BrickBack — how it came to be

Why at all

My son and I had built the UCS Millennium Falcon (75192) — over 7,500 parts across 17 numbered bag groups. At some point the set sat disassembled in one big bag, and rebuilding it failed on a single thing: finding a specific part in the pile. There are apps that recognise piles (Brickit) or individual parts (Brickognize), but none that sort a disassembled heap back into its original build-stage bags. That's exactly what I wanted.

How it works

The flow is deliberately simple: pick a set, take a photo of the parts spread out flat, and the app recognises each part and shows — via a coloured overlay — which bag it belongs to. Sort it away, take the next photo of what's left, until the pile is empty. At the end you get a missing-parts list with bag assignments that doubles as a shopping list.

What's inside

The app is written in Flutter (Android first, iOS to follow), state runs through Riverpod, and the local database uses drift on SQLite — sorting progress survives app restarts. Technically the project is made of four building blocks:

• Inventory & bag mapping — the actual core. Which part goes into which bag lives in no database. I compute it once per set, offline, from the text layer of the instruction PDF (inventory, quantities, bag badges); OpenCV finds the part callouts, Brickognize provides part identity — with a built-in check that the sum of all bags plus spares equals the full inventory.
• Object detection — a YOLOv8n detector crops each part out of the photo. The training data is rendered synthetically from LDraw 3D models; the model runs on-device as TFLite.
• Classification — each cropped part becomes a part number. In the MVP via the Brickognize API, later a custom on-device model that only knows the ~700 part types of the given set.
• Matching — deduct a recognised part from the remaining demand; when ambiguous, suggest the bag with the largest remaining need.

The backend

So the app doesn't have to pre-compute every set itself, there's a small content service: a FastAPI app that holds versioned set bundles (inventory + bag mapping + optional recognition data) in PostgreSQL and serves them read-only to the app. The offline pipeline that produces the mappings shares its data schema with the service, so both sides speak the same contract. The app itself needs no server in the MVP — the backend is the infrastructure behind it that supplies new sets.

What was trickier

• Grey on grey: the Falcon is almost monochrome. The app recommends a dark, high-contrast surface, otherwise the parts blur together.
• Confusable groups: 1×2 plate vs. tile, shallow slopes seen from above — barely distinguishable. Below a confidence threshold the app asks rather than guesses.
• Trademark: "LEGO" must not appear in the app name. Hence BrickBack and the phrasing "suitable for building bricks".

Where it stands today

The sorting workflow works for the first set (75192) with Brickognize as the recogniser. Next up: the mapping pipeline for more sets, custom on-device models, and the path into the app stores. The store links will go here once the app is published.