pctablet505
Work / Computer vision

Reading damaged barcodes on phones

Read 90%+ of damaged barcodes correctly on its damaged test set, where a commercial scanning SDK read 70%, with models trained on 10M+ synthetic images.

  • Computer vision
  • Samsung R&D Institute India
  • Jul 2022 – Aug 2024
  • Built
  • 2 min read
70% → 90%+
damaged barcodes read correctly on the damaged synthetic test set, a commercial scanning SDK against this reader
○ reported, from my account
10M+
synthetic training images
○ reported, from my resume
10+
1D barcode formats
○ reported, from my account
under 100 ms
per scan of an undamaged code
○ reported, from my account
up to 350 ms
per scan of a damaged code, depending on the damage
○ reported, from my account
under 1 MB
per model, on a Galaxy S21 FE
○ reported, from my account
5 901234 123457a random string, valid for its format59012341234575 901234 123457drawn from the EAN-13 tablesblurlinesspotsglarereflectiontearscratchlabels stay inside the picture10M+ imagesYOLOMask R-CNNfound by YOLO, cut out by Mask R-CNNU-Nettrained pixel to pixelclassifierEAN-13UPC-ACode 128Code 39ITFCodabar…10+ 1D formats, each read in its own wayconfidencehighread ✓5901234123457confidencelowrefused ✗no guessdamaged codes read correctly, on the damaged synthetic test setcommercial SDK70%this reader90%+under 100 ms a scanup to 350 ms damagedeach model under 1 MB

1 of 8Make the data

Every training image starts as a random string that follows its format's rules, drawn as a real barcode.

8 steps · 55 s
Illustration drawn by code, not a recording: a real EAN-13 encoding is made, damaged, found, restored, named and read, and a code beyond repair is refused.

Knox Capture lets a phone do an enterprise barcode scanner's job. On industrial sites the codes are blurred, partly covered, seen at an angle or shining under lights, and a scanner that reads the wrong number is worse than one that reads nothing. I built the part that reads damaged codes on the phone, and knows when to give up.

Problem

Industrial barcodes arrive blurred, partly covered, at an angle or under glare. A plain decoder either fails or returns a wrong code, and a wrong code is the costly failure.

Approach

  • Data first. A synthetic-data engine writes random strings that follow each format's rules and draws them as barcodes. Then it damages them on purpose: blur, lines, spots, glare, reflections, tears and scratches, each drawn from a probability model. Perspective, curvature and lighting make each one a camera view, blended onto photos of real objects. 10M+ images in all.
  • Labels that stay true. The hardest part: every transform also moves the labels, the bounding box and the barcode's outer and inner rectangles. Algebra and trigonometry set how far a code can turn, move or zoom before a label leaves the picture.
  • A chain of small models. YOLO finds the barcode, Mask R-CNN cuts it out, a U-Net trained pixel to pixel restores the damaged bars, a classifier names the format among 10+ 1D formats, then an open-source decoder reads it.
  • Knowing when to stop. Every read carries a confidence score. A wrong read costs far more than no read, so a code beyond recovery is refused instead of guessed.
  • On the phone. Android, the hardware-accelerated camera and TensorFlow Lite, mostly on a Galaxy S21 FE: each model under 1 MB, under 100 ms per scan for an undamaged code and up to 350 ms for a damaged one.

Results

On the damaged synthetic test set, 90%+ of codes were read correctly, where a commercial scanning SDK read 70%. Real photos were checked too, by hand, on a smaller set. Refusing hopeless codes cut wrong reads.

More in computer vision