❝

TL;DR: tesseract image.png stdout reads the text out of an image. It works when the text sits in a block, which is the case with most terminal screenshots, scanned pages, and rendered charts. When it does not, the fix is to crop, not to sharpen the contrast. And it can still produce incorrect results, so check the output precisely when a wrong character matters.

Say a step in your pipeline needs to interpret the text in images: a number of screenshots from a CI run, a figure inside a PDF, a diagram on a bug report. And it needs that text on stdout, so the next command in the pipeline can work with it.

The OCR engine is Tesseract. Getting the most out of it needs a preparation step for the images, which sips (on macOS) or ImageMagick (on Linux) does.

Install Tesseract with brew install tesseract on macOS and apt install tesseract-ocr on Linux Ubuntu or Debian.

A terminal screenshot, read nearly right

Take a screenshot of a terminal session with monospaced text, light on dark. Here is a real one, against a PostgreSQL container:

$ psql -At -d app -c 'select current_database(), current_user;'
app|app

$ psql -d app
FATAL:  password authentication failed for user "app"

Use tesseract to process that image and output to stdout:

tesseract screenshot.png stdout

You would get the following output:

$ psql -At -d app -c ‘select current_database(), current_user;'
app|app

$ psql -d app
FATAL: password authentication failed for user "app"

Looking closely, what worked well: the quotes are there, the pipe is there, and app|app came through intact. What did not: the double space after FATAL: collapsed to a single space, and the straight quote before select came back as a curly one. These are small OCR errors you may get.

Tesseract normally writes to a file; stdout as the output target sends the recognized text to standard output instead, which is what makes it pipeable:

tesseract screenshot.png stdout | grep app

Two possible failures: nothing and wrong output

Run tesseract on an image whose text it cannot find and you get no output, exit status 0, and no warning. In a pipeline, that is worse than an error, because an empty result is indistinguishable from an image with no text at all.

The other failure situation is getting a wrong output that looks right. Take a terminal screenshot containing the following build log:

$ ./build.sh --target release
  CC      src/parser.c
  CC      src/lexer.c
  LD      bin/tool

42M    /tmp/build-stage/logs/app.log
79M    /tmp/build-stage/cache/blob.bin
31M    /tmp/build-stage/output/layer.bin

$ ls -la /tmp/build-stage
total 152M
drwxr-xr-x   8 user  staff   256B  app.log

When testing that, here is what came back:

$ ./build.sh --target release
cc src/parser.c
cc src/lexer.c
LD bin/tool

42M /tmp/build-stage/logs/app. log
79M /tmp/build-stage/cache/blob.bin
31M /tmp/build-stage/output/layer.bin

$ ls -la /tmp/build-stage
total 152M
drwxr-xr-x 8 user staff 256B app.log

The sizes came through exactly: 42M, 79M, 31M. The paths came through with one exception, app.log arriving as app. log. And CC came back lowercase as cc.

Nothing in the output looks broken, but it could cost you at some point. A grep for 42M would find it, and a grep for the path would almost find it. Regarding the build flags, though they are wrong, only a human reading them would catch the error.

The recommendation: crop, not contrast

The rule for improved results is not about contrast — it is about layout context. Take a title card that says "New Members", set at a 28-degree angle the way a graphic or a photo caption tends to be. Tesseract has to decide that what it is looking at is a line of text, and a tight crop of a slanted word gives it no baseline to work with. In this case, the whole image reads as empty. But when you do a wider crop, with the surrounding frame included, some text is found:

$ tesseract tight-crop.png stdout
                              # empty

$ tesseract wide-crop.png stdout
Ww Men, Yor

The recommendation is to crop to the text region; as we can see, the wide crop finds the line, even if the 28-degree angle still mangles the words.

On macOS, the sips -c command helps with that preparation step:

sips -c 300 800 --cropOffset 0 0 title.png --out crop.png

tesseract crop.png stdout

Contrast work does not help: on the tight crop, thresholding at 100, 128, 160, or 190 still returns nothing.

The equivalent for sips on Linux is ImageMagick.

Where this leaves you

Use OCR in situations where the text is the image, and do a crop, as a preparation step, when it is not. Do not depend only on it: when the output feeds a script and a wrong character matters (a version number, a port, a path), check it against something you trust. In a way, OCR is guessing.

Reply

Avatar

or to participate