OCR based Trade Recommendations for The Patrician II

I recently bought The Patrician II on GoG. Having fond childhood memories of that game, which has been created by a studio in my hometown (Ascaron, they sadly don't exist anymore), I played for about half an hour when I got annoyed by the fact that I'd have to figure out the best trading routes again, comparing prices against profitable ranges. I used to know them by hard, but that was a long time ago...

There are price tables for this online and since it was a relaxed sunday morning I thought about how to automate some of the procedure. I came up with this plan:

For the OCR, we could go all crazy trying to code this ourselves in Python e.g. using pytesseract and Pillow for image pre-processing. I played around with that, but it isn't easy to produce good reliable results and I did not want to turn this into a week long project. I found Capture2Text as a great tool for that exact purpose since it came with a commandline interface and was able to target a specific area of the screen. The "trading goods" interface in the game looks like this:

We can tell Capture2Text to capture the window rectangle (having measured the pixel coordinates beforehand). It will cast it to a black-and-white image and apply some filters as a pre-processing step, producing something like this internally:

That is actually a pretty good result for the OCR that comes next, helping it to distinguish the actual characters from any background noise. The resulting recognized text may be something like this:

• The Trading Goods window containing the price table for the current town is always in the same place, with the same dimensions. Assuming a fixed desktop resolution of 1280 x 720 pixels (the maximum the game allows... yes, it is an older game) the area of interest will always be in the same place, so we know how to capture it into a screenshot.
• OCR can be used as a way to translate that image of the price table to text that can be processed programmatically.
• The price table data is of a fixed format that we can parse and compare against recommended prices from an online source.
• Run all this while playing the game to get live recommendations for goods to trade with profit in the current town.

Capture2Text produces somewhat differing results on repeated runs, so there are a number of quirks we'd have to handle to get reliable, consistent results. We then have to parse, clean and analyze the output and compare to a price table from the web to give recommendations for the current town.

I wrote a script that does that.

Running it in a console on a second screen while playing the game, it produces output like this every two seconds: Profit!