Can AI finally read messy auction catalogs from the 1800s?

Auction catalogs from the nineteenth and early twentieth centuries sit in archives across Europe. They record who sold what, when, and for how much — information that provenance researchers need to trace artworks across time. The problem: every catalog formats its lots differently. One page uses numbered paragraphs, another mixes text and images in irregular columns, a third abbreviates artist names in ways that change from sale to sale. Until now, large-scale analysis meant hiring people to read each page and type the data by hand.

A paper submitted to arXiv on 31 August 2026 describes a system that automates this extraction. The authors — Mathias Zinnen, Alisha Mund, Sabine Lang, Lukas Hüttner, Thomas Gorges, and Vincent Christlein — call it Lot Machine. It targets German Sales, a database of historical auction and sales catalogs from the 19th and 20th centuries. The paper was accepted at the VISART Workshop, part of ECCV 2026.

How the pipeline works

Lot Machine treats each catalog page as a visual and textual problem at once. A Vision-Language Model (VLM) receives an image of the page and a prompt asking for structured metadata: lot number, artist, title, medium, dimensions, provenance notes, estimate, hammer price. The model outputs JSON. That sounds straightforward. In practice, the formatting variance breaks naive prompting.

The team built a manually annotated test set of representative catalog pages. They then evaluated multiple prompt strategies and constrained decoding frameworks. Constrained decoding forces the model to emit valid JSON by restricting the token vocabulary at each generation step. Without it, models often produce almost-valid JSON with missing braces or trailing commas — useless for downstream ingestion.

Deployment modes reflect real constraints

Cultural heritage institutions face three practical limits: budget, compute, and data privacy. The paper benchmarks three deployment modes that map to those limits.

Commercial endpoints — think GPT-4o or Claude via API — set the performance ceiling. They achieve the highest extraction accuracy but send page images to external servers. For many institutions, that violates privacy policies or donor agreements.

Institutional gateways — shared GPU clusters run by university consortia or national libraries — offer a middle ground. Models like Llama-3.1-70B or Qwen2-VL-72B run behind institutional firewalls. Accuracy drops slightly but remains usable. Data never leaves the network.

Local deployments run quantized models on a single workstation. A 7B or 8B parameter model fits in limited VRAM. The paper finds this feasible only when constrained decoding is enforced. Without it, the smaller models drift into invalid JSON often enough to break automated pipelines. With it, they produce valid output at a fraction of the cost.

What the numbers show

Results span 5 tables and 6 figures. Commercial endpoints reach the highest F1 scores on lot-level field extraction. Institutional gateways trail by a small margin. Local quantized models with constrained decoding close most of the gap. Without constrained decoding, local models produce invalid JSON in a significant share of cases — the exact percentage appears in Table 3 of the paper.

Human-in-the-loop correction remains necessary. The pipeline does not claim full autonomy. It reduces the manual workload from "transcribe everything" to "verify and fix model output." How much verification time that saves depends on the catalog's condition and the model choice. The paper does not give a single hours-saved figure; it varies by deployment mode and collection.

What this changes for researchers

If you work with German Sales today, you search by keyword or browse page images. You cannot query structured fields across the corpus because that layer does not exist. Lot Machine creates it.

The authors release code, benchmark data, and prompt templates. That means another team can adapt the pipeline to a different catalog corpus — French sales, British sales, dealer stock books — by annotating a few hundred pages and retraining or re-prompting. The paper's supplementary material includes the annotation guidelines.

Where the explanation stops

The paper does not report how many catalogs in German Sales remain unprocessed. It does not estimate the total compute cost to run the pipeline on the full corpus. It does not compare against traditional OCR plus layout analysis, only against VLM baselines. And it does not claim the pipeline works equally well on catalogs outside the German Sales conventions — though the authors suggest the approach transfers with new annotations.

For now, the takeaway is concrete: a VLM pipeline can turn messy historical catalog pages into queryable data, provided you choose a deployment mode that matches your privacy budget and enforce JSON structure during generation. The rest is engineering.