chart-critic

Can a vision model act as a useful chart referee? My hunch is that the misleading-chart playbook is small and well-catalogued, and most of it is mechanical to check once you can read the axes off the image.

What it’s poking at:

• A skill-shaped critique pass: a checklist of known tricks (zero-suppressed y-axis, inconsistent intervals, dual axes with mismatched scales, area-as-volume, truncated time windows, missing error bars on noisy data) run against an input chart
• Lossy data reconstruction: inferring the underlying table from a rasterised plot, with the agent honest about how precisely it can read the values rather than pretending to recover exact numbers
• Redraw as the receipt: the deliverable isn’t a verdict, it’s a side-by-side, original chart on the left, a redrawn version with a sane y-axis and any uncertainty made visible on the right
• Evaluation is the hard part: it needs a corpus of known-bad charts with ground-truth data to tell whether the agent catches the tricks and whether its redraw is faithful. News and finance social feeds are a steady source of material

Corpus

Examples being collected to test the agent against. Each entry pairs an input image with the trick(s) the agent should catch.

1. Zero-suppressed y-axis (Wikipedia, “Misuse of statistics”)

Same data, two y-axis choices. The left chart’s baseline sits just below the smallest bar, making a small difference look dramatic; the right chart starts at zero and shows the true proportions. The expected agent output is the right-hand chart, generated from data inferred from the left.

Source: Wikimedia Commons.

2. Same chart, misleading panel only (no answer key)

The left panel from entry 1, cropped out on its own. With the honest panel removed the agent has to infer the true magnitude from the misleading axis alone, which is the actual job. Same source image, cropped at x=240.