The ActiveVision benchmark shows that today’s leading multimodal large language models (LLMs) solved 10.6% of tasks that require looking at an image more than once. In a test of 17 iterative-perception challenges, humans succeeded 96.1% of the time while GPT-5.5 managed 10.6% and Claude Fable 5 managed 3.5%; eleven of the tasks saw zero correct answers from the models.

Why current vision models stumble

Most vision systems treat an image as a one-off input. A “vision encoder” extracts features once, then hands them to the language model for reasoning. The pipeline cannot ask for a second glance, zoom in on a region, or re-evaluate a hypothesis. Humans, by contrast, make an initial guess, shift focus, gather new evidence, and adjust their answer. The benchmark formalizes that loop: each task forces a model to observe, propose an action, observe the result, and repeat until it reaches a correct conclusion.

What the benchmark tested

Researchers built 17 scenarios that require repeated observation. The results are stark:

  • Human participants: 96.1% success
  • GPT-5.5: 10.6% success
  • Claude Fable 5: 3.5% success
  • Eleven tasks: every tested model scored zero

Even when models generated code to re-process the image, they missed errors in their own output, confirming that the limitation is architectural rather than purely data-driven.

Stakes for developers and industry

If you are building autonomous robots, inspection drones, or any system that must verify visual information over time, a single-shot vision model is risky. The benchmark shows that current LLM-based vision pipelines cannot reliably handle feedback-driven perception, so they will miss defects, miscount items, or misinterpret dynamic scenes. Adding more training data or scaling up parameters will not close the gap; the missing ingredient is a mechanism for controlled, iterative observation.

Counter-argument: can larger models help?

Adding more training data or scaling up parameters will not close the gap; the missing ingredient is a mechanism for controlled, iterative observation.

Paths forward

Researchers suggest a complementary direction:

  • Architectural redesign – embed a controllable visual module that can request new views, crop regions, or alter lighting conditions based on the model’s internal state.

Takeaway: Current multimodal LLMs excel at answering static image questions but fall short when a problem requires looking again. True visual intelligence will need models that can control their own sight, not just read a picture once.