PaperBanana-Interact: Scientific Diagram Refinement
with Multi-Turn Human Feedback

Xueqing Wu1 2 *, Ashwin Balasubramanian2, Bingxuan Li2 3, Dawei Zhu4, Kai-Wei Chang1, Yale Song2, Yiwen Song2, Rui Meng2, Nanyun Peng2
1University of California, Los Angeles  2Google  3University of Illinois Urbana-Champaign  4Peking University
*Work done while Xueqing was a student researcher at Google Cloud AI Research.
arXiv Paper Code Benchmark Demo
PaperBanana-Interact teaser

Diagram generators may fail to fully satisfy user preferences in a single turn, necessitating multi-turn refinement with human feedback. We introduce MTPaperBananaBench to benchmark this task, and propose PaperBanana-Interact to mitigate common failure modes in baseline systems.

Abstract

Recent efforts have aimed to automate scientific diagram generation from paper content. However, fully satisfying an author's visual and communicative preferences in a single turn is challenging: in our formative user study (N = 14), all participants requested further revisions after viewing an initial draft, and 86% of them rated the refined diagrams as more satisfactory. Despite the clear demand, this multi-turn workflow remains largely underexplored. To bridge this gap, we present MTPaperBananaBench, a benchmark for multi-turn diagram generation containing 292 images annotated with 3,518 user requirements. To avoid expensive human studies and enable scalable benchmarking, we construct a user simulator that, at each turn, identifies unsatisfied requirements and converts k of them into natural language feedback. Evaluating both requirement satisfaction and overall diagram quality reveals two key failure modes shared across baseline systems: (1) quality drift, where diagram quality progressively declines over turns, and (2) forgetting, where previously implemented features are lost in subsequent turns. To address these issues, we introduce PaperBanana-Interact, a multi-agent system that refines diagrams via an internal critique-and-refine loop. PaperBanana-Interact consistently improves rather than degrades diagram quality across iterations, outperforming baselines by 11.9–18.6 points in quality score and reducing forgetting by 3.7–6.2 points.

MTPaperBananaBench: Benchmarking Multi-Turn Diagram Refinement

At the core of the benchmark is an interaction process in which user requirements gradually emerge as the diagram evolves. Since collecting human interactions at scale is prohibitively expensive and hard to standardize, we develop a user simulator driven by a pool of requirements hidden from the diagram generator at the initial turn. At each subsequent turn, the simulator identifies unfulfilled requirements, spanning content, layout, and visual representation, and formulates k of them into natural language feedback. The final diagram is evaluated on (1) satisfaction of the user requirements and (2) overall diagram quality.

Annotation and inference pipeline of MTPaperBananaBench

Annotation and inference pipeline of MTPaperBananaBench. The multi-turn interaction is driven by a user simulator that converts unsatisfied requirements into natural language feedback at each turn.

Baselines Suffer from Quality Drift and Forgetting

Baseline systems, including leading generative models (NanoBananaPro, GPT-Image-2) and an agentic baseline (PaperBanana-DirectRefine), effectively address refinement requests on individual turns: satisfaction rates reach 68.6% for PaperBanana-DirectRefine and 56.2% for NanoBananaPro.

However, their performance degrades significantly across multi-turn interactions. Failure analysis reveals two key failure modes:

  • Quality drift: image quality progressively deteriorates over turns (50.3 → 19.0 for NanoBananaPro; 50.3 → 47.1 for PaperBanana-DirectRefine).
  • Forgetting: later refinements overwrite previously satisfied requests, affecting 14.1–22.9% of such requests across all baselines.

PaperBanana-Interact

PaperBanana-Interact is a multi-agent system that refines diagrams via an internal critique-and-refine loop. A critic reviews each revision and proposes suggestions, and a refiner and a visualizer generate the next revision accordingly. PaperBanana-Interact features two key designs:

  • A multi-objective critic assesses the diagram against multiple constraints: the current user request, all prior user requests, adherence to the source context, and presentation quality.
  • A summarizer compresses the long, multi-image interaction history into a compact textual memory capturing relevant visual details, shared across all subsequent agents.
PaperBanana-Interact overview

PaperBanana-Interact overview. A summarizer compresses the interaction history into textual memory; an internal loop of multi-objective critic, refiner, and visualizer produces each revision.

Results

On MTPaperBananaBench, PaperBanana-Interact consistently improves rather than degrades diagram quality across iterations. It outperforms baselines by 11.9–18.6 points in quality score and reduces the forgetting rate by 3.7–6.2 points, mitigating both quality drift and forgetting.

Example Trajectory

Below is a complete refinement trajectory produced by PaperBanana-Interact, one turn per slide. Each slide shows the user feedback given at that turn, followed by the diagram the system produces in response. Starting from the initial draft at t = 0, the user raises a new requirement at every turn, and PaperBanana-Interact satisfies it while preserving the requirements met in earlier turns, so the diagram improves steadily instead of drifting.

BibTeX

@misc{wu2026paperbananainteractscientificdiagramrefinement,
      title={PaperBanana-Interact: Scientific Diagram Refinement with Multi-Turn Human Feedback}, 
      author={Xueqing Wu and Ashwin Balasubramanian and Bingxuan Li and Dawei Zhu and Kai-Wei Chang and Yale Song and Yiwen Song and Rui Meng and Tomas Pfister and Nanyun Peng},
      year={2026},
      eprint={2608.30241},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2608.30241}, 
}