TIB - Leibniz Information Centre for Science and Technology · L3S Research Center
Scientific figure comprehension and reasoning using multimodal AI requires integrating visual perception with domain-specific reasoning to extract meaningful knowledge, often not presented in the text of a research publication. The Sci-ImageMiner benchmark dataset, accompanied by a community-driven competition, raises the bar over prior scientific competitions by curating a comprehensive, expert-annotated dataset across four end-to-end complementary tasks. The competition attracted 68 active participants and 1,263 public/private submissions from 9 January 2026 to 8 April 2026. Our results show that state-of-the-art multimodal models perform well on classification and summarization tasks but struggle with data extraction and scientific reasoning, particularly in visual question-answering. These findings reveal key limitations and highlight challenges and opportunities for improving domain-aware multimodal AI systems. Overall, the Sci-ImageMiner benchmark and competition establish a rigorous platform for advancing research in scientific figure comprehension and reasoning and demonstrate the potential of state-of-the-art approaches for a challenging and complex research area.
The released benchmark covers ALD and ALE papers across experimental and simulation use cases. It provides train, dev, and gold-standard test splits with figure images, captions, bounding boxes, and task-specific annotations. Task definitions are summarized from the official competition task description.
We collected 205 research publications in ALD/E materials science, encompassing both experimental and simulation-based studies. MinerU is used to extract textual content in structured JSON format along with high-resolution figures in JPEG format, preserving semantic structure and visual fidelity for downstream multimodal analysis as illustrated in Figure 1.
The dataset is organized into two primary categories: ALD and ALE, which are further divided into experimental and simulation-based studies. It is partitioned into standard train/dev/test splits with the statistics shown in Table 1 and the organization illustrated in Figure 2.
content.json (sections and figure captions), along with
figures and annotation JSON files inside the images directory.
The four panels below summarize the official competition task definitions with compact examples from the task website while preserving the input/output structure.
A supervised multi-class image classification task. Systems must identify one of the 49 figure classes from a scientific figure.
Structured reconstruction of tabular data encoded in scientific charts. Systems identify field labels, table structure, and textual or numerical cell values.
| Time (s) | Mass Change (ng/cm²) |
| 0 | 0 | 2000 | -500 | 4000 | -1000 | 6000 | -1500 |
Generate concise, factual summaries that capture key trends, relationships, and scientific insights presented in the figure. Systems must ground their interpretation in the visual content.
Fine-grained reasoning over figures through natural-language questions that reference axes, legends, data patterns, and scientific meaning.
The strongest systems were not single-model baselines. They combined vision-language models with task structure: hierarchy-aware classification, chart-to-table training data, context retrieval, alignment training, and answer-type-aware VQA pipelines.
The competition continues beyond the official competition period. The benchmark is now in its post-competition phase, and the leaderboard will remain open indefinitely for new submissions. Evaluate your multimodal system on the public leaderboard using the resources below.
Use the GitHub repository to access the competition dataset, Codabench submission-format guidelines, and the evaluation scripts used on the Codabench competition pages.
Use the Hugging Face dataset to visually explore the data. It supports image class style loading by train/dev/test split.
Join the official Sci-ImageMiner Google Group for competition announcements and a place to ask questions and interact with the participant community.
@article{ahmed2026icdar,
title = {ICDAR 2026 Competition on Information Extraction from
Atomic Layer Deposition/Etching (ALD/E) Scientific Figures},
author = {Ahmed, Fahad and Auer, S{\"o}ren and D'Souza, Jennifer},
journal = {arXiv preprint arXiv:2607.26848},
year = {2026},
url = {https://arxiv.org/abs/2607.26848}
}