Do Multimodal Models Decide Consistently on Equivalent Optical Spectra?

Classés selon la proximité de leur titre et de leur résumé avec ceux de cet article. Sélectionnez-en pour les lire ensemble.

Les articles les plus proches

1. Proche Contexte plus large Pas pertinent

   Evaluating large language models on multimodal chemistry olympiad exams

   Yiming Cui, Xin Yao, Yuxuan Qin et al. · 2025 · Communications Chemistry · cité 17 fois · Pas encore résumé

   «Evaluating large language models on multimodal chemistry olympiad exams» teste le raisonnement visuel et textuel de modèles multimodaux sur des questions de chimie, plutôt que sur des spectres optiques équivalents.

   Résumé scientifique

   Multimodal scientific reasoning remains a significant challenge for large language models (LLMs), particularly in chemistry, where problem-solving relies on symbolic diagrams, molecular structures, and structured visual data. Here, we systematically evaluate 40 proprietary and open-source multimodal LLMs, including GPT-5, o3, Gemini-2.5-Pro, and Qwen2.5-VL, on a curated benchmark of Olympiad-style chemistry questions drawn from over two decades of U.S. National Chemistry Olympiad (USNCO) exams. These questions require integrated visual and textual reasoning across diverse modalities. We find that many models struggle with modality fusion, where, in some cases, removing the image even improve…

2. Apparenté Contexte plus large Cité par cet article Pas pertinent

   Unified neural network model for predicting optical responses in gold nanostructures

   Xiao-Bo Wang, Yu-Chen Cao, Feng Feng et al. · 2025 · Journal of Applied Physics · cité 6 fois · Pas encore résumé

   «Unified neural network model for predicting optical responses in gold nanostructures» prédit les spectres de nanostructures d’or à partir de leur géométrie, tandis que cet article évalue des décisions sur des spectres mesurés.

   Résumé scientifique

   In this paper, we present a deep neural network model capable of simultaneously predicting the optical transmittance spectra of three distinct gold nanostructures: nanodisks, truncated cones, and nanocones. Our model achieved a mean squared error loss as low as 0.0015. By systematically varying geometric parameters, we generated a comprehensive dataset to train the model, which captures complex dependencies between structure and optical response. A key finding is the redshift in resonance dip associated with increasing sidewall tilt. We further validated the model by comparing its predictions with experimental results, showing excellent agreement. This work demonstrates a powerful, data-driv…

3. Apparenté Même question Cité par cet article Pas pertinent

   SpecVQA: A Benchmark for Spectral Understanding and Visual Question Answering in Scientific Images

   Jialun Shen, Han Lyu, Su-Yang Zhong et al. · 2026 · arXiv.org · preprint, non relu par les pairs · cité 1 fois · Pas encore résumé

   SpecVQA évalue la compréhension de spectres et les réponses à des questions scientifiques sur des images, comme cet article évalue les décisions de modèles multimodaux sur des spectres.

   Résumé scientifique

   Spectra are a prevalent yet highly information-dense form of scientific imagery, presenting substantial challenges to multimodal large language models (MLLMs) due to their unstructured and domain-specific characteristics. Here we introduce SpecVQA, a professional scientific-image benchmark for evaluating multimodal models on scientific spectral understanding, covering 7 representative spectrum types with expert-annotated question-answer pairs. The aim comprises two aspects: spectra scientific QA evaluation and corresponding underlying task evaluation. SpecVQA contains 620 figures and 3100 QA pairs curated from peer-reviewed literature, targeting both direct information extraction and domain-…

4. Apparenté Contexte plus large Cité par cet article Pas pertinent

   Probing the limitations of multimodal language models for chemistry and materials research

   Nawaf Alampara, Mara Schilling-Wilhelmi, Martiño Ríos-García et al. · 2024 · Nature Computational Science · cité 89 fois · Pas encore résumé

   MaCBench évalue des modèles vision-langage sur des tâches de chimie et de science des matériaux, dont l’interprétation de résultats, sans se limiter à l’équivalence des formats spectraux.

   Résumé scientifique

   Recent advancements in artificial intelligence have sparked interest in scientific assistants that could support researchers across the full spectrum of scientific workflows, from literature review to experimental design and data analysis. A key capability for such systems is the ability to process and reason about scientific information in both visual and textual forms—from interpreting spectroscopic data to understanding laboratory set-ups. Here we introduce MaCBench, a comprehensive benchmark for evaluating how vision language models handle real-world chemistry and materials science tasks across three core aspects: data extraction, experimental execution and results interpretation. Throug…

5. Apparenté Contexte plus large Pas pertinent

   Reading and steering representations of materials-science mechanisms in an open-weight language model

   Markus J. Buehler · 2026 · AI for Science · cité 1 fois · Pas encore résumé

   «Reading and steering representations of materials-science mechanisms in an open-weight language model» étudie les représentations internes et le raisonnement en science des matériaux, plutôt que la cohérence des décisions sur des spectres.

   Résumé scientifique

   Large language models can answer scientific questions, yet a correct output does not reveal whether the model represents or uses the governing physics. Here, using three open-weight Gemma 4 instruction-tuned models (google/gemma-4-E4B-it, google/gemma-4-12B-it, and google/gemma-4-31B-it) we identify three experimentally separable signatures of materials-science mechanism information: selective concept readability, relational encoding of qualitative constitutive orientation, and causal, context-dependent control of constrained engineering answers. Across 50 held-out descriptions, three independently fitted Jacobian lenses produced nearly identical concept ranks, but 36 prompts recovered no de…

6. Apparenté Contexte plus large Pas pertinent

   Good Scores, Bad Data: A Metric for Multimodal Coherence

   V. Srinivasan · 2026 · arXiv.org · preprint, non relu par les pairs · Pas encore résumé

   «Good Scores, Bad Data» mesure la cohérence multimodale sur des images Visual Genome et COCO, tandis que cet article teste la stabilité des décisions sur des spectres.

   Résumé scientifique

   Multimodal AI systems are evaluated by downstream task accuracy, but high accuracy does not mean the underlying data is coherent. A model can score well on Visual Question Answering (VQA) while its inputs contradict each other. We introduce the Multimodal Coherence Score (MCS), a metric that evaluates fusion quality independent of any downstream model. MCS decomposes coherence into four dimensions, identity, spatial, semantic, and decision, with weights learned via Nelder-Mead optimization. We evaluate on 1,000 Visual Genome images using DETR, CLIP, and ViLT, and validate on 150 COCO images with no retraining. Across three fusion architectures, MCS discriminates quality with higher sensitivi…

7. Apparenté Contexte plus large Pas pertinent

   MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models

   Chaoyou Fu, Peixian Chen, Yunhang Shen et al. · 2023 · Computer Vision and Pattern Recognition · cité 1 927 fois · Résumé IA disponible

   MME propose un benchmark général de perception et cognition multimodales, alors que cet article cible la cohérence des réponses à des spectres représentés de plusieurs façons.

   Résumé scientifique

   Multimodal Large Language Model (MLLM) relies on the powerful LLM to perform multimodal tasks, showing amazing emergent abilities in recent studies, such as writing poems based on an image. However, it is difficult for these case studies to fully reflect the performance of MLLM, lacking a comprehensive evaluation. In this paper, we fill in this blank, presenting the first MLLM Evaluation benchmark MME. It measures both perception and cognition abilities on a total of 14 subtasks. In order to avoid data leakage that may arise from direct use of public datasets for evaluation, the annotations of instruction-answer pairs are all manually designed. The concise instruction design allows us to fai…

8. Apparenté Même question Pas pertinent

   Same Answer, Different Representations: Hidden instability in VLMs

   Farooq Ahmad Wani, Alessandro Suglia, Rohit Saxena et al. · 2026 · Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing · cité 4 fois · Pas encore résumé

   «Same Answer, Different Representations» étudie la stabilité des modèles vision-langage sous des représentations différentes, question également testée ici avec des formats équivalents de spectres.

   Résumé scientifique

   The robustness of Vision Language Models (VLMs) is commonly assessed through output-level invariance, implicitly assuming that stable predictions reflect stable multimodal processing. In this work, we argue that this assumption is insufficient. We introduce a representation-aware and frequency-aware evaluation framework that measures internal embedding drift, spectral sensitivity, and structural smoothness (spatial consistency of vision tokens), alongside standard label-based metrics. Applying this framework to modern VLMs across the SEEDBench, MMMU, and POPE datasets reveals three distinct failure modes. First, models frequently preserve predicted answers while undergoing substantial intern…

Les travaux antérieurs sur lesquels il s’appuie

1. Apparenté Contexte plus large Pas pertinent

   Large Language Models Are Not Robust Multiple Choice Selectors

   Chujie Zheng, Hao Zhou, Fan-Dong Meng et al. · 2023 · International Conference on Learning Representations · cité 725 fois · Pas encore résumé

   «Large Language Models Are Not Robust Multiple Choice Selectors» teste les biais de position dans les choix multiples, non les variations de représentation des données scientifiques.

   Résumé scientifique

   Multiple choice questions (MCQs) serve as a common yet important task format in the evaluation of large language models (LLMs). This work shows that modern LLMs are vulnerable to option position changes in MCQs due to their inherent"selection bias", namely, they prefer to select specific option IDs as answers (like"Option A"). Through extensive empirical analyses with 20 LLMs on three benchmarks, we pinpoint that this behavioral bias primarily stems from LLMs' token bias, where the model a priori assigns more probabilistic mass to specific option ID tokens (e.g., A/B/C/D) when predicting answers from the option IDs. To mitigate selection bias, we propose a label-free, inference-time debiasin…

2. Apparenté Contexte plus large Pas pertinent

   Sufficient Context: A New Lens on Retrieval Augmented Generation Systems

   Hailey Joren, Jian-Yi Zhang, C. Ferng et al. · 2024 · International Conference on Learning Representations · cité 141 fois · Pas encore résumé

   «Sufficient Context» distingue les questions répondables de celles privées d’informations suffisantes, comme cet article teste l’abstention lorsque la portion pertinente du spectre manque.

   Résumé scientifique

   Augmenting LLMs with context leads to improved performance across many applications. Despite much research on Retrieval Augmented Generation (RAG) systems, an open question is whether errors arise because LLMs fail to utilize the context from retrieval or the context itself is insufficient to answer the query. To shed light on this, we develop a new notion of sufficient context, along with a method to classify instances that have enough information to answer the query. We then use sufficient context to analyze several models and datasets. By stratifying errors based on context sufficiency, we find that larger models with higher baseline performance (Gemini 1.5 Pro, GPT 4o, Claude 3.5) excel …

3. Apparenté Même méthode, autre problème Pas pertinent

   Jev for Scientific Decisions: Evaluating Semantic Choices and Their Consequences

   Bo-Yuan Deng, Shu-Yi Fan, Hong-Yang Zhang et al. · 2026 · arXiv.org · preprint, non relu par les pairs · cité 13 fois · Pas encore résumé

   «Jev for Scientific Decisions» évalue Jev sur des choix sémantiques scientifiques, tandis que cet article inclut Jev 1.13 pour décider à partir de spectres.

   Résumé scientifique

   Scientific workflows often require choosing among known relations before a deterministic calculation can proceed. Whether observations share a culture, treatment or reference standard can change the scientific meaning of the resulting count or comparison. Once the evidence is prepared and the calculation is written as code, this choice reduces to one semantic judgment among listed candidates. We evaluate Jev on this step, using a harness that follows its documented guidance. The study compares twelve model configurations on twenty Choices in ten scientific cases built from published sources, each repeated five times. We measure semantic selections, downstream outputs and final claim labels s…

4. Apparenté Même méthode, autre problème Pas pertinent

   From Text Decisions to Pixels: An Study of Jev-Style Visual Choice Model

   Xun-Lan Zhou, Xian-Liang Yang, Li Zhao · 2026 · arXiv.org · preprint, non relu par les pairs · cité 7 fois · Pas encore résumé

   PixelJev adapte une interface de choix Jev aux images et évalue des modèles multimodaux, tandis que cet article teste Jev sur des représentations graphiques de spectres.

   Résumé scientifique

   Visual software often needs a decision over supplied alternatives rather than a generated explanation. We present PixelJev, a native-image decision interface that maps an image, a task instruction, and a runtime candidate set to a structured choice and candidate-conditioned probabilities using small open multimodal models. Its initial realization unifies recognition and multiplechoice visual question answering through an existing language-model readout, with separately evaluated options for frozen inference, language-side adaptation, and held-out calibration. Across seven benchmark evaluations, 64-shot source adaptation raises Pets accuracy from 60.13% to 92.40% across optimization seeds and…

5. Apparenté Base de cet article Pas pertinent

   MOCLIP: a foundation model for large-scale nanophotonic inverse design

   S. Rodionov, A. Burguete-Lopez, M. Makarenko et al. · 2025 · Nature Communications · cité 5 fois · Pas encore résumé

   MOCLIP fournit les données expérimentales de métasurfaces dont cet article évalue les spectres et les décisions multimodales.

   Résumé scientifique

   Foundation models are transforming artificial intelligence by enabling generalizable, data-efficient solutions across diverse domains and applications. However, the lack of large and diverse datasets remains a key barrier to their development in nanophotonics. This work presents MOCLIP (Metasurface Optics Contrastive Learning Pretrained), a nanophotonic foundation model that encodes metasurfaces’ structural and spectral information into a shared latent space via contrastive learning, using an experimentally acquired dataset with sample density approaching the scale of ImageNet-1K. The study demonstrates MOCLIP’s inverse design capabilities, including high-throughput zero-shot prediction at 2…