MatPlotAgent: Method and Evaluation for LLM-Based Agentic Scientific Data Visualization

Scientific data visualization plays a crucial role in research by enablingthe direct display of complex information and assisting researchers inidentifying implicit patterns. Despite its importance, the use of LargeLanguage Models (LLMs) for scientific data visualization remains ratherunexplored. In this study, we introduce MatPlotAgent, an efficientmodel-agnostic LLM agent framework designed to automate scientific datavisualization tasks. Leveraging the capabilities of both code LLMs andmulti-modal LLMs, MatPlotAgent consists of three core modules: queryunderstanding, code generation with iterative debugging, and a visual feedbackmechanism for error correction. To address the lack of benchmarks in thisfield, we present MatPlotBench, a high-quality benchmark consisting of 100human-verified test cases. Additionally, we introduce a scoring approach thatutilizes GPT-4V for automatic evaluation. Experimental results demonstrate thatMatPlotAgent can improve the performance of various LLMs, including bothcommercial and open-source models. Furthermore, the proposed evaluation methodshows a strong correlation with human-annotated scores.