Abstract
Multiple-choice questions (MCQs) are a widely used educational tool, particularly in domains such as visualization literacy that require broad conceptual coverage and support diverse real-world applications. However, designing high-quality visualization literacy MCQs remains challenging, as instructors must coordinate multimodal elements (e.g., charts, question stems, and distractors), address diverse visualization tasks, and accommodate learners with heterogeneous backgrounds. Existing visualization literacy assessments primarily rely on standardized, fixed item banks, offering limited support for iterative question design that adapts to differences in learners' abilities, backgrounds, and reasoning strategies. To address these challenges, we present VizQStudio, a visual analytics system that supports instructors in iteratively designing and refining visualization literacy MCQs using MLLM-powered simulated students. Instructors can specify diverse student profiles spanning demographics, knowledge levels, and learning-related traits. The system then visualizes how simulated students reason about and respond to different question components, helping instructors explore potential misconceptions, difficulty calibration, and design trade-offs prior to classroom deployment. We investigate VizQStudio through a mixed-method evaluation, including expert interviews, case studies, a classroom deployment, and a large-scale online study. Our results indicate that MCQs designed with VizQStudio can support measurable learning gains and, within our exploratory online sample, yielded observed post-test outcomes similar to established benchmark questions, while enabling greater flexibility and scalability during the design process. Overall, this work reframes MLLM-based student simulation in assessment authoring as a design-time, exploratory aid. By examining both its value and limitations in realistic instructional settings, we surface design insights that inform how future systems can support instructor-centered, iterative, and responsible uses of AI for multimodal assessment design in visualization literacy and related domains.
| Original language | English |
|---|---|
| Pages (from-to) | 7468-7485 |
| Number of pages | 18 |
| Journal | IEEE Transactions on Visualization and Computer Graphics |
| Volume | 32 |
| Issue number | 8 |
| Early online date | 22 May 2026 |
| DOIs | |
| Publication status | Published - 1 Aug 2026 |
Bibliographical note
Publisher Copyright:© 1995-2012 IEEE.
Funding
This work was supported by RGC GRF under Grant 16218724.
UN SDGs
This output contributes to the following UN Sustainable Development Goals (SDGs)
-
SDG 4 Quality Education
Keywords
- MLLM agents
- multiple choice question design
- Student simulation
- visualization literacy education
Fingerprint
Dive into the research topics of 'VizQStudio : Iterative Visualization Literacy MCQs Design with Simulated Students'. Together they form a unique fingerprint.Cite this
- APA
- Author
- BIBTEX
- Harvard
- Standard
- RIS
- Vancouver