大模型时代自动问答系统及评价体系综述

大模型时代，自动问答系统呈现出诸多新的特征。通过文献阅读和梳理，对自动问答系统特征和评测体系进行总结与归纳，从问答模型推理训练的训练数据、预训练框架、模型后处理、模型高效微调等阶段，对比大模型发展初期 “追求数据和参数规模”的训练方法和如今“注重数据和模型效率”之间的差异，系统分析基于大模型的自动问答系统新的特征。总结当前各种类型的自动问答大模型评测体系，并详细梳理自动化评价体系HELM (Holistic Evaluation of Language Model) 在自动问答任务上的数据集、评价指标和量化计算方法。未来基于大模型的自动问答系统研究将会围绕多模态融合、高安全性、高可解释性、低资源消耗、结合大模型和自动化的综合评价体系这几个方面进一步拓展与深化。

In the era of large models (LMs), question answering (QA) systems exhibit new characteristics. This paper reviews QA system features and evaluation methods. From the stages of training data for question-answering model inference training, pre-training frameworks, model post-processing, and efficient model fine-tuning, it contrasts the early “pursuit of data and parameter scale” training methods with the current “emphasis on data and model efficiency,” and systematically analyzes the new characteristics of large-model–based automatic question-answering systems. The study summarizes existing QA evaluation methods, reviewing HELM (Holistic Evaluation of Language Models) regarding datasets, metrics, and quantitative assessment for QA tasks. Future research on LLM-based QA systems will focus on multimodal fusion, security, interpretability, efficiency, and integrated automated evaluation.