Developing and validating a scenario-based cognitive bias questionnaire

Introduction and Objectives: Human decision-making often suffers from cognitive biases, leading to suboptimal outcomes across various domains. Existing tools to evaluate biases lack comprehensiveness and reliability, highlighting a crucial gap in research. To address this, we aim to develop a comprehensive questionnaire covering diverse biases in decision-making, leveraging advanced deep learning methods for item clustering and selection.

Methods: Our methodology includes an in-depth literature review, content validity assessment through expert input, pilot testing, and data analysis using exploratory factor analysis (EFA), confirmatory factor analysis (CFA), and item response theory (IRT). We identified 230 biases, refined to 92 items using deep learning clustering techniques, and conducted EFA to reveal five factors explaining 75% variance.

Results: The developed scale exhibited strong reliability (Cronbach's alpha 0.909) and validity (CFI > 0.9) with pilot studies (N=87) across the five factors: Belief, Social, Memory, Information, and Calculation biases. Item response theory (IRT) analysis identified and removed 12 low-discriminating items, enhancing the scale's efficacy in measuring cognitive biases.

Conclusion: Our study addresses critical gaps in existing tools for measuring cognitive biases, providing a robust and comprehensive instrument developed using advanced deep learning techniques. By advancing decision science and highlighting individual differences in cognitive bias, this research contributes significantly to improving decision-making outcomes.