Making decisions with evidential probability and objective Bayesian calibration inductive logics

Mantas RADZVILAS; William PEDEN; Francesco DE PRETIS

doi:10.1016/j.ijar.2023.109030

Making decisions with evidential probability and objective Bayesian calibration inductive logics

Mantas RADZVILAS, William PEDEN, Francesco DE PRETIS^*

^*Corresponding author for this work

Department of Philosophy

Research output: Journal Publications › Journal Article (refereed) › peer-review

Abstract

Calibration inductive logics are based on accepting estimates of relative frequencies, which are used to generate imprecise probabilities. In turn, these imprecise probabilities are intended to guide beliefs and decisions — a process called “calibration”. Two prominent examples are Henry E. Kyburg's system of Evidential Probability and Jon Williamson's version of Objective Bayesianism. There are many unexplored questions about these logics. How well do they perform in the short-run? Under what circumstances do they do better or worse? What is their performance relative to traditional Bayesianism?

In this article, we develop an agent-based model of a classic binomial decision problem, including players based on variations of Evidential Probability and Objective Bayesianism. We compare the performances of these players, including against a benchmark player who uses standard Bayesian inductive logic. We find that the calibrated players can match the performance of the Bayesian player, but only with particular acceptance thresholds and decision rules. Among other points, our discussion raises some challenges for characterising “cautious” reasoning using imprecise probabilities. Thus, we demonstrate a new way of systematically comparing imprecise probability systems, and we conclude that calibration inductive logics are surprisingly promising for making decisions.

Original language	English
Article number	109030
Number of pages	37
Journal	International Journal of Approximate Reasoning
Volume	162
Early online date	19 Sept 2023
DOIs	https://doi.org/10.1016/j.ijar.2023.109030
Publication status	Published - Nov 2023

Bibliographical note

Funding Information:
We thank Daniele Tortoli (University of Modena and Reggio Emilia, Italy) for his very valuable support in accelerating computations in our research. The work described in this article was partly supported by a Senior Research Fellowship award from the Research Grants Council of the Hong Kong SAR, China (“Philosophy of Contemporary and Future Science”, Project no. SRFS2122-3H01 ), German Research Foundation project SP 279/21-1 (project no. 420094936 ), and the University of Modena and Reggio Emilia , Italy. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of this article. The Authors are grateful to the University of Modena and Reggio Emilia for providing open access for this article.

Funding Information:
The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Mantas Radzvilas reports that financial support was provided by German Research Foundation (project SP 279/21-1, project no. 420094936). William Peden reports that financial support was provided by Research Grants Council, University Grants Committee, project no. SRFS2122-3H01 of the Hong Kong SAR, China. Francesco De Pretis reports that financial support was provided by University of Modena and Reggio Emilia.We thank Daniele Tortoli (University of Modena and Reggio Emilia, Italy) for his very valuable support in accelerating computations in our research. The work described in this article was partly supported by a Senior Research Fellowship award from the Research Grants Council of the Hong Kong SAR, China (“Philosophy of Contemporary and Future Science”, Project no. SRFS2122-3H01), German Research Foundation project SP 279/21-1 (project no. 420094936), and the University of Modena and Reggio Emilia, Italy. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of this article. The Authors are grateful to the University of Modena and Reggio Emilia for providing open access for this article.

Publisher Copyright:
© 2023 The Author(s)

Keywords

Agent-based modelling
Imprecise probability
Machine learning
Decision under uncertainty
Frequentist statistics
Objective Bayesianism

Access to Document

10.1016/j.ijar.2023.109030Licence: CC BY

https://linkinghub.elsevier.com/retrieve/pii/S0888613X23001615

Cite this

@article{f413d3acfc8649768f9751696fc8a662,

title = "Making decisions with evidential probability and objective Bayesian calibration inductive logics",

abstract = "Calibration inductive logics are based on accepting estimates of relative frequencies, which are used to generate imprecise probabilities. In turn, these imprecise probabilities are intended to guide beliefs and decisions — a process called “calibration”. Two prominent examples are Henry E. Kyburg's system of Evidential Probability and Jon Williamson's version of Objective Bayesianism. There are many unexplored questions about these logics. How well do they perform in the short-run? Under what circumstances do they do better or worse? What is their performance relative to traditional Bayesianism?In this article, we develop an agent-based model of a classic binomial decision problem, including players based on variations of Evidential Probability and Objective Bayesianism. We compare the performances of these players, including against a benchmark player who uses standard Bayesian inductive logic. We find that the calibrated players can match the performance of the Bayesian player, but only with particular acceptance thresholds and decision rules. Among other points, our discussion raises some challenges for characterising “cautious” reasoning using imprecise probabilities. Thus, we demonstrate a new way of systematically comparing imprecise probability systems, and we conclude that calibration inductive logics are surprisingly promising for making decisions.",

keywords = "Agent-based modelling, Imprecise probability, Machine learning, Decision under uncertainty, Frequentist statistics, Objective Bayesianism",

author = "Mantas RADZVILAS and William PEDEN and {DE PRETIS}, Francesco",

note = "Funding Information: We thank Daniele Tortoli (University of Modena and Reggio Emilia, Italy) for his very valuable support in accelerating computations in our research. The work described in this article was partly supported by a Senior Research Fellowship award from the Research Grants Council of the Hong Kong SAR, China (“Philosophy of Contemporary and Future Science”, Project no. SRFS2122-3H01 ), German Research Foundation project SP 279/21-1 (project no. 420094936 ), and the University of Modena and Reggio Emilia , Italy. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of this article. The Authors are grateful to the University of Modena and Reggio Emilia for providing open access for this article. Funding Information: The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Mantas Radzvilas reports that financial support was provided by German Research Foundation (project SP 279/21-1, project no. 420094936). William Peden reports that financial support was provided by Research Grants Council, University Grants Committee, project no. SRFS2122-3H01 of the Hong Kong SAR, China. Francesco De Pretis reports that financial support was provided by University of Modena and Reggio Emilia.We thank Daniele Tortoli (University of Modena and Reggio Emilia, Italy) for his very valuable support in accelerating computations in our research. The work described in this article was partly supported by a Senior Research Fellowship award from the Research Grants Council of the Hong Kong SAR, China (“Philosophy of Contemporary and Future Science”, Project no. SRFS2122-3H01), German Research Foundation project SP 279/21-1 (project no. 420094936), and the University of Modena and Reggio Emilia, Italy. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of this article. The Authors are grateful to the University of Modena and Reggio Emilia for providing open access for this article. Publisher Copyright: {\textcopyright} 2023 The Author(s)",

year = "2023",

month = nov,

doi = "10.1016/j.ijar.2023.109030",

language = "English",

volume = "162",

journal = "International Journal of Approximate Reasoning",

issn = "0888-613X",

publisher = "Elsevier Inc.",

}

TY - JOUR

T1 - Making decisions with evidential probability and objective Bayesian calibration inductive logics

AU - RADZVILAS, Mantas

AU - PEDEN, William

AU - DE PRETIS, Francesco

N1 - Funding Information: We thank Daniele Tortoli (University of Modena and Reggio Emilia, Italy) for his very valuable support in accelerating computations in our research. The work described in this article was partly supported by a Senior Research Fellowship award from the Research Grants Council of the Hong Kong SAR, China (“Philosophy of Contemporary and Future Science”, Project no. SRFS2122-3H01 ), German Research Foundation project SP 279/21-1 (project no. 420094936 ), and the University of Modena and Reggio Emilia , Italy. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of this article. The Authors are grateful to the University of Modena and Reggio Emilia for providing open access for this article. Funding Information: The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Mantas Radzvilas reports that financial support was provided by German Research Foundation (project SP 279/21-1, project no. 420094936). William Peden reports that financial support was provided by Research Grants Council, University Grants Committee, project no. SRFS2122-3H01 of the Hong Kong SAR, China. Francesco De Pretis reports that financial support was provided by University of Modena and Reggio Emilia.We thank Daniele Tortoli (University of Modena and Reggio Emilia, Italy) for his very valuable support in accelerating computations in our research. The work described in this article was partly supported by a Senior Research Fellowship award from the Research Grants Council of the Hong Kong SAR, China (“Philosophy of Contemporary and Future Science”, Project no. SRFS2122-3H01), German Research Foundation project SP 279/21-1 (project no. 420094936), and the University of Modena and Reggio Emilia, Italy. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of this article. The Authors are grateful to the University of Modena and Reggio Emilia for providing open access for this article. Publisher Copyright: © 2023 The Author(s)

PY - 2023/11

Y1 - 2023/11

N2 - Calibration inductive logics are based on accepting estimates of relative frequencies, which are used to generate imprecise probabilities. In turn, these imprecise probabilities are intended to guide beliefs and decisions — a process called “calibration”. Two prominent examples are Henry E. Kyburg's system of Evidential Probability and Jon Williamson's version of Objective Bayesianism. There are many unexplored questions about these logics. How well do they perform in the short-run? Under what circumstances do they do better or worse? What is their performance relative to traditional Bayesianism?In this article, we develop an agent-based model of a classic binomial decision problem, including players based on variations of Evidential Probability and Objective Bayesianism. We compare the performances of these players, including against a benchmark player who uses standard Bayesian inductive logic. We find that the calibrated players can match the performance of the Bayesian player, but only with particular acceptance thresholds and decision rules. Among other points, our discussion raises some challenges for characterising “cautious” reasoning using imprecise probabilities. Thus, we demonstrate a new way of systematically comparing imprecise probability systems, and we conclude that calibration inductive logics are surprisingly promising for making decisions.

AB - Calibration inductive logics are based on accepting estimates of relative frequencies, which are used to generate imprecise probabilities. In turn, these imprecise probabilities are intended to guide beliefs and decisions — a process called “calibration”. Two prominent examples are Henry E. Kyburg's system of Evidential Probability and Jon Williamson's version of Objective Bayesianism. There are many unexplored questions about these logics. How well do they perform in the short-run? Under what circumstances do they do better or worse? What is their performance relative to traditional Bayesianism?In this article, we develop an agent-based model of a classic binomial decision problem, including players based on variations of Evidential Probability and Objective Bayesianism. We compare the performances of these players, including against a benchmark player who uses standard Bayesian inductive logic. We find that the calibrated players can match the performance of the Bayesian player, but only with particular acceptance thresholds and decision rules. Among other points, our discussion raises some challenges for characterising “cautious” reasoning using imprecise probabilities. Thus, we demonstrate a new way of systematically comparing imprecise probability systems, and we conclude that calibration inductive logics are surprisingly promising for making decisions.

KW - Agent-based modelling

KW - Imprecise probability

KW - Machine learning

KW - Decision under uncertainty

KW - Frequentist statistics

KW - Objective Bayesianism

UR - http://www.scopus.com/inward/record.url?scp=85173128443&partnerID=8YFLogxK

U2 - 10.1016/j.ijar.2023.109030

DO - 10.1016/j.ijar.2023.109030

M3 - Journal Article (refereed)

SN - 0888-613X

VL - 162

JO - International Journal of Approximate Reasoning

JF - International Journal of Approximate Reasoning

M1 - 109030

ER -

Making decisions with evidential probability and objective Bayesian calibration inductive logics

Abstract

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Philosophy of Contemporary and Future Science

Cite this

Making decisions with evidential probability and objective Bayesian calibration inductive logics

Abstract

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Projects

Philosophy of Contemporary and Future Science

Cite this