ASA Connect

Hi All,

I've been reading about aleatory and epistemic uncertainty, mainly in the machine learning literature, and as far as I can tell:

Epistemic uncertainty = parameter uncertainty (although the ML folks call it model uncertainty)

Aleatoric uncertainty = uncertainty in a prediction

So from a Bayesian perspective it's the variance of the posterior distribution (epistemic) versus variance of the posterior predictive distribution (aleatory). No one seems to have phrased it like this, so I'm wondering if I'm missing something.

Many thanks,
Stan

------------------------------
Stanley E. Lazic, PhD
https://stanlazic.github.io/
------------------------------

Hi Stan,

Have you looked at Andrew Gelman's Bayesian coverage of this?
Ex.  http://stat.columbia.edu/~gelman/research/unpublished/augie3.pdf

Depending on how far along you've been reading, here's another good'n (although it doesn't address your question directly):
http://www.stat.columbia.edu/~gelman/research/published/philosophy_online4.pdf

Just for fun, since you seem to be interested in the philosophy of statistical science, you might also enjoy Deborah G. Mayo (https://en.wikipedia.org/wiki/Deborah_Mayo) if you haven't read something from her already. I haven't read everything she's written...but I've enjoyed everything I've read.






------------------------------
Glen Wright Colopy
DPhil Oxon
Data Scientist at Cenduit LLC, Durham, NC
------------------------------

Original Message:
Sent: 04-24-2020 16:58
From: Stanley Lazic
Subject: Aleatory vs. Epistemic uncertainty

Hi All,

I've been reading about aleatory and epistemic uncertainty, mainly in the machine learning literature, and as far as I can tell:

Epistemic uncertainty = parameter uncertainty (although the ML folks call it model uncertainty)

Aleatoric uncertainty = uncertainty in a prediction

So from a Bayesian perspective it's the variance of the posterior distribution (epistemic) versus variance of the posterior predictive distribution (aleatory). No one seems to have phrased it like this, so I'm wondering if I'm missing something.

Many thanks,
Stan

------------------------------
Stanley E. Lazic, PhD
https://stanlazic.github.io/
------------------------------

Not sure, but doesn't a subjective Bayesian place all the uncertainty -- both types -- into one basket, so that both distributions could encompass either or both types of uncertainty? I am thinking that the posterior distribution refers to the distribution of the parameter, while the predictive posterior distribution refers to the distribution of future observations of data, and that both distributions could reflect aleatory uncertainty (to the extent a random process is generating data) and epistemic uncertainty. Maybe this is too simple-minded.
By the way, the aleatory-epistemic distinction has been discussed in many fields. A recent Royal Society Open Science article on communicating uncertainty emphasizes it, but coming up with a clear definition seems tricky. A couple of paragraphs on that are at http://for-sci-law.blogspot.com/2019/06/aleatory-and-epistemic-uncertainty.html.

------------------------------
David Kaye
Penn State Law (emeritus)
------------------------------

Original Message:
Sent: 04-24-2020 16:58
From: Stanley Lazic
Subject: Aleatory vs. Epistemic uncertainty

Hi All,

I've been reading about aleatory and epistemic uncertainty, mainly in the machine learning literature, and as far as I can tell:

Epistemic uncertainty = parameter uncertainty (although the ML folks call it model uncertainty)

Aleatoric uncertainty = uncertainty in a prediction

So from a Bayesian perspective it's the variance of the posterior distribution (epistemic) versus variance of the posterior predictive distribution (aleatory). No one seems to have phrased it like this, so I'm wondering if I'm missing something.

Many thanks,
Stan

------------------------------
Stanley E. Lazic, PhD
https://stanlazic.github.io/
------------------------------

Stanley

In the Bayesian framework you can deal with all forms of uncertainty.

There is the uncertainty associated with randomness (aleatory uncertainty) from dice, playing cards, tossed coins or other random devices. But there is a second form of uncertainty associated with one's own lack of knowledge and this is called epistemic uncertainty.

Spiegelhalter (The Art of Statistics) gives the example of comparing a lottery ticket, whose outcome is determined by chance, (aleatory uncertainty) and a scratchcard. The outcome of the scratchcard is determined beforehand, you just don't know the outcome. That's your personal ignorance - epistemic uncertainty.

Frequentist statistics handle only aleatory uncertainty but Bayesian statistics handles both.

Blaise

Blaise F Egan

Chartered Statistician

------------------------------
Blaise Egan
Lead Data Scientist
British Telecommunications PLC
------------------------------

Original Message:
Sent: 04-24-2020 16:58
From: Stanley Lazic
Subject: Aleatory vs. Epistemic uncertainty

Hi All,

I've been reading about aleatory and epistemic uncertainty, mainly in the machine learning literature, and as far as I can tell:

Epistemic uncertainty = parameter uncertainty (although the ML folks call it model uncertainty)

Aleatoric uncertainty = uncertainty in a prediction

So from a Bayesian perspective it's the variance of the posterior distribution (epistemic) versus variance of the posterior predictive distribution (aleatory). No one seems to have phrased it like this, so I'm wondering if I'm missing something.

Many thanks,
Stan

------------------------------
Stanley E. Lazic, PhD
https://stanlazic.github.io/
------------------------------

Dear Glen, David, and Blaise,

Many thanks for your comments and links to resources. I'll give them a read.

Kind regards,
Stan

------------------------------
Stanley E. Lazic, PhD
https://stanlazic.github.io/
------------------------------

Original Message:
Sent: 04-24-2020 16:58
From: Stanley Lazic
Subject: Aleatory vs. Epistemic uncertainty

Hi All,

I've been reading about aleatory and epistemic uncertainty, mainly in the machine learning literature, and as far as I can tell:

Epistemic uncertainty = parameter uncertainty (although the ML folks call it model uncertainty)

Aleatoric uncertainty = uncertainty in a prediction

So from a Bayesian perspective it's the variance of the posterior distribution (epistemic) versus variance of the posterior predictive distribution (aleatory). No one seems to have phrased it like this, so I'm wondering if I'm missing something.

Many thanks,
Stan

------------------------------
Stanley E. Lazic, PhD
https://stanlazic.github.io/
------------------------------

I hate to say it like this but, make sure the papers you are reading are written by people that know what they are talking about! 

As a student of "Data Science", I've found that the books we use in ML classes and indeed the professors of those classes, have little to no clue what they are talking about when it comes to probability and often statistics. Not to mention that the models they make are based upon deterministic models and solutions, probability not used. 

This past fall, I took a class on "Pattern Recognition"  as an online student. I found so many mistakes in the class that I had to talk to the prof after class many times. The material he spoke about and the mistakes he made were all written into the textbook we used! 

Also, when you go back to a simple modeling scheme, you have 2 types of error, systematic and random, or biased and unbiased. Every model you make will change the amount of biased error in your model.

Biased error + Random error => Prediction error.  If you remember back in your first stats class, Var(A + B) = Var(A) + Var(B) - 2*Cov(A,B). So, Var(prediction) = Var(biased error) + Var(Random Error) - 2*Cov(biased, random). 

(Thanks for spotting my mistake Eric.) 

------------------------------
Andrew Ekstrom

Statistician, Chemist, HPC Abuser;-)
------------------------------

Original Message:
Sent: 04-24-2020 16:58
From: Stanley Lazic
Subject: Aleatory vs. Epistemic uncertainty

Hi All,

I've been reading about aleatory and epistemic uncertainty, mainly in the machine learning literature, and as far as I can tell:

Epistemic uncertainty = parameter uncertainty (although the ML folks call it model uncertainty)

Aleatoric uncertainty = uncertainty in a prediction

So from a Bayesian perspective it's the variance of the posterior distribution (epistemic) versus variance of the posterior predictive distribution (aleatory). No one seems to have phrased it like this, so I'm wondering if I'm missing something.

Many thanks,
Stan

------------------------------
Stanley E. Lazic, PhD
https://stanlazic.github.io/
------------------------------