ASA Connect

Is anyone else tired of reading about how projections of infections could be as high as xxx,xxx by August for the country/world/state?  I find it personally exhausting that as the bloated news cycle continues, long term projections are reported as point estimates and delivered as truth.  And people I'm talking to accept it as gospel truth.

The fact that the time series for this outbreak is non-stationary, with wild variance, inaccurate and vastly incomplete data (with only 3 months worth of data) amounts to the idea that long term projections are mostly garbage, and should never be reported as point estimates.

Is there any sort of education or spotlighting we're doing in combination with news outlets to inform the general public about statistical literacy? 

*I want to make it clear that my intention is not to undermine any of the hard work many in our profession are doing.  But should we play our role in advising that that projections are inherently tricky and should be treated as such?

------------------------------
Weston McManus
SAS Institute
------------------------------

This was a surprising by the 2nd bullet when I read the document on the link below that was posted recently on this list. 

2. Be mindful that forecast models are often built with the goal of change, which affects their shelf life.
The irony of many COVID-19 modeling purposes is that in some cases, especially for forecasting, a key purpose in building and disseminating the model is to invoke behavior change at individual or system levels-e.g., to reinforce the need for physical distancing.

This makes it difficult to assess the performance of forecasting models since the results of the model itself (and reactions to it) become part of the system. In these cases, a forecasting model may look like it was inaccurate, but it may have been accurate for an unmitigated scenario with no behavior change. In fact, a public health success may be when the forecasts do not come to be!

Perhaps the is the underlying motivation for the wide spread circulation of long term and inherently unstable projections.  

source:  https://www.jhsph.edu/covid-19/articles/10-tips-for-making-sense-of-covid-19-models-for-decision-making.html


------------------------------
Eugene Komaroff
Professor of Education
Keiser University Graduate School
------------------------------

Original Message:
Sent: 05-05-2020 09:32
From: Weston McManus
Subject: Education on Covid 19 Projections

Is anyone else tired of reading about how projections of infections could be as high as xxx,xxx by August for the country/world/state?  I find it personally exhausting that as the bloated news cycle continues, long term projections are reported as point estimates and delivered as truth.  And people I'm talking to accept it as gospel truth.

The fact that the time series for this outbreak is non-stationary, with wild variance, inaccurate and vastly incomplete data (with only 3 months worth of data) amounts to the idea that long term projections are mostly garbage, and should never be reported as point estimates.

Is there any sort of education or spotlighting we're doing in combination with news outlets to inform the general public about statistical literacy?

*I want to make it clear that my intention is not to undermine any of the hard work many in our profession are doing.  But should we play our role in advising that that projections are inherently tricky and should be treated as such?

------------------------------
Weston McManus
SAS Institute
------------------------------

​I agree that forecasting of the infections and deaths should be interpreted appropriately. The projections should always be reported with the 95% confidence interval and the assumptions used in the statistical model. To deliver the relevant statistical literacy to the general public, we should first conduct a background review of how people interpret the existing results, for example how newspapers write the story about projections.

------------------------------
Paul Lee
Assistant Professor
Hong Kong Polytechnic University
------------------------------

Original Message:
Sent: 05-05-2020 09:32
From: Weston McManus
Subject: Education on Covid 19 Projections

Is anyone else tired of reading about how projections of infections could be as high as xxx,xxx by August for the country/world/state?  I find it personally exhausting that as the bloated news cycle continues, long term projections are reported as point estimates and delivered as truth.  And people I'm talking to accept it as gospel truth.

The fact that the time series for this outbreak is non-stationary, with wild variance, inaccurate and vastly incomplete data (with only 3 months worth of data) amounts to the idea that long term projections are mostly garbage, and should never be reported as point estimates.

Is there any sort of education or spotlighting we're doing in combination with news outlets to inform the general public about statistical literacy?

*I want to make it clear that my intention is not to undermine any of the hard work many in our profession are doing.  But should we play our role in advising that that projections are inherently tricky and should be treated as such?

------------------------------
Weston McManus
SAS Institute
------------------------------