John,
Never underestimate your students abilities. I've taught 200-300 stats students and tutored hundreds more. Out of all of those students, I can only think of 5-6 that just weren't very good. I also give my students questions from the PhD qualifying exams for Industrial Engineers at WAyne State University. Once you remove all the Calculus and proof stuff, there is about 70% of each exam that the students could do.
If you weave everything in well, and give them a formula or 2, they can do wonderful things. I'm going to give my students something like:
Suppose there are 10,000,000 people in the state. Calculate the number of people that are hospitalized and the number that die under the given situations.
1) In state A, they decide "exposure parties" are a great idea.
2) In state B, they decide "closing the state" can't happen.
3) In state C, they decide "Stay at home" is a good idea and mostly do it.
I will give them an equation and the values to use, and see what they get.
Then I will ask, between the 3 states, which one do you think had the "best" idea, based upon the outcomes? (All reasoned and reasonable answers are corrrect.)
I will follow that up with:
Suppose that each person in the hospital cost $10,000. Each person that died cost $300,000. During the pandemic, economic output was $X per worker regardless of state. Based upon Decision Analysis, which state did best? worst? ( I might have state A and B do better economically. might not.)
And follow that up with:
Do you take issue with the costs used above? Why or why not?
Based the economic and personal outcomes, which state do you think did "best"? (All reasoned and reasonable answers are correct.)
(I also require them to write their answers in english, not mathenese.)
I expect 80% to 90% of my students to get every point on this question.
I'll also have the testings and quarantining questions I wrote earlier..... (I think I wrote them here.)
------------------------------
Andrew Ekstrom
Statistician, Chemist, HPC Abuser;-)
------------------------------
Original Message:
Sent: 04-20-2020 13:40
From: John Conrad
Subject: False positive, false negative, sensitivity, specificity of COVID testing (for teaching)
Hi Jennifer,
I am teaching Intro Statistics at a local Community College and I am also interested in including something on the COVID -19 testing in class. I am probably looking at something even more basic in the sense that I would first like to discuss how viruses spread and how the professionals approach sampling and testing.
If you come across any good worksheets or discussions about this please let me know.
Also, thank you for posting your interest in teaching this topic...I think this COVID-19 could be a great teaching "moment" for us. The only problem is that much of the discussions so far are on such a high level my students really can't follow what is being said...I need something appropriate for a first year college student in their first Statistics course and just learning about Hypothesis Testing, etc.
John Conrad
------------------------------
John Conrad
Teacher
Hudson Valley Community College
Original Message:
Sent: 04-14-2020 14:08
From: Jennifer Ward
Subject: False positive, false negative, sensitivity, specificity of COVID testing (for teaching)
Thank you, Alicia! I especially appreciate your explanations of PPV and NPV where you clearly specify what the denominator value is.
Thank you also for the link to the COVID-19 test materials!
------------------------------
Jennifer Ward
Clark College
Original Message:
Sent: 04-13-2020 18:39
From: Alicia Toledano
Subject: False positive, false negative, sensitivity, specificity of COVID testing (for teaching)
Jennifer, yes, you've got the correct terminology.
David, great resource (http://for-sci-law.blogspot.com/)!
Quick-and-easy definitions for those new to diagnostic tests:
- False positive result: Positive test result for someone who does not have the condition of interest (here, COVID).
- False negative result: Negative test result for someone who does have COVID.
- Sensitivity: Ability of a test to identify (give a positive result for) someone with COVID; percentage of people with COVID whose test values are positive (denominator = # of people with COVID).
- Specificity: Ability of a test to give a negative result for someone who does not have COVID; percentage of people without COVID whose test results are negative (denominator = # of people without COVID).
Also of interest are predictive values - because when someone gets tested, they don't yet know whether they have COVID. These depend on prevalence of COVID, which could vary by state, suspected risk factors, etc. When evaluating test performance in a group of interest (representative prevalence):
- Positive predictive value (PPV): Percentage of people with a positive test result who have COVID (denominator = # of positive test results).
- Negative predictive value (NPV): Percentage of people with a negative test result who do not have COVID (denominator = # of negative test results).
It often surprises people that in populations with a low prevalence of disease, tests with extremely high sensitivity and specificity (extremely low false negative and false positive rates) have very low positive predictive values. Making a 2-by-2 table shows why: Take a large population with a low prevalence of disease. Obtain expected cell counts using sensitivity and specificity. Most of the positive test results are from people without the condition of interest - because there are orders of magnitude more of them to begin with!
When evaluating test performance in an enriched group - let's say, using a new test on some number of people already confirmed to have COVID and an equal number of people already confirmed to be COVID-free - we have to adjust back to the prevalence in the population of interest.
One advantage of living in the US is that FDA's decisions are available on the internet for anyone who searches - pending time to post and with varying levels of detail.
To find the latest posted materials on COVID tests:
https://www.fda.gov/emergency-preparedness-and-response/coronavirus-disease-2019-covid-19/coronavirus-disease-2019-covid-19-frequently-asked-questions
(accessed 2020-04-13) notes: "Currently there is no FDA-approved or cleared test to diagnose or detect COVID-19 because the virus that causes COVID-19 is new. Therefore, the FDA has issued several Emergency Use Authorizations (EUAs) ..."
The EUA list is posted at:
https://www.fda.gov/medical-devices/emergency-situations-medical-devices/emergency-use-authorizations#covid19ivd
There are 33 entries in the "Test Kit Manufacturers and Commercial Laboratories Table"
To see what goes to doctors, in the "Authorization Documents" column click "HCP;" similarly, for "Patients." To see numbers on sensitivity and specificity click "IFU" and look at Clinical Evaluation (or Clinical Performance).
There are 10 entries in "High Complexity Molecular-Based Laboratory Developed Tests." To see numbers on sensitivity click "EUA Summary" and again look for Clinical Evaluation.
[NOTE: Documents also report Analytical Sensitivity and Analytical Specificity for bench testing.]
Hope that helps,
Alicia
------------------------------
Alicia Toledano
President
Biostatistics Consulting, LLC
Original Message:
Sent: 04-10-2020 14:20
From: Jennifer Ward
Subject: False positive, false negative, sensitivity, specificity of COVID testing (for teaching)
Hello everyone,
I teach introductory statistics and I'm having a hard time finding information about the false positive rates, false negative rates, sensitivity, and specificity of COVID testing*. I'm not sure where or in what kinds of medical literature to find these details, either. If there is more than one diagnostic test, won't these 4 rates vary by manufacturer?
If you're from the medical/statistical world and want to share more background information about these probabilities, please share!
Thanks for your help. I look forward to using what I learn as a teaching tool this term. :D
-Jennifer
*My apologies if I use the wrong medical terminology. I welcome corrections (in a private message) so that I'm not teaching my students incorrectly. :)
------------------------------
Jennifer Ward
Clark College
Vancouver, WA
------------------------------