ASA Connect

Hello Everyone,

Example: I have data for all of the arrests in two different boroughs of NYC. I am being asked to use a proportion test to compare the proportion (number of female arrests)/(number of all arrests) between the two boroughs to see if there is a significant difference in the proportion of female arrests. However, I have ALL of the arrest data for the two boroughs for a specific time period. Does it make sense to do a hypothesis test/proportion test to assert if there is a significant difference even though we already have the true population proportions? 

My opinion is no, however a colleague disagrees


Thanks

I agree with your colleague that you *should* do the test.  You are not really interested in the rates at the present time but rather in what those rates tell you about the long-run arrest policy or method. Say the policy was totally fair, unbiased, and 30% of arrests in both areas were female, but just by chance the rates in the sample were 29% and 31% in boroughs A and B.  Would you want to say, "Look, borough B has a higher female arrest rate?"  It did, in that period, but does that mean that its true long-term rate or the practices that influence that rate are different?

This kind of situation is complicated, and some will probably say that testing is invalid because the arrests aren't a random sample from a larger population, but they are a *sample* from a larger population in a sense, even if not quite random.

If there is a difference in the percentage of female residents in the two communities, that may complicate the analysis,

------------------------------
Edward Gracely
Drexel University
------------------------------

Original Message:
Sent: 12-17-2019 16:00
From: Melissa N
Subject: Hypothesis Testing for Entire "Population"

Hello Everyone,

Example: I have data for all of the arrests in two different boroughs of NYC. I am being asked to use a proportion test to compare the proportion (number of female arrests)/(number of all arrests) between the two boroughs to see if there is a significant difference in the proportion of female arrests. However, I have ALL of the arrest data for the two boroughs for a specific time period. Does it make sense to do a hypothesis test/proportion test to assert if there is a significant difference even though we already have the true population proportions?

My opinion is no, however a colleague disagrees


Thanks

I have heard this question or something similar many times and it often waxes philosophical. I would report confidence intervals but with a footnote that acknowledges the issue.

------------------------------
Helen Burn
Highline College
------------------------------

Original Message:
Sent: 12-18-2019 07:15
From: Edward Gracely
Subject: Hypothesis Testing for Entire "Population"

I agree with your colleague that you *should* do the test.  You are not really interested in the rates at the present time but rather in what those rates tell you about the long-run arrest policy or method. Say the policy was totally fair, unbiased, and 30% of arrests in both areas were female, but just by chance the rates in the sample were 29% and 31% in boroughs A and B.  Would you want to say, "Look, borough B has a higher female arrest rate?"  It did, in that period, but does that mean that its true long-term rate or the practices that influence that rate are different?

This kind of situation is complicated, and some will probably say that testing is invalid because the arrests aren't a random sample from a larger population, but they are a *sample* from a larger population in a sense, even if not quite random.

If there is a difference in the percentage of female residents in the two communities, that may complicate the analysis,

------------------------------
Edward Gracely
Drexel University

Original Message:
Sent: 12-17-2019 16:00
From: Melissa N
Subject: Hypothesis Testing for Entire "Population"

Hello Everyone,

Example: I have data for all of the arrests in two different boroughs of NYC. I am being asked to use a proportion test to compare the proportion (number of female arrests)/(number of all arrests) between the two boroughs to see if there is a significant difference in the proportion of female arrests. However, I have ALL of the arrest data for the two boroughs for a specific time period. Does it make sense to do a hypothesis test/proportion test to assert if there is a significant difference even though we already have the true population proportions?

My opinion is no, however a colleague disagrees


Thanks

I'll answer with another old-man's story.

35 years ago I was the "outside member" on a dissertation committee for a student in the excellent Accounting Department at the University of Tennessee. His data were all the Fortune 500 companies for a particular period of years, and his research questions were limited to understanding something about this very same set of companies and years. No sampling. He had the whole universe. Yet, the "final" draft of his dissertation contained numerous p-values. (Confidence intervals? He had used SAS, and back then, SAS didn't report many CIs.) In his dissertation defense meeting, I made the same--obvious--point you are making here.

There was a little give-and-take, but we all soon agreed that all the p-values had to go. If there were any CIs, he would have nixed those, too. The analyses stood just fine; in fact, it was clearer without all this unnecessary complexity (statistical goo). The rest of the student's defense was excellent, and I'll bet he went on to have a fine career.* (Yet, if/when he tried to publish the work, did some journal reviewer or editor require him to add back in all those damn p-values?)

------------------------
*He was actually a student who minored in statistics, a formal program I created and ran. A few weeks later he and another statistics-minor student who was also a soon-to-graduate PhD in Accounting were talking in the hallway. They looked up and saw me and told me that both had accepted faculty positions and both were given sizeable starting salary bumps because they had minored in statistics. They told me what those salaries were, and each was materially greater than mine as an associate professor in Statistics.

I had a terrific relationship with terrific Statistics Chair David Sylwester. In his office I joked to him about this "inequity:"

David: Do you want to be an accountant?

Me: No.

David: Then get out!



------------------------------
Ralph O'Brien
Professor of Biostatistics (officially retired; still keenly active)
Case Western Reserve University
http://rfuncs.weebly.com/about-ralph-obrien.html
------------------------------

Original Message:
Sent: 12-17-2019 16:00
From: Melissa N
Subject: Hypothesis Testing for Entire "Population"

Hello Everyone,

Example: I have data for all of the arrests in two different boroughs of NYC. I am being asked to use a proportion test to compare the proportion (number of female arrests)/(number of all arrests) between the two boroughs to see if there is a significant difference in the proportion of female arrests. However, I have ALL of the arrest data for the two boroughs for a specific time period. Does it make sense to do a hypothesis test/proportion test to assert if there is a significant difference even though we already have the true population proportions?

My opinion is no, however a colleague disagrees


Thanks

What a timely question. I am working with a PI who is developing a similar study in that we will have all cases from a specific local population and there is a desire to look for a change in a ratio based on a policy change.  I tend to agree with Edward Gracely's comment. In medical research at my small hospital, we very often will be able to use medical records to pull all the data for a given study population.  But I still run significance tests because I determine this population is still a kind of sample of the larger population.  Even though it is not a purposefully drawn sample intended to be representative of the full population. This flaw in design is unavoidable for residents attempting to do basic research using local population data, but we make sure they document it in the limitations section of any poster or publication.

If there were an instance where the entire population of a country were surveyed (i.e. the full U.S. Census), then I could see dropping the idea of statistical tests of significance.

------------------------------
David Metcalf
Research Statistician
Henry Ford Allegiance Health
------------------------------

Original Message:
Sent: 12-17-2019 16:00
From: Melissa N
Subject: Hypothesis Testing for Entire "Population"

Hello Everyone,

Example: I have data for all of the arrests in two different boroughs of NYC. I am being asked to use a proportion test to compare the proportion (number of female arrests)/(number of all arrests) between the two boroughs to see if there is a significant difference in the proportion of female arrests. However, I have ALL of the arrest data for the two boroughs for a specific time period. Does it make sense to do a hypothesis test/proportion test to assert if there is a significant difference even though we already have the true population proportions?

My opinion is no, however a colleague disagrees


Thanks

It depends on why you are comparing these two proportions and what you/your colleague are planning on doing with this information.If you are only interested in these two groups of people at a specific point in time, then, of course, there is no point in introducing sampling variability into the analysis as no sampling has happened. The proportions are likely numerically different, though, so you will have to decide based on background knowledge whether the difference is large enough to be of interest or concern.

If, however, you might want to generalize the findings to other "similar" boroughs, or future (and, possibly, different) pool of arrests in these boroughs (assuming that there is no trend), then conducting the proportion test is appropriate. Therefore, you might have to clarify with your colleague the goals of this analysis.

------------------------------
Victoria Liublinska Prince
Sr. Statistician / Data Scientist
Harvard Business School
------------------------------

Original Message:
Sent: 12-17-2019 16:00
From: Melissa N
Subject: Hypothesis Testing for Entire "Population"

Hello Everyone,

Example: I have data for all of the arrests in two different boroughs of NYC. I am being asked to use a proportion test to compare the proportion (number of female arrests)/(number of all arrests) between the two boroughs to see if there is a significant difference in the proportion of female arrests. However, I have ALL of the arrest data for the two boroughs for a specific time period. Does it make sense to do a hypothesis test/proportion test to assert if there is a significant difference even though we already have the true population proportions?

My opinion is no, however a colleague disagrees


Thanks

I think I agree with Victoria Prince that the answer is "it depends". In particular, I think, it depends on on the purpose. Similar questions have arisen previously and I repeat some points from previous threads here. For one, the first two pages of Thomas Lumley's book Complex Surveys: A Guide to Analysis Using R, includes a good discussion on these matters. Sometimes the purpose is to generalize from a sample to a population, sometimes it is to generalize to the process that created the data, sometimes there are elements of both. Also, Michael Höhle had in interesting article in Significance June 2017 relevant to this theme, see an open access preliminary version at http://staff.math.su.se/hoehle/naming/Naming_Uncertainty-r01.html (I have a reservation towards the term "super-population model", a population model is a model and not a larger population). This discussion is not all all new, see for example the article by Deming and Stephen from 1941, On the Interpretation of Censuses as Samples, JASA 36 No 213; 45-49, and the article by Deming from 1975, On probability as a basis for action, The American Statistician 29 No 4; 146-152. To conclude, there are a lot of cases where p-values, confidence intervals and other inferential information make good sense for data from a whole population. There are also cases where it is not so.

------------------------------
[Tore][Wentzel-Larsen][Tore Wentzel-Larsen
Researcher
Norwegian Centre for Violence and Traumatic Stress Studies,
Regional Center for Child and Adolescent Mental Health, Eastern and Southern Norway]
------------------------------

Original Message:
Sent: 12-18-2019 10:03
From: Victoria Prince
Subject: Hypothesis Testing for Entire "Population"

It depends on why you are comparing these two proportions and what you/your colleague are planning on doing with this information.If you are only interested in these two groups of people at a specific point in time, then, of course, there is no point in introducing sampling variability into the analysis as no sampling has happened. The proportions are likely numerically different, though, so you will have to decide based on background knowledge whether the difference is large enough to be of interest or concern.

If, however, you might want to generalize the findings to other "similar" boroughs, or future (and, possibly, different) pool of arrests in these boroughs (assuming that there is no trend), then conducting the proportion test is appropriate. Therefore, you might have to clarify with your colleague the goals of this analysis.

------------------------------
Victoria Liublinska Prince
Sr. Statistician / Data Scientist
Harvard Business School

Original Message:
Sent: 12-17-2019 16:00
From: Melissa N
Subject: Hypothesis Testing for Entire "Population"

Hello Everyone,

Example: I have data for all of the arrests in two different boroughs of NYC. I am being asked to use a proportion test to compare the proportion (number of female arrests)/(number of all arrests) between the two boroughs to see if there is a significant difference in the proportion of female arrests. However, I have ALL of the arrest data for the two boroughs for a specific time period. Does it make sense to do a hypothesis test/proportion test to assert if there is a significant difference even though we already have the true population proportions?

My opinion is no, however a colleague disagrees


Thanks

Original Message:
Sent: 12-18-2019 11:19
From: Tore Wentzel-Larsen
Subject: Hypothesis Testing for Entire "Population"

I think I agree with Victoria Prince that the answer is "it depends". In particular, I think, it depends on on the purpose. Similar questions have arisen previously and I repeat some points from previous threads here. For one, the first two pages of Thomas Lumley's book Complex Surveys: A Guide to Analysis Using R, includes a good discussion on these matters. Sometimes the purpose is to generalize from a sample to a population, sometimes it is to generalize to the process that created the data, sometimes there are elements of both. Also, Michael Höhle had in interesting article in Significance June 2017 relevant to this theme, see an open access preliminary version at http://staff.math.su.se/hoehle/naming/Naming_Uncertainty-r01.html (I have a reservation towards the term "super-population model", a population model is a model and not a larger population). This discussion is not all all new, see for example the article by Deming and Stephen from 1941, On the Interpretation of Censuses as Samples, JASA 36 No 213; 45-49, and the article by Deming from 1975, On probability as a basis for action, The American Statistician 29 No 4; 146-152. To conclude, there are a lot of cases where p-values, confidence intervals and other inferential information make good sense for data from a whole population. There are also cases where it is not so.

------------------------------
[Tore][Wentzel-Larsen][Tore Wentzel-Larsen
Researcher
Norwegian Centre for Violence and Traumatic Stress Studies,
Regional Center for Child and Adolescent Mental Health, Eastern and Southern Norway]

Original Message:
Sent: 12-18-2019 10:03
From: Victoria Prince
Subject: Hypothesis Testing for Entire "Population"

It depends on why you are comparing these two proportions and what you/your colleague are planning on doing with this information.If you are only interested in these two groups of people at a specific point in time, then, of course, there is no point in introducing sampling variability into the analysis as no sampling has happened. The proportions are likely numerically different, though, so you will have to decide based on background knowledge whether the difference is large enough to be of interest or concern.

If, however, you might want to generalize the findings to other "similar" boroughs, or future (and, possibly, different) pool of arrests in these boroughs (assuming that there is no trend), then conducting the proportion test is appropriate. Therefore, you might have to clarify with your colleague the goals of this analysis.

------------------------------
Victoria Liublinska Prince
Sr. Statistician / Data Scientist
Harvard Business School

Original Message:
Sent: 12-17-2019 16:00
From: Melissa N
Subject: Hypothesis Testing for Entire "Population"

Hello Everyone,

Example: I have data for all of the arrests in two different boroughs of NYC. I am being asked to use a proportion test to compare the proportion (number of female arrests)/(number of all arrests) between the two boroughs to see if there is a significant difference in the proportion of female arrests. However, I have ALL of the arrest data for the two boroughs for a specific time period. Does it make sense to do a hypothesis test/proportion test to assert if there is a significant difference even though we already have the true population proportions?

My opinion is no, however a colleague disagrees


Thanks

Well, if a client approached me and said that they only wanted to compare the observed two groups at that time and not to generalize to anything, my first response would be, "Why?" 

If the client could provide a rationale for why that specific comparison was important that did not implicitly assume that it generalized to anything else, so be it. no p value or CI needed.

I can picture such reasons for some comparisons. Maybe there is a resource allocation issue for a comparison of the number of chronically ill people in two areas.  I just want to know how much money each area needs right now, and even small numerical differences are real, assuming I have complete data.

It is, however, hard for me to see why that would be of interest for a comparison of female arrest rates. Even if the goal was to allocate jail space or female-specific councilors, they'd still want to know if the pattern is likely to continue (hence generalize).

I would pin them down and make sure they truly understand that no generalization means that they cannot make assertions about whether the difference represents an underlying pattern or is likely to continue. Many non-statisticians don't understand that,

Ed

------------------------------
Edward Gracely
Drexel University
------------------------------

Original Message:
Sent: 12-18-2019 11:35
From: Melissa N
Subject: Hypothesis Testing for Entire "Population"

Original Message:
Sent: 12-18-2019 11:19
From: Tore Wentzel-Larsen
Subject: Hypothesis Testing for Entire "Population"

I think I agree with Victoria Prince that the answer is "it depends". In particular, I think, it depends on on the purpose. Similar questions have arisen previously and I repeat some points from previous threads here. For one, the first two pages of Thomas Lumley's book Complex Surveys: A Guide to Analysis Using R, includes a good discussion on these matters. Sometimes the purpose is to generalize from a sample to a population, sometimes it is to generalize to the process that created the data, sometimes there are elements of both. Also, Michael Höhle had in interesting article in Significance June 2017 relevant to this theme, see an open access preliminary version at http://staff.math.su.se/hoehle/naming/Naming_Uncertainty-r01.html (I have a reservation towards the term "super-population model", a population model is a model and not a larger population). This discussion is not all all new, see for example the article by Deming and Stephen from 1941, On the Interpretation of Censuses as Samples, JASA 36 No 213; 45-49, and the article by Deming from 1975, On probability as a basis for action, The American Statistician 29 No 4; 146-152. To conclude, there are a lot of cases where p-values, confidence intervals and other inferential information make good sense for data from a whole population. There are also cases where it is not so.

------------------------------
[Tore][Wentzel-Larsen][Tore Wentzel-Larsen
Researcher
Norwegian Centre for Violence and Traumatic Stress Studies,
Regional Center for Child and Adolescent Mental Health, Eastern and Southern Norway]

Original Message:
Sent: 12-18-2019 10:03
From: Victoria Prince
Subject: Hypothesis Testing for Entire "Population"

It depends on why you are comparing these two proportions and what you/your colleague are planning on doing with this information.If you are only interested in these two groups of people at a specific point in time, then, of course, there is no point in introducing sampling variability into the analysis as no sampling has happened. The proportions are likely numerically different, though, so you will have to decide based on background knowledge whether the difference is large enough to be of interest or concern.

If, however, you might want to generalize the findings to other "similar" boroughs, or future (and, possibly, different) pool of arrests in these boroughs (assuming that there is no trend), then conducting the proportion test is appropriate. Therefore, you might have to clarify with your colleague the goals of this analysis.

------------------------------
Victoria Liublinska Prince
Sr. Statistician / Data Scientist
Harvard Business School

Original Message:
Sent: 12-17-2019 16:00
From: Melissa N
Subject: Hypothesis Testing for Entire "Population"

Hello Everyone,

Example: I have data for all of the arrests in two different boroughs of NYC. I am being asked to use a proportion test to compare the proportion (number of female arrests)/(number of all arrests) between the two boroughs to see if there is a significant difference in the proportion of female arrests. However, I have ALL of the arrest data for the two boroughs for a specific time period. Does it make sense to do a hypothesis test/proportion test to assert if there is a significant difference even though we already have the true population proportions?

My opinion is no, however a colleague disagrees


Thanks

It seems to me the most important piece of information would be whether the differences really are "different."

------------------------------
Michael Mout
MIKS
------------------------------

Original Message:
Sent: 12-17-2019 16:00
From: Melissa N
Subject: Hypothesis Testing for Entire "Population"

Hello Everyone,

Example: I have data for all of the arrests in two different boroughs of NYC. I am being asked to use a proportion test to compare the proportion (number of female arrests)/(number of all arrests) between the two boroughs to see if there is a significant difference in the proportion of female arrests. However, I have ALL of the arrest data for the two boroughs for a specific time period. Does it make sense to do a hypothesis test/proportion test to assert if there is a significant difference even though we already have the true population proportions?

My opinion is no, however a colleague disagrees


Thanks

What role does "really" play in this question? Is it a nod towards the "fact" that hypothesis testing gives us a sense of reality? If so, I beg to differ. The differences are the differences; they represent the entire population, hence they are real. I am asking myself whether a p-value adds to or detracts from the clarity of the statistical picture.

I think questions like these are at the heart of a lot of hypothesis testing. We need clarity on them, especially in the discussions begun by the ASA about what p "really" means and how or if we should be using it. I vote that a p value is not a litmus test for reality. 

------------------------------
Sheila Braun
------------------------------

Original Message:
Sent: 12-18-2019 10:21
From: Michael Mout
Subject: Hypothesis Testing for Entire "Population"

It seems to me the most important piece of information would be whether the differences really are "different."

------------------------------
Michael Mout
MIKS

Original Message:
Sent: 12-17-2019 16:00
From: Melissa N
Subject: Hypothesis Testing for Entire "Population"

Hello Everyone,

Example: I have data for all of the arrests in two different boroughs of NYC. I am being asked to use a proportion test to compare the proportion (number of female arrests)/(number of all arrests) between the two boroughs to see if there is a significant difference in the proportion of female arrests. However, I have ALL of the arrest data for the two boroughs for a specific time period. Does it make sense to do a hypothesis test/proportion test to assert if there is a significant difference even though we already have the true population proportions?

My opinion is no, however a colleague disagrees


Thanks

​Depending on what the purpose of the question might be, I would assume the arrests are a realization of a (perhaps pseudo-) random process.  In that case, it makes sense to consider a statistically based comparison.  What the arrests represent is another question.  The number of arrests are not a fixed feature of the underlying population, and may vary--among other things--according to the level of enforcement.  Even the underlying population is probably not fixed, people come, people go, people are born, people die, people are incarcerated, people are released from incarceration to the borough.

------------------------------
Raoul Burchette
Biostatistician III
Kaiser Permanente, Pasadena, CA USA
------------------------------

Original Message:
Sent: 12-17-2019 16:00
From: Melissa N
Subject: Hypothesis Testing for Entire "Population"

Hello Everyone,

Example: I have data for all of the arrests in two different boroughs of NYC. I am being asked to use a proportion test to compare the proportion (number of female arrests)/(number of all arrests) between the two boroughs to see if there is a significant difference in the proportion of female arrests. However, I have ALL of the arrest data for the two boroughs for a specific time period. Does it make sense to do a hypothesis test/proportion test to assert if there is a significant difference even though we already have the true population proportions?

My opinion is no, however a colleague disagrees


Thanks

Hi Melissa,

David Spiegelhalter talks about this in his recent book The Art of Statistics. The short answer is: Yes, this is fine, assuming the point is inferential and not descriptive (as Victoria Prince discussed).

The reason is that one can make inferences about a process or mechanism, which can be done even if you have the whole "population". As a side note, we (statisticians) need to stop presenting statistics as inference from samples to populations to users that don't actually do any sampling, i.e. for all experimental researchers. Teaching randomisation-based inference is more appropriate and connects better with their day-to-day work. I do like the concept of making an inference about a data generating process or mechanism, as it subsumes both perspectives.

------------------------------
Stanley E. Lazic, PhD
https://stanlazic.github.io/
------------------------------

Original Message:
Sent: 12-17-2019 16:00
From: Melissa N
Subject: Hypothesis Testing for Entire "Population"

Hello Everyone,

Example: I have data for all of the arrests in two different boroughs of NYC. I am being asked to use a proportion test to compare the proportion (number of female arrests)/(number of all arrests) between the two boroughs to see if there is a significant difference in the proportion of female arrests. However, I have ALL of the arrest data for the two boroughs for a specific time period. Does it make sense to do a hypothesis test/proportion test to assert if there is a significant difference even though we already have the true population proportions?

My opinion is no, however a colleague disagrees


Thanks

I fully agree that statisticians (and others also) "need to stop presenting statistics as inference from samples to populations to users that don't actually do any sampling", and I also agree that making inference about a data generating process or mechanism is more appropriate as a general perspective. It is simply not so that all random variation stems from sampling from a population. It had been an advantage if this practice had been discontinued already in Deming's time, but it is even more important now with increasing availability and use of large data sets originating from complicated processes. To elaborate, I think that the word "population" has been used to denote profoundly different concepts, and that this term should now be reserved to a well-defined finite population from which it is possible, at least in principle, to draw a simple random sample. A broader use of the term "population" is quite old and may be traced down at least to Ronald Fisher, I may give specific citations when back at office next year.  In particular, random observations modeled as stemming from a continuous distribution should not, I think, be thought of as sampling from a population, there are profound differences between finite and infinite entities. The broader use of the term population by Fisher is a bit surprising, Fisher did important work together with Kolmogorov and was well aware of the subtleties of the foundations of probability, including the existence of non-measurable sets.

------------------------------
Tore Wentzel-Larsen
Researcher
Norwegian Centre for Violence and Traumatic Stress Studies,
Regional Center for Child and Adolescent Mental Health, Eastern and Southern Norway]
------------------------------

Original Message:
Sent: 12-18-2019 11:00
From: Stanley Lazic
Subject: Hypothesis Testing for Entire "Population"

Hi Melissa,

David Spiegelhalter talks about this in his recent book The Art of Statistics. The short answer is: Yes, this is fine, assuming the point is inferential and not descriptive (as Victoria Prince discussed).

The reason is that one can make inferences about a process or mechanism, which can be done even if you have the whole "population". As a side note, we (statisticians) need to stop presenting statistics as inference from samples to populations to users that don't actually do any sampling, i.e. for all experimental researchers. Teaching randomisation-based inference is more appropriate and connects better with their day-to-day work. I do like the concept of making an inference about a data generating process or mechanism, as it subsumes both perspectives.

------------------------------
Stanley E. Lazic, PhD
https://stanlazic.github.io/

Original Message:
Sent: 12-17-2019 16:00
From: Melissa N
Subject: Hypothesis Testing for Entire "Population"

Hello Everyone,

Example: I have data for all of the arrests in two different boroughs of NYC. I am being asked to use a proportion test to compare the proportion (number of female arrests)/(number of all arrests) between the two boroughs to see if there is a significant difference in the proportion of female arrests. However, I have ALL of the arrest data for the two boroughs for a specific time period. Does it make sense to do a hypothesis test/proportion test to assert if there is a significant difference even though we already have the true population proportions?

My opinion is no, however a colleague disagrees


Thanks

Hi Melissa,
Yes, it makes sense to do the test.  For the specific, observed time period from which data were obtained, either the observed proportions (out of all female arrests for that period) are the same or they are different for the two boroughs, to infinitely many decimal places. About that there can be no doubt, or relevance.  Since the observed arrest counts will vary across cohort time periods, the interest should be on whether the two proportions from the given observed time period are enough different to be deemed "significant."  This takes the view that the time period that was observed constitutes one random period taken from the population of all cohort time periods, and the test is an attempt to answer whether the two observed proportions differ enough to be worthy of notice.
Robert "Mick" Norton

------------------------------
Robert Norton
------------------------------

Original Message:
Sent: 12-17-2019 16:00
From: Melissa N
Subject: Hypothesis Testing for Entire "Population"

Hello Everyone,

Example: I have data for all of the arrests in two different boroughs of NYC. I am being asked to use a proportion test to compare the proportion (number of female arrests)/(number of all arrests) between the two boroughs to see if there is a significant difference in the proportion of female arrests. However, I have ALL of the arrest data for the two boroughs for a specific time period. Does it make sense to do a hypothesis test/proportion test to assert if there is a significant difference even though we already have the true population proportions?

My opinion is no, however a colleague disagrees


Thanks

Hey everyone,

I appreciate all of the responses. Some people have said that it is okay to do this type of test if we want to say something about future arrests or long term arrests. I agree, but this is not the goal.

Some more background regarding the approach:

The null hypothesis is that the two proportions are equal and the alternative is that they are not equal. So he is trying to say something about our actual proportions at that moment. Using the same proportions that we are hypothesizing about, we get a test statistic and compare it to a normal distribution with the idea that our data is one sample/realization of one world and in another universe/hypothetical situation our data can be different. Using the clt, our hypothetical data will be normally distributed and so this is possible.

It just seems like a reach just to add a p-value to a difference that we can already calculate because we have the actual data.

------------------------------
Melissa N
------------------------------

Original Message:
Sent: 12-17-2019 16:00
From: Melissa N
Subject: Hypothesis Testing for Entire "Population"

Hello Everyone,

Example: I have data for all of the arrests in two different boroughs of NYC. I am being asked to use a proportion test to compare the proportion (number of female arrests)/(number of all arrests) between the two boroughs to see if there is a significant difference in the proportion of female arrests. However, I have ALL of the arrest data for the two boroughs for a specific time period. Does it make sense to do a hypothesis test/proportion test to assert if there is a significant difference even though we already have the true population proportions?

My opinion is no, however a colleague disagrees


Thanks

In light of the proposed hypotheses, I would first ask if this is a reasonable hypothesis.  Is there any reason to suppose that the boroughs are sufficiently and the law enforcement sufficiently similar in both boroughs that such a hypothesis is reasonable?  If not, it is a waste of time, and possibly worse, if people try to draw "conclusions" from such an exercise.  I can think of analyses that could be useful, however, depending on what the purpose is.

------------------------------
Raoul Burchette
Biostatistician III
Kaiser Permanente, Pasadena, CA USA
------------------------------

Original Message:
Sent: 12-18-2019 11:38
From: Melissa N
Subject: Hypothesis Testing for Entire "Population"

Hey everyone,

I appreciate all of the responses. Some people have said that it is okay to do this type of test if we want to say something about future arrests or long term arrests. I agree, but this is not the goal.

Some more background regarding the approach:

The null hypothesis is that the two proportions are equal and the alternative is that they are not equal. So he is trying to say something about our actual proportions at that moment. Using the same proportions that we are hypothesizing about, we get a test statistic and compare it to a normal distribution with the idea that our data is one sample/realization of one world and in another universe/hypothetical situation our data can be different. Using the clt, our hypothetical data will be normally distributed and so this is possible.

It just seems like a reach just to add a p-value to a difference that we can already calculate because we have the actual data.

------------------------------
Melissa N

Original Message:
Sent: 12-17-2019 16:00
From: Melissa N
Subject: Hypothesis Testing for Entire "Population"

Hello Everyone,

Example: I have data for all of the arrests in two different boroughs of NYC. I am being asked to use a proportion test to compare the proportion (number of female arrests)/(number of all arrests) between the two boroughs to see if there is a significant difference in the proportion of female arrests. However, I have ALL of the arrest data for the two boroughs for a specific time period. Does it make sense to do a hypothesis test/proportion test to assert if there is a significant difference even though we already have the true population proportions?

My opinion is no, however a colleague disagrees


Thanks

Original Message------

In light of the proposed hypotheses, I would first ask if this is a reasonable hypothesis.  Is there any reason to suppose that the boroughs are sufficiently and the law enforcement sufficiently similar in both boroughs that such a hypothesis is reasonable?  If not, it is a waste of time, and possibly worse, if people try to draw "conclusions" from such an exercise.  I can think of analyses that could be useful, however, depending on what the purpose is.

------------------------------
Raoul Burchette
Biostatistician III
Kaiser Permanente, Pasadena, CA USA
------------------------------

I think that Michael Mout has made the most relevant comment in asking what an important (rather than statistical) difference is.

The pervasive (and perverse) notion that one can make decisions solely on P-values (i.e., no subject matter knowledge needed!) is just nuts.

What size of difference that is to be considered important is not easy to determine nor might there be much of any consensus nor might any consensus be constant over time.  The point is that "importance" is not an intrinsic property of the data.

------------------------------
Jim Baldwin
Retired
------------------------------

Original Message:
Sent: 12-18-2019 14:32
From: Michael Mout
Subject: Hypothesis Testing for Entire "Population"

Again, in my opinion, what is more important is not the statistical significance, but the size of the difference in real world terms.

For example a relatively small difference, like 1 or 2% may be significant at something like the .001 level, depending on sample size; but, in this case it doesn't seem like sample size makes sense to me since were talking about the population.

OTOH, a difference of say 10 or 20% may indicate huge bias, especially if you take into account other populations of similar make up.

M. Lynn Mout, MS, Cstat, Csci
4957 Gray Goose Ln, Ladson, SC 29456
804-314-5147(Mbl), 843-871-3039 (Home)

Original Message------

In light of the proposed hypotheses, I would first ask if this is a reasonable hypothesis.  Is there any reason to suppose that the boroughs are sufficiently and the law enforcement sufficiently similar in both boroughs that such a hypothesis is reasonable?  If not, it is a waste of time, and possibly worse, if people try to draw "conclusions" from such an exercise.  I can think of analyses that could be useful, however, depending on what the purpose is.

------------------------------
Raoul Burchette
Biostatistician III
Kaiser Permanente, Pasadena, CA USA
------------------------------

As W. Edwards Deming explained many years ago, most of the time a statistician is not interested in the enumerators task of counting exactly what happened at a particular time in the past, but in the analytic task of using past data as a basis for predicting the future.

But can you use past data as a basis for predicting the future? You can when the process from which data is coming from is approximately stable, so that e.g.  deviations from average or a systematic model over time are approximately random.

This is commonly simply assumed. Such assumptions have led to disaster in the past. The mortgage crisis of a decade ago is perhaps a salient example of the tendency to project data taken over a short period of time far into the future. In general, the naive use of statistics, ignoring the spadework necessary to establish the reasonableness of predictability assumptions, has tended to overestimate the reliability of hypothesis tests of this kind. 

Perhaps, if it turns out the two years of data are stable over time, or perhaps changing in a systematic way (e.g. monthly results fit a regression line with approximately random residuals), you could conduct a hypothesis test that could serve as a prediction valid for perhaps the next few months into the future. By I doubt it would have much validity beyond that time. 

------------------------------
Jonathan Siegel
Deputy Director Clinical Statistics
Bayer HealthCare Pharmaceuticals Inc.
------------------------------

Original Message:
Sent: 12-17-2019 16:00
From: Melissa N
Subject: Hypothesis Testing for Entire "Population"

Hello Everyone,

Example: I have data for all of the arrests in two different boroughs of NYC. I am being asked to use a proportion test to compare the proportion (number of female arrests)/(number of all arrests) between the two boroughs to see if there is a significant difference in the proportion of female arrests. However, I have ALL of the arrest data for the two boroughs for a specific time period. Does it make sense to do a hypothesis test/proportion test to assert if there is a significant difference even though we already have the true population proportions?

My opinion is no, however a colleague disagrees


Thanks

I think it fascinating that this elementary statistical inference (difference in two sample proportions) has such a high level of "researcher degrees of freedom".

My best answer would be that it depends. What does it depend on? Well, the intention of the person asking you to do that analysis. Any analysis is "wrong" if the two of you don't share an understanding of the problem at hand. The only way to find that out is through a conversation. If this is a "sample" to project latter time points, perhaps you'll want to consider trend or serial correlation over time. Does the client want to adjust for differences in the number of women who may be living in each borough? Or demographic differences? Or type of crime? Is he or she trying to determine crime commission from arrest records (this is a type of sampling, but certainly not a probability sample). Or perhaps trying to predict the likelihood of a women being arrested in each borough (or perhaps of a women who has committed a crime).

In fact, I recently encountered a very similar question. In that situation, the two groups were drastically different. I discussed with the client at length that raw numbers were more clear that statistical jargon--or even percentages. We also did the tests but discussed only reporting them in footnotes. So I believe it could be reasonable to try to account for variability in the reported numbers. Though reporting effect sizes, together with some exploration of why there is a difference, would probably be more interesting.

------------------------------
Daniel Coven
Graduate Statistics Consultant
Arizona State University
Daniel.Coven@asu.edu
------------------------------

Original Message:
Sent: 12-17-2019 16:00
From: Melissa N
Subject: Hypothesis Testing for Entire "Population"

Hello Everyone,

Example: I have data for all of the arrests in two different boroughs of NYC. I am being asked to use a proportion test to compare the proportion (number of female arrests)/(number of all arrests) between the two boroughs to see if there is a significant difference in the proportion of female arrests. However, I have ALL of the arrest data for the two boroughs for a specific time period. Does it make sense to do a hypothesis test/proportion test to assert if there is a significant difference even though we already have the true population proportions?

My opinion is no, however a colleague disagrees


Thanks

I tend to view this in the context of a permutation test: membership of a female in a borough is a random process under the null. Are the observed data consistent with the null? Under the null the odds of being arrested in the boroughs should be equal (see subsequent comment on over-simplification). Fisher's exact test, which can be shown to be a permutation test, can therefore be used analyze whether the between-borough ratio of the odds (odds ratio) is close to 1.

You get a permutation P value form Fisher's exact test, which is a measure of how far the data or more extreme data are from the null. The permutation P value is quite different from a "regular P value" because it is conditional on the margins (no sampling). You can also get an exact "confidence interval" on the odds ratio, from which, with a little algebra, you can get an exact "confidence interval" on the probability difference (more accessible by free-range humans).

This type of analysis is of course way over-simplified because the interpretation is conditional on assuming other factors are the same across boroughs, for example, that the factors leading to arrest are homogeneous across boroughs.

------------------------------
Brent Blumenstein
------------------------------

Original Message:
Sent: 12-17-2019 16:00
From: Melissa N
Subject: Hypothesis Testing for Entire "Population"

Hello Everyone,

Example: I have data for all of the arrests in two different boroughs of NYC. I am being asked to use a proportion test to compare the proportion (number of female arrests)/(number of all arrests) between the two boroughs to see if there is a significant difference in the proportion of female arrests. However, I have ALL of the arrest data for the two boroughs for a specific time period. Does it make sense to do a hypothesis test/proportion test to assert if there is a significant difference even though we already have the true population proportions?

My opinion is no, however a colleague disagrees


Thanks

I would tend to think of the true, "population" proportions in this case as unobservable parameters that characterize the 2 boroughs. In any given time period, we can think of the observed counts of females arrested, roughly, as realizations of a binomial process (of course in reality the denominator [# of total arrests] isn't fixed, so there may be better approaches). But it's similar to flipping a coin to test whether it's fair: there's no real population to sample from, just an unobservable probability of heads that we'd like to draw inference about from a sample of coin tosses. 

Vince   

------------------------------
Vincent Staggs, PhD
Research Faculty, Biostatistics & Epidemiology Core, Children's Mercy Hospitals & Clinics;
Associate Professor, School of Medicine, University of Missouri-Kansas City
------------------------------

Original Message:
Sent: 12-17-2019 16:00
From: Melissa N
Subject: Hypothesis Testing for Entire "Population"

Hello Everyone,

Example: I have data for all of the arrests in two different boroughs of NYC. I am being asked to use a proportion test to compare the proportion (number of female arrests)/(number of all arrests) between the two boroughs to see if there is a significant difference in the proportion of female arrests. However, I have ALL of the arrest data for the two boroughs for a specific time period. Does it make sense to do a hypothesis test/proportion test to assert if there is a significant difference even though we already have the true population proportions?

My opinion is no, however a colleague disagrees


Thanks

I agree that one wants to perform tests of statistical significance in order to know whether one is observing a trend (or that we are not merely observing random variation).  I also agree that it is crucial to quantify and appraise the difference between the situations, which first means quantifying the male-female difference in each situation.  It is not possible to do that based solely on the proportion women make up of arrested persons.  That information allows one to determine the ratio of the male arrest rate to the female arrest rate assuming that women are half the population.  That is, for example, where women are 10% of arrests the male-female ratio would be 9.0, and where women are 20% of arrests the ratio would be 4.0.  But the rate ratio is not a sound measure of association because it is affected by the prevalence of an outcome.  We need the actual male and female rates in order to quantify the male-female difference in each setting.  This is the point of Section I.C of my November 2016 comments for the Commission on Evidence-Based Policymaking. https://www.regulations.gov/document?D=USBC-2016-0003-0135

But there is a further complication with arrests.  Table 1 of the above comments shows that a situation where adverse outcome rates of two groups are 20% and 37% is essentially the same as a situation where the rates are 5% and 13%.  Probit d' value of approximately .5 in each situation.  But if all arrests are among persons, say, aged 20 to 50, and that population is half the population for both men and women in each situation, the figure become 40% and 74% (probit d' = .9) in the first situation and 10% and 26% (probit d' = .64) in the second situation.  

Further, assume we observe the table 1 values over a certain period but double those values over the double that period.  Same issue.  Yet all time periods are arbitrary.  I discuss this issue in the Addendum to the Ferguson Arrest Disparities page of Ferguson Arrest Disparities subpage of the Discipline Disparities page of jpscanlan.com.  I don't know the solution.

Incidentally, in the comments (at 27) and things I posted here in 2015 I discuss that policies aimed at addressing racial disparities in adverse criminal justice outcomes in Ferguson, MO would likely increase the measures of racial disparity on which the Department of Justice had relied in its actions against Ferguson.  Below is a discussion of the way those measures have in fact increased in recent years. 

https://fedsoc.org/commentary/blog-posts/usual-but-wholly-misunderstood-effects-of-policies-on-measures-of-racial-disparity-now-being-seen-in-ferguson-and-the-uk-and-soon-to-be-seen-in-baltimore

------------------------------
James Scanlan
James P. Scanlan Attorney At Law
------------------------------

Original Message:
Sent: 12-20-2019 11:49
From: Vincent Staggs
Subject: Hypothesis Testing for Entire "Population"

I would tend to think of the true, "population" proportions in this case as unobservable parameters that characterize the 2 boroughs. In any given time period, we can think of the observed counts of females arrested, roughly, as realizations of a binomial process (of course in reality the denominator [# of total arrests] isn't fixed, so there may be better approaches). But it's similar to flipping a coin to test whether it's fair: there's no real population to sample from, just an unobservable probability of heads that we'd like to draw inference about from a sample of coin tosses. 

Vince   

------------------------------
Vincent Staggs, PhD
Research Faculty, Biostatistics & Epidemiology Core, Children's Mercy Hospitals & Clinics;
Associate Professor, School of Medicine, University of Missouri-Kansas City

Original Message:
Sent: 12-17-2019 16:00
From: Melissa N
Subject: Hypothesis Testing for Entire "Population"

Hello Everyone,

Example: I have data for all of the arrests in two different boroughs of NYC. I am being asked to use a proportion test to compare the proportion (number of female arrests)/(number of all arrests) between the two boroughs to see if there is a significant difference in the proportion of female arrests. However, I have ALL of the arrest data for the two boroughs for a specific time period. Does it make sense to do a hypothesis test/proportion test to assert if there is a significant difference even though we already have the true population proportions?

My opinion is no, however a colleague disagrees


Thanks

in my opinion, here we find an item of the common confusion between a tolerance interval and a confidence interval. A fraction of the Universe or Population is a fixed number without any standard error. In fact, for populations, we talk about standard deviations and never talk we about standard errors. Of course, errors could anyway exist as "measure errors". In addition, if tolerance intervals are extrapolated by a sample they too will have their standard errors. 
Therefore, I would be very careful in adding the info Confidence Interval to a fraction of the Population distribution as if a different sample could exist.  The frequentist interpretation of a Confidence Interval corresponds to the hypothesis of repeated samples in the same population. Each sample would show a different statistic whose distribution would be its sampling distribution. However, if no repetitive sample is possibile, no different statistic would show up and so speculating about p values would be meaningless.
I witnessed a extreme case of this confusion when somebody showed me a panel of Banks, actually all banks of the Region he was invetigating, under a Random Effects model. Of course this case should have been done under a Fixed Effects model. Each bank is a dummy variable, a column in the Design Matrix, without any need of speculating how the other banks would behave. The other Banks do not exist if you take them all in your analysis.

The fact that packages always show p alues or similar is due to an "economic" way of doing. They print everything. What applies is up to you to decide. 

Ulderico Santarelli

------------------------------
[Ulderico] [Santarelli]
[Las Vegas][Nevada]
------------------------------

Original Message:
Sent: 12-17-2019 16:00
From: Melissa N
Subject: Hypothesis Testing for Entire "Population"

Hello Everyone,

Example: I have data for all of the arrests in two different boroughs of NYC. I am being asked to use a proportion test to compare the proportion (number of female arrests)/(number of all arrests) between the two boroughs to see if there is a significant difference in the proportion of female arrests. However, I have ALL of the arrest data for the two boroughs for a specific time period. Does it make sense to do a hypothesis test/proportion test to assert if there is a significant difference even though we already have the true population proportions?

My opinion is no, however a colleague disagrees


Thanks

This is correct if the purpose is to generalize from a sample to a well-defined population. Then, if the sample is the whole population there is no need for modeling (not for a fixed effects model either). However, as stated by several contributors to this thread, including Stanley Lazic and Vincent Staggs, not all statistical inference is about generalizing from a sample to a population. Often, and increasingly often I think, the purpose of statistical inference is to make statements on the process that created the data. Sampling from a population may or may not be part of this process. In case, sampling from a population is often only a tiny part of the randomness involved. This is not primarily a question of whether the purpose involves prediction of the future or not. In the case of flipping of a coin by Vincent Staggs, there is no population, there is just a coin with unknown head probability. If the purpose is to learn about this probability, this does not necessarily involve prediction. Concerning the original case of Melissa Nunez, the aim may be to compare the two boroughs  as they are. Then there is no need for statistical inference if error-free data on the entire boroughs are available. If, however, the purpose is to compare the processes that made the two boroughs  different (or not), there is nothing wrong with using a model-based approach.

------------------------------
Tore Wentzel-Larsen
Researcher
Norwegian Centre for Violence and Traumatic Stress Studies,
Regional Center for Child and Adolescent Mental Health, Eastern and Southern Norway]
------------------------------

Original Message:
Sent: 12-20-2019 13:44
From: Ulderico Santarelli
Subject: Hypothesis Testing for Entire "Population"

in my opinion, here we find an item of the common confusion between a tolerance interval and a confidence interval. A fraction of the Universe or Population is a fixed number without any standard error. In fact, for populations, we talk about standard deviations and never talk we about standard errors. Of course, errors could anyway exist as "measure errors". In addition, if tolerance intervals are extrapolated by a sample they too will have their standard errors.
Therefore, I would be very careful in adding the info Confidence Interval to a fraction of the Population distribution as if a different sample could exist.  The frequentist interpretation of a Confidence Interval corresponds to the hypothesis of repeated samples in the same population. Each sample would show a different statistic whose distribution would be its sampling distribution. However, if no repetitive sample is possibile, no different statistic would show up and so speculating about p values would be meaningless.
I witnessed a extreme case of this confusion when somebody showed me a panel of Banks, actually all banks of the Region he was invetigating, under a Random Effects model. Of course this case should have been done under a Fixed Effects model. Each bank is a dummy variable, a column in the Design Matrix, without any need of speculating how the other banks would behave. The other Banks do not exist if you take them all in your analysis.

The fact that packages always show p alues or similar is due to an "economic" way of doing. They print everything. What applies is up to you to decide.

Ulderico Santarelli

------------------------------
[Ulderico] [Santarelli]
[Las Vegas][Nevada]

Original Message:
Sent: 12-17-2019 16:00
From: Melissa N
Subject: Hypothesis Testing for Entire "Population"

Hello Everyone,

Example: I have data for all of the arrests in two different boroughs of NYC. I am being asked to use a proportion test to compare the proportion (number of female arrests)/(number of all arrests) between the two boroughs to see if there is a significant difference in the proportion of female arrests. However, I have ALL of the arrest data for the two boroughs for a specific time period. Does it make sense to do a hypothesis test/proportion test to assert if there is a significant difference even though we already have the true population proportions?

My opinion is no, however a colleague disagrees


Thanks

Hi Melissa,

That's a great question and I think the answer, in part, depends on the purpose of your study. For example, if you plan to use your results in the future, then to have a real population from which you could draw relevant inferences, you would have to data on all possible scenarios relating to female arrests (which you don't have). Sometimes, I tell my students to think of such situations as a sample taken as a snapshot in time rather than a population.

Matt

------------------------------
Matthew Brenneman
Instructor of Mathematics & Statistics
Embry-Riddle Aeronautical University
------------------------------

Original Message:
Sent: 12-17-2019 16:00
From: Melissa N
Subject: Hypothesis Testing for Entire "Population"

Hello Everyone,

Example: I have data for all of the arrests in two different boroughs of NYC. I am being asked to use a proportion test to compare the proportion (number of female arrests)/(number of all arrests) between the two boroughs to see if there is a significant difference in the proportion of female arrests. However, I have ALL of the arrest data for the two boroughs for a specific time period. Does it make sense to do a hypothesis test/proportion test to assert if there is a significant difference even though we already have the true population proportions?

My opinion is no, however a colleague disagrees


Thanks

Let's not get caught up in the "magic" of hypothesis testing. Often there is no need for a test at all. The questions here seem to be: a) is any difference "real" or might it be due to chance? and b) is the observed difference big enough to be actionable?
If the answer to (b) is "no" then stop. If there appears to be a big enough difference to make a difference, then we do need to ask whether it might be due to chance. In this particular situation, viewing the data as a snapshot in time is both statistically sensible and operationally sensible--after all, what is probably wanted, if the difference is large, is a warning that something should be done so the difference won't be that large in the future. A confidence interval is a relatively intuitive way to compare the observed difference to an estimate of the likely magnitude of the underlying variation. With population values, the concept of "underlying variation" is harder to get at, but with proportions, the usual estimates are a reasonable thing to do.

------------------------------
Paul Velleman
Professor Emeritus
------------------------------

Original Message:
Sent: 12-23-2019 14:45
From: Matthew Brenneman
Subject: Hypothesis Testing for Entire "Population"

Hi Melissa,

That's a great question and I think the answer, in part, depends on the purpose of your study. For example, if you plan to use your results in the future, then to have a real population from which you could draw relevant inferences, you would have to data on all possible scenarios relating to female arrests (which you don't have). Sometimes, I tell my students to think of such situations as a sample taken as a snapshot in time rather than a population.

Matt

------------------------------
Matthew Brenneman
Instructor of Mathematics & Statistics
Embry-Riddle Aeronautical University

Original Message:
Sent: 12-17-2019 16:00
From: Melissa N
Subject: Hypothesis Testing for Entire "Population"

Hello Everyone,

Example: I have data for all of the arrests in two different boroughs of NYC. I am being asked to use a proportion test to compare the proportion (number of female arrests)/(number of all arrests) between the two boroughs to see if there is a significant difference in the proportion of female arrests. However, I have ALL of the arrest data for the two boroughs for a specific time period. Does it make sense to do a hypothesis test/proportion test to assert if there is a significant difference even though we already have the true population proportions?

My opinion is no, however a colleague disagrees


Thanks

It depends on whether you are trying to use your data to estimate a parameter. If yes, then a hypothesis test makes sense.  If not, then a hypothesis test does not make sense.

------------------------------
John Czarnek
------------------------------

Original Message:
Sent: 12-17-2019 16:00
From: Melissa N
Subject: Hypothesis Testing for Entire "Population"

Hello Everyone,

Example: I have data for all of the arrests in two different boroughs of NYC. I am being asked to use a proportion test to compare the proportion (number of female arrests)/(number of all arrests) between the two boroughs to see if there is a significant difference in the proportion of female arrests. However, I have ALL of the arrest data for the two boroughs for a specific time period. Does it make sense to do a hypothesis test/proportion test to assert if there is a significant difference even though we already have the true population proportions?

My opinion is no, however a colleague disagrees


Thanks

I believe the first step is to get clear on whether the objective of the research is to estimate parameters (e.g., sampling a human population) or test hypotheses (e.g., sampling a statistical population, such as from a sampling distribution of proportions). If the former, one needs to understand survey sampling techniques and appropriate weighted analysis, and for the latter the potential confounding effects of lurking variables on experimental designs. 


------------------------------
Eugene Komaroff
Professor of Education
Keiser University Graduate School
------------------------------

Original Message:
Sent: 12-26-2019 11:59
From: John Czarnek
Subject: Hypothesis Testing for Entire "Population"

It depends on whether you are trying to use your data to estimate a parameter. If yes, then a hypothesis test makes sense.  If not, then a hypothesis test does not make sense.

------------------------------
John Czarnek

Original Message:
Sent: 12-17-2019 16:00
From: Melissa N
Subject: Hypothesis Testing for Entire "Population"

Hello Everyone,

Example: I have data for all of the arrests in two different boroughs of NYC. I am being asked to use a proportion test to compare the proportion (number of female arrests)/(number of all arrests) between the two boroughs to see if there is a significant difference in the proportion of female arrests. However, I have ALL of the arrest data for the two boroughs for a specific time period. Does it make sense to do a hypothesis test/proportion test to assert if there is a significant difference even though we already have the true population proportions?

My opinion is no, however a colleague disagrees


Thanks