ASA Connect

Dear ASA Colleagues:

Have you ever computed power to compare two proportions or for logistic regression, when the data came from a complex survey design with weighting, stratification, and clustering?

If so, I'd appreciate any references to books, articles, or syntax with SAS, Stata, R, Python, or another package.

------------------------------
Brandy Sinco, BS, MA, MS
Statistician Senior
Michigan Medicine
------------------------------

Hi Brandy,

The sample size annexes in the 2018 World Health Organization's Vaccination Coverage Cluster Survey Reference Manual has a section on power and sample size when the inferential goal is to compare proportions in a) two strata, or b) pre- and post- surveys.  The equations assume that the test will be a survey-adjusted Rao-Scott chi-square test, which is the same as a survey-adjusted logistic regression if the only independent variable is categorical for the two groups you are comparing. If you intend to adjust for other covariates in your regression, that should only increase your power and the WHO numbers should turn out to be conservative.  (A hand-wavy argument, I know...)  The relevant sections are Annex B1 and B3.  The manual is available here:  https://www.who.int/immunization/documents/who_ivb_18.09/en/

There is also an Excel spreadsheet that helps you do the sample size calculations in the manual without needing to work the equations.  You'll want to look at the worksheet named 'Comparative'.
https://www.who.int/immunization/monitoring_surveillance/routine/coverage/sample_size_calculator_survey.xlsx?ua=1

The other approach is to do a simulation which is straightforward if you have a handle on the likely design effects or effective sample sizes.  I'd be happy to discuss any of these ideas with you if that would help.  My e-mail address is: dale.rhoda@biostatglobal.com.

-Dale

------------------------------
Dale Rhoda
President
Biostat Global Consulting LLC
------------------------------

Original Message:
Sent: 11-11-2020 14:23
From: Brandy Sinco
Subject: Power for Complex Survey Design - Compare Proportions, Logistic Regression

Dear ASA Colleagues:

Have you ever computed power to compare two proportions or for logistic regression, when the data came from a complex survey design with weighting, stratification, and clustering?

If so, I'd appreciate any references to books, articles, or syntax with SAS, Stata, R, Python, or another package.

------------------------------
Brandy Sinco, BS, MA, MS
Statistician Senior
Michigan Medicine
------------------------------

Thanks Dale.  That is an excellent starting point.

------------------------------
Brandy Sinco, BS, MA, MS
Statistician Senior
Michigan Medicine
------------------------------

Original Message:
Sent: 11-12-2020 08:49
From: Dale Rhoda
Subject: Power for Complex Survey Design - Compare Proportions, Logistic Regression

Hi Brandy,

The sample size annexes in the 2018 World Health Organization's Vaccination Coverage Cluster Survey Reference Manual has a section on power and sample size when the inferential goal is to compare proportions in a) two strata, or b) pre- and post- surveys.  The equations assume that the test will be a survey-adjusted Rao-Scott chi-square test, which is the same as a survey-adjusted logistic regression if the only independent variable is categorical for the two groups you are comparing. If you intend to adjust for other covariates in your regression, that should only increase your power and the WHO numbers should turn out to be conservative.  (A hand-wavy argument, I know...)  The relevant sections are Annex B1 and B3.  The manual is available here:  https://www.who.int/immunization/documents/who_ivb_18.09/en/

There is also an Excel spreadsheet that helps you do the sample size calculations in the manual without needing to work the equations.  You'll want to look at the worksheet named 'Comparative'.
https://www.who.int/immunization/monitoring_surveillance/routine/coverage/sample_size_calculator_survey.xlsx?ua=1

The other approach is to do a simulation which is straightforward if you have a handle on the likely design effects or effective sample sizes.  I'd be happy to discuss any of these ideas with you if that would help.  My e-mail address is: dale.rhoda@biostatglobal.com.

-Dale

------------------------------
Dale Rhoda
President
Biostat Global Consulting LLC

Original Message:
Sent: 11-11-2020 14:23
From: Brandy Sinco
Subject: Power for Complex Survey Design - Compare Proportions, Logistic Regression

Dear ASA Colleagues:

Have you ever computed power to compare two proportions or for logistic regression, when the data came from a complex survey design with weighting, stratification, and clustering?

If so, I'd appreciate any references to books, articles, or syntax with SAS, Stata, R, Python, or another package.

------------------------------
Brandy Sinco, BS, MA, MS
Statistician Senior
Michigan Medicine
------------------------------

oops I missed the "survey" part...sorry

------------------------------
Alberto Nettel-Aguirre
Associate Professor
University of Calgary
------------------------------

Original Message:
Sent: 11-11-2020 14:23
From: Brandy Sinco
Subject: Power for Complex Survey Design - Compare Proportions, Logistic Regression

Dear ASA Colleagues:

Have you ever computed power to compare two proportions or for logistic regression, when the data came from a complex survey design with weighting, stratification, and clustering?

If so, I'd appreciate any references to books, articles, or syntax with SAS, Stata, R, Python, or another package.

------------------------------
Brandy Sinco, BS, MA, MS
Statistician Senior
Michigan Medicine
------------------------------