Tasneem, I cannot agree more with Steven's insightful inputs. Given that most health-related instruments/scales/surveys/constructs/questionnaires were NOT 'professionally developed' and 'few of them have received much critical evaluation' (Teresi & Fleishman 2007), it is essential to test the basic psychometric properties (reliability, validity, etc) of these instruments/questionnaires, even though for those claimed as 'standard' ones, which in fact never exist for your own sample before testing ("Validity", Zumbo 2007).
Back to your data, for a questionnaire with 5-10 questions, with each has a 1-5 Likert-type response, it is crucial to test at least the following 2 things, for each of the two time points you have (Pre-, and Post-):
1. Can the individual scores from each of these 5-10 questionnaires be summarized into a single 'Total Score'? The big assumption behind this single 'Total Score' is the 'unidimensionality', i.e., each and all of the 10 questions is measuring the single same thing('Construct', e.g., Happiness, Stress, Satisfaction, etc). To test this assumption, you need to run a Confirmatory Factor Analysis (CFA). And when you run this CFA, please keep in mind that your data are actually ordinal, not continuous. (Moreover, although rarely addressed, Longitudinal Measurement Invariance (Brown 2006) is actually another assumption needed for meaningful Pre-/Post- comparison of data like yours).
2. How 'reliable' the questionnaire is, especially, how 'internally consistent' the 10 questions are? This can be assessed by the Cronbach's Alpha (Cronbach 1951). And if possible, you might also want to test reliability using other methods, such as Test-Retset, Alternative-Form, Split-Haves, and you might want to read Cmrmines & Zeller's classic booklet (SAGE, 1979) on this topic.
After passed these two basic tests, the ordinary biostatistical techniques come in, and it's totally up to you to make a choice from many different options.
Sincerely yours,
Chengwu Yang （杨成武）
______________________
Chengwu Yang, MD, MS, PhD
Assistant Professor of Biostatistics
Department of Public Health Sciences
College of Medicine, The Pennsylvania State University
A210, ASB 3400H, 600 Centerview Drive, Hershey, PA 17033
Email:
yangc@psu.edu; Phone: 717-531-3016; Fax: 717-531-0146
http://profiles.psu.edu/profiles/ProfileDetails.aspx?From=SE&Person=244 -------------------------------------------
Original Message:
Sent: 06-19-2012 08:35
From: Steven Pierce
Subject: Comparing before and after scores based on questionairre
Before you start doing longitudinal analyses, you should first make sure that you can extract a meaningful measurement from each questionnaire. Taking the sum of the items might (or might not) produce a coherent, unidimensional measurement, so that should be explicitly tested. Establish that the measure has reliability and validity at the pre-test, then confirm that the same measurement structure applies at the post-test before you worry about longitudinal comparisons. Such comparisons are only sensible if a given score has the same meaning at each time point. You can use confirmatory factor analysis and structural equation modeling to investigate these issues.
Assuming the scores end up properly being continuous, then a simple paired t-test could answer your primary question, as could a variety of other statistical methods.
-------------------------------------------
Steven Pierce
Associate Director
Center for Statistical Training and Consulting
Michigan State University
-------------------------------------------