Discussion: View Thread

Hello Consulting Section Members:

I am writing to get some opinions on collecting and summarizing focus group data.

I have 35 participants who are asked 45 open-ended questions, with most of the questions having three parts.  Focus groups sessions are recorded and then transcribed.  The transcriber types the conversations into Excel.  That means that each response to a questions is in one cell.  One response may be one word or 50 sentences, depending on the question and the respondent.  All questions/responses are typed into one worksheet; so I must scroll through the document to reach the 45 questions.

The problem is that I find it very hard to read focus group conversations in Excel. Also, some of the responses are very long (a page), and sometimes I read through all of it only to learn they never answered the question.

1.  What format is the ideal format for a transcriber to use when entering focus group responses into some electronic format?
2.  Are you familiar with software that is more appropriate for displaying focus group data or facilitates the review of focus group data?

Thank you,
MJ-

-------------------------------------------
Monica Johnston
Statistical Consultant & Instructor
Mostly Math
-------------------------------------------

I am reminded of the Henny Youngman punchline:  "Don't do that." (in response to "Doctor, it hurts when I do this.")

Unless this was a very exploratory and pilot survey, the amount of text generated in such a non-structured fashion would seem to result in an enormous amount of work even if it were all easily readable.  How was this expected to be analyzed?

Jim

-------------------------------------------
James Baldwin
Station Statistician
US Forest Service
-------------------------------------------

I agree with Jim. I would only use open-ended questions for a pilot study.
Open-ended questions provide information, but not useful data.
Usually with more than 20 respondents, it's more info than useful or necessary.

-------------------------------------------
David Bristol
Statistical Consulting Services
-------------------------------------------







Original Message:
Sent: 07-17-2013 13:48
From: James Baldwin
Subject: Entering and Summarizing Focus Group Data

I am reminded of the Henny Youngman punchline:  "Don't do that." (in response to "Doctor, it hurts when I do this.")

Unless this was a very exploratory and pilot survey, the amount of text generated in such a non-structured fashion would seem to result in an enormous amount of work even if it were all easily readable.  How was this expected to be analyzed?

Jim

-------------------------------------------
James Baldwin
Station Statistician
US Forest Service
-------------------------------------------

With this many respondents and given the number of responses each you should be using narrative software
I have used N*Vivo extensively, others I know use Ethnograph.  Both are specifically designed to deal with narrative responses.
I like N*Vivo as it takes MS Word documents directly.  The transcriptionist should be typing the responses verbatim not trying
to pre-code or squeeze them into cells in an Excel sheet.  Using MS word the transcriptionist should be able to identify respondents and sessions and sections, etc by following labeling guidance for the software.

Neither software is really cheap but both are worth the investment.

Sounds like you need someone to help you who knows qualitative analysis of complex narrative data.  Sounds like an interesting study.  Most focus groups I have run have a maximum of 10 people and 4 to 9 questions in a 90 minutes period.  How long did it take to go through 45 items?

Start with the web pages for N*Vivo and Ethnograph as they will have some great background and then do a quick search on Google Scholar for examples of how to apply these approaches.



-------------------------------------------
William Grant
Professor, Emergency Medicine
SUNY Upstate Medical University
-------------------------------------------







Original Message:
Sent: 07-17-2013 13:48
From: James Baldwin
Subject: Entering and Summarizing Focus Group Data

I am reminded of the Henny Youngman punchline:  "Don't do that." (in response to "Doctor, it hurts when I do this.")

Unless this was a very exploratory and pilot survey, the amount of text generated in such a non-structured fashion would seem to result in an enormous amount of work even if it were all easily readable.  How was this expected to be analyzed?

Jim

-------------------------------------------
James Baldwin
Station Statistician
US Forest Service
-------------------------------------------

Thank you all for your responses so far! I appreciate the links to papers and the suggestions for particular software. 

Jim, to answer your question.....These data are supposed to be summarized by identifying themes in responses to questions. (By the way, there are 35 respondents (about 9 people in each of 4 customer groups.)  For a particular question, I need to read the 9 responses from respondents within one customer group and identify or develop one comment that is representative of most/all 9 respondents in that customer group.

Thank you,
MJ-

-------------------------------------------
Monica Johnston
Statistical Consultant & Instructor
Mostly Math
-------------------------------------------







Original Message:
Sent: 07-17-2013 13:48
From: James Baldwin
Subject: Entering and Summarizing Focus Group Data

I am reminded of the Henny Youngman punchline:  "Don't do that." (in response to "Doctor, it hurts when I do this.")

Unless this was a very exploratory and pilot survey, the amount of text generated in such a non-structured fashion would seem to result in an enormous amount of work even if it were all easily readable.  How was this expected to be analyzed?

Jim

-------------------------------------------
James Baldwin
Station Statistician
US Forest Service
-------------------------------------------

After my struggles with qualitative data and how to fit it into a statistical analysis -- I now send the transcriptions back to the PI to code. My experience is that this is not a task for statisticians.  The coding and categorizing needs to be completed by the primary investigator of the project or her/his coding team.  The PI has an idea of what he/she had in mind when developing the questions.  There are particular skills and training that are needed to do this reliabily -- you will need a second coder on some of the interviews to make sure there is agreement on the codes/themes. 

Another piece of advise from my experience is that the PI and his/her staff should not break themes into too small of bits -- I have 1000 yes/no questions from pre and post interviews -- the small bits lost most of the thematic data that might have been available.  By the way of the 1000 bits only 6 were related to the outcomes of interest.  

Some of my PI use Altas software -- this software will generate a dataset that you can read directly into SPSS.  The more committed qualitative PIs - transcribe datasets, develop codebooks, translate the codebooks into ACCESS data forms, and do not put the actual text into the access tables -- There is a second database or word document that is created that keeps track of memorable quotes that illustrate points they want to make.

-------------------------------------------
Barbara Hanusa
Senior Statistician, CHERP
VA Pittsburgh Health System
-------------------------------------------







Original Message:
Sent: 07-17-2013 15:26
From: Monica Johnston
Subject: Entering and Summarizing Focus Group Data

Thank you all for your responses so far! I appreciate the links to papers and the suggestions for particular software. 

Jim, to answer your question.....These data are supposed to be summarized by identifying themes in responses to questions. (By the way, there are 35 respondents (about 9 people in each of 4 customer groups.)  For a particular question, I need to read the 9 responses from respondents within one customer group and identify or develop one comment that is representative of most/all 9 respondents in that customer group.

Thank you,
MJ-

-------------------------------------------
Monica Johnston
Statistical Consultant & Instructor
Mostly Math
-------------------------------------------

Sounds like a job for text analysis software...Atlas has that capability...SPSS has a separate program for that...then there's SAS...

Once you decide on what program you'll use for the text analysis, then you'll know what "format" is needed...

-------------------------------------------
Wayne Fischer
Statistician
University of Texas Medical Branch
-------------------------------------------







Original Message:
Sent: 07-17-2013 16:12
From: Barbara Hanusa
Subject: Entering and Summarizing Focus Group Data

After my struggles with qualitative data and how to fit it into a statistical analysis -- I now send the transcriptions back to the PI to code. My experience is that this is not a task for statisticians.  The coding and categorizing needs to be completed by the primary investigator of the project or her/his coding team.  The PI has an idea of what he/she had in mind when developing the questions.  There are particular skills and training that are needed to do this reliabily -- you will need a second coder on some of the interviews to make sure there is agreement on the codes/themes. 

Another piece of advise from my experience is that the PI and his/her staff should not break themes into too small of bits -- I have 1000 yes/no questions from pre and post interviews -- the small bits lost most of the thematic data that might have been available.  By the way of the 1000 bits only 6 were related to the outcomes of interest.  

Some of my PI use Altas software -- this software will generate a dataset that you can read directly into SPSS.  The more committed qualitative PIs - transcribe datasets, develop codebooks, translate the codebooks into ACCESS data forms, and do not put the actual text into the access tables -- There is a second database or word document that is created that keeps track of memorable quotes that illustrate points they want to make.

-------------------------------------------
Barbara Hanusa
Senior Statistician, CHERP
VA Pittsburgh Health System
-------------------------------------------







Original Message:
Sent: 07-17-2013 15:26
From: Monica Johnston
Subject: Entering and Summarizing Focus Group Data

Thank you all for your responses so far! I appreciate the links to papers and the suggestions for particular software. 

Jim, to answer your question.....These data are supposed to be summarized by identifying themes in responses to questions. (By the way, there are 35 respondents (about 9 people in each of 4 customer groups.)  For a particular question, I need to read the 9 responses from respondents within one customer group and identify or develop one comment that is representative of most/all 9 respondents in that customer group.

Thank you,
MJ-

-------------------------------------------
Monica Johnston
Statistical Consultant & Instructor
Mostly Math
-------------------------------------------

SAS, SPSS and there is some pen source software
http://www.umass.edu/qdap/

http://rqda.r-forge.r-project.org/


-------------------------------------------
Chris Barker, Ph.D.
Consultant and
Adjunct Associate Professor of Biostatistics
www,barkerstats.com

---
"In composition you have all the time you want to decide what to say in 15 seconds, in improvisation you have 15 seconds."
-Steve Lacy
-------------------------------------------







Original Message:
Sent: 07-17-2013 17:32
From: Wayne Fischer
Subject: Entering and Summarizing Focus Group Data

Sounds like a job for text analysis software...Atlas has that capability...SPSS has a separate program for that...then there's SAS...

Once you decide on what program you'll use for the text analysis, then you'll know what "format" is needed...

-------------------------------------------
Wayne Fischer
Statistician
University of Texas Medical Branch
-------------------------------------------







Original Message:
Sent: 07-17-2013 16:12
From: Barbara Hanusa
Subject: Entering and Summarizing Focus Group Data

After my struggles with qualitative data and how to fit it into a statistical analysis -- I now send the transcriptions back to the PI to code. My experience is that this is not a task for statisticians.  The coding and categorizing needs to be completed by the primary investigator of the project or her/his coding team.  The PI has an idea of what he/she had in mind when developing the questions.  There are particular skills and training that are needed to do this reliabily -- you will need a second coder on some of the interviews to make sure there is agreement on the codes/themes. 

Another piece of advise from my experience is that the PI and his/her staff should not break themes into too small of bits -- I have 1000 yes/no questions from pre and post interviews -- the small bits lost most of the thematic data that might have been available.  By the way of the 1000 bits only 6 were related to the outcomes of interest.  

Some of my PI use Altas software -- this software will generate a dataset that you can read directly into SPSS.  The more committed qualitative PIs - transcribe datasets, develop codebooks, translate the codebooks into ACCESS data forms, and do not put the actual text into the access tables -- There is a second database or word document that is created that keeps track of memorable quotes that illustrate points they want to make.

-------------------------------------------
Barbara Hanusa
Senior Statistician, CHERP
VA Pittsburgh Health System
-------------------------------------------

Statistics is not really about means, but about variation. Means are easy, but are meaningless (pun intended :-) without some measure of variation. I'm not sure how you defined your customer groups, or why you needed 9 customers in each, but I would guess it might be able to capture the variation among customers within a group and among groups. By developing a single summary comment for each question that is "representative" of most of the 9, you are throwing out the baby with the bathwater, i.e. you are losing what could be very important information about how variable your customer base is in each group with respect to each question. Perhaps there are some questions with very little variation among the answers, others with a lot. Perhaps for some questions there is little variation among responses in one group, a lot in others. The fact that some questions elicit a lot of variation in response, might in itself be very interesting... perhaps more interesting than the average or "representative" response. Of course, not knowing your study nor its objectives, this may not be relevant but I thought I'd throw it out there... 


-------------------------------------------
Manuela Huso
Research Statistician
US Geological Survey
-------------------------------------------







Original Message:
Sent: 07-17-2013 15:26
From: Monica Johnston
Subject: Entering and Summarizing Focus Group Data

Thank you all for your responses so far! I appreciate the links to papers and the suggestions for particular software. 

Jim, to answer your question.....These data are supposed to be summarized by identifying themes in responses to questions. (By the way, there are 35 respondents (about 9 people in each of 4 customer groups.)  For a particular question, I need to read the 9 responses from respondents within one customer group and identify or develop one comment that is representative of most/all 9 respondents in that customer group.

Thank you,
MJ-

-------------------------------------------
Monica Johnston
Statistical Consultant & Instructor
Mostly Math
-------------------------------------------

Monica,

I prefer using Word or another word processing program when working with qualitative/open ended survey data. The reason being is that when you use Excel, the data is placed in an artifical format that is more suitable for handling numbers than text. I like being able to page up/down and code without having to deal with cells. Programs such as Nvivo work directly with text files and provide nice coding tools. The R program has a text mining package call "TM" that works really well with qualitative data.

Thanks,

Dan

-------------------------------------------
Daniel Butorovich
Research Analyst
Cochise College
-------------------------------------------

Here is a paper I wrote a few years back about using Excel for double data entry -- with examples for both quantitative and qualitative type data. Most clients bring their data to me in Excel format because that's what they know. I use the paper to coach them into using double data entry (and the other tips in the paper) to create as clean a data set as possible.

Elliott AC, Hynan LS, Reisch JS, Smith JP. "Preparing Data for Analysis Using Microsoft Excel," J Investig Med, 54(6):334-41, 2006.

-------------------------------------------
Alan Elliott
Director, Statistical Consulting Center
Department of Statistical Science
Southern Methodist University
-------------------------------------------







Original Message:
Sent: 07-17-2013 13:53
From: Daniel Butorovich
Subject: Entering and Summarizing Focus Group Data

Monica,

I prefer using Word or another word processing program when working with qualitative/open ended survey data. The reason being is that when you use Excel, the data is placed in an artifical format that is more suitable for handling numbers than text. I like being able to page up/down and code without having to deal with cells. Programs such as Nvivo work directly with text files and provide nice coding tools. The R program has a text mining package call "TM" that works really well with qualitative data.

Thanks,

Dan

-------------------------------------------
Daniel Butorovich
Research Analyst
Cochise College
-------------------------------------------

Hi Monica, for focus groups, the questions and answers are understandably open-ended.
  
 You have a digital recording of the interview and that may be the best method for storing the interview.

This paper may have some links to software recommendations

http://www.academia.edu/746649/Methods_of_data_collection_in_qualitative_research_interviews_and_focus_groups

googling on 'data storage open ended questions" turns up some software, though -doesn't look inexpensive

http://www.qsrinternational.com/products_nvivo.aspx




-------------------------------------------
Chris Barker, Ph.D.
Consultant and
Adjunct Associate Professor of Biostatistics
www,barkerstats.com

---
"In composition you have all the time you want to decide what to say in 15 seconds, in improvisation you have 15 seconds."
-Steve Lacy
-------------------------------------------

Dear Monica Johnston,
You can use Atlas.ti  software.
Best regards

-------------------------------------------
[Dr MITONGA KABWEBWE Honoré, Bsc.Stat., MPH, PhD

UNIVERSITY OF NAMIBIA
FACULTY OF HEALTH SCIENCES
SCHOOL OF MEDICINE
DEPARTMENT OF COMMUNITY MEDICINE
Tel(W): +264612065013
Tel(H): +26461272323
FAX : +26461254130

Fax2Email: +264(0)88633115

Cell:+264856056256 /+264816291762
P.O. BOX 10661-KHOMASDAL
WINDHOEK
NAMIBIA


]
-------------------------------------------

It is at times odd how many different uses people find for Excel.

I too would use something else for this.  The only possible advantage that I can see for using Excel in this context is that you could fairly quickly compare the responses within a subject across questions, or within questions across respondents.  If I wanted to retain that advantage I would probably go to a database program and use a VARCHAR field.  

-------------------------------------------
David Mangen
Owner
Mangen Research Associates, Inc.
-------------------------------------------

My only comment is: what a marvelous group that people can get help so quickly, of such diversity, and on a topic that is not commonly treated in statistical education. I am very proud of self-giving us!

Nayak

  

-------------------------------------------
Nayak Polissar
Principal Statistician
The Mountain-Whisper-Light Statistics
-------------------------------------------







Original Message:
Sent: 07-17-2013 14:50
From: David Mangen
Subject: Entering and Summarizing Focus Group Data

It is at times odd how many different uses people find for Excel.

I too would use something else for this.  The only possible advantage that I can see for using Excel in this context is that you could fairly quickly compare the responses within a subject across questions, or within questions across respondents.  If I wanted to retain that advantage I would probably go to a database program and use a VARCHAR field.  

-------------------------------------------
David Mangen
Owner
Mangen Research Associates, Inc.
-------------------------------------------

Another tool that could be of use in such qualitative research studies is EthnoHub:

http://ethnohub.com

-------------------------------------------
Karen Copeland
Boulder Statistics
-------------------------------------------







Original Message:
Sent: 07-17-2013 15:16
From: Nayak Polissar
Subject: Entering and Summarizing Focus Group Data

My only comment is: what a marvelous group that people can get help so quickly, of such diversity, and on a topic that is not commonly treated in statistical education. I am very proud of self-giving us!

Nayak

  

-------------------------------------------
Nayak Polissar
Principal Statistician
The Mountain-Whisper-Light Statistics
-------------------------------------------

I would agree with those who suggest sticking with Word or some other text editor, and using verbatim transcriptions and qualitative/text analysis software, such as NVivo, Ethnograph, RQDA, etc.  You should find a lot if you google "NVivo alternatives".  The wealth of information gained from qualitative data can be well worth the time and effort it takes to analyze!   It's really not supposed to become just an exercise in number-crunching.  You might talk with an anthropologist who has a background in ethnography, for detailed advice.  Research Methods in Anthropology, by H. Russell Bernard, may be helpful as well.  If not, it is at least interesting.  My own background is in anthropology, so I may be biased about some of that.

-------------------------------------------
David G. McBride, Ph.D.
Senior Epidemiology Specialist, Office of Epidemiology
Missouri Department of Health and Senior Services
-------------------------------------------

Maybe you can try DtmVic software, which is free.
http://www.dtmvic.com/05_SoftwareE.html

There's tutorial with examples
http://www.dtmvic.com/06_TutorialE.html

and users' guide
http://www.dtmvic.com/06_ManualE.html

Vesna


-------------------------------------------
Vesna Ilakovac
School of Medicine, J.J.Strossmayer University of Osijek
Osijek, HR-31000, Croatia

-------------------------------------------