Discussion: View Thread

This message has been cross posted to the following eGroups: Statistical Consulting Section and Statistical Education Section .
-------------------------------------------
Hi All,

I have an opportunity to do some educational research at about 10-15 different schools in Georgia.  I would be the PI for the study and expect to have at most one other researcher helping me with the study.

Recently, I created a pre-statistics course for a private university (where I am Adjunct Faculty).  One of the purposes of the course was to reduce student statistics anxiety.  The course was approved and accepted by the university and I was selected to teach its pilot in November.  One of my main outcomes is (read, I know it worked if)  a statistically significant reduction in student anxiety after taking the class is observed - a classic before and after quasi-experiment. (I can't choose which students take the class and I can't compare it to students who have not taken the pre-statistics class unless I wait for both group's performances in the real stats class.)  Hence, my concerns.

1) The university will most likely not like to wait for the following semester to get data from the pilot.

2) As far as the university is concerned, the results of the pilot are just what's expected for continual improvement for the class.  My understanding is that an IRB is not needed if an instructor collects data just to improve his class and has no intentions to publish the results.

To get around #1 above, I wanted to compare the anxiety levels of students taking introductory statistics classes at various colleges and universities around the state.  This includes research institutions, teaching instituitions, small schools, large schools, public schools, private schools, HBCUs and PWIs, undergraduate and graduate classes, stats classes for specific populations and stats classes for the general population offered through MATH or STAT departments..  Theses are not my factors of primary interest, but I'll have to account for them in my analysis.  Specifically, I wanted to contact instructors at the various schools that teach intro stats and ask them to administer the STARS assessment in the first week of their class, then again towards the end of the semester.  This is my full involvement with the students at the various schools.

Unfortunately, because of #2 above, my university will not be providing an IRB.  After all, it's just curriculum improvement to them.  In order to report out on the effectiveness of my intervention however, I'm going to need to compare it to some baseline - to the anxiety levels of other students in other intro stats courses is just one option.  Regardless, I think that this is information worth publishing.  My questions follow:

1. In this situation, would I need an IRB for each school that I contact?
2. Can I get the IRB under my company name and not my school's name?
3. Must I have a Co-PI at the other school before it will accept my proposal?
4. Do I need to have the IRB accepted at my "home institution" before these other schools consider it?
5.  Do you think this research would fall under the Exempt category?
6. What are the steps that I need to take to get the approvals before the Fall semester starts?
7. Where can I (as an independent statistical consultant) get Human Subjects training in order to do this study?

I usually do physical science research so IRBs are completely new to me.  I thank you for all of your suggestions.

Ray

-------------------------------------------
Raymond Mooring
Senior Statistical Consultant
Analysis Made Easy
-------------------------------------------

Your situation is complicated, and I suspect nobody can give you a definitive answer.

On one level it's simple.  The applicable Federal law states that educational research is exempt from IRB requirements.  But you face several issues because the statute doesn't define educational research, and with your outcome being "stats anxiety" you are in a grey area.  Furthermore, you are not allowed to decide for yourself that your project is exempt: the IRB itself must declare your project exempt.  (So, de facto, you are not exempt from applying for IRB approval, but they may exempt you from consent procedures, annual review, etc.)   

So I think you will have to contact the IRB at each institution to find out what their policies are.

I have served on the IRB at two different institutions, and most of my funded research involves cross-institutional collaborations, so I have a bit of experience with these issues.  Based on that, here is my take on your specific questions:

1.  It is uncommon for an institution's IRB to waive their own review and just accept another institution's approval.  Again, the statute only requires that you receive approval from some IRB, any IRB, anywhere.  But institutional policies usually require their own IRB to review and approve any research carried out on their campuses or by their faculty.

2.  This isn't a meaningful question.  Any IRB will require you to identify by name all key personnel who are involved in carrying out the research.  If you make personnel changes, you will have to go back to the IRB to get approval for that amendment.  The IRB approves the protocol and the personnel as a package.  Your company is relevant only to the extent that it might raise conflict-of-interest issues.

3.  Whether you will need a co-PI at each institution will depend on institutional policy.  I think in most cases, you will be required to do that, so the IRB has somebody that they can hold accountable.  But you might find some institutions willing to let you go on your own for a small project like this.

4.  In multi-institutional research, each institution typically issues an approval conditional on the approval of all of the other involved institutions, and will require you to submit copies of all the other approvals before starting the research.  As a practical matter, you should begin the process at each institution as soon as possible and not wait for any one institution to act first.

5.  Probably, though some IRBs may balk at the stats anxiety outcome.  If your outcome were a measure of achievement, there would be no question about it--but you would still have to get the IRB to declare your project as meeting the requirements for exempt status.

6.  Each IRB will have its own application process and procedures.  You need to contact them directly and find out what to do.  With July about 1/3 over I would say you are already behind schedule if you want to start your research in the Fall semester.  While some IRBs can turn these things around in a matter of a week or two, if anybody has questions or concerns, it could take a few months--especially over the summer with staff and IRB members taking vacations.

7.  Each IRB will have its own policies about which human subjects training course(s) they accept.  In the worst case scenario, you will have to take a separate course for each institution!  More likely, there will be some on-line course that several of them accept, and, in my experience, it is often possible to persuade the remainder to accept that one just for you if you are only going to do one project there. 

Frankly, even apart from this rigmarole, I really question whether these other institutions constitute a reasonable control group for your study--but that's another issue and I'll leave it at that.

-------------------------------------------
Clyde Schechter
-------------------------------------------

To add to this post and to my post of yesterday: In my and my colleagues' experience, the IRB's concern about "statistics anxiety," absent plans to present or publish your results, will be whether the nature of the questions could cause students discomfort or embarrassment. I haven't found IRBs to be unreasonable in evaluating this, but we did have a case on one of our campuses in which the IRB objected to use of overly personal questions without a consent form that indicated clearly to students that they had a right to opt out. Additionally, the researchers had gone ahead without consulting the campus IRB, which found out about it only when students complained to the dean, so all faculty were strongly admonished by the administration not to take such initiative in the future. So, yes, you should run it by the IRB and not assume it's exempt without being told so.

I have found our IRB leaders, whether on campus or in the private group I mentioned, to be very accommodating in responding to our informal descriptions of the study and letting us know whether we should file with them and how. My experience with IRBs is that they tend to be detail-oriented, and want their jurisdiction and concerns respected, but they also want to support good research and help faculty to carry it out properly.

One other thought: Since you are an adjunct faculty at a university, and most universities contract with CITI, you should have no problem in creating a login using your affiliation and doing the online training without charge.

-------------------------------------------
Annette Gourgey
-------------------------------------------
CUNY






Original Message:
Sent: 07-09-2012 11:07
From: Clyde Schechter
Subject: IRB Policies for External Researcher

Your situation is complicated, and I suspect nobody can give you a definitive answer.

On one level it's simple.  The applicable Federal law states that educational research is exempt from IRB requirements.  But you face several issues because the statute doesn't define educational research, and with your outcome being "stats anxiety" you are in a grey area.  Furthermore, you are not allowed to decide for yourself that your project is exempt: the IRB itself must declare your project exempt.  (So, de facto, you are not exempt from applying for IRB approval, but they may exempt you from consent procedures, annual review, etc.)   

So I think you will have to contact the IRB at each institution to find out what their policies are.

I have served on the IRB at two different institutions, and most of my funded research involves cross-institutional collaborations, so I have a bit of experience with these issues.  Based on that, here is my take on your specific questions:

1.  It is uncommon for an institution's IRB to waive their own review and just accept another institution's approval.  Again, the statute only requires that you receive approval from some IRB, any IRB, anywhere.  But institutional policies usually require their own IRB to review and approve any research carried out on their campuses or by their faculty.

2.  This isn't a meaningful question.  Any IRB will require you to identify by name all key personnel who are involved in carrying out the research.  If you make personnel changes, you will have to go back to the IRB to get approval for that amendment.  The IRB approves the protocol and the personnel as a package.  Your company is relevant only to the extent that it might raise conflict-of-interest issues.

3.  Whether you will need a co-PI at each institution will depend on institutional policy.  I think in most cases, you will be required to do that, so the IRB has somebody that they can hold accountable.  But you might find some institutions willing to let you go on your own for a small project like this.

4.  In multi-institutional research, each institution typically issues an approval conditional on the approval of all of the other involved institutions, and will require you to submit copies of all the other approvals before starting the research.  As a practical matter, you should begin the process at each institution as soon as possible and not wait for any one institution to act first.

5.  Probably, though some IRBs may balk at the stats anxiety outcome.  If your outcome were a measure of achievement, there would be no question about it--but you would still have to get the IRB to declare your project as meeting the requirements for exempt status.

6.  Each IRB will have its own application process and procedures.  You need to contact them directly and find out what to do.  With July about 1/3 over I would say you are already behind schedule if you want to start your research in the Fall semester.  While some IRBs can turn these things around in a matter of a week or two, if anybody has questions or concerns, it could take a few months--especially over the summer with staff and IRB members taking vacations.

7.  Each IRB will have its own policies about which human subjects training course(s) they accept.  In the worst case scenario, you will have to take a separate course for each institution!  More likely, there will be some on-line course that several of them accept, and, in my experience, it is often possible to persuade the remainder to accept that one just for you if you are only going to do one project there. 

Frankly, even apart from this rigmarole, I really question whether these other institutions constitute a reasonable control group for your study--but that's another issue and I'll leave it at that.

-------------------------------------------
Clyde Schechter
-------------------------------------------