Discussion: View Thread

Pro bono consulting for statistically naive clients

  • 1.  Pro bono consulting for statistically naive clients

    Posted 01-26-2012 12:43
    Hi, All.

    I've considered doing pro bono stats consulting for, say, local non-profit organizations.  Here's a challenge I anticipate: How do I describe to a statistically naive client how a statistician might help her, especially when I'm naive about her organization?  I'll elaborate below, but here's the bottom line: I'd appreciate ideas about strategies to initiate and foster productive consulting relationships with such clients.

    An example, for concreteness: Suppose I contact a local animal shelter and meet with someone "in charge" to discuss types of statistical help I could provide.  Having experience with such shelters only via visits to adopt pets, I have little idea how they operate or what kinds of decisions they make that empirical data might inform.

    Like many consultants, I'm used to learning about a client's needs and circumstances and explaining which statistical techniques might be most appropriate.  Most of my (paying) clients, however, are seasoned researchers -- usually faculty members in the social, behavioral, or health sciences -- with whom I already share considerable background knowledge about empirical research, basic statistics, and the publication process.  Furthermore, these clients initiate our consulting relationships with fairly specific research questions and aims in mind.

    In contrast, I can imagine offering my services to pro bono clients who have no experience with even basic statistics and little idea how to formulate questions amenable to data collection and statistical analysis.  To complicate matters further, with such clients I'd usually initiate the relationship and would enter their environment with very little understanding of their mission, day-to-day operations, resources, administrative/managerial issues, etc.

    Without having tried this, I suspect I'd do something like the following in an initial meeting with a pro bono client who has very little stats background:

    • Ask him to describe some basic information about the organization.
    • Give him a brief, accessible overview of types of issues statistics can help address.
    • Brainstorm with him about questions that are relevant to the organization and amenable to data collection and analysis.

    Based on what I glean from that meeting, I could do my homework and give the client more specific ideas tailored to his situation.

    Has anyone on this list worked with pro bono clients, especially outside of your usual consulting context?  Either way, do you have ideas or know of resources that might be useful for consultants who are new to this activity?  A possibly useful resource I've encountered in casual web browsing is the STATCOM initiative:

      www.amstat.org/education/statcom


    Thanks in advance!


    -------------------------------------------
    Adam Hafdahl
    Statistical Consultant
    ARCH Statistical Consulting, LLC
    -------------------------------------------


  • 2.  RE:Pro bono consulting for statistically naive clients

    Posted 01-26-2012 12:52
    You should check out the ASA communities:  http://community.amstat.org/statisticswithoutborders/home/
    [Statistics without Borders] and http://www.statisticianswithoutborders.net/index.html [Statisticians without Borders (clinical research)].

    We offer statistical services throughout the world (and lots in the US) to non-profits etc.


    -------------------------------------------
    Elaine  Allen
    Babson College
    -------------------------------------------








  • 3.  RE:Pro bono consulting for statistically naive clients

    Posted 01-26-2012 13:06
    First of all I don't see why you lump naive clients with pro bono clients.  I have come across many paying clients that are pretty naive about statistics.  Next I would take the approach to talk about general classes of problems that require ststistical expertise rather than jump right in to their specific problems.  I find many common themes for clients from a wide variety of fields.  Most are involved in some form of experimentation.  So first explain to them how statisticians can help with the design of an experiment by determining what form of randomization is best to use and how many samples are needed to get precise estimates or to draw inferences to the population based on the sample.  Then mention that statisticians have special techniques and models depending on the type of data.  If data are gathered over time and predicting the future values is important we use time series methods.  For time to event data we have survival analysis.  Continuous variables may be studied comparing means, medians and/or variances while categorical data is dealt with using contingency table analysis.  You could go on and on this way.  After this introduction to statistics they may be prepared to describe specific problems.  At that point you can enter a dialogue as you would with clients that have a basic understanding of what statisticians do.

    -------------------------------------------
    Michael Chernick
    Director of Biostatistical Services
    Lankenau Institute for Medical Research
    -------------------------------------------








  • 4.  RE:Pro bono consulting for statistically naive clients

    Posted 01-26-2012 13:24

    This is an interesting question.  I have done some consulting work without pay but they have approached me or I got their name from another statistician.  Most of my stat work without pay came from some professors at a local university that would get a lot of stat requests and they would occassionally talk me into helping them.  I recall once working for the Indiana Arts Commission and they wanted help in an algoritm to spread out some state money to help the arts.  It was a fun project and they gave me a free poster as payment.  I think there are a lot of folks looking for free stat consulting and if you want to help out I would advise getting to know some of the professors at your local universion.
    -------------------------------------------
    Rocco Brunelle
    Senior Statistician
    Bowsher Brunelle Smith LLC
    -------------------------------------------








  • 5.  RE:Pro bono consulting for statistically naive clients

    Posted 01-26-2012 13:25
    I find myself in a similar situation in a corporate environment, trying to communicate to project leaders ways that statistics could benefit their projects.  It really comes down to what the value of statistics is.  I can't say I've been fully successful, but I have some ideas:
    1. Try to break down the skills you have into a discrete set of things.  Possibilities include predictive modeling, identifying potential causative agents, optimization, and inference, just to name a few.
    2. Show a few pictures illustrating case studies representing each item.  Hopefully the picture will stick in their head and they may see something that reminds them of it.  (This may happen after some time.)  For instance, one of the items I illustrated was a classification problem, and someone from a diagnostic assay project asked, since a diagnostic assay is classifying, if classification methods could be applicable to diagnostics.  Bingo!
    3. A colleague made a reasonable suggestion:  ask them where they have the biggest or most complex data.  Chances are that will lead to a statistics problem.
    4. Similarly, do they have "pain points" involving data?
    5. Seymour Geisser was famous for saying that statistics is really about prediction.  Are there things that they would benefit from being able to predict?  Are adoptions seasonal?  What about animal arrivals?  Do either relate to other factors?
    Good luck!
     

    -------------------------------------------
    James Garrett
    Manager, R&D Statistics
    Becton Dickinson
    -------------------------------------------








  • 6.  RE:Pro bono consulting for statistically naive clients

    Posted 01-26-2012 17:23
    I agree, you need to understand their world view and perspectives.  This means that you need to spend some time gathering background information on your target organization both from the general field and specific to the site you want to deal with.  Volunteer to really understand what issues they are facing. Get into the scholarship in the field.  Google Scholar on Animal Shelters, for example.  What problems are they facing.  Remember that many non-profits are so busy keeping their head above water that they don't have time to bail the boat.  And, you are often asking them to think in a different way about their organization.

    Let's take the animal shelter example.  A quick Google search showed a recurrent theme posted on animal shelter sites:  people adopting usually have no idea of the time and expense of caring for a pet.  Why don't you draw up a quick little comparison chart that would show the relative costs of caring for a dog, cat, bird, turtle, etc.  You could get information from searching, from talking with local vets, talking with animal trainers, talking to pet store owners, etc.

    Then you could go back to the animal shelter and have a chart that would help the shelter educate its patrons.  The shelter wants to place pets but wants to do that with an increased probability of success and some assurance that the pet won't be returned because it was 'too expensive to keep'

    It's not a sexy cox proportional analysis of 'survival' but it is putting reality into practice in a way that the shelter may not be able to do.

    You just can't show up and expect that in a hour you will have a project.  Some of the pro bono work I have done with non-profits has taken a year or more to clearly identify appropriate projects.  You need to understand them as much as they need to understand you.

    Keep at it, it can be great fun and very rewarding.



    -------------------------------------------
    William Grant
    Professor, Emergency Medicine
    SUNY Upstate Medical University
    -------------------------------------------








  • 7.  RE:Pro bono consulting for statistically naive clients

    Posted 01-26-2012 18:11
    These are all interesting ideas but I suggest another approach. Rather than explaining statistics to the client, who at this point does not see the relevance of the subject, let's get the client to talk about their organization and the challenges that they face. Most clients can go on for quite some time on this topic -- it's dear to their hearts -- and what they say will be very revealing. As we listen, we ought to be able to identify a few areas where we might be able to help.

    At this point, we have an engaged client, since we are talking about their problems rather than our skills. Then, when we need to describe our statistical methodology, we can do so in the terms on their problem rather than someone else's or something out of a textbook. And let's not bore them with the technicalities but rather let's emphasize the ways in which our analyses can help them make better decisions or solve their current dilemma.

    The point is, let's listen to our clients, naive or not, paying or not, and talk to them on their terms rather than on ours. Let's avoid giving them a broad overview of statistics hoping that they will latch on to something we say.

    For what it's worth.

    -- Tom 


    -------------------------------------------
    Thomas Sexton
    Professor and Associate Dean
    Stony Brook University
    -------------------------------------------








  • 8.  RE:Pro bono consulting for statistically naive clients

    Posted 01-27-2012 08:00
    I tell my students that the Fundamental Law of Statistical Collaboration is (from Stephen Covey's 7 Habits), "Seek first to understand, then to be understood."

    This applies to all clients, from those naive about statistical methods to those published in stat journals.

    The suggestion above to research the client and produce a simple graphic is a great suggestion too, as long as it's presented more like, "Is this what you're interested in?" rather than, "Here's an example of what basic statistics can do for you."  

    -------------------------------------------
    Eric Vance
    Assistant Research Professor
    Director of LISA (Laboratory for Interdisciplinary Statistical Analysis)
    Virginia Tech
    -------------------------------------------








  • 9.  RE:Pro bono consulting for statistically naive clients

    Posted 01-27-2012 08:45
    I agree with Eric and Tom.  To clarify my previous suggestion, I was not saying to hit them with a broad overview of statistics but rather under the premise that they are very naive at least give them an idea as to where statistics impacts real applications the most.  One such area is the design and analysis of experiments.  For many clients this may strike a cord.  The point is to initiate a dialogue that takes you to a point where the client can express his problem in a way that the statistician can see how statistics can play a role.  Any way to productivrly start that dialogue is fine.  It is just that these two approaches will probably not work.
    1.  The statistician gives an overview lecture on statistics to the client hoping that a light bulb will strike somewhere along the line.
    2. The client goes into a detailed discussion of his problem without having the slightest idea as to what statistics can do.

    In both of these scenarios the communication is one way.  There is a tendency for each to use the jargon he is most comfortable with.  I just think these two approaches are likely to lead to ineffective communication, delaying or preventing the goal of a solution to the problem.  The most important aspect of consulting is effective communication.  Once you have that an intelligent client and a skilled statistician will have a much easier road to a solution.  Without it you may never get to a reasonable solution no matter how smart you are. 

    -------------------------------------------
    Michael Chernick
    Director of Biostatistical Services
    Lankenau Institute for Medical Research
    -------------------------------------------








  • 10.  RE:Pro bono consulting for statistically naive clients

    Posted 01-27-2012 08:47
    Eric
    Good point about how to present it.  Thanks

    Bill

    -------------------------------------------
    William Grant
    Professor, Emergency Medicine
    SUNY Upstate Medical University
    -------------------------------------------








  • 11.  RE:Pro bono consulting for statistically naive clients

    Posted 01-27-2012 10:55
    The comments emphasizing "listen to and learn their needs and desires, rather than try to sell them on our skills and offerings" match most closely with my experience.

    Even working with long-time scientists, I occasionally have my most effective influence when I let them show what they have done and what they are trying to accomplish (they are already vested in it).  In a recent case, an engineer had already collected appropriate data and had it in a spreadsheet, but it wasn't arranged for simple plotting and analysis.  After 20 minutes of rearranging the data appropriately and creating and labeling plots, he had a picture that told a clear story.  A couple of days later, the clear outcome and decision was shared with me after the scientist had shown the plot and the new questions it raised with his colleagues.  I didn't know the processes or ramifications involved -- they did -- but I was able to ask the questions made obvious by appropriately plotting data they already had.

    A similar thing can happen frequently in non-profit organizations.  I have sometimed volunteered time with educators (especially at the K-12 level).  The most effective cases involved conferring first with the teacher on what the particular students' needs were, and tailoring my involvement to their needs, with a goal at helping expand the horizons and ability to proceed self-sufficiently with a new skill.  If many of us do this effectively -- in baby steps in various organizations -- then some grateful benefactors will have a tendency to share the practicality of statistical consulting by word of mouth with people we know.  Like others have said, we won't immediately wow anyone, but the rewards and potential long-term impact are great.

    -------------------------------------------
    Mark Martin
    Sr. Biostatistician
    Siemens Healthcare Diagnostics
    -------------------------------------------








  • 12.  RE:Pro bono consulting for statistically naive clients

    Posted 01-27-2012 12:01
    Your examples sound good and I would agree with your approach.  I don't think the situations are the same as with a naive client as the question was originally raised.  In your first example you say that once the data was plotted properly the result was clear and was explained to you to your satisfaction although you said that you didn't know all the details and the ramifications.  That concerns me a bit.  Again I would emphasize that clear communication is important.  If you had enough detail to understand the problem and know that the statistical approach and conclusion were sound then it is fine.  But sometimes crucial details are left out and the solution may not be as clear cut as you think.  I would prefer to feel that i understood the problem well and that he understood the reason the statistical methods used were appropriate and understood any limitations based on statistical assumptions. Clear communication on both ends is needed.

    In the case of the naive client some discussion of what statistics can and can't do is very important and should be made upfront.  For example, clients often come to me and describe the medical details of the problem and then thinking that is enough they ask me to tell them how many patients are needed to get significant results.  This may be a misunderstanding of what statisticians can do.  Maybe they really think we can provide them with a sample size large enough to guarantee success.  Inspite of all the detailed information they gave crucial information required for the sample size problem is missing.  What do I need?
    1, the primary endpoint(s)
    2. what is a clinically significant difference or precision of estimation desired
    3. are they testing a hypothesis about the difference in the endpoints between groups or are the looking to obtain precise estimates
    4. do they have literature to help tell them the magnitude of the variability across patients
    5. if information for sample size estimation is not available would an adaptive design with an interim look for sample size reassessment be feasible and if so what initial N would be economically feasible for the interim analysis

    Sometimes some of this information has come out before I question them but not all of it (except on rare occasions) and it is not uncommon for them to naively request a sample size estimate from me without have provided any of this information.  So a dialogue must go on until I have gotten sufficient information on 1-5 above.  Also before I do any of this if the question was raised in a way that they seem to think I have magical powers to give them publishable clinically significant results I need to explain to them that I can't guarantee successful research.  They also must be made clear of the difference between clinical and statistical significance. Even with a sample size large enough that a clinically significant difference would be detected with high power, it is possible that the difference is not clinically significant or it is significant in the wrong direction.  Too often clients want to define clinical significance to be the difference that is statistically significant (with high power) for a suitably small sample size.  So they make the difference large, running the risk that for a smaller but truly clinically significant difference that could have been detected with a larger sample size but not with the chosen size.

    The point is that the statistician must understand the clients problem well enough that the statistician and the client are both sure that they got the solution to the "right" problem.  The client must be able to commuicate the information that the statistician needs to solve the problem and understand the limitations of the methodology and the results well enough to not have false expectations or publish incorrect conclusions.

    I hope this illustrates why clear two-way communication is essential to good consulting.  Who speaks first is less important as long as both get to speak enough to avoid misunderstandings.

    -------------------------------------------
    Michael Chernick
    Director of Biostatistical Services
    Lankenau Institute for Medical Research
    -------------------------------------------








  • 13.  RE:Pro bono consulting for statistically naive clients

    Posted 01-30-2012 10:37
    I agree with everyone who has said something on the order of "listen and understand" and want to take it a bit further.  Don't assume that the naive client actually does need statistics.  I was the "go to" person for statistics at Goodyear for a long time. 

    One time a colleague came to me and described a problem involving inferring the size and shape of elliptical particles based on the shadow they cast in an electron microscope experiment.  He specifically asked for a statistical method.  I told him, "Let me think about this for a while."  After thinking I got back to him and told him I thought that I could solve it using solid geometry instead of statistics.  He expressed surprise but indicated that he just needed a solution and that the method was irrelevant.  Sure enough, solid geometry gave us a direct method.

    Later on he or we probably used statistics to look at the distribution of particle sizes and shapes.  (It was too long ago for me to remember the follow-up except that he applied for a patent and omitted my name from the list of inventors, but that's a different story.)
    -------------------------------------------
    Emil M Friedman, PhD
    emil.friedman@alum.mit.edu (forwards to day job)
    emilfrie@alumni.princeton.edu (home)
    http://www.statisticalconsulting.org



  • 14.  RE:Pro bono consulting for statistically naive clients

    Posted 01-30-2012 10:57
    It would seem to me that the object is three dimensional and the shadow is two dimensional.  The angle that light hits the particle should determine the shadow.  But wouldn't there be a problem of uniqueness for the object? Suppose the light is coming straight down from the top of the ellipsoid.  Then you can determine two axes but not the third.  I think there must be an additional assumption such as the third axis is very small in size.

    So the answer via solid geometry is the easy part.  But there are some interesting statistical issues that affect the accuracy of the solution. For example
    1.  How accurately do we know the angle of incidence for the light hitting the particle?
    2. How accurately can we measure or estimate the semi-major and semi-minor axes of the elliptical shadow?

    Shouldn't this be brought up in the initial consultation?  Statisticians know that point estimates do not provide a complete answer to a problem unless the signal to noise ratio is very large.
    -------------------------------------------
    Michael Chernick
    Director of Biostatistical Services
    Lankenau Institute for Medical Research
    -------------------------------------------