ASA Connect

 View Only

  • 1.  Probability for error of a human reviewer

    Posted 03-10-2016 08:43

    Good afternoon,

    I am looking for publications on the probability that a human reviewer overlooks a defect, an error message in a protocol or the like.
    A rule of thumb I heard about several times states this should be in the range of .01 to .1. Is there any published empirical evidence for this?

    Kind regards,

    Christian Graf

    ------------------------------
    Christian Graf
    Dipl.-Math.
    Qualitaetssicherung & Statistik
    ------------------------------


  • 2.  RE: Probability for error of a human reviewer

    Posted 03-11-2016 03:13

    It sounds like you are looking at a human factors experiments. The error rate will not be constant. There is a lot that goes into an error rate. 

    ------------------------------
    Andrew Ekstrom

    Statistician, Chemist, HPC Abuser;-)

    Original Message


  • 3.  RE: Probability for error of a human reviewer

    Posted 03-14-2016 07:47

    Hello Andrew,

    definitely. I agree that the probability shouldn't be constant.

    I meanwhile found some publications on human realiability assessment that give error-probabilities for situations with varying factors such as routine/non-routine, under stress/no-stress, complexity, availability of a manual/action plan and so on. Research has been done especially for safety-critical situations like in aviation or nuclear power plants. There exist several systematic approaches to assess human reliability. A quite interesting topic.

    Kind regards,

    Christian

    ------------------------------
    Christian Graf
    Dipl.-Math.
    Qualitaetssicherung & Statistik

    Original Message


  • 4.  RE: Probability for error of a human reviewer

    Posted 03-14-2016 13:17

    Survey research is another field which may be relevant here. There is a discussion of coding errors in data processing in chapter 16 of Moser & Kalton's 'Survey Methods in Social Investigation'. I expect that there are more recent texts covering this, but this is what I had to hand (and remembered from student days). No doubt changes in data capture methods will have had their effect on practice, but the earlier research may still be of interest.

    Kind regards,

    Robin Darton 

    ------------------------------
    Robin Darton
    Senior Research Fellow
    Canterbury, Kent
    United Kingdom

    Original Message


  • 5.  RE: Probability for error of a human reviewer

    Posted 03-15-2016 03:34

    Thank you for the source and your comment, Robin.
    The research I found is mostly from the 90's. A source frequently mentioned in more recent publications is 'Kirwan, B., A guide to practical human reliability assessment. Taylor & Francis, 1994.' You can partially access it on google books.

    Best regards,
    Christian

    ------------------------------
    Christian Graf
    Dipl.-Math.
    Qualitaetssicherung & Statistik

    "To call in the statistician after the experiment is done may be no more than asking him to perform a post-mortem examination: he may be able to say what the experiment died of."

    Ronald Fisher in "Presidential Address by Professor R. A. Fisher, Sc.D., F.R.S. Sankhyā: The Indian Journal of Statistics (1933-1960), Vol. 4, No. 1 (1938), pp. 14-17'"

    Original Message