A difficult circumstance arises when the algorithm is so complex that it's hard to know if it's correct or not. Look at any issue of JASA or Biometrics and you will see analyses like that. If the result is counter-intuitive, or disagrees with a simpler analysis, does that mean that the complex analysis is wrong? There are unfortunate stories about this, such as analyses of the relation of air pollution to health (submitted to Congress but subsequently retracted) and the recent front-page scandals about genomic screening in cancer chemotherapy. Huge data sets, complex algorithms: how can the analyst check his/her work and how can the reader believe the results? I worry that our tools have become too complex for us to understand how they work or what they produce.
Larry Muenz, statistical consultant
-------------------------------------------
Larry Muenz
-------------------------------------------