Blog Viewer

Principle B: Integrity of data and methods

  
The ASA Committee on Professional Ethics seeks input on the Ethical Guidelines for Statistical Practice, which are permanently linked here:
http://www.amstat.org/ASA/Your-Career/Ethical-Guidelines-for-Statistical-Practice.aspx


If you would like to contribute a recommendation for revision to the Guidelines, or for a comment for the linked discussion, we have created discussion threads for each of the Guidelines' individual principles. Please comment on the principle(s) most directly related to your suggestion(s).

Your suggestions should be as specific and complete as possible so that the Committee or its designated Working Group can review and consider your suggestions and input. All suggestions received through these discussion threads will be considered by the Committee.


The ethical statistician is candid about any known or suspected limitations, defects, or biases in the data that may impact the integrity or reliability of the statistical analysis. Objective and valid interpretation of the results requires that the underlying analysis recognizes and acknowledges the degree of reliability and integrity of the data.

The ethical statistician:

  1. Acknowledges statistical and substantive assumptions made in the execution and interpretation of any analysis. When reporting on the validity of data used, acknowledges data editing procedures, including any imputation and missing data mechanisms.
  2. Reports the limitations of statistical inference and possible sources of error.
  3. In publications, reports, or testimony, identifies who is responsible for the statistical work if it would not otherwise be apparent.
  4. Reports the sources and assessed adequacy of the data; accounts for all data considered in a study and explains the sample(s) actually used.
  5. Clearly and fully reports the steps taken to preserve data integrity and valid results.
  6. Where appropriate, addresses potential confounding variables not included in the study.
  7. In publications and reports, conveys the findings in ways that are both honest and meaningful to the user/reader. This includes tables, models, and graphics.
  8. In publications or testimony, identifies the ultimate financial sponsor of the study, the stated purpose, and the intended use of the study results.
  9. When reporting analyses of volunteer data or other data that may not be representative of a defined population, includes appropriate disclaimers and, if used, appropriate weighting.
  10. To aid peer review and replication, shares the data used in the analyses whenever possible/allowable, and exercises due caution to protect proprietary and confidential data, including all data that might inappropriately reveal respondent identities.
  11. Strives to promptly correct any errors discovered while producing the final report or after publication. As appropriate, disseminates the correction publicly or to others relying on the results.
​​
1 comment
10 views

Permalink

Tag

Comments

02-08-2020 16:37

I have a suggested addition to Principle B. In considering the existence of legacy models and algorithms that are in place across the many contexts of practicing statistics and data science, I suggest the ASA Guidelines incorporate a new element of Principle B that mirrors one element from the Association of Computing Machinery, namely, 3.6, "Retire legacy systems with care".  Organizations that employ statisticians and data scientists regularly utilize models and algorithms that are fixed at a point in time while the data on which they are deployed may actually change over time. Sensitivity checks are needed over time to ensure that decision-making these models and algorithms support is consistently effective and that assumptions are tested and met. When those sensitivity checks are failed, then those models and algorithms certainly need to be reconsidered by the ethical statistician and data scientist, especially if they are to be “retired”. Thus, my recommendation is that the new item (B 12) be considered:
"Performs sensitivity checks over time to ensure that the decision-making supported by models and algorithms is effective, and that assumptions are tested and met over time. When those sensitivity checks are failed, then those models and algorithms should be reconsidered by the ethical statistician and data scientist, especially if they are to be “retired”."