Discussion: View Thread

Hello:

Can anyone provide me with guidance on how to validate a scoring tool?

We are trying to develop a scoring tool  (rather improve an exisiting toool) to capture the severity of a clinical conidtion called incontinence-associated dermatitis (IAD).

The objective of the tool would be to track clinically relevant changes in the condition and help categorize IAD into either a mild, moderate, severe, or very severe cases.   We ultimately want a tool for use in future comparative studies.

I have read about face/content validity; criterion/construct-related validity; inter-rater reliability or reproducibility and test-retest reliability...all of which sound relevant. 

The questions are:
How do I do about proving my tool is valid in the eyes of the clinical users and the FDA reviewer?
What statistics or tests do I use to show validity?
What is the criteria for validity?
How do I determine the sample size for the study or studies that I need to prove validity/reliability.

Thanks much!


-------------------------------------------
Shelley-Ann Walters
3M
-------------------------------------------

Hello Shelly-Ann,

I am attaching a paper (if I handled the attachment procedure properly), a paper that is not statistically great, a paper that shows the steps the authors took to validate a tool. There seems to be lots of papers out there that define the various types of validation, but, I feel, what you need are some papers that actually are validating tools (instruments.) This is one.

I am also co-authoring a paper on validation of a pruritis (itchiness) scale to be used in a veterinary setting. It is in press (we just got the proofs), but that won't help you unless you wait a few weeks (or months.) 

You can also go to Google Scholar (in Google, just type "Google scholar".) There, if you search for "questionnaire validation", you should see lots of references to specific, published validation exercises. I tried that and it looked like there would be some nice examples. You can try other keywords in Google scholar, too.

So, one way to go on this is to find papers that carried out a validation exercise and emulate those papers whose statistical methods seem sound. 

Some time ago I perused a textbook on questionnaire construction and validation, but I don't remember the title. Also, I thought that doing everything the textbook recommended would be overkill. 

So, bottom line, I suggest that you find some good validation papers that have passed peer review in a good journal and whose statistical methods seem acceptable to you and just do what they did. 

By the way, questionnaire, tool, instrument--they can all be considered similar as far as validation goes.

I hope this helps you.

Best wishes,

Nayak



-------------------------------------------
Nayak Polissar
Principal Statistician
The Mountain-Whisper-Light Statistics
-------------------------------------------