Discussion: View Thread

This message has been cross posted to the following eGroups: Young Professionals Group and Statistical Consulting Section .
-------------------------------------------

Hello all:

I thought of this model specification test. Most likely, a similar idea has been developed in the past, and I would like to obtain some references from you.

The setup is as follows. There is a GLM

E[Y] = g(X'beta)

There is a large number of experimental units. Each unit has its own response vector, but the number of observations and the design are the same for all units. The model is fitted separately for each unit.

There are no units for which the true beta or the true response are known. Therefore, to test for possible model misspecification I introduce a binary factor, X_random, whose levels are assigned at random so that it is not associated with Y, and include it in the model:

E[Y] = g( (X | X_random)' beta )

If the model is well specified, the Type III p-value for X_random must be U(0, 1). I can collect p-values across all of the units and test them for uniformity.

Have anyone seen something like this before?

Regards,

Nik

Sorry, here's a more readable version:

​

Hello all:

 

I thought of this model specification test. Most likely, a similar idea has been developed in the past, and I would like to obtain some references from you.

 

The setup is as follows. There is a GLM

 

E[Y] = g(X'beta)

 

There is a large number of experimental units. Each unit has its own response vector, but the number of observations and the design are the same for all units. The model is fitted separately for each unit.

 

There are no units for which the true beta or the true response are known. Therefore, to test for possible model misspecification I introduce a binary factor, X_random, whose levels are assigned at random so that it is not associated with Y, and include it in the model:

 

E[Y] = g( (X | X_random)' beta )

 

If the model is well specified, the Type III p-value for X_random must be U(0, 1). I can collect p-values across all of the units and test them for uniformity.

 

Have anyone seen something like this before?

 

Regards,

Nik

It's not exactly in the same vein, but i have seen some propose the addition of pseudo variables to help with model selection.

Wu, Boos, D. D., and Stefanski, L. A (2007), "Controlling Variable Selection by the Addition of Pseudo Variables," Journal of the American Statistical Association, No. 477, p. 235-243.

You're situation is quite a bit different but I read what you had sent and immediately thought of this.

Hope it helps.

-------------------------------------------
Jason Brinkley
East Carolina University
-------------------------------------------



Original Message:
Sent: 09-26-2014 11:12
From: Nikita Tuzov
Subject: Model specification test - some advice needed.

Sorry, here's a more readable version:

​

Hello all:

 

I thought of this model specification test. Most likely, a similar idea has been developed in the past, and I would like to obtain some references from you.

 

The setup is as follows. There is a GLM

 

E[Y] = g(X'beta)

 

There is a large number of experimental units. Each unit has its own response vector, but the number of observations and the design are the same for all units. The model is fitted separately for each unit.

 

There are no units for which the true beta or the true response are known. Therefore, to test for possible model misspecification I introduce a binary factor, X_random, whose levels are assigned at random so that it is not associated with Y, and include it in the model:

 

E[Y] = g( (X | X_random)' beta )

 

If the model is well specified, the Type III p-value for X_random must be U(0, 1). I can collect p-values across all of the units and test them for uniformity.

 

Have anyone seen something like this before?

 

Regards,

Nik