ASA Connect

------Original Message------

I am interested in finding the variables significant for the response and predict its influence. Hence Principal Component Analysis is not going to help in this case.

Thank you

------------------------------
Uday Jha
Rochester Institute of Technology

Original Message:
Sent: 01-03-2017 13:18
From: David Wilson
Subject: Predicted R-Squared

 

Without knowing what you are trying to accomplish, I suggest investigating the use of principal components.

With 1000 potential "predictor" variables and 100 observations you can often "perfectly fit" any response, even if the "response" come from a random number generator.  In other words, you can "predict" history but those predictions will be completely useless in predicting the results of future experiments.

If you are able to reduce the number of predictor variables to something "reasonable", you might have something useful, but you'll need to work closely with an experienced consultant.  R.I.T. should have people like that, but consultants don't work for free.

------------------------------
Emil M Friedman, PhD
emilfriedman@gmail.com
http://www.statisticalconsulting.org

Original Message:
Sent: 01-03-2017 14:47
From: Uday Jha
Subject: Predicted R-Squared

I am interested in finding the variables significant for the response and predict its influence. Hence Principal Component Analysis is not going to help in this case.

Thank you

------------------------------
Uday Jha
Rochester Institute of Technology

Before constructing any model for prediction purposes it is vital that you do an Exploratory Data Analysis.

This involves understanding the distributions of all your variables and whether there are missing values, outliers and particularly influential points. You should also understand the correlations between your variables and would include an exploratory principal components analysis. This may well lead to the conclusion that are your variables fall into a number of distinct groups. If so, you should ask whether you need to have all of them present in your model.

In the first instance you should try to use subject matter expertise to reduce the number of variables. If you have no subject matter expertise there are a few empirical things you can try. You might need to choose a random sample of the variables to get a model that can be estimated with only 100 observations.

I would do a random forests prediction, as that produces a variable importance score that tells you which variables are the key ones from the point of view of predictive accuracy.

A CART model would also be worth looking at - it will throw away most of the variables and use just a few of them.

The LASSO method also picks out a subset of variables and rejects the rest.

One way or another you need to get the number of variables down to something that can be estimated with 100 observations.

------------------------------
Blaise Egan
Lead Data Scientist
British Telecommunications PLC

Original Message:
Sent: 01-03-2017 14:47
From: Uday Jha
Subject: Predicted R-Squared

I am interested in finding the variables significant for the response and predict its influence. Hence Principal Component Analysis is not going to help in this case.

Thank you

------------------------------
Uday Jha
Rochester Institute of Technology

The usual recommendation for a MR study is that you have 20 cases per predictor. That means eliminating 995 variables which is unrealistic. Stepwise and other predictor-selection procedures would merely capitalize on chance. If you can select a small number of predictors based on theory (independently of the data), you could do an analysis but it appears that your study is purely exploratory. There is, of course, the issue of how your sample of cases was chosen which raises a whole new set of concerns. 

------------------------------
Chauncey Dayton

Original Message:
Sent: 01-05-2017 05:48
From: Blaise Egan
Subject: Predicted R-Squared

Before constructing any model for prediction purposes it is vital that you do an Exploratory Data Analysis.

This involves understanding the distributions of all your variables and whether there are missing values, outliers and particularly influential points. You should also understand the correlations between your variables and would include an exploratory principal components analysis. This may well lead to the conclusion that are your variables fall into a number of distinct groups. If so, you should ask whether you need to have all of them present in your model.

In the first instance you should try to use subject matter expertise to reduce the number of variables. If you have no subject matter expertise there are a few empirical things you can try. You might need to choose a random sample of the variables to get a model that can be estimated with only 100 observations.

I would do a random forests prediction, as that produces a variable importance score that tells you which variables are the key ones from the point of view of predictive accuracy.

A CART model would also be worth looking at - it will throw away most of the variables and use just a few of them.

The LASSO method also picks out a subset of variables and rejects the rest.

One way or another you need to get the number of variables down to something that can be estimated with 100 observations.

------------------------------
Blaise Egan
Lead Data Scientist
British Telecommunications PLC


Original Message:
Sent: 01-03-2017 14:47
From: Uday Jha
Subject: Predicted R-Squared

I am interested in finding the variables significant for the response and predict its influence. Hence Principal Component Analysis is not going to help in this case.

Thank you

------------------------------
Uday Jha
Rochester Institute of Technology

------Original Message------

Before constructing any model for prediction purposes it is vital that you do an Exploratory Data Analysis.

This involves understanding the distributions of all your variables and whether there are missing values, outliers and particularly influential points. You should also understand the correlations between your variables and would include an exploratory principal components analysis. This may well lead to the conclusion that are your variables fall into a number of distinct groups. If so, you should ask whether you need to have all of them present in your model.

In the first instance you should try to use subject matter expertise to reduce the number of variables. If you have no subject matter expertise there are a few empirical things you can try. You might need to choose a random sample of the variables to get a model that can be estimated with only 100 observations.

I would do a random forests prediction, as that produces a variable importance score that tells you which variables are the key ones from the point of view of predictive accuracy.

A CART model would also be worth looking at - it will throw away most of the variables and use just a few of them.

The LASSO method also picks out a subset of variables and rejects the rest.

One way or another you need to get the number of variables down to something that can be estimated with 100 observations.

------------------------------
Blaise Egan
Lead Data Scientist
British Telecommunications PLC
------------------------------