Discussion: View Thread

This message has been cross posted to the following eGroups: Statistical Consulting Section and Statistics in Marketing Section .
-------------------------------------------
Hello,

I am working on a post-promotional analysis and I am a little confused.

I have monthly transactional data on several markets (some were selected in a previous analysis to have the promotion).

I am now trying to show that the markets that had the promotion had a lift or performed better than the markets that were not selected for the promotion.

I ran a time series analysis model by market and now have both actual and predicted values for each market. I was thinking about using the predicted values as a baseline and use some calculations to show if the actual values had a lift over the baseline or not but I am not sure if this is the best way. 

Does anyone have any suggestions?

Thanks,

-------------------------------------------
Kenita Hall
Student

-------------------------------------------

You may want to consider your time series model, but add another parameter (mean shift) for those markets that received the promotion. The idea is to put the possibility of a lift into your analysis of the model instead of your analysis of (in effect) the residuals.

Exactly how you do this would require more knowledge of the problem. For example, if all markets are about the same size, perhaps one mean-shift parameter would provide an adequate representation. Or if the markets vary a lot in size, perhaps you can get away with one parameter if you analyze the data on a log scale. There are other, more complex, possibilities (different shift parameter for each promotional market, one overall shift parameter with random effects for the variation in shifts, ...) but the simpler the solution (assuming it's a reasonable solution...) the easier it will be to explain.

-------------------------------------------
Joseph Voelkel
Rochester Institute of Technology
-------------------------------------------

This is called intervention analysis.  The citation is:
-------------------------------------------
Chuck Coleman
U.S. Census Bureau and Private Consultant
-------------------------------------------







Original Message:
Sent: 01-13-2011 10:29
From: Joseph Voelkel
Subject: Post Promotional Analysis

You may want to consider your time series model, but add another parameter (mean shift) for those markets that received the promotion. The idea is to put the possibility of a lift into your analysis of the model instead of your analysis of (in effect) the residuals.

Exactly how you do this would require more knowledge of the problem. For example, if all markets are about the same size, perhaps one mean-shift parameter would provide an adequate representation. Or if the markets vary a lot in size, perhaps you can get away with one parameter if you analyze the data on a log scale. There are other, more complex, possibilities (different shift parameter for each promotional market, one overall shift parameter with random effects for the variation in shifts, ...) but the simpler the solution (assuming it's a reasonable solution...) the easier it will be to explain.

-------------------------------------------
Joseph Voelkel
Rochester Institute of Technology
-------------------------------------------

I suggest considering simpler options, such as using a decision tree approach or CHAID. While time series methods have their moments, in my experience, the extent of complexity they introduce does not always justify the extra light they may shed on the topic. ------------------------------------------- Mansour Fahimi VP, Statistical Research Services Marketing Systems Group -------------------------------------------
Original Message: Sent: 01-13-2011 10:36 From: Charles Coleman Subject: Post Promotional Analysis This is called intervention analysis. The citation is: Intervention Analysis with Applications to Economic and Environmental Problems G. E. P. Box and G. C. Tiao Journal of the American Statistical Association Vol. 70, No. 349 (Mar., 1975), pp. 70-79 ------------------------------------------- Chuck Coleman U.S. Census Bureau and Private Consultant -------------------------------------------
Original Message: Sent: 01-13-2011 10:29 From: Joseph Voelkel Subject: Post Promotional Analysis You may want to consider your time series model, but add another parameter (mean shift) for those markets that received the promotion. The idea is to put the possibility of a lift into your analysis of the model instead of your analysis of (in effect) the residuals. Exactly how you do this would require more knowledge of the problem. For example, if all markets are about the same size, perhaps one mean-shift parameter would provide an adequate representation. Or if the markets vary a lot in size, perhaps you can get away with one parameter if you analyze the data on a log scale. There are other, more complex, possibilities (different shift parameter for each promotional market, one overall shift parameter with random effects for the variation in shifts, ...) but the simpler the solution (assuming it's a reasonable solution...) the easier it will be to explain. ------------------------------------------- Joseph Voelkel Rochester Institute of Technology -------------------------------------------
Original Message: Sent: 01-13-2011 10:12 From: Kenita Hall Subject: Post Promotional Analysis This message has been cross posted to the following eGroups: Statistical Consulting Section and Statistics in Marketing Section . ------------------------------------------- Hello, I am working on a post-promotional analysis and I am a little confused. I have monthly transactional data on several markets (some were selected in a previous analysis to have the promotion). I am now trying to show that the markets that had the promotion had a lift or performed better than the markets that were not selected for the promotion. I ran a time series analysis model by market and now have both actual and predicted values for each market. I was thinking about using the predicted values as a baseline and use some calculations to show if the actual values had a lift over the baseline or not but I am not sure if this is the best way. Does anyone have any suggestions? Thanks, ------------------------------------------- Kenita Hall Student -------------------------------------------

I would respectfully disagree.  Complexity is simply a degree of freedom issue.  Although one might attempt a few time-series models, the choice of the appropriate model typically involves only 3 or less parameters (df).  On the other hand CHAID or the decision tree approach, are very large d.f. modeling procedures.  They requires very, very large data set to allow any generalizability.  Furthermore, the time-series approaches, either the Voelkel or Box and Tiao, directly answer the research question.

My own experience with CHAID and decision tree approaches would suggest that one should always hold a good proportion of the data out for cross-validation or do some jack-knifing procedure.  Simple, I think not.

-------------------------------------------
Allen Fleishman, PhD
Allen Fleishman Biostatistics Inc.
-------------------------------------------







Original Message:
Sent: 01-13-2011 11:50
From: Mansour Fahimi
Subject: Post Promotional Analysis

I suggest considering simpler options, such as using a decision tree approach or CHAID. While time series methods have their moments, in my experience, the extent of complexity they introduce does not always justify the extra light they may shed on the topic.

-------------------------------------------
Mansour Fahimi
VP, Statistical Research Services
Marketing Systems Group
-------------------------------------------

<sup>I have to agree with Allen on this.  If the objective is a nonparametric approach to regression or classification then trees might be a tool to use.  But if your data is a time series then at least you should be looking questions of stationarity and the autocorrelation or periodic nature of the data.  It is okay to replace complex tools with simpler ones is the simpler ones can give adequate (less accurate maybe) results But you don't pick a simpler tool just because it is simpler.  Chances are it will give the wrong answer or not even address the underlying question.  Forget for the moment that trees are the wrong tool simple tree algorithm like CHAID are inadequate.  CHAID is really outdated because it was before cross-validation and pruning which were the key ideas  of Breiman , Olshen and Stone that made trees workable.  CHAID trees would tend to be too big and would not be good for prediction and estimation while CART.  Now added complications like bagging and boosting improve CART.  Ed George and his coauthors have come up with BART.  Maybe someday we will come up with the ultimate classifier and call it SMART.

The bottom line is first get the right tools. If you have a correct tool that is very simple it may not be adequate and so additional complexity is neede to make it adequate.</sup> 



-------------------------------------------
Michael Chernick
Director of Biostatistical Services
Lankenau Institute for Medical Research
-------------------------------------------






Original Message:
Sent: 01-13-2011 13:52
From: Allen Fleishman
Subject: Post Promotional Analysis

I would respectfully disagree.  Complexity is simply a degree of freedom issue.  Although one might attempt a few time-series models, the choice of the appropriate model typically involves only 3 or less parameters (df).  On the other hand CHAID or the decision tree approach, are very large d.f. modeling procedures.  They requires very, very large data set to allow any generalizability.  Furthermore, the time-series approaches, either the Voelkel or Box and Tiao, directly answer the research question.

My own experience with CHAID and decision tree approaches would suggest that one should always hold a good proportion of the data out for cross-validation or do some jack-knifing procedure.  Simple, I think not.

-------------------------------------------
Allen Fleishman, PhD
Allen Fleishman Biostatistics Inc.
-------------------------------------------







Original Message:
Sent: 01-13-2011 11:50
From: Mansour Fahimi
Subject: Post Promotional Analysis

I suggest considering simpler options, such as using a decision tree approach or CHAID. While time series methods have their moments, in my experience, the extent of complexity they introduce does not always justify the extra light they may shed on the topic.

-------------------------------------------
Mansour Fahimi
VP, Statistical Research Services
Marketing Systems Group
-------------------------------------------