ASA Connect

Error is a hypothetical quantity. Residual is realized or estimated error. There is no such thing as estimated residual.

Residual = Observed - Expected (or Estimated from the model)  at each point on the independent variable under regression.  The sum of the residuals must equal to zero.  The residuals are not independent events and they are observable quantities.  

Error = Disturbance = Deviation of observed value from its true value (which is generally not observable). Often it is the population mean (or median from skewed distribution).  It is sometimes called statistical error.  The sum of this quantity is not necessarily zero and they are independent.

Estimated Residual = Probably a term that some statistical software uses for the residual.

In the context of regression (linear or nonlinear in parameters), residual is the correct term to use.  Residuals lend themselves for normality, homoscedasticity, and model appropriateness analyses.

Ajit K. Thakur, Ph.D.

Retired Statistician

Residuals sum to zero in a regression model if there is an intercept in the model, as would be the case in simple linear regression. The residuals may or may not sum to zero if there is not an intercept in the model - it depends on the predictor variables in the model and their relationship to one another (essentially whether or not there exists a linear combination of the predictor variables that is constant for all the cases in the data set).

Erick - 

I think 'estimated residual' may be used because it is the difference between the result from an estimated (never exactly correct) model and the observed.  I think a well-known friend insisted on "estimated residual," but many use "residual" for short anyway.
     
 
It's not really an 'error.'  You might call it a "prediction error."  In physical science, however, if the model is known 'exactly,' and the residual is really just a measurement error, then maybe we have a "residual" or "error?" 
    
Language does tend to evolve, and also be used differently in different subject matter contexts, which can make things appear sloppy:  (1) Sigma in regression is the standard deviation of the random factors of the estimated residuals, but I have heard that a well-known software system calls it a standard error.  However, sigma is still a standard deviation.  (2) In "statistical learning," I can see "residual" would make sense, as that discipline looks at bias-variance tradeoff from the "correct" model.  (3) The physical science application might be somewhat different.  Terminology can often be confusing.

Cheers - Jim 

If Y is the response variable in the regression, then for a given value X of predictors:

The error at X is the difference between the actual value of Y and the actual conditional mean of Y given X: Error = Y - E(Y|X).

The residual is the difference between the actual value of Y and the regression estimate Y-hat at X: Residual = Y - Y-hat.

Since Y-hat is the estimate of the conditional mean E(Y|X), then we can consider the residual to be an estimate of the error at X.

(Note that y-hat is also our estimate of Y.)

Thank you very much for your reply.