I agree with Gretchen. There has been a shortage of substantive questions, but I
enjoy the more social dialogue as well. To restore balance, I have a statistics
question. I am working with a client to assess trends in estuarine water quality for a
30 year record of near monthly data at a number of locations. For this we are using a
generalized additive model (gam) with independent variables that include smooth
terms for long term trend, seasonal cycle, and sometimes other forcing functions such
as fresh water input to the estuary. The dependent variable is typically a measure of
eutrophication such as total nitrogen concentration (TN). In the early part of the
record, many of the dependent variables were censored by detection limits so that if a
water constituent was observed to be below a limit of detection, the data were left
censored at the limit of detection. Because some constituents such a TN are
computed as the sum of nitrogen constituents, this can also lead to interval censored
data. To accommodate these censored data, we have implemented the
expectation-maximization (EM) algorithm to obtain maximum likelihood parameter
estimates (mle), but there is a wrinkle. The gam procedure that we use ( the
r-package mgcv) uses an optimization that is constrained for smoothness rather than
a global optimization. I have two questions. 1) Is anyone aware of other
applications, with citations please, of the EM algorithm where the optimization step
is a constrained optimization rather than a global optimization? 2) Are there tools
available for obtaining inference on the parameter estimates for a constrained EM
process? For example, is it possible to inflate the standard errors obtained from the
last iteration gam fit to adjust for the uncertainty of some of the data being censored?
Elgin S. Perry, Ph.D.
Statistics Consultant
377 Resolutions Rd.
Colonial Beach, Va. 22443
ph. 410.610.1473