Discussion: View Thread

  • 1.  warning multinomial analysis spss

    Posted 01-19-2015 04:55
    What about this warning that appears in multinomial analysis?


    ...Warnings
    Unexpected singularities in the Hessian matrix are encountered. This indicates that either some predictor variables should be excluded or some categories should be merged.
    The NOMREG procedure continues despite the above warning(s). Subsequent results shown are based on the last iteration. Validity of the model fit is uncertain.

    -------------------------------------------
    Efthalia Massou
    PhD candidate - Researcher
    Panteion University of Social and Political Sciences
    -------------------------------------------


  • 2.  RE: warning multinomial analysis spss

    Posted 01-19-2015 12:14
    There are a couple of possibilities.

    This warning will be produced when there is a category of the dependent
    variable for which one of the predictors is constant. If this is the case, you
    can diagnose the problem by examining the regression coefficients resulting
    from the last iteration, which are shown in the Parameter Estimates table. Look
    for a set of coefficients where the magnitude of the intercept is very large,
    and one of the predictor coefficients is also large, in the opposite direction.
    If there is only one logit where this pattern occurs, the category of the
    dependent variable used in the numerator of that logit is where the problem is
    occurring. If all sets of coefficients have this pattern, the category used as
    the reference category, which is used in the denominator in forming all logits,
    is the problematic category. You should find that for the value of the
    dependent variable identified, the value of the identified predictor is a
    constant. This results in the suggestion that you may want to consider
    combining categories of the predictor variable.

    It is also possible for this message to occur when the procedure is attempting
    to fit a model where quasi-complete separation is an issue. Increasing the
    number of step-halvings allowed during estimation (Maximum step-halving under
    Iterations in the Criteria dialog or the value for the MXSTEPS criterion on the
    CRITERIA subcommand) and beginning checking for separation earlier than the
    default 20th iteration (Check separation of data points from iteration under
    Iterations in the Criteria dialog or CHECKSEP keyword in syntax) may result in
    a message about quasi-complete separation instead of the message you saw.
    A quasi-complete separation means that for a subset of the data you're able to
    predict perfectly (complete separation means you can predict perfectly for all of
    the data).

    -------------------------------------------
    David Nichols
    Lead Statistician, SPSS Support
    IBM
    -------------------------------------------




  • 3.  RE:warning multinomial analysis spss

    Posted 01-19-2015 13:25

    And how do i continue? My results aren't reliable?

    -------------------------------------------
    Efthalia Massou
    PhD candidate - Researcher
    Panteion University of Social and Political Sciences
    -------------------------------------------





  • 4.  RE:warning multinomial analysis spss

    Posted 01-19-2015 21:23

    Kal'hesperas, Efthalia,

    At this point, your regression data are not as reliable as you want them to be. The reason is that the Hessian matrix used in the common Gauss-Newton regression is a trouble spot as a consequence of your data and your choice of variables (exactly why this is the case is hard to evaluate without knowing a bit more about your model and the intent of your analysis).

    David Nichols suggests two very useful general directions to think about: 1.) that one of the variables you identified behaves like a constant over the course of the regression steps and thus makes problems on the level of matrix inversion; 2.) that only a subset of your data works well with the regression model you selected, and some data are disruptive outliers.

    A good way of troubleshooting is to keep all your current variables in the beginning, but to simplify the model by reducing the levels of the response variables that you allow. First combine the two lowest levels of a response variable and run the regression, then combine the two highest levels and run the regression. Work your way through all your response variables systematically. If SPSS stops complaining, you have fixed your problem (few investigators are that lucky. . . ).

    Next, try to group one of the variables that changes the least in your data set with one that changes rather strongly, and see whether that fixes the problem.

    An easy but fairly uncommon reason for the error message would exist if the covariates allow prediction of your main variables so accurately that the variance is zero for all practical purposes.

    More discussion contributions are found on the web page http://stats.statsexchange.com/questions/10146/unexpected-singularities -in-the-hessian-matrix-error-in multinomial-logistic-regression.html

    If none of these strategies listed above helps, I would encourage you to think about an alternative iteration algorithm.The Levenberg-Marquardt method is quite successful, in problematic multinomial logistic regression as well. One reason for its popularity is that it can handle local nonlinearities in datasets, can be used to correct a non-invertible Hessian matrix, and may fix the root cause of your error messages with these two properties. The code for the Levenberg-Marquardt method is available in the SPSS Business Analytics Package (at least since 2011, according to the IBM flyer), and David Nichols would give you some guidance (thanks, David!). If your unversity has not purchased the Business Analytics Package for SPSS, you could use SAS or R software.

    There is even help relatively nearby: Prof. Lourakis at the Institute of Computer Science in Heraklion is a leading expert on the Levenberg-Marquardt algorithm and has written a .NET-implementation that might help you. You will find his (protected) email on his web site.

    Also have a look at the (free) introductory text on "Iterative Methods for Optimization" on the SIAM website(www.siam.org/books/textbooks/fr18_book.pdf), and perhaps on the textbook by Nocedal and Wright on Numerical Optimization.

    Hope this helps,

    ___________________________________________________________

    Gerhart Graupner, M.D., Ph.D.

    Graupner Consulting Services



    ------Original Message------


    And how do i continue? My results aren't reliable?

    -------------------------------------------
    Efthalia Massou
    PhD candidate - Researcher
    Panteion University of Social and Political Sciences
    -------------------------------------------