Discussion: View Thread

  • 1.  Chess cheating statistical detection

    Posted 10-04-2022 20:04
    Todays breaking news announcement of detection of cheating  "statistically extraordinary" in chess - using confidential statistical methods.

    The WSJ has a paywall https://www.wsj.com/articles/chess-cheating-hans-niemann-report-magnus-carlsen-11664911524
    excerpting (cut/paste)
    ....Chess Investigation Finds That U.S. Grandmaster 'Likely Cheated' More Than 100 Times
    An internal report reviewed by The Wall Street Journal alleges a previously unknown pattern of likely widespread cheating by Hans Moke Niemann, the player whose September victory over Magnus Carlsen has rocked the chess world
    chess.com has a cheat detection system, that uses statistics . however the details are confidential.

    and
    ...While we don't doubt that Hans is a talented player, we note that his results are statistically extraordinary."...

    Some sketchy details -It seems that chess.com might have a way to compare to chess moves from a sophisticated chess software program,
    Stockfish is a "strong chess program "  https://stockfishchess.org/

    Some additional technical details on statistical detection of cheating in chess

    https://cse.buffalo.edu/~regan/chess/fidelity/Methodology.html 

    and a reference to "chess.com statisticians"

    https://www.chess.com/article/view/chess-com-fair-play-and-cheat-detection

    ------------------------------
    Chris Barker, Ph.D.
    2022 Statistical Consulting Section
    Chair-elect
    Consultant and
    Adjunct Associate Professor of Biostatistics
    www.barkerstats.com


    ---
    "In composition you have all the time you want to decide what to say in 15 seconds, in improvisation you have 15 seconds."
    -Steve Lacy
    ------------------------------


  • 2.  RE: Chess cheating statistical detection

    Posted 10-05-2022 12:47
    I was reading the BBC news this morning and it seems like this may be snowballing?

    After chess, cheating rows rock poker and fishing - BBC News



    ------------------------------
    Mary Kwasny
    Professor
    ------------------------------



  • 3.  RE: Chess cheating statistical detection

    Posted 10-05-2022 15:21
    Another article (About Online Chess Cheating) from chess.com discusses a bit more about their methods, though it is still very vague. It goes beyond accuracy (from what I gathered, how many times players move matches a bot of the same player level), because if an opponent makes a mistake early, sometimes the best move is easy for rookie players to find. 

    The article points to other third-party for cheat detection (GitHub - MGleason1/PGN-Spy: Chess game analyser for detection of cheating)... might be a good place to start for those looking to dive into the topic more.

    There is also more sophisticated cheating software based on AI, which tries to emulate "human" play to evade cheating software (one example: Maia Chess)

    ------------------------------
    Bryan McComb, PhD
    Manager, Biostatistics
    Pfizer
    ------------------------------

    Original Message


  • 4.  RE: Chess cheating statistical detection

    Posted 10-06-2022 13:31

    A report from chess.com about cheating is now available. This report appears to have been issued in response to the reporting in the media. There are minimal technical details about the statistical methods. The article below has a link to the report. I included a link to a duplicate copy of the report on my webpage.

    https://www.nbcnews.com/news/us-news/us-chess-grandmaster-likely-cheated-100-online-games-investigation-fin-rcna50972

    www.barkerstats.com/PDFs/ASA/CNSL/Oct.-2022-Final-H-Niemann-Report.pdf

    excerpting some of the details of the "strength score" and reference to a "chess engine"  from (adobe physical page) 9 of the report

    ...Chess.com utilizes sophisticated statistical methods that dig deep into
    the probability that any individual player could have achieved their results based on past performance.
    Our detection system requires robust methodologies beyond simply looking at best moves, player rating,
    and centipawn loss. To effectively identify the vast majority of cheating, Chess.com computes an
    aggregate Strength Score. Strength Score is a measurement of the similarity between the moves made
    by the player, and the moves suggested as "strongest moves" by the chess engine

    ------------------------------
    Chris Barker, Ph.D.
    2022 Statistical Consulting Section
    Chair-elect
    Consultant and
    Adjunct Associate Professor of Biostatistics
    www.barkerstats.com


    ---
    "In composition you have all the time you want to decide what to say in 15 seconds, in improvisation you have 15 seconds."
    -Steve Lacy
    ------------------------------

    Original Message