Cool.

My problems are sequential.

My students always make the mistake of asking, "How many problems are on the exam?" I tell them, 10.... But there is part i to xiii on many.

Perhaps one of my departments should look into making their dept final exams sequential instead of "Find the mean of this data. Find the median of that data. Find the variance of a 3rd set of data."

Original Message:

Sent: 03-27-2023 11:14

From: Monnie McGee

Subject: Have you used Chat GPT to answer your stats exam questions?

A colleague of mine and I have played around with ChatGPT and exams. Here are the results:

So I ran (or attempted to run) all of the first introductory stats exam through ChatGPT.

First note that this had to be done question by question, and this was pretty time consuming. Several times the AI timed out, and I sometimes had to log back in again. In practice this would not be a very efficient method of cheating.

The exam has 30 questions. 25 are multiple choice, and 5 are true false.

__Multiple choice__: Two of the problems had pictures in them, which I couldn't figure out how to easily load into the AI. This fact in of itself could help to deter AI use.

One other problem was just too difficult to format for the AI, it was the 4^{th} of a series of problems referring to some tables, and in general the AI does not handle sequential problems very well. It is almost certain that if I had successfully loaded the problem, the AI would get it wrong.

Again, this provides another tip for discouraging AI use, make problems sequential rather than stand alone.

Anyway, of the 25 multiple choice problems, I was able to get the AI to try 22 of them.

In one problem, I tried 3 times and each time the AI timed out, so I marked that as wrong.

In the remaining 21 problems, the AI got 9 correct. Clearly the AI gets a failing grade on my multiple choice problems.

__True/False__ was a different story. I got the AI to respond to all 5 problems, sometimes very quickly, and the AI got 4 of the 5 correct. This is actually better than the performance of many of the students on T/F problems.

Overall, the AI got 13 correct of the 27 problems I gave it, which is an F.

So to discourage AI use in exams, I recommend the following:

- Include figures and graphs.
- Make questions sequential rather than stand alone.
- Avoid true/false problems.

Obviously these conclusions are based on only one exam.

Cheers,

Monnie

------------------------------

Monnie McGee

Associate Professor

Southern Methodist University

Dallas,TX United States

Original Message:

Sent: 03-24-2023 07:28

From: Robert Borotkanics

Subject: Have you used Chat GPT to answer your stats exam questions?

Good question! I asked it some questions, starting at the most basic and worked forward in increasing difficulty. Here are the results I attained:

1) Control question. I asked it what time it was. Did not know, but told me how I could find out.

2) Asked it to describe difference between mean and median. Response was correct.

3) Asked under what conditions should I use mean and median. Response was correct.

4) Asked if I should use the mean when my continuous data are positively skewed. Response was correct.

5) Asked if I could use Chi-squared statistic to do a between group comparison of continuous data. Response was correct.

6) Asked what the assumptions are for linear regression. Response was incomplete and out of scope (so incorrect). It provided an incomplete list of assumptions and how to interpret residuals.

Happy chatting!

------------------------------

Robert

Original Message:

Sent: 03-23-2023 13:02

From: Andrew Ekstrom

Subject: Have you used Chat GPT to answer your stats exam questions?

Hello Everyone,

I'm wondering if anyone here has used Chat GPT to answer questions on their stats exams?

Over the last 5 years, I have taught as many in person stats classes as I have online. I give my online students a few days to get teh exams done. My in person students get the class period. I've noticed that their grades tend to be the same.

This semester, I have given my students exams and asked Chat GPT for answers. The problems my students did well on, Chat GPT did poorly on. The problems my students did poorly on, Chat GPT did well on.

So, I'm wondering:

1) If anyone else has done this with their exams? If so, how did it do?

2) If my online students do just as well as my in class students, and clearly are not using ChatGPT or other sources for answers, should I be worried about them cheating or should I be more inclined to let my in person students do take home exams?

------------------------------

Andrew Ekstrom

Statistician, Chemist, HPC Abuser;-)

------------------------------