24

Background: I've flagged over 1,200 answers I believed were ChatGPT/AI, with only a handful of declines (12, I believe) and some where the moderator decided not to take action. Since the strike, I've commented on hundreds more, as nicely as I can, with comments that have been reviewed (and even reworded somewhat) by a moderator. While I don't always get it right, I feel I have a pretty good eye for it.

Every so often, we'll have someone who wants to know why (the collective) we think something is AI-based. Often, this isn't even the person who posted the suspected-AI content. And all too often, I see others in comments answer the question, explaining exactly what the post contained that indicates that it might be AI-based.

Why is this a bad idea, and what should we do instead?

16
  • 2
    "Ironically, the comments that ask this often don't even claim not to be using AI, just using some "fuzzy wording" to not say they are or aren't." I've seen a similar sentiment on another meta post recently and honestly this feels very, very wrong to me. People have no obligation whatsoever to deny or confirm arbitrary suspicions in specific ways to satisfy the accuser's taste in rebuttals. If someone were to direct an outlandish accusation at me, "Why would you think that?" – and various less pleasantly worded variants – very much would be intended to challenge the accusation. Commented Jul 14, 2023 at 13:58
  • 3
    @MisterMiyagi I'm not talking about responses that simply ask "Why" - That's fine. I'm talking about replies like "I know [xyz tag] well" - It doesn't say they aren't using it, but it tries to make you think they are saying they aren't using it. Commented Jul 14, 2023 at 14:03
  • 11
    And yes, under the current policy, they do have an obligation to say if they posted something that came from ChatGPT or any other source. That's the plagiarism policy, and it long predates the AI policy. Commented Jul 14, 2023 at 14:04
  • 4
    Those comments perfectly underline why I feel this is critical. Both assuming they are trying to trick you and that they have plagiarised and are obligated to disclose that is presupposing that they are guilty. People have absolutely no obligation to disclose that they did not plagiarise. Commented Jul 14, 2023 at 14:10
  • @MisterMiyagi Fair enough - I'm going to edit that part out of the question, as it was a tangential anecdote anyway and not important to the main point. Commented Jul 14, 2023 at 14:18
  • "we'll see someone want to know why" Could the question link to some of these? I don't see them very often if at all and I want to make up my own mind about the possible intention of such comments. Commented Jul 14, 2023 at 22:37
  • @NoDataDumpNoContribution Not coincidentally, there's an example today in another Meta post that prompted this post. Quite a few people there have posted in answers or comments why they thought the answers were AI-based. The first second comment under that question has a link to a deleted question with an exchange Commented Jul 15, 2023 at 1:43
  • Maybe not the best example of a pure "Why do you think this is GPT" comment, but finding comments is a needle/haystack problem, unfortunately. I have over 3,300 "likely GPT" answers in a "Saves" list, and finding those "every so often" ones after they've passed is tough. I'll try to remember to link the next one here. Keep in mind, I'm not saying that all such questions have bad intentions, but that even if the intentions are benign, providing that information publicly can be then used by someone else. Commented Jul 15, 2023 at 1:44
  • 3
    @NoDataDumpNoContribution Another one in a different form. The user asks "why the downvotes?", when I'd already posted the AI policy on 10 (some deleted) of the user's previous questions over the last month, so they likely already knew why the answer was getting downvotes. Commented Jul 15, 2023 at 4:58
  • Out of curiosity, I asked ChatGPT this exact question. I was going to post it as an answer, but I'm about 100% sure it would be deleted b/c it is AI generated. Commented Jul 16, 2023 at 23:27
  • 1
    Why would anyone want to engage in coments about downvotes or flags? And why should generative AI break the consensus? The use of flags and downvotes is a widely debated topic. We have been discussing their combination with automatically generated or customized comments for years. While flags and downvotes should be used appropriately, it is generally advised not to engage in commenting about those. Commented Jul 17, 2023 at 7:04
  • 1
    @MetroSmurf Of course, the ChatGPT model training doesn't include any information on the current Strike, or that a Mod has said that commenting actually might be a good course of action while the strike is ongoing. Commented Jul 17, 2023 at 10:30
  • 2
    @DragandDrop Agreed that, in the normal operation of the site, flagging anonymous is the best action, and was my preferred action for many months. However, with SE's change in policy, the moderation team can no longer do anything about it, and subsequently (as I think most should be aware) went on strike. In the meantime, comments have been recommended by the moderators as a polite warning both to the user posting the information as well as to those reading it. Commented Jul 17, 2023 at 10:32
  • 1
    "Every so often, we'll have someone who ..." blethers in comments. Ignore comments. Keep up the good work flagging chatgpt (and ignore comments)
    – Fattie
    Commented Jul 17, 2023 at 11:54
  • Humm, I will comment in Mod flag. And to be Frank, I won't comment on post. Because those comment will stay for years, and will trigger serial downvote. Just flag them, and don't try to be the next Sarah Connors, cause those AI post will eventually be worthy user with rep. And the down vote will be back. Commented Jul 17, 2023 at 11:57

2 Answers 2

30

Commenting at all isn't productive. One of two things happens: you falsely accuse someone of violating existing policies, publicly, in a way likely to attract further action from other users who may not be as restrained as you are resulting in unchecked serial voting or worse, or you make it obvious to the user that their current methods of trying to avoid detection are failing prior to their actions being properly dealt with by a mod.

A false accusation followed by retaliation from others through serial downvoting can't be easily cleaned up; leave these exceptions to the mods. Raise a flag, cast a downvote if you think the answer is wrong or low quality.


I do understand the seeming need to inform the OP and future visitors that a post is GPT generated, however, unless you actually see something wrong with the answer or something likely to inject vulnerabilities... the possibility that it's gpt generated isn't relevant to anyone other than mods, the people who can do something about it (assuming we get out of the current mess while retaining that ability.)

3
  • 16
    As several people pointed out in reply to your answer on another related Meta question here, a Moderator has specifically said that, "there may be value in posting comments now, in order to warn your fellow users, when you would not have done so under normal circumstances when SO's legendary moderation is in play" Commented Jul 14, 2023 at 14:50
  • 2
    "Replying at all isn't productive." - do you refer to the first comment that points out that something looks like AI-generated content, or do you mean replying to a comment by the author of the answer where they deny AI use?
    – Bergi
    Commented Jul 15, 2023 at 20:24
  • 3
    the former. The best leaving a comment pointing out something looks like gpt can accomplish is vigilante justice, only mods can actually take the action that is needed. Informing these users that they're failing to avoid detection also helps them improve how they are avoiding detection.
    – Kevin B
    Commented Jul 16, 2023 at 2:34
22

tl;dr Don't spill the beans, or some bad actors will just avoid those things when using AI in the future

Related reading: @Makyen's answer to "How can we determine whether an answer used ChatGPT?"

In my experience, we have several categories of users using AI to assist (entirely or partially) in writing answers to questions. To recap from another post I made on MSE:

  • Users who have used it (mostly) responsibly. The current policy for Stack Overflow is still that this is not allowed (but now impossible to enforce, of course).

  • Then we have users who I honestly believe are trying to be helpful in answering a question, but they don't understand the perils of unvalidated AI output, and post without verifying (and often not even fully understanding) their answer. This is not responsible use. Most of these users will stop when the policy is pointed out, but some simply don't agree with the policy and will continue. These then, become "bad actors" (a term that means the person isn't acting in good faith).

  • Then we have the "bad actors" who are simply attempting to "farm rep" for one reason or another (perhaps even for spam purposes). This is also not remotely acceptable, as the trust and reputation earned on SE should represent the community's assessment of the user's expertise. Some of these users are doing it without knowledge of the policy, and stop when it is pointed out, and others, sadly, have no regard for the rules.

  • And finally we have spammers of various sorts, either explicitly in the answer or through profile links. Clearly these are also "bad actors".

The problem we have here is that the bad actors, given knowledge of how we identified something as likely ChatGPT/AI, will simply use that information to modify their answers in the future to attempt to evade detection. Since bad actors are almost always new users, they'll often simply delete their account and start over with a new one that doesn't have the "tarnish" of the old, already identified account. Of course, moderators have tools to help identify this (after the strike is over, of course), but I believe they generally have to look manually.

Even if the user is asking in good faith, others who aren't so well-inclined might see the info and use it more maliciously.

It may be tempting to want to "prove" you are right to the user, but telling them why you identified it as likely-AI is just exposing us to more potential ChatGPT which will be harder to detect in the future.

So what to do?

A standard response could be something like:

I really do want to believe you here, but one of your answers included information that is very rarely done by a human author, but quite often done by ChatGPT. Then you had another 6 answers that also had things we see often from ChatGPT. It just seems unlikely that this is all your content. Again, if you used AI to assist with any answer, I would encourage you to delete it.

That keeps it vague enough to not give anything critical away. Also, absolutely do re-evaluate your stance, back-down, apologize if you think you might be wrong at second glance. I tend to have varying levels of responses based on how confident I am that the content is from ChatGPT/AI.

Another possible reply that I've used is something like:

Unfortunately I don't go into specifics on the indications of why this might be AI-based. We do have some folks here who appear to attempt to hide their AI use, and they might use that information to do so.

I don't consider this "security through obscurity"; just trying to avoid giving out the password in comments ;-).

16
  • 10
    So the recommendation is not telling people why something is suspected to be AI generated... by vaguely telling people why something is suspected to be AI generated? Commented Jul 14, 2023 at 13:49
  • @MisterMiyagi I don't think the "vaguely" here gives anything away. Unlike folks in comments in this question that are being very specific about the things the user did that look like GPT. The "vaguely" answer here is just side-skirting the question. Obviously they know they did something in one of their answers to give it away - They just don't know exactly what in most cases. Commented Jul 14, 2023 at 13:56
  • 6
    If you want to write an announcement / PSA kind of post, I'd strongly suggest you leave your personal opinion out of it. Stick to the facts...
    – Cerbrus
    Commented Jul 14, 2023 at 14:00
  • 3
    @NotTheDr01ds But what's the intention there? I mean, you are either vague enough to not disclose anything or, well, not vague enough and still disclose something. If you don't want to disclose anything… why reply at all? This seems to just needlessly string along folks. Commented Jul 14, 2023 at 14:00
  • 6
    @Cerbrus As I said in the answer, I fully support the responsible use of AI here. But no, an AI answer that looks human but still isn't validated and backed by human-expertise is still a useless AI answer. Commented Jul 14, 2023 at 14:11
  • 2
    @MisterMiyagi Not responding at all is clearly okay as well! Just don't give specifics either way. Commented Jul 14, 2023 at 14:12
  • 2
    @NotTheDr01ds "Not responding at all is clearly okay as well!" Not responding at all is clearly not being recommended by this answer. Commented Jul 14, 2023 at 14:24
  • @MisterMiyagi Which do I recommend? I recommend giving a vague reply or let them know why you won't tell them (which I'll edit in as well). I don't consider it "stringing them along" -- I am trying to assume that there's a chance that I might be wrong about it being AI-based, so I consider a "vague" response more polite than just not replying at all. But I'm not going to fault someone else for not replying. Commented Jul 14, 2023 at 14:30
  • 9
    Imo this is bound to frustrate users suspected of using GenAI even more. Better not comment at all about why you think something than this vague comment containing nothing actionable imo.
    – Erik A
    Commented Jul 14, 2023 at 14:35
  • 4
    @NotTheDr01ds Sorry, I am confused now. You've styled this up as a PSA and request for people to act a specific way, so what you show in the answer presumably is what you recommend to/request from people, isn't it? Or is this really just intended to show your standard response? Commented Jul 14, 2023 at 14:37
  • 1
    I don’t support any use of AI, specifically, ChatGPT or OpenAI as I have both seen the garbage it’s generated attempted to generate content for experimental purposes to confirm the output is absolutely garbage. So I will start to vaguely tell users their content is AI generated but I will also downvote those contributions. Commented Jul 15, 2023 at 3:04
  • 3
    Frankly, if you’re using an indicator that can easily be thwarted just by knowing what it is, it’s a bad indicator. The biggest tell for AI-generated text is that it is blind to context a human would understand immediately, and if someone wants to get around that, they’ll need to make significant advances in the field of genAI or actually write most of the post themselves. I’m happy to be the catalyst for either of those options.
    – ColleenV
    Commented Jul 15, 2023 at 10:25
  • 3
    @ColleenV That an indicator can be thwarted by the author knowing what it is and making changes doesn't make the indicator bad. Calling such "bad" is, at best, a very limited view. Such an indicator can, in fact, be a good, or even excellent, indicator. It could be something that, if the indicator is tripped, then there's a 99.99% chance that the thing detected is a real true positive. Such indicators routinely don't detect all real true positives, which is why many indicators are typically used.
    – Makyen Mod
    Commented Jul 16, 2023 at 1:45
  • 5
    Telling people who are trying to avoid detection what is looked for will result in some of those people spending the time to make sure they aren't triggering those things. Giving them that information does make it harder to detect what one is looking for, when there are people actively trying to avoid detection. For AI-generated content on SO, there definitely are people actively trying to avoid detection.
    – Makyen Mod
    Commented Jul 16, 2023 at 1:45
  • 4
    @Makyen The fact that it is easily thwarted and you have to maintain secrecy for it to work makes it a bad indicator in the sense that you’re only going to catch the laziest or least malicious posters. All keeping the secret does is add a short delay while the people actively trying to get past your screening figure out the you’re keying off of “Here is an example” or whatever. They will figure it out even without being told, and there are many already getting past you.
    – ColleenV
    Commented Jul 16, 2023 at 11:11

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .