12

There was a recent "poll" here on Meta.TeX.SE asking about the use of generative AI in answers. Read all about it here:

Should AI answers be banned?

The results from that discussion were mixed. However, current network policy is to disallow AI-generated answers. While there's no effective way of moderating this (as there's no guarantee of always getting true positive/negative evaluations from posts by heuristics), content moderation of such answers is allowed (see the discussion under the heading AI-generated posts in Moderation strike: Results of negotiations ; also the accepted answer here).

The consequence is a request from Stack Exchange to have a per-site discussion on whether or not to place a banner at the top of the answer box stating that AI-generated answers are not allowed. A pilot was recently graduated on SO; here is a visual of the banner:

enter image description here

The current proposal, if the community wants this banner, is to have one of the following options:

We will initially offer two banner text options that all sites in the Stack Exchange network can opt-in to. Those options are the following:

  • Reminder: Answers generated by artificial intelligence tools are not allowed on TeX - LaTeX. Learn more

  • Reminder: Answers generated by artificial intelligence tools must be cited on TeX - LaTeX. Learn more

Both options have a "Learn more" link, which will point to a help center article, whose contents should also be a part of the community discussion - this article should explain what the site's policy on AI-generated content is (here's SO's article, as an example).

The banner will display once users select the answer field, removing the ability to show the "new contributor" banner. All users will see this banner when posting an answer with the option to dismiss. Once dismissed, logged-in users will not see this banner again.

For Action

  • Participate by casting a vote to this question either in favour of such a banner (up-vote) or against it's display in the answer box (down-vote).

  • If you'd like to add a general item of discussion on this topic or propose content related to a specific help center article, please post an answer below.

If there's no real success in this endeavour (minimal voting, a close-to-zero score with plenty of votes, or complete community disinterest with a large negative score, for example), the status quo will probably remain.


Update (Feb 10, 2024)

The response to this question has very low. A total of 17 votes (14 in favour and 3 against). That doesn't sound like sufficient evidence to make a decision either way.

Regardless, StackExchange has made the decision to roll out a network-wide Help Center article that will become available as of February 14th, 2024 (see Proposed changes to Help Center articles to include mention of AI-generated content policy). This article will contain the default messaging related to AI-generated content as mentioned in the Code of Conduct, which is that it should be cited (see Is uncited LLM usage considered a CoC violation under the Inauthentic Usage policy?). Sites can update the article to suit their individual needs/recommendations.

Debate around the activation of a banner is still open as "no site will have either banner on by default".

9
  • 3
    Can you give us some feeling on how big the problem on tex.se is? I'm aware of one user who serial posted AI non-sense at the start of 2023 and maybe one high rep user who use it (sudden deviation from his otherwise unique language pattern in an answer, but might also just be plagiarised instead of AI generated). Are there are a lot more cases? Jan 13 at 15:48
  • 1
    @samcarter_is_at_topanswers.xyz: Good question! As moderators we don't have any additional information on this. Discussions around various heuristics that could test for/alert on this seem to be comparably good/bad at identifying AI-generated content compared to a human's interpretation. Perhaps, because of this mixed result on efficacy, the onus has shifted to pre-emptively post such a policy reference in the answer box. On TeX - LaTeX thus far, active users on the site have made their own judgement calls on whether posts are human/AI-generated and then flagging them.
    – Werner Mod
    Jan 14 at 19:25
  • 9
    The second banner doesn't make sense. I assume it means that the AI tool must be cited, but what it actually says is that AI-generated answers must be cited. That doesn't say anything about what must be in the AI-generated answers themselves.
    – cfr
    Jan 15 at 4:15
  • On the substantive issue: if we don't use one of these banners, does that mean people should no longer flag answers they think are AI-generated because such answers will be acceptable with or without citation? Or is there some existing policy-basis for the flags? Frankly, I don't like either banner. I'd prefer something to the effect that purely AI-generated answers aren't acceptable and AI-generated content should be identified and cited. Aside from anything else, if questions include AI-generated code, how can they be answered without including AI-generated content?
    – cfr
    Jan 15 at 4:22
  • 2
    So reading David's answer the banner text should be: »Answers including especially LaTeX Code generated by AI are not allowed on TeX.stackexchange.com« including a link to an explanation: AI generated LaTeX Code nowadays (beginning of 2024) often contains low level code with errors, which are really difficult to spot. // If somebody with better English skills will improve this explanation, you are welcome!
    – Keks Dose
    Jan 23 at 11:15
  • @KeksDose the site is not offered any choice of banner wording, just a choice of using the wording suggeted or no banner at all, given that choice I would say no banner is far preferable. Jan 27 at 11:34
  • 1
    @cfr no the choice of banner or no banner is not a choice of policy. So we can choose a site specific policy but not a site specific banner, which makes no sense Jan 27 at 11:35
  • @DavidCarlisle Thanks. That's loopy. I'm against using either of these banners in this case. Actually, I'm against any banner in that case. I don't see the problem this is meant to solve. (On TeX SE, that is. May well be a problem elsewhere.)
    – cfr
    Jan 29 at 18:42
  • @cfr get all your friends to agree and vote down then:-) so far there are only two down votes... Jan 29 at 19:01

3 Answers 3

8

I voted down (no banner) the network policy of allowing each site to make its own policy, but only offering a choice of one of two fixed texts for the banner makes little sense.

There are many levels of AI use and it doesn't make sense to ban everything. Chatgpt for example is very good at English but not that good at TeX. If someone for whom English isn't a first language writes a good technical answer but then asks chatgpt to re-write the English prose should that be banned? Is it logically any different from writing the answer in French then getting google translate to write it in English?

The second banner intends I think that any such answer adds "written with the help of ..." although as @cfr commented, what it actually says doesn't make sense. So that would be more or less acceptable, but not reliably enforcable and serves no practical use. A bad answer is a bad answer whether or not AI was used to generate it, and similarly a good answer is a good answer. I can't see that it matters how the answer was constructed.

8
  • 1
    This also depends on the volume of AI-generated posts I guess (also asked by samcarter in a comment). If that is small then the community can check the quality of each post, point out issues, offer alternatives etc. However, if the amount is increasing then we could use a banner, because it would not be feasible to check everything, especially since an AI-generated answer often looks plausible and it takes time to find the mistakes (if any).
    – Marijn
    Jan 22 at 13:06
  • 1
    Regarding the spelling assistance/translation versus code generation: I guess here the idea is that it is easier to explain and enforce a total ban than to say which parts of a post are allowed to be AI-generated/improved, which checks are expected of the poster, what attribution/reproducibility looks like etc. Again that depends on the volume of such posts, if there are not too many then we can probably allow for some forms of AI-generated content.
    – Marijn
    Jan 22 at 13:12
  • 1
    the fact that it's easier to enforce a total ban doesn't mean it's a good idea. I see nothing wrong with a technical answer that's AI generated if it's right. A correct answer from chatgpt is better than a wrong answer from a human even if the chatgpt answer was necessarily a lucky guess. @Marijn Currently a chatgpt answer is highly unlikely to be right or tested for tex answers as it hasn't been trained that way but that doesn't mean a language model couldn't be trained to give accurate answers to a large percentage of questions here. Jan 22 at 13:57
  • 1
    I agree that AI can be helpful, that was not the point. The point is that it is only possible to verify/curate if there are not too many such posts. If there are many then this is a problem, as it takes far more time to check an answer than it takes to produce it with AI. In that case a total ban is probably better to avoid being overrun by posts of unknown quality - some good, some bad, but there are not enough volunteers to tell which is which.
    – Marijn
    Jan 22 at 14:04
  • 1
    shrug, you could say the same about accepting answers from random human users on the internet. That's how the network works, it tolerates bad answers they just don't get votes or get negative votes. Having a blanket ban on posts that someone thinks might possibly be ai generated just increases the load having to police an unenforcable rule. Jan 22 at 14:10
  • @Marijn Why pick AI-generated posts in particular? Copy-paste answers are also quick to produce and often of equally unknown quality (depending on where they're copied from), but we don't ban those.
    – cfr
    Jan 22 at 14:10
  • 1
    @cfr because copy-pasting an answer from somewhere either takes time to tailor it to the question or it is immediately clear that the answer does not apply/is copied from another answer on the same question/has other problems. That is the difference with AI generated answers, those are quick to produce and look appropriate for the question.
    – Marijn
    Jan 22 at 14:15
  • @Marijn Perhaps you can immediately identify such answers, but I doubt I'm the only one unable to do so. In any case, the site is setup to filter answers by votes. If people looking for a solution find the answer doesn't work, they'll not upvote/will downvote. Besides, the problem is currently non-existent on this site, so banning potentially good answers is a cost with almost no benefit.
    – cfr
    Jan 22 at 14:26
5

I think people are asking the wrong question. The question isn't whether to remind people of site policy on AI-generated content. It is whether to single this policy out for special treatment.

One way to convey the importance of a message is to repeat that message again and again. If a single message is repeated every time somebody writes an answer the implication is clear: this message is extremely important. If the message is repeated over and over to the exclusion of all others, it must be of overwhelming importance.

A better question might be: Is TeX SE's policy on AI-generated content one we want to draw particular attention to? It's not enough to think the policy matters. Do we want to give this degree of priority to this policy as opposed to any other?

I think the answer is no. I see no evidence that violations of TeX SE's policy on AI-generated content are causing problems which justify prioritising that policy, let alone doing so to the exclusion of all others.

1

I share the same opinion as @marjin in the comment replies to this post by David Carlisle on the current question.

Human-written bad answers have certain characteristics given below that are immediately obvious:

  • Very short
  • Out of context (if copied and pasted)
  • Easy to judge the direction in which the person is going

If human content is any longer than a paragraph, it requires some effort to make it at least appear meaningful. This is not the case with AI-generated content, where:

  • It is very easy to generate long answers
  • The answers almost always seem relevant to the context at hand
  • Even if no solution exists for the current "thing", AI can still generate a solution by taking components from other things and adapting them to the current context, making it difficult to determine if it will actually work or not

If an answerer is willing to put in the effort to write a good answer, per can certainly use AI for perself and use those hints to form per's own answer. In this case, having the citation guidelines (the second proposed option above) will help, requiring citation for:

  • model* (e.g.: OpenChat Aura)
  • client* (e.g.: https://chat.lmsys.org)
  • prompt* (e.g.: "How to accomplish this using context typesetting?")
  • parameters*:
    • temperature: (e.g.: 0.2)
    • top P: (e.g.: 0.2)

However, if there is no deterrent to AI answers, the positive use (the latter point) will be far outnumbered by poor quality, unverified mass AI postings.

Other factors that don't help in motivating quality AI answers include:

  • SE (Stack Exchange) does not allow flagging answers based on whether they are correct or not, which doesn't help with this situation.
  • Most popular LLMs (Large Language Models) do not provide options (temperature, top-p, etc.) to tune the generation, and many people don't know or use them either, resulting in more "creative" answers than "definitive" answers.
  • Even if the generator is tuned to "definitive", the "mold other related things to the current one" aspect still applies.

Update: The comment from @cfr motivated a very nice summary:

The Problem is that the answers from AI can't be easily determined if they are problematic or not; and require good amount of effort.

The Enforcement is to eliminate direct AI answers, whether problematic or not.

The Idea is that the non-problematic AI answer is substantially vetted by the answerer perself before posting for fear that if done otherwise "guidelines explicitly warned", so "downvotes", "negative comments", and "mod actions" etc.

- cfr "If problem is "can't spot problematic answers", how's the rule¹ supposed to be enforced? Is the idea that [it₁] serves a purpose [regardless]?"

3
  • If the problem is that we can't spot the problematic answers, how is the rule supposed to be enforced? Is the idea that the rule serves a purpose even though it can't be enforced?
    – cfr
    Jan 26 at 3:53
  • hi @cfr the problem is that the answers from AI can't be easily determined if they are problematic or not; and require substantial effort. Enforcement is to eliminate direct AI answers, whether problematic or not. Idea is that the non-problematic AI answer is vetted by the answerer perself before posting for fear that if done otherwise "guidelines explicitly warned", so "downvotes", "negative comments", "mod actions" etc. > - cfr "If problem is "can't spot problematic answers", how's the rule¹ supposed to be enforced? Is the idea that [it₁] serves a purpose [regardless]?" Jan 26 at 6:57
  • 2
    If we had a reliable way to identify AI generated answers, your argument would have some merit and it might be a question of weighing the loss of good answers against the harm of bad ones. But we don't have a reliable way to identify AI generated answers. So there's no reason to think the negative consequences you mention will be targeted at AI generated answers.
    – cfr
    Jan 27 at 3:36

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .