15

Some time ago, I asked how we should handle bug reports: Questions which are bug reports. The top answer at the time split questions into two

  • Those which are asked where is seems clear that the poster knows they are bug reports
  • Those where it becomes clear they are due to bugs or where this is 'obvious' to another user, but where the original poster did not know that

The suggestion then was that the first case get closed and the second get answered with

This is basically a bug in the package/program and as such, it should be reported using the way explained in its manual.

I've been prompted to raise this again following some discussion about making sure things are transparent to users, in particular new users. Looking over meta, I think we have some issues:

  • What has tended to happen 'in the wild' doesn't match the above (see Answer reveals problem is package bug: Delete question? for a statement that suggests a different policy, for example)
  • The nature of the closure types available to us has changed since the earlier discussion
  • There is a need to consider people other than the OP arriving via search: do they get useful information? In particular, will they pick up subtleties in comments (as opposed to actual answers)?
  • Is the speed of (likely) fixing a bug relevant?
  • What (if anything) is different for questions where the OP knows something is a bug but is seeking a workaround (and does this vary between things solvable in TeX/Lua code and those which need work on supporting binaries)?
  • What (if anything) is different between bugs which are fixed and bugs which are still 'live' (and of course does this have a knock-on if the bug is live when the question is asked but fixed X months later)?
  • How do such questions relate to formal issue trackers (where available)?

Given the above, I'd like to seek a clear position which can be used as a reference when handling these cases. This will be useful as a reference point and can be linked to as part of discussion of handling of individual questions.

2
  • Some examples that are not closed 79161 , 41828 , 215610
    – percusse
    Dec 6, 2016 at 14:39
  • 1
    @percusse Indeed: one of the things I noticed when reviewing the situation after discussion is that there's not necessarily a consistent position
    – Joseph Wright Mod
    Dec 6, 2016 at 16:03

3 Answers 3

Reset to default
16

One man's bug is another man's feature...

I don't think whether the OP considers it to be a bug is that relevant.

If the problem has been identified and fixed by the package author in an updated version of the package then we can consider treating the question by some special mechanism for "bug reports" otherwise it is best to assume that it's a permanent feature that may need a permanent workaround and an answer on the site is as good a place as any to detail that.

If the problem was accepted by a package author and it's already fixed so the OP just needs to update, I think it can be closed initially. (Note that in comments it has been suggested that an answer here is useful. If the question is answered with a code workaround if the package is not updated, then I'd agree (this is no different from if the question and been answered before an update became available) but if the answer is just "you have an old version, update your system" then to me just leaving that as a comment and closing seems OK, but if someone answers I wouldn't object)

If later the package gets fixed, answers can be updated (I do that for my own answers about my own packages (or the latex kernel) when I remember, but I don't delete the answer or vote to close the question)

Otherwise if the bug is in one of the many packages with uncertain or slow update mechanisms, having a workaround in the answer is a good thing and the question should be left open as normal.

9
  • 1
    What do you mean by: "If the problem was accepted by a package author and it's already fixed"? I'm not sure what context you are describing there, but I think if the bug exists in any version of the package, keeping the question open and answered is helpful to other users who might not realize they are using an outdated version.
    – Tyler
    Dec 5, 2016 at 22:27
  • @Tyler existing answers I didn't suggest closing, with my own i just add a heading at the top saying update fixed in 2015 release or whatever. But if a new question comes in describing a bug fixed in a 2015 latex release and the answer is just to use a current release, the question and its answer really don't have much use and may as well be closed (9 times out of 10 it'll be a duplicate of a contemporary question anyway)
    – David Carlisle
    Dec 6, 2016 at 8:04
  • 4
    I guess the key issue here is that statement 'really don't have much use': the point raised in discussion which I've tried to elude to in the question, and which @Tyler mentions, is that there will be people who still see such bugs and who may not know they need to update (or may not be able to, though that's more tricky), and who arrive at a question via search rather than posting themselves. For them, closure doesn't necessarily help unless it points to some useful source of information (closing as a dupe).
    – Joseph Wright Mod
    Dec 6, 2016 at 9:13
  • 2
    Could you elaborate on why you see “you have an old version, update” as being a close reason, rather than an answer? It’s a good solution to the OP’s problem. It looks like an answer, it quacks like an answer…
    – Peter LeFanu Lumsdaine
    Dec 8, 2016 at 14:00
  • 1
    @PeterLeFanuLumsdaine Oh because usually it's no use to the OP: people who are prepared to update usually try that before asking, so if they don't have admin access and admins won't update what they need is either a code workaround (which is hard to make an answer for once you no longer have the old code available) or instructions on how to install a single package locally from ctan, of which we already have several in depth step by step answers.
    – David Carlisle
    Dec 8, 2016 at 14:28
  • @JosephWright Don't people closing as 'off-topic' normally add an explanation which says an update fixes it? Then this ends up in the box highlighting the closure, so it is not just buried in comments somewhere. (Of course, this relies on people actually adding the comment, but that does happen most of the time - or does it not? Is that the problem?)
    – cfr
    Dec 9, 2016 at 2:29
  • Aren't there at least some things which are clearly bugs? For example, code taken straight from the manual of TikZ fails to compile at all or gives completely different output from that shown. Isn't that a bug by the lights of the package's author? (Even if somebody else considers it a feature.)
    – cfr
    Dec 9, 2016 at 2:37
  • Thank you for your answer: both this one and the one by E.P. are useful and I think 'closer together' than perhaps is immediately obvious. I feel a formal 'statement of policy' is needed so have posted my own answer, which I will likely accept in due course. I do hope this makes sense (and would welcome feedback on that answer.)
    – Joseph Wright Mod
    Dec 11, 2016 at 10:48
  • @JosephWright yes no argument with your answer, seems fine to me.
    – David Carlisle
    Dec 11, 2016 at 14:25
8

In general, questions due to bugs should be treated no differently from any others. Thus provided the question is clear and reproducible it should be kept open, and (ideally) answered.

For issues which are acknowledged bugs which have been fixed, it may be that the most straight-forward answer is 'update your TeX system' (for bugs in (La)TeX packages)/'update XXX' (for bugs in e.g. editors). This information constitutes an answer. Suggested text for updating the TeX system is

This behaviour is due to a bug in <name-of-package> which is fixed in the <date/version> release. Where possible, you should update your TeX distribution to install the current release. If this is not possible, for example if you want to change only this one package or where you do not have the privileges to update the entire distribution, you will need to install the package locally. This option should be considered a last resort, since for more complex packages there may be package dependencies that will make local installation more complicated and error-prone.

If a more detailed analysis of the bug (perhaps including a work-around and version(s) affected) is available this should also be part of the answer. If there are more details of the bug (and fix) in a public issue tracker then a link is appropriate.

Where behaviour appears to be defective and is not explicitly covered in documentation, it may be a bug. Likely examples are loading a package with particular options leading to excess characters in output, leading to a complete failure to complete the TeX run, or where code given in the relevant documentation cannot be reproduced. (A counter-case would be the fact that booktabs does not work with vertical rules: this is a documented design decision not a bug.) For such 'open' cases, a statement that the behaviour 'appears to be a bug' may be appropriate.

Where possible, any steps taken to report the issue should also be included, e.g. 'I have reported this issue by email'/'I have logged the bug as YYY in the issue tracker'.

5
  • although the date of an answer is present in the posting, someone reading the answer a year or two later may not bother to read the date. so it would be useful to specify which release -- version and date -- for future reference. (after all, sometimes bugs, like "the cat" in the song, come back.)
    – barbara beeton
    Dec 11, 2016 at 13:03
  • 2 minor points. I think there should be some degree of consensus in the 'appears to be a bug' case, unless the relevant author/developer has already recognised it as a bug (but not yet fixed it). Otherwise, the answer may be little more than a statement of opinion e.g. 'I don't like ...'. E.g. it would be very misleading to write this about vertical rules + booktabs, given the package author's views. Also '(La)TeX packages' is a bit confusing given the ambiguity of 'packages' in the case of LaTeX.
    – cfr
    Dec 11, 2016 at 14:48
  • More generally, I guess I'm not that convinced by the 'appears to be a bug' case in general, even though I think that there are some cases in which things really are fairly obviously bugs, even if the package author fails to accept that (or, more usually, has failed so far to say one way or another). Especially since many users regard lack-of-feature-X as a bug - after all, bug trackers usually encompass feature requests.
    – cfr
    Dec 11, 2016 at 14:56
  • @cfr I've tried to cover the 'might be by design' case but I'm wary that saying 'anything that the package author doesn't explicitly mark has to be regarded as a feature': there are a fair few things that are objectively bugs that we never hear back about from the original authors (the code could be many years old).
    – Joseph Wright Mod
    Dec 11, 2016 at 15:19
  • @cfr I've adjusted my suggested text to reflect the fact that trackers are for issues not bugs. Feature requests are I think covered reasonably by meta.tex.stackexchange.com/questions/2623/…
    – Joseph Wright Mod
    Dec 11, 2016 at 15:23
5

Thanks for posting this - it goes a long way towards addressing the concerns I raised in chat. This site is entitled to whatever customs it deems necessary to keep up the quality of the content (or, as I suspect many here would phrase it, to "not talk about the things we cannot fix"), but the site does owe a minimal measure of clarity in why it shuts threads down. This is both to the OP, but also to future visitors who arrive via search and are trying to evaluate whether this site will be helpful to their question or not. (In particular, such users will have no knowledge of meta, and no access to chat.)

That's as far as the transparency of closures goes: having an informal consensus is good, but past a certain point this site should have some appropriate go-to place where newer users can catch up with what the deal is. (In usenet parlance "lurk for a while and read the FAQ" only works if the FAQ is accessible, discoverable, and up-to-date, though I would question the requirement to "lurk for a while".) This thread is precisely that place.

On the other hand, I'd like to say I'm really weirded out by an attitude that I've seen several times on this site, including David Carlisle's answer here, which goes along the lines of

if a question asks about behaviour which can be reasonably attributed to TeX, but which turns out to be due to a bug in a related binary (such as MikTeX or TeXlive) which has already been solved, the question should be closed.

This strikes me as wrong-headed and entirely against the spirit of Stack Exchange. Again, this site's community is entitled to whatever standards the consensus feels are reasonable (as long as they are transparent enough), but I really disagree with this one. In short:

If the answer is "this behaviour is due to a bug in (X), which has since been patched; to solve the problem, simply update your system", then that is still an answer.

I would argue that the correct approach to this class of questions is to keep them open and provide the above as an answer.

There are multiple reasons for this.

  • The typical audience for a Stack Exchange page is normally much broader than just the original poster. Over a several-years lifetime, very many more people will see the thread, and find it helpful (or not).
  • Normally, this additional audience comes in directly from a search engine, and they will normally not be registered on the site. This means that arguments like "if people have more questions they can comment" don't actually work.
  • TeX error messages are normally very specific, which means that if I encounter that error, and the error message is reproduced in some question on this site, I'm very likely to end up there.
  • More generally, Stack Exchange pages tend to be ranked very highly on multiple search engines.
  • It is very common for many TeX users to run their distributions, packages, and associated software using very old versions. This is partly due to the outstanding stability of TeX, but it can also be because e.g. Ubuntu came pre-packed with texlive 2011 even in 2015 (or so). This means that even for "solved" bugs there is still a significant user base running software without the patch.

Ultimately, the approach in my first quote only serves to litter the internet with yet another useless search result that is simply not helpful. If the answer is simply "oh, I just need to update and everything will be fine", the Stack Exchange is the best place to be told that in a simple and clear manner. Closing those questions instead of answering them strikes me as being deliberately unhelpful to the majority of the question's audience.

15
  • We do sometimes "answer" a question in comments, and then it is closed. For visitors who are not registered here, they'll probably read that and take it as a possible solution and attempt it. No?
    – Werner Mod
    Dec 5, 2016 at 7:42
  • somewhat bemused that you single me out:-) I was suggesting that we close less often than we do now and already we close much less often than most stackexchange sites. Note I didn't suggest closing old answers, just adding a banner on them saying "package was updated in xxx release. For new questions that come in which have an answer "works for me with current latex, update your distribution" that's useful info to give the OP but not that useful as a long term resource, so I'd comment and vote to close normally.
    – David Carlisle
    Dec 6, 2016 at 8:55
  • Presumably your suggested text would include a link to the 'How to update' question. How do you feel the case where people have a 'fixed' TeX system should be handled? Something like 'If possible, you should update your TeX system [link]; if you cannot do that, you will need to carry out a local installation of XXX [link]' (XXX = package in question)?
    – Joseph Wright Mod
    Dec 6, 2016 at 9:28
  • @DavidCarlisle Here the focus is specifically on questions where there is an identifiable bug, even if fixed. That's distinct from a more general 'cannot reproduce the fault' situation, which I feel is reasonably clear.
    – Joseph Wright Mod
    Dec 6, 2016 at 9:53
  • @JosephWright yes but as I tried to say in my answer I don't think it should be identified as a bug unless it is acknowledged as such and by the package author. Otherwise it's best to assume that it is a permanent feature (that may need a permanent workaround)
    – David Carlisle
    Dec 6, 2016 at 9:56
  • @DavidCarlisle Ah, I see what you are getting at and entirely agree. However, it wasn't clear to me that was what you meant in your answer: perhaps you could add a version your comment above to your answer?
    – Joseph Wright Mod
    Dec 6, 2016 at 10:43
  • @JosephWright added some more text, can't say whether it's any clearer:-)
    – David Carlisle
    Dec 6, 2016 at 12:44
  • 1
    @DavidCarlisle I singled you out (purely) because your statement is on this thread ;-). I vaguely agree with your first comment, except that I don't see the need for a "banner": an answer will do perfectly fine. I also strongly dispute your claim that this is not useful as a lasting resource. (As an example, consider that Ubuntu, which is not the most keep-it-stable long-release-cycle minded distro around, came with texlive 2009 in the 12.04 version, and there are still people running that distro.) I just don't get the rationale for closing a question that has an answer (but it's up to y'all).
    – E.P.
    Dec 6, 2016 at 13:01
  • @E.P. as I just said in my expanded answer, I think it's OK to answer if you are answering (with code) as if the package hadn't been updated, but an answer that just says "you've an old release, update" seems pretty pointless as a long term resource. (but useful as a comment to the OP)
    – David Carlisle
    Dec 6, 2016 at 13:04
  • 1
    @DavidCarlisle I don't see the pointlessness. There are lots of people running outdated systems, and if all they need to do is update then all you need to do is to say that (in a place that's clearly labeled as the answer). Leaving that information out, or putting it in a comment (that might not even be immediately visible, possibly buried in a long conversation) strikes me as simply degrading the quality of that communication, and just about deliberately unhelpful. I do see, though, that my chances of changing attitudes around here are close to nil, so this is probably my cue to bow out.
    – E.P.
    Dec 6, 2016 at 13:08
  • @E.P. just to show every rule has an exception, I just answered a question about a bug fixed in 1989:-) tex.stackexchange.com/a/342988
    – David Carlisle
    Dec 8, 2016 at 8:03
  • TeX error messages are normally very specific - I'm obviously using a different system. Could you give me a link to the version of TeX you use? It sounds like a fantastic improvement!
    – cfr
    Dec 9, 2016 at 2:35
  • @cfr You got there a day before I did. Those "very specific" TeX error messages are like going to a doctor with an earache, and he tells you that you have otalgia. The problem is that due to the multi-package nature of TeX, the error message is often thrown by a package or core feature that the user does not understand, in arcane terms.
    – user103221
    Dec 10, 2016 at 2:43
  • Thank you for your answer: both this one and the one by David are useful and I think 'closer together' than perhaps is immediately obvious. I feel a formal 'statement of policy' is needed so have posted my own answer, which I will likely accept in due course. I do hope this makes sense (and would welcome feedback on that answer.)
    – Joseph Wright Mod
    Dec 11, 2016 at 10:48
  • @JosephWright I agree with your answer.
    – E.P.
    Dec 11, 2016 at 18:00

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .