11

For some time now I consider the review process broken, particularly the First Post queue, as too many review tasks are just rubber-stamped. This has been discussed on meta.so, and one possible solution is the Honeypot system.

My question: should we go for an installment of Honeypot here on TX.SX, or should that be dealt with on a more personal case-to-case basis by our moderators?

  • 1
    Could you say what your concern with the first-post queue is? It isn't one of the livelier clues, and I don't recall, offhand, what kind of trouble there is with it. Are newbies getting treated harshly? – Charles Stewart May 15 '13 at 7:45
  • 4
    @CharlesStewart As I said, too many first posts are rubber-stamped. No editing, no voting, no comments. If you follow the queue and see what speed is applied there by certain users, without any real reviewing, then it smells a lot like a bot. In particular when some users don't sleep, seemingly. – mafp May 15 '13 at 7:50
  • 2
    Ah, I see. The set of overly light reviewers is very small: I think raising this on a case-by-case basis would work. – Charles Stewart May 15 '13 at 7:54
  • @Papiro Your post exemplifies what is wrong with your approach toward reviewing: it is not about winning, it is a duty for the community. And "Honeypot" is the name of the project, if you'd have cared enough to follow the link. – mafp May 17 '13 at 15:05
  • @Papiro: See this comment. I assume the upvote came from you after a quick review. – Werner Jun 2 '13 at 17:24
5

I've looked a bit at the review queue: I note that Papiro, who is the most prolific "No Action Needed" reviewer, actually does go back and comment on quite a few of the reviewed posts, so "rubber stamping" isn't really accurate.

I'm going to propose on Meta that we abolish the six review status screens: having a league table for what should be mostly unsung work, and which has widely acknowledge perverse incentives, doesn't seem wise to me. The review history pages provide the useful picture on what kind of reviewing activity is happening.

Would having a statement of good reviewing practice, perhaps as a question, be useful? It could explicitly discourage over-frequent use of the "No Action Needed" button on first-post reviews.

My guess is that the SX team would be reluctant to implement the honeypot for this site yet, since the overall level of review activity is relatively low.

  • 3
    1) I can't agree with your assessment of the situation: right now 19 of todays 22 first posts are reviewed by the two new users I have in mind, and almost all of their review tasked were rubber-stamped. Yes, Papiro edited one of the questions. Maybe I should present more hard numbers to make my case. But please remember that this is the first contact to the new users. I know that my experience 6 months back when I came along was much better than what is happening now. – mafp May 15 '13 at 14:24
  • 2) There is an awful lot discussion about this on Meta.SO. It is already hard to find the right questions there to leave one's comments ;-) 3) A good FAQ would be helpful, yes. 4) Maybe Honeypot is overshooting, that is why I ask. – mafp May 15 '13 at 14:25
  • @mafp: The point about going back is that if it happens, it shows that the "No Action Needed" choices are being made as part of a workflow that does give further thought to at least some of the posts. – Charles Stewart May 15 '13 at 14:26
  • 1
    @mafp: I posted at meta.stackexchange.com/questions/180323/… – Charles Stewart May 15 '13 at 14:28
  • I know. But here you can see whether someone came back to edit, and he didn't. – mafp May 15 '13 at 14:28
  • 3
    @mafp: My impression is that Papiro is trying to do the right thing, but needs some guidance. Asking a moderator to talk this over with him is the right thing. I'll ask on chat. – Charles Stewart May 15 '13 at 14:42
  • @CharlesStewart Thanks, Charles. However, I read Bill's answer as: use Honeypot, and your problems go away. – mafp May 17 '13 at 15:12
  • @mafp (That's meta.so's Bill the Lizard, I guess you mean) the SX team have been reluctant to roll out the honeypot on other sites, at least yet. – Charles Stewart May 19 '13 at 14:40
4

Regardless of the matter, please do not anonymize certain users if you are going to address them by the implications of the questions. If that's Papiro you have in mind say openly that you have Papiro in mind.

This is not a good way to discuss matters, since you are observing one user's behavior and trying to generalize it to all other users with another terrible, terrible system called Honeypot which is shifting from hereditary monarchy to only elected monarchy which is again nothing but a monarchy (since you have to agree with the majority of the closing votes). I don't know why this is even considered as a system in the first place. But one of the amazing feats of TeX.SX is that if someone comes up with a solid argument, previous opinions and their owners do respect the right(er) argument and change their mind with an impressive attitude almost always in a fun way. What this SO honeypot thingy is offering is to kill that feat and obey to what the top guys think.

You are for example closing questions too fast occasionally (I don't need to make a case for it anyway, which is again my point because I'm addressing you directly) but I don't need to ask a question and say should we introduce another review system with a time limit etc. because that would be wrong. Well, instead I propose that we quit using the review system completely because the system is broken in every aspect and amplifies conflicts but not because of your or a few other users.

My opinion: is that we have enough site reviewers (maybe too many lately) and everybody is imposing a certain way on the newcomers which is not good at all. A mild consensus is OK but pushing new users around is not good with the automated welcome scripts and other stuff. Maybe we don't need to welcome the new users that much. Maybe we were better off with occasional pushes if needed.

If we need to talk to Papiro then we ask a moderator to invite in the chatroom, instead of calling in the indirect mod-hammers to punish.

Papiro is just an example, I don't care too much about this review nonsense to be honest. The more reviews we do, the worse the newcomer experience becomes. Because this thing is taken a little too seriously. We might need to leave the new users alone for a while. They are not stupid per se and are not in need of strict guidance. They are just new and that's about it. If they stick around they get the main idea by screwing up a few times and that's OK too. It looks terrible when a user slightly makes a mistake and 4-5 nannies jump on the new user for fixing this and that, oh image is wrong oh your name is not needed, oh that thing with your thanks is not ok. That's SO uncool.

And if Papiro gets the juice out of this thing, then fine, let him/her do it, we all have weird addiction to this nice place. As long as it doesn't get in the way it's ok for me (which doesn't mean that it's ok for everyone else just for me).

  • I am trying to do my best. Sincerely. This matter is not only against me. The user Anthon (tex.stackexchange.com/users/28931/anthon) also was "anonymized" (@texenthusiast Oh yes. – mafp) in a comment that was deleted. BTW, peace. – user13907 May 19 '13 at 20:18
  • @Papiro I have no objection for your contribution. It's just the motivation of what we do is getting a little weird. SO has their own rules. If you want to retag every question that's also fine. I'm just not in favor of using the SO rules for the edits/close question behavior. If you like what you are doing just keep doing it. It's a massive effort and I don't belittle that. But if you can combine the effort with something that is also beneficial for the site in general that would be optimal instead of what you would like to see. – percusse May 19 '13 at 20:40
  • 2
    I pretty much disagree with everything you write, sorry. First, I did not anonymised a single user, I've seen at least four users with such behaviour, that makes it a pattern. Charles Stewart brought up Papiro in his answer, but I don't want to point fingers on single users, but address that general pattern. Secondly, I think that the quality of reviews does matter. On one hand, I've seen new users being, say, irritated when their "Thank you" were removed without comment etc, on the other hand there have been users who appreciated a "Welcome", followed by a tip how to improve posts. – mafp May 20 '13 at 8:32
  • Thirdly, this is not about punishing anybody. I think to get quality reviews we need a delicate balance between reward (to get reviews done at all), and checks (to avoid rubber-stamping). This balance might even need recalibration from time to time. Honeypot is one tool to push the equilibrium slightly to the checks side, and my question is about whether this would be a helpful tool around here. Maybe less reward would be better, as you say that we have too many reviewers/too much review. Then let's discuss that. – mafp May 20 '13 at 8:39
  • Finally, but off-topic, I doubt that I close questions too fast. Quite contrary, I think I am on the Leave open-leaning side of tx.sx. Most question that I vote "Leave open" are closed anyway, while I hardly ever experience the other way round. Of course that does not mean that I never made a mistake among my 400+ Close reviews. – mafp May 20 '13 at 8:43
  • @mafp We have at least 40 other active reviewers. 4 users are hardly a pattern. And also the same number almost guarantees quality reviews but do we need to have this tight review process I don't think so. When I say you close too fast I mean of course the other 4 fast closers anyway in a particular question so I don't need to include them and make a pattern out of it. Strict discipline is not making any good as far as I can see in the last two year run. Honeypot is just cracknut invention suitable only for SO and that we don't need to agree about of course. (ctd.) – percusse May 20 '13 at 11:52
  • So we have users who love PSTricks and bounties we have retaggers, we have every kind of other esoteric behavior and Papiro and possibly others have different taste for editing. I don't know why but they enjoy editing. Fine with me as long as it doesn't get in the way. Rubber-stamp is not a problem for me. It's just another thing. I don't need to introduce a system that would affect everybody because of four users. And we didn't even ask their opinion which is my main objection for a system. – percusse May 20 '13 at 11:55
  • @mafp A recent example; 41 mins and the question is closed but OP is still editing his question tex.stackexchange.com/questions/115229/… This system is broken. I can't imagine adding another stupid add-on which would make it even worse. – percusse May 20 '13 at 15:03
  • I don't see anything wrong with this example. As stated originally, the question was a duplicate. You probably know that editing a closed question automatically nominates it for reopening, and this is what happened here. I consider the period in which the question was closed a good thing, as it sends possible answers to the older question. And it happens regularly that older questions that are identified as originals receive some new love, exactly because they were found original to some new duplicate. – mafp May 20 '13 at 16:21
  • @mafp 41 minutes for a new user is not even nice let alone welcoming. What you write here is not even related. If you close every question and wait for a fix from the OP, that's SO behavior and it's not what we do here. – percusse May 20 '13 at 16:23
  • A remark regarding the extent of the problem: you rightly say that four people are a minority among 40 reviewers, but when those four people do 90 per cent of the First Post reviews, it actually matters. – mafp May 20 '13 at 16:25
  • You seem to read a closed question as some kind of negative feedback. It is not. It just says: as currently stated, your question has been asked before in this or that question. I think that is fine. – mafp May 20 '13 at 16:29
  • @mafp Not from the OPs point of view if he/she is desperately looking for an answer. In that question we had some angry comments too until we have reached to a common conclusion and I've voted for reopening. – percusse May 20 '13 at 16:43
  • So the OP was angry in that question that his/her question was closed (I can't see dead people)? If so, do you think the anger was justified? – mafp May 20 '13 at 16:49
  • 2
    percusse, @Papiro, mafp: This kind of discussion is more efficient and, if we can stay polite, much friendlier if it is done via chat. With respect to thanks hunting and similar points, I'd expect a wider set of opinions on chat than in a meta comment thread. – Charles Stewart May 21 '13 at 18:12

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .