I'm not a fan of the 'golden hammer', to be honest. It is too easily abused or inadvertently misused, and its effect is determined rather arbitrarily in practice, since questions are often not even tagged appropriately in the first instance. Incorrect tagging further exacerbates the problem of inadvertent misuse, by which I mean intending to vote, but actually closing unilaterally. Furthermore, I'm not convinced that a golden badge for even an appropriate tag guarantees any particular expertise in identifying duplicate questions, especially since people often seem to erroneously think of duplication in terms of duplicate answers.
Fortunately, however, actions of this kind - inadvertent or otherwise - are, as Alan Munn points out, easily reversed. Moreover, refusing to participate in this system on the grounds that it is unlike 'normal' human environments is, I think, wrong-headed. I can think of various more-or-less persuasive reasons for refusing to participate, including a judgement that the system is unfair, arbitrary, anti-democratic, non-consensual, objectionably hierarchical and supportive of anti-social, tyrannical and community-destroying behaviour. But the claim that it should be rejected because it fails to reflect 'normal' human environments is, in my opinion, untenable.
First, SE is not a 'normal' human environment in any sense. No online environment is normal, but SE is doubly abnormal, if not triply so. It is downright weird in so many other ways that the voting system is surely a mere trifling detail. The only adequate way to reject its abnormality is to never visit. I'm not at all suggesting you do that: I'm just saying that this is the logical implication of your position. Indeed, I would prefer to view this as an reductio ad absurdum of that position.
Second, 'normal' is a fraught word and it isn't clear what is meant here. Statistically normal? That's no objection: if SE were friendlier and more helpful than statistically normal human environments, that would be a point in its favour and certainly no reason against. Presumably, there is rather a notion of what a human environment ought to be like. And the objection is that the system is unfair, unjust etc. and that human environments ought not to be this way. I think this is arguably correct. Whether or not it justifies participating in the site but not participating in this particular aspect of the system, I'm not sure.
If the objection is that the voting/hammer system is anti-egalitarian, that's true, but the entire site is designed to promote anti-egalitarian aims, so it isn't clear why this particular feature should be picked out for rejection.
If the objection is that the voting/hammer system is not effectively meritocratic, although this is (presumably) the intention, that is not implausible in the sense that the system in practice provides rather arbitrary powers which typically bear a somewhat tenuous relationship to pertinent expertise. However, this is equally true of many, if not most, other aspects of SE's design, so it is not clear why this feature should be singled out for rejection.
It is, of course, true that paradigmatic cases of 'voting' are one-person-one-vote. However, this is not an essential feature of voting systems and is, in fact, an extremely recent innovation which is still far from perfectly realised even by the best of democratic systems. Moreover, outside of special contexts, one-person-one-vote is the exception rather than the rule. (Statistically) normal human environments are ones in which the power of individuals varies hugely. Most of these don't use voting at all, of course, but many do: when shareholders in companies vote, the rule is not one-person-one-vote and many of those most affected by their decisions may have no votes at all. And there is probably less reason to think shareholders enjoy votes in proportion to their expertise than there is to think that SE users do. This is not to say that such systems are just, but they are certainly 'normal' human environments at this point in time.
I would say that the SE environment generally, including its voting system, is, like all human environments, messy, flawed and imperfect. Parts of it reflect design purposes I don't endorse; other parts imperfectly realise purposes I do. As with all human systems, SE's users introduce additional flaws into the environment. They further frustrate the ends of the designers - both those I agree with and those I don't - and they find ways to use features for purposes which were never intended. In this sense, SE is absolutely typical of human environments - 'absolute' because true in the strongest possible sense. There are simply no exceptions because no human environment is perfectly designed, no human designer pursues entirely ideal ends and no human beings fail to frustrate both the best and worst of other humans' purposes.