I feel there are really 3 types of people in Gamedev:
1 - Those that will just get upvotes (yayz)
2 - Those that will only get downvotes (and quickly disappear, rightfully so, for being trolls, bots, etc.)
3 - Those that get mixed votes, often starting with a few downvotes, but quickly moving towards a positive count. A lot of people that are active now art part of that group I believe. It takes a few down votes to `fit in` not in that you need to agree with everyone else, but it teaches everyone how to behave as a microcosm. I'm pretty sure 90-99% of my downvotes were warranted, and I hope I've grown as a person from them. I don't think I'm immune to downvotes, etc.
That being said, I've seen very "partisan" threads where the OP was not a stellar community member and decided to downvote people who disagreed with his approach. It happened to me as well. That didn't stop me from getting above 10k rep though (many thanks to my articles, I agree, but still).
In the greater scheme of things, these random downvotes pale in comparison to your contribution, and if you stick around long enough, your rep will go up. I know I've already upvoted you 2-3 times myself :P