A shout-out to the people of Meta Stack Overflow
Jeff Atwood calls it human unit tests and cheating. Alex Miller calls it User Based Monitoring. Namely, relying on user feedback and bug reports to let us know when we break something.
This works remarkably well – when something is wrong, we'll know about it pretty quickly. You can be sure that Meta Stack Overflow will have a new post, complaining about the bug we just introduced. In no time.
This system has been working great so far. But why? Why do people help us, for free, even though we prove bug after bug after bug that after one thing is fixed, another issue is right around the corner?
I've been on both sites of the fence. Before I started to work for Stack Exchange, I was one of the wise-cracking members of the Meta community. Now I am one of the people who answer to those crowds.
So after having seen both sides, here are some reasons why I think this “cheating” works so great.
Many of our users are devs.
This may be the biggest “cheat”: Obviously most users on Meta Stack Overflow are themselves programmers, so you can assume they have certain skills that J. Random User may not have:
Many of them know that a bug report needs more than a
It doesn't work for me, please fix.
to help you fix the problem. I say “many of them”, because somewhat surprisingly there are developers (who presumably deal with bug reports all the time) that don't go much further than the above “please fix”.
That said, most of them are able to give you a few more details – and that helps us a lot.
If you need additional details, they'll be able to provide them without you having to give them a step-by-step tutorial on how to find out the version of their browser, or whether there are any JavaScript errors on the page that's causing the problem.
They may even be able to fix things. It's not unusual to have someone already offer a solution for the problem; maybe they single-stepped through (even minified!) JavaScript to find the problem, or already found the CSS issue that's causing the IE8 rendering problems. Sometimes (especially when the problem is in server-side code), this may be no more than guesses, but sometimes it's hitting the nail on the head.
They know what problems to look for. As developers, they have an understanding of typical problems, and will try things. Sometimes this may give you a useless
If I unplug my network cable in the midst of doing X after having de- and re-activated JavaScript three times, my proxy cache throws an excption that causes the CSS load to fail on the next page.
– but sometimes, they may have thought of an edge condition that you didn't have on your mind, and which is worth considering.
Incentives always help.
The same mechanisms that make people ask great questions and give awesome answers on the main Q&A sites also work on Meta Stack Overflow: Public recognition of your knowledge and skills, the fun of solving problems, and not least: reputation and badges.
When someone finds a bug, describes it well, maybe even offers a fix, and includes screenshots with freehand circles, this will ususally be rewarded through upvotes and confirming comments. Okay, it helps if a bug materializes in a particularly funny way.
We try to give feedback.
We try to explain decisions, bug fixes, and feature implementations. We're not always good at that, but we try. And this feedback is always meant to be a bit of recognition.
You've found a bug, we fix it, and we'll try to explain it. A little post-mortem, if you will. You deserve it.
You've made a feature request, we liked it, we implemented it. Now if that's not deserving of a “Thank you” – you have just helped us make the site more awesome!
Even if we don't take a request because we disagree with it, we try to explain why we disagree. Maybe not anymore when the feature gets requested for the umpteenth time, but in general, a well-written and well-argued feature request is worthy of a reply.
And of course: For someone who cares about the site, it's nice to see that the feature they suggested is now available for them to use! Which reminds me of …
We have users who care.
A bug report or feature request is ususally not going to come from someone who has no real interest in your site. Even if you make it really easy for them, it takes time to write up what you have to say, so in most cases, you can assume the user cares about the issue they're talking about.
Granted, sometimes it'll just be someone who feels like ranting, or is trying to prove to you how smart they are, or how stupid you are. But the vast majority of requests will come from users to whom your site is important in some way.
And fortunately, Stack Exchange is in the convenient position of having lots of users who care. They care enough to not only help us with pointing out our bugs, but also by helping other users (who might not yet be as experienced) to use the sites.
So here's to you, people of Meta!
Thanks a lot for all the work you do in helping us and others. You're a crazy bunch, and you're an awesome bunch.
Also, waffles. That is all.
next post: Introducing Lyfe: Yield in JavaScript