Sunday, November 27, 2005

Why software ships with tons of bugs

Here's a great article (written by a SourceGear developer) that explains why Windows Vista will probably ship with tens of thousands of bugs:

My life as a Code Economist

The main point of the long article is that you do it because you're willing to ship a product with known bugs that aren't too bad (1) because if you don't, you'll never ship anything, and (2) because shipping with not-so-bad bugs you know about is better than shipping with horrible bugs you don't.

That's something that I didn't fully understand before I started working at Microsoft. I rarely posted any of my own products if they had bugs. They were small, and I could do that. But that isn't possible in the real world. And, in the real world, lots of things are bugs that you don't consider bugs. When you work on something big with a lot of people, you realize that you can just absolutely nitpick the thing to death. There are a thousand little silly things you could change. If I went through any of my popular apps like EclipseCrossword or StickyPad with a scrutinizing eye, I'd have hundreds of things to change. I don't, because I don't have time to change those things. What I didn't understand before is that a feature request is a bug, random things that suck are bugs, and things that need to be revisited in the future are bugs too. They all have the same effect; they're all areas for improvement.

In a way, I understood this to some degree. If there was a behavior change that would require an option that could be set, I'd have to weigh how hard it was going to be to fix against how popular I thought that feature would be. I've even, in a few cases, chosen to work on new features instead of fixing bugs. StickyPad will sometimes lose some of your text if you use it on US English Windows with the Windows UI in Korean. I consciously decide not to care too much about Asian languages or Hebrew or Russian or any of the hard stuff, because I don't have enough knowledge in that area or the time to test every language. So, I ship StickyPad with lots of known bugs regarding those wacky languages.

This is the sort of thing that should be taught in a software engineering class, because it's always been true, and it will always be true. It can even apply to things that aren't software. It's something that I understand much better now that I've spent time in the real world, but it couldn't have hurt to have it drilled into my head earlier, instead of the much more worthless crap I got from computer science classes. Of course, computer science has fairly little to do with building software as a practice, a business, a large-scale undertaking—but it still couldn't hurt to be a little more broad...

1 comment:

Luke said...

I agree. It would at least been useful for my Microsoft interviews, where I was like, "Ship? With Bugs! No way!" Now, working at GC Image, its more like, "Ship? Allright, how many do we have? How big are they? Let's mark them to be fixed later."

I also frankly don't believe the elaborate white box/black box path coverage testing stuff we learn is too useful. More useful stuff would be, when do you delay shipping vs ship with bugs? How much of a risk do you want to take that your customers will go elsewhere because of bugs, even if they aren't important?

Of course, you sort of keep score with bugs, so people will feel bad if their bugs are posted. That's what happens to me. And then if I get a whole bunch of bug reports from other developers, I'll point out some bugs of their own, so they remember that they too have problems, and should not be spending their whole day generating bug reports against me.

The facts of life about bugs...