Monday, February 11, 2013

How To Write The Perfect Bug Report

Like many other developers, the people who I interact with most often in my codebases and issue trackers are not other software engineers—they’re my clients.

My partners have generally never been involved in the development of any software project at any point in their lives prior to working with me. This is not to say that they aren’t technically savvy—on the contrary, they usually are. However, there are a certain number of things about bug reporting that aren’t obvious to someone who hasn’t been doing it for years.

So, I thought about it, and instead of having to give a mini-overview every time I want to bring this up, I thought it might be nice to have something to refer to that explains what you should do when you encounter an issue. Big or small, enterprise or start-up, everything applies.

The Basics

Issue trackers vary widely in terms of features from one to another. Whether it’s a simple list of items on an Google Docs spreadsheet or a custom JIRA installation with multiple plugins and tens of fields, everything I’ll write in this post applies. No matter what, there are certain fields that your issue tracker should ALWAYS account for. They are:

  • Assignee
  • Title
  • Description
  • Milestone
  • Labels / Tags / Categories (optional but recommended)

If your issue tracker doesn’t support the first four, find another one. Labels are good but not necessary.

Now that we’ve got a bug tracker, let’s start off with a fairly common scenario.

I Can’t Login! @#$%!

You woke up this morning to a nice cup of coffee, opened up your application to get work done…and you can’t login.

First thing, DON’T PANIC. It’s easy to feel helpless if you’re not the developer, since you may think there’s not much you can do.

Wrong! Your role is super important. You speak for the users. The better you do this, the quicker the issue will get fixed and the happier everyone will be. So if you find that you’re stressed, just take a deep breath.

We want to make sure that everything we do from this point forward is clearly communicated, straightforward, and easy for the assignee to work with.

Let’s handle the low-hanging fruit, shall we? This means verifying that you actually have an account and then checking that you’re actually putting in the right password. This may sound dumb, but just double check. Always double check.

After you’ve made sure that you didn’t make a careless mistake, try to make whatever broke break again. Depending on the issue, this process can take a fair amount of patience and/or creativity. Software can be very picky, so make sure to be precise. If you encountered the bug when writing your login credentials in ALL CAPS, try to reproduce it with all caps. Then, try with lowercase letters. Still working? Now try with a mix of uppercase and lowercase. Then try with only spaces. Do whatever you need to do to corner it!

After much experimenting, you’ve found that no user can login, even with the correct username and password, and no error message is displayed.

Whatever is happening isn’t intended behavior, so it’s confirmed: you’ve found a bug.

The Issue Title

The issue title comes first for a couple of reasons.

  1. It’s the first thing other team members will see.
  2. A clear issue title helps you decide out who to assign it to.
  3. Writing the title forces you to describe the problem clearly and succintly.

Before going into details, I’m just going to start off with some anti-examples. The following are bad issue titles:

  • login
  • login problem
  • login form not working
  • can’t login
  • login page
  • SITE IS BROKEN

I don’t like these issue titles because they are vague and don’t help the assignee understand what is actually wrong. The best issue titles are specific and to-the-point. Brevity is always appreciated, but the sweet spot for length is probably around 6-10 words.

Your issue title should briefly identify two things:

  1. The undesired outcome
  2. When or how it happens

With all that in mind, here’s the issue title I would write:

Login fails silently when user enters correct credentials

Onwards!

The Assignee

Once we have a title, you should have a good idea for who needs to work on this. There are generally two scenarios here.

1. You know who does what.
2. You know who knows who does what.

If you know who is responsible for whatever feature or bug you’ve just encountered, make them the assignee. If you don’t know who is responsible—even if there is the slightest doubt, assign the issue to the person who will know who to assign it to.

The Description

As with a good title, a description should be short and sweet, but also a bit more detailed. There’s actually a straightforward recipe.

  • Write in sentence or two what you tried to do and what should’ve happened.
  • If the bug seems really specific, give steps on how to reproduce it. Note: if all logins fail (and not just yours) and there’s not really a pattern, you can skip this step.
  • Attach screenshots if applicable.

Given the above, here’s what I might write as the description for the login issue:

I just tried to log into the site from the login form on /accounts/login and nothing appeared to happen, even when using my correct username and password.

If there was some sort of pattern, or only certain credentials fail, this is what I might write:

I just tried to log into the site from the login form on /accounts/login and nothing appeared to happen, even when using my correct username and password. More specifically, it seems that passwords with weird characters cause the issue (my password has a space).
 
1. Use credential with space in password.
2. Fill in form with correct credentials.
3. Submit form and notice that the user is not logged in and no error message is shown.

We’re almost done! On to milestones.

Milestones

Milestones are like goals. Trying to ship software without goals just isn’t going to happen. And if it miraculously does, it’ll take way more time than you originally planned, and will probably have costed way more money and energy than you had the budget for. This is why you need milestones, and why issues need to be placed into them.

Milestones can be numbered, given code names, be descriptive, or anything—it really doesn’t matter. All that matters is that it’s clear what is high priority (due sooner) and what’s “low” priority (due later). I put low in quotes because, in my opinion, everything is high priority. It’s just a matter of how high.

So file the issue into the appropriate milestone.

Personally, I think users not being able to log in is a pretty big deal, so I would file this under the “STOP WHAT YOU’RE DOING AND DO THIS NOW” milestone.

Labels / Tags / Categories

Every bug tracker gives this a different name, but I’m going to call them labels for the sake of simplicity.

I have a love-hate relationship with this one. I think labels have a tendency to be overused, and are mostly just there to be pretty, but I think there’s something to be said for being able to categorize and group issues by similarity.

In my projects, I use six labels1.

  • question: for features we need to discuss or questions
  • enhancement: improvements, can be design, backend, frontend, anything
  • bug: something isn’t working the way it should
  • tweak: changes to existing features or designs
  • needs more info: not enough information in the issue to make a next step
  • duplicate: issue was already reported

Some bug trackers will also have an “invalid” label. I think this is a bad label, because it makes the reporter feel stupid if it’s ever used. This is bad karma and throws a negative wrench in the relationship between everyone involved. Just ask for closer instructions to reproduce, and tag “needs more info”. If the reporter can’t do it, close it and say it couldn’t be reproduced. Yay, no feelings hurt.

The issue we’re discussing (not being able to log in) should just be labeled as “bug” and that’s that.

Conclusion

There’s nothing else, really. I’m sure that there’s way more I could write about this topic, but after reading this, you should feel confident enough to write a bug report that is concise, information-dense, and makes it as easy as possible for the responsible person to resolve.

And remember what I said about not panicking? Well, because you didn’t panic, the bug report you wrote was easy to read and contained straightforward instructions to reproduce. The developer was able to pinpoint the problem in the login logic, run tests to verify he didn’t break anything else, and push a fix to production in no more than 15 minutes.

The most important thing? Everyone did their jobs with a smile on their face. :)

I hope this was helpful to you. Have a comment? I’d love to hear your thoughts. Send me an email or reach out via Twitter.

1 If you use GitHub and like these labels, you may be interested in the script I wrote that replaces the default labels with these ones.




comments powered by Disqus