25 – Bug Writing Guidelines

December 5, 2009 – 10:17 pm

Please note… This information no longer exists at the referenced locations.  This is only a copy of what was available in 2003.

Basic Linux Training™

Bug Writing Guidelines

Henry White

Table of Contents

Writing bug reports

Most software packages have their own webpage – usually something along the lines of http://www.[package].org/, a mailing list, and some formal mechanism for filing bug reports. This is one of the open secrets that makes it possible to get bugs and security patches in as short a turn-around time as possible ;-)

One of the most useful and widely used bug tracking systems is Bugzilla which came out of the Mozilla project when Netscape released the code several years ago. That’s what we’ll use for discussion purposes and we’ll be following Mozilla’s Bug Writing Guidelines, originally written by Christine Begle; and the Mozilla QA Bug Writing Guidelines originally written by Eli Goldberg.

By following these guidelines, you can help ensure that your bugs stay at the top of the programmers’ and engineers’ TODO list and get fixed ahead of others that may have been filed days or even weeks before. Bugzilla is the preferred method of submitting a bug – the linked entry form incorporates parts of these guidelines. (Again, other distributions and packages will probably have similar guidelines and their unique forms, so you’ll have to adapt to whichever one applies. The basic principles and concepts are the same.)

How to Write a Useful Bug Report

Useful bug reports are ones that get bugs fixed. A useful bug report normally has two qualities:

  1. Reproducible. If an engineer can’t see it or conclusively prove that it exists, the engineer will probably stamp it “INVALID”, and move on to the next bug. Every relevant detail you can provide helps.
  2. Specific. The quicker the engineer can isolate the issue to a specific problem, the more likely it’ll be expediently fixed. If you’re crashing on a site, please take the time to isolate what on the page is triggering the crash, and include it as an HTML snippet in the bug report if possible.
    (Specific bugs have the added bonus of remaining relevant when an engineer actually gets to them; in a rapidly changing web, a bug report of “foo.com crashes my browser” becomes meaningless after the site experiences a half-dozen redesigns and hundreds of content changes.)

Let’s say you crash at foo.com, and want to write up a bug report:

BAD: “My browser crashed. I think I was on foo.com. I think that this is a really bad problem and you should fix it or else nobody will use your browser. By the way, my sister thinks your icons really suck. Oh, and my mom’s home page doesn’t look right, either, it’s all messed up. Thx 4 fixing theze bugz.”

GOOD: “I crashed each time when I went to foo.com, using Mozilla on a Win NT 4.0 (Service Pack 5) system. The build ID is 20000609. I also rebooted into Linux, and reproduced this problem using the 20000608 Linux build.”

The first one is mostly whining with next to no useful content. Just human nature, this one is NOT going to get a lot of attention if the crash at foo.com is not immediately reproducible ;-)

Contrast that with the second one – the version right down to the build (and the latest service pack) for NT and the same problem with Linux and that build! This stands out as someone who has genuinely tried to put a little effort into it from the user end and will almost certainly be a priority. The programmers and engineers know for example that this bug is not specific to one build, but quite likely they can quickly narrow it down since the build numbers are consecutive.

We’d also recommend reviewing a few of the better bug reports mentioned in the Bug Writing Guidelines, such as 2683, 3092 or 4044. Bug submissions that do not meet these “Useful Bug Report” criteria tend to be investigated on a time-available basis, if investigated at all.

How to Enter your Useful Bug Report into Bugzilla

Before you enter your bug, you need to make sure it has not been previously reported. There is a tutorial on the best ways of doing this.

Next, be sure that you’ve reproduced your bug using a build released within the last few days. The development process throughout Linuxdom moves at lightning speed, and the bug you’ve found may already have been fixed. (Nightly builds can be downloaded from the mozilla.org binaries page. Again, an open-secret that makes development run much faster and smoother than charging users to do beta testing only to insult them by turning around and charging them fullprice for the official release ;-) What is amazing is that millions of users are eager to pay twice! <shudder>)

If you’ve discovered a new bug using a current build, report it in the guided Bugzilla entry form.

  1. Select the product that you’ve found a bug in.
  2. If you haven’t logged into Bugzilla already, you’ll need to enter your email address, password, and press the “Login” button. (If you don’t yet have a password, leave the password text box empty, and press the “email me a password” button instead. You’ll receive an email message with your password shortly.)

Now, fill out the form. Here’s what it all means:

Where did you find the bug?

Product: In which product did you find the bug?

You just filled this out on the last page.

Version: In which product version did you find the bug?
We’re not yet using this field. Just leave the default value as you found it. ;)

Component: In which component does the bug exist?
Bugzilla requires that you select a component to enter a bug. (If they all look meaningless, click on the Component link, which links to descriptions of each component, to help you make the best choice.)

Platform: On which hardware platform did you find this bug?(e.g. Macintosh, SGI, Sun, PC.)
If you know the bug happens on all hardware platforms, choose ‘All’. Otherwise, select the platform that you found the bug on, or “Other” if your platform isn’t listed.

OS: On which Operating System (OS) did you find this bug? (e.g. Linux, Windows NT, Mac OS 8.5.)
If you know the bug happens on all OSs, choose ‘All’. Otherwise, select the OS that you found the bug on, or “Other” if your OS isn’t listed.

How important is the bug?

Severity: How damaging is the bug?
This item defaults to ‘normal’. (To determine the most appropriate severity for a particular bug, click on the Severity link for a full explanation of each choice, from Critical to Enhancement.)

Who will be following up on the bug?

Assigned To: Which engineer should be responsible for fixing this bug?
Bugzilla will automatically assign the bug to a default engineer upon submitting a bug report; the text box exists to allow you to manually assign it to a different engineer. (To see the list of default engineers for each component, click on the Component link.)

Cc: Who else should receive e-mail updates on changes to this bug?
List the full e-mail addresses of other individuals who should receive an e-mail update upon every change to the bug report. You can enter as many e-mail addresses as you’d like; e-mail addresses must be separated by commas, with no spaces between the addresses.

You would not normally change either of these fields from their default values.

What else can you tell the engineer about the bug?

URL: On what URL did you discover this bug?

If you encountered the bug on a particular URL, please provide it (or, them) here. If you’ve isolated the bug to a specific HTML snippet, please also provide a URL for that, too or, preferably, return to the bug after you’ve submitted it and add the HTML snippet as an attachment.

Summary: How would you describe the bug, in approximately 60 or fewer characters?
A good summary should quickly and uniquely identify a bug report. Otherwise, developers cannot meaningfully query by bug summary, and will often fail to pay attention to your bug report when reviewing a 10 page bug list. Think of it as a “title”.

A summary of “Drag-scrolling any web page crashes Mac builds” is a useful title. “Crash” or “Drag Crash” would be examples of a bad title.

Description: What else can you tell the engineer about this bug?
Please provide as detailed of a problem diagnosis in this field as possible, including as much as possible of the following information:

Overview Description: More detailed expansion of summary.

    Drag-selecting any page crashes Mac builds in NSGetFactory

Steps to Reproduce: The minimal set of steps necessary to trigger the bug. Include any special setup steps.

    1) View any web page. (I used the default sample page,
       resource:/res/samples/test0.html)

    2) Drag-select the page. (Specifically, while holding down the
       mouse button, drag the mouse pointer downwards from any
       point in  the browser's content region to the bottom of the
       browser's content region.)

Actual Results: What the application did after performing the above steps.

    The application crashed. Stack crawl appended below from MacsBug.

Expected Results: What the application should have done, were the bug not present.

    The window should scroll downwards. Scrolled content should
    be selected. (Or, at least, the application should not crash.)

Build Date & Platform: Date and platform of the build that you first encountered the bug in. (The build date can be found as part of the window title, in YYYYMMDDHH format.)

    2000060709 installer build on Mac OS 8.6

Additional Builds and Platforms: Whether or not the bug takes place on other platforms or browsers.

    - Occurs On
            Seamonkey (20000605 build on Windows NT 4.0)

     - Doesn't Occur On
            Seamonkey (20000602 build on Red Hat Linux;
            feature not supported)
            Internet Explorer 5.0 (Windows NT 4.0)
            Netscape Communicator 4.5 (Mac OS)

Additional Information: Minimized HTML snippets, Talkback crash IDs, and any other debugging information. For crashing bugs:

  • Win32: if you receive a Dr. Watson error, please note the type of the crash, and the module that the application crashed in. (e.g. access violation in mozilla.exe)
  • Mac OS: if you’re running MacsBug, please provide the results of a how and an sc.
  • Unix: please provide a minimized stack trace, which can be generated by typing gdb mozilla core into a shell prompt.
    *** MACSBUG STACK CRAWL OF CRASH (Mac OS)

    Calling chain using A6/R1 links
     Back chain  ISA  Caller
     00000000    PPC  0BA85E74
     03AEFD80    PPC  0B742248
     03AEFD30    PPC  0B50FDDC  NSGetFactory+027FC
    PowerPC unmapped memory exception at
        0B512BD0 NSGetFactory+055F0

You’re done!

After double-checking your entries for any possible errors, press the “Commit” button, and your bug report will be part of the Bugzilla database.

(The Mozilla QA Bug Writing Guidelines were originally written by Eli Goldberg. Thanks to Claudius Gayle, Jan Leger, Peter Mock, Chris Pratt, Chris Yeh and Felix Miata for contributing to this document. Additional suggestions welcome.)

Bugs

(Read the bug writing guidelines as well, for tips on how to write useful bug reports.)

What Is Bugzilla?

Bugzilla is a database for bugs. It lets people report bugs and assigns these bugs to the appropriate developers. Developers can use Bugzilla to keep a to-do list as well as to prioritize, schedule and track dependencies.

Not all ‘bugs’ are bugs. Some items in the database are known as Enhancement Requests or Requests For Enhancement (RFE for short). An RFE is a bug whose severity field is set to ‘enhancement’. People often say ‘bug’ when they mean ‘item in Bugzilla’, so RFEs often wind up being called bugs. Enter the tasks you’re planning to work on as enhancement requests and Bugzilla will help you track them and allow others to see what you plan to work on. If people can see your flight plan, they can avoid duplicating your work and can possibly help out or offer feedback.

Anatomy Of A Bug

Bugs and RFEs are composed of many fields. Some of them are described here.

Component
The Mozilla application is composed of many different components such as the networking library, javascript, and the layout engine. Carefully read the component descriptions if you’re unsure of which component to attribute your bug to. Some components are very similar. For example, problems with table layout should be assigned to HTML Tables, not to Layout. The component you choose determines which person Bugzilla assigns the bug to.
Status Whiteboard
The Status Whiteboard is used for writing short notes about the bug.
Keywords
This field is used to store various keywords. For instance, the BugAThon uses it to to note when bugs have test cases (using the keyword testcase).
helpwanted
Developers can use this field for posting helpwanted notices. Write helpwanted in the Keywords field of one of your bugs or RFEs and others searching for things to do will find it. People interested in getting involved can search Bugzilla for bugs and RFEs marked helpwanted. Maybe one of them will have the expertise and resources to help you with your problem.For instance, if your code has a bug on some platform which you don’t have access to, or with some language you don’t know, try adding this keyword.

Or, as with any developer, you’re probably swamped with bugs. Some of these bugs will be lower priority than others. Try marking bugs that you realize you won’t be able to get to any time soon as helpwanted. Someone else with different priorities may see this and help you out.

When marking bugs (or RFEs) helpwanted, be sure to add a comment carefully explaining what work needs to be done, and how to do it, if you know. The better you can explain the problem and its solution, the more likely it is that someone can provide a useful solution.

Target Milestone
The Mozilla project uses target milestones to plan Mozilla’s development process. If a bug is marked mozilla0.9.3, it means work on the bug is expected to be completed by Mozilla 0.9.3. This field should only be set by the person responsible for the bug.
Dependency
If a bug can’t be fixed until another bug is fixed, that’s a dependency. For any bug, you can list the bugs it depends on and bugs that depend on it. Bugzilla can display a dependency graph which shows which bugs it depends on and which are dependent on it.
Attachment
Adding an attachment to a bug can be very useful. Test cases, screen shots and editor logs all can help pinpoint the bug and help the developer reproduce it. If you fix a bug, attach the patch to the bug. This is the preferred way to keep track of patches since it makes it easier for others to find and test. To make a patch, you need to generate a diff file containing the differences between the file with your changes and the original file in the repository. To generate a patch file enumerating changes for all files in the current directory try

    cvs diff -u > mypatch.diff

To apply a patch, go to the proper directory and

    patch < bugpatch.diff

Life Cycle Of A Bug

What happens to a bug when it is first reported depends on who reported it. New Bugzilla accounts by default create bugs which are UNCONFIRMED – this means that a QA (Quality Assurance) person needs to look at it and confirm it exists before it gets turned into a NEW bug.

If you have been working in Bugzilla for a while and believe you are worthy of being able to create NEW bugs, mail Gerv. When a bug becomes NEW, the developer will probably look at the bug and either accept it or give it to someone else. If the bug remains new and inactive for more than a week, Bugzilla nags the bug’s owner with email until action is taken. Whenever a bug is reassigned or has its component changed, its status is set back to NEW. The NEW status means that the bug is newly added to a particular developer’s plate, not that the bug is newly reported.

Those to whom additional permissions have been given have the ability to change all the fields of a bug (by default, you can only change a few). Whenever you change a field in a bug its a good idea to add additional comments to explain what you are doing and why. Make a note whenever you do things like change the component, reassign the bug, create an attachment, add a dependency, or add someone to the CC list. Whenever someone makes a change to the bug or adds a comment, the owner, reporter, the CC list and those who voted for the bug are sent an email (unless they have switched it off) showing the changes to the bug report.

If you are doing much work in Bugzilla, you might want to check out the QA Help Page, which has various useful resources and things to do.

When a bug is fixed it’s marked RESOLVED and given one of the following resolutions.

FIXED
A fix for this bug has been checked into the tree and tested by the person marking it FIXED.
INVALID
The problem described is not a bug, or not a bug in Mozilla.
WONTFIX
The problem described is a bug which will never be fixed, or a problem report which is a “feature”, not a bug.
LATER and REMIND
These are both deprecated. Please do not use them.
DUPLICATE
The problem is a duplicate of an existing bug. Marking a bug duplicate requires the bug number of the duplicating bug and will add a comment with the bug number into the description field of the bug it is a duplicate of.
WORKSFORME
All attempts at reproducing this bug in the current build were futile. If more information appears later, please re-open the bug; for now, file it.
MOVED
The bug was specific to a particular Mozilla-based distribution and didn’t affect mozilla.org code. The bug was moved to the bug database of the distributor of the affected Mozilla derivative.

QA look at resolved bugs to make sure the appropriate action has been taken. If they agree, the bug is marked VERIFIED. Bugs remain in this state until the product ships, at which time the bug is marked CLOSED. Bugs may come back to life by becoming REOPENED.

Be careful when changing the status of someone else’s bugs. Instead of making the change yourself, it’s usually best to make a note of your proposed change as a comment and to let the bug’s owner review this and make the change themselves. For instance, if you think one bug is a duplicate of another, make a note of it in the Additional Comments section.

If you make a lot of useful comments to someone’s bugs they may come to trust your judgement and ask you to go ahead and make the changes yourself, but unless they do, its best to be cautious and only make comments.

The forms prompt you for information that is essential to the developers. They need this information to understand and reproduce the bug. Being able to understand and reproduce bugs is the first step towards getting the bug fixed.

Copyright © 1997-2003 Henry White. All Rights Reserved.
Reproduction or redistribution without prior written consent is strictly prohibited. Address comments and inquiries to info@basiclinux.net

Sorry, comments for this entry are closed at this time.

Proudly powered by WordPress. Entries (RSS) and Comments (RSS). Silver Light 3-Column theme by Bob, modified by Ms. Crafty.