Recommended
How to run a successful bug bash
Is your newest software feature behaving oddly?
Are you worried about your upcoming release, unsure if you’ve tested thoroughly enough?
Have users been mentioning defects more and more?
If the answer to any of the above questions is yes, then it’s a good idea to organize a bug bash.
With this procedure, you’ll unearth a good portion of the bugs in your software, and do so quickly and inexpensively. It’s an excellent method for eradicating all your software’s issues.
If you’re unsure what a bug bash is, don’t worry—we’ve explained it all below.
Table of Contents
What is a bug bash
A bug bash is the software development version of a pest control extermination—the two procedures are equally aggressive.
A collaborative, team-wide (sometimes company-wide) event, a bug bash brings together many different roles and departments in one room.
The goal is to explore every nook and cranny of your software, and look at it from all possible angles and in all possible environments, in order to find as many bugs as possible.
Here’s a photo of a typical bug bash:
Participants search for bugs both on desktop and mobile devices. Furthermore, they’re equipped with a spreadsheet to report the errors they come across.
This is a standard bug bash set-up.
Ron Patton first coined the term bug bash in his book Software Testing, defining it as:
A bug bash is a procedure where all the developers, testers, program managers, usability researchers, designers, documentation folks, and even sometimes marketing people, put aside their regular day-to-day duties and “pound on the product” — that is, each exercises the product in every way they can think of.
Since participants are emulating end-users, technical knowledge isn’t necessary. Sales, Marketing, and other non-technical departments can join in.
However, there should be some developers and testers, as they’re most familiar with the product.
Given the extensive results bug bashes can yield, sometimes even executive-level employees take part. Brad Silverburg is reported to have said:
If the most productive, efficient developer participates in the bug bash, the bug bash is worth participating in. Whoever can take part, should—including senior management.
The effectiveness of bug bashes is proven by the number of companies that employ them. For example, Microsoft’s Windows Insider organized a bug bash earlier this year:
Windows Insider organized 53 quests to test Windows 11.
Bug bash attendees would accomplish quests that interested them, uncovering defects and reporting back to Windows Insider.
Once the event finished, participants were awarded for their efforts—they received the badge displayed in the tweet above.
Bug & crash reporting tool for mobile apps.
The badge served as an acknowledgment of attendance and as a thank-you.
Gamification is common in bug bashes, and participants are often granted badges, awards, or even trophies.
For example, here’s a picture of Classy’s most productive bug bash attendee:
Whoever finds the most bugs is awarded this trophy, in recognition of their bug-finding prowess.
These fun, silly gamification elements are an excellent method to make bug bashes enjoyable as well as productive.
Why is having a bug bash important
As a general rule, software performs better the fewer bugs there are. Software bugs are defects, errors that prevent the product from functioning correctly.
The chief benefit of a bug bash is that it unearths many such errors in a short time.
By blocking out one afternoon and inviting 20-odd participants, you’ll likely find more bugs than a few regular testers would in a week.
For example, here’s the success GeekyAnts had:
The company had an impressive result—they located over fifty bugs in only one hour. The event was hugely productive.
Part of the reason bug bashes are so effective is that they bring together a large, diverse pool of people.
Each individual at the bug bash will provide a different perspective, notice different features and functionalities, and make different suggestions.
Bhumika Srinivas also highlighted this. She described the reason for the practice’s efficacy as follows:
No two people at the bug bash will use the software in the same way simply because there are so many possible approaches.
It’s a numbers game—the more people there are, the more perspectives there are, and consequently, the more errors are found.
Furthermore, not everyone has the same definition of quality. For example, developers working on your mobile app for months might not consider a five-second loading time an error.
However, most smartphone users are accustomed to the breakneck speeds of Whatsapp and Instagram and expect an app to load in two or three seconds.
As a result, someone will likely report the slower speed if your app isn’t up to scratch.
This scenario highlights the fact that not everyone has the same definition of quality. Or, as Gerald Weinberg put it:
The benefit of a bug bash is that with so many individuals with different areas of expertise working together, someone is bound to notice any bugs that were previously overlooked simply because they have a high quality standard for that product portion.
As a result, once the event is over, your product will be of higher quality and provide more value to the user—all because of the bug bash.
When to hold a bug bash
There’s not much sense in holding a bug bash at the beginning of development. Your software is in its starting stages, and the codebase is still undeveloped and unreviewed.
In the traditional software development life cycle, testing is held directly after development. All that’s left afterward is integration or deployment and, eventually, maintenance.
The graph below displays this:
The reasoning is simple—if testing is done simultaneously with development, the testers might forget to test later features.
Errors can slip through just because the software is still evolving, and the QA team might not keep up.
As such, it’s recommended to test the software once it’s fully built. That significantly reduces the chance of later code changes going untested.
Bug bashes follow the same logic. You’re aiming to find all errors in the software, so why organize the event if bugs can creep in later?
Therefore, bug bashes are most often held right before a major release, or a new feature is introduced. That way, the new software is deployed with minimal bugs.
Mattermost also recommends this, as shown below:
If possible, organizing the bug bash at least one week before the release is best.
That way, your developers should have plenty of time to remove the errors, and you can deploy bug-free software.
However, some companies also organize exploratory bug bashes—events unrelated to a specific feature or release.
The difference between the two is explained below:
Exploratory bug bashes are usually held monthly. They serve as a QA check for the entire software, testing the product as a whole.
As such, they operate on a regular schedule and are a standard part of software testing.
On the other hand, feature-specific bug bashes are those tied to a particular feature and are conducted before the feature’s release.
Exploratory bug bashes complement software testing well, but feature-specific bug bashes are considered the more useful, as they essentially prepare a feature for release.
Consequently, they are also more common.
How to manage a bug bash
To reap the benefits of a bug bash, you need to make sure your event is well-organized. Otherwise, you risk participants not uncovering very many bugs.
When managing your bug bash, make sure it features these three components:
Participants should know what feature they’re testing. It’s also helpful to have a set schedule so participants know how much time is allocated for each activity.
Finally, attendees will need an easy bug reporting method.
The following sections detail how to achieve these requisites and what should be done before, during, and after the bug bash.
Before the bug bash
Before the bug bash starts, you’ll need to define the players that participate in it—the event requires more than just testers looking for bugs.
The graph below displays all the characters:
The moderator is the chief organizer, managing logistics such as location, tables, food, drinks, etc.
Furthermore, they ensure everyone is focused yet comfortable during the event. This will probably be your role.
The participants are the individuals actually searching for bugs. Anyone from the company can participate.
Finally, the stager sets up the testing environment. They provide all testing devices, tools, and templates for bug reporting.
As such, this role is usually reserved for developers with domain knowledge.
Once all roles are defined, it’s time to send invitations. After all, your co-workers can’t attend if they don’t know the event’s details.
The invitation should be sent via email, as instant messages are easily overlooked.
If possible, a C-level executive should send it, as their voice will carry more weight and incentivize employees.
If you’re unsure how to word the invitation, here’s a template:
It’s always good to mention free food and drinks, as most employees will appreciate it.
Furthermore, it’s best to send the email a few weeks before the event. This allows plenty of time to prepare beforehand.
During the bug bash
During the bug bash, all participants should be welcomed and given detailed instructions before the event starts.
You’d do well to inform them about the session’s scope, bug reporting method, and timeline.
A bug bash is always a time-boxed event. It can last however long you think is necessary, but the standard timeframe is ninety minutes.
For your first bug bash, try the following approach:
Use the first ten minutes to provide instructions and host a quick Q&A session.
Next is an hour-long uninterrupted block devoted to the participants’ bug finding.
Simultaneously, the moderator and stager should categorize bugs as reported, so there’s less triage post-event.
Finally, utilize the last twenty minutes with each participant briefly elaborating on the bugs they found.
To streamline the process, it’s worth investing in a bug reporting tool.
For example, with Shake, all you need to do is shake your mobile device, and a comprehensive report will immediately be sent to your developers.
Have a look:
The bug report contains all possible details—screenshots, screen recordings, steps to reproduce the bug, network specifications, app version, etc.
This tool allows bug bash participants to report bugs quickly and easily, significantly improving the event’s productivity.
After the bug bash
Once the bug bash is over, it’s clean-up time. The testers have uncovered all the bugs they could, and now your QA testers need to categorize them.
Bugs are usually sorted based on their impact on the software—in other words, according to their severity.
For example, this is a minor bug:
The text should read No stops at the moment, but the No stops portion is not displayed. However, while irritating, this doesn’t affect functionality all that much.
However, no user wants to see this message:
This error message appears when an app crashes. Such an event essentially renders an app useless, which is why it’s classified as a major bug.
However, there can also be invalid bugs. These are explained below:
Duplicate bugs are the same defect reported twice, creating nothing but clutter.
A defect is declared not a bug when the reported error doesn’t actually affect the software’s functionality.
Sometimes, defects can’t be fixed. For example, when software is too old or the programming language doesn’t have the capacity to solve a problem.
Finally, if a report is missing any information (e.g., expected and actual result), this is considered an incomplete bug report.
After the bug bash is finished, it’ll be your QA team’s job to sort through all findings, reject any invalid bug reports, and assign severity to the existing bugs.
Conclusion
A bug bash is a collaborative effort where employees gather and do their best to find bugs.
Usually held right before a software release, bug bashes are handy as they combine different perspectives, making it easy to find many different bugs.
Before the event, ensure you define everyone’s role. It’d also be good to provide a bug reporting tool to streamline the entire process. Afterward, your QA team will have to triage all the results.
All in all, bug bashes are worth the effort—they’re an excellent method for unearthing many software errors very quickly.
