Guidelines for Running a Bug Bash

My previous blog on Bug Bashing covered the what, why and when. The focus of this blog is how to run a successful bug bash: what you will do before, during and after the Bug Bash.

General

• A Bug Bash should be short, a period of 30 minutes should be sufficient.
• An additional 30 minutes should be allotted for post bug bash discussion and raising issues.
• The organiser will typically be the tester from the relevant team but could be facitialited by others team members such as the Product Manager.
• I would suggest that the organiser does not participate in the bug bash, instead, the organiser supports users and answer any questions.
• A Bug Bash should have a minimum of 5 participants to ensure there is sufficient feedback.
• Decide what channels(emails, slack etc)  will be used for Bug Bash communications( invites, updates, questions) and inform participants. Slack, yammer, or communicator are good for asking questions but I have found it unsuitable for people raising issues(i.e sharing multiple screenshots and stack traces etc). Bug bashers should be encouraged to only use these channels for questions.

Before

• Endeavor to give participants plenty of notice but make them aware that the scheduling of the bug bash kick off time can potentially change due to unforeseen issues such as environmental changes or change of release timelines. Thus participants may need to exercise a degree of flexibility.
• Try to have the team members who developed the software be present to support users and observe the feedback first hand.
• Provide any relevant documentation to the participants.
• Each participant should have their own account and data(avoids overlapping with other users).
• Try to involve people from outside the team and from outside your engineering department, people that have not actively been involved with the product. These people are seeing the product for the first time and can give valid feedback regarding functionality, usability, and performance.
• For obvious reasons, the environment used during the Bug Bash should be available and stable. The environment may be a test environment or perhaps a production environment with a feature toggle.
• If test environments are shared with other teams, ensure these teams are aware of the Bug Bash to prevent redeployments and other related issues.
• If the product has a web front-end and needs to support multiple browser versions (or even devices) ensure you perform an inventory of what the participants have available and assign browser version(device) to the users, to ensure user use a diverse range of browsers and/or devices.
• If the system or functionality is large you can assign specific functional areas to each of the participants or ask them to complete a particular user flow. Although ensure that you are not overly restricting your participants.
• In your defect management tool, perhaps create a category for bug bash defects. For example in Jira Create a bug bash story or label in Jira so there is somewhere to group potential defects.
• Bug bashers should be directed towards functional issues that would impact support staff and customers, rather than edge cases/obscure issues.

During

• Open the Bug Bash by explaining why you are running the bug bash and give a brief overview of the feature to be bug bashed.
• Ensure that you inform all participants of what you expect from them in case they believe they find an issue.
  • If an issue is found during a bug bash, a participant should not spend time investigating the issue.
  • Instead simply note the problem and continue on with further testing. This maximizes testing time and avoids investigation of known issues
  • They should capture the key information and raise a defect after the bug bash after discussing it with the bug basher organiser
• The participants of the Bug Bash can perform the work from their own desk, but in some cases, it might be good to have all participants in a single room as this has the following benefits:
  • An issue is encountered that affects all participants this can easily be communicated.
  • It ensures the participants are truly dedicated to the Bug Bash and are not distracted by their regular day-to-day work or colleagues.

After

• Allow 30 minutes for feedback and to raise any issues.
• Gather feedback immediately after the Bug Bash completes when the information is fresh in peoples minds.
• Rewarding defects. Typically there is a prize for “Best Bug” found during the Bug Bash. Personally, I have found that praising people who find important issues and updating them when the defects are fixed has a more motivating effect than a prize. I have heard people say “It’s great to know that I made a difference to the product”
• Metrics: ensure that the issues that were discovered in the Bug Bash are correctly labeled. This will aid support for a  future bug Bash. Triage the Bug bash defects and ensure action is taken to resolve the highest priority defects.

 

Summary

This blog focused on how to run a successful bug bash, what you will do before, during and after the Bug Bash.

• They key to success is in the preparation including finding a diverse range of participants, ensuring a stable environment is available and participants understand what their role is in the Bug Bash.
• During the Bug Bash keep participants focused on exploring the software rather than investigating issues encountered.
• After the Bug Bash, capture all information immediately from or participants. Get support for a future Bug Bash by resolving any issues encountered and use these metrics to gain support from your Managment team.

---

This is a personal blog. The opinions expressed here represent my own and not those of my current or previous employers.