Guidelines for Running a Bug Bash

My previous blog on Bug Bashing covered the what, why and when. The focus of this blog is how to run a successful bug bash: what you will do before, during and after the Bug Bash.

*The original version of this blog was written in 2018 and updated in 2024.

General

• A Bug Bash should be short, a period of 60 minutes to 90 minutes should be sufficient. For a 60 minute session, 10 minutes for the introduction, 30 minutes for the bug bash, an additional 20 minutes should be allotted for the bug bash debrief.
• The organiser will typically be the tester from the relevant team but could be facilitated by others team members such as the Product Manager, UX or team lead.
• It is better if the organiser does not participate in the bug bash, instead the organiser supports users and answers any questions.
• A Bug Bash should have a minimum of 5 participants to ensure there is sufficiently diverse feedback.
• Decide what channels(MS team, slack, email)  will be used for Bug Bash communications( invites, updates, questions) and inform participants. Slack, MS teams are good for asking questions but I have found it unsuitable for people raising issues(i.e sharing multiple screenshots and stack traces etc). Participants should be encouraged to only use these channels for questions.

Before

• Endeavour to give participants plenty of notice but make them aware that the scheduling of the bug bash can potentially change due to unforeseen issues such as environmental changes or change of release timelines. Thus participants may need to exercise a degree of flexibility.
• Endeavour to have the team members who developed the software be present to support users and observe the feedback first hand. Also for pairing, it can beneficial to have those who worked on the solution pair with those unfamiliar with the solution. 
• Provide any relevant documentation to the participants.
• Ensure Participant should  have access to any relevant systems and   have their own account and data(avoids overlapping with other users).
• Try to involve people from outside the team and outside your engineering department, people that have not actively been involved in the development of the product. These people are seeing the product for the first time and can give valid feedback regarding functionality, usability, and performance. People who work in customer support, training and sales are often excellent contributors.
• For obvious reasons, the environment used during the Bug Bash should be available and stable. The environment may be a test environment or perhaps a production environment with a feature toggle.
• If test environments are shared with other teams, ensure these teams are aware of the Bug Bash to prevent redeployments and other related issues.
• If the product has a web front-end and needs to support multiple browser versions (or even devices) ensure you perform an inventory of what the participants have available and assign browser version(device) to the users, to ensure user use a diverse range of browsers and/or devices. The same advice applies to testing mobile apps.
• If the system or functionality is large you can assign specific functional areas to each of the participants or ask them to complete a particular user flow. Although ensure that you are not overly restricting your participants.
• Participants should be directed towards issues that would impact customers and other key stakeholders, rather than edge cases/obscure issues. Also feedback on feature improvements should be encouraged.
• Create a document for collecting feedback and a shared folder for uploading images, videos, logs. Typically I share a spreadsheet and a shared drive folder for artefacts. The spreadsheet has tabs for the different functional areas  and a few columns to capture the person name, description of the issue and link to any artefacts.
• If you are finding it difficult to find volunteers, offering food for in-person bug bashes(pizza, cookies) or for remote bug-bashes a reward for best bug can increase participation.
• I am a huge advocate of Heuristics and Mnemonics, i will often suggest heuristics that may apply to the feature for the Bug Bash and make this Ministry of Testing article to participants.

During

• Open the Bug Bash by explaining why you are running the bug bash and give a brief overview of the feature to be bug bashed. Often it make sense to demonstrate the functionality.
• If changing existing functionality, it may be worthwhile noting any potential risks.
• After experimenting with different ways of organising people. I feel pairing someone faimialr with the application and feature with someone unfamiliar works well. 
• After the initial overview of the functionality, set a timer for 20 minutes and states this time is for focused testing. (Repeat for another 10 minutes).
• Ensure that you inform all participants of what you expect from them in case they believe they have found an issue.
  • If an issue is found during a bug bash, a participant should not spend time investigating the issue.
  • Instead simply note the problem in the feedback sheet and continue on with further testing. This maximizes testing time and avoids investigation of known issues.

• A Bug Bash can be performed fully remotely , but in some cases, if all or some of the people are in the office you could consider a hybrid or full in person options.

After

• Allow Approximately 20 minutes for debriefing so that all participants can provide feedback on any issues that have been captured.
• Gather feedback immediately after the Bug Bash is completed when the information is fresh in peoples minds.
• Rewarding defects. Typically there is a prize for “Best Bug” found during the Bug Bash. Personally, I have found that praising people who find important issues and updating them when the defects are fixed has a more motivating effect than a prize. I have heard people say “It’s great to know that I made a difference to the product”
• Triage the Bug bash issues and ensure action is taken to resolve the highest priority issues.
• After triage add the key issues to Jira(or your bug tracking tool). If it’s Jira consider adding a bug bash label for reporting purposes. 

Summary

This blog focused on how to run a successful bug bash, what you will do before, during and after the Bug Bash.

• They key to success is in the preparation including finding a diverse range of participants, ensuring a stable environment is available and participants understand what their role is in the Bug Bash.
• During the Bug Bash keep participants focused on exploring the software rather than investigating issues encountered.
• After the Bug Bash, capture all information immediately from or participants. Get support for future Bug Bashes by resolving any issues encountered and use these metrics to gain support from your Management team.

---

This is a personal blog. The opinions expressed here represent my own and not those of my current or previous employers.