How Swimply is prioritizing quality with Jam to reproduce bugs in one click
The Problem
Reproducing bugs was taking nearly one hour per ticket
Swimply is an online marketplace that allows homeowners to share their passion spaces with their local communities. They have started with swimming pools, but pools are just the beginning, as they are expanding into new shared experiences like tennis courts, basketball courts, yards, and more!
Swimply is one of the fastest-growing consumer apps available today, and the platform serves a high volume of users. It's essential to deliver a high-quality experience to their users during this moment of rapid growth.
The team focused on this outcome is a small but mighty team that includes 12 Engineers, 2 Quality Assurance Engineers, and 1 Head of Product. All work side by side to investigate bugs found via automated testing for products pre-released, but they also manually test to reproduce small bugs that can occur in the wild.
To reproduce these pesky bugs, the QA team was responsible for retracing a user’s journey by following a long series of events. They introduced a rigorous workflow that included logging into multiple systems to grab screen recordings, screenshots, network calls, and console logs.
“Sometimes we weren’t able to reproduce the bug even if we followed the exact same steps” - Christina Stutes, Quality Assurance Automation Engineer
The goal was to reproduce the bug and get a ticket to the engineering team to quickly fix it, but as the Swimply platform is so robust, reproducing these bugs became extremely time-consuming and tedious to narrow down the steps taken to trigger the bug. Constantly retracing their steps to identify what they did to get that outcome.
Once they were able to get a visual of the bug, the team would have to manually upload screenshots or paste recording links to Jira from their legacy vendor. This meant a lot of “context switching” and as a result, it was difficult to capture all the bugs in their platform as the complexity continued to grow.
The team needed a solution that would allow them to quickly reproduce a bug while seamlessly fitting into their current tooling stack and workflows. They needed a tool that would enable product velocity while enabling high-quality execution. The team and product were growing and they needed to act quickly.
The Solution
Finding a tool that reduces the reproduction steps down to just one click
Christina Stutes, Quality Assurance Automation Engineer at Swimply was told about a new tool that would make reproducing and reporting bugs so much easier. The Head of Product shared a Jam with her and she quickly installed it to start playing around with the browser extension.
Within the first 10 minutes, she was blown away by how easy it was to take an instant replay of the last 30 seconds of the screen to capture the bug with just one click. It also had an integration with Jira, so it could immediately populate the ticket with all the diagnostics that the engineering team needed to quickly identify the bug.
“When we first found Jam, it was like a light at the end of a tunnel! Before we would try to repro the issue with a screen recording, guess what needs to be shared from the network and console tab, and then open up Jira to create a ticket. It’s AMAZING to have all of that in one place. It saves us so much time to just grab the 30 seconds video and share it with the devs. All the information is there and we can create a bug ticket on the spot!” - Christina Stutes, Quality Assurance Automation Engineer
Instant Replay of the bug to remove reproduction steps
The old process to reproduce a bug included:
- Run through the required steps to hopefully get a visual repro of the bug
- If the visual was validated, run it again and take a screen recording using Loom
- Go back and clear the network tab to see if it’s possible to collect the network tab or logs
With Jam, now the QA, PM, and engineering team can click on the browser extension when they experience a bug, to instantly record the last 30 seconds of the browser’s state – reducing a process that used to take upwards of an hour per bug ticket to a few mere seconds.
Integrates with Jira and plays well with other tools
One other pleasant surprise for the Swimply team was that Jam integrates with their existing issue-tracking tool, Jira.
“You are telling me, I don’t have to grab a link and then go create a Jira ticket and then paste that link in the ticket? It just hit all of the pain points and allowed us to get time back and find more bugs.” - Christina Stutes, Quality Assurance Automation Engineer
With a seamless integration with Jira, the team can now send the Jam link directly to Jira, without needing to copy and paste links or attachments. Removing the time that it takes to craft a perfect bug report for the engineering team to have all the visual and technical details required to fix the bug.
Using JamGPT to validate
The QA team is typically the first line of defense before creating tickets for engineers. They look at the console logs and network requests to identify potential reasons for a bug to help their engineering team be able to track the error faster.
It is important that the QA team provides as much technical detail as possible for the bug investigation, so the engineering team can use their time to fix the bug vs. researching on their own. The Swimply QA team is now using JamGPT to find the root cause, removing additional steps.
“We may think it’s a UI issue, but with JamGPT – I can see that it was actually a network call!” - Christina Stutes, Quality Assurance Automation Engineer
JamGPT is now helping the QA engineers to validate their assumptions about a bug before they send the ticket to their engineering team. With their legacy workflow engineers used to spend hours investigating the root cause of a bug, but now they can focus on fixing bugs and shipping new features.
The Outcome
1 minute per bug report, manual work reduced, and improved product velocity
Jam has shortened the bug repro and bug ticket submission process that used to take over an hour to just a few seconds. It has given the QA team time back to focus on testing the new latest features that are shipping (there are so many awesome features on the horizon!) vs spending so much time trying to find and fix bugs in the wild that need an immediate remedy.
“It cuts back so much wasted time, because we can easily grab it and send all of the information to our developers vs us having to see it once and then spend - 10, 20, 30 minutes trying to reproduce. It gives us extra time to put towards work that matters,” - Christina Stutes, Quality Assurance Automation Engineer
The impact on the team’s velocity was significant after the effort to reproduce and report a bug decreased by 60%.
“If we are testing our web application on desktop we always create our Jira tickets through Jam. This cuts down on the amount of time we would spend gathering all the information to plug into the ticket. This way, all the information is there and we get that time back.” - Christina Stutes, Quality Assurance Automation Engineer
Implementing Jam into their debugging workflow has exemplified the team’s commitment to building community through introducing more shared passion spaces and launching in new markets.
It’s exciting to see their significant growth and Jam is so happy to support Christina and her team, as they build this awesome shared space platform.