Mobile & Web
A Complement to Beta Testing
With thousands of users around the globe, Blendr offers a social networking experience unlike any other. Since 2009, the company has been connecting members of its community based on shared interests via an assortment of applications for iPhone, Android and Facebook.
Among those thousands of users is an active, eager beta community. As the company’s testing challenges grew, VP of Technology Aaron Judd began searching for a way to combine what he liked best about the beta community (real-world feedback) with professional testing. Having worked with uTest at a previous company, Aaron was already sold on the concept of in-the-wild testing, making his choice all the more easy.
"We were trying to find a balance between what our beta community does well and what our in house QA team does. uTest was a perfect fit, since they would give us real-world testing like our beta community, but they would report bugs into a dedicated platform, making it easier for us to sort through and analyze."
-- Aaron Judd, VP of Technology, Blendr
This brief case study will highlight how Blendr was able achieve in-the-wild testing of their nine different products. Along the way, we’ll cover the basics of defining a testing scope, analyzing bug reports, working with a project management team and other aspects of the uTest experience.
Blendr: Setup, Scope and Tester Selection
Of course, detailed bug reporting wasn’t the only reason Aaron decided to leverage uTest. Other factors included speed, ease-of-use, device coverage and perhaps most important of all, location.
"We asked (uTest) for more aggressive testing in our target areas, which happen to be London and Australia," said Aaron, who noted that his criteria was easily met with the help of his dedicated uTest project manager.
From there, Aaron would begin the process of selecting a test team for their exploratory test cycles. As mentioned previously, Blendr was looking for ways to combine their beta community, their in-house QA team and uTest…so that’s exactly what they did.
"We actually invited members of our beta community to join the uTest as testers," he said. "About 20 people joined and have been testing our projects through uTest ever since. We want the solid testers that are always there, but we also want new people all the time for their fresh eyes. It’s not about having the perfect tester."
With the setup complete, uTest would go on to run an exploratory test cycle every week, covering multiple versions of iOS and Android, as well as Facebook.
Upon completion of their first test cycle, Aaron and his team would immediately realize the advantage that can only be provided by professional testers.
"With uTest, we were able to require screenshots and video attachments from the testers," said Aaron. "It’s important to us because a lot of times even if you describe something, it can still be hard to reproduce. uTest made it much easier for us to prioritize the bugs being reported."
Aside from the screenshots and videos, Aaron and his team would receive bugs sorted by frequency, type, device, OS and a host of other criteria.
"The range of bugs by device and location was very helpful. For example, we discovered some issues in London with Vodaphone that we wouldn’t have known about without uTest, because our in-house team isn’t in London, so they simply can’t capture that type of real-world experience."
-- Aaron Judd, VP of Technology, Blendr
"The complexity of our application makes real-world testing an absolute must. Between our in-house team, our beta users and uTest, we feel extremely confident about our testing efforts," said Aaron.
With dozens of projects completed – currently spanning nine different products – Blendr is confident in uTest’s ability to continually meet their in-the-wild testing needs.
"I’ve always enjoyed all the people I’ve met and worked with at uTest. All the project managers I’ve worked with have always given 110%. It’s been a great experience."