VML

Working with some of the biggest, most successful companies on the planet, VML is a full-service digital marketing agency comprised of “thinkers, dreamers, constructors and builders.” With more than 800 employees and offices around the globe, the company specializes in client solutions at the intersection of marketing and technology.

In 2011, as the company began developing a sophisticated web product for one of its high-profile clients, Senior Project Manager Nick Langworth realized that their testing efforts lacked a real-world component, prompting him to look for new ways of getting testing results outside of their internal lab.

“The software was passing all of the major tests internally,” said Nick. “But we weren’t entirely pleased with the real-world results. We needed a way to test our software outside of the sterile lab environment – and that’s where uTest came into play.”

This case study will detail how VML leveraged uTest for “in-the-wild” testing of a client application, including a summary of their testing requirements, testing process, the role of uTest’s project management team and other aspects of the uTest experience.

Testing Challenges

As is often the case, VML had found that their test results varied depending on the environment. While the application would pass with flying colors in the lab, the real-world results were a different story. Nick explained that certain variables (OS versions, plug-ins, etc.) would have a major effect on the application’s overall performance. After learning about uTest through one of its own partners, Nick was pleased to learn that “in-the-wild” testing was even easier than he imagined.

“The uTest community was exactly what we were looking for,” said Nick. “We needed testers – lots of testers – that weren’t affiliated with our partners and who could give us a fresh, objective perspective before we sent this product out to the general public. Basically, we needed to make sure our software worked on real machines.”

Working with his uTest project manager, Nick drafted a simple test case that would cover the following areas:

• Installation
• Exception handling
• Index verification
• Indexing time

 

After specifying other requirements in terms of Windows® operating systems, Nick’s project was accepted by more than 200 testers from all over the world, who immediately began executing the test case.