Mobile & Web
"There's Nothing Like Real-World Testing"
Founded in 1992, Numerex (NASDAQ: NMRX) is revolutionizing the machine-to-machine (M2M) business-to-business market. Specializing in end to end delivery of solutions including cellular and satellite devices, networks and applications, the company strives to make the M2M experience easy, scalable and secure with a vast array of advanced technologies.
In 2011, Darrin Wilkey – the company’s Senior Director of LBS Product Management – was leading the development of a software application designed to track vehicles and assets. In terms of application complexity, this project ranked near the top, with robust functionality and lots of potential iterations depending on how users logged in. The project also had a multilingual aspect, adding to the difficulty of relying completely on internal testing.
"We searched around for (testing) solutions, and found uTest to be ideal because of their global base of testers. The fact that they could aggregate testers by language was extremely helpful. Obviously, this was not something we could have accomplished in-house."
-- Darrin Wilkey, Senior Director of LBS Product Management, Numerex
This brief case study will show how uTest was able to help Numerex with two highly complex testing projects – both for software and hardware – via their global community of professional software testers. Subject matter will include the role of uTest’s project management team; the challenge of performing global testing; testing results and more.
Numerex: Testing Setup and Scope
With a fast-paced development schedule, Darrin and his team were immediately impressed with uTest’s speed and scalability. After a short kickoff call with his dedicated uTest project manager (there’s one assigned to every account) Numerex had their testing plan formalized, a QA team selected and were ready to get started.
"We found that uTest had really great systems to manage the testing process," said Darrin. "We saw testers signing up for the project almost immediately after we finished the test plans."
Like many uTest customers, Numerex kicked things off with an exploratory test cycle, where testers were allowed to search the application without a formalized test plan. From there, his team would follow precise directions in what is considered to be a more traditional functional test cycle. In each instance, Darrin was impressed with the quantity and quality of his test cycle results.
"I was somewhat surprised by how thorough the reporting was," said Darrin. "It was not a gloss-over testing; people were very thorough, even during exploratory testing. I was amazed at how serious the testers took these assignments. There was nothing amateur about it."
In-the-Wild Hardware Testing
After nearly three months of functional testing of their software application, Darrin decided to leverage uTest’s global community for a hardware project. While this assignment required less expertise on the part of the community (they would simply install a device on their roof), it was complex in terms of geography and logistics, as Darrin’s team would need to monitor device activity in select areas around the world.
"We had devices in New Zealand, Poland, Pakistan, Spain, Australia, Brazil and Argentina," said Darrin. "Taking this type of hardware and placing it all around the world was a little daunting, but uTest really helped us streamline the process."
Previously, Numerex had leveraged a beta community for such assignments – a process Darrin found to be difficult and time-consuming.
"We’ve traditionally identified customers to be beta testers, but it’s hard to get people in all of these regions who have the time to do it," he said.
With two highly complex projects in the books, Numerex is confident in uTest’s ability to meet any testing challenge it encounters. In the future, as Numerex continues to push the boundaries of M2M technology, the company plans to continue leveraging uTest for its in-the-wild testing needs.
"We can simulate functionality in a lab, but there’s nothing like real world testing," he said. "We’ve had a great experience with uTest."