Company Size

Enterprise

Company Location

Atlanta, GA

Testing Type

Functional

App Type

Mobile & Web

Testing Coverage

Global

Numerex

"There's Nothing Like Real-World Testing"

Founded in 1992, Numerex (NASDAQ: NMRX) is revolutionizing the machine-to-machine (M2M) business-to-business market. Specializing in end to end delivery of solutions including cellular and satellite devices, networks and applications, the company strives to make the M2M experience easy, scalable and secure with a vast array of advanced technologies.

In 2011, Darrin Wilkey – the company’s Senior Director of LBS Product Management – was leading the development of a software application designed to track vehicles and assets. In terms of application complexity, this project ranked near the top, with robust functionality and lots of potential iterations depending on how users logged in. The project also had a multilingual aspect, adding to the difficulty of relying completely on internal testing.

"We searched around for (testing) solutions, and found uTest to be ideal because of their global base of testers. The fact that they could aggregate testers by language was extremely helpful. Obviously, this was not something we could have accomplished in-house."
-- Darrin Wilkey, Senior Director of LBS Product Management, Numerex

This brief case study will show how uTest was able to help Numerex with two highly complex testing projects – both for software and hardware – via their global community of professional software testers. Subject matter will include the role of uTest’s project management team; the challenge of performing global testing; testing results and more.

Numerex: Testing Setup and Scope

With a fast-paced development schedule, Darrin and his team were immediately impressed with uTest’s speed and scalability. After a short kickoff call with his dedicated uTest project manager (there’s one assigned to every account) Numerex had their testing plan formalized, a QA team selected and were ready to get started.

"We found that uTest had really great systems to manage the testing process," said Darrin. "We saw testers signing up for the project almost immediately after we finished the test plans."

Like many uTest customers, Numerex kicked things off with an exploratory test cycle, where testers were allowed to search the application without a formalized test plan. From there, his team would follow precise directions in what is considered to be a more traditional functional test cycle. In each instance, Darrin was impressed with the quantity and quality of his test cycle results.

"I was somewhat surprised by how thorough the reporting was," said Darrin. "It was not a gloss-over testing; people were very thorough, even during exploratory testing. I was amazed at how serious the testers took these assignments. There was nothing amateur about it."

In-the-Wild Hardware Testing

After nearly three months of functional testing of their software application, Darrin decided to leverage uTest’s global community for a hardware project. While this assignment required less expertise on the part of the community (they would simply install a device on their roof), it was complex in terms of geography and logistics, as Darrin’s team would need to monitor device activity in select areas around the world.

"We had devices in New Zealand, Poland, Pakistan, Spain, Australia, Brazil and Argentina," said Darrin. "Taking this type of hardware and placing it all around the world was a little daunting, but uTest really helped us streamline the process."

Previously, Numerex had leveraged a beta community for such assignments – a process Darrin found to be difficult and time-consuming.

"We’ve traditionally identified customers to be beta testers, but it’s hard to get people in all of these regions who have the time to do it," he said.

Next Steps

With two highly complex projects in the books, Numerex is confident in uTest’s ability to meet any testing challenge it encounters. In the future, as Numerex continues to push the boundaries of M2M technology, the company plans to continue leveraging uTest for its in-the-wild testing needs.

"We can simulate functionality in a lab, but there’s nothing like real world testing," he said. "We’ve had a great experience with uTest."

Open Announmcement
uTest and Appplause Banner

Heard the news?

In early 2014, uTest will be changing its name to Applause. With an expanded app quality offering including test services, tools, SDKs & integrated analytics, we will do even more to help companies delight their web & mobile users.

The uTest name will live on in our tester community, with an expanded offering for all facets of a tester’s professional life. This includes paid testing projects, hands-on training, networking, events, career resources & more.

While we’ve got a lot of work to do, we’re pumped to tackle both of these bigger, bolder challenges. Get the details on this news here.