How Usability Testing Prevents Horror Stories

Posted in Testing Services by Brian Buntz on December 3, 2013

Strangled babies and irradiated patients provide some recent examples of why medical device designers cannot assume their products will be used the way they think they are going to be used, says Edith Folger, senior software quality engineer at Boston Scientific.

Folger, speaking Tuesday at BIOMEDevice San Jose, recounted Angelcare Monitors Inc.'s recent recall of 600,000 baby monitors after two reports of infants who grabbed the power cord of the device, wound it around their neck, and then suffocated. The electrical cord, which measures roughly 11 feet in length, was designed to connect the sensor pad of the device to a nursery monitor. But the consequences of its poor usability were tragic.

AngelCare monitor
The cord design of the Angelcare baby monitor was responsible for the strangulation of two infants.

Another example that highlights the importance of usability testing can be found in a CT machine at Cedar Sinai that emitted eight times the standard radiation dose. The incident, which occurred at Cedar Sinai in 2009, went unnoticed by CT technicians and radiologist for months. The machine informed technicians of the radiation dose being used in the scans, but did nothing to flag the problem.  

Folger told these horror stories to hammer home the message to assume nothing when designing medical devices and choosing technologies to be used within them. When choosing materials or software to be used in a medical device, thoroughly consider how they might cause confusion or might be inadvertently misused, Folger says.

When introducing Folger before her talk, AgaMatrix CTO Sridhar Iyengar recalled how his company ran into a usability issue while obtaining regulatory approval for its iPhone-compatible IBGstar glucose meter.

“FDA asked us: ‘How do we know that people who use this product know how to download the app from iTunes?’” Iyengar said. “Common sense would indicate that it is easy to do so. But FDA informed us that there is no data to support it.”

The company had to identify people who had never used iTunes before and organize a study to determine whether they can figure out how to download the app. “In 2009, it was not easy to find people who had never used iTunes,” Iyengar explained. But they managed to find a number of them and successfully completed the study, which also served to inform the company of the importance of usability engineering.

Usability testing should begin at the prototyping stage, and it should be considered when identifying specific requirements or specifications to address. “This doesn’t fall under the regulations so much,” Folger said. But such testing can prove to be invaluable.

CT scan
In 2009, it was revealed that hundreds of patients who underwent routine CT scans received between six to eight times the recommended radiation dose. The problem could have been prevented by programming the software to alert technicians of the problem.

This process is informed by market research, which leads to the formulation of usability requirements and specifications, Folger said. “There should be a basis from a user as to why the product was developed.” Later on, usability studies can be used when the product is more mature that can be used in the regulatory approval process.

When developing technology that has a screen interface, automation can be used to help facilitate product testing. Folger mentioned screen recorders and testing sleds—which can be used to show how people use a screen interface. Possible areas of confusion should be identified as they can lead to patient harm.

Brian Buntz is the editor-in-chief of MPMN. Follow him on Twitter at @brian_buntz and Google+.