Researchers Find Large Number of EHRs Do Not Meet Usability Standards

A report by MedStar Health’s National Center for Human Factors in Healthcare finds that a significant percentage of electronic health record (EHR) vendors failed to meet federally mandated user-centered design requirements and did not conform to usability testing standards for their EHRs, yet their products were certified as having met all the requirements of the government’s meaningful use program for EHRs. The findings, reported today in the Journal of the American Medical Association, are based on publicly available information supplied by the EHR vendors to the Office of the National Coordinator for Health Information Technology (ONC) between April 2013 and November 2014.

The study authors are experts in human factors and safety science and have studied EHRs extensively. They are Raj Ratwani, PhD; Rollin J. (Terry) Fairbanks, MD, MS; A. Zachary Hettinger, MD, MS; and Natalie Benda, MS. The investigators studied official reports submitted by the EHR vendors to the federal government attesting to the user-centered design (UCD) process they had followed to develop their products and providing results of usability testing they had conducted. Specifically, the study focuses on the computerized order entry function, since it is primarily used by clinicians and presents significant safety hazards when not designed well. The authors conclude that enforcement of existing standards and oversight of usability processes are necessary to meet usability and safety goals for the next generation of EHRs.

The ONC put the user-centered requirements in place to help assure that EHRs are optimized for patient safety, with a focus on usability, meaning that clinicians using them to accomplish tasks find the experience to be satisfying, efficient and effective. “The reason the ONC requires EHR vendor products to be certified as having followed UCD processes—an approach that puts the cognitive workflow and information needs of the clinical user at the forefront of design and development—is to ensure EHRs support clinical staff as they perform their work and reduce the potential for error and patient harm,” says Ratwani, the Human Factor Center’s scientific director and the study’s principle author.

The MedStar study found that among the problems were failure to adequately test the usability of an EHR and failure to document that an EHR was developed with a UCD process. Among the specific findings:

  • 63 percent of the vendors whose reported results were analyzed failed to enroll the recommended number of users—at least 15— in tests on their EHRs.
  • 17 percent used no physician participants in usability testing of systems intended for physician use.
  • 12 percent of vendor reports lacked enough detail to determine whether physicians participated.
  • 34 percent of vendors did not state, as required, the UCD process they had followed.

Researchers compiled their study by examining available reports from the top 50 EHR vendors, as measured by the number of meaningful use attestations they had made between April 1, 2013, and Nov. 30, 2014.


About the National Center for Human Factors in Healthcare

The National Center for Human Factors in Healthcare occupies a unique position in the United States as a large human factors program embedded within a healthcare system. It brings together human factors scientists, systems safety engineers, health services researchers, and clinicians to conduct safety science and applied research in medicine to improve safety, quality, efficiency, and reliability. The center is part of the MedStar Institute for Innovation and is affiliated also with the MedStar Health Research Institute. MedStar Health, the parent organization, is the largest not-for-profit healthcare provider in the Maryland and Washington, D.C., region, with 10 hospitals and an extensive ambulatory services network, and is the medical education and clinical partner of Georgetown University.

For Media Contact: 

Ann Nickels
Assistant Vice President
PR & Communications
410-772-6661 
[email protected]