From the Chair: Usability… Still In the Eye of the Beholder

by Karen Bell, MD, MMS on May 11, 2011

One of the reasons that “usability” of EHRs has been so elusive is that it has had no clear definition.   We’ve heard that it is a barrier to full EHR adoption, but without a sense of what it is and how it could be measured, it’s difficult to improve.    We have defined its attributes –efficiency, efficacy, and satisfaction as one carries out one’s tasks – and identified some early basic ways to assess usability as it pertains to safety.     But we have yet to come to consensus on how to achieve an environment where focused, organized, comprehensive clinical information is immediately available in the time and format that is needed and how to assure that data can be quickly and easily incorporated into the electronic record of choice.

Discussions are now in progress at both federal and state levels on the topic, and the National Institute of Standards and Technology (NIST) has been working on a Usability Framework that will be shared in early June.   National Coordinator for Health IT, Dr. Farzard Mostashari, has signaled the need to progress in this arena, and to do so with a goal of improvement…not a goal of measurement for certification or other qualifying purposes.   The Certification/Adoption Work Group of the federal Health IT Policy Committee has heard a full day’s worth of testimony on usability in preparation for making recommendations to the full Committee.    As a result of this activity and the work done to date, we now have a foundation on which to build:

  1. A usable system has access to comprehensive data and integrates all elements of data input, storage, and use.   This requires interoperability and assurance that all components (or modules) of the system are integrated, as is the case for CCHIT Certified® 2011 products.
  2. Usability refers to how an EHR supports clinical workflow — not a unique measure, objective, or criteria.   Including practicing clinicians in the process of determining and assessing usability is therefore critical to its success.
  3. Tools will be required that can quickly find and filter accurate data that is specific to the user and presented as information that can be organized to support real time patient care in different settings   –  one size does not fit all situations or exam rooms.
  4. Usability testing will change as hardware and operating systems evolve –  what may be applicable in a Windows desk top environment may  not work in an  iPad or other mobile smart device environment.
  5. Usability must address the access needs of persons with disabilities.
  6. Attention to patient communications (both efferent and afferent) will need to be incorporated into the usability framework.  Use of EHRs affects the patient-clinician interaction.
  7. Vendors understand that usability will provide a marketing edge.    Many of the larger more stable vendors have already invested in usability programs in house with clear principles and practices.   These programs are not, however, transparent, and many products are reconfigured to meet customized needs at installation, necessitating assessment in the working environment.
  8. User centric design and human factors research are robust scientific fields that can guide the development of usable systems, as well as add to the current measurement base.
  9. Usability testing should result in demonstrated improvement in specific outcomes, such as measures of patient safety, provider productivity, and clinician satisfaction with workflows.
  10. Usability will likely be improved as we migrate to a “plug and play” EHR environment, where one individual workflow application can be exchanged for another which will integrate with those already in place.

So…what can we expect in the coming months?    Clearly, we will be seeing greater access to interoperable health data as one of the key components of usability.   We can also expect NIST to continue to refine and develop a framework based on human factors research and user centric design that will guide the development of more robust usability testing.    I hope that we will see ONC Certification Criteria move toward integration testing and criteria that are oriented toward clinical workflow.    Ultimately, new applications and approaches will evolve and usability testing will evolve along with them.    In the meantime, CCHIT will continue to conduct basic patient safety focused usability testing that has been validated and demonstrated to have inter-rater reliability on all CCHIT Certified® 2011 ambulatory EHR products.

Karen M. Bell, MD, MMS
Chair, Certification Commission

Karen Bell, MD, MMS, is Chair of the Certification Commission for Health Information Technology (CCHIT®. Dr. Bell has wide and varied expertise in health information technology (HIT), quality assurance and clinical practice, in both the private and public sectors. Previously, she served as Senior Vice President, HIT Services, Masspro, the federally-contracted Quality Improvement Organization within Massachusetts, where she oversaw the development, implementation and distribution of products and services to support adoption of electronic health records (EHRs) within the health care system. Between 2005 and 2008, Dr. Bell was Director, Office of Health Information Technology Adoption, Office of the National Coordinator (ONC), U.S. Department of Health and Human Services (HHS), and, in 2006, served as Acting Deputy of ONC. She was ONC’s representative on CCHIT’s Board of Commissioners from 2006 to 2008.

Prior appointments held by Dr. Bell include Division Director, Quality Improvement Group/Office of Standards and Quality for the Centers for Medicare and Medicaid (CMS), and Medical Director of Blue Cross Blue Shield (BCBS) of Rhode Island and of Anthem BCBS of Maine.

She received her medical degree from Tufts University School of Medicine, Boston, and her master of medical science degree from Brown University, Providence, R.I. Dr. Bell has clinical experience as a board certified physician in internal medicine and also was an Associate Professor at the University of Rochester, and Clinical Instructor at Harvard University School of Medicine.

{ 3 comments }

Part three of David Tao’s certification lessons learned

by admin on May 5, 2011

David Tao continues sharing his experiences with CCHIT and the certification process. In this edition, he describes four prototypical controversies he faced in content, vocabulary, and transport.

I could go on with more examples. That’s part of the challenge of interoperability work, as evidenced by the fact that ONC and the HIT Standards Committee still have all of the above issues to ponder for Stages 2 and beyond. I should mention that one more big controversy arose in the very first year, when CCHIT was newly formed and struggling to figure out how to set the interoperability bar. Eyes were upon those first members to resolve the CDA vs. CCR debate. Unlike the four issues above, where we took a stand, we weren’t ready to force a decision in 2006, and then HITSP was formed by ONC to deal with such “standards harmonization” issues. So at least our punt had a receiver (and football wasn’t locked out)!

Read the entire post.

{ 0 comments }

David Tao continues his retrospective on certification lessons learned

by CCHIT Staff on April 28, 2011

More from David Tao on his experiences with CCHIT and the certification process.

One thing I learned is that no group can please all the people all the time. Sometimes, people wondered why we proposed some criteria and not others, or proposed the timeframes that we did. So we tried to explain these “whys” in a white paper Interoperability, Supplying the Building Blocks for a Patient-centered EHR in mid-2009 which still exists on the website of writer John Morrissey. It described the context for what we were doing and the “methodical march” to increasing levels of information interoperability that we proposed. I don’t think this was well understood by the public, who might have thought some of what we did was arbitrary. But here’s what was in our mind (well, I can only speak for myself – mine at least).

Read David’s insights. Or start with Part 1 of this series.

{ 4 comments }

David Tao shares certification lessons learned

by CCHIT Staff on April 26, 2011

Seasoned CCHIT Work Group member, David Tao will be publishing some of his thoughts and lessons learned about certification.

From the first post in the series.

It’s been a long time since I blogged, even though there have been many HIT topics to blog about, such as Stage 2 meaningful use, the ONC Standards and Interoperability Framework, the ONC Strategic Plan, the NPRM on Medicare Shared Savings (Accountable Care Organizations), and more. I’m getting back on the wagon. One topic I’ve wanted to blog about for a while, even though it’s not a current “hot topic” in the news, is my experience in the Certification Commission for Health Information Technology (CCHIT) prior to the establishment of the ONC certification program. This bubbled to the surface when I read a HISTalk “Time Capsule” posting. So I’ll write another mini-series (probably two or three parts). This first post will simply introduce the “cast of characters” and how they interacted. The next posts will talk about what the Interoperability Workgroup of CCHIT did, and how its work through 2009 related to the ONC-ATCB certification program as we know it today. There are useful lessons learned.

Read the entire post.

{ 0 comments }

Testimony before the Certification/Adoption Work Group Health Information Technology Standards Committee

by CCHIT Staff on April 22, 2011

Dr. Karen Bell, CCHIT Chair, delivered testimony to the the Certification/Adoption Work Group Health Information Technology Standards Committee on April 21, 2011, regarding usability:

In keeping with CCHIT’s mission, we first undertook the development of usability rating in 2008, launching it with our independently developed, voluntary CCHIT Certified® 2011 Ambulatory EHR program in October 2009. With the release of this first version of usability testing, we recognized that more sophistication would need to be added to the program in the future. Adapting a description developed by the National Institute of Standards & Technology (NIST) in 2007, we defined usability as the effectiveness, efficiency, and satisfaction with which the intended users can achieve their tasks in the intended context of product use. In the development of its “first-step” usability rating instrument, CCHIT endeavored to reflect the perceived usability of an ambulatory EHR application as rated by its content experts (i.e., 3 trained and experienced testing jurors that include at least one practicing physician) during an inspection process that follows a typical clinical workflow.

This focus on inspecting an EHR following a comprehensive clinical workflow differentiates our CCHIT Certified programs from the current ONC-ATCB testing, which does not, since ONC testing is meant to be executed criteria by criteria to allow certification of EHR technology modules for which there is no expected integration testing. We therefore believe that a usability testing process should be based on and limited to an integrated testing approach to certification.

Download the testimony (PDF).

{ 0 comments }

Technical issues with CHPL? Please let ONC know

by CCHIT Staff on April 18, 2011

ONC has notified us that they are having problems with EHR technology display functions on their Certified Health IT Product List (CHPL).

We have already notified ONC of issues. Vendors should report any problems—with specific examples—to ONC.certification@hhs.gov with “CHPL” in the subject line.

ONC is working to resolve these issues.

{ 0 comments }

From the Chair: Interoperability 101

by Karen Bell, MD, MMS on April 11, 2011

If you practice medicine, you need the latest knowledge, tools and instruments specific to your specialty….and ready access to as much relevant information as possible about each patient. The latter is the driving force behind the current push for you to adopt and use an electronic health record (EHR) as part of your basic set of tools. Like your other major technical investments, your EHR must meet your needs for functionality. It must also possess state of the art security and be “interoperable” to the extent possible. At present, however, true interoperability – the ability to work with and communicate with other sources of information that give you the full picture of your patients’ health and history—remains the holy grail of medical informatics.

Why is interoperability such a challenge? And when can you expect to “plug and play”, meaning easily get information to and from any lab that is convenient for your patient, get a feed from a patient’s Holter monitor while he is wearing it, or see a current problem list to which all of your patient’s doctors have contributed? The challenges to achieving this type of data access are both technical and cultural.

The key to understanding interoperability on the technical side is to understand the various categories of standards necessary for disparate electronic information sources to communicate reliably and usefully – vocabulary or terminology standards, messaging standards, and a standard approach to implementation guidance. Using the transmission of lab orders and results as an example, we need a consistent vocabulary that goes far beyond the name of the test. We need standard terminology for everything related to the patient, for information identifying you as the ordering clinician, for presentation of results, for how the normal ranges are presented, for qualifying information about the test, for example, was the sample hemolyzed), and for qualifying information about the result such as recommendations for a follow up test. At present, we have standard terminologies for many of the actual test names, but little else.

Messaging standards determine how the information or data elements are packaged for secure travel over the internet. The good news is that there are a limited but sufficient number of messaging standards at present and a lot of work has been done to date to select the one that is most appropriate for a given situation. Continuing with our example with respect to lab data, the HIT community has agreed that HL7 (Health Level 7) is the most appropriate approach for messaging laboratory related information.

The most difficult step in the technical process is standards for implementation guidance. Simple translation: when the data are delivered to a disparate EHR system, implementation guides allow the second system to accept and integrate them in a way that makes them as presentable to the clinician as if it were entered locally. Implementation guides are specific for different types of data and need to work for all types of EHRs. While initial work on implementation guides for several types of lab data has commenced, a lot more work is necessary in this area before the industry can achieve true interoperability for lab data and information. Meanwhile, implementation guidance still needs to be written for many other types of data.

One frequently hears the terms “syntactic interoperability” and “semantic interoperability.” The former simply refers to the basic capability to communicate and exchange information and is the first step. Semantic interoperability is the ability to automatically interpret the data in the information exchanged meaningfully and accurately in order to produce useful results as defined by the end users of both systems. Much of the interoperability currently available clinically is in the first category: summary documents can be exchanged, opened, read, and stored. The data elements in these documents, however, may need to be extracted and manually keyed into the new system if they are to be used independent of the initial document.

The cultural challenges to interoperability may be even more daunting than the technical challenges. There needs to be both demand and a business case on the part of participants and the technology vendors to make the necessary investments in interoperability and put it to use. Laboratories with multiple different technological platforms will require a return on investment if they need to replace these platforms in order to be interoperable with EHRS. Providing data from a home based system that monitors anticoagulant effect to a clinician who is responsible for dosing will require that the clinician be compensated in some way for the time and expertise necessary to appropriately monitor the patient virtually. Lastly, and especially relevant to laboratory data, we need to assure that our laws and regulations allow for and support the exchange of electronic data. CLIA, the Clinical Laboratory Information Act, has been a barrier to health information exchange between clinical labs and others who need access to laboratory information. This Federal legislation has been under review for several years with the intent to issue guidance or modify its restrictions so as to allow information to flow where it is needed, not just to the ordering clinician.

Efforts in all these areas have intensified with the passage of ARRA (the American Recovery and Reinvestment Act of 2009). We still, however, have a long way to go to achieve true semantic interoperability in the healthcare industry. This should not stop us, however, from taking advantage of what information can be exchanged now, which includes patient summary data and documents. In order to facilitate this exchange, the Office of the National Coordinator for Health Information Technology (ONC), an agency of HHS, has developed a set of standards, services, and policies which allows participating providers to send but not query, find, and access, patient summary information and documents to known and trusted recipients. Known as the Direct Project, this set can be used by clinicians or institutions, public health departments, and laboratories to “push” information to receiving entities. A number of Health Information Exchange (HIE) organizations at local, regional, and statewide levels are also exchanging summary documents and public health information. Check with your local medical society, Regional Extension Center (REC), or HIE program to learn more about how you may participate in information exchange, using your certified EHR.

Karen M. Bell, MD, MMS
Chair, Certification Commission

Karen Bell, MD, MMS, is Chair of the Certification Commission for Health Information Technology (CCHIT®. Dr. Bell has wide and varied expertise in health information technology (HIT), quality assurance and clinical practice, in both the private and public sectors. Previously, she served as Senior Vice President, HIT Services, Masspro, the federally-contracted Quality Improvement Organization within Massachusetts, where she oversaw the development, implementation and distribution of products and services to support adoption of electronic health records (EHRs) within the health care system. Between 2005 and 2008, Dr. Bell was Director, Office of Health Information Technology Adoption, Office of the National Coordinator (ONC), U.S. Department of Health and Human Services (HHS), and, in 2006, served as Acting Deputy of ONC. She was ONC’s representative on CCHIT’s Board of Commissioners from 2006 to 2008.

Prior appointments held by Dr. Bell include Division Director, Quality Improvement Group/Office of Standards and Quality for the Centers for Medicare and Medicaid (CMS), and Medical Director of Blue Cross Blue Shield (BCBS) of Rhode Island and of Anthem BCBS of Maine.
She received her medical degree from Tufts University School of Medicine, Boston, and her master of medical science degree from Brown University, Providence, R.I. Dr. Bell has clinical experience as a board certified physician in internal medicine and also was an Associate Professor at the University of Rochester, and Clinical Instructor at Harvard University School of Medicine.

{ 2 comments }

MGMA releases 2011 EHR Survey report

by CCHIT Staff on April 6, 2011

The MGMA has released results from its annual EHR survey.

The use of electronic health records (EHRs) by medical practices and other organizations in the United States has increased measurably in the past decade. Despite the potential to improve the quality of patient care and enhance practices’ financial performance, the technology remains far from universal.

To better understand the current state of EHR use, MGMA conducted a study funded by PNC Bank to explore the barriers and benefits of EHR adoption. MGMA collected data between Oct. 1, 2010, and Nov. 9, 2010, from 4,588 healthcare organizations nationwide that responded to the survey. The data represent the aggregate experience of more than 120,000 physicians in medical practice.

To download your copy, visit the MGMA 2011 EHR survey.

{ 0 comments }

Back to the future

by Sue Reber on April 5, 2011

I enjoyed the recent reminiscence in HISTALK of a reprinted editorial from January 2006 describing the beginning days of EHR certification. It was written even before CCHIT launched its first certification program in May 2006, a full five years ago. We would go on to announce the first CCHIT Certified EHR products in July.

That was in the heady days of CCHIT’s “start-up” phase when–like all new ventures operating on a lot of passion and not much funding–neither our small staff nor our dedicated volunteers could predict where this would eventually take us. So here’s a little recounting.

First, I have to chuckle at the early reference to our name and its unfortunate phonetic pronunciation. While I’ve spent the last six years trying to get folks to say C-C-H-I-T, I long ago accepted that this marketing faux pas is the result of hiring the marketing director after the founders–all brilliant physicians and IT executives—who came up with an elegant name with initials not even a mother could love.

C’est la vie. You’ve got to be prepared to laugh at yourself.

Much has changed since those early days. We’ve seen certification blossom from begrudging acceptance to deeper Federal involvement with associated financial rewards. During that time, the industry has truly moved from stick to carrot. The work that CCHIT pioneered beginning in 2005 has grown to become a prerequisite, now managed by the government, for federal incentive funding intended to further encourage health IT adoption.

CCHIT continues to develop its own programs with its broadly representative volunteer stakeholders to fill in the gaps left by the government’s program. Those gaps are most notable in special care settings–behavioral health and long term and post acute care–and in special populations or medical practices–child heath, women’s health, cardiovascular medicine, dermatology and oncology–or for special purposes, such as clinical research.

Some things stay the same. We’ve certified more than 400 individual products, including both those in our independently developed CCHIT Certified® program and those in the newer ONC Authorized Testing and Certification Body (ONC-ATCB) program. Yet this comment: “Some vendors are complaining about the cost of certification and interference with their business” is still largely true. Most vendors, however, would probably agree that product certification has now become a necessity for competing in the health IT market. And most, if not all, buyers know to ask about certification.

Some things that this missive from the past asked for as additions to the first proposed CCHIT program have been added incrementally:

  • “Release the individual scoring sheets of the products it evaluates.” This was never necessary since all CCHIT Certified products meet 100 percent of the criteria in a juried pass/fail inspection. Clinicians shouldn’t have to worry about which EHR capability isn’t there or isn’t working. For, ONC-ATCB products, each tested criteria is listed separately on our product pages so buyers can easily see if it’s a complete EHR or an EHR module.
  • “Add a category for patient safety.” Though this category was not called out separately in CCHIT development activity after 2006, many criteria were added in subsequent years to support this need. Additional attention was given to CPOE, the medication administration record and medication reconciliation, alerts and reminders, performance and quality reporting, and exchange of information across care settings. CCHIT does not certify EHR implementations but, in its independent program, it verifies with live reference sites that key capabilities are implemented and working properly before full certification is granted.
  • “Add criteria for usability.” With the CCHIT Certified 2011 Ambulatory EHR program, launched in October 2009, we began requiring basic usability testing, allowing vendors to opt-in or out of reporting their usability score. There was great concern about how objective this testing would be but, with a clear Usability Testing Guide, most vendors embraced the opportunity to demonstrate these first steps in improving the usability of their products. Later this month, CCHIT has been invited to testify at a hearing held by the Certification/Adoption Workgroup of the HIT Policy Committee on usability of EHRs to help the committee formulate recommendations to the National Coordinator for Health IT for inclusion of usability testing in the government’s program.

Finally, I would agree with the writer’s original assessment from 2006: “Damned if CCHIT’s work groups didn’t go off and actually get some real work done that will have a significant impact on the industry.”

That work hasn’t stopped.

{ 1 comment }

Meaningful Yoose Rap

by CCHIT Staff on March 15, 2011

{ 5 comments }