ONC Health IT Certification Program: Enhanced Oversight and Accountability

Federal RegisterOct 19, 2016

Ask Donna

What actually matters in this document.

Text

DEPARTMENT OF HEALTH AND HUMAN SERVICES

Office of the Secretary

45 CFR Part 170

RIN 0955-AA00

ONC Health IT Certification Program: Enhanced Oversight and Accountability

AGENCY:

Office of the National Coordinator for Health Information Technology, Department of Health and Human Services.

ACTION:

Final rule.

SUMMARY:

This final rule finalizes modifications and new requirements under the ONC Health IT Certification Program (“Program”), including provisions related to the Office of the National Coordinator for Health Information Technology (ONC)'s role in the Program. The final rule creates a regulatory framework for ONC's direct review of health information technology (health IT) certified under the Program, including, when necessary, requiring the correction of non-conformities found in health IT certified under the Program and suspending and terminating certifications issued to Complete EHRs and Health IT Modules. The final rule also sets forth processes for ONC to authorize and oversee accredited testing laboratories under the Program. In addition, it includes provisions for expanded public availability of certified health IT surveillance results.

DATES:

These regulations are effective December 19, 2016.

The incorporation by reference of the publication listed in the rule is approved by the Director of the Federal Register as of December 19, 2016.

FOR FURTHER INFORMATION CONTACT:

Michael Lipinski, Office of Policy, Office of the National Coordinator for Health Information Technology, 202-690-7151.

SUPPLEMENTARY INFORMATION:

Commonly Used Acronyms

CAP Corrective Action Plan

CEHRT Certified Electronic Health Record Technology

CFR Code of Federal Regulations

CHPL Certified Health IT Product List

EHR Electronic Health Record

HHS Department of Health and Human Services

HIT Health Information Technology

ISO/IEC International Organization for Standardization/International Electrotechnical Commission

NVLAP National Voluntary Laboratory Accreditation Program

OMB Office of Management and Budget

ONC Office of the National Coordinator for Health Information Technology

ONC-ACB ONC-Authorized Certification Body

ONC-ATCB ONC-Authorized Testing and Certification Body

ONC-ATL ONC-Authorized Testing Laboratory

PoPC Principles of Proper Conduct

Table of Contents

I. Executive Summary

A. Purpose of Regulatory Action

B. Summary of Major Provisions

1. ONC Direct Review of Certified Health IT

2. ONC-Authorized Testing Laboratories

3. Transparency and Availability of Identifiable Surveillance Results

C. Costs and Benefits

1. Costs

2. Benefits

II. Provisions of the Final Rule

A. ONC's Role Under the ONC Health IT Certification Program

1. Review of Certified Health IT

a. Authority and Scope

(1) Requirements of the Program

(2) Review of Uncertified Capabilities

(3) Scope of Review

b. ONC-ACB's Role

c. Review Processes

(1) Notice of Potential Non-Conformity or Non-Conformity

(2) Corrective Action

(3) Suspension

(4) Termination

(5) Appeal

d. Consequences of Certification Termination

(1) Certification Ban, Recertification, and Heightened Scrutiny

(2) ONC-ACB Response to a Non-Conformity

2. Establishing ONC Authorization for Testing Labs Under the Program; Requirements for ONC-ATL Conduct; ONC Oversight and Processes for ONC-ATLs

a. General Comments on the ONC-ATL Approach

b. Regulatory Provisions for Inclusion of ONC-ATLs in the Program

(1) § 170.501 Applicability

(2) § 170.502 Definitions

(3) § 170.505 Correspondence

(4) § 170.510 Type of Certification

(5) § 170.511 Authorization Scope for ONC-ATL Status

(6) § 170.520 Application

(7) § 170.523 Principles of Proper Conduct for ONC-ACBs

(8) § 170.524 Principles of Proper Conduct for ONC-ATLs

(9) § 170.525 Application Submission

(10) § 170.530 Review of Application

(11) § 170.535 ONC-ACB Application Reconsideration

(12) § 170.540 ONC-ACB Status

(13) § 170.557 Authorized Certification Methods

(14) § 170.560 Good Standing as an ONC-ACB

(15) § 170.565 Revocation of ONC-ACB Status

(16) § 170.570 Effect of Revocation on the Certifications Issued To Complete EHRs and Health IT Module(s)

B. Public Availability of Identifiable Surveillance Results

III. National Technology Transfer and Advancement Act and the Office of Management and Budget Circular A-119

IV. Incorporation by Reference

V. Collection of Information Requirements

A. ONC-AA and ONC-ACBs

B. ONC-ATLs

C. Health IT Developers

VI. Regulatory Impact Statement

A. Statement of Need

B. Alternatives Considered

C. Overall Impact

1. Executive Orders 12866 and 13563—Regulatory Planning and Review Analysis

a. Costs

(1) Costs for Health IT Developers To Correct a Non-Conformity Identified by ONC

(2) Costs for ONC and Health IT Developers Related to an ONC Inquiry Into Certified Health IT Non-Conformities and ONC Direct Review

(3) Costs for Health IT Developers and ONC Associated With the Appeal Process Following a Suspension/Termination of a Complete EHR's or Health IT Module's Certification

(4) Costs for Health Care Providers to Transition to Another Certified Health IT Product When the Certification of a Complete EHR or Health IT Module That They Currently Use Is Terminated

(5) Costs for ONC-ATLs and ONC Associated With ONC-ATL Accreditation, Application, Renewal, and Reporting Requirements

(6) Costs for ONC-ATLs and ONC Related to Revoking ONC-ATL Status

(7) Costs for ONC-ACBs To Submit Identifiable Surveillance Results to the CHPL

(8) Total Annual Cost Estimate

b. Benefits

c. Accounting Statement and Table

2. Regulatory Flexibility Act

3. Executive Order 13132—Federalism

4. Unfunded Mandates Reform Act of 1995

Regulation Text

I. Executive Summary

A. Purpose of Regulatory Action

The ONC Health IT Certification Program (“Program”) was first established as the Temporary Certification Program in a final rule published on June 24, 2010 (“Temporary Certification Program final rule” (75 FR 36158)). It was later transitioned to the Permanent Certification Program in a final rule published on January 7, 2011 (“Permanent Certification Program final rule” (76 FR 1262)). Since that time, we have updated the Program and made modifications to the Program through subsequent rules as discussed below.

In November 2011, a final rule established a process for ONC to address

instances where the ONC-Approved Accreditor (ONC-AA) has engaged in improper conduct or has failed to perform its responsibilities under the Program (76 FR 72636). In September 2012, a final rule (“2014 Edition final rule” (77 FR 54163)) established an edition of certification criteria and modified the Program to, among other things, provide clear implementation direction to ONC-Authorized Certification Bodies (ONC-ACBs) for certifying Health IT Modules to new certification criteria. On September 11, 2014, a final rule provided certification flexibility through the adoption of new certification criteria and further improvements to the Program (“2014 Edition Release 2 final rule” (79 FR 54430)). On October 16, 2015, the Department of Health and Human Services (HHS) published a final rule that identified how health IT certification can support the establishment of an interoperable nationwide health information infrastructure through the certification and adoption of new and updated vocabulary and content standards for the structured recording and exchange of health information (“2015 Edition final rule” (80 FR 62602)). The 2015 Edition final rule modified the Program to make it open and accessible to more types of health IT, and health IT that supports various care and practice settings. It also included enhanced surveillance, disclosure, and other requirements. These requirements were designed to support the reliability of health IT certified under the Program and increase the transparency of information about such health IT (referred to as “certified health IT” throughout this final rule).

With each Program modification and rule, we continue to address stakeholder concerns, provide additional guidance, and improve oversight. In keeping with this approach, in the “ONC Health IT Certification Program: Enhanced Oversight and Accountability” proposed rule (81 FR 11056) (“Proposed Rule”) we put forth several new proposals for comment, based on feedback from stakeholders and our own experience administering the Program. Importantly, we explained that the adoption and use of certified health IT has increased significantly since the Program was established, and that this trend will continue, including for settings and use cases beyond the Medicare and Medicaid EHR Incentive Programs (“EHR Incentive Programs”). As certified health IT becomes more integral to the delivery of care, and as certified capabilities increasingly interact with other capabilities in certified health IT and with other products, we seek to strengthen oversight of the performance of certified health IT capabilities and ensure that concerns within the scope of the Program continue to be appropriately addressed.

We explained in the Proposed Rule that we had delegated authority to ONC-ACBs to issue certifications for health IT on our behalf through the Permanent Certification Program final rule (81 FR 11057). In addition to issuing and administering certifications, ONC-ACBs are responsible for conducting ongoing surveillance to assess whether certified health IT continues to conform to the requirements of the Program. An ONC-ACB's surveillance encompasses conformity assessments based on adopted certification criteria as well as certain other regulatory requirements (

e.g.,

§§ 170.523(k) and (l)). However, under this approach, which is consistent with customary certification programs and International Organization for Standardization/International Electrotechnical Commission 17065:2012 (ISO/IEC 17065),

ONC-ACBs do not have the responsibility to address the full range of requirements applicable to health IT certified under the Program. For example, an ONC-ACB's conformity assessment may not encompass certain interactions among certified capabilities and other capabilities or products that are not certified under the Program. Similarly, an ONC-ACB's assessment of certified capabilities may be limited to certain functional outcomes and may not encompass the combined or overall performance of certified health IT in accordance with Program requirements. Separately, in some instances an ONC-ACB may be responsible for administering Program requirements but may be unable to do so effectively due to practical challenges. In contrast, ONC is well-positioned to review certified health IT against the full range of requirements under the Program. Therefore, to enhance Program oversight and the reliability and safety of certified health IT, we have finalized provisions of the Proposed Rule that set forth a regulatory framework for ONC to directly review certified health IT and take appropriate responsive actions to address potential non-conformities and non-conformities.

The international standard to which ONC-ACBs are accredited (

see also

§ 170.102). To meet these certification criteria, health IT must be able to perform required specifications and capabilities and, more generally, to do so in an accurate and reliable manner. For example, health IT certified to § 170.315(a)(1) (Computerized provider order entry—medications) must “[e]nable a user to record, change, and access medication orders.” Satisfying this criterion also plainly requires that the health IT perform this function accurately and reliably. For example, when a user enters a medication order for a patient, the health IT must accurately record the medication ordered and associate it with the patient selected by the user. Similarly, when a user accesses a list of medication orders for a particular patient, the health IT must not display medication orders for a different patient.

While certification criteria define the required capabilities of certified health IT, ensuring that health IT can perform certified capabilities is not the only requirement of the Program. In the 2015 Edition final rule, we adopted Program requirements under which developers must disclose on an ongoing basis any known material information about limitations and additional types of costs associated with any certified capabilities of their health IT (80 FR 62720). We have also adopted other Program requirements such as those related to the use of the ONC Certification and Design Mark and the submission of complaints to ONC-ACBs (§ 170.523(l) and (n), respectively). Developers must also submit

information that is required to be included on the CHPL (80 FR 62725).

Finally, in previous rulemakings we have highlighted that there are certain overarching requirements of the Program, in addition to those described above, that are necessary to ensuring its basic integrity and effectiveness (

see, e.g.,

80 FR 62710 n.170), thereby ensuring that the National Coordinator can meet his or her responsibilities under section 3001(b) of the PHSA. These requirements are part of the bases on which other requirements of the Program are understood and assessed.

A prime example is the duty of developers who participate in the Program to cooperate with the surveillance of their certified health IT. The Permanent Certification Program final rule incorporated requirements for ONC-ACBs to conduct surveillance to ensure that certified health IT continues to conform to the requirements of certification when it is implemented “in the field” (76 FR 1282). More recently, in the 2015 Edition final rule, we expanded these surveillance requirements and also stated our expectations for the performance of certified health IT in production environments. We explained that health IT developers have a responsibility to make their certified capabilities available to purchasers and users in a manner that allows them to be used for their intended purposes, including any uses reasonably within the scope of the health IT's certification (80 FR 62710). We stated that health IT would no longer conform to the requirements of its certification if customers or users were restricted from successfully implementing and using the technology for any purpose contemplated by the certification criteria to which the technology was certified (80 FR 62711). As an illustration, we said that a developer's failure to supply training materials and instructions necessary to access and successfully use data export capabilities described by § 170.315(b)(6) would constitute a non-conformity (80 FR 62711). Similarly, technical or other limitations that substantially interfere with the ability to access or use certified capabilities (or any aspect or intended uses of such capabilities) would give rise to a non-conformity (80 FR 62711). Further, even in the absence of any actual impairment, if a developer's actions would be likely to substantially impair the ability of one or more users (or prospective users) to implement or use certified capabilities for any purpose within the scope of applicable certification criteria, the technology would no longer conform to the requirements of its certification (80 FR 62711). Thus, we explained that the failure to disclose known material information about limitations or types of costs associated with certified health IT not only violates the express disclosure requirements at § 170.523(k)(1), but also constitutes a non-conformity to the certification criteria associated with the potentially affected capabilities (80 FR 62711).

Consistent with these established principles under the Program, certified health IT must be designed and made available to users in ways that allow certified capabilities to be used in an accurate and reliable manner, including in a manner that does not cause or contribute to serious risks to public health or safety or to other outcomes that are inconsistent with the National Coordinator's responsibilities under section 3001(b) of the PHSA. This requirement applies to the use of certified capabilities individually and in combination with other certified and uncertified capabilities of health IT. Just as the failure to disclose known material limitations or types of costs may impair the use of certified capabilities, the failure to design and make certified capabilities available so that they perform in an accurate and reliable manner impairs the safe and effective use of certified capabilities and is a non-conformity under the Program.

It is important to note that the foregoing examples and analysis assume that the putative non-conformity is a result of the actions of the developer or factors that are reasonably within the developer's ability to influence or control. As we have explained on prior occasions, a non-conformity does not arise when certified health IT fails to perform in an acceptable manner but where the failure is the result of factors that are far removed from the control or responsibility of the developer (80 FR 62710).

These principles are further elaborated and applied in the responses to specific comments throughout the remainder of this section (II.A.1.a) of the final rule. We have also included numerous examples to assist readers in understanding these principles and how ONC would apply them in particular circumstances.

Comments.

Many commenters believed that ONC should review certified health IT against specific standards, implementation specifications, certification criteria, or other express requirements, preferably developed through formal rulemaking; otherwise, developers would have insufficient guidance to design and implement their products in a manner that complies with Program requirements, and any determinations made by ONC could be ad hoc and have the potential to be unfairly applied. For these reasons, several commenters urged us to initiate separate rulemaking to identify and adopt new certification criteria that would prescribe specific requirements that ONC would apply when reviewing certified health IT and determining whether it conforms to Program requirements.

Response.

These comments raise many of the same concerns expressed in comments on the 2015 Edition proposed rule regarding then-proposed requirements for ONC-ACBs to conduct in-the-field surveillance of certified health IT. As we explained in finalizing those requirements, we understand the desire for bright-line rules; yet experience suggests that the fast-paced nature of technological change in the health IT landscape makes it impracticable to anticipate and prescribe detailed rules for every conceivable situation in which health IT may not conform to Program requirements (

see

80 FR 62709). In practice, certified health IT may be integrated with a wide range of other systems, processes, and workflows and may be customized and used in many different ways. These circumstances, which are inherent to the production environment, are too numerous and varied to anticipate or to reduce to simple rules of universal application.

For the same reasons, we do not believe that adopting certification criteria would provide the clarity or certainty sought by advocates of that approach. We believe that clarity and predictability are best achieved by articulating and explaining the basic principles that govern our review of certified health IT, as we have done in our previous response above and in the examples and discussion of potential non-conformities throughout this section of the preamble. These principles are consistent with those that govern an ONC-ACB's surveillance of certified health IT in the field (80 FR 62709). As such, they will ensure that ONC's review of certified health IT is consistent and based on clear and predictable principles.

Comments.

Multiple commenters stated that a non-conformity should be defined as occurring only when certified health IT can no longer complete or repeat the certification test procedures against which it was previously tested and on the basis of which the health IT was certified.

Response.

We expressly rejected these arguments in the preamble to the 2015 Edition final rule (80 FR 62709). There, we explained that an ONC-ACB's

assessment of certified health IT in the field is not limited to aspects of the technology that were tested in a controlled environment. Rather, an ONC-ACB must consider the unique circumstances and context in which the certified health IT is implemented and used in order to properly assess whether it continues to perform in a manner that complies with its certification.

Testing is an important part of an ONC-ACB's overall analysis of health IT under the Program. For practical reasons, however, testing focuses on particular use cases and necessarily reflects assumptions about how capabilities will be implemented and used in practice. Thus, while test results provide a preliminary indication that health IT meets the requirements of its certification and can support the capabilities required by the certification criteria to which the technology was certified, that determination is always subject to an ONC-ACB's ongoing surveillance, including the ONC-ACB's evaluation of certified capabilities in the field. Indeed, a fundamental purpose of in-the-field surveillance is to identify deficiencies that may be difficult to anticipate or that may not become apparent until after certified health IT is implemented and used in a production environment. That purpose would be entirely frustrated if an ONC-ACB's assessment of technology in the field were confined to those aspects of the technology's performance specifically delineated in test procedures.

For these same reasons, we again reject the position that Program requirements should be rigidly defined by test procedures instead of more meaningful performance outcomes. In assessing putative non-conformities in the course of ONC direct review, we consider the unique circumstances and context in which the certified health IT is implemented and used in order to properly assess whether it continues to perform in a manner that complies with the Program (

see,

80 FR 62709).

Comments.

Several commenters observed that the performance of health IT may be impacted by providers' implementation choices or other factors that the developer of the health IT may be unable to reasonably anticipate or control. One commenter explained that health IT developers do not necessarily control which third-party products their customers may deploy in conjunction with the developer's certified health IT and that it is not unusual for interface issues to arise because of updates to these unsupported products or uses. Commenters noted that developers may find it particularly difficult to anticipate and address interactions of their certified health IT with third-party products that are not certified under the Program or with capabilities or aspects of certified health IT that are not directly governed by certification criteria.

Response.

In the 2015 Edition final rule, we recognized there may be instances in which the failure of certified health IT to perform required capabilities in the field may be due to factors that are beyond the ability of the health IT's developer to reasonably influence or control (80 FR 62710). Because the requirements of the Program focus on the responsibilities of health IT developers and those aspects of their technology that they can reasonably influence or control, we explained that the failure of health IT to perform in an acceptable manner would not constitute a non-conformity if the failure was caused exclusively by factors far removed from the control or responsibility of the developer.

We also explained that, in evaluating non-conformities in the field, ONC-ACBs are required to determine the reasons for the failure of health IT to function in an acceptable manner, taking into account the roles of the technology as well as the health IT developer, users, and other parties. If an ONC-ACB finds that the developer or its technology were a substantial cause of the failure, the ONC-ACB would conclude that the health IT does not meet the requirements of its certification. By contrast, if the ONC-ACB finds that the failure was caused exclusively by factors far removed from the control or responsibility of the developer, the ONC-ACB would regard those factors as beyond the scope of the health IT's certification and would not find a non-conformity.

For example, in the 2015 Edition final rule we provided a hypothetical scenario in which a health IT developer's certified health IT could not demonstrate required capabilities in the field due to factors that were far removed from the developer's control or responsibility (80 FR 62710). In the scenario, a customer had instructed the developer to configure the certified health IT to use clinical decision support content from a third-party vendor with whom the developer had no sublicensing agreement. The customer agreed that it would be responsible for maintaining the necessary licenses for access to the third-party vendor's content. Despite the developer's warning, the customer failed to maintain the necessary licenses and access to the content was suspended, which prevented the certified health IT from functioning as expected.

These same principles apply equally to ONC's review of certified health IT. If in the course of reviewing certified health IT, ONC determines that the failure of the health IT to perform in an acceptable manner is the result of factors that, because they are far removed from the control or responsibility of the developer, were not within its ability to reasonably influence or control, ONC would not conclude that the certified health IT is non-conforming.

(2) Review of Uncertified Capabilities

In the Proposed Rule, we proposed that ONC could review the interaction of certified capabilities of health IT with uncertified capabilities. As defined earlier in section II.A.1.a of this final rule, we use the term “certified capabilities” to refer to any capabilities or other aspects of health IT that are certified under the Program. In contrast, other aspects of health IT are referred to as “uncertified capabilities” throughout this final rule. Uncertified capabilities may be integrated with certified capabilities within a single certified health IT product (

i.e.,

a certified Complete EHR or certified Health IT Module) or may be part of other health IT products or services that are not certified under the Program.

Comments.

Several commenters supported our proposal to review certified health IT in a manner that recognizes that, in practice, certified capabilities frequently interact with uncertified capabilities, whether because a developer of certified health IT includes additional capabilities in its certified health IT product or because the developer's certified health IT product is deployed with or configured to work with other health IT products that are not certified under the Program. One commenter stated that a significant limitation of the Program to date has been the lack of an effective means to evaluate how certified capabilities of health IT are performing once they are deployed in the field and interact with other capabilities or products that are not certified under the Program.

In contrast, some commenters, including one health IT developer, suggested that it would be appropriate for ONC to review uncertified capabilities, but only in certain limited circumstances. One commenter recommended that such review be limited to situations in which a developer integrates uncertified “components” with its certified health IT in a manner that directly causes a material adverse impact on the ability of the certified health IT to function in accordance with certification requirements.

Other commenters categorically opposed this aspect of our proposal. Some of these commenters assumed, however, that ONC would review and make determinations about the performance of capabilities or products

that the commenters regarded as clearly beyond the scope of the Program. Some commenters even assumed that ONC would review health IT products that are not certified under the Program at all. According to these commenters, ONC's review of uncertified capabilities or other products would be inconsistent with the voluntary nature of the Program and would be a significant overstep of ONC's authority. One commenter, for example, stated that ONC had no authority to investigate uncertified “components” of certified health IT or to dictate how a developer builds and modifies a product in response to market mandates.

Response.

It appears that many commenters interpreted this aspect of our proposal in a manner that was more far-reaching than we had either contemplated or proposed. The confusion appears to have resulted from our summary of the major provisions of the Proposed Rule, which stated that ONC's direct review “may include certified capabilities and non-certified capabilities of the certified health IT” and “would extend to the interaction of certified and uncertified capabilities within the certified health IT and to the interaction of a certified health IT's capabilities with other products” (81 FR 11058).

In explaining the purpose of the Proposed Rule, we stated that as certified capabilities of health IT interact with other capabilities in certified health IT and with other products, ONC's direct review would ensure that concerns

within the scope of the Program

can be appropriately addressed (81 FR 11057). As this statement suggests, the purpose of direct review is to evaluate and determine whether capabilities and other aspects of health IT that are certified under the Program conform to the Program's requirements. Nevertheless, because certified capabilities are frequently integrated or deployed with uncertified capabilities, evaluating whether a certified capability under review (the “target capability”) conforms to the requirements of the Program may require understanding how the target capability is interacting with other capabilities of health IT. Those other capabilities may be certified under the Program or they may be uncertified capabilities. In the case of an uncertified capability, the capability may be part of the same “product” as the target capability or it may be part of a different product, which may or may not be certified under the Program. Whatever the case, to ensure that ONC can properly evaluate whether the target capability is functioning in an acceptable manner, we proposed that ONC may have to consider the interaction of the target capability with other capabilities that affect its performance, which could include uncertified capabilities, as discussed above. We did

not

propose, however, that uncertified capabilities would themselves become a target of ONC's review. In this sense, our statement that ONC's “review” would extend to the uncertified capabilities was somewhat inexact because ONC would be concerned with only the effects of the uncertified capabilities on the target capability, not with the performance of the uncertified capabilities in isolation. In other words, ONC's consideration of uncertified capabilities would be ancillary to its review of certified capabilities and limited to the extent necessary to determine whether those certified capabilities are functioning in a manner consistent with Program requirements.

As an illustration, consider a Health IT Module designed for ambulatory settings and that is certified to, among other criteria, § 170.314(b)(5) (Incorporate laboratory tests and values/results). Under the process established by this final rule, ONC could initiate direct review if, for example, it had reliable information that the Health IT Module were receiving and incorporating lab results incorrectly in a manner that was causing or contributing to missed diagnoses or improper management of serious medical conditions. ONC's review of the Health IT Module would be based on the Health IT Module's certified capabilities, which include the capability to incorporate lab results according to the standard specified in § 170.205(j) and, at a minimum, the version of the standard specified in § 170.207(c)(2). However, it may be that the lab results are being corrupted before they are received by the certified capability. To determine whether that is the case, it may be necessary for ONC to examine the capabilities of upstream health IT systems from which the Health IT Module receives lab results. This may include examining certified capabilities or uncertified capabilities of the upstream systems to the extent that those capabilities could be causing or contributing to incorrect data being transmitted to the receiving Health IT Module.

We reiterate that ONC does not intend to review the functioning of uncertified capabilities except to the extent that an uncertified capability interacts with and affects the performance of a certified capability that is under review. If ONC commenced review of certified health IT based on a reasonable belief that the certified health IT may not conform to the requirements of the Program, but subsequently determined that the problem or deficiency was related solely to the functioning of uncertified capabilities in isolation, ONC would cease its review of the certified health IT. We note that, as discussed subsequently in section II.A.1.a.(3) of this preamble, ONC may share any information obtained in connection with its review with other relevant agencies, to the extent permitted by law, including agencies with applicable federal oversight or enforcement authority.

With these clarifications, we believe the concerns raised in connection with this aspect of our proposal are misplaced. Contrary to those concerns, this final rule does not establish a process for ONC to make determinations about uncertified capabilities, nor to dictate how developers design uncertified capabilities within certified health IT or other technologies. ONC's consideration of uncertified capabilities will be ancillary to ONC's review of certified capabilities and limited to aspects of uncertified capabilities that interact with certified capabilities and are relevant to evaluating the performance of those certified capabilities. Further, we reiterate our expectation that direct review will occur relatively infrequently and will focus on situations that pose a risk to public health or safety or where ONC-ACBs may be unable to respond effectively.

Comments.

A number of commenters raised concerns that the application of direct review to uncertified capabilities would be contrary to ONC's policy of encouraging flexibility in the way that health IT systems are configured and used. Commenters also expressed concern that direct review of uncertified capabilities could create regulatory uncertainty and would diminish innovation. Noting that developers regard the uncertified aspects of their health IT as a key area of differentiation from their competitors, commenters expressed fear that direct review of uncertified capabilities would crowd out innovation in this important area and diminish overall incentives to innovate and improve health IT capabilities.

Response.

We are sensitive to the competition and innovation concerns raised by commenters. We believe that those concerns can be effectively addressed by clearly communicating the scope of ONC's direct review under this final rule and the limited extent to which it will impact developers of uncertified capabilities. We have

explained the potential scope of ONC's review under the processes established by this final rule, including the extent to which ONC would consider the impact of uncertified capabilities on the performance of certified capabilities. In addition, section II.A.1.a.(3) of this preamble describes the types of circumstances in which ONC may invoke the processes for direct review set forth in this final rule.

To further communicate our intent and address the concerns raised by commenters, we reiterate that the purpose of direct review is to ensure that certified health IT functions in a manner that is consistent with the requirements of the Program. In the event that ONC determines that an uncertified capability is causing a certified capability to function in a manner inconsistent with Program requirements, ONC's determination would relate to the functioning of the certified capability at issue. Even in the event that an uncertified capability is identified as the cause of, or a contributing factor toward, certified health IT functioning in a manner inconsistent with Program requirements, direct review would not dictate whether or in what manner the uncertified capability should be modified. Any corrective action to be taken by the developer in response to a determination of non-conformity by ONC would relate to bringing the certified capability or capabilities into conformity. For example, appropriate corrective action might involve the developer taking steps to ensure that the certified capability does not interact with the uncertified capability that is causing it to function in an unsafe manner.

Comments.

A number of commenters expressed concern that extending ONC's review to uncertified capabilities or to uncertified products would conflict with or duplicate oversight of health IT by other federal agencies.

Response.

We acknowledge that the investigatory and enforcement authorities of other federal agencies might apply, in certain circumstances, to the performance and functioning of certified health IT. For several reasons, however, we disagree that ONC's review will conflict with or duplicate other oversight of health IT.

First, as discussed above, while ONC's review may encompass uncertified capabilities, ONC would only be concerned with aspects of the uncertified capabilities that interact with the certified capabilities that are the subject of ONC's review, and only to the extent necessary to assess whether the certified capabilities are functioning in accordance with Program requirements. This limited and ancillary consideration of uncertified capabilities would be unlikely to create any significant conflict with or duplication of any other agency's authority. Moreover, to the extent that ONC's review does uncover issues that fall within the purview of other agencies with relevant oversight or enforcement responsibilities, ONC could coordinate with and share any information or evidence it has obtained with such agencies, to the extent permitted by federal law, and, if appropriate, could pause or end its review.

Second, as discussed below in section II.A.1.a.(3) of this preamble, we have narrowed the scope of direct review under this final rule based in part on the ability of other agencies to provide appropriate oversight of certain types of non-conformities that would otherwise warrant ONC's review. For example, at this time, we have not finalized in this rule processes for ONC direct review of a suspected non-conformity solely on the basis that certified health IT may be compromising the security or protection of patients' health information (

see

section 3001(b)(1) of the PHSA) or increasing health care costs as a result of, for example, inefficiency or incomplete information (

see

section 3001(b)(3) of the PHSA). Our decision not to establish regulatory processes for such oversight at this time is based in part on the recognition that other agencies have the ability to investigate and respond to these types of issues and our desire to make the most efficient use of limited federal resources.

Third, far from conflicting with or duplicating the efforts of other agencies, we expect direct review to promote greater alignment in the oversight of health IT. Direct review allows ONC to coordinate with and provide expertise to other agencies, and to share any information or evidence ONC has obtained, as permitted by federal law. For example, ONC could quickly marshal and deploy resources and specialized expertise while working with federal counterparts to ensure a coordinated review and response to potential non-conformities. This approach is consistent with our inter-agency efforts to avoid regulatory duplication and promote appropriate, risk-based oversight of health IT, including efforts described in the Draft Food and Drug Administration Safety and Innovation Act (FDASIA) Health IT Report,

published jointly with the Food and Drug Administration (FDA) and the Federal Communications Commission (FCC). Indeed, the need for effective coordination could be especially important in responding to serious risks to public health or safety that arise from the complex interaction of health IT products that may include certified capabilities regulated by ONC as well as uncertified capabilities that may be subject to FDA, FCC, or another agency's oversight.

Draft FDASIA Health IT Report: Proposed Strategy and Recommendations for a Risk-Based Framework (April 2014), available at

https://www.healthit.gov/sites/default/files/fdasia_healthitreport_final.pdf.

Finally, we note that ONC may elect to not initiate direct review (or, if it has initiated direct review, to cease such review) at any time and for any reason, including if ONC believes that another agency is better situated to investigate or address a suspected non-conformity, or if ONC believes that direct review could duplicate or interfere with the oversight or enforcement activities of other agencies. ONC may also coordinate with and share any information or evidence it has obtained, through its direct review or otherwise, with other agencies, to the extent permitted by federal law. We also anticipate that ONC may coordinate with ONC-ACBs, ONC-ATLs, the ONC-AA, and other entities in appropriate circumstances and consistent with applicable federal law.

(3) Scope of Review

We proposed that ONC may exercise direct review of certified health IT when there is reason to believe that the certified health IT may not conform to the requirements of the Program. We explained that ONC's review could be in response to concerns that certified health IT may be leading to medical errors or other outcomes that are inconsistent with the National Coordinator's responsibilities under section 3001 of the PHSA. We also stated there could also be other exigencies, distinct from public health or safety concerns, that for similar reasons would warrant ONC's direct review and action. In addition, we proposed that ONC may directly review certified health IT in situations that present unique challenges or issues that ONC-ACBs may be unable to effectively address without ONC's assistance or intervention. We listed a variety of factors in this regard that could help inform ONC's decision whether to initiate direct review in individual cases, specifically:

• The potential nature, severity, and extent of the suspected non-conformity or non-conformities, including the likelihood of systemic or widespread issues and impact.

• The potential risk to public health or safety or other exigent circumstances.

• The need for an immediate and coordinated governmental response.

• The potential for inconsistent application of certification requirements in the absence of direct review.

(

see

Comments.

A majority of commenters agreed that ONC should directly review certified health IT that could be leading to medical errors or other risks to public health or safety. One commenter representing health care professionals noted a strong need for ONC to adjust the Program to focus on the safety, usability, and interoperability of certified health IT, citing widespread concerns among the medical community about these issues. The commenter stated that ONC could play a valuable role in ensuring that the appropriate parties are identifying, analyzing, and correcting health IT safety concerns by quickly resolving non-conformity issues.

Several commenters who otherwise opposed direct review, including health IT developers, stated that it may be reasonable for ONC to review non-conformities as a “true last resort” when risks to patient safety are sufficiently compelling or when there is a gap or overlap in the ability of ONC-ACBs to effectively address the risk.

A small number of commenters categorically opposed this aspect of our proposal and stated that whether certified health IT is leading to medical errors or other risks to public health or safety is either beyond the scope of current certification criteria, other Program requirements, or section 3001(c)(5) of the PHSA. A few commenters, including one ONC-ACB, stated that health IT-related safety risks should not be addressed through the Program because there might be other channels, such as the proposed Health IT Safety Collaborative,

through which these issues could be more effectively dealt with, including by identifying health IT safety-related issues, defining appropriate best practices and criteria, and making objective assessments. Commenters also urged ONC to continue to support existing private-public initiatives that are developing a framework for the identification of health IT safety incidents to expand knowledge for all stakeholders.

See

Department of Health and Human Services,

Justification of Estimates for Appropriations Committee (Office of the National Coordinator for Health Information Technology),

app. IV,

https://www.healthit.gov/sites/default/files/final_onc_cj_fy_2017_clean.pdf

(2016) (proposing that Congress provide ONC with authority to establish a Health IT Safety Collaborative and provide adequate confidentiality protections).

See also

ONC,

Health IT Safety Center Roadmap, http://www.healthitsafety.org/uploads/4/3/6/4/43647387/roadmap.pdf

(2015) (containing task force recommendations for the development of a national Health IT Safety Center); Food and Drug Administration,

Draft FDASIA Health IT Report, https://www.healthit.gov/sites/default/files/fdasia_healthitreport_final.pdf

(2014) (recommending establishment of a Health IT Safety Center as a key component of a risk-based approach to health IT safety oversight and efforts to create a sustainable, integrated health IT learning system that avoids regulatory duplication and leverages and complements existing public and private sector activities to improve the safety and safe use of health IT).

Response.

We thank commenters for their feedback and suggestions on this aspect of our proposal. Based on the comments, and consistent with the focus of the Proposed Rule, we continue to believe that direct review by ONC is necessary to address potential non-conformities and non-conformities in certified health IT that may be leading to medical errors or contributing to other risks to public health or safety. As we have explained, although ONC-ACBs play an important role in the Program, addressing the full range of these suspected non-conformities is beyond the scope of their responsibilities under the Program. In addition, ONC-ACBs may as a practical matter lack the expertise and resources to effectively respond to certain types of non-conformities, such as widespread or systemic problems with certified capabilities. Other agencies may similarly be unable to effectively respond to these issues, especially when the underlying causes are unclear or involve complex interactions among multiple health IT capabilities or products. As the capabilities of certified health IT evolve and become ubiquitous in the delivery of care, the National Coordinator has a responsibility to continually update and enhance oversight of the Program so that certified health IT continues to improve, and does not compromise, patient safety.

Addressing these types of issues will promote greater confidence in the safety of certified health IT and protect the integrity and effectiveness of the Program. Accordingly, § 170.580(a)(2) addresses the process for ONC to directly review certified health IT when the health IT may be causing or contributing to conditions that pose a serious risk to public health or safety. We note that the policy we have finalized is consistent with the general sentiment expressed by commenters, as we understand it, that ONC should exercise direct review judiciously, focusing on risks to public health or safety that are serious and on non-conformities that cannot be effectively addressed by ONC-ACBs. As we stated in the Proposed Rule, we expect that ONC's exercise of direct review will be relatively infrequent. We discuss these considerations in detail in our responses to the comments summarized immediately below.

We agree with commenters that advancing health IT safety is a shared responsibility and will require a concerted commitment by all relevant stakeholders, including through current public-private efforts and proposed initiatives such as the Health IT Safety Collaborative. We continue to strongly support these efforts and recognize the vital role they play in promoting the safety of health IT and the use of health IT to improve the safety and quality of care. We regard ONC's direct review as complementary to these efforts.

We disagree with the view expressed by some commenters that concerns related to the safety of certified health IT are beyond the scope of current certification criteria, other Program requirements, or section 3001(c)(5) of the PHSA. We refer commenters to our discussion of these issues in section II.A.1.a.(1) of this preamble.

Comments.

We received relatively broad support for our proposal to enhance oversight of non-conformities that pose a risk to public health or safety, including through the direct review of such issues by ONC. A significant number of commenters urged us to prioritize public health and safety over other concerns by narrowing the scope of ONC's review to focus exclusively or primarily on non-conformities that pose serious risks to public health or safety. Commenters stated that this narrower focus would

allow ONC to concentrate its resources and provide more effective oversight of safety issues.

Many commenters also recognized the need for and supported ONC's review of non-conformities that, for other reasons, would be difficult for ONC-ACBs to effectively address.

Commenters were less supportive of applying ONC oversight of the Program to the other areas we had proposed, such as widespread non-conformities that could compromise the security or protection of patients' health information in violation of applicable law, or that could lead to inappropriate claims for reimbursement under federal health care programs. A substantial majority of commenters urged us to significantly narrow and more clearly define the types of non-conformities that ONC could potentially review. Commenters were concerned that, as proposed, ONC could conceivably review non-conformities that implicate any of a wide and diverse range of potential subjects, from security breaches, to anti-competitive practices, to conditions giving rise to health disparities. This could lead to regulatory uncertainty or arbitrary enforcement, and could discourage innovation in health IT.

For many of the same reasons, commenters urged us to clarify the specific types of circumstances or situations in which ONC would be likely to initiate direct review of certified health IT. While we had proposed several factors that ONC would consider in determining whether to initiate direct review, a number of commenters stated that these factors were too numerous or open-ended to provide useful guidance to stakeholders. Several commenters urged us to provide guidelines or examples explaining when ONC would be likely to initiate direct review. One commenter explained that by clarifying our methodology we could make the direct review process fairer and more equitable and establish confidence both in the process and its outcomes.

Response.

We agree with commenters that the types of non-conformities ONC may review and, equally important, the types of circumstances in which ONC will take action to enforce Program requirements should be made as clear as possible and should be applied in a consistent and judicious manner. Such clarity and consistency help enable developers to design and make their certified health IT available in a manner that consistently meets Program requirements and the expectations of purchasers, licensees, and users of certified health IT. We also appreciate that uncertain or unnecessary regulation can have unintended consequences, including reducing incentives to invest in and to innovate the technologies that will make it possible to use health IT and health information to improve health and the delivery of care.

In light of these and other considerations described below, we have reconsidered and revised our proposal in several key respects. Importantly, while the PHSA provides the National Coordinator the authority to directly review certified health IT in the broad range of circumstances we proposed, at this time we have finalized a regulatory framework for the exercise of such review in a more limited set of circumstances. This scope of review is consistent with our expectation stated in the Proposed Rule that direct review will be relatively infrequent and will focus primarily on issues that pose a risk to public health or safety (81 FR 11058) or that ONC-ACBs may be unable to effectively address without ONC's assistance or intervention (81 FR 11061). While we stated that there could be other exigencies in addition to risks to public health and safety that could also warrant ONC's review, we agree with commenters that the need for additional ONC oversight in these areas is less pronounced at this time. In particular, we note the active oversight in these areas by other agencies, as discussed below. In light of this existing oversight and the limited resources at ONC's disposal, we agree with commenters that it is advisable to focus ONC's resources in areas in which, at this time, additional and direct oversight by ONC is most vital to ensuring the integrity and effectiveness of the Program. We believe that focusing ONC's review in these areas will help foster alignment and coordination with other agencies and promote confidence in the performance of certified health IT and the nation's health IT infrastructure, which will in turn support innovations and investments in health IT.

For all of these reasons, we have finalized processes in this rule for ONC to exercise direct review of certified health IT in two distinct sets of circumstances.

First, ONC may elect to directly review certified health IT when there is reason to believe that the certified health IT may be causing or contributing to serious risks to public health or safety. In these circumstances, ONC's direct review of certified health IT may be necessary to protect the public from certified health IT that is unsafe and to ensure the basic integrity and effectiveness of the Program. As explained in section II.A.1.a.(1) of this preamble, it is a requirement of the Program that certified health IT be made available in a manner that does not cause or contribute to serious risks to public health or safety. However, responding to the full range of these suspected non-conformities is beyond the scope of an ONC-ACB's expertise and responsibilities under the Program. In contrast, ONC is well-placed to respond to these issues, through the direct review processes established by this final rule, bringing to bear needed expertise and resources and coordinating activities with federal counterparts and other relevant entities to ensure a coordinated review and response to public health and safety concerns (81 FR 11061).

The circumstances described above do not encompass all possible non-conformities of certified health IT. For example, certified health IT may not conform to the requirements of the Program if it is causing or contributing to other outcomes—distinct from risks to public health or safety—that are inconsistent with the National Coordinator's responsibilities, such as compromising the security or protection of patients' health information in violation of applicable law (

see

section 3001(b)(1) of the PHSA) or increasing health care costs resulting from, for example, inefficiency or incomplete documentation (

see

section 3001(b)(3) of the PHSA). At this time, however, we believe that other agencies are in the best position to provide effective federal oversight and enforcement in these areas. For example, within HHS, the Office for Civil Rights' (OCR) enforces the Privacy, Security, and Breach Notification Rules promulgated under the Health Insurance Portability and

Accountability Act of 1996 (HIPAA) and amended by the HITECH Act, and the Office of Inspector General (OIG) enforces a range of federal laws related to fraud, waste, and abuse. Therefore, we have not at this time finalized regulatory processes by which ONC would directly review certified health IT solely on the basis of circumstances distinct from public health or safety concerns or in cases where practical challenges prevent an ONC-ACB from effectively investigating the suspected non-conformity or providing an appropriate response, as discussed above (

compare

81 FR 11061). We will continue to assess the need to exercise direct review in these additional circumstances, as necessary

Such agencies could include, for example, the Centers for Medicare and Medicaid Services, the Food and Drug Administration, the HHS Office for Civil Rights, the HHS Office of Inspector General, the Department of Veterans Affairs, the Federal Communications Commission, or state Medicaid agencies. We note that to the extent ONC exercises its discretion to engage in any efforts to identify or address non-conformities, such efforts and any resulting remediation (or the absence of such efforts or remediation) are not intended to impact the materiality of any non-conformity in a matter addressed by another agency; and nothing in this final rule is intended to supplant, delay, or in any way limit oversight or enforcement by other agencies, including any investigation, decision, legal action, or proceeding.

Example E in section II.A.1.a.(3) of this preamble illustrates the complementary roles of ONC's direct review and the activities of other agencies.

Finally, our decision to focus ONC's review, at this time, on the types of non-conformities described above allows us to provide a more structured decision-making regulatory framework to support the exercise of ONC's discretion to initiate review of certified health IT in the circumstances we have described. In contrast to the framework set forth in the Proposed Rule, we have simplified and defined with greater specificity the factors ONC will consider in determining whether to initiate direct review of a suspected non-conformity. The updated regulatory framework, which we have finalized at § 170.580(a)(2), provides a more sequential and targeted set of factors that ONC will consider when determining whether to initiate direct review. We have also eliminated duplicative or redundant factors included in the Proposed Rule, as discussed in more detail in our responses to comments on those factors below. These revisions will provide clear and predictable guidelines that will promote compliance with Program requirements while preserving incentives to develop and adopt new and innovative technologies.

Comments.

Several commenters suggested that ONC should focus its oversight on risks to public health or safety that are “clear,” “severe,” “immediate,” “extreme,” or otherwise compelling. A few commenters stated that ONC should not exercise direct review unless the risk to patient safety or public health poses imminent risks to public health or safety. Commenters stated that focusing on these types of risks would ensure that ONC's limited resources are used to mitigate the problems or issues with certified health IT that pose the most serious risks of harm to patients and the public. Separately, some commenters stated that exercising direct review of all potential risks could be counter-productive in that it may discourage efforts to implement and use health IT to improve patient safety and care.

Relatedly, commenters requested additional specificity regarding the types of risks to public health or safety that could trigger ONC's review or give rise to a non-conformity. One commenter requested that ONC provide examples to illustrate how certified health IT might contribute to risks to patient safety and public health.

Response.

We agree that not every risk to public health or safety necessitates ONC's direct review. We are also cognizant of the need to prioritize ONC's limited resources by focusing on the kinds of problems and other issues that, if not addressed through ONC's direct review, are most likely to lead to harm to patients or the public and undermine confidence in health IT and the integrity of the Program.

As described in section II.A.1.a.(1) of this preamble, to conform to the requirements of the Program certified health IT must be designed and made available to users in a way that allows certified capabilities to be used in an accurate and reliable manner. This includes making capabilities available in a manner that does not cause or contribute to medical errors or other conditions that give rise to serious risks to public health or safety. Direct review would be appropriate if ONC had reason to believe that certified health IT were causing or contributing to conditions that present a serious risk to public health or safety, including conditions that could result in serious injury or death, whether to a patient or to any other person.

Our focus on risks to public health or safety that are “serious” is consistent with the Proposed Rule, in which we suggested that ONC's direct review would be appropriate in response to certified health IT causing or contributing to medical errors or other exigent circumstances that call for an immediate or coordinated governmental response (81 FR 11058;

compare

proposed § 170.580(a)(1)(ii) through (iii) at 81 FR 11082). This focus also aligns with the general sentiment expressed by commenters that ONC's review of matters involving public health or safety should focus on risks that are “clear,” “severe,” “immediate,” “extreme,” or otherwise compelling. We note that these terms are not self-defining and that assessing whether certified health IT poses serious risks to public health or safety will necessarily involve a careful consideration of the relevant facts and circumstances in each case. To this end, ONC would consider the nature, extent, and severity of the risk and the conditions giving rise to it, in light of the information available to ONC at the time. In addition to any other factors that may be relevant, ONC would consider the apparent severity of the harm that might result, or has resulted, from the suspected unsafe conditions, including the likelihood of death or serious injury; the number of persons who may be harmed in the event that the harm were to materialize; and the likelihood that harm will in fact materialize if appropriate action is not taken. ONC would also consider the extent to which the risk of harm may be imminent such that an immediate or coordinated governmental response is necessary to significantly reduce the likelihood of actual harm occurring or recurring (§ 170.580(a)(2)(i)(B)). In evaluating whether the risk of harm may be imminent, ONC would also take into

account any actions being taken to mitigate the risk, to the extent that ONC is aware of those actions. We have declined to adopt commenters' suggestions that ONC should focus exclusively on the “imminence” of a potential risk to public health or safety when determining whether to exercise direct review. While the nature of public health or safety risks dictates that in most cases they will be imminent, we can envision scenarios in which a risk might not be strictly “imminent” at the time ONC determines that it will initiate its review but might nonetheless lead to serious harm if not addressed. For example, ONC might decide to exercise direct review if it became aware of information about a serious safety risk that a developer, in concert with its healthcare provider customers, is managing by way of a complex series of manual “work-arounds” until the scheduled release of the developer's next software update. While the developer may assert that the risk to patients is not imminent because of the existence of the manual work-arounds, it may be necessary—both to protect patients and the integrity and effectiveness of the Program—for ONC to review the safety risk at issue immediately and not have to wait until such time as the manual work-arounds fail. ONC may, as part of direct review in this instance, determine that the risk to patient safety is such that, for the health IT to remain certified, the developer must rectify the deficiency by way of a patch and not wait until the developer's next scheduled software release.

Separate from information about unsafe conditions in particular, ONC could conclude that certified health IT poses a serious risk to public health or safety were it aware of information calling into question the validity of the health IT's certification. Such information might include, for example, credible allegations that a health IT developer obtained or maintained any part of the certification of its health IT by means of false or misleading statements or representations to an ONC-ACB; misrepresented or made false or misleading statements to customers or users about the certification or certified capabilities of the health IT; concealed problems, deficiencies, or potential non-conformities; or took other actions that would be likely either to compromise or to circumvent processes under the Program for testing, certifying, and conducting ongoing surveillance and review of certified health IT. These circumstances present a serious risk to public health or safety because obtaining and maintaining a valid certification is fundamental to ensuring that health IT meets Program requirements, including requirements essential to providing basic assurance that health IT is able to perform required capabilities in an accurate and reliable manner. Indeed, customers, implementers, and users rely on the certifications issued on behalf of ONC to provide this basic assurance so that they can select appropriate technologies and capabilities, identify potential implementation or performance issues, and implement certified health IT in a predictable, reliable, and successful manner (80 FR 62709). Where the validity of a certification is called into question, these and other persons are unknowingly deprived of this basic assurance upon which they rely.

To further illustrate these principles and how they would be applied in practice, we offer the following contrasting examples.

Example A:

ONC receives multiple, detailed reports that a cloud-based EHR system (certified to the 2015 Edition) has become so slow that it may take up to five minutes to load a patient's record or to display information within a patient's record, such as the patient's medication and medication allergy lists. When providing emergency treatment, clinicians cannot wait five minutes for this information and must order medications with incomplete information about patients' current medications and medication allergies. Even when treatment is not urgent, the system's delays in responding lead many clinicians to assume that the EHR is not working and to order medications based on their best recollection of patients' current medications and allergies.

Clinicians at several hospitals in multiple states are experiencing these problems. There is no indication that these hospitals are maintaining substandard hardware or network infrastructure below the recommendations from the health IT developer, nor that they have customized their health IT in a way that would adversely affect system performance. The health IT did not behave this way when it was installed, but as the clinical data and number of records has grown the speed of the EHR's responsiveness has decreased.

In this example, ONC may initiate direct review of the certified health IT. The facts suggest that several capabilities of the certified health IT are implicated, including § 170.315(a)(6) (Problem list) and § 170.315(a)(7) (Medication list). The capabilities as implemented appear to be performing or interacting in a way that is causing or contributing to a serious risk of harm to public health or safety. The risk of harm is serious for several reasons. First, clinicians are abandoning use of the capabilities and resorting to memory to order medications for patients, which could result in severe harm to patients, including serious injury or death. Moreover, the risk is imminent because it is likely that harm will occur soon unless immediate action is taken to address the unsafe conditions. Further, the extent of the risk is large because the unsafe conditions have been reported at several hospitals in multiple states and may therefore put at risk a large number of patients.

Assuming ONC were to initiate direct review, it would examine the certified capabilities to determine why they are not performing in an accurate and reliable manner and whether the cause of the problem was within the ability of the health IT developer to reasonably influence or control. The facts suggest that the problem is common across multiple customers and is not the result of any actions of the developer's customers or users. Because the problem developed over time, the developer would have been aware of the problem and could have prevented it by employing best software practices to prevent a system related slow-down under load. If this were established, ONC would find a non-conformity.

Example B:

ONC receives credible information from multiple sources that a large hospital's EHR system, which is certified to the 2015 Edition, is dropping medication orders. While the cause of the dropped orders is not yet clear, data in patients' records is not being recorded in a consistent and reliable manner, which is leading to patients not receiving medications.

Based on the information it has received, ONC believes that the EHR system's computerized provider order entry (CPOE) capability for medications (§ 170.315(a)(1)) may be interacting with other capabilities within the EHR or within other health IT in a way that is causing or contributing to orders not arriving when they are needed. This poses a serious risk to public health or safety because there is an imminent risk that patients will not receive needed or even life-saving medications that have been ordered for them, which could result in severe harm.

Accordingly, ONC initiates review of the certified health IT. However, during the course of its review, ONC determines that the hospital had chosen not to install and maintain the minimum specified hardware and

network requirements published by the developer of the certified health IT. As a direct result of the substandard hardware and network connectivity, the certified health IT is suffering system timeouts, losing network packets, and not operating correctly. Based on these findings, ONC finds that while the certified capability is not performing in an acceptable manner, the reason for the substandard performance is that the hospital has chosen not to follow the developer's minimum hardware and network recommendations. The hospital's decision to intentionally disregard the developer's clear instructions regarding the safe use of its technology is a factor that is beyond the ability of the developer to reasonably influence or control. Therefore, ONC would not find a non-conformity and would cease its review. ONC may, however, refer the matter (and information or evidence obtained as a result of its review) to other agencies with applicable oversight or enforcement responsibilities, as discussed above in this section of the preamble.

Example C:

ONC receives multiple reports from a large hospital concerning a potential problem with its EHR. Over the past week, several patients with congestive heart failure (CHF) had to be readmitted because of CHF exacerbations. Clinical and IT staff at the hospital have investigated the problem and believe that it is due to an error in the hospital's EHR, which is certified to the 2015 Edition. The hospital reports that its CHF patients are all given electronic scales that record their weight and automatically transmit the daily weight back to the hospital's EHR. The weight can be tracked and the patients can be alerted if they are gaining too much weight (from excess fluid, one of the signs of a CHF exacerbation) and need to adjust their CHF medications accordingly. The readmissions happened due to inaccurate weight data being presented to clinicians, which caused the clinicians to not adjust diuretic medication to manage patients' fluid status appropriately.

Based on these facts, ONC may initiate direct review of the certified health IT. ONC could form a reasonable belief that the certified health IT may be causing or contributing to serious risks to public health or safety, in violation of Program requirements. A number of certified capabilities appear to be implicated, including § 170.315(e)(3) (Patient Health Information Capture) and certified capabilities that interact with vital signs data (which is part of the Common Clinical Data Set (§ 170.102)). Although the cause of the problem is not yet clear, it is reasonable to believe that it may be a result of one or more of these certified capabilities or of their interaction with other uncertified capabilities or products. Meanwhile, the occurrence of multiple readmissions in the past week suggests that, if the certified health IT is causing or contributing to these risks to public health or safety, the risks are sufficiently serious as to constitute a non-conformity and to warrant ONC's review.

Example D:

ONC becomes aware of a patient safety hazard at a large area hospital. In one reported case, a patient with chest pain entered the emergency department (ED) of the hospital. In the ED, nurses enter protocol orders for patients with chest pain on behalf of the attending physician. On this occasion, an attending physician accessed the patient's record in the EHR and, observing that no blood tests had been ordered, proceeded to order the tests from the standard order set. Contemporaneously, a nurse was in the process of entering the same tests from the same order set. The nurse completed her order a few seconds before the physician completed hers. Neither the nurse nor physician recall any duplicate order alerts, although hospital IT staff state that clinical decision support (CDS) was active in the EHR system and had been configured to intercept and display alerts when duplicate orders are entered. The duplicate orders were noticed later when the physician was reviewing the patient's record in the EHR. At that time, the physician cancelled the nurse's order, which thereafter was no longer displayed in the EHR. The EHR continued to display the physician's order with a status of “pending collection.” The lab system assumed that the identical lab requests for the same patient were duplicates and cancelled the physician's request because the nurse's request had arrived first. The lab system, however, did not create an outgoing interface message to the ordering EHR indicating that the physician's “duplicate” request had been cancelled. As a result, the physician's order continued to be displayed in the EHR with a status of “pending collection.”

Back in the ED, alert staff noticed that the labs had not been drawn within the expected time frame, and reordered the tests. Fortunately no harm resulted to the patient. However, the hospital's clinical staff and leadership believe the EHR presents a serious patient safety hazard. The clinicians report the incident to ONC and note that in a large and busy ED it is not uncommon for clinicians to enter contemporaneous orders; and that they expect the EHR to alert them when this occurs and to intercept duplicate orders before they are transmitted. The hospital's IT staff and the EHR developer, with whom the IT staff have been working to analyze this incident, believe that the EHR was configured to provide these CDS interventions. Neither the hospital's IT staff nor the EHR developer has been able to ascertain why these safeguards appear to have failed in this case. Based on these facts, ONC could form a reasonable belief that the certified health IT may be causing or contributing to a serious risk to public health or safety. As noted by the hospital's clinical staff and leadership, duplicate orders are not uncommon, especially in a large and busy ED. If not detected, the duplicate orders may lead to a wide range of serious hazards, such as administration of unnecessary tests or excessive medication dosages. And as illustrated by this example, the failure to detect and intercept duplicate orders may also have downstream effects that could prevent the fulfillment of orders and result in patients not receiving timely test results and treatment. The severity and extent of the harm that could occur is significant and is likely to materialize unless the cause of the problem is isolated and resolved. That the hospital's IT staff and the EHR developer are cooperating and yet have been unable to ascertain the cause of the problem is also relevant to ONC's consideration because it suggests that the problem could reoccur and that the full extent of the problem, including for other hospitals or facilities that use the developer's EHR, is not known.

While the risk to public health or safety is clear, to initiate direct review, ONC must have a reasonable belief that the certified health IT may be causing or contributing to that risk. Here, there are at least two certified capabilities that are potentially implicated: CPOE (§ 170.315(a)(3)) and CDS (§ 170.315(a)(9)). This nexus to certified capabilities is sufficient for ONC to initiate direct review.

Concurrently, ONC might direct the responsible ONC-ACB to perform surveillance of issues that are within the scope of its responsibilities and expertise. Here, an ONC-ACB could conduct in-the-field surveillance of the CPOE and CDS capabilities to determine whether there is a non-conformity to the requirements of § 170.315(a)(3) or (a)(9). For example, the ONC-ACB would be well-positioned to determine through in-the-field surveillance whether the certified CDS capability, when properly configured to intercept and alert users to

duplicate orders, consistently triggers those interventions in a reliable manner in a production environment.

On the other hand, an ONC-ACB may be unable to analyze other possible non-conformities. For example, it may be that the CDS reliably displays alerts as intended but that the alerts are designed in a way that makes them susceptible to being inadvertently overridden. These usability considerations are within the scope of the Program's requirements but may be best suited for ONC to review. ONC could also examine the interaction of the certified capabilities with the receiving lab system (which may or may not be certified under the Program), which in this example is critical to isolating and understanding the nature of the problem and assessing whether the certified health IT conforms to Program requirements. In reaching that determination, ONC would consider whether the EHR developer could have reasonably anticipated that the lab system would cancel the orders without sending a notification of the cancellation and whether it could have taken reasonable steps to mitigate this risk (such as warning users to manually confirm the orders or providing a bi-directional interface that ensures that users are able to view when orders are in fact received and filled). This may require analyzing the EHR developer's interfaces and contractual agreements with the lab system as well as the EHR developer's field testing and quality assurance procedures. Again, these factors may be beyond the expertise of the ONC-ACB and better suited for ONC's review.

As the foregoing examples illustrate, the particular facts and circumstances that may trigger ONC's review of certified health IT will be unique to each case, as will be the analysis of the issues relevant to determining whether the certified health IT conforms to Program requirements. Nevertheless, we believe the examples above will help stakeholders understand the types of risks to public health or safety that may prompt ONC's review and that may lead to a finding of non-conformity. We anticipate issuing additional guidance on these and other aspects of this final rule as appropriate.

Comments.

A small number of commenters distinguished between risks to patient safety and those related to broader public safety or public health. Some commenters stated that direct review would not be appropriate in circumstances that pose a risk of harm to public health but not specifically to patient safety. In contrast, one commenter posited that public health considerations may justify or weigh in favor of direct review in certain situations, such as where problems with certified health IT may adversely impact socially or medically vulnerable populations.

Response.

We intend the term public health or safety to encompass risks to both patients and other persons. Given the central role of health IT in delivering care, it is likely that ONC's oversight will focus on risks of harm to patients. However, we would be no less concerned if certified health IT were causing or contributing to risks of harm to persons other than patients, and we believe that the National Coordinator's responsibility to provide for effective oversight of certified health IT so that it does not create unreasonable risks of harm to patient safety applies with equal force to risks involving public health.

We note that under the approach we have finalized, ONC would consider the potential nature of a public health or safety risk when reaching a determination whether to initiate direct review. Thus ONC's determination would take into account the impact that the potential risk is having, or might have, on a patient(s). This determination would necessarily involve an analysis of the risk as it relates to the affected patient population.

Comments.

A number of commenters voiced concerns about the factors that ONC would consider when determining whether to initiate direct review, characterizing those factors as overly broad and creating a risk of arbitrary application. Commenters noted in particular that the phrase “other exigent circumstances” was ambiguous. Some commenters suggested that ONC's potential reliance on such an open ended factor would enable ONC to exercise direct review in an unaccountable manner. Commenters requested clarification or reconsideration of the inclusion of “other exigent circumstances” as a factor to be considered by ONC when initiating direct review.

Response.

We identified a number of factors in the Proposed Rule that ONC might consider when determining whether to exercise its discretion to initiate direct review. These factors were included to provide health IT developers with some comfort that while ONC's authority to initiate direct review is broad, ONC's use of direct review would be guided by principles that focus ONC's limited resources on the oversight of non-conformities that pose substantial risks to the integrity and effectiveness of the Program. Indeed, the inclusion in the proposal of the phrase “other exigent circumstances” was intended to narrow ONC's discretion rather than, as suggested by commenters, provide ONC with a degree of flexibility that would make ONC's exercise of direct review unaccountable. Notwithstanding this, we acknowledge commenters' concerns regarding the open-ended nature of the phrase “other exigent circumstances.” We maintain that there could be other exigencies, distinct from public health or safety concerns, that pose risks to the integrity and effectiveness of the Program and warrant ONC's direct review and action. However, at this time, our decision to focus on public health and safety risks (in addition to non-conformities over which, for practical or other reasons, ONC-ACBs may be unable to provide effective oversight) at this stage of our administration of the Program has enabled us to omit any reference in the final rule to ONC considering “other exigent circumstances” when determining whether to exercise direct review.

We clarify that while under the processes established by this final rule ONC would not, at this time, initiate direct review solely on the basis of exigencies other than serious risks to public health or safety, and while ONC's review would focus on aspects of health IT that are certified under the Program, ONC would not be precluded from sharing, to the extent permitted by federal law, any information or evidence (including about other exigent circumstances or problems with uncertified capabilities of health IT) with other relevant agencies, including law enforcement or other agencies who may be able to address such matters. Conversely, ONC may receive information about potential non-conformities or non-conformities from other agencies in the course of their oversight, enforcement, or other activities. As an illustration, consider the following example.

Example E:

A Health IT Module certified to the 2015 Edition (“the EHR”) is the subject of a “ransomware” attack. The attacker gained unauthorized access to the EHR at multiple health care facilities and deployed malicious software that rendered patients' electronic health information completely inaccessible to clinicians and other users of the EHR. Several of these facilities have reverted to backup systems, including in some cases paper records and manual workflows that significantly increase the risks of medical errors and harm to patients. Several federal agencies (“the Agencies”) are currently investigating the attack. The Agencies request the

assistance and expertise of ONC's Chief Privacy Officer to better understand the role of the EHR in contributing to the incident. The investigation quickly reveals that the attacker exploited a vulnerability in the operating system software (OS) used in conjunction with the EHR. The OS was out of date and no longer receiving security updates. The Agencies, concerned about the prospect of additional security breaches, share this information confidentially with ONC.

For the reasons stated earlier in section II.A.1.a.(3) of this preamble, ONC would not initiate direct review of the certified health IT solely on the basis of security incidents or other exigencies that are distinct from risks to public health or safety. At this time, we believe that other agencies are currently best positioned to provide effective oversight and enforcement of health IT with respect to these potential exigencies. Nevertheless, as the facts of this example make clear, these exigencies may also give rise to serious risks to public health or safety. Where certified health IT may be causing or contributing to risks of this kind, ONC may initiate direct review to protect the public and the integrity and effectiveness of the Program.

Here, ONC initiates direct review based on the information received from the Agencies. To ensure that ONC's review assists and does not in any way hinder the ongoing investigation, ONC carefully coordinates with the Agencies and shares information and evidence it obtains during its review. ONC's review confirms that the developer of the EHR requires users to install and use a version of the OS that is no longer supported by the OS manufacturer and is not receiving security updates. All certified capabilities of the EHR are affected by this requirement, which exposes users to vulnerabilities and attacks that could compromise patient data and result in serious harm to patients. At the same time, ONC finds that the developer could have reasonably anticipated, and avoided, these risks because the OS manufacturer had published many notices that the version of the OS was being retired and would no longer receive security updates. Based on these findings, ONC issues a notice of non-conformity to the developer.

By contrast, if ONC had found that the health IT developer offers an upgrade path to the latest versions of the operating system software, and encourages its users to upgrade, ONC would not find a non-conformity if users decided to not install the upgrade.

Comments.

Commenters suggested that we clarify our proposed methodology for assessing the “nature, severity, and extent” of a suspected non-conformity and the significance of this factor to ONC's determination whether to initiate direct review.

Response.

In response to the concerns raised by commenters, we have made a number of adjustments in the final rule that will create greater predictability for the process that ONC will use to determine when to initiate direct review.

The proposals in the Proposed Rule outlined a direct review process in which ONC would exercise wide latitude to consider and weigh factors when determining whether to initiate direct review. As proposed, ONC might evaluate a number of factors that could be relevant to the particular circumstances at issue at the same time. However, at this time, we have chosen to narrow the scope of potential non-conformities and non-conformities ONC will review as described above. Given this narrower scope, we are able to delineate the specific factors that ONC will consider and apply when determining whether to initiate direct review of certified health IT.

Under the final rule, the nature, severity, and extent of a non-conformity would be relevant if ONC were to initiate review of a suspected non-conformity on the basis of public health or safety concerns. In that instance, ONC would have a reasonable belief that certified health IT may be causing or contributing to conditions that pose a serious risk to public health or safety. The potential nature, severity, and extent of the suspected conditions giving rise to that risk would be directly relevant to this determination, as would the need for an immediate or coordinated governmental response. These considerations are described in greater detail earlier in section II.A.1.a.(3) of this preamble. We have expressly included these considerations as factors that ONC will consider when determining whether certified health IT may be causing or contributing to risks that are sufficiently serious as to suspect that the certified health IT does not conform to the requirements of the Program and ONC's direct review.

Separately, and as also discussed in section II.A.1.a.(3) of this preamble, ONC may directly review certified health IT when a suspected non-conformity, while based on requirements of the Program that are generally within the scope of an ONC-ACB's responsibilities to administer and enforce, presents issues that may prevent the ONC-ACB from effectively investigating or responding. The nature, severity, and extent of a suspected non-conformity may be relevant to this determination. For example, the suspected non-conformity may be so systemic, complex, or widespread that an ONC-ACB would lack the resources or expertise to effectively investigate or respond to it. On this basis, ONC may directly review the suspected non-conformity.

Comments.

One commenter suggested that ONC include additional factors for assessing when to exercise its direct review. This commenter recommended that ONC develop an additional factor that ensures that ONC's decision to initiate direct review takes into account the impact of non-conformities on socially and medically vulnerable populations.

Response.

Under the final rule, ONC will consider the potential nature, severity, and extent of a public health or safety risk when reaching a determination as to whether to initiate direct review. This determination would take into account the potential impact the risk is having, or might have, on a patient(s) or the public. We anticipate that an analysis of the affected population could be relevant to that determination. For example, an issue might present a less serious risk of harm to patients at a large tertiary hospital with in-house IT staff and robust quality assurance processes than to patients served by a safety-net provider with no in-house IT expertise and less extensive quality controls and resources than might be available to a large institution.

Comments.

Many commenters expressed support for ONC direct review in situations where ONC-ACBs may be unable to effectively investigate or respond to potential non-conformities. Several commenters recognized that there may be a variety of situations in which ONC-ACBs are unable to effectively investigate and respond to non-conformities, such as where doing so would require access to confidential or other information that is unavailable to an ONC-ACB, would exceed the resources or capacity of an ONC-ACB, or would involve novel or complex interpretations or application of certification criteria or other Program requirements. One commenter recommended that ONC invest in and empower ONC-ACBs to enable them to investigate and address non-conformities that are currently beyond the scope of their responsibilities under the Program.

All three ONC-ACBs commented on this aspect of our proposal. One ONC-ACB related that in its own surveillance it had encountered scenarios in which

ONC's direct oversight would have proven beneficial to the situation and its resolution. Another ONC-ACB stated that it had received complaints from users of certified health IT that raised issues (including issues related to patient safety) that were beyond the scope of the ONC-ACBs' accreditation and ability to address but that could be governed by the broader requirements of the Program. The remaining ONC-ACB did not believe that ONC should enforce any Program requirements that ONC-ACBs themselves could not administer in accordance with their accreditation; however, the ONC-ACB did support ONC's direct review of non-conformities whose nature, severity, or extent would be likely to quickly consume or exceed an ONC-ACB's resources or capacity.

Some commenters suggested that ONC should only intervene due to ONC-ACB limitations in very limited circumstances and that ONC should use its discretion in this respect as a “last resort.” One commenter suggested that ONC refine the factors that it will consider when determining whether to initiate direct review on this basis. Another commenter suggested that ONC should only initiate direct review on the basis of ONC-ACB limitations when clearly defined criteria are met; the commenter provided the example of a non-conformity involving the interaction of two health IT products certified by separate ONC-ACBs and having a proven and urgent impact on patient safety.

Response.

We thank commenters for their support and thoughtful comments on this aspect of our proposal. We have adopted the proposed approach to ONC direct review when ONC-ACBs may lack necessary expertise or resources, with the following clarifications. ONC may exercise direct review on the basis of suspected non-conformities that, while generally within the scope of an ONC-ACB's responsibilities and expertise, may present issues that could prevent an ONC-ACB from effectively investigating or providing an effective response. In these circumstances, ONC's direct review of the certified health IT is appropriate to help ensure consistency in the effective oversight and administration of the Program. Specifically, under the processes established in this final rule, ONC may directly review certified health IT if investigating or responding to a suspected non-conformity may require access to confidential or other information that is unavailable to an ONC-ACB (§ 170.580(a)(ii)(A)); may require concurrent or overlapping reviews by multiple ONC-ACBs (§ 170.580(a)(ii)(B)); or may exceed the scope of an ONC-ACB's resources or expertise (§ 170.580(a)(ii)(C)).

In response to the comments and to provide additional clarity regarding the types of circumstances that may exceed an ONC-ACB's resources or expertise, we provide the following example, which includes three alternative scenarios. The scenarios, which are mutually exclusive, illustrate how variations in facts and circumstances may give rise to different issues that necessitate different levels of involvement and forms of collaboration between ONC and ONC-ACBs.

Example F:

An EHR system certified to the 2015 Edition is in use by several major hospitals and health systems, including their ambulatory clinics, in multiple states. During a span of two weeks, over a dozen users at multiple health care facilities report to ONC and to the ONC-ACB that the EHR is displaying inaccurate or missing diagnoses (problems) and that, as a result, patients are not receiving appropriate care. In one reported instance, a patient was diagnosed with renal impairment, and this diagnosis was entered into the patient's active problem list in the EHR by her primary care physician (PCP). The PCP then referred the patient to an orthopedist for an unrelated musculoskeletal issue. The orthopedist is affiliated with the same health system as the PCP and has access to the same instance of the EHR. When the orthopedist accessed the patient's problem list, the diagnosis for renal impairment was missing from any relevant sections as displayed in the EHR. Unaware of this diagnosis, the orthopedist prescribed a medication for musculoskeletal pain that should either be avoided or minimized in patients with renal impairment. As a result, the patient suffered acute renal failure. Similar instances involving other missed or inaccurate diagnoses and resulting harm to patients have also been reported to ONC and the ONC-ACB.

Based on the information described above, the ONC-ACB initiates in-the-field surveillance of the certified health IT, as required by § 170.556(b), to assess whether the problem list capability continues to conform to the requirements of the certification criterion at § 170.315(a)(6) (Problem list). Separately, because the certified health IT may be performing in a manner that is causing or contributing to a serious risk to public or health or safety, ONC also initiates direct review of the certified health IT on this basis. ONC does not exercise exclusive

review under the Program at this time.

Under the final provisions, ONC may assert exclusive review of certified health IT as to any matters under its review and any similar matters under surveillance by an ONC-ACB. In determining if matters are similar, ONC will, as proposed, consider whether the matters are so intrinsically linked that divergent determinations between ONC and an ONC-ACB would be inconsistent with the effective administration or oversight of the Program.

Scenario 1

The ONC-ACB's in-the-field surveillance reveals that the cause of the issue is a software error that is only found in one EHR “workflow.” The EHR presents the user with multiple ways, or screens, to accomplish the same task. In this case, the PCP modified the problem list from a “quick summary screen,” which due to a software error did not write the updated diagnosis (problem) back to the database. This led to a situation where the PCP thought the diagnosis had been updated, but in fact on the back end, the list had not been updated. The EHR, when tested for certification, had presented the “standard office visit” screen for diagnosis list modification but not the “quick summary screen,” which is an alternate workflow available only in production.

The ONC-ACB concludes that the failure of the problem list capability to function in accordance with § 170.315(a)(6) was reasonably within the control of the developer, who should have anticipated the risk during the course of normal software development. Any additional read/write/display functionality may initially contain code errors, and all functions of certified health IT should be subjected to adequate testing. The developer could have reasonably taken actions to avoid the risk by employing an adequate software regression testing methodology.

Based on the surveillance and analysis above, the ONC-ACB finds a non-conformity to § 170.315(a)(6) and requires the developer to take corrective action, pursuant to § 170.556(d), including by submitting a CAP in accordance with §§ 170.556(d)(1)-(4) that addresses how the developer will resolve the identified non-conformity and related deficiencies across all of the developer's customers and users. ONC, in coordination with the ONC-ACB, concurs with the ONC-ACB's finding of non-conformity and, at this time, forbears from taking any action against the developer because the non-conformity involves a straightforward violation of a certification criterion, which is well within the scope of the

ONC-ACB's responsibilities and does not appear to exceed the ONC-ACB's resources. ONC continues to closely monitor the situation and coordinate with the ONC-ACB. If at any time ONC were to believe that the ONC-ACB could not effectively administer the necessary corrective action or that ONC's direct intervention were necessary to more quickly and effectively mitigate the risk to public health or safety, ONC could immediately issue a notice of non-conformity and notice of suspension, as described in section II.A.1.c of this preamble.

Scenario 2

The ONC-ACB's in-the-field surveillance reveals that the missing diagnosis was due to a system workflow implementation that the healthcare organization had customized. Contrary to the developer's recommendations, the healthcare organization had removed the problem list from the “quick visit” EHR workflow that is presented to ambulatory PCPs. This resulted in the PCP not being able to quickly and easily update the problem list properly, resulting in incomplete problem lists.

In contrast to scenario 1, the ONC-ACB finds that there is no non-conformity because these factors are beyond the developer's ability to reasonably influence or control. ONC concurs with the ONC-ACB's determination and ceases its direct review of the certified Health IT Module(s).

Scenario 3

Based on its in-the-field surveillance, the ONC-ACB finds that the problem list capability is functioning in accordance with § 170.315(a)(6). Specifically, the ONC-ACB concludes that the issue is not the result of any technical or functional deficiencies with the problem list capability but rather the manner in which the problem list's user interface has been designed, which is unintuitive and appears to h

This text is long and has been trimmed here. Open the source document for the complete record.

This is a copy of a public record, reproduced as it was published. It is not legal advice, and it may not be the version a court would rely on. Check the official source before you cite it.