Real World Testing Plan
Executive Summary
This is the Real World Testing plan for CY 2022 for Carefluence’s certified Open API solution.
As ONC has stated in its rule, “The objective of real-world testing is to verify the extent to which certified health IT deployed in operational production settings is demonstrating continued compliance to certification criteria and functioning with the intended use cases as part of the overall maintenance of a health IT’s certification.” In accordance with the ONC’s recommendation, Carefluence intends to conduct Real World Testing to verify that our Certified EHR continues to perform as intended with regard to interoperability and data exchange criteria. This test plan focuses on demonstrating that certified capability is utilized in real world care settings.
Health IT Developers have been given maximum flexibility to develop innovative plans and measures for Real World Testing. As we develop a plan to execute Real World Testing, the overall complexity of the workflows and use cases within the care settings are being consider in which Carefluence’s Health IT products are deployed.
We have included our milestones and timeline for completing Real World Testing below.
Developer Attestation
This Real-World Testing plan is complete with all required elements, including measures that address all certification criteria and care settings. All information in this plan is up to date and fully addresses the health IT developer’s Real World Testing requirements
| Authorized Representative Name: | Aditya Ayyagari |
| Authorized Representative Email: | aditya.ayyagari@carefluence.com |
| Authorized Representative Phone: | (248) 686-2600 x1011 |
| Authorized Representative Signature: |   |
| Date: | 11/10/2021 |
General Information
Plan Report ID Number: CF_RWT_2021
Developer Name: Carefluence
Product Name(s): Carefluence Open API
Version Number(s): 1.0
Certified Health IT Product List (CHPL) ID(s):
| CHPL Product Number | CHPL Product Number | CHPL URL |
| 1.0 | 15.04.04.2657.Care.01.00.0.160701 | https://chpl.healthit.gov/#/listing/7849 |
Timeline and Key Milestones
| Key Milestone | Date / Time Frame |
| Develop list of clients to assist with Real World Testing | Q1 2022 |
| Preparation and scheduling testing with clients | Q2 2022 |
| Collection of information as laid out by the plan | Q2 and Q3 2022 |
| CY 2022 Real World Testing plan will be completed according to ONC and ONC-ACB requirements and expectations. | Q4 2022 |
| Document our CY 2022 test results | January 2023 |
| Submit Real World Testing Report to ONC-ACB | February 2023 |
Standards Updates
| Standard (and version) | N/A |
| Updated certification criteria and associated product | N/A |
| Health IT Module CHPL ID | N/A |
| Method used for standard update | N/A |
| Date of ONC-ACB notification | N/A |
| Date of customer notification (SVAP only) | N/A |
| Conformance measure | N/A |
| USCDI-updated certification criteria (and USCDI version) | N/A |
Justification for Real World Testing
Real World Testing is intended to verify that deployed Certified Health IT continues to perform as intended by demonstrating that certified capability for interoperability and data exchange is successfully utilized in the real world.
All measures reasonably align with the elements within the Real World Testing plan, the scope of the certification, the types of settings in which the certified health IT is marketed, and other factors relevant to the implementation of the certified Health IT Module(s). Each element within the plan is relevant to the overall strategy for meeting the Real World Testing condition and maintenance of Certification requirements.
In each measurement description, we elaborate specifically on our justification for choosing this measure and the expected outcomes. Each measurement was chosen to best evaluate compliance with the certification criteria and interoperability of exchanging electronic health information (EHI) within the certified EHR.
Measures Used in Overall Approach
For each measurement/metric, describe the elements below:
- Associated certification criteria
- Description of each measurement/metric
- Justification for each measurement/metric
- Expected Outcomes for each measurement/metric
- Care Settings Targeted will be ambulatory care settings
We will be using reporting methods and screenshots to capture and demonstrate compliance. This methodology uses the logging or reporting capabilities to examine functionality performed in the system over a given interval. This measurement was chosen to best evaluate compliance with the certification criteria and interoperability of exchanging electronic health information (EHI).
In instances where no or little evidence exists due to lack of adoption of a certified measure or other reasons, we will demonstrate the requirements in a semi-controlled setting in a production site, otherwise known as a “real world” environment.
Care Settings Targeted
Carefluence is an Open API platform that supports HIT Vendors in fulfilling interoperability requirements for medical clinics in ambulatory and acute care settings. All measures outlined in the Real World Testing are designed for and will be performed within the ambulatory care settings through HIT Applications.
Real World Testing: Application Programming Interfaces
Measure 1
| Associated Certification Criteria: | 170.315 (g)(7): Application Access – Patient Selection |
| Measurement: | Count the number of API queries for a patient search made in a real world setting. |
| Description: | During the testing period, a report will be generated which tracks how many patient searches have been done by OpenAPI queries in the real world. |
| Justification: | OpenAPI queries result in the selection of a patient and demonstrate the audit log of search or selection of patient. |
| Expected Outcomes: | Demonstrate our compliance with the certification criteria and report the OpenAPI queries in selecting patients in a real world setting. |
| Associated Certification Criteria: | 170.315 (g)(8): Application Access – Data Category Request |
| Measurement: | Count the number of API queries for the category of clinical data set made in a real life setting. |
| Description: | During the testing period, a report will be generated which tracks and log the OpenAPI queries made for specific data category requests in the real world. |
| Justification: | Demonstrate Audit log report of OpenAPI queries with a patient as a subject and fetch that patient-specific data by category. |
| Expected Outcomes: | Demonstrate our compliance with the certification criteria and report the OpenAPI queries with a patient as a subject in real world setting. |
| Associated Certification Criteria: | 170.315 (g)(9): Application Access – All Data Request |
| Measurement: | Count the number of API queries for generating CCDA and all data requests |
| Description: | In real world settings, make all data request with patient as a subject in query and generate the CCDA 2.1 |
| Justification: | Demonstrating for a patient, the query of generating all data request and generate CCDA 2.1 |
| Expected Outcomes: | Demonstrate our compliance with the certification criteria and synchronize the OpenAPI query for a patient as a subject, to fetch the CCDA in real world setting |
Metrics and Outcomes: RWT Result – Year 2022
|
Associated Certification Criteria |
Outcomes,Methodology and Key findings |
Measurement/Metric (Count the Queries) |
| 170.315 (g)(7): Application Access – Patient Selection | Spirometry device integration with an EHR in a real-world setting using Carefluence OpenAPI to search and select patient with Spirometry orders. This workflow helped us measure the number of queries for the associated certification criteria within the reporting period. | 445 |
| 170.315 (g)(8): Application Access – Data Category Request | Spirometry device integration with an EHR in a real-world setting was using Carefluence OpenAPI to query different resource endpoints such as Diagnostic order, encounter etc., with the selected patient as a subject in query. This workflow helped us measure the number of queries for the associated certification criteria within the reporting period. | 445 |
| 170.315 (g)(9): Application Access – All Data Request |
CCDA 2.1 file generation in an EHR in a real-world setting was using Carefluence OpenAPI to request a bundle of FHIR resources with the selected patient as a subject in query and generate a CCDA 2.1 file. This request is launched typically in transition of care by providers, or view, download, transmit from patient portal. Further, a patient portal on iOS app connecting to an EHR was generating the CCDA file using the same set of queries before writing to Health App. This workflow helped us measure the number of all data request queries for the associated certification criteria within the report period.
|
1644 |
Real World Environment: Carefluence OpenAPI platform 1.0 was fully integrated with ModuleMD’ s EHR and is licensed to host the integrated OpenAPI to run on the servers of EHR’s Infrastructure. Real World Testing results were reported with the Spirometry device integration workflow using EHR’s hosted instance of Carefluence OpenAPI for interoperability.
WITHDRAWN PRODUCTS
Following product was withdrawn effective Jan 1 2023.
| Product Name(s): | Carefluence OpenAPI |
| Version Number(s): | 1.0 |
| CHPL Product Number(s): | 15.04.04.2657.Care.01.0.0.160701 |
| Date(s) Withdrawn: | Dec 31, 2022 12:00:00 AM ET |
|
Inclusion of Data in Results Report: [Provide a statement as to whether any data was captured on the withdrawn products. If so, this data should be identified in the results report.] |
RWT results for year 2022 was captured with this product and only at the end of the year after reporting period, we have withdrawn the product. Starting Jan 1 2023, we have Carefluence OpenAPI R4 version which is g10 certified is in Deployment for all the customers. |
![Logo-WHT-nds[1740]](https://carefluence.com/wp-content/uploads/2018/04/Logo-WHT-nds1740-200x45.png "Logo-WHT-nds[1740]"){kind=logo}