Phase 1 Deliverables

• Collaboration network and infrastructure drives progress toward a shared future vision for evidence-based, health IT-enabled care transformation that realizes the Quintuple Aim
  • A refined, expanded, and further vetted shared future vision
  • Defined requirements for the AHRQ DKP and other knowledge ecosystem components and related healthcare performance improvement enablers
  • A Roadmap Execution Steering Committee
  • A Coordinating Center to ensure that Roadmap execution leverages and supports the many other initiatives working to enhance the knowledge ecosystem
  • A plan to establish a PPP to drive Roadmap execution in Phases 2–4
• Production version of AHRQ Digital Knowledge Portal that delivers tools and resources that support the four Phase 1 clinical targets
  • A concept demonstration for this was produced during ACTS Roadmap development (68)
• Concept demonstration and requirements for a Production AHRQ DKP that:
  • Makes all tools and resources more FAIR and useful
  • Makes evidence, guidance, tools, and data on the Phase 1 clinical targets more computable and interoperable
  • Demonstrates tooling to support computable processing of offerings around the knowledge ecosystem (see the What a Digital Knowledge Ecosystem Will Enhance section)
• Concept demonstrations and requirements for tooling that makes information processing in the knowledge ecosystem cycle steps (see Figure LHS Functions That the Knowledge Ecosystem Supports) more computable and useful and information transfer between these steps more seamlessly interoperable, which includes:
  • A national/international reference architecture to undergird interoperable DKPs from AHRQ and other sources that support developing, disseminating (e.g., via interoperable public and private marketplaces), accessing and implementing FAIR/computable knowledge assets, and integrating them with data for user-friendly application in various workflows
  • An open, public-sector marketplace (Figure Digital Healthcare Knowledge Ecosystem)
• Living evidence, guidance, and CDS interventions/eCQMs for the four Phase 1 clinical targets, produced and maintained in a manner that demonstrates more seamless flow from studies to systematic reviews to guidance to CDS/eCQMs to implementing the CDS/eCQMs into information systems and care delivery workflows to data about care delivery processes and results to QI efforts on the target and evidence generation and requirements for additional research
• A Living Guidance Development Playbook for creating and maintaining living, computable evidence, guidance, and CDS/eCQMs that is coordinated with investments from other stakeholder organizations to generate supporting educational materials (e.g., courses, learning communities)
• Results from developing and clinically implementing the AHRQ Digital Knowledge Portal and living CDS interventions and eCQMs for the four Phase 1 clinical targets in at least five different CDOs, affecting care for at least 50,000 patients, which includes assessing and documenting the implications of these results for developing and implementing the AHRQ DKP and other knowledge ecosystem components during the Phase 2 pilots
• A playbook for successfully implementing CDS interventions and eCQMs for the four Phase 1 clinical targets (applicable to other targets as well) (i.e., Implementation Playbook) that is coordinated with investments from other stakeholder organizations to generate supporting educational materials (e.g., courses, learning communities)
• Requirements and project plans for full Production versions of the AHRQ DKP and other knowledge ecosystem components and for producing and testing the tooling to make ecosystem cycle functions more computable and usable

Phase 1 Tasks, Timeline & Costs

Table 2. ACTS Phase 1 Create/Use Governance & Collaboration Tasks, Timeline 

Table 3. ACTS Phase 1 Enhance/Leverage Infrastructure Tasks, Timeline  [1]

Table 4. ACTS Phase 1 Enhance/Develop Living, Computable Guidance Tasks, Timeline 

Table 5. ACTS Phase 1 Enhance Guidance Implementation & Assessment Tasks, Timeline 

Table 6. ACTS Phase 1 Evaluate/Plan Roadmap Execution Tasks, Timeline 

Phase 1 Create/Use Governance & Collaboration

Despite many diverse efforts to better put evidence into practice and foster LHSs, progress toward the Quintuple Aim has been disappointingly slow. There still isn’t a widely adopted shared vision for the desired future state or a clear path to achieve it. The governance and collaboration efforts in Phase 1 drive the broad adoption of a shared future vision for a knowledge ecosystem and benefits it delivers. This work builds on the foundation established by the ACTS Stakeholder Community  (52) and Future Vision Workgroup (see Appendix B, Future Vision) (e.g., by engaging more stakeholders, covering more perspectives around the ecosystem cycle, and adding additional future vision details). These governance and collaboration mechanisms also guide execution of the Phase 1 tasks to drive progress toward this vision and ensure that the federally funded actions and results are amplified by complementary actions by other sectors.

Form Phase 1 Roadmap Execution Steering Committee for Concept Demos

The broadly representative Roadmap Execution Steering Committee oversees requirements development and concept demonstrations for a new AHRQ DKP and other interoperable knowledge ecosystem components (e.g., new or enhanced computable guidance, and DKPs and marketplaces from other organizations). These concept demonstrations drive knowledge ecosystem cycle enhancements for the four initial clinical target areas in ways that align with a consensus future vision.

The Roadmap Execution Steering Committee will:

• Refine the four proposed Phase 1 clinical targets to define exactly what aspects of these targets will be addressed by enhanced guidance content and technology for creating and applying this content based on the most pressing healthcare needs; includes defining a consensus future vision addressing how the knowledge ecosystem cycle should function for those targets
• Ensure that concept demonstrations are designed, executed, and evaluated in a manner that leverages major efforts focused on the clinical targets (e.g., USPSTF for preventive care, ACTS COVID Collaborative participant efforts for pandemic response, the American Medical Association’s (AMA’s) efforts to enhance blood pressure control nationwide, PCCDS-LN’s OAP) and pertinent knowledge ecosystem enhancements (including refining and using pertinent standards)
• Ensure that the concept demonstrations provide requirements and valuable models for desired knowledge ecosystem components from AHRQ and others (e.g., DKPs, computable resources, marketplaces – and a reference architecture to enable interoperability across these)
• Set the stage for broad scaling of potential ecosystem enhancements to all other clinical improvement targets, any care delivery setting, and critical ecosystem stakeholder needs as outlined in the consensus future vision
• Identify critical dependencies for achieving the knowledge ecosystem/LHS future vision (e.g., payments, incentives) and outline a stakeholder-driven plan to address these dependencies and policy drivers to accelerate the “slow walk to value based care” (69) (e.g., by realigning business incentives and policies with shared LHS goals, building on current policy/payment efforts by public payers/regulators such as CMS (as outlined in Appendix A, Interplay With a Sampling of Other Strategic Plans, Priorities & Initiatives) and similar efforts by private payers and other market drivers)
• Oversee the development and execution of the Roadmap Research and Implementation Evaluation Plan

The Roadmap Execution Steering Committee will be open to contributions from individuals, organizations, and initiatives in a position to contribute to and benefit from achieving LHS/knowledge ecosystem goals; it will provide a route for them to engage with the Roadmap Execution Steering Committee and provide input and feedback. The Roadmap Execution Steering Committee will be seeded by work and stakeholders from the ACTS Stakeholder Community  (52), ACTS COVID‑19 Guidance to Action Collaborative Learning Community (70) and other ACTS pilots (see Appendix E, AHRQ-Funded ACTS Pilots), and other related initiatives such as MCBK (71) and many others (see Figure 8. Sampling of Non-AHRQ Initiatives Addressing the Knowledge Ecosystem With Which Roadmap Execution Coordinates).

The Roadmap Execution Steering Committee provides a forum to help stakeholders advance their healthcare and public health-related goals in a manner that simultaneously supports and guides Roadmap execution. Table 7. Sample Stakeholder Concerns and Actions provides examples of these synergies.

Table 7. Sample Stakeholder Concerns and Actions

Establish Collaboration Infrastructure

Digital collaboration tools such as web meeting platforms, shared online documents for group editing, online databases for gathering and processing information about participant activities and insights, and a web-based collaboration platform all played important roles in developing this Roadmap (e.g., see the ACTS COVID Collaborative Learning Community web platform (55)). Roadmap execution will require even more robust and tightly managed infrastructure to support the highly collaborative and coordinated activities needed for successful Roadmap execution. This task includes reviewing strengths and limitations of the collaboration infrastructure used in Roadmap development and ACTS pilot efforts, along with other available collaboration tools, to design and implement infrastructure to support Phase 1 collaboration and coordination needs.

Support Phase 1 Roadmap Execution Steering Committee

Refine Four Clinical Targets for Concept Demos

The Phase 1 Roadmap Execution Steering Committee selects which specific aspects of the initial four clinical target areas will be addressed through efforts to improve specific care decisions and actions through enhanced CDS interventions and other enhanced knowledge ecosystem cycle components. For example, more specific targets might include:

• Pandemic response: Managing chronic COVID symptoms, selecting appropriate medications for acute treatment, assessing disease severity for triage decisions
• Hypertension: Support for patient self-monitored blood pressure, medication selection, patient self-management
• Pain management / opioid use: Non-opioid management of chronic non-cancer pain, appropriate opioid use for severe acute pain
• Prevention: Supporting specific types of cancer screening, counselling on lifestyle changes

These specifics will be chosen based on the potential to drive population health improvements, evidence supporting the value of interventions focused on the topic, and other pragmatic considerations such as major complementary national initiatives that could enhance Roadmap-supported efforts to address the target while simultaneously enhancing pertinent knowledge ecosystem components. Healthy People 2030’s healthcare objectives  (72) will inform target refinement.

Establish Criteria to Select Resources for AHRQ DKP & Other Concept Demos

The Roadmap Execution Steering Committee will establish the criteria to select existing resources pertinent to the four targets form AHRQ (Appendix F, AHRQ Offerings & the Knowledge Ecosystem Cycle) and others to ensure that the AHRQ Portal and Phase 1 concept demonstrations are designed, executed, and evaluated in a manner that fully leverages the best available resources.

Establish Criteria to Select Clinical Sites to Participate in Guidance Implementation

There is a short timeline for developing, implementing, and evaluating CDS interventions and eCQMs for the Phase 1 targets. Selecting implementation sites with a proven track record will help ensure success, though may limit results scalability. The Roadmap Execution Steering Committee will establish selection criteria that balance these types of tradeoffs in ways that optimize success and scalability of Phase 1 implementation efforts. 

Support Requirements Development for Concept Demos

This task supports the direct work to develop the concept demonstration requirements that are covered by tasks 1.5.3, 1.5.5, and 1.6.1. The Roadmap Execution Steering Committee ensures broad stakeholder engagement in requirements development to ensure that concept demonstration tools and resources developed from those requirements are very likely to evoke strong interest in wide use and high expected valued for addressing the targeted needs and activities.

Support Agile Development for Concept Demos

This task supports the direct work to develop the concept demonstrations that are covered by tasks 1.5.4, 1.5.6, 1.6.2, 1.6.3, 1.6.4, 1.6.5, 1.6.6, and 1.6.7. The Roadmap Execution Steering Committee ensures broad stakeholder engagement in concept demonstration development to ensure that concept demonstration tools and resources are very likely to evoke strong interest in wide use and high expected valued for addressing the targeted needs and activities.

Support Concept Demo Feedback Evaluation & Phase 2 Project Plan Development

This task supports the direct evaluation and project planning work that is covered by 1.14. The Roadmap Execution Steering Committee ensures broad stakeholder engagement in these activities to ensure that all key stakeholder perspectives are included, and their implications for evaluation and planning are addressed.

Establish the Coordinating Center

The Coordinating Center is responsible for coordination among pertinent AHRQ (Figure 9. How AHRQ Supports Key Tasks in the Knowledge Ecosystem Cycle) and external (Figure 8. Sampling of Non-AHRQ Initiatives Addressing the Knowledge Ecosystem With Which Roadmap Execution Coordinates) initiatives that are related to all Phase 1 tasks and deliverables.

The Coordinating Center will be established and applied to supporting Phase 1 activities by building on related public and private efforts (e.g., by HL7 [FHIR accelerators (73)], the COVID‑19 Evidence Network to support Decision-making (COVID-END) (74), and other pertinent initiatives described in Appendix A, Interplay With a Sampling of Other Strategic Plans, Priorities & Initiatives). The Coordinating Center will leverage and support related activities under AHRQ’s CDS Connect sustainability, PPP, and innovation collaborative efforts, as appropriate.

Institute Project Management for Phase 1 Activities

Comprehensive, sophisticated project management approaches and tools will be required to ensure that the many interdependent Phase 1 tasks are executed efficiently and effectively.

Cultivate Opportunities for Public & Private Organizations to Contribute & Benefit from Phase 1

Although ambitious, the Phase 1 tasks and deliverables will be most directly driven by only a relatively few targets, organizations, and tools. In this task, the Coordinating Center will cultivate opportunities for public and private organizations not directly funded by investments supporting Phase 1 activities to contribute to and benefit from all Phase 1 activities. For example, by addressing additional topics and implementation sites that harmonize with Phase 1 work but are funded by other sources.

Coordinate Concept Demo Requirements Gathering, Development, Evaluation &Phase 2 Pilot Planning with Other Initiatives

While the Roadmap Execution Steering Committee identifies the broad group of pertinent stakeholders and ensures their engagement in these activities, the Coordinating Center is responsible for actually coordinating pertinent activity within AHRQ (Appendix F, AHRQ Offerings & the Knowledge Ecosystem Cycle) and in other initiatives Appendix A, Interplay With a Sampling of Other Strategic Plans, Priorities & Initiatives to optimize processes and results in Phase 1.

Hold an Annual Meeting

During Roadmap development, Stakeholder Community members expressed a desire to have face-to-face meetings for networking, deep dives into challenging issues and to realize other collaboration benefits from such gatherings. Although there weren’t resources to support such meetings, the project team did leverage gatherings such as the 2019 AMIA Annual Symposium where many Stakeholder Community members were present for such purposes (e.g., gaining consensus around knowledge ecosystem frameworks).

For Roadmap execution, an annual gathering will be important to achieve the collaboration and coordination goals. The meeting should be in-person, but if obstacles such as the COVID‑19 pandemic prevent this, a robust virtual event will be substituted. Many organizations have shown that such virtual events—when planned and executed carefully—can achieve many of the same results as face-to-face gatherings.

Plan to Form a PPP to Manage Roadmap Execution

Given the scope and impact of Roadmap activities related to public and private sector activities and goals, a PPP to support and guide Roadmap execution will help ensure that it fully leverages stakeholder resources in addressing their needs. Once formed in Phase 2, the PPP assumes responsibility for Roadmap-related governance and coordination activities conducted by the Roadmap Execution Steering Committee and Coordinating Center. It also enhances these efforts over the remaining 10-year Roadmap horizon, ensuring that the knowledge ecosystem/LHS goals and future vision are achieved in ways that meet stakeholder needs and enable them to drive progress. Planning for this PPP in Phase 1 will leverage and support related activities under AHRQ’s CDS Connect sustainability, PPP, and innovation collaborative efforts, as appropriate.

Phase 1 Enhance/Leverage Infrastructure

These Phase 1 activities will produce concept demonstrations for:

• AHRQ DKP functions that advance FAIRness, computability, and usefulness for AHRQ resources and tools that support the knowledge ecosystem cycle (see Figure 9. How AHRQ Supports Key Tasks in the Knowledge Ecosystem Cycle)
• Similar key knowledge ecosystem cycle functions in other public and private DKPs
• The reference architecture that supports interoperability within and across these platforms
• A free/open Federal marketplace for “one-stop shop” access to critical resources around the knowledge ecosystem cycle and interoperability between this Federal-integrated marketplace and other private-sector marketplaces

These concept demonstrations use core clinical target specifics validated by the Roadmap Execution Steering Committee as central use cases. Other uses cases may be added by the Phase 1 Roadmap Execution Steering Committee (e.g., based on strong interest and supported by pertinent stakeholders). These demonstrations illustrate how enhanced information flow for these clinical targets around the knowledge ecosystem cycle could be enabled by a new AHRQ DKP and other interoperable knowledge ecosystem components (e.g., new or enhanced digital guidance, and DKPs and marketplaces from other organizations).

Develop an AHRQ DKP Concept Demo

This task addresses AHRQ’s central goal in establishing the ACTS initiative. That is, providing an approach for AHRQ programs (Appendix F, AHRQ Offerings & the Knowledge Ecosystem Cycle) to make their resources more FAIR, computable, and useful in ways that support its mission of producing evidence and ensuring that it is understood and used to improve health and healthcare. Work under this task leverages the Phase 1 Create/Use Governance & Collaboration tasks to produce an AHRQ DKP concept demo for delivering valuable tools and resources from AHRQ —enterprise wide, along with other selected sources—in ways that are sensitive to user roles, context, and needs. The AHRQ DKP concept demo will illustrate how AHRQ can help users better leverage its own offerings in the context of information and tools available from other public and private sources.

Produce a Directory of AHRQ Tools & Resources

To seed the AHRQ DKP concept demo, a Production-grade portal is developed to provide valuable tools and resources from AHRQ (Appendix F, AHRQ Offerings & the Knowledge Ecosystem Cycle) and other selected sources that can help stakeholders with knowledge ecosystem cycle activities pertinent to the four Phase 1 targets. The portal draws from all pertinent tools (e.g., EvidenceNOW Tools for Change (75), Prevention TaskForce (76), CDS Connect Repository  (49), evidence and guidance (e.g., from AHRQ EPCs (77), SRDR+ (43), and the USPSTF (78)), QI resources (e.g., from CQuIPS (47)), and healthcare performance data (e.g., from the Center for Financing, Access, and Cost Trends [CFACT] (79)) from all three AHRQ centers and all AHRQ websites. This task involves identifying and producing a living directory of these pertinent AHRQ tools and resources that will feed thee AHRQ portal concept demo.

The Roadmap Execution Steering Committee identifies and iteratively refines selection criteria for directory entries by reviewing sample high-value content from AHRQ centers/websites and synthesizing criteria for items most valuable for ecosystem cycle activities. Work under this task includes tagging these resources to indicate the clinical target covered, ecosystem cycle step supported, and other metadata determined to be important. A structured, web-based data entry tool (e.g., analogous to the one used to manage knowledge ecosystem enhancement approaches in the ACTS COVID Collaborative (80)) is used to compile this information. The directory listings will be made computable using the standard FHIR Citation Resource to describe each directory entry with rich metadata to support FAIRness.

Augment the Directory With Authoritative Select Non-AHRQ Tools & Resources

As for the AHRQ offerings, produce a living directory of pointers to valuable, free/open online tools and resources from authoritative non-AHRQ sources to populate the concept demo portal. For example, to provide access to offerings from other HHS/Federal sources as well as from private, vetted, evidence-based sources such as specialty societies and academic medical centers. These offerings are selected to support decisions and actions around the knowledge ecosystem cycle for each clinical target per the Roadmap Execution Steering Committee-defined criteria. Directory entries are selected and tagged in a similar manner to entries from AHRQ.

Define Requirements for the AHRQ Portal Concept Demo Website

This task defines requirements for the AHRQ Portal concept demo website that provides users with information, resources, and tools from AHRQ and other sources. This information is focused on the four Phase 1 targets and is delivered by the portal in a user-friendly manner that is highly responsive to the user’s role in the ecosystem cycle and specific need. Information is delivered through interfaces for searching, browsing, and APIs. Requirement definition addresses repository/database capabilities, content tagging/indexing details (e.g., which tags to use and how to apply them), UI and API specifications, and content management functions. The content that populates this portal is developed under tasks 1.5.1 (AHRQ) and 1.5.2 and (non-AHRQ). The requirements leverage CEDAR API  (42) background activities and tooling.

Requirements include determining how best to manage information on the portal website within the scope of available concept demo time and resources. That is, addressing when the website will provide access to living content posted on the (AHRQ or others) source’s website versus when (for expediency) a snapshot of the source content will be posted on the AHRQ Portal concept demo website along with notification that a more complete/current version might be available from the original source.  

Test the hypothesis that private/commercial vendors will want to have pointers to their material in the Portal and help establish commerce mechanisms (see 1.6.6, Produce an Open, Unified, Integrated Federal Marketplace Concept Demo) because the knowledge ecosystem cultivated by this Roadmap will evolve into a compelling “front door” and communication for reaching and supporting customers.

Produce the AHRQ Portal Concept Demo Website

Produce a fully functional Production portal that helps users addresses needs throughout the knowledge ecosystem cycle for the four Phase 1 targets, while also serving as a concept demonstration for portal capabilities within an AHRQ DKP that is capable of providing pertinent information about any topic that AHRQ offerings cover. Leverage the preliminary portal concept demo work (68) by the Making GRADE the Irresistible Choice (MAGIC) Evidence Ecosystem Foundation in producing this deliverable.

Determine Requirements to Expand the AHRQ Portal Concept Demo Website into an AHRQ DKP Concept Demo

This AHRQ DKP delivers to different user roles (e.g., patients, care teams, QI teams, guideline developers, systematic reviewers, CDS implementers, public health professionals, policy makers, and many others) pertinent, vetted, and curated information from AHRQ and others that is responsive to their needs at all steps around the knowledge ecosystem cycle. This information spans the four knowledge representation levels (81), and includes narrative (L1), semi-structured (L2), structured (L3), and executable (L4) resources. Computable artifacts (i.e., L3, L4) are provided via CDS Connect.

This AHRQ DKP concept demo includes all AHRQ Portal concept demo functions, and it demonstrates how all current AHRQ offerings could be made more FAIR, while actually delivering this FAIRness for offerings related to the four Phase 1 targets. The broader AHRQ DKP concept demonstration also illustrates how selected AHRQ resources could become more computable and standards-based, and how AHRQ tools could better support development of computable resources. Examples of these enhancements are outlined in the What a Digital Knowledge Ecosystem Will Enhance section.

In addition to the AHRQ DKP (see Figure 10. AHRQ DKP) improving access to and use of AHRQ offerings, it is also the seed for an open, public-sector “one-stop-shop” portal and marketplace where users can easily find and access (e.g., via federated search, browse, and API functions) free information, tools, and resources from Government agencies and others that support decisions and actions throughout the knowledge ecosystem cycle. This AHRQ DKP-seeded marketplace interoperates smoothly with other DKPs and marketplaces as depicted in Figure 11. Digital Healthcare Knowledge Ecosystem. The AHRQ DKP concept demo illustrates how AHRQ can contribute to the world’s repository of computable content for each ecosystem cycle step as well as contributing tooling to create this computable content (see section 1.6. Develop concept demos for integrated tools that support the knowledge ecosystem cycle).

The concept demo will determine what content should be placed within the AHRQ DKP versus what content is accessed via proper interfaces and metadata from primary sources, balancing single source of truth (which favors linking to primary source) and integration complexity (which favors managed data redundancy). Once developed, the ADKP concept demo will provide a springboard for defining requirements for a full Production ADKP platform (e.g., related to servers, software, search engine, UI, dashboard, data aggregation/visualization, API, data analytics as well as processes for change management, metadata management and system maintenance, and content management [master index, metadata taxonomy/ontology, mechanism to tag/classify artifacts, etc]).

Requirements for this AHRQ DKP concept demo will be determined by feedback collected from review of the AHRQ Portal Concept Demo, output from the AHRQ-funded ACTS Evidence/Guidance Computability Tools Requirements Pilot, as well as from AHRQ-sponsored challenges and connectathons (details to be guided by the Roadmap Execution Steering Committee), and other related discussions among stakeholders within and across all ecosystem cycle steps, e.g., as fostered by the Roadmap Execution Steering Committee and the Coordinating Center.

These AHRQ DKP concept demo requirements include seeding requirements for the reference architecture that will undergird the knowledge ecosystem (see Figure 11. Digital Healthcare Knowledge Ecosystem). These reference architecture requirements support system interoperability and resource computability across public and private DKPs and marketplaces. The AHRQ DKP and reference architecture requirements will leverage and build on report deliverables from the ACTS Marketplaces Workgroup (Appendix D, Marketplaces). They include standardized approaches to indexes, metadata, taxonomies, APIs, sandboxes, trusted curation, and other features needed to create a knowledge ecosystem that makes content and resources from AHRQ and others more FAIR, computable, useful, and widely used. The requirements for these also build on the ACTS Infrastructure/ Standards Workgroup landscape analysis (Appendix C, Infrastructure) to drive industry consensus and adoption around standards and a reference architecture for content representation, interoperability, and consumption needed to achieve the ACTS Future Vision. Work from relevant standards and FAIRness efforts (e.g., the MCBK Standards Work Group  (82), OMG BPM+ Health COVID‑19 Navigator Tiger Team  (83) and other related initiatives listed in Appendix A, Interplay With a Sampling of Other Strategic Plans, Priorities & Initiatives).

There have been various efforts to develop models and schemas related to the knowledge ecosystem cycle that will be leveraged to create the AHRQ DKP and reference architecture (see Appendix A, Interplay With a Sampling of Other Strategic Plans, Priorities & Initiatives), and leaders from many of these have participated actively in Roadmap development. Figure 16–Figure 20 provide a sampling of these models and schemas.

Figure 16. Model for an Industry-Scale Knowledge Ecosystem

Figure 17. Knowledge Representation Levels & Implementation Tiers

Figure 18. Schema for Evidence-Related FHIR Resources (84)

Figure 19. FHIR and the Evidence Ecosystem Cycle (85)

Figure 20. HL7 Standards & Ecosystem Cycle (86)

Figure 21. FHIR-Based Knowledge Representation Specifications  (87)

Explanation about Figure 21. FHIR-Based Knowledge Representation Specifications  (84) from its author, Bryn Rhodes: “This diagram depicts four categories of specifications, with representative examples of each category, illustrating how the various pieces can be used together to deliver shareable clinical reasoning artifacts such as quality measures and decision support rules. Along the bottom of the diagram are the foundational standards, including FHIR, with the five layers of Foundation, Conformance, Administration, Clinical, and Reasoning, as well as expression language and integration standards including FHIRPath, Clinical Quality Language (CQL), CDS Hooks, and SMART on FHIR. In the middle left are the Model Implementation Guides (IGs), typically derived from Administration and Clinical resources such as Patient, Encounter, and MedicationRequest. Model IGs are typically built to address a broad range of use cases, focused on a particular target realm or domain. In the middle right are the Specification IGs, which derive from the FHIR Clinical Reasoning resources to provide implementation guidance and conformance requirements for the creation, distribution, evaluation, and maintenance of shareable clinical knowledge. For example, the Quality Measure IG provides guidance on and conformance requirements for the use of the FHIR Measure and Library resources to create and share clinical quality measures, and the Clinical Practice Guidelines IG (CPG-on-FHIR) demonstrates how to build computable guideline (88) content. And finally, at the top of the diagram, the Content IGs are FHIR IGs, but not necessarily balloted as HL7 standards, rather these use the FHIR publication toolchain to support authoring and distribution, but the content is stewarded by separate authorities such as quality agencies and guideline developers; groups that have their own governance and maintenance policies. For example, the Healthcare Effectiveness Data and Information Set (HEDIS) IG contains HEDIS quality measures expressed using FHIR Measure and Library resources, and conforming to the Quality Measure IG profiles, while the CDC Opioid Prescribing IG and World Health Organization (WHO) Antenatal Care IGs contain decision support content to streamline guideline implementation.”

Produce the AHRQ DKP Concept Demo

Using Agile methods, develop an AHRQ DKP concept demo that illustrates what could be done to make AHRQ tools and resources more FAIR, computable, and useful in ways that address all the defined requirements. Below are notes about interplay between the AHRQ DKP concept demo and related efforts and resources in AHRQ centers (see Appendix F. AHRQ Offerings & the Knowledge Ecosystem Cycle).

• CEPI:
  • Enhance CDS Connect website into CDS Connect 2.0. By 2025, CDS Connect 2.0 provides the primary tool for creating Federal CDS interventions [L3/L4] (freely available to anyone who wants to use this tool) and the primary source for accessing these interventions—as well as interventions from anyone else that wants to make them available via this free marketplace.
    • Evolve CQL authoring tool into the unified Federal CDS authoring environment (e.g., add BPM+ capabilities, cover all types of CDS interventions that are needed/valuable)
    • Build out marketplace functionality (e.g., trust, digital rights management [DRM], curation, product labeling)
  • Explore enhancements to an NGC successor, if becomes available, so there’s seamless integration across all separate websites, works with APIs/portal, tied into CDS Connect 2.0 efforts
    • Optimize ways that guidelines get into NGC and out of NGC, get vetted for trust, support development of CDS interventions, etc. all happen in ways that leverage—and add value back to—pertinent components of the knowledge ecosystem
  • Enhance other CEPI evidence-based resources and websites into 2.0 versions:
    • SRDR+
    • Effective Healthcare (EPC)
    • Academy for Integrating Behavioral Health and Primary Care
    • Digital Healthcare Research
    • Patient-Centered Medical Home
    • Practice-Based Research Networks (PBRNs)
    • S. Health Information Knowledgebase/eCQMs
  • CFACT: coordinate with data harmonization project; establish proper tagging to enable interface/integration with other components of AHRQ DKP
    • Data Harmonization Effort
      • Medical Expenditure Panel Survey (MEPS)
      • Healthcare Cost and Utilization Project (HCUP) User Support (HCUP-US)
      • Healthcare Cost and Utilization Project Network (HCUPnet)
      • Consumer Assessment of Healthcare Providers and Systems (CAHPS)
      • National Healthcare Quality and Disparities Reports (NHQDR)
    • CQuIPS: coordinate and develop interfaces with patient, safety, and quality content, resources, and tools to ensure proper integration within AHRQ DKP
      • AHRQuality Indicators™
      • Patient Safety Network (PSNet)
      • NHQDR
      • Patient Safety Organizations
      • Patient Safety Organizations Privacy Protection Center
      • Morbidity & Mortality Rounds on the Web
      • CAHPS
    • Cross-cutting / Office of Communications (OC)/ chief information officer (CIO): coordinate and develop interfaces with AHRQ.gov/OC/CIO efforts to ensure proper integration within AHRQ DKP
      • gov
      • Project Resource Online Database
        • Develop Concept Demos for Integrated Tools That Support the Knowledge Ecosystem Cycle

As illustrated in Figure 11. Digital Healthcare Knowledge Ecosystem, the AHRQ DKP is one component of the much larger and complex knowledge ecosystem, and, as shown in Figure 9. How AHRQ Supports Key Tasks in the Knowledge Ecosystem Cycle the AHRQ DKP supplies only a portion of the critical knowledge ecosystem tools and functions. In this series of tasks, concept demos are produced for integrated tools that support key activities around the knowledge ecosystem cycle—complementing and interoperating seamlessly with AHRQ DKP resources and functions.

With use cases based on living resources being developed under Table 4. ACTS Phase 1 Enhance/Develop Living, Computable Guidance Tasks, Timeline  and implemented under Table 5. ACTS Phase 1 Enhance Guidance Implementation & Assessment Tasks, Timeline & Costs, these concept demos address how stakeholder can create and consume information around the ecosystem cycle more efficiently and effectively and achieve better results. With support from the Roadmap Execution Steering Committee and the Coordinating Center, these tasks are executed in a manner that fully leverages current and emerging resources and approaches from AHRQ (see Appendix F, AHRQ Offerings & the Knowledge Ecosystem Cycle, other national initiatives (see Appendix A, Interplay With a Sampling of Other Strategic Plans, Priorities & Initiatives), and other drivers in the healthcare environment. For example, other current and emerging DKPs such as the application marketplaces from EHR vendors and other IT suppliers.

An important resource for all the subtasks below is the living repository of Approaches to Improving the Knowledge Ecosystem Cycle (80) being developed by the ACTS COVID Collaborative  (55) described in E.3.1, ACTS COVID‑19 Evidence to Guidance to Action Collaborative Pilot.

Determine Requirements for Ecosystem Cycle Support Tooling Concept Demos

The Roadmap Execution Steering Committee will engage stakeholders in each individual knowledge ecosystem cycle component and the entire cycle to define requirements for a series of integrated ecosystem cycle support tooling concept demos that make the entire cycle more computable, FAIR, and useful. Once implemented in subsequent Roadmap execution Phases, these tools will improve the efficiency and effectiveness of each cycle component and the ability of the knowledge ecosystem to broadly support LHS and realize the Quintuple Aim.

It is important to note that aim for these requirements it to produce systems that only automate and standardize things that should be more automated (as outlined in the What a Digital Knowledge Ecosystem Will Enhance section). There are important judgement activities at each step of the knowledge ecosystem cycle (e.g., related to evidence evaluation and clinical decisions), and trying to fully automate activities that require human judgement is inappropriate, dangerous, and not at all what is intended for this task or the broader Roadmap.

These requirements will be informed by the AHRQ DKP requirements, including requirement for the underlying reference architecture. These requirements will leverage, adapt, and extend currently available offerings for each component and national/international efforts to support computable guidance development (e.g., clinical practice guidelines on FHIR IG (86) (89) and its application by authors within the COVID‑19 Digital Guidelines Workgroup  (90), the MCBK initiative (71), and others).

The requirements will also address the following:

• For tooling that supports identifying, processing, and communicating a specific type of knowledge:
  • Setting a defined schema for the knowledge type (e.g., FHIR resources)
  • Demonstrating functional data entry forms (UIs) so people can specify the knowledge in ways that are converted to the defined schema
  • Demonstrating functional data transformations so knowledge in structured data formats can be automatically converted to the defined schema
  • Demonstrating functional UIs for people to search and view the specific knowledge
• For standards supporting the infrastructure (data transfer schema, terminologies):
  • Demonstrating the standard is adequate to convey the desired knowledge
  • Coordination with the standard developer to adjust the standard (or adjust the implementation) as needed to convey the desired knowledge

All the subtasks outlined below use the Roadmap Execution Steering Committee-selected clinical dimension for each of the four Phase 1 clinical targets as core use cases.

Produce a Computable Literature Management Tooling Concept Demo

Using Agile methods, produce (or adapt from available system(s)) a computable literature management tooling concept demo to illustrate creating and managing a living repository of current and emerging high-impact studies on specific targets. Leverage capabilities related to this that are currently available in systems (e.g., the L·OVE platform (91)). This computable literature management concept demo will automate the steps to populate the living repository of high-impact studies and will make information about the studies computable using standards. This task builds on the Tool 1 requirements for computable study details results being produced within the ACTS COVID computable evidence/guidance requirements pilot (see E.3.2, Evidence/Guidance Computability Tools Requirements Pilot).

This task includes documenting stakeholder feedback about how the concept demo could be enhanced and applied to improving efficiency and effectiveness of topic-focused literature surveillance (including both processes and technologies), and synthesizing best practices and processes based on that feedback. These results will inform the Living Guidance Playbook (described in 1.11).

Produce a Systematic Reviews Tooling Concept Demo

Using Agile methods, produce (or adapt from available system(s)) a concept demo of tooling to create and maintain living reviews in a computable fashion, including automating manual processes prone to error (e.g., data entry). This concept demo interoperates seamlessly by receiving computable data from literature surveillance concept demo tool (described in 1.6.2) and providing computable data to the concept demo guideline management tool (described in 1.6.4). It leverages Tool 2 requirements for making systematic reviews computable from the ACTS tool requirements pilot (see E.3.2, Evidence/Guidance Computability Tools Requirements Pilot), cultivates synergies with efforts by systematic review platforms (92), and leverages and seamlessly integrates with current and planned SRDR+ functionality.

This task includes documenting stakeholder feedback about how the concept demo could be applied to improving efficiency and effectiveness of producing living systematic reviews (including both processes and technologies) and synthesizing best practices and processes based on that feedback. These results will inform the Living Guidance Playbook (described in 1.11).

Produce a Living Guidelines Tooling Concept Demo

Using Agile methods, produce (or adapt from available system(s)) a concept demo tool that supports developing and maintaining living guidelines in a computable, standards-based, automated fashion, leveraging current and planned SRDR+ functionality. This guideline tooling concept demo interoperates seamlessly by receiving computable data from the systematic reviews tooling concept demo (described in 1.6.3) and providing computable data to the concept demo CDS/eCQM authoring tool (described in 1.6.5). This tooling could leverage or emulate tools such as MAGICapp (93). It also leverages computable guidance rationale and processing value sets for guideline concepts and requirements for Tools 3 and 4 from the ACTS tooling pilot (see E.3.2, Evidence/Guidance Computability Tools Requirements Pilot).

This task includes documenting stakeholder feedback about how the concept demo could be applied to improving efficiency and effectiveness of producing living guidance (including both processes and technologies) and synthesizing best practices and processes based on that feedback. These results will inform the Living Guidance Playbook (described in 1.11).

Produce a CDS/eCQMs Authoring Tooling Concept Demo

Using Agile methods, produce (or adapt from available system(s)) a concept demo of authoring tooling to create living CDS/eCQMs in a computable, standards-based, automated fashion, leveraging current and planned CDS Connect Authoring Tool functionality. This CDS/eCQMs authoring tool concept demo includes functions beyond what CDS Connect will have available during Phase 1 (e.g., taking BPM+ artifacts and putting them into CQL and other CDS interventions). This concept demo illustrates enhanced interoperability by seamlessly receiving computable data from the guideline management tool concept demo (described in 1.6.4) and providing standards-based CDS interventions and eCQMs for implementation. This interoperability will be demonstrated in Phase 2 (task 2.12).

This task includes documenting stakeholder feedback about how the concept demo could be applied to improving efficiency and effectiveness of authoring tools for living CDS/eCQMs (including both processes and technologies) and synthesizing best practices and processes based on that feedback. These results will inform the Living Guidance Playbook (described in 1.11).

Produce an Open, Unified, Integrated Federal Marketplace Concept Demo

Using Agile methods, produce a concept demo for an open, integrated Federal marketplace with a “one-stop-shop” user experience. This marketplace concept demo disseminates in a limited-function Production fashion (analogous to the Production capabilities of the AHRQ Portal concept demo) the CDS/eCQMs for the four targets that are produced and implemented during Phase 1 (e.g., L3s gathered into the AHRQ Digital Knowledge Portal concept demo). This function builds on CDS Connect Repository (49) and any enhancement available during Phase 1. The concept demo more broadly illustrates the full reference architecture-enabled capabilities and other reference architecture-enabled requirements established for the AHRQ DKP.

The Federal Marketplace concept demo illustrates seamless interoperability with other marketplaces (e.g., from EHR vendors and private/commercial medical publishers), and with the AHRQ DKP. This marketplace concept demo addresses DRM and integration of L3/L4s into local systems (e.g., app store-like functionality). It also illustrates seamless interoperability with the CDS Authoring Tool concept demo in the previous cycle step and CDS/eCQM integration into clinical systems in the following cycle step.

The infrastructure that provides DKPs and marketplaces must answer these questions:

• What are the contents?
• How can these contents be accessed?
• What do the contents do (e.g., what parts/activities in the knowledge ecosystem cycle to they support)?
• Can the contents be trusted?
• How can the contents be used?

To demonstrate interoperability with other marketplaces, this task includes conducting a private/commercial marketplace environmental scan and outreach (supported by the Roadmap Execution Steering Committee and Coordinating Center) for collaboration on the concept demo. This work builds on private/commercial marketplace stakeholder engagement from ACTS Roadmap Development and COVID Collaborative and includes considering the implications of blockchains (94)  (95) for marketplace development.

For Federal / HHS Agencies interested in participating, identify high-value content relevant to clinical targets/use cases and assemble it into the free/Federal marketplace concept demo, which includes concept demos of new and/or enhanced HHS Agency DKP infrastructure for making this content more computable and usable in ways that will better meet stakeholder needs. Do likewise for PRIVATE DKPs/marketplaces (e.g., commercial publishers and EHR vendors who are interested in participating in concept demos of interoperable marketplaces and who have high-value content relevant to clinical targets/use cases and clients interested in that content and in participating in ACTS the Phase 1 marketplace concept demo). These stakeholders would participate for the value proposition of increasing the reach and value of their offerings and wouldn’t be funded by the Federal Roadmap investment.

This task coordinates with and leverages related Federal initiatives identified in Appendix A, Interplay With a Sampling of Other Strategic Plans, Priorities & Initiatives, such as PCCDS-LN’s Knowledge Network Proposal  (96), the Logica Roadmap  (97), and other relevant efforts as outlined in this Roadmap and identified by the Roadmap Execution Steering Committee. It builds on the CDS Connect Life Cycle  (98) (see Figure 22. CDS Connect Life Cycle). It also builds on and leverages earlier proposals for producing an open Federal marketplace and knowledge ecosystem (see Figure 23. Earlier Logica Draft Proposal ($25 million / 5 Years) for the Marketplace & Pertinent Parts of the Reference Architecture and Figure 24. Earlier Proposal from the PCCDS-LN (Middleton, Richardson, Blumenfeld))

Figure 22. CDS Connect Life Cycle (98)

This task includes documenting stakeholder feedback about how the reference architecture and this concept demo could be applied to address needs in a Production system unified Federal/public marketplace and spur advances in private-sector marketplaces that are fully interoperable with each other and with public marketplaces. Synthesize best practices and processes based on that feedback for use in Phase 2 piloting.

Figure 23. Earlier Logica Draft Proposal ($25 million / 5 Years) for the Marketplace & Pertinent Parts of the Reference

Figure 24. Earlier Proposal from the PCCDS-LN (Middleton, Richardson, Blumenfeld) (96)

Produce a Concept Demo Registry of Care Process and Results Data

Using Agile methods, produce (or adapt from available system(s)) a concept demo registry of care process and results data from CDO implementation sites (Table 5. ACTS Phase 1 Enhance Guidance Implementation & Assessment Tasks, Timeline ). This registry concept demo will illustrate seamless interoperability/data uploading of computable data from CDO information systems and feeding computable data into literature surveillance concept demo tool, including the complex issues associated with using population data from registries to support evidence generation.

This task includes documenting stakeholder feedback about how the concept demo registry could be applied to QI efforts in participating CDOs, performance reporting, new evidence creation, and healthcare performance assessment functions of AHRQ’s CFACT (e.g., ways the computable data about care processes and results can enhance CFACTs resources). Synthesize best practices and processes based on that feedback and use these to inform the Guidance Implementation Playbook (described in 1.13).

Phase 1 Enhance/Develop Living, Computable Guidance

These tasks are to accelerate—in the near term—availability of current, evidence-based guidance and tools to support population health and care decisions and actions for high stakes conditions and to do this in a manner that enhances care in real-world settings (Table 5. ACTS Phase 1 Enhance Guidance Implementation & Assessment Tasks, Timeline & Costs and provides a foundation for efforts to make the knowledge ecosystem cycle more computable, efficient, and effective (Table 3. ACTS Phase 1 Enhance/Leverage Infrastructure Tasks, Timeline & Costs ). The tasks create living computable guidelines based on living systematic reviews and feed living CDS interventions and eCQMs focused on the specific aspects of the four Phase 1 targets prioritized by the Roadmap Execution Steering Committee (described in 1.3.1). The Steering Committee likewise determines the criteria (described in 1.3.2) to select resources from AHRQ and other sources (e.g., government agencies, academic centers, specialty societies, commercial vendors, etc.).

Suitable owners will be identified to take responsibility for maintaining the living guidance and related materials for the specific targets related to preventive care (e.g., USPSTF (99), hypertension control (e.g., AMA’s Improving Outcomes division (100)), pain and opioids (e.g., CDC (101)), and COVID (e.g., CDC (102), NIH (103)) The Steering Committee and the Coordinating Center will cultivate synergies with other similar living guidance efforts on other targets in both the public and private sectors to amplify returns (e.g., best practices generated people supported) from work on the Roadmap-funded targets.

Initial evidence and guidance processing will leverage available tools and formats since infrastructure to make this work fully computable and standards-based doesn’t yet exist. Work on the infrastructure concept demos in Phase 1 (Table 3. ACTS Phase 1 Enhance/Leverage Infrastructure Tasks, Timeline & Costs ) and pilots in Phase 2 (Table 9. ACTS Phase 2 Enhance/Leverage Infrastructure Tasks, Timeline & Costs) will provide a path to the desired computability. To start, use the Roadmap Execution Steering Committee, Coordinating Center, and AHRQ DKP/knowledge ecosystem concept demo work to determine how to best to manage this information initially.

An important resource for all the subtasks below is the living repository of Approaches to Improving the Knowledge Ecosystem Cycle (80) being developed by the ACTS COVID Collaborative  (55) (see E.3.1, ACTS COVID‑19 Evidence to Guidance to Action Collaborative Pilot). Cultivate synergies between tasks in this group and internal AHRQ efforts (Appendix F, AHRQ Offerings & the Knowledge Ecosystem Cycle) and external efforts (Appendix A, Interplay With a Sampling of Other Strategic Plans, Priorities & Initiatives)

• Produce/Enhance a Living Directory of High-Impact Studies

This task will produce (or adapt) and leverage “living repository” of current/emerging high-impact studies on each of the four Phase 1 clinical targets, leveraging a literature surveillance system like the Living OVerview of Evidence (L·OVE) / Epistemonikos platform (91). In this Phase, the directory is initially a list of pointers to these resources where they currently reside and/or an actual aggregation of these resources. AHRQ Portal Concept Demo (described in 1.5.4) is used when it becomes available to store and present these studies.  

• Produce/Enhance Living Systematic Reviews

Produce and/or leverage living systematic reviews on each of the four Phase 1 targets. Leverage repositories of living reviews, e.g., as pointed to from sources such as the ACTS COVID Collaborative’s Approaches to Improving the Knowledge Ecosystem Cycle (80). Also use output from the high-impact studies directory produced in 1.7. If possible, have AHRQ EPCs create, maintain, and leverage the living systematic reviews. Leverage current and emerging SRDR+ functionality.

• Produce/Enhance Living Clinical Guidelines

Produce and/or adapt a living clinical guideline on each of the four Phase 1 targets. Leverage the living systematic reviews produced in task 1.8.

• Produce/Adapt at Least One High-Impact, Living CDS Intervention & Corresponding eCQM

Produce and/or adapt at least one high-impact, living CDS intervention and corresponding eCQM to address each of the four Phase 1 clinical targets. Leverage the Roadmap Execution Steering Committee, especially members from clinical sites that will implement the intervention/measure (see Phase 1 Enhance Guidance Implementation & Assessment), to determine which interventions would be most valuable given implementation site expertise, evidence regarding intervention value, and broader stakeholder needs for specific high-value interventions. Leverage the living guidance produced in task 1.9. See Table B-1. Interventions Underlying the Future Vision for a sampling of CDS intervention type candidates for this work.

Cultivate opportunities to put the evidence-based guidance into practice through individualized, integrated care plans, especially for patients with multiple chronic conditions. This population is an important focus for AHRQ efforts and a major opportunity to reduce morbidity and costs. Explore synergies with the AHRQ–NIDDK e-care plan project-related work (50) (104), including clinician- and patient-facing electronic care plan applications (105), and an HL7 FHIR IG (106).

Document stakeholder feedback about how the concept demo could be applied to improving efficiency and effectiveness of producing living CDS/eCQMs. Synthesize best practices and processes to accomplish this step based on stakeholder feedback. This task includes documenting stakeholder feedback about producing this content and synthesize best practices and processes based on that feedback. This information is used to inform the Guidance Development Playbook (described in 1.11).

• Produce Initial Living CDS/eCQMs

This subtask produces the initial version of the living CDS eCQMs in the early part of Phase 1, so that it can be implemented in the first half of Phase 1 in clinical sites (see 1.12). It provides (or adapts) living, high-impact CDS interventions and associated eCQMs (building on proven tools, likely developed at one or more of the implementation sites).

• Produce a Living Guidance Development Playbook

Produce a playbook for creating and maintaining living computable evidence, guidance, and CDS/eCQMs that will support efficient/effective creation—across public and private developers—of high-value computable guidance and related tools. The playbook will provide practical guidance and tools for those in involved in creating and processing computable studies, systematic reviews, guidelines, and CDS interventions and eCQMs. It leverages existing resources (e.g., as outlined in Figure 21. FHIR-Based Knowledge Representation Specifications  (84))

This playbook will be developed and applied in coordination with educational, change management, workforce development offerings (e.g., courses, learning communities, IGs, training, and mentoring programs, etc.) to ensure that these knowledge ecosystem outputs can be effectively produced and consumed by the healthcare and public health sectors. These complementary offerings are developed with investments outside core Roadmap funding from other stakeholder organizations, and the Roadmap Execution Steering Committee and Coordinating Center will cultivate these collaborations.

Phase 1 Enhance Guidance Implementation & Assessment

These tasks are to accelerate—in the near term—application of current, evidence-based guidance and tools to support population health and care decisions and actions for high stakes conditions. Although Roadmap funding directly supports guidance implementation and assessments in a relatively few organizations, the Roadmap Execution Steering Committee and Coordinating Center will cultivate synergies with a much larger group of public health, care delivery, and other implementation and assessment initiatives. These interdependent efforts will focus on the four Phase 1 targets and others beyond in both the public and private sectors. These sectors currently invest billions of dollars in these efforts, but since there is little or no opportunity for large-scale, robust shared learning and cross-fertilization, progress toward shared goals is painfully slow. The governance, coordination, and collaboration mechanisms proposed in this Roadmap provide a path to faster learning and progress.

• Integrate CDS/eCQM & AHRQ Digital Knowledge Portal Concept Demos into Information Systems & Clinical Workflows at Five CDOs

Demonstrate CDS/eCQM (and AHRQ DKP) integration and use in systems and care workflow. At least five different CDOs implement at least one intervention/measure for each of the four Phase 1 targets, affecting care for at least 200,000 patients. Select implementation sites based on proven success implementing CDS interventions for the clinical target. Use an Agile approach and keep interventions updated as the living CDS/eCQMs evolve. Leverage and build on work of AHRQ’s EvidenceNOW initiative (66) (especially for the hypertension target) and the work of the CMS Transforming Clinical Practice Initiative (107).

Document stakeholder feedback on refinements and extensions to the CDS/eCQMs that were implemented in ways that would improve care processes and outcomes for the clinical target more efficiently and effectively. Synthesize best practices and processes to accomplish this step based on stakeholder feedback. Use this information to inform both the Guidance Development Playbook (described in 1.11) and the Guidance Implementation Playbook (described in 1.13).

• Select CDO Sites, Providers & Patients for Guidance Implementation & Assessment

Based on criteria from the Roadmap Execution Steering Committee, select CDO sites, providers, and patients for guidance implementation and assessment. CDO selection criteria includes proven success implementing CDS interventions for the clinical target, and strong interest in building on this work in ways that enhance LHS performance in their organization and help scale learning to broader knowledge ecosystem enhancements (including providing input to optimize AHRQ DKP function and value).

• Integrate Interventions into Systems, Reengineer Workflows & Train Staff

Prepare CDOs for successful guidance implementation and assessment by integrating the interventions into local systems, reengineering workflows with affected stakeholders to optimize intervention use and value, and training staff in the new tools and workflows.

• Deploy Tools & Workflow into Care Delivery Activities

After appropriate user engagement and preparation and intervention testing and validation, release the tools for clinical use. Leverage processes and systems developed prior to deployment to monitor progress and respond to issues that arise.

• Produce an Implementation Playbook

Using Agile methods, produce a playbook for successful CDS/eCQMs implementation to improve care processes and outcomes, vetted by activities implementing the CDS interventions in real-world settings (see 1.12) for the Phase 1 clinical targets/use cases. Apply the playbook to CDS/eCQM implementation tasks in CDOs and use those implementation experiences to enhance the evolving playbook. The playbook will have an overarching focus on CDS intervention and eCQM implementation, with additional sections covering issues specific to CDS/eCQMs for each clinical target.

This playbook is informed by and builds on previous guidebooks on improving care process and outcomes via CDS and other approaches  (108) (75) (107) (109) (110) (111) (112), as well as the approaches for implementing CDS-enabled care process and outcome improvements for COVID from the ACTS COVID Collaborative (113). This evolving playbook supports—and is supported by—activities implementing the CDS interventions in CDOs per the Resource/Guidance Implementation tasks. It has overarching focus on QI (including via CDS intervention implementations) to address a broad range of clinical targets, with sections covering issues specific to QI and CDS interventions for each Roadmap-specific target. The playbook also leverages AHRQ-funded Digital Healthcare and other research and guidance (e.g., AHRQs COVID Health Services Research (HSR) awards (64), CQuIPS resources (47), see Table B-7. Current AHRQ (& Other) Resources for LHSs). Playbook development informs—and is informed by—requirements development the AHRQ DKP concept demo (1.5.5).

As with the Guidance Development Playbook, the Guidance Implementation Playbook will be developed and applied in coordination with educational, change management, workforce development offerings (e.g., courses, learning communities, IGs, training, and mentoring programs, etc.) to support effective implementations. These offerings are developed with investments outside core Roadmap funding from other stakeholder organizations, and the Roadmap Execution Steering Committee and Coordinating Center will cultivate these collaborations.

Phase 1 Evaluate/Plan Roadmap Execution

This Roadmap is based on an Agile, continuous learning approach to transformation, so incremental evaluation and planning and research are central to the approach. This task group has two components: 1) iterative evaluation and planning; 2) and research. The latter is based on leveraging synergies with the AHRQ Digital Health Research Strategic Plan initiative.

• Gather & Analyze Feedback on Concept Demos

These tasks focus on evaluating all the other Phase 1 activities and outcomes and applying these results to developing requirement and project plans to build on each of those efforts in larger-scale, more Production-grade activities in Phase 2.

• Develop a Plan to Evaluate Phase 1

Develop a plan to gather and analyze results and feedback from all the Phase 1 activities, and vet this information (e.g., explore its implications for Phase 2 activities) with pertinent stakeholders via the Roadmap Execution Steering Committee and Coordinating Center.

• Implement Evaluation Plan & Develop Requirements & Project Plan for Phase 2 Pilots

Implement the evaluation plan with oversight from the Roadmap Execution Steering Committee and develop requirements and project plans for the Phase 2 pilots.

• Evaluate Phase 1 Concept Demo Processes & Outcomes

Evaluate the concept demo processes and outcomes with an emphasis on the implications for the Phase 2 pilots.

• Provide Requirements & a Phase 2 Project Plan to Convert the AHRQ DKP Concept Demo into a Fully Functional AHRQ DKP

Determine requirements and a Phase 2 project plan to convert the AHRQ DKP Concept Demo into a fully functional AHRQ DKP that makes all current and planned AHRQ offerings more FAIR, computable, and useful.

• Provide Requirements & a Phase 2 Project Plan to Build the Literature Surveillance Concept Demo into a Production-Grade Literature Surveillance Tool

This effort includes documenting which portions of this tool are provided or supported by the AHRQ DKP. For example, NIH might be the organization that actually provides the literature surveillance tool, but the AHRQ DKP might need to have functions that support AHRQ-funded researchers in using the tool to make results from AHRQ-funded studies computable via the tool and available via the AHRQ DKP. This work also includes defining synergies between this tool and enhancements emerging to AHRQ tools such as SRDR+, EPC report enhancements, etc. (see Appendix F, AHRQ Offerings & the Knowledge Ecosystem Cycle).

• Provide Requirements & a Phase 2 Project Plan to Build the Concept Demo Evidence Synthesis Tool into a Production-Grade Living Systematic Review Management Tool

Includes documenting which portions of this evidence synthesis tool are provided or supported by the AHRQ DKP (e.g., requirements for SRDR 3.0, requirement for funneling AHRQ-funded research results in a computable into this Production-grade living systematic review management tool, implications for supporting EPCs in doing systematic reviews, etc.).

• Provide Requirements & a Phase 2 Project Plan to Build the Living Computable Guidance Concept Demo into a Production-Grade Living Guidance Management Tool

Includes documenting which portions of this guidance tool are provided or supported by the AHRQ DKP (e.g., to drive production of USPSTF recommendations).

• Provide Requirements & a Phase 2 Project Plan to Pilot the Authoring Tool & Scale CDS/eCQM Development

Provide requirements and Phase 2 project plan to build the concept demo CDS authoring tool into a Production-grade living CDS/eCQM authoring tool for producing CDS interventions (see Table B-1. Interventions Underlying the Future Vision) and eCQMs to better support care (for patients and populations) and QI (e.g., care process/outcome enhancement and quality reporting) for the four Phase 1 clinical targets. These requirements will be the focus of Phase 2 pilots that interoperate seamlessly with other ecosystem cycle components (e.g., receiving computable data from the guidance management concept demo tool). This work also includes documenting which portions of this CDS/eCQM tool are provided or supported by the AHRQ DKP (e.g., related to the evolutionary trajectory of the CDS Connect authoring tool).

• Provide Requirements & a Phase 2 Project Plan to Build the Public Marketplace Concept Demo into a Production-Grade Open/Free Marketplace

Includes requirements to make the concept demo reference architecture fully operational and to make the free/open marketplace a valuable source for free Federal resources (e.g., from HHS, VA, DOD, and others) in a manner seamlessly interoperable with other public and private marketplaces. Consider capabilities of current and planned repositories and marketplaces in developing these requirements (e.g., CDS Connect for CDS artifacts, ECRI Guidelines Trust, GIN library and registry, and Logica/Graphite Marketplace, SRDR+ for evidence/systematic reviews).

• Provide Requirements & a Phase 2 Project Plan to Scale the CDS/eCQM Implementation

Provide a Phase 2 project plan to scale the CDS/eCQM implementation and care/QI improvement to additional sites and clinical targets as outlined in the Phase 2 tasks. The expanded implementation includes sites with a broad array of different characteristics such population demographics, locations served, health IT sophistication, etc.). Ensure that initial results from highly selected implementation sites are broadly scaled to all populations and settings, especially those currently underserved.

• Provide Requirements & a Phase 2 Project Plan to Build the Care Process/Results Registry Concept Demo into Production-Grade Tool

Provide requirements and Phase 2 project plan for building the concept demo into Production-grade tooling for augmenting how care results are currently measured and leveraged. Document which portions of this tool are provided or supported by the AHRQ DKP.

• Produce a Phase 1 Report

Produce a Phase 1 report that outlines Phase 1 activities, results, and next steps.

• Develop & Execute Research Plan

The Roadmap’s research activities are encompassed by those that will be undertaken by AHRQ’s DHRSP. This Roadmap task group is to cultivate synergies between those activities, and valuable research opportunities raised by the Roadmap and its execution. In addition, cultivate synergies between related AHRQ research investments and Roadmap execution (e.g., the R18 research funding for Disseminating and Implementing Patient-Centered Outcomes Research [PCOR] Evidence into Practice through Interoperable CDS  (114)) to scale CDS enabled by patient-centered outcome research, “across different healthcare systems and technologies (e.g., different EHRs) and that disseminate lessons learned about how to achieve CDS scalability and interoperability.”

• Define Strategies to Incorporate AHRQ's DHRSP into Phase 1 & Apply Roadmap Results to the DHRSP

This planning task is to define specific strategies and opportunities to incorporate pertinent activities and investments pursued under AHRQ's DHRSP efforts into Phase 1 Roadmap efforts and apply results from these Roadmap efforts to amplify investments and support execution on the DHRSP.

The DHRSP strategic planning guide suggests these DHRSP strategic pillars, which are all highly interdependent with Roadmap goals and activities, most intensively #3:

1. 1. Healthcare Equity—Inequity mitigations improve outcomes and advance the access and equity objectives of AHRQ’s mission
   2. Optimized Care Delivery—Research provides insights, strategies, and tools to leverage telehealth appropriately, promote care planning, and manage care transitions more effectively
   3. Evidence-Based Practice—Evidence and guidance is quicker and easier to find and apply to improve care decisions, actions, and outcomes
   4. Engaged and Empowered Patients and Caregivers—Digitally enabled engagement strategies empower patients and caregivers to achieve patient-centered care and desired outcomes

The DHRSP strategic pillar/knowledge ecosystem enablers—data, advanced analytics, and knowledge (which includes decision support, clinical guidelines, and research results)—are also closely related to the Roadmap activities and goals.

• Ensure Bi-directional Support Between DHRSP & ACTS Phase 1 AHRQ DKP, Knowledge Ecosystem Enhancement Concept Demos & Resource Enhancement/Development Tasks

This task executes the plan outlined in the prior task, ensuring that the synergies both advance DHRSP goals and accelerate progress toward the AHRQ DKP, other knowledge ecosystem components, and LHS/Quintuple Aim goals.

[1] The AHRQ DKP concept demo explores how—within AHRQ’s scope—its DKP could create and process computable information around the ecosystem cycle. These other knowledge ecosystem concept demos are broader, covering needs for functions critical for each cycle step that might be beyond AHRQ’s current scope to provide (e.g., tools that cover all aspects of computable systematic reviews and guidance for all users).