ANSWER
What role does nursing play in the digital health transformation of the twenty-first century? The answer to this critical question may be determined by how well-prepared nursing is to participate in the design processes associated with this evolution.
Aims
This paper aims to introduce basic terminology and tools to encourage more nursing participation in user-centered design. This is part of a six-step design process that includes a new analytic framework that combines computer science disciplinary expertise with nursing methodology. Descriptive Interpretation.
Methods The analytic framework and recommended research process were created throughout two projects, both of which used a collaborative mixed-methods design. These digital health intervention studies drew primary methodological drivers from the software development life cycle and Interpretive Description.
ORDER WITH US AND GET FULL ASSIGNMENT HELP FOR THIS QUESTION AND ANY OTHER ASSIGNMENTS (PLAGIARISM FREE)
Results
An analytic framework was conceived as part of an interdisciplinary research process using aspects of software development practice, allowing nurses to integrate their disciplinary expertise in user-centered digital design. The framework enables nurses to parse collected data into a robust set of functional and non-functional requirements for software developers while conducting extensive interpretive analysis.
Conclusion
Nursing must play a more significant role in advancing technological innovation in healthcare. However, nursing voices in this area need to be improved by a lack of familiarity with design thinking and the associated practical experience. As a means of expanding our disciplinary design capacity, tools and processes are introduced to enhance an existing nursing methodology.
Innovation and improvement, methods, nurses, nursing influence, organization, and service delivery are all keywords.
Go to:
Background
Nursing has recognized the role of patients (Dabbs et al., 2009) and practitioners (Narasimhadevara et al., 2008) in designing and advancing technological health solutions for more than a decade. The transition to a digital health era has resulted in a significant paradigm shift in care delivery, partly due to the increasing complexity and costs associated with the provision of healthcare around the world (LeRouge et al., 2013). This evolution, which extends far beyond traditional healthcare systems, includes the rise of consumer-driven health and associated technologies such as personal wearable devices and mobile health applications. These networked appliances monitor, collect, and collate data from their surroundings and can communicate that data via an internet connection, resulting in a vast knowledge exchange network (Meinert et al., 2018). As data flows more easily between these sectors, this technology enables new connections between home and formal healthcare settings (Evans, 2016). Nursing is confronted with the challenges and opportunities associated with advancing digital health in this context.
As we explore the possibilities of this new era, it is essential to remember that not all health innovation is technological. Innovative solutions may use various approaches to pursue new products and processes that can change behavior and influence the overall culture of a healthcare environment (Chamberlain & Partridge, 2017; Groeneveld et al., 2018). The primary goal of this paper is to introduce key design elements to a larger nursing audience and demonstrate how these principles can be applied in developing digital health solutions. A brief overview of design terminology is provided as a foundation for introducing a framework to assist nurses engaged in collaborative digital design. This paper will demonstrate the value of the nursing perspective in developing and implementing digital solutions into healthcare systems in the hope that more nurses from diverse practice areas will have increased opportunities to participate in this type of work. The analytic framework in this paper was developed by combining critical aspects of computer science’s software development life-cycle (SDLC) with Interpretive Description (ID), a qualitative nursing methodology designed to address clinical needs and questions pragmatically. When applied to data collected during a user-centered design (UCD) process, the framework is proposed as a way for design teams to deliver outputs that support both the technological development of a digital solution and the identification of critical context or process elements for the successful adoption and use of the completed innovation.
Designing for Digital Health
Although this paper focuses on design in the context of digital health innovation, the design field has a broad scope that is used in some form by all disciplines. In healthcare, design thinking is frequently used with a user-centered focus (Dopp et al., 2019; LeRouge et al., 2013; Wolstenholme et al., 2017). While this emphasis is common among tech developers in other fields, UCD is especially well-aligned with pursuing patient and family-centered care in health (Witteman et al., 2015). The National Institute for Health Research in the United Kingdom funded a multi-year project on user-centered healthcare design (Wolstenholme et al., 2017). This work resulted in a set of guiding design principles that advocate for design to go beyond improvement in pursuit of innovation and to be done with and for people in a holistic approach that recognizes that most living occurs outside formal care encounters (Wolstenholme et al., 2017). Aspects of these principles have been echoed in the work of others who have emphasized the importance of collaboration with end-users in order to improve the uptake and sustained use of new technologies (Dopp et al., 2019; Gurses et al., 2009; LeRouge et al., 2013; Kildea et al., 2019; Witteman et al., 2015; Wray et al., 2019). (Dabbs et al., 2009; Narasimhadevara et al., 2008).
The concept of UCD has come to reside within a complex set of nomenclature over time as the design industry has evolved, with each term having distinct meanings and attributes. Participatory design is widely regarded as a solid overarching principle to situate more specific design approaches (Kildea et al., 2019; Sonney et al., 2019; Wolstenholme et al., 2016). Along with UCD, concepts such as co-design (Wolstenholme et al., 2016), human-centered design (Sonney et al., 2019), and the distinction between patient- and person-centered design (Kildea et al., 2019) have emerged. Co-design emphasizes the patient or user as a partner rather than the subject of the design (Wolstenholme et al., 2016). In contrast, human-centered design is based on empathy, which drives the partnership throughout the design process (Sonney et al., 2019). The shift from a patient- to a person-centered design was intended to honor consumers’ holistic needs both in and out of their patient roles (Kildea et al., 2019).
This increase in design-thinking parallels discussions in healthcare about patient-centered care, specifically whether or not that concept should be transitioned to patient-partnered care. In the same vein, Kildea et al. (2019) have proposed participatory stakeholder co-design as an even more inclusive approach to collaborative innovation. This term encompasses patients and practitioners as equal and active collaborators in the design process (Kildea et al., 2019). The pursuit of participatory stakeholder co-design has added value from a nursing standpoint, as the discipline continues to see technological innovation delivered into practice settings without consultation or opportunities for design collaboration.
This is a common nursing experience. Despite the research, policy, and application efforts led by UCD advocates in health, design thinking, and genuinely collaborative design processes are yet to be commonplace in the global healthcare landscape. Part of the difficulty in moving this discussion forward may be due to the complexities of the terminology and the interchangeable use of terms with distinct meanings. According to Witteman et al., the foundational term UCD will be used in this paper (2015). In this context, a user is defined as anyone who will interact with the digital health solution. Design is guided by the understanding that a ‘product is most likely to fulfill user needs when its development process is based on iterative cycles in which potential users are consulted early and frequently’ (Witteman et al., 2015, p. 3).
Increasing nursing participation at UCD
As efforts to promote the effectiveness, efficiency, and innovative potential inherent in UCD continue, nursing has a new opportunity to establish a more significant presence in this movement. Patients and families are already strong advocates for the patient and family-centered care, which is a critical foundation of nursing education and codes of practice in both the UK and North America (Canadian Nurses Association, 2017; Wolstenholme et al., 2016), and the promotion of UCD appears to be a natural extension of these values. The question now is how to support these efforts best. While nurses have a strong foundation on which to build their design skills, there is a need for additional knowledge of the UCD process and additional practical support to facilitate this work. The interdisciplinary analytic framework proposed in this paper is intended to serve as one practical solution to this need. It is presented here within a six-step UCD study process to familiarize readers with this design. Nursing and patient voice continue to be underrepresented in digital health design. This will only change once nurses can demonstrate the value of nursing knowledge and tools in successfully developing and adopting new digital solutions. Even nurses not preparing to lead this design work will benefit from learning more about how collaboration can improve digital health tools in healthcare systems.
Go to: Developing an Analytic Framework
UCD in digital health necessitates extensive collaboration among all potential users of a proposed product or solution and various academic disciplines, health institutes, and industries. This work was created as a result of such a diverse collaboration. To develop this analytic framework, research leaders from nursing and computer science combined the qualitative methodology and rigor of ID, which originated in nursing, with an analytic framework based on best practices in software development. The authors initially used the framework to support the development of a mobile application for pediatric patients transitioning to adult care with chronic health conditions. This team has recently used it in the first phase of a UCD process to create a new patient portal for use in an acute care pediatric setting. While it was designed to aid researchers in analyzing data gathered during design sessions to guide technology development, the team quickly realized the framework’s potential in supporting the evaluation of existing digital tools. When used in the early stages of design, the framework assists researchers in determining the desired functional and nonfunctional features of the technology – information required by the development team to create a successful intervention. However, because it is embedded in the overarching methodology of ID, it encourages a broader investigation of many other factors that may influence the successful adoption and long-term use of the new technology.
Identification Methodology
ID is a qualitative methodology explicitly developed to address needs in applied disciplines “for those whose mandate requires informed action” (Thorne, 2016: 15). Thorne and colleagues first described ID (Thorne et al., 1997), and Thorne herself developed it as a method of generating clinically relevant knowledge for health disciplines (Thorne, 2008; Thorne, 2016).
ID is a call for researchers to use the foundational roots of the profession as a starting point for an exploration that should ultimately provide new insights and understanding into phenomena of interest. It is firmly rooted in the disciplinary structure of nursing itself (Thorne, 2014). The methodology is not overly prescriptive, allowing researchers to study without a stated allegiance to any one existing theory (Thorne, 2014). Being free of a single prescriptive theoretical influence allows nursing researchers to pursue interdisciplinary collaborations and cooperative solutions to complex clinical problems. The collaboration that resulted in this work aimed to push the boundaries of nursing capacity in designing digital health solutions while maintaining a critical disciplinary context. This increased knowledge is critical for strengthening the nursing voice, research in design, and the rapid advancement of health technology development.
Although using a framework to support the analysis of qualitative data is not novel, it is essential to consider the impact of such a tool on the analytic outputs. Qualitative methodologies, such as ID, allow study participants’ experiences and expertise to guide the discovery of new insights and potential solutions. The information should not be forced into preconceived or existing knowledge structures. This proposed framework is only prescriptive in extracting specific digital solution design and construction data. The advantage of using this tool within ID is that the methodology’s analytic proclivities allow for the pursuit of other critical questions Thorne (2014) recommended, such as: what else might be happening here?; what might we be missing? How else might we be thinking about this phenomenon? These are beneficial queries for design work.
The identification process is based on comparing and considering data elements in relation to one another (Thorne, 2008). This framework can be used to investigate practical needs related to specific required design elements while also considering how more significant contextual drivers may influence the uptake, use, and effectiveness of a deployed digital solution. While this type of examination is necessary for a successful design process, it also honors ID’s primary goal of reintroducing “analysis back into the context of the practice field, with all of its inherent social, political, and ideological complexities” (Thorne, 2016, p. 50). This approach’s pragmatic nature is also in line with certain fundamental principles and practices in computer science related to software development.
Tools and SDLC
Software professionals use the SDLC to refer to the standard, sequential phases involved in creating an application or component. While there is no single, definitive template for how an SDLC should be structured, most SDLCs share a core conceptualization (Mishra & Dubey, 2013). Gathering requirements, analysis and design, implementation, testing, deployment, and maintenance must all be included in new software development. Depending on the SDLC model used, these steps may occur only once or many times, especially in iterative approaches such as UCD. (2012) (Balaji and Sundararajan Murugaiyan).
The development of software is becoming a much more common research intervention. These projects must integrate the SDLC’s requirements gathering and documentation mechanisms with the research process. There are few tools available to support this type of research. The requirements-gathering phase of any SDLC model is one of the most important because it defines the fundamental parameters for the application’s functionality. This urgency is heightened when digital solution development is combined with health research. Members of the development team will interpret the data provided by participants and, to some extent, impose their views and preferences into the design of the code once they begin to create the components of the digital solution. Misinterpretation risks can be reduced by incorporating software development idioms into a comprehensive analytic framework.
Despite its youth, software development is a vast field with numerous specialties, paradigms, and practices. This analytic framework incorporated elements that blend well with qualitative research. This included standard and practical techniques for documenting and communicating user requirements, preferences, and contexts, as well as a list of functional and non-functional requirements. The primary goal of requirements gathering is to collect data about functional solution requirements. This section will go over the solution’s capabilities, focusing on what eventual functionality it will support. However, researchers must also identify any critical nonfunctional requirements that govern how the software achieves its functionality. Considerations such as application responsiveness, security, and privacy may be included. Table 1 shows examples of functional and nonfunctional requirement types identified using the proposed framework.
Table 1 depicts the functional and non-functional requirements for software development.
Functional requirements examples (what technology must do)
Users must be able to create accounts.
Users must be able to record 0 or more symptoms per day, and user transactions must be reported.
Nonfunctional requirements examples (criteria for how the system must operate)
Colorblind users must be able to read screens.
Reports must be generated in no more than 90 seconds, and the system must be available and operational 99.995% of the time in a given year.
Launch in a new window
Because these requirements play such an essential role in developing digital solutions, they must be documented clearly and consistently. In addition to the existing methodological directives within ID, these critical aspects of software development practice form the foundation of the proposed framework.
User stories and personas are two additional tools nurses at UCD can use to support precise requirements gathering in digital health intervention research. When combined with the proposed framework, this data suite provides a solid foundation for the successful completion of software development within a research endeavor.
When software intervention is the primary focus of a planned study, user stories are a valuable data collection tool. User stories serve as a template for transcribing individual requirements, identifying desired function(s), the intended use of the function(s), and anticipated outcomes. They can be modified to allow for more semi-structured questioning, as is common in traditional qualitative interviews. A typical user story is structured as follows: ‘as a (role), I want (function), so that (outcome).’ As a patient (role), for example, I want an easy tracking tool (function) to reduce the time required to manage my condition (outcome). The likelihood of clear communication between participants, researchers, and software developers is increased by phrasing each requirement in this manner.
In addition to user stories, using personas for requirements gathering and documentation is a recommended tool in UCD. Personas are a technique for identifying and communicating each key stakeholder’s primary needs and concerns for a specific solution (Ferreira et al., 2017; LeRouge et al., 2013; Wray et al., 2019). Previously, using this suite of tools as an example, user stories aided in the development of three distinct personas: patients with chronic illnesses, family caregivers for those patients, and healthcare practitioners providing specialized care to this patient group. A persona stereotypically represents a specific user group (Ferreira et al., 2017). The persona includes a brief, fictional biography based on an amalgamation of the themes and requirements shared by that stakeholder type; it essentially serves as a vehicle for summarizing critical aspects of the data gathered during the UCD process. This is especially useful when a solution must serve multiple end users, such as patients, clinicians, and analysts (LeRouge et al., 2013). As shown in Figure 1, an effective user-story format pairs a description of the requirement, or feature, with an intended outcome tailored to each persona.
An external file containing a picture, illustration, or other data.
10.1177 1744987120913590-fig1.jpg is the name of the object.
Figure 1 shows a user-story template.
Visit: Putting it all together in a UCD project.
In order to support the advancement of UCD in digital health, a foundation has been laid out that details the unification of ID and elements of software development. When aspects of both ID qualitative research and software development are incorporated into the study, the application of this analytic framework is most effective. Following is an example of this process: situating the proposed framework.
Prerequisites and assumptions
The application of this analytic framework is based on identifying a problem that is thought to have a digital health solution. As a result, it is most relevant in studies that prioritize the development of technological intervention. Other research considerations (e.g., maximizing patient voice, expanding the body of knowledge, etc.) may still be included in the study objectives. However, they must be accomplished within the context of developing the most effective digital solution possible.
Go to: UCD Study Process in Six Steps
A complete study overview is provided here to articulate how the analytic framework is used as part of a UCD process. This exercise also demonstrates the intersections of computer and nursing science and guides those who require this methodological approach. The recommended research procedure is comprised of six steps:
Problem identification: To specify the parameters of the problem under study, the study team is guided by ID principles and those reflected in UCD work. This work should include carefully crafting a research question aligned with ID methodology. If possible, users (patients, practitioners, etc.) should be involved from the start.
Technology solution hypothesis/research question: the proposed analytic framework has been designed specifically to meet the needs of research teams looking to implement a digital solution to an identified health problem. These issues may be addressed in a formal clinical setting, but this approach can also be applied to developing other health-behavior interventions. Once a digital health solution-focused hypothesis or research question has been developed, additional questions about the introduction, uptake and sustained use of the new technology may be included.
Data collection and analysis: Once an initial technological intervention has been identified, the study team can begin data collection, guided by the ID parameters in this case. Following ID, researchers can address the proposed questions using various data types, including interviews, texts, focus groups, and observation. As previously stated, this work must also meet the needs of gathering software development requirements, including prompts that reveal the proposed solution’s functional and non-functional requirements. Because the intervention will be developed over time and will be disconnected from data collection and analysis (i.e., the technology developers will most likely not participate in all data collection), special effort must be made to codify stakeholder needs in a format that translates to instructions for how the system must operate. Second, context about user experiences and goals should be included in the requirements documentation so that developers can make the best design decisions. This is where personas, created from user-story data, can be most helpful in providing developers with a ‘living’ sketch of the various users.
To complete this data set, functional requirements (what the intervention should do) must be collected with the participation of all stakeholders. This could be patients, practitioners, or a combination of these groups and other possible end-users. In order to ensure maximum effectiveness and successful integration of the proposed innovation, regulators and institutional administrators, for example, may need a voice.
There is also a requirement for the technology development team to establish any nonfunctional requirements associated with developing and deploying the proposed digital solution. This includes privacy, security, data retention, application minimum performance levels, required system operational hours, and minimum availability, among others. These software or development requirements are likely best addressed by that disciplinary expertise. In an interdisciplinary team, however, all researchers can play a role in identifying the most appropriate sources for these requirements (regulators, legislation, ethics board, etc.). (regulators, legislation, ethics board, etc.). This is just one area demonstrating the benefits of the ongoing collaboration between nursing and computer science in UCD research, with a shared goal of delivering a comprehensive and evidence-based design. The application of user stories and personas is another. These tools are derived from the disciplinary knowledge of computer science; however, our research has demonstrated the value of nurse team members translating the collected user-story data into corresponding personas in this work. This has proven to be a pivotal opportunity to imbue the design process with relevant nursing knowledge and expertise and is something many more nurses, even those not on formal design teams could potentially engage in.
The data analysis in this process should begin while data collection is ongoing, a common feature of many qualitative methodologies, ID included. The research team can use the proposed analytic framework as the data is collected through user stories or other means commensurate with ID. The framework is intended to ensure that critical digital design requirements are captured within an exploration of a clinical problem to which a technological solution is envisioned. A sample layout of these analytic elements is included in Figure 2. While there is flexibility in how the framework can be laid out, certain key elements should always be included, as noted in the sample. For example, key features identified by users for inclusion in the digital solution should be listed along with their intended outcomes; this data is most easily collected in the previously detailed user-story format. Functional and nonfunctional requirements must also be pulled from the user data. The sample layout includes a reminder for coding staff about functional and non-functional requirements, as this can be new terminology for health researchers.
An external file containing a picture, illustration, or other data.
The object name is 10.1177 1744987120913590-fig2.jpg
Figure 2. \sRequirements and analysis output example.
In the additional data prompts, data related to other questions or from collection methods beyond the user stories can be explored. Researchers may create multiple columns to explore potential strengths and challenges to the planned introduction of the new technology. However, there are no limits to what can be included here. Ultimately, all research questions should be represented in the analytic framework in addition to the requirements of the actual technical development of the intervention.
Intervention design: once the data collection and analysis are complete, the outputs can be relayed to a development team. This may be members of the research group or a contracted service. The technology developers will create an initial representation of the intervention based on the data collected. This could take many forms based on various situational constraints. Storyboards, wireframes, or prototypes are all helpful at this stage. The design(s) should be tested with all stakeholders engaged in the UCD at a minimum and ideally with other participants involved in the data collection.
Intervention/solution building or development: guided by design, the software development team will create the intervention to meet the required specifications. The outputs delivered through the analytic framework do not prescribe using any particular technology or software development methodology for this step. Presuming that the completed solution satisfies the specification and the research team can verify that the software operates as intended, the development team should be free to select the most appropriate tools for the project.
Testing the effectiveness of the intervention: finally, the research team evaluates the digital intervention to determine whether it produces the intended results or outcomes identified in the study hypothesis of the primary research question. The activities in this step should be dictated by the overarching research methodology and protocols selected for the study. It is critical in this step to evaluate not just the technical aspects of the intervention but its integration into the existing processes or system. Users must continue to be consulted in this phase to monitor for unintended consequences or complications that can often occur with new technologies.
Go to: \sAdditional operational considerations in UCD research.
Typically, in an SDLC, the solution is deployed into operations once the application development phase has been completed. At that point, it would be considered ‘under maintenance’ or in active use, with continuing development to address any issues or defects discovered. However, interventions created primarily for a study may not follow the same pattern. Depending on the situation, or program of research, the solution could be decommissioned, reworked, and re-deployed for future research or deployed for general use. Suppose the solution moves beyond use by the research team or institution alone. In that case, it should be transitioned to an operational team to manage and maintain it until the point of decommissioning.
Go to: \sDiscussion
Even though nursing has been engaged in discussions related to UCD for more than a decade, design thinking is currently a rare feature of nursing education or practice. However, some of the work that has been done on collaborative design, especially within the healthcare setting, has demonstrated strong synergies between these approaches and nursing’s disciplinary commitment to patient and family-centered care (Sonney et al., 2019; Witteman et al., 2015; Wolstenholme et al., 2016). (Sonney et al., 2019; Witteman et al., 2015, Wolstenholme et al., 2016). While not all design or innovation is focused on delivering technological solutions, in an era of digital health transformation, nurses must be better equipped to engage when it is. The consequence of not being able to advocate for and actively contribute to technology development will be the continued appearance of digital ‘solutions’ in the practice setting that ostensibly cause more problems than they purport to solve. These digital problems are not only visited by nurses; they are also suffered by patients. As healthcare delivery becomes more techno-centric, there is an urgent need to ensure everyone benefits in this digital transformation. Nurses will be essential in monitoring the progression of the touted promises in digital health and as advocates for and active partners in UCD processes.
In order to become more engaged in the design, especially when the planned outcome is a new digital solution, nurses require a foundational understanding of UCD and practical tools that can enhance their disciplinary contribution to these processes. This paper’s UCD process and proposed analytic framework include key computer-science principles integrated into a nursing methodology. This contribution is meant to support a broader understanding of the elements of UCD, allowing nurses to identify opportunities for engagement more readily.
This UCD process, including user stories, personas, and the analytic framework, has proven to be a fundable methodological foundation for this type of research. The framework has proven successful in the foundational phases of two ongoing studies delivering clear outputs to a software development team while allowing health researchers to complete a full ID analysis. ID is meant to address practical clinical problems, and introducing new technology into health systems is challenging. The authors have succeeded in intervention development and integration by integrating ID and computer science in this process and tool. Ultimately, the framework was created to advance UCD’s understanding and capabilities of two currently under-represented groups in digital health development: patients and nurses.
Go to: \sConclusion
As healthcare has undergone rapid transformation in this century, a disconnect seems to have occurred between nursing informatics and nursing, as if the detailed technical aspects of digital health were best handled by a small subset of the profession. Given the unrelenting pace of technological advancement, it is imperative that nurses reconcile that nursing informatics is nursing and that every nurse has a role to play in our digital health future. If we cannot embrace this evolution as a profession, the technological transformation will continue without us. Digital health is an irreversible entity, and while there is still time to engage, the opportunity to exert significant disciplinary influence upon the tools and processes associated with this transformation will not be interminable. Nursing has much to offer in this critical moment. Ethical expertise, in-depth knowledge of the complexities that the social determinants of health visit upon all intended innovation, and, as the most significant global healthcare workforce, a far-reaching connection to patients are also key players in this evolution. A nursing innovation movement is gaining momentum worldwide, driven by this wide breadth of disciplinary knowledge. This paper has introduced tools to encourage more nurses to step boldly into UCD, employing their professional proclivity for creative problem-solving to help deliver the promise of a healthier digital future for us all.
QUESTION
Review the steps of the Systems Development Life Cycle (SDLC) as presented in the Resources.
Reflect on your own healthcare organization and consider any steps your healthcare organization goes through when purchasing and implementing a new health information technology system.
Consider what a nurse might contribute to decisions made at each stage of the SDLC when planning for new health information technology.
Post a description of what you believe to be the consequences of a healthcare organization not involving nurses in each stage of the SDLC when purchasing and implementing a new health information technology system. Provide specific examples of potential issues at each stage of the SDLC and explain how the inclusion of nurses may help address these issues. Then, explain whether you had any input in the selection and planning of new health information technology systems in your nursing practice or healthcare organization and explain potential impacts of being included or not in the decision-making process. Be specific and provide examples.
· McGonigle, D., & Mastrian, K. G. (2022). Nursing informatics and the foundation of knowledge (5th ed.). Jones & Bartlett Learning.
o Chapter 9, “Systems Development Life Cycle: Nursing Informatics and Organizational Decision Making” (pp. 191–204)
o Chapter 12, “Electronic Security” (pp. 251–265)
o Chapter 13, “Achieving Excellence by Managing Workflow and Initiating Quality Projects”
· Agency for Healthcare Research and Quality. (n.d.a). Health IT evaluation toolkit and evaluation measures quick reference guide Links to an external site.. Retrieved January 26, 2022, from https://digital.ahrq.gov/health-it-evaluation-toolkit
· Agency for Healthcare Research and Quality. (n.d.b). Workflow assessment for health IT toolkit Links to an external site.. Retrieved January 26, 2022, from https://digital.ahrq.gov/health-it-tools-and-resources/evaluation-resources/workflow-assessment-health-it-toolkit