Journal of Medical and Clinical Nursing Studies

Building Confidence Using Online Healthcare Simulations: A Critical Analysis Through User Experience (UX) Design

Anthony Basiel^1*, Mike Howarth², Joelle Salje³ and Louise Morrell⁴

¹Department of Science and Engineering, Solent University, Southampton, United Kingdom
²Middlesex University, Bournemouth, United Kingdom.
³Department of Social Sciences and Nursing, Solent University, Southampton, United Kingdom
⁴Department of Art and Music, Solent University, Southampton, United Kingdom

*Corresponding author
Anthony Basiel, Department of Science and Engineering, Solent University, Southampton, United Kingdom..

ABSTRACT
Healthcare learning simulations have grown and matured over the past 40 years on substantive and methodological grounds and looks to be increasing in the future. This small pilot study suggests a well-organised project can assess the usability of a simulation with a limited sample size, conducted over five weeks and with zero budget. This investigation asks, ‘What is the impact on the confidence level of healthcare workers (Nursing students) using learning simulations? Can computing science user experience testing techniques be used to generate evidence to validate the conclusions? A computing science user experience (UX) 5-part Sprint methodology was used to collect evidence from 30 undergraduate Nursing students who used an online 2D medical learning simulation. This cross-discipline project synthesised a variety of technologies and tools to collect data to inform the design of possible improvements to the UX design of the simulation. UX tests were performed to provide evidence on the Nursing students’ experience. Although there was not enough time to validate improvement on the Nurses’ knowledge and skills related to the simulation scenarios, the conclusion is that using the UX modified online medical learning simulation did have a positive impact on the Nurses’ confidence level. Further research is recommended to explore the use of Augmented Reality and 3D Virtual Reality rooms to promote the development of next generation simulation solutions.

Introduction
An instructional model with undergraduate Nursing students writing scripts for face-to-face role-playing scenarios was presented by co-author Joelle Salje, Head of Nursing Simulations at Solent University, UK at the Association for Simulated Practice in Healthcare Conference in 2022 [1]. Also at the conference was Anthony Basiel who presented a case study using online learning simulations to promote communication skills for nurses through a circle discussion debriefing model at Bournemouth University [2]. They agreed to carry out a cross-discipline collaboration between the Nursing School and the School of Computing Science. For this paper participants are referred to using capitalisation for their titles as Nursing or UX Researcher students and Facilitators as staff.

The project was conducted as part of an undergraduate simulated nursing placement. Simulated placements in undergraduate nursing programmes have been gaining traction since 2022 when the NMC released RN6(D) permitting for 600 hours of clinical placements to be replaced with simulated placement [3]. During this time, OMS has proven to be an effective option for delivering simulated placement to undergraduate students [4].  

Anthony Basiel’s students in the MSc Computing User Experience (UX) Design course carried out an assessment case study aimed to examine and evaluate a 2D online healthcare learning simulation from the Oxford Medical Simulation software company. See Figure 1 [5]. The purpose of this study was to explore learning technology design and the impact on the confidence of healthcare stakeholders. The SMART (specific, measurable, achievable, relevant, and time-bound) objectives of the student investigation asked what is the impact on the confidence level of healthcare workers (Nursing students) using learning simulations [6]? Can computing science user experience testing techniques be used to generate evidence to validate the conclusions?

Cross-disciplinary collaboration can often lead to the development of new perspectives and innovative solutions to unexpected problems. The result can be breakthroughs that may not have been possible within the confines of a single discipline or research ethos. According to ChatGPT 3.5 a new perspective can result in breakthroughs that may not have been possible within the confines of a single discipline [7].

The platform for the investigation was the Oxford Medical Simulation [5]. The 2D online version was used for this study, not the 3D virtual reality option. Microsoft Teams was used to host the Nurses and Computing students sharing their experience as a group using two different medical scenarios. The sessions were recorded, then auto-transcription text was generated into Microsoft Stream. The Nurses also posted comments and questions in the text chat. Microsoft Form surveys collected data for the critical UX review using numerical and open-ended feedback. Finally, interview data from online meetings with the Nurses were recorded and auto-transcribed. The analysis was conducted by UX Research students who created Microsoft channels in the project Teams account to store files, conduct cohort meetings and to share knowledge. There was also a debriefing session using a 360* augmented reality camera (Insta360) in a previous study which provided a unique immersive experience for the students participating in a circle discussion. Solent University ethical policies and procedures were followed in this case study [8]. 

Summary
A cross-disciplined collaboration between Solent and Bournemouth Universities provided research models detailed in the next chapter along with a variety of data collection tools which offer some unique perspectives. Through a mix of data collection and analysis tools a rich underpinning of the remote stakeholder’s environment was provided. SMART objectives of the project looked for evidence to see the impact through the online learning simulation. Finally, through a pilot case study approach and a UX 5-part Sprint evidence was generated to cross-check and confirm any actions reviewed in the following pages.

Methodologies
A focused case study methodology was used for the project [9]. Forty-four stakeholders were involved: 30 undergraduate Nursing students formed 3 cohorts, with UX Research students designing online learning simulation prototypes. The Facilitators were computing and nursing tutors. The project sample size (n = 30) was in three cohorts of 10 for data collection.

The Nurses demographics were established by a 5-part Sprint approach and online survey [10]. ‘Pain Points’ or areas of potential difficulty of the simulation design, were then used to develop prototype solutions and conduct usability tests to validate the solutions.

Mind Maps and Rich Pictures
The concept of mapping processes to promote critical thinking can allow the identification of relationships among data (system components) in the form of a diagram or map according to Tattersall [11]. Therefore, project researchers used this concept to note corrélations with directional arrows and lines to reveal relationships in a Figma Mind Map (Figure 2). Tattersall also recognises the value of the process to promote a common language amongst the stakeholders and generate a meta-view of the issues.

Rich Pictures
Some UX students refined the Figma Mind Map images in Figure 2 into Rich Pictures. This type of diagram is a Soft Systems Methodology (SSM) approach to refine complex problems and facilitate change among people who hold different views, goals and agendas [12,13]. Rich Pictures are tools to make sense of a system and its behaviour. Where and how visuals are placed in the picture helps to orient the viewer to what is happening, allowing the whole system to be seen at once. Conte says Rich Pictures offers a ‘map’ of a system which can enable patterns to be surfaced and identified [14]. This can lead to user experience (UX) design prototypes to be developed and tested.

Commercial Market Research
The UX students also took a commercial investigation approach [15]. Market research was conducted by each group to see what the online learning simulation competition was doing. Table 1 compares features of two simulations: 1) Shadow Health Simulations [16]. 2) NurseSim [17].

Scenario realism, assessments and feedback, interactivity, adaptability of the platform, how accessible it is to the users and how the software can be integrated with other learning platforms or healthcare systems were key features explored.

Figure 3 illustrates the results that might improve the nursing students learning experience in the simulation. They focus on a strategy for designing and development of screen layout.

Journey Map
A Journey Map is an infographic visualisation of the process that a nursing student goes through to accomplish a task, according to Salazar [18]. Journey Maps are useful in communicating the general narratives and themes uncovered by research to understand how a user works toward a goal over time.  The map of the Nurses experience using the online healthcare learning simulations is seen in Figure 4.

The following elements are highlighted:
Awareness: The Nurses (virtually) attended the two 2D online OMS events through Teams with a Facilitator (UX Tutor/Nurse Lecturer). They click on the virtual healthcare tools and people to work through the scenario. The stakeholders (Nurses and UX reviewers) were acquainted with the navigation features using an induction simulation video.

Pain Point Identification: A variety of data collection tools and techniques were used to identify areas of frustration and improvements to the Nurses learning experience. For example, observations were noted when the Nurses did the live simulated events and critically reviewed later. The Teams live text chat also provided insight into the nurses’ experiences. Online surveys collected quantitative data via Likert scale responses. Additionally, open-ended survey responses also gave additional qualitative data to validate the statistics. 

Training: The OMS virtual environment system provided prompts to aid the Nurses towards a solution to the patient’s condition. A blue glow aura outlined an active element such as a piece of medical equipment (e.g. the patient monitor) or support person (e.g. another Nurse) as the mouse rolled over the screen object. Joelle, the simulation Facilitator, had to give instructions to the Nurses on how to proceed in the limited time allocated by the software providing important feedback.

Patient Monitoring: The virtual patient communicates verbally with the scenario stakeholders. Nurses make decisions to resolve the patient’s condition based on these prompts and medical read outs from the devices in the medical ward.  

Diagnosis and Treatment: The OMS is originally designed to be used by an individual, not a group as we have done in our project. The various clicks and actions made during the session were recorded and matched against a standardised response. This ‘tick-box’ feedback also provided a score to identify any omissions or errors in medical procedures. A text summary of the sequence of events is available to the individual for reflection on lessons learnt. In this project the final diagnosis and treatment was a group effort.

Summary
A synthesis of a case study design and computing science user experience design testing tools and techniques is a key contribution to knowledge of this investigation. For example, Mind Maps and Rich Pictures informed the overview of the project, its components and stakeholders. This group exercise helped to form a common framework and language for the researchers. A snapshot of the features was used in various medical learning simulations through market research. A UX journey map of the specific area of improvement for the Nurses using the OMS platform was generated by each research group.  

Results
Regulations for data protection GDPR and Solent University research ethics policies and procedures in the project were followed to ensure protection of all stakeholders [19,20].
The three groups of UX researchers collected evidence to inform the designs of the prototype interventions using several methods. Project Data supports the prototype in the online presentation slides [21]. The student UX Researchers presented their research and findings to local usability design consultants in a ‘Dragon’s Den’ style review panel. Table 2 shows a screen grab of the prototypes.

Self-Debriefing Feedback (Image 1)
Self-debriefing is an enhancement to the current text-based feedback given to individuals using the current OMS software. The healthcare user is guided to start at the top tab with the ‘home’ icon as a feature of the interface design. Text and audio prompts instruct the learner to go through a series of prompts to debrief their lessons learnt. They can type the text or use a voice-to-text dictation feature. The survey results support increased accessibility to promote the use of the debriefing stage of a learning simulation event.

Help Prompts (Image 2)
The Nurses had different experiences and capabilities with medical technology as identified by the UX Researcher’s online surveys. For example, devices available to the Nurses in the hospital ward were not always easily identifiable. For this reason, UX cohort 2 included an optional ‘bubble help’ in clinical settings’ feature. For example, in a clinical setting the patient monitor provides an explanation alongside the patient monitor of the device functionality. 

Patient Vital Signs (Image 3)
The patient data was found to change too rapidly due to the critical symptoms to be read by Nurses looking around the room. Since this version of the simulation was 2D and not 3D virtual reality, the UX researchers were limited as to how they could display information where is could be seen.

The solution, based on survey and interview data, was to use the bottom right of Image 3 to click on a link where user could ‘call up’ the vital signs monitor in more detail. This is an example of user interface design principles and Laws of UX being applied to the prototype [20,21]. A live sample Figma Prototype can be seen online to illustrate a real-time experience [23]. 

The results from usability tests were favourable from the majority of the Nurses on ease of use, satisfaction, and the Nurses’ ability to complete the required tasks. Nurses reported that the changes would improve their learning experience leading to improved confidence. More specific data analysis of each group can be seen in the presentation slides of the UX student Researcher [21].

There is not enough data to show an impact on the student Nurses medical knowledge or physical skills improvement. However, based on the survey and interview data the Nurses did report that the use of the simulations was a factor in building their confidence in the scenarios conducted in the study. The results appear to confirm that, according to McGaghie, healthcare learning simulations have grown and matured over the past 40 years on substantive and methodological grounds and looks to be increasing in the future. Yet, simulation-based medical education is a complex service intervention that needs to be planned and practised with attention to organisational contexts [24]. This small pilot study might suggest a well-organised project can assess their usability with a limited sample size and was conducted over five weeks and with zero budget.

Summary
The key take-a-ways of the project are summarised here. The user experience Pain Points identified were:

• A patient vital signs readout modification - A new location and functionality.
• Help prompts to identify simulation components for new users.
• An end-of-simulation debriefing tool that promotes reflection on practice.

The three UX Research groups produced prototypes to address this issue using a variety of usability testing methodologies.

Discussion
Pre-study: 360* Augmented Reality (AR) Camera
A project was carried out prior to this study, to explore the use of a 360* augmented reality camera in the middle of a discussion circle called The Socratic discussion model. Basiel and Howarth originally proposed technical and pedagogic models to promote engagement and creativity in webinar designs [2]. The Socratic model can be specifically applied to open-ended problems and online learning simulation debriefing scenarios. This paper synthesises the learning designs and technologies to promote confidence levels with simulation participants and suggests that the challenge can been met.
Figure 5 shows a representation of the Socratic discussion circle to support a blended learning design. It is taken from a still from the Augmented Reality camera in the middle of the student table. Virtual online participants are seen on the large TV screen contributing to the debate to promote creative solutions. 

Next Generation - Now
A physical 3D virtual reality room of the healthcare learning simulation debriefing can be used instead of the Augmented Reality 360* discussion. Students can enter the physical room with the scenario projected on the walls, rather than the flat 2D simulation our Nurses experienced. This includes touch screen drag and drop features. For example, the healthcare participant in Figure 6 can put his hand on the screen and have a patient’s x-rays enlarged on the wall. Any files can be uploaded to the system to create a real-life role-play simulation that is built on actual case studies. In this situation GDPR and appropriate ethical protocols are followed to keep data anonymous. Additions and improvements from the synthesis of the technologies presented in this paper and pedagogic designs may inform new blended learning simulation solutions. 

Summary Discussion
The research questions addressed in this study were: 
‘Is there an impact on the confidence level of healthcare workers (Nursing students) using an online learning simulation?’  And, ‘Can computing science user experience testing techniques be used to generate evidence to validate the conclusions?’

The case study approach and a user experience 5-part Sprint methodology was used to collect data from stakeholders. Academic and market research provided secondary data to provide a theoretical foundation and commercial context to the use of the 2D OMS platform. A user journey map generated by each group followed the path of the Nurses as they did a specific task in the simulation. This methodology helped identify specific areas the learners found difficult. High fidelity interactive prototypes were designed, developed, and tested. Survey results indicated that the use of the online medical simulations has a positive influence on the confidence level of the Nursing student participants. Further study is encouraged to measure the impact on knowledge and skill levels of healthcare workers. The use of 360* augmented reality cameras for debriefing circles after learning simulations has been shown in another area to be useful for this research and development. Finally, learning simulations 2D screens or virtual reality headsets can inform 3D immersive rooms to open learning providing the opportunity for a new level of learning simulations. Research into the technology and pedagogy provided in this study sets a path to educating our next generation [25,26].

Acknowledgements
The authors wish to thank the Nursing and Computing Science students for their participation in this case study. Without their support and contributions this project would not have been a cross-disciplinary success.

References

1. ASPiH. 2022. https://aspih.org.uk/event/conference-2022/
2. Basiel A, Howarth M. The 70:20:10 Webinar Design’ - Basiel and Howarth. 2021. https://www.amazon.co.uk/Industry-Practices-Processes-Techniques-Education/dp/9811935165
3. NMC. Council approves continued use of recovery standards to increase flexible use of simulation. 2021. https://www.nmc.org.uk/news/news-and-updates/council-approves-continued-use-of-recovery-standards-to-increase-flexible-use-of-simulation/
4. Devereux E. How nurse education is embracing simulated practice learning. 2023. https://www.nursingtimes.net/news/education/how-nurse-education-is-embracing-simulated-practice-learning-16-06-2023
5. OMS (Oxford Medical Simulation). 2022. https://oxfordmedicalsimulation.com/ 
6. US Department of Health. Objectives and goals: Writing meaningful goals and SMART objectives. 2023. www.health.state.mn.us/communities/practice/resources/phqitoolbox/objectives.html
7. https://www.federalregister.gov/index/2023/health-and-human-services-department 
8. ChatGPT 3.5. 2022. https://chat.openai.com/c/c582683c-a887-44f8-ab25-050340fe2d30 prompt: Is a cross-disciplinary approach a trend in UK higher education research?
9. Solent University (SU) Ethics Policy and Procedures. 2022. https://www.solent.ac.uk/research-innovation-enterprise/research-at-solent/support/ethics-and-integrity
10. Hyett Nerida (PhD Candidate), Kenny Amanda, Dickson-Swift Amanda. Methodology or method? A critical review of qualitative case study reports. International Journal of Qualitative Studies on Health and Well-being. 2014. 9: 1.
11. Kaley Anna. UX in Design Sprints. 2023. https://www.nngroup.com/videos/design-sprints,  https://www.youtube.com/watch?v=yZigSuvVr28
12. Tattersall C, Watts A, Vernon Stephen. Mind mapping as a tool in qualitative research. Nursing Times. 2007. 103: 32-33.
13. Checkland P. Soft Systems Methodology. John Wiley & Sons Ltd, from: Rosenhead J (ed.), Rational Analysis for a Problematic World. 1989. 8: 273-289.
14. Checkland Peter, Scholes Jim. Soft systems methodology in action: a 30-year retrospective ([New ed.] ed.). Chichester, UK: Wiley. 1999.
15. Conte KP, Davidson S. Using a ‘rich picture’ to facilitate systems thinking in research coproduction. Health Res Policy Sys. 2020. 18: 14.
16. McQuarrie Edward. The Market Research Toolbox: A Concise Guide for Beginners0 Sage Publishers. 2016. https://books.google.co.uk/books?id=p5vCBwAAQBAJ&lpg =PP1&ots=E6AOoApF4I&dq=market%20research%20techniques%20&lr&pg=PP1#v=onepage&q=market%20research%20techniques&f=false
17. Shadow Health Simulations. 2023. https://evolve.elsevier.com/education/simulations/shadow-health
18. NurseSim. 2023. https://nursesim.com.au/
19. Salazar Kim. 7 Ways to Analyze a Customer-Journey Map. 2020. https://www.nngroup.com/articles/analyze-customer-journey-map/
20. GDPR. 2023. https://gdpr-info.eu/ 
21. Solent University. Research Integrity. Solent University Southampton. 2023. https://www.solent.ac.uk/research-innovation-enterprise/research-at-solent/support/ethics-and-integrity
22. Project Data. 2023. https://drive.google.com/drive/folders/1lc1TZcAi6p8xLVCm6QcoLhoFui 6KUOJg?usp=sharing
23. Laws of UX. Aesthetic-Usability Effect. 2023.  https://lawsofux.com/aesthetic-usability-effect/
24. Figma Sample Prototype. 2023. https://www.figma.com/proto/RbAIZY9M16sdle7LwY1tBf/Usability -%26-UX?page-id=78%3A1905&node-id=78-1911&viewport =803%2C554%2C0.48&scaling=scale-down&starting-point-node-id=78%3A1906
25. McGaghie WC, Issenberg SB, Petrusa ER, Scalese RJ. A critical review of simulation-based medical education research: 2003-2009. Medical Education. 2009. 44: 50-63.
26. Salazar Kim. 7 Ways to Analyze a Customer-Journey Map. 2020. https://www.nngroup.com/articles/analyze-customer-journey-map/
27. Solent University (SU) Ethics Policy and Procedures. 2022. ‘https://www.solent.ac.uk/research-innovation-enterprise/research-at-solent/support/ethics-and-integrity