Feedback Data Analytics Medical Education Simulation Training Central Venous Catheter Clinical Competence

External authors:

• Rene de la Fuente ( Pontificia Universidad Catolica de Chile )
• Victor Galvez-Yanjari ( Pontificia Universidad Catolica de Chile )
• Alejandro Delfino ( Pontificia Universidad Catolica de Chile )
• Ricardo Lira ( Pontificia Universidad Catolica de Chile )
• Claudia Hurtado ( Complejo Asistencial Dr Sotero Rio )

Abstract:

Background Process mining is an emerging discipline that allows for the analysis of procedural executions performed in a training context, providing objective information about adherence with a normative procedural model (similarity), the number of repetitions of steps (reworks), and performance metrics, which can be used as objective feedback for trainees to guide learning through a process-oriented feedback approach. The aim of this study was to assess whether interventions based on information derived from process mining analysis improve the attainment of procedural proficiency. Methods Twenty anaesthesia and emergency medicine residents participated in a training program on ultrasound-guided internal jugular central venous catheter placement that took place in a simulated environment. The participants were randomized into a process-oriented training group (n = 10), which received supplementary interventions during training according to the information obtained with process mining tools, and a control group (n = 10), for whom the simulation-based training program was unchanged. Video recordings of each student were obtained before and after the training. Two blinded observers evaluated each recording using a global rating scale (primary outcome) and checklist. Procedure execution time and process-oriented metrics (rework and similarity) were measured. The pre- and posttraining performance indicators were compared within groups and between groups. The interrater reliability of the global rating scale scores was calculated using the intraclass correlation coefficient. We used the Wilcoxon signed-rank test for intragroup comparisons and the Mann-Whitney test for intergroup comparisons. Statistical significance was set at P < .05, adjusted for multiple comparisons. Results There were no differences between groups in the pretraining measures. During post training, both groups showed improved performance in ultrasound-guided central venous catheter placement compared with their pretraining performance. The global scale results, checklist results, and execution times were not significantly different between the control and process-oriented groups. However, the process-oriented group showed a significant improvement in similarity to the expected performance and a greater reduction in rework than did the control group. Conclusion The process-oriented approach, along with the procedural training program, decreases rework and increases adherence to the reference process model of the ultrasound-guided central venous catheter in the execution of the procedure in a simulated environment. Moreover, classic procedural assessment indicators do not capture this effect.