Delivering safe and effective medication management technology now and for the future

Project description, background and aims

Description

The Sydney Children’s Hospitals Network plans to adopt new eHealth systems to significantly reduce patient risk and improve outcomes for patients. In August 2016 SCHN will commence implementation of two major electronic health record components:

1) an electronic medication management system (eMM) for all patients across the two major paediatric hospitals in New South Wales; and

2) an electronic medical record for oncology patients (eMR) across the same hospitals.

This project presents an innovative and comprehensive program to evaluate and learn from the implementation of these two major eHealth applications and their impact on clinical work, patient outcomes, including medication outcomes, across two hospitals, and costs in relation to benefits generated. Our multi-method design recognises the complexity of assessing the multiple facets of eHealth system impact on delivery models. Importantly, the project will test the translation of research findings to enhance the eHealth systems and then measure the results on performance.

The project is based upon an innovative, rigorous, two stage design. IT system safety and effectiveness data generated from a stepped wedge cluster randomised controlled trial (SWCRCT) at the first hospital (stage 1) will be used to modify the IT system design, re-design work processes and inform training programs through a collaborative process with partners. In stage 2 this enhanced eMM model will be implemented at hospital 2 and the success of these changes in improving system effectiveness will be measured using a second SWCRCT.

Exploiting the SWCRCT design within an action-research model is highly innovative and novel. This project will generate the first Australian cost-effectiveness data on eMM, providing policy-makers with new evidence critical to prioritising health care investments.

Considering the costs of health IT – NSW alone has allocated $2.6 billion to 11 current health IT projects – such information is critical.

Background

The complex medication decision process in children often requires information about age, weight, dosing ranges and off-label use of medicines. Systematic reviews show errors occur in 5-27% of all medication orders for children. Prescribing errors account for 3-37% of errors, dispensing 5-58%, administration 72-75% and documentation 17-21%, although accuracy of estimates is affected by the great variability in definitions and measurement methods used. There is much less research about the harms associated with medicines use. Research has found that 19.2% of paediatric in-patients experienced adverse drug events (ADEs); 12.3% were potential ADEs, 7.0% were actual ADEs, and 3.6% were preventable. Paediatric patients are at particular risk of certain errors such as tenfold dosing errors facilitated by errors in dose calculation, poor documentation of decimal points and confusion with the use of zeros, some errors have led to death.

eMM are expected to reduce medication errors and ADEs significantly. However, rigorous evidence demonstrating these effects is limited.  A systematic review identified eight studies of eMM effectiveness among paediatric patients. Meta-analysis showed a significant reduction in prescribing error risk (RR 0.08) but not in ADEs or mortality. There are no Australian studies of eMM use in a paediatric setting. Previous studies have often relied upon incident reports to measure error rates, which are generally unreliable. Only one study has used a control group. Internationally there is currently insufficient evidence to demonstrate clinical benefit from eMM in paediatric patients.

eHealth systems will have an increasing impact on the delivery of health care services, yet, despite their widespread effects and vast cost, they are rarely subject to rigorous research. This limited evidence-base significantly hinders improvements and innovation in the design, implementation and use of health IT systems.

Evidence of eHealth system effectiveness is crucial to facilitate policy-makers’ and health care organisations’ informed decisions about investments and prioritisation of clinical IT systems. Implementation of eMM within Australian hospitals is central to state eHealth strategies. Most eMM adopted in Australia are designed overseas for different health delivery models. In the absence of local studies which demonstrate eMM effectiveness within Australian health care organisations, claims of benefits made by vendor IT companies are rarely able to be assessed or successfully challenged. Consequently, securing design changes to IT systems to integrate with Australian care models is much more difficult. This situation typically results in the introduction of workarounds which are inefficient and increase risk.

Aims

1. Quantify the safety and effectiveness of an eMM to reduce medication errors (including adverse drug events-ADEs), and average length of stay among paediatric patients using a stepped-wedge cluster randomised controlled trial
2. Assess the extent to which feedback (from Aim 1) and subsequent modifications of eMM design can improve eMM effectiveness in reducing medication errors
3. Assess the effects of an eMR in paediatric oncology on workflow, efficiency and patient outcomes
4. Conduct a cost-effectiveness study of eMM use in two paediatric hospitals

Publications

Journal articles

2024

1. Badgery-Parker T, Li L, Fitzpatrick E, Mumford V, Raban MZ, Westbrook JI. Child age and risk of medication error: a multisite children's hospital study. The Journal of Pediatrics. 2024 Sept;272:1-8. 114087. doi: 10.1016/j.jpeds.2024.114087
2. Mumford V, Raban MZ, Li L, Fitzpatrick E, Woods A, Merchant A et al. Developing a process to measure actual harm from medication errors in paediatric inpatients: from design to implementation. British Journal of Clinical Pharmacology. 2024 Jul;90(7):1615-1626. Epub 2024 Mar 26. doi: 10.1111/bcp.16052
3. Li L, Badgery-Parker T, Merchant A, Fitzpatrick E, Raban MZ, Mumford V et al. Paediatric medication incident reporting: a multicentre comparison study of medication errors identified at audit, detected by staff and reported to an incident system. BMJ Quality and Safety. 2024 Apr 15;016711. Epub 2024 Apr 15. doi: 10.1136/bmjqs-2023-016711
4. Westbrook JI, Li L, Woods A, Badgery-Parker T, Mumford V, Merchant A et al. Risk factors associated with medication administration errors in children: a prospective direct observational study of paediatric inpatients. Drug Safety. 2024 Jun;47(6):545-556. Epub 2024 Mar 5. doi: 10.1007/s40264-024-01408-6

2023

1. Lichtner V, Prgomet M, Gates P, Franklin BD. Automatic dispensing cabinets and governance of controlled drugs: an exploratory study in an intensive care unit. European Journal of Hospital Pharmacy. 2023 Jan;30(1):17-23. Epub 2021 May 11. doi: 10.1136/ejhpharm-2020-002552
2. Westbrook J, Li L, Raban MZ, Mumford V, Badgery-Parker T, Gates P et al. Immediate and long-term effects of an electronic medication management system on paediatric prescribing errors: a stepped-wedge cluster randomised controlled trial with one-year follow-up. 2022 Sept 29. doi: 10.21203/rs.3.rs-2083575/v1

2022

1. Westbrook JI, Li L, Raban MZ, Mumford V, Badgery-Parker T, Gates P et al. Short- and long-term effects of an electronic medication management system on paediatric prescribing errors. npj Digital Medicine. 2022 Dec 13;5(1):1-9. 179. doi: 10.1038/s41746-022-00739-x

2021

1. Westbrook JI, Li L, Raban MZ, Woods A, Koyama AK, Baysari MT et al. Associations between double-checking and medication administration errors: a direct observational study of paediatric inpatients. BMJ Quality and Safety. 2021 Apr;30(4):320-330. 011473. Epub 2020 Aug 7. doi: 10.1136/bmjqs-2020-011473
2. Gates PJ, Hardie R-A, Raban MZ, Li L, Westbrook JI. How effective are electronic medication systems in reducing medication error rates and associated harm among hospital inpatients? A systematic review and meta-analysis. Journal of the American Medical Informatics Association. 2021 Jan 15;28(1):167-176. Epub 2020 Nov 8. doi: 10.1093/jamia/ocaa230

2020

1. Baysari MT, Van Dort BA, Prgomet M, Zheng WY, Raban MZ, Dalla-Pozza L et al. The efficiency-thoroughness trade-off after implementation of electronic medication management: a qualitative study in paediatric oncology. International Journal for Quality in Health Care. 2020 Nov 9;32(8):511-516. Epub 2020 Oct 16. doi: 10.1093/intqhc/mzaa086
2. Westbrook JI, Sunderland NS, Woods A, Raban MZ, Gates P, Li L. Changes in medication administration error rates associated with the introduction of electronic medication systems in hospitals: a multisite controlled before and after study. BMJ Health and Care Informatics. 2020 Aug 1;27(3):1-9. e100170. doi: 10.1136/bmjhci-2020-100170

2019

1. Lichtner V, Baysari M, Gates P, Dalla-Pozza L, Westbrook JI. Medication safety incidents in paediatric oncology after electronic medication management system implementation. European Journal of Cancer Care. 2019 Nov 1;28(6):1-9. e13152. Epub 2019 Aug 22. doi: 10.1111/ecc.13152
2. Gates PJ, Baysari MT, Gazarian M, Raban MZ, Meyerson S, Westbrook JI. Prevalence of medication errors among paediatric inpatients: systematic review and meta-analysis. Drug Safety. 2019 Nov;42(11):1329-1342. Epub 2019 Jul 9. doi: 10.1007/s40264-019-00850-1
3. Baysari MT, Hardie RA, Barclay P, Westbrook JI. Effects of an electronic medication management system on pharmacists’ work in a paediatric hospital. Journal of Pharmacy Practice and Research. 2019 Aug;49(4):317-323. doi: 10.1002/jppr.1507
4. Gates PJ, Baysari MT, Mumford V, Raban MZ, Westbrook JI. Standardising the classification of harm associated with medication errors: the Harm Associated with Medication Error Classification (HAMEC). Drug Safety. 2019 Aug 1;42(8):931-939. Epub 2019 Apr 23. doi: 10.1007/s40264-019-00823-4
5. Gates PJ, Meyerson SA, Baysari MT, Westbrook JI. The prevalence of dose errors among paediatric patients in hospital wards with and without health information technology: a systematic review and meta-analysis. Drug Safety. 2019 Jan 21;42(1):13-25. Epub 2018 Aug 16. doi: 10.1007/s40264-018-0715-6

2018

1. Baysari MT, Hardie RA, Lake R, Richardson L, McCullagh C, Gardo A et al. Longitudinal study of user experiences of a CPOE system in a pediatric hospital. International Journal of Medical Informatics. 2018 Jan 1;109:5-14. doi: 10.1016/j.ijmedinf.2017.10.018
2. Mumford V, Baysari M, Kalinin D, Raban MZ, McCullagh C, Karnon J et al. Measuring the financial and productivity burden of paediatric hospitalisation on the wider family network. Journal of Paediatrics and Child Health. 2018 Sept;54(9):987-996. Epub 2018. doi: 10.1111/jpc.13923

2016

1. Westbrook JI, Li L, Raban MZ, Baysari MT, Mumford V, Prgomet M et al. Stepped-wedge cluster randomised controlled trial to assess the effectiveness of an electronic medication management system to reduce medication errors, adverse drug events and average length of stay at two paediatric hospitals: A study protocol. BMJ Open. 2016 Oct 21;6(10):1-11. e011811. doi: 10.1136/bmjopen-2016-011811

Health Innovation Series

2024

1. Merchant A, Fitzpatrick E, Westbrook JI, Raban MZ. The curious case of the 100-fold overdose. Australian Institute of Health Innovation, Macquarie University, 2024. 2 p. (Health Innovation Series; 21). doi: 10.25949/CTFN-KP70
2. Fitzpatrick E, Merchant A, Westbrook JI, Raban MZ. The dangers of copying a previous order. Don’t be a copycat. Australian Institute of Health Innovation, Macquarie University, 2024. 2 p. (Health Innovation Series; 20). doi: 10.25949/YNMY-SN59
3. Fitzpatrick E, Merchant A, Raban MZ, Westbrook JI. Does your drop-down menu present safe options? Australian Institute of Health Innovation, Macquarie University, 2024. 2 p. (Health Innovation Series; 19). doi: 10.25949/P1W7-P418
4. Raban MZ, Merchant A, Fitzpatrick E, Woods A, Westbrook JI. What time is the next dose due? Avoid errors by updating the schedule! Australian Institute of Health Innovation, Macquarie University, 2024. 2 p. (Health Innovation Series; 18). doi: 10.25949/FC6Q-K566

2023

1. Fitzpatrick E, Merchant A, Raban MZ, Westbrook JI. Is your administration documentation accurate? Check fields that auto-populate! Australian Institute of Health Innovation, Macquarie University, 2023. 2 p. (Health Innovation Series; 17). doi: 10.25949/E50T-VD39
2. Merchant A, Fitzpatrick E, Raban MZ, Westbrook JI. A mix of prescribing systems may be a recipe for disaster. Australian Institute of Health Innovation, Macquarie University, 2023. 2 p. (Health Innovation Series; 16). doi: 10.25949/WN2B-BX78
3. Merchant A, Fitzpatrick E, Westbrook JI, Raban MZ. Default first dose times can cause deadly double doses. Australian Institute of Health Innovation, Macquarie University, 2023. 1 p. (Health Innovation Series; 15). doi: 10.25949/H47E-TH56
4. Fitzpatrick E, Merchant A, Raban MZ, Westbrook JI. How free-text fields can lead to medication errors. Australian Institute of Health Innovation, Macquarie University, 2023. 2 p. (Health Innovation Series; 14). doi: 10.25949/BEDT-WG68
5. Fitzpatrick E, Merchant A, Westbrook JI, Raban MZ. First in line: optimising order sentence display reduces selection errors. Australian Institute of Health Innovation, Macquarie University, 2023. 2 p. (Health Innovation Series; 13). doi: 10.25949/0EFY-SV08
6. Fitzpatrick E, Merchant A, Westbrook JI, Raban MZ. Making order sentences work for you: search tips for prescribers. 2023. 2 p. (Health Innovation Series; 12). doi: 10.25949/2QEY-3D18
7. Woods A, Merchant A, Raban MZ, Fitzpatrick E, Gamboa S, Westbrook JI. Keeping staff safe when handling hazardous medication: it’s not only cytotoxics that are risky. Australian Institute of Health Innovation, Macquarie University, 2023. 2 p. (Health Innovation Series; 11). doi: 10.25949/8RFE-PS46
8. Fitzpatrick E, Merchant A, Raban MZ, Westbrook JI. Stop! Is that weight out-of-date? preventing dose errors in children. Australian Institute of Health Innovation, Macquarie University, 2023. 2 p. (Health Innovation Series; 10). doi: 10.25949/76Q9-B991
9. Merchant A, Fitzpatrick E, Raban MZ, Westbrook JI. Is the rounding rule in your dose calculator causing dose errors in children? Australian Institute of Health Innovation, Macquarie University, 2023. 2 p. (Health Innovation Series; 9). doi: 10.25949/99b1-sk02
10. Merchant A, Westbrook JI, Fitzpatrick E, Raban MZ. Can’t find a medication in the electronic medication system? How to prevent errors and user frustration. Australian Institute of Health Innovation, Macquarie University, 2023. 2 p. (Health Innovation Series; 8). doi: 10.25949/m2h3-1w46

2022

1. Merchant A, Raban MZ, Fitzpatrick E, Westbrook JI. Dose calculator: Missing in action! Australian Institute of Health Innovation, Macquarie University, 2022. 2 p. (Health Innovation Series; 7). doi: 10.25949/w6vs-dd62
2. Merchant A, Fitzpatrick E, Westbrook JI, Raban MZ. Accidental prescribing of extended-release opioids. Australian Institute of Health Innovation, Macquarie University, 2022. 1 p. (Health Innovation Series; 6). doi: 10.25949/5ka7-ab42
3. Raban MZ, Merchant A, Fitzpatrick E, Westbrook JI. Pre-operative medication frequencies matter. Australian Institute of Health Innovation, Macquarie University, 2022. 1 p. (Health Innovation Series; 5). doi: 10.25949/7yc3-a257
4. Raban MZ, Merchant A, Fitzpatrick E, Westbrook JI. Preventing dangerous intraspinal injections. Australian Institute of Health Innovation, Macquarie University, 2022. 1 p. (Health Innovation Series; 4). doi: 10.25949/dy3f-pa76
5. Merchant A, Fitzpatrick E, Westbrook JI, Raban MZ. Double dose trouble: systemic intranasal medication: Can you spot the problem? Australian Institute of Health Innovation, Macquarie University, 2022. 2 p. (Health Innovation Series; 3). doi: 10.25949/PRND-F975
6. Merchant A, Raban MZ, Westbrook JI. Caution: editing within a dose calculator can result in large dose errors. Australian Institute of Health Innovation, Macquarie University, 2022. 1 p. (Health Innovation Series; 2). doi: 10.25949/MWNH-P929
7. Raban MZ, Merchant A, Westbrook JI. Prescribing an IV in an electronic medication system: What could possibly go wrong? Australian Institute of Health Innovation, Macquarie University, 2022. 2 p. (Health Innovation Series; 1). doi: 10.25949/N04Z-4Y85

Conference proceedings and poster presentations

2024

1. Raban MZ, Merchant A, Clay C, Fitzpatrick E, Westbrook JI. The Health Innovation Series: translating research evidence into practice. In Bichel-Findlay J, Otero P, Scott P, Huesing E, editors, MEDINFO 2023 - The future is accessible: Proceedings of the 19th World Congress on Medical and Health Informatics. Amsterdam: IOS Press. 2024. p. 339-343. (Studies in Health Technology and Informatics). doi: 10.3233/SHTI230983
2. Mumford V, Raban MZ, Li L, Merchant A, Fitzpatrick E, Badgery-Parker T et al. The economics of medication safety: a cost-benefits analysis framework for evaluating an electronic medication system. In Bichel-Findlay J, Otero P, Scott P, Huesing E, editors, MEDINFO 2023 - The future is accessible: Proceedings of the 19th World Congress on Medical and Health Informatics. Amsterdam: IOS Press. 2024. p. 1390-1391. (Studies in Health Technology and Informatics). doi: 10.3233/SHTI231209
3. Westbrook JI, Li L, Woods A, Badgery-Parker T, Mumford V, Raban MZ. Stepped-wedge cluster RCT to assess the effects of an electronic medication system on medication administration errors. In Bichel-Findlay J, Otero P, Scott P, Huesing E, editors, MEDINFO 2023 - The future is accessible: Proceedings of the 19th World Congress on Medical and Health Informatics. Amsterdam: IOS Press. 2024. p. 329-333. (Studies in Health Technology and Informatics). doi: 10.3233/shti230981
4. Mumford V, Raban MZ, Li L, Westbrook JI. Identifying actual harm from prescribing errors in acute paediatric care. Abstract from ISQua 40th International Conference 2024, Istanbul, Turkiye

2023

1. Mumford V, Gu Y, Westbrook JI. Measuring parent preferences for care during a child’s hospital admission: an unlabelled DCE survey. Abstract from 44th Australian Health Economics Society Conference 2023, Adelaide, Australia
2. Mumford V, Gu Y, Westbrook JI. Measuring parent preferences for care during a child’s hospital admission: a Discrete Choice Experiment. Abstract from 14th Meeting of the International Academy of Health Preference Research 2023, Sydney, Australia
3. Mumford V, Gu Y, Westbrook JI. Measuring Parent Preferences for Care During a Child’s Hospital Admission: A Discrete Choice Experiment. The patient. 2023;16(4).
4. Mumford V, Raban MZ, Li L, Westbrook J. Measuring actual harm from medication administration errors in an acute paediatric hospital. 2023. Abstract from ISQua 39th International Conference: Technology, Culture, and Coproduction: Looking to the horizons of quality and safety. 2023, Seoul, South Korea
5. Mumford V, Westbrook JI. Evaluating complex interventions in medication safety. Abstract from ISQua 39th International Conference: Technology, Culture, and Coproduction: Looking to the horizons of quality and safety. 2023. Seoul, South Korea

2022

1. Raban MZ, Merchant A, Badgery-Parker T, Westbrook J. Longitudinal assessment of technology-related prescribing errors following implementation of an electronic prescribing system at a paediatric hospital. Abstract from ISQua 38th International Conference. 2022, Brisbane, Queensland, Australia.

2021

1. Mumford V, Raban MZ, Gates P, Westbrook JI. Evaluating parent preferences for preventing and managing medication errors in paediatric acute care: Designing a discrete choice experiment. Abstract from ISQua 37th International Conference: Emotion, Inspiration and Creativity: Pathways to Global Health Quality. 2021, Online
2. Van Dort BA, Baysari MT, Prgomet M, Zheng WY, Raban MZ, Dalla-Pozza L et al. Impact of electronic medication management on the physical location of work in a paediatric setting. In Marcilly R, Dusseljee-Peute L, Kuziemsky CE, Zhu X, Elkin P, Nohr C, editors, Context sensitive health informatics: the role of informatics in global pandemics. Amsterdam: IOS Press. 2021. p. 72-76. (Studies in Health Technology and Informatics). doi: 10.3233/SHTI210640

2019

1. Mumford V, Raban MZ, Gates P, Westbrook JI. Measuring actual harm from paediatric medication errors. Abstract from ISQua 36th International Conference: Innovate, Implement, Improve: beating the drum for safety, quality and equity. 2019, Cape Town, South Africa
2. Mumford V. Evaluating parent preferences for preventing and managing medication errors in paediatrics: a discrete choice experiment. Absract from 41st Australian Health Economics Society Conference. 2019, Melbourne, Australia

2018

1. Baysari MT. eMeds - Changes in nurses’ and doctors’ perceptions across the first six months of use. Abstract from eMedication Management Conference. 2018 Mar 21-22, Sydney, Australia
2. Raban MZ, Baysari MT. Implementation of eMMS in paedicatrics – key lessons learnt from a rigorous, multi-method evaluation. eHealth NSW’s eMR Connect Clinical Leaders’ Network. 2018 May 10.
3. Westbrook JI, Baysari M, Li L, Raban MZ, Mumford V, Gates P, Koyama A, Hardie R, Woods A, Kim T, Dong C, Fitzpatrick E. Evaluation of eMM in paediatrics. Abstract from eMedication Management Conference. 2018 Mar 21-22, Sydney, Australia.

2017

1. Mumford V, Baysari M, Kalinin D, Raban MZ, McCullagh C, Karnon J, Westbrook J. Measuring the financial and productivity burden of paediatric hospitalisation on the wider family network. Abstract from 39th Australian Health Economics Society Conference. 2017 Sept 21-22, Sydney, Australia.
2. Mumford V, Baysari M, Kalinin D, Raban MZ, McCullagh C, Karnon J, Westbrook J. Measuring the non-medical out-of-pocket costs for families with a child in hospital. Abstract from 2nd EnCourAge Symposium, Macquarie University. 2017 Oct 27, Sydney, Australia. (2nd Prize for long case presentation)
3. Baysari M. Understanding the user experience of eMM implementation: a 6-month longitudinal study. 2017. Paper presented at 52nd Annual Conference of the Human Factors and Ergonomics Society of Australia: Sustainable Ergonomics, HFESA 2017, Wollongong, Australia.
4. Hardie R-A, Baysari MT, Lake R, Richardson L, McCullagh C, Westbrook JI. User perceptions of the implementation of an electronic medication management system in a paediatric setting. In Ryan A, Schaper LK, Whetton S, editors, Integrating and Connecting Care - Selected Papers from the 25th Australian National Health Informatics Conference, HIC 2017. Vol. 239. IOS Press. 2017. p. 41-47. (Studies in Health Technology and Informatics). doi: 10.3233/978-1-61499-783-2-41
5. Li L, Kyng T, Westbrook J. Determining sample sizes for assessing inter-observer reliability for direct observational studies. 2017. Paper presented at The 61st World Statistical Congress, Marrakech, Morocco.
6. Mumford V, Baysari M, Kalinin D, Raban MZ, McCullagh C, Karnon J, Westbrook JI. Measuring the impact of a child’s stay in hospital on the wider family network [Abstract and Poster]. Abstract from ISQua 34th International Conference: Learning at the System level to Improve Healthcare Quality and Safety. 2017 Oct 1-4, London, UK.
7. Westbrook JI, Li L, Kim T, Baysari MT. Medication administration practices on the weekend versus weekday? A direct observational study of 227 paediatric nurses. Abstract from ISQua 34th Internatioal Conference: Learning at the System Level to Improve Healthcare Quality and Safety. 2017 Oct 1-4, London, UK.

2016

1. Baysari M. CHW users’ views of the eMM. Abstract from Delivering Safer and Better Care for Children in Hospital: The Sydney Children’s Hospitals Network Symposium Symposium. 2016 Sept 6, Sydney, Australia.
2. Li L. Challenges in the design and analysis of stepped-wedge cluster randomized controlled trials to evaluate the effectiveness and safety of electronic medical record systems. 2016. Abstract from XXVIIIth International Biometric Conference, Victoria, Canada.
3. Mumford V, Karnon J. Approaches to assessing the cost-effectiveness of eMM in paediatrics. Abstract from Delivering safer and better care for children in hospital: The Sydney Children’s Hospitals Network Symposium. 2016 Dec 6, Sydney Australia.
4. Westbrook JI. Assessing the effects of eMM in paediatrics. Abstract from 5th Annual eMedication Management Conference. 2016 Mar 15-16, Sydney, Australia.
5. Westbrook JI. Observational study of medication administration at CHW: preliminary results. Abstract from Delivering Safer and Better Care for Children in Hospital Symposium. 2016 Dec 6, Sydney, Australia.