An Audit on Admission Outcome Measures in a National Child and Adolescent Mental Health Inpatient Unit

Introduction

Mental disorders in young people are ideally treated in a community setting (rather than in a hospital) within child and adolescent mental health services (CAMHS), thus avoiding risks towards the young person’s psychosocial development, educational achievements etc.¹ However, those with complex or severe mental health symptoms may require care and treatment in an inpatient mental health setting.¹ The National Institute for Health and Care Excellence (NICE) guidance ² recommends that “all healthcare and CAMHS professionals should routinely use, and record in the notes, appropriate outcome measures”. Such outcome measures may take the form of self-/clinician-reported questionnaires and screening items.

Outcome measures help provide an understanding of the effectiveness of the service being provided while monitoring young people’s mental health status within an inpatient setting.³ Completing outcome measures is an important aspect for one’s admission as it would highlight any changes in the individual’s mental health from admission to discharge. They also help to contextualise the significant changes in the patient’s mental health variables.

The aims of this research were to analyse the use of pre- and post-admission outcome measures for admissions to the Young People’s Unit (YPU), Mount Carmel Hospital, Malta, in 2022; and to suggest recommendations for implementation to improve the completion rate of pre-post-admission outcome measures.

Materials and Methods

Study Sample and Procedure

The study sample included all YPU admissions between the 1st of January 2022 and the 31st of December 2022. A retrospective case note analysis was carried out to analyse the completion of pre- and post-admission outcome measures; the data were recorded and tabled accordingly.

Outcome Measures

The outcome measures below were selected because they were the assigned measures for admissions within the inpatient unit. The first outcome measure was the Health of the Nation Outcome Scales for Children and Adolescents (HoNOSCA), which is a 15-item practitioner-rated scale for young people aged 5 to 17 years which assesses multiple mental health components, such as overactivity, self-harm, hallucinations/delusions, peer relationships etc. It has shown good apparent face validity and interrater reliability (intraclass correlation coefficient of more than 0.8 in eight of the subscales).⁴ The second outcome measure was the practitioner-rated 6-item Kutcher Adolescent Depression Scale (KADS-6), which assesses depressive symptoms, and has reported sensitivity and specificity rates of 92% and 71% respectively.⁵ It also showed good reliability (α = 0.74) and convergent validity with the Beck’s Depression Inventory-II (r = 0.44).⁶ The third outcome measure was the Moods and Feelings Questionnaire (MFQ) ⁷ with reported excellent internal reliability (α = 0.91 to 0.93) and concurrent validity of more than 0.40 in most of its items.⁸ The fourth outcome measure was the Tool for Assessment of Suicide Risk - Adolescent (TASR-A).⁹ No information is available on the tool’s validity and reliability since this was not designed as a predictive or diagnostic tool, but as a means of summarising one’s suicidal assessment.¹⁰ The fifth outcome measure was the Children’s Global Assessment Scale (CGAS), which reported evidence of discriminant validity and excellent interrater reliability (intraclass correlation coefficient = 0.84).¹¹ At YPU, these outcome measures are usually completed by the on-call doctors or doctors carrying out the pre-/post-admission reviews. The relevant demographic data were also recorded for the purpose of this audit study.

Guideline

The NICE guidance ² used for comparison in this audit was NG134, 1.1.2.5: “All healthcare and CAMHS professionals should routinely use, and record in the notes, appropriate outcome measures (such as those self-report measures used in screening for depression or generic outcome measures used by particular services) …”.

Ethics

Research governance was granted by the Data Protection Officer of the Malta Mental Health Services, Government of Malta. Approval to present and publish was also obtained by the Chairman of the Department of Psychiatry (Malta Mental Health Services, Government of Malta). All data were stored in a password-protected spreadsheet within an encrypted computer.

Results

A total of 83 YPU admissions were recorded during the year 2022. Of these, 63.9% (n = 53) were females whilst 36.1% (n = 30) were males (Figure 1). Out of the 83 YPU admissions, 24% of which were re-admissions (n = 20). Pre-admission outcome measures were completed on the day of admission or within 24 hours of the admission. Post-admission outcome measures were completed on the day of discharge (post-admission review).

Figure 1

Admissions to YPU in 2022 Divided by Gender

Results (Figure 2, Figure 3) revealed that the Moods and Feelings Questionnaire was the least completed measure with a 0% completion rate for both the pre-admission and the post-admission phases. During the pre-admission phase, the KADS-6 was completed most (n = 22, 26.5%), followed by the TASR-A (n = 18, 21.7%), the HoNOSCA (n = 12, 14.5%), and the CGAS (n = 7, 8.4%). During the post-admission phase, the KADS-6 (n = 13, 15.66%) and the HoNOSCA (n = 13, 15.66%) were completed the most, followed by the TASR-A (n = 12, 14.50%) and the CGAS (n = 10, 12.04%).

Figure 2

Results Showing Completed Pre-Admission Outcome Measures

Figure 3

Results Showing Completed Post-Admission Outcome Measures

In females, the most completed outcome measure was KADS-6 (n = 27, 35.1%), followed by TASR-A, HoNOSCA, and CGAS. In males, the most completed outcome measure also was the KADS-6 (n = 12, 35.3%), followed by TASR-A whilst HoNOSCA and CGAS were completed equally.

Only 27.7% (n = 23) and 20.5% (n = 17) of admissions had outcome measures completed during the pre-admission and post-admission phase respectively. This meant that 72.3% (n = 60) of admissions did not have any outcome measures completed upon admission, while 79.5% (n = 66) of admissions did not have any outcome measures completed upon discharge. It was also noted that the date was not always appropriately documented on the outcome measure sheets.

Discussion

This audit study looked at the completion rate of outcome measures in a national inpatient unit within CAMHS. This first audit cycle showed a generally low completion rate of outcome measures. This was even lower than previous similar research in CAMHS ¹², which also reported low outcome measure completion rates with subsequent improvement in completion rates by introducing a Commissioning for Quality and Innovation target (e.g., CGAS completion rates improved from 41% to 75%).

One may postulate different reasons for the low completion rates of outcome measures. One possible reason might be staff burnout. The high workload and pressures placed on mental health staff might account for overwhelming levels of burnout, decreasing efficiency.¹³ Another possible reason for the low completion rates is understaffing within the inpatient unit, which has been heavily documented in mental health settings.^14,15 Furthermore, since in YPU only doctors generally complete these outcome measures, this adds more work to the already overstretched working schedule. Therefore, all members of the multidisciplinary team should be able to complete such questionnaires with patients.

Conclusion and Recommendations

This first audit cycle highlights the low completion rates of outcome measures within a national inpatient child and adolescent mental health inpatient unit. Therefore, the authors are putting forward the following recommendations:

1. Recommendations for Practice: i) Psychoeducation material and sessions for mental health professional staff members about the importance of outcome measures in practice; ii) to include outcome measures with patient admission and discharge papers; iii) to encourage all members of the multidisciplinary team (including non-medical staff) to complete outcome measures; iv) to include the date and time on the outcome measure forms, v) to work on an official inpatient package of outcome measures.
2. Recommendations for Research: i) To carry out the second loop of the audit after three months of psychoeducation, thus completing the audit cycle; ii) To carry out a qualitative study to explore reasons for the low completion rates of outcome measures; iii) To carry out a prospective service evaluation using outcome measure analysis to assess the effectiveness of inpatient admissions.

Summary Box

What is already known

• The use of outcome measures is recommended by NICE guidelines and may take the form of questionnaires
• These will help contextualise the patient’s feelings and monitor their progress
• These outcome measure should be applied to all admission to an inpatient young people’s unit by mental health professionals

New findings

• Further emphasis on the importance of outcome measures is required, as evidenced by low completion rates

Acknowledgements

I would like to express my gratitude towards Dr Daniel Vella Fondacaro for his advice, supervision, and guidance. Moreover, I would like to thank the Chairman of the Department of Psychiatry (Prof. Anton Grech), and the Data Protection Officer (Mr. Oswald Balzan) of the Malta Mental Health Services for granting me permission to carry out this research.

Abbreviations

CAMH Child and Adolescent Mental Health Services

CGAS Children’s Global Assessment Scale

HoNOSCA Health of the Nation Outcome Scales for Children and Adolescents

KADS-6 6-item Kutcher Adolescent Depression Scale

MFQ Moods and Feelings Questionnaire

NICE National Institute for Health and Care Excellence

TASR-A Tool for Assessment of Suicide Risk – Adolescent

YPU Young People’s Unit

References

1. Edwards D, Evans N, Gillen E, et al. What do we know about the risks for young people moving into, through and out of inpatient mental health care? Findings from an evidence synthesis. Child Adolesc Psychiatry Ment Health. 2015;9:55.
2. National Institute for Health and Care Excellence (NICE). Depression in children and young people: identification and management. NICE guideline NG134. London: NICE; 2019 [cited 2023 May 8]. Available from: https://www.nice.org.uk/guidance/ng134
3. Kilbourne AM, Keyser D, Pincus HA. Challenges and opportunities in measuring the quality of mental health care. Can J Psychiatry. 2010;55:(9)549–57.
4. Gowers SG, Harrington RC, Whitton A, et al. Brief scale for measuring the outcomes of emotional and behavioural disorders in children. Health of the Nation Outcome Scales for children and adolescents (HoNOSCA). Br J Psychiatry. 1999;174:413–6.
5. LeBlanc JC, Almudevar A, Brooks SJ, Kutcher S. Screening for adolescent depression: comparison of the Kutcher Adolescent Depression Scale with the Beck depression inventory. J Child Adolesc Psychopharmacol. 2002;12:(2)113–26.
6. Quintão S, David S, Gusmão R, Kutcher S. Contribution to the validation of the Kutcher Adolescent Depression Scale (KADS-6) in a Portuguese population. Psicol Reflex Crit. 2015;28:313–21.
7. Angold A, Costello EJ, Messer SC, Pickles A. Development of a short questionnaire for use in epidemiological studies of depression in children and adolescents. Int J Methods Psychiatr Res. 1995;5:(4)237–49.
8. Thabrew H, Stasiak K, Bavin LM, Frampton C, Merry S. Validation of the Mood and Feelings Questionnaire (MFQ) and Short Mood and Feelings Questionnaire (SMFQ) in New Zealand help-seeking adolescents. Int J Methods Psychiatr Res. 2018;27:(3)e1610.
9. Chehil S, Kutcher P. Suicide risk management: a manual for health professionals. 2nd ed. Hoboken (NJ): Wiley-Blackwell; 2012.
10. Shaffer D, Gould MS, Brasic J, et al. A children's global assessment scale (CGAS). Arch Gen Psychiatry. 1983;40:(11)1228–31.
11. Perlman CM, Neufeld E, Martin L. Suicide risk assessment guide: a resource for health care organizations. Toronto (ON): Ontario Hospital and Canadian Patient Safety Institute; 2011.
12. Hall CL, Moldavsky M, Baldwin L, et al. The use of routine outcome measures in two child and adolescent mental health services: a completed audit cycle. BMC Psychiatry. 2013;13:270.
13. Morse G, Salyers MP, Rollins AL, Monroe-DeVita M, Pfahler C. Burnout in mental health services: a review of the problem and its remediation. Adm Policy Ment Health. 2012;39:(5)341–52.
14. Cranage K, Foster K. Mental health nurses' experience of challenging workplace situations: a qualitative descriptive study. Int J Ment Health Nurs. 2022;31:(3)665–76.
15. Samartzis L, Talias MA. Assessing and improving the quality in mental health services. Int J Environ Res Public Health. 2019;17:(1)249.

Figure

Test image