An analysis of the effect of educational environment on burnout: a cross-sectional observational study of trainee doctors at the Malta Foundation Programme

There is no alternative to learning when managing real patients in the clinical context.¹ A suboptimal educational environment has been associated with effects on both patient care and trainee wellbeing.  Patient care may suffer when the learning environment is not optimal eg higher complication rates.² Trainees exposed to suboptimal educational environments are known to be at risk of deleterious consequences to their mental health and their educational achievement.  Mental health issues may include stress, anxiety, depression, burnout, emotional exhaustion and depersonalization.³

Defining the educational environment remains a challenge mainly because of differences in existing definitions which highlight the different dimensions and elements that are taken into consideration by different authors.  For example, Kilty et al.³ describe the educational environment as encompassing “the social, cultural and material context in which residents learn while they work”. On the other hand, the MACY Foundation⁴ defines the educational environment as:

“social interactions, organizational cultures and structures, and physical and virtual spaces that surround and shape participants’ experiences, perceptions and learning”

Attempts to measure the educational environment have led to the development of numerous instruments such as DREEM (Dundee Ready Educational Environment Measure)⁵, STEEM (Surgical Theatre Educational Environment Measure)⁶, ATEEM (Anaethetic Theatre Educational Environment Measure)⁷ and PHEEM (Postgraduate Hospital Educational Enivornment Measure).⁸ The latter is widely used and has been validated in various countries, settings and cultures.

The concept of burnout was introduced by Freidenburger⁹ and Maslach¹⁰ working independently from one another while studying volunteers working with people with social problems. Burnout has been defined as a psychological syndrome that consists of emotional exhaustion, depersonalization and reduced personal accomplishment that is directly related to caregiving activities.¹¹ Burnout can result in either of three dimensions: personal burnout, work-related burnout and client-related burnout.  These three subscales form the Copenhagen Burnout Inventory¹², which is the burnout measure recommended for use by the British Medical Council.¹³

The consequences of burnout among physicians in practice and those in training are wide ranging.  Literature shows that burnout is associated with negative effects at both the personal and the institutional level.¹⁴ Patients may be affected by poor patient outcomes, decreased patient satisfaction and suboptimal patient care practices.¹⁵ Burnout has also been associated with direct effects on the individuals effected – stress,¹⁶ depression,¹⁷ anxiety,¹⁷ alcoholism,¹⁸ cynicism¹⁹ and suicide.²⁰

The role of the educational environment in the development of burnout has been studied in many countries.  This is the first such study in a Maltese population.  The educational environment has been described as “a crucial factor”²¹ contributing to trainee wellbeing and learning achievement.  A significant correlation between the educational environment and the prevalence of burnout has been outlined in other studies.^22-23

The aim of this study was to analyse the effect of the educational environment on the prevalence of burnout among doctors within the Malta Foundation Programme.

Methodology

A cross-sectional observational methodology was adopted for this study.  An online Google forms questionnaire was distributed among all doctors within the Malta Foundation Programme.  These were divided into 124 first-year Foundation doctors (FY1s), 133 second-year Foundation doctors (FY2s) and 113 extended Foundation doctors (extended FYs).  The latter category were FYs (foundation doctors) who had completed their two-year training programme, but were in the process of being accepted into further training as BSTs (basic specialist trainees) or trainee GPs (general practitioners).  This group of doctors are no longer under the responsibility of the Foundation Programme. However, they provided an opportunity of sampling trainees at the end of their two-year training programme.  The other two groups were sampled at 3-months (FY1s) and one year (FY2s) of training.

FY2s and extended FYs were invited to participate in July/August 2020 whereas FY1s were invited in October 2020.  The invitation was sent through the Year representatives to all FY doctors in their respective year.  The questionnaire was accompanied by a covering letter and a participant information sheet.  Informed consent was obtained from all respondents.  All replies were online with complete anonymity of the respondents being ensured.  A reminder was sent one week after the original invitation request.  Data collection was completed two weeks after the reminder.

The Faculty Research Ethics Committee of the University of Malta provided ethical approval.  The Malta Foundation Programme authorised distribution of the questionnaire to FY doctors.

The instruments used were the Postgraduate Hospital Educational Environment Measure (PHEEM) for the evaluation of the educational environment, and the Copenhagen Burnout Inventory (CBI) for the evaluation of burnout.

Copenhagen Burnout Inventory

The CBI is a 19-item questionnaire validated and used in many countries which include Denmark, New Zealand, Sri Lanka, Taiwan and Hong Kong.²⁴ It is also the measure recommended for use by the General Medical Council of the United Kingdom.¹³ The CBI is made up of three distinct subscales: personal burnout, work-related burnout and client-related burnout. 

The personal burnout subscales is made up of six items. It measures the level of burnout in people irrespective of their employment status.  The work-related burnout subscale consists of seven items and measures a person’s attribution of any experienced fatigue and exhaustion to his or her work.  Client-related burnout subscale is made up of six items that measure whether a person considers any experienced fatigue or exhaustion to be related to work with client e.g. patients or students.¹²

Each item on the CBI is scored on a 5- point Likert scale as follows:

100 – Always or to a very high degree

75 – Often or to a high degree

50 – Sometimes or somewhat

25 – Seldom or to a low degree

0 – Never/almost never or to a very low degree 

Postgraduate Hospital Educational Environment Measure

PHEEM was the instrument of choice to measure the educational environment as perceived by the trainee doctors.  PHEEM has been validated in many countries and in different settings.  Minimal changes to the wording of the instrument were made in consultation with the original author⁸ to ensure relevance to the local setting without loss of validity.

PHEEM is a 40-item questionnaire scored on a Likert scale as follows:

4 – Strongly agree

3 – Agree

2 – Uncertain

1 – Disagree

0 – Strongly disagree.

PHEEM consists of three sub-scales, each measuring perceptions of role autonomy (14 items), teaching (15 items) and social support (11 items). When interpreting the scores for each of the three sub-scales, the authors recommend the following schema:

1. Perceptions of role autonomy

 0-14 – very poor

15-28 – a negative view of one’s role

29-42 – a more positive perception of one’s role

43-56 – excellent perception of one’s job

1. Perceptions of teaching

  0-15 – very poor quality

16-30 – in need of some retraining

31-45 – moving in the right direction

46-60 – model teachers

• Perceptions of social support

  0-11 - non-existent

12-22 - not a pleasant place

23-33 - more pros than cons

34-44 - a good supportive environment.

Roff et al⁸ recommend the following interpretation of the overall score:

0-40 – very poor

41-80 – plenty of problems

81-120 – more positive than negative but room for improvement

121-160 – excellent

Roff et al.⁸ recommend that PHEEM can be used to identify specific strengths and weaknesses of the educational environment.  Each individual item can be analysed by itself and any items having a mean score of 3.5 or more would be considered as a very positive point. Conversely, any item with a mean score of 2 or less signifies a problem area.  Should the mean score of any item fall between 2 and 3, that specific item within the environment could potentially be enhanced.

Statistical analyses

Data was analysed using SPSS v25.0.  A Shapiro-Wilk test determined that data was normally distributed.  Descriptive statistics were used to outline the demographic data of the study. Cronbach’s alpha was used to test reliability of both PHEEM and CBI together with their respective sub-scales.  As recommended in the literature, a score of more than 50 on any of the CBI subscales was used as a cut off point for the presence of burnout in that subscale.

One-way ANOVA was used to assess for significant differences between cohorts on the CBI subscales. An independent t-test analysis was to analyse for significant differences between genders and also for differences between those suffering from burnout and those not.  Spearman correlation was carried out to assess for significant correlation between the respective subscales of PHEEM and CBI.

The threshold for statistical significance was a p<0.05 and 95% confidence interval.

Results

Descriptive Statistics

The questionnaire was electronically distributed to 370 Foundation doctors (124 FY1s, 133 FY2s and 113 extended FYs).  The responses received were from: 39 FY1s (31.5%), 33FY2s (24.8%) and 26 extended FYs (23.0%).

Most worked over the 48 hours stipulated in the European Working Time Directive as shown in Table 1.

60.2% of the respondents were female with 94.9% being single. The ages of the participants ranged between 23 and 32 years with a mean of 24.7years and a mode of 23 years.  87.8% were Maltese, 8.2% from another EU country and 4.1% from a non-EU country.

Reliability

Both PHEEM and CBI total scores and their respective subscales showed good reliability with high Cronbach’s alpha values as shown in Table 2.

Analytical statistics

The Educational Environment assessment produced a mean overall score of 86.95 with a standard deviation of 20.76 (range 19-139). This means that, overall, the foundation doctors’ perception of the educational environment is more positive than negative but there is ample room for improvement.  The overall mean for each of the PHEEM subscales is shown in Table 3.

The results for each foundation year cohort have been presented in a separate paper (preprint).

The level of burnout is high across all stages of Foundation training in Malta. Burnout peaks midway through the two-year training period. Table 4 shows the respective mean scores for each cohort for each of the three subscales of the CBI.

A one-way ANOVA analysis for significant between group differences for foundation year cohorts showed significant differences for each of the CBI subscales: personal burnout (F:8.098, p=0.001), work-related burnout (F=10.086, p=<0.001) and client-related burnout (F=7.772), p=0.001).

An independent t-test analysis for the differences between genders, identified a statistically significant higher level of work-related burnout in males (t_{72.94 1}= -2.075=, p=0.042).  Males also had a statistically significant higher client-related burnout (t_72.833 = -2.278, p=0.026).

The assessment of the subscales of PHEEM and CBI showed a highly significant negative correlation between autonomy, teaching and social support, and personal, work-related and client-related burnout. These results are summarized in table 5.

Spearman correlation analysis showed that higher CBI total scores were correlated with lower PHEEM total scores (r_s=-0.463, p<0.001).

When comparing the two groups (burnout vs no burnout) no statistically significant difference was found regarding age, gender, marital status, nationality, or number of hours worked.

There was a significant difference between those suffering from burnout on any of the CBI scales (score >50) and those not suffering from burnout on the total PHEEM score.  Those who scored high on any of the three CBI subscales, had a low perception of the educational environment. Table 6 depicts these results.

The differences between those suffering from burnout on any of the CBI subscales (score>50) and those not suffering from burnout on each of the three PHEEM subscales (role autonomy, teaching and social support) are explained in Tables 7, 8 and 9.

(T=t-test, DF=degrees of freedom)

Discussion

Foundation doctors are at a very delicate stage of their career.  They also have to face a duality of roles – that of a doctor entrusted with very sensitive situations and that of a trainee that is still learning on the go.  They have to face situations that are highly emotional and stressful – death, failure of treatment, delivering bad news and dealing with uncertainty.  At the same time, they also need to deal with situations at home and at their workplace that are not directly the effect of their role with patients. Trainees are also learning to improve their skills, prepare for exams, further their education and keep in touch with new knowledge in medicine.  Most trainees are expected to cope well under these conditions. Some may, however, find the burden overwhelming, thereby increasing the odds for burnout.

The study was carried out during the Covid-19 pandemic.  The lowish response rate (26.5%) may be a reflection of an overworked cohort who find little time to reply to lengthy questionnaires.  Some of the results regarding the levels of burnout may have been strongly influenced by the changes in working practice brought about by dealing with an increased workload in a pandemic. 

Extended FY doctors may also be more prone to burnout as they are in a stage in their career when uncertainty prevails.  Much of their career will depend on the upcoming post which they may or may not be able to secure.  Such uncertainty may increase the odds for burnout.

This study analysed the impact of the educational environment, as perceived by doctors within the Malta Foundation Programme, on burnout in these doctors.  The results show that there is a significant association between the learning environment and burnout. It is the first study of its sort that has been carried out among Foundation doctors in Malta. Another local study was conducted by Camilleri²⁵ who after a qualitative study “Stress and Coping in Junior Doctors” concluded that, particularly in the early stages of their training, junior doctors were subjected to high levels of stress and experienced problems coping.  These finding were reflected in a study by Taylor-East, Grech and Gatt.²⁶

A correlation analysis showed a highly significant negative correlation between all subscales of the educational environment and all subscales of CBI.  Trainees with a poor perception of the educational environment were more likely to be suffering from burnout.  This negative association between burnout and a low perception of role autonomy, teaching and social support is evident in all three subscales of CBI - personal, work-related and client-related.  The only exception is the association of teaching with personal and client-related burnout.

The prevalence of burnout among doctors within the Malta Foundation Programme is high.  The prevalence of burnout is on the higher side when compare to that in other countries.^27-28 William et al.²⁹ in a literature review, reported a prevalence of burnout ranging between 27% and 75% among residents. McCray³⁰ also reported a prevalence between 47% and 76%.  The results reported in this study fall within the higher part of these reported ranges.

The number of hours that these doctors work is well in excess of the EWTD.  However, no association between the number of hours worked and the prevalence of burnout could be found in this study.

The lowish scores for all three subscales of PHEEM indicate that while, in general, the educational environment is considered positive by the trainees, considerable improvement is needed. This improvement could result in a reduction in the prevalence of burnout in these trainee doctors.

There is a negative relationship between social support and total CBI and its subscales.  Previous reports have outlined this inverse relationship between prevalence of burnout and collegial support.³¹ When analysing the individual items in PHEEM, it becomes apparent that the items with the lowest scores are those associated with out of hours duties (number of hours worked, catering facilities while on call, inappropriate bleeping, a blame culture, poor accommodation when on call).

Autonomy and teaching were also significantly and negatively associated with all three CBI subscales. This was also reported by Papaefstathiou in a study of Greek residents.²⁷ Zis et al.³² have also reported that each additional point for autonomy was associated with a reduction in the likelihood of burnout.

The negative correlation between autonomy and burnout has also been described by Llera and Durante²² and Eckelberry³³. This negative correlation between autonomy and burnout is not limited to medicine but it has also been described in non-medical, service-related occupations.³⁴

These findings can serve as a starting point in the setting up of curricular modifications aimed at improving the overall educational environment.  Though many interventions to tackle burnout are aimed at the individual (for example, aimed at increasing the individual’s resilience), burnout needs to be tackled at an organisational and institutional level too for better results. 

Limitations

The low response rate despite the reminder sent is a major limitation of this study.  Performing a study at the height of a pandemic was not ideal.  Foundation doctors who were overwhelmed with clinical work might have been less likely to respond to a rather lengthy questionnaire. 

This low response rate could introduce bias in that trainees interested or suffering from burnout may have been more inclined to reply.

A qualitative study may be indicated to analyse in depth the reasons behind the trainees’ perceptions. The quantitative nature of this study does not allow for in depth analysis of these perceptions. 

No study using the CBI in a Maltese population was identified.  Therefore, a cut-off point of 50 used in other studies was adopted.

A repeat of this study when the pandemic is over will give a clearer picture and will identify the effect the pandemic may have had on the results of this study.

Conclusion

This study has confirmed the correlation that exists between the educational environment and burnout among early medical trainees in Malta.  The high level of burnout identified is a cause for concern and calls for action aimed at improving the wellbeing of these young doctors. 

At a local level, medical students receive training in stress management and in building of resilience.  Postgraduate trainees in difficulty are offered help directly on an individual basis by the Malta Foundation Programme.  However, cases still persist of trainees who decide to quit the medical profession as they feel they cannot cope with the burdens of the job.

Efforts need to be directed at the institutional and organizational level.  Changes at this level should help in reducing the prevalence of burnout among these trainees and, as a result, improve retention rates and patient outcomes. 

An improved educational environment can improve the quality and safety of patient care, as well as improve the mental and physical health of the trainees. One has to remember that these trainees form a very important part of the medical team and that their work is invaluable to the smooth running and objectives achievement of any department.

SUMMARY

What is already known about this subject?

• A suboptimal educational environment has been associated with effects on both patient care and trainee wellbeing
• Burnout has been defined as a psychological syndrome that consists of emotional exhaustion, depersonalization and reduced personal accomplishment that is directly related to caregiving activities.
• Burnout can result in either of three dimensions: personal burnout, work-related burnout and client-related burnout.

What are the new findings?

• There is a significant association between the learning environment and burnout.
• A correlation analysis showed a highly significant negative correlation between all subscales of the educational environment and all subscales of the Copenhagen burnout inventory within the Malta Foundation Programme.
• The prevalence of burnout among doctors within the Malta Foundation Programme is high.
• The educational environment is mostly perceived as positive by the trainees but considerable improvement is needed.

References

1. Nordquist J, Hall J, Caverzagie K, Snell L, Ming-Ka C, Thoma B, Razack, S et al. The clinical learning environment. Medical Teacher 2019; 41(4), 366-372.
2. Bansal N, Simmons KD, Epstein AJ, Morris JB, Kelz RR. Using patient outcomes to evaluate general surgery residency program performance. JAMA Surg.2016; 151, 111-119.
3. Kilty C, Flood P, Fu N, Horgan M, Higgins A, Bridget M et al. A national stakeholder consensus study of challenges and priorities for clinical learning environments in postgraduate medical education. BMC Medical Education. 2017. DOI: 10.1186/s12909-017-1065-2.
4. Josiah M., Jr. Foundation. 2018. Improving environments for learning in the health professions. Recommendations from the Macy Foundation. Conference. New York: Josiah Macy Jr. Foundation.
5. RoffS. The Dundee Ready Educational Environment Measure (DREEM)—a generic instrument for measuring students’ perceptions of undergraduate health professions curricula, Medical Teacher 2015; 27:4, 322-325, DOI: 1080/01421590500151054
6. Cassar K. Development of an instrument to measure the surgical operating theatre learning environment as perceived by basic surgical trainees. Medical Teacher 2004 May;26(3):260-4. doi: 10.1080/0142159042000191975. PMID: 15203505.
7. Holt MC and Roff S. Development and validation of the Anaesthetic Theatre Educational Environment Measure (ATEEM). Medical Teacher 2004; 26, No. 6: 553–558.
8. Roff S, McAleer S and SkinnerA. Development and validation of an instrument to measure the postgraduate clinical learning and teaching educational environment for hospital-based junior doctors in the UK, Medical Teacher 2005; 27:4, 326-331, DOI: 10.1080/01421590500150874
9. Freudenberger HJ. Staff burnout. Journal of Social Issues;30:159-165.
10. Maslach C. Burned-out. Human Behaviour 1976; 5: 16-22.
11. Maslach C and Leiter MP. Understanding the burnout experience: Resent research and its implications for psychiatry. World Psychiatry 2016; 15(2): 103-111. https://doi.org/10.1002/wps.20311
12. Kristensen TS, Borritz M and Christensen KB. The Copenhagen Burnout Inventory: A new tool for the assessment of burnout. Work & Stress 2005; 19(3): 192-207.
13. General Medical Council. Burnout report. 2020. Available online at https://www.gmc-uk.org/help/education-data-reporting-tool-help/burnout-report Accessed 14th December 2020.
14. Rodrigues H, Cobucci R, Oliviera A, Cabral JV, Medeiros L, Gurgel K et al. Burnout syndrome among medical residents: A systematic review and meta-analysis. PLoS ONE 13(11); e0206840.
15. Ishaak WW, Lederer S, Mandili C, Nikravesh R, Seligman L, Vasa M et al. Burnout during Residency Training: A Literature Review. Journal of Graduate Medical Education 2009; 1(2): 236-242
16. Simpkin AL, Khan A, West DC, Garcia BM, Sectish TC, Spector ND et al. Stress from uncertainty and resilience among depressed and burned out residents over a 2-year period: a cross-sectional study. Academic Paediatrics 2018; 18(6): 698-704. doi: 10.1016/j.acap.2018.03.002. Epub 2018 Mar 7. PMID: 29524616.
17. Hakanen JJ and Schaufeli WB. Do burnout and work engagement predict depressive symptoms and life satisfaction? A three-wave seven-year prospective study. Journal of Affective Disorders 2012; 141: 415-424
18. Jackson ER, Shafelt RD, Hasan O, Satele D and Dyrbye LN. Burnout and alcohol abuse/dependence among US medical students. Academic Medicine 2016; 91(9): 1251-1256.
19. Prins JT, Van Der Heijden FM, Hoekstra-Weebers JE, Bakker AB, Van der Wiel HBM, Jacobs B et al. Burnout, engagement and resident physicians’ self-reported errors. Health Manag. 2009; 14: 654-666
20. Van der Heijden F, Dilling G, Bakker A, and Prins J. Suicidal thoughts among medical residents with burnout. Arch Suicide Res. 2008; 12:344-346
21. Puranitee P, Stevens FFCJ, Pakakasama S, Plitponkarnpim A, Arj-OngVallibhakara, Busari JO et al. Exploring burnout and association with the educational climate in pediatric residents in Thailand. BMC Medical Education 2019; 19:245-25.
22. Llera J and Durante E. Correlation between the educational environment and burn-out syndrome in residency programs at a university hospital. Arch Argent Pediatr 2014; 112(1):6-11
23. Van Vendeloo SN, Prins DJ, Verheyen CCPM, Prins JT, van den Heijkant F, van der Heijden FMMA et al. The learning environment and resident burnout: a national study. Perspect Med Educ 2018; 7(2):120-125.
24. Fernando BMS and Samaranayake DL. Burnout among postgraduate doctors in Colombo: prevalence, associated factors and association with self-reported patient care. BMC Medical Education. 2019; 19:373-384.
25. Camilleri R: Stress and Coping in Junior Doctors. Unpublished Bachelor’s degree dissertation, University of Malta, 2006.
26. Taylor-East R, Grech A and Gatt C. The Mental Health of Newly Graduated Doctors in Malta. Psychiatria Danubina 2013; Vol. 25, Suppl. 2:250-255.
27. Papaefstathiou E, Tsounis A, Papaefstathiou E, Malliarou M, Sergentanis T and Sarafis P. Impact of hospital educational environment and occupational stress on burnout among Greek medical residents. BMC Research Notes 2019; 12:281-287.
28. Salpigktidis I, Paliaouras D, Gogakos AS, Rallis T, Schizas NC, Chatznikolaou F et al. Burnout syndrome and job satisfaction in Greek residents: exploring differences between trainees inside and outside the country.  Ann Transl Med 2016; 4:444
29. William W, Lederer S, Mandili C, Nikravesh R et al. Burnout during residency training. Journal of Graduate Medical Education 2009; 236-242.
30. McCray L, Cronholm P, Bogner H, Gallo J and Neill R. Resident Physician Burnout: Is There Hope? Family Medicine 2008; 40(9): 626-632.
31. Dyrbye L , Thomas M, Harper W, Massie SF, et al. The learning environment and medical student burnout: a multicentre study. Medical Education 2009; 43: 274-282.
32. Zis P, Anagnoustopoulos F and Sykioti P. Burnout in medical residents: a study based on the job demands – resources model. Scientific World Journal 2014; 2014:673279.
33. Eckelberry J, Lick D, Boura J, Hunt R, et al. An exploratory study of resident burnout and wellness. Academic Medicine 2009; 2: 269-277.

Bakker AB and Demerouti E. Job demands-resources theory: taking stock and looking forward. J Occup Health Psychol. 2016; 22(3):273-285.

Figure

Test image