Key resources for evaluation of workforce change programs and interventions
Click on the headings below to read more.
Workforce change – the concept
Despite the benefits of family-sensitive approaches, family support (particularly for children) through programs or interventions are not routinely offered in mental health services. Research has shown that mental health workers rarely acknowledge the children of adults who experience mental illness or their parenting responsibilities. This trend is also reflected internationally.
Literature and research on the range of innovations and evidence-based practices in health care settings have clarified barriers to the uptake of family interventions (see GEMS research summary edition 6 – Issues for mental health workers when wrking with children and parents). Key issues that need to be addressed in order to include children and families in programs or interventions are:
- policy and management
- interagency collaboration
- deficiencies in worker attitudes, skills and knowledge.
Few studies have been undertaken in the area of workforce change, so there is little evidence to illustrate what works and what doesn’t work in making workforce practice more family-focused, centred or friendly.
Theoretical background
Kylie Eddy recently reviewed the core training components of COPMI workforce development packages. She found that the training focused on:
- the impact of parental mental illness on children and families
- working and communicating with families using strengths-based approaches
- resources and programs available for young people and their families
- identifying the needs of local areas
- planning for collaboration in service delivery (Whitman et al., 2009).
A number of studies have now shown that many workers have skill and knowledge deficits in regard to working with:
- Children
- Parents with a mental illness
- The family as a unit (Maybery & Reupert, 2009; Dean & Macmillan, 2001; Korhonen et al., 2008; Bibou-Nakou, 2003)
A recent Delphi study sought opinions from experts in the COPMI workforce training area about what modules should be include in a training package (Whitham, Eddy, Maybery, Reupert & Fudge, 2009). Column one (Modules) in the table below highlights seven core modules, including child development and information about carers.
Rankings (and overall average rank) of modules considered of greatest ‘learning need’ for different mental health professionals (modified from Whitman et al., 2009).
| Modules | GPs | OTs | Social workers | MH nurses | Psychiatrists | Psychologists | Average ranking |
| Mental illness | 2 | 1 | 1 | 6 | 7 | 5 | 3.56 |
| Child development | 5 | 2 | 3 | 2 | 3 | 3 | 3.00 |
| Parenting | 3 | 3 | 2 | 1 | 1 | 1 | 1.83 |
| Families | 4 | 5 | 6 | 6 | 4 | 4 | 4.83 |
|
Supporting families |
1 | 4 | 5 | 3 | 2 | 2 | 2.83 |
| Carers | 7 | 7 | 6 | 5 | 6 | 7 | 6.33 |
| Roles and responsibilities | 6 | 6 | 4 | 4 | 5 | 6 | 5.17 |
The table also shows experts’ ranking of the most important training needs of various professions (1 being the most important). Mental health practitioners (eg mental health nurses, psychiatrists and psychologists) mainly needed training about parenting and family needs, while non mental health practitioners (eg GPs, OTs and social workers) needed training about mental illness. This information provides a useful theoretical backdrop to the area of workforce training.
Measures of workforce change
A number of instruments measure workforce change. However, as far as is known at the time of writing, few studies have detailed the psychometric properties of these measures.
Preventive Child-Focused Family Work (PCF-FW) questionnaire
The PCF-FW is one of the more rigorously developed measures. Korhonen and colleagues (2009) have published details of the development of their PCF-FW questionnaire for adult psychiatric settings. The 133-item instrument was ‘developed to measure support for parenting, dependent children and family relationships in order to promote child development and mental health in adult psychiatry’.
Structure of the measure
The PCF-FW focuses upon:
- Background information
- Gathering information about the family
- Planning and implementing family meetings
- Support for the family’s support network
- Support for parenthood
- Support for children
- Limiting issues for family work
Construct validity
The measure was developed in conjunction with reviews from experts in the field and with registered and psychiatric nurses, and is reported to have strong content and construct validity.
Reliability
While Cronbach’s alpha coefficients are reported between 0.55-0.95 for subscales, only the one subscale was below .70 with many showing very good to excellent reliability.
Evaluation and/or research that has used this measure
Korhonen et al. (2009) report a cross-sectional study using the measure.
Availability of the scale
Please contact the author, email: Teija.Korhonen@uku.fi, regarding availability.
Family focused mental health practice questionnaire (Maybery, Goodyear & Reupert, 2010)
This questionnaire is a collaboration of Maybery, Goodyear, Reupert, Victorian FaPMI coordinators and Dr. Rob Lees and colleagues from British Columbia, Canada. The family focused mental health practice questionnaire has been developed over seven years and is designed for use in adult mental health settings. The measure is based upon a review of workforce barriers (Maybery & Reupert, 2009) and focuses upon policy, confidence, and skills and knowledge of workers working with families where a parent has a mental illness.
There are two versions of the measure. The main measure has 49 items (16 subscales) and a shorter version has 33-items (eight of the 16 subscales). The short version of the measure is being used to evaluate the COPMI-developed Keeping Families and Children in Mind eLearning course.
Construct validity
The 49 item measure was developed in conjunction with reviews from experts in the field and is considered to have good face validity however construct validity has not been established at this point.
Reliability
Cronbach’s alpha coefficients are reported between 0.70-0.90 for most of the subscales (also see pdf under availability of the scale below).
Evaluation and/or research that has used this measure
Australian and Canadian studies including almost 600 participants have been undertaken using this measure. For details email: darryl.maybery@monash.edu.au.
Family focused mental health practice questionnaire
This 49-item measure includes the sixteen subscales: Workplace Support, Location issues, Time and workload, Policy and procedures, Professional development, Co worker support, Family and parenting support, Worker confidence, Support to carers and children, Engagement issues, Assessing the impact on the child, Training, Skill and knowledge, Service availability, Connectedness and Referrals.
Definitions of each of these concepts are available with the measure.
The measure takes between 10 and 20 minutes to complete and participants respond on a seven point scale from 1 (strongly disagree) to 7 (strongly agree). Participants also have the option of responding that the item is not applicable.
Evaluation measure
This 33-item evaluation measure includes eight subscales:
- Support to carers and children
- Engagement issues
- Interprofessional collaboration
- Skill and knowledge
- Assessing and responding
- Family and parenting support
- Worker confidence
- Connectedness
This version also asks for information about participant background, the type of training being undertaken, and knowledge and practices involving families with a parent with a mental illness. As indicated above this version of the measure is being used to evaluate the Keeping Families and Children in Mind eLearning course.
Availability of the measure
A copy of the ‘Family focused mental health practice questionnaire‘ is available for downloading.
Evaluation measure
Please contact the author regarding availability: darryl.maybery@monash.edu.au
Other measures
A number of untested measures of COPMI workforce activities are in development or piloting phases. One example is an evaluation instrument developed by Brooks and colleagues from the NSW Institute of Psychiatry which aims to evaluate the Crossing Bridges package.
COPMI programs and interventions
Key readings
GEMS research summary edition 6: This GEM provides a brief summary on ‘Issues for mental health workers when working with children and parents’.
Conway, J., McMillan, M. & Becker, J. (2006). Implementing workforce development in health care: a conceptual framework to guide and evaluate health service reform. Human Resource Development International, 9, 129-39.
Korhonen, T., Vehviläinen-Julkunen, K. & Pietilä, A. (2008). Implementing child-focused family nursing into routine adult psychiatric practice: hindering factors evaluated by nurses. Journal of Clinical Nursing, 17, 499-508.
Korhonen, T., Vehvilainen-Julkunen, K., Pietilä, A. & Kattainen, E. (2009). Preventive child-focused family work: development of instrument in adult psychiatry. Journal of Psychiatry and Mental Health Nursing, 16, 9, November, 804-12.
Maybery, D. & Reupert, A. (2006). Workforce capacity to respond to children whose parents have a mental illness. Australian and New Zealand Journal of Psychiatry, 40, 657-64.
Maybery, D. & Reupert, A. (2009). Parental mental illness: a review of barriers and issues for working with families and children. Journal of Psychiatry and Mental Health Nursing, 16, 9, 784-91.
Slack, K. & Webber, M. (2008). Do we care? Adult mental health professionals’ attitudes towards supporting service users’ children. Child Family Social Work, 13, 72-9.
Other references
Bernheim K. F. & Switalski, T. (1988). Mental health staff and patient’s relatives: how they view each other. Hospital and Community Psychiatry, 39, January, 63-8.
Bibou-Nakou, I. (2003). ‘Troubles Talk’ among professionals working with families facing parental mental illness. Journal of Family Studies, 9, 2, 248-66.
Dean, C. & Macmillan, C. (2001). Serving the children of parents with a mental illness: barriers, break-throughs and benefits. Australian Infant, Child, Adolescent and Family Mental Health Association, 4th National Conference.
Whitham, Eddy, Maybery, Reupert & Fudge. 2009. Use of a web-based delphi study in the development of a training resources for workers supporting families where parents experience mental illness. International Journal of Mental Health Promotion, 11, 2 42-54.