Assertive Outreach Teams
Listening to practitioners, and placing the resources where they can make a real impact.
A Review of Seven Practice Development Programmes
Part 3: Evaluation a Strengths Approach to Practice Development
Developing a tool to guide and capture the strengths messages
The rationale underpinning a strengths approach led to the development of a 13- item tool to help capture some measurement of how practice development and team functioning can influence positive changes in team practice. The tool should also act as a prompt to areas of good practice, so the items are designed to be a sufficiently comprehensive reflection of ethical issues, care process, team functioning, and an underpinning knowledge base. Yet, this also needs to be a manageable tool (i.e. fitting one side of paper) if it is to engage practitioners interests and be used.
-
Ethical practice
- Service users are fully involved in determining the ‘priorities’ for the working relationship.
- As a team, we regularly examine and review the impact of ‘values and principles’ on our practice. Care process
- Time for creative approaches to ‘engagement’ is a priority.
- Our assessment of needs includes the identification of service user ‘strengths’.
- We identify and manage the appropriate opportunities for ‘positive risk-taking’ in practice.
- Our working practice draws on a broad range of ‘practical and research based’ approaches. Team working
- We are clear about our ‘purpose’ as an assertive outreach team.
- We have a clearly agreed model of ‘team-working’ (within team).
- We have effective systems of ‘support and supervision’.
- We have clear processes of ‘decision-making’ in the team.
- We ‘link’ effectively with other parts of the mental health system (including primary care). Knowledge in practice
- We make full use of the ‘diverse’ knowledge, skill and experience within the team.
- We access appropriate expertise from outside of the team.
Developing the method of evaluation
The method focuses on evaluation of changes in practice that gives useful feedback to the teams about their global functioning, as well as indicators to priorities they may choose to focus specific attention on. This is not the same as the approach established in many of the well known research projects examining the effectiveness of assertive outreach. The research approach has been one of top-down influences, whereby external sources examine questions relating to their desire to compare outcomes to other known and established research. The evaluation approach is a bottom-up influence, whereby the practitioners are engaged in reflecting on how their team is performing in relation to practice-based outcomes.
Each individual within the team is asked to evaluate ‘the team’ on a 1-5 Likert Scale across each of the 13 items in the evaluation tool. The responses at a given point in time are amalgamated to give a ‘whole team’ mean score response for each of the 13 items (not identifying which individual said what). For each of the teams this process is repeated at reasonable intervals (between 6 and 12 months). Of the seven teams, five completed the evaluation at 3 points, and two completed it at 2 points in time. The timescales for the evaluations and the numbers of practitioners responding on each occasion are listed below in Table 1.
- When amalgamating all seven team’s results together, a total of 74 practitioners completed the first point of evaluation; 72 completed the tool at the second point; and 59 practitioners provided a third response.
- Two of the teams did not reach a third point of evaluation because the length of the practice development contracts did not permit sufficient time.
Evaluation results
The following figures do not identify changes in the separate teams. Table 2 shows mean scores as calculated by amalgamating the results of 7 teams at the first and second points of evaluation, and the relevant 5 teams at the third point. They also show the ranked order, out of 13, that the items achieved at each point of evaluation.
- The overall mean scores at the base of the 3 columns show an upward trend in the scores across the three points of evaluation, from 3.42>3.77 > 3.80. This indicates that a focus on practice development can enable practitioners to spend more time reflecting on and developing their practice as an on-going function of their work, with positive results.
- The shift in ‘identifying strengths’ (item 4), up to overall 1st at the second point of evaluation, but slipping back to overall 4th position indicates this item to be consistently evaluated high. However, a few practitioners did reflect that the second point of evaluation indicated a degree of over-confidence in how they thought they were doing in relation to this aspect of their work; but the detailed analysis through practice development helped them achieve a more realistic reflection at the third point.
- Working with service user priorities (item 1) demonstrates a steady improvement in relative ranking from 8th > equal 6th > 5th, and a significant improvement in mean scores from a baseline of 3.41 at the 1st point of evaluation to 3.92 at the 3rd point.
- ‘Positive risk-taking’ (item 5) demonstrates a steady progression in its overall ranking from equal 5th > 3rd > 1st indicating that practice development support through a strengths approach has contributed to increasing the confidence of practitioners in this challenging area of practice.
- Reflection on ‘values and principles’ (item 2) has remained relatively low in rank order (12 > 12 > 10), but the improvement in mean scores from 3.15 > 3.73 implies that practice development has raised the awareness and significance of this item within individual and team practice.
- ‘Clarity of assertive outreach purpose’ (item 7) has remained very high in relative rank order (2 > equal 4 > 2), but the overall mean scores demonstrate a significant improvement from 3.66 > 4.03. However, the ‘model of team-working’ (item 8) continues to show modest results with changes in rank order (7 > equal 6 > 11), with improvements in overall mean scores from 3.50 > 3.67; this may partly be reflecting the differences in team-working models that persist across the 7 teams.
- The ‘knowledge’ and ‘skill’ elements of the evaluation (items 12 & 13) are modestly ranked in the mid-range of all 13 items, but both demonstrate significant improvements in their overall mean scores: 3.61 > 3.97 for using knowledge within the team, and 3.31 > 3.85 for accessing external expertise.
In Table 3 the results also identify the percentage shifts of opinion across the different points of evaluation.
- All 13 items demonstrate positive opinion shifts between the 1st and 2nd points of evaluation, but 6 of the items then demonstrate relatively small backward slides between the 2nd and 3rd points of evaluation.
- This does not present any cause for concern within teams, as it is partly a reflection that very large initial positive shifts could not be followed by further moves in a positive direction; and some of the negative changes were described by the practitioners within teams as a readjustment in the form of a reality check on what they were actually doing in practice. Some of the percentage opinion shifts within individual teams were as large as 35%.
- The important differences are those recorded between the initial baseline 1st point and the 3rd point of evaluation. In this respect, the final column in Table 3 demonstrates that whatever interim readjustments may have been made, the practitioners across the board have recorded positive opinion shifts for all 13 items as a result of direct practice development input, and the impact it has on increasing the priority for individual and team reflection on practice.
Table 4 identifies the items in rank order by the overall percentage opinion shifts between the 1st and 3rd points of evaluation. It also includes the overall mean score for each item at the 3rd point.
- The two items with the largest opinion shifts (items 6 & 2) started with a low baseline evaluation initially. So, they still remain 7th and 10th in overall ranking at the third point of evaluation. However, the ‘values and principles’ (item 2) clearly has benefited from the affect of increasing the focus on reflective practice.
- ‘Identifying strengths’ (item 4) appears comparatively low in this table, but started from a relatively high baseline and retained a relatively high mean score. However, many practitioners did comment on how easy it can be to slip out of a strengths mode of thinking when attention is drawn to other priorities or crises.
- ‘Service user priorities’ (item 1) records the 5th highest percentage improvement of 10.2%. It also rates the 5th highest ranking at the third point of evaluation, suggesting that a strengths approach can have a positive impact on refocusing attention onto the things that service users want to achieve for themselves.
- ‘Positive risk-taking’, with a strong basis in strengths working, is the one item that manages to combine a start from a relatively high baseline and a relatively high percentage opinion shift, to finish as the highest ranked item, and one of only two items to rate a mean score of >4 (out of 5) across the third point of evaluation for all participating teams.
- ‘Clarity of decision-making’ and ‘Links with other parts of the system’ (items 10 & 11) are both ranked relatively low and score relatively lower on opinion shift. Achieving mean scores at the third point of 3.31 and 3.36 respectively is not evidence of poor functioning, just relatively low within the 13 identified items. Specific attention was not given to these during the practice development initiatives, but the implication is that they benefited less from an anticipated ‘ripple effect’ across all areas of functioning.
Note: ‘significance’ in the above commentary is the author’s subjective use of the term, based also on anecdotal feedback from individual practitioners; it is not ‘statistical significance’, as no statistical analysis has been run on this data.