Relationship-based practice evaluation framework appendix 2

Food Matters is a national charity dedicated to empowering people. We support communities and organisations to develop the skills, knowledge and capacity to make inclusive, representative and transparent decisions. Food Matters is supporting and facilitating a process to develop a framework for evaluation of the Relationship-Based Practice (RBP) model of social work adopted by East Sussex County Council and Brighton & Hove City Council Children’s Services. 

Food Matters has wide experience and well-developed expertise for innovating and championing participatory approaches to facilitated evaluation processes, particularly in the food sector but not exclusively so.  Our approach is based on the use of Participatory Appraisal (PA) in a range of different circumstances and with diverse groups and organisations because we know that lasting and meaningful change is more likely when the people who will be affected by a change participate in deciding what the change should be.    

The project aim is to use co-production, with care-experienced young people, social work practitioners, student social workers and parents, to design a new evaluation framework. The aim is that the evaluation framework will be meaningful for the families they work with and will develop better understanding of how social work and more specifically Relationship-based practice is creating positive change.  

Food Matters’ experience tells us that to deliver meaningful and useful evaluations we need to maximise inclusion and participation. We do this by designing workshops around participants and include a combination of activities aimed at facilitating participation, the sharing of opinions and perspectives and engagement in a sequence of questions. The workshops use a range of accessible, interactive and visual participatory tools and exercises that facilitate engagement in a mutually respectful and non-judgemental manner.    

A core principle of Food Matters’ participatory research and evaluation work is to ensure that the processes we use are two-way, meaning the co-production process benefits the participants beyond the workshop. In the context of this work, we will develop a workshop programme that encourages participants to share experiences and learn from each other.    

• The two co-production workshops were held at the Linklater Pavilion in Lewes on 13 and 16 February 2023.  
• Workshop participants included care-experienced young people from both Brighton & Hove and East Sussex alongside social workers, youth workers and student social workers from both local authorities.
• The workshops were designed and facilitated by Callum Etches, Izzy Hutcheson-Lovett, Alex Ward and Ben Messer from Food Matters. 

• Assessing the impact of RBP by focusing on the relationship that develops between young people and their social workers or care practitioners
• RBP evaluation framework focuses on developing processes to help young people:
  • review and assess their relationships with social workers and care providers
  • acknowledge the impact the relationships have on their experience of care and support 
  • determine what, how, where, when and by whom evaluation should be undertaken 

• Participatory processes for the co-production – between care-experienced young people and social workers – of reflection and assessment tools 

Processes aim to: 

• structure conversations about care experience with an emphasis on the relationships that develop between young people and social workers/care practitioners
• determine a range of indicators of positive relationships
• explore how the indicators can be used in different ways as criteria against which relationships can be assessed
• identify appropriate methods for engaging young people in evaluation 

• Participatory approaches to develop the evaluation framework
• Establish supportive, safe and non-judgemental environments for the work
• Clarity about the purpose of the work, participant anonymity and how outcomes may be used 
• 2 separate facilitated, participatory sessions
• Sessions will support care-experienced young people and social workers to collaborate and co-produce relationship evaluation indicators and creative ways in which they can be used to evaluate the RBP approach 

WHAT do young people think they should be asked?

Indicators of good RBP.

HOW do they think they should be asked?

Tools and approaches to use.

WHEN and where is it best to do this?

When, how often and in what environment.

WHO should be carrying out the evaluation?

Support with the evaluation process 

Welcome and introductions 

The first half of Session 1 (on 13 February) established a safe environment and a sensitive and supportive approach to the co-production process. The purpose and aims of the process were clearly explained to participants – care-experienced young people and care practitioners (social workers, youth workers and student social workers). The session facilitators and participants introduced themselves to each other and shared hopes and expectations for the co-production sessions.   

Experiences 

The second half of Session 1 focused on developing an understanding of the participants’ experiences of relationships between young people and social workers (care practitioners, care professionals). This was facilitated using a continuum or line exercise to evaluate and share experiences of relationships – from the point of view of both the young people and the care professionals – and to determine key factors or indicators of what makes supportive or less supportive relationships. 

Indicators 

The key output of Session 1 was a list of indicators (or criteria) against which relationships could be evaluated.  These indicators provide the basis for the RBP evaluation framework with care-experienced young people by responding to the first Key question – WHAT do young people think they should be asked? 

The indicators against which RBP can be evaluated are expressed in this initial draft list as positive statements of what a supportive relationship is. The list is refined and ranked later in this report. 

A supportive relationship...

• is built on mutual trust
• is where I feel listened to, understood and heard
• is where I feel able and encouraged to be an individual and express myself
• is where I know something about the people supporting me
• is where I feel support is tailored to my needs
• is where I can access support when I need it
• is one that supports me to be treated as equal to others
• is one that makes me feel safe
• is one that empowers me and enables me to make decisions for myself
• is where private things about my life are only shared with others when really needed
• is where I understand what information has to be shared with others and why
• requires people to be as honest as they can be to each other
• requires creativity and flexibility
• requires two-way communication and sharing on both sides
• is one that gives me some choice in the decisions that affect me
• is where my feedback is valued and leads to change
• is where I am properly looked after by my foster carer 
• is one where I feel respected
• is where someone speaks up for me and advocates on my behalf
• is when the reasons for decisions that affect me are clearly explained
• is one that is as transparent as possible 

Confirming and prioritising the indicators

The second session (on 16 February) began by confirming and prioritising the draft list of indicators.  Participants used a dot voting exercise to indicate their agreement with the statements somewhere on the scale between ‘strongly agree’ and ‘strongly disagree’. 

Young people used a sticky star.

Care professionals used an orange circle to indicate their individual scores for each indicator. 

Participants then discussed the scoring patterns and amended indicator statements where necessary.

Key comments

The young people commented on how they felt that many of these indicators of a supportive relationship are not currently being met.

Participants acknowledged that many of these indicators are connected, for example, respect is built on honesty, mutual responsibility and trust.

While participants agreed with all statements listed, it was highlighted that there needs to be clear examples of what building these indicators looks like in practice:

• many young people felt that they could only provide examples of negatives, as they felt some of the positives have yet to be modelled or experienced.  

Participants acknowledged that these indicators could mean different things to different people:

• for example, ensuring safety may mean different things from the perspective of a young person and a social worker.  

Participants acknowledged that some of these indicators are age dependent:

• for example, involvement in decision making may not be suitable for younger age groups

Participants highlighted the need for social workers to be proactively involved in managing the other care relationships of young people:

• for example, the experience of many of the young people was that their social worker was not ensuring that their foster carer was supporting them appropriately. 
• young people also emphasised the need for social workers to help ensure that their relationships in schooling were appropriately managed. Many voiced that they felt ‘singled out’ in the schooling system, and that this led to them being treated differently by teachers. 
• many social workers were unaware of these schooling dynamics and agreed that they should take a proactive role in understanding and addressing these challenges.  

While both social workers and young people agreed that flexibility and creativity were important aspects of a supportive care relationship, the social workers voiced their concern for their lack of ability to be flexibility within local authority processes. Social workers voiced that they were often not able to make decision themselves and that this has made it more difficult to built supportive relationships.  

While social workers agreed with all statements, they voiced concern about how the administrative elements of their work left little time to build supportive relationships with young people:

• one social worker commented: “I write about children more than I see them” 

The final stage of the confirmation process involved each participant voting for their TOP 5 most important indicators.  The draft indicators were amended based on participants’ comments and the outcome of the scoring is shown below in this finalised and ranked indicators list.

Top 5 most important indicators

Following the session the list of indicators has be clustered into themes for the purpose of developing Feedback Dial tools. 

The clustering is shown in Appendix B.  

Having spent time determining WHAT should be asked in an evaluation the remaining time in Session 2 focused on the remaining key questions: HOW, WHEN, WHERE and WHO.   

Three ‘Islands’ – work tables – were set up around the room and participants, either individually or in small groups, were encouraged to visit each island. At each island a facilitator asked participants to consider the way evaluation is carried out currently and how it could be in the future.   

Conversations and discussions were documented by participants and facilitators on flip-charts. The transcribed comments are in Appendix A. 

It is important to note that responses to the questions and suggestions for the way feedback could work was varied and often contradictory. This highlights the need for the development of a feedback framework and process that is flexible and where possible responsive to the needs of individual young people. The full transcription of comments is given in Appendix A. 

How 

• Use a flexible approach – not ‘one size fits all’
• Need to balance the value in engaging in conversations during the feedback with the intensity that is felt by the young person – ‘being put on the spot is not helpful’
• Less intense if the young person can consider the questions and complete the feedback on their own and in their own time
• The potential for a digital and online feedback should be considered
• Feedback Dials should be explored because they: 
  • allow easy visual comparison of feedback given at different times
  • are useful as a basis for conversation
  • could be used to set targets between feedback sessions
  • are better completed digitally but print-outs may be easier for younger children 

The idea of feedback dials based on the indicators was suggested as an option to be explored and some young people and social workers had used similar tools before.  A draft Feedback Dial tool has been included alongside this report. 

When 

• As part of an ongoing process: ‘so it’s not a shock…’ 
• At regular points in time: ‘properly organised and scheduled (not just turning up)’
• Flexible and responsive to the young person’s needs
• Although regular the feedback should not be too often – perhaps every 6 months or once every year 

Where 

• Listen to the young person’s preference
• In the young person’s own space
• Online – on their own ‘phone 

Who 

• Listen to the young person’s preference: ‘You should get a choice who you do it with’
• Someone trusted: ‘Someone I trust and have a good relationship with’
• Someone objective and not a social worker, for example, Independent Visitors
• Apps: 
  • Better as more young people communicate this way
  • Quicker
  • Not biased by carer 

Whilst some young people were very keen on using an app based on the indicators to provide feedback others were more interested in feedback in-person based on casual conversation around the indicators. 

Other comments 

A suggestion from young people was for other young people to be involved in the feedback process, perhaps through buddy schemes: ‘we (the children + YP) should get feedback from other children and YP’ 

Some young people were open to giving feedback in group settings ‘depending on who is in the group’ while others preferred a 1-to-1 setting where ‘my words can be better understood/not twisted’ 

Several participants expressed doubt that anonymous feedback was possible as information or opinions shared may identify them 

This sentiment was shared by a social worker, noting that feedback on social work needs to identify the staff member in question, in order to meaningfully address issues: ‘can feedback lead to meaningful action if it is anonymous? 

1. Criteria for the RBP evaluation should be built around the key indicators of a ‘supportive relationship’ identified by participants.
2. Feedback methods need to be varied to meet different needs of participants and provide different options, including: 
   • The basis for informal conversations
   • Feedback Dials
   • Scoring forms showing strength of agreement
   • Digital versions of forms and dials
   • Option of anonymity
   • Possible group sessions as well as 1-to-1 feedback
3. For some young people honest feedback is easier to give to a trusted person not directly involved in providing care e.g. Independent Visitors
4. More work needs to be undertaken to provide practical examples of the actions and behaviours that contribute to the indicators of a ‘supportive relationship’. 
5. Young people need to be consulted again once an approach is developed. 
6. All outputs from the co-development process need to be shared with the young people who have been involved.  

The 4 facilitators from Food Matters who were involved in these co-production sessions would like to express our sincere gratitude to the participants – the young people, and social work professionals. Throughout both sessions participants engaged with the activities and responded to the questions honestly and energetically. Without this the sessions would not have been so successful and productive and we would like to say thank you. 

We would also like to thank the staff from East Sussex County Council and Brighton & Hove City Council social services who organised the sessions as well as participating when required. 

Ben Messer, Callum Etches, Izzy Hutcheson-Lovett and Alex Ward 

Food Matters       

24 February 2023 

Transcribed comments from the work islands  

HOW do young people think they should be asked? Tools and approaches

Now: How have you been asked to give feedback up until now? 

• Forms – but they don’t capture my experience or change anything
• In a form – boring
• Forms – can do it independently when you want
• Tick-box and scaling 1 to 10
• Smiley faces don’t involve giving a score
• Letters – emails – questionnaires
• Simplicity of the wording – childish – needs to be age appropriate
• Form – too much writing – don’t want to take up whole day
• Being put on the spot is not helpful
• Not asked for feedback – need to voice my opinions – more of an effort made to hear my voice
• East Sussex activity week to seek views in half term 

Future: How would you use the indicators of a supportive relationship? 

• Different ways depending on the situation
• Focus more on conversations – better than forms and writing
• Be flexible
• Provide distractions/games to make conversations easier
• Better online – can consider what you think in your own time – anonymous
• Doing activities I enjoy with people I trust and having a natural conversation
• Depends on personal preference
• Ask: ‘is there anything you’d like to change or be done differently?’
• 360 degree feedback – allows SW being evaluated to respond to feedback that is given by YP: Questions – YP feedback – reply or thoughts on feedback – steps to be taken to improve or change
• Potential for using feedback dials:
  • Useful to compare dials over time
  • Better as a digital feedback tool than as a paper chart
  • Could set targets for how to score higher next time
  • Complete with someone external/objective – e.g. Independent Visitors (E.Sussex)
  • Can fill in dial sections – younger people – or create a star chart image
  • Could use for many different relationships
  • Better if anonymous 

WHEN is it best to do this? When and how often? 

Now: When are you currently asked to give feedback? 

• When the SW needs feedback
• Before meeting in a visit – 1 or 2 days before
• I don’t talk to anyone – I don’t trust anyone 

Less good: 

• Have to think on the spot
• SW asks in person – verbally – stays while you are filling in the form – awkward ‘looking over your shoulder’
• Too vague – ‘how are things going?’
• Don’t think I’ve ever given it
• Only given it once a year – would like it more often 

Better: 

• When I choose or need to – sometimes I have nothing to say
• Online:
  • In your own time
  • Tick box approach
  • Choice of when to fill it in
  • Keep it simple – don’t ask too much
  • Make it anonymous – makes more honesty possible 

Future: When would you like to be asked to give feedback? 

• Flexible approach to how often – would like to be given the option, but not longer than a year and not only when there is a big meeting
• As part of an ongoing process so its not a shock and we can co-create/co-review
• Once a year
• When its properly organised and scheduled (not just turning up)
• Regular points in time
• It shouldn’t take too long – 20 to 30 minutes 
• When something bad happens or goes wrong
• When things are not at crisis point – things are a bit calmer 

Where is it best to do this?  

Now: Where is feedback currently given? 

• Home or school – good: people were asking my opinion
  • home is my own space
  • school – nothing is private – other people just want to know everything 
• Feedback obtained in person – feel under pressure
• YP don’t get asked for feedback
• Small café: not good, very cramped, no choice, felt as though people were listening 

Future: Where would you like to give feedback? 

• Where the young person wants to go
• In your own space
• On my own phone
• Office or coffee shop
• On an App – the way YP communicate, quicker, un-biased by carer
• Library – quiet 
• Not in class
• In private and in person
• Out and about – foster placement – at the end of meetings 

Who should be carrying out the evaluation? Who should be doing it with you? 

Now: Who do you currently give feedback to? 

• Whoever I have a connection with and trust
• Youth workers, Foster carers, Independent visitors
• When I have a problem I tell my parents – I don’t wait until I’m asked
• LAC review meeting
• Issues – nothing ever changes
• I sometimes don’t give feedback 

Future: Who would you like to give feedback to in the future? 

• You should get a choice who you do it with
• Key worker – someone who is there 24/7
• Placement support
• Independent visitors
• Foster carer
• Someone I trust and have a good relationship with
• YP needs to know there’s someone they can tell if the SW breaks their trust – to feel safe
• I like the supervising social worker: 1 to 1 – they actually get stuff done
  • Group (feedback): 
  • can encourage discussion
• depends who’s in the group
• Young people getting feedback from other young people
• Buddy schemes for/with people with lived experience
• Anonymity: 
  • can feedback lead to meaningful action if anonymous?
  • If with a social worker can’t be anonymous
  • Anonymous chat-line
• Apps:
  • Better as more young people communicate this way
  • Quicker
  • Not biased by carer 

Themed clustering of indicators for the development of three Feedback Dials  

Dial A 

Key relationship principles (based on indicator scoring) 

How is your relationship with your social worker? 

• It is built on mutual trust
• It supports me to be treated as equal to others
• It makes me feel safe
• I feel listened to, understood and heard
• It is creative and flexible
• I feel respected
• It empowers me and enables me to make decisions for myself 

Dial B 

Information and communication  

How is information shared and used 

• I know something about the people supporting me
• Private things about my life are only shared with others when really needed
• I understand what information has to be shared with others and why
• People are as honest as they can be to each other
• Two-way communication and sharing on both sides
• Feedback is valued and leads to change
• The reasons for decisions that affect me are clearly explained
• As transparent as possible 

Dial C 

Individual growth and development 

• I feel able and encouraged to be an individual and express myself
• I am given some choice in the decisions that affect me
• Someone speaks up for me and advocates on my behalf 

Practicalities of support 

• Support is tailored to my needs
• I can access support when I need it
• I am properly looked after by my foster carer 

See the draft Feedback Dial Tool Excel document demonstrating the potential of this approach