Robert and I (Phil) arrived late on Sunday night into Mbeya and so we started our first morning with the Malila team. We took some time for introductions and to look through what we had planned for the week ahead.
Overall Plan
Our plan over the first couple of days is to:
Start by hearing some stories of what makes the team proud/excited about the work, getting grounded in the idea of stories of change!
Make a timeline of the past 1-2 years, to get a sense of context and history, and to start surfacing some of the key changes that they have seen
Make an actor map, to think about who plays a role in making SE happen and is affected by SE activities. This serves as a basis to start finding outcomes as we think about how these actors have changed, and helps us think about who we might need to talk to.
Start writing up outcomes - by this point we should have enough ideas about changes that have happened that we can start writing “outcomes statements”. We’ll need to explain the “discipline” of writing statements, and then work together to capture what we already know.
On Wednesday we are going out to a village to meet with people to hear about outcomes they are aware of, so on Tuesday we will also need to think through about the most appropriate ways we can do this with them.
On Thursday, we hope to have lots of rough outcomes from our work with the community that we will need to clean up and organise. We also plan to start doing some initial analysis - what patterns do we see in the outcomes we have collected so far. Who is changing? In what ways? Is this what we hoped for?
On Friday, we plan to review the week and think about how the experience of the week can inform the rest of the OH research, and start making further plans.
We asked the team what it is that they feel they/the CBO need to learn about in order to strengthen the work moving forward. While the team didn’t have a lot to add to the overall evaluation questions, there were quite a few questions about how we were going to appropriately engage the Malila language CBO. (Some of the team are members of the CBO themselves). They felt we would need to go over the road map with the other CBO members who were due to join us in the afternoon.
What are we wanting to learn about?
Unfortunately only one of three of the CBO members was able to join us for the afternoon session (a second was delayed in his travels and arrived just as we were finishing). We reviewed the roadmap for the week, emphasising that this week was a preparation to get the research going.
We first looked at trees! Inspired by Klaas de Vries video…
Outcome Harvesting is about looking for the fruit of our work, and seeing - like a good farmer - what do we need to do to nurture the ‘seed’ (the things we have done - materials, translations, trainings etc.) that has been planted. We lifted four scenarios we might find:
Lots of fruit on the tree! Celebrate and thank God, and learn about what made this tree so healthy, and how can we help it to keep bearing fruit.
Small seedlings - initial signs of change, how do we encourage them to grow?
Tree with no fruit - places where we expected to see change, but didn’t find any? Why hasn’t the fruit grown? Does it need more time, or a different strategy?
Unhealthy “bad” fruit - are these unintended consequences of our work? If so, we will need to rethink strategies or review our own behaviour, possibly asking for forgiveness, and looking at how we ourselves need to change.
We also looked at a simple “theory of change” for SE (also being developed by Klaas and others), to start to think about what kinds of changes might we be looking for:
Overall, the big dream is to see communities transformed by God’s word….
…This will happen as individuals' hearts are changed, and they change their behaviour and relationships. …
…This is supported by people hearing, and seeing God’s word in their own language…
…Which is made possible by people having access to materials in the MT (Reach) - who is helping to distribute the materials? How is this showing signs of sustainability beyond the project's own actions?
… of course these Materials need to be produced …. but what signs are there that more people are becoming involved in producing MT materials that can support SE (e.g. songs, bible studies ) and that the capacity to produce materials (and translation) is growing.
… and all these things need to be supported by the legs! The relationships, cooperation and collaboration needed to support SE. Who is taking on new roles? What new relationships have formed? What new forms of cooperation and collaboration (e.g. CBO) has been established?
We might see some early signs of transformation, and some stories about individual changes, which we will need to try to understand more deeply. A lot of “impact stories” are often about people being moved and encouraged by hearing / reading scripture in the MT, so we will probably hear these stories, which will affirm the work that has been done. But the “lower” level stories are also important, and can potentially give us some of the best insights into how the work needs to move forward in the future, especially as we think about whose role is it in the long term (i.e. a sustainable SE ‘system’ of actors).
Discussing the research with CBO members
As the team anticipated, we then had a lot of questions about why we were doing the research from the CBO director, so we prioritised talking these through in the afternoon, and though we didn’t make much progress on our “road map” we hopefully built some trust and understanding with the CBO.
One of his concerns was whether the research was being used to determine whether the language community warrants having OT translation (which is the CBOs priority). There was also concern about previous research that was done, and has never come back to the community to support them (and had therefore undermined the value for him of “research” - possibly a negative outcome!)
The challenges we faced today are a reminder of how important it is to think through about who needs to be involved, and to make sure that discussions with teams and partners explores their interests and concerns. There is power in conducting evaluation and research which needs to be wielded carefully! This makes me wonder if we as LTT could have helped the team to prepare better - at the same time, some of these issues only surface once conversations start ‘for real’ and so we need to make sure we plan in enough time to work through any potential issues!
Inspiring Stories
We finished the day off by asking about the scripture engagement stories which made the participants proud or excited! These help us to get tuned into stories, and start to get an insight into the outcomes we might find. Here is a quick summary of the things that I was able to pick up!
At the 2011 dedication of Mark, many people came, including kids, to buy the books - there was a real thirst!
The pastors shared testimonies of how they were preaching in Malila, and people hearing in the MT understand better than in Swahili.
When they had MLE in preschools, they saw how the children were able to learn much better, and there was a lot of support from the communities.
Former pre-school teachers are now using MLE methods in government schools.
In the last couple years, a person in Dar has been distributing books to the Malila diaspora.
Recorded Scriptures are available in music libraries.
The Language Committee has registered as an NGO.
Comments
Post a Comment