Reconstructing a Program Theory. Midterm Evaluation of the Nzotheka Project by Theatre for a Change
Sex work is heavily stigmatized and discriminated against in Malawi. Police, health workers and sex workers themselves are not aware of the laws and policies that protect sex workers. Furthermore sex workers are more exposed to sexually transmitted diseases including HIV/AIDS due to pervasive low sexual and reproductive health.
In 2015 Theatre for a Change, through funding from Comic Relief and in partnership with the National Association for People Living with HIV and AIDS in Malawi (NAPHAM), SOS Children’s Village, the Malawi Red Cross, Chisomo Children’s Club, and the Microloan Foundation, began implementing the Nzotheka Project. The Nzotheka project aims to improve the sexual and reproductive health of women in sex work (WSW), and girls who have been sexually exploited (GBSEs).
Scope of Work
The External Evaluation of the Nzotheka Project aimed to report progress against project logframe indicators, identify when and how much change has occurred against project outcomes and the extent to which these changes can be attributed to the project.
In order to achieve these objectives, the External Evaluation utilizes Contribution Analysis as its primary evaluation approach. Contribution Analysis is an approach for assessing causal questions and inferring causality in real-life program evaluations.
At Midterm, the study aimed to construct a credible programme theory (or contribution story), based on a shared consensus of what makes the project work or not, and appraise whether this contribution story is likely to result in project impact. At Endline, the evaluation will gather existing quantitative and qualitative evidence, assemble it in relation to outcomes achieved and provide a final judgement on the extent to which project activities likely led to project impact.
Method
To construct a credible programme theory, the Midterm Study systematically reviewed each project outcome and developed distinct results chains from intervention activities. In order to assess the plausibility of whether these results chains will deliver impact, the study appraised each results chain against criteria of relevance, internal consistency, and external plausibility.
As part of the theory of change (TOC) building exercise, we organized two Performance Story Workshops with project stakeholders. The first workshop was a 1-day workshop attended by the Project Team. The second workshop was a 2-day workshop attended by WSW, Umodzi Network members, Mobile Health Clinic Staff, Police Peer Facilitators, and representatives of the Lilongwe District Social Welfare Office, NAPHAM, and Malawi Red Cross.
In order to assess and further develop the consensus theory of change based on criteria of relevance and internal consistency, the Baseline Study conducted key informant interviews and focus groups with relevant stakeholders.
Insights
Many Women in Sex Work (WSWs) view police as having significant influence over their safety. WSW report many cases of abuse, harassment, rape and discrimination perpetrated by police officers. A WSW summarized, “When police officers find us in our rooms they force us to sleep with them and we have nowhere to report this since it’s the police officers who are abusing us, we end up being scared”. Others mention that when they are arrested on “vagabond” and “rogue” charges they have to have sex with police officers in order to be released.
Encouraging joint meetings between WSWs and Police can reduce stigma on both sides. A number of police listening club members suggested inviting local WSW representatives to listen to the radio programme with them on alternate months. They argued this will enable fellow police officers to develop personal relationships with local WSW and understand their needs and rights.
Engaging local leaders can support the protection of WSWs. The project has successfully engaged local chiefs in the identification and selection of both WSW and GBSE beneficiaries. This strategy has resulted in Chiefs having a stake in project activities. As one Chief reports now that he has worked with TfaC he is even more willing and motivated to continue his involvement. He states, “If they find problems in solving some issues [for WSW and GBSEs] I can try to help them with ideas. I can set up community meetings for them, try to come up with activities to entertain people".
Results
Through an innovative and participatory stakeholder workshop, the Midterm Study was able to discuss and explore assumptions of the project, risks to implementation, and strategies to promote future impact. Engaging diverse stakeholders including implementing partners, WSWs, police, and district officials, in the reconstruction of the project theory of change synthesized existing knowledge on what could work to improve the sexual and reproductive health of WSWs and led to new partnerships between often divergent stakeholders.
