Terms of Reference for the final evaluation of project “Non-state Actors Promoting Budget Transparency”
Terms of reference
Non-state Actors Promoting Budget Transparency (BTAP project)
Vietnam (national level): Hoa Binh and Quang Tri provinces
March 2015 – March 2018
€6,000 – €7,000
Evaluation commissioning manager
Nguyễn Thu Hương (Oxfam)
Trần Thị Thanh Thủy (Oxfam) and Nguyễn Thùy Linh (CECEM)
Background, rationale and purpose of the evaluation
BTAP is a three-year project funded by the European Committee that runs from March 2015 to March 2018, seeking to address the mismatch between the top-down budget process and bottom-up socio-economic development planning at a local level. The overall objective of the projectis to contribute to increase public Budget Transparency, Accountability and People Participation (BTAP) in Vietnam in order to enhance development outcomes for women and men from marginalised communities in the provinces of Hoa Binh and Quang Tri.The specific objective is that empowered civil society organisations (CSOs), community-based organisations (CBOs) and communities in Hoa Binh and Quang Tri are able to effectively influence the public budget process of poverty reduction and development programmes (including on health issues) with wider national impacts on relevant laws and decrees.
The project´s theory of change is based on the belief that transparency, accountability and people´s participation in local public budget processes leads to positive impacts on resource allocation and management for poverty reduction and development programmes. By being engaged in the budget process, poor people, ethnic minorities and women will be able to shape budget allocation and expenditure for relevant programmes and benefit from the development outcomes. For this to occur, conditions for BTAP are required simultaneously in four domains: a) citizens with access to budget information, with capacities to analyze and decide on public budgets, and empowered to negotiate spaces for participation; b) elected officials, such as People’s Councils at commune, district and provincial levels, with increased technical expertise and timely access to accurate budget information and mechanisms to facilitate oversight, people’s participation and accountability; c) local government authorities, such as People’s Committees, with the capacities and political incentives to improve BTAP, as a result of pressure from CBOs, MOs, People´s Councils, informed and empowered citizens, and champions within government; and d) an enabling national environment, including legal frameworks, national policies, open information and favourable media and public opinion. This project will introduce changes in all four spheres through a pilot intervention aimed at improving BTAP in selected poverty reduction and development programmes. The aim is for the local level intervention to become a “model” of BTAP, to be adapted and replicated by other provinces and at national government level, while also employed as evidence for advocacy work to change relevant laws and decrees in favour of BTAP.
Particularly, expected results of the project are:
· Result 1: Poor people, ethnic minorities and women at local level have increased awareness of the budget process and are able to exert influence on this process of poverty reduction and development programmes (including on health issues).
· Result 2: The Budget Transparency Coalition is expanded to include multiple stakeholders and strengthened to promote and advocate for budget transparency, accountability and participation at local and national levels, including in relevant laws and decrees at national level.
· Result 3: Targeted People’s Councils from commune to provincial levels are more effective in the oversight of the budget process, promoting budget transparency, accountability and people’s participation.
· Result 4: Targeted People’s Committees from commune to provincial levels open spaces for civil society’s meaningful engagement in public budget formulation and budget monitoring of poverty reduction and development programmes (including on health issues).
There are multiple implementing partners in the project who have been selected for their expertise that supports the project objectives and their capacity to manage resources and implement project activities. They are 3 CSOs including Center for Development and Integration (CDI), Action to Community Development Center (ACDC), and Center for Community Empowerment (CECEM); and four local implementing partners including Farmer´s Union in Hoa Binh (HBFU), Women’s Union in Quang Tri (QTWU) and Provincial People’s Councils at communal, district and provincial levels in the two provinces.
As planned, a final evaluation of the project will be conducted to assess the achievements of the project, Oxfam’s and its partners’ contribution to the realization of the theory of change, and to document key lessons learnt from the project. In addition, the evaluation is aimed to assess what strategies and approaches Oxfam and partners have undertaken to influence the changes on public budget transparency, accountability and people participation in Vietnam.
Oxfam plans to use the evaluation to document and share lessons learnt on fiscal justice programming.It is expected that the evaluation will inform the implementing team and partners in their future work. Moreover, the evaluation will be shared widely across the Oxfam confederation, with key partners and interested donors to inform future project design and implementation.
In line with our contractual obligation with the donor, the evaluation will be an accountability tool to demonstrate to the donor, Oxfam and other relevant stakeholders how funding was spent and what results were achieved.
1. The specific objectives of the evaluation
The evaluation will cover the whole project’s interventions at all levels of commune, district and nation. Specifically, the evaluation will seek to assess to what extend and how the identified specific objective and expected results have been achieved, and how these achievements have contributed to the overall objective of increasing public Budget Transparency, Accountability and People Participation in Vietnam, and enhancing development outcomes for women and men from marginalised communities in the selected provinces.
The audience for the final report is Oxfam staff from senior management to front line programme officers, implementing partner organisation staff, beneficiaries and stakeholders to the extent that the country team want to socialize findings, the current donor and future donors (as applicable).
2. Overarching questions for the evaluation
These are the overarching questions that will guide the evaluation. It is expected that these questions will be unpacked and broken down into sub-questions during the inception phase to ensure that the evaluation responds to the information needs of all stakeholders. The evaluation will aim to:
- Assess the overall achievements of the project and how the project has contributed to empowering civil society organisations (CSOs), community-based organisations (CBOs) and communities in Hoa Binh and Quang Tri to be able to effectively influence the public budget process of poverty reduction and development programmes (including on health issues) with wider national impacts on relevant laws and decrees over the project’s lifetime, looking particularly to what awareness has been raised and capacities has been built of poor people, ethnic minorities, women, and targeted organizations that lead to their actions on fiscal justice.
- Assess the effectiveness of the strategies and approaches that Oxfam and partners have used to influence changes towards increased public Budget Transparency, Accountability and People Participation in Vietnam, and enhanced development outcomes for women and men from marginalised communities in the selected provinces. Highlight examples of good practices are and how they could be scaled up and sustained beyond the project.
- Assess the appropriateness and relevance of the project in meeting actual needs of the target groups in the policy environment and country context of Vietnam.
- Assess the sustainability and recommend next steps for the intervention.
3. Approach, methods, and ethical considerations
The evaluation team will propose the evaluation design and methodology and develop the data collection tools. It is anticipated that the evaluation team will utilize both quantitative and qualitative methods of data collection and that participatory tools and techniques for data collection will be employed. Oxfam has used process tracing and contribution analysis extensively in the past, but other evaluation approaches that allow to infer causality are also acceptable. The evaluationis expected to be gender-sensitive, rights-based, inclusive, culturally-sensitive and comply with professional ethical standards. Individuals should not be mentioned in the report. This includes complying with Oxfam’s Responsible Data policy. Innovative approaches that would foster interaction and learning across the partners, that would include capacity building on evaluation, would be welcomed.
The full evaluation design and methodology will be presented in the inception report, and will include as a minimum:
- The evaluator’s understanding of the evaluation questions
- Description of the evaluation approach proposed
- Sources of information for primary and secondary data collection
- Tools to be used for data collection
- Data analysis methods that will be carried out
- Data interpretation approach
- A detailed evaluation workplan.
Stakeholders to be consulted:
o Staff involved in project implementation, implementing partners, other CSOs involved
o Local community actors and citizens engaged in the project, with a balance between gender and age
o Any government ministries or department staff that were engaged in the lifetime of the project.
Information sources to be consulted:
- Founding project documents (contracts, project proposal, governance documents)
- Reports (annual and biannual narrative and financial)
- Project products such as research, social media content, awareness raising materials, etc.
4. Evaluation team qualifications and skills
The following skills and expertise are required:
- Relevant university degree (Masters preferred) in a related field of study (such as research methods, evaluation, social sciences etc.) or equivalent training and work experience.
- Proven track record in undertaking evaluations, including impact assessments.
- Excellent understanding of policy, campaigns and influencing approaches and tools.
- Demonstrable skills and experience in qualitative research methods in general and exploring causal links between activities and policy outcomes. Previous experience using process tracing and contribution analysis is a plus.
- Interest in, understanding of and practical experience of development finance issues such as budget transparency, people’s participation in budget allocation and spending, and civil society activism, campaigning and advocacy.
- Strong understanding of gender and Governance/Active Citizenship
- Excellent communication and facilitation skills.
- Flexibility and openness to work collaboratively with the evaluation manager, the project teams and partners in a co-creative evaluation process.
- Fluency in English, ability to communicate complex information using simple and concise language.
5. Schedule, budget, logistics and deliverables
- January 2018: Selection of consultant(s)
- February 2018: Desk review and Detailed evaluation plan/tools design
- February – mid March 2018: Data collection
- Late March 2018: Draft report and feedback process, Validation exercise
- 31 March 2018: Final report
The total amount of funds available for the evaluation is EUR 6,000– 7,000 including tax. It is expected that all travel days, transport, accommodation and any other expenses incurred are incorporated in the stated fees of the evaluators. No additional payments or reimbursements outside the contractually agreed fees will be made.
Payments will be made according to the following payment schedule: 25% upon starting the contract (January), 25% on delivery of the first draft (mid March) and the remaining 50% upon delivery of a satisfactory final report (end of March).
The evaluation would be conducted mainly in Hanoi, with two trips to Hoa Binh and Quang Tri.
Partners, beneficiaries and stakeholders should be included in the design and delivery of the evaluation as far as possible bearing in mind geographical and financial constraints.
Field visits for data collection will be organised to facilitate project sites and face to face discussions with stakeholders. Details of the logistics regarding visits will be managed by CECEM.
1. Evaluation inception report or evaluation plan that will need to be approved before moving on to the next phase (including draft tools)
2. Brief progress report from the field work
3. Facilitation of a participatory workshop (TBC)
4. Evaluation draft report for comments
5. Final evaluation report of publishable quality, including an executive summary
6. One PowerPoint (up to 10 slides) of evaluation findings and recommendations
All deliverables in English.
6. Dissemination strategy, plan and responsibilities for sharing and using the findings.
The findings will be shared with communities, partners, and key stakeholders (internal and external). The country teams and the reference group will consider and agree what is the most appropriate format and process for sharing the evaluation findings with communities (women, men, boys and girls), partners and key stakeholders.
The evaluation report (or the summary) which is in English may be translated into Vietnamese to facilitate community and stakeholder accessibility In Vietnam. The report will be shared with the donor and a meeting will be organised before project close to discuss the findings.
The full evaluation report or the executive summary and management response will be posted on Oxfam websites and made available for publishing to institutional donors. This is consistent with Oxfam’s commitment to transparency and accountability.
The findings will be used to inform program development, organisational learning, accountability and advocacy. Following the completion of the evaluation, a management response to the findings and recommendations will be developed.
7. Process of the selection of the evaluator or evaluation team and expectations for evaluation proposal
Oxfam invites bids from individuals or teams that combine the experience and skills described above.
Tenders must include:
1. A cover letter of no more than 2 pages introducing the evaluator (or members of the evaluation team) and describing how the skills and competencies described above are met, with concrete examples as appropriate, including previous experience working as a team. Please also use this cover letter to indicate consultants’ availability at critical periods.
2. A light touch technical proposal of no more than 2 pages explaining the consultant’s approach to conducting this evaluation, including indicative timeframe of availability.
3. A 1-page budget covering all major anticipated costs (Oxfam prefers to pay an agreed price for the totality of the work including the field trips, which will be paid in phased instalments).
4. A CV detailing relevant skills and experience of no more than 2 pages, including contactable referees.
5. One example of a relevant previous evaluation.
Selection criteria to be applied:
- Quality, suitability and feasibility of the methodological proposal
- Profile and competencies of the evaluation team measured in knowledge, experience, composition and other relevant skills as outlined above
- Suitability of the financial proposal for the activities laid out in the methodology within the financial availability of the project.
Tenders should be sent to email@example.com by 17.00pm of Friday 19th January 2018.
Shortlisted candidates will be invited to make an interview with the evaluation reference group in the last week of January 2018.
NOTE: Once the selection process has been completed, the ToR will be updated accordingly.