Mixed-method Evaluation of US Embassy Foreign Assistance Programs – ACCESS and TVET
US Embassy in Georgia;
Georgia;
United States Department of State (DOS);
30.09.2022
30.12.2024
Caucasus;
The goal of this project is to evaluate US Embassy’s Public Affairs Foreign Assistance Programs in Georgia, namely the English Access Micro-Scholarship Program (ACCESS) and the Technical and Vocational Education and Training Support Program (TVET). By employing a mixed-method evaluation (comprising both quantitative and qualitative data), we aim to arrive at a better understanding of:
- How effectively the current program designs are achieving the desired results of building skills, developing desired attitudes, and increasing awareness among participants;
- The extent to which participants are maintaining the skills, attitudes, and awareness after partaking in the program; and
- The extent to which participants are implementing activities that make use of the knowledge gained during the program, after its conclusion.
To evaluate the ACCESS Program, we will use a modified Kirkpatrick model, which is an internationally-recognized framework for assessing training programs. Having tailored the model in accordance with the project’s goals, we will investigate the following seven aspects of the ACCESS Program: Context & Needs; Participant Selection; Training Inputs and Process; Participant’s Reaction; Participant’s Learning; Participant’s Practice; and Training Program Impact. With these goals in mind, the evaluation team will conduct a desk review, as well as baseline, endline, and 6-month follow-up surveys with beneficiaries, and in-depth interviews with the relevant stakeholders. In addition, we will organize a ripple effects mapping workshop – a qualitative method that will help us to capture the wider impacts and adaptability of the program, beyond its intended outcomes. Moreover, we will conduct an online sense-making workshop with the participation of stakeholders at the end of the evaluation to discuss and agree on the findings,, before reaching a consensus on recommendations, all of which will be incorporated into the final evaluation report.
As for the evaluation of the TVET Program, we will examine not only the performance of the two projects implemented under this Program but also the broader TVET system and its elements to better understand the performance of these two projects. Guided by these goals, the evaluation team will carry out the following activities: desk review; in-depth interviews with program stakeholders; and survey with beneficiaries and their parents. Furthermore, similar to the previous evaluation, this will also entail conducting a sense-making workshop with program stakeholders.
