Out-of-School Time

Program Evaluation: Assessing What Works

How do you know if your after-school program is working and if it’s meeting the expectations of students, staff, parents, and community partners? Through surveys, focus groups, and other data sources, program directors are able to gain valuable information that not only influences program direction but also helps in fulfilling grant reporting requirements.

Despite the obvious benefits of a thorough assessment, many out-of-school time programs don’t have the resources to bring in outside evaluators or to develop their own scientifically tested evaluation tools. To address that problem, NWREL has just published a resource guide that offers sample surveys, focus group questions, and helpful advice for administering both.

Out-of-School Time Program Evaluation: Assessing Stakeholder Opinions Through Surveys & Focus Groups draws on NWREL’s experience in evaluating 11 districtwide programs around the Northwest. The tools used in these evaluations are reproduced, along with some of the findings of the studies and a review of related literature.

"A lot of programs are at a loss for what to do (about required annual performance reports and evaluation in general)," says Brenda Britsch, a NWREL evaluation associate. "We wanted to provide a way to help." Britsch notes that programs can simply use the tools to do the evaluations themselves or administer the surveys and then have NWREL analyze the data. Besides being cost-effective, using the NWREL templates allows programs to compare their findings with those of other after-school centers.

Britsch and fellow evaluator Elke Geiger recommend using multiple data sources when conducting a comprehensive program evaluation. Student surveys and focus groups can uncover satisfaction with enrichment activities and opinions on how well the after-school program links with the regular school day. Staff surveys yield information about professional development needs, communication with teachers and parents, and the perceived impact of the program on students. Questioning parents reveals what they think about everything from safety issues to transportation. Teachers are asked about their perceptions of the program, as well as how it has affected their individual students.

In addition to surveys and focus groups, Britsch and Geiger suggest using test scores, grades, teacher reports, and attendance and behavior data as part of the overall evaluation plan."Your evaluation should provide useful information that can be directly linked to your program goals," says Geiger. She notes that much of the evaluation tends to lead to program improvement rather than simply addressing accountability issues.

After conducting focus groups around the region, Britsch and Geiger uncovered a series of emerging themes. These included operational issues; linking the program with the school day; communication needs; programming; and partnership, collaboration, and sustainability issues.

To learn more about the evaluators’ findings and helpful advice on conducting your own evaluations, view Out-of-School Time Program Evaluation online at www.nwrel.org/ecc/21century/publications/ost_tools.pdf