STEM in Afterschool Impacts & Outcomes

The Afterschool Alliance recently released an analysis of the impacts and outcomes of STEM learning in afterschool. The report identifies evaluation studies that have been done in a wide range of afterschool programs across the country and highlights common themes, laying the foundation for future evaluation studies of STEM in afterschool.

 To find the 19 studies highlighted in the report, the Alliance cast a wide net among out-of-school time intermediaries (including the Coalition for Science After School) and program providers themselves. The report includes large national programs, (such as FIRST Robotics and the 4-H Science Initiative), statewide projects and initiatives (like After-School Math PLUS in New York and Missouri, and the Student Science Enrichment Program in North Carolina), as well as local groups (such as Project Exploration in Chicago and Techbridge in Alameda County, California).

The Alliance was able to identify three overarching themes that were examined and/or defined in the evaluation studies:

·      Improved attitudes toward STEM fields and careers

·      Increased STEM knowledge and skills

·      Higher likelihood of graduation and pursuing a STEM career

Further details of how each of the evaluation studies addresses each theme—and what the actual outcomes of each theme look like—are available in the full report.

Individual organizations can take action by conducting evaluation studies of their own programs. Evaluation studies are an excellent way for afterschool organizations and programs to identify what is and isn’t working in their program, as well as provide tangible evidence of success to funding organizations and community advocates. Afterschool programs can look for success indicators in two ways:

• Youth development outcomes:
• Increased graduate rates
• Likelihood of attending college
• Better attendance in school
• Increased community service or involvemnet
• Science and Math education outcomes:
• Higher test scores
• Demonstrated understanding of math and/or science concepts through high-quality projects and schoolwork
• Likelihood of pursuing a STEM college degree and/or career

If you’re looking for more resources on doing an evaluation study within your program, see our Assessment & Evaluation Resources page.

Evaluation and Afterschool

Evaluation and assessment are important components of ensuring your program's long-term success. As stated on the Coalition website:

"Program assessment and evaluation provide critical feedback for ensuring that a program is making the most of this opportunity. Additionally, these reviews can prove useful in securing funding."

Two Coalition members have recently completed major evaluation studies. Destination ImagiNation, a national non-profit afterschool program, underwent an evaluation by the University of Virginia. They found that Destination ImagiNation participants outperformed their non-participant peers on creative thinking, critical thinking, and problem-solving--all stated goals of the program.  

Project Exploration, located in Chicago, found that they have had a major impact in participants' decisions to stick with STEM education and career goals. The study was done by the Center for Research, Evaluation, and Assessment (REA) at the Lawrence Hall of Science, UC Berkeley and looked at ten years' of data collected by Project Exploration as well as surveys of more than 30% of Project Exploration participants. They also found that their youth-centered model was effective in encouraging positive youth development outcomes. You can read the full report, or listen to a podcast about the study at the National Conference on Science and Technology in Out-of-School Time presented by Project Exploration co-founder and director Gabrielle Lyon and REA Research Specialist Bernadette Chi. 

The Coalition website also has a list of tools available for program evaluation, as well as a list of independent evaluators from across the country. Looking for even more evidence of why evaluation is so important? Download this podcast from our conference in September, presented by Dr. Alan Friedman of The Museum Group. The podcast, called Three Reasons to be Eager to Do Evaluation: Your Funders, Your Organization and Your Students, presents compelling reasons why evaluation and assessment should be made a regular part of program design and implementation.

Program Evaluation and Teaching Effectiveness

The New York Times today published an article about a new study showing that student rankings of teacher effectiveness correlate with how much students learn in a school year, as measured by an increase in test scores. The study, which was funded by the Bill & Melinda Gates Foundation, is part of a larger effort by the Foundation to work with several districts to improve their teaching effectiveness.

Students filled out confidential surveys about their teachers, and researchers found that teachers who received positive reviews also tended to have students whose test scores improved over the school year. However, teachers who had their classes spend "a lot of time... practicing for the state test" consistently had lower rankings by students as well as lower test scores. This interesting finding has a lot of implications for education reform, and raises some questions about teaching effectiveness in out-of-school time as well.

Many afterschool STEM programs--including museum exhibits and programs, summer camps, and other opportunities for learning in out-of-school time--undergo evaluation to find out how well they are fulfilling their organization's mission or program's goals. This process, though often time-consuming, can be very important for keeping the program up-to-date with current educational standards and expectations as well as essential for proving success to current or potential funders. But how often does that evaluation process include input from students or participants? 

Since afterschool program effectiveness many not be measured against test scores, another metric must be used to gather data or feedback on the program's teaching effectiveness. Participant feedback--from students or adults--provides meaningful data on how well the program is doing, and what it could do to improve. Though this study specifically addressed in-school time teaching, hopefully it will provide afterschool programs with an impetus to have themselves evaluated and make an even greater case for out-of-school time education.