ASU Education Partnerships

Our Goals

A primary focus of the evaluation efforts is to examine ASU University-School Partnerships' engagements and effectiveness on participating students' academic skills and achievement, attitudes toward learning and school, and behaviors in and out of the classroom. In addition, the evaluation efforts examine both the use of ASU and community resources by teachers and educational leaders and the degree to which professional development has affected practice. The evaluation efforts engage parents and family members who participate in the ASU University-School Partnership, to assess their understanding of program impact. The following is a summary of our goals:

• Utilize a rigorous design and develop sound and reliable evaluations to assess the effects of educational programs, innovations, interventions and reforms in a variety of diverse settings;
• Monitor ASU's teacher education production and diversity outcomes;
• Engage in scholarly activities and disseminate the findings;
• Evaluate the specific programs sponsored by Carstens Family Funds within Murphy School District;
• Maintain a monitoring system for outcomes related to VP engagements;
• Increase the visibility of VPEP's school partnerships; and
• Assist in efforts to improve Arizona's Math and Science talent pool.

Activities of the Evaluation Unit

Program Evaluation

Through the generous support of a private donor, a comprehensive evaluation plan has been implemented to identify successful programs within the ALPHA Partnership. Evaluation efforts are guided by a team of evaluation experts. The following is a brief description of specific programs that were evaluated for the 2005/06 academic year in the Murphy School District:

• Arizona Quest for Kids (QFK): Provides mentoring, enrichment opportunities, family support, and college financial support to low income students
• Bridges Learning Development Lab: Offers individualized support in the development of students' cognitive and perceptual skill across all grade levels.
• Bridges Learning Discovery Program: Offers individualized support in the development of cognitive and perceptual skills in preK-2 students.
• Creciendo Unidos (Growing Together): Provides workshops to support family bonding, communication skills, and cultural ties.
• Olweus Bullying Prevention Program: Aims to prevent bullying in elementary schools.

Data Profiling

The office's evaluation activities also include data profiling efforts for focus schools and districts within ASU's University-School Partnerships. Examples of demographic and achievement measures that are being compiled include:

Other Projects

AZIMASE: The Arizona Initiative in Math and Science Education (AZIMASE) is a state-wide, public-private collaborative model designed to improve educational outcomes in STEM (i.e., Science, Technology, Engineering, and Math) fields.

ASYouth: The VPEP office offer students in the ALPHA Partnership a variety of high quality summer enrichment programs. In addition, survey instruments have been developed to collect information from participating students and their parents.

Evaluation of Mesa Writing Project: Evaluation of Mesa Writing Project evaluation plan for the English Writing Project has been developed and appropriate assessment instruments are currently being reviewed.

The Greater Phoenix Region (GPR) Education Indicators Project: Part of the Arizona Competitiveness Indicators project led by the Office of the President, the GPR collects educational data from approximately 700 K-12 schools from 60 school districts in Arizona.

What is evaluation?

Evaluation is the systematic assessment of program operations or outcomes that concentrates on the effects of a program on its intended beneficiaries. It consists of the use of data to make strategic decisions and the development of appropriate measures to monitor progress. Evaluation also pays attention to process, or the way a program is conducted. Once the program's processes and outcomes are determined, the evaluation efforts focus on the merits of the program by comparing evidence collected to a set of expectations. Evaluation therefore is a means of contributing to the improvement and understanding of the impact of a program. Evaluation efforts must be comprehensive and incorporate multiple methods, measures, criteria, perspectives, and interests in order to best contribute to program improvement and assessment of outcomes.

Where does evaluation fit within the office of the VPEP?

Through the Office of University-School Partnerships, Arizona State University is committed to providing responsible and responsive leadership to Arizona's Pre K-12 education system in four central areas: (1) Early Childhood, (2) Education Teachers, (3) Education Leaders, and (4) Students, Schools and Families. A primary focus of the VPEP's evaluation efforts is to examine ASU University-School Partnerships' engagements and the impact they have on students' academic skills and achievement, attitudes toward learning and school, and behaviors in and out of the classroom. Evaluation efforts examine the use of ASU and community resources by teachers and educational leaders, the degree to which professional development has affected practice, and the engagement and understanding of program impact by parents and family members. Therefore, evaluation is critical to identifying and maintaining effective programs and partnerships. The following is a summary of VPEP's evaluation-related goals:

• Develop sound and reliable evaluation plans with a rigorous design to understand the effects of educational innovations, interventions, reforms, and programs and strategies in diverse settings,
• Monitor ASU's teacher education production and diversity outcome;
• Engage in scholarly activities and knowledge dissemination;
• Evaluate specific programs sponsored by Carstens Family Funds in Murphy School District;
• Maintain a monitoring system for outcomes related to VP engagements;
• Assist in efforts to maintain visibility of activities related to VP engagements; and
• Assist in efforts to improve Math and Science talent pool in AZ.

What evaluation models and theories are being used?

• M.E.H.M.E.T. (Measurable Engagements, Hyper-Modelistic Evaluation Theory)
• Layers and Levels of Measurable Engagements
• Systems Thinking
• ASU Biophysical Organizational Model
• Quality Touchstones
• Program Effectiveness Scale / Rating System

Why is there a need for evaluation?

Evaluation can provide information on which programs and program supports are achieving desired outcomes, whether programs should be continued, and what should be done next in relation to the targeted programs and supports. Evaluation results are critical components to effective partnerships in that they provide a way to make judgments on program impacts and performance. Evaluation also is essential in sustaining the partnerships themselves.

Where can I get more information?

To contact the Research and Evaluation Unit, click here.

What is the primary focus of evaluation in the office of the VPEP?

The broad evaluation purpose of the efforts of the Research and Evaluation Unit at the ASU Office of the VPEP is to examine ASU University-School Partnerships' impacts on participants. Specifically, evaluation efforts examine impact on participant students' academic skills and achievement, attitudes toward learning and school, and behaviors in and out of the classroom. Evaluation also examines teachers' and school leaders' use of ASU & community resources and the degree to which professional development has impacted practice. Evaluation efforts include parents and family members who participate in ASU University-School Partnership programs to understand their perspective of the impact of such programs. In addition, evaluation focuses on identifying program factors, if any, that are related to improved outcomes for students, teachers, leaders, schools, and families.

What specific programs are being evaluated?

Evaluated programs vary based on findings and funding for each academic year. For the 2005/2006 school year, evaluation efforts centered around five programs in the Murphy Elementary School District:

• Arizona Quest for Kids (QFK): Provides mentoring, enrichment, family support, and college financial support.
• Bridges Learning Development Lab: Offers individualized support in development of cognitive and perceptual skills.
• Bridges Learning Discovery Program: Focuses on the development of cognitive and perceptual skills in preschool through the second grade.
• Creciendo Unidos (Growing Together): Provides workshops that support family bonding, communication skills, and cultural ties.
• Olweus Bullying Prevention Program: Aims to prevent bullying incidences in elementary grades.

In addition, the Research and Evaluation Unit also examines the growth of its school-based partnerships in relation to Adequate Yearly Progress and AZLearns school status. Other programs and efforts in which evaluations are being conducted include the Alpha Student Teacher Incentive Program, the AmeriCorps Student Teacher Service Award, ASYouth summer programs for low income students, the Mesa Writing Project, and the Arizona Initiative in Math and Science Education (AZIMASE).

To learn more about the Research and Evaluation Unit of the ASU Office of the Vice President for Education Partnerships (VPEP), click here.

What is QT?

QT is an abbreviation for Quality Touchstones, which are areas of questioning with the purpose of guiding selection of the most promising programs/strategies in an organization. Each QT area incorporates the use of specific questions that can aid in identification, selection, and continued implementation of the most effective programs.

For more information on Quality Touchstones, click here.

What kind of rating system is used by the office of the VPEP to evaluate programs?

The Program Effectiveness Scale/Rating System was developed by the Research and Evaluation Unit of the ASU Office of the Vice President for Education Partnerships (VPEP) for the purpose of classifying overall impressions of reviewed programs into an understandable format. Following thorough review and statistical analysis of a program's effectiveness in relation to its goals and targeted outcomes, overall results can be compiled based on the criteria of this scale in order to rate its overall effectiveness and recommendations for its continuation. It is important to note that the purpose of this scale is to judge the effectiveness of a program during a particular time period of evaluation at a specific site or sites; the scale is not intended to represent the global effectiveness of a program.

For more information on the Program Effectiveness Scale/Rating System, click here.

What is M.E.H.M.E.T.?

M.E.H.M.E.T. is the primary theory incorporated by the Research and Evaluation Unit of the ASU Office of the VPEP; it is an abbreviation for Measurable Engagements, Hyper-Modelistic Evaluation Theory. This conceptual framework helps in understanding the layers and levels of measurable engagements as well as multiple stakeholder perspectives needed for effective evaluation of university-school partnerships. M.E.H.M.E.T. offers a model of the context needed in constructing an appropriate evaluation design for these purposes.

For more information on M.E.H.M.E.T., click here.

What types of measures are being collected for the Partner schools?

The office's evaluation activities include data profiling efforts for focus schools and districts in Alpha, Beta, and other partnerships. The following are a few examples of demographic and achievement measures that are being compiled:

• Student and Student Body Descriptors (i.e., total student population, gender and ethnicity distribution, English language learners, and students on free or reduced lunch)
• FTE Figures (i.e., number of FTE teachers)
• Community Descriptors (i.e., median household income)
• School Descriptors (i.e., Title I school designation)
• Student Achievement Tests (i.e., AIMS, SAT9, AZAC)
• Student Academic Progress
• School Profiles (AZLEARNS achievement profiles in the categories of excelling, highly performing, performing, and underperforming).

What is the strategic roadmap and how does it relate to evaluation?

Strategic roadmaps help to align institutional capabilities and long-term goals so that they are consistent with the organizational vision and mission. Roadmaps provide guidance in development of strategy and process, which includes continuous evaluation.

For more information on ASU's Strategic Roadmap, click here.