In order to achieve greater accountability and more effective use of public resources, millions of dollars and human resources are committed each year
In order to achieve greater accountability and more effective use of public resources, millions of dollars and human resources are committed each year to the collection of data and to the measurement of impacts associated with educational practice and programs. Too often the evaluative process is disconnected from internal feedback loops, which prevents the application of findings into actionable knowledge as a way to improve program impacts.
RISE Framework
ISKME’s approach to the evaluation enterprise entails the incorporation of evaluative evidence into the initial design and ongoing operation of programs and policy decisions. The purpose of this approach is to embody learning in action, and provide a process through which stakeholders are empowered to collaboratively test ideas and identify opportunities for improvement. The foundation of this work is based in understanding the goals of grant-funded programs and initiatives and the needs of the communities they impact—their unique context, practices, and perspectives. This action oriented, human centered approach, which we call the RISE Framework, is part of every aspect of our research, and is guided by the following principles:
-
Relational – Stakeholders at all levels are engaged in the evaluation effort in order to create a collaborative experience and calibrate expectations toward robust research outputs.
-
Inquiry based – The needs of internal and external stakeholder groups are gathered and cycled into research design, implementation and outputs.
-
Social impact oriented – All projects are viewed through the ultimate lens of desired social impacts based on client needs—whether they be enhanced student learning, instructional development, or education policy change, to name a few.
-
Empathetic – Tested diversity and inclusion frameworks serve as a starting point for our research services, so that the needs of all affected populations are served.
Action Oriented Research
ISKME’s action oriented approach to research is illustrated in the infographic above. Based on the specific context, a customized pathway for social impact is developed, as outlined below.
Design: Design evaluative measures and metrics in collaboration with clients as part of early program design or start of a program’s implementation, in these three steps:
1. Identify project stakeholders
2. Identify what success looks like, meaning, how will things be different if the project succeeds
3. Illuminate measurable outcomes (even if they might be hard to measure).
Test: Pilot test measures and metrics by gathering input from stakeholders, including those who represent lived experience (e.g., students, educators), including:
4. Apply framework to test for diversity and inclusion so that all affected stakeholders are included and served
5. Go through iterative design process, test ideas
6. Create feedback loops with various stakeholders
Improve: Iterate as program is implemented, refine as needed:
7. Devise monitoring mechanisms to be put in place
8. Develop a plan for transforming findings into action (such as enacting policy, moving past a pilot phase, turning theory into practice).
9. Make improvements to proposal submission, and receive funding!
Integrate: Supporting the incorporation of evaluation findings into program and policy decisions so that the full potential of evaluation products is realized. Finally, and iteratively:
10. Collaboratively analyze research data with input from stakeholders
11. Integrate findings from research into funded programs
This approach is exemplified through the knowledge management and open education research ISKME has conducted over the past 18 years. Measuring the impact of, for example, open education, isn’t just about assessing adoption rates or costs saved through the use of OER. Through our work with open education field-builders, including the William and Flora Hewlett Foundation, the Shuttleworth Foundation, the Michelson 20MM Foundation, and UNESCO, we have developed “hard to measure” social impact metrics that serve as a starting point for the clients we support. With an eye toward lived experience and the authentic audiences, it is important that research for social impact in education is also based on the need for culturally relevant, accessible learning experiences for all students. In doing so, this approach strives to look beyond traditional measures of efficiency and return on investment, to meaningful high impact change.
To consider how research for social impact could be applied to your work, please complete this short survey.