This page is focused on the evaluation questions we will raise to get feedback on the evaluation criteria for VALUING practice-based Doctoral Programs.
A critical starting point in thinking evaluatively about such a program is to ask: What would success look like? Given that it’s a program of practice, should the success of this program be measured if the practice itself becomes enhanced as a result of the program? Or should the program be judged by the quality of the dissertations? Given the orientation towards practice, it is especially important that the criteria for success are not defined purely by traditional academic norms. Moving from one education paradigm to another necessitates that one does not use older criteria of success to judge the success of a new innovation.
One solutioning around the criteria of success problem is to have students who have successfully completed the program define what success means to them and in what ways the program has helped them achieve their criteria for success. In almost all cases, students joining the program had already embarked upon multiple years (in some cases multiple decades) of practice.
An important set of evaluation questions needs to focus on: How did their practice become enhanced as a result of this graduate program? Did the collaborative structure of the program help them in any tangible ways with problem-solving in their contexts? By the same token, was the program lacking in actually providing skills to solve problems in the students’ contexts?
A deeper question is: How do such innovations that start in specific parts of the university percolate and impact educational systems more broadly? For example, does the learning that comes from students working collaboratively help transform educational systems more broadly?
The Table below describes some of the questions that can help evaluate EdD programs
| Evaluation Questions to Evaluate EdD Program |
|---|
| Judging success Why did you join the program? What did success mean to you when you started this program? Have these criteria of success been met according to you? How? Has your definition of what constitutes success changed over time? |
| Value added to Individuals In what ways did the program help the students’ abilities to solve problems in their settings? Did it help their ability to analyze problems and conceptualize solutions? Did it increase their confidence to solve problems? Did it enhance their confidence as leaders? Did it make them better consumers of evidence? Did it enhance their ability to negotiate and collaborate with partners to solve problems? What were the unintended negative impacts of the program? Was the focus on taking an indigenous-lens sufficient? |
| Value added to Systems What specific skills did the program provide to help solve systemic problems? Did the program help improve Educational systems? How? Did it help students understand how to address bottlenecks within systems? Did it provide a deeper understanding of contextual/systemic challenges? In what ways did it help with addressing social inequities? |
| Supports Did the students feel sufficiently supported during the program? Did they feel sufficiently supported after the completion of their Doctoral degree? |
| Mechanisms In what ways did the program bring clarity on the role of values in guiding action? In what specific ways did the program bring a social justice dimension to their work? Did the collaborative ethos of the program help them become better collaborators? |
| Learning and Spread What are the lessons from this program for traditional PhD programs in Education? What are lessons learned for other practice-based programs? What specific aspects of such a program are relevant for a program focused on public health? |
| Ideas for improvement What were specific suggestions to improve the program? What more can be done to bring a specific social justice perspective to the work? What more can be done to enhance focus on indigenous practice? Were their specific skills they think they should have learned that they were not taught? What specific post-doctoral supports could be provided to enhance the program? What support systems are needed to help improve the program? How could the program better balance rigor and utility? |