By: Sheryl Nussbaum-Beach
Dean Shareski is a community leader with PLP. He asks some interesting questions in his building of community within our cohorts. He does the same on his own blog. Today he asked:
In the effort to be fiscally responsible and focus our attention on what truly matters, we hear this question asked almost everyday as we make choices on how to support our school division: “Will it improve student learning?” That’s a simple and yet very complicated question. It’s pretty easy to suggest that almost anything you do will improve student learning but not everything will translate into the classroom and impact student learning immediately or directly.
Here’s what I want to know:
1. How do you measure the effectiveness of professional learning?
2. Is there a time frame in which the professional learning translates into student learning?
I’m not sure I’m articulating myself particularly well here but am searching for some ideas that will help me solidify my own beliefs. I’m really curious to hear your thoughts on this one. Thanks in advance.
I responded and since I have to wait for him to approve my comment I decided to share my thoughts here. (I am not very patient–working on it).
My response to Dean’s questions …..
The obvious answer is the effectiveness has to be measured first on how well the PD met the objectives it set out to accomplish. Having clear cut objectives – intentional direction – is a hallmark of PD that results in prof. learning. If the goal of the PD is to teach teachers how to be more effective math teachers, what are the specific objectives that will help them become more effective? It can’t be all theoretical, compelling case for improving hype. There needs to be specifics in terms of measurable change.
First you set the goal. Then you set the specific measurable objectives. Then you actually follow-up in ways that are not just self reported data (maybe classroom observation, focus group with students, looking at student work, etc.) For example, if an objective was: all learners, as a result of the PD, will be able to teach specific tested concepts maybe… quadratic equations by using a 21st Century strategy -let’s say the TPACK method.
Then there needs to be both formative and summative evaluation for the learners (teachers) who participated in the PD. So the first thing you do is embed lots of process pieces that result in created artifacts that show whether teachers mastered the concepts being presented. Mastery needs to take place at all levels of Bloom,s or some other cognitive measure. Are there examples of mastery through knowledge, comprehension, synthesis or analysis? Can they evaluate their own product in terms of the objective-how about others work? You provide places for them to metacognate about what they are learning and to hear each other explain what the learning means. Collaboratively you look at examples of best practice and then they create something that will show they mastered the concept. You ask lots of questions, have your learners do most of the talking, and generalize to authentic examples.
Here is where you measure for student learning and it occurs immediately-3 months-6 months- 1 year out. After the PD is over you visit classrooms to see how the new professional learning is going. You use observation check lists that look for specific use in relations to your intended objectives. You look for scale- how are the concepts being used in generative ways you hadn’t considered
in your PD. You talk to kids and ask, “What are you doing? What are you suppose to be learning? Explain to me how to work that equation. What is TPACK?” “What can you do differently now that you couldn’t do before in terms of your own personal learning and understanding of this concept?” You record the answers so you can share with others in the school and in the blogosphere. You talk to the teacher and ask them to share what’s working from the PD, what’s not, lessons learned. You ask them, “What could I have done differently in my PD offering to have helped you grasp this concept better?” “What did I do that really worked?” “What do you need from me now?” “What professional learning goal can I help you achieve- one you are interested in achieving, not one I mandate, that is related to this concept?”
The effectiveness of PD, the kind that results in professional learning- is dependent on more than the teacher getting it when you are presenting the content. It starts with the PD provider’s planning and ends with follow-through in the classroom. Ongoing authentic assessment of how well the PD was delivered and then follow through to help the individual teacher with what they need to be successful or even what they are passionate about learning as an extension of the PD is what makes PD effective.
The less obvious is that the effectiveness of professional learning is not always measurable. Value add is a tough thing to measure empirically. For example, I was told a story yesterday by a teacher who said that for years she had been trying to get her principal and teachers to buy into the reasons and ways to bring about 21st Century reform and transformation in to her school. She felt she was banging her head against the wall. She said after her principal had heard me speak about learning communities as a PD means- something clicked. She finally got it and has been implementing sound 21st Century change since.
The point I am trying to make isn’t oh look I am great. Rather, that keynote was delivered over a year ago and I left having no idea of how effective I really was. I mean I had my objectives for the audience. I knew what I wanted to convey and what I wanted them to walk away with and how I was going to embed process into my keynote to make that happen, but I never knew the value add I had given that principal. It wasn’t measurable then. Make sense?
One of the reasons I love what we do together in PLP is that we are developing PD in an ongoing, job embedded capacity. Remember, it isn’t just about the learners- it is about you as a learner. And not everything that is of value can be measured.
Hope this helps. I am very interested in what others have to say.
Photo credit: http://jperk30.edublogs.org/
Latest posts by Sheryl Nussbaum-Beach (see all)
- “We have done this before, we can do it again.” - October 15, 2020
- Are you Remote Learning or Connected Learning? - October 6, 2020
- Passion Based Learning: Personalize Your Blended Curriculum - September 4, 2020
Hi Sheryl,
Interesting ideas.
Here is my response http://suifaijohnmak.wordpress.com/2010/08/28/how-to-measure-the-effectiveness-of-professional-development-activities/ to your post
Thanks for your insight.
I agree with your community and process over time focus and really believe that good PD must start with the teachers and finding out where they are coming from, so to speak. Too many time we have provided methodology for teaching quadratic equations or designing better graphic organizers when the teacher’s concern is that they haven’t seen half the class for the last week. I would eschew top down planing for a community first emergent research agenda with ongoing support and measurable, but teacher defined goals. Research academic and PD professionals might benefit from a little humility, even if their ideas have merit.
“Remember, it isn’t just about the learners- it is about you as a learner. And not everything that is of value can be measured.” Love this thought I might go so far as to say that most things of value can not be measured at least not in a quantifiable way. We really need to start thinking of teachers as learners, give them the same space for learning that we give our students and the same sort of respect as learners.
What a great look at professional development! I’m currently a math instructor in one of the largest districts in the state that prides itself on it’s outstanding pd. The sad part is, when I first started with this district, I was excited and inspired by all of the things that I was learning. However, when I went back to the classroom, all attempts to implement change as a result of my development were shot down immediately. I quickly realized that the pd we were provided was “all theoretical, compelling case for improving hype”. I believed it all, but without an outlet, I quickly became jaded. I can say, from “the other side of the desk” that the efforts by administration to see, measure, and continue to direct the effects of pd are absolutely critical. I love to learn. I love to grown. I love to try new things! So naturally, I come away from pd experiences hungry for ways I can change and apply what I’ve learned immediately! But without the encouragement and continued support from admin as I grow and change, permanent improvement is impossible. This makes me think of dessert rose flowers. They grow where water is minimal and the heat is intense! They are gorgeous to look at and when cared for properly, the blooms are overwhelming! However, they quickly outgrow their pots and out of their love for life-giving sunshine often grow taller than they can support alone. They must be staked up and tied together. If this care is taken, these flowers are breathtaking. But left on their own, their love for the sun and for growth, even with minimal resources, will be their ruin. They grow too tall, eventually lay down, and are unable to thrive.
I beg all of you in the pd world, to follow up. Check on us. Help us measure. Help us track trends. We have to know that you care too. It’s not enough to depend on our love for teaching, learning, students, technology. If you do, we’ll grow. We’ll grow fast. But eventually, we’ll become top-heavy and lay down and die. Like the desert roses, if we are staked up with support systems and tied together in collaborative teams, we will use our minimal resources and our love for the students and we will overwhelm you with our growth and take your breath away with the beauty we can create.
In Conclusion, the overall message is no different than what our grandparents used to say
in that ‘a penny saved is a penny earned’. A hot water boiler is a vital component of a central heating
system of a building. That is why if a hotel experiences a boiler problem (and does not have a contingency
plan in place), they could not only soon have irate guests refusing to pay for their stay, but
also get a bad name as being a hotel that is unreliable; not a hotel that people
would recommend to others.