Bill Gates Talks MET, Teacher Effectiveness
Whether you love or hate the Bill & Melinda Gates Foundation's work in education, it has become an influential part of the education policy world, specifically in discussions about teacher quality.
Some time ago, I interviewed Bill Gates while at the American Federation of Teachers conference, where he had come to address the union's delegates. We spoke about the foundation's $500 million Intensive Partnerships for Effective Teaching project and the $45 million Measures of Effective Teaching study.
I apologize to all for taking so long to transcribe this for you, but a funny thing happened on my way to the keyboard, and I got immersed in a special project.
Note that this interview took place before the preliminary MET results came out. You'll recall that they found some correlations between value-added data and student surveys. Those findings caused some backlash, too,with at least one scholar criticizing how the Foundation interpreted those results, as I reported a few weeks ago.
Disclosure: The Gates Foundation provides support for Editorial Projects in Education, the publisher of Education Week, but the paper retains sole control over the content. And if your sense of balance is still offended, well, let me put it this way: We've done similar Q & A features on this blog with the National Education Association.
Q: What's the most difficult or challenging part of this work so far?
A: Of course, the particulars of finding the right districts to work in and making sure that the union was really willing to allow observations and that everybody was going to push forward on it.
It's fantastic that we found the four locations where we feel great about the partnerships and that we're on the schedule we've set out for ourselves to get that kicked off. ... Pittsburgh is a place that I went and met with the superintendent, actually a couple years ago, and was very pleased that they were one of the ones that really showed they wanted to try these changes. So we're at the very beginning of this, and there will be a lot of course correction, a lot learned.
For me, I'm just kind of surprised how nascent the field is that there hasn't been clear-cut measurement systems that improve quality and that the teachers really enjoy more than a sort of pure how many-years have you been on the job, and do you have a master's degree type approach.
Q: And why do you think that is? Why has it taken so long that someone's doing a study of this magnitude, and to parse the attributes of good teaching?
A: We had the highest college-graduation rate in the world and we were doing very well through the 1960s, and it was only in the 70s that we started to realize that on a competitive basis or even absolute basis, our comparative position was weakening, and that's gone on a number of years as other countries have done better. The urgency of, hey, we've really got to be smarter about this has grown. And I think, I hope we're at a kind of reflection point where parents, teachers, and politicians say, wow, we are really not where we want to be on an absolute or a relative basis. So now we've really got to delve into improving teaching.
I think it was known for a long time that some teachers are dramatically better than others. Anecdotally, everybody's had that experience, someone who really made the subject interesting and encouraged you and someone who explained concepts in the right way, but being systematic about it requires a lot of effort, and teachers have to be willing to engage in that effort. ...
There is some earlier work like in Tennessee in the 1980s that guides some of our work, so it's not like there was nothing. But if there was a great evaluation system that improved people and that was good, we would just grab that and promote its widespread usage.
We've got really two goals: We'll tune [our measurement systems] so these teachers involved like it, and the achievement is good, and they're telling other teachers about that. Because our dream here is not just to have four great locations in the country, it's to have a fundamental approach for constant improvement that is great for teachers and for students.
Q: Your earlier work in small schools was dealing with the structure of schools. This is a lot about the attributes of teachers. I'm wondering if you have an idea of what lies in the nexus there. How do you create school structures, leadership structures, other things that will help [effective teaching practices] transfer?
A: We think that the personnel system that does the measurement and incents improvement and helping other teachers improve is a first-class system. Incentive systems are very first-class systems, and so if you get that in place, then it can cause powerful change. We've seen in spaces like health care, if you can get the incentive system messed up, you can get costs out of control. We've seen in technology where the incentive for entrepreneurs, risk-taking is very, very good, you can get some magical things to happen. Here it's the personnel system, and in and of itself it's a very powerful factor if it's embraced after it's proved itself out.
We do think some of the structural things are complementary. Having 1,500 kids all in one big high school where you don't break it down into smaller groups—we still think that's a mistake and we actually have some pretty clear objective third-party data to say that, but we admit there's a ceiling. You don't just break a high school down into pieces and get the equivalent of schools in the suburbs or the best charter schools. Unless you're helping that teacher get better in the classroom, you can only go so far.
Q: What are the roles of school leaders in helping improve the effectiveness of teachers in the building?
A: They have a central role to play in this evaluation system. As a teacher's doing well, they're talking to them, talking about what aids there are to help them improve. They play a very central role. It's like management. Great software writers do have managers, and they have to have a very constructive relationship in order for that to work. People have to work well together and when things aren't going well they have to talk about what's going to change. Great principals and superintendents, that's another leverage point. But it's also our view that you could do a lot there and still you'd hit a ceiling. Those things are necessary but not sufficient. We've funded a lot of principal training about how you look at data, how you deal with infinite numbers of issues a principal has to deal with, and I think we got good gains on that.
Q: One of the most interesting components of MET is the study of student impressions. Can student impressions be part of a formal teacher evaluation?
A: I know there are cases where it's worked. The school I went to, which is a private high school, the student impressions are a major part of a fairly rigorous evaluation they do. What they found is that it aligns with the other [measures]. that is, it's not really an outlier.
When people first hear that, even when I first heard that, I was like, OK, we've got to be careful, because some of my best teachers I didn't love them. Actually, some of them I did love, but some of them it was kind of a tough-love situation where they pushed me quite a bit. So you do have to be careful with the student survey not to get any sort of popularity contest or ability of somebody to manipulate the thing. I doubt anyone would want to rely on that alone than they would want to rely on test scores or videos.
It happens to be a measure that is very low overhead. You can gather that data very very quickly. ... And preliminary data about some of those questions, are very predictive, very correlated with student achievement.
It's not to say that [student feedback alone] is our personnel system. I think it's wonderful because one of the problems we have is to get multiple data points so a teacher will not feel like it can be capricious. And as you move outside of math and reading, the test score data is not helpful there, and so if we can have things like this, we can bring more teachers in.
Q: What's your reaction to the fact that there have been some teachers and unions that have taken this work on, committed to the grants, signed contracts around it? After all, this is pretty scary stuff for a lot of teachers.
A: Absolutely. These are professionals, and when people change evaluation systems, that is a scary thing. If the imperative wasn't so dire in terms of the need to improve, if we were only going for small gain, then the comfort of the status quo should win out.
It's certainly unfortunate that we have in parallel with this a lot of state budgets that are very tight, and the funding for the state education system as a whole is subject to a lot of uncertainty and in some instances, cuts. If you had your druthers, you wouldn't be doing [this work] at the same time as there are those very tough things. But the work is so urgent it's not like on the behalf of students or teachers that you want to delay this stuff.
I'm impatient; I'm like you, see the videos of people who solve disruption in the classroom, who explain a concept well. And now that you can show me what good teaching is, finally, let's talk about how you transfer that to the other teachers...
Q: Any ideas about how to do that yet?
A: We will put great teaching out on the Internet for any teacher to look at. We'll organize that in a way they can find various things and learn from it. We will put great assessments out on the internet so a kid can self-assess and a teacher can assign a kid to self-assess to see what that kid may be missing. ...
It is strange how little we know about best practices. It's really very unusual. There are some teachers who are good with the kids who are behind and not with the kids who are ahead; there are some who are good at keeping the class calm but not explaining the concepts.
Some time ago, I interviewed Bill Gates while at the American Federation of Teachers conference, where he had come to address the union's delegates. We spoke about the foundation's $500 million Intensive Partnerships for Effective Teaching project and the $45 million Measures of Effective Teaching study.
I apologize to all for taking so long to transcribe this for you, but a funny thing happened on my way to the keyboard, and I got immersed in a special project.
Note that this interview took place before the preliminary MET results came out. You'll recall that they found some correlations between value-added data and student surveys. Those findings caused some backlash, too,with at least one scholar criticizing how the Foundation interpreted those results, as I reported a few weeks ago.
Disclosure: The Gates Foundation provides support for Editorial Projects in Education, the publisher of Education Week, but the paper retains sole control over the content. And if your sense of balance is still offended, well, let me put it this way: We've done similar Q & A features on this blog with the National Education Association.
Q: What's the most difficult or challenging part of this work so far?
A: Of course, the particulars of finding the right districts to work in and making sure that the union was really willing to allow observations and that everybody was going to push forward on it.
It's fantastic that we found the four locations where we feel great about the partnerships and that we're on the schedule we've set out for ourselves to get that kicked off. ... Pittsburgh is a place that I went and met with the superintendent, actually a couple years ago, and was very pleased that they were one of the ones that really showed they wanted to try these changes. So we're at the very beginning of this, and there will be a lot of course correction, a lot learned.
For me, I'm just kind of surprised how nascent the field is that there hasn't been clear-cut measurement systems that improve quality and that the teachers really enjoy more than a sort of pure how many-years have you been on the job, and do you have a master's degree type approach.
Q: And why do you think that is? Why has it taken so long that someone's doing a study of this magnitude, and to parse the attributes of good teaching?
A: We had the highest college-graduation rate in the world and we were doing very well through the 1960s, and it was only in the 70s that we started to realize that on a competitive basis or even absolute basis, our comparative position was weakening, and that's gone on a number of years as other countries have done better. The urgency of, hey, we've really got to be smarter about this has grown. And I think, I hope we're at a kind of reflection point where parents, teachers, and politicians say, wow, we are really not where we want to be on an absolute or a relative basis. So now we've really got to delve into improving teaching.
I think it was known for a long time that some teachers are dramatically better than others. Anecdotally, everybody's had that experience, someone who really made the subject interesting and encouraged you and someone who explained concepts in the right way, but being systematic about it requires a lot of effort, and teachers have to be willing to engage in that effort. ...
There is some earlier work like in Tennessee in the 1980s that guides some of our work, so it's not like there was nothing. But if there was a great evaluation system that improved people and that was good, we would just grab that and promote its widespread usage.
We've got really two goals: We'll tune [our measurement systems] so these teachers involved like it, and the achievement is good, and they're telling other teachers about that. Because our dream here is not just to have four great locations in the country, it's to have a fundamental approach for constant improvement that is great for teachers and for students.
Q: Your earlier work in small schools was dealing with the structure of schools. This is a lot about the attributes of teachers. I'm wondering if you have an idea of what lies in the nexus there. How do you create school structures, leadership structures, other things that will help [effective teaching practices] transfer?
A: We think that the personnel system that does the measurement and incents improvement and helping other teachers improve is a first-class system. Incentive systems are very first-class systems, and so if you get that in place, then it can cause powerful change. We've seen in spaces like health care, if you can get the incentive system messed up, you can get costs out of control. We've seen in technology where the incentive for entrepreneurs, risk-taking is very, very good, you can get some magical things to happen. Here it's the personnel system, and in and of itself it's a very powerful factor if it's embraced after it's proved itself out.
We do think some of the structural things are complementary. Having 1,500 kids all in one big high school where you don't break it down into smaller groups—we still think that's a mistake and we actually have some pretty clear objective third-party data to say that, but we admit there's a ceiling. You don't just break a high school down into pieces and get the equivalent of schools in the suburbs or the best charter schools. Unless you're helping that teacher get better in the classroom, you can only go so far.
Q: What are the roles of school leaders in helping improve the effectiveness of teachers in the building?
A: They have a central role to play in this evaluation system. As a teacher's doing well, they're talking to them, talking about what aids there are to help them improve. They play a very central role. It's like management. Great software writers do have managers, and they have to have a very constructive relationship in order for that to work. People have to work well together and when things aren't going well they have to talk about what's going to change. Great principals and superintendents, that's another leverage point. But it's also our view that you could do a lot there and still you'd hit a ceiling. Those things are necessary but not sufficient. We've funded a lot of principal training about how you look at data, how you deal with infinite numbers of issues a principal has to deal with, and I think we got good gains on that.
Q: One of the most interesting components of MET is the study of student impressions. Can student impressions be part of a formal teacher evaluation?
A: I know there are cases where it's worked. The school I went to, which is a private high school, the student impressions are a major part of a fairly rigorous evaluation they do. What they found is that it aligns with the other [measures]. that is, it's not really an outlier.
When people first hear that, even when I first heard that, I was like, OK, we've got to be careful, because some of my best teachers I didn't love them. Actually, some of them I did love, but some of them it was kind of a tough-love situation where they pushed me quite a bit. So you do have to be careful with the student survey not to get any sort of popularity contest or ability of somebody to manipulate the thing. I doubt anyone would want to rely on that alone than they would want to rely on test scores or videos.
It happens to be a measure that is very low overhead. You can gather that data very very quickly. ... And preliminary data about some of those questions, are very predictive, very correlated with student achievement.
It's not to say that [student feedback alone] is our personnel system. I think it's wonderful because one of the problems we have is to get multiple data points so a teacher will not feel like it can be capricious. And as you move outside of math and reading, the test score data is not helpful there, and so if we can have things like this, we can bring more teachers in.
Q: What's your reaction to the fact that there have been some teachers and unions that have taken this work on, committed to the grants, signed contracts around it? After all, this is pretty scary stuff for a lot of teachers.
A: Absolutely. These are professionals, and when people change evaluation systems, that is a scary thing. If the imperative wasn't so dire in terms of the need to improve, if we were only going for small gain, then the comfort of the status quo should win out.
It's certainly unfortunate that we have in parallel with this a lot of state budgets that are very tight, and the funding for the state education system as a whole is subject to a lot of uncertainty and in some instances, cuts. If you had your druthers, you wouldn't be doing [this work] at the same time as there are those very tough things. But the work is so urgent it's not like on the behalf of students or teachers that you want to delay this stuff.
I'm impatient; I'm like you, see the videos of people who solve disruption in the classroom, who explain a concept well. And now that you can show me what good teaching is, finally, let's talk about how you transfer that to the other teachers...
Q: Any ideas about how to do that yet?
A: We will put great teaching out on the Internet for any teacher to look at. We'll organize that in a way they can find various things and learn from it. We will put great assessments out on the internet so a kid can self-assess and a teacher can assign a kid to self-assess to see what that kid may be missing. ...
It is strange how little we know about best practices. It's really very unusual. There are some teachers who are good with the kids who are behind and not with the kids who are ahead; there are some who are good at keeping the class calm but not explaining the concepts.
