Value of Value-added
August 15, 2008
Value-added is a major state initiative in Ohio and elsewhere and represents an important education policy trend. The basic premise of “value-added” is that progress is best measured by following a cohort of students over time rather than comparing cross-sectional achievement scores for a grade or school each year. For example, standard metrics of school improvement in 7th grade might show:
Year 1 – Average test scores in 7th grade were X%
Year 2 – Average test scores in 7th grade were Y%
As a result, if Y > X, then one may conclude that the 7th grade instruction in that school was effective.
The problem with this is that it does not actually measure progress. A better way to assess student achievement is to measure:
Year 1 – Test scores of a class in 7th grade
Year 2 – Test scores of that same class in 8th grade
Hence, the change in test score, or “value added,” is better able to capture the influence of a given teacher, school, or program on student achievement.
@sineadmullen, a researcher for Stanford University (and good friend), wrote on Twitter:
Doing some value-added modeling for my research. After being in classrooms all year, really makes me wonder what this actually captures
She followed this up with a thoughtful e-mail:
I think value-added modeling is a valuable tool in some respects, but it has some pretty major limitations. Clearly, we want teachers who can improve student test scores, they must be doing something right. However, it is unclear what it is that they are doing that makes their value-added scores high.
In the last year, I went into numerous schools throughout NYC and observed teachers with very high value added scores from the last three years (top quartile) and teachers who below average value-added scores (3rd quartile) (I was blind to who was who at the time). In general, the level of instruction was pretty bad. But what I found astounding was that I could not predict who was the more effective teacher.
My favorite teacher that I saw was one of the most incredible teachers I have ever seen including my own education. She connected with the students, they were excited about working, she had them doing rigorous work, having incredible conversations, their behavior was amazing. I mean really it was astounding. She was a low value-added teacher.
Other teachers who were high in value-added were the most boring classrooms I had ever stepped foot in. I even fell asleep during some of them. Often times the behavior management was good (and as part of our study we may find that the teachers with the best management were most effective in improving scores). But the level of work was mind numbingly boring. Students didn’t get meaningful feedback. They just went through the motions, practiced over and over again reading comprehension passages and math problems. The sort of the thing that obviously would translate to a higher test scores, but certainly not a great education.
So while I think value-added scores are important…they really miss the boat in showing us who the great teachers are or how great teaching translates into test scores. And in places like NYC, where they are hoping to make decisions about hiring and firing based on value-added rankings, I would be real hesitant. It seems to miss something real big.
Sinead criticizes value-added because she witnessed bad teaching achieving good value-added results. But the problem is not with the measurement method (ie. value-added), but rather with the outcome measurement (ie. test scores). While Value-Added is a conceptually powerful tool to track student progress, policymakers need to better understand the assessments being used to measure student outcomes.
Welcome to Smart Solutions Blog/Web Page
January 14, 2008
Note: Please click here to read the full blog. Also, please do not hesitate to e-mail me or call me at 216-374-6723.
Welcome to Smart Solutions new K12 web page. First, let me introduce myself.
My name is Nitin Julka and I have been “blogging” since 2002. I am an avid consumer of the news, politics, blogs, think tanks, magazines, podcasts, and other media. If you google me, you will find a lot of results. To save you the energy, you can read about me on my personal web page.
I am committed to improving education through the use of technology. Smart Solutions is my family business, and after completing an undergraduate degree in Computer Science (and Psychology) from University of Wisconsin-Madison, a stint in Consulting at CRA International, and finally an MBA from Columbia Business School, I have returned home to the business.
While we are not a large company with a large marketing department, I find that blogging is a good way to create a “voice” for my enterprise. Therefore, following in the illustrious footsteps of Richard Edelman, Andy Rotherham, and John Halamka, I have started this blog.
This blog will focus on Ohio, education technology, and education policy. I always welcome feedback and suggestions – please e-mail me any questions or comments that you have about anything. I am especially interested in building a community of educators who are committed to furthering collaboration, 21st century skills, and education through the use of technology.
If you are an educator in Ohio and Michigan, and wish to learn more about our company or services, please contact k12@ssik12.com .
Looking forward to getting to know you!