[ad_1]
A new study by a team of economists at MIT shows a new way of evaluating Value-Added Models (VAMs). VAMs are used to assess schools and teachers.
Various states in the U.S. have recently started assessing schools and teachers using Value-Added Models, or VAMs. The idea is simple. A VAM looks at the year on year changes in the standardized test scores of students. The schools and teachers are then rated accordingly. When it is found that the scores regress or improve, schools and teachers get either the blame, or the credit. This process is actually similar to the one used in determining credit scores.
The use of VAMs has however resulted in a lively debate, with opponents claiming that standardized tests could result in a misleading guide to educator quality, while proponents argue that they bring useful metrics and accountability to education evaluation. Although VAMs do make adjustments for differences in the characteristics of a student, educators argue that these adjustments are not enough.
The new MIT study has developed an innovative way to evaluate and improve VAMs. The researchers used data from Boston schools with admissions lotteries, and used the random assignment of students to schools to determine how similar groups of students do in various classroom settings.
Josh Angrist, the Ford Professor of Economics at MIT and co-author of the paper explains that value added models have high stakes and it is critical that VAMs result in a reliable guide to school quality.
The researchers have shown that existing VAMs are prone to underestimating the amount of improvement test scores actually achieve at some schools. It is however also true that conventional VAMs do deliver an approximate figure for improvement that can’t be discounted.
Peter Hull, PhD and co-author adds that although conventional VAMs are biased, the team was able to demonstrate that the bias is modest and that in Boston, VAMs are useful for generating predictions of school quality.
The paper, called “Leveraging Lotteries for School Value-Added: Testing and Estimation,” was published in the Quarterly Journal of Economics.
The conclusions reached by the authors result from an analysis of data from Boston’s public school system. Academic years from 2006 to 2014 were taken into account. The data includes a sample of about 28,000 students at 51 different schools.
The test scores were taken from fifth- and sixth grade results in the Massachusetts Comprehensive Assessment System (MCAS), in English language arts and math. The team use this data to first replicate conventional VAMs and then developed their new hybrid VAM model that combines the new school quality estimations with the older approach.
The study utilizes the fact the Boston school system makes use of a centralized assignment system for students. This system uses a lottery feature to determine which students will attend schools that are in high demand. An element of chance therefore determines where a large portion of sixth graders will be enrolled in middle school. This gave the researchers the random assignments they needed to derive higher resolution comparisons of the effects schools have on the student’s achievements.
As the students in this pool of applications are different only in where they were offered a place, researchers are able to make proper comparisons to see how those who were admitted via the lottery perform when compared to those who had not been admitted. This difference in performance should therefore reflect school quality rather than any other differences.
On the other hand, when two schools are compared without using random assignments, it becomes extremely difficult to ensure that the students being evaluated are similar in other ways. In this situation, what may appear as a lack of student achievement, may be as result of the school having more disadvantaged students.
The authors note that the findings form part of a broader political debate on education systems in general. Charter schools are often debated at length, as they receive public funding but may be operated privately and use nonunion teachers, in contrast to traditional public schools. Pilot schools are a hybrid model, with more space for variations in curriculum and scheduling than public schools, but with unionized teachers.
Angrist notes that school performance is a vital topic, both for educators to evaluate and for researchers to examine. It may indeed be more pressing in school districts where test scores have been low constantly, and where larger inequalities in school quality might be found.
[ad_2]
Source link
