Low-performing subgroups often invisible to policy boards
(Calif.) Two years ago when a group of school districts operating under a federal waiver agreed to hold themselves answerable for the performance of subgroups of just 20 students they knew a lot more kids would become visible for accountability purposes.
But what they didn’t know until now is that often the subgroups that became visible were also the lowest performing.
“Obviously more students become visible at a lower subgroup size, but what was surprising was that in a high proportion of schools, those were the lowest performing students,” said Heather Hough, executive director of a research partnership between a university collective and the California Office to Reform Education, or CORE.
“We think this really strengthens the case for lowering the subgroup size,” she said. “Because it is particularly those students that you do want visible for both school accountability and school improvement purposes.”
The findings come as the California State Board of Education is in the final stages of constructing new accountability measures that go well beyond ranking schools by test scores. Empowered by the newly adopted Every Student Succeeds Act, the state board is considering a matrix of multiple measures for evaluating school and student performance – including school climate, parent and community engagement as well as college and career readiness.
The CORE districts – which won a federal waiver in 2013 relieving them from many of the mandates of the No Child Left Behind Act – have been operating under a similar accountability system for the last year. That system, the School Quality Index, limits test scores and student academic measures to account for no more than 60 percent of the evaluation, and other factors – such as a school’s culture, climate and social settings – account for the other 40 percent.
In another major policy shift, the CORE districts – Fresno, Garden Grove, Long Beach, Los Angeles, Oakland, Sacramento, San Francisco, Sanger and Santa Ana Unified – agreed to lower the subgroup threshold from 100 students, as defined by NCLB, to 20 students.
To get a better sense of the impact of change to the reporting pool, researchers at the Policy Analysis for California Education – a non-partisan research center based at Stanford University, the University of Southern California, and the University of California Davis – undertook a study to examine how many more students were now being included in the accountability review. They also wanted to understand how school performance changed as a result – that is, how many schools now had new subgroups and how were those students doing.
Perhaps the key finding is that for many schools, their lowest performing racial or ethnic subgroup couldn’t be identified at the larger reporting level. Indeed, for 47 percent of the CORE schools, results for the lowest performing subgroup were only reported when the threshold was dropped from 100-plus student to 20-plus students.
For instance, at the 100 student subgroup, only 7 percent of the CORE schools reported even having an African American subgroup. But when the pool is reduced to 20 students, the number of schools reporting jumped to 41 percent.
Thus, only 37 percent of test scores in math from African-American students were reported at the 100 student subgroup level. But when the pool is reduced to 20 students, the percentage of test scores from African-American students jumped to 88 percent.
The same was true for students with disabilities: at the 100 student subgroup, only 25 percent of test scores from SWD were reported. But when the pool is reduced to 20 students, the percentage of test scores counted jumped to 92 percent.
Hough said that having this information available to school administrators as well as state policy makers is critical in helping inform decisions about where to employ scarce resources.
“When a community is looking across this multiple measures of school performance and wanting to ask how schools are doing having this data, down to the 20-student subgroup is very important,” she said.
Other key highlights from the report:
- A substantially higher percentage of student data is reported at smaller subgroup sizes. For example, when the subgroup size is reduced to 20+ from 100+, six times as many schools report results for African-American students.
- At a subgroup size of 20+, approximately 10 times as many schools report results for all student subgroups than at a subgroup size of 100+.
- The reduction in subgroup size substantially changes school rankings; at the 20+ threshold, most schools’ weighted test scores are lower than at the 100+ threshold. However, the schools identified in the bottom 5 percent tend to be the same regardless of subgroup threshold.
- The 20+ subgroup size presents clear advantages in terms of the number of students represented, particularly in making historically underserved student populations visible. This increased information about subgroup performance may better support a continuous improvement framework, helping inform and empower educators to improve student outcomes.