The November 7, 2022 edition of the La Canada Flintridge Outlook Valley Sun newspaper proclaimed, “LCUSD Scores Among Top in California.” The accompanying graphs, provided to the Outlook by the La Cañada Unified School District (LCUSD), showed LCUSD as top 3 in the state of California in all grades tested for both English and Mathematics based on the 2022 California Annual Assessment of Student Performance and Progress (CAASPP) assessment results:
One had to look closely to notice the footnote – “based upon comparable high performing Unified School Districts.” The Outlook story followed the presentation of the district’s Spring 2022 CAASPP assessment results given by LCUSD Executive Director of Programs & Services Karen Hurley to the LCUSD Governing Board at its October and November meetings.
LCUSD Governing Board member Caroline Anderson commented later during the November 15th meeting that she was proud the district retained its top 3 status in the state. While LCUSD Superintendent Wendy Sinnette was careful in her various emails and reports to the district community and Governing Board to qualify the “top 3” characterization as “compared to the high performing unified school districts with whom we have traditionally measured and assessed our results,” Anderson omitted the qualification and continued an exaggeration the district has maintained for many years. La Cañada Math Parents (LCMP) has reported on this exaggerated claim several times in the past (see LCUSD Continues to Overstate Its Rankings from Nov. 2019 and When Top 3 Isn’t Top 3 from a year earlier.)
Superintendent Sinnette and Programs & Services Executive Director Hurley both deserve credit for clarifying this year that the district’s stellar CAASPP performance was among a handful of high performing Unified school districts with whom we have traditionally measured and assessed our results. The district did not make such qualifications in the past and the “top 3 in the state” adage is repeated so often in the community that rarely a week passes by without a realtor, parent or administrator crowing about La Canada schools being “top 3 in the state.”
To reiterate from our past articles highlighting this problem, LCUSD ranks in the top 3 among a handful of selected Unified school districts, omitting comparisons with thousands of schools that are contained within other types of school districts. Further, given LCUSD has only one high school in its district, it should be comparing La Canada High School’s standardized test results against other individual high schools, not multi-high school district averages. While the distinction may seem trivial, a comparison of the district’s claimed rankings versus its actual ranking against all public schools in California reveals a large disparity in rank:
The Actual Rank in the table above was calculated by LCMP based on the raw data files provided by the California Department of Education and detailed data tables are presented below. Rather than report the mean scale score of LCUSD’s grades as the state does, LCMP rank orders all public schools that had CAASPP scores reported by the state and shows the ordinal ranking of each school’s grade-level cohort for a given subject test (i.e. English Language Arts or ELA, and Mathematics,) and for the district rank averages the three elementary school ranks together. This methodology normalizes the data to account for differences in the difficulty of the CAASPP assessments from year to year, changes in test conditions such as those caused by policy responses to the COVID-19 pandemic, as well changes in the format of the test as occurred in 2021. As can be seen, LCUSD was not top 3 in any grade level + CAASPP subject area, and compared to 2019 LCUSD dropped in rank in most grade-level cohorts.
Why Just These Fourteen School Districts?
When asked in the past why LCUSD chose to limit its statewide comparison to just a limited number of Unified school districts, administrators variously claimed that they were just following the practice of individuals who held their administrative position previously, or that the selected districts had similar student demographics, school size, and socioeconomic status. The latter explanation would make sense if widening the comparison led to misleading conclusions, but the opposite motive seems to be at play here – narrowing the comparison inflates the status of LCUSD schools. To test the district’s second explanation – that the chosen set of districts have similar student demographics, school size, and socioeconomic status, LCMP examined objective measures of the fourteen Unified school districts LCUSD chose:
And here are the same metrics of the other districts omitted from the comparison that eclipse us in actual rankings when all districts in California are compared:
Comparing the non-Unified higher-performing districts to the LCUSD-selected cohort of high-performing Unified school districts, we looked at the mean, standard deviation, and mean absolute deviation to measure the variance in the cohorts. Metrics in green have a lower variance in the second group, while metrics in red have a higher variance in the second group. As can be seen, the non-Unified higher-performing district were more similar to LCUSD than the 14 districts LCUSD chose to compare itself to in all metrics except Zillow Home Value Index, # of High Schools, and two racial sub-categories. In other words, there is no statistical data to support the district’s claim that the districts it chose to compare to are more “comparable” to LCUSD than those they omitted.
Why Just Unified School Districts?
As LCMP has explained multiple times in the past, there are five types of school districts in California:
- Unified School Districts include elementary, middle and high schools covering all pre-college grades K through 12.
- Elementary School Districts generally include schools covering grades K through 8, though some limit themselves to just K-3, K-5 or K-6.
- High School Districts generally include just high schools (i.e. grades 9 through 12.)
- Joint Districts – “Joint” denotes that the district includes territory from more than one county. There are joint elementary, joint high school, and joint union districts of both types.
- Union Districts – “Union” denotes that the district was formed from two or more districts. There are both union elementary and union high school districts. A “joint union” district means it was formed from two or more districts, and includes territory from more than one county.
By narrowing the comparison to only other Unified school districts of “like” kind, vast numbers of other schools were omitted from the comparison. As LCMP stated in its 2018 analysis, “Since only 37.7% of school districts in California are Unified or Joint Unified districts, thousands of public schools were omitted from the comparison.” Schools in the five major different types of districts are indistinguishable from each other on the outside — their students are held to the same state education standards, take the same standardized tests, use the same types of instructional materials, and hire from the same pool of certificated teachers.
Comparing Against All Districts
Given there is no good reason to limit comparing LCUSD against just Unified school districts, LCMP downloaded the raw data sets from the California Department of Education’s website for the 2022 test and re-ran the district’s comparisons against districts of all five types listed above.
CAASPP Mean Math Scale Scores – Districts:
As a reminder, here is LCUSD’s table comparing LCUSD against their carefully selected high-performing unified school districts by CAASPP Mathematics mean scale score:
LCUSD shows the #1 district in green, the #2 district in yellow, and the #3 district in orange. Here are the actual results comparing LCUSD against all districts in California, not just Unified districts, using the same color coding scheme:
LCMP generated the above table by ranking all public school districts by math mean scale score for each grade, then averaging the rank across all grades taught in that district. Mean scale scores cannot be combined or averaged across grades because the mean scale score ranges are different for each grade.
CAASPP Mean ELA Scale Scores – Districts:
And repeating from earlier, here is LCUSD’s table showing how LCUSD fared against fourteen “comparable” high-performing unified school districts by CAASPP English Language Arts (ELA) mean scale score:
Here are the actual results comparing LCUSD against all districts in California in ELA, not just Unified districts:
Comparing Against All Schools
As explained in the article “When Top 3 Isn’t Top 3” from LCMP’s January 2019 newsletter, LCUSD also benefits from being a “boutique” district — it is unusually small with just one high school, whereas 94% of California public high schools are in districts that include two or more high schools. Given almost all high-performing California public high schools reside in districts with two or more high schools, LCHS and LCHS 7/8 appear better when comparing them by district mean scores than when comparing by school mean scores. High-performing high schools in districts with more than one high school appear lower in district rankings because their scores are averaged with other lower performing high schools in the same district. For this reason, when evaluating the district’s middle and high schools by looking at their statewide rankings as LCUSD does regularly, the district should compare LCHS and LCHS 7/8 test scores against other schools’ test scores, not against averaged district scores.
Additionally, during the Q&A after Hurley’s presentation to the LCUSD Governing Board at its November 15, 2022 meeting, Board Vice-Chair Joe Radabaugh asked Hurley if the data were available to the Board and the community “to allow us to look outside our comparative set of high performing districts” to see which schools were above us and how we can learn from them. Hurley did not answer the question. The district does not make such data available and the California Department of Education’s CAASPP dashboard does not allow school-to-school comparisons based on mean scale scores or “met or exceeded” performance. Let us examine Radabaugh’s question. LCMP’s analysis of all public schools in California reveals hundreds of schools that are outperforming LCUSD’s students in grade level cohorts in mathematics and English language arts. More concerning, evaluating LCUSD elementary schools by the district’s average scores masks issues at individual schools.
Shown below is a table of how all of the district’s schools did by grade level and by CAASPP assessment (i.e. ELA and Math) compared to how the district claimed they did when looking at district grade-level mean scores. Keep in mind that there are more public elementary schools than middle/intermediate schools, and more middle schools than high schools so we show both the ordinal rank as well as the percentile within that grade for better context, along with the number of schools in each grade with a score:
When comparing LCUSD elementary schools, the grade-level cohort with the highest ranking among the three schools is in bold. Note that Palm Crest Elementary (PCR) for the first time outperformed both La Canada Elementary (LCE) and Paradise Canyon Elementary (PCY) in all grades and subjects. In a typical year, top performing grade-level cohorts are more evenly distributed between the three elementary schools. Note also that for the first time LCMP has been analyzing CAASPP scores, an LCUSD grade-level cohort ranked first among all public schools in California. In this case PCR’s 6th graders ranked first in ELA among all 4,173 public schools that reported scores – an extraordinary accomplishment given PCR is a traditional public school and ranked better than specialized magnate and charter schools.
The grade-level cohorts in red are the lowest performing across the entire district in ELA and Mathematics. LCMP highlights these extremes to show that the situation at a specific school can be masked by looking only at the district ranking. For example, LCUSD claims its 4th graders rank 1st in the state in ELA, yet PCR’s 4th graders rank 27th among all 5,741 schools while LCE’s rank 254th. This is worth further exploration because for many years LCUSD parents could trust that their student’s grade-level cohort consistently ranked in the top two percentiles in the state. Currently this is only true for PCR and LCHS.
To answer Board member Radabaugh’s question about finding other schools that outperform LCUSD schools, LCMP shows below the tables of top 30 schools (plus LCUSD’s schools if they fall outside the top 30) in each grade, first by CAASPP Mean ELA scores:
TOP 30 CALIFORNIA PUBIC SCHOOLS BY CAASPP MEAN ELA SCALE SCORE (BY GRADE):
Click on a thumbnail to see the table at full resolution.
TOP 30 CALIFORNIA PUBIC SCHOOLS BY CAASPP MEAN Mathematics SCALE SCORE (BY GRADE):
And below are the top 30 schools in California ranked by CAASPP Mean Math Scale score for 2022 in each grade:
Conclusions
LCUSD’s continued practice of overstating its rankings may not be intentional, and it has made progress since 2019 when it used to conveniently omit stating the limits of its comparison, but the district should altogether stop perpetuating the misconception that it is a “top 3” district. The results from the 2022 CAASPP assessments were actually mostly positive for the district when compared to the broader state averages. Shown below is the historical results of LCUSD schools on the CAASPP mathematics assessment by mean scale score compared to California:
It should be noted that data are not shown for 2020 & 2021 because California schools did not take the CAASPP exam in 2020 due to the COVID-19 pandemic, and school districts were allowed to take a shortened version of the exam in 2021 or take alternate local assessments as LCUSD chose to do. The “∆” column shows the change in mean scale score from the previous test period. Green numbers indicate an increase while red numbers in parentheses represent a decrease in performance. As can be seen in the 2022 mean scale score for California, the CAASPP results were grim, wiping out five years of steady gains since the CAASPP was first administered in 2015. That trend was less prevalent among LCUSD schools. Some schools fared exceedingly well, notably PCR which showed improved scores in all grade levels over the last test period in 2019. However, other grade level cohorts at LCE and PCY, particularly LCE 4th & 5h grades, and PCY 3rd, 5th and 6th grades showed steep declines in mathematics scores.
The unevenness in performance among LCUSD elementary grade-level cohorts is more clearly seen in graphs of the above data:
The real truth is that LCUSD is an exceptionally high-performing district and some of its schools have top 3 performing grade level cohorts in certain CAASPP subject tests. PCR’s #1 rank in 6th grade ELA is a monumental achievement. There is much to be proud of and no reason to exaggerate.
La Cañada Math 