Illustration generated by ChatGPT 4.0.
Published: December 09, 2023
In a press announcement dated October 18, 2023, the La Canada Unified School District (LCUSD) proudly proclaimed that “LCUSD Maintains Outstanding Performance on State Test:”
And in keeping with recent practice, the La Canada Flintridge Outlook Valley Sun newspaper dutifully relayed the district’s message in its October 29, 2023 edition with the headline, “CAASPP Test Scores Released, LCUSD Ranks High.” Both headlines were accurate. And to its credit, LCUSD seemed to have finally stopped the deceptive practice over the past five years of claiming that all of its grades performed in the “top 3 in California” on the English Language Arts (ELA) and Mathematics portions of the California Annual Assessment of Student Performance and Progress (CAASPP).
More impressively, LCUSD ranked first in two metrics when the comparison was widened to all schools and districts. Bafflingly, the district chose to ignore these achievements in its press release, and instead buried the noteworthy accomplishments at the tail end of a presentation to the LCUSD Governing Board at its November meeting.
CAASPP Results: A 10,000-foot View
For context, the CAASPP is administered annually by the California Department of Education (CDE) to roughly 3 million public school students in grades 3 through 8, as well as grade 11. The CAASPP was first administered at the end of the 2014-15 school year and was skipped in 2020 due to the COVID-19 pandemic. In addition, school districts were allowed to take a shortened version of the exam in the second year of the pandemic or take alternate local assessments. LCUSD chose the latter, administering local assessments in 2021. Last year’s CAASPP results for California were abysmal and wiped out all the gains from the previous five years. Shown below are the overall percentage of students in California who met or exceeded grade level standards in ELA and Math mean scale scores from 2015 to 2023:
Note that in the first year of testing after the COVID-19 pandemic — 2022, ELA scores had regressed to a level prior to 2016, and the drop was even worse in Math — all the way back to 2015. Even worse, note that the ELA result for 2023 was worse than 2022. While Math scores rebounded slightly last year, in the words of EdSource they “leave California far behind pre-Covid levels of achievement.”
The post-COVID drop in LCUSD was far more modest:
The COVID drop last year in ELA regressed only to the district’s 2018 level and last year’s improvement returned the district almost to the pre-COVID 2019 level. Meanwhile LCUSD students’ Math CAASPP results dropped far more in the first post-COVID year of 2022, but rebounded far more than in ELA and returns the district’s students almost to their 2019 level. In brief, LCUSD did not suffer nearly as much as California during the pandemic and has recovered much further since. This is the first good news worth celebrating in La Cañada.
The second bit of good news for LCUSD, which has been true for as long as LCMP has been tracking standardized test performance, is that LCUSD vastly outperforms the state average in all subject assessments as seen when plotting the above graphs on the same graph with the vertical axis from 0 to 100% Met or Exceeded:
Reporting Improves from Recent Exaggerated Claims
Before noting further district achievements, it is worth reviewing what the district focused on in its official pronouncements first. LCUSD and the Outlook were much more careful in their claims this year:
In reviewing LCUSD’s performance relative to other school districts within the state, the focus is limited to “unified” school districts that enroll students in grades K-12. Aside from a small 45-student district in remote Northern California that tested only 12 students, LCUSD ranked first among unified districts on the English language arts assessment, boasting an impressive 88.66% of students meeting or exceeding the state target. In mathematics, LCUSD students excelled with 85.12% meeting or surpassing the state-defined standards. LCUSD placed second in the state compared to other unified school districts, by only 0.84% behind San Marino Unified.
LCUSD Press Release, October 18, 2023
Note that the district correctly characterized the comparison cohort — “‘Unified’ school districts that enroll students in grades K-12.” This continues a trend over the last two years of using language to more accurately describe the basis for comparison. To understand why the “top 3 in California” claim made in years prior was deceptive, see previous La Cañada Math Parents (LCMP) articles from 2018, 2019, and 2022.
Unfortunately, LCUSD did not leave well enough alone and at the LCUSD Governing Board meeting on November 14, 2023 the district reverted to its old ways and for most of the report once again limited its comparison to a handful of carefully selected Unified school districts. Further, the district could not repeat the “top 3 in all grades” claim of recent years because it dropped out of the top 3 in grades 3 & 4 ELA and mathematics CAASPP mean scale scores.1 Here are the district’s grade-level mean scale score results on the 2023 CAASPP assessment as reported by district staff at the November Governing Board meeting:
To understand the limited basis of comparison, notice the footnote – “based upon comparable high performing Unified School Districts.” As we’ve elucidated in our past coverage of how LCUSD presents its annual CAASPP results, limiting the comparison to just Unified School Districts is misleading because only 33.7% of public school districts in California are Unified or Joint Unified school districts. The remaining 66.2% of districts are other types of public school districts – elementary school districts, high school districts, joint districts, union districts, and offices of education. There are over 4,000 public schools and county offices of education in California that are not in Unified or Joint Unified school districts and were therefore omitted from LCUSD’s comparison. The district narrows its comparison even further by looking at only fourteen districts it characterizes as “comparable high-performing.”
To reframe the problem we have brought to light in our past articles on the topic, LCUSD ranks in the top 3 among a handful of selected Unified school districts, omitting comparisons with thousands of schools that are contained within other types of school districts. Further, given LCUSD has only one high school in its district, it should compare La Canada High School’s standardized test results against other individual high schools, not multi-high school district averages. While the distinction may seem trivial, a comparison of the district’s claimed rankings versus its actual ranking against all public schools in California reveals the large disparity in rank:
The Actual Rank in the table above was calculated by LCMP based on the raw data files obtained from the California Department of Education.3 The detailed data tables for each grade are presented below so readers can see exactly where each LCUSD school grade-level cohort ranked. Rather than report the mean scale score of LCUSD’s grades as the state does, LCMP rank orders all public schools that had CAASPP scores reported by the state and shows the ordinal ranking of each schools grade-level cohort for a given subject test (i.e. ELA and Mathematics), and for the district rank the three elementary school ranks are averaged together. This methodology normalizes the data to account for differences in the difficulty of the CAASPP assessments from year to year, changes in test conditions such as those caused by policy responses to the COVID-19 pandemic, as well changes in the format of the test like the one that occurred in 2021 when the test was made shorter in length. As can be seen, LCUSD was not top 3 in any grade level in either CAASPP subject area, and compared to 2022 LCUSD’s performance was mixed — it dropped in rank in six grade-level cohorts, but increased in eight others.
Why Limit Comparisons to Unified Districts?
Given LCUSD fares well when compared to all schools and districts, why does LCUSD limit its annual CAASPP analysis to only other California Unified districts? Clearly the district broadens the claim when it suits them. For example, during its CAASPP presentation to the Governing Board on November 14, 2023 the district presented two new slides on 2023 CAASPP & CAST Celebrations, and noted LCUSD ranked in the top 3 in ELA mean scale score in grades 5, 7, and 11 “compared to all CA public school districts in the state” and that the Palm Crest Elementary (PCR) 6th graders finished 1st place in ELA “compared to all public schools in CA:”
This would seem to indicate that the district’s motive is to make itself look better than it actually is, at least in the realm of annual standardized test performance. After all, parents are the customers of public schools and many parents move to better neighborhoods in the hope of sending their kids to higher quality public schools. When the district shows favorably in a broad comparison – bring attention to it. When it doesn’t, change the basis of comparison until it does. What is especially vexing about this rationale is that the district’s accomplishments when compared against ALL public school districts are quite extraordinary as the next section highlights.
When asked in the past why LCUSD chose to restrict its statewide comparison to just a limited number of Unified school districts, administration initially claimed that they were just following the practice of individuals who held their administrative position previously. When it was pointed out there is no policy or obligation to keep with past practices, administration changed its explanation by stating that the selected comparison districts had similar student demographics, school size, and socioeconomic status. As LCMP explained last year about this practice, “the latter explanation would make sense if widening the comparison led to misleading conclusions, but the opposite motive seems to be at play here – narrowing the comparison inflates the status of LCUSD schools.” And as we pointed out last year, we tested the district’s second explanation by examining the metrics of the fourteen Unified school districts LCUSD chooses to compare itself to versus the other non-Unified districts omitted from the comparison that perform better than LCUSD in actual rankings. We found that the non-Unified districts were more similar to LCUSD than the 14 districts LCUSD chose to compare itself to in all but two metrics:
And here are the same metrics of the other districts omitted from the comparison that eclipsed LCUSD in actual rankings when all districts in California are included in the comparison:
To reiterate our conclusion from last year, there is no statistical data to support the district’s claim that the districts it chose to compare itself against are more similar to LCUSD than the higher-performing non-Unified districts they omitted.
The third explanation offered by the district defending its decision to limit its CAASPP comparison to just a handful of Unified school districts was offered by a current LCUSD Governing Board member in response to our past articles. He stated that limiting the comparison to just Unified school districts is appropriate because Unified school districts cover all pre-college grades K-12 and non-Unified school district include only subsets of that grade range, and it is better to compare against districts that are administratively similar in grades covered. This justification would make sense if a district’s CAASPP scores are computed based on an average of grade-level mean scale scores. But they aren’t. The CDE provides both grade-level mean scale score CAASPP data as well as quartile performance (e.g. “% of students who Meet or Exceed grade level standards.”) Mean scale scores should not be averaged across grades or compared between grades as the scale score ranges differ by grade, and the ranges change every year. So district metrics are calculated solely on the % of students Met or Above grade level standard.
As we stated last year, “schools in the five major different types of districts are indistinguishable from each other on the outside — their students are held to the same state education standards, take the same standardized tests, use the same types of instructional materials, and hire from the same pool of certificated teachers.” Parents looking to move into public school districts so their children can access higher quality schools care not whether the districts that contain those schools are Unified or non-Unified. What matters is how individual schools in those districts perform.
Noteworthy Accomplishments Buried
New to the district’s annual report to the school board but omitted from its press release were the following slides that highlighted new accomplishments that should have been more widely announced. The first of those new slides we’ve already mentioned above:
All of the above claims are true. In fact, some of the celebratory accomplishments are unprecedented. The fourth bullet in slide 15 of Hurley’s presentation to the Board was made almost in passing:
“Compared to all public schools in CA, PCR 6th grade ELA mean scale scores placed them 1st in the state.”
To do this once would be exceptional. After all, this means Palm Crest Elementary (PCR)’s 6th graders scored higher on average on the 2023 ELA portion of the CAASPP exam than 4,318 other public schools in California, including specialized schools like charter schools, magnet schools, and college prep academies co-located on college campuses. In the ten plus years LCMP has tracked LCUSD’s state assessment performance data, we’ve observed this feat only one other time – last year, and again by PCR 6th graders in ELA. So the PCR 6th grade teaching team and its students placed first two years in a row. Achieving such a rare accomplishment two years in a row is especially difficult because it involves an entirely different cohort of students, and neither cohort ranked 1st in prior grades. Clearly PCR’s 6th grade teachers deserve special recognition – Amy Tsai, Jane Chang, and Debbie Au:
LCUSD should have recognized the PCR 6th grade teaching team with a special award for its extraordinary accomplishment last year, but failed to even inform the teachers about their feat. This year they finally recognized the PCR 6th grade team when they did it again (and parents brought the feat to the attention of the cabinet), but it was mentioned only in passing during a Board presentation.
The stubborn insistence by the district to stick with its past practice of limiting comparison to 14 “comparable high-performing” Unified districts on CAASPP performance is perplexing. How top 3 among 15 cherry-picked districts is considered better than 1st among 4,318 other schools will ever remain a mystery.
Comparing Against All Districts
Given there still remains no good reason to limit comparing LCUSD against just Unified school districts, LCMP conducted its own analysis of CAASPP data and re-ran the district’s comparisons against districts of all types again. The results were actually quite stellar, which again raises the question of why limit comparisons to just a handful of Unified districts?
CAASPP Mean ELA Scale Scores – Districts:
As a reminder, here is LCUSD’s slide comparing the district against their carefully selected “high-performing comparable” Unified school districts by CAASPP ELA mean scale score:
LCUSD shows the #1 district in green, the #2 district in yellow, and the #3 district in orange. Note that LCUSD dropped out of the top three in 3rd and 4th grades. Here are the actual results comparing LCUSD against all districts in California, not just Unified districts, using the same color coding scheme:
LCMP generated the above table by ranking all California public school districts by ELA mean scale score for each grade, then averaging the rank across all grades taught in that district. Last year LCUSD ranked 7th in this all-district ranking. This is a laudable improvement over last year.
CAASPP Mean Math Scale Scores – Districts:
Shown below is a repeat of the slide LCUSD staff presented at the Nov. 14, 2023 LCUSD Governing Board meeting, showing LCUSD compared against fourteen “comparable high-performing” unified school districts by CAASPP Mathematics mean scale score:
Though the above table is sorted alphabetically with the exception that LCUSD is listed at the top, a quick comparison of top 3 placings, highlighted in green for 1st, yellow for 2nd, and orange for 3rd, shows LCUSD in 2nd place behind San Marino. So in the above comparison LCUSD is 2nd out of 15 districts with 778 other districts omitted. And below are the LCMP-compiled results comparing LCUSD against all districts in California in ELA, not just Unified districts:
Last year LCUSD ranked 11th in this all-district math ranking, so it improved one spot from 2022. To summarize, when using LCUSD’s method LCUSD ranked 2nd among 15 (i.e. 86th percentile) “comparable high performing Unified school districts.” When compared to all districts in California LCUSD ranked 10th among 793 districts (i.e. 98th percentile.)
All School Comparisons
La Cañada Math Parents has regularly extended our CAASPP analysis to the school level since it shows how the district’s three elementary schools do relative to one another, and how they compare to the top schools in the state. One problem with the way the district reports its CAASPP scores is it averages the scores across the three elementary schools. This masks weaknesses at individual schools, and also downplays remarkable achievements at the highest performing school as seen above in PCR’s 6th grade ELA performance. The last reason LCMP publishes school rankings is because LCUSD is a “boutique” district — it is unusually small with just one high school, whereas 94% of California public high schools are in districts that include two or more high schools. And as we explained in our 2019 article “When Top 3 Isn’t Top 3” almost all high-performing California public high schools reside in districts with two or more high schools. Thus LCHS and LCHS 7/8 look better when comparing them by district mean scores than when comparing by school mean scores. High-performing high schools in districts with more than one high school appear lower in district rankings because their scores are averaged with other lower performing high schools in the same district. For this reason, when evaluating the district’s middle and high schools by looking at their statewide rankings as LCUSD does regularly, the district should compare LCHS and LCHS 7/8 test scores against other schools’ test scores, not against averaged district scores. LCUSD now seems acutely aware of this distinction as it reports top individual school rankings when they are better than the district ranking.
The California Department of Education’s CAASPP dashboard does not allow school-to-school comparisons based on mean scale scores or “Met or Exceeded” performance, which is why LCMP conducts its own ranking of all public schools in California. Doing so reveals dozens of schools that are outperforming LCUSD’s students in grade level cohorts in mathematics and English Language Arts.
Shown below is the table of how all of the district’s schools did by grade level for each CAASPP assessment (i.e. ELA and Math) in 2023 compared to how the district claimed they did when looking at district grade-level mean scores in its more limited comparison:
When comparing between LCUSD elementary schools, the grade-level cohort with the highest ranking among the three schools is shown in bold. Last year Palm Crest Elementary (PCR) for the first time outperformed both La Canada Elementary (LCE) and Paradise Canyon Elementary (PCY) in all grade level cohorts. In a typical year, top performing grade-level cohorts are more evenly distributed between the three elementary schools. This year LCE’s 4th graders eclipsed PCR (and PCY.)
The grade-level cohorts in red are the lowest performing across the entire district in ELA and Mathematics. LCMP highlights these extremes to show that the situation at a specific school can be masked by looking only at the district ranking. For example, LCUSD claims its 3rd graders rank 6th in the state in Math, yet PCR’s 4th graders rank 37th among all 5,788 schools (i.e. 99.4th percentile) while PCY’s rank 346th (only 94th percentile.)
Shown below are the tables of top 30 schools (plus LCUSD’s schools if they fall outside the top 30) in each grade, first by CAASPP Mean ELA scores in 2023:
TOP 30 CALIFORNIA PUBIC SCHOOLS BY CAASPP MEAN ELA SCALE SCORE (BY GRADE):
Click on a thumbnail to see the table at full resolution.
TOP 30 CALIFORNIA PUBIC SCHOOLS BY CAASPP MEAN Mathematics SCALE SCORE (BY GRADE):
Below are the top 30 schools in California ranked by CAASPP Mean Math Scale score for 2023 in each grade:
Final Observations
LCUSD’s CAASPP results for 2023 were solid, certainly far better than they were for California as a whole. The district’s grade-level cohort rankings improved in eight grade-level cohorts, while dropping in six using LCMP’s ordinal ranking methodology. And the crowning achievement was PCR’s 6th grade ELA CAASPP ranking first in the state among ALL schools for a second year in a row. Lastly the district has made steady progress since 2019 in not overstating its performance. If there was one area of concern, it is that the 3rd and 4th grade scores lagged all other grades. In an embarrassment of sorts, the district can no longer claim that all grades performed in the “top 3” among the cherry-picked 14 “comparable high-performing unified” districts.
For consistency, we show below the mean scale score for all LCUSD schools at each grade level, with the California score at the top for comparison purposes:
Remember that data are not shown for 2020 & 2021 because California schools did not take the CAASPP exam in 2020 due to the COVID-19 pandemic, and LCUSD administered local assessments in 2021 instead of CAASPP. The “∆” column shows the change in mean scale score from the previous test period. Green numbers indicate an increase while red numbers in parentheses represent a decrease in performance.
Overall LCUSD’s 2023 CAASPP results were excellent and demonstrate that the district remains an exceptionally high-performing district with some of its schools top 3 in certain grade-level cohorts. PCR’s #1 rank in 6th grade ELA for a second year in a row is an unprecedented achievement. And we applaud the district for being more honest in its CAASPP performance claims and hope once and for all they end the short-sighted practice of limiting comparisons to a handful of cherry-picked districts.
- Last year and in 2019 all LCUSD grade-level cohorts ranked first, second or third in the district’s limited comparison to “comparable high-performing districts.” ↩︎
- The absence of data for 2020 and 2021 was due to the CAASPP exam being canceled by the CDE in 2020 at the start of the COVID-19 pandemic, and the fact that in 2021 the CAASPP exam was made optional to California schools since most schools chose to hold classes remotely during the 2020-21 school year. During the 2020-21 school year, LCUSD chose not to take the shortened 2021 CAASPP assessments and used local assessments instead. ↩︎
- Readers can download the data for themselves from the CDE’s CAASP Research Data Portal. ↩︎
La Cañada Math 