The Georgia Department of Education released 2021 Milestones results on Monday, August 16. As Dr. Herring described in her blog, the test results measure student achievement after an unusual and challenging period of learning. APS students were remote for the last 2.5 months of spring 2020 and all of fall 2020. Families then had a choice of remote or in-person during spring 2021. Milestones exams were administered in late April and early May 2021.

The graph below shows Milestones math proficiency by race/ethnicity for students in grades 3-8.

Use the “Comparison” option to view the results by other groups. The “Poverty” comparison shows results for students who directly certified for free lunch compared to those who did not.

In addition to the challenges of remote learning, only 34% of enrolled students in grades 3-8 took the exams. Most students who were remote during the spring 2021 semester opted to not take the exam, as well as some students who were receiving in-person instruction. GADOE reports test results and take-rates by school here.

The Milestones exam was not administered in 2020. The low exam take-rate this year means that we cannot directly compare the proficiency rates to 2019 as a measure of pandemic-related learning loss because the sample might not be representative. To account for this, the graph below shows 2019 and 2021 proficiency rates for students who took both exams¹.

The graph shows that the math proficiency rate for Black students, the largest racial group in APS, fell 18 percentage points from 25% to 7%. The math proficiency rate for White students, the second largest racial group, fell 20 percentage points from 84% to 64%. Use the subject and comparison filters to see additional data.

The proficiency rate drops by race/ethnicity are much larger than the district’s overall change in proficiency. The overall proficiency rate for math fell 7 percentage points, from 35% to 28% and the ELA proficiency rate fell 1 percentage point, from 37% to 36%. The overall drops were smaller because the composition of students taking the exams changed, an example of Simpson’s Paradox. In 2019, 74% of tested students were Black, and 15% of tested students were White. In 2021, 56% of tested students were Black, and 30% were White.

The different composition of test takers in 2021 is due to differential rates of in-person schooling and test taking by school. Some schools in high-income neighborhoods, such as Brandon, WT Jackson, and Mary Lin had test take-rates above 80%, while most APS schools in low-income neighborhoods had test take-rates below 40%.

Proficiency rates are helpful for understanding relative achievement levels, but sometimes the change in proficiency rates can create a misleading comparison. For example, the maximum amount that black math proficiency could have fell was 25 percentage points, and the binary proficient/non-proficient doesn’t tell us how much already non-proficient black students changed.

The next graph uses change in average scale score to better show relative change. Average scale score is impacted by changes from any student, regardless of their starting point².

The graph shows that all student groups had larger drops in math scores than ELA. Black students had the largest drop in math scores, at 30 scale score points. White and Multiracial students shared the largest drop in ELA scores at 20 points.

The “Change in Average Scale Score” graph also has an X-axis option to toggle from scale score to z-score. Z-score is a statistic that allows comparisons across exams by standardizing the scale score. See here for more details. Switching to Z-score slightly increases the size of math loss relative to ELA because the math exam tends to have a larger standard deviation than the ELA exam.

The drops in student achievement detailed in this blog post emphasize the need for academic recovery. Dr. Herring details the APS Academic Recovery Plan in her Milestones blog post.

To get email alerts for similar posts, please subscribe to the blog.

1. Student scores were adjusted slightly to equalize scores across grades within year. The 2019 to 2021 comparison is comparing student proficiency in grades 3-6 in 2019 to to their proficiency in grades 5-8 in 2021. Differences in baseline proficiency rates in these grade bands could bias the analysis. We used within-year differences in the state average scale score by grade to adjust test scores.
2. The Milestones is rescaled at each grade-level so that 475 is the cutoff between beginner and developing, and 525 is the cutoff between developing and proficient.