This is the 4th in this year’s series of posts by PhD students on the job market.
In my job market paper, I develop a method that makes the assessments schools already set and grade comparable across schools, without a new assessment. Any school system that records students’ grades and follows each student between schools can then compare learning across all its primary schools, every year. Using South Africa’s school administration records, I apply the method to 1.2 million students as they move from primary to secondary school.
When education economists evaluate a policy, intervention or shock, we usually care about how learning was affected, yet across the developing world usable, comparable measures of learning are hard to come by. The World Bank’s flagship education review recommends “more and better measurement”. The evidence suggests the binding word is better: Bergbauer, Hanushek and Woessmann (2024) find that assessments are associated with learning gains, typically of about a year of schooling, only when they let schools be compared with one another, which is what lets parents, schools and governments see who is falling behind and act.
Yet, at most a quarter of countries test every student on a common assessment at the end of primary school. In most countries, South Africa among them, the only primary school results are schools’ own grades, so the same grade represents different amounts of learning at different schools. In the same review, the Bank lamented that there is “no effective way to aggregate the results of classroom-level formative assessment by teachers into … reliable system-level information”. This is exactly what my paper provides.
Design
I estimate a two-way fixed-effects model, adapted from the labour model of workers moving between firms (Abowd, Kramarz and Margolis 1999), with students as workers and schools as firms: it regresses students’ secondary school grades on their primary school grades, with fixed effects for both the primary school they left and the secondary school they entered. Recovering learning outcome comparability requires common assessment across students, yet my method relies only on local commonality: students from multiple primary schools writing the same assessment at a given secondary school, and students from one primary school writing the assessments of several secondary schools.
Figure 1 illustrates how the model measures the difficulty of each primary school’s assessments. Two (representative) students enter Secondary 1, one from Primary A with a primary school grade of 58%, and one from Primary B with a grade of 72%. They write Secondary 1’s assessments soon after arriving. They both achieve 65% (in expectation). Had the two primary schools set equally hard assessments, B’s student, who left with the higher grade and so had learned more, would have scored higher at Secondary 1 too (an assumption I test). Instead, their identical performance at Secondary 1, despite different primary school grades, reveals the gap in difficulty between the two primary schools’ assessments, measured by each school’s difficulty wedge (αj). I normalise Secondary 1 as the reference secondary school (γ1 = 0) and measure wedges against the average primary school. Equating each student’s primary and secondary grades requires an additive wedge of αA = +7 for Primary A and αB = −7 for Primary B. A positive wedge denotes a primary school with harder assessments; a negative wedge, one with easier assessments.
While this calibrates primary schools feeding into a single secondary school, linking across secondary schools requires students from the same primary school branching into different secondary schools. Consider a second student from Primary B (also leaving with 72%) who attends Secondary 2 and achieves 60%. Because the primary wedge is a school-level parameter fixed at αB = −7, equating 72% and 60% requires subtracting a further 5 percentage points. This defines Secondary 2’s destination effect (γ2 = −5), where a negative value means students score lower there than at the reference school.