Supplemental instruction (SI) is a widely implemented peer-led academic support program. Assessments of SI's effectiveness are often confounded by student ability, achievement, motivation and demographic characteristics, and frequently limited by available resources. We modelled the impact of SI attendance on the grades, pass rates and retention of over 4000 university students. Percentage grade increases of up to 10.4% were associated with a full semester of SI attendance, and largest for males, younger students and those who had not previously engaged with the program. Furthermore, pass rates were 10% higher for full attendees. Finally, full attendance was associated with a 3% and 9% increase in the probability of reenrolling the following year for students who had and had not previously engaged, respectively. These estimates were calculated controlling for measures of ability, achievement and motivation, plus several demographic factors. Our methods can be applied to evaluate any opt-in academic support program.