Waterford Upstart and the What Works Clearinghouse
What is the What Works Clearinghouse? Run by the US Department of Education, the What Works Clearinghouse (WWC) catalog categorizes studies of learning programs, following their own educational standards and criteria. These rankings are shared with educators to provide an unbiased indication of a program’s effectiveness, allowing them to know what program will work best in their schools.
WWC coordinates with the Every Student Succeeds Act (ESSA) of 2015. The What Works Clearinghouse provides schools with guidance on intervention-based programs by assigning them to ESSA Tiers of effectiveness. According to the research available, the WWC organizes programs into the following categories:
- ESSA Tier 1 — Strong Evidence; at least one high-quality randomized control experimental study shows evidence that the program is effective
- ESSA Tier 2 — Moderate Evidence; at least one high-quality quasi-experimental study shows evidence that the program is effective
- ESSA Tier 3 — Promising Evidence; at least one high-quality correlational studies shows evidence that the program is effective
ESSA Tier 4 — Demonstrates a Rationale; the program has a well-defined theory of action and an effort to measure its effectiveness is currently underway
Only programs that meet these criteria are assigned an ESSA Tier.
The Waterford Upstart reading program, which teaches students early literacy skills through adaptive curriculum, meets ESSA Tier 1 standards according to the WWC. This evaluation was based on a 2016 study of the program conducted by ETI Consulting. But which standards does Waterford Upstart meet to earn this categorization?
High-Quality Study Design
For a program to meet ESSA Tier 1 standards, it must be both well-designed and well-implemented. For example, it must have a fully randomized assignment between the control and intervention group. This helps ensure that differences observed between students are primarily due to their group assignment.
The Waterford Upstart study evaluated by the WWC (available here) included two groups: one intervention group, which received access to the Waterford Upstart Early Reading curriculum, and one control group, which received access to the Waterford Upstart Math & Science curriculum.
These groups were both assigned randomly, with the only difference being the curriculum available to them. They were both evaluated on their early literacy abilities at the beginning and end of the study.
Strong Sample Size and Setting
ESSA Tier 1 programs must have a sample size of at least 350 students from more than one school district. There can be no significant differences in demographics between the control and intervention group.
ETI Consulting evaluated 497 students from eighteen school districts as part of the Waterford Upstart study. All lived in a rural Utah area, with over 80% of these students coming from families living in poverty levels at or under 185%. Over 95% of the students’ caregivers had graduated high school, and 38% of the caregivers had earned a bachelor’s degree.
No significant differences in demographics were found between the control and intervention group.
Significant Positive Effects
Programs that are assigned an ESSA Tier 1 categorization must show a statistically significant positive effect on a relevant learning outcome. This is evaluated using the WWC Improvement Index, which indicates how large the intervention group’s improvement was compared to the control group as a percentile.
The Waterford Upstart study evaluated literacy improvement through student scores on the Brigance Inventory of Early Development. The WWC assigned the program an Improvement Index of 14, indicating a significant and effective impact on early literacy.
No Strong Negative Findings
Finally, all ESSA Tier 1 programs must show no negative impact on the learning measurements studied. Students who completed the Waterford Upstart study saw no negative impact on early reading skills. On average, the students in the intervention group saw significant literacy growth.
Follow the link to view the full WWC evaluation. Plus, learn more about how Waterford Upstart teaches children early literacy skills at home.
