Deep Dives

The Issue

It’s that time of year when many people have to make some important decisions: Should the turkey be roasted or deep fried? Who should make the potato salad? And for many teachers, is this intervention working, or do I need to change something? Of course, this important decision can’t be made until after you have screened your students, identified their instructional needs, set appropriate goals, provided high-quality instruction to help them meet those goals, and collected data on a regular basis to track their progress. Now with all this data, you are prepared to determine if your students are making enough progress. What may be less clear, however, is how you make this determination. This is where decision rules come into play. By applying decision rules, teachers can work with other members of the data team to interpret their students’ data and assess whether their instructional practice should change or be maintained. Now, whether this concept is new to you or you were already familiar, one important consideration is that there isn’t just one decision rule to choose from. Naturally, this may raise questions: Is one decision rule better or worse than the others? How do the rules differ? And perhaps most importantly, how do the different decision rules influence your instructional decisions? To answer these questions, it’s important to examine what the research says about the different decision rules used in progress monitoring.

The Research

Collecting progress monitoring data is an essential component of the response to intervention (RTI) and multi-tiered systems of support (MTSS) frameworks, as it is used to inform decisions about student progress (Parker et al., 2018; Stecker et al., 2008). Commonly, Curriculum-Based Measures (CBMs) are used as a reliable and valid means of evaluating this progress, as they are sensitive to changes over time (Ardoin et al., 2013; Hintze et al., 2018). When examining student progress, teachers should collaborate with members of the data team (e.g., reading specialists, speech-language pathologists, school psychologists) to make informed decisions about their instructional practices. For instance, when progress monitoring data reflect that students are adequately responding to instruction, the data team may conclude that the current instructional practices are effective and should be maintained. When students have made sufficient progress toward their goals, the data team may suggest reducing or discontinuing supplemental support and redirecting those resources to students with greater instructional needs. When students are not making sufficient progress, the data team may determine that the intervention is insufficient and should be changed so that students don’t continue to fall further behind (Fuchs & Fuchs, 2017).

The determinations for what adequate progress looks like compared to inadequate progress depend on the guidelines used to assess the data; these guidelines are the decision rules. 

Decision rules typically use student data plotted on a chart for visual analysis. The x-axis reflects the time, usually measured by the number of weeks, and the y-axis displays the skill being monitored, such as the number of words read correctly per minute (WRCM). A goal line (also known as an aimline) is also plotted to reflect students' trajectories toward their performance goals. The criteria set by a decision rule are then applied to interpret students’ progress. There are several options for data teams to make these decisions; however, they commonly conclude that patterns falling below the goal line reflect insufficient progress, patterns falling in line with the goal line reflect adequate response, and patterns exceeding the goal line reflect that the student is making sufficient progress. However, despite sharing these criteria, each rule differs in how the data is used to compare to the goal line, and depending on the circumstance, different rules may be more or less of an accurate reflection for when a change is actually warranted. This is because the accuracy of the recommendations is influenced by several factors such as how long the data was collected for, variability in student performance (e.g., if the child was feeling ill or distracted on one of the days data was collected), and the validity of the student's baseline performance. Therefore, no single decision rule would be considered a one-size-fits-all solution for making accurate decisions from the data; instead, more understanding of their individual strengths and weaknesses is needed.

Though several options exist, two of the most common rules used within MTSS are the data-point and the trend-line rule (Ardoin et al., 2013). Additionally, guidance from the National Center on Intensive Intervention (2024) includes the median rule, which is arguably a type of data-point rule. The data-point rule, as the name suggests, uses a set number of consecutive data points that fall above, below, or in line with the goal line to make determinations about students’ progress. Researchers have examined several variations of the data-point rule, including as few as three data points to as many as six, but the general rule of thumb is to have at least four (Van Norman et al., 2024). Overall, the data-point rule is more susceptible to inaccuracies as compared to the other decision rules. However, it does have the benefit of being simple to visualize and easy to apply. Consequently, this simplicity is what contributes to it being susceptible to inaccuracies. For example, if a student was adequately responding to the instruction they received, but during the last three weeks of progress monitoring they read passages that were significantly more difficult and were battling a cold, one can imagine that there may be a lot of variability in the recorded scores. This variability isn’t so much a reflection of their actual reading progress, but if using the data-point rule, those last three scores may fall below the goal line, and a data team may falsely conclude that the instruction that was in fact working now needs to be changed. Amid these concerns, research evidence suggests that the data-point rule is most useful when data teams collect at least 10 weeks of data when using curriculum-based measures of reading (CBM-R; Van Norman et al., 2024) or in instances when the growth in students’ progress is substantially higher or lower than the goal line (Parker et al., 2018; Van Norman & Christ, 2016; Van Norman & Parker, 2017). In the latter condition, if student data is much higher than or much lower than the goal line, the decisions are more accurate, as temporary fluctuations are less likely to make a significant impact.

The median rule is a bit different from traditional data-point decision rules in that it uses the last three data points ordered from largest to smallest, and selects the middle number, or the median, to compare to the corresponding score from the goal line (Good et al., 2015; Parker et al., 2018). If the median is below, then a change is needed; if it is above, the goal is intensified; and if it is the same, then the intervention should continue as usual. The median rule has been shown to outperform the data-point rule in accurately identifying children who did need an instructional change because they were not making sufficient progress (i.e., a true positive). However, the median rule may also overestimate the number of children who need an instructional change when they are actually making sufficient progress (i.e., a false positive), especially if data isn’t collected for a sufficient amount of time, which ranges from 10 weeks (Van Norman et al., 2024) to 18 weeks (Parker et al., 2018). Though the median rule shows promise in supporting accurate instructional decisions, data teams should still draw upon their expertise, especially when decisions are more high-stakes, like referrals for special education services (Christ et al., 2013).

Lastly, the trend-line rule is a well-supported approach that uses statistics to estimate a line of best fit that considers all of the available data for the student. Much like the other decision rules, data teams base their decision on whether the slope of the trend line falls above, below, or in line with the slope of the goal line (Van Norman et al., 2024). Though earlier studies examining CBM-R for elementary students have found that the trend-line rule shows promise in accurately recommending whether a change is needed, it is similar to the median rule in that it also requires more data than teams typically assume is necessary to provide reliable results (Christ et al., 2012; Christ et al., 2013). Across studies, the median rule and trend-line rule perform similarly, though in some instances, more weeks are needed to supply accurate information about predicting true and false positives (Parker et al., 2018). Trend-line rules work best when there is at least 10 weeks of data (Van Norman et al., 2024); however, the decisions from this rule are less reliable when the data includes values that are extremely high or extremely low (Nelson et al., 2017). Similar to the data-point rule, if the trend line is much higher or much lower than the goal line, decisions are more accurate (Parker et al., 2018; Van Norman & Christ, 2016; Van Norman & Parker, 2017).

Across each of the decision rules, considerations for accuracy must be made in conjunction with how students’ goal lines are established, given that the goal line reflects students’ expected performance. For example, when monitoring WRCM, some goal lines may be set using a rate of improvement standard of 1.5 WRCM per week, whereas other approaches may use the standards provided by assessment platforms. Another approach is to set the rate of improvement by subtracting the student’s baseline score from their benchmark score and dividing that by the number of weeks (Parker et al., 2018; Shapiro & Guard, 2014; Van Norman et al., 2018).

Because these approaches set different expectations for target growth, the progress monitoring data will relate to each of these expectations differently.

For example, if a student is scoring far below the benchmark, then a goal line based on an end-of-year benchmark would have a steeper rate of improvement than a set rate of improvement goal, making it more difficult to detect progress. Conversely, using a goal line that uses normative data from similarly performing peers may better reflect a student’s response to instruction. Therefore, by changing the way the goal line is determined, the same progress monitoring data may lead to different instructional decisions even if the same decision rule were used.

An additional consideration must be made for the role of measurement error and the variability in student growth over time. The underlying assumption when assessing progress over time is that growth occurs in a constant, linear fashion; however, this conceptualization is oversimplified, as students’ growth occurs more sporadically. Students may be more responsive to instruction once they’ve had an opportunity to get into a routine, or have spent more time with the material. Being that instructional decisions rely on a snapshot in time to track progress, it may be more difficult to capture these fluctuations, which, in turn, complicates the decision accuracy given their sensitivity to performance at specific time points (Van Norman et al., 2018). Further, measurement error introduces an additional challenge. Progress monitoring measures such as WRCM assume that alternate passages are comparable in their difficulty level, but the passage used to record data at time point one may be more difficult than the passage used at time point four. As such, it is difficult to determine what changes are reflective of the variability in the measures used as opposed to the student’s actual progress (Hintze et al., 2018).

Lastly, it is important to consider that decision rules are commonly evaluated by their accuracy. However, accuracy can be thought of as 1) correctly predicting whether students will meet their spring benchmark (e.g., Parker et al., 2018), or 2) correctly identifying when an intervention should be maintained or changed (e.g., Van Norman & Christ, 2016). This is an important distinction as accurately predicting future performance doesn’t necessarily mean that the instructional recommendations are appropriate. Both conceptualizations are represented in the literature, making comparisons across studies more difficult. Nonetheless, data teams should be clear on their purpose for progress monitoring, particularly when selecting decision rules, as their determinations for accuracy may be impacted by this.

In Sum

Though the research guidance on which decision rules to use when making decisions about instructional changes is still emerging, data teams can rely on some key commonalities to inform their decisions. The accuracy of the data-point, median, and trend-line rule generally improves when the data are collected for at least 8-10 weeks (though some conditions require more time), and when a student's rate of improvement appears significantly higher or significantly lower than the goal line. Though the accuracy of a decision rule can be influenced by many factors, they are still an essential component of the MTSS and RTI process, as they help data teams be responsive to students' data. When applying decision rules, special attention should be given to the manner in which the goal line is set, the potential for measurement error, and the variability in how students progress. Because inaccurate decisions may result in teachers unnecessarily continuing supplemental instruction for students who no longer need it, prematurely discontinuing support for students who are benefiting from it, or delaying instructional adjustments for students who require more intensive support, it is important that data teams exercise their professional judgment when interpreting progress monitoring data.

Questions we just don’t have the answers to:

1. How much influence does the type of progress monitoring data collected (e.g., CBM-R, CAT, multi-component) have on the accuracy of a decision rule?
2. Does the accuracy of a decision rule vary across different CBM-R measures such as WRCM or Non-word fluency?
3. Which decision rules work best for older students who may not be using CBM-R for progress monitoring?
4. How do the different variables that impact instructional decisions, such as how the goal line is selected, a student’s initial performance, or the decision rule applied, interact with one another?
   1. Is one factor more predictive of accuracy than the others?
5. Do instructional decisions change when they use automated tools versus manual calculations?
6. When looking at the accuracy of decision rules, is it necessary to account for school-level factors such as the type of school, or characteristics of the teacher?

Nitty-gritty of some of the research:

Van Norman, 2026

Participants

• A database of 1,456 6th-grade students from 35 states

Study Design

• This study examined whether different types of progress monitoring assessments impacted instructional decisions when using the trend-line rule. Specifically, Passage Oral Reading Fluency (PORF) data were used to assess progress monitoring from CBM, and Star Reading (SR) data were used to assess progress monitoring from Computer Adaptive Tests (CATs). The study evaluated how much time was needed to provide accurate recommendations when making lower- and higher-stakes decisions such as referring for an evaluation.

Findings

• When applying the trend line decision rule, the recommendations to change or maintain an intervention, whether using PORF or SR were equal to chance level. Further, at least five months of data were needed to make accurate recommendations when making low-stakes decisions. The authors concluded that the trend line decision rule may be insufficient for accurately determining whether students should change or maintain their instruction, particularly when using PORF or SR as the progress monitoring tool unless they score just below the benchmark. When initial scores fall significantly below the benchmark, alternative options should be considered.

Notes/Limitations

• The authors highlight that future studies should explore how the type and intensity of the supplemental support that students receive affect decision rule accuracy. This study examined decision rules for 6th grade students, whereas the majority of the literature has focused on elementary students, so generalizability across ages remains uncertain.

Van Norman et al., 2018

Participants

• A database of 1,820 3rd-grade students who participated in Tier 2 reading interventions across 290 schools in the upper Midwest.

Study Design

• This study used simulations informed by previous CBM-R progress monitoring data to determine how the three-point data-, median, and trend-line rules were influenced by students’ initial performance, the way the goal line was determined, and the duration of progress monitoring for students receiving Tier 2 interventions. Students were grouped in profiles based on their initial reading performance (i.e., very low, low, or average). Goal lines were either 1) fixed to reflect what vendors typically used as a criterion for typical rate of improvement, 2) normative using the rate of improvement guidance from AIMSweb, or 3) benchmark based on calculation of the weekly rate of improvement needed to reach benchmark. Progress duration spanned from 4-16 weeks.

Findings

• The data-point rule performed poorly across each of the conditions, while the median and trend-line rules performed similarly well in recommending accurate decisions. Out of the different goal line options, the normative goals were most appropriate when students initially scored significantly below the benchmark. The benchmark goal line was most appropriate when students’ initial score fell just below the target. In most conditions, the median rule provided strong evidence of decision accuracy after 10 weeks. When using an end-of-year benchmark, or when a fixed goal line was applied, students who scored far below benchmark rarely demonstrated sufficient growth to reach the target.

Notes/Limitations

• Though the findings concluded that the accuracy of decisions depended on a combination of initial performance, the way the goal line was determined, and duration, the influence of each of these factors remains unclear. It is also important to note that this study relied on the use of simulations of student performance, which may differ from using actual data. Lastly, the authors used a three-point data-point rule despite previous research establishing three points as being insufficient, so results may have differed if more data points were used. Given that different goal lines produce different expectations, the authors suggest using a fixed goal line as an ambitious target, a benchmark goal for students scoring in the average range, and a normative goal line as a standard to evaluate the relative degree of responsiveness in comparison to similarly performing peers.

Parker et al., 2018

Participants

• A database of 4,500 2nd-3rd grade students receiving tier 2 reading intervention.

Study Design

• This study examined the accuracy of the data-point, trend line, and median rule in predicting reading outcomes. The authors evaluated how the data-point, trend line, and median decision rules performed across different durations ranging from 8-28 weeks, when looking at WRCM in higher-quality and lower-quality conditions. Goal lines were generated by subtracting students’ fall screening score from their spring benchmark score to calculate a rate of improvement. Data quality was based on the WRCM standard error of estimate (SEE) such that values above 10 were considered “poor quality”, whereas values below 10 were considered “good quality”.

Findings

• The data-point decision rule performed poorly in identifying students who would meet the benchmark across conditions; however, it was strong in identifying students who would not meet the benchmark. The trend-line and median rule performed similarly and accurately identified students who would meet the spring benchmark around 18 weeks. Conversely, these rules were not as accurate at identifying students who would not meet the spring benchmark for this duration, but the median line rule was more accurate than the trend line. The quality of the data set had more of an effect on students in 3rd grade as opposed to students in 2nd grade.

Notes/Limitations

• Each of the decision rules had concerns with consistently producing accurate recommendations when the duration for data collection was less than 18 weeks. Specific to the data-point rule, the authors only used three points, though standard recommendations suggest at least using four points, if not more. Teachers should be aware that shorter periods incorrectly suggest that the intervention was effective when it was not. Importantly, these results were collected using the spring benchmark as the goal line, and that can be seen as an ambitious target for many struggling readers. Lastly, the results from this study add evidence for the utility of the median rule because it reflects the student’s level as opposed to estimating their growth, while also accounting for extreme data points.

Additional Resources:

National Center on Intensive Intervention | Decision Rules for Analyzing Academic Progress Monitoring Data

IRIS Center | How can teachers determine whether students are making appropriate progress?