State departments of education across the country are engaging their constituencies in the concept of “N” size. N size refers to the minimum number of students required in a group before the group is used for accountability and reporting purposes.
The opportunity for states to re-set their N size is an issue brought about by the increased flexibility granted to State Education Agencies (SEA’s) as a result of the re-authorization of No Child Left Behind (NCLB) or the Every Student Succeeds Act (ESSA). The 2016-17 school year is a “transition” year with ESSA taking full effect in the 2017-18 school year. Consequently, states are making numerous decisions this year that will significantly impact their accountability systems and one of those topics is N size. The size of the sub group that becomes reportable will potentially have a profound impact on a large number of students, schools and districts. In making this decision, there are issues to consider. First let’s look at what gets reported by using a sample school district that has five elementary buildings, one middle school and one high school.
In our sample school district with seven school buildings, eight report cards will be issued: one for each of the five elementary buildings, one for the middle school, one for the high school and one for the district. To illustrate what gets reported let’s use Students with Disabilities as a subgroup and the fifth grade math test as our assessment. In our sample school district’s state, 30 is the N size required for a subgroup to be reported.
Number of Fifth Grade Students with Disabilities In Sample School District
Elementary A: 17
Elementary B: 27
Elementary C: 34
Elementary D: 16
Elementary E: 12
Middle School: 0 (No 5th grade test)
High School: 0 (No 5th grade test)
District: 106
Using the above data, Students with Disabilities would be reported as a subgroup for accountability purposes on the report card for Elementary C and the district. Both had 30 or more fifth grade Students with Disabilities who took the state math assessment in their respective sub groups. For Elementary A,B, D and E Students with Disabilities would NOT be a reported sub group because the N size in these buildings was lower than 30. If the N size was 10 instead of 30, however, EVERY elementary building in our sample district as well as the district would have had Students with Disabilities as a reported sub group. N size matters!
There are at least three considerations when determining the size of a sub group in statistical evaluation in education. First, the data needs to be reliable (consistent over time), valid (measures what was intended) and statistically significant (whether the results are due to chance or some other factor). Second, the data reported should not be “personally identifiable”. Third, setting the N size too high may leave a significant number of students outside the accountability system. Let’s take a closer look at these three factors.
The issue of a test being valid, reliable and statistically significant is essential or the data we are studying is meaningless. Although absolutely imperative, the conversation of the validity, reliability and statistical significance of an assessment will need to be left to the judgment of each SEA in their selection of a testing company to create and statistically prove these three factors in the development phase of the assessment. These statistical factors focus on the exam itself and not the specific size of the sub group. Nevertheless, these issues are fundamental in the success of any accountability system.
Another major consideration is the protection of individual students to insure that data reported to the public is not “personally identifiable”. In the era of NCLB and now ESSA more and more student information is being published for public consumption. Data is reported at the state, local and building level in areas such as assessment scores, race/ethnicity, socio-economic status, disability and the list goes on. The requirement for SEA’s to report this information and at the same time protect student rights under the Family Education Rights and Privacy Act (FERPA) is a tightrope at best and a quagmire at worst. As states contemplate the issue of N size, thoughtful consideration must be given to the possibility that lowering the N size may increase the possibility of personally identifiable information of a student or students being compromised. A violation of FERPA by an SEA could open a Pandora’s box of charges and law suits that would be tied up in the court system for years. The National Center for Education Statistics published an article that speaks to this issue in depth: “Statistical Methods for Protecting Identifiable Information in Aggregate Reporting”
The third factor to consider when re-setting the N size that I will discuss involves the “inclusion” of as many students as possible in the reporting. Inclusion ensures that no single group of students gets left behind or is out of the spotlight because the group or groups they belong to do not meet the established N size and their data is subsequently not reported. Child rights advocates and representatives from numerous groups that focus on specific children’s issues, like the rights of minority children or students with disabilities, cry foul at a system that statistically “leaves these kids out” by inflating the N size. Our sample school district above highlights this issue. The sample district had a total of 106 fifth grade Students with Disabilities. With an N size of 30 none of the Students with Disabilities data in Elementary schools A,B,D or E were reported. Numerically, out of 106 fifth grade Students with Disabilities who took the state math assessment, 72 students scores went unreported. That number represents 67.9% of the students in the district who fall into this sub group. If the N size was 10 in the sample, 100% of the students data would have been published. Conversely, there are also implications for districts and individual school buildings if a lower N size is adopted. In our sample district, four of the five elementary buildings, or 80%, would ADD a sub group that would be reported. Considering there are currently ten (10) sub groups that must be reported by federal legislation and that three new subgroups will be added with ESSA (Children of Military Personnel, Homeless and Foster Children) buildings or districts may be looking at an additional 2,3,4 or more subgroups to their respective report cards with a lower N size. The Ohio Department of Education looks at this issue in depth in their Every Student Succeeds Act Discusion Guide.
Including as many kids as possible in an effort to uncover areas that need to be addressed while protecting the FERPA rights of all children presents a lot of issues to carefully consider. Let’s just hope these tests are valid, reliable and statistically significant!