Part 1: Participation Rates
Assessment data submitted to the Department via EdFacts, as of March 25, 2019, indicated the percentage of all tested students assessed with an AA-AAAS in 2017-18 statewide was as follows: Reading/language arts: 2.02%, Mathematics: 2.01%, Science: 1.97%
As outlined in the June 4, 2019 1.0 percent consequences letter, PA must demonstrate that it made progress in reducing the participation of students who participated in the AA-AAAS in the 2018-19 school year. The data contained in the table below represents preliminary rates of participation in the AA-AAAS for the 2018-19 school year. Accountability data is being verified at the state level during the submission window of this report. Therefore, the final percentage rates will be solidified and reported via Ed Facts by October 1, 2019.
| Subject | 2017-18 | 2018-19 (preliminary projections) |
|---|---|---|
| Reading/ELA | 2.02 | 1.96 |
| Mathematics | 2.01 | 1.96 |
| Science | 1.97 | 1.95 |
Preliminary projections of the 2018-19 data indicate a decrease in AA-AAAS participation for all three tested subjects. While the state continues to exceed the 1.0 percent threshold, the needle is moving in the right direction. Pennsylvania's 1.0 Percent Compliance Plan of Action will continue to build upon the current work that is being done in the state to address the 1.0 percent requirement.
Part 2: PA's 1.0 Percent Compliance Plan of Action
PDE submitted a 1.0 percent waiver request in November 2017. The request was denied by the US Department of Education as PDE did not meet the requirement in 34CFR 200.6(c)(4) that, in order to be eligible for such a waiver, the State must assess at least 95 percent of all students and 95 percent of children with disabilities on the reading/language arts, mathematics, and science assessments. Although the state did not receive waiver approval, PDE implemented a plan of action to address the 1.0 percent requirement in accordance with available technical assistance resources via the National Center on Educational Outcomes (NCEO). The information below outlines the action steps that have been taken to date, as well as plans for additional measures to come into compliance with the 1.0 percent requirement. The information is formatted according to the relevant sections of the original waiver requirements. The "Developing a 1% Cap Waiver or Waiver Extension Request" tool developed by the National Center on Educational Outcomes (NCEO) was used as a guide in development of this plan.
Requirement 3 (§200.6(c)(4)(iii)): Provide assurances that the state has verified that each LEA that the state anticipates will assess more than 1.0 percent of its assessed students in a subject using the AA-AAAS did the following:
(A) followed the state's participation guidelines;
Pennsylvania has a process in place by which all Local Education Agencies (LEAs) in the state are required to submit a written justification document to the Bureau of Special Education (BSE) for each year they anticipate exceeding the 1.0 percent threshold of students who will be assessed in a subject using the AA-AAAS. While this process has been in place for three years, the document LEAs are required to submit was updated in the 2018-19 school year. The new requirements of the document include a more detailed analysis of participation data to identify trends and/or anomalies that may be occurring within the LEA's participation data. The document also requires the LEA to provide assurance that IEP teams are adhering to PA's identified six criteria of eligibility in making participation decisions for students who participate in the AA-AAAS. The Superintendent or Chief School Officer and the Special Education Administrator are required to sign the justification document to provide assurance that the contents of the LEAs plan are accurate. The document entitled, 'PA Alternate System of Assessment 1.0 % Participation Cap Justification' can be accessed on the PDE website:
LEAs were formally notified of the new 1.0 Percent Cap Justification document through a Penn*Link, which is the official electronic mail service in which PDE conveys information to LEAs through statewide distribution lists.
PDE, Bureau of Special Education representatives are currently participating in the National Center for Educational Outcomes (NCEO) and Center for Standards and Assessment (CSAI) Peer Learning Group entitled, "Guiding and Evaluating District Justifications for Exceeding the 1% Threshold". PDE plans to make publicly available the 'PA Alternate System of Assessment 1.0% Participation Threshold Justification' data for each LEA beginning in the 2019-20 school year. LEAs will be required to submit their justification documentation to BSE by October 15, 2019. LEAs will be notified in writing and through statewide training of PDEs plan to make the justification information publicly available. PDE will post the list of each LEA who submits a justification document on the PDE website by November 1, 2019. The department will ensure that no personally identifiable information is released within the public posting. Requests to view the individual justification reports or specific questions about individual LEA reports can be made to the Bureau of Special Education (BSE) or directly to the LEA. Each LEA will identify a point of contact, such as a Special Education Administrator or Superintendent to be contacted for questions.
(B) will address any disproportionality in the students taking the AA-AAAS
Data regarding the percentage of students who participate in the AA-AAAS is provided annually to the BSE. The annual report of participation and performance data is provided from the alternate assessment vendor and is based upon data entered by the LEA into the online alternate assessment enrollment system1. This report entitled, '2018 PASA Annual Participation and Performance Report' provides a breakdown of AA-AAAS participation and performance data by gender, primary disability category, ethnicity, economically disadvantaged, and English Learner status. A summary included in the 2018 report indicates the overall total of 19,209 students assessed on the PA Alternate System of Assessment (PASA) in reading/language arts, mathematics, and science were male, white/not Hispanic, with a primary disability category being intellectual disability or autism.
1The online enrollment system for the PASA is referred to as the PASA Digital system. Accountability data may vary from enrollment data based upon final attribution rules including full academic year.
While the state has not formally identified areas of disproportionality present within statewide AA-AAAS participation data, review of data indicates a growing trend of students with a primary disability category of learning disabled who are participating in the AA-AAAS. The following represents a three-year comparison of students who participated in the AA-AAAS with a primary disability of learning disability;
- 2015-16 1.5%
- 2016-17 9.6%
- 2017-18 8.8%
The large increase of students within the primary disability category of learning disability who participate in the AA-AAAS sparked an increased focus on providing appropriate eligibility determination resources for IEP teams, as referenced below.
Requirement 4 (§200.6(c)(4)(iv)): Submit a plan and timeline by which the following will be accomplished: (A) state will improve the implementation of its participation guidelines, including if necessary, revising its definition of "students with the most significant cognitive disabilities"; (B) state will take additional steps to support and provide appropriate oversight to each LEA that the state anticipates will assess more that 1.0 percent with the AA-AAAS to ensure that only students with the most significant cognitive disabilities take the AA-AAAS (this must include a description of how the state will monitor and regularly evaluate each of these LEAs to ensure that the LEA provides sufficient training for IEP team members); and (C) state will address any disproportionality in the percentage of students taking the AA-AAAS.
(A) Plan and timeline for improving the implementation of participation guidelines (and possibly revising definition of students with the most significant cognitive disabilities)
The BSE revised the state's AA-AAAS participation eligibility criteria in 2018. PA conducted a statewide, online survey regarding the eligibility criteria. The purpose was to identify areas of improvement and focus within the existing AA-AAAS eligibility criteria. Following analysis of the survey results, PA engaged stakeholders in three separate meetings in three areas of the state (Pittsburgh, Harrisburg, and Malvern). The meetings occurred in May of 2018. Groups were comprised of parents (including representation from The Parent Education and Advocacy Leadership (PEAL) and Local Task Forces, special educators, special education administrators, and service providers (school psychologists). The meetings were representative of three geographical regions of PA. Stakeholders added a definition of students with the most significant cognitive disabilities, further defined criteria per survey data, and suggested examples of non-eligible considerations for school teams.
As a result of the input garnered from the survey data and stakeholder meetings, the BSE developed a new resource to include the updated AA-AAAS participation criteria entitled, 'PASA Eligibility Criteria: Decision Making Companion Tool'. The new tool was released to the field in the Fall of 2018. The resource is intended for IEP teams to review each of PA's six criteria of eligibility, along with additional considerations related to each of the criteria to provide further clarity and specificity. The IEP team must be able to answer 'yes' to all six criteria in order to deem a student eligible to participate in the AA-AAAS.
The decision to participate in the AA-AAAS is made by the Individualized Education Program (IEP) team and documented accordingly in the student's IEP. PA updated the IEP document and annotated IEP document in the Fall of 2018 to include the updated AA-AAAS eligibility criteria in section IV of the document. IEP teams now have the criteria available on the IEP template as well as the decision-making companion tool to use as a reference when making alternate assessment eligibility determinations.
- Attachment 5 - Annotated IEP document
Both statewide survey data and the stakeholders indicated a need for additional resources designed for parents on the AA-AAAS. The BSE developed a new resource released in the Fall of 2018 entitled, 'A Parent Guide to the Pennsylvania Alternate System of Assessment'. The guide includes frequently asked questions expressed by parents and answers to these questions written in familiar, everyday language.
- Attachment 6- A Parent Guide to the PASA
(B) The state will take additional steps to support and provide appropriate oversight to each LEA that the state anticipates will assess more that 1.0 percent with the AA-AAAS to ensure that only students with the most significant cognitive disabilities take the AA-AAAS (this must include a description of how the state will monitor and regularly evaluate each of these LEAs to ensure that the LEA provides sufficient training for IEP team members).
Monitoring of LEAs: Intensive Needs Review Process
BSE initiated a new monitoring system during the 2018-19 school year to focus on LEAs that exceeded the 1.0 percent cap. This process is referred to as the Intensive Needs Review (INR) Process. The PDE reviewed 2015-16 and 2016-17 participation data and identified 22 LEAs that exceeded the 1.0 percent threshold by a rate of 4.0 percent or greater and did so for two consecutive years. LEAs with fewer than 100 total students enrolled in the tested grade levels were excluded from the identification process. The identified LEAs received written notification in August of 2018 of the requirement to participate in the INR process. Each identified LEA was required to submit a completed 'Intensive Needs LEA Review' data protocol. The data protocol provided the LEA with a prescribed framework in which to analyze their participation data. LEAs were required to provide PASA participation data by disability type, gender, ethnicity, economic disadvantaged status, and English Learner status. The LEA was required to review each set of data and identify anomalies or patterns. Additionally, the LEA was required to compare the subgroup participation rates to the LEAs overall student and special education populations to identify possible disproportionality. The 22 identified LEAs were also required to submit two IEP's of students who took the AA-AAAS to BSE for review. The purpose was to determine if eligibility determinations were being documented appropriately within the IEP. BSE provided a webinar training for all affected LEAs on September 21, 2018. During this webinar, BSE representatives presented information on each component of the INR data protocol document and the overall process requirements.
The INR submission and two IEPs for each of the 22 identified LEAs were reviewed by BSE representatives. Any identified area of need requiring additional action and/or training was identified by the BSE, and BSE representatives then met with each LEA to review the results. Some actionable steps required by the BSE included evidence of staff training, evidence of IEP Team meetings, and/or additional data clarification. All LEAs received final notice from the BSE upon completion of the required actions.
While only 22 LEAs were required to submit the Intensive Needs LEA Review document to BSE for formal review, all LEAs in the state were provided the tool to use as an internal resource in examining alternate assessment participation data.
BSE reviewed the 2018-19 justification statements from each of the affected 'intensive needs' LEAs. While most were still projecting AA-AAAS participation to exceed the 1.0 percent threshold, all 22 LEAs projected some level of decrease in the percentage of students planned to take the alternate assessment in 2018-19 compared to the prior year. This may be attributed to the process of analyzing participation data, and reconvening IEP teams of students who did not truly meet the state's updated eligibility criteria to participate in the AA-AAAS.
Differentiated Monitoring System
BSE has established administration procedures for monitoring of special education program implementation, including evaluating the appropriateness, effectiveness, and accountability of special education services and programs across the commonwealth referred to as the Special Education Compliance Monitoring for Continuous Improvement (CMCI). BSE is offering a pilot for a new Differentiated Monitoring Support (DMS) to begin the 2019-20 school year. The DMS pilot will schedule the LEAs monitoring activities based upon identified risk factors. PASA participation rates will now be included as one component in identifying if the LEA is at a high, medium or low risk in regard to AA-AAAS participation rates. LEAs determined to be at a medium or high risk will be subject to additional oversight and monitoring procedures as a component of their overall special education monitoring. BSE anticipates the addition of AA-AAAS participation data review to the special education cyclical monitoring will heighten the focus on analyzing participation data for LEAs and ensuring that IEP teams are making alternate assessment participation eligibility determinations appropriately.
At the time of this submission, the form being used to analyze AA-AAAS participation data as part of the new DMS pilot is in draft form. The attachment below is a draft of the planned document.
Training
The BSE offers several statewide training opportunities throughout each school year to ensure LEAs receive current information regarding policy and accountability requirements related to the alternate assessment. An annual statewide webinar entitled, 'PASA Getting Ready Annual Updates' is provided each Fall prior to enrollment in the alternate assessment. It is designed for Special Education Administrators and/or Alternate Assessment Coordinators and provides information related to meeting the 1.0 percent threshold requirement including federal regulatory language, State and LEA procedures, and implications for exceeding the threshold. Additionally, new test administration procedures are emphasized. The webinar is broadcast live, as well as recorded, and made available on the PDE and the Pennsylvania Training and Technical Assistance Network (PaTTAN) websites. In the September 2018 training, the updated AA-AAAS eligibility criteria and new tool for IEP teams were introduced during this training.
BSE and PaTTAN (Pennsylvania Training and Technical Assistance Network) offered additional statewide training and technical assistance opportunities at the state's annual Autism Conference and Special Education Leadership Academy in 2018. Additionally, all assessment coordinators and assessors who participate with the AA-AAAS are required to complete a series of online training modules annually. The updated eligibility requirements and decision-making companion tool for IEP teams were added to the training contents in 2018.
Plans for 2019 training include increased focus on the INR data protocol. While LEAs who will be identified as an intensive needs status will be required to complete and submit the review to BSE, the PDE will train all LEAs on using the tool as a means to examine their LEAs participation data internally, and determine if patterns or anomalies within the data exist. A focus on identification of students outside of the disability categories typically associated with the alternate assessment (i.e., learning disability) will be covered. LEAs will receive information on how to use the tool appropriately, and guidance on how to re-examine eligibility decisions with the IEP team when necessary.
(C) state will address any disproportionality in the percentage of students taking the AA-AAAS.
PDE will continue to collect and analyze AA-AAAS participation data at the state level. The annual participation and performance information will continue to be made publicly available.
The state requires all LEAs in the state to examine their own participation data and provide a written justification to BSE if they anticipate exceeding the 1.0 percent threshold requirement. The LEA is required to provide a breakdown of their participation data by primary disability category on the justification form, and provide explanation of any higher than expected numbers of students who participate in the AA-AAAS within the subgroup of disability category.
LEAs who are subject to the Intensive Needs Review process are required to break down their participation data by disability category, ethnicity, gender, economically disadvantaged status, and English Learner participation to identify any patterns or anomalies that may indicate disproportionality among any of the subgroups participating in the AA-AAAS within the LEA. BSE reviews this data with the LEA and provides technical assistance and/or required follow up action when necessary.
Plan of Action Summary
In conclusion, PDE has implemented a comprehensive actionable plan to address the 1.0 percent threshold requirement. BSE representatives are active participants in the technical assistance opportunities offered through the National Center on Educational Outcomes (NCEO) 1% Community of Practice calls, as well as and the NCEO and Center on Standards and Assessment Implementation (CSAI) Peer Learning Groups. The department was represented at the, "Supporting States in Implementing ESSA's 1.0 Percent State-Level Cap on Participation of Students in the AA-AAAS" convening in October of 2018. The support and resources gained from these opportunities has been instrumental in the development and implementation of PA's current actions to address the 1.0 percent requirements.
PDE will continue to utilize these resources in future development and refinement of plans to address the 1.0 percent requirement. Specifically, PDE will implement the following actions to come into compliance with the 1.0 percent requirement:
- Continue to require all LEAs in the state that anticipate exceeding the 1.0 percent threshold to provide a written justification to BSE annually.
- BSE will make publicly available LEA PASA 1.0 Percent Cap Justifications by November 1, 2019.
- Continue to implement the 'Intensive Needs LEA Review' for LEAs that exceed the threshold in accordance with established State criteria.
- BSE will conduct further training for LEAs in 2019 on the Intensive Needs Review data protocol. While the identified 'intensive needs' LEAs will be required to submit the data protocol to BSE, all LEAs will be trained on the use of this tool as a way to examine participation data internally and identify patterns, anomalies, or areas of possible disproportionality among subgroups participating in the AA-AAAS.
- Beginning in the 2019-20 school year the state will pilot the DMS as part of CMCI. LEAs who show a high level of alternate assessment participation rates will be subject to a medium or high-risk status in the new system requiring additional oversight by the BSE in this area.
- PDE recognizes the importance of students with disabilities overall participation in statewide assessments and the need to address compliance with the measured achievement of at least 95% of all students and students with disabilities enrolled in the grades for which the AA-AAAS is required (Requirement 2, §200.6(c)(4)(ii)). Therefore, stakeholder input will be garnered over a two-year period beginning during the 2019-20 school year to analyze current participation and opt- out rates in the state. The stakeholder input will also be used to develop resources that may assist LEAs in the improvement rate of students with disabilities who participate in statewide assessments.