Final Evaluation Report AY 1996-07 Early Reading Intervention Project Springfield Public Schools Springfield, Massachusetts August 22, 1997 Submitted by David J. Chard, Ph.D. Department of Special Education The University of Texas at Austin
	Introduction In the Fall of 1996, Springfield Public Schools began an Early Intervention effort whose primary goal was to improve the reading success of children district wide. This multifaceted effort included professional development opportunities focusing on recent research in reading development, assessment, and instruction. As a result of this district initiative, 27 schools developed an early reading program focused on preventing reading difficulties for all children. In an attempt to evaluate the overall effectiveness of the program and to determine the relative effectiveness of individual interventions an assessment was completed prior to and following administration of the reading interventions. This report is divided into five parts. First, a brief description of each of the four interventions included in the overall assessment will be provided. Second, a description of each of the assessment measures will be outlined. Third, descriptive data will be presented for each program across measures. Fourth, comparative statistics will be presented to identify which of the four programs was most effective overall. Finally, the report will be concluded with instructional implications to assist district and school administrators in decision-making regarding early interventions in reading.
	The Programs Many teachers reported using combinations of commercially published early reading programs along with their own materials. In the process of sorting through the assessment data, program descriptions were distilled into four primary programs: (1) Springfield Early Intervention in Reading (Rigby), (2) Other, (3) Open Court, and (4) Houghton Mifflin Early Success. Table 1 illustrates the number of schools and students receiving each of the four programs.

	Table 1. Numbers of schools and students in each early reading program.
	Program Number of Schools Number of Students 1  Springfield EIR 15 266 2  Other   68 3  Open Court 2 27 4  Early Success 7 114
	The Assessment Measures Decades of research in reading instruction has led to the identification of several elements that should be included to maximize the effect of early reading instruction. These elements included frequent opportunities to read orally with teacher feedback; opportunities to read silently; explicit instruction in phonological awareness, letter-sound correspondence, word reading and writing. As a consequence of this research, six measures were developed and administered by teachers and support staff early in the school year and again mid-way through the school year at the completion of the early intervention programs. The first two measures focused on letter names and letter sounds. These measures were included to gauge students' familiarity with the letters of the alphabet. The third and fourth assessments were measures of phonological awareness or the ability to manipulate sounds in words. To this end, both measures were administered in the absence of print. More specifically, when presented with a consonant-vowel-consonant word (e.g., bet) students were asked to segment the sounds in the word. (e.g., /b/ /e/ /t/). Students were given partial or full credit for successful segmentation of each sound part. The fourth assessment measured students' ability to blend individual sound parts (e.g., /t/ /a/ /n/) into full words (e.g., tan). The fifth assessment measured students' dictation skills. Teachers dictated words while students wrote the words. The sixth assessment measured students' word reading skills. The measure required students to read common words in lists. The final assessment accompanied the Dictation measure and is referred to as Additional Words. Once students finished the dictation tasks, testers asked them to add any other words they knew how to write. Copies of each assessment accompany this report as Appendix A.
	Descriptive Data In this section mean scores on each of the seven measures are presented for all four reading programs. On Figures 1-7, Group 1 represents students receiving Springfield EIR (a modification of the program developed by Dr. Barbara Taylor), Group 2 represents students receiving Other, Group 3 represents students receiving Open Court and Group 4 represents students receiving HM Early Success. In addition, descriptive data is reported for each of the seven assessment measures at Posttest. The descriptive data include mean, standard deviation, median, and range. Figure 1. Pre- and Posttest Mean Scores on Letter Name Identification.
	Figure 2. Pre- and Posttest Mean Scores for Letter Sound
	Figure 3. Pre- and Posttest Mean Scores for Segmenting
	Figure 4. Pre- and Posttest Mean Scores for Blending.
	Figure 5. Pre- and Posttest Mean Scores on Dictation
	Figure 6. Pre- and Posttest Mean Scores on Word Reading.
	Figure 7. Pre- and Posttest Mean Scores on Ad Words.

	Table 2. Descriptive data for Letter-Name Identification.
	Programs Springfield EIR Other Open Court HM Early Success 1  Mean 50.7 38.3 43.6 49.5 2  Standard 6.12 19.14 11.99 7.91 3  Deviation         4  Median 52.0 50.0 44.79 50.73 5  Range 3-54 0-54 11-54 0-54
	Table 3. Descriptive data for Letter-Sound Identification.
	Programs Springfield EIR Other Open Court HM Early Success 1  Mean 19.64 13.88 16.89 18.80 2  Standard 5.14 9.85 5.38 5.93 3  Deviation         4  Median 21.0 17.5 19.0 20.00 5  Range 0-26 0-25 2-25 0-25
	Table 4. Descriptive data for Segmenting.
	Programs Springfield EIR Other Open Court HM Early Success 1  Mean 12.69 8.76 10.44 12.55 2  Standard 4.72 5.74 5.03 4.36 3  Deviation         4  Median 14.0 9.0 11.0 13.0 5  Range 0-18 0-18 0-18 0-18
	Table 5. Descriptive data for Blending.
	Programs Springfield EIR Other Open Court HM Early Success 1  Mean 11.64 9.12 13.30 13.48 2  Standard 5.89 6.86 5.47 5.47 3  Deviation         4  Median 13.0 8.0 15.0 15.0 5  Range 0-18 0-24 0-18 0-19
	Table 6. Descriptive data for Dictation.
	Programs Springfield EIR Other Open Court HM Early Success 1  Mean 9.54 6.71 5.96 8.31 2  Standard 4.22 4.78 4.53 3.99 3  Deviation         4  Median 11.0 7.0 5.0 9.0 5  Range 0-17 0-16 0-16 0-15
	Table 7. Descriptive data for Word Reading.
	Programs Springfield EIR Other Open Court HM Early Success 1  Mean 11.25 4.90 6.81 9.39 2  Standard 4.30 5.35 5.74 4.75 3  Deviation         4  Median 13.0 3.0 5.0 10.5 5  Range 0-15 0-15 0-15 0-15
	Table 8. Descriptive data for Ad Words.
	Programs Springfield EIR Other Open Court HM Early Success 1  Mean 8.43 1.69 .15 6.74 2  Standard 7.11 1.38 .46 5.13 3  Deviation         4  Median 7.0 1.38 .46 5.13 5  Range 0-44 1-6 0-2 0-21

	The descriptive data provide a general overview of the assessment results. With the exception of the blending measure, the Springfield EIR program would appear to be superior to the other programs at improving students' early reading skills. However, because students' scores were not equivalent at pretest and because data on students receiving both Other programs and Open Court was confounded by the small number of students receiving these programs, it is difficult to interpret the data. As a consequence, comparative statistics were employed and pretest scores were used as a covariant to provide a more accurate analysis.
	Comparative Analysis Analysis of covariance was used to test the relative effectiveness of each program on each of the seven measures. Effectively, the null hypothesis is that all programs are equally effective. Therefore, significance indicates that one program is more effective than another. When a significant difference was identified in each analysis, further comparisons were completed to identify the program(s) that were significantly more effective than others. Significant differences in adjusted means were identified on all seven measures. This indicated that further comparisons were required. For each measure the three lowest scoring groups were compared to the highest scoring group. The group receiving Springfield EIR had the highest adjusted mean on each measure. Thus, Other, Open Court and Early Success were compared with Springfield EIR on all measures. Table 9 summarizes the findings of the detailed comparisons of adjusted means.
	Table 9. Summary of findings of final comparisons.
	Measure Finding Letter-Name Identification Spgfld. EIR/Early Success>Other/Open Court Letter-Sound Identification Spgfld. EIR/Early Success>Other/Open Court Segmenting Spgfld. EIR/Early Success>Other/Open Court Blending Spgfld. EIR/Other/Early Success>Open Court Dictation Spgfld. EIR>/ Other/Open Court / Early Success Word Reading Spgfld. EIR/Early Success>Other/Open Court Ad Words Spgfld. EIR/Open Court>Early Success
	In sum, on all measures students receiving instruction with the Springfield EIR program outperformed all others on every measure. Statistically, on all measures with the exception of Dictation and Ad Words, students receiving instruction with Houghton Mifflin's Early Success program also outperformed students receiving Other programs or Open Court. Students receiving Other programs outperformed Open Court only on the Blending measure and students receiving instruction with Open Court outperformed students in Other programs and Early Success only on the Ad Words measure.
	Instructional Implications It is clear from the results of these assessments that of the four programs being employed in the Early Intervention Program in reading instruction in 1996-97, the Springfield EIR and Houghton Mifflin's Early Success were the most successful programs. In this comparison, the "Other" programs can be considered our comparison group. Children receiving the Springfield EIR and Houghton Mifflin's Early Success Program have made considerable improvement over those children receiving the "Other" programs on all measures. A careful look at the descriptive data supports these statements. In all cases Springfield EIR and Early Success appear to result in higher actual scores. With this information, teachers utilizing the "Other" programs or Open Court should be encouraged to reevaluate their programs and instructional delivery to determine where the weaknesses lie. This reevaluation may not result in a change in instructional programs but should result in some instructional change. Examples of instructional components to consider are grouping practices, instructional intensity, allocated time, and engaged time. Once consideration is given to instructional, it is advisable for further evaluation of this effort to take place. Specifically, schools or school districts that have made similar attempts to improve reading instruction in this way should be contacted. Using common measures as these schools, another comparison should be done to identify strengths and weaknesses of the Springfield program. There are a number of limitations to the findings reported here. First, data was collected by teachers and support staff. Consequently, it is impossible to account for the technical adequacy of the resultant scores. Efforts were taken to minimize bias by scripting assessment procedures and providing data tables for gathering data. However, discrepancies in administration no doubt did occur. Secondly, while statistically it is possible to compare different size groups, assumptions associated with comparative statistics may be violated and, subsequent findings may be biased.
	You may download, print, and make copies of these pages provided you show the copyright notice. Intervention | Early Success Education Place | Site Index Copyright © 2001 Houghton Mifflin Company. All Rights Reserved.