A National Study of School Effectiveness for Language Minority Students' Long-Term Academic Achievement Final Report: Project 1.1
Overall Conclusions and Major Policy Implications
Summary Of Findings Across All Research Sites
Each of the research contexts of this study illustrate varying aspects of the major factors that appear to influence the academic success of language minority students who begin their U.S. schooling with no proficiency in English. Overall, the five school districts examined have attempted to address the dimensions of the Prism Model of Language Acquisition for School (Thomas & Collier in Ovando & Collier, 1998, p. 89), as they continue to improve programs for their English language learners. The Prism Model focuses on four developmental processes that school-age students experience through Grades K-12–sociocultural, linguistic, cognitive, and academic processes. These processes develop subconsciously, occur simultaneously, and are interdependent. The findings of this research study demonstrate that it is crucial that educators provide a socioculturally supportive school environment for language minority students that allows natural language, academic, and cognitive development to flourish in both L1 and L2, comparable to the sociocultural support for ongoing language, academic, and cognitive development that native-English speakers are provided in school.
The qualitative findings from each research site illustrate that each school context is different, and significant elements within each educational context can have strong influence on students' academic achievement in the long term. Overall, we have found that the following major factors influence the success of programs for English language learners:
- The potential quality of the type of program for ELLs. This refers to the power of a particular program's features to influence student achievement. Some school programs for ELLs are "feature-rich," with enhanced potential to affect student achievement, while others are "feature-poor," with little or no theoretical reason to believe that their use will help ELLs to close the achievement gap.
- The realized quality of the type of program for ELLs. This is the degree of full and effective implementation of a program in terms of administrative support; teacher skills and training to deliver the full instructional effect of the program; and the degree to which program installation, processes, and outcomes are monitored and formatively evaluated.
- The breadth of program focus. This refers to an instructional focus on the Prism Model dimensions of linguistic, cognitive, and academic development to native-speaker levels of English, as well as in students' primary language, in a supportive sociocultural school environment, as contrasted with a narrow and restrictive instructional focus, such as "just learning enough English to get by."
- The quality of the school's instructional environment. This refers to the degree to which the school becomes an additive language-learning environment rather than a subtractive environment, including parental engagement and support of the instructional program. In an additive bilingual environment, students acquire their second language without any loss of their primary language. Students who continue to develop cognitively in their primary language and develop age-appropriate proficiency in both first and second language can outscore monolinguals on school tests (Baker & Prys Jones, 1998).
- The quality of available instructional time. This is the degree to which instructional time is used effectively so that students receive maximally comprehensible instruction for an instructionally optimum time period, in classrooms where English language learners are not isolated, but where all students interact together and where instruction is driven by students' cognitive, academic, and linguistic developmental needs.
Overall, programs for English language learners that "score high" in these five major factors are long-term and enriched forms of bilingual/ESL instruction that provide for most or all of the documented achievement gap to be closed in the long term. Programs that "score low" on these major factors are remedial, short-term, and ineffective.
SUMMARY OF QUANTITATIVE ANALYSES
Major findings from the quantitative analyses that are statistically and practically significant for decision-making are presented below. For decision-making purposes, a 4 NCE difference between groups is considered a small but significant difference (equivalent to 0.2 of a national standard deviation [s.d.]), 5 NCEs an actionable significant difference (0.25 of a national s.d.), 6 NCEs a moderate significant difference (0.3 of a national s.d.), and 10 NCEs a very large significant difference (0.5 of a national s.d.).
ENGLISH ACHIEVEMENT FINDINGS
Focusing first on program comparisons, we summarize English language learners' long-term achievement on nationally standardized tests (ITBS, CTBS, Stanford 9, Terra Nova) in English Total Reading (the subtest measuring academic problem-solving across the curriculum–math, science, social studies, literature), for students who entered the U.S. school district with little or no proficiency in English in Grades K-1, and following them to the highest grade level reached by the program to date:
- English language learners immersed in the English mainstream because their parents refused bilingual/ESL services showed large decreases in reading and math achievement by Grade 5, equivalent to almost 3/4 of a standard deviation (15 NCEs), when compared to students who received bilingual/ESL services. The largest number of dropouts came from this group, and those remaining finished 11th grade at the 25th NCE (12th percentile) on the standardized reading test. (pp. 113-114, 122-124, Figures C-1, C-2, Tables C-1, C-2, C-10, C-11)
- When ESL content classes were provided for 2-3 years and followed by immersion in the English mainstream, ELL graduates ranged from the 31st to the 40th NCE with a median of the 34th NCE (23rd percentile) by the end of their high school years. (pp. 112-114, 126-127, 241-256, Figures C-1, C-2, E-1, E-6, E-7, E-8, E-9, E-14, Tables C-1, C-2, E-1, E-6, E-7, E-8, E-9, E-14)
- 50-50 Transitional bilingual education students who were former ELLs, provided with 50 percent instruction in English and 50 percent instruction in Spanish for 3-4 years, followed by immersion in the English mainstream, reached the 47th NCE (45th percentile) by the end of 11th grade. (pp. 112-114, 126-127, Figures C-1, C-2, Tables C-1, C-2)
- 90-10 Transitional bilingual education students who were former ELLs reached the 40th NCE (32nd percentile) by the end of 5th grade. (In 90-10 TBE, for Grades PK-2, 90 percent of instruction is in the minority language, gradually increasing English instruction until by Grade 5, all instruction is in the English mainstream for the remainder of schooling.) (pp. 119-122, Figure C-8, Table C-7)
- 50-50 One-way developmental bilingual education students who were former ELLs reached the 62nd NCE (72nd percentile) after 4 years of bilingual schooling in two high-achieving school districts, outperforming their comparison ELL group schooled all in English by 15 NCEs (almost 3/4 of a national standard deviation–a very large significant difference). By 7th grade, these bilingually schooled former ELLs were still above grade level at the 56th NCE (61st percentile). (A one-way program is one language group being schooled through two languages.) (pp. 48-52, 58, Figures A-1, A-3, Tables A-5, A-6)
- 90-10 One-way developmental bilingual education students who were former ELLs reached the 41st NCE (34th percentile) by the end of 5th grade. (90-10 means that for Grades PK-2, 90 percent of instruction is in the minority language, gradually increasing English instruction to 50 percent by Grade 5, and a DBE program continues both languages in secondary school.) (pp. 119-122, Figure C-8, Table C-7)
- 50-50 Two-way bilingual immersion students who were former ELLs attending a high-poverty, high-mobility school: 58 percent met or exceeded Oregon state standards in English reading by the end of 3rd and 5th grades. (Two-way is two language groups receiving integrated schooling through their two languages; 50-50 is 50 percent instruction in English and 50 percent in the minority language.) (pp. 201-204, Figures D-4, D-6, Table D-18)
- 90-10 Two-way bilingual immersion students who were former ELLs performed above grade level in English in Grades 1-5, completing 5th grade at the 51st NCE (51st percentile), significantly outperforming their comparison groups in 90-10 transitional bilingual education and 90-10 developmental bilingual education. (pp. 119-121, Figure C-8, Table C-7)
SPANISH ACHIEVEMENT FINDINGS
A goal of one-way and two-way bilingual education is to graduate students who are fully academically proficient in both languages of instruction, to prepare these students for the workplace of the 21st century. We summarize native-Spanish-speakers' long-term achievement on nationally standardized tests (Aprenda 2, SABE) in Spanish Total Reading (the subtest measuring academic problem-solving across the curriculum–math, science, social studies, literature), following them to the highest grade level reached by the program to date:
- In 50-50 Two-way bilingual immersion, Spanish-speaking immigrants after 1-2 years of U.S. schooling achieved at a median of the 62nd NCE (71st percentile) in Grades 3-6. These immigrants arrived on or above grade level and maintained above grade level performance in Spanish in the succeeding two years. (pp. 199-200, Figure D-2, Tables D-5, D-6)
- In 90-10 Transitional bilingual education classes, native-Spanish speakers reached the 56th to 60th NCE (61st to 68th percentile) for Grades 1-4, and after moving into all-English instruction in Grade 5, they tested at the 51st NCE, still on grade level in Spanish reading achievement. (pp.117-119, Figure C-5, Table C-4)
- In 90-10 Developmental bilingual education classes, native-Spanish speakers reached the 56th to 63rd NCE (61st to 73rd percentile) for Grades 1-4, and in Grade 5 they outperformed the TBE comparison group by 4 NCEs at the 55th NCE (60th percentile). (pp. 117-119, Figure C-5, Table C-4)
- In 90-10 Two-way bilingual immersion classes, native-Spanish speakers reached the 58th to 65th NCE (64th to 76th percentile) for Grades 1-4, and in Grade 5 they outperformed the TBE and DBE comparison groups by a significant 6 NCEs at the 61st NCE (70th percentile). (pp. 117-119, Figure C-5, Table C-4)
- In reading achievement across the curriculum, native-Spanish speakers outperformed native-English speakers when tested in their native language, for Grades 1-8, regardless of the type of bilingual program Spanish-speaking students received. Native-Spanish speakers remained significantly above grade level at every grade except sixth grade (at the 49th NCE), reaching the 64th NCE (74th percentile) in 8th grade. (pp. 117-119, Figure C-3, Table C-3)
ACHIEVEMENT FINDINGS IN OTHER SUBJECTS
- We chose the reading subtest of the standardized tests (results presented above) as the "ultimate" measure of attainment, because LM students' reading scores were consistently the lowest among the subjects, and this is the measure that most closely correlates with the standardized tests required for admission to post-secondary education. Generally, LM students achieved 5-10 NCEs higher in English language arts, math, science, social studies, and writing. (pp. 46-53, 111-114, 119-122, 241-256, Figures A-4, A-5, C-9, C-10, E-1 to E-14 and accompanying tables)
- In Spanish math, native-Spanish speakers generally outperformed native-English speakers tested in English math. When comparing native-Spanish speakers' achievement in Spanish math by program, for Grades 2-5, students attending all three bilingual program types achieved at or above the 55th NCE (60th percentile). But the Spanish speakers attending 90-10 Two-way bilingual immersion classes outperformed the Spanish speakers in 90-10 TBE and 90-10 DBE classes by 3-6 NCEs on Spanish math achievement, reaching the 59th NCE (66th percentile) by 5th grade. (pp. 114, 117-118, Figures C-4, C-6, Tables C-3, C-4)
ACHIEVEMENT OF NATIVE-ENGLISH SPEAKERS IN TWO-WAY BILINGUAL EDUCATION
- Native-English speakers in two-way bilingual immersion programs maintained their English, added a second language to their knowledge base, and achieved well above the 50th percentile in all subject areas on norm-referenced tests in English. These bilingually schooled students equaled or outperformed their comparison groups being schooled monolingually, on all measures. (pp. 46-53, 119, 124, 201-204, Figures A-3 to A-5, D-1, D-3, D-5, D-7, D-9, Tables A-1 to A-11, C-4, C-12, C-13, D-1 to D-4, D-7, D-8, D-10, D-12, D-13, D-15, D-17 to D-10)
INFLUENCE OF STUDENT BACKGROUND ON STUDENT ACHIEVEMENT
- Socioeconomic status (SES) typically influenced from 3-6% of LM students' reading achievement as measured by standardized tests, for both enrichment dual language programs and ESL content programs. In selected circumstances (e.g., oral proficiency of Spanish speakers learning English) the effect of SES explains as much as 11-12% of achievement. However, the effect of number of years of program participation on reading achievement varied with the program type. For one-way and two-way dual language programs, up to five years of program participation accounted for 6-9% of ELLs' reading achievement on standardized tests. For Spanish speakers learning English, 20% of oral proficiency was attributable to program exposure while program exposure accounted for 15% of oral proficiency for English speakers learning Spanish. In the case of the ESL Content program, years of schooling accounted for less than 2% of end-of-school reading achievement as measured by standardized tests. Thus, a strong dual language program can "reverse" the negative effects of SES more than a well-implemented ESL Content program by raising reading achievement to a greater degree. (pp. 56-57, 204-206, 256-258, Tables A-18, D-20, E-16 to E-18)
- The One-way developmental bilingual education program in Northern Maine influenced 8.5% of former ELLs' eventual reading achievement, exceeding the effects of low socioeconomic status at less than 4%. The Two-way bilingual immersion program at Grant Community School exerted a powerful and significant effect on Spanish-speaking students' scores on oral English development and influenced about 6 percent of their standardized reading scores as assessed in English, while SES accounted for about 4%. In this high-poverty school, SES alone accounted for 14 percent of the observed achievement variance overall. Thus, the school's dual language program is reducing the negative effects of SES by significant amounts for Spanish speakers learning English and taking the statewide assessment in English. (pp. 56-57, 204-206, 256-258, Tables A-18, D-20, E-16 to E-18)
- Number of years of primary language schooling, either in home country or in host country, had more influence than socioeconomic status when the number of years of schooling was 4 or more years. In addition, the L2 academic achievement of older immigrant arrivals with strong grade-level schooling completed in L1 in the home country was less influenced by low socioeconomic status and more dependent on number of years completed. Likewise, students of low socioeconomic status who were born in the U.S. or arrived at a very young age achieved at high levels in L2 when grade-level schooling was provided in both L1 and L2 in the U.S. (pp. 257-258, Figures C-1, E-6, E-7, Tables C-1, E-6, E-7, E-17, E-18)
- When immigrants were schooled all in English in the U.S., students who received 4-5 years of L1 schooling in home country (arriving at ages 10-12) scored 6 NCEs higher in English reading in 11th grade than those who received 1-3 years of home country schooling (arriving at ages 7-9). (pp. 248-251, Figures E-6, E-7, Tables E-6, E-7)
- Immigrants with interrupted schooling in home country achieved significantly below grade level, when provided instruction only in English. Those one year below grade level on arrival were at the 29th NCE (16th percentile) on the English reading test by 11th grade, those two years below grade level on arrival at the 26th NCE (13th percentile), those three years behind at the 20th NCE (8th percentile), and those four years behind at the 19th NCE (7th percentile). (pp. 251-253, Figure E-8, Table E-8)
- Gender differences among Hispanic students were found to be significant in only two subject areas–math and science. Hispanic males outperformed Hispanic females by 4 NCEs in math and 6 NCEs in science on the 11th grade tests in English. (p. 256, Figure E-14, Table E-14)
MAJOR POLICY IMPLICATIONS
- Enrichment 90-10 and 50-50 one-way and two-way developmental bilingual education (DBE) programs (or dual language, bilingual immersion) are the only programs we have found to date that assist students to fully reach the 50th percentile in both L1 and L2 in all subjects and to maintain that level of high achievement, or reach even higher levels through the end of schooling. The fewest dropouts come from these programs.
- Parents who refuse bilingual/ESL services for their children should be informed that their children's long-term academic achievement will probably be much lower as a result, and they should be strongly counseled against refusing bilingual/ESL services when their child is eligible. The research findings of this study indicate that ESL or bilingual services, as required by Lau v. Nichols, raise students' achievement levels by significant amounts.
- When English language learners (ELLs) initially attend segregated, remedial programs, these students do not close the achievement gap after reclassification and placement in the English mainstream. Instead, they maintain or widen the gap in later years. Therefore, their average achievement NCE at reclassification should be as high as possible, since this is likely to be their highest achievement level that they reach during their school years. Ideally, instructional gains are best accomplished in an enrichment (not a remedial) program.
- Students with no proficiency in English must NOT be placed in short-term programs of only 1-3 years. In this study and all other research studies following ELLs long term, the minimum length of time it takes to reach grade-level performance in second language (L2) is 4 years. Furthermore, only ELLs with at least 4 years of primary language schooling reach grade-level performance in L2 in 4 years. As a group, students with no primary language schooling (either in home country or host country) are not able to reach grade-level performance in L2.
- The strongest predictor of L2 student achievement is amount of formal L1 schooling. The more L1 grade-level schooling, the higher L2 achievement.
- Bilingually schooled students outperform comparable monolingually schooled students in academic achievement in all subjects, after 4-7 years of dual language schooling.
- Students who receive at least 4-5 years of grade-level L1 schooling in home country before they emigrate to the U.S. typically reach the 34th NCE (23rd percentile) by 11th grade when schooled all in English in the U.S. in an ESL Content program, and then the mainstream. These students are on grade level when they arrive, but it takes them several years to acquire enough English to do grade-level work, which is equivalent to interrupting their schooling for 1 or 2 years. Then they have to make more gains than the average native-English speaker makes every year for several years in a row to eventually catch up to grade level, a very difficult task to accomplish within the remaining years of K-12 schooling.
- The highest quality ESL Content programs close about half of the total achievement gap.
- When ELLs initially exit into the English mainstream, those schooled all in English outperform those schooled bilingually when tested in English. But the bilingually schooled students reach the same levels of achievement as those schooled all in English by the middle school years, and during the high school years the bilingually schooled students outperform the monolingually schooled students (see Figure C-2).
- Students who receive at least 5-6 years of dual language schooling in the U.S. reach the 50th NCE/percentile in L2 by 5th or 6th grade and maintain that level of performance, because they have not lost any years of schooling. Students raised in a dual language environment need at least 4 years of schooling in L1 and 4 years of schooling in L2 to achieve on grade level in either of the two languages. Providing bilingual schooling in the U.S. meets both needs simultaneously, typically in 4-7 years, leading to high academic achievement in the long term.
- Bilingual/ESL Content programs must be effective (at least 3-4 NCE gains per year more than mainstream students are gaining per year), well implemented, not segregated, and sustained long enough (5-6 years) for the typical 25 NCE achievement gap between ELLs and native-English speakers to be closed. Even the most effective programs can only close half of the achievement gap in 2-3 years, the typical length of remedial ELL programs. Therefore, short-term, remedial, and ineffective programs cannot close the large achievement gap and should be avoided.
- An enrichment bilingual/ESL program must meet students' developmental needs: linguistic (L1-L2), academic, cognitive, emotional, social, physical. Schools need to create a natural learning environment in school, with lots of natural, rich oral and written language used by students and teachers (L1 and L2 used in separate instructional contexts, not using translation); meaningful, ‘real world' problem-solving; all students working together; media-rich learning (video, computers, print); challenging thematic units that get and hold students' interest; and using students' bilingual-bicultural knowledge to bridge to new knowledge across the curriculum.
Back to Main Report