A National Study of School Effectiveness for Language Minority Students’ Long-Term Academic Achievement

Final Report

Executive Summary

Principal Investigators:

Wayne P. Thomas

Virginia P. Collier

Purpose. Our research from 1985 to 2001 has focused on analyzing the great variety of education services provided for language minority (LM) students in U.S. public schools and the resulting long-term academic achievement of these students. This five-year research study (1996-2001) is our most recent overview of the types of U.S. school programs provided for these linguistically and culturally diverse students, especially focusing on English language learners’ (ELLs/LEPs) academic achievement in Grades K-12. This study includes qualitative and quantitative research findings from five urban and rural research sites in the northeast, northwest, south-central, and southeast U.S. It is designed to answer urgent policy questions of interest to the federal and state governments of the United States, since this demographic group is projected to be 40 percent of the school-age population by the 2030s and most U.S. schools are currently under-educating this student group. Overall, this research provides whole school district views of policy decision-making that is data-driven regarding designing, implementing, evaluating, and reforming the education of LM students.

Analyses. As principal investigators, we established a collaborative research agreement with each school district that chose to participate, to follow every LM student who entered the school district for every year of his/her attendance in that school district, by each program type attended including the mainstream, and by cohorts of similar student background (e.g. socioeconomic status, primary language [L1] and second language [L2] proficiency upon entry, prior schooling). Measures of student achievement were those administered by the school district, including standardized test scores. We reported generalizations across school districts based on group performance on standardized measures, in normal curve equivalents (NCEs—equal-interval percentiles). Quantitative analyses proceeded through five research stages (presented in detail in report), each stage followed by collaborative interpretation of the results with school district staff. Qualitative analyses from interviews, school visits, surveys, and source documents, included historical demographic patterns of linguistically diverse groups of each U.S. region, the sociolinguistic and social context for the school programs, and specific implementation characteristics of each program type, including a case study of one school innovation.

Research sites, student samples, and program types analyzed. By written agreement, the school districts participating in each of our studies are promised anonymity until they choose to self-identify. For this study, four sites decided to self-identify—Madawaska School Department and School Administrative District #24, both located in northern Maine; Houston Independent School District in Texas; and Grant Community School in Salem, Oregon. The total number of student records collected in the five school districts featured in this report was 210,054. (One student record includes all the school district records for one student collected during one school year, such as student background characteristics, the grade level and school program(s) that student attended, and academic achievement measures administered to that student during the school year.) Over 80 primary languages were represented in the student samples, but the data analyses in three of the five research sites focused on Spanish speakers, the largest language group in the U.S. (75 percent of the U.S. LM school-age population). The student samples included newly arriving immigrants as well as ethnolinguistic groups of French cultural and linguistic roots in the northeast and students of Spanish-speaking heritage in the south-central U.S. The analyses focused on student outcomes from eight major different program types for LM students—90-10 two-way bilingual immersion (or dual language), 50-50 two-way bilingual immersion, 90-10 one-way developmental bilingual education, 50-50 one-way developmental bilingual education, 90-10 transitional bilingual education, 50-50 transitional bilingual education, English as a Second Language (ESL) taught through academic content, and the English mainstream.

FINDINGS: Qualitative findings are presented in the full report. Major findings from the quantitative analyses that are statistically and practically significant for decision-making are presented below. For decision-making purposes, a 4 NCE difference between groups is considered a small but significant difference (0.2 of a national standard deviation [s.d.]), 5 NCEs an actionable significant difference (0.25 of a national s.d.), 6 NCEs a moderate significant difference (0.3 of a national s.d.), and 10 NCEs a very large significant difference (0.5 of a national s.d.).

ENGLISH ACHIEVEMENT FINDINGS: Focusing first on program comparisons, we summarize English language learners’ long-term achievement on nationally standardized tests (ITBS, CTBS, Stanford 9, Terra Nova) in English Total Reading (the subtest measuring academic problem-solving across the curriculum—math, science, social studies, literature), for students who entered the U.S. school district with little or no proficiency in English in Grades K-1, and following them to the highest grade level reached by the program to date:

• English language learners immersed in the English mainstream because their parents refused bilingual/ESL services showed large decreases in reading and math achievement by Grade 5, equivalent to almost 3/4 of a standard deviation (15 NCEs), when compared to students who received bilingual/ESL services. The largest number of dropouts came from this group, and those remaining finished 11^th grade at the 25^th NCE (12^th percentile) on the standardized reading test. (pp. 113-114, 122-124, Figures C-1, C-2, Tables C-1, C-2, C-10, C-11)

• When ESL content classes were provided for 2-3 years and followed by immersion in the English mainstream, ELL graduates ranged from the 31^st to the 40^th NCE with a median of the 34^th NCE (23^rd percentile) by the end of their high school years. (pp. 112-114, 126-127, 241-256, Figures C-1, C-2, E-1, E-6, E-7, E-8, E-9, E-14, Tables C-1, C-2, E-1, E-6, E-7, E-8, E-9, E-14)

• 50-50 Transitional bilingual education students who were former ELLs, provided with 50 percent instruction in English and 50 percent instruction in Spanish for 3-4 years, followed by immersion in the English mainstream, reached the 47^th NCE (45^th percentile) by the end of 11^th grade. (pp. 112-114, 126-127, Figures C-1, C-2, Tables C-1, C-2)

• 90-10 Transitional bilingual education students who were former ELLs reached the 40^th NCE (32^nd percentile) by the end of 5^th grade. (In 90-10 TBE, for Grades PK-2, 90 percent of instruction is in the minority language, gradually increasing English instruction until by Grade 5, all instruction is in the English mainstream for the remainder of schooling.) (pp. 119-122, Figure C-8, Table C-7)

• 50-50 One-way developmental bilingual education students who were former ELLs reached the 62^nd NCE (72^nd percentile) after 4 years of bilingual schooling in two high-achieving school districts, outperforming their comparison ELL group schooled all in English by 15 NCEs (almost 3/4 of a national standard deviation—a very large significant difference). By 7^th grade, these bilingually schooled former ELLs were still above grade level at the 56th NCE (61^st percentile). (One-way is one language group being schooled through two languages.) (pp. 48-52, 58, Figures A-1, A-3, Tables A-5, A-6)

• 90-10 One-way developmental bilingual education students who were former ELLs reached the 41^st NCE (34^th percentile) by the end of 5^th grade. (90-10 means that for Grades PK-2, 90 percent of instruction is in the minority language, gradually increasing English instruction to 50 percent by Grade 5, and a DBE program continues both languages in secondary school.) (pp. 119-122, Figure C-8, Table C-7)

• 50-50 Two-way bilingual immersion students who were former ELLs attending a high-poverty, high-mobility school: 58 percent met or exceeded Oregon state standards in English reading by the end of 3^rd and 5^th grades. (Two-way is two language groups receiving integrated schooling through their two languages; 50-50 is 50 percent instruction in English and 50 percent in the minority language.) (pp. 201-204, Figures D-4, D-6, Table D-18)

• 90-10 Two-way bilingual immersion students who were former ELLs performed above grade level in English in Grades 1-5, completing 5^th grade at the 51^st NCE (51^st percentile), significantly outperforming their comparison groups in 90-10 transitional bilingual education and 90-10 developmental bilingual education. (pp. 119-121, Figure C-8, Table C-7)

SPANISH ACHIEVEMENT FINDINGS: A goal of one-way and two-way bilingual education is to graduate students who are fully academically proficient in both languages of instruction, to prepare these students for the workplace of the 21^st century. We summarize native-Spanish-speakers’ long-term achievement on nationally standardized tests (Aprenda 2, SABE) in Spanish Total Reading (the subtest measuring academic problem-solving across the curriculum—math, science, social studies, literature), following them to the highest grade level reached by the program to date:

• In 50-50 Two-way bilingual immersion, Spanish-speaking immigrants after 1-2 years of U.S. schooling achieved at a median of the 62^nd NCE (71^st percentile) in Grades 3-6. These immigrants arrived on or above grade level and maintained above grade level performance in Spanish in the succeeding two years. (pp. 199-200, Figure D-2, Tables D-5, D-6)

• In 90-10 Transitional bilingual education classes, native-Spanish speakers reached the 56^th to 60^th NCE (61^st to 68^th percentile) for Grades 1-4, and after moving into all-English instruction in Grade 5, they tested at the 51^st NCE, still on grade level in Spanish reading achievement. (pp.117-119, Figure C-5, Table C-4)

• In 90-10 Developmental bilingual education classes, native-Spanish speakers reached the 56^th to 63^rd NCE (61^st to 73^rd percentile) for Grades 1-4, and in Grade 5 they outperformed the TBE comparison group by 4 NCEs at the 55^th NCE (60^th percentile). (pp. 117-119, Figure C-5, Table C-4)

• In 90-10 Two-way bilingual immersion classes, native-Spanish speakers reached the 58^th to 65^th NCE (64^th to 76^th percentile) for Grades 1-4, and in Grade 5 they outperformed the TBE and DBE comparison groups by a significant 6 NCEs at the 61^st NCE (70^th percentile). (pp. 117-119, Figure C-5, Table C-4)

• In reading achievement across the curriculum, native-Spanish speakers outperformed native-English speakers when tested in their native language, for Grades 1-8, regardless of the type of bilingual program Spanish-speaking students received. Native-Spanish speakers remained significantly above grade level at every grade except sixth grade (at the 49^th NCE), reaching the 64^th NCE (74^th percentile) in 8^th grade. (pp. 117-119, Figure C-3, Table C-3)

ACHIEVEMENT FINDINGS IN OTHER SUBJECTS:

• We chose the reading subtest of the standardized tests (results presented above) as the "ultimate" measure of attainment, because LM students’ reading scores were consistently the lowest among the subjects, and this is the measure that most closely correlates with the standardized tests required for admission to post-secondary education. Generally, LM students achieved 5-10 NCEs higher in English language arts, math, science, social studies, and writing. (pp. 46-53, 111-114, 119-122, 241-256, Figures A-4, A-5, C-9, C-10, E-1 to E-14 and accompanying tables)

• In Spanish math, native-Spanish speakers generally outperformed native-English speakers tested in English math. When comparing native-Spanish speakers’ achievement in Spanish math by program, for Grades 2-5, students attending all three bilingual program types achieved at or above the 55^th NCE (60^th percentile). But the Spanish speakers attending 90-10 Two-way bilingual immersion classes outperformed the Spanish speakers in 90-10 TBE and 90-10 DBE classes by 3-6 NCEs on Spanish math achievement, reaching the 59^th NCE (66^th percentile) by 5^th grade. (pp. 114, 117-118, Figures C-4, C-6, Tables C-3, C-4)

ACHIEVEMENT OF NATIVE-ENGLISH SPEAKERS IN TWO-WAY BILINGUAL ED:

• Native-English speakers in two-way bilingual immersion programs maintained their English, added a second language to their knowledge base, and achieved well above the 50^th percentile in all subject areas on norm-referenced tests in English. These bilingually schooled students equaled or outperformed their comparison groups being schooled monolingually, on all measures. (pp. 46-53, 119, 124, 201-204, Figures A-3 to A-5, D-1, D-3, D-5, D-7, D-9, Tables A-1 to A-11, C-4, C-12, C-13, D-1 to D-4, D-7, D-8, D-10, D-12, D-13, D-15, D-17 to D-10)

INFLUENCE OF STUDENT BACKGROUND ON STUDENT ACHIEVEMENT:

• Socioeconomic status (SES) typically influenced from 3-6% of LM students’ reading achievement as measured by standardized tests, for both enrichment dual language programs and ESL content programs. In selected circumstances (e.g., oral proficiency of Spanish speakers learning English) the effect of SES explains as much as 11-12% of achievement. However, the effect of number of years of program participation on reading achievement varied with the program type. For one-way and two-way dual language programs, up to five years of program participation accounted for 6-9% of ELLs’ reading achievement on standardized tests. For Spanish speakers learning English, 20% of oral proficiency was attributable to program exposure while program exposure accounted for 15% of oral proficiency for English speakers learning Spanish. In the case of the ESL Content program, years of schooling accounted for less than 2% of end-of-school reading achievement as measured by standardized tests. Thus, a strong dual language program can "reverse" the negative effects of SES more than a well-implemented ESL Content program by raising reading achievement to a greater degree. (pp. 56-57, 204-206, 256-258, Tables A-18, D-20, E-16 to E-18)

• The One-way developmental bilingual education program in Northern Maine influenced 8.5% of former ELLs’ eventual reading achievement, exceeding the effects of low socioeconomic status at less than 4%. The Two-way bilingual immersion program at Grant Community School exerted a powerful and significant effect on Spanish-speaking students’ scores on oral English development and influenced about 6 percent of their standardized reading scores as assessed in English, while SES accounted for about 4%. In this high-poverty school, SES alone accounted for 14 percent of the observed achievement variance overall. Thus, the school’s dual language program is reducing the negative effects of SES by significant amounts for Spanish speakers learning English and taking the statewide assessment in English. (pp. 56-57, 204-206, 256-258, Tables A-18, D-20, E-16 to E-18)

• Number of years of primary language schooling, either in home country or in host country, had more influence than socioeconomic status when the number of years of schooling was 4 or more years. In addition, the L2 academic achievement of older immigrant arrivals with strong grade-level schooling completed in L1 in the home country was less influenced by low socioeconomic status and more dependent on number of years completed. Likewise, students of low socioeconomic status who were born in the U.S. or arrived at a very young age achieved at high levels in L2 when grade-level schooling was provided in both L1 and L2 in the U.S. (pp. 257-258, Figures C-1, E-6, E-7, Tables C-1, E-6, E-7, E-17, E-18)

• When immigrants were schooled all in English in the U.S., students who received 4-5 years of L1 schooling in home country (arriving at ages 10-12) scored 6 NCEs higher in English reading in 11^th grade than those who received 1-3 years of home country schooling (arriving at ages 7-9). (pp. 248-251, Figures E-6, E-7, Tables E-6, E-7)

• Immigrants with interrupted schooling in home country achieved significantly below grade level, when provided instruction only in English. Those one year below grade level on arrival were at the 29^th NCE (16^th percentile) on the English reading test by 11^th grade, those two years below grade level on arrival at the 26^th NCE (13^th percentile), those three years behind at the 20^th NCE (8^th percentile), and those four years behind at the 19^th NCE (7^th percentile). (pp. 251-253, Figure E-8, Table E-8)

• Gender differences among Hispanic students were found to be significant in only two subject areas—math and science. Hispanic males outperformed Hispanic females by 4 NCEs in math and 6 NCEs in science on the 11^th grade tests in English. (p. 256, Figure E-14, Table E-14)

MAJOR POLICY IMPLICATIONS:

• Enrichment 90-10 and 50-50 one-way and two-way developmental bilingual education (DBE) programs (or dual language, bilingual immersion) are the only programs we have found to date that assist students to fully reach the 50^th percentile in both L1 and L2 in all subjects and to maintain that level of high achievement, or reach even higher levels through the end of schooling. The fewest dropouts come from these programs.

• Parents who refuse bilingual/ESL services for their children should be informed that their children’s long-term academic achievement will probably be much lower as a result, and they should be strongly counseled against refusing bilingual/ESL services when their child is eligible. The research findings of this study indicate that ESL or bilingual services, as required by Lau v. Nichols, raise students’ achievement levels by significant amounts.

• When English language learners (ELLs) initially attend segregated, remedial programs, these students do not close the achievement gap after reclassification and placement in the English mainstream. Instead, they maintain or widen the gap in later years. Therefore, their average achievement NCE at reclassification should be as high as possible, since this is likely to be their highest achievement level that they reach during their school years. Ideally, instructional gains are best accomplished in an enrichment (not a remedial) program.

• Students with no proficiency in English must NOT be placed in short-term programs of only 1-3 years. In this study and all other research studies following ELLs long term, the minimum length of time it takes to reach grade-level performance in second language (L2) is 4 years. Furthermore, only ELLs with at least 4 years of primary language schooling reach grade-level performance in L2 in 4 years. Students with no primary language schooling (either in home country or host country) are not able to reach grade-level performance in L2.

• The strongest predictor of L2 student achievement is amount of formal L1 schooling. The more L1 grade-level schooling, the higher L2 achievement.

• Bilingually schooled students outperform comparable monolingually schooled students in academic achievement in all subjects, after 4-7 years of dual language schooling.

• Students who receive at least 4-5 years of grade-level L1 schooling in home country before they emigrate to the U.S. typically reach the 34^th NCE (23^rd percentile) by 11^th grade when schooled all in English in the U.S. in an ESL Content program, and then the mainstream. These students are on grade level when they arrive, but it takes them several years to acquire enough English to do grade-level work, which is equivalent to interrupting their schooling for 1 or 2 years. Then they have to make more gains than the average native-English speaker makes every year for several years in a row to eventually catch up to grade level, a very difficult task to accomplish within the remaining years of K-12 schooling.

• The highest quality ESL Content programs close about half of the total achievement gap.

• When ELLs initially exit into the English mainstream, those schooled all in English outperform those schooled bilingually when tested in English. But the bilingually schooled students reach the same levels of achievement as those schooled all in English by the middle school years, and during the high school years the bilingually schooled students outperform the monolingually schooled students (see Figure C-2).

• Students who receive at least 5-6 years of dual language schooling in the U.S. reach the 50^th NCE/percentile in L2 by 5^th or 6^th grade and maintain that level of performance, because they have not lost any years of schooling. All students who are raised in a dual language environment need at least 4 years of schooling in L1 and 4 years of schooling in L2 to achieve on grade level in either of the two languages. Providing bilingual schooling in the U.S. meets both needs simultaneously, leading to high academic achievement in the long term.

• Bilingual/ESL Content programs must be effective (at least 3-4 NCE gains per year more than mainstream students are gaining per year), well implemented, not segregated, and sustained long enough (5-6 years) for the typical 25 NCE achievement gap between ELLs and native-English speakers to be closed. Even the most effective programs can only close half of the achievement gap in 2-3 years, the typical length of remedial ELL programs.

• An enrichment bilingual/ESL program must meet students’ developmental needs: linguistic (L1-L2), academic, cognitive, emotional, social, physical. Schools need to create a natural learning environment in school, with lots of natural, rich language (L1, L2), both oral and written, used by students and teachers; meaningful, ‘real world’ problem-solving; all students working together; media-rich learning (video, computers, print); challenging thematic units that get and hold students’ interest; and using students’ bilingual-bicultural knowledge to bridge to new knowledge across the curriculum.

©2002 Center for Research on Education, Diversity & Excellence. All rights reserved.