Research Evidence Five Standards for Effective Pedagogy and Student Outcomes

Technical Report No. G1 March, 2002 The following report is the first in a series to present research evidence on the effectiveness of the Five Standards for Effective Pedagogy. Research Evidence Five Standards for Effective Pedagogy and Student Outcomes Tharp, Estrada, Dalton, and Yamauchi propose the Five Standards for Effective Pedagogy as critical for improving learning outcomes for all students, and especially those of diverse ethnic, cultural, linguistic, or economic backgrounds. The Five Standards are: • Standard I -- Teachers and Students Producing Together Facilitate learning through joint productive activity among teacher and students. • Standard II -- Developing Language and Literacy Across the Curriculum Develop competence in the language and literacy of instruction across the curriculum. • Standard III -- Making Meaning -- Connecting School to Students' Lives Contextualize teaching and curriculum in the experiences and skills of students' homes and communities. • Standard IV -- Teaching Complex Thinking Challenge students toward cognitive complexity. • Standard V -- Teaching Through Conversation Engage students through dialogue, especially the Instructional Conversation. These standards are discussed extensively in Teaching Transformed (Tharp et al., Westview Press, 2000), but three clarifications are made here. First, these pedagogy standards are not intended to represent the full spectrum of complex tasks that comprise teaching; rather, they represent instructional activities that promote active student learning and must be adapted to varying contexts and diverse student needs. Second, these standards do not stand in opposition to small-group direct instruction. In fact, Tharp and Gallimore used the terms "direct" and "effective" interchangeably in their report on reading comprehension in the Kamehameha Early Education Program. Third, we do not propose these standards should be used to the exclusion of other strategies. Our data suggests that teachers who use the standards at higher rates are more likely, not less, to use a variety of other effective teaching strategies . A highly abstracted model of instruction using the Five Standards for Effective Pedagogy consists of a teacher and a small group of students having an instructional conversation while collaborating on a cognitively challenging activity contextualized in students' personal, social, or cultural knowledge and experience. The overarching goals or instruction are to foster complex thinking by all students, and language and literacy development in the language of instruction, as well as in the content domains. The latter is especially important for English Language Learners. An instructional unit developed at an American Indian reservation middle school provides a concrete illustration of the Five Standards model. Tribal leaders were asked to speak to students at an assembly, after which an eighth-grade teacher team created subject area units related to the topics presented. In mathematics, students worked in small groups to generate student surveys on the issues presented, the data from which served as the basis for a unit on fractions, decimals, and percents, with survey results presented in multiple representations such as pie charts, graphs, and frequency distributions. The unit culminated with student presentations and letters written to the tribal council to share survey results. In science, students focused on the issue of local water quality. They worked in small groups to sample several local water sources, including water fountains in the school itself, to examine for chemical content and pollutants. Five Standards and Student Academic, Cognitive, and Affective Outcomes Recent research has found a consistent relationship between use of the standards and a wide range of student outcomes. This research has examined teachers' use of the standards, both separately and in combination, with a variety of methods including case studies of multiple classrooms, short-term randomized designs and quasi-experimentation in single classrooms, and longitudinal studies of entire schools. Several studies (Saunders, 1999; Saunders, & Goldenberg, 1999a, 1999b, 2001, in press; Saunders, O'Brien, Lennon, & McLean, 1998) have found instructional conversations useful in assisting literacy development. Instructional conversations are planned, goal-directed conversations on an academic topic between a teacher and a small group of students. Although instructional conversations can be used to meet any learning goal in any content area, these studies have focused attention on the effectiveness of instructional conversations in developing thematic understanding of literature. In a recent study (Saunders & Goldenberg, in press), fourth-grade English Language Learners (ELLs) read a short story and then were randomly assigned into one of two kinds of lessons. The experimental group participated in an instructional conversation (IC); the control group participated in a directed reading lesson suggested in the teacher’s current reading series. Both groups achieved equivalent levels on post-tests of literal comprehension (76%), but a significantly larger number of students in the IC group (63%) demonstrated a clear understanding of the story theme than in the control group (13%). In another study, Saunders and Goldenberg (1999a) found that instructional conversation and contextualization (CTX) greatly assisted the reading comprehension and thematic understanding of students with varying levels of English proficiency. Fourth- and fifth-grade students were randomly assigned into four experimental conditions: (1) IC: teacher-led small group discussions of story content and theme; (2) CTX: students wrote in literature logs about personal experiences related to story content and theme; (3) IC + CTX; or (4) Control: reading and study only. The study found a strong independent IC effect on comprehension, with all students in the IC group scoring .75 standard deviations higher than controls. There was also a strong effect for instructional conversation and contextualization combined: students in the IC + CTX group scored 1.07 standard deviations higher than students in the control group. This effect held up for both fluent and limited English proficient students. The additional time spent sharing and discussing literature logs (contextualization) proved to be quite helpful for LEP students, but of lesser importance for fluent English proficient students. Among LEP students, 69% of the students in the IC + CTX group successfully explained and 56% successfully exemplified thematic understanding. The percentages among LEP students in both the IC and CTX conditions were virtually the same as that for LEP students in the control group: 6%-19% for explanation and 13%-25% for exemplification. There were no significant differences on thematic understanding for fluent English proficient students. The percentages of fluent-English proficient students who successfully explained and exemplified the story’s theme were equivalent across the IC, CTX, and IC + CTX groups: 69% could explain it, and 46-62% could exemplify it (the percentages for fluent English proficient controls were 46% and 31%, respectively). Doherty and Pinal (2002) used the Standards Performance Continuum (Doherty, Hilberg, Epaloose, & Tharp, 2002) to examine the influence of teachers' use of joint productive activity (JPA) during language arts instruction on the metacognitive development of predominantly Latino ELL students. In joint productive activity, the teacher and a small group of students co-construct meaning from a text. The teacher is a full collaborator in the activity, modeling his/her use of effective reading comprehension strategies while assessing and assisting students' comprehension efforts. This study found a significant positive association between teachers' use of JPA and students' self-reported use of cognitive reading strategies: teachers' use of JPA predicted students' self-reports of effective comprehension strategy use; JPA was unrelated to self-reports of ineffective strategy use. This study also found that self-reported effective strategy use predicted achievement gains on standardized comprehension tests, whereas ineffective strategy use, unrelated to JPA, predicted declines in comprehension achievement. A set of studies by Estrada over a four-year period have consistently shown a positive relation between implementation of the Five Standards and student outcomes in first and fourth grades. These studies also demonstrated that it is possible to assist teachers to implement the Standards and to garner student gains in achievement parallel to teachers' gains in pedagogical capacity. In the first year, first graders whose teachers were stronger implementers of features of the Five Standards scored higher in reading and language on the SABE. Fourth-grade students whose teachers were stronger implementers scored higher in reading and language on the SAT9 (Estrada, in press). Teacher ratings of student performance showed the same pattern. In the second year, in a subsample of six first grades, the vast majority of students in strong implementers' classrooms reached grade level in reading, whereas less than half did so in weaker implementers' classrooms (Estrada & Imhoff, 2001). Professional development provided to the same teachers in the third year produced parallel increases in teachers' implementation of the Standards and in student performance. Virtually 100% of students reached grade level in reading in strong implementers' classrooms, whereas 69% did so on average in weaker implementers' classrooms (Estrada & Imhoff, in press). Teachers continued to implement the Standards in the fourth year, and preliminary analyses of reading data indicate that all of the teachers maintained or increased gains in student achievement. Hilberg, Tharp, and DeGeest (2000) examined the efficacy of the Five Standards in mathematics instruction in a quasi-experimental study in an American Indian middle school. Two groups of American Indian eighth grade students were randomly assigned to either Five Standards or Traditional conditions for a one-week unit on fractions, decimals, and percents. Students in the Five Standards condition outperformed controls on tests of conceptual learning at the end of the study and exhibited better retention of unit content two weeks later. Two studies recently conducted at one of our OERI-funded Research and Demonstration Schools document the relationship between teachers' use of the Five Standards and student achievement and provide strong support for their effectiveness with diverse students. The school, serving predominantly low-income Latino ELLs, ranked in the second decile of California schools in 2001. For both studies, teachers' use of the standards was recorded with the Standards Performance Continuum (Doherty, et al., 2002), and student achievement gains were estimated from standardized test scores (SAT-9) from two consecutive years. The first study found that teachers' overall use of the standards reliably predicted achievement gains in comprehension, language, reading, spelling, and vocabulary (Doherty, Hilberg, Pinal & Tharp, 2002). In the second study, cluster analysis of data on teacher’ use of the Five Standards and classroom organization generated a four-group taxonomy of pedagogy (high vs. low use of the standards) and organization (whole class vs. activity settings). Analyses found that students whose teachers used the Five Standards extensively and their classroom organization consisted of multiple, simultaneous, diversified activity settings as proposed by Tharp et al. (2000) showed significantly greater achievement gains on all SAT-9 tests than students whose teachers had not similarly transformed their teaching. In fact, students whose teachers had transformed both their pedagogy and organization were the only group to evidence achievement gains; students in all other groups evidenced declines in achievement from the prior year. Teachers' use of the Five Standards has been linked to factors critical to school performance such as motivation, perceptions, attitudes, and inclusion. Predominantly Latino ELL students in classrooms where the Five Standards were used only slightly or moderately spent more time on-task, perceived greater cohesion in their classrooms, and perceived themselves as better readers having less difficulty with their work (Padron & Waxman,1999). American Indian students in mathematics classes integrating the Five Standards reported more positive attitudes toward mathematics (Hilberg, Tharp, and DeGeest. 2000). Findings, replicated over two years with two cohorts of students (Estrada & Imhoff, 2001, 2002), indicated that, across language programs, peer inclusion was greater in classrooms in which students participated in more peer joint productive activities (or peer collaboration). Effective Instructional Models Using the Five Standards O.L.A. is a language arts program for grades 3-5 developed by CREDE researchers in southern California (Saunders & Goldenberg, 2001). Tightly aligned with the Five Standards and exemplifying, in particular, Contextualization, Cognitive Complexity, and Instructional Conversation, O.L.A. effects have been tested (Saunders, 1999) and replicated (Saunders & Goldenberg, 1999b) with both longitudinal and short-term quasi-experimental designs. Comparisons of randomly selected matched samples of O.L.A and non-O.L.A students indicate the program produces higher levels of Spanish literacy, significantly higher levels of English literacy, and important literacy-related practices and attitudes for significantly larger numbers of students (Saunders, 1999). By grade 5, O.L.A students, on average, score at least one half of a standard deviation higher than matched controls on standardized tests of English reading and approximately .60 to .75 standard deviations higher on standardized tests of English language expression and mechanics (Saunders, 1999; Saunders & Goldenberg,1999b; Saunders et al., 1998). Researchers have also documented the effectiveness of the Five Standards using the Sheltered Instruction Observational Protocol (Echevarria, Vogt, & Short, 2000). Sheltered instruction, grounded in two decades of classroom-based research, is an approach for teaching content to English language learners in strategic ways that make the subject matter concepts comprehensible while promoting students' English language development. SIOP has a strong Language and Literacy Development component, CREDE’s second standard, and incorporates elements of the other four standards. Studies on the effects of sheltered instruction found that ELLs in middle school classes of teachers trained in sheltered instruction out-performed control students on overall gains in expository writing, and made significant improvement in all areas measured by a writing rubric: language production, focus, support/elaboration, organization, and mechanics (Echevarria, Short, & Powers, 2002). These consistent findings from instructional models and programs, and controlled and correlational studies demonstrate a systematic relationship between use of the Standards for Effective Pedagogy and improved student performance across a broad range of outcomes. Taken together, these findings provide strong support for the instructional effectiveness of the Standards for Effective Pedagogy. References • Doherty, R. W., Hilberg, R. S., Epaloose, G., & Tharp, R. G. (2002). Development and validation of the Standards Performance Continuum: A performance-based measure of the Standards for Effective Pedagogy. Manuscript submitted for publication. • Doherty, R. W., Hilberg, R. S., Pinal, A., & Tharp, R. G. (2002). Transformed pedagogy, organization, and student achievement. Paper presented at the annual conference of the American Education Research Association, New Orleans, LA. • Doherty, R. W., & Pinal, A. (2002). Joint productive activity and the development of metacognitive thinking. Manuscript submitted for publication. • Echevarria, J., Short, D., Powers, K. (2002). Using sheltered instruction to improve the achievement of English language learners. Manuscript in preparation. • Echevarria, J., Vogt, M. E., Short, D. (2000). Making content comprehensible for English language learners: The SIOP model. Boston: Allyn and Bacon. • Estrada, P. (in press). Patterns of language arts instruction activity: Excellence, inclusion fairness, and harmony in first and fourth grade culturally and linguistically diverse classrooms. In H. C. Waxman & R. G. Tharp & R. S. Hilberg (Eds.), Observational research in U.S. Classrooms: New approaches for understanding cultural and linguistic diversity. Cambridge, MA: Cambridge University Press. • Estrada, P. & Imhoff, B. D. (2002, in preparation). Peer joint productive activity and peer inclusion in culturally and linguistically diverse classrooms. • Estrada, P., & Imhoff, B. D. (2001). Patterns of language arts instructional activity: Excellence, inclusion, fairness, and harmony in six first grade classrooms. Paper presented at the annual meeting of the American Education Research Association, Seattle, WA. • Estrada, P., & Imhoff, B. (in press). One road to reform: Professional development, pedagogy, and student achievement in the context of state reform of literacy instruction. In S. Stringfield & A. Datnow (Eds.), The imperfect storm: Successes and failures of school reform efforts in multicultural/multilingual settings. Cambridge, MA: Cambridge University Press. • Hilberg, R. S., Tharp, R. G., & DeGeest, L. (2000). The efficacy of CREDE's standards-based instruction in American Indian mathematics classes. Equity and Excellence in Education, 33(2), 32-39. • Padron, Y. N., & Waxman, H. C. (1999). Classroom observations of the Five Standards of Effective Teaching in urban classrooms with English language learners. Teaching and Change, 7(1), 79-100. • Saunders, W. (1999). Improving literacy achievement for English learners in transitional bilingual programs. Educational Research and Evaluation, 5(4), 345-381. Saunders, W., & Goldenberg, C. (1999a). The effects of instructional conversations and literature logs on the story comprehension and thematic understanding of English proficient and limited English proficient students. Santa Cruz, CA: Center for Research on Education, Diversity & Excellence, University of California. • Saunders, W., & Goldenberg, C. (1999b). The effects of comprehensive Language Arts/Transition Program on the literacy development of English learners (Technical Report). Santa Cruz, CA: Center for Research, Diversity & Excellence, University of California. • Saunders, W., & Goldenberg, C. (2001). Opportunities through Language Arts: Overview video, video guide, and program manual. • Saunders, W., & Goldenberg, C. (in press). The effects of an instructional conversation on transition students' concepts of friendship and story comprehension. In R. Horowitz (Ed.), The evolution of talk about text: Knowing the world through classroom discourse. Newark, DE: International Reading Association. • Saunders, W., O'Brien, G., Lennon, D., & McLean, J. (1998). Making the transition to English literacy successful: Effective strategies for studying literature with transition students. In R. Gersten & R. Jimenez (Eds.), Promoting learning for culturally and linguistically diverse students. Monterey, CA: Brooks Cole Publishers. • Tharp, R. G., Estrada, P., Dalton, S. S., & Yamauchi, L. (2000). Teaching transformed: Achieving excellence, fairness, inclusion, and harmony. Boulder: Westview Press. Contributors R. William Doherty Center for Research on Education, Diversity & Excellence University of California, Santa Cruz Jana Echevarria California State University, Long Beach Peggy Estrada Center for Research on Education, Diversity & Excellence University of California, Santa Cruz Claude Goldenberg California State University, Long Beach R. Soleste Hilberg Center for Research on Education, Diversity & Excellence University of California, Santa Cruz William M. Saunders California State University, Long Beach Roland G. Tharp Center for Research on Education, Diversity & Excellence University of California, Santa Cruz This work was supported under the Education Research and Development Program, PR/Award No. R306A60001, the Center for Research on Education, Diversity & Excellence (CREDE), as administered by the Office of Educational Research and Improvement (OERI), National Institute on the Education of At-Risk Students (NIEARS), US Department of Education (USDOE). The contents, findings and opinions expressed here are those of the author and do not necessarily represent the positions or policies of OERI, NIEARS, or the USDOE.