Lynda

The Harmful Impact of the TAAS System of Testing in Texas
Most interesting.
 
Lynda
----- Original Message -----

Home Page



THE HARMFUL IMPACT OF THE TAAS SYSTEM OF TESTING IN TEXAS:
BENEATH THE ACCOUNTABILITY RHETORIC

Linda McNeil
Department of Education and Center for Education
Rice University
Houston, Texas
&
Angela Valenzuela
Department of Curriculum and Instruction and the Center for Mexican American Studies
University of Texas at Austin
Austin, Texas


This paper is not to be cited, reproduced or distributed without the prior written consent of the authors.

Copyright © Linda McNeil and Angela Valenzuela

Introduction

Those who promote state systems of standardized testing claim that these systems raise the quality of education and do so in ways that are measurable and generalizable. They attribute low test scores to management’s failure to direct its "lowest level" employees (i.e., the teachers) to induce achievement in students. In Texas, the remedy to this situation has been to create a management system that will change behavior, particularly the behavior of teachers, through increased accountability. The means of holding teachers and administrators accountable is the average scores of each school’s children on the state’s standardized test, the Texas Assessment of Academic Skills, or "TAAS." As our report will show, this over-reliance on test scores has caused a decline in educational quality for those students who have the greatest educational need.

Texas, a state with a history of low educational achievement and low investment in public education, has put into place an accountability system that hinges on the testing of children. The test has high stakes consequences for the children: not passing the high school-level test is a bar to graduation (regardless of the student’s accomplishments and courses passed); and soon, scores on the reading section of the test will determine whether a child can be promoted from third to fourth grade.1

 The scores on the test are also used as the chief means of monitoring the performance of teachers, principals, and schools. School-level aggregations of children’s scores are used to rate principals, schools, and even superintendents.

The rhetoric surrounding this accountability system is that it is raising educational quality. Politicians claim that this testing system is "saving" the Texas schools. The system is gaining national recognition as an exemplary accountability system, because scores on the state test have, in most districts, been rising. The system’s popularity is further bolstered by the idea that it must be improving the education of Latino and African American children, since, in many parts of the state, their test scores are also rising.

Before other states rush to copy the Texas system, it is important to look behind the reported numbers. It is necessary to see the effects that the test and the testing system are having on the children and on what they are learning. It is essential to examine the effects of this system of testing on teachers. And it is critical to analyze the effects of this system of testing on children’s lives.

We present here our strong assessment that the TAAS system of testing is reducing the quality and quantity of education offered to the children of Texas. Most damaging are the effects of the TAAS system of testing on poor and minority youth.

Our analysis draws on emerging research on high stakes testing and on our individual investigations. These include a longitudinal analysis of instruction when test-prep materials become the curriculum, a multi-year study of Latino children’s high school experiences, investigations of Latino elementary schools, and research in urban schools with predominantly Mexican American and African American children (McNeil, 1988, 2000; Valenzuela, 1997, 1998, 1999). Our investigations have entailed the documentation of the effects of centralized testing in Texas beginning with the Perot reforms of the 1980s and on through the 1990s, when such tests were increasingly tied to "high stakes" for children and school personnel (McNeil, 1987, 2000). Our investigations have included interactions over the past ten years with literally hundreds of public school teachers, representing all subjects and grades and a wide mix of urban and suburban districts, whom we have encountered through research projects and the teacher enhancement and school reform programs at the Rice University Center for Education. Our research required fieldwork in schools and classrooms and frequent interactions with students, teachers, and administrators, whose voices and experiences are vital to capture.

In essence, our investigations have permitted us to gather and triangulate data from a variety of sources over a multi-year period. From these investigations, we understand that there are a wide variety of responses to the TAAS. However, those we report are the most characteristic among the many schools, teachers, and students included in our research. The effects of the TAAS, which we describe herein, represent strong, persistent trends emerging from the data. We are not describing isolated or aberrant cases. To the contrary, we have unavoidably encountered this evidence in overabundance, and we have encountered this evidence even in the course of pursuing other research topics.

Our analysis reveals that behind the rhetoric of rising test scores are a growing set of classroom practices in which test-prep activities are usurping a substantive curriculum. These practices are more widespread in those schools where administrator pay is tied to test scores and where test scores have been historically low. These are the schools that are typically attended by children who are poor and African American or Latino, many non-English-language dominant. These are the schools that have historically been under-resourced. In these schools, the pressure to raise test scores "by any means necessary" has frequently meant that a regular education has been supplanted by activities whose sole purpose is to raise test scores on this particular test.

Because teachers’ and administrators’ job rewards under the TAAS system of testing are aligned to children’s test scores, the TAAS system fosters an artificial curriculum. It is a curriculum aimed primarily at creating higher test scores, not a curriculum that will educate these children for productive futures. The testing system distances the content of curriculum from the knowledge base of teachers and from the cultures and intellectual capacities of the children. It is creating an even wider gap between the curriculum offered to children in traditionally high-scoring schools (white, middle and upper-middle class) and those in typically minority and typically poor schools.

In this paper we present what we are seeing as the direct, negative educational consequences of this system of testing, and its harmful effects on the quality of education available to disadvantaged Latino and African American children. Under the TAAS system of testing, curriculum, instruction, school resources, and children are all adversely affected:

  • The TAAS system of testing reduces the quality and quantity of curriculum.
  • The TAAS system distorts educational expenditures, diverting scarce instructional dollars away from such high quality curricular resources as laboratory supplies and books toward test-prep materials and activities of limited instructional value.
  • TAAS provokes instruction that is aimed at the lowest level of skills and information, and it crowds out other forms of learning, particularly for poor and minority students.
  • TAAS-based teaching and test-prep violates what is known about how children learn.
  • The TAAS is divorced from children’s experience and culture.
  • The test is imposing exit measures that are particularly inappropriate for LEP (Limited English-Proficient) students.
  • TAAS is widening the gap between the education of children in Texas’ poorest (historically low- performing) schools and that available to more privileged children.

An analysis of the actual effects of the TAAS on children’s educational experience is especially important given the publicity that has surrounded both the TAAS scores and Texas children’s performance on the National Assessment of Educational Progress (NAEP) exams. The latter exams present a mixed picture: there are some gains in the early grades, but Texas NAEP gains in the early grades do not seem to hold up in later grades. Also, the attention has been more on the rate of improvement on NAEP rather than actual improvement. Highly touted rates of improved scores (for example, that Texas was described as in the top four "most improved states") mask the fact that even after such "gains," Texas students were still at or below average, registering lower than 21 of 40 participating states. Hence, neither the TAAS, nor attempts to interpret Texas’s NAEP rate gains as support for TAAS, represent the actual status of education for youth in Texas.2

The Texas Accountability System and Minority Youth

The Texas Assessment of Academic Skills (TAAS) system of testing and test-driven curriculum was enacted by the Texas state legislature in the spring of 1990 (Texas Education Agency, 1998). It is the most recent in a series of centralized, standardizing "reforms" in that state. It differs from earlier test systems in being increasingly tied to teacher and principal job security and pay.

The policy rhetoric surrounding the TAAS goes like this: By measuring student performance on a computer-scored standardized test, inferences can be drawn about the quality of teacher and principal performance, as well as the aggregate quality of the school. School ratings, periodically published in full-page spreads in the state’s newspapers, now serve as barometers for the condition of education in the state.

Hence, important judgments about personnel and the quality of schooling now revolve around a single indicator. There are clearly serious problems when any single indicator is used to assess the quality of so complex an enterprise as educating children. This is why the use of a single indicator to assess learning or to make decisions about tracking, promotion, and graduation violates the ethics of the testing profession (Heubert & Hauser, 1998).

However, in this paper, our focus is not on the technical problems that are typically raised for this and other tests (e.g., probable impacts of particular questions, cut-off scores for passing, psychometric properties of the test, etc.), all of which are quite real. Rather, we want to focus on another consequence which is frequently overlooked: the direct, negative impact that this accountability system is having on the education of this state’s most economically disadvantaged, minority children. What we are seeing is not the "misuse" of the TAAS testing system, but the playing out of its inherent logic at the expense of our poorest, minority children.

These are the children who comprise the preponderance of minority youth in large urban school districts. For example, in the Houston Independent School District (HISD), 52.5 percent and 34.1 percent of all students are Latino or African American, respectively, while 10.6 percent are Anglo. Moreover, 73 percent of the total are on free or reduced lunch (HISD District and School Profiles, 1997-98).

To fully assess the impact that this system of testing is having on the education of minority youth, it is essential to understand how this system is operationalized in classrooms. One of the misleading features of this testing system is the notion that publishing scores disaggregated by the race and ethnicity of the children alerts the public to inequities. In reality, these scores are misleading and diversionary: The scores loom so large that they overshadow discussion of other, more telling indicators of quality of education, among these the degree of segregation, the level of poverty, or the number of students graduating, taking the SAT, and going to college. Furthermore, the scores mask the inequities produced when schools raise test scores at the expense of substantive learning.

Drawing on our collective and extensive research, throughout this paper we illuminate the ways that this testing system harms the educational quality and educational opportunity of minority and economically disadvantaged youth. Our critique is organized around very specific and documented harmful effects of the TAAS system of testing on the quality of the curriculum, the quality of instruction, and on resource allocation for the education of minority youth.3 Because of their dramatically increasing representation in our nation’s largest school districts, a section below also addresses in some detail the even greater damaging effects of the TAAS test for Limited English Proficient (LEP) youth in our schools.4

The Educational Impact of the TAAS

In many urban schools, whose students are overwhelmingly poor and African American and Latino, the TAAS system of testing reduces the quality and quantity of subjects being tested by TAAS.

The pressure to raise TAAS scores leads teachers to spend class time, often several hours each week, drilling students on practice exam materials. This TAAS drill takes time from real teaching and learning: Much of the drill time is spent learning how to bubble-in answers, how to weed out obviously wrong answers, and how to become accustomed to multiple-choice, computer-scored formats. In the name of "alignment" between course curricula and test, TAAS drills are becoming the curriculum in our poorest schools.

The pressure to raise TAAS scores leads teachers to substitute commercial TAAS-prep materials for the substance of the curriculum. Principals, deans of instruction, and other building or central office administrators urge or even require teachers to set aside the course curriculum and to use the TAAS-prep materials in their place.

Although TAAS is supplanting a more substantial curriculum throughout the state, the problem emerges unevenly. It is more common in traditionally low-performing schools, the schools attended by low-income and non-Anglo children. In contrast, middle-class children in white, middle class schools are reading literature, learning a variety of forms of writing, and studying mathematics aimed at problem-solving and conceptual understanding. In essence, these children continue to receive an education appropriate for their age and grade level, while poor and minority children are devoting class time to practice test materials whose purpose is to help children pass the TAAS. The TAAS system of testing thus widens the gap between the public education provided for poor and minority children and that of children in traditionally higher-scoring (that is, Anglo and wealthier) schools.

Advocates of a state standardized system of testing frequently make the argument that "before TAAS, minority children were receiving nothing. Now at least, they are getting something (even if it is just exposure to the kinds of information that will be tested in the multiple-choice format)."

We have seen no studies that have documented this claim. But what we have seen is a reduction in content, even in those schools historically under-served and under-resourced, when the TAAS becomes the focus. An experienced Anglo English teacher at Seguín High School (a pseudonym),5 a predominantly Mexican HISD school, underscores this point. She commented that she teaches "less" English each year.6

Less as time goes on. Less as time goes on with the TAAS test thing. Because we have to devote so much time to the specific functions of the TAAS test, it’s harder and harder [to teach English].

Subjects tested by TAAS (reading, writing, and mathematics) are reduced, in the test and in the test-prep materials, to isolated skills and fragments of fact. This artificial treatment of subjects into isolated components may enable children to recognize those components on a multiple-choice test. However, this treatment does not necessarily enable children to use these components in other contexts. For example, high school teachers report that although practice tests and classroom drills have raised the pass rate for the reading section of the TAAS at their school, many of their students are unable to use those same skills for actual reading. These students are passing the TAAS reading by being able to select among answers given. But they are not able to read assignments, to make meaning of literature, to complete reading assignments outside of class, nor to connect reading assignments to other parts of the course such as discussion and writing.

Middle school teachers report that the TAAS emphasis on reading short passages, then selecting among answers given to questions based on those short passages, has made it very difficult for students to read a sustained reading assignment. After several years in classes where the "reading" assignments were increasingly TAAS practice materials, middle school teachers in more than one district have reported that children were unable to read a novel even two years below grade level.

In writing, students are increasingly being asked to write repetitively in only the format of that year’s TAAS writing objective. One African American parent, whose first son received an excellent fourth grade education in a Texas elementary school a few years ago, reported that when her second son entered fourth grade at the school in the fall of 1998, the entire fourth-grade curriculum had disappeared. The teachers, facilities, and principal were all the same, but the fourth grade curriculum had essentially been replaced by daily writing of "the persuasive essay," according to the strict TAAS format for that essay.

The approved format for the TAAS essay, for all children in the state, is the "five paragraph essay." Within this essay format, long discredited by teachers of writing, each of five paragraphs contains exactly five sentences. Each paragraph begins with a "topic sentence," followed by three "supporting sentences," and a "concluding sentence," which essentially recaps the topic sentence. Paragraph one serves the essay in the same way the topic sentence serves each paragraph; the concluding paragraph essentially plays the role in the essay that each concluding sentence plays in its own paragraph.

Teachers of writing who work with their students on developing ideas, on finding their voice as writers, and on organizing papers in ways appropriate to both the ideas and the paper’s intended audience find themselves in conflict with this prescriptive format. The format subordinates ideas to form, sets a single form out as "the essay," and produces, predictably, rote writing. Writing as it relates to thinking, to language development and fluency, to understanding one’s audience, to enriching one’s vocabulary, to developing ideas, has been replaced by TAAS writing to this format. "Writing" as TAAS prep no longer bears any resemblance to what research shows to be a developmental activity in children’s language competency.

However, the essay is easily scorable in the testing of large populations because what counts is the strict format rather than the development of ideas. Many teachers describe having to teach this format in terms of its intended audience, "Just think of yourself as writing for someone sitting in a cubicle in a bureaucracy, counting sentences and indentations." Others report teaching their students that there is "TAAS writing" and "real writing." Still others say that in their school teachers are required to have students do only TAAS writing, the five-paragraph essay, weekly or even daily until after the TAAS test (in other words, until late spring).

Students also complain about the persuasive essay. For example, an intellectually gifted, Mexican American senior male assigned to (i.e., misplaced) in Seguín High School’s "regular" track, feels that the school’s priorities are misguided. In his view, the school should be preparing students for the SAT. On the subject of many of his peers’ academic limitations, he provided the following commentary:

I’ve realized that the kids have good arguments here, but they have absolutely no argument skills. They probably have persuasive skills for the TAAS. Argument skills, none. The only argument they have is probably to curse. Say the F-word and that’s it.

Reflecting on a conversation he had on the subject of the college application process with his friends, he further places the onus of students’ limitations on the school itself:

They [friends] were telling me that they are kind of fed up with that [the persuasive essay], because here are kids that are filling out scholarship applications and essays. They are writing essays, and they don’t know how to write anything else but a persuasive essay for the TAAS. They prepare us for the TAAS, and we are not just test takers. I try to explain to people that we are not just persons who need to develop critical thinking skills on how to choose between five things. How to choose the correct answer out of five. That’s nothing!

The required "TAAS objectives" or "TAAS prompts" which are to be drilled each day are often presented to teachers as five- to ten-minute exercises. However, teachers report that drilling to these prompts, often required by the administration if their children are poor and minority (with a history of low scores), frequently usurps so much of the class period that little time is left for teaching and learning.

It is a myth that TAAS sets the minimum standards and that teachers are encouraged to go beyond that. In many schools, it is the best-prepared teachers with the richest curriculum who are required to scale back in order to teach to the sequence and format of the TAAS. In low-performing schools, even the most knowledgeable teachers are asked to set aside their lesson plans and materials to teach to the TAAS.

Whether children were being taught "nothing" before or whether they were being given a meaningful curriculum, the pressure to raise TAAS scores shows no evidence of opening children’s access to great literature, to conceptual understanding in mathematics, to fluency in writing, or to other learning experiences that seriously address previous inadequacies in their education. Nor does TAAS seem likely to do so. Under the current accountability system based on this test, financial rewards go to those schools whose scores go up, not necessarily to those in need of serious upgrading of staff and materials. In addition, the statewide system of testing has not been accompanied by a parallel investment that could reduce inadequacies and inequities in low-performing schools. The result is that many very real problems persist, problems which are not addressed by more, and more centralized, measurement and testing, or problems which testing may exacerbate by its focus on a narrow set of measures rather than a comprehensive look at children’s learning.

The TAAS system of testing reduces the quality and quantity of course content in subjects not tested by TAAS, because teachers are encouraged or required to substitute TAAS test preparation activities for the curriculum in those subjects.

The study of science, social studies, art, and other subjects that are not examined by the TAAS are all undermined by the TAAS system. For example, many science teachers in schools with poor and minority children are required by their principals to suspend the teaching of science for weeks, and in some cases for months, in order to devote science class time to drill and practice on the math sections of the TAAS. The first loss, of course, is the chance to learn science. The second is the chance to learn to become highly knowledgeable in mathematics. Many science teachers have little background in mathematics; the "mathematics" they are doing is drill and practice with commercial TAAS-prep materials.

The direct loss of both science and mathematics learning is clear. Less obvious, but equally important, is the way this TAAS practice widens the gap in science learning between children in middle class, higher performing schools, who continue to study science, and poor and minority children, whose science classes are more frequently sacrificed for TAAS prep.

Social studies is another core subject that is frequently suspended or interrupted for TAAS preparation. For instance, all history and social studies teachers in one Latino high school were told several months in advance of the TAAS to spend twenty minutes a day preparing students for the TAAS. According to one of these teachers:

Twenty minutes is just too much. By the time we get to teaching our history lesson, most of the time is gone. And then it’s hard to re-capture the rhythm of the previous day’s lesson, which was also interrupted by the TAAS.

The content of elective subjects is also often set aside to make time for TAAS prep. Art teachers report that they are required to drill on the grammar sections of TAAS. An ROTC instructor was assigning the five-paragraph essay each week, not to link writing to ROTC but to add another drill to the regimen, this time in a rote essay.

The TAAS system distorts educational expenditures, diverting scarce instructional dollars away from such high quality curricular resources as laboratory supplies and books toward test-prep materials and activities of limited instructional value beyond the test.

Under TAAS, there is a widespread press to spend instructional dollars on test-prep materials and activities. These include expenditures on expensive materials for "alignment" and "accountability" systems and consultants. It also includes diverting dollars from Texas’ classrooms to out-of-state vendors of tests, test-prep materials, consultants and related materials.

The press to spend instructional dollars for test prep is felt most especially in schools with large populations of poor and minority children, which have been historically underfunded. In these schools, scarce instructional dollars are being diverted into materials and activities whose only value is to increase TAAS scores, not to produce educated children well prepared for college or future work. For example, to the extent such schools had fewer sets of classroom novels and other reading materials before, the pressure of the TAAS test does not lead to such purchases. Rather, it tends to lead to the purchase of costly, commercial test-prep materials. These provide practice in answering multiple choice, recall questions pertaining to brief passages that are written explicitly for test-prep purposes.

This diversion of dollars further widens the gap between the quality of education offered to poor and minority children and that provided to wealthier children. Middle class and wealthy districts either do not spend money on these TAAS-related systems or they have the capacity to make up the difference in local funding for schools; either way, wealthier districts continue spending money on high quality instructional materials, which advance their children’s education and place them in the national mainstream of what is considered to be a quality education.

Increasingly, expenditures for management and "alignment" systems are displacing instructional expenditures, and expenditures for management conferences and consulting around increasing compliance with TAAS are displacing programs of teacher learning and professional pedagogical development. As the state curriculum becomes increasingly test-driven, it becomes difficult to disaggregate test-prep from "curricular" expenditures. Administrators argue that materials aimed at boosting test scores are in fact instructional expenses.

Parents and taxpayers, however, are often surprised to learn that what may appear to be curricular and instructional, in budgets and expenditure lines, do not resemble instructional materials in the tradition sense. Software, for example, can be a means of providing students with up-to-date scientific or historical information, with activities for manipulating geometric figures or for learning to conduct information searches. However, in schools dominated by TAAS, software often focuses on test-prep drills, little different from drill-and-practice worksheets. "Reading" may make the lay person think of books, magazines, anthologies of literature, but in TAAS controlled schools, it may take the form of purchased practice tests of short reading passages.

The justification for these materials is often that they give students practice with the "basics," which they must master before "going on." However, among the schools we have visited, those where even low-achieving students are taught the subjects tend to have students who both learn more and score higher on TAAS than those schools where TAAS-prep has displaced the curriculum.

In districts where schools’ TAAS scores are tied to incentive pay for teachers or principals, and where TAAS-based performance contracts have replaced tenure, there is an even greater tendency for school personnel to shift dollars away from instruction and into the expensive TAAS-prep and "alignment" materials and consultants. Again, frequently these incentives are applied in schools or districts whose populations are poor or minority or both. And, again, because such consultants and materials are narrowly focused on boosting test scores, they are unlikely to enhance children’s capacity for thinking and learning in the many realms beyond the TAAS test.

The inversion of incentives needs to be systematically examined. If only those schools whose scores increase receive additional funding (a form of merit reward), then the incentive to focus only on TAAS will increase (to the detriment of more substantial learning). Likewise, if the neediest schools, which are least likely to have adequate resources, are trapped in that need until they can raise scores, they will see no compensating investments to bring their students opportunities to learn in line with more privileged schools. There is no plan to make a massive investment in the neediest schools. In fact, much of the public rhetoric is that "low performing schools" do not deserve additional public investment. It is as if their poverty is tied in some Puritanical sense to lack of virtue, with low scores as their Scarlet Letter of guilt.

The TAAS system of testing emphasizes the lowest level of information and skills, crowds out other forms of learning, and disengages students in many urban schools, particularly those whose children are poor and non-Anglo.

Teachers report that the pressure to drill for TAAS has caused them to omit or severely decrease other forms of learning because of the lack of time or because their principals are urging them to devote time only to those activities which will be measured by the TAAS. Library research, independent projects, science experiments, oral histories, long-term writing assignments, writing assignments different from those being tested in a particular year, longer-term reading assignments that include related writing and speaking activities – all are being reduced, even though they are highly motivating for children. Such instructional activities, which engage children in higher-order problem solving and thinking, are deleted in those schools (poor, minority) where TAAS scores have been low.

The rigor of the academic learning is also sacrificed by courses like "TAAS Math" and "TAAS English," which certain students are required to take to enable them to pass the exit exam. The curriculum in these classes is predictably fragmented and incoherent, because teachers teach abstracted pieces of the curriculum in order to cover the various segments of the exam. Referred to as "local credit," such courses do not count toward the 24 credits that are required for high school graduation. Since they are not "real" math and English courses, they are not taken very seriously by the students.

Placement in these courses also undermines learning due to wasteful and bureaucratic scheduling issues. If a student is enrolled in a fall TAAS course and takes and passes the exam in October, he or she will have to wait out the rest of the semester in that "non-course" until it is over. An administrator from one of the larger, well-integrated HISD schools where such courses are offered told us that students start skipping class after they pass the TAAS test "since the course doesn’t count toward high school credit anyway." According to this individual, TAAS courses are also wasteful if the students fail the fall exam, because they have to sign up a second time in another such course. This expands to a whole year the time that students are not in the regular curriculum. In addition to the TAAS courses, students must of course meet the normal math and English requirements for graduation. Additional requirements tend to also translate either into summer school attendance or graduating off schedule, if at all.

While Anglo and middle class children in this same HISD school overwhelmingly pass the TAAS test within the context of the normal curriculum, an entire segment of the school’s population is being subjected to months, if not years, of TAAS prep. By treating these students as if they are unteachable, the system itself engenders a cumulative deficit in students’ knowledge, encouraging their resistance not to education, but to schooling (Valenzuela, 1999). Since the content of schooling is already deemed by many regular track (non-honors or college-bound) youth to be boring, unrewarding, and irrelevant to their lives, we are seeing the TAAS system of testing promote an even greater sense of alienation.

Preliminary research is showing that those schools that score higher on TAAS (usually wealthier, with fewer minority children) rarely teach directly to the TAAS. They teach children; they teach science, math, social studies, literature, writing, the arts. They teach the subjects. A tortured logic governs the highly prescriptive administration of the TAAS in predominantly minority schools: If the scores increase, it is because the school taught more to the test; however, if the scores decrease, the school needs to teach more to the test.

Yet, teaching to the test and thereby improving scores does not indicate increased learning or improved capacities for complex problem solving. For example, one largely Hispanic, traditionally low-performing high school with virtually no library, a severe shortage of textbooks, and little laboratory equipment for its students, spent $20,000 (almost its entire instructional budget) for a set of commercial test-prep materials. Even the school’s best teachers were required to set aside their high-quality lessons and replace them with the test-prep materials. Scores on some sections of the TAAS did go up, but teachers report that students’ actual capacity to read, to handle high school level assignments, to engage in serious thought and be able to follow through on work actually declined.

This school, touted in the newspapers for increasing the TAAS passing rate on reading, is now searching for a way to counter what is seen by the faculty as a serious deficiency in the students’ ability to read. It is clear that higher scores do not mean that children are learning to a higher level. Such scores may mean that nothing is being taught except TAAS-prep.

The TAAS system of testing goes against what is known in research on children’s learning.

Research on children’s learning shows that learning is not linear, that it must build on what children already know and understand, that it must engage children’s active thinking, and that it must engage many senses (Gardner, 1991; Ohanian, 1999; Sacks, 1999). In striking contrast, the TAAS reinforces one particular mode of learning. This cognitive impact of the test has not been seriously investigated. Classroom observations and teacher reports, however, raise critical questions about the sort of learning that is reinforced, those which are subordinated to TAAS formats, and those which are increasingly structured out of test-dominated classrooms.

The TAAS mode of learning is to "master" brief, discreet, randomly selected pieces of information. The reading comprehension and grammatical sections of the writing TAAS, for example, cover isolated skills through very brief written passages. These written passages are not intended to build a cumulative knowledge base; they are not meant to connect with children’s understanding. The isolated skills are presented in fragments, carefully sequenced to match the fragmented and isolated skills in the Texas curriculum frameworks. Learning fragments of fact and skill out of context is known to be counterproductive to understanding and to building cumulative skills which can be applied in an unfamiliar setting or to unfamiliar information in the future.

Two features of the TAAS and TAAS-prep materials are especially damaging to learning. The first is that under the TAAS system, students are to choose among possible answers that are given to them; they rarely have to think on their own, puzzle out a problem, come up with a possible answer, or articulate an idea. This engenders passivity and a dependent learning style that fails to develop many essential cognitive skills. The second is that TAAS presents the child with choices, of which all but one are incorrect. To the extent that children, especially in poor and minority schools, are taught a curriculum and test drills that are in the TAAS format, they are spending three-quarters of their learning time considering erroneous, "wrong" material. It is doubtful that there is any respectable learning theory that advocates children’s continual exposure to incorrect material.

Again, the TAAS system places most at risk the children in schools that heavily emphasize raising TAAS scores (usually poor and minority). These children not only fail to learn the same rich, complex material that children in middle class schools learn, but they are simultaneously required to devote hours and hours each week to a de facto worthless curriculum. By keeping children focused on these drills and these disembodied facts, the TAAS system of testing is denying them access to forms of knowledge and ways of knowing that can lead them beyond this minimal level, into higher forms of learning. That is why one teacher said that yes, under TAAS, certain students in her school who previously were not being taught much math (these were bilingual students, recently immigrated), are "getting more math now that we are testing everyone." But she cautioned, "but of course, it’s not real math – it’s not what you would want for your children. It’s just TAAS Math." The opportunity costs of spending weeks, months and even years on test drills which narrow learning modes and close off complex thought may be one of the costliest effects of the TAAS system of testing. It is a cost being borne by the least-well served children in our schools.

The generic curriculum inherent in the TAAS system of testing is divorced from children’s experiences, language and cultures

.

The TAAS system of testing is not respectful of, nor does it build on, children’s personal experiences, the cultures of their families, nor the variations in learning style and interests that span any classroom. ("Subtractive schooling" [Valenzuela, 1997, 1998, 1999] is a term that captures this problematic, ubiquitous feature of public schooling for U.S. minorities.) This use of a generic curriculum is frequently further aggravated by culturally and socially distant teachers who teach the exam through traditional, teacher-centered, lecture formats.

Important lessons about culture can be learned from the experiences of one Latina teacher, Ms. Moreno (a pseudonym), who teaches in one of the larger, virtually all-Mexican high schools. Through a rather energetic and lively style of teaching that includes "TAAS Pep Rallies" for her students, a majority of students in her classes every semester are able to pass the test.

When asked why they were able to learn from Ms. Moreno, students often referred to her use of "cariño" (or "affection") in the classroom. They also said that she taught "the Mexican way," meaning that she used a lot of Spanish in the classroom and welcomed a high degree of interaction. An excerpt from an LEP student’s written commentary about her heroic teacher appears below (in her exact words):

One of the heroes I know is Ms. ____, she is a person that cares about people. I have her TAAS class and she is teachers in a way that everybody can understand her because, she talks in Spanish, and English. I think she is a strong person. Because she spent many hours teaching her students in the mornings and afternoons, she is always telling us that we can come to her room during our lunch period that she is going to be ready to teaches us. I come during my lunch period and she teaches me every problem I don’t understand, and I feel very comfortable with her because she treats us like friends not like a student. (Written assignment for an English class by a senior female student at Seguín dated September 14, 1994)

Ms. Moreno was so committed to helping students pass the test that she, of her own accord and without additional pay, held classes five days a week at seven o’clock in the morning, to help students pass the TAAS. As one can see from her student’s letter, her lunch hours were also dedicated to helping children. Through her commitment, cultural sensitivity, and her Mexican brand of caring, she was able to help more children pass the test as a single individual than most of her colleagues combined. She embodies the ethos that Ladson-Billings (1994) identifies as central to culturally relevant pedagogy for African American youth. Motivated by a sense of shared fate and shared cultural assumptions, effective teachers of African American children see their role as one of "giving back to the community." Ms. Moreno was no different and just as effective.

Ms. Moreno attributed her success—and in contrast, the majority of her colleagues’ lack of success—to the following: test anxiety and teachers’ inability to teach to "this population." To prepare her students, she used typical English as a Second Language (ESL) methodologies, even if her students were not in the ESL program (i.e., were Mexican American). She lamented her colleagues’ penchant for teaching TAAS Math through standard, teacher-centered lecture formats. She viewed manipulatives as particularly essential, because they help students visualize relationships and therefore make sense of, and retain, information. While she was a firm believer that multi-sensory approaches work best for underprivileged youth, Ms. Moreno’s capacity to help her students rise to her expectations were also clearly mediated by culturally-relevant pedagogy.

The research literature is clear that the learning of abstractions which have little connection to children’s lives and cultures, or which present a monocultural, technical view of knowledge, yield little in long-term learning (e.g., Ladson-Billings, 1994; Valenzuela, 1999). It is unfortunate and of great consequence that problems of abstractions and technical, fragmented sorts of facts and formulas associated with the TAAS combine with an historic inability of non-minority, middle-class teachers to promote widespread academic success with minority, low-income youth in segregated settings (DeVillar et al., 1994). This Latina teacher’s example strongly suggests how real academic achievement among underprivileged youth can be enormously improved through cultural awareness, sensitivity, and a commitment to social justice that no commercial test-prep materials can possibly package. Attracting and supporting those teachers who are knowledgeable about their subjects, about children’s learning, and also about children’s cultures is a national need. It is one that will be difficult to fill when teachers are asked to bracket off their concern for children’s whole development in order to process them through testing systems which undermine, rather than foster, these relationships.

The TAAS exit test is particularly inappropriate for Limited English Proficient (LEP) students.

The TAAS system of testing has many negative effects on teaching, learning, and curriculum across the state. And, as we have discussed, many of the most harmful effects fall on those students whose education has been historically under-resourced and whose academic opportunities have, by race and economic circumstance, been severely limited.

Even given these general patterns, there is the need to focus more closely on the most rapidly growing student population in Texas: the children who, with their families, are recent immigrants. English is not the language of their family.

Harris County, Texas helps to put this issue into perspective. The county has around 3,000,000 people. The latest census data show that there are 875,000 Latinos in the county. While many of these are descendants of original Mexican families in this region, and others are second- or third- (or more) generation citizens, many others are recent arrivals. The schools call them "immediate immigrants" -- children who have arrived within the past three months. Several suburban counties in Texas show an increase of more than 80% in their Hispanic populations within the past five years.

The presence of these children, and their need for a quality education, is no longer an issue at the margins – either in Texas or in many other states. How they are taught, how they learn, how their educational progress is evaluated are of central importance to these children, to their families, and to their communities. The failure of the TAAS system of testing to encompass a model of education which is appropriate to these children is telling not just because of problems the testing system is generating for these children. It is also important because it brings to light the ways the test, and the testing system, slight the relationship between children’s learning and who they are.

The TAAS exit test results in a gap in student achievement that may be directly attributed to the test and not to these students’ abilities. As newcomers at the high school level, non-fluent speakers of English are typically placed in the English as a Second Language (ESL) curriculum (or "the ESL ghetto," as one scholar refers to it [Valdés, 1998]; Valenzuela [1999] refers to this placement practice in her own work as "cultural tracking"). In the ESL track, students spend much of their day in courses that focus on English vocabulary and English reading without regard to their prior academic training. Their remaining subject matter courses are characteristically remedial and rarely, if ever, honors or college-bound. Rather than inferring that the immigrants’ much needed – though often deficient – school administered language support systems should be removed, it is necessary to focus attention on several key consequences of "cultural tracking." These relate directly to LEP students’ test performance.

If lucky, LEP students are placed in ESL subject matter courses like "ESL-Math" and "ESL-Biology," which approximate courses offered in the regular, mainstream, English-only track. Teachers offering these courses tend to be sensitive to issues that face language learners, particularly the sheer amount of time – between 5 and 7 years -- that it takes a young adult to gain native fluency in a second language. Consequently, teachers offer these subject matter courses to keep students from falling behind academically while they learn English. As one Seguín ESL teacher phrased it, "It’s unfair to put their academics on hold while they (students) take time out to learn the language." Another motivation for teaching these courses is the ESL teachers’ recognition of ample academic talent among their immigrant population. These teachers seek to capitalize on and nurture immigrant students’ talents in and through ESL subject matter courses.

Theory would predict the prevalence of academic talent among many ESL youth, especially those schooled in their own country for many years prior to entering U.S. schools (Cummins, 1984; Skutnabb-Kangas & Cummins, 1988). That is, the more a student is schooled in one’s first language, the greater their conceptual grasp of academic subject matter (e.g., punctuation, how to summarize, arrive at conclusions, write papers, etc.). The greater their grasp of academic subject matter, the easier it is to transfer this knowledge to the second language.

Unfortunately, the more common situation in high schools -- even those throughout the Southwestern United States -- is a scarcity of ESL subject matter courses either because of a lack resources to offer such courses or because of philosophical opposition or both (Olsen, 1997; Valdés, 1998). This results in ESL-remedial course placements, which in turn stall students’ learning of the mainstream curriculum. This process masks how the intellectual abilities and potential of so many talented immigrant students get compromised.

To appreciate and understand the magnitude of this waste in talent, one has to consider the characteristics of the immigrant population. Research shows that Mexican immigrants are a select group with average education levels that are higher than the national average for Mexico. They are risk-takers able to delay gratification by accumulating capital to effectuate their passage across the border (for reviews, see Buriel, 1987, 1994). Studies, including Valenzuela’s (1999), further show immigrants and their children possess a progressive orientation toward schools and U.S. society, generally. Moreover, since only 15 percent of the middle-school-aged population attend secondary school in Mexico, to come across a critical mass of such students at the high school level is to encounter a truly "elite" crowd that their ESL and Spanish language teachers readily recognize.

What must also be considered is that Mexico has a challenging national curriculum which is publicly subsidized and thus accessible to most through the sixth year. By the fourth grade, students know the anatomy of the human body. By the sixth grade, students take the equivalent of ninth-grade level geometry in the U.S.

Unfortunately, most school personnel are not sensitive to these kinds of possibilities—either because they cannot read a transcript from Mexico or because they simply assume that a good education is not possible there. As a result, youth get systematically channeled into the "ESL ghetto." Even in instances when they are allowed to enjoy subject matter courses, immigrant youth are systematically denied the opportunity to achieve at an advanced academic level since none of these courses are offered at an honors’ level. This structure helped Valenzuela to understand immigrant youth who said in interviews that they used to know math, or who further reported that they used to be smarter. In a word, many were "de-skilled" as a result of having been schooled in the U.S.7

Even if one considers that some amount of de-skilling is occurring, the quantitative part of the Seguín High School study arrived at three significant and interrelated findings. First, within the regular, non-honors, track, immigrant youth outperform their U.S.-born counterparts. In multivariate analysis, these differences were consistently statistically significant, suggesting that immigrant youth have an edge, academically speaking. A second finding from survey data corroborated the ethnographic evidence that immigrant youth experience school significantly more positively than their U.S.-born peers. That is, they see teachers as more caring and accessible than their U.S.-born counterparts, and they rate the school climate in more positive terms as well. They are also much less likely to evade school rules and policies. These students’ attitudes contrast markedly with that of their second- and third-generation counterparts whose responses in turn are not significantly different from one another. Particularly striking is how generational status—and not gender or curriculum track placement—influences orientations toward schooling. These findings concur with research from numerous other large- and small-scale studies (e.g., Matute-Bianchi, 1991; Kao and Tienda, 1995). Third, the level of schooling youth attain in Mexico or Latin America and students’ grades are significantly correlated. That is, for each year of schooling attained in Mexico or Latin America, achievement goes up, even after controlling for the quality of education they received (based on a subjective measure of school quality).

These three pieces of evidence together suggest that immigrant youth should be passing the TAAS test. Since their chances of passing the TAAS test are lower than their English-dominant counterparts, it is logical to assume that their poor passing rates on the TAAS test suggest more their difficulties with the English language nature of the test, than their potential to achieve academically at a high level. Their teachers frequently refer to this barrier as a chief explanation. In short, children of limited English proficiency are especially handicapped in their ability to exhibit their knowledge by the TAAS exit test.

In conclusion, the TAAS is a ticket to nowhere

. It is harmful to instruction by its rigid format, its artificial treatment of subject matter, its embodiment of discredited learning theories, its ignoring of children’s cultures and languages, and its emphasis on the accounting of prescribed learning. The test itself, and the system of testing and test preparation, have in poor and minority schools come to usurp instructional resources and supplant the opportunity for high quality, meaningful learning.

This system of testing is therefore not the benign "reform" its political advocates claim. Nor is it the remedy for a malfunctioning bureaucratic system that is merely in need of stricter internal management and accountability. The TAAS system of testing exerts a direct, negative impact on the curriculum, creating new problems outlined herein and exacerbating old ones related to historic inequities between rich/majority and poor/minority children. In addition, it masks the real problems of inequity that underlie the failure to adequately educate children. By shifting funds, public attention, and scarce organizational and budgetary resources away from schools and into the coffers of the testing industry vendors, the futures of poor and minority children and the schools they attend are being compromised.

For the children, successful performance on the TAAS in no way ensures either a quality education or a promising future. An education aimed at TAAS scores does unequivocally reduce children’s chances for a real education. The pressure to raise scores is greatest in our poorest, historically least-well-funded schools. To raise scores in those schools -- absent a major investment in teacher knowledge, school facilities, and instructional materials -- educators are diverting time, energy and dollars away from the kind of instruction available in middle class schools and into materials whose only purpose is to raise TAAS scores.

We conclude with an episode from Seguín High School that poignantly captures the human side of this misdirected and injurious policy:

I attended Seguín’s high school’s graduation ceremony. In the middle of the ceremony after the class song was played, about eight students stood up to chant the words scrawled on a large banner they held in their hands: "14 YEARS OF SCHOOL. MADE IT THIS FAR. WHY CAN’T WE WALK?" After the students chanted these phrases several times, three cops and six ushers approached the crowd to take away their banners. The audience booed the cops, including all or most of the graduates sitting in their seats. The hundreds of boos, which included parents’, brought the ceremony to a halt. Some students were escorted out of the audience by the police while others left on their own. I could clearly see how this state-level policy of linking the TAAS test to high school graduation was sensed by everyone as unjust. It was only too fitting to see how this policy was "policed" in a final show of force to the would-be high school grads. (Valenzuela’s June 5, 1996 field notes)

Rather than youth failing schools, schools are failing our minority youth through the TAAS system of testing. In short, we fail to see how the state’s interest is served by a policy that simultaneously diminishes young people’s access to a substantive education and closes off their opportunity for a high school graduation, especially when this route represents their best hope for a socially productive life.

There is a critical need for additional independent research that examines the effects of the TAAS system of testing on the curriculum in various school subjects, on children’s capacities to learn and their sense of themselves as learners, on teachers’ work, and on teacher exit (especially highly educated, highly qualified, and effective teachers) from public schools where TAAS is aligned with administrative bonuses and "performance contracts." Virtually none of the effects on teaching, curriculum, and children which we summarize here are captured by analyses and re-analyses of individual or aggregate test score data.

In addition, there is a need for studies that do not rely merely on officially reported data on test scores and on tested students. There has been much analysis of the test score numbers, including analyses disaggregated by race. However, such studies have primarily relied on numbers provided by the state education agency and/or school districts. Furthermore, such studies are often carried out primarily by analysts employed by, or on contract to, the state or employed by organizations with continuing state contracts for such studies or for "TAAS implementation," TAAS consulting, or "teacher/administrative training" based on TAAS.

There is also a need for public discourse surrounding the many ways to assess children’s learning and evaluate their academic progress. "Alternative assessments" in most subject areas are not "alternative" except in the policy context. Through such professional organizations as the National Council of Teacher of English, the National Council of Teachers of Mathematics, the National Writing Project (and many regional writing projects), the Coalition of Essential Schools, and others there is a broad and deep expertise on ways of understanding how and whether children are learning. Authentic means of documenting children’s learning do not derive from management systems, nor are they intended to be tools of management. They are "authentic" to the extent that they are premised on a rich and complex view of curriculum subjects, on children’s intellectual engagement with learning, and on an understanding of learning as developmental. They are "authentic" if they are able to encompass the relational part of teaching and learning. Ideally they can encompass learning that honors the imagination and the open-ended possibilities that substantive learning supports.

Authentic forms of assessment are ways of sharing information among children, their teachers, and their parents. They should be means by which adults can learn more about how to effectively teach not in a generic sense, but how to better teach "this child, these children."

There is at present an enormous gap in vocabulary and in ideology between those who teach and those who set policy. This gap must be addressed if the system is ever to be corrected; that is, if what is known about teaching and learning are ever going to shape large educational systems rather than be compromised by them. Ways of governing and managing schools, even big school systems, should not depend on forms of assessing children that undermine the very learning schools are intended to foster.



(Message over 64 KB, truncated)