My Photo
Buy Your Copy Now!
Blog powered by Typepad
Member since 12/2003

« Would a Real Blogger? | Main | THRASS Learning »

Thursday, August 10, 2006

Comments

Ed Darrell

Another strategy I found useful over the past three years in a suburban Dallas district's alternative school was to treat the students as if they were there to learn the good stuff, and repeat it as often as they needed it in order to get it, in different forms.

Strangely enough, when we concentrated on the material, other discipline problems dropped to almost nothing for the vast majority of the students.

Our state test scores went to first among the district's high school programs in my area.

JohnL

I'm still cautious about KIPP schools. I wrote an entry about how little the five pillars address instructional factors. If KIPP schools are consistently successful by objective standards, they're worth examining and, perhaps, emulating. Absent such evidence, whatever we can say about KIPP schools is simply language, words, opinions,argument...and people can say anything.

TMAO

A huge difference between KIPP schools and reform-focused schools like Mathson: KIPP and schools like them can control the type of student that enrolls as well as removing (their phrase is counsel-out) students who do not respond to the model. True public schools, charged with educating everyone, cannot. I don't begrudge any student the opportunity or the reality of receiving a quality education, but I struggle with a continued inequitable distribution of resources, and the celebration of a model that cannot possible reach all students.

Peter Campbell

There are two issues that need to be addressed: (1) does KIPP do what it claims to do? i.e., raise achievement for low-income minority children and (2) if so, what are the implications of KIPP "working," i.e., the consequences of KIPP being successful at what it claims to do.

I will dissect each of these issues.

1 - Does KIPP do what it claims to do? i.e., raise achievement for low-income minority children?

To begin with, we have to determine what we mean by "achievement." KIPP defines achievement in the following two ways: (1) increased test scores over time and (2) high rates of enrollment at prestigious high schools. Overall, KIPP points to the following evidence of achievemen: (1) students who enter KIPP in the 5th grade and finish in the 8th grade have higher test scores when they graduate and (2) a large percentage of students that leave KIPP in the 8th grade go on to prestigious high schools. Therefore, KIPP is "successful" because it has raised the achievement of low-income minority children.

This sounds great. From this track record of success, it seems like we can infer that KIPP can take any low-income minority child, raise his/her test scores, and send him/her on to a great high school, presumably on to college, and then presumably on to a great life. If this were true, I'd be the first person in line to lead the celebratory KIPP parade.

Unfortunately, this is not true. It is, in fact, too good to be true.

a) A recent SRI report on Bay Area KIPP schools - http://www.sri.com/policy/cep/pubs/choice/KIPPYear1Report.pdf - shows that at least some KIPP students have already been succeeding at other schools prior to entering a KIPP school. In addition, a large part of KIPP's success is based on the fact that KIPP students have motivated parents who push them in ways that other underprivileged kids don't. Given these two factors -- motivated parents and already successful students -- how much credit can we reasonably ascribe to KIPP?

b) KIPP students spend approximately 70% more time at school than their regular public school peers. So how much of KIPP's success is simply due to the fact that children spend exponentially more time there?

c) KIPP schools claim they are responsible for dramatic leaps in achievement. According to its FAQ (http://www.kipp.org/01/kippfaq.cfm#04),

"The average KIPP student who has been in a KIPP school for three years starts fifth grade at the 44th percentile in math, the 34th percentile in reading, and the 44th percentile in language, as measured by nationally norm-referenced exams. After three years in a KIPP school, these same students are performing at the 83rd percentile in math, the 58th percentile in reading, and the 67th percentile in language."

The obvious problem with averaging anything is that the average often does not depict the typical outcome. If there is one outcome that is very far from the rest of the data, then the average will be strongly affected by this outcome. In short, some really high achievers will make the others look pretty good, even if these others are not doing so well.

For example, let's say there were 5 students taking a test. The scores (out of 100 possible points) were as follows: 45, 47, 52, 98, 99. The average of these five scores is 68.2 So you could truthfully and accurately say, "Student scores were near the 70th percentile." But how many students scored a 70? None. If you look at the scores, 3 out of the 5 did really badly. But 2 of the 5 did really, really well. The result? It looks like great things are happening when, in fact, they are not.

To my knowledge, no statistical analysis of KIPP achievement data exists, an analysis that determines the variance and standard deviation of these data. The variance and standard deviation describe how spread out the data is. If the data all lie close to the average, then the standard deviation will be small, while if the data are spread out over a large range of values, it will be large. Having outliers -- very high achievers that make the KIPP scores seem better than they really are -- will increase the standard deviation. If the standard deviation is small, then the claim about "leap in achievement" would be quite substantive. But if the standard deviation is large, then the claim about "leap in achievement" would be pretty sketchy. In fact, it would be misleading at best.

d) There is strong evidence that kids that graduate from KIPP in 8th grade are not the same kids that began KIPP in 5th grade. If these were the same kids, then the numbers might indeed be impressive. But how many of the 5th graders dropped out after being at KIPP for a year or two? It's entirely possible that a large number of the original 5th graders dropped out, leaving only the higher-performing students, resulting in fewer 8th graders. With fewer 8th graders, it's easier to report higher graduation rates and higher college enrollment rates. It also makes the achievement data look better than it might actually be. The enrollment data I obtained through the California Department of Education shows the same cohort of students enrolled in CA KIPP schools over several years. What's most noticeable to me about the data is the enrollment numbers for African-American boys. For example, KIPP Bridge College Preparatory in Oakland had 35 AA boys in 5th grade, but only 8 remained by the time they reached the 8th grade. What happened to these children? Also, how many of these 8th graders were high-performing students who were attracted or recruited from outside KIPP schools and encouraged to enroll at KIPP? According to the SRI report mentioned above,

"Given increasing interest in KIPP schools on the part of parents and students, some principals expressed concern about “creaming” already high-performing students from local schools when there remains a large number who are low-performing and underserved. One principal expressed dismay with the school’s struggle to enroll Title I students, whom she considers to be her target population."

e) KIPP's FAQ (link above) claims that KIPP has a broad-based curriculum and does not shirk on subjects that are not tested under NCLB, e.g., social studies. According to the SRI report on Bay Area KIPP schools, "Students have 90 minutes of (English Language Arts) and math every day. They also have 90 minutes of social studies and science on alternating days." (p. 33) The report does not indicate what is actually taught in the social studies and science blocks, nor how it is taught. Moreover, there is no evidence in this report that KIPP students receive a "broad-based education" other than the fact that science and social studies are taught for 90 minutes every other day. Because students are not tested in these subjects, we have no way of knowing if they are learning anything. More troubling, we have no way of knowing if the instruction they receive is substantive or superficial other than anecdotes from KIPP teachers themselves.

f) KIPP can never scale enough to raise the achievement of a large enough number of children to justify the amount of praise and attention it receives, notwithstanding all of the questions and issues I raise above. KIPP relies on energetic idealists in their 20's who are single and have no kids to work 10 hour days, an extra day on Saturday, and an extra month in the summer. There are only so many people who are willing to do this. There are even fewer who can do this because of their family commitments. They have to go home, fix dinner, do the dishes, walk the dog, and help with their kids' homework. Certainly some kids can pull themselves up out of the inner-city despite the tremendous odds. Certainly some great schools have formed and will continue to form in poor neighborhoods and attract motivated teachers, students, and parents to work together to improve the educational outcomes of poor kids. KIPP is a good example of this. But the dozens of examples of personal success pale in comparison to the hundreds of thousands of personal failures. The 40 or so KIPP schools make up a tiny fraction of the thousands and thousands of schools where children are ground up and spat out. So why do so many poor kids fail? Why are so many poor children chewed up and spat out?

In sum, KIPP does not raise achievement for low-income minority children. There is evidence that the average test scores of children who stay enrolled at KIPP until 8th grade do go up. But there is no strong evidence that suggests that KIPP CAUSES this increase in test scores. At best, there is a CORRELATION between children who stay enrolled at KIPP until 8th grade and higher test scores and a CORRELATION between children who stay enrolled at KIPP until 8th grade and enrollment in prestigious high schools.

Finally, it should be noted that there is no longitudinal data at all that looks at the long-term success of KIPP graduates in high school, college, and beyond. The statement that KIPP prepares students for college and life after college does not have a shred of evidence to back it up.

---

2 - What are the implications of KIPP "working," i.e., the consequences of KIPP being successful at what it claims to do?

KIPP schools are made up almost entirely of black or Hispanic students. KIPP's success undergirds the recent law passed by the Nebraska legislature, allowing for segregated schools in Omaha. In other words, looking at KIPP as an example, the argument could be made that while segregated schools might seem bad, they actually "work." Of course, what they work at doing is the question.

Why should this matter to us? After all, proponents of KIPP argue, "these children" need the basics in fifth grade because NO ONE TAUGHT THEM WELL ENOUGH BEFORE! If the school system they were in beforehand hadn't been so screwed up and awful, they could start where they're supposed to, with fifth grade material. And maybe they wouldn't all have to spend 70% more time in school just to catch up.

But let's not forget Brown v. Board, the Supreme Court decision that was supposed to end segregated schools in America:

"Segregation of white and Negro children in the public schools of a State solely on the basis of race, pursuant to state laws permitting or requiring such segregation, denies to Negro children the equal protection of the laws guaranteed by the Fourteenth Amendment -- even though the physical facilities and other "tangible" factors of white and Negro schools may be equal."

Think of it. The Court said that segregation denies to low-income minority children the equal protection of the laws guaranteed by the Fourteenth Amendment EVEN THOUGH the facilities of these segregated schools may be equal. So if we look at KIPP from the perspective of the Brown decision, we can see that KIPP denies to low-income minority children the equal protection of the laws guaranteed by the Fourteenth Amendment EVEN IF these segregated schools raise their achievement. So EVEN IF KIPP lived up to all its praise -- which I have shown it does not -- it would still be unconstitutional. As the Court argued in the decision,

"Segregation of white and colored children in public schools has a detrimental effect upon the colored children. The impact is greater when it has the sanction of the law, for the policy of separating the races is usually interpreted as denoting the inferiority of the negro group. A sense of inferiority affects the motivation of a child to learn. Segregation with the sanction of law, therefore, has a tendency to [retard] the educational and mental development of negro children and to deprive them of some of the benefits they would receive in a racial[ly] integrated school system."

The decision, delivered by Chief Justice Warren, took this view on the importance of public education:

"Today, education is perhaps the most important function of state and local governments. Compulsory school attendance laws and the great expenditures for education both demonstrate our recognition of the importance of education to our democratic society. It is required in the performance of our most basic public responsibilities, even service in the armed forces. It is the very foundation of good citizenship. Today it is a principal instrument in awakening the child to cultural values, in preparing him for later professional training, and in helping him to adjust normally to his environment. In these days, it is doubtful that any child may reasonably be expected to succeed in life if he is denied the opportunity of an education. Such an opportunity, where the state has undertaken to provide it, is a right which must be made available to all on equal terms."

Maybe KIPP is all we can hope for. Maybe we have to throw up our hands and say, "We can't beat racism, so let's join it." Maybe we have to face the facts and say, "Poverty will always be with us, so we just to have to make the best of it." Maybe we have to admit that KIPP is not as great as it claims to be, but -- because it works for some kids -- then that is enough.

Imagine if we said the same thing about world hunger. Maybe we we have to throw up our hands and say, "We can't beat world hunger." Maybe we have to face the facts and say, "Hunger will always be with us, so we just to have to make the best of it." Maybe we have to admit that current solutions are not as great as they claim to be, but -- because not all children end up starving to death -- then that is enough.

Gee. Come to think of it, that IS what we say about world hunger. In fact, it's also what we say about AIDS. It's what we say about homelessness. It's what we say about drug addiction. It's what we say about global poverty. In fact, it's what we say about other people's suffering as a whole. We accept, in full self-fulfilling prophecy mode, that these problems can never be solved. We accept that the best we can do is make something intolerable a little more tolerable. The question is, tolerable for whom?

For all the kids that are not lucky enough to get a place at KIPP, it is not tolerable. For all the kids that do make it into KIPP but are not able to endure the 10-hour days and two hours of homework every night and who eventually drop out or are "counseled out," it is not tolerable. And even for those kids who do make it into KIPP and make it out of KIPP, their "success" is not tolerable because it comes at a price, a price that is too high to pay.

Peter Campbell
Missouri State Coordinator
Assessment Reform Network, The National Center for Fair and Open Testing (FairTest)
http://transformeducation.blogspot.com

The comments to this entry are closed.

Pages

What I'm Tweeting

    follow me on Twitter