Month: March 2012

What’s a teacher to do?

New York City found out on February 28 that an elementary school teacher I know well rates a “34 (7-73) 32 (5-84)” in Reading and a “63 (41-82) 77 (42-91)” in Math.

A few months ago, teachers had received from their school summary documents that looked like this:
Math result for a NYC teacher
What is an individual teacher to do about any of this?  What, on a day to day basis should a teacher do to “improve” on a 34 and maintain a 77?

These numbers are somewhat related to the wonderful awful formula:

NYC value added model for teachers

What are the values of these variables for any particular teacher? Which of these variables are under an individual’s control?  On what day of the year?

I venture that neither common sense, habituation into any cultural world, guesswork, or any other process proceeding from the individual teacher as teacher or person, is likely to help in answering these questions.  I suspect that a whole new class of professional consultants is now being inducted into fuller and fuller participation in new polities in all sorts of institutions.  They will be sold as the interpreters of the ratings.  They will also be people with children and mortgages who will have ever more interest in keeping the formula opaque.  They will be joined by the psycho/socio-metricians tinkering with the formula to “improve” it so that they can report to the New York Post that “the complaints of the teachers have been addressed,” various software engineers, etc.  And the web keeping everybody in place will get tighter and more difficult to escape.

The question we need to raise is, of course, whether teachers should have to ask questions about manipulating variables on a formula.  The formula may be wonderful as a research tool, but it is awful as a method for hiding political decisions and making it appear that these decisions are removed from precisely political activity at all levels of schooling.  As a political tool it may be intended to take the place of a terminating Skynet where evaluation, like the response to some foreign threat.  Evaluation, it appears, is taken out of the political realm of principals meeting teachers in a school, and into the realm of automatized mechanisms noone quite understand but are un-impeachable, as well as altogether unaccountable.  That people will be hurt people is their problem will the newly powerful say: “good” teachers (the top 50%? 75%? 25%?) “have nothing to fear” and “bad” teachers should fear dismissal (unless the whole exercise is pointless).

Whether any of this will do anything to improve education in any of its senses in the question may be a question one asks at one’s peril.

On Political Deep Play – a coda on experimentation

My entry from March 2 played, very seriously, with the kind of deep play policy makers in the world of schooling engaged in when they released invalid scores purported to tell how well individual teachers taught.  On March 9th, James B. Stewart of the New York Times, asked “Would Americans be better off if General Motors and Chrysler had simply gone bankrupt, without benefit of taxpayer assistance?” and he raised the question of the kind of evidence one could use to answer such a question.  What picked my curiosity is the following comments:

Unlike a science experiment, in which variables can be changed and the experiment repeated, we can’t turn back the clock, let the auto companies go bankrupt and compare the results with what we have today, which is an American auto industry that is, by nearly all measures, healthier than it’s been in many years. G.M. and Chrysler, not to mention Ford, which didn’t get taxpayer money but benefited indirectly, are profitable, hiring more workers, competing more effectively, gaining market share and building better cars and trucks.

He then proceeded to make comparisons with other companies that were, or not, helped by the government when they face bankruptcy.  Essentially, he was using history rather than “evidence-based” empirical research to argue in favor of a political decision.

Now, of course, history, like anthropology, is precisely not an experimental science and yet it may more useful to “politic” makers, that is politicians, as actors, rather than “policy” makers as advisors to the actor.  The very small group (Obama, Geithner, ??) who decided to bail out General Motors could not rely on “evidence.”  They had to rely, in the best sense of all these words, their ideology, their common sense, and the conversations they must have had.

In other words, they placed a major bet.  It looks like they won.  But this was about the deepest of deep plays.  The only deeper I can imagine is Roosevelt or Wilson getting America into World Wars.

Back in New York, it is probably the case that Bloomberg made a similar bet when he had the test scores released.  He could not wait for the “evidence” that this release would lead to better teaching.  By the time this evidence was in, then the political problem would probably have been moot.  We can disagree with his decisions.  We can note the irony that people who have prided themselves on being “data-driven” made a major decision in the absence of data.  But we see the decision for what it is, a political decision, not a policy decision.  And as one approaches political decisions, history, and anthropology, may be more useful than “experimental” social sciences.

How can we convince policy/politic makers that evidence-based research is not the way to a better democracy?

MOOC: Education, degrees, careers?

Stanford offered a class, on Machine Learning . 104,000 students registered. 13,000 completed the course. Most of them must have learned something but none got a State recognized certificate or a degree. So, at this point, they completed “for nothing”–that is just for the education of it. Some of them may also perform their professional tasks better. I suspect those will not accept for very long to be passed for promotion by people who have a degree.

I ended a recent paper for the Gordon Commission on the Future of Assessment with comments about the possible disassociation of schooling from assessment.  My argument was that the association between the two has been 1) good for schooling in that it has massively increased its reach and claim on resources, and also 2) terrible as it makes it less and less relevant to education.  The recent, and ongoing, controversy about publicizing teachers’ scores on invalid measures is a case in point: teacher-ing, with ever more symbolic violence (a.k.a “accountability”), is made into a skill for putting measurable bits into students’ head.  That teacher-ing might be about participating in shaping a mind is left aside as not a concern for the State.  But why, may we ask, should the State be concerned with education?  Well, of course, because

the tests
…..that lead to the degrees
……….that accredited  schools grant
……………that employers use to open or close doors to careers

are essential for the representation that a political system is indeed “democratic” and that state rewards are indeed distributed on merit rather than birth privilege (in its racial, ethnic, class, gender, etc. forms).

In many ways, as the people of the School have been saying, focusing on tests leading to degrees is a radical narrowing of what was the mission of schooling.  Arguable, the battle has been lost as much (most?) of what was included in this mission has now been distributed out to the family, the media, religious institutions, etc.  But schooling, as an institution, appeared to remain central because it has kept its monopoly on the granting of degrees.

What if this changed?  What if a successful challenge was mounted to legitimize other doors to adult careers than those controlled by the current schools, colleges and universities (and their teachers or faculty)?

In the past week, the New York Times published evidence that this challenge has started.  One is an opinion piece by Charles Murray asking an “energetic public interest law firm” to challenge “the constitutionality of the [bachelor’s degree] as a job requirement” (March 8, 2012).  The Supreme Court, I did not know, has made it unconstitutional to make test scores the key to employment unless there is a tight link between the test and the job.  Demonstrating the link between almost any college degree and almost any job might be difficult.

The other report may be a more immediate and less ideological challenge, and possibly much more difficult for school people to block (particularly since some of them are profiting from this challenge).  I am talking here about “Massive Open Online Courses” (MOOCs).  Stanford offered a classes for 160,000 students in 190 countries!  Another class, on Machine Learning, was given for 104,000 students with 13,000 completing the course.

The figures are astounding.  They are about something that is happening now and will have social and political consequences.  Of the 13,000 who complete the course on Machine Learning, most of them must have learned something but none got a State recognized certificate or a degree.  So, at this point, they completed the course “for nothing”–that is just for the education of it.  Some of them may also perform their professional tasks better.  I suspect those will not accept for very long to be passed for promotion by people who have a degree.

If the State finds ways to accredit (“give credit for”) the taking of MOOC (perhaps by asking a company like ETS to give an independent and controlled test), the implications for universities and their faculty are staggering.  The New York Times quote one of the Stanford professors who taught one of the MOOC as saying that he does not want to go back to teaching just twenty students in a small classroom.  When Stanford has found a way to charge people for the course, and reward the faculty member in commensurate fashion, the whole economic basis of colleges is transformed.

We may be seeing the end of schooling as we have known it (and for people of my generation profited from it).  It is going to be quite a ride.

Value Added Deep Play

The publishing of individual teachers scores by New York City is a research boon as it allows us to test various analytic methods that will allow for understanding more systematically the networks of authority and power in which we are all caught, and particularly the relationship of motivation to act to social consequences. In this effort we might do worse than facing the extent to which social life is, also, (deep) play, and then making suggestions to policy makers about what is most probably a fundamental feature of humanity

Geertz’s “Deep play: Notes on the Balinese cockfight” (1972) is justly famous and yet its central argument, that the Balinese, sometimes, play “deeply’ in such a way as possibly to endanger their status, has had surprisingly little impact.  Maybe it is because it is set in a mysterious island far, far away.  Maybe it is because “we” do not bet on cockfights.  Maybe it is because, though quite a few of “us” play more or less deeply in Las Vegas and other such venues, “our” institutions, dominant ideologies, etc., do not play much and frown on any play dangerous enough to risk our health or well-being.  Quite the reverse indeed as “we” pride ourselves on ever more detailed protective regulation of what must be done while “at play” (in play grounds, riding cycles, etc.).  But, of course, we, personally and as implicated by more or less enlightened leaders do “deep play”–though not necessarily in ways so labeled.

Sherry Ortner is one of the few who worked at expanding what Geertz must have meant.  She presented a study (1999) of Sherpas helping people climb the Everest and other Himalayan mountains as a study of, precisely, deep play given the well-known dangers involved in such endeavors.  But Sherpas, and the rich people who hire them to make it possible for them to reach the summit, may still be dismissed as exotic people living at the edge of our safety nets and of only passing interest.  And yet we, social scientists, should pay attention.

A few years ago, I got interested in deep play as something potentially central to culture theory in anthropology.  I was working with Mary Cotter analyzing a moment in a woman’s hospital labor when she tells as a joke about an earlier labor when she, a physician herself, had lied to an anesthesiologist about pain she was not feeling to get more pain killers, thereby endangering her and her child (Cotter 1996; Varenne and Cotter 2007).  Everyone present, anesthesiologist, husband, nurse, researcher, laughed.  The joke was certainly “play” in the usual sense.  Was her earlier lie also a gamble, deep play?

In a forthcoming paper in the Educational Researcher, Jill Koyama and I propose we answer this question positively and expand the relevance of “(deep) play” to include all acts, by an individual or a polity through its leaders, that threaten their status or the status of some of the people the act involves.  In our perspective, it does not matter what the motivation of the actor (gambler) may be, or the type of rewards the act might produce, or its initial cost.  What would matter is the level of uncertainty about the act producing what it is supposed to produce and the severity of the effects, should the act not produce what was hoped for.  It has made sense to think about the activity of traders of Wall Street as, also, gambling.  What about the activity of politicians and other regulators when act to, in what would be their term most probably, “generate reform of our educational system in fundamental ways” (from the introductory paragraph of the report A Nation at Risk, National Commission on Excellence in Education 1983)?

In this perspective, the decision by the New York City Department of Education to publish the names of individual teachers and their score, is a prime example of very deep play by policy makers and the politicians who accepted their recommendations.  Who made the decision is actually unclear, but it was forcefully defended by Mayor Bloomberg.  There is no evidence, “data based” or otherwise, that such a publication will achieve what reformers wish to achieve.  There is much debate about the validity of the figures.  But, somewhere, a decision in uncertainty was taken.  It involved ten of thousands who, willy nilly, are caught in its consequences and for whom the act might transform status in what Garfinkel once referred to, chillingly, as “successful degradation ceremonies” (1956).

And now, unsurprisingly, a slow motion drama is unfolding.  This drama, like the Balinese cockfight, is taking place in the public square, and so will many of the status changes that will result as the main bet, as well as many of the side bets that are also being made, is followed by its consequences.  Political careers will be made and lost.  Administrators will be fired while others will move up.  Unions might be reinvigorated, or not.  One can hope that the shame as much as half of the teachers of New York City are experiencing this month as they are publicly labeled “below average” will be short lived (though one can imagine the psychological scars some will bear as journalists and parents berate them).

It may seem insensitive for social scientists to watch the unfolding of the drama as evidence for refining our understanding of social processes.  And yet it is our duty to do such.  Publishing individual scores, whatever good might come of it, is, also a full scale experiment which would probably never have been allowed to proceed if it had been proposed as “research.”  University review boards are very vigilant about the extent to which individuals might be harmed by research, even unwittingly or in indirect ways.  But, of course, policy acts are not systematically reviewed for the harm they might cause.  Perhaps they could not be or political decisions could not be taken in a timely fashion.  But the issue remains and as social scientists who advise policy makers, we must pay attention and play our analytic role.  The publishing of the scores is a research boon as it allows us to test various analytic methods (as Koyama and I are doing) that will allow for understanding more systematically the networks of authority and power in which we are all caught, and particularly the relationship of motivation to act to social consequences.  In this effort we might do worse than facing the extent to which social life is, also, (deep) play, and then making suggestions to policy makers about what is most probably a fundamental feature of humanity.