One of the ways that I can accustom myself to inconvenient phenomena is to imagine that I will stand trial for ethnographic malpractice. An attorney has brought a claim against me on behalf of my study’s readers. The trial will be held at a courtroom near the site of study, and witnesses who know about my subject will be called. The important thing about these witnesses is that they will be the ones I most fear hearing from because what they know is least convenient for the impressions I have given the reader.
Mitchell Duneier, “How Not To Lie With Ethnography,” Sociological Methodology (2011)
I have an article in the Chronicle of Higher Education on Alice Goffman’s book On the Run.
It’s both 10,000 words long and subscriber-only, so here’s a quick summary:
(1) On the Run is full of improbable stories of various kinds. In the article I looked into several (there were many others that I didn’t have room to explore), and was unable to confirm any of them.
(2) Assuming the events she described actually happened as she described them, most of the stories I investigated could have been confirmed, in least in part, with a little cooperation from Goffman herself. She declined to provide almost any, citing confidentiality concerns. (She ended up answering one question about one issue, and the implausibility of her answer only brought her veracity into further doubt).
(3) Goffman is abusing the concept of subject confidentiality in order to keep her work from being fact-checked. I determined the identities of most of her primary subjects, as well as the location of what the book calls the “6th Street” neighborhood, without much difficulty. It was made clear to her that neither I nor the CHE had any interest in compromising the anonymity of her subjects, even though we were under no ethical, let alone legal, obligation to protect them from her failure to do so. It was also made clear to her that she could be interviewed on background, in a way that would preserve subject confidentiality completely, and which, if even a couple of the stories investigated in the article then checked out, would have led to the article not being published. Again, she refused.
(4) Nobody at Princeton, which gave her a doctorate on the basis of the research that eventually became the book, or the University of Chicago Press, which published the hardback version, or the American Sociological Review, which published Goffman’s article featuring much of the book’s quantitative data, actually checked any of Goffman’s purported research, beyond confirming that she did hang out on 6th Street for a time, and did know her research subjects. (A similar level of investigation led Jesse Singal to conclude a couple of months ago that On the Run is “almost entirely true.”).
(5) This spring, the University of Wisconsin, where Goffman is on the faculty, conducted an investigation, which led to a public statement that concerns about research misconduct on Goffman’s part were “without merit.” The documents generated by this investigation are subject to the state’s open records law. I filed a request for those documents two months ago. As of today the university has failed to comply. (According to regulatory guidance from the Wisconsin attorney general’s office, a request of this type should be complied with within ten business days).
(6) All of the above raises questions about how social science work in general, and ethnographic work in particular, is evaluated and rewarded. From the article:
In May 2015, the academic world was rocked by news that a paper published in Science appeared to have been based on a fake study. The paper was co-authored by Donald Green, a prominent political scientist at Columbia, but it was actually the work of Michael LaCour, a graduate student at UCLA. The paper reported that a single brief conversation between people who had a stake in the issue and those they were interviewing could lead to significant changes in attitudes toward gay marriage.
The most striking aspect of the LaCour scandal is that, at no point in the submission, review, and publication process did anyone — including Green, the paper’s reviewers, and the editors of Science — have any basis other than, apparently, an implicit faith in that process for their belief that LaCour’s data were genuine.
The Duke University sociologist Kieran Healy had this reaction:
Science is often bitterly competitive but it depends on honesty. It is not set up to weed out liars. Imagine what research, or talks, or conferences would be like if you had to routinely question not simply the quality or competence but the actual honesty of speakers. The same goes for supervision. Consider having to check not just the quality of your grad students’ work, but whether they were lying to you about their data. Much of what we do would become simply impossible.
To which a skeptic might reply: If science is bitterly competitive, and it isn’t set up to catch liars, and there are great rewards for liars who don’t get caught, then one doesn’t need a Ph.D. in social science to realize that this system will produce a whole lot of lying, and that a lot of that lying won’t ever be discovered.
It’s not yet clear that Goffman engaged in the sort of wholesale fabrication that LaCour committed, but it’s also far from clear that she didn’t. And that fact points to a problem that goes far beyond Alice Goffman and On the Run.