FOIA Coordinator by Edward Vielmetti

PACER redaction failures: about 0.01% in a 1.8 million document sample

PACER is a public system for access to electronic court records. Wholesale downloads of files from it, initiated by Carl Malamud in the RECAP project, have provided an insight into systematic ways that failures in the court system for the courts and parties to litigation to improperly redact documents.

A study of redaction failures gives a sense for how widespread is the problem of improper redactions in court documents. The specific failure they are looking at is the "redaction rectangle" failure, where a black box is drawn over text in such a way that the underlying text is still in the document. Quoting from author Timothy B. Lee, emphasis added:

So how many PACER documents have this problem? We're in a good position to study this question because we have a large collection of PACER documents—1.8 million of them when I started my research last year. I wrote software to detect redaction rectangles—it turns out these are relatively easy to recognize based on their color, shape, and the specific commands used to draw them. Out of 1.8 million PACER documents, there were approximately 2000 documents with redaction rectangles. (There were also about 3500 documents that were redacted by replacing text by strings of Xes, I also excluded documents that were redacted by Carl Malamud before he donated them to our archive.)

Next, my software checked to see if these redaction rectangles overlapped with text. My software identified a few hundred documents that appeared to have text under redaction rectangles, and examining them by hand revealed 194 documents with failed redactions. The majority of the documents (about 130) appear be from commercial litigation, in which parties have unsuccessfully attempted to redact trade secrets such as sales figures and confidential product information. Other improperly redacted documents contain sensitive medical information, addresses, and dates of birth. Still others contain the names of witnesses, jurors, plaintiffs, and one minor.

One document in 10,000 seems like a needle in the haystack, but knowing that you can automatically check a sample document and flag it as a possible redaction failure (with a 1 in 10 chance of finding a real failure) makes this a fruitful place for further study. At the very least, there's the opportunity to flag a document before releasing it or to automatically warn PDF creators that their workflow includes a step that shouldn't be there.

For more on this, I recommend the text Public Access to Court Records - Protecting Personal Sensitive Information, delivered as an ABA Webinar on the topic in March 2011.

Northwest Herald: Overzealous redaction from the Upper Illinois River Valley Development Authority

Thanks to the Northwest Herald of McHenry County, Illinois for this shining example of an overly zealous redaction effort, where almost every word is blacked out with what looks like magic marker. It's also notable as an example of bad redaction, in that the magic marker was inexpertly wielded in spots leaving letter ascenders and descenders unprotected, sometimes enough to discern the words below the ink with the naked eye.

The story is FOIA response regarding Lakewood sportsplex filled with redactions, written by Kevin P. Cramer and published December 29, 2010.

SportsPlex FOIA response

Partially unredacted FBI informant file provides link to Ernest Withers, civil rights photographer

An AP story, which the Sydney Morning Herald gave the headline Civil rights photographer was FBI spy, does a rewrite of a story by a newspaper called the Commercial Appeal.

The newspaper did not have access to Withers' informant file because it is sealed. The Justice Department twice denied The Commercial Appeal's requests for that file and won't acknowledge it exists, the newspaper reported.

Instead, the government released 369 pages related to a 1970s public corruption probe that targeted Withers, who pleaded guilty in 1979 to extorting kickbacks from a nightclub owner while he was a Tennessee Alcoholic Beverage Commission agent. Those pages included redacted references to informants, but in one instance the FBI failed to hide a single reference to Withers' informant number, ME 338-R.

As with all AP rewrites, it pays to read the original story. Photographer Ernest Withers doubled as FBI informant to spy on civil rights movement; He provided agency with insider's view of volatile period reads the September 12 story with Marc Perrusquia's byline.

Censors overlooked a single reference to Withers' informant number. That number, in turn, unlocked the secret of the photographer's 1960s political spying when the newspaper located repeated references to the number in other FBI reports released under FOIA 30 years ago. Those reports -- more than 7,000 pages comprising the FBI's files on the 1968 sanitation strike and a 1968-70 probe of the Invaders -- at times pinpoint specific actions by Withers and in other instances show he was one of several informants contributing details.

The link in the Appeal's story goes back to the original document, which the Appeal published. Pages from FOIA reveal Withers as informant shows a three page transmission on form FD-36 (Rev 7-27-76)

The full report is on the Commercial Appeal as Withers Exposed. It uses the Document Cloud system to highlight and sift through records to reconstruct information from linked files released in previous FBI FOIAs.