I am happy with their lack of expertise and hope it stays that way, because I cannot remember a single case where redactions put the citizenry at a better place for it.

Of course if it's in the middle of an investigation it can spoil the investigation, allow criminals to cover their tracks, allow escape.

In such case the document should be vetted by competent and honest officials to judge whether it is timely to release it, or whether suppressing it just ensures that investigation is never concluded, extending a forever renewed cover to the criminals.

Secure systems are not exactly the right environment for quick release and handling. So documents invariably get onto regular desktops with off the shelf software used by untrained personnel.

Since hundreds of people were involved the most likely explanation is incompetence

> strict ethical standards to not use the information (for what little that may be worth)

If it's worth so little to your eyes/comprehension you will have no problem citing a huge count of cases where lawyers do not respect their obligations towards the courts and their clients...

That snide remark is used to discredit a profession in passing, but the reason you won't find a lot of examples of this happening is because the trust clients have to put in lawyers and the legal system in general is what makes it work, and betraying that trust is a literal professional suicide (suspension, disbarment, reputational ruin, and often civil liability) for any lawyer... that's why "strict" doesn't mean anything "little" in this case.

> What happens in a court case when this occurs? Does the receiving party get to review and use the redacted information (assuming it’s not gagged by other means) or do they have to immediately report the error and clean room it?

Typically, two copies of a redacted document are submitted via ECF. One is an unredacted but sealed copy that is visible to the judge and all parties to the case. The other is a redacted copy that is visible to the general public.

So, to answer what I believe to be your question: the opposing party in a case would typically have an unredacted copy regardless of whether information is leaked to the general public via improper redaction, so the issue you raise is moot.

My guess would be that if the benefitting legal party didn't need to declare they also benefitted from this (because they legally can't be caught, etc.) they wouldn't.

I know and am friends with a lot of lawyers. They're pretty ruthless when it comes to this kind of thing.

Legally, I would think both parties get copies of everything. I don't know if that was the case here.

> Edit: after reading up on this it looks like attorneys have strict ethical standards to not use the information (for what little that may be worth), but the Associated Press was a third party who unredacted public court documents in a separate Facebook case.

Curious. I am not a litigator but this is surprising if you found support for it. My gut was that the general obligation to be a zealous advocate for your client would require a litigant to use inadvertently disclosed information unless it was somehow barred by the court. Confidentiality obligations would remain owed to the client, and there might be some tension there but it would be resolvable.

It already seems that they blacked out more than the law allowed, so following neither.

Not that it matters much what the law says if the goal is to protect the man who hands out pardons...

> made their redactions with actual ink, and then re-scanned every page

That's not very competent.

> going analog is foolproof

Absolutely not. There are many way's to f this up. Just the smallest variation in places that have been inked twice will reveal the clear text.

It's like Russian spies being caught in the Netherlands with taxi receipts showing they took a taxi from their Moscow HQ to the airport: corrupt organizations attract/can only hire incompetent people...

https://www.vice.com/en/article/russian-spies-chemical-weapo...

Anyone remember how the Trump I regime had staff who couldn't figure out the lighting in the White House, or mistitled Australia's Prime Minister as President?

I would just do the digital version of that: add 100% black bars then screenshot page by page and probably increase the contrast too.

The bigger difference from my perspective is that they have competent people doing the strategy this time. The last Trump administration failed to use the obvious levers available to accomplish fascism, while this one has been wildly successful on that end. In a few years they will have realigned the whole power dynamic in the country, and unfortunately more and more competent people will choose to work for them in order to receive the benefits of doing so.

> William Barr. Barr is contemptible, yes, but smart AF

You mean the guy who covered up for Epstein's 'suicide' and expected us morons to believe it?

It’s easy to appear competent when you’re sitting on your butt doing nothing. Had exactly did Barr and Co. accomplish in terms of moving forward the agenda people voted for? These guys were so eager to win accolades from liberals they couldn’t even pick the lowest hanging fruit. Totally pathetic effort after the stellar performance by the legal eagles in the Obama administration. Trump 2.0 is pursing a very aggressive legal strategy. It has a bunch of very smart people racking up wins in areas such as funding cuts, education, civil rights, deployment of national guard, etc. It also has people that are… struggling. But, unlike with Trump 1.0, they’re actually trying to move the ball forward for their team.

> but smart AF. When Barr’s DOJ released a redacted version of the Mueller Report, they printed the whole thing, made their redactions with actual ink, and then re-scanned every page to generate a new PDF with absolutely no digital trace of the original PDF file.

This is a dumb way of doing that, exactly what "stupid" people do when their are somewhat aware of the limits of their competence or only as smart as the tech they grew up with. Also, this type of redaction eliminates the possibility to change text length, which is a very common leak when especially for various names/official positions. And it doesn't eliminate the risk of non-redaction since you can't simply search&replace with machine precision, but have to do the manual conversion step to printed position

Or it is just the result of firing people who were competent and giving insufficient training to people who had never done this before.

Somewhat related, I once sent a FOI request to a government agency that decided the most secure way to redact documents was to print them, use a permanent marker, and then scan them. Unfortunately they used dye based markers over laser print, so simply throwing the document into Photoshop and turning up the contrast made it readable.

And that's just in the non-adversarial simple case.

If you don't know the provenance of images you are putting black box on (for example because of a rogue employee intentionally wanting to leak them, or if the image sensor of your target had been compromised to leak some info by another team), your redaction can be rendered ineffective, as some images can be made uncroppable by construction .

(Self-plug : https://github.com/unrealwill/uncroppable )

And also be aware that compression is hiding everywhere : https://en.wikipedia.org/wiki/Compressed_sensing

I was thinking I understand what's going on but then I came to the image showing the diff and I don't understand at all how that diff can unredact anything.

It's actually quite easy to open the pdf and see that there are several different elements per page to the document, eg the main text, an image, the footer, the title.

Randomly removing these by trial and error will usually quite easily allow you to find the watermark and nix it, with the advantage that even a sophisticated recipient will not be able to find out from the pdf file what the watermark was.

Normally, I'd never attribute to intention what can be blamed on incompetence. Especially if the government is doing it. But sure, if I were the intern tasked with this job...

There's also metadata in the image files. What specifically would be sensitive in the pdf with screenshots metadata that is also not present in the sceenshot image metadata?

USA is still very high, so they can go much much lower, but I think they might go to some still lower places, finding them where we didn't even know such places could exist. Some ideas:

- Leave NATO

- Start openly supporting Russia and North Korea

- Arrest whole International Criminal Court

- Preventively invade China

This low https://en.wikipedia.org/wiki/Child_abuse_in_Pakistan aka a society where child abuse is simply accepted and mainstream, with the child abuse of child labour and dhijhadism being just additional nightmare fuel on top.

I'm not too concerned about the US. They've made their bed.

I'm more concerned with them dragging everyone else down, and someone much worse taking their place.

Maybe it was always part of the plan. Plausible Deniability.

The really interesting bit is whether they can go another term.

That seems like a dangerous approach. Though printer drivers do often use rasterization, especially when targeting cheap printers, many printers can render vector graphics and text as well. Print-to-PDF will often use the later approach, unless of course the source program always rasterizes it's output when sending it out to the printer driver, or the used Print-to-PDF driver is particularly stupid.

Me too.

care to share the script?

Yes but don't tell them they're doing it wrong.

This was my initial reaction to this news. I mean think about it

The Trump team knows that nobody is gonna buy whatever they put out as being the full story. Isn't this just the perfect way to make people feel like they got something they weren't supposed to see? They can increase trust in the output without having to increase trust in the source of it

And as far as I've heard there hasn't been anything "unredacted" that's been of any consequence. It all just feels a little too perfect.

"Never ascribe to malice that which is adequately explained by incompetence."

Colleagues whose full time job is doing this sort of thing for various bits of the government have told me this is exactly the case here. People from all over the government have been deputized to redact these documents with little or no prior training.

It seems insane that nobody at the other end runs something as simple as MAT or imagick (twice) over it to take the text layers out before uploading though. I hope this is at least partially intentional.

My understanding is that many people were fired and replaced by loyalists at the FBI. I think there are a lot of incompetent people working there right now.

Yeah — don't attribute to resistance what can adequately be explained by idiocy.

Let people believe it's deliberate sabotage. Unfortunately, in real life, minions of a dictator serve the dictator; they don't risk their live or safety for a noble cause. Any screw-ups are a result of gross incompetence that is typical for every dictatorship.

It’s funny that this effort, the largest exertion of FBI agents second only to 9/11, seems to be unprepared to redact. Cynically, I’m prepared for it to be part of a generative set of PDFs derived from the prompt “create court documents consistent with these 16 PDFs which obscure the role of Donald Trump between 1993 and 1998.”

It seems like a common tactic by this administration is to just not do what they are required to do until they have been told 50 times and criminal charges are being filed. I suspect the actual truth here is 'don't do this' turned into 'you have 1 day to do this and keep my name out of the release' which led to lots of issues. They probably spent more time deciding the order of pages to release, and how to avoid releasing the things damaging to the administration, than actually doing the work needed to release it. Now they will say 'look, see! You didn't give us enough time and our incompetence is the proof'

Considering the Comey, James, and Adams debacles, seems quite likely they're purged most people with a shred of competence.

Fine, but the teeth of this act belong to some future justice department. I predict Trump will issue blanket pardons for everyone involved, up to Bondi; and that none of them will respect a congressional subpoena.

> My point with this is that lawyers, which the Department of Justice is full of, are no strangers to this process so should be able to do it adequately. If they reveal something damaging to their client this way, they themselves can get sued for whatever the damages are. So it's something they're careful about, for good reason.

> So in my opinion, it's unlikely that this is an act of resistance. Lawyers won't generally commit overt illegal acts,

Political redaction in this release under the Epstein Transparency Act is an overt, illegal act.

Does that reconfigure your estimation of whether DoJ attorneys that aren't the Trump inner-circle loyalists installed in leadership roles might engage in resistance against (or at least fail to point out methodological flaws in the inplmentation of) it?

You guessing my password is not the same as a know and expected behavior of a program. Adobe has a specific feature to redact. PDF is a format known to have layers. Lawyers are trained on day one not to make this mistake. (I am a recovering lawyer). This is either incompetence or deliberate disclosure.

If you were blind would a screen reader read the documents? Thats not a hack.

If someone sends me a document with text in it that they meant to remove but didn't and then I read that text, I haven't hacked anything they're just incompetent.

Hacking is unauthorised use of a system. Reading a document that was not adequately redacted can hardly be considered hacking.

I’m not an attorney or anything, but the relevant federal statute is explicitly about unauthorized access of computer systems (18 USC 1030).

Opening someone else’s laptop and guessing the password would absolutely fall under that definition, but I think it’s very much questionable if poking around a document that you have legitimately obtained would do so.

But copying and pasting text of publicly released documents is not illegal. Accessing someone’s computer is illegal. While maybe it could fall under the umbrella of hacking in some general way, articles, and especially titles, should be more precise.

I guess but if you write something down real small and I squint at it is that still hacking?

It’s not a hack. It’s known, expected behavior of the program. Adobe has a specific feature to redact. Color filled boxes is not it.

> limit/lack of authorization

By serving up the PDF file I am being authorized to receive, view, process, etc etc the entire contents. Not just some limited subset. If I wasn't authorized to receive some portion of the file then that needed to be withheld to begin with.

That's entirely different from gaining unauthorized entry to a system and copying out files that were never publicly available to begin with.

To put it simply, I am not responsible for the other party's incompetence.

But this isn’t an unexpected technique it’s literally the core design of the pdf format. It’s a layered format that preserves the layers on any machine. Adobe has a redaction feature to overcome the default behavior that each layer can be accessed even if there is a top layer in front.

It's also the meaning used in the title of this very Web site.

You have a better editor?

The average office worker has it on their computer, illustrating how commonplace unredacting could be. Any text tool will work, even some designed to detect bad redactions in PDFs via drag and drop (now specifically trained on these known bad redactions). https://github.com/freelawproject/x-ray

Someone wanted to make sure to be the first?

IKR?!

I don't think there is a grand conspiracy here. Any schmoe can download these files, select with their mouse, and copy paste into a document.

> It would be bad for certain people to get him on a stand and force him to spill the beans.

Yep. I think this sort of thing is actually their biggest concern with releasing the docs. They can redact or lose documents that say anything directly incriminating about Trump and his associates and dismiss everything Epstein and testimonies from the 2020s say about him as confabulation, but there are other people who might want to take the administration down with them if they get caught or even just get fed up of being doorstepped by the media, and some of them might have receipts.

Right, I’m aware of the excuses the government uses to keep secrets.

But on principle, what right does the government have to keep secrets from its own people? I don’t believe we had that button at the founding, it was added somewhere along the way. I’m asking what is the justification for this, and whether in the grand scheme of things that outweighs the principle of the government not being a separate entity from the people.

There are multiple ways to approach witness protection. For example if we have a problem with witnesses being harmed we could make being involved with witness harm at any layer of indirection a capital offense. We can probably think of other options besides the government being allowed to keep secrets from its own people.

Is the Department of Justice not a part of the government?

Competence and possibility of malicious compliance are interesting questions, but I think the more appropriate question is if DoJ will be sued for violating the law by redacting unrelated content?

It's no problem if they leak the fact that an FBI office printer was used to print the documents the FBI released.

Most likely "I Am Me, I Am Free: The Robot's Guide to Freedom. - David Icke"

This seems highly misleading.

> - Remove the data. This is much harder than it sounds. Many PDF tools won't let you change the content of a PDF, not because it isn't possible, but because you'll likely massively screw up the formatting, and the tools don't want to deal with that.

Compared to other formats this is actually relatively easy in a PDF since the way the text drawing operators work they don't influence the state for arbitrary other content. A lot of positioning in a PDF is absolute (or relative to an explicitly defined matrix which has hardcoded values). Usually this makes editing a PDF harder (since when changing text the related text does not adapt automatically), but when removing data it makes it much easier since you can mostly just delete it without affecting anything else. (There are exceptions for text immediately after the removed data, but that's limited and relatively easy to control.)

> - Replace the data. This what what all the "blackout" tools do, find "A" and replace with "🮋". This is effective and doesn't break formatting since it's a 1-to-1 replacement.

That's actually rather tricky in PDFs since they usually contain embedded subset fonts and these usually do not have "🮋" as part of the subset. Also doing this would break the layout since "🮋" has a different width than most letters in a typical font, so it would not lead to less formatting issues than the previous option. Unless the "🮋" is stretched for each letter to have the same dimensions, but then the stretched characters allow to recover the text.

> The problem with "replacing" is that not every PDF tool works the same way, and some, instead, just change the foreground and background color to black; it looks nearly the same, but the power of copy-and-paste still functions.

PDF does not have a concept of a background color. If it looks like a background color in PDF, you have a rectangle drawn in one color and something in the foreground color in front of it. What you usually see in badly redacted PDF files is exactly this, but in opposite color: Someone just draws a black box on top of the characters. You could argue that this is smarter since it would still work even if someone would chnage colors, but of course, PDF is a vector format. If you just add a rectangle, someone else can remove it again. (And also copy & paste doesn't care about your rectangle)

>- Remove the data. This is much harder than it sounds. Many PDF tools won't let you change the content of a PDF, not because it isn't possible, but because you'll likely massively screw up the formatting, and the tools don't want to deal with that.

>- Replace the data. This what what all the "blackout" tools do, find "A" and replace with "🮋". This is effective and doesn't break formatting since it's a 1-to-1 replacement. The problem with "replacing" is that not every PDF tool works the same way, and some, instead, just change the foreground and background color to black; it looks nearly the same, but the power of copy-and-paste still functions.

You're making it sound way harder than it is, when both adobe acrobat and the built-in preview app on mac can both competently redact documents. I'm not aware of instances of either (or any other purpose-made redaction tools) failing. I wouldn't homebrew a python script to do my redaction either, but that doesn't mean doing redactions properly in some insurmountable task for some intern.

> Then you have the computer illiterate, who think changing the foreground and background color to black is good enough anyway

To be fair, this works if you print out those copies and then re-scan them.

Thanks for this. Really quells the urge I get every so often to just code my own PDF editor, because they all suck and certainly it couldn't be THAT hard. Such hubris!

qpdf has a redaction option. It’s routinely used to anonymize medical records for studies.

Even that might leak to length attacks: one reasonable plaintext would lead to black bars of 1135 px, another to 1138 px, and with enough redactions you can converge on what the plaintext might be.

The only safe way for journalists is to paraphrase what the document said and to say "an unnamed source claims that ..." and to guarantee with your reputation, and the reputation of your publisher, that you are being faithful to what the original source said. For even better results, combine multiple sources.

Unfortunately paraphrasing things and taking editorial responsibility have both been deprecated in favour of rereleasing press releases in the house style, so it's difficult to get the actual journalism these days.

> Perfectly censoring documents, particularly digital ones is actually surprisingly difficult.

But the difficult part is easily repeatable once it's figured out, which is why it surprises me that it's not built into Acrobat as a tool already.

sorry, yes, that one.

Great contest. And a great entry, I had a big chuckle running it and unredacting my documents, even photos!

Link broken. (See reply with link that works.)

AKA https://www.justice.gov/multimedia/Court%20Records/Matter%20...