Here are excerpts from the post:
One year ago […] the Foreign Intelligence Surveillance Court (also known as the “FISA Court”, or “FISC”), which oversees surveillance policy in the US, began publishing a docket after 35 years of silence.
Ever since, I’ve been following the Court by running @FISACourt on Twitter […]
@FISACourt is powered by an open source tool I wrote to watch the Court’s public docket site. Within 5 minutes of any change to the docket, the tool automatically posts a tweet with a link to the HTML change, emails me, and texts my personal cell phone. When that happens, I quickly investigate and read anything that was posted, and follow up with a hand-written explanation. […]
In another step, the FISA Court recently relaunched their site, now at fisc.uscourts.gov. […]
I’m now doing a proper crawl of the entire FISC website, and saving data for every filing. […]
That data is in YAML, a humane data format that lends itself well to human readability, and line-by-line diffs.
I’m also downloading the actual PDFs, and watching them for any future changes. As the Court showed with its unexplained withdrawal detailed above (and as the Supreme Court frequently demonstrates), it’s important to watch the public record for revisions of its history.
If you want to use the FISA Court’s data, or the scraper, or have any questions at all, please open an issue over at the project. It’s all public domain and you don’t need my or anyone’s permission, but I love to help out and talk about this stuff. […]
For more details, please see the complete post.