Schuman: U.S. House Convenes Second Public Meeting on Legislative Bulk Data

Daniel Schuman, Esq., of the Sunlight Foundation has posted House Convenes Second Public Meeting on Legislative Bulk Data, at the Sunlight Foundation Blog.

Here is an excerpt:

On January 30th, the House of Representatives held a public meeting on its efforts to release more legislative information to the public in ways that facilitate its reuse. This was the second meeting hosted by the Bulk Data Task Force where members of the public were included; it began privately meeting in September 2012. (Sunlight and others made a presentation at a meeting, in October, on providing bulk access to legislative data.) This public meeting, organized by the Clerk’s office, is a welcome manifestation of the consensus of political leaders of both parties in the House that now is the time to push Congress’ legislative information sharing technology into the 21st century. In other words, it’s time to open up Congress.

The meeting featured three presentations on ongoing initiatives, allowed for robust Q&A, and highlighted improvements expected to be rolled out of the next few months. In addition, the House recorded the presentations and has made the video available to the public. The ongoing initiatives are the release of bill text bulk data by GPO, the addition of committee information for docs.house.gov, and the release of floor summary bulk data. It’s expected that these public meetings will continue at least as frequently as once per quarter, or more often when prompted by new releases of information. [...]

The Bulk Data Task Force was formed in part in response to the #freeTHOMAS movement. That movement seeks free public bulk access to the contents of the THOMAS U.S. federal legislative database, which is gradually being superseded by a new database called Congress.gov.

For more details, please see Daniel’s complete post.

Mill: Sunlight Foundation releases Congress API

Eric Mill of the Sunlight Foundation points us to Sunlight Congress API released yesterday.

Here is a description:

A live JSON API for the people and work of Congress, provided by the Sunlight Foundation.

Features

Lots of features and data for members of Congress:

• Look up legislators by location or by zip code.
• Official Twitter, YouTube, and Facebook accounts.
• Committees and subcommittees in Congress, including memberships and rankings.

We also provide Congress’ daily work:

• All introduced bills in the House and Senate, and what occurs to them (updated daily).
• Full text search over bills, with powerful Lucene-based query syntax.
• Real time notice of votes, floor activity, and committee hearings, and when bills are scheduled for debate.

All data is served in JSON, and requires a Sunlight API key. An API key is free to register and has no usage limits.

We have an API mailing list, and can be found on Twitter at @sunlightlabs. Bugs and feature requests can be made on Github Issues. [...]

About the source of the bill data, Eric says:

it’s built on the github.com/unitedstates work that GovTrack and Sunlight and others created, which ultimately comes from THOMAS.

He adds:

there’s a mix of other (documented) official sources too. One of the API’s purposes is to connect and de-silo information.

For more details, please see the Sunlight Congress API site.

For more information on the github.com/unitedstates repository, which was co-developed by Eric, Dr. Joshua Tauberer of GovTrack, and Derek Willis of the New York Times, please see the post entitled New Congressional Data Available for Free Bulk Download: Bill Data 1973- , Members 1789-

HT @konklone

Congress.gov: New Official Source of U.S. Federal Legislative Information

The U.S. Congress has launched a new official legislative information system for the U.S. federal government: Congress.gov.

According to Alex Howard of O’Reilly:

[...] the new Congress.gov features responsive design, adapting to desktop, tablet or smartphone screens. It’s also search-centric, with Boolean search and, in an acknowledgement that most of its visitors show up looking for information, puts a search field front and center in the interface. The site includes member profiles for U.S. Senators and Representatives, with associated legislative work. In a nod to a mainstay of social media and media websites, the new Congress.gov also has a “most viewed bills” list that lets visitors see at a glance what laws or proposals are gathering interest online. (You can download a fact sheet on all the changes as a PDF).

Click here for Alex’s complete post about Congress.gov.

Daniel Schuman of the Sunlight Foundation observes that Congress.gov does not provide “for public comment on the design process [or] computer-friendly bulk access to the underlying data.”

Daniel, Tom Bruce of the Legal Information Institute, and others have recently written recommendations to Congress about providing public bulk access to congressional data.

Tom and his team have been consulting with the Library of Congress on their legislative metadata, and Congress.gov appears to reflect their work. Tom describes this new approach to legislative metadata approach in a series of posts here.

Kim Nayyer of the University of Victoria Law Library has also written about Congress.gov at Slaw.ca.

Nick Judd and Miranda Neubauer have written a post about Congress.gov at TechPresident: What Congress.gov Means for a Congressional API.

Bruce, Mill, Schuman, Tauberer & Wonderlich: Recommendations on Public Access to Legislative Data

Tom Bruce of the Legal Information Institute; Daniel Schuman, Eric Mill, and John Wonderlich, all of the Sunlight Foundation; and Dr. Joshua Tauberer of GovTrack and POPVOX, have posted a new report entitled On Public Access to Legislative Information: Recommendations to the Bulk Data Task Force (2012).

The report “provides a roadmap” that the U.S. Congress’s Bulk Data Task Force can use “to implement[]” free public “bulk access to” the THOMAS database of U.S. federal legislative information.

The report is a product of the effort — known as #freeTHOMAS — to provide free online public access in bulk to THOMAS.

For more information, please see Daniel’s post entitled How to #FreeTHOMAS: A report on implementing bulk access.

HT @danielschuman

Lulofs on Legal Open Data and freeTHOMAS

Meg Lulofs, JD, MSLIS, has posted has posted Open Data in a Librarian Hat: What’s Your Number One?, on the VoxPopuLII blog, published by the Legal Information Institute at Cornell University Law School.

In this post, Ms. Lulofs discusses the prioritization of applications of legal open government data, with particular reference to the effort to make legislative data from the US federal THOMAS system freely available on the Web (the movement commonly known as #freeTHOMAS).

Suhaka and Tauberer on Business Models for Reuse of Open Legislative Data

Karen Suhaka of LegiNation and Dr. Joshua Tauberer of GovTrack gave a “lightning talk” presentation entitled Legislative Transparency: A Round Up of Efforts and Results to Date (featuring this GoogleDoc spreadsheet on Open Gov Business Models) at IOGDC 2012 Virtual Conference: International Open Government Data Conference, 7 July 2012.

Part of the presentation concerned descriptions and examples of several different business models for using or reusing legislative and other government data. The legislative examples included:

• BillTrack50 (an example of the “Freemium” business model)
• GovTrack (“Advertising”)
• LexisNexis (“Pay services”)
• POPVOX (“Startup”)
• Sunlight Foundation (“Nonprofit”)
• THOMAS (“Government”)

The authors invite you to contribute additional business models and examples of use and reuse of open government data, to their GoogleDoc spreadsheet.

HT @Smoodle.

Google Endorses freeTHOMAS: Free Public Access to U.S. Federal Legislative Data in Bulk XML

Google has announced its support for Congress’s providing free public access in bulk XML to U.S. federal legislative data, in a new post by Seth Webb, Google’s Senior Policy Manager, entitled In Support of Legislative Transparency, on the Google Public Policy Blog.

The effort to persuade Congress to provide free public access to U.S. federal legislative data in bulk XML is popularly known as freeTHOMAS (hashtag #freeTHOMAS).

Mr. Webb writes:

We believe the ability to download bulk legislative data in formats like XML on a regular basis provides tremendous benefits. Website and app developers can use such data to provide up-to-date information on bills. Researchers can use it to perform studies. And politically-curious citizens can use it to follow legislation moving its way through Congress. [...] We applaud Congress for the work that it’s done to promote openness and look forward to a future of increased legislative transparency.

For more information, please see the complete post.

For more information about freeTHOMAS, please see Schuman: Major Transparency Milestone in Bulk Access Statement; the new post by David Moore of Participatory Politics Foundation entitled Next Steps in #freeTHOMAS Campaign; and Open Congress’s THOMAS bulk data access wiki.

HT @garvinfo

Tauberer, Bruce, and Schuman Quoted: Congressional Data May Soon Be Easier to Use Online

David A. Fahrenthold has published Congressional Data May Soon Be Easier to Use Online, Washington Post, June 8, 2012.

The article describes recent efforts to petition Congress to make U.S. federal legislative data — specifically the data underlying the THOMAS legislative information system — publicly available free of charge in bulk XML. Those efforts are popularly known as the freeTHOMAS movement (hashtag #freeTHOMAS).

The article contains quotations from Dr. Joshua Tauberer of Govtrack, Tom Bruce of the Legal Information Institute, and Daniel Schuman of the Sunlight Foundation.

For more information about freeTHOMAS, please see Schuman: Major Transparency Milestone in Bulk Access Statement; the new post by David Moore of Participatory Politics Foundation entitled Next Steps in #freeTHOMAS Campaign; and Open Congress’s THOMAS bulk data access wiki.

Schuman: Major Transparency Milestone in Bulk Access Statement

Daniel Schuman of the Sunlight Foundation has posted Major Transparency Milestone in Bulk Access Statement, on the Sunlight Foundation Blog.

In this post, Mr. Schuman reports that on 6 June 2012 the leadership of the U.S. House of Representatives issued a statement expressing its support for providing public bulk access to U.S. federal legislative data, the goal of the movement commonly known as “freeTHOMAS” (Twitter hashtag #freeTHOMAS.)

In the statement the House leadership identified as one of its goals the “provi[sion of] bulk access to legislative information to the American people without further delay,” characterized provision of bulk access to legislative data as “among our top priorities in the 112th Congress,” described a new task force whose charge is “to expedite the process of making public information available to the public,” and “direct[ed] the task force to begin its [...] work immediately.”

The full text of the House leadership’s statement appears at the bottom of this post, and is also available here.

Mr. Schuman characterizes the House leadership’s statement as momentous, and expresses substantial optimism about the statement. He writes:

[T]oday is a milestone for legislative transparency. [...] The debate over whether there should be bulk access to legislative data is over. Because bulk access is a top priority of the 112th Congress, we expect to see tangible progress in the upcoming months. The remaining questions largely concern how bulk access should be implemented to meet the needs of the public while respecting the legitimate concerns of Congress and its support agencies.

Certain aspects of the House leadership’s statement raise questions, however. Mr. Schuman notes that the task force will include no individuals from outside of government, and that the task force does not appear to be required to consider input from the public. In addition, the House leadership’s statement does not set a deadline for the completion of the task force’s work, does not require the task force to issue a report or to make any of its deliverables public, and does not require the task force to make use of the 2008 Library of Congress memorandum that addresses many of the issues that appear to be on the task force’s agenda. Further, the language of the House leadership’s statement, to the extent that it is contained in a press release, appears to lack the authority of language in legislation or a congressional report or print.

Mr. Schuman expresses optimism that the task force will invite input from the public, and states that “Sunlight [Foundation] and our friends in the transparency community stand ready to be of assistance as the technical, policy, and scope issues are addressed.”

Mr. Schuman also suggests that the public should continue to monitor this issue, to make sure that the promise of the House leadership’s statement is fulfilled. He writes:

While this is clearly progress, there’s still much more to do. We will be monitoring this issue closely.

For more information, please see Mr. Schuman’s complete post.

Click here for other commentary about #freeTHOMAS.

Here is the full text of the House leadership’s statement:

House Leaders Back Bulk Access to Legislative Information

Posted by Speaker Boehner Press Office
June 6, 2012
Press Release

WASHINGTON, DC – House Speaker John Boehner (R-OH), Majority Leader Eric Cantor (R-VA), Legislative Appropriations Subcommittee Chairman Ander Crenshaw (R-FL), and Oversight & Government Reform Committee Chairman Darrell Issa (R-CA) released the following statement today regarding House efforts to provide bulk access to legislative information:

“The coming vote on the Legislative Branch appropriations bill marks an important milestone for the House of Representatives: the moment lawmakers agree to free legislative information from the technical limits of years past and embrace a more open, more transparent, and more effective way of doing the people’s business. Our goal is to provide bulk access to legislative information to the American people without further delay.

“The bill directs a task force to expedite the process of making public information available to the public. In addition to legislative branch agencies such as the Library of Congress and the Government Printing Office, the task force will include representatives of House leadership and key committees, as well as the Clerk of the House and the House Chief Administrative Officer.

“This is a big project. That’s why accomplishing it rapidly and responsibly requires all those with a role in the collection and dissemination of legislative information to be at the table together. Because this effort ranks among our top priorities in the 112th Congress, we will not wait for enactment of a Legislative Branch appropriations bill but will instead direct the task force to begin its important work immediately.

“The offices involved in this project have been instrumental in using new technology to make the House more open. We pledged to make Congress more transparent and accessible, and from our efforts to provide legislation and updates in XML, to the video streaming and archiving of committee hearings, to our search for new ways to engage and serve the American people through events like last year’s ‘Hackathon’ – and more – we’re working to keep that pledge. Bulk data is the next and a very important step. We look forward to the task force’s report and to beginning implementation of this project as soon as possible.”

# # # # #

House Committee Votes to Indefinitely Delay Public Bulk Access to U.S. Federal Legislative Data

[UPDATE, 7 June 2012]: Schuman: Major Transparency Milestone in Bulk Access Statement

[UPDATE, 5 June 2012, 18:00 Eastern: Daniel Schuman reports: Issa Offers #FreeTHOMAS Amendment to Leg Approps Bill.]

[UPDATE, 5 June 2012: Daniel Schuman says the language of the print has been amended; see his new post entitled Bulk Access Language Tweaked by Approps.]

The U.S. House Committee on Appropriations has voted to approve a committee print directing an indefinite delay on a decision whether to provide U.S. federal legislative data in bulk XML to the public, according to Daniel Schuman’s new post entitled Bulk Access Developments after the H. Approps Hearing, on the Sunlight Foundation Blog.

According to the approved committee print, the decision will be delayed indefinitely while a new task force meets to consider the committee’s concerns. Those concerns mostly involve what are described as the “unresolved” “challenge of authenticating downloads of bulk data legislative data files in XML,” and costs and other issues that purportedly may arise from the alteration of legislative data by downstream users.

Here is the language that, according to Mr. Schuman, has been approved by the committee:

During the hearings this year, the Committee heard testimony on the dissemination of congressional information products in Extensible Markup Language (XML) format. XML permits data to be reused and repurposed not only for print output but for conversion into ebooks, mobile web applications, and other forms of content delivery including data mashups and other analytical tools. The Committee has heard requests for the increased dissemination of congressional information via bulk data download from non-governmental groups supporting openness and transparency in the legislative process. While sharing these goals, the Committee is also concerned that Congress maintains the ability to ensure that its legislative data files remain intact and a trusted source once they are removed from the Government’s domain to private sites.

The GPO currently ensures the authenticity of the congressional information it disseminates to the public through its Federal Digital System and the Library Congress’s THOMAS system by the use of digital signature technology applied to the Portable Document Format (PDF) version of the document, which matches the printed document. The use of this technology attests that the digital version of the document has not been altered since it was authenticated and disseminated by GPO. At this time, only PDF files can be digitally signed in native format for authentication purposes. There currently is no comparable technology for the application and verification of digital signatures on XML documents. While the GPO currently provides bulk data access to information products of the Office of the Federal Register, the limitations on the authenticity and integrity of those data files are clearly spelled out in the user guide that accompanies those files on GPO’s Federal Digital System.

The GPO and Congress are moving toward the use of XML as the data standard for legislative information. The House and Senate are creating bills in XML format and are moving toward creating other congressional documents in XML for input to the GPO. At this point, however, the challenge of authenticating downloads of bulk data legislative data files in XML remains unresolved, and there continues to be a range of associated questions and issues: Which Legislative Branch agency would be the provider of bulk data downloads of legislative information in XML, and how would this service be authorized. How would ‘‘House’’ information be differentiated from ‘‘Senate’’ information for the purposes of bulk data downloads in XML? What would be the impact of bulk downloads of legislative data in XML on the timeliness and authoritativeness of congressional information? What would be the estimated timeline for the development of a system of authentication for bulk data downloads of legislative information in XML? What are the projected budgetary impacts of system development and implementation, including potential costs for support that may be required by third party users of legislative bulk data sets in XML, as well as any indirect costs, such as potential requirements for Congress to confirm or invalidate third party analyses of legislative data based on bulk downloads in XML? Are there other data models or alternative[sic] that can enhance congressional openness and transparency without relying on bulk data downloads in XML?

Accordingly, and before any bulk data downloads of legislative information are authorized, the Committee directs the establishment of a task force composed of staff representatives of the Library of Congress, the Congressional Research Service, the Clerk of the House, the Government Printing Office, and such other congressional offices as may be necessary, to examine these and any additional issues it considers relevant and to report back to the Committee on Appropriations of the House and Senate.

As Daniel Schuman and Eric Mill have noted (here and here), the committee print language respecting the task force is problematic from the perspective of those who favor increased government transparency, increased participation of the public in government policy, or efficiency in government operations, because, according to the language of the committee print, the task force:

• will have no members from outside of the government
• has no deadline to complete its work
• is not required to receive input from the public
• is not required to make its report public
• is not required to take into consideration the 2008 Library of Congress memorandum that already addressed this issue in detail.

For commentary about this new House policy, click here.