EOT Harvest, part II: How we got involved

I became aware of the EOT Harvest through meetings and presentations at the Depository Library Council conference, so when I saw that the project was seeking volunteers to nominate content for the archive, I already had some notion of what the project may involve.

I saw the notice on my favorite blog, Free Government Information (FGI). The notice briefly described the project and called for volunteers to nominate, through an online form, U.S. Federal government domains to be archived.
I used the generic e-mail address that was provided for further questions, and e-mailed the following:

Dear EOT project managers,
I saw the call for volunteers on FGI and thought this may be an opportunity to involve students taking my Government Information Sources (Fall 2012) course. Since they are new to GovDocs I would have to have something a little more contained and targeted for them. If there are any specific agencies/sub domains and such that I can have students work on, I would be glad to help. This is both a great learning opportunity as well as an act of civic responsibility. If this is at all on interest we can pursue this further.
Thank you.
Debbie Rabina

The first promising sign was a reply that came within 24 hours from one of the partners, the Library of Congress. They were excited about the suggestion and invited further discussion.
I cannot emphasize enough the importance of the relationship between volunteers and the EOT team. In our case it is making this project so much easier. In the weeks since we began work, I have been e-mailing the EOT team regularly, usually several times a week, At least one team member gets back to me within hours–including during weekends. This is instrumental to keeping the pace of the project going. Students are waiting for answers so that they can submit nominations, and every effort is made not to slow them down.

Following some e-mail exchanges and a conference call between myself and the EOT team, we came up with a project for students. During the course of the semester, students will systematically locate social media sites that are maintained by all three branches of the federal government such as, for example, NASA’s Twitter feed.

Sixteen students are involved in the project, all taking LIS 613 Government Information Sources. Most students are new to government information and the project was initially not very clear to them.

Several elements contributed to getting students to understand and get excited about the project. These included several classroom discussions that were enforced by a detailed write-up of the assignment, published literature of the 2008 EOT harvest, and finally, a conference call with the EOT team.
Sitting in our classroom in New York City, with a combination of Skype, a land-line conference call and Power Point slides, we discussed the project and our role and students had an opportunity to have their questions answered. It was after this conference call, which took place during Week 3 of the semester, that we finalized the workflow and began nominating websites for inclusion in the 2012 EOT archive.

As the project and our own involvement become clear, and lessons about government information began to emerge:

Lesson #1: The amount at at-risk information is enormous.
Students were under the impression that the Federal Depository Library Program preserves all content authored by the government and were surprised to learn that most agencies’ web content is not part of the distribution and preservation efforts of GPO. This required us to set up a good workflow that would capture as much of this information as possible.

Coming next: Our workflow

Introducing the End of Term Harvest project at Pratt SILS

I am excited to be contributing this semester to the End of Term (EOT) harvest project.

What, you may rightly ask, is the EOT harvest?
So here’s the short answer: The EOT harvest archives web information from the U.S. federal government in the months before and after presidential elections.

Who is behind the EOT Web Archive: The archive is created through a partnership among several organizations, among them The Internet Archive, the Library of Congress, California Digital Library, University of North Texas and the Government Printing Office.

More detail: The federal government authors a lot of content in many formats, over 95% of it electronically born. Some of this content is very well managed and protected by legislation. It is cataloged and authenticated and receives metadata and is maintained in archive quality file formats. These include bills and laws and titles such as the Congressional Record or the Code of Federal Regulations. Such content is available from FDsys, the content management system maintained by the Government Printing Office.
But other types of government content do not benefit from these practices. These include websites and their content created by over 500 government agencies.

At the end of a presidential term, particularly if a new president is elected, this web content is in high risk of disappearing from the web. This content reflects a record or blue print of the government. The EOT project was created to collect, preserve, archive and maintain this content. During the 2008 presidential election, the EOT harvest collected 16 terabyte of content that is now made available on the Internet Archive website.

Now the project continues for the 2012 presidential elections and students taking Government Information Sources (LIS 613) at Pratt SILS, are volunteering for the project. Very briefly, we are collecting URLs of social media websites of government agencies, such as the State Dept. twitter feed, for inclusion in the 2012 EOT Archive.

In the next blog posts I will describe in detail our involvement, our workflow, the questions we are encountering, solutions we are finding and lessons we are learning.