Archive-It

August 27th, 2024

by Bridget Collings, Web Archivist

Harold Washington Library Center Winter Garden (Image: Spiro Bolos, CC BY-SA 2.0)

On August 14th, the Archive-It community convened for its 18th annual Partner Meeting in Chicago, IL. Held at the Harold Washington Library Center of the Chicago Public Library, the event brought together 11 Archive-It staff members and 30 partners and friends for a day of productive discussions and networking. This year’s meeting was scheduled to align with the Society of American Archivists’ ARCHIVES * RECORDS 2024 conference and was organized in collaboration with ArchivesSpace, who conducted their annual forum in the morning, allowing our session to follow in the afternoon.

The afternoon was structured into two parts: 

1. An update on Archive-It services
2. Facilitated breakout sessions

Kody Willis, Product Operations Manager, began the afternoon session with an overview of Archive-It services and introduced the staff members who would provide detailed updates. A Q&A session followed this.

Kody Willis provides an overview of AIT stats.

Key Updates and Developments

Alex Dempsey, Engineering Manager, presented a detailed overview of recent advancements and ongoing initiatives. Highlights included:

• Crawling Enhancements: Upgrades to the Standard crawler, improved video collection capabilities, and performance enhancements to Brozzler.
• Reporting Improvements: Enhancements to host reporting.
• Access Upgrades: Hardware improvements for better load times and stability, migration to Plausible, the launch of ARCH, and updates to CommWebs access sites.
• Future Infrastructure Efficiencies: Simplified crawl management, redesigned archive-it.org for a more intuitive user experience, and new data deduplication options.

Kody Willis then shared insights on Archive-It’s statistics and trends. He discussed the complexities of crawling platforms like Twitter/X and social media, emphasizing ongoing efforts to archive streaming videos and explore better QA methods.

Sylvie Rollason-Cass, Senior Web Archivist, demonstrated new features, including options for excluding video content to optimize data budgets and improvements in crawl scheduling. She also issued a call for testers interested in contributing to these developments. If you want to get involved, contact us via a support ticket.  

Watch a recorded summary of Archive-It’s web archiving services update

Community Webs Program and Vault Service Updates

Anna Trammell, Program Manager, provided an update on the CARTA Program and the Community Webs Program. She detailed CommWebs participants, the collections created, and the recent symposium funded by the Mellon Foundation. She encouraged attendees to apply for the CommWebs program and attend upcoming information sessions.

Updates on the Vault service were also presented, including details about a complimentary ½ TB account for CommWebs partners. To learn more about Vault, check out Vault’s help center or contact vault@archive.org.

ARCH Demonstration

Karl Blumenthal, Senior Web Archivist, offered an overview and demonstration of ARCH (Archives Research Compute Hub), discussing recent updates, ongoing development, and how to get started with the platform.

To learn more, check out ARCH’s help center or contact arch@archive.org. 

Karl Blumenthal provides an overview of ARCH.

Q&A Session

The Q&A session addressed several key topics:

• The potential for publishing datasets on ARCH and what this entails for partners.
• Enhanced language processing and search functionality, including support for indigenous languages on archive.org.

Breakout Discussions

Following a brief break, attendees participated in facilitated breakout discussions on various themes:

• Web Archiving within Communities: Led by Anna Tremmel.
• Digital Preservation / Vault: Led by Jefferson Bailey and Kyrie Whitsett.
• Quality Assurance & Social Media: Led by Tanya Ulmer and Kody Willis.
• Making Web Archives More Usable: Led by Sylvie Rollason-Cass.
• Office Hour with a Web Archivist: Led by Karl Blumenthal.
• Engineering & WA Technologies: Led by Alex Dempsey.
• Fostering Community Among Web Archivists: Led by Bridget Collings.

Partners had the chance to participate in two sessions, where they shared valuable insights and questions with the larger group. These discussions aimed to foster productive exchanges of ideas and strengthen the sense of community among web archivists. We are excited about the continued collaboration and innovation that this year’s meeting has sparked.

We extend a special thank you to ArchivesSpace for their partnership and to all our attendees for their active participation. We look forward to reconnecting at future virtual and in-person meetings!

Subscribe to our monthly newsletter to stay informed about our latest updates, upcoming events, recommended readings, and other web archiving news.