Preserving Digital Private Archives and Collections: How a Flemish Network Tackles the Challenges
In the digital age, cultural heritage institutions have faced a growing influx of born-digital and digitized archives and collections. For many organizations, particularly smaller private archives, managing and preserving this digital content has proven a complex and daunting challenge. However, a collaborative network in Flanders, known as AIDA, has developed a comprehensive solution to tackle this issue head-on.
The Rise of Digital Private Archives and Collections
The volume of digital archives and collections held by cultural heritage institutions in Flanders has grown exponentially over the past decade. Where once these materials were measured in gigabytes, they now exceed the terabyte threshold. This shift has brought new challenges to the forefront, as institutions grapple with issues of file format obsolescence, physical media degradation, and the need to provide meaningful access to researchers.
While larger cultural heritage organizations have made progress in addressing these concerns, many smaller private archives have struggled to keep up. These Digital Private Archives and Collections (DPAC) often lack the resources, expertise, and control over their materials that is typically afforded to public archives or major institutional collections.
“For legal reasons, records from government organizations (public archives) also gained a lot of attention. These cases have one thing in common that greatly simplifies digital preservation: there is quite a large degree of control over the materials by the archival institution,” explains the AIDA network. “With DPAC, this is rarely the case. There is also little control over or participation in the creation of digital documents.”
Forming AIDA: A Cooperative Network for Digital Preservation
Recognizing the growing urgency of the DPAC challenge, a consortium of seven small to medium-sized cultural heritage institutions and two heritage service providers in Flanders came together to form the AIDA (Automating Ingest Digital Archives) network in 2017. Their goal was to develop a comprehensive, cooperative solution to the preservation of digital private archives and collections.
From the outset, AIDA established several core principles to guide its work:
- Cooperation and Transparency: AIDA would operate as a cooperative network, emphasizing transparency, shared interests, and trust among its members.
- Knowledge Sharing: The free sharing of knowledge, including the use of open-source software, would be essential to the network’s success.
- Balanced Partnerships: AIDA would maintain a balanced relationship with its service providers, avoiding vendor lock-in or over-reliance on a single technical partner.
- Experimentation and Expertise: AIDA would encourage experimentation and the gradual build-up of digital preservation expertise among its members.
With these principles in place, AIDA set out to tackle the preservation of DPAC through a series of practical, hands-on projects, each designed to address specific challenges and expand the network’s capabilities.
The AIDA Approach: From Pilot to Production
Digital Repair Café: Rescuing Data from Obsolete Media
AIDA’s first project, the Digital Repair Café, tackled the pressing issue of retrieving data from obsolete digital media. The network’s members conducted an inventory of the obsolete media in their collections, ranging from 5.25-inch floppy disks to quarter-inch cartridges (QIC). The project then involved:
- Acquiring the necessary hardware components to capture data from these legacy formats
- Developing workflows and documentation for the capture process
- Conducting pilots with AIDA members to test the workflows
- Sharing the expertise and resources with the broader cultural heritage sector in Flanders
By establishing the Digital Repair Café, AIDA ensured that its members could safely and effectively extract data from obsolete media, preserving valuable digital content that might otherwise have been lost.
SCALA: Building a Preservation System for DPAC
With hands-on experience in digital preservation under their belt, AIDA embarked on a larger-scale project called SCALA, which focused on implementing a customized digital preservation system for DPAC. This multi-phase initiative was designed to address several key requirements:
- Open and Well-Documented Software: To avoid vendor lock-in, AIDA insisted on using open-source software that was well-documented and maintained.
- Customizable Workflows: The preservation system needed to be flexible enough to accommodate AIDA’s specific workflows and requirements for DPAC.
- Standardized Information Packages: AIDA chose to adopt the E-ARK standard for the creation and exchange of Submission Information Packages (SIPs) and Archival Information Packages (AIPs).
- Reuse of Existing Infrastructure: By partnering with the Flemish service provider meemoo, AIDA was able to leverage existing storage and preservation infrastructure, reducing costs and avoiding the need to build everything from scratch.
The SCALA project resulted in a proof-of-concept digital preservation system, built on the RODA open-source software and integrated with meemoo’s storage infrastructure. This setup allowed AIDA to gain hands-on experience with the ingest, storage, and management of DPAC, while maintaining the flexibility and control that was essential to the network’s approach.
Organizational and Business Modeling
With the technical foundations established, AIDA turned its attention to the organizational and business aspects of its solution. Recognizing the need for a sustainable and scalable model, the network engaged the services of a cooperative and social entrepreneurship advisory center.
Through a series of intensive sessions, AIDA developed a comprehensive business plan that addressed key elements such as:
- Organizational Structure: AIDA is in the process of transforming from an informal network into a formal non-profit organization, with a steering committee and board of directors representing its member institutions.
- Funding Model: The core of AIDA’s funding will come from membership fees, ensuring that the network remains “for the members, by the members.”
- Service Offerings: AIDA has defined six clusters of services, including a knowledge network, strategy development, user support, experimental projects, pre-ingest tools, and the preservation environment.
By carefully considering the organizational and business aspects, AIDA has laid the groundwork for a robust and sustainable solution that can continue to evolve and serve the needs of its members and the broader cultural heritage community.
Lessons Learned and Future Outlook
The AIDA network’s journey in tackling the preservation of DPAC has yielded several valuable lessons that may benefit other cultural heritage organizations facing similar challenges:
- Start Small, but Start: AIDA emphasizes the importance of taking action, even with limited resources. Well-defined, small-scale projects can provide invaluable practical experience and serve as showcases to secure further support.
- Communication is Key: In a cooperative model, effective communication and knowledge-sharing among partners are essential to maintain a thriving network and avoid single points of failure.
- Leverage Standards and Existing Infrastructure: By adopting standards such as E-ARK and partnering with established service providers, AIDA was able to reduce costs and complexities while maintaining flexibility and control.
- Prioritize the Organizational Aspects: A well-structured organization, with clear roles, responsibilities, and business models, is crucial for the long-term sustainability and scalability of digital preservation initiatives.
As AIDA transitions from the pilot phase to full-fledged production, the network plans to scale up its services and accept new members, both within Flanders and potentially beyond. The knowledge network will continue to be a priority, with the goal of sharing AIDA’s expertise and documentation more widely within the cultural heritage community.
By demonstrating the success of a cooperative, open-source approach to the preservation of DPAC, AIDA hopes to inspire and empower other small to medium-sized cultural heritage institutions to take on the digital preservation challenge. The network’s journey serves as a shining example of what can be achieved through collaboration, technical expertise, and a strategic organizational foundation.
To learn more about AIDA’s initiatives and access the network’s growing collection of resources, visit https://itfix.org.uk/networking-support/.
