DARIAH-EU and the DARIAH Working Groups organise a series of Open Meetings from September to December 2021 with the goal of presenting current work undertaken by different Working Groups and to give them a platform to share future plans and to and start a dialogue for potential future collaborations.
On Thursday November 4, 2021 (10:30-12:00 CET), the Sustainable Publishing of Metadata Working Group will give an update on their development status and the achievements reached during the past year. The working group Sustainable Publishing of Metadata will also introduce its plans for the coming year. The timing, full program and some information about each of the talks can be found here below.
Please keep in mind that in order to participate in the meeting you need to register via Zoom here:
10.30 Introduction WG (Johan Van der Eycken, State Archives Belgium)
The Sustainable Publishing of Metadata Working Group focuses on developments regarding data sharing and interoperability as well as collaboration with other research infrastructures and projects, devoted to collecting and making available all types of (research) data, a field which is in full development.
10.40 Legal aspects of Open Data platforms: ‘the SODHA case’ (Oliver Lenaerts, State Archives Belgium)
In this contribution the legal aspects of Open Data platforms will be discussed by means of ‘the SODHA’ case. SODHA is the federal Belgian data archive for social sciences and the digital humanities. On the website, users can find and deposit social science and digital humanities data in reusable form. Open data is strongly promoted by enabling reuse of research data and by safely preserving datasets in the long term. This comes with some very interesting challenges about Legal Governance Issues that will be discussed during this talk.
10.50 Introducing CLARIAH-VL: Advancing the Open Humanities Service Infrastructure (Tom Gheldof, coordinator CLARIAH-VL/DARIAH-BE)
The CLARIAH-VL Open Humanities Service Infrastructure, funded for at least two years by the Research Foundation Flanders (FWO), is being developed by an interdisciplinary team of researchers, digital humanities experts and infrastructural specialists, from the universities of Ghent, Antwerpen, Leuven and Brussels, together with linguists from the Dutch Language Institution (INT). CLARIAH-VL will offer a unique combination of high-quality and user-friendly tools and resources that can be seamlessly embedded into the everyday workflows of humanities researchers, designed to improve the utility, accessibility, reusability, and sustainability of their research processes and data.
11.00 KBR Virtual Lab: a Belgian contribution to DARIAH (Julie Birkholz, KBR)
Belgium’s National Scientific Library- KBR, is responsible for preserving historical & cultural objects and for ensuring & providing access to the public. The proposed Virtual Lab allows KBR to develop an infrastructure to facilitate data-on-demand services for the public and researchers to generate data corpora from available KBR data. This includes the technical & practical development of authentication & authorisation for users, to implement APIs to query & export KBR’s data, with support from BELNET. Providing collections as data affords digital humanities research on cultural heritage collections, allowing us to pose different questions to understand our history. E-infrastructures provide opportunities for facilitating access & use of these collections. The KBR Virtual Lab is a contribution to DARIAH and the open data directive, by developing an infrastructure for Belgian & European researchers to access KBR’s collections as data. This infrastructure will contribute to the establishment of KBR’s Digital Data Strategy (2022-2024) and implement the feasibility studies of data.kbr.be & goals of the Digital Research Lab to facilitate text and data mining on the collections. It also contributes goals in the future Action Plan (2022- 2024) in supporting digitisation and facilitating KBR’s data as FAIR – findable, accessible, interoperable & reusable. The Virtual Lab can be used as a best practice for other federal science organizations in considering how to make collections as data available to users, and integrate data-on-demand services to automate the compiling of data corpora.
This contribution focuses on the accomplishments of the European Holocaust Research Infrastructure (EHRI) during its first and second life cycle phase as well as its plans for the future in becoming an ERIC. Since 2010 EHRI has searched for and discovered ways to connect archival holding institutes which wish to share collection descriptions and metadata with other institutes, researchers and other types of users. Crucial in these attempts are the portal website and the research program created by EHRI. The presentation will illustrate the work done on data exchange by EHRI as well as the learning curve of the consortium, and its plans for the future.
11.30 The EHRI (Mobile) Data Integration Lab: what it is, what it solves and which the challenges are (Herminio García González, EHRI Data integration specialist)
The European Holocaust Research Infrastructure (EHRI) envisages to publish the Holocaust relevant collections meta-data through the EHRI portal in a centralized fashion. Although great effort was done previously to develop tools that could alleviate the process of data integration within the EHRI portal, it is still a hard task that requires some IT knowledge, let alone non-experts users. Therefore, the creation of a data integration lab is seen as a way to help institutions in their data integration endeavors while also establishing more sustainable connections with them. After an overview of the state-of-the-art data integration processes inside the EHRI project, further challenges that the lab will face will be described.
We are very pleased to announce the launch of the Social Sciences and Digital Humanities Archive ‘SODHA’. SODHA is a data archive – an archive for datasets produced in the course of scientific investigations – in which researchers can deposit and find reusable data in any discipline of social sciences and the (digital) humanities.
Datasets published through SODHA receive a Digital Object Identifier (DOI), which makes them likelier to be harvested by search engines, thus boosting their online visibility. This not only highlights researchers’ accomplishments in the form of data collections alongside academic publications; it also makes datasets citable and findable, like other traditional sources such as articles, book chapters, books, conference proceedings, etc.
In this way data collected and prepared thanks to public funding are made accessible for reuse, and researchers’ efforts are publicly rewarded. They become thus findable, accessible, interoperable and reusable — in one word: FAIR data.
As a new service of the State Archives of Belgium, SODHA is committed to preserving research materials with the same quality standards and warranties already adopted by its home federal archive institution. SODHA also promotes open science and transparency by encouraging researchers to deposit their data under an open licence, which facilitates data reuse and enables scientific replication. SODHA relies on the open-source Web application Dataverse. Thanks to the BISHOPS project (Belgian Infrastructure for Social Sciences and Humanities Open Science), we are now broadening SODHA’s scope to include the (digital) humanities.
Would you like to browse the SODHA dataset collections or to deposit your own? Access SODHA here: https://www.sodha.be/
Archives Portal Europe (APE) aggregates archival material from and about Europe, currently providing access to archival descriptions from more than 1,100 institutions of more than 30 countries in a single research point. It is currently the largest aggregator of its kind in the archives domain, and it aims to connect all archival collections related to the history of European countries, from the large state archives to local and private archives. The portal allows to conduct research with an unprecedented outlook based on cross-country archival comparisons and multilingual search. Thanks to this new tool, the functions indicated by Unsworth as “scholarly primitives” are available to researchers, and expressed in new forms.
This workshop presents APE to both researchers and archivists as a tool that allows new types of historical research. Researchers will learn how to conduct research on APE in the most effective way possible; archivists will be presented with the metadata solutions applied by APE in order to provide the best possible returns to the users’ search queries, while at the same time being sustainable and feasible for the partner institutions.
This is embedded in a more general reflection on how metadata can inform researchers, connecting the work on APE with DARIAH’s Working Group on Sustainable publishing of (meta)data, DARIAH’s involvement in the Social Sciences and Humanities Open Cloud (SSHOC) project, and the Parthenos project.
Following the individual presentations, speakers and attendees will be invited to reflect together on the role of metadata and cloud computing in digital archival research, and how archivists and researchers can collaborate to make digital archives as open as possible by improving searchability and retrievability.
Authors: Dorien Styven (project manager), Marius Caragea (IT specialist) and dr. Veerle Vanden Daelen (curator)
Kazerne Dossin – Memorial, Museum and Documentation Centre on Holocaust and Human Rights opened its doors on 1st December 2012. The institute inherited its archival collections from its predecessor, the Jewish Museum of Deportation and Resistance (JMDR). The JMDR had been inaugurated in 1995 at the former SS-Sammellager Mecheln, better known as the Dossin barracks, in Belgium. Between the summer of 1942 and 1944 over 25,500 Jews, Roma and Sinti were deported from here to concentration camps the East, mostly to Auschwitz-Birkenau.
From the mid-1990s onwards the JMDR collected archival materials on the Holocaust in Belgium. The way in which these were catalogued and made available to the public changed dramatically over the past few years as Kazerne Dossin and EHRI became partners in sharing (meta)data with a broader audience. Using the example of the Lewkowicz family collection (unique, persistent identifier KD_00404) this article focuses on the evolution Kazerne Dossin went through while standardizing descriptions, and on the tools EHRI provided to optimize the workflow for collection holding institutes, a process from which Kazerne Dossin benefited as a test case.
The Birth of an Archival Collection
In 1994 a small team of two researchers started to collect documents, photos, objects, testimonies and other relevant archival materials to create the Jewish Museum of Deportation and Resistance’s permanent exhibition. To facilitate and catalogue all possible objects for the exhibit, all items – whether they were obtained physically or digitally, and whether they fell under the copyright of other institutes or under the copyright of the JMDR itself – were described in French on item-level in one large MS Access database.
Each item received a persistent, unique identifier and was attributed to one of four archival series: the A-series for documents and objects, the J-series for newspapers and magazines, the P-series for photos and the T-series for precious prints. The items were thus rather organised according to museum standards and not as a unit on collection level. Each of the four series had its own consecutive numbering and four customized metadata schemas. These were not compliant with the international archival standards: the museum staff developed their own standard.
As a result of the inauguration of the JMDR in 1995, survivors started donating their private collections to the JMDR. The accumulation of these valuable archival sources and information consequently led to the establishment of a documentation centre onsite. The staff members continued to describe archival additions on item-level in French according to the metadata schemas that were developed for exhibition purposes, thus dividing the donated collections of survivors and assigning items from the donation to one of the four series, which themselves became collections.
The Lewkowicz family collection, for example, contained both photos and documents. The photos were added, numbered and catalogued according to the metadata schema of the P-series, while the documents were added to the A-series. The only link between the item descriptions was the name of the donor as indicated in the metadata field Côte d’obtention.
This work method resulted in very detailed and rich descriptions which, because of their original purpose, very often also contained biographical information on the depicted persons or persons mentioned in the document. However, because initially intended to serve the creation of the museum exhibit, the descriptions were work-documents meant for and only accessible to documentation centre staff members. As there were no collection-level descriptions either, the context of the collection and many items as well as the link with the donor was unclear. The system of item descriptions proved hard to use for both non-employees and employees not acquainted with the work methods. The knowledge transfer among staff members and between “generations” of employees was challenging.
The JMDR described its archival holdings on its website in eleven to seventeen collections (the number depends on the version of the institute’s website). The identifiers of the collections changed whenever the website was updated. The collections shown included government series such as the municipal Jewish Register of Belgium and immigration files of Jewish deportees, as well as the four large series created by the JMDR. However, the non-standardized, full text collection descriptions did not contain content details for the A, P, J and T-series. There was for example no information on and no reference to the Lewkowicz family collection in the A- and P-series, so users were not aware of the existence of this collection. Due to a lack of alternatives, when launching its website in 2012 Kazerne Dossin took over the collection presentation method on its webpage. The work method would, however, change after the development of a portal website suited to share collection descriptions.
Adapting the workflow
As of 2011, with the switch from JMDR to Kazerne Dossin, the museum made it its explicit mission to collect, research and share metadata and digital copies of archival collections related to the history of the Holocaust in Belgium. In 2012, the Kazerne Dossin board of directors decided to invest in a digital asset management system. The Media Haven software package was selected to create a closed source portal website via which Kazerne Dossin could share its archival collections, both images of the scanned archives and metadata, with partner institutes worldwide.
At the same time, the European Holocaust Research Infrastructure expressed its wish to bring Kazerne Dossin and its archival collections on board of its archival portal (https://portal.ehri-project.eu/) as a test case to analyse methods to standardize archival processes and to identify much-needed tools to help collection holding institutes to share their descriptions with researchers and other institutes. This partnership would lead to growing professionalization in preservation, digitization and description standards for Kazerne Dossin.
During EHRI’s first phase (2010-2015), Kazerne Dossin focused on reworking the archival methods and materials it inherited from the JMDR. EHRI coached Kazerne Dossin when introducing good practices and standardized archival methods, and stepped in to adapt the existing infrastructure (as the institute’s databases were created and maintained in MS Access). The most challenging and at the same time most important step in this phase was, however, to master the jargon, or in other words: to learn to speak ICT and implement ICA standards (https://www.ica.org) in order to communicate with the IT staff members of EHRI and the software provider, and transmit this to the documentation centre’s staff.
The partnership with EHRI allowed Kazerne Dossin to quickly grow towards a more standardized and thus more broadly usable format. A first important step was to accord unique and persistent identifiers to all archival collections received as of 2012. Also, all new collections would be described on collection-level according to the ISAD(G) standard (https://www.ica.org/en/isadg-general-international-standard-archival-description-second-edition) and no longer on item-level (except for photos). This had the double advantage of (1) allowing to process and provide information on archival collections much faster (since writing descriptions on a collection level are considerably less time-intensive than on item level) and (2) allowing more broadly sharable descriptions as collection-level faces fewer challenges on privacy regulation than item level descriptions.
Subsequently, based on the name of the donor, items donated before 2012 and catalogued in the A, P, T and J series were recombined into their original collection and described on collection level. The documents and photos of the Lewkowicz family collection were thus reunited in one collection level description. The context of the collection was researched and added to the description. The cataloguing of pre-2012 donations is an ongoing process. Today, dozens of collection descriptions are available via the public end of Kazerne Dossin’s portal website, including the Lewkowicz family collection.
Harvesting and sharing descriptions
Until 2015, due to many changes in Kazerne Dossin’s work flow, EHRI exported the Kazerne Dossin collection level descriptions in CSV from the old MS Access database, which served as a “draft catalogue”. This database was intended as a tool to assemble collection descriptions, before uploading the metadata together with digital images to the portal website. This situation was not ideal, but it allowed Kazerne Dossin to share 253 collection descriptions on the EHRI portal.
That same year Kazerne Dossin tested the APE (http://www.archivesportaleuropefoundation.eu/) data preparation tool designed by Archives Portal Europe to import xml files with a valid Encoded Archival Description (EAD) exported from a portal website, into APE to share the collection level descriptions with a broader audience. Although Kazerne Dossin assumed it would be able to export valid EAD directly from its portal website, this tool indicated that the Kazerne Dossin xml files were not compliant with the APE portal site as they did not contain valid EAD. Therefore, Kazerne Dossin could serve again as a test case to install and explore the new tools EHRI was developing.
The EAD Conversion Tool (ECT) was created by EHRI to allow collection holding institutes to map and convert bulk xml files into individual xml files each containing metadata of one collection; the Metadata Publishing Tool (MPT) was developed for collection holding institutes in need of a way to present the xml files for harvesting and publishing on the EHRI portal (https://ehri-project.eu/ehri-for-institutions#EAD). Kazerne Dossin served again as a test case since it was not able to export valid EAD, nor was an OAI-PMH available. Kazerne Dossin therefore needed both tools developed by EHRI to be able to provide partners with publicly available xml files containing valid EAD for sharing purposes.
The implementation of tools
EHRI implemented both tools at Kazerne Dossin in June 2017. For the moment, valid EAD is created by converting Kazerne Dossin’s portal website’s raw XML export using the ECT on a client computer. The converted output is then used for the MPT to create a browsable set of files that the harvesters can locate. EHRI assisted in providing documentation, live demo’s and tweaking the tools until Kazerne Dossin was able to get an error-free output (EAD conversion and publishing).
How does it work in practice? Via an Application Programming Interface (API), Kazerne Dossin can export all collection level descriptions from its portal website as a bulk xml. Kazerne Dossin and EHRI together mapped the metadata in this bulk xml on the international standards. During this stage, Kazerne Dossin and EHRI used a trial and error method. The exports were run by EHRI until an error occurred after which the error was solved. The errors in the xml’s on collection level included missing <p>tags; the usage of capital letters in some metadata fields; ISO codes for language, script and country (which were introduced); readability and misunderstandings when mapping the metadata.
The ECT could thus successfully convert the bulk xml in single xml files, one per collection, compliant with EAD. As Kazerne Dossin had restructured its collection level metadata schema in a prior project phase to fit ISAD(G), no principal, conceptual differences in data structure were identified during the mapping phase.
The Metadata Publishing Tool was then used to create URL’s for every xml file. Two folders – one containing the xml files with valid EAD and another regarding resource synchronization – were offered to interested partners via an accessible endpoint on the Kazerne Dossin server. The folder processed by the Publishing tool is deployed to a simple website (collections.kazernedossin.eu) that the harvesters can access. For security reasons on the server, the folder browsing functionality is disabled and a direct link to the folder with the exported EAD files could not be provided. One of the collections currently awaiting export is the Lewkowicz family collection.
With the current manual process, periodical exports can be planned. However, in order to automate the process, an extra (tool) implementation is most likely required between Kazerne Dossin’s portal website’s API: providing automatically/scheduled raw XML exports at regular bases and the ECT will transform the exports to valid EAD.
Today, Kazerne Dossin has advanced enormously in its archival work thanks to EHRI. Although the resource sync tool is not yet fully operational and the ultimate goal is to have both the ECT and MPT in an automated process (with auto-generated files automatically published on the website), Kazerne Dossin now has the opportunity to share valid EAD with partner institutions. Kazerne Dossin and its archival holdings’ visibility has improved and it gives the documentation centre the opportunity to join other initiatives and connect its (meta)data with partner projects and institutions worldwide. Hundreds of Kazerne Dossin’s collections will be shared online, honouring thousands of deportees such as Rosine Régine Lewkowicz, her parents, brother and sister who were all killed in the Holocaust, leaving only one brother to survive.
The Working Group on “Sustainable Publishing of Metadata” has been active within the Digital Research Infrastructure for the Arts and Humanities (DARIAH) European network since 2016. The working group aims to present and share information regarding ongoing and established projects which deal with the creation, reproduction and sharing of metadata between institutes and researchers within the field of Humanities and Social Sciences. The contributions on this blog will document both the activities of the working group as well as present projects dealing with this use of metadata. We wish to realize a collaboration between European and pan-European research infrastructures and in this way stimulate the exchange of expertise between them as well as projects led by partners. This working group aims to disseminate best practices and the use of standards (EAD, DDI, RIC, IIIF, …). Its members are (next to DARIAH), among others, involved in the research infrastructures CESSDA (Consortium of European Social Science Data Archives), EHRI (European Holocaust Infrastructure), APE (Archives Portal Europe), Europeana,…