Metadata problems and interface ideas: Report from the Dutch team
Emily Hansell Clark
Technologies of archival practice—of categorizing, cataloguing, preserving, and providing access—add additional traces that frame archival documents and shape their use and meaning. A technology that PICCH investigates is natural language processing (NLP), which is used to automatically produce written transcripts for audio and audiovisual media. One of our findings is that problematic or contentious terms from the content of colonial media make their way into archival metadata such as transcripts (through NLP) as well as descriptive summaries and keywords. One possible solution to this is to build a digital thesaurus of contentious terms so that users can better identify historical media of interest without needing to use outdated and problematic terms themselves, for example typing them into the digital archival interface’s search box. However, updating individual terms that appear in the archive does not necessarily update or identify the colonial tropes or ideas that use these terms. For example, in a historical media document that describes a particular culture as “primitive” in comparison with the West, what contemporary term could be substituted for “primitive”? None: because it is not just the word “primitive” that needs to be updated, it is the underlying ideology of a great divide in time and space between “the West” and “the rest”—a notion that unfortunately still appears in discourse today.
Language is fundamental to interacting with the digital media archive: transcripts, metadata, and search functions are all created and conducted using the medium of language. When contentious colonial terms appear in metadata at NISV, we can see that language and the technologies that rely on it can serve to regenerate colonial terms, tropes, and ideologies—even while striving to make the historical archive more accessible (including accessible for scholarly critique and critical reuse). Thus, my finding is that natural language processing, automatically generated transcripts, digitization and digital interfaces, and tools such as thesauri constitute advances in archival access, but not necessarily in terms of coloniality or polyvocality. In order to achieve the later, additional layers of creative and critical thinking are required.
Artistic interventions
One place to look for ideas and alternatives is in artistic interventions that have been conducted in the colonial archive—some of which have taken place using the collections of the NISV. For example, the 'Unlocking Sounds' project (2016 –2017), a collaboration between the Tropenmuseum/Research Center for Material Culture and NISV, invited three musical artists to delve into ethnographic sound collections housed at NISV to creatively repurpose sounds of the past. One of the artists, Parrish Smith, brought together sounds from the ethnographic archive with an exploration of his Surinamese roots and inspiration from the anticolonial Surinamese writer and activist Anton de Kom.
In the film Zij Noemen Me Baboe, the director Sandra Beerends creatively repurposes images of life in the colonial Dutch East Indies from the silent home films of Dutch civil servants who were stationed there with their families in the early XX century (including films now held by NISV). In the film, Beerends, who is herself of Dutch and Indonesian descent, combines these images and sets them to a new soundtrack and narrative to tell a story from the perspective of a baboe: one of many Indonesian nannies employed to take care of the children in colonial Dutch families. The experiences of the fictional narrator draw from the actual experiences of many former baboes whom Beerends interviewed. Thus, Beerends recontextualizes colonial images to tell an anticolonial story, reversing who has the power to narrate and frame the past.
In a third example, the researcher, theatre-maker, and composer meLê yamomo uses historical sound recordings and images to comment on European colonial entanglements in Southeast Asia and to explore what other perspectives and experiences the colonial archive may reveal. This has yielded multimodal theater pieces such as Echoing Europe: Postcolonial Reverberations alongside yamomo's scholarly projects that aim to analyze, make accessible, and even to repatriate colonial archives that document Southeast Asian pasts in Europe. In both art and scholarship, yamomo aims to understand media within its colonial context, but also to question aspects of that context (for example, archives of Southeast Asian sound recordings that are held by institutions in Europe and largely not accessible to researchers or other users located in South Asia).
In these three artistic examples, sounds and images from the colonial archive are reappropriated in music, film, and theatre works that reflect personal and critical engagements that challenge dominant European narratives about the past. These works stand on their own—but they also have implications for the European colonial media archive. Each of these projects offers ideas about the value and potentialities of separating sounds and images from the forms of language (both narration within a piece of media as well as the accumulated metadata) that frame them and construct their meaning. When separated from language, what other possibilities exist for understanding the meaning of sounds and images from colonial settings in different ways, from different perspectives, and for different purposes?
Archival technologies
Incorporating such critiques about
image, sound, and the coloniality/centrality of language into modes of
providing archival access does not require reinventing the wheel: currently
ongoing projects in archival technology echo these themes and suggest
possibilities for imagining a less language-centric digital archival interface.
This critique is not new in the archives world: in 2015, at the EuropeanaTech
Conference in Paris, digital designer George Oates made a plea for the end of
the text-base search box as the primary point of access to cultural heritage
archives and databases. Oates's critique was not about coloniality per se, but
rather a more general argument that text-based item-level searches quickly lead
to a loss of context and understanding of the value of items in heritage
collections—value that is shaped by the relationships between individual items
and collections with particular histories. Similarly, in 2013, Mitchell
Whitelaw suggested the concept of the "generous interface" ,
which he presented and expanded on at EuropeanaTech 2018. In contrast to the
search bar where the user is required to come up with their own terms that will
hopefully yield the information they are seeking, the generous interface starts
with collections overviews, allows visual browsing and different entry points
into collections, and encourages user exploration, potentially leading to
unexpected encounters with collections materials of interest.
There are also currently developing possibilities for image- or sound-based, rather than text-based, searching of collections. For example, technologies such as neural networks for visual search and audio fingerprinting for audio search may yield new options for the colonial archive. Using neural networks for automatic visual analysis, Arnold and Tilton (2019) and Wevers and Smits (2020) demonstrate modes of searching and analyzing photo and audiovisual collections that query images or use one image to return other similar images. In the colonial media archive, this could be a way of searching particular images of people, objects, or nature that are depicted or framed in certain ways, but are not necessarily catalogued as such. For example, for many media depictions of Dutch colonial life in the East Indies, metadata might identify who the white Dutch people in the scene are, but not the Indonesian people who are often relegated to the background in image and in history. Neural networks could provide a way of searching for particular faces or types of faces that were also presences in (and could provide alternative perspectives on) colonial history or family heritage. Similarly, audio fingerprinting, currently being developed for archival use at NISV, could help to trace uses and reuses of particular music or sounds as backgrounds for colonial media such as news reports. Background music was important in Dutch news reports about colonial events (such as those of Polygoon news journals) in informing Dutch publics not just about what was going on in the Dutch colonies, but also how to feel about it (for example, musical soundtracks that suggested Dutch heroism in situations that we now view as acts of colonial violence and oppression).
In summary, after identifying problems with the centrality and coloniality of language in the media archive, we can turn to artistic interventions that suggest the potential value of separating sound, image, and language; and to recent technological developments that may provide opportunities for bringing these critiques to bear on modes of archival access. In the context of the PICCH project, what does this all mean for our goal of promoting polyvocality in the colonial media archive? I argue that by "un-framing" sounds and images by detaching them from the entry point of language, the archive can promote different modes of discovery, encounter, and reframing for and by a diversity of potential users with scholarly, artistic, and/or social activist goals. The "voice" to which "polyvocality" refers does not necessarily have to speak using language: new ways of assembling images and sounds of the past can also provide a way of giving voice to different perspectives on history and on the present world.
Works cited
- Arnold, T. & Tilton, L. (2019). Distant viewing: analyzing large visual corpora. Digital Scholarship in the Humanities, 34(1). i3–i16, https://doi.org/10.1093/llc/fqz013
-
Wevers, M. & Smits, T. (2020).
The visual digital turn: Using neural networks to study historical images. Digital Scholarship in the Humanities, 35(1). i3–i16,
https://doi.org/10.1093/llc/fqy085