Introduction

CLARIAH’s ‘Narrativizing Disruption’ (NarDis) research pilot investigates how (digital) humanities researchers who use audio-visual materials in their research practices can benefit from digital tools that help them better understand how media representations construct meaning. The project’s premise is that media researchers who analyse audio-visual materials and mediated events often reconstruct narratives, and are therefore storytellers themselves.1 The pilot delves deeper into the relationship between scholarly search and storytelling – by concentrating on how, specifically, the Media Suite’s exploratory search tools can help to understand how ‘disruptive’ media events are constructed as narratives across media, and instilled with specific cultural-political meanings. Disruptive media events are by definition unexpected; these events are shocking and often unexpected, such as natural disasters or terrorist attacks.2 Disruptive events are difficult to narrativise due to their haphazard unfolding, with reporting often taking place across different media platforms simultaneously. Researchers aiming to analyse the representation of disruptive events would thus benefit from tools that offer explorations of different perspectives on such events.3

The project uses a case study approach in order to understand how exploratory search and narrative formation relate by investigating how exploratory search tools can support media researchers’ interpretations of ‘disruptive’ media events as narratives.4 To make this more concrete, NarDis investigates media researchers’ use of exploratory search tools and narrative formation around two specific disruptive events, namely the case studies of (1) the assassination of Dutch politician Fortuyn in 2002 and (2) the North Sea Flood of 1953.

This project approaches this question by testing the Media Suite’s (exploratory) search tools, namely the Comparative Search recipe and the Explore recipe (DIVE+) (described below), and examines how media researchers use and create narratives to understand disruptive media events. In the Media Suite, the term ‘recipe’ refers to each of the tools that researchers can use to engage with the collections that are accessible through the Media Suite.5 Exploratory search ‘describes an information-seeking problem context that is open-ended, persistent, and multifaceted, and information-seeking processes that are opportunistic, iterative, and multitactical.’6 In other words, this type of searching is about exploring search results when the person who is browsing is investigating a topic without a clearly formulated research question. Indeed, exploring a topic could support the development of a research question. Exploratory search tools therefore aim to support complex problem solving by aligning search tools with more iterative (human) processes of learning. Instead of providing direct information retrieval, exploratory search tools support human curiosity, creativity and experienced serendipity, by offering search results that are indirectly related to a search query (e.g. not presenting a direct answer to the query ‘date Fortuyn assassination’, but for instance offering search results about what else happened on that date).

The principal investigators (Sabrina Sauer and Berber Hagedoorn) focused on two specific Media Suite recipes in this research project. First, the linked-data digital cultural collection browser DIVE+ – the Explore recipe – is a recipe that ‘provides access to heritage objects from heterogeneous collections, using historical events and narratives as context for searching, browsing and presenting the objects’.7 This recipe is based on linked data and the linked data model, which connects data from collections that are otherwise not readily connected in one searchable interface. What makes the linked data model so interesting is that it allows users to explore connections between, for example, objects in digital heritage collections that would not seem to be connected, but that are related (based on semantic similarities between their metadata descriptions and collection descriptions and thesauruses). The Explore recipe’s data is constantly enriched via crowdsourcing and other user contributions, and offers a highly dynamic and visually rich interface. At the time of the research pilot (4 December, 2017), DIVE+8 or the Explore recipe contained 1,106,844 entities, from Delpher (KB), the Amsterdam Museum, Tropenmuseum and the Netherlands Institute for Sound and Vision (Open Images collection).

Second, the Media Suite’s Comparative Search recipe is a tool for exploring radio and television programme descriptions, television subtitles and historical newspaper articles. The tool affords searches across the audio-visual collection of the Netherlands Institute for Sound and Vision (NISV) and a selection of newspapers of the Koninklijke Bibliotheek (KB, Royal Library of the Netherlands). Comparative Search (based on the previously developed tool AVResearcherXL) allows for ‘distant-reading’ exploration of collections, based on word frequencies appearing in the archival descriptive metadata and/or within the documents’ content (for instance subtitles or speech transcripts, when available). It is used for research into representation and discourses.9

In this brief project overview, we provide details of our methodology – how we used these Media Suite recipes for our research – in order to explain how the existing features and functionalities of these recipes supported this research. We do so to make explicit which challenges we encountered during the research pilot, and to identify a number of opportunities for further Media Suite development, especially for the Explore recipe.

Methodology and corpus selection

NarDis combined two methods to reach its research insights. First, the investigators defined their general exploratory research process of a disruptive event, such as the Fortuyn assassination, into generalised steps: explore contextual information about the event across archives such as Delpher, Google Scholar, YouTube, academic databases, the NISV’s audio-visual collection (using faceted search to sort results by name, date, eye-witnesses, assassin, genre of media, and by broadcaster) and by visiting archives physically. After collecting materials, ideas about the event would be refined, analysed, and written up.10

In an ideal use case, the Media Suite tools support this process by aiding with:

  1. Archive selection for research purposes; by giving researchers an overview of digitally searchable archives and providing one point of access to these different archives, researchers can save time and spend less time perusing different archives and interface.
  2. Decision making about collection selection to search, which has ramifications: are collections physically or virtually accessible, and in what material form (physical format, digitised, or digital)?
    • Collections that are accessible in the Media Suite are investigated with the Collection Inspector, which provides statistics about how complete the metadata fields are in a particular collection. This helps researchers identify which collections could be useful to explore. Multi-layered single collection search – a tool to help specify how certain topics, genres or time periods are represented in the collection – is used with the NISV collection.
  3. Searching in collections for the title of the event (e.g. ‘Assassination of Pim Fortuyn’) or implicated persons.
    • The first search step entails looking for the main media outlets (e.g. NOS news about Fortuyn’s assassination, newspapers, and first-person accounts as close to the event in time as possible, e.g. 6 May 2002)
    • The second step involves looking for discussions of the event in the media (e.g. political talk shows, more distant to the event than directly after: this way, discourses surrounding the event are collected for analysis)
  4. Corpus analysis by supporting – for instance – discourse analysis: what narratives can be discerned about the event? Ideally, Exploratory Search and Comparative Search offer support for analysis by displaying materials in such a way that the timeline of the event becomes clearer, or suggest central media objects such as eye-witness accounts, or news reports about the event. This could aid researchers in analysing narratives about disruptive events.
  5. Serendipitous insight generation : ideally, the above steps leave room for researchers to discover unexpected insights, new narratives and discourses. For example, when unexpected media objects are presented (such as radio broadcasts) that mention Fortuyn’s assassination in a different context (e.g. when discussing disaster tourism: people visiting the site of his assassination). Discovering a multitude of narratives around the event can grant insight in the multi-interpretability of past events.

This first method was combined with a second approach. As a second method, the principal investigators organised nine user sessions in which (digital) humanities researchers were invited to explore disruptive events with the Media Suite’s Explore recipe, in order to gain more user perspectives and validate research insights. During these sessions, users were presented with a search task (see Figure 1), created posters about their exploratory search processes, and discussed – in a focus group session – their use of the recipe. Combined, these user perspectives led us to insights into corpus selection and usability of the Media Suite.

Figure 1 

Exploratory Search task, user session 17 May, 2017.

Using the Explore and Comparative Search recipes: Insights on corpus selection and usability

The Explore recipe: browse and explore

Our pilot research into the benefits and challenges of using the Explore recipe indicates that the recipe allows limited exploration of narratives around disruptive media events. In this section, we discuss the benefits and challenges of using this recipe as part of our research methodology. Upon accessing the Explore recipe, users are presented with featured example searches, and the following categories that can be used to search entities: media object, actor, event, place, and concept.11 The single search box invites users to start exploring, and suggests search terms by showing users a number of popular search queries.

A clear limitation of the Explore recipe is its limited corpus; while many different collections are accessible via the recipe, these collections do not provide data about Fortuyn’s assassination. Interpreting this particular disruptive event was therefore impossible. Due to technical challenges in making more collections accessible as linked data, the collections that are currently available in the Explore recipe do not include the entire NISV collection. Searching for ‘Pim Fortuyn’, or ‘aanslag’ (assault), or ‘moordaanslag’ (murder), or ‘Media Park’ (the location of Fortuyn’s assassination) yielded no results. Therefore, one of the recommendations of our research pilot is to clarify which collections can be explored within the recipe, for instance in a tutorial or an information box. As there was no relevant material available, the researchers decided to explore alternative Media Suite recipes (the Comparative Search recipe), while simultaneously further investigating alternative disruptive events using the Explore recipe.

The North Sea Flood (‘Watersnoodramp’) of 1953 proved to be a more fruitful event to investigate with the Explore recipe. This query generated 7 events and 61 objects. However, it was unclear why search results are labelled as events or objects; radio bulletin transcripts could contain information about events, and be included as an object as well. Instead of supporting exploration, this experienced randomness of search result labelling confused researchers taking part in the pilot user study: researchers do not necessarily share the developers’ idea about what constitutes an object or event. This confusion could make the recipe less useful to researchers.

Assessing how search queries and search results match also proved challenging. Search result thumbnails (which show a small image of a search result) indicate what an object is (for example a news item, with a tag stating: ‘matches search query’), but do not indicate how the object and query match exactly. It would be helpful to show excerpts of text mentioning the exact search query when the user hovers over the thumbnail. This way, it would become possible to assess how search queries match search results.

Deeper exploration of collections using the Exploration path

One of the elements of the Explore recipe that would support narrative formation around disruptive events, is the recipe’s exploration pane (Figure 2). The exploration pane gives an overview of those search results that users have bookmarked as they explore. As these bookmarked search results are presented as a search pathway, or history, these results form a kind of ‘search narrative’ (Figure 2). One of the pilot’s recommendations was to allow users to annotate these ‘search narratives’. That way, users can describe connections between search results in order transform a ‘search narrative’, into an ‘event narrative’ (Figure 3); annotating how results connects and facilitate the interpretation of what kind of discourse about the event is constructed in each search result. Such annotation within the Explore recipe interface allows for interpretation within the recipe’s interface.

Figure 2 

DIVE+ Exploration path for the query ‘Watersnoodramp’ (North Sea Flood).

Figures 3a and 3b 

Recommendations to incrementally adapt the Explore recipe on the level of audio-visual annotation by the user (in grey).

Comparative search

Upon entering the Comparative Search recipe of the Media Suite, users can query and compare particular collections. In the case study of the assassination of Pim Fortuyn, the researchers searched for the key words ‘Pim Fortuyn’ and ‘moord’ (‘murder’). Figure 4 shows the result of this query. The Comparative Search recipe makes it immediately clear to the researchers that the collection of the NISV contains material about this case that fits the chronology of the assassination. In other words, the recipe allows the researchers to explore whether or not a collection contains materials to further investigate. It would be useful if, after having inspected this collection, the materials within this collection could be selected for further investigation using tools such as the Explore recipe. This alternative exploration of collections could trigger insights; the Explore recipe could for instance allow researchers to ‘zoom in’ on collections by looking at how different entities within the collections (such as people and objects) relate across the selected collections.

Figure 4 

Comparative Search for ‘Pim Fortuyn’ and ‘murder’ using Comparative Search recipe.

Final reflection: Challenges and opportunities of the Media Suite’s Exploratory Search recipes

The Comparative Search and Explore recipes posed certain challenges to the NarDis research pilot, particularly in terms of tool functionality and the collection accessibility. The user sessions that were organised further support these challenges.

The Comparative Search recipe provides a good indication of whether or not materials about a topic, or in our case person, can be found in selected collections. This suggests that as part of a research method, researchers could use this recipe as a first step in their Media Suite exploration process. However, access to collections is not immediately evident as it is not possible to access collections directly from the Comparative Search recipe interface. It would be useful if users could select collections and access them in a tool such as the Explore recipe. That way, the latter recipe could be used to ‘zoom in’ on collections found using the Comparative Search recipe.

It is crucial to make more (complete) collections available as linked (open) data in the Explore recipe if this tool is to be used more extensively. For instance, to understand how disruptive events are represented in media, it would be useful to include newspaper articles, news programs in video, more transcripts of radio bulletins, current events talk shows, and tv guides to indicate how a disruptive event intervened in the programmed broadcasting flow. Apart from issues with collection accessibility, the user tasks and focus group sessions in our user study identified problems with users’ understanding of how the Explore recipe creates connections between search results. More transparency about how linked (open) data creates connections will help users more readily embrace the type of exploratory search supported by the recipe. Currently, this lack of transparency made researchers question how the tool shapes event representation; if one does not know what is missing, or how search results relate, it becomes impossible to contextualise what is presented by the recipe. This also threatens the recipe’s perceived usefulness as part of a scientific methodology.

A recommendation resulting from the pilot was to allow users to export the Explore recipe’s exploration paths into a user’s personal research space. The fact that this recommendation was implemented by the end of the pilot means that researchers will now be able to compare saved exploration paths, to start comparing different search and event narratives around disruptive events in one interface. Provided that more complete collections become available in the Media Suite, this could expedite research processes.

Overall, the pilot results show that analysing how the Media Suite’s search recipes facilitate research into narratives around disruptive events is no easy task; one disruptive event, the Fortuyn assassination, had to be abandoned due to a lack of accessible material. Access to collections across the different (search) recipes of the Media Suite is clearly crucial. Ideally, having more access to collections across the different tools of the Media Suite allows researchers to more fully understand how tools shape interpretation, which is not only key for the humanities as a field, but also for researchers who want to investigate how narratives are shaped across audio-visual media.