Improving the User Experience with Audiovisual Content: The Project “Europeana Media”

1 Introduction

^[1]Audiovisual documents speak of the entire 20th century—its fictions and realities, its politics and high culture as well as citizens’ daily lives. Increasingly so, throughout the 20th and 21st centuries, audiovisual documents have become the major source for entertainment, information, and creativity. Over the past decades, memory organisations with audiovisual collections around the globe have been making the transformation to a fully digital working environment. Analogue collections (film and videotape) are, or will be, digitised and workflows adapted to manage born-digital content. Today, audiovisual content is made available online for the various user groups that collection owners (archives and libraries as well as media outlets) cater to.

The impact and popularity of video is also reflected in the collections accessible via Europeana, made available via dedicated projects and domain aggregators, such as the European Film Gateway (EFG), EUscreen and Europeana Sounds or via singular partnerships with audiovisual archives, national aggregators and as part of thematic aggregators such as Europeana Fashion. The quantity of audiovisual content on Europeana has massively increased with projects such as EUscreenXL, contributing 1,000,000 audiovisual items, Europeana Sounds, which added 735,000 audio objects, and European Film Gateway, which contributed more than 650,000 pieces of film-related content to Europeana. Europeana has grown to become one of the largest and most diverse inventories of audiovisual heritage in the world. Today, these EU funded projects have evolved into expert hubs and continue to attract more interest from AV collection owners to contribute to Europeana. On Europeana, the popularity of audiovisual content remains uncontested—reports have consistently quoted a greater interest for audiovisual items from Europeana visitors than for other kinds of material.^[2]

Over the past several years improvements have been made to the presentation of images and text on Europeana through the integration and implementation of the International Image Interoperability Framework^[3] (IIIF) Presentation API. However, the Europeana Core Service Platform lacks structural support for video items on Europeana Collections. The Task Force “Audiovisual Media in Europeana” that operated with support from the Europeana Network published its final report in June 2017.^[4] It identified three main shortcomings in the user experience that hinders AV material from reaching its full potential:

1. No unified playout of videos (different content providers use different media players)

2. Lack of clear and intuitive information about potential reuse options for AV content

3. Playout on Europeana offers no enhanced functionalities to engage with AV content

Our project therefore aimed to unlock the potential of audiovisual (AV) media in Europeana by increasing the appeal, accessibility, re-use, and interaction of Europe’s AV heritage available through Europeana Collections and 3rd party platforms that use Europeana content.

As a result of the project an enhanced unified playout service (EUPS) for time-based media integrated into the Europeana Core Service Platform was developed. This service offers an interoperable and sustainable solution for audiovisual media playout on Europeana, providing ease of use, intuitive design and functionalities tailored specifically to the Europeana’s key target groups. In particular, it delivers a suite of functionalities that offers researchers and educators the possibility to better incorporate audiovisual content from Europeana in their working environment. Providing, for example, video fragment quoting and embedding as well as support for subtitles.

The project Europeana Media was co-financed by the Connection Europe facility of the European Union and started in September 2018 and ended in February 2020. It was coordinated by Istituto Luce-Cinecittà.^[5] Besides the Europeana Foundation,^[6] other partners were TIB,^[7]NISV,^[8] ATIT^[9] andNoterik.^[10]

2 Method

Successful products and services are always geared to the needs and requirements of their users. Ideally, they solve an existing problem and offer the user noticeable added value. This also applies to Europeana Media. In this project we worked according to the user-centred design approach. With this approach, the future users of the unified playout service with their tasks, goals and characteristics were placed at the centre of the development process for all design decisions. The principles of dialogue design (ISO 9241-110^[11]) in terms of good usability (ISO 9241-11) and user experience (UX; according to ISO 9241-210) were consistently applied.

First we analysed the user needs and requirements, developed a requirements catalogue^[12] and user stories for the different target groups. The requirements were prioritized and implemented into the first prototype. A first usability test^[13] with participants from the relevant target groups was carried out in order to capture the real interaction of users with the player: Do users find the features they are looking for? Are there problems with certain workflows? Which terms cause difficulties? Based on the results of the usability test, the player prototype was revised, tested again in a second usability test^[14] and finally implemented and rolled out.

Fig. 1

User-Centred Design approach pursued by Europeana Media^[15]

2.1 User Requirements^[15]

The aim of the Europeana Enhanced Unified Playout Service (EUPS) is to deliver functionalities to better access and incorporate AV content from Europeana. In a first stage, the partners specifically researched what functionalities users in general (the public at large), as well as those more specifically targeted by the project (the education and research communities) expect from such a Universal Player beyond those functionalities that have become common to media users.

Work on the extraction of user requirements started with the composition of a first version of user stories, based on interviews with users from research and education and on the user needs analysis that was carried out in past research and development projects. A long list of functionalities was extracted from the user stories and reviewed by all partners. This document describes player functions from basic (play, pause, fast forward, rewind and stop, “player head”) to additional functionalities (full screen video view, picture quality control, closed captioning or subtitles) and even specialist functionalities such as a media library, playlists and indexing, tagging, lyric or artwork discovery, basic recording, editing and sharing, time stretching, chapterisation, bookmarking, auto-resume, etc. The final version of the long list retained only functionalities for players and interface that were considered technically achievable and in line with Europeana’s mission and typical strategic and policy constraints (for example regarding IPR).

On the basis of the functionalities list, a user questionnaire was prepared; this was used for in-depth interviews and surveys between 15 October and 9 November 2018. The surveys and interviews involved a representative sample of users from the three target groups (education, research and general public) regarding their typical usage behaviour, tasks, needs and expectations related to the functionalities of the EUPS. These interviews were carried out via Skype in the first week of November 2018 and were recorded for quantitative and qualitative analysis. The in-depth interviews on average lasted 67 minutes and involved altogether 17 users.

This work resulted in a EUPS Requirements Catalogue,^[16] which gave the Europeana Media project team a better understanding of how and why users wanted these functionalities. This list was not to be considered the ultimate wish list of functional specifications, but rather a realistic indicator for the Europeana Media consortium to understand how users work with video, what they would like to do with video resources and how important functionality can be for them. This catalogue helped partners in selecting and prioritising the most relevant design elements and functionalities of the EUPS. The comprehensive functionalities list is provided in Figure 2.

Fig. 2

Survey results-priorities according to survey sample

The results of the user survey largely confirmed the initial functionalities list that was extracted from the first version of the user stories. This common pattern of use of online media resources has evolved from the experience with the existing available media resources such as Windows Media Player, QuickTime, iTunes, VLC media player, and media platforms such as YouTube and Vimeo. Besides the standard functionalities that users such as educators and researchers adopt in their daily practice (play, pause, scrub, and full screen), educators and researchers also want to use media in the same way as they use other content. With such content they are adopting web 2.0 functionalities, allowing for much greater interaction, better usability, options for sharing and commenting, etc. Functionalities such as adding, annotating and commenting, downloading or embedding, subtitling, creating and sharing playlists are important for users in education and research. Professional users in our target groups also expressed their interest in functionalities to manage citations (especially for researchers) and get clear information about the rights for use or re-use.

Not all functionalities that were expressed in the analysis were selected to go through to the development phase: functionalities such as “in text search” are not a function of the player itself. Still these results were considered valuable user feedback to Europeana. Other functionalities were put on the long term evolution path because time and resources would not allow for all functions to be implemented (increase/decrease speed, jump x seconds forward/backwards). Finally, some functionalities for the time being could not be implemented because they require structural changes to the way the Europeana platform communicates with its users: under this last category we find functionalities that essentially require the use of personal accounts on Europeana, for example storing, recalling and sharing personalised information such as annotations, tags, etc.. Again they are not an immediate function of the player but they are closely related to the user experience and therefore retained as valuable feedback for the Europeana Foundation for future developments.

The Requirements Catalogue was officially adopted by the consortium as the basis for the development roadmap in order to decide about the prioritisation of the functionalities and as input for the design of the evaluation and usability testing.

2.2 Usability testing

The Europeana Media project usability evaluation was conducted in two iterations. In the first evaluation, which took place in June 2019 in Leuven/Belgium, 14 participants comprising 5 educators, 4 researchers and 5 general public EU citizens were recruited through purposive sampling. The test was conducted using hybrid techniques, which include, online questionnaire, observation, think aloud, tasks, and post tasks questionnaires as well as post-test interviews. All participants were presented the first clickable version of the EM wireframe and a number of scenario

Fig. 3

Usability Test Setting

tasks based on the use cases defined in the requirement analysis. The objectives^[17] of the test were to:

1. Determine design inconsistencies and usability problem areas within the user interface and content areas of the unified player and item page. Potential sources of error may include:

   1. Navigation errors – failure to locate functions, excessive keystrokes to complete a function, failure to follow recommended screen flow.

   2. Presentation errors – failure to locate and properly act upon desired information in screens, selection errors due to labelling ambiguities.

   3. Control usage problems – improper toolbar or entry field usage.

2. Testing of the web application under controlled test conditions with representative users. Data is used to access whether usability goals regarding an effective, efficient, and well received player interface have been achieved.

3. Establishment of baseline user performance and user satisfaction levels of the player interface for future usability evaluation.

The results of the first evaluation were used to redesign the Europeana media player and editor. These were then reviewed by the team, and the final version of the wireframe was created and further reviewed. Finally, the visual design of the playout system was created. The second usability test was conducted in December 2019 in Hannover with 12 new participants, consisting of 4 researchers, 4 educators and 4 EU citizens from the general public. The general goal of the second test was to assess the changes resulting from the results of the first evaluation on the whole and to enrich them with objectively measured data. The objectives and the general test protocols were the same as for the first evaluation. Finally, a summative evaluation of the two tests was conducted. This showed that the results of the three areas Average Task Completion Rate, the Average Task Completion Time and the system usability satisfaction have improved significantly (see Table 1).

Table 1

Results of the summative evaluation

Key Metrics	First Evaluation	Second Evaluation
Average Task Completion Rate	78 %	92 %
Average Task Completion Time	178.22 secs	159.80 secs
System Usability Satisfaction	57 (OK)	85 (Excellent)

3 What’s in it for users?

As mentioned before, the objective of the Europeana Media project is to offer heritage institutions an enhanced media playout on Europeana with a user friendly design and that could foster audiovisual content reusability. The developed Enhanced Unified Playout Service (EUPS) unlocks the potential of audiovisual media in Europeana by increasing the appeal, visibility, reuse, research and interaction with Europe’s AV heritage available directly through Europeana Collections and 3rd party platforms that use Europeana content.

This unified media player has been integrated in the Europeana Collections portal and delivers a suite of functionalities that will give researchers, educators and citizens the possibility to better access and incorporate AV content from Europeana into their working environments and practices.

In particular, the EUPS will offer users the possibility to

1. Embed and share audiovisual content in third-party platforms (e.g. Wordpress,^[18] Moodle,^[19] Drupal^[20] etc.) and on social media,

2. Display available subtitles in multiple languages,

3. Annotate and share videos or fragments with time-aligned annotations or user generated subtitles,

4. Create and share playlists of selected audiovisual items.

The version of the EUPS integrated in Europeana supports at its launch the first two functionalities listed above, offering to all users of the Europeana Collections portal the possibility to embed audiovisual content from the portal on educational platforms like Moodle or MediaWiki^[21] or on CMS platforms like Wordpress and Drupal (and in general on any third-party platform that supports iFrame or oEmbed standards), this serves to stimulate and foster the reuse of Europeana content in different contexts, from scientific online journals, to MOOCs or users’ blogs, raising the visibility and awareness about the Europeana initiative and its potential, especially -but not only- in the fields of education and research.

The full set of functionalities of the EUPS, including user-generated annotations and subtitles as well as playlists creation, are available on the version deployed on the EUscreen portal^[22] (the portal of the Europeana domain aggregator for audiovisual heritage) that supports user management and registration. The full version of the EUPS will also be deployed on the Europeana Collections portal once the website will support user management and when a general policy for the display and management of user generated content (like annotations, tags, etc.) has been defined by the Europeana Foundation.

The new unified player handles and displays by default all the audiovisual formats supported by the HTML5 specifications, with the exclusion of the content streamed by closed platforms, like YouTube or Vimeo, because of limitations derived from their terms of service. A special attention has been given to display copyright information. In fact, videos or just snippets can now be embedded in other environments such as blogs or learning environments, and the embedded content will always display the source of the material, the material’s rights statement and a link back to the source record (e.g. onEuscreen.eu^[23] orEuropeana.eu).

3.1 Annotations

Through the annotation functionality, users are able to create time-stamped annotations in the audio or video files. This could be used by presenters to create points of illustration, or by teachers who could add bookmarks at which to pause and discuss the content in a classroom. In the screenshot below the editing of a note about the program at a specific point in the video is shown (see Fig. 4).

Fig. 4

Annotation functionality of the EUPS

3.2 Playlist

The playlist functionality allows users to create a playlist of selected videos or snippets. Users are able to create named playlists and populate the list with videos from their bookmarks. Once again this may benefit educators who can create playlists of material relevant to the topic of a lesson. In the screenshot below we see an example of a playlist on Italian history containing two videos. The user is able to add further videos, edit the playlist, preview it, or embed it in third-party platforms or just share it (see fig. 5).

Fig. 5

Playlist functionality of EUPS

3.3 Subtitles

Subtitles or captions are a specific type of annotation which provide a translation or transcription of the speech content in the video. Below is a screenshot that shows the addition of subtitles in a video. Users can generate subtitles in multiple languages for the same video that then can be displayed upon request (see Fig. 6).

Fig. 6

Subtitle functionality of EUPS

Moreover, the EUPS interface has been customised by the Europeana UX Team to be nicely embedded in the Europeana Collections portal item page, and users of the portal, when displaying an audiovisual item on it, will be present an item page that is directly embedding the EUPS as default player and where users can enjoy the item (play/pause, play full screen, switch subtitles, etc.) and eventually embed it on third-party platforms or social media (via link, iFrame or oEmbed code). In this way the user experience of consuming audiovisual content on Europeana will be more uniform and streamlined and reuse and share of Europeana AV content, especially in education and research, will be fostered.

4 Conclusion

As mentioned above, the Europeana platform does not currently provide any features that require a user profile, such as video playlists and annotations. However this is on the Europeana.eu Roadmap. Europeana is currently adding support for further annotations in the form of IIIF-manifests for other media formats such as text and images. This has being done by integrating code from the Mirador^[24] player for these use cases.

Currently, also only the functionalities permitted by content providers are supported on the Europeana portal. In order to allow users to fully benefit from the new EUPS, the project team is continuing to reach out to content providers in order to obtain permission to offer users the new functionality. This has brought up new challenges, as content providers sometimes prefer to select a subset of functionality for certain datasets.

Regarding the usability of the EUPS, we have shown that it has improved substantially during the course of the project by applying the method of user-centred design. The EUPS now provides users with a uniform experience throughout the portal for almost all of the content providers on the platform. Because the player’s codebase is open-source, improvements can be made and initiated from the Europeana community. This can also be useful for debugging issues specific to content providers.

Artificial Intelligence (AI) might play an interesting role in enhancing video content in the future. AI could manifest added value by providing conversational interfaces like Siri to enhance search directly inside the player as well as enabling automatic audio transcription. Other possibilities included enhanced timeline and the possibilities of tracking and display in the player multiple tracks simultaneously, like subtitles/captions together with other annotations (e.g. contextual info or tags). Future Work includes benchmarking of available AI services like speech-to-text and visual analysis as well as services that enable location based search and display. In order to facilitate motion analysis, the possibility of providing slow play functions/frame by frame play could also be considered.