Audio games: investigation of the potential through prototype development Jarosław Beksa
Sonia Fizek
Philip Carter
Auckland University of Technology Gamification Lab Auckland University of Technology Computer and Mathematical Centre for Digital Cultures Computer and Mathematical Sciences Leuphana University Sciences Auckland, New Zealand Lüneburg, Germany Auckland, New Zealand +64 27 318 6550 +64 9 921 9999 x 5300
[email protected]
[email protected]
ABSTRACT The goal of the described projects was to verify the potential of the audio games design concept. The following introductory questions were set at the initial stages. Is it possible to create immersive and playable computer games based on sound only? May such games be controlled using the touch screen? Will they be attractive for sighted players? And finally, can audio games become a new commercially successful trend in the electronic entertainment?
[email protected]
1. INTRODUCTION 1.1 Audio Games Audio games exist only in the world of sounds. Contrary to popular video games, they do not rely on visuals (Collins 2013) – the sonic layer becomes the basis for gameplay mechanics and interface design. The game world is created in the player’s imagination based on the omnipresent soundscape.
This article gives an overview of the development of three genres of audio games: a story-driven role-playing game (RPG), an interactive audiobook with RPG elements, and a set of casual sound-based games. The first two projects have been completed and evaluated in the commercial setting. This allowed for the evaluation of their marketability potential. The last one is a work-in-progress at its final stage of completion.
Audio games originally started as an entertainment tailored for visually impaired and were developed mostly by amateurs, game accessibility researchers, and occasionally by visually impaired programmers. One of the first commercial games of that genre was Real Sound – Kaze No Regret (1999), an audio adventure game created for Sega Dreamcast and Sega Saturn. Recently, an increasing interest in audio games has been noticed among sound artists and game developers.1
Categories and Subject Descriptors
1.2 Project goals
H.5.2 [User Interfaces]: Auditory feedback, Input devices and strategies, Interactions styles, User-centered design; K.8.0 [General]: Games.
General Terms
Our main goal was to verify the audio games design concept. To achieve that we set the following questions:
Design, Management
Keywords
Audio game, interactive audiobook, gesture based audio user interface.
1
More audio games may be found on the AudioGames.net website and in the “Sonic Mechanics: Audio as Gameplay” article by Aaron Oldeburg (2013).
Is it possible to create computer games based on sounds only? Can audio games be equally or more involving than video games? Can audio games be played successfully on touch screen devices? Will such games be attractive for sighted players? Can audio games become a new successful trend in the electronic entertainment?
2. PROTOTYPES AND USER TESTS 2.1 Adventures of Jolan The first audio game prototype was developed for the PC Windows platform. It was based on the Last Crusade – an audio game engine developed at the University of North Carolina by Patrick Dwyer & Peter S. Van Lund. Adventures of Jolan is an audio computer role-playing game (RPG) enriched with a non-linear story and a simple character development system. To enhance the player’s immersion in the story world, we implemented a range of voice recordings, sound effects and background music. The prototype did not have any graphical interface. The keyboard was used as an input device.
2.2 User tests Twelve computer game players (both male and female) were invited to participate in the tests. They were divided into three focus groups: Sighted high school students Sighted college / university students Visually impaired of various education levels Participants were asked to play the game for 20 minutes and share their general opinions about it. To our surprise, the game was very well received by sighted players. Some of their opinions were as follows (Beksa and SzewczykBiedrzycka) 2: “Revolution in the gaming system”, “More relaxing than video games”, “Improves imagination” “There is no need to stare at the screen”, “Saves sight”, “Porting to a mobile platform would be a great idea”.
Sound effects Dialog recordings
3.1 Game story We acquired a non-linear story through a contest announced in December 2008 by the Polish Telecom. From over 130 submitted scripts we selected 1812: Heart of Winter written by Magdalena Reputakowska and Maciej Reputakowski. The interactive story is set in the 19th century during the times of Napoleon Bonaparte’s army invasion on Russia. Is has both, historical and fantasy elements. A non-linear plot allows the player to complete the game in numerous different ways, encouraging gamers to play more than once and enhancing the replayability factor.
3.1.1 Binaural dialog recordings All the game dialogs were recorded using a binaural technique, which uses a special microphone arrangement and is intended for replay using headphones [5]. The sounds are recorded with microphones embedded in the ear canals of an artificial mannequin head (the so called “dummy head”). When they are then listened to via headphones, there is a realistic perception of three-dimensional auditory space. The player feels as if they were thrown into the middle of the scene. Any set of headphones that provides good stereo channel isolation is sufficient in order for the recording effects to be heard. In 1812: Heart of Winter, all the actors were recorded from different angles and distances to create an even more immersive sound effect.
Test results assured us that the audio game concept is very promising and gave us further motivation to develop the first large-scale multiplatform audio game.
3. INTERACTIVE AUDIOBOOK For commercial implementation we chose an interactive audio story (interactive audiobook) with RPG elements. 3 The production required the development of the following elements:
Game story Audio game engine Audio game editor Music
2
Authors translation
3
Interactive audiobooks were also developed and discussed by Röber, N., Huber, C., Hartmann, K., Feustel, M., and Masuch, M.
Figure 1. Binaural recording of dialogs scheme
3.1.2 Music and sound effects The game’s atmosphere was enhanced by dedicated music composed by Marcin Przybyłowicz and a rich variety of sound effects. Each location in the game is defined by a unique set of ambience effects. This further enhances the feeling of immersion and gives the player the possibility to create associative cognitive maps of the in-game locations, (2006). Although belonging to the same genre, 1812: Heart of Winter was a parallel project realised with a completely new engine and interface design concept.
which “are complex mental interpretations of a real or fictional environment, and its components that live in the fictional plane” (Nitsche 2008, 161). The development of 1812: Heart of Winter was a large-scale endeavour, which involved 36 voice actors, includes over 60 minutes of dedicated music, more than 6000 sound files, and over 10 hours of game play. The game was also developed in three languages: Polish (full version), English (demo version), and French (demo version).
Developing an accessible user interface on keyboard devices (PC, Symbian and Android phones) was relatively easy. Users were able to customize buttons according to their own preferences. The real challenge was the development for touch screen devices. After many sessions with visually impaired players, we designed a gesture-based interface with corner buttons. This approach was device independent – the interface was consistent for all the devices.
3.2 Game engine The Game Engine is based on the library developed by BL Stream [6]. It is capable of displaying graphics, handling I/O and playing up to 5 ogg streams simultaneously. Currently the engine is available on four platforms: PC Windows, iOS, Android and Symbian S60. The Game Engine allows running any audio game developed in the Game Editor and is fully accessible to visually impaired players. A minimalistic graphics interface was prepared to increase the attractiveness for visual players. Also, due to the users’ suggestions, the text and audio elements were synchronized, turning the game into a combination of an e-book, audiobook and a role-playing game. Figure 3. Touchscreen user interface design
3.3 Game Editor To accelerate the process of story implementation and to manage multiple story plots, we developed an editor, which allows creating any audio game (from simple interactive audio books to advanced role playing audio games). It runs on the PC platform and uses a graphical user interface (GUI). Figure 2. Game Engine screens (from left): main menu, dialogues, inventory The Game Engine can be easily customized to any game created with the Game Editor and localized in any language.
3.2.1 User interface Creating a multiplatform audio game required a complex user interface design. Our goal was to create an engine fully accessible to visually impaired and similar for each platform. Every action performed in the game is represented by a kinesonically syncretic interactive sound, fused not to image, but to action. As Collins explains, “… interactive sound is event-driven, and the sound controlled by an action or occurrence that is initiated by the game or by the player” (2013, p. 32). We used sounds as feedback to acknowledge the events.
Figure 4. Game Editor story view
4. MARKET RESULTS 1812: Heart of Winter was released in November 2011 in Poland. The game was published by the Polish Telecom and was neither advertised nor promoted on a large scale. Despite this fact, during the past three years it has been downloaded more than 60 000 times in Poland (all platforms) and acquired excellent user reviews. An average rating on App Store was 4,5 stars out of 5. It was available free of charge for four platforms (there was a short period when the game was available at different price levels to test the sales potential). The most surprising fact was that over 95% of downloads came from sighted players (survey data from PC version downloads). Gaming web-portals and communities also valued the interactive audiobook design concept: "The first interactive audiobook in the history of Polish literature" www.masz-wybor.com.pl "Heart of winter is one of the best productions of 2011." www.dubscore.pl "A brilliant combination of an audio book with a roleplaying game” www.mackozer.pl "Heart of winter is something we’ve been waiting for” www.tyflopodcast.net
5. CASUAL AUDIO GAMES Audio-based games are also naturally adaptable to a genre focused on non-visual space exploration, where binaural audio technology is used to create 3D spatial environments for the player to traverse through. In the recent years, several games based on moving through space have been created: Papa Sangre (2010), Papa Sangre II (2013), and Blindside (2012), among others.
Audio Game Hub project is being developed at the Gamification Lab, Centre for Digital Cultures (Leuphana University Lüneburg).
5.1.1 User interface The Audio Game Hub interface uses a set of gestures and corner buttons, which allows the player to navigate through the available options and menu elements. A minimalistic graphical interface was implemented to reduce the entry barrier for sighted players. It can be disabled and enabled at any time. This feature will be used for further user tests with sighted players. The main focus was laid on audio interface. Each user action is represented by an interactive sound. Also, every mini game consists of unique sounds and voices. Numerous audio samples were recorded using the binaural method. Each game starts with an audio tutorial explaining the rules and game mechanics. We also implemented an array of background sounds to represent each of the game worlds and enhance the immersion of the player. Selected interactive sounds used in the game’s menu and in individual games have been assigned to the same actions and events. This points to another crucial aspect of interactive sounds - their repeatability, which “… establishes an expectation that we will hear the same sound as a reaction to the same action” (Collins 2013, 33). To minimize the problems with “dead” borders of touchscreen devices, we implemented a calibration step, where users can learn the physical size of their active touchscreen area. In all the games subtle audio cues are available, informing about reaching a corner button or the edge of the screen.
In our current project we are focusing on adapting multiple genres, belonging to the casual games category, into the audio sphere. A combination of simple arcades and platform games constitutes a perfect playground sample, as it allows for the evaluation of various mechanics and interface solutions in a homogeneous project. Figure 5. Audio Game Hub main menu
5.1 Audio Game Hub The Audio Game Hub (2014) is a practice-led research project and a work-in-progress. It includes a collection of several mini-games, each of which requires a different type of audio gameplay mechanics. The application is developed for mobile platforms (iOS, Android, Windows Phone) and as a desktop version (Windows, OSX). It may be operated with: touchscreen, touchpad, keyboard and mouse. The
5.1.2 The Hunt The Hunt is a shooting game, where user has to hit moving targets – forest animals. With each level of difficulty the targets move faster. The aiming system in The Hunt (see figure 6) is based on a 2D soundscape represented by stereo panorama (the “x”
axis) and pitch (the “y” axis). There are two sound sources placed within the 2D game world and available to the player:
than other players after hearing the trigger time. The game also supports one and two player modes.
The target board (A) The aiming point (B)
Figure 8. Samurai user interface scheme
5.1.5 Labyrinth
Figure 6. Game mechanics in The Hunt In each round the target sound (A) is placed along the “x” axis (heard to the left or to the right), and along the “y” axis (heard in a higher pitch when placed up, and in a lower pitch when down). The aiming point sound (B) corresponds to the finger position. When the finger moves closer to the target, the corresponding targeting sound is played more frequently (z – distance between the target and the aiming point).
Labyrinth is a classic arcade maze. The player’s goal is to find the exit by following the “guiding” sound. When getting closer to the exit, the guiding sound becomes louder. The game consists of randomly generated rooms, whose borders are drawn by sounds signifying the player bumping into a wall.
5.1.3 Animal Farm Animal Farm is an adaptation of a classic memory game. The player’s task is to find matching pairs of farm animals stored in boxes. To locate the box, the interactor needs to move a finger on the screen (looking for a wooden box sound). The box is opened by a double tap gesture, which triggers the sound of a selected animal.
Figure 9. Labyrinth game screen
5.2 User Tests During the development process (Agile method) each of the mini games and the main user interface was tested with visually impaired players in three countries: Germany, Poland and New Zealand. Gathered feedback allowed us to improve user interface and game mechanics so that they can be played by visually impaired who never had contact with touchscreen devices before.
Figure 7. Animal Farm game screen
5.1.4 Samurai Samurai is a reflex based game for up to four players. The screen is divided into four equal parts corresponding to the game space occupied by each of the players. The goal of the game is to beat other opponents by touching the screen faster
We also tested the Audio Game Hub on two groups of sighted students (5 each): Group one: play games with visual aids on, then again with visuals off Group two: play games with visual aids off, then again with visuals on First prototype test results have shown that immersion is increasing when graphics interface is turned off. Visual aids also affect sighted players’ entry barrier.
6. CONCLUSIONS Our work has demonstrated a great potential of interactive audiobooks with RPG elements and other genres of audio games. User tests and players’ reviews demonstrated that this form of entertainment may be attractive for both, sighted and visually impaired players. Currently, we are experimenting with various types of casual audio games and the most effective sound and gesture interface solutions. Since the Audio Game Hub is a project-in-progress, the testing data with sighted and visually impaired players are in the process of evaluation.
7. REFERENCES [1] Collins, K. 2013. Playing with sound: a theory of
interacting with sound and music in video games. The MIT Press. [2] Oldenburg, A. 2013. “Sonic Mechanics: Audio as
Gameplay”. In Game Studies.org, vol. 13. Issue 1. Available at: http://gamestudies.org/1301/articles/oldenburg_sonic_ mechanics [Accessed 15th Jan. 2015]. [3] Dwyer, P. & VanLund. P. S. RPG Game Engine and
Map Maker. University of North Carolina. Available at: http://www.cs.unc.edu/Research/assist/et/projects/RPG /index.html [Accessed 15th Jan. 2015].
[4] Beksa, J., Szewczyk-Biedrzycka A. 2008. Audio game
test report. Consumer Research Centre. Polish Telecom R&D. [5] Wenzel, E.M., Arruda, M., Kistler, D. J. & Wightman,
F. L. 1993. “Localization using nonindividualized head-related transfer functions.” J. Acoust. Soc. Am. 111-123. [6] BL STREAM. Available at: http://www.blstream.com
[Accessed 15th Jan. 2015]. [7] Nitsche, M. 2008. Video Game Spaces: image, play,
and structure in 3D worlds. MIT Press. [8] AudioGames. Available at:
http://www.audiogames.net/list-games [Accessed 15th Jan. 2015]. [9] Röber, N., Huber, C., Hartmann, K., Feustel, M., and
Masuch, M. 2006. “Interactive Audiobooks: Combining Narratives with Game Elements.” In Technologies for Interactive Digital Storytelling and Entertainment Lecture Notes in Computer Science. Volume 4326, pp 358-369.
