The Three Dimensions of Book Evolution in

The Three Dimensions of Book Evolution in Ubiquitous Computing Age: Digitalization, Augmentation, and Hypermediation Arum Park and Kyoung Jun Lee

Federico Casalegno

School of Business and Management Research Institute Kyung Hee University Seoul, Korea [email protected], [email protected]

Mobile Experience Laboratory Massachusetts Institute of Technology Cambridge, USA [email protected]

Information technology has changed the forms of publications and the way of producing, distributing, and consuming those (Park & Lee 2009). For example, the invention of typography, a new information technology at that time, provided the first uniformly repeatable commodity, the first assembly line, and the first mass production as the way of producing books (McLuhan 1962: 124 p). In addition, the printing technology broke the monopoly of library (McLuhan 1962: 206-207p) – the way of distributing and consuming books.

Merriam-Webster Online as ‘a book composed in or converted to digital format for display on a computer screen or handheld device.’ The recent development in e-book is the emergence of e-book hardware and services integrated as a product service system. Like the product service system i-tunes.com coupled with iPod of Apple in music industry, e-book industries are competing with Amazon.com’s Kindle, Barnes & Noble’s Nook, Apple’s iPad with iBook. Amazon’s Kindle is a vertical integration model which combines content (‘Kindle Store’), device (‘Kindle), and platform, which benchmarked the success model of Apple’s iPod/iTunes model. Kindle’s business model changed the way of distributing and consuming the publications. Kindle makes every activity related to human readers online. They buy the device ‘Kindle’ from Amazon.com and buy e-books from Amazon through Kindle or PC, and read the books on Kindle. The ordered e-books are delivered to Kindle through Amazon’s Whispernet using Sprint EVDO network. The readers do not pay the air fee. Since 1998, Amazon created a new way of buying books from consumer’s view and distributing books from publisher’s view with its Web site Amazon.com. Amazon also provides the Kindle selfpublishing solution, Digital Text Platform, which allows authors to publish their own content and share the revenue.

The information and communication technologies have the new terms such as e-book and m-book emerged, therefore now the term ‘publication’ does not mean just a paper book. The emergence of Kindle by Amazon.com, Nook by Barnes & Noble, and iPad by Apple have accelerated the evolution of books by information technology. However, the authors of this paper have found that the evolution of books by information technology is not just one dimension, which is towards e-book. It has been noticed that academia and industries have exerted their efforts to other dimensions of book evolutions. This paper analyzes the three dimensions of book evolution from information technology perspective.

The transformation of the paper book into digital e-book gives much benefit (Pastore 2008). However, the limitation of e-book is that most e-books only transform the twodimensional pages of the paper book into the two-dimensional electronic (or digital) book. There is no dimension added to ebook. Actually, e-book reduces the hundreds of pages of a book into a one-page screen, which causes human readers to feel inconvenient and unnatural. There have been researches on the attempt to build 3D model of e-books for more natural and convenient reading (Chu et al. 2004; Hong, Chi & Card 2005; Almeida et al. 2006), but there have been not yet 3D e-books emerging in industry.

Abstract — This paper suggests a framework for the book evolution in ubiquitous computing age. Based on the review of the academic researches and industrial development, we suggest the three dimensions of the evolution of books: digitalization, augmentation, and hypermediation. Based on the framework, we suggest three research and business development directions: 1) Open Hypermediation, 2) e-Book Augmentation, and 3) MobileDevice Based Augmentation. Keywords - Augmented Book; ubiquitous computing; Business Model

I.

II.

e-book;

hypermediation;

INTRODUCTION

DIGITALIZATION OF BOOKS: THE FIRST DIMENSION

The mainstream dimension of book evolution has been digitalization, that is, evolution of paper books to e-book since Project Gutenberg found in 1971. E-book is defined by

Human readers will not feel comfortable in reading the ebook the same as they read paper books, until they have an ebook reader which is composed of hundreds of thin physical computer screens and lightweight as paper books. Since such an e-book reader is technically or economically infeasible, e-

book readers will not replace paper books, which means that ebooks and paper books will coexist for a considerable amount of time. III.

AUGMENTATION OF BOOKS: THE SECOND DIMENSION

The second dimension of book evolution is augmentation: the evolution of paper books to augmented books. While the current e-book is digitalizing the current two-dimensional printed book into two-dimensional computer screen, the “augmented book” is augmenting a book experience by adding some dimensions or functionality to the traditional book or electronic book. Augmented book is an instance of so-called Augmented Reality. Augmented reality (AR) is a term for a live direct or indirect view of a physical real-world environment whose elements are merged with (or augmented by) virtual computer-generated imagery - creating a mixed reality. Augmented Reality is also defined as what combines real and virtual interactively in real time into three-dimensional (3D) display (Azuma 1997). Similar to the definition of Augmented Reality, many researches on augmented books are augmenting the paper books into 3D screens (Yang et al. 2008; Dünser & Hornecker 2007a; Grasset, Dünser & Billinghurst 2008a; Ucelli et al. 2005; Park and Woo 2010; Farbiz et al. 2005). There also exists a research and a commercial augmented book which produces sounds rather than or as well as 3D screen (Gotzen & Rocchesso 2006; Maeil Business Newspaper 2007). The company Semtoyou, which produces sensor modules and systems in Korea, developed an augmented book system which consists of the paper books with special carbon coated pages and optical reader embedded pen (Figure 1) which has mp3 player, LCD screen, and flash memory. If the human readers touch the pen to the carbon coated book, the optical sensors in the pen are activated and the pen produces the sound corresponding to the words or statements in the book and displays the relevant texts on its own LCD screen. The pen also has the Bluetooth capability for communicating with headphones or remote display. The augmented book introduces a new way of consuming books. While reading a book, the human reader recognizes the invisible codes coated on each page with the optical pen and enjoys the relevant multimedia content.

Figure 1. Optical Reader-Embedded Pen with Speaker and LCD Display

IV.

HYPERMEDIATION: THE THIRD DIMENSION

The researches referred in the above are augmenting the books with internal link systems rather than using the standardized communication protocol such as HTTP. However, researches such as Norrie & Signer (2003) and Fujinami & Inagawa (2009) augmented books by linking with outer

systems using standard protocols. Though it has not been active in industry yet, we suggest the hypermediation, which means hyperlinking from a media to another media (Bolter & Grusin 2000), as the third dimension of book evolution. Ubiquitous computing (UC) originally means the embedding computing elements into real world, which contrasts to the concept of virtual reality (VR). UC emphasizes the construction of computer-embedded real world, i.e. u-space, while VR pursues the construction of computer-generated virtual world, i.e. the cyberspace. Books and publications are a kind of media therefore we need to foresee the future of media in ubiquitous computing age for the prospect of them. How will the media change in ubiquitous computing age? What is the core of media change? Based on the essential meaning of ubiquitous computing explained above, the ubiquitous computerization of media is not just a digitalization of existing media but the embedding of computing elements into the real world media. The digitalization of newspaper is online news. However, the ubiquitous computerization of real newspaper should be the embedding of computing elements into newspaper. If we can imagine as far as possible, the top page of newspaper will have video display and a speaker too. However, it will be almost impossible to realize in near future due to technical and economic constraints. Therefore, we expect that the real ubiquitous computerization of media will be implemented by embedding lightweight hyperlinks into the real world media rather than embedding strong computing facilities. For example, instead of embedding the computer display into the newspaper, the human readers may read an URL of a video on the newspaper with mobile devices such as mobile phones and watch the video through the screen of mobile devices. Speakers are the same. The ubiquitously computerized media provides the URL, which is machine readable and transferred to human’s mobile digital devices, as well as the human recognizable content, only which traditional media have provided. We believe that this is the unique characteristics of media in ubiquitous computing age (Lee & Ju 2007). The new media in ubiquitous computing age can be envisioned as a media that appeal to the digital systems owned by human beings as well as the human sensory systems (Lee et al. 2008; Yoon, Park, & Lee 2009). Ubiquitous society can be envisioned as a society with media-embedded product, media embedded place, and commerce-embedded media (Lee 2007). The examples include U-exhibition embedding the commercial functions into the spatial media like museums or art galleries (Jun & Lee 2008), U-photo embedding the commercial function into the existing photo media (Lee, Ju & Lee 2007), U-display embedding the commercial functions into the existing display (Yoon & Lee 2009), and U-Billboard embedding the commercial functions into the service space (Lee & Lee 2008). Using this concept, the publication in ubiquitous computing age will be new kinds of publications that provide additional information or content to human’s devices through the digital systems or recognizable codes, i.e. hyperlinks, such as RFID tag, color bar codes, and 2D bar codes, and therefore may

realize new experience of book reading and incur commercial transaction. SK Telecom’s ‘Touch-Book Store’ prototype service launched in 2007 (Telecom Korea 2007) is an attempt to augment a book for the purpose of transaction rather than just for augmented information. To use the service, mobile users install external RFID reader into their cell phones, which will be available for free in bookstores, and put the phones on the tags on book shelves. The provided information includes book summary, book review and rating. Users are able to buy and present books or save wish lists with their own mobile phones. Gatenby (2005)’s Galatea also augmented books using LED for recommending books in book store.

where anyone can create and share contents like posting a book review on a blog. For each category of e-book and augmented book, the industry examples are mapped into the stages. We have not found the industry examples in the third stage of hypermediation.

In 2009, French gaming culture quarterly magazine AMUSEMENT (www.amusement.fr) n°4 was published with an RFID tag, fixed in the middle of its page 2, connecting it to the web. Using Violet’s RFID reader ‘Mir:ror’ (Figure 2), the human readers can access the online content (e.g. game and video) available exclusively to them. AMUSEMENT acts as irrefutable proof that the internet and printed press are no longer separate entities but intertwined extensions of one another (Flatley 2009).

Figure 3. The Three Dimensions and Examples in Industry

VI.

CONCLUSTION

Based on the framework suggested in this paper, we propose the future directions for research and developments.

Figure 2. Violet’s RFID Reader ‘Mir:ror’

V.

INTEGRATIVE FRAMEWORK FOR THE BOOK EVOLUTION

Figure 3 shows the three evolution paths of books by modern information technology and the industry examples. Paper books have been evolved into e-books by digitalization and into augmented books by augmentation. The intersection of the two sections shows either the evolution of e-book into augmented e-book or the evolution of augmented book into digitalized augmented book. Figure 3 also shows the stages of hypermediation which is the third dimension of book evolution: internal/closed link, external/closed link, and external/open link. The hypermediation will evolve from internal links to external links and from closed (proprietary) links to open links. Inagawa & Fujinami (2008) illustrated an augmented book

A. Open Hypermediation One direction of e-book and augmented book evolution is open hypermediation. As of 2010, the degree of hypermediation of e-books is in very early stage. For example, Amazon’s Kindle does not allow human readers to enjoy the external links in WWW, but only give some internal links such as dictionary. The limitation in hypermediation seems to come from the Kindle’s networking capability that allows only EVDO connection rather than Wi-Fi. The iPad of Apple has both Wi-Fi and 3G connection the same as iPhone. Therefore ebook in iPad will have more hypermediating capability than Kindle. The more hypermediation in e-books will give more opportunity for e-book businesses to make revenue using advertising such as content match advertising such as Tag Match Advertising (Jun & Lee 2008) and keyword search advertising. B. e-Book Augmentation The current e-book business initiatives do not seem to yet consider the augmentation of e-book itself. However, authors expect that e-book itself will be integrated with some other devices such as mobile phone using NFC (Near Field Communication), Bluetooth or ZigBee. The fact that iPad of Apple will have Bluetooth capability supports this expectation.

The augmentation of e-book will also open new business opportunities to industry. C. Mobile-Deviced Based Augmentation The limitation of most of existing researches on augmented books lies in the requirements of specific and unpopular computer interface such as head mounted display (Yang et al 2008; Ucelli et al. 2005) , ARToolkit (Dünser & Hornecker 2007b; Grasset, Dunser & Billinghurst 2008b), bar code scanner (Tallyn et al. 2005), projector with cameras (Gupta & Jaynes 2006; Wu, Robinson & Mazalek 2008), and RF reading computer servers(Gomez 2003). Park & Lee (2008) introduced a mobile RFID-based augmented book as a publication in ubiquitous computing age. The RFID-based augmented book attaches RFID to its pages. Human users recognize the RFID tags with their own mobile phones and enjoy additional relevant information. As of 2010, RFID reader imbedded phones are not in popular use. However, the authors expect mobile phones to have some kinds of automatic identification modules, such as RFID, bar codes, or color codes etc., in themselves. ACKNOWLEDGMENT This research is done within the Green Connected Home Alliance between MIT - Massachusetts Institute of Technology and FBK - Fondazione bruno Kessler. This research is partially supported by Kyung Hee University. This research is also supported by the Ubiquitous Autonomic Computing and Network Project, the Ministry of Knowledge and Economy 21st Century Frontier R&D Program in Korea. Kyoung Jun Lee appreciates the support of Fulbright program for visiting MIT Mobile Experience Lab from Sep. 2009 to Jan. 2010. REFERENCES [1]

[2]

[3]

[4]

[5]

[6] [7]

[8]

[9]

A. Dünser and E. Hornecker, “An observational study of children interacting with an augmented storybook,” Proceedings of Edutainment’07, pp. 305-315, 2007. A. Gőtzen and D. Rocchesso, “PEEK–A–BOOK: PLAYINGWITH AN INTERACTIVE BOOK,” Proceedings of the 12th International Conference on Auditory Display, London, UK, pp. 265-267, 2006. A. Park and K. Lee, “An Integrative View on Publication in Ubiquitous Era: Definition of U-Publication and its Relationship with e-Book,” Studies of Korean Publishing Science, vol. 35, no.1, pp.105-138, 2009. A. Park and K. Lee, “Design and Evaluation of U-Publication: TagEmbedded Publication System and Business Model(in Korean),” Journal of Inteligence and Information Systems, vol. 14, no. 3, pp. 41~57, 2008. C. Wu, S. Robinson and A. Mazalek, “Turning a page on the digital annotation of physical books,” Proceedings of the Second International Conference on Tangible and Embedded Interaction, pp. 109-116, 2008. D.A.G. Gatenby, “Galatea: Personalized Interaction with Augmented Objects,” Master's thesis, Massachusette Institute of Technology, 2005. E. Hornecker and A. Dünser, “Supporting Early Literacy with Augmented Books – Experiences with an Exploratory Study,” Proceedings of GI-Jahrestagung GI-Edition -Lecture Notes in Informatics (LNI), Bonner Kollen Verlag in press, pp. 555-559, 2007. E. J. Yoon, A. Park and K. J. Lee, “Design of Ubiquitous Sound Service Business Model as a Commerce-Embedded Media,” Proceedings of the 11th International Conference on Electronic Commerce, pp. 296-301, 2009. E. L. Eisenstein, The Printing Revolution in Early Modern Europe, Cambridge University Press, 1983.

[10] E. Tallyn, D. Frohlich, N. Linketscher, B. Signer and G. Adams, “Using paper to support collaboration in educational activities,” Proceedings of th 2005 conference on Computer support for collaborative learning: learning 2005: the next 10 years!, pp. 672-676, 2005. [11] F. Farbiz and A. D. Cheok, "Live three-dimensional content for augmented reality," IEEE Transactions on multimedia, vol. 7, no. 3, pp. 514 – 523, 2005. [12] G. Ucelli, G. Conti, R. D. Amicis and R. Servidio, “Learning Using Augmented Reality Technology: Multiple Means of Interaction for Teaching Children the Theory of Colours,” INTETAIN 2005, LNAI, vol. 3814, pp. 193-202, 2005. [13] H. Gomez, “Parent and child reading, designing for an interactive, dimensional reading experience,” In Proceedings of Technology for Interactive Digital Storytelling and Entertainment, 2003. [14] H. S. Lee and K. J. Lee, “U-Commerce in Service Space: Business Model Analysis and Case Study,” Journal of Intelligence and Information Systems, vol. 14, no. 2, pp. 45-62, 2008. (in Korean). [15] H. Yang, K. Cho, J. Soh, J. Jung and J. Lee, “Hybrid visual tracking for augmented books,” LNCS, vol. 5309, 161–166, 2008. [16] J. Jun and K. J Lee, “Design and Analysis of Business Model using Mobile RFID in the Exhibition Space and its Cases,” Journal of Intelligence and Information Systems, vol. 14, no. 4, pp. 47-68, 2008.(in Korean). [17] J. Jun and K. J. Lee, “Design of Tag Match Advertising System and the Evaluation of the Business Model,” IEEE International Workshop on Ubiquitous Network Computing 2008 (UNC'08), pp. 1527-1532, December 2008. [18] J. L. Flatley, Print magazine + RFID = hyperbole, http://www.engadget.com/2009/03/23/print-magazine-rfid-hyperbole/, Mar. 2009. [19] J. Park and W. Woo, “Multi-layer Based Authoring Tool for Digilog Book,” Lecture Notes in Computer Science, vol. 5709, pp. 234-239, 2010. [20] J.D. Bolter and R. Grusin, Remediation: Understanding New Media, MIT Press, 2000. [21] K. Fujinami and N. Inagawa, “Page-flipping detection and information presentaion for implicit interaction with a book,” International Journal of Multimedia and Ubiquitous Engineering, vol. 4, no. 3, pp. 93-111, 2009. [22] K. J. Lee, “Web 2.0 from the Perspective of Business Model,” Communications of the Korea Information Science Society, vol. 25, no. 10, pp. 16-22, 2007. (in Korean). [23] K. J. Lee, H. S. Lee and J. J et al. “A Demonstration of U-Commerce & U-Media with Mobile RFID and ZigBee,” UbiComp, 2008. [24] K. J. Lee, J. Ju and K.J. Lee, “Ubiquitous Business Model by Seamless Integration of Photo and Location Information: UPhoto”, Proceedings of 2007 Fall Korean Operations Research and Management Science Society (KORMS), pp. 440-447, 2007. (in Korean). [25] K. Lee and J. Ju, “Ubiquitous Commerce Business Models Based on Ubiquitous Media,” Proceedings of the 10th International Conference on Business Information Systems, pp. 510-521, April 2007. [26] L. Hong, E. H. Chi and S. K. Card, “Annotating 3D electronic books,” CHI '05 extended abstracts on Human factors in computing systems, pp. 1463 – 1466, 2005. [27] M. McLuhan, The Gutenberg galaxy: the making of typographic man, University of Toronto Press, 1962. [28] M. Pastore, 30 Benefits of Ebooks, Epublishers Weekly, 2008. [29] M.C. Norrie and B. Signer, “Switching Over to Paper: A New Web Channel,” Proceedings of the Fourth International Conference on Web Information Systems Engineering, 2003. [30] Maeil Business Newspaper, Semtoyou: A pen reading fairy tales (in Korean), July 7, 2007. [31] N. Inagawa and K. Fujinami, “Making Reading Experience Rich with Augmented Book Cover and Bookmark,” Computer-Human Interaction: 8th Asia-Pacific Conference, Seoul, South Korea, pp. 157-166, 2008.

[32] R. Almeida, P. Cubaud and J. Dupire, et al, “Experiments Towards 3D Immersive Interaction for Digital Libraries,” Lecture Notes in Computer Science, vol. 3942, pp. 1348-1357, 2006. [33] R. Grasset, A. Dünser and M. Billinghurst “The Design of a MixedReality Book: Is It Still a Real Book?,” Proceedings of the 7th IEEE/ACM International Symposium on Mixed and Augmented Reality, pp. 99-102, 2008. [34] R. Grasset, A. Dunser and M. Billinghurst, "Edutainment with a mixed reality book: a visually augmented illustrative childrens' book," Advances in Computer Entertainment Technology, vol. 352, pp. 292-295, 2008. [35] R. T. Azuma, “A Survey of Augmented Reality,” Presence: Teleoperators and Virtual Environments, vol. 4, pp. 355–385, August 1997. [36] S. Gupta and C. Jaynes, “The universal media book: tracking and augmenting moving surfaces with projected information”, Proceedings of the 5th IEEE and ACM International Symposium on Mixed and Augmented Reality, pp. 177-180, 2006. [37] Telecom Korea, SK Telecom Starts Mobile RFID Bibliography Service, Jan. 18, 2007. [38] Y. Chu, D. Bainbridge, M. Jones and I. Witten, “Realistic books: a bizarre homage to an obsolete medium?,” JCDL’04. ACM Press, pp. 7886, 2004.