cross-platform these solutions. Even more, many ... the platform by using mobile devices, but also in order to test and to analyze usability issues in ... work in progress on developing a framework for testing and optimizing a LMS mobile web ...
Usability evaluation of a LMS mobile web interface Daniel Ivanc, Radu Vasiu, Mihai Onita Communications Department, “Politehnica” University of Timisoara Bv, V. Parvan nr. 2, Timisoara, Romania {dan.ivanc,radu.vasiu,mihai.onita}@cm.upt.ro
Abstract. The growth in the use of mobile devices in everyday life determined an increasing demand to access educational information using mobile technology. Nevertheless, most existing computer based Learning Management Systems still do not have advanced access support for mobile devices. This paper presents the basis of an in-progress research, aiming to enhance our understanding of mobile learning usability considerations and measurement. It also provides the starting point for performing a usability evaluation of the MyMobile web interface of our university’s Moodle LMS. The main contribution of this paper is our proposed approach, based on four perspectives that have to be considered when designing or testing a mobile learning solution: Pedagogical usability, Usability of the device, Usability of the content and Usability of the mobile web interface. Additionally, metrics, methods and guidelines for usability testing are given. The main goal of the paper is to provide an overview on a proposed framework for testing and optimizing a LMS mobile web interface from a usability perspective. Keywords: mobile learning, usability, metrics, mobile devices
1
Introduction
Mobile technology is recognized to be one of the most significant directions of nowadays knowledge based society concept. The growth in the use of mobile devices as mobile phones, smartphones, tablets, netbooks and notebooks in everyday life determined an increasing demand to access educational information, instruction and tools using mobile technology. Many studies have given encouraging results for using mobile technologies to support students in the teaching and learning process. Nevertheless, most existing computer based Learning Management System still do not have advanced access support for mobile devices, and there are deficiencies in cross-platform these solutions. Even more, many mobile device’s browsers do not support scripting or plug-ins, and do not have available memory to display desktop web-pages and graphics. This influences a lot the usability of mobile learning systems. Also, web content that is mostly the format of electronic learning content is not always suited for mobile browsers and the ability to display information in various multimedia formats is limited. These usability issues of mobile devices and learning
must be considered when different types of interfaces and device oriented applications are developed and tested [1]. Moodle is one of the most popular open-source e-Learning platforms. It has proven to be a serious competitor to other paid worldwide-known solutions and is usually the first choice when a low-cost, robust eLearning solution is needed. It has been used for several years in the eLearning Centre of our university and several add-ons and tools have been developed. Because very few open source mobile extensions for Moodle were available and none complied with our needs, the first option was to develop a new mobile user interface for Moodle. This was needed not only for obtaining a functional access to the platform by using mobile devices, but also in order to test and to analyze usability issues in mLearning and especially to develop a usability testing methodology for mLearning because our research revealed that there is a lack of studies on this aspect. However the release of the 2.0 version of Moodle provided a new way of developing mobile friendly interfaces and the release of MyMobile Moodle mobile-optimized interface, based on jquerymobile was considered and later adopted for providing mobile access to the eLearning platform and for continuing the research on mLearning usability testing. Starting from the question of whether there are specific methods and metrics for mLearning usability testing, the goal of this paper is to provide an overview on our work in progress on developing a framework for testing and optimizing a LMS mobile web interface from a usability perspective. It provides an introduction to mobile learning usability testing, presents several usability testing methods and metrics, selected or adapted from literature review, the setup needed for the testing, and the proposed framework for the design and implementation of usability testing of a LMS mobile web interface.
2
Mobile Learning and Usability
2.1
Defining Usability
ISO/IEC 9126-1 defines usability as “the capability of the software product to be understood, learned, used and be attractive to the user, when used under specified conditions”. This definition is primarily concerned with a software product; however, it can be applied to mobile learning software taking into consideration features specific to mobile phones and eLearning aspects. ISO 9241-11 defines usability as: “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use” and suggests the possibility of using specific metrics for measuring performances. A challenge with definitions of usability is that it is very difficult to specify what its characteristics and its attributes should be, in particular because the nature of the characteristics and required attributes depend on the context in which the product is used [2]. Three critical elements are also revealed: usability of the product relates to
specific users, the ones that it was designed for; these specific users have to follow some specified goals for the product was designed for; a specific context of use. 2.2
Mobile learning usability testing
Most of the mobile devices that are used in mobile learning were not designed for educational purpose, and usability issues continue to be the most serious restraining. Frequent approaches to usability incline to be limited to metrics relating to time taken to complete a task, effort, throughput, flexibility and the user’s satisfaction and attitude. Usability has to be considered in a different manner when it is being evaluated in the context of teaching and learning, and the concept of pedagogical usability can be helpful when considering the close relationship between usability and pedagogical design. 2.3
A four perspectives approach
The usability is reviewed as a determinant factor of success of an educational platform and it should be given special attention also when talking about accessing it on mobile devices that were not designed for educational purposes. Based on the fact that the device’s and UI usability issues influence in different ways the user experience [19], and on the fact that mobile learning is usually considered an extension to eLearning and have similar characteristics, we consider that the usability of a mLearning platform has to be regarded from four different perspectives that are presented in Table 1 and described in the following paragraphs. Table 1.
A four perspectives usability testing approach
Device Usability Pedagogical Usability 2.3.1
Web User Interface Mobile Learning Usability Usability Testing Educational Content Usability
Pedagogical usability
Pedagogical usability is the analysis of the way an educational application (tools, content, tasks and interface) supports students in their learning process within various learning contexts according to learning objectives. It should be especially concerned with educational aspects such as the learning process, purposes of learning, user’s needs, the learning experience, learning content and learning outcomes [3]. When developing or improving a mobile learning application is not sufficient to intend that people can use it, they must want to use it [4]. Some relevant pedagogical usability metrics and a description of what each of the usability metrics measures are provided in Table 2. Our research on pedagogical usability testing needs to go further and establish practical methods for evaluating the mLearning platform from the pedagogical point of view.
Table 2.
Metric Instruction Learning content relevance Learning content structure Tasks
Measurement Whether the application’s instruction is clear or if it needs a lecturer’s intervention. The extent to which the content of the application supports students in their learning. The degree to which the content of the application is organized in a way [clear, consistent and coherent] that supports learning.
Learner variables Collaborative learning Ease of use
Learner control
Motivation
2.3.2
Pedagogical usability metrics and their measurements [3]
The extent to which the tasks performed on the application help students to achieve their learning goals. The degree to which learner variables are considered in the application. The extent to which the application allows students to study in groups. The provision by the application of clear directions and descriptions of what students should do at every stage when questions to the mobile class activities are answered. The characteristics possessed by the application that allow students to make instructional choices. The extent to which the learning material is broken down into meaningful units. The extent to which the learning material in the application is so interesting to students that it compels them to participate. The degree to which the application motivates students.
Usability of the device
Any usability testing methodology has to account for the current limitations of mobile devices that are supposed to be used as learning mediums and has to provide information on the ease-of-use, effectiveness and efficiency of the mobile user interface. There are a series of known usability issues of the mobile devices that can affect the overall usability of the mLearning system: small screen sizes, low screen resolution or the form factor, low storage capacity and network bandwidth, limited processor performance, compatibility issues, lack of data input capability, short battery life, the use of the devices more often on the move. There are also a multitude of mobile user interfaces types with their own usability issues: scroll-and-select interfaces, tilting/sensor based interfaces, speech-based user interfaces, interfaces that use a stylus, and nowadays most common touch interfaces. Other specific characteristics are provided by the different operating systems interaction facilities and navigation scheme of the applications. There are a wide variety of devices, possessing different characteristics, which the application must be adaptable to all of them. These usability issues of mobile devices must be considered and carefully
examined during the usability testing of a mobile learning interface in order to select an appropriate research methodology and reduce the effect of contextual factors in the usability testing outcomes [5]. 2.3.3
Usability of the educational content
There could be usability issues regarding the content delivered by the mLearning application or web user interface. The format of electronic learning content is not always compatible to most mobile browsers, scripts and plug-ins are usually not supported and the ability to display information in various multimedia content is limited. Integrated graphics and animation should be provided in compatible mobile formats as well as audio and video files. Placing large images, video, PDF, MS Office files and usually any types of similar resources directly on the front of course pages should be avoided, links to images and video via activities, resources and pages is recommendable to be used. These and similar multimedia content compatibility issues must be considered when structuring the mLearning content. As a result of the usability testing of the content/courses provided by the mobile web interface we intend to elaborate a guideline for designing, developing and especially organizing content in a Moodle course for best use on mobile devices. 2.3.4
Usability of the mobile web interface
In this case, the mobile web interface strictly refers to the elements and structure of the mobile-friendly MyMobile Moodle interface. The usability testing can be similar to any other mobile web-page or application, and unlike the other perspectives, it can be evaluated using heuristic methods. The goals of the usability testing, in this case are: discovering navigation issues, improving the positioning and use of the menu elements, discovering bugs and verifying compatibility and interoperability of the UI elements on different devices, assuring efficiency, learnability and satisfaction in using the system and completing the tasks.
3
Research questions and Test preparations
Our first research question was if we could use the classical methods and techniques used in computer web interfaces usability testing for mLearning web interface usability testing. Even if there is not much literature on this topic, the main answer was that these methods are usually used with the observation that some need to be adapted to be used for mobile devices and some cannot be used at all because of the lack of technical solutions or specific software. There is a need in developing specific methods in evaluating mobile learning applications and we are looking forward to researching and providing such methods. The second research question was if the metrics used in computer web interfaces usability testing could be used for our purpose. The review of mobile UI usability
testing guidelines from the research literature reveals some specific metrics that we proposed to use for mLearning user interface testing. The third research question is what metrics of the above are best to be used in order to evaluate the usability of a mobile web interface of LMS? 3.1
Users, materials, devices and location
Usability testing of the mLearning MyMobile web interface for our Moodle eLearning platform will be performed by four groups of students and one group of experts: ─ students that are already using our university’s Moodle eLearning platform, that are used with the eLearning platform’s characteristics and with the course format ─ future users of the mLearning platform which are not used with the existing eLearning platform ─ students that use their own mobile devices (used with the specific UI) ─ students that use the provided mobile devices ─ experts from the eLearning department with professional background in the domain There are three types of devices that are proposed to be used for the evaluation: ─ old-generation smartphones with medium-size display and normal or full keyboard ─ new-generation smartphones / superphones with full touch-screen displays ─ new-generation tablets We also want to test the ability of these devices to support technologies used by the MyMobile UI. Their features also dictate whether certain UI features will be rendered, identified and visible on these devices. The usability test must set up in a way that is as close to the normal context as possible. Testing in a usability lab offers some benefits because it can be designed to create the ideal testing environment by using specific equipment for recording the session, controlled logging software, special equipment for testing mobile devices. On field and remote testing is much closer to the normal context of use and requires special applications that have the ability to automatically collect user interface events as the user interacts with the application, to capture screenshots and eventually, make an audio-video recording of the user. Even if closer to the real life use of the final product, testing out of the lab has specific disadvantages [6][7]. The testing will take place in one of the universities laboratories, a quiet and airconditioned room, equipped with audio-video recording system. One of the difficulties was to find a mobile and tablet usability testing kit [8] wishing to capture the full context of usage. There are some testing methods which use a static camera, telling the user to put a mobile device on a table, operating it within a designated area (where the camera is pointing). Such testing fails to capture
the true experience of real usage context. It does not allow the user to hold the device in his hand, and operate it as he or she would do in real life scenarios. More than that: the fact that the user is instructed not to move the device from a designated area, adds on stress during testing, which in itself is an un-wanted cognitive load, thus being a negative interference. The conditions considered when building a homemade mobile and tablet usability testing kit are: ─ the testing device should allow users to hold the mobile device (smartphone, tablet) the way they would hold it in real life ─ the testing device should be flexible to fit many types of mobile devices, which differ in size, and in screen resolution. ─ the device should look professional and communicate trust The solution that meets the challenges: ─ GPS mounts and mobile device holders ─ a high-end HD camera, which proves to produce high quality outcome in various light conditions, with autofocus ─ a glass-fiber lead to be used as a camera mount which gives both stability and flexibility. This allows to adjust the camera to fit many screen sizes, and also keeps a steady picture when the user moves the device around while using it. 3.2
Methods
User-based evaluation methods are mostly used in mLearning usability testing. These methods involve collecting quantitative and qualitative data from users while or after running through well prepared scenarios. Users are invited to do typical tasks with a product, or simply asked to explore it freely, while their behaviors are observed and recorded in order to identify design flaws that cause user errors or difficulties. During these observations, the time required to complete a task, task completion rates, and number and types of errors, are recorded. [9]. Users are also asked to complete questionnaires/surveys that provide useful qualitative data from users. Although the data collected is subjective, it provides valuable information on what the user wants. Professional observers and audio-video recording are also used to provide useful information by using the “think aloud protocol” and by measuring performance data. Traditional laboratory testing can be complemented by using UI event logging systems that provide useful information for measuring performance data and usability metrics [10]. Table 3. Usability evaluation methods [11] User testing
Thinking aloud protocol
User speaks aloud about what he is thinking throughout the evaluation.
Wizard of Oz Log file analysis Observation
This is done to watch the action in its setting,
and recording as many details as possible, without obstructing the experience of the subjects being studied. Measuring performance Cognitive walkthrough
Inquiry
Heuristics evaluation
This aims to complement observation, in order to get more insight into what is observed. This involves Usability Experts and requires an interactive prototype.
Guidelines review
Interviews Inspection
The researcher formulates questions about the product, based on the issues of interest. Interview representative users are then asked these questions, in order to gather desired information
Questionnaires for satisfaction Questionnaires for preferences Questionnaires for cognitive workload
3.3
Metrics
Upon review of the measures’ relative appearance in the reviewed literature the core constructs for the measurement of usability appear to be: ─ Efficiency: Degree to which the product is enabling the tasks to be performed in a quick, effective and economical manner or is hindering performance ─ Effectiveness: Accuracy and completeness with which specified users achieved specified goals in particular environment ─ Satisfaction: The degree to which a product is giving contentment or making the user satisfied These three dimensions also reflect the ISO 9241 standard making a strong case for its use in related future studies. [12] Based on the GQM model, Table 4 provides a list of usability characteristics, goals, guidelines and metrics that can help to collect quantitative or qualitative data during the usability evaluation process [13]. Table 4. Measures, goals, guidelines and metrics MEASURE
GOAL
Effectiveness Accessibility
GUIDELINE
METRIC
Ease of understanding content
Time taken to understand the content
Satisfaction
Help
Ease of navigation of help topics
Is/ not easy to learn how to navigate help topics
Interactivity
Ease of interaction
Amount of interaction required
Navigation
Ease of use Ease of customization Ease of navigation
Is or not easy to use Allow/ not allow customization Provide/ not provide easy navigation Time taken to load the application Time taken to learn the application Number of mistakes made before knowing how to use Time taken before to respond Amount of task effort required Amount of time taken before knowing what to do Time taken to complete a task Number of time the user follows the wrong path when attempting a task Provide/not task related clues Provide/not error recovery assistance Provide/not help Allow/ Not cater for personalization Rating scale of content organization Rating scale of multimedia usage Use or not appropriate controls Use or not narratives for content structure Rating scale of narratives appropriateness Use or not familiar mental models Rating scale of user familiarity with user interface Rating scale of consistency during navigation across system Is or not easy to navigate Rating scale on whether the system is attractive Is or not attractive Provide or not sufficient help information
Time taken Effort required
Loading application Time to learn Time taken to respond Amount of task effort
Features
Familiarity
Provision of task related clues. Provision of error recovery Provision of help. Allow personalization Organize content appropriately. Coherent multimedia usage. Use of appropriate controls. Narratives to structure the content. Mental models
Consistency
Navigation
Attractiveness Use appropriate font
style, size, and colors Help
Sufficient help information
Preciseness
Feedback
Organization of help topics Messages precision
Helpful messages Suitability for all users
Rating scale of usefulness of help information Provide or not precise messages Rating scale of message preciseness Provide or not helpful feedback messages Provide or not suitable feedback messages for all categories of users.
Usability can also be defined by other several quality components [14][15]: ─ Learnability: how easy users accomplish their basic tasks the first time they encounter the design. ─ Efficiency: number of steps it takes for a user to complete a task ─ Memorability: how easy is to memorize how to use the interface of the system and how easily users can reuse the system after a break ─ Errors: how many errors do users make using the interface of the system and how serious are these errors? ─ Satisfaction: how do users like using the system’s interface ─ Understandability (display load, clarity of operation possibilities, completeness of operation menu) ─ Operability (ease of input entering, display self-adjustment possibilities, messages conciseness, ease of output use, parameters self-adjustment possibilities, tasks based on user location) ─ Attractiveness (ease of use - displays per output, ease of use - displays per task) Mathematical formulas, the methods they were obtained and methods of application for some of the listed usability metrics can provide the source for developing new personalized metrics [16][17]. Based on testing methods and metrics, a usability testing framework for developing or testing mobile learning applications or mobile web interfaces can be structured and useful usability design guidelines can be modeled [18]. This paper presents a work in progress and the reviewed and listed metrics have to be also analyzed from a pedagogical perspective so that only relevant metrics are used. The final step would be the analysis and interpretation of the collected data and the use of the results in improving the mobile web user interface of the LMS and the educational content provided.
4
Usability Testing Framework
Fig. 1. A Framework for the Design and Implementation of Usability Testing of a LMS mobile web interface
In this section, the authors are proposing a generic framework (Figure 1) for the design and implementation of the usability testing of a LMS mobile web interface. The design of the usability testing should start with identifying research questions and objectives, as these determine the selection of the methods, location, tools and metrics used during the test. A survey should be done before choosing the mobile devices to be used for the test as they should be representative for the majority of the users of the system and should include different types of devices in terms of operating system, keyboard, display size and multimedia characteristics. The selected representative mobile devices should be used in the first stage for testing their own usability issues and the possibility to display the educational content provided by the learning platform. Experimented users should be used in identifying the main usability issues of the devices and educational content that could influence the UI usability testing and the pedagogical usability testing. Furthermore this is the step where experts are selecting the usability attributes to be measured, methods and metrics that should be used in the further main usability test, that are best to answer the research questions and objectives. After deciding if a laboratory or a field testing is required, and after determining the representative user profile of the learning platform, the next important step is selecting the usability attributes to be measured, methods and metrics that are best for answering the research questions and objectives from the UI and pedagogical usability testing points of view. The guidelines for the main usability test, the setup of the environment and equipment, the preparing of questionnaires and all the necessary preparations can be done based on the results of the previous steps. Running the actual usability testing, data collection, analysis and interpretation, are the last steps, followed by required changes of the mLearning platform. As this is a research in progress paper, the proposed framework still has to be improved, detailed and validated by a real test.
5
Discussions
The first contribution of this paper is an usability testing approach based on four perspectives that have to be considered when designing or testing a mobile learning web interface: Pedagogical usability, Usability of the device, Usability of the educational content and Usability of mobile web interface. There certainly are direct usability influences among the four domains. For example, usability issues of mobile devices accentuate usability issues of the UI [19]. Therefore, because the school cannot ask the students to change their mobile devices, usability improvements should be done to improve the user experience on the existing devices. The existing educational content cannot be easily changed, but changes can be easily done in the way courses are structured, and multimedia content can be adapted be used both on eLearning or mLearning web interfaces.
Further research should be done on understanding the connections and influences between the usability issues that belong to each of the four usability testing perspectives and identify the methods and metrics that can reveal best results when used in a realistic usability test. Concerning usability metrics that were mentioned in this paper, the next step should be the selection of the measuring formulas, or developing specific ones based on quality metric frameworks [16] for learning environments, and also the preparation of specific questionnaires elements, as our review of the literature indicates a lack of specific usability metrics for mobile learning environments.
6
Conclusions
Even if it is one of the most popular open-source eLearning platforms, Moodle is still in its beginnings in providing educational content on mobile devices. One of the biggest challenges that it has to overcome is the usability issue that mobile devices, mobile applications and web interfaces reveal. As the usability is reviewed as a determinant factor of success of an educational platform, it is very important to achieve a high usability level also on the mobile web interface. This research-in-progress presents the basis of a continuing research, which aims to enhance our understanding of mobile learning usability considerations and measurement and it provides the starting point for performing a usability evaluation of the MyMobile web interface of Moodle LMS. The proposed framework still has to be improved, detailed and validated by a real test. Additionally, metrics, methods and guidelines for usability testing were given. However, in order to test their applicability, they have to be applied on the real mLearning web interface with real users in the form of an actual evaluation. Research progress in usability design and testing guidelines, focusing on pedagogical usability also, rapidly improvements in the usability of mobile devices and their performances will influence in a good way the development of mobile learning. Acknowledgments: “This work was partially supported by the strategic grant POSDRU/88/1.5/S/50783(2009) of the Ministry of Labor, Family and Social Protection, Romania, co-financed by the European Social Fund–Investing in people.”
References 1. Bekim Fetaji and Majlinda Fetaji. Analyses and review of M-learning feasibility, trends, advantages and drawbacks in the past decade (2000-2010). In Proceedings of the 5th European conference on European computing conference (ECC'11), World Scientific and Engineering Academy and Society (WSEAS), Stevens Point, Wisconsin, USA, 2011, pp.474-479.
2. Abran, A., Khelifi, A., Suryn, W.,& Seffah, A.(2003). Consolidating the ISO usability models. Proceedings of 11th International Software Quality Management Conference (Springer), April 23 – 25, 2003. Glasgow, Scotland, UK 3. Kukulska-Hulme, A. Mobile usability in educational contexts: What have we learnt? International Review of Research in Open and Distance Learning, 8(2):1-16, 2007 4. Pakiso Joseph Khomokhoana, Using mobile learning applications to encourage active classroom participation: Technical and pedagogical considerations, Department of Computer Science and Informatics, University of the Free State, Bloemfontein - South Africa, May 2011, pp. 58 5. Bekim Fetaji and Majlinda Fetaji. 2011. Analyses and review of M-learning feasibility, trends, advantages and drawbacks in the past decade (2000-2010). In Proceedings of the 5th European conference on European computing conference (ECC'11), Remi Leandre, Metin Demiralp, Milan Tuba, Luige Vladareanu, and Olga Martin (Eds.). World Scientific and Engineering Academy and Society (WSEAS), Stevens Point, Wisconsin, USA, 474479 6. Carol M. Barnum, Usability Testing Essentials: Ready, Set...test!, ISBN: 978-0-12375092-1, Elsevier, 2011 7. A. Kaikkonen, A. Kekäläinen, M. Cankar, T. Kallio, and A. Kankainen. Usability testing of mobile applications: A comparison between laboratory and field testing. Journal of Usability Studies, 1(1): 4--17, 2005. 8. Spotless Interactive - http://www.spotlessinteractive.com/news/mobile-usability-testingkit.php , January 2012 9. Mary Ann Chiramattel Kunjachan, Evaluation Of Usability On Mobile User Interface, University Of Washington, Bothell 10. Xiaoxiao Ma, Bo Yan, Guanling Chen, Chunhui Zhang, Ke Huang, and JillDrury, A Toolkit for Usability Testing of Mobile Applications, Computer Science Department, University of Massachusetts, MobiCase 2011 11. Bernhaupt, R., Mihalic, K. & Obrist, M. (2008): Usability Evaluation Methods for Mobile Applications. In: Lumsden, J. (Ed.). Handbook of Research on User Interface Design and Evaluation for Mobile Technology, IGI Global. 745-758. 12. Coursaris, C., and Kim, D., (2006). A Qualitative Review of Empirical Mobile Usability Studies, Proceedings of the Americas Conference on Information Systems (AMCIS), Acapulco, Mexico, August 4-6, 2006. 13. Kantore, A and van Greunen, D. Metrics for m-learning usability evaluation, WWW2010 September 2010, Durban. 14. Jakob Nielsen. Usability 101: Introduction to Usability, 2003.Available at http://www.useit.com/alertbox/20030825.html 15. Ruti Gafni, Usability Issues in Mobile-Wireless Information System, Issues in Informing Science and Information Technology Volume 6, 2009 16. Gafni, R. Framework for quality metrics in mobile-wireless information systems. Interdisciplinary Journal of Information, Knowledge, and Management, 3, 23-38.2008 17. Gafni, R. Quality Metrics for PDA-based M-Learning Information Systems. A. Koohang, Ed.) Interdisciplinary Journal of E-Learning and Learning Objects, 5, 2009 18. Fetaji Majlinda, Fetaji Bekim (2011). Devising M-learning Usability Framework. In Information Technology Interfaces (ITI), Proceedings of the ITI 2011 33rd International Conference, pp. 275 - 280. IEEE, New York, USA, 6 / 2011. ISBN 978-1-61284 19. Azham Hussain, Maria Kutar, Apps vs Devices: Can the Usability of Mobile Apps be Decoupled from the Device?, IJCSI International Journal of Computer Science Issues, Vol. 9, Issue 3, No 3, May 2012 pp.11-16
