Experimentally Investigating the Effects on

Explicitness of Consequence Information in Privacy Warnings: Experimentally Investigating the Effects on Perceived Risk, Trust, and Privacy Information Quality Completed Research Paper

Gökhan Bal Goethe University Frankfurt Grüneburgplatz 1, 60323 Frankfurt, Germany [email protected] Abstract Utility that modern smartphone technology provides to individuals is most often enabled by technical capabilities that are privacy-affecting by nature, i.e. smartphone apps are provided with access to a multiplicity of sensitive resources required to implement context-sensitivity or personalization. Due to the ineffectiveness of current privacy risk communication methods applied in smartphone ecosystems, individuals’ risk assessments are biased and accompanied with uncertainty regarding the potential privacy-related consequences of long-term app usage. Warning theory suggests that an explicit communication of potential consequences can reduce uncertainty and enable individuals to make better-informed cost-benefit trade-off decisions. We extend this design theory to the field of information privacy warning design by experimentally investigating the effects of explicitness in privacy warnings on individuals’ perceived risk and trustworthiness of smartphone apps. Our results suggest that explicitness leads to more accurate risk and trust perceptions and provides an improved foundation for informed decision-making. Keywords: Information Privacy, Risk Communication, Mobile Computing, Privacy Behavior, Privacy Calculus

Thirty Fifth International Conference on Information Systems, Auckland 2014

1

IS Security and Privacy

Introduction The use of information technology by individuals in everyday life is driven by their striving for utility. Particularly modern smartphone technology has rapidly become indispensable for many people, as they offer utility in manifold ways. Smartphones provide users with access to the Internet, connectivity in various ways, context-sensitivity, personalization, and entertainment. Utility is enabled and promoted by two main factors. Firstly, smartphone applications (“apps”) are provided with access to a multiplicity of device resources required to implement utility (e.g. access to positioning features to implement locationbased services). Secondly, the major app markets are centralized and open, i.e. apps can be developed and provided to customers easily by anyone. In this way, the global app economy became worth $68 billion in 2013 and is projected to grow to $143 billion in 2016 (Vision Mobile 2014). While offering significant utility, smartphone app usage is also considerably affecting users’ information privacy1. This is caused by the same capabilities of smartphone platforms that enable utility, specifically, allowing apps to access and process a multiplicity of sensitive resources on the device such as positioning features, the contacts database, or device sensors. Numerous research studies reveal that users are often not aware of sensitive information flows (Egele et al. 2011; Enck et al. 2010), which as a consequence, negatively influences the users’ ability to assess the potential privacy risks of app usage. Risk assessments are rather accompanied with uncertainty regarding the actual consequences of usage behavior (Acquisti and Grossklags 2005). Current privacy risk communication methods in smartphone ecosystems have been shown to be ineffective in informing the users about privacy-affecting properties of individual apps. Smartphone users rather rely on other trust anchors such as app ratings, user reviews, or visual appearance in risk assessments and decision-making (Chia et al. 2012; Krasnova et al. 2013). Warning theory suggests that a more explicit communication of potential risks can reduce uncertainty and help individuals making more informed decisions regarding product use (Laughery and Smith 2006). So far, the potential effects of explicitness have not yet been examined empirically in privacy risk communication. Grounded in warning design theory and smartphone privacy literature, we propose a design for privacy warnings with explicit consequence information and experimentally study its effects on perceived privacy information quality, perceived privacy risk, perceived trustworthiness, and app preference in the context of smartphone app markets. We contribute to privacy warning design theory with knowledge to improve the quality of individuals’ privacy risk and trust assessments by providing an improved foundation for making informed decisions. The research results are relevant for app providers, app market providers, and privacy warning research. The paper is structured as follows. Section 2 provides the theoretical background of our research. This includes relevant theories from information privacy research, a discussion of the nature of privacy risks in smartphone app ecosystems to recognize the potential privacy-related consequences of app usage, and relevant design knowledge from warning research. Section 3 presents our proposed design for privacy warnings for smartphone apps with explicit consequence information and concludes with a set of testable hypotheses. Section 4 elaborates on the used methodology to test the effectiveness of our proposed warning scheme, including a description of the experimental user study that we have conducted. Section 5 presents the results of the user study, which we discuss in Section 6. Implications and limitations of our findings are discussed in Section 7, while Section 8 concludes the article.

Theoretical Background The discussion of prior literature starts with a literature-based privacy risk analysis of smartphone app usage. The aim is to recognize and conceptualize potential privacy-related consequences of app usage to inform the design of more explicit privacy warnings. Next, we provide an overview of relevant theories from information privacy research to explain factors that influence individuals’ privacy behavior. We then discuss relevant theories from warning research that provide design knowledge for effective warnings. Finally, we discuss the effectiveness of existing privacy indicators in smartphone app ecosystems to further motivate the need for an improved privacy risk communication. 1

In the rest of this article, we refer to the concept of information privacy when using the term privacy.

2


Explicitness of Consequence Information in Privacy Warnings

Privacy Risks in Smartphone App Usage Prior to designing warnings, a hazard analysis to understand the nature and the causes of potential risks should be performed (Smith-Jackson and Wogalter 2006). The privacy-affecting nature of smartphone app usage mainly emerges due to technical capabilities of smartphones that are primarily intended to enable utility. For example, apps are provided with access to sensitive resources in order to enable personalized or context-sensitive services. Since many apps in addition have Internet-access capabilities, apps in principle can share any collected sensitive information with external entities (e.g. app provider or advertisement networks). Using the information-flow tracking system TaintDroid, Enck et al. (2010, 2011) demonstrated that many Android apps leak sensitive data without informing the user. Similar behavior has been observed on the iOS platform (Egele et al. 2011). On most smartphone platforms, applications have to request permissions to access sensitive resources, which the users have to accept if they intend to use the application or the respective functionality. Permission-based security mechanisms are meant to enforce informed consent and to provide privacy transparency. Another function of those mechanisms is to drive app developers to follow the principle of least privilege. However, application analysis tools revealed that many apps are over-privileged, i.e. they request more permissions than actually required to implement functionality (Felt, Chin, et al. 2011; Wei et al. 2012). This suggests that app developers do not always comply with the principle of least privilege, regardless of whether harm is intended or not. Irrespective of the permissions granted, Davi et al. (2010) showed that current permission-based security mechanisms are vulnerable and can be circumvented. Thus, malicious applications can gain unauthorized access to sensitive resources. Moreover, inter-application communication capabilities allow apps to exchange messages, which is another way to circumvent permission-control mechanisms (Chin et al. 2011). Enck et al. (2009) argue that privacy and security risk assessments should not isolate single permissions from each other. Rather, risk assessments should consider potentially dangerous combinations of permissions (e.g. Internet access in combination with call listening and audio recording capabilities) since the overall risk might be greater than the sum of the risks posed by single permissions. A more sophisticated risk assessment framework for smartphone apps has been proposed by Zhang et al. (2013). It calculates an intrusiveness score for individual apps that is based on the apps’ individual dataaccess patterns. The context of access (e.g., whether or not the app is idle during data access) is a particularly considered factor in the assessment. The researchers’ analysis of 33 apps across four different categories revealed that each app appears to have an individual privacy fingerprint based on its dataaccess behavior. These findings underline the need for considering the application-specific and dynamic aspects of data access in privacy risk assessments (Bal 2012). The potentials of harming user privacy when long-term smartphone data is available have been further demonstrated by several data-mining based information extraction approaches. These show how individuals implicitly reveal additional information about themselves when access to sensitive resources on the device is granted to third parties. Weiss and Lockhart (2011) used available accelerometer data to accurately predict user traits such as sex, height, and weight. Min et al. (2013) could infer smartphone users’ social network structure by analyzing communication logs on the devices. Similar results are demonstrated by Eagle et al. (2009). The possibility to predict individuals’ personality traits by analyzing smartphone usage data has also been demonstrated (Chittaranjan et al. 2011). Further, individuals’ movement patterns can be predicted by using smartphone data (Phithakkitnukoon et al. 2010). The potentials for user identification and authentication by analyzing smartphone data have been demonstrated in numerous studies (Conti et al. 2011; E. Shi et al. 2011; W. Shi et al. 2011). Table 1 summarizes the different factors that have an impact on the privacy risks in smartphone app usage. The risk analysis suggests that the risks and their potential consequences are multifaceted and not limited to the leakage of single information items. We conclude the hazard analysis by emphasizing the importance of considering the dynamic and long-term aspects of data access when designing privacy warnings. We want to note that the severity of privacy risks is further dependent on factors such as the purpose and functionality of the app, or on the app provider’s intentions to share personal data with third parties. In this work we abstract from those additional factors to gain first understanding of the potential effects of explicitness and suggest that future work on privacy warning design should consider more factors. In this study, we focus on the risks coming from data-mining approaches (cf. Table 1).


3


Table 1. Privacy Risk Factors in Smartphone Usage Factor

Explanation

Data Leakage

Many apps leak private data items without informing the user (Egele et al. 2011; Enck et al. 2010). Data leakages represent the most basic level of information privacy risks. They violate the principle of user control and transparency.

Over-privilege

Many apps possess more privileges to access private data than required to implement the functionality. Consequently, the privacy principles of data minimization and purpose-binding are threatened (Felt, Greenwood, et al. 2011; Wei et al. 2012).

Privilege escalation

Permission-based security mechanisms are vulnerable and can be circumvented. Consequently, there is a risk of unauthorized access to private data by harmful apps (Davi et al. 2010). This risk can rather be mitigated by improved implementations of permission-based security mechanisms.

Inter-application communication

Apps can communicate with each other. In this way, private data can be shared with apps that don’t possess the required permissions (Chin et al. 2011). While this capability provides some benefits, the risk is unauthorized access to private data.

Combinations of permissions

Some permission combinations are dangerous, e.g. an app with Internet, audio recording, and phone listening capabilities could record phone calls and share the recordings with external servers (Enck et al. 2009). The risk is permission misuse.

Out-of-context data access

Apps might access sensitive resources out of the usage context (Zhang et al. 2013). This poses a risk to the principle of purpose-binding and transparency.

Data mining

Smartphone data can be used to infer additional information about the user (e.g. Min et al. 2013). The privacy risk is the unintended, implicit revelation of private information and consequently leads to a bias in users’ risk awareness.

Information Privacy: Influencing Factors of Privacy Behavior Individuals’ privacy behavior2 is affected by various factors. While privacy concerns have been shown to negatively influence privacy behavior (Choi and Choi 2007; Malhotra et al. 2004; Smith et al. 1996; Zhou 2011), Milne and Boza (1999) suggest that building trust is more effective than trying to reduce privacy concerns. The positive effects of trust on privacy behavior have been demonstrated in various contexts (Doney and Cannon 1997; George 2004; Pavlou 2003; Slyke et al. 2006; Smith et al. 2011; Wang et al. 2004; Xu et al. 2005; Zhou 2011). Trust on the other hand, has been shown to be positively affected by the presence of privacy seals or notices (LaRose and Rifon 2006; Wang et al. 2004). Consequently, privacy behavior can be influenced by the availability of privacy notices. Furthermore, trust has been shown to reduce perceived risk (Jarvenpaa et al. 2000; Xu et al. 2005; Zhou 2011), while also a direct, negative relationship between perceived risk and privacy behavior has been suggested (Malhotra et al. 2004; Xu et al. 2005). Moreover, the privacy calculus theory suggests that privacy behavior is the outcome of a riskbenefit analysis of individuals in which potential risks of behavior are weighed against the benefits (Dinev and Hart 2006; Smith et al. 2011). An accurate calculus requires the availability of both benefit and risk information that serve as input variables to the calculus. Potential benefits are most often clear from the

In the rest of this article, we use the term “privacy behavior” to denote any behavior-related outcome of information privacy, e.g. behavioral intentions to use, actual behavior, willingness to share data. 2

4



purpose of a service (e.g. personalization). Furthermore, the benefits are what drive individuals toward products. Privacy risk information on the other hand is not always present and accessible. To the contrary, privacy decision-making is often based on incomplete information and consequently leads to uncertainty regarding the potential consequences of privacy behavior (Acquisti and Grossklags 2007). Regarding the usefulness of reducing uncertainty by providing more risk information, Acquisti and Grossklags (2007) argue that even with access to complete information, individuals would be unable to process and act optimally on large amounts of data and consequently people rather rely on simplified mental models and approximate strategies in privacy decision-making. Lederer et al. (2004), however, argue that illuminating information disclosure can constructively contribute to the user’s mental model and understanding of privacy risks. We therefore aim for designing privacy indicators that more accurately and efficiently communicates the potential consequences of privacy behavior. The ultimate goal is to provide individuals with an improved foundation for privacy calculi that help in identifying the less privacy-intrusive choices when alternatives are available. We consult warning theory in the following to inform our design search process for more effective privacy warnings.

Warning Theory Warnings are safety communications used to inform people about hazards so that undesirable consequences are avoided or minimized (Wogalter 2006), i.e., warnings are artifacts designed for risk communication. The term “hazard” covers all kinds of potential vulnerabilities and risks encountered during product use, when performing tasks, and in the environment. In this work, the hazards of interest are privacy risks caused by information technology use, particularly the long-term use of smartphone apps. According to the hazard control hierarchy concept from safety engineering, warnings should only be the last resort of hazard control (Wogalter 2006). The first in the sequence should be designing the hazard out or eliminating it, e.g. through alternative design. The next suggested action in the sequence is protecting people from the hazard to reduce the likelihood that people come in contact with the hazard. Designing out the privacy risks of smartphone app usage, as identified in our privacy risk analysis, would require removing the data-access capabilities that enable utility. This would negatively influence the user experience of smartphone usage and is therefore not considered an option. Protecting users from the hazard, on the other hand, can be achieved by providing users with enhanced privacy control mechanisms to better control sensitive-information flows and adapt the privacy risks according to their preferences. While some tools have been developed to provide users with a more fine-grained control over personal data (Bai et al. 2010; Conti et al. 2010; Zhou et al. 2011), the inherent limitations of such privacy tools are related to their usability and perceived usefulness. Firstly, such tools often require a certain degree of technical knowledge. Secondly, the perceived usefulness of privacy protection tools is affected by the perceived privacy risks of a product. Thus, privacy control tools are only useful if users have a fair understanding of the risks. Moreover, the effectiveness of such tools is seldom being evaluated. All things considered, we believe that privacy risk communication is a key area which needs improved design work. The Communications-Human Information Processing (C-HIP) model describes a set of stages involved in warning processing that can be used to systematically examine and improve the effectiveness of warnings (Smith-Jackson and Wogalter 2006). The consecutive stages are defined as attention switch (the warning must have characteristics that make it noticeable), attention maintenance (the warning must be able to hold attention so that users can encode its message), comprehension (the receiver of the warning must be able to extract meaning from the warning and activate relevant information in memory), attitudes/beliefs (the warning must be able to influence the receiver’s hazard-related attitudes and beliefs), motivation (the warning must be able to push the receiver towards safe behavior), and behavior (the receiver must perform the safe behavior). Warning theory further provides design knowledge for improving warning effectiveness in the different stages of warning processing. Suggested factors to influence warning effectiveness are size (Barlow and Wogalter 1991), location (Frantz and Rhoades 1993), timing (Egelman et al. 2009), colors used (Kline et al. 1993; Young 1991), signal words or pictorials, message length (Laughery and Wogalter 1997), or explicitness (Laughery et al. 1993). Explicitness has emerged as an particularly efficacious design factor for warning effectiveness (Laughery and Wogalter 2006). Explicitness in the context of warnings is defined as information that is specific, detailed, clearly stated, and leaves nothing implied, while explicitness can refer to hazard information, consequence information, or the instructions to avoid the hazard. Laughery and Smith (2006) suggest a set of principles regarding explicitness that may be useful for the warnings designer, e.g. “do not assume


5


everybody knows.”, “do not rely on inference.”, “be careful about assuming that hazards and consequences are open and obvious.”, “technical jargon is usually not a good way to achieve explicitness.”. Laughery and Wogalter (2006) suggest that more specific information about hazards and consequences can reduce uncertainty and enable people to make better-informed cost-benefit trade-off decisions regarding the need to comply. It is suggested that explicitness of warning information is especially effective and useful when the injury severity is high. Further, warning research suggests that the contents of risk communication should focus on risk severity instead of likelihood of injury since risk severity has been shown to be the better predictor of perceived risk (Wogalter et al. 1999). The effects of explicitness of consequence information in privacy warnings so far have not been empirically investigated.

Privacy Warning Effectiveness and Alternative Approaches Information security and privacy warnings have been shown to be ineffective in risk communication. Individuals often have difficulties in understanding security warnings (Bravo-Lillo et al. 2011). Privacy indicators such as privacy policies are not processed by consumers since they tend to be too long or written in a technical or juridical language (Milne and Culnan 2004). Specifically in smartphone ecosystems there are no reliable privacy risk indicators currently in app markets (Chia et al. 2012). A commonly used approach to provide privacy indicators and risk information is to face smartphone users with permission requests to access sensitive resources. Research studies have already demonstrated the ineffectiveness of this approach (Kelley et al. 2012). As a result of the ineffectiveness of existing privacy indicators, people rather act based on their general attitudes or rely on other trust anchors such as user reviews or app ratings (Chia et al. 2012; Krasnova et al. 2013). In attempts to provide individuals with more effective privacy risk information, researchers have designed and tested a variety of alternative privacy indicators. Kelley et al. (2009) made use of the nutrition-label approach to design consumer privacy notices. Their attempt was to bring text-based privacy policies to a format which allows for more efficient information gathering and easier comparison between different policies. While the researchers could show that comparison is easier, the contents of the indicators are still based on privacy policies that do not provide information about the potential risks of a service. The P3P standard enables machine-readable privacy policies that can be retrieved automatically by web browsers and other user agents. Cranor et al. (2002) have developed the Privacy Bird as a P3P user agent that can compare P3P policies against users’ privacy preferences. However, privacy risk communication of Privacy Bird focuses on potential policy mismatches and does not inform about potential consequences. The approach taken by Lin et al. (2012) is to model privacy information as expectations. The rationale of this approach is to trust on the accuracy of crowd opinions regarding the appropriateness of permission requests. While we believe that especially expert knowledge can help, no risk information is provided that can help the users in making more informed decisions. Thompson et al. (2013) tackled another aspect of privacy risk communication, namely helping users to attribute data access to the software responsible for it. The purpose is to help users identify misbehavior. Another approach tested is the use of the metaphor of growing eyes on the display with data disclosure (Schlegel et al. 2011). A problem of this approach is that it might disturb the primary task in the long run. Another factor that has been shown to be relevant in privacy risk communication is the timing of information (Egelman et al. 2009). Altogether, privacy risk communication in the digital world is lagging behind risk communication in the physical world. Privacy warnings or indicators are most often ineffective in influencing privacy behavior. Camp (2009) suggests that established risk communication methods from warning research should be applied to improve the effectiveness of computer security and privacy warnings. We recognize that the long-term data-access behavior of apps significantly influences the privacy-related consequences of app usage and, based on knowledge from warning theory, we suggest introducing explicit consequence information into privacy warnings.

Designing Explicit Privacy Warnings for Apps The key findings from our privacy warning design search process are as follows. Privacy behavior can be positively influenced by providing trust indicators (e.g. privacy seals) or by employing risk communication methods (e.g. privacy warnings). Privacy decision-making is often accompanied with uncertainty regarding potential consequences (Acquisti and Grossklags 2005). As a consequence, individuals rely on other trust anchors than risk information (Chia et al. 2012; Krasnova et al. 2013). Computer security

6



warning research suggests applying established methods from general warning research (Camp 2009) to security and privacy warning design. A specific factor that can help to reduce uncertainty in privacy decision-making is explicitness in warning messages (Laughery et al. 1993). Explicitness can be added to the hazard information, the consequence information, or the instructions to avoid the hazard. While privacy research has so far focused on informing about the hazard (e.g. potential data access or intentions to share with third parties), we want to investigate the potential effects of explicitness in consequence information. To the best of our knowledge, this has not yet been empirically investigated in the context of information privacy risk communication. Regarding the nature of potential consequences of privacy behavior, our hazard analysis suggests that smartphone applications differ in their data-access behavior (Zhang et al. 2013), thus considering dynamic data-access patterns in risk analyses is essential (Bal 2012). The implicit revelation of private information through potential application of data-mining techniques is one type of potential consequences of long-term privacy behavior. We propose adding information about such potential revelations to privacy warnings as the typification of privacy-related consequences. We believe that this will reduce uncertainty about the risks and consequently lead to more accurate privacy risk assessments, i.e. individuals will be better able to distinguish apps with high risk severity from apps with low risk severity. Figure 1 conceptually illustrates our design suggestion for the warning’s contents. It includes both hazard information (data-access permissions) and consequence information (implicit revelations). Since timing is another critical factor in warning design and placement (Egelman et al. 2009), we suggest introducing such warning information into app markets in order to maximize the chances to influence users’ behavior (e.g. selecting the less privacy-intrusive app when alternatives are available). In that way, contact with the hazard can be avoided. Figure 1. Privacy Warning with Explicit Consequence Information

We used the proposed conceptual design in an experimental study to investigate its effects on individuals’ risk and trust perceptions and their privacy behavior. Backed up by existing theory, we formulated a set of testable hypotheses that we used in our study to investigate on the effectiveness. Since we aim for contributing with design knowledge for more effective privacy risk communication methods, our hypotheses are directed and express positive expectations regarding the outcomes. Yet, effective privacy warnings do not necessarily increase risk perception or decrease trust in general, they rather aid users in making better assessments regarding these two variables. For example, with improved privacy risk communication, individuals should be better able to distinguish apps with high privacy risk severity from apps with low risk severity and perceived risk and trust should be affected accordingly. Inspired by the notions used in a previous study about physical-product warning explicitness (Laughery et al. 1993), we use the following abbreviations to classify apps with varying privacy risk severity and number of requested permissions in our testable hypotheses. H-S refers to apps with high privacy risk severity; L-S refers to apps with low privacy risk severity. Analogous, L-P refers to apps with a low number of data-access permission requests; H-P refers to apps with a high number of permission requests.


7


Due to the ineffectiveness of current privacy risk communication in smartphone app markets (e.g. permission request screens), users rely on other trust anchors (Chia et al. 2012). We specifically assume that users base their judgments on the quantity of permission requests rather than on the quality of their privacy-related consequences when no other cues are available even though the relationship between the number of permissions requested and the privacy risk severity is not necessarily linear. Based on this assumption, we are primarily interested in scenarios in which the quantity and the quality of permission requests regarding privacy consequences are diametrical (H-S/L-P and L-S/H-P) and therefore we blind out the “trivial” cases (L-S/L-P and H-S/H-P). We want to measure individuals’ risk perceptions with two related variables. On the one hand, we are interested in measuring individuals’ perceived privacy risk for individual apps as an absolute measure. On the other hand, we investigate the participants’ ability to sort different apps according to their privacy-intrusiveness, which we consider as a measure of relative perceived privacy risk among different alternatives. The latter will provide more direct insight into individuals’ ability to distinguish privacy-intrusive apps from less privacy-intrusive apps. To find support for our expectations, we test the following hypotheses: Hypothesis 1a: The perceived privacy risk of an H-S/L-P app will be higher when explicit consequence information is available in privacy warnings. Hypothesis 1b: The perceived privacy risk of an L-S/H-P app will be lower when explicit consequence information is available in privacy warnings. Hypothesis 1c: Individuals will rank an H-S/L-P app more privacy-intrusive when explicit consequence information is available in privacy warnings. Hypothesis 1d: Individuals will rank an L-S/H-P app less privacy-intrusive when explicit consequence information is available in privacy warnings. Hypothesis 1e: The perceived privacy risk of an H-S/L-P app will be higher than the perceived privacy risk of an L-S/H-P app when explicit consequence information is available in privacy warnings. Prior research further suggests that trust is affected by the presentation of privacy notices and also a mediating effect of perceived risk on trust has been suggested (Lim 2003). Based on these findings, we suggest that more effective privacy notices will affect the trustworthiness of an app: Hypothesis 2a: The perceived trustworthiness of an H-S/L-P app will be lower when explicit consequence information is available in privacy warnings. Hypothesis 2b: The perceived trustworthiness of an L-S/H-P app will be higher when explicit consequence information is available in privacy warnings. Hypothesis 2c: The perceived trustworthiness of an H-S/L-P app will be lower than the perceived trustworthiness of an L-S/H-P app when explicit consequence information is available in privacy warnings. We expect that the new privacy warnings will improve risk assessments abilities and as a consequence, privacy decisions will be improved. We therefore hypothesize that privacy behavior will be affected towards safer choices when making potential consequences explicit: Hypothesis 3a: App preference will be affected by the availability of explicit consequence information in privacy warnings. Hypothesis 3b: App preference will result in the selection of apps with lower privacy risk severity when explicit consequence information is available. Prior research suggests that the presence of privacy notices positively affects the perceived trustworthiness of a service provider (LaRose and Rifon 2006; Wang et al. 2004). In our context, app market providers are service providers offering smartphone apps to smartphone users. Based on these findings, we further expect that more effective privacy notices in an app market will have a positive effect on the perceived trustworthiness of the app market as the service provider: Hypothesis 4: The perceived trustworthiness of an app market will be higher when it provides privacy information with explicit consequence information.

8



We introduce explicit privacy warnings to provide additional trust and risk information, but ultimately we aim for increasing the quality of privacy information. We expect that adding explicit consequence information to privacy warnings will increase the perceived privacy information quality. We further expect that explicit consequence information in privacy warnings will make the comparison of apps regarding their privacy properties easier and more enjoyable: Hypothesis 5a-5f: The perceived information quality (amount / believability / interpretability / relevance / understandability / correctness) of privacy warnings will be higher when explicit consequence information is available. Hypothesis 6: The perceived enjoyment of comparing apps regarding the privacy properties will be higher when explicit consequence information is available in privacy warnings.

Method We tested the effectiveness of the proposed privacy risk communication scheme with explicit consequence information by conducting online experiments in which individual participants are presented several app descriptions in a fictive app market called “SMART App Shop”. The anonymous online experiment was conducted in the beginning of the year 2014 during a period of seven weeks.

Participants Participants were recruited through advertisements in online platforms such as Facebook, student forums on university web sites, or by e-mail. Recipients of the advertisement were asked to further spread the advertisement to friends, colleagues, or relatives. We invited people to partake in an online experiment about “Experiences with App Discovery”. We wanted to mitigate priming effects by mentioning the topic of privacy upfront. There were no specific requirements for participation, thus the sample was from the general population. The participants remained anonymous and they were not compensated by any financial means. In total, 94 participants started the experiment, out of which 71 participants completed it (attrition rate = 24.47%). In the data analysis, we only used the data from the 71 individuals who completed the experiment. The sample included 24 female and 47 male participants with an average age of 32.87 (SD = 10.02; range = 18 - 60). Regarding experiences with smartphone usage, 52.1% of the participants had most experiences with the Android platform, 35.2% with the iOS platform, 7% with the Symbian platform, 4.2% with the Windows Phone platform, and 1.4% with the Blackberry platform. The average number of apps that the participants used regularly (at least once a week) was 10.07 (SD = 11.81). Regarding the period of using a smartphone, 9.8% of the participants used a smartphone for less than 1 year, 38% between 1-3 years, and 52.1% for more than 3 years.

Materials We implemented the experiment as an online survey using the LimeSurvey 3 software. The survey first collected some demographic information and information about prior experiences with smartphone usage. We further measured privacy concerns with smartphone apps and general trust towards apps. Since existing scales for privacy concerns and perceived trustworthiness were not specific enough for our purposes, we developed the required measurement instruments that are more targeted towards our smartphone app scenarios4. The used smartphone privacy concern scale consists of 10 items (Cronbach’s Alpha = .891; example item: “It is important to me to know which personal data is accessed by my apps.”). The trust scale consists of 3 items (Cronbach’s Alpha = .783; example item: “I believe that apps only access personal data when it is required for the functionality.”). Both scales were rated on 6-point Likert scales ranging from “strongly disagree” to “strongly agree”. We further included the 10-item General SelfEfficacy scale in our survey (example item: “I can always manage to solve difficult problems if I try hard enough.”), which had to be rated on the proposed 4-point Likert scale ranging from “not at all true” to

3

https://www.limesurvey.org/

The scales were developed by employing PCA-based exploratory factor analysis on a pool of items that we iteratively developed and improved in multiple rounds of focus group meetings. 4


9


“exactly true” (Schwarzer and Jerusalem 1995). To measure perceived privacy risk of individual apps, we slightly modified the 3-item perceived risk scale developed by Featherman and Pavlou (2003) to make it applicable for our purposes (example item: “Using this app would cause me to lose control over the privacy of my personal information.”). To measure perceived trustworthiness of specific apps, we used a slightly adapted version of the trust scale developed by Wang et al. (2004) (example item: “I believe that the app provider is not likely to sell my personal information.”). We measured six dimensions of perceived privacy information quality based on the scales developed by Lee et al. (2002). The six dimensions that we were interested in were amount (example item: “The privacy information is of sufficient volume.”), believability (4 items; example item “The privacy information is believable.”), interpretability (4 items, example reversed item: “The privacy information is difficult to interpret.”), relevance (3 items; example item: “The privacy information is useful.”), understandability (4 items; example item: “The privacy information is easy to comprehend.”), and correctness (4 items; example reversed item: “The privacy information is incorrect.”). To measure the perceived trustworthiness of the fictive app market used in the study, we used another 8-item perceived trustworthiness scale (example item: “I can count on SMART App Shop to protect my privacy.”), based on the trust instruments developed by Jarvenpaa et al. (2000) and Fogel and Nehmad (2009). Finally, we used and slightly adapted the 2-item enjoyment and ease of comparison scale (example item: “Comparing apps in the SMART App Shop regarding their privacy friendliness was an enjoyable experience.”) used by Kelley et al. (Kelley, Bresee, et al. 2009).

Experimental Design The design of our experiment is partly adapted from and inspired by the experiments conducted by Laughery et al. (1993) to study the effectiveness of explicit information in physical product warnings. We used a between-subjects design, while we manipulated the contents of the privacy warnings in the experimental part of the survey. Figure 2 illustrates the process of the experiment. The survey started with the collection of demographic information and questions about smartphone usage experiences, followed by the smartphone app privacy concern, trust, and self-efficacy scales. We explicitly measured privacy concern and trust before the actual experiment started to avoid any bias that might have been caused by the experimental scenario. Subsequently, introductory information about the experiment has been presented to explain the experimental setting, in which the participants were smartphone users who search for some apps with specific functionality (flashlight app and weight diary tool) and therefore visit the fictive app market called “SMART App Shop” to use its search function. The actual search was simulated and part of the background story, i.e. the participants did not have to type in anything. The random group assignment took place transparently while the introductory page was shown (random number generation). Following the introductory page, they were presented a list of four flashlight apps and respective descriptions that pretendedly resulted from the search (cf. Figure 3). The following information about the apps was provided: generic name (e.g. “Flashlight 1”), icon, short description of the core functionality, and privacy information. The list of apps was static, i.e. all participants were presented the same set of apps with the same icons and descriptions in the same order, only the privacy information presented was different between the two experimental conditions (without explicit consequence information in the control group and with explicit consequence information in the experimental group). To test our hypotheses, we have manipulated the privacy information of the apps in two ways. Firstly, explicit consequence information (framed parts in Figure 3) was only shown in the experimental group (between-subjects design), while the control group was only presented permission requests. Secondly, the four apps presented have been manipulated by their number of permissions requested and their privacy risk severity based on the actual permissions requested (within-subjects design). Thus, we have created different app profiles that were required for our hypotheses testing (cf. Figure 4). As mentioned before, we did not use the apps matching the “trivial” profiles for hypothesis testing. Yet, we introduced them to the experiment for the sake of completeness. The set of permission requests that we used in our experiment is a subset of existing permissions in the Android framework and smartphone ecosystem, while some of them have no or little privacy consequences (e.g. “control vibration” or “prevent device from sleeping”) and others have high privacy consequences (e.g. “precise location” or “read call history”). To create the required app profiles, we used a respective subset of those permissions for each app. The potential privacy-related consequences per app were then based on and derived from the permissions granted to an individual app. For example, an app that requested only permissions with low or no privacy consequences was classified as having low privacy risk severity, whereas an app that had at least one permission with high privacy consequences has been classified as having high privacy risk severity.

10 Thirty Fifth International Conference on Information Systems, Auckland 2014


Figure 2. Process of the experiment with between-subjects design.

Figure 3. The flashlight apps as presented in the experiment. The parts in dotted frames (explicit consequence information) were only shown in the experimental condition.


11


Figure 4. Risk severity and permission profiles of the apps used in the experiment.

The classification of the permissions regarding the degree of their potential privacy consequences has been performed in a rationale and qualitative approach prior to the start of the study. We introduced two app categories (flashlight apps and health apps) to split hypothesis testing between these two categories. While we targeted the flashlight app profiles towards the hypotheses concerning user perceptions (hypotheses 1a-e and 2a-c), the health app profiles were designed in a way to best help testing the hypotheses regarding privacy behavior (app selection according to hypotheses 3a and 3b). All hypotheses could also well be tested with one category of apps, but we wanted to limit the required number of apps per app category (more app profiles would be necessary) and avoid boredom effects by just using one category. We were not interested in effects caused by the category of the apps. The first question asked the participants to state their preferred flashlight app based on the available information. Next, they were asked to put the four flashlight apps into an order regarding their privacy-intrusiveness. After that, the participants had to answer the questions regarding perceived privacy risk and perceived trustworthiness of the apps. These questions had to be answered individually for each app. The same procedure followed in the next app category (health apps) on the following page of the survey. After this experimental part, the group splitting ended and all participants had to answer the questions about perceived privacy information quality in the SMART App Shop, perceived trustworthiness of the SMART App Shop, and enjoyment of comparison of apps in the SMART App Shop. The experiment ended afterwards.

Results Prior to hypothesis testing, we were interested in whether the two experimental groups that resulted out of the random assignment are from the same population. The assignment resulted in 33 participants in the control group and 38 participants in the experimental group. We calculated Mann-Whitney U tests5 to determine whether the two groups differ regarding the control variables age, self-efficacy, privacy concern, trust, period of using smartphones, and number of apps used regularly. The test statistics did not reveal any significant differences between the two groups regarding the tested parameters. To test the distribution regarding gender and smartphone platform used, we calculated Pearson’s chi-square test

In our data analyses, we only calculated non-parametric tests since normality tests on our data revealed that they are significantly non-normal. Thus, the assumptions for using parametric tests are not met. 5

12 Thirty Fifth International Conference on Information Systems, Auckland 2014


statistics. The test did not reveal any statistically significant difference between the two groups regarding those parameters. We therefore reject the assumption that the two data samples are from different populations regarding the tested characteristics. To test Hypothesis 1a, we conducted a Mann-Whitney U test with perceived privacy risk of the “Flashlight 4” app (H-S/L-P) as dependent variable and the experimental condition as independent variable. Perceived privacy risk of this app in the control group shows a mean rank of 33.91, while the mean rank in the experimental group is 37.82. The mean ranks differ in the expected direction. However, the test statistics revealed that the difference is statistically not significant (cf. Table 2). Hypothesis 1a is therefore not supported. To test Hypothesis 1b, we calculated a Mann-Whitney U test with perceived privacy risk of the “Flashlight 1” app (L-S/H-P) as dependent variable (cf. Table 2). The mean rank of perceived privacy risk of this app in the control group was 38.56 and in the experimental group 33.78. The test results suggest rejecting Hypothesis 1b, since the difference was statistically not significant. In the ranking task, the participants sorted the four apps according to the perceived privacy-intrusiveness. We used the assigned ranking position (1 to 4) of each app as its perceived privacy-intrusiveness value. Thus, a higher value is associated with higher perceived privacy-intrusiveness. We conducted a MannWhitney U test with the participant-assigned privacy-intrusiveness rank of the “Flashlight 4” app (H-S/LP) as the dependent variable and the experimental condition as the independent variable. The mean rank of privacy-intrusiveness of this app in the control group is 30.12 and in the experimental group 41.11. The test statistics (cf. Table 2) revealed that the mean ranks differ statistically significant, p < .01. Hypothesis 1c is therefore supported. We calculated the same test with the privacy-intrusiveness rank of the “Flashlight 1” app (L-S/H-P). The mean privacy-intrusiveness rank in the control group is 42.36 and in the experimental group 30.47. The test statistics (cf. Table 2) revealed a statistically significant difference between the two mean ranks, p < .01. Therefore, Hypothesis 1d is supported. To test Hypothesis 1e, we have designed the apps “Flashlight 1” (L-S/H-P) and “Flashlight 4” (H-S/L-P) accordingly. We conducted a Wilcoxon Signed Rank test for dependent samples (experimental group) with perceived privacy risk of “Flashlight 1” (Mdn = 3.00) and “Flashlight 4” (Mdn = 3.83) as input variables. The test revealed a statistically significant difference, z = -1.738, p < .05, r = -.28. As a consequence, Hypothesis 1e is supported. To test Hypothesis 2a, we calculated a Mann-Whitney U test with perceived trustworthiness of the “Flashlight 4” app (H-S/L-P) as dependent variable. The mean rank of perceived trustworthiness of this app in the control group is 39.06, while the mean rank in the experimental group is 33.34. The test statistics (cf. Table 2) revealed that the difference is statistically not significant. Hypothesis 2a has therefore to be rejected. To test Hypothesis 2b, we conducted the same test with the perceived trustworthiness of the “Flashlight 1” app (L-S/H-P) as dependent variable. The mean rank of perceived trustworthiness of this app in the control group was 26.62 and in the experimental group 44.14. The difference was statistically significant, p < .001 (cf. Table 2). Hypothesis 2b is therefore supported. A Wilcoxon Signed Rank test for dependent samples revealed that perceived trustworthiness of “Flashlight 1” (Mdn = 4.50) differs significantly from perceived trustworthiness of “Flashlight 4” (Mdn = 2.25) in the experimental group, z = -3.501, p < .001, r = -.57. Hypothesis 2c is therefore supported. To test Hypothesis 3, we analyzed the app preferences in the health application category, where the apps, except for “Health 2”, did not differ in the numbers of permissions requested but in the privacy risk severity. In the control group, 12.1% of the participants preferred “Health 1”, 60.6% preferred “Health 2”, 3% preferred “Health 3”, and 24.2% preferred “Health 4”. Whereas in the experimental group 15.8% preferred “Health 1”, 57.9% preferred “Health 2”, 23.7% preferred “Health 3”, and 2.6% preferred “Health 4”. A Pearson’s chi-square test revealed that there was a significant association between the experimental group and the app preferences χ2 (3) = 12.05, p