From the People's Perspective: Assessing the ...

FROM THE PEOPLE’S PERSPECTIVE

1

From the People’s Perspective: Assessing the Representational Validity of a Coding Scheme of Citizens’ Legal Communication about Ballot Initiatives Robert C. Richards, Jr. Department of Communication Arts & Sciences, The Pennsylvania State University

Author Note Robert C. Richards, Jr., Department of Communication Arts & Sciences, The Pennsylvania State University. The author thanks Professor Dr. James P. Dillard for assistance in designing the study reported in this paper, Professor Dr. John Gastil for supervising the research and providing valuable comments on the manuscript, and David Brinker for assistance in conducting the study. Correspondence concerning this article should be addressed to: Robert C. Richards, Jr., Department of Communication Arts & Sciences, The Pennsylvania State University, University Park, PA 16802. E-mail: [email protected]


2

Abstract A weakness of coding schemes used in analyzing citizens’ legal communication about proposed laws is the lack of evidence that such codes correspond to concepts in citizens’ minds, viz., evidence of representational validity (Poole & Folger, 1981). This study aims to address that weakness by using a sorting exercise to assess the representational validity of codes from a coding scheme of citizens’ legal communication about proposed laws (Richards, 2012; Richards & Gastil, 2013). The results furnish evidence that topical concepts referred to by the codes are recognized by ordinary persons, but the extent of recognition varies. Multidimensional scaling and cluster analyses indicated that the codes were organized along two dimensions and seven clusters, six of which could be readily interpreted. Findings support suggestions in previous research that strategic and realistic cognitive schemata influenced citizens’ decision making and communication about proposed laws (Richards, 2012; Richards & Gastil, 2013). Keywords: legal communication, democratic deliberation, direct democracy, ballot initiatives, representational validity


3

From the People’s Perspective: Assessing the Representational Validity of a Coding Scheme of Citizens’ Legal Communication about Ballot Initiatives

Citizens’ legal communication in lawmaking processes warrants particular attention from legal communication scholars, because that communication concerns the implementation of political principles at the foundation of the rule of law, including those of democratic selfgovernance (Christiano, 1996) and popular control over the state (Mattson, 1998; Pettit, 2012; Richardson, 2002). Understanding the attributes of such communication is vital in order to discover the causal influences on such communication, as well as to diagnose flaws in and design effective reforms to direct-democratic processes (Gastil, 2000). Inquiry into citizens’ legal communication about proposed laws has been conducted in the domains of administrative rulemaking (Cuéllar, 2005; Epstein, Farina, & Heidt, 2014; Farina, Epstein, Heidt, & Newhart, 2012; Farina, Newhart, & Heidt, 2012; Kwon & Hovy, 2007; Kwon, Shulman, & Hovy, 2006; Shulman, 2007, 2009), constitutional reform (Gastil & Wilkerson, 2013), and ballot-initiative elections (Gastil & Knobloch, 2010; Gastil & Richards, 2013; Knobloch, Gastil, Reed, & Walsh, 2013; Richards, 2012; Richard & Gastil, 2013). To carry out this research, a number of schemes for coding attributes of citizens’ communication about proposed laws have been developed (Cuéllar, 2005; Farina et al., 2012a; Gastil & Wilkerson, 2013; Kwon et al., 2006; Richards & Gastil, 2013; Shulman, 2007, 2009), but all such schemes share a common weakness: the extent to which those codes reflect citizens’ own understanding has not been determined; that is, their representational validity has not been assessed (Poole & Folger, 1981). The question has remained unanswered: To what extent do these codes reflect citizens’ own thinking about proposed laws? Answering this question is a priority, to ensure that


4

research on communication concerning this core dimension of democracy is rooted in the social reality of the citizens being studied. The study reported in this paper seeks to address that question, by employing a sorting exercise to measure the extent to which citizens recognize topics about a proposed law, identified in previous research on citizens’ legal communication about ballot initiatives. Results were further analyzed to detect associations and distinctions between those topics. The results offer evidence that ordinary people recognize those topics, and that those topics are linked to higher level categories and dimensions, which may be related to structures or processes in citizens’ minds (Richards, 2012; Richards & Gastil, 2013). This paper proceeds as follows. First, the background of this study is described. Then the concept of representational validity is defined. Next, previous literature on citizens’ communication about proposed laws and previous research involving the coding scheme examined in this study are summarized. The research questions and methodology of this study are set out. The results are described and discussed. Limitations of the study and future research paths are outlined. Finally, potential implications of the findings are set forth in the conclusion. Background This study is part of a larger research project concerning the communication of legal information1 about ballot initiatives to citizens who are not lawyers (Richards, 2012; Richards & Gastil, 2013). In nearly half of U.S. states, citizens may use statewide ballot-initiative procedures to bypass the legislature by writing and directly enacting their own laws (Lupia & Matsusaka, 2004). Yet these initiative processes have serious flaws (Gastil & Richards, 2013), of at least two kinds. First, voters often harbor inaccurate beliefs about the legal effects of the initiatives they 1

In this paper “legal information” means structured data that express the content and consequences of rules that are enforceable by the state, or other ideas about such rules (Richards, 2009b). “Legal communication” means the exchange of legal information (Drucker, 2005; Richards, 2009a).


5

vote on (Gastil, Reedy, & Wells, 2007; Manweller, 2005; Miller 2009; Wells, Reedy, Gastil, & Lee, 2009). Second, courts frequently invalidate initiatives as inconsistent with constitutional or other law (Manweller, 2005; Miller, 2009). Both of these problems seem to stem from citizens’ failure to understand the legal aspects of ballot initiatives (Richards, 2012; Richards & Gastil, 2013). U.S. voters’ principal source of legal information about ballot initiatives is the “explanatory statement” describing each initiative, which is written by state government lawyers or persons supervised by such lawyers (Richards & Gastil, 2013). In sixteen U.S. states, explanatory statements appear in the official voter guide, a pamphlet published by the state government and distributed to voters (Brien, 2002; Cronin, 1989; Magleby, 1984, 1995). In eight states that do not publish voter guides, explanatory statements are distributed in a variety of ways, including being published in newspapers and on government Websites and posted at polling places.2 If voters are not being adequately informed about the legal aspects of ballot initiatives, and explanatory statements are voters’ chief source of such legal information, then flaws in those statements may contribute significantly to the gap in voters’ legal knowledge about initiatives. Yet how should explanatory statements’ legal communication be evaluated? Two bodies of theory suggest ways of addressing this question. The first is plain legal language (PLL) theory (Barnes, 2006; Tiersma, 1999). According to this theory, the understanding of legal information by individuals who are not lawyers is expected to increase to

2

See Arkansas Code Annotated § 7-9-114 (2014); Florida Statutes, tit. IX, § 101.161 (1)-(2) (2014); Florida Administrative Code §1S-2.009 (2014); Michigan Constitution, art. XII, § 2 (2014); Michigan Compiled Laws § 168.22e (2014); Missouri Revised Statutes § 116.025 (2014); Nevada Constitution, art. 19, § 2(4); Nevada Revised Statutes § 295.170; North Dakota Century Code §§ 16.1-01-07, 16.1-06-09 (2014); Oklahoma Statutes § 34-17 (2014); South Dakota Codified Laws §§ 12-13-1, 12-13-9.1, 12-13-11, 12-13-23, 12-13-25.1, 12-16-15, 12-16-16 (2014).


6

the extent that the communication of such information to them is consistent with the individuals’ own practices of legal communication (Barnes, 2006; Tiersma, 1999). In the ballot-initiative setting, PLL theory predicts that legal communication practices in explanatory statements are likely to be effective to the extent that such practices are consistent with citizens’ own practices of communicating about the legal aspects of initiatives. PLL theory thus suggests that evaluating the effectiveness of legal communication in explanatory statements requires the development of a descriptive model of citizens’ legal communication about proposed laws. The second relevant body of theory is social cognition (Fiske, 1993; Kunda, 1999). According to social cognitive theory, individuals’ cognition of and behavior toward others, including communicative behavior, are influenced by processes and structures in their minds, such as heuristics and schemata (Fiske, 1993; Kunda, 1999). In the legal domain, social cognitive theory and research have furnished evidence of the influence of cognitive structures in individuals’ decision making and communication in several contexts (Kovera & Borgida, 2010; Spellman & Schauer, 2013). For example, Pennington and Hastie (1986, 1988, 1991, 1992) demonstrated that a narrative schema in jurors’ minds leads jurors to impose a story-structure on trial evidence. Sunwolf (2006, 2010) has shown that a “decisional regret” cognitive schema leads jurors to share stories about potential adverse consequences of their decisions. Theories of social cognition suggest that a descriptive model of citizens’ legal communication about ballot initiatives should account for cognitive structures and processes that may influence citizens’ thinking and communication about the legal aspects of initiatives. From such a descriptive model, a normative model of initiative legal communication can be developed, that specifies the attributes of effective communication to voters about the legal


7

aspects of initiative. This normative model can be used to evaluate explanatory statements for their effectiveness in communicating about the legal aspects of ballot initiatives. In order to analyze citizens’ legal communication about ballot initiatives, a coding scheme of the attributes of such communication was necessary. Analysis of citizens’ deliberations about ballot initiatives during the 2010 Oregon Citizens’ Initiative Review (CIR) (Knobloch et al., 2013) indicated that such a coding scheme should cover at least three communicative dimensions: topics—such as bases for legal challenges—functions—such as evaluation—and discursive modes, especially narrative (Richards, 2012; Richards & Gastil, 2013). In addition, it was desired that the representational validity of such a scheme should be assessed. Representational Validity Representational validity, according to Poole and Folger (1981, p. 26), “reflects the meanings of utterances in the culture studied.” This type of validity is thus equivalent to the emic perspective on communication, meaning understandings held by the members of the community being studied (Keating, 2001, p. 288; Pike, 1954). In the present research, a coding scheme would be considered to have representational validity to the extent that it designates concepts present in the minds of ordinary citizens when they think or communicate about proposed laws. To determine whether a coding scheme had already been developed that was suitable for use in this research, and to identify previous research on legal communication via explanatory statements, a search of political communication and legal studies literature was conducted. Literature Review No empirical study of the communication of legal information in explanatory statements appears to have been published. Although some studies have identified broad classes of information contained in voter guides (California Commission on Campaign Financing, 1992;


8

Center for Governmental Studies, 2008; Sutro, 1994), those classes have not included legal communication attributes. Of the studies that have measured the readability of voter guides (California Commission on Campaign Financing, 1992; Center for Governmental Studies, 2008; Kruse, 2001; Magleby, 1984), none has identified particular legal communication attributes of voter guides, or related those attributes to readers’ understanding of legal information about initiatives. Wells, Reedy, Gastil, and Lee (2009) found discrepancies between voter-guide descriptions of the legal effects of a ballot initiative and voters’ understanding of those effects, but did not identify particular attributes of legal communication in voter guides, or examine associations between such attributes and shortcomings in voters’ knowledge about initiatives. Regarding research on various modes of communication used in explanatory statements, Freelon, Kriplean, Morgan, Bennett, and Borning (2012) explored the use of a Web-based crowdsourcing platform for a voter guide, but did not address legal communication attributes of the guide. Attributes of citizens’ legal communication about proposed laws have been identified in some empirical research on democratic deliberation. For example, the “topic dictionaries” that Gastil and Wilkerson (2013) created to classify citizens’ deliberations during the Australian Citizens’ Parliament (ACP) include two law-related topics: “Rights” and “Referendum,” and five other topics containing law-related subtopics: “Duplication,” “Elections,” “Indigenous,” “Republic,” and “Term/Office,” which encompass current and proposed statutes and constitutional provisions (pp. 149, 151). Gastil and Wilkerson (2013) do not report codes for functions or discursive modes, or discuss representational validity. Black and Lubensky (2013) described narratives shared by ACP participants, but did not discuss legal topics or functions. Two studies of citizens’ communication during the Oregon Citizens’ Initiative Review, other than preliminary studies to the research reported here, have addressed CIR panelists’ legal


9

communication. Gastil and Knobloch (2010) and Knobloch et al. (2013) identified two functions— inquiring and persuading—and five legal topics raised by 2010 Oregon CIR panelists: the intended effects, probable effectiveness, potential unintended consequences, and implementation of initiatives, as well as other jurisdictions’ laws. These studies do not address discursive modes or the representational validity of the specified topics and functions. Research on public comments about proposed administrative regulations has also identified attributes of citizens’ legal communication. The items in Cuéllar’s (2005, p. 431) “sophistication” scale measuring attributes of public comments about proposed regulations concern four topics— the content of the proposed law, its status (or genre) as a regulation, statutes in force, and alternative means of achieving the policy objective—and two functions of citizens’ legal communication: the application of law to facts, and persuasion through “logical argument.” In addition, Cuéllar’s (2005, p. 432) “categories of commenter concern” can be grouped into law-related topical classes: the policy objectives, content, compliance costs, and language of proposed laws, reasons for choosing lawmaking rather than non-legal approaches, alternative means for addressing the policy objective, and public participation in the regulatory process. Cuéllar (2005) does not discuss discursive modes or representational validity. Kwon and Hovy (2007; Kwon et al., 2006) used topical codes to classify citizens’ comments on a proposed environmental regulation. The code called “Legal” covers laws in force as well as “legal proceedings,” and other codes include law-related subtopics: “Economic” (encompassing compliance costs), “Environment” and “Health (policy objectives), “Government responsibility” (policy objectives and content of the proposed regulation), “Policy” (content of the proposed regulation), and “Pollution” (the need for the proposed regulation) (Kwon et al., 2006, p. 158). The authors also identify the topic of reasons supporting arguments related to the


10

proposed regulation. Further, Kwon et al. (2006) describe two functions of citizens’ legal communication: persuasion (in the “Removing”, “Reasoning,” and “Request” frames, and the account of argument structure) and seeking information (in the “Request” frame) (p. 164). Moreover, Kwon et al.’s (2006) opinion classes regarding the proposed regulation—supporting, opposing, and “proposing a new idea” (p. 159) —relate to the functions of evaluating and expressing opinions (not necessarily for purposes of persuasion) about proposed laws. These authors do not address discursive modes or representational validity. Shulman’s (2007, 2009) coding schemes designating features of citizens’ comments on a proposed environmental regulation cover topics, functions, discursive mode, and affect. The topical code “Legal” covered laws in force and case law, while other codes include law-related subtopics: “Agency Mission” (covering legal responsibilities of the regulating agency), “Children’s Health” and “Public Interest or Health” (policy objectives), “Economic” / “Economic Issues” (compliance costs), “Proposal” (changes to the proposed regulation), “Public Health and Safety” (policy objectives), and “Suspicion-Corruption” (the rulemaking process) (Shulman, 2007, pp. 63-64; 2009, p. 48). In addition, the “Personal Experience” code includes the narrative discursive mode (Shulman, 2007, p. 64; 2009, p. 48), and several codes refer to the function of persuasion. Neither study addressed the representational validity of the codes. Farina et al. (2012a, 2012b; Epstein et al., 2014) have employed a scheme consisting of three topics, one function, and one discursive mode to classify citizens’ comments on proposed regulations. One topic, “Unintended Consequences,” is expressly law-related, while the others encompass law-related subtopics: “complexity … of interests and practices” includes implementation, effectiveness, and compliance costs of the proposed regulation, and “contributory context” covers effectiveness (Farina et al., 2012a, pp. 1204, 1209, 1211). Further,


11

the functional code, “reframing the regulatory issues,” also designates the topics of the need for, the symbolic effects of, and unintended consequences of the proposed regulation (Farina et al., 2012a, p. 1214). Moreover, Farina et al. (2012a, 2012b, Epstein et al., 2014) include in their classification scheme the narrative discursive mode. The authors do not address the representational validity of their codes. Bryer’s (2013) scheme for coding citizens’ comments about proposed regulations includes two law-related codes: opinions that support or oppose the proposed regulation. Bryer (2013) does not discuss functions, discursive modes or representational validity. Some findings from research on citizens’ legal communication in litigation are relevant to this inquiry. First, Pennington and Hastie (1986, 1988, 1991, 1992) showed that a narrative schema strongly influences jurors’ application of law to facts. This is relevant to the ballotinitiative context because deliberating citizens frequently discuss the application of proposed initiatives to hypothetical factual scenarios (Richards, 2012; Richards & Gastil, 2013). In addition, Sunwolf’s (2006, 2010) studies of decisional regret theory suggest that a “decisional regret” cognitive schema leads jurors to experience “anticipated regret” and then to share narratives about potential adverse consequences of their verdicts. This relates to the directdemocracy setting because unintended consequences of ballot initiatives are a common topic of discussion among deliberating citizens (Richards, 2012; Richards & Gastil, 2013). Pennington and Hastie’s (1986, 1988, 1991, 1992) and Sunwolf’s (2006, 2010) lines of research suggest that, in the ballot-initiative setting, citizens’ use of narrative discourse is likely frequently to occur in association with the function of applying law to facts, and with the topic of unintended consequences. Further, those lines of research suggest that particular cognitive structures may


12

influence associations between storytelling, the application function, and the discussion of unintended consequences. Thus previous research discloses little about the legal communication attributes of ballotinitiative explanatory statements, but suggests a number of attributes with which to develop a coding scheme of citizens’ legal communication about proposed laws. The representational validity of previously identified codes has not been assessed, however. Previous Research and Development of the Coding Scheme Earlier stages of this research have been reported in four previous studies (Gastil & Richards, 2013; Richards, 2010, 2012; Richards & Gastil, 2013). Richards (2010) analyzed the descriptions of the legal communication practices of government lawyers who advise non-lawyer official legislators and regulators about proposed laws, 3 and identified twenty-one attributes of legal communication about proposed laws. These descriptions were considered indirect accounts of the attributes of citizens’ legal communication about ballot initiatives, because the latter are assumed to be closely analogous to those of official legislators and regulators who are not lawyers. During a content analysis of the transcripts of the 2010 Oregon CIR, those twenty-one initial codes were used, and open coding (Lindlof & Taylor, 2011) was performed for purposes of adding new codes in order to designate additional legal communication attributes observed during the content analysis (Gastil & Richards, 2013; Richards, 2012; Richards & Gastil, 2013). Four hundred and eighteen additional codes were added to the coding scheme, bringing the total number of codes to 439. Of the twenty-one initial codes, seventeen were observed in the

3

See Carter, 2001; Clark, 1998; Coonjohn, 1994; Edwards, 1987; Glennon, 1998; Grant, 2001; Marcello, 1996; Marchant, 2002; McGarity, 1998; Powers, 2002; Purdy, 1987.


13

transcripts of the citizens’ deliberations.4 The four codes not observed in the CIR transcripts5 described topics that were likely to arise only in litigation, and that would not be expected to be reflected in citizen-to-citizen communication about proposed laws. The observation of seventeen of the initial codes in the transcripts of citizens’ discussions during the 2010 Oregon CIR can be viewed as some evidence of the validity of those codes (DeVellis, 2012). To simplify the expanded coding scheme, semantically similar codes were aggregated into 42 broad categories (Lindlof & Taylor, 2011). The coding scheme currently has two levels: a granular level consisting of 439 codes, and a more general level consisting of 42 broad categories.6 The coding scheme includes most of the attributes of citizens’ legal communication about proposed laws identified in the published literature reviewed above.7

4

Those seventeen were: 1. Bases for Legal Challenges to the Proposed Law; 2. Delegation of Regulatory Authority by the Proposed Law; 3. Explanation of the Proposed Law (Detailed); 5. History of Proposed Law (i.e., events following the origin of the proposed law); 6. Language Choices Made in Phrasing the Proposed Law; 7. Legal Effects of the Proposed Law, on Laws Having a Topic Different from the Proposed Law; 8. Legal Effects of the Proposed Law, on Laws Having Same Topic as the Proposed Law; 10. Origin of the Proposed Law; 11. Other Jurisdictions’ Laws Similar to the Proposed Law; 12. Alternative Means Other than Lawmaking of Achieving the Policy Objectives of the Proposed Law; 14. Policy Objectives of the Proposed Law; 15. Policy Reasons for Choosing Lawmaking Rather than Alternative Means of Achieving the Policy Objectives; 16. Effects of the Proposed Law on Public Administration and Enforcement; 17. Regulations In Force that Relate to the Proposed Law; 19. Statutes In Force that Relate to the Proposed Law; 20. Statutory Framework that Forms Part of the Context of the Proposed Law; 21. Summary of the Proposed Law. 5 Those four were: 4. Factors Courts Will Use in Interpreting the Proposed Law; 9. Location of the Proposed Law in a Statutory or Administrative Code; 13. Broad Policy Framework that Forms Part of the Context of the Proposed Law; and 18. Regulatory Framework that Forms Part of the Context of the Proposed Law. 6 The topical categories were: 1. Administration or Enforcement of Law; 2. Alternative Policy Approaches; 3. Bases for Legal Challenges; 4. Deliberation Process; 5. Descriptions of Proposed Law (e.g., in Ballot Title, Voters' Guide, etc.); 6. Facts or Evidence; 7. Fiscal / Budgetary Issues; 8. Consequences of the Proposed Law; 9. Negative or Unanticipated Consequences; 10. All Other Consequences. 11. Effectiveness of the Proposed Law; 12. Means (i.e., the Proposed Law as an instrument for achieving intended effects); 13. Need for the Proposed Law; 14. Objectives or Purposes of the Proposed Law; 15. Other Instrumental Aspects of the Proposed Law; 16. Language or Drafting of Law; 17. Law Currently in Force (in this jurisdiction); 18. Legislative Procedure; 19. Other Jurisdictions' Laws or Policies; 20. Political / Policy Issues or Interests; 21. Proposed Law [including descriptions, explanations, history, origins, etc.]; 22. Regulations or Rulemaking; 23. Values; 24. Voters’ or Citizens’ Roles in Lawmaking; 25. Witnesses’ / Experts’ Opinions of the Proposed Law. The functional categories were: 26. Applying Law to Facts; 27. Comparing Laws or Facts; 28. Evaluating Laws; 29. Explaining Laws / Sharing Information; 30. Persuading Others. The discursive-mode categories were: 31. Narrative; 32. Non-Narrative. The motivational categories, applicable only to narratives, were: 33. Express Anticipated Regret; 34. Express Anticipated Regret and Responding with a Positive Counterfactual Narrative; 35. Compare Proposed Law or Facts to Similar Laws or Facts; 36. Consider Alternative Policy Approaches or Reforms; 37. Evaluate Laws; 38. Express Opinions [without necessarily


14

The major findings from this stage of the research were as follows. Regarding the narrative discursive mode, Oregon CIR 2010 participants were observed frequently to tell stories, most often in connection with the function of applying law to facts, and frequently in connection with the topic of unintended consequences and the motivations8 of expressing anticipated regret and seeking information (Richards, 2012; Richards & Gastil, 2013). In all modes of discourse, 2010 Oregon CIR participants frequently discussed the topics of unintended consequences and policy objectives of proposed laws, as well as the functions of applying law to facts and evaluating laws. Three tentative conclusions were drawn from these findings: That a “strategic” schema in CIR participants’ minds may have influenced them to discuss the policy objectives of proposed laws and to evaluate those laws in terms of the likelihood of achieving those policy objectives; that a “realistic” cognitive schema, similar to Sunwolf’s (2006, 2010) decisional-regret schema, may have influenced CIR participants to tell stories about unintended consequences of proposed laws; and that CIR participants used narrative discourse to apply the proposed law to hypothetical factual situations in order to educate themselves about the legal nature and

intending to persuade]; 39. Engage in Humor; 40. Persuade Others; 41. Seek Information or Understanding; 42. Share Information. 7 Two topics and one function identified in previous literature are not reflected in the current coding scheme: the topics of legal proceedings (Kwon et al., 2006), and the costs of complying with the proposed law (Cuéllar, 2005; Epstein et al., 2014; Farina et al., 2012a, 2012b; Kwon et al., 2006; Shulman, 2006, 2009), and the function of reframing the issues (Epstein et al., 2014; Farina et al., 2012a, 2012b). These attributes are not currently included in the coding scheme because they were observed neither in the government lawyers’ accounts of their legal communication with official lawmakers (Richards, 2010) nor in the 2010 Oregon CIR transcripts (Richards, 2012; Richards & Gastil, 2013). The topics of legal proceedings and compliance costs will be added to the coding scheme and used in the next stage of this research. Because the function of reframing issues is difficult to operationalize, more research is needed before a decision is made regarding whether it will be added to the coding scheme. 8 The coding scheme originally treated motivations as distinct from functions, but during the coding of transcripts of citizens’ deliberations from the 2010 Oregon CIRs it was discovered that motivations often cannot readily be distinguished from functions during content analysis. Therefore, codes and categories for motivations will be transferred to the “functions” component of the coding scheme in the next stage of this research.


15

consequences of initiatives (Richards, 2012; Richards & Gastil, 2013).9 These conclusions suggested that a normative model of the communication of legal information about proposed laws should include the topics of Policy Objectives (relating to the strategic schema), Unintended Consequences (relating to the realistic schema), and the functions of Evaluating the Proposed Law (relating to the strategic schema) and Applying Law to Facts (relating to the learning activity described above, which may be influenced by a narrative schema), and should employ the narrative discursive mode in connection with the topic of Unintended Consequences and the application function. A study applying this normative model to the explanatory statements for the initiatives examined during the 2010 Oregon CIRs showed that the statements did not address any of those topics or functions, except for one mention of the application function (Gastil & Richards, 2013; Richards & Gastil, 2013). Thus, initial studies have furnished some evidence of the validity of seventeen codes in the initial coding scheme, and in those studies the scheme has been expanded to cover additional attributes, patterns in citizens’ legal communication about ballot initiatives have been identified and tentative explanations for those patterns have been set forth, and a preliminary normative model of the communication of legal information about proposed laws to citizens has been produced and tested. The next task is to assess the representational validity of the coding scheme. Description of This Study After a determination that the topical portion of the coding scheme could be treated as analytically distinct from the functional, discursive-mode, and motivational portions, a decision was made to undertake a study to assess the representational validity of the topical codes in the

9

Oregon CIR participants’ use of narrative in connection with the application function may be influenced by a narrative cognitive schema (Pennington & Hastie, 1986, 1988, 1991, 1992).


16

coding scheme. To evaluate the representational validity of the topical codes, a sorting exercise was designed, similar to those frequently used by communication researchers (Goldsmith & Baxter, 1996, Study 3; Roskos-Ewoldsen & Roskos-Ewoldsen, 2008). Research Questions This study was intended to address the following research questions: RQ1. To what extent do subjects recognize topical codes from a coding scheme of attributes of citizens’ legal communication about proposed laws, as evidenced by items associated with each code being grouped together, in the results of a sorting exercise in which subjects are asked to group together items associated with topical codes? RQ2. What higher-level dimensions of the topical codes are revealed through analysis of the results of the sorting exercise? RQ3. What distinctions and associations among topical codes are revealed through analysis of the results of the sorting exercise? Methodology Materials and Measures In selecting topical codes to include in the study, the goal was to include as many topical codes and categories from the coding scheme as possible, while keeping the number of items to be sorted low enough to enable subjects feasibly to organize them during a single session. To accomplish this goal, several steps were taken. First, the seventeen original codes that had been validated during the previous stage of the research were reduced to twelve,10 by combining those that were closely semantically similar and unlikely to be distinguished by ordinary people. 10

The twelve codes were: 1. Bases for Legal Challenges to the Proposed Law; 2. Delegation of Regulatory Authority by the Proposed Law; 5. History/Origins of Proposed Law (i.e., events following the origin of the proposed law); [incorporates: 5. History of the Proposed Law; and 10. Origin of the Proposed Law.]; 6. Language Choices Made in Phrasing the Proposed Law; 7. Effects of the Proposed Law on Other Laws [incorporates: 7. Legal Effects of the Proposed Law, on Laws Having a Topic Different from the Proposed Law; and 8. Legal Effects of the


17

Second, the categories not associated with those twelve codes11 were reduced to six in number, by omitting those that were not strictly law-related or were difficult to operationalize in a sorting exercise.12 In each of the remaining six categories, the code most frequently observed during the previous stage of research was chosen for inclusion in this study.13 This brought the number of topical codes to be examined in the study to eighteen. (See Tables 1 and 2.) Next, 54 items for the sorting exercise were created by selecting for each topical code three passages from the transcripts of citizens’ deliberations during the 2010 Oregon CIR to which the code had been assigned during the previous stage of research (Richards, 2012; Richards & Gastil, 2013 ).14 Each passage was no more than 40 words in length. Passages chosen were those considered most likely to convey their meanings to those unfamiliar with the context

Proposed Law, on Laws Having Same Topic as the Proposed Law]; 11. Other Jurisdictions’ Laws Similar to the Proposed Law; 12. Alternative Means Other than Lawmaking of Achieving the Policy Objectives of the Proposed Law; 14. Policy Objectives of the Proposed Law; 15. Policy Reasons for Choosing Lawmaking Rather than Alternative Means of Achieving the Policy Objectives; 16. Effects of the Proposed Law on Public Administration and Enforcement; 17. Regulations In Force that Relate to the Proposed Law; 19. Statutes In Force / Statutory Framework that Relate to the Proposed Law [incorporates: 20. Statutory Framework that Forms Part of the Context of the Proposed Law]. In addition, it was decided to consolidate the codes 3. Explanation of the Proposed Law (Detailed), and 21. Summary of the Proposed Law with the “Means” code, taken from the remaining categories. 11 The categories corresponding to the twelve topical codes were: 1. Administration or Enforcement of Law (Code 16); 2. Alternative Policy Approaches (Code 12); 3. Bases for Legal Challenges (Code 1); 5. Consequences of Proposed Laws (Codes 7 & 8); 14. Objectives or Purposes (Code 14); 16. Language or Drafting of Law (Code 6); 17. Law Currently in Force (in this jurisdiction) (Code 19); 19. Other Jurisdictions’ Laws or Policies (Code 11); 21. Proposed Law [i.e., the law contained in the ballot measure that is the subject of the deliberation] (Codes 3, 5, 10, 15, 21); 22. Regulations or Rulemaking (Code 17). 12 The following topical categories were dropped: 4. Deliberation Process; 5. Descriptions of Proposed Law (e.g., in Ballot Title, Voters' Guide, etc.); 6. Facts or Evidence; 10. All Other Consequences of the Proposed Law; 15. Other aspects related to the instrumental view of the proposed law; 18. Legislative procedure; 20. Political / Policy Issues or Interests; 23. Values; 24. Voters’ or Citizens’ Roles. The following topical categories were retained: 7. Fiscal / Budgetary Issues; 9. Negative or unanticipated consequences; 11. Effectiveness of the Proposed Law; 12. Means [i.e., the Proposed Law viewed as an instrument for achieving intended effects]; 13. The Need for the Proposed Law; 25. Witnesses’ / Experts’ Opinions of Law. 13 Those six additional topical codes were: Effectiveness of Proposed Laws (Category 11); Experts’ Opinions of Proposed Law (Category 25); Fiscal Effects of Proposed Laws (Category 7); Means, including Summaries, Descriptions, and Explanations of the Proposed Law (Codes 3 and 21) (Categories 12 and 21); Negative or Unintended Consequences of Proposed Laws (Category 9); the Need for the Proposed Law (Category 13). 14 Full text of the passages is available at: https://dl.dropboxusercontent.com/u/16652392/SortingExercise1ItemswithConcepts10-6-14.pdf .


18

of the 2010 Oregon CIRs. Where transcripts offered no suitable passages, fictional passages were written in colloquial phrasing intended to resemble the passages taken from the transcripts.15 Subjects Subjects were 120 undergraduate students enrolled in an introductory speech course at a large public university in the northeastern United States. Subjects were given the choice to participate in the study in exchange for course credit or to complete an alternative assignment. A total of 109 subjects completed the study, of whom 41% were male and 59% female. They ranged in age from 18 to 33 years, with median and mean of 21. Seventy-seven percent of subjects completing the study were White, 9% Asian or Asian American, 4% African or AfricanAmerican, 4% Hispanic or Latino/a, 2% Middle Eastern, and 3% of mixed race. Twenty-nine percent of subjects completing the study identified as politically moderate, 17.4% as conservative, none as very conservative, 25.7% as liberal, 6.4% as very liberal, and 21% unsure. Procedures

The study was conducted online from November 4-December 10, 2013. Before the start of the study, each subject was sent an email message containing participation instructions and a unique numeric identifier. When the study had begun, subjects logged on to the university’s online research participation platform, and then to a secure area of the OptimalWorkshop online card-sorting platform, where an informed-consent form was displayed. Consenting subjects were shown instructions for the study and then prompted to enter their numeric identifier in a Web form. Subjects who declined to consent were taken to the alternative assignment. Consenting subjects were then asked to read a fictional ballot initiative and explanatory statement concerning imposing mandatory-minimum criminal sentences for repeat offenses

15

Fictional passages were written for six items: three items for Means, and three items for Experts’ Opinions of Proposed Laws. Minor edits were made to passages used in several other items to improve clarity.


19

concerning driving while intoxicated or committing certain sex crimes.16 After reading those materials, subjects were taken to an online sorting space in which all 54 items were displayed onscreen on digital “cards.” Subjects were instructed to sort the cards into eighteen piles, with each pile containing items whose statements belonged together, in the subject’s view, and then to label each pile with a name that, in the subject’s view, best described the items in the pile. For piles for which the subject could not think of a label, the subject was instructed to use the label “Not sure.” After completing the sorting and labeling tasks, subjects were asked to complete a questionnaire containing twelve four-point Likert-type items measuring cultural worldview (Kahan et al., 2007), as well as demographic items.17 18 Data Analysis Examination of the similarity matrix, and multidimensional scaling (MDS) and cluster analysis, were used to analyze the data. First, the similarity matrix, displaying the number of subjects who grouped together each pair of items, was manually examined. For 12 of the 18 codes, mean frequency was calculated for the three items associated with each code. For the other six codes mean frequency was calculated for all three items and for the two highest-loading items (Table 1), because one item of the three grouped weakly with the others.19 Next, in preparation for MDS and cluster analysis, similarity data were converted to dissimilarity frequency scores. These were then transformed using the Delta method, to enhance co-occurrence information and goodness of fit and to prevent skewing (Roskos-Ewoldsen & Roskos-Ewoldsen, 2008; Van der Kloot & Van Herk, 1991). 16

The initiative was modeled on Oregon’s 2010 statewide Measure 73, which was the subject of the first 2010 Oregon Citizens’ Initiative Review (Knobloch et al., 2013). 17 Cultural worldview data were not used in the analyses presented here and so are not reported. 18 Subjects were not asked to complete items rating passages along possible multidimensional scaling dimensions (Roskos-Ewoldsen & Roskos-Ewoldsen, 2008), since results of prior research had not clearly indicated the nature or number of possible dimensions. 19 A code was considered to have been grouped with others “weakly” if it was grouped with at least one of the other items associated with its code, by fewer than 10% of subjects.


20

To identify higher-level dimensions associated with items and codes, MDS was conducted on the Delta-transformed dissimilarity data using PROXSCAL in SPSS (Borg & Groenen, 2005; Borg, Groenen, & Mair, 2013).20 To determine the appropriate number of dimensions, models were estimated for one through five dimensions; stress, dispersion accounted for, and Tucker’s coefficient of congruence (Borg & Groenen, 2005; Tucker, 1951) were calculated; a scree plot was created; and Kruskal and Wish’s (1978) “elbow” method and 0.05 stress threshold were used. To determine associations and differentiation of the sorted items by category, a cluster analysis was performed using ultrametric trees (Aldenderfer & Blashfield, 1984; RoskosEwoldsen & Roskos-Ewoldsen, 2008). Preliminary analyses were performed on both the frequency dissimilarity data and the Delta-transformed dissimilarity data, using single-, average-, and Ward-linkage (Aldenderfer & Blashfield, 1984; Bailey, 1994; Roskold Ewoldsen & Roskold Ewoldsen, 2008). Single-linkage and average-linkage methods yielded highly imbalanced clusters21 (Aldenderfer & Blashfield, 1984). Since similarity matrix and MDS results suggested that more balanced clusters more accurately represented the data, the Ward method was chosen, as it generally yields “clusters of relatively equal sizes” (Aldenderfer & Blashfield, 1984, p. 43). Because the Ward method is “sensitive to transformations of the data” (Roskos-Ewoldsen & Roskos-Ewoldsen, 2008, p. 300), cluster analyses were performed using the frequency dissimilarity data. To determine the appropriate number of clusters, a scree plot of distance accounted for by each additional dimension was constructed and the “elbow” test used to identify 20

PROXSCAL was configured for ordinal proximity transformations in which tied observations remained tied, there were no restrictions on common space, and the Torgerson initial configuration was used (Borg & Groenen, 2005). 21 On frequency dissimilarity data both single-linkage and average-linkage methods produced four clusters, each containing three items, and one cluster containing 42 items, whereas on Delta-transformed data the single-linkage method yielded four clusters of which three contained three items each and one contained 45 items, and the averagelinkage method produced five clusters, three of which contained three items each, one of which contained a single item, and one contained 44 items.


21

the number of clusters after which additional clusters yielded diminishing returns (Aldenderfer & Blashfield, 1984; Roskos-Ewoldsen & Roskos-Ewoldsen, 2008). The plot indicated a sevencluster solution. Hierarchical cluster analysis for a seven-cluster solution was then performed in SPSS, with Ward’s method and intervals measured with squared Euclidean distances. Results Results: Similarity Matrix Average frequencies for groupings of items associated with each code appear in Table 1. Groupings of three codes were very pronounced, as their items were grouped together by an average of more than 50% of subjects: Fiscal Effects of the Proposed Law, Experts’ Opinions, and Other Jurisdictions’ Laws. Groupings of six codes were somewhat less marked, with average frequency of 20-49%: Delegation of Regulatory Authority, Bases for Legal Challenges, History/Origins of the Proposed Law, the Need for the Proposed Law, Language Choices Affecting the Wording of the Proposed Law, and the Effects of the Proposed Law on Other Laws. Groupings of eight codes were less strong, though still with an average frequency of greater than 10% of subjects: Alternative Means for Choosing the Policy Objective, Policy Reasons for Choosing Lawmaking, Regulations in Force, Policy Objectives of the Proposed Law, Negative or Unintended Consequences of the Proposed Law, Effectiveness of the Proposed Law, the Proposed Law as a Means for Achieving the Policy Objective, and Public Administration Effects of the Proposed Law. The items for Statutes in Force grouped together relatively weakly, with an average frequency of fewer than 10% of subjects.


22

Table 1. Frequencies of Groupings of Code-Related Items, of Sorted Items Designating Topics of Citizens’ Legal Communication about Proposed Laws Code Name Fiscal Effects Experts’ Opinions Other Jurisdictions’ Laws Delegation of Regulatory Authority Bases for Legal Challenges History / Origins of Proposed Laws Need Language Choices Effects on Other Laws Alternative Means Policy Reasons for Choosing Lawmaking Regulations in Force Policy Objectives Unintended Consequences Effectiveness Means Public Administration Effects Statutes in Force

No. of Items

Mean Frequency

Mean %

3 3 3

66.0 64.7 61.3

60.6% 59.3% 56.3%

3

48.0

44.0%

3

43.7

40.1%

3 3 3 3 3 [2]

35.7 30.3 27.7 24.3 20.7 [30.0]

32.7% 27.8% 25.4% 22.3% 19.0% [27.5%]

3 3 3 [2] 3 3 [2] 3 [2]

19.0 16.7 14.0 [37.0] 14.0 13.7 [20.0] 13.3 [32.0]

17.4% 15.3% 12.8% [33.9%] 12.8% 12.5% [18.3%] 12.2% [29.4%]

3 [2] 3 [2]

11.7 [17.0] 10.0 [12.0]

10.7% [15.6%] 9.2% [11.0%]

Note. N = 109 subjects. Data are from the similarity matrix. Brackets indicate figures for codes when the single weakest item associated with the code is omitted. The single weakest item is an item that fewer than 10% of subjects grouped with at least one other item associated with the code. “Mean Frequency” is the arithmetic mean of the frequency with which subjects grouped together each pair of items associated with the code. “Mean %” is the Mean Frequency divided by the total number of subjects (109). See text for details. Six of the codes were found to have a single “weak” item, however, which fewer than 10% of subjects grouped with at least one other item associated with the same code. When those weak items are dropped, the strength of the groupings of the remaining items associated with these codes increases, often substantially (Table 1, figures in brackets): the frequencies for Policy Objectives, Means, and Alternative Means exceed 20% of subjects (rising to 33.9%, 29.4%, and 27.5%, respectively), those for Effectiveness and Public Administration Effects increase within the lower tier (to 18.3% and 15.6%, respectively), and that for Statutes in Force reaches 11% of subjects.


23

Results: Multidimensional Scaling Regarding the MDS analysis, a two-dimensional solution yielded stress of 0.059, dispersion accounted for of 0.94, and coefficient of congruence of 0.97, with diminishing returns for each additional dimension added thereafter. 22 As discussed below, the two-dimensional solution also yields readily interpretable dimensions. Therefore, applying Kruskal and Wish’s (1978) “elbow” test and considering the proximity of the result to the 0.05 stress threshold, the two-dimensional solution was chosen. Applying thresholds commonly used in factor analysis (Brown, 2006), loadings of items on each dimension were categorized as “weak” for values |0.100| through |0.399|, and “moderate to strong” for values ≥|0.400|. Items with loadings of