However, the verbal ability of survey respondents is one factor that can determine
... els of respondent verbal ability and by calculated grade level readability ...
Survey data quality: Does verbal ability and text readability matter? James Cole, PhD Abstract
Indiana University
This study examined data quality by levels of respondent verbal ability and grade level reading estimates of survey text. Results indicate that students with low verbal ability (relative to their college peers) were significantly more likely to take longer to complete each screen, skip items, break-off from the survey, “skip through” screens, and straight-line items. The estimates of grade level readability were accurate in predicting the occurrence of straight-lining, but not other suspicious data.
Results, continued
Results, continued Table 5. Hanging
Is low verbal ability, as compared to very high verbal ability respondents, associated with suspicious data quality?
Introduction Krosnick, Narayan, and Smith (1996) identify three regulators of “satisficing” (a source of survey data error): task difficulty, performance ability, and motivation. Task difficulty has to do with how familiar the language is to the respondent. Performance ability generally refers to the cognitive task required to recall the information needed to provide an accurate or best-estimate answer. Motivation is how willing the respondent is to provide an accurate or best-estimate answer. One-way survey researchers have tried to decrease task difficulty and accommodate low ability is to write items that use plain, simple language that is easily understood by the target sample (Fowler, 2008). Use of focus groups, cognitive interviews, and pilot testing are important components for determining task difficulty for the survey respondent. However, the verbal ability of survey respondents is one factor that can determine how difficult is the task of responding to a survey (Krosnick, 1999) and is often over looked. This study examined data quality by levels of respondent verbal ability and by calculated grade level readability using commonly used estimates of grade-level readability, Flesch-Kincaid, ARI, and FOG. One tool for aligning text difficulty with appropriate reading skills has been the use of grade-level calculators. Though not widely used in survey research, they are used in fields such as health in the development of their promotional, educational materials, as well as surveys (e.g., NCI, NIH, CDC, various state depts of health). In addition, some health science IRB’s are now starting to require this information for informed consent documents. There is some research regarding use of grade level estimations in survey research (e.g., Calderon, Morales, Liu, & Hays, 2006), but much is still not known. Research questions include: 1. Is low verbal ability, as compared to very high verbal ability respondents, associated with suspicious data quality? 2. Are commonly used readability programs that provide grade-level reading estimates associated with indicators of suspicious data quality?
As indicated in Table 1, those with low verbal scores took significantly longer to complete each screen (p
