Uncertainty in Environmental Risk Assessment

CHAPTER 9

UNCERTAINTY IN ENVIRONMENTAL RISK ASSESSMENT

Glenn W. Suter II

A bstract--Risk is a result of uncertainty concerning the consequences of a hazardous situation. This paper discusses three aspects of the uncertainty associated with environmental risks: (1) the sources of uncertainty, (2) the types of uncertainties associated with three classes of assessment endpoints, individuals, aggregates, and systems, and (3) the ways that uncertainty is realized in the assessment process. The occurrence of uncertainty and the potential for validation of methods are discussed for assessments based on mathematical models, observational studies, and experimental studies. Finally, ways to reduce uncertainty in risk assessments are discussed, and examples of the utility of uncertainty analysis in risk assessment are presented.

Although all scientific activities are concerned with identifying and reducing uncertainty, uncertainty plays a particularly important role in risk assessment. Without uncertainty there is no risk (R. Wilson and E. A. C. Crouch, 1987). Risk arises from the existence of a hazard and some uncertainty about its effects. A hazard is a hypothesized conjunction of a potentially harmful agent and a potentially susceptible subject. An example is the occurrence of an animal carcinogen in human drinking water. Risk is the probability that a specified harmful effect will occur, or, in the case of a graded effect, the relationship between the magnitude of the effect and its probability of occurrence. If it is certain that the undesired event will occur or that it will not occur (i.e., the probability of occurrence of an exactly specified event is known to be 1 or is known to be 0), there is no risk.

G. M. von Furstenberg, Acting under Uncertainty: Multidisciplinary Conceptions © Springer Science+Business Media Dordrecht 1990

204

Acting Under Uncertainty: Multidisciplinary Conceptions

There have been many discussions of the nature of uncertainty in risk assessment and its estimation. The reader is referred to Owen Hoffman and Charles Miller (1983), G. W. Suter, L. W. Barnthouse and R. V. O'Neill (1987), and W. D. Rowe (1977). Those discussions emphasize error· in mathematical models. This paper differs in acknowledging that many assessments, or major components of assessments, make little or no use of mathematical models but rather are based on experiments or observations. Nevertheless, since all three approaches generate quantitative estimates of the response of systems to hazards, a common language and set of concepts could be useful in planning and carrying out assessment programs. This paper discusses three aspects of uncertainty (Table 1): (1) sources of uncertainty, (2) types of uncertainty associated with three classes of assessment endpoints, individuals, aggregates, and systems, and (3) the ways that uncertainty is realized in the assessment process. The occurrence of uncertainty and the potential for validation of methods are then discussed for studies based on mathematical models, observational studies, and experimental studies. Finally, ways to reduce uncertainty in risk assessments are discussed, and examples of the utility of uncertainty analysis in risk assessment are presented.

I.

SOURCES OF UNCERTAINTY

Uncertainties in risk assessment have three sources: (1) the inherent randomness of the world (stochasticity), (2) imperfect or incomplete knowledge of things that could be known (ignorance), and (3) mistakes in execution of assessment activities (error). Stochasticity Stochasticity is that portion of uncertainty that can be described and estimated but can not be reduced because it is characteristic of the system being assessed. It results from the intrusion of uncertainties at small scales into large· scale phenomena through the action of a variety of agencies termed randomness amplifiers or multipliers (E. H. Mercer, 1981; G. Kolata, 1987a). The small-scale uncertainty may be associated

Uncertainty in Environmental Risk Assessment

205

Table 1 Aspects of Uncertainty in Environmental Risk Assessment Sources of Uncertainty Stochasticity: the inherent unpredictability of the environment. Ignorance: lack of knowledge. Error: inadvertent mistakes. Expressions of Uncertainty for Different Types of Endpoints Individuals (Identity/Likelihood): uncertainty concerning the occurrence of effects on a particular individual expressed as either uncertainty concerning the identity of victims or likelihood that a particular individual will experience effects. Human populations (Frequency): effects on human populations are usually treated as frequencies rather than probabilities. Systems (Credibility): uncertainty concerning the occurrence or magnitude of effects on a characteristic property of a system (e.g., rate of growth of a population). Realization of Uncertainty in the Assessment Process Conceptual: uncertainty in the basic design of the assessment. Quantitative: uncertainty in quantification including sampling, measurement, and statistical analysis. Error: inadvertent mistakes.

with truly random quantum-mechanical processes. For example, random decay of a radio-isotope may cause a mutation in a proto-oncogene, converting it to an active oncogene. The cell containing the active oncogene will multiply uncontrollably, eventually killing the

206


unlucky organism. Alternatively, the random decay may cause a mutation in the DNA of a germ cell that results in a selectively advantageous trait and, through reproduction and selection, changes the evolutionary course of a species. In these cases, biological reproduction serves as the randomness multiplier. The small-scale source of stochasticity need not itself be truly random. Systems characterized by nonlinear dynamics display sensitive dependence on initial conditions (see also the discussion by William Brock elsewhere in this volume). As a consequence, a deterministic system can have an unpredictable outcome because the initial conditions The original example of this can not be perfectly specified. phenomenon, and the example that has the greatest general relevance to risk assessment is the weather. E. N. Lorenz (1963) demonstrated that small changes in the initial state of even the simplest weather model eventually result in large differences in the predicted weather. Even if perfect measurements of all relevant parameters were available for enough grid points to fill the memories of all computers with meteorological data, the weather at some distant time would still be unpredictable because of lack of knowledge of parameter values at subgrid scales. The randomness-multiplying properties of current weather models result in a doubling time of uncertainties of about 2 to 2.5 days (J. J. Tribbia and R. A. Anthes, 1987). Ignorance The second source of uncertainty in risk assessment is ignorance, defined as the lack of knowledge of some aspect of a system that is potentially knowable. In some cases this is a fundamental ignorance of some scientific issue. For example, ignorance of the phenomenon of acid rain made assessments of the environmental effects of sulfur oxide emissions incomplete prior to the late 1970s. This fundamental ignorance results in undefined uncertainty, the "unknown unknowns" that can not be described or quantified (Suter, Barnthouse, and O'Neill, 1987). More commonly, ignorance is simply a result of constraints on our ability to accurately count or measure everything that pertains to a risk estimate. The constraints may be ethical (toxicity tests can not be performed on humans), or practical (sufficient time and resources are not available to perform toxicity tests on all species that will be exposed to a chemical).


207

Error

The third source, human error, is a proverbial inevitability and, to the extent that it occurs in risk assessments, it contributes to uncertainty concerning the accuracy of predictions. Examples include incorrect measurements, misidentifications, data recording errors, data entry errors, and computational errors. With reasonable attention to quality assurance and quality controls, error can be a very small source of uncertainty relative to stochasticity and ignorance.

II.

UNCERTAINTY AND ENDPOINTS

The uncertainties that contribute to the probabilities of effects in risk assessments depend on the organizational level of the assessment endpoint. The endpoint of an assessment is the formal expression of the value that is being protected. Examples are the expected number of deaths in a population exposed to a pollutant or the percent reduction in harvestable biomass of a fishery.

Risks to Individuals: Identity/Likelihood The lowest organizational level of endpoint that is appropriate f or risk assessments is the individual. Individual human lives have value both to individual humans and to society and this value is reflected in the use of individual probabilities of death or injury as the most common endpoints for risk assessments. For example, a pack-per-day cigarette smoker has a 0.004 annual probability of premature death (Wilson and Crouch, 1988). It is possible to derive true individual-level risks if there is information on the frequency of occurrence of the undesired event in specific individuals. Examples are an individual epileptic's probability of experiencing a seizure, or a particular typist's error rate. However, this class of situations is not the usual subject of risk assessment. In practice, the individual probabilities are derived from observations of populations. The populations of smokers observed in epidemiological studies have experienced particular frequencies of

208


fatal lung cancers, and the population frequencies are treated as equivalent to an individual probability. The explicit uncertainty at this level is the identity of the individuals who will actually experience the effect, given the probability that is shared by the members of the exposed population. A case of pure identity uncertainty would be two individuals standing in a large open field with a very high charge difference between the field and an overhead cloud. One will be struck by lightning, but which? This identity uncertainty allows smokers to rationalize their high-risk habit. In most real cases there is additional uncertainty due to the use of population data to estimate individual risks. These uncertainties involve both the ability of the population data to represent specific individuals of interest, and the uncertainty in the population data per se. In a few cases it is possible to treat higher-level entities as individuals in risk assessments. For example, a purpose of the EPA acid lakes survey was to estimate the probability of acidification of individual lakes from the frequency of acidification in a sample of lakes.

Risks to Human Populations: Frequency Populations are the highest organizational level addressed by human health risk assessments, and the lowest organizational level with which ecological risk assessments are concerned. Human populations are treated as simply collections of individuals. To the risk manager who is not concerned with the identity of victims, the endpoints are aggregate characteristics: the frequency of effects in the exposed population and the expected number of effects (the product of frequency and number exposed). These are commonly referred to as individual risk and population risk, respectively. Because it is both unfair to expose an individual to a high risk simply because he has no neighbors and because it is unreasonable to cause harm to more people than necessary by conducting a hazardous activity in a densely populated area, both of these endpoints are taken into account (c. C. Travis et aI., 1987). These frequencies and numbers of effects are usually treated deterministically. In that case, the exposed individuals are treated as being at risk but the population is assumed to be subject to a prescribed level of effect. The uncertainties associated with estimation of a true frequency of effects from an observed frequency, with extrapolating frequencies between human populations, with extrapolating frequencies


209

in test animals to frequencies in humans, and with estimation of the number of exposed individuals in the population are usually either not included in risk analyses or are incorporated as safety factors. Like human populations, populations of nonhuman organisms, biotic communities, ecosystems, and regions can be treated as aggregates of lower-level entities. Frequencies or probabilities of effects on those individual entities can be calculated without identifying the individual entities involved. For example, the national water quality criteria assume a lognormal distribution of sensitivity of species within a biotic community, and set the criterion at the lower fifth percentile (c. E. Stephan et aI., 1985). In other words, a randomly chosen species has a 0.05 probability of toxic effects at the criterion.

Risks to Systems: Credibility Unlike human populations, populations of nonhuman organisms and higher levels of ecological organization may also be treated as entities with their own characteristic properties that may be at risk. For popUlations, these include density, age class structure, extinction, and anthropocentric properties such as standing board-feet of timber or harvestable biomass of fish. For communities, ecosystems, regions, and the globe, they include productivity, diversity, climate, and aesthetic quality. Uncertainties in assessing effects on these systems are different from the identity uncertainty that concerns individuals. It is possible for epidemiologists to find large numbers of individual humans that are exposed to some hazard and to estimate a probability of occurrence of an effect from the frequency of occurrence. For the individual, that probability is an expression of uncertainty concerning his fate. However, it is difficult to find replicate popUlations of organisms that are exposed to the same hazard so that the frequency of popUlation effects can be determined and there are clearly no replicate global environments. Thus ecological risk assessments seldom derive a probability of effects on an individual ecological system from the frequency of occurrence of effects in similar systems. This is not to say that there is no frequentist interpretation of the probabilities derived from analysis of uncertainty in ecological risk models. Rather, when we say that there is a 30% chance that an effluent will lead to the extinction of a fish population, we are saying that we are

210


uncertain about the effects of this effluent on the fish population and, in the cases where we are this uncertain about the effects of a hazard on some ecological endpoint, we expect that the effect will occur 30% of all cases. This type of probability has been termed "credibility" by Bertrand Russell (1948) and others (see I. J. Good, 1983). This type of probability is familiar as the weather forecaster's probability of rain; given the uncertain ty in the forecast models there is a certain credibili ty to the occurrence of rain. Experience with meteorological models indicates that these credibilities approximately equal the actual frequencies of occurrence of the predicted weather types. That is, rain occurs on approximately 30% of those days on which the forecast was a 30% chance of rain. Although traditional actuarial concepts of risk to humans derive probabilities from frequencies of occurrence, probability as credibility is also relevant to human risks because of the uncertainties in the use of epidemiological and test data to estimate frequencies of effects. In other words, if the uncertainties in epidemiology and toxicology are accounted for, then the probability of a particular frequency or magnitude of effects can be calculated.

III.

REALIZATION OF UNCERTAINTY

Although the sources of uncertainty are the same for all risk assessments, and the choice of endpoints constrains the expression of uncertainty in the ways just described, different components and methods of assessment result in different realizations of uncertainty. That is, how uncertainties appear in risk assessments depends on the phase of the assessment and the assessment methods used. In general, the realized uncertainty is distinct in the conceptualization of the assessment, in the derivation of quantitative results and in the commission of errors. Conceptual uncertainty is the uncertainty associated with the underlying assumptions of the mathematical model, observational study, or experimental design chosen to represent the situation being assessed. It is composed largely of ignorance about the nature of the phenomena being represented and about how to adequately represent it. Stochasticity also contributes to conceptual uncertainty by preventing a precise definition of the past or future state of the


211

system being represented. Conceptual uncertainty can be resolved only through validation. Quantitative uncertainty arises in the inherently quantitative processes of sampling, measurement, counting, and calculation as the conceptualization of the model, experiment, or observational study is being carried out. It is distinguished by being readily estimated by conventional techniques and, as a result, it is the only uncertainty that is routinely recognized in risk assessments. Error, as discussed above, is the result of simple mistakes in execution of some task. Although error rates can be measured and estimated, error is usually eliminated or minimized until the residue is felt to be small enough to be ignored. In the following three sections, I discuss the realization of uncertainty in three types of risk studies. They are mathematical models, observational studies (e.g., epidemiological studies and pollution monitoring studies), and experimental studies (e.g., toxicity tests and radiotracer studies). An entire risk assessment may be based on a single type of study (e.g., an integrated pollutant fate and effects model--S. M. Bartell, R. H. Gardner, and R. V. O'Neill, 1987) or different parts of the assessment may be based on different types of study (the most common pattern is pollutant fate modeling accompanied by effects estimation using toxicity tests).

IV.

UNCERTAINTY IN MATHEMATICAL MODELS

Mathematical models are obviously simple abstractions of any real-world situation, and are therefore unlikely to receive automatic acceptance. As a result, much attention has been directed to models as sources of uncertainty and to the ways in which models propagate the uncertainty in input data.

Conceptual Uncertainty Conceptual uncertainty is more obviously a problem for mathematical models than for other assessment approaches. This is because mathematical models are more obviously conceptual

212


abstractions of the situation being assessed than are field or laboratory studies, although all are in fact abstractions. Model conceptualization can be divided into two stages, scenario formulation and model formulation, and different kinds of uncertainty arise in each stage. The scenario is the description of the situation being assessed that the model will attempt to represent. The scenario is defined by (1) the purpose of the assessment (e.g., setting national criteria for a chemical, licensing a specific effluent source, emergency planning, or assessing a new technology), (2) the nature of the hazard and endpoints, and (3) the conceptions of the nature of the relationship between the hazard and the endpoints. These include the physical and temporal scale and resolution, heterogeneity, and interactions with other hazards. Many elements of the scenario are set by law, policy, or other considerations that are external to the assessment process and are not sources of uncertainty. The elements that are most likely to be uncertain are those that relate to (3), the conceptualization of the causal pathway between the hazard and its effects on the endpoints. Model formulation is the process of converting the scenario into a mathematical model, usually in the form of a computer code. One source of uncertainty in model formulation is the aggregation of the large number of entities and phenomena associated with any assessment into a finite number of variables. The resulting aggregation error (R. V. O'Neill, 1973) can cause underestimation of risks by masking extreme responses as when sensitive species are aggregated with other species at the same trophic level (O'Neill, Bartell, and Gardner 1983). Another source of uncertainty is the incorrect bounding of the model. An example with spatial bounds is the approximately 10 km boundaries that were typically used for assessments of the effects of stack gas emissions bef ore the discovery of regional pollution problems such as acid rain. The problem can also be incorrectly bounded in other ways, such as leaving out delayed effects, effects on reproductive processes, or the role of biological processes in chemical fate. A third source of uncertainty in model formulation is the spatial and temporal scale of resolution. For example, the annual average water quality is the wrong scale if the effects of an effluent are episodic (Le., caused by low flow rates of the receiving river or variation in the process generating the effluent). Finally, uncertainty can result from the choice of functional forms. The form of the relationships among variables is often not well defined, and linearization and other approximations are commonly used to facilitate computation.


213

It is possible to evaluate conceptualization uncertainty by comparing models that use different assumptions and structures (R. H. Gardner et at., 1980; BIOMOVS, 1988a). Such comparisons indicate whether models of a phenomenon are robust with respect to possible alternate formulations or whether there is a large variance in results among models. Comparison of models can also be used to test the significance of assumptions about model parameterization. For example, R. H. Gardner, W. G. Cale, and R. V. O'Neill (1982) developed criteria for aggregating individuals and species in ecological studies by examining the sensitivity of 40 alternative models to aggregation of variables representing ecosystem components. They found that components with identical turnover rates could be aggregated in all models without introducing uncertainty.

Quantitative Uncertainty

Implementation of mathematical models requires that numeric values be assigned to the parameters. Methods for estimating the uncertainty of individual parameters and the total uncertainty associated with the parameters of a model are relatively well developed and straightforward. Many environmental parameters are highly variable, difficult to measure, or both. The way in which parameter uncertainties are estimated depends on whether the uncertainty is primarily due to stochasticity or ignorance. Stochastically variable parameters of environmental risk models are often associated with the stochasticity of the weather. The weather determines dispersion of air pollutants, the dilution of water pollutants, and much of the variance in the parameters of popUlations of organisms. Because predictions of weather parameters are no more accurate than monthly climatology for projections greater than a week (Tribbia and Anthes, 1987) and risk predictions must be made much further in advance than one week, weather-related parameters should be treated as random. Temperature, dilution volume, and survival of larval fish are only estimable asf requency distributions of observed past temperatures, dilution volumes, and larval survivals. In the engineering risk assessments that determine the frequency of episodic pollution, many of the parameters, such as time to failure of valves, are similarly stochastic (A. Kandel and E. Avani, 1988).

214


Uncertainty in an individual parameter resulting from ignorance consists of the variance of the measured value on which it is based plus the uncertainty associated with the extrapolating between the entity measured and the entity represented by the parameter. For example, a model of chemical fate in water would include a parameter that estimates the bioaccumulation of chemicals from water by fish (BCF). Because that parameter is difficult to measure, it is usually estimated from the octanol/water partitioning coefficient (P) using the equation (G. D. Veith et aI., 1980): (1) log BCF

= 0.76 log P - 0.23.

The variance in the BCF parameter results from the uncertainty in P (for highly hydrophilic or hydrophobic chemicals, different determinations of P may vary by as much as a factor of 1O--C. T. Garten Jr. and J. R. Trabalka, 1983) and the uncertainty in the statistical model. The model only represents an average relationship for many chemicals and we are ignorant of the actual relationship for a particular new chemical and a particular species of fish. The uncertainty concerning a new BCF can be estimated as the variance in individual observations of BCF in an errors-in-variables regression model (W. E. Ricker, 1973; Suter et aI., 1987), given an estimate of the ratio of the variances of BCF and P. In some cases, there is no series of observations from which to estimate the frequency distribution of a stochastic parameter, or no measurement or extrapolation model for estimation of a frequency distribution due to ignorance. In those cases, it is necessary to resort to expert judgements, but these judgements tend to be highly inaccurate (B. Fischoff et aI., 1981). Experts are most reliable when they are like bookies or weather forecasters in having (1) well defined feedback of success for a well defined population of events, and (2) a sizeable body of experience with few factors affecting the outcome. Techniques for eliciting best estimates of a parameter and its distribution are being developed (M. G. Morgan, M. Henrion, and S. C. Morris, no date; T. s. Wallsten and R. G. Whitfield, 1986). The process of determining the combined effects of uncertainty in all of the parameters of a mathematical model on the output parameters is known as uncertainty analysis or error analysis (O'Neill, 1973). For simple models, it is possible to use analytical approaches


215

(IAEA, 1989). For example, if the model is additive (e.g., the dietary intake of a toxic chemical is the sum of the intake in meat, fruit, water, etc.) and the parameters are independent and normally distributed, then the total variance is the sum of the variances of all parameters. Uncertainty analyses for more complex models typically employ a numeric technique called Monte Carlo simulation in which the model is iteratively parameterized and run using parameter values chosen randomly from the frequency distributions of the parameters. The result of Monte Carlo simulation is a frequency distribution of the output parameter derived as shown in Figure 1. Error Errors in mathematical modeling result from mathematical errors, programming errors, input errors, and errors introduced by computational techniques. Errors are detected and either eliminated or minimized rather than estimated. Errors can be detected by the usual processes of careful checking, such as having all large data sets entered twice and checking for incongruities. A more general a pproach is model verification, the determination that the model, as implemented, produces the intended results. The model should provide correct solutions of the equations and should correctly simulate the situation for which the model was designed. Validation Validation is the process of determining how well the model predicts situations other than the one used in writing the model (J. B. Mankin et aI., 1975). Such data are seldom available, particularly for site-specific models, and are expensive and time consuming to obtain. However, because the validity for models used for common problems is important, validation exercises have been carried out for a number of assessmen t models including radionuclide transport models (BI OM 0 VS, 1988a; M. R. Buckner, 1981; J. C. Golden, E. S. Chandrasakaran, and B. Kahn, 1982; C. W. Miller and C. A. Little, 1982), and models of pollutant fate in the Great Lakes (D. McKay, 1988). Ideally, the validation data set would include a variety of relevant conditions and the validity of the model could then be quantified by regressing predicted

216


DISTRIBUTIONS OF INPUT VALUES FOR PARAMETERS x, y, ANDz

f(x)

~ x

fry)

~ y MODEL DOSE

= g(x, y, z)

DISTRIBUTION OF MODEL PREDICTIONS

~~ -

VALUES OF DOSE

Figure 1. A Diagrammatic Representation of Error Analysis. Source: Hoffman and Gardner, 1983.

values of the output parameters against observed values over the range of conditions (IAEA, 1989).


V.

217

UNCERTAINTY IN OBSERVATIONAL STUDIES

The effects of exposure to toxic chemicals and other hazards are often assessed based on statistical summaries of data from observational studies such as epidemiological and accident studies. Observational studies are also used for assessment of fate or exposure to existing pollution (e.g., assessment of acid rain from measurements of precipitation or surface water pH). Such empirically-based assessments have different conceptual, quantification, and error uncertainties than mathematical models. Conceptual Uncertainty

Observational studies suffer from two basic conceptual problems. One is temporal (observational studies assume that the future will be like the past) and the other is categorical (observational studies assume that the measured population is the same as the population of interest). The temporal similarity assumption may take the form of an assumed identity (the future death rate due to the hazard will be the same as the past rate) or of an assumed trend (the death rate will continue to increase at the current rate). These assumptions are intuitively unappealing because things invariably change, but they lie at the foundation of all inference from experience. The categorical uncertainty results from the assumption that two populations are identical or differ by only the factor being studied. This problem is well recognized in epidemiological studies where, for example, breast cancer rates in the U.S. general population may be compared to rates in Japan or Africa to determine effects of dietary fat on cancer rates (Kolata, 1987b). In the absence of a complete understanding of the factors controlling cancer rates, the confounding effects of differences among populations other than dietary fat are difficult to define or quantify. Ouantitative Uncertainty

A major source of quantitative uncertainty is sampling. Any sample of organisms, environmental materials, or time periods will have a mean and variance that deviates from the popUlation mean and

218


variance in ways that are described by conventional statistics. A somewhat more difficult problem relates to scale of observation; how much uncertainty results from treating an entity as homogeneous that is heterogeneous at a finer scale? Examples include populations that are largely tolerant of pollutants but contain susceptible subpopulations, annual samples of pollutant concentrations that miss acutely toxic episodes, and spatial sampling of pollution that fail to recognize that certain ecosystem types tend to accumulate pollutants. If the small-scale sUbpopulations are adequately represented by the sampling scheme but are not recognized in the analysis, then the uncertainty will simply be larger than it would have been if the sUbpopulations were distinguished. If subpopulations are not represented by the sampling scheme because they are too rare or if they are overrepresented because they are readily sampled, then the results can be biased. These uncertainties due to sampling a heterogeneous population can be estimated and minimized using the techniques of survey statistics. Error Errors in observational studies result from misidentifications, errors in data recording and tabulation, and calculation of summary statistics. If measurements are involved, errors can originate in inaccuracies and imprecisions in the methods and their execution. If human subjects are involved, any lack of completeness and veracity in their responses may also introduce errors. Tommy Wright (1983) presents methods for reducing error in observational studies.

VI.

UNCERTAINTY IN EXPERIMENTAL STUDIES

Experimental studies are primarily used in effects assessment. Dose response functions, median lethal concentrations (LC5Qs), and other statistical summaries of the results of toxicity tests are the primary expressions of toxic effects in risk assessments. However, exposure assessments are occasionally based on experimental studies involving releases of dyes, tracers, or small amounts of the pollutant, rather than by modeling pollutant transport and fate.


219

Conceptual Uncertainty Experiments, unlike observational studies, lend themselves to elucidation of causation because they allow control of conditions. However, this control introduces conceptual problems. If it is simply assumed that the fish in a tank or the rat in a cage is a model of the fish community in a river or of humans, then that conceptual leap is a source of uncertainty. This uncertainty is typically addressed by applying a safety factor that is felt to be of sufficient magnitude to cover differences in species, conditions, life stages, and other factors (M. L. Dourson and J. F. Stara, 1983). The imprecision of safety factors as estimates of this uncertainty is reflected in the fact that they are usually round orders of magnitude (e.g., 10, 100, 1000). Quantitative Uncertainty Quantitative uncertainty arises in experimental studies from both uncertainty in the experimental results and extrapolation of the results so that they express the desired endpoint. Uncertainty in the results is due to variance among experimental units, sampling, measurement, and stochasticity of the system in which the experiment is being conducted. Statistical methods for design and analysis of experiments are well developed and well known to experimenters. Examples of uncertainty estimates for experimental results are confidence intervals on dose response functions and variances on LC so values. If the experimenter or the risk assessor using experimental results does not simply assume that the experimental system and the system being assessed are equivalent, plus or minus a margin of safety, then extrapolation models must be used to make the translation. For example, rainbow trout are commonly used for toxicity testing but Atlantic salmon are not. An assessor who possesses test results for rainbow trout and is interested in estimating the effects on Atlantic salmon could use the equation: (2) log Y = 1.2 log x - 0.51,

220


where y is LCso values for Atlantic salmon and x is LCso values for rainbow trout (Suter et aI., 1986). The uncertainty in estimating a salmon LC so from this equation results in a prediction interval (the interval in which 95 percent of individual observations would be expected to occur) of .±. 0.87 log units (i.e., orders of magnitude) at mean x. Although regression analysis is the most common means of making these extrapolations, other approaches have been used including sensitivity distributions (Stephan et aI., 1985), and relative potency models (T. D. Jones et aI., 1988). Error Experiments potentially contain the same errors of observation, measurement, tabulation, and calculation of results as observational studies. In addition, because the experimenter creates and controls the conditions under which the results occur, there are potential errors due to incorrect experimental conditions. For example, in a toxicity test the age, strain, or species of the test organisms, the health of the organisms, the temperature and other physical conditions, and the dose or exposure concentration may not be correctly specified. Validation Validation of experiment-based risk assessment requires a comparison of the experimental results with the response of actual environments exposed to actual hazards. As has been mentioned previously, the necessary monitoring studies are difficult and expensive and often give ambiguous results. Another possibility is comparison of results from one or more experimental systems with differing degrees of complexity. For example, Jeffrey H. Giddings and Paul J. Franco (1985) compared results of tests of the same toxic material employing single species, a laboratory microcosm, and outdoor ponds. Similarly, John Cairns Jr. and Donald S. Cherry (1983) and James E. Clark et aI. (1986) have compared responses of organisms in laboratory toxicity tests to those of organisms caged in polluted environments. This approach can be less costly than monitoring the response of real biotic communities in the field, and is more likely to reveal the cause of differences between results, but it is a less complete form of validation.


221

The most common form of validation for experiments is comparison of other investigators' results for the same or similar pollutants in similar experimental systems. This validation will indicate whether there may be flaws in the execution of the experiment or whether peculiarities of a particular experimental system unduly influence the results but it does not indicate whether critical features of the environment are included in the experimental systems. Finally, if an investigator is suspicious of the results, he or she may repeat the experiment to verify the results. This within-laboratory replication can only reveal flaws in execution of the experiment.

VII.

REDUCING UNCERTAINTY

The obvious way to reduce uncertainty is to do a better job of modeling, observing, and experimenting. Modelers may obtain better parameter estimates, or a better understanding of mechanisms. Those conducting observational studies can obtain larger samples and longer time series, better define the populations being sampled, and reduce errors in measurements. Experimentalists can increase replication, provide more realistic experimental conditions, include more of the range of relevant experimental subjects and conditions, and take more care in conducting the experiment and recording results. Another means of reducing uncertainty is to reduce the number of components in the assessment that contribute to uncertainty. For example, if a screening model of pollutant exposure indicates that some of the exposure routes are trivial, they can be left out of the final model. By thus reducing the scope of the model, uncertainty that is irrelevant to the exposure is eliminated. A less obvious means of reducing uncertainty is optimization of the mix of modeling, observation, and experimentation in the risk assessment. The relative magnitudes of uncertainty associated with these techniques differs depending on the problem being addressed. For example, observational studies, unlike models and experiments, are inherently realistic so they potentially have low conceptual uncertainty. However, when causation is not understood, experimental studies will have lower conceptual uncertainty than observational studies which may lead the investigator astray with coincidences and extraneous

222

Acting Under Uncertainty: MUltidisciplinary Conceptions

correlations. In addition, methods differ in the extent to which their uncertainties are objectively quantifiable. In general, it is desirable to maximize the extent to which uncertainties are quantified so as to increase confidence in decisions. Kenneth H. Reckhow (1983) provides an example of using observational data to reduce uncertainty in an assessment model. When modeling the effects of a development on the phosphorous loading of a lake, he suggests that, rather than modeling the entire phosphorous budget of the lake, an assessor should use observed values for phosphorous loading from the undisturbed portions of the watershed and only model the input from the portion that would be disturbed by the proposed development. Reckhow provides procedures for combining uncertainty from observational variables with uncertainty from model output. Mathematical models can be used to guide and refine observational studies as a means of reducing uncertainty. Detailed exposure models can be used to determine the exposure of individuals or sUbpopulations rather than simply comparing exposed and unexposed populations. Similarly, chemical fate models can be used to optimize sampling designs in exposure assessments. Empirical models provide a compromise between observational studies and mathematical simulation models. For very regular and well-understood systems, either observation or theory is successful. For example, if you want to know what time the sun will rise you can either use observations of the sunrise for a few days before the day of interest, or you can model the velocities and gravitational interactions of the sun and earth and either way come up with a reliable prediction. However, most systems involving risks are neither well understood nor regular. In those cases empirically-based mathematical models can minimize the conceptual uncertainties. For example, fish populations tend to vary unpredictably so that simple statistical summaries of fisheries data provide little information, while attempts to model fish populations from bioenergetics or other ecological theory have little hope of success. However, simple empirical fisheries models can summarize fisheries data using just enough biological theory (e.g., annual generations, age-specific mortality) to allow modeling the consequences of life-stage-specific pollution effects without losing the realism of data from observations of the dynamics of real populations (Barnthouse et aI., 1989).


223

VIII. USES OF UNCERTAINTY

A simple example of the use of uncertainty in environmental decision making is portrayed in Figure 2a. The curve is a cumulative probability function for the predicted concentration of a pollutant in some environmental medium. Such curves can be generated by probabilistic models of environmental fate (M. A. Parkhurst, Y. Onishi, and A. R. Olson, 1981; D. M. DiToro, 1986). The spread of the distribution results both from the combined variances of stochastic environmental parameters and from uncertainty concerning pollutant-specific parameters such as partition coefficients and degradation rates. The probability of exceeding some benchmark value such as a water or air quality standard can be read off the curve as indicated by the arrow and dashed line. Similarly, probabilistic risk analyses can be used to set standards based on the probability that some threshold for unacceptable effects will be exceeded. Suter, Vaughan and Gardner (1983); Suter, Rosen and Linder (1986); Suter et al. (1987) and Barnthouse et al. (1987) described methods for calculating probability distributions for various toxicological effect endpoints ranging from LCso values to proportional reductions in populations. Figure 2b shows such a distribution plotted as a cumulative probability. A standard could be set at the concentration having a 5% probability of exceeding the chosen effect threshold. Probability distributions such as these can clarify the relationship between decision making and uncertainty. They can be used to justify additional measurement or testing by showing how the reduction in uncertainty obtained by the additional studies can be expected to steepen the curves. This decrease in uncertainty may change the outcome of the decision, and will certainly increase the confidence in the decision. These curves also make clear the advantage of estimating the expected effects and associated uncertainty compared with using worst-case assumptions or arbitrary safety factors. In the latter approach, there is no scale of badness or safety equivalent to the probability scale, so if someone puts forward a higher arbitrary safety factor or devises a worse case than the proposed worst case, there is no objective way to compare the consequences.

224


(a)

o~--------~--------~

CONTAMINANT CONCENTRATION (b) 1.0

0~-7--------------~

CO NTAMINANT CONCENTRATION

Figure 2. Two Applications of Probabilistic Risk Estimates (a) A probability (frequency) function is used to estimate the frequency with which a standard or other action level will be exceeded. (b) A cumulative probability function for the threshold for significant effects on a species or biotic community is used to select a concentration with an X% chance of exceeding the threshold. Source: Suter, Barnthouse, and O'Neill, 1987.


225

IX. CONCLUSIONS

The explicit role of uncertainty in risk assessment has led to the criticism that risk assessment is simply an excuse for not doing the right thing. For example, Langdon Winner (1987, p. 143) has written in this regard: What otherwise might be seen as a fairly obvious link between cause and effect, for example, air pollution and cancer, now becomes something fraught with uncertainty. Clearly, links between pollutants and cancer rates are less obvious to other observers; see for example Bruce N. Ames (1983). Given such differences of perspective, denial of uncertainty can lead only to the courtroom, a poor forum for settling technical issues. Although merely acknowledging uncertainty can lead to a paralysis of authority, systematic identification and quantification of uncertainty allows comparison of assumptions, models, and data. This can lead to efficient reduction of critical uncertainties, more fully informed decisions, increased openness of the decisionmaking process, and even the possibility of consensus among interested parties. The real problem with risk assessment for Winner and the many like-minded environmentalists is not the acknowledgment of uncertainty but its acknowledgment and use by one side only. If the burden of proof is equally apportioned between the advocates of safety and the advocates of development, then analysis of uncertainty becomes a genuinely two-edged sword.

REFERENCES

Ames, Bruce N. (1983), "Dietary Carcinogens and Anticarcinogens," Science, 221, pp. 1256-64.

226


Barnthouse, L. W. et a1. (1987), "Estimating Responses of Fish Populations to Toxic Contaminants," Environmental Toxicology and Chemistry, 6, pp. 811-24. Barnthouse, L. W.; Suter, G. W. and Rosen, A. E. (1989), "Inferring Population-level Significancefrom Individual-level Effects: An Extra polationf rom Fisheries Science to Ecotoxicology ,to in G. W. Suter and M. A. Lewis, eds., Aquatic Toxicology and Hazard Assessment: Eleventh Symposium, Philadelphia: American Society for Testing and Materials. Bartell, S. M.; Gardner, R. H. and O'Neill, R. V. (1988), "An Integrated Fates and Effects Model for Estimation of Risk in Aquatic Systems," in W. J. Adams et aI., eds., Aquatic Toxicology and Hazard Assessment: Tenth Volume, Philadelphia: American Society for Testing and Materials, pp. 261-74. BIOMOVS (1988a), Progress Report 5, Stockholm: National Institute of Radiation Protection. BIOMOVS (1988b), Technical Report 1; Scenario B3, Release of Radium-226 and Thorium-230 To A Lake, Stockholm: National Institute of Radiation Protection. Buckner, M. R., ed. (1981), Proceedings of the First SRL Model Validation Workshop, DP-1597,Aiken, SC: Savannah River Laboratory. Cairns, John, Jr. and Cherry, Donald S. (1983), "A Site-specific Field and Laboratory Evaluation of Fish and Asiatic Clam Population Responses to Coal Fired Power Plant Discharges," Water Science and Technology, 15, pp. 10-37. Clark, James E. et a1. (1986), "Field and Laboratory Toxicity Tests with Shrimp, Mysids, and Sheepshead Minnows Exposed to Fenthion," in T. M. Poston and R. Purdy, eds., Aquatic Toxicology and Environmental Fate: Ninth Volume, Philadelphia: American Society for Testing and Materials, pp. 161-76. DiToro, D. M. (1986), "Exposure Assessment for Complex Effluents: Principles and Possibilities," in H. L. Bergman, R. L. Kimerle, and A. W. Maki, eds., Environmental Hazard Assessment for Effluents, New York: Pergamon Press, pp. 135-51. Dourson, M. L. and Stara, J. F. (1983), "Regulatory History and Experimental Support of Uncertainty (Safety) Factors," Regulatory Toxicology and Pharmacology, 3, pp. 224-38.


227

Fischoff, B. et al. (1981), Acceptable Risk, Cambridge: Cambridge University Press. Gardner, R. H. et al. (1980), "Comparative Error Analysis for Six Predator-Prey Models," Ecology, 61, pp. 323-32. Gardner, R. H.; Cale, W. G. and O'Neill, R. V. (1982), "Robust Analysis of Aggregation Error," Ecology, 63(6), pp. 1771-79. Garten, C. T., Jr. and Trabalka, J. R. (1983), "Evaluation of Models for Predicting Terrestrial Food Chain Behavior of Xenobiotics," En vironmental S cienceand T echnology,17(10),pp.590-95. Giddings, Jeffrey M. and Franco, Paul J. (1985), "Calibration of Laboratory Bioassays with Results from Microcosms and Ponds," in Terence P. Boyle, ed., Validation and Predictability of Laboratory Methods for Assessing the Fateand Effects of Contaminants inAquatic Ecosystem, Philadelphia: American Society for Testing and Materials, pp. 104-19. Golden, J. C.; Chandrasekaran, E. S. and Kahn, B. (1982), "Monitoring the Critical Radiation Exposure Pathways at a BWR Nuclear Power Station," Health Physics, 42, pp. 777-88. Good, I. J. (1983), Good Thinking, The Foundations of Probability and Its Applications, Minneapolis: University of Minnesota Press. Hoffman, F. o. and Gardner, R. H. (1983), "Evaluation of Uncertainties in Radiological Assessment Models," in John E. Till and H. Robert Meyer, eds., Radiological Assessment, Washington, D.C.: U. S. Nuclear Regulatory Commission, pp. 11-1--11-45. Hoffman, F. o. and Miller, C. W. (1983), "Uncertainties in Environmental Radiological Assessment Models and their Implications," in Proceedings of the Nineteenth Annual Meeting on Environmental Radioactivity, Bethesda, MD: National Council on Radiation Protection and Measurement, pp. 110-38. International Atomic Energy Agency (IAEA) (1989), Procedures for Evaluating the Reliability of Predictions Made by Environmental Transfer Models, Vienna, Austria: IAEA. Jones, T. D. et al. (1988), "Chemical Scoring by a Rapid Screening of Hazard (RASH) Method," Risk Analysis, pp. 99-118. Kandel, A. and Avani, E. (1988), Engineering Risk and Hazard Assessment, Boca Raton: CRC Press.

228


Kolata, G. (1987a), "What Does It Mean To Be Random?" Science, 231, pp. 1068-70. Kolata, G. (1987b), "Dietary Fat-Breast Cancer Link Questioned," Science, 235, p. 436. Lorenz, E. N. (1963), "Deterministic Nonperiodic Flow," Journal of the Atmospheric Sciences, 20, pp. 130-41. Mankin, J. B. et al. (1975), "The Importance of Validation In Ecosystem Analysis," in G. S. Innis, ed., New Directions in the Analysis

of Ecological Systems, Simulation Councils Proceedings, Series 1(1), LaJolla, CA: Simulation Councils, pp.63-72. McKay, D. (1988),

"On

Reviewing

Environmental

Models,"

Environmental Science and Technology, 22(2), p. 116. Mercer, E. H. (1981), The Foundations of Biological Theory, New York: John Wiley and Sons. Miller, C. W. and Little, C. A. (1982), A Review of Uncertainty

Estimates Associated with Models for Assessing the Impact of Breeder Reactor Radioactivity Releases, ORN L-5832, Springfield, VA: National Technical Inf ormation Service. Morgan, M. G.; Henrion, M. and Morris, S. C. (no date), Expert Judgements for Policy Analysis, Springfield, VA: National Technical Information Service. O'Neill, R. V. (1973), "Error Analysis of Ecological Models," in D. J. Nelson, ed., Radionuclides in Ecosystems, Springfield, VA: National Technical Information Service, pp. 898-908. O'Neill, R. V.; Bartell, S. M. and Gardner, R. H. (1983), "Patterns of Toxicological Effects in Ecosystems: A Modeling Study," Environmental Toxicology and Chemistry, 2, pp. 451-61. Parkhurst, M. A.; Onishi, Y. and Olson, A. R. (1981), "A Risk Assessment of Toxicants to Aquatic Life Using Environmental Exposure Estimates and Laboratory Toxicity Data," in D. R. Branson and K. L. Dickson, eds., Aquatic Toxicology and Hazard Assessment, Philadelphia: American Society for Testing and Materials, pp. 59-71. Reckhow, Kenneth H. (1983), "A Method for the Reduction of Lake Model Prediction Error," Water Research, 17, pp. 911-16. Ricker, W. E. (1977), "Linear Regressions In Fisheries Research,"

Journal of the Fisheries Research Board of Canada, 30, pp.409-34.


229

Rowe, W. D. (1977), An Anatomy of Risk, New York: John Wiley and Sons. Russell, Bertrand (1948), Human Knowledge, Its Scope and Limits, Part V. New York: Simon and Schuster. Stephan, C. E. et al. (1985), Guidelines for Deriving Numeric National Water Quality Criteria for the Protection of Aquatic Organisms and Their Uses, PB85-227049, Springfield, VA: National Technical Information Service. Suter, G. W., II, Vaughan, D. S. and Gardner, R. H. (1983), "Risk Assessment by Analysis of Extrapolation Error, A Demonstration for Effects of Pollutants on Fish," Environmental Toxicology and Chemistry, 2, pp. 369-78. Suter, G. W., II; Rosen, A. E. and Linder, E. (1986), "Analysis of Extrapolation Error," in L. W. Barnthouse and G. W. Suter II, eds., User's Manual for Ecological Risk Assessment, Springfield, VA: National Technical Information Service, pp. 49-81. Suter, G. W., II et al. (1987), "Endpoints for Responses of Fish to Chronic Toxic Exposures," Environmental Toxicology and Chemistry, 6, pp. 793-809. Suter, G. W., II; Barnthouse, L. W. and O'Neill, R. V. (1987), "Treatment of Risk in Environmental Impact Assessment, Environmental Management, 11, pp. 295-303. Travis, C. C. et al. (1987), "Cancer Risk Management," Environmental Science and Technology, 21, pp. 415-20. Tribbia, J. J. and Anthes, R. A. (1987), "Scientific Basis of Modern Weather Prediction," Science, 237, pp. 493-99. Veith, G. D. et al. (1980), "An Evaluation of Using Partition Coefficients and Water Solubility to Estimate Bioconcentration Factors for Organic Chemicals in Fish," in J. G. Eaton, P. R. Parrish and A. C. Hendricks, eds., Aquatic Toxicology, Philadelphia: American Society for Testing and Materials, pp. 116-29. Wallsten, T. S. and Whitfield, R. G. (1986), Assessing the Risks to Young Children of Three Effects Associated With Elevated Blood-Lead Levels, Argonne, IL: Argonne National Laboratory. Wilson, R. and Crouch, E. A. C. (1987), "Risk Assessment and Comparisons: An Introduction," Science, 236(4799), pp. 267-280. H

230


Winner, Langdon (1983), The Whale and the Reactor, Chicago: University of Chicago Press. Wright, Tommy, ed. (1983), Statistical Methods and Improvement of Data Quality, Orlando, FL: Academic Press.