Subjective probabilities may help to quantify probabilities. Question: ... 36
enumerator dummies. Selection into missing and don't know (linear probability).
5 ...
Vulnerability to downside risk in developing countries: What can subjective probabilities tell us? Ethan Ligon University of California, Berkeley
Felix Povel Georg-August-University, Göttingen
International Conference on Social Cohesion and Development, OECD, January 2011 1
Preview Motivation • Vulnerability decreases current wellbeing, impacts on behavior, may lead to poverty in future. • Measures of vulnerability have to quantify probabilities. • Subjective probabilities may help to quantify probabilities. Question: • To what degree do subjective probabilities help to quantify the likelihood of experiencing a shock in future? Answer: • Subjective probabilities have some predictive power. • Predictive power and potential biases can partly be assessed and dealt with.
Contribution: • New empirical approaches to evaluate prediction accuracy of subjective probabilities. 2
Data Two wave panel from rural households in Thailand (N=2186) collected in 2007 and 2008, respectively. Data set includes • conventional survey information (household demographics, wealth, occupation, education, health, etc.); • subjective probabilities of different risks; and • records of experienced shocks.
forward looking subjective probabilities from wave 1 wave 1
wave 2
retrospective shock information from wave 2 May 2007
April 2008
3
Results
Descriptive statistics Subjective risk probability in wave 1 Shocks in wave 2 missing “don’t know” probability illness 0.018 0.156 0.826 0.188 Note: share of 2186 households in cells type of event
4
Results Selection into missing and don‘t know (linear probability) Correlates of “missings” or “don’t knows” dependent variable equals 1 if probability is missing or unknown; 0 otherwise (wave 1) Event type:
illness
Household characteristics
p-value
0.071
0.716
F-stat
1.56
0.79
Respondent characteristics
p-value
0.457
0.162
F-stat
0.97
1.49
Location dummies
p-value
0.000
0.000
F-stat
75.75
21.79
Enumerator dummies
p-value
0.000
F-stat
32.18
Constant
0.131
-0.017
(0.219)
(0.184)
2,058
2,058
Observations Adjusted Rsquared 0.035 0.323 Note: OLS regressions; standard errors in parentheses and clustered at village level; * denotes significance at 10%-level, ** at 5%-level, *** at 1%-level; 110 sub-district; 36 enumerator dummies.
5
Results Test of null hypothesis that respondents provide objective probabilities. ID 1 2 3 … … … 2184 2185 2186
probability 0.20 0.00 0.67 … … … 0.36 0.49 0.59
dummy 1 0 0 1 … … … 0 1 0
dummy 2 1 0 1 … … … 1 0 0
… … … … … … … … … …
dummy 1000 E[dummy] 0 0.20 0 0.00 1 0.67 … … … … … … 0 0.36 0 0.49 1 0.59
actual shock 0 1 0 … … … 0 0 0
6
Results Test of null hypothesis that respondents provide objective probabilities. ID 1 2 3 … … … 2184 2185 2186
probability 0.20 0.00 0.67 … … … 0.36 0.49 0.59
dummy 1 0 0 1 … … … 0 1 0
dummy 2 1 0 1 … … … 1 0 0
… … … … … … … … … …
dummy 1000 E[dummy] 0 0.20 0 0.00 1 0.67 … … … … … … 0 0.36 0 0.49 1 0.59
actual shock 0 1 0 … … … 0 0 0
riskprob23 97.5
80 60 40 20 0
1000 R²
# of regressions
Regress dummy 1 on probability Regress dummy 2 on probability … Regress dummy 1000 on probability
mean
100
2.5
.68
.7
.72
.74
R squared
Regress actual shock on probability If R² is within the 95% confidence interval we cannot reject the null hypothesis. 7
Results
Test for “objectivity” of subjective probabilities actual R 2.5th 97.5th event type mean squared percentile percentile illness 0.009 0.230 0.265 0.301 Note: “actual R squared” is result from OLS regression of shock dummy on subjective probability; “2.5th percentile”, “mean”, and “97.5th percentile” are results from distribution of 1000 R squared obtained by regressing 1000 simulated shock dummies on subjective probabilities.
8
Results Correlates of subjective probabilities (OLS) Correlates of subjective probabilities dependent variable is subjective probability (wave 1) Event type: illness Household p-value 0.000 0.000 0.000 0.000 0.000 characteristics F-stst 4.86 3.77 4.08 5.20 4.26 Respondent p-value 0.106 0.004 0.003 characteristics F-stst 1.47 2.20 2.29 Location p-value 0.000 0.000 0.000 0.000 0.000 dummies F-stst 43.31 26.84 43.00 29.77 26.18 Enumerator p-value 0.000 0.000 dummies F-stst 10.88 8.96 Constant 0.060 0.097 0.050 0.207 0.190 (0.129) (0.149) (0.135) (0.137) (0.142) Observations 1,540 1,520 1,520 1,540 1,520 Adjusted Rsquared 0.050 0.053 0.058 0.153 0.164 Note: OLS regressions; standard errors in parentheses and clustered at village level; * denotes significance at 10%-level, ** at 5%-level, *** at 1%-level; 110 sub-district dummies included; 36 enumerator dummies included; p-values and F-statistics from tests for joint significance.
9
Results Kolmogorow-Smirnow-Test for equality of distributions KS tests for illness - Thailand distributions 2 1.5 1 0
.5
Density
1 .5 0
Density
1.5
distributions
-.5
0 value
.5
base
-.5
resp bias
.5
base
kernel = epanechnikov, bandwidth = 0.0473
enum bias
kernel = epanechnikov, bandwidth = 0.0473
1.5 1 .5 0
0
.5
1
Density
1.5
2
distributions
2
distributions Density
0 value
-.5
0 value base
.5 enum+resp bias
kernel = epanechnikov, bandwidth = 0.0473
-.5
0 value enum bias
.5 enum+resp bias
kernel = epanechnikov, bandwidth = 0.0442
10
Results Correlates of shock occurrence (OLS) Event type: probability
Correlates of shock occurrence dependent variable is shock dummy (wave 2) illness 0.112*** (0.040) 0.155*** (0.044)
“enumerator“ residual equal coefficient
0.008 7.27
dummy
-0.030
“enumerator“ residual*dummy
(0.020) 0.155*** (0.044)
Observations 2,082 1,540 1,540 1,540 2,082 Adjusted R0.086 0.063 0.067 0.070 0.092 squared Notes: OLS regressions; standard errors in parentheses and clustered at village level; * denotes significance at 10%-level, ** at 5%-level, *** at 1%-level; household characteristics and 110 sub-district dummies included but not reported; “equal coeff” denotes test results for equality of coefficient of subjective probability and coefficient of residual from subjective probability regression on base specification plus enumerator dummies.
11
Conclusion Don’t overestimate usefulness of subjective probabilities: •
Subjective probabilities not equal to objective ones
•
Many potential biases
Don’t underestimate usefulness of subjective probabilities either: •
Probabilities significantly correlated with subsequent shock occurrence
•
Potential to contribute to quantification of risk exposure
Straightforward test of “objectiveness” of probabilities is feasible.
Sources of potential bias can be detected.
Bias can – at least partly – be dealt with. 12
Appendix A: Subjective probabilities equal to objective ones? Null hypothesis: Respondents provide objective probabilities.
Test: Is R² from regressing actual shock dummy on subjective probability significantly different from the R² received by regressing simulated shock dummies on objective probabilities? Ingredients: shock dummy subjective probability cross-sectional mean objective probability
regression
with
with
13
Appendix A: Subjective probabilities equal to objective ones? R²:
where
and
14
Appendix A: Subjective probabilities equal to objective ones?
From individual to cross sectional variance:
plug
into
15
