An expert systems approach to econometric modelling

~?

ATHEMATICS AND COMPUTERS N SIMULATION

ELSEVIER

Mathematics and Computers in Simulation 39 (1995) 379-383

An expert systems approach to econometric modelling L.T. Oxley Department of Economics, University of Edinburgh, 50 George Street, Edinburgh EH8 9JZ, UK

I. Introduction

Many areas of economics involve the formulation of models and the testing of certain hypotheses. Estimation and hypothesis testing can now be routinely undertaken using many estimation packages, for example, M I C R O F I T 3.0 (Pesaran and Pesaran, [1]), and PC-GIVE (Hendry, [2]). Many new estimation techniques, for example Johansen [3] maximum likelihood methods, can be "routinely" used by relatively inexperienced users. New statistical tests are rapidly incorporated into existing estimation packages. However, how 'best' to 'discover' an adequate model with known, powerful, statistical properties, remains to many, if not the majority of economists, an act of "alchemy" and not science to paraphrase Hendry [4,5] (see also McAleer, Pagan and Volker, [6] and Lovell [7]). Demystification of the expert's expertise in an operational, constructive and open manner is the motivation behind attempts to incorporate methods from the discipline of artificial intelligence, particularly expert systems, into economics. This paper will investigate how an expert systems approach might be applied to the problem of econometric model formulation. In Section 2, we will discuss current "best-practice'econometrics to give an outline of the candidate econometric methods available. Section 3 will outline the power of expert systems applications. Section 4 describes via a schematic diagram, how an expert system may be constructed for use in applied economics and Section 5 concludes.

2. Some issues in current best-practice methods

How best to formulate and estimate economic models is of perennial concern to applied economists (see for example, McAleer, Pagan and Volker [6]). However, of late one of the main issues has become the choice of econometric methodology. The 1985 World Congress of the Econometric Society was the venue for the first major debate on best practice methodologies (see for example Pagan [8]). The three major alternatives presented there were: Hendry's, 0378-4754f95/$09.50 © 1995 Elsevier Science B.V. All rights reserved

SSDI 0 3 7 8 - 4 7 5 4 ( 9 4 ) 0 0 0 8 7 - Z

380

L.T. Oxley / Mathematics and Computers in Simulation 39 (1995) 379-383

"General to Specific"; Leamer's "Extreme Bounds Analysis, (EBA)"; and Sims' "Vector AutoRegressive, (VAR)". Although the use of such stereotypes commands less support than it did, see Pagan [9], they represent the beginnings of a continuing debate on best practice methods. In a recent update to his view of the mid-1980s, Pagan [9] discusses some of the recent developments in econometric methodology. In particular he considers the progress made by the " t h r e e methodologies" of 1987 noting some convergence between the structural and V A R modellers and a virtual stagnation of the Bayesian approach most often associated with the work of Leamer. " T h e E T Dialogue", Hendry et al. [10] presented a number of experts' views on econometric methodology. Finally, a number of recent texts, for example, Darnell and Evans [11], discuss various characteristics of the competing methodologies from a philosophy of science viewpoint, see Dharmapala [12]. Current best practice, however, is still an issue of debate. The dominant British approach follows many the tenets of Hendry (see Ericsson et al. [13] and Gilbert [14]). In contrast the dominant United States approach follows many of the tenets of Sims [15]. In this paper we will generally be considering aggregate time series data and methods generally used in its analysis.

3. Expert systems and knowledge elicitation Expert systems allow the formalization and implementation of rule-based decision processes. They codify the views and practices of expert practitioners. Furthermore, they allow the user to interact with and be interrogated by, the system. Finally, they are able to explain their reasoning and choice of solution. They have been applied to a wide variety of domains to solve a wide variety of tasks, particularly those involving classification diagnosis and configuration. More complex systems have been applied to tasks such as simulation, Ushold [16], and equation solving, Bundy [17]. The development of expert systems, including complex tasks such as eliciting knowledge from experienced practitioners in the field, and other tasks such as reconciling different expert views, is now a well researched process. Given the sometimes conflicting views of expert applied econometricians and the number of non-expert users of powerful econometric methods, the use of expert systems methods in econometrics seems an ideal partnership.

4. Econometric modelling, specification searches and expert systems The primary sources of information for use in econometric modelling are economic theory, data, a measurement system and alternative models for explaining the data. The challenge facing applied economists is to discover an adequate representation of the data, an adequate model specification. Differences in formulating a model specification may arise through different modelling paradigms, different ways of specifying auxiliary assumptions about a paradigm, or different strategies adopted in the process of model construction. Economic theory may indicate which variables to include, however, it is often not forthcoming on other issues, such as functional form. In these cases specification searches, see McAleer [18], are


381

often utilized where a specification search is a set of procedures followed in moving from an initial to a final model specification. It is in specification searching that expert systems can be seen to have a role. Although there are some dissenters, see Learner [19], a consensus has developed that diagnostic checks are essential in evaluating the adequacy of models. Failing a diagnostic check identifies some form of model inadequacy. The tests themselves rarely suggest which alternatives to follow, see McAleer [18]. Moreover, within this specification search approach, the applied economist is faced with a series of (often implicit) questions, where in many cases experts have developed their own "rules-of-thumb". In other cases, evidence from econometric theory can give some guidance. Such questions might include; how many tests should be used; are the tests independent; what order of tests is optimal; can the tests be used jointly; how many non-nested alternatives should be considered; what are the small sample properties of the tests; are the tests powerful, are the tests robust? An expert system could incorporate the following features to aid non-expert users in specification searches. This system would either operate within an existing estimation package or act as an interface to it. The features could easily include a sophisticated help/advice system including state-of-the-art knowledge elicited from experts; a system with various levels of automated actions including routine reactions to diagnostic check results; a sophisticated warning mechanism for example, minimum degrees of freedom or violation of optimal test sequences; the incorporation of s t r o n g / w e a k economic priors which would influence/guide the advice offered by the system about model development. The system would have the facility to allow the "novice" user to follow a pre-determined "best-practice" path (perhaps guided by strong or weak economic priors) or to be overruled at any juncture. Furthermore, it would incorporate sophisticated report and internal backtracking facility allowing a detailed description of the actual specification search adopted. This would allow others to challenge/validate the search procedure followed. However, the system would not be a static tool. It could be used to consider developments in best-practice method itself. A schematic view of a possible system based upon the generally tenets of specification searches advocated in McAleer [18], but incorporating elements of Bayesian methods particularly the idea of strong and weak priors, is presented as Fig. 1. A typical " r u n " of the system might involve the following: (i) The economic model and set of hypotheses to test would be specified by the user. This might include rival hypotheses to encompass. (ii) Data appropriate for estimation would be collected or input via databases. These might include direct interfaces to e.g. E U R O S T A T DOSES project (iii) The user would be asked what level of automated decision making is required; what level of warnings. Help and advice will be available at this and all levels. (iv) S t r o n g / w e a k priors may be specified which will guide systems reactions to intermediate results, pre-tests and diagnostic checks. (v) Normally, the next stage involves estimation of a preliminary model. Exceptions would include models with very strong priors. Diagnostic checks would normally be called-upon. If the model "passes", the next stage would be an automatic report writing stage. If the model "fails" advice will be given on possible respecification. The model would then be re-estimated and the cycle continued until a "pass" is attained.

382


us[a INT[RFAC[

l

I LEU[L OF MUTOMRTED ACTIONS

PM[-T[$T ~RRNING~ VtS/NO

j

I .................................................. ..............

J

[

[ ~

L[~P[L OF A[eOA~ WRItinG

1

J

I

INPUT

....

I..............

"

[CONOMIC

tAIORS

-:--:--\:-/--'--;;---':---:-"

_ . _

P R [ L I NI NAIIV ( PME ) TIE~TS

I

i

o ......

MOBIL

JJ

.oout.[

-~

R[POMT PACILITV WMITING

$TmONG

4

pmss

]

FAll.

]

c.[cNs

[CONOMIC

1

ORIORS

0

i

I

• ~ AN[ND,'R[SPECIFY r1OO[L ( AtlTOHATI C/~ANUAL )

PRELININARVMoD[L

÷

......

/

/

.[L,

*

Fig. 1. Flowchart o f possible expert system.

(vi) Variations might include: (a) no role for pre-tests or diagnostics, (b) the specification of a general model at the outset with no role for economic priors.

5. Conclusions Used carefully, the methods from expert systems could usefully be used to aid the construction of econometric models. Using best-practice methods elicited from expert practitioners more applied economists' could undertake applied research.

References [1] M.H. Pesaran and B. Pesaran, M I C R O F I T 3.0 (Oxford Electronic, Oxford 1991). [2] D.F. Hendry, PC-GIVE. An interactive econometric software modelling system, Oxford: Institute of Economics and Statistics, 1989. [3] S. Johansen, Statistical analysis of cointegrating vectors, J. Econom. Dynamics Cont. 12, (1988) 231-254. [4] D.F. Hendry, Econometrics--alchemy or science?, Economica 47 (1980) 387-406.


383

[5] D.F. Hendry, Econometrics: Alchemy or Science?: Essays in Econometric Methodology, (Basil Blackwell, Oxford, 1992). [6] M.J. McAleer, A. Pagan and P. Volker, What will take the con out of econometrics?, Amer. Econom. Rev. 75 (1985) 293-307. [7] M. Lovell, Data mining, Rev. Econom. Statist. 65 (1983) 1-12. [8] A. Pagan, Three econometric methodologies: a critical appraisal, J. Economic Surveys 1 (1987) 3-24. [9] A. Pagan, Three econometric methodologies: an update, in: Oxley et al., eds., Surveys in Econometrics, (Basil Blackweli, Oxford, 1994) chapter 3 [10] D.F. Hendry, E.E. Leamer and D.J. Poirier, A conversation on econometric methodology, Econometric Theory 6, (1990). [11] A. Darnell and L. Evans, The Limits of Econometrics (Edward Edgar, 1990). [12] D. Dharmapala, On the history and methodology of econometrics, J. Econom. Surveys 7 (1993) 85-104. [13] N.R. Ericsson, J. Campos and Hong-Anh Tran, PC-GIVE and David Hendry's econometric methodology. Board of Governors of the Federal Reserve System, International Finance Discussion Paper, # 406, 1991. [14] C. Gilbert, Professor Hendry's econometric methodology, Oxford Bul. Econom. Statist. 48 (1986) 283-307. [15] C. Sims, Macroeconomics and reality, Econometrica 48 (1980) 1-48. [16] M. Uschold, The use of typed lambda calculus for comprehension and construction of simulation models in the domain of ecology, Ph.D. Thesis, University of Edinburgh, 1991. [17] A. Bundy, L. Byrd, G. Luger, C. Mellish, R. Milne and Palmer, Solving mechanics problems using meta-level inference, Proceedings of IJCAI-6 and Expert Systems in the Micro-Electronic Age (1979). [18] M.J. McAleer, Sherlock Holmes and the search for truth: a diagnostic tale, in:Oxley et al., eds. Surveys in Econometrics (Basil Biackwell, Oxford, 1994) chapter 5.