Application of Artificial Neural Networks to the Design Optimization of ...

4 downloads 215877 Views 835KB Size Report
application of artificial neural networks to capture structural design expertise is demonstrated. The principal ad- vantage of a trained neural network is that it ...
NASA

Technical

Memorandum

Application

of Artificial

Networks

to the Design

of Aerospace

Laszlo

Berke

Lewis

Research

Cleveland,

Surya

Ohio

N. Patnaik

Ohio

Aerospace

Brook

Park,

Pappu Lewis

Center

Institute Ohio

L. N. Murthy Research

Cleveland,

Center

Ohio

National Aeronautics Space Administration

and

Office of Management Scientific and Technical Information Program 1993

Structural

4389

Neural Optimization Components

about

Summary

the electrochemical

system, The

application

structural

design

vantage

expertise

of a trained

computational For

of artificial

the class

expert

the present

effort,

nonlinear

code

for a ring straints

used.

were

generated

by using were

processed

put pairs,

which

were

used

to develop

the code

NETS.

Optimum

conditions

wcre

Neural

prediction

of optimum

for most

of the output

However, must

results

from

be exercised

within

selected

predicted

by using

the present

to ensure error

designs

design

have

artificial

for new net-

both

neural

arc some

targets

are

bounds.

As

Introduction The

nervous basic

forming

such

with

from

neurons

a biological

extremes, mans

system

design:

neural

as slugs

around

only

lel, information-processing, the ultimate

grossing

at a rapid

pace.

cations,

neurons

number

power.

nections

reside

intellectual terms,

with

The

in the cerebral

capabilities.

is the

size

of

hard

Other

cover.

hard-wired

life support

emotions, number

inherited of neurons

cognition

emerged,

lions

of years

some

level

opportunity

tive

tissue

tense and

studies theway

with

creatures

what

exhibit

more

it does.

We

literature (Garret,

designer

neural

a fundamental

future

only

lt193,

Rumelhart

(ref.

appear

major

in the number

explore

size

computationally

the subject

of in-

to its capabilities havelearncd

much

The

expert

ANN

fced-fl)rward, tion ANN

training

current

limitations

objective

of thc

in

used

in

investigation

the applicability

of ANN

feasible

for con-

optimization. design

supervised algorithm.

paradigm.

clustering

was

optimiza-

of variables

to further

structural

The

and is illustrated

techniques,

to have

to (ref.

structural

As in structural

programming

can

Man.,

of ANN.

as an expert

and Hajcla

4).

sampling

J. lntel.

to the theory

network

reported

cogni-

applica-

and NcCleland

mil-

This

in

of loan officers,

a small

Jr. ct at.,

ancestors

ventional

of

applica-

Industrial

model.

to feed-

nel models inanarrowdo-

jobs

by Bcrkc

level

especially

here was

enhance-

applications

experts.

introduction

mathematical

the problem

will

to perform

the mathematical

when

their

for business

and

or

with

capabilities

Neural

awareness

instinctive

proappli-

numbers with

intc_ practical

neural

problem

size,

These

attempted

1-4)).

net capabilities

problem

At a critical

hundreds

entities.

are

ANN

promising

is diverse;

and (refs.

using

in the

market

was suggested

tion,

perform

way

J.H.,

use of a trained

tissue

and self-organizing

It has been

net

1) provide

neumath-

also.

here

be published

in ANN

of expertise

tool

Neural

at a "toy"

in threat avoidance.

and physiology

are growing

network

science,

conventional

thousand.

living

stock

neural

medical

in the near

is being

bc given

The

than

and

as-

encompassing

technology,

as a profit-making

in its protective

responding

programmable,

and it is trainable. on its anatomy

our human

to input,

of our

in physical

only

their

ANN

Vision,

of expertise

artificial

advancements

to a few

order

functions

classification,

accumulation The

accumulation

found

tax auditors,

is and

quick-response

ancestors.

All living

or engaging

wet

connectivities,

reaction

brain

tions

tions

paral-

of most

no

including

beginning

has hard-wired,

it does

newspaper,

our reptilian

and their

for

and their con-

cortex,

of the 3-pound

from

of

allow

human

the seat

cerebral

functions,

of cognitive

ing

capabilities

ago.

The

up for packaging

parts

areas.

to miniaturization

cortex,

a six-page

and crumpled

have

many

and complexity

of our neurons

The

main

and hu-

and the

brain

and engineering.

or greater

lower

between

of neurons

tissue.

approaching learning

of this massively

respect

majority

1/4 in. thick,

difference

the

and

In current

in computational

science

sophisticated

implcmentalions.

psychology,

limited

computer

muhidisciplinary,

science,

neurons

other

and diversity

areas

living

technology

processing

The dozens

organization

of the various

follows

to many

is the number

Their

the specialization

to humans

network.

with

billions,

the connectivities.

slugs

connected

is truly

millions

ultimate

capabilities

of intriguing

of this activity.

science,

approach

fidelity

this

arithmetic

understanding,

learning,

computer

mcnts

same

language

in computer

connectivilies

others. Y-MP's

real-lime

of increasingly

in software

physiology,

ematics,

that caution

than Cray

time,

advancing

models

and

research

robiology,

rapidly

network

memory,

same same

to the

introduction

natural

sociative

the

close

better

is not

cerebral

calculator.

and the

tasks

activi-

and

The

a thousand

At the

come

sized

in hardware

perception,

that

modeling

cannot card

now,

future.

certain

is such

of meaning.

motiwtled

(ANN)

found

variables

to perform

difficulty

perception

The

parameters.

indicate

all

would

in the ncrw)us physical

known

in the foreseeable

for example,

Biotechnology

neu-

was

has evolved

have

trained

design

study

that

the

designs

cortex

that occurs

the measurable

for us is not

known

of a credit

and out-

artificial

lobe

con-

codes. input

likely

and

activity

in which

meaning

technology

designs

and dynamic

a trained

work.

to be satisfactory

neural

to obtain

design

net

multiple

the optimization

data

with

code with

for static

design

in

an artificial

A set of optimum

wings

optimum ral network

of a

difficult,

acquire

Vision,

design.

development

optimization

were

trivial

new

the

and

ad-

it requires

be extremely

algorithms

two aircraft

principal

an acceptable

would

a structural

NETS

and

is that

ties

to capture

The

addressed,

system

programming

network

network

to produce

of problems

conventional

networks

is demonstrated.

neural

c|'fort

neural

but the way

More

and classification

model

considered

learning This

is the simplest

sophisticated of dala

here

and an error

and most

approaches (ref.

is based

on

back-propaga-

5) or other

popular

inw_lving candidate

approaches, such as functional links (ref. 6) or radial base functions (RBF), are under investigation at this time. An ANN is trained first, by utilizing available information generated from several similar optimum designs of aerospace structural components. The trained artificial neural network, as the expert designer, is then used to predict an optimum design for a new situation. This situation should resemble closely, though not identically, the conditions of the training set, bypassing conventional reanalysis and optimization iterations. The major advantage of a trained neural network as an expert designer over the traditional computational approach is that results can be produced with trivial computational effort. Further, the predictive capability of a trained network is insensitive to numerical instabilities and convergence difficulties typically associated with computational processes (e.g., during reanalysis, direction generations, one-dimensional searches, and design updates of the nonlinear optimization schemes). The disadvantage in generating sufficient design sets to train the artificial neural network is the potential expense. In this report, the feasibility of ANN as an expert designer is considered for a complex set of engineering problems, representative of the optimum designs of structural components of the aerospace industry. The components are a ring, an intermediate complexity wing, and a forward swept wing. The number of design variables used in the optimization problems range from 60 to 157 and the number of behavior constraints range from 200 to 400. The design load conditions and constraint limitations are selected to ensure that, at optimum, all three types of behavior constraints (i.e., stress, stiffness, and frequency) become active. The design sets required to train the neural networks for the three components are generated with an optimization code CometBoards, which is described later. The neural network training is carried out through the code NETS, developed at NASA Johnson Space Center (ref. 10). The optimization code CometBoards was run on a Convex mainframe computer at NASA Lewis Research Center to generate the training data sets, and NETS was run on a SUN SPARC workstation to train the neural networks. This report is divided into the following five subject areas: (1) a feed-forward back-propagation artificial neural network, (2) structural optimization, (3) code CometBoards, (4) discussion of neural net results, and (5) conclusions.

A Feed-Forward Artificial Neural

as knowledge is accumulated. The term "neural network" refers to a collection of neurons, their connections, and the connection strengths between them. Figure ] shows an idealized neural network where the artificial neurons are shown as circles, the connections as straight lines, and the connection strengths (or weights) as calculations derived during the learning process for a problem. This network contains three layers-an input layer, an output layer, and a hidden layerwith each layer consisting of several neurons or nodes. The adaptation scheme used is based on the popular delta error back-propagation algorithm. In error back-propagation, the weights are modified to perform a steepest-descent reduction of the sum of the squares of the differences between the generated outputs and the desired outputs as indicated in the training pairs. The optimal number of nodes in the hidden layer and the optimal number of hidden layers can be problem dependent. These numbers, however, should be kept low for computational efficiency. A rule of thumb is to start with a single hidden layer with the number of nodes equal to about half the total number of variables in the input and output layers. The numbcr of nodes and layers should be increased if convergence difficulties are encountered, but should not exceed the total number of input and output variables. A simpler network with no hidden layers may be computationally efficient, but it represents only linear mapping between input and output quantities. These are known as flat networks and can be inadequate to model function determines

nonlinear relationships. The activation the response of the neuron and is the

only source of introducing nonlinearities in the input-output relationships. The details of the back-propagation scheme have been described by Rumelhart and NcCleland (ref. 7). A brief discussion of the theoretical background follows. A typical neural net configuration consists of an input layer, an output layer, and one hidden layer (as shown in fig. 1). Each layer consists of several nodes or neurons. To gain

_

layer

Back-Propagation Network layer

Neural biological

network simulations represent attempts to emulate information processing. The fundamcntal proces-

sor is thc neuron, or brain cell, which receives input from many sources and processes these to generate a unique output. The output, in turn,'can be passed on to other neurons. Learning is accomplished by changing connection strengths

Input layer Figure1.--A simpleneuralnetworkmodel.

E i = (T/-Y/)

x-

where Ti and Yi are the target and the actual output for node i, respectively. The error signal in equation (4) is multiplied by the derivative of the activation function for the neuron in question to obtain

I=-z

"LJ

¢5i,k =

Figure

2.--A

single processing

(4)

element,

Ei

(5)

where the subscripts i and k denote the ith neuron in the output layer k. The derivative of the output Yi of the sigmoid function is obtained as follows:

insight into the mechanism of information processing, it is better to focus on a single node (fig. 2), which receives a set ofn inputs xi, i = 1,2 .... , n. Thcse inputs arc analogous to electrochemical signals received by neurons in mammalian brains. In the simplest model, these input signals are multiplied by the connection weights wi), and the effective input to the elements is the weighted sum of the inputs as follows:

--' = V_(1-_) 5Z The strength of connections ceding hidden layer to the adjusted

by an amount

(6)

between all neurons in the preira neuron in the output layer is

Awpi,k as follows:

/I

Zj = Z

wq x i

(1)

AW pi, k = rlSi, kYp, )

(7)

i=1

In the biological system, a typical neuron may only produce an output signal if the incoming signal builds up to a certain level. This biological characterstic is simulated in the artificial neural network by processing the weighted sum of the inputs through an activation function F to obtain an output signal as follows: V = F(Z)

(2)

The type of activation function that was used in the present study is a sigmoid function. The sigmoid function is given by the expression 1 F (Z) = 1 + e_,Z+T ,, ,

(3)

This expression was adopted by the NETS computer code that was used in the present study. In equation (3), Z is the weighted input to the node, and T is a bias parameter used to modulate the element output. The principal advantage of the sigmoid function is its ability to handle both large and small input signals. The determination of the proper weight coefficients and bias parameters is embodied in the network learning process, which is essentially an error minimization problem. In the delta error back-propagation approach, the nodes are initialized arbitrarily with random weights. The output obtained from the network is compared to the actual output (supervised

learning)

and the error

Ei is computed

as follows:

In equation (7), Yp.j denotes the output of neuron p in the hidden layer j immediately before the output layer; Avvpi,k is the change in value of the weight between neuron p in the hidden layer to neuron i in output layer k; and 1/ denotes a learning rate coefficient (usually selected between 0.01 and 0.9). This learning rate coefficient is analogous to the step size parameter in a numerical optimization algorithm. Rumelhart and NcCleland (ref. 1) present a modification to the approach by including a momentum term as follows:

Awt+lpi,

k = ll¢_i,kYp,

j + O_Awtpi,

k

(s)

Superscript t denotes the cycle of weight modification. The inclusion of the oe term, which would incorporate a memory in the learning process, increases the stability of the scheme and helps in preventing convergence to a local optimum. This approach is applicable, with some variations, in the modification of weights in other hidden layers. The output of hidden layers cannot be compared to a known output to obtain an error term. Hence, the following procedure is used. The _'s for each neuron in the output layer are first computed as in equation (5) and used to determine the weights of connections from the previous layer. These 5's and w's are used to generate the 6's for the hidden layer immediately preceding the output layer as follows:

aP'J

= YP'J

Z

ai'kWpi'k

)i

(9)

where 6r, j is the 6corresponding to the pth neuron in the hidden layer and Yp,j is the derivative of the activation function of this neuron as computed in equation (6). Once the 6's corresponding to this hidden layer are obtained, the weight of connections fied by an application change in the indices.

of the next hidden layer can be modiof cquation (7) with appropriate This process is then repeated for all

where fn represents natural frequencies of the structure and fno represents the limitations on these frequencies. In a mathematical programming technique, the optimal design point _pt is reached starting from an initial design Zo in, say, K iterations. The design is updated at each iteration by the calculation of two quantities, a direction 0vector, and a step length

a.

The design

remaining hidden layers. The process must bc repeated for all input training patterns until the desired level of error is attained. A modification of the approach is to present a sum of errors of all training patterns from the very beginning.

Structural

Optimization

The structural design scribed as the following: Findthc

optimization

problem

can be de-

n dcsignvariablcs,_withinprcscribedupper

and lower

bounds,

make the scalar

(zti "