Do Networks Really Work? - DBast wiki

Download PDF
51 downloads 20079 Views 2MB Size Report
Comment
Although cooperative, interorganizational networks have become a common mechanism ... delivery of public services, evaluating their effectiveness is extremely ...
Do Networks Really Work? A Framework for Evaluating Public-Sector Organizational Networks Author(s): Keith G. Provan and H. Brinton Milward Source: Public Administration Review, Vol. 61, No. 4 (Jul. - Aug., 2001), pp. 414-423 Published by: Blackwell Publishing on behalf of the American Society for Public Administration Stable URL: http://www.jstor.org/stable/977503 Accessed: 13/03/2009 22:27 Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you may use content in the JSTOR archive only for your personal, non-commercial use. Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at http://www.jstor.org/action/showPublisher?publisherCode=black. Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed page of such transmission. JSTOR is a not-for-profit organization founded in 1995 to build trusted digital archives for scholarship. We work with the scholarly community to preserve their work and the materials they rely upon, and to build a common research platform that promotes the discovery and use of these resources. For more information about JSTOR, please contact [email protected].

Blackwell Publishing and American Society for Public Administration are collaborating with JSTOR to digitize, preserve and extend access to Public Administration Review.

http://www.jstor.org

KeithG. Provan Universityof Arizona

H. BrintonMilward Universityof Arizona

Do

Networks

Really

Work?

A

Framework for

Evaluating Public-Sector Organizational Networks Although cooperative, interorganizational networks havebecomea commonmechanism for ofpublicservices, delivery theireffectiveness isextremely evaluating andhasgenerally complex beenneglected. Tohelpresolvethisproblem, wediscusstheevaluation of networks of community-based, mostlypubliclyfundedhealth,humanservice,andpublicwelfareorganizations. withpressures Consistent to perform effectively froma broadrangeof keystakeholders, we arguethatnetworks mustbe evaluated at threelevelsof analysis:community, network, and organization/participant levels.Whilethethreelevelsarerelated,eachhasitsownsetofeffectivenesscriteriathatmustbe considered. Thearticleoffersa generaldiscussion of network effectiveness, followed byarguments explaining effectiveness criteria andstakeholders at each levelofanalysis.Finally, thearticleexamines howeffectiveness atonelevelof network analysis mayormaynotmatcheffectiveness criteria atanother levelandtheextenttowhichintegration acrosslevelsmaybe possible. An importantissue in the delivery of publicly funded healthandhumanservices at the local-communitylevel is the integrationand coordinationof organizationalproviders into service-deliverynetworks.The developmentand utilizationof these networkshas been a focus of organizational and public policy scholars since at least the 1960s. Much of the early work on the topic (Levine and White 1961; Warren,Rose, and Bergunder1974) focused on the importanceof cooperativerelationshipsamong individual organizations,such as referralsand joint programs,how they work,andthe impactof such relationshipson organizational structureand behavior. Recently,the focus has broadenedfrom a concernwith individual relationships among organizations to an examinationof the multiple interactionsthat comprise full networks, including discussion of how public policy is implemented through networks of cooperating service providers(Agranoff 1991;Alter andHage 1993; Jennings and Ewalt 1998; O'Toole 1997). Empirical researchers, often using sophisticated network-analysis techniques, have triedto understandexactly how agencies coordinate and integrate their activities, often emphasizing differ414 Public Administration Review* July/August 2001,Vol.61, No.4

ences in networkstructuresand governance(Bolland and Wilson 1994; Laumann and Knoke 1987; Provan and Milward 1995). Whathas been lackingin most of this work,however,is an examination of the relationship between interorganizational network structuresand activities and measures of effectiveness. Evaluatingnetworkeffectiveness is critical for understandingwhethernetworks-and the network formof organizing-are effectivein deliveringneededserKeithG. Provanis a professorin the Schoolof PublicAdministration and Policyat the University of Arizona.Dr.Provan'sresearchinterestsfocuson the studyof interorganizational and networkrelationships,includingnetworkstructure, evolution,governance,and effectiveness.Hisempiricalwork on thesetopicsincludesstudiesof buyer-supplier relations,small-firm manufacturingnetworks,and especiallynetworkrelationsamongpublicand notfor-profit healthand humanserviceagencies. Recentworkin thislastarea has focusedon theorganizationof networksof servicedeliveryforindividuals with seriousmentalillnessundermanaged care, and on communitybased diseasepreventionnetworks.Email:[email protected]. H.BrintonMilwardis theMcClelland Professorof PublicManagementat the University of Arizona.Hisresearchinterestsrevolvearoundthe intersection of publicpolicy and management.Dr.Milwardhas focusedmuchof his recentworkon understanding howto efficiently managenetworksof organizations thatjointlyproducepublicservicessuch as mentalhealthand the impactof privatizationon public-serviceprovision.Email:bmilward@ bpa.arizona.edu.

vices to communitymembers.Evaluationof networkeffectiveness is especially importantfor those who formulate publicpolicy at local, state,andnationallevels, so that scarcepublic fundingcan be allocatedto service-delivery mechanismsthat are utilizing resourcesefficiently while adequatelyservingpublic needs. In O'Toole's (1997) terms,if we are to "treatnetworks seriously,"we must understandwhetherthey work. To do this, it is firstnecessaryto understandwhatnetworkeffectiveness means and what issues must be consideredin its evaluation.Unfortunately,thereis little agreementamong organizationaland public-policy scholars or among public administratorsabout how community-basednetworks of healthandhumanserviceorganizationsshouldbe evaluated. The problemis especially acute in view of the multiple stakeholdersthat can and do lay claim to the organizations that comprise a network,and in view of the fact thatmost healthand humanservices are now providedby private,not-for-profitagencies thatareonly loosely monitored and evaluatedby the public entities thatfund them. Inthis articlewe developa rationalefortheneedto evaluate publicly funded, community-basednetworksof organizations,while acknowledgingthe difficultiesinherentin such an evaluationeffort. We then devote most of the article to proposinga frameworkfor networkevaluation.Our model focuses on evaluationof networks at three broad levels of analysis:community,network,and organization/ participantlevels. All three must be considered, though not necessarily equally. It is not our intentionto develop detailed mechanismsfor evaluatingnetworks,but to discuss both the rationalefor evaluationand the issues that must be considered for determiningwhether a network appearsto be performingat a level thatjustifies continued public support.

Issuesfor EvaluatingNetwork Effectiveness To date, there has been very little work devoted to understandingandassessing networkeffectiveness.The prevailing view has been that interdependentgroups of two or more organizationsthat consciously collaborate and cooperate with one anotherare more effective at providing a complex arrayof community-basedservices than the same organizationsare able to do when they go their own ways (Alter and Hage 1993). The logic behind this belief is powerful, and it builds on concepts from game theory that cooperation will produce outcomes that are more favorableto both partiesthanwhen the partiescompete (Axelrod 1984). The belief has been especially strong in health and human services, where norms of competition have not been nearly as strong as they have in the for-profittbusiness sector.Cooperationis particularlyap-

pealing when the profit motive is absent, because the potential downsides of cooperation, such as reduced autonomy, sharedresources,and increaseddependence, are less likely to be seen as a threatto survival.In the public sector resources are often scarce, clients have multiple problems, service professionals are trained in narrow functional areas, and agencies maintain services that fit narrowlyspecified funding categories. Under conditions like these, networks of providers offer a way to provide services effectively while still maintaining acceptable levels of organizationaland professional autonomy. But do public-sector networks really work? Because good comparativenetworkdata that are tied to outcomes arescarce(Lehmanet al. 1994;ProvanandMilward1995), it is still prematureto concludethatnetworksare effective mechanismsfor addressingcomplex policy problems,despite their promise. Yet a reasonable assessment of networkeffectivenessis criticalto justify involvementby provider agencies andto justify public supportof the concept. The difficulties of assessing networkeffectiveness are closely relatedto thoseof evaluatingorganizations,butthey are even more complex. Most recent attemptsto evaluate organizationaleffectiveness have been based on the concept of satisfyingthe organization'skey stakeholders(Freeman 1984). Probablythe most critical stakeholdergroup thatmustbe satisfiedis customers,andthis view has formed the basis of the qualitymovementin recentyears (Deming 1986). By satisfying customersand by maintaininga customer-drivenfocus, the organizationwill presumablybe effective, not only to its customersand clients, but also to other stakeholders,such as suppliers, shareholders,and employees, all of whom stand to benefit by reaping the rewardsthat accrueto a customer-drivenorganization. This approachis logical and certainlyappealingto customers and clients. However, clients representonly one groupof constituents,andfor public-sectororganizations, they may not even be the most critical stakeholdergroup. For instance, welfare agencies andjails both must satisfy the needs of taxpayersandpoliticians,often at the expense of welfare recipientsand inmates.Even for organizations like public schools, where studentsand families are typically viewed as the most importantconstituentgroup and wheresatisfyingtheirneeds is criticalfor success, students' needs are likely to be fragmented,resulting in multiple constituentgroups with very different views about how success shouldbe measured.Giftedstudentsandtheirfamilies may have one view of whatthe school shouldprovide, whereasstudentswith learningdisabilitiesand theirfamilies may have quite a differentview. For all the problemsassociatedwith evaluatingand assessing the effectiveness of organizations,evaluationbecomes even more complex when addressing networks. Networksmustcontendwith thejoint-productionproblem Work?415 Really DoNetworks

of multiple agencies producingone or more pieces of a Knoke 1987; Milwardand Wamsley 1985), we focus on single service. This means, of course, that assessing the networksof serviceprovidersoperatingin local communieffectiveness of a networkis more complex than evaluat- ties. The recent devolutionmovementin the U.S. federal ing a single organization.Most problematicis the fact that government(SmithandLipsky1993)andothergovernments multiple organizationsrequiredealing with multiple sets aroundthe worldhas reinforcedthe importanceof dealing of constituencies. The joint productionof services may with andpayingfor problemsat the local level. Whenpubsatisfy clients with multiple needs, but it may also raise lic services cannot or should not be centralized,often for substantialproblemsregardingresourcesharing,political political reasons,community-basednetworksare the logiturfbattles,regulatorydifferences,andthe like.Thus,some cal mechanismsfor providingpublic services that are, at stakeholdergroupsmay be quite satisfied with delivering least in theory,relativelyefficientand effective. marginal-qualityservicesto clients, so long as the services are deliveredby individualagencies they can understand at the andcontrol.The prospectof high-quality,efficientservices NetworkEffectiveness Level Community deliveredby multipleagencies may not be as appealingto At the broadestlevel of analysis,community-basednetsome stakeholdergroupswhen the coordinatingactivities of these agencies are not well understoodand when their worksmust be judged by the contributionthey maketo the communities they are trying to serve. Most problems in actions cannotbe readily monitoredand controlled. Consistent with a multiple-stakeholder perspective, the publicdomain,particularlyin healthor humanservices, evaluationof networkeffectivenesscan be viewed at three areessentiallycommunityproblemsthatmustbe addressed levels of analysis:the community,the networkitself, and at the community level. While the concept of "commuthe network'sorganizationalparticipants.These levels are nity" is difficult to define with any precision, we use the of concern to three broadcategoriesof networkconstitu- term loosely to describe the local area that is served by a ents:principals,who monitorandfundthe networkandits network.These areas seldom coincide precisely with the activities;agents,who workin the networkboth as admin- boundariesof a municipality,althoughat least some poristratorsand service-level professionals;and clients, who tion of the populace of the town, city, or county presumactuallyreceive the services providedby the network.We ably standsto benefit, directly or indirectly,from the seruse agency theory (Famaand Jensen 1983) as an organiz- vices providedby the network. Thus, networks must be evaluated as service-delivery ing frameworkfor the primarynetworkconstituentgroups, recognizingthatan agentat one level may be a principalat vehicles that provide value to local communitiesin ways another level. The three levels of analysisandtheirimportancefor each Table1 Summaryof NetworkEvaluationRelationships of the major constituentgroups will Levelsof Effectiveness criteria be discussed separately,although in networkanalysis Keystakeholdergroups Community Principals and * Clients Cost to community practice there may be considerable * Clientadvocacygroups * Buildingsocialcapital overlapacross levels. An overview of * Funders * Publicperceptionsthatproblem the relationshipswe explore and re* Politicians is beingsolved * Regulators * Changesin the incidenceof the problem latedeffectivenesscriteriais presented * Generalpublic * Aggregateindicatorsof clientwell-being in table 1. Ourfocus hereis community-based Network Principals * Networkmembership and agents growth * Primaryfundersand regulators * Rangeof servicesprovided networks, which primarily deliver a * Networkadministrative * Absenceof serviceduplication varietyof health and humanservices. organization * Relationship strength(multiplexity) While some of the organizationsin* Memberorganizations * Creationand maintenanceof network volved in these networks are public administrative organization(NAO) * Integration/coordination of services entities,most arenot-for-profit.None* Costof networkmaintenance theless, most of the networks them* Membercommitment to networkgoals selves canreadilybe viewed as publicAgentsand clients * Agencysurvival sector networks, since much of the Organization/ participant * Memberagency boardand * Enhancedlegitimacy fundingthat is receivedby participatmanagement * Resourceacquisition ing agenciesis public.While some au* Agencystaff * Costof services * Individual thorshavefocusedon policy networks, clients * Serviceaccess * Clientoutcomes mostly involvingrelationsamongfed* Minimumconflictfor eral, state, and local governmententimultiprogram agencies ties (Derthick 1970; Laumann and acrossmultiplenetworks 416 Public Administration Review* July/August 2001,Vol.61, No.4

that could not have been achieved throughthe uncoordinatedprovisionof servicesby fragmentedandautonomous agencies. Puttingaside the significantproblemsof actual measurement,the goal of most public networksis to enhanceclient servicesthroughimprovedaccess, utilization, responsiveness,and integration,while maintainingor reducing costs. From a community-levelperspective, networkeffectivenesscan best be evaluatedfirstby assessing aggregate outcomes for the population of clients being servedby the network,andsecond,by examiningthe overall costs of treatmentandserviceforthatclientgroupwithin a given community. In stakeholderterms, a networkmust satisfy the needs andexpectationsof those groupswithin a communitythat have both a directandindirectinterestin seeing thatclient needs areadequatelymet. Obviously,the specific groupof clients served by the networkshould be satisfied, at least in the aggregate.However,the clientsof publiclysupported agencies often arenot a politicallypowerfulinterestgroup by themselves.Quite the reverse,often they are viewed as public problems-the homeless, abusedchildren,welfare recipients, etc. Thus, other groups that represent the community's and clients' interests must be satisfied by network activities. In agency-theoryterms, these are the principals,whose role it is to fund and/ormonitorthe activities of their agents (network agencies), who provide services to clients. These stakeholdersmay include consumeradvocacygroups such as the Alliance for the Mentally Ill, local funderslike UnitedWay,and local officials, both elected and appointed.It also includes the general public, which pays for manyof the services neededby clients throughtaxes and which reapsthe indirectrewardsof a healthier,safer community. Satisfying these groups is obviously problematic, especially since they may not agree on eithernetworkgoals or what would constitute successful network outcomes. While stakeholdergroupsmay agree on some goals, such as cost containment, it is likely they will disagree on others, such as outreach to the underserved. This conflict is often exacerbatedby severe resource constraints that make compromise difficult. Should social services be targetedto adults or to families and children? Should behavioral health resources be used primarily for those with mental illness or for those with substance abuse problems? These types of questions present an obvious dilemma to network evaluators, who ultimately must determinewhich groups constitute the targetpopulation that is to be evaluated. Often, networksof health and humanservice agencies try to satisfy the needs of these diversestakeholdergroups by minimizing problems. In mental health, for instance, this meansnot only ensuringthatcosts arekeptundercontrol, but also avoidingvisibly bad client outcomes, which

would create problems with the public, the media, and elected officials. Agencies providing services to a common pool of clients may work together to minimize the visibility of problemsto the generalpublic-for example, homeless persons in big cities aggressively begging and sleeping on steam grates in downtownbusiness districts. Agencies may also work togetherto minimize morbidity and mortalityrates, such as the suicide rate among the seriously mentally ill, the overall rates of serious crime, or the prevalenceof preventablediseases.These areall benchmarksof how well or how poorly a communityis doing in comparison to similar cities. They are community-level indicatorsof the effectivenessof service-deliverynetworks thatdo not necessarilyreflectthe effectivenessof individual networkmembersor the success of individualintegrated programsfor specific clients. A final way for networksto be evaluatedat the community level is by their contributionto the buildingof social capital(Putnam1993). Fountain(1998) discusses this concept as an importantoutcome of the cooperationand collaboration among agencies and firms. By working together,organizationsin a communitylearn to understand and trustone another,as well as learn whom not to trust. This learningcan be extremelyimportant,not just for the productionof currentservices, but also for the joint productionof services to be performedin the future,particularly in otherservice domains.In theory,a networkof human service agencies, criminal justice agencies, and business firms could be unsuccessful in reducing neighborhoodcrime (a community-leveloutcome) undera particularfederallyfundedprogram.However,the social capital these agencies build could then be drawn on for the smooth and successful implementationof a laterprogram involving many of the same agencies and firms. The network becomes a more efficient and effective service-delivery mechanism,benefitingthe communityin ways that would not have been possible if no social capitalhad been createdand maintained.

at the NetworkLevel Effectiveness While a networkmay benefit the communityin which it is embedded,especially the pool of clients it serves, it must become a viable interorganizationalentity if it is to survive. Effectiveness at the community level means the networkis likely to have considerablelegitimacy and externalsupportby satisfying the needs of clients and other community-interestgroups. However, network effectiveness may come at a cost that is too high to sustainthe involvement of individualnetworkmembers.A networkis not simply one more communityproviderorganization;it is a collection of programsand services that span a broad rangeof cooperatingbutlegally autonomousorganizations. Work?417 Reolly DoNetworks

To operateeffectively, member agencies must act as a network,which means incurringorganizingand transaction costs. These costs may be assumed directly by network members,especially in networks that evolve informally. In formally constructed and taxpayer-funded public-sector networks, however, network growth and maintenanceis often led, coordinated,and governedby a central, local administrativeentity. Lawless and Moore (1989) and Mandell (1984) have referredto this entity as a networkbroker,though we preferthe term networkadministrativeorganization (NAO). In its key role as disseminatorof funds, administrator,and coordinatorof the network, in an agency-theory context, the NAO is both the agent of the community and the principalof the network participants. The effectiveness of a networkand its NAO can be assessed in a numberof differentways, many of which depend on the relativematurityand developmentof the network. The simplest way of evaluating network-level effectiveness is by the ebb and flow of agencies to and from the network.There is no minimumnumberof organizations requiredto make a network succeed; however, networks obviously need to attractand retain members, particularlyduringearly growth,if they are to surviveas a viable form of social organization. Once a networkbecomes well established,effectiveness is not contingenton simplyattractingmoreandmoremembers. Large networks have obvious political advantages, but they may not be particularlyefficient mechanismsfor servicedelivery.While newly establishednetworksshould be gaining members,maturenetworksmay have a rate of networkentrythatonly modestly exceeds the rate of exit, as peripheralagencies drop out as part of the process of networkrefinement,particularlyas core membersworkto enhance service quality and become more efficient. It is likely that many peripheralagencies will be attractedto the network; they may be linked informally, primarily throughreferrals,but effective networks will maintaina limitedcoreof agenciesthatprovidecriticalservices.Some of these core providersmay be the productsof the merger and consolidationof otheragencies as the networkmoves toward greater efficiency. While there is no theoretical upperlimit to the numberof agencies thatcan be partof a network,after surpassinga certainsize, any networkwill become less effective because of increasingcoordination costs, especially in the absence of an NAO. A closely relatedway of assessing network-leveleffectiveness is by the range of actualservices providedby the network, ratherthan simply the number of agencies involved. One key advantageof a networkis that it allows for the provision of a broadrange of services that collectively addressthe full needs of clients. Yet these services may or may not be adequatelyprovidedby the network. 418 Public Administration Reviewa July/August 2001,Vol.61, No.4

At one extreme, only a limited range of services may be offeredby the agencies comprisedby the network,forcing clients to go outside the formalnetworkto meet their full treatmentneeds. At the otherextreme,too many agencies and programsmay be involved, resulting in a confusing arrayof services with considerableduplicationof effort. Thus, network-leveleffectiveness can be judged partly by the extent to which services thatare actuallyneeded by clients areprovidedby the network.Determiningwhatmix of agencies is sufficientto providea complete inventoryof services is rathersubjective,althoughmost professionals within a given service domain could probably agree on which services are most critical and which might be consideredperipheral.While an effective mix of services may evolve informally,for the most part, decisions about the appropriatemix of services often rest with the NAO because it may fund the network and authorizeagencies to provide particularkinds of treatmentfor clients. In this way, the NAO acts as both the agent of the community, ensuringthatneeded services areprovided,andthe principal of the networkparticipants,monitoring,coordinating, and fundingtheir activities. Newly evolving networksmay be effective if network membersprovide essential services. The mix of network services should graduallyexpand to include both critical and more peripheralservices within the network's core service domain. As the network continues to evolve and mature,effectiveness would then be judged by the mix of agencies across service domains. For example, recently formed networks in criminal justice might include the police (probablyas the core agency,or NAO), courts,jails, halfway houses, probation departments,hospital emergency rooms, etc. As the networkevolves, other agencies could be added,including those in substanceabuse, child and family services, and mental health. Maturenetworks would also include schools, religious organizations, social welfare, housing, andcommunityoutreachprograms. The point is that over time, successful networks should be able to broaden their web of ties, moving from the provision of critical core services by a limited numberof closely linked agencies, to inclusion of agencies and services thatmight be seen as moreperipheral,andthus more loosely connected.Highly effective, maturenetworkshave a seamless qualitythatallows information,resources,and clients to flow smoothly across agencies and programs that span the organizationalfield for health and human services in a community. A third way of evaluatingnetwork effectiveness is to assess the strengthof the relationshipsbetween andamong networkmembers,especially acrossthe full network.During initialformation,the ties amongmemberagencies will be tentativeandcalculated.The networkis new, and agencies that have operatedlargely independentlyor only in-

formally with other agencies are now expected to share resources, information,and clients. Because agencies in the public sector work togetherinformally,the transition to a better-developednetworkis often not as disruptiveas in business networks,where firms must shift from norms of competitionto cooperation.Nonetheless, all organizations are likely to experiencea periodof transitionalcommitment as they move from informal,casual, and easily brokenties to relationshipsthat are either formalized or ones that are less formal but based on trust and commitment built on a history of interactions(Ring and Van de Ven 1994). It remainsto be seen how the new emphasison competitive contractingby public and nonprofitentities (Milwardand Provan 1993; Smith and Lipsky 1993) will affect the need for cooperationamong membersof a service-deliverynetwork. One networkconcept that is particularlysalient in this regard is multiplexity, which refers to the strength of ties

betweennetworkagencies(Scott 1991).Twoorganizations aresaidto havemultiplexties if they areconnectedin more than one way-through referralsand planning links, for example. Such a tie is strongerthan a single link (that is, referralsonly) becausethe relationshipis maintainedeven if one of the two links is broken.It follows that relationships involvingfouror five differenttypes of ties arestronger still. For evaluatingnetworkeffectiveness, multiplexitycan be a particularlyuseful measure.During the early development of networkrelationships,ties among most memberstendto be relativelyweak,or loosely coupled,as agencies test each other's commitmentand reliability.As the networkmatures,some of these links will completely dissolve as agencies discover which relationshipswork and which do not. Otherrelationshipsmay be maintainedat a low level, based on the need for only limited contact and involvementamong network members providing certain types of services. However,if a networkis working well and is to be sustainedover time, the ties among many network agencies will gradually strengthen, particularly amongthose with complementaryservices. Effective,maturenetworksmighthave a majorityof agenciesconnected throughtwo or threedifferenttypes of programsor client services as well as throughgeneral informationsharing andfriendship.Multiplexity,andhence the strengthof the network,will be high, reflectingcommitmentsamongnetwork agencies to one anotherthroughmultipleactivities. A final way of assessing network-level effectiveness is by evaluating its administrativestructure.While the existence of a distinct NAO is not critical to networksuccess, it generally indicates the network is a viable form and resources have been committed to developing the network.While small networks can survive and prosper in the absence of an NAO, such an absence means that

networkgovernanceis left to networkparticipants.In this case, the community has no designated agent to guide, coordinate,and legitimize network activities or to monitor service provision. Such a structureis highly unusual in largernetworksand is likely to produceweak network outcomes. Non-NAO networks require a high level of commitmentto network goals and to interorganizational cooperation by member agencies that is difficult to sustain. For instance, Provanand Milward's(1995) work on mentalhealthnetworksdemonstratesthatin Tucson,Arizona, the absence of a strong NAO-in this case a core mental health agency-resulted in largely informal cooperation and coordination among the many providers. Although there were many links across the network, resulting in high overall integrationamong provideragencies, actual services were not well coordinatedand client outcomes were not favorable. An importantway of assessing network effectiveness throughthe NAO is to evaluate the extent to which the NAO acquiresandthen distributesresourcesfor andto the network. In most communities in the United States, for instancethe success of the UnitedWay, a federatedfunding organizationfor communityservice providers,is measured, at least by its member agencies, by its capacity to attractand distributecommunityfunds to these agencies. In mental health, the core agency is often the principal conduitfor statementalhealthfunds andis responsiblefor obtainingand distributingthese funds.As with the United Way,this puts the core agency in a powerfulposition relative to networkparticipants. Despite this imbalancein powerwithinthe network,the role of the NAO, broker,or core agency is criticalfor networksuccess. Whenfunds aredistributeddirectlyto many providers,thereis a far greaterincentivefor these providers to offer duplicate services and to compete with one anotheras each agency scramblesto get as much as it can. It is the NAO'sjob to ensurethatresourcesare distributed in ways that maximize overall network effectiveness for the benefit of the communityand the populationof clients being served, even at the expense of individualnetwork members.Thus, the NAO acts in its capacity as agent for the communityand as principalto its networkmembers, and not as the agent of members.This role, then, legitimizes the networkandthe activitiesperformedby network members on behalf of clients. Strong central control of resources is not, of course, sufficient to ensure network success;however,as ProvanandMilward(1995) find, such control is an importantindicatorthat a group of service providershave an incentive to cooperate, both with one anotherand with the core agency, to ensure that services are providedefficiently across multiple agencies.

Work?419 Really DoNetworks

at the Organization/ Effectiveness Participant Level Althoughnetwork-andcommunity-leveloutcomes are valid ways of evaluatingnetworks, it is importantto recognize that individual agencies and their managers are still motivated partly by self-interest. For organizations consideringbecomingpartof a network,the relevantquestion is, how can networkinvolvementbenefit my agency? Despite the broadervalue that may accrue to clients and the communityat large as a result of the integrateddelivery of services througha network,networkmembersstill strive to ensure the survival of their own agency. Networks can contributesignificantly to organization-level outcomes. Conversely,the success of network members is criticalto overallnetworkeffectiveness,althoughsometimes network success can be enhancedthroughthe failure of individual members,resulting in some interesting evaluationproblems. The importanceof networkinvolvementfor individual agencies can be evaluatedon four primarycriteria:client outcomes,legitimacy,resourceacquisition,andcost. How networkagencies might enhancetheireffectiveness in the last threeareascan be demonstratedusing the example of a networkof communityhealth and humanservice agencies affiliatedwith a city's United Way organization.Although United Way is not normally supportedby public funds, its affiliatedagencies often receive significantpublic funding, and United Way's own funding decisions are influencedby those of public agencies. Agencies typicallyjoin the UnitedWayto enhancetheir legitimacyin the communityandto acquireresources,primarilyfunding,morereadilythanthey could on theirown. By becomingpartof the UnitedWaynetworkin theircommunity,agenciesacquirestatusandacceptabilitytheycould acquireon its own only after many years of effort.Agencies also become legitimized as part of the broadernetwork of United Way agencies, allowing them to shareinformation,ideas, and clients. The importanceof network affiliationfor gaining legitimacyis just as relevantin publicly funded systems, especially for small, relatively unknown agencies with modestpowerbases in a community (Wiewel and Hunter1985). A closely related outcome of network affiliation for individual agencies is resource acquisition. In United Way networks, small agencies have limited fund-raising capacities, while large agencies generally prefer to devote their limited resources to service provision instead of fund-raising, making network affiliation quite attractivefor both. In most communities, general fundraising can be done far more efficiently and effectively when it is centralized through the United Way than if agencies attemptedto raise funds on theirown. The fund420 Public Administration Review* July/August 2001,Vol.61, No.4

ing of United Way can also have multiplier effects, as when United Way dollars are matched by funds from outside grant sources or when city or county funds are turnedover to United Way for allocation to agencies, as they are in some communities. Forproblemssuchas mentalhealthandsubstanceabuse, access to state funds may be possible only throughaffiliation with an NAO or core agency, such as a regional behavioralhealthauthority.Fundsthatarechanneledthrough an NAO may also interactwith legitimacy effects by enhancing the visibility of affiliatedagencies and their programs,helping them to attractnonfinancialresourceslike boardmembers,key staff, and even clients. While the benefits of network membership are most apparentto smalleragencies thathave low legitimacy and modest capacities to attractresourceson their own, these agencies are also likely to experience the greatest costs. Larger,more visible agencies may get less out of network involvement,buttheircosts are also likely to be lower.For instance, as Provan,Beyer, and Kruytbosch(1980) find, large agencies can exert influence over the United Way, often by threatof withdrawal;an option that is not viable for small agencies. Large agencies often have legitimacy that is well established, allowing them to exert influence in the communitythat may extend to United Way's fundraising efforts. Thus, these agencies may be nearly as importantto United Way as United Way is to the agency, minimizingthe costs of affiliation.Assessing the value of network membership to these large, powerful agencies makes sense only if benefits, in the form of modest legitimacy and resourcegains, are consideredrelativeto costs, which will tend to be relativelylow. A fourth benefit that agencies may derive from network involvement is enhancedclient outcomes. Through the integrationof services, the clients of network agencies can receive a broadrange of needed and coordinated services. Thus, an agency can see benefits to their clients that would not be possible when only one or two uncoordinatedservices are providedby any one agency. For example, an agency providing crisis services to the homeless is likely to see clients less frequentlyif it is partof a network that includes agencies offering a range of supportservices. This allows the crisis agency to devote more time and resources to a smaller pool of especially needy clients. This is the logic of the CommunityCareNetworks, discussed by Weiner and Alexander (1998). In general, agencies will join a network if agency management believes their specific clients can be better served through the integratedservices providedby networkmembersand if the agency's services can be offered more efficiently and effectively.

acrossLevelsof Analysis Integration Network effectiveness is likely based on interactions across all three of the levels of analysis discussed here. Althougheach stakeholdergroupwill be most concerned with effectivenessat one particularlevel of networkanalysis, it is only by minimally satisfying the needs of each group-principals, agents, and clients-that network effectiveness can be fully realized.As figure 1 shows, outcomes at each level of analysishave a directeffect on outcomes at another level. In addition, while each of the broadly defined stakeholdergroups is unique conceptually, in practicethey overlapso that outcomes that satisfy one groupcan at least partiallysatisfy anothergroup.For instance, while principals, like the general public and funders,may be most concernedwith networkeffectiveness at the communitylevel, effectiveness at this level can only be achievedif most (althoughnot all) individualclientsareservedreasonablywell by networkproviders.Similarly,participantorganizationscan often enhancetheirsurvival and resource acquisition by responding to the expectationsof an NAO, broker,or core agency. Figure1 Relationshipsbetween Effectivenessat DifferentLevelsof NetworkAnalysisand Influence by KeyStakeholders Community-leveleffectiveness

Key stakeholders Principals Clients

Organization/ participant-level effectiveness

*

gents

Network-level effectiveness

At the same time, however, network effectiveness at one level does not ensure effectiveness at the other two levels. For example, one importantcaveat regardingorganization/participant-leveloutcomes is that it is not the role of network administrativeentities like United Way or mental health centers to enhance the well-being of individual networkmembers.These NAOs work to satisfy theirprincipalsby enhancingcommunity-leveloutcomes. Network-level effectiveness is also emphasized as the NAO strives to ensure its own survival throughnetwork growth and diversity of services. Thus, the network is considered successful if the community in general, and clients in particular,are better served by an integrated network of providers.

In addition,networksuccess andoverallclient outcomes may be best achievedthroughactions thatruncompletely counterto the goals of organization-levelstakeholders(indicatedin figure 1 by the absence of an arrowgoing from community-levelto organization-leveleffectiveness). For example, a providernetwork may have too many small agencies to achieve the desired economies of scale in the productionof a service. Underthis condition,bankruptcy of several of the agencies could actually increase the network's effectiveness. In addition, the embeddedness (Granovetter 1985) of networks means that while an agency's success may be boosted by its networkinvolvement, the agency may also be hurt as the inappropriate behaviorof a few memberagencies ripplethroughoutthe system.These pointsillustratethe inherenttensionin community service networksbetween the needs and expectations of community-level,network-level,andorganization/ participant-levelstakeholdersand the effectiveness measures valued by each group. The resolutionof this problemis not easy. It does mean, however,thatwhile communitynetworksthatare successful are likely to be effective at all three levels of analysis, stakeholderneeds and expectations are not necessarily consistentacrosslevels. For instance,the communitymay be best served by a networkthat first focuses on the full range of needs of a particularclient group, and then attemptsto coordinateandintegratethe deliveryof required services throughspecific agencies. This may mean shifting resources away from those agencies whose services do not fit network-determinedneeds or that duplicatethe mix of services already provided. Thus, an individual agency may be doing a good job on its own, but the particularservices the agency providesmay be deemedeither nonessential or too costly by the network administrative organization.Thisnetwork-levelassessmentmaythenforce the agency to close down or shift its service focus. In this case, the NAO acts as the agent of its relevantcommunity constituency,representinga particularset of communitylevel intereststhatarenot necessarilyconsistentwith those of some networkmembers. As an example of this interplaybetween community, network,andorganizationlevels, some local mentalhealth authoritieshave shiftedfundingandserviceprioritiesfrom residential to outpatient treatment as a cost-saving device. As a consequence, network agencies that formerly provided residential treatmenthave had to develop different services or go out of business. From communitylevel and network-levelperspectives, this networkcould be judged successful if the shift in prioritiesattractednew members and resources, or if it improved client access and the quality of needed services while reducing costs. From the perspective of some individual organizational members, however, the network failed because its serWork?421 Really DoNetworks

vice direction is now incompatible with these agencies' goals and competencies. Sometimes, the incentive for making community-and network-leveldecisions over organization-levelneeds is based on legal mandate. For instance, one way for less powerful client groups to ensure their needs are reasonably met is to sue public agencies. Judicial takeoversof mental health, corrections, and school systems have resulted from such lawsuits in the United States. In these cases, publicnetworksmaybeartheresponsibilityof implementing change across provider agencies in ways that clearly favorthe activitiesof some types of providers,and some sets of clients, over others.Such legal mandatesmay also mean, however, that network-, organizational-,and community-leveloutcomes in one service domainmay be substantiallyenhanced,as new resourcesflow to the network and its providers,sometimes at the expense of providers and clients in other service domains.

Conclusions This article has emphasized that it is both reasonable and desirable to evaluate networks based on their effectiveness. We discussed the substantialdifficulties associated with evaluatingorganizationaleffectiveness and indicated that, in many ways, assessing network effectiveness is even moreproblematicbecausekey stakeholders and their interests are so diverse. Despite these problems,establishing whetheror not a networkis effective is critical from the perspectives of those organizations that make up the network, those who are served by the network,and those whose policy and funding actions affect the network. The approachused here to evaluatenetworkeffectiveness was not to offer a list of specific andwell-testedmethods. Rather,threebroadlevels of analysis were identified thatresearchers,policy makers,or practicingnetworkadministratorswouldneedto considerin determiningwhether or not a networkis effective. Specifically,public networks can and should be evaluatedat community,network,and levels of analysis. The different organization/participant views of effectiveness at each level need to be considered andresolved,especially in a system thatonly workseffectively throughcooperation. It is importantto note that public-sectornetworks are differentfrom those in the for-profitworld, where the financial performanceof memberfirms is commonly seen as a viable way of assessingnetworkeffectiveness(Saxton 1997). While some stakeholders,like employees or communityleaders,might arguethatnetworkgoals shouldinclude theirinterests,the expectationof improvedfinancial performanceencourages business firms to participatein networks,even thoughgains mightnot be immediate.Pre422 Public Administration Review* July/August 2001,Vol.61, No.4

sumably,the needs of other stakeholdersare reasonably satisfied when performancegains are realized. In the public sector,the needs of constituentgroupsare more diverseand more politicized. In particular,the rationale for public networksis most apparentat what we have referredto as the communitylevel. Public-sectornetworks are most effective when they enhancethe capacityof organizations to solve problems and to serve clientele. This may meanthatsome service providersare dropped,others are expanded,and still others shift their focus. The organizing dilemma is that networksoften comprise well-establishedprogramsand organizationsthat may be highly resistantto change or dismantling.For a networkto work effectively,the needs andinterestsof the people who work for and supportthese programsand organizationsmust be satisfied, while building a cooperative network of relationshipsthatcollectively provides interorganizational services more effectively and efficiently than a system based on fragmentedfunding and services. A fundamentalproblemwith any effortto evaluatepublic networks is that external stakeholdergroups seldom exist for networksas they do for individualorganizations. That is, effectiveness tends to be seen by externalgroups as depending on what specific service providers either do or do not do, ratherthan how well services are provided as a result of network activities. Stakeholderstend to evaluate,reward,or punishindividualagencies, regardless of the network's role in enhancing or limiting client outcomes. Despite the prevalence of networks in the delivery of health and human services in most communities, individual organizationshave constituency groups, but networks do not. Thus, the extent to which community- and network-level decisions can be made at the expense of networkparticipantsis partlypolitical, andpowerful organizational stakeholders may be able to resist community-and network-levelpressures.If constituency groups are active at the networklevel, it is often through a coalition of agencies, like a human services coalition, that protects and expands funding for human services in legislatures and city councils. Despite these problems,networksfundedby the public sector can and should be evaluated.While there will be disagreementaboutgoals and methods,public-sectornetworks cannot be evaluatedsolely on sustainmentof particularprograms,agencies, or networkforms. Rather,the taskfornetworkorganizersis to minimallysatisfytheneeds and interestsof stakeholdersat networkand organization levels, while emphasizingthe broaderneeds of the community and the clients the networkmust serve. Communityvalue may be createdby providingclients with improved access to services, enhanced utilization, reduction in unneeded services, lower overall costs, enhancedclient satisfaction,andimprovedoutcomes.These,

cation of the Dynamic Network Framework.Human Relain turn,will make the communitya more productiveand tions 42(12): 1167-84. viable place to live. The advantageof public-sectornetworks is that many of the individualswho are employed Lehman,Anthony F., Letitia T. Postrado,Dee Roth, Scott W. McNary,and HowardH. Goldman.1994. Continuityof Care by network organizationsare professionals, with values and Client Outcomes in the RobertWood JohnsonFoundaand commitmentto clients and the public good that often tion Programon ChronicMental Illness. MilbankQuarterly outweigh theircommitmentto specific programsor orga72(1): 105-22. nizations.Thus,organization-andnetwork-leveleffective- Levine, Sol, and Paul E. White. 1961. Exchange as a Concepness criteriacan be mostly satisfied by focusing on comtual Frameworkfor the Study of InterorganizationalRelamunity-levelgoals. Service-deliverynetworksmustbe built tionships.AdministrativeScience Quarterly5(4): 583-601. andmaintainedat the organizationandnetworklevels, but Mandell, Myrna P. 1984. Application of Network Analysis to overallnetworkeffectivenesswill ultimatelybe judgedby the Implementationof a Complex Project.HumanRelations community-levelstakeholders. 37(6): 659-79.

Acknowledgments The authorsthankCharlieWise for his commentson an earlier versionof this article,and threeanonymousPARreviewers.

References Agranoff,Robert. 1991. HumanServices Integration:Past and PresentChallengesin PublicAdministration.Public AdministrationReview 51(6): 533-42. Alter,Catherine,andJeraldHage. 1993. OrganizationsWorking Together.NewburyPark,CA: Sage Publications. Axelrod,Robert.1984. TheEvolutionof Cooperation.NewYork: Basic Books. Bolland,JohnM., andJanV. Wilson. 1994. ThreeFaces of InteRelagrativeCoordination:A Model of Inter-Organizational tionsin HealthandHumanServices.HealthServicesResearch 29(3): 341-66. Deming, W. Edward.1986. Out of the Crisis. Cambridge,MA: MIT. Derthick,Martha.1970. TheInfluenceof FederalGrants.Cambridge,MA: HarvardUniversityPress. Fama, Eugene F., and Michael C. Jensen. 1983. Separationof OwnershipandControl.Journalof LawandEconomics26(2): 301-25. Fountain,JaneE. 1998. Social Capital:Its Relationshipto Innovationin Science andTechnology.Science and Public Policy 25(2): 103-15. Freeman,RobertEdward.1984.StrategicManagement:A StakeholderApproach.Boston, MA: PitmanPublishing. Granovetter,MarkS. 1985. EconomicAction and Social Structure:The Problem of Embeddedness.AmericanJournal of Sociology 91(3): 481-510. Jennings, EdwardT., and Jo Ann G. Ewalt. 1998. Interorganizational Coordination,AdministrativeConsolidation, and Policy Performance.Public AdministrationReview 58(5): 417-28. Laumann,EdwardO., and David Knoke. 1987. The Organizational State. Madison,WI: Universityof WisconsinPress. Lawless, Michael W., and Rita A. Moore. 1989. InterorganizationalSystems in Public Service Delivery:A New Appli-

Milward,H. Brinton,and Keith G. Provan. 1993. The Hollow State: PrivateProvision of Public Services. In Public Policy for Democracy,edited by Helen Ingramand Steven Rathgeb Smith, 222-37. Washington,DC: BrookingsInstitution. Milward,H. Brinton,and GaryL. Wamsley. 1985. Policy Subsystems, Networks and the Tools of Public Management.In Policy Implementationin Federal and UnitarySystems, edited by Kenneth Hanf and Theo A.J. Toonen, 105-30. Dordrecht,Netherlands:MartinusNijhoff. O'Toole, LaurenceJ. 1997. TreatingNetworksSeriously:Practical and Research-BasedAgendas in PublicAdministration. Public AdministrationReview 57(1): 45-52. Provan,KeithG., and H. BrintonMilward. 1995. A Preliminary Theory of Network Effectiveness:A ComparativeStudy of Four Community Mental Health Systems. Administrative Science Quarterly40(1): 1-33. Provan,KeithG., JaniceM. Beyer,andCarlosKruytbosch.1980. EnvironmentalLinkagesandPowerin Resource-Dependence Relations between Organizations.AdministrativeScience Quarterly25(2): 200-25. Putnam,Robert. 1993. MakingDemocracy Work:Civic Traditions in Modern Italy. Princeton,NJ: Princeton University Press. Ring, Peter S., andAndrewH. Vande Ven. 1994. DevelopmentalProcessesof CooperativeInterorganizational Relationships. Academyof ManagementReview 19(1): 90-118. Saxton, Todd. 1997. The Effects of Partnerand Relationship Characteristicson Alliance Outcomes.Academyof ManagementJournal40(2): 443-61. Scott, John. 1991. Social NetworkAnalysis:A Handbook.London: Sage Publications. Smith, Steven Rathgeb, and Michael Lipsky. 1993. Nonprofits for Hire. Cambridge,MA: HarvardUniversityPress. Warren,RolandL., StevenM. Rose, andAnn F. Bergunder.1974. The Structureof UrbanReform.Lexington, MA: Lexington Books. Wiewel, Wim, and Albert A. Hunter. 1985. The InterorganizationalNetworkas a Resource:A ComparativeCase Study on OrganizationalGenesis.AdministrativeScience Quarterly 30(4): 482-96. Weiner,BryanJ., andJeffreyA. Alexander.1998. The Challenges of GoverningPublic-PrivateCommunityHealthPartnerships. Health Care ManagementReview 23(2): 39-55.

Work?423 Really DoNetworks