Performance Measurement, Information Technology and Participatory Process in Metro Manila Water Distribution by
Peter D. COOK 1 Jonathan STEVENS 2 Dr. Aura MATIAS 3
ABSTRACT
The paper describes the use of new technology and process change to create a multi-faceted system of communication and feedback between the privatized utilities, the public and the MWSS (regulator of the water distribution system in the Metro Manila area) in the Philippines. It describes a technique for public performance assessment based on service quality indicators constructed both from utility-generated data and from user-generated data and the information system that was developed along with GIS technology to communicate it. The MWSS used this process to facilitate public participation in evaluating utility performance and interacting with utility managers to establish priorities for service improvements. The MWSS adopted four new means of communicating with the stakeholders, using different technologies. This paper describes the changes in the participatory process that resulted from the use of the new technologies and how these resulted in positive actions for consumers, utility managers and the regulatory agency. Finally, it presents lessons learned for Public Participation in government and implications for accountability of all parties that can be applied in many countries.
1 Founding Member Building and Sharing Partnerships (b-sharp) 2329 N. Jackson St. Arlington, VA 22201
[email protected]
2 Vice President PM Global Infrastructure, Inc. 1915 I St., NW, Suite 700 Washington DC 20006
[email protected]
3 Professor University of the Philippines National Engineering Center Manila, Philippines
[email protected]
1 Introduction Public reporting of service performance generally refers to the publication by regulatory agencies of data provided to them by service providers (Kingdom and Jaganathan, 2001). However, to be effective, this information must be communicated to the stakeholders in a process that facilitates feedback and leads to service improvements. Easily understandable, and accessible, service performance information enables the public to put pressure on poor performing utilities to provide better services. The Public Performance Assessment (PPA) process in Metro Manila represents a major extension of current efforts in developing countries to improve utility performance measurement and public feedback for service improvements. It can also be used to set a baseline and monitoring system for service levels under privatization of utilities that incorporates the user perspective and deals with changes in service to different groups of users (e.g., low income households). 2 The Public Performance Assessment (PPA) Project The PPA project was designed to help the Metropolitan Manila Water and Sewer System regulatory agency (MWSS) independently monitor and evaluate the performance of the two private Concessionaires who were awarded concessions in 1997 for delivering water and sewer services to 14 million people (the largest privatization of water distribution in the world). The goals of the Project were to:
Increase accountability for water service performance
Increase the transparency of water service performance
Support good government: people-oriented and service-driven Get customer feedback for better service delivery
Promote results-oriented water service performance
The specific objectives of the PPA project were to:
Improve water distribution service delivery in Metro Manila in an accountable, transparent manner;
Design, implement and evaluate a Pilot Public Performance Assessment (PPA) system for Metropolitan Manila, including:
o
Performance measurement,
o
A PPA information system with data viewing by location and data sharing,
o
An improved public information dissemination and feedback program.
Effect rapid transfer of technology and expertise to Philippine institutions (MWSS and University of the Philippines) for future sustainability and independence of the PPA.
Its initial form was partly a research project and partly an implementation project. The project was created in response to a major concern expressed by both public interest groups and local governments that service might decrease in some areas or for some low-income households in favor of wealthier households as a result of privatization. There were also concerns about the transparency and accountability of the MWSS in monitoring the services provided by the utilities. In response to these concerns, the MWSS and the World Bank created the PPA Project to enhance the information available to MWSS-RO, the utilities and the public concerning the performance of the two Concessionaires in all aspects and to provide this information to the public using independent observers. The new performance information system combined data from both service providers and service users by location and provided both map-based and table-based presentations. The greater availability of information to the public is intended to improve service decisions through feedback from the users, as depicted in Figure 1.
PUBLIC PERCEPTION
SERVICE IMPROVEMENTS
CONCESSIONAIRE
CONSUMER GROUPS
PPA Information
REGULATOR
Figure 1: Public Information Feedback to Improve Service
3 Why is this project different? This is different from other potential improvements for several reasons. First, it establishes a baseline of service measurement in local areas (called Barangays) that is more detailed than available before in either Concessionaire documents or MWSS Regulatory Office information systems (including the baseline studies for privatization). This baseline provides service performance information with a customer focus as well as the usual utility focus, and is therefore more meaningful to a range of stakeholders in assessing the service provided and the changes in service since privatization. Secondly, the PPA contains a public dissemination and feedback process that is revolutionizing the role of public participation for the Regulatory Office. The RO has recognized that it now has the tools to be more proactive in their approach to interacting with stakeholders. As a result it has organized four different ways to communicate (a) through the traditional media news releases, (b) by “Road Shows” to bring performance data maps to the local governments, (c) by providing a hands on “Performance Café” for customers and interest groups at the MWSS headquarters, and (d) sponsoring a “Performance Corner” in the Concessionaire local offices where customers can see how their service compares with other locations. Also an MWSS Internet web site is being created to show performance on the web (among other things). Thirdly, local competency and expertise for future sustainability of the PPA was developed from inception through completion of the project. Filipino experts from the University of the Philippines National Engineering Center (UP-NEC) were designated by the MWSS as the recipient of the transfer of technology component of the project. Consultants and local counterparts worked together on “twin” arrangements thus building in local content with a recognized, independent, trust-worthy organization. The prominent role of UP-NEC as an independent assessor of performance counteracts the mistrust of public officials that is common among interest groups and the public in Metro Manila. The result is the start of a win-win-win situation with the Regulatory Office, the Concessionaires and the other stakeholders. The new information that is now available through new technologies is being used to
improve water service in Metro Manila through a much more effective interaction of all three groups. This is also a good foundation for monitoring any changes in privatized water service throughout the region. 4 What was produced? The PPA project produced (a) a set of products for improving the capability of the MWSS regulatory office, (b) a set of performance measures, (c) an information system for decision support and (d) a new process for communication and feedback among the stakeholders (Renardet/Berger Group 2001). 4.1 PPA Products The First Phase of the PPA Project was successful in several aspects:
The PPA Project created a performance measurement system that is: -
practical
-
easy to comprehend and apply
-
comprehensive yet flexible
-
promotes feedback to both MWSS and Concessionaires; and
-
adapted to Philippine conditions.
A PPA database was created and implemented with a GIS-based Data Viewer that was customized to MWSS-RO needs for Decision-Support;
An information system was developed for use by the Concessionaires for data sharing with the MWSS-RO as a “Partnership Tool”;
A Public Information Plan and program were created and partially implemented as part of the Phase 1 pilot, and Phase 2 is continuing the program with UP-NEC support; and
A plan for sustainable funding of the PPA Activities was developed (with 0.5% to 1% of the tariff rate per cubic meter as a user fee).
4.2 Performance Measures PPA performance indicators were developed from a combination of sources including World Bank research in other countries, other current performance practice, concessionaire contractual obligations, and the analysis of the available performance data. The performance criteria established for the PPA were based on previous work by Stevens and Cook (1997, 1993). They are:
Network Quality (pressure and hours of service)
Water Quality (bacterial contamination and chlorine content)
Risk of Communicable Diseases (Health Quality)
Service Quality (response to complaints and speed of resolution)
Coverage (percent households by location, including low-income areas)
The performance criteria are evaluated from two different points of view – that of the Concessionaire/utility (i.e. Provider-level) and that of the User / Customer (i.e., Consumer-level). For each viewpoint, performance indicators were identified and calculated for 100 Barangays throughout the Metro Manila area. This also allowed the ranking of Barangays according to their performance in each indicator and for combined performance across all indicators. The Consumer-Level performance indicators are based on consumer responses to a set of survey questions that reflect perception of service provision while the Provider-Level performance indicators are based on review and analysis of Concessionaire data. Statistical measures of performance were calculated from provider data, using new techniques not previously employed by the Concessionaires. These indicators
were aggregated and combined to accomplish a range of performance reporting objectives as a means of decision-support for the MWSS Regulatory Office. (Note that Health Risk was kept separate from the other indicators and coverage by household was considered but not retained, due to lack of detailed data) The most effective aspect of this performance system was the ability to see local performance results in color-coded maps (see below). This attracted both public and utility manager interest. 4.3 Information System The PPA project served two useful purposes with respect to MWSS’s Information System. First, it created a database of performance measures, combined with a database of information concerning the water distribution system infrastructure, and a digital geographic (GIS) base map. Second, it started a process of re-engineering the Management Information System (MIS) for MWSS, which was still in an undeveloped state and did not serve the agency’s goals. Another important contribution of the PPA project was the development of a software tool called the PPA Pilot Data Viewer. This allows the MWSS staff, the Concessionaires and the Public (in selected locations) to view the data on performance, by category throughout the Metropolitan Area, either in map-format or table format. It provides automatic color-coding of Barangays and/or municipalities according to performance ratings and then displays them on the screen or on a printed map. This technology also allowed MWSS to map low-income areas and their service levels. The UP-NEC is also creating an internet-based website for allowing the public and other stakeholders to access service performance information over the worldwide web. One of the side-effects of the PPA process was the realization by the MWSS that the present information system provides only 30% of the information that the management needed to make regulatory decisions. This led the MWSS-RO to conduct a business process re-engineering study with the objective of improving the decision-support system and make it more oriented to specific decisions. Finally, it led to a proposal for a more developed MIS with the PPA integrated as a component, which is now being implemented, with extensive on-the-job training. 4.4 Improved Feedback Process Before the PPA project, MWSS was a purely reactive organization with one-way communication, which issued press releases responding to issues raised by the press or relayed news of happenings in the Metro Manila water system and the MWSS organization. It also tried to create positive stories in the press to improve the MWSS image as a regulator, with little success. The PPA project instituted a four-pronged strategy to create a more pro-active approach. The four components were: (a) “Road Shows” to bring performance data maps to the local governments, (b) a hands-on “Performance Café” for customers and interest groups at the MWSS headquarters, (c) a “Performance Corner” in the Concessionaire local offices where customers can see how their service compares with other locations and (d) an MWSS Internet web site is being created to show performance on the web. These are illustrated in Figure 2. Road Shows The Road Show component was the first one implemented by the MWSS, and the first to produce results. The Road Show was created as a new means of providing feedback to those communities (Barangays) that had been surveyed for their opinions about water service performance in their areas. The steps taken by the MWSS for a Road Show evolved over time. After this evolution they were: 1.
Prepare reports, maps and survey results for the Barangay (using PPA info system with GIS mapping)
2.
Organize a Barangay meeting with Concessionaire Managers present
3.
Present performance and survey results
4.
Get feedback on Issues
5.
Have Concessionaires show how they plan to improve performance
6.
Elicit an Action Plan and commitments from the Concessionaire
The Road Shows were originally conceived by the MWSS as means of presenting survey results to the public, without any intervention by the Concessionaire. This generated some interest in what service they were receiving, but raised some questions that only the Concessionaire could answer. Then the Barangay representatives in the meeting asked MWSS to invite Concessionaire managers so that they could ask them questions. The MWSS responded to this request and brought the managers to the next meeting. When the questions became specific as to when and where the service would be improved, the managers were put on the spot, but they gave the information. This gave an opening for the MWSS to ask the managers to make a commitment for improving service by a certain date. The MWSS asked for this commitment in writing and then followed up to see if the commitments were observed. The evolved form of the Road Shows was a major form of empowerment for the citizens of Metro Manila who participated. They were using the MWSS as a vehicle for increasing the accountability of the Concessionaires to their concerns. The MWSS greatly benefited from showing a proactive side to the public and increasing the transparency of their actions to monitor the Concessionaires. Performance Café The Performance Café, is a room designated at MWSS headquarters for showing performance to MWSS staff, Concessionaires or general public. This room has information about PPA on the walls and has a computer access to performance data that can be used by anyone with help from PPA staff. They can create and print maps of performance for any indicators in any location or for the area as a whole and give them to the visitors. They can also print out a table of performance indicators for any selected Barangay. The Performance Café represents a major step in making information available to the public in a way that is easy to understand because it is color-coded on a map and it shows a 5-point scale from very good to very poor. It also showcases the GIS technology that can give an answer to the question, “What service are we getting in my Barangay?” Another innovation that was implemented in the Performance Café was the mapping of squatter areas, which are the main low-income areas in Manila. The Concessionaires created new programs to specifically help the squatter areas and also increase their revenues and reduce leakage. The extent of these programs in low-income areas can be seen using the GIS technology for mapping. This has a limitation, however, in that a person must come to MWSS headquarters in order to access the information. Another limitation is that accessing the data requires the assistance of PPA staff. These limitations can be overcome with the implementation of the other two components of the PPA public feedback program. Performance Corners in Concessionaire Offices In order to place the results of the PPA performance analysis close to the consumers, the project recommended the creation of “Performance Corners” in the local offices of the Concessionaires where many people go to pay their water bills. The idea is to have a place designated in these offices where MWSS staff can place maps and description of the service performance of these offices. This would be an extract of the information from the PPA Performance Café that is oriented to the local area served by the office. To complement the Performance Corner, the PPA project also recommended that the Chief Regulator also give an award for best performance and most improved performance that could be displayed in those offices. The idea is that this would give a positive incentive for the Concessionaires to adopt the Performance Corner concept. This component is still under discussion between MWSS and the Concessionaires.
Figure 2: IT Links Between MWSS, Concessionaires and the Public
Concessionaires Management and IT Depts.
MWSS Technical Regulation Dept. Customer Service Reg. Dept.
Raw Data, Perf. Reports and Maps
Public Performance Assessment (PPA) Unit
PPA Perf. Data and Maps
Performance Corners
MWSS “Road Shows”
PPA Perf. Data and Maps
MIS Unit
PPA
Survey Results
Perf. Data and Maps
MWSS Performance Cafe
Website with Performance Pages
University of Philippines NEC Survey Teams
Performance Website Another PPA recommendation that is now being implemented is the development of an MWSS website. This website would provide the public with an overview of the MWSS mission and responsibilities. It had been recommended as part of the proposed MIS development before the PPA project, but not implemented. The PPA expanded on the recommendations for this website to include an interactive performance page. This page would give a visitor the ability to enter a location and see the service performance indicators for that location. This component will expand the availability of the information in the Performance Café to those who have access to a computer and the Internet. 5 Stakeholder Comments Another method used to gain stakeholder feedback during the PPA project was a series of Stakeholder Forums held in Manila. The final stakeholder forum in June 2001 elicited comments from a wide range of stakeholders, notably: “The PPA system has given the MWSS-RO a solid basis for assessing performance and we will use this for both our internal validation of Concessionaire performance and as a basis for being more proactive with our customers.” Col. Angel Agustin, Deputy Administrator, Customer Service Regulation, MWSS-RO. “We do not want the PPA to be used as a hammer, but as a Partnership Tool…It may also give us key information (performance by location) to use in rate rebasing discussions.” Mr. Antonio Aquino, President, Manila Water Company, Inc. “The PPA has great potential for giving important information on Concessionaire performance to people in poor communities. It gives us more hope that there is a structure or system where good performance information can be obtained.” Sister Annie Abion of CHHED Foundation, NGO activist, Manila. “…the inclusion of consumer perception through the Consumer Survey was a breakthrough…we can see a model on how we should really measure our performance and improve the delivery of service to the general public …rest assured that CONSUMERNET will be one with you, in partnership with you, for the effective implementation of the project.” Mrs. Teresa Mahiwo, CONSUMERNET, NGO activist, Manila.
In addition, the National Economic Development Agency (NEDA), which is the national economic planning body in the Philippines, has stated that this is one project that has increased accountability and transparency and not just talked about it. They believe that it could be applied to privatizations of water supplies in other cities in the country. They also stated that the approach may well be applicable to other sectors, such as energy and sanitation. 6 PPA, Privatization, Transparency and Accountability The PPA project also made visible a basic defect in the privatization contracting system used by the MWSS. The performance clauses in the contract were focused on providing 24 hour service at a minimum pressure level to all connected consumers, and on increasing the percent coverage of households in each municipality in the Metro area. This contract was based on a capital budget for improvements intended to achieve this type of performance that was established in a baseline engineering study carried out prior to the bidding on the Concession contracts.
If the performance is not achieved the penalty in the contract is an all or nothing taking of the Performance Bond on the Concession. This turned out to be unrealistic and not suitable for accountability in the Metro Manila situation. After the Concession agreement and before the PPA project was started, the MWSS technical staff and the Concessionaires were beginning to realize that the broad 24 hour pressure target for all locations was not achievable with the planned investments. Instead, they were depending on renegotiation of the concession agreements (called rate rebasing), which was scheduled to occur at the five-year mark, to renegotiate the performance clauses. This realization was not made transparent to the public, although performance maps provided by one Concessionaire to the MWSS as part of required reports showed this to be the case. The PPA process and the start of rate rebasing negotiations caused this issue to surface inside the MWSS. The Concessionaires were not willing to discuss the issue because of the all-or-nothing nature of the contract clauses. There was no allowance for partial performance. It is now apparent that the privatization agreement was not completely transparent and accountability was being side-stepped by both parties. This was due to the unrealistic nature of the performance clauses. It was also due to the lack of attention of the baseline report to the variations in existing performance by location throughout the Metro area. If the PPA had been implemented prior to the privatization, with its emphasis on identifying service by location, there would have been a basis for accountability. It would also have served as a starting point for monitoring actual performance changes by location, and led to increased transparency for the public, since the actual performance would have been clear from the start. This transparency would have led to a better estimate of the necessary capital investments and also would have given the Concessionaires a more realistic basis for making their bids. There would have been no need to hide the difficulty of meeting the performance targets. 7 Project Replication Similar PPA-type projects have been proposed for Jakarta and other cities in Asia, to complement performance analysis with more effective pressure for service improvements. NGOs have expressed interest in implementing more projects of this kind in order to increase transparency of service provision for privatized systems in other countries. The most promising area for replication of the PPA is as a model baseline for privatizations, which would combine information on infrastructure with service performance from both water user and provider (utility) perspectives. The baseline would then serve as a foundation for an on-going monitoring system to be used by the regulatory agency for annual performance assessments and for measuring progress toward contractual service goals by location.
Acknowledgements The authors wish to acknowledge the support and encouragement of Vijay Jagannathan of the World Bank, who worked closely with the MWSS to create this new kind of project. Also we acknowledge the key support of the management of the MWSS who agreed to pursue a project that took a new consumeroriented perspective and involved leadership in a research effort.
References Report: Renardet/Berger Group (2001). Metropolitan Manila Water and Sewerage System Public Performance Assessment (PPA) Project, MWSS, Manila, Philippines, April Paper: Kingdom, W., and Vijay Jagannathan, (2001). “Utility Benchmarking: Public Reporting of Service Performance,” in Viewpoint no. 229. World Bank, Washington DC, March. Report: Stevens, Jonathan and Peter Cook, (1997). Performance Measures for Infrastructure, Findings for the Water Sector, East Java-Bali Urban Development Project Case Study, The World Bank, Washington DC, December. Paper in Book: Stevens, Jonathan, and Peter Cook (1993). "The Service-Level Approach to Measuring Asset Performance," in Ernst Lutz, ed., Toward Improved Accounting for the Environment, Washington, D.C., The World Bank. Paper: Tynan, N. and William Kingdom, (2002). “A Water Scorecard: Setting Performance Targets for Water Utilities,” in Viewpoint no. 242, The World Bank, Washington DC, April.
