2015 International Conference on Data and Software Engineering
Service Orchestration using Enterprise Service Bus for Real-time Government Executive Dashboard System Kabul Kurniawan
Ahmad Ashari
Department of Computer Science & Electronics Gadjah Mada University Yogyakarta
[email protected] Abstract—The increasing implementation of various platforms and technology of information systems also increases the complexity of integration when the integration system is needed. This is just like what has happened in most of government areas. As we have done from a case study in Sleman, a regency of Yogyakarta Indonesia, it has many departments that use different platform and technology on implementing information system. Integration services using point-to-point method is considered to be irrelevant whereas the number of services are growing up rapidly and more complex. So, in this paper we have proposed a service orchestration mechanism using enterprise service bus (ESB) to integrate many services from many departments which used their owned platform and technology of information system. ESB can be the solution of nto-n integration problem and it strongly supports the implementation of service oriented architecture (SOA). This paper covers the analysis, design and implementation of integration system in government area. Then the result of this integration has been deployed as a single real time executive dashboard system that can be useful for the governance in order to support them on making decision or policy. Functional and performance testing are used to ensure that the implementation of integration does not disrupt other transaction processes. Keywords— Service Orchestration Architecture (SOA), Enterprise Service Bus (ESB); Government Executive Dashboard, Information System Integration.
I.
INTRODUCTION
The utilization of information technology, especially information systems in Sleman doesn’t have a policy or special rules to develop information systems centrally. Every department has been given a freedom to develop their own information systems in accordance with duties and functions in each work unit. It causes the existing information systems heterogeneous and varied in terms of platforms, databases and programming languages [1]. The increasing implementation of various, different platform and technology of information would impacts the increasing quantity and complexity of integration when integration is needed. Integration services using point-to-point method is considered to be irrelevant whereas the number of services are growing up rapidly and more complex. The interactions between
Department of Computer Science & Electronics Gadjah Mada University Yogyakarta
[email protected] services implemented with point-to-point architecture have some problems and limitations [3]. Service Oriented Architecture (SOA) is a technology that helps integrate heterogeneous systems, as it provides a data bridge between incompatible technologies and is used nowadays all over the world. A system based on SOA will package functionality as a suite of interoperable services that can be used within multiple separate systems from several domains [2]. With SOA, the bus architecture of the EAI has further evolved to a new architectural concept, the ‘enterprise service bus’ (ESB). The idea of using an ESB is to provide a unified, standard based messaging layer where all the interactions between services take place [3]. Andy Katz [4] has implemented the open source enterprise service bus to connect distributed and heterogeneous applications and services. ESB can be used to be as centralized management for the applications that have different message formats and incompatible transport bindings. ESB can do the mediation of those differences. Mulik [5] presented an implementation of ESB to connect many corporate functions and services with the business division in an enterprise corporation. He said that an enterprise corporation has many division, many corporate function and services and need real-time and looselycoupled integration. ESB implementation helps the client to integrate many application on their business division with the applications owned by the corporation in real-time and looselycoupled. Every function of ESB such as routing, transformation and orchestration have been applied to realize the integration process. Safuwan [6] did the research to integrate enterprise resource planning (ERP) software through service oriented architecture (SOA). The research said that the implementation of enterprise service bus (ESB) is the solution of n-to-n integration complexity problem. ESB concept strongly support the implementation of service oriented architecture (SOA) paradigm. Interaction between service components are done through ESB mediator. It has brought the nature of loosecoupling of the interactions among services and accommodated the management of distributed system.
978-1-4673-8430-8/15/$31.00 ©2015 IEEE
207
2015 International Conference on Data and Software Engineering Then, in this paper we have proposed a service orchestration mechanism using enterprise service bus (ESB) to integrate many services from many departments in Sleman, which used their owned platform and technology of information system on executing their business process. Then the result of this integration has been deployed as a single real time executive dashboard system that can be useful for the government in order to support them on making decision or policy. II.
services provider or service requester. Services may include special components like orchestration engines, adapters for data or resource, adapters to external systems with the transformation of message or transport protocol conversions. ESB can mediate messages between components, deciding where to route messages, and transforming the messages. ESB requires a persistent memory such as connected to the database. Figure 2 shows the architecture of ESB [10].
BACKGROUND ON SOA & ESB
According to Erl [7] service oriented architecture (SOA) is modeling software built with the service oriented approach. Service oriented is an ideal approach that has a vision that every process or resources of software cleanly partitioned from each other. Each resource is called a service. This service represents a business logic or automation logic in an enterprise system. Each service has its own autonomy that makes it independent from the others. Each service can communicate with one to another via a standardized protocol to make it easier to integrate new services and rearrangements collection of service due to changing business processes. As shown in Figure 1 – each process is partitioned as a resource that called a service.
Figure 2 – The Architecture of ESB III. RESEARCH PROCESS This research aims develop a real time executive dashboard system based on service orchestration mechanism using enterprise service bus (ESB). This orchestration will involve many web services from many departments in government that used their owned platform and technology of information. This research will adopt the following research phases as shown in Figure 3.
Figure 1 – Process Encapsulation as a Service According to Chappel [8] enterprise service bus is a standards-based integration platform that combines messaging, web services, data transformation and intelligent routing to connect and coordinate the interaction of a number of diverse applications in a company with transactional integrity. Bus service approach to integration is the use of technology that provides a bus for application integration. Different applications do not communicate with each other directly but communicate via SOA middleware’s backbone. Architectural feature that most distinguishes ESB is a distributed nature of topological integration. ESB is a set of middleware services that provide integration capabilities. This Middleware services is the heart of ESB architecture that puts the message to be routed and transformed [9]. Every component can take on the role of
Figure 3 – Research Proses A. Requirement Gathering The first phase is requirement gathering. It will collect and analyze the requirements in order to develop executive dashboard system based orchestration service. First, we do several interviews to the government official to collect data and information required to be analyzed and identified the case study in order to develop real time executive dashboard system. The interviews were conducted with the chief of information organization on communication & informatics department. We addressed on the following type of questions: • What kind of data and information required should be displayed on Executive Dashboard?
208
2015 International Conference on Data and Software Engineering • Where could the data and information required be retrieved from?
department of education, youth and sports of Sleman to record the schools ranging from elementary to high school in Sleman.
• Who owns and manages the data and information?
C. Architectural Design & Implementation Based on the analysis of the functional needs described above, we make an architectural design of system integration to meet the needs of the integration process. Figure 4 shows the architectural design of system integration using the enterprise service bus (ESB). Every system will be connected as a centralized database on the ESB.
• What type of platform, information system and database they used to manage the data and information? • How does the information system connect? • How to represent the executive dashboard system? Those questions has been asked in order to get information in detail so it could be used to analyze and identify the requirement of system development. B. Case Study Analysis & Existing Application Identification This section will discuss about the analysis of the existing application system and determine the information system and database that will be the case study object of this research. First, we do an observation. The observation is carried out to all departments in Sleman in order to know what kind of implemented information system, performed business processes and the platform of database that is used for storage and processing of data. The application or databases selected in this case study has relevance to the issue of integration of the system, data source needed and the relation between the application or database. Based on the observations of some departments, the following databases are used as the objects of the research e.g. population database, poverty database, license database and school database. Table 1 shows the selected database as object of the research. Table I. The Selected Database as Object of the Research Databases Population Database Poverty Reduction Database Licensing Database School Database
Owner Department of Population and Civil Registrar Department of Social and Labor Department of Investment and Integrated Licensing Department of Education Youth and Sports
Platform MS-SQL Oracle
Figure 4 – Architectural Design System Integration using The Enterprise Service Bus (ESB) 1.
Physical Network Architecture
As shown in Figure 5, the network used in this research is the local network owned by the government of Sleman. The Executive dashboard server (192.168.35.xx) and ESB middleware server (202.162.35.xx) will be located at the secretariat office of Sleman. The ESB middleware server is locally connected through the central tower. The population database server (192.168.12.xx) is located at the department of population and civil registration. The poverty database server (192.168.100.xx) is located at the department of labor and social affairs. Then the licensing database server (192.168.80.xx) is located at the department of investment and integrated licensing. Last, the school database server (192.168.100.xx) is located at the department of education, youth and sports.
PostgreSQL MySQL
Population database is the database which used to store and processes the entire people of Sleman. It’s also coupled utilization delivered with the population information system to manage data management and the population in department of population and civil registrar. Poverty database is owned by the department of social and labor of Sleman. Then the licensing database, It’s used by the department of Investment and Integrated Licensing. This database is used together with the licensing information system to perform data processing related to the permitting. Then the last is School database used by the
Figure 5 – Physical Network Architecture
209
2015 International Conference on Data and Software Engineering 2.
Real-time Executive Dashboard System
Real-time executive dashboard is an application that can real-time display the summary and report a variety of information from multiple data sources in a certain scope of company or agency useful in supporting and determining the direction of policies and decisions. The data of population, data on the number of poor people, data on the number of license and schools statistical data at a particular area and at a certain time, for example the area villages, sub-districts or districts in a given year. Based on the analysis of the table structure and their relationships, we obtained information in which every database has a regional property which covers an area of villages, subdistricts and districts. For example, for the population data, every data can be displayed by village, sub-district and district levels. Figure 6 shows the use case diagram of integration process.
Figure 7 – Architectural Design of Integration by Service Orchestration The operations are orchestrated by patterns-utilizations and rules that have been defined within the ESB. The results of orchestration service mechanism are then poured into a new service that can be accessed by client applications. For example, the client can simply use the service operation "ExecutiveAllProxy" then the client will get all the data from each database in different regions and periods given year. 3.
Figure 6 – Use Case Diagram Integration Process Access many services performed simultaneously to obtain a single output actually can be optimized by doing a web service orchestration. Web service orchestration is a process to combine and coordinate several different web services which is defined by a certain pattern. It has been defined to obtain the expected results. Web service orchestration is one of many features that are owned by ESB. By determining the scenario to display data from various databases using the service by leveraging the ESB, it can be described as an architectural design of integration by service orchestration mechanism as shown in Figure 7. As shown from the Figure, the ESB initially create some services and operations from various database sources. For example, to get the amount of population data can make an operating under the name "getPopulationByVillage", then to obtain data at the district level using "getPopulationBySubdistric" operation.
Orchestration Process Design
Orchestration process are done in a ESB proxy service. The process begins when there is an incoming request sent by the client, the proxy service will call with the mediator "send" on the operation which will be executed. If operations are invoked more than one then it must be subjected to clone using a mediator "clone" in order to be able to call a proxy to more than one operation in parallel. After the successful operation is executed and returns the response, then the responses of each of these operations will be collected by the mediator "aggregate" for later combined into a single response. Once the merger is completed before the response can be returned to the client. Figure 8 shows a proxy service that is used to perform the orchestration of multiple services and the operation of four different databases. For example proxy "VillageExecutiveProxy" is used to invoke the operations to get the data in the village.
Figure 8 – VillageExecutiveProxy Service Orchestration
210
2015 International Conference on Data and Software Engineering There are three mentioned proxy service that will generates a single response, services included "VillageExecutiveProxy", "SubdistrictExecutiveProxy" and "DistrictExecutiveProxy. The three proxy service could be orchestrated back and it will generate a new proxy service. The purpose are used to determine orchestration addressing where the proxy that will be called if the client sends a request with certain parameters. The process of selecting a proxy service is done with mediators "Filter" within the ESB. mediator "Filter". it will determine for example if a client simply sends the parameter village, sub-district and year, the proxy will choose a calling "VillageExecutiveProxy". Then, if the client only sends the parameters districts and in the proxy will be called is "SubDistrictExecutiveProxy", but if the proxy only sends the parameters of the proxy will call "DistrictExecutiveProxy" and in the end if any parameter specifying the client is not included in the proxy will not invoke any operations. Here as illustrated in Figure 9.
getSchoolByDistrict
School Service
SELECT
4. Orchestration Implementation The orchestration process service is divided into two stages. The first stage is to do the orchestration service to get data from all databases and the second stage is to determine the orchestration addressing, where the proxy will be called if the client sends a request with certain parameters. Oorchestration process is divided into village, sub-districts and districts. Here the pseudo code orchestration made for sub region as shown in Figure 10
Figure 10 – Executive Proxy Service Algorithm
Figure 9 – Routing Appropriated Proxy Service As described in the analysis of previous systems that were developed a service orchestration of services that are made from each database. So it needs to be made in advance service for each of these databases. Scenario that is done is to get the data from each database based on region village, district and county. Each database service created their own operations so that each service has several operations. Here are some of the services that make for each database as shown in Table 2. Table II. OPERATION AND SERVICE PROVIDED Operations getPopulationByVillage getPopulationBySubDistrict getPopulationByDistrict getProvertyByVillage getProvertyBySubDistrict getProvertyByDistrict getLicenseByVillage getLicenseBySubDistrict getLicenseByDistrict getSchoolByVillage getSchoolBySubDistrict
Services Population Service Population Service Population Service Poverty Service Poverty Service Poverty Service License Service License Service License Service School Service School Service
Query Type SELECT SELECT SELECT SELECT SELECT SELECT SELECT SELECT SELECT SELECT SELECT
On Proxy "ExecutiveAllProxy" orchestration process begins when there is a parameter village, district and year are entered, then the proxy service will "call” the operation. If operations are invoked more than one then it must be subjected to cloning using the syntax "clone" in order to be able to call a proxy more than one operation in parallel.After the successful operation executed and returns the response, then the responses of each of these operations will be gathered with the syntax "aggregate” and then combined into a single response. Once the merger is completed before the response can be returned to the client as a response. Furthermore, the process orchestration to districts and counties use the same way as is done in the village, if the parameters passed just load the districts and in the proxy will perform cloning, calling and aggregation operations throughout the districts. However, if the parameters passed only a few short years, the proxy will perform cloning, calling and aggregation operations throughout the district. The three proxy service orchestration result of the above each of which is a new service that generates a single response. Services include "ExecutiveProxyService", "ExecutiveSubDistrictProxyService" and "ExecutiveDistrictProxyService" are orchestrated third service can then return to the orchestration of the second stage. The following pseudo code of the orchestration service as shown in Figure 11. The process of selecting a proxy is done with the syntax "Filter" in the ESB. for example "Filter" will determine if a client simply sends the parameter village, district and year, the proxy will choose to call "Village-Paramed ExecutiveProxy".
211
2015 International Conference on Data and Software Engineering Table III. The Results of Performance Testing Operations
Figure 11 – Executive Proxy Service Routing Algorithm Then, if the client only sends the districts parameter, it will call "SubDistrict-Paramed ExecutiveProxy", however if the proxy only sends the null parameters, the proxy will call "District – Paramed ExecutiveProxy" and if any parameter specifying the client is not included, so the proxy will not invoke any operation. D. Functional & Performance Testing Functional testing is performed to determine the functionality of the system that has been developed. The test is performed by running applications and services developed and implemented in each process. Based on the functional test, the executive dashboard system can real-time display the information retrieved from multiple-connected database, using orchestration mechanism with ESB as shown in Figure 12.
getPopulationByVillage getPopulationBySubDistrict getPopulationByDistrict getProvertyByVillage getProvertyBySubDistrict getProvertyByDistrict getLicenseByVillage getLicenseBySubDistrict getLicenseByDistrict getSchoolByVillage getSchoolBySubDistrict getSchoolByDistrict AVERAGE
AET (Second) 0,078 0,083 0,079 0,077 0,077 0,078 0,081 0,079 0,080 0,082 0,092 0,084 0,081
ATPS 12,74 12,05 12,64 12,92 12,93 12,80 12,36 12,64 12,50 12,10 10,90 11,86 12,37
ABPS 4306,12 3348,51 2796,09 4894,79 3930,72 2720,73 27897,59 41509,76 95591,25 13374,43 12223,91 13461,10 18837,92
IV. CONCLUSION The results show that the enterprise service bus (ESB) can be used as a mediator data integration and information systems specially integration process in Sleman District Executive dashboard system can real-time display the information retrieved from multiple-connected database through orchestration mechanism with ESB. Moreover, the results of performance testing show that the average execution time (AET) for service operations amounted to 0,081 seconds. Furthermore, the system has an average amount of execution about 12.37 times and it produces the data about 18837.86 bytes or equivalent to 18.83 KB per second. These results indicate that the execution time is considered to be small enough so that the services developed considered will not disrupt the transaction process. REFERENCES [1]
Figure 12 – The Display of Real-time Executive Dashboard Performance testing conducted to determine the performance of the services that have been developed. At this performance testing we used a computer server with specifications speed 3199.954 MHz of CPU, 3.0 GB of RAM, centos linux OS and running through the Sleman local network. The tests are conducted to know the average of execution time (AET), average of transaction per Second (ATPS), and average of bytes per second (ABPS). The following test results for each operation of several services that have been developed. Testing is done for each operation for 60 seconds and given with virtual user (thread) about 10 then the experiments are repeated up to 10 times and then generate its average. The results as shown in Table 3.
Prasojo W. B.,”Information Integration Approach Using Service Oriented Architecture to Support Government Public Services Sleman”. Gadjah Mada University, Yogyakarta, 2014. [2] SOA – Terminology Explained, 2010. Retrieved from: http://wzcww.lemongrassconsulting.com/knowledge/terminologyexplained [3] Anonymous, “The Evolution of Integration:A Comprehensive Platform for a Connected Business”, WSO2. 2015. [4] Andy Katz, “Implementing an Open Source Enterprise Service Bus”, IEEE Journal. 2006. [5] Mulik, S., “Using Enterprise Service Bus (ESB) for connecting Corporate Functions and Shared Services with Business Divisions in a large Enterprise” I EEE Journal. Mumbai, India. 2009. [6] Safuwan, “Integrasi Perangkat Lunak Enterprise Resource Planning (ERP) dengan Menggunakan Metode Service Oriented Architecture (SOA)”. Institut Teknologi Sepuluh Nopember, Surabaya.2010. [7] Erl, T., “Service-Oriented Architecture: Concepts, Technology, and Design”, Prentice Hall PTR, Upper Saddle River, New Jersey 07458. 2005. [8] Chappell, D., “Enterprise Service Bus”. Gravenstein Highway North, O’Reilly Media, Inc.2004. [9] Andary, J.F. and Sage, A.P., “The role of service oriented architectures in systems engineering”, Information Knowledge Systems Management 9 (2010), IOS Press.2010. [10] Juric, M.B., Loganathan, R., Sarang, P., dan Jennings, F., “ SOA Approach to Integration”, Packt Publishing, Birmingham, B27 6PA, UK 2007
212
