Continuous Software Engineering

Continuous Software Engineering Keynote at Software Engineering 2016, Wien Prof. Dr. Wilhelm (Willi) Hasselbring http://se.informatik.uni-kiel.de/ http://kosse-sh.de/

24.02.2016

W. Hasselbring

1

Agenda 1. Entwicklung des Gebietes Continuous Software Engineering 2. Kontinuierliche Modernisierung 3. Kontinuierliche Qualitätssicherung 4. Kontinuierliche Bereitstellung und Lieferung 5. Welche Auswirkungen hat das auf die Softwarearchitekturen? 6. Was bedeutet das für das Software Engineering? 24.02.2016

W. Hasselbring

2

Continuous Software Engineering ist schon lange ein Thema

[Weber 1999] 24.02.2016

W. Hasselbring

3

Continuous Software Engineering: The Driving Forces

[Weber 1999] 24.02.2016

W. Hasselbring

4

Kontinuierlich ein Thema “Software Center is an experiment to establish an effective, scalable, and long-term software engineering research collaboration between academia and industry.” Jan Bosch, Preface

Springer-Verlag, 2014 24.02.2016

Siehe auch [Fitzgerald and Stol 2015]: “Continuous Software Engineering: A Roadmap and Agenda” W. Hasselbring

5


W. Hasselbring

6

Continuous Software Engineering: Life Cycle

[Weber 2002] 24.02.2016

W. Hasselbring

7

Von der dynamischen Analyse zum Modell-basierten Testen

[van Hoorn et al. 2011, 2014]


W. Hasselbring

9

Continuous Improvement Process Kontinuierliche Qualitätssteigerung und Kostensenkung

Bildquelle: Wikipedia 24.02.2016

W. Hasselbring

10

Continuous Integration Developing Software

Performing Automated Build

Publishing Reports

Performing Automated Test

24.02.2016

W. Hasselbring

11

Kontinuierliche Qualitätssicherung Beispiel: Regressions-Benchmarking Integrated into Continuous Integration Setup [Waller et al. 2015] Should include automated anomaly detection [Marwede et al. 2009, Ehlers et al. 2011] https://build.se.informatik.uni-kiel.de/jenkins/job/kieker-nightly-release/plot/ 24.02.2016

W. Hasselbring

12

Kontinuierliche Resilienz Fehlerinjektion zur Prüfung der Fehlertoleranz: Chaos Monkey - randomly shuts down virtual machines Latency Monkey - simulates a degradation of service Conformity Monkey - detects instances that aren’t coded to best-practices Security Monkey - searches out security weaknesses Doctor Monkey - performs health checks Janitor Monkey - searches for unused resources Quelle: https://github.com/Netflix/SimianArmy 24.02.2016

W. Hasselbring

13

Kontinuierliche Softwarearchitektur Validation im Continuous Integration

Quelle: [Goldstein & Segall 2015] Siehe auch: [Frey et al. 2013, Fittkau et al. 2014b] 24.02.2016

W. Hasselbring

14

Präskriptive und deskriptive Modelle • Für Continuous Integration ist es sinnvoll präskriptive und deskriptive (Architektur-) Modelle zu kombinieren – Präskriptive Modelle kommen aus der Softwareentwicklung (Forward Engineering) – Deskriptive Modelle kommen aus der Beobachtung der im Betrieb befindlichen Softwaredienste (Reverse Engineering)

24.02.2016

W. Hasselbring

15


W. Hasselbring

16

Continuous Software Engineering: Objects of Interest

[Weber 2002] 24.02.2016

W. Hasselbring

17

24.02.2016

W. Hasselbring

18

Zusammenarbeit von Entwicklung und Betrieb

Maßnahmen: • Entwicklung einer Kultur der (agilen) Zusammenarbeit zwischen Entwicklungsabteilung und Systembetrieb. • Qualitätssicherung und Effizienzsteigerung durch Automatisierung von Entwicklungs- und Betriebsaufgaben. 24.02.2016

W. Hasselbring

19

24.02.2016

W. Hasselbring

20

Deployment Pipelines für Continuous Deployment

Beispiel: Deployment Pipeline @ Otto.de. Quelle: [Breetzmann et al. 2014] 24.02.2016

W. Hasselbring

21

Kontinuierliches Monitoring

24.02.2016

W. Hasselbring

22

Monitoring Frameworks

[van Hoorn et al. 2012]

[Fittkau et al. 2013, 2015a] Vortrag morgen 14:00 h, Sitzung “Empirical Software Engineering 3” zu [Fittkau et al. 2015d] 24.02.2016

W. Hasselbring

23

Software zum Anfassen und Erleben solutions.hamburg, September 2015

Siehe auch [Fittkau et al. 2015c]

Siehe auch [Fittkau et al. 2015b] 24.02.2016

W. Hasselbring

24


W. Hasselbring

25

Agilität vs. Softwarearchitektur Neal Ford (ThoughtWorks Inc.), 2010:

– “Architecture is the stuff that's hard to change later. – And there should be as little of that stuff as possible. – By deferring important architectural and design decisions until the last responsible moment, you can prevent unnecessary complexity from undermining your software projects.”

24.02.2016

vs.

[Reussner & Hasselbring 2008, Hasselbring 2005] W. Hasselbring

26

Renaissance & Innovation in Architecture •

• •

•

“Organizations have accepted that "cloud" is the de-facto platform of the future, and the benefits and flexibility it brings have ushered in a renaissance in software architecture. The disposable infrastructure of cloud has enabled the first "cloud native" architecture, microservices. Continuous Delivery, a technique that is radically changing how tech-based businesses evolve, amplifies the impact of cloud as an architecture. We expect architectural innovation to continue, with trends such as containerization and software-defined networking providing even more technical options and capability.”

Quelle: http://www.thoughtworks.com/radar 24.02.2016

W. Hasselbring

27

DevOps & Softwarearchitektur “The deployment pipeline is the place where the architectural aspects and the process aspects of DevOps intersect.” [Bas et al. 2015]

24.02.2016

W. Hasselbring

28

Microservice Architekturen als Enabler für DevOps

Backend Integration Proxy

[Bas et al. 2015]

[Steinacker 2014]

“Scalability is managed by each service individually and is included in its SLA in the form of a guaranteed response time given a particular load.” [Bas et al. 2015, Chapter 4] 24.02.2016

[Kraus et al. 2013]

“The trade-off between many small components and a few large components must be considered in component and system design.” [Hasselbring 2002]

W. Hasselbring

29

Tracking

Link Handler

Auth

AfterSales

User

Order

Insights

Promotion

Product

Search & Navigation

Shoppages

Backoffice

Page Assembly Proxy

z.B. durch Microservices

Aufteilung der Funktionen

Dimensionen der Skalierung

z.B. durch Datenbank-Replikation Duplikation 24.02.2016

W. Hasselbring

30

Polyglot Persistence Tracking

Order

Recommendation

Search

Product

Page Assembly

Backend Integration

Wide Column Database

24.02.2016

Full-Text Database

Graph Database

W. Hasselbring

Relational Database

Time Series Database

31

Wiederverwendung von Komponenten

Quelle: [Weber 2002] 24.02.2016

W. Hasselbring

32

Wiederverwendung von Middleware Component A

Open Source Middleware

Component B

Beispiel: https://github.com/otto-de/ 24.02.2016

W. Hasselbring

33


W. Hasselbring

34

Continuous Quality of Software Knowledge ? • “The engineering of Software Infrastructure depends on the existence of knowledge.” [Weber 2002] • DFG Schwerpunktprogramm 1593: Design for Future - Managed Software Evolution – Knowledge Carrying Software

• Continuous Software Engineering  Continuous Quality of Software Knowledge Potentieller Themenbereich für einen neuen SPP? 24.02.2016

W. Hasselbring

35

Predigt • Möglichst viel automatisiert messen • Dies kontinuierlich überwachen (Monitoring, Darstellung in Dashboards) • Instrumentierung zum Monitoring sollte schon initial in die Software integriert werden, statt dies erst nach Problemen im Betrieb zu tun: • So können die Anforderungen des Betriebs hinsichtlich Application-Performance-Management und Fehlerdiagnose direkt berücksichtigt werden • Performance-Abweichungen werden unmittelbar erkannt, bevor sie über die Deployment-Pipeline in den Betrieb propagiert werden 24.02.2016

W. Hasselbring

36

http://eprints.uni-kiel.de/31415

Veranstaltungshinweise

• Keynote “Microservices for Scalability” at ICPE 2016, Delft (NL), March 15, 2016 • , April 12-13, 2016 Microservice Architectures and Continuous Delivery http://www.softwareforen.de/goto/sar • DevOpsDays Kiel, May 12-13, 2016 http://www.devopsdays.org/events/2016-kiel/

• KoSSE-Tag zu DevOps, June 1, 2016 http://kosse-sh.de/

• Symposium on Software Performance November 08–09, 2016 in Kiel (Descartes/Kieker/Palladio Days 2016) http://www.performance-symposium.org/

24.02.2016

W. Hasselbring

37

References

[Ba s et al. 2015] Len Bass, Ingo Weber, Li ming Zhu: “DevOps: A Software Architect’s Perspective”, Addison-Wesley 2015. [Breetzmann et al. 2014] Robert Breetzmann, Stephan Kra us, Christian Stamm: “Null Toleranz für Fehler: Wie wir a uf otto.de die Qualität hoch halten“, OBJEKTspektrum 4/2014, 18-23. [Ehl ers et al. 2011] Jens Ehlers, André va n Hoorn, Jan Waller, Wilhelm Hasselbring: “Self-Adaptive Software System Monitoring for Performance Anomaly Localization“, In: 8th IEEE/ACM International Conference on Autonomic Computing (ICAC 2011). [Fi ttkau et al. 2013] Fl orian Fittkau, Ja n Waller, Christian Wulf, Wilhelm Hasselbring: “Li ve Trace Visualization for Comprehending Large Software Landscapes: The ExplorViz Approach“, In: 1s t IEEE International Working Conference on Software Visualization (VISSOFT 2013). [Fi ttkau et al. 2014a ] Florian Fittkau, André van Hoorn, Wilhelm Hasselbring: “Towards a Dependability Control Center for La rge Software Landscapes“, In: 10th European Dependable Computi ng Conference (EDCC 2014). [Fi ttkau et al. 2014b] Florian Fittkau, Phil Stelzer, Wilhelm Hasselbring: “Live Visualization of Large Software Landscapes for Ensuring Architecture Conformance“, In: 2nd International Works hop on Software Engineering for Sys tems-of-Systems 2014 (SESoS 2014). [Fi ttkau et al. 2015a ] Florian Fittkau, Sascha Roth, Wilhelm Hasselbring: “ExplorViz: Vi sual Runtime Behavior Analysis of Enterprise Application Landscapes“, In: 23rd European Conference on Information Systems (ECIS 2015). [Fi ttkau et al. 2015b] Florian Fittkau, Erik Koppenhagen, Wilhelm Hasselbring: “Research Perspective on Supporting Software Engineering via Physical 3D Models”. In: 3rd IEEE Working Conference on Software Visualization (VISSOFT 2015). [Fi ttkau et al. 2015c] Fl orian Fittkau, Al exander Kra use, Wilhelm Hasselbring: “Exploring Software Ci ties in Vi rtual Reality”. In: 3rd IEEE Working Conference on Software Visualization (VISSOFT 2015). [Fi ttkau et al. 2015d] Florian Fittkau, Alexander Kra use, Wilhelm Hasselbring: “Hierarchical software landscape visualization for system comprehension: A controlled experiment”. In: 3rd IEEE Worki ng Conference on Software Visualization (VISSOFT 2015). [Fi tzgerald a nd Stol 2015] Brian Fitzgerald, Kl aas-Jan Stol: “Continuous Software Engineering: A Roadmap and Agenda”, In: Journal of Sys tems and Software, July 2015. [Frey et a l . 2013] Sören Frey, Wi lhelm Hasselbring, Benjamin Schnoor: “Automatic Conformance Checking for Mi grating Software Systems to Cl oud Infrastructures and Platforms”, In: Journal of Software: Evolution and Process, 25(10): 1089-1115. [Gol dstein & Segall 2015] Ma ayan Goldstein, Itai Segall: “Automatic and Continuous Software Architecture Validation”, In: 37th International Conference on Software Engineering (ICSE 2015) Softwa re Engineering In Practice Tra ck. [Ha sselbring 2002] Wi lhelm Hasselbring: “Component-Based Software Engineering”, In: Handbook of Software Engineering a nd Knowledge Engineering. World Scientific Publishing, Si ngapore, pp. 289-305, 2002. [Hei nrich et al. 2014] Robert Heinrich, Eri c Schmieders, Reiner Jung, Ki ana Rostami, Andreas Metzger, Wilhelm Hasselbring, Ralf Reussner, Kl aus Pohl: “Integrating Run-Time Observations a nd Design Component Models for Cl oud System Analysis“, In: 9th Workshop on [email protected] 2014. [Kra us et al. 2013] Stephan Kraus, Guido Steinacker, Oliver Wegner: “Teile und Herrsche – Kl eine Sys teme für große Architekturen“, OBJEKTspektrum 5/2013, 8-13. [Stei nacker 2014] Guido Steinacker: “Scaling with Microservices and Vertical Decomposition”, http://dev.otto.de/2014/07/29/scaling-with-microservi ces-and-vertical-decomposition/, 2014. [va n Hoorn et al. 2011] André va n Hoorn, Sören Frey, Wolfgang Goerigk, Wilhelm Hasselbring, Holger Knoche, Sönke Köster, Harald Kra use, Marcus Porembski, Thomas Stahl, Ma rkus Stei nkamp, Norman Wittmüss: “DynaMod Project: Dynamic Analysis for Model-Driven Software Modernization”. In: 1s t International Workshop on Model-Driven Software Mi gra tion (MDSM 2011). [va n Hoorn et al. 2012] André va n Hoorn, Jan Waller, Wilhelm Hasselbring: “Ki eker: A Fra mework for Application Performance Monitoring a nd Dynamic Software Analysis”, In: 3rd joint ACM/SPEC International Conference on Performance Engineering (ICPE 2012). [va n Hoorn et al. 2014] André va n Hoorn, Christian Vögele, Eike Schulz, Wilhelm Hasselbring, Helmut Krcmar: “Automatic Extra ction of Probabilistic Workload Specifications for Load Testing Ses sion-Based Application Sys tems” In: 8th International Conference on Performance Eva luation Methodologies and Tools (ValueTools 2014), [Wa l ler et a l. 2015] Jan Waller, Nils Ehmke, Wilhelm Hasselbring: “Including Performance Benchmarks into Continuous Integration to Enable DevOps“, In: ACM SIGSOFT Software Engi neering Notes, 40(2). [Weber 1999] Herbert Weber: “Continuous Engineering of Information and Communication Infrastructures”, Second International Conference on Fundamental Approaches to Software Engi neering (FASE 1999). [Weber 2002] Herbert Weber: “Continuous Software Engineering”, Fraunhofer ISST, 2002.