CMMI - Center for Systems and Software Engineering

Process Improvement in the Aerospace Industry CMMI and Lean Six Sigma USC CS510

Rick Hefner, Ph.D. [email protected]

Agenda

• Current Challenges Facing the Aerospace Industry • Current Industry Approaches – Capability Maturity Model Integrated – Lean Six Sigma

• Northrop Grumman Approach

2

NDIA Top 5 Systems Engineering Issues (2006) • Key systems engineering practices known to be effective are not consistently applied across all phases of the program life cycle. • Insufficient systems engineering is applied early in the program life cycle, compromising the foundation for initial requirements and architecture development. • Requirements are not always well-managed, including the effective translation from capabilities statements into executable requirements to achieve successful acquisition programs. • The quantity and quality of systems engineering expertise is insufficient to meet the demands of the government and the defense industry. • Collaborative environments, including SE tools, are inadequate to effectively execute SE at the joint capability, system of systems (SoS), and system levels. 3

Systems Engineering Update, NDIA Top 5 Issues Workshop. July 26, 2006. Briefing by Mr. Robert Skalamera

Agenda

• Current Challenges Facing the Aerospace Industry • Current Industry Approaches – Capability Maturity Model Integrated – Lean Six Sigma – Agile


4

Heritage of Standards for Systems Engineering

2002

2002

ISO/IEC 15504

ISO/IEC 19760

(FDIS)

(PDTR)

1998 1994

EIA / IS 632 1994 1974 1969

Mil-Std499

Mil-Std499A

Mil-Std499B (Not Released)

EIA 632 (Full Std)

1998

1994

(Trial Use)

ISO/IEC 15288 (FDIS)

(Interim Standard)

IEEE 1220

2002

1998

IEEE 1220 (Full Std)

EIA/IS 731 SE CM (Interim Standard)

2002

CMMISE/SW/IPPD

Legend Supersedes Source for

5

Standards for Systems Engineering, Jerry Lake, 2002

The Frameworks Quagmire Sarah A. Sheard, Software Productivity Consortium

6

http://stsc.hill.af.mil/crosstalk/1997/sep/frameworks.asp

Two Complimentary Approaches to Process Improvement Data-Driven (e.g., Lean Six Sigma)

Model-Driven (e.g., CMMI)

• Clarify what your customer wants (Voice of Customer) – Critical to Quality (CTQs)

• Determine the industry best practice – Benchmarking, models

• Determine what your processes can do (Voice of Process) – Statistical Process Control • Identify and prioritize improvement opportunities – Causal analysis of data

7

• Anticipate your customers/ competitors (Voice of Business) – Design for Six Sigma

• Compare your current practices to the model – Appraisal, education • Identify and prioritize improvement opportunities – Implementation – Institutionalization • Look for ways to optimize the processes

Agenda



8

What is the Capability Maturity Model Integrated?

• The CMMI is a collection of industry best-practices for engineering, services, acquisition, project management, support, and process management – Developed under the sponsorship of DoD – Consistent with DoD and commercial standards

Three Constellations sharing common components and structure • CMMI for Development - used by engineering organizations • CMMI for Acquisition - used by buyers (e.g., govt. agencies) • CMMI for Services - used by service providers (e.g., help desk)

9

Basic Building Blocks – 22 Process Areas Implemented by the organization

Implemented by each project

Project Management • Project Planning • Project Monitoring and Control • Supplier Agreement Management • Integrated Project Management) • Risk Management • Quantitative Project Management 10

Engineering • Requirements Development • Requirements Management • Technical Solution • Product Integration • Verification • Validation

Support • Configuration Management • Process and Product Quality Assurance • Measurement and Analysis • Decision Analysis and Resolution • Causal Analysis and Resolution

Process Management • Organizational Process Focus • Organizational Process Definition • Organizational Training • Organizational Process Performance • Organizational Performance Management

Expected Practices Provide Guidance for Implementation & Institutionalization Project Planning – Implementation SG 1 Establish Estimates SP 1.1 Estimate the Scope of the Project SP 1.2 Establish Estimates of Work Product and Task Attributes SP 1.3 Define Project Lifecycle Phases SP 1.4 Estimate Effort and Cost SG 2 Develop a Project Plan SP 2.1 Establish the Budget and Schedule SP 2.2 Identify Project Risks SP 2.3 Plan Data Management SP 2.4 Plan the Project’s Resources SP 2.5 Plan Needed Knowledge and Skills SP 2.6 Plan Stakeholder Involvement SP 2.7 Establish the Project Plan SG 3 Obtain Commitment to the Plan SP 3.1 Review Plans That Affect the Project SP 3.2 Reconcile Work and Resource Levels SP 3.3 Obtain Plan Commitment 11

Project Planning - Institutionalization GG 2 Institutionalize a Managed Process GP 2.1 Establish an Organizational Policy GP 2.2 Plan the Process GP 2.3 Provide Resources GP 2.4 Assign Responsibility GP 2.5 Train People GP 2.6 Manage Configurations GP 2.7 Identify and Involve Relevant Stakeholders GP 2.8 Monitor and Control the Process GP 2.9 Objectively Evaluate Adherence GP 2.10 Review Status with Higher Level Management GG 3 Institutionalize a Defined Process GP 3.1 Establish a Defined Process GP 3.2 Collect Improvement Information

Practice Ratings for the Organization/Projects

12

How is the CMMI Used for Process Improvement?

IDEAL Model

www.sei.cmu.edu/ideal/ 13

Typical CMMI Benefits Cited in Literature • Reduced costs – 33% decrease in the average cost to fix a defect (Boeing) – 20% reduction in unit software costs (Lockheed Martin)

• Faster Schedules – 50% reduction in release turnaround time (Boeing) – 60% reduction in re-work following test (Boeing)

• Greater Productivity – 25-30% increase in productivity within 3 years (Lockheed Martin, Harris, Siemens)

• Higher Quality – 50% reduction of software defects (Lockheed Martin)

• Customer Satisfaction – 55% increase in award fees (Lockheed Martin) 14

Agenda



15

What is Lean Six Sigma (LSS)?

• Lean Six Sigma is a powerful approach to improving the work we do • LSS improvement projects are performed by teams • Teams use a set of tools and techniques to understand problems and find solutions

16

+

• Lean Six Sigma integrates tools and techniques from two proven process improvement methods

Six Sigma • A management philosophy based on meeting business objectives by reducing variation – A disciplined, data-driven methodology for decision making and process improvement

• To increase process performance, you have to decrease variation • Greater Too early

Defects

Too late

Defects

Delivery Time Spread of variation too wide compared to specifications

17

Too early

Too late

Reduce variation Delivery Time Spread of variation narrow compared to specifications

predictability in the process • Less waste and rework, which lowers costs • Products and services that perform better and last longer • Happier customers

DMAIC Roadmap

Define

Measure

Improve

Control

Define project scope

Identify needed data

Explore data

Identify possible solutions

Define control method

Establish formal project

Obtain data set

Characterize process & problem

Select solution

Implement

Evaluate data quality Summarize & baseline data

[Hallowell-Siviy 05] 18

Analyze

Update improvement project scope & scale

Document Implement (pilot as needed) Evaluate

DMAIC Toolkit

Define Benchmark Contract/Charter Kano Model Voice of the Customer Voice of the Business Quality Function Deployment

Measure GQIM and Indicator Templates Data Collection Methods Measurement System Evaluation

Analyze

Improve

Control

Cause & Effect Diagrams/ Matrix

Design of Experiments

Statistical Controls:

Failure Modes & Effects Analysis

Modeling

Control Charts

Statistical Inference

Tolerancing

Reliability Analysis Root Cause Analysis, including 5 Whys Hypothesis Test

ANOVA

Time Series methods

Robust Design Systems Thinking Decision & Risk Analysis PSM Perform Analysis Model

Non-Statistical Controls: Procedural adherence Performance Mgmt Preventive measures

19

Design for Six Sigma (e.g., DMADV)

Define Define project scope Establish formal project

Measure Identify customers

Explore data

Research VOC

Design solution

Benchmark

Quantify CTQs

20

Analyze

Predict performance

Design

Verify

Develop detailed design

Evaluate pilot

Refine predicted performance Develop pilot

Scale-up design Document

Lean

• Series of tools and techniques refined by Toyota and called the “Toyota Production System” – Called “Lean” by Womack, Jones and Roos in The Machine That Changed the World

• Focused on increasing efficiency by eliminating non-value added process steps and wasteful practices • Being adopted world-wide by both manufacturing and transactional based organizations • Utilizes tools like “Value Stream Mapping,” “Just in Time” and “Kaizen” LEAN FOCUS: ELIMINATE WASTE AND REDUCE CYCLE TIME 21

Wastes in Production

CORRECTION

• WAITING

Repair or Rework

Any non-work time waiting for tools, supplies, parts, etc..

PROCESSING Doing more work than is necessary

Types of Waste

INVENTORY

MOTION Any wasted motion to pick up parts or stack parts. Also wasted walking

OVERPRODUCTION Producing more than is needed before it is needed

CONVEYANCE

Maintaining excess inventory of raw mat’ls, Wasted effort to transport materials, parts, or parts in process, or finished goods into or finished goods. out of storage, or between processes.

22

Organizational Adoption: Roles & Responsibilities • Champions – Facilitate the leadership, implementation, and deployment • Sponsors – Provide resources • Process Owners – Responsible for the processes being improved • Master Black Belts – Serve as mentors for Black Belts • Black Belts – Lead major Six Sigma projects – Typically requires 4 weeks of training

• Green Belts – Lead minor Six Sigma teams, or serve on improvement teams under a Black Belt – Typically requires 2 weeks of training 23

A Typical Lean Six Sigma Project in Aerospace

The organization notes that systems integration has been problematic on past projects (budget/schedule overruns)

A Six Sigma team is formed to scope the problem, collect data from past projects, and determine the root cause(s)

The team’s analysis of the historical data indicates that ineffective peer reviews are leaving significant errors to be found in test

Procedures and criteria for better peer reviews are written, using best practices from past projects

A pilot project uses the new peer review procedures and criteria, and collects data to verify they solve the problem

The organization’s standard process and training is modified to incorporate the procedures and criteria, to prevent similar problems on future projects 24

Agenda



25

Northrop Grumman Approach:

Mission Success Requires Multiple Approaches

Risk Management Systems Engineering Independent Reviews & Cost Estimates

Dashboards for Enterprise-Wide Measurement

Program Effectiveness

Training, Tools, & Templates

Process Effectiveness

Mission Assurance & Enterprise Excellence

Communications & Best-Practice Sharing

Operations Effectiveness

CMMI Level 5 for Software, Systems, and Services ISO 9001 and AS-9100 Certification 26

Six Sigma

Robust Governance Model (Policies, Processes, Procedures)

Organizational Infrastructure Required for CMMI Level 3 Process Group

Training Program

Measurement Repositories Predictive Modeling

Best-Practice Libraries

Audits & Appraisals

Defects per component

Policies, Processes, Templates & Tools

Process Improvement

Communications

25 20 15 UCL

10

_ X

5 0 1

11

21

31

41

51

61

71

Component #

Developing and maintaining mature processes requires significant time and investment in infrastructure 27

Northrop Grumman Approach:

Institutionalizing Our Improvements We systematically analyze quality and process data and trends to determine how to improve our processes

We improve our process assets based on internal and external best practices Disposition

Analysis ISO/AS9100 Findings CMMI Appraisal Findings

• Systems/ Software Engineering Process Group

Customer Comments

• QMS Working Group

Independent Audits Lessons Learned & Metrics Tools

28

Increasing program efficiency

• Program Management Advisory Board

Industry Standards Internal Best Practices

Configuration Control Board

Information

Deployed to programs

Policy Process

Six Sigma Projects

Procedures

External Best Practices

Checklists and Guides Templates and Examples

eToolkit StartIt!

msCAS My MS Portal

PAL Workbench

PCDB

Northrop Grumman Approach: Lessons Learned • Multiple improvement initiatives helps encourage a change in behavior as opposed to “achieving a level” – Reinforces that change (improvement) is a way of life

• Benefits results from institutionalizing local improvements across the wider organization – CMMI establishes the needed mechanisms

• CMMI and Lean Six Sigma compliment each other – CMMI can yield behaviors without benefits – Lean Six Sigma improvements based solely on data may miss innovative improvements (assumes a local optimum)

• Training over half the staff as Lean Six Sigma Green Belts has resulted in a change of language and culture – Voice of Customer, data-driven decisions, causal analysis, etc. – Better to understand/use tools in everyday work than to adopt the “religion” 29