Chap 4 The Simplex Method

College of Management, NCTU

Operation Research I

Fall, 2008

Chap 4 The Simplex Method The Essence of the Simplex Method 9 Recall the Wyndor problem Max Z = 3x1 + 5x2 ≤ 4

S.T. x1

2x2 ≤ 12 3x1 + 2x2 ≤ 18 x1, x2 ≥ 0

9 8 corner point solutions. 5 out of them are CPF solutions. 9 Each corner-point solution lies at the intersection of two constraint boundaries. ¾

General speaking, for a LP problem with n variables, each of its corner-point solutions lies at the intersections of n constraint boundaries.

9 For any LP with n decision variables, two CPF solutions are adjacent to each other if they share n – 1 constraint boundaries. ¾

For example, (0, 0) and (0, 6) are adjacent. (0, 0) and (4, 3) are not.

9 Two adjacent CPF solutions are connected by a line segment that lies on these same shared constraint boundaries, call an edge of the feasible region. 9 Note that two edges emanate from each CPF solution. Thus, each CPF solution has two adjacent CPF solutions (in the Wyndor example). ¾

How many adjacent CPF are there for an n decision variable case?

9 Optimality Test—Consider a LP problem that possesses at least one optimal solution. If a CPF solution has no adjacent CPF solutions that are better, then it must be an optimal solution. ¾

(2, 6) is the optimal solution in the Wyndor Example.

Solving the Wyndor Example (from a geometric viewpoint) 9 Initialization—Choose (0, 0) as the initial CPF solution for convenient reason. ¾

Jin Y. Wang

Optimality Test—(0, 0) is not an optimal solution (adjacent solutions are better).

Chap4-1



Fall, 2008

9 Iteration 1—Move to a better adjacent CPF solution, (0, 6). ¾

Between the two edges of the feasible region that emanate from (0, 0), choose to move along the edge that leads up the x2 axis.

¾

Why choose this one?

¾

Stop at the first new constraint boundary: 2x2 = 12.

¾

Solve for the intersection of the new set of constraint boundaries: (0, 6).

¾

Optimality Test: still not an optimal

9 Iteration 2—Following the same procedures, move to a better adjacent CPF solution (2, 6). 9 Optimality Test—(2, 6) is an optimal solution.

Key Solution Concepts Behind the Simplex Method 9 Concept 1: For any problem with at least one optimal solution, finding one requires only finding a best CPF solution. Thus, we focus solely on CPF solutions. ¾

The number of CPF solutions is finite. This simplifies the searching space tremendously.

9 Concept 2: The simplex method is an iterative algorithm.

Find an initial CPF solution

Yes

Stop

Is Current CPF solution optimal? No

Perform an iteration to find a better CPF

Jin Y. Wang

Chap4-2



Fall, 2008

9 Concept 3: Whenever possible, chooses the origin to be the initial CPF solution. 9 Concept 4: Given a CPF solution, it is much quicker computationally to gather information about its adjacent CPF solutions than about other CPF solutions. ¾

While moving from the current CPF solution to a better one, it always chooses a CPF solution that is adjacent to the current one.

¾

The entire path followed to eventually reach an optimal solution is along the edges of the feasible region.

9 Concept 5: After the current CPF solution is identified, the simplex method examines each of the edges of the feasible region that emanate from the current CPF solution. ¾

Among the edges with a positive rate of improvement in Z, it chooses to move along the one with the largest improvement rate.

9 Concept 6: How the optimality test is performed efficiently? ¾

Check the improvement rate of each edge emanate from the current CPF solution.

¾

If all of them are negative, the current CPF solution is optimal.

Solving the Wyndor Example (a algebraic procedure) Max Z = 3x1 + 5x2 ≤ 4

S.T. x1

2x2 ≤ 12 3x1 + 2x2 ≤ 18 x1, x2 ≥ 0 9 Setting up the Simplex method ¾

Introducing slack variables to transfer the functional inequalities into equalities and obtain the augmented form of the model. Max Z = 3x1 + 5x2 St x1 + x3 =4 2x2 + x4 = 12 3x1 + 2x2 + x5 = 18 x1, x2, x3, x4, x5 ≥ 0

¾

If a slack variable equals 0 in the current solution, then this solution lies on the constraint boundary for the corresponding functional constraint.

¾

A value greater than 0 means that the solution lies on the feasible side of this constraint boundary.

Jin Y. Wang

Chap4-3


¾


Fall, 2008

What if the value of a slack variable is less than 0?

9 More terminologies ¾

An augmented solution is a solution for the original variables and the slack variables (feasible or infeasible).

¾

A basic solution is an augmented corner-point solution (feasible or infeasible). For example, (4, 6) is a corner-point solution and (4, 6, 0, 0, -6) is the corresponding basic solution.

¾

A basic feasible (BF) solution is an augmented CPF solution. For example (0, 6) is a CPF solution and (0, 6, 4, 0, 6) is a BF solution.

9 There are 3 equations with 5 variables. So, we can have 2 free variables. That is, if two of the variables are set, the value of the other three variables can be calculated. ¾

The simplex method uses zero for this arbitrary value.

¾

That is, two of the variables (called the nonbasic variables) are set to zero, and then the simultaneous solution of the three equations for the other three variables (called basic variables) is a basic solution.

9 Some properties of a basic solution ¾

Each variable is designated as either a basic or a non-basic variable.

¾

The number of basic variables equals to the number of functional constraints.

¾

Therefore, number of non-basic variables equals the total number of variables (including slack variables) minus number of functional constraints.

¾

The non-basic variables are set to zero.

¾

The values of the basic variables are obtained as the simultaneous solution of the system of equations.

¾

The set of basic variables is often referred to as the basis.

¾

If the basic variables satisfy the non-negativity constraints, the basic solution is a BF solution.

9 Let x1 = x4 = 0 (nonbasic variables), obtain x3 =4, x2 = 6, and x5 = 6 (basic variables). It is a BF solution. Jin Y. Wang

Chap4-4



Fall, 2008

9 Two BF solutions are adjacent if all but one of their basic (nonbasic) variables are the same. (Note: not the values.) ¾

That is, moving from the current BF solution to an adjacent one involves switching one variable from nonbasic to basic and vice versa. (0, 6, 4, 0, 6)

(2, 6, 2, 0, 0)

(4, 3, 0, 6, 0)

(4, 0, 0, 12, 6) (0, 0, 4, 12, 18) 9 We could deal with the objective function equation at the same time as the new constraint equation. Note that Z is not really a decision variable. Max Z S.T. Z - 3x1 - 5x2 = 0 x1 =4 2x2 = 12 3x1 + 2x2 = 18 x1, x2 ≥ 0 The Algebra of the Simplex Method 9 Recall the Wyndor Problem again. 9 Initialization ¾

Add the slack variables

Max Z S.T. Z - 3x1 - 5x2 =0 (0) x1 + x3 =4 (1) 2x2 + x4 = 12 (2) 3x1 + 2x2 + x5 = 18 (3) x1, x2, x3, x4, x5 ≥ 0 ¾ Pick x1 and x2 be the nonbasic variables (starting from the origin). x1 = x2 = 0. ¾

Easily obtain x3 = 4, x4 = 12, and x5 = 18 (x3, x4, and x5 are basic variables).

¾

Why it is easy? z

Jin Y. Wang

Each equation has just one basic variable, which has a coefficient 1, and this basic variable does not appear in any other equation (and the objective function). Chap4-5



Fall, 2008

9 Optimality test Z - 3x1 - 5x2 = 0 Î Z (to be maximized) = 3x1 + 5x2 ¾

Can we improve the objective value any more (Z = 0 currently)?

9 Determining the direction of movement ¾

If we change the value of x1, or x2, can we increase the value of objective function? Which one looks more promising?

Pick to increase, called the entering basic variable. 9 Determining where to stop. (How far could we increase the entering variable?) x1 remain zero. x1 + x3 =4 Î x3 = 4 Î no bound on x2 2x2 + x4 = 12 Î x4 = 12 –2x2 Î x2 ≤ 6 Î minimum 3x1 + 2x2 + x5 = 18 Î x5 = 18 – 2x2 Î x2 ≤ 9 ¾ x2 can be increased to 6, where x4 will drop to zero. ¾

x4 is called the leaving basic variable (since it becomes zero).

¾

These procedures are referred to as the minimum ratio test.

¾

Only consider the equations where the coefficient of the entering basic variable is strictly positive (>0).

9 Solving for the New BF Solution ¾

Initial BF solution: Nonbasic: x1 = x2 = 0, Basic: x3 = 4, x4 = 12, x5 = 18.

¾

New BF solution: Nonbasic: x1 = x4 = 0, Basic: x3 = ?, x2 = ?, x5 = ?

¾

We would like to convert the system of equations to a more convenient form for obtaining the variable values and conducting the optimality test.

¾

z

Objective function equation only contains the nonbasic variables.

z

Each functional constraint equation contains only one basic variable and its coefficient equals to 1.

The original algebra form =0 Z – 3x1 – 5x2 x1 + x3 =4 2x2 + x4 = 12 3x1 + 2x2 + x5 = 18

Jin Y. Wang

(0) (1) (2) (3)

Chap4-6


¾

¾


Fall, 2008

Perform the elementary algebraic operations (as learned in linear algebra) to produce the convenient form (called Gaussian elimination). z

Multiply (or divide) an equation by a nonzero constant.

z

Add (or subtract) a multiple of one equation to (or from) another equation.

Note that the coefficients of x2 are -5, 0, 2, and 2. We want these coefficients to become 0, 0, 1, and 0.

9 Optimality test (for the new BF solution) Z – 3x1 + 5/2x4 = 30 Î Z = 30 + 3x1 – 5/2x4 ¾

Do we need to continue?

9 Determine the direction of movement

9 Determine where to stop

Jin Y. Wang

Chap4-7



Fall, 2008

9 Solving for the new BF solution

9 Optimality Test ¾

Do we need to continue?

9 Optimal solution is

Simplex Method in Tabular Form 9 Logic is identical to that of the algebraic form. 9 Only records the essential information. It can significantly reduce the effort of calculations. Z – 3x1 – 5x2 =0 x1 + x3 =4 2x2 + x4 = 12 3x1 + 2x2 + x5 = 18 Basic Variable Z x3 x4 x5 9 Optimality test ¾

Jin Y. Wang

Z 1 0 0 0

Coefficient of: x2 x3 x1 0 -3 -5 1 0 1 0 2 0 3 2 0

Right Side x4 0 0 1 0

x5 0 0 0 1

0 4 12 18

If every coefficient in Z row (row 0) is nonnegative ( ≥ 0).

Chap4-8



Fall, 2008

9 Iteration ¾

Determining the entering basic variable by selecting the variable with the negative coefficient having the largest absolute value in row 0. Put a box around the column below this coefficient, and call this the pivot column. Coefficient of: Right Side Basic Variable x2 x3 x4 x5 x1 Z 1 0 0 0 0 Z -3 -5 x3 0 1 0 1 0 0 4 x4 0 0 2 0 1 0 12 x5 0 3 2 0 0 1 18

¾

Determining the leaving basic variable by applying the minimum ratio test. z

Pick out each coefficient in the pivot column that is strictly positive (>0).

z

Divide each of these coefficients into the right side entry for the same row.

z

Identify the row that has the smallest of these ratios.

z

Put a box around this row and call it the pivot row. Also, call the number that is in both boxes the pivot number.

Basic Variable Z x3 x4 x5

¾

Z 1 0 0 0

Coefficient of: x1 x2 x3 x4 0 -3 -5 0 1 0 1 0 0 2 0 1 3 2 0 0

x5 0 0 0 1

Right Side 0 4 12 18

Ratio 0 4 12/2=6 Î min 18/2=9

The basic variable for the pivot row is the leaving basic variable, so replace that variable by the entering basic variable in the basic variable column of the next simplex tableau.

9 Use elementary row operations to construct a new simplex tableau in proper form by using Gaussian elimination. ¾

Jin Y. Wang

Divide the pivot row by the pivot number.

Chap4-9



Fall, 2008

¾

Use elementary row operations to make other coefficients in pivot column to be zero. Coefficient of: Basic Right Ratio Variable Z x1 x2 x3 x4 x5 Side 1 0 5/2 0 30 Z -3 0 x3 0 1 0 1 0 0 4 4/1=4 x2 0 0 1 0 1/2 0 6 x5 0 3 0 0 -1 1 6 6/3=2 Basic Variable Z x3 x2 x1

Z 1 0 0 0

Coefficient of: Right Side x1 x2 x3 x4 x5 0 0 0 3/2 1 36 0 0 1 1/3 -1/3 2 0 1 0 1/2 0 6 1 0 0 -1/3 1/3 2

Ratio

Tie for the entering basic variable Z = 3x1 + 3x2 9 Selection may be made arbitrarily. Tie for the leaving variable—degeneracy Z – 3x1 – 5x2 = 0 x1 + x3 = 4 2x2 + x4 = 12 3x1 + 2x2 + x5 = 12 9 x2 is the entering variable, x4 and x5 reach zero simultaneously. 9 All the tied basic variables reach zero simultaneously. The one not chosen to be the leaving basic variable also will have a value of zero in the new BF solution. If x4 is chosen as the leaving basic variable + 5/2x4 = 30 Z – 3x1 x1 + x3 =4 x2 + 1/2x4 =6 3x1 – x4 + x5 = 0 So, the value of basic variable x5 is zero (called a degenerate basic variable). Jin Y. Wang

Chap4-10



Fall, 2008

9 If the degenerate basic variable retains its value of zero until it is chosen to be a leaving basic variable, the corresponding entering basic variable also must remain zero, so the value of Z must remain unchanged. 9 If x1 is the entering basic variable and x5 is the leaving basic variable, x1 will remain zero because any increment of x1 will result in a negative value of x5.

9 If Z may remain the same rather than increase at each iteration, the simplex method may then go around a loop. ¾

Rarely happen and ease to avoid.

No Leaving basic variable—unbounded Z x1 is the entering basic variable Z –3x1 + 5x5 = 30 -x1 + x3 = 4 -2x1 + x4 = 9 Multiple optimal solution 9 If there are multiple optimal solutions, two of them must be CPF. 9 How to tell and how to find the other optimal solutions? ¾

At least one of the nonbasic variables has a coefficient of zero in the final objective row. Increasing any such variable will not change the value of Z.

¾

Just perform one of iteration of simplex method will find it. Z – 0x3 + x5 x1 +

= 18 x3 3x3 + x4

x2 – 3/2x3

Jin Y. Wang

=4 – x5 = 6 + 1/2x5 = 3

Chap4-11


¾


Fall, 2008

Rewrite the algebraic form

9 All optimal solution can be written as linear combination of these two CPF. (x1, x2, x3, x4, x5 ) = w1(2, 6, 2, 0, 0) + w2(4, 3, 0, 6, 0) or (x1, x2) = w1(2, 6) + w2(4, 3) w1 + w2 = 1, w1, w2 ≥ 0 Not all models can be stated in standard form 9 It causes the difficulty of finding the initial BF solution. Equality constraints 9 Adopt 3x1 + 2x2 = 18 instead of 3x1 + 2x2 ≤ 18 in the Wyndor Problem. 9 Replace this equality by 3x1 + 2x2 ≤ 18 and 3x1 + 2x2 ≥ 18 is workable. But it is not a good idea since the increment of number of constraints. 9 After introducing slack variables, we have Max Z = 3x1 + 5x2 S.T. x1 + x3 =4 2x2 + x4 = 12 3x1 + 2x2 = 18 x1, x2, x3, x4 ≥ 0 9 It is not easy to get the initial feasible solution anymore. ¾

The origin point is not feasible.

9 Introduce an artificial variable x5 and construct the artificial problem Max Z = 3x1 + 5x2 S.T. x1 + x3 =4 2x2 + x4 = 12 3x1 + 2x2 + x5 = 18 x1, x2, x3, x4, x5 ≥ 0 9 Assign an overwhelming penalty to having x5 > 0 by changing the objective function Z = 3x1 + 5x2 to Z = 3x1 + 5x2 – Mx5 (M is a huge positive number, Big M method).

Jin Y. Wang

Chap4-12


¾


Fall, 2008

x5 needs to be zero to assure the feasibility. So, impose a huge penalty when it becomes non-zero.

Max Z = 3x1 + 5x2 – Mx5 S.T. x1 + x3 2x2 + x4 3x1 + 2x2 + x5

=4 = 12 = 18

x1, x2, x3, x4, x5 ≥ 0 9 Simplex method will try very hard not to have x5 > 0 in the optimal solution. 9 The initial basic variables are x3, x4, and x5. So, we have to rewrite the objective function in term of nonbasic variables.

Î Z = –18M + (3M+3)x1 + (2M+5)x2 9 Please finish the rest procedures (in tabular or algebraic form). See page 128 for details. 9 What does it mean by having at least one non-zero artificial variable when the simplex method stops?

Negative right hand side 9 Multiply both sides by –1.

Functional constraints in ≥ form 0.6x1 + 0.4x2 ≥ 6 9 Introduce a surplus variable x5 to make it be equality. Then, introduce an artificial variable x6 and treat it as before. 0.6x1 + 0.4x2 Æ0.6x1 + 0.4x2 – x5

≥ 6

= 6

Æ0.6x1 + 0.4x2 – x5 + x6 = 6 Minimization 9 Multiply the objective function by –1 Min Z = 0.4x1 + 0.5x2 Max –Z = – 0.4x1 – 0.5x2

Jin Y. Wang

Chap4-13



Fall, 2008

Example—Radiation Therapy Example Min Z = 0.4x1 + 0.5x2 S.T. 0.3x1 + 0.1x2 ≤ 2.7 0.5x1 + 0.5x2 = 6 0.6x1 + 0.4x2 ≥ 6 x1, x2 ≥ 0 9 Apply above procedures Max –Z = –0.4x1 – 0.5x2 – Mx4 – Mx6 S.T. 0.3x1 + 0.1x2 + x3 0.5x1 + 0.5x2

= 2.7 + x4

0.6x1 + 0.4x2

=6 – x5 + x6 = 6

x1, x2, x3, x4, x5, x6 ≥ 0 9 Rewrite the objective function in term of nonbasic variables and solve by the simplex method

Jin Y. Wang

Chap4-14



Fall, 2008

Recall the radiation therapy example 9 The big M method requires extra calculation after the initial BF solution (for the original problem) is found. 9 When the artificial variables are equal to 0, they remain zero for the rest simplex iterations. Summary of the Two-Phase Method 9 Initialization: Revise the constraints by introducing artificial variables. 9 Phase 1: The objective is to find a BF solution for the real problem. ¾

Min Z = sum of artificial variables, subject to revised constraints

9 Phase 2: The objective is to find an optimal solution for the real problem. 9 Phase 1 problem for radiation therapy example

Jin Y. Wang

Chap4-15



Fall, 2008

9 Phase 2 problem for radiation therapy example ¾

Drop the artificial variables and restore the original objective function.

Variables with a bound or with no bound 9 xj ≥ L Let xj’ = xj – L so xj’ ≥ 0 and can be treated as usual

9 xj ≤ U Let xj’ = U – xj so xj’ ≥ 0 and can be treated as usual

9 xj does not have a bound xj = xj+ – xj-

Jin Y. Wang

, xj+ , xj- ≥ 0

Chap4-16



Fall, 2008

Shadow Price—Find managerial decisions on resource allocations. 9 The shadow price for resource i (denoted by yi* ) measures the marginal value of this resource. 9 The rate which Z could be increased by (slightly) increasing the amount of this resource (bi) being made available. 9 Shadow price yi* equals to the coefficient (positive value) of the ith slack variable in the final objective row. 9 The final iteration of the Wyndor example Z

+3/2x4

+x5

= 36

x3 +1/3x4 –1/3x5 = 2 x2 x1

+1/2x4

=6

–1/3x4 +1/3x5 = 2

¾

The shadow prices are:

¾

If the unit cost for resource 2 is 1, do you want to go for it?

9 Notice that the increment of resources is not unlimited.

Jin Y. Wang

Chap4-17



Fall, 2008

9 Binding constraints are:

Sensitivity Analysis—Evaluate estimates of model parameters 9 Define the allowable range of parameters without changing the optimal solution (or optimal basis). 9 Sensitive parameters: The parameters that cannot be changed without changing the optimal solution. 9 As for bi:

9 As for ci:

Jin Y. Wang

Chap4-18



Fall, 2008

9 As for aij:

9 Sensitivity report provided by the Excel Solver

Jin Y. Wang

Chap4-19