CSC 445/545 Notes 1.6: How fast is the Simplex method?

Klee-Minty, n=2

Maximize
10 X1 + 1 X2 
subject to
     1 X1 + 0 X2  <=   1 
    20 X1 + 1 X2  <= 100 

X1 , X2 >= 0

The initial dictionary:
X3 =   1  - 1 X1 + 0 X2 
X4 =  100 -20 X1 - 1 X2 
-------------------------------------------------
z  =  -0  +10 X1 + 1 X2 

X1  enters.   X3  leaves.  z =  0 

After   1 pivot:
X1 =   1 + 0 X2 - 1 X3 
X4 =  80 - 1 X2 +20 X3 
-------------------------------------------------
z  =  10 + 1 X2 -10 X3 

X2  enters.   X4  leaves.  z = 10 

After   2 pivots:
X1 =   1 - 1 X3 + 0 X4 
X2 =  80 +20 X3 - 1 X4 
-------------------------------------------------
z  =  90 +10 X3 - 1 X4 

X3  enters.   X1  leaves.  z = 90 

After   3 pivots:
X3 =    1 - 1 x1 + 0 X4 
X2 =  100 -20 X1 - 1 X4 
-------------------------------------------------
z  =  100 -10 X1 - 1 X4 

The optimal solution: 100
X1  =  0  X2  = 100  X3  =  1  X4  =  0  

Klee-Minty, n=3


Maximize
100 X1 +10 X2 + 1 X3 

subject to
     1 X1 + 0 X2 + 0 X3  <=  1 
    20 X1 + 1 X2 + 0 X3  <= 100 
   200 X1 +20 X2 + 1 X3  <= 10000 

X1 , X2 , X3 >= 0

The initial dictionary:

X4 =      1 -  1 X1 + 0 X2 + 0 X3 
X5 =    100 - 20 X1 - 1 X2 + 0 X3 
X6 =  10000 -200 X1 -20 X2 - 1 X3 
-------------------------------------------------
z  =  -0 +100 X1 +10 X2 + 1 X3 

X1  enters.   X4  leaves.  z = -0 

After   1 pivot:
X1 =     1 +  0 X2 + 0 X3 -  1 X4 
X5 =    80 -  1 X2 + 0 X3 + 20 X4 
X6 =  9800 - 20 X2 - 1 X3 +200 X4 
-------------------------------------------------
z  =  100  +10 X2  + 1 X3 -100 X4 

X2  enters.   X5  leaves.  z = 100 

After   2 pivots:
X1 =     1 + 0 X3 -  1 X4 + 0 X5 
X2 =    80 + 0 X3 + 20 X4 - 1 X5 
X6 =  8200 - 1 X3 -200 X4 +20 X5 
-------------------------------------------------
z  =  900 +  1 X3 +100 X4 -10 X5 

X4  enters.   X1  leaves.  z = 900 

After   3 pivots:
X4 =     1 -  1 X1 + 0 X3 + 0 X5 
X2 =   100 - 20 X1 + 0 X3 - 1 X5 
X6 =  8000 +200 X1 - 1 X3 +20 X5 
-------------------------------------------------
z  =  1000 -100 X1 + 1 X3 -10 X5 

X3  enters.   X6  leaves.  z = 1000 

After   4 pivots:
X4 =     1 -  1 X1 + 0 X5 + 0 X6 
X2 =   100 - 20 X1 - 1 X5 + 0 X6 
X3 =  8000 +200 X1 +20 X5 - 1 X6 
-------------------------------------------------
z  =  9000 +100 X1 +10 X5 - 1 X6 

X1  enters.   X4  leaves.  z = 9000 

After   5 pivots:
X1 =     1 -  1 X4 + 0 X5 + 0 X6 
X2 =    80 + 20 X4 - 1 X5 + 0 X6 
X3 =  8200 -200 X4 +20 X5 - 1 X6 
-------------------------------------------------
z  =  9100 -100 X4 +10 X5 - 1 X6 

X5  enters.   X2  leaves.  z = 9100 

After   6 pivots:
X1 =     1 + 0 X2 -  1 X4 + 0 X6 
X5 =    80 - 1 X2 + 20 X4 + 0 X6 
X3 =  9800 -20 X2 +200 X4 - 1 X6 
-------------------------------------------------
z  =  9900 -10 X2 +100 X4 - 1 X6 

X4  enters.   X1  leaves.  z = 9900 

After   7 pivots:
X4 =      1 -  1 X1 + 0 X2 + 0 X6 
X5 =    100 - 20 X1 - 1 X2 + 0 X6 
X3 =  10000 -200 X1 -20 X2 - 1 X6 
-------------------------------------------------
z  =  10000 -100 X1 -10 X2 - 1 X6 

The optimal solution: 10000
X1  =  0  X2  =  0  X3  = 10000  X4  =  1  
X5  = 100  X6  =  0  

Klee-Minty, n=4

Maximize
1000 X1 +100 X2 +10 X3 + 1 X4 
subject to
       1 X1 +  0 X2 + 0 X3 + 0 X4  <= 1 
      20 X1 +  1 X2 + 0 X3 + 0 X4  <= 100 
     200 X1 + 20 X2 + 1 X3 + 0 X4  <= 10000 
    2000 X1 +200 X2 +20 X3 + 1 X4  <= 1000000 

X1 , X2 , X3 , X4 >= 0


After  15 pivots:
X5 =        1   - 1 X1  + 0 X2 + 0 X3 + 0 X8 
X6 =      100   -20 X1 -  1 X2 + 0 X3 + 0 X8 
X7 =    10000  -200 X1 - 20 X2 - 1 X3 + 0 X8 
X4 =  1000000 -2000 X1 -200 X2 -20 X3 - 1 X8 
-------------------------------------------------
z  =  1000000 -1000 X1 -100 X2 -10 X3 - 1 X8 

The optimal solution: 1000000
X1  =  0  X2  =  0  X3  =  0  X4  = 1000000  
X5  =  1  X6  = 100  X7  = 10000  X8  =  0  


The General Problem:

Maximize   
10n-1 x1 + 10n-2 x2 + 10n-3 x3 + ... + 10n-n xn 

subject to

       X1                                  <= 1
    20 X1 +     X2                         <= 100
   200 X1 +  20 X2  +    X3                <= 10000
  2000 X1 + 200 X2  + 20 X3  +    X4       <= 1000000
 20000 X1 +2000 X2  +200 X3  + 20 X4 + X5  <= 100000000
.
.
.


X1, X2, X3, ..., Xn >= 0
Theorem [Klee-Minty, 1972]

The Klee-Minty examples take 2n - 1 iterations when the variable to enter is chosen using the Maximum Coefficient rule.

Proof: Problems 4.2 and 4.3.

Similar examples exist for Largest Increase rule [Jeroslow, 1973].


So why is the Simplex Method useful?

In practice, it usually takes less than 3m/2 iterations, and only rarely 3m, for m < 50, and m+n < 200 [Dantzig, 1963].

Monte Carlo studies of larger random problems- similar results, see table in text [Kuhn and Quandt, 1963]

The Largest Increase rule may require fewer iterations but it requires more work per iteration. Thus, the Maximum Coefficient rule may be faster.


Application of Maximum Increase Rule:
Dictionary:

Xi  =   Bi  -  s * Xj  + ....

If Xj enters and Xi leaves:

s Xj=   Bi    - Xi + ...

So 
 
  Xj =  Bi/s  - Xi + ...

Looking at the z row:

z= z '   +    r Xj

Replacing Xj in the z row:

z= z '   +    r (Bi / s ) - ...

So the change to z is (r/s) * Bi
where r is the coefficient of the 
        entering variable in the z row

Bi is the constant term in the pivot row

-s is the coefficient of Xj in the 
      pivot row of the dictionary
Largest Increase Rule: Choose the entering variable to maximize this quantity.