CSC 445/545 Notes 5.1: The Revised Simplex Method

Note: A standard convention in matrix algebra is that a vector x is by default a column vector and x^T is a row vector. The text does not follow this standard convention, but I am using it. Hence, the notation here may differ a bit from the text.

x= [X1]
   [X2]
   [X3]
     .
     .
     .
   [Xn]


X^T= [X1 X2 X3 ... Xn]

Notes 5.1, Page 2.

In the Simplex Method, what information is needed to find the next basic feasible solution?

Coefficients of non-basic variables in the z row to determine the pivot column.
The pivot column and the current solution to determine the pivot row.

The new solution is then determined from the pivot column and the old solution.

X4 =   5 - 2 X1 - 3 X2 - 1 X3 
X5 =  11 - 4 X1 - 1 X2 - 2 X3 
X6 =   8 - 3 X1 - 4 X2 - 2 X3 
-------------------------------------------------
z  =   0 + 5 X1 + 4 X2 + 3 X3 

X1  enters.   X4  leaves.  

X1 =   2.50 - 1.50 X2 - 0.50 X3 - 0.50 X4 
X5 =   1    + 5 X2    + 0 X3    + 2 X4 
X6 =   0.50 + 0.50 X2 - 0.50 X3 + 1.50 X4 
-------------------------------------------------
z  =  12.50 - 3.50 X2 + 0.50 X3 - 2.50 X4

Notes 5.1, Page 3.

For this problem we have:

Maximize c^T x

c^T = [ 5     4     3     0     0     0]

x= [X1 X2 X3 X4 X5 X6]^T

subject to A x = b

A = [  2     3     1     1     0     0 ]
    [  4     1     2     0     1     0 ]
    [  3     4     2     0     0     1 ]

b = [ 5]
    [11]
    [ 8]

Initially, the basis corresponds to the slack variables X4, X5, X6:

HB^T= [ 4     5     6 ]

The non-basic variables are X1, X2, and X3:

HN^T= [1     2     3]

Notes 5.1, Page 4.

Split the matrix A into two parts:

AB = columns corresponding to HB^T
AN = columns corresponding to HN^T

HB^T =[4     5     6 ]

AB =[1     0     0]
    [0     1     0]
    [0     0     1]

HN^T =[1     2     3]

AN =[2     3     1]
    [4     1     2]
    [3     4     2]

Create CB, CN, and XB, XN the same way:

CB = [0     0     0]^T

CN=  [5     4     3]^T

XB= [X4 X5 X6]^T
XN= [X1 X2 X3]^T

Notes 5.1, Page 5.

The system of equations can then be expressed as:

(a) A x = b   =>    AB * XB + AN *  XN = b

(b) z= CB^T * XB + CN^T * XN

From (a), XB = AB ^-1 * b - AB ^-1 * AN *  XN 

             = AB ^-1 * [ b - AN *  XN ]

Plugging this into the formula for z:


z= CB^T * AB ^-1 * [ b - AN *  XN ] + CN^T * XN

So the dictionary with basis AB reads:


   XB  = AB ^-1 * [ b - AN *  XN ]
  -------------------------------------------
z= CB^T * AB ^-1 * [ b - AN *  XN ] + CN^T * XN

Notes 5.1, Page 6.

The dictionary:

   XB  = AB ^-1 [ b - AN *  XN ]
  -------------------------------------------
z= CB^T * AB ^-1 * [ b - AN *  XN ] + CN^T * XN

============================================================= 

[X4]   [1 0 0 ] -1     [ 5]   [ 2 3 1] [X1]
[X5] = [0 1 0 ]     *( [11] - [ 4 1 2] [X2] )
[X6]   [0 0 1 ]        [ 8]   [ 3 4 2] [X3]

=============================================================

X4 =   5 - 2 X1 - 3 X2 - 1 X3 
X5 =  11 - 4 X1 - 1 X2 - 2 X3 
X6 =   8 - 3 X1 - 4 X2 - 2 X3 
------------------------------
z  =   0 + 5 X1 + 4 X2 + 3 X3 

=============================================================

   [0 0 0]*[1 0 0]-1   [ 5]   [2 3 1] [X1]
z=         [0 1 0]  * ([11] - [4 1 2] [X2])
           [0 0 1]     [ 8]   [3 4 2] [X3]

 + [5  4  3] * [X1]
               [X2]
               [X3]

Notes 5.1, Page 7.
Step 1: Find coefficients of non-basic variables in the z row.

z=  CB^T * AB ^-1 [ b - AN *  XN ] + CN^T * XN

Let y^T= CB^T * AB^-1

Find y^T by solving   y^T AB = CB^T
or equivalently AB^T y = CB.

Set z=  CB^T * AB ^-1  b + 
        [CN^T - CB^T * AB^-1 * AN ] * XN

     =  y^T  b + [CN^T - y^T AN ] * XN
=======================================================================
[Y1 Y2 Y3] [1 0 0]=  [0 0 0]
           [0 1 0]          
           [0 0 1]      Y1=0 Y2=0, Y3=0
 
  [0 0 0][ 5]             [0 0 0][2 3 1] [X1]
z=       [11] + ([5 4 3]-        [4 1 2])[X2] 
         [ 8]                    [3 4 2] [X3]

z  =   0 + 5 X1 + 4 X2 + 3 X3

The y^T b gives the constant term in the z row and the rest gives the terms corresponding to the non-basic variables.

Notes 5.1, Page 8.

Step 2: Determine the leaving variable.

Let t be the change to the entering variable.

Increase t until the value of some variable, the leaving variable is zero.

How do the values of the basic variables change with t?

XB  = AB ^-1 [ b - AN *  XN ]

XB  = XB' - AB ^-1 AN *  XN 

XB''  = XB' - t * d

where d is the column of AB ^-1 AN that corresponds to the entering variable.

But d= AB^-1 a  
where a is the entering column 
taken from the initial problem.

To find d, solve AB d = a.

Notes 5.1, Page 9.

Solving AB d = a:

Recall:  z  =   0 + 5 X1 + 4 X2 + 3 X3

Choose X1 to enter. Which variable leaves?

 
AB =[1     0     0]
    [0     1     0]
    [0     0     1]

a=  [2]    Column for entering variable in
    [4]    original tableau.
    [3]

The solution is d=[2]
                  [4]
                  [3]

Notes 5.1, Page 10.
Which equation imposes the tightest constraint?

b=  [ 5]       5 - 2X1 >= 0    X1 <=  5/2 (*)
    [11]      11 - 4X1 >= 0    X1 <= 11/4  
    [ 8]       8 - 3X1 >= 0    X1 <=  8/3

The first basis variable X4 leaves because the first equation is the tightest. The value of the entering variable is 5/2 because this is the tightest constraint.

Updating the rest (plug in the value of the entering variable):

X4=  5 - 2(5/2)  = 0
X5= 11 - 4(5/2)  = 1
X6=  8 - 3(5/2)  = 1/2

The new solution is:
(don't forget to include the value of the entering variable in place of the value of the leaving variable)

X1=  5/2 
X5=   1
X6=  1/2

Notes 5.1, Page 11.

Step 3: Updating all the variables:

HB^T= [1 5 6]
HN^T= [4 2 3]

AB =[2     0     0]
    [4     1     0]
    [3     0     1]

AN =[1     3     1]
    [0     1     2]
    [0     4     2]

Create CB, CN, and XB, XN the same way:

CB= [ 5  0  0]^T    CN= [ 0  4  3]^T

XB= [X1 X5 X6]^T   XN= [X4 X2 X3]^T

The current solution is: [5/2  1  1/2]^T

The objective function value Z is:
(previous value of Z) 
+ (coeff. of entering var. in Z row) 
* (new value of entering variable)

= 0  + 5 * 5/2 = 25/2

Notes 5.1, Page 12.

The Revised Simplex Method uses more work to determine the

z row coefficients,
the column that should enter, and
the pivot row

but then it takes less work to pivot.

Notes 5.1, Page 13.

Summary of the steps of the Revised Simplex Method:

Maintain for each step:

HB- the current basis header.
HN- the header for the non-basic variables (optional).
b- the current solution.
AB- the columns from A which correspond to the basis in the same order as in HB. [Actually, most programs maintain some factorization of this matrix].
AN- the columns from A which correspond to the basis in the same order as in HN. [Actually, you can get these columns from A when you need them so you don't really have to store this.]
CB- the costs of the basic variables in the same order as HB.
CN- the costs of the non-basic variables in the same order as HN.
z- the current value of the objective function.

Notes 5.1, Page 14.

The Revised Simplex Algorithm

Step 1: Determine pivot column.

Solve AB^T y = CB for y.

Compute [CN^T - y^T AN] * XN
to get coefficients of non-basic variables.
Look for a positive coefficient, say r 
corresponding to non-basic Xj.

Step 2: Find pivot row.

Solve AB d = a 
where a is column in A for entering variable Xj.

To find leaving variable, find the tightest constraint.

For i= 1, 2, ..., m,
      bi - di * Xj  >= 0   

Say Xj <= s is the tightest constraint.

Notes 5.1, Page 15.

Step 3: Update variables (pivot).

Update basis header HB by replacing entering variable by leaving one. Update HN by replacing leaving variable by entering one.
The new value of z = z + r * s.
Set Xj = s in the new solution. Plug this value for Xj into the other equations to update the values of the other variables: Xi= bi - di * s.
In AB, replace the column for the leaving variable with the column in A that corresponds to the entering variable.
In AN, replace the column for the entering variable with the column in A that corresponds to the leaving variable.
Update CB, CN so they match the basis headers (take from original costs).

Notes 5.1, Page 16.

We completed the first iteration on the example. The result was:

Iteration #1:

HB^T= [1 5 6]
HN^T= [4 2 3]

z= 25/2

The current solution is: b= [5/2  1  1/2]^T

AB =[2     0     0]
    [4     1     0]
    [3     0     1]

AN =[1     3     1]
    [0     1     2]
    [0     4     2]

CB= [5     0     0]^T
CN= [0     4     3]^T

XB= [X1 X5 X6]^T
XN= [X4 X2 X3]^T

Notes 5.1, Page 17. Iteration #2:
Step 1: Determine pivot column.

Solve AB^T y = CB for y.

[2     4     3] [Y1]     [5]
[0     1     0] [Y2]  =  [0]
[0     0     1] [Y3]     [0]

Y1= 2.5000, Y2=0, Y3=0

Compute [CN^T - y^T AN] * XN to get coefficients of non-basic variables. Look for a positive coefficient, say r corresponding to non-basic Xj.

[0     4     3]  - [ 2.5 0 0] [1     3     1]
                              [0     1     2]
                              [0     4     2]

=  [ -2.5000   -3.5000    0.5000]

The third variable should enter. Looking at XN, this is X3:

XN= [X4 X2 X3]^T

r = 1/2

Notes 5.1, Page 18.

Step 2: Find pivot row.

Solve AB d = a 
where a is column in A for entering variable Xj.

[2     0     0] [d1]     [1]
[4     1     0] [d2]  =  [2]
[3     0     1] [d3]     [2]

d = [0.5]
    [0  ]
    [0.5]

Find the tightest constraint:

b= [5/2  1  1/2]^T

X1 <=  5/2  - 0.5 * X3   X3 <= 5
X3 <=   1   - 0   * X3   NO CONSTRAINT
X3 <=  1/2  - 0.5 * X3   X3 <= 1     (*) s=1

So the third basis variable exits.

HB^T= [1 5 6],    X6 exits.

Notes 5.1, Page 19.

Step 3: Update variables (pivot).

HB^T= [1 5 3]     HN^T= [4 2 6]

z= 12.5 + r *s = 12.5 + 0.5 * 1= 13

X1 <=  5/2  - 0.5 * 1    = 2
X5 <=   1   - 0   * 1    = 1
X6 <=  1/2  - 0.5 * 1    = 0

Note also X3= s = 1.

b= [2  1  1]^T

AB =[2     0     1]
    [4     1     2]
    [3     0     2]

AN =[1     3     0]
    [0     1     0]
    [0     4     1]

CB= [5     0     3]^T
CN= [0     4     0]^T

XB= [X1 X5 X3]^T
XN= [X4 X2 X6]^T

Notes 5.1, Page 20.

Iteration 3:
Step 1: Determine pivot column.

Solve AB^T y = CB for y.

[2     4     3] [Y1]     [5]
[0     1     0] [Y2]  =  [0]
[1     2     2] [Y3]     [3]

Y1= 1,  Y2=0, Y3=1

Compute [CN^T - y^T AN] * XN to get coefficients of non-basic variables. Look for a positive coefficient, say r corresponding to non-basic Xj.

[0     4     0]  - [1 0 1] [1     3     0]
                           [0     1     0]
                           [0     4     1]

= [ -1    -3    -1]

All the coefficients are negative so the current solution is OPTIMAL.