<< . .

( 16)

. . >>

a step-function, is discussed. Before the problem is defined, it is necessary
to cover some concepts and transformations in Section 2.2 and Section 2.3, to
explain the problems and algorithms that will be introduced in later sections
and chapters. As part of the computer software package SCOM (that was con-
structed by Craven and Islam), “nqq” is used as a DE solver in the algorithms
in this research. This program is described in Section 2.4. Then a simplified
control problem is introduced in Section 2.5. The computational algorithms ,
which are used to solve this simplified control problem, are indicated in Sec-
tion 2.6. Some problems with different fitting functions will be discussed later
in Chapter 5. Lastly, graphs and tables in Section 2.7 represent all the comput-
ing results of this problem. The analysis of the results is discussed in Section

2. Piecewise-linear Transformation
The idea of piecewise-linear transformation of the time variable was first in-
troduced by Teo [40, 1991], but the time intervals were mapped into
instead of where is the number of time intervals.
In Lee, Teo, Rehbock and Jennings [51, 1997], the time transformation is de-
scribed by where is a piece-wise constant transformation. In
this book, a similar idea is used, but the implementation is a little simpler, not
requiring another differential equation. A non-linear transformation of the time
scale given by Craven [15, 1998], is introduced in Section 2.3. The transfor-

mations in this chapter are the essential part of the algorithms in the research,
and will be directly applied in Section 2.6.
Consider a financial optimal control model, to minimize an integral:

subject to:

The time interval [0,1] is divided as follows:

so there are subintervals:

A scaled time is constructed here to replace the real time in computation;
will be used later in the “nqq” package (differential equation solver).
The scaled time takes values:

where thus the time intervals [0, 1] is divided into equal intervals.
The relationship between and can be expressed by the following formula:

Thus the subdivision point maps to point
Now the control takes values:
The STV Approach to Financial Optimal Control Models 23

in successive subintervals:

transforms to:
Then the dynamic equation

The objective function transforms to the sum of integrals:

3. Non-linear Time Scale Transformation
In some time-optimal control problems or optimal control problems that are
over a long time interval, the large number of subintervals of the time scale will
cause computational difficulty. The optimal control functions lead to stable
values monotonically when time becomes large. So it would be useful if
a suitable non-linear transformation of time scale is practical, (see reference
[15, 1998]). With a suitable non-linear time scale, a few subintervals may be
enough to get the same accuracy of large subintervals.
Goh and Teo [33, 1987] introduced a change of time scale for a time-optimal
control problem while the terminal time T is a variable. Let where
is the new time variable mapping in [0,1]. is again written for then
problem becomes a fixed-time optimal control problem with time interval [0,1]
and parameter T. This allows T to be computed accurately without being
interpolated between subdivision points.
This makes the original financial optimization problem as the following:

subject to:

Transform into:

subject to:

This transformation will be used later in the computational methods 4.1 - 4.5
for the financial model in Chapter 4.
Now consider, an objective function with a discount factor where is
positive, thus:

Define time as:

The objective function becomes:
The STV Approach to Financial Optimal Control Models 25

and the differential equation changes from to:

For T is fixed and large, the denominator satisfies If
T is a variable to be optimized over, then replaces the variable T, in an
For example, in a fixed-time optimal control problem, if there is some “dis-
tinguished time” then an additional constraint must
be satisfied. A transformation of to maps to thus to a
subdivision point. In particular:

At the piecewise-linear transformation is not differentiable, and the
co-state may be discontinuous.

4. A Computer Software Package Used in this Study
The computer package SCOM ([18, 2001]) is a tool to solve step-function
optimal control problems on MATLAB 5.2, on a Macintosh computer. It is
noted that a program written and compiled in C language will run faster than a
MATLAB program for the same computation. However MATLAB is a matrix
computation language, so it requires much less programming work in calcu-
lating matrix operations on MATLAB than other computer languages, such as
C language and Fortran, etc. The program “constr” is a constrained optimiza-
tion package in MATLAB™s Toolbox, based on a SQP (sequential quadratic
programming) method. “constr” will use gradients if supplied; otherwise it
will estimate gradients by finite difference. In non-linear programming meth-
ods, the SQP method is very successful. The method closely mimics Newton™s
method for constrained optimization, just as it is done for unconstrained opti-
mization. Using a quasi-Newton updating method , an approximation is made
of the Hessian of the Lagrangian function at each major iteration. It is then used
to generate a Quadratic Programming sub-problem whose solution is used to
form a search direction for a line search procedure. In the research, we build
another subroutine to be called as the objective function in “constr”.

The state functions in this research were solved by the differential equation
solver “nqq” in the SCOM package (see details in Appendix C). When results
of the state come out on the grid-points, the objective function becomes
a function of N variables that can also be calculated by
When the subroutines for the state functions and objective function are con-
structed, the MATLAB program “constr” is then used to obtain the optimal
solution of the problem with respect to optimal switching times. In the pro-
gram, the control is approximated by a step-function.

5. An Optimal Control Problem When the Control can
only Take the Value 0 or 1
From the point of view of control theory, the bang-bang optimal control
happens when the systems of the optimal control problems are linear in control.
The Nerlove-Arrow model [63, 1962] is an example of a bang-bang control
following a singular arc control. Now, introduce a typical bang-bang optimal
control problem.
We consider an optimal control problem:

subject to:

In this case, the state function is assumed piece-wise smooth, and the
control jumps many times between 0 and 1 during the time interval [0, T].
Here, simplify the problem to T = 1. Now use a step-function:

to approximate the control Here the target function is: Let

Observe that if having instead then
would be optimal, with But if must be either 0 or 1, then
The STV Approach to Financial Optimal Control Models 27

will jump many times between 0 and 1. Suppose a further restriction is
made that, for some given integer time interval [0, 1] is divided into equal
takes a constant value (1 or 0) on each subinterval
So if takes the values in this pattern 1, 0, 1, 0, ¦, 1, 0
on the successive subinterval, then each subintervals contributes to
as But the optimal value J = 0 is not
reached, unless an infinite number of jumps are allowed.
Now a term is added to the objective function. (2.7) is modified as

Here, K is the cost of changing control and is the number of time intervals.
The objective function of the optimal control problem becomes a cost function.
The algorithms in the next section are used for solving these kind of optimal
control problems.

6. Approaches to Bang-Bang Optimal Control with a Cost
of Changing Control
In this section, the computational methods for solving bang-bang optimal
control problems with a cost of switching control are introduced. Simply mod-
ifying certain parts of these methods can satisfy a class of similar problems.
Suppose the number N of switching times is fixed by N = nn, say switch-
ing times Now consider (2.7) as a function of the switching
times, say Then the minimum of this function
is computed, starting from a small value of N. Here the components of the
vector are the lengths between each switching
time. The vector um must satisfy the upper and lower bounds 0 and 1, thus
and a constraint must be satisfied. The
vector represents the values of the state func-
tion takes at each switching time. The algorithms are as follows:

Algorithm 2.1 Main Program (see project1_1.m in Appendix A.1)
Step 1. Initialization. Set the maximum number of function evaluations, par,
which is the system parameter of the MATLAB “constr” function, and
also another system parameter par(13) = 1 (1 represents the number
of the equation constraint in the minimization problem), and a vector of
the parameters which are used in the whole subroutines, par = [number
of the state components, number of control components, nn= number of
total subintervals], arbitrary starting lengths of switching time intervals

and vector ul = lower bound of um,
vector uu = upper bound of um, thus Initialize the value
of the state function xinit.
Step 2. Call the MATLAB “constr” function. In turn, “constr” calls the “Mini-
mizing Program” to calculate the minimization of the calling program with
respect to the optimal vector um.
Step 3. Input the optimal result um into “Minimizing Program” again to ob-
tain the results of the objective function (the last value of the calcula-
tion) and the state vector xm, corresponding to the optimal um.
Step 4. Attach a cost K to nn. Add to the objective function (2.7) and
Step 5. Set a bigger nn; then go back to Step 2; EXIT when the result of cost
function J stops decreasing.
Algorithm 2.2: Minimizing Program (see project1_2.m in Appendix A.1)
Step 1. Initialization. Input vectors um, par and initial state xinit. Set the
initial state xm(1) = xinit, and nx = par(1), the number of the state
components, nu = par(2), the number of the control components; initial-
ize scaled time subinterval counter it = 1, hs = 1/nn(length of
each equal subinterval). Choose the “Input function for dynamic equation”
as the right side of equation (2.9) to calculate the differential equation, input
Step 2. Call SCOM package function “nqq” with the stated “Input function
for dynamic equation” to solve the differential equation (2.9) of the state
function Tabulate the solutions as the components of the vector

Step 3. Set the initial state zz = 0. Set initial scaled time subinterval
counter it = 1 again, choose the “input function for integration calcula-
tion” for SCOM function “nqq” for solving the integration of the objective
function (2.7), input vector xm and um.
Step 4. Call SCOM function “nqq” with the stated “Input function for integra-
tion calculation” to calculate:

The STV Approach to Financial Optimal Control Models 29


by solving the differential equation:


Tabulate the results in as the components of vector

Step 5. Take the last result of the vector jm as the value of the objective func-
tion, and calculate the constraint function of “Minimizing program”, which
Algorithm 2.3: Input function for dynamic equation (see project1_3.m in
Appendix A.1)
Step 1. Initialization. Input scaled time subinterval counter it, the length of
subintervals hs, and vector um, and set the number of subintervals nn =
Step 2. Set the control policy as vector with al-
ternating values 1 and 0 (as in 2.10) in successive subintervals.
Step 3. Construct the right side of the differential equation for the state func-
tion using the piecewise-linear transformation in (2.5).
Algorithm 2.4: Input function for integration calculation (see project1_4.m
in Appendix A.1)
Step 1. Initialization. Input scaled time subinterval counter it, the length
of subintervals hs, vector xm representing the values of the state function
at each switching time, and also a new initial state for integral, and the
vector um. Set the number of subintervals nn = 1/hs.
Step 2. Use the linear interpolation to get an estimate “xmt” of the state, in a
time between grid-points where
Step 3. Add up components of the lengths of the switching time intervals in
um to obtain the switching times in (2.4).
Step 4. Construct the right side of the equation (2.4) to obtain the time variable

Step 5. Calculate the integral in (2.6) at scaled time
When the number of switching time N increases, the cost function decreases
because of the better approximation. While calculating a minimization prob-
lem the term KN increases with N increasing, and decreases
with bigger N. It can be found that the cost of changing control is very criti-
cal in the cost function. A proper chosen cost K will efficiently lead the cost
function to the minimum. The analysis of the cost will be discussed in later

7. An Investment Planning Model and Results
In this section, the computational results reported by a set of graphs will
be introduced to verify the algorithms developed in Section 2.6. First we will
introduce the fitting function. In this example, the fitting function is set to be
The state function is used to approximate this fitting function. The
formulae for this financial optimal control model is shown as follows:

subject to:

where x(t) = stock price, u(t) = the proportion of total investment in stocks
compared to other forms of financial investment.
Although the above model is an illustrative model, it can, however, represent
an interesting financial decision making problem. The state equation represents
the dynamics of the price of a stock. It is assumed that the change in the
price of a stock is determined by the proportion of allocation of total funds for
purchasing a stock. The objective of this control problem is to determine the
value of u(t) (which only takes the value of 1 or 0) which can optimize the
objective function so as to minimize the deviation of the state variable from
its target value specified. Therefore this model (2.11 to 2.15) is an investment
planning model with some sub-utilization criterion included in the model.
The target function is First set (n is the number of time
intervals), and control takes as 1,0 in time intervals The model
2.11 to 2.15 was solved with these parameter values and the results are shown
The STV Approach to Financial Optimal Control Models 31

in Appendix B.1 and are represented by Figure 2.1. In Figure 2.1, it is shown
that during the planning horizon [0,1], the control only jumps once at time
The state vector xm takes the value [0.127, 0.127] at the grid-points
of two subintervals [0, 0.127] and [0.127, 1]. The approximation between the
state function xm and the fitting function is not very good when the
number of the switching times is quite small is only 2). “*“” represents
the state function and “.” represents the fitting function in the
following graph.

Figure 2.1. Plot of n=2, forcing function ut=1, 0

Then increase to 4. During the whole time [0,1], the control jumps three
times. A better approximation is shown in Figure 2.2 with more jumps of the

Figure 2.2. Plot of n=4, forcing function ut=1,0,1,0

Set and run the program. A much better approximation than

<< . .

( 16)

. . >>

Copyright Design by: Sunlight webdesign