LINEBURG

to cover some concepts and transformations in Section 2.2 and Section 2.3, to

explain the problems and algorithms that will be introduced in later sections

and chapters. As part of the computer software package SCOM (that was con-

structed by Craven and Islam), “nqq” is used as a DE solver in the algorithms

in this research. This program is described in Section 2.4. Then a simplified

control problem is introduced in Section 2.5. The computational algorithms ,

which are used to solve this simplified control problem, are indicated in Sec-

tion 2.6. Some problems with different fitting functions will be discussed later

in Chapter 5. Lastly, graphs and tables in Section 2.7 represent all the comput-

ing results of this problem. The analysis of the results is discussed in Section

2.8

2. Piecewise-linear Transformation

The idea of piecewise-linear transformation of the time variable was first in-

troduced by Teo [40, 1991], but the time intervals were mapped into

instead of where is the number of time intervals.

In Lee, Teo, Rehbock and Jennings [51, 1997], the time transformation is de-

scribed by where is a piece-wise constant transformation. In

this book, a similar idea is used, but the implementation is a little simpler, not

requiring another differential equation. A non-linear transformation of the time

scale given by Craven [15, 1998], is introduced in Section 2.3. The transfor-

22 OPTIMAL CONTROL MODELS IN FINANCE

mations in this chapter are the essential part of the algorithms in the research,

and will be directly applied in Section 2.6.

Consider a financial optimal control model, to minimize an integral:

subject to:

The time interval [0,1] is divided as follows:

so there are subintervals:

A scaled time is constructed here to replace the real time in computation;

will be used later in the “nqq” package (differential equation solver).

The scaled time takes values:

where thus the time intervals [0, 1] is divided into equal intervals.

The relationship between and can be expressed by the following formula:

where

Thus the subdivision point maps to point

Now the control takes values:

The STV Approach to Financial Optimal Control Models 23

in successive subintervals:

Define

transforms to:

Then the dynamic equation

The objective function transforms to the sum of integrals:

3. Non-linear Time Scale Transformation

In some time-optimal control problems or optimal control problems that are

over a long time interval, the large number of subintervals of the time scale will

cause computational difficulty. The optimal control functions lead to stable

values monotonically when time becomes large. So it would be useful if

a suitable non-linear transformation of time scale is practical, (see reference

[15, 1998]). With a suitable non-linear time scale, a few subintervals may be

enough to get the same accuracy of large subintervals.

Goh and Teo [33, 1987] introduced a change of time scale for a time-optimal

control problem while the terminal time T is a variable. Let where

is the new time variable mapping in [0,1]. is again written for then

problem becomes a fixed-time optimal control problem with time interval [0,1]

and parameter T. This allows T to be computed accurately without being

interpolated between subdivision points.

This makes the original financial optimization problem as the following:

24 OPTIMAL CONTROL MODELS IN FINANCE

subject to:

Transform into:

subject to:

This transformation will be used later in the computational methods 4.1 - 4.5

for the financial model in Chapter 4.

Now consider, an objective function with a discount factor where is

positive, thus:

Define time as:

The objective function becomes:

The STV Approach to Financial Optimal Control Models 25

and the differential equation changes from to:

For T is fixed and large, the denominator satisfies If

T is a variable to be optimized over, then replaces the variable T, in an

interval

For example, in a fixed-time optimal control problem, if there is some “dis-

tinguished time” then an additional constraint must

be satisfied. A transformation of to maps to thus to a

subdivision point. In particular:

At the piecewise-linear transformation is not differentiable, and the

co-state may be discontinuous.

4. A Computer Software Package Used in this Study

The computer package SCOM ([18, 2001]) is a tool to solve step-function

optimal control problems on MATLAB 5.2, on a Macintosh computer. It is

noted that a program written and compiled in C language will run faster than a

MATLAB program for the same computation. However MATLAB is a matrix

computation language, so it requires much less programming work in calcu-

lating matrix operations on MATLAB than other computer languages, such as

C language and Fortran, etc. The program “constr” is a constrained optimiza-

tion package in MATLAB™s Toolbox, based on a SQP (sequential quadratic

programming) method. “constr” will use gradients if supplied; otherwise it

will estimate gradients by finite difference. In non-linear programming meth-

ods, the SQP method is very successful. The method closely mimics Newton™s

method for constrained optimization, just as it is done for unconstrained opti-

mization. Using a quasi-Newton updating method , an approximation is made

of the Hessian of the Lagrangian function at each major iteration. It is then used

to generate a Quadratic Programming sub-problem whose solution is used to

form a search direction for a line search procedure. In the research, we build

another subroutine to be called as the objective function in “constr”.

26 OPTIMAL CONTROL MODELS IN FINANCE

The state functions in this research were solved by the differential equation

solver “nqq” in the SCOM package (see details in Appendix C). When results

of the state come out on the grid-points, the objective function becomes

a function of N variables that can also be calculated by

“nqq”.

When the subroutines for the state functions and objective function are con-

structed, the MATLAB program “constr” is then used to obtain the optimal

solution of the problem with respect to optimal switching times. In the pro-

gram, the control is approximated by a step-function.

5. An Optimal Control Problem When the Control can

only Take the Value 0 or 1

From the point of view of control theory, the bang-bang optimal control

happens when the systems of the optimal control problems are linear in control.

The Nerlove-Arrow model [63, 1962] is an example of a bang-bang control

following a singular arc control. Now, introduce a typical bang-bang optimal

control problem.

We consider an optimal control problem:

subject to:

In this case, the state function is assumed piece-wise smooth, and the

control jumps many times between 0 and 1 during the time interval [0, T].

Here, simplify the problem to T = 1. Now use a step-function:

to approximate the control Here the target function is: Let

Observe that if having instead then

would be optimal, with But if must be either 0 or 1, then

The STV Approach to Financial Optimal Control Models 27

will jump many times between 0 and 1. Suppose a further restriction is

made that, for some given integer time interval [0, 1] is divided into equal

subintervals

takes a constant value (1 or 0) on each subinterval

So if takes the values in this pattern 1, 0, 1, 0, ¦, 1, 0

on the successive subinterval, then each subintervals contributes to

as But the optimal value J = 0 is not

reached, unless an infinite number of jumps are allowed.

Now a term is added to the objective function. (2.7) is modified as

follows:

Here, K is the cost of changing control and is the number of time intervals.

The objective function of the optimal control problem becomes a cost function.

The algorithms in the next section are used for solving these kind of optimal

control problems.

6. Approaches to Bang-Bang Optimal Control with a Cost

of Changing Control

In this section, the computational methods for solving bang-bang optimal

control problems with a cost of switching control are introduced. Simply mod-

ifying certain parts of these methods can satisfy a class of similar problems.

Suppose the number N of switching times is fixed by N = nn, say switch-

ing times Now consider (2.7) as a function of the switching

times, say Then the minimum of this function

is computed, starting from a small value of N. Here the components of the

vector are the lengths between each switching

time. The vector um must satisfy the upper and lower bounds 0 and 1, thus

and a constraint must be satisfied. The

vector represents the values of the state func-

tion takes at each switching time. The algorithms are as follows:

Algorithm 2.1 Main Program (see project1_1.m in Appendix A.1)

Step 1. Initialization. Set the maximum number of function evaluations, par,

which is the system parameter of the MATLAB “constr” function, and

also another system parameter par(13) = 1 (1 represents the number

of the equation constraint in the minimization problem), and a vector of

the parameters which are used in the whole subroutines, par = [number

of the state components, number of control components, nn= number of

total subintervals], arbitrary starting lengths of switching time intervals

28 OPTIMAL CONTROL MODELS IN FINANCE

and vector ul = lower bound of um,

vector uu = upper bound of um, thus Initialize the value

of the state function xinit.

Step 2. Call the MATLAB “constr” function. In turn, “constr” calls the “Mini-

mizing Program” to calculate the minimization of the calling program with

respect to the optimal vector um.

Step 3. Input the optimal result um into “Minimizing Program” again to ob-

tain the results of the objective function (the last value of the calcula-

tion) and the state vector xm, corresponding to the optimal um.

Step 4. Attach a cost K to nn. Add to the objective function (2.7) and

calculate

Step 5. Set a bigger nn; then go back to Step 2; EXIT when the result of cost

function J stops decreasing.

Algorithm 2.2: Minimizing Program (see project1_2.m in Appendix A.1)

Step 1. Initialization. Input vectors um, par and initial state xinit. Set the

initial state xm(1) = xinit, and nx = par(1), the number of the state

components, nu = par(2), the number of the control components; initial-

ize scaled time subinterval counter it = 1, hs = 1/nn(length of

each equal subinterval). Choose the “Input function for dynamic equation”

as the right side of equation (2.9) to calculate the differential equation, input

um.

Step 2. Call SCOM package function “nqq” with the stated “Input function

for dynamic equation” to solve the differential equation (2.9) of the state

function Tabulate the solutions as the components of the vector

Step 3. Set the initial state zz = 0. Set initial scaled time subinterval

counter it = 1 again, choose the “input function for integration calcula-

tion” for SCOM function “nqq” for solving the integration of the objective

function (2.7), input vector xm and um.

Step 4. Call SCOM function “nqq” with the stated “Input function for integra-

tion calculation” to calculate:

where:

The STV Approach to Financial Optimal Control Models 29

when:

by solving the differential equation:

when:

Tabulate the results in as the components of vector

Step 5. Take the last result of the vector jm as the value of the objective func-

tion, and calculate the constraint function of “Minimizing program”, which

is:

Algorithm 2.3: Input function for dynamic equation (see project1_3.m in

Appendix A.1)

Step 1. Initialization. Input scaled time subinterval counter it, the length of

subintervals hs, and vector um, and set the number of subintervals nn =

1/hs.

Step 2. Set the control policy as vector with al-

ternating values 1 and 0 (as in 2.10) in successive subintervals.

Step 3. Construct the right side of the differential equation for the state func-

tion using the piecewise-linear transformation in (2.5).

Algorithm 2.4: Input function for integration calculation (see project1_4.m

in Appendix A.1)

Step 1. Initialization. Input scaled time subinterval counter it, the length

of subintervals hs, vector xm representing the values of the state function

at each switching time, and also a new initial state for integral, and the

vector um. Set the number of subintervals nn = 1/hs.

Step 2. Use the linear interpolation to get an estimate “xmt” of the state, in a

time between grid-points where

Step 3. Add up components of the lengths of the switching time intervals in

um to obtain the switching times in (2.4).

Step 4. Construct the right side of the equation (2.4) to obtain the time variable

OPTIMAL CONTROL MODELS IN FINANCE

30

Step 5. Calculate the integral in (2.6) at scaled time

When the number of switching time N increases, the cost function decreases

because of the better approximation. While calculating a minimization prob-

lem the term KN increases with N increasing, and decreases

with bigger N. It can be found that the cost of changing control is very criti-

cal in the cost function. A proper chosen cost K will efficiently lead the cost

function to the minimum. The analysis of the cost will be discussed in later

sections.

7. An Investment Planning Model and Results

In this section, the computational results reported by a set of graphs will

be introduced to verify the algorithms developed in Section 2.6. First we will

introduce the fitting function. In this example, the fitting function is set to be

The state function is used to approximate this fitting function. The

formulae for this financial optimal control model is shown as follows:

subject to:

where x(t) = stock price, u(t) = the proportion of total investment in stocks

compared to other forms of financial investment.

Although the above model is an illustrative model, it can, however, represent

an interesting financial decision making problem. The state equation represents

the dynamics of the price of a stock. It is assumed that the change in the

price of a stock is determined by the proportion of allocation of total funds for

purchasing a stock. The objective of this control problem is to determine the

value of u(t) (which only takes the value of 1 or 0) which can optimize the

objective function so as to minimize the deviation of the state variable from

its target value specified. Therefore this model (2.11 to 2.15) is an investment

planning model with some sub-utilization criterion included in the model.

The target function is First set (n is the number of time

intervals), and control takes as 1,0 in time intervals The model

2.11 to 2.15 was solved with these parameter values and the results are shown

The STV Approach to Financial Optimal Control Models 31

in Appendix B.1 and are represented by Figure 2.1. In Figure 2.1, it is shown

that during the planning horizon [0,1], the control only jumps once at time

The state vector xm takes the value [0.127, 0.127] at the grid-points

of two subintervals [0, 0.127] and [0.127, 1]. The approximation between the

state function xm and the fitting function is not very good when the

number of the switching times is quite small is only 2). “*“” represents

the state function and “.” represents the fitting function in the

following graph.

Figure 2.1. Plot of n=2, forcing function ut=1, 0

Then increase to 4. During the whole time [0,1], the control jumps three

times. A better approximation is shown in Figure 2.2 with more jumps of the

control.

Figure 2.2. Plot of n=4, forcing function ut=1,0,1,0

32 OPTIMAL CONTROL MODELS IN FINANCE

Set and run the program. A much better approximation than

Copyright Design by: Sunlight webdesign