Transcript Ingen lysbildetittel - Institutt for teknisk kybernetikk, NTNU

TTK4135 Optimization and control
Spring semester 2005
Scope - this you shall learn
Optimization - important concepts and theory
Formulating an engineering problem into an optimization
problem
Solving an optimization problem - algorithms, coding and
testing
Course information
Lectures are given by professor Bjarne A. Foss
The course assistant is Mr. K. Rambabu.
1
TTK4135 Optimization and control
B.Foss
Spring semester 2005
Course information
All course information is provided on the web-pages for the
course: www.itk.ntnu.no/fag/TTK4135. There will be no handout of material.
Every student must access the course web-pages at least
every week to keep updated course information (eg. changes
in lecture times, information on mid-term exam)
All students should subscribe to the email-list: 4135-optreg
The deadlines for all assignments (“øvinger” and the helicopter
lab. report) are absolute.
There will be 1-2 “øvingstimer” with assistants present ahead
of the deadline for every assignment.
A minimum number of “øvinger” and the helicopter lab.report
must be approved to enter the final examination.
I will not cover the complete curriculum in my lectures; rather
focus on the most important and difficult parts.
2
TTK4135 Optimization and control
B.Foss
Spring semester 2005
Grading
The final exam counts 70% on the final grade
The mid-term exam is graded. It counts 15% on the final
grade.
Please note that only this semester’s mid-term exam
counts. A mid-term grade from last year will not be
acknowledged.
The project report (based on the helicopter laboratory) is
graded. It counts 15% on the final grade.
Please note that only this semester’s report counts. A
report grade from an earlier year will not be acknowledged.
To ensure participation from all students 4 groups will be
selected for an oral presentation of their laboratory work.
This presentation will influence the grade on the report.
3
Finally
I welcome constructive criticism on all aspects of the course,
including
my lectures.
TTK4135 Optimization
and control
B.Foss
Spring semester 2005
Preliminary lecture plan
The content of each lecture is specified in the following slides.
All lectures are given in lecture halls EL 3 and EL 6.
The mid-term examination is on 2004-03-11.
The final examination is on 2004-05-23.
4
TTK4135 Optimization and control
B.Foss
Spring semester 2005
Content of Lecture #1 - 2004-01-10
Optimization problems appear everywhere
Stock portfolio management
Resource allocation (airline companies, transport companies, oil
well allocation problem)
Optimal adjustment of a PID-controller
Formulating an optimization problem: From an engineering
problem to a mathematical description.
Case: a realistic production planning problem
Defining an optimization problem
Definition of important terms
Convexity and non-convexity
Global vs. local solution
Constrained vs. unconstrained problems
Feasible region
Reference: Chapter 1 in Nocedal and Wright (N&W)
5
TTK4135 Optimization and control
B.Foss
Spring semester 2005
Content of Lecture #2 - 2004-01-14
Karush Kuhn-Tucker (KKT) conditions
Sensitivities and Lagrange-multipliers
Reference: Chapter 12.1, 12.2 in N&W
6
TTK4135 Optimization and control
B.Foss
Spring semester 2005
Content of Lecture #3 - 2004-01-17
Linear algebra (App. A.2 in N&W)
Norms of vectors and matrices
Positive definit and indefinite matrices
Condition number, well-conditioned and ill-conditioned linear
equations
Subspaces; null space and range space of a matrix
Eigenvalue and singular-value decomposition
Matrix factorization: Cholesky factorization, LU factorization
Sequences (App.A.1, Ch.2.2 “Rates of …” in N&W)
Convergence to some points; convergence rate; order notation
Sets (App.A.1 in N&W)
Open, closed, bounded sets
Functions (App.A.1 in N&W)
Continuity, Lipschitz continuity
Directional derivatives
7
TTK4135 Optimization and control
B.Foss
Spring semester 2005
Content of Lecture #4 + #5 - 2004-01-21/28
Linear programming - LP
Mathematical formulation
Condition for optimality - the Karush-Kuhn-Tucker (KKT)
conditions
Basic solutions - basis for the Simplex method
The Simplex method
Understanding the solution - Lagrange variables
The dual problem
Obtaining an initial feasible solution
Efficiency of algorithms
LP example - production planning
Reference: Ch.12.2,13-13.5 in textbook
8
TTK4135 Optimization and control
B.Foss
Spring semester 2005
Content of Lecture #6 - 2004-01-31
Quadratic programming - QP
Mathematical formulation
Convex vs. non-convex problems
Condition for optimality - KKT conditions
Special case: No inequality conditions
Reduced space methods
The active-set method for convex problems
Understanding the solution - Lagrange variables
The dual problem
Obtaining an initial feasible solution
Efficiency of algorithms
QP example - production planning (varying sales price)
Reference: Ch.12.2,16.1-16.4,(16.5),16.8 in textbook
9
TTK4135 Optimization and control
B.Foss
Spring semester 2005
Content of Lecture #7 - 2004-02-04
Quadratic programming - QP
The active-set method for convex problems
The active-set method for non-convex problems
QP example - production planning (varying sales price)
Reference: 16.4,16.5,16.8 in textbook
10
TTK4135 Optimization and control
B.Foss
Spring semester 2005
Content of Lecture #8 - 2004-02-07
Quadratic programming - QP
The active-set method for non-convex problems
Reference: 16.5,16.8 in textbook
11
TTK4135 Optimization and control
B.Foss
Spring semester 2005
Content of Lecture #9 - 2004-02-11
Repetition of LP, QP
--Optimality conditions
Necessary and sufficient conditions for optimality
Iterative solution methods
Starting point
Search direction
Step length
Termination criteria
Convergence
Reference: 2.1, 2.2 in textbook
12
TTK4135 Optimization and control
B.Foss
Spring semester 2005
Content of Lecture #10 - 2004-02-14
Line search methods
Choice of 
Wolfe-conditions
Back-tracking
Curve-fit and interpolation
Convergence of line-search methods - Theorem 3.2
Convergence rate
Reference: 3.1-3.4 in textbook
13
TTK4135 Optimization and control
B.Foss
Spring semester 2005
Content of Lecture #11 - 2004-02-18
Practical Newton-methods
Approximate Newton-step
Line search Newton
Modified Hessian
Reference: 6 - 6.3 in textbook
Computing gradients
Reference: 7 - 7.1 in textbook
14
TTK4135 Optimization and control
B.Foss
Spring semester 2005
Content of Lecture #12 - 2004-02-21
Quasi Newton methods
DFP and BFGS methods
Rosenbrock example for illustration
Reference: 8 - 8.1 in textbook
Information on the mid.term examination
15
TTK4135 Optimization and control
B.Foss
Spring semester 2005
Content of Lecture #13 - 2004-03-07
Mid-term examination
16
TTK4135 Optimization and control
B.Foss
Spring semester 2005
Content of Lecture #14 - 2004-04-01
Mid-term examination - once again
Model Predictive Control (MPC)
The MPC principle
Formulation of linear MPC
Formulating the optimisation problem which is a QP-problem
Reference: Ch.1 and 2 – Note on MPC by M.Hovd
17
TTK4135 Optimization and control
B.Foss
Spring semester 2005
Content of Lecture #15 - 2004-04-04
Linear Quadratic Control (LQ-control)
Formulation of the LQ-problem
Finite horizon LQ-control
Reference: Ch.1-1.2 - Note on LQ-control by B.Foss
18
TTK4135 Optimization and control
B.Foss
Spring semester 2005
Content of Lecture #16 - 2004-04-08
Linear Quadratic Control (LQ-control)
Infinite horizon LQ-control
State-estimation (repetition from TTK4115)
Reference: Ch.1.3-1.4 - Note on LQ-control by B.Foss
19
TTK4135 Optimization and control
B.Foss
Spring semester 2005
Content of Lecture #17 - 2004-04-11
Model Predictive Control (MPC)
Feasibility and constraint handling
Target calculation
Robustness
Reference: Ch.4 – 6, 8, 9 – Note on MPC by M.Hovd
20
TTK4135 Optimization and control
B.Foss
Spring semester 2005
Content of Lecture #18 - 2004-04-18
Nonlinear programming - SQP
Line-search in nonlinear programming
l1 exact merit function
Exact merit function
Reference: 15.3,18.5,18.6 in textbook
21
TTK4135 Optimization and control
B.Foss
Spring semester 2005
Content of Lecture #19 - 2004-04-25
Nonlinear programming - SQP
Computing the search direction
Solving nonlinear equtions
Quasi-Newton method for computing the Hessian
Reference: 11.1,18.1-18.4,18.6 in textbook
22
TTK4135 Optimization and control
B.Foss
Spring semester 2005
Content of Lecture #20 - 2004-05-02
Nonlinear programming - SQP
Reduced Hessian methods
Convergence rate
Maratos effect
Reference: 18.7,18.10,18.11 in textbook
23
TTK4135 Optimization and control
B.Foss
Spring semester 2005
Content of Lecture #21 - 2004-05-09
SQP – final remarks including examples
Repetition
Repetition of main topics
Course evaluation
----------------------
24
TTK4135 Optimization and control
B.Foss
Spring semester 2005