Transcript The Maximum Principle
The Maximum Principle of Optimal Control: A History of Ingenious Idea and Missed Opportunities
Hans Josef Pesch 1 , Michael Plail 2 1 University of Bayreuth, Germany 2 Steinebach, Wörthsee, Germany Optimization Day, University of Southern Australia, Adelaide, Australia, January 29, 2011
Outline
•
Carathéodory‘s Royal Road of the Calculus of Variations
•
Hestenes‘ secret report and first formulation
•
Bellman‘s and Isaacs‘ regrets Missed Opportunities to the Maximum Principle of Optimal Control
•
Pontryagin and his students: adoration and embitterment Hans Josef Pesch, Roland Bulirsch:
The Maximum Principle, Bellman‘s Equation, and Carathéodory‘s Work
J. of Optimization Theory and Applications, Vol. 80, No. 2, Feb. 1994 Hans Josef Pesch, Michael Plail:
The Maximum Principle of Optimal Control: A History of Ingenious Idea and Missed Opportunities
Control and Cybernetics, Vol. 38, No. 4A, 973-995, 2009.
Carathéodory‘s Royal Road in the Calculus of Variations Relationship between
Hilbert‘s Independence Theorem
and
Hamilton-Jacobi Equations
allows the reduction of
Problems of the Calculus of Variations
to
Problems of Finite Optimization
Carathéodory‘s Royal Road in the Calculus of Variations Lagrangian problems: precursors of optimal control Search for -curves that extremize subject to implicit differential equations
for line elements of curves with
DOF:
n - p
controls
Carathéodory‘s Royal Road in the Calculus of Variations Stage 1: Definition: extremal (minimal or maximal)
Different from today‘s terminology: weak extremum / minimum / maximum
closer neighborhood Stage 2: Proof of necessary Legendre-Clebsch condition
or in today‘s terminology for minimization has a positive definit Hessian for fixed
Carathéodory‘s Royal Road in the Calculus of Variations Stage 3: Caratheodory‘s equivalent variational problems Let then independent of
Carathéodory‘s Royal Road in the Calculus of Variations adding a null Lagrangian Let Then: integration along two curves yields Thus can be passed by one and only one minimal curve
Carathéodory‘s Royal Road in the Calculus of Variations Stage 4: Caratheodory‘s existence result for a special problem If there exists with for all and all with then the solutions of are extremals of
Carathéodory‘s Royal Road in the Calculus of Variations Stage 5: Caratheodory‘s sufficient condition
If there exists for which there hold and for sufficiently small , then the solutions of yield
Carathéodory‘s Royal Road in the Calculus of Variations Hence we have to determine the functions such that or (as function of ) possesses a minimum for
(Carathéodory, 1935)
That is the so-called Bellman Equation on Carathéodory‘s Royal Road
Carathéodory‘s Royal Road in the Calculus of Variations Stage 6: Caratheodory‘s formulation of Weierstraß‘ condition Substituting the fundamental equations and replacing by yields Hence we obtain the necessary condition of Weierstraß
Carathéodory‘s Royal Road in the Calculus of Variations Stage 7: Lagrangian variational problems Similarly the Weierstraß necessary condition takes the form (Carathéodory: 1926) Exit to the Maximum Principle?
Exit to the Maximum Principle from C‘s Royal Road Recall Caratheodory‘s Hamiltonian Carathéodory‘s closed approach to optimal control (from 1935) Today‘s Hamiltonian call them controls
Exit to the Maximum Principle from C‘s Royal Road With the maximizing Hamiltonian for and the costate we obtain as long as By means of the Euler-Lagrange equation
canonical equations
and because of
Exit to the Maximum Principle from C‘s Royal Road Furthermore Hence, must have a maximum with respect to along a curve From here it is still a big step to Missed Carathéodory the exit?
Constantin Carathéodory (1873 - 1950) 1904 1932
Constantin Carathéodory (1873 - 1950)
•
Born in Berlin to Greek parents, grew up in Brussels
•
(father was the Ottoman ambassador) to Belgium The Carathéodory family was well-respected in Constantinople (many important governmental positions)
•
Formal schooling at a private school in Vanderstock (1881-83); travelling with is father to Berlin, Italian Riviera;
• •
grammar school in Brussels (1985); high school Athénée Royal d'Ixelles, graduation in 1891 Twice winning of a prize as the best mathematics student in Belgium Trelingual (Greek, French, German), later: English, Italian, Turkish, and the ancient languages
•
École Militaire de Belgique (1891-95), École d'Application (1893-1896): military engineer
•
War between Turkey and Greece (break out 1897); British colonial service: construction of the Assiut dam (until 1900); Studied mathematics: Jordan's Cours d'Analyse a.o.; Measurements of Cheops pyramid (published in 1901)
Constantin Carathéodory (1873 - 1950)
• •
Graduate studies at the University of Göttingen (1902-04) (supervision of Hermann Minkowski: dissertation in 1904 (Oct.) on Diskontinuierliche Lösungen der Variationsrechnung In March 1905: venia legendi (Felix Klein)
•
Various lecturing positions in Hannover, Breslau, Göttingen and Berlin (1909-20)
•
Prussian Academy of Sciences (1919, together with Albert Einstein)
•
Plan for the creation of a new University in Greece (Ionian University)
• •
(1919, not realized due to the War in Asia Minor in 1922); the present day University of the Aegean claims to be the continuation
•
University of Smyrna (Izmir), invited by the Greek Prime Minister (1920); (major part in establishing the institution, ends in 1922 due to war University on Athens (until 1924) University of Munich (1924-38/50); Bavarian Academy of Sciences (1925)
•
C. played a remarkable opposing role together with the Munich „Dreigestirn“ (triumvirate) (Perron, Tietze) within the Bavarian Academy of Science during the Nazi terror in Germany
The Maximum Principle (first formulation, controls, 1950)
Thus, has a maximum value with respect to along a minimizing curve .
Research Memorandum RM-100, Rand Corporation, 1950
I became interested in control theory in 1948.
At that time I formulated the general control problem of Bolza …, and observed the maximum principle … is equivalent to the conditions of Euler Lagrange and Weierstrass … Magnus Rudolph Hestenes
(1906 – May 31, 1991)
It turns out that I had formulated what is now known as the general optimal control problem. Missed opportunity
The Maximum Principle (Bellman‘s & Isaacs‘ Equation, 1951+) Richard Ernest Bellman
(Aug. 26, 1920 – March 19, 1984)
Rufus Philip Isaacs
(1914 – 1981)
Isaacs in 1973 about his
Tenet of Transition
of 1951
Once I felt that here was the heart of the subject ….. Later I felt that it … was a mere truism.
Thus in (my book) Differential Games
it is mentioned only by title. This I regret. I had no idea, that Pontryagin‘s principle and Bellman‘s maximal principle (a special case of the tenet, appearing a little later in the Rand seminars) would enjoy such a widespread citation.
Missed opportunities
The Maximum Principle (1956)
This fact is a special case of the following general principle which we call maximum principle Doklady Akademii Nauk SSSR, Vol. 10, 1956
Lev Semenovich Pontryagin
(Лев Семёнович Понтрягин) (Sept. 3, 1908 – May 3. 1988)
The Maximum Principle (1956) Vladimir G. Boltyanski Revaz V. Gamkrelidze proved the Maximum Principle
Boltyanski in 1991 about the Maximum Principle of 1956
By the way, the first statement of the maximum principle was given by Gamkrelidze, who has established (generalizing the famous Legendre Theorem) a sufficient condition for a sort of weak optimality problem. Then, Pontryagin proposed to name Gamkrelidze‘s condition Maximum Principle. … Finally, I understood that the maximum principle is not a sufficient, but only a necessary condition of optimality.
Pontryagin was the Chairman of our department at the Steklov Mathematical Institute, and he could insist on his interests.
So, I had to use the title Pontryagin‘s Maximum Principle in my paper. This is why all investigators in region of mathematics and engineering know the main optimization criterium as the Pontryagin‘s Maximum Principle.
Gamkrelidze in 2008 about Pontryagin
My life was a series of missed opportunities, but one opportunity, I have not missed, to have met Pontryagin.*
*
at the Banach Center Conference on 50 Years of Optimal Control in Bedlewo, Poland, on September 15, 2008
Plail, M.:
Die Entwicklung der optimalen Steuerungen
. Vandenhoeck & Ruprecht, Göttingen, Germany, 1998
Carathéodory‘s words: I will be glad if I have succeeded in impressing the idea that it is not only pleasant and entertaining to read at times the works of the old mathematicial authors, but that this may occasionally be of use for the actual advancement of science.
Besides this there is a great lesson we can derive from the facts which I have just referred to. We have seen that even under conditions which seem most favorable very important results can be discarded for a long time and whirled away from the main stream which is carrying the vessel science. … If their ideas are too far in advance of their time, and if the general public is not prepared to accept them, these ideas may sleep for centuries on the shelves of our libraries … awaiting the arrival of the prince charming who will take them home. (C.C. 1937) Constantin Carathéodory
( Κωνσταντίνος Καραθεοδωρή) * Sept. 13, 1873 in Berlin; † Feb. 2, 1950, Munich