Selected Titles in This Series
41 Georgi V. Smirnov, Introduction to the theory of differential inclusions, 2002
40 Robert E. Greene and Steven G. Krantz, Function theory of one complex variable,
2002
39 Larry C. Grove, Classical groups and geometric algebra, 2002
38 Elton P. Hsu, Stochastic analysis on manifolds, 2002
37 Hershel M. Farkas and Irwin Kra, Theta constants, Riemann surfaces and the Inodular
group, 2001
36 Martin Schechter, Principles of functional analysis, second edition, 2002
35 James F. Davis and Paul Kirk, Lecture notes in algebraic topology, 2001
34 Sigurdur Helgason, Differential geometry, Lie groups, and symmetric spaces, 2001
33 Dmitri Burago, Yuri Burago, and Sergei Ivanov, A course in metric geometry, 2001
32 Robert G. Bartle, A modern theory of integration, 2001
31 Ralf Korn and Elke Korn, Option pricing and portfolio optimization: Modern methods
of financial mathematics, 2001
30 J. C. McConnell and J. C. Robson, Noncommutative Noetherian rings, 2001
29 Javier Duoandikoetxea, Fourier analysis, 2001
28 Liviu I. Nicolaescu, Notes on Seiberg- Witten theory, 2000
27 Thierry Aubin, A course in differential geometry, 2001
26 Rolf Berndt, An introduction to symplectic geometry, 2001
25 Thomas Friedrich, Dirac operators in Riemannian geometry, 2000
24 Helmut Koch, Number theory: Algebraic numbers and functions, 2000
23 Alberto Candel and Lawrence Conlon, Foliations 1,2000
22 Gunter R. Krause and Thomas H. Lenagan, Growth of algebras and Gelfand-Kirillov
dimension, 2000
21 John B. Conway, A course in operator theory, 2000
20 Robert E. Gompf and Andras I. Stipsicz, 4-manifolds and Kirby calculus, 1999
19 Lawrence C. Evans, Partial differential equations, 1998
18 Winfried Just and Martin Weese, Discovering modern set theory. II: Set-theoretic
tools for every mathematician, 1997
17 Henryk Iwaniec, Topics in classical automorphic forms, 1997
16 Richard V. Kadison and John R. Ringrose, Fundamentals of the theory of operator
algebras. Volume II: Advanced theory, 1997
15 Richard V. Kadison and John R. Ringrose, Fundamentals of the theory of operator
algebras. Volume I: Elementary theory, 1997
14 Elliott H. Lieb and Michael Loss, Analysis, 1997
13 Paul C. Shields, The ergodic theory of discrete sample paths, 1996
12 N. V. Krylov, Lectures on elliptic and parabolic equations in Holder spaces, 1996
11 Jacques Dixmier, Enveloping algebras, 1996 Printing
10 Barry Simon, Representations of finite and compact groups, 1996
9 Dino Lorenzini, An invitation to arithmetic geometry, 1996
8 Winfried Just and Martin Weese, Discovering modern set theory. I: The basics, 1996
7 Gerald J. Janusz, Algebraic number fields, second edition, 1996
6 Jens Carsten Jantzen, Lectures on quantum groups, 1996
5 Rick Miranda, Algebraic curves and Riemann surfaces, 1995
4 Russell A. Gordon, The integrals of Lebesgue, Denjoy, Perron, and Henstock, 1994
3 William W. Adams and Philippe Loustaunau, An introduction to Grabner bases,
1994
(Continued in the back of this publication)

Introduction to the
Theory of Differential
Inclusions

Introduction to the
Theory of Differential
Inclusions
Georgi V. Smirnov
Graduate Studies
in Mathematics
Volume 41
American Mathematical Society
Providence, Rhode Island

Editorial Board
Steven G. Krantz
David Saltman (Chair)
David Sattinger
Ronald Stern
2000 Mathematics Subject Classification. Primary 34A60, 34D20, 49K24;
Secondary 49J24, 49J52, 93D15.
ABSTRACT. Differential inclusions theory is presented at an elementary level. The emphasis is
given to applications such as viability theory, controllability, necessary conditions of optimality,
asymptotic stability at first approximation, and the stabilization problem. The text is intended
for graduate students who specialize in pure and applied analysis.
Library of Congress Cataloging-in-Publication Data
Smirnov, Georgi V.
Introduction to the theory of differential inclusions / Georgi V. Smirnov.
p. cm. - (Graduate studies in mathematics, ISSN 1065-7339 j v. 41)
Includes bibliographical references and index.
ISBN 0-8218-2977-7 (alk. paper)
1. Differential inclusions. I. Title. II. Series.
QA371.S517 2001
515'.35-dc21
2001053414
Copying and reprinting. Individual readers of this publication, and nonprofit libraries
acting for them, are permitted to make fair use of the material, such as to copy a chapter for use
in teaching or research. Permission is granted to quote brief passages from this publication in
reviews, provided the customary acknowledgment of the source is given.
Republication, systematic copying, or multiple reproduction of any material in this publication
is permitted only under license from the American Mathematical Society. Requests for such
permission should be addressed to the Assistant to the Publisher, American Mathematical Society,
P. O. Box 6248, Providence, Rhode Island 02940-6248. Requests can also be made by e-lnail to
reprint-permission@ams.org.
@ 2002 by the American Mathematical Society. All rights reserved.
The American Mathematical Society retains all rights
except those granted to the United States Government.
Printed in the United States of America.
@ The paper used in this book is acid-free and falls within the guidelines
established to ensure permanence and d urabili ty.
Visit the AMS home page at URL: http://www . ams . org/
10 9 8 7 6 5 4 3 2 1
07 06 05 04 03 02

Contents
Preface
Xl
Introduction
XllI
Part 1. Foundations
Chapter 1. Convex Analysis
31.1. Convex sets
31.2. Convex functions
31.3. Differential properties of convex functions
31.4. Problems
3
3
13
24
30
31
31
37
41
44
47
49
56
61
Chapter 2. Set-Valued Analysis
32.1. Set-valued maps
32.2. Derivatives of set-valued maps
32.3. Lipschitzian approximations
32.4. Extension theorem
32.5. Fixed point theorems
32.6. Convex processes
32.7. Structure of a convex process
32.8. Problems
-
. .
Vll

Vlll
Contents
Chapter 3. N onslllooth Analysis 65
9 3 . 1 . Method of metric approximations 65
9 3 . 2 . Mordukhovich normal cone 67
9 3 .3. Separation theorem for non convex sets 72
9 3 . 4 . Nonsmooth calculus 75
9 3 . 5 . Lagrange multipliers 82
9 3 . 6 . Problems 83
Part 2. Differential Inclusions
Chapter 4. Existence Theorems 87
9 4 . 1 . Background notes 88
9 4 . 2 . Lipschitzian differential inclusions 90
9 4 . 3 . Upper semi-co11tinuous differential inclusions 96
9 4 . 4 . Discontinuous differential equations 103
9 4 . 5 . Existence of optimal solutions 106
9 4 . 6 . Dependence on initial conditions 109
9 4 . 7 . Discrete approximations 113
9 4 . 8 . Problellls 116
Chapter 5. Viability and Invariance 119
9 5 . 1 . Monotone solutions to a differential inclusion 119
9 5 . 2 . Viability problem 122
9 5 . 3 . Invariant sets 127
9 5 . 4 . Existence of periodic solutions 130
9 5 . 5 . Pursuit in a differential game 132
9 5 . 6 . Problems 135
Chapter 6. Controllability 139
9 6 . 1 . Duality relation 139
8 6 . 2 . Controllability of convex processes 145
9 6 . 3 . Controllability at first approximation 147
8 6 . 4 . Controllability of son1e mechanical systems 152

Contents
IX
96.5. Problems
Chapter 7. Optimality
97.1. Optimal solutions to discrete- time inclusions
97.2. Optimal solutions to differential inclusions
97.3. Time-optimal problem
97.4. Problems
Chapter 8. Stability
98.1. Lyapunov direct method
98.2. Linear-selectionable differential inclusions
98.3. Weak asymptotic stability of convex processes
98.4. First approximation techniques
98.5. Stability of a missile uniform motion
98.6. Problems
Chapter 9. Stabilization
39.1. Lyapunov functions for convex processes
99.2. Stabilization problelll
99.3. Weak asymptotic stability and stabilizability
39.4. Stabilizers for some mechanical systems
99.5. Problems
153
157
157
160
165
168
171
171
176
185
189
194
196
199
200
202
205
208
210
213
219
225
Comments
Bibliography
Index

Preface
The aim of this text is to provide an introductory treatment of the theory of
differential inclusions that will be accessible to anyone having a basic knowl-
edge of analysis, theory of functions, and ordinary differential equations.
The present book is an expanded record of lectures given at the Inter-
national School for Advanced Studies (SISSA), Trieste, Italy, and at the

Universities of Evora and Porto, Portugal, during the last few years.
Most of the material in this text was written when the author was visit-
ing SISSA. I acknowledge the opportunity offered by the Functional Analysis
Sector of SISSA to work there. I wish to express my thanks to Boris Mor-
dukhovich, who read the draft of the manuscript and made important re-
marks. I also thank the students who made suggestions that have improved
the presentation.
Georgi Smirnov

Introd uction
In this text we consider differential inclusions
X E F(x),
where F is a set-valued map which associates with any point x E R n a
set F(x) c R n . Differential inclusions serve as models for many dynam-
ical systems. Obviously, any process described by an ordinary differential
equation
x == f(x)
can be described by a differential inclusion with the right-hand side F(x) ==
{f(x)} as well. A system of differential inequalities
. i < f i ( 1 n ) ' _ 1
x _ X,...,X ,2==,n
can also be considered as a differential inclusion. If an implicit differential
equation
f(x, x) == 0
is given, then we can put F(x) == {v I f(x, v) == O} to reduce it to a differen-
tial inclusion. Differential inclusions are used to study ordinary differential
equations with an inaccurately known right-hand side. Suppose that the
right-hand side of a differential equation is in an E-neighborhood of a given
function f(x). Then any solution of the differential equation is a solution
to the differential inclusion
x E f(x) + EBn,
where Bn is a unit ball in R n centered at zero.
Differential inclusions play a crucial role in the theory of differential
equations with a discontinuous right-hand side. The investigation of such
-
XIII

XIV
Introduction
equations is of great importance since they model the performance of various
mechanical and electrical devices as well as the behavior of automatic control
systems. Differential equation
x == f(x)
with discontinuous f is a rather unpleasant object from the mathematical
point of view. In particular it is impossible to prove existence theorems.
However if solutions of the differential equation with discontinuous right-
hand side are regarded to be solutions to the differential inclusion
x E n cl cof(x + f.B n ),
€>o
then it is possible to develop a rigorous mathematical theory of discontinuous
systems.
One of the most important examples of differential inclusions comes from
control theory. Consider a control system
x == f (x, u), U E U,
where u is a control parameter. It appears that the control system and the
differential inclusion
x E f(x, U) = U f(x, u)
uEU
have the same trajectories. If the set of controls depends on x, that is,
U == U (x), then we obtain the differential inclusion
x E f(x, U(x)).
The equivalence between a control system and the corresponding differential
inclusion is the central idea used to prove existence theorelns in optimal
control theory.
Since the dynamics of economical, social, and biological macrosystems is
multi valued, differential inclusions serve as natural models in macrosystem
dynamics. Differential inclusions are also used to describe some systems
with hysteresis.
Differential inclusion is a generalization of the notion of an ordinary dif-
ferential equation. Therefore all problems considered for differential equa-
tions, that is, existence of solutions, continuation of solutions, dependence
on initial conditions and parameters, are present in the theory of differential
inclusions. Since a differential inclusion usually has many solutions start-
ing at a given point, new issues appear, such as investigation of topological
properties of the set of solutions, selection of solutions with given properties,

Introduction
xv
evaluation of the reachability sets, etc. To solve the above problems special
mathematical techniques were developed.
Thus, the differential inclusions are not only models for many dynam-
ical processes but they also provide a powerful tool for various branches
of mathematical analysis. As we have mentioned the differential inclusion
techniques are applied to prove existence theorems in optimal control the-
ory. They are used to derive sufficient conditions of optimality, play an
essential role in the theory of control under conditions of uncertainty and in
differential games theory.
In this text we discuss many important problems of differential inclu-
sions theory. For simplicity of presentation we consider only differential
inclusions with a convex-valued right-hand side. The mathematical difficul-
ties appearing in the nonconvex case and outstanding techniques developed
to overCOlne them are outside of the frame of this text. This assumption
enables us to apply approximation techniques in order to solve Inany in-
teresting problems. Namely, we approximate differential inclusions under
consideration by differential (or even discrete-time) inclusions with simpler
structure. We then solve the problem for the simplified dynamical systems
and obtain the result for the original problem using a limiting procedure.
This is the central idea of many proofs in the text that allows us to avoid
complicated mathematical constructions. Note that this method simplifies
the proofs of some classical theorems from the theory of ordinary differential
equations (the Hukuhara theorem, the Kneser theorem, for instance).
The required mathematical background is knowledge of ordinary differ-
ential equations theory, the theory of functions, and functional analysis at an
elementary level. The text is intended for graduate students who specialize
in pure and applied analysis.
The text is organized as follows. The first chapter contains a brief in-
troduction to convex analysis. In the second chapter we consider set-valued
maps. The third chapter is devoted to the Mordukhovich version of non-
smooth analysis. Chapter 4 contains the main existence theorems and gives
an idea of the approximation techniques used throughout the text. Chapter
5 is devoted to the viability problem, that is, the problem of selection of a
solution to a differential inclusion which is contained in a given set. The
controllability problem is considered in Chapter 6. In Chapter 7 we discuss
extremal problems for differential inclusions. Stability theory for differential
inclusions is presented in Chapter 8. In the last chapter we deal with the
stabilization problem.

XVI
Introduction
At the end of the book we suggest further reading intended to provide
more detailed coverage of the relevant topics, but no attempt is made to
survey the literature on the subject.

Part 1
Foundations

Chapter 1
Convex Analysis
Convex analysis studies properties of convex sets and convex functions. The
study is based on a systematic use of the relationship between sets and func-
tions, which consists of the possibility of identifying the functions with their
epigraphs and the sets with their indicator functions. These identifications
make it easy to combine geometric and analytic methods.
The main concept of convex analysis consists of the fact that a closed
convex set is the intersection of all the closed half-spaces which contain it.
This property of convex sets follows from the separation theorem, which is a
formalization of one intuitive fact: two convex sets without common points
can be separated by a hyperplane.
Convexity implies many useful properties of sets and functions. For
example, a convex set either has nonempty interior or is contained in a
plane and its interior relative to the plane is nonempty. Bounded convex
functions are continuous and directionally differentiable. However, convex
functions, in general, are not differentiable in the usual sense. For t11is reason
convex analysis contains a concept of subdifferential which is a replacement
for the ordinary gradient and plays an important role in optimization. The
rules of subdifferential calculus are similar to the rules of classical analysis.
In this chapter we present some results of convex analysis that are es-
sential to the rest of the book.
1.1. Convex sets
Throughout this text we denote the set of real numbers by R and the usual
n-dimensional space of vectors x == (xl,... , x n ), where xi E R, i == 1, n ,
-
3

4
1. COllvex Analysis
by R n . By zn we denote the set of n-dimensional vectors with integer
COlTIpOnents. The inner product of two vectors x and y in R n is expressed
by
(x, y) == x 1 y1 + . . . + xnyn.
The nOrlTI of a vector x E R n is defined by Ixl == (x, x)1/2. Let C be a
linear operator from R n to R m . The adjoint linear operator from R m to
R n is denoted by C*. If C is an m x n real matrix corresponding to the
linear operator C (we use the same sYlTIbol), then the transposed matrix C*
corresponds to the adjoint operator. The unit linear operator frOlTI R n to
R n will be denoted by E. If there is a danger of confusion concerning the
dimension, this (n x n)-matrix will be denoted by En. We denote the unit
ball in R n by Bn:
Bn == {x E R n II x I < I}.
The open unit ball in R n is defined by
o
Bn== {x E R n Ilxl < I}.
Let A eRn. The distance function d(., A) : R n  R is defined by
d(x, A) == inf{lx - all a E A}, x ERn.
Let A E R. Then put by definition
AA == {Aa I a E A}.
For two sets A and B in R n their sum is defined by
A+B=={a+blaEA, bEB}.
The closure cIA and interior intA of A are defined by the formulas
cIA = n(A + EBn),
€>o
intA == {a E A I E > 0, a + EBn C A}.
The boundary of A is defined by
bdA == cIA \ intA.
o 0
Obviously clBn== Bn, intBn ==Bn, and bdBn == {x Ilxl == I}.
A set A C R n is said to be convex if Ax + (1 - A)y E A whenever x E A,
YEA, and A E [0, 1]. By definition it follows that an intersection of any
nUlTIber of convex sets is a convex set, and if A c R n , BeRn are convex,
and ex and (3 are real numbers, then the set exA + (3B is convex. If A is
convex, then intA and cIA are also convex sets.

1.1. Convex sets
5
Convex hulls. Let A c R n . The intersectio11 of all convex sets c011tail1ing
A is called the convex hull of A a11d is de110ted by coA.
A vector sum
A1 X 1 + . . . + Amxm
is called a convex combination of Xl . . . X m if Ai > 0, i == 1, m, and Al +
. . . + A 7n == 1. Obviously if Xl, . . . , X m are vectors from A, then any C011vex
combination of Xl, . . . , X m belongs to coA. The following inverse stateme11t
is very important.
Theorem 1.1 (Caratheodory). Let A c R n . For any X E coA there exist
Xl, . . . X m E A such that m < n + 1 and
X == A1X1 + . . . + Amxm,
where Ai > 0, i == I,m , and Al +... + Am == 1.
Remark. 111 other words, any point X E coA ca11 be expressed as a convex
combination of at most n + 1 points from A.
Proof. Obviously, the set of all convex combinatio11s
(1.1)
X == A1X1 + . . . + Amxm,
Xi E A, Ai > 0, i == 1, m , Al + . . . Am == 1
is convex and contains A. Consequently for a11Y X E coA t11ere exist points
of A satisfying (1.1). Let us show that the number of nonzero terms in (1.1)
can be reduced if m > n + 1. Without loss of ge11erality we can assume that
Ai > O. Consider (n + 1 )-dilnensional vectors (Xi, 1) == (xl,.. . , xi, 1), i ==
1, m . Since m > n + 1 these vectors are linearly depe11dent. Conseque11tly
there exist ai, i == 1, m , at least one of which is nonzero, such that
(1.2)
(1.3)
a1X1 + . . . + amX m == 0,
a1 + . . . + am == O.
From (1.3) it follows that at least one coefficient ai is positive. Set
E == Inin {Ail ai I ai > 0, i == 1, m }.
Let the minimum be achieved for i == io. Then
J1i == Ai - E ai > 0, i == 1, m,
and J1io == O. From (1.1) and (1.2) we have
m m m
I:: J-LiXi = I:: AiXi - E(I:: QiXi) = X.
i==l i==l i==l

6
1. Convex Analysis
From (1.3) we obtain
m m m
LJ.li = L'\ - E(Lai) = 1.
i==l i==l i==l
Therefore the point x can be expressed as a sum of a smaller number of
nonzero terms. Thus, we can reduce the number of terms whenever m >
n + 1. The theorem is proved. 0
Corollary 1.1. The convex hull of a compact set is a compact set.
Proof. Recall that a set in R n is compact if and only if it is closed and
bounded. Let A c R n be compact. By Theorem 1.1 the convex hull coA is
bounded. Let us prove that coA is closed.
Let Xk E coA, k == 1,2,..., and limxk == Xo. By Theorem 1.1 we have
n+1
Xk = L Ai,kXi,k,
i==l
(1.4 )
where
(1.5)
Xi,k E A, A1,k + . . . + An+1,k == 1, Ai,k > o.
The sequences {Xi,k} k l' {Ai,k} k l' i == 1, n + 1 are bounded and therefore
contain convergent subsequences. Without loss of generality we can assume
that Xi,k  Xi,O, Ai,k  Ai,O. Since A is compact, we have Xi,O E A. Taking
the limit in (1.4) and (1.5) we obtain
n+1
Xo = L Ai,OXi,O,
i==l
where
Xi,O E A, A1,0 + . . . + A n +1,0 == 1, Ai,O > o.
Hence Xo E coA. Thus, coA is a closed set.
o
Topological properties. A convex set always can be imbedded into a
subspace translated by a vector in such a way that it has a nonempty interior
as a subset of the translated subspace.
Theorem 1.2. Let A c R n be a convex set. Then either intA =I 0 or A is
contained in a subspace translated by a vector.
Proof. Let Xo E A. There exists r < n linearly independent vectors
Xl - Xo, . .. , X r - Xo, where Xi E A. Suppose first that r == n. Consider
the set
 == co{ XO, . .. , xn} == {AOXO + . . . + AnXn I AD + . . . + An == 1, Ai > o}.

1.1. Convex sets
7
Obviously  c A. Let us show that int =1= 0. For arbitrary x E  consider
the system of linear equations
x - Xo == Al (Xl - XO) + . . . + An(Xn - XO),
where AI, . . . , An are unknown variables. Since the vectors Xl - xo, . .. , x n -
Xo are linearly independent, the system has a unique solution Ai(X), i == 1, n .
The solution continuously depends on X due to Cramer's rule. Set x ==
(xo + . . . + xn)/(n + 1). Obviously x E  and Ai(X) == l/(n + 1), i == 1, n .
Define AO(X) == 1 - (AI (x) +... + An(X)) and note that AO(X) == l/(n + 1).
We observe that Ai(X) > 0, i == 0, n , whenever X is in a neighborhood of X.
Hence for all x in a neighborhood of x we have
x == AO(X)XO + . . . + An(X)Xn E .
Thus intA =1= 0.
Now consider the case r < n. Let X be the subspace spanned by vectors
of the form
x == a1(x1 - xo) +... + ar(xr - xo),
where ai E R. By construction A - Xo eX, that is, A c Xo + X. The
theorem is proved. D
The subspace X constructed in the proof of the theorem is r-dimensional,
and the set A - Xo regarded as a subset of X has nonempty interior. (This
fact can be proved similarly to the first part of the theorem.) It is easy to
verify that X does not depend on the choice of Xo and the vectors Xi"- XO, i ==
1, r . This justifies the following definitions.
The intersection of all subspaces containing A-x, where x is an arbitrary
point in the convex set A, is called carrier subspace and is denoted by LinA.
The relative interior of a convex set A, which we denote by riA, is defined as
the interior which results when A is regarded as a subset of LinA translated
by a vector x E A. In other words,
riA == {x E A I 3E > 0, x + (EBn n LinA) C A}.
The set rbA == cIA \ riA is called the relative boundary of A.
Separation theorems. Now we proceed to study an important concept
of convex analysis - the concept of separation. It turns out that any two
convex sets without common points can be separated by a hyperplane. The
theorems presented below formalize the separation property.

8
1. Convex Analysis
Let A c R n and x E R n . The projection of x onto A is the set defined
by
7r(x,A) == {a E A II X - al == d(x,A)}.
Lemma 1.1. If x E Rn and A C R n is a closed convex set, then 7r(x, A)
consists of a single point; that is, there exists a unique point in A nearest to
x.
Proof. Since A is closed, there exist nearest points. To prove the unique-
ness, suppose the contrary. Let a1 E 7r(x, A) and a2 E 7r(x, A) be two
different nearest points. Set b == (a1 + a2)/2. Since A is convex, b E A.
Since the seglnent connecting x and b is the height of the isosceles triangle
x, aI, a2, we have Ix - bl < Ix - all == Ix - a21. Hence a1 and a2 are not the
nearest points, a contradiction. 0
Lemma 1.2. Let A C R n be a convex set. If Xo E riA, Xl E cIA, and
Xo :/= Xl, then X,A == (1 - A)XO + AX1 E riA for all A E [0,1[.
o
Proof. Since Xo E riA, there exists E > 0 such that U == xo+LinAnE Bnc A.
Let A E [0,1[. Set V == (X,A - (1 - A)U)/ A. Obviously V c Xo + LinA and
Xl == (X,A - (1 - A)XO)A E V. Since Xl E cIA, there exists Y1 E V n A. Put
W == (1- A)U +AY1. Since A is convex we have W C A. Show t11at X,A E W.
By the definition of V there exists Yo E U satisfying Y1 == (X,A - (1- A)YO)/ A.
Consequently
X,A == (1 - A)YO + AY1 E (1 - A)U + AY1 == W.
Hence X,A E riA.
o
Proposition 1.1. Let Al C R n and A 2 C R n be convex sets. If riAl n
rL4 2 :/= 0, then
cl(A1 n A 2 ) == clA 1 n c1A 2 .
Proof. Let x E riAl n riA 2 , and let X E clA 1 n c1A 2 . By Lemma 1.2
AX + (1 - A)X E riAl n riA 2 whenever A E [0,1[. Therefore X E cl(A 1 n A 2 ),
and we get clA 1 n clA 2 C cl(A 1 n A 2 ).
The inverse inclusion is obvious. 0
Lemma 1.3. Let A C R n be a convex set and let the point Xl E cIA be such
that Xl f/. A. Then any neighborhood of Xl contains points which are not
contained in cIA.
Proof. Consider a point Xo E riA. Then the points of the ray Xo + A(X1 -
xo), A > 0 are not contained in cIA when A > 1. Indeed, if A > 1 and
X == Xo + A(X1 - xo) E cIA, then
Xl == (X + (A - l)xo)/ A E riA
due to Lemma 1.2. Since Xl f/. A, we reach a contradiction.
o

1.1. Convex sets
9
Theorem 1.3. Let A c R n be a convex set and let Xo tt cIA. Then there
exist a vector x* =I=- 0 and a positive number E such that
(x, x*) < (xo, x*) - E
for all x E A.
Proof. Let y == 7r(xo, cIA) (see Lelllllla 1.1). By the definition of a projection
we have Ix - xol > Iy - xol for all x E cIA. Since AX + (1 - A)Y E cIA for all
x E A and A E [0,1], we obtai11
lAx + (1 - A)Y - xol 2 == (y - Xo + A(X - y), y - Xo + A(X - y))
== Iy - xol 2 + 2A(X - y,y - xo) + A21x - yl2 > Iy - xol 2 .
Consequently
2(x - y, y - xo) + Alx - yl2 > 0
for all A E [0, 1]. In particular, if A == 0, then we have
(1.6)
(x - y, y - xo) > o.
Set x* == Xo - Y and E == Ix* 1 2 . Since Xo tt cIA, we get y =I=- Xo. Therefore
x* =I=- 0 and E > O. Inequality (1.6) can be represented in the form
(x, x*) < (y, x*) == (xo, x*) - (x*, x*) == (xo, x*) - E.
Since x E A is arbitrary, we obtain the result.
D
Remark. In particular we have proved the following proposition. If a point
Xo does not belong to a convex closed set A, then
(x - 7r(xo, A), Xo - 7r(xo, A)) < 0
for all x E A.
Theorem 1.4. Let A E Rn be a convex set and let Xo tt A. Then there
exists a vector x* =I=- 0 such that
(x, x*) < (xo, x*)
for all x E A.
Proof. If Xo tt cIA, then the result follows from Theorem 1.3. Let Xo E
cIA. By Lemllla 1.3 there exists a sequence Xk  Xo with Xk tt cIA. By
Theorem 1.3 there exist nonzero vectors xk and positive numbers Ek such
that
(x, xk) < (Xk, Xk) - Ek
whenever x E A. Dividing by Ixkl, we obtain
(1.7)
(x, xk/lxkl) < (Xk, xk/lxkl)

10
1. Convex Analysis
for all x E A. Without loss of generality we can assume that
xk/lxkl  x*, Ix*1 == 1.
Taking the limit as k  00 in (1.7), we obtain
(x, x*) < (xo, x*)
for all x E A.
o
Theorem 1.5. Let Al and A 2 be convex sets in R n satisfying Al n A 2 == 0.
Then there exists a nonzero vector x* such that
(Xl, x*) < (X2, x*)
for all Xl E Al and X2 E A 2 .
Proof. Set A == Al -A 2 . Since Al nA 2 == 0, we have 0 (j. A. By Theorem 1.4
there exists x* :f. 0 such that
(X, x*) < (0, x*)
for all X E A, that is, for all X == Xl - X2, where Xl E AI, X2 E A 2 . Hence
(Xl - X2, x*) < 0, Xl E AI, X2 E A 2 .
This ends the proof.
o
If one of the sets is compact and the other is closed, then a strong version
of the separation theorem can be proved.
Theorem 1.6. Let Al E R n be a convex compact set and let A 2 E R n be a
convex closed set. If Al n A 2 == 0, then there exist a nonzero vector x* and
a positive number E such that
(Xl, x*) < (X2, x*) - E
for all Xl E Al and X2 E A 2 .
Proof. Set A == Al - A 2 . Since Al n A 2 == 0, we have 0 (j. A. Show that the
set A is closed. Indeed, let al k E AI, a2 k E A 2 and ak == al k - a2 k  a.
" "
Since Al is compact, the sequence {al,k} contains a convergent subsequence.
Without loss of generality the sequence {al,k} cnverges, that is, al,k  al E
AI' Since the sequence {ak} converges and the set A 2 is closed, we obtain
a2,k  a2 E A 2 . Hence a == al - a2 E A. Thus, the point Xo == 0 is not
contained in the closed set A. Applying Theorem 1.3, we obtain the result.
o

1.1. Convex sets
11
Examples of convex sets. Consider several examples of convex sets. The
convex hull of finitely many points is called a polytope. By Corollary 1.1
a polyt ope is compact. If Xl, . . . , Xn+l E R n and the vectors Xi - Xl, i ==
2, n + 1, are linearly independent, then th e polyt ope  == co{ Xl, . .. , Xn+l}
is called a simplex. The points Xi, i == 1, n + 1 are called vertices of the
simplex. As it follows from the proof of Theorem 1.2 int :/=- 0.
A set KeRn is called a cone if Ax E K when X E K and A > o. A
convex cone is a cone which is a convex set. From the definition it follows
that
AlXl + . . . + AmXm E K
whenever Xl, . . . , X m E K and AI, . . . , Am are positive. It is easy to check
that if Kl and K 2 are convex cones in R n , then Kl + K 2 is a convex cone.
Denote by R+. the positive orthant, that is, R+. == {x E R n I xi > 0, i ==
1, n }. Obviously R+. is a closed convex cone. The cone R we shall denote
by R+.
The cone
K* == {x* I (x, x*) > 0, \Ix E K}
is called a conjugate cone of K. Obviously K* is a convex cone. Moreover,
K* is closed. Indeed, let x k E K* and x k  x*. Then we have
(x, xk) > 0
for all x E K. Taking the limit, we obtain x* E K*. The cone -K* is called
a polar cone of K.
The following useful result is a simple consequence of the definition.
Proposition 1.2. Let Kl and K 2 be convex cones in R n . Then
(K l + K2)* == K; n K 2 .
Let K == {x I x ==  f 1 aiXi, ai > O,i == I,N }. Since {x I x == ax, a >
O}* == {x* I (x*,x) > O}, from Proposition 1.2 we get
K* == {x* I (X*,Xi) > 0, i == I,N }.
To study local properties of a convex set A in a neighborhood of a point
x E A the tangent cone is used. Let A c R n and x E A. The set
U A-x
T(x,A) = cl A
A>O
is called the tangent cone to A at x. If 0 E A, then we shall denote T(O, A)
by coneA.
The set (A - x) / A is monotone in A in the following sense.

12
1. Convex Analysis
Lemma 1.4. Let A be convex and x E A. Then
A-x A-x
/3 c 0:
whenever 0 < ex < {3.
Proof. Let 0 < ex < {3 and YEA. Set z == (1- ex/(3)x + (ex/{3)y. Obviously
z E A and
y - x == {3-1 ( /3 Z _ /3 x + x _ X ) = Z - x E A - x .
(3 ex ex ex ex
The lemma is proved. D
It is possible to give an equivalent definition of the tangent cone using
the distance function.
Proposition 1.3. Let A be a convex set and x E A. Then
T(x,A) == {v Ilim,\-ld(x+'\v,A) == O}.
,x!0
Proof. Let v E T(x, A). Then for any E > 0 there exists 8 > 0 such that
d(v, (A - x)/8) < E. By Lemma 1.4 we have
,\-ld(x + '\v, A) == d(v, (A - x)/'\) < d(v, (A - x)/8) < E
for all ,\ E]O, 8[, and therefore
(1.8) lim'\ -ld(x + '\v, A) == o.
,x!0
If a vector v satisfies (1.8), then, given E > 0, there exists ,\ > 0 such
that d(v, (A - x)/'\) < E, that is,
U A-x
vEcl A '
,x>o
The lemma is proved.
D
The polar cone of the tangent cone T(x, A) is denoted by N(x, A) and
IS called the normal cone to A at x E A. In other words N(x, A) ==
-(T(x, A))*.
Proposition 1.4. Let A be a convex set and let x E A. Then
N(x,A) == {v* I (v*,y - x) < 0, Vy E A}.
Proof. Since A - x c T(x, A), we have
(1.9) (v*, y - x) < 0
for all yEA and v* E N(x, A).

1.2. Convex functions
13
Suppose that v* satisfies (1.9) for all YEA. Consider a vector v E
T(x, A). Given E > 0, there exist yEA and A > 0 satisfying I (y-x)/ A -vi <
E. Note that
(v*,v) == (v*,v - (y - X)/A + (y - X)/A)
< (v*,v - (y - X)/A) < Elv*l.
Since E is an arbitrary positive number, we obtain v* E N(x, A). 0
1.2. Convex functions
Let f be a function whose values are real or +00 and whose domain is R n .
The effective domain of a function f, which is denoted domf, is the set
defined by
domf == {x E R n I f(x) < +oo}.
The set
epif == {(X, a) E R n x R I f(x) < a}
is called the epigraph of f.
It should be mentioned that the epigraph of f completely determines
the function. Indeed,
f(x) == inf{ a I (x, a) E epif}.
Thus fUl1ctions defined on Rn are related to sets in Rn+1, and this corre-
spondence makes it possible to study the functions via sets.
Now let us define convex functions. We define f to be a convex function
on R n if epif is a convex set. Obviously the domain of a convex function is
a convex set. It is easy to check that the function f is C011vex if and only if
(1.10)
f( AX 1 + (1 - A)X2) < Af(X1) + (1 - A)f(X2)
for all Xl, X2 E dOlnf and A E [0,1]. By induction we see that f is convex if
and only if
f(A1 X 1 + . . . + AmXm) < A1f(X1) + . . . + Amf(xm)
for all Xl, . .. , X m E domf and for all Ai > 0, i == 1, m , satisfying Al + . . . +
Am == 1.
Examples of convex functions. Note that the supremum
f(x) == SUp{fi(X) liE I}
of a family of convex functions is a convex function. Indeed, the epigraph of
f is the intersection of epigraphs of the convex functions fi and consequently
is a convex set. From (1.10) we see that the sum of finitely many convex
functions is also a convex function.

14
1. Convex Analysis
Now consider a few examples of convex functions. Let A c R n be a
convex set. The function
S(x*, A) == sup{ (x, x*) I x E A}
is called the support function of A. Since the support function is a supremum
of linear functions, it is convex.
The support function allows us to express the inclusion x E A in an
analytical form.
Theorem 1.7. Let A be a convex closed set. Then x E A if and only if
(1.11)
(x, x*) < S(x*, A)
for all x* ERn.
Proof. If x E A and x* E R n , then by the definition of the support function
we obtain (1.11). Suppose Xo satisfies (1.11) for all x* E R n and Xo f/. A.
By Theorem 1.3 there exist x* and E > 0 such that
(x, x*) < (xo, x*) - E
whenever x E A. This inequality implies
S(x*, A) < (xo, x*) - E,
a contradiction.
D
The following theorem establishes a connection between the support
function and the distance function.
Theorem 1.8. Let A be a convex set. Then
d(x, A) == sup{ (x, x*) - S(x* , A) I x* E Bn}.
Proof. Fix arbitrary x E Rn and denote d(x, A) by p. Obviously x E
cl(A + pBn). By Theorem 1.7 we have
(x, x*) < S(x*, cl(A + pBn)) == S(x*, A) + P
for all x* or equivalently
p > (x, x*) - S(x*, A), Vx* E Bn.
Suppose that
p > sup{ (x, x*) - S(x*, A) I x* E Bn}.
Set a == n(x, cIA) and x* == (x - a)/Ix - al. Since S(x*, A) == (a, x*) (see
the remark after Theorem 1.3), we have d(x, A) == p > (x, x*) - (a, x*) ==
Ix - al == d(x, A), a contradiction. D

1.2. Convex functions
15
Corollary 1.2. Let KeRn be a convex cone, and let 0'. > O. The conjugate
cone of the cone
K 1 == {(x,xo) ERn x R I xo > O'.d(x,K)}
has the form
K; == {(x*,xo*) E R n x R I xo* > 0, x* E O'.xo*(B n n K*)}.
Proof. Let (x*, xo*) E R n x R be such that xo* > 0 and x* E O'.xo* (Bn n K* ) .
Consider any point (x, xO) E K 1 . Invoking Theorem 1.8, we get
(x*, x) + xO*xo > (x*, x) + O'.xo*d(x, K)
(1.12)
== (x*, x) + O'.xo* sup [(y*, x) - S(y*, K)] ==
y*EB n
sup [(x* + O'.xo*y*, x)].
y*EBnn-K*
Since y* == -x*j(O'.xo*) E Bn n -K*, we see that the right side of (1.12) is
nonnegative, that is, (x*, xo*) E Ki.
Now assume that (x*, xo*) E Ki. Then for any (x, xO) E K 1 we have
(x*, x) + xO*xo > O. Putting x == 0, we conclude that xo* > O. Let x E R n .
Setting xO == O'.d(x, K) and applying Theorem 1.8, we get
o < (x*, x) + O'.xo* d(x, K)
== sup [(x*, x) + O'.xo* (x, y*) - O'.S(y* , K)] == sup [(x* + O'.xOy* , x)].
y*EB n y*EBnn-I<*
In other words, we have
(-x*, x) < S(x, O'.xo*(B n n - K*))
for any x ERn. Froln Theorem 1.7 we obtain x* E O'.xo*(B n n K*). This
completes the proof. D
The support function of a compact set is Lipschitzian.
Proposition 1.5. Let A c R n be a closed convex set satisfying A c mBn,
where m > O. Then
IS(x, A) - S(x;, A)I < mlx - x;1
for all xi E R n and x2 ERn.
Proof. Let xi E R n and x2 E R n . Since the set A is closed convex and
bounded, there exists Xl E A satisfying S(xi,A) == (xi,x1). We have
S(x, A) - S(x;, A) == (x, Xl) - S(x;, A)
< (x, Xl) - (x;, Xl) < mlx - x;l.
This ends the proof.
D

16
1. Convex Analysis
Remark. From this result we see that the support function of a convex
compact set satisfies the Lipschitz condition. Hence it is sufficient to define
it on al1 arbitrary dense countable set in Rn. Since the support function is
positively homogeneous, that is,
S(AX*, A) == AS(X*, A), A > 0,
we see that it can be defined on a countable set dense in bdBn.
The indicator function of a set A is defined by
8(x,A) == { 0, f x E A,
+00, If x (j. A.
The indicator function of a convex set is obviously convex.
Let A c R n be a convex set such that 0 E A. The function
p,(x, A) == inf{a > 0 I a-Ix E A}
is called the Minkowski function of A. Assume that 0 E intA; then for any
x E A we have a-Ix E A, if a is sufficiel1tly large. Hence p,(x, A) is always
a finite number. The convexity of the Minkowski function is a consequence
of the following result.
Proposition 1.6. Let A c Rn be a convex set such that 0 E intA. Then
1. p,(AX, A) == AJ-l(X, A) for all A > 0;
2. p,(x, A) < 1, if x E A, and p,(x, A) > 1, if x (j. A;
3. p,(x + y, A) < J-l(x, A) + p,(y, A) for all x, y E R n .
Proof. 1. By definition we have
p,(Ax,A) == inf{a > 0 I a-lAx E A} == inf{A,8I,8 > 0, ,8-Ix E A}
== Ainf{,8 > 0 I ,8-1x E A} == Ap,(x,A).
2. If x E A, then obviously p,(x, A) < 1. Let x (j. A. If p,(x, A) < 1, then
there exists a < 1 satisfying a-Ix E A. Since 0 E A and A is convex, we
have
x == (1 - a)O + a(a-Ix) E A.
We obtain a cOl1tradiction. Hence p,(x, A) > 1.
3. Let E > 0 and  == p,(x, A) + p,(y, A) + E. There exist numbers a and
,8 satisfying  == a +,8, a > p,(x, A), ,8 > J-l(Y, A). Froln 1 and 2 we obtain
a-Ix E A and ,8-Iy E A. Consequently
x + y x + y a -1 ,8 ,8 -1 A
a x+ yE .
 a+,8 a+,8 a+,8
Thus we have
p,(x + y, A) <  == p,(x, A) + p,(y, A) + E.

1.2. Convex functions
17
Si11ce E > 0 is arbitrary, we obtain the result.
D
One Inore exaInple of a convex function is given by the following propo-
sition.
Proposition 1.7. Let f : R n x R m  R U { +oo} be a convex function, and
let G c R n x R m be a convex set. Then the function
cjJ(x) == inf f(x, y),
yEG(x)
where G(x) == {y E R m I (x,y) E G}, is convex.
Proof. Let Xl E dOIncjJ, X2 E dOIncjJ, and E > o. There exist Y1 E G(X1)
and Y2 E G(X2) satisfying f(X1, Y1) < cjJ(X1) + E and f(X2, Y2) < cjJ(X2) + E,
respectively. Let a E [0,1]. Since aY1 + (1 - a)Y2 E G(ax1 + (1 - a)x2), we
have
cjJ(aX1 + (1 - a)x2) == inf f(ax1 + (1 - a)x2, y)
yEG(axl +(1-a)x2)
< f(ax1 + (1 - a)x2, aY1 + (1 - a)Y2)
< a f ( Xl, Y1) + (1 - a) f ( X 2, Y2) < a cjJ ( Xl) + (1 - a) cjJ ( X2) + E.
Since E > 0 is arbitrary, we obtain the result. D
Froln Proposition 1.7 we see, for example, t11at the distance function
x  d(x, C) is convex if CeRn is convex.
Continuity properties. The following theoreln shows that COI1vex fUI1C-
tions are locally Lipschitzial1.
Theorem 1.9. Let f : R n  RU {+oo} be a convex function. Assume that
f(x) < b for all x E Xo + EBn, where E > o. Then f is Lipschitzian in a
neighborhood of xo.
Proof. Let us show that f is bounded from below on t11e set Xo + EBn. Let
z E Xo + EBn. There exists z' E Xo + EBn such t11at Xo == ! (z + z'). Since f
is convex we have
1 1 ( ' )
J(xo) < 2 J(z) + 2 J z ·
Consequently
f(z) > 2f(xo) - f(z') > 2f(xo) - b.
Set 8 == E/2, c == Inax{2f(xo) - b, b}. Then If(z)1 < c when z E Xo + 28Bn.
Let Xl E Xo + 8Bn and X2 E Xo + 8Bn. Put a == I X 1 - x21 and X3 ==
X2 + (8/a)(x2 - Xl). Obviously X3 E Xo + 28Bn. Since
8 a
X2 == 8 X1 + 8 X3 ,
a+ a+

18
1. Convex Analysis
we have
8 a
!(X2) < 8 !(XI) + 8 !(X3)'
a+ a+
Consequently
a a
!(X2) - !(XI) < a + 8 (J(X3) - !(xr)) < -;51!(X3) - !(xI)1
2ca 2c
< T = Tl x 2 -xII.
Replacing Xl by X2 and X2 by Xl and repeating our reasoning, we obtain the
result. 0
Corollary 1.3. Assume that the effective domain of a convex function f
coincides with R n . Then f is locally Lipschitzian (satisfies the Lipschitz
condition on a neighborhood of each point).
Proof. Let Xo E R n . There exists a simplex L; == co{ Xl, . . . X n +1} such
that Xo E intL;. Let X E L;. Thel1 there exist positive numbers AI, . .. , An+1
satisfying Al + . . . + An+1 == 1 and X == A1X1 + . . . + An+1Xn+1. Hence
f(x) < A1f(X1) + . . . + A n +1f(x n +1) < l11ax f(Xi).
1,n+1
Thus f is bounded on a neighborhood of x. Applying Theorelu 1.9 we obtain
the result. 0
Function f : R n  R U { +oo} is said to be lower semi-continuous at a
point Xo if
liminf f(x) > f(xo).
xxo
The function f is said to be lower semi-continuous if it is lower seml-
continuous at any point X E dOluf.
Function f : R n  R U { +oo} is said to be upper semi-continuous at a
point Xo if
limsupf(x) < f(xo).
xxo
The function f is said to be upper semi-col1tinuous if it is upper semi-
continuous at al1Y point X E domf.
Theorem 1.10. The following conditions are equivalent:
1. The epigraph of f is a closed set.
2. The set Co: == {x I f(x) < a} is closed for every a E R.
3. f is lower semi-continuous.

1.2. Convex functions
19
Proof. Let epif be closed. Consider a sequence Xk  Xo satisfying f(Xk) <
a. Without loss of generality f(Xk)  /l < a, where /l is real or -00. If /l
is a finite number, then (Xk, f(Xk))  (xo, /l) E epif. Hel1ce f(xo) < /l < a.
This implies Xo E Ca.
Now let us show that /l == -00 hnplies f(xo) == -00. Indeed, if f(xo) ==
/lo is finite, then f(Xk) < /lo-E for k sufficiently large. COl1sider the sequence
(Xk, /lo - E) E epif. Its limit (xo, /lo - E) obviously belongs to epif. Hel1ce
/lo == f(xo) < /lo - E. We obtain a contradiction. Hel1ce, if /l == -00, then
f(xo) == -00; that is, Xo E Ca. Thus C a is closed.
Suppose that the second condition is satisfied. Let Xk  Xo and f(Xk) 
a. Let E > O. Then f(Xk) < a + E when k is sufficiel1tly large. Since C a + E
is closed, we obtain f(xo) < a + E. Hence f is lower sen1i-continuous.
Finally let Condition 3 be satisfied. If (Xk, ak) E epif and (Xk, ak) 
(xo, aD), thel1
f(xo) < Ihninf f(Xk) < aD;
k-+oo
that is, (xo, aD) E epif. Thus epif is closed. The t11eorem is proved. D
Note that from the second condition follows imlnediately the lower semi-
continuity of the indicator function of a closed set.
Conjugate functions. The function
f*(x*) ==sup((x,x*) - f(x))
x
is called the conjugate of f. By definition we obtain
f(x) + f*(x*) > (x, x*)
for all x and x*. This relation is known as Fenchel's inequality.
Examples. COl1sider two examples. Let A c R n be a convex closed set.
Put f(x) == 8(x, A). Then we have
f*(x*) == sup[(x, x*) - 8(x, A)] == sup(x, x*) == S(x*, A).
x xEA
Now let us fil1d the conjugate function of f*(x*) == S(x*,A). From
Theorem 1.8 we have
f** (x) == sup [(x, x*) - S(x*, A)]
x* ERn
== SUp a sup [(x, x*) - S(x*, A)] == sup ad(x, A) == 8(x, A).
aO x*EB n aO

20
1. Convex Analysis
Note that the epigraph of t11e conjugate function is an intersection of
closed half-spaces. lIence the conjugate function is convex and lower semi-
continuous.
Froln Fenchel's inequality we obtain
(1.13)
f**(x) == sup((x*, x) - f*(x*)) < f(x).
x*
We already know that £5**(x, A) == £5(x, A) whenever A c R n is a convex
closed set. It turns out that for any convex lower selni-continuous functions
we have f** == f.
We need the following auxiliary result.
Lemma 1.5. Let f : R n ---t R U {+oo} be a convex lower semi-continuous
function with nonempty effective domain. Then dOInf* =I=- 0.
Proof. Let Xo E domf. Then (xo, f(xo) - 1) tj. epif. By Theorem 1.3 there
exists a vector (y*, (3) E Rn+1 such that
sup ((y*, x) + f3Q) < (y*, xo) + f3(f(xo) - 1).
(x,a)Eepif
Obviously f3 =I=- O. On the other hand, if f3 > 0, then the supremum in the
left side is equal to +00. Hence f3 < O. Dividing by 1f31 we obtain
f*(y* 11(31) == sup ((y* 11f31, x) - f(x))
xEdolnf
< (y* 11f31, xo) - (f(xo) - 1) < +00.
The lemn1a is proved.
D
Now we are in a position to establish the Inain result of the theory of
conjugate convex functions.
Theorem 1.11 (FeI1chel-lVloreau). Let f : R n ---t R U {+oo} be a convex
lower semi-continuous function. Then f** == f.
Proof. If f(x) - +00, the equality f** == f is evident. Therefore we suppose
that dOInf =I=- 0. Suppose that f**(xo) < f(xo), where Xo E domf**. Let us
show that
(1.14 )
((xo, f**(xo)) + EB n + 1 ) n epif == 0
if only E > 0 is sufficiently sInal!. Indeed, in the opposite case there exists a
sequence (Xk, Qk) E epif such that (Xk, Qk) ---t (xo, f**(xo)). Since f is lower
semi-continuous, we have
f** (xo) < f(xo) < liln inf f (Xk) < f** (xo),
koo
a contradiction. Thus (1.14) is satisfied if E > 0 is sufficiently sn1all.

1.2. Convex functions
21
Si11ce the epigraph of f is closed, by Theoren1 1.6 there exists a vector
(y* , (3) E Rn x R such that
(1.15) (y*,xo) + (3f**(xo) > sup{(y*,y) + (3a I (y,a) E epif}.
In other words the point (xo, f** (xo)) can be separated fro111 the set epif
by a hyperplane H. Observe that if {3 > 0, then the supre111un1 in the right
side of (1.15) is equal to +00. Thus {3 < O. Show that the hyperplane H is
not vertical, or equivalently, that {3 < O. Suppose that {3 == O. Froll1 (1.15)
we have
(y*, xo) > sup{ (y*, y) lyE dOlnf}.
Taking z* E domf* (dolnf* =I 0 due to Lelnn1a 1.5) and t > 0, we obtain
f*(z* + ty*) == sup{(z* + ty*,y) - f(y) lyE d0111f}
< sup{ (z*, y) - f(y) lyE domf} + t SlIp { (y*, y) lyE dOll1f}
== f*(z*) +tsup{(y*,y) lyE dOlnf}.
Combining this inequality with the previous one, we get
f**(xo) > (z* + ty*, xo) - f*(z* + ty*)
> (z*, xo) - f* (z*) + t( (y*, xo) - sup{ (y*, y) lyE dOlnf})  +00
when t  00. Thus Xo tj. domf**, a contradiction.
Thus {3 < O. Set x* == y* /1{31. Dividing (1.15) by 1{31, we obtain
(x*, xo) - f**(xo) > sup{ (x*, y) - a I (y, a) E epif} == f*(x*);
that is,
(x*, xo) > f**(xo) + f*(x*).
This contradicts Fenchel's inequality. Thus f** (xo) > f (xo). C0111bining
this with inequality (1.13), we obtain t11e result. D
Corollary 1.4. Let f : Rn  R U {+oo} be a convex function. Assume
that f is continuous at a point Xo E dOlnf. Then f** (xo) == f (xo) .
Proof. Consider the function cjJ : Rn  R U {+oo} defined by epicjJ ==
cl epif. The fU11ction cjJ is convex and lower selni-continuous. Observe
that cjJ(x) < f(x) for all x E Rn and cjJ(xo) == f(xo). We obviously have
cjJ*(x*) > f*(x*), x* E Rn , and hence cjJ**(x) < f**(x) for all x ERn. Ap-
plyi11g Theorem 1.11 and taking account of (1.13), we get f(xo) == cjJ(xo) ==
cjJ**(xo) < f**(xo) < f(xo). D
Corollary 1.5. If KeRn is a closed convex cone, then K** == K.

22
1. Convex Analysis
Proof. Set f(x) == fJ(x, K). Then we have
f*(x*) == sup((x*, x) - fJ(x, K))
x
Hence f**(x)
fJ(x, K**).
== sup{(x*,x) I x E K} == fJ(x*, -K*).
fJ(x, K**). By the Fenchel-Moreau theorem fJ(x, K)
o
Let A : R n  R m be a linear operator. Let Y c R m . By A -l(y) we
denote the set {x E R n I Ax E Y}.
Corollary 1.6. Let K c R m be a convex closed cone, and let A : Rn  R m
be a linear operator. Assume that AR n - K == R m . Then (A -1 K)* == A * K* .
Proof. Let x* E A * K* and Ax E K. Then there exists y* E K* such that
x* == A *y*. Since
(x*, x) == (A *y*, x) == (y*, Ax) > 0,
we have
A*K* c (A- 1 K)*.
Now let us prove the opposite inclusion. Let x E (A * K*)*. Then for all
y* E K* we have
o < (A * y* , x) == (y*, Ax),
that is, Ax E K**. From Corollary 1.5 we obtai!1 Ax E K. Thus (A* K*)* c
A-I K. This implies
(A -1 K)* c (A * K*)**.
In a view of Corollary 1.5 it remains to prove that the cone A * K* is
closed. Consider a sequence Yk E K* such that A *Yk  x*. We claim that
the sequence is bounded. Indeed, suppose it is not true. Let {Yk } be a
p
subsequence which tends to i!1finity, and let v E R m . There exist x E R n
and Y E K such that v == Ax - y. We have
(Ykp/IYkpl,v) == (Ykp/IYkpl,Ax - y) < (A*Ykp/IYkpl,x).
Without loss of generality Yk /IYk I  b E bdBn. Taking the limit, we
p p
get (b, v) < O. Since v E R m is an arbitrary vector, we obtain b == 0, a
contradiction. Thus the sequence {Yk} is bounded. Let Yk be a convergent
p
subsequence. Obviously Yk  y* E K* and A *y* == x*. Thus the cone
p
A * K* is closed. This ends the proof. 0
Corollary 1.7. Let K 1 and K 2 be closed convex cones in Rn. Then we
have
(K1 n K 2 )* == cl(K; + K 2 ).

1.2. Convex functions
23
Proof. Applying Proposition 1.2 and Corollary 1.5 we obtain
(K 1 n K 2 )* == (K;* n K*)* == ((K; + K)*)*
== (K; + K)** == cl(K; + K).
This ends the proof.
D
Corollary 1.8. Let
K == {x I (xi, x) > 0, i == 1,N }.
Then
N
K* = {x* I x* = LC\:iX;, C\:i > 0, i = 1,N }.
i==l
Proof. From Corollary 1.7 we have
N
K* = cl{x* I x* = LC\:iXi, C\:i > 0, i = 1,N }.
i==l
Show that the cone
N
Q = {x* I x* = LC\:iXi, C\:i > 0, i = 1,N }
i==l
is closed. Set 1_ == {i I xi E -Q} and 1+ == {i I xi (j. -Q}. Obvi-
ously the cone Q- == {x* I x* == EiEI _ aixi, ai > 0, i E I_} is a sub-
space. Indeed, for each i E I_there are f3'fn > 0, m E 1_, such that
xi == - EmEI- f3'fnx:n. Therefore for any vector EiEI- aixi E Q- we have
- EiEL C\:ixi = ErnEL (EiEL C\:i/ 3 :n) x:n E Q-.
Let yZ == E f 1 aikxi, aik > 0, i == 1, N , k == 1,2, . . . ; and let limkoo Yk
== Xo =F O. Denote by zZ the vector EiEI_ aikxi E Q-. Put ak == EiEI+ aik+
IzZI. Without loss of generality ak =F 0, aik/ak ---7 ai > 0, and zZ/ak ---7 z* E
Q- as k ---7 00. Obviously EiEI+ ai + Iz*1 == 1. Show that the seque11ce
ak is bounded. Suppose the contrary. Without loss of generality ak ---7 00.
Passing to the limit in the equality yZ/ak == EiEI+ (aik/ak)xi+zZ/ak, we get
o == EiEI+ aixi + z*. From the definition of 1+ we see that ai == 0, i E 1+.
Therefore z* == 0, a contradiction. Thus the sequence ak is bounded. Since
Y'k = ak (EiEI+ (C\:ik/ ak)xi + z'k/ ak ), we see that ak ---t a > O. Taking the
limit, we get x(j = a (EiEh C\:ixi + z*) E Q. D

24
1. COl1vex Analysis
1.3. Differential properties of convex
functions
No,v we proceed to study differel1tial properties of C011vex functions.
Directional derivative. Let f : R n  R be a function. T11e directional
derivative of f at x with respect to a vector v is defined to be t11e lilnit
Df( )( ) 1 . f(x + AV) - f(x)
x v == un \ '
A!O /\
if it exists (+00 and -00 beiI1g allowed as liInits).
We l1eed t11e following technical.lelnn1a.
Lemma 1.6. Let f : R  R U {+oo} be a convex function. If AO < Al <
A2, AO, AI, A2 E don1f, then
f(Al) - f(AO) f(A2) - f(AO)
< .
Al - AO - A2 - AO
Proof. Set Q == (AI - AO)/(A2 - AO). Then we have
Al - AO A2 - Al
f(>\l) = f(a)..2 + (1 - a)..o) <).. ).. f()..2) + ).. ).. f()..o).
2- 0 2- 0
SubtractiI1g f(AO) and dividing by Al - AO we obtain (1.16). 0
(1.16)
Convex fUl1ctions are not differentiable in the usual sense in general.
Nevertheless the directional derivative (finite or infinite) always exists.
Theorem 1.12. Let f : R n  R U {+oo} be a convex function, and let
x E domf. Then Df(x)(v) exists for any v ERn. Moreover, Df(x)(v) is a
positively homogeneous convex function of v.
Proof. If x + AV (j. dOlnf for all A > 0, then Df(x)(v) == +00. If x + AV E
dOlnf when A > 0 is sufficiel1tly slnall, then by Lelnn1a 1.6 the function
\ f(X+AV)-f(x) \
/\ A ' /\>0
is nondecreasing. Hence the liInit always exists.
Let Q > O. Then we have
Df( )( ) 1 . f(x + AQV) - f(x)
X QV == In1
A!O A
= a lim f (x + )..av) - f (x) = aD f (x )( v ).
A!O ACt

1.3. Differential properties of convex functions
25
If a E [0,1], then we have
f(x + A( av 1 + (1 - a)v2)) - f(x)
A
f(a(x + AV1) + (1 - a)(x + AV2)) - af(x) - (1 - a)f(x)
A
f(x + AV1) - f(x) ( 1 ) f(x + AV2) - f(x)
< a A + -a A .
Taking the lin1it as A 1 0 we obtain
Df(x)(av1 + (1- a)v2) < aDf(x)(V1) + (1 - a)Df(x)(V2)'
This ends tl1e proof.
D
Subdifferential. Let f : R n  R U { +oo} be a convex function. The set
8f(x) == {x* E R n I f(y) - f(x) > (x*,y - x), Vy ERn}
is called the subdifferential of f at x. Obviously 8f(x) is a closed C011vex
set. In genera], the subdifferentiallnay be en1pty. If 8f(x) is not empty,
then f is said to be subdifferentiable at x.
The subdifferential is a replacelnent for the usual gradie11t a11d plays an
ilnportant role in optin1ization. Indeed, froln the definition of the subdif-
ferential it follows that a convex fU11ction f achieves its Ininimal value at a
point x if and only if 0 E 8f(x).
Examples. Consider a few exalnples. Let f(x) == Ixl. Then
8 f( ) { Bn, x == 0,
x == x/lxi, x =I O.
If f(x) == 8(x, A), where A is a convex set, and x E A, then 8f(x) == N(x, A).
Set f(x) == S(x, A). Then
8f(x) == {x* E A I (x*,x) == f(x)}.
Now consider an exan1ple of a convex function which is not sllbdifferen-
tiable everywhere it1 dOlnf. Set
f(x) == { - V I - x 2 , x E [-1,1],
+00, x  [-1,1].
It is easy to check that 8f(l) == 8f(-I) == 0.
The following result shows a local nature of the subdifferential.
Theorem 1.13. Let f be a convex function, and let x E domf. Then
8 f(x) == 8v D f(x) (0),
where 8v stands for the subdifferential of the function v  Df(x)(v).

26
1. Convex Analysis
Proof. Let x* E 8f(x), a11d let A > o. Then we have
f(x + AV) - f(x) > A(X*,V)
for all v. Dividing by A and taking tl1e limit, we obtain Df(x)(v) > (x*v)
for all v. Hence 8f(x) C 8v D f(x)(0).
Let x* E 8vDf(x)(0). Then Df(x)(v) > (x*v) for all v. Let A E]O,l[.
Invoking Lemlna 1.6, we obtain
f(y) - f(x) = f(x + (y - x)) - f(x) > f(x + A(Y  x)) - f(x)
> Df(x)(y-x) > (x*,y-x)
for all y ERn.
D
For differentiable convex functions the subdifferential coincides with the
usual gradient.
Theorem 1.14. Assume that a convex function f is differentiable at x.
Then
8f(x) == {\7j(x)}.
Proof. Since 8f(x) == 8vDf(x)(0) and Df(x)(v) == (\7f(x),v), we get
\7 f(x) E 8vD f(x) (v).
Let x* E 8vD f(x)(O) be such that x* =I- \7 f(x). Then we have
(x*, v) < D f(x)(v) == (\7 f(x), v).
Hence
(\7f(x) - x*,v) > 0
for all v E R n . Therefore \7 f (x) == x*.
D
The following result has many applicatio11s.
Theorem 1.15. Let f be a convex positively homogeneous function with
domf == R n . Then
f(x) == S(x, 8f(0)).
Proof. Since f is positively homoge11eous and continuous, we have f(O) == O.
By definition
8f(0) == {x* I f(x) > (x*,x), \Ix}.
Hence
f(x) > S(x,8f(0)).
To prove the inverse inequality suppose that there exists x satisfying
f(x) > S(x, 8f(0)).

1.3. Differential properties of convex functions
27
Since (x, 8(x, 8f(0)))  epif and epif is a closed set, by Theorem 1.3
there exists a nonzero vector (y*, a*) E Rn x R such that
(1.17) a*a+ (y*,x) < a*sup{(x*,x) I x* E 8f(0)} + (y*,x)
for all (x, a) E epif. If a* > 0, then the left side of (1.17) can be made
arbitrarily large. Therefore a* < o. If a* == 0, then we have y* == 0, since x
is an arbitrary vector. Hence a* < O. Without loss of generality a* == -1.
Thus we have
-a + (y*,x) < (3 == -sup{(x*,x) I x* E 8f(0)} + (y*,x)
for all (x, a) E epif. Let x E domf, and let A > O. Then we have
- f(AX) + (y*, AX) < (3.
Since f is positively homogeneous, dividing by A, we obtain
-f(x) + (y*,x) < (3/A.
Taking the limit as A  00 we obtain y* E 8f(0). Let x == 0 and a == O.
Then from (1.1 7) we have
sup{ (x*, x) I x* E 8f(0)} < (y*, x).
We achieve a contradiction and the end of the proof.
o
Corollary 1.9. Let f be a convex function, and let x E int domf. Then
Df(x)(v) == 8(v,8f(x)).
Proof. From Theorems 1.13 and 1.15 we obtain
D f ( x ) ( v) == 8 ( v, 8v D f ( x ) ( 0 )) == 8 ( v, 8 f ( x ) ) .
This ends the proof.
o
Corollary 1.10. Let f be a convex function, and let domf == Rn. Then f
is subdifferentiable and the set 8 f (x) is bounded.
Proof. By Corollary 1.3 the function f is locally Lipschitzian. Consequently
its directional derivative is locally bounded. From Corollary 1.9 it follows
that 8f(x) =I- 0 and the set 8f(x) is bounded. 0
Corollary 1.11. Let f be a convex function, and let domf == Rn. Then
the function (x, v)  D f(x)(v) is upper semi-continuous.
Proof. Consider a point (xa, va) E R n x R n . Let a sequence (Xi, Vi) 
(xa, va) be such that
limsup Df(x)(v) == ,lim Df(Xi)(Vi).
(x,v)---+ (xo, vo) ---+oo
By Corollary 1.10 all sets 8f(Xi) are contained in a ball of sufficiently large
radi us. Therefore there are points xi E 8 f (Xi) , i == 1, 2, . . ., such that

28
1. Convex Analysis
(Vi, Xi) == S(vi,8f(Xi)). The sequence {xi} contains a convergent subse-
quence. Witllout loss of generality xi  xo. By definitioll of subdifferential
we have
f(y) - f(Xi) > (xi, y - Xi), \I Y ERn.
Taking the limit, we obtain XC) E 8f(xo). Applying Corollary 1.9, we get
.Jim D f (Xi) ( Vi) == .lhn S ( Vi, 8 f (Xi)) == ,lhn (Vi, xi)
 ---+ 00  ---+ 00  ---+ 00
== (vo, xC)) < S ( vo, 8 f (xo)) == D f (xo) ( vo).
This ellds the proof. '
D
Subdifferential calculus. Some results froln the ordillary differential cal-
culus call be extended into a subdifferential calculus.
Let f : R n  R U {+oo} be a convex function. Then from the definition
of subdifferential we have
8(af)(x) == a8f(x), \I a > O.
Now cOllsider two convex functions f1 : R n  R U {+oo} and f2 : R n 
R U {+oo}. It is easy to check that
8(f1 + f2)(X)  8f1(X) + 8f2(X), \I X ERn.
The equality in this formula is not valid in general. However, under SOlne
additional assumptions it holds.
Theorem 1.16 (Moreau-Rockafellar). Let f1 : R n  R U {+oo} and f2 :
R n  R U {+oo} be convex functions. Assume that there is a point x E
domf1 n domf2 and that f1 is continuous at x. Then the equality
8(f1 + f2)(X) == 8f1(X) + 8f2(X)
holds at any point x E dOlnf1 n domf2.
Proof. It suffices to prove that
8(f1 + f2)(X) C 8f1(X) + 8f2(X), \I x ERn,
holds at any point x E domf1 n domf2. Put f == f1 + f2. Let x E domf1 n
domf2, and let x* E 8 f (x). Consider the sets
Al == {(x,fL) ERn x R I fL> f1(X) - f1(X)}
and
A 2 == {(X,fL) E R n x R I fL < -f2(X) + f2(5:) + (x*,x - x)}.
Obviously Al =I- 0 and A 2 =I- 0. Since x* E 8f(x), we have Al n A 2 == 0. By
Theorem 1.5 there exists a nonzero vector (z*, fL*) E R n x R such that
(1.18) (Xl, z*) + fL1fL* < (X2, z*) + fL2fL*

1.3. Differential properties of convex functions
29
for all (Xl, J.-l1) E Al and (X2, J.-l2) E A 2 . Since J.-l1 can be arbitrarily large, we
conclude that J.-l* < O. Show that J.-l* < O. Indeed, if J.-l* == 0, then we have
(1.19)
(Xl, z*) < (X2, z*)
for all Xl E domf1 and X2 E dOlnf2. Set X2 == x. Since f1 is continuous at X,
we have x + EBn C domf1 when E > 0 is sufficiently small. Let y E Bn. Put
Xl == X + EY. From (1.19) we have (y, z*) < O. Since y E Bn is an arbitrary
vector, we obtain z* == 0, a contradiction. Thus J.-l* < O. Dividing (1.18) by
IJ.-l* I, we get
(Xl, 1J.-l*1- 1 z*) - J.-l1 < (X2, 1J.-l*1- 1 z*) - J.-l2
for all (X1,J.-l1) E Al and (X2,J.-l2) E A 2 . Hence
(Xl, 1J.-l*1- 1 z*) - f1(X1) + f1(X)
( 1. 20 )
< (X2, 1J.-l*1- 1 z*) + f2(X2) - f2(X) - (x*, X2 - x)
for all Xl E domf1 and X2 E domf2.
Setting X2 == x, from (1.20) we obtain
f1(X1) - f1(X) > (1J.-l*1-1z*,X1 - x), V Xl E domf1,
that is, 1J.-l*1- 1 z* E 8f1(X). Now let Xl == x. Then from (1.20) we get
f2(X2) - f2(X) > (x* -1J.-l*1- 1 z*,x2 - x), V X2 E domf2.
In other words, x* - I J.-l * 1-1 z* E 8 f2 (x). Thus
x* == 1J.-l*1-1z* + (x* -1J.-l*1- 1 z*) E 8f1(X) + 8f2(X).
The theorem is proved.
D
Remark. Note that the convex functions on R n can be identified with their
epigraphs, while the convex sets in R n can be identified with their indicator
functions. These identifications Inake it easy to pass back and forth between
a geometric approach and an analytic approach. For example, here we
considered an analytic approach to differential calculus for convex functions.
However, there is another way. We can define directional derivatives and
subdifferentials and investigate their properties using a geometric approach.
Namely, let f : Rn  R U { +oo} be a convex function and X E domf. It is
easy to prove that
epiD f (x) (-) == T( (x, f (x)), epif)
and
8f(x) == {x* I (x*, -1) E N((x, f(x)), epif)}
(see Propositions 1.3 and 1.4). This idea will be used in Chapter 3, where a
nonsmooth calculus is developed.

30
1. Convex Analysis
1.4. Problems
1. Let f : R n  R be a twice continuously differentiable function, and
let its Hessian matrix \l;xl(x) be positive definite for every x ERn.
Prove that f is strictly convex; that is, f(AX1 + (1- A)X2) < Af(X1) +
(1 - A)f(X2), for all A E]O, 1[ and Xl =I- X2.
2. Let A be a convex set. The set
AO == {x* I (x,x*) < 1, Vx E A}
is called the polar of A. Let A be a closed convex set such that
o E intA. Prove that AOO == A. (Hint. Set f(x) == 8(x, A) and use the
Fenchel-Moreau theorem.)
3. Let A be a convex set. A point X E A is called an extreme point of A
if there is no way to express x as a convex combination AY + (1 - A)Z
such that YEA, Z E A and A E]O, 1[, except by taking Y == Z == x.
The set of all extreme points of a convex set A is denoted by extA.
Let A c R n be a convex cOlnpact set. Prove that A == co extA. (Hint.
Use induction over the dimension of the space.)
4. Let Al and A 2 be compact sets in Rn. The Hausdorff distance be-
tween Al and A 2 is defined by h(A 1 ,A 2 ) == inf{h I Al C A 2 +
hBn, A 2 C Al + hBn}. Verify that h satisfies all axioms of distance.
5. Let Al and A 2 be convex cOlnpacts in R n . The geometric difference
*
of Al and A 2 is defined by Al - A 2 == {x I x + A 2 c AI}. Assulne
*
that 1Bn C Al - A 2 , 1 > o. Show that there exist 8 > 0 and l > 0
such that for all convex compacts A and A satisfying h( AI, A) < d
* *
and h(A 2 , A) < d, the h1equality h(A 1 - A 2 , A - A) < ld holds
whenever d < 8.
6. Let f1 : R n  Rand f2 : R n  R be C011vex continuous at x == x
functions satisfying 11 (x) == f2(X). Show that olnax{f1, f2}(X) ==
CO(Of1(X) U 8f2(X)). (Hint. Use Theorem 1.15.)
7. Let f : R n  R be a function. Show that if f*(x) == f(x) for all
x ERn, then f(x) == IxI2/2.
8. Show that AO == A c R n if and only if A == Bn.
9. Show that (J.L(., A))*(x*) == 8(x*, AO).

Chapter 2
Set- Val ued Analysis
The Inaterial in this chapter is basic to the rest of the text. We present here
t11e fundamental concepts and results associated with set-valued maps. We
study various continuity properties (upper and lower semi-continuity, Lips-
chitzian continuity) of set-valued maps. We then proceed by studying the
selection problen1, that is, the problem of constructing single-valued Inaps
with graphs contained in the graph of a given set-valued map. We also de-
fine set-valued derivatives, which will playa crucial role in the study of the
local structure of set-valued maps. Furthermore, we construct Lipschitzian
approxilnations of upper semi-continuous set-valued maps, which are ex-
tensively used in subsequent chapters. For upper sen1i-continuous maps we
establish an extension property needed in viability theory. We also study
convex processes, which are Inultivalued analogues for linear operators, and
prove a Inultivalued version of the Jordan theorem, which is essential for the
study of controllability, stability and stabilizability of differential inclusions.
2.1. Set-valued maps
In this section we consider the main notions and results concerning set-
valued maps, their continuity and selections.
Let X and Y be two normed spaces. A set-valued map F from X to Y
is a map that associates with any x E X a set F(x) c Y. A set-valued map
F is completely characterized by its graph. The graph is denoted grF and
is defined by
grF == {(x,y) E X x Y lyE F(x)}.
-
31

32
2. Set- Valued Analysis
The set
domF == {x E X I F(x) =I- 0}
is called the domain of F. The image of F is defined as ilnF == {y I 3x E
X : y E F(x)}.
The inverse n1ap F- 1 : Y  X is defined by
F-1(y) == {x E X I (x,y) E grF}.
By FIA we denote the restriction of F to a set A c X.
Examples of set-valued maps. Consider several exalnples of set-valued
lnaps.
Let I : X  Y be a single-valued lnap. Then its inverse 1-1 : Y  X
is usually set-valued.
Let V : X  R U {+oo} be a function. Define the set-valued n1ap
V+(x) == { V(x) + R+, V(x) < +00,
0, V(x) == +00.
Obviously, the domain of V+ coincides with the set of the points x such that
V(x) < 00, and grV+ is the epigraph of V.
The following lnap is widely used in control theory. Let I : X x U  Y
be a single-valued map, where U is a set of parameters. Then we can define
the set-valued map
F(x) = f(x, U) = U f(x, u).
uEU
Continuity. A set-valued map F is called upper semi-continuous at Xo E X
if for any opel1 set !VI containing F(xo) there exists a neighborhood 0 of
Xo such that F(O) c !VI. A set-valued lnap F is said to be upper semi-
continuous if it is so at every point Xo EX.
A set-valued n1ap F is called lower semi-continuous at Xo E X if for any
Yo E F(xo) and any neighborhood O(Yo) of Yo there exists a neighborhood
O(xo) of Xo such that
F(x) n O(Yo) =I- 0
for all x E O(xo). A set-valued lnap F is said to be lower selni-continuous if
it is so at every point Xo EX.
A set-valued map F : X  Y is said to be continuous at Xo E X if it is
both upper and lower selni-continuous at Xo. It is called continuous if it is
continuous at every point x EX.
Let F : X  Y be a set-valued lnap. The norms in X and Yare denoted
by I · Ix and I . Iy respectively. We say that F is Lipschitzian if there exists

2.1. Set-valued maps
33
l > 0 such that
F(XI) C F(X2) + llxI - x2lx B y,
for all Xl E X and X2 E X, where By == {y E Y Ilyl < I}. A set-valued
map F is said to be locally Lipschitzian if for any X E X there exist E > 0
and l > 0 such that
F(XI) C F(X2) + llxI - x21x B y
for all XI,X2 E x+EBx.
Proposition 2.1. Let F : X  Y be an upper semi-continuous map with
closed values. Then grF is closed.
Proof. Consider a sequence (Xi, Yi) E grF converging to a point (xo, YO) E
X x Y. Assume that Yo tt F(xo). Then there exists an open set M containing
F(xo) such that Yo tt clM. Since F is upper semi-continuous there exists
a positive integer i* such that Yi E M for all i > i*. Hence Yo E clM, a
contradiction. D
There is a partial converse to this result.
Proposition 2.2. Assume that grF is closed and the set
M == cl{F(x) Ilx - xol < ()},
where () > 0, is compact. Then F is upper semi-continuous at xo.
Proof. Suppose that there exist a neighborhood N of F(xo) and sequences
{Xi} and {Yi} such that Xi  xo, Yi E F(Xi), and Yi tt N. Since M is
con1pact, the sequence {Yi} contains a convergent subsequence. Without
loss of generality we can assume that Yi  Yo. Since grF is closed, we have
Yo E F(xo). This contradiction proves the proposition. D
From Propositions 2.1 and 2.2 we obtain the following result.
Proposition 2.3. Let F and G be upper semi-continuous set-valued maps
with closed values. Assume that the set
M == cl{F(x) Ilx - xol < ()},
where () > 0, is compact. Then the set-valued map F(x) n G(x) 'lS upper
semi-continuous at xo.
Continuity properties of parameterized set-valued maps are determined
by corresponding properties of parameterizing functions.
Proposition 2.4. Let U be a compact, and let f : R n x U  R m be a con-
tinuous (Lipschitzian) in X function. Then the set-valued map X  f(x, U)
is continuous (Lipschitzian).

34
2. Set- Valued Analysis
Proof. Obviously the set-valued map x ---t f(x, U) has a closed graph and is
locally bounded. By Proposition 2.2 it is upper semi-continuous. The lower
selni-continuity follows from the continuity of the function x ---t f(x, u) for
every fixed u E U. If the function x ---t f(x, u) is Lipschitzian, then the
set-valued n1ap x ---t f(x, U) is obviously Lipschitzian. 0
An intersection of two lower selni-continuous Inaps is not lower semi-
continuous in general. However, if the maps are COl1vex valued and the
relative interior of the intersectio11 satisfies SOlne additional conditions, the
intersection turns out to be lower selni-continuous.
Proposition 2.5. Let G : R n ---t R m and F : R n ---t R m be lower semi-
continuous maps with convex values. Assume that G is locally bounded and
that there exists "I > 0 such that
F(x) n (G(x) + z) i= 0
for all x E R n and z E "IBm. Then the set-valued map F(x) n G(x) is lower
semi-continuous.
Proof. Let Yo E F(xo) n G(xo), and let E > O. There exists b > 0 such that
G(x) C bBm for each x E Xo + 'f}Bn, where'f} > O. Set a == E(l + 4b/'Y)-1.
Since G and F are lower semi-continuous, there exists 8 E]O, 'f}[ such that
G(x) n (Yo + aB m ) i= 0 and F(x) n (Yo + aB m ) i= 0
whenever x E Xo + 8Bn. Consequently for any x E Xo + 8Bn t11ere exists
g(x) E G(x) satisfying Iyo - g(x)1 < a and, hence, such that
(2.1) g(x) E F(x) + 2az,
where Iz/ < 1. By assumption there exist f(x) E F(x) and g(x) E G(x) such
that
(2.2) 'Y Z == f(x) - g(x).
Set () == "1/ ("I + 2a). Then we have 1 - () == 20a/'Y. Multiplying (2.1) by ()
and using (2.2), we obtain
2()a
()g(x) E ()F(x) + -(f(x) - g(x)) c ()F(x) + (1 - ())F(x) - (1 - ())g(x).
"I
Since F(x) and G(x) are convex, we have
()g(x) + (1 - ())g(x) E F(x) n G(x).
From the inequality
/()g(x) + (1 - ())g(x) - YO/ < ()Iyo - g(x)/ + (1 - ())/YO - g(x)/
< a() + 2b(1 - ()) < a(l + 4b/'Y) == E
it follows t11at F( x) n G (x) is lower semi-continuous at xo. 0

2.1. Set-valued maps
35
The selection problem. Consider a set-valued map F : R n  Rm. A
single-valued map f : R n  R m satisfying f(x) E F(x) for all x E R n
is called a selection. Below we establish sufficient conditions for the exis-
tence of selections with some regularity properties such as continuity and
measurability.
First we establish properties of the distance function.
Proposition 2.6. Let A, BeRn, and let a > O. Assume that B c A +
aBn. Then
d(x, A) < d(x, B) + a.
Proof. Let x E R n and E > o. There exists yEA such that d(x, A+aBn) >
d(x, y + aBn) - E. Since d(x, y + aB n ) > Ix - yl - a, we have
d(x, B) > d(x, A + aB n ) > Ix - yl- a - E > d(x, A) - a-E.
Since E is an arbitrary positive number, we obtain the result.
o
Lemma 2.1. Let A eRn. Then the function x  d(x, A) is Lipschitzian
with the constant 1.
Proof. Let Xl, X2 E R n , and let E > o. There exists Y1 E A satisfying
I X 1 - Y11 < d(X1, A) + E. We have
d(X2, A) < I X 2 - Y11 < I X 2 - xII + I X 1 - Y11
< I X 2 - xII + d(X1, A) + E.
Exchanging Xl and X2, we obtain
Id(X1, A) - d(X2, A)I < I X 1 - x21 + E.
Since E is an arbitrary positive nUlnber, we obtail1 the result.
o
Lemma 2.2. Let F : R n  Rm be a continuous set-valued map with
domF == R n . Then the function p( x, y) == d(y, F (x)) is continuous.
Proof. By Lelnma 2.1 the function p is Lipschitzian in y. Hence it is
sufficient to prove that p is continuous in x for any fixed y == Yo.
Since p(x, Yo) == d(yo, clF(x)), we can assun1e without loss of generality
that F has closed values. Show that the function p(x) == d(yo, F(x)) is lower
semi-continuous at any Xo E R n . Suppose that it is not true. Then there
exist E > 0 and a sequence Xi  Xo such that
(2.3) p(xo) > p(Xi) + E.
Let points Zi E F(Xi) be such that p(Xi) == Iyo - zil. From (2.3) it follows
that the sequence {Zi} is bounded. Without loss of generality Zi  zoo By
Proposition 2.1 we have Zo E F(xo). From (2.3) we obtain
p(xo) > Iyo - zol + E.

36
2. Set- Valued Analysis
This contradicts the definition of p.
Now let us prove that p(x) is upper semi-continuous at any point Xo E
R n . Suppose that there exist E > 0 and a sequence Xi  Xo such that
(2.4) p(Xi) > p(xo) + E.
Let Zo E F(xo) be such that p(xo) == Iyo - zol. Since F is lower semi-
continuous there exists a sequence {Zi} such that Zi E F(Xi) and IZi - zol <
E/2 for i sufficiently large. Hence we have
p(Xi) < Iyo - zil < Iyo - zol + Izo - zil < Iyo - zol + E/2.
This contradicts (2.4). The lemma is proved. D
Recall the following characterization of measurable functions.
Theorem 2.1 (Lusin). Let A c R n be a compact set. A function f : A 
R m is measurable if and only if for every E > 0 there exists a compact
subset A€ c A such that meas(A \ At:) < E and the restriction of f to At: is
continuous.
If a set-valued map F : R n  R m has closed convex values, then the pro-
jection 7r(Y, F(x)) consists of a unique point (see Lemma 1.1). The following
theorem establishes properties of the projective selection.
Theorem 2.2. Let F : R n  R m be a continuous set-valued map with
closed convex values. If the function 9 : R n  R m is continuous (measur-
able), then the function f(x) == 7r(g(x), F(x)) is continuous (measurable).
Proof. Assume that 9 is continuous and consider the set-valued map G(x) ==
g(x) + d(g(x), F(x))Bm. By Lemma 2.1 and Lemma 2.2 it is continuous.
Obviously f(x) == F(x) n G(x). From Proposition 2.1 the graphs of F and
G are closed. Hence the graph of f is closed as an intersection of closed
graphs. From the definition of G and Proposition 2.2 we see that f is an
upper semi-continuous map. Since f is single-valued, it is a continuous
function.
Now let 9 be measurable. By the Lusin theorem for any E > 0 there exists
a closed set At: C R n such that meas(R n \ At:) < E and the restriction glA€
is continuous. The above arguments imply the continuity of fIA€. Applying
the Lusin theorem again, we see that f is measurable. D
Below we prove the existence of a measurable selection. This statement
is a version of an implicit function theorem.
Theorem 2.3 (Filippov). Let f : R n x R k  R m be a continuous function,
and let v : R n  R m be a measurable function. Assume that U C R k is a

2.2. Derivatives of set-valued maps
37
compact set such that v(x) E f(x, U) for almost all x. Then there exists a
measurable function u : R n  U satisfying v(x) == f(x, u(x)).
Proof. Consider the set-valued Illap
U(x) == {u E U I v(x) == f(x,u)}.
Since f is continuous and U is compact, the map U(x) has compact values.
Select u(x) == (u 1 (x),. . . , uk(x)) E U(x) so that the first cOlllponent u 1 (x)
is the smallest possible. If there is more than one such point u E U (x),
it is required that u 2 (x) be the smallest among these, and further that
u 3 (x) be the smallest among these, and so forth, to define a unique vector
u(x) E U(x). We shall prove that u(x) is measurable, and this need only be
delllonstrated for a compact set A c R n .
Suppose that u 1 (x),. . . , u p - 1 (x) are measurable 011 A (if P == 1, nothing
is assumed here), and prove that uP(x) is measurable 011 A. Let f. > O. By the
Lusin theorem there exists a compact set A E C A such that meas(A \A E ) < f.
and the functions u 1 ( X ), . .. , u p - 1 ( x) and v ( x) are continuous on A E . Let Q
be a real number. Show that the set
AE,a == {x E A E I uP(x) < Q}
is closed. Suppose that AE,a is not closed. Then there exists a sequence
Xi E AE,a such that limioo Xi == X and uP(x) > Q. Since U is compact, the
sequence {u( Xi)} contains a convergent subsequence. Without loss of gen-
erality the sequence U(Xi) itself converges to a vector il E U. By continuity
we have
. . .
,lim V(Xi) == v(x), ,lim UJ(Xi) == uJ(x) == ilJ j == 1,p - 1.
oo oo
Thus we have v(x) == f(5;, il). But ilP < Q < uP(x), which contradicts the
definition of uP(x). Therefore uP(x) is measurable on A E . By the Lusin the-
orem there exists a compact subset A 2E C A E such that Illeas(A E \ A 2E ) < f.
and uP(x) is continuous on A 2E . Since f. is arbitrary, applying Lusin's theo-
rem again, we see that uP(x) is measurable on A. Hence u(x) is measurable
on the entire space R n . 0
2.2. Derivatives of set-valued maps
This section is devoted to a concept of directional derivative of a set-valued
map.

38
2. Set- Valued Analysis
Tangent and contingent cones. First we introduce two cones that are
used to study local properties of sets. Let X be a 110rmed space. The tangent
cone to a set A c X at a point x E A is defi11ed by
T+(x, A) == {v llim A -ld(x + AV, A) == O}.
'x!0
T11e contingent cone to a set A c X at a poh1t x E A is defined by
T_(x,A) == {v Ilhninf A- 1 d(x + Av,A) == O}.
'x!0
It is easy to see that tangent and c011tingent C011es are always closed and
T+(x, A) c T_(x, A).
If A is convex and x E A, then by Lelnma 1.4 we have
Al1d(x + A1V, A) < A 2 1 d(x + A2V, A)
whenever 0 < Al < A2. Hence lim,X!o A- 1 d(x+AV, A) exists for each v. Thus
if A is convex, the11 T+(x, A) coincides with the tangent cone introduced h1
Section 1 for convex sets and T+(x, A) == T_(x, A) for all x E A.
In Propositions 2.7 and 2.8 below we prove two estimates of tangent
cones. We will use them later in the book.
Proposition 2.7. Let A eRn, a > 0, x E A, and let D == {(x,x O ) E
R n x R I x O > ad( x, A) }. Then the inclusion
T+((x, 0), D)  {(v, v o ) E R n x R I v O > ad(v, T+(x, A))}
holds.
Proof. Let v O > ad(v, T+(x, A)) == alv - wi, where w E T+(x, A). Then we
have
ad(x + AV, A) < ad(x + AW, A) + Aalv - wi < ad(x + AW, A) + AVO.
Dividh1g by A and taking the limit, we derive
limA- 1 ad(x + Av,A) < v o .
'x!0
Hence (v, v O ) E T+((x, 0), D).
D
Let U C R k , and let f : R n x U  R m be a function. Assume that f
is differentiable in x and that the set f (x, U) is convex for all x E R n . For
(x, it) E R n x U denote v == f(x, il) and set
C == \7 xf(x, it), K == cone(f(x, U) - v).
The following result contains an estimate for the tangent cone to the graph
of the set-valued Inap x  f(x, U).

2.2. Derivatives of set-valued maps
39
Proposition 2.8. The following inclusion holds:
{(X, v) ERn x R m I v E Cx+K} C T+((x,v),grf(.,U)).
Proof. By Proposition 2.4 the set-valued map x  f(x, U) is Lipschitzian.
The result follows froln Lemlnas 2.3 and 2.4 belovl. D
Lemma 2.3. Let v E Cx + K. Then we have
lim A -ld( v + AV, f(x + AX, U)) == o.
A!O
Proof. Let 1] > O. There exist u E U and (3 > 0 such that Iv - Cx -
(3(f(x, u) -v)1 < 1]. Since the set f(X+AX, U) is convex, for A(3 < 1 we have
d(v + AV, f(x + AX, U)) < d(v + A(CX + (3(f(x, u) - v)), f(x + AX, U))
+A1] < IV+A\7xf(x,it)X+A(3(f(x,u) -v)
- f(x + AX, it) - A(3(f(x + AX, u) - f(x + AX, it))1 + A1]
< If(x, it) + A \7 xf(x, it)x - f(x + AX, it) I + A(3(lf(x, u) - f(x + AX, u) I
+If(x, it) - f(x + AX, it) I) + A1].
Since f is continuous and 1] is an arbitrary positive number, dividing by A
and taking the limit as A ! 0, we obtain the result. D
Let X and Y be normed spaces.
Lemma 2.4. Let a set-valued map F : X  Y be Lipschitzian with a con-
stant l > O. Then
(2.5) d ( ( X, Y ) , gr F) < d (y, F ( X )) < (1 + l) d ( ( X, Y ) , gr F) .
Proof. Define the norm in X x Y by l(x,y)lxxY == Ixlx + Iyly. Let E > O.
There exists z E F(x) satisfying d(y, F(x)) + E > Iy - zly. Since
d ( ( X, Y ), gr F) < I ( X, y) - (x, z ) I x x y == I y - z I y
and E > 0 is arbitrary, we obtain the left inequality in (2.5).
Let E > 0, and let (x, iJ) E grF be such that
d((x, y), grF) + E > I(x, y) - (x, iJ)lxxY == Ix - xix + liJ - yly.
There exists z E F(x) satisfying liJ - zly < llx - xix. Hence we have
d(y, F(x)) < Iy - zly < liJ - zly + liJ - yly < llx - xix + liJ - yly
< (1 + l)(lx - xix + liJ - yly) < (1 + l)(d((x, y), grF) + E).
Since E > 0 is arbitrary, we obtain the right inequality in (2.5). D
For the contingent cone to the set gr f (., U) at (x, v), one can establish
the following inclusion.

40
2. Set- Valued Analysis
Proposition 2.9. Let U C R k be a compact set, and let 1 : R n x U ---+ Rm
be a twice differentiable in x function. Assume that the function and the
derivatives are continuous in (x, u) and it is a unique vector in U satisfying
v == I(x, u). Then
T_((x, v), grf(., U)) C {(x, v) I v E Cx + T_(iJ, f(x, U))}.
Proof. Let (x, v) E T-((x,v),grf(.,U)). Then by definition there exist
sequences Ai! 0, Ui E U, and Yi E R n , i == 1,2, . . . , such that
.liln A;II(x + Ai X , V + Ai V ) - (Yi, I(Yi, ui))1 == o.
 ---+ 00
Observe that
.liln A;ll x + Ai X - Yi I == O.
 ---+ 00
Since 1 is locally Lipschitzian in x, we have
(2.6)
.liln A;llv + AiV - I(x + AiX,Ui)1 == o.
 ---+ 00
Since U is cOlnpact, without loss of generality Ui ---+ Uo E U (we can choose
a convergent subsequence if needed). Since 1 is twice differentiable in x and
the derivatives are continuous in (x, u), from (2.6) we have
o == .liln Iv - A;I(f(x + Ai X , Ui) - I(x, u))1
 ---+ 00
== .lim Iv - (\1 xl(x, Ui)X + A;l (I(x, Ui) - I(x, u))) I.
 ---+ 00
The existence of the limit implies that the sequence A;lll(x,Ui) - l(x,u)1
is bounded. Consequently v == I(x, uo), that is, Uo == u. Thus we obtain
.liln Iv - (Cx + A;l(l(x,Ui) - l(x,u)))1 == O.
 ---+ 00
We see that t11e sequence A;l(l(x,Ui) - I(x,u)) converges to a vector w E
T_(v, I(x, U)). D
From Len11na 1.4 and Propositions 2.8 and 2.9 we derive the following
resul t.
Theorem 2.4. Let U C R k be a compact, and let 1 : R n x U ---+ R m be a
continuous twice differentiable in x function. Assume that the set I(x, U)
is convex for all x E R n , the derivatives are continuous in (x, u), and u is a
unique vector in U satisfying v == 1 (x, u). Then
T + ( ( x, v), gr 1 ( ., U)) == T - ( ( x, v) , gr 1 ( ., U)) == {( x, v) I v E C x + K}.

2.3. Lipscl1itzian approxi111ations
41
Derivatives of set-valued maps. If j : R  R is a smooth function, then
the graph of its directional derivative x  j'(x)x at a point x is a tangent
to the graph of f at the point (x, j(x)). Following this interpretation of the
directional derivative, we can define derivatives of a given set-valued Inap
as set-valued Inaps whose graphs are cones locally approxilnating the graph
of the Inap.
Let X and Y be nOI'lned spaces. Consider a set-valued n1ap F : X  Y.
Let (x, iJ) E grF. The set-valued n1ap D+F(x, iJ) : X  Y defined by
gr D + F ( x, iJ) == T + ( ( x, iJ) , gr F)
is called the derivative of F at the point (x, iJ). In other words
Y E D + F ( x, iJ) ( x ) {::} ( X, y) E T + ( ( x, iJ) , gr F) .
The set-valued Inap D_F(x, iJ) : X  Y defined by
grD_F(x, iJ) == T_ ((x, iJ), grF)
is called the contingent derivative of F at the point (x, iJ). In other words
Y E D_F(x, iJ) (x) if and only if (x, y) E T- ((x, iJ), grF).
Proposition 2.10. Assume that a set-valued map F : X  Y satisfies the
Lipschitz condition with a constant l > O. Then
D+F(x, iJ)(x) == {y Ililn A -ld(iJ + AY, F(x + AX)) == O},
..\10
D_F(x,iJ)(x) == {y lliminf A-1d(iJ + Ay,F(x + AX)) == O}.
..\10
Proof. By Lelnlna 2.4 we have
d((x + AX, iJ + AY), grF) < d(iJ + AY, F(x + AX))
< (1 + l)d((x + AX, iJ + AY), grF).
Dividing by A and taking the lilnit, we obtain the result.
o
2.3. Lipschitzian approximations
In this section we show that any bounded upper semi-continuous set-valued
map with closed convex values can be approxilnated with any given accuracy
by a locally Lipschitzian set-valued map.
To prove this theoren1 we need an ilnportant notion of partition of unity.
Let X c R n be a set and {Oi}iEI be an open covering of X; that is, X c
UiEIOi, and each Oi is open. We say that an open covering {Oi}iEI is locally
finite if for each i E I the set {i' E I I Oi n Oi' :I 0} is finite. A family of
functions {Pi(X)}iEI defined on X is called a Lipschitzian partition of unity

42
2. Set- Valued Analysis
subordinated to a locally finite covering {Oi}iEI if
1. pi(.) is locally Lipschitzian for all i E I,
2. Pi (X) > 0 for x E Oi n X a11d Pi (X) == 0 for x E X \ Oi,
3. for each x E X, L:iEI Pi(X) == 1.
Since the covering is locally finite, the SUln in the last condition is taken
over a finite number of indices and, hence, is well defined.
Lemma 2.5. Let X eRn. For any locally finite open covering {Oi}iEI of
X, there exists a Lipschitzian partition of unity subordinated to it.
Proof. For all i E I define functions qi : X  R+ by
qi(X) == d(x, X \ Oi)
and functions Pi : X  R+ by
qi ( X )
Pi(X) = L: .( ) '
JEI qJ x
For at least one j E I we have x E OJ. Therefore L:jEI qj(x) > o. Conse-
quently, the functions Pi are well defined. Moreover, we see that for each
x E X, L:iEI Pi(X) == 1, and that Pi(X) > 0 for x E Oi n X, and Pi(X) == 0
for x E X \ Oi.
Now show that each function Pi is locally Lipschitzian.
Let Xo EX. Assume that Xo E Oio. There exists t > 0 such that
Xo + tBn C Oio. Since the covering is locally finite, there are only finite
many indices i E I such that (xo + tBn) n Oi f=. 0. Since the set Xo + tBn
is cOlnpact, there exist M > 0 and m > 0 sucl1 that m < L:jEI qj(x) < 1\11
whenever x E Xo + tBn. Denote by III the number of elements in I. For any
Xl and X2 in Xo + tBn we have
I ( ) ( )1 - Iqi(X2) L:jEI qj(X1) - qi(X1) L:jEI qj(x2)1
Pi Xl - Pi X2 -
L:jEI qj(X1) L:jEI qj(X2)
Iqi(X2) L:jEI qj(X1) - qi(X1) L:jEI qj(x2)1
< 2
m
1
< 2 2)lqi(X2)qj(XI) - qi(xdqj(XI)1 + Iqi(xI)qj(xI) - qi(XI)qj(X2)1)
m
JEI
1 (1 + III)M
< 2 L qj(xI)lxl - x21 + qi(XI) L IXI - x21 < 2 IXI - X21,
m m
JEI JEI
since for each i the function qi is Lipschitzian with constant equal to 1 (see
Lemma 2.1). D

2.3. Lipschitzian appl'OXilnations
43
(( Theorem 2.5. Let F : R n  R n be an upper semi-continuous set-valued
map with closed convex values. Assume that there exists b > 0 such that
F( x) C bBn for all x E R n . Then there exists a sequence of locally Lips-
chitzian set-valued maps Fk : Rn -t R n , k == 0, 1, . . ., satisfying the follow-
ing conditions:
1. F(x) c ... C Fk+1(X) C Fk(X) C ... C Fo(x) C bBn for all x ERn;
2. given € > 0 and x E R n , there exists a positive integer k( €, x) such
that Fk(X) C F(x) + EBn whenever k > k(€, x).
Proof. Consider an orthonormal basis {e1,... , en} in R n . Fix Po > 0
and consider the points pom 1 e1 + ... + pomne n , where m i , i == l,n , are
integers. We shall denote such points by x(m 1 ,... ,mn) or x, where m E zn
is a vector with integer components. Consider the open cube QO == {x ==
q 1 e1 + .. . + qne n I qi E] - Po, po[, i == 1, n }. Obviously, {x + QO}mEZ n is
an open locally finite covering of R n .
Consider the convex sets C == clcoF(x + 2QO). Let {P(-)}mEzn
be a locally Lipschitzian partition of unity subordinated to the covering
{x + QO}mEZ n . Define the set-valued map
Fo(x) = 2: p(x)C.
mEZ n
Obviously Fo is locally Lipschitzian convex-valued and Fo(x) C bBn for all
x.
To construct the map F 1 we set PI == po/3 and repeat the procedure
described above using PI i11stead of Po. As a result we obtain the locally
finite open covering {x!n + Q1 }mEZn of R n , the sets C!n == cl coF( x!n + 2Q1 ),
and the map
Ft(x) = 2: p(x)C.
mEZ n
The map F 1 has the same properties as Fo.
We show now that F1(X) C Fo(x) for all x. Let x ERn. Let
Zo(x) == {m E Zn I x E x + QO}
and
Zl(X) == {m E Zn I x E x +Q1}.
Consider mo E Zo(x) and m1 E Zl (x). Then we have x!n 1 +2Q1 C xo +2Qo.
Consequently C!n 1 C Co. Since the sets C and C!n are convex, we have
C = 2: p(x)C c 2: p(x)C.
mEZo(x) mEZo(x)

44
2. Set- Valued Analysis
Therefore
H(x) = L p(x)C = L p(x)C
mEZ n 7nE Z l(x)
C L p(x) L P?n'(X)C1
mE Z l(x) m'EZo(x)
L Pn(x) L P?nI(X)C, = Fo(x).
mE Z l(x) m'EZ n
Sin1ilar inclusion holds for F(x), that is, F(x) C Fo(x). Indeed, F(x) C
C if m E Zo (x). The sets C are convex; therefore we have
F(x) C L P?n(x)C = L P?n(x)C = Fo(x).
7nE Z o(x) mEZ n
It is clear that using Pk+1 == Pk/3, k == 0,1,..., we can construct a
seque11ce of set-valued n1aps Fk : R n ---7 R n , k == 0,1, . . . , satisfying the first
condition of the theorem.
Show that the second condition is satisfied. Let x E R n and E > O.
Since the Illap F is upper selni-continuous, there exists k( E, x) such that
F(y) C F(x) + EBn whe11ever y E x + Qk(E,X). Let k > k(E, x). Put Zk(X) ==
{m E zn I x E x + Qk}. If m E Zk(X), tl1en we 11ave
x + 2Qk C x + 3Qk.
Consequently for all k > k(E, x) and y E x+2Qk we have F(y) C F(x)+EBn
whenever m E Zk(X). The set F(x) + EBn is convex and closed; therefore
we obtain
C == cl coF(x + 2Qk) c F(x) + EBn,
for all m E Zk(X). Thus
Fk(X) = L p(x)C c F(x) + tBn
mEZk(X)
whe11ever k > k(E, x). The theorelll is proved.
D
2.4. Extension theorem
In this sectio11 we prove an extension theorem for convex-valued upper semi-
c011tinuous Illaps. It turns out that sucl1 a map defined on a closed set in
R n can be extended to an upper seIlli-continuous Illap defined on the whole
space.
Tl1e following auxiliary Ie III III as are devoted to the construction of a
t;pecific covering used in tl1e proof of the extension theoreill.

2.4. Extel1sion tl1eorell1
45
If A and B are sets in R n , then the distance between them is denoted by
d(A, B) == inf{la - bll a E A, b E B}. The dialneter of a set A is denoted
by diamA == SUp{la1 - a211 aI, a2 E A}.
Lemma 2.6. Let CeRn be a closed set. Then there exists a collection of
closed cubes F == {Pk} r 1 such that
1. UkPk == R n \ C,
2. intPk n intP k , == 0 for all k =I- k',
3. dialnPk < d(Pk, C) < 4dialnP k .
Proof. Let {e1,... , en} be an orthonorlnal basis h1 Rn. Set Qk == {x ==
q1 e1 +. . . +qnen I qi E [- 2- k , 2- k ], i == 1, n }. Consider the collection of cubes
Mk == {x + Qk I m E zn}, where x == 2-km1e1 + ... + 2- k m n e n , m ==
(m1,... ,m n ) E zn, k == 0, ::l:1,.... For each cube fron1 1v1k the length of
the edge and the dialneter are equal to 2- k + 1 and y0i2-k+1, respectively.
Consider the sets
Ok == {x I y0i2- k + 2 < d(x, C) < y0i2- k + 3 }, k == 0, ::l:1,... .
Obviously we have
00
R n \ C == U Ok,
k==-oo
and therefore
R n \ C == U P,
PEFo
where
Fo = U{P I P E Mk, pnD k i= 0}.
k
Show that
dialnP < d(P, C) < 4diamP, P E Fo.
Let P E Mk. Then diamP == y0i2- k + 1 . Since P E Fo, there exists x E
P n Ok. Consequently
(2.7) d(P, C) < d(x, C) < y0i2- k + 3 == 4diamP
and
d(P, C) > d(x, C) - diamP > y0i2- k + 2 - y0i2- k + 1 == dialnP.
From (2.7) we see that the cubes have no COlnmon points with C.
Let P E Fa. If P' E Fo and PcP', then froln (2.7) we obtain diamP' <
4diamP. Obviously if P' E Fo, P" E Fo, and PcP', PcP", then intP' n
intP" =I- 0. Note that if P' E lvlk', P" E lvlk", k' > k", and intP' nintP" =I- 0,
then P' c P". Hence for any P E Fo there exists a unique lnaxilnal cube
containing P, and the interiors of lnaxhnal cubes have no COlnmon points.

46
2. Set- Valued Analysis
Denote by F the set of maximal cubes from Fo. Obviously F satisfies all
the required conditions. 0
Lemma 2.7. If PI E F, P2 E F, and PI n P2 =10, then
 diamP2 < diamP 1 < 4diamP2.
Proof. Since d(P 1 , C) < 4diamP 1 and PI n P2 =I 0, we have d(P 2 , C) <
4dialnP 1 +diamP1 == 5diamP 1 . Besides, diamP2 < d(P 2 , C), and tllere-
fore diamP2 < 5diamP1. There exists an integer k such that diamP2 ==
2 k diamP 1 . Hence diamP2 < 4diamP 1 . Using the same reasoning, we obtain
diamP 1 < 4diamP2. This ends the proof. 0
Lemma 2.8. Let P E F. Then at most N == 12 n cubes from F have
common points with P.
Proof. If P E Mk, then there exist 3 n cubes from Mk (including P), which
have common points with P. Each cube from Mk contains at most 4 n cubes
from F with the diameter greater than or equal to (1/4)diamP. Now the
result follows from Lemma 2.7. 0
Let Pk E F, and let Xk be its center. Fix E E]O,1/4(, and put Ok
(1 + E) (intPk - xk) + x k . Obviously we have Pk C Ok.
Lemma 2.9. Any point x E Rn \ C belongs to at most N cubes Ok.
Proof. Show that if pnOk =I 0, then pnPk =10. Indeed, consider all cubes
Pi that have common points with Pk. Since their diameters are greater than
or equal to diamPk/4, we have Ok C UP i . Therefore P n Ok =I 0 only if
P n Pk =10. Since each x E Rn \ C belongs to a cube P, by Lemma 2.8 we
obtain the result. 0
From the above lemmas we derive the following result.
Lemma 2.10. Let CeRn be a closed set. Then there exists a locally finite
open covering {Ok} r 1 satisfying
c1diamOk < d(Ok, C) < c2diamOk,
where C1 and C2 are positive constants.
Now we are in a position to prove the main result of this section.
Theorem 2.6. Let CeRn be a closed set, and let F : C ---+ R n be an upper
semi-continuous map with closed convex values satisfying F(:) c bBn for
all x E C. Then there exists an upper semi-continuous map F : Rn ---+ R n
with closed convex values satisfying F(x) C bBn, for all x ERn, and such
that F(x) == F(x) whenever x E C.

2.5. Fixed point theorems
47
Proof. By Lemlna 2.10 there exists a locally finite open covering {Ok} k 1
satisfying
c1dialnOk < d(Ok, C) < c2dialnOk.
From Lemma 2.5 it follows that there exists a Lipschitzian partition of
unity {Pk(.)} k 1 subordinated to the covering. Fix points Xk E C satis-
fying d(C,Ok) == d(Xk,Ok) and set
F(x) == { E r 1 Pk(x)F(Xk), X tJ. C;
F(x), x E C.
Since the covering {Ok} is locally finite, only finitely many terlns in the
- -
definition of F differ from zero. Therefore F is upper semi-continuous in
the set R n \ C and has closed convex values. Besides, F(x) c bBn for all
x ERn.
Fix y E C and show that F is upper semi-continuous at y. Note that if
x E Ok, then we have
Iy - xl > d(Ok, C) > c1diamOk
and, hence,
Iy - xkl < Iy - xl + Ix - xkl < Iy - xl + d(Ok, C)
< Iy - xl + c2diamOk < cly - xl,
where c == (1 + C2/C1).
Let E > O. There exists 8 > 0 such that F(x) c F(y) + EBn whenever
x E Cn(y+8Bn). If Iy-xl < 8/c and x tJ. C, then x E Ok only if IY-Xkl < 8.
Hence
F(Xk) c F(y) + EBn.
Since the set F(y) + EBn is convex, we obtain
F(x) = LPk(X)F(Xk) C LPk(X)(F(y) + EBn) = F(y) + EBn.
k k
The theorem is proved.
D
2.5. Fixed point theorems
Here we establish sufficient conditions for a set-valued F which maps a
convex compact set A c R n into itself to have a fixed point, that is, a point
x E A satisfying x E F(x).

48
2. Set- Valued Analysis
Brouwer fixed point theorem. Recall the falllous result of Brouwer.
Theorem 2.7 (Brouwer). Let A c Rn be a convex compact set. If f : A 
A is a continuous map, then there exists a point x E A satisfying f (x) == x.
Corollary 2.1. Let L; c R n be a convex compact set, and let x E intL;.
Assume that a continuous map f : L;  Rn satisfies the following condition:
If(x) - xl < Ix - xl, \Ix E bdL;.
Then x E in1f.
Proof. Consider the Ivlinkowski function J.-L( x, L; - x) of the set L; - x. Set
p(x) == { X + (x - x)/ J.-L(x - x, L; - x), x  L;,
x, x E L;.
Define the l11ap cp : L;  L; by cp( x) == p( x - f (x) + x). By Proposition 1.6 the
IVlinkowski function is continuous and takes value 1 on the boundary of the
set. Therefore cp is continuous. By the Brouwer fixed point theorelll there
exists x E L; such that cp(x) == x. Show that x - f(x) + x E L;. Suppose that
x - f(x) + x  L;. Thel1 we have 1 < J.-L( - f(x) + x, L; - x), x E bdL;, and
x == x + (x - f(x))/ J.-L( - f(x) + x, L; - x). Consequently we l1ave
II. ( - f( x ) + x L; - X ) = Ix - l(x)1 < 1
,....., , 1 - .... 1 - ,
x-x
a contradiction. Therefore x - f(x) + x E L;. Hence
x == p(x - .f(x) + x) == x - f(x) + x,
and we get x == f(x).
D
a-selectionable maps. Let X c Rn. We say that a set-valued map F :
X  Rn is a-selectionable if there exists a decreasing sequel1ce of cOlllpact-
valued maps Fk : X  Rn with closed graphs such that for any k == 1,2, . . . ,
Fk has a continuous selection and F(x) == n r 1 Fk(X) for any x E X.
From Theoreills 2.5 and 2.2 we see that an upper sellli-continuous lllap
F : Rn  Rn with con1pact convex values contained in a ball bBn is a-
selectionable. Later we show that the same is true for lllaps associated with
the reachability set of a differential inclusion with upper semi-continuous
right-hand side.
The notion of a a-selectionable map is useful in fixed point theory due
to the following result.
Theorem 2.8. Let A c Rn be a convex compact set, and let F be a a-
selectionable map from A into A. Then F has a fixed point; that is, there
exists x E A such that x E F(x).

2.6. Convex processes
49
Proof. Let fk be a continuous selection of Fk. By Brouwer's theorem
the Inap x  7r(fk(X), A) has a fixed point Xk E A, and IXk - fk(Xk)1
d(fk(Xk), A).
For any m < k we have
(2.8) d(Xk, Fm(Xk)) < d(Xk, Fk(Xk)) < IXk - fk(Xk)1 == d(fk(Xk), A).
TIle cOlnpactness of A allows us to aSSUlne that Xk converges to a point
x E A. Note that
(2.9)
fk(Xk) E Fk(Xk) C F'In(Xk).
The map Fm is upper selni-continuous. Therefore, given E > 0, there exists
ko such that Fm(Xk) C Fm(x) + EBn whenever k > ko. Invoking Proposi-
tion 2.1 and Proposition 2.6, froln (2.8) we obtain
(2.10)
d(x, Fm(x)) < IXk - xl + d(Xk, Fm(Xk)) + E < IXk - xl + d(fk(Xk), A) + E
whell k > ko. Since the sequence of compact sets Fm(x) decreases to F(x),
there exists mo SUCll that Fm(x) C F(x) + EBn when m > mo. Observe that
fk(Xk) E Fk(Xk) C Fm(Xk) C Fm(x) + EBn C F(x) + 2EBn C A + 2EBn.
Taking the limit as k  00 in (2.10), we get x E Fm(X)+3EBn C F(x)+4EBn.
Since E > 0 is arbitrary, we obtain x E A and x E F(x). D
Corollary 2.2 (Kakutani's fixed point theoreln). Let A C Rn be a com-
pact convex set, and let F : A  A be an upper semi-continuous set-valued
map with convex compact values. Then F has a fixed point.
2.6. Convex processes
A set-valued Inap A : Rn  R m is called a convex process iff its graph
grA is a convex cone. Convex processes are multivalued analogues of linear
operators and play an important role in set-valued analysis. Obviously a
convex process A has the following properties:
1. A(AX) == AA(x), V A > 0, x E domA;
2. A(X1) + A(X2) C A(X1 + X2), VX1, X2 E domA.
We say that a convex process A : R n  R m is closed if its graph is
closed. It is called strict if domA == R n .
Let A : R n  R m be a convex process. The adjoint process A * : R m 
R n is defined by
A * ( v *) == {x * I (x * , x) < (v * , v) , V ( X, v) E gr A } .

50
2. Set- Valued Analysis
In other words (v*,x*) E grA* iff (-x*,v*) E (grA)*. Obviously the adjoint
process is always closed.
Example. Consider a set-valued map A(x) == Cx+K, where C: R n  R m
is a linear operator and K c R m is a closed convex cone. Obviously grA
is a closed convex cone. To calculate the adjoint process, show that the
conjugate cone of grA is given by
(grA)* == {(x*,v*) I x* == -C*v*, v* E K*}.
Indeed, let x* == -C*v* where v* E K*. Then for any vector v == Cx + y,
where y E K, we have
(x*,x) + (v*,v) == (x*,x) + (v*, Cx) + (v*,y) == (v*,y) > o.
Therefore (v*,v) > (C*v*,x). Hence C*v* E A*(v*).
Now suppose that (x*, x) + (v*, v) > 0 for all (x, y) satisfying v == Cx+y,
where y E K. Put Y == o. Then we see that (x*, x) + (v*, Cx) > 0 for any
x ERn. Hence x* == -C*v*. Put x == o. Then we have (v*, y) > 0 for all
y E K. Hence v* E K*. Thus we obtain
{ C * v * V * E K * ,
A* ( v* ) == fA , ,
VJ v*  K*.
In particular if A(x) == Cx, then A * (v*) == C*v* for all v* E R m ; that is,
when the convex process reduces to a linear operator, the definition of the
adjoint process reduces to the customary one.
Now let us establish a few properties of convex processes.
Lemma 2.11. Let A : R n ---7 Rm be a closed convex process. Then the
following equalities hold:
1. (-A)*(-) == A*( -.),
2. (A- 1 )*(.) == _(A*)-l( -.),
3. A**(.) == -A(-.),
4. A(O) == (domA*)*,
5. A-I (0) == (im( - A*))*.
Proof. The first equality follows from the equivalence of the relations
( v * , x *) E gr ( - A) * ,
( - x * , v *) E (gr ( - A) ) * ,
(-x*,x) + (v*, -v) > 0, V(x,v) E grA,
x* E A*( -v*).

2.6. Convex processes
51
The second equality is a consequence of the equivalence of the following
relations:
(x*, v*) E gr(A -1)*,
( - v * , x *) E (gr A -1 ) * ,
(-v*,v) + (x*,x) > 0, V(x,v) E grA,
-x* E A*( -v*),
-v* E (A*)-l( -x*).
To prove the third equality, note that the following relations are equiv-
alent:
( x, v) E gr A * * ,
( - v, x) E (gr A * ) * ,
( - v, v *) + (x, x *) > 0, V ( v * , x *) E gr A * ,
(-v, v*) + (x, x*) > 0, V( -x*, v*) E (grA)*,
(-v, v*) + (-x, x*) > 0, V(x*, v*) E (grA)*,
(-x, -v) E (grA)** == grA,
v E -A(-x)
(we used Corollary 1.5).
Now prove the fourth equality. Let v E A(O). Then (0, v) E grA, and
for any (x*, v*) E (grA)* we have (v*, v) > O. It implies the inclusion
A(O) c (domA*)*.
Let v E (domA*)*. Then (0, -x*) + (v, v*) > 0 for each v* such that
there exists x* satisfying (-x*,v*) E (grA)*. Consequently (O,v) E (grA)**.
Applying Corollary 1.5, we obtain (0, v) E grA.
To prove the last equality, note that from the first and the fourth equal-
ities we have
A- 1 (0) == (dom(A- 1 )*)* == (doln( _(A*)-l( -.)))*
== (-dom(-(A*)-l))* == (-dom(A*)-l)* == (-imA*)* == (im(-A*))*.
The lemma is proved.
D
The following result contains a multi valued analogue for the rule (v*, Cx)
== (C*v*,x), which holds for any linear operator C.
Theorem 2.9. Let A : R n  R m be a convex process. Assume that Xo E
int domA and va E domA *. Then
sup (x*,xo) == inf (v,v).
x*EA*(v o ) vEA(xo)

52
2. Set- Valued Analysis
Proof. Consider the function
cI> ( x) == inf { (va, v) + 8 ( ( x, v), gr A) I v E R m }.
By Proposition 1.7 cI> is convex. Show that cI>(x) > -00 for any x E domA.
Suppose that there exists a sequence Vk E A(x) such that (va, Vk) ---+ -00 as
k ---+ 00. Then (x*, x) < (Va, Vk) for any x* E A* (va). Since the left side of
the inequality is finite, the right side cannot tend to -00.
Since Xo E int domA there exists a silnplex  == co {Xl, ... , X n +1} C
domA such that Xo E int . Let x E . Then froln convexity of cI> we derive
cI>(x) < max{cI>(xi) Ii == 1,n+ 1}. Thus cI> is bounded in a neigllborhood
of Xo. By Theorem 1.9 cI> is continuous at Xo. By Corollary 1.4 cI>**(xo) ==
<I>(xo). Hence we have cI>(xo) == sup{ (x*, xo) - cI>*(x*) I x* E Rn}. Observe
that
cI>*(x*) == sup{(x*,x) - inf{(vo,v) + 8((x,v),grA)}}
x v
== sup{(x*,x) - (va, v) - 8((x,v),grA)} == 8((-x*,vo), (grA)*).
(x,v)
Thus we obtain
sup{(x*,xo) - 8((vo,x*),grA*)} == cI>(xo) == inf{(va,v) + 8((xo,v),grA)}.
x* v
This ends the proof.
D
The results about adjoint of product and SUln of linear operators are
valid for convex processes too.
Theorem 2.10. Let Al : R n ---+ R m and A 2 : R rn ---+ R k be convex processes.
Assume that either int imA1 n ri domA2 =I- 0 or ri imA1 n int dOlnA 2 =I- 0.
Then the map A 2 A 1 : R n ---+ R k is a convex process and (A 2 A 1 )* == At A 2 .
Proof. Put
K 1 == {(x, y, z) E R n x R m x R k I (x, y) E gr AI} ,
K 2 == {(x,y,z) E R n x R m x R k I (y,z) E grA 2 }.
Obviously K 1 and K 2 are convex cones. Observe that (x, z) E grA 2 A 1 if
and only if there exists y E R m such that (x, y, z) E K 1 n K 2 . Hence A 2 A 1
IS a convex process.
By definition the following relations are equivalent:
( z* , x *) E gr ( A 2 AI) * ,
(-x*, z*) E (grA 2 A 1 )*,
(-x*, x) + (0, y) + (z*, z) > 0, \f (x, y, z) E K 1 n K 2 ,
(-x*,O,z*) E (K 1 nK 2 )*.

2.6. Convex processes
53
Since riK1nriK2 =I=- 0, by Proposition 1.1 cl(K1 n K2) ==clK 1 nclK 2 . In-
voking Corollary 1.7, we obtain
(K 1 n K2)* == (cl(K1 n K 2 ))* == (clK1 n clK 2 )* == cl(K; + K 2 ).
Let us show that the cone K; + K 2 is closed. Suppose that there exist
sequences ai E K; and bi E K 2 such that liln( ai + bi) == c* tf. Ki + K 2 .
Observe that either lail  00 or Ibil  00 as i  00. Otherwise the
sequences ai and bi would be bounded and would contain subsequences
converging to a* E Ki and b* E K 2 . It would imply that c* == a* + b* E
Ki + K 2 . Assume that lail  00. Then we have
a b
 +   0 as k  00.
lail lail
Without loss of generality
*
a i * *
lail -t a E Kl as k -t 00,
and therefore
bi * *
-  -a E K 2 as k  00.
lail
Hence there exists a vector y* E R m such that a* == (0, y*, 0) E Ki and
-a* == (0, -y*, 0) E K 2 . Consequently we have (y*, Y1) > 0 > (y*, Y2) for all
Y1 E imA1 and Y2 E dOll1A 2 , a contradiction. Thus (K 1 n K2)* == Ki + K 2 .
Froln this we see that for any vector (-x*, 0, z*) E (K 1 n K2)* there exists
a vector y* E R m such that (-x*, y*, 0) E Ki and (0, -y*, z*) E K 2 ; in
other words (y*, x*) E grAi and (z*, y*) E grA 2 . It implies that (z*, x*) E
gr(Ai A 2 ). D
Theorem 2.11. Let Al : R n  R m and A 2 : R n  R m be convex pro-
cesses. Assume that int dOlnA 1 n int domA2 =I=- 0. Then the map Al + A 2 is
a convex process and
(AI + A 2 )* == A! + A 2 .
Proof. Consider convex processes Ao : R n  R n x R n , Al : R n x R n 
R m x Rm, A2 : R m x R m  R m given by
Ao(x) == (x, x),
A 1 (X1,X2) == {(Vl,V2) I VI E A 1 (X1), V2 E A2(X2)},
A2 ( VI, V2) == VI + V2.
Observe that Al + A 2 == A2A1Ao. We see that Al + A 2 is a convex process.
By Theoreln 2.10 (AI + A 2 )* == AoAiA2. Hence we have
AoA!A2(V*) == AoA!(v*, v*) == Ao(A!(v*), A 2 (v*)) == A!(v*) + A 2 (v*).
This ends the proof. D

54
2. Set- Valued Analysis
Whe11 A : R n  R m is a convex process, we define
IAI == sup inf Ivl.
xEBnndomA vEA(x)
Theorem 2.12. Let A : R n  R m be a strict closed convex process. Then
1. A is Lipschitzian with the constant IAI,
2. Ix*1 < IAllv*1 for all v* E dOlnA* and x* E A*(v*),
3. domA* is a closed convex cone,
4. the restriction of A* to the subspace domA* n -domA* is a linear
operator.
Proof. 1. Show that IAI < 00. Consider a shnple x E == CO{X1'. .. , X n +1}
such that Bn C E. Let Vi E A(Xi) , i == 1, n + 1. If x E Bn, then there
exist numbers Ai > 0, i == 1, n + 1, such that 2:i Ai == 1 and x == 2:i AiXi.
Obviously we have v == 2:i AiVi E A(x) and Ivl < maxi IVil. Hence IAI < 00.
Let Xl E R n and X2 E R n . Put a == IX2 - xII and consider the vector
X3 == a- 1 (x2 - Xl). We obviously have IX31 == 1, X2 == Xl + aX3, and
A(X1) + aA(x3) C A(X2). Let VI E A(X1). For any E > 0 there is a
vector V3 E A(X3) such that I V 31 < IAI + E. There exists a vector V2 E A(X2)
satisfying V2 == VI +av3. Hence we obtain IV2-V11 == alv31 < (IAI+E)lx2-X11.
Since E > 0 is arbitrary and A is a closed process, we obtain the result.
2. Let x* E A*(v*). Then we have (-x*,x) + (v*,v) > 0 for all (x, v) E
grA. Put X == Ix* 1- 1 x*, fix E > 0, a11d choose a vector v E A(x) n (IAI +E)Bm.
Then we have
Ix*1 == (x*, x) < (v*, v) < (IAI + E)lv*l.
Since E > 0 is an arbitrary number we obtain Ix*1 < IAllv*l.
3. Consider a sequence vi E domA*. Let lim vi == v*. There exists a
sequence xi E A*(vi). From the inequality Ixil < IAllvil we see that t11e
sequence xi is bounded. Therefore it contah1s a convergent subsequence.
Without loss of generality xi converges to a vector x*. Sh1ce grA* is closed,
we have x* E A*(v*); that is, v* E domA*.
4. From the inequality Ix*1 < IAllv*1 we obtain A*(O) == {O}. Let
v* E domA * n -domA *. Then we have
A*(v*) + A*( -v*) c A*(O) == {O}.
Hence A*(v*) is a single point set and A*( -v*) == -A*(v*). Thus we have
A*(av*) == aA*(v*) for all a E R. Moreover, since A* is single valued, the
equality A*(vi) + A*(v2) == A*(vi + v2) holds for all vi and v2 in R m . The
theorem is proved. D

2.6. Convex processes
55
A convex process A : R n  Rm is called bounded if there exists a
constant b > 0 such that Iv I < blxl for all x E R n and v E A(x). From
Theoreln 2.12 we see that the adjoint process of a strict closed convex process
is bounded.
We say that a nonzero vector x E R n is an eigenvector of a convex
process A : R n  R n if there is a nUlnber A E R such that AX E A(x). The
number A is called an eigenvalue of A.
Theorem 2.13. Let KeRn be a nonzero convex closed cone which does
not contain a line, and let A : R n  R n be a closed bounded convex process
such that K c domA and A(x) n K =10 for all x E K. Then there exists an
eigenvector of A contained in the cone K and corresponding to a nonnegative
eigenvalue.
Proof. 1. Suppose that A(x) nintK =10 for all x E K, x =I O. Consider the
set
o == {w E R 13 x E K n bdBn : (A(x) - wx) n K =l0}.
Since sufficiently small w > 0 belongs to 0, we conclude that 0 =I 0. Let
us prove that the set 0 is bounded from above. If this is not the case, then
there exist sequences Wk  00 and Xk  x such that
Xk E K n bdBn, (w;;l A(Xk) - Xk) n K =10.
Taking the limit we obtain -x E K. This contradicts the inclusion x E K
because the C011e K does not contain a line.
Let Wo == sup 0 > O. Since the cone K is closed and the process A
is closed and bounded, there exists a vector Xo E K n bdBn such that
(A(xo) - woxo) n K =I 0. Suppose that there exists a nonzero vector Zo E
(A(xo) - woxo) n K. Then we have
A ( XO + 2:0 ) - Wo (xo + 2: 0 )
 A(xo) + A (  ) - woxo - Zo  Zo + A (  ) .
2wo 2 2 2wo
Since Zo =I 0, we obtain A(zo) nintK =I 0. Hence, (A(yo) -wOYo) nintK =I 0,
where Yo == (xo + zo/( 2w o))/l x o + zo/( 2w o)l. This implies that sup 0 > woo
Thus, Zo == 0 and WOXO E A(xo).
2. To reduce the general case to that one considered in part 1, introduce
the restriction of the process A to the subspace K - !{. This allows us to
aSSUlne that intK =I 0. Let Xo E intK. Consider the sequence of linear
operators Ak : R n  R n defined by
Akx == k- 1 (xo, x)xo.

56
2. Set- Valued Analysis
Observe that (A(x) + k- 1 (xo, x)xo) n intK =i 0 for all x E K, x =i O. By
part 1 there exist numbers Wk > 0 and vectors Xk E K n bdBn such that
WkXk E (A + Ak)(Xk) == A(Xk) + k- 1 (xo,Xk)XO' Without loss of generality
the sequel1ces {Wk} and {Xk} converge to Wo and Xo respectively. Taking the
Ihnit we reach WOXO E A(xo), Wo > 0, and the end of the proof. 0
2.7. Structure of a convex process
Here we study the structure of a convex process. The lnain result of this
section is a lnultivalued versiOl1 of the Jordan theoren1.
We say that a subspace X c Rn is invariant by a convex process A :
Rn  Rn if A(x) c X whenever x E X.
Consider a closed strict convex process A : Rn  Rn. By Theorem 2.12
the restriction of A* to the subspace domA* n -domA* is a linear operator.
Denote by J C dOlnA * n -domA * the maximal subspace invariant by A * .
Put 1== J1-.
Lemma 2.12. The subspace I is the minimal subspace invariant by A.
Proof. Consider vectors x E I, v E A(x), and v* E J. The set A*(v*)
consists of one point al1d is contained in J. By Theoren1 2.9 we have
(v*,v) > (A*(v*),x) == O.
Since v* E J is an arbitrary vector, we see that v E I. Thus I is invariant
by A.
Suppose that there is a subspace II C I invariant by A and such that
II =i I. Then we have A(O) C II. From Lemlna 2.11 we obtain If C dOlnA*.
Put J1 == If. Obviously J c J 1 C dOlnA * n -domA * . Consider points
x E II, V E A(x) C II, and v* E J 1 . By Theorem 2.12 the set A*(v*)
consists of one point. Froln Theoreln 2.9 we have
(A*(v*), x) < (v*, v) == O.
Since x E II is an arbitrary vector, we see that J 1 is invariant by A*. Hence
J is not the lnaximal invariant subspace, a contradiction. 0
Consider convex cones
Lk(A) == (A - AE)-k(O), k == 1,2, . . .
and put
L(>') = U Lk(>')'
kl
It is easy to see that Lk(A) C Lm(A) whenever m > k.

2. 7. Structure of a convex process
57
Froin Theorein 2.12 (Proposition 2) we see that the set of eigenvalues of
A* is boullded alld closed. Denote by Ao(A*) the maxilnal eigenvalue of the
process A *. If A * has no an eigenvector , put AO ( A *)  - 00.
Theorem 2.14. Assume that J  {O} and A > Ao(A*). Then
L(A) == R n .
Proof. By Theorein 1.4 the equality to be proved is equivalent witll
(2.11 )
(L(A))*  {O}.
From Theoreins 2.10 and 2.11 and Lemina 2.11 we have
(Lk(A))*  ((A - AE)-k(O))*  cl iln( -((A - AE)k)*)  -cl im(A* - AE)k.
Consider the convex process B  A-AE. Obviously dOInB* == domA*. Put
!vI = n cl imB*k.
k>l
To prove (2.11) it suffices to show that !vI  {O}. Suppose the contrary.
1. Show that imB*k is closed. Indeed, since A > AO (A *), we have
{O}  B*-k(O)  -(B-k)*(O) == -(domB- k )* (see Theoreins 2.10 and
2.11 and Lemma 2.11). Hence domB- k == R n . By TIleorem 2.12 (Propo-
sition 3) the cone dom(B-k)* is closed. From Lemina 2.11 we obtain
ilnB*k  donlB*-k  -dom(B- k )*. Thus imB*k is a closed cone and
!vI  nk2::1 ilnB*k.
2. Let x* E !vI. Let us show that B*-l(x*) n!vI f=. 0. For any positive
integer k there exists a finite collection of vectors {v k i}  1 such that x* E
,
B*(v k 1)' V k i E B*(v k i + 1)' i  1, k . We have vk i E B*-i(x*). Fix i. Froin
", ,
the argument of the previous paragrapll, Leinma 2.11, and TIleorem 2.12
(Proposition 2) we see that tIle sequence {vk,i} k i is bounded. TIle sequence
{ v k ,l} contains a convergent subsequence {v k1 ,1}. The sequence {V k1 ,2} con-
tains a convergent subsequence {v k2 ,2}' etc. Set vi  liInkioovki,i. Then
we have
(2.12)
Vki+l,i E B* (Vki+l ,i+1).
Taking the limit as ki+1  00 in (2.12) and applying Theorem 2.12, we
obtain vi E B*(vi+1). Taking the lilnit as k1  00 in the illclusion x* E
B*(v k 1)' we have x* E B*(vi). Thus vi E M and vi E B*-l(x*).
1 ,
3. Show that N  M n -!vI == {O}. Suppose that tIlere exists a
nonzero vector q EN. Then from the previous paragraph we have q E
N c domB*-l n -domB*-l. By Theorein 2.12 (Proposition 4) the set
B*-l (q) consists of one point and is contained in !vI (paragraph 2). By the
saIne argument we have B*-l(_q) E M. Consequently B*-l(q) E N. Since
kerB*-l  {O}, we see that B*-l maps N onto itself and has an inverse

58
2. Set- Valued Analysis
li11ear operator which coincides with the restriction of the process B* to the
subspace N. Thus B* maps N onto itself. Since A*(q) - Aq == B*(q) E N,
we have A * (q) E N whenever q EN. It implies that N is invariant by A * .
Therefore N c J == {O}, a contradiction.
4. The cone Aif does not contain a line, t11e process B*-l is bounded, and
B*-l(x*) n M =10. Therefore by Theorem 2.13 there exists an eigenvector
x* E M of the process B*-l corresponding to a nonnegative eigenvalue J-L.
Since B*(O) == {O}, we see that J-L is positive. From the inclusion
J-Lx* E B*-l(x*) == (A* - AE)-l(x*)
we have
(A+  )X*EA*(X*).
Thus we have proved that the process A * has an eigenvector with an eigen-
value bigger than AO (A *), a contradiction. D
Corollary 2.3. Under the conditions of Theorem 2.14 there exists a positive
integer k such that
Lk(A) == R n .
Pro of. Let I; == co{ Xl, . . . , X n +1} be a simplex satisfying 0 E intI; . For an y
i == 1, n + 1 there is k i such that Xi E Lk i (A). Put k == max{ k i I i == 1, n + I}.
Since the cone Lk(A) is convex, we get I; C Lk(A). Therefore Lk(A) == R n .
D
Consider the factor-space R n / I. An element x of R n / I is a class of
vectors X E R n such that for any Xl E x and X2 E x we have Xl - X2 E I. In
particular 6 == I. Define the set-valued map A : R n / I  R n /1 as follows:
A(x) == {v 13x Ex: A(x) nv =l0}.
Let us identify R n / I and J according to the following rule: x  X E J
iff X Ex.
Lemma 2.13. The map A : R n / I  R n / I is a linear operator. The adjoint
operator A* coincides with the restriction of A* to the subspace J.
Proof. It is easy to check that A is a convex process. Show that A( x) == {v I
A(x)nv =l0} for any X E x. Indeed, let x' -x E I; that is, x' == x+y, where
y E I. By invariance we have A(y) c I. From the inclusion A(x) + A(y) C
A(X') we see that for any v E A(x) there is a vector v' E A(X') such that
v' - v E I. Thus we obtain {v I A(x) n v =l0} c {v I A(X') n v =l0}. The
inverse inclusion follows from the same argument.

2. 7. Structure of a convex process
59
Now we prove the inclusion
(2.13)
A(x) - A(x) c I, Vx ERn.
Suppose that there is a vector v E A(x) such that A(x) - v ct I. Then there
exists y E A(x) such that y  I + v. By Theorem 1.6 there exists a vector
y* =I 0 satisfying
(2.14)
and
(y, y*) < (v, y*),
y* E J C dOlnA* n -domA*.
Recall that the restriction of A * to J is a linear operator. Let z E A (x ) .
Then we have (z, y*) - (x, A*(y*)) > O. Since y* E J, we obtain (z, -y*) -
(x, A*( -y*)) > O. Thus (z, y*) == (x, A*(y*)) for all z E A(x). Hence
(v, y*) == (x, A*(y*)) == (y, y*). This contradicts (2.14), and (2.13) is proved.
From (2.13) we see that the set A(x) consists of one class. Consequently
A(O) == 0 and A( -x) + A(x) c A(O) == 0; that is, A( -x) == -A(x). Thus A
is a linear operator.
Let us find A*. Consider vectors v* E J, x ERn, v E A(x) and v' E A(x).
Since v' - v E I, we have (v, v*) == (v', v*). The restriction of A* to J is
single-valued. From Theorem 2.9 we obtain
(x, A*(v*)) == sup (x*, x) == inf (v*, w) == (v, v*)
X*EA*(v*) WEA(x)
for each v E A(x) + I. Consequently (x,A*(v*)) == (v*,A(x)) whenever
x E J  R n / I. The lemma is proved. D
Consider now the factor space R n / J. Its elements we shall denote by x*.
Identify R n / J with I and consider the set-valued map A' : R n / J ---+ R n / J
gi ven by
A'(v*) == {x* 13v* E v* : A*(v*) n x* =l0}.
Lemma 2.14. The set-valued map A' is a convex process coinciding with
the adjoint of the restriction of A to I. The cone domA ' does not contain
proper subspaces invariant by A'. Any eigenvalue of the process A' is also
an eigenvalue of A*.
Proof. It is easy to check that A' is a convex process. Then we can show
that {x* I A*(v*) n x* =l0} == {x* I A*(w*) n x* =l0} whenever v* - w* E J
(see the proof of Lemma 2.13).
Let us find the adjoint process of the restriction of A to I. Let x* E
A * ( v*). Consider vectors x* E x* n I and v* E v* n I. Then there are z* E J
and w* E J such that x* E A * (v* + w*) + z*. Consequently we have
(x, x*) == (x* - z*, x) < (v, v* + w*) == (v, v*)

60
2. Set- Valued Analysis
whenever x E I and v E A(x).
Now let x* E I, v* E I, and (x, x*) < (v, v*) for any x E I and v E A(x).
Then in particular we have 0 < (v, v*) for any v E A(O). Fron1 Lelnn1a 2.11
and Theoreln 2.12 we obtain v* E don1A*. By Theoreln 2.9 we have
S(x,A*(v*)) > sup (y*,x)== inf (v,v*) > (x,x*).
y*EA*(v*) VEA(x)
Hence x* E UY*EA*(v*) 7r(Y*, I). Therefore there is a vector z* E J satisfying
x* + z* E A*(v*). Thus x* E A'(v*).
Since I is the Ininhnal space invariant by A, the cone domA' does not
contain nonzero sllbspaces invariant by A'.
It ren1ains to prove that any eigenvalue of the process A' is also an
eigenvalue of A * .
Show that A*(xi) == A* (x*) + A*(y*) whenever y* E J and xi == x* + y*.
Indeed, A*(x*) + A*(y*) c A*(xi). Since the restriction of A* to J is a
linear operator, we have
A*(x*) == A*(x*) + A*(y*) - A*(y*) c A*(x) - A*(y*)
== A*(x* + y*) + A*( -y*) c A*(x*).
Thus A*(xi) == A*(x*) + A*(y*).
Let AX* E A' (x*) and x* E x*. There exists a vector y* E J such that
AX* E A*(x*)+y*. If A is an eigenvallle of the restriction of A* to J, then we
have nothing to prove. Otherwise V\Te have in1(A*-AE) IJ== J. Consequently
there is a vector z* E J satisfying y* == A*(z*) - AZ*. Fron1 this we obtain
A(X* + z*) E A*(x*) + A*(z*) == A*(x* + z*). Show that x* + z* =I- o. If
x* + z* == 0, then we have x* E J. Hence x* == 0*. It contradicts the
definition of eigenvalue. The lelnna is proved. D
Denote by AI and by E] the restrictions of A and E to I respectively.
Slunn1arizing the results of this section, we obtain the following theoreln.
Theorem 2.15. Let A : R n  Rn be a strict closed convex process, and let
A > Ao(A*). Then
1. The map A : R n / I  Rn / I is a linear operator and its adjoint A *
coincides with the restriction of A* to the subspace J.
2. There exists a positive integer k such that I == (AI - AE] )-k(O).
Thus we see that a strict closed convex process has the following struc-
ture. With any linear manifold parallel to I it associates another linear Inan-
ifold parallel to I, and this correspondence is a linear transformation. More-
over, if x E I, then there exists a finite collection of vectors Yi E I, i == 1, k,

2.8. ProblelTIs
61
sucl1 that
AY1 E A(Y1) ,
Y1 + AY2 E A (Y2 ) ,
(2.15) *** * ** * ***
Yk-1 + AYk E A(Yk),
Yk x.
Vector Y1 is an eigenvector of A. Vectors Yi, i == 2, k, can be considered as
principal vectors of A. Inclusions (2.15) imply tl1at subspace I is a 'cyclic'
subspace of A corresponding to one 'Jordal1 block'. Tllus Tl1eorein 2.15 can
be interpreted as a Inultivalued version of tl1e Jordall tl1eorein froin linear
algebra.
2.8. ProbleITls
1. Let X c R n be a con1pact set, let Al c R m and U C R k be convex
sets, let A : R k  R m be a linear operator, and let f : Rn  R m be
a continuous function. Assun1e that for any x E X there exists u E U
sa tisfying
f(x) + Au E A1.
Prove that for any E > 0 there exists a Lipschitzian Inap u : X  U
such that
f(x) + Au(x) E !vI + EBm
whenever x EX.
2. Let F : R n  Rm be a set-valued Inap. Its graph is assun1ed to
be closed and convex. Show that if F(xo) c Al Bn, !vI > 0, Xo E
int dOInF, then F is Lipschitzian in a neighborhood of Xo.
3. Prove that a set-valued n1ap F : R n  R m with cOlnpact convex
values is continuous if and only if the function x  S(y*,F(x)) is
continuous for all y* E R m .
4. Let F : [a, b]  Rn be a set-valued map with conlpact convex values.
Its Rieinann integral can be defined as a lilnit in the sense of the
Hausdorff distance of the integral sunlS I:k F(k)(tk+1 - tk), where
a == to < tl < ... < tN == b is a partition of the interval [a, b]
and k E [tk, tk+l]. Prove that the integral exists whenever F is
continuous ahnost everywhere. (Hint. Show that S(x*, J: F(t)dt) ==
J: S(x*, F(t))dt for piecewise set-valued Inaps, and use the previous
problem. )

62
2. Set- Valued Analysis
5. Let P and Q be two polytopes in R n , and let A(t) be an (n x n)
matrix depending on t E [0, T]. It is assumed that the components
of the matrix are analytical functions and that the set-valued map
*
F(t) == A(t)P - A(t)Q, t E [0, T] (see Probleln 1.5) is nonempty.
Show that F is continuous almost everywhere.
6. Construct an (n x n) matrix A(t), t E [0, T], with infinitely differen-
tiable components and two polytopes P and Q in R n such that the
*
set-valued map F(t) == A(t)P - A(t)Q, t E [0, T], is discontinuous in
a set of positive measure.
7. Find T+(O, A) and T_(O, A), where A c R is given by
(a) A == {O, 1, 1/2,... ,1/n,...},
(b) A == {O, 1, 1/2, . . . ,1/ 2 n , . . . },
(c) A=={0,1,1/2,... ,1/n!,...}.
8. Let a ( t) > 0, t E R, be a differentiable function, and let A ( t), t E R,
be a (m x n) matrix with differentiable components. Consider the
set-valued map F(t) == {x E R n I A(t)x E a(t)Bm}, t E R. Let
(t, x) E grF. Show that
DF(t, x)(l)
{ Rn IA(t)xl < a(t),
== {v I (A.(t)x + A(t)v, A(t)x) < a(t)a(t)}, IA(t)xl == a(t).
9. Let G : R  R m be a continuous set-valued map with compact convex
values, and let A(t), t E R, be a (m x n) matrix with differentiable
components. Set F(t) == {x E Rn I A(t)x E f G(s)ds}, t > O. It is
assumed that intf G(s)ds =1= 0, t > O. Prove the following equalities:
(a)
{ Rn A(t)x E int f G(s)ds,
DF(t, x)(l) == {v I A.(t)x + A(t)v
E T(A(t)x, f G(s)ds)} + G(t), otherwise,
(b)
{ Rn A(t)x E int f G(s)ds,
DF(t, x)( -1) == {v I A.(t)x + A(t)v + G(t)
C -T(A(t)x, f G(s)ds)}, otherwise,
whenever (t, x) EgrF, t > O.
10. Show that the set-valued map F : B 2  B 2 defined by F(x)
B 2 \ (x + B2) is continuous and has no continuous selector. (Hint.
Use the Brouwer theorem.)
11. Show that a function f : R n  R n satisfying j(O) == 0 and differen-
tiable at x == 0 has an inverse defined in a neighborhood of the origin,

2.8. Problems
63
provided that det\7 f(O) i- O. (Hint. Apply the Brouwer theorem to
the map x  x + (\7 f(O))-l(y - f(x)) with Ixl and Iyl sufficiently
small. )

Chapter 3
N onsmooth Analysis
In many branches of lnathelnatics it is necessary to study differential prop-
erties of nondifferentiable functiollS. For this reaSOll various generalizations
of classical calculus llave been developed. In this chapter we present non-
smooth calculus oriented to derive necessary conditions of optimality for
extremal problelns. NOllsmooth analysis considered here is a generalization
of convex analysis. We prove a separation theoreln for nonconvex sets and
then, following the convex analysis scheme, develop a nonsmooth calculus
and derive necessary conditions of opthnality for a general nonsmootll math-
ematical programlning problem.
3.1. Method of metric approximations
The method of metric approximations is used to derive necessary condi-
tions of optimality for general nonsmooth extremal problems. The main
idea of the lnethod is to approximate all constraints and the functional by
smootll distance-like functions in order to obtahl a slnooth unconstrained
optimization problem. Applying the Ferlnat theorenl and taking the limit
in necessary conditions for the unconstrailled probleln, Olle can derive nec-
essary conditions of opthnality for the original extremal problem.
The Lagrange multipliers rule. To illustrate the method of metric ap-
proximations, we derive the Lagrange multiplier rule for a mathematical
programming probleln with smooth data. Consider the minimization prob-
lem
-
65

66
3. Nonsmooth Analysis
(3.1)
f(x)  inf,
gi(x) == 0, i == 1, m ,
h j (x) < 0, j == 1, k ,
where f : R n  R, gi : R n  R, i == 1, m , fj : R n  R, j == 1, k, are
continuously differentiable functions.
Let A E R, J..l == (J..l1,... , J..lm) E R m , and v == (vI,... , v k ) E R k . Intro-
duce the Lagrange function:
L(x, A, J..l, v) == Af(x) + (J..l, g(x)) + (v, h(x)).
Theorem 3.1. Let x be a local solution of problem (3.1). Then there exist
A > 0, J..li, i == 1, m , and v j > 0, j == 1, k , satisfying the following conditions:
1. "\1xL(X,A,J..l,V) == 0,
2. vjhj(x) == 0, j == 1, k (the complementarity condition),
3. (A 2 + (J..l1)2 + . . . + (J..lm)2 + (v 1 )2 + . . . + (v k )2)1/2 == 1.
Proof. Let, < f(x). Set
[ m k ] 1/2
<I>(x,')') = (f+(x,')'))2 +  (gi(x))2 + H (h(x))2 ,
F(x, ,) == q>(x, ,) + Ix - x1 2 ,
where f+(x,,) == max{f(x) - "O}, h(x) == max{hj(x),O}. We observe
that q>(x, ,) > 0 for all x E R n and that F(x, ,)  00 as x  00. Con-
sequently F(x, ,) achieves its global minimum at a point x, thanks to the
Weierstrass theorem. Note that
(3.2) q>(x" ,) + Ix, - xl 2 == F(x" ,) < F(x, ,) == f(x) - ,.
If q>(x" ,) == 0, then x, satisfies all constraints of problem (3.1) and f(x,) <
, < f(x). Since x is a local solution to minimization problem (3.1), from
(3.2) we have q>(x" ,) > 0 for, sufficiently close to f(i). Since q>(x" ,) > 0,
the function F is differentiable in x at the point x,. Hence
(3.3)
"\1xF(x",) == O.
Set
. J
\ _ f+(x" ,) i _ g(x,) . _ - j _ h+(x,) . - - k
A')' - <I> ( ) , J-LI' - <I> ( ) '  - 1, m, v I' - <I> ( ) ' J - 1, ·
x" , x, , , x" ,

3.2. Mordukhovich normal cone
67
Then (3.3) can be written in the form
m k
(3.4) )",./\7 f(x"() + L J.l V' gi(x"() + L v? V' h j (x"() + 2(x"( - x) = O.
i=1 j=l
Note that
(3.5)
A, > 0, v > 0, j == 1, k,
and
( 3.6) ( ( A, ) 2 + (J-l) 2 + . . . + (J-lr;) 2 + (v  ) 2 + . . . + (v;) 2 ) 1/ 2 == 1.
Let'Y i f(x). Without loss of generality A, ---+ A, J-l ---+ J-li, v4 ---+ v j .
From (3.2) we have x, ---+ x. Taking the limit in (3.4)-(3.6), we obtain the
result. D
3.2. Mord ukhovich norlllal cone
Consider a set-valued map F : R n ---+ R m . The upper topological limit of
F(x) as X tends to Xo is defined by
limsupF(x) = n cl U F(x).
xxo €>o Ix-xol€
It is the set of all limiting points limioo Vi, where Vi E F(Xi) and Xi ---+ Xo
as 't ---+ 00.
Let A c R n be a nonempty set. Put P(x, A) == cone(x - 7f(x, A)) ==
Ua:>o a(x - 7f(x, A)). The Mordukhovich normal cone to A at X E A is
defined by
N(x,A) == limsupP(x',A).
X/x
Obviously N(x, A) is a nonempty closed cone.
Main properties. Let x E cIA. Define the inverse projection by
7f-1 (x, A) == {y I x E 7f(Y, A)}.
Lemma 3.1. Let x E cIA. Then the following equality holds:
N(x, A) == lim sup cone(7f-1(x, A) - x).
xx)
xEclA
Proof. Let x* == limkoo x k ' where x k E cone(7f-1(xk, A) - Xk), and Xk ---+
X, Xk E cIA. Observe that x k == Ak(Yk - Xk), where Ak > 0 and Yk E
7f-1(Xk, A). If Y E 7f-1(x, A), then we have ax + (1- a)y E 7f-1(x, A) for all

68
3. Nonsmootl1 Analysis
a E [0,1]. Therefore without loss of ge11erality IYk -xkl < Ilk. By definition
we have Yk - Xk E Yk - 7r(Yk, A). Thus we have Yk  x and
x* E lim sup cone(Yk -7r(Yk,A)) C N(x,A).
koo
Now, let x* E N(x,A) == limsupxxcone(x - 7r(x,A)). Then we have
x* == limkoo x k ' where x k == Ak(Xk - Wk), Wk E 7r(Xk, A), and Xk  x.
Since Xk - Wk E 7r- 1 (Wk, A) - Wk and Wk  x, Wk E cIA, we obtain the
result. D
Theorem 3.2. Let A c R n be a convex set, and let x E cIA. Then the
Mordukhovich normal cone to A at x coincides with the normal cone of
convex analysis.
Proof. Show that
N(x,A) == {x* I (x*,x - x) < 0, \Ix E A};
then the result will follow from Proposition 1.4. Let x* be such that (x*, x-
x) < 0 for all x E A. Let x E A. Then we have
Ix - x - x* 1 2 == Ix - xl 2 - (x*, x - x) + Ix* 1 2 > Ix* 1 2
for all x E A. Thus 7r(x*lk+x,A) == {x}, and therefore
x* == k(x*lk+x-7r(x*+x,A)) == lim k(x*lk+x-7r(x*+x,A)) E N(x,A).
koo
Let x* E N(x, A). Then we have x* == limkoo x k , x k E cone(7r- 1 (xk, A)
- Xk), and Xk  x, Xk E cIA thanks to Lemlna 3.1. Since A is convex,
from Zk E 7r- 1 (Xk, A) we obtain 7r(Zk, cIA) == {Xk}. Since x k == ak(zk - Xk),
ak > 0, we have (x k , X-Xk) < 0 for all x E A (see the proof of Theoreln 1.3).
Taking the limit, we obtain (x*, x - x) < 0 for all x E A. D
From Lemma 3.1 we derive the following property of the Mordukhovich
normal cone.
Theorem 3.3. Let x E cIA. Then the following equality holds:
N(x, A) == lim sup N(x, A).
xx,
xEclA
Denote the polar cone -(T_(x, A))* by N_(x, A). We need the following
lemma.
Lemma 3.2. For any x E cIA the following 'inclusion holds:
N_(x,A) c N(x,A).

3.2. lvlordul{hovich nornlal cone
69
Proof. Let x* E N_(x, A), and let Wk E 1f(x + x* /k, A). Since Ix + x* /k-
wkl 2 < Ix* /kI2, we obtain Ix - wkl 2 + 2(k- 1 x*, x - Wk) < O. Consequently
we have 2(x*, IWk - xl- 1 (Wk - x)) > klwk - xl. Witllout loss of generality
IWk - xl- 1 (Wk - x)  w. Since x* E N _ (x, A), we derive (x*, w) < O. Hence
klx - wkl  0 as k  00. Thus k(x + x* /k - Wk) == k(x - Wk) + x*  x*.
Tllis hnplies x* E N(x, A). D
Anotller useful represelltation of the 1Vlordukhovich normal cone is con-
tained in the following theorem.
Theorem 3.4. Let x E cIA. Then the following equality holds:
N(x,A) == limsupN_(x,A).
xx,
xEclA
Proof. By LeInma 3.2 and Theorem 3.3 we have
limsup N_(x, A) C N(x, A).
xx,
xEclA
Now let us prove the inclusion
cone(1f-1(x, A) - x) c N_(x, A).
Let Z E 1f-1(x, A), yEA. Then Iz - yl2 > Iz - x1 2 , and we obtain Iy - x +
x - zl2 > Ix - z12. Hence we have
(3.7) Iy - xl 2 > 2(y - x, z - x).
Any vector wET _ (x, A) n bdBn can be represented in the form
W == liln (y - x)/Iy - xl.
yx
yEclA
Dividing (3.7) by Iy - xl and taking the limit as y  x, we obtain z - x E
N_(x, A). Applyhlg LemIna 3.1, we obtain the result. D
In many problems we will have to find normal cones to Cartesian prod-
ucts.
Theorem 3.5. Let A C R n , BeRm, and let x E cIA, y E clB. Then
N((x, y), A x B) == N(x, A) x N(y, B).
Proof. Froln the definition of contingent cone we derive
T_((x, y), A x B) c T__(x, A) x T_(y, B)
and therefore we have
N_((x, y), A x B)  N_(x, A) x N_(y, B).

70
3. Nonsmooth Analysis
Applying Theorem 3.4, we obtain
N((x, y), A x B)  N(x, A) x N(y, B).
On the other hand, by definition we have
N((x, y), A x B) == limsup cone((x', y') - 7r((x', y'), A x B)).
(x' ,Y')-+(X,y)
It is easy to check that 7r((x,y),A x B) == 7r(x,A) x 7r(y,B). Therefore we
have
N((x, y), A x B) C limsup cone(x' - 7r(x', A)) x cone(y' - 7r(Y', B))
(x' ,Y')-+(X,y)
== limsup cone(x' - 7r(x', A)) x limsup cone(y' - 7r(Y', B))
x' -+x y' -+y
== N(x, A) x N(y, B).
The theorem is proved.
o
Consider a set A c R n and a linear operator A : R n  R n .
Theorem 3.6. Let x E A-1A. IfkerA == {O}, then
N(x, A-I A) == A * N(Ax, A).
Proof. Let x E A-I A. Then we have T_ (x, A-I A) == A -IT_ (Ax, A). In-
deed, by definition we have
T_(x,A-1A) == {v I liminfA- 1 d(x + Av,A- 1 A) ==O}
A!O
== {v lliminf A- 1 d(Ax + AAv, A) == O}
.-\!O
== {v I Av E T_(Ax,A)} == A- 1 T_(Ax,A)
and hence
coT_ (x, A -1 A) == A -lcoT_ (Ax, A).
We have (see the proof of Corollary 1.6)
(3.8)
N _ (x, A-I A) == cl(A * N_ (Ax, A)).
Applying Theorem 3.4, we obtain the result.
o

3.2. Mordukhovich norlnal cone
71
Adjoint set-valued maps. Consider a set-valued map F : R n  R m
with closed convex values. Let (x, v) E grF. The adjoint set-valued map
F*(x, v) : R m  Rn at the point (x, v) is defined by
F*(x,v)(v*) == {x* E R n I (x*, -v*) E N((x,v),grF)}.
The adjoint set-valued map is a nonconvex generalization of the adjoint
convex process. It provides a dual characterization of a set-valued map local
properties, while the derivatives give a primal description of the map local
structure.
Theorem 3.7. Assume that a set-valued map F : R n  R m is Lipschitzian
with a constant l > O. Then the following inclusion is fulfilled:
(3.9)
F*(x, v)(v*) C llv*IBn.
Moreover, the map (x, v, v*)  F(x, v)(v*) is upper semi-continuous.
Proof. Let (x*,-v*) E N((x,v),grF). By definition there exist sequences
ak > 0, (Xk,Vk)  (x, v) and (xk,v'k) E 7f((Xk,vk),grF) such that
(x*, -v*) == lim ak(xk - xk,Vk - v'k).
k--+oo
Let (x11",v11") E 7f((x,v),grF). To prove the theorem it is sufficient to show
that Ix - X 11" I < llv - v11"l. Suppose that Ix - X 11" I - llv - V 11" I > O. Consider
x, == x11" + ,(x - x11"), , > O. Let v, E 7f(v, F(x,)). Applying Proposition
2.6 we have
Iv - v, I == d( v, F( x,)) < Iv - v11" I + d( V 11" , F( x,))
< Iv - V 11" I + llx, - X 11" I == Iv - V 11" I + ,llx - X 11" I.
From this we obtain
I(x, v) - (x" v,)1 == (Ix - x,1 2 + Iv - v,12)1/2
< ((1 - ,)2Ix - X 11" 1 2 + (Iv - V 11" I + ,llx - X 11" 1)2)1/2
== (Ix - X 11" 1 2 + Iv - V 11" 1 2 + r(,))1/2,
where
r(,) == -2,lx - x11"I(lx - x11"l-llv - V 11" I) + ,21x - x11"12(1 + l2).
Obviously r(,) < 0 when, > 0 is sufficiently small. Thus there exists
, > 0 such that I (x, v) - (x" v,) I < I (x, v) - (x 7r , v11") I, a contradiction. Thus
inclusion (3.9) is proved.
By Theorem 3.3 the graph of the map (x, v, v*)  F*(x, v)(v*) is closed.
From (3.9) and Proposition 2.2 we derive upper semi-continuity of F*. 0
Now establish another property of adjoint maps.

72
3. NOllS1TIOoth Analysis
Theorem 3.8. Assume that a set-valued map F : R n  Rm is Lipschitzian.
Let (x, v) E grF, and let F*(x,v)(v*) i- 0. Then
S( -v*, F(x)) == -(v, v*).
Proof. Let x* E F*(x, v)(v*). Then we have (x*, -v*) E N((x, v), grF). By
definition of IVlordukhovich's normal cone there exist sequences (Xk, Vk) E
R n x R n , D:.k > 0, and (x'k,v'k) E 1f((Xk,vk),grF) such that Xk  x, Vk  v,
D:.k((Xk, Vk) - (xl;, v'k))  (x*, -v*) as k  00. Observe that
I(Xk, Vk) - (xl;, vk)1 < I(Xk, Vk) - (x'k, 1u)l, V w E F(xl;).
Consequently we have 1f(vk,F(x'k)) == {v'k} (see Len1ma 1.1). He11ce we
obtain
(D:.k(Vk - v'k), w - v'k) < 0, Vw E F(x'k).
Since F is Lipschitzian, taking the lhnit, we have
(v*,w - v) > 0, V w E F(x).
Thus we get
S(-v*,F(x)) == -(v,v*).
This ends the proof.
D
3.3. Separation theorelll for nonconvex
sets
Obviously, the separation theorem for nonconvex sets can be of local nature
only. For this reason we introduce the notio11 of an extremal collection of
sets.
Let Ai eRn, i == 1, m , i > 2, be nonelnpty sets. We say that the
collection {Ai} in 1 is extrelnal at a point x E n  1 clA i if there exist se-
quences {ai,k} r l' ai,k ERn, i == I,m , and {Wk} r l' Wk > 0, such that
lilnk-+oo ai k == 0, i == 1, m , limk-+oo Wk == 00, and
,
m
n(clA i - ai,k) n(x + 'YkBn) = 0, k = 1,2,. .. ,
i==l
where'rk == Wk(I:  1I a i,kI 2 )1/2.
The following result is a nonconvex version of the separation theorem
(Theorem 1.5) proved in the first chapter.
Theorem 3.9. Let the collection of sets {Ai} i 1 be extremal at a point
x E n  1 clA i . Then there exist xi E N(x, Ai), i == 1, m , such that
m
m
L xi = 0 and
i==l
L Ixil 2 = 1.
i==l

3.3. Separation tl1eorelll for nonconvex sets
73
Proof. Si11ce the collection {Ai} in 1 is extren1al at x E n n 1 clA i , there exist
sequences {ai,k} r l' ai,k ERn, i == I,m , and {Wk} r l' Wk > 0, such that
lin1k--4oo ai,k == 0, i == 1, m , Ih11k--4oo Wk == 00, and
m
n(clA i - ai,k) n(i; + 'YkBn) = 0, k = 1,2,. . . ,
i=l
where 'rk == Wk(L:  1 lai,kI 2 )1/2.
Fix positive integers k and p and consider the function
( 1n ) 1/2
<l>(x) = L d 2 (x + ai,k> Ad + d 2 (x, X + 'YkBn) + Ix - xl.
i=l p
Since .p(x)  00 as x  00, by the Weierstrass theorelll there exists Xp,k E
R n such that
.p(Xp,k) < .p(x), for all x ERn.
Let Wp,i,k E 7r(X p ,k + ai,k, Ai), i == 1, m , and Wp,O,k E 7r(X p ,k, X + /'kBn). Put
aD k == 0 and consider the function
,
( m ) 1/2
w(x) =  Ix + ai,k - w p ,i,kI 2 +  Ix - xl.
=o
Obviously W(Xp,k) < w(x) for all x ERn. Set
( m ) 1/2
O'.p,k =  IXp,k + ai,k - W p ,i,kI 2
=o
Observe that Qp,k > O. Suppose that Xp,k =I=- x. Then W is differe11tiable at
Xp,k. Hence
m '"
L * Xp,k - X
V W(X p k)== X p ik+ I AI =O,
, " p x k-X
i=O p,
where x;,i,k == Q;'k(Xp,k + ai,k - Wp,i,k), i == 0, m . Thus we have
m m
(3.10) '" X p * i k E Bn and '" I x p * i kl 2 == 1.
 "p "
i=O i=O
If Xp,k == x, then we can obtain (3.10) using the function
( m ) 1/2
w(x) =  Ix + ai,k - W p ,i,kI 2
=o
(3.11)
Observe that
( m ) 1/2
IXP,k - xl < <l>(Xp,k) < <l>(x) < L lai,kl 2
p i=l

74
3. NOllS1ll00th Analysis
and
IX;,o,kl == a;Ld(xp,k' x + kBn) == In ax { a;L(lxp,k - xl - k), O}
< max { a;,l(p - Wk) (  la i 'kI 2 ) 1/2 ,0 } .
Let k ---+ 00. Then we have IX;,o,kl == 0 whenever k is sufficiently large.
Without loss of generality x;,i,k ---+ x;,i' i == 1, m . Taking the limit in (3.10),
we derive
(3.12)
m m
L X;,i E  En and L IX;,iI 2 = 1.
i=l i=l
Since Xp,k + ai,k ---+ X (see (3.11)), by definition of the Mordukhovich
cone we have
(3.13) X;,i E N(x, Ai), i == 1, m .
Without loss of generality x;,i ---+ xi when p ---+ 00. Taking the limit in (3.12)
and (3.13), we obtain
m m
LX; = 0, L Ix;12 = 1, and x; E N(x,A), i = I,m.
i=l i=l
The theorenl is proved.
D
Let us consider an example that shows 110W the separation theorein can
be used in nonsinooth analysis. We need the following auxiliary result.
Lemma 3.3. Let x E cIA eRn, and let x* E N_(x,A). Then, given E > 0,
there exists  > 0 such that
(x*, Y - x) < Ely - xl
for all yEA n (x +  Bn).
Proof. Suppose that there exist E > 0 and a sequel1ce Yk ---+ X such that
(x*, Yk - x) > EIYk - xl.
Dividing by IYk - xl, we obtain
(x*, (Yk - x)/IYk - xl) > E.
Without loss of generality (Yk - x)/IYk - xl ---+ w E bdBn. Taking the limit,
we obtain (x*,w) > E and w E T_(x,A), a contradiction. D
N ow consider a set A c R n and a linear operator A : R m ---+ Rn.

3.4. Nonsnlooth calculus
75
Theorem 3.10. Assume that A is closed. Let x E A-I A, and let N(Ax, A)
n kerA* == {O}. Then the following inclusion holds:
N (x, A -1 A) c A * N (Ax, A) .
Proof. Let x* E N(x, A-I A). Since A is closed, the set A -1 A is also closed.
By Theorem 3.4 there exist sequences xk  x, xk E A-I A and x k  x*
such that x k E N-(Xk, A-I A). Consider the sets
A k ,l == R m x A x R+,
A k ,2 == {(X, Ax, (Xk,X - Xk)) E R m x R n x R I x E R m }.
By Lemma 3.3 there exist 7p > 0, p == 1,2, . .. , such that
(xX:, x - Xk) < Ix - xkl
p
for all x E A-I A n (Xk + 7pBm). Thus the collection of sets {A k ,l, Ak,2}
is extremal at the point (Xk, AXk, 0) E Ak,l n A k ,2. By Theorem 3.9 there
exists a vector (zZ, yZ, JLk) such that l(zZ, yZ, JLk) I == 1,
( z Z, y Z, JL k) E N ( ( x k, Ax k, 0) , A k, 1 ) ,
and
-(zZ, yZ, JLk) E N( (Xk, AXk, 0), A k ,2).
Applying Theorem 3.5, we see that zZ == 0, yZ E N(Axk, A), and JLk < O.
From Theorem 3.2 and Proposition 1.4 we have
(A(x - Xk), yZ) + JLk(xk, x - Xk) < 0, \Ix E R m .
Hence A *yZ == -JLkxk. Without loss of generality tIle sequellce (yZ, JLk)
converges to a vector (y*, JL*). Obviously I (y*, JL*) I == 1 and JL* < o. Suppose
that JL* == o. Then y* E N(Ax, A) and A *y* == 0, a contradictioll. Thus
JL* < 0, and we obtain
x* == IJL*I- 1 A*y* E A*N(Ax,A).
This ends the proof.
D
3.4. Nonsrnooth calculus
Let f : R n  R U {:i::oo} be a function. Assume that If(x)1 < 00. The
M ordukhovich subdifferential of f at x is defined by
8f(x) == {x* I (x*, -1) E N((x, f(x)), epif)}.
If f is a convex function, then from Theorem 3.2 we see that the Mor-
dukhovich subdifferential coincides with the usual subdifferential of convex
analysis (see Remark at the end of Chapter 1).

76
3. NOnS1TIOoth Analysis
The singular subdifferential of f at x is defined by
as f(x)  {x* I (x*, 0) E N((x, f(x)), epif)}.
Frol11 Theoreln 3.3 we obtain the following result.
Theorem 3.11. Let f : R n  Rn {:!::oo} be lower semi-continuous, and let
If(x)1 < 00. Then the following equality holds:
af(x)  Ii!nsup af(x').
X/X
,
f(x') f(x)
Let A c R n . Fro111 the definitions we derive
a8(5:, A)  a S 8(5:, A)  N(5:, A).
The lower and upper Dini derivatives are defined by
D - f( )( ) 1 . . f f(x + av') - f(x)
x v  I1111n
alO,v/v a
and
. f(x + av') - f(x)
D+ f(x)(v) == hn1sup
alO,v/v a
respectively. In this chapter we shall use the properties of lower Dini deriv-
ative.
Proposition 3.1. Let f : R n  R U { +oo} be a function and x E d0111f.
Then
epiD- f(x)  T_((x, f(x)), epif).
If f is Lipschitzian, then
D - f( )( ) 1 . . f f(x + av) - f(x)
x v  Iln In .
alO a
Proof. Follows in1111ediately fro111 definitions.
D
The lower Dini subdifferential of f at x is defined by
a-f(x)  {x* I (x*,v) < D-f(x)(v), \Iv ERn}.
Froln Proposition 3.1 we see that
a- f(x)  {x* I (x*, -1) E N_((x, f(x)), epif)}.
Obviously N_ (5:, A)  a- 8(5:, A) whe11ever x E A.
FrOln Theoreln 3.4 we obtai!1 the following result.

3.4. Nonsmootl1 calculus
77
Theorem 3.12. Let f : R n  R n {:!:oo} be lower semi-continuous, and let
If(x)1 < 00. Then the following equality holds:
8 f(x) == li!n sup 8- f(x').
x' x,
f(x') f(x)
Corollary 3.1. Assume that f satisfies the Lipschitz condition with a con-
stant L > o. Then 88f(x) == {O} and 8f(x) C LBn for all x ERn.
Proof. By definition 8- f(x) == {x* I (x*,v) < D- f(x)(v), V v ERn}.
Since D- f(x)(v) < Llvl, froln Tl1eorelns 3.4 and 3.12 we obtai!1 the result.
o
We need t11e following technicallemlna.
Lemma 3.4. Let Y be a compact set, and let f : R n x Y  R be a con-
tinuous function. Assume that f is differentiable in x and that \7 xf is
continuous in (x, y). Then the function
fo(x) == Inax{f(x, y) lyE Y}
is directionally differentiable and
Dfo(x)(v) == Inax{(\7 x f(x,y),v) lyE Y(x)},
where Y(x) == {y E Y I fo(x) == f(x,y)}.
Proof. Let Yo E Y(xo). Then we have
fo(xo + AV) - fo(xo) > f(xo + AV, Yo) - f(xo, Yo).
Dividing by A and taking tl1e li!nit as A 1 0, we obtai!1
1 . . f fo(xo + AV) - fo(xo) > (  f( ) )
un In \ _ V X XO, Yo , v .
.-\10 /\
Since Yo E Y(xo) is arbitrary, we 11ave
lim inf fo(xo + ).. - fo(xo) > max{ (\7 xf(xo, y), v) lyE Y(xo)}.
.-\10
Let y E Y(xo + AV). Then we have
fo(xo + AV) - fo(xo) < f(xo + AV, y) - f(xo, y).
Applying the Inean-value theoreln we see t11at there exists (y) E [0, A] such
that
fo(xo + ).. - fo(xo) < (\7 xf(xo + (y)v, y), v).
Consequently
1 . fo(xo + AV) - fo(xo)
un sup \
.-\10 /\

78 3. Nonsmooth Analysis
(3.14) < Ihnsupmax{(\7 x f(xo + (y)v,y),v) lyE Y(xo + AV)}.
AlO
Let Ai 1 0, i E]O, Ai[, and Yi E Y(xo + AiV) be such that
lhnsup max{ (\7 xf(xo + (y)v, y), v) lyE Y(xo + AV)}
Alo
== ,lim (\7 xf(xo + iV, Yi), v).
oo
Since Y is compact, without loss of generality Yi  Yo E Y. Since f(xo +
AiV,Yi) > f(XO+AiV,y) for all Y E Y taking the limit, we obtain f(xo,yo) >
f(xo, y) for all y E Y. Hence Yo E Y(xo). From (3.14) we derive
lim sup fo(xo + A - fo(xo) < max{ (\7 xf(xo, YO), v) lyE Y(xo)}.
AlO
This ends the proof. 0
Theorem 3.13. Let x E cIA. Then the following equality holds:
N(x, A) == cone {) d(x, A).
Proof. It is easy to see that
8-d(x, A) == N_(x, A) n Bn.
Let x (j. cIA and 1r(x, A) == {w}. From Lemma 3.4 we see that the function
x  d(x,A) == -max{-Ix - all a E cIA} is differentiable and \7d(x,A) ==
(x - w)/Ix - wi. Moreover, observe that if 1r(x, A) contains more than one
point, then 8- d(x, A) == 0. Invoking Theorems 3.4 and 3.12, we obtain
8d(x, A) == N(x, A) n Bn.
This ends the proof.
o
Now we establish a property of the Dini subdifferential needed later.
Lemma 3.5. Let E > o. If x* E 8- f(x), then there exists"Y > 0 such that
f(x) < f(x) - (x*, x - x) + Elx - xl
for all x E x + "YBn.
Proof. Suppose that there is a sequence Xk  x such that
f(x) > f(Xk) - (x*, Xk - x) + Elxk - xl.
Dividing by IXk - xl, we obtain
(3.15) (x*, (Xk - X)/IXk - xl) > IXk - xl- 1 f(x + (Xk - X)/IXk - xl) + E.
Without loss of generality (Xk - X)/IXk - xl  v. From (3.15) we have
(x*, v) > D- f(x)(v) + E,
a contradictiol1.
o

3.4. NOllsmooth calculus
79
Now we are in a position to prove a nonconvex version of the ]Vloreau-
Rockafellar theoren1.
Theorem 3.14. Let 11 : R n  R U {:i:oo} and 12 : R n  R U {:i:oo} be
lower semi-continuous functions. Assume that
8 S 11 (x) n (-8 S 12(X)) == {O}.
Then the following inclusion holds:
8(11 + 12)(X) C 811 (x) + 812(X).
Proof. Put I == 11 + 12. Let x* E 81(x). By Theorem 3.12 there exist
sequences Xk  x and xk  x* such that xk E 8- I(Xk) and I(Xk)  I(x).
Consider the sets
A k ,l == {(x,J-l) E R n x R I J-l > 11 (X) - 11(Xk)},
A k ,2 == {(x, J-l) E R n x R I J-l < - 12(X) + 12(Xk) + (Xk, x - Xk)}.
By Lemma 3.5 there exist 'rp > 0, P == 1,2, . . . , such that
f(Xk) < f(x) - (xL x - Xk) + !Ix - xkl
p
for all x E Xk + 'rpBn. Thus the collection of sets {A k ,l, Ak,2} is extremal at
the point (Xk, 0) E Ak,l nA k ,2. By Theorem 3.9 there exists a vector (zic, J-lk)
such that I (zic, J-l k) I == 1,
(zic, J-lk) E N((Xk, 0), Ak,l),
and
-(zic, J-lk) E N((Xk, 0), Ak,2).
Since A k ,2 is an affine transformation of the set epil2, invoking Theoreln 3.6,
we obtain
(zic,J-lk) E N((Xk,11(xk)),epil1)
and
(-J-lkXk - zic, J-lk) E N((Xk, 12(Xk)), epiI2).
Without loss of generality (zic,J-lk)  (z*,J-l*). Taking the limit, we derive
( z * , J-l *) E N ( ( x, 11 ( x) ), ep ill) , and (- J-l * x * - z * , J-l *) E N ( ( x, 12 ( X ) ) , epi 12) .
If J-l* == 0, we obtain a contradiction. Thus we have J-l* < 0 and
1J-l*1-1Z* E 811(X), x* -1J-l*1-1z* E 812(X).
The theorem is proved.
o

80
3. Nonsl1100tl1 Analysis
Corollary 3.2. Let Ai C R n , i == 1, m , be nonempty sets and x E n n 1 clA i .
Assume that the system of inclusions
m
x; E N(x,A i ), i = I,m , LX; = 0
i==l
has only the trivial solution xi == 0, i == 1, m . Then the inclusion
(3.16)
( m ) 1n
N x, n clA i C h N(x, A)
holds.
Proof. C011sider tl1e functio11S fi (x) == 8 (x, cIA i ), i == 1, m . Since 8 fi (x) ==
8 s fi(X) == N(x, Ai), i == 1, m , applying Tl1eorelll 3.14, by induction we
obtain
m m
8 S L fi(X) C L 8 s fi(X)
i==l i==l
a11d
m 1n
8 L fi(X) C L 8!i(x).
i==l i==l
Botl1 i11clusions are equivale11t with (3.16).
D
Corollary 3.3. Let A == {x I f(x) < O}, where f : R n  R is a Lips-
chitzian function. Assume that f(x) == a and a tJ. 8f(x). Then the following
inclusion holds:
N(x, A) c cl cone 8f(x).
Proof. By Theoren1 3.4 we have
N((x, 0), epif) == limsup N_((x, f(x)), epif).
x--+x
Observe that
N - (( x, f (x)), epif) == cl cone (8- f (x), -1) c cl cone (LBn, -1),
where L is t11e Lipschitz constant (see Corollary 3.1). Hence we 11ave
N((x, 0), epif) == cl C011e (8f(x), -1).
Observe that the sets Al == epif a11d A 2 == {(x, {t) E R n x R I {t == a} satisfy
all C011ditio11S of Corollary 3.2. Therefore we obtain
N(x, A) x R == N((x, 0), A x {O}) == N((x, 0), Al n A 2 )
c N((x, 0), epif) + {a} x R == cl cone (8f(x), -1) + {a} x R.
Thus we have N(x, A) c cl C011e 8f(x). D

3.4. N0l1S1TIOotll calculus
81
Consider Lipschitzian functions fi : Rn  R, i == 1, m . Put
f ( x) == nlax {f 1 ( X ), . . . , f1n ( X ) }
and
I ( X) == {i I f ( x) == fi ( X ) } .
Corollary 3.4. The following inclusion holds:
8 f ( x) C co { 8 fi ( x) liE I ( x ) } .
Proof. Observe that
m
epif = n epiJi.
i==l
Applying Corollaries 3.1 and 3.2, we obtain the result.
o
Theorem 3.15. Let f(x) == <p(Ax), where <p : R n  R is a Lipschitzian
function and A : R m  Rn is a linear operator. Then
8f(x) c A*8<p(Ax).
Proof. Let x* E 8f(x). By Theoreln 3.12 there exist sequences Xk  x and
x k  x* such that x k E 8- f(Xk) and f(Xk)  f(x). Consider the sets
A k ,l == {(x,Y,J.L) E R 1n X R n x R I J.L > <p(x)} == R m x epi<p,
A k ,2 == {(X, Ax, (Xk,x - Xk) + f(Xk)) E R m x R n x R I x E R 1n }.
By Lenllna 3.5 there exist !p > 0, P == 1,2, . . . , such that
f(Xk) < f(x) - (Xk, x - Xk) + Ix - xkl
p
for all x E x k + !pBn. ThllS the collection of sets {Ak, 1, A k ,2} is extrelnal
at the point (Xk, AXk, f(Xk)) E A k ,l n A k ,2. From Theorenl 3.9 we see that
there exists a vector (zk' Yk' J.Lic) such that I (zk' Yk' J.Lk) I == 1,
(zk, Yk, J.Lk) E N((Xk, AXk, f(Xk)), A k ,l),
and
-(Zk' Yk, J.Lk) E N((Xk, AXk, f(Xk)), A k ,2).
Applying Theoreln 3.2 and Proposition 1.4, we obtain
N( (Xk, AXk, f(Xk)), A k ,2) == {(z*, y*, J.L*) I z* + A *y* + J.L*xk == O}.
FrOln Theorenl 3.5 we get zk == 0 and (Yk,J.Lk) E N((Axk,f(xk)),epi<p).
Therefore A *Yk == -J.Lkxk. Without loss of generality tIle sequence (Yk' J.Lk)
converges to a vector (y*, J.L*). Then we have A *y* == -J.L*x*, I (y*, J.L*) I == 1,
and (y*, J.L*) E N((Ax, <p(Ax)), epi<p). By Corollary 3.1 J.L* =1= O. Conse-
quently J.L* < 0, and we get
x* == IJ.L* 1-1 A *y* E A * 8<p(Ax).
This ends the proof.
o

82
3. NonSlllooth Analysis
3.5. Lagrange multipliers
Let f : Rn  R be an arbitrary function. The following result is a non-
Sillooth analogue of the Fermat theorem.
Theorem 3.16. Let x be a point where f achieves its local minimum. Then
o E {)f(x).
Proof. Observe that 0 E {)- f(x)
Lemma 3.2 {)- f(x) C {)f(x).
{x* I (x*,v) < D- f(x)(v)}. By
D
Now consider the following problem:
(3.17)
f(x)  inf,
m
X E n Ai.
i=l
We assume that f is Lipschitzian and the sets Ai, i == 1, m , are closed. The
following theorem contains necessary conditions of optimality for problem
(3.17) .
Theorem 3.17. Let x be a local solution of (3.17). Then there exist A > 0
and xi E N(x, Ai), i == 1, m , such that
m
o E 'xof(x) + LX;
i=l
and
m
,x+ Llx;1 > o.
i=l
Proof. Suppose that there exist xi E N(x, A) such that
m
m
LX; = 0 and L Ix;1 > O.
i=l i=l
Then we put A == 0 and obtain the result.
Now assume that the system of inclusions
m
x; E N(x,Ad, i = I,m , LX; = 0
i=l
has only trivial solution xi == 0, i == 1, m . Consider the function
m
<I> (x ) = f(x) + L 8(x, Ai).
i=l

3.6. Problems
83
Obviously <I> has a local minimum at x. By Theorems 3.16 and 3.14 and
Corollary 3.2 we have
o E 8<I>(x) c 8j(x) + 8  8(x, Ai) = 8j(x) + 88 (x, n A)
= 8j(x) + N (x, n ai) C 8j(x) +  N(X,Ad.
This ends the proof. 0
3.6. Problems
1. Let f : R n -7 R be a convex function. Prove that the Mordukhovich
subdifferential of f coincides with the subdifferential of convex anal-
YSIS.
2. Let A eRn. The Clarke tangent cone to A at x E A is defined by
TCI(X, A) == { V I lim A -ld(y + AV, A) == O } .
y----+-x
A!O,yEA
Show that the cone TCI(X, A) is always convex. Prove that the Clarke
normal cone NCI(X, A) defined as the polar cone of the Clarke tangent
cone coincides with the cone cl coN(x, A).
3. A closed convex cone KeRn is said to be a tent to the set A
at x E A if there exists a continuous map r : K -7 Rn such that
x+x+r(x) E A, x E K, and r(x)/lxl-7 0 as x -7 O. Let cjJ: R n -7 R
be a locally Lipschitzian function, and let 'lj; : R n -7 R be a positively
homogeneous convex function satisfying epi'lj; C T + (( x, cjJ( x) ), epicjJ).
Prove that epi'lj; is a tent to epicjJ at (x, cjJ( x)).
4. Let A c Rnbeaclosedset, and let x E A. AssumethatintTcI(x,A) =I-
0. Prove that any convex cone satisfying K c T+(x, A) is a tent to
A at x. (Hint. Show that in a neighborhood of x the set A can be
represented as the epigraph of a locally Lipschitzian function.)
5. Let F : R n -7 R n be a Lipschitzian set-valued map with convex
compact values. Consider the map
F(x,x O ) == {vo(v, 1) E R n x R I v O E [a,b], v E F(x)}.
Prove that (x*, x O *, v*, v o *) E gr co F* (x, x O , v, v O ) (.) if and only if
(x*, v*) E gr co F* (x, v) (.), x o * == 0, and (v*, v) + v o * == O. (Hint. Use
Theorems 3.4 and 3.7.)

84
3. Nonsnl00tl1 Analysis
6. Consider the set A == {x E Rn I fk(X) < 0, k == I,m , fk(X) == 0, k ==
m + 1, m + p}, where the functions fk : R n  R n , k == 1, m + p,
are continuously differentiable. Assulne that the gradients \7 fk(X),
k == m + 1, m + p, are linearly independent and there exists a vector
y E Rn such that (\7 fk(X), y) < 0 , k E {k == 1, m I fk(X) == O}, and
(\7 fk(X), y) == 0, k == m + 1, m + p. Prove that
N(x, A) == N_(x, A)
{ m+p }
= x* = 2:: Ak \J fk(x) I Ak > 0, Akfk(x) = 0, k = 1, m ·
k==l
7. Let A == {x E R n I fk(X) < 0, k == I,m }, where t11e functions
fk : R n  R n , k == 1, m , are locally Lipschitzian. Assume that
o tJ: co{8fk(X) I k E K(x)}, where K(x) == {k == I,m I fk(X) == O}.
Prove that the cone N(x, A) is contained in the set
U {  Ak 8 fk(x) I Ak > 0, Akfk(x) = 0, k = 1, m } ·
8. Let F : R n  R m be an upper semi-continuous set-valued map with
compact values, and let 9 : R m  R be a locally Lipschitzian function.
Set f(x) == min{g(y) lyE F(x)}. Prove the inclusion
8f(x) c U{F*(x, y)(y*) I y* E 8g(y), y E F(x), f(x) = g(y)}.
(Hint. Evaluate the Mordukhovich subdifferential of the function
9 (y) + 6 ( ( X, y) , gr F) . )
9. Let F : R n  R m be an upper semi-continuous set-valued map with
compact values. Consider the function f(x) == d(z, F(x)). Show t11at
8f(x) c U{F*(x, y)(y*) Ily*1 < 1,
(y*, y - z) == Iy - zl, y E F(x), f(x) == Iy - zl}.
10. Consider the Inathematical programming problem
fo(x)  inf,
fk(X) < 0, k == 1, m ,
fk(X) == 0, k == m+ 1, m+p,
where the functions fk : R n  R, k == 0, m + p, are locally Lips-
chitzian. Let x E Rn be a local solution to this p roblem. Prove that
t11ere exist Lagrange multipliers Ak, k == 0, m + p, satisfying
(a) 0 E 2:  0 Ak 8 fk(X) + 2: ;+ +1IAkl(8fk(X) U 8( - fk)(X));
(b) Ak > 0, k == O,m ;
(c) Akfk(X) == 0, k == I,m ;
(d) 2: ;+6 A == 1.

Part 2
Differential Inclusions

Chapter 4
Existence Theorems
In this chapter we introduce the concept of solution to a differential inclusion
x(t) E F(x(t))
and prove the main existence theoreins used throughout the text. If we deal
with an ordinary differential equation
x(t) == f(x(t))
with continuous right-hand side, it is natural to define solutions as contin-
uously differentiable functions satisfying the equation in all points of SOlne
tilne interval. In many applied problems we have to consider differential
inclusions with an upper semi-continuous right-hand side, but the class of
continuously differentiable functions is not large enough to guaralltee the
existence of solutions. For example, the Cauchy problem
{ I,
x E [-1,1],
-1
,
x(O) == Xo
has no continuously differentiable solution whenever Xo =I- 0 and T > Ixol.
In order to prove the existence for such problenls we define solutions in the
class of absolutely continuous functions, that is, in the class of functions
with Lebesgue integrable derivative satisfying the relation
x < 0,
x==O
,
x > 0,
t E [0, T],
x(tI) - x(to) = {t! x(t)dt.
Jto
In the first section we recall definitions and results concerning the absolutely
continuous functions, which are needed later.
-
87

88
4. Existence Theorenls
'Ve then prove a result on the existence of continuous lnaps fro111 the
space of absolutely continuous functions to the set of solutions to a differ-
ential il1clusion with Lipschitzian right-hand side (Theoreln 4.5). Roughly
speaking, this theore111 shows that the space of absolutely continuous func-
tions can be continuously 'projected' onto tl1e set of solutions. We shall
frequently use this result to construct fan1ilies of solutions with given prop-
erties. Nalnely, we shall 'project' fan1ilies of functions which are 'quasi'-
solutions and have the required properties on the set of solutions. In lnany
cases this gives the desired result.
We also study three types of approxi111atiol1s for differential inclusions.
The first one is an approxilnation of upper selni-continuous differential in-
clusions by Lipschitzian ones. The second type is a local approxhnation of
a Lipschitzian differential inclusion in a neighborhood of a given solution
(variational differential inclusiol1). And finally for Lipschitzian differential
inclusions we consider discrete-tilne approxiI11ations. These approxhnatiol1
techniques are the 111ain tools used in the boole N an1ely, we approxilnate
differential inclusions under consideration by differential or discrete-tiIne in-
clusions with sin1pler structure. We then solve the probleln for the si111ple
inclusion and, passing to the lin1it, obtain the result for the original probleln.
This is the central idea of lnany proofs in later chapters. This 111ethod also
silnplifies the proofs of SOlne classical theorelns fro111 the theory of ordinary
differential equations, like the Hukuhara and Kneser theoren1s.
We apply tl1e developed techniques to study differential equations with
discontinuous right-hand side and to prove the existence theorelns in opthnal
control proble111s.
4.1. Background notes
Let us recall SOlne notations and results we use in the book. The Banach
space of integrable functions x : [0, T]  Rn with the norln
Ix(-)ILI = 1 T Ix(t)ldt
is denoted by £1 ([0, T], R n ). The Hilbert space of lneasurable functions
x : [0, T]  R n sucl1 that
1 T Ix(t)1 2 dt < 00
is denoted by £2 ([0, T], R n ). The il1ner product is defined by the forlnula
(x ( . ), y ( · )) = 1 T (x (t), y (t) ) dt.

4.1. Bacl{ground notes
89
The Banach space of continuous functions x : [0, T]  R n \\Tith tIle 110rn1
I x ( . ) Ie == n1ax { I x ( t ) I I t E [0, T]}
is denoted by C([O, T], R n ).
Recall that a function x : [0, T]  R n is called absolutely continuous if,
given E > 0, there is () > 0 such tl1at for any cou11table collection of disjoint
subintervals [t, t%] of [0, T] satisfying
I) t% - tk) < 8
,ve have
L Ix(tZ) - x(t)1 < E.
An absolutely continuous function is continuous and has bounded variati011.
Any Lipscl1itzian function is absolutely continuous. An absolutely conti11U-
ous x : [0, T]  R n is differentiable ahnost everY\\Tl1ere, and its derivative
xC) is a Lebesgue integrable functi011. lVloreover, the Ne,vton-Leibniz for-
n1ula is true; tl1at is,
t"
x( t") - x( t') = r x( t)dt
it'
for all t', t" E [0, T], t' < t". Hence any absolutely continuous function
x : [0, T]  R n can be represented in the for1n
x(t) = x(O) + I t x(s)ds.
We denote tIle space of absolutely continuous functions x : [0, T]  R n with
the norn1
Ix(')IAC = Ix(O)1 + I T Ix(t)ldt
by AC([O, T], R n ). Obviously it is ison1etrically ison10rphic to the Cartesian
product R n x £1([0, T], R n ) and tl1erefore is a Banacl1 space.
Later we often use tIle following results. The first one is a well-kno,vn
Arzela-Ascoli compactness tl1eoren1. Recall tl1at a set X C C([O, T], R n ) is
called equicontinuous if, given E > 0, tl1ere exists () > 0 such tl1at Ix( t") -
x( t') I < E for eacl1 x(.) E X whenever t', t" E [0, T] a11d It II - t'l < ().
Theorem 4.1 (Arzela-Ascoli). If a set X C C([O, T], R n ) is bounded and
equicontinuous, then it contains a uniformly convergent sequence Xi (.) E
X, i == 1,2,. . .,. that is, there exists xC) E C( [0, T], R n ) such that IXi C) -
x(.)lc  0 as i  00.
In particular this theore1n implies tl1at a bounded set of absolutely con-
tinuous functions X such that Ix(t)1 < b for all x(.) E X contains a unifor111ly
convergent subsequence.

90
4. Existence Tl1eorel11s
The next two results relate the L 1 -col1vergence and the pointwise con-
vergence.
Theorem 4.2 (Lebesgue). Assume that a sequence Xi(.) E L 1 ([0,T],R n )
converges to afunctionx(-) almost everywhere and IXi(-)1 < cP(t), t E [O,T],
i == 1,2,..., where cP(-) E L 1 ([0, t], R). Then x(.) E L 1 ([0, T], R n ) and the
sequence Xi (-) converges to x(-) in L1 -norm.
Theorem 4.3. Assume that a sequence Xi(-) E L 1 ([0,T],R n ) converges to
a function x(.) E L1 ([0, T], R n ) in L 1 -norm. Then there exists a subsequence
{Xip(-)}  1 converging to x(.) almost everywhere in [0, T].
To estiInate a solution of a differential inclusiol1, it is useful to have the
following result.
Theorem 4.4 (Gronwall's inequality). If a(.) E AC([O, T], R) satisfies
I a ( t ) I < l ( t ) a ( t) + P ( t ) , t E [0, T],
where l (-) E L 1 ([0, T], R), l( t) > 0, and p(-) E L 1 ([0, T], R), then
la( t) I < eJ l(s)ds la(O) I + I t Ip( s) leJ l(r)dr- J; l(r)dr ds, t E [0, T].
4.2. Lipschitzian differential inclusions
Consider a differential inclusion
(4.1)
x ( t) E F ( x ( t ) ), t E [0, T],
where F : R n  R n is a set-valued l11ap. By S[r,T] (F, xo) we denote the
set of solutions to inclusion (4.1) with the initial condition XO, that is, the
set of functions x(-) E AC([T, T], R n ) with X(T) == Xo satisfying (4.1) ahnost
everywhere. We shall olnit the index [T, T] if it does not cause any confusion.
If CeRn, then we put
S(F, C) = U S(F, x).
xEC
If C == R n , then we shall write
S(F) == S(F, R n ).
Throughout this section we assun1e that F satisfies the following condi-
ti 0 ns :
1. the sets F ( x) are closed and convex for all x E R n ;
2. the set-valued n1ap F is Lipschitzian with a constant l > o.
Such differential inclusions are called Lipschitzian.

4.2. Lipschitzian differential inclusions
91
Control systems. Many examples of Lipschitzian differential inclusions
are provided by control theory. Consider a differential equation depending
on a parameter u E U C R k ,
(4.2)
x == f(x, u),
where f : R n x U  R n . In what follows (4.2) will be referred to as a
control system, while u will be called the control. The cOlltrol u is thought
of as a function of t. We assume that f is a continuous function satisfying
Lipschitz condition in x and such that the set f(x, U) is closed and convex
for all x E R n . By an admissible control we mean any nleasurable function
u : [0, T]  R k satisfying the inclusion u(t) E U almost everywhere and
such that the function t  f(x, u(t)) is Lebesgue integrable for all x ERn.
Thus, for example, all bounded measurable functions u(.) satisfyhlg u(t) E U
almost everywhere are admissible controls. If u(.) is an adlllissible control,
then the Cauchy problem
x == f(x, u(t)), x(O) == Xo
has a solution x(-) which is referred to as a trajectory of system (4.2) corre-
spondhlg to tIle control u(.).
Consider the connection between control systellls and differential inclu-
sions. We can associate the differential inclusion
(4.3)
X E U f(x,u)
uEU
with control system (4.2). Obviously any trajectory of (4.2) is a solution
to (4.3). By Theorem 2.3 the converse statenlent is also true. Thus system
(4.2) is equivalent to differential inclusion (4.3).
Continuous maps to the set of solutions. Now we proceed to prove
one of the most important theorems that we shall often use. This result
cOlllbines an existence theorem with the Gronwall inequality and allows us
to 'project' continuously the space of absolutely continuous functions onto
the set of solutions. We postpone III ore detailed comments 011 its use to the
end of the proof.
Theorem 4.5. Let 8 > 0, p(.) E L 1 ([0, T], R), and M c AC([O, T], R n ) be
given. Let ro : R n  R n be a continuous map such that Iro(x) - xl < 8 for
all x E Rn. Assume that each x(.) E IvI satisfies
d ( x ( t ) , F ( x ( t ) )) < p ( t ), t E [0, T] .
Then there exists a map r : IvI  S(F) satisfying the following conditions:
1. r is continuous in the norm of the space AC([O, T], Rn);
2. r ( x ( . ) ) ( 0) == ro ( x ( 0 )) for all x (-) EM;

92
4. Existence Tl1eorenls
3. if x(.) E 1\1 n S(F) and ro (x(O)) == x(O), then r(x(.)) == x(-);
4. the inequalities
(4.4)
I x ( t) - r ( x (- ) ) ( t ) I < (( t ), t E [0, T],
d
Ix(t) - d{ (x(o))(t)1 < l(t) + p(t), t E [0, T],
(4.5)
where
(t) = 8 e /ltl + it el(ltl-lsl)p(s)ds ,
are satisfied.
Remark. In this theoren1 T can be l1egative, that is, solutions defined in
the seg111ent [T,O] are also alloV\'ed.
To prove the theore111 we need the following auxiliary result.
Lemma 4.1. Assume that the sequences {Xi(-)} r l and {Yi(-)} r 1 converge
in L 1 ([0, T], R n ) to Xo (.) and Yo (.) respectively. Assume besides that there
exists a function b(.) E L1 ([0, T], R) such that IWi (t) I < b( t), t E [0, T], i ==
1,2,.. ., where Wi(t) == lU(Xi(t), Yi(t)) -Yi(t) and w(x, y) == 7r(Y, F(x)). Then
the sequence {Wi(-)} r l converges to wo(-) == w(xo(-),yo(')) - yo(.) in the
norm of L1([0, T], Rn).
Proof. Suppose that the staten1ent of the le111111a is not true. Then there
exist subsequences {Xik' (-)} k 1 and {Yik (-)} r l' and a nU111ber E > 0 such that
IWik(-) - wO(')ILl > E. Vlithout loss of generality we can aSSU111e that the
sequences {Xik(')} k 1 and {Y i k(.)} r 1 converge aln10st everywhere (we can
choose a convergent subsequel1ce if needed). The n1ap (x, y)  w(x, y) is
continuous due to Theore111 2.2. Therefore Wik (.) converges to Wo (.) ahnost:
everyvvhere. Since Wik (.) < b( t), t E [0, T], fron1 the Lebesgue theoren1 we
obtain 11Uik(') - wO(')ILl  0 as k  00, a contradiction. D
Proof of Theorem 4.5. Let x(-) E AC([O, T], Rn). Set w(x(.))(t)
1f(x(t), F(x(t))). Since
Iw(x(-))(t)1 < Ix(t)1 + Iw(x(.))(t) - x(t)1
== Ix(t)1 + d(x(t), F(x(t)))
and F is Lipschitzian, fron1 Proposition 2.6 we have
Iw(x(.))(t)1 < 2Ix(t)1 +d(O, F(x(t))) < 2Ix(t)1 +d(O, F(x(O))) +llx(t) -x(O)I.
This inequality i111plies w(x(.)) E L1([0, T], R n ), and ,ve can define the op-
erators
J(x(o))(t) = ro(x(O)) + it w(x(-))ds, t E [0, T],

4.2. Lipscl1itzian differential inclusions
93
I(x(.))(t) = x(O) + I t w(x(-))ds, t E [O,T].
Let xo(.) E A1. Define the sequence of functiollS Xk(.) E AC([O, T], Rn) by
X1C) == J(xoC)), Xk(.) == I k - 1 (X1C)), k == 2,3,... .
By asslunption we have
(4.6)
IX1(t) - xo(t)1 == d(xo(t), F(xo(t))) < p(t).
Integrating tllis inequality, we obtain
IXl(t) - xo(t)1 < Ixo(O) - ro(xo(O))1 + I t p(s)ds
(4.7) < 8 + I t p(s)ds, t E [0, T].
Fronl the Lipschitz condition and Proposition 2.6 we obtain
I X k+1(t) - xk(t)1 == IW(Xk(.))(t) - w(xk-1(.))(t)1
== d(Xk(t), F(Xk(t))) < d(Xk(t), F(Xk-1(t))) + llxk(t) - Xk-1(t)1
(4.8) == llxk(t) - Xk-1(t)l.
Integrating (4.8), we have
I t l:h+l(S) - xk(s)lds < I t llxk(S) - Xk-l(S)lds
(4.9) < I t ll s IXk(q) - Xk-l(q)ldqds .
For any k > 0 the follo,ving inequality holds:
(4.10) I X k+l(t) - xk(t)1 < 8 (l1:)k + I t (l(ltl ! Isl))k p(s)ds .
We call prove this fact by induction. Indeed, for k == 0 the i1lequality is valid
due to (4.7). Suppose it is true for k - 1, k > 1. Frolll (4.9) we have
{t ( (llsl)k-1 {S (l(lsl-lql))k-1 )
I X k+l(t) - xk(t)1 < Jo l 8 (k _ I)! + Jo (k _ I)! p(q)dq ds
_ r t d ( 8 (lISI)k + r (l(lsl-lql))k p(q)dq )
Jo k! Jo k!
=  (ll t I) k i t (l (I t I - I s I) ) k ( ) d
u k' + k' p s s,
. 0 .
and, thus, (4.10) is proved. Froln (4.8) and (4.10) for k > I,ve have
. . [ (lltl)k-1 {t (l(ltl _ Isl))k-1 ]
( 4.11 ) I x k+ 1 ( t) - X k ( t ) I < l 8 (k _ I)! + J 0 (k _ I)! p( s ) ds .

94
4. Existence Theorellls
From this inequality we see t11at {Xk(.)} k 0 is a Cauchy sequence in the space
£1 ([0, T], R n ). Consequently it converges to a function Vo (.) E £1 ([0, T], R n ).
Moreover, froln (4.11) we see that x k (t) converges to Vo (t) almost every-
where. Define
r(xoO) (t) = ro(xo (0)) + it vo( s )ds, t E [0, T].
Since
Z zk
1 + I! + . . . + k! < e Z , z > 0,
from (4.10), (4.11), and (4.6) we obtain
Ixo(t) - xk(t)1 < (t)
and
Ixo(t) - xk(t)1 < l(t) + p(t).
Taking the limit in the inequalities
Ixo(t) - r(xo(.))(t)1 < Ixo(t) - xk(t)1 + IXk(t) - r(xo(.))(t)1
< (t) + IXk(t) - r(xoC))(t)1
and
Xo(t) -  r(xo(.))(t) < Ixo(t) - xk(t)1 + Xk(t) -  r(xo(.))(t)
< l(t) + p(t) + Xk(t) - :{ (xoO)(t) ,
we obtain (4.4) and (4.5). If xo(.) E M n S(F) and ro(x(O)) == xo(O),
then by construction we have Xo (.) == Xl (.) == ... == Xk (.) == .... Hence
r(xoC)) == xo(.). The map ro is continuous; therefore Lemma 4.1 ilnplies
the continuity of the map Jk 0 J : M ---+ AC([O, T], R n ). From (4.11) we see
that the sequence JkoJ(xo(.)) uniforlnly in xo(.) E M converges to v(xo(.)).
Hence the map r : M ---+ AC([O, T], R n ) is continuous. Invoking Lemma 2.2
and taking the limit in (4.8), we obtain r(xo(.)) E S(F). The theoreln is
proved. 0
This theorem is one of the Inain tools we use to study differential inclu-
sions. The main idea of its application is the following. Suppose we need
to construct a family of solutions with given properties. Then at first we
construct a falnily M of functions which are sufficiently close to the solution
set and have the required properties. After that we map the falnily Minto
the solution set with the help of Theorem 4.5. Froln inequalities (4.4) and
(4.5) we conclude that the sets !vI and r(M) are sufficiently close to each
other. Quite often such estimates together with continuity of r allow us

4.2. Lipscl1itzian differential inclusions
95
to conclude that r(NI) is the set of solutions ,vith the desired properties.
Consider a few exalnples.
Corollary 4.1 (Existence theorem). For any Xo E R n there exist solutions
to differential inclusion (4.1) with x (0) == xo.
Proof. Consider the set NI consisting of one function z(t) - O. Let ro(x) ==
Xo + x for each x ERn. By Theorem 4.5 there exists a solution x(-) E
S(F, xo) satisfying
Ix(t)1 < Ixolelltl + 1 t el(ltl-lsDd(O, F(O))ds .
This ends the proof. 0
Corollary 4.2 (Arcwise connectedness of the solution set). The set of so-
lutions S[o,T](F,xo) is arcwise connected; that is, for any xo(.) E S(F,xo)
and X1(-) E S(F,xo) there exists a continuous map cjJ : [0, 1]  S(F,xo)
satisfying cjJ( 0) == Xo (-) and cjJ( 1) == Xl (.) .
Proof. Consider the set NI == {x(-) E AC([O, T], R n ) I x(.) == (1 - A)XO(.) +
AX1(.), A E [0, I]}. By the Lipschitz condition and Proposition 2.6 we have
d ( (1 - A) x 0 ( t) + AX 1 ( t ), F ( (1 - A) x 0 ( t) + AX 1 ( t ) ) )
< d(xo(t), F(xo(t))) + Alxo(t) - x1(t)1 + Allxo(t) - x1(t)1
< Ixo(t) - x1(t)1 + llxo(t) - x1(t)l.
Let p( t) be equal to the right side of the last inequality. Obviously p(.) is
integrable. Set ro (x) x. By Theoreln 4.5 there exists a continuous map
r : NI  S(F,xo) satisfying r(xo(.)) == xo(-) and r(x1(-)) == X1(.). Setting
cjJ(A) == r((l- A)XO(-) + AX1(.)), we obtain the result. 0
Define the reachability set of differential inclusion (4.1) by
R[O,T](F,xo) == {x(T) I x(.) E S[o,T](F,xo)}.
Corollary 4.3 (Boundary property). Let x* E bdR[o,T] (F, xo), and x* ==
x(T), where x(-) E S[o,T](F,xo). Then x(t) E bdR[o,t](F,xo) for all t E
[ 0, T] .
Proof. Suppose that there exist t E [0, T[ and E > 0 such that x(t) + EBn C
R[o,t] (F, xo). Fix a point y (j. R[O,T] (F, xo) satisfying Iy - x* I < E exp( -lit -
TI). Put NI == {x(-)} and ro(x) == x - x* + y. By Theorem 4.5 there exists
a solution y(.) E S(F) satisfying y(T) == y and
Iy(t) - x(t)1 < Iy - x*le(llt-TI) < E.
Therefore y(t) E R[o,t](F,xo). Hence y E R[O,T](F,xo), a contradiction. 0

96
4. Existence Tl1eorenls
4.3. Upper semi-continuous differential
incl usions
In this section ,,,e study differential inclusions
( 4.12)
x(t) E F(x),
,,,here F : R n --7 R'n is an upper sell1i-continuous ll1ap with closed convex
values. '''Ie aSSlune that there exists a constant b > 0 such that F(x) c bBn
for all x E R n . In order to establish properties of (4.12) we approxill1ate its
right-hand side by Lipschitzian set-valued 111aps Fk, k == 0,1, . . . , and apply
Theoren1 4.5 to the differential inclusions
(4.13 )
X E Fk(X), k == 0,1,... .
The ll1ain result of this section (Theorell1 4.6) sho,,,s that the solution sets
of (4.13) approxin1ate that of (4.12).
Approximation theorem. First we establish two sin1ple auxiliary results.
Lemma 4.2. Let A c R n be a convex compact, and let v(.) E L 1 ([0, T], R n )
be a function satisfying v(t) E A almost everywhere in [0, T]. Then
 iT v(s)ds E A.
Proof. By the definition of the Lebesgue integral we have
T i
1 1 ( )d 1 . L 111eas(Llk) i
T v s s == . 1111 V k ,
o 1,OO T
k
,,,here {Ll1}  i 1 is a partition of the segl11ent [0, T], and the silnple function
Vi (t) == v1, t E Ll1, satisfies Iv( t) - Vi (t) I < Iii, t E [0, T]. Obviously we can
choose v1 E A. The set A is convex; therefore we have
'"'"' n1eas( Ll) i A
 T Vk E .
k
Since A is closed, we obtain t11e desired result. D
Lemma 4.3. Let A c R n be a convex compact set. Assume that Xk(') E
AC([O, T], R n ) is a sequence such that Xk(t) --7 x(t) for all t E [0, T] and
Xk(t) E A for almost all t E [0, T]. Then x(.) E AC([O, T], R n ) and x(t) E A
almost everywhere.
Proof. Since A is bounded, "\ve have IXk(t)1 < l, k == 1,2,... , t E [0, T].
Let t1, t2 E [0, T] and t1 < t2. Then we 11ave
IXk(t1) - Xk(t2)1 < l t2IXk(t)!dt < *2 - tI.
tl

4.3. Upper sell1i-col1tinuous differential inclusions
97
Taking the lilni t as k  00, we see that x ( .) satisfies the Lipschitz condi tiOl1
with the constant l. Hence x(.) E AC([O, T], R n ). By Len1111a 4.2 we have
Xk(t + h) - Xk(t) _  {t+h. ( )d A [ [
h - h it Xk ssE , 'If t E 0, T , h > O.
Taking the lilnit as k  00, we obtain
x(t+h)-x(t) A
hE.
Since A is cOIn pact, we have
. () 1 . x(t+h)-x(t) A
x t == In1 E
h10 h
for ahnost all t E [0, T]. D
Vrheorem 4.6. Let F : R n  R n and Fk : R n  R n , k = 0,1,..., be
1 set-valued maps with closed convex values. Assume that
1. the maps F and Fk, k == 0,1, . . ., are upper semi-continuous;
2. F(x) c ... C Fk(X) C Fk-1(X)... C Fo(x) C bBn for all x ERn;
3. for any E > 0 and any x E R n there exists a positive integer k( E, x)
such that Fk(X) C F(x) + EBn whenever k > k(E, x).
If a sequence of solutions Xk (.) E S[O,T] (Fk), k == 0, 1, . . . , uniformly con-
verges to a function x(.), then x(.) E S[O,T] (F).
Proof. Since Xk(t) E bBn, k == 1,2,. . . , by Len1n1a 4.3 the function x(.) is
absolutely continuous. Let to E [0, T] be such that there exists the derivative
x(to). Fix E > o. There exists a positive integer ko satisfying Fko(X(tO)) C
F(x(to)) + EBn. The n1ap Fko is upper sen1i-continuous; therefore there
exists 1] > 0 such that
Fko(x) C Fko(X(tO)) + EBn
whenever x E x(to) + 21]Bn. Since Xk(.) uniforlnly converge to xC), there
exist a positive integer k 1 > ko and a positive nUlnber  < 1] such that
I x k ( t) - x ( t ) I < 1] and I x ( t) - x ( to) I < 1]
for all k > k 1 and t E]to - , to + [n[O, T]. Thus, if k > k 1 and t E
]to - , to + [n[O, T], then we have
Xk(t) E Fk(Xk(t)) C Fko(Xk(t)) C Fko(X(tO)) + EBn C F(x(to)) + 2EBn.
Fro111 Len1n1a 4.3 we obtain
X(t) E F(x(to)) + 2EBn

98
4. Existence Theorel11s
for almost all t E]to -'"'(, to+'"'([n[O, T]. In particular x(to) E F(x(to)) +2EBn.
Since E > 0 is arbitrary and the set F(x(to)) is closed, we obtain
x(to) E F(x(to)).
The theoreln is proved.
o
Now we are in a position to establisll the Inain properties of the solution
set to a differential inclusion with upper selni-continuous right-hand side.
Corollary 4.4 (Existence theorem). For any Xo E R n there exists a solu-
tion x(.) of differential inclusion (4.12) with x(O) == Xo.
Proof. By Theorem 2.5 there exists a sequence of locally Lipschitzian set-
valued maps Fk : R n ---7 R n , k == 0,1,. .. , approximating F and satisfying
Fk(X) C bBn. From Corollary 4.1 it follows that there exist solutions Xk(-) E
S[O,T] (Fk, xo). Since Xk(t) E bBn and Xk(t) E Xo + TbB n for all t E [0, T]
and k == 0, 1, . . . , from the Arzela-Ascoli theorem we see that the sequence
{Xk(-)} k 1 contains a uniformly convergent subsequence. By Theorem 4.6
the limit function x(.) is a solution to inclusio11 (4.12). 0
Corollary 4.5 (Compactness of the solution set). Let CeRn be a com-
pact set. Consider a sequence of solutions Xk(') E S[o,T](F,C). There ex-
ists a subsequence Xk p (.) which uniformly converges to a solution x(.) E
S[O,T] (F, C) .
Proof. It is a trivial consequence of the Arzela-Ascoli theorem and Theoren1
4.6. 0
Note tllat solution sets of differential inclusions with nonconvex right-
hand side are not compact in general. Indeed, consider the set-valued Inap
F : R ---7 R defined by F(x) == {:f:1}. Obviously the 'saw-toothed' functions
{ t - 2i/k, 2i/k < t < (2i + l)/k,
Xk(t) = (2i + 2)jk - t, (2i + l)jk < t < (2i + 2)jk,
i == 1, k/2, k == 2,4, . . .
are solutions to the differential inclusion x E {:f:1}, and they uniformly
converge to xo(t) - 0, t E [0,1]. Nevertheless the function xo(-) is not a
solution of the differential inclusion.
Corollary 4.6 (Connectedness of the solution set). The solution set S ==
S[O,T] (F, xo) c C ([ 0, T] , R n ) is connected; that is, it cannot be represented as
a union of two closed sets without common points.

4.3. Upper semi-continuous differential inclusions
99
Proof. Suppose that 5  51 U 52, where 51 c C([O, T], R n ) and 52 C
C([O, T], R n ) are closed sets satisfying 51 n 52  0. From Corollary 4.5 we
see that both 51 and 52 are cOlnpact. Consequently we have
inf{lx1(.) - x2(.)lc I X1(.) E 51, X2(.) E 52}  8 > O.
Consider the functio11 0 : C([O,T],R n ) -7 Rdefi11ed byO(x(.))  d(x(.),5 1 )-
8/2. We see that O(X1C))  -8/2 for all X1(.) E 51 and O(X2(.)) > 8/2 for all
X2(.) E 52. Fix X1(.) E 51 a11d X2C) E 52. By Theorem 2.5 there exists a se-
quence of locally Lipschitzian set-valued Inaps Fk : R n -7 R n , k  0, 1, . . . ,
approximating F and satisfyi11g Fk(X) C bBn. From Corollary 4.2 it fol-
lows that there exist c011tinuous maps CPk : [0,1] -7 5[0,T] (Fk, xo) such that
CPk(O)  X1(.) and cpk(l)  X2(.). Since the functions 00 cpk : [0,1] -7 rare
continuous, 00 cpk(O) < 0, and 00 cpk(l) > 0, there exist Ak E [0,1] such that
o 0 cpk(Ak)  0, k  0,1, . . .. By the Arzela-Ascoli theorem the sequence of
sol u tions cp k ( Ak) ( .) E 5[0, T] ( F k, xo) contains a uniformly convergent su bse-
quence. By Theorem 4.6 the limit function xC) is a solution to differential
inclusion (4.12). On the other hand, since the function 0 is continuous, we
have O(x(.))  0; that is, x(.)  51 U 52  5, a contradiction. 0
Remark. If F( x)  {f (x)}, where f : R n -7 R n is a continuous function,
then Corollary 4.6 is equivalent to the Kneser theorem.
Corollary 4.7 (Boundary property). If x* E bdR[o,T](F,xo), then there
exists x(.) E 5[0,T](F, xo) such that x(T)  x* and x(t) E bdR[o,t](F, xo)
for all t E [0, T].
Proof. By Theorem 2.5 there exists a sequence of locally Lipschitzian set-
valued Inaps Fk : R n -7 R n , k  0, 1, . . . , approximati11g F and satis-
fying Fk(X) C bBn. Consider a sequence x k E bdR[o,T](Fk,xo) satisfying
d(x*, bdR[o,T](Fk,xo))  Ix* - xkl. By Corollary 4.5 the reachability sets
are closed and x k E R[O,T] (Fk, xo). Consider a sequence of solutio11s Xk (.) E
5[0,T] (Fk, xo) such that xk(T)  x k . By the Arzela-Ascoli theorem the se-
quence {Xk(.)} k 1 contains a uniformly convergent subsequence. Without
loss of generality Xk(.) uniformly converge to x(.). By Theorem 4.6 we have
xC) E 5[0,T](F,xo). Show that x(T)  x*. Suppose that Ix(T) - x*1 > o.
Then there exists 8 > 0 such that x* + 8Bn C R[O,T](Fk,XO), k  0,1,....
Let Y E x* + 8 Bn. There exist Yk ( .) E 5[0,T] (Fk, xo), k  0, 1, . . ., satis-
fying Yk(T)  y. By the Arzela-Ascoli theorem the sequence {Yk(.)} k 1
contains a uniformly convergent subsequence. Without loss of general-
ity Yk(.) uniformly converge to a function y(.). By Theorem 4.6 we have
y(.) E 5[0,T](F,xo). Hence we obtain x* + 8Bn C R[O,T](F,xo), a contra-
diction. Prove that x(t) E bdR[o,t] (F, xo) for all t E [0, T]. Indeed, if there
exist t E [O,T[ and E > 0 such that x(t) + EBn C R[o,t](F,xo), then for k

100
4. Existence Tl1eorelTIS
sufficiently large we have
E
Xk(t) + 2 Bn c x(t) + EBn C R[o,t] (F, xo) c R[o,t] (Fk, xo).
This contradicts Corollary 4.3.
D
Remark. If F(x) == {f(x)} where f : Rn  Rn is a continuous function,
then we obtain the Hukuhara theoreln. In the theory of ordinary differential
equations Hukuhara's theoreln is derived froln the Kneser theoren1. Here
we have obtained a direct proof.
If the right-hand side of a differential inclusion is upper selni-continuous,
then not all solutions with x(T) E R[O,T](F, xo) satisfy x(t) E R[o,t](F, xo),
as it was in the case of Lipschitzian differential inclusions (see Corollary 4.3).
Indeed, consider the following differential h1clusion:
(xl, x 2 ) E F(x 1 , x 2 ),
where
{ B2, x 2 < 0,
F(x 1 , x 2 ) == 6B 2 , x 2 == 0,
{O}, x 2 > O.
Obviously the point (3,0) is a boundary point of the set R[O,5](F, (0, -4)),
since this set is contained in the half-plane x 2 < O. The trajectory
(1 2)(t) == { (0, (8t)/9), t E [0,9/2],
x,x (6(t-9/2),0), tE]9/2,5]
satisfies (x1(5), x2(5)) == (3,0) and (x 1 (t), x 2 (t)) 9!' bdR[o,t] (F, (0, -4)) when
t E [0,9/2[.
Time-dependent differential inclusions. We shall consider also thne-
depende11t differential inclusions
,
x == F ( t, x ), t E [0, T] .
Let us establish a convergence property for theln.
Lemma 4.4. Assume that a set-valued map F : [0, T] x R n x R m  R n
has closed convex values. Let the set-valued map (x, y)  F( t, x, y) be upper
semi-continuous for almost all t E [0, T], and let F(t, x, y) C b(t)B n for all
(t, x, y) E [0, T]' where b(.) E £1 ([0, T], R). Assume that functions Xk (.) E
AC([O, T], R n ), k == 0,1, . . ., satisfy
(4.14) Xk( t) E coF( t, Xk( t), 'l}k (t )Bm) + 'l}k( t)Bn,

4.3. Upper selni-continuous differential inclusions
101
where 1]k(t) > 0, liInk-too1]k(t) == 0 almost everywhere, and l1]k(t)1 < 1](t),
k == 1,2,..., 1](-) E £1([0, T], R). Then the functions Xk(.) are equicontinu-
ous on [0, T],. and if a subsequence Xk p (.) uniformly converges to a function
x(-), then x(-) is a solution to the differential inclusion
(4.15 )
X(t) E F(t, x(t), 0).
t
Proof. Froln (4.14) we have
( 4.16)
IXk(t)1 < b(t) + 1]k(t).
Let E > 0, and let 0 < a1 < /31 < a2 < ... < al < /31 < T. There exist
8 > 0 and ko such that the inequalities E I 1 (/3i - ai) < 8 and k > ko imply
t lf3i b(t)dt <  and I T 1]k(t 
i== 1 
or, by virt ue of (4.16),
I I r f3i
L I X k(,6d - Xk(ai)1 = L in xk(t)dt
i==l i==l Qi
( 4.17)
<E
for all k == 0,1, . . .. Hence the functions Xk (-) are equicontinuous.
Suppose that a subsequence Xk p (.) uniformly converges to a function
x(.). Froln (4.17) we see that x(.) is absolutely continuous.
Prove that x (-) is a solution to differential inclusion (4.15). Since the
functions 1]k p (.) converge to zero in the norln of L 1 ([0, T] , R), there exists a
subsequence converging ahnost everywhere. For the sake of brevity denote
this subsequence by {1]i ( . )} and denote the corresponding subsequence of
{Xk p (-)} by {Xi(.)}. Since the set-valued Inap (x, y)  F(t, x, y) is upper
sen1i-continuous, and Xi(t)  x(t), 1]i(t)  0 ahnost everywhere, we have
F(t, Xi(t), 1]i(t)B m ) C F(t, x(t), 0) + vi(t)B n ,
where Vi(t)  O. COInbining this inclusion with (4.14), we obtain
Xi(t) E F(t, x(t), 0) + (Vi(t) + 1]i(t))Bn.
In terIns of support function this inclusion caI1 be written as
(Xi(t),p) < S(p, F(t, x(t), 0) + (Vi(t) + 1]i(t))B n )
(see Theoreln 1.7). Consequently we 11ave
( 4.18) S ( t) == liln sup (x i ( t ) , p) < S (p, F ( t, x ( t ), 0) ) .
 -t 00
Let a, /3 E [0, T] be such that a < /3. Then we have
(Xi(,6) - xi(a),p) = r f3 (xi(t),p)dt < r f3 s.up(Xj(t),p)dt
JQ JQ J?

102
4. Existence Theorellls
(the latter integral obviously exists). Since SUPj2i(Xj(t),p) does not increase
with increasing i and tends to the left side of (4.18), as i  00, applying the
Lebesgue theoren1, we obtain
(x({3) - x(a),p) < il!l J: (Xj(t),p)dt < 1(3 S(t)dt.
Since the interval ]a,,B[ is arbitrary, we have
( i; ( t ) , p) < S ( t) < S (p, F ( t, x ( t ), 0) )
for almost all t E [0, T]. The same is true for a countable, everywhere dense
set of vectors p. From Theorem 1.7 and Proposition 1.5 (see also the remark
after Proposition 1.5) we obtain (4.15). D
Lemn1a 4.4 can be applied to prove al1 existence theorem for a time-
dependent differential incl usiol1 /
/
( 4.19) i; E F ( t, x ) , ) t E [0, T].
Theorem 4.7. Let F : [0, T] x Rn  R n be a set-valued map with closed
convex values. Assume that
1. the set-valued map x  F(t, x) is upper semi-continuous for almost
all t E [0, T];
2. for any x E R n there exists a measurable function t  f (t, x) satisfy-
ing f(t, x) E F(t, x);
3. there exists a function b(.) E £1([0, T], R n ) such that If(t, x)1 < b(t),
t E [0, T] .
Then for any Xo E R n there exists a solution x(.) to differential inclusion
(4.19) with x(O) == xo.
Proof. Put
hk == T jk, tk,i == ihk, i == 0, k ,
where k == 1,2,.... Let us define a functiol1 Xk(.). Put Xk(tk,O) == Xo. If
Xk(tk,i) == Xk,i, i > 0, is already defined, then set
Xk(t) = Xk,i + r t f(s, Xk,i)ds, t E]tk,i, tk,i+1]'
Jtk1i
The functions Xk (.) are absolutely continuous and satisfy
Xk(t) E F(t, Xk,i) n b(t)Bn.
Obviously
I t k 1 i + 1
IXk(t) - Xk,il < mx b(t)dt,
 t k .
11.

4.4. Discontinuous differential equations
103
and the integral in the right side tends to zero as k  00. From LemIna 4.4
and the Arzela-Ascoli theoren1 we see'that the seq'uence {Xk(.)} contains a
uniforlnly convergent subsequence, and the lilnit function is a solution to
differential inclusion (4.19). 0
4.4. Discontinuous differential equations
Upper selni-continuous differential inclusions are mainly applied in the the-
ory of differential equations with discontinuous right-hand side. Consider a
differential equation
( 4.20)
x == f ( x ), t E [0, T],
where f : R n  R'n is a bounded function. If f is not continuous, then the
Cauchy probleln
( 4.21)
x == f(x), x(O) == Xo
Inay have no solution. For exaInple, consider the following Cauchy probleln:
. _ { I, x < 0
x - -1, x > 0 '
x(O) == O.
It is clear that the problen1 has no solutions in the usual sense. Indeed, if
x(t) < 0 the solution has the forln x(t) == t+c_, and the form x(t) == -t+c+
whenever x(t) > o. As t increases, each solution reaches the point x == 0
and cannot leave it. The function x(t) - 0 does not satisfy the equation,
since for it x == 0 =/:. F(O) == -1.
However, we face discontinuous differential equations in many applica-
tions. A large number of problelns froln Inechanics and electrical engineering
leads to differential equations with discontinuous right-hand sides because
Inany physical laws are expressed by discontinuous functions, for exaInple,
a dry friction force or jump-like transition characteristic of SOlne electronic
devices. Many differential equations appearing in control theory model ob-
jects with variable structure or with sliding Inotions and are discontinuous.
Another motivation to consider discontinuous right-hand sides is of a n1athe-
Inatical nature. Nan1ely, if the right-hand side is continuous but cOInplicated,
it can be useful to approxilnate it by a sin1ple discontinuous function, for
exaInple, by a piecewise constant or piecewise linear function. This Inethod
is often used in technical problems. Consider SOlne examples of differential
equations with discontinuous right-hand sides.

104
4. Existence Tl1eorellls
1. Systems with dry friction. A general for111 of a 011e-din1ensio11al
111echanical systelTI with dry friction is
x == v,
v == - f(v) - kx + u(t),
vvhere
f(v) == { f, v > 0,
- f, v < 0,
n10dels the dry friction force, kx is an elastic force, and u(t) is an external
force. If v == 0, the dry friction force takes a value fo E [- f, f] that places
the other forces in equilibril1l11. The value of the frictio11 force depends 011ly
on the sign of the velocity (it alvvays opposes the velocity) and does not
depend on its value. This siinple 1110del of dry friction is known as Coulomb
law.
2. Electronic oscillator. The work of an electronic oscillator in ITIany sit-
uations can be 1110deled by the following discontinuous differential equatiol1:
.. . b { O, x < 0,
x + ax + x == b . 0
, x > .
This is a typical exa111ple of a systen1 with variable structure having a daInp-
inglike Inechanisn1 which acts to increase the energy when the ainplitude of
the n10tion is sn1all and to decrease the energy when the an1plitude is large.
As a result, the systein reaches a so-called lilnit cycle which is independent
of the initial cOl1ditions.
3. Autopilot. Consider 011e axis of an aircraft attitude cOI1trol systelTI.
Let <p be the angle between the desired heading of the aircraft and its actual
heading. If <p is slTIall enough, the angular Inotion of the aircraft can be
described by the differential equation
<p == -k0 + T,
where T is a torqlle created by aircraft's control ele111ents. The zero equi-
libriu111 position of this systen1 is unstable without control. Therefore it is
necessary to construct a device, which COlnpares the desired heading with
the actual heading and produces an error signal which drives the aircraft's
control eleinents in order to reduce the heading error to zero. In other words,
it is necessary to find a function T == T( <p) that n1akes the systelTI stable.
The sin1plest stabilization law is given by
T(<p) == { T,
-T
,
<p < 0,
<p > O.

4.4. Discontinuous differential equations
105
The autopilot generates a torque of constant absolute value. The sign of
the torque depends on the sign of the deviation angle <po The basic control
element required has ideal relay characteristics whic11 can easily be imple-
n1e11ted.
Since the Cauchy problem for a differential equation with a discontinuous
right-hand side, in general, may have no solution in the usual sense, we have
to generalize the notion of solution in order to ensure its existence.
Filippov solutions. The generalized solution lTIUst necessarily satisfy nat-
ural requirements. The solution to the Cauchy problelTI must exist for any
initial condition. For differential equations with a continuous right-hand side
the generalized solution must coincide with the usual one, and the properties
of the solution set, like connectedness and compactness, must be the same.
The simplest way to introduce such a generalized solution is the following.
Define the set-valued map
F(x) = n cl cof(x + EBn).
E>O
Since f is bounded and grF is closed, the map F is upper semi-continuous
due to Proposition 2.2. Note t11at f(x) E F(x) for all x E R n and F(x) ==
{f(x)} whenever f is continuous at x. Consider the differe11tial inclusion
X E F ( x ), t E [0, T].
Let Xo E R n . By Corollary 4.4 the Cauchy problem
( 4.22)
X E F(x), x(O) == Xo
always has a solution. Moreover, the properties of a solution set such as
compactness, connectedness, and boundary property (see Corollaries 4.5 -
4.7) are similar to that of a solution set to an ordinary differential equa-
tion with continuous right-hand side. Motivated by these observations we
introduce the following definition. An absolutely continuous function is said
to be a solution (or Filippov solution) of the Cauchy problem (4.21) for a
differential equation with discontinuous right-hand side if it is a solution to
the Cauchy problem (4.22).
To understand the geometrical sense of the definition, consider a function
f : R n  R n discontinuous on a smooth surface S. The surface separates
the space into two domains D+ and D-. The set F(x), XES, is the seglTIent
with the ends
f-(x) == lim f(x') and f+(x) == lim f(x').
X/X,X/ED- X/X,X/ED+

106
4. Existence Tl1eorenls
If F(x) lies on one side of the tangent plane T(x, S) to the surface S at
xES, tl1en the solutions pass frolll one side of the surface to the other.
If tl1e segn1ent F(x) intersects T(x, S), tl1en the h1tersection point fs(x) ==
T(x, S) n F(x) detern1i11es the velocity of the n10tio11 X == fs(x) along the
surface. If fs(x) =1= f-(x) and fs(x) =1= f+(x), such a solution is called a
sliding solution. If the vectors f (x) are directed to the surface on both sides,
the11 all solutions starting in a 11eighborhood of S approach the surface frolll
both sides as t h1creases. Therefore a solutio11 starth1g at xES belongs to
S for t > o. If tl1e vectors f (x) are directed away frolll the surface on botl1
sides, the11 any solution starting at xES lllay either leave the surface or slide
alo11g S. 111 the latter case the solution lllay leave S at any 11101llent. Such
a lllotion is unstable and usually cannot be observed in physical systen1s.
4.5. Existence of optimal solutions
Consider a differential" incl USi011
( 4.23)
x ( t) E F ( x ( t ) ) , t E [0, T],
where F : R n  R n is an upper sen1i-continuous set-valued n1ap with closed
convex values contah1ed in a ball of radius b > O. Let Co c R n and C 1 C R n
be closed sets.
We shall deal with two probleills. The first one, the Mayer problem, is to
111inin1ize, over all solutio11s to differential inclusio11 (4.23) defined 011 a fixed
th11e h1terval and satisfying bou11dary conditions, a functional depending on
the final position x(T). The second problelll, known as the time-optimal
problem, is to find a solution to (4.23) such that the thlle of tra11sfer frolll an
initial set to a terillinal set is Illh1in1al. Later we show that the tin1e-optin1al
problen1 can be reduced to the lVlayer proble111. However we prefer to study
this problelll separately because of its importance. Below we prove son1e
existe11ce results for these pro bleills.
Mayer problem. Let <p : R n  R be a c011tinuous function. The problelll
is to find a solution x(-) to (4.23) satisfying the bou11dary conditio11s
X(O) E Co, x(T) E C 1
a11d such that
<p(x(T)) < <p(x(T))
for any trajectory x(.) of (4.23) satisfying x(O) E Co and x(T) E C 1 .
Theorem 4.8. Let the set Co be compact. Assume that there exists a tra-
jectory x(.) of differential inclusion (4.23) satisfying the boundary conditions
x(O) E Co and x(T) E C 1 . Then there exists an optimal trajectory.

4.5. Existence of optimal solutions
107
Proof. Indeed, the set R[O,T](F, Co) n C 1 is nonen1pty. By Corollary 4.5 it
is compact. Applying the Weierstrass theoreln, we obtain t11e result. D
Time-optimal problem. Let T* be a positive nUlnber. The problem is to
find a time 0 < T < T* and a solution x(.) to (4.23) satisfying the boundary
conditions
x(O) E Co, x(T) E C 1
and such that
X(t) tJ C 1
for any solution xC) E S[o,T*](F, Co) and any t E [0, T[.
Theorem 4.9. Let the set Co be compact. Assume that there exist T > 0
and a trajectory x(.) of differential inclusion (4.23) satisfying the boundary
conditions x(O) E Co and x(T) E C 1 . Then there exists a time-optimal
trajectory.
Proof. Consider the sets 8 == {(x, T) I x E R[O,T](F, Co), T E [0, T*]} and
8 1 == {(x, T) I x E C1, T E [0, T*]}. The intersection 8 n 8 1 is nonelnpty.
Show that the set 8 is compact. Indeed, let Xk(.) E S[o,T*](F, Co) and
Tk E [0, T*], k == 1,2,..., be sequences such that lilnk-too(xk(Tk), Tk) ==
(x, T). Without loss of generality the sequence Xk(.) uniforlnly converges to
a solution xC) E S[O,T*] (F, Co), thanks to Corollary 4.5. Since
Ix(T) - TI < Ix(T) - xk(T)1 + IXk(T) - xk(Tk)1 + IXk(Tk) - xl
< Ix(T) - xk(T)1 + biT - Tkl + IXk(Tk) - xl  0, k  00,
we get x(T) == x. Therefore (x, T) E 8. By the Weierstrass theoreln the
continuous function cjJ : 8 n 8 1  R defined by cjJ(x, T) == T attains its
minin1um. D
Note that the convexity of F is essential for the existence of optimal
solutions. Without convexity the set R[O,T](F, Co) n C1 is not closed in
general, as we can see from the following exalnple. Consider the differential
incl usion
( 4.24 )
(X 1 ,X 2 ) E F(x1,x2),
where the right-hand side F : R 2  R 2 is given by
F(x1,x2) == {(v 1 ,v 2 ) I vI == (v 2 )2 - (x2)2, v 2 E [-1, I]}.
Consider the set of solutions with the initial condition (xl, x2) (0) == (0,0). If
x2(t) 0, then xl (t) o. If x2(t)  0, then xl (t) < 1 on the intervals where
x2(t) =I- O. Since x 1 (t) < 1 for all t, we see that x1(1) < 1 for all solutions
and the point (1, 0) does not belong to the reachabili ty set R[o, 1] (F, (0, 0) ) .

108
4. Existence Theorems
Consider the solutions (xk, x) (-) E 5[0,1] (F, (0, 0)) k == 1, 2, . . ., sucl1
that x 2 ( t) == 1, t E [2i / k, (2i + 1) / k [, and x 2 ( t) == -1, t E [( 2i + 1) / k, (2i +
2)/k[, i==O,l,.... TheI1wehave
2 [ / ] '1 ( 1 1 1
Xk(t) E 0,1 k, xk t) > 1 - k 2 ' xk(l) > 1 - k 2 '
Thus the point (1,0) belongs to the closure of the reachability set. Therefore
R[O,l] (F, (0, 0)) is not closed.
Optimal control problem. Many physical and technical systen1s can be
n10deled by differential equations depending on SOI11e paral11eters. Usually
tl1ese paran1eters are controls that we 11ave at our disposal. Consider a
control systel11
( 4.25)
x == f(x,u), u E U C R 7n , t E [O,T],
where u is a control paral11eter varying in a giveI1 set U. We assume that
f : R n x R k  R n is a continuous fUl1ction. We can consider opti111ization
problen1s for the differential inclusion generated by (4.25). These problems
are known as optimal control problems.
Let T* > O. Consider the following optin1al control probleln:
(4.26) cp(T, x(T))  inf;
( 4.27) x ( t) == f (x ( t), u ( t) ), t E [0, T];
(4.28) u(t) E U;
(4.29) x(O) E Co, x(T) E C 1 , T E [0, T*].
The triple (T, u(.), x(.)), vvhere T E [0, T*], u(.) is a control, and x(.) is
a trajectory associated to the control u(.) and satisfying (4.29), is called
control process. A control process (T, u(.), x(.)) is said to be an optimal
control process for problen1 (4.26) - (4.29) if
cp(T, x(T)) < cp(T, x(T))
for all control processes. If cp does not depend on T and the parameter T
is fixed, we get the IVlayer problen1; and if cp(T, X (t)) == T, we have the
tin1e-optin1al pro blen1.
The equivalence between a control systen1 and tl1e corresponding differ-
ential inclusion (Theorel11 2.3) allows us to prove (following the proof of the
previous theorel11) the existeI1ce of solutions to the optil11al control problem.
Theorem 4.10. Let the sets U and Co be compact, and let the set C 1 be
closed. Assume that the function cP is continuous and the set f(x, U) is
convex for all x E R n . If there exists a control process for problem (4.26) -
(4.29), then there exists an optimal control process.

4.6. Depel1dence on initial conditions
109
4.6. Dependence on initial conditions
In this section we study the dependence of tIle solution set S[o,T](F, x) on
the initial condition x E R n . In other words we investigate tIle set-valued
lnap x  S[O,T] (F, x).
Continuity. First consider tIle case wIlen F is an upper senli-continuous
set-valued map.
Theorem 4.11. Let F : R n  R n be an upper semi-continuous map with
closed convex values satisfying F(x) C bBn, where b > o. Then the set-
valued map S[o,T](F,.) : R n  C([O, T], R n ) is upper semi-con.tinuous.
Proof. By Theoreln 4.6 the graph of tIle nlap S(F, .) is closed. Fronl Corol-
lary 4.5 we see that the set S(F, C) is cOlnpact ill C([O, T], R n ) whenever
CeRn is cOlnpact. Applyillg Propositioll 2.2, ""'e obtaill the result. D
When F is Lipschitziall, it is possible to prove a strollger result.
Theorem 4.12. Assume that the set-valued map F : R n  R n with closed
convex values satisfies the Lipschitz condition with a constant l > o. Then
the set-valued map S[O,T] (F,.) : R n  AC([O, T], R n ) satisfies the Lipschitz
condition with the constant 1 +IT exp(lT) . Moreover, if Xo (.) E S[O,T] (F, xo),
then there exists a continuous selection cP( x) E S[O,T] (F, x) satisfying cP( xo) ==
Xo ( . ) .
Proof. Let Xl, X2 E R n and XI(.) E S[o,T](F,XI). Applyillg Theorelll 4.5
witll !vI == {Xl C)} and ro (x) x + X2 - Xl, we see that tllere exists a solutioll
X2 ( .) E S[O,T] (F, X2) satisfying
IXI(t) - x2(t)1 < IXI - x2lle lT .
Hence
IXIC) - x2(.)IAC < 1 + lTe lT .
To prove the second part of the theorenl we apply Theorenl 4.5 witll
A1 == {xo (.) + x - Xo I x E R n } and ro (x) == x. We conclude tllat tllere exists
a continuous map r : A1  S(F). Set cP(x) == r(xo(.) + x - xo). Tllis ellds
the proof. D
Differentiability. Now let us study the derivative and COlltillgent deriv-
ative of the set-valued lnap S[o,T](F,.) : R n  AC([O,T],R n ) at a point
(x(O), xC)) E grS(F,.) C R n x AC([O, T], R n ). For brevity we use the fol-
lowing notations:
D+(u) == D+S(F, .)(X(O), xC))(u),
D - ( u) == D - S (F, .) ( x ( 0 ), xC) ) ( u ) ,

110
4. Existence Theorenls
where U E Rn, and
F ( t, x) == {v E R n I (x, v) E T + ( ( x ( t ), 5; ( t ) ), gr F) } ,
F ( t, x) == {v E R n I (x, v) E T _ ( ( x ( t ), 5; ( t ) ), gr F) } ,
where (t,x) E [O,T] x R n . The differential inclusions
x E F(t, x)
and
x E F ( t, x )
play an important role in the estiinating of derivatives of the map x 
S (F, x). We sl1all call thein variational inclusions or first approximations
along the solution x(.). They are silnilar to the variational equation along a
solution to an ordinary differential equation because they approxiinate the
perturbation of the solution caused by a sinall perturbation of the initial
conditions.
Theorem 4.13. Assume that the set-valued map F : Rn  Rn with closed
convex values satisfies the Lipschitz condition with a constant l > o. Then
for any u E R n the inclusions
(4.30)
(4.31 )
hold.
S[o,T](F,u) c D+(u),
D_(u) c S[o,T](F,u)
Proof. Prove first inclusion (4.30). Let x(.) E S[o,T](F, u), and let A > o.
Put
p(t, A) == A -ld(5; + AX(t), F(x(t) + AX(t))).
From Proposition 2.10 we have liInA!O p(t, A) == o. Froin the Lipschitz con-
dition and Proposition 2.6 we have
p(t, A) < A -ld(5;(t), F(x(t))) + Ix(t)1 + llx(t)1
== Ix(t)1 + llx(t)l.
The function in the right side is integrable. By the Lebesgue theorem we
obtain
(4.32) lim rTp(t,>')dt=O.
A!O Jo
Applying Tl1eorein 4.5 with !vI == {x(.) + AX(.)} and ro(x) == x, we see that
there exists a solution x A (-) E S[o,T](F,x(O) + AU) satisfying
Ix,\(-) - (5:(.) + >'X('))IAC < >.1 T el(T-s) p(s, >')ds.

4.6. Dependence on initial conditions
III
From (4.32) we obtain
limA- 1 Ix A (.) - (x(.) + Ax(.))IAC == O.
AiD
Fron1 Theoreill 4.12 we know that the n1ap x  S(F, x) is Lipschitzian.
Applying Proposition 2.10, we obtain x(.) E D+(u).
Now we prove inclusion (4.31). Let xC) E D_(u). By Theorem 4.12 and
Proposition 2.10 there exist sequences Ak 1 0 and Xk(.) E S[O,T] (F, X(O)+AkU)
such that
lin1 Ak1Ixk(.) - (x(.) + AkX(.))IAC == o.
k-7OO
Obviously x(O) == u. Ivloreover, there exist subsequences {Ak p } and {Xkp(.)}
such that
lilll A k -=-llxk (t) - (£(t) + Ak x(t))1 == 0
p-700 P P P
for ahnost all t E [0, T]. Applying Proposition 2.6, we have
lilll inf A -1 d ( £ ( t) + AX ( t ) , F ( x ( t) + AX ( t ) ) )
AiD
< lilll inf A k 1 d(£( t) + Ak x( t), F(x( t) + AkpX( t)))
p-700 P P
< liIll (A k 1 d(Xk p (t), F(Xkp(t))) + Ak1Ixkp(t) - (£(t) + AkpX(t))1
p-700 p p
+A;:llxkp(t) - (x(t) + AkpX(t))I) = 0
for ahnost all t E [O,T]. This inequality in1plies (4.31).
D
Corollary 4.8. Let F(t,x) == F(t,x) for all (t,x) E [O,T] x R n . Then
D + ( u) == D - ( u) == S ( F, u) == S ( F , u).
Theoreill 2.4 contains the sufficient conditions guaranteeing the equality
F (t, x) == F (t, x) for differential inclusions generated by control systeills.
Suppose that the right-hand side is single-valued, that is, F(x) == {f(x)},
and f is Lipschitzian and directionally differentiable. Then froill Corol-
lary 4.8 we see that the solutions to the differential equation
X == f(x)
are directionally differentiable in initial data. If f is continuously differ-
entiable, then we obtain the classical theoreill on the differentiability with
respect to the initial data.

112
4. Existence Theorems
Uniform differentiability. Let F : R n  R n be a set-valued Inap with
closed convex values satisfying the Lipschitz condition with a constant l >
O. Since the map x  S[o,T](F,x) is Lipschitzian (Theorem 4.12), froln
Proposition 2.10 a11d inclusion (4.30) we see that there exists a map 0 : R+ x
AC([O,T],R n )  AC([O,T],R n ) such that for any x(-) E S[o,T](F,u), U E
R n we have
1. X(.)+AX(-)+O(A,X(.)) ES[O,T](F,x(O)+AU), A > O,
2. lim,X!o A -llo(A, x(.)) lAc == O.
Our ailn is to construct a map 0 : R+ x AC([O, T], R n )  AC([O, T], R n )
satisfying the above two conditions and such that the map x(.)  O(A,X(.))
is continuous in the norln of AC([O, T], R n ) and the convergence in the sec-
ond condition is uniform in xC).
Theorem 4.14. Let Xi(.) E S[o,T](F), i == 1, k , and let
X == co{ Xl (- ), . . . , X k ( . )} c A C ( [0, T], R n ).
Assume that co{ (Xi (t), Xi (t)) I i == 1, k } c grF almost everywhere. Then
there exists a map 0 : R+ x X  AC([O, T], R n ) satisfying the following
conditions.
1. x(-) + AX(.) + O(A, x(-)) E S[O,T] (F, x(O) + AX(O)), A > O.
2. The map x(.) ---+ O(A, x(.)) is continuous in AC([O, T], R n ).
3. lim,X!o maxx(')EX A -llo(A, x(.)) lAc == o.
Proof. Set r == {)' == ()'1,... ,)'k) E R k I I:  l)'i == 1, )'i > 0, i == 1, k }.
Define a function p : [0, T] x R+ x r  R+ by
p(t, A, ,) = A -ld ((t) + A  ,iXi(t), F (i:(t) + A  ,iXi(t)) ) .
As in Theorem 4.13 we have lim,X!o p(t, A,)') == 0 almost everywhere and
k k
(4.33) p(t, A, ,) < L ,iXi(t) + l L ,iXi(t) < m(t),
i=l i=l
where m(t) == maxi IXi(t)1 + lmaxi IXi(t)l. Let)'l E rand)'2 E r. Then we
have
p ( t, A, )'1) - p ( t, A, )'2)
< A -ld (  + A t ,iXi(t), F ( i:(t) -1' Xi(t) ) )
1=1 1 1
-A -ld ( + A  ,Xi(t), F (i:(t) + A  'Xi(t)) )

4. 7. Discrete approxinlatiollS
113
k k k
+l Lb1 - I')Xi(t) < Lb1 - I')Xi(t) + l Lb1 - I')Xi(t)
i=l i=l i=l
( 4.34)
k
< m(t) L 11'1 - I'I.
i=l
Set p(t, A) == max')'Er p(t, A, ",/). Let r' be a countable everywhere dense
subset in r. Since p(t, A,.) is continuous, we have p(t, A) == sUP')'Er' p(t, A, ",/).
Therefore the function p ( ., A) is measurable.
Show that
( 4.35)
lim p ( t, A) == 0, t E [0, T].
A10
Let t E [0, T] be such that limAlo p(t, A, ",/) == 0, V",/ E r. Suppose that there
is a sequence Aj 1 0 such that liInioo p( t, Aj) > O. Let ",/j E r be such that
p( t, Aj) == p( t, Aj, ",/j). Without loss of generality ",/j  "'/0 Eras j  00.
(We can select a convergent subsequence, if needed.) Froln (4.34) we have
k
p ( t, ).. j) = p ( t, ).. j , I' j) < p ( t, ).. j , 1'0) + m ( t) L 1 I' j - I' b I.
i=l
Taking the limit as j  00, we obtain a contradiction.
From (4.35), (4.33), and the Lebesgue theorem we 11ave
( 4.36) lirn (T p( t, ).. )dt = O.
A10 Jo
Applying T11eorem 4.5 with M == {x(.) + AX} and ro(x) == x, we see that
there exists a continuous in the norln of AC([O, T], R n ) map r : IvI  S(F)
such that r(x(.) + AX('))(O) == x(O) + AX(O) and
Ir(xC) + )..x(.)) - (x(.) + )..xC)) lAC < ).. faT el(T-s) p(s, )")ds
whenever x(.) E X. Froln (4.36) we obtain
Ihn Inax A-II r ( X ( .) + AX ( . )) - (x ( .) + AX ( . ) ) lAC == O.
A10 x(')EX
Setting O(A, x(.)) == r(x(.) + AX(')) - (x(.) + AX(')), we obtah1 the result. 0
4.7. Discrete approximations
We proceed to study discrete approximations for a differential inclusion
( 4.37)
X E F(x).

114
4. Existel1ce Tlleorell1s
Theorem 4.15. Let F : R n  Rn be a set-valued map with closed convex
values satisfying F(x) C bBn, where b > o. Assume that F is Lipschitzian
with a constant l > O. Then for any solution x(.) to differential inclusion
(4.37) there exist solutions Zi,N, i == 0, N , N == 1,2,..., of the discrete
inclusions
Zk,N E Zk-1,N + 6NF(Zk-1,N), k == 1, N ,
ZO,N == x(O), 6N == TIN
such that the functions
WN(t) == 6 N 1 (Zk,N - Zk-1,N), t E [(k - 1)6N, k6N[, k == 1, N,
converge to x(.) in the norm of the space £2([0, T], Rn).
( 4.38)
Proof. Let N be a positive integer. Consider the following Ininilnization
probleln
N j k8 N
L Ii;(t) - vkl dt  inf,
k==l (k-1)8N
where t11e Ininimuln is taken over all vectors (VI,... , V N) E Rn x ... x
Rn == RnN. Obviously this problem has a unique solution (V1,N, . . . , V N,N ).
Consider t11e function VNC) : [0, T]  Rn defined by
VN(t) == Vk,N, t E [(k -1)6N,k6N[.
Obviously we have
lim (T Ii;(t) - vN(t)ldt = O.
Noo Jo
Set
YN(t) = x(O) + 1 t VN(S)
and denote Yk,N == YN (k6 N ), k == 0, N . Let us define solutions of discrete
inclusion (4.38) as follows:
ZO,N == x(O),
Wk,N == 1f(Vk,N,F(Zk-1,N)),
ZkN==Zk-1N+6NWkN, k== l , N .
, , ,
Using the definitions, the Lipschitz condition, and Proposition 2.6, we have
(4.39) I W 1,N - V1,N! == d( V1,N, F(YO,N)), I Z 1,N - Y1,N I == 6N I W 1,N - V1,N I,
IWk,N - Vk,NI == d(Vk,N,F(Zk-1,N))
(4.40) < d(Vk,N, F(Yk-1,N)) + llzk-1,N - Yk-1,N!, k == 2, N ,
and
IZk,N - Yk,N! < I Z k-1,N - Yk-1,N! + 6N!Wk,N - Vk,N!

4. 7. Discrete approxilnatiol1s
115
(4.41) < IZk-1,N - Yk-1,N I (1 + 6 Nl) + 6 Nd( Vk,N, F(Yk-l,N)), k == 2, N .
By hlduction froln (4.39) - (4.41) we obtain
k
IZk,N - Yk,NI < L(1 + 8Nl)k- i 8Nd(Vi,N, F(Yi-l,N)), k = 1, N .
i==l
COllsequently we have
N N k
L IZk,N - Yk,NI < L L(1 + 8Nl)k- i 8Nd(Vi,N, F(Yi-l,N))
k==l k==li==l
N N
< L(1 + 8Nl)N8N Ld(Vi,N,F(Yi-l,N))
k==l i==l
N N
= N 8 N (1 + 8Nl)N L d( Vi,N, F(Yi-l,N)) < Te lT L d( Vi,N, F(Yi-l,N)).
i==l i==l
Combining this inequality with (4.39) and (4.41), we obtain
N N
(4.42) L8Nlwk,N - Vk,NI < (1 + lTe 1T ) L8Nd(Vk,N,F(Yk-l,N)).
k==l k==l
The inequality
N N l k8N
L8 N d(Vk,N,F(Yk-l,N)) = L d(Vk,N,F(Yk-l,N))dt
k==l k==l (k-1)8 N
N l k8N
< L (d(Vk,N,F(YN(t))) + lb(t - (k -1)8 N ))dt
k==l (k-1)8 N
= I T d(VN(t), F(YN(t)))dt + lbT8 N /2
< I T Ix(t) - vN(t)ldt + I T lIYN(t) - x(t)ldt + lbT8N /2
< (1 + IT) I T Ix(t) - vN(t)ldt + lbT8N/2
and (4.42) hnply that the functions
WN(t) == Wk,N, t E [(k - 1)6N, k6N[, k == 1, N
satisfy
lirn (T IWN(t) - vN(t)ldt = O.
N-+oo Jo
Consequently we have
lirn (T IWN(t) - x(t)ldt = 0
N-+oo Jo

116
4. Existence Theorems
and
lirn (T IWN(t) - x(t)1 2 dt < lirn 2b (T IWN(t) - x(t)ldt = O.
Noo Jo Noo Jo
The theorem is proved.
D
Now establis11 a converse statement.
Theorem 4.16. Let F : R n ----t R n be a set-valued map with closed con-
vex values contained in a ball bBn. Assume that F is Lipschitzian with a
constant l > o. Then for any solution Zi, i == 0, N , of the discrete-time
inclusion
Zk E Zk-1 + 8F(Zk-1), k == 1, N , 8 == TIN
there exists a solution x(.) E S[O,T] (F, zo) satisfying
Ix(t) - y(t) I < 8be lT , t E [0, T]
and
Ix(t) - w(t)1 < 81b(e lT + 1), t E [0, T],
where w(t) == 8- 1 (Zk - Zk-1), t E [(k - 1)8, k8[, k == 1, N , and y(t)
Zo + Jw(s)ds.
Proof. Let t E [(k - 1)8, k8[. Then applying Proposition 2.6, we get
d(iJ(t), F(y(t))) == d(8- 1 (Zk - Zk-1), F(y(t)))
< d(8- 1 (Zk - Zk-1),F(Zk-1)) +lly(t) - Zk-11
== ll(t - (k - 1)8)(zk - zk-1)18 + Zk-1 - Zk-11 < 81b.
Applying Tlleorem 4.5 with M == {y(.)} and ro(x) == x, we obtain the
result. D
From Theorems 4.15 and 4.16 we see that a discrete-time h1clusion is a
good approximation for a Lipschitzian differe11tial inclusion.
4.8. Problems
1. Let F : R n ----t R n be a bounded Lipschitzian set-valued Inap with
closed convex values, and let Vo E F(xo). Prove that there exists
a solution x(.) E S[o,T](F, xo) satisfying x(O) == vo. (Hint. Apply
Theorem 4.5 with Ivl == {xo + tvo}.)

4.8. Problems
117
2. Let F : R n  R n be a bounded Lipschitzian set-valued map with
closed convex values, a11d let Vo E F(xo). Prove that there exists
a continuously differentiable solution x(.) E S[O,T] (F, xo) satisfying
x(O) == vo. (Hint. Consider polygonal approximatio11s x(N)(.) with
the nodal points xN) = XQ, vN) = VQ, xk N ) = xk N ) + ( T /2 N )vk N ),
(N) F( (N) ) I (N) (N) I - d( ( (N) )) k - N
v k E x k ' v k - 1 - v k - Vk-1,F x k ' - 0,2 . De-
fine continuous approximations v(N) (.) of t he d erivatives x(N) (.) by
interpolating Ih1early the points vk N ), k == 0, 2 N , and show that the
sequence v(N) (.) satisfies the conditions of the Arzela-Ascoli theorem.)
3. Prove Theorem 4.10.
4. Consider the optimal control problem
( 4.43)
<p(T, x(T)) + I T 'IjJ(x(t), u(t))dt  inf,
x ( t) == f ( x ( t ) , u ( t ) ) , t E [0, T],
u(t) E U,
x(O) E Co, x(T) E C 1 , T E [0, T*].
Assume that the functions f, cp, and 'ljJ are continuous. More-
over there exists a constant c > 0 such that 'ljJ(t, x, u) > c for all
(t, x, u). Let (8, p(.), q(.), t(-), z(.)) be an optimal control process for
the problem
cp( t(8), z( 8)) + 8  inf,
dt(O) p(O)
dO 'IjJ ( z ( 0), q ( 0) ) , 0 E [ 0, 8],
dz ( 0) = p ( 0) f ( z ( 0 ), q ( 0 ) ) 0 [ 0 8 ]
dO 'IjJ ( z ( 0), q ( 0) )' E, ,
q ( 0) E U, P ( 0) E [0, 1],
z(O) E Co, z(8) E C 1 , t(O) == 0, t(8) < T*.
Assume that
p( 0) >
'IjJ(i(O),q(O)) - a > 0, 0 E [0,8].
Prove that there exists O(t), the inverse of the function t(O), and the
control process ('r, u(.), x(.)), where x(t) == z(O(t)), u(t) == q(O(t)),
T == £(8) is optimal for problem (4.43).
5. Consider problem (4.43). Assume that If(x,u)1 < b(l + lul), the sets
Co e C1 are compacts, and the set
F(x) == {(vo,v) E R x R n I v O > 'ljJ(x,u), v == f(x,u), u E U}

118
4. Existellce Theorellls
8.
( 4.44 )
is convex for all x. It is also assulned that either U is compact or U is
closed and 'ljJ(x, u) > c(l + luI 1 +1'), 'r > o. Prove that problem (4.43)
has an optimal solution, provided that there exists an admissible one.
(Hint. Use the previous problem and Theorem 4.10.)
6. Study the exis tence i n the brachistochrone problem
l b V I +x2
a vx dt -+ inf x(a) = A, x(b) = B.
(Hint. Use Problem 4.)
7. Study the existence in the surface of revolution of minimum area
probleln
l b x V I + i; 2 dt -+ inf x(a) = A, x(b) = B.
(Hint. Use Problem 4.)
Consider the linear control systeln
x == Cx + u, u E U,
where C is an n x n matrix, and U is a convex compact set satisfying
U C Bn. Consider the discrete control system
(4.45) Xi+1 == (E - TC)-l xi + TUi, Ui E U, i == 0, k - 1.
Let a number T > 0 and a positive integer k be such that TIT <
k < (TIT) + 1 and T/k < ICI- 1 . Show that for any trajectory xC)
of systeln (4.44) with x(O) == 0 there exists a trajectory of discrete
system (4.45) Xo, . . . , Xk, Xo == 0 SUCll that
T
IXk - x(T)1 < k"(2 + IGI)2e 21C1T .

Chapter 5
Viability and
Invariance
The viability problelll is to select trajectories of a differential inclusiol1 which
are 'viable' in the sense that they satisfy given constraints. We derive nec-
essary and sufficient conditions for a differe11tial inclusion to have viable
trajectories. This problen1 is closely related to the probleln of finding n10l10-
tOl1e solutions to a differential inclusio11, and all proofs here are based on
silnple Ino110tonicity ideas. Another problelll considered 11ere is the i11vari-
ance probleln: we find conditions guara11teeil1g that all trajectories of a
differential inclusion satisfy given constraints. We apply the obtained re-
suIts to prove the existence of periodic solutions to a differential inclllsion
with a tiIne periodic right-hand side and to construct pursuit controls in
differential galnes.
5.1. Monotone solutions to a differential
incl usion
We start with a very siInple theoren1 concerning the existence of monotol1e
solutions. Consider a differe11tial inclusion
(5.1)
x ( t) E F ( x ( t ) ) , t E [0, T],
where F : R n -4 Rn is a set-valued Inap. Let V : R n x [0, T] -4 R be a
function. We say that a solution x(-) E S[O,T] (F) is lnonotone with respect
to V(-) iff
v ( x ( t2 ), t2) < V ( x ( t 1 ), t 1 ) , \j t 1 , t2 E [0, T], t 1 < t 2 .
-
119

120
5. Viability and Invariance
In this section we establish sufficient conditions for existence of a mono-
tone solution under rat11er restrictive assuillptions 011 the map F and the
fUl1ction V.
Theorem 5.1. Assume that the set-valued map F : R n  R n has closed
convex values contained in a ball of radius b > 0 and is Lipschitzian with
a constant l > O. Let function V : Rn x [0, T]  R be Lipschitzian with
a constant L > 0, and for any (x, t) E Rn x [0, T[ let there exist a vector
v E F(x) such that
D-V(x, t)(v, 1) < o.
Then, given Xo ERn, there exists a solution xC) E S[o,T](F, xo) monotone
with respect to V.
Proof. Fix Xo E R n and f > O. Consider a function (x, t) == V(x, t) -
V(xo,O) - ft. Our first issue is to prove the existence of a solution x€(.) E
S[O,T] (F, xo) such that  (x€ (t), t) < f for all t E [0, T]. For any function
xC) E AC([O,T],R n ) define
8(xC)) == max{t E [0, T] I max (x(s), s) < f, (x(t), t) < O},
sE [O,t]
and put
i == sup{8(x(.)) I xC) E S[o,T](F, xo)}.
Show that there exists x ( .) E S[O,T] (F, xo) such that i == 8 (x ( . ) ) . Indeed,
consider a sequence Xi (.) E S[O,T] (F, xo), i == 1, 2, . .. such that 8( Xi C)) i i.
Since the map F is bounded, by the Arzela-Ascoli theorem the sequence Xi(.)
contains a uniformly convergent subsequence. Without loss of generality the
sequence Xi(.) uniformly converges to a function xC). By Theorelll 4.6 we
have xC) E S[o,T](F,xo). Taking the limit in the inequalities
max (Xi(S), s) < f,
sE[O,O(Xi('))]
 (Xi ( 8 (Xi C ) ) ) , 8 (Xi ( . ) )) < 0,
we obtain i == 8( x(.)).
Now prove that i == T. Suppose that i < T. Then we have (x(i), i) ==
o. There exists a vector fJ E F(x(i)) such that
( 5.2) D -  ( x ( i), i) ( fJ, 1) < - f .
Consider the function y(t) == x(i)+(t-i)fJ. Since F is Lipschitzian, we have
d(iJ(t), F(y(t))) == d(fJ, F(x(i) + (t - i)fJ)) < llfJllt - il.
From this inequality and Theorem 4.5 we see that there exists a solution
xC) E S[i,T] (F, x ( i)) such that
(5.3) Iy(t) - x(t)1 < a(t - i) = it e1(t-S)Zlvl(s - i)ds.

It is easy to check that
lim a(t-f) =0.
tlt (t - t)
Since the function V is Lipschitzian with a constant L > 0, from (5.3) we
have
(5.4)
 ( x ( t ), t) <  (y ( t ), t) + La (t - i).
From inequality (5.2) we see that there exists a number i E]i, i + E(4L(b +
1) )-1 [ such that
(y(i), i) == (x(i) + (i - i)v, i) < -E(i - i)/2
and
- '" E - '"
a(t-t) < 2L (t-t).
From this and (5.4) we obtain (x(i), i) < o.
Let t E [i, i]. Then from (5.4) we have
(x(t), t) < (x(i) + (t - i)v, t) + La(t - i)
< (x(i),i)+L(t-i)+L(t-i)lvl+La(t-i) < 4(b: 1) + 4(bb: 1) +  < E.
Thus we see that (x(t), t) < E whenever t E [i, i]. Moreover, we have
(x(i), i) < O. This contradicts the definition of i. Consequently we obtain
i == T. Denote x€ (.) == x(.). The solution x€ (.) satisfies the inequality
(5.5) max  (x€ (t), t) < E.
tE [O,T]
Consider a sequence Ei 1 o. Applying the Arzela-Ascoli theorem and
Theorem 4.6, without loss of generality we conclude that the sequence X€i (.)
uniformly converges to a solution xC) E S[o,T](F, xo). Taking the limit in
(5.5), we obtain V(x(t), t) < V(x(O),O) whenever t E [0, T]. In particular
we have V(x(T), T) < V(x(O), 0) == V(xo, 0).
To prove the existence of a monotone trajec tory we divide the segment
[0, T] into k subintervals [iT/k, (i + l)T/k], i == 0, k - 1. Arguing as above,
we find a solution xl(.) E S[O,T/k] (F, xo) satisfying
V(xl(T/k), T/k) < V(xl(O), 0).
Then we find a solution x(-) E S[T/k,2T/k](F,Xk(T/k)) satisfying
V(x(2T / k), 2T /k) < V(x(T /k), T /k),
etc. Define a solution Xk(.) E S[o,T](F, xo) by
Xk(t) == xi(t), t E [iT/k, (i + I)T/k], i == 0, k - 1.

122
5. Viability and Invariance
By COI1structioI1 the SOllltion Xk (.) satisfies
(5.6) V(xk((i + I)T/k), (i + I)T/k) < V(xk(iT/k), iT/k), i == 0, k - 1.
Let t1 E [0, T] and t2 E [0, T] be SUCl1 that t1 < t2. There are i and j
such that t1 E [iT/k, (i + l)T/k] and t2 E [jT/k, (j + l)T/k]. Since F is
bouI1ded aI1d V is Lipschitzian, from (5.6) we obtain
V(Xk(t2), t2) < V(xk(jT/k),jT/k) + L(b + I)T/k
(5.7) < V(xk(iT/k), iT/k) + L(b + l)T/k < V(Xk(t1), t1) + 2L(b + I)T/k.
Applyh1g Arzela-Ascoli theoreln and Theoreln 4.6, without loss of generality
we conclude that the sequeI1ce XkC) uniformly converges to a solution xC) E
S[o,T](F,xo). Taking the Ihnit in (5.7), we see that the solution xC) is
Inonotone with respect to the function V. 0
In tl1e next section we use Theoreln 5.1 to study the viability problen1.
Then we shall COlne back to the existeI1ce of monotoI1e trajectories and
obtain a more general result.
5.2. Viability problem
In this sectioI1 we obtain I1ecessary and sufficient cOI1ditions for the existence
of a solution to a differential inclusion
(5.8)
x(t) E F(x(t))
contai11ed h1 a given set CeRn.
Theorem 5.2. Let CeRn be a closed set. Assume that the set-valued map
F : C  R n with closed convex values contained in a ball of radius b > 0 is
upper semi-continuous. Then the following conditions are equivalent.
1. For any point Xo E C there exists a solution x(.) E S[o,oo[(F, xo)
satisfying
X(t) E C, V t E [0,00[.
2. For any point x E C the following condition holds:
F(x) n T_(x, C) i- 0.
Proof. Suppose that the first condition is satisfied. Let Xo E C, and let
xC) E S[o,oo[(F, xo) be a solution satisfying x(t) E C, t > O. Sh1ce F is upper
seIni-coI1th1uOUS, there exists a sequence tk 1 0 such that
1
F(x(t)) c F(x(a)) + k Bn, V t E [a, tk]'

5.2. Viability problem
123
By Lemina 4.2 we have
(5.9) X(tk) - x(O) =  (t k x(s)ds E F(x(O)) + k 1 Bn.
tk tk Jo
Consequently the sequence t k 1 (x(tk) - x(O)) is bounded. Without loss of
generality t k 1 (x(tk) - x(O))  v as k  00. Sh1ce F(x(O)) is closed, from
(5.9) we obtain v E F(x(O)). On the other hand we have
lim inf A -ld(x(O) + AV, C)
,XlO
(5.10) < liminft k 1 d(x(0) + tk(t k 1 (x(tk) - x(O)) + lXk), C),
koo
where lXk == v - t k 1 (x(tk) - x(O))  0 as k  00. Obviously the right side
of (5.10) is less than or equal to
liminf(t k 1 d(x(tk), C) + IlXkl) == o.
koo
Thus v E F(x) n T_(x, C).
Now suppose that the second condition is satisfied. Applying Theo-
reIn 2.6, we extend the set-valued Inap F to the whole space. We denote this
extension again by F. Obviously the extension satisfies F(x) nT_(x, C) =1= 0
whenever x E C. Let us show that for any Xo E C there is a sollltion
x(.) E S[o,oo[(F, xo) satisfying x(t) E C, t > O.
Assume that F : R n  R n is Lipschitzian with a constant l > O. We
need the following auxiliary result.
Lemma 5.1. For any x E R n there exists a vector v E F(x) such that
D-d(x,C)(v) < ld(x,C).
Proof. Let y E 7f(x, C). There exists a vector w E F(y) nT_(y, C). By the
Lipschitz condition there is a vector v E F(x) satisfying
(5.11)
Observe that
D- d(x, C)( v) == lim inf A-I (d(x + AV, C) - d(x, C))
,Xlo .
== lim inf A-I (d(x + AV, C) - d(y + AW, C) + d(y + AW, C) - d(x, C))
,Xlo
< liminf A -l(lx - yl + Alv - wi + d(y + AW, C) - d(x, C))
,XlO
== liln inf A -ld(y + AW, C) + Iv - wi == Iv - wi.
,X10
Froin this and (5.11) we obtain D-d(x, C)(v) < ld(x, C). The lemma is
proved. D
Iv - wi < llx - yl == ld(x, C).

124
5. Viability and Invariance
End of the proof of Theorem 5.2. Let T > O. Consider the function
V(x, t) == e-ltd(x, C) defined in the set R n x [0, T]. From Lelnma 5.1 we see
that for any (x, t) E R n x [0, T[ there is a vector v E F(x) such that
D-V(x, t)(v, 1) == e- lt D- d(x, C)(v) -le-ltd(x, C) < O.
By Theoren1 5.1 for any Xo E R n there exists a solution x(.) E S[O,T] (F, xo)
Inonotone with respect to V. In particular if Xo E C, then we have
e-ltd(x(t), C) == V(x(t), t) < V(x(O), 0) == 0;
that is, x(t) E C for all t E [0, T].
Thus if F is Lipschitzian (or locally Lipschitzian), then the existence
of a solution xC) E S[O,T] (F, xo) satisfying x(t) E C for all t E [0, T] is
proved. To prove this claim for an upper selni-continuous map F, let us
approximate F by a sequence of Lipschitzian maps. By Theoreln 2.5 there
exists a sequence of locally Lipschitzian set-valued maps Fk : Rn  Rn, k ==
0, 1, . . . , satisfying the following conditions:
1. F(x) c ... C Fk+1(X) C Fk(X) C ... C Fo(x) C MBn for all x ERn;
2. given E > 0 and x E Rn, there exists a positive integer k( E, x) such
that Fk(X) C F(x) + EBn whenever k > k(E,X).
Since F(x) C Fk(X), we have Fk n T_(x, C) -I- 0 whenever x E C. From the
above considerations we conclude that, given Xo E C, there exists a solution
x k ( .) E S[O, T] (Fk, xo) satisfying
( 5.12) x k ( t) E C, t E [0, T], k == 1, 2, . .. .
Since Fk (x) C bBn, the set of functions x k (.) is bounded and equicontin-
uous. By the Arzela-Ascoli tl1eoreln there exists a uniformly convergent
subsequence. Without loss of generality the sequence Xk (.) uniformly con-
verges to a functio11 xC). By Theorem 4.6 x(.) E S[O,T] (F, xo). Taking the
Ihnit in (5.12), we obtain x(t) E C, t E [0, T].
Now we can repeat t11e previous reasoning for the seglnent [T,2T] with
the initial condition x(T), etc. Finally we obtain x(.) E S[o,oo[(F, xo) satis-
fying x(t) E C, t > o. 0
It is easy to prove a time-dependent version of Theorem 5.2. Consider
a differential inclusion
(5.13)
x(t) E F(t, x).
The theorem proved below contains necessary and sufficient conditions for
t11e existence of solutions to (5.13) satisfying
(5.14)
x(t) E C(t),
for all t E [to, 00[, where C : R  R n is a set-valued Inap.

5.2. Viability problem
125
Theorem 5.3. Assume that the set-valued map C : R  Rn has a closed
graph and the set-valued map F : grC  Rn is upper semi-continuous and
has closed convex values contained in a ball of radius b > O. Then the
following conditions are equivalent.
1. For any point (to, xo) E grC there is a solution x (-) to inclusion (5.13)
with x(to) == Xo satisfying (5.14).
2. For any (t, x) E grC the following condition holds:
F(t, x) n D_C(t, x)(l) =1= 0.
Proof. Consider the differential inclusion
dx
ds E F(t, x),
dt == 1
ds
and the set grC c R x Rn. The result now follows fron1 Theoreln 5.2. 0
In practical problems we usually need to guarantee condition (5.14) only
on a final time interval. Consider a set-valued Inap C : [0, T]  Rn with a
closed graph and an upper semi-continuous set-valued Inap F : grC  Rn
with closed convex values contained in a ball of radius b > O.
Theorem 5.4. The following conditions are equivalent.
1. For any point (to, xo) E grC there is a solution x (.) to inclusion (5.13)
with x(to) == Xo satisfying (5.14) for all t E [to, T].
2. For any (t, x) E grC, t < T, the following condition holds:
F(t, x) n D_C(t, x)(l) =1= 0.
Proof. Applying Theorem 5.3 to the set-valued map
C(t) == { C(t), t E]O, T[,
R n , t 5t'] 0, T[,
we obtain the result.
o
Note that the convexity of the right-hand side is essential for the viability
property. Indeed, consider the set C == B2 and the differential inclusion
(xl, x 2 ) E F(x 1 , x 2 ),
where F : B2  R 2 is given by
F(x 1 ,x 2 ) == {(-1,0), (1,0)}.
It is easy to see that the differential inclusion has no viable solution with
the ini tial condition (0, 1) E C.

126
5. Viability and Invariance
Now we are in a position to prove a new result on the existence of
monotone solutions.
Theorem 5.5. Let CeRn be a closed set, and let F : C --t R n be an
upper semi-continuous set-valued map with closed convex values contained
in a ball of radius b > o. Assume that V : C --t Rand W : C --t Rare
continuous functions. Then the following conditions are equivalent:
1. For any point Xo E C there exists a solution xC) E S[o,oo[(F, xo)
satisfying
(5.15) V(X(t2)) - V(X(t1)) + l t2 W(x(s))ds < 0, \;f t2 > tl > O.
tl
2. For any point x E C there exists a vector v E F(x) such that
D-V(x)(v) < -W(x).
Proof. Suppose that the first condition is satisfied. Let Xo E C, and
let x (-) E S[O,oo[ (F, xo) be a sol u tiOll satisfying (5.15). As in the proof of
Theorem 5.2 we show that there exists a sequence tk 1 0 such that fJ;l(x(tk)-
x(O)) --t v E F(x(O)). From (5.15) we have
V(xo + tk(tJ;l(x(tk) - xo))) - V(xo)
tk
= V(X(tk)) - V(xo) < _ t k W(x(s))ds.
tk tk J o
Taking the limit, we obtain D-V(xo)(v) < - W(xo).
Now suppose that the second condition is satisfied. Let Xo E C. Consider
the set A == C n (xo + bBn). Fix E > O. Since W is continuous, there exists
a positive integer k such that IW(X1) - W(X2)' < E whenever Xl E A and
X2 E A satisfy 'Xl - x21 < blk. Consider the differential inclusion
(5.16)
( X, Q) E F (x) X {O} c R n x R
and the set-valued map G : R --t Rn x R defined by
o
G(t) == {(x, a) , V(x) < a - t(W(xo) + E), x E C n (xo + k- 1 b Bn)}
o
U{(x, a) , x E C \ (xo + k- 1 b Bn)}.
The second condition implies that for any (t, x, a) E grG there exists v E
F(x) satisfying (v,O) E D_G(t, x, a)(l). From Theorem 5.3 we see that
there exists a solution (x(-), a(.)) to (5.16) with (x(O), a(O)) == (xo, V(xo))
such that (x(t), a(t)) E G(t), t > o. In other words
V(x(t)) < V(xo) - t(W(xo) + E), t E [0, Ilk].

5.3. Invariant sets
127
Hence there exists a solution x(.) E S[O,l/k] (F, xo) satisfying
1
V(x(l/k)) < V(xo) - k (W(xo) + f).
Put Xl == x(l/k). Arguing as above, we prove that there exists a solution
x(.) E S[1/k,2/k](F,X1) satisfying
1
V(x(2/k)) < V(Xl) - k (W(Xl) + f),
etc. Thus we see that there exists a solution Xk(-) E S[o,l](F,xo) satisfying
(5.17)
J
V(xk(j/k)) - V(xk(i/k)) + L W(xk(p/k))/k < o.
p=
By the Arzela-Ascoli theoreln the sequence Xk(') contains a uniformly con-
vergent subsequence. Without loss of generality Xk (.) uniforlnly converges
to a function x(.). By Corollary 4.5 x(.) E S[o,l](F,xo). Since V and Ware
continuous, froln (5.17) we obtain
l t2
V(X(t2)) - V(X(t1)) + W(x(s))ds < 0, V 1 > t2 > t1 > O.
tl
Now we can extend the solution to the interval [0,2], etc. Finally we find a
solution satisfying (5.15). D
5.3. Invariant sets
In this section we study invariant sets of a differential inclusion
(5.18)
x(t) E F(x(t)).
We say that a set CeRn is invariant by differential inclusion (5.18) if,
given X E C, any solution x(.) E S(F, x) satisfies x(t) E C, t > O.
Theorem 5.6. Let F : R n  R n be a set-valued map with closed convex
values, and let CeRn be a closed set. Assume that F is Lipschitzian with
a constant l > O. Then the following conditions are equivalent.
1. The set C is invariant by differential inclusion (5.18).
2. For any point x E C the following inclusion holds:
F(x) c T_(x, C).

128
5. Viability and Invariance
Proof. Let C be invariant by inclusion (5.18). Consider points x E C
and v E F(x). Applying Theorem 4.5 with M consisting of one function
y(t) == x + tv, we see that there exists a solution x(.) E S(F, x) satisfying
Ix(t) - (x + tv)1 < I t e1(t-s)d(v, F(x + sv))ds
< I t el(t-s)sllvlds = Ilvlt 2 j2+o(t 2 ).
By invariance x(t) E C and hence we have
liminf a- 1 d(x + av, C) < liminf a- 1 lx + av - x(a)1
a!O a!O
< liminf(llvla/2 + o(a)) == O.
a:!O
Thus v E T_(C,x).
Now suppose that the second condition is satisfied. Consider a solution
xC) E S(F). The functions xC) and g(t) == d(x(t), C) are absolutely con-
tinuous. Suppose that there exist the derivatives x(t) and g(t) at a point
t. Let y E 7r(x(t), C) and w E T_(y, C). There exist sequences ak 1 0 and
Uk  0 such that y + akw + akuk E C, k == 1,2, . . .. Then we have
g(t) == lima- 1 d(x(t + a), C)
a!O
== lima- 1 (d(x(t) + a(x(t) + o(a)), C) - d(x(t), C))
a!O
< lim a;;l(lx(t) + akx(t) - (y + akw)I-lx(t) - yl) < Ix(t) - wi.
k ---4 00
Since w E T_(y, C) is an arbitrary vector we obtain
g(t) < d(x(t), T_(y, C)) < d(x(t), F(y))
< lly - x(t)1 == ld(x(t), C) == 19(t).
If g(O) == 0, then we have g(t) = 0 due to the Gronwall inequality. The
theorem is proved. 0
Note that in the case of the upper semi-continuous right-hand side the
condition F(x) c T_(x, C), \Ix E C, is neither necessary nor sufficient.
Indeed, let C ==] - 00,0] c R, F(x) == {x 1 / 3 }. Obviously F(x) c T_(x, C)
for all x E C. However there exists a solution x(.) E S(F,O) leaving C.
On the other hand the set C ==] - 00,0] c R is invariant by a differential
inclusion with the right-hand side F(x) == {O} everywhere except the point
x == 0, where we put F(O) == [0,1]. However F(O) ct T_(O, C).
Assume that an upper semi-continuous map F : C  R n with closed
convex values satisfies the condition F(x) c T_(x, C), \Ix E C. Since the
map F is defined on C only, the question whether there exist solutions that

5.3. Invariant sets
129
leave C is not appropriate. The possibility of having solutions to the differ-
ential inclusion that do leave C depends entirely on what happens outside
C. Therefore we may ask whether there exists an upper semi-continuous
extension F of the map F such that C is invariant by the inclusion
X E F(x).
The following example shows that in general the answer is negative.
Let C be the union of the two circles C 1 == bdB2 + (1,0) and C2
bdB2 - (1,0), and let F(x) be the vector of unit length tangent to C at x in
the direction of counterclockwise rotation for C 1 and of clockwise rotation
for C 2 (at the origin F is the vector (0,1)). The map F is continuous and
satisfies the condition F(x) c T_(x, C), \:Ix E C. There are two solutions
to the Cauchy problem x E F(x), x(O) == 0, namely the two rotations with
angular speed 1, counterclockwise on C 1 and clockwise on C2. Let F be any
upper selni-continuous extension of F to a neighborhood of C. Consider
a neighborhood of the origin where F is defined and such that the second
component of a vector from F is at least 1/2 on it (it is 1 at (0,1)). Let
8 > 0 be so small that no solution to x E F(x), x(O) == 0, can leave this
neighborhood on [0, 8]. If C is invariant by the differential inclusion, then
R[o,o] (F, 0) c C. By Corollary 4.6 the set R[o,o] (F, 0) is connected and
therefore contains points both of C 1 and C 2 (belonging to the two solutions
mentioned above). Hence R[o,o] (F, 0) contains the origin. However the mean
value theorem implies that for every solution x(.) we have Ix( 8) I > 8/2, a
contradiction. Thus C cannot be invariant by the inclusion x E F.
We face another situation if C is convex. In this case it is possible to
construct an extension having the invariance property.
Theorem 5.7. Let CeRn be a closed convex set, and let F : C  R n
be an upper semi-continuous map with closed convex values. Assume that
F(x) C T_(x, C) for all x E C. Then there exists an upper semi-continuous
- -
set-valued map F : R n  R n with closed convex values such that F(x) ==
F( x), x E C, and the set C is invariant by the differential inclusion
x E F(x).
Proof. Set F(x) == F(1f(x, C)). Let v E F(x). Then (x - 1f(x, C), v) < 0
(see Remark after Theorem 1.3). Consider a solution x(.) E S(F, C) and
suppose that x(T) t/. C, where T > O.
Let x t/. C. From Lemmas 1.1 and 3.4 we see that the function d 2 (., C)
is differentiable at x and \7d 2 (x, C) == 2(x - 1f(x, C)). Let t E [0, T] be such

130
5. Viability and Invariance
that x(t) <t C. Then we have
 d 2 (x(t), C) = (Vd 2 (x(t), C), x(t)) = 2(x(t) - 1f(x(t), C), x(t)) < o.
Consequently  d2(x(t), C) < O. From this we obtain
o < d 2 (x(T), C) - d 2 (x(0), C) = I T  d 2 (x(t), C)dt < 0,
a contradiction.
D
5.4. Existence of periodic solutions
In this section we establish sufficient conditions for the existence of periodic
solutions to a differential inclusion with periodic right-hand side. First we
prove that the reachability set of a differential inclusion
x(t) E F(t, x(t))
is a O"-selectionable Inap. N alnely, the followh1g result is true.
Theorem 5.8. Let CeRn be a convex compact set, and let F : [0, T] x
Rn  R n be an upper semi-continuous set-valued map with closed convex
values contained in a ball of radius b > o. Assume that F(t, x) nT(x, C) :/=- 0
for all (t, x) E [0, T] xC. Then the set-valued map <I> : C  C defined by
<I> ( x) == {y E R n I :3 x (-) E 5[0, T] (F, x ) ; x ( t) E C, t E [0, T] ; x (T) == y}
is 0" -selectionable.
Proof. From Theoreln 5.3 we see that <I>(x) :/=- 0 for all x E C. By The-
orems 2.6 a11d 2.5 there exists a sequence of locally Lipschitzian set-valued
Inaps Fk : [0, T] x R n  R n , k == 0, 1, . . . , satisfying the following conditions:
1. F(t,x) C ... C Fk(t,X) C Fk-1(t,X) C ... c Fo(t,x) C bBn for all
( t, x) E [0, T] x R n ;
2. for any E > 0 and any (t, x) E [0, T] x R n there exists a positive integer
k(E, t, x) such that Fk(t, x) C F(t, x) + EBn whenever k > k(E, t, x).
Fix a positive integer k. The Inap Fk is Lipscllitzian with a constant l > 0 on
the set [0, T] x (C + Bn). Take E E]O, min{I/(kl), e- lT /(2kT)} and consider
the function
(t, x) == e-ltd(x, C) - Et.
By Lelnlna 5.1 for any (t, x) E [0, T] x Rn there is a vector v E Fk(t, x)
satisfying
D-(t,x)(I,v) < -E.

5.4. Existence of periodic solutions
131
Since
D-(t, x)(l, v) == -le-ltd(x, C) - E + e- lt Dd(x, C)(v),
by Corollary 1.11 the function D-(., .)(1, v) is upper semi-continuous.
Hence there is 8 E]O, E[ such that D-(t', x')(l, v) < -E/2 for all (t ' , x') E
(t, x)+8Bn+1. Cover the compact set [0, T] x (C+ETe lT Bn) by the balls ofra-
dius 8 and choose a finite subcovering. Let the points (ti, Xi), i == 1, N , be the
centers of the balls from the subcovering, and let Vi E Fk(ti, Xi), i == 1, N ,
be vectors satisfying D-(ti, xi)(l, Vi) < -E.
By Lemn1a 2.5 there exists a Lipschitzian partition of unity {pi(., .)} f 1
subordinated to the covering. Put
N
f€(t, x) = I:>i(t, X)Vi'
i=l
We have
D(t, x)(l, fE(t, x)) == e-lt(D-d(x, C)(fE(t, x)) -ld(x, C)) - E
N
< I:>i(t, x)D-(t, x)(l, Vi) < -  .
i=l
Let x E (-) E S[O,T] (fE' C). The function t ---7 (t, xE(t)) is absolutely
continuous, and at points where  and X E are differentiable we have
 (t, x€(t)) = D-(t, x€(t))(l, x€(t))
E
= D-(t,x€(t))(l, f€(t,x€(t))) < - 2 '
From this we obtain
Therefore
Et
(t, x€(t)) < -2'
d(x€(t), C) <  TeIT <  .
Observe that
d(f€(t, x), Fk(t, x)) = d (  Pi(t, X)Vi, Fk(t, X))
N N 1
< I:>i(t,X)Vi - LPi(t,X)Wi < if. < k '
i=l i=l
where Wi E Fk(t, x) are such that IVi - wil < l(lti - tl 2 + IXi - xI 2 )1/2. Thus
- 1
x€(t) E Fk(t,x€(t)) = Fk(t,x€(t)) + k Bn, t E [O,T],
and
1
x€(t) E C + k Bn, t E [0, TJ.

132
5. Viability and Invariance
Define the lTIap <I> k : C  R n by
<I>k(X) == {y E R n 13x(.) E S[O,T] (Fk, x);
1
x(t) E C + k Bn, t E [0, T]; x(T) = y}.
The map x  R[O,T] (fE' x) is a continuous selection of <I>k. Obviously <I> (x ) c
<I> k ( X ) , x E C, k == 0, 1, . . . . Applying the Arzela- Ascoli theorem and
Lemma 4.4, we conclude that gr<I>k is closed and <I>(x) == nkO <I>k(X). Thus
<I> is O"-selectionable. D
Now consider a differential inclusion
(5.19)
x(t) E F(t, x(t)),
where F is periodic in t of period T > 0; that is, F(t + T, x) == F(t, x) for
all (t, x) E R x Rn.
Theorem 5.9. Let CeRn be a convex compact set, and let F : R x R n 
R n be an upper semi-continuous set-valued map with closed convex values
contained in a ball of radius b > o. Assume that F(t, x) nT(x, C) =I- 0 for all
(t, x) E [0, T] xC. Then there exists a periodic solution xC) to differential
inclusion (5.19) with period T such that x(t) E C, t E [0, T].
Proof. By Theorem 5.8 the set-valued map <I> : C  C defined by
<I>(x) == {y E R n I 3 x(.) E S[o,T](F,x); x(t) E C, t E [0, T]; x(T) == y}
is O"-selectionable. Invoking Theorem 2.8, we see that there exists a solution
xC) E S[O,T] (F) such that x(O) == x(T) and x(t) E C, t E [0, T]. Since F is
periodic in t of period T, we conclude that there exists a periodic solution
of (5.19) contained in C. D
Note that convexity of values F(t, x) is essential for the existence of
periodic solutions. Indeed, consider the set-valued map F : R 2  R 2 defined
by
F(x 1 ,x 2 ) == {(v 1 ,v 2 ) I vI == :1:1, v 2 == Ix 1 1}.
Let C == B2. The condition F(x) n T(x, C) =I- 0 is obviously satisfied.
However the inclusion x E F(x) has no a periodic solution, since x 2 (t) >
x 2 (0) whenever t > O.
5.5. Pursuit in a differential game
Consider two controllable objects in R n . One of thelTI pursues another. The
goal of the first object is to intercept the second, while the aim of t11e second

5.5. Pursuit in a differential galne
133
is to escape from pursuit. One faces this situation when one aircraft pursues
another. The pilots control the aircrafts, keeping their goals in view and
using current inforlnation about the situation: the positions of the aircrafts,
their velocities, and technical possibilities, but nothing can be known about
the future behavior of the aircrafts. Control problems of this type are known
as differential games.
To construct a Inathematical model of the pursuit process, denote by x
the phase vector of the pursuer and by y the phase vector of the evading
object. Assulne that the dynamics of these objects is described by a pair of
control systems
x == f(x,u), u E U;
iJ == g(y,v), v E V.
The phase vectors x and y can be represented in the form x == (xl, x 2 )
and y == (y1, y2), respectively, where xl and y1 are the coordinates of the
objects while x 2 and y2 are their velocities. It is assumed that the pursuit
is finished if xl == y1, that is, if the coordinates of the objects coincide;
or, Inore generally, the pursuit is finished if Ixl - y11 < l, that is, if the
coordinates are close enough.
Introd ucing vectors z == (x, y) and F == (f, g) we can write the pair of
differential equations in the form
i == F( z, u, v), u E U, v E V.
Denote by M the set of points z on which the game terminates. This is a
general differential game. If the dynamics is described by a linear control
system
( 5.20 )
i == Az - u + v, u E U, v E V,
where U and V are convex compact sets, and the terminal set has the form
M == M + lvlo, where M is a linear subspace and Mo is a convex compact
set, the differential game is called a linear differential game.
The general differential game problem is to find a time-optimal pursuit
control u(z) and a time-optimal evasion control v(z). This problem (even
the linear one) is extremely complicated and is still far from being solved.
Here we consider a simple example of a linear differential game and show
that using geometric ideas of the viability theory a pursuit control (not
necessarily optimal) can be easily found.
De110te by L the orthogonal complement of M, L == M.l, and by 7r the
orthogonal projection onto L. The pursuit process starting at z == Zo and
governed by the controls u(.) and v(.) terminates at the moment t == T only
if for any measurable function v(t) E V, t E [0, T], there exists a measurable

134
5. Viability and Invariance
function u(t) E U, t E [0, T], such that
7reTAzo E Mo + iT (7re(T-r)A u (r) - 7re(T-r)A v (r)) ds.
In terlTIS of the Riemann integral (see Problems 1.5 and 2.4) this condition
can be written as
7reTAzo E W(T) = Mo + iT (7re(T-r)Au ..:. 7re(T-r)Av) ds,
if the integral exists. The set W (t), after the change of variable s == T - r,
takes in the fornl
(5.21) W(t) = Mo + it (7re SA U ..:. 7re SA V) ds.
We do not study the linear differential game in its general setting (see Prob-
lem 5 below). Instead we consider a simple example and show that the
set-valued map W(t) satisfies some viability conditions and tllerefore can be
used to construct a pursuit control.
Assume that the behavior of the pursuer and of the evader is described
by the control system
x == -ax + p, p E pBn,
y == -{3y + q, q E ()" Bn,
where n > 2. We have the control p ill our disposal and know the current
positions x(t) alld y(t). The aim is to find a cOlltrol p == p(t, x, y), t E [0, T],
and a set of initial positions (xo, Yo, xo, Yo) such that the pursuit process
terminates at a time not exceeding T, that is x(t) == y(t) for some t < T.
Put zl == x - y, z2 == x, and z3 == y. We shall use the notation z ==
(zl,z2,z3) E R 3n . Now the motion of the objects can be described by the
following control system:
i == Az - u + v,
where
A= (
E -E )
-aE 0,
o -{3E
u = ( -),
and
v=().
It is easy to see that in this example M == M == {z I zl == O}, and therefore
VV (t) == ')'( t )Bn, where
r 1 - e- as 1 - e-{3s
-y(t) = Jot w(s)ds and w(s) = a P - (J CT,
and
1 e- at 1 - e-{3t 3
7re tA z = zl + - a z2 - (J Z.

5.6. Problems
135
Let T > O. The pursuit objective now is to find a control u  u(t, z),
t E [0, T], such that for any initial position z there exists T < T such that
zl(T)  o. Assume that
p a
(5.22) p > cr, and 0: > (3 .
Under these c011ditions the function w(s), s E [0, T], is positive. C011sider
the set
C(t)  {z E R 3n l1fe(T-t)A z E W(T - t)}, t E [0, T].
Put
" 1fe(T-t)A z . { !1fe(T-t)A z! }
u(t, z) = -p l7fe(T-t) A z l illm 1, 'Y(T _ t) ·
Obviously the function z  il(t, z), t < T, is locally Lipschitzian. Using the
result of Problelll 2.8 one can easily to check the condition DC(t, z)(l) n
(Az - il(t, z) + v) =I- 0 for all v E aBn, whe11ever (t, z) E grC. Let Zo E C(O)
be an initial point, and let v(t), t E [0, T], be a continuous function. Then
frolll Theorem 5.4 we see that the Cauchy problelll
(5.23)
(5.24)
i  Az - il(t, z) + v(t),
z(O)  Zo,
has a unique solution z(.) satisfying z(t) E C(t). Therefore there exists
T < T such that zl(T)  o.
5.6. Problems
1. Let F : R n  R n be a bounded upper sellli-co11tinuous set-valued
map with closed convex values, let P c R n be a C011vex cOlllpact set,
and let y(.) E AC([O, T], R n ). Assullle that a nonnegative function
C) E AC([O, T], R) satisfies the differential i11equality
sup max ((iJ(t), q)
pEbdP -qEN(p,p)nbdB n
(5.25) -(t)S( -q, P) - S(q, F(y(t) + (t)p))) < 0,
for all t E [0, T]. Prove that for any Xo E y(O) + (O)P there exists a
solution x(.) E S[o,T](F, xo) satisfying x(t) E y(t) + (t)P, t E [0, T].
2. Let F : R n  R n be a bounded Lipschitzian with constant l > 0
set-valued map with closed convex values, let P  Bn, and let yC) E
AC([O, T], R n ). Show that the function
(t) = eltl xo - y(O)1 + I t e1(t-s) p(s)ds, t E [0, T],

136
5. Viability and Invariance
where p(t) == d(iJ(t),F(y(t))), satisfies (5.25) and, therefore, there
exists a solution xC) E S[o,T](F,xo) satisfying Ix(t) - y(t)1 < (t),
t E [0, T] .
3. Consider the systelTI
x == ax + by + cp(t, x, y),
iJ == ex + dy + V; ( t, x, Y ) .
It is assumed that the roots Al and A2 of the characteristic polynomial
det ( a - A b ) == 0
e d- A
satisfy ReAl . ReA2 > O. The functions cp and V; are continuous and
T-periodic. Moreover
lim sup I ( <p, 1/J )( t, x, y) I = 0,
l(x,y)loo t I(x, y)1
Prove that the system has at least 011e T-periodic solution.
4. Consider a differential game described by the control system
x == p, P E pBn,
y == q, q E aBn,
where n > 2. The terlTIinal set is {(x,y) II x - yl < l}, l > a 2 j(2p).
Show that if Ixo - Yo + Txol < l + pT 2 - aT, then the game ca11 be
terminated at a time not exceeding T. Find a pursuit control.
5. Consider linear differential game (5.20). Assume that 0 E U, 0 E V,
IvIo == {O}, intW(t) =I 0, t > 0, the normal cone N(7re tA z, W(t)) is
a ray at any point 7re tA z E W (t), and for any x* =I 0 there exists
only one point u E U satisfying S(x*, U) == (x*, x). Show that for any
initial point Zo satisfying 7reTAzo E W(T) the game can be terminated
at a time not exceeding T. Find a pursuit control. (Hint. Use the
result of ProblelTI 2.9(b).)
6. Consider a differential game described by the system
x == p,
x ERn
,
y E R n ,
Ipl < p,
Iql < a,
y == q,
where p > a. The terminal set is given by M == {(x, y) E R n x R n I
x == y}. Write down this problem in the form (5.20); then find the
set W (t) and a pursuit control.
7. Do the same for the terminal set given by IvI == {(x, y) E R n x R n I
Ix - yl < l}.

5.6. Problems
137
8. Do the same for the differential game
x == p, x E R n , Ipl < p,
j.j == q, Y E R n , I q I < (J,
where p > (J. The terminal set is given by M == {(x,y) E Rn x Rn I
x == y}.
9. Do the same for the terminal set given by M == {(x, y) E Rn x Rn I
Ix - yl < l}.

Chapter 6
Controllabili ty
The set of x-controllability of a differential inclusion is the set of all initial
points such that at least one solution to the corresponding Cauchy problem
reaches the point x. In applications, it is usually of interest to k110W w11ether
the x-controllability set contains a given point. For example, it is clear
that the time-optimal problem with the boundary conditions x(O) == Xo
and x(T) == Xl makes sense only if Xo belongs to the xl-controllability
set. If the x-controllability set contains a neighborhood of the point x,
the differential inclusion is said to be locally controllable around x. Using
some ideas of duality and the multivalued version of the Jordan theorem for
convex processes, we establish necessary and sufficient conditions of global
controllability for convex processes. We then derive local controllability
of a differential i11clusion from the controllability of a convex process that
approximates the differential inclusion in a l1eighborhood of an equilibrium
point.
6.1. Duality relation
Consider the linear differential equation
x(t) == Cx(t) + f(t)
and its adjoint
x*(t) == -C*x*(t) - g(t).
Integrating the equality
d
dt (x, x*) = (Cx + f, x*) - (x, C*x* + g),
-
139

140
6. Controllability
we get the well-known Green formula
(x(T),x*(T)) - (x(O),x*(O)) = 1 T ((f(t),X*(t)) - (x(t),g(t)))dt.
Here we establish a multivalued analogue of the Green formula in terms of
convex allalysis.
Let A : R n  R n be a closed strict convex process. Consider differential
incl usions
(6.1)
x(t) E A(x(t))
and
(6.2)
x*(t) E -A*(x*(t)).
Let T > O. Consider two convex cones
PT == {p E R n I 3x(-) E S[o,T](A,p), x(T) == a}
and
QT == {q E R n 13x*(.) E S[o,T](-A*,q)}.
Our ahn is to establish a duality relation between PT and QT.
Theorem 6.1. The following equality holds:
P T == -QT.
Proof. Let q E QT and p E PT. Then there are trajectories x*(.) E
S[o,T](-A*,q) and x(.) E S[o,T](A,q).with x(T) == o. Observe that
(q,p) = (x*(O), x(O)) - (x*(T), x(T)) = -1 T :t (x*(t), x(t))dt
= -1 T ((x*(t), x(t)) + (x*(t), x(t)) )dt < 0,
where the last inequality follows from the definition of the adjoint process.
Hence -QT CPT.
Now let -q E PT. Then the trajectory x(t) = 0, t E [0, T] is a solution
to the following extremal problem:
(6.3) minimize{ -(q, x(O)) I x(.) E S[O,T] (A), x(T) == O}.

6.1. Duality relation
141
Let N be a positive integer. Put T == T / N and consider the minimization
problem
T ( q, [ Vk ) + T ? (Vk' Vk)  inf,
VN E A(O),
VN-1 E A( -TVN),
(6.4)
*** *** ***
Vk E A( -T(VN + VN-1 + . . . + Vk+1)),
*** *** ***
VI EA(-T(VN+...+V2)).
We shall denote a vector (VN,... , VI) E R Nn by v. Consider the vector
ij == (q, . . . , q) E R N n, the cone
lC == {( X N, . .. , Xl, V N, . . . , VI) E R 2 N n I (x k, V k) E gr A, k == 1, N },
and the linear operator A : R Nn ---+ R Nn X R Nn give11 by
Av == A( V N, . . . , VI)
== (0, -TVN, -T(VN + VN-1),.. . , -T(VN + .. . + V2), VN, VN-1,. . . , VI).
Problem (6.4) can be written in the form
(6.5)
w(v) ---+ inf,
Av E lC,
where w(v) == T(ij, v) + T(V, v). Since w(v) ---+ 00 as Ivl ---+ 00, we have
w(v) > 0 whenever Ivl > p. By the Weierstrass theorem the problem
w ( v) ---+ inf,
Av E lC, Ivl < p
has a solution v. Obviously w(v) < o. Therefore v is also a solution to
problem (6.5). The function
(v) == T(ij, v) + T(V, v) + 8(v, A -llC)
(here A -1 is a set-valued map) achieves its global minimum at the point v.
Therefore its subdifferential at v contains zero. From Theorem 1.16 we 11ave
(6.6) 0 E 8(v) == Tij + 2TV + 88(v, A -llC) C Tij + 2TV - (A -lK)*.
Check that AR Nn -lC == R 2Nn . Let (x, w) == (X!\T, . . . , Xl, WN, . . . , WI) E
R Nn X R Nn . Show that there exists a vector v satisfying
(x,w) E Av -lC.

142
6. Controllability
This inclusion is equivalent to
VN E A( -XN) + WN,
VN-1 E A( -TVN - XN-1) + WN-1,
(6.7)
*** *** ***
Vk E A(-T(VN +... + Vk+1) - Xk) + Wk,
*** *** ***
VI E A(-T(VN +... + V2) - Xl) + WI.
Since dOlnA == R n , we see that system of inclusions (6.7) has a solution
- - ( ) .c ( - - ) E R N n R N n
v - V N , . . . , VI lor any x, W x.
Applying Corollary 1.6, from (6.6) we obtain
o E Tij + 2TV - A*J(*.
It implies that there exists a vector (x*, v*) E J(* such that
T(q,... ,q)+2T(VN,... ,VI)
( N-1 N-2 )
= -7 L Xk + Vtv, -7 L Xk + Vtv -1' . . . , -7X + V2 , vt .
k=l k=l
Put Yk == Xk/T, k == 1, n . Then we have
yr E -A*(q + 2V1),
*** *** ***
(6.8)
( k-1 )
Yk E - A * q + 7  Yi + 2Vk ,
=1
*** *** ***
( N-1 )
Ytv E - A * q + 7 h Yi + 2v N .
Let us esthnate the norm IVk I. To this end c011sider the function
(t - T) VI - T (V2 + . . . + V N ) , t E [0, T],
(t - 2T)V2 - T(V3 + ... + VN), t E [T,2T],
x( t) ==
* * * * **
***
(t - kT)Vk - T(Vk+1 + . . . + VN), t E [(k - l)T, kT],
* * * * **
***
(t - NT)vN, t E [(N - l)T, NT].
Put c == e T1A1 . Applying Theorem 4.5 to differential inclusion (6.1) with
M == {x(.)}, we see that there exists a solution x(.) E S[O,T] (A) with x(T) ==
o satisfying
Ix(O) - x(O)1 < c I T d(i(t),A(x(t)))dt

6.1. Duality relation
143
N l kT N l kT
= c L d(Vb A(x(t)))dt < c L d(Vk, A(x(k1')))dt
k=l (k-1)T k=l (k-1)T
N l kT 1 N
+clAI L It - k1'llvkl dt = -c1AI1'2 L IVkl
k=l (k-1)T 2 k=l
(see Proposition 2.6 and Theorem 2.12 (Proposition 1)). Since the function
x(t) = 0, t E [O,T] solves minimization problem (6.3), we have (-q,x(O)) >
o. Observe that iJ == 0 is an admissible point for minimization problem (6.5).
Therefore we obtain
(6.9)
N
o > (-q,x(O)) +1' L(Vk,Vk) + (q,x(O)) - (q,x(O))
k=l
N N
> (q, x(O) - x(O)) + l' L(Vk' Vk) > -lqllx(O) - x(O)1 + l' L(Vk' Vk).
k=l k=l
Thus we have
N
Iqllx(O) - x(O)1 > 1'L(Vk,Vk)'
k=l
Combining this with (6.9), we obtain
N 1 N 1 L N I" I
1'L(VbVk) < 2 1qlcIAI1'2Llvkl = 2 1qlclAI1'T k Vk .
k=l k=l
Since the arithmetic mean is always less thaI1 or equal to the root mean
square, we have
N ( N ) 1/2
 (Vb Vk) <  IqlclAIT Lk=lk' Vk)
From this we obtain
N ( 1 ) 2 1
 (Vk,Vk) < 2 1qlciAIT N '
Finally we get
(6.10)
IVkl < JN ' k = 1,N,
where c is a constant.

144
6. Controllability
Consider the function
q + tyi,
q + Tyi + (t - T )Y2'
* * * * **
q + T(yi + . . . + Yk)
+(t - kT)Yk+l'
* * * * **
t E [0, T],
t E [T,2T],
***
xiv(t) ==
t E [kT, (k + l)T],
***
q + T(yi + · . . + YN-l)
+(t - (N - l)T)YN' t E [(N - l)T, NT].
From (6.10), (6.8), and Theorem 2.12 (Proposition 2) we have
Ixiv(O)1 == Iql,
Ixiv(kT)1 < Ixiv((k - l)T)1 + TIAllxiv((k - l)T)1 + 2TIAll v kl
< (1 + rIAI)lxjV((k - l)r)1 + 2C , k = 1, N.
Hence we obtain
IxjV(kr)1 < (1 + TI ) N Iql + 2C (1 + (1 + TI )
( TIAI ) N-l )
+...+ 1+ N
( TIAI ) N 2c (( TIAI ) N ) 2c
= 1 + N Iql + .jN 1 + N - 1 < clql + .jN (c - 1).
From this we have
Ixiv(t)1 < c(lql + c), t E [0, T],
and
Ixiv(t)1 < IAlc(lql + c), t E [0, T]
whenever N is sufficiently large.
Now let N tend to infinity. From the above inequalities and the Arzela-
Ascoli theorem we see that the sequence xiv (.) contains a uniformly conver-
gent subsequence. Without loss of generality xiv(.) uniformly converges to
a function x* ( . ). Since
xiv(t) E -A*(xiv(t) + ZN(t)),
where SUp{IZN(t) II t E [0, T]}  0 as N  00, we have x* (.) E S[O,T] (-A*, q)
due to Theorem 4.6. This ends the proof. 0

6.2. Controllability of convex processes
145
6.2. Controllability of convex processes
Consider a convex process A : Rn  Rn and the differential inclusions
(6.11 )
x(t) E A(x(t))
and
(6.12)
x*(t) E -A*(x*(t)).
In the previous section we defined convex cones PT and QT. Obviously
PTI c PT2' QT2 C QTl
whenever T 1 < T2. Put
P = U P T and Q = n QT.
T>O T>O
From Theorem 6.1 we see that P* == -Q.
We say that a convex process A : Rn  Rn is controllable if P == Rn.
We say that a convex process A : Rn  Rn is controllable at time T if
PT == Rn. These two definitions are equivalent. Indeed, controllability at
time T implies controllability. If the process is controllable, then consider
a simplex  == CO{X1,. .. ,X n +1} such that 0 E int. There are positiv e
numbers Ti, i == 1, n + 1, and trajectories Xi (.) E S[O,Ti] (A, Xi), i == 1, n + 1,
such that xi(T i ) == O. Since the set of solutions to the differential inclusion
(6.11) is a convex cone and any poin t X E  can be represented as a convex
combination of th e point s Xi, i == 1, n + 1, we see that PT == Rn, where
T == max{Ti Ii == 1,n+ I}.
The following result allows us to answer the question whether a convex
process is controllable using inforlnation about the structure of the adjoint
process.
Theorem 6.2. Let A : Rn  Rn be a closed strict convex process. Then
the following conditions are equivalent.
1. The process A is controllable.
2. The adjoint process has neither proper invariant subspace nor eigen-
vectors.
Proof. Let A be controllable. Suppose that A * has a proper invariant
subspace L. Since the restriction of A* to L is a linear operator (see The-
orem 2.12) that maps L into itself, we see that the differential inclusion
(6.12) has a solution with x*(O) == x* whenever x* E L. Hence L c Q.
Since P* == -Q, we see that P :I Rn, and therefore A is not controllable, a
contradiction.

146
6. Controllability
Suppose that there is an eigenvector x* of the process A *. Let A be a
corresponding eigenvalue. Observe that the function e->-.t x * is a solution to
(6.12). Hence x* E Q and P =I=- R n , a contradiction. Thus the first condition
in1plies the second one.
Now suppose that the second condition is satisfied. In view of Theo-
rem 6.1 it suffices to show that Q == {O}.
First prove that Q does not contain a line. Let L == Qn-Q =I=- {O}, and let
x* E L be a nonzero vector. There exists a solution x* ( .) E S[O,oo[ ( - A * , x*).
Obviously we have -x* (-) E S[O,oo[ ( - A *, -x*). From this and the definition
of Q we obtain x*(t) E Q and -x*(t) E Q, for all t > O. In other words
x*(t) E L, t > O. Thus we see that x*(.) is a solution to the linear differential
equation
x*(t) E -A*IL(X*(t)).
Therefore x* (.) is differentiable at any point t > O. Taking the limit as t 1 0
in the inclusion
x*(t) - x*
t EL, t > 0,
we obtain -A*(x*) == {x*(O)} E L. Since x* E L is an arbitrary point, we
see that L is invariant by A *, a contradiction.
Suppose that Q =I=- {O}. Consider convex processes Ak : Rn  R n ,
k == 1, 2, . . . , defined by
Ak(X*) == R[O,l/k] ( -A*, x*).
Obviously Q C domAk and Ak(x*) n Q =I=- 0 for all x* E Q and k == 1,2, . . . .
Since Q does not contain a line, we can apply Theorem 2.13 and conclude
that there exist vectors x k E Q n bdBn and numbers Ak > 0, k == 1,2,. . . ,
satisfying AkXk E Ak(xk). Let xk(.) E S[o,I](-A*,x k ) be such that
{Ilk
AkXk = Xk + Jo x'k(t)dt.
From TheorelTI 2.12 (Proposition 2) and the Gronwall inequality we see
that the sequence of functions x k ( .) is bounded and equicontinuous. By
the Arzela-Ascoli theorem it contains a uniformly convergent subsequence.
Without loss of generality the sequence xk(.) uniformly converges to a func-
tion x*(.). By Theorem 4.6 x*(.) E S[O,I] ( -A*). Put x* == x*(O). Obviously
Ix*1 == 1.
Let E > O. Then we have
A*(Xk(t)) C A*(x*) + EBn, t E [0, Ilk]
when k is sufficiently large. Consequently
(Ilk
AkX'k = x'k + Jo x'k(t)dt E x'k +  (-A*(x*) + EBn).

6.3. Controllability at first approxinlation
147
From this we obtain
( 6.13)
k(Ak - l)xk E -A*(x*) + tBn.
Since the set in the right side of (6.13) is bounded and the sequence xk
converges to x* E bdBn, we see that the sequence k(Ak - 1) is bounded.
Without loss of generality k(Ak - 1) ---+ A as k ---+ 00. Taking the limit in
(6.13), we obtain
AX* E -A*(x*) + tBn.
Since t is an arbitrary positive nUlllber, we see that x* is an eigenvalue of
A *, a contradiction. Thus the second condition implies controllability of A.
o
Consider a convex process of the forlll A(x) == Cx + K, where C is an
(n x n)-matrix and KeRn is a closed convex cone. The adjoint process is
gi ven by
A* ( * ) == { C*v*, v*EK*,
v 0, v* tJ. K*
(see Example in Section 2.6). Therefore we obtain the following result.
Corollary 6.1. The following conditions are equivalent.
1. The process x ---+ Cx + K is controllable.
2. The linear operator C* has neither proper invariant subspaces con-
tained in K* n - K* nor eigenvectors contained in K* .
6.3. Controllability at first
approximation
Consider a differential inclusion
(6.14)
x(t) E F(x(t)),
where F : R n ---+ R n is a set-valued map. We assume that the origin is an
equilibrium position of inclusion (6.14), that is, 0 E F(O).
We can approximate F in the vicinity of the origin by a convex process
A : R n ---+ R n satisfying grA c T+( (0,0), grF). As we know frolll Theo-
rem 4.14 the set of solutions to the variational inclusion
x(t) E A(x(t))
is a tangent cone to the set of solutions to inclusion (6.14). Therefore one
can try to derive controllability of the original differential inclusion from

148
6. Controllability
controllability of the variational inclusion provided that the latter is con-
trollable. This type of controllability is referred to as controllability at first
approximation.
Differential inclusion (6.14) is said to be locally controllable around x == 0
at time T if there exists t > 0 such that for any Xo E tBn there exists a
solution x(.) E S[O,T] (F, xo) satisfying x(T) == O.
Primal form of controllability conditions. The following result con-
tains sufficient conditions of local controllability at first approximation.
Theorem 6.3. Let F : R n  R n be a Lipschitzian set-valued map with
closed convex values, and let A : Rn  R n be a closed strict convex process
satisfying grA c T+((O, 0), grF). Assume that A is controllable at time
T > O. Then differential inclusion (6.14) is locally controllable around x == 0
at time T.
Proof. To say that inclusion (6.14) is locally controllable around x == 0 at
time T amounts to saying that 0 E intR[o,T] ( - F, 0). Consider a simplex  ==
co{ Xl, . .. , X n +1} satisfying 0 E int. Since Rn == R[O,T] ( - A, 0) , there are
solutions Xi(') E S[O,T] ( -A, 0) such that xi(T) == Xi, i == 1, n + 1. Consider
the set X == co{ Xl (.), . . . , Xn+1 (.)}. By Theorem 4.14 there exists a map
o : R+ x AC([O, T], R n )  AC([O, T], Rn) satisfying the following conditions:
1. x(.) + o(, x(.)) E S[O,T] ( -F, 0),  > 0;
2. the map x(.)  o(A, x(.)) is continuous in the norm of AC([O, T], R n );
3. lim.x!O  -1 maxx(')EX IO(A, x(.)) lAC == O.
Put a == min{d(x, !) I X E bd}. There exists AD > 0 such that
max lo(o,x('))1 < oa.
X(')EX
Let X E . Then there is a unique collection of numbers G"i > 0 such
that E G"i == 1 and x == E G"iXi. Consider a continuous map f : o  Rn
gi ven by
f(>..ox) = f (Ao  O"iXi) = Ao  O"iXi(T) + 0 (Ao,  O"iXiC)) (T).
Let x E ! and x == E +l G"iXi E bd. Then we have
If(Aox) - Aoxi = 0 (Ao,  O"iXiC)) (T) < Aoa < IAox - Aoxl.
By Corollary 2.1 ox E imf. Hence ! o C R[O,T] ( - F, 0). The theorem is
proved. 0

6.3. Controllability at first approximation
149
Consider the differential inclusion
(6.15) x E f(x, U) = U f(x, u),
uEU
where f : Rn x Rm  Rn is a continuous function differentiable in x,
and U c R m is a compact set. Assume that there is Uo E U such that
f(O, uo) == O. Let the derivative \l xf(x, u) be continuous in (x, u), and let
the set f(x, U) be convex for all x ERn.
From Theorem 6.3 and Proposition 2.8 we obtain the following result.
Corollary 6.2. If the convex process x  \l xf(O, uo)x + conef(O, U) is
controllable at time T > 0, then differential inclusion (6.15) is locally con-
trollable around x == 0 at time T > O.
Dual form of controllability conditions. In Theorem 6.3 we have de-
rived local controllability of a differential inclusion from controllability of its
first approximation. Obviously the first approximation is controllable only
if it is a strict convex process. Such an approximation may not exist. Nev-
ertheless it is possible to prove sufficient conditions of local controllability
in a dual form using nonstrict first approximations. From Theorem 6.1 we
know that controllability of a strict convex process A is equivalent to the
following condition: the adjoint differential inclusion
(6.16)
x * ( t) E - A * ( x * ( t ) ) , t E [0, T]
does not have a nontrivial solution. The last condition does not require
strictness of A and implies local controllability of the original differential
inclusiol1.
Theorem 6.4. Let F : R n  R n be a Lipschitzian set-valued map with
closed convex values, and let A : R n  R n be a closed convex process sat-
isfying grA c T+((O,O),gr}'). Assume that differential inclusion (6.16) has
only a trivial solution. Then differential inclusion (6.14) is locally control-
lable around x == 0 at time T.
Proof. Note again that local controllability of inclusion (6.14) around x == 0
at time T is equivalent to the condition 0 E intR[o,T] (-F, 0).
- -
Consider set-valued maps F : Rn x R  R n x R and A : Rn x R  R n x R
given by
F(x,x o ) == {(v,v o ) E R n x R I v O > d(v,F(x))}
and
A(x,x o ) == {(v,v o ) E R n x R I v O > (1 + l)d((x,v),grA)},
where l > 0 is the Lipschitz constant of F. The map F is obviously Lips-
chitzian with the constant l. From Lemma 2.4 we have
grF == {(x,xo,v,v o ) E R n x R x R n x R I v O > d(v,F(x))}

150
6. Controllability
c {(x,xo,v,vo) ERn x R x R n x R I vO > (1 +l)d((x,v),grF)}.
Applying Proposition 2.7, we obtain
T+ ((0,0,0,0), grF)
c {(x, x O , v, va) E R n x R x R n x R I vO > (1 + l)d((x, v), T+((O, 0), grF))}
c {(x, x O , v, va) E R n x R x R n x R I vO > (1 + l)d((x, v), grA)} == grA.
- - -
In other words A is a first approxilnation of F at the origin. Moreover A is
a closed strict convex process. By Corollary 1.2 we have
(6.17) (grA)* == cone{ (x*, 0, v*, 1) I (x*, v*) E (1 + l)( (grA)* n B 2n )}.
Obviously R[O,T] (-A, (0,0)) is a C011vex cone contained in R n x R+.
Observe that there is no vector (v, v O ) E (R[O,T] ( -A, (0,0)))* with v i- o.
Indeed, if such a vector exists, then by Theorem 6.1 there is a solution
(x*(.), xo*(.)) E S[O,T] ( -A*, -(v, va)). From (6.17) we see that x*(.) E
S[o,T](-A*, -v) is a nontrivial soution to (6.16), a contradiction. Thus
we have i11t(R n x R+) c R[O,T] (-A, (0,0)).
Let E == CO{X1'... ,X n +1} C R n be a simplex satisfying 0 E intE. Put
a == min{d(x, E) I x E bdE} and b == !ae- lT . Since int(R n x R+) c
R[O,T] ( - A, (0,0)), there are solutions (Xi (.), x?(.) ) E S[O,T] (- A, (0,0)) such
that xi(T) == Xi and x?(T) == b, i == 1, n + 1. Consider the set X ==
CO{(X1(.),X(.)),... , (Xn+1(.),X+1(-))}. By Theorem 4.14 there exists a
map (0,0°) : R+ x AC([O, T], R n x R) ---+ AC([O, T], R n x R) satisfying the
following conditions:
1. A (x (. ), xO (- )) + (0('\, (x ( .) , x O ( . )) ), 0 0 (A, (x (. ), xO (. ) ) ) )
E S [0, T] ( - F, (0, 0) ), A > 0;
2. the map (x(.), x O (.)) ---+ (0('\, (x(.), x O (.))), 0°('\, (x(.), x°(-)))) is con-
tiI1UOUS in the norm of AC([O, T], R n x R);
3.
limA- 1 In ax I(O(A, (x(.),xo(.))),Oo(A, (x(.),xo(.))))IAC == O.
Alo (x(.),xO('))EX
Let AD > 0 be such that
max I (0('\0, (x (. ), xO (. ))), 0° (AD, (x ( .), x O ( .)))) lAC < bAo.
(x(.),xO('))EX
From the defillitio11 of F we have
AOX O (t) + 0° ('\0, (x (- ), xO (. ))) (t)
> d('\ox(t) + 0('\0, (x(.), x°(-)))(t), F(AoX(t) + O(Ao, (x(.), xO(.)))(t))),
for almost all t E [0, T] and for all (x(-), x O (.)) EX.

6.3. Controllability at first approxilnatiol1
151
Applying Theoren1 4.5 with
M == {AOXC) + O(AO, (x(.), xOC))) I (x(.), x o (.)) E X}
and ro(x) == x, we see that there exists a continuous map 01 M 
AC([O, T], R n ) such that
AOX(.) + O(AO, (xC), x o (.))) + 01(X(.)) E S[O,T] ( -F, 0)
and
101(X(.))IAC < e lT I T (>.oiP(t) + 0 0 (>'0, (x(.),xo(.)))(t))dt
= e1T(>'ob + 10 0 (>'0, (x(.),xo(')))IAC <  >'oa
for all (xC), x O (.)) E X.
Let x E . Then there exists a unique collection of numbers (J"i > 0 such
that L: (J"i == 1 and x == L: (J"iXi. Consider the continuous map f : AO  R n
given by
f(x) = f (>'0  (TiXi)
= >'0  (TiXi(T) + 0 ( >'0, (  (TiXi('),  (TiX?O) ) (T)
+01 (  (TiXiO) (T).
Let x E !  and x == L: +1 1 (J"iXi E bd. Then we have
If(>'ox) ->'oxi = 0 (>'0, (  (TiXi('),  (TiX?O)) (T)
+01 (  (TiXiO) (T) < >'oa < I>'ox - >'oxl.
By Corollary 2.1 we have AOX E ilnf. He11ce !AO c R[O,T] ( -F, 0). The
theorem is proved. D
Example. To compare the sufficient conditions given by Theorem 6.3 and
by Theoreln 6.4, consider the differential inclusion
(6.18) (xl, x 2 ) E F(x1, x 2 ),
where the set-valued Inap F : R 2  R 2 is given by
F(x 1 ,X 2 )== { [-1,1] X [_Xl, xl], X > O,
[-1,1] x {O}, x < O.

152
6. Controllability
It is easy to see that there is no strict controllable convex process approxi-
mating F at zero and therefore Theorem 6.3 cannot be applied. Nevertheless
we can establish local controllability using Theorem 6.4. Consider the con-
vex process A : R 2 ---7 R 2 given by
gr A == {( x l , X 2 , v I , V 2 ) I x l > 0, - X 1 < V 2 < x l }.
Since
(gr A) * == {( x * 1 , X * 2 , V * 1 , V * 2 ) I x * 1 > 0, x * 2 == 0, v * 1 == 0, x * 1 > I v * 21 } ,
we see that any solution (x*l, x*2) (t) to the differential inclusion
(x*1,x*2) E _A*(x*1,x*2)
satisfies
_X*l(t) > 0, x*2(t) == 0, x*l(t) == 0, -x*l(t) > Ix*2(t)l.
Therefore (x*l, x*2)(t) _ (0,0), and we derive local controllability of differ-
ential inclusion (6.18) from Theorem 6.4.
Thus Theorem 6.4 provides sufficient conditions for local controllability,
which are stronger than that provided by Theorem 6.3 even for very simple
systems. However it is not easy to check the conditions of Theorem 6.4,
while Theorem 6.3 combined with Theorem 6.2 gives sufficient conditions of
local controllability in a convenient algebraic form.
6.4. Controllability of some mechanical
systems
Here we show how the developed techniques work in examples that come
from mechanics.
Controlled point body. The motion of a point body subjected to a force
is described by the following control system:
xl == x 2
,
x 2 == U
,
(6.19)
U E [-1,1].
The control Uo == 0 corresponds to the zero equilibrium position. Obviously
the convex process A(x) == Cx + K, where
c=( )

6.5. Problems
153
and
K == {(W 1 ,W 2 ) I wI == O},
approximates (6.19) in a neighborhood of the origin. The transposed oper-
ator C* has neither nontrivial subs paces nor eigenvectors contained in the
conjugate cone K* == {( w 1 *, w 2 *) I w 2 * == O}. By Corollary 6.1 the process
A(x) is controllable. Applying Corollary 6.2 we see that system (6.19) is
controllable around zero.
Oscillator subjected to a unilateral force. The motion of an oscillator
subjected to a unilateral force is described by the following equations:
xl == x 2
,
x 2 == -xl + U,
(6.20)
U E [0, 1].
The control Uo == 0 corresponds to the equilibrium position. The convex
process A(x) == Cx + K, where
c=(_ )
and
K == {(w 1 ,w 2 ) I wI == 0, w 2 > O}
is a first approximation for system (6.20) near zero. The transposed ma-
trix C* has neither nontrivial subspaces nor eigenvectors contained in the
conjugate cone K* == {(w 1 *,w 2 *) I w 2 * > O}. From Corollary 6.1 and Corol-
lary 6.2 we conclude that system (6.20) is controllable around the origin.
6.5. Problems
1. Let C : R n  R n be a linear operator, and let KeRn be a closed
convex cone. Show that the following conditions are equivalent:
(a) The convex process A(x) == Cx + K is controllable.
(b) For some m > 1, K + C K + . . . + cm K == K - C K + . . . +
(-l)mcmK == Rn.
2. Show that if K is a subspace, then controllability of the process
A(x) == Cx + K is equivalent to the condition
K + CK +... + C n - 1 K == R n .
The subspace K can be represented as K == DR m , where D : R m 
R n is a linear operator. Prove the Kalman criterion of controllability:
the controllability of A( x) is equivalent to the condition
rank[D, CD, . .. , C n - 1 D] == n.

154
6. Controllability
3. Study the controllability of the system
x == u,
y == -y + u,
U E [-1, 1]
around the origin.
4. Do the saIne for the system
x == -x + U,
Y == -y + U,
U E [0, 1].
5. A set-valued map F : [to, t1]  R n with closed values is measur-
able if the function t  d(x, F(t)), t E [to, t1], is Ineasurable for all
x E R n . Let K ( t) c Rn be a closed convex cone depending on
t E [to,t1], and let x*(.) E L 1 ([to,t1],R n ). Assuine that the set-
valued map t  K(t) is measurable and it: 1 (x*(t), x(t))dt > 0, when-
ever x(-) E Loo([to, t1], R n ) and x(t) E K(t), t E [to, t1]' Prove that
x*(t) E K*(t), t E [to, t1]'
6. Let Ko c R n be a closed convex cone, and K(t) c R n x Rn be a
closed convex cone depending on t E [to, t1]' The set-valued Inap
t  K(t) is assumed to be measurable. Consider the set-valued Inap
A : [to,t1] X R n  R n defined by A(t,x) == {v E R n 1 (x, v) E
K(t)}. For each t E [to,t1] the Inap x  A(t,x) is a convex process.
Assume that there exists a function ')'(.) E L 1 ([to, t1], R) such that
A ( t, x) n ')' ( t) Bn i- 0 for all (t, x) E [to, t 1] X R n . Set
P == {p E R n 13 x(-) E S[to,tl](A,p) X(t1) E Ko}
and
Q == {q E R n 13 x*(-) E S[to,tl](-A*,q) X*(t1) E -K}
Show that P* == -Q. (Hint. Use the previous problein.)
7. Show that the condition A(t, x) n ')'(t)Bn i- 0 for all (t, x) E [to, t1] x
R n , where ')'(.) E L 1 ([to, t1], R), is essential. (Hint. Consider the
cones Ko == {O} c Rand K(t) == {(x, v) E R 2 1 -x < v(l - t)t},
t E [0,1].)
8. Let A : R n  R m be a linear operator, let F : Rn  Rn be a
Lipschitzian with a constant l > 0 set-valued map with closed con-
vex values, and let Co C R n be a closed set. Consider a solution
x(.) E S[O,T] (F, Co). Assume that there are given a closed convex
cone K(t) C T+((x(t), i(t)), grF) depending on t E [0, T] and a closed

6.5. Problems
155
convex cone Ko C T+(x(O), Co). The set-valued map t  K(t) is as-
sumed to be measurable. It is also assumed that Ko is a tent to
the set Co at x(O) E Co. Assume that Ax(T) tJ. intAR[o,T] (F, Co).
Prove that there exist a nonzero vector v* E R m and a function
p(.) E AC([O,T],R n ) such that (p(t),p(t)) E -K*(t), p(O) E -Ko,
andp(T) == -A*v*. (Hint. Consider the set-valued maps F(x,x O ) ==
{(v, vO) E R n x R I v O > d(v, F(x))}, F'(t, x, x O ) == {(v, vO) E R n x R I
v O > (1 + l)d((x, v), K(t))}, and the sets C == Co x {O} c R n x R
and K == Ko x {O} c R n x R. Following the proof of Theorem 4.14,
show that the cone R[O,T] (F' , K) is a tent to the set R[O,T] (p, C) at
the point (x(T),O). Following the proof of Theorem 6.4, show that
the cone L == cl{(Ax,x O ) E R n x R I (x,x O ) E R[O,T](F',K')} does
not coincide with the cone (Ax, x O ) E {R n x R I x O > O} and that
this implies the existence of a vector (v*, v O *) E L * with v* =I 0, and
apply the result of Problem 6.)

Chapter 7
Optimality
In this chapter we deal with extremal problelns for differential inclusions.
The existence of optimal solutions has already been discussed in Chapter 4.
Here we derive necessary conditions of optimality. First we study an op-
timization problem for a discrete-thne inclusion. We then obtain neces-
sary conditions of optimality for differential inclusions using the method
of discrete-time approximations. Especial emphasis is given to the thne-
optimal problem because of its practical importance.
7.1. Optimal solutions to discrete-time
incl us ions
In this section we study the optimality of solutions to discrete-time inclu-
sions. Since the discrete-time extremal problems are mathelnatical program-
ming problems of special type, the necessary conditions of optimality can
be easily obtained from general results of nonsmooth analysis.
Let F : R n  R n be a set-valued map with closed values, and let
Ak : R(k+1)n  R n , k == 0, N , be linear operators. The matrix of Ak has the
form (AO,k, A 1 ,k, . .. , Ak,k) , where Ai,k, i == 0, k , are n x n matrices. Consider
a discrete-time inclusion
(7.1)
Vk+1 E F(Ak(VO, VI,. .. , Vk)), k == 0, N - 1.
Let Co c R n and CN C R n be nonempty closed sets, and let <P : R n  R
and Wk : R n  R, k == 0, N , be functions. Our aim is to find a trajectory
-
157

158
7. Optimality
{Vk} f 0 of (7.1) satisfying the boundary conditions
(7.2) vo E Co, AN(VO,... ,VN) E CN
and such that
N N
<I>(AN(VO,'" , VN)) + L W[«(Vk) < <I>(AN(VO,'" , VN)) + L Wk(Vk)
k=O k=O
for any other trajectory {vk} f 0 of (7.1) satisfying (7.2).
Theorem 7.1. Assume that the set-valued map F : R n ---+ R n is Lips-
chitzian and that the functions  : Rn ---+ Rand Wk : R n ---+ R, k == 0, N , are
locally Lipschitzian. Let kerAiv == {O}. If the trajectory {vk} f 0 is optimal,
then there exist a number A > 0 and vectors xo, . .. , xiv, qo, . .. , qN, y*, z* E
R n such that
N
x;j E N(vo,C o ), y* E N(LAi,NVi,CN),
i=O
N
z* E 8<I>(LAi,NVi), qk E 8Wk(vd,
i=O
xk E F* ( )-: Ai,k-l Vi, Vk ) ( >..qk + >"Ak,NZ* + )-: Ak,iXi+1 + A k,NY* ) ,
=O =k
k == 1,N,
N-1
>..qO + >"A;j,NZ* + x;j + L A;j,i X i+1 + AO,NY* = 0,
i=O
N
>.. + ly*1 + L Ixil > o.
i=O
Proof. We shall denote by V a vector (Vo,... ,VN) E R(N+I)n. Consider
the set
F == {( Vo, VI, . . . , V N, WI, . . . , W N)
E R(2N+I)n I Vo E Co, Wk E F(Vk), k == 1, N }
and the linear operator A : R(N+I)n ---+ R(2N+I)n defined by
A V == A ( Vo, . . . , V N )
== (vo, Aovo, A 1 (vo, VI),... , AN-I(VO,... , VN-I), VI,... , VN).
Put W(V) == 'E r 0 Wk(Vk). Observe that V == (vo,. . . , VN) is a solution of
the following minimization problem:
(ANV) + w(V) ---+ inf,
AVEF, ANVECN.

7.1. Optimal solutions to discrete-time inclusions
159
By Theorem 3.17 there exist X* E N(V, A-I F), Xiv E N(V, A N 1 CN), and
A > 0 such that
(7.3) 0 E A8(<I>(A n V) + w(V)) + X* + Xiv
and A + IX*I + I Xiv I > O.
Let (xo,xi,... ,xN,Yi,... 'YN) E N(AV,F) n kerA*. By Theorems 3.5
and 3.7 we have
(7.4)
IXkl < llYkl, k == 1, N ,
where l > 0 is the Lipschitz constant of F. The matrix A* has the form
E Ao 0 Ao 1
, ,
o 0 Ai 1
,
A(; N-2
,
Ai N-2
,
A(; N-1
,
Ai N-1
,
o 0
E 0
o 0
o 0
o 0 0 0 A N - 1 ,N-1 0 0 E 0
000 0 0 00 OE
From (7.3) we see that YN == O. From (7.4) we have x N O. Hence
YN-1 == 0, etc. By induction we obtain Yk == xk == 0, k == 1, N, and Xo == o.
Thus we have N(AV,F) nkerA* == {O}. Applying Theorem 3.6, we obtain
X* E A*N(AV,F). Since kerA N == {O}, we have Xiv E ANN(ANV,CN).
From (7.3) and Theorems 3.5 and 3.15 we see that there exist vectors z* E
8<I>(A N V), (xo,xi,... ,xN,Yi,... 'YN) E N(AV,F),andy* E N(ANV,C N )
such that
qk E 8Wk(Vk), k ==. 0, N , x(; E N(vo, Co),
xk E F*(A k - 1 (vo,. . . , Vk-1), Vk)(-Yk), k == 1, N,
N-1
)..qo + )"A'Q,NZ* + x'Q + L A'Q,iXi+1 + A'Q,NY* = 0,
i=O
N-1
)..qk + )"A'k,N Z * + L A'Q,iXi+1 + Yk + A'k,NY* = 0, k = 1, N,
i=k
and
2
N-1 N N
).. + x'Q + L A'Q,iXi+1 + L I)"A'k,N Z * + Ak,Ny*1 2 + L I A 'k,NY*12 > O.
i=O k=l k=O
From the last inequality we derive
N
).. + ly*1 + L Ixil > o.
i=O
The theorem is proved.
D

160
7. Optimality
7.2. Optimal solutions to differential
incl us ions
Consider a differential inclusion
(7.5)
x ( t) E F ( x ( t ) ) , t E [0, T],
where F : R n  R n is a Lipschitzian set-valued map with closed convex
values contained in a ball of radius b > O. Let Co c R n and C 1 C R n be
closed sets, and let <p : R n  R be a Lipschitzian function. The problem is
to find a solution xC) to (7.5) satisfying the boundary conditions
(7.6)
x(O) E Co, x(T) E 01
and such that
<p(x(T)) < <p(x(T))
for any trajectory xC) of (7.5) satisfying (7.6).
The following theorem contains necessary conditions of optimality for
the problem under consideration.
Theorem 7.2. Let x(.) be an optimal trajectory. Then there exist A > 0
and an absolutely continuous function p(.) E AC([O, T], R n ) such that
1. p(O) E N(x(O), Co), p(T) E -Ao<p(x(T)) - N(x(T), C 1 );
2. p ( t) E co F* ( x ( t ) , i; ( t ) ) ( - p ( t ) ) ;
3. Ip(T)1 + A > O.
Proof. Let N be a positive integer. Consider the following auxiliary prob-
lem: minimize the function
W(XO,V1,... ,VN)
( N ) N k8 N
= <p XQ + 6N I:: Vk + (xQ - x(O))2 + I:: 1 (Vk - f(t))2dt,
k==l k==l (k-1)8 N

7.2. Optimal solutions to differential inclusions
161
where 6 N == T / N, over the variables (XO, VI, . . . , V N) satisfying the following
inclusions
Xo E CO,
VI E F(xo),
V2 E F(xo + 6 N V 1),
(7.7)
****** *********
VN E F (xo + 6 N - Vk) ,
N
Xo + 6 N L Vk E CN,T = C 1 + "7N B n.
k=l
From Theorem 4.15 we see that there exists a sequence TJN  0 as N  00
such that inclusions (7.7) are compatible for any N. Since
lim w(xo, VI,. .. , V n ) == 00,
I(XO,Vl,... ,vn)loo
by the Weierstrass theorem we conclude that the auxiliary problem has a
solution (XN,O, VN,l, . . . , VN,N). Applying Theorem 7.1, we see that there
exist AN > 0 and vectors xC;, . .. , xjy, y*, z* E R n such that
xC; E N(XN,O, Co), y* E N (XN'O + 6 N f VN,i, CN,T) ,
z* E 8<p ( XN'O + 6 N ) VN'i ) ,
=o
Xk E P* (XN,O+6 N }-: VN'i'VN'k) (6 N ANZ*
+6 N }.-: X;+l + 6 N y* + ANqN'k) ,
N
ANZ* + LX; + y* + 2AN(XN,O - x(O)) = 0,
i=O
k == 1, N,
N
AN + ly*1 + L Ix;1 > 0,
i=O
where
l k8N
qN,k == 2(VN,k - x(t))dt.
(k-1)8 N

162
7. Optinlality
Define the function x N ( .) : [0, T]  Rn as
k-l
XN(t) = XN,O + 8 N 2: Vi + (t - (k -1)8 N )Vk, t E [(k -1)8 N , k8 N [, k = 1, N .
i=l
Define the fU11ction q N ( .) : [0, T]  Rn as
qN(t) == qN,k/ 8 N, t E [(k - 1)8N, k8N[, k == 1, N .
Put PN,k == -y* - ANZ* - 'L, f k+1 x;, k == 0, N , and consider the piecewise
linear function PN(.) : [0, T]  Rn given by
PN(t) == PN,k-1 + 8 N 1 (PN,k - PN,k-1)(t - (k - 1)8 N ),
t E [(k - 1)8N, k8N[, k == 1, N .
Wit110ut loss of generality
(7.8) iT IPN(t)ldt + AN = 1.
Observe that PN (-) satisfies the following inclusions:
(7.9) PN(O) - 2AN(XN(0) - x(O)) E N(XN(O), Go),
(7.10) PN(T) E -AN8rp(XN(T)) - N(XN(T), GN,T),
(7.11) PN(t) E F*(XN(t) + 8NbB n ,!c N (t))(-PN(t) + ANqN(t) + 8NlBn).
Let us show that the sequence of functions {!eN (. ) } N 1 tends to i ( . )
in the norm of L2([0, T], R n ) and the sequence {XN,O} N 1 tends to x(O).
Suppose that there exist (J" > 0 and a cou11table subset of positive integers
N such that
(XN,O - £(0))2 + iT (£N(t) - i(t))2dt > a
for all N E N. Since the sequence {XN,O} N 1 is bounded, from t11e Arzela-
Ascoli theorem we see that the sequence {x N ( .) } N 1 contains a uniformly
convergent subsequence. Without loss of generality the sequence XN(.) con-
verges uniformly to a continuous function x(.). From Lemma 4.4 we conclude
that x(.) is a solution to differential inclusion (7.5). Moreover, x(T) E G 1 .
Note that
W(XN,O, VN,l,.. . , VN,N) > rp(xN(T)) + (J" > rp(x(T)) + (J" /2
whenever N E N is sufficiently large. Applyi11g Theorem 4.15 we see that
there exists a sequence {(ZN,O,. . . , ZN,N) } N 1 of solutions to t11e discrete-
time inclusions
ZN,k E ZN,k-1 + 8NF(ZN,k-l)' k == 1, N ,

7.2. Optimal solutions to differential inclusions
163
with the boundary conditions ZN,O == x(O), ZN,N E CN,T and such that the
functions W N ( .) : [0, T]  R n defined by
1 -
WN(t) == WN,k == 8 N (ZN,k - ZN,k-1), t E [(k - 1)8N, k8N[, k == 1, N,
converge to i(.) in the norm of L 2 ([0, T], R n ). For N E N sufficiently large
we have W(ZN,O, WN,l,' . . , WN,N) < <p(x(T)) + a /4. Since
W(ZN,O,WN,l,'" ,WN,N) > W(XN,O,VN,l,'" ,VN,N)
and <p(x(T)) > <p(x(T)), we have
<p(x(T)) + a /4 > W(ZN,O, WN,l,' .. , WN,N) > W(XN,O, VN,l, . . . , VN,N)
> <p(x(T)) + a/2 > <p(x(T)) + a/2
whenever N E N is sufficiently large, a contradiction. Thus, we have
lim ( (XN,O - x(O))2 + fT(!iN(t) - 5;(t))2dt ) == O.
Noo Jo
From the inequality
TNT
{ IqN(t)ldt = L IqN,kl < 2 ( liN(t) - 5;(t)ldt
Jo k=l Jo
we see that the functions qN(') converge to zero in the norm of L1([0, T], R n ).
Without loss of generality the functions liN(') - i(.) I and IqN(') I tend to zero
almost everywhere. From Theorem 3.7, the Gronwall inequality, differential
inclusion (7.11), and the Arzela-Ascoli theorem we see that the sequence of
functions PN(') contains a uniforlnly convergent subsequence. Without loss
of generality P N (.)  p(.) in the uniform norm. Applying Lemma 4.4, we
obtain
(7.12)
p(t) E coF*(x(t), £(t))( -p(t)).
Taking into account (7.8), we can assume that AN  A > O. Since XN(O) 
x(O) as N  00, we have
p(O) E N(x(O),C o )
thanks to Theorem 3.3.
Observe that CN,T == {x I d(x, C 1 ) < 'r/N}' Applying Corollary 3.3 and
Theorem 3.13 and taking the limit in (7.10), we obtain
p(T) E -Ao<p(x(T)) - N(x(T), C 1 ).
Taking the liInit in (7.8), we obtain
(7.13)
I T Ip(t)ldt + A = 1.

164
7. Optimality
From Theorem 3.7, inclusion (7.12), and the Gronwall inequality we have
Ip(t)1 < Ip(T)lel(T-t), t E [0, T],
where l > 0 is the Lipschitz constant of F. Consequently (7.13) is equivalent
with Ip(T)1 + ,\ = 1. The theorem is proved. 0
Remark. From Theorem 3.8 we see that an optimal trajectory satisfies the
following maximum principle:
(p ( t ), 5: ( t )) = S (p ( t ) , F ( x ( t ) ) )
for almost all t E [0, T].
Mayer optimal control problem. Consider the Mayer optimal control
pro blem
cp(x(T))  inf,
x ( t) = f ( x ( t ) , u ( t ) ) , t E [0, T],
u(t) E U,
x(O) E Co, x(T) E C 1 ,
where the set U C R k is compact and the sets Co c R n and C 1 C R n
are closed. Necessary conditions of optimality for this problem have the
following form.
Theorem 7.3. Let (u(.), x(.)) be an optimal process. Assume that f : Rn x
R m  R n is differentiable in x and the derivative \7 xf(x, u) is continuous in
(x, u), the set f(x, U) is convex for all x E R n , and for almost all t E [0, T]
there exists a unique point u(t) E U satisfying 5:(t) = f(x(t), u(t)). Then
there exist a number ,\ > 0 and an absolutely continuous function p(.) E
AC([O, T], R n ) satisfying the following conditions:
1. p(O) E N(x(O), Co), p(T) E -'\8cp(x(T)) - N(x(T), C1);
2. p(t) = -(\7 xf(x(t), u(t)))*p(t);
3. maxuE U (p ( t ) , f ( x ( t ) , u)) = (p ( t ) , f ( x ( t ) , u ( t ) ) ) ;
4. Ip(T)1 + ,\ > O.
Proof. By definition x* E F*(x,v)(v*) is equivalent with (x*, -v*) E
N((x, v), grF). Invoking Theorem 3.4 we obtain
N((x,v),grF) = limsup N_((x',v'),grF)
(x' ,v')---+(x,v)
(x' ,v')EgrF
= limsup -(T_((x',v'),grF))* c limsup -(T+((x', v'), grF))*.
(x',v')---+(x,v) (x',v')---+(x,v)
(x'v')EgrF (x'v')EgrF

7.3. Time-optimal problem
165
Applying Proposition 2.8, we obtain
N((x,v),grF) c limsup {(y,w) I w E \lxf(XI,U')Y
(x' ,v')---+(x,v)
+cone(f(x ' , U) - f(x ' , u ' )), v' == f(x ' , u ' )}*.
The conjugate cone is given by
{ (y, w) I w E \l x f ( x' , u ' ) y + cone (f ( x', U) - f ( x' , u ' ) ), v' == f ( x', u ' ) } *
== {(y*, w*) I y* == -(\lxf(x ' , u'))*w*, w* E (cone(f(x ' , U) - f(x ' , u ' )))*}
(see Exalnple in Section 2.6). Since the point u(t) E U such that i:(t) ==
f(x(t), u(t)) is unique, applying Proposition 1.4, we obtain the result. D
7.3. Time-optimal problem
Consider the time-optimal problem
T  inf,
x ( t) E F ( x ( t ) ) , t E [0, T],
x(O) E Co, x(T) E C 1 , T E [0, T*],
where F : R n  Rn is a Lipschitzian set-valued map with convex closed
values contained in the ball of radius b > 0, and Ck, k == 1,2, are closed
sets. Let T > 0 be an optimal time and let x(.) be an optimal trajectory.
The time-optilnal problem is equivalent to the following Mayer problem:
t(T)  inf,
d
dT (Y(T), t(T)) E F(Y(T), t(T))
== {a(v, 1) E R n x R I a E [1/2,3/2], v E F(Y(7))}, 7 E [0, T],
(y(O), t(O)) E Co x {O}, (y(T), t(T)) E C 1 x R.
Indeed, since dt(7)/d7 E [1/2,3/2], any trajectory (Y(7), t(7)) of the Mayer
problem is associated with one and only one trajectory x( t) == y( 7( 7- 1 (t))) of
the time-optimal problem. The trajectory (y( 7), i( 7)) == (x( 7), 7), 7 E [0, T],
is, obviously, an optimal solution to the Mayer problem. By Theorem 7.2
there exist A > 0 and a function (Py,Pt)(.) E AC([O, T], R n x R) such that
1. (Py , Pt) (0) E N ( (y ( 0 ), i ( 0 ) ) , Co x {O}),
(Py,Pt)(T) E -A8(y,t)t(T) - N((y(T), i(T)), C 1 x R),
2. (Py, Pt) ( 7) E co F* (y ( 7 ), i ( 7 ), y ( 7 ), i ( 7 ) ) ( - (Py , Pt) ( 7 ) ) , 7 E [0, T],
3. I (Py, Pt) (T) I + A > O.
Using Theorem 3.5 and Problem 3.4, we obtain

166
7. Optimality
1. (Py, Pt) (0) E N (x (0), Co) x R,
(py,pt)(T) E -A(O, 1) - N(x(T), C 1 ) x {O},
2. Py(T) E co F*(X(T), X(T))( -Py(T)),
Pt ( T) == 0, (Py ( T ), i; ( T )) + Pt ( T) == 0, T E [0, T],
3. I(Py,Pt)(T)1 + A > O.
Putting P == Py, we get
1. p(O) E N(x(O), Co), p(T) E -N(x(T), C 1 ),
2. P E co F* ( x ( T ) , i; ( T ) ) ( - P ) ,
o < A == -Pt ( T) == (p ( T ), i; ( T ) ) , T E [0, T],
3. l(py,pt)(T)1 + A > O.
By TheorelTI 3.8 (p(T), i;(T)) == S(p(T), F(X(T))). Show that p(T) =I- O.
Indeed, if p(T) == 0, then passing to the limit in the equality
A == - Pt ( T) == (p ( T ) , i; ( T )) == S (p ( T ) , F ( x ( T ) ) )
as TiT, we obtain A == pt(T) == S(O,F(x(T))) == 0, a contradiction. Thus
we have proved the following result.
Theorem 7.4. Let T be an optimal time and let x ( .) be an optimal trajec-
tory. Then there exists an absolutely continuous function p(.) such that
1. p(O) E N(x(O), Co), p(T) E -N(x(T), C 1 ),
2. P E coF*(x(t), i;(t))( -p), (p(t), i;(t)) == S(p(t), F(x(t))) (const) >
0, t E [0, T],
3. p(T) =I- O.
Example. Consider one example. The rectilinear motion of a hydroplane
can be described by the control system
(7.14)
x == -F(x) + u, u E [-1,1],
where u is the applied force and F(v) is the resistance. The resistance F(v),
v > 0, can be modeled as a piecewise linear function
{ av, v E [0, VI],
F( v) == -{3v + (a + (3)V1, V E [VI, V2],
av - (a + (3)(V2 - VI), V E [V2,00[,
where a and (3 are positive constants satisfying V2 max{ a, {3} < 1. We put
F( v) == - F( -v) if V < O. Introducing the velocity V == X, we can write down
the control system in the form
x == V,
iJ == -F(v) + u, u E [-1,1].

7.3. Time-optimal problelTI
167
Consider the problem consisting of choosing u(.) so as to Ininimize the time
needed to take the hydroplane froin a given initial position (xo, vo) to the
position (0,0). This time-optilnal problem can be formalized as follows:
T  inf,
(x, v) E (v, -F(v) + [-1,1]),
( x, v ) ( 0) == (x 0, vo), ( x, v ) (T) == (0, 0).
Let (x, v)(.) be an optimal trajectory. By Theorein 7.4 there exists a nonzero
absolutely continuous function (p, q)(t) E R x R satisfying
(7.15) (p, q) ( t) E (0, - p ( t) + f ( v ( t) ) q ( t ) ) , t E [0, T],
where
{ a, v E] - 00, -V2 [U] - VI, VI [U]V2, +00[,
f ( v) == - (3, v E] - V2, - vI [U] VI, v2 [,
[ - (3 , a], v E {:::i::v1, :::i::v2},
and
( 7 .16) q ( t ) u ( t) == Inax ( q ( t ) u) == I q ( t ) I , t E [0, T] .
UE[-l,l]
From (7.15) we obtail1 p(t) = P == (const). If v(t) tJ. {:::i::v1, :::i::v2}, t E [to, t1] C
[0, T], then f(v(t)) E {a, -(3} and the function q(t), t E [to, t1], satisfies the
differential equation q(t) == -p + fq(t), t E [to, t1]' Hence
q( t) = eftq(to) +  (1 - eft), t E [to, tl]'
Froin this we see that q(t), t E [to,t1], is monotone. Therefore (7.16) ilnplies
that the optimal control in this interval is either
u(t) == { -I, t E [to, 7[, or u(t) = { I, t E [to, T[,
1, t E]T, t1], -1, t E]T, t1],
where T E [to, t1] is the point satisfying q( T) == O. (Such a point Inay
I10t exist.) Show that the optilnal trajectory cannot move along the lines
v == :::i::v1 and v == :::i::v2. Indeed, if v(t) E {:::i::v1,:::i::v2}, then 0 == (t) ==
-F(v(t)) + u(t), t E [to, t1]' Since Vk In ax { a, (3} < 1, k == 1,2, we obtain
u =1= :::i::1. Therefore q(t) - 0, t E [to, t1]' On the other hand by (7.15) we
have q(t) E -p + f(v(t))q(t), t E [to, t1]' This implies (p, q(t)) - (0,0),
t E [to,t1]' Hence (p,q(t)) = (0,0), t E [O,T], a contradiction. Thus we see
that the function q(t), t E [0, T], is Inonotone and there exists at Inost one
point T E [0, T] such that q( T) == O.
It is easy to verify that there is only one solution (x+, v+)(t), t E] -00,0],
to the differential equation
x == v,
v == -F(v) + 1,

168
7. Optimality
satisfying (x+, v+)(O) == (0,0), and there is only one solution (x-, v_)(t),
t E] - 00,0], to the differential equation
x == v,
v == -F(v) - 1,
satisfying (x-, v-)(O) == (0,0). The functions X:f:(t) and V:f:(t) are monotone
and satisfy the inequalities x+(t) > 0, v+(t) < 0, x_(t) < 0, and v-(t) > 0,
whenever t E] - 00,0]. Moreover limt_oo x+(t) == +00, limt_oo v+(t) ==
-00, limt_oo x-(t) == -00, and limt_oo v_(t) == +00. Thus we see that
the optimal control has the following form. If the initial point (xo, vo) is
situated above the curve C == {(x+, v+)(t) It E] - 00, OJ} U {(x-, v_)(t) It E
] - 00, OJ}, then
u(t) == { -I, t E [0, [,
1 , t E [ T, T],
where T satisfies (£, V)(T) E C and (£, v)(t) tt C, t E [0, T[. If the initial
point (xo, vo) is situated below the curve C == {(x+, v+)(t) It E] - 00, OJ} U
{(x_, v-)(t) It E] - 00, OJ}, then
u(t) == { I, t E [0, [,
-1 , t E [T, T],
where T satisfies (£, V)(T) E C and (£, v)(t) tt c, t E [0, T[.
7.4. Problems
1. Solve the brac histoch rone problem
l b v I +x2
a -IX dt ---t inf x(a) = A, x(b) = B.
(Hint. Use Problem 4.4.)
2. Solve the surfa ce of re volution of minimum area problem
l b x V ! + j; 2 dt ---t inf x(a) = A, x(b) = B.
(Hint. Use Problem 4.4.)
3. Study the time-optimal problem for system (7.14) without the as-
sumption V2 max{ a,,B} < 1.
4. Solve the following time-optimal problem:
T  inf,
x == - max{O, x} + 9 + u, u E U,
(x, x)(O) == (xo, vo), (x, x)(T) == (9,0),

7.4. ProblelTIs
169
where 9 > 0 is a constant and
(a) U  [-1,1],
(b) U  [0,1],
(c) U  [-1,0].
(Remark. The equation describes the vertical motion of a point l11ass
attached to an elastic band. When the band is stretched a positive
amount x, it applies, according to the Hook law, a restoring force
proportional to x. Whe11 unstretched, the force is equal to zero. The
constant 9 is the gravity acceleration. The control u is a force we
have at our disposal.)
5. Let F : R n  R n be a Lipschitzian with a constant l > 0 set-valued
map with closed convex values, let Co c R n and C1 C R n be closed
sets, and let <p : R n  R be a Lipschitzian function. Consider the
Mayer problem
<p(x(T))  inf,
x ( t) E F ( x ( t ) ) , t E [0, T],
x(O) E Co, x(T) E C 1 .
Let xC) be a solution to this problem. Assume that there are given
a closed convex cone
K ( t) C T + ( ( x ( t) , £ ( t) ), gr F)
depending on t E [0, T] and closed convex cones
Ko c T+(x(O), Co) and K 1 c T+(x(T), C 1 ).
The set-valued map t  K(t) is assumed to be measurable. It is also
assumed that Ko and K 1 are tents to the sets Co and C1 at x(O) E Co
and x(T) E C 1 , respectively. Let <p+ : R n  R be a convex positively
homogeneous function satisfying
epi<p+ c T + (( x(T), <p( x(T))), epi<p).
Prove that there exist ,\ > 0 and a function pC) E AC([O, T], R n )
such that
( a) (p ( t) , P ( t )) E - K * ( t ), t E [0, T],
(b) p(O) E -KG and p(T) E -8<p+(0) + Ki,
(c) ,\ + Ip(T)1 > O.
(Hint. Use Problem 3.3. Apply the result of Problem 6.8 to the
differential inclusion (Y1, Y2, Y3, Y4) E (F(Y1), 0, 0, 0), the set of initial
points Co x C1 x epi<p, linear operator A : R n x R n x R n x R 
Rn x R n x R defined by A(Y1, Y2, Y3, Y4)  (Y1 - Y2, Y1 - Y3, Y4), and
the trajectory (Y1, Y2, Y3, Y4)  (x(t), x(T), x(T), <p(x(T))).)

170
7. Optimality
6. Consider the IVlayer problenl
-x2(1)  i1lf,
( Xl, X 2) E co { ( -1, 0), (1, 0), (0, -I)} + (0, I xII) , t E [0, 1],
(Xl, X2)(0) == (0,0),
and study the optimality of the trajectory (Xl, X2)(t) = (0,0) using
Theorelll 7.2 and using the previous problem. Compare the results.
7. Let F : R n  R n be a Lipschitzian with a constant l > 0 set-valued
map with closed convex values and let Co C R n and C 1 C R n be
closed sets. Consider the tillle-optimal problem
T  inf,
x ( t) E F ( X ( t ) ) , t E [0, T],
x(O) E Co, x(T) E C 1 .
Let x(.) be a solution to this problem, and let T be the optimal
tillle. Assume that there are given a closed convex cone K(t) C
T+((x(t),i(t)),grF) depellding on t E [O,T] and closed convex cones
Ko C T+(x(O), Co) and K 1 C T+(x(T), C 1 ). The set-valued map
t  K(t) is assullled to be measurable. It is also assumed that Ko
and K 1 are tents to the sets Co and C 1 at x(O) E Co and x(T) E C 1 ,
respectively. Prove that there exists a function p(.) E AC([O, T], R n )
SUCll that
(a) (p( t), p( t)) E - K* (t), (p( t), i( t)) = (const) > 0, t E [0, T],
(b) p(O) E -K5 alld p(T) E Ki,
(c) Ip(T)1 > o.
(Hint. Use Problelll 5.)
8. Let F : R n  R n be a set-valued map witll closed convex graph. Let
Co C R n and C 1 C R n be closed convex sets, and let cp : R n  R
be a convex function. Suppose that a solution x(.) to the differential
inclusion x(t) E F(x(t)), t E [0, T], satisfies x(O) E Co and x(T) E
C 1 , and that there exists an absolutely continuous function p(.) E
AC([O, T], R n ) such that
(a) p(O) E N(x(O), o), p(T) E -ocp(x(T)) - N(x(T), C 1 );
(b) p( t) E F* (x( t), x( t)) ( -p( t)).
Prove that x(.) is a solution of the problelll
cp(x(T))  inf,
x ( t) E F ( X ( t ) ) , t E [0, T],
x(O) E Co, x(T) E C 1 .
(Hint. Use Tlleorem 3.2 and Problelll 3.1.)

Chapter 8
Stability
In this chapter we consider several concepts of stability for differential in-
clusions. We present two main methods for investigating stability: t11e Lya-
punov direct method and the method based on the study of a first approxi-
mation. We consider first approximations of two types: linear-selectionable
differential inclusions and differential inclusions with a convex process in the
right-hand side. The main part of this chapter is devoted to stability prop-
erties of these inclusions. In both cases we obtain simple algebraic criteria
of stability.
8.1. Lyapunov direct method
Consider a differential inclusion
(8.1)
x(t) E F(x(t)),
where F : R n ---+ R n is a set-valued map satisfying 0 E F(O); that is, the
origin is an equilibrium position of differential inclusion (8.1).
The concepts of stability. The equilibrium position x == 0 of differential
inclusion (8.1) is said to be stable (weakly stable) if, given any E > 0, there
exists fJ > 0 such that for any Xo E fJB n , each (at least one) solution x(.) of
(8.1) with x(O) == Xo satisfies Ix(t)1 < E for all t > O. Note that this requires
solutions starting near the origin to exist for all t > O. The equilibriulTI
position x == 0 is said to be asymptotically stable (weakly asymptotically
stable) if, in addition to being stable (weakly stable),
(8.2)
lim x ( t) == o.
t ---+ 00
-
171

172
8. Stabili ty
Whel1 the equilibriu111 position x == 0 is not weakly stable, we say that
it is unstable.
T11ese concepts of stability have Inotivations cOIning from technical prob-
lellls. The stability has the following heuristic n1eaning. Think of the origin
as a desired steady-state position of a differel1tial equation
x(t) == f(x(t))
vvhich serves as a lnodel for our system. Because of unpredictable perturba-
tions the dynalnics should be described by a differential inclusion
x(t) E F(x(t)).
Stability guarantees that every close-to-zero-state value taken by the system
in its future evolution is not too far fron1 the desired one. Asymptotic
stability ilnplies that the alnplitude of initial perturbations decreases and
eventually vanishes.
Weak stability and weak aSYlnptotic stability concepts are lnotivated by
another probleln. We have a control systeln
x == f ( x, u) , u E U,
and f(O, uo) == 0 for some Uo E U. Our goal is to answer the question: is it
possible to keep the system in the vicinity of the zero equilibrium position or
to drive it to t11e equilibriun1 positiol1 fron1 any initial position in the vicinity
of the origin? This problen1 is close to the controllability probleln. For this
reaSOl1 weak aSYlnptotic stability is often called asymptotic controllability.
To c11aracterize an aSYlnptotic behavior of a given solution to (8.1), so-
called Lyapunov exponel1t is used. Let f : R  R be a continuous function.
The Lyapunov exponent of f is defined by
x[f(.)] = -lim sup! In If( t) I.
too t
It is easy to check that the Lyapunov exponent possesses the following prop-
erties:
1. x[(f + cp)(-)] > lnin{x[f(-)]' X[cp(-)]},
2. x[(fcp)(.)] > x[f(.)] + X[cp(.)],
3. X [ (f cp ) ( . )] == X [f (- )], where 0 < a < cp ( t) < b < 00.
If f : R  R n is a vector fUl1ction, then the Lyapunov exponent is defined
as the n1inilnal value of the Lyapunov exponents of the components:
X [f (- )] == lnin {X [f 1 (- )], . .. , X [fn (- ) ] } .

8.1. Lyapunov direct Inetl10d
173
Consider a continuous fUl1ction V : R n  R such that V(O) == o. "\Ve
say that V is positive (negative) definite if V (x) > 0 (V (x) < 0) for all
x :I O. The function V is said to be positive (negative) semi-definite if
V(x) > 0 (V(x) < 0) for all x :I o.
A very powerful tool in stability al1alysis is provided by tI1e Lyapunov
direct method. The idea of the lnethod is to fil1d a Lyapunov function, that
is, a function defined in the vicinity of the origin such tI1at its upper or lower
Dini derivative with respect to (8.1) is definite or selni-definite. Then it is
possible to answer the question whether the equilibriul11 positiol1 is stable
or not.
Stability and asymptotic stability. We sI1all aSSUlne tI1at tI1e set-valued
map F : R n  R n with compact convex values is boul1ded and upper sen1i-
C011tinuous. Recall that the upper Dini derivative of a function f : R n 
R U { +oo} is defined by
D + f( )( ) 1 . f(x + av') - f(x)
x v == llnsup .
alO,v'---+v a
We sI1all use tI1e following auxiliary result.
Lemma 8.1. Let w : [0, T]  Rand h : [0, T]  R be continuous functions.
Assume that D+w(t)(l) < h(t), t E [0, T[. Then
w(T) - w(O) < I T h(t)dt.
Proof. Set to == O. Let E > O. Since D+w(to)(l) < h(t), there exists 8 0 > 0
such that
w(to + 8 0 ) < w(to) + 8 0 h(to) + 8 0 E.
By induction we define ti+ 1 == ti + 8 i , i == 0, 1, . . .. For al1Y i tI1ere exists
8 i > 0 such that
(8.3) w( ti + 8 i ) < w( ti) + 8 i h( ti) + 8 i E.
Set t == SUp{ti Ii == 0,1,...}. Let us show that t == T. Suppose that £ < T.
This implies that for any 8 > 0 we have
W(ti + 8) > W(ti) + 8h(ti) + 8E
whel1ever i is sufficiently large. Taking the lilnit we obtail1
w(£ + 8) > w(£) + 8h(£) + 8E
for all 8 > O. Hence D+w(t)(l) > h(i), a contradiction. Thus £ == T. Froln
(8.3) we obtain
00
w(T) - w(O) < L Oih(ti) + TE.
i=l

174
8. Stability
If 8 i > 0 are sufficiently slnall, then we have
00 r T
'L Oih(ti) < io h(t)dt + E.
i=l 0
Hence
w(T) - w(O) < 1 T h(t)dt + (T + 1)E.
Since f. > 0 is arbitrary, we obtain the result.
o
Theorem 8.1. Assume that there exist a number'TJ > 0, a positive definite
function V : R n  R, and a negative semi-definite function W : R n  R
such that
D+V(x)(v) < W(x)
for all v E F(x), Ix I < 'TJ. Then the equilibrium position x == 0 of differential
inclusion (8.1) is stable.
Proof. Let f. E]O, 'TJ[. Consider the set n == {x Ilxl == f.}. Denote by w the
minimum of V in O. Since V is positive definite, we have w > O. Let a > 0
be such that V(x) < w for alllxl < a. Consider a solution xC) of (8.1) with
Ix(O)1 < a. The solution certainly exists in some interval [0, T[.
Let us estimate the derivative
D+V(x(t))(1) = limsup V(x(t + T(1 + 0))) - V(x(t)) .
T !O,B---+O T
Let Ti 1 0 and ()i  0 be sequences such that
D+V(x(t)) (1)
1 . V ( x ( t) + Ti T i - 1 ( X (t + Ti (1 + () i)) - x ( t ) )) - V ( x ( t ) )
== 1m sup .
i---+oo Ti
Take f. > O. Since F is upper semi-continuous, froln Lemma 4.3 we get
r t + Ti (l+B i )
Ti-1(X(t + Ti(1 + Oi)) - x(t)) = T i - 1 it x(s)ds C F(x(t)) + EBn
whenever i is big enough. Without loss of generality the sequence T i - 1 (X(t+
Ti (1 + ()i)) - x( t)) converges to a vector v. The set F(x( t)) is closed and the
number f. > 0 is arbitrary. Therefore v E F(x(t)). From this we get
D + V(x(t))(l) < D+V(x(t))(v) < W(x(t)).
By Lemma 8.1 we have
V(x(t)) - V(x(O)) < 1 t W(x(s))ds < 0
for all t E [0, T[. Hence
V(x(t)) < V(x(O)) < w.

8.1. Lyapullov direct Inetl10d
175
Froin this we see that Ix(t)1 < E for all t E [0, T[. This in1plies that xC)
exists for all t > 0 and that the zero equilibriuln position is stable. D
Theorem 8.2. Assume that there exist a number 1] > 0, a positive definite
function V : R n  R, and a negative definite function W : R n x R  R
such that
D+V(x)(v) < W(x)
for all v E F(x), Ixl < 1]. Then the equilibrium position x == 0 of differential
inclusion (8.1) is asymptotically stable.
Proof. By Theorem 8.1 we know that there exists some a > 0 such that any
solution x(.) of (8.1) with Ix(O) I < a exists in [O,oo[ and Ix( t) I < 1] for all
t > O. Let us show t11at x(t)  0 as t  00. Suppose that V(x(t)) > l > 0
for all t > O. There exists 6 > 0 such that V(x) < l for all Ixl < 6.
We conclude that the solution x(.) satisfies Ix(t)1 > 6 for all t > O. Set
j.L == - Inax{W(x) I 6 < Ixl < 1]}. Then we have W(x(t)) < -j.L for all t > O.
Arguing as in the proof of Theoreln 8.1 and applying Lemlna 8.1, we get
V(x(t)) < V(x(O)) - j.Lt.
This implies V (x( t)) < 0 for all t large enough, a contradiction. We see that
the function V(x(t)) takes positive values as small as we want for large t.
Since V(x(t)) is nonincreasing, we have V(x(t))  0 as t  00. Thus we
have also Ix(t)1  0 as t  00. D
Theorem 8.3. Let V : Rn  R be a continuous function with V(O) == 0,
and let W : R n  R be a negative definite function such that
D+V(x)(v) < W(x)
for all v E F(x), Ix I < 1], where 1] > o. Assume that for each 6 > 0 there
exists at least one x with Ixl < 6 such that V(x) < O. Then the equilibrium
position x == 0 of differential inclusion (8.1) is unstable.
Proof. Let xC) be a solution of (8.1) with Ix(O)1 < 1] and V(x(O)) < O.
There exists 6 > 0 such that V(x) > V(x(O)) for all Ix I < 6. Arguing as in
the proof of Theorem 8.1 and applying Lemma 8.1, we get
V(x(t)) < V(x(O)) + it W(x(s))ds < V(x(O)).
Hence we cannot have Ix(t)1 < 6. Set j.L == -lnax{W(x) I 6 < Ixl < 1]}.
Then we have
V(x(t)) < V(x(O)) - tj.L
for all t > O. Since the last expression approaches -00 as t  00, we see
that there exists T > 0 such that Ix(T) I == 1]. Thus no matter how slnall
an a > 0 we take, the solution xC) of (8.1) with Ix(O) I < a will satisfy

176
8. Stabili ty
the relation Ix(T)1 == "7 at some T > O. We conclude that the equilibriulTI
position x == 0 of (8.1) is unstable. 0
Weak stability and weak asymptotic stability.
Theorem 8.4. Assume that there exist a number "7 > 0, a positive definite
function V : R n  R, and a negative semi-definite (definite) function W :
R n  R such that for any x E 'r/Bn there is a vector v E F(x) satisfying
D-V(x)(v) < W(x).
Then the equilibrium position x == 0 of differential inclusion (8.1) is weakly
stable (asymptotically stable).
Proof. Let E E]O,'r/[. Consider the set !l == {x Ilxl == E}. Denote by w
the minimum of V in!l. Since V is positive definite, we have w > o. Put
C == {x E EBn I V(x) < w}. Let a > 0 be such that V(x) < w for all
Ixl < a. Applying Theorem 5.5, we see that for any Xo E aBn there exists
a solution x(.) E S[o,oo[(F, xo) satisfying
V(x(t)) - V(x(O)) < I t W(x(s))ds.
Arguing as in the proofs of Theorems 8.1 and 8.2, we obtain the result. 0
8.2. Linear-selectionable differential
incl us ions
In this section we consider a differential inclusion
(8.4)
x(t) E A(x(t)),
where the set-valued map A : R n  R n has the form
A(x) == {v I v == Cx, C E C}.
Here C is a convex compact set in the space of real (n x n)- matrices C == II Cij II.
Such set-valued maps A are called linear selectionable, since the right-hand
side is a union of linear maps. Obviously A(O) == {O}. We say that a linear
selectionable set-valued map is asymptotically stable if the zero equilibrium
position of differential inclusion (8.4) is asymptotically stable.
Linear systems. We first consider the simplest linear-selectionable inclu-
sion, namely, a linear system
(8.5)
x == Cx
,
where C is an (n x n)-matrix. The roots AI,.. . , An of the characteristic
polynomial det(C - AE) == 0 of C are referred to as the spectrum of C.

8.2. Linear-selectionable differential inclusions
177
Since any solution to (8.5) can be represented as a linear cOlTIbination of the
functions eAitpi(t), i == 1, n , where pi(t) are polYl10mials with vector coeffi-
cients, we see that the equilibrium position x == 0 of (8.5) is asymptotically
stable if and only if all eigenvalues of C have negative real part.
Let us show that frolTI asymptotic stability of the zero equilibriulTI po-
sition (8.5) follows the existence of a quadratic Lyapunov function satisfy-
ing the conditions of Theorem 8.2. Later we establish the existence of a
Lyapunov function for a general asymptotically stable linear-selectionable
differential inclusion, but the proof will not be constructive as in the case of
a linear system.
First consider the following problem. Let W(x) be a quadratic form.
Our aim is to find a quadratic form V(x) satisfying
(8.6)
(VV(x), Cx) == AV(X) + W(x)
for all x E R n . We associate symlTIetric matrices V == Ilvi,j II and W ==
Ilwi,jll with the forms V(x) and W(x), that is, V(x) == (x, Vx) and lV(x) ==
(x, Wx). Then (8.6) cal1 be written as
(8.7)
V C + C* V == A V + W.
Equation (8.7) is a linear equation with n(n + 1)/2 unknowns Vi,j, i
1, n , j == 1, n , i > j. This equation can be solved for any symmetric matrix
W if and only if the linear system
(8.8)
V C + C* V == A V
has a nonzero determinant, that is, when A is not a root of the characteristic
equation for tl1e linear system
(8.9)
V C + C* V == o.
The following result allows us to calculate all roots of the characteristic
equation for systen1 (8.9).
Theorem 8.5. All roots of the characteristic equation for system (8.9) are
given by the formula
(8.10)
A == m1A1 + . . . + mnAn,
where Ai, i == 1, n , are eigenvalues of C and mi, i == 1, n, are nonnegative
integers satisfying
(8.11)
m1 + . . . + m n == 2.
Proof. Let Ci, i == 1, n , be the eigenvectors of C* corresponding to eigen-
values Ai, i == 1, n . Consider the quadratic form
V (x) == (C1, x) ml . . . (c n , x) m n ,

178
8. Stability
where mi, i == 1, n , are nonnegative integers satisfying (8.11). We have
(\7V(x),Cx) == m1(c1,X)m 1 -1(C2,X)m 2 ... (c n ,X)m n (C1,CX)
+m2 (C1, x)m 1 (C2, x)m 2 -1 . . . (c n , x)m n (C2, Cx) + . . .
+mn (C1, X)m 1 (C2, X)m 2 . . . (C n , x)m n -1 (C n , Cx)
== (m1 A 1 + . . . + mnAn)V(X).
Thus, the matrix V corresponding to the form V(x) is a nontrivial solution
of (8.8) with A satisfying (8.10). That is, any number A == m1 A 1 +. . .+mnAn,
where mi, i == 1, n , are nonnegative integers satisfying (8.11), is a root of
the characteristic equation for system (8.9).
Now it remains to prove that any root can be represented in the form
(8.10) with nonnegative mi satisfying (8.11). If all numbers (8.10) with non-
negative mi satisfying (8.11) are different, then we have n(n+ 1)/2 different
roots of the characteristic equation for system (8.9) and hence the result. In
the general case the result can be obtained taking the limit. This completes
the proof. 0
Theorem 8.6. Assume that all eigenvalues Ai, i == 1, n , of C have negative
real part. Then there exists a positive definite quadratic form V (x) satisfying
(8.12)
(\7V(x), Cx) == A V(x) - (x, x),
for all x E R n , where
(8.13)
a > A > 2max{ReAi Ii == 1,n }.
Proof. Since all eigenvalues Ai, i == 1, n , of the matrix C have negative real
part, Theorem 8.5 implies that A satisfying (8.13) exists and cannot be a
root of the characteristic polynomial for system (8.9). Consequently, there
exists a unique quadratic form V(x) satisfying (8.12).
Let us show that the forin is positive definite. Let V be a symmetric
matrix satisfying V(x) == (x, Vx). Then (8.12) is equivalent to
V C + C* V == A V - E
or
v ( C -  E) + ( C* -  E) V = - E.
Consider the linear differential equation
(8.14)
(8.15)
x = (C -  E) x.

8.2. Linear-selectionable differential inclusions
179
All eigenvalues of the matrix C - E have negative real part. From (8.14)
we see that for any trajectory of (8.15)
d
(8.16) dt V(x(t)) = -(x(t), x(t)).
If the form V(x) takes negative values, then by Theorem 8.3 the equilibriulTI
position x == 0 of (8.15) is not stable, a contradiction. If V(x) is positive
semi-definite and there exists a point x such that V(x) == 0, then from (8.16)
we have (VV(x), (C - E)x) < O. Hence there exists a point z such that
V(z) < 0, a contradiction. Thus V(x) is a positive definite quadratic form.
D
Corollary 8.1. If all eigenvalues of C have negative real part, then there
exists a positive definite quadratic form V ( x) such that
(VV(x), Cx) == -(x, x)
for all x ERn.
Corollary 8.2. If all eigenvalues of C have negative real part, then there
exists a positive definite quadratic form V ( x) such that
(VV(x), Cx) < AV(X),
where 0 > A > 2max{ReAi Ii == 1,n }.
Asymptotic stability of linear-selectionable differential inclusions.
Let A : R n  R n be a linear selection able set-valued map. By IAI denote
max{ICII C E C}.
Lemma 8.2. If A : R n  R n is an asymptotically stable linear selectionable
set-valued map, then there exist constants a > 0 and I > 0 such that
Ix( t) I < alx(O) Ie-It, t > 0,
for any solution x(.) of differential inclusion (8.4).
Proof. Show that there are 8 > 0 and T > 0 such that for all solutions
xC) of (8.4) with Ix(O)1 < 8 we have Ix(t)1 < 8/e for t > T. If we assume
the contrary, then for any 8 > 0 there exist solutions Xk (.), k == 1, 2, . . . ,
and a sequence tk  00 such that IXk(O)1 < 8 and IXk(tk)1 > 8/e. Since
x == 0 is an aSYlTIptotically stable equilibrium position, 8 > 0 can be taken
so small that for all solutions with Ix(O)1 < 8 we have Ix(t)1 < 1, t > 0,
and x(t)  0 as t  00. Moreover there exists J.l > 0 such that for all
solutions with Ix(O)1 < J.l we have Ix(t)1 < 8/e, t > O. Then J.l < IXk(t)1 < 1
for t E [0, tk], k == 1,2,.... Indeed, if IXk(t*) I < J.l for some t* E [0, tk],
then the solution z(t) == Xk(t + t*) satisfies Iz(O)1 < J.l and IZ(tk - t*)1 >
8/ e. This contradicts the choice of J.l. Consider the restrictions of solutions
Xk(.) to the interval [0, t1]. By the Arzela-Ascoli theorem we can choose a

180
8. Stabili ty
uniformly convergent subsequence. From this subsequence we can choose a
new subsequence convergent on [0, t2], etc. The limiting function x(.) is a
solution satisfying Ix(O)1 < 8 and J-l < Ix(t)1 < 1, t > 0 (see Theorem 4.6).
Therefore x(t) does not tend to zero, a contradiction. Thus there are 8 > 0
and T > 0 such that for all solutions with Ix(O)1 < 8 we have Ix(t)1 < 81e
when t > T.
Put b == sup{lx(-)lc I x(.) E S[O,r] (A, B n )}. By Corollary 4.5 b is finite.
Let x(.) E S[o,oo[(A) (from the Gronwall inequality we know that xC) exists
on [O,ooD. The function 8x(t)/lx(0)1 is also a solution to (8.4). Therefore
we have Ix(t)1 < blx(O)I, t E [O,T], and IX(T)I < Ix(O)l/e. Hence Ix(t)1 <
blx(O)l/e, t E [T,2T], and Ix(2T)1 < Ix(0)l/e 2 , etc. By induction we obtain
I x ( t ) I < b I x (0) I I e k , t E [kT, (k + 1) T],
and
I x (kT ) I < I x (0) I I e k , k == 0, 1, 2, . . . .
Set a == eb and ry == liT. It is easy to see that
I x ( t ) I < a I x (0) Ie -')'t , t > O.
This ends the proof.
o
Theorem 8.7. The following conditions are equivalent:
1. The equilibrium position x == 0 of differential inclusion (8.4) is asymp-
totically stable.
2. There exist numbers T > 0, 8 E [0,1 [, and a compact neighborhood of
the origin N such that
x + TCX E 8N
for all x E bdN and C E C.
3. There exist a piecewise linear positive definite function
(8.1 7)
V(x) == Inax{l(x;,x)11 i == I,m }, m > n
and a number fJ > 0 such that
DV(x)(v) < -fJV(x)
for all x E R n and v E A(x).
Proof. Assume that the first condition is satisfied. The proof of the second
condition consists of several steps.
1. Consider the function
V(x) = sup {h OO Ix(t)ldt I x(.) E S[Q,oo[(A, x) } .
By Lemma 8.2 V(x) < alxl/ry for all x ERn. Moreover V( -x) == V(x) and
V(ax) == aV(x) for all x E R n and a > O.

8.2. Linear-selectionable differential inclusions
181
2. Show that V is convex. Let E > O. Consider two points Xl E Rn and
X2 ERn. There exists x(.) E S[o,oo[(A, Xl + X2) such that
(8.18) V(XI + X2) < 1 00 Ix(t)ldt + E.
By Theoreln 2.3 there exists a Ineasurable map C(t) E C, t > 0, such that
x(t) == C(t)x(t). Denote by (t, s) the fU11damental matrix of this linear
equation. We have
1 00 Ix(t)ldt = 1 00 1<I>(t, O)(XI + x2)ldt
< 1 00 1<I>(t, O)xlldt + 1 00 1<I>(t, 0)x2ldt < V(Xl) + V(X2)'
COlnbining this with (8.18), we get
V(XI + X2) < V(XI) + V(X2) + E.
Since E > 0 is arbitrary and V is positively homogeneous, we conclude that
V is convex.
3. Show that
D+V(x)(v) < -Ixl
for all x E R n and v E A(x). Let E > 0 and a > O. Consider a nonzero
vector X ERn. Let x(.) E S[o,oo[(A, x) be such that
(8.19) V(x) < 1 00 Ix(t)ldt + aE.
From the definition of V we see that
(8.20) V(y(a)) + 1 01 ly(t)ldt < V(x)
for any y(.) E S[O,a:] (A, x). COlnbining (8.19) and (8.20), we get
V(y(a)) - V(x) < i oo Ix(t)ldt + 1 01 Ix(t)ldt -1 01 ly(t)ldt - V(x) + aE
< i oo Ix(t)ldt + 1 01 Ix(t) - y(t)ldt - 1 00 Ix(t)ldt + aE
(8.21) = -1 01 Ix(t)ldt + 1 01 Ix(t) - y(t)ldt + aE.
Observe that Ix(t)1 > Ixl - 2tlAllxl and Ix(t) - y(t)1 < 4tlAllxl for t > 0
sufficiently small. From (8.21) we obtain
(8.22) V(y(a)) - V(x) < -alxl + 3a 2 1Allxl + aE.

182
8. Stability
Let v E A(x). Then there exists C E C such that v == Cx. Put y(t) ==
etCx. Since V is convex and homogeneous, Theorem 1.9 implies that V is
Lipschitzian with a constant lv. From (8.22) we have
V(x + av) - V(x) < V(y(a)) - V(x) + lvl x + aCx - eaCxl
< -alxl + aE + o(a).
Dividing by a and taking the lilnit as a 1 0, we obtain
D+V(x)(v) < -Ixl + E.
Since E > 0 is arbitrary, we have
D+V(x)(v) < -Ixl
for all x E R n and v E A(x).
4. Denote by M the set {x I V (x) < I}. Obviously M is convex and
compact. Let x E R n and C E C. Then there exists 70 > 0 such that
..., ..., 7
(8.23) V(x + TeX) < V(x) - 2 1xl
for all T E [0, TO]' Since V(x) < Ixl, from (8.23) we get
V(x + TeX) < (1 - ; ) V(x).
In other words
(8.24) x + TeX E (1 - ; ) M
for all 7 E [0,70], X E bdM, and C E C .
5. Consider a finite collection of points yi E bdBn, i == 1, m, m > n,
and the set
N == {x II(yi,x)1 < S(yi,coM), i == I,m }.
By Theoreln 1.7 coM c N. Observe that N == -N, coM == -coM, and
S(yi,N) == S(yi,coM), i == I,m . Let E > O. From Theorem 1.8 alld
Proposition 1.5 we see that tIle vectors yi, i == 1, m , can be chosen in such
a way that
N c coM + EBn
if only m is sufficiently large. If E > 0 is small enough, then using com-
pactness of bcIN and C, from (8.24) we COllclude that tllere exist 7 > 0 and
fJ E [0, 1 [ such tllat
(8.25) x + TCX E fJN
for all x E bcIN and C E C. Thus the second condition is satisfied.
To derive the third condition put
V ( x) == In ax { I (x: , x) I I i == 1, m } ,

8.2. Linear-selectionable differential inclusions
183
where xi == yi/S(yi,coM). Obviously V(x) is t11e IVli11kowski fU11ction of
N. Let x E R n and C E C. Then from (8.25) a11d Lelllilla 1.6 we obtain
DV(x)(Gx) < V(x + rex) - V(x) < -OV(x),
T
where B == (1 - b) / T. Thus the third c011dition is satisfied.
The first c011dition follows from t11e third one due to Theorelll 8.2. D
Now derive a criterion of asymptotic stability for linear-selectionable
differential h1clusions in an algebraic form.
We say that an (m x m)-matrix H == "hijll has a strictly negative-
dominant diagonal according to rows if
h ii + L Ihijl < 0, i = I,m.
j#i
Theorem 8.8. Assume that C == co{ C 1 , . .. , CN }. Then the following con-
ditions are equivalent:
1. The equilibrium position x == 0 of differential inclusion (8.4) is asymp-
totically stable.
2. There exist a number m > n, an (n x m)-matrix X* of rank n, and
(m x m)-matrices Hk = IIh;)II, k = 1, N , each with strictly negative-
dominant diagonal according to rows such that
(8.26)
CkX*==X*HZ, k== I,N .
Proof. Assume that the first condition is satisfied. By Theoren1 8.7 there
exist a positive defi11ite fU11ction
V(x) == lllax{l(xi,x)11 i == I,m }, m > n,
and a nUlnber B > 0 such that
DV(X)(CkX) < -BV(x)
for all x E R n and k == 1, N . Since V(x) is positive defi11ite, the rank of the
lllatrix X* == (xi, . .. , x:n) equals n. FrOlll Lemma 3.4 we 11ave
DV(x)( v) == III ax { (xi, v)sign(xi, x) liE I(x)},
w 11ere I ( x) == {i == 1, N I V (x) == I (xi, x) I}. T11erefore
max{(xi,CkX)sig11(xi,x) liE I(x), k == I,N }
(8.27) < -B max{ (xi, X)Sig11(xi, x) I i == 1, m }
for all x ERn. Fix i == 1, m . From (8.27) we see that (Ckxi + Bxi, x) < 0
whenever x E K i == {x I (xi, x) > I (xj, x) I, j == 1, 11L }. 111 other words,

184
8. Stability
(CZ + () E)xi E - Kt. Applying Corollary 1.8, we see that there exist aj > 0
and {3j > 0, j == 1, N , such that
CZxi == - ( 0 + I=(aj + ,8j) + ai - ,8i ) xi + I=(aj - ,8j)xj.
j=l j#i
Let Hk be the matrix whose i-th row is
( a1 - ,81, · . · , - (0 +  (aj + ,8j) + ai - ,8i) ,... , am - ,8m ) ·
Obviously Hk has a strictly negative-dominant diagonal according to rows
and
CZX* == X*H k , k == 1,N .
Assume that the second condition is satisfied. Let X* == (xi,. .. , x:n).
Since rankX* == n the function
V(x) == Inax{l(xi,x)11 i == I,m }
is positive definite. Put l(x) == {i == I,m I V(x) == l(xi,x)I}. Let i E l(x).
Then froln (8.26) we obtain
( m )
* * . * _ (k) * . *
(C k X i,x)slgn(xi'x)- hij Xj'X slgn(Xi'X)
)=1
m ( m )
(k) * (k) (k) *
< L h ij I(Xj, x)1 < h ii + L Ih ij I I(Xi, x)l.
j = 1 j #i
From this and Lemma 3.4 we get
DV(X)(CkX) < -BV(x), k == 1, N,
where 0 = - max{h;;) + 2:j#i Ih;;) I I k = 1, N } > O. Applying Theorem
8.7, we see that the first condition is satisfied. D
Remark. The matrix relation (8.26) can be understood as a condition
expressing generalized similarity of the collection of matrices C k , k == 1, N ,
to the collection of matrices H k , k == 1, N, with strictly negative-dominant
diagonals according to rows.

8.3. Wealc asynlptotic stability of convex processes
185
8.3. Weak asymptotic stability of
convex processes
Let A : Rn --7 Rn be a closed strict convex process. C011sider a differe11tial
incl usion
(8.28)
x(t) E A(x(t)).
We say that the convex process A is weakly asymptotically stable if for
any x E Rn there exists a solution xC) E S[o,oo[(A, x) satisfying
lim x ( t) == o.
t-+oo
If the process A is weakly asymptotically stable, then it is easy to see
that the zero equilibrium position of differential inclusion (8.28) is weakly
asymptotically stable.
Recall that by Theorem 2.12 the restriction of A* to the subspace domA*
n -domA* is a linear operator. Denote by J c domA* n -domA* the
maximal subspace invariant by A*. Put I == J.L. Consider convex cones
Lk(A)==(A-AE)-k(O), k==1,2,....
Denote by Ao(A*) the maximal eigenvalue of the process A*. If A* has no
eigenvalue, put Ao(A*) == -00.
Lemma 8.3. Let x E Lk(A) and A < o. Then there exists a solution x(.) E
S[O,oo[ (A, x) satisfying
lin1 x ( t) == o.
t-+oo
Proof. Put Xk == x. Since x E Lk(A), then there exists a collection of
vectors xo, Xl, . . . , Xk-1 such that
Xk E (A - AE)-1(Xk_1)'... Xl E (A - AE)-l(xO), Xo == o.
Show that the function
x(t) = eAt ( (::)! XI + ... + ;, Xk-l + Xk)
is a solution to be found. Indeed,
( tk-1 t k - 2 )
x(t) = eAt (k _ l), AX I + (k _ 2)! (Xl + AX2) +... + (Xk-l + AXk)
( tk-1 t k - 2 )
E eAt (k _ l)! A(XI) + (k _ 2)! A(X2) +... + A(Xk)
( ( tk-1 t k - 2 ))
c A eAt (k _ l)! XI + (k _ 2)! X2 +... + Xk = A(x(t)).
Obviously x(t) --7 0 as t --7 00. D

186
8. Stability
Froln this lelnma and Theorem 2.14 we obtain the following result.
Theorem 8.9. Let J == {O} and Ao(A*) < O. Then the convex process A is
weakly asymptotically stable.
To derive necessary and sufficient conditions of weak asymptotic stabil-
ity, we need the following auxiliary result.
Lemma 8.4. Assume that a convex process A : R n ---+ R n is weakly asymp-
totically stable. Then there exist numbers  > 0 and a > 0 such that for any
Xo E R n there is a solution xC) E S[o,oo[(A, xo) satisfying
(8.29) Ix(t) I < alxole- 1t , t > o.
Proof. Consi der a si mplex E c R n containing a unit ball centered at zero.
Let Xk, k == 1, n + 1, be tIle vertices of the simplex. Since A is weakly
asymptotically stable, there exist trajectories Xk(.) satisfyi ng the conditions
x k ( 0) == x k , lhn x k ( t) == 0, k == 1, n + 1.
too
There exists a number r > 0 such that IXk(r)1 < lie for all k == 1, n+ 1.
Let y E bd Bn. Then there exist numbers Ak > 0, k == 1, n + 1, such
that L + i Ak == 1 and y == L + i AkXk. Obviously, the function x(., y) ==
L  i AkXk(.) is a trajectory of differential inclusioll (8.28) and satisfies the
inequality Ix(r,y)1 < lie. Hence, for any y E R n the function Xy(.) ==
Iylx(., yllyl) is a solution to (8.28) and satisfies the inequality IXy(T)1 < Iyl/e.
Set
 == 1 I r, a == e lnax { I x k ( t ) II t E [0, r], k == 1, n + I}.
For every Xo E R n we define a solution xC) to (8.28) satisfying the initial
conditioll x(O) == Xo as follows:
x(t) == Xx(mr)(t - mr), t E [mr, (m + l)T], m == 0,1,... .
It is easy to check that the trajectory xC) satisfies condition (8.29). D
With the help of the results on the structure of a convex process obtained
in Section 2.6, the investigatioll of aSYlnptotic behavior of solutions to a
process call be reduced to the study of asymptotic properties of a process
satisfying the conditions of Theorem 8.9 and asymptotic stability of a linear
operator.
The following theorem contains necessary and sufficient conditions of
weak asymptotic stability.
Theorem 8.10. Let A : R n ---+ R n be a closed strict convex process. Then
the following conditions are equivalent.
1. The process A is weakly asymptotically stable.

8.3. Weak asymptotic stability of convex processes
187
2. Only a trivial solution to the differential inclusion
(8.30)
x*(t) E -A*(x*(t))
has nonnegative Lyapunov exponent.
3. The restriction of the convex process A * to J is an asymptotically
stable linear operator and Ao(A*) < o.
Proof. Assume that the first condition is satisfied. Suppose that there
exists a nontrivial solution x*(-) to (8.30) with x[x*(-)] > O. Let Xo ERn.
By Lemma 8.4 there is a solution x(-) to differential inclusion (8.28) with
x[x(.)] > o. Observe that the Lyapunov exponent of the function (x* (-), x(-))
is positive. Taking the lhnit in the inequality
r t d
(x*(t),x(t)) = (x*(O),xo) + Jo ds (x*(s),x(s))ds
= (x*(O),xo) + I t ((x*(s),x(s)) + (x*(s),x(s)))ds > (x*(O),xo)
as t  00, we obtain (x*(O), xo) < O. Since Xo is an arbitrary vector, we have
x*(O) == O. From Theorem 2.12 (Proposition 2) and t11e Gronwall inequality
we obtain x*(t) 0, a contradiction.
Now assume that the second condition is satisfied. Suppose that A*IJ is
not asymptotically stable. Then we see that the differential inclusion (8.30)
has a solution with x*(O) == x* E J and x[x*(-)] < 0, a contradiction.
Suppose that the process A* has an eigenvector x* corresponding to an
eigenvalue A > o. Observe that the function x*(t) == e-Atx* is a solution to
(8.30) and x[x*(-)] > 0, a contradiction. Thus the second condition implies
the third one.
Assume that the third condition is satisfied. Let Xo E Rn. Our aim
is to find a solution x(.) E S[o,oo[(A, xo) satisfying lhnx(t) == 0 as t  00.
From Theorem 8.9 and Leinina 2.14 we see that the restriction of A to
I is a weakly asymptotically stable convex process. By Lemma 8.4 there
exist numbers , > 0 and a > 0 such that for any Xo E I there is a
solution x(-) E S[o,oo[(A, xo) satisfying (8.29). Let T > 0 be such that
(3 == ae-,T < 1. Consider the projection of Xo onto I: Yo == 7r(xo, I).
Let YO(-) E S[o,T](A,yo) be a solution satisfying (8.29). By Theorem 2.12
the process A is Lipschitzian with the constant IAI. Applying Theorem
4.5 with M == {yo(.)} and ro(x) XO, we see that there exists a solution
xo(.) E S[O,T] (A, xo) satisfying Ixo(t) -yo(t)1 < Ixo -yole IAlt . Put Xl == XO(T).
Now we do the saIne with the point Xl and obtain a point Y1 == 7r(X1, I),
solutions Y1(-) E S[0,T](A,Y1), X1(.) E S[O,T] (A, Xl), and a point X2 == X1(T),
etc. By induction we define sequences of points {Xk}, {Yk} and solutions
Yk(.) E S[O,T] (A, Yk), Xk(.) E S[O,T] (A, Xk).

188
8. Stability
Define a function xC) E 5[O,r] (A, xo) as
x ( t) == X k (t - k,), t E [k" (k + 1),], k == 1, 2, . . . .
SI10W tIlat lilnx(t) == 0 as t  00. Put Zk == 7r(Xk, J). By Condition 3 and
Le11l11la 2.13 all solutions to the linear differential equation
(8.31)
x(t) E A(x(t))
tend to zero as t goes to infinity. IVloreover there exist nun1bers ,\ > 0 and
d > 0 such that any solutioll x(.) to (8.31) satisfies
Ix(t)1 < blx(O)le- At , t > O.
DeIl0te by x(t) the class in R n / 1 cOlltaining x(t). Obviously x(.) is a solution
to (8.31). Hence the functioll 7r(x(t), J) also is a solution to (8.31) (we
identify R n / 1 and J). Consequently we Ilave
IZkl < blzole- Akr , k == 0,1, . .. .
Hence Te obtain
IYkl < IYk-1(') - Ykl + IYk-1(T)1 < IYk-1(T) - xkl + IYk-1(,)1
< IZk_11e 1Alr + ,8IYk-11 < blzoleIAlr-A(k-1)r + ,8IYk-11.
Set b == blzolelAlr and a == e-,\r. The previous inequality can be \vritten in
the fornl
IYkl < ba k - 1 + ,8IYk-11, k == 1,2,... ,
where a < 1 and (3 < 1.
Show that IYkl tends to zero when k goes to infinity. Without loss of
generality a i=,8. TIlen \ve have
IYkl < ba k - 1 + ,8IYk-11 < ba k - 1 + ,8ba k - 2 + (32IYk-21 < ...
< b(a k - 1 + ,8a k - 2 + ... + (3k- 2 a + (3k-1) + (3kl yo l
k ,8k
= " a a = (3 + (3kl yo l.
Froln this we obtain linl Yk == 0 as k  00.
Let us estinlate the nor11l 17r(x(t), 1) I. Let t E [kT, (k + 1),]. TheIl we
have
17r(x(t),1)1 < IYk(t - kT)1 + IYk(t - k,) - 7r(x(t), 1)1
< IYk(t - kT)1 + IYk(t - kT) - x(t)1 < alYkl + IZkle lAlr .
Therefore the norin Ix(t)1 == (17r(x(t), 1)1 2 + 17r(x(t), J)12)1/2 tends to zero
when t goes to infinity. TIlus the process A is weakly aSYlnptotically stable.
D
Consider a convex process of the fornl A(x) == Cx + K, where C is an
(n x n)-nlatrix and KeRn is a closed convex cone. Denote by J the

8.4. First approxi111atiol1 teclll1iques
189
Inaxilnal subspace contained in K* n - K* invariant by C*. The adjoint
process is given by
A* ( * ) == { C*v*, V*EK*,
v 0, v* rt K*
(see the Exaln pIe in Section 2.6). Therefore we obtain the following result.
Corollary 8.3. The following conditions are equivalent.
1. The process x  Cx + K is weakly asymptotically stable.
2. The restriction of C* to J is an asymptotically stable linear operator,
and all eigenvectors of C* contained in K* correspond to negative
eigenvalues.
We say that a convex process A : R n  R n is unstable if there is a point
x E R n such that all trajectories xC) E S(A, x) are unbounded.
Theorem 8.11. Let A : R n  R n be a closed strict convex process. If
either the restriction of the convex process A* to J is an unstable linear
operator or Ao(A*) > 0, then A is unstable.
Proof. Suppose that for any Xo E R n there exists a bounded solution xC)
to (8.28) with x(O) == Xo. It is easy to see that there exists a nontrivial solu-
tion x* C) to differential inclusion (8.30) with positive Lyapunov exponent.
Observe that the Lyapunov exponent of the function (x*(.), x(.)) is positive.
Taking the limit in the inequality
(x*(t), x(t)) = (x*(O), xo) + it :s (x*(s), x(s))ds
= (x*(O),Xo) + i t ((x*(s),x(s)) + (x*(s),x(s)))ds > (x*(O),Xo)
as t  00, we obtain (x*(O), xo) < O. Since Xo is an arbitrary vector, we have
x* (0) == O. FrOln Theorem 2.12 (Proposition 2) and the Gronwall inequality
we obtain x* (t) 0, a contradiction. D
8.4. First approximation techniques
Consider a differential inclusion
(8.32) x(t) E F(x(t)),
where F : R n  R n . Assume that 0 E F(O). We can approxiInate F in
the vicinity of the origin by a set-valued map A whose graph is a cone and
first investigate stability of the zero equilibriuln position of the differential
inclusion
x(t) E A(x(t)).

190
8. Stability
Our aim is to figure out when the zero equilibrium position of the original
differential inclusion has the saIne stability properties.
Asymptotic stability. If we consider Lipschitzian set-valued map F :
R n  R n , then for the zero equilibrium position to be stable it is nec-
essary to have F(O) == {O}. Indeed, let v E F(O) be a nonzero vector.
Put y(t) == tv. From Theorem 4.5 we conclude that there exists a solution
x(.) E S(F,O) such that Ix(t) - y(t)1 < (const)t 2 whenever t is sufficiently
small. Therefore there are E > 0 and solutions Xk(') E S(F, x(l/k)) given
by Xk(t) == x(t + Ilk) that leave the E-neighborhood of the origin while
Xk(O) == x(l/k)  0 as k  00. Thus x == 0 cannot be a stable equilibrium
position when F(O) i- {O}.
If F(O)  {O}, then a linear-selectionable set-valued Inap A : R n  R n
can be considered as a suitable first approxilnation. To derive asymptotic
stability of the zero equilibrium position of differential inclusion (8.32) from
asymptotic stability of a linear-selectionable map approxhnating F at the
origin, we need the following auxiliary result.
Lemma 8.5. Assume that F : R n  R n is a Lipschitzian set-valued map
satisfying 0 E F(O). Let c > 0, and let a function a : R+  R+ be such that
a(p)  0 as P 1 O. Then for any x E R n and E > 0 there exists Po > 0 such
that
(8.33 )
(p- 1 F(px) + a(p)Bn) n cBn C (D_F(O, O)(x) + EBn) n cBn
whenever p E]O, po].
Proof. Suppose that there exist E > 0, a sequence Pi 1 0, and points
Vi E (pi! F(Pi X ) +a(Pi)Bn) ncBn such that Vi tJ. (D_F(O, O)(x) +EBn) ncBn.
Without loss of generality Vi  Vo E cBn. Since Vo tJ. D_F(O, O)(x) + EBn,
from Propositions 2.10, 2.1, and 2.6 we obtain
o < liln inf p- 1 d(p v o, F(px)) < li!ll inf d( Vo, pi! F(Pi X ))
p!O oo
< li!ll inf( d( Vi, pi 1 F(Pi X ) + a(Pi)Bn) + Ivo - Vi I + a(Pi)) == 0,
oo
a contradiction.
D
The following result combined with Theorem 8.8 provides a useful tool
for the investigation of aSYInptotic stability.
Theorem 8.12. Assume that the set-valued map F : R n  R n is Lips-
chitzian. Let A : R n  R n be an asymptot'ically stable linear-selectionable
set-valued map satisfying T_((O,O),grF) C grA. Then the zero equilibrium
position of differential inclusion (8.32) is asymptotically stable.

8.4. First approxilnatioll techniques
191
Proof. By Theorem 8.7 there exist numbers 7 > 0, 8 E [0, 1 [, and a convex
compact neighborhood of the origin N such that
( 8.34 )
x+7A(x)c8N
for each x E bclN. Fix Xo E bclN. Let E > O. By Lelnlna 8.5 tl1ere exists
Po > 0 SUCl1 that
F(pxo) c D_F(O, O)(pxo) + PEBn C pA(xo) + PEBn
for all p E [0, po]. If E > 0 is sufficiently small, then from this and (8.34) we
get
1+8
pXO + TF(pxo) c 2 pN
for all p E [0, po]. Since F is Lipschitzian, there is 'f}o > 0 such tl1at
3+8
(8.35) px + TF(px) c 4 pN
for all p E [0, 'f}o] and x E (xo + 'f}oBn) n bclN. Now, we cover each point
Xo E bclN by such 'f}o-neighborhood and choose a finite subcovering. Let
fj be the minimal radius of a subcovering elelnent. Denote by V(x) the
Minkowski function J.L(x,N). If V(x) < fj, then from (8.35) we obtain
3+8
x + TF(x) c 4 V(x)N.
Let v E F(x). Invoking Lelnlna 1.6, we get
DV(x)(v) < V(x + TV) - V(x) < -9V(x),
7
where e == (1 - 8)/(47). Applying Theorem 8.2, we obtain the result. 0
Weak asymptotic stability. As we have Inentioned, weak asymptotic
stability is connected with properties of a control system
x == f ( x, u), u E U.
In this case it is natural to assume that F(O) =I- {O} and consider a convex
process as a first approximation as we have done in Section 6.3.
Theorem 8.13. Assume that the set-valued map F : R n  Rn is Lip-
schitzian. Let A : R n  R n be a closed strict convex process satisfying
grA C T+((O, 0), grF). If A is weakly asymptotically stable, then the zero
equilibrium position of differential inclusion (8.32) is weakly asymptotically
stable.
Proof. Consi der a si mplex  C R n containing a unit ball centered at zero.
Let Xi, i == 1, n + 1, be the vertices of the simplex. Since A is weakly

192
8. Stabili ty
asymptotically stable, by Lemma 8.4 there exist trajectories Xi (.) satisfying
the conditions
Xi (0) == Xi, I Xi ( t) I < a I Xi Ie - l't , t 2: 0, i == 1, n + 1.
Let T > 0 be such that xi(T) E , i == 1, n + 1.
Let X == CO{X1(.)'... ,Xn+1(.)} C AC([O,T],R n ). By Theorem 4.14
there exists'\* and a map 0: [0,'\*] x X  AC([O,T],R n ) satisfying the
following conditions:
1. '\x ( .) + 0 ( '\, X ( . )) E S [0, T] (F, X ( 0 ) ) ;
2. 0 ( '\, X ( . ) ) ( t) E ,\  , t E [0, T].
Set r == {I' == (1'1,... , I'n+1) E Rn+1 I E +t I'i == 1, I' > 0, 'l ==
1, n + I}. Let I' == (1' 1 , . .. , I'n+ 1 ) E r and ,\ E [0,'\ * ]. Put
x A ,')'(.) = A  '"lx iO + 0 (A,  '"lXiO) ·
Denote b == Inax{lxill i == 1, n + I}.
Observe that
(8.36)
1 1
IXA,')'(t)I < Aab + 4 Ab = A(a + 4 )b,
1 1 1
(8.37) xA,')'(T) E 4 AI; + 4 AI; = 2 AI;.
Let E > o. Put'\o == min{'\*, E/((a + 1/4)b)}. Choose 8 == '\0. Let
Xo E 8Bn c ,\o. Define a solution x(.) E S[o,oo[(F, xo) by induction. There
exists 1'0 == (1'6, . .. , 1'+1) E r such that Xo == '\0 E +t I'&Xi. Define
X ( t) == X AO ,1'0 ( t ) , t E [0, T].
From (8.36) and (8.37) we have
I X ( t) I < E, t E [0, T],
and
1
x(T) E 2 AoI;.
Set '\k == '\0/2k, k == 1, 2, . . .. Suppose that x(.) is already defined for
t E [0, (k - l)T] and satisfies
E
Ix(t)1 < 2 k - 1 ' t E [(k - 2)T, (k - l)T]
and
x((k - l)T) E '\k.

8.4. First approxill1atioll tecllniques
193
There exists k == (k, . .. , +1) E r such that x( (k -1 )T) == Ak E +/ kXi.
Define
x(t) == XAk(Yk (t - (k - l)T), t E [(k - l)T, kT].
From (8.36) and (8.37) we llave
1 E
Ix(t)1 < Ak(a + 4 )b < 2 k ' t E [(k - l)T, kT]
and
1
x(kT) E -Ak E == Ak+1E.
2
Thus x(t)  0 as t  00. The tlleorem is proved.
D
Consider the differential inclusion
(8.38)
X E f(x, U) = U f(x, u),
uEU
where f : R n x R m  R n is a continuous functiol1 differentiable in x,
and U c R m is a compact set. Assume that there is Uo E U such that
f(O,uo) == O. Let the derivative \!xf(x,u) be continuous in (x,u), and
let the set f ( x, U) be convex for all x E R n . From Theorem 8.13 and
Proposition 2.8 we obtain the following result.
Corollary 8.4. If the convex process x  \! xf(O, uo)x + conef(O, U) is
weakly asymptotically stable, then the equilibrium position x == 0 of differen-
tial inclusion (8.38) is weakly asymptotically stable.
Instability. Instability at first approximation can be established with the
help of the following obvious theorem.
Theorem 8.14. Let A : R n  R n be an unstable convex process satisfying
(8.39)
grF n (EBn X R n ) c grA,
where E > O. Then the zero equilibrium position of differential inclusion
(8.32) is unstable.
Note that COl1dition (8.39) cannot be replaced by
(8.40)
T_((O,O),grF) c grA
as it would be natural to expect. Indeed, consider the differential inclusiol1
(8.41)
(X 1 ,X 2 ) E F(x 1 ,x 2 ) == (x 1 ,x 2 + (x 1 )3) + [-1,1] x {O}.
The graph of the convex process
A(x 1 ,x 2 ) == {(v 1 ,v 2 ) E R 2 1 v 2 == x2}

194
8. Stabili ty
is the contingent cone to grF at the origin. The adjoint process is
A * ( *1 *2 ) == { (0, v*2), v*l == 0,
v , v 0, V*l:/= o.
It is easy to see that (0,1) is an eigenvector of A* corresponding to the
eigenvalue 1. By Theorem 8.11 A is unstable. However the zero equilibrium
position of differential inclusion (8.41) is weakly asymptotically stable. To
show this observe that the solution to t11e differe11tial equation
(8.42)
(Xl, x 2 ) == (-xl, x 2 + (xl )3)
tends to zero if the initial point belongs to the curve x 2 == _(x 1 )3/4. Con-
sider the function
{ ( Xl - 1 ) x 2 + ( x 1 ) 4 / 4, x2 < 0,
V ( x 1 , x2 ) ==
(xl + 1)x 2 + (x 1 )4 /4, x 2 > o.
If xl E] -1, 1[ and (x 1 ,x 2 ):/= (0,0), then V(x 1 ,x 2 ) > O. Let (x 1 ,x 2 ) E !Bn.
Then we have
DV(X 1 ,X 2 )(_X 1 + 1,x 2 + (x 1 )3) == 0,
DV(x 1 ,x 2 )(-x 1 + 1,x 2 + (x 1 )3) == 0,
DV(x 1 ,x 2 )(-x 1 -1,x 2 + (x 1 )3) == 0,
DV(x 1 , x 2 )( _xl - 1, x 2 + (xl )3) == 0,
x 2 < O.
,
xl < 0 x 2 == O.
, ,
x 2 > 0;
xl > 0, x 2 == O.
If the initial point is sufficiently close to zero, then by Theorem 5.5 inclusion
(8.41) has a trajectory satisfying V(x 1 (t), x 2 (t)) V(x 1 (0), x 2 (0)). Thus
the zero equilibrium position of (8.41) is weakly stable. Since any trajectory
along which V is a constant Ineets the curve x 2 == -i(x 1 )3, we see that
(0,0) is a weakly asymptotically stable equilibriuln position.
8.5. Stability of a lllissile uniforlll
lllotion
Consider a model of a missile moving in the vertical plane. We can vary the
thrust of its jet propulsion in value and direction. If we neglect the angular
motion and size of the missile and suppose that the velocity of the missile is
always directed along the longitudinal axis, then the motion of the missile

8.5. Stability of a missile uniform motiol1
195
mass center is described by the following equations:
1 . 1 2 .2
..1 .1 U Z - U Z
Z == -az +
V (.i 1 )2 + (z2)2'
1 .2 2 . 1
..2 .2 U Z + u z
z == -az +
V (z2)2 + (z2)2'
(u 1 ,u 2 ) E U == {(u 1 ,u 2 ) I V (u 1 )2 + (u 2 )2 < b,
u 2 1u 1 < tan 7], u 1 > O},
(8.43 )
(8.44 )
where a > 0 stands for a coefficient of air resistance, b is the maximal thrust,
and 7] is the maximal angle between the longitudinal axis of the missile and
that of the jet propulsion. Our aim is to study the stability of the object
uniform motion along the zl-axis with the constant maximal speed zl == bla.
The control (u6, u6) == (b,O) corresponds to the uniform motion.
Let us introduce new variables xl == zl - bla, x 2 == z2, x 3 == z2. System
(8.43) now can be written in the form
b u1(x1+Q)-u2x3
xl == -a(x 1 + _) + a ,
a V (x 1 + *)2 + (x 3 )2
x 2 == x 3
,
u 1 x 3 + u 2 (x 1 + Q)
x 3 == -ax3 + a
V (x 1 + *)2 + (x3)2'
(u 1 , u 2 ) E U,
( 8.45 )
and the original problem is reduced to the study of the system's (8.45) zero
equilibrium position.
Consider the first approximation of system (8.45),
( :: ) = ( f  ) ( :: ) + ( :: ) ,
where (w 1 ,w 2 ,w 3 ) E K == {(w 1 ,w 2 ,w 3 ) I wI < 0, w 2 == O}. Obviously, the
transposed matrix
( -a 0 0 )
C* == 0 0 0
010
has the eigenvector ( -1, 0, 0) corresponding to the eigenvalue -a, the eigen-
vector (0,0,1) corresponding to the eigenvalue 0, and the invariant subspace
spanned by the vectors (0,0,1) and (0,1,0) corresponding to one Jordan

196
8. Stability
block of the matrix C* with the eigenvalue O. Sh1ce
K* == {(w 1 *,w 2 *,w 3 *) I w 1 * < 0, w 3 * == O}
we observe that the invariant space spanned by the vectors (0,0,1) and
(0,1,0) is not contained in the cone K*. The vector (-1,0,0) belongs to
the cone K*, but it corresponds to the negative eige11value -a. By Corol-
laries 8.3 and 8.4 the zero equilibrium position of system (8.45) is weakly
asymptotically stable.
8.6. Problems
1. Consider a discrete-time inclusion
(8.46 )
Xt+ 1 E F (Xt), t == 0, 1, . . . ,
where F : R n  R n is a set-valued map. Assume that the point x == 0
is an equilibrium position of inclusion (8.46), that is, 0 E F(O). We say
that the equilibrium position x == 0 of (8.46) is weakly asymptotically
stable if given E > 0, there is 8 > 0 such that for any x E 8Bn there
exists a solution Xt, t == 0,1, . . . , of (8.46) with Xo == x satisfying
Xt E E Bn , t == 0, 1, . . ., an d li m x t == O.
t-+oo
Assume that there exist a number rJ > 0, a number 1/ E]O, 1[, and a
positive definite function V : R n  R such that
inf V ( v) < 1/ V ( X )
VEF(x)
for all Ixi < rJ. Show that the equilibrium position x == 0 of discrete
system (8.46) is asymptotically stable.
2. Consider a linear discrete systelTI
(8.47)
Xt+1 == CXt, t == 0,1,. . . ,
where C is an n x n matrix. Prove that the equilibrium position x == 0
of system (8.47) is asymptotically stable if and only if all eigenvalues
of C have absolute value less than 1.
3. Show that if all eigenvalues Ai, i == 1, n , of the matrix C have absolute
value less than 1, then there exists a positive definite quadratic form
V(x) such that
V(CX) < AV(X),
where 1 > A > max{IAill i == 1,n }.

8.6. ProblelTIs
197
4. Let A : Rn  R n be a strict closed convex process. Consider the
discrete- tilne inclusion
( 8.48 )
Xt+ 1 E A (Xt), t == 0, 1, . . . .
To characterize the asymptotic behavior of a sequence Xt, t == 0,1, . . . ,
the Lyapunov exponent
X[Xt] = -limsup  In IXtl
t-too t
is used. Show that the following conditions are equivalent:
(a) The equilibriuln position x == 0 of inclusion (8.48) is weakly
asymptotically stable.
(b) The discrete-tilne inclusion
X;+l E (A*)-l(x;), t == 0,1,...
has only trivial solution with nonnegative Lyapunov exponent.
(c) Eigenvalues of the restriction of A * to J have absolute values
less than one and Ao(A*) < 1.
(d) There exist 8 E]O, 1[ and a convex positively homogeneous func-
tion V(x) such that V(O) == 0, V(x) > 0, if x =1= 0, and for all
x E Rn a vector v E A(x) satisfying V(v) < 8V(x) can be
found.
(Hint. Use Problems 1 - 3.)
5. Assume that x == 0 is a weakly asymptotically stable equilibrium po-
sition of discrete-time inclusion (8.48), and A approximates F in a
neighborhood of the origin. Prove that x == 0 is a weakly asymptoti-
cally stable equilibrium position of discrete-time inclusion (8.46).
6. Let A : R x Rn  R n be a set-valued map satisfying the following
conditions:
(a) The map x  A(t, x) is a strict closed convex process;
(b) A is periodic in t of period T > 0;
(c) the map t  A(t, x) is measurable for all x E R n and
max min Ivl < l(t), t E [0, T],
xEB n vEA(t,x)
where l(-) E £1 ([0, T], R).
Consider tIle differential inclusion
(8.49)
X E A(t, x).
Denote by A(x) the closure of the set of points x(T), where x(.) is a
solution to the Cauchy problem
x ( t) E A ( t, x ( t ) ) , x ( 0) == x, t E [0, T].

198
8. Stabili ty
(8.50 )
Obviously A is a convex process. With differential inclusion (8.49)
we associate the discrete-time inclusion
x p + 1 E A (x p ), p == 0, 1, . .. .
Denote by :J the maximal subspace invariant by A *. Prove that the
following conditions are equivalent:
(a) The equilibrium position x == 0 of differential inclusion (8.49)
is weakly asymptotically stable.
(b) The differential inclusion
x* E -(A*)(t, x*)
has only trivial solution with nonnegative Lyapullov exponent.
(c) Eigenvalues of the restriction of A * to :J have absolute values
less than one and Ao(A*) < 1.
(Hint. Use Problem 4.)
Consider the differential inclusion
7.
(8.51)
X E F(t, x),
where F : Rx R n  R n is a set-valued map with closed convex values.
Assume that
(a) 0 E F(t,O) for all t E R.
(b) F is periodic in t of period T > O.
(c) The map t  F(t, x) is measurable for all x E R n and F(t, x) c
b(t)B n for all (t, x) E [0, T] x R n , where b(-) E L1 ([0, T], R).
(d) The map x  F(t, x) is Lipschitzian for all t E [0, T], with
a constant k( t), where k(.) E £1 ([0, T], R); that is, F( t, Xl) C
F(t, X2) + k(t)lx1 - x21Bn for all Xl and X2 in R n and t E [0, T].
Assume that the equilibrium position x == 0 of differential inclusion
(8.49) is weakly asymptotically stable alld that A(t, x) is a first ap-
proximation of the map x  F(t, x) at x == 0 for all t. Prove that
x == 0 is a weakly asymptotically stable equilibrium position of differ-
ential inclusion (8.51). (Hint. Use ProblelTIs 1 and 4.)
8. Specify Problems 6 and 7 for a time-periodic control system.

Chapter 9
Stabilization
Consider a differential inclusion
x(t) E F(x(t)).
Assume that x == 0 is a weakly asymptotically stable equilibrium position.
The problem we consider in this chapter is to find a selection f(x) E F(x)
such that f(O) == 0 and the zero equilibrium position of the differential
equation
x(t) == f(x(t))
is asymptotically stable. This problem is referred to as a stabilization prob-
lem, and the selection f ( x) is referred to as a stabilizer. Usually the differ-
ential inclusion is generated by a control systen1
x == f ( x, u ) , u E U.
In this case the stabilizability, that is, the existe11ce of a stabilizer, is equiv-
alent to the existence of a feed-back control u(x) E U such that the origin
is an asymptotically stable equilibrium position of the differential equation
x == f(x, u(x)).
To solve the stabilization problem we construct a Lyapunov function V for
a weakly aSYlnptotically stable convex process of first approximation and
show that the stabilizer can be found solving the following minimization
problem:
min V (x + TV) == V (x + T f ( x ) ) , T > O.
VEF(x)
We also discuss the relationship between the concepts of weak asymptotic
stabili ty and stabilizabili ty.
-
199

200
9. Stabilization
9.1. Lyapunov functions for convex
processes
In Section 8.1 we constructed a Lyapunov function for an aSYlnptotically
stable linear operator. Here we show that it is possible also to construct a
Lyapunov function for a weakly asymptotically stable closed convex process
A : Rn  R n . Namely, we construct a convex positively homogeneous
function V : Rn  R n such that for any x E R n there exists v E A(x)
satisfying
V(x + TV) < 8V(x),
where 8 E [0,1[. Invoking Lelnma 1.6, we obtain
DV(x)(v) < V(x + TV) - V(x) < -BV(x),
T
w11ere () :=: (1 - 8)/T.
Let A : Rn  R n be a weakly asymptotically stable strict closed convex
process. Recall that by J c domA * n -domA * we denote the maximal
subspace invariant by A*, we denote J..l by I, and Ao(A*) stands for the
maxhnal eigenvalue of the process A*. If A* has no eigenvalue, put Ao(A*) :=:
-00. We also use the notation
Lk(A) :=: (A - AE)-k(O), k:=: 1,2,. . . .
Since R n :=: I x J, every x E Rn can be represented in the form x :=: (x I, x J ),
where XI E I and XJ E J. By Theorem 8.10 the restriction of the convex
process A* to J is an asymptotically stable linear operator and Ao(A*) < O.
Fix A E]Ao(A*),O[. Let  c I be a simplex containing the ball BI :=:
BnnI. Denote by xp, i :=: 0, m , the vertices of the simplex. By Theorem 2.14
xr E L ki (A) for some k i . Moreover, xr does not belong to Lk(A) if k < k i .
From the definition of the cones L ki (A) we derive the existence of a finite
set of nonzero vectors {Yi,j} i 1 in I that satisfy the following inclusions:
AYi,l E A(Yi,l),
Yi,l + AYi,2 E A(Yi,2),
(9.1)
****
* * **
****
Y i k'-l + A Y i k.
,  , 
x

E A(Yi,ki)'
Y i k..
, 
Fix a number a > 1. Let lvII be the convex hull of the points
(9.2) Zi,j :=: (a/IA/)ki-jYi,j, j:=: 1, k i , i:=: 0, m.
Let us consider the linear operator A : J  J. From Lemma 2.13 we
see that A is asymptotically stable. By Corollary 8.2 there exists a positive

9.1. Lyapunov functions for convex processes
201
definite quadratic form W : J ---7 R which is a Lyapunov function for the
differential equation
xJ(t) == AXJ(t).
Denote by MJ the ellipsoid {x E J I W(x) < I}. Let w > O. Consider the
convex compact set
Mw == M/ x wMJ c I x J == R n .
Denote by V (x) the Minkowski function of the set Mw.
Theorem 9.1. If w > 0 is sufficiently small, then there are 7 > 0 and
8 E [0, 1[ such that for each x E R n there exists v E A(x) such that
V(x + 7V) < 8V(x).
Proof. For all i
inclusion
0, m , the vector Vi,l == AZi,l E A(Zi,l) satisfies the
Zi,l + IAI- 1V i,l == 0 E int 1\11/.
For all points Zi,j, j > 1, i == 0, m , we consider the vectors
Vi,j == (1/ a )IAl z i,j-1 + AZi,j.
Obviously, we have Vi,j E A(Zi,j) and
Zi ) ' + IAI- 1v i ) . = Zi ) '-1 E int M/.
, , a'
From the above reasoning we conclude that there exist nun1bers 7/ > 0
and 8/ E]O, 1[ such that for each x/ E bd1\lI/ there exists a vector v E
A( (x/, 0)) satisfying the inclusion
(XI,O) +7/V E 8/M/.
Moreover, there exist numbers 7 J > 0 and 8 J E] 0, 1 [ such that for all
XJ E bdM J the inclusion
xJ + 7JAxJ E 8JMJ
is fulfilled. Set 7 == min { 7/ , 7 J }, 8 == max { 8/, 8 J} and choose w from the
interval ]0, (1- 8)/(47IAlb)[, where b == max{lxJII XJ E bdMJ}.
Let x == (x/,xJ) E bdMw. Then there exists a nUlnber 1] E [0,1]
such that x/ E bd ( 1] M / ) . Observe that there exists a vector v == (v / , 0) E
A ( (x / , 0)) satisfying the inclusion
( x / , 0) + 7 ( V / , 0) E (81]1\11/, 0) C (81\11/, 0) .
Since A is Li pschi tzian with the constant I A I, there exists v == (v / , v J) E A ( x )
satisfying Iv - vi < IAllxJI < IAlwb < (1 - 8)/(47). Since lv/ - v/I <
(1 - 8) 1 ( 47 ), we have
(9.3)
1-8 1-8 1+8
XI +TVI E XI +Tih + 4 BI C 6'rJMI + 4 MI C 2 MI.

202
9. Stabilization
On the other hand, we have
1+6
(9.4) XJ + TVJ == XJ + T AXJ E 6wMJ c wMJ.
2
Summation of (9.3) and (9.4) yields
1+6 1+6
x + TV E 2 (l\IiI, wMJ) C 2 Mw.
Thus for any x E R n there exists v E A(x) such that
1+6
V(x + TV) < 2 V(x).
The theorem is proved.
D
9.2. Stabilization problem
We now proceed to study the stabilization problem for a differential inclusion
(9.5)
x(t) E F(x).
We assume that 0 E F(O). The following theorem contains sufficient con-
ditions for stabilizability of inclusion (9.5) at first approximation. First we
construct a discontinuous stabilizer f (x). Solutions to the differential equa-
tion
(9.6)
x(t) == f(x(t))
are regarded to be solutions to the differential inclusion
i; E n cl cof(x + fBn).
€>o
Theorem 9.2. Assume that F : Rn  Rn is a Lipschitzian set-valued map
with closed convex values. Let A : Rn  Rn be a closed strict convex process
satisfying grA C T+((O,O),grF). If A is weakly asymptotically stable, then
there exist a neighborhood n of the origin and a map f : n  R n that
satisfies the following conditions:
1. f(O) == O.
2. f(x) E F(x), \Ix E n.
3. The equilibrium point x == 0 of the differential equation (9.6) is
asymptotically stable. Moreover, there exist constants a > 0 and () > 0
such that
(9.7) Ix(t) I < alx(O) le- Bt , t > 0,
(9.8) Ix(t) I < alx(O) le- Bt , t > 0
for any solution x(.) to (9.6) with sufficiently smalllx(O) I.

9.2. Stabilization problem
203
Proof. By Theorem 9.1 there exist numbers T > 0, 8 E [0,1[, and a convex
positively homogeneous function V(x) such that V(O) == 0, V(x) > 0, if
x =1= 0, and for any x E bdM (M == {x I V(x) < I}) there exists a vector
v E A(x) satisfying x + TV E 8M.
Let Xo and Vo be vectors from the sets bdM and A(xo) respectively which
satisfy Xo + TVo E 8M. If E1 > 0 is sufficiently small, then the inclusions
x E Xo + E1Bn, V E Vo + E1 B n imply that x + TV E (1 + 8)M.
From Proposition 2.10 we see that there exists E2 > 0 such that the
inequality
p- 1 d(p v o, F(px)) < E1
holds for all x E Xo + E2Bn, P E]0,E2[. Set EO == min{E1,E2}. Let x E
bdM n (xo + EoBn), P E]O, EO[. Then there exists a vector v(p, x) E F(px)
satisfying Ivo - p- 1 v(p, x) I < E1. We observe that
1+8
x + Tp-1 v (p, x) E M.
2
Now, we cover every point Xo E bdM by such Eo-neighborhood and
choose a finite subcovering. Let"E be the minimal radius of a subcovering
element. If V(x) < "E, then by the above considerations we conclude that
there is a vector v E F(x) satisfying
1+8
X+TV E 2 V(x)M.
Let us consider the set-valued maps
1+8
G(x) = {v E R n I x + TV E 2 V(x)M}, H(x) = G(x) n F(x)
defined on the set n == {x I V(x) < "E}. We take any single-valued map
j(x) E H(x). Obviously j(O) == O. Consider the set-valued map
<I>(x) = n cl co f(x + T]Bn).
1]>0
Let v E <I>(x). Then we have
DV(x)(v) < [V(x + TV) - V(x)] < -OV(x),
T
where () == (1- 8)/(2T). Froln Theorem 8.2 we see that the zero equilibrium
position of differential equation (9.6) is asymptotically stable. Moreover,
any solution x(.) of (9.6) satisfies
(9.9) V(x(t)) < V(x(O))e- Ot , t > 0,
whenever x(O) E f2. By construction
x(t) E G(x) = T- 1 ( 1 ; 8 V(x(t))M - x(t))

204
9. Stabilization
(9.10) C 7- 1 e- Ot V(x(0)) ( 1 ; 8 M - M ) .
Froln (9.9) and (9.10) we see that there exists a constant a > 0 such that
(9.7) and (9.8) are satisfied for aI1Y solution x(.) to (9.6) with sufficiently
snlalllx(O)I. The theoreln is proved. 0
Remark. For exan1ple, f(x) E F(x) can be chosen fron1 the condition
Inin V(x + TV) == V(x + T f(x))
vEF(x)
if F( x) is a cOInpact.
Consider the differential inclusion
X E f(x, U) == U f(x, u),
uEU
where f : R n x R m  R n is a continuous function differentiable in x,
and U C R 1n is a cOInpact set. Assun1e that there is Uo E U such that
f(O, uo) == O. Let the derivative \l xf(x, u) be continuous in (x, u), and let the
set f(x, U) be convex for all x ERn. Froln Theoreln 9.2 and Proposition 2.8
we obtain the follo,ving result.
(9.11)
Corollary 9.1. If the convex process x  \l xf(O, uo)x + conef(O, U) is
weakly asymptotically stable, then differential inclusion (9.11) is stabilizable.
TheoreI11 9.2 shows that there exists a stabilizer f(x) such that the origin
is an aSYInptotically stable equilibrilun point of differential equation (9.6)
with discontinuous right-hand side. A natural question arises: is there a
stabilizer which Inakes the right-hand side of (9.6) a continuous function?
A positive answer is given by the following theore111.
Theorem 9.3. Under the conditions of Theorem 9.2 there exists a contin-
uous stabilizer f (x) .
Proof. As in the proof of Theoreln 9.2 we observe that there exist nlunbers
T > 0, 8 E]O, 1[, and a convex positively hOlnogeneous function V(x) such
that V(O) == 0, V(x) > 0, if x i- 0, and for any x froln the neighborhood of
the origin n the set-valued Inap
H(x) = 7- 1 ( 1; 8 V (x)M - x) nF(x),
where M {x I V(x) < I} is nOl1eInpty. Let us consider the set-valued
111ap
H(x) = 7- 1 ( 3: 8 V (x)M - x) nF(x).

9.3. Weal( aSYlnptotic stability and stabilizability
205
The Inap H(x) has convex cOInpact values. By Proposition 2.3 it is up-
per selni-continuous. Let f(x) be the projectioll of the origin OlltO H(x).
Obviously
(r- 1 ( 3:8 V (X)M -x) +z) nF(x):I 0
whenever Izl is slnall enough. By Proposition 2.5 H(x) is cOlltinuous. Fronl
Theorem 2.2 we see that f (x) is a continuous functioll. Asynlptotic stability
of the zero equilibrium position of the differential equatioll
x(t) == f(x(t))
can be proved sinlilarly to the last part of Theoreln 9.2.
o
Remark. From the algorithmic point of view the construction of a contin-
uous stabilizer f (x) is Inore cOInplex thall the procedure described ill the
relnark after Theoreln 9.2.
9.3. Weak asymptotic stability and
stabilizability
In this section we study the relationship between weak aSYInptotic stability
and stabilizability. COllsider a differential inclusion
(9.12)
x(t) == F(x(t)),
where F : R n --t R n is such that 0 E F(x). Obviously, if the stabilizatioll
problem for inclusion (9.12) is solvable, then the origill is its weakly aSYInp-
totically stable equilibriunl position. As we know frolll Theoreln 9.2, for the
first approximation these conditions are equivalellt. In general stabilizability
cannot be derived froln weak asymptotic stability.
Consider the differential inclusion generated by the following differential
equation with discontinuous right-hand side:
( '1 . 2 ) _ f( 1 2 ) _ { (xl, x 1 (x 2 )1/3), (xl, x 2 ) E 0 1 ,
X , X - x, X - ( 1 2 ) ( 1 2 ) ()
- X , X , X , x E  [,2,
where 0 1 == {(xl, x 2 ) I xl > 0, Ix 2 1 < (xl )2}, O 2 == R 2 \ 0 1 . Let (X6, x6) E
O 2 . Then the trajectory
(9.13)
(x 1 (t), x 2 (t)) == e- t (x6, x6)
of system (9.13) obviously tends to zero when t --t 00.

206
9. Stabilization
Let (xb, x6) E 0 1 , and let x6 > 0 (the case x6 < 0 is similar to that one).
Then the trajectory
(x 1 (t), x 2 (t))  (x6 et , ((2/3)(x6(e t - 1) + (3/2)(x5)2/3)3/2)
satisfies x 2 (t)  ((2/3)(x 1 (t)+(3/2)(X5)2/3-xb))3/2. We see that there exists
t* such that x 2 ( t*)  (x 1 ( t*) ) 2 if (xb, x6) is sufficiently close to the origin.
Hence (x 1 (t*), x 2 (t*)) E O 2 . Thus, the equilibrium position (xl, x 2 ) == (0,0)
of system (9.13) is weakly asymptotically stable. Nevertheless, it is not
stable. Indeed, let xb > O. Then the trajectory (etxb,O) of system (9.13)
does not tend to the origin. Thus the stabilization problem is not solvable.
In order to derive stabilizability frOlTI a stability concept, we introduce
the following definition. We say that the equilibrium positio11 x == 0 of dif-
ferential inclusion (9.12) is weakly exponentially stable if there exist positive
constants a, (), and  such that for each Xo E Bn at least one trajectory
xC) of (9.12) with x(O) == Xo satisfies
Ix(t)1 < alxole- Bt , t > 0,
Ix(t) I < alxole- Bt , t > O.
From Theorem 9.2 we see that if a differential inclusion is stabilizable at
first approximation, then its zero equilibrium position is weakly exponen-
tially stable. Below we prove a partial inverse of Theorem 9.2.
Theorem 9.4. Let F : R n ----7 R n be a Lipschitzian set-valued map with
closed convex values contained in a ball of radius b > o. Assume that
T_((O, 0), grF)  T+((O, 0), grF) == coT+((O, 0), grF). If the zero equilibrium
position of (9.12) is weakly exponentially stable, then the convex process A
defined by grA == T+((O, 0), grF) is strict and weakly asymptotically stable,
and differential inclusion (9.12) is stabilizable.
Proof. Let i E R n . Since the equilibrium position x == 0 is weakly expo-
nentially stable, there exist trajectories Xk (.), k  1, 2, . . . , of system (9.12)
with Xk(O) == i/k satisfying
(9.14) IXk( t) I <  Ifle- Ot , t > 0,
(9.15) IXk(t)1 <  Ifle-Ot, t > 0,
where a > 0 and () > 0 and k is big enough. Introduce a new time T ==
1 - (t + 1)-1. Obviously T varies in [0, 1[ when t varies in [0,00[. Let
Yk(T) == Xk(t( T)), T E [0,1[, k == 1,2,. . . .
Then we have
(9.16)
d
dr Yk(r) E (1 - r)-2 F(Yk(r)).

9.3. Weak asymptotic stability and stabilizability
207
Froln (9.14) and (9.15) we obtain
(9.17) IYk(r)1 <  Ixle-O«l-T)-l_l), r E [0,1[,
(9.18) IYk(r)1 <  lxl(l- r)-2 e -O«1-T)-1_1), r E [0,1[.
Consequently Yk(T)  0 and Yk(T)  0, k == 1,2,..., as T  1. Put
by definition Yk(l) == 0, k == 1,2,.... From (9.17) and (9.18) we see that
the sequence {kYk(.)} r 1 is bounded in sup-norm and equicontinuous. By
the Arzela-Ascoli theorem it contains a uniformly convergent subsequence.
Without loss of generality kYk (.) uI1iformly converge to a continuous function
y(.). From (9.16) and (9.18) we see that there exists c > 0 such that
kYk(T) E k(l - T)-2 F(Yk(T)) n cBn
(9.19) C k(l - T)-2(F(k- 1 Y(T)) + lIYk(T) - k- 1 Y(T)I B n) n cBn,
where l > 0 is the Lipschitz constant of F. From Lemma 8.5 we see that
kYk(T) E (1 - T)-2(D_F(0, O)(Y(T)) + Ek(T)Bn) n cBn
== (1 - T)-2(A(Y(T)) + Ek(T)Bn) n cBn,
where SUPT Ek(T)  0 as k  00. Applying Lemma 4.4, we get
Y(T) E (1- T)-2 A(Y(T)), T E [0,1[.
From (9.17) we obtain
IY(T)I < alxle- O ((l-T)-l_l), T E [0,1[.
Thus, the function x(t) == Y(T(t)) satisfies
x(t) E A(x(t))
and
Ix(t) I < alxle- Ot , t > O.
Therefore A is a strict weakly asymptotically stable convex process. From
Theorem 9.2 we derive stabilizability of differential inclusion (9.12). D
Consider the differential inclusion
X E f(x, U) = U f(x, u),
uEU
where f : R n x R m  R n is a continuous function differentiable in x,
and U c R m is a compact set. Assume that there is Uo E U such that
f(O, uo) == o. Let the derivative \l xf(x, u) be continuous in (x, u), and let
the set f(x, U) be convex for all x ERn.
From Theorems 9.4 and 2.4 we obtain the following result.
(9.20)

208
9. Stabilizatiol1
Corollary 9.2. Let f : R n x U ---7 R m be twice differentiable in x function.
Assume that the derivatives are continuous in (x, u) and Uo is a unique vector
in U satisfying f(O, uo) == o. If the zero equilibrium position of differential
inclusion (9.20) is weakly exponentially stable, then the convex process x ---7
 xf(O, uo)x + conef(O, U) is weakly asymptotically stable and differential
inclusion (9.20) is stabilizable.
Remark. The uniqueness of Uo E U is essential. Indeed, the zero equi-
librium position of the control system x == ux, lul < 1, Uo == 0, is weakly
exponentially stable. The control u -1 is a constant stabilizer. N everthe-
less the first approximation is x == o.
9.4. Stabilizers for some mechanical
systems
In Section 6.4 we established the controllability of two mechanical control
systems. It was done using the method of first approxilnation. From the re-
sults of this chapter it is clear that the stabilization problem for the systems
is also solvable. It suffices to find Lyapunov functions for the corresponding
convex processes of first approximation.
Stabilization of a point body. Recall that the motion of a point body
subjected to a force is described by the following control system:
xl == x 2
,
x 2 == U
,
(9.21)
u E [-1,1].
The control Uo == 0 corresponds to the zero equilibrium position. The convex
process A(x) == Cx + K, where
c=( )
and
K == {(w 1 ,w 2 ) I wI == O},
approximates (9.21) in a neighborhood of the origin. The transposed oper-
ator C* has neither nontrivial subspaces nor eigenvectors contained in the
conjugate COI1e K* == {( w 1 *, w 2 *) I w 2 * == O}. Therefore, thanks to Corol-
lary 2.3 for all negative A the cone Lk(A) coincides with the whole space for
SOlne k. Indeed, we observe that
L 1 (A) == {(x 1 ,x 2 ) I (x 1 ,x 2 ) == (a, Aa + {j), a,{j E R} == R 2 .

9.4. Stabilizers for SOlTIe ll1ecl1anical systelTIS
209
Let us take the points
x == (1,0), xr == (0, 1), x == (-1, -1)
for the vertices of the polyhedron. Systelll of inclusions (9.1) reduces to
the systelll
AYi,l E A(Yi,l),
Yi,l + AYi,2 E A(Yi,2) ,
x Yi,2.

This is a systelll of linear inequalities. Solving it for i == 0,1,2, with
A < 0 and using normalization (9.2) with some a > 1, we obtain points
Zi,j == (a/IAI)2-jYi,j, j == 1,2, i == 0,1,2. The Minkowski function of the
polyhedron CO{Zi,j I j == 1,2, i == 0,1, 2} is the Lyapunov function to be
found.
Stabilization of an oscillator subjected to a unilateral force. The
motion of an oscillator subjected to a unilateral force is described by the
equations
Xl == x 2
,
x2 == -xl + U,
U E [0, 1].
The control Uo == 0 corresponds to the equilibriulll position. The convex
process A(x) == Cx + K, where
C=(_ )
(9.22)
and
K == {(w 1 ,w 2 ) I wI == 0, w 2 > O},
is a first approximation for system (9.22) near zero. The transposed ma-
trix C* has neither nontrivial subspaces nor eigenvectors contained in the
conjugate cone K* == {(w 1 *,w 2 *) I w 2 * > O}. Therefore by Corollary 2.3
for all negative A the cone Lk(A) coincides with the whole space for some
k == k(A). In the previous example k(A) == 2 for all A < o. The example
under consideration is of a different nature.
Show that k(A) ---7 00 as A ---7 -00. Indeed, by definition
k
Lk(A) = -co U(C - AE)-iK.
i=O
If A < 0, then we have
(C _ AE)-i = (( _ ) + IAI ( ))-t

210 9. Stabilization
= ( A2  l Y ((  -) + IAI ( ) Y
= ( A 2  1 Y  (  ) IAl i - j ( -  ) j
= ( A2l Y [(  (-l)P( 2) IAl i - 2P ) ( )
(9.23) + (  ( -l)P ( 2p  1 ) IAl i - 2 P-l) ( -)].
Denote e1 == (1,0), e2 == (0,1), li(A) == -(C - AE)-ie2. Observe that the
COlle Lk(A) is spanned by the convex hull of the points {lO(A),.. . , lk(A)}.
From (9.23) we obtain
(el,li(A)) = ( A2l Y (  (-l)P( 2P1 )IAl i - 2P - 1 ).
Obviously, for allY k there exists a A < 0 such that (e1, li(A)) > 0, i == 1, k,
whenever A < A. Hence Lk(A) i=- R 2 . Thus k(A) ---7 00 as A ---7 -00.
To find the Lyapllnov function we choose A < 0, set
E == co{ x == (1,0), xf == (0, 1), x == (-1, -I)},
and examine the compatibility of systelTI (9.1) for i == 0,1,2, and k i ==
1,2, . . .. For some k i the system turns out to be solvable. After normal-
ization (9.2) we obtain the vertices of the polyhedron which generates the
Lyapunov function.
9.5. Problems
1. Consider the control system
xl
x 2 ==
x 2 + u 1 ,
xl + u 2 ,
where u == (u 1 ,u 2 ) E U == {(u 1 ,u 2 ) I u 1 < 0, u 2 > O}. Show
that there is no smooth solution to the corresponding stabilization
problem; that is, there is no slTIooth function
u == u(x) == (u1(x1,x2),u2(x1,x2))

9.5. Problems
211
such that u(O) == 0, and the equilibrium position xl == 0, x 2 == 0 of
the system
Xl == x 2 + u 1 (xl, x 2 ),
x 2 == xl + u 2 (xl, x 2 )
is asymptotically stable.
2. Consider the control system
N
X = f(x, u) = fo(x) + L u q fq(x)
q==l
with u == (u 1 , . . . , uN) E U, where U c R N is convex. Assume that
the stabilization problem for the first approximation of this system is
solvable. Prove that there exists a Lipschitzian function u : R n  U
with u(O) == 0 and such that the zero equilibrium position of the
differential equation x == f (x, u( x)) is asymptotically stable. (Hint.
Use the result of Problem 2.1.)
3. Assume that the stabilization problem for the inclusion
x E \1 xf(O, uo)x + cone(f(O, U) - f(O, uo))
is solvable. Prove that there exists a Lipschitzian stabilizer. (Hint.
Use the previous problem.)
4. Suppose that the function (x, u)  f(x, u) is differentiable, that U is
a convex set, and that the stabilization problem for the first approx-
imation of the system x == f(x,uo) + \1uf(x,uo)(u - uo), u E U, is
solvable. Show that there exists a Lipschitzian stabilizer u( x) that
solves the stabilization problem for the system x == f(x, u), u E U.
5. Assume that the linear system
x == Cx + u, u E U,
where U c R n is a convex compact, is locally O-controllable. Let
T > 0 and 'TJ > 0 be such that 'TJBn C R[O,T] ( -Cx - U, 0). Show that
if A < -max{IC/,'TJ-1(2+ ICI)2 exp (2ICIT)} and k > T/AI, then
Lk(>") = -  (c - >"E)-iK = -  (E + 1>"1-1C)-i (1Jo aU ) = R n ,
(Hint. Apply the result of Problem 4.8 to the system
( -1 ) -1 1
Xi+l = E + 1>"1 C Xi - u, u E U, i = 0, k - 1.)
6. Prove that for the control system
x == Cx + Bw, w E R k ,

212
9. Stabilization
to be stabilizable it is necessary and sufficie11t that there exists a
matrix D such that the zero equilibrium position of the systelTI
x == (C + BD)x
is asymptotically stable.
7. Denote by A(8) the subspace of all points Xo E R n such that the
solution x(.) to the differential equation
x == Cx
with the initial condition x(O) == Xo satisfies x[x(-)] > -8. Consider
the adjoint linear differential equation
x* == -C*x*.
Denote by A + (8) the subspace such that the solution x* (.) to the
adjoint equation with an initial condition x*(O) == x* E A+(8) satisfies
x[x*(.)] > 8. Prove that A1-(8) == A+(8).
8. Let k be a positive integer. S110w that for any Xo E Lk(A) there exists
a trajectory x(.) of the differential inclusion x E Cx + K satisfying
x(O) == Xo and x[x(.)] > -A.
9. Denote by P(8) the set of all points Xo E R n such that control systelTI
x == Cx + w, w E K, where KeRn is a closed convex cone, has a
trajectory x(.) satisfying x(O) == Xo and x[x(.)] > -8. Let Q(8) be the
set of all points Xo E Rn such that the adjoint differential equation
x* == -C*x* has a solution x*(-) with x*(O) == Xo and x[x*(-)] > 8
satisfying x*(t) E K*, t > O. Prove that the convex cones P(8) and
Q(8) satisfy the equality Q(8) == -P*(8) for all 8.
10. Show that for all 8 the equality clP(8) == cl (A(8) + co UA<8 Loo(A))
holds.
II. Let £ be the cone of initial points Xo E R n such that there exists
a trajectory x(.) of the system x == Cx + w, w E K, where K c
Rn is a closed convex cone, with x(O) == Xo satisfying x(t)  0 as
t  00. Denote by AD the subspace such that any solution x(.) to
the differential equation x == Cx with x(O) E A D has zero Lyapunov
exponent. Let A D == {O}. Prove that the equality cl P(O) == cl £ holds.

Comments
In these bibliographical comments we do not review all works 011 differential
inclusions. We mention only books devoted to t11e theory of differential
inclusions or its applications, fundamental papers that are i11teresth1g from
the historical point of view and show the development of the theory, and
SOlne papers closely connected with the material of the text. The works are
cited in chronological order.
Introduction. The theory of differe11tial inclusions appeared in 30's when
lVlarchaud [77] and Zarelnba [128] proved the first existence theorems. How-
ever at that time there were no applications of the results obtained, and
mathelnaticians did not pay much attention to the subject. At the end of
50's optimal control theory was developed (see Belhnan [16], Pontryagin,
Boltyanski, Galnkrelidze, and Mischenko [95]). It was a great impetus for
differe11tial inclusions theory. Filippov [41] and Wazewski [125] established
the correspondence between c011trol systems and differential inclusions and
derived existence results for optimal c011trol problems. Another motivation
caIne froln the theory of differential equations with discontinuous right-hand
side. Filippov [42] suggested to define solutions of such systems as solutions
to differential inclusions. Since the early 60's the theory of differential in-
clusions can be considered as an independent branch of mathematics with
its own problems, methods, and applications.
There exist Inany monographs devoted to various aspects of differential
inclusions theory. In Aubin and Cellina [9] the emphasis is given to selection
problems, existence theorems, and viability theory. The theory of differential
equations with discontinuous right-hand side is considered in Filippov [45].
In Tolstonogov [119] differential inclusions in Banach space are studied.
-
213

214
Comments
The book of Deimling [40] contains Inany results concerning the existence
of solutions and qualitative properties of solution sets.
The discussion of the role of differential inclusions in ecoll0mics, sociol-
ogy, and biology can be found in Aubin [8], where a complete presentation
of viability theory is given. Extremal problems for differential inclusions
are considered in Pshenichnyi [98], Clarke [34], Blagodatskikh and Filip-
pov [18], Mordukhovich [83], and Kisielewicz [63]. Differential inclusions
are applied to differential games theory in Krasovski and Subbotin [66] and
Aubin [8], and to the theory of control under conditions of uncertainty in
Kurzhanski [69]. Krasnoselski and Pokrovski [65] used differential inclu-
sions to describe systems with hysteresis. Applications in Inechanics are
considered in Monteiro Marques [80].
More complete information on differential inclusions can be obtained
from the bibliography contained in the mentioned books.
Chapter 1. 1'here are many monographs on convex analysis. We mention
only books by Rockafellar [99] and Pshenichnyi [98]. The solutions of the
problelns to this chapter also can be found there.
Chapter 2. The theory of set-valued maps is considered in Castaing and
Valadier [28], Pshenichnyi [98], Aubin and Cellina [9], Aubin and Ekeland
[11], Aubin and Frankowska [12]. The concept of directional derivative for
set-valued maps was introduced by Aubin [7]. The approximation theorem
from Section 3 belongs to Haddad and Lasri [55]. The first extension theo-
rem for set-valued maps was proved by Cellina [29]. The decomposition of
the compliment of a closed set into cubes used in Section 4 first appeared
in Whitney [126]. Elementary proofs of the Brouwer fixed point theorem
can be found in [15], [120]. The notion of a a-selectionable Inap appeared
in Haddad and Lasri [55]. Many other results on the selection problem can
be found in Aubin and Cellina [9].
Convex processes were introduced by Rockafellar [99]. Other set-valued
versions of Perron's theorem can be found in Aubin and Ekeland [11] and
Aubin, Frankowska, and Olech [13]. The structure of convex processes was
studied in Aubin, Frankowska, and Olech [13] and Smirnov [106].
Problems 4 and 5 are from Pontryagin [93].
Chapter 3. Nonsmooth calculus appeared at the end of the 60's. The first
concepts are described in Joffe and Tikhomirov [60] and Pshenichnyi [98],
for example. A very successful version of nonsmooth analysis was developed
by Clarke [32]. A complete presentation of this approach is given in Clarke
[ 34 ] .

Comments
215
Here we consider the version of nonsmootIl allalysis developed by Mor-
dukhovich [81], [82], [83], [84], [87]. The separation theorem for nonconvex
sets was proved by Kruger and Mordukhovich [68].
Related nonsmooth calculus appeared in Ioffe [57] and Rockafellar [101],
Rockafellar and Wets [103]. An alterllative approach can be found in Suss-
lnann [116].
Problems 1, 2, and 6-10 are from Mordukhovich [83]. (Many of them
are simplified versions.) The concept of tent was introduced by Boltyanski.
Problems 3 and 4 go back to Rockafellar [100] and Watkins [124].
Chapter 4. The facts from tIle theory of functions and functiollal allaly-
sis gathered in Section 1 are proved in the text book by Kolmogorov alld
Fomin [64], for exalnple. The first version of the existellce theorem com-
bined with the Gronwall inequality was proved by Filippov [43]. The con-
tinuous version of Filippov's result presellted here is taken from Polovinkin
and Slnirnov [92]. Continuous versions of the Filippov theorenl for differ-
ential inclusions with nonconvex-valued right-hand side and some related
results are obtained in Polovinkin and Smirnov [92], Cellina [30], Ornelas
[89], Colombo, Fryszkowski, Rzezuchowski, and Staicu [37], Fryszkowski
and Rzezuchowski [52], Staicu and Wu [113], alld Snlirnov [108].
The nlost general existence result for differential inclusions with convex-
valued right-hand sides was obtai1led by Davy [38]. In the llOllconvex case it
is much more difficult to prove existence theorelns. The first existence the-
orems in the nonconvex case were proved by Filippov [43] for Lipschitzian
differential inclusions. Then ill Filippov [44] the existence result for dif-
ferential inclusions with continuous right-Iland sides was obtained. A very
general existellce theoreln which covers the results of Davy [38] and Filippov
[44] was proved by Olech [88]. For differential inclusions with llonconvex
lower semi-continuous right-hand sides, existence theorelTIS were proved by
Bressall [20] and Lojasiewicz [74]. A general existence result in Banach
spaces was obtained by Bressan and Cololnbo [26].
Some uniqueness results are cOlltained in Aubill and Cellina [9] and
Cellina [31].
Qualitative properties of solutioll sets to differential inclusions with con-
vex-valued right-hand sides, were considered by Davy [38] and the non-
convex case was studied by Bressan [21], [22], [23], [24], and De Blasi and
Pialligiani [39]. Qualitative properties for Lipschitzian differential inclusions
were largely studied in Bressan, Cellina, and Fryszkowski [25] alld Smirnov
[108] .
The exalnples of discontinuous differential equations are taken from An-
drollov, Vitt, and Khaikin [1].

216
COlnments
Many results on the existence of optimal trajectories are gathered in
Kisielewicz [63].
The results on the differentiability with respect to initial conditions and
estimates of tangent cones to solution sets appeared in Frankowska [47],
Polovinkin and Slnirnov [91], and Frankowska and Kaskosz [50].
The approximation result of Section 7 is taken froln Slnirnov [107].
SOlne other approximation theorems can be found in Pshenichnyi [98], Mor-
dukhovich [83], and Wolenski [127].
Problem 2 is a special case of a lnore general result froln Filippov [43].
Problelns 4 and 5 are froln Galnkrelidze [53]. Problem 6 in the fralne of
optilnal control theory was considered by Susslnann [114]. Probleln 8 is
froln Bushenkov and Smirnov [27].
Chapter 5. Different results concerning monotone solutions were obtained
by Roxin [105], Krbec [67], Aubin, Cellina, and Nohel [10], and Aubin
[7] (see also [9]). The viability theorem was proved in Haddad [54]. The
material of Section 3 is taken from Clarke [32] and from Aubil1 and Cellina
[9]. The existel1ce theorem for periodic solutions was obtained by Haddad
and Lasri [55]. Viability theory is presented in Aubin and Cellina [9] and
Aubin [8].
There are many monographs on the theory of differential games. We
lnention only the books by Friedman [51] and Krasovski and Subbotin [66].
The approach presented here is due to Pontryagin [93], [94] and Pontryagin
and lVlischenko [96].
Probleln 3 is due to Krasnoselski. Problem 4 is from Pontryagin [93].
Problem 5 is a shnplified version of more general results from Pontryagin
and Mischenko [96] (see also Friedlnan [51]).
Chapter 6. Controllability of convex processes was studied in Aubin, Fran-
kowska, and Olech [13]. The result on continuous differentiability on initial
conditions is taken froln [92]. Controllability at first approximation was con-
sidered in Blagodatskikh [17], Frankowska [46], and Polovinkin and Slnirnov
[92]. Dual forms of controllability conditions can be found, for example, in
Clarke [34], Polovinkin and Smirnov [92], Mordukhovich [83], Frankowska
and Kaskosz [50], Sussmann [115], [117], and Than [121]. Controllability
at high order approximation is considered in Frankowska [49] and Sussmann
[115], [11 7], [118].
Problems 1 and 2 are from Aubin, Frankowska, and Olech [13]. The
techniques needed to solve Problem 5 can be found in Ioffe and Tikhomirov
[60]. Problems 6 and 7 are from Polovinkin and Smirnov [91]. Problem 8
is a special case of a more general result from Polovinkin and Smirnov [92].

COlnments
217
Chapter 7. First results on necessary conditions of optil1lality for differen-
tial inclusions were obtained by Boltyanski [19], Clarke [33], and Pshenich-
nyi [97] (see also Pshenichnyi [98] and Clarke [34]). The lllethod of discrete
approxinlations goes back to Halkin [56], Pshenichnyi [97], [98], and 1\1101'-
dukhovich [82], [83], [85], [86]. The necessary conditions proved here are
taken frolll Slllirnov [107]. SOl1le other approaches to tIle problel1l are con-
sidered in Frankowska [48], Kaskosz and Lojasiewicz [61], [62], Lojasiewicz,
Plis, and Suarez [75], Polovinkin and Sl1lirnov [91], [92], Aseev [5], [6], Ioffe
[58], Ioffe and Rockafellar [59], Loewen and Rockafellar [71], [72], Loewen
and Vinter [73], Pinho and Vinter [90], Rockafellar [102], Rowland and
Vinter [104], Sussmann [115], [117], [118], and Vinter and Zheng [122].
The problem with state constraints is considered in Arutyunov and Aseev
[2], Arutyunov, Aseev, and Blagodatskikh [3], and Vinter and Zlleng [123].
The model of hydroplane considered in Section 3 is a lllodification of a
model from Andronov, Vitt, and Khaikin [1].
The lllechanical model used in Problelll 4 is due to Clarke [34]. Problelll
6 is frolll Polovinkin and Smirnov [92]. Problelll 8 is frolll Pshenichnyi [98].
Chapter 8. The concepts of stability for differential inclusions were intro-
duced in Roxin [105]. Stability of an equilibrium position is cOlllprehen-
sively studied in Filippov [45]. Lyapunov functions are used to investigate
the weak stability problem in Roxin [105], Krbec [67], Aubin [7], Aubin and
Cellina [9], and Aubin [8]. The construction of a quadratic Lyapunov func-
tion for a linear asymptotically stable systelll appeared in Lyapunov [76].
For the general case, converse Lyapunov theorellls are proved in Arzarello
and Bacciotti [4], Clarke, Ledyaev, and Stern [36], and Lin, Sontag, and
Wang [70]. ASYlllptotic stability for linear-selectionable differential inclu-
sions was studied by Molchanov and Pyatnitski [78], [79]. IV1any other
results on stability and aSYlllptotic stability can be found in Filippov [45].
The results on weak asymptotic stability of convex processes and weak as-
Ylllptotic stability at first approximation are taken frolll Slllirnov [106]. The
case of periodic differential inclusions is considered in Slllirnov [110].
Problellls 4-7 are from Smirnov [110].
Chapter 9. Literature on the stabilization problelll is reviewed in Bacciotti
[14] and Sontag [112]. The main results of this chapter are taken frolll
Smirnov [106]. Some other aspects concerning the approach considered here
are contained in Bushenkov and Slllirnov [27] and Snlirnov [111], [109].
Another point of view on the relation between weak aSYlllptotic stability
and stabilizability can be found in Clarke, Ledyaev, Sontag, and Sllbbotin
[35] .
All problems are from Bushenkov and Slllirnov [27].

Bibliography
[1] A.A. Andronov, A.A. Vitt, and S.E. Khaikin, Theory of Oscillators, Pergaillon Press,
Oxford, 1966.
[2] A.V. Arutyunov and S,rv1. Aseev, Investigation of the degeneracy phenomenon of
the maximum principle for optimal control problems with state constraints, SIAIvI J.
Contr. Opt., 35 (1997), 930-952.
[3] A.V. Arutyunov, S.Iv1. Aseev, and V.I. Blagodatskikh, First-order necessary condi-
tions in the problem of optimal control of a differential inclusion with phase con-
straints, Russ. Acad. Sci, Sb, Ivlath., 79 (1994), 3-32.
[4] E. Arzarello and A. Bacciotti, On stability and boundedness for Lipschitzian dif-
ferential inclusions: The converse of Lyapunov's theorems, Set-Valued Analysis, 5
(1997), 377-390.
[5] S.lV1. Aseev, Smooth approximations of differential inclusions and the time-optimality
problem, 1\'udy Ivlat. Inst. Steklov, 200 (1991), 27-34 (in Russian).
[6] S,rvI. Aseev, A method of smooth approximations in the theory of necessary optimality
conditions for differential inclusions, Izv. Ivlath., 61 (1997), 235-258.
[7] J .-P. Aubin, Contingent derivatives of set-valued maps and existence of solutions to
nonlinear inclusions and differential inclusions, Advances in Ivlath. Suppl. Studies,
Acad. Press, N. Y., 1981, pp. 160-232.
[8] J.-P. Aubin, Viability Theory, Birkhauser, Boston, 1991.
[9] J.-P. Aubin and A. Cellina, Differential Inclusions, Springer-Verlag, 1984.
[10] J.-P. Aubin, A. Cellina, and J. Nohel, Monotone trajectories of multivalued dynam-
ical systems, Annali di Matem. pura e applic., 115 (1977), 99-117.
[11] J.-P. Aubin and I. Ekeland, Applied Nonlinear Analysis, vViley, N. Y., 1984.
[12] J.-P. Aubin and H. Frankowska, Set-valued Analysis, Berlin, Birkhauser, 1990.
[13] J .-P. Aubin, H. Frankowska, and C. Olech, Controllability of convex processes, SIAIvI
J. Control, 24 (1986), 1192-1211.
[14] A. Bacciotti, Local stabilizability of nonlinear control systems, World Scientific, 1992.
[15] L. Baez-Duarte, Brouwer's fixed point theorem and a generalization of the formula
for change of variables in multiple integrals, J. Ivlath. Anal. Appl., 177 (1993), 412-
414.
-
219

220 Bibliography
[16] R. Belhnan, Dynamic Programming, Princeton Univ. Press, 1957.
[17] V.I. Blagodatskikh, Local controllability of differential inclusions, Diff. Equations, 9
(1973).
[18] V.I. Blagodatskikh and A.F. Filippov, Differential inclusions and optimal control,
Proc. Steklov Inst. ivlath., 169 (1986), 199-259.
[19] V.G. Boltyanski, The 'method of local cross sections in the theory of optimal processes,
Diff. Equations, 4 (1968).
[20] A. Bressan, On differential relations with lower semicontinuous right-hand side, J.
Diff. Equat., 37 (1980), 89-97.
[21] A. Bressan, Directionally continuous selections and differential inclusions, FunIc
Ekvac., 31 (1988), 459-470.
[22] A. Bressan, On the qualitative theory of lower semicontinuous differential inclusions,
J. Diff. Equat., 77 (1989), 379-391.
[23] A. Bressan, Upper and lower semicontinuous differential inclusions: a unified ap-
proach, in Nonlinear Controllability and Optinlal Control, Ed. H.J. Sussillann, Ivlar-
cel Dekker, Inc., Ne\\T York, 1990.
[24] A. Bressan, On the set of trajectories for a non-convex differential inclusion, Bull.
Pol. Acad. Sci., 1'.tJath., 39 (1991), 111-115.
[25] A. Bressan, A. Cellina, and A. Fryszkowski, A class of absolute retracts in spaces of
integrable functions, Proc. Anler. 1'.tJath. Soc., 119 (1991), n. 2.
[26] A. Bressan and G. Cololnbo, Generalized Baire category and differential inclusions
in Banach spaces, J. Diff. Equat., 76 (1988), 135-158.
[27] \!.A. Bushenkov and G.\!. Sll1irnov, Stabilization Problems with Constraints: Anal-
ysis and Computational Aspects, Gordon and Breach, 1997.
[28] C. Castaing and 1'./1. Valadier, Convex Analysis and lvleasurable lVlultifunctions, Lect.
Notes in 1'.tJath., n. 580, Springer-Verlag, 1977.
[29] A. Cellina., A theorem on the approximation of compact multivalued mappings, Rend.
Acc. Naz. Lincei, 47 (1969), 429-433.
[30] A. Cellina, On the set of solutions to Lipschitzian differential inclusions, Diff. and
Int. Equat., 1 (1988), 459-500.
[31] A. Cellina, On uniqueness almost everywhere for monotonic differential inclusions,
Nonlin. Anal. Theory 1'./Ieth. Appl., 25 (1995), 899-903.
[32] F.H. Clarke, Generalized gradients and applications, Trans. Alner. 1'.tJath. Soc., 205
(1975), 247-262.
[33] F.H. Clarke, Necessary conditions for a general control problem, in Calculus of Vari-
ations and Control Theory, N. Y., 1976, pp. 257-278.
[34] F.H. Clarke, Optimization and Nonsmooth Analysis, \i\Tiley-Interscience, N. Y., 1983.
[35] F.H. Clarke, Y.S. Ledyaev, E.D. Sontag, and A.I. Subbotin, Asymptotic controlla-
bility implies feedback stabilization, IEEE Trans. AUtOlll. Control, 42 (1997), 1394-
1407.
[36] F.H. Clarke, Y.S. Ledyaev, and R.J. Stern, Asyrnptotic stability and smooth Lya-
punov functions, J. Diff. Equat., 149 (1998), 66-114.
[37] R.1'.tJ. Cololnbo, A. Fryszkowski, T. Rzezuchowski, and V. Staicu, Continuous selec-
tions of solution sets of Lipschitzian differential inclusions, Funkcialaj Ekvacioj, 34
(1991), n. 2.
[38] J .L. Davy, Properties of solution set of a generalized differential equations, Bull.
Austr. 1'./Iath. Soc., 6 (1972), 372-398.

Bibliography
221
[39] F.S. De Blasi and G. Pianigiani, On the solution sets of nonconvex differential in-
clusions, J. Diff. Equat., 128 (1996), 541-555.
[40] K. Deilllling, Nlultivalued Differential Equations, de Gruyter, Berlin, 1992.
[41] A.F. Filippov, On some questions in the theory of optimal regulation, Vestn. j\/Ioslc
Univ. Ser. j\/Iat. j\/Iekh. Astr. Fiz. Khiln., (1959), 25-32 (translation in J. SIAj\/I
Control, Ser A, 1 (1962), 76-84).
[42] A.F. Filippov, Differential equations with discontinuous right-hand side, j\/Iat.
Sborn., 51 (1960), 99-128 (in Russian).
[43] A.F. Filippov, Classical solutions of differential equations with multivalued right-
hand side, SIAj\/I J. Control, 5 (1967), 609-621.
[44] A.F. Filippov, On the existence of solutions of multivalued differential equations,
Mat. Zalnetki, 10 (1971), 307-313 (in Russian).
[45] A.F. Filippov, Differential Equations with Discontinuous Right-Hand Sides, I(hnver
Acad. Publishers, 1988.
[46] H. Frankowska, Controlabilite locale et proprietes des semigroupes de correspon-
dances, C.R.A.S., 299 (1983), Ser. 1, 165-168.
[47] H. Frankowska, Inclusions adjointes associees aux trajectoTies minimales
d'inclusions differentielles, C.R.A.S., 297 (1983), Ser. 1, 461-464.
[48] H. Frankowska, The maximum principle for a differential inclusion problem, Lect.
Notes ContI'. Inf. Sci., 62 (1984), pp. 517-531.
[49] H. Frankowska, Local controllability of control systems with feedback, J. Optiln. The-
ory Appl., 60 (1989), 277-296.
[50] H. Frankowska and B. Kaskosz, Linearization and boundary trajectories of non-
smooth control systems, Canadian J. j\/Iath., XL (1988), 589-609.
[51] A. Friedlnan, Differential Games, vViley-Intersci., New York, 1971.
[52] A. Fryszkowski and T. Rzezucho\vski, Continuous version of Filippov- Wazewski re-
laxation theorem, J. Diff. Equat., 94 (1991), 254-265.
[53] R.V. Galnkrelidze, Principles of Optimal Control Theory, PienU111 Press, Ne",' York,
1978.
[54] G. Haddad, Monotone trajectories of differential inclusions and functional differen-
tial inclusions with memory, Israel J. j\/Iath., 39 (1981), 83-100.
[55] G. Haddad and J.-j\/1. Lasri, Periodic solutions of functional differential inclusions
and fixed points of a-selectionable correspondences, J. j\/Iath. Anal. Appl., 96 (1983),
295-312.
[56] H. Halkin, Extremal properties of biconvex contingent equations, in Ordinary Differ-
ential Equations, L. Weiss, Ed., Acadelnic Press, N. Y., 1972, pp. 109-119.
[57] A.D. Ioffe, Approximate subdifferentials and applications, I-III, Trans. A111er. Ivlath.
Soc., 281 (1984); IvIathe111atika, 33 (1986), 111-128; 36 (1989), 1-38.
[58] A.D. Ioffe, Euler-Lagrange and Hamiltonian formalisms for dynamic optimization,
Trans. Ailler. j\/Iath. Soci., 349 (1997), 2871-2900.
[59] A.D. Ioffe and R.T. Rockafellar, The Euler and Weierstrass conditions for non-
smooth variational problems, Calc. Var. Partial Diff. Equat., 4 (1996), 59-87.
[60] A.D. Ioffe and V.j\/1. Tikho111irov, The Theory of Extremal Problems, North-Holland,
Amsterda111, 1979.
[61] B. Kaskosz and S. Lojasiewicz, A maximum principle for generalized control systems,
Nonlin. Anal., 9 (1985), 109-130.

222 Bibliography
[62] B. Kaskosz and S. Lojasiewicz, Lagrange-type extremal trajectories in differential
inclusions, Syst. Control Lett., 19 (1992), 241-247.
[63] Iv1. Kisielewicz, Differential Inclusions and Optimal Control, I(luwer Acad. Publish-
ers, 1991.
[64] A.N. Kohnogorov and S.V. Fonlin, Introductory Real Analysis, Dover, N. Y., 1975.
[65] IvI.A. Krasnoselski and A.V. Pokrovski, Systems with Hysteresis, Springer-Verlag,
1989.
[66] N.N. Krasovski and A.I. Subbotin, Game- Theoretical Control Problems, Springer-
Verlag, 1987.
[67] P. Krbec, Weak stability of multivalued differential equations, Czechosl. Ivlath.
Journ., 26 (1976), 470-476.
[68] A.Y. Kruger and B.S. J\lIordukhovich, Extremal points and Euler equations in non-
smooth optimization, Dokl. Akad. Nauk BSSR, 1980, n. 8, 684-687 (in Russian).
[69] A.B. Kurzhanski, Control and Observation under Conditions of Uncertainty,
Ivloskva, Nauka, 1977 (in Russian).
[70] yr. Lin, E.D. Sontag, and Y. Wang, A smooth converse Lyapunov theorem for robust
stability, SIAJ\lI J. Control Opt., 34 (1996), 124-160.
[71] P.D. Loewen and R.T. Rockafellar, Optimal control of unbounded differential inclu-
sions, SIAJ\lI J. Control Opt., 32 (1994),442-470.
[72] P.D. Loewen and R.T. Rockafellar, New necessary conditions for the generalized
problem of Bolza, SIAJ\lI J. Control Opt., 34 (1996), 1496-1511.
[73] P.D. Loewen and R.B. Vinter, Pontryagin type necessary conditions for differential
inclusion problems, Syst. Control Lett., 9 (1987), 263-265.
[74] S. Lojasiewicz, The existence of solutions for lower semicontinuous orientor fields,
Bull. Acad. Polon. Sci. Ser. Sci. J\lIath., 28 (1980), 483-487.
[75] S. Lojasiewicz, A. Plis, and R. Suarez, Necessary conditions for nonlinear control
system, J. Diff. Equat. 59 (1985), n. 2.
[76] A.J\lI. Lyapunov, Probleme General de La Stabilite du Mouvement, Annals of Math-
eillatics Studies, no. 17, Princeton Univ. Press, Princeton, N.J., and Oxford Univ.
Press, London, 1947.
[77] A. Ivlarchaud, Sur les champs continus de demi-cones convex et leurs integrales,
COlnp. J\lIath., 3 (1936), 89-127.
[78] A.P. Ivlolchanov and E.S. Pyatnitski, Stability criteria for selector-linear differential
inclusions, Soviet Math. Dokl., 36 (1988), 421-424.
[79] A.P. Ivlolchanov and E.S. Pyatnitski, Criteria of asymptotic stability of differential
and difference inclusions encountered in control theory, System Control Lett., 13
(1989), 59-64.
[80] M.D.P. J\lIonteiro Ivlarques, Differential Inclusions in Nonsmooth Mechanical Prob-
lems, Birkhauser, 1993.
[81] B.S. IVlordukhovich, lvlaximum principle in the problem of time optimal control with
nonsmooth constraints, J. Appl. Ivlath. Ivlech., 40 (1976),960-969.
[82] B.S. J\lIordukhovich, Metric approximations and necessary conditions for optimality
for general classes of nonsmooth extremal problems, English translation in Soviet
Ivlath. Dokl., 22 (1980), 526-530.
[83] B.S. J\lIordukhovich, Approximation Methods in Problems of Optimization and Con-
trol, Nauka, Ivloskva, 1988 (in Russian, 2nd English edition to appear in Wiley-
Interscience) .

Bibliography
223
[84] B.S. Ivlordukhovich, Generalized differential calculus for nonsmooth and set-valued
mappings, J. Ivlath. Anal. Appl., 183 (1994), 250-288.
[85] B.S. Ivlordukhovich, Discrete approximations and refined Euler-Lagrange conditions
for nonconvex differential inclusions, SIArvI J. Control Optilniz., 33 (1995), 882-915.
[86] B.S. rvlordukhovich, Optimization and finite difference approximations of nonconvex
differential inclusions with free time, in Proceedings of nvIA Workshop on Non-
Sillooth Analysis and Geometric Ivlethods in Detenninistic Optinlal Control, rvIin-
neapolis, Feb. 1993, B.S. Ivlordukhovich and H.J. Sussillann, Eds., IrvIA Vohllnes in
rvlath. and its Applications 78, Springer-Verlag, N. Y., 1996, pp. 153-202.
[87] B.S. Ivlordukhovich and Y. Shao, Nonsmooth sequential analysis in Asplund space,
1\'ans. Anler. Ivlath. Soc., 348 (1996), 1235-1280.
[88] C. Olech, Existence of solutions of non-convex orientor fields, Boll. Un. rvlat. It.,
(3) 11, suppl., (1975), 189-197.
[89] A. Ornelas, A continuous version of Filippov- Gronwall inequality for differential
inclusions, Atti. Accad. Naz. Lincei Cl. Sci. Fis. rvlat. Natur. Rend. rvlat. Appl., 1
(1990), 105-110.
[90] M.R. Pinho, and R.B. Vinter, An Euler-Lagrange inclusion for optimal control prob-
lems, IEEE 1\'ans. Autom. Control, 40 (1995), 1191-1198.
[91] E.S. Polovinkin and G.V. Smirnov, An approach to differentiation of set-valued maps
and necessary conditions of optimality for differential inclusions, Diff. Equations, 22
(1986), 660-668.
[92] E.S. Polovinkin and G.V. Smirnov, Time-optimal problem for differential inclusions,
Diff. Equations, 22 (1986), 940-952.
[93] L.S. Pontryagin, Linear differential games of pursuit, Ivlath. USSR Sb. 40 (1981).
[94] L.S. Pontryagin, Linear differential games, Proc. Steklov Inst. rVlath., 185 (1990),
225- 232.
[95] L.S. Pontryagin, V.G. Boltyanski, R.V. Galllkrelidze, and E.F. IvIischenko, The
Mathematical Theory of Optimal Processes, Wiley-Interscience, New York, 1962.
[96] L.S. Pontryagin and A.S. rVIischenko, Linear differential games (analytic theory on
the basis of alternating integration), Proc. Steklov Inst. Math., 185 (1990), 233-240.
[97] B.N. Pshenichnyi, Necessary conditions of extremum for differential inclusions, Cy-
bernetics, 12 (1976).
[98] B.N. Pshenichnyi, Convex analysis and extremal problems, Nauka, Ivloskva, 1980 (in
Russian).
[99] R.T. Rockafellar, Convex Analysis, Princeton Univ. Press, 1970.
[100] R.T. Rockafellar, Clarke's tangent cones and the boundaries of closed sets in R n ,
Nonlin. Anal., 3 (1979), 145-154.
[101] R.T. Rockafellar, Extension of subgradient calculus with applications to optimization,
Nonlin. Anal., 9 (1985), 665-698.
[102] R.T. Rockafellar, Equivalent subgradient version of Hamiltonian and Euler-Lagrange
equations in variational analysis, SIAIvI J. Control Opt., 34 (1996), 1300-1314.
[103] R.T. Rockafellar and R.J.-B. Wets, Variational Analysis, Springer, 1998.
[104] J.D.L. Rowland and R.B. Vinter, Pontryagin type conditions for differential inclu-
sions with free time, J. Ivlath. Anal. Appl., 165 (1992), 587-597.
[105] E. Roxin, Stability in general control systems, J. Diff. Equat., 1 (1965), 115-150.
[106] G.V. Smirnov, Weak asymptotic stability for differential inclusions, I-II. Automat.
Reillote Control, 51 (1990), n. 7 part 1, 901-908; n. 8 part 1, 1052-1058.

224
Bibliography
[107] G.V. Sl11irnov, Discrete approximations and optimal solutions to differential inclu-
sions, Cybernetics, 27 (1991), 101-107.
[108] G.V. Sl11irnov, The topological properties of an integral funnel of a differential in-
clusion with Lipschitzian right-hand side, Diff. Equations, 27 (1991), 157-165.
[109] G.V. Sl11irnov, Unilateral stabilization at first approximation, J. Ivlath. Anal. Appl.,
192 (1995), 469-501.
[110] G.V. Sl11irnov, Weak asymptotic stability at first approximation for periodic differ-
ential inclusions, Nonlinear Diff. Equations, 2 (1995), 445-461.
[111] G.V. Sl11irnov, Stabilization by constrained controls, SIAM J. Control Opt., 34
(1996), 1616-1649.
[112] E.D. Sontag, Mathematical Control Theory. Deterministic Finite Dimensional Sys-
tems, Springer-Verlag, New York, 1990.
[113] V. Staicu and H. Wu, Arcwise connectedness of solution sets to Lipschitzian differ-
ential inclusions, Boll. Unione Ivlat. Ital. ses. A, V-A (1991), 253-257.
[114] H.J. Sussl11ann, From the brachystochrone to the maximum principle, in Proceedings
of the 35th IEEE Conference on Decision and Control, Kobe, Japan, Dec. 1996, IEEE
publications, 1996.
[115] H.J. Sussmann, A strong maximum principle for systems of differential inclusions,
in Proceedings of the 35th IEEE Conference on Decision and Control, Kobe, Japan,
Dec. 1996, IEEE publications, 1996, pp. 1809-1814.
[116] H.J. Sussmann, Multidifferential calculus: chain rule, open mapping and transversal
intersection theorems, in Opti111al Control: Theory, Algorithl11s, and Applications,
W.W. Hager and P.IvI. Pardalos, Eds., Kluwer Academic Publishers, 1997.
[117] H.J. Susslnann, Geometry and optimal control, in Mathematical Control Theory, J.
Baillieul and J.C. Willel11s, Eds., Springer-Verlag, 1998.
[118] H.J. Sussl11ann, Transversality conditions and a strong maximum principle for sys-
tems of differential inclusions, in Proceedings of the 37th IEEE Conference on De-
cision and Control, Tampa, FL, USA, Dec. 1998.
[119] A.A. Tolstonogov, Differential Inclusions in Banach Space, Nauka, Novosibirsk, 1987
(in Russian).
[120] T. Traynor, An easy analytic proof of Brouwer's fixed point theorem, Atti Sem. Ivlat.
Fios. Univ. Ivlodena, XLIV (1996), 479-483.
[121] H.D. Tuan, Controllability and extremality in nonconvex differential inclusions, J.
Optim. Theory Appl., 85 (1995), 435-472.
[122] R.B. Vinter and H. Zheng, The extended Euler-Lagrange condition for nonconvex
variational problems, SIAIvI J. Control Opt., 35 (1997), 56-77.
[123] R.B. Vinter and H. Zheng, Necessary conditions for optimal control problems with
state constraints, Trans. Anler. Math. Soc., 350 (1998), 1181-1204.
[124] G.G. Watkins, Clarke's tangent vectors as tangents to Lipschitz continuous curves
J. Opt. Theory Appl., 45 (1985).
[125] T. Wazewski, Systemes de commande et equations au contingent, Bull. Acad. Polon.
ScL Ser. ScL IVlath., 9 (1961), 151-155.
[126] H. Whitney, Analytic extensions of differentiable functions defined in closed sets,
Trans. Al11er. Ivlath. Soc., 36 (1934), 63-89.
[127] P.R. Wolenski, The exponential formula for the reachable set of a Lipschitz differen-
tial inclusion, SIAM J. Control, 28 (1990), 1148-1161.
[128] S.K. Zarel11ba, Sur les equations au paratingent, Bull. ScL Ivlath., 60 (1936), 139-160.

Index
Arzela-Ascoli theorem, 89
Asymptotic controllability, 172
Boundary, 4
linear, 133
Dini
derivative lower, 76
derivative upper, 76
lower subdifferential, 76
Directional derivative, 24
Closure, 4
Cone, 11
Clarke normal, 83
Clarke tangent, 83
conjugate, 11
contingent, 38
convex, 11
normal, 12
polar, 11
tangent, 11, 38
Control, 91
admissible, 91
problem, 108
system, 91
trajectory of, 91
Control process, 108
optimal, 108
Convex combination, 5
Convex hull, 5
Convex process, 49
bounded, 55
closed, 49
controllable, 145
eigenvalue of, 55
eigenvector of, 55
strict, 49
unstable, 189
weakly asymptotically stable, 185
Convex set, 4
Equilibrium position
asymptotically stable, 171
stable, 171
unstable, 172
weakly asymptotically stable, 171
weakly exponentially stable, 206
weakly stable, 171
Extreme point, 30
Fenchel's inequality, 19
Filippov solution, 105
First approximation, 110
Function
conjugate, 19
convex, 13
definite, 173
effective domain of, 13
epigraph of, 13
lower semi-continuous, 18
semi-definite, 173
upper semi-continuous, 18
Gronwall inequality, 90
Indicator function, 16
Interior, 4
Differential game, 133
Lagrange function, 66
Lebesgue theorem, 90
-
225

226
Index
Locally finite covering, 41
Lusin theorem, 36
Lyapunov direct method, 173
Lyapunov exponent, 172
Lyapunov function, 173
:tvlaximum principle, 164
Mayer problem, 106
Minko\vski function, 16
Ivlordukhovich
normal cone, 67
subdifferential, 75
Partition of unity, 41
Polar set, 30
Polytope, 11
Projection, 8
Reachability set, 95
Relative boundary, 7
Relative interior, 7
Selection, 35
Set-valued map, 31
contingent derivative of, 41
continuous, 32
derivative of, 41
domain of, 32
graph of, 31
image of, 32
linear-selectionable, 176
Lipschitzian, 32
locally, 32
lower semi-continuous, 32
restriction of, 32
a-selectionable, 48
upper semi-continuous, 32
Simplex, 11
Spectrum of a matrix, 1 76
Stabilization problem, 199
Stabilizer, 199
Strictly negative-dominant diagonal 183
Subdifferential, 25
singular, 76
Support function, 14
Tent, 83
Time-optimal problem, 106
Variational inclusion, 110
Vertex of a simplex, 11

Selected Titles in This Series


( Continued from the front of this publication)


2 Jack Graver, Brigitte Servatius, and Herman Servatius, Combinatorial rigidity,
1993
1 Ethan Akin, The general topology of dynamical systems, 1993