Malcolm Adams
Victor Guillemin
Measure Theory
and Probability
BlRKHAUSER
Boston • Basel • Berlin

Malcolm Adams
Department of Mathematics
University of Georgia
Athens, Georgia 30602
Victor Guillemin
Department of Mathematics
MIT
Cambridge, MA 02139
To Jon Bucsela in memoriam
Library of Congress Cataloging-in-Publication Data
Adams, Malcolm Ritchie.
Measure theory and probability / Malcom Adams, Victor Guillemin.
p. cm.
Originally published: Monterey Calif. : Wadsworth & Brooks/Cole
Advanced Books and Software, c 1986.
Includes bibliographical references and index.
ISBN 0-8176-3884-9 (he : alk. paper). - ISBN 3-7643-3884-9 (he :
alk. paper)
1. Measure theory. 2. Probabilities. I. Guillemin, V., 1937-
II. Title.
[QA273.A414 1995]
515\42~dc20 95-46511
CIP
Printed on acid-free paper
© 1996 Birkhauser Boston
Reprinted with corrections from the
1986 Wadsworth edition.
Birkhauser
)®
Copyright is not claimed for works of U.S. Government employees.
All rights reserved. No part of this publication may be reproduced, stored in a retrieval system,
or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording,
or otherwise, without prior permission of the copyright owner.
Permission to photocopy for internal or personal use of specific clients is granted by
Birkhauser Boston for libraries and other users registered with the Copyright Clearance
Center (CCC), provided that the base fee of $6.00 per copy, plus $0.20 per page is paid directly
to CCC, 222 Rosewood Drive, Danvers, MA 01923, U.S.A. Special requests should be
addressed directly to Birkhauser Boston, 675 Massachusetts Avenue, Cambridge, MA 02139,
U.S.A.
ISBN 0-8176-3884-9
ISBN 3-7643-3884-9
Typesetting by ASCO Trade Typesetting, Hong Kong
Printed and bound by Maple-Vail, York, PA
Printed in the U.S.A.
987654321

Preface to the 1996 Edition
We have used Measure Theory and Probability as our standard text in
the basic measure theory courses at M.I.T. and the University of Georgia for over
ten years and have been agreeably surprised at the enthusiasm with which students
have reacted to the 5-3 mix of measure theory and probability. It is not clear that
we've converted lots of aspiring mathematicians into probabilists, but we do seem
to have left the non-mathematicians, our students from electrical engineering and
computer science, feeling upbeat about the Lebesgue theory and its practical uses.
On die down side, our students have been annoyed at the plethora of typos and
silly mistakes in the first edition. (For instance the absence of a superscript bar on
the right hand side of identity
(v, w) = (w, v)
made a travesty of our definition of Hilbert space!) This edition has been ridded
of these errors thanks to the efforts of the editorial staff at Birkhauser, our former
students, Leonard Shulman, Roy Yates, Gregg Wornell, and Mastafa Terab, and,
in particular, thanks to Professor Bo Green and a diligent group of his students at
Abilene Christian University (who went through the book with a fine tooth comb
and assembled a pretty definitive list of errata). To them our warmest thanks.
vii

Preface to the First Edition
Probability theory became a respectable mathematical discipline only
in the early 1930s. Prior to that time it was viewed with scepticism by some
mathematicians because it dealt with concepts such as random variables and
independence, which were not precisely and rigorously defined. This situation
was remedied in the early 1930s largely thanks to the efforts of Andrei Kolmo-
gorov and Norbert Wiener, who introduced into probability theory large infusions
of measure theory. In retrospect, it was fortunate that the kind of measure theory
they needed was already available; it had, in fact, been created some thirty years
earlier by Henri Lebesgue, who had not been led to the invention of Lebesgue
measure by problems in probability but by problems in harmonic analysis. It
seems strange that it took more than 30 years for this fusion of probability and
measure theory to occur. In fact, since that time, probability theory and measure
theory have become so intertwined that they seem to many mathematicians of
our generation to be two aspects of the same subject. It also seems strange that
the basic concepts of the Lebesgue theory, to which one is naturally led by
practical questions in probability, could have been arrived at without probability
theory as their main source of inspiration.
Saddled as we are with the fact that the theory of measure didn't develop
along these lines, this doesn't mean we cannot teach the subject as if it had
developed this way. Indeed, we believe (and this is the reason we wrote this
book) that the only way to teach measure theory to undergraduates is from the
perspective of probability theory. To teach measure theory and integration theory
without at the same time dwelling on its applications is indefensible. It is unfair
to ask undergraduates to learn a fairly technical subject for the sake of payoffs
they may see in the distant future. On the other hand, the applications of measure
theory to areas other than probability (e.g., harmonic analysis and dynamical
systems) are fairly esoteric and not within the scope of undergraduate courses.
Of course probability theory, taught in tandem with measure theory, is also not
thought of as being within the scope of an undergraduate course, but we feel this
ix

X
Preface
is a mistake. Discrete probability theory is taught at many institutions as a
freshman course (and at some high schools as a senior elective). The kinds of problems
we will be interested in here, i.e., the amorphous set of problems that go under
the rubric of the law of large numbers, are lurking in the background in these
discrete probability courses, and are often so bothersome to bright students that
they arrive, unaided, at quite original ideas about them. By formulating these
problems in measure theoretic language, one is often doing little more than
vindicating for undergraduates their own intuitive ideas and, at the same time of
course, convincing them that the measure theoretic methods are worth learning.
By now we have probably given you the impression that this book is basically
about probability. On the contrary, it is basically about measure theory. Sections
1.1 and 1.2 nominally discuss probability, but primarily discuss why measure
theory is needed for the formulation of problems in probability. (What we hope
to convey here is that had the Lebesgue theory of measure not existed, one would
be forced to invent it to contend with the paradoxes of large numbers.)
Section 1.3 deals with the construction of Lebesgue measure on Rn (following the
metric space approach popularized by Rudin [see References, p. 199]. In §1.4
we briefly revert to probability theory to draw some inferences from the Borel-
Cantelli lemmas, but §§2.1-2.5 are straight measure theory: the basic facts about
the Lebesgue integral. Only to illustrate these facts do we return to probability
at the end of the chapter and discuss expectation values, the law of large numbers,
and potential theory.
Sections 3.1-3.5 are also consecrated entirely to measure theory and
integration: £l9$£2, abstract Fourier analysis, Fourier series, and the Fourier integral.
Fortunately, the last two items have some beautiful probabilistic applications:
Polya's theorem on random walks, Kac's proof of the classical Szego theorem,
and the central limit theorem. With these we end the book. All told, taking into
account the fact that we have packed quite a few applications to probability into
the exercises, the ratio of measure theory to probability in the book is about 5
to 3.
The notes on which this book is based have served for several years as material
for a course on the Lebesgue integral at M.I.T. and for a similar course at
Berkeley. They have been the basis for a leisurely semester course and an
intensive quarter course and have proved satisfactory in both (though in using these
notes in a quarter course, we had to delete most of the material in §§2.6-2.8 and
§§3.6-3.8). We divided the book into the three chapters not just for aesthetic
reasons, but because we found that in teaching from these notes we were devoting
approximately the same amount of time to each of these three chapters, i.e., four
weeks in a typical twelve-week semester course. Incidentally, we found it very
effective, for motivational purposes, to devote the first three class periods of the
course to the material in §§1.1 and 1.2, even though in principle this material
could be covered in a much more cursory fashion. We discovered that with these
ideas in mind, the students were much better able to endure the long arid trek
through the basics of measure theory in §1.3.

Preface
xi
We would like to thank Marge Zabierek for typing the notes on which this
book is based, and we would like to thank our students at Berkeley and M.I.T.,
in particular, Tomasz Mrowka, Mike Dawson, Harold Naparst, Mike Conwill,
Christopher Silva, and Ken Ballou for suggestions about how to improve these
notes and for weeding out what seemed to have been an almost endless number
of errors from the problem sets.
We have dedicated this book to Jon Bucsela, to whom we owe an exhaustive
revision of the manuscript before we had the final version typed. His untimely
death in the spring of 1984 was a source of acute grief to all who knew him.
Malcolm Adams
Victor Guillemin

Suggestions for
Collateral Reading
For background in probability theory, we recommend Feller, An
Introduction to Probability Theory and Its Applications,* We feel that at the
undergraduate level, this is the best book ever written on probability theory. Its
charm resides in the fact that there are literally hundreds of illustrative examples.
This makes it hard to read through from cover to cover, but it is a gold mine of
ideas.
Another beautiful book, though more advanced than Feller, is Kac's 96-page
monograph in the Carus series, Statistical Independence in Probabilityy Analysis
and Number Theory, Our treatment of Bernoulli sequences and the law of large
numbers in §1.1 was largely borrowed from this book, and one can go there to
find further ramifications of these topics.
There are several treatments of measure theory in conjunction with probability
written for graduate students. In our opinion, the best of these is Billingsley's
book Probability and Measurey which a bright undergraduate will, with a little
effort, find accessible if he or she ignores the more technical sections toward
the end.
Finally, for material on metric spaces and compactness, we have attempted
to remedy the fact that we presuppose a nodding acquaintance with these topics
by summarizing the main facts in the appendix. To learn this material, however,
we recommend either Hoffman (Analysis in Euclidian Space) or Rudin
(Principles of Mathematical Analysis),
♦For complete bibliographic information for the titles listed here, see the Reference section on
page 202.
xii

Contents
Chapter 1
Measure Theory 1
§1.1 Introduction 1
§1.2 Randomness 14
§1.3 Measure Theory 24
§1.4 Measure Theoretic Modeling 42
Chapter 2
Integration 53
§2.1 Measurable Functions 53
§2.2 The Lebesgue Integral 60
§2.3 Further Properties of the Integral; Convergence Theorems 72
§2.4 Lebesgue Integration versus Riemann Integration 82
§2.5 Fubini Theorem 89
§2.6 Random Variables, Expectation Values, and Independence 102
§2.7 The Law of Large Numbers 110
§2.8 The Discrete Dirichlet Problem 115
Chapter 3
Fourier Analysis 118
§3.1 ^-Theory 118
§3.2 ^-Theory 124
§3.3 The Geometry of Hilbert Space 130
§3.4 Fourier Series 137
§3.5 The Fourier Integral 145
§3.6 Some Applications of Fourier Series to Probability Theory 156
§3.7 An Application of Probability Theory to Fourier Series 164
§3.8 The Central Limit Theorem 170
xiii

xiv
Contents
Appendix A Metric Spaces 178
Appendix B On $£p Matters 183
Appendix C A Non-Measurable Subset
of the Interval (0,1] 199
References 202
Index 203

Measure Theory
and Probability

Chapter 1
Measure
Theory
§1.1 Introduction
In this section we will talk about some of the mathematical
machinery that comes into play when one attempts to formulate precisely what
probabilists call the law of large numbers.
Consider a sequence of coin tosses. To represent such a sequence, let H
symbolize the occurrence of a head and T the occurrence of a tail. Then a
coin-tossing sequence is represented by a string of H's and T's, such as
HHTHTTTHHT...
Now, let Sff be the number of heads seen in the first N tosses. The law of large
numbers asserts that for a "typical" sequence we should see, in the long run,
about as many heads as tails. That is, we would like to say that
(1) lim£-i
for the "typical" sequence of coin tosses.
We do not expect this assertion to be true for all sequences, because
it is possible, for instance, for our sequence of coin tosses to be all heads.
Experience tells us, however, that such a sequence is not typical.
In what follows, we describe a mathematical model of coin tossing in which
we precisely define what is meant by a "typical" sequence of coin tosses. With
this model, the law of large numbers can be rigorously demonstrated.
Because James Bernoulli first stated the law of large numbers, in the
seventeenth century, we will call a sequence of coin tosses a Bernoulli sequence.
1

2
Chapter 1 Measure Theory
Let 0$ represent the collection of all possible Bernoulli sequences. Notice that
<% is an uncountable set. (See exercise 1.) This fact is also clear from the
following proposition.
Proposition 1. If we delete a countable subset from ^, we can index what is
left by points on the real interval / = (0,1].
Proof. We construct a map / -> 0$ that is one to one and fails to be onto by
a countable set. The map is constructed as follows.
Every coel can be written in the form
(2) *>=ZS «i = <U
Because the a^s determine co, we introduce the notation
co = . ax a2 #3...
which is called the binary expansion of co. From this representation we
produce a Bernoulli sequence by putting an H in the nth term of the sequence
if an = 1 or a T if an = 0. Unfortunately, this does not give a well-defined map
l-*@ because co does not necessarily have a unique binary expansion. For
example, co = \ has the two binary expansions
.1000... and .0111...
To avoid this problem we prescribe that, if co has a terminating and a
nonterminating expansion, we give it the nonterminating one.
This convention gives a one-to-one map / -* & that is not onto because it
misses out on those Bernoulli sequences that end in all tails. Let ^dcg denote
the collection of Bernoulli sequences that, after a certain point, degenerate to
all tails. We claim that ^dcg is countable.
Proof. Let ^Jeg be the Bernoulli sequences that have only tails after the
fcth toss. Then ^dcg *s finite and
^deg = Q ^Seg
fc=l
is a countable union of finite sets. Thus ^deg is countable. □
Because ^deg is a countable subset of the uncountable set ^, we consider
it to be negligible in our consideration of "typical" elements of ^. Thus, for
all intents and purposes, we can consider & to be identified with /.
In order to describe other features of our model of ^, we need some
familiarity with the idea of Lebesgue measure. We will not yet attempt to define
Lebesgue measure precisely, but we will describe some of the properties it

§1.1 Introduction
3
should have. We ask the reader to believe that it exists until we examine it
more rigorously.
A measure \i on a space AT is a nonnegative function defined on a prescribed
collection of subsets of X, the measurable sets. If A is a measurable set,
the nonnegative number /i(A) is called the measure of A. Of course we will
require that jx have certain properties so that it behaves as our intuition tells
us a measure should behave. For example, we will require additivity: If A and
B are measurable and disjoint, then A u B is measurable and fi(A u5) =
fi(A) + /x(B). This and other properties of measures will be discussed in §1.3.
The particular measure in which we are interested here is called Lebesgue
measure (denoted fxL) and is defined on certain subsets of the real line R. For
the intervals
(a,b), (a,*], [a,6], [«,&)
the Lebesgue measure is just the length, b — a. More generally, by the
property of additivity, if
i=l
is a finite disjoint union of finite intervals A,-, then A is Lebesgue measurable
and
i=l
Using the concept of Lebesgue measure, we can now formulate what we
will call the Borel principle.
Borel principle. Suppose £ is a probabilistic event occurring in certain
Bernoulli sequences. Let 0&E denote the subset of & for which the event occurs.
Let BE be the corresponding subset of /. Then the probability that E occurs,
Prob(E), is equal to fJ.L{BE).
Let us show that this principle works for some simple probabilistic events.
1. E is the event that H appears on the first toss.
BE = {fl)6/;fl) = .l...}=(i,l]
so fiL(BE) = \.
2. E is the event that the first N tosses are a prescribed sequence.
BE = {co e /; co = ,a\ a2a3.. .an .. }
where au...,aN are prescribed and everything else is arbitrary. Let s =
. a1 a2... aN 0 0..., then BE = (s, s 4- (1/2N)] so that fiL(BE) = 1/2* as expected.

4
Chapter 1 Measure Theory
3. E is the event that H occurs in the Nth place.
BE = {coel; co = .ax a2...aN^ 1 aN+1...}
Fix a particular s = .ax ...a^^ 1000.... Then B£ contains the interval
(s, s + (1/2N)]. We can choose the al9..., aN_t in 2N~l different ways, and each
of these intervals is disjoint from the others; so
MBE) = 2-(i,)4
13 15 3
4 8 2 8 4
The shaded region is BE for the event that H occurs on the third toss.
4. E is the event that, in the first N tosses, exactly k heads are seen.
Be = {co € /; co = .a\di... aN ..., where exactly £ of the first N at9s are 1}
Fix .£*!,...,%, k of which are 1. Let s = .<z1a2...0NOOO..., so that ££
contains (s,s + (1/2N)]. There are (£) such intervals, all mutually disjoint, so
Pl(Bb)
-m
5. Start with X dollars and bet on a sequence of coin tosses. At each toss
you win $1.00 if a head shows up and you lose $1.00 if a tail shows up. What
is the probability that you lose all your original stake? To discuss this event
we introduce some notation.
Rademacher Functions
For coel we define the kth Rademacher function Rk by
(3) Rk(co) = 2ak-l
where co = . ax a2... is the binary expansion of co. Note that
„ , , f+1 ifak= 1
(4) -„-, , , ,. Q
so Rk(co) represents the amount won or lost at the kth toss.
To familiarize ourselves with the Rademacher functions, we graph the first
three, Riico), R2(co), and R3(co).

§1.1 Introduction
5
*»
+ 1-
-1-*-
R2
+1-
i _/
l ^
[to)
v-
1
1
I
4
■ ■ — ~w
1
1
I
2
LJ-™ -™"
I
1
3
4
— ™ "•
1
1
1
**(»)
+ 1-
H h
-!-*■
Using the Rk\ we can describe BE for event 5. First consider the event Ek,
representing loss of the original stake at the fcth toss. Let
(5)
Sk(co) = £ *,(«)
Sk(co) gives the total amount won or lost at the fcth stage of the game. Then
BEk ={coe /; Si(co) > -X for / < k, and Sk(co) = -X}
and
*E = U BEk

6
Chapter I Measure Theory
We will postpone the computation of fxL(BE) to §1.4 because BE is not a finite
union of intervals.
Now we return to the law of large numbers. Our assertion is that "roughly
as many heads as tails turn up for a typical Bernoulli sequence." We formulate
this statement mathematically as follows.
For (Del, with co = .a1...aN..., let sN((o) = at + a2 + •• • + aN. This
sum gives the number of heads in the first N terms of the Bernoulli sequence
corresponding to o. Now fix e > 0 and consider
f T\sN(co) ll 1
MH^~ 2\>e\
i
2
This set represents the event that, after the first N trials, there are not "roughly
as many heads as tails." We can restate our assertion as follows.
Theorem 2. (Weak law of large numbers)
ViX^n) -*" 0 as N -* °°
Proof. We first describe BN using Rademacher functions. Recall that
Rk(co) = 2ak-l
where co = . ax a2.. • ak Thus
N
SN(co) = X Rk(o>) = 2(flx + a2 + ••■ + flj - N = 2sN(o)) - N
Now
N 2
> eo\2sN((o) - N\> 2eN
which is equivalent to |5N(a))| > 2sN. So, by altering e slightly, we restate the
theorem as follows.
Let AN = {weI; \SN(w)\ > Ns}
Then jUL(/4N) -> 0 as AT -> oo
To prove this form of the theorem, we will need the following special case
of Chebyshev's inequality.
Lemma 3. Let / be a nonnegative, piecewise constant function on (0,1]. Let
a > 0 be given. Then

§1.1 Introduction
7
fiL({(oeI;f((o)>
*})<\[ fdx
aJo
(Here the integral Jo fdx is the usual Riemann integral.)
Notice that we know how to compute fiL({co e I;f{co) > a}) because {co e I;
f(co) > a} is a finite union of intervals.
Proof of lemma. When / is piecewise constant, there exist xl9...,xk with
0 = xx < • • • < xk = 1 and / = ct on (xh xi+1) i = 1,..., k — 1. (This is what
we mean by piecewise constant.) Then
I
i k-i
fdx = X Ci(xi+1 - Xi) > X'cr(xl+1 - xt)
0 i=l
where Z' means sum over the i's such that c{ > a. Therefore,
Z,ci(*m - xt) > a£'(xm - xt) = ocfiL({coeI;f(co) > a})
so
1 f1
- fdx > fiL({(oeI;f(co) > a})
aJo
Now we continue with the proof of our theorem. Notice that
AN = {coeI;\SN(co)\>Ns}
= {coeI; SN(co)2 > N2€2}
An application of the preceding lemma gives
hMh) = M{">e/; |S*(<d)|2 > N2e2}) < ^ P S^dx
To exploit this inequality we need to compute jo S%dx- However
f1S^x= [\YJRk)2dx= t ['rUx + Z ^^Rjdx
JO JO *=lJo i+JJO
Because Rk = 1, each of the first N terms is equal to one. What about
I
i
RtRjdx i ^/?
Suppose i <j. Let J be an interval of the form (//2\(J + l)/2'], 0 < / < 2'.
Then Rt is constant on J and Rj oscillates 2(j — i) times so that

8
Chapter 1 Measure Theory
i
Rjdx = 0
Thus
which proves that
Thus
I
1
RiRjdx = 0
1
i
S*dx = N
o
^(^)<(^iJN = ^t->0 as N^oo □
The astute reader has probably noticed that we haven't proved exactly
what we said we intended to prove at the beginning of this section. Namely,
we wanted to prove that, for a "typical" Bernoulli sequence,
(6) l_£»£Uoas^oo
By "typical" we should mean equation 6 fails on a set of zero probability.
By the Borel principle, an event E has probability zero if the corresponding
set BEcz I has Lebesgue measure zero. The only sets we know thus far with
zero Lebesgue measure are finite collections of points. When we extend
Lebqsgue measure to a collection of sets much bigger than the collection of
intervals, we will find many more sets of measure zero. In fact we can describe
these sets now without developing the general theory of Lebesgue measure.
Given a subset A czR and a countable collection of sets {At}jiu we will
say the >4£'s are a countable covering of A if A c [j^ A{.
Definition 4. A set A c R has Lebesgue measure zero if, for every e > 0, there
exists a countable covering {At} of A by intervals such that
(7) f>t04,)<e
Remarks.
1. In this definition we can allow the i4f's to be finite unions of intervals.
2. If A has Lebesgue measure zero and B c A, then B has Lebesgue measure
zero.

§1.1 Introduction
9
3. A single point has Lebesgue measure zero.
4. If A1, A2,... is a countable collection of sets, each having Lebesgue
measure zero, then \J flx At has Lebesgue measure zero. In particular,
countable sets have Lebesgue measure zero.
Proof. (Remarks 1, 2, and 3 are clear.) To prove remark 4, choose e > 0.
Because A{ has Lebesgue measure zero, there exists a countable collection of
intervals Aitl,Ait29... covering A{ such that
j-i *
The collection {AtJ} is countable, it covers (J?^ Ah and
i,j i=l j=l i=l ^
Now let N = {co e I; (sn(co)/n) -► 1/2 as n -► cxd}. N is called the set of normal
numbers. Let Nc denote the complement of N.
Theorem 5. (Strong law of large numbers) Nc has Lebesgue measure zero.
Remark. Nc is uncountable; in fact, Nc contains a "Cantor set."
Consider the map a:I -► J defined by
o-(co) = . ax 11 a2 11 a3 1 1...
for co = ,a1a2a3 This map is one to one, so its image is uncountable.
Notice also that the image is contained in Nc. In fact, if co' = a(co)9 then
s3n(co') > 2n\ so
3n "3
Now we will prove theorem 5. Let
An = {coel; \Sn(co)\ > en} = {coel; S*(©) >e4n4}
Then, by Chebyshev's inequality,
MlW.) < i I sndx where | #ix = | ( £ Kfc J dx
e n Jo Jo Jo \*=i /
Multiplying out the integrand, we obtain five kinds of terms:
1. Rt a=l,...,n
2. RlR] oc*P

IU
Chapter 1 Measure Theory
3. R*RfiR7 OL^P^y
4. R^Rp ol^P
5. RaRpRyRd a^P^y^d
Because R£ = 1 and R^Rj = 1, J£fl£<frc = JoRlRjdx = 1.
We claim that the other terms all integrate to zero. In fact,
and
R%RpRydx= RpRydx = 0
Jo Jo
RlRpdx = RaRpdx = 0
Jo Jo
What about RaRpRyRdl Assume a < p <y <8 and consider an interval of
the form (//2y, (/ + 1)/2V]. Ry is constant on J and, because a < /? < y, RaRfiRy
is constant on J as well. Finally, Rd oscillates 2(8 — y) times on J, so
1
RaRfiRyRddx = 0
and
RaRpRyRddx = 0
Because there are n terms of the form R* and 3n(n — 1) terms of the form
^a^fl>
J Sw4dx = 3n2-2n<3n2
and
3
*&>
Hl(A„)?Z \-r-7 3«2^
Lemma 6. Given 5 > 0, there exists a sequence £,, e2,... such that s„ -> 0 and
(8) 2i<5
n=l " «>w
Proof. Choose, for instance, e„ such that
e4 = cn~1/2 for some constant c.

§1.1 Introduction
11
Then
oo 3 3 co j
If c is chosen large enough, this quantity is less than 5. V
Finally, for each n, set
Bn = {co; \Sn(co)\ > snn}
Hd^n) < 3/e£n2, hence Yj?=i A*l(Ai) < <5. Notice that the B„'s are finite unions
of intervals since Sn is piecewise constant. Thus, if we can show that Nc a
(j£=i An ^e theorem will be proved.
Now Nc <= (Jw°°=1 Bn if N s fln^i5«- Bu*> if coef|„% ££, then, for each
n, \Sn(co)\ < e„n; that is, \Sn(co)/n\ < en. Because en-^0, we conclude that
| Sn(co)/n| -► 0; that is, co e N. □
Remarks.
1. We have just proven theorem 5 by showing that
(9) fiL(N<) = 0
Notice that we needed a relatively sophisticated definition of "measure zero"
to make sense of this statement, because Nc is such a bad set. In particular Nc
is uncountable. (The only intervals of length zero are points, and Nc is not
even a countable union of such sets.) Later, when we discuss the connection
between measure and integration, we will see that this example provides a
good illustration of why Riemann integration is inadequate for probability
theory.
2. Notice that the strong law of large numbers (theorem 5) does not
indicate at what point we can expect about as many heads as tails. In §3.8 we
will discuss the central limit theorem, which has some bearing on this question.
Exercises for §1.1
1. Prove that the set $ of Bernoulli sequences is uncountable by the Cantor
diagonal argument.
2. a. Let coel = (0,1]. Show that co can be written in the form Yj%i aJlK
a{ = 0,1. Show that this expansion is unique when we restrict to
nonterminating series.
b. Show that, for any integer k,coeI can be written in the form J^ ajk\
where a{ = 0,1,..., k — 1. Show that the expansion is unique when we
restrict to nonterminating series.
3. A gambler has an initial stake of one dollar. Calculate the probability of

Chapter 1 Measure Theory
ruin at times 1, 3, and 5. Show that the chance of eventual ruin is at least
70%.
Show that
I
Ry>Rv2''RyJx = 0
or 1
for any sequence yx <y2<"' <yn- When is the answer one?
Define the Rademacher functions on the whole real line by requiring
them to be periodic of period one—that is, by setting jRfc(x + 1) = Rk(x).
With this definition, show that Rk+l(x) = Rk{2x) and, by induction, that
Rk(x) = R^l^x).
Show that
Rn(x)= -sgn[sin(27c2n~1x)]
except at a finite number of points. (Notation: For any number a, sgna is
one if a is positive and minus one if a is zero or negative.) We will see later
in the text that some interesting analogies exist between the Rademacher
functions and the functions sin(27c2n~1x).
Prove that
2t - 1 = £ Rk(t)2~k
Every number ewe(0,1] has a ternary expansion
°> = X) ai3~'
with a{ = 0, 1, or 2 (see exercise 2). We can make this expansion unique
by selecting, whenever ambiguity exists, the nonterminating expansion—
that is, the expansion in which not all af's from a certain point on are equal
to 0. With this convention, define
Tk{a>) = ak - 1
Draw the graph of Tk for k = 1,2, 3. Can you discern a general pattern?

§1.1 Introduction
13
9. Obtain a recursion formula for the 7^'s similar to the recursion formula
for the Rk's in exercise 5.
10. Let C be the set of all numbers on the unit interval [0,1], which can be
written in the form
00
<o = Z a*3~*
k=l
with ak = 0 or 2. Show that C is uncountable. (C is called the Cantor set.)
(Hint: Use the Cantor diagonal process.)
11. Prove that the Cantor set (see exercise 10) can be constructed by the
following procedure: From [0,1] remove the middle third, (i,f); from the
remainder—that is, the intervals [0,y] and [f, 1]—remove the middle
thirds, and so on, ad infinitum. The remainder is the Cantor set.
I 2
3 3
12 1 2 2 8
9 9 3 3 9 9
12 1 2 2 8
9 9 3 3 9 9
12. Show that the Cantor set is of measure zero.
13. Describe geometrically the set a(I) discussed on page 9.
14. Show that the nonnormal numbers are dense in the unit interval.
15. a. Show that a positive number c3 exists such that, for all JV,
/ [SN(x)]6dx<c3N3
Jo
b. Let An be the set {co e /; |S„(co)| > en}. Show that the Lebesgue measure
of An is less than c3e"6n"3.
16. More generally, show that a positive number cK exists such that, for all N,
L
[SN(x)]2Kdx < cKNK.
0
17. Prove a refinement of the strong law of large numbers, which says that
-£-►0 as N-+oo
N*

i«t Chapter 1 Measure Theory
for any 3 > j. (Hint: Use exercise 16. We will see later that, for <5 = j9 the
situation is much more interesting.)
18. Prove that
(Hint: By induction. Write
etSn(x)dx= etS»-iix)etR»ix) dx
Jo Jo
Break up the unit interval into 2""1 equal subintervals on each of which
#„__! is constant. Show that
f etS»™dx = (^yA f ets^x)dx
if J is one of these intervals.)
19. From exercise 18, derive the formula
i:—&rm
1=0
20. Let / be a nonnegative monotone function defined on the unit interval.
Prove Chebyshev's inequality
■» «;/.'•
fiL({(oeI;f(co)>a})<-\ fdx
aJo
with the integral on the right being the Riemann integral.
21. We have already defined Lebesgue measure for two kinds of sets: finite
unions of intervals and sets of Lebesgue measure zero. Show that these
two definitions are not contradictory; that is, show that the interval [a, fc],
a < b, is not a set of measure zero. (Hint: Use the Heine-Borel property
of compact sets.)
§1.2 Randomness
In §1.1 we saw how to identify the set 0& of Bernoulli sequences
with the set of points on the unit interval J. In terms of this identification, a
probabilistic event E, associated with Bernoulli sequences, gets identified with
a subset BE of /. We saw that, at least for simple events, the Borel principle

§1.2 Randomness
15
applies; that is,
(1) Prob(E) = fiL(BE)
We will attempt in this section to describe some slightly more complicated
probabilistic events in measure theoretic terms.
Example 1. Gambler's Ruin
A gambler has X dollars and bets at even odds on a coin flip. What is the
probability of his ruin?
We discussed this event already in §1.1. We showed that
BE=\jBEk
where
BEk = {coeI; Si(co) > -X for / < k, and Sk(co) = -X}.
After developing some measure theoretical tools, we will see that
(2) fiL(BE) = £ KBBk) = 1
fc = l
In other words, with probability one, if a gambler bets long enough, he will
eventually lose all his money no matter how big his initial stake.
Example 2. Random Patterns
Pick a finite pattern of coin tosses, for example, T H H T. Let E be the event
that T H H T occurs in a given Bernoulli sequence. Then
BE = {cogI; there exists n0
with R„o((o) = -1, Rno+1((o) = 1, Rno+2((o) = 1, and R„0+3((o) = -1}
We will prove in §1.4 that this set is of measure one. In fact we will prove
that, if one fixes any finite pattern, this pattern appears infinitely often in a
Bernoulli sequence with probability one.
This result can be interpreted as follows. Put a monkey in front of a
telegraph key and let him punch a series of dots and dashes as he pleases.
With probability one, the monkey will eventually tap out in Morse code all
the sonnets of Shakespeare infinitely often.

16
Chapter 1 Measure Theory
Example 3. Random Variables
In example 1 let jR„ be the amount of money won or lost at the nth toss. jRn
can be thought of as a function on the set 38 or, via the identification 38 <-> J,
as a function on the unit interval. It is, of course, just the nth Rademacher
function, discussed in §1.1. Rn is a typical example of what probabilists call a
random variable. It is a variable—that is, a quantity that one can measure each
time one performs a sequence of Bernoulli trials—and it is random, because
the values it assumes are a matter of hazard or chance. Another example of a
random variable is the sum
which is the total amount won or lost by the nth stage of the game. Notice
that the set BEk in example 1 is completely described by the S„s. This is
not surprising. Most interesting random events are describable by random
variables. For instance, consider winning streaks. Suppose that, starting at time
t = n, a gambler tosses an unbroken sequence of heads for a certain length of
time. The relevant random variable connected with this phenomenon is the
variable /„, which counts the number of times H occurs consecutively starting
with the nth toss.
Example 4. Expectation Values
Let 38 be the set of Bernoulli sequences. A random variable associated with
the Bernoulli process is, by definition, a function /: 38 -► R. Thanks to the
identification of 38 with J, we can also think of / as a function on J. In Chapter
2 we will address the question of what kinds of functions correspond to the
"physically interesting" random variables. For these functions we will be able
to define the Lebesgue integral
(3) jVdfe
The probabilists call equation 3 the expectation value of the random variable
/. Roughly speaking, it is the value that / is "most likely" to assume in a series
of frequently repeated experiments. To use a simplistic example, for the
Rademacher function jRn, the integral in equation 3 turns out to be just the
usual Riemann integral
which, as we saw in the previous section, is zero; that is, the "most likely" value

§1.2 Randomness
17
of Rn is zero (even though Rn takes only the values +1 and — 1). We will justify
this somewhat paradoxical assertion in §2.6.
Example 5. Random Walks
A Bernoulli sequence, that is, a sequence of coin tosses, can be considered to
describe a random walk on the real line. That is, a particle is placed at the
origin; a flip of a head causes the particle to move one step forward, and a
tail moves it one step backward. As one tosses the coin an infinite number of
times, the particle moves erratically backward and forward along the real line.
We will call the path traced out by such a particle a random path and the
sequence of motions itself a random walk. Obviously each Bernoulli sequence
gives rise to a random path and vice versa. If we denote by 0t the set of random
paths, we can identify & with ^ and, by means of binary expansions, both 0t
and S& with the unit interval /. Probabilistic events associated with 0t can be
reinterpreted as events associated with & and vice versa. For example
gambler's ruin <-> passing through —X for the first time
In probability jargon the space of all possible outcomes of a probabilistic
process is called the sample space. For Bernoulli sequences, the sample space
is 3&\ for random walks the sample space is 0t. For all intents and purposes,
0t and & are identical, even though one thinks of 0t in connection with the
motion of particles and & in connection with games of chance.
Example 6. Random Walks with Pauses
To perform a random walk with pauses, one needs a gadget of the type
depicted in the figure below. Place a particle at the origin of the real line and
spin the pointer. If it lands on +1, move the particle one unit to the right; if
it lands on — 1, move the particle one unit to the left; and, if it lands on 0,

Chapter J Measure Theory
leave the particle fixed. By repeating this operation infinitely often, we get a
random walk with pauses. Let $bp be the sample space of this process. Identify
<%p with / using ternary expansions of points, a> e J; that is, each coel can be
written as
(4) «=ZS at = 0,1,2
The ternary expansion of co is then
co = . ax a2 a3...
Notice that ^ = . 10 0 0... or . 0 2 2 2...; so, in order to make ternary
expansions unique, we will always choose the nonterminating expansion in cases
like the one above (see §1.1, exercise 2). Now make the identification
+ 1~1
0~0
-l<->2
to identify a random walk with pauses with the digits in such a ternary
expansion. This identification gives a map / -+ <%r The Borel principle in this
instance says that, if an event E associated with this random process
corresponds to the subset BE of /, then, just as before,
(5) Prob(£) = iiL(BE)
We suggest you check equation 5 for a few simple events. (See exercise 4.)
Example 7. Random Walks in the Plane
For two-dimensional random walks, we need a gadget similar to the one
we used on the previous page:
N
W X E
s

§1.2 Randomness
19
Let Z2 = {(m, n); ra, n integers} denote the integer lattice in the plane.
Place a particle at (0,0) e Z2 and spin the pointer. If it lands on N, move the
particle to (0, l)eZ2; if it lands on E, move the particle to (l,0)eZ2, and so
on. By repeating this operation ad infinitum, one produces a random walk,
the successive stages of which are indexed by an infinite sequence such as
(6) NSSEWE...
Let ^piane be the sample space of this process—that is, the set of all sequences
like the one in display 6. We can identify each sequence with a point coel
using base-four expansions; that is, we I can be written as
(7) *>=£S ** = 0,1,2,3
The base-four expansion of co is then. ax a2 a3..., which can be identified with
a sequence like the one in display 6 by means of the correspondence
0 <-> East
l^West
2<-»North
3<->South
Of course we must deal with the problem of nonuniqueness in this
identification as above, by selecting nonterminating rather than terminating expansions
whenever ambiguity exists. Just as in example 6, to every event E associated
with this process there corresponds a subset BE of/. We urge you to check that
Prob(E) = iiL{BE)
for a few simple, typical events. (See exercise 5.)
Example 8. The Discrete Dirichlet Problem
Let Q be a smooth, bounded region in the plane with boundary B. An
important problem in electrostatics is the Dirichlet problem: Given a
continuous function / on B, find a function u satisfying
Am = 0 in Q
(8)
u = / on B
where Am = (d2/d2x)u + (d2/d2y)u.
This problem has a discrete analogue that is itself quite interesting. Let Q

Chapter 1 Measure Theory
be a finite subset of Z2. A point p = (w, n) of CI is an interior point if its four
next-door neighbors
(m,n + 1), (m + l,w), (m,n — 1), and (m — l,n)
are also in Q; otherwise, p is a boundary point. For instance, in the figure below,
pi is an interior point and p2 a boundary point of the shaded region.
For a function u on Z2, we define Adiscretew by the formula
(Kiscrcuu)(m9n)
(9) w(m, n + 1) 4- w(m, n — 1) + w(w + 1, n) + w(m — 1, n)
= - w(m, n)
[Notice that the first term on the right is just the average of u over the
next-door neighbors of the point (m, n).] The discrete analogue of the Dirichlet
problem is to find a function u: Z2 -» R such that
(10) Adiscreteu = 0
at the interior points of Q, and
(11) " = /
on the boundary, 3Q, of Q, / being a given function on 3Q. One can solve this
problem elegantly by using the random walk described in example 7: Given
a point peQ and a random path co starting at p, let F(a>,p) be the value of/
at the first point at which co hits 3Q. [If co never hits the boundary, set
F(co, p) = 0.] If we fix p and regard F as a function of the random path co alone,
then F is a random variable in the sense of example 3. We will show in §2.8
that its expectation value is the value at p of the solution of the Dirichlet
problem described in equations 10 and 11.

§1.2 Randomness 21
Example 9. Randomized Series
Probabilistic considerations have another way of entering into classical
analysis. Consider the series
00 1 00 (-W
Z± and E^
«=i« «=i n
The first of these series diverges, whereas the second converges. We can enrich
this problem by adding a probabilistic component. Consider a general series
00+1
z—
where the plus or minus is determined by the flip of a coin; that is, for each
co e I we get a series
(12) £ *!.(*)
w=1 n
with Rn being the nth Rademacher function. Now let E be the event that this
series converges:
BE = ico e I; ^ converges\
Whatis/iL(BBJ?
In §3.3 we give a series of exercises in which we sketch a proof that
l*L{BE) = 1. The intuition behind this result is that a typical Bernoulli sequence
has roughly as many pluses (heads) as minuses (tails).
Example 10
We end this section by considering a collection of sample spaces that includes
all of those we have considered up to this point.
Take n marbles of k various colors. Say the colors are labeled cl9c2,--.,ck
and suppose that, of the N marbles, Nj of them have the color Cj, 1 <j<k.
Now put all of the marbles into a cylindrical wire cage that can be spun on
its axis to mix the marbles fairly well within the cage. After the marbles are
mixed, a blindfolded assistant removes one marble from the cage. If the marble
has the color cj9 one gets as a reward a preassigned number, rj9 of dollars.
(Incidentally, we will allow r- to be positive or negative.) After the color of the
marble is recorded, the marble is returned to the cage and the process is
repeated.
The probability that the color Cj will be chosen is

22
Chapter J Measure Theory
and
k k
(13) I>j=1 because £ W), = AT
Notice that this game can serve as a model for coin tossing (and random walks)
by allowing only two colors of equal number, say cx = red and c2 = white,
and setting the rewards at +$1.00 for a red marble and —$1.00 for a white
marble.
With three colors of equal number, say cx = blue, c2 = white, and c3 =
red, and rewards rx = +$1.00, r2 = 0, and r3 = —$1.00,'we get a model for
the random walk with pauses.
If we alter our process slightly by allowing the r£'s to be vectors in R2, we
can model random walks in the plane; namely, take four colors of equal
numbers with rx = (0,1), r2 = (1,0), r3 = (—1,0), and r4 = (0, — 1).
In §2.6 we will develop a measure theoretic model for this process based
on a "Borel principle" similar to that in the preceding examples.
Exercises for §1.2
1. Under the correspondence <% «-* /, describe the subset of / corresponding
to the event that a run of 15 heads will occur before a run of 11 tails.
2. Describe the subset of / corresponding to the event that no run of heads
longer than 15 occurs in a Bernoulli sequence.
3. Prove that the pattern H T has to occur infinitely often in a Bernoulli
sequence (with probability one) using the Borel principle.
4. With the ternary numbers as a model for the random walk with pauses,
test the Borel principle by using it to compute the probability of
a. a pause at time t = 1.
b. a pause at time t = n.
c. forward motion at times t = 1,2,3,..., n.
d. forward motion at times t = fe,/c+ l,...,fe + n.
5. With the quaternary numbers as a model for the random walk in the
plane, test the Borel principle by using it to compute the probability that
a. the first move is due east.
b. the wth move is due east.
c. the first n moves lie on a straight line.
6. With the ternary numbers as a model for the random walk with pauses,
prove that with probability one an infinite number of pauses occur. (Hint:
See §1.1, exercise 12.)

§1.2 Randomness
23
7. Sum the series
l±?
by the following procedure. For each Bernoulli sequence, put a (+) sign
in the feth place if a head comes up and a (—) sign if a tail comes up. What
is the sum? (Hint: See §1.1, exercise 7.)
8. Let Q be the subset
{(0,0), (1,0), (0,1), (-1,0), (0,-1)}
of Z2. (That is, Q consists of the origin and its four next-door neighbors.)
Check directly that the recipe described in example 8 for solving the
"discrete Dirichlet problem" on Q is correct.
9. For the process described in example 10, show that, if one uses an equal
number of marbles of each color, the sample space of the process can be
identified with the unit interval using expansions in base fe.
10. For the ordinary random walk starting at the origin, show that the
probability of a particle's being in position k at time t = n is
0 if |fe| > nor if n + Zeis odd
(*)
m
where r = —-— otherwise
11. (On Markov processes.) Let P = (Py), — oo < ij < oo, be an infinite
matrix with the following properties:
(i) ^>0
(**) («) £j Pu = 1 for all i
(Hi) For fixed i, Pu = 0 for all but finitely many/s.
For the "generalized random walk" associated with P, a particle moves
along the line according to the following probabilistic rule: If the particle
is at position i at time t = w, then at time t = n + 1 it can be at any position
j for which Pi} # 0, and the probability of its being there is P0.. (For
instance, if Pti = 1 and Ptj = 0 for i # ;, then the particle stays forever
at its initial position.) The matrix P is called the matrix of transition
probabilities associated with the process.
a. Show that the process described in example 10 is a process of this kind.
(Think of the position of the particle as being the total number of
dollars won or lost by time t = k.)
b. For the process described in example 10, show that the matrix of
transition probabilities is of the form Ptj = P£_j.
c Show, conversely, that if Ptj = P^j the corresponding process is a
process of the kind described in example 10.

24
Chapter 1 Measure Theory
12. a. Show that, if P and Q are matrices of the form in equation (**), the
usual matrix product PQ is well-defined and is of the same form.
b. Show that, for the generalized random walk associated with P, if the
position of a particle at time zero is i, the probability that its position
at time t = nisj is just the i —jth entry of the matrix Pn.
13. Show that, for the matrix PM+1 = Pi+iti = i> Pij = 0> otherwise, the
generalized random walk is the usual random walk. Derive the formula (*) of
exercise 10 by computing directly the i —jth. entry of Pn. (Hint: Consider
the vector space V consisting of all finite sums:
Y,ckekt cksR
On this vector space consider the linear mapping "multiplication by
(e~l + e')/2." Show that, if we take for a basis of V
e~kt e~l 1 el ekt
...c ,...e ,x,e,...,e ,...
then, in terms of this basis, this linear mapping has P as its matrix.)
14. Can you construct a measure theoretic model for random walks in space
similar to the measure theoretic model for random walks in the plane?
(Hint: Expansions in base six.)
§1.3 Measure Theory
We mentioned earlier that Lebesgue measure assigns to each set
A, belonging to a certain collection of subsets of R, a nonnegative number
liL(A) called the Lebesgue measure of A. We also mentioned that \iL has certain
additivity properties. We will now study these properties in more detail. We
need to begin with a large number of technical definitions. Keep in mind the
vague notion of Lebesgue measure we have already discussed so as to put
these technicalities in perspective.
Let AT be a fixed set. Suppose A and B are subsets of X. We recall the
following notation:
Notation Meaning
0 empty set
AkjB = {xeX;xeA or xeB} union of A and B
Ar\B = {xeX;xeA and x e B} intersection of A and B
Ac = {x e X; x $ A) complement of A
B — A = {xeX; xeB and x$ A} B minus A
S(A9 B) = (A — B)<u(B — A) symmetric difference of A and B (see figure,
page 25)

§1.3 Measure Theory
25
A ring of sets in X is a nonempty collection ® of subsets of X satisfying the
following two properties
1. AuBe® whenever/!, BeM
2. A- Be0t whenever A, Be®,
Remark. 0e@ since A - A = 0. Also if A, Be® then A n Be® since
A H B is obtained from A U £ by deleting A - 5 and 5 - A.
Two examples with which we will soon be very familiar follow.
Example 1. Let 2X denote the set of all subsets of X; 2X is a ring.
Example 2. Let X = Rn. Suppose {ax,...,an) and (bl9...9bn) are given, with
each at <bhi= 1,2,..., n.
Let A be the set of points xeR" such that
(1) at <xt < bt i= l,...,n
A is called a multi-interval. More generally, a multi-interval is a set of the form
shown with perhaps some of the <'s replaced by a <.
Define ®Leh by Ae®Leh<=>A = \J?=iAh where the A/s are a disjoint
collection of multi-intervals. We let the reader check that ®Leh is a ring.
Now, fix a ring 0t of subsets of X. Let \i be a nonnegative set function on
^; that is, to each AetX, p assigns a nonnegative number \i{A).
Definition 3. \i is additive if /z(A u5) = /z(A) + /z(£) whenever A, Be® are
disjoint.
Example 4. 0t — ®Leh. Suppose Ae®Lch is a multi-interval described by the
inequalities
at <Xi<bi i = 1,...,m
(Again, some <'s may be replaced by <'s.) We define
(2) rtA) = (bi-a1)(b2-a2)-(bn-an)

26
Chapter 1 Measure Theory
More generally, if A = (JfLi Ai is a disjoint union of multi-intervals, we define
(3) A*M)*5>Wi)
I
t=i
Then u is a well-defined additive set function on ^Leb*
Proposition 5. Let 0t be a ring of subsets of AT and /z an additive, nonnegative
set function on 0t. Then
1. M0) = O.
2. (monotonicity) HA9Be@ with A c 5, then /z(A) < n(B).
3. (finite additivity) If Al9A2,.-*,An€<% are mutually disjoint, then
MU?=i^-) = Z?=iM^).
4. (lattice property) If A, B e 0t then fi(A uB) + /z(A n5) = /z(A) + fi{B).
5. (finite subadditivity) For any Al9..., Ane@, fi(\JUi A{) < JJ.j /*(>!;).
Aw/.
1. A e ®9 ii(A) = ii(A u0) = /i(X) + ix{0\ so /i(0) = 0.
2. £ = (£ - A) u 4 is disjoint, so /x(£) = /*(£ - A) + /*(A) > M>0-
3. Induction on n.
4. A = {A-B)Kj(AnB)
B = (B-A)Kj{AnB)
AKjB = (A-B)Kj{B-A)Kj(AnB)
so fx(A) = fi(A - B) + fi(A n 5)
p(JB) = /i(B-i4) + /iC4nB)
/*(4 u B) = |i(i4 - £) + /i(B - A) + ^(i4 n 5)
5. Induction on m; case n = 2 follows from item 4. □
So far we have done nothing very deep. We have just given an abstract
setting for the situation in example 4. Our eventual purpose is to extend the
definition of the set function in example 4 to a much larger ring of subsets of
R. For instance, this ring should contain the sets of measure zero described
in §1.1. In order to carry out this extension in a natural way, we will need the
following refinement of additivity. As the proof of theorem 7 will suggest, this
property is much more intricate than finite additivity.
Definition 6.
1. Let 0t be a ring of subsets of X and \i an additive set function on 31.
We say \i is countably additive on 0t if, given any countable collection
{A£}f* x c 0t with the Afs mutually disjoint and such that A = \Jf^t A{ is
also in 0t9 then

§1.3 Measure Theory
27
(4) M^)=ZA*Ml)
i=l
2. A countably additive, nonnegative set function fi on a ring ^ in X is called a
measure.
Warning. Equation 4 makes no sense unless we assume Ae0t,
Theorem 7. If X = Rw, ^ = ^Lcb5 and \i is the set function in example 4, then
/* is a measure.
Lemma 8. Let A e @Leh9 and let e > 0 be given. There exist F,Ge ^Leb such
that F is closed, G is open, F ^ A^G, and
/i(F) > /i(i4) - s
li(G) < ii(A) + e
Proof. Suppose A is a multi-interval given by the inequalities
ax <xt <bt i= l,...,n
where some of the <'s may be replaced by <'s. We can find a 8 such that
Y\l(bi-d)-(ai + d)l = Y\(bi-ai-2d)>Y\(bi-ai)-s
»=1 i=l i=l
and
II [ft + d) - (a, - *)] = n& - a. + 25) < FI& - «i) + «
t=l 1=1 i=l
Let F be given by the inequalities
at + 5 < Xi < bt — 8 i = 1,..., n
and G by the inequalities
af — <5 < x, < 6f + <5 i=l,...,n
We then have
H(F) > ii(A) - s
11(G) <; 11(A) + 6
Now, if A = U}sl i4£ is a disjoint union of multi-intervals, find for each At an
Ff and G£ such that

28
Chapter 1 Measure Theory
fift) 2: p{At) -1
MGi)^K) + |
Then, with F = (JJ=1 Ft and G = (J?=1 Gf, we have
MO * E A*(G|) <S E U^) +1] = A*W) + « V
Now, take {A^t to be a disjoint collection of sets in ^Leb, and suppose
/I = (J.%! /!(is also in ^Leb. Notice that (Jf^ At e A, so
(5) fi(A) £ // f (J X^ = £ ^(yl,) for every N
Thus
(6) li(A) :> £ M4)
£=1
Choose a closed stt F ^ A such that /*(F) > fi(A) — e, and for each A{
choose an open set Gx containing A{ with /z(Gf) < ^(Ai) + 6/2*.
Because F is closed and bounded, it is compact. Because it is covered by
the Gf's, it must be covered by a finite number of them, say Gx, G2,..., GN. Then
li{A) - a < ii(F) < ji(jj Gf) < t m) < t [a*Wi) + j] ^ Z MWi) + £
Being true for all e, this yields
(7) l*(A) zttiAi)
Putting inequality 7 together with inequality 6 shows that \i is a measure. □
We have now constructed a measure on a collection of subsets of R". The
sets on which this measure is defined, MLeb, are very simple, however. As
remarked above, the property of countable additivity will allow us to extend
this measure to a much larger ring of sets.
Let \i be a measure on a ring M in X. We attempt to extend \i to the ring
2X by mimicking the definition of measure zero in §1.1.

§1.3 Measure Theory
29
Definition 9. Let A be a subset of X. A number / > 0 will be called an
approximate outer measure of A if there exists a covering of A by a countable
collection of sets Al9A29A39... with each A{eM such that
(8) i^At)<l
Remark. I is allowed to be + oo.
Definition 10. Let A be a subset of X. The outer measure of 4, /i*(>4), is the
greatest lower bound of the set {/: / is an approximate outer measure of A).
If this set is empty, then ix*{A) = + oo.
We now have a set function, fj,*, on the ring 2X. Unfortunately, /x*
is not generally a measure (see, for example, exercise 1, or, for a more
rewarding example, Appendix C). We will show, however, that \i* is a
measure on a large ring of subsets of X; this ring will be called the ring of
measurable sets in X.
Proposition 11.
1. If A g 0, then y*(A) = fx(A).
2. If A <= B9 then fi*(A) < fx*(B).
3. fi* is countably subadditive; that is, if Al9 A2, A39... are subsets of AT, then
Proof.
1. Covering A by the sequence Ax = A9 A2 = 0, A3 = 0,..., we see that
fi(A) is an approximate outer measure for A9 so
(9) ti*(A) < ti(A)
To prove the other inequality, let e > 0 be given. Because f**(A) is the greatest
lower bound of all approximate outer measures of A9 a cover {A^%t c= 0t
must exist such that
(10) fi*(A) + e^t^Ai)
i=l
Let A\ = Al9 A'2 = A2 — Al9 A'3 = A3 — {At u A2)9 and so on. Then the
A"s are mutually disjoint and
(11) fi*(A) + e > t VW)
i=l
If we let A" = A\ n A9 we have that A[ ^ A for all i9 the /4J"s are mutually

Chapter 1 Measure Theory
disjoint, and still
(12) M*(^) + e>X>W)
Now, since A" ^ a for all i and (j£x At => A, we must have {JfLx A" = A. So
V(A) = L-i A*W) and thus
(13) /i*(^) + £ > /*M)
Because this inequality is true for all £, we have
(14) »*(A) > n(A)
2. If / is an approximate outer measure for B, then surely it is for A. Thus
fJL*{A) < fi*{B)
3. Given e > 0, for each i we can find a cover, {;4,j}j!Li <= ^, of ^ such that
(15) ^%4,) + J>lM^)
Then the countable collection {Aitj}™j=1 covers A = (J£x A{ so that
*,j=i t=i i=i
(16)
<
-J+f/1%4,)
1 = 1
This holds for all e > 0, so
(17) /*%4) <: |>%4,) □
i=l
Remark. This proof is essentially the same as that used in §1.1 to show that
a countable union of sets of measure zero is itself of measure zero.
Now our original ring, 0ty is a subset of 2X. We wish to find a larger ring,
M, containing ^, that will be the measurable sets. Our strategy will be to
think of 2X as a metric space and define a distance function on it, so that,
roughly speaking, Jt will be the closure of 0t in 2X with respect to this distance
function. (For a quick review of metric spaces, see Appendix A.)

§13 Measure Theory
31
For A, B c= X we define the distance from A to B by
(18) d(A,B) = fi*lS(A,Bn
where S(A, B) is the symmetric difference S(A9 B) = (A — B)v(B — A).
If A and 5 are the unit squares pictured, then d(A, B) = 1^.
Caution.
1. d(A9 B) may be + oo.
2. Although we are calling d a. distance function, d(A9 B) = 0 does not
necessarily imply A = B.
Proposition 12. Suppose A, B, Ce2x. Then
1. d(A9B) = d(B9A)
2. d(A9A) = 0
3. d(A9B) + d(B9C)>d(A9C)
Proof.
Lemma 13. 1. S(A9B) = S{B9A)
2. S(A9A) = 0
3. S(A9B)vS(B9C)=>S(A9Q
Proof. Items 1 and 2 are obvious. To see item 3 we have
S(A9B) = {A-B)kj{B-A)
and S(B9C) = {B-C)u{C- B)
so- S(A9B)uS(B9C) = {A-B)u{B-A)kj{B- C)u(C - B)
But ^~Cc(A-B)u(5-C)

32
Chapter I Measure Theory
and C - A s (B - A) u(C - B)
so S(/4, C) = {A - C)u(C - A) £ S(X,5)u S(B, Q V
The proposition follows from the lemma. □
Note. d(A, B) = 0 if n*{S(A, B)) = 0; that is, A and B symmetrically differ by
a set of outer measure zero.
Although the preceding note says that d is not quite a distance function
in the standard sense, we can still use d to define the notion of convergence in
2X. That is, we say a sequence {/lj}j™i e2x converges to A e2x, written At -* A,
ttd(A„A)-*0.
Proposition 14. The Boolean operations in 2X are continuous with respect
to d. That is, if An -► A and Bn -* B in 2X, then
An<oBn-* AkjB
A„nB„-* AnB
An-Bn-+A-B
and Acn -> Ac
Proof.
Lemma 15. If AltA2, B1,B2e2*, then
1. S(A<uB\) = S(AuBl)
2. S(^! uAj.B, uB2) s S^.BJuS^.Bj)
3. S(A, ni42,B, nB2) cS^LB^uSf^,^)
4. S(>l1-yl2,B1-B2)sS(yl1,B1)uS(i42,B2)
Pro©/.
1. S{A,B) = (/4 - B)u(B - A) = (/InB°)u(Bn/lc)
so S(AC, W) = (/4£ n B) u (B° n ,4) = S(/4, B)
2. S(/l1u4B1uB2) = [(>l1u^)-(B1uB2)]uPiuB2)-(/(1u/l2)]
= [(^1u^2)n(B1uB2)c]u[(B1uB2)n(/41u/42)c]
= [(^! u/42)n(Bc1 nB2)] u [(Bj uB2)n(^C! n/42)]
s (/41 n B0!) u (/12 n B^) u (Bt n A',) u (B2 n 42)
= S(>l1,Bl)uS(/l2,B2)
3. S^j n /42,B! n B2) = S(A\ u 4C2, B^ u B2)
SSMl.BDuS^Bi)
= S(X„B1)uS(/(2,B2)
4. S(/4! - 42, Bt - B2) = S(.At n 4C2, B, n Bc2)
sS(i41(B,)uS(4fli)
= S(>l1,B1)uS(>l2,B2) V

§1.3 Measure Theory
33
From the lemma we have the immediate corollary
1. d{A,B) = d(Ac9Bc)
2. d(AluA2,BiKjB2) < d(AuBx) + d(A2,B2)
3. d(Ai n A29B1 nB2) < d(AuB{) + d(A2iB2)
4. d{Ax - A2,BX - B2) < d(AuBl) + d(A2iB2)
from which the proposition follows. □
Proposition 16. fx* is continuous in the following sense: Let A, Be2x and
suppose either fi*(A) or fi*(B) is finite, then
(19) \fi*(A) - fx*(B)\ < d(A,B)
Proof. Suppose fi*(B) < oo; also assume fji*(B) ^ n*(A). Then
li*(A) = d(A,0)
<d(B,0) + d(B,A)
= V*(B) + d(B9A)
Thus
\lx*{A) - ^(B)\ = ii*(A) - fi*(B) < d(B, A) □
Definition 17. Let MF be the closure of 0t in 2X. That is, A e JtF if and only
if there exists a sequence of sets {i4£}," t c 0t such that d(Ah A) -► 0 as i -» oo.
Theorem 18. 1. JtF is a ring.
2. For A e JtF, fi*(A) < oo.
3. fi* is a measure on ^F.
1. Assume A, Be JtF. We need to show that AuB and A — B are in *MF»
Now, because A, BeJtF there are sequences {Aj-3^ and {5j£i in 0t such
that y4f -► A and ££ -» B. By the continuity of the Boolean operations
At kjB^AkjB
Ai-B^A-B
so JtF is a ring.
2. A eJF implies that there is a sequence {A^fLx <= ^ with 4£ -► A. For
some m, then, d{An9 A) < 1.

j*» Chapter I Measure Theory
Thus
lx*(A) < fi*{An) + 1 < oo
3. We first show that fx* is additive, or—what amounts to the same
thing—we will prove the lattice property; that is, if A, Be JtF then
(20) ii+(A uB) + fx*(A n B) = fi*(A) + fi*{B)
Choose An -» A and Bn -» B in 01. Because on 9t, /z* = \i, fi* is additive on M.
Thus
li*(An u Bn) + m*(^4„ n B.) = ji*(4,) + /**(£„)
But An u Bw -*• X u B and y4„njB„->An5,so the continuity of y* implies
li*(A u B) + /x*(/l n5) = /**(/*) + /**(£)
We now prove countable additivity. Let {A^fLi be a mutually disjoint
sequence in JtF with 4 = (j£t >4£ also in ^#f. By the subadditivity of ft* we
know that
(21) »*(A) < £ /i%4,)
i=l
Furthermore
i=l
H*(A) > n* ( y XjJ = £ m*(^i) for all N
That is,
(22) /*%4)>f>*K) D
1=1
Definition 19. A is a measurable set, AeJi,\ithere exist {AJ£*x cr ^F such
thati4 = Ui-Mi-
Theorem 20. \iAsM then Xe^F<=>/z*(i4) < oo.
Pnra/. Part 2 of theorem 18 gives "=>", so to establish the theorem we must
show that, if fi*(A) < oo and AeJi, then AeM¥.
Because A e Jt, there exist A£ g M? such that A = \]%\ At. We can assume
this union is disjoint for, if it isn't, we can replace the i4t's by ^'s as follows:
so
1,=AX

§13 Measure Theory
35
A2 — A2 — A1
^3 = A3-(A1uA2)
and, because M¥ is a ring, we know A( e M¥. Thus we can assume A = [jflx At
is a disjoint union.
Now consider ix*(A). First, subadditivity gives fi*(A) < Yj^i /J*(^i)- We
claim that, in fact, ix*(A) = Yfii l**^). To see this, notice that
so
[JA^A
i=l
Because this equality holds for any N, we have
1 = 1
and thus
(23) £ ix*(Ai) = lx*(A)
Now, fix e > 0 and let BN = \J"=1 i4£; then BNsJtF, and
(24) W '
because J^ /i*(i4£) is convergent. Thus /I eJt¥ because BN-+ A and ^#F is
closed. □
We now consider properties of the collection M.
Definition 21. Let Sf be a collection of subsets of a set X. ¥ is called a a-ring
if
1. it is a ring and
2. given {Aj^ in Sf, \)?=x A£ is also in Sf.
Theorem 22. J( is a a-ring.
/V00/. First we will show property 2.
Suppose y41,y42>... are elements of Jt. Let /I = (J^ ^£. Because each
y4£e^#, there must be {A^}^ in MF such that

36
Chapter 1 Measure Theory
At = U *u
i=i
Then A = U*v/=i ^y' a countable union, so /I eJi.
Now we will show that Ji is a ring. It suffices to show that if A9 BeM
ihQTiA-BeJt.
First, suppose A e M¥ and write B = U£ t J5£ with J5£ e */#F. Because JiF is
a ring, A nBteJiF so i4n5 = U«^i^n^ *s a member of Jt. Moreover,
li*(A n B) < ix*(A) < oo, so Ar\BeMF. Now ^-J5 = ^-(^nJ5) and,
because c^F is a ring of which A and ^4 n J5 are members, we have A — Be JtF.
Now let A be a general element of ^# and write ^4 = \j£x i4£ with A£ g ^#f.
Then
but from the discussion above, A,- — J3 e ^#F, so we are done. □
Theorem 23. If A1, A2,... is a countable collection of disjoint sets in ^#, then
Proof. Let >4 = (j£ i /l^, y4 e ^#. We consider two cases separately.
1. ix*(A) < oo
Because At c /I, /x*(y4f) < oo so A and all of the A?s are elements of JtF.
Because ju* is a measure on ^#F, we have then
(x*(A)=fifi*(Ai)
2. fj.*(A) = oo
In this case subadditivity tells us that
oo = fi*(A) ^ f /,%4i)
t=l
00
SO £ A**(4) = °° □
Now that we have constructed measurable sets in the abstract case, let us
return to the example of Lebesgue measure in R".
Example 24. X = Rn, ^ = ^Leb = finite unions of multi-intervals, and fi is
as given in example 4. Here we call J( the set of Lebesgue measurable sets in

§1.3 Measure Theory
yj
Rn, and the extension of \x to Jt (the restriction of /x* to Jt) is called Lebesgue
measure \ih.
What do the sets in Jt look like? First we remark that RneJt. Indeed let
IN = {xeRn; -N <Xi<N9i= l,...,n}
Then
00
R" = (J iN and each INeM c JtF
N=l
Proposition 25. Every open subset of Rn is in Jt.
Proof. Let c = (al9a2,--.,an,blib2,...,bn)e R2n with the a£'s and fy's rational
and a{ < b(. Let Ic = {xe Rn; ai<xi<bi,i= 1,2,..., n}. The collection { Jc} is
countable.
Now let 0 be any open subset of Rn; 0 is equal to the union of all sets Ic
such that Ic c (9, (If xeO we can find a c such that xe/cc 6?.) Because such
a union is countable, GeJt.
Corollary 26. Every closed subset of Rn is in Jt.
Proof. A is closed so Ac is open. A = Rn — Ac and, because Jt is a ring and
Rn, Ac 6 ^, we have AeJt.
Corollary 27. All countable unions and intersections of closed and open sets
are measurable.
We have shown that the measurable sets are a <x-ring containing the open
subsets of Rn. Are they the smallest <x-ring with this property? That is, if one
starts with the closed and open sets, forms countable unions and intersections,
and then from these forms further countable unions and intersections, and so
on, does one eventually end up with all measurable sets? The answer is no,
unfortunately; so we are forced to make the following definition.
Definition 28. The Borel sets are the smallest a-ring containing the open sets.
Although not all measurable sets are Borel (for an example see Halmos,
P. Measure Theory. [Van Nostrand: Princeton, NJ] p. 67), the following
theorem says that any measurable set is close to being a Borel set.
Theorem 29. HAeJt there exists a Borel set B ^ A such that [j*(A - B) = 0;
that is, A can be written as A = (A — B)vB9 where B is Borel and
tx*(A -B) = 0.

38
Chapter 1 Measure Theory
Lemma 30. XiAeJl and if e > 0 is given, then there exists a Borel set G such
that G Z5 A and n*(G - A) < e.
Proof. First suppose ix*(A) < oo. Then by definition of /x* we can find a cover
A c UJSjAj such that
where each of the A/ 's is a multi-interval; so we take G = U£j Ar-, which is Borel.
More generally, if AeM we can write A = (j£x A,- where each A{tJ(F.
By the preceding argument we can find Borel sets G{ with At c Gt and
/z*(G£ - A,) < e/2l. Then with G = (JSi Gi we have
/**(G - /4) < a V
Lemma 31. If AeJt there exists a Borel set F c A with
li*(A -F)<e
Proof. Choose a Borel set G such that Ac a G and /x*(G — Ac) < e by lemma
30. Let F = Gc. Then /4 - F = G - Ac and
^♦(,4 - F) = /x*(G - Ac) < e V
Now we prove theorem 29. Take A e Jt. For every N choose a Borel set
FN(=A such that /x*(A - FN) < l/N. Let F = (j£=1 F^; then F is Borel and
/**(>! - F) < ix*(A -FN)<~- for every N
N
Thus ix*(A - F) = 0. □
We conclude this section with a few remarks about notation. Let X be a
set, 0t a ring of subsets of X, and /x a measure on ^2. By theorem 18 \x
extends to a measure on a much larger ring of sets, Ji¥. In fact \x can be
regarded as a measure on the a-ring Jt, providing we define it to take the
value + oo on sets A that are in M but not in JtF. Note that proposition 5 is
then still true if one observes the usual addition conventions for + oo, namely
( + oo) + a=+oo foraeR
and (-f-oo) + (-foo) = -hoo
Moreover, \i is countably additive on Jl by theorem 23. Note that, if X itself
is in Jif, these problems with infinity don't arise; that is, J(F = J(. For all
examples of measures that we will encounter in this text, the set X is either in
JtF or in Jl—that is, X satisfies the conditions of the following definition.

§1.3 Measure Theory
39
Definition32. X is cr-finite if there exist sets X^J^p, i= 1,2,..., with
For example, Rw is a-finite because
R"=LM
i=l
with Bi being the ball of radius i about the origin.
Exercises for §1.3
1. Let X be an uncountable set. Let & be the collection of all finite subsets of
X. Given Aeffllet fi(A) be the number of elements in A. Show that 0t is a
ring and that \x is a measure on ^?. Identify /x*. What are «/# and Jt/i Is
every subset of X measurable?
2. Let X be an infinite set and let ^ be the following collection of subsets:
Ae0t\i and only if A is finite or AC is finite. Let \x be the following function
on M\ fi(A) = 0 if A is finite, and fi(A) = 1 if >4C is finite. Is \i a measure?
3. a. Let X be an infinite set and St the collection of all countable subsets
of*. Is^aa-ring?
b. Let \i be a measure on 0t. Show that there exists a function f\X ^
[0, oo) such that
(*) AiM) = Z /(*)
foralM60.
c. Show that the function / in part b has to have the following
two properties: (1) The set {xeX;f(x) ^0} is countable and (2)
d. Show that, if/ has the properties in part c, the formula (*) defines a
measure on 0t.
4. Let X be the real line and & — t%Leh. (That is, finite unions of intervals.)
Given AeSt\t\. fi(A) = 1 if, for some positive e, A contains the interval
(0, e). Otherwise let fx(A) = 0. Show that \i is an additive set function but is
not countably additive.
5. Let F be a continuous, monotone increasing function on the real line. If A
is an interval with endpoints a and b, let
VM) = F(b) ~ F(a)
More generally, if A is a disjoint union of intervals
N
A=\J A,
i=l

40
Chapter 1 Measure Theory
let fiF(A) = Yj=i ^f(^i)- Show that fxF is a measure on the ring ^Leb; that is,
prove it is countably additive.
Remark. If one takes for F an antiderivative of the function
(\/^j2n)e~x212, fxF is called the Gaussian measure. We will encounter it
several times later on.
6. a. Let A be a measurable subset of R. One says that the density of A is
well defined if the limit
r-oo 2.1
exists. If the limit exists, this expression is called the density of A. Can
you produce an example of a measurable set A whose density is not
defined?
b. Show that, if A1 and A2 have well-defined densities and are disjoint,
then Ax u A2 has a well-defined density and
D(AlvA2) = D(A1) + D(A2)
c. Show that there exist sets A and Ai9 i= 1,2,..., with well-defined
densities such that
00
^ = (J At (disjoint unions)
t=i
but
D(A)^ltD(Ai)
7. Let X be a set, 0t a a-ring of subsets of AT, and \ix and fi2 measures on 9t.
Let ££ be the family of all those sets Aefflfor which ixx(A) = fi2(A).
Assume Xe@ and fx1(X) = f*2{X) < oo. Show that S£ has the following
properties:
(i) XeS£.
(**) (ii) If A, Be& and B s A9 A - BeS£.
(iii) If A( e if, i = 1,2,... and /I = (J^ y4£ (disjoint union)
thence if.
Remark. A collection of sets if having the properties listed in (**) is
called a A-system.
8. Let X be the three-element set {Px ,P2,P3 }, and let ^ be the ring of subsets
of AT. Let ix t and \x2 be measures on 0t. When is the set ££ a ring? Show that
S£ doesn't always have to be a ring.
9. Show that the example described in exercise 1 is not cr-finite.
10. Remember that a metric space is complete if every Cauchy sequence

§1.3 Measure Theory
41
has a limit. Show that, with respect to the distance function d(A9 B) =
fi*(S(A, B))9 2X is complete.
11. a. Given any collection # of subsets of a set X, show that there is a
smallest ring of sets 0t containing <€. (That is, M has the property that it
contains #, and any ring that contains # contains M) Describe
explicitly how to construct 0t from #.
b. Show that the ring ^Leb is the smallest ring containing the multi-
intervals.
12. Given any collection # of subsets of a set X, show that there exists a
smallest a-ring of sets 0ta containing #. This justifies definition 28.
13. Show that for any 8 > 0 there exists an open dense subset U of R with
iiL(U) < 5.
14. Let ceR". Given any subset A of Rn, let A + c = {weRn;w - ceA}.
Prove that, if A is measurable, then A + c is measurable and
(**) M^ + c) = ixL(A)
(Hint: First, prove this for multi-intervals. Next, show that in equation (**)
the outer measures are equal.)
15. Let /: R -► R be the linear mapping x -> ax + b9 a and b being constants
with a > 0. Show that, if X is measurable, f(A) is measurable and
16. Let ^ be a Lebesgue measurable subset of R and let
CA = {(x,y)eR2;xeA}
Such sets are called cylinder sets. Show that the collection of these sets
forms a ring 0tc, Show that the set function \ic defined by
Pc(CA) = VM)
is a measure on this ring. Show that, if S is a proper subset of R and
fxL(A) # 0, the set
/I x S = {(x9y); xeA9yeS}
is not a measurable subset of R2 with respect to the measure \xc. (Hint:
What is its outer measure, computed with respect to ficl)
17. Let /: Rw -> Rn be a continuous map. Show that, if A is a Borel subset of
R", then f~\A) is a Borel subset of Rw. Define
VM) = M/_1(>4))
Show that fif is a measure on the Borel subsets of Rn.
18. Let £% be a ring and /x a measure on ^. Prove that, if Al9A2,*.*,An are
in 0t then

42
Chapter 1 Measure Theory
fi(A1 u • • • u An) = £ /x(^) - £ /x^ n Ay)
£</<*
+ (-irV(^in---n^J
19. Let X be a set, 0t a ring of subsets of X9 and /x a measure on ^. Let
/x* be the corresponding outer measure. Show that if A eJi¥ then, for all
E^X,
(t) »*(A n £) + ju*(/1c n £) = ,u*(£)
(Hint: First check this for /I e^.)
20. Prove the converse result; that is, suppose n*(A) is finite and A satisfies
property (f) for every subset E of X. Prove that A e Jt¥. (Hint: Show that,
for every set A and every e > 0, there exists a set EeJi such that E => A
and ju*(£) < fi*(A) + e. Use property (f) to conclude that d(E9A) < e).
Remark. In many textbooks the property (f) is used as the definition of
a measurable set.
§1.4 Measure Theoretic Modeling
Now that we have developed the basic notions of measure theory
we can examine a little more closely the ideas involved in what we have called
the "Borel principle." First we provide some definitions.
Definition 1. Let X be a set and 3* a ring of subsets of X.
1. ^isa/ieWifXe^.
2. !F is a a-field if X e !F and if ^ is a a-ring.
Definition 2. Let X be a set and !F a field of subsets of X. Suppose \x is a
measure defined on «^\ Then ju is a probability measure iffi(X) = 1. In this case
the triple {X9 «^, /*} is called a probability space.
Example 3. Let X be the unit interval /, and let 3F be the measurable subsets
contained in /. Then !F is a cx-field and the Lebesgue measure is a probability
measure.
Now let X be the sample space of a probabilistic process. A measure
theoretic model of the process is a <7-field &* of subsets of X and a probability
measure /x defined on !F so that, for any "plausible" event E in X9 we have

§1.4 Measure Theoretic Modeling
43
BEe^ and Prob(E) = ix(BE\ where BE is the set of points in X for which E
occurs.
Of course this definition is not precise; the word plausible is left to be
interpreted by the modeler. We need to include the plausibility qualification
because the desired measure may not be defined on all subsets of X. For
example, as we show in Appendix C, not every subset of the unit interval is
Lebesgue measurable.
Let us consider some examples of measure theoretic models.
I. Discrete Probability Theory
Suppose the sample space X is finite or countable, say X = {xl9x2,x39...}.
Further, suppose that each point x£ has the probability ft of occurring and
that Yj Pi = 1- The measure theoretic model for this process is given by letting
^ be the collection of all subsets of X and by defining \i as
(1) fx(A)= £ Pl torA^X
xteA
It is left as an exercise for the reader to check that \x is a measure. (See exercise
3 in §1.3.)
Notice that in this case we need not interpret the word plausible because
SF contains all subsets of X; that is, all events are considered plausible.
II. Bernoulli Sequences and Random Walks
In this case the sample space can be identified with the unit interval /, and the
measure theoretic model is given by the Borel principle. In §1.2 we saw that,
for many "plausible" events E, BE is a finite union of intervals (and thus
measurable) and that Prob(£) = fi(BE) in these cases. The events considered
there were rather simple; let us now confirm that BEs^ for some more
complicated, yet still "plausible," events.
1. Let E be the event that a prescribed finite pattern, for example, H T T H,
occurs infinitely often. To describe BE we let En be the event that the pattern
occurs beginning at the nth step. Because BEn is described by a finite number
of conditions on the Rademacher functions, it is a finite union of intervals.
(If the pattern is HTTH, then BEn = {(oeI; Rn{co)=l, Rn+1(co)= -1,
/J„+2(fi)) = — 1, Rn+3(co) = 1}.) Thus, because
(2) BE=f] [jBEn
it is Borel.

44
Chapter 1 Measure Theory
2. (Law of large numbers) Let E be the event that a Bernoulli sequence
obeys the law of large numbers. That is, with Sn(co) = ££=1 Rk((o)9
SJco)
BE = \(dgI\ -^-^-+ 0 as n -> oo
■{•
We know this is measurable because we showed its complement has
measure zero. However, let us describe BE as a Borel set.
Recall that the statement
5w(co)
-> 0 as n -► oo
means that, for every integer r > 0, there is a fc> 0 such that
S>)
If we let
(3)
we can write
(4)
< - whenever n > k
r
f r |S„(<b)| 11
r=lfc=in>k
which is Borel, because each An%r is a finite union of intervals.
3. Let E be the event that £ (Rn(a>)/n) converges. We claim BE is Borel. Let
(5)
*=1
Then the Cauchy condition tells us that X(K„(co)/n) converges if, for every
integer r > 0, there is a fc> 0 such that
| Tn(co) - Tm(co)\ < - for all m9n>k
If we let
n,r = \«
we see that
(6)
which is Borel, because Amnr is a finite union of intervals.
bb - n u n ^.n,

§1.4 Measure Theoretic Modeling
45
We have now shown that these "plausible" events correspond to
measurable sets. Thus, if we assume that the Borel principle holds, we can
determine the probability of these events by finding the measure of these sets. We
know already that, for the law of large numbers, the set described has measure
one. We now develop the necessary tools to determine the measure of the set
described in event 1.
This example is a special case of the following general situation: Start with
a countable collection of events
{E1,E2,...}
and define a new event E to be the event that infinitely many of the events Et
occur. Can we determine Prob(£) if we know Prob(jEt) for all i? Two theorems
address this problem; they are called the Borel-Cantelli lemmas. In order to
formulate them, we first restate the problem in measure theoretic terms.
Let X be the sample space of our process, equipped with a a-field !F and
a measure \i. Let Bt denote the subset of X on which E{ occurs. We assume
that Bte^ and that Prob(jEj) = /x(Bf). If we let BE denote the subset of X
corresponding to the event E, then, in terms of the 5/s,
(7) BE=(\{JB„
k=ln>k
We give this a name.
Definition 4. Given sets Bt, B2,B3,... in &r, then
(8) {Bl.;i.o.}=limsupBn= f] [j Bn
is called "Bi9 infinitely often" or the limes supremum of the Bf's.
Theorem 5. (First Borel-Cantelli lemma) Given BUB2>... in ^, let B =
{Bt; i.o.}. Then £>i fifa) < oo implies that fi(B) = 0.
Proof. Let Ak = \J„>kBn so that B = [)?=! Ak\ in particular, B ^ Ak for all
k. Now, by subadditivity
Thus, because Y£=i M(Ai) < oo, for e > 0 there is a k > 0 such that
M(4k) ^ I KBn) < e
n^k
Because B ^ Ak we have that fi(B) < e, and because e is arbitrary we must
have fi(B) = 0. □

46
Chapter 1 Measure Theory
Application. (Run lengths) For oel define the nth run-length function /„
by letting l„(co) be the number of consecutive l's in the binary expansion
of co starting at the nth place. That is, ln(co) = k if Rn(co) = 1, Rn+1(co) =
1,..., Kn+k_i(a>) = 1, and Rn+k(co) = -1.
Now take a sequence of non-negative integers, ri,r2,r3,..., and let
En denote the event that ln(uj) > rn. Let E = {En ;i.o.}. Then
BEn = {coeI; R„(co) = Rm+1(a>) = • • • = R,vl(a)) = 1}
so ii{BEr) = (|)rn and we can use theorem 5 to conclude the following.
Corollary. If £n°°=1 (1/2/" < oo then fi(BE) = 0.
The second Borel-Cantelli lemma supplies a partial converse to the first.
It is restricted by applying only to independent events.
Definition 6. Two events Et and E2 are independent if the outcome of Ex
tells us nothing about the outcome of E2.
Let us try to make this definition more precise by restating it in measure
theoretic terms. Knowing that the event Ex occurs means that the elements
of the sample space in which we are interested are already in BEr Now, for
what proportion of the elements in BEl does the event E2 occur? Clearly the
answer is
(9) KBElnBS2)
1 ' KBEl)
This ratio is called the conditional probability of E2 given Et. Now, if E2
is independent of Eu this conditional probability is just the probability of E2
computed without prior knowledge of Ex—that is, n(BE2); hence
KBe^ mow
This leads us to the following measure theoretic definition.
Definition 7. Let A" be a sample space with c-field !F and probability
measure \i. Two sets Al9 A2e!F are independent if
(10) VL(AlnA2)-11^)^2)
Example 8. Given X = /, \i = Lebesgue measure, Al = {coel; R^co) = 1},
and A2 = {coel; R2(co) = 1}, then
^=(il] and ^2 = (ii]u(il]

§1.4 Measure Theoretic Modeling
47
AtnA2 = &ll
Thus, ii(Ax n A2) = i = (i)2 = KA1)fi(A2)
Definition*). More generally, Al9A2,...,A„ are independent if, for any
sequence of integers 1 <i1 < i2 < • • • < ik < n, we have
(11) MX£i n Ai2 n • • • n XJ = ji^i,)/z(Af2) • • • fi(Aik)
Further, a countable collection of sets is independent if every finite subcollec-
tion is independent.
Example 10. Let At = {coel; R^co) = 1}. It is left as an exercise to prove that
the collection Ax, A2,... is independent.
Theorem 11. (Second Borel-Cantelli lemma) Let (AT, ^, //) be a probability
space and let Al9A2,... be an independent collection of sets from J5". Suppose
that Yj^i^t) = °°jtnen Ki^il i-°-}) = 1-
Lemma 12. Let i4l9 A2,... be an independent collection of sets in «F. Then
A\, Ac2, A%,... is an independent collection of sets in SF.
The proof of the lemma is left to the reader. (See exercise 10. We suggest
that the reader give this exercise a few moments of thought before continuing
the chapter.)
Proof of theorem. Let A = {At; i.o.}. Then
A=(\{JA„ so A<={J()A<n
k=ln^k k=l»>k
To show that fi(A) = 1, it is enough to show that fi(Ac) = 0; and, to establish
this fact it is enough to show (by subadditivity) that
«(.q/:)-°
Now, by independence,
\n=k J n=k
but fi(Ac„) = 1 — fi(An\ which in turn is less than or equal to e~^A^ because
it is true in general that 1 — x < e~x. (Prove this inequality yourself!) Thus

48
Chapter 1 Measure Theory
(12) \i I f] Acn ) < n e-^ = g-H-*"M">
\n=k J n=k
But g-Il-*^") -» 0 as / -» oo, because £^=1 //(XJ = oo. Thus v(C)n>kAcn) = 0.
D
Example 13. Let Hn denote the event of a head at the nth toss of a Bernoulli
sequence. The corresponding subset of / is
An = {coeI;Rn(a>)=l}
In exercise 11 the reader will show that these sets are independent.
Furthermore, fi{An) = j for each n9 so £^04J = oo. Thus a head occurs infinitely
often in a Bernoulli sequence with probability one. (This result can be proved
much more trivially. What is the proof?)
Example 14. Example 13 is an example of a finite pattern (the pattern H)
occurring infinitely often in Bernoulli sequences. More generally we now show
that any finite pattern occurs infinitely often in Bernoulli sequences with
probability one. For simplicity of notation, consider the particular pattern
HTTH.
Proposition 15. The pattern HTTH occurs infinitely often in a Bernoulli
sequence with probability one.
Proof. Let En be the event that HTTH occurs starting at step n, and let Bn
be the corresponding subset of /. Because the A„'s are independent and Bn =
AnnAcn+1nAcn+2nAn+29 we have fi(B„) = (£)4 = ^; so ^=1 »(Bn) = 00.
Unfortunately, B„ and Bn+l are not independent, so the second Borel-Cantelli
lemma does not apply. However, the sets B„,Bn+4,Bn+B,... are independent;
in particular, BuBS9B99...9B4k+l9...are independent and
00
fc=l
so the second Borel-Cantelli lemma applies to give
M{*4*+i;i.o.})=l
But {B4k+1; i.o.} c {Bn;lo.} so
1 = /*({B4k+1; i.o.}) < fi({B„; i.o.}) < 1
Thus ix({Bn9 i.o.}) =1. □

§1.4 Measure Theoretic Modeling
49
Remark. This same proof works for any finite pattern of H's and T's—for
example, Shakespeare's sonnets translated into Morse code, with the dots and
dashes changed to H's and T's.
Exercises for §1.4
1. We will say that an event E involving Bernoulli sequences is plausible if
the subset BE of the unit interval corresponding to it is a Borel subset.
Show that the following events are plausible:
a. A gambler quadruples his initial stake. (Beware: he will not quadruple
his initial stake if he gets wiped out beforehand.)
b. In an infinite sequence of trials, a gambler breaks even an infinite
number of times.
c. In an infinite sequence of trials, arbitrarily long run lengths occur.
d. In an infinite sequence of trials, H comes up "on the average" more
often than T.
(Incidentally, event d shows that "plausible" does not necessarily mean
"probable.")
2. Show that, for random walks on the line, the following events are
plausible:
a. The origin is visited infinitely often.
b. Every integer point on the real line is visited infinitely often.
3. Show that, for random walks in the plane, the following events are
plausible:
a. The origin is visited infinitely often.
b. Every point (m, n) is visited infinitely often.
4. Let / be a function from the integers to the real numbers. Show that, for
random walks on the line, the event
00
Z /(*») < °°
i=l
is plausible, nt being the position at time i.
5. Let S = ZS=i ± 2~l be the series obtained by flipping a coin to decide
whether a plus sign or a minus sign goes into the ith place. Show that the
event |S| < e is plausible, and compute its probability. {Hint: See §1.2,
exercise 7.)
6. Let AT be a set, SF a a-field of subsets of AT, and \x a probability measureon
SF. Let Al9A29A3,...bea. sequence of subsets of AT belonging to !F.
a. Show that, if Ax 3 A2 3 A3 • • •, then
\i=l / i-
ftm n(Ai)

50
Chapter 1 Measure Theory
b. Show that, if At ^ A2 e A3--, then
\f=l / i-^oo
7. Let AT be a set and Al9A2,A3,...a. sequence of subsets of X. The set
k=1 \n>k J
is denoted as liminf A„ or as {An; a.a.} (abbreviations for limes infimum of
A„ and "An, almost always," respectively). Show that {Acn; a.a.} is the
complement of {An; i.o.}. If X is a probability space and the >ln's are
measurable, conclude that the probability of {Acn; a.a.} is zero if and only
if the probability of {An; i.o.} is one.
8. Let AT be a set, & a a-field of subsets of X, and \i a probability measure on
<F. Show that, if Al9A2,A39... are in ^, then
^(liminf An) < liminffi(An) < limsup/x(i4„) < fi(\imsupAn)
9. a. Let X be a set and let A x and A2 be subsets of AT. Show that the smallest
(7-field containing ^ and A2 consists of at most 16 sets. (Hint: Take
unions of the four sets: Atr\ A2, A\ n A2,AX n >42, A\ n A2O
b. Let Al9A29.-.9Ak be subsets of X. Let <Fk be the smallest <r-field
containing the A^s. Show that !Fk has at most 22k members.
c. Show that the upper bound in part b cannot be improved. (Hint: Let
M be the Jc-element set {px,..., pk}, and let X = 2M be the set of subsets
of M. Let A( be all subsets of M that contain the point p(.
10. Let X be a set, ^ a a-field of subsets of X, and /x a probability measure on
#". Suppose that ^ ,...,>!„ are independent sets belonging to &.
a. Show that Ac1,A2,...9A„2ire independent.
b. Let A be any one of the sets Al n A2, A\ n Aj, ^ n ^2, A\ n 42.
Show that A, 43, >44,..., A„ are independent.
c. Let 3Fk be the smallest subfield of & containing A l9...9 Ak. Show that if
Ae3Fk then A, Ak+l,..., An are independent.
d. Let 3Fk be the smallest subfield of SF containing Al9...9Ak and ^,_k be
the smallest subfield containing Ak+l,..., A„. Show that, if A e !Fk and
A'e&n-k9 then 4 and >4' are independent.
11. a. Let A{ be the subset of the unit interval corresponding to the event
"H at the ith trial" in a Bernoulli sequence. Show that the A?s are
independent.
b. Let Bt be the subset of the unit interval corresponding to the event
"HTHattheith,* -I- lstandi + 2nd trial." Show that BuB4r,Bl9Bl0,
... are independent.
12. a. For the random walk with pauses, let A{ be the subset of the unit

§1.4 Measure Theoretic Modeling
51
interval corresponding to the event "a pause at time i." Show that the
Ai's are independent.
b. For the random walk in the plane, let A{ be the subset of the unit
interval corresponding to the event "an eastward move at time i."
Show that the A?s are independent.
13. Let N be a large positive integer. Prove that in a Bernoulli trial run lengths
of length N occur infinitely often with probability one.
14. Prove that run lengths of arbitrary length occur infinitely often with
probability one. (Hint: Consider the random pattern:
THTHHTHHHT...TH...HT
with the last term involving n H's. Let En be the event that this pattern
occurs infinitely often and let E be the intersection of the events En.
15. For the random walk with pauses, prove that with probability one there
are infinitely many pauses. Use the Borel-Cantelli lemma. (An alternative
proof of this fact was suggested in §1.2, exercise 6.)
16. Let N be a large integer. Prove that the random walk on the line, starting
at zero, passes either through the point N or the point — N with
probability one. Conclude that it passes through N with probability at least ^.
17. Let Z2 be the integer points in the plane, and let Q be a finite subset of
Z2 containing the origin. Prove that a random path starting at the origin
hits dQ in a finite time with probability one.
The path depicted hits dSl on the ninth step.
18. In proving the law of large numbers in §1.1, we used Chebyshev's
inequality to prove
> e j ) < 3>T V4
PI ]<*>;
Sn(co)\

52
Chapter 1 Measure Theory
Choose a sequence en -*> 0 such that ^=1 n 2en 4 is finite and let An be the
set
«f|>,}
Use the first Borel-CanteUi lemma to prove that fi({An; i.o.}) = 0 and
deduce from this the law of large numbers.
19. Let X = {x1,x2,...} be a countable set, Pl9P29... a sequence of non-
negative numbers such that £P£ = 1, and [i the measure
ri(A)= £ P(
xteA
Show that X cannot contain an infinite sequence of independent sets
i41,i42,... such that, for all i, ^(Ai) = \. (Hint: Start by observing that
every point xeX must lie in one of the four sets Ax r\A2, A\ nA2,
At n Ac2, or A\ n Ac2. Thus the measure of the one-point set {x} is less
than or equal to £. Notice, by the way, the moral of this exercise: A discrete
measure theoretic model for the Bernoulli process does not exist. Just let
A( be the subset of AT corresponding to the event "an H at the ith trial.")
<co;

Chapter 2
Integration
Now that we have the tools of measure theory, we are ready to
discuss integration. The student of the Riemann integral is accustomed to
considering only integrals over subsets of Rn. However, we will see that
integrals can be defined whenever we have a triple (X, #", pi), where X is a set,
SF is a a-field of subsets of X9 and \i is a measure defined on #\ Such a triple
is called a measure space. Our basic example of a measure space is, of course,
Lebesgue measure on the Borel sets of [0,1] or of the whole real line. The
last section of Chapter 1 suggests that the theory of probability is rife with
other examples. Notice that for the real line some sets have infinite measure—
for instance, fiL(R) = oo. We will allow this to occur in general. (See the
comments at the end of §1.3.)
§2.1 Measurable Functions
In the study of integration, it is convenient to allow functions to
assume the values + oo and — oo. To make this notion concrete, we define the
extended real number system to be the set R u { + oo} u { — oo}.
The elements + oo and — oo in the extended reals have the special
properties
1. — oo<a<+oo, aeR
2. a + ( + oo) = ±oo, aeR
3. a-(±oo)=+oo, aeR9a>0
4. — l*(±oo) = +oo
Now let (AT, SF, fi) be a measure space. Let / be a function on X with values
in the extended real numbers.
53

54
Chapter 2 Integratio
Definition 1. The function / is measurable if, for all aeR, the set {xeX
f{x) > a) is an element of #\
Measurable functions can be characterized in various ways.
Proposition 2. The following are equivalent:
1. For all a e R, {x; f(x) >a}e&
2. For all aeR, {x; f(x) > a} e ^
3. For all aeR, {x; f(x) <a}e^
4. For all a e R, {x; f(x) <a}e^
Proof.
lo4: The sets in items 1 and 4 are complementary and, because 3F is a
c-field, we know that A e & o Ac e 3F.
2o3: Same as above.
1=>2: For all aeR
{x;/(x)>a} = ni|X;/(x)>a-iJ
By item 1 each set {x; /(x) > a — 1/n} e &. Because !F is a c-field, the
countable intersection is in «^\
2=>1: For aeR
{x;/(x) > a} = Q |x;/(x) > a + U
By item 2 each set {x; /(x) > a + 1/n} e^. Because 3? is a a-field, the
countable union is in &. □
In keeping with our notion of extended real numbers, we define the
extended Borel sets as the collection of subsets ofRu{ + oo}u{ — oo} having
one of the following forms:
A, i4u{ + oo}, Au{ — oo}, Av{ + co,—co}
where A is a Borel set. One can easily see that the extended Borel sets form a
o--field.
Theorem 3. Conditions 1 through 4 of proposition 2 are equivalent to
5. For every extended Borel set B
(1) {x;/(x)e£}e^
Proof It is obvious that 5 => 1, 2, 3, 4. We will show that 1, 2, 3,4 => 5.
Let # be the collection of all subsets CofRu{ + oo}u{ — oo} with the
property that

§2.1 Measurable Functions
55
(2) {x;f(x)eC}e&
We need to show that the extended Borel sets are contained in c€.
Note that, by items 1,2,3, and 4, the sets (a, -f oo], [a, + oo], [— oo, a), and
[— oo, a] are members of #.
Also notice that ^ is a <r-field. Indeed, if /4fe#, 1 < i < oo, then
jx; f(x)e H a\ = H {*; fix) eAt}e^
so flJii A^V. Similarly, (J^ >i£e«.
Now notice that the extended Borel sets are the smallest <r-field containing
all the infinite intervals mentioned above. Thus the extended Borel sets must
be contained in #. □
Example 4. Let X = R", and let & = Jt be the Lebesgue measurable sets.
If/: R" -► R is continuous, then / is measurable. Indeed, if a e R, then {x e R";
/(x) > a} is open and thus is a Borel set.
Example 5. Let X = S3, the sample space for the Bernoulli process. As usual,
identify <% with /, the unit interval.
Consider Rn(coi), the Rademacher functions. These are measurable because
they are piecewise constant; namely, for any subset AofRu{ + oo}u{ — oo},
we have that {coel; Rn(co) e A} is a finite union of intervals.
Similarly, if we let Sn{co) = ]T£=1 Rk(co), we see that Sn is also piecewise
constant and hence measurable.
Notation. In §1.2 we used the term random variable for a function /: X -> R
when (X9^,fj) is a probability space. The interesting functions on X are
always measurable. For this reason we use the name random variable
interchangeably with measurable function when discussing probabilistic notions.
Example 6. Let T((o) be the number of times the random walk
corresponding to a) returns to 0, and let ln(co) be the number of consecutive /f's beginning
at the nth toss of co. As an exercise, show that T and /„ are random variables.
(See exercise 4.)
Remark. The variables /„ and T are pathologically discontinuous. The
possibility of integrating such functions, that is, computing their expectation values,
vindicates the work we are about to put into the theory of integration. For
example, we will show in Chapter 3 that T = + oo with probability one; that
is, with probability one a random walk returns to 0 infinitely often.
We continue now with the study of the properties of measurable functions.
Let (X, &, fj) be a measure space.

56
Chapter 2 Integration
Theorem 7. Let / and g be measurable functions on X. Let max(/,0) be
the function on X defined by max(/,0)(x) = max[/(x),^(x)]. Then max(/,^)
is measurable. Similarly, if min( f,g)(x) = min[/(x),^(x)], then min(/,0) is
measurable.
Proof, {x; max(/,^)(x) > a} = {x; f(x) > a} u {x; #(x) > a} and is thus an
element of &.
Similarly, {x; min(/,^)(x) > a) = {x; fix) > a) n {x; #(x) > a} is also
in^. □
Corollary 8. Let / be a measurable function on X. Let
(fix) if f(x) > 0 (-fix) if/(x) < 0
/+(X) = 10 if/(x)<0 aDd f-{x) = { 0 if/(x)>0
Then /+(x) and /_(x) are measurable.
Proof. f+ix) = max(/, 0) and /_(x) = max(-/, 0). D
Notice that fix) = /+(x) — /-(x), so we have just shown that every
measurable function can be written as the difference of two nonnegative measurable
functions. (See the figure below.)
X±
/X
We now consider how measurable functions behave under limits. First, if
fi> fi> • • • are functions on X, define functions sup f and inf f on X by
(3)
sup/w = sup{y;.(x); 1 ^ / < 00}
i>l
inf fi{x) = influx); 1 < i < 00}
«>i
Theorem 9. ISfufi* -. - is a sequence of measurable functions, then sup£>0/;.
and infj>0yi are also measurable.
Proof ForaeR,
{x; supy;(x) >a}=\J {x; /f(x) >a}e&

§2.1 Measurable Functions
57
Similarly,
{x; infflx) >a}=(\ {x; /,(x) >a}e& □
i=l
Now, if/l5/2,... is a sequence of functions, let gk = s\xpn>kfn. Notice that
gx > 02 ^ 03 ^ ' * * so that
lim gk(x) = inf 0k(x)
fc->oo fc>0
Recall that
lim sup fi = inf gk
k>0
is called the "lim sup (limes supremum) of the sequencefl9f2,...." Similarly,
lim inf f( = suphk where fck = inf/n
fc>0 n>k
Theorem 10, If fl9f2,... is a sequence of measurable functions on X, then
lim sup/w and lim inf fn are measurable.
Proof. From the previous theorem, gk = supn^k/n and hk = infw>k/w are
measurable. Applying theorem 9 once again, we have that
lim sup /„ = inf gk and lim inf fn = sup hk
fc>0 k>0
are also measurable. □
Corollary 11. Let /i,/2»--- ^e a sequence of measurable functions on X.
Suppose that the /„'s converge pointwise to a function /; then / is also
measurable.
Proof f = lim sup /„ = lim inf fn □
Thus we see that we can generate new examples of measurable functions by
taking pointwise limits. Another way of getting new measurable functions is
by composition.
Theorem 12. Let /: X -► R be a measurable function. Let g be a continuous
function on R. Then g°f is measurable.
Proof Note that we are not allowing / to take the values of + oo or — oo
so that g °/is defined.
Now, for all aeR, let Oa = {teR; g(t) > a}. Then {xeX; g°f(x) > a} =
{xeX; fix) e &a}. Because g is continuous, (9a is a Borel set. So {x e X; f(x) e
&a} e ^", because / is measurable. □

58
Chapter 2 Integration
Example 13. If/: X -» R is measurable, then eixf, e~f2,2> sin/, |/|, and so on
are measurable.
In a similar vein we have the following theorem.
Theorem 14. Let fx, /2, ...,/„ be measurable functions on X with ft: X -► R,
1 < i < n. Let G: Rw -* R be continuous. Then G(/l5. ..,/„) is measurable.
We leave the proof to the reader as an exercise.
Example 15. Let ft: X -> R, i = 1, 2, be measurable functions. Since x + j;
and xy are continuous functions from R2 to R, fx + /2 and /^ are
measurable.
Exercises for §2.1
1. Given a set X and subsets Al9 A2,..., let
X+ = limsupylB = {An; i.o.} = f| (J A„
*=1 «>k
and
00
A. = liminf^ = {An; a.a.} = (J f) An
Let /+, /_ and fn be the characteristic functions of the set A+, A. and An,
respectively. Prove that /+ = limsup/, and /_ = liminf/,.
2. a. Let X be a set, <F a <r-field of subsets of X, and /* a measure on !F. A
map / = (/i ,..,/„) of AT into R" is said to be measurable if each of the
coordinate functions,/., is measurable. Show that/is measurable if
and only if f~l{B)e& for every Borel set B c Rw.
b. Let g be a real-valued function on R". Then g is 5ore/ measurable if,
for all numbers a, the set {xeRw; g(x) > a} is a Borel set. Prove that,
if /: X -► Rw is measurable and g: Rw -► R is Borel measurable, then
0 o/: AT -► R is measurable.
3. a. Let /„: X -> R, n = 1,2,..., be a sequence of measurable functions.
Show that the set of points, x e X, where the sequence {/„(x); n = 1,2,
...} converges, is measurable.
b. From part a deduce that the set of points, coel, for which the
"randomized harmonic series"
«=i «
converges, is a measurable subset of J.

§2.1 Measurable Functions
59
4. The examples below describe some random variables that occur naturally
in the theory of Bernoulli trials. Show that these random variables are
measurable functions on the unit interval /. (As usual we make the
identification 88 = Bernoulli sequences = /.)
a. R = ruin time = the time it takes a gambler with an initial stake of N
dollars to be reduced to penury.
b. l„ = the number of successive heads that appear beginning with the
nth toss.
c. T = the number of times a random walk returns to the origin.
d. limsupSw/n, where Sn is the sum of the first n Rademacher
functions. This random variable measures "violation of the law of large
numbers."
5. Let T be the random variable in part c of exercise 4. Show that the set
{coeI;T(co)< oo}
is uncountable and dense in /. (We will show in Chapter 3 that this set is
of measure zero.)
6. a. Let S be the random variable in part d of exercise 4. Show that, for
every subinterval of the unit interval and for every real number a with
— 1 < a < 1, S((o) = a uncountably often. (Hint: Suppose that co =
.a1a2a3...is2i sequence for which S(co) = a. Let
co' = ax bxa2a3b2a4asa6b3ana^a^a^b^,..
Show that S(co') = a no matter what the fy's are.)
b. On the other hand, show that the set {weI; S(co) ^ 0} is of measure
zero.
7. a. Let Q be a finite subset of Z2 containing (0,0) as an interior point. (See
§1.2, exercise 8.) Let H be the time at which a random walk starting
at the origin hits 5Q. (H is called the "hitting time.") Show that H,
regarded as a function on the unit interval, is measurable and is finite
except on a set of measure zero. (Hint: See §1.4, exercise 17.)
b. Assume that the points (1,0) and (0,1) are also interior points. Show
that the set
{(dgI;H(co)= +oo}
is uncountable.
8. Let /: R -> R be monotone increasing. Show that / is measurable.
9. A function /: R -> R is upper-semicontinuous at x if for every e > 0 there
exists a 5 > 0 such that f(y) < f(x) + e when \x — y\ < 5. Show that if/
is upper-semicontinuous at all points of R, it is measurable.
10. Two functions / and g are said to be equal almost everywhere if / = g
except on a set of measure zero. Show that if / = g almost everywhere
and if f is measurable, g is also measurable.

60
Chapter 2 Integration
11. Let (X, ^, n) be a measure space with fx(X) < oo. A sequence of
measurable functions fn:X->R, n= 1,2,..., is said to converge to zero in
measure if for all e > 0
lim^{xeX;|/„(x)|>e} = 0
n-*co
a. Prove that, if/, converges to zero pointwise except on a set of measure
zero, then /„ converges to zero in measure.
b. Show that the converse of part a is not true. (Hint: Let X = J, !F be
the Borel subsets of /, and \i be Lebesgue measure. Let Ax = [0,£],
A2 = [i H ^3 = [0,i], ^4 = B,fl, A5 = [|,J], ^ = [|, 1], ^ =
[0,i], A8 = [i,|], and so on. Let fn be the function that is 1 on An
and 0 elsewhere.)
§2.2 The Lebesgue Integral
Let (X9^9fi) be a measure space and s: X -> R be a measurable
function. The function s is called a simple function if it takes on only a finite
number of values.
Example 1. Let £e«f and define
:eE
fl ifxe
'""[O ifx^
M*)= IA :,^E
Then l£(x) is called the characteristic function of E; it is clearly simple.
In general, if s is a simple function taking on the values cx,..., cN9 let
£.• = s-1^) = {xeX; s(x) = c£} {1 < i < AT}
We then have that
N
s(x) = X CilEt(x)
It is easy to define the integral of nonnegative simple functions.
Definition 2. Let s: X -► R be a nonnegative simple function and let £ e3F.
Let c!,..., cN be the distinct nonzero values of s and let Ex? = s~* (c£), 1 < i < AT.
Define the integral of 5 over £ with respect to \x as the sum
(1) /£(5)=fciM(£n£l.)
Remark. This value may be + oo because fi(E n jE£) can be + oo.

§2.2 The Lebesgue Integral
61
Proposition 3. Let st: X -► R, i = 1, 2, be nonnegative simple functions and
letEeJ^.
1. (linearity)
a. IE{csx) = cl^Si) for c e R, c > 0
b. ^(S! + s2) = J^) + IE(s2)
2. (monotonicity) If sx < s2 then /^Sx) < IE{s2)
Proof, la is clear.
lb: Let cx,..., cm be the distinct values of Sj and dx,..., dn be the distinct
values of s2. Let E£ = s^fo), 1 < i < m, and i? = sj1^), 1 <; < n. The E^'s
form a mutually disjoint cover of AT and so do the F/s. Thus E{ n jF}, 1 < i < m
and 1 <j <n, also form a mutually disjoint cover of X, and 5j + s2 has the
constant value ct + dj on ££ n jF}. Hence
IE(Sl + 52) = £(c£ + dy)/i(£i n .F) n £)
= £cil/«<*i n^n £) + E^ZMBinFjO £)
* i j
= X c,./i(£,. n £) + Y,dj»(Fjn E)
i J
= h(si) + IE(s2)
2: s2 — $i is a nonnegative simple function, and s2 = sx + s2 — sl9 so
J(s2) = I(SX) + 7(52 - St). Q
We now extend our notion of integration to nonnegative measurable
functions by approximation with simple functions.
Definition 4. Let / be a nonnegative measurable function from X into the
(nonnegative) extended real numbers and let Ee&. Then the integral of/ on
E with respect to \x is defined by
(2) I fdfi = sup{/£(s);0 < s </, s simple}
The following proposition shows that this definition of the integral is
consistent with our previous definition when / is a simple function.
Proposition 5. IE(s) = J£ s d\i if s is a nonnegative simple function.
Proof. Clearly IE(s) <$Esdfx because s is a simple function with s < s. To
show equality let 5' be any simple function with 0 < s' < s. By monotonicity
J£(s') < J£(s). Hence sup {J£(s'); 0 < s' < s, s' simple} < IE(s). D
At this point the reader is probably asking why the integral of nonnegative
measurable functions should be approximated by the integrals of simple

62
Chapter 2 Integration
functions. We justify this by showing that nonnegative measurable functions
can themselves be approximated by nonnegative simple functions.
Theorem 6. Let / be a nonnegative measurable function on X with values
in the (nonnegative) extended real numbers. There exists a sequence of non-
negative simple functions
0<5l <s2 <••</
such that s( -> /pointwise. Moreover, if/is bounded, then st -» /uniformly.
Proof. We begin by defining sn. Consider the interval [0, n) on R. Divide this
interval into nln subintervals of length 1/2"; namely, let
(3) J, = IteR;1-^- < t < jV 1 < i < n2n
Let Et = rl{fi) and Fn = /_1([n, + oo]). Together the £/s with Fn form
a mutually disjoint cover of X (n is fixed).
Define
(4) sn(x) = f(~-yEi{x) + nlFn(x)
Notice that on Et we have
i-1 . i . i-\
— <f<y; and sn = -^-
Thus, sn(x) < f(x) for xeEhi= 1,..., nln
Similarly, on Fn
n< f and sn = n
so
s„(x)</(x) forx6Fw
Hence
sn < f on all of X
Notice also that sn < sn+1. Indeed, let / be one of the intervals
Notice that / = /' u I" where
I 2" '2"/'

§2.2 The Lebesgue Integral
63
_ p - 2 2i - l\ _ [2t-l 2/ ^
~ 2n+1 ' 2n+1 y a _ 2n+1 92n+1)
Let £ = /-l(/), E' = f~\I'\ and £" = /_1(/"). Then
SnM = ^r forxeE
whereas
sm+i(x) = -^r forxeE' and 5B+1(x) = -^T1- forxeE"
Then, because
E = E'uE" and ^—^ < ^-J-
we have shown that
sn(x) <sn+1 (x) for all x e E
This argument is clearly independent of which It we began with. It also works
with minor changes on [n, + oo], so
sn(x) < sn+ j (x) for all x e X
We now show that sn-*f pointwise. Two separate cases are involved.
Case 1. f(x) = H-oo
In this case xeFn for all n, so sn(x) = n for all n. That is, sn(x) -> + oo.
Case 2. /(x) is finite
Say f(x) < n0. Then, for all n> n0, /(x) lies on one of the intervals /£; that
is,
But then
s«W = -yr so |/(x) - sn(x)\ < —
for n > n0, proving that sn(x) -> f(x).
Finally, suppose that / is bounded, say f(x) < n0 for all xeX. The
preceding argument shows that for n > n0
|/(x)-5„(x)|<i for all xe*
That is, sH->f uniformly on X. □

64
Chapter 2 Integration
Remark. Hidden in this proof is a crucial ingredient of the Lebesgue theory.
In the Riemann theory one approximates a function / by simple functions by
dividing the domain of the function into small intervals, as shown in part (a)
of the figure. In the Lebesgue theory one approximates by dividing the range
of the function into small intervals, as shown in part b of the figure.
(b)
Graph of/ (The dark lines represent the approximating function.)
The second procedure has two conspicuous advantages. First, the x axis
no longer has to be the real line; it can be any measure space X. Second, one
gets a good approximation of / by simple functions without assuming / to
be continuous.
We now look at some properties of the integral.
Proposition 7. Let £, Fe^ and let / and g be nonnegative measurable
functions.
1. (monotonicity) If f <,g then

§2.2 The Lebesgue Integral
65
2. IfFcFthen
3. H>(F) = Othen
fdfi< gdfi
JE JE
J/*** J/**
l/d/i = °
Proof.
1. This is obvious because, if s is a simple function and s < f, then s < g.
2. We first verify this for
f=\G where Gs^
Then J£(lG)d/i = /£(1G) = [i(E n G) and likewise JF(lG)d/x = j*(F n G).
ButEnGcFnG, hence
/x(F n G) < fi(F n G)
Now, by the linearity of IE we know that J£ s rf/x < JF s <fyx when s is a simple
function. But, by the definition oi\EfdpL and $Ffdft, the statement has to
be true in general.
3. If / = s is a simple function, this assertion is clear; thus
sup{/£(s); 0 < s < /} = 0 □
Remark. We will defer to §2.3 the proof that, for nonnegative measurable
functions,
I (/i+/2)^= I /i^+ I /2<fo
Je Je Je
Unfortunately, this fact, which looks as though it should be practically
obvious, requires a somewhat delicate proof.
We can also now prove Chebyshev's inequality in full generality.
Theorem 8. (Chebyshev) Let / be a nonnegative measurable function. For
Ee& and c> 0, let Ec = {xsE;f{x) > c}. Then
(5) M£c)<- fdli
»*\L

66
Chapter 2 Integration
Proof. Because f >c on Ec,
f dfi (by mono tonicity)
I cdii<\ j
JEe JEe
But cdix = IEe{c) = cfi{Ec)
JEe
Thus cfi(Ec)< fdfi< fd\i □
JEC JE
Corollary 9. If / is a nonnegative measurable function with jEfdfi< oo,
then {x e E; f(x) = + oo} has measure zero.
Notation. If a property holds on a set E e 8F except for a subset of measure
zero, we say that the property holds almost everywhere on E (abbreviated as
a.e.). Thus corollary 9 can be restated as
I,
(6) fdfi < oo =>/(x) < oo a.e. on E
Proof. Let An = {xe£; f(x) > n}9 and let A = {xeE; f(x) = + oo}. By
Chebyshev
ii(An)<z\fdli
.sif
But A c An for all n, so
ii(A) < fi{A„) < - \fdfi for all n
njE
Because \Ef d\i < oo we must have fi(A) = 0. □
Corollary 10. Let / be a nonnegative measurable function and let EslF. If
lEfdfi = 0, then / = 0 a.e. on E.
Proof. Let /I = {xgE; f(x) ^ 0}, and let An = {xe£; /(x) > 1/n}. Because
y4 = (J"=i An it is enough to show that fi(A„) = 0. By Chebyshev
■*"J,
Another property we can now prove is that the integral behaves nicely on
disjoint unions of sets.

§2.2 The Lebesgue Integral
67
Theorem 11. Let / be a nonnegative measurable function on X. Let Al9A2,
... be a sequence of pairwise disjoint members of #\ Let A = \JfLx Ar Then
f/«*#« = £ f -
(7) /<*/•=! /<fc
J A i-1 Ji4,
/V00/ First we prove the theorem in the case that f(x) = lE(x) for some
Ee#\ In this case
1Edfi = ^(y4 n £) and \Ed\i = fi(A( n £)
By countable additivity of ^, we know that
That is,
r _?° r
f Ma* = Z f :
•M *=1 .M,
By the linearity of IA and J^., the theorem is now automatically true for all
simple functions. We prove the general case with / being an arbitrary non-
negative measurable function as follows.
For e > 0 pick a simple function s < f with
[■
fdfj. < IA(s) + e
Because the theorem is true for simple functions, we have that
«-l 1=1 J A,
[ fdn<± [ ,
J A « = 1 J At
so fdfj. < X | fdfi + s
JA i=
Because £ is arbitrary, we must have
(8) f fdfi < £ I fdfi
\fdn<t \ .
J A i=l jAt
To prove the opposite inequality, first consider two disjoint sets Al9A2e SF.
Let sx and s2 be simple functions with 0 < sf < / and
Sidfi> ,
JAi jAt
(9) | s(dfi> | fdii-1 for/=l,2

68
Chapter 2 Integration
Let s = max(5x, s2); then 0 < s < f, and s is a simple function. Also sx < s and
s2 < s, so we can replace s( by s in inequality 9; namely,
Sdfi>\
JA{ JA{
/d\i — - fori =1,2
Adding these inequalities gives
or
Jsdfi+ sdfi> fdfi+ fdfi — i
A\ JA2 JAi JA2
sdii>\ fdii+ I fdii-i
JA JAx JA2
because the theorem is known to be true for simple functions.
Now iAfdfi>jAsdfi9 so
I fdfi> \ fdti+\ fdp-i
JA JAV JA2
Because e is arbitrary,
f fdti> \ fdgi+ f fdii
JA JAi JA2
Thus we have the inequality for A1 and A2. An induction argument gives
(10) f fdn>t \ fdfi
JAlvA2yj-~vAn i=l JAi
Returning to the general situation—that is, A = \jf=i At—we have
[fd»>[ fdfL
J A JAiKJA2KJ'~KJAn
because Ax u A2 u • • • u An c A. Hence, by inequality 10,
fd\i for all n
[ fdn>t I ,
JA «=1 JAi
That is,
fdfi
[ fdn>t [ -
J A *=1 JAi
Application. Theorem 11 tells us that we can use the integral to define new
measures. For example, we define Gaussian measure, \iG, on the measurable

§2.2 The Lebesgue Integral
69
subsets of R by
(11) /xG(A) = -Lf e~x2*diiL
x/27r ja
Theorem 11 says that this is countably additive and so is indeed a measure.
In fact, \iG is a probability measure because
'in Jr
ldfiL= 1
The theorem also has the following corollary.
Corollary 12. Suppose / and g are nonnegative measurable functions and
Ee#\ Then, if / = g a.e. on E,
JE JE
gdfi
Proof. Let Ax = {x eE; f(x) = g(x)} and A2 = {xeE; f(x) / g(x)}. Clearly
Ax and A2 are disjoint. Moreover, by assumption fi(A2) = 0, so
JA2 JA2
gdfi = 0
Also $Al f dfi = JXi gd\x because / = g on Ax. Thus
fdfi= fdfi+ fdfi= gdfi+ gd\i= gd\i □
JE .Mi Ji42 .Mi JA2 JE
Exercises for §2.2
1. Let / be the unit interval. Show that
1
xdfiL = -
i 4
using only properties of the Lebesgue integral discussed in this section^
(Hint: Approximate x from above and from below by simple functions.)
2. Let J be the interval 1 < x < oo. Show that
IO-
+ oo
using only properties of the Lebesgue integral discussed in this section.

70
Chapter 2 Integration
3. Let (X, &9 fj) be a probability space. Let /: X -+ [0, + oo) be a random
variable (that is, measurable function). The integral
E = \xfdfi
is called the expectation value of / (or "most likely value" of f) and the
integral
K = £(/-JE)2**
is called the variance off Show that, if the variance off is small,/deviates
from its expectation value with very small probability. Explicitly, show
that the probability that / deviates by s from E—that is,
fi({xeX;\f(x)-E\>e})
is less than or equal to s~2V.
4. Let Hn be the number of heads occurring in the first n trials of a Bernoulli
sequence. Compute its expectation value and variance.
5. Consider the "random" series
< 1 1 1
~2~4~8~
with the assignment of a plus or minus in the nth term being decided by
the toss of a coin. Compute its expectation value and variance.
6. Let (X, #*, fi) be a measure space and/a nonnegative measurable function.
Forallfle(0,oo),let
Q>(a) = fi{{xeX;f(x)>a})
Suppose Jx/k<fy* < 00, k > 0. Show that there exists a constant C > 0
such that
0(a) < Ca'k
That is, show that $ goes to zero at least as fast as a~k as a -> +00.
7. Let J be a finite subinterval of the real line and f:J-+R a simple
function taking on values cx,..., cn. The function/is called a step function
if y "~1(c£) is a finite union of intervals for each i. Given a simple function
s: J -> R and a positive number e, show that there exists a step function/
such that
(*) j\s-f\diJiL<e

§2.2 The Lebesgue Integral
71
(Hint: Show that, if A is a measurable subset of J, there exists a finite
union of intervals B such that d(A, B) = fi(S(A, B)) < e. Now prove the
inequality (*) for s = 1A. Proceed.)
8. Let X be a countable set; that is,
X = {■X1,x2,x3,...)
and let Pl9P2,P3,... be a sequence of nonnegative numbers such that
£Pf= 1. For/I c* let
x{eA
We saw in §1.4 that \i is a probability measure on the cr-field of all subsets
of X. Prove that every function /:AT->Ru{±oo} is measurable, and
prove that, for / nonnegative,
I
X
9. Let /: R -> [0, oo) be measurable. Given aeR, let fa(x) = f(x — a). Show
that fa is measurable and that
{/.<*-/.
(Hint: See §1.3, exercise 14.)
10. Let (X, J*\ //) be a measure space and A and £ be measurable subsets of
X. Show that, if fx(S{A, B)) = 0, then, for every nonnegative measurable
function /,
1/*-1.
B
11. Let (X, #",/*) be a measure space and / and g positive measurable
functions. Show that, if g is simple, then, for all £e^,
\ (f + Q)dfx= I fdp+ I gdfi
jE JE JE
12. Let (X,^^) be a measure space with fi(X) < oo. Let / be a bounded
nonnegative measurable function and let {sn} be the sequence of simple
functions constructed in theorem 6. Show that, for Ee^,
sndfi^>
JE JE
sndfi^> fdfx
(Hint: The sequence sn converges uniformly to /. Moreover,

72
Chapter 2 Integration
\(f-sjdn+ I sKdn= I fdfi
JE JE JE
by exercise 11.)
§2.3 Further Properties of the Integral; Convergence
Theorems
The reader has probably noticed that the definition of the integral
does not make evaluation of integrals easy. The set of simple functions s with
0 < s < f is a formidably large set. In this section we will develop some
effective techniques for computing integrals and for manipulating integrals
and limits. The three key results are the monotone convergence theorem,
Fatou's lemma, and the Lebesgue dominated convergence theorem.
Let fx, f2,... be a sequence of measurable functions with
0</i</2<- -
Note that / = lim^^ f( exists and is measurable.
Theorem 1. (Monotone convergence theorem) Let / and fh i= 1,2,3,..., be
as above. Then
(1) I /4i = lim| Jfrfp forEe^
JE i~*oo JE
To prove this theorem we need the following lemma.
Lemma 2. Let / be a nonnegative measurable function on X and let El9E2,
E3,... be a sequence of sets in & with Ex c E2 a E3 c • • •. Set E = (Jji1 Et;
then
f fdfi = Km(
JE i-*x>jEi
fdfi
Proof. Let At = £t
A2 = E2 - Et
A3 = E3- E2
and so on. Then the ^4,'s are pairwise disjoint and
0 At = E and Q At = E„

§2.3 Further Properties of the Integral; Convergence Theorems
73
So by countable additivity
fdn
\fdn=t \
JE i=l JA{
>=i Ja,
= lim X | fdn
n-*oo £=1 ,
= lim fdfi
We now prove the monotone convergence theorem. We have
0<fi<f2<-<f=limfn
By monotonicity
\ fidfi< I f2dfi<-'< I fdix
JE JE JE
so lim^oo J£/nd^ exists and must be less than or equal to §Efdti. Let a =
lim,,.^ J£/nrf^. We must establish that
(2) a > [fdfi
Let s be any simple function, with 0 < s < /, and let c e R, with 0 < c < 1.
Let
En = {xeE;fn(x)>cs(x)}
Notice that Ex <= E2 <= • * * because ft < f2 < * * *. Also notice that U"=1 En =
E. Indeed, if x e 2s with s(x) = 0, then xe£„ for all n, and, if x6E with s(x) ^ 0,
then because c < 1
fix) > s(x) > cs(x)
So, for some n, fn(x) > cs(x) because
that is, xeEn.
Taking integrals, we get
a=lim fnd\i> fnd\i> fnd\i> csd\i
n^cojE JE JE„ JEn
because fn> cs on E„. We apply the lemma to E = \J™=X En to get

74
Chapter 2 Integration
a > lim cs(x)dfi= cs(x)dfx = c s{x)dfi
n-*oojEn JE JE
Because this is true for all c with 0 < c < 1, it must also be true that
a > sdfi
By now taking the supremum over all simple s with 0 < s < f, we get
inequality 2. □
Remark. Thanks to the monotone convergence theorem, we can apply
theorem 6 of §2.2 to the evaluation of integrals. If / is a nonnegative
measurable function and sn is the sequence of simple functions constructed in theorem
6 of §2.2, then
(3) j\**-> [ fdn
JE JE
We will use this formula to clear up some matters that we left dangling in §2.2.
Theorem 3. Let / and g be nonnegative measurable functions and let c > 0
be a real number. For E e $F we have
1.
2.
-I
= { fdn+[
JE JE
cfdfi = c fdfi
E JE
(f + g)dfi= I fdn+l gdfi
E JE JE
Proof.
1. The first part is clear because we know that, if s > 0 is a simple function,
then IE{cs) = dE(s) and also that s < f if and only if cs < cf.
2. Again we know that, ifs1 and s2 are nonnegative simple functions, then
h(si + s2) = /£(sx) + IE(s2)
Now, choose a monotone sequence of simple functions
0 <s2 <s2 < *••
with sn-+ f pointwise. Similarly, choose simple functions
0 < s'i < s'2 < • • •
with s'„ -► g pointwise.

§2.3 Further Properties of the Integral; Convergence Theorems
75
Note that
0 < s1 + s\ < s2 + y2 <
and s„ + s'„^ f + g pointwise. By formula 3
(f + g)dfi= lim (sn + s'„)dfi
n-*oo Je
| (/ + «7)^=lim I
JE n-*ooji
= limf s„dfi+ s'ndn)
n-oo \JE JE /
= lim sndfi+ lim ^ d/i
n-*aojE n-*aojE
/<^ + ^ n
Corollary 4. Let /i,/2,... be a sequence of nonnegative measurable
functions. Then ^Jij ./j is a nonnegative measurable function and
f fl/.)^= E f/„^ for£e^
j£\n=l / n=l j£
/V00/I Let F„ = ££=j fk. Then Fx < F2 < • • •, and the F„'s are all measurable.
Now apply the monotone convergence theorem and theorem 3. □
So far we have integrated only nonnegative measurable functions. When
extending our definition to more general measurable functions, we must
beware of the problem of adding + oo to — oo.
Let / be an arbitrary measurable function from X into the extended real
numbers. Recall that
/+ = max(/,0) and /_ = max(-/,0)
are nonnegative measurable functions with
/ = /+-/-
Lemma 5. The following two conditions are equivalent.
■•I'
1. |/| dfi < oo
I. /+ d\i < oo and I /_ d\i < oo
JE JE

76
Chapter 2 Integration
Proof. Notice that \f\ = f+ + /_, so
(4) [ \f\dfi= [ f+dii+ \ f.dti V
J E J E J E
Definition 6. A measurable function / is integrable over E if either of the
equivalent conditions of lemma 5 holds. In this case we write fe «£?(/*,£) or
/ g S£(\x) on E. If E = X we write / e J^(/x). For / e S£(\i, E) we define
(5)
[ fdfi= \ f+dix- [ f.dti
J E J E J E
Theorem 7. (Linearity) Let /, g e £f(n, E) and c e R. Then
a. cfe£f{fi,E) and c/d/i = c /d/x
Je Je
b. f + ge&(n,E) and (f + g)dfi = fdfi + gdfi
J E J E J E
Proof.
a. If c > 0, then c{f+) = (c/)+ and (c/)_ = c(/_), so
cfdfi= cf+dfi— cf-dfi
JE JE JE
= c f+dfi-c\ f.dfi
JE JE
= c fdfi
A similar argument treats the case of c < 0.
b. Let h = f + g. We first deal with the special case that none of /, g9 or h
changes sign on £. The six subcases are:
1. />O,0>O,fc>Oon£
2. /<O,0<O,fc<Oon£
3. / > 0, g < 0, h > 0 on E
4. f>0,g<0,h<0onE
5. f <0,g>0,h>0onE
6. /<O,0>O,fc<Oon£
Case 1 has been dealt with previously. Case 2 can be reduced to case 1
because we can rewrite the formula as

§2.3 Further Properties of the Integral; Convergence Theorems
77
f (-*)«! = f (-/)*!+ \(-9)dil
JE JE JE
Case 3: rewriting h—f+g as f = h + (—g) reduces this to case 1.
Similarly, cases 4, 5, and 6 can be reduced to case 1.
Now to complete the proof we write E = E1u E2W"Kj E6 so that Et
is the set for which case i holds, i= 1,2,...,6. Then, because jEfd^ =
Zf=i Je,/^A*> anc* similarly for g and h, the theorem follows by applying it to
each E( separately and summing. □
Corollary 8. (Monotonicity) Let /, g e S£(/x, £) with / < #. Then
(6)
[ fdli< [
JE JE
gdfi
Proof. Because / < g, g — f > 0, so
0< J (g-f)dfx= J gdii- \ fdfi
JE JE JE
by linearity.
Corollary 9. life $£ (/x, £), then
(7)
\ fdix\< \
JE I JE
\fW
Proof. Because / < \f\, by monotonicity
[ fdn* \\f\dp
JE JE
Similarly, -/ < |/|, so
-f/<*/*< f |/| i|i
Je Je
That is,
1
fdp\<Z {\f\dft
E | JE
D
We end this section with the other convergence theorems mentioned
above.

78
Chapter 2 Integration
Lemma 10. (Fatou's lemma) Let /i,/2,... be a sequence of nonnegative
measurable functions, and let / = liminf/,. Then
(8) I /^<liminf| f^
JE JE
Proof. Let gk = infn^k /„ and ak = inf„^ JE fn d[i. Then
gi<g2^'" and al^a2^-"
By definition
/ = lim gk and lim inf f„dfx = lim ak
k-*oo JE k->ao
Notice that ak > \E9k dp because gk < fn for n > k. Hence, by the monotone
convergence theorem,
fdfx = lim gkdp < lim ak = liminf fkd[i D
JE k->aojE k->ao JE
Theorem 11. (Lebesgue dominated convergence theorem) Let /i,/2>/3»--
be a sequence of measurable functions, and let EefF. Assumptions:
1. limn_>00 f„(x) exists for all xeE.
2. There is a nonnegative measurable function ge^(fiyE) with g > \fn\ on
£,n = 1,2,....
Conclusion: The function f(x) = limn_>00 fn(x) is integrable and
lim fndfi = lim fndfi
j£n->co n-+cojE
Proof. By Fatou's lemma,
liminf|/n|d/x
\\f\dix= [ lir
< lim inf \fn\dfi<\ gd\x
Hence fe&(fi,E).
Now we note that # -b /„ is nonnegative, so by Fatou's lemma
I lim inf(g + Qd\x < liminf | (g + f„)d/i
JE JE

§2.3 Further Properties of the Integral; Convergence Theorems 79
But liminf(0 + /„) = g + lim /„ = g + /
and liminf {g + fn)dfj.= gd\x + liminf fnA\i
JE JE JE
so we have fd/i< liminf fnd\i
JE JE
Repeating this argument, with g + /„ replaced by g — /„, we get
- fdfx < liminff - fndfi)
= -lim sup ( fndfi)
so fdfi > lim sup! f„dfi\
Combining these results
limsupM fndfi\< /d/i<liminf( fndA
But it is always true that lim inf < lim sup, thus we get equality:
/<//* = liminfM /n^J = limsupM /nd/xJ = limM fndA D
Corollary 12. Let fx ,/2,... be a sequence in S£(fi9 E) with
\dfi < oo
2 f i/.i
n=l JE
Then
!• Z^=i fn converges absolutely a.e. on E and is integrable on E.
2. j£Zr=i/„^ = Z„°°=ijE/„^.
Frw/. Let g = £*=1 l/J. Corollary 4 tells us that
\ Gdfi= £ \fn\dfi< oo
JE «=1 JE
so geJ?(fi,E). In particular, # is finite a.e. on E, so Z^=i/n converges
absolutely a.e. on E. To prove part 2 let F„ = XZ=i /k- Then I^J ^ Z*=i l/*l ^

Chapter 2 Integration
g so, by the dominated convergence theorem, Y*=ifn = ^mn-*oo^n *s *n
j£f(A*,£)and
{ (tfn)dv= f limFn^ = lim f /*,<*/*= £ [ f.d/i D
jE\n=l / j£n-»oo n-»oo J£ «=1 j£
Exercises for §2.3
1. Let (X, «^", /*) be a measure space and / be a bounded nonnegative
measurable function. Show that
This formula is Lebesgue's original definition of the Lebesgue integral.
(Hint: See formula 3, page 74.)
2. Let/„: R -► R be 1/n times the characteristic function of the interval (0, n).
Show that/, -► 0 uniformly but that ]"/„ d//L = 1. Why isn't this example a
counterexample to the Lebesgue dominated convergence theorem?
M
Compare J lim inf /„ &\iL and lim inf \fn d[iL for the sequence /„ in exercise
2. Can the inequality in Fatou's lemma be replaced by an equality?
For n = 1,3,5,... let fn be the characteristic function of the interval (0, \\
and for n = 2,4,6,... let /„ be the characteristic function of the interval
(^, 1). Compare J lim inf/„ &\iL and lim inf \fn dfiL.
Let (X,^,fi) be a measure space. A measurable function f:X->R is
mean-square integrable if \f2dn < oo. Show that, if pi(X) < oo, every
mean-square integrable function is integrable. (Hint: Consider separately
the integral of |/| over the set where |/| > 1 and the integral over the set
where |/| < 1.)
Let X = /, ^ the Borel sets, and \i Lebesgue measure. Show that there
exists an integrable function on X that is not mean-square integrable.
Let X = R, 3F the Borel sets, and \i Lebesgue measure. Show that there
exists a mean-square integrable function on X that is not integrable.

§2.3 Further Properties of the Integral; Convergence Theorems
81
8. Let (X9^,fi) be a measure space and /„, n = 1,2,..., a sequence of
measurable functions. Then /„ is said to converge to zero in measure if,
for all e > 0,
fi({xeX;\fn(x)\>e})^0
as n -► oo. (Compare with exercise 11 of §2.1.) Show that, if j \fn\ &\x -► 0,
then fn converges to zero in measure. Show that the converse is not true.
(Hint: See exercise 2.)
9. Let /: R -► R be an integrable function. Show that, if
J/*1 = °
for every subinterval J of the real line, then / = 0 a.e.
10. Let J be a finite subinterval of the real line and /: J -> R an integrable
function. Show that for every e > 0 there exists a step function g such that
\f-g\dn<e
(Hint: See §2.2, exercise 7.)
11. Let (X,^,fi) be a measure space and / and g measurable functions.
Show that, if/ is integrable and g is bounded and measurable, then fg is
integrable.
12. Let (X^.fi) be a measure space and / a nonnegative measurable
function. For Ae^let
l*AA) = \ fdf*
Show that fif is a measure on &. Moreover, show that, if g is a nonnegative
measurable function, then for Ee^
9dnf= gfdfi
JE JE
13. a. In exercise 12, let A e SF and let / be the characteristic function of the
set A. Describe the measure \if.
b. Suppose, more generally, that / is a simple function; that is
<*) f=iciiAl
i=l
Describe the measure \if.
14. a. In exercise 12, show that, if/is bounded, S£(E,\if) contains S£(E,\ji).
Moreover, show that, for g e £f(E, /x)

82
Chapter 2 Integration
9dflf= gfdfJ,
JE JE
b. Show that S£(E,\xs) need not necessarily contain £f(E,fi) if / is not
bounded.
15. a. Let (X,^,fj) be a measure space and f:X^>R a measurable
function. For every Borel set ^^R, let
vf(A) = tif-H*))
Show that this formula defines a measure vf on the Borel sets of R.
Moreover, show that, if/us a probability measure, so is vf.
b. If / is the function (*), describe the measure vf.
16. A function g: R -» R is Borel measurable if, for every Borel set A ^ R,
g~x(A) is a Borel set. Let vf be the measure in exercise 15 and let g be a
nonnegative Borel-measurable function on the real line. Show that
(**) gdvf = \ g(f(x))dfi
Jr jx
(Hint: What does equation (**) say when g is a simple function?)
§2.4 Lebesgue Integration versus Riemann Integration
By the results of §2.3 we can now integrate complicated limits and
sums of series. What about simple integrals? We will show that in the Lebesgue
theory, just as in the Riemann theory, these integrals can be evaluated by the
second fundamental theorem of calculus; that is, for the Riemann integral one
has the following theorem.
Theorem. Let g be a continuous function on an interval [a, fc] c R. Then g
has an antiderivative G and
(bgdx = G(b)-G(a)
where $9dx denotes the Riemann integral of g on the interval [a,fc].
We will show that the same is true for the Lebesgue integral when our
measure space (X, 2F, \x) is X = [a, b\ \i = \xL, and !F is the field of Lebesgue-
measurable subsets of X. Rather than prove the fundamental theorem of
calculus directly for the Lebesgue integral, we prove a much more general
theorem, as follows.

§2.4 Lebesgue Integration versus Riemann Integration
83
Theorem 1. Let / be a bounded Riemann-integrable function on [a, ft] with
Riemann integral \bafdx. Then fe «£?(/xL, [a, ft]) and
(1) [bfdx=\ fdfiL
Ja J[a,b]
Before proving this theorem, let us recall how the Riemann integral is
defined.
Riemann Integral
A partition P of [a, ft] is a finite, ordered sequence of points
a = x0 < xx < x2 < * • * < xN = ft
The maximum of the numbers xi+l — xh i = 0,..., N — 1 is denoted m(P) and
is called the mesh width of the partition.
Fix a partition P of [a, ft], and let / be a bounded function on [a, ft]. Let
and
Define
Mi = sup{/(x); Xj-x < x < xt}
mt = inf{/(*); xi^1 < x < xt}
U(fP)=tMi(*i-Xi-i)
and
L(fP)=fjmi(xi-xi.1)
i=l
to be the upper and lower Riemann sums, respectively.
Notice that L{fP) < U(fP). In fact, it is true that, if Px and P2 are any
two partitions of [a, ft], then L{f Px) < U{f P2). To see this we introduce the
notion of refinement. A partition F is called a refinement of P if the ordered
sequence of points comprising F contains the points of P as well as some
additional points; that is, F = P plus additional points. Clearly, if F is a
refinement of P, then L(/, F) > L(f P) and U(f9 F) < U(f P). Now, if Px and
P2 are two partitions of [a, ft], let P be a partition of [a, ft] that refines both
Px and P2 simultaneously. Then
(2) L(f, P,) < L(/, P) < U(f, P) < U(f, P2)

84
Chapter 2 Integration
We define
fdx = inf { [/(/, P); P is a partition of [a, fc] }
(3)
r t
fdx = sup {L(/, P); P is a partition of [a, fc] }
Note that by inequality 2
(4) [ fdx< [fdx
Ja Ja
Definition 2. The function / is Riemann integrable if JJ fdx = \baf dx. In this
case we define
(5)
I fdx = I fdx = I fdx
Ja Ja Ja
To compare the Riemann integral with the Lebesgue integral, we will first
show the following lemma.
Lemma 3. There exists a sequence of partitions P1,P2,P3,... such that
1. Pk is a refinement of Pfc_ x.
2. The monotone sequence
U(/,i\)£l/(/,P2)*"'
converges to \bafdx.
3. The monotone sequence
Uf,Pi)*L{f,PK'
converges to \hafdx.
4. The mesh width m(Pk) -> 0 as k -> oo.
Proof. We describe how to define Pk. Let P^ be a partition with
i/(/,ifl<j'V*+£
and P£ a partition with

§2.4 Lebesgue Integration versus Riemann Integration
85
uf,pn*jl
fdx-k
We know that J^ and Fk' exist by the definition of \bafdx and \bafdx. Let Pk
be a partition with m(Pk) < 1/fc refining Pk, Pk, and P*^ all at once. Then
L(f,Pk)>L(f,Pu)>^fdx-1-
U(f,Pk)<iV(f,Fk)<jbfdx + l-
lim L(/,P,)= | Vdx and WmV(f,Pk)= [" fdx V
k-+oo Ja k-*co Ja
and
So
For a fixed partition P, define the simple functions
f(a) at x = a
m1 on x0 < x < xx
Lp{x) = ^ m2 on xx < x < x2
and
mN on Xx-i < x < xN
Cf(a) at x = a
Mx on x0 < x < xx
t/P(x) = < M2 on xx < x < x2
MN on Xpf-i < x < xN
Notice that L (x) < f(x) < UP(x) and
I LP(x)dfxL = £ ^(x, - x,.!) = L(/,P)
J[a,b] i=1
See figure on page 86.

Chapter 2 Integration
Graph of/
C mn
xQ = a x,
*3
x=b
f ™3
Graph of LP
xQ = a xx x2 x3
xn-\ Xn — b
Also, if F is a refinement of P, then
£/P<(x) < UP(x) and LP,(x) > LP(x)
Now choose a sequence of Pk's as in lemma 3 and let
Lk = LPk and Uk = UPk
Then
L1 < L2 <-</<"< U2 < Ux
Let L(x) = linv^ Ln(x) and ir(x) = lim,,.^ Un(x). By construction, L(x) <
f(x) < U(x), and, by the dominated convergence theorem,

§2.4 Lebesgue Integration versus Riemann Integration
87
and
lim LndfxL= LdfiL
n->ao J[a,b] J[a,b)
lim f UmdpL=[ UdfiL
n->co J[a,b] J[a,b]
Now we know that
f LndfxL = L(f,Pn) and ! UndfiL= U(fPn)
J[a,b] J[a,b]
and we chose the P's so that
lim L(fPn) = [fdx and lim !/(/,PJ = [ fdx
Hence, we conclude
LdnL=\ fdx and Ud/xL= \ fdx
J[a,b] Ja_ J[a,b] J a
To prove the theorem, we assume that / is Riemann integrable; that is,
p r*
fdx = fdx
Ja Ja
In other words,
Ld\iL= UdfiL= fdx
J[a,b] J[a,b] J a
Thus i[a,b](U - V)d\xL = 0. But U > L so U - L > 0; hence, U - L = 0 a.e.
Now L<f<U so f=U = L a.e.
and / is Lebesgue integrable with
f fdliL=[bfdx
J[a,b] Ja
Remark. The standard notation for the Riemann integral of a function /
over an interval [a, b] is the notation we have been using—namely,
(6) f(x)dx
I

88
Chapter 2 Integration
Unfortunately, the Lebesgue integral has no such standard notation.
Heretofore we have been using the notation
(7) f fdfiL
J[a,b]
or
(8) jV*iL
with / denoting the interval [a, ft].
Now that we've shown that the Lebesgue integral and the Riemann
integral are identical for Riemann-integrable functions, we will be less
methodical with our notation. We will sometimes use display 6 for Lebesgue
integrals and will sometimes use displays 7 and 8 with the subscript deleted
from \iL when it is clear from the context that \i is \iL.
Exercises for §2.4
1. Compute the Lebesgue integral
of the following functions: x2, x3, sinx, ex, xlogx. (You are encouraged to
use the tools of elementary calculus in making these computations.)
2. In the proof of theorem 1, choose an x e [a, ft] that is not equal to any of
the partition points of any of the Pk's. Show that / is continuous at x if
and only if U(x) = L(x).
3. Conclude from exercise 2 that, if a bounded function / on the interval
[a, ft] is Riemann integrable, then it is continuous almost everywhere.
4. a. Conversely, suppose that / is a bounded function that is continuous
almost everywhere on the interval [a, ft]. Conclude from exercise 2 that
]imU(f9Pk) = limL(f9Pk)
b. Deduce that, if / is bounded and continuous a.e. on [a, ft], then it is
Riemann integrable.
5. (Improper integrals)
a. Let / be a nonnegative measurable function on the interval J = (0,1].
Suppose that / is Riemann integrable on all of the intervals [a, 1],
0 < a < 1. Show that
fdpL = lim f(x)dx
J J a->0Ja
with the integral on the right being the Riemann integral.
b. Compute the Lebesgue integral over J of the function f(x) = 1/^/x.

§2.5 Fubini Theorem
89
6. a. Let / be a nonnegative measurable function on the interval J = [1, oo).
Suppose that / is Riemann integrable on all of the intervals [1, a],
a > 1. Show
fdnL = lim f(x)dx
b. Compute the Lebesgue integral over J of the function /(x) = 1/x2.
7. Let /(x) = (sin x)/x. Show that
lim f(x)dx
a-*°o J i
exists. On the other hand, show that f(x) is not Lebesgue integrable over
the interval [l,oo).
8. Let /: R -+ R be continuous and have a continuous first derivative /' that
is positive everywhere. If \i is Lebesgue measure on R, show that the
measure vf of §2.3, exercise 15, is of the form
vf(A) = j" 0<fo where l/g(x) = /'(/"'(*))
§2.5 Fubini Theorem
Section 2.4 and the convergence theorems of §2.3 allow us to
compute a number of integrals on subsets of R. In order to compute integrals
on subsets of R", we must justify the use of iterated integrals. This is the
purpose of Fubini's theorem.
The general situation is as follows: Let {X,Jt,fji) and (y,«yT,v) be two
measure spaces. Let X x Y denote the space
(1) *x r={(x,j/);xe*,j/ey}
If A a X and Bc7, then A x B (= X x Y. On the other hand, you should
notice that most subsets of X x Y are not of this form.
Definition 1. A x B <= X x Yis called a product set \iAtJt and Be Jf. The
smallest a-field in X x Y containing all product sets A x Bis denoted M x
jV.
In a moment we will define a measure on Jt x Jf. First we must describe
sets in J( x Jf in terms of J( and JT,
Definition 2. For E <= X x Y and xeX fixed, let Ex = {ye Y; (x,y)eE}.
Then Ex is called the x-slice of E.

90
Chapter 2 Integration
Notice that, if E and F are subsets of X x Y, then
(EnF)x = ExnFx
(2) (E-F)X = EX-FX
K = (Ec)x
and, if Ex, E2, E3,... are subsets of X x 7, then
\n=l /* n=l
Proposition 3. If E e ^# x Jf, then £x e Jf.
Proof. Fix xeAT and let £fx be the collection of all sets E ^ X x Y with
Ex e Jr. Note that
1. Sfx contains all product sets A x B.
2. Sfx is a (7-field.
Thus ^. contains the smallest cr-field containing all product sets—namely,
Jt xJf. □
Corollary4. Let f:Xx y-»Ru{±oo} be measurable with respect to
Jt x Jf. For x0eX fixed, define fXo: 7-> R by /Xo(.y) = /(x0,.y). Then, for
each x0eX, fXQ is a measurable function on Y
Proof. Fix x0 e AT. If a e R we need to show that
{yeY;fXo(y)<a}ejr
Let £ = {(xj)el x Y;f(x,y) <a). EeJt x Jf because / is measurable
and {yeY; fXo(y) < a} = EXoeJf by proposition 3. □
Remark. We could just as easily have studied y-slices as x-slices. The
corresponding proposition and corollary are obviously true for j/-slices as well.
Thus far we have made no assumptions about the measure spaces (X, Jt> fi)
and (7, JT, v). We will now assume that both of these spaces are a-finite. (Recall
that a measure space (X, Ji^ fi) is cr-finite if there exist X/ e ^#, i = 1,2,3,...
with /i(Xt) < oo and
0 xi=x
(See definition 32 in §1.3.)

§2.5 Fubini Theorem
91
We now use the measures \i and v to define a measure on Ji x Jf. First
let's recall how one computes the areas of regions in the plane in elementary
calculus. Consider the region in the figure below.
For each point x on the interval [a, 6], we let <f>E(x) be the length of the interval
Ex\ that is, <t>E[x) = y2 — y^. For reasonable-looking regions, such as the one
we've drawn here, <f>E{x) is a continuous function of x, so the Riemann integral
(3)
r
<f>E(x)dx
is well defined. In elementary calculus one proves that display 3 gives the area
of the region E (or uses the integral as the definition of area). To define a
measure on the product cr-field, J( x Jf, we will mimic this process. Let
£eJ x ./. For each xeX,ExeJf, so we can define a function </>E: X -► R by
(4) fc(x) = v(Ex)
We would like to define the measure of E to be the integral
(5)
J.
<f>E(x)dfx
To use this, we have to check that <f>E(x) is a measurable function on X. Proof
of this fact requires a general argument about tx-fields called the n-k theorem.
Definition 5. Let Z be a set and let Sf be a collection of subsets of Z. £f is
called a X-system if the following three properties hold.
312. If £iC£2C£3c
is an increasing sequence with each Ene£f, then
X3. If £, Fe^ and £ c F, then F - Ee^.

92
Chapter 2 Integration
Definition 6. A collection s# of subsets of Z is called a n-system if
Til. ,4, Bes/^AnBes^.
Theorem 7. (7r-A theorem) If $f is a A-system and ^ is a 7r-system with
s0 <^¥, then the smallest <7-field containing j^, <r(«s/), is contained in £f.
Proof.* First notice that, if a collection / of subsets of Z is both a A-system
and a rc-system, then it must be a cr-field.
Now let l(s/) be the smallest A-system containing s/. If we can show that
l(srf) is also a 7r-system, then we will know it is a cr-field. Because <r(jtf) is the
smallest a-field containing stf and l(s#) is the smallest A-system containing j/,
we will have
which proves the theorem.
To see that l(s/) is a rc-system, we need to show that if A,Bel(sf) then
AnBel(sf).
For 4 c Z let
^ = {5cZ;in5Gi(i)}
Notice that if ,4 e /(j</), then ^ is a A-system. Indeed, the three properties
are satisfied as follows.
JUL If A e /(j/), then Ze^, because /I n Z = /I.
X2. If Ael(s#) and Ft <= F2 cr F3 c ••• are in ^, then (A n Fx) c (A n F2) c
• • • is an increasing sequence in l(s/)9 so (J*=1 (A nEn) = A n (U*=1 F„) is
also in J(j^); that is, (J~=1 £ne^.
X3. If Ael(st) and F, Fe^ with E c F, then XnF, infGl(i), and
(yln£)c(inF). So >4 n(F - F) = (>4 nF) - (A n£)e!(4 that is,
F-Ee%.
Furthermore, HA, Be srf, then AnBes/ a /(j</), soBe&A; that is, s/ a^A
when y4ej^. Thus we see that J(j</) <= &A when y4Gj^, by the minimality of
l{s/). In other words, we now know that, if A e st and B e l(st\ then An Be
Thus, if B e l(sf) we have shown that st c <&B. Again using the minimality
of Z(jaO, we get l(s/) c <§B for B e \(s4)\ that is, An Be l(s/) when A and 5 are
elements of l(jtf). This is property n\ for /(.s/). D
With the 7T-A theorem we can now prove the following proposition.
♦This proof is rather technical, and you might want to skip it when reading this material for the
first time.

§2.5 Fubini Theorem
93
Proposition 8. If E e Ji x Jf and <j>E: X -* R is defined by ^£(x) = v(£x), then
<j>E is measurable.
Proof. First assume that v(7) < cxd. By the n-X theorem, the proposition
will follow from the following three facts.
a. Let Sf be the collection of sets E such that </>E is measurable. Then Sf is a
A-system.
b. All product sets, A x B, with Xeif and 56^T, are in ^.
c. The collection j</ of product sets is a 7r-system.
We prove these statements as follows.
a. XI. <j>x x y(x) = v(y) for all x; hence, is clearly measurable.
12. Let Ex £ E2 e • • • be an increasing sequence with En e Sf\ that is, </>En(x)
is measurable. Let £ = (j£=i £„. We need to show that <f>E(x) is
measurable.
Now
+b(x) = v(EJ = v[ Q (£jj = lim v[(fiJJ = lim ^(x)
Thus ^£ is the pointwise limit of measurable functions and so is
measurable as well.
X3. Let E,Fe$f with E cz F. Then F = E u (F - E) is a disjoint union,
and Fx = Ex u (F - E)x is also disjoint. Thus v(Fx) = v(Ex) + v[(F -
E) J; that is
Because </>E and ^F are measurable, <f>F-E is too.
b. Let £ = Ax5bea product set. Then
_fJ5 ifx
~\<f> ifx
Gi4
so
(6) «x)-io ifxM
That is, <f>E(x) = v(B)lA(x)9 which we know to be measurable.
c. Let A x B and A' x 27 be two product sets. Then
(A xB)n(A' x B') = {Ac\A')x(Br\B')
so the collection of product sets is a 7r-system.

94 Chapter 2 Integration
Now, what happens if we drop the assumption that v{Y) < oo? Because Y
is c-finite we can find a sequence of subsets Yl9 Y2, Y39... of Y with Y{eJf,
v{Y^ < oo, and
00
^ = U ^ (disjoint union)
i=l
LetEe^T x Jf. Then
00
E = (J ££ (disjoint union)
i=l
where £< = E n (X x Yt) and
(7) ^(x) = f ^,(x)
i=i
We proved above that ^£. is measurable, so by equation 7, ^£ is measurable.
□
We can now use display 5 to define a measure on Jt x Jf.
Definition 9. Let EeJtxJf and define
(8) tt'(£) = j^£(x)d/i
to be the product measure of E.
Proposition 10. n' is a measure.
JV00/I We must check that ri is countably additive. Let EuE2>... be a
pairwise disjoint family of sets in Jl x ^T. Let £ = (J"=1 £„. We need to show
that
n'(E)=fjn'(En)
»=i
Now, Ex = (J^ (En)x is a disjoint union, so
v(Ex)= Zv[(£„)J
n=l
00
that is, <j>E(x) = £ <t>En{x)
n=l
We can now apply the monotone convergence theorem to get

§2.5 Fubini Theorem
95
JX n=ljx
that is, n\E) = g *'(£„) □
n=l
Notice that we could repeat this whole procedure using ^-slices instead of
x-slices. Ostensibly this method gives a different measure n" on Jt x Jf. Our
first version of Fubini's theorem is that these two methods yield the same
measure.
Theorem 11. (Fubini, version 1)
n = n
Proof. First assume that ix(X) and v(7) are finite. By the %-X theorem, it is
enough to establish the following three assertions.
a. Let Sf = {E e Jt x JT; n\E) = n"(E)}. Then Sf is a A-system.
b. The product sets are in Sf.
c. The product sets form a 7c-system.
We proved assertion c above. Let us look at assertions a and b.
a. U. n\X x Y) = ix(X)v(Y) =n"(X x Y)9 so X x Y e#>.
\2. Let Ex s £2 c • • • be an increasing sequence with F„ e £f. We wish to
show that E = (J*=1 En eSf. It is clear that lim^^ n'{En) = rc'(F) and
\imn^Q07i"(En) = n"(E). But te'(F„) = n"(En\ because EneSf. Hence
7c,(£:) = 7c"(£:);thatis,£G^.
3l3- Let E.FeSf with F c F. Then F = E u (F - F) is a disjoint union,
so n'(F) = 7c'(F - F) + te'(F) and n'\F) = te"(F - E) + tt"(F). Hence
^(F - E) = tc"(F - F), because tc'(F) = tc"(F) and n\E) = n"(E).
b. Let E = y4 x B be a product set. Then ^£(x) = v(jB)1^(x), so
n\E)=[<t,E(x)dn = v{B)n(A)
Reversing the procedure gives
tt"(F) = n(A)v(B)
ThusFe^.
Now, what happens if fi(X) and v(7) are not finite? Because X and 7 are
<7-finite, we can find subsets XteJt,i = 1,2,3,..., and Yje Jf, j = 1,2,3,...,
such that fi(Xi) and v(l^) are finite and

96
Chapter 2 Integration
X = [j Xi (disjoint union) and Y = [j Yj (disjoint union)
MEeJt x J^let
Then
00
(9) E= (J £u (disjoint union)
By what we proved above,
so, by equation 9,7c'(JB) = n"(E). D
Definition 12. The measure n' = %" is denoted p x v and is called the product
measure on Jt x Jf.
Example 13. Let X = Y = R and ^# = .yf = &l9 the Borel sets in R. Also
let ju = v = pl9 Lebesgue measure on R. We claim that M x Jf = 08 2, the
Borel sets in R2, and p x v = p29 Lebesgue measure on R2.
Proof. Notice that, if J and J are intervals in R, then J x J is a product set
so J x J g M x Jf. Now, $b2 is the smallest a-field containing sets of the form
J x J, so $2 c Jl x tAr.
We now show Jt x Jf c <%2\ Fix an interval / c R. Let
^ = {5cR;/x Be@2)
Note the following.
1. ^7 contains all intervals JcR.
2. Jfj is a <7-field. Indeed, suppose Bl9B29... are elements of 0bl9 then
/xfQlj/U Q(/xBn)e^2
\n=l / n=l
Because Si1 is the smallest <7-field containing the intervals, items 1 and 2
imply that SlcSII\ that is, \{Be08l we have I x Be@2.
Now fix a Borel set Be£x. Let ^B = {A c R; A x 5e^2}. Note the
following.

§2.5 Fubini Theorem
97
1. @IB contains all intervals /cRby the above result.
2. @lB is a <7-field by an argument just like the one above.
Because ^\ is the smallest a-field containing all of the intervals, we have
^i <= ®b\ that is, A x Be@2 for any A, Be^x. But ^ x ^ is the smallest
t7-field containing all sets of the form A x B9 so we have shown that SSl x Stx c
^2 and thus J^ x ^ = ^2.
We now show that Mi x Mi = ju2. Let
KN = {(*,)OeR2;-Ar<x,);<N}
It is enough to show that, for all N, Mi x Mi = M2 on Borel subsets of KN.
(Why?) To establish this fact, let ^N be the Borel subsets B of KN for which
Mi * Vi(B) = n2(B)
c^v is a A-system:
XI. MiXMi(^) = 4N2 = M2(^iv).
XI. Let Bt c B2 cz • • • be an increasing sequence with each Bne£fN. Let B =
U»=1Bn;then
(Hi x Mi)(5) = I™ (Aii x A*i)(Bn) = lim n2{Btt) = ^2(5)
n-*ao n-»oo
X3. Let £ and F be elements of £fN with F c F. Then F = E u (F - £), so
Mi x Mi(^ - E) = Mi x Mi(^) - Mi x Mi(£)
= M2W - M2(£) = M2(^ " E)
Next, let ja/jy = {I x J; I, J subintervals of [—N,N]}. «s/N is a 7c-
system because
(It x Jx) n (72 x J2) = (/x n J2) x (Jx n J2)
Furthermore, jrfN <= ^ because
M2(/ x J) = (length/) (lengthJ)
whereas (mi x Mi)(J x J) = MiCO'MiO^)
= (length/) (lengthJ)
From the rc-A theorem we conclude that SfN contains the smallest a-field
containing stN. Hence M2 = Mi x Mi on all Borel subsets of KN. □
Exercise. Write R3 = R2 x R1 and show that ^3 = @2 x ^1 andM3 = M2 x
Mi-
We return now to the general situation: (X9 Jty \x) and (Y, J^*, v) are <7-finite

98
Chapter 2 Integration
measure spaces and (X x Y9 M x jV, fx x v) is the measure space constructed
above.
Theorem 14. (Fubini, version 2) Let/:X x 7-► R be a nonnegative
measurable function. Then
1. a. For each xeX9 f(x9y) is a measurable function of y.
b. For each yeY9 f(x9 y) is a measurable function of x.
2. a. jV /(x, j;) rfv is a measurable function of x.
k I* /(x> JO <fy* *s a measurable function of y.
3. f T f /(*, y) dvl d/i = f [ f /(*, y) dfA dv = f /(x, y) d(/, * v)
JxLJY J JrLJ* J J*xy
Proof. Part 1 was proved at the beginning of the section.
To prove parts 2 and 3, first note that they are true for f(x9y) = 1e(x,j;) if
EeJt x Jf. Indeed
I,
l£(x,y)dv = v(Ex) = (f>E(x)
Y
which we have shown to be measurable, and similarly
lE(x,y)dfi = i*(Ey) = \I/E(y)
I
)x
is also measurable. The fact that
m\E(x9y)dv \dfx= lE(x,y)dii \dv = \ lE(x9y)d(fi x v)
Y J JyLJX J JXxY
is version 1 of the Fubini theorem.
Now, because parts 2 and 3 are true for characteristic functions, they
must be true for simple functions by linearity. To prove the theorem in
general, let f{x,y) be a general nonnegative measurable function and choose
an increasing sequence of nonnegative simple functions
0 < sx < s2 < • - •
with sn->f pointwise. Then, for x e X fixed,
sn(x9y)dv^> f(x9y)dv
by the monotone convergence theorem. Thus \yf(x9y)dv is a measurable
function of x, because we know that Jy s„(x, y) dv is measurable. Similarly,
\xf(x>y)dn is a measurable function of y.

§2.5 Fubini Theorem
99
Furthermore, part 3 holds for all sn; that is,
msn(x,y)dv \dfi= sn(x9y)dn\dv= sn(x,y)d{n x v)
r J JyLjx J Jxxy
Applying the monotone convergence theorem to each of these terms
separately yields
m/(x,j,)dv~L = [ [ f f(x,y)dfi~\dv = [ f(x9y)d(n xv) D
Theorem 15. (Fubini, version 3) Let/be integrable onXx Y. Then
1. a. For almost all x, /(x, y) is integrable as a function of y.
b. For almost all y, f(x9 y) is integrable as a function of x.
2. a. j"y /(x, j;) dv is equal a.e. to an integrable function of x.
b. ]x /(x, y) d\i is equal a.e. to an integrable function of y.
3. f [ [ f(x9y)dv~\dfz = f I" [ f(x,y)d»\dv = [ /(x, jOdfo x v).
/V00/. Write / = /+—/_ where /+ and /_ are nonnegative. Because / is
integrable with respect to /* x v, we know that jxxy/+(x,j;)d(ju x v) and
ixxYf-(x>y)d(fi> x v) are finite. Version 2 of Fubini then gives
f Mx,y)d(n x v) = [ [" f /+(x,^v~L
JXxY Ja LJ< _J
< 00
Thus jV/+(x,y)dv is finite a.e., and similarly $xf+(x9y)d/j. is finite a.e. This
gives part 1. Parts 2 and 3 follow by applying version 2 of Fubini to/+ and/_
separately and then adding. □
A Final Remark. Instead of considering only products of two measure
spaces, we could have considered products of three or more measure spaces.
For instance, let (XhJKh jx^ i = 1,2,3, be measure spaces. One can define a
product <7-field
Jtx x M2 x Jtz
by defining it as either
{Jtx x Jli) x M-$ or M^ x (Jt2 x Jt3)
or as the smallest <j-field containing the product sets
iT.j X iT_2 X iT,g

100
Chapter 2 Integration
with AteJii9 i = 1,2,3. It turns out that these three definitions give the same
(7-field. (See exercise 3.)
Moreover, we can define on Jtx x Jt2 x Jtz a product measure \xx x
1*2 x /*3 w^h ^e property that, on product sets,
(10) (Mi x \i2 x fi3){A1 x A2 x A3) = MiO^M^W^a)
That is, we can define this measure as
Mi x (M2 x Ms) or (Mi x M2) x Ma
You will be asked in exercise 3 to show that these definitions are the same.
By simply using the Fubini theorem twice, we get analogous statements
for this triple product. In particular, iff is integrable on A\ x X2 x X3 (with
respect to/i! x ix2x ju3), then the various partial integrals make sense a.e. and
(11) fd(ixx x fx2 x /x3)= ( /<*Mi W2 Ui3
JXlxX2xX3 J*3LJ*2\J*i / J
What we have said about products of three measure spaces applies equally
well to products of any finite number of measure spaces. (See exercise 4.) Here
the Fubini theorem looks like
(12) f fd(fx1x-"Xfzn)=! -\[ fdixX-d^
JXlX'"XXn JXn |_J*l J
Exercises for §2.5
1. Let R be the region
* = {(*, j>);-1<x<1,-1<j><1}
in the plane. Compute the Lebesgue integrals
xy2dfx2 (x2 + y2)dfj,2 and ye*
JR JR JR
d\i2
2. Let (X, M, fj) and (Y, Jf9 v) be a-finite measure spaces. Call a product set
A x B finite if fi(A) < oo and v(B) < oo. Show that the product measure
fx x v is the only measure satisfying
(fi x v)(A x B) = fi(A)v(B)
for all finite product sets A x B. (Hint: Use the n-X theorem.)
3. a. Let (Xi9 Mi9 fit), i= 1, 2, 3, be a-finite measure spaces. Show that
(J(1 x M2) x Jt3 = Jt^ x (M2 x M3)

§2.5 Fubini Theorem
101
and that these equal the smallest a-field containing the product sets
Ax x A2 x A39 where A^Jt^ i = 1, 2, 3.
b. Show that (ix1 x fx2) x fx3 = fj,1 x (/z2 x fi3).
c. Call a product set Ax x A2x A3 finite if ^(^f) < oo,i = 1,2,3. Show
that the measure in part b is the only measure satisfying
(j"i x /i2 x n3)(A1 x A2 x A3) = jM^JiM^W^a)
for all finite product sets Ax x A2 x A3.
4. Generalize exercise 3 to n-fold products.
5. In part a of exercise 3, let Xx= X2 = X3 = R, JK1 = Ji2 = M3 = the
Borel sets of R, and ixx = fx2 = fx3 = Lebesgue measure. Show that Jtx x
Jt2 x J(3 is the Borel sets of R3 and ixx x \i2 x \i3 is Lebesgue measure.
Can you prove an equivalent statement for R4? for Rw?
6. Let (X,Jt,fj) and (Y, Jf9v) be <7-finite measure spaces, and let E be in
Jl x Jf. Show that E is of measure zero if and only if Ex is of measure
zero for almost all xeX.
7. a. Let (X, Jl, \x) and (Y, Jf, v) be a-finite measure spaces, and let /: X -*
R and g: Y -► R be measurable functions. Let h(x9 y) = f(x)g(y). Show
that h is a measurable function on X x Y.
b. If/ and g are integrable, show that h is integrable and that its integral
is
(L-XJ
gdv
8. (The integral as "area under the curve.") Let {X9Jt9\x) be a a-finite
measure space, and let /: X -* R be a nonnegative measurable function.
Let
Af = {{x,t)eX xR;0<t< f(x)}
Show that Af is a measurable subset ofXxR (that is, belongs to M x
(%x) and that the measure of Af with respect to the product measure
A* x jULebisequaltoJx/d/i.
9. Show that property XI of a A-system can be replaced by the property
12'. If Al9 A2,... are disjoint subsets of Sf9 then (J^ i4£ is in y.
10. Let Ibea set and ^ a a-field of subsets of X. Let ixx and /z2 be finite
measures on & with the property that \i\[X) = ^PO- Show that the
collection of sets ' , A ^ , ^ , ,,,
{Ae&;iix(A) = »2(A)}
is a A-system.
11. Show that Lebesgue measure is the only measure on the Borel sets of the
interval [0,1] with the property that, for all subintervals J, fi(J) = length
of J. (Hint: Use exercise 10 and the n-X theorem.)

102
Chapter 2 Integration
12. a. Show that for the function
xy
f(*,y) =
(x2 + y2)2
the iterated integrals
r [ P /<* y) dAdx and j \ [ fl f(*> y) dx~\ dy
exist and are equal.
b. Show that / is not integrable over the square — 1 < x < 1 and — 1 <
13. Let X = Y = R and M = Jf = Borel sets. Let \i be Lebesgue measure,
and let v be counting measure; that is,
v(B) = number of elements in B
Let EeM x Jf be the set
E = {(x,y)eXx Y;x = y}
Recall that </>E(x) = v(Ex) and \jfE{y) = /i(£y). Show that <f>E and ^£ are
measurable but that
j ^(x)rf/i^j
<f>E(x)dfj.^ il/E(y)dv
§2.6 Random Variables, Expectation Values, and
Independence
In the next two sections we will discuss some probabilistic
applications of the material in §2.5. Let X be a set, 2F & a-field of subsets of X9
and [i a probability measure on X. A random variable f is, by definition, a
measurable function f:X-*Rv{±oo}. For instance, let X = J = @ and let
(1) /
-K"+i;*)
Interpreted probabilistically, / is the number of times H comes up in the first
n stages of a Bernoulli sequence. It is a "random quantity" or "random
variable" that can be measured each time we perform a sequence of Bernoulli
trials.
The expectation value of a random variable is its integral
E(f) = [fdv

§2.6 Random Variables, Expectation Values, and Independence
103
Of course, the expectation value need not always be defined; that is, / need
not always be integrable. However, all the random variables we will consider
do have well-defined expectation values. For instance, if we integrate equation
1 over the unit interval we get
J>-«/>-!
representing the fact that n/2 is the number of heads "most likely" to turn up
in a sequence of n Bernoulli trials. (This meaning of expectation value will
become clearer in §2.7.)
Given a random variable /,: X -► R and a Borel subset A ^ R let
(2) f*f(A) = tilf-HAn
The right-hand quantity is well defined because f~1(A)e^. (See §2.1.) We
leave it for the reader to check that equation 2 defines a measure on the Borel
subsets of R. (See §2.3, exercise 15.) We will call this measure the probability
distribution associated with the random variable /. If, for two random
variables / and g, [if = [ig9 we will say that / and g are identically distributed. The
essential property for us of the measure fxf is the following.
Theorem 1. Let ^ be a nonnegative Borel-measurable function on R. Then
(3) f *(/)**- f ***/
Jx Jr
Proof. First, suppose that ^ is the characteristic function of a Borel subset
XgR. Then ^(/) is the characteristic function of f~1(A), so the left-hand
side of equation 3 is ix\_f~l(A)~\ and the right-hand side is fif(A). By equation
2 these quantities are equal. Next observe that equation 3 holds for finite linear
combinations of characteristic functions of sets—that is, for simple functions.
Finally, by theorem 6 of §2.2 there exists an increasing sequence sn of non-
negative simple functions with sn -► <j>. Then sn(f) -► </>(f); so equation 3
follows from the monotone convergence theorem. □
Corollary 2. Let ^ be a Borel-measurable function on R. Then <j> is integrable
with respect to the measure \is if and only if <j>(f) is integrable with respect to
\i. When such is the case, equation 3 holds.
Proof. Let <j> = ^+ — ^_ and apply theorem 1 to ^+ and ^_ separately. □
Notice that if <j>(x) = x, then equation 3 becomes
[ xdfif= \ fd/i = E(f)
Jr J*

104
Chapter 2 Integration
This shows that, if random variables f and g are identically distributed, they
have the same expectation value. More generally, if / and g are identically
distributed, then, for every Borel-measurable function <j> on R,
<j>diif= L
</>dtig
so by equation 3
f Kf)dii- (
Jx Jx
(4) | *(/)**- «*)«*
For instance, taking ^(x) = x2 we get
g2d\i
//>*.-/.
Given several random variables /i,...,/„, let F:X->Rn be the map
F(x) = (/i (x),... ,/„(x)). If A is a Borel subset of Rn, set
(5) nfl fn{A) = nlF-i(A)-]
This formula defines a probability measure on the Borel subsets of Rw (check
this!) called the joint probability distribution associated with /i,...,/„. The
analogue of equation 3,
(6)
<f>(fi,--->fn)dn= I ^dfifi fn
Jx Jr"
holds for any nonnegative Borel-measurable function (j> on Rw and is proved
in exactly the same way.
A set of random variables fi9...9fn is said to be independent if, for any
sequence of Borel subsets Ax, A2,..., An of R, the sets
are independent as subsets of X. An infinite sequence of random variables fl9
f2,...is said to be independent if every finite subsequence is independent. A
very simple criterion for independence in terms of the joint probability
distributions of the/J's is the following.
Theorem 3. The random variables fl9...,/, are independent if and only if
the probability measure At/lt...f/n is equal to the product measure \iSx x fxf2 x
•"X/i/n.
Remark. The product is, of course, defined as in §2.5.

§2.6 Random Variables, Expectation Values, and Independence
105
Proof. To check that the measures agree, it is enough, by the n-X theorem,
to check that they agree on sets of the form At x • • • x An. By definition
and by independence the right-hand expression is equal to /*[/f H^i)] x
• • • x li[fnl(An)] which equals fifl(Ai) x •. - x (An) which equals(/Aflx • • • x
/X/b)(Ai X • • • X An). □
One consequence of theorem 3 is the identity
(7) J/iX-x/.^ = £(/l)x-x£(/B)
Indeed, if we apply equation 6 to the function faxl9..., x„) = xt • • • xn we get
A x--x/.dfi= xix2--xndiifx /n
Jx Jr»
x1x2'-xndfxfl x ••- x^
Jr»
=(Lxi^-)(1X2^)"(Lx"^)
by Fubini's theorem. However, the ith term on the right is just E(fy.
A remark about independence that will be useful below is the following.
Let /i,...,/, be independent, and let fa,..., <j>n be Borel-measurable functions
on R. Then
(8) Mfl)>~.,<f>n(fn)
are independent. In fact, let Al9...,AH be Borel subsets of R, and let A't =
^f1 (At). Then the A"s are also Borel subsets, and
M(/i)rl(>4i) = /r1w) *-i,...,n
By assumption, the sets on the right are independent; hence, so are the sets on
the left.
Example 4. If ft and f2 are independent, then /i3 and |/2| are independent.
The notion of independence plays a central role in measure-theoretic
models of probabilistic processes. For instance let's go back to the gambling
process described at the end of §1.2. Recall that this process involves a cage
filled with colored marbles. There are assumed to be k different colors, with
N( marbles of each color i and JV = J]f=1 Nt marbles in all. The process consists
of mixing the marbles, then drawing a marble out of the cage. If the color of
the marble is i, the player receives a reward (or penalty) of rt dollars. The

106
Chapter 2 Integration
marble is then replaced, the marbles are again thoroughly mixed, another
marble is drawn, and the game continues. \ifn is the amount of the reward or
penalty at the nth draw, then fn takes on the values of r!,...,rk with the
probabilities of pl9...9pk, where pi = Ni/N. What is an adequate measure-
theoretic description of this situation? We claim that the data needed to
"model" this process are: (a) a set X9 a c-field 3F of subsets of X, and a
probability measure /x on SF\ and (b) an infinite sequence of random variables
/i>fiy - • • with the following properties:
(9) The y|'s are independent
and
(10) /^)=Ep;
rjeA
for every Borel subset A of R. Indeed, the yj's give an identification of points
xeX with infinite sequences of draws from the cage, Le.,fk(x) describes what
happens in the sequence corresponding to x at the fcth draw. Property 9 just
says that what happens at the nth draw is independent of what happens at any
of the other draws. This is justified by the fact that the marbles are thoroughly
mixed after each draw. If one sets A = {rm}, then, by equation 10, the
probability that at stage n the reward or penalty incurred will be rm is just
which is what we expect because of the number of marbles of color m in the
cage. Notice that equation 10 implies that the //s are identically distributed.
We will now show that a probabilistic model with all the above features
does exist. In fact, we will show that we can even take for X the unit interval
J, for & the Borel subsets of/, and for the probability measure on 3F ordinary
Lebesgue measure.
Theorem5. There exist bounded measurable functions fuf29... on I such
that property 9 and equation 10 hold with fx = fxL = Lebesgue measure.
Proof. Decompose the unit interval into k disjoint subintervals Ix,..., Ik such
that Jj is of length ph and define /i by setting
ft=rt on/, l = lf...f* .
Next decompose each of the intervals It into k disjoint intervals Ilt w, m = 1,...,
k, such that IUm is of length pfpw. (Because It is of length pt and J]pm = 1, such
a choice of 7/tW's is clearly possible.) Define f2 by setting
fi = rm on Ilttn

§2.6 Random Variables, Expectation Values, and Independence
107
Notice that, if At = {r,} and Am = {rm},
fi-i(Al)nf2-HAJ = Iltm
so ^fi'i(Al)nf2-HAM=PlPm
On the other hand,
k
fi'HAi) = // and f2x{AJ = (J IUm (disjoint union)
k
so M[/i_1Mi)] = Pi and m[/2_1WJ] = Z P/Pm = Pm
From these computations we conclude that
or, in other words, fC1(Al) and f2~1(AJ are independent. Now, suppose that
Bt and 2*2 are arbitrary Borel subsets of R. Then
B1=B'1v(\Ja) and B2 = B'2J (J Xm)
with B[ and £2 containing none of the r/s. Then /J"1 (2*1) and f2~1(B2) are
empty; so
/i"1(51)= U /i_1(^) (disjoint union)
r,eB,
and
/2"1(^2)= U /2_1(^m) (disjoint union)
are independent. (Why?) Because Bx and 2?2 are arbitrary, ft and /2 are
independent. We let the reader check that ft and f2 satisfy equation 10 and
move on to the construction of/3. Decompose Ilm into k disjoint subintervals
Ii,m.n>*=l>...9 K of length ViVmVn and define f3 by setting
f3 = rn on Ilttnt„
One checks that fl9 f2, and f3 are independent in exactly the same way as
above. It is also clear by now how to construct /4, fs, and so on. We leave
details to the reader. □
Remark. Let fe = 2, rx = 1, r2 = — 1, px = i, and p2 = h Then the functions
constructed above are exactly the Rademacher functions!

108
Chapter 2 Integration
Stage 1 Stage 2
1
1
1
This figure indicates the first two stages in the construction of the/'s with the data k = 3,rl = \,
r2 = 0, r3 = -1, and/>! = p2 =p3 =* £.
Exercises for §2.6
1. Let v be a probability measure on the Borel sets of the real line. Then v
is said to be supported on the Borel set A if v(Ac) = 0. Using theorem 1,
show that, if A is a Borel set containing the image of/, then nf is supported
on A. In particular, if / is a simple function taking on the values rl9...9rh,
then fif is supported on {rt,..., rfc}.
2. Let X = / be the unit interval, and let \i be Lebesgue measure. Describe
the measure \is for the function f(x) = x2.
3. If, for i = 1,2,/ = Ri is the i th Rademacher function, what is the measure
fxfif Verify directly that ju/l>/2 = fxfl x ju/2.
4. (The unfair coin.) Using theorem 5, let fe = 2, rt = 1, r2 = —hPi = P> and
p2 = 1 — p. Describe the first three of the functions /x, /2, /3,....
5. In exercise 4, let Sn = ft H + /,. Compute the expectation value E of
Sn and the variance
V(Sn)=f(Sn-E)2dfiL
6. (The random walk with pauses.) Using theorem 5, let k = 3, rx = 1, r2 = 0,
r3 = — 1, and px = p2 = p3 = ^. We have already drawn the graphs of/!
and /2. Draw the graph of/3. Can you discern a pattern?
7. a. Let Rt be the ith Rademacher function, and let
/*(*>= zQf)w
Compute
I etf*ix)dx
(Hint: Use independence.)

§2.6 Random Variables, Expectation Values, and Independence
109
b. Using the formula
2x - 1 = lim fk(x)
(proved in exercise 7 of §1.1) deduce Vieta's formula
8. a. Let ft, f2,... be as in theorem 5, and let Sn — f^ + • • ■ + /„. Prove that
ets»Wdx = YJProb(Stt = r)etr
b. Conclude from part a that
(*) £P">MS„ = r)e» = (£ Pie»>Y
(Hint: Write the integral on the left in part a as
e*fie*f2 x ... x etfndx
1
I
and use independence.)
9. Let /i, f2,/3,... be independent, identically distributed random variables
taking on the value of 1 with probability p and the value of 0 with
probability 1 — p, where 0 < p < 1. (That is, using theorem 5, take k = 2,
ri = 1, r2 = 0, Pi = p, and p2 = 1 - p.) Let Sn = /x + • • • + /„. Show that
(**) Prob(Sn = r) = (")p'(l-p)n-'
if 0 < r < n and is zero otherwise. (Hint: Use exercise 8.)
10. Let {rx, r2,. •.} be a countable subset of R, and let pj, p2,... be a countable
sequence of nonnegative numbers with
Zp«-i
For every subset 4 of R, let
(t) v(A)=£p*
i"| e A
Show that there exists a sequence /i,/2,... of independent, identically
distributed random variables on the unit interval such that fifl = fif2 =
... = v.
11. For i = 1,..., n let {X,-, ^£, /jj be a probability space, let M± x-xMn
be the product of the Jtl^, and let \ix x • • • x \in be the product measure

110
Chapter 2 Integration
on Jt± x ••• x Mn. For each i let ft be a measurable function on Xt.
Consider /, as a function on Xl x • • x Xn by setting
y;.(x1,...,xn) = /i(xl)
Show that the /£'s, regarded as functions on Xx x ••• x Xn, are
independent.
§2.7 The Law of Large Numbers
Let's now return to the question posed at the beginning of §2.6.
What does the expectation value of a random variable really represent?
Consider a simple probabilistic process (such as the toss of a coin) and a
numerical quantity Q associated with the process. (For instance, for the toss
of a coin let Q = 1 if an H occurs and Q = — 1 if a T occurs.) Now repeat
the process and again measure the quantity Q; repeat it a third time and
again measure Q, and so on ad infinitum. Let AVn(Q) be the average value of
Q, averaged over the first n stages of this infinite sequence of experiments.
Does AVn(Q) tend to a limit as n tends to infinity? The answer is yes, provided
that, each time the experiment is repeated, the conditions under which it is
performed are not biased by the results of the preceding trials. (For instance,
if the experiment consists of drawing a marble from a cage, recording its color,
and then replacing, it, the marbles must be thoroughly mixed each time.) We
will show that, if these experimental requirements are met, then, not only does
AVn(Q) tend to a limit as n -► oo, but in fact
(1) lim AVn(Q) = E(Q)
n-*oo
is the expectation value of Q. (We mean, of course, that equation (1) holds
with probability one.) To see this, let's first describe the experimental
setup above in somewhat more precise terms. Let /„ be the measured value of
the quantity Q at the nth stage of the sequence of experiments. Then, under
the hypotheses above, the ^'s are independent, identically distributed random
variables. The underlying space X on which they are defined is, technically
speaking, the totality of "all infinite sequences of repetitions of the experiment."
For instance, if the experiment consists of the toss of a coin, X is the set
0$ of all Bernoulli sequences as in §1.1. Actually it isn't terribly important
to describe X this explicitly. What is important are the i.i.d. (independent,
identically distributed) random variables fl9f2,... and their common
probability distribution fxfl = fifl = • • •. For instance, for the experiment described
in §2.6 (a colored marble drawn from a cage) we showed that X could be taken
to be a very simple set: the unit interval. The important point was that on the

§2.7 The Law of Large Numbers
111
unit interval we could produce a sequence of independent random variables
fiyfi*- • aH with the probability distribution of equation 10 in §2.6.
The following result is what is traditionally called the law of large numbers.
Fix a set X, a <7-field & of subsets of X, and a probability measure \i on «^\
Theorem 1. Let fx ,/2,... be a sequence of bounded random variables on X
that are independent and identically distributed. Let E = E(f^) = E(f2) = • • •
be the common expectation value of the /.'s. Let X0 be the set of points xeX
for which
(2) /t(*)+ -+/■(*) _,g
ft
as n -► oo. Then i*(X0) = 1.
Remark. The assumption that the /£'s are bounded is not essential, but it
simplifies some details of the proof.
Proof. Replacing ft by f{ — E9 we can, without loss of generality, assume that
E = 0. Let
V = (\f*d»)2 and w = \f?
dfi
Because the /f's are identically distributed, these quantities are the same for
all f s. The first step in the proof will be to establish the following inequality
for all s > 0:
/f y. |/i(x)+ "-+/.(*)! ^ /1\ ^ 3n(n-l)K + K^
^reX'|—s—|>8jJ-—s?—
(3)
The left-hand side of inequality 3 is equal to
//({xeX;(A + ---+/J4 >nV})
and, by Chebyshev's inequality, this is less than
(^)!(/i
I'8"/J
so inequality 3 reduces to
(4) [(/i + -"+ fnfdfi < 3n(n - \)V + nW
If we multiply out the expression on the left, we get five sorts of terms—
namely,

112
Chapter 2 Integration
j flfjdn a^p
J flMidix a^P^l
J faffifyfsdn a^p^j^S
J fiffidp
The first integral is equal to W, and the second integral is equal to
(Jj*d^(J$d^=V
by equation 7 of §2.6. Similarly, the third integral is equal to
(//2<fc)(//***)(/A**)
and the fourth and fifth integrals are equal to
(J fa d^j (j ffi dp\ (J /7dji) (J hdl^j and J fid/* J ffid/jL
Because the expectation values are zero, these three terms are zero.
Because there are exactly n integrals of the first type and 3n(n — 1) integrals
of the second type (see §1.1), the sum of all these integrals is the right-hand
side of inequality 4.
Now choose a sequence of numbers st, e2» £3* • • • such that sn -> 0 and
» 3n(n- l)V + nW
L, 4-4 < °°
(See lemma 6 in §1.1.) Let
A,
n = IxeX;
/iM+ - + «*)
>£n[
Then, by inequality 3, Yj?=i l*{AH) < 00. So, by the first Borel-Cantelli lemma,
fi{An; i.o.) = 0. This result means that, if we exclude a set of measure zero from
X9 then for x in the complement
/i(*) + -+/.(x)
<£„

§2.7 The Law of Large Numbers
113
for all but finitely many values of n. This clearly implies that
/i(x)+ -+/.(*)
-> 0 as n
oo
□
Exercises for §2.7
1. Show that, if the /„'s in theorem 1 are the Rademacher functions, then
equation 2 is the strong law of large numbers formulated in §1.1.
2. Let (AT, «^, fi) be a probability space. Recall (§2.3, exercise 8) that a sequence
of measurable functions fn9n= 1,2,... converges to zero in measure if for
all e > 0
ti({xeX;\fn(x)\ >e})->0 as n-+co
Show that, if {fn} converges pointwise to zero almost everywhere, then /„
converges to zero in measure. (Hint: Let An = {xeX; \fk(x)\ < eforfc > n}.
Show that Al ^ A2 ^ A3 ^ • • • and that X — (J"=1 An is of measure zero.)
3. Deduce from exercise 2 and theorem 1 the weak law of large numbers. Show
that, if/*i,/2>--- are a sequence of bounded independent, identically
distributed random variables and if £ is their common expectation value, then
for all e > 0
Prob
/i + -+/.
-£
>e)->0
as n-
oo
Show that, if/i,/2>... are as in exercise 3, then
I/1 + -+/.
Prob
-E
*-)'&>
where Kis the common variance of the//s, i.e., V = ^(f — E)2 d\i. Use this
to give another proof of the weak law of large numbers.
Let /i,/2,.. be bounded, independent, identically distributed random
variables with E = E(fx) = E(f2) = • • = 0. Let Sn = fx + • • • + /„. Show
that if a > 0 then
Sn(x)/na/2)+a -► 0 a.e. as n -> oo
(Compare with exercise 17 in §1.1.) (Hint: Show that there exists a constant
Ck such that
\
S?dn<Cknk
for every integer k > 0.)

114
Chapter 2 Integration
6. Let /i,/2» • • be independent, identically distributed random variables on
the unit interval. Suppose that the common probability distribution of
the fiS is given by equation 9 of §2.6, with k = 2, rx = 1, r2 = 0, px = p,
and p2 = 1 — p (p being any number between zero and one). Let Sn =
f\ + " • • + fn- Show that, if <j> is any bounded measurable function on the
interval [0,1],
<•> «Kr)]-MD*-K)(i)^-^
(Hint: See section 2.6, exercise 9.)
Remark. We will denote the right-hand side of equation (*) by Bn{<f>9p).
Notice that it is a polynomial of degree n in p. We will call it the nth
Bernstein polynomial associated with </>.
7. Show that for n very large Sn(x)/n is very close to p for most values of x.
Explicitly show that for all 5 > 0
(**)
<{■
jiNxeflU];
S„(x)
-p
>*})*(£)'<,-'>
(Hint; See exercise 4.)
8. Let <j> be a continuous function on the interval [0,1]. Show that
(t)
■['©]■
► ^(p) as n -► oo
(Hint: Given 8 > 0 choose 5 so that |^(s) — <l>(t)\ < e when 0 < s, t < 1 and
|s - t| < 5. Let /j be the subset of 0 < x < 1 on which |(Sn(x)/n) - p\ < 5,
and let I2 be the complementary set. Show that
im-
m
dx < e
and estimate
im-
m
dx
using inequality (**) and the fact that <j> is bounded.)
9. Show that the convergence in display (t) is uniform in p. By equation (*)
conclude that, as functions of p, the Bernstein polynomials Bn(<l>) converge
uniformly to <f> on the interval [0,1]. (The result we have asked you to prove
is a constructive form of the Weierstrass approximation theorem: Given a
continuous function ^ on the interval [0,1], there exists a sequence of
polynomials Bn converging uniformly to ^ as n -► oo.)

§2.8 The Discrete Dirichlet Problem
115
§2.8 The Discrete Dirichlet Problem
Let 6 be an open set in R2. A twice-differentiable function /: 0 -*
R is called harmonic on 0 if
d2f d2f „
Now let Q be a compact subset of R2 with a continuous boundary dCl. Suppose
that g : dCl -* R is continuous. The classical Dirichlet problem asks one to find
/: Q -* R such that / is harmonic on Int Q and / = g on 3Q.
Many solutions to this problem have been discovered, some of which are
quite ingenious. In particular, in Two dimensional Brownian motion and
harmonic functions (Tokyo: Proc. Imp. Acad., 20, 706-714 [1944]), S. Kakutani
showed how to construct / using probabilistic methods. He used a kind
of limit of the random walk in R2 called the Wiener process or Brownian
motion. Although the theory of the Wiener process is beyond the scope of this
book, we can understand the ideas behind Kakutani's construction by looking
at a discrete version of the Dirichlet problem due to Courant (Courant, R.,
Friedrichs, K. O., and Lewy, H. Ueber die partiellen Differenzengleichungen
der mathematischen Physik. Math. Ann. Vol. 100. pp. 32-74 [1928]). (In fact,
Courant showed that the solution to the classical problem can be obtained
as a limiting case of the solution of the discrete problem described below!)
Before we describe this discrete version of the Dirichlet problem, we need
to translate the definition of harmonic functions into a form that is easily dealt
with measure theoretically.
Theorem. (Mean value property) Let 0 c R2 be open and let /: (9 -> R be
harmonic. Let x0 e 0 and assume that the circle of radius a around x0 lies
entirely in 0. Then
(1) /(*o) = (^j £ 7(*o + «") d9
Conversely, if /: 0 -* R is continuous and equation 1 holds for all x0 and a
such that the circle of radius a around x0 lies entirely in 0, then / is twice-
differentiable and harmonic in (9.
For a proof of this theorem see, for example, L. Ahlfors, Complex Analysis
(New York: McGraw-Hill [1953]).
Using this characterization of harmonic functions, we can formulate a
plausible discrete analogue of the Dirichlet problem. The space R2 is replaced
by the integer lattice
Z2 = {(m, n); m, n are integers}
and the compact region O becomes a finite subset of Z2.

116
Chapter 2 Integration
For x e Z2, there are four nearest neighbors, xN, xs, x£, and xw, as pictured
below.
xN
xwm mx mxE
•
If x e Q we say x e Int CI if xN, xs, x£, and x^ are all in CI as well. We then define
dCl = CI - IntCl.
To define harmonic functions on Intfi, the integral in equation 1 is
translated to be the average over the nearest neighbors. Namely, if /: CI -► R
we say / is harmonic on Int CI if
fix) = \U(XH) + /(*s) + f(xE) + /W)]
for all xg IntQ.
Now let's consider the following problem.
Discrete Dirichlet Problem
Given g: dCl -► R find /: fi -> R such that / is harmonic on Int fi and f = g
on dCl.
We ask you to solve this problem by yourself. The following three exercises
should be of some help.
1. Let 0tXQ denote the set of all random walks on Z2 with x0 as the starting
point. This set can be identified with the set of all sequences of APs, E%
S's, and W's (for example, N W W E S N...). Assign to N, E, 5, and W the
numerical values 0,1,2, and 3. Let J = (0,1] = the half-closed unit interval.
If coel, the quaternary expansion of co gives rise to a sequence of 0's, l's,
2's and 3's and hence to a sequence such as that above. Therefore, we can
identify J with 0tXo. (For the details of this identification, see §1.2.) Now
suppose x0 g CI, Consider the random walk rw e 0tXQ indexed by co el. Two
possibilities exist: Either rw stays inside Intfi forever, or it eventually gets
to a boundary point xb(co). (For instance, if x0edCl, then xb(co) = x0.)
a. Show that the first of these two possibilities occurs with probability
zero. (See §1.4, exercise 17.)
b. Let fXo(co) = g [x6(co)]. Show that fXo is a measurable function of co e I.

§2.8 The Discrete Dirichlet Problem
117
2. Let 0t^.Q be the set of all random walks starting at x0 that move directly
to xN on the first step. Define ^f0,^0, and #J£ similarly.
a. Show that 0tXQ = «Jo u ^o u ^o u ^ (disjoint union) and show
that, under the correspondence MXQ ~ J, 0$Xo corresponds to the
interval (0,£], fflXo10 the interval (i,£], and so on.
b. There is an obvious bijective map p: fflXo -> fflXfl. Namely, take the
random walk whose first position after x0 is xN and think of it as a
random walk starting at xN. Show that, if we identify 0tXft with (0,1]
as in exercise 1 and identify M*o with (0,£| as in part a above, the
mapping p becomes the mapping a> -> 4co.
c. Show that, with the identifications in parts a and b,
w /*»=/*0(f)
Obtain comparable identities for fXE,fXs, and/X|K.
3. Define /: Q -> R by setting
/(*o)= fx0(o>)dfi
for all x0eQ, with fi being Lebesgue measure. Prove that / is harmonic
and equal to g on 3Q.

Chapter 3
Fourier
Analysis
§3.1 if ^Theory
Let (X, #", fi) be a measure space and let /: X ->C be a complex-
valued function. We can write f(x) = u(x) + it;(x), where u and t? are real-
valued functions on X and i = */— 1.
Definition 1. f = u + iv is measurable if m and i; are both measurable.
Note. tff = u + iv is measurable, then |/| = y/u2 + u2 is measurable by
Theorem 14 in §2.1.
Proposition 2. Let / = u + iv be measurable. The following two statements
are equivalent.
1
\u\dfi < oo and |i;|
Jx Jx
1. |/|<fo<co
Proo/. Notice that |u| + \v\ > (u2 + t>2)1/2 £ |u| (or |p|). But |/| = (m2 + v2)1*2,
so integration yields
[ \u\dfi+ [ \v\dfi> f l/lrf/x
118

§3.1 &l-Theory 119
and
\x[ndli~\xMdlX (0rLW*1)
The first of these inequalities shows that statement 2 implies statement 1; the
second inequality shows that statement 1 implies statement 2. □
Definition 3. If / = u + iv is a complex-valued measurable function on X,
we say / is integrable if
I
\f\dp< co
X
In this case we define the integral of / to be the complex number
(1) fd\x — udfi + i\ vdpt
JX JX JX
We denote by S£ 1{X,fj) the set of all such functions:
(2) ^{X.ii) = j/:X->C,measurable; | \f\dpt < ool
Proposition 4. Let f,ge££?x(X,pi), ceC, then
1. f + ge<el{X,p!) and | (f + g)dfi = I fd/i + | ^d/i
2. cfe&\X,ix) and | (cf)dpi = c \ fdpL
3. | /£fo = /<fo
Jx Jx
If fdn\< [
\Jx I Jx
4. /d/i < \f\dfi
Proof. We leave proofs of 1,2, and 3 as exercises. To prove 4 let a = $xfdl*.
Then (5/|a|)\xfd\i = \a\ is a positive real number. Let # = (fl/|fl|)/ and write
g = u + iv, where u and v are real valued. Then
fdfi \ = \a\ =— /d/* = #4u
IJx I Mjx Jx
= u dfx + i i; d\x
Jx Jx

120
Chapter 3 Fourier Analysis
Thus $x v d\i = 0 because the left-hand side is real. Therefore
If fdfi\= [ udix<[ \u\dpi< \ \g\dti = \ \f\dii
\jx I Jx Jx Jx Jx
since \f\ = \g\. Q
Parts 1 and 2 of proposition 4 are a proof that the space S£ 1{X,ii) is a
vector space over the complex numbers. In fact, S£x{X,\i) is a normed vector
space.
Definition 5. Let V be a vector space over C. A norm on V is a function
|| • ||: V -> R with the following properties:
a. \\v\\>0,veV
b. \\v\\ =0ot; = 0
c. \\cv\\ = \c\\\vlceC,veV
d. ||t; + w||< ||t;|| + ||w||
Given a norm || • || on a vector space K, we can define a metric d(-9 •):
K x F ^ R on F by d(u, w) = || i; — w ||. It is easy to check that d satisfies the
properties of a metric; that is,
1. d(v9 w) = d(w9 v\ v9weV
2. d(v, w) -+- d(w, u) > d(v> m), u, v, w e K
3. d(u, w) = 0 if and only if v = w
(See Appendix A for a review of metric spaces.)
There is a natural candidate for a norm on the space S£ 1(X,fx).
Definition^ Let feSel(X,ii). We define H/IU = j*l/l4u to be the if1-
norm of/.
Unfortunately, ||. Id does not quite satisfy property b. Instead it satisfies
property b':
ll/l!i=0~/ = 0a.e.
This statement means that two functions / and g in ££?l(X,ii) have to be
considered the same if they are equal a.e. With this convention, it is easy to
show the following theorem.
Theorem 7. || • || x is a norm on if1 (AT, /*).
Proof. Properties a, b', and c are obvious. Property d follows by integrating
the inequality
|/(x) + g(x)\ <: |/(x)| + \g(x)\ Q

§3.1 ^-Theory
121
We will say that a sequence of functions fne££l(X,ii\ n= 1,2,...,
converges to a function fe&1(X,ii) in the J^-norm (or simply converges in
(3) U/^/II^O as *->ao
Convergence in this sense is not the same as pointwise convergence almost
everywhere. In the exercises, you will find examples of sequences that converge
in one of these two senses but fail to converge in the other. (See exercises 1
and 2.) The best one can conclude about the relationship between these two
notions of convergence is the following.
Theorem8. Suppose /„, n=l,2,..., converges to / in the if^-norm.
Then there exists a subsequence /„£, i = 1,2,..., that converges to / almost
everywhere.
We will, in fact, prove a somewhat stronger result. Recall that, for a metric
space (K, d), a sequence vneV9n= 1,2,..., is said to be a Cauchy sequence if
d{vm9vn) -> 0 as m, n -> oo. In particular, a sequence of functions/„g ££l(X,iji),
n= 1,2,..., is a Cauchy sequence if
(4) ll/m-/Jli->0 as m,w->oo
Theorem 9. Let /n, n = 1,2,..., be a Cauchy sequence in if1. Then there
exists a subsequence /„., i = 1,2,..., that converges almost everywhere to an
if1 function /. In addition, the original sequence converges to / in the
J^-norm.
Proof. Choose nx such that, for m, n > nu ||/m — fn\\i <i- Next choose
n2 > n1 such that, for m, n > n29 \\fm — fn\\i <i- Continuing inductively,
choose nt > n-^ such that, for m, n > ni9 \\fm — fn\\x < 1/2I+1. We will show
that the subsequence {f„t} converges pointwise almost everywhere. By
construction
(5) ll/.l+|-/»,lli<2HT
Let ^ = fni and let gt = fHi - fni_x for i > 2. Then
(6) /„, = t 9r
r=l
and || 3, ||! <l/2r.Thus
r=lJX

122
Chapter 3 Fourier Analysis
so, by the corollary of the Lebesgue dominated convergence theorem (corollary
12 of §2.3), the series
GO
r=l
converges pointwise almost everywhere; and, in view of equation 6, the
sequence {f„{} converges almost everywhere. Let / be the pointwise limit of the
/,{'s. By assumption,/is defined for almost all xeX, and we can define it at
the remaining points of X by setting it equal to zero at these points. It remains
for us to show that fn -> f in 5£1. Given e > 0, there exists an n0 such that
\fm-fn\dfJ.<S
for m, n > n0. Fixing n > n0 and letting m -> oo, we get
e > lim inf |/m - fn\ dp > lim inf |/m - fn\dfi
>Jl/-/J^=ll/-/Jli
by Fatou's lemma. Hence /„ converges to / in the <£ ^norm. □
Recall that a metric space (K, d) is complete if every Cauchy sequence v„ e V
has a limit veV. (Intuitively speaking, there are no "holes" in V.) A normed
vector space (K, || • ||) that is complete with respect to the metric
d(v,w)= \\v- w\\
is called a Banach space. By theorem 9, <£x{X,\x) has this property, so we
conclude the following.
Theorem 10. S£ * (X, p) is a Banach space.
Exercises for §3.1
1. Let / be the unit interval 0 < x < 1, and let lk%n be the subinterval
k fc+! „ ,
-<x < 0<fc<n
n n
Let/x be the characteristic function of/0>1,/2 and/3 the characteristic
functions of /0,2and 7i>2, /4,/s, and /c the characteristic functions of
/0,3, /lf3, and /2,3, and so on. Show that the sequence {/„} converges to
0 in S£X{1) but does not converge pointwise anywhere.

§3.1 ^-Theory
123
2. Let /„ be the function on the interval (0,1] that is equal to zero for
1/n < x < 1 and is equal to n for 0 < x < 1/n. Show that /„ converges
point wise to zero everywhere as n -► oo but does not converge in ifl.
3. In exercise 1 extract a subsequence of the sequence {/„} that converges
pointwise almost everywhere.
4. Let AT be a finite interval and \i Lebesgue measure on X. Show that
there exists a countable family of functions {/> i = 1,2,3,...} with the
property that the /f's are dense in S£x(X,\ji). That is, given any function
/ e .3? * (AT, /*) and any number e > 0, then, for some fh \\ f{ — /1| t < e. (Hint:
See §2.2, exercise 7.)
5. Let AT be a set, and let @(X) be the set of all bounded, complex-valued
functions on X. Forfe@(X) let
||/||= sup |/(x)|
xeX
Show that || || is a norm, and show that @(X) is a Banach space with
respect to this norm.
6. In exercise 5 suppose the set X is infinite. Show that, if /i,/2,.-. is a
sequence of functions in @(X), there exists a function fe@(X) such that
n/-y;ii>i
for all i. (Compare with exercise 4.)
7. a. Let p and q be numbers greater than 1 with (1/p) + (1/q) = 1. Prove
that
, ap bq
ab< — + —
P Q
for any pair of nonnegative numbers a and b.
b. Let (X,^9fi) be a measure space, and let / and g be nonnegative
measurable functions. Prove that
(*) jfgdp < ( [rd^fAg^d^
(Hint: Let a = (J/p^)1/p and 0 = ($gqdfi)1/q. At each point xeX, apply
the inequality in (a) with a = /(x)/a and b = g(x)/P, and integrate with
respect to x.)
8. Let / and g be as in exercise 7. Show that
a\ i/p / r \ i/p / r \ i/p
(/+0)p^J ^(J/P^J +(K<H
(Hint: Write (/ + ^ = /(/ + g)p~l + g(f + 0)'"1 and apply equation (*)
to each of the two products /(/ + g)p~l and g(f + g)p~l)

124
Chapter 3 Fourier Analysis
9. a. Let 1 < p < oo. A complex-valued measurable function /: X -* C is
said to be ifMntegrable if \\f\pdp < oo. Denote by £fp(X,ii) the set
of all such functions. Show that S£P(X9 /*) is a vector space. That is,
show that, if / and g are in S£P{X9 /*), so is / + g, and that, if / is in
£fp(X,n), any constant multiple of/ is in £fp(X9fi) as well.
b. If/ei?*(X,/41et
/ r V/p
WfWp
-(Jl/I'*)'
Show that || • ||p is a norm on S£p(X9\i).
§3.2 ^2-Theory
Let (AT, !F9 fi) be a measure space. A measurable function/: X -► C
is said to be S£2-\ntegrab\e or square-integrable if
(1) f |/|2^<oo
We denote by 5£2{X9\x) the set of all such functions; that is,
(2) S£\X9fi) = if: X -* C,measurable; | |/|2 d\i < oo 1
Definition 1. The quantity
(3) ii/ii2=(£i/i2^)i/2
is called the i?2-norm of fe$e2{X9\i).
We will see in a moment that equation 3 does indeed define a norm and
that ££2 is a Banach space with respect to this norm. First, however, we will
establish a few elementary facts about S£2.
Theorem 2. If/ and g are in S£2{X9/4 fg is in S£*{X9fi).
Proof Let X1 = {xeX; |/(x)| > |<?(x)|} and let X2 = {xeX; \g(x)\ > |/(x)|}.
Then, on Xl9 \fg\ < |/|2; and, on X2, \fg\ < \g\2. So
f \f9\dfi< f |/|2^+ [ \g\2dfi
JX JXi J AT 2
*mi + Mi
n

§3.2 &2-Theory
125
Corollary 3. If \i(X) < oo, then S£2(X, p) is contained in J&?1 (AT, /*).
Proof, li fi(X) < oo, the constant function 1 is in JSf2(AT,/x). □
Corollary 4. If/ and # are in $£2{X9fi), so is / + g.
Proof. It is enough to show that \f + g\2 is in JSP1, but |/ + #|2 < |/|2 +
2|/||fl| + I0I2. D
Corollary 4 says that J£?2(X, p) is a vector space over the complex numbers.
We will soon show that it has some other nice properties as well. First,
however, we need to discuss briefly the subject of inner product spaces.
Definition 5. A vector space V over the complex numbers is an inner product
space if it is equipped with a mapping
<-,*>:Kx V-+C
such that
1. <>! +V2iW} = <U1,W> + <>2,W>
2. <cu,w> = c(v,w}
3. <u,w> = (wpv)
4. <u, t;> > 0 and <u, t?> = 0 if and only if v = 0
An example of an inner product space with which you are already familiar
is the finite dimensional space C". If v — (a!,..., an)eCn and w = (bl,...ib„)e
C", the inner product of v and w is
(4) tfl,5
We will show that the much more complicated space j£?2(AT,m) is also an
inner product space. Indeed, by theorem 2, the quantity
(5) </,<?>= f fgdfi
is well-defined for/, #e j£?2(AT,/j); and it is obvious from proposition 4 of §3.1
that it satisfies properties 1-3. It doesn't quite satisfy property 4; in fact, if
<//> = Jl/l2^ = 0
the most we can conclude is that / = 0 a.e. But, if we put in force the
convention that an ££2 function is zero "in the S£2 sense" when it is zero a.e.,
then property 4 is reinstated and we have proved the following theorem.

126
Chapter 3 Fourier Analysis
Theorem 6. £?2(X, pi), equipped with the inner product given by equation 5,
is an inner product space.
We now prove a few facts that are true for inner product spaces in general
and, thus, for <£2{X, fi) in particular. Given an inner product space (7, < , »
and veV, let
(6) iM=y<^>
By property 4 this is well-defined and is zero if and only if v = 0. We call ||i;||
the norm of the vector veV.
Theorem 7. (Schwarz's inequality) If v, w e V, then
(7) \<v,n>\<\\v\\\\w\\
Proof. If w = 0, the inequality is obvious. If w ^ 0, consider, for Ae C,
0 < <t; + Xw,v + Aw> = <v,v> + I<u,w> + l<v,w> + |A|2<w,w>
Letting A = — <t;, w}/(w, w>, this inequality implies
<w,w>
or, equivalently c
|<u,w>|2<<t;,i;><w,w> □
Corollary 8. (Triangle inequality) For v,weV
(8) ll» + w||<H| + ||w||
Proof. Squaring equation 8 we get
(9) l|t; + w||2<||t;||2 + 2||t;||||w|| + ||w||2
But ||i? + w||2 = <t; + w,v + w> = <i;,i;> + <i;,w> + <w,t;> + <w,w>=||y||2 +
2Re(v, w> + || w||2, so equation 9 reduces to the inequality
2K*<»,w><2||t;||||w||
which is an immediate consequence of inequality 7. □
From this corollary we conclude the following.
Corollary 9. The norm || • || on V is a norm in the sense of definition 5 of §3.1;
that is, (K, || • ||) is a normed vector space.
In particular, we restate this result for the vector space if2(AT,/x).

§3.2 ^-Theory
127
Corollary 10. ££2(X, pi) is a normed vector space.
Moreover, applying Schwarz's inequality to ££2(X,ii\ we deduce the
following.
Corollary 11. If/ and g are in if2, then
(10) ll/fflli^ll/llillfllli
An inner product space (V, < , >) that is complete with respect to the norm
|| • || (that is, one that is a Banach space with respect to this norm) is called a
Hilbert space. For example, Cn is a Hilbert space.
We will show that if2(X,/x) is a Hilbert space. To simplify the proof we
will make an assumption about the underlying measure space (X, ^, pi). We
recall from the last paragraph of §1.3 the following definition.
Definition 12. A measure space (X, ^, pi) is a-finite if there exists a sequence
Xne&r,n= 1,2,... with (Jn°°=1 Xn = X and pi(Xn) < oo.
Without loss of generality, one can assume that Xx £ X2 ^ • * •.
Theorem 13. if2(Ar,/x) is complete with respect to the norm || • ||2; that is, it
is a Hilbert space.
Proof. Assume that X is a-finite. (See exercise 9 for a way to get rid of this
assumption.) Let {/„} be a Cauchy sequence in £f2(X9pi). Choose Xn9s as in
definition 12. By corollary 3, {/„} is a Cauchy sequence in £f1(Xl9pi). So, by
theorem 9 of §3.1, we can extract from {/„} a subsequence {/i,„} that converges
a.e. on Xx. Repeating the process, extract from this subsequence a smaller
subsequence {f2t„} that converges a.e. on X2. Continuing inductively, one
obtains for each i a subsequence^!^,,} of {/-!,„} that converges a.e. on
XL. Now apply the Cantor diagonal process: The subsequence flil9 f2t2>...
converges a.e. on X and its pointwise limit is equal a.e. to a measurable
function g. Let gx =/n, g2=fi.2> and so on. Because the 0„'s are a
subsequence of the /„'s, they are also a Cauchy sequence in if2(Ar,/x). So, for
any e > 0, there exists an n0 such that ||^m — gn\\2 < e when m,n> n0.
By Fatou's lemma, with n > n0 fixed and m -► oo,
J-
liminf|0m - g„\ dpi < liminf
But the term on the left is
\g-gn\2d\i
\0m ~ 9n\2 dp < e
J'
because gm converges to g pointwise a.e. Hence, we conclude that g is in

128
Chapter 3 Fourier Analysis
if2(Ar,/z) and that gn converges to g in if2(Ar,/z). Finally, if a subsequence of
a Cauchy sequence converges, the sequence itself converges as well, so /„ also
converges to g. □
The following general fact about inner product spaces will be useful in §3.3.
Theorem 14. Let V be an inner product space. Then the inner product < •, • >
is continuous in both variables with respect to the norm given by equation 6.
In other words, i(vn-+v and wn -► w, then (vn, w„> -> <u, w>.
Proof. livn^v with respect to || • ||, then || vn — v \\ < 1 for n large, so
\\vn\\<\\vn-v\\ + \\v\\<l + \\v\\
for n large. Then
Kvniw„} - (v,w}\ < !<!;„,w„> - <X,w>| + !<!;„,w> - <t;,w>|
^ I<v„, w„ - w>| + !<!;„ - v, w>|
^ll^llllw^-wll + ll^-^IMIwll
^(l + ll»ll)l|wll-w|| + ||i;il-r||||iy||
Hence | <t;„, w„> — <u, w>| tends to zero as n -> oo. D
Exercises for §3.2
1. Let V be an inner product space. Show that Schwarz's inequality
K^w>|<|M|||w||
is an equality if and only if w = 0 or v = cw for some complex number c.
2. Let V be an an inner product space. Show that
(*) ll^ + w||2+||i;-w||2 = 2(||t;||24-||w||2)
(Geometrically, the sum of the squared lengths of the diagonals in the
figure below is equal to the sum of the squared lengths of the sides.)
w

§3.2 ^-Theory
129
3. Let V be a normed vector space whose norm satisfies the identity in
equation (*) in exercise 2. Show that there exists an inner product on V
such that
\\v\\=y/<^>
(Hint: Show that
2Re<v,w> = \\v + w\\2 - \\v\\2 - \\w\\2
if an inner product exists.)
4. Let X = (0,1], equipped with Lebesgue measure. Show that the function
f(x) = x"3/4 is in se^X,!*) but not in S?2(X,fi).
5. Let X = [1, oo), equipped with Lebesgue measure. Show that/(x) = x~3/4
is in &2(X,ti) but not in &l{X,n).
6. Let aua2>... be a sequence of positive numbers with Y*=ian = °°- Let
sn = £?=1 a(. Show that
Z^=oo
but that
oo a
(Hfwt: Let /„ = £?=1 *ilst and let gn = £"=i ajs2. Compare fn with logs„
and gn with l/s„.)
7. Let (X, &, fi) be a measure space that is cx-finite but not finite. Show that
&2(X,jj) is not contained in if l(X,fi). (Hint: Use exercise 6.)
8. Let (X,&9fi) be a measure space and fl9f29-.. a sequence of if2
functions on X. Let
AT' = {xeX\ fi(x) =£ 0 for some i}
Show that X' is a-finite; that is, show that it is a countable union of
measurable sets of finite measure. (Hint: Let Emt„ = {xeX; \fn(x)\ > 1/m}.
Show that X' = LLnEm,n)
9. Using exercise 8, show that theorem 13 is still true without the hypothesis
that X is (7-finite.
10. Let (X, #", fi) be a measure space. Prove that J§?P(X, fi) is a Banach space—
that is, complete with respect to the norm || • ||p. (See §3.1, exercise 9.)
11. (Sobolev's inequality) Let / be a function on the interval [0,1] that is
continuous and has a continuous first derivative /'. Show that
sup l/(x)-/(y)|<||/'||2
0<x,y<l

130
Chapter 3 Fourier Analysis
§3.3 The Geometry of Hilbert Space
In this section we will discuss some of the geometric properties of
a Hilbert space ££ with inner product < , >. When we apply this material in
the later sections of this chapter, ££ will always be if2(Ar, pi) where (AT,^, pi)
is a measure space.
Definition 1. Iff^ge^^v/es^y f is orthogonal tog {written f Lg)ii{f9g} = 0.
Theorems (Pythagoras) If /, ge <£ with fig, then ||/||2 + \\g\\2 =
II/+0II2.
Proof.
\\f + 9\\2 = <f + 9,f + 9>
= </,/> + </.*> + <*,/> + <g,g>
ll2 + \\g\\2 □
More generally, suppose that /x ,/2,... ,/„ e if with /£ _L yj-, i ^./. Then by
induction it is easy to prove that || f, + f2 + • • • + fn\\2 = £?=11|/,||2.
One basic example of a Hilbert space you should always keep in mind is
C" with the inner product given by equation 4 of §3.2. This Hilbert space has
finite "dimension." We will see, however, that some Hilbert spaces are "infinite
dimensional"; in fact, these are the spaces that are most interesting to us.
Definition 3. A sequence (/>i9 <f>2, <f>$,... in S£ is called orthonormal if
Example 4. Let ^£ be C" with the inner product given by equation 4 of §3.2.
Leti;1=(l,0,...,0),i;2 = (0,l,0,...,0),...,i;ll = (0,...,0,l). Then t^,...,*;,, is
an orthonormal sequence.
Example 5. Let X = [—7r,7t], pi = Lebesgue measure, and 5£ = if2(AT,ju).
Let <j)k = (\ls/hi)eikx, — oo < k < oo. It is easy to check that the ^k's form an
orthonormal sequence. Indeed
f M-^ = 2^f e^'^dii
-{
1 iffc=J
0 if k*j

§3.3 The Geometry of Hilbert Space
131
Recall now that an orthonormal basis in Cn is, by definition, an
orthonormal sequence of vectors vl9v2,...9vn. This definition depends on the dimension
of Cn. Another characterization of orthonormal basis in Cn allows us to turn
the tables and determine the dimension from the length of the basis: Namely,
if v1,v2i...ivn is a basis of C", then there are no nonzero vectors that are
simultaneously orthogonal to all the t^'s. This motivates the following definition.
Definition 6. An orthonormal sequence ^, ^2>•••is called complete if, for any
fe&, the conditions
f-Lfa i=l,2,...
imply / = 0.
Remark. If if = ££2(X, /*), we must interpret / = 0 as / = 0 a.e.
Definition 7. Let if be a Hilbert space, and suppose that <f>l9<f>2,...,<f>nis a.
complete orthonormal sequence in S£. Then ^£ is said to have dimension n. If
5£ contains an infinite orthonormal sequence <j>x, </>2,..., ^ is said to be infinite
dimensional
We leave it to the reader to check that the dimension of a Hilbert space
is well-defined (see exercise 1).
Remark. Example 5 shows that, if X = [ — n9iz] and \x = \iL9 then £P2{X9jx)
is infinite dimensional. We will see in §3.4 that the ^k's described in example
5 are complete.
In infinite dimensions the notion of completeness of an orthonormal
sequence replaces the notion of an orthonormal basis for finite dimensions.
We now study some of the properties of a complete orthonormal sequence
in if.
Let <j>x, <j>2,... be a complete orthonormal sequence in 5£. Given f e if, let
ci = </>^»>5 this is called the ith Fourier coefficient of/ with respect to the
sequence <j>x, </>2,.... The formal series Yj%i ci</>i is called the Fourier series of
/ with respect to the sequence (j>u </>2....
Theorem 8. Let <t>u<l)2,... be a complete orthonormal sequence in S^. Let
f^5£ and let c, = </,^/>. Define Sn = £?=1 c^ to be the nth partial sum of
the Fourier series of/. Then Sn -► / in £f.
Proof. Write / = / - Sn + Sn. Notice that </ - Sn, ^> = 0 as long as i < n
because
<f,</>i> = ci = <Sn,(f>iy for i<n

132
Chapter 3 Fourier Analysis
Thus (/ — Sn) J_ S„ because Sn is a linear combination of ^'s with i < n. Then,
by theorem 2,
imi2 = ii/-sji2 + usnii2
> iisji2
Now, ||S„||2 = <Sn,Sny = (t^t^
= i\ct\2
1=1
So, we have shown that
(2) £k-|2<:|mi2 for all n
1=1
That is, Y,T=i \ct\2 converges.
Now consider Sn — Sm for n > m. Sn — Sm = /j"=OT+i Ci</>i, so by theorem 2
l|S„-SJ|2= t \c,\2
i=m+l
Because we have shown that Yj=i \ci\2 converges, we can conclude that
the sequence of S„'s is Cauchy. Because «£f is complete, we know that there is
a g e 5£ such that Sn -> g in if.
To finish the proof we need to show that / = g. Notice that, by theorem
14 of §3.2, (g,(f>i> = lim,,.^ <SW,^> = ct. Thus </ - g,^} = c, - c£- = 0 for
all i. Because the ^'s are complete, we conclude that / = g. □
Theorem 9. (Plancherel) Let <j>u <j>2,... be a complete orthonormal sequence
in if. For/eif, let ct = </,<*£>. Then ||/||2 = £Si |c,|2.
Proof. S„ -> / in if so, by the continuity of the inner product,
<s„,s„> -*</,/> = imi2
But ^.S,,) = £?=1 IcJ2, so we conclude that
I kil2 = ll/ll2 D
1=1
You may have noticed that the proof of theorem 8 did not use the
completeness of the orthonormal sequence until the final line. When the
sequence is not necessarily complete, we get the following.

§3.3 The Geometry ofHilbert Space
133
Theorem 10. Let <j>i, ^2> • • • be an orthonormal sequence in ££. For fe <£, let
c,= </»^i> and S„ = £"=1 c,^. Then S„ is Cauchy in if and thus converges
to a limit g e if. Moreover, <#, ^> = </, ^> for all i.
Proof. This proof is the same as the proof of theorem 8 with the last sentence
omitted. □
If the sequence {</>„} is not complete, we get the following in place of the
Plancherel theorem.
Theorem 11. (Bessel's inequality) Let / and c£ be as in theorem 10. Then
ll/li2>ZN2
f=l
Proof. This inequality is a direct consequence of equation 2. □
Corollary 12. The Fourier coefficients ct tend to zero as i: -► oo.
An important example of a Hilbert space is the space whose elements are
infinite sequences
(3) s = (a1,a2ia3i...)
of complex numbers satisfying
(4) £ lfl'l2 < °°
The set of all such sequences is denoted I2 (read as "little S£2"). It is easy to see
that it is a Hilbert space. In fact, it is a Hilbert space of the form if2(X,/i).
Take for X the set of positive integers—that is, X = {1,2,3,...}. Let 3F be the
(7-field of all subsets of X9 and let \x be the counting measure:
fi(A) = number of points in A
A function on X is just a sequence such as in equation 3. In the exercises we will
ask you to show that such a sequence is in &2(X, p) if and only if equation 4
holds. We will also ask you to show that for two such sequences
s = (al9a2,a3,...)
and
*= (bl9b29b39...)
their inner product is

134
Chapter 3 Fourier Analysis
(5) I a,b,
i=l
(Compare with equation 4 of §3.2.) Notice that Schwarz's inequality for I2 says
that
(6)
E«A
/oo \l/2/oo \l/2
few) CH
We will use this fact in §3.4.
Notice that, if we are given any Hilbert space 5£ and a complete ortho-
normal sequence <j>x, </>2,... in £f, then, by the Plancherel formula, the sequence
of Fourier coefficients
s = (c1,c2,...)
is in I2. Conversely, we claim that, given a sequence
(ci,c2,c3,...)e/2
the sequence of partial sums
converges in if to a limiting element /. Indeed, for m > n
m
USm-S„||2= I \c,\2
»=«+!
so Sn is a Cauchy sequence and the assertion follows from the completeness
of ^£. Thus, if we have a complete (infinite) orthonormal sequence in ££, we
get a bijective map of 5£ onto I2. This statement is rather surprising in view of
the fact that S£ can be, in principle, a much more complicated space than
I2—for example, the space of square-integrable functions on Rn.
Exercises for §3.3
1. Show that the dimension of a Hilbert space is well-defined.
2. Let X be the set of positive integers and pi the counting measure on X.
Show that S£2(X,\x) = I2. Moreover, show that the 5£2 inner product on
S£2(X^\i) is the inner product given by equation 5.
3. Let / be an <£2 function on the interval [ — n, %]. Show that
I
f(x)e inx dx^O as n -> oo
(Hint: See corollary 12.)

§3.3 The Geometry ofHilbert Space
135
4. Let (AT, «^, fi) be a measure space. A sequence fn e if 2(AT, fj),n = 1,2,..., is
said to converge in the if2 sense to fe£f2(X,n) if ||jf — fn\\2->0 as
n -> oo. Prove that, if/„ -^ /in the S£2 sense, there exists a subsequence/„.,
i = 1,2,3,..., that converges to / a.e.
5. Let X be the unit interval and \x Lebesgue measure. Show that S^2-
convergence in S£2(X,\i) does not imply pointwise convergence a.e., and
vice versa.
6. (if 2-convergence of the randomized harmonic series) Let \x be Lebesgue
measure on the unit interval, and let R„ be the nth Rademacher function.
Let
s--!,(t>'
Show that Sn converges in the S£2 sense to a function //Eif2(/,/z).
7. For fixed m and n with m> n, let A be the set
{coeI\ \Sk{a>) — Sn{a>)\ > £ for some k between m and n)
Prove that
fx(A)<^[(Sm-Sn)2dfx
Here are some hints:
(i) With k fixed, let J <= / be a union of intervals of the form i/2k < t <
(i + l)/2*, with i between zero and 2k — 1. Show that, if n < k < m,
]jRmRHdp = 0.
(ii) If J is as in part i, show that
f (Sm-Sk)(Sk-S„)dfx = 0
and also show that
f (Sm-Sn)2dfi> ((Sk-Sn)2dn
JJ JJ
(Hi) For n < fc < m, let /4k be the set
{coeI; \Sj{co) - Sn(co)\ < e for n <j < k and \Sk(co) - Sn(co)\ > e}
Show that A = \Jk=n Ak (disjoint union) and
H(Ak)<\[ (Sm-Sn)2dfi

136
Chapter 3 Fourier Analysis
8. Using exercise 7 prove that the randomized harmonic series
I(t)«
converges almost everywhere to the function H of exercise 6.
9. (The Gram-Schmidt process)
a. Let S£ be an inner product space and/i,/2,...,/n elements of S£.
Show that, if/i,...,/„ are linearly independent, then there exists an
orthonormal sequence fa,.., fa such that, for all i,
(*) <t>i = Ysaijfj
j<i
with au > 0.
(Hint: Let fa = c^/i where c[l is the length of ft. Let ^2 = c2[/2 —
(f29fa)fal where cj1 is the length of/2 — (/i^iWi- Continue.)
b. Let fx ,/2»• • • be an infinite sequence of elements of S£. Suppose that,
for all n, fx,... 9f„ are linearly independent. Show that there exists an
orthonormal sequence fa, fa,... such that equation (*) holds for all i.
10. a. Let S£ = ££2([-1,1]). Let); = l,/2 = x,f3 = x2, and so on. Apply
the Gram-Schmidt process to this sequence (see exercise 9). Show
that the resulting ^-'s are polynomial functions of x. (These functions
are called the Legendre polynomials) Compute the first few of these
functions.
Answer: fa = —j=. fa = -x
1
3
*-4(§H) *>-M*-r)
b. Use the Weierstrass approximation theorem (see §2.7, exercise 9) to
show that the Legendre polynomials are a complete orthonormal
sequence in S£2([— 1,1]).
11. a. Show that the Haar functions,
#00(x)=l for0<x<l
and
#..*(*) = <
-2"2 for^-Ux<^-i
2" 2n
2nl2 [ox±zA<x<h_
2" 2n
0 elsewhere

§3.4 Fourier Series
137
where n > 1 and 1 < k < 2W, are an orthonormal sequence in S£2([0,
1]).
b. Suppose that fe &2([0,1]) and (/, Hn,k) = 0 for all n, fc. Show that, if
A is an interval of the form [fc/2B, Z/2n], with 0 < k < I < 2W, then
1
(**) fdfi = 0
c. Show that equation (**) holds for every subinterval A of [0,1].
Conclude that {H„fk} is a complete orthonormal sequence.
d. Let Rn+1 be the n -h 1st Rademacher function. Show that
1 2"
*Ni+l = ^ Zj "».*
§3.4 Fourier Series
We pointed out in §3.3 that the functions
VV^J
einx — oo < n < oo
form an orthonormal sequence in the space if2 [—tt, jt]. We will show in this
section that this orthonormal sequence is complete. In the course of proving
this result, we will also prove a number of classical results about convergence
of Fourier series. To begin, note that, by corollary 12 of §3.3,
(1)
yjllt J -K
tends to zero as |n| -► oo, provided that / is an S£2 function on the interval
[-7C,7T].
Let / be a measurable function defined on the whole real line. We will say
that / is periodic of period 2% if
(2) f(x + 2n)=f(x) a.e.
Given any measurable function defined on the interval (—n, 7t], one can extend
it uniquely to a periodic function on the whole real line by requiring that
equation 2 hold. Moreover, if/ is periodic of period 2n and integrable over
the interval [—7c,7t], then by equation 2 it is integrable over every compact
subinterval of the real line. In fact, if / is a subinterval of length 2n9 then
(3) \ fdx = fix
J>-Lj

138
Chapter 3 Fourier Analysis
To see this, suppose that / is of the form [a — 7t, a + 7t]. Then
f(x)dx = f(x)dx+ f{x)dx- f(x)dx
JI Jn J —n J —n
But by periodicity the first term and third term cancel.
We will first study convergence of Fourier series for functions that are
rather nicely behaved. Let / be a continuous function periodic of period 2n,
and let x0 be a point on the interval [ —7c,7i]. Suppose that the right and left
derivatives of/exist at x0; that is, the limits
(4)+ lim /(*>-^o>
and
(4)_ lim MZ/M
x-(x0)_ X — X0
exist. We will prove the following theorem.
Theorem 1. The series
1 °°
S cnelnx
y/27tn=
converges at x = x0 and its limit is f(x0).
Proof. Let SN(f)(x0) be the ATth partial sum of this series. By equation 1
SN(f)(x0) )= £ ckeik*<>
4j"/M(.l/'"-"",)J*
Setting
(5) DN(x) = ±- t eik*
znk=-N
we get for SN(f)(x0) the formula
S*(/)(*o) = j * f(x)DN(x0 - x)dx

§3.4 Fourier Series
139
Making the change of coordinates x0 — x -► x, this integral becomes
fx0+n
f{x0-x)DN(x)dx
Jx0-n
So, by equation 3 we get finally
(6) SN(f)(xo) = P f(x0 - x)DN(x)dx
To estimate the right-hand side of this formula, we note first the following
properties of DN(x):
I DN(x)dx =
(7) | DN{x)dx=l
and
i ei(N+l)x _ -iNx
<8> W°> e«-i
Proof of properties 7 and 8. To obtain equation 7, just integrate equation 5
term by term and note that all terms except k = 0 have integral zero. To obtain
equation 8, rewrite DN(x) as
1 /2N
-iJVx/ y gikx
27C \fc=o
and note that, with a = eIJC, the second factor is just
2N a2N+1 - 1
V^afc, which equals —.
Next we note that the denominator in equation 8 has a zero of first order
at x = 0. In fact,
eix - 1 d
and eix — 1 has no zeroes on the interval [—n,7t] except at x = 0; so the
function
eix - 1
is continuous on this interval providing we define it to be — i at x = 0.

140
Chapter 3 Fourier Analysis
We now return to the proof of theorem 1. By equation 7
/(xoHj f(x0)DN(x)dx
Subtracting this from equation 6, we get
(9) SN(f)(x0) - f(x0) = f* [/(x0 - x) - /(x0)]D*(x)<lx
Set
, , /(x0 - x) - /(x0) /(x0 - x) - /(x0) f x ^
*W = ?TTi = -x [T^Ti)
The second factor on the right is continuous on the interval [—7t,7t], as we
just observed. The first factor is continuous except at x = 0, and at x = 0 it is
continuous on the left and on the right by assumptions 4±. Hence g is
piecewise continuous and a fortiori in «Sf2. By equations 8 and 9
SN(f)(*o)-f(xo) = ^^ gMeW+^dx-^j" g(x)e-iNxdx
The first term on the right is the — (N + l)st Fourier coefficient of g9 and the
second term is the iVth Fourier coefficient; so by equation 1 both these terms
tend to zero as N -► oo. □
A continuous function / is called piecewise differentiable if its domain
of definition is a finite union of closed intervals and if, on each of these
intervals, df/dx exists and is continuous. It is clear that, if / is piecewise
differentiable, it satisfies assumptions 4± at all points in its domain of
definition. We will show that for such functions theorem 1 can be considerably
improved.
Theorem 2. Let / be a continuous function that is periodic of period 2n. If
/is piecewise differentiable on the interval [—7c,7t], then SN(f) converges to
/ uniformly and absolutely on this interval.
Proof. Let g be the derivative of/ By assumption, g is defined and
continuous on the interval [—7c,7t] except at a finite number of points, and we will
define it everywhere by defining it arbitrarily at these points. We will first show
that, if cn(f) and cn(g) are the nth Fourier coefficients of/ and g, respectively,
then
(10) cn{g) = incn(f)
Proof of equation 10. We can find a0 = — n < at < a2 < • • * <ar = n such
that g is continuous on (af,ai+i). Then

§3.4 Fourier Series
141
T ge~inx dx = Y. \ 9e~inx dx = H\ (if) *~inx d*
fe~inx\ +in\ fe~inxdx\
U, J a, J
= fe-inx\ +in I fe~inxdx
| —n J—n
However, the first term vanishes because / is periodic.
Remark. Integration by parts is justified on [af, aI+1] because this integral
equals the Riemann integral.
To prove theorem 2 it is enough to show that £|c„(/)| < oo. Indeed,
because we already know by theorem 1 that Sn(f) -► / pointwise, this fact will
imply that the convergence is absolute and uniform. Because g is in if2,
Zlcw(0)l2 < °°> so by equation 10
E"2kC0i2<a>
Therefore, by Schwarz's inequality for I2 (see equation 6 of §3.3),
i^--i(whcj4^)\^M2)
Because both terms on the right are finite, so is their product. □
Let's now return to the proof that the functions <t>n = (l/^/2n)einx form a
complete orthonormal sequence. We have to show that, if/ is an if2 function
with </, <f>n) = 0 for all n, then / = 0 a.e. We will first show that, if / has this
property, then
(id
[b fdx = 0
for every subinterval [a,fc] of [—n,it]. Let e > 0 and let Xt be the function
indicated in the following figure:
y=\
y = 0

142
Chapter 3 Fourier Analysis
This function is piecewise differentiate. So, by theorem 2, SN(xe) -> Xe
uniformly and, hence, a fortiori in if2. Thus
0 = </,SNfc.)>-*</,*.>
that is, </, /£> = 0. As e tends to zero, xe converges in if2 to the characteristic
function of the interval [a,fc]. So, by a repetition of the preceding argument,
lim fXedx= fdx = 0
e-0 J -* Ja
Thus, we have established equation 11.
Consider now the collection of all measurable subsets of the interval
[—7r,7c] for which
(12)
fdx = 0
This collection contains all the subintervals of [—7r, n] and is a A-system;
so, by the 7r — A theorem (theorem 7 of §2.5), it contains all Borel subsets of
[—7T, 7r]. Because every measurable set is a disjoint union of a Borel set and
a set of measure zero, equation 12 holds for all measurable sets A. Prom
now on we will assume that / is real-valued. (If not replace / by its real
and imaginary parts.) Let A+ be the set where / > 0 and A- be the set
where / < 0. Then
/ \f\dx= f fdx- [ fdx = 0
J-* Ja+ Ja-
so / = 0 a.e.
The Plancherel theorem now gives us
P \f\2dx= £ \cj
J-n n=-oo
(13)
with
(H) C„—i=r/Me-** dx
y/2K J -K
for feJ?2( — 7i, 7c). We will discuss some applications of this identity in the
exercises.
Exercises for §3.4
1. a. Let / be an integrable function on the interval [—n9 n\. Let
(f(x) when |/(x)| < M
Show that j" \fM - f\ dfx -> 0 as M -> oo.
*Uy~' ]n when |/(x)| > M

§3.4 Fourier Series
143
b. Show that the nth Fourier coefficient of/ tends to zero as n -* oo. (Hint:
What can you say about the nth Fourier coefficient of /M?)
2. Let / be a periodic function of period 2n possessing continuous derivatives
up to order k. Show that
kcn(f)
\ax j
3. a. Let / be the function
<0)=«»>"
,, s fO for -n < x < 0
11 for 0 < x < n
What are its Fourier coefficients?
b. Prove that
00 1
„5(2n-l)2
4. The zeta function
7C2
~8
and
n=\n o
C(s) = I n"s s > 1
is of considerable importance in number theory. By judicious use of the
Plancherel theorem, evaluate this function at s = 2 and s = 4. Can you
devise a method for evaluating £(s) at all even integers? (//int: See
exercise 2.)
a. Let / = f(x, t) be a function that has continuous second derivatives in
x and t and is periodic of period 2n in x. Let cn(t) be the nth Fourier
coefficient of f(x, t), regarded as a function of x (that is, with t fixed).
Show that, if / is a solution of the heat equation
{) dt dx2
thencn(t) = e-"2tcn(0).
b. Given a function f0{x) that is periodic of period 2rc and has continuous
second derivative, show how to construct a solution of the heat
equation (*) with initial data: /(x,0) = /0(*)-
The Weierstrass approximation theorem says that, if / is a continuous
function on the interval [a, fc] and 6 > 0, there exists a polynomial p with
SUP |/(X) - p(x)| < 6
a^x<b
We sketched a proof of this theorem in exercises 8 and 9 of §2.7. Deduce
from theorem 2 a second proof. Here are some hints:
(i) Show that one can assume — n < a < b <n.

144
Chapter 3 Fourier Analysis
(ii) Show that every continuous function / on the interval [a, b] can be
extended to a continuous function on [—7r,7c] that is periodic of
period 2n.
(iii) Show that, if / is a continuous function that is periodic of period In,
there exists a piecewise differentiate function f0 that is periodic of
period 2n and is e/4 close to /; that is,
SUP |/-/ol<T
-ic<x£n 4
(Hint: See figure.)
(iv) Let Sn(x) = Y,-Ncneinx> the JVth partial sum of the Fourier series for
f0. Show that, for N sufficiently large,
sup \SN-f\<-
(v) In the formula for SN9 replace einx by J]?=0 (l/r!)(mx)r with k large.
7. Show that in theorem 1 it is enough to assume that / is continuous and
dijfferentiable from the left and from the right at x0 and is piecewise
continuous elsewhere.
8. a. Compute the Fourier series of the sawtooth function
s(x)
{x — Tt
\x -I- n
— n for 0 < x < n
for —7t < x < 0

§3.5 The Fourier Integral
145
b. Show that the series
1
£ -einx
converges everywhere on the interval — n < x < n except at the origin.
9. Let / and g be if2 functions on the interval (—71,71]. Extend them to
functions on R by requiring that they be periodic of period 2n. Show that
the convolution
«-±r.
is in if 1( — n9n) and that its Fourier coefficients cn are just
where a„ and bn are the Fourier coefficients of/ and #.
§3.5 The Fourier Integral
Let / be a complex-valued integrable function defined on the real
line. Its Fourier transform is the function
-I'
(1) fiy)=\me-">dx
Notice that this function is well-defined because the absolute value of the
integrand is |/(x)|. Indeed
\M\ =
f{x)e-^dx\< \\f(x)\dx
J'
so f(y) is bounded by the JSP1-norm of/ (In exercise 2 you will be asked to
show that f(y) is continuous and that f(y) -» 0 as y -> + oo.)
On the interval (—n,n)9 i?2-integrable functions are automatically if1-
integrable; however, for functions defined on the real line, this is no longer
the case (see §3.2, exercise 5). Therefore, equation 1 does not make sense if
the integrand is an arbitrary S£2 function. Nevertheless, we will show that
equation 1 can be appropriately defined for S£2 functions and that, just as for
Fourier series, the 5£2-theory of the Fourier integral is remarkably simple and
elegant.
We will start by studying the Fourier transform for a very well-behaved
class of functions.

146
Chapter 3 Fourier Analysis
Definition 1. Let / be a complex-valued function defined on the real line
whose derivatives of all orders—that is, df/dx, d2f/dx2, and so on—exist and
are continuous. Then / is called a Schwartz function if, for each pair of
nonnegative integers m and n, there exists a constant C (depending on m and
n) such that
(2)
<®
<C
We will denote by 5 the set of all Schwartz functions. Iff and g are in S,
so is / 4- g; and, iff is in S, constant multiples off are in S and so are xf and
df/dx. Also, equation 2 implies that, given N, there is a constant C, depending
on AT, so that
(3) i/mi < c(i + \x\2rN
So Schwartz functions go to zero very rapidly as x -► ± oo. The basic example
of a Schwartz function, about which we will have much to say in the next two
sections, is the function e~(x2/2).
By equation 3 Schwartz functions are S£ Mntegrable; so their Fourier
transforms are defined. We will prove that the Fourier transform of a Schwartz
function is again a Schwartz function. To see this fact we need a fundamental
property of the Fourier transform.
Lemma 2.
1. LetfeS and let g(x) = xf(x). Then g{y) = yf^\(d/dy)}(y).
2. Let/e S and let h = df/dx. Then k(y) = yf^itfiy).
In other words, up to factors of ^f — 1, the Fourier transform interchanges
the operations "differentiation by x" and "multiplication by x."
Proof. By definition
f(x)e~ixydx
yw-Jj
The integrand on the right is differentiable with respect to y, and the derivative
is again integrable; so the left side is differentiable with respect to y, and
= -i xf(x)e-ixydx
= -ig(y)

§3.5 The Fourier Integral
147
which proves part 1. To prove part 2 we note that
= -//oo<J--'">«*
K)e'ixydx
= iyj7(x),
The integration by parts is justified by the fact that / is going to zero very
rapidly as x -► + oo. V
It follows from the lemma that, if / is a Schwartz function, then (d/dy)f
and yf are the Fourier transforms of Schwartz functions, and by induction
ym(dn/dyn)f is the Fourier transform of a Schwartz function for all m and n.
In particular, ym(dn/dyn)f is bounded; so/is a Schwartz function.
Example. Let / = e~{x2/2\ We will show that
(4) /(y) = v^^(y2/2)
That is, up to a constant, / is its own Fourier transform.
Proof. Notice that / satisfies the differential equation
(5) f + */=0
Indeed, up to a constant factor, / is the only solution of this equation, for, if
dh , r,
— + xh = 0
dx
then (d/dx)ex2/2h = ex2/2[(dh/dx) 4- xh] = 0. So ex2/2h is equal to a constant C
and ft = Ce~ix2,2). By lemma 2, / satisfies equation 5 if / does; so f(y) is a
constant multiple of e~{y2,2\ All that remains to check is that this constant is
JIjl. But, if f(y) = Ce~{y2/2\ then
= /(0) = JV(':
72)
rf),
The integral on the right can be evaluated by elementary means and shown
to be yfhi. □
We can now state the first main result of this section.

148
Chapter 3 Fourier Analysis
Theorem 3. The mapping / -> / maps S bijectively onto itself. Moreover, if
feS and g = /, then / = g where
(6) g(x) = —\g(y)eixydy
Remarks.
1. The function g in display 6 is called the inverse Fourier transform of g.
Notice that it is very simply related to the Fourier transform of g—namely,
-(s)«-
(7) M-[^)«-y)
From this identity it is clear that the inverse Fourier transform maps the
Schwartz space into itself.
2. Equation 6 implies that the Fourier transform is injective as a map of
S into S. (That is, iffeS and g = / = 0, then, by equation 6,/ = 0.) It also
implies that the inverse Fourier transform is surjective as a map of 5 into S.
(That is, iffeS and # = /, then/ = g) But, because of the simple relation
(equation 7) between the Fourier transform and its inverse, we conclude that
both the Fourier transform and the inverse Fourier transform are injective
and surjective. So, if we can prove equation 6, we will have automatically
proved the rest of theorem 1. Incidentally, equation 6 is usually referred to as
the Fourier inversion formula.
For the proof of equation 6 we will need some additional properties of the
Fourier transform.
Lemma 4. Let / and g be Schwartz functions. Then
(8) jf(y)g(y)dy = jf(xMx)dx
Proof. By Fubini's theorem
jto)g(y)dy = ^f{x)e-^dx\g{y)dy
= \(\g{y)e-ixydy\f{x)dx
= J6(x)Xx)dx
The interchange of integrations is justified by equation 3.

§3.5 The Fourier Integral 149
Lemma 5. Let feS and let a be a real number. Then, iffa(x) = /(x + a),
(9) faiy) = e*f(y)
Proof. By definition
/.GO- |/(x + a)<T'*Mx
So, if we make the change of variables x = s — a, this becomes
Liy) = *iay IfWe-tods = eia*f(y)
Lemma 6. Let fe S and let a be a positive number. Then, if fa(x) = /(x/a),
(10) Liy) = af(ay)
Proof. By definition
fa(y) = ^ffye-lxydx
So, if we make the change of variables x = as, this becomes
h(y) = a^me-ia**ds = aKay)
We will now prove equation 6. Let/ = /(x) be an arbitrary Schwartz function,
and let g = e~{y2/2a2). Then, combining equations 4, 8, and 10, we get
(11) J f(y)e-{y2/2a2) dy = Jhta | /(x)£T<fl2*2'2> dx
When we make the substitution ax = s, the right side becomes
(12) V&f/Qe-*2*
Now let a -> + oo. Then e"(y2/2fl2) tends to e° = 1 uniformly on compact sets;
so the left side of equation 11 tends to J/QO dy. On the other hand, by display
12, the right side of equation 11 tends to
y/2Hf(0)[e-<*
i2/2)ds = 2tc/(0)
and we obtain

150
Chapter 3 Fourier Analysis
(13) /(0)
-spw*
Now let g be the function g(x) = f{x + a). Replacing / by g in equation 13
and taking into account equation 9 we get
M = 9(0) = ^ [ mdy = ^\f(y)eiaydy □
Next we will show that the Fourier transform preserves 5£2-norms up to
a scalar factor.
Theorem 7. Let / be in 5. Then
(14) ll/lli = 2*||/||5
Proof. If we take complex conjugates of both sides of the identity
we get
= ±\e*f(y)dy
f{*) = ^e-ixytfmdy
That is, 2nf is the Fourier transform of / Let g = f, so § = 2nf. Then, by
lemma 4
|l/l2^ = J//^ = J/ff^ = Jj»ix
= \f(2nf)dx = 2n\\f\2dx D
Remark. This identity is called the Plancherel formula for the Fourier
transform.
Using theorems 3 and 7 we can now define the Fourier transform of an
arbitrary if2 function. The idea for this definition is based on a theorem about
metric spaces: Suppose M and N are metric spaces and A is a dense subset of
M. A map fiA^Nis called uniformly continuous if, for every £ > 0, there
exists a S > 0 such that dN(f(x)9f(y)) < e whenever djuix, y) < 8. The theorem
we will need is the following.
Proposition 8. If/: A -> N is uniformly continuous and N is complete, there
exists a unique continuous mapping g: M -+ N extending/

§3.5 The Fourier Integral
151
A proof of this fact is outlined in Appendix A.
By the Plancherel theorem, \\f — 0\\% = 2n\\f- g\\l; so the Fourier
transform is uniformly continuous as a map of S into J?2. Moreover, as we
saw in §3.2, <£2 is complete. Therefore, if we can show that S is dense in J?2,
then, by proposition 8, there is a unique extension of the Fourier transform
from S to S£2. In other words, if we can show that S is dense in &2, we
will have succeeded in our goal of extending the Fourier transform to S£2
functions.
To show that S is dense in &29 we need to show that there is a large supply
of Schwartz functions. The results that we describe next make this point. We
say that a function /: R -► R is C00 if all of its derivatives—that is, df/dx,
d2f/dx2, and so on—exist and are continuous.
Lemma 9. There exists a C00 function f0 that is zero for x < 0 and positive
for x > 0.
Proof. The function
"(1/x) forx>0
«-fc"
fo(x' '" forx<0
has this property.
Lemma 10. Given an interval (a, b)9 there exists a C00 function fx such that
/i = 0 for x${a,b) and/t > 0 for xe(a9b).
Proof. Let f0 be as in lemma 9 and let
/iM = /o(x-fl)/0(fr-x) V
Lemma 11. Given an interval (a, b), there exists a C00 function f2 such that
f2(x) = 0 for x < a, /2(x) = 1 for x > b, and 0 < f2 < 1 on the interval (a, b).
Proof. Let
^(sjds
/2(*) = "
J —oo
(s)ds
Lemma 12. Given £ > 0, there exists a C00 function / of the type depicted in
the following figure.

152
Chapter 3 Fourier Analysis
a a+ e b—e b
Proof. By lemma 11 there exists a C00 function g such that g = 0 for x < a,
g = 1 for x > a + e, and 0 < g < 1 on the interval (a, a + s). Similarly, there
exists a C00 function ft such that ft = 0 for x < ft — e, ft = 1 for x > ft and
0 < ft <_1 on the interval (ft - £, b). Now let /(x)_= #(x) [1 - ft(x)]. V
Let S be the closure of S in if2; that is, feS if and only if there exists a
sequence f„eS,n= 1,2,..., such that || f„ — /1| 2 -» 0. It is clear that, if g and
ft are in 5, then # + h is in S and constant multiples of g are in S.
Proposition 13. Let A_ be a finite union of intervals. Then the characteristic
function \A of /4 is in S.
/V00/. It is enough to prove this proposition for an interval A = [a, ft].
Given e > 0, let /be as in lemma 12. Because/is C00 and is zero outside [a, ft],
/ is in S. Moreover, \f — 1A\ < 1 on (a9a + s) and (ft — e,ft), and f=lA
elsewhere, so
J|/~lyl|2dx<2e □
Proposition 14. Let >4 be a measurable set of finite measure. Then the
characteristic function 1A of A is in S.
Proa/ Choose £ > 0. By the definition of J(¥ (see §1.3), there exists a finite
union of intervals B such that fi(S(A,B)) < s. Then
[\lB-lA\2dx = ti(S(A,B))<e □
Now let / be a nonnegative J£?2 function. By theorem 6 of §2.2, there exists
an increasing sequence of simple functions sn > 0 such that s„ -> / By
proposition 14, s„eS. Moreover
l/~s«l2^-^0 as n->oo
by the monotone convergence theorem, so feS. Finally, let / be a complex-.

§3.5 The Fourier Integral
153
valued if2 function. Then
/ = Re(f)+ - Re(f). + >/=Ilm(/)+ - ,/=! Jm(/)-
so / is in S. Thus we have proved the following theorem.
Theorem 15. S is dense in if2.
We can now prove the second main result of this section.
Theorem 16. There is a unique linear mapping
(15) ~:i?2->if2
and a unique linear mapping
(16) ~:£e2^&2
such that, restricted to S, equations 15 and 16 are the usual Fourier transform
and inverse Fourier transform. These mappings are bijective and satisfy the
Fourier inversion formula
/=(/r
and the Plancherel formula
ll/ll! = 2K||/||!
Proof. We have already indicated how the Fourier transform can be
extended to S£2. The inverse Fourier transform can be extended the same way.
Moreover, the Fourier inversion formula and the Plancherel formula hold on
5; so, by continuity, they hold on S£2. □
Remark. For a general if2 function /, the integral in equation 1 doesn't
make sense. How then do you evaluate/? Of course, you have to approximate
/by functions for which equation 1 does make sense, and then take limits.
See exercise 4 for an explicit way to carry out this manipulation.
Exercises for §3.5
1. Show that, if/eif^R), then for every e > 0 there exists a Schwartz
function g such that \\f — g\\i < e.
2. If/ is in ifx (R), show that / is continuous and /(<!;) -> 0 as ^ -> ± oo. (Hint:
These assertions are true when feS. Now use exercise 1.)
3. Show that, if / is both in &l(R) and in «^2(R), the two definitions of
/—that is, equation 1 and the definition by continuity—coincide.

154
Chapter 3 Fourier Analysis
4. a. For/e«£P2(R)andM>0,let
r ( \ _ J/M when M ^ M
when |x| > M
Show that \\fM - f\\2 -► 0 as M -+ oo.
Show that, if /eif2(R), then
lim
Af-*oo J -M
(*) lim f(x)e~ixydx
M-+ao J -M
exists, in the sense of if2, and is equal to /. (Equation (*) is often used
as the definition of the S£2 Fourier transform.)
Show that, if/, ge&2{R\ then
-(s)0.«
(This identity is called Parseval's identity.) (Hint: The real part of </, #>
is equal to i(||/ + 0ll5-!/»!-11*111).)
6. a. Compute the Fourier transform of xe *2/2 and of x2e *2/2. Can you
devise a scheme for computing the Fourier transform of xme~x2/2 for
any ml
b. Show that for every integer m there exists a polynomial //m(x) of
order m such that the Fourier transform of Hm(x)e~x2/2 is a constant
multiple of itself. Moreover, show that one can choose the Hm's so that
the sequence Hme~x2/2, m= 1,2,..., is orthonormal. (The polynomial
Hm(x) is called the mth Hermite polynomial.) (Hint: Use exercise 5.)
7. a. Let c be a positive number and let fc be the function
'-6
.. . fe"" forx>0
UX)= 0 forx<0
Show that its Fourier transform is l/(c -I- iy).
Use the Plancherel formula to compute the integral
I
dy
-„c2 + y2
8. a. Let / be the characteristic function of the interval [—1,1]. Show that
its Fourier transform is (2 sin y)/y.
b. Use the Plancherel formula to compute
Jl(^
) dx
9. a. If/ and g are in Sf1, the convolution off and g is the function

§3.5 The Fourier Integral
155
(f*9)(x)=\f(x-y)g(y)dy
Show that / * g is in S£l (R) and that || / * g \\ x < \\ f \\ x \\ g || x. (Hint: Use
the Fubini theorem.)
b. Show that the Fourier transform of /* g is the product fg.
c. Conclude from part b that the convolution operation is associative
and commutative.
10. a. Show that the function
satisfies the heat equation dg/dt = d2g/dx2 for 0 < t < oo and — oo <
x < oo.
b. Show that, iff e S, the function
ix(x,t) = gt*f= g(t9x - y)f(y)dy
satisfies the heat equation for 0 < t < oo and — oo < x < oo.
c. Show that, as t -► 0 +, /x(x, t) -> f(x). (Hint: Using part b of the
previous exercise, show that J \(lt(y) — f(y)\ dy -► 0 as t -► 0 +. Here ixt(x) =
fx(x,t).)
11. a. Let AT be a set, 3F be a c-field of subsets of AT, and /x a probability
measure on X. Given a random variable/: X -► R, the function
XrW= f «v*i
is called the characteristic function of /. Show that x7 is continuous
and |x/(OI < 1.
b. Suppose / is bounded. Show that all the derivatives—(d/dt)xf,
(d2/dt2)xf, and so on—exist and are continuous. Show that
12. a. Let (AT, «^\ /x) be as in exercise 11. Show that, if two random variables
/ and g are identically distributed, then Xf = Xg-
b. Conversely, show that if X/ = Xg then f and g are identically
distributed. (Hint: Let h(x) be a Schwartz function and ft(t) be its Fourier
transform. Using the Fourier inversion formula, show that
00 Xf(t)kt)dt = f h(f)dn
-co J*
2*1

156
Chapter 3 Fourier Analysis
and interpret the right-hand side as
I-**'
where vf is the probability distribution of/)
13. Let (X9^,fi) be as in exercise 11. Show that, if/ and g are independent
random variables, then X/+g = XfXg- (Hint: See equation 6 of §2.6.)
14. a. Let X be the unit interval, & the Borel sets, and /x Lebesgue measure.
Show that, if/ = R„ is the nth Rademacher function, then xf = cos t.
b. Let S„ = Yj=i Rt> Show that the characteristic function of Sn is (cos t)n.
c Using part b of exercise 11, show that
fsn2fcd/x = (~l)k^[(cosOn]|
(Compare with §1.1, exercise 19.)
15. Let H = X^=i(Vw)^n be the randomized harmonic series. Let % be its
characteristic function. Show that
*w=flc°s(0
§3.6 Some Applications of Fourier Series to
Probability Theory
Let pn, — oo < n < oo, be a sequence of nonnegative real numbers
with the following three properties:
(1) Pn = P-n
(2) pn = 0 for all but finitely many n's
and
(3) Zp.= l
We will consider in this section a generalized version of the random walk
in which a point-mass moves randomly along the real line with transition
probabilities #_,. To be more specific, suppose that, at time fc, the position of
the point-mass is the integer point i. At time k + 1 the point-mass is allowed to
move to any integer positionj for which p£_j is nonzero and the probability of
its moving to this position is assumed to be p£_j. (For instance, if px = p^ = \

§3.6 Some Applications of Fourier Series to Probability Theory
157
and the other pk's are zero, the process we have just described is the usual
random walk.) To normalize, we will assume that the position at time zero is
the origin.
The basic random variables associated with this process are for each k9
(4) the difference between the positions at time k and time k + 1
Notice that the probability distributions v associated with these random
variables do not depend on k; in fact, they are all just the measure
(5) v(A)=Y,Pr
reA
for every Borel subset A of R. Indeed, if A is the one-element set consisting of
the integer r, then v(A) = pr9 the probability that the point-mass moves r units
to the right (or left) at time k.
Now let X = I be the unit interval, fF = Bj the Borel sets, and fx = fxL. In
§2.6 we showed that there exist independent, identically distributed random
variables ft: I -> R, i = 1,2,..., such that equation 5 is their common
probability distribution. If we take (X9lF,fx) to model the sample space of the
process described above and the /fc's to model the random variables described
in display 4, it is clear that we get an adequate measure theoretic model of
this process. In this model the sum
(6) Sn=tft
1=1
is the position of the point-mass at time n.
Let's consider the question of when and how often the point-mass returns
to its initial position. In our model the probability that the point-mass returns
to its initial position at time n is the measure of the set
{coeI;Sn(co) = 0}
In the remainder of this section, we will show that
(7) »({Sn = 0;Lo.})=l
That is, with probability one the point-mass returns infinitely often to its initial
position. (Incidentally, we will assume from now on that the transition
probability p0 is less than one, for otherwise equation 7 is trivially true: The
point-mass stays at the origin forever with probability one.)
The first step in the proof will be to get a simple description of the measure
of the set where S„ = 0. This step will be done using Fourier series. Consider
the sum
(8) g(t)= £ pne
int

158
Chapter 3 Fourier Analysis
By equation 2 this sum is finite; so there are no problems of convergence. We
will show the following proposition.
Proposition 1. The measure of the set
{coeI;Sn(co) = r}
is the rth Fourier coefficient of the function gn.
Proof. We claim that for all k
(9) g(t) = E(eitf*) = I e^dpt
Indeed, display 4 tells us that fk is an integer-valued function, taking on only
a finite number of integer values. In addition, it tells us that the measure of
the set
Em = {(oeI;fk(a)) = m}
is just pm. Thus the right-hand side of equation 9 is
Z^(/k = m) = ^/- = #
as claimed. Because /i,...,/, are independent, so are eitfl,...,eitfn; so
E(eits») = E(eitf> x ••• x eitf») = E(e^) x ••• x E(eitf»)
by equation 6 of §2.6. By equation 9 the right-hand side is g(t)n. On the other
hand, because S„ is also an integer-valued function, taking on only a finite
number of integer values,
E(e^) = (e^d/i = 2>(S„ = r)el»
Comparing the Fourier coefficients of this series with those of g(t)n, we see
that fi(Sn = r) is the rth Fourier coefficient o{g(t)n. □
Before continuing with the proof of equation 7, we point out a few
properties of the function g that we will use in our proof:
(10) g is real-valued,
(11) <7(0) =1, <7'(0) = 0, and 0"(O)<O,
(12) \g(t)\ < 1,
and
(13) \g(t)\ < 1 except at a finite number of points on the interval [—n, it]

§3.6 Some Applications of Fourier Series to Probability Theory
159
Proof of properties 10-13: By equation 1
so g is real-valued. Differentiating equation 8,
0(0) = EP„, g'(0) = i^npn, and s"(0) = -2>2p„
By equation 3 the first sum is 1, and by equation 1 the second sum is zero.
Finally, the third sum is negative, because, by equation 3, pn ^ 0 for some n # 0
(because we are assuming that p0 ^ 1). To prove inequalities 12 and 13, note
that
g(t) = Reg(t) = YJPncosnt
so
l0tol<IpJcosnt|<Xp„=l
with equality holding if and only if cos nt = ± 1 whenever pn # 0. v
Using these facts we will prove the following proposition.
Proposition 2. The sum
(14) JM{^U» = 0})
n=0
is infinite.
Proof. By proposition 1 this sum is identical to the sum
The integrand in the integral on the right is nonnegative and monotone-
increasing (why?) and converges to the limit
1
1-0(0
except at those points where g(t) = — 1. Because the points with this property
are finite in number, by equation 13, we get from the monotone convergence
theorem
00 1 Cn dt
Because the integrand on the right is nonnegative, we can show that the

160
Chapter 3 Fourier Analysis
left-hand side is infinite by showing that
dt
i:
+ 00
for some e > 0. Let C be a constant with
g"(0)>-2C
Because g(0) = 1 and #'(0) = 0, we get from Taylor's formula with remainder
that
g(t) > 1 - Ct2 > 0
on a small interval, — e < t < e. Hence
f£ dt fE dt
If the events Sn = 0 were independent, we would now be finished: We
could deduce that fx({Sn = 0; i.o.}) = 1 from proposition 2 using the second
Borel-Cantelli lemma. However, because these events are not independent, we
have to resort to a slightly more complicated argument. Let k be a positive
integer and, for every positive integer /, let
fl = fk+l
Let Bk be the set of co e / where
(15) Z/iM#0f r<K and £ fta) = 0
»=i f=i
Similarly, let B\ be the set of coel where
(16) tyi'M^O, r</, and £//(a)) = 0
Because the ^s in display 15 and the /"s in display 16 are independent, Bk
and B\ are independent.
Let pk = n(Bk) and pz' = ii(B\). We claim that, for k = /, pfc = p/. Indeed, let
^ be the joint probability distribution associated with fi,...,fk, and let n' be
the joint probability distribution associated with //,..., fk. By theorem 3 of
§2.6,
n = fxfi x fxf2 x • • • x fifk = v x • • • x v
and a similar identity holds for n', so
n = n'

§3.6 Some Applications of Fourier Series to Probability Theory
161
Now pk is the measure with respect to /x of the set in display 15. But, by the
definition of n, this measure is the same as the measure with respect to n of
the set
<(xi,...,xn)eRn; |>f t* 0, r < k, and J>, = 0>
Similarly p'k is the measure of this set with respect to n'. Because n = n\
Pk = Pk as claimed. By definition, Bk is the set of random paths that return
for the first time to the origin at time k; so the sum
oo
is the probability that a random path returns at least once to the origin.
Similarly Bk n B\ is the set of random paths that return to the origin for
the first time at time k and for the second time at time k + /. Because Bk and
B\ are independent, the probability of this event—that is, the measure of
Bk n B\—is pkpt; and the probability that a random path returns at least twice
to the origin is
L PkPi = [ LPk)\LPi) = P
JM=1 \k=l J \l=l J
We leave for the reader to show, by a similar argument, the following
proposition.
Proposition 3. If p is the probability of a random path returning to its initial
position at least once, the probability of its returning at least k times is pk.
We will deduce from this proposition that p must equal 1. In fact, suppose
that p < 1. Let
Ak = {a>eI;Sk(co) = 0}
and let
(17) h = £ 1^
Notice that, for m < oo, the set
(18) {(oeI;h(a)) = m}
is the set of random paths that return to the origin exactly m times; so, by
proposition 3, the measure of this set is pm — pm+l or pw(l — p). For m = oo,
the set in display 18 is the set of paths that return to the origin infinitely often;
so, by the proposition, its measure is less than pk for all k; in other words, it
is zero (assuming that p < 1). Thus

162
Chapter 3 Fourier Analysis
I
hdfi = £ mpm(\ - p) < oo
f m<oo
On the other hand, by equation 17
oo
I,
and the quantity on the right is infinite, by proposition 2; so we get a
contradiction and conclude that p = 1.
Let Ck be the set of paths that return to the origin at least fc times. Because
p = 1, p(Ck) = 1 by proposition 3. Because Q => C2 ^ C3 • • •,
"(,QC')
1
Hence, we conclude that random paths return to the origin infinitely often with
probability one.
Exercises for §3,6
1. Show that, for the classical random walk (p_x = p1 = j), every integer
point neZ is visited at least once with probability one. (Hint: Suppose
that the random walk visits the integer point n with probability p < 1.
If n > 0, consider the set of random walks having the following two
properties: (i) The first n moves are to the left, (ii) The origin is never
revisited. Prove that the probability that a random walk belongs to this set
is (1 — p)(l/2") > 0, contradicting equation 7.)
2. Show that, for the classical random walk, every integer point is visited
infinitely often. (Hint: Let q be the probability that the point n is visited
at least once, and let p be the probability that the random walk returns
at least once to the origin. Show that the probability that n is visited at
least k times is qpk~l. (Compare with proposition 3.) Now use the fact that
p = q=l.)
3. a. For the generalized random walk with transition probabilities
satisfying equations 1 through 3, let nl9...,nk be those integers for which
pn t* 0. Let A be the set of those integers that can be written in the
form rxnx + h rknk) with integers rx,..., rfc, and let n be the greatest
common divisor of nl9...9nk. Show that A consists of all integer
multiples of n.
b. We will call an integer point a on the real line accessible if a e A. Show
that a is visited with probability greater than zero if and only if it is
accessible.
4. Show that, if an integer point a is accessible, then with probability one it
is visited infinitely often by the generalized random walk. (Compare with
exercises 1 and 2.)

§3.6 Some Applications of Fourier Series to Probability Theory
163
5. a. Let g(t) be the function given by equation 8. Show that g(t) = 1 if and
only if t is a multiple of 2n/n (the n here being the same n as in exercise
3, part a).
b. Conclude that g(t) = 1 if and only if cos kt = 1 for all admissible fc's.
c. Show that if g{t) = 1 then
Conclude that g"(t) < 0 when g(t) = 1.
6. Let
TN(0) = t Prob(Sfc = 0) and TN(r) = £ Prob(5fc = r)
*=i fc=i
Show that, if r is admissible, lim[7^(0) — TN(r)~] is finite as N -> oo and is
equal to
1 f* 1 -cosrt
(*) ^- "J rr*
(Hint: Use part b of exercise 5 to show that the integrand in equation (*)
is a bounded function of t)
7. a. Show that, for the classical random walk (p_x = px = \\
Prob(S2„
-•»-#)
b. By Stirling's formula (see S. Lang, A Complete Course in Calculus.
Reading Mass., Addison-Wesley, 1968), there exists a number 0, with
0 < 0 < 1, such that
n\ = x/2nnnne-nee/12n
Deduce from Stirling's formula that
Prob(S2„ ~ 0) = -=
for n large.
c. Prove from part b that
£ Prob(S2n = 0) = oo
n=l
[Notice that Prob(S2n+1 = 0) = 0. Why?]
8. a. For the "unfair coin" [the process described in theorem 5 of §2.6
with fe = 2, rt = 1, r2 = -1, Pi = p ±\, and p2 = (1 - p) # ±], let
S„ = /i+ ••+/„. Prove that
Prob(S2„ = 0) = (2nj p"(l - pf and Prob(S2n+1 = 0) = 0

164
Chapter 3 Fourier Analysis
b. Using Stirling's formula show that
X Prob(Sn = 0) < oo
and conclude (by the first Borel-Cantelli lemma) that Prob(S„ = 0;
i.o.) = 0. Why doesn't this contradict equation 7?
9. Suppose that the transition probabilities pn, — oo < n < oo, satisfy
equations 1 and 3 but not equation 2. Show that if £ n2p„ < oo, proposition 2 is
still valid.
10. a. For the generalized random walk with transition probabilities
satisfying equations 1 through 3, let Nn(k) be the number of times the point k
is visited during the interval of time 0 < t < n. Show that the
expectation value of Nn(k) is
£ Prob(S, = k)
1=0
(Hint: Let Br be the set of paths that visit the point k exactly r times
during the period 0 < t < n. Show that
AT„(fc) = LrlBr
Let A{ = {o; S^o) = k}. Show that
Nn(k) = t UO
1=0
b. Show that the expectation value of Nn(k) is
-r
2tiJ_
cos kt— at
1 -g
c Using exercise 6 show that, if k is admissible, the expectation value of
Nn(k) differs from the expectation value of Nn(0) by a quantity that
tends to a finite limit as n -► oo.
§3.7 An Application of Probability Theory to Fourier Series
In this section we are going to discuss a classical theorem about
Fourier series due to G. Szego. This theorem not only is of considerable
theoretical interest, but also has a number of practical real-life applications.
For those who want to learn about these applications, we recommend the
very readable book by Grenander and Szego (Toeplitz Forms and Their
Applications. Berkeley, Calif.: University of California Press, 1958).

§3.7 An Application of Probability Theory to Fourier Series
165
Szego proved his theorem in 1916, and since then several other proofs of
it have been discovered. The proof described below is due to Mark Kac and
consists of reversing one of the key arguments of the previous section.
We begin by making a few definitions: Let 88 be the Borel subsets of the
real line. Suppose we are given a probability measure \i and a sequence of
probability measures /*!, /x2> • • • on ^- We will say that \in converges weakly to
\x if, for every bounded continuous function /,
h-l>
a) yd^^fdix
The notion of weak convergence will play an important role, not only in the
following discussion, but also in the formulation of the central limit theorem
in the next section.
The second notion we will need involves some elementary linear algebra.
Let T = (atj), 1 < ij < N9 be an N x N matrix of complex numbers with
(2) au = an
It is a standard theorem in linear algebra that T has N real eigenvalues; that
is, the equation
det(A - T) = 0
has N real roots Xx,..., XN (potentially occurring with multiplicities). For every
subset A e ^, let fi(A) be the number of A£'s contained in A9 counting
multiplicities. The measure fi defined by this recipe is called the spectral measure of
T. If / is a bounded continuous function then
(3) jfdii = jtf(X,)
Having the notions of weak convergence and spectral measure, we can
state a provisional form of the Szego theorem. Let a„, — oo < n < oo, be a
sequence of complex numbers satisfying
(4) 0-n=^
and
(5) t l«nl<<*>
n=— oo
Associated with these numbers is the infinite matrix
T = (amn) with amn = am_„ - oo < m, n < oo
Matrices of this form are called Toeplitz matrices. Notice that, by equation 4,

166
Chapter 3 Fourier Analysis
T satisfies the symmetry condition in equation 2. Let
(6) TN = (am_n) 0<m,n<N-l
be the AT x AT principal minor of this matrix. The Szego theorem in its
provisional form says that, if fiN is the spectral measure of TN9 then fiN/N has
a weak limit as N -> oo. We will identify this limit shortly, but before doing
so let's observe an interesting tie-in between Toeplitz matrices and Fourier
series. Let
(7) <Z(0)= f a„eM
n=-co
By equation 5, q is continuous. Moreover, by equation 4
m = lL*-nein0 = YJanein° = q(e)
That is, q is real-valued. Let i?2 be the space of i?2-integrable functions on
the interval [ —7c,7c] and let
Tq: &2 -► J&?2
be the linear mapping that sends fe £f2 to qfeS£2. Then, by equation 7
(8) Tf«"= J) a„e<<«+^= J an.m
^fcrf
That is, an_m is the matrix associated with Tq in terms of the basis einB9
— oo < n < oo, of i?2. The matrix 7^ has a similar description: Let VN be the
vector subspace of «£?2 spanned by the functions eM9 0 < n < N — 1, and let
P:J2f2->Fw
be the orthogonal projection of i?2 onto J^. In other words, iff is in i?2 and
its Fourier series is
— oo
then
Let
be the linear mapping
(9)
JV-1
pf= EV-
0
(VN:VN-*VN
(Tq)Nf = PTqf

§3.7 An Application of Probability Theory to Fourier Series
167
Then, for 0 < m < N - 1,
(Tq)Neime = Y an_mein°
n=0
by equation 8; so the matrix associated with (Tq)N in terms of the basis ein09
0 < n < N - 1, is exactly TN.
Consider the probability measure
®
/*L
on the interval [ — n97t\. If we think of the function q: [ — n,7t] ->R as a
random variable, its probability distribution \x is defined by the formula in
equation 3 of §2.6; that is, for every Borel function /,
(10) [fdii = ^\*flq(0ndiiL
We can now state the Szego theorem in its sharp form.
Theorem 1. Let fiN be the spectral measure of 7^, and let \x be the measure
in equation 10. Then
(id ^
weakly as N -► oo.
Let us see what equation 11 says in concrete terms: Let
Aj"> i=l,...,AT
be the eigenvalues of 7^, and let / be a bounded continuous function on the
real line. Then by equation 3
so equation 11 is equivalent to the assertion that
(12) jjt fMN)) ^^[ /M] de
as N -> oo, for every bounded continuous function /.
We will now give a heuristic justification of equation 12 in terms of
probability theory. Suppose that
an = Pn — 00 < M < 00

168
Chapter 3 Fourier Analysis
with the p„'s being transition probabilities satisfying equations 1 through 3 of
§3.6. As in §3.6 we will consider the random walk based on these transition
probabilities; but, instead of assuming that the random path starts at the
origin, we will assume that it starts at one of the points x = 0,..., N — 1 and
that all these points are equally likely as starting points. We will confine
ourselves to a fixed finite interval of time 0 < t < m. Thus a sample path can
be described as a zigzag line consisting of m segments. An example of a path
for m = 3 is shown in the following figure. The t coordinate indicates time,
and the x coordinate indicates position.
t
(2, 3) f
(2,2)i
>(3,1)
I 4- x
(2,0)
We will compute the return-time probabilities for this process as in §3.6
but with a "confinement" condition imposed: What is the probability that a
random path returns to its initial position at time t = m and stays confined in
the box 0 < x < N — 1 and 0 < t < m? (For instance, in the figure the
confinement condition says that, at times t = 0, t = 1, t = 2, and t = 3, the x
coordinate of the path has to lie in the interval 0<x<N — 1.) Let us denote
the probabilities in question by p(m, N). We claim that
(13) p(m,N) = i£(A<.">r
JM £=1
Proof. We will prove the case of m = 3, the general case being essentially no
more difficult. Let k be a point on the interval 0 < x < N — 1. Because our
random path has to start at some point in this interval and because all N
points are equally likely, the probability that it starts at k is 1/AT. What is the
probability that at t = 0 its position is fe, at t = 1 its position is Z, at t = 2 its
position is n, and at t = 3 its position is again k? Clearly this is
( "y jPl-kPn-lPk-n
The probability p(3, N) is therefore the sum

§3.7 An Application of Probability Theory to Fourier Series
169
(14) (jjlZPl-kPn-lPk-n
over 0 <k,l, n < N - 1. But the sum in display 14 is also just l/N times the
trace of the matrix T$\ that is,
(15) p(3,N) = (^) traced
In terms of the eigenvalues X\N\ this trace is just
EOT
establishing equation 13 for m = 3. □
Let's now drop the constraint condition—that is, no longer require the
random path to be constrained to lie on the interval 0 < x < N — 1 but only
require that its initial position lie on this interval. What is the probability,
for an unconstrained random path, that its positions at time t = 0 and at
time t = m coincide? Clearly this is just the "return at time m" probability
computed in §3.6; that is, it is just
**-*£
q(0)mdO
where q(6) = Y,Pneine by proposition 1 of §3.6. Now it is intuitively clear that
as N-> oo, with m fixed, p(m, N)-> p(m). Indeed, if we make the interval
[0, N — 1] extremely large relative to m, relatively few paths with an initial
point in this interval leave it before t = m in view of property 2 of §3.6. Hence,
we conclude
<"> sJ^-sJI
mmde
for f(x) = xm. By taking linear combinations of xm's we see that equation 16
is true for any polynomial function /, and a simple application of the Weier-
strass approximation theorem shows that it is true in general.
The following series of exercises will give you a chance to strip the
probabilistic scaffolding from this proof. Henceforth, an, — oo < n < oo, will be an
arbitrary sequence of complex numbers satisfying equations 4 and 5; that is,
the a„'s will not necessarily be the transition probabilities associated with a
random walk.
1. Show that
271 J-
q3dO= £ arasa,
i r+s+t=0

170
Chapter 3 Fourier Analysis
2. Suppose that all but finitely many of the a„'s are zero. Show that
3. Let a'n, — oo < n < oo, be another sequence of numbers satisfying
equations 4 and 5. Suppose that, for all n, \an — a'n\ < e. Let M be the larger of
the two sums
Efc.1 and £WI
Show that
Z ar<*s<tt- Z (
r+s+t=0 r+s+t=0
<3eM2
4. Similarly, show that
(*) (— ) | trace Tj - trace (T^)31 < 3eM2
5. Show that equation (*) is true without the assumption that all but finitely
many of the an's are zero.
6. Prove equation (*) with the power 3 replaced by the power m.
7. Let a = min q and b = max q. Show that the eigenvalues of 7^ lie on the
interval a < X < b, (Hint: Use equation 9.)
8. Let / be a continuous function on the interval [a, b]. By the Weierstrass
approximation theorem, there exists for every s > 0 a polynomial function
p such that sup|/ — p\ < e. (See §2.7, exercise 9, or §3.4 exercise 6.) Use
this theorem in conjunction with exercise 7 to prove that fiN converges
weakly to \i.
9. (Szego's original version of the Szego theorem) Let DN be the determinant
of TN. Show that, if q is bounded below by a positive number, then DN>0
and
lim Dk,N = exp|^- \ log q(9) dO]
N-oo L2nJ-n J
§3.8 The Central Limit Theorem
In §2.7 we proved the following version of the law of large numbers.
Theorem 1. Let/i,/2,... be a sequence of bounded random variables on X
that are independent and identically distributed. Let E = £(/j) be the common
expectation value of the /£'s. Let X0 be the set of points xe X for which

§3.8 The Central Limit Theorem
111
n
as n -> oo. Then /jl{X0) = 1.
In other words, if
S.(x) = /i(x) + "- + /.(x)-n£
then
(2) ^U
n
as n -> oo with probability one. In many practical problems, one would like
to know how fast this convergence is. A close look at the proof of theorem 1
gives the following theorem.
Theorem 2. Let/t ,/2> • • • be as in theorem 1, and let a > 0 be given. Then
as n -> oo with probability one; that is, Sn(x)/n -> 0 faster than n~il,2)+a with
probability one.
Proof. The first step of the proof is to verify the following lemma.
Lemma 3. For each k> 0 there is a Ck > 0 such that
(4)
£tfk^<Ckn*
This lemma can be proved by induction. The case of k = 2 is done in the
proof of theorem 1. The induction step is fairly messy but straightforward,
and we leave it to the reader. (See §2.7, exercise 5.)
Now, given a > 0, choose k so that 1/fc < a; notice that equation 2 and
Chebyshev's inequality give
(W>«})-'({i^>'"})
- E2knk+2ka ]x
(5) *tt*s)xSr*l*
< <
— Ob Olrrt —
£2kn2ka - e2*„2+/>
where J? = 2fca - 2 = 2fc(a - 1/k) > 0.

172
Chapter 3 Fourier Analysis
Now, as in §1.1, choose a sequence el9e2,... with en -► 0 and
CO C
La 2k 2+P
„=l tn n
Let
< oo
_ f r y. l ^W J . 1
71 "~ J X e A' M(l/2)+a n (
Then, by equation 5, Y*=i A*(4„) < °°. So, by the first Borel-Cantelli lemma,
fx({An; i.o.}) = 0. But, for x in the complement of {An\ i.o.}, we must have
Sn(x)
M(l/2)+«
<£„
for all but finitely many n's. Hence, we conclude
S„(jc) ^
__->0 as M^°°
with probability one. □
A natural question to ask at this point is "What happens to SJn112 as n
-> oo?" To formulate the answer to this question, we need a definition. Let
/ be a bounded random variable with expectation value E. The integral
-J,"-
V(f) = ^(f-E)2dfi
is called the variance of /. It is regarded by probabilists as a good
measure of "deviation of / from its expectation value," because by Chebyshev's
inequality,
(6) ll{xeXt\f-E\>M}^^(f-Efdn = ^-
For instance, inequality 6 says that the set where / deviates by one unit from
its expectation value is less than V(f). If V(f) is very small, so is this
deviation.
Notice that, if v is the probability distribution associated with /, then by
equation 3 of §2.6
= f {x-Eft
V(f)=\(x-E)2dv
So, if two random variables are identically distributed, they have the same
variance.
Let's now return to the question posed earlier. Because/i,/2,... are
identically distributed, V(fx) = V(f2) = •••.

§3.8 The Central Limit Theorem
173
Theorem 4. Let a = V(fl) = V(f2) = • •. Then, for every pair of numbers a
and b, with a < b
(7) JxeX;a<^-<b)^-^[be-t2f2adt
as n tends to infinity.
This theorem is called the central limit theorem. It is sometimes stated as
saying that, if the deviations of the /£'s from their expectation value E, for
1 < i < n, are rescaled by the factor n1/2, then these deviations tend to be
normally distributed for n large.
Notice that, if we denote by \in the probability distribution of the random
variable SJnl/2 and by J the interval (a, ft), the left-hand side of equation 7 is
fi„(J). The right-hand side, on the other hand, is
J2%G J J
(8) V.(J)- 7=\e-'^'dliL
The measure \ia defined by equation 8 is called the Gaussian or normal
distribution with variance a. The central limit theorem says that, for every
interval J,
(9) \in(J)^\ia(J) as n->oo
Proof of theorem 4. Replacing f{ by ft — E, we can assume that E = 0. We
will first prove a statement similar to equation 9 for the Fourier transforms
of/*„and/v
Lemma 5. Let
Xn(t)=\ e-ixtdfin
Then, for fixed t, xn{t) -+ e~ff'2/2 as n -► oo.
Proof By equation 3 of §2.6,
= f f e-Wirfdi*} x ••• x M e-itf»'fndv\

174
Chapter 3 Fourier Analysis
by independence. Because the yj's are identically distributed, equation 3 of §2.6
tells us that
I e-itf^dfi= I e-'W"d/i = ---= I e-*fnisT"dii
Jx Jx Jx
In fact, each of these expressions is equal to
1
e-itx/s/"dv
R
where v is the common probability distribution of the /,'s. Therefore, letting
/ be any one of the ft% we get for xn the formula
do) ^w=(jv^^y
Now notice that, for t fixed,
where rn is a bounded function that tends to zero uniformly as n -> oo.
Integrating the right-hand side, taking into account the fact that E = \fd\i = 0,
we get
(11) £^//^^=1_^(1+fin)
where
<* Jx
S« = T / rndfi
The Lebesgue dominated convergence theorem implies that
(12) e„ -► 0 as n -> oo
If we substitute equation 11 into equation 10, we get
'■'"-['-©"-H"

§3.8 The Central Limit Theorem
175
Therefore, the proof of lemma 5 reduces to showing that
(13) [i_(£)(i+0"
• e-<"2/2
as n -> oo. For this proof, set a = —at2/2 and take the log of both sides. The
left-hand side becomes
»log[l + Q(l+o]
or, with s = (1 + £„)/n,
(l+£rt)
s
In view of equation 12, the limit of this expression as n tends to infinity is
log(l + as) d
km = — log(l + as)\s=0 = a
s-o s as
which is exactly the log of the right-hand side of equation 13. V
We now return to the proof of theorem 4. We begin by proving an
essentially equivalent statement. We will show that, if/ is a Schwartz function,
then
(14) f /<te,-> [ fdixa
Jr Jr
as n -> oo. To see this, note that
L^-sLd.'^*)*-
!H/M*
The last expression, however, limits to
4= f f(y)e-**l2'dy
12-kg Jr
by lemma 4 of §3.5 and lemma 5 of this section. Hence, we have established
equation 14.
To show that equation* 14 is essentially equivalent to equation 9, consider
a Schwartz function ft of the form indicated in the following figure. (We
showed in §3.5 that Schwartz functions of this type do exist.)

176
Chapter 3 Fourier Analysis
y=l
a a+ t
Let J be the interval (a, b). It is clear that
\fsdfxn<fxn(J)
so by equation 14
liminf/i„(J) > lim ftd\in = fzd\i,
n-*oo J J
rb-B
> e-x2/2°dx
Ja+E
The last inequality holds for all e > 0, so we obtain
liminf/*„(J)> e~x2/2(rdx = fia(J)
A similar argument shows that
limsupA*n(J)<A*ff(J)
so we conclude that the limit exists and is equal to the expression on the
right. □
Another formulation of the central limit theorem is that the sequence of
measures \xn converges weakly to the measure fj,ffasn-+ oo. Recall from §3.7 that
this statement means that
(15)
Jr Jr
for every bounded continuous function /. By equation 14 we know this fact
to be true when / is a Schwartz function; and, by approximating by Schwartz
functions, one can easily show it to be true for any bounded continuous
function.
Example: coin tossing. Suppose that in theorem 4 the /£'s are the Rademacher
functions Rt. The strong law of large numbers says that (SJn)(co) -> 0 as

§3.8 The Central Limit Theorem
177
n -> oo. A gambler might want to know how many trials it takes to be
reasonably sure this quantity is near zero. For example he or she might want
to know that \Sn\/n < .01 with a probability of 99%. Because a = ViRJ =
V(R2) = ••• = !, one gets from equation 7 the estimate
.99 = JLeX;{^<m\)
(16) =/i(|x6X;^<.01>Aj)
*-t= e-t2*dt
V27r J-.01JH
.Oly/n
ly/n
By numerical methods one can show that, if
2= I e-'2/2dt = .99
/2n j-a
then a = 2.57.... Hence, by equation 16, .01 y/n « 2.57 or n » 66,000; that is,
after 66,000 tosses one can be 99% sure that \Sn\/n < .01.

Appendix A
Metric Spaces
We collect here, in a minimal sense, the important facts about
metric spaces used in the text. This exposition is in no way complete and is
meant only as an easy reference for the reader who is already familiar with
these concepts. For a more thorough treatment of this matter, see W. Rudin,
Principles of Mathematical Analysis, 3rd Ed. (New York: McGraw-Hill, 1976).
Let M be a set.
Definition 1. A metric on a set M is a map d( •, •): M x M -> R satisfying the
properties
1. d(x,y) = d(y,x)
2. d(x,y)>0
3. d(x9 y) = 0 if and only if x = y
4. d(x, j;) < d(x, z) + d(z, j;)
If d is a metric on M, the pair (M, d) is called a metric space.
Example 2. Let (K, || • ||) be a normed vector space (see §3.1). Then V is a
metric space with metric
d(x,3>)=l|x-)i
Metric spaces are nice because they allow us to define the basic topological
objects we are used to considering in Rn—for example, open and closed sets,
compactness, convergence, and so on. We first discuss convergence.
178

Metric Spaces
179
Definition 3. Let (M, d) be a metric space. Let xx, x2, x3,... be a sequence in
M. We say {xw} is a Cauchy sequence if
(1) d(x„,xm)->0 as n,ro->oo
We say x„ converges to x e M (written x„ -► x), if
(2) d(x, x„) -> 0 as n -> oo
Proposition 4. If xt, x2,... is a sequence in M with x„ -► x e M, then the x/s
form a Cauchy sequence.
Proof. d(xni xm) < d(xn9 x) + d(x, xm) -> 0 as n, m -> oo D
Thus, just as in Rn, every convergent sequence in (M, d) is a Cauchy
sequence. The converse, however, is not true in general.
Example 5. Let M = R — {0} and let d(x9 y) = |x — y\ for x, yeM. Clearly
the sequence x„ = 1/n is Cauchy, and yet there is no xeM such that x„ -> x.
Experience in Rn tells us that it is nice to be able to use display 1 as a
criterion for convergence. This motivates the following definition.
Definition 6. (M, d) is called complete if every Cauchy sequence in M
converges to an element of M.
Example 7. Rn with the metric
/ n \l/2
(3) d(x9y) = ^Z(xi-yi)2)
is a complete metric space. We leave this fact for the reader to check. It follows
from the fact that R with the metric in example 5 is complete.
When we consider convergence of sequences, it is sometimes useful to
know if a subsequence converges. One check for this is that the sequence be
contained in a compact set. To define compactness in a metric space, we need
to study the open and closed sets.
Let (M, d) be a metric space and let A c= M be a subset. We say x e M is a
limit point of A if for every s > 0 there is an xe e A with xe ^ x and d(x, x£) < e.
The set A is called_c/osed if it contains all its limit points. In general, if A c= M,
the closure of A, A, is the smallest closed set containing A, It is easy to check
that A consists of the points in A together with all the limit points of A. A set
1/cMis called open if its complement is closed.
Proposition 8. If U a M is open, then for each x0 e U there is an r > 0 such
that the ball

180
Appendix A
(4) Br(x0) = {x e M; d{x9 x0) < r}
is contained in U.
Proof, By definition, U is open if and only if Uc is closed. Now x0 £ Uc9
so it is not a limit point of Uc. Hence there exists an r > 0 such that
Br(xo)nUc = 0. a
A set K <=. M is called compact if, whenever {Ua} is a collection of open sets
covering K9 there is a finite subcollection [/ai,l/a2 l/^ covering K. (A
collection {Ua} of sets covers K if K c (J [/a.)
Proposition 9. Let (M,d) be a metric space and let xl9x2,... be an infinite
sequence in M. Suppose there is a compact set X c M such that xneK for all
n. Then there is a subsequence xni, x„2,... of the xw's that converges.
Proof. It is enough to show that the set A = {xj, x2,...} has a limit point.
Suppose this is not true. Then for each yeK there is an sy such that, if
By = {x; d(x9y) < sy}9 then By contains at most one point of A. The collection
{By}yeK is an open cover of K with no finite subcover. This fact contradicts
the compactness of K. □
This proposition gives a nice criterion for existence of convergent
subsequences. The rub is that compactness is generally hard to check. However,
in the case that the metric space is Rw with the usual metric given by equation
3, the Heine-Borel theorem gives a simple criterion for compactness.
Theorem 10. (Heine-Borel) Every closed, bounded subset of Rw is compact.
Remark. A set A cz Rn is bounded if there is a number M > 0 such that
d(x, y) <M for all x, y e A.
Warning. The Heine-Borel theorem is not true in a general metric space.
To prove theorem 10 we need the following lemma.
Lemma 11. Let (al9a2,•••,«„) and (bi9..,bn) be elements in Rn with aj < bj
for all j = 1,2,..., n. Then the closed multi-interval J = {x e Rn; at < x£ < fcj
is compact.
Proof. Suppose, on the contrary, that J is not compact. Then there must be
a cover of J by open sets {Ua} that has no finite subcover.
If w& let Jf c R be the interval It = [ai9 fcj, we note that J = Ix x I2 x
• • • x J„. Let ct = j(a( + bt) and let
ir = lai9Ci] It = [ci?bj i = l,...,n

Metric Spaces
181
Then there are 2" multi-intervals of the form
/±x/± x-x/±
all of which are covered by the Ua's. At least one of these multi-intervals must
not allow a finite subcover because J doesn't. Choose one such multi-interval
and call it Jx. Now repeat this process ad infinitum to get a sequence of
multi-intervals
J => Jx =) J2 => • • *
none of which can be covered by finitely many of the l^'s.
For each fc take xkeJk, Notice that the sequence {xfc}£L, is a Cauchy
sequence because the size of Jk decreases as fc -> cxd. Because Jx is closed there
is a limit point x0 e Jx. Also, for each / the tail of the sequence {xfc}£L, is in the
closed set Jt. Hence x0eJt for each /; that is, x0e f]fit Jt. Choose <x0 so that
x0 6 Uao. By proposition 8 there is an r > 0 such that jBr(x0) c Uao. We claim
that, for some fc, Jk c Br(x0). This claim contradicts the construction of the
Jks and thus proves the lemma. To prove this claim let
/ n \l/2
^ = (Z(^-^)2J
and note that for x, y e Jk, d(x, y) < A/2*. Now, x0 e Jk for all fc, so we have that,
if xe Jk, then d(x9x0) < X/2k\ that is, Jk c Bx/2k(x0). Choosing fc large enough
so that X/2k < r we are finished. V
Proof of theorem 10. Let C be a closed, bounded subset of Rn. Because C is
bounded, it is contained in some closed multi-interval J. Because C is closed,
the set Rn — C = U0 is open. Now let {Ua} be an open cover of C; the collection
{Ua} u {U0} is then an open cover of J. By the lemma, J is compact so
there is a finite subcover. If U0 is among these, throw it out; what's left still
covers C. □
Finally, we can combine proposition 9 with theorem 10 to get the
Bolzano-Weierstrass property.
Theorem 12. (Bolzano-Weierstrass) Every bounded infinite set in Rn has a
limit point.
Now suppose that (M,dM) and (N,dN) are two metric spaces. A function
/: M -> N is continuous at x0eM if for every e > 0 there exists 5 > 0 such
that dM(x0iy) < d implies that dN(f(x0),f(y)) < e. The function / is called
continuous if it is continuous at each point of M. Notice that when checking
continuity the 8 may vary depending on x0. The function / is called uniformly
continuous if the 6 can be chosen independently of x0; namely, / is uniformly
continuous if for every s > 0 there is a 8 > 0 such that dN(f(x),f(y)) < e
whenever dM(x, y) < 5.

182
Appendix A
Proposition 13. Let/: M -* N be uniformly continuous and let xl5 x2,... be
a Cauchy sequence in M. Then/(x1),/(x2),... is a Cauchy sequence in N.
Proof. Given s > 0 we need to find K such that dN(f(xi)9f(Xj)) < s
whenever i, j > K. Because / is uniformly continuous, there is a 8 > 0 such that
dM(x,y) < 8 implies that dN(f(x),f(y)) < e. Because the x,'s are a Cauchy
sequence, we can find K such that dM(xh xf) > 8 whenever ij < K. This is the
K we sought. □
Now let (M, dM) be a metric space and let A c M be a subset. A is
automatically a metric space with the metric induced from dM. In §3.5 we
encounter a continuous function f:A->N and we wish to extend it to a
continuous function g:M ->N. When/is uniformly continuous, we can make
this extension if we assume that N is complete and that points in M can be
"approximated" by points in A in the following sense: A subset A cz Mis called
dense if A = M.
Theorem 14. Let (M, dM) and (AT, djy) be metric spaces. Let A c M be a dense
subset of M, and let /: A -► JV be uniformly continuous. Assume that N is
complete. Then there exists a unique continuous mapg:M -> AT such that
0(x) = /(*) f°r all x g ^
/Voq/I We begin by defining g. Let x e M. If x e X we set #(x) = /(x). If x £ A
then, because 4 is dense in M, x must be a limit point of A, Choose a sequence
xl5x2,... in A with Xj-^xinMas/-> oo. By proposition 13 the sequence
/(x£) in N is Cauchy. Because N is assumed to be complete, we know that
this sequence has a limit; define this limit to be #(x). Notice that this definition
is independent of the choice of sequence xl9x2i... because, if x\9x'29... is
another such sequence, then f(x[) -> g(x). In this fashion it is also easy to see
that, given s > 0, there exists a 5 > 0 such that, if dM(x, y) < 8 for x e M and
ye A, then dN[g(x\g(yy] < e. To prove continuity of g, take x, yeM with
dM(x, j;) < 8/2. Because A is dense in M, we can find zeA with dM(x,z) < 8/2.
Then dM(y,z) < dM(y,x) + dM(x>z) < ^ an(* so
dw[0(x),0(y)] £ 4v[0(*)>0(*)] + dN[g{z\g(yy] < 2s
whenever dM(x,y) < 8/2.
The uniqueness of g is a direct consequence of its continuity. □

Appendix B
On <£p Matters
You recall that in §3.8 we proved the following improved version
of the law of large numbers.
Theorem. Let/i,/2>--- be bounded, independent, identically distributed
random variables on the probability space (X9fi). Let E be the common
expectation value of the /j's, and let Sn(x) = fx(x) + • • • + fn(x) — nE. Then,
for any a > 0,
as n -> co with probability one.
To prove this theorem we assumed that the /j's were bounded so that we
wouldn't have to worry about the integrability of the functions S*k. Actually,
if a > 1/fc it is easy to see that equation 1 holds as long as
Juir
(2) I l/l* < oo
Indeed, in order to get the estimate in inequality 4 of §3.8, all you need to know
is that
(3)
[\fi\lf£ x - x #| < oo when lt + - + I, = 2k
It turns out that inequality 3 follows from inequality 2 once we have some
basic facts about if p-spaces, which you have already proven in the exercises
of §3.1 (see corollary 5 of this appendix). Here we first review those basic facts
183

184
Appendix B
(don't peek until you've looked at exercises 7, 8, and 9 of §3.1 and exercise 10
of §3.2) and then we develop some ifp analogues of some of the i?2-theory in
Chapter 3.
Basic Theory
Let (X,n) be a measure space. Recall that, for p > 1, S£p{Xy\x) (or just S£p
if X,fi is understood) is the set of complex-valued measurable functions
f:X^C such that
(4) f|/|'4i<oo
The value
(5) ii/iip=(Li/|pd/x)1/P /ejsfp
is called the JS?p-norm of/.
Theorem 1. S£P(X, fi) is a vector space and || • ||p is a norm on S£P(X, /i).
To prove theorem 1 we will need the following lemma of calculus.
Lemma 2. Let #(t) be a convex function on the interval (a, b); that is, <j>"(t) > 0
for all te(a,b). Let x, ye(a,b) and let a and P be nonnegative numbers with
a + p = l. Then
^(ax + #y) < ou/>(x) + /ty(j;)
/Vaq/I Suppose this lemma is not true. Then, for some a, /?, x, and y as above,
we have
</>(<zx + ft;) > <x^(x) + jS^O)
or
<xO(ax + ft;) ~ <*(*)] > PW(y) - <*(** + ft,)]
because a + /? = 1. Dividing by a/?(j; — x), we get
#(coc + fly) - </>(x) ^ fly) - ^(ax + ft;)
(ax + ft;) - x y — (ax + ft;)
By the mean value theorem there exist f and rj with x<£<ax + Py<ri<y
such that ^'(f) > fiW- This contradicts f(0 > 0. V

On Sev Matters
185
Proof of theorem 1. First, to check that J?p(X,fi) is a vector space, we need
to see that, if/,ge ££P(X, fi), then J|/ + g\p < oo. To do this, notice that for
p > 1 the function <ft(t) = tp is convex; hence, with a = /? = ^, we conclude from
lemma 2 that
@m+5i*i)'4i/r+5i*r
pointwise in X. Integrating this inequality gives
J 1/ + 0|'4* < 2'j Q|/| + ^Id)'** < 2*"1 (j l/IM/z + | IflfK^
Hence,/ + 0 e JS?'(*f ji).
To show that || • ||p is a norm on £fp(X9fi)9 we need to prove the triangle
inequality; that is, \\f + g \\p < \\f\\p + II0II p. (The rest of the norm properties
are obvious.) To prove this inequality, we need the following.
Lemma 3. Let / and g be nonnegative measurable functions on X, and let
p and q be numbers greater than 1 with (1/p) + (1/q) = 1. Then
(6) ffgdp < [\fPd^JP(\ ^^J"
Proof Let a and b be positive numbers. Define the numbers x and y by
a = ex,p and b = ey,q. Then, because ex is convex and (1/p) + (1/q) = 1, we have
from lemma 2 that
ab = e(x/p)+(y/q) < igx + \ey = iap + ±fcf
p q p q
Now let
a = -m n-ttt and b =
(W (I9**)'"
and integrate to get inequality 6.
To prove the triangle inequality, consider
= 11/1(1/1 + \9\y~1 dp + j \g\(\f\ + \g\r~1 dp

186
Appendix B
Apply lemma 3 with q = p/(p — 1) to get
| (i/i + M)p^ < (| mp^)1/P (ja/i+ i^dp^)(p 1>/P
so
(7)
(J !**)"([
(jfl/l + \g\rdfij" :
\(p-D/p
(l/l + Wdftj
f + gL*[ (l/l + MW <\\f\\p +\\g\\
Remark. The triangle inequality for || • ||p, inequality 7, is called Minkowski's
inequality.
We can now give an argument to show that inequality 2 implies inequality
3. First, we have the following proposition.
Proposition 4. (Holder's inequality) Let p and q be numbers greater than 1
with (lip) + (1/q) = 1. Let fe&p(X9fi) and gtSe\X,\i\ Then fge&l(X,p)
and
I
fgdfi
^ UUgh
Proof. By lemma 3 we have
j\fg\dii = j\f\\9\dii£m,M,
sofge^X.^and
Ufgdfi\^j\fg\dn^\\f\\p\\g\\q
D
Corollary 5. Let pt and p2 be greater than 1, and let /i e J?"'(X, /i) and
f2 e 2 "*{X, n). Then ft f2 e #'•'*«'»+'*>(X, /t) and
II/1/2L II /1ll„II/2
Proof. Consider
l/l/2|PlP2/(Pl+P2>^= |/1|PlP2/(Pl+P2)|y2|PlP2/(Pl+P2)^

On Sev Matters
187
Notice that/x e&p*(X9/i) implies that//lP2/(p'+P2)e J^*'2^2, and, similarly,
//ip2/(Pl+P2)eif(P1+P2)/Pl.Now[p2/(pi +pa)] + q,i/(Pi +pa)] = i, so we can
apply proposition 4 to conclude that
f /r \P2/(pi+P2) / r \pi/(pi+p2)
j (i/iii/2Dpip2/(pi+p2)^ < y i/iipi^j m i/2ip2^j
so
11/1/2 HplP2/(Pl+P2) <ii/iIU/2iu n
Now take k functions in iffc,/l5/2,...,/fc. By corollary 5, fx &£&**,
Applying corollary 5 again gives/i/^ = (fif2)he^m- Continuing in this
fashion, we get/i/2 x • • • x fke S£1; hence inequality 2 implies inequality 3.
Notice also that corollary 5 gives the following.
Corollary 6. Suppose fi(X) < 00. Then &p(X,ii) <= Se\X,\i) for 1 < r < p
and
(8) 11/11, <c||/||p for fe&IX.p)
where c = [/*(*)] (p~r)/pr-
Proof. Because fi(X) < 00 the constant function 1 is in «£?5(X, fi) for all s > 1.
Let 5 = pr/(p - r). Then, by corollary 5Je&p*p+s) = Ser and
11/11, <ll/llpll Ills □
To continue now with the basic properties of ifp-spaces, recall that a
normed vector space is called a Banach space if it is complete in the metric
topology.
Theorem 7. JSf P(X, p) is a Banach space.
Proof. We will prove this theorem in the case that X is a-finite; that is,
X = (JjSj Xt with Xx < X2 < • • • and /*(AT£) < 00 for all i. We leave the more
general case to the reader (see exercises 8 and 9 in §3.2).
Let {fn}™=i be a Cauchy sequence in «S?p(Z,/z); that is, given e > 0 there is
an N such that || /„ — fm \\p < e whenever n,m> N. We wish to show that there
is a function feJ?p(X,n) such that /„ -> f in «£?p; that is, given e > 0 there is
an N such that \\fn — /||p < e when n> N.
We proceed-as in the proof of theorem 13 of §3.2. Because ii{Xx) < °°
we get from corollary 5 that $gp{Xu\i) c S£\Xu\i)> and the /„'s form a
Cauchy sequence in i?1^,/*). Thus, from theorem 9 of §3.1, we can extract a
subsequence {/i,„}^=i that converges a.e. on Xx. Similarly, because n(X2) < 00

188
Appendix B
we can extract a subsequence {/2fW}"=i from the sequence {/i,n}"=i that
converges a.e. on X2. Continuing inductively we extract a subsequence {fiin}ZLi
from {/•-!,„}"=i that converges a.e. on Xt. By the Cantor diagonal process,
the subsequence fiti,f2,2> • • • converges a.e. on X to a measurable function /
on X. Let gx = /ltl, g2 = /2,2, anc* so on. The sequence {gn}T=i is Cauchy in
&p(X9ii); that is, for s > 0 there is an JV such that \\gm — gn\\p < e when
m,n> N.
By Fatou's lemma, with n > N fixed and m -> oo,
liminf |fc - flj'dji < liminf |^m - gn\pd\i < e
The term on the left is J I/— gn\pdfi9 so this inequality shows thatfeJ?p(X,fi)
and that gn-+ f in J?p(X9n). Because {^„}^=i is a subsequence of {/„}£Li, it
follows that fn -► f in J£?p as well. D
Representation Theorems
Recall that, for the Hilbert space £f2(X, n), we have Schwarz's inequality
If/^Ull/IUMU fovf,ge<?2(X,ti)
I V I
The generalization of this inequality to if-spaces is Holder's inequality
I fgdp\z \\f\\P\\g\\q, feX1X,n\ gB<e\X,ti\ - + -=1
This inequality can be interpreted in the following way: For geJ?q(X,ij)
consider the linear map
(9) /,: X"(X, rf^C where /,(/) = jfg dp
It is an immediate consequence of Holder's inequality that this map is
continuous. Indeed, to show that a map F: ££p(Xy pi) -> C is continuous at /0,
we need to show that, given e > 0, there is a 3 > 0 such that \\f — f0 \\p < 3
implies that \F(f) — F(f0)\ < e. In this case
Uf) - Wo)l = \IJJ - /o)l < 11/ -fo\\P\\9\\q
so, if we take 3 = e/\\g \\q9 we are done.
The purpose of this section is to convince you that, in fact, every
continuous linear map /: ££P{X, n)-+Cis of the form lg for some g e £?q(Xy fx) where
(1/p) + (l/q) = 1. We will not prove this fact in general; we will just prove some
special cases of it. If you want to see the general proofs, we recommend the

On Sev Matters
189
treatment in Reed and Simon (Methods of Modern Mathematical Physics:
Functional Analysis, vol. 1. New York: Academic Press, 1980).
We first introduce some nomenclature. Let (K, || • ||) be a normed, complex
vector space. A continuous linear map /: V -> C is called a continuous linear
functional. The dual space V* is the space of all continuous linear functionals
/: V -► C. Notice that V* is a vector space.
Proposition 8. Let I: V -> C be a linear functional. / is continuous if and only
if there is a constant c> 0 such that
(10) l(v)<c\\v\\ fomllveV
Proof. The argument that inequality 10 implies continuity is the same as the
argument that lg is continuous. To show that continuity of / implies inequality
10, notice that, because / is continuous at zero, if we choose e > 0 there is a
5 > 0 such that ||u|| < 8 implies \l{v)\ < e. Let c = 2e/d. Then for ve V notice
that
\M<> - I'Gr)I=>>»<'
Hence |/(v)| < (2fi/«)|N| = c||w||. D
Now let H be a Hilbert space with inner product <•,•>. If v e H we define
lv e H* by lv(w) = < w, t;>, weH. (lv is continuous by Schwarz's inequality.) This
definition gives a map
L:H-*H* by L(v) = lv
It is easy to see that Lis one to one. Indeed, if L{v) = 0 then lv(w) = <», w> = 0
for all w; hence i; = 0. The surprising fact is that L is onto.
Theorem 9. The map L:H -> H* defined by L(v) = lv is bijective.
Remark. We call this theorem a representation theorem because it
"represents" the abstract space H* in terms of the known space H.
To prove theorem 9 we will need a geometric result on Hilbert spaces,
which is interesting in its own right. Let V <= H be a vector subspace of H. If
V is also a closed subset of H (in the norm topology), then it is automatically
a Hilbert space itself; in this case V is called a Hilbert subspace of H. If weH
we write w ± V if <w, »> = 0 for all veV. Let V1 be the set of all weH such
that w J_ K. Notice that K1 is a vector subspace of /f.
Proposition 10, Let V c: H be a Hilbert subspace of the Hilbert space ff.
Assume that V ^ H. Then there exists weV1 such that w # 0.

190
Appendix B
Proof. Because V # H there is an x e H such that x $ V. Let
a = inf ||x — v\\
veV
We claim that there is a y e V such that || x — y || = a. To see this fact choose
vneV such that ||x — vn\\ -»a as n -► oo. Then
||o. - vm\\2 = 2b. - x||2 + 2\\vm - x||2 - 4||i(t,n + oj - x||2
<2||t,n-x||2 + 2||i,m-x||2-4a2
Hence the vn's form a Cauchy sequence in if. Because V is closed in if, the
sequence of i;„'s converges to y e V and
||x-y|| = lim||x-i>B||=a
n->oo
Now let w = x — j;, w # 0, because j/eK and x £ V. We claim that w e K1.
If this claim is not true, then there is a v0 e V such that <w, v0} ^ 0. Let
boll2
y =y+ .... in ^o
Then y'eKand
I foil
<WM>0> ,|2
!|x-/|r = ||x — y — lln 2 v0
,2 , I<W^q>|2 <W,P0) vn , <W,Pq>,
boll2 Kll2^ * °> Kll2
II .,2 , IN^^O/ N^j^O// v \YV9U0/ , v
= l|x,y.|2^l<w?i;o>|2
" '" Kll2
<II*-J>II2
This inequality contradicts the fact that ||x — y|| = a. □
We now prove theorem 9. Let /eH*. We want to find veH such that / = /„.
If / = 0 we can take v = 0, so we will assume / ^ 0 from now on. Let K cz H
be the subspace of if defined by
K = {wetf;/(w) = 0}
Note that K # H because / ^ 0. If {vn} is a Cauchy sequence in K, then vn -> i;
for some veH.By the continuity of /, we see that /(»„) -► l(v) so /(») = 0. Hence
veK. Thus we have seen that K is closed. From proposition 9 there exists
w # 0 in K1. Let

On $£* Matters
191
Z(w)
V = - rrW
Then
l) IMI2
Let x be any element of H. Notice that
(x-j$v) = lix)-l{x) = 0
so
X-M?VeK
ButveK1, so
\\v\\:
0 = ( x - -^2V,v ) = <x,v> - l{x)
That is, l(x) = <x, i?> for all xeH. □
Now let's return to ifp-spaces. In the beginning of this section, we showed
that, if ge£?q with (1/p) + (1/q) = 1, then g defines a continuous linear
functional lg e £fp* given by lg(f) = \fgi[i for / e ££p. This definition gives a map
L :&-+&** byL(g) = lg
This map is one to one because if lg = 0 then, in particular, ^(1^) = J^ g dfx = 0
for all measurable sets A with pt(A) < oo. This implies g = 0 a.e.
In fact, when X is a-finite, it is also true that L is onto; that is, L is an
isomorphism. We won't prove this in full generality; instead we'll prove the
following special case.
Theorem 11. Assume that fi(X) < oo and let 1 < p < 2 and q = p/{p — 1).
Then L\S£q'->£fp* is an isomorphism.
Proof. Let Ze«£fp*. We want to find ge&q such that / = lg. Recall that,
because \i{X) < oo, corollary 6 implies that S£\X,\£) c J?p(X9fi) and that
there is a constant a > 0 such that
ll/llp<a||/ll2 for/Gif2
Now, by proposition 8, there is a constant c> 0 such that
|/(/)l<c|mip torfeX'

192
Appendix B
Thus
|/(/)l<c||/||p<ca||/||2 for/eJS?2
Applying proposition 8 again we see that, if we restrict / to ££2, we get
something in if2*. By theorem 9 there is a g e ££2 such that
■J-
1(f) = ] fid? for/ei?2
We claim that g is actually in if*. To see this, fix K > 0 and set
*»-fi
tO it\g(x)\>K
Then ft(x) is bounded, so heSC2(X,n) and
|/(A)|~ |rf4t£c||fc».
"I
\J\g(x)\^K J
\J\g(x)\£K J
So
(f lffMlf4*Y
This bound holds for all K, so, by the monotone convergence theorem, we
conclude that g(x)e ££q.
Finally, because £f2(X,fj) is dense in ££p(Xy\i) (e.g., the simple functions
are dense in JSfp), it is clear by continuity that
4
1(f) = fgdfi for all / e <£\X, fi D
Remark. It is an easy exercise to extend this result to the case when X is
(7-finite.
Convolution
In the exercises of §3.5, we defined the convolution of / and g when / and g
are in ^(R) by

On &p Matters
193
(11) f*0{x) = j^fix-y)g(y)dy
In those exercises you proved the following proposition.
Proposition 12. f*geif X(R) and
ii/*0iii<imiiii0iii
Proof. Consider the iterated integral
\f(x-y)g(y)\dxdy=W\\dg\\i
JrJr
JRJR
By Fubini's theorem (theorem 15 of §2.5), the integral
1
f{x-y)g(y)dy
R
makes sense for almost all x and is equal a.e. to an &l function. Thus
f*geSCl(R) and a second application of Fubini's theorem gives
\\f*g\\i=\\\f(x-y)g{y)dy\dx
JrJr
JrJr
\f(x - y)g(y)\dydx
\f(x-y)g(y)\dxdy
JRJR
= II/Illll0lll □
Corollary 13. Convolution is a continuous map from if1 x if1 ->• SC1; that
is, if/, -»/in JS?1 and gn-*ginSe1, then/, * gn -> f * g in S£*.
Proof. Choose n large enough so that ||/ — /Ji < 1. Then ||/,Hi<
(l + ll/IIJ-Now
ll/*0-/.*ftJiHI(/-/,)**+ /.*(*-ft,)111
*ll/-/Jlill*lli + ll/.llill0-ftJi
£fl/-/JillffOi+(l + fl/lli)ll0-flJ->O as n^co
a

194
Appendix B
2. (f*g)*h(x) = | |r/(x - z - y)g(y)dyh(z)dz
f(x — s)g(s — z)dsh(z)dz, s = z + y
R
Proposition 14. Let f,g,he &\R). Then
1. f*g = g*f
2. (f*g)*h = f*(g*h)
Proof.
1. f*g(x)=( f(x-y)g{y)dy
= /(s)0(x-s)ds, s = x-y
= g*f(x)
*g)*k{x)sShlt
JrJi
= /(x-s) g(s-z)h{z)dzds
Jr Jr
= f*(9*h)(x) Q
One of the main strengths of the convolution is that it tells us what
corresponds to a product in Fourier transform land.
Proposition 15. (f*g)* = f$ for /, g e JSf^R).
JYw/. (/ * 0) A(£) = f e-ix* f /(x - ,)flf(y) rfj, dx
Jr Jr
= f f ^/(x-j^OOdxdy
JrJr
JrJr
=M)^e-Mg(y)dy
The convolution can be extended to S£p-spaces in various combinations.
For instance, if/ e Sep and g e <£q for (1/p) + (1/q) = 1, then /(x - y)g(y) e S£1

On S£v Matters
195
for each fixed x; so equation 11 makes sense. In fact by Holder's inequality
we have
(12) l/*0<*)|£ 11/11,11*11, for all x
Hence, / * g is a bounded function when fe S£p and g e «£?*, (1/p) + (1/q) = 1.
Proposition 16. Let fe&p, geS£q, and (1/p) + {1/q) = 1. Then f*g is a
bounded, uniformly continuous function on R.
Proof. Let s > 0 be given. We need to find 3 > 0 such that if |x — y\ < 8 then
1/ * g(x) — f * g{y)\ < e. Notice that
\f*g(x)-f*g(y)\ =
I
U(x-z)-f(y-z)Mz)dz
*Ux-f,hM<
where fx(z) = f(x — z). Now let ^ be a compactly supported continuous
function with
ll/-0llP<e
(You can prove that such ^'s exist.) It is easy to see that there is a S > 0 such
that if |x — y\ < S then
Ux-<f>y\\P<e
Then ||/, - fy\\p < \\fx - d>x\\P +HX- <M, + ||/, - 4IIF < 3e □
Another combination for which the convolution can be defined is feJ?p
zndge&1.
Proposition 17. Let / e S£p, 1 < p, and g e J?l. Then / * g is well-defined and
is in S£p. Moreover
ll/^llp<ll/llpll^lli
Proof. Let q = p/(p — 1) and notice that
| \f(x - y)g(y)\dy = | \g(y)\1/p\f(x - y)\ \g(y)\l">dy
Now, by proposition 12, \g(y)\ \ f(x — y)\p is integrable for almost all x, because
\g\ and \f]p are both in <£x. Thus, \g(y)\llp\f{x - y)\ is in <£p for almost all x.
Hence, for almost all x we have by Holder's inequality
[ \f(x - y)g(y)\dy < Q \g(y)\ |/(x - y)\'dy\" \\g\\\f*

196 Appendix B
Thus we have that / * g is defined for almost all x; furthermore, we get
\f*g(x)\p < (| \g(y)\ \f(x - y)|'dy) MP
By Fubini's theorem
U*gVP < (^ \g(y)\\f{x - y)\"dxdy^ MP
< miMi*"
so ii/*0iiP<imipii0iii n
Another important use of the convolution is that it allows us to give
explicit smooth approximations to «£?p functions. We describe this in the
following.
Proposition 18. Let feJ?1 and let ^ be a Schwartz function. Then <f>*f is
C00.
Proof, j^(4*/)(x) = jj J 4(x - y)f(y)dy
= \d^x-y)Mdy
where the differentiation under the integral can be justified by the dominated
convergence theorem. □
Now let <f>0(x) be a smooth function with support in (— 1,1) and such that
<f>0(x)dx = 1
Let <f>k(x) = k<j>0(kx\ fc=l,2,.... Then <l>k(x) is smooth, supported in
(-l/fc,l//c),and
1
<f>k{x)dx= 1
R
Theorem 19. Let/ e S£p. Then <j>k * /converges to / in ifp, as k -► oo.
/V00/. Let 0 be a smooth, compactly supported function with || / — g \\p < e.
Then
11^*/ - /||p < 11^*/ - 4*0llP + 11^*9 - 0II, + \\g - /lip

On &p Matters
197
By proposition 17, \\<f>k*(f - g)\\p < ||/- g\\p because \\<t>k\\x = 1. Thus
\\</>k*f-f\\P<2s+\\<f>k*g-g\\p
Now notice that
\<f>k*g(x) - g(x)\ = lg(* -y)- 0(*)M(y)<ty
because \<l>k(y)dy = 1. Because # is compactly supported, it is uniformly
continuous; that is, given s > 0 there is a 5 > 0 such that if | j/| < 5 then
\g(x — y) — 0(y)| < e. Now ^k is supported in (— 1/fc, 1/k), so, if we take fe large
enough, we can make
\<f>k*g(x) - g(*)\ < e when xesuppg
Furthermore, if g is supported in (a, b), then <j>k * gf is supported in (a — l//c,
b + 1/fe). Thus, for large enough fe, we can conclude
Uk*g-g\\p<e D
Fourier Transform in «Sf P(R)
In §3.5 we defined the Fourier transform
We then used the density of the Schwartz space S in if2, along with
the Plancherel formula, to extend the Fourier transform to an isomorphism of
S£2 onto if2. The same techniques can be used to extend the Fourier
transform to S£p functions for 1 < p < 2. We need the following two results.
Proposition 20. The Schwartz space S is dense in ifp(R), p > 1.
We leave the proof of this proposition to the reader as an exercise. It is
essentially the same as the proof for p = 2 (theorem 15 of §3.5).
The other ingredient we need is an S£v replacement for the Plancherel
formula.
Theorem 21. (Hausdorff-Young inequality) Let 1 < p < 2 and q = p/(p — 1).
Then, for Schwartz functions/
(13) ||/||€ < C||/||p for some constant C
A good reference for the proof of this is the book by Reed and Simon (see
the reference section, page 202).

198
Appendix B
This result tells us that the Fourier transform is uniformly continuous as a
map from the dense set S c i?p(R) into the space &q{R). By theorem 14 of
Appendix A we can then extend it to a map of «£?P(R) into if9(R), and
inequality 13 still holds for all/e&P(R).

Appendix C
A Non-Measurable Subset
of the Interval (0,1]
Let Sl be the set of complex numbers of modulus one (complex
numbers of the form, c = a + ib, a, b e R, with a2 + b2 = 1.) This set forms a group
under complex multiplication: If c\, c-i e S{ then c\c2 and c\/ci e S1. As a set
one can identify Sl with the interval, / = (0,1], by means of the map
(1) f:I—>S\ f(t) = e27rit.
(Notice that this map is both one-one and onto.) We will use the identification (1)
to transport the Lebesgue outer measure, /x*, on subsets of / to subsets of Sl; in
other words if A is a subset of Sl we will define /x*(A) to be fA*(f~l (A)). (Thus,
to show that there exists a non-measurable subset, U\ of / it will suffice to show
that there exists a non-measurable subset, £/, of Sl and then take V = /-1 (£/)•)
Let QCM be the set of rational numbers in the interval, /, and let SlQ = f(QCM).
We claim
Lemma 1. Sq is a subgroup ofS1. In other words, ifc\ and c-i are in SlQ so are
c\C2 and c\/C2.
Proof. By assumption cx = e2nitx and c2 = e2nitl with tu t2 6 Q O /. Choose Ji
and S2 in QHI so that (t\ +1\) — s\ and (t\ —12) — S2 are integers. It's clear that
Clc2 = e2ltiSi and c{/c2 = e2***2.
D
Given a subset, A, of 51 and co G Sl let coA = {coaya e A}. We leave the
following as an exercise
199

200
Appendix C
Lemma 2. For all subsets, A.ofS1 and all co e Sl
(2) tf(fi>A) = tf(A).
Hint: §1.3 exercise 14.
Since the rational numbers on the interval, /, are a countable set, so are the
points of Sq. Let
{a>i,i = 1,2,...; a* e SlQ]
be a sequence to which every element of Sq belongs. Without loss of generality
we can assume that o>,- ^ coj for i ^ j. We will say that a subset, B, of Sx has the
coset property (with respect to SXQ) if the sets
Bj =(OjBt j = 1,2,...,
are mutually disjoint: Bj O Bt = 0 if j' ± k. It is clear that sets with this property
exist. In fact let c be any point in Sl and let B be the one element set consisting
of c. Then /?,- H Bj ^<j> implies ccoi = ca>j or, dividing by c, on = a>7, i.e. i = 7.
Let's pick a set, 5wflJO with the coset property which is as "large as possible,"
i.e. which is so large that if we add one more point to it, it no longer has the coset
property. l We claim that one of the sets
(3) Bk=cokBmaX9 * = 1,2,...,
is non-measurable. Indeed, by assumption, these sets are mutually disjoint; and
the fact that Bmax is "as large as possible" implies that
(4) \JBk = S\
(Proof: if this were not the case, there would exist a c 6 Sl not in the union of the
fljk's; and by adding c to Bmax we would get a larger set with the coset property.)
Suppose now that the Z?*'s are all measurable. Then, since the union (4) is disjoint:
(5) EM*(ft) = M*(S,) = M*(/) = l.
But, by (2),
(6) M*(ft) = M*(*m«x).
Our naive intuition tells us that such a set has to exist; however, the existence of such a set involves
some delicate issues in set theory (such as the axiom of choice and Zorn's lemma). Suffice it to say
that the standard axioms of set theory permit us to assert that such a set exists, confirming our naive
intuition.

A Non-Measurable Subset of the Interval (0,1 J
201
so the left hand side of (5) is either zero or infinity depending on whether fi*(Bmax)
is zero or greater than zero. This argument by contradiction proves that one of the
Bk's has to be non-measurable.
Exercise. In fact show that all the Bkys are non-measurable and that Bmax is also
non-measurable.
Hint: See the exercise in §1.3 cited in the previous hint.

References
P. Billingsley, Probability and Measure. New York: Wiley, 1979.
W. Feller, An Introduction to Probability Theory and Its Applications, 3d ed. New York: Wiley,
1967.
U. Grenander and G. Szego, Toeplitz Forms and Their Applications. Berkeley, Calif.: University
of California Press, 1958.
K. Hoffman, Analysis in Euclidean Space. Englewood Cliffs, N.J.: Prentice-Hall, 1975.
M. Kac, Statistical Independence in Probability Analysis and Number Theory. Mathematical
Association of America (Cams Mathematical Monograph, no. 12) 1959.
S. Lang, A Complete Course in Calculus. Reading, Mass.: Addison-Wesley, 1968.
M. Reed and B. Simon, Methods of Modern Mathematical Physics: Functional Analysis, vol. 1,
revised and enlarged edition. New York: Academic Press, 1980.
W. Rudin, Principles of Mathematical Analysis, 3rded. New York: McGraw-Hill, 1976.
202

Index
almost always (a.a.), 50
almost everywhere (a.e.), 66
approximate outer measure, 29
Banach space, 122, 187
Bernoulli sequence, 1,2, 43-44
Bernstein polynomial, 114
Bessel's inequality, 133
binary expansion, 2
Bolzano-Weierstrass property, 181
Borel-Cantelli lemmas, 45, 47
Borel measurable, 58
Borel principle, 3-4
Borel sets, 37
extended, 54-55
Cauchy sequence, 179
central limit theorem, 170-177
characteristic function
of a random variable, 155
of a set, 60
Chebyshev's inequality, 6, 9, 65-66
complete metric space, 122, 179
complete orthonormal basis, 130, 131
conditional probability, 46
convergence
in^1, 121
in #2, 124
in metric spaces, 179
in 2X, 32
weak, 165
convex function, 184-185
convolution, 145, 154, 192-197
dimension, 131
discrete Dirichlet problem, 19-20,
115-117
discrete probability theory, 43
dual space, 189
expectation value, 16-17, 70, 102,
103
extended Borel sets, 54-55
extended real numbers, 53
Fatou's lemma, 78
field, 42
Fourier coefficient, 131
Fourier series, 131
Fourier transform, 137-145
Fourier transform in £p, 197-198
Fubini's theorem
version 1,95-96
version 2, 98-99
version 3, 99
gambler's ruin, 15
Gaussian measure, 68-69, 173
Gram-Schmidt process, 136
Haar functions, 136-137
Hausdorff-Young inequality, 197-198
203

204
Index
Heine-Borel theorem, 180-181
Hermite polynomials, 154
Hilbert space, 127
Holder's inequality, 186
identically distributed, 104
independent events, 46
independent random variables, 104
independent sets, 46-47
infinitely often (i.o.), 45
inner product, 125
integrable, 76
integral
of integrable functions, 76
of nonnegative measurable
functions, 61-63
of nonnegative simple functions,
60-62
inverse Fourier transform, 148
joint probability distribution, 104
X-system, 91-93
law of large numbers, 1-2, 44
strong, 9-11
weak, 6-9,113
2Oj0, 2(|i, E), 76
2\ 118-124
£2, 124-129
/2, 133-134
£p, 124, 184-188
Lebesgue dominated convergence
theorem, 78-80
Lebesgue measure, 3, 36-39
Lebesgue measure zero, 8-9
Legendre polynomials, 136
lim inf, 50, 56-57
lim sup, 45, 56-57
linear functional, 189
marbles, 21-22, 105-106
Markov process, 23
measurable function, 53-58
measurable set, 34
measure, 27
space, 53
theoretic model, 42-43
mesh width, 83
metric space, 178-182
Minkowski's inequality, 186
monotone convergence theorem,
72-74
multi-interval, 25
Parseval's identity, 154
partition, 83-84
piecewise constant, 6-8
piecewise differentiable, 140
TT-X theorem, 92-94
ir-system, 92
Plancherel formula, 132, 150
probability distribution, 103
probability measure, 42
product measure, 94, 96
product set, 89
Rademacher functions, 4-6
random variable, 16, 55, 102
random walk, 17
generalized, 156
with pauses, 17-18
in the plane, 18-19
refinement, 83
representation theorems, 188-192
Riemann integrable, 84
Riemann integral, 83-88
Riemann sums (upper and lower), 83
ring, 25
run lengths, 46
sample space, 17
Schwartz function, 146
Schwarz's inequality, 126
set function, 25
additive, 25-26
countably additive, 26-27
Shakespeare, 15, 49
<r-field, 42
a-finite, 39
a-ring, 35, 42
simple function, 60
slice, 89
spectral measure, 165
square-integrable functions, 124
symmetric difference, 25
Szego theorem, 165

Index
205
ternary expansion, 12, 17-18
Toeplitz matrix, 165
triangle inequality, 126
unfair coin, 108, 163-164
uniform continuity, 150, 182
variance, 70, 172
vector space norm, 120
Weierstrass approximation theorem,
114, 143-144
winning streaks, 16