/
Текст
Probability and its Applications A Series of the Applied Probability Trust Editors: J. Gani, C.C. Heyde, T.G. Kurtz Springer New York Berlin Heidelberg Barcelona Hong Kong London Milan Paris Singapore Tokyo
Probability and its Applications Anderson: Continuous-Time Markov Chains. Azencott/Dacunha-Castelle: Series of Irregular Observations. Bass: Diffusions and Elliptic Operators. Bass: Probabilistic Techniques in Analysis. Choi: ARMA Model Identification. de la Pena/Gine: Decoupling: From Dependence to Independence. Galambos/Simonelli: Bonferroni-type Inequalities with Applications. Gani (Editor): The Craft of Probabilistic Modelling. Grandell: Aspects of Risk Theory. Gut: Stopped Random Walks. Guyon: Random Fields on a Network. Kallenberg: Foundations of Modern Probability. Last/Brandt: Marked Point Processes on the Real Line. Leadbetter/Lindgren/Rootzen: Extremes and Related Properties of Random Sequences and Processes. Nualart: The Malliavin Calculus and Related Topics. Rachev/Ruschendorf: Mass Transportation Problems. Volume I: Theory. Rachev/Ruschendorf: Mass Transportation Problems. Volume II: Applications. Resnick: Extreme Values, Regular Variation and Point Processes. Shedler: Regeneration and Networks of Queues. Thorisson: Coupling, Stationarity, and Regeneration. Todorovic: An Introduction to Stochastic Processes and Their Applications.
Hermann Thorisson Coupling, Stationarity, and Regeneration With 27 Illustrations Springer
Hermann Thorisson Science Institute University of Iceland Dunhaga 3 107 Reykjavik Iceland E-mail: hermann@hi.is Homepage: www.hi.is/~hermann Series Editors J. Gani Stochastic Analysis Group, CMA Australian National University Canberra ACT 0200 Australia C.C. Heyde Stochastic Analysis Group, CMA Australian National University Canberra ACT 0200 Australia T.G. Kurtz Department of Mathematics University of Wisconsin 480 Lincoln Drive Madison, WI 53706 USA Mathematics Subject Classification (1991): 60Gxx, 60Jxx, 60Kxx, 60D05, 60B15, 60A10, 60F99 Library of Congress Cataloging-in-Publication Data Thorisson, Hermann. Coupling, stationarity, and regeneration / Hermann Thorisson. p. cm. — (Probability and its applications) Includes bibliographical references and index. ISBN 0-387-98779-7 (hardcover : alk. paper) 1. Random variables. 2. Stochastic processes. I. Title. II. Series. QA273.T4395 2000 519.2—dc21 99-40961 Printed on acid-free paper. © 2000 Springer-Verlag New York, Inc. All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer-Verlag New York, Inc., 175 Fifth Avenue, New York, NY 10010, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use of general descriptive names, trade names, trademarks, etc., in this publication, even if the former are not especially identified, is not to be taken as a sign that such names, as understood by the Trade Marks and Merchandise Marks Act, may accordingly be used freely by anyone. Production managed by Jenny Wolkowicki; manufacturing supervised by Jeffrey Taub. Typeset by Thorir Magnusson using the author's TgX files. Printed and bound by Maple-Vail Book Manufacturing Group, York, PA. Printed in the United States of America. 987654321 ISBN 0-387-98779-7 Springer-Verlag New York Berlin Heidelberg SPIN 10660218
TileinkaS per Rannveig
Solarhjort leit ek sunnan fara, hann teymdu tveir saman; faetr hans stoSu foldu &, en toku horn til himins. Ur Solarljodum
Preface This is a book on coupling, including self-contained treatments of station- arity and regeneration. Coupling is the central topic in the first half of the book, and then enters as a tool in the latter half. The ten chapters are grouped into four parts as follows: Chapters 1-2 form an introductory part presenting basic elementary couplings (Chapter 1 on random variables) and the classical triumphs of the coupling method (Chapter 2 on Markov chains, random walks, and renewal theory). Chapters 3-7 present a general coupling theory highlighting maximal couplings and convergence characterizations for random elements, stochastic processes, random fields, and random elements under the action of a transformation semigroup. Chapters 8-9 present Palm theory of stationary stochastic processes associated with a simple point process. Chapter 8 treats the one- dimensional case and Chapter 9 the higher-dimensional case. Chapter 10 deals with regeneration, both classical regenerative processes and three generalizations: wide-sense regeneration (as in Harris chains); time-inhomogeneous regeneration (as in time-inhomogeneous recurrent Markov chains); and taboo regeneration (as in transient Markov chains). It ends with a section on perfect simulation (coupling from-the-past). This enormous chapter is thrice the size of a normal chapter, and is really a book within the book. For more information on the content of the book, see the introductions to the chapters. Also, the table of contents provides a structural review.
viii Preface The book should be of interest to students and researchers in probability, stochastic modelling, and mathematical statistics. It is written with a Ph.D. student in mind, and the first two chapters can be read at the master's level and even at an advanced undergraduate level. The book is mathematically self-contained, relying only on basic measure- theoretic probability. Measure-theoretic language is suppressed in the first two chapters, and then enters heavily in Chapter 3 to be used explicitly for the rest of the book; Ash (1972) is used as reference, but Billingsley (1986) is also fine. Some prior knowledge of elementary Markov chain theory would be useful, at least in Chapter 2; Karlin and Taylor (1975) and Cinlar (1975) are excellent, and the compact first two sections of the first two chapters in Asmussen (1987). Some Conventions In order to make clear what results belong to the measure-theoretic background, the term 'Fact' is used for results stated without proof, while the terms 'Theorem' and 'Lemma' are reserved for results that are proved here. Facts of basic importance throughout the book are restricted to Chapter 3 (Sections 3 and 4). Sections are enumerated within chapters. For instance, the 4th section in the 3rd chapter is referred to within the chapter only as 'Section 4'; but in the other chapters as 'Section 4 in Chapter 3' or 'Chapter 3 (Section 4)'. Subsections are enumerated within chapters and sections: the 5th subsection of the 4th section in the 3rd chapter is referred to within the chapter as 'Section 4.5'; but in the other chapters as 'Section 4.5 in Chapter 3'. The same goes for Theorems, Lemmas, Facts, Remarks, and Figures. Definitions are stated in the text, and only indicated by writing the concept being defined in italics (we also use italics for emphasis). Figures are placed in the text precisely where they should be consulted (mostly), but the text does not rely on them. We use both parenthesis () and brackets [] for comments that can be skipped. Historical and bibliographic notes are deferred to a separate section at the end of the book. The symbol X (and X', X, X\, ...) is reserved for real-valued random variables. The symbol Sk always denotes a sum, Sk = Sq+ Xx + 1- -X*. On the other hand, S is either a sequence of Sk (one-sided sequence in Chapters 2, 3, and 10; two-sided in Chapter 8), or a d-dimensional random vector (Chapters 7 and 9). The symbol U is reserved for a variable uniform on [0,1]. The symbol Z is reserved for processes. The symbol Yoften denotes a random element in a general space; and P(Y € •) is the distribution of Y. Errors are bound to abound in the book, in spite of all the thinning attempts. For errata, and even some notes and references, see my homepage (www. hi. is/~hermann) or Springer's (www. springer-ny. com). If you find an error or have a comment, please send me an informal note by e-mail (To: hermann@hi.is; Subject: book).
Preface ix Acknowledgements It took four long years to write this book, word for word, from the first word on the first page to the last word on page four-hundred-seventy-eight. Previously, the book had been in preparation for five years; and before that, subconsciously for years. I would like to thank Torgny Lindvall for introducing me to this field of study and for grooming me for this task, and Peter Jagers for his influence and for the focused comments on the book. Also thanks to S0ren Asmussen, Karl Sigman, Peter Glynn, Serguei Foss, and Richard Gill, who have influenced this work in various ways throughout the years. Special thanks to Olle Nerman, Olav Kallenberg, David Blackwell, Henry Berbee, Jakob Yngvason, Peter Donnelly, and Olle Haggstrom for illuminating observations; to Rolando Rebolledo, who got me started on this project by inviting me to Chile to lecture in the fall of 1991; to Vladimir Kalashnikov for the collaboration on the Petrozavodsk proceedings, which helped get things into perspective; to Christian Meise for reading and rereading the book, and for the thicket of detailed comments; to Diemer Salome for comments on the first four chapters; to Damien White, Remco van der Hofstad, Erik van Zwet, Karma Dajani, Ronald Meester, and Adam Shwartz for comments on the first two; to Vladimir Bogachev and Andrew Nobel for comments on the third; to David Svensson for comments on the eighth; to my next-door colleague Magnus Halld6rsson for reading and commenting on parts of the book, and for the many discussions; and to my fellow probabilist Ott6 Bjornsson for his long-lasting interest and support. Also thanks to the copyeditor David Kramer for excellent suggestions while basically accepting my Icelandic English, and for all the commas; to my colleague Robert Magnus for straightening out many unclear sentences, and for some British moderation; and to the production editor Jenny Wolkow- icki for the insightful finishing touch. After all this feedback the book should be in pretty good shape; but whatever mistakes there are, they are all mine. I would like to thank John Kimmel, Springer's statistics editor, for his deep understanding of what a work like this is about, and for never rushing me in spite of the writing going almost two years beyond deadline and the book becoming more than twice the planned size. Many thanks to Porir Magnusson for his expert LaTeX-ing, and for his support and the patience it must have taken to work along with me these four years. Also thanks to my son Freyr (Hermannsson) for calmly FreeHand-ing the figures during these hectic last weeks. I am grateful to the University of Iceland and its Science Institute for funding this project, and for providing the freedom necessary for this task. I am also grateful to the Icelandic Science Foundation for travel grants. And now, as this book goes to print, I am informed that I have been awarded the Olafur Danielsson Prize in Mathematics for my research, most ofwhich can be found in some form in this book. I am deeply moved and will use the generous sum to recover from this work, and prepare for the next.
x Preface Music, - from Johann Sebastian Bach to Par Lindh Project (PLP), - played an important role in getting me through the composition of this book. So, on another note, thank you Keith Emerson for the High Level Fugue and the Endless Enigma; evermoving without ever moving. Also thanks to Ian (Thick as a Brick) Anderson for Twelve Dances with God, and to King Crimson for both Discipline and Indiscipline, and for completing this trinity of sons IN THE COURT OF THE CRIMSON KING. Finally, Rannveig, Freyr, and Nanna, thank you for so lovingly supporting me through these Ten Dances with Chance. Reykjavik October 1999 Hermann Porisson
Contents Preface vii 1 RANDOM VARIABLES 1 1 Introduction 1 2 The i.i.d. Coupling - Positive Correlation 2 3 Quantile Coupling - Stochastic Domination 3 4 Coupling Event - Maximal Coupling 6 5 Poisson Approximation - Total Variation 11 6 Convergence of Discrete Random Variables 15 7 Continuous Variables - Hitting the Limit 18 8 Convergence in Distribution and Pointwise 21 9 Quantile Coupling - Dominated Convergence 26 10 Impossible Coupling - Quantum Physics 27 2 MARKOV CHAINS AND RANDOM WALKS 33 1 Introduction ' 33 2 Classical Coupling - Birth and Death Processes 33 3 Classical Coupling - Recurrent Markov Chains 38 4 Classical Coupling - Rates and Uniformity 44 5 Ornstein Coupling - Random Walk on the Integers 47 6 Ornstein Coupling - Recurrent Markov Chains 52 7 Epsilon-Coupling - Nonlattice Random Walk 57 8 Epsilon-Coupling - Blackwell's Renewal Theorem 62 9 Renewal Processes - Stationarity 68 10 Renewal Processes - Asymptotic Stationarity 72
xii Contents 3 RANDOM ELEMENTS 77 1 Introduction 77 2 Back to Basics - Definition of Coupling 78 3 Extension Techniques 80 4 Conditioning - Transfer 86 5 Splitting 93 6 Random Walk with Spread-Out Step-Lengths 98 7 Coupling Event - Maximal Coupling 104 8 Maximal Coupling Two Elements - Total Variation 107 9 Hitting the Limit 113 10 Convergence in Distribution and Pointwise 117 4 STOCHASTIC PROCESSES 125 1 Introduction 125 2 Preliminaries - What Is a Stochastic Process? 126 3 Exact Coupling - Distributional Exact Coupling 136 4 Distributional Coupling 140 5 Exact Coupling - Inequality and Asymptotics 142 6 Exact Coupling - Maximality 146 7 Coupling with Respect to a Sub-cr-Algebra 149 8 Exact Coupling - Another Proof of Theorem 6.1 153 9 Exact Coupling - Tail cr-Algebra - Equivalences 156 5 SHIFT-COUPLING 161 1 Introduction 161 2 Shift-Coupling - Distributional Shift-Coupling 162 3 Shift-Coupling - Inequality and Asymptotics 165 4 Shift-Coupling - Maximality 169 5 Shift-Coupling - Invariant cr-Algebra - Equivalences .... 174 6 e-Coupling - Distributional e-Coupling 178 7 e-Coupling - Inequality and Asymptotics 180 8 e-Coupling - Maximality 187 9 e-Coupling - Smooth Tail cr-algebra - Equivalences 188 6 MARKOV PROCESSES 195 1 Introduction 195 2 Mixing and Triviality of a Stochastic Process 195 3 Markov Processes - Preliminaries 201 4 Exact Coupling 204 5 Shift-Coupling 207 6 Epsilon-Coupling 209 7 Stationary Measure 213 7 TRANSFORMATION COUPLING 217 1 Introduction 217
Contents xiii 2 Shift-Coupling Random Fields 218 3 Transformation Coupling 222 4 Inequality and Asymptotics 225 5 Maximality 228 6 Invariant a-Algebra and Equivalences 231 7 Topological Transformation Groups 238 8 Self-Similarity - Exchangeability - Rotation 241 9 Exact Transformation Coupling 244 8 STATIONARITY, THE PALM DUALITIES 249 1 Introduction 249 2 Preliminaries - Measure-Free Part of the Dualities 251 3 Key Stationarity Theorem 254 4 The Point-at-Zero Duality 258 5 Interpretation - Point-Conditioning 264 6 Application - Perfect Simulation 271 7 The Invariant cr-Algebras I and J 281 8 The Randomized-Origin Duality 284 9 Interpretation - Cesaro Limits and Shift-Coupling 288 10 Comments on the Two Palm Dualities 290 9 THE PALM DUALITIES IN HIGHER DIMENSIONS 295 1 Introduction 295 2 The Point-Stationarity Problem 296 3 Definition of Point-Stationarity 302 4 Palm Characterization of Point-Stationarity 308 5 Point-Stationarity Characterized by Randomization 317 6 Point-Stationarity and the Invariant a-Algebras 320 7 The Point-at-Zero Duality 323 8 The Randomized-Origin Duality 328 9 Comments 335 10 REGENERATION 337 1 Introduction 337 2 Preliminaries - Stationarity 339 3 Classical Regeneration ' 346 4 Wide-Sense Regeneration - Harris Chains - GI/GI/k .... 358 5 Time-Inhomogeneous Regeneration 372 6 Classical Coupling 385 7 The Coupling Time - Rates and Uniformity 399 8 Asymptotics From-the-Past 422 9 Taboo Regeneration 436 10 Taboo Stationarity 451 11 Perfect Simulation - Coupling From-the-Past 467
xiv Contents Notes 479 References 491 Index 509 Notation 517
Chapter 1 RANDOM VARIABLES 1 Introduction Coupling means the joint construction of two or more random variables (or processes), usually in order to deduce properties of the individual variables or gain insight into distributional similarities or relations between them. In this chapter and the next the method is introduced through a series of basic elementary examples. The arguments are carried out in full detail at an undergraduate level, suppressing measure-theoretic language. Advanced readers should be able to fill in any missing measure-theoretic notation or find it at the beginning of Chapter 3, where we return to the definition of coupling. Let us spend a few lines on terminology before turning to the examples. A copy or a representation of a random variable X is a random variable X with the same distribution as X. Denote this by X^X. A coupling of a collection of random variables Xi, i € I, [where I is some index set] is a family of random variables (Xi : i € I) such that Xi = xu i e i. Note that only the individual Xt are copies of the individual Xi, while the whole family (X, : i 6 I) is typically not a copy of the family (Xi : i 6 I). In other words, the joint distribution of the Xt need not be the same as that of 1
2 Chapter 1. RANDOM VARIABLES the Xi. In fact, the Xi need not even have a (specified) joint distribution. On the other hand, we write (Xi : i 6 I) in parentheses to stress that the Xi have a joint distribution. A trivial but often useful coupling is the independence coupling consisting of independent copies of the Xi. Thus a coupling has fixed marginal distributions (the distributions of the individual Xi), and the trick is to find a dependence structure (joint distribution) that fits one's purposes. 2 The i.i.d. Coupling - Positive Correlation A self-coupling of a random variable X is a family (Xt : i £ I) where each Xi is a copy of X. A trivial (and not so useful) self-coupling is the one with all the Xi identical. Another trivial self-coupling is the i.i.d, coupling consisting of independent copies of X. As an example of an efficient use of the i.i.d. coupling we shall prove the following result. For every random variable X and nondecreasing bounded functions / and g, the random variables f(X) and g(X) are positively correlated, that is, Cov[f(X),g(X)]^0. (2.1) In order to prove this claim let X' be an independent copy of X [thus (X, X') is an i.i.d. coupling of X]. The additivity of covariances yields Cov[/(X) - f(X'),g(X) - g(X')} = Cov[f(X),g(X)} - Cov[f(X),g(X')] - Cov[f(X'),g(X)} + Cov[f(X'),g(X% Since X and X' are independent, the middle terms on the right are zero, and since X and X' have the same distribution, the remaining terms on the right are identical. Thus Cov[f(X),g(X)] = i Cov[/(X) - f(X'),g(X) - g(X')}. Since the mean of both f(X) - f(X') and g{X) - g{X') is zero, we have Cov[f(X)-f(X'),g(X)-g(X')} = E[(f(X)-f(X'))(g(X)-g(X'))}, which is positive, since / and g nondecreasing implies that f(x) — f(y) and g(x) — g(y) are either both ^ 0 or both ^ 0. Thus (2.1) holds.
Section 3. Quantile Coupling - Stochastic Domination 3 3 Quantile Coupling — Stochastic Domination In this section we produce a coupling that turns so-called stochastic domination into ordinary (pointwise) domination. Another application can be found in Section 8. See also Section 9. 3.1 The Coupling Consider a random variable X with distribution function F, that is, P{X ^ x) = F(x), i£l Let F"1 be the generalized inverse of F (or quantile function) defined by F-1{u) = ini{xeR:F{x)^u}, u e [0,1]. Note that if F is continuous and strictly increasing, then F_1 is the ordinary inverse of F (see Figure 3.1). FIGURE 3.1. The generalized inverse F_1. Let U be uniform on [0,1] (this is short for saying that U is a random variable that is uniformly distributed on [0,1]). Then the random variable X = F~1(U) is a copy of X, since [note that F-1(u) ^ x if and only if u ^ F(x)] P{X < i) = P(F_1 {U) ^x) = P(U ^ F{x)) = F(x), xeR. Thus letting F run over the class of all distribution functions (using the same U) yields a coupling of all differently distributed random variables. Call it the quantile coupling. Since F_1 is nondecreasing, we have, according to Section 2, that the quantile coupling consists of positively correlated random variables. We might even think of this coupling as a maximal dependence coupling because knowing the value of only one of its variables, namely the value of U itself, gives us the value of all the others.
4 Chapter 1. RANDOM VARIABLES 3.2 Application — Stochastic Domination Let X and X' be two random variables with distribution functions F and G, respectively. If there is a coupling (X, X') of X and X' such that X is pointwise dominated by X', that is, X^X', then {X ^ x} D {X' ^ x}, which implies P(X ^ x) ^ P(X' ^ x) and thus F(x) ^ G(x), x e M. (3-1) If (3.1) holds, then X is said to be stochastically dominated (or dominated in distribution) by X'. Denote this by D X ^ X'. We shall now show that the quantile coupling turns stochastic domination back into pointwise domination: due to (3.1), G(x) ^ u implies F(x) ^ u and thus {x e R : G{x) )ti}C{iel: F(x) ^ u} and thus F~\u) ^ G~\u), which yields F~\U) ^ G~\U) [see Figure 3.2]. FIGURE 3.2. Turning stochastic domination into pointwise domination. We have established the following result. Theorem 3.1. Let X and X' be random variables. Then D X ^X' if and only if there is a coupling (X,X') of X and X' such that X <X'.
Section 3. Quantile Coupling - Stochastic Domination 5 3.3 What For? The direct usefulness of Theorem 3.1 is mainly due to the fact that it is easier to carry out arguments using pointwise domination than stochastic domination. As an illustration of this we shall prove the following result. Corollary 3.1. Let Xi,X2,X[, and X2 be random variables such that X\ and X2 are independent, X[ and X'2 are independent, Xx J X[ and X2 J X2. Then X1+X2^X[+X2. (3.2) Proof. Let (Xi,X[) be a coupling of Xi and X[ such that XY ^ X[ and let {X2,X^} be a coupling of X2 and X2 such that X2 f X2. Let {XUX[) and (X2,X2) be independent. Then (Xi + X2,X{ + X2) is a coupling of Xi + X2 and X[ + X2, and Xl+X2^X[+X'2. This implies (3.2). □ A more substantial example of obtaining a distributional result through a pointwise argument by way of coupling can be found in Section 9, where we use the quantile coupling to obtain the distributional version of dominated convergence from the standard pointwise version. 3.4 On the General Coupling Idea Theorem 3.1 and Corollary 3.1, illustrate two general points about coupling. Firstly, a coupling characterization of a distributional property deepens our understanding of that property: according to Theorem 3.1, stochastic domination is simply the distributional form of pointwise domination. Secondly, the coupling characterization can also be directly useful because it is easier to argue pointwise (as in the proof of Corollary 3.1) than in distribution. 3.5 Variations on the Quantile Coupling Let U be uniform on [0,1] and define X = F-\U), X' = G~1(1-U). Then (X, X') is still a coupling of X and X', since 1 — U is also uniform on [0,1]. Now G_1(l — u) is nonincreasing in u, and an obvious modification
6 Chapter 1. RANDOM VARIABLES of the last three lines in Section 2 yields that X and X' are negatively correlated. More generally, if we put X = F~l{(a + bU) mod 1), X' = G'1 ((c + dU) mod 1), where a,c € M and b, d = ±1, and x mod 1 means the fractional part of x, x mod 1 = x — [x], then (X,X') is a coupling of X and X'. Here we could even allow a,b,c, and d to be random variables that are independent of U. If we take G = F, these modifications of the quantile coupling yield a nontrivial self-coupling of X. The quantile approach is used heavily in simulation to generate random variables with specified distributions. 3.6 Comment D „ „ Suppose X ^ X' and apply Theorem 3.1 to obtain a coupling (X, X') such that X ^ X'. Then for each bounded nondecreasing function g we have g(X) ^ g(X') and thus E[g(X)} < E[P(X')]. (3.3) Conversely, suppose (3.3) holds for all bounded nondecreasing functions g. Fix anigl and take g = l^oo) to obtain from (3.3) that P(X >x)^ P{X' > x), x € BL Thus X ^ X' if and only if (3.3) holds for all bounded nondecreasing g. This is taken as the definition of stochastic domination in higher dimensions and, more generally, in partially ordered spaces. Theorem 3.1 can in fact be extended to partially ordered Polish spaces; cf. Lindvall (1992), Chapter IV.1 (Strassen's theorem). 4 Coupling Event - Maximal Coupling Let Xi, i € I, be a collection of discrete or continuous random variables. We shall construct a coupling such that the variables coincide maximally. We first treat the discrete case and start by establishing an upper bound on the coincidence probability. In Section 5 we give an application of this coupling.
Section 4. Coupling Event - Maximal Coupling 7 4.1 The Coupling Event Inequality — Discrete Variables Suppose (Xi : i £ I) is a coupling of X^, i £ I, and let C be an event such that if C occurs, then all the Xi coincide, that is, C C {Xi = Xj for all j, j £ I}. Call such an event a coupling event. Consider first the discrete case: let all the Xi take values in a finite or countable set E and denote the probability mass functions by pi, that is, for x £ E, P(Xi=x)=Pi(x). For all i,j £ I and x £ E we have P(Xi =x,C) = P(Xj = x,C)^ Pj{x) and thus for all i £ I and x £ E P{Xi =x,C) ^infp^x). Summing over x £ E yields the following basic coupling event inequality. Theorem 4.1. If C is a coupling event of a coupling of discrete random variables Xi, i £ I, taking values in a finite or countable set E, then P(C) <Y,infPi(x). (4.1) 4.2 Maximal Coupling — Discrete Variables We shall now construct a coupling with a coupling event C such that (4.1) holds with identity. Call such a coupling maximal and C a maximal coupling event. Put c := 2_, inf Pj(x) (the maximal coupling probability). xeE'€ If c = 0, take the Xi independent and C = 0. If c = 1, take the Xi identical and C = 0 = the set of all outcomes. If 0 < c < 1, let us mix these couplings as follows. Let /, V, and Wi, i £ I, be independent random variables such that / is 0-1 valued with P(I = 1) = c, P(y = x) =mipi(x)/c, x £ E. P(Wi =x) = (pi{x) - cP{V = x))/(l -c), x£ E.
8 Chapter 1. RANDOM VARIABLES Define, for each i £ I, [Wi if 7 = 0. K J Then P(Xt =x)= P(V = x)P(I = 1) + P(Wi = x)P(7 = 0) = P(Xi = a:). Moreover, C= {1= 1} is a coupling event and P(C) has the desired value c. We have established the following result. Theorem 4.2. Suppose Xi, i £ I, are discrete random variables taking values in a finite or countable set E. Then there exists a maximal coupling, that is, a coupling with coupling event C such that P(C) = £ MPi(x). »€I x€E 4.3 The Coupling Event Inequality — Continuous Variables Now let the X{ be continuous random variables with densities /,, that is, for intervals A P{Xi £ A) = / fi (which is short for / fi(x)dx). J A J A It is a little harder to establish the coupling event inequality in this case, and we shall make the simplifying assumption that the X, are either finitely or count ably many, that is, I = {l,...,n} or I={1,2,...}. Suppose (Xi : » £ I) is a coupling of Xi, i £ I, and C is a coupling event. Then, for intervals A and i,j £ I, P(Xi £A,C) = P(Xj £A,C)^ [ fj. (4.3) J A Consider first the finite case I = {1,..., n} and define a partition of E by A1 ={x£ E: /!(x) = inf fj(x)} and recursively for 1 < k ^ n Ak = {x £ E : fk(x) = inf fj(x)}\(A1U---UAk-1).
Section 4. Coupling Event - Maximal Coupling 9 Then (4.3) yields the inequality in P(Xi GAnAk,C)^ [ fj= [ inf fk, (4.4) JAr\Ak JAnAk Kj<n while the equality follows from the definition of Ak- Sum over A; £ I to obtain, in the finite case, that P(Xi g A,C)^ /inf/j, iGl. (4.5) Ja^1 In the countable case I = {1,2,...} fix n < oo to obtain that (4.4) still holds for i,k ^ n. This yields (4.5) with infjgi fj replaced by infi^j^n fj. Sending n —> oo yields (4.5), since infi^-^n fj decreases to inf j€i fj. Take A = E in (4.5) to obtain the following coupling event inequality. Theorem 4.3. If C is a coupling event of a coupling of continuous random variables with densities f\, fi,... (or /i,..., fn), then P(C)^Jmifi- (4-6) 4.4 Maximal Coupling — Continuous Variables Call a coupling and event achieving identity in (4.6) maximal. The construction in Section 4.2 extends with an obvious modification to the continuous case. Put /inf J iGI fi (the maximal coupling probability). If c = 0, take the Xt independent and C = 0. If c = 1, take the Xt identical and C = 0. If 0 < c < 1, mix these couplings as follows. Let /, V, and Wi, i £ I, be independent random variables such that / is 0-1 valued with P(I = 1) = c, V has density inf file, i€I Wi has density (fi - inf /j)/(l - c). JGI Define Xi by (4.2). Then (Xj : i 6 I) is a coupling of the Xi, since for intervals A, P{Xi £i) = P{V £ A)P(7 = 1) + P(Wi £ A)P(7 = 0) = P(Xi e A). Moreover, C = {I = 1} is a coupling event, and P(C) has the desired value. We have established the following result.
10 Chapter 1. RANDOM VARIABLES Theorem 4.4. Suppose X\,X2, ■ ■ ■ (or X\,..., Xn) are continuous random variables with densities /i, /2, • • • (or f\,..., /„). Then there exists a maximal coupling, that is, a coupling with coupling event C such that \C) = Jmifi. 4.5 Comments It is often natural to take C={Xi = Xj for alH,./el}. By definition, any coupling event of (Xt : i £ I) is contained in this set, and thus the maximal couplings in Theorems 4.2 and 4.4 are also maximal with this choice of C. In particular, for two discrete random variables X and X' there exists a coupling (X,X') such that, with A denoting minimum, p(i = i') = ^p(i = i)AP(r = i) (4.7) X and for two continuous random variables X and X' with densities / and /' there exists a coupling (X,X') such that P(X = X')= f fAf' (see Figure 4.1). (4.8) t v Maximal coupling probability ■ FIGURE 4.1. The maximal coupling probability. Call these couplings maximal (without a reference to a particular coupling event). In simulation a maximal coupling of continuous random variables X and X' with densities / and g can be generated as follows. Choose a point uniformly at random under the /-curve and let its ^-coordinate be a realization of X. If the point happens to be under the g-curve, let its x-coordinate also be a realization of X'. If not, choose a new point uniformly at random under
Section 5. Poisson Approximation - Total Variation 11 the g-curve and above the /-curve and let its x-coordinate be a realization oiX'. This simulation procedure extends to a collection Xi,..., Xn of random variables with densities fi, ■ ■ ■, fn as follows. Choose a point uniformly at random under fx, consider the densities under which this point falls, and let its x-coordinates be the realizations of the corresponding variables. Then pick a point uniformly at random above these densities and under one of the remaining densities, consider the densities under which the point falls, and let its ^-coordinates be the realizations of the corresponding variables. Repeat this until no density remains. This yields a coupling (Xi,... ,Xn) such that all subcollections (Xni,..., Xnk) are maximal couplings. In fact, repeating this ad infinitum yields a coupling of a countable collection of continuous random variables such that each subcollection is a maximal coupling. We shall refer to the representation (4.2) of X, as a splitting representation. In Chapter 3 (Section 7) we extend the results of this section to arbitrary collections of random elements. 5 Poisson Approximation - Total Variation The following well-known approximation Bin(n,p) « Poi(np) (5.1) can be established and made precise by coupling. 5.1 Approximating a 0-1 Variable Let X be a 0-1 variable with P(X = 1) = p where 0 ^ p ^ 1 and let X' be Poisson p. Let (X, X') be a maximal coupling of X and X'. In order to determine the maximal coupling probability P(X = X'), recall that for all real x it holds that 1 + x ^ ex, which yields P(X = 0) = 1 - p ^ e~p = P{X' = 0), and note that P(X = l)=p> pe-p = P(X' = 1). This and (4.7) yields P(X = X') = P{X = 0) A P(X' = 0) + P(X = 1) A P{X' = 1) — 1 — p + pe~p. Since e~p ^ 1 - p, this implies that P(X = X') ^ 1 - p2 and thus P(XjtX')^p2. (5.2)
12 Chapter 1. RANDOM VARIABLES 5.2 Sums of Independent 0-1 Variables Let Xi,... ,Xn be independent 0-1 variables with P(Xi = 1) = pi, where 0 < pi ^ 1. Put x = x1 + --- + xn. Let X[,...,X'n be independent Poisson variables, X[ with parameter p;. Recall that X' := X[ + ■ ■ ■ + X'n is Poisson pi + • ■ • + p„. Let (Xi, X[),..., (Xn, X^) be independent pairs such that for each i, (Xi, X[) is a maximal coupling of Xi and Xs'. Put X = Xx+--- + Xn and X' = X{+--- + X;. Then (X,X') is a coupling of X and X', and P(X ^ X') ^ P(Xi ^ Xs' for some i) ^ ^ P(X< ^ X?). Applying (5.2) yields P(X ^ X') ^ ^ pi (5.3) If we take pi = p, then X is binomial (n,p), and thus we have the following clear and intuitively appealing random variable formulation of (5.1): Bin(n,p) differs from Poi(np) with probability at most np2. In order to use the above coupling to formulate (5.1) in terms of total variation distance between distributions we take an excursion into that topic for the next two subsections. 5.3 Total Variation — Definition and Identities Let X and X' be random variables with distributions A and /i, that is, for each (Borel) set A, X(A) = P{X g A) and n(A) = P{X' € A). The total variation distance between A and // is simply twice the supremum distance ||A-/i||:=2sup|A(A)-/i(A)|. (5.4) A
Section 5. Poisson Approximation - Total Variation 13 The reason for multiplying by 2 and using the phrase 'total variation' is the following. Suppose X and X' are discrete with probability mass functions p and q, or continuous with densities / and g. Then twice the supremum of A — /t equals the actual total variation (the total of the variation) of p — q, or / — <?, namely IIA -/ill = $>(*)-<?(*)| or ||A-/i|| = ||/-3|. (5.5) X We shall establish (5.5) and two other useful identities: Theorem 5.1. If X and X' are discrete with probability mass functions p and q, or continuous with densities f and g, then (5.5) holds and ||A -/i|| = 2 ^(x)-<?(*))+ or ||A-/i|| = 2 /"(/-$)+, (5.6) x J ||A - /*|| = 2 - 2 5^p(a:) A g(a:) or ||A - /i|| = 2 - 2 f f A g. (5.7) x •* Here we have used the following standard notation: for real numbers a and b let a+ = a V 0, where a V b = maximum of a and 6, a~ = — (aAO), where aA6= minimum of a and 6. Proof. We shall carry out the proof of Proposition 5.1 in the discrete case, the continuous case is analogous. It is clear that for sets A, X(A) - p(A) < 5>(*) - q(x))+ X and that equality holds if we take A = {x : p(x) > q(x)}. Thus sup(A(A) - n(A)) = $>(*) - q(x))+, (5.8) A x ( and similarly, sup(/i(A) - \{A)) = Y,(p(x) ~ <?(*))-• (5-9) A ( From Yl,x P(x) — 1 = Sx <z(x) ^ follows that £(p(a:) - g(x))+ = 5Z(p(a:) - «(*))-. (5.10)
14 Chapter 1. RANDOM VARIABLES Combining (5.8), (5.9), and (5.10) yields Sup|A(A)-MA)|=5>(a0-*(a0)+> A and thus (5.6) holds. From \p-q\ = (p-1)+ + (p - g)~ together with (5.6) and (5.10) we obtain (5.5). Finally, (p - q)+ = p - p A q together with (5.6) and ^p(i) = 1 yields (5.7). □ 5.4 Total Variation and Coupling Let (X, X') be a coupling of two random variables X and X', and let C be a coupling event. Since C implies that X = X', we have for (Borel) sets A, P(XeA,c) = P(x'eA,c) and thus P{x e A) - P(x' e A) = P(x e A) - P{x' g A) = P(x e A, cc) - P(x' e A, cc) ^ P(CC). Apply (5.4) to obtain the coupling event inequality ||P(Xe-)-P(*'e-)IK2P(<7c). (5.n) From (5.7) we see that in the discrete and continuous cases this is just a total variation formulation of Theorems 4.1 and 4.3 specialized to two variables. We also see that the coupling is maximal if and only if identity holds in (5.11). Thus when (X,X') is a maximal coupling, we have ||P(x e •) - P(x' g -)ll = 2P(^ ¥> x'). (5.12) 5.5 Back to the Poisson Approximation Combining (5.3) and the coupling event inequality in the form (5.11) [and with C = {X = X'}} yields n ||P(XG0-Poi(pi + ---+Pn)|K2^p2_ »=i
Section 6. Convergence of Discrete Random Variables 15 In particular, if p, = p, then X is binomial (n,p), and thus we have the following precise formulation of (5.1): || Bin(n,p) - Poi(np)|| ^ 2np2. If a Poisson parameter c is given and n ^ c, then taking p = c/n yields || Bin(n, c/n) - Poi(c) || ^ 2c2/n. tv Sending n to infinity yields in particular, with —> denoting convergence in total variation, Bin(n,c/n) -4 Poi(c), n ->■ oo, (5.13) which further implies 0 (c/n)x(l-c/n)n-x -> eTc— asn->oo, iEZ+, (5.14) where Z+ are the nonnegative integers. 5.6 Comment The above results can be much sharpened and extended; see Barbour, Hoist, and Janson (1992). We just mention here Le Cam's theorem: with X as in Section 5.2, ||P(X£-)-Poi(p1+---+pn)|| ^2 max p^ and in particular, ||Bin(n,p)-Poi(np)|| ^ 2p. 6 Convergence of Discrete Random Variables Let Xi,..., Xqo be discrete random variables taking values in a finite or countable set E. We shall first show that convergence in total variation, like (5.13), is (somewhat surprisingly) equivalent to the apparently weaker pointwise convergence of probability mass functions, like (5.14). We shall then show that these distributional modes of convergence can be turned by coupling into a convergence where the random variables actually hit the limit and stay there. 6.1 Mass Function Convergence <& Total Variation Convergence Suppose P(Xn = x) —)■ P(Xoo = x) as n —> oo for each x £ E. (6.1)
16 Chapter 1. RANDOM VARIABLES Then (P(Xoo = x) - P(Xn = x))+ ->• 0, and since (P(Xoo = x) - P(Xn = X))+ ^ P(Xoo = X) and 53 P(^oo = x) = 1< 00, xeE we have by dominated convergence that 53(P(Xoo=x)-P(Xn = x))+^0, n^oo. xeE Now (5.6) yields convergence in total variation: Xn % Xoo, n ->• oo. (6.2) Conversely, it is clear that (6.2) implies (6.1). Thus (6.1) and (6.2) are equivalent. 6.2 Hitting the Limit We now show that if (6.1) holds, then there exists a coupling {X\,..., Xoo) of X\,..., Xqo and a finite random integer K such that Xn = Xoo, n>K. (6.3) We obtain this by elaborating on the maximal coupling construction in Section 4.2. Note that (6.1) implies, for all x £ E, qn(x) := inf P(Xk = x) t PiX^ = x) as n -> oo. (6.4) Put go = 0 and let K, V\, V2,..., W\, Wi,... be independent random variables such that for 1 ^ n < oo and x £ E P(K = n)=^2 gn(x) - 53 9n-i(x), P(y = ^ = /(«n(a:) - ff„-i(a:))/P(A- = n) if P(K = n) > 0, 1 " ^ [arbitrary if P{K = n) = 0, PW = ^ = /(P(X» = *) - ?n(x))/P(ir > n) if P{K > n) > 0, 1 " X; [arbitrary if P(K > n) = 0. The random variable _K" is finite, since by dominated convergence and (6.4) P{K ^ n) = 53 Q1"^) T 53 p(^°° = a;) = 1 as n ->■ oo.
Section 6. Convergence of Discrete Random Variables 17 Define, for 1 < n ^ oo, Xn=[V« i{n>K> (6.5) \Wn \{n<K. This is a coupling of Xi,..., Xoo, since for 1 ^ n < oo and each x £ E P(Xn =x)= Y, P(Vk = X^P(K = fc) + P(Wn = X^P(K > n) = Y (qk(x) - qk-i(x)) + (P{Xn = x) - qn(x)) = P(Xn = x), while Xqo = Vk, and thus [due to (6.4)] for each x £ E P{X00=x) = Y (9k(x)-qk-i{x))=P{X0O=x). l<fc<oo Clearly, (6.3) holds. 6.3 Converse Conversely, suppose (6.3) holds. Then {K ^ n} is a coupling event of the coupling (Xn,Xoo) of Xn and Xqo. Applying the coupling event inequality (5.11) and the finiteness of K yields ||P(Xn G •) - P(*oo G Oil < 2P(K >n)^0, n ^ oo, which implies (6.1). Since (6.1) in turn implies (6.4), which was in fact the condition under which we established (6.3), we have established the following equivalences. Theorem 6.1. Let X\,..., Xoo be discrete random variables taking values in a finite or countable set E. Then the three claims lim P{Xn =x) = P(X00 = x), x G E, [this is (6.1)] n—»oo Xn -4 Xqq, n —> oo, [£/i«s is (6.2)] liminf P(Xn = x) = P(X00 = x), x G £, [ttis is (6.4)] are equivalent and hold if and only if there exists a coupling (Xi,..., X^) ofXu...,X oo and a finite random integer K such that Xn — Xoo, n ^ K, [this is (6.3)].
18 Chapter 1. RANDOM VARIABLES 7 Continuous Variables - Hitting the Limit Let X\,..., Xqo be continuous random variables with densities /i, •• •, /oo- How should Theorem 6.1 be extended to this case? This section is structured as the previous one and gives the answer at the end. 7.1 Density Convergence =>■ Total Variation Convergence Replacing probability mass functions by densities in the argument in Section 6.1 yields that the following analogue of (6.1): the densities /i, • ■ ■, /oo can be chosen so that (7.1) fn{x) —> foa(x) as n —> oo for each x £ E, implies convergence in total variation, Xn 4 Xoo, n -> oo. (7.2) However, the converse is not as obvious. In fact, it is no longer true, as we shall see in a while. 7.2 Hitting the Limit We shall now show that the condition (7.1) is sufficient to hit the limit, that is, if (7.1) holds, then there exists a coupling (Xi,..., X^) of Xi,..., Xao and a finite random integer K such that Xn = X00, n > K. (7.3) This follows by a coupling construction analogous to the one in Section 6.2. Let us go through the essential part of it again. Put go = 0 and for n > 1 gn= inf fk- Let K, V\, V2, ■ ■ ■, Wi, W2, ... be independent random variables such that for 1 ^ n < 00, K is integer valued and V(K = n) = / gn — I gn~\, V has Jdensity (#" ~ 9n-i)/P(K = n) if P(K = n) > 0, [arbitrary density if Y?{K = n) = 0, w hag f density (/„ - gn)/P(K > n) if P(K > n) > 0, (arbitrary density if P(K > n) = 0. Defining Xn as at (6.5) yields the desired result, since (7.1) implies that gn t /oo as n ->■ 00.
Section 7. Continuous Variables - Hitting the Limit 19 7.3 Converse? Note that (7.3) was actually established under gn f /oo, that is, under liminf /„ is a density of X^, (7.4) n—>oo which is weaker than (7.1). We shall now show that (7.4) is the correct condition, that is, that (7.4) is implied by (7.3). Suppose there is a coupling and a finite K such that (7.3) holds. Then {K ^ n} is a coupling event of the coupling (Xn,..., Xco) of Xn,...,X^, and (4.5) yields for (Borel) sets A, P(i"oo &A,K^n)^gn, 1 ^ n < oo. J A Now, gn increases to liminf n_>oo /n, and thus [by monotone convergence and since K < oo] P(Xoo G A) < / liminf /„. (7.5) yA n->oo Also, since J gn ^ J fn = 1, we have J" lim infn_>oo /n ^ 1- Thus, for each (Borel) set A, 1 = P^ £i) + P(Xoo G Ac) ^ / liminf/„+ / liminf/n^l. J A n^°° Va= n^°° This cannot hold unless (7.5) holds with identity. Thus (7.4) holds. 7.4 Pointwise Convergence of Densities Is Too Strong We have established the equivalence of (7.3) and (7.4). For discrete variables there were two more equivalences, which both break down in the continuous case. We start with (7.1) and (7.4): certainly, (7.1) implies (7.4), and the following example shows that (7.1) is, in fact, strictly stronger than (7.4). Example 7.1. Let the random variables X\,.. .,Xoa be [0,1) valued and have densities /i,..., /«, defined on [0,1) as follows: /i(a:) = /0o(a:) = l) *€[0,1); and for n = 2m + k where m > 1 and 0 < k < 2m [each n > 1 can be written uniquely in this way] put (see Figure 7.1 on the next page) (2, arG[*2-"\(A + l)2-m), W 1 2 — (1 — 2-m)-1, x$ [k2-m,{k+l)2-m).
20 Chapter 1. RANDOM VARIABLES Example 7.1 Example 7.2 FIGURE 7.1. The functions /„ when n = 12 (m = 3 and k = 4). Then for each x G [0,1) there are infinitely many n such that fn(x) = 2, and thus limsup/n(x) = 2^ 1 =/oo(x), x G [0,1). n—>oo Hence (7.1) does not hold. On the other hand, for each x G [0,1), 2-(l-2~m)-1 ^gn(x)<l. This yields, as n —> oo, gn(x) ->• 1 = /oo(a:), a; £ [0,1), and thus (7.4) holds. 7.5 Total Variation Convergence Is Too Weak Finally, consider (7.4) and (7.2). In Section 7.2 we showed that (7.4) implies (7.3) and in Section 7.3 that (7.3) implies (7.2). Thus (7.4) implies (7.2), and the following example shows that (7.4) is, in fact, strictly stronger than (7.2). Example 7.2. Let the random variables Xi,... ,Xoo be [0,1) valued and have densities /i,..., /oo defined on [0,1) as follows: foo(x) = l, a:€ [0,1); and for n = 2m + k where m > 0 and 0 ^ k < 2m put (see Figure 7.1) (0, x G [k2~m, (k + l)2"m), {X |(l-2-m)-\ xi [fc2"m,(fc+l)2-m). Then, due to (5.6), ||P(Xn G •) - P(*oo G Oil = 2/(/oo - fn) + = 2-2~m->0, n->oo.
Section 8. Convergence in Distribution and Pointwise 21 Hence (7.2) holds. On the other hand, for each x £ [0,1) there are infinitely many n such that fn{x) = 0, which yields liminf fn(x) = 0, x <= [0,1), n—»oo and thus (7.4) does not hold. 7.6 What Has Been Achieved? In Sections 7.2-7.5 we have established the following result. Theorem 7.1. If X\,...,Xoo are continuous random variables with densities fi, ■■■, foo, then lim /„ is a density of X^ [this is (7.1)] n—»oo is strictly stronger than liminf/„ is a density of X^, [this is (7.4)] n—»oo which is strictly stronger than Xn -4 Xqo, n ->■ oo. [this is (7.2)] Moreover, (7.4) holds if and only if there exists a coupling (X\,... ,-Xqo) of X\,..., Xqo a-nd a finite random integer K such that Xn = Xqo, n ^ K. [this is (7.3)] In Chapter 3 (Section 9) we shall extend this coupling result to general random elements. 8 Convergence in Distribution and Pointwise Let Xi,..., Xoo be random variables with distribution functions Fi,..., Fqo- The Xn tend pointwise (or surely, or realizationwise) to Xoo if Xn ->• Xoo, n ->• oo, (8.1) which is short for Xn(u>) —> Xoo(w), n —)• oo, for all outcomes to.
22 Chapter 1. RANDOM VARIABLES This means that the Xn close in on the limit without necessarily hitting it as in (7.3). In order to compare (8.1) to (7.3) note that (8.1) can be rewritten as follows: for each e > 0 there is a finite random integer Ke such that \Xn — Xoo| ^ e, n > Ke, (see Figure 8.1). (8.2) 11 Illustration of (7.3) n i-» X„(co) XJlfo) K(fo) i k Illustration of (8.1), that is, of (8.2) n H> XJfit) XO0) KM FIGURE 8.1. Comparison of (7.3) and (8.1). In this section we shall dig out the distributional form of pointwise convergence. The result, once more, is stated at the end of the section. 8.1 Total Variation Convergence Is Too Strong The distributional condition we are looking for should be implied by point- wise convergence. This excludes convergence in total variation, as can be seen by the following example. Put Xn = \jn and X^ = 0. Then certainly (8.1) holds, but Xn does not tend to Xoo in total variation, since clearly ||P(xne-)-P(*ooe-)ll = 2^o. Even the much weaker condition Fn(x) ->-F0O(x), n->oo, iel, (8.3) is too strong, since in our example Fn(0) = 0/>1 = FQO(0). 8.2 Pointwise Convergence =>■ Convergence in Distribution The distributional form of pointwise convergence turns out to be the following slight weakening of (8.3): Fn(x) —> Foo(x), n —> oo, for all x where Fqq is continuous. (8.4)
Section 8. Convergence in Distribution and Pointwise 23 This is called convergence in distribution and is denoted by Xn -> Xqo, n ->■ oo. In order to see that pointwise convergence implies convergence in distribution assume that (8.1) holds and apply its equivalent form (8.2) to obtain that for all x £ E and e > 0 Fn(x) = P(Xn ^x,Ke^n) + P{Xn ^x,KE>n) ^P(Xoo ^x + e)+P(Ke >n) ->• Fqo (x + e), n ->■ oo, and Fn{x) >P{Xn^x,Ks^n) ^PiXcc ^x~e,Ke^n) —> Fqo (x — e), n —> oo. Thus for all x <E E and e > 0 Fqo (x - e) ^ lim inf Fn (x) ^ lim sup Fn (x) ^ F*, (x + e), and sending e to 0 shows that (8.4) holds. 8.3 Turning Distributional Convergence into Pointwise We shall now use the quantile coupling (Section 3.1) to reverse the above implication, that is, turn convergence in distribution into pointwise convergence (see Figure 8.2). FIGURE 8.2. Turning convergence in distribution into pointwise convergence. First we need the following fact. Lemma 8.1. For a nondecreasing real function f, the set of points where f is not continuous is either finite or countable.
24 Chapter 1. RANDOM VARIABLES Proof. That / is not continuous at u is equivalent to the left-hand limit being less than the right hand limit, f(u—) < f(u+). To each such u we can associate a rational number that lies in the interval [f(u—),f(u+)). These intervals are disjoint (because / is nondecreasing), and thus we have established a one-to-one correspondence between {ti£l: f(u~) < f(u+)} and a subset of the rational numbers. Since the rationals are countable, the set {u £ E : f{u—) < /(«+)} is either finite or countable. □ Recall that the generalized inverse of a distribution function F is F_1(w) =inf{a:e E : F(x) > w}, w<E[0,l]. Clearly, F~l is nondecreasing, F(F-l{u)-)^u^F(F-l(u)), «G[0,1], (8.5) and F"1 is continuous at u and F(x-) ^ w ^ F(x) Since F^1 is nondecreasing, the set of points where F^1 is not continuous is finite or countable. Thus there is a random variable U that is uniform on [0,1] and takes values in the set of points at which F^1 is continuous. Use this U to define the quantile coupling Xn = F~1(U), O^n^oo. We shall show that (8.4) implies FnXu) ~* •P,o^1(u)> n-t-oo, for all u where F^1 is continuous, (8.7) which yields the desired result that Xn —> Xoo as n —> oo. 8.4 Establishing That (8.4) Implies (8.7) Fix u G [0,1] and put x = liminf F~l(u). Since F^ is nondecreasing and thus is discontinuous at only finitely or countably many points, we can fix an arbitrarily small e > 0 such that Foo is continuous at both x — e and x + s. Let rik,k ^ 1, be a sequence of integers such that F_1(m) =x. (8.6) x - s < F~* (u) ^ x + e, k > 1.
Section 8. Convergence in Distribution and Pointwise 25 Applying (8.5) yields Fnk(x-e) ^ u ^ Fnk{x+e), k > 1. Send k to infinity and use (8.4) and the choice of e to deduce Fxj (x - e) ^ u ^ Fqo(x +e). Then send s to zero to obtain Fooix-) ^u^ Foo(a:). (8.8) Replace x by i/ = lim sup F^^tt) n—>oo in the above argument to obtain Fcoiy-) ^u^ Food/). (8-9) If Fj;1 is continuous at u, we obtain from (8.6), (8.8), and (8.9) that F-1(«) = ^ = liminfF-1H, n—j-oo F^(u) = 2/ = limsupF-1(M). n—j-oo Thus (8.7) holds. 8.5 What Has Been Achieved? In Sections 8.2-8.4 we have established the following result. Theorem 8.1. Let X\,..., Xqq be random variables. Then V D V A„ -» Aqo, n -» 00, if and only if there exists a coupling (Xi,..., X^) of X\,..., X^ such that Xn ->• Xqo, n ->■ oo. We finally mention that the definition (8.4) of convergence in distribution can be seen to be equivalent to E[/(Xn)] -»■ E[/(Xoo)], n -»■ oo, for all bounded continuous functions /. This is taken to be the definition of convergence in distribution for random elements in metric spaces. In Chapter 3 (Section 10) we extend Theorem 8.1 to random elements in a separable metric space.
26 Chapter 1. RANDOM VARIABLES 9 Quantile Coupling - Dominated Convergence The pointwise version of the dominated convergence theorem [see Ash (1972)] states that if Xi, X2,.. ■,X,*,,X are random variables such that |Xn| sj X, 1 ^ n < oo, E[X] < oo and Xn ->• X^ as n ->■ oo, then E[|Xoo|] < oo and E[Xn] ->• E[Xoo] as n ->• oo. (9.1) Using the quantile coupling it is straightforward to extend this result to the following distributional form. Theorem 9.1. // X\, X2, ■ ■ •, Xqo, X are random variables such that D \Xn\ sj X, 1 ^ n < oo, E[X] < oo and Xn —> X,*, as n -^ oo, then (9.1) ZioMs. Proof. Under the assumptions of the theorem we have d D X+ ^ X and X+ ->• X+, as n ->■ oo, _ D r> Xn ^ X and Xn —>■ X^ as n —>• oo. Apply the quantile coupling in Sections 3 and 8 to turn these distributional relations into pointwise ones, that is, to obtain copies of X+ and X~ that are pointwise dominated by a copy of X (Section 3) and converge pointwise to copies of X+, and X^, respectively (Section 8). By the pointwise version of dominated convergence, this together with E[X] < oo implies E[X+] < oo and E[X+] ->• E[X+] as n ->• oo, E[X^] < oo and E[X"] ->• E[X^] as n ->• oo. Thus E[|Xoo|] = E[X+] + E[X-] < oo and E[Xn] = E[X+] - E[X"] -> E[X+] - E[X"] = E[Xoo] as n -> oo, and the proof is complete. D In Chapter 2 we shall need the following extension to a continuous index.
Section 10. Impossible Coupling - Quantum Physics 27 Corollary 9.1. If Xt, t £ [0, oo], and X are random variables such that \xt\^x, te[o,oo), E[X] < oo and Xt —> Xqo as t —» oo, then E[|Xoo|] < oo and ~E[Xt] -> E[Xoo] as t -t oo. Proof. A collection of real numbers like E[Xt], t e [0, oo), tends to a limit if and only if it tends to this limit along all subsequences E[Xt(nj], n ^ 1, where the t(n) increase to oo as n —> oo. Apply Theorem 9.1 to Xt^ to obtain E[Xt{n)] ->■ E[X oo] as n -> oo. D 10 Impossible Coupling - Quantum Physics We end this first chapter on a rather different note: the coupling aspect of a (the?) problem in quantum physics. 10.1 A Surprising Experimental Result The following experiment has been carried out. Some material (calcium, carefully excited by laser) sends off particles (photons) in pairs, one particle to the left and the other to the right. Measuring devices are placed on each side of this material with measurements made when particles pass through. What is being measured is the so-called polarization of the particle, which can be either 1 or —1 and depends on the angle in the plane orthogonal to the direction of movement. 0. When the measuring devices are aligned to measure polarization in the same direction, say 0°, the same measurement is always recorded on both sides. 1. When the left device is tilted 30° and the right device is kept at the initial 0° position, then the measurements agree | of the time. 2. When the left device is rotated back to its 0° position and the right device is tilted —30° (that is, 30° in the opposite direction), then the measurements also agree | of the time. 3. When the left device is again tilted 30° and the right device is kept at its new —30° position (that is, the total relative rotation is 60°), then the measurements agree ~ of the time.
28 Chapter 1. RANDOM VARIABLES 10.2 Why Surprising? On the basis of the above empirical facts it is now natural to build the following model. Consider a particular pair of photons, and set X = the polarization of the left particle in the 0° direction = the polarization of the right particle in the 0° direction, Y = the polarization of the left particle in the 30° direction, Z ~ the polarization of the right particle in the -30° direction; see Figure 10.1. Setup 0 -D- Measurements agree all the time. Setup 1 -D- Measurements agree 3/4 of the time. Setup 2 -D- Measurements agree 3/4 of the time. Setup 3 -D- Measurements agree 1/4 of the time. FIGURE 10.1. The experimental setups. Interpreting the relative frequencies as probabilities, we have and P(Y = X)=I>(X = Z)= f, P(Y = Z)=\.
Section 10. Impossible Coupling - Quantum Physics 29 By basic rules of probability, P{Y = Z) ^P(Y = Z,X = Z) = P(Y = X,X = Z) = P(F = X) - P{Y = X,X^Z) ^P(Y = X)-P{X^Z) = P(Y = X) + P{X = Z)-l, that is, P{Y = Z)^P(Y = X) + P{X = Z)-l. (10.4) Combine this, (10.2), and (10.3) to obtain the following contradiction: This contradiction is derived in an ordinary probabilistic way from straightforward empirical facts: real life seems to contradict probability theory... 10.3 Predicted by Quantum Theory Because of this apparent contradiction it is all the more annoying (for probabilists) that the empirical results are in fact predicted by quantum mechanics, which calculates the probabilities as follows: P(F = X)= P{X = Z) = cos2 30° = 1 - sin2 30° = 1 - (|)2 = |, P(F = Z)=cos260° = (f)2= I. 10.4 No Contradiction at the Level of Observation Note that X, Y, and Z refer to polarization as intrinsic properties of the particles, thought of as existing simultaneously without interaction with the macro world (without being measured). If we instead stay at the level of observation (measurement), then it turns out that the contradiction disappears. It is clear that we are dealing with three experimental setups (leaving out the one with the measuring devices aligned). First consider Setup 1: the case when the left device is tilted 30° and the right device is kept at the initial 0° position. Put X\ = observed polarization of the right particle in the 0° direction, Y\ — observed polarization of the left particle in the 30° direction.
30 Chapter 1. RANDOM VARIABLES In addition to the measurements agreeing | of the time it has been recorded that —1 and 1 are observed in equal proportions on both sides. Specify the complete joint distribution of X\ and Yx as p(x1 = -i,r1 = -i) = p(x1 = i,y, = i)= §, p(x1 = -i,r1 = i) = P(A:1 = i,Yi = -i)= |. This is in accordance with the relative frequencies since P(Yi=-l)=P(X!=-l) = P(Xi = -l,Yi = -1) +P(X1 = -l,Yi = 1) _ i_ ~ 2' P(X, = Yx) = P(Xi = -1, Yi = -1) + P(Xi = 1, Yi = 1) _ 3 4" Now consider Setup 2: the case when the left device is at the 0° position and the right device is tilted —30°. Put X2 = observed polarization of the left particle in the 0° direction, Z2 = observed polarization of the right particle in the —30° direction. Letting (X2,Z2) have the same distribution as (Xi,Y\) again yields probabilities in accordance with the relative frequencies, P(Y2 = -1) = P(X2 = -1) = \ and P(X2 = Y2) = f. Finally, consider Setup 3: the case when the left device is tilted 30° and the right device -30°. Put Y3 = observed polarization of the left particle in the 30° direction, Z3 = observed polarization of the right particle in the —30° direction. The measurements now agree only | of the time, but it has still been recorded that —1 and 1 are observed in equal proportions on both sides. Specify the complete joint distribution of Y3 and Z3 as p(y3 = -i, z3 = -i) = P(r3 = 1, z3 = 1) = |, p(y3 = -1, z3 = 1) = P(y3 = 1, z3 = -i) = §. This is in accordance with the relative frequencies, P(y3 = -1) = P(Z3 = -1) = I and P(y3 = Z3) = \. We have managed to account for all three experiments, and thus the contradiction is not at the level of observation. The contradiction appears when we assume that each particle has a polarization in a direction where we do not make a measurement.
Section 10. Impossible Coupling - Quantum Physics 31 10.5 What Has This to Do with Coupling? We have created three pairs (Xi,Y\), (X2,Z2), and (Y3,Zz)- What we proved in Section 10.2 is that there is no coupling of these pairs such that the X-variables agree, the y-variables agree, and the Z-variables agree. More precisely, there is no jointly distributed triple (X, Y, Z) such that (X,Y) = (X^Y,), (X,Z) = (X2,Z2), (Y,Z)^(Y3,Z3). That is, although reality seems to be able to construct a coupling, we can't. 10.6 Does Probability Not Suffice in the Micro World? It is one of the implications of quantum theory that polarization cannot be measured simultaneously in all three directions; only one measurement on each particle is possible. The reason we have measurements in pairs is that we have two particles. The above contradiction further suggests that polarization exists in the micro world only through interaction with the macro world (only by being measured). Is there then nothing, no reality, behind the observations? Or does probability not suffice to describe it? One school of thought claims that classical probability (that is, Kolmogorov's axioms) is too narrow. It should be replaced by quantum probability (an axiom system more general than Kolmogorov's) in a similar way as Newton's theory had to be replaced by Einstein's in physics. Applying quantum probability there is no longer a contradiction to be derived from the assumption that polarization exists in all three directions. See Kummerer and Maassen (1998) and Accardi (1998) for such viewpoints. Note that there are finitely many possible outcomes in each individual experiment, so the contradiction does not appear to have to do with countable additivity. Since Kolmogorov's axioms otherwise reflect properties of relative frequencies, it is hard to swallow that they should not apply. And so it is not surprising that there are other attempts to get rid of the contradiction. See Maudlin (1994) and Gill (1998, 1999) for the following point of view. Behind the attempt in Section 10.2 to create a model are several implicit assumptions. One assumption is that measuring the polarization in a particular direction does not affect the polarization in the other directions. In other words, an interplay between the micro and macro worlds is not allowed. Allowing a local interplay is not a serious crime against physical ideas, but it turns out that a nonlocal interplay is needed to get rid of the contradiction. Nonlocal means that the experimental setup on the left, for instance, affects the polarization of a particle measured on the right. This is not easy to accept, but for an Einsteinian realist this is easier to accept than having to discard Kolmogorov's axioms, which is too close to discarding 2 + 2 = 4.
32 Chapter 1. RANDOM VARIABLES 10.7 What Does This Teach Us About Coupling? The above excursion into the quantum experience shows that we have to be careful when assuming existence of couplings. For empirical or intuitive reasons joint distributions may appear to exist when they do not. In Chapter 3 (Sections 3 through 5) we shall consider some safe methods for constructing couplings. The next chapter, however, is devoted to the classical triumphs of the coupling method.
Chapter 2 MARKOV CHAINS AND RANDOM WALKS 1 Introduction We now turn to the coupling of Markov chains in discrete and continuous time, random walks, and renewal processes, the aim being to establish asymptotic properties such as asymptotic stationarity. We start with the earliest example, the classical coupling, which we present first in the pleasant context of birth and death processes. 2 Classical Coupling - Birth and Death Processes A continuous-time irreducible nonexplosive birth and death process is a collection of random variables (stochastic process) Z = (2s)sg[0,oo) taking values in the state space E = {0,1,...} and developing in time (as the time parameter s increases) in such a way that Z changes state only finitely often in finite time intervals (nonexplosion) and whenever Z visits a state i, it stays there an exponential length of time (sojourn time) with parameter depending only on i, and then jumps either one step up to i + 1 (a birth) or one step down to i — 1 (a death, this occurs only if i > 0) with positive probabilities depending only on i. Irreducibility follows from the positivity of the birth and death probabilities (irreducibility means that each state is visited with positive probability starting from any other 33
34 Chapter 2. MARKOV CHAINS AND RANDOM WALKS state). Finally, we let the paths be right-continuous, that is, Zt = Zt+, where Zt+ = lims;t Zs. 2.1 Notation Let A be the distribution of Zq, the initial distribution. Let P\ indicate this. Let Pj indicate that Z starts in state j, that is, Z0 = j. The semigroup of transition matrices is Pl = (Ptj:i,j e E), t > 0, [semigroup because PlPs = Pt+S] where P'- denotes the probability of going from i to j in a time interval of length t, P£ =P(Z.+t=j|Z. = t), s,t^Q, i,j€E. If we treat the initial distribution A as a row vector, then the row vector AP* represents the distribution of Zt, px(zt = j) = \pj, t^o, jeE. 2.2 The Classical Coupling Let Z' be a differently started independent version of Z, that is, Z' is independent of Z and has the same semigroup of transition matrices but another initial distribution A', say. Let T be the time when Z and Z' first meet, T = inf{i >0 : Zt = Z[) (see Figure 2.1). Let Z" be the process that follows the path of Z' up to T and then switches to Z, z,,[z't if*<r, ' \zt iit^T. At time T the processes Z and Z' are in the same state and will continue as if they both were starting anew in that state. Therefore, modifying Z' by switching to Z at time T does not change its distribution, that is, Z" is a copy of Z'. Thus (Z, Z") is a coupling of Z and Z', the classical coupling. 2.3 The Coupling Time T - Asymptotic Loss of Memory The time T when Z and Z" merge is called a coupling time or coupling epoch. By definition (Section 4.1 in Chapter 1) the event {T < t} = {Zt = Z't'}
Section 2. Classical Coupling - Birth and Death Processes 35 FIGURE 2.1. The classical coupling in the birth and death case. is a coupling event for the coupling (Zt, Z[') of Zt and Z[. The coupling event inequality (5.11) in Chapter 1 yields the coupling time inequality ||XPl - \'Pl || < 2P(T > t), t ^ 0. (2.1) The coupling is called successful if P(T < oo) = 1. This implies asymptotic loss of memory P(T<oo) = l => ||AP* -A'P'H -+0, t-^oo. (2.2) 2.4 Recurrence of the Birth and Death Process implies T < oo The process Z is called recurrent if each state j £ E is recurrent: Pj(tj < oo) = 1, where r,- is the time of first visit to state j (re-entrance if Z starts in j): Tj=M{t>0:Zt- ?j,Zt=j}. Recurrence implies, by irreducibility, that Pa(tj < oo) = 1 for all initial distributions A and all states j, since otherwise there would be states i and j such that Z could go from j to i and never return to j, contradicting the recurrence of j. By the birth and death property Z and Z' cannot pass each other without meeting (since jumps cannot happen simultaneously due to the exponen- tiality of the sojourn times in the individual states). Thus if Z starts above Z', we have T < r0, while if Z starts below Z', we have T < Tq, that is, T < t0 V To (see Figure 2.1). (2.3) If Z is recurrent, this implies that P(T < oo) = 1, and (2.2) yields that an irreducible recurrent birth and death process forgets how it started: for all initial distributions A and A', Z recurrent => ||XP* - A'P*|| -> 0, t -> oo. (2.4)
36 Chapter 2. MARKOV CHAINS AND RANDOM WALKS 2.5 Recurrence Implies Existence of a Stationary Vector Let Z be recurrent. Fix an arbitrary state k and let i/, be the expected amount of time spent in i between two entrances to k, Ei rTk / 1{Za=i} Jo ds i e E. (2.5) Then the row vector v with entries i/j, i 6 E, is stationary: vPl = v, t > 0. (2.6) This can be seen as follows. Note that Vi = E* r-oo poo I l{z,=i,Tk>s}ds = / Pk(Zs = i,rk > Jo J Jo s) ds. Note also that we can tell whether the event {rk > s} happens or not by observing Z only in the time interval [0, s], and thus, conditionally on {Zs = i,rk > s}, the process Z starts anew in state i at time s, that is, Ptij=Pk(Zs+t = j\Zs=i,rk >s). These two observations yield /•OO ViP^ = / Pk (Zs+t =j,Zs = i,rk> s) ds. Jo Sum over i to obtain the first equation in /•OO r-Tk *=j Pk(Zs+t=j,Tk>s)ds = Ek[J l{Za+t-_ = E4/ liz,=j}ds vP] ■■3} ds r fTk ~\ r rTk+t = Efc[j l{Zs=j}ds\+Ek[J 1 {Z,=j} ds\. Since Z starts anew in state k at time rk, the last term can be replaced by Efc[Jo l{z„=j} ds]. This yields the first equation in V.=j} ds = Efc[y l{z.=j}ds + E; U' !{Z.=j} rfs that is, (2.6) holds. Note also that 5>i = E J P 53 l{Zj=i} dsl = Efc [ I'" ll - Etlnt]. (2.7)
Section 2. Classical Coupling - Birth and Death Processes 37 2.6 Positive Recurrence Implies Asymptotic Stationarity The process Z is called positive recurrent if each state j £ E is positive recurrent: rrij := Ej[r,] < oo. In this case, due to (2.6) and (2.7), the row vector ir = u/mk is a stationary distribution for Z, that is, irPt=Tr, t^Q, and yVj = 1. Choose Z' stationary to obtain that Z is asymptotically stationary: putting A' = 7r in (2.4) yields \ nt tv XP ->• 7T, t ->• OO. In other words (cf. Theorem 6.1 in Chapter 1), we have established the following result: for all initial distributions A and all j £ E it holds that Z positive recurrent => P(Zt = j) —> ftj, t —> oo. (2.8) Remark 2.1. The proof of (2.8) works for all stationary distributions ir. Thus 7r must be unique. In particular, 7r does not depend on the arbitrary fixed state k. For each j e E, let hj be the expected sojourn time in j, hj = E^inf {t > 0 : Zt- = j, Zt ? j}}, (2.9) and note that v^ = h^. This yields (by taking k = j) that the unique stationary distribution tt is given by 7Tj = hj/m.j, j G E. 2.7 Null Recurrence Implies Asymptotic 'Nullity' A recurrent process Z is called null recurrent if each state j e E is null recurrent: mJ = %h] = °°- In this case (2.6) yields ViP\k ^ uk, and since vk = hk < oo (A* is the expected value of an exponential random variable and thus hk < oo) and P\h > 0 for all t > 0 (some t > 0 suffices of course), we obtain Vi < oo, ie E. (2.10) Take a finite set of states B and put A' = v(-\B), that is, V = \vjlY,i£Bvu J e B, J 10, j^B.
38 Chapter 2. MARKOV CHAINS AND RANDOM WALKS Then A' ^ vIY^i£Bvi [entrywise], which implies X'Pj ^ ^V XagB ^ and thus by (2.6), *'^<"j/I>. 3€E. (2.11) i£B This yields the second inequality in \Z' has initial distribution A'] p(zt = j) < \p(zt = j) - p(z; = j)\ + p(z; = j) *Z\P(Zt=])--P(Z't=j)\ + uj/,£iui. Apply (2.4) to obtain limsupP(Zt=j)<^/2I/i- (2-12) Sending B "[ E yields ^ieB Vi t X)ieB "» = mfc = °°) an(l tnus we have established the following result: for all initial distributions A and all j £ E, Z null recurrent =*■ P(Zt = j) ->• 0, t ->• oo. Remark 2.2. In the null recurrent case no stationary distribution exists, since if a stationary 7r existed, then we could let it be the initial distribution of Z and obtain ttj = P{Zt = j) —► 0 as t —► oo, that is, 7Tj = 0 for all j, which contradicts Y^i^E^i ~ 1- Remark 2.3. The result in Section 2.6 that limt_).00P(Zt = j), j £ E, is a stationary distribution was obtained under the condition my. < oo for some state k. The result in this subsection that limt_>00P(Zt = j) = 0, j £ E, was obtained under the condition m^ = oo for some state fc. Since a stationary distribution cannot be identically 0, it follows that either all states are positive recurrent or all states are null recurrent. That is, a recurrent Z is either positive recurrent or null recurrent. 3 Classical Coupling - Recurrent Markov Chains The argument in the previous section went basically as follows. In order to deduce asymptotic properties of the process Z, let a differently started version Z' run independently of Z until it hits Z, at time T, say. At time T switch from Z' to Z to obtain a copy Z" of Z' that sticks to Z from time T onward. Establish that T is finite with probability one to deduce that Z and Z' have the same asymptotic behaviour. In the positive recurrent case, choose Z' stationary to obtain that Z is asymptotically stationary. In the null recurrent case, choose Z' such that P(Zt' = j) is close to 0 for all t to obtain that P(Zt = j) is close to 0 asymptotically.
Section 3. Classical Coupling - Recurrent Markov Chains 39 *Z'andZ" ° * / \ p '0--0' 2' 0 T FIGURE 3.1. Classical coupling of two discrete-time Markov chains. This argument extends immediately to finite and countable state space Markov chains in both discrete (see Figure 3.1) and continuous time, except for the proof of the finiteness of T: the inequality (2.3) relies on the birth and death structure. In fact, we cannot establish that T is finite in the null recurrent case. We deal with these complications below, first in continuous and then in discrete time. 3.1 Continuous Time — Preliminaries A continuous-time stochastic process Z = (-^s)se[o,oo) with a finite or countable state space E is called a Markov jump process (or a continuous-time Markov chain) if Z is piecewise constant and right- continuous (as a function of s) and satisfies the Markov property: the future is independent of the past given the present, that is, P(Z.+t=j\Zh,0 4:h4:8;Za = i) = P!j, s,t>0, i,j £ E, where P/- depends only on t, i, and j. The independence of s is called time-homogeneity. For convenience we assume time-homogeneity here to be part of the Markov property. Below we outline some properties of Markov jump processes needed here. For formal proofs, see, for instance, Asmussen (1987). A Markov jump process behaves as follows: it stays in a state i an exponential length of time with parameter depending only on i and then jumps to a new state with probabilities depending only on i (if the process is nonexplosive, then this description is equivalent to the Markov property). Thus a birth and death process is the special case with E the nonnegative integers and with jumps of size one being the only possible jumps. In addition to the terminology and facts from the birth and death case in Section 2 we need the following. Assume that Z is irreducible, that is, for each i, j £ E there is a t > 0 such that Py > 0. Then Z can go from i to j
40 Chapter 2. MARKOV CHAINS AND RANDOM WALKS through a finite sequence of states i = io,«i, ■ • -in-i;*n — j- Conditionally on going through these states the sojourn times in io,i\,... ,in are distributed like independent exponential random variables, say X0,... ,Xn, with parameters depending on these states. Since for all t > 0 we have P{X0 H 1- Xn_i < t < X0-\ h Xn) > 0, it follows that Z can enter the state j before time t and not leave it until after time t, that is, Z can be in j at time t. Thus irreducibility implies that Ftj >0, t>0, i,j€E. (3.1) The process Z is transient if each state j E E is transient: Pj{Tj < CO) < 1. Transience implies (since each time Z leaves the state j it has the probability Pj(tj = oo) > 0 of never entering j again) that P\(kj < oo) = 1 for all initial distributions A and all j E E where (with sup 0=0) Kj = sup{£ > 0 : Zt = j} — the time of the last exit from j. An irreducible Markov jump process is either recurrent or transient. This can be seen as follows. Let Z start in some state i and suppose there exists a recurrent state j. Then, by irreducibility, Z is sure to visit j (because otherwise it could go from j to i and never return to j, contradicting the recurrence of j). Each time Z leaves j it has a positive probability of visiting i before returning to j (because otherwise it would always return to j before visiting i and thus could not go from j to i, contradicting the irreducibility of Z). Since, by irreducibility of Z and by recurrence of j, Z leaves j infinitely often, it will eventually get back to i with probability one, that is, i is also recurrent. This yields the desired result that if one state is recurrent, then all the states are recurrent. 3.2 Continuous Time - The Theorem We shall now establish the following result. Theorem 3.1. Let Z be an irreducible recurrent Markov jump process with a finite or countable state space E. Then, with k a fixed state, the row vector v with entries defined at (2.5) is stationary and the entries are finite. Further, Z is either positive recurrent or null recurrent. If Z is positive recurrent, then the classical coupling (defined in Section 2.2) is successful, the row vector ■k — (hj/m.j : j € E) [hj is expected sojourn time in j, see (2.9)]
Section 3. Classical Coupling - Recurrent Markov Chains 41 is a unique stationary distribution, and for all initial distributions A and all j e E, PX{Zt=j)-^7Tj, £->0O. (3.2) // Z is null recurrent, then no stationary distribution exists, and for all initial distributions A and all j 6 E, Px{Zt=j)^Q, i->oo. (3.3) Proof. For the stationarity of v, see Section 2.5, and for the flniteness of Vi, see (2.10). In order to establish the rest of the theorem, let Z' be a differently started independent version of Z. Then the bivariate process \ZS, Zs)sg[0jOo) is a Markov jump process with state space E2 and transition probabilities pUi'Hi,r) = pijpi'j'^ l > °> *'*'. J'.J" e E- These are strictly positive due to (3.1), that is, (Zs,-Zs)se[o,oo) is also irreducible. Thus (Zs, Zj)se[o>00) is either recurrent or transient. We treat these two cases separately. Case 1: suppose (Zs, Zj)se[0>Oo) is recurrent. Let T be the classical coupling time. Instead of (2.3) use that for an arbitrary j, T ^ T(jj) — tne time of the first visit of (Zs, Z's)s&[0,oo) t0 (hi)- Since (Zs, Zj)se[0jOo) is recurrent, this implies P(T < oo) = 1 for all initial distributions A and A', and the same argument as in Section 2 yields the desired results. Case 2: suppose (Zs, Z's)S£[0,oo) is transient. Then for an arbitrary j, KU'i) = t'ie ilme of the last exit of (Zs, Zj)se[o,oo) from (j,j) is finite with probability one. Letting Z' have the same initial distribution as Z yields P(Zt = j)2 = P((Zt,Zfi = (j,j)) ^ P(kuj) > t), and thus for all initial distributions A and all j £ E, P(Zt = j) -)• 0, t -)• oo. (3.4) It follows from this that Z cannot have a positive recurrent state k because if it had, then we could choose the initial distribution of Z to be the stationary ■k = v/mk to obtain from (3.4) that ttj = P(Zt = j) —> 0 as t —> oo, that is, itj = 0 for all j, which contradicts ^ieE^i = 1- Thus Z is null recurrent, and (3.4) is the desired limit result. The nonexistence of a stationary distribution follows from Remark 2.2 at the end of Section 2.7. □
42 Chapter 2. MARKOV CHAINS AND RANDOM WALKS 3.3 Discrete Time — Preliminaries A discrete-time stochastic process Z = (Zt)8° with a finite or countable state space E is called a Markov chain in discrete time if P(Z„+i =j\Z0,...,Zn-i;Zn = i) =Pij, n>0, i,j £ E, where P$j depends only on i and j. As in the continuous-time case we assume here that time-homogeneity (the fact that P^ does not depend on n) is part of the Markov property. The matrix Pn of n-step transition probabilities is the nth power of P = (Pij : i,j £ E). We use notation and terminology from the continuous-time case with the following modification (to conform with standard practice even if it is not formally needed): let tj be the time of first visit to state j (revisit if Z starts in j), that is, Tj = inf{n > 0 : Zn - j}. Again the same argument as in Section 2.5 yields vP — v, where v is the row vector with entries {k is a fixed state) r n ~\ Vi = Efc 2_, l{z„=i} (note that now v^ = 1). (3.5) We can use the same facts as in the continuous-time case except (3.1). Irreducibility means that Pf" > 0 for some n > 0, and this does not imply that it holds for all n > 0. In particular, periodicity can pop up in discrete time. Therefore, we need the following property. The Markov chain Z is called aperiodic if each state j 6 E is aperiodic: gcd{n > 1 : P£ > 0} = 1; here gcd denotes greatest common divisor, the largest integer that is a factor of all the integers in the set. 3.4 Discrete Time — The Theorem We shall now establish the following result. Theorem 3.2. Let Z be an irreducible recurrent Markov chain in discrete time with a finite or countable state space E. Then, with k a fixed state, the row vector v with entries defined at (3.5) is stationary and the entries are finite. Further, Z is either positive recurrent or null recurrent, and either is aperiodic or has no aperiodic state.
Section 3. Classical Coupling - Recurrent Markov Chains 43 If Z is aperiodic and positive recurrent, then the classical coupling (defined in Section 2.2) is successful, the row vector ■k = (1/rrij :jEE) is a unique stationary distribution, and for all initial distributions A and all j e E, P\(Zn=j)-^TTj, n-^oo. If Z is aperiodic and null recurrent, then no stationary distribution exists, and for all initial distributions A and all j 6 E, P\(Zn = j) ->0, n->oo. Proof. Suppose Z has an aperiodic state h. Then, by Lemma 3.1(6) below, there is an integer n such that P£h > 0 and P^1 > 0 [take B = {k ^ 0 : Pfih > 0}]. For alii £ E and all integers /, k and m ^ 0, we have p^k+m ^ P\hPhh^bA- ^v irreducibility, we can find / and m such that P\h > 0 and P% > 0. Thus P/+n+m > 0 and pl.+n+1+m > 0. Thus either all states are aperiodic or no state is aperiodic. The rest of the proof is the same as in the continuous-time case (see the proof of Theorem 3.1), except that the irreducibility of the bivariate Markov chain (Zk,Z'k)1f is harder to establish, since we cannot rely on (3.1). We have only that Pf™ > 0 for some n > 0, not necessarily all. In order to establish that (Z&, ZjJ.)g° is irreducible, let h £ E be fixed and hjii'ij' € E arbitrary. We must show that there is an n such that both P"j > 0 and P"-, > 0. We have, for all integers l,m,l',m' ^ 0 and for all n ^ (l + m) V(/' + m'), P"i v- pi pn-1 — m r>m „„J pn v. of pn—l' — m'pm' ij ^ rihrhh rhi ana r{,y ^ rilhrhh rhy. Use the irreducibility of Z to find l,m,l',m' such that p/h>o, ph™>o, p/;h>o, p$>o. According to Lemma 3.1(6) below, if we take n large enough, then both PhY'~m > 0 and P^l'-m' > 0 [take B = {k > 0 : P*fc > 0}]. □ Lemma 3.1. (a) Let A be a subset of Z that is aperodic [gcd A = 1], additive [a, 6 6 A implies a + b £ A], and closed under negation [a 6 A => -ae 4]. Then A = 1. (6) Lei B be a subset of the nonnegative integers that is aperiodic and additive. Then there is an integer ub such that n 6 B for all n ^ rig. PROOF, (a) Since A is closed under addition and negation, it follows that A contains dZ, where d = min{fc ^ 1 : k £ A}. For each 6 £ A there is
44 Chapter 2. MARKOV CHAINS AND RANDOM WALKS a k £ Z such that 0 ^ b — kd < d, and due to the definition of d we have b - kd = 0. Thus A = dZ, and the aperiodicity of ^4 yields d = 1. (6) It is no restriction to assume that 0£B. The set i:={fc£Z: there is an n^ € B such that n^ + k £ B} is aperiodic [since B is so, take n^ = 0 to see that B is a subset of ^4], additive [take na+b — na + nb], and closed under negation [take n_a = na + a\. Thus A = Z, due to (a). Thus IgtI. We shall show that nB = n\ does the trick. Each n~^n\ can be written in the form n = n\ + mn\ + k, where m ^ 0 and 0 ^ k < m. Thus n = (ni + m — k)rii + k{n\ + 1), which lies in B [since B is additive, since both n\ and n\ + 1 are in B, and since both n\ + m — k and k are nonnegative]. □ 3.5 Comment on the Strong Markov Property A countable state space Markov jump process Z satisfies the strong Markov property at hitting times: for a hitting time r = inf{i > 0 : Zt £ A} of a subset A of E it holds that, conditionally on r < oo, the process (Zr+S)se[0ioo) is a version of Z and is conditionally independent of (Zs)s€[0 T\ given ZT. In other words, at the time r the process Z starts anew in state ZT independently of how it got there. The same comment applies to Markov chains in discrete time. Thus, when proving in Section 2.2 that Z" is a copy of Z', we were actually using the strong Markov property of the bivariate process (Zs, Zj)s€[0oo) at T = hitting time of the diagonal {(j,j) ■ j £ E} of E2. In fact, the strong Markov property holds at stopping times, that is, random times r such that for each t ^ 0 the event {r ^ t} is determined (measurably) by (Zs)s€[0t] [see, for instance, Theorem 3.1 in Chapter 1 of Asmussen (1987)]. A Markov process with this property is called a strong Markov process. 4 Classical Coupling - Rates and Uniformity Have another look at the coupling time inequality (2.1) ||APt-A'P'|K2P(T>i). (4.1) If we knew not only that T is finite but also how fast P(T > t) goes to zero, then we would obtain a rate result for the convergence of the Markov process. Also, if T is stochastically dominated by a finite random variable
Section 4. Classical Coupling - Rates and Uniformity 45 with distribution that does not depend on the family Pl, t > 0, as long as it lies in some fixed class of transition matrices, then we would obtain uniform convergence over that class. In this section we shall take a closer look at the classical coupling time T in two simple cases. 4.1 Birth and Death Processes Let Z be the irreducible nonexplosive recurrent birth and death process considered in Section 2. There we proved for the classical coupling time T that T ^ T0 V T0. If we know, for instance, that tq and t'0 have finite a-moments for some a > 0, that is, E[t£] < oo and E^"] < oo, then E[Ta] < E[r0a] + E[r^a] < oo and thus taP(T > t) < E[Tal{T>t}] -> 0, £ -> oo, which together with (4.1) yields the following rate result: for a > 0, E[r0a],E[r^a]<oo => ia||AP' - A'P*|| -> 0, t -> oo. It is worth noting that for this rate result we needed only recurrence, not positive recurrence as in the generalization to Markov jump processes mentioned in Section 4.3 below. 4.2 Finite State Space — Doeblin's Argument The classical coupling was introduced by Doeblin in 1938 in order to establish asymptotic stationarity of a regular discrete-time finite-state Markov chain Z = (Zfc)g°. Regularity means that there is an integer m and an s > 0 such that P%>e, i,jeE. ■ (4.2) Doeblin argued along the following lines. Let a differently started version Z' run independently of Z until the two chains meet, at a time T, say. From T onward let the two chains run together. Regularity implies that if Z has not met Z' up to time mk, then it will meet Z' before time mk + m with probability no less than e. Thus P(T > km) < (1 -e)k -> 0, k -> oo. (4.3)
46 Chapter 2. MARKOV CHAINS AND RANDOM WALKS Thus the chains eventually coincide and we obtain |P(Z„ = j) - P(Z'n = j)\ *C P(Zn ?Z'n)^0, n^ oo. Add to this the observation that maxie£ P,™ is nonincreasing and that mini€E P?j is nondecreasing in n to deduce that P(Zn = j) has a limit. (Nowadays one usually takes Z' stationary, which makes the last sentence unnecessary.) Doeblin's argument in fact yields that P(Zn = j) goes to the limit itj at a geometric rate: from (4.3) and (4.1) with A' = ■k (and with t replaced by n) we get ||APn - tt|| < 2(1 - £)["/m' (here [x] = sup{n e Z : n < x}), which yields the following geometric rate result: Vp< {l-e)-l/m : pn||APn-7r|| -> 0, n -> oo. Note also that if we let Vm,e denote the class of transition matrices P satisfying (4.2) with m and e fixed, then we obtain the following uniform convergence result: sup ||APn - tt|| -> 0, n -> oo. The rate result also holds uniformly: Vp < (1 - e)~l/m : pn sup ||APn — vr|| —> 0, n -> oo. Pevm,£ Remark 4.1. Regularity is equivalent to irreducibility and aperiodicity. This can be seen as follows. Regularity obviously implies irreducibility and also that which implies aperiodicity. Conversely, irreducibility and aperiodicity together with PV> ^ PljPj]~l and Lemma 3.1(6) yield that PV> > 0 for all m large enough, which, together with finiteness, implies regularity. 4.3 Comment on the Countable Markov Case In the irreducible positive recurrent Markov jump case (and discrete-time aperiodic Markov chain case) the following holds for the classical coupling time T. Let j be an arbitrary fixed state and let Z and Z' have initial distributions A and A', respectively. If a > 0 and E[7f ] < oo, E[rja] < oo, Ej[r.,a] < oo,
Section 5. Ornstein Coupling - Random Walk on the Integers 47 then E[Ta] < oo, which yields (as above) ilAP'-A'P'H-^O, t->oo. Moreover, if 3p>l: E[pTj] < oo, E[pT'i] < oo, Ej[pTi]<oo, then there is a p > 1 such that E[pT] < oo, which yields (as above) ^IIAP'-A'P'H-^O, t->oo. This and more elaborate rate and uniformity results are established in Chapter 10 (Section 7.5). See also Section 5 in Chapter 4. 4.4 Comment on Diffusions A diffusion on [0, oo) is a continuous-time Markov process with state space [0, oo) and continuous as a function of time. Two diffusions Z, Z' have to meet before both have hit 0. Thus T < r0 V Tq holds, where T is the classical coupling time and tq, Tq the hitting times of 0. The limit results for birth and death processes were based on this inequality, and thus analogous results hold for diffusions. 5 Ornstein Coupling — Random Walk on the Integers The classical coupling need not be successful in the null recurrent and transient cases. Integer-valued random walks are either null recurrent or transient (see Remark 5.1 below). In this section we construct a coupling of such walks that is always successful, provided that the step-lengths satisfy a certain aperiodicity condition. 5.1 The Walk Let X\,X%,... be i.i.d. finite integer-valued random variables that are independent of the finite integer-valued random variable So- Put Sk = S0 + X! + --- + Xk, 0<fc<oo. Then 5 = (5fc)g° is called a random walk on the integers with step-lengths X^,X2, ■ ■ ■ and initial position So- We further assume that the step-lengths are strongly aperiodic: there is an h such that P(Xi = ft) > 0 and gcd{n 6 Z : P(Xi - h = n) > 0} = 1. Note that strong aperiodicity implies that the step-lengths are aperiodic, that is, gcd{n 6 1 : P(Xi = n) > 0} = 1.
48 Chapter 2. MARKOV CHAINS AND RANDOM WALKS The converse is not true, however. The step-lengths can be aperiodic without being strongly aperiodic. For instance, step-lengths with distribution P(Xi = 1) = P(-X"i = —1) = | are aperiodic but not strongly aperiodic. Note in this example that the difference X\ — X[ of two i.i.d. step-lengths X\ and X[ will take the values 2, 0, and -2, that is, the difference is not aperiodic. This is the reason we assume strong aperiodicity: we shall need step-length aperiodicity for the difference of two walks. Remark 5.1. Clearly, 5 is a Markov chain with state space Z. When 5 is irreducible and aperiodic, it will be either transient or null recurrent. It cannot be positive recurrent because if it were, then Theorem 3.2 would yield that 7r = (1/rrij : j £ Z) is a stationary distribution, but the expected recurrence times m.j are obviously all identical, say rrij = a, and positive recurrence would imply Y^^j = 2 l/a = °°i which contradicts X^j = 1- 5.2 Ornstein Coupling Let 5' be a differently started independent version of 5, that is, 5' is a random walk on the integers, S'k = S^ + X[ + --- + X'k, 0^fc<oo, that is independent of S and has the same step-length distribution. Let h be as above. Since X\ — h is aperiodic, there is a constant c such that X\ — h is aperiodic on {|^i — h\ ^ c}, that is, we can take c large enough so that gcd{n 6 Z : P(Xi - h = n, \XX - h\ < c) > 0} = 1. (5.1) Put S£ = S£ and for k ^ 1, x,l = [X'k ii\Xk-X'k\^c, k \Xk ii\Xk-X'k\>c. By symmetry [since (Xk,X'k) ^ (X'k,Xk)} P(Xk = n, \Xk - X'k\ < c) = P(X'k = n, \Xk - X'k\ < c), which yields the second equality in P(X'k' = n) = P(X'k = n, \Xk - X'k\ <c)+ P(Xk = n, \Xk - X'k\ > c) = P(X'k =n), n e Z. Thus the step-length distribution of the random walk 5", S'k'= SH + X^ + ■ ■ ■ + X'k\ 0^k<oo,
Section 5. Ornstein Coupling - Random Walk on the Integers 49 is the same as that of 5'. Since the initial positions are the same, 5" is a copy of 5'. The random walk R = {Rk)^ defined by Rk = Sk- S'k\ 0 < k < oo, has step-lengths Xk — Xj!, k > 1, which are symmetric and bounded: Xi - X[' = X'l - XY and \XX - X['\ < c. Further, with h as above and n 6 Z, we have P(Xi - X[' = n) > P(Xi -X'1=n, \XY -X[\^c) > P(Xi - h = n, \Xi - h\ < c, X{ = h) = P(Xi - A = n, |Xi - /iK c)P(X{ = A), which together with (5.1) and P(X[ = h) > 0 implies that R has aperiodic steplengths, that is, gcd{n 6 Z : P(Xi - X(' = n) > 0} = 1. Such a random walk is irreducible and recurrent (see next subsection), and thus K = inf{fc > 0 : Sk = Sk} = the time of the first visit of R to 0 is finite with probability one. Let 5'" be the copy of 5' that sticks to the path of 5" up to time K and then switches to 5. Then (5,5'") is a coupling of 5 and 5' with coupling time K. This together with the coupling time inequality \\P(Ske-)-P(S'ke-)\\^2P(K>k) yields the following result. Theorem 5.1. Let S be an integer-valued random walk with strongly aperiodic step-lengths. Let S' be a differently started version of S. Then there exists a successful coupling of S and S', and \\P(Ske-)-P(S'ke-)\\^o, k^oo. The above coupling of random walks was introduced by Ornstein in 1968 and is named after its inventor.
50 Chapter 2. MARKOV CHAINS AND RANDOM WALKS 5.3 The Difference Walk R is Irreducible and Recurrent Here is an elementary argument showing that an integer-valued random walk R = (-Rfc)o° witn symmetric bounded aperiodic step-lengths is irreducible and recurrent (as a Markov chain). Put R° = (RDF, where R°k = Rk - fl0- Clearly, the set A = {n G Z : P(R°k = n) > 0 for some k} is additive. Due to the step-length symmetry, A is closed under negation. Due to the step-length aperiodicity, A is aperiodic. Thus A coincides with the integers [see Lemma 3.1(a)]. Thus the Markov chain R is irreducible. In order to establish recurrence, we shall first show that P( sup |Jfc|=oo) =1. (5.2) ^0<fc<oo ' Fix an r > 0 and an n > 0 so large that p := P(fl° G [-2r, 2r}) < 1. Then, for 1 ^ k < oo, P(K € [-r,r],R°2n G [~r,r],... ,R%n G [~r,r}) ^pk. Send k —»• oo to obtain P(sup0<fc<oo |.R£n| ^ r) = 0. Since r is arbitrary, this yields P(sup0^fc<oo \Rk\ = oo) = 1. This implies (5.2). Next, note that by step-length symmetry, P( sup Rk = oo) =p( inf Rk = -oo). (5.3) ^0^k<oo ' \0^k<oo I Put M„ = sup0^fc^n Rk and M^ = sup0^fe<oo flfc. Since {Moo = oo} = {sup0<fc<oo(.Rn+fc — Rn) — oo}, it follows by the step-length independence that the events {Mx = oo} and {Mn > x} are independent and thus P(M„ > x, Moo = oo) = P(Mn > x)P(Moo = oo). Send n -» oo to obtain P(Moo = oo) = P(Moo > z)P(Moo = oo) and then x —> oo to obtain P(Moo = 00) = P(Moo = OO)2. Thus P(sup0<fc<oo Rk — oo) = 0 or 1 (example of Kolmogorov's 0-1 law). This, together with (5.3) and (5.2), shows that P( sup Rk = oo ) = 1 and P( inf Rk = -oo ) = 1. ^O^fc<oo ' \0^k<oo I Thus R changes sign infinitely often. Due to the step-length bounded- ness, there is a constant c such that R cannot change sign without visiting {0,1,..., c - 1}. Thus R visits a finite set of states infinitely often. Thus, due to irreducibility, the Markov chain R is recurrent.
Section 5. Ornstein Coupling - Random Walk on the Integers 51 Remark 5.2. Note that the proof of the infinite number of visits to [0, c) does not depend on R having integer-valued step-lengths. Thus we have established the following result. Let R be a random walk on R with symmetric step-lengths that are nonzero with positive probability and bounded by a constant c > 0 with probability one. Then R visits [0, c) with probability one. 5.4 Comment on Nonidentically Distributed Step-Lengths The Ornstein coupling can be modified to apply to step-lengths Xi, X2, ■ ■ ■ that are independent but not identically distributed and not necessarily integer valued. Assume there is an integer-valued strongly aperiodic random variable V and a p > 0 such that for each k > 1, P(Xk =n) >pP(V = n), nel. Let (this is no restriction, see Section 5 in Chapter 3) Ii,I2,... be i.i.d. 0-1 variables such that the pairs (Xi,Ii), (X2,I2), ■ • • are independent and, for each k > 1, P(Xk=n,Ik = l)=pP(V=n), neZ. Let Vi, V2, ■ ■ ■ be i.i.d. copies of V, let V\, V2, ■ ■ ■ be independent of (Xi,Ii), (X2,h), ■ ■ ■, and put, for k > 1, x, (yfc if 4 = 1, k \xk if 4 = 0. Let So and 50 be independent, integer valued, and independent of (Xi ,I\), (X2,I2),..., and Vi, V2,.... Put, for k > 1, Sk =S0 + X1 +...+Xk and S'k = 50 + X[ + ■ ■ ■ + X'k. It is no restriction to assume that V is bounded. Then Rk=Sk-S'k, 0^fc<oo, forms an integer-valued random walk with symmetric bounded aperiodic step-lengths. Thus the time K of the first visit of this random walk to 0 is finite, and we have established a successful coupling of the differently started versions 5 and 5'. Remark 5.3. Another coupling that works for step-lengths that are independent, integer valued, but not necessarily identically distributed is the Mineka coupling; see Lindvall (1992), Section 14 in Chapter 2.
52 Chapter 2. MARKOV CHAINS AND RANDOM WALKS 6 Ornstein Coupling - Recurrent Markov Chains The Ornstein argument went as follows: in order to make two random walks meet, start with independent walks and then change them by letting the step-lengths coincide when the step-length difference is large. We shall now apply this trick in the recurrent Markov case to the random walk formed by times of visits to a fixed state: let the excursions of the process between two visits coincide when the difference of the excursion-lengths is large. This yields a coupling that is successful even when the recurrence is null. Trivial modifications are needed: in continuous time to have the random walk integer valued, in discrete time to have the random walk strongly aperiodic. 6.1 Continuous Time Let Z — (Z«)s€[o)00) be an irreducible recurrent countable state space Markov jump process. Fix a state j and let Sn be the time of the (n + l)st visit to j at integer time, that is, S0=mi{keZ+:Zk=j}, and recursively for n ^ 0, 5„+i = inf{fc 6 Z+ : k > Sn and Zk = j}. That P(5„ < oo) = 1, n > 0, can be seen as follows. By recurrence the state j is entered infinitely often (with probability one). Since at each entrance to j the process starts anew, the sojourn times in state j form an i.i.d. sequence of exponential random variables, and thus (with probability one) there are infintely many sojourn times greater than 1. Each sojourn interval that is greater than 1 contains an integer, and thus (with probability one) there are infinitely many integers k such that Zk = j. Thus P(5n < oo) = 1, n >0. Clearly, 5 = (5„ ){j° is a random walk [since Z starts anew from the state j at the times 5„] on the integers, and the step-lengths Xk, k ^ 1, are strongly aperiodic because, due to (3.1), P(X1 = l) = Pj(Zi=j)>0, P{X1 =2)2 Pj(Zi = i)Pi{Z1 = j) > 0, i ± j. Let Z' be a differently started independent version of Z and define 5' in the analogous way. Then 5' is a differently started independent version of S, and due to the strong aperiodicity, the Ornstein construction in the previous section yields a successful coupling of 5 and 5'. But this does not suffice; we need a successful coupling of Z and Z'. For that purpose
Section 6. Ornstein Coupling - Recurrent Markov Chains 53 <*- u o > a o §•£ '5 a |S as. -a f independent - D 0 1 2 3 4 5 6 7 8 10 11 12 15 16 17 18 S0 St S2 Sj FIGURE 6.1. The sequence S splits Z into a delay and cycles. we introduce the concepts of delay and cycles: the increasing sequence of random times 5 splits Z into the delay D = (Zt)0^t<So (see Figure 6.1) and the sequence of cycles (excursions, blocks) Ck = (ZSk^+t)o^t<xk, 1 < k < oo. Since Z starts anew from state j at the times Sn, it follows that the cycles are i.i.d. and independent of the delay. Note that Z is uniquely determined by the delay and cycles, and in particular, we obtain So as the length of D, and Xk as the length of Ck- In the same way 5' splits Z' into a delay D' and cycles C'k that are copies of C\. Define Z" by mimicking the definition of 5" in the previous section: let Z" be the process with delay D" = D' and cycles C£, 1 < n < oo, defined by C" = Again by symmetry C" is a copy of C'n [just as in the previous section X'^ was a copy of X'n\. Since the pairs (Cn,C'n), 1 ^ n < oo, form an i.i.d. sequence that is independent of D', the cycles C£, 1 < n < oo, of Z" form an i.i.d. sequence that is independent of D" = D'. It follows that Z" is a copy of Z'. Let 5" be the sequence of integer times at which Z" is in the fixed state 3- Put K = inf{n > 0 : Sn = S£}. Since (as in Section 5.2) Rn = S„ - S£, 1 ^ n < oo, forms an integer-valued random walk with symmetric bounded aperiodic step-lengths, it follows (see Section 5.3) that K is finite with probability one, and thus so is T = SK = S'Ji. Note that the pairs (C„,C^'), 1 < n < oo, form an i.i.d. sequence that is independent of the pair (D,D'). For each k > 0, the event {K = k}
54 Chapter 2. MARKOV CHAINS AND RANDOM WALKS is determined by (D, D",Ci,C",..., Ck,C'£) and thus is independent of (Cfc+„,CJ[.'+n), 1 < n < oo. It follows that the cycles Ck+i,Ck+2,- ■■ and CK+1,CK+2, ■ ■ ■ are i.i.d. copies of C\ and independent of (D,Clt...,CK) and (D",C^...,C'k). Thus both Z and Z" start anew at time T from the state j independently of how they got there. Let Z"' be the process that sticks to the path of Z" up to time T and then switches to Z, that is, Z'" has delay D"' = D" and cycles n [Cn iin>K. Then Z'" is a copy of Z" (and thus of Z'), since both Z and Z" start anew at time T independently of how they got there. Thus (Z, Z'") is a coupling of Z and Z' with coupling time T, and since T is finite, we have obtained the following result (apply the coupling time inequality (2.1) for the latter statement). Theorem 6.1. There exists a successful coupling of two differently started versions of an irreducible recurrent countinuous-time countable state space Markov jump process. Moreover, with Pl, t ^ 0, denoting the semigroup of transitiori matrices, HAP'-A'P'H-^O, i-*oo, for all initial distributions A and A'. This result enables us to improve the convergence result in the null recurrent case (Theorem 3.1). Corollary 6.1. With k a fixed state let v be the row vector with entries defined at (2.5). Let c be a finite constant. In the null recurrent case, P(Zt 6 A) -> 0, t->oo, uniformly in subsets A of E satisfying ^2ie^ Vi ^ c. Proof. As in Section 2.7 let A' be the stationary vector v conditioned on a finite set of states B. Use (2.11) to obtain the second inequality in P(Zt e A)< £ A'P/+ ||AP'- A'P'U i£A i£B
Section 6. Ornstein Coupling - Recurrent Markov Chains 55 Thus, with sup„A<jc denoting supremum in A C E satisfying ~^2ieA Vi < c, sup P(zt eA)^c/J2"i + Wxpt ~ x'ptW ■msJc ieB -+cl2_,vii * —^ oo, (by Theorem 6.1) i£B ->0, BtE, since Eies^i -*• Y.ieEvi =mk = oo- n 6.2 Discrete Time Let Z = (Zjt)§° be an irreducible aperiodic recurrent discrete-time countable state space Markov chain. If the random walk formed by the visits to a fixed state j has strongly aperiodic step-lengths, then the argument in the continuous-time case works as it stands. If this is not the case, we proceed as follows. Let Jo,Ii,... be independent 0-1 variables such that P(Ik = 1) = \ for k > 1. Then (Zk,Ik)'o' ls an irreducible aperiodic recurrent countable state space Markov chain. Let j be a fixed state and Tj{n) be the time of the nth (re)visit of Z to j. Clearly, the set B = {k > 1 : Pj(Tj(n) = k) > 0 for some n} is aperiodic and additive. Thus by Lemma 3.1(6) there are integers n, n' and h such that both Pj(r,-(n) = ft) > 0 and Pj(r,-(n') = h + 1) > 0. This and P(i,i)(T-a,i)=*)>2-"Pi(ri(n)=fc), Ol, n>l, (6.1) yields the strong aperiodicity of the time r^i) between two (Zk,Ik)'o' visits to (j, 1). Put So = inf{fc 6 Z+ : (Zk,Ik) = (j, 1)} = tWi1) and recursively, for n > 0, 5n+1 = inf{fc 6 Z+ : k > Sn and (Zk,Ik) = (j, 1)} = r(jil)(n + 1). Then [repeating the argument starting with the second paragraph of the previous subsection with Z replaced by (Zk, h)™} there is a successful coupling ((Zk,Ik)?,(Zk",Ik")%>) of (Zfc,Jfc)8° and (Z£,I£)8°, where (Z^)°° is any differently delayed version of (Zk, 4)g°. Then (Z, Z'") is a successful coupling of (Z, Z'), and we obtain a discrete-time version of Theorem 6.1. Theorem 6.2. There exists a successful coupling of two differently started versions of an irreducible aperiodic recurrent discrete-time countable state
56 Chapter 2. MARKOV CHAINS AND RANDOM WALKS space Markov chain. Moreover, with P denoting the one-step transition matrix, \\XPn - A'Pn|| -> 0, n-+oo, for all initial distributions A and A'. As in the continuous-time case this result enables us to improve the convergence result in the null recurrent case (Theorem 3.2). The proof is the same as that of Corollary 6.1. Corollary 6.2. With k a fixed state let v be the row vector with entries defined at (3.5). Let c be a finite constant. In the null recurrent case, P(Z„ £A)->0, n -> oo, uniformly in subsets A of E satisfying ^2n^A Vi ^ c. Remark 6.1. A coupling of discrete time Markov chains that has not been mentioned here is the Vasershtein coupling (see Lindvall (1992)). It is obtained by simply maximal coupling (as in Section 4 of Chapter 1) at each step the transitions of two differently started versions Z and Z' of a discrete time Markov chain. This should not be confused with Griffeath's maximal coupling which obtains identity at all times in the coupling time inequality: ||P(Z„ € ■) - P(Z'n 6 -)ll - 2P(T >n), n> 0, see Theorem 6.1(a) in Chapter 3 and the observation at (3.1) in Chapter 6. 6.3 Aperiodicity Versus Strong Aperiodicity — Shift-Coupling Although the 0-1 variable trick in Section 6.2 results in a successful coupling of the Markov chains, it does not result in a successful coupling of the random walks (r3'(n))^=0 and (tJ(^))^=0 formed by successive visits to the fixed state j; the two walks will not merge in the end. We only obtain that there are two finite random times M and M'" [namely the random integers M and M'" such that Tj(M) = t^"{M'") = T = tUa)(K) = t^a)(K)] such that rj(M + fc) = rj"(M'"+fc), Jb > 0, that is, the random walks (rj(n)^L0 and (r'"(n)^=0 merge in the end only modulo the random time shift M — M'". Applying the 0-1 variable trick in the Ornstein coupling of integer-valued random walks S and 5' in Section 5 allows us to replace strong aperiodicity of the step-lengths by the weaker condition that the step-lengths are only aperiodic. This results in a coupling such that the walks merge only up to a random time shift (see Section 7.5 for a formal statement).
Section 7. Epsilon-Coupling - Nonlattice Random Walk 57 Such a coupling is called a successful shift-coupling. It does not yield the limit result ||P(S* e •) - P(S'k £ -)ll -> 0 as fc -> oo but only the weaker time-average (Cesaro) result n-l r«-1^P(Ske-)-n-1^P(S[£-) fc=o fc=o 0, k -» oo. We establish this and many more results about shift-coupling in Chapters 5 and 6. 7 Epsilon-Coupling - Nonlattice Random Walk We shall now apply the Ornstein approach and the 0-1 variable trick from Section 6.2 to random walks with step-lengths that do not take values exlusively in the integers or in any other lattice dlj,d> 0. We will only be able to make the random walks come e-close and stay e-close from there on; they will not merge. In fact, they will not even come e-close at the same time; they come e-close only modulo a random time shift (cf. Section 6.3 above and Section 7.7 below). In spite of these limitations we can use this coupling to prove Blackwell's renewal theorem (Section 8) and other important renewal results (Section 10). Although the arguments are now becoming familiar, we once more go through the details. 7.1 Nonlattice Random Walks A random variable X, and its distribution function F, is nonlattice if Vd>0: P(XedZ)<l. Note that a discrete random variable can be nonlattice (for instance, X can have 1 and y/2 as its only values). Say that X can be close to a point x, and that a; is a point of increase of F, if V<5 > 0 : P(X e [x - S, x + 6}) > 0 [that is, F(x - S) < F{x + 6)}. Thus X is nonlattice if and only if the points of increase of F are not all contained in the same lattice. Let S and S' be differently started independent versions of a random walk (see Section 5.1) with nonlattice step-lengths. 7.2 Merge When Difference of Geometric Sums Is Not e-Small Let Ii, I2, ... , I[, I'2, ... be i.i.d. 0-1 variables that are independent of 5 and 5' and such that P(/i = 1) = \. Put K0 = 0 and recursively for n ^ 1 Kn = ini{k > Kn-i : h = !}■
58 Chapter 2. MARKOV CHAINS AND RANDOM WALKS The Kn split the step-lengths into cycles C„ = {XKn-x+\,- ■ ■ ,Xk„), n^l, that are i.i.d. and independent of the initial position So- Let Yn be the sum over the cycle Cn, Yn = XKn_1+1+---+XKn, n>\. Define C'n and Y'n in the analogous way. Fix an e > 0 and define a new sequence of cycles C^, n ^ 1, as follows: fc; if|yn-^K£) n \cn if \Yn - y^\ > e. By symmetry [since {Cn,C'n) = (C'n,Cn)\ P(Cn e B, \Yn - yj ^ e) = P(c; g b, |Y„ - y;\ <: s) for sets B of cycle values. This yields the second equality in p(c;' g b) = p(c; g b, |y„ - y^i ^ e) + P(cn g b, |y„ - y^\ > £) = P(CnGB), n^l. Thus C'^Cn, n^l. Also, note that the pairs of cycles (Cn, C^'), n ^ 1, form an i.i.d. sequence that is independent of (So, S0). Let Y^' be the sum over the cycle C'^, that is, y„ [^ if \Yn~Y^\^s, n \y„ if |yn - y^| > e. 7.3 The Difference Walk Has Nontrivial Step-Lengths Let R = (fln)o° be the random walk with initial position So — S'0 and step-lengths Yn — Y£, n ^ 1. These are symmetric and bounded by e: Yj. - Y/' = Y{' - yx and | y - Y/' | sC e. We shall prove that P(Yi - Y{' ^ 0) > 0 [recall that e > 0 is arbitrary]. Put A — {x G K : Yi - Y/ can be close to x}.
Section 7. Epsilon-Coupling - Nonlattice Random Walk 59 Firstly, A is nonlattice, since X\ is nonlattice and if Xi can be close to x, then x is in A: for all 6 > 0, P(Y1 -Y{£[x-S,x + S\) > 2"3P(X! + X2 - X[ £ [x - 6, x + 6}) > 2-3P(Xx £ [x - 6/3, x + 6/3})3 > 0. Secondly, A is additive, since Yi - Y{ can be close to x if and only if there are k and k! such that Xi + • • • + X^ — X{ — • • • — X'k, can be close to x and since for all 6 > 0, P(Xi + • • • + Xk+n -X[ X'k,+n, £[x + y-S,x + y + 6}) > P(XX + • • • + Xfc - X[ X'k, £ [x - 6/2,x + 6/2]) P(Xj +... + Xn-X[ X'n,£[y- 6/2,y + 6/2}). Thirdly, if x £ A, then —x £ A, since Y\ - Y{ is symmetric. Fourthly, if Xk £ A and xk —> x as k —» oo, then x £ A, since for each 6 > 0 there is a k such that |xfc — x\ < 6/2 and thus, with F the distribution function of Vi -1?. F(i - <J) ^ F(xfc - 6/2) < F{xk + 6/2) <C F{x + 6). The only subset of K with these four properties is K itself (see Lemma 7.1 below). Thus Y\-Y{ can be close to e/2. Thus Yi-Y/' can be close to e/2. Thus P(Yi - Y/' ^ 0) > 0. 7.4 The Epsilon-Coupling A random walk like R with step-lengths that are symmetric, bounded by e, and nonzero with positive probability visits [0, e) with probability one (see Remark 5.2). Thus M = inf{ra > 0 : Rn £ [0,e)} is finite with probability one. Define a new sequence of cycles C^", n > 1, by switching from the C£ to the C„ after the Mth cycle: c,u = {Cl ifn^M, [Cn if n>M. The C^", n > 1, are i.i.d. copies of Ci and independent of 5q [since both CM+n, n ^ 1, and C^^, n > 1, are i.i.d. copies of Ci and independent of (So.C'i', • • • ,Cm)]- For fc > 1, let X'k" and 4" be such that [with K% the nth. k such that J£" = 1] (X^Li+1,...,X£-',„) = C", n>i.
60 Chapter 2. MARKOV CHAINS AND RANDOM WALKS The random walk 5'" with initial position S'0" = S'0 and step-lengths X{", X'-l', ... is a copy of 5', since 5'" has the same initial position as 5', since the step-lengths of S'" are obtained in the same way from the cycles C^", n > 1, as those of 5' from the cycles C'n, n > 1, and since both cycle sequences are independent of S'Q and both contain i.i.d. copies of C\. This yields the following result. Theorem 7.1. For an arbitrary e > 0, the pair (5,5'") is a coupling of the differently started nonlattice random walks S and S'. Moreover, Km and KM are finite and SKu+k-S%,»+k = RM€[0,e), k>0. Finally, Km is a randomized stopping time for S in the presence of S'Hm, that is, for each n ^ 0, the event {Km = n} and the variable S'^„, are conditionally independent of S given (S0,..., 5„); and K'm is a randomized stopping time for S'" in the presence of Skm ■ Proof. Only the randomized stopping time claim has not been proved. The event {KM=n} and the variable S'^„, are determined by (Ik)o°> (^)o°) 5', and (50, ■.. ,5n), which are independent of (Xk)^+i- Thus {Km =n), S'^,,,, and [So, ■ ■ ■, Sn) are independent of (XjOIh-i- Since 5 is determined by (So, ■ ■ ■ ,Sn) and (Xk)^+i, we obtain that {Km = n) and S'^„, are conditionally independent of 5 given (So, ■ ■ ■, Sn). In the same way we see that K'm is a randomized stopping time for 5'" in the presence of Skm ■ ^ 7.5 Analogous Result for Integer-Valued Random Walks A straightforward modification of the above argument yields the following result. Theorem 7.2. Let S and S' be differently started versions of an integer- valued random walk with aperiodic step-lengths. Then there exists a coupling (S,S') of S and S' and two finite random integers K and K' such that SK+k = S'K,+k, 0 ^ k < 00, and such that K is a randomized stopping time for S, that is, for n ^ 0, {K = n} is conditionally independent of S given (So, ■ ■ ■, Sn), and such that K' is a randomized stopping time for 5".
Section 7. Epsilon-Coupling - Nonlattice Random Walk 61 Proof. In Section 7.2 take e = 1 to obtain that Y\ — Y[' can at most take the values —1, 0, and 1. In Section 7.3 take 5 = 0 to obtain that A = {x£l: P(Xi - Y( = x) > 0} is aperiodic and additive and closed under negation. Thus A = Z by Lemma 3.1(a). Thus Yi - Y{' can take all the values —1, 0, and 1. In Section 7.4 take e = 1 to obtain the desired result. □ 7.6 The Reals Have No Proper Closed Nonlattice Subgroup In Section 7.3 we needed the following result. Lemma 7.1. Let A be a subset of M. that is nonlattice [there is no d > 0 such that dTL contains A], additive [x,y £ E implies x + y £ E], closed under negation \x £ E implies —x £ E], and closed [xk £ A and Xk —>• x as k —» oo implies x £ A}. Then A = E. Proof. Since A is nonlattice and closed under negation, A n (0, oo) is not empty. Put d = inf AD (0,oo). Then d £ A [since A is closed], and thus A contains dJL [since A is closed under addition and negation]. There is an x £ A such that x 0 dZ [since A is nonlattice], and thus d = 0 [since if d > 0 there would be an integer k such that 0 < x — kd < d, but x — kd£ A, contradicting the definition of d]. Take dk £ A n (0, oo) such that dk i d = 0. Let x be an arbitrary real number and let n* be such that n^dk ^ x < rikdk + dk- Then nud^ —> x, and thus x £ A [since n^dk £ A and A is closed]. Thus A = E. □ 7.7 Comment on Nonlattice Versus Strongly Nonlattice Call a random variable X with distribution function F strongly nonlattice if there is a point x such that X can be close to x and X — x is nonlattice. Suppose S and S' have strongly nonlattice step-lengths (instead of only nonlattice). Then the Ornstein construction in Section 5 yields a difference walk Rk — Sfc — S'l having symmetric nonlattice step-lengths bounded by some (large enough) constant c. Such a walk is e-recurrent for all e > 0 (in order to establish this we can use the result from Remark 5.2 that R visits [0, c] infinitely often: to be able to deduce that [0,e] is also visited infinitely often it only remains to show that the symmetry and nonlatticeness implies that each time R is in [0, c] it has a conditional probability greater than some strictly positive p of hitting [0,e]). Thus we obtain the result that the coupling (5,5'") of the strongly nonlattice random walks 5 and 5' satisfies SK+k - S'H+k =RK£[0,e], k> 0, where K = inf{fc > 0 : Rk £ [0,e]}. That is, the walks really come (and stay) e-close, not only modulo a random time shift.
62 Chapter 2. MARKOV CHAINS AND RANDOM WALKS 7.8 Comment on Successful Coupling Consider a random walk S with nonlattice step-lengths taking only rational values. Let S have a rational initial position and let S" be a version of S having a nonrational initial position. Then Sk is rational for all k, and S'k is nonrational for all k, and thus there can be no random integers K and K' such that Sk = S'K,. In particular, there can be no successful coupling of S and S' (that is, no K such that Sk = S'K). There are, however, nonlattice step-lengths that allow successful coupling, for instance continuous step-lengths. We shall prove this (and a more general result, 'spread-out' step-lengths) in Chapter 3 when we have introduced the most useful 'conditioning' and 'splitting' techniques. 8 Epsilon-Coupling - Blackwell's Renewal Theorem Informally, Blackwell's renewal theorem can be stated as follows. A room is lit by one light bulb. When it burns out, a new one is installed immediately. Then, as time passes, the expected number of light bulb installations in a time interval of length h will tend to h/m, where m is the expected life length of a light bulb. This is one of those intuitively obvious facts with no elementary proof. Or so it seemed for many years until a coupling proof emerged. Even the coupling argument was not fully elementary at the begining but has little by little been refined down to the acceptably elementary form presented in the previous section. Below we introduce renewal terminology, state the theorem, and complete the proof. 8.1 The Renewal Process A renewal process S = (Sfc)^0 is a random walk with nonnegative initial position and strictly positive step-lengths, that is, So is a nonnegative random variable and Sk = So+Xi +---+Xk, 0 O < oo, where Xi,X2, ... are i.i.d. strictly positive random variables that are independent of So. It is customary to think of the Sk as times when something happens. Thus in renewal context we refer to the k in Sk as index, not as time. Call So [the time when the first light bulb is installed] the delay time and denote its distribution function by G. Call X\, X2, ■ ■ ■ [the life lengths of the successive light bulbs] the recurrence times and denote their common distribution function by F. Say that a renewal takes place at time Sit [the time of the (k + l)th light bulb installation]. Put m = E[Xi] = the mean recurrence time.
Section 8. Epsilon-Coupling - BlackwelPs Renewal Theorem 63 Let N(t, t + h] denote the number of renewals in the time interval (t, t + h], N(t, t + h]:= #{k ^0:t<Sk^t + h}, t, ft > 0. Thus N(t, t + h) = Nt+h - Nt, where (see Figure 8.1) Nt '■= inf {k ^ 0 : Sk > t} = the number of renewals in [0, i\. o o o o -—• •—• • • • ► o s0 s, s, s3 s4 s5 I <«— x, —*• h-x,-*i -<—x3—*-\+ x4 *-h-x5-*i FIGURE 8.1. The renewal process S and some associated random variables. Take an x > 0 such that F(x) < 1 and let Ln be the nth k > 1 such that Xk > x. Then Ln is the sum of n geometric variables with parameter 1 - F(x), and thus E[Ln] = ra/(l - F(x)) < oo. Since Sk > xl{Xl>x} H 1- a;l{Xfc>x}, we have Nt ^ inf{fc : l{x!>x} H 1" !{xfc>x} > V1} = ^[t/sl+i. and thus E[Nt] < oo, t > 0. (8.1) If S0 = 0 identically, then S is zero-delayed. In particular, 5 has the zero- delayed version S° := (Sk - 50)C° = (X1 + ■ ■ ■ + Xfc)~. The event {Nt— n} is determined by (So, ■ ■ •, S„) and thus is independent of Xn+i, Xn+2, This implies that Xn,'+i,Xn,+2, ■ ■ ■ are i.i.d. copies of Xi. Thus N[SN,,SNt + ft], the number of renewals in [Sn,, SWt + ft], is a copy of N%, which together with N(t, t + ft] ^ N[SNt, SNt + ft] yields N(t,t + h]^N%, £,ft>0. (8.2) In order not to have the renewal process stuck in a lattice dZ we need the assumption that F is nonlattice [recall the definition: P(Xi £ dZ) < 1 for all d > 0].
64 Chapter 2. MARKOV CHAINS AND RANDOM WALKS 8.2 Blackwell's Renewal Theorem - Idea of Proof We are now ready to state Blackwell's renewal theorem. [Sharpened versions of this result can be found in Chapter 3 (Theorem 6.2) and Chapter 10 (Theoerem 7.6).] Theorem 8.1. If F is nonlattice, then lim E[N(t, t + h]]- h/m [= 0 if m = oo] for all delay time distributions G- The obvious coupling approach to proving this theorem would be [station- arity part] to look for a version S' with a delay distribution G' making E[iV'(£, t + h]] = h/m (close to 0 when m = oo) for all t and then [coupling part] construct a coupling such that the two processes have common renewals from some random time T onwards. It turns out that e-close renewals (Theorem 7.1) suffice; everything works out fine by sending e 4- 0 in the end. We carry out the proof in the next four subsections. 8.3 Stationarity Part of the Proof The following is the key stationarity result (we use the corollaries). Theorem 8.2. For the zero-delayed renewal counting process N° we have f Jo (1 - F{x))E[N°_x] dx = t, 0^t<oo. PROOF. Note that (5°+1 - Xi)^ is a copy of S°. This yields the distributional identity in oo oo oo Nt-x - 22 1{S°n<it-x} = 2^ l{S»-X!^t-i} = 2^ l{x+S°-X!^t}- Since Xi is independent of Yln=\ ^{x+s^~Xi^.t}^ this yields oo (1 - F(x))E{N?_x] = E[l{Xl>x} J2 Mx+sz-x^t}} • n=l Integrate over x (and move out sum and expectation) to obtain /■oo °° r pXi / (l-F(x))E[N^x]dx = Y,V \ l{x+s^Ximdx\. Jo n=1 lJo J
Section 8. Epsilon-Coupling - Blackwell's Renewal Theorem 65 Variable substitution yields the first equality in r°° °° r rs* / (l-F(x))E[N?_x}dx = J2v\ W> J° „=i lJs?t-x1 dy = 5^E[5°At-(5°-X1)At' n=l oo = £(E[S° At]- E[5°_! A t]) (since (5° - X>) £ S°_J n=l = lim E[5° At]- E[5n A t] (telescoping sum) n—>oo = t, (S° A t increases to t as n —» oo and 5q A t = 0) and the theorem is established. □ When m < oo define a distribution function Goo by Goo(x) = E[Xi A x]/m, 0 ^ x < oo. Corollary 8.1. If m < oo, t/ten Gqo /tas density (1 — F)/m and G = Goo => E[JV(i,i + ft]] = ft/m, t,h>0. Proof. For 0 ^ i ^ oo we have Xi A x = J"0X l{Xi>j/} dy, which yields the first equality in H[X1Ax]=E[[ l{Xl>y}dy] Jo = [Xp(X1>y)dy= [\l-F(y))dy. Jo Jo This shows that Goo has density (1 — F)/m. Take G = Goo and use this density to obtain the second equality in E[Nt] = E[Nt°_So] = m_1 / (1 - F(x))E[N?_Jdx = t/m (by Theorem 8.2) Jo and thus E[N(t, t + h]] = E[JVt+h] - E[iVt] = ft/m as desired. □ For 0 < a < oo, define a distribution function Ga by = JE[Xi A x]/E[Xi A o], 0 < i <C o, 1 1, a ^. x < oo.
66 Chapter 2. MARKOV CHAINS AND RANDOM WALKS Corollary 8.2. For a < oo, Ga has density (1 — F)l[0,a]/E[Xi A a] and G = Ga =>• E[N(t,t + h]] ^h/E[Xl A a], t,h>0. Proof. The first display in the proof of Corollary 8.1 shows that Ga has the desired density. Take G = Ga and use this density to obtain the second equality in E[N(t,t + h}} = E[JV°+ft_5o] - E[N°_So] = E[XX A a]"1 f (1 - F(x))(E[N°+h_x] - E[iV°_J) dx Jo /•oo ^ E[XX A a]"1 / (1 - F(i))(E[^.J - E[JV°_J) di Jo = E[Xi A a]"1 ((t + h) -t) (due to Theorem 8.2) and thus E[N(t, t + h]] ^ A/E[Jf i A a] as desired. D Note that h/E[Xi Ao] 4- ft/m as a —» oo, and thus Corollary 8.2 implies that we can choose the delay time distribution G' of N' so that E[N'(t,t + h]] is close to zero uniformly in t when m = oo. 8.4 Coupling Part of the Proof The proof of Theorem 8.1 is based on Theorem 7.1. Let N"' be the counting process associated with S'" and put T - Skm ■ In the time interval [T, oo) the renewals of S stay at distance Rm ahead of those of 5"', and thus t>T =>• iV"'(i-RM,* + ft-fiM] = iV(i,i + /i]. Now, Rm £ [0,e), and thus for e < h, N'"(t, t + h- e]l{Tm ^ N(t, t + h]l[T<t} ^ N'"{t -e,t + h]. Subtract the term (N'"(t,t + h] - N'"(t,t + h - e])l{r>t} on tlie left and add the term N(t,t + h]l^r>t} m tne middle and on the right to obtain (after taking expectations and using N"' = N') the coupling inequality E[N'(t, t + h-e]]- E[N"'(t, t + h}l{T>t}} <^E[N{t,t + h}} (8.3) ^ E[N'(t -e,t + h]) + E[N{t,t + h)l{T>t}).
Section 8. Epsilon-Coupling - Blackwell's Renewal Theorem 67 8.5 Completing the Proof When m < oo When m < cxd put G' = Goo and subtract E[N'(t,t + h]} = h/m on all three sides of (8.3) to obtain \E[N(t,t + h]]-h/m\ <; e/m + E[N'"(t, t + h}l{T>t}] + E[N(t, t + h]l{T>t}]. Both N'"(t,t + h]l{T>t} and N(t,t + h]l{T>t} tend to 0 with probability one as t —>• oo and both are [see (8.1) and (8.2)] stochastically dominated by the finite mean random variable N%. Thus by dominated convergence (see Corollary 9.1 in Chapter 1), E[N"'{t,t + h]l{T>t}]-+0, t-+oo, E[N(t,t + h]l{T>t}]->0, *-^co, (8.4) which yields lim sup |E[iV(t, t + h]]- h/m\ < e/m. t—too Sending e 4- 0 yields the desired result when m < oo. 8.6 Completing the Proof When m = oo Whenm = oo put G' =Ga and apply E[N'(t-e, t + h]] < (h + e)/E[Xi Aa] to the second inequality in (8.3) to obtain E[N(t, t + h]}^(h + e)/E[Xl A a] + E[N(t, t + h}l{T>t}}. Send t —»• oo and use (8.4) to obtain the inequality in lim sup E[N(t, t + h]]^(h + e)/E[Xi Aa]->Oaso-^oo, t—>oo while the limit result is due to monotone convergence. This completes the proof of Blackwell's renewal theorem. 8.7 Comment on the Two-Sided Case If we drop the condition that the Xn are strictly positive and only assume (in addition to the nonlatticeness) that the random walk S has positive drift, that is, 0 < m := E[Xi] < oo, then Theorem 8.1 still holds. This can be established along the above lines (since Theorem 7.1 holds for any nonlattice random walk) with the following modifications.
68 Chapter 2. MARKOV CHAINS AND RANDOM WALKS Note that Nt ■■= inf{fc ^ 0 : 5fc > 0 no longer equals N[0,t]:=#{k>0:0^Sk^t}. Both are, however, finite with probability one due to the strong law of large numbers, which says that P(Sfc/fc —> m as k —>• oo) = 1 and thus (since m > 0) P(Sfc -» oo as k -» oo) = 1. In fact, E[iVt] < oo and E[iV(-oo, ft]] < oo; see, for instance, Asmussen (1987). Moreover, (SN,+k — Sn,)^Lo 1S a zero-delayed version of S, and (t,t + h] C (—oo, Sn, + h], which yields, instead of (8.2), that N(t,t + h] ^N°{-oo,h}. This allows us to use dominated convergence in Section 8.5. Finally, the stationarity results hold if we replace (1 - F(x))E[N^_x] by P(S°No > x)E[N°(-x,t - x}} in Theorem 8.2 and redefine Ga(x) = E[S%S A a A x]/E[S%. A a], 0 ^ x < oo, Goo(:r) = E[5^. A i]/E[5^.], 0 ^ x < oo. 9 Renewal Processes — Stationarity We shall now establish a full-fledged stationarity result for renewal processes using a method that differs from the one in Section 8.3, where we only established a minimal stationarity result in order to prove Blackwell's renewal theorem. The result below will be used in the next section together with epsilon-coupling to obtain asymptotic stationarity for nonlattice renewal processes. The constructive approach of this section will be used in Chapter 8 to develop a general theory (Palm theory) on the relation between stationary processes and processes consisting of a stationary sequence of cycles. In Chapter 9 we move on with the same idea to processes with d-dimensional time parameter (random fields). 9.1 Structure of the Stationary Version - Intuitive Motivation Let 5 be a renewal process with m < oo (see Section 8.1). Let Xq be a random variable such that Xo ^ So and such that the pair (Xo,5o) is independent of Xi,X2, If we think of 5 as a sequence of times when a light bulb burns out and is replaced by a new one, then So can be seen as the residual life of the light bulb in use just before time 0, and Xo as its
Section 9. Renewal Processes - Stationarity 69 total life. Put S-i = S0 - X0. For t ^ 0, put At =t - Sn,-i Bt = SNt-t Dt = XNt =At + Bt Ut = At/Dt age at time t, residual life at time t, total life at time t, relative age at time t; see Figure 9.1. FIGURE 9.1. The age, residual life, total life, and relative age processes.
70 Chapter 2. MARKOV CHAINS AND RANDOM WALKS In order to get an intuitive feeling for the stationary version of the Markov process (As,Bs,Ds,Us)se[o,oo), suppose F has a density /, consider a zero- delayed renewal process and make the thought experiment of selecting a time r uniformly at random in [0, oo). Then the Markov process from time r onward should be stationary. Further, the relative position of r in the renewal interval where it landed should be uniform. Finally, the probability of r being in an interval of length x should be proportional to xf(x)dx because (due to the law of large numbers) the relative number of intervals of length x is f(x) dx and the probability of landing in a particular interval of length x is proportional to x (this length-biasing , the fact that an interval selected in this manner is longer than an ordinary interval, is called the inspection paradox or waiting time paradox). Thus a stationary version should be obtained by placing the origin uniformly in the initial interval after choosing its length Xo according to the density xf(x)/m, that is, according to the distribution function F00(x) = E[X1l{Xl^x})]/m. Note that if Xo has this distribution, then for nonnegative (Borel measurable) functions g, E[g(X0)} = E[Xl5(X!)]/m. (9.1) 9.2 The Stationarity Theorem - Proof We shall now prove that the above guesswork is correct. Theorem 9.1. Let S be a renewal process with finite mean recurrence times and delay time So = (1 — Uq)Xq, where Xo has the distribution F^ and is independent of (Xi,X2, ■ ■ ■) and Uq is uniform on (0,1) and independent of(Xo, Xi, X2, ■ ■ ■ )■ Then (AS,BS,DS,US) se[o,oo) «s stationary and S0 = (1 - Uo)X0 and - S-i = U0X0 both have the distribution function Goo from Section 8.3. PROOF. Since U0 and (1 — U0) are both uniform on [0,1] and both independent of Xo, it holds that So and — S_i have the same distribution, and the common distribution function is Goo, since P(U0Xo ^x) = P(U0^ (Xo A x)/Xo) = E[(X0Aa;)/Xo] = E[Xi A x]/m (due to (9.1)) = G00(x). (by definition)
Section 9. Renewal Processes - Stationarity 71 Note that the process Z° = (^s:-Bs>-Ds't/s°)s6[0,oo) is determined by (X\, X2, X3,...) in the same way as the process Z = (As_l+S, Bs_1+S, Ds_1+S, Us_1+S)se[o,oo) by (X0,Xi,X2,...)- Since (X2,X3!.-.) = (^i>^2,...), (9-2) this yields the first equality in 1 r-Xa+t E[T0It l^dS Xq = x r 1 rXi+t = E -an Xi=x l{z;<*}d* Xi=x , 0 < x < 00, z € With z fixed call this expression g(x) and apply (9.1) to obtain Efei l{Z^}dS = E r-x'i+t /" 1+ 1 / / l{z,°<z}dsj/m. (9.3) Since J70 is independent of Z and since (At, Bt, Dt, Ut) = Zu0x0+t, we have 1 [Xa+t in r 1 rXo+t P((At,Bt,Dt,Ut)<z) = E[TJ l{z.$z}d8 Combining this and (9.3) yields the first identity mP{{At,Bt,Dt,Ut)^z) = E r *{z;&}ds fx1+t + E / • l{z°^z}ds lJXi = E[Jt Mzs^yds = E[| ' l{z.<z}da]+n[J l{z^z}ds\ (by (9.2)) Thus the distribution of (At,Bt,Dt,Ut) does not depend on t. Since the process (AS,BS,DS, ^s)o<s<<x> is Markovian, this implies stationarity. □
72 Chapter 2. MARKOV CHAINS AND RANDOM WALKS 10 Renewal Processes - Asymptotic Stationarity The e-coupling constructed in Section 7 can be used for more than Black- well's renewal theorem. Below we apply it together with Theorem 9.1 to obtain total variation convergence of the total life and convergence in distribution of the age, the remaining life and the relative age, in the nonlattice case. The lattice case is dealt with at the end of the section using the results for Markov chains from Section 3. We start with a general coupling result. 10.1 Epsilon-Coupling and Piecewise Constant Processes Let Z — (Zs)se[o,oo) an<l Z' = (^s)s6[o,oo) be two continuous-time stochastic processes. For e > 0, say that (Z,Z',T,T') is a successful e-coupling of Z and Z' if (Z, Z') is a coupling of Z and Z', and T and T' are finite random times such that Zt+s = Z'T,+S, 0 ^ s < oo, \T-T'\ ^e. A stochastic process Z is stationary if (Z(+S)s6[o,oo) = Z, 0 ^ t < 00. Theorem 10.1. Let Z and Z' be piecewise constant right-continuous processes with finitely many jumps in finite time intervals. Suppose Z' is stationary and there is for each e > 0 a successful e-coupling of Z and Z'. Then Zt -* Zqi * "*■ °°- PROOF. Let Z and Z' be an e-coupling (0 < e < 1) of Z and Z' with finite times T and T". From the piecewise constancy we deduce that for t > 1, Zt = Z't on C = {TVT' ^t, no Z' jump in [t-e,t + e]}. Thus (by definition) C is a coupling event of the coupling (Zt,Z't) of Zt and Zj, and the coupling event inequality (5.11) in Chapter 1 yields ||P(zte-)-P(^e-)ll ^ 2P(T V T' > t) + 2P(Z' jumps in [t-e,t + e]), since Cc - {T\/T > i}\J {Z' jumps in [t- e,t+e]}. Use the stationarity of Z' and Z' = Z' to obtain ||P(ZtG-)-P(^G-)ll ^ 2P(T V T > t) + 2P(Z' jumps in [1 - e, 1 + e]).
Section 10. Renewal Processes - Asymptotic Stationarity 73 Sending t —>• oo yields limsup||P(Zt e-)-P(Zo g-)II ^ 2P(Z'jumps in [l-e,l+e]). t—>oo The right-hand side tends to P(Z' jumps at 1) as e 4- 0, and thus limsup||P(Zt £-)-P(Zo G-)ll < 2P(Z'jumps at 1). t—>oo Let V be uniform on [1,2] and independent of Z' and use the stationarity of Z' to obtain P(Z' jumps at 1) = P(Z' jumps at V"). Since there are only finitely many jumps in finite intervals and V is uniform and independent of Z', we have P(Z' jumps at V) = 0. Thus limsup ||P(zte-)-P(Zoe 011 = 0, t—>oo and the proof is complete. □ 10.2 Nonlattice Renewal Processes We now use Theorems 7.1, 9.1, and 10.1 to establish asymptotic stationarity (see Sections 8.1 and 9.1 for the definitions of S, At, Bt, Dt, and Ut). Theorem 10.2. Let S be a renewal process with nonlattice finite mean recurrence times. Let Y and U be independent random variables, Y with distribution Fqq and U uniform on [0,1). Then Dt%Y, Ut%U, both At and Bt 4 UY, P{Ut ^u,Dt < x) -tuF^x), we [0,1], i€[0,oo), P(At >x,Bt >y) -> 1-Goo(i+j/), x,y e [0,oo), as t —> oo. Proof. Theorem 7.1 yields a successful e-coupling of the stochastic process (AS,BS,DS, Us)se[o,oo) and its stationary version (Theorem 9.1) and thus of each of the piecewise constant processes [0,oo)i (l{C/^u,£)^x})se[0,oo), (l{^,>x,BJ>y})s6[0,oo)j and their stationary versions. A reference to Theorem 10.1 yields Dt % Y and P{Ut <u,Dt^x) -+P(U^u,Y^x), P{At >x,Bt>y)-+ P(UY > x, (1 - U)Y > y).
74 Chapter 2. MARKOV CHAINS AND RANDOM WALKS Since P(UY > x) = 1 - G00(x), it only remains to check that P{UY >x,(l- U)Y >y) = P(UY >x + y). This follows from the independence of U and Y and the following [condition or\Y = z and take a = x/z and b = y/z]: P{U > a, (1 - U) > b) = P(a < U < 1 - 6) = P(C/>a + 6), a, 6^0, and we are through. □ Remark 10.1. If the recurrence times are continuous (or, more generally, spread out), then the above convergence in distribution can be replaced by convergence in total variation, since then the successful epsilon coupling can be replaced by a successful (exact) coupling; see Theorem 6.1 in Chapter 3. Remark 10.2. Since the events {At < x}, {Bt < x}, and {Dt < x} are all contained in {N[t — x, t + x] > 0} and since P(N[t - x,t + x] > 0) < E[JV(t -2x,t + x}}, it follows from Blackwell's renewal theorem (Theorem 8.1) that for every 0 ^ x < oo, m = oo P(At < x) -> 0, t-*oo, P(Bt < i) -> 0, t -> oo, P(D( < x) -> 0, t-^-oo. But P(C/f ^ u) should still go to u, or what? I don't know. 10.3 Integer-Valued Renewal Processes The lattice version of Theorem 10.2 is easily deduced from Theorem 3.2. Without loss of generality we can take the span of the lattice dZ+ to be one, d = 1. The result for general d follows by change of scale, that is, by replacing 5 by (Sk/d)f=0. Theorem 10.3. Let S be an integer-valued renewal process with aperiodic recurrence times, that is, P(Xi € nZ+) < 1 for all n > 1. If m < oo, then for all integers i > j ^ 0, P(Dn = i,An = j) -*• P(Xi = i)/m, n -> oo. If m = oo, then for all integers i and j, P(£>„ = i, An = j) -» 0, n -> oo.
Section 10. Renewal Processes - Asymptotic Stationarity 75 Proof. Apply Theorem 3.2 to ((Dn,An) : n ^ 0), which is an irreducible aperiodic recurrent Markov chain with state space E = {(i,j):P(Xl=i)>0, i>j>0}. The time between two ((£>„, An) : n ^ 0) visits to (i,j) € E is of the form X\ H + Xk, where K is the first k such that Xk = i. Since Kis geometric with parameter P(Xi =t), we have E[K] = l/P(Xi=i). Since the event {K < k} is determined by X\,... ,Xk-i, it is independent of Xk- Thus the expected time between two (Dn, An)g° visits to (i,j) is m/P(Xi = i), due to the following lemma. □ Lemma 10.1. Let K be a nonnegative random variable. Then E[K] = P{K ^ 1) + P{K ^ 2) + ... . Further, if X\, X2, ... are nonnegative random variables such that for all k ^ 1 it holds that E[Xk] = E[Xi] and that the event {K < k} is independent of Xk, then E[Xi +...+XK] = E[K]E[Xi] (Wald's identity). Proof. The first claim follows from K = 1{k^i} + ^{k^2} + The second claim follows from E[Xi +---+XK] = E[X!l{x^1}] + E[X21{K^2}] + ... = E[Xl}E[l{K^1}] + E[X2}E[l{K^2}}} + ... = E[X1}(P(K>1)+P(K>2) + ...) = EiX^EiK], where the second equality is due to {K ^ k} being the complement of {K < k] and thus independent of Xk, and the third equality is due to E[Xk] = E[Xi]. D 10.4 More on Integer-Valued Renewal Processes The limit result in the finite mean case, m < oo, can be stated as follows. As time passes the total life becomes length-biased, that is, for integers i such that P(Xi = i) > 0, P{Dn = i)/P{Xi = i) -> i/m, n -> oo, and the age becomes uniformly distributed on the total life, that is, for integers i > j ^ 0 such that P(Xi = i) > 0, P{An = j\Dn = i) -> 1/t, n -^ oo. Results for the residual life Bn are easily deduced from Theorem 10.3 because Bn = Dn — An. In particular, we have the following results.
76 Chapter 2. MARKOV CHAINS AND RANDOM WALKS Corollary 10.1. Let S be an integer-valued renewal process with aperiodic recurrence times. Then for all integers J ^ 0 and k ^ 1, P(^n = j, Bn = k) -> P(Xi = j + k)/m, n -> oo, P{Bn =j + l)= P{An = j) -> P(Xi > j)/m, n -»• oo. The lattice version of Blackwell's renewal theorem follows immediately from the last observation by noting that N{n} := #{fc > 0 : 5fc = n) = \{An=0} and thus E[iV{n}] = P(An = 0): Corollary 10.2. // 5 is an integer-valued renewal process with aperiodic recurrence times, then E[N{n}] -> 1/m, n -^ oo. Summary This chapter started by presenting the classical coupling (Sections 2, 3, 4). We showed that it is successful for irreducible recurrent birth and death processes and for irreducible (aperiodic in the discrete-time case) positive recurrent Markov chains. Then the Ornstein coupling was introduced (Sections 5 and 6). We showed that it is successful for integer-valued random walks with strongly aperiodic step-lengths without any recurrence condition and then applied it to construct a successful coupling of irreducible (aperiodic in the discrete-time case) null recurrent Markov chains. Finally (Sections 7 through 10), the Ornstein idea was used to construct successful epsilon-couplings of random walks with nonlattice step-lengths. When applied to renewal processes, this rendered Blackwell's renewal theorem and several other results on asymptotic stationarity. * * * This ends the introductory part of the book. The next five chapters present a general coupling theory: Let me take you down 'cause I'm going to Strawberry Fields, Nothing is real
Chapter 3 RANDOM ELEMENTS 1 Introduction This chapter consists of two parts: Sections 2-5 and Sections 6-10. The first part introduces general coupling tools, and the second part generalizes some of the results from Chapters 1 and 2. After a measure-theoretic review of terminology in Section 2, Section 3 explains what is meant by extending the underlying probability space and collects some extension techniques. Sections 4 and 5 are devoted to particularly important extension techniques: conditioning, transfer, and splitting. These sections may seem rather technical, but the extension methods are quite probabilistic in nature, and are used frequently throughout the book. In Section 6, transfer and splitting are used to construct a successful coupling of random walks with step-lengths that are spread out (continuous step-lengths are a special case of spread out). In Section 7, splitting is used to construct a maximal coupling of an arbitrary collection of random elements. In Section 8, we consider the special case of two random elements and formulate the maximal coupling result in terms of total variation. In Section 9, splitting is used to turn lim inf convergence of distributions to a distribution into a pointwise convergence where the random elements actually hit the limit and stay there. In Section 10, we use transfer (and Theorem 6.1 in Chapter 1) to turn convergence in distribution into pointwise convergence for random elements in a separable metric space (Dudley's extension of the Skorohod coupling), and then re-prove this result in the case when the space is also complete (the Skorohod coupling) after generalizing of the quantile coupling. 77
78 Chapter 3. RANDOM ELEMENTS 2 Back to Basics - Definition of Coupling A random element in a measurable space (E,£) defined on a probability space (ft, J7, P) is a measurable mapping Y from (ft, J7, P) to (E,£), that is, {Y G A} 6 T, A G £, where {Y G A} := {w G ft : Y(w) G A} =: Y~l A. We also say that Y is supported by (ft, J7, P) and that Y is an ,F/£ measurable mapping from ft to E. Note that if we replace P by another probability measure Q, then Y is the same measurable mapping but a different random element. Also, if we replace J7 by a, larger a-algebra and/or £ by a smaller, then Y is the same mapping but not the same measurable mapping. If we replace T by a smaller cr-algebra and/or £ by a larger, then Y need not even be measurable. A random variable X is a random element in (R, B), where E denotes the set of real numbers (the line) and B its Borel subsets, that is, B is the smallest cr-algebra on R containing the open sets (the cr-algebra generated or induced by the open sets), B = B{R) = a{A C E : A open}. An extended random variable X is a random element in ([-oo,oo],B([-oo,oo])). When the line is regarded as time, we often call an extended random variable T a random time. If T cannot take the values — oo and oo, then T is a finite random time. The distribution of a random element Y [under P] is the probability measure on (E,£) induced by Y, namely PY"-1. Since P(Y G A) =PY~lA, A££, a more probabilistic notation for the distribution of Y is P(V G •). A random element Y is canonical if Y is the identity mapping, that is, if (ft, J7) = {E, £) and Y(u) = u, u G ft. ThenP(Y G ■) = P- A random element Y in {E,£) defined on a probability space (ft,^", P) is a copy or representation of Y if P(Y g-) =P(YG-); this is denoted by Y = Y. A random element Y has always a canonical representation, the canonical random element on (E,£,P(Y G ■)).
Section 2. Back to Basics - Definition of Coupling 79 2.1 Coupling Random Elements — Definition For each i in an index set I let Yi be a random element in a measurable space (Ei,£i) defined on a probability space (fij,.^,?;). A family of random elements (Y : i G I) defined on a common probability space {(l,^, P) is a coupling of Yi, i € I, if Yi^Y for each j£l. Note that the Yi need not be defined on a common probability space, in other words need not have a joint distribution. Thus 'coupling' can be seen to refer to the fact that the copies % are defined on a common probability space, have a joint distribution, live together. Writing (Y : i £ I) in parentheses indicates that this is the case. For any collection of random elements Yi, i € I, there is always at least one coupling, the independence coupling , consisting of independent copies of the Yi. This follows from the product measure theorem (Fact 3.1 below). 2.2 Coupling Probability Measures — Rephrasing the Definition In terms of distributions the definition of coupling can be rephrased as follows. For each i in an index set I let Pi be a probability measure on a measurable space (Ei,£i). Define the product space 0(^i,fO:=(II£;*'0f') i€l i€l iel where Y\ieI Ei is the Cartesian product of the Ei, YI Ei := {y = {yi : i £ I) : Vi £Ei,i£ I}, and 0ieI £t is the product cr-algebra, that is, the smallest cr-algebra on Yli^jEi making the ith projection mapping taking y in Yliei^i to ^i Vlx Ei measurable for all i £ I (the cr-algebra generated or induced by the projection mappings): (g) Si := a{{y : y{ G A} : i G I and A G Si}. iei A probability measure P on (&i€i(Ei,Si) is a coupling of Pj, i G I, if Pj is the ith marginal of P, that is, if Pi is induced by the ith projection mapping: P({y.yi&A}) = Pi(A), AeSh iel.
80 Chapter 3. RANDOM ELEMENTS 2.3 The Relation Between the two Formulations The latter definition of coupling can be seen as a canonical version of the former by the following identification: let (Ei,£i,Pi) be the probability spaces supporting the canonical copies of the individual random elements Yi, that is, Pi = Pi{Yi € ■), » € I, and let (fliei ^i) ®iei &> P) be the probability space supporting the canonical copy of the coupling (Y* : i € I), that is, P = P((£ :<€!)€■). Note that here we treat the expression (Yi : i € I) not as a collection of individual random elements in (Ei,£i), i € I, but as a single random element in 0ieI(.Ej,£j) defined by {Yi-.iG I)((j) := (Yi{u>) : i € I), w € ft. The distribution of this random element, P((li : i € I) € •), is the joint distribution of the Yt, i € I. 3 Extension Techniques Finding random elements with particular properties (having a particular joint distribution with those already introduced) can be an essential task in constructing couplings. These new random elements are often brought into existence by extension, by extending the underlying probability space. In this section, and Sections 4 and 5, we give several ways of doing this. Let us start by making precise what we mean by extension. 3.1 Extending the Underlying Probability Space - Definition A probability space (£l,!F, P) is an extension of another probability space (ft, T, P) if (Q,^", P) supports a random element £ in (fi, J") having P as distribution. If Y is a random element in {E,£) denned on (fi, T, P), then the random element Y defined on (A,^, P) by Y{Q) = Y{£(Q)), Q£&, (see Figure 3.1) is a copy of Y, since for A € £, P(Y £A) = P((erU) = P(Y~1A) =P(y € A).
Section 3. Extension Techniques 81 (fi,J-,P)i »{E,£) (ft.-F.P) FIGURE 3.1. The original random element Y induced by Y. Say that Y is induced by Y and call Y an orginal random element. Thus, in particular, £ is the original random element induced by the canonical random element on (ft, T, P). In addition to the original random elements (which we shall think of as 'old') the probability space (A,^,P) may support 'new' random elements not induced by random elements already supported by (ft, T, P). These 'new' random elements we shall call external. Convention of the common probability space. When there is no risk of confusion, we often extend the underlying probability space (ft, T, P) without changing its name: after extending (ft, T, P) to obtain (Cl,!F,P) we rename the extension (ft, J7, P), and the induced Y we rename Y. This identification of Y and Y explains the term 'original' for Y. This procedure enables us to assume, when convenient, that all the random elements to be considered in a certain context are defined on a common probability space, which we then denote by (ft,T,P). Call this the convention of the common probability space. Typically, in probability theory, it is not the actual underlying probability space that matters but the (joint) distributions of the random elements under consideration. It is, however, crucial that new random elements be introduced in a consistent manner and, as the example in Section 10 of Chapter 1 shows, we must be careful here. For the rest of this section, and in the next two, we give several safe ways of introducing new random elements. But first we consider at some length an extension that does not yield any new random elements. 3.2 Reduction Extension - Deleting a Null Event Let (ft, T, P) be a probability space. An element lj € ft is an outcome and a set A € T is an event. If P(A) = 0, then A is a null event. If P(A) = 1, then A is an almost sure (a.s.) event. Any statement that holds on an a.s. event (that is, for all outcomes in the event) is an a.s. statement. It is common practice to remove a null event (or a set contained in a null event, an outer null set) from the underlying probability space, thereby Y
82 Chapter 3. RANDOM ELEMENTS getting rid of some unpleasant outcomes (turning some a.s. statement into a pointwise one). Although this is certainly a reduction of the space, it is in fact an extension in the above sense of the word. This can be seen as follows. Let fi be a subset of Cl of inner probability one, that is, containing an almost sure event A. (Note that Cl has inner probability one if and only if its compliment, which we are going to delete, is an outer null set.) Reduction extension. Define f, P, and £ by T := JFn Cl : = {B D Cl : B € T), [the trace of Cl on T ], P(Bnfi):=P(5), -BeJT, (3.1) £(u>) := u>, Q G ft. Note that P is well-defined on T because if B nCl = B' nft, then BC\A = B' n A, and thus P(B) = P(B n A) = P{B' n A) = P(B'). The above reduction is an extension, since {£ € B} = B D fi, and thus P({GB)=P(B), B£J, that is, £ has distribution P, as desired. We can remove a countable number of null events, since their union is a null event, but not an uncountable number unless, of course, we know that their union is a null event. It is also worth noting that deleting a null event is measure dependent: a null event with respect to P need not be a null event with respect to another probability measure Q unless, for instance, Q has density with respect to P (which is the same as saying that Q is absolutely continuous with respect to P, that is, the null events of P are also null events of Q). 3.3 Reduction Extension — Deleting an Inner Null Set We shall now show that the above reduction extension (3.1) can be generalized to a set Cl with outer probability one, that is, to a set Cl $ T such that P(A) = 1 for all A e F that contain Cl. Note that Cl has outer probability one if and only if its complement is an inner null set, that is, if and only if the complement of Cl has inner probability zero: P(A) = 0 for all A £ T contained in the complement of Cl.
Section 3. Extension Techniques 83 Thus reduction to a set of outer probability one is the same as deleting an inner null set. In order to show that this is allowed we must check that if (ft, T, P) is a probability space and A is a subset of ft with outer probability one, then P as defined at (3.1) is well-defined on T. This follows by noting that if B and B' are two sets in T such that B n fi = B' (1 fl, then the events B\B' and B' \B are both in the complement of ft, and thus both have probability zero, that is, P(B) - P(-B'). Thus Bnft = B'nft implies P(J5 D fl) = P(B' n ft), that is, P is well-defined. The reduction extension cannot be generalized beyond sets ft of outer probability one because the complement of such a set would contain a nonnull event, and thus deleting it would result in loss of mass. We must be careful when deleting two or more inner null sets because the union of two inner null sets need not be an inner null set, as the following trivial example shows. Consider T = {0, ft} where ft has more than one element; let P be the 0-1 measure. Then any nonempty proper subset A of ft is an inner null set, since it contains only one element of J-', namely 0, which is a null event. But ft \ A is also an inner null set. Thus if we first delete A and then ft \ A, we have deleted all of ft. The lesson to be drawn from this example is that after deleting an inner null set the next subset to be deleted must be an inner null set of the reduced space (in the example ft\ A is not a null set of {0, ft \ A}). This is never the case when the second set is the complement of the first as in the above example. Also, more care must be shown when deleting a single inner null set than when deleting a single null event. For instance, if Y is a random element in (E,£), A € £, and P(Y G A) = 0, then the event {Y G A} is a null event and may be deleted. But what if A g £ and P(F G B) = 0 for all B G £ contained in A? Then, in general, {Y G A} cannot be deleted, because although A is an inner null set with respect to P(Y G •), the set {Y G A} need not have inner measure zero with respect to P, as the following trivial example shows. Let ft = {a, &}, T = {0, {a}, {&}, {a, &}}, and P({a}) = P({6}) = \. Let Y be the random element in (ft,{0,ft}) defined by Y(a) = a and Y(b) = b. Then A = {a} is an inner null set with respect to the 0-1 measure P(Y G •) on (ft, {0,ft}). But {Y G A} = {a}, and thus {Y e A} is an event with positive probability P(Y G A) = \, that is, {Y £ A} is not an inner null set with respect to P. However, although not allowed in general, we shall in the next subsection consider an important case where {Y G A} can be deleted if A is an inner null set with respect to P(Y G •). This is the case when (ft, T) is the product of two measurable spaces and Y is the projection on the second space. This is a particularly important case because all our remaining extensions (all the proper extensions, the product space extensions) yield a new random element Y satisfying this condition and reduction is often carried out at the end of such an extension (see Remark 4.1 below).
84 Chapter 3. RANDOM ELEMENTS 3.4 Deleting an Inner Null Set of a Product Space Component Let P be a probability measure on the product space (fi,.F) = (ni,.Fi)®(n2,Jr2) and let Y be the random element in {Sl2,J-2) defined on (ft,T) by Y(loi,u2) = u2 [Y is the projection of (ft,T) on (ft2,fa)]- Let G be a subset of ft2 of outer probability one with respect to P(Y € •). We shall show that the set {Y g G} = fti x Gc can be deleted from (ft, T). [Since the reduction extension cannot be stretched beyond subsets ft of outer probability one, this of course means implicitly that fti x G has in fact outer probability one with respect to P.] Put ft := Q,i x G and define fa P and £ as at (3.1). We must prove that P is well-defined on T. First consider product sets: if B\ and B[ are in fa, and B2 and B'2 are in fa, and Bi x (B2 D G) = B[ x (B2 n G), then ^ = B[ and both B2 \ B2 and B2 \ B2 are in the complement of G, which implies P(Y € B2 \ B'2) = P(r € B2 \ B2) = 0. Hence \P{B, x B2) - P(B[ x B2)| = |P(5! x (B2 \ B'2)) - P(Bl x (B'2 \ B2))\ ^ P{Y e B2 \ B'2) + P(Y e B'2 \ B2) = 0. Thus Bj x (B2nG) = B[ x (-B2nG) implies P(Bi xB2) = P(B[ x52), that is, P is well-defined on T\ x (T2 PlG). Since P is a probability measure, it follows from this that P as denned at (3.1) is a well-defined probability measure on the algebra of all finite unions of disjoint sets in T\ x (J"2nG). Thus, by the Caratheodory extension theorem (see Ash (1972), Theorem 1.3.10), P extends from this algebra to a unique probability measure (denote it by P) on f. Now, P(f £BiX B2) = P(B! x (B2 n G)) = P{Bl xB2), Bx£Ty, B2 £T2, and thus £ has distribution P under P, as desired. This together with {£ e A} = An ft finally yields thatP satisfies P(A n ft) = P{A) for all Aef, that is, P is well-defined on T by (3.1).
Section 3. Extension Techniques 85 3.5 Independence Extension In Chapters 1 and 2 we freely introduced new independent random elements without changing the name of the previously introduced elements or the probability measure. Although this could (together with deleting a null event) be named the 'tool of the unconscious probabilist', it is in line with the convention of the common probability space and is allowed due to the following product measure theorem. Fact 3.1. Let I be an arbitrary index set. For each i€l let Pi be a probability measure on a measurable space (Ei,£i). Then there exists a unique probability measure 0iel Pi on ®;6i(-E'i,£i) such that ^^({y&YlEi-.yi, £Ah,...,yin € Ain}) iel iel = Pi1(Ail)---Pin(Ain) for all integers n > 0, all i\,... ,i„ € I, and all A^ E S^,..., Ain € £ jn . For a proof, see Halmos (1950), Section 38, Theorem B and comment (2). (When I is countable, then Fact 3.1 is a consequence of Fact 4.3 below; comment (2) in Halmos takes care of generalization to an arbitrary I.) The measure ®iej Pi is called the product measure and ®(Ei,£i,Pi) := (l[Ei,(g)£i,(g)Pi) iei iei iei iei the product probability space. Suppose (ft, T, P) is the probability space we wish to extend to support independent random elements Y;, i € I, that are independent of the random elements already supported by (ft, T, P). If Yt is to be a random element in (Ei,£i) and to have distribution Pt, then this is achieved by putting (n.^.P) := (H,^,P) ®^{Ei,£uPi), iei £(uj,y) :=uj, weft, yeJjEi, Yi(u,y) := y{, i€l, weft, y £^\Ej. jei Call an external random element Yi obtained in this way independent (it is independent of the original random elements). 3.6 Consistency Extension If we wish to introduce dependent random elements, some restrictions are needed. A measurable space (E,£) is Polish if there exists a metric on E
86 Chapter 3. RANDOM ELEMENTS such that E is complete (each Cauchy sequence converges to a limit in E) and separable (E has a countable dense subset) and such that £ is generated by the open sets. If our (fi, !F) is Polish, then we can introduce any collection of random elements, provided that they take values in Polish spaces and that the proposed finite-dimensional distributions are internally consistent and consistent with P. This is due to the Kolmogorov extension theorem. Fact 3.2. For each i in an arbitrary index set I let (Ei,£i) be Polish. Assume that for each finite nonempty subset J of I we are given a probability measure Pj on ®ipj{Ei,£i). Assume that the Pj are consistent, that is, if K is a subset of J, then Pj({(yi :i€J)€Y[Ei:(yi:i€K)€ A}) = PK(A), A G (g) £. Then there exists a unique probability measure P on 0i6l(.Ej,£i) such that for all J P({(»i : i G I) G Y[Ei : (Vi : i G J) G A}) = Pj(A), A G ® fc. i€/ i€J For a proof, see Ash (1972), Section 4.4.3. The extension of (fi, T, P) is analogous to the one in the previous subsection. We shall not use this extension much because the methods in the next two sections fit our purposes better, in particular by not demanding that (fi,.F) be Polish. 4 Conditioning — Transfer In this section we consider an extension technique that is particularly useful for coupling purposes. The idea is straightforward: if the value of some random element is y, then we introduce a new random element with conditional distribution depending on y. Let us first recall some properties of conditional distributions before giving the restriction needed for this to be allowed. 4.1 Conditional Distribution — Regularity — Probability Kernel Let (fi, J-,P) be a probability space supporting the random variable X. If X is nonnegative, then the expectation (or expected value, or mean) of X is E[X]:= fxdl>= f X{u)P{dj).
Section 4. Conditioning - Transfer 87 If X is not nonnegative, then the expectation of X is E[X] := E[X+] - E[X~], provided that either E[X+] < oo or E[X~] < oo. Let Q be a sub-cr-algebra of T. If X has a well-defined expectation, then the conditional expectation of X given Q is the a.s. unique Q/B{[— oo, oo]) measurable function E[X|<?] from ft to [-00,00] satisfying f E[X\g\{u)P{du) = [ X{u)P(du), A eg, J A J A that is, E[E[X\g]lA] = E[X1A], A €Q. Here a.s. unique means that two functions with this property are a.s. identical, they are called versions of E[X|£]. If Y\ is a random element in {E\,S\) supported by (ft,T,P), then the conditional expectation of X given Y\ is the a.s. unique cr(Yi)/B([— 00,00]) measurable function E[X\Y1]:=E[X\a(Y1)] while the conditional expectation of X given the value of Y\ is the a.s. unique £i/B([—00,00]) measurable function E[X|Yi = •] satisfying / E[X|yi = y]P(Y1 g dy) = E[Xl{Yl€B}], BgS,. Jb A cr(Vi)/B([—00,00]) measurable function h is a version of E[X|Fi = •] if and only if h(Yi) is a version of E[X|Fi]. The conditional probability of an event A € T given g, given Y\, or given the value of Y\, is P{A\YX) V{A\Y, = ■) = E[U|r1], = E{1A\Y1=.], respectively. If Y2 is a random element in (E2,S2) supported by (ft,T,P) and we Pick a particular version of P(Y2 G B\Yi = •) for each B £ £2, then P(^2 G -\Y\ = ■) is the conditional distribution of Y2 given the value of Yi. Another pick results in a different version of P(Y2 € -|Vi = •)• The random element Y2 is conditionally independent of another random element Y0 given Yi if P(r2 G -|^i = 0 is a version of P(Y2 G MY^Yo) = (-,•))•
88 Chapter 3. RANDOM ELEMENTS If we consider P(Y2 € B\Yi = y) as a function of B keeping y fixed, then it need not be a probability measure for all y. If P(>2 € -|Yi = y) is a probability measure for each y € E\, then we say that the conditional distribution V{Y2 € -\Y\ = •) is regular. Obviously there exists a regular version of P(l*2 G -|Vi = •) if Y\ is discrete, that is, if E\ is finite or countable and S\ the power set of E\ containing all its subsets. A more useful condition is given by the following theorem, where the condition is placed on Y2 rather than Yi. Two measurable spaces {E,£) and {G,G) are Borel equivalent (or Borel isomorphic) if there exists a bijection (that is, an invertible mapping) / from E to G such that / is £/Q measurable and its inverse f~l is Q/£ measurable. The bijection / is a Borel equivalence. A measurable space {E,£) is a standard space if it is Borel equivalent to (G,G), where G is a Borel subset of [0,1] and Q are the Borel subsets of G. Fact 4.1. There exists a regular version ofP(Y2 6 -|Yi = •) if (£2,£2) is a standard space. Any Polish space is a standard space. For a proof, see Ash (1972), Theorem 6.6.5 and Problem 8 in Section 4.4 (and the solution on pages 442-443). A function Q(-, •) from E\ x £2 to [0,1] is an ((Ei,£i), (E2,£2)) probability kernel if Q(-,A) is £i/B([0,1]) measurable for each A e £2 and Q{y, ■) is a probability measure on (E2,£2) for each y € E\. Thus P(l2 € -|Yi = •) is regular if and only if it is a probability kernel. 4.2 Conditioning In Suppose we need the existence of a random element Y2 having a particular conditional distribution given the value of another random element Y\. Then the underlying probability space can be extended to support Y2, provided that the proposed conditional distribution is regular, is a probability kernel. This is due to the following theorem. Fact 4.2. Let Pi be a probability measure on (Ei,£i) and let Q(-, ■) be an ((Ei,£i), (E2,£2)) probability kernel. Then there exists a unique probability measure P on (Ei,£i) ® (E2,£2) such that P(AlxA2)= f Q{yi,A2)Pi{dyi), A^£x, A2 € £2. For a proof, see Ash (1972), Section 2.6.2. We can now condition in a new random element as follows. Let Yi be a random element in (Ei,£i) defined on (Q,?,P) and let Q{-,-) be an
Section 4. Conditioning - Transfer 89 ((Ei,£i),{E2,£2)) probability kernel. Note that Q{Yl (•),•) is an ((fi, T), {E2,£2)) probability kernel. Conditioning extension. Define T, P, £, Y, and Y2 by ($,?):= (n,F)®(E2,£2), P{AxB):= [ Q{Y1(uj),B)P{(Lj), A£T, B e £2, J A £(u,y):=u, u € 0, y £ E2, Yl{u,y):=Yl{u), lo € ft, y G E2, y2(w,y) :=y, wen, !/e£2- Theorem 4.1. T/ie conditional distribution ofY2 given Y\ is Q(Yi,-) or, equivalently, for y € E\, the conditional distribution ofY2 given Y\ = y is Q{y,-)- Moreover, ifYo is a random element defined on (fi, T', P) and YQ its induced copy, then Y2 is conditionally independent of Yo given Y\. Proof. With B € £2 and A e £\ ® £0, P(Y2 € B,(Yl,Y0) € A) = P({(yi,lo) & A} x B) = E{l{iYl,yo)eA}Q(Y1,B)}. Thus [since (Y\,Yo) is a copy of (Yi, Yo)] P(Y2 € B,(YUY0) EA) = E[l{(PliP2)eA}Q(Yi,B)], that is, Q(Yi,-) is a version of P(Y2 € -|Yi, Y0), which yields the desired results. □ Call an external random element Y2 obtained by the conditioning extension conditional. 4.3 Conditioning in Countably Many Times It is clear that the conditioning extension can be repeated finitely many times. It is not as clear, however, that it can be repeated countably many times, but this is in fact true due to the Ionescu Tulcea theorem.
90 Chapter 3. RANDOM ELEMENTS Fact 4.3. Let (E\,£i), (E2,£2), • ■ • be a sequence of measurable spaces. Let Pi be a probability measure on (Ei,£i) and let, for 2 ^ n < oo, Q„{-,-) be an ( (^) (Ei,£i),(En,£n)\ probability kernel. Then there exists a unique probability measure P on ®Ki<00{Ei,£i) such that P{AY x---xAnx Y[ Ei) n<i<oo = / (/ (•••(/ Qn{{yi,---,yn-i),dyn))---)Q2{yi,dy2))Pi{dyl) JA\ J A2 J An for all finite n ^ 2 and all Al € £j,..., An € £n. For a proof, see Ash (1972), Section 2.7.2. 4.4 On Conditional Independence Conditional independence comes up naturally when conditioning in a new random element, and since it can be quite tricky to handle, we shall have a look here at some of its basic properties. Let Y0,Yi,... be random elements in (E0,£o)> (Ei,£i), ■ ■ ■, respectively, defined on a probability space (n,J",P). Interpret the statements below about P(-|lo) to mean that there is a version of P(-|Y0) sucn that the statements hold. Say that Y\ and Yi are conditionally independent given Ya if P(Yi e -,Y2 e -\Y0) = P(Yi € -\Y0)P{Y2 £-\Y0), (4.1) that is, for all bounded £i/B measurable functions ft, i = 0,1,2, it holds that V[fo(Y0)f1(Y1)f2(Y2)}=E[fo(Y0)E{f1(Y1)\Y0}E[f2(Y2)\Y0}}. (4.2) This is equivalent to p(y2 e -|y0,Fi) = p(f2 e -|io), (4.3) that is, for all bounded £i/B measurable functions fi,i = 0,1,2, it holds that E[f0(Y0)fl(Yl)f2(Y2)]=E[f0(Y0)f1(Y1)E[f2(Y2)\Yo}}. (4.4) In order to establish this equivalence note that the left-hand sides of (4.2) and (4.4) coincide, and so do the right-hand sides, since E[/o(io)E[/1(yi)|yo]E[/2(y2)|r0]] = E[E[/0(y0)/i(Vi)E[/2(y2)|yo]in]] = E[fo{Y0)f1(Y1)E[f2(Y2)\Y0]].
Section 4. Conditioning - Transfer 91 Due to (4.3), we also say (as in Section 4.1) that Y2 is conditionally independent of Y\ given Yo, or that Y2 depends on Y\ only through Yq. It is clear from (4.1) that conditional independence is symmetric in Yi and Y2. Note that in (4.4) we may replace /o(Y0)/i(Yi) by g(Y0,Yi) where g is any bounded S0 ®£\/B measurable function. Lemma 4.1. The statement Y3 depends on Y2 only through (Yi,Y0) (4.5) and on Yi only through Yo is equivalent to the statement Y3 depends on (Y2, Yi) only through Y0. (4.6) Proof. If (4.5) holds then P(Y3 € -\Y2, Yi, Yo) = P(Y3 G -\YU Y0) = P(Y3 G -\Y0), that is, (4.6) holds. Conversely, if (4.6) holds then so does the latter part of (4.5) since P(Y3 € -in, Y0) = E[P(Y3 e -\Y2, Yu Y0)\Ylt Y0] = P(Y3 G -|^o), which in turn yields the second equality in P(Y3 € -|Y2, Yu Y0) = P(Y3 € -\Y0) = P(Y3 € -^i, Y0), that is, the first part of (4.5) holds also. □ A random element Y\ and an event A are conditionally independent given a random element Y0 if Y\ and 1^ are conditionally independent given Y0. This implies that Y\ and Ac are conditionally independent given Y0. Two random elements Y\ and Y2 are conditionally independent given an event A, P{A) > 0, if P(Yi £-,Y2£-\A)= P(Yi € -|A)P(y2 € -\A). Note that this does not imply that Y\ and Y2 are conditionally independent given Ac. Conditional independence extends to more than two random elements as follows: Yi,..., Yn are conditionally independent given Yq if P(Yi €•,...,Yn€-|Y0)=P(Y1 €-|Yo')---P(Yn€-|Yo). This is equivalent to any subfamily (Yj : i £ I) being conditionally independent of the rest (Yj : i £ I) given Y0. Countably many random elements Yi,Y2,... are conditionally independent given Y0 if Y\,...,Yn are so for each n. Finally, Yi,..., Yn (or Y\, Y2,... ) are conditionally i.i.d. given Y0 if they are conditionally independent given Y0 and the conditional distribution of Yi given the value of Y0 has a version that does not depend on i.
92 Chapter 3. RANDOM ELEMENTS 4.5 Typical Application - Transfer The conditioning extension is often applied in the following situation. Let Y\ be a random element in (Ei,E\) defined on (fi,T,P) and suppose we have managed to construct a pair (Y{,Y2) on some probability space (fi',.F',P') where Y2 is a random element in some measurable space (£2,£2) and Y[ is a random element in (2?i,£i) such that Further, suppose there exists a regular version Q(-, •) of the conditional distribution of Y2 given Y{ (according to Fact 4.1 this holds in particular when (£^,£2) is Polish, or standard, which is the main reason why we sometimes assume Polishness). Then we can transfer Y2 to (fl,,?-,P) as follows. Theorem 4.2. With (Y{,Y2) as above the conditioning extension yields a random element Y2 such that (yuy2)Z(y;,y±), and given Y\ the external random element Y2 is conditionally independent of any original random element Yq. This transfer procedure can be repeated countably many times. Proof. This follows immediately from Theorem 4.1 except the final claim, which is due to Fact 4.3. □ Thus if we are working with Y0 and Y\ defined on (fi,.F,P), then by the common probability space convention we could have taken (fi,.F,P) large enough to support a random element Y2 such that (Y\,Y2) is a copy of (Y{,Y2) and such that, given Yi, the random element Y2 is conditionally independent of Y0- Call such an external random element Y2 transferred {Y2' has been 'transferred' from (fi',.F,P') to (fi,.F,P)). We give an application of this transfer approach in Sections 5, 6, and 10, and then repeatedly in the next chapters. Remark 4.1. In the next chapters (see Section 2.12 of Chapter 4) we will have use for the following immediate consequence of the reduction result in Section 3.4 above [which is applicable because Y2 is the projection of (n,T)®(E2,£2)on(E2,S2)}: If Y2 takes values in a subset G of E2 or, more generally, if G has outer measure one with respect to P'^' € ■), then we may assume that Y2 takes values in G. This can be a useful observation because we may need a random element Y2 in a space (G,Q), and although there is not a regular version of the
Section 5. Splitting 93 conditional distribution of Y2' given Y{ when Y£ is regarded as a random element in (G, Q), it may exist when Y"2' is regarded as a random element in a larger space (£2,£2), where E2 is a set containing G, and £2 is such that Q = £2 H G. Typically, in applications G is not an element of £2- 5 Splitting Consider the following example. Let X be a continuous random variable with density / and suppose / > pg where 0 < p < 1 and g is a density. We would then like to say that X has density g with probability p. But how are we going to tell whether X is governed by g or not? We can do so if g and h := (/ - pg)/(l — p) have disjoint supports, that is, if there is a Borel set B such that g = 0 outside B and h = 0 on B. When X is in B then X has density g (see Figure 5.1). (l-p)ft The set B FIGURE 5.1. If X falls on the left-hand side then it is governed by g. More generally, we can tell whether X is governed by the density g or not whenever X can be split as follows: there are random variables /, V, and W such that X = IV + {1- I)W, where / is a 0-1 variable with P(7 = 1) = p and, conditionally on J = 1, the variable V has density g.li I = 1, then X = V, so X has density g. When this is not the case (see Figure 5.2 on the next page), we cannot tell whether g is governed by g or not unless we extend the underlying probability space bringing into existence such a 0-1 variable i\ We do this below for general random elements and then prove a more general splitting result.
94 Chapter 3. RANDOM ELEMENTS PS FIGURE 5.2. When is N(0,1) uniform on [-1,1] ? 5.1 Splitting Indicator Let Y be a random element in a measurable space {E,£) defined on a probability space (fi, J7,P). Let v be a subprobability measure on {E,£) with mass \\u\\ = i/(E). Suppose 0 < \\u\\ < 1 and v is a component or part of the distribution of Y, that is, P(F £-)^v (short for P(Y € A) ^ v{A) for all A € £). Let I' ,V, W be independent random elements defined on some probability space (fi', JF',P') with distributions P'(/' = 1) = ||i/|| and P'(/' = 0) = 1 - ||i/||, P'(V'G-) = "/IIHI, P'(W €.) = (P(Y€.)-*,)/(!-H). Then the random element Y' defined by Y' = if I' = 1, if J' = 0, certainly is a copy of Y. Call Y' a splitting representation of V. Due to Theorem 4.2 and Y' = Y we can now transfer /' to (fi, T, P) to obtain a 0-1 variable / such that (Y,I) — (Y',I'), that is, P(J = 1) and P(Y e-|J= 1) = W Call / a splitting indicator. After this splitting extension we can tell when Y is governed by *VIMI: when / = 1, then the conditional distribution of Y is ^/||^||. Moreover, if Y\ is another random element that was supported by (fi, T, P) before the splitting, then Y\ is conditionally independent of / given Y. In particular, if Y\ is independent of Y, then Y\ is also independent of (Y, I). Due to Theorem 4.2, splitting can be repeated countably many times.
Section 5. Splitting 95 5.2 Generalization Beyond a Single Component Let 1/1,1/2,- ■ ■ be subprobability measures on {E,£) and suppose P(Y £•) = J/j +l/2 + .... Then clearly an analogous argument to the one in the previous subsection yields an extension of ($l,T, P) supporting a nonnegative integer-valued random variable K (a splitting variable) that tells us which of the components Vi, 1 ^ i < oo, governs the distribution of Y, that is, P(K = t) = INI, P(Y g -\K = i) = i/i/\\i/i\\ (arbitrary when H^H = 0). More generally, let Y\ and Y2 be random elements in (E\,£\) and (E2,£2), respectively, defined on a probability space (fi, T, P). Let /i be a probability measure on a Polish space (E3,£3), let v{-, ■) be an ((E3,£3), (E2,£2)) probability kernel, and suppose P(Y2 G A) = /" i/(i, A)/i(di), A € £2. Then (fi, JF, P) can be extended to support a random element Y3 (a splitting element) in (£3,£3) having distribution /i and such that P(Y2£-\Y3 = y) = l/(y,-), y € E3. Furthermore, Y\ is conditionally independent of Y3 given Y2, and in particular, if Yi is independent of Y2, then Y\ is independent of (Y2,Y3). This generalized splitting can be repeated countably many times. It is a special case of Theorem 5.1 below (take Yq nonrandom). 5.3 Conditional Splitting In the next section we shall need to be able to split conditionally on the value of a random element Y0. Theorem 5.1. LetY0, Y\, andY2 be random elements in (E0,£0), {E\,£\), and (E2,£2), respectively, supported by a probability space (il,J-, P). Let {E3,£3) be a Polish space and /i(-, •) be an ((Eo, £0), (E3, £3)) probability kernel. Let v{-,-) be an {{E0,£o)®(E3,£3),{E2,£2)) probability kernel and suppose, for yQ € EQ and A £ £2, that P(Y2 £ A\Y0 =y0) = J v{(y0, y3), A)^(y0,dy3). (5.1) Then (fi, T, P) can be extended to support a random element Y3 in (E3,£3) such that for y0 € E0 and y3 € E3, P(Y3e-\Y0 = y0) = ti(y0,-), P(^2 € -\Y0 = y0,Y3 = y3) = v({y0,y3),-). (5.2)
96 Chapter 3. RANDOM ELEMENTS Moreover, Y\ is conditionally independent ofY3 given (Yb,l^), and in particular, ifY\ is independent of [Yq,Y2), thenYi is independent of (lo,^2,^3) • Conditional splitting can be repeated countably many times. Proof. Due to Fact 4.2, there is a probability space (ft', T', P') supporting random elements yo', Y2', and F3' such that for A0 € £0, A2 € £2, and A3 € £3, p'(y0'€^0,y2'€^2,y3'€^3) r r (5-3) = / (/ "{{yo,y3),A2)n{yo,dy3))P{Y0 £dy0). J A0 J A3 Take A3 = E3 and compare with (5.1) to see that (Y0',Y2') is a copy of {Yo,Y2). Since (E3,£3) is Polish, this allows us to transfer (Theorem 4.2) Y3' to (ft,T,P) to obtain a random element Y3 such that (lo^,^) is a copy of (y0',r2',y3'). Thus p(y2 e -\y0 = -,y3 = -) = p'(y2' e -|y0' = -,y3' = •)• Due to (5.3), the right-hand side equals ^((-,-)j') and (5.2) follows. Theorem 4.2 also yields that Y\ is conditionally independent of Y3 given (Y0, Y2). This in turn yields the independence claim. Finally, due to Theorem 4.2, this type of extension can be carried out countably many times. □ In Chapter 10 (Section 4.5 on Harris chains) we need the following conditional version of the 0-1 variable splitting in Section 5.1 above. Corollary 5.1. LetYo, Y\, andY2 be random elements in some measurable spaces (Eo,£q), {E\,£\), and (.E^,^), respectively, supported by a probability space (ft, T, P). Suppose P(y2 € -\Yo = •) has a regular version Q{-; ■) and there is a subprobability measure v on (£^,£2) such that 0 < ||j/|| < 1 and Q{yo, 0 >v, y0£E0. (5.4) Then (Cl,J-,P) can be extended to support a 0-1 variable I such that for y0 £ E0 and y2 £ E2, Y\ is conditionally independent of I given (yo,y2), (5.5a) P(j = i|y0=»o) = IHI, (5-56) P(y2G-|yo = 0o,-T=l) = jAj, (5.5c) P(f = l|y0=y0, Y2=y2)= V{dy^ (5.5rf) Q{yo,dy2) This conditional splitting can be repeated countably many times.
Section 5. Splitting 97 Proof. In Theorem 5.1 take E3 = {0,1} and, for y0 £ E0, Myo,{i}) = IMI and Myo,{o}) = i-|HI, "((2/o, 1), 0 = Tj^jj and i/((y„, 0), •) = t _ ^ to obtain (5.1) from (5.4). This yields the desired results (take I = Y3) except (5.5rf). In order to obtain (5.5d), take A0 £ £q and A2 £ £2 and deduce from (5.56) and (5.5c) that P(y0 € A0,Y2 £A2,I = 1) = P(y0 e A))K^2)- Combine this and v{dy2) II J An J A A0 Ja2 Q{yo,dy2) u{dy2) II JAo J A; P(Y0 edy0,Y2 edy2) P{Y0 £dy0)Q(y0,dy2) Ia2 Q{Vo,dy2) = 11 P(Vo € dy0)p{dy2) J An J A2 = P(y0 e a0)p{a2) to obtain P(Y0£A0,Y2 £A2,I = l) v{dy2) -II J An J A: P(Y0£dy0,Y2£dy2). Ia0Ja2 Q(Vo,dy2) This yields (5.5rf). □ 5.4 Review of Splitting - Brownian Motion As a review of this section, consider the example of a standard Wiener process (Brownian motion), namely, a real-valued Markov process (VFs)se[0,oo) with continuous paths, W0 = 0, and stationary independent increments. Then for each t £ [0,oo), Wt is normal with mean 0 and variance t. In particular, W\ is N(0,1); see Figure 5.2. According to the first subsection (take Y — W\), we can introduce a splitting indicator I such that 7 = 1 indicates that W\ is uniform on [-1,1]. The reason why this is more useful than a splitting representation (V, /'), where Y' is only a copy of W\ and not W\ itself, is that by introducing I we can split without losing the process (Ws)se[0iOOy This means, for instance, that we can repeatedly split in the same process: first split W\, then W2, and so on. This yields a sequence of splitting
98 Chapter 3. RANDOM ELEMENTS indicators 7i,/2,... such that for each n ^ 1, 7n = 1 indicates that the N(0,n) variable Wn is uniform on [—1,1]. Moreover, suppose we allow Wq to be a random variable taking values in a bounded interval [a, 6]. Then the conditional distributions of W\ given W0 = x, x € [a, 6], have a common uniform component. Thus according to Corollary 5.1 (take Y0 = W0 and Y2 = W\), we can split W\ such that I is independent of W0, and such that given I = 1, W\ is uniform on [—1,1] and independent of Wq. 6 Random Walk with Spread-Out Step-Lengths In this section we apply splitting and transfer to the coupling of random walks. At the end of the section the coupling result is used to sharpen Blackwell's renewal theorem. Let S = (5fc)g° be a random walk on the line, that is, Sk =S0 + Xl +--- + Xk, 00<oo, where the step-lengths Xi, X2,... are i.i.d. finite random variables that are independent of the initial position S0. Let S' be a differently started version of S, that is, let S' be a random walk with the same step-length distribution as 5. In Chapter 2 we showed in the lattice case (for integer-valued walks with strongly aperiodic step-lengths, Theorem 5.1 of Chapter 2) that there exists a successful coupling of S and 5', that is, a coupling (5, S') of S and 5', and a finite random integer K such that Sn = S'n, n ^ K. In the nonlattice case (when the step-lengths are strongly nonlattice, Section 7.7 in Chapter 2) we only managed to obtain that for each e > 0, there is a coupling (5, S') of S and S' and a finite random integer K such \Sn - S'n\ = \SK -S'K\^e, n> K. In this section we shall establish that a successful coupling actually exists in the nonlattice case, provided that we assume that the step-lengths are spread owf, namely, provided that there exists an integer r ^ 1 and a nonnegative Borel measurable function / such that J*R / > 0 and P(Jfi+ ■■■ + *,.€ A) £ //, A£B. (6.1) J A Theorem 6.1. Let S and S' be differently started versions of a random walk with spread-out step-lengths. Then there exists a successful coupling (S,S') ofS andS', and ||P(5„G-)-P(S;e-)H-^0, m-kx),
Section 6. Random Walk with Spread-Out Step-Lengths 99 where || • || denotes the total variation norm. Moreover, the coupling time K is a randomized stopping time for both S and S'. We prove this coupling result in the next four subsections. (The limit result follows in the standard way; see Chapter 2, Section 2.3, or the next chapter, Theorem 5.1. And the randomized stopping time claim follows in the same way as in the proof of Theorem 7.1 in Chapter 2.) 6.1 Key Idea of Proof - Uniform Step-Lengths Suppose (6.1) holds with r = 1 and / = l[0,2]/2, that is, the step-lengths are uniformly distributed on [0,2]. Suppose also that S0 = 1 and S'0 = 0. Let S" be the copy of S' with Stf — 0 and fcth step-length defined by „ .= [Xk + 1 if Xk^ 1, * ' \Xk-l \iXk> 1. Then Rk := Sk — SJj.', 0 ^ k < oo, forms an integer-valued random walk with symmetric aperiodic bounded step-lengths, and thus (see Chapter 2, Section 5.3) M : = inf{fc ^ 0 : Rk = 0} = inf{fc ^ 0 : Sk = S'£} is finite with probability one. Define a copy S"' of 5' by switching from S" to S at time M, g.^is- if n<M, " \Sn if n^M, (and delete the null set {M = oo}) to obtain that (5, S'") is a successful coupling of S and S' with time K = M. 6.2 Splitting Part of Proof - Step-Lengths with Uniform Part Now suppose (6.1) holds with r = 1 and / = c\a,b] f°r some constants a < b and c > 0. Clearly, (6.1) still holds if we replace [a,b] by the subinterval [a, a + 2d(s, s')], where s and s' are any real numbers and d(s,s') '■= sup{x € [0, (6 - a)/2] : \s — s'\/x is an integer}. By recursive conditional splitting (see Section 5) extend the underlying probability space to support 0-1 variables h, I2,... such that given (50,5q), the pairs (X\, /i), (X2, h), ■■■ are conditionally i.i.d. and such that for each k ^ 1 and all real s and s', P(Ik = l\(S0,S'0) = (s,s')) = 2cd{s<sl)
100 Chapter 3. RANDOM ELEMENTS and, given (S0,SQ,Ik) = (s, s',1), the conditional distribution of Xk is uniform on [a, a + 2d(s>s/)]. Let S" be the copy of S' with S0' = S0 and fcth step-length defined by (Xk if Jfc = 0, X£ := < Xk + d{So,S'0) if 4 = 1 and Xk^a + d{So,s'0), [Xk - rf(s0,s^) if h = 1 and Xfc > a + rf(s0,s^)- Conditionally on (S0,S0'), fl*:=(S*-S£')Ms„,sj'). 0<*<oo, forms an integer-valued random walk with symmetric aperiodic bounded step-lengths. Thus M := mi{k > 0 : Rk = 0} = inf{fc ^ 0 : Sk = S£} is finite with probability one. Now note that both IM+1,Ijf+2,... and X^f+1,X^f+2,... are sequences of i.i.d. copies of X\ and that both sequences are independent of (So, S",..., S'm). Since Sm = S^, this means that we again obtain a copy S"' of S' by switching from S" to S at time -R" = M, that is, we have again established the existence of a successful coupling. 6.3 Transfer Part of Proof - Uniform Part After r Steps Now allow r > 1 but still assume that / = cl[a b]. Then the random walk (Skr)kxL0 with initial position S0 and fcth step-length Lk '■= -X"(fc-l)r+l + • • ■ + Xkr has a uniform component in one step. Proceed as in the previous subsection to obtain a copy L'k' of Lk such that given (So,S0), Lk — L'k is symmetric and takes the values 0 and ±d(s0,s") with positive probabilities, and (-X"(fc-i)r+i, • • • ,Xkr,Lk,Lk), k ^ 1, is a conditionally i.i.d. sequence. Since L'fc' is a copy of Lk, we can recursively apply the transfer extension (Theorem 4.2) to obtain (-X"(/t_i)r+1, • • • :Xkr) such that -XV'fc-D,..!, • • •, Xkr are i.i.d. copies of X\ with sum L'fc' and, given (S0,S0), (-X"(fc-l)r+l, • • • >-X"fcr,£fc,£fc,-X"(fc_l)r + l, • • • ,Xkr), K ^ 1, is a conditionally i.i.d. sequence. Let S" be the copy of S' with S0' = S0 and step-lengths X[',X%,.... Conditionally on (S0, S0'), ijfc := (Sfcr - 5i'r)/d(So,5»), 0 ^ fc < oo,
Section 6. Random Walk with Spread-Out Step-Lengths 101 forms an integer-valued random walk with symmetric aperiodic bounded step-lengths. Thus M = inf{k ^0:Rk=0} = inf{fc ^ 0 : Skr = Skr} is finite with probability one. Both XMr+i,XMr+2,--- and X^r+1,X^r+2,... are sequences of i.i.d. copies of X\, and both sequences are independent of (Sq , S",..., 5^r). Since Smv = "5^r, this means that we obtain a copy S'" of S' by switching from S" to S at time K = Mr, that is, we have once more established the existence of a successful coupling. 6.4 Final Part of Proof - Always Uniform Part After 2r Steps We shall now complete the proof of Theorem 6.1 by showing that the situation dealt with in the previous subsection is always the case (when the step-lengths are spread-out). Lemma 6.1. // (6.1), holds then there are constants a, b, and c such that a < b, c > 0, and P(X1+--- + X2r£A) > c / 1M], A£B. J A Proof. Due to (6.1), P(Xj + --- + x2re A) > J (Jf(x-y)f(y)dy)dx, AeB. (6.2) It is no restriction to assume that / ^ 1 and that / = 0 outside a finite interval [a,b] (since if this is not the case, we can replace / by l[a,b]f A 1, where a and b are such that faf> 0). Let gn, 0 ^ n < oo, be nonnegative continuous functions that are 0 outside the interval [a,b], bounded by 1, and such that (see Ash (1972), Section 2.4.14) / |/-ffn|-»0, n -» oo.
102 Chapter 3. RANDOM ELEMENTS Use 0 ^ / ^ 1 to obtain the latter inequality in | J f{x - y)f(y) dy- J f(x' - y)f(y) dy\ ^ | f{x-y)f(y)dy- gn(x-y)f(y)dy\ + | / 9n(x-y)f{y)dy- / gn{x' -y)f(y)dy\ + | J f(x' - y)f(y) dy- J gn{x' - y)f(y) dy\ ^ 2 / \f ~9n\ + | (gn(x-y) -gn(x' -y))f(y)dy\. Since gn(x — y) — gn(x' — y) is bounded and goes to 0 as x' —> x, we obtain by bounded convergence that limsupl [ f(x-y)f{y)dy- [ f(x' - y)f(y) dy\ ^ 2 /' \f - gn\. x'—^x J J J The right-hand side goes to 0 as n —> oo, and thus lim \ [f{x- y)f(y) dy - [ f(x' - y)f(y) dy\ = 0, x G R, x'->x J J that is, J f(x — y)f(y) dy is continuous as a function of x. Take d such that / f{d - V)f(y) dy >0 and put = \jf(d-y)f(y) dy. Take a and b close enough to d for J f(x — y)f(y) dy ^ c to hold when a ^ x ^ b. Then J f(x — y)f(y)dy ^ cl[a,i,](a;), a; G K, and a reference to (6.2) completes the proof. □ 6.5 The Renewal Theorem - Spread-Out Version Let 5 be a renewal process, that is, let the X^ be strictly positive and So nonnegative. For B G B([0, oo)), let N(B) be the number of renewals in B, oo k=0 Blackwell's renewal theorem (Theorem 8.1 in Chapter 2) says that if Xt is nonlattice, then for h G [0, oo), E[N(t,t + h]]->h/E[Xi], t-»oo.
Section 6. Random Walk with Spread-Out Step-Lengths 103 When Xi is spread out and E[Xj] < oo, Theorem 6.1 can be used to sharpen this to hold with (t, t + h] replaced by t + B where B is any Borel subset of [0, h]; and the convergence is uniform in B. Theorem 6.2. Let S be a renewal process such that Xx is spread out and E[X\\ < oo. Then, for each h € [0, oo) and with A the Lebesgue measure on [0, oo), E[N(t + B)] -> A(S)/E[X!] uniformly in B e B([0, h]), (6.3) as t —¥ oo. COMMENT. Note that (6.3) cannot hold for general nonlattice renewal processes (with EfXJ < oo) as the following example shows. Let B be the nonrational subset of [0, h] and note that X(B) = h. Let S be zero-delayed with rational recurrence times. Then for all rational t, E[N(t + B)] = 0, which does not tend to h/E[Xi]. PROOF. Let S' have the same recurrence time distribution as S and the delay time distribution Goo from Corollary 8.1 in Chapter 2. According to that corollary, E[N'{0,t]] = t/E[Xi], t e [0,oo). Thus the measure with mass E[N'(B)] at B £ B([0,oo)) must coincide with A/E[Xi]. Let S, S', and K be as in Theorem 6.1, put T = Sk, and take h g [0, oo) and BeB{[0,h]).Then E[N(t + B)] = E[N(t + B)], E[N'(t + B)] = E[N'{t + B)} = A(B)/E[A-!], N(t + B) = N'(t + B) on{T^t}- This yields the two equalities in \E[N{t + B)] - A(S)/E[X1]| = |E[JV(* + B)] - E[N'{t + B)]\ ^E[\N(t + B)-N'(t + B)\] = E[\N(t + B)-N'(t + B)\l{T>t}}. Since \N{t + B) - N'(t + B)\ ^ N([t, t + h}) + N'([t, t + h}), we obtain sup \E[N(t + B)] - A(B)/E[Jfi]| BeB{[o,h}) (64) < E[N([t, t + h})l{T>t}] + E[N'([t, t + h])l{r>t}]. BothN{[t,t + h])l{T>ty and N'([t,t + h])l{T>t} tend to 0 with probability one as t -> oo, and both are [see (8.1) and (8.2) in Chapter 2] dominated
104 Chapter 3. RANDOM ELEMENTS in distribution by the finite-mean random variable N%. Thus by dominated convergence (see Corollary 9.1 in Chapter 1) E[N([t,t + h])l{T>t}]->0 and E[N'([t,t + h])l{T>t}] -> 0 as t -¥ oo. This and (6.4) yield (6.3). □ Remark 6.1. The measure with mass E[N(B)] at B G B([0,oo)) is denoted by E[N] and called the intensity measure of the renewal process S. The uniform rate result (6.3) says that E[N] tends to A/E[Xi] in total variation on bounded intervals (total variation is defined in Section 8 below). If the condition E[Xi] < oo is sharpened to E[Xj] < oo, then this result can be improved to hold in total variation on the whole half-line; see Section 7.5 of Chapter 10. 7 Coupling Event - Maximal Coupling In Chapter 1 we established the existence of a maximal coupling of a collection of discrete random variables and of a countable collection of continuous random variables, that is, a coupling making all the variables coincide with maximal probability. We now extend this result to general random elements in an arbitrary space and start by establishing the key measure-theoretic result. 7.1 Greatest Common Component Let (E,£) be a measurable space, I an arbitrary index set, and m, i G I, a collection of measures on (E,£). A measure v on {E,£) is a common component of the /jj, i G I, if it is a component of each Hi, that is, if v ^ m, i G I. Moreover, v is a greatest common component of the /x*, i G I, if all other common components are components of v. Theorem 7.1. Any collection of measures pn, i G I, on an arbitrary space (E, £) has a unique greatest common component, which we denote by f\iej Hi- It holds that (/\/ii)(i4) = sup{i/(A):i/<A*<,«'eI}, Ae£. (7.1) Comment. Note that in general, (/^(AJjMnfi/^AJiiel}. t€l
Section 7. Coupling Event - Maximal Coupling 105 In order to see this, consider a collection of probability measures Hi, i G I, and suppose there is a j and a k such that Hj(Ac) = Hk{A) = 0. Then ( /\ &)(E) = ( f\/*)(Ac) + ( /\ w)(A) < ^(Ac) + N(A) = 0, iei iei iei while inf{/Xj(£) : i G 1} = 1. PROOF. Uniqueness is obvious: if v and z/ are two greatest common components, then by definition, v(A) ^ v'(A) and v(A) ^ ^'(^4), A £ £, and thus v = v'. In order to establish existence define a set function n by /i(A) := sup{v(A) : v ^ Hi,i & I}. Clearly, /j ^ /jj, j 6 I, and z/ ^ /x for all common components v. Thus the set function n is a greatest common component if it is a measure. Since \x is nonnegative, it only remains to show that /j, is u-additive. For that purpose, let Ai,A2}... be an arbitrary sequence of disjoint sets in £. We must establish that H{AY U A2 U • • •) = ji(Ai) + /i(A2) + • ■ • . (7.2) For each j ^ 1, let v^\, i/j2, ■ ■ ■ be a sequence of common components of the [ii,i € I, such that "jkiArftfiiAj), fc->oo. For each k ^ 1, define a common component vk of the m, i G I, by Vk :=^u(-nAi) + z/2/fc(-nA2) + ••• and note that for each j we have Vk{Aj) = Vjk(Aj). Thus "t(^)tM^j). fc->oo, j ^ 1, and /x(^! \jA2U---)^vk(A1uA2U---) = VkiAi) + vk(A2) + ■ ■ ■ . Sending k —> oo yields, due to monotone convergence, fi{Al U42U-)^ M^O + /i(A2) + • • ■ . (7.3) In order to establish the converse inequality, let v[, v'2,... be a sequence of common components of the /i;, i G I, such that v'k(A1UA2U---)-tn(A1uA2U---), k^oo.
106 Chapter 3. RANDOM ELEMENTS Since for k ^ 1, i/k{A1uA2\J---) = i/t(Ai) + i/k(A2) + ■■■ ^ n{Ai) +/i{A2) + • • ■ , we obtain, by sending k —» oo, /i(Ai U A2 U • • • ) ^ M^l) + M^) + • • • • This together with (7.3) yields the desired result, (7.2). □ 7.2 Coupling Event Inequality Let Yi, i e I, be a collection of random elements in a general space (E,£). Call an event C a coupling event of a coupling (Y, : i 6 I) of Yi, i e I, if Y, = y} on C, i,j e I. Call lc a coupling indicator if C is a coupling event. Theorem 7.2. If C is a coupling event, then /\p(yie-)^P(^e-,C), jel, (7.4) and, with || • || denoting total mass (total variation norm), || /\P(yj € -)|| ^ P(C)- COUPLING EVENT INEQUALITY (7.5) Proof. Fix a j e I. For all i e I, P(Yi e •) = P(£ € •) ^ P(£ € -,C) = P(y,- e -.C). Thus P(Yj e -,C) is a common component of the P(Yi € •), i € I, which yields (7.4), and (7.5) follows by evaluating (7.4) at E. □ 7.3 Maximal Coupling A coupling (Yi : t e I) with coupling event C is maximal if the coupling event inequality is an equality, ||/\p(yie-)|| = P(C). (7-6) This is equivalent to /\P(Yie-) = P(^e-,C), jel, (7.7) ten
Section 8. Maximal Coupling Two Elements - Total Variation 107 since (7.6) follows by evaluating (7.7) at E, and conversely, if (7.7) does not hold, then with fi := f\ielP{Yi € ■), there is a j and an A such that n{A) >P(Yj e A, C), by (7.4), which together with fi(Ac) ^ P(Yj &AC,C) yields ||p|| > P(Yj e A,C) + P(YJ e AC,C) = P(C), that is, if (7.7) does not hold, then neither does (7.6). Theorem 7.3. There exists a maximal coupling of any collection of random elements Yi, i g I, in an arbitrary space (E,£). Proof. Let I, V, and Wj, j e I, be independent. Let I be a 0-1 variable with p(/ = i) = ||/\p(yi€-)|. iei When P(I = 1) = 0, let the Yj, j e I, be independent and take C = 0. When P(J = 1) = 1, let the Yj, j e I, be identical and take C = fi. When 0 < P(J = 1) < 1, let V and Wj, j e I, be independent random elements in {E, £) that are independent of I and have distributions P(Ve-) = /\P(Yi&.)/p(l=i), iei P(Wj e ■) = (P(Vi e •) - Ap(y» e -))/p(^ = 0). Put, for j e I, V if 1=1, Y = J 'K if/ = 0, and C = {/ = 1} to obtain the desired result. □ 8 Maximal Coupling Two Elements - Total Variation We shall now consider the case of only two random elements Y and Y' in an arbitrary space (E, £) and formulate the results of the previous section in terms of total variation distance between the distributions of Y and Y'. We first establish a decomposition result for differences of measures, like P(Y e •) — P(Y' e •), then have a look at some basic properties of the total variation norm, and finally present the reformulation. 8.1 Difference of Measures — Mutual Singularity Two measures v+ and v~ on £ are mutually singular if they put all their mass in separate parts of E, that is, if there is a set A+ e £ such that v+{E\A+)=0 and v~{A+) = 0. (8.1)
108 Chapter 3. RANDOM ELEMENTS Denote this by v+ Lv . According to the next theorem v+Lv~ <£> z/+ A v~ (E) = 0; (8.2) here v+ A v~ is the greatest common component of v+ and v~. Theorem 8.1. Let [i and // be bounded measures on £ and let /1A/1' be their greatest common component. Then (fi- fi')+ := fi - fi A fi', (8.3) (fi-fj,') := fi'- fi A fi', are the unique measures satisfying fi-fi' = {fi- fi')+ -{n- fi')~, (8.4) Further, let f and f be densities of [i and fi' with respect to some measure X, for instance with respect to A = [i + fi'. Then / A /' is a density of /jA/i', (8.5) (/ - f')+ *s a density of (/j - /i')+, (8.6) (/ — /')" is a density of (fi — //)". (8.7) Proof. Let / and /' be densities of \x and fi' with respect to some measure A, and note that /A/' = /-(/-/')+=/' -(/-/')"• (8-8) Let v be the measure with density /A/', and note that v ^ fiA/j,'. Also, fi—u and // — v have densities (/ — /')+ and (/ — /')~ and are mutually singular since (/ - /')+ = 0 outside B+ = {x : f(x) > f'(x)} and (/ - /')~ = 0 on B+. Now, [i A n' — v is a component of both [i — v and n' — v and thus has mass zero both outside and inside B+. Thus v' = \x A fi', and we have established the theorem except for the uniqueness result. In order to establish uniqueness of the decomposition (8.4), suppose there is another decomposition /j, — fi' = v+ — v~ where v+ ± v~. Let A+ be as at (8.1) and B+ be such that (/x - n')+(E \B+) = 0 and (/x - /x')~(s+) = °- Then (»-»')(■ n A+) = v+> o, (/x-/x')(-nB+) = (/x-/x')+^0, and thus by additivity (fi - fi'){- n (A+ U £?+)) ^ 0. In the same way we obtain (fj, - //)(• n (A+ n B+)c) ^ 0. Therefore, (li - aO(- n (A+ u B+) n (A+ n B+V) = °> which implies (fi - //)(■ n B¥) = (/j - /j')(- n A+), that is, i/+ = (/j - /x')+- Thus also v~ = (fi — /i')~. □
Section 8. Maximal Coupling Two Elements - Total Variation 109 Corollary 8.1. Let Hi,H2,--- be a sequence of bounded measures on £. Then f\ Hii f\ m, n->oo. Further, if ft, fi, ■ ■ ■ are densities of Hi, [i2, ■ ■ ■ with respect to some measure X, for instance with respect to X = ^2l<i<oc 2~lHi> then inf fi is a density of A Hi- l<i<oo ' x l^i<oo Proof. The measures f\i<nHi decrease setwise to a component \x of all the n\,H2,---, and if there were an A such that n(A) < (f\i<oc m)(A), then there would be an n such that {/\i<n Hi){A) < {f\i<oc Hi){A), which cannot hold, since f\i<oc Hi is a component of f\i<nHi- Thus /* = f\ Hi- l^i<oo To establish the second half of the corollary observe that (h\ A H2) A H3 = f\i<3Hi- Applying this and (8.5) repeatedly yields that f\i<nHi has the density inf^„/j. By monotone convergence {/\i^nHi){A) = JA inf i^n ft decreases to JA infi<0O ft for A e £. Thus fi{A) = / inf ft, Ae£, J A l$«<oo IA and the proof is complete. □ 8.2 Total Variation The difference \x — h' oi two bounded measures is still a bounded u-additive set function but not necessarily nonnegative, therefore 'signed': a real- valued function v defined on £ is a bounded signed measure if it is bounded, sup \v{A)\ < 00, Ae£ and (j-additive, that is, for each sequence Ai,A2,... of disjoint sets in £, v{Ax U A2 U • • •) = u{Ai) + v(A2) + ■■■ . The total variation norm of v is |H| := sup v{A) - inf v{A). (8.9) A€£ Ae£ If v is a measure, then clearly ||i/|| = v{E) = the mass of v. For a real-valued function g defined on E let g € £ denote that g is £/B-measurable.
110 Chapter 3. RANDOM ELEMENTS Theorem 8.2. Let fi and fi' be bounded measures onS with densities f and f with respect to some measure X, for instance with respect to X = [i + fi'. Then there is a set A+ € £ such that Ha* - a*'II = (a* - »')(A+) - (a* - n')(E \ A+) = ll(^-/x') + || + ||(/x-/x')-|l = J(f-f')+d\ + J(f-f')-d\, = f\f-f'\dX J (8.10) = f fdX+ f f'dX - 2 f f A f'dX = ||Ai|| + ||A«'||-2||AiAAi'|| = SUP [ 9dn - gd/i'j - inf N gdfi - gdfi'j. In particular, when [i and fi' have the same mass, ||/j|| = \\(J.'\\, then \\li-fi'\\=2fr-n')(A+) = 2snp(fi-fi')(A) Ae£ = 2 sup \/i(A) - fj,'{A)\ Ae£ = 2||(/x-//) + l = 211^-^)"! = 2J(f-f')+dX = 2 ju-fydx = 2 sup ( gdfi- gdfi'j, (8.11) and when [i and fi' are probability measures, \\fi\\ = ||/i'|| = 1, then ||At-Ai'|| = 2-2||A.AAi'|| = 2-2 [fAf'dX. (8.12)
Section 8. Maximal Coupling Two Elements - Total Variation 111 Proof. From (8.4) we deduce -\\(/i - //)"ll ^ {fi- fi')(A) ^ ||(/i - fi')+\\ and that there is a set A+ € S such that (fl-fl')(A+) = \\(n-n')+\\, (n-n')(E\A+) = -\\(ii-fi')-\\. Thus sup(/x - a*') W = (A* - A*')(^+) = ll(A* - A«')+ll, ^ (8.13) inf (ax - fi'){A) = (a, - a*')(S \ A+) = -||(a. - A»')"ll- This yields the first and second equality in (8.10). The third equality in (8.10) follows from (8.6) and (8.7), the fourth and fifth are immediate, the sixth follows from (8.5), and the seventh from the third equality and sup (ax - n'){A) < sup ( fgdfi - [gdfi) < /(/ - f')+d\, Ae£ ge£ V J ' J f(Ax-Ax')(^)^ mf (Jgdti-Jgdfi')^-J(f-f')-dX in Ae£ If ||Ai|| = ||Ai'||,then(Ai-Ai')(£) = 0 and thus (a* - fi')+{E) = (A» - A»')~(£)- Hence (8.11) follows from the first and second equality in (8.10) together with (8.13), (8.6), (8.7), and (8.14). Finally, (8.12) follows from the fifth and sixth equality in (8.10). □ If a*i , At2, ■ ■ ■, A4 are bounded measures let tv fln -t (1, Tl ->■ 00, denote that ||ax„ — a*|| -> 0 as n -> oo. The following result (Scheffe's theorem) is an easy consequence of Theorem 8.2. Corollary 8.2. Let fix, fi2, ..., fi be probability measures with densities Ai fi, ■ • • i / with respect some measure A. If fn —¥ f as n —► oo a.e. X, then tv fin —¥ fi as n —► oo. Proof. According to the sixth equality in (8.11), we have ||At„ — At|| = /(/ - fn)+d\ . Now (/-/„)+ ^ / and / fdX < oo, and the desired result follows by dominated convergence. □ The converse does not hold, see Example 7.2 in Chapter 1.
112 Chapter 3. RANDOM ELEMENTS 8.3 The Coupling Event Inequality - Maximal Coupling We are now ready to reformulate the results of Section 7 in the case of two random elements Y and Y' in an arbitrary space (E, £). The total variation distance between the distributions of Y and Y' is [see (8.11)] ||P(y g ■) - p(y' e Oil =2 suP(P(r g A) - p(y g a)). (8.15) If (Y, Y') is a coupling of Y and y with coupling event C [that is, Y — Y' on C], then P(r G A) - P(y' G A) = P(f G A) - P(f' G A) = p(y g a, cc) - P(y' g a, cc) < P(CC) and applying (8.15) yields the following coupling event inequality, ||P(y G •) - P(i" G OIK 2P(CC). (8.16) This inequality we wrote in Section 7 as ||P(y G0AP(y'G0ll ^P(C). (8.17) That (8.16) is a reformulation of (8.17) follows from the first equality in (8.12). Equality holds in (8.16) if and only if it holds in (8.17). Thus the coupling is maximal if and only if ||P(y GO- P(l" G Oil = 2P(CC). (8.18) If the set {y = Y'} is an event [is measurable], then by definition {Y — Y'} is a coupling event, and since C is a subset of {Y = Y'} [and thus P(y jt f) ^ P(CC)], we can [due to (8.16)] rewrite (8.18) as l|P(y go- p{y' g oil = 2P(f ? y'). (8.19) The set [Y = Y'} is measurable when, for instance, (E,£) is Polish (see (7.5) in Chapter 4). 8.4 Comment on Signed Measures Although we shall not need this result here, it is in fact true (and not too hard to prove, see Ash (1972), Theorem 2.1.2) that for any bounded signed measure v on £ there are unique measures v+ and v~ on £ such that v = v+ — v~ and v+± v~ [Jordan Hahn decomposition]. The measures v+ and v~ are the positive and negative parts of v and \v\ := i/+ + v~ is called the total variation measure. Note that in general, v+(A) jt MA))+, v-{A)jt(v{A))-, \u\(A)jt\u(A)\.
Section 9. Hitting the Limit 113 9 Hitting the Limit In Chapter 1 (Sections 6 and 7) we established that a sequence of discrete random variables can be coupled in such a way that the variables hit the limit in finite time and stay there if and only if their probability mass functions converge pointwise to the probability mass function of the limit variable, and that a sequence of continuous random variables can be coupled in this way if and only if the lim inf of their densities is a density of the limit variable. We shall now extend this to random elements in an arbitrary space. 9.1 Coupling Index Inequality Let Y\,..., Yqc be random elements in an arbitrary space (E, £). Call a random variable K in {1,..., oo} a coupling index of a coupling (1t\,..., Y^) ofY1,...,Y00ii Y„ = Yoo for n ^ K. Theorem 9.1. If K is a coupling index, then for 0 ^ n ^ oo, l\ p(ne)^p(y00G-,K^n), || f\ P(Yfc € -)ll ^ V{K ^ n)> COUPLING INDEX INEQUALITY ||P(yooe-)- A P(ne-)ll<P(tf>n)- (9-1) PROOF. This follows from Theorem 7.2, since {K ^ n} is a coupling event for the coupling (Yn,..., Y^) of Yn,..., Y^. □ 9.2 Hitting the Limit If K is finite, we obtain from (9.1) that /\ Pfte-)tP(^e-), n-K». (9-2) In fact the following holds. Theorem 9.2. Let Y\,..., Y^ be random elements in an arbitrary space. There exists a coupling with a finite coupling index if and only if (9.2) holds and if and only if /\ P(y*G-)tP(noG-). n-Kx>, (9.3)
114 Chapter 3. RANDOM ELEMENTS and if and only if liminf//t is a density of Y^, (9-4) k—+oo where /i, /2, • • • are the densities o/Yi, Y%,... with respect to some measure A, for instance with respect to A = Xa</t<oo 2-fcP(YJs: G •). Comment. In Chapter 1, Theorems 6.1 and 7.1, we showed that these equivalent conditions are in general strictly stronger than total variation convergence [Yn —>• Y^ as n —>■ oo], except in the discrete case [when E is finite or countable). PROOF. By Corollary 8.1, f\n<k<00 ~P(Yk <E •) has density infnsjfc<00 fk. As n —► oo, this density increases to lim infk->oo fk and thus by monotone convergence f\n<k<oc PQ'fc £ ') increases setwise to a measure with density liminffc^oo fk. Thus (9.4) and (9.3) are equivalent. From (9.3) we obtain that An<fc<t30P(y* G ") ^ P(^oo € •) and therefore A„^<00P(n € •) = A„a<°oP(y* e 0; hence (9.3) implies (9.2). Conversely, since AnO^oo p(** G ') ^ A„£*<ooP(** e 0. we have that (9.2) implies that lim^oo /\n<k<00 P(Yk € •) ^ P(Voo € •)> and since liirin^oo An</fc<ooP(^: e ') has mass ^ 1, it follows that (9.3) holds. Thus (9.2) and (9.3) are equivalent. Due to (9.1), the existence of a coupling with a finite coupling index implies (9.2), and the converse follows from the next theorem. □ 9.3 Maximality at Each Index Call a coupling (Y\,..., Yoo) with coupling index K maximal at each index if the coupling index inequality is an equality, || [\ P(ne-)||=P(*"<n), 0<n<oo, (9.5) or equivalently [see (7.6) and (7.7)], if f\ P(n G-)=P(Y00 €-,-K"<n), (Kn<oo. (9.6) Theorem 9.3. For any sequence of random elements in an arbitrary space there exists a coupling that is maximal at each index. Proof. Let fi\,... , fioo be the distributions of Y\,... , l^oo- Put vn= /\ (*k, 1 ^ n < oo, Vaa = lim A \xk and Vq — 0. n~+00 ' *
Section 9. Hitting the Limit 115 Let Vi,V2, ■ ■ ■ ,Wi,..., Woo, K be independent random elements. Let K be {1,..., oo} valued with distribution Y{K = n) = \\vn\\-\\vn-X\\, 1 ^ n < oo, P(tf = oo) = l-||"oo||, and note that P(K^n) = J2 (lkl|-|l"*-ill) = IKII, K"<oo. (9.7) For 1 ^ n < oo, let the distribution of Vn be -p(Vn G •) = K - ^n-\)fP(K = n) (arbitrary if ¥{K = n) = 0). For 1 ^ n ^ oo, let the distribution of Wn be [note that P(K ^ n + 1) = 1 — ||i/n||, even for n = oo] P(TynG-) = (/in-^n)/P(^^n + l) (arbitrary if P(K^n + l) =0). Put f = (V*r on{K<n + l}, " [Wn on{K^n + l}. Then (Yi,..., Yoo) is a coupling of Y\,..., Y^, since p(fnG-)= Y. ?(vke-)'P(K = k) + -p(Wne-)-p(Kzn + i) l^k<n+l = fin, 1 ^ n ^ 00. Clearly, K is a coupling index, and by (9.7) the coupling is maximal at each index. □ 9.4 No Final Element Suppose a sequence Yi, Y^,... (without a final element Y^) is given. Call K a coupling index if {K ^ n} is a coupling event for Yn, Yn+i,... for all n < oo. Then || /\ P(Yke-)\\>P(K^n), l^n<oo. n^fc<oo
116 Chapter 3. RANDOM ELEMENTS Let Yoo be a random element such that P(yoo G ") 2 lim A P(n G •)• n—>oo ' * Then Theorem 9.3 yields the following: there exists a coupling of Yx, Yi, • • • that is maximal at each index, that is, a coupling with coupling index K such that || f\ P(ne.)ll=P(tf<n)> (Kn< oo. And Theorem 9.2 yields the following: there exists a coupling of Yx, y2, • • • with a finite coupling index if and only if lim A P(Yjfc G •) is a probability measure and if and only if lim inf k-^oa fk is a probability density. 9.5 Continuous Index Consider a continuous index family Yt, t £ (0, 00]. Then a random variable K in (0, 00] is a coupling index if Yt = Yoo for t ^ K. Since ~P(K ^ t) is right-continuous in t, while || A(<s<oo PQ's G -)ll need not be [and not left-continuous either], it is clear that Theorem 9.3 must be modified. We also leave out the density part of Theorem 9.2, but the rest of the above results can be transferred from discrete to continuous index. Theorem 9.4. Let Yt, t € (0, oo], be random elements in an arbitrary space. If K is a coupling index, then ||P(^ocG-)- A P(nG-)ll<P(tf>*). 0<t<oo, and there exists a coupling with a finite coupling index if and only if /\ P(y, e-)tP(noG-), t->oo. Further, let t\,t2,--- be an increasing sequence of positive real numbers such that tn —¥ 00 as n —> 00. There is a coupling that is maximal at each index in {^1,^2, ■ • • }, that is, || /\ P(Yse-)\\ = P(K^t), te{h,t2,...},
Section 10. Convergence in Distribution and Pointwise 117 or equivalently, /\ P(ys6-) = P(te-,if^t), *€{*,,t2,...}. PROOF. Let /xs be the distribution of Ys. Put t(s) = tn if tn ^ s < tn+\. In the proof of Theorem 9.3 replace {1, ...,oo} by {ti,t2, ■ ■ ■ ,00} and W\,..., Woo by Ws, s G (0, 00], where Ws is a random element with distribution (/xs — z/((s))/(l — ||^t(»)||) if Ik^s)!! < 1 and otherwise an arbitrary distribution. This yields a {*i, *2i - - - } valued coupling index K. The rest of the theorem follows as in the discrete index case. □ 10 Convergence in Distribution and Pointwise In Chapter 1 (Section 8) we turned convergence in distribution for random variables into pointwise convergence. We shall now extend this result to random elements in a space {E, £) where E is separable metric and £ its Borel subsets. In Section 1 below we have a look at the definition of convergence in distribution and its basic properties and then prove the coupling result in Section 2. In Chapter 1 the quantile coupling was used to turn convergence in distribution into pointwise convergence. The quantile coupling represents all random variables as measurable functions of a uniform random variable. In Section 3 below we extend this result to random elements in a Polish space and in Section 4 use the construction to give another proof of the coupling result in the special case of a Polish space. This latter coupling result is due to Skorohod (1956), while the extension to a separable space was established by Dudley (1968). 10.1 Convergence in Distribution The following result is basic. Fact 10.1. Let Y\,Yz,..., Y be random elements in (E,£) where E is a metric space with metric d and £ are the Borel subsets of E. The following four conditions are equivalent: (a) for bounded continuous functions f from E to M., lim E[/(y„)] = E[/(y)], n—>oo (6) for Ae£ with P(Y G boundary of A) = 0, lim P(Yn G A) = P{Y G A), n—+00
118 Chapter 3. RANDOM ELEMENTS (c) for open subsets A of E, liminf P(yn GA)Z P(Y G A), n—>oo (d) /or closed subsets A of E, HmsupP(y„ G A) < P(y G A). n—>oo Moreover, if E is the real line and F\, F2,. ■ ■, -F are the distribution functions of the random variables Y\, Y2,.. ■, Y then these equivalent conditions are also equivalent to (e) for x GM. such that F is continuous at x, lim Fn(x) = F(x). n—>oo For a proof, see Ash (1972), Theorem 4.5.1 and Theorem 4.5.4. (The first of these theorems is the basic result for so-called weak convergence of measures. We have stated the result here in the special case of probability measures and in a random element form). The sequence Y\,Yi,. ■. is said to converge in distribution to Y if one of the equivalent conditions in Theorem 10.1 holds (and the distributions are said to converge weakly). Denote this by Yn —> y, n —► 00. Note that convergence in distribution is weaker than convergence in total variation. The latter is defined for {E, £) arbitrary [see Section 8.2] and means that (a) holds uniformly in / G £ bounded by one [or any fixed constant] and without the continuity restriction. Total variation convergence also means that (6) holds uniformly in A G £ and without the restriction that P(y G boundary of A) = 0. Further, note that if the sequence Yi,Y2,... converges pointwise to Y, that is, Yn->Y, n ->■ 00, (short for lim d(Y{to),Yn(to)) = 0,w G 0) n—>oo then f(Yn) —► f(Y) pointwise as n —> 00 for continuous functions /, and thus by bounded convergence it follows from (a) that pointwise convergence implies convergence in distribution. Finally, note that if Yn %■ Y and Yn 4 Y', then, due to (a), E[f(Y)] = E[/(y')] for bounded continuous functions /, and thus [since bounded continuous function are a measure determining class] Y and Y' have the same distribution, that is, the limit random element is distributionally unique.
Section 10. Convergence in Distribution and Pointwise 119 10.2 Turning Distributional Convergence into Pointwise We shall now show that if Y\,Yi,. ■. ,Y are random elements in a space (E,£), where E is a separable metric space and £ its Borel subsets, then V D V Yn-¥Y, n -¥ oo, if and only if there is a coupling (Yi, Yi,..., Y) of Y\, Yi,..., Y such that Yn —> Y, n —► oo. The 'if part was established in Section 10.1, and the 'only if part follows from the next theorem. Theorem 10.1. Let E be a separable metric space and £ its Borel subsets. If a sequence of random elements in (E,£) with distributions Pi,P2,... converges in distribution to a random element with distribution P then there exist random elements Y^\Y^2\ ... ,Y with distributions Pi,P2, ■ ■ ■ ,P such that y(n) —>■ Y pointwise in the metric as n —>■ oo. PROOF. Let d be the metric. A set A in £ is called a P-continuity set if P(dA) — 0 where dA denotes the boundary of A. Due to separability, for any fixed k ^ 1, E can be covered with countably many sets of diameter less than 1/k. Further, these sets may be taken to be P-continuity sets since d{x £ E : d(y,x) < r} is a subset of {x £ E : d(y,x) = r} and the spheres around y are P-continuity sets except for countably many radii r. Note also that the sets can be made disjoint because d(A D B) is a subset of dA n dB. Let {An, A\2, • ■ • } be a countable partition of E into P-continuity sets with diameter less than 1. Let {A^iijA^a,--.} be a countable partition of An into P-continuity sets with diameter less than |. Continue this recursively: let {A(k+i)i1...iki,-*4-(fc+i)ii...ifc2) • • • } be a countable partition of Au\...ik into P-continuity sets with diameter less than 1/fc. Since countable unions of countable classes are countable, it follows that for each k ^ 1, {^kii...ik '■ (ii, • • • ,ik) € {1,2,... }k} is a countable partition of the set E into P-continuity sets with diameter less than 1/k. Let Y be a random element with distribution P and define a sequence of positive random integers Mi, M2,... by (Afi,...,Aft) = (ii,...»*) if YeAkil...ik. (10.1) Thus T>((M1,...,Mk) = (i1,...,ik))=P(Akh...ik). For each n ^ 1, let M[n\ M2, ■ ■ ■ be a sequence of positive random integers with distribution defined by P((M1(n))...,Afin)) = (ii,...it))=Pn(Atil...<J.
120 Chapter 3. RANDOM ELEMENTS We shall prove in the next subsection that the M[n', M% , ■ ■ ■ can be chosen in such a way that for each 1 ^ k < oo there exists a finite random integer Nk such that (M[n\...,Mikn)) = (M1,...,Mk), n^Nk. (10.2) Assume at this point that (10.2) holds. Let Vfcj- i be an Aki1...ik valued random element that is independent of (M{ ,..., Mfc ) and has distribution Pn(-\Aki1...ik). Define a random element Y^ by n(B) = *&U if (Af1(B),...,AfW) = (i1>...,ifc)- Then Y^n' has the distribution Pn. Since j ^ k implies that Aji1..jj is a subset of i4fci1...jfc, it follows from (10.1) and (10.2) that if n ^ Nk and j ^ fc, then y and y("' are in the same Aki1...ik set. Thus d(Y,Y!>n)) < Ilk if n ^ ATfc and j ^ k. Define y(") = yfc^) where kn = sup{fc ^ n : P(Nk > n) < l/k}. Then y(n) has distribution Pn,kn —► oo as n —► oo, fc„ ^ km if n ^ m, and d(y,y(n)) < l/fcm if O Nkm and n ^ m. Since the iVjfc are finite, this yields limsup d(yy'n^) ^ l/km -> 0 as m -> oo, n—>oo that is, limn-yoo d(y, y(")) = 0. Thus it only remains to establish (10.2) to complete the proof. □ 10.3 Proof of (10.2) - Transfer We now show that the Al[n\ M^n\ ... can be chosen so that (10.2) holds. In the proof we shall use Theorem 6.1 in Chapter 1 and the transfer extension (Section 4.5 of this chapter) inductively. Since P{dAki1...ik) = 0, we have the following convergence of probability mass functions: for all (ii,...ik) € {1,2,... }k, P((M<n\...,Af<n>) = (i1,...,it)) ->P((Mi,...,Mfc) = (h,...,ik)), n-> oo.
Section 10. Convergence in Distribution and Pointwise 121 Take k = 1 and apply Theorem 6.1 in Chapter 1: there is a coupling (M[1\m[2),...,Mi) of M1(1),M1(2),...,Mi and a finite random integer N\ such that M[n)=Mx, n>N1. (10.3) Apply transfer: since M\ is discrete, there exists a regular conditional distribution of (M[l), m[2\ ..., Mu Nx) given M\, which together with Mi = Mi means that we may take Mi := M\. Then apply transfer count- ably many times recursively in n: each M{ ' is discrete, and thus there exists a regular conditional distribution of (Mj ,M^ , ■■■) given M[n', which together with M[n) = M[n) implies that we may take M[n) := M[n) for all n. Due to Mi = Mi and M[n) = M[n\ 1 ^ n < oo, we obtain from (10.3) that (10.2) holds for k = 1. In order to establish (10.2) inductively for finitely many k, repeat the above argument. Fix k ^ 2 and suppose (10.2) holds with k replaced by 1 through k — 1. Apply Theorem 6.1 in Chapter 1 to obtain that there is a coupling ((M[1\...,M^),(M[2\...,Mi2)),...,(Ml,...,Mk)) of the collection of /c-dimensional random integer vectors (M1(1),...,M(1)),(M1(2),...,Mf)),...,(M1,...,Mfc) and a finite random integer Nk such that (Min),...,Min)) = (M1,...,Mk), n^Nk. (10.4) Apply transfer to obtain that we may take (Mi,...,Mk):=(Mi,...,Mk), (M1^,...)M^)1):=(Af1(n))...,Af^1), l^noo. (10.5) Then apply transfer countably many times recursively in n to obtain that we may take M{kn) := M{kn\ lsCn<oo. (10.6) Due to (10.5) and (10.6), we obtain from (10.4) that (10.2) holds for k. In order to establish (10.2) for all k note that in the Arth step of the recursive transfer construction of (Mi,M2, ...),(Ni,N2,...), and {M[n\ M<n),...), 1 sC n < oo, we did not [see (10.5) and (10.6)] redefine (Mi,M2, ...),(NU..., Nk-!), and {M[n\ ..., M{kn_\), 1 ^ n < oo. Thus the recursive transfer construction is consistent and can be repeated countably many times recursively in k. Thus (10.2) holds for all k.
122 Chapter 3. RANDOM ELEMENTS 10.4 Representation on the Lebesgue Interval The Lebesgue interval is the probability space ([0,1],<B[0,1], A) where A is the Lebesgue measure on ([0,1],$[0,1]). We shall now generalize the quantile coupling (Section 3 in Chapter 1) by showing that all random elements in Polish spaces can be represented as measurable mappings of a single uniform variable. This result can be stated as follows. Theorem 10.2. For each probability measure P on a Polish space (E,£) there is a random element, defined on the Lebesgue interval, with distribution P. Proof. Recall that (E,£) Polish means that there is a metric d on E making E a complete [Cauchy sequences converge] separable [there is a countable dense subset] metric space with S the Borel sets. For each k ^ 1, construct a countable [E is separable] partition Ak — {Ak\,Ak2, ■ ■ ■ } of E into sets (in £) of diameter less than 1/k. Let Ak+i refine Ak, that is, let each set in Ak be a union of sets in Ak+i [see the second paragraph of the proof of Theorem 10.1 and note that the P-continuity requirement is not needed in this proof]. Then construct countable partitions h = {hi,Ik2,---} of [0, 1] into subintervals whose length is X(Iki) = P{Akl) with Ik+i refining Ik- Arrange the indexing so that Aki 2 Ak+i,i if and only if Iki 2 h+i,i- The construction can be carried out inductively because if ai,a,2, ■.. are nonnegative numbers adding to the length of an interval, then the interval can be split into subintervals of lengths ai, <i2,.... Let \jki be some element of A^ and define, for u G [0,1], Yk(u) = yki if uelki. (10.7) Since [for fixed w] the set {Yk (u), Yk+i («),...} is a subset of a single element of {Aki, Ak2, ■ ■ ■ }, its diameter is at most 1/k. Thus Yjt(u), Yk+i (u),... is a Cauchy sequence for each u. Thus the limit Y(u) exists \E is complete] and d(Y(u),Yk(u))^l/k. (10.8) For B G £ let Bk be the closure of {y G E : d(y, x) < 1/k for some x £ B} (1/k neighborhood of B). With k fixed and L the set of positive integers i such that B n A^ /0we have X(Yk GB)^ X(Yk G (J Aki) = £ \(Yk G Aki) = £ \(Iki) = Y,P(Akt) = P([JAkt) = P( U AkinB)^P(Bk).
Section 10. Convergence in Distribution and Pointwise 123 If B is closed, then Bk decreases to B as k —> oo, and it follows that limsup X(Yk € B) ^ P(B), for closed subsets B of E. Thus, by definition [see (d) of Fact 10.1], it holds that Yk converges in distribution to a random element with distribution P. Now Yk converges to Y pointwise, and thus in distribution. Since such a limit is distributionally unique, we deduce that Y has distribution P. □ 10.5 The Skorohod Coupling in the Polish Case We shall now elaborate on the construction in the previous subsection to reprove Theorem 10.1 in the special case of random elements in a Polish space (which means that in addition to separability we also have completeness). The proof is quite different and yields the extra result that the random elements are all supported by the Lebesgue interval. Theorem 10.3. // a sequence of random elements in a Polish space (E,£) with distributions P\,P2,... converges in distribution to a random element with distribution P, then there exist, on the Lebesgue interval, random elements Y^\Y^2\ ... ,Y with distributions P\,P2, ■ ■ ■, P such that y(n) _^ Y pointwise in the metric as n —> oo. PROOF. Construct the countable partitions Ak of the preceding proof, but this time require that each Aki be a P-continuity set [see the first paragraph of the proof of Theorem 10.1]. Consider the countable partitions Ik as before and, for each n, construct successively finer [as k increases] partitions ^{/^^•■.Iwith A(/£>) = Pn(Akl). Inductively, arrange the indexing so that il? < iff if and on\y if Iki < Ikj [here I < J means that the right endpoint of / does not exceed the left endpoint of J). In other words, arrange that for each k the families Ik, Ik \lk ■>••■ are ordered similarly. Define Yk by (10.7), as before, where yki 6 Aki, and define Yln\u) = yki if uElff. Again Yk(u) converges to a limit Y(u) satisfying (10.8) and Y^n\u) converges as k —> oo to a limit Y^n\u) satisfying d(Y^(u),Yk{n)(u)) ^l/k. (10.9)
124 Chapter 3. RANDOM ELEMENTS And again Y has distribution P and Y^ distribution Pn. Since the Aki are P-continuity sets, we have by the convergence in distribution assumption [see (6) of Fact 10.1] A(Ffc(n) = Vti) = Pn(Aki) -»■ P(Aki) = X(Yk = yki), n -»■ oo, that is, for each fc, the probability mass functions of Yk converge pointwise to that of Yfc as ra —► oo. Theorem 6.1 in Chapter 1 now yields that Y^ tends to Yk in total variation as n -> oo, and thus [since X(Yk' = yki) = M^ki ) an^ -M^* = 2/fcj) — ^(^fci)] for any set L of nonnegative integers A(U4?))^A(U/^)' n^°°- (10-10) Fix /c and j and choose L = {i21:Iki<Ikj} = {i21:lW<I%)} to obtain from (10.10) that the left endpoint of/^ goes to the left endpoint of Ikj as n —> oo. Similarly, the right endpoint of /Ji," goes to the right endpoint of Ikj. Hence, if u is in the interior of Ikj, then for all sufficiently large n, u lies in 4"\ so that Yk(n)(u) = Yk(u) and, by (10.8) and (10.9), d(Y(u),Y^(u))^d(Y(u),Yk(u))+0 + d(Y^(u),Yk{n)(u))^2/k. Send first n —► oo and then k —> oo to obtain if w is not an endpoint of any Ikj, then Y^n\u) -» Y(u) as n —► oo. The set of endpoints of the Ikj is countable and thus has Lebesgue measure zero so that if Y^n\u) is redefined as Y{u) on this set, then Y^™) still has distribution Pn, and there is now convergence for all u. D
Chapter 4 STOCHASTIC PROCESSES 1 Introduction In this chapter we shall be concerned with coupling general stochastic processes (in one-sided time) in such a way that they ultimately merge. This was our main concern in the first half of Chapter 2 and in Section 5 of Chapter 3. Recall, for instance, the classical coupling: two differently started versions of a Markov chain run independently until they meet, at a time T say, and then run together from time T onward. For lack of a better term we shall call this kind of coupling exact coupling. The qualifier 'exact' here refers to the fact that the processes coincide exactly from T onward as opposed to what was the case in the latter part of Chapter 2 and will be the case in Chapter 5, where the processes merge only modulo a random time shift (shift-coupling, epsilon-coupling). Exact coupling is in fact what some writers still call only 'coupling'. The word coupling then refers to the merging, and not to joint construction in general as in this book and as is becoming more and more common. Section 2 starts with preliminaries establishing notation and discussing the definition of a stochastic process to make sure that our simple abstract framework and the reasons why it is chosen are understood. Section 3 then introduces exact coupling and its distributional version, which is obtained by replacing pointwise merging by distributional merging. Distributional exact coupling is not as intuitively appealing as its nondis- tributional counterpart but has several merits. It applies (slightly?) more generally and has the same distributional implications. It is easier to establish and can serve as a first step in constructing a nondistributional exact 125
126 Chapter 4. STOCHASTIC PROCESSES coupling. Section 4 takes a brief look at distributional coupling in general and establishes the coupling event inequality. Section 5 presents the coupling time inequality and the resulting limit theory. Section 6 proves the central maximality result, and Section 8 reformulates the proof after the concept of coupling with respect to a sub- c-algebra has been introduced in Section 7. Section 9 introduces the tail cr-algebra T and proves a result on maximal coupling with respect to T that leads to a basic set of equivalences between successful exact coupling, convergence in total variation, and distributional identity on T■ This chapter is followed by Chapter 5, where analogous theory is established for two generalizations of exact coupling: shift-coupling and epsilon couplings. Chapter 6 then considers the implications of these three sets of coupling results in the Markov case, while Chapter 7 extends the view beyond stochastic processes. 2 Preliminaries - What Is a Stochastic Process? Before turning to the coupling theory we spend a number of pages (a third of the chapter!) on some general but simple aspects of stochastic processes that are basic for our purposes. The impatient reader could skim rapidly through this section. Much of it is a motivation for the technically straightforward property of 'shift-measurability', which we often impose in order to be able to shift our processes randomly, and which is satisfied in the standard settings such as when the state space is Polish and the paths right continuous. 2.1 Classical Definition The classical definition of a stochastic process goes as follows: a stochastic process with index set 1 and state space (E,£) is a family Z = (Zs)s^i, where the Zs are random elements defined on a common probability space (0, T, P) and all taking values in (E,£). We shall here think of the index set as time and mostly restrict the use of the term 'stochastic process' to the following four cases: I = R two-sided continuous time, I = [0, oo) one-sided continuous time, I = Z two-sided discrete time, I = {0,1, 2,... } one-sided discrete time. In this chapter we shall, in fact, only be concerned with one-sided processes. But in this section the index set I is kept general for later purposes. Warning. To avoid misunderstanding it should be stressed that here the role of the index set I is different from the role it had in Chapter 3. We are not going to couple the collection of random elements Zs, s 6 I, that is,
Section 2. Preliminaries - What Is a Stochastic Process? 127 their joint distribution will be fixed. We shall couple Z and another process Z'\ the collection to be coupled will be Z and Z'. 2.2 Stochastic Process as a Random Mapping in {El,£l) Let (El,£l) denote the product space {E\£l):=®{E,£). Rather than regarding Z in the classical way as a family of random elements in (E,£), we can equivalently regard Z as a random mapping, that is, as a single random element in {El,£l) defined by Z(w) = (Zs(u>))sei, a; G ft. The two points of view are equivalent because each Zs is a measurable mapping from (Cl, T) to (E, £) if and only if Z is a measurable mapping from (ft,.?-) to(E\£l). The distribution of a stochastic process Z is the distribution of Z as a random element in (El,£l). The distribution of Z is uniquely determined by the finite-dimensional distributions, that is, by the distributions of the (En,£n) valued random elements {Ztx, ■ ■ ■ ,Ztn), t\,...,tn G I, n ~£ 1. These finite-dimensional distributions are in turn determined by p(ztl e Ai,...,ztn e Ai), Au...,Ane£, h,...,tnei, n > 1. In particular, when Z is real valued, that is, (E,£) = (R, B), then the finite-dimensional distributions are determined by the finite-dimensional distribution functions P(Ztt < xi,...,Ztn < xn), xi,...,xneR, ti,...,tn e I, n > 1. Finally, for Polish (E,£), Kolmogorov's consistency theorem (Fact 3.2 in Chapter 3) states that if a consistent collection of finite-dimensional distributions is given, then there always exists a stochastic process having these finite-dimensional distributions. 2.3 Path Space {H,%) - Standard Settings The paths of Z are the realizations Z(u)),u> G ft, of the random mapping Z. Sometimes restrictions are put on the paths, for instance in the continuous- time case that they are continuous, or right-continuous, or right-continuous with left-hand limits, or more generally that they lie in some subset H of El. In this case it is natural to consider Z as a random element not in {El,£l) but in (H,H), where H is the a-algebra on H generated by the
128 Chapter 4. STOCHASTIC PROCESSES projection mappings taking z in H to zt in E, t 6 I. Note that H is the trace of H on £l, that is, ft := £l n# := {inffiie^1}. Again the two points of view are equivalent: Z has H valued paths and each Zt is a measurable mapping from (fi, .F) to (£, £) if and only if Z is a measurable mapping from (0, J7) to (H,H). When a particular patft set iJ is given, call (H,"H) the pot/i space of Z. (One could conceive of allowing 7i to be more general than just the trace of H on £l, but we shall not do so here.) Note that H need not be an element of £l. In particular, H is not an element of £l in the standard settings in continuous time when (E, £) is Polish, I = R or I = [0, oo), and # is one of the sets Ce(P) = continuous maps from I to E, -De (I) = right-continuous maps from I to E with left-hand limits, ■Re(I) = right-continuous maps from I to E. Both C.e(I) and -De(I) can be metrized such that the Borel cr-algebras, Ce(I) and T>e(I), are the traces of the respective path sets Ce(I) and -De(I) on £l (see Ethier and Kurtz (1986)), but we shall not need this fact here [except for three isolated results, Theorem 7.4 in Chapter 5, Theorem 5.4 in Chapter 8, and the (v)-part of Theorem 3.4(6) in Chapter 10]. The distribution of a stochastic process Z with path space (H, V.) is the distribution of Z as a random element in (H,H). The distribution is again uniquely determined by the finite-dimensional distributions. However, even if (E, £) is Polish and a consistent collection of finite-. dimensional distributions is given, there need not be a stochastic process with path space (H,H) having these finite-dimensional distributions. This has to be checked in each individual case. One example where it can be established is for the Wiener process: there is a one-sided continuous-time real-valued process with path space (C,e(I),C,e(I)) having independent stationary increments (see Billingsley (1986) or Kallenberg (1997), we will only use this fact for an isolated example, Section 8.1 in Chapter 7). 2.4 Observing a Process at a Random Time We would like to be able to observe a stochastic process at a random time. There is no complication with this in discrete time, but in continuous time there is. So let Z be a one-sided continuous-time stochastic process with state space (E,£) and let T be a random time in [0, oo). By Zt we mean the E valued mapping defined on fi in the obvious way: zT{w) ■■= zT(W)(w), w e fi.
Section 2. Preliminaries - What Is a Stochastic Process? 129 This mapping need not be T'/£ measurable. A condition sometimes imposed to take care of measurability complications is the following: a continuous-time one-sided stochastic process Z is jointly measurable if the mapping taking (u, t) 6 ft x [0, oo) to Zt(uj) 6 E is T ® B[0, oo)/£ measurable. This condition implies that Zt is measurable, since Zt is the composition of two measurable mappings: the first, taking u> to (ui,T(ui)), is TjT® B[0,oo) measurable, and the second, taking (u>,T(u>)) onward to Zt(u\(u>), is T ® B[0, oo)/£ measurable. However, Zt being measurable is not all we need, as is illustrated in the next subsection. 2.5 Joint Measurability Is Not Enough Consider the following example. Example 2.1. Take (£,£) = (R,B) and let (Q,f,P) be the Lebesgue interval: ft =[0,1], T =£([0,1]), P = Lebesgue measure. Put T(u) = u and let Z, Z' be the one-sided continuous-time stochastic processes defined by Zs(u) =0, s € [0,oo), weft, Z;(w) = l{0,i,...}(s-w), s6[0,oo), weft. Trivially, Z is jointly measurable. Also, Z' is jointly measurable, since as a mapping from ft x [0, oo) to E, it is a composition of two measurable mappings: the first, taking (w, t) in ft x [0, oo) to t — u in R, is T ® B[0, oo)/5-measurable, and the second, taking t — uj in R onward to l{o,i,...} (t — cj) in R, is B/B-me&sm&ble. Now, the finite-dimensional distributions of Z and Z' are identical, and thus Z and Z' have the same distribution. In fact, for 0 ^ u ^ 1, P(Z'ti =0,...,Zi=0,T^u) = P(Ztl =0,...,ZK =0,T^u)=u. Thus (Z,T) = (Z',T). In spite of this, ZT and ZT do not h ave the same distribution, since certainly Zt = 0, while ZT = 1.
130 Chapter 4. STOCHASTIC PROCESSES 2.6 Canonical Joint Measurability May Be Needed Example 2.1 shows the following: knowing that (Z,T) = (Z',T) and that Zt and Z'T are random elements does not suffice to deduce that Zt and Z'T have the same distribution. This is typically the sort of conclusion we would like to be able to draw, so what went wrong? Not surprisingly, the condition we need is joint measurability of the canonical versions of the processes: say that a continuous-time one-sided stochastic process with path space (H, T-Cj is canonically jointly measurable if the mapping (z, t) 6 H X [0, oo) to zt 6 E is H ® B[0, oo)/£ measurable. This is the condition needed to draw the conclusion, from the fact that (Z,T) and (Z',T) have the same distribution, that Zt and ZT also have the same distribution. Thus in Example 2.1 there is no way to find a path space such that the processes are canonically jointly measurable. Canonical joint measurability on the other hand implies joint measurability, since a canonically jointly measurable Z, as a mapping from fi x [0, oo) to E, is the composition of two measurable mappings: the first, taking (u,t) to (Z(u),t), is T ® B[0, oo)/?^® B[0, oo) measurable, and the second, taking (Z(u),t) onward to Zt(oj), is H®B[Q, oo)/£ measurable. Hence, canonical joint measurability is a strictly stronger condition than joint measurability. 2.7 In Fact, Shift-Measurability May Be Needed Rather than observing our process only at a random time we would also like to be able to observe the whole process from that time onward. Say- that a path set H of a one-sided continuous-time stochastic process Z is internally shift-invariant if {(zt+s)se[o,°o) ■ z € H} = H, t e [0, oo). When this is the case, define the shift-maps 8t, t 6 [0, oo), from H to H by OtZ = (Zt+s)s€[0,oo)> Z E H. Say that Z is shift-measurable if Z has a path set H that is internally shift-invariant and the mapping taking (z,t) 6 H x [0, oo) to 6tz 6 H is H <8> B[0, oo)/?^ measurable. A stochastic process with an internally shift-invariant path set H is shift- measurable if and only if it is canonically jointly measurable: canonical
Section 2. Preliminaries - What Is a Stochastic Process? 131 joint measurability is (trivially) equivalent to the mapping taking (z, t + s) to zt+s being H ® #[oj00)/£ measurable for each s £ [0, oo), which in turn is equivalent to shift-measurability, since % is generated by the projection mappings. 2.8 Shift-Measurability Holds in the Standard Settings The standard cases where the state space (E, £) is Polish and the path sets are Ce[0,oo), De[0,oo), and Re[0, oo), respectively, are all covered by shift-measurability. In fact, we only need E separable metric but not necessarily complete. This is a corollary of the next theorem, as can be seen as follows. First note that Ce[0, oo), De[0, oo), and Re[0, oo) are all internally shift-invariant and thus it suffices to establish canonical joint measurability. Now recall that separability of a metric space is equivalent to second countability, that is, every open cover has a countable subcover. Thus if G, is open we can cover it by open balls whose closure lies in G. And since G is second countable as a subspace of E, we can cover it by countably many of these balls. Theorem 2.1. Suppose E is topological, £ is generated by the open sets, and the paths of Z are right-continuous (an element of E is a limit of a function if the function eventually stays in any neighbourhood of the element). If every open G C E is the union of countably many open sets Gj whose closure Gj lies in G, then Z is canonically jointly measurable. PROOF. Let H consist of the right-continuous elements oi E^°'°°K In order to show that g : (z, t) h-> zt is H ® B[0, oo)/£ measurable, take d > 0, put Ld = {0,d,2d,...} and [t\d = sup{s 6 Ij : s < t}, and define 9d ■ (z,t) ^ z[t]d+d. Note that (z, t) t-» (z, [t]d + d) is H <g> B[0, oo)/H'® B{Ld) measurable, (z,t) t-» zt is H ® B(Ld)/£ measurable, gd is a composition of these two mappings. Thus gd is %®B[Q, oo)/£ measurable. By right-continuity, gd —» g pointwise as d I 0, and thus the measurability of g follows from the next lemma. □ Lemma 2.1. Suppose E is topological, £ is generated by the open sets, fn are measurable mappings from some measurable space (K,JC) into (E,£),
132 Chapter 4. STOCHASTIC PROCESSES and fn —¥ f pointwise as n —> oo. // every open G C E is the union of countably many open sets Gj whose closure Gj lies in G, then f is measurable. Proof. We must show that f~1(G) G K. for open G C E. Now, xef-\G) f(x) GG => 3j: f{x) G Gj if a- 3j : fix) eGj 4= 3j,i : fn(x) eGj, n^i, 3j,i : x G f^iGj), n Js i, lj,i:xef)f^(Gj) j,i n^i and the proof is complete. □ 2.9 Killing - Birth - Shifting to Infinity Consider a one-sided continuous time stochastic process with state space (E, £). (The following applies also to processes in discrete and/or two-sided time with obvious modifications.) In order to be able to hide the process from a random time onward (killing) and/or prior to a random time (birth) we introduce a new state A icemetery or censoring state) external to E. For 0 $C t $C oo, define the killing maps Kt taking z G (E U {Zi})!0-00) to ntz G (E U {Zi})[°'°°) by \zs H0^s<t, \A if t ^ s < oo, and the birth maps pt taking z G (£U {Zi})!0-00) to j3tz G (El) {Zi})!0-00' by fzi ifO<a<t, \ zs if t -C s < oo.
Section 2. Preliminaries - What Is a Stochastic Process? 133 Note that there are no joint measurability complications with killing and birth: the mapping taking (z, t)G(EU {A})^0'^ x [0, oo] to ntz 6(£U {A}f^ is (a(S U {{4}}))!0-00) ® B[0, oo}/(a(£ U {{Zi}}))^00) measurable, and so is the mapping taking (z, t) to j3tz. If the process has a path space (H,7i), it is natural to consider nt and fit as mappings from the set Ha to Ha where Ha := |J KtPsH. This set is internally shift-invariant, and if the process is shift-measurable, then so are processes with state space (E U {A}, a(£ U {{Zi}})) and path space (Ha,T-La), where 1-La is (of course) generated by the projection mappings. It can be convenient to have the shift maps also defined for t = oo. Let that is, #oo is the mapping from Ha to Ha defined by (900z)s = A, 00<oo, zGHa- For t < oo extend the definition of 9t in the obvious way to Ha- 6tz = (zt+s)s€[o,oo), z 6 Ha- If Z is a one-sided stochastic process and T a nonnegative random time then 8tZ,PtZ and ktZ denote the Ha valued mappings defined on fi by (6tZ)(lu) = 6T(L0)Z{u), (f3TZ)(oj) = f3T(Lo)Z(oj), (nTZ)(u>) = kt(u)Z{u), for w£(l. 2.10 A Countable Product of Polish Spaces Is Polish Existence of regular conditional distributions is the key to applying the conditioning extension (Chapter 3, Section 4), which we use quite heavily. According to the following result, discrete-time stochastic processes Z with a Polish state space have also a Polish path space and thus (Chapter 3, Fact 4.1) have regular conditional distributions. Theorem 2.2. A countable product of Polish spaces is Polish.
134 Chapter 4. STOCHASTIC PROCESSES Proof. Let (Ei,£i), (E2,£2),-- ■ be a sequence of Polish spaces. Let dk be a metric making Ej. complete and separable and £k its Borel subsets. Define a metric d on rji° E/. by oo d = Y^2~k Adk- 1 In order to show that d is complete let zn = (z^)^=1, 1 ^ n < oo, be a Cauchy sequence in rji° E/. with respect to d. Then z%, 1 ^ n < oo, is a Cauchy sequence in Ej. with respect to dk, for each k, and since E\. is complete, this sequence has a limit Zk- Put z = (zk)f. Since for each k, dk(z^,Zk) -> 0 as n -> oo, it follows by dominated convergence that d(zn, z) —> 0 as n —> oo. Thus d is complete. In order to establish separability let, for each k, Ak be a dense countable subset of Ej. with respect to dk and let ak be a fixed element of Ak- Then the YlT Ek subset oo A = [j A1x-- xAkx {ak+1} x {ak+2} x ■■■ is countable, since the finite product Ai x ■ ■ ■ x Ak is countable and since a countable union of countable sets is countable (note that A is different from the uncountable set ni°^*)- Fix a z = (zfc)i° m TIT Ek- For each e > 0 there is an n such that 2~n < 2_1e and for each k < n a bk in Ak such that dk(zk,bk) ^ 2_1e. Put b = (blt... ,bn,an+i,an+2,. ■ ■ )■ Then b is in A and n oo d(z, b) sC 2~xe Yl2~k + Y1 2~k ^ 2_l£ + 2~" < e- 1 n+l Thus the countable set A is dense in \\T Ek, that is, \\T Ek is separable. In order to establish that 0^° Sk is the Borel c-algebra BIXYT Ek), reca^ that separability and second countability are equivalent for a metric space, and thus each Ek has a countable base. The sets AiX ■■■ x Anx En+l x En+2 x • ■ ■ , 1 ^ n < oo, where the Ak range over the countable base of Ek, form a countable base for \\T ^k- Thus #(ni° Ek) is contained in ®^° £k- Conversely, for a fixed n let A be the largest sub-c-algebra of £n making the nth projection mapping <B(ni° Ek)/A measurable, that is, oo A= {Ae£n:E1 x ■■■xEn_1 xAxEn+l x ■•• G fi(JJ^*)}-
Section 2. Preliminaries - What Is a Stochastic Process? 135 Then A contains the open sets of £n and thus A = £n. Hence, for each n, the nth projection mapping is B(Y[^° E/.)/£n measurable. It follows that 0J°£fc is contained in B([\f Ek), that is, we have proved that ®^°£fc = m? Ek). □ 2.11 Weak-Sense-Regular Conditional Distributions In the standard settings in continuous time we shall not establish the existence of regular conditional distributions [in fact, (CE([0, oo)),Ce([0, oo)) and (De([0, oo)),T>e([0, oo)) are both Polish when (E,£) is Polish; see Ethier and Kurtz (1986), but we shall not use this fact here]. We shall establish a weaker result, which turns out (next subsection) to be all we need to carry out the much-used transfer extension. Let Y\ and Y2 be random elements in some measurable spaces [E\,£\) and (£2,£2), respectively, defined on the probability space (fi,T,P). Say that there exists a weak-sense-regular conditional distribution of Y2 given Yi if (.E2,£2) can be embedded into a larger measurable space where regular conditional distributions exist, that is, if there is a measurable space (£3, £3) and a subset G of E3 (typically G £ £3) such that (£2, £2) and (G, £3 C\G) are Borel equivalent and there exists a regular version of I3 given Y\ where Y3 is the random element in (£3,£3) defined by Y3 = /(I2) with / the Borel equivalence. Theorem 2.3. Let Z be a one-sided continuous-time stochastic process with a Polish state space (E,£) and right-continuous paths. Let Y be some random element defined on the same probability space (fi, T, P) as Z. Then there exists a weak-sense-regular conditional distribution of Z given Y. PROOF. Let Q+ be the nonnegative rationals. In the above definition put Yx := Y, Y2:=Z and (E2,£2) := (RE([0, oo)),nE([0,oo))), (£3,£3):=(£°+,£0+) and G := {(zs)seQ+ : z G KE([Q, 00))}. Let / be the bijection from Re([0, 00)) to G defined by f(z) = (zs)seQ+, z S Re([0, 00)). Now Y3:=(Zs)sm+ is a random element in (JE°+,£°+) and (Zs)sm+ =f(Z). By Theorem 2.2, (£°+,£°+) is Polish, and thus [Fact 4.1 in Chapter 3] there exists a regular conditional distribution of (Zs)s6q+ given Y. □ 2.12 Transfer Revisited We shall now show that weak-sense-regularity (and thus Theorem 2.3) suffices for extension purposes. Consider again the transfer extension in Section 4.5 of Chapter 3, that is, let Yx be a random element in (£i,£i)
136 Chapter 4. STOCHASTIC PROCESSES defined on a probability space (fi, T, P) and suppose we have managed to construct on another probability space (fi', T', P') a pair (Y{, Y2) where Y2 is a random element in some measurable space (Z?2, £2) and Y{ is a random element in (E\,£i) such that This time assume only that there exists a weafc-serase-regular version of the conditional distribution of Y2' given Y{. Theorem 2.4. With Y\ and (Y/, Y2) as above, (fi, T, P) can be extended to support a random element Y2 in (£2,£2) such that (YUY2) = {YIX)- (2-1) Proof. Let / be the bijection (the Borel equivalence) from E2 to the subset G of Ez and let Y3' be the random element in (E3,£3) defined by YI = /(Y2). Since (by assumption) there exists a regular version of the conditional distribution of Y3' given Y{, we can use the transfer extension of Section 4.5 in Chapter 3 (see Remark 4.1 at the end of that subsection) to obtain a G valued random element Y3 in (E3,£3) such that (YltY3)2(Y{,Y& (2-2) Since both Y3 and Y3 are G valued, they can be considered as random elements in {G,£% C\G) rather than (£3,£3). After this modification, (2.2) still holds and also Y2 = /_1(Y3'), and finally we can define Y2 = f~1(Yz) to obtain from (2.2) that (2.1) holds, as desired. □ It is also readily checked that given Y\, Y2 is conditionally independent of any random element Yq supported by (fi, T, P) before the extension. 3 Exact Coupling - Distributional Exact Coupling After these lengthy preliminaries we now return to coupling. This section introduces exact coupling and its distributional version. 3.1 Exact Coupling — Definition Let Z and Z' be one-sided discrete- or continuous-time stochastic processes with a general state space (E, £) and a general path space (H, %). We shall use notation in accordance with continuous time, but all we need to switch to discrete time is to substitute, for instance, t and s by n and k. Recall that (Z, Z') is a coupling of Z and Z' if Z = Z and Z' = Z'.
Section 3. Exact Coupling - Distributional Exact Coupling 137 A nonnegative random time T (integer-valued in the discete-time case) is a coupling time (or coupling epoch) if 6TZ = 6TZ' on {T < 00} (see Figure 3.1). (3.1) The triple (Z,Z',T) is an exact coupling of Z and Z', P(T < 00) is the success probability, and (Z, Z',T) is successful if P(T < 00) = 1. Using the convention that a process is absorbed in the cemetery state when shifted to infinity, we can rewrite (3.1) simply as 6tZ = 6tZ . Also, using the birth maps [see Section 2.9], (3.1) can clearly (if unclear, consult the end of next subsection) be rewritten as pTz = pTz'. (3.2) Note that T+ 1 is also a coupling time. FIGURE 3.1. Exact coupling: the processes merge from time T onward. 3.2 Distributional Exact Coupling — Definition We now weaken the requirement that Z and Z' ultimately coincide and demand rather that Z behave distributionally from a time T onward as Z' does from a time X" onwards: say that (Z,Z',T,T') is a distributional exact coupling of Z and Z' if (Z, Z') is a coupling of Z and Z' and D (3TZ = Pt'Z1 (see Figure 3.2). (3.3) The bold parts have the same distribution. jO/hy 0 r FIGURE 3.2. Distributional exact coupling: the processes merge in distribution.
138 Chapter 4. STOCHASTIC PROCESSES Call T and X" distributional coupling times. If T s T', call T a single distributional coupling time. Note that if (Z, Z',T) is an exact coupling, then (Z,Z',T,T) is a distributional exact coupling with a single distributional coupling time T. We shall use the word nondistributional to distinguish an exact coupling from a distributional one. Otherwise, we use the same terminology in both cases. From (3.3) it follows that rp D fit This can be seen as follows: T is the time when flrZ exits from A, and T is recovered in a measurable way from /3tZ, since it is the pointwise limit of Tn = sup{fc J> 0 : (PrZ)k/n = A}/n, which are measurable mappings of (3tZ; in the same measurable way X" is recovered from /3T>Z'; thus (3.3) yields T = T". Moreover, when Z and Z' are discrete-time processes or continuous-time shift-measurable, then (Z,Z',T,T') is a distributional coupling of Z and Z' if and only if (6TZ,T) = (6T,Z',T'). (3.4) This can be seen as follows. From shift-measurability and QTZ = eT^rZ and QT.Z' = 6TI(3T,Z' we see that 9TZ is the same measurable mapping of (PtZ, T) as 9t> Z' is of (PtiZ'jT1). We have just seen that T is the same measurable mapping of (3tZ as X" is of Pt'Z1 . Thus 6tZ is the same measurable mapping of /3t'Z as 9T'Z' is of /3T'Z'. Thus (3.3) implies (3.4). Conversely, for 0 ^ s < oo, (B Z) ={A on{T>5}, {PT )s \{6TZ)S-T on{T^5}, and thus by shift-measurability PtZ is a measurable mapping of (9tZ,T). Since faZ' is the same measurable mapping of (8t'Z\T'), we have that (3.4) implies (3.3). For an example of a distributional exact coupling consider the classical coupling. Recall that a nondistributional exact coupling is obtained by letting two differently started versions of a Markov chain, Z and Z', run independently until they meet, say at time T, and letting the chains run together from time T onward. A distributional exact coupling (with a single time T) is obtained by letting the chains continue to run independently after meeting at time T (that is, if we allow the chains to stay independent and do not introduce the chain Z" as in Section 2.2 of Chapter 2).
Section 3. Exact Coupling - Distributional Exact Coupling 139 3.3 The Hats May Be Dropped in the Distributional Case We shall now show that if we have a distributional exact coupling of Z and Z', then we can take Z and Z' to be the original processes Z and Z'. Theorem 3.1. Suppose (Z, Z',T,T') is a distributional exact coupling of Z and Z'. Then the underlying probability space can be extended to support random times T and X" such that (Z, T) = (Z, f) and (Z', T) = (Z', f"). In particular, (3TZ = (3T.Z'. (3.5) PROOF. This follows from the transfer extension in Section 4.5 of Chapter 3. In order to obtain T take Yx := Z and (Y{,Y2) := (Z,T) and define T : = Y2. In order to obtain T" take Yx := Z' and (Y{,Y£) := (Z',f') and define T':=Y2. D This theorem motivates dropping the hats when discussing distributional coupling, at least when there is no danger of confusion. We shall say that T and X" are distributional coupling times for Z and Z' if (3.5) holds. 3.4 Turning Distributional into Nondistributional In the standard settings a distributional exact coupling can always be turned into a nondistributional one. Theorem 3.2. Let (Z,Z',T,T') be a distributional exact coupling of Z and Z'. Suppose there exists a weak-sense-regular conditional distribution of Z given (3fZ [this holds in discrete time when the state space is Polish and in continuous time when the state space is Polish and the paths are right-continuous}. Then the underlying probability space (fi,^7,P) can be extended to support T and Z" such that (Z, T) = (Z, f) and (Z", T) = (Z', f") and (Z,Z",T) is a nondistributional exact coupling of Z and Z' PROOF. Let T be as in Theorem 3.1. Obtain Z" by applying the transfer extension in Section 2.12 as follows. Take Yx := pTZ and (F/,!^') : = {Pf,Z',Kf,Z') to obtain Y2 such that (PrZ,Y2) = (l3t.Z',KT,Z'). Define Z" by ((3tZ",KTZ"):=(PtZ,Y2).
140 Chapter 4. STOCHASTIC PROCESSES Since (/3tZ",ktZ") is a copy of (/3f,Z',Kf,Z'), it follows that (Z",T) is a copy of (Z',2") because (Z",T) is determined in the same measurable way by {PtZ",ktZ") as (Z1 ,f') is by (/3f,Z',Kf,Z'). D 4 Distributional Coupling Distributional coupling concepts will play some role in what follows, and before continuing with stochastic processes, we shall now devote a whole section to the simplest of them all, the distributional version of a coupling event. The section ends with a general comment on distributional coupling. 4.1 Distributional Coupling Events Let (Y,Y') be a coupling of two random elements Y and Y' in an arbitrary space (E,£). Two events C and C" are distributional coupling events of (Y,Y') if Y has the same distribution on C as Y' has on C", that is, if p(y e-,C) = P(y'e-,c"). (4.1) Note that P(C) = P(C"). If C = C, call C a single distributional coupling event. If C is a coupling event [Y = Y' on C], then clearly C is a single distributional coupling event. We shall use the word nondistributional to distinguish a coupling event from a distributional one. We shall first show that a coupling with distributional coupling events can always be unhatted, that is, we can take Y and Y' to be the original Y and Y'. Theorem 4.1. Let (Y,Y') be a coupling of Y and Y' with distributional coupling events C and C'. Then the underlying probability space can be extended to support events C and C' such that (y,ic) = (r,i(j) and (Y',ic>) = (Y',id,). In particular, P(Y e-;C) = P(y'G-;C"). (4-2) Proof. Apply the splitting extension in Section 5.1 of Chapter 3. Due to (4.1), P(f G -,C) is a component of both P(Y G ■) and P(Y' G •)■ Let I and /' be 0-1 variables such that P(y e -,/ = 1) = P(Y' G -,/' = 1) = P(y G -,C). Take C = {I = 1} and C" = {/' = 1} to obtain the desired result. □ We now show that a coupling with distributional coupling events can always be turned into a coupling with a nondistributional coupling event.
Section 4. Distributional Coupling 141 Theorem 4.2. Let (Y,Y') be a coupling of Y and Y' with distributional coupling events C and C". Then the underlying probability space can be extended to support an event C and a Y" such that (Y,lc) = (Y,ld) and (F", lc) = (f, ld.) and (Y,Y") is a coupling ofY and Y' with C a nondistributional coupling event. Proof. Let C be as in Theorem 4.1. Let W be a random element in (E, £) that is independent of C and has distribution P(F' € -\C'C). Define Y" by Y" := Y on C and Y" : = W on Cc. D 4.2 The Coupling Event Inequality — Maximality The distributional coupling event inequality is a corollary to the nondistributional one. (It can also be established directly in exactly the same way as in the nondistributional case.) Theorem 4.3. Let (Y,Y') be a coupling of Y and Y' with distributional coupling events C and C. Then, with \\ • || the total variation norm, ||P(F € •) - P(F' € -)l| <2P(CC). COUPLING EVENT INEQUALITY PROOF. This follows from Theorem 4.2 and the coupling event inequality in the nondistributional case [Section 8.3 in Chapter 3]. □ A coupling with distributional coupling events such that the coupling event inequality is an identity is distributionally maximal and its events are distri- butionally maximal. Such a coupling always exists, since a nondistributional maximal coupling (Section 8.3 in Chapter 3) is in particular distribution- ally maximal. We now show that there exists an 'unhatted' coupling that is distributionally maximal. Theorem 4.4. Let Y and Y' be random elements in an arbitrary space (E,£). Then the underlying probability space can be extended to support events C and C such that (4.2) holds and ||P(Fe-)-P'CF'(E-)ll=2P(Cc), that is, (Y,Y',C,C) is distributionally maximal. Proof. This is an immediate consequence of Theorem 4.1 and the existence of a maximal coupling in the nondistributional case [Section 8.3 in Chapter 3]. □
142 Chapter 4. STOCHASTIC PROCESSES 4.3 Comment on Distributional Coupling Up to now we have been concerned with using coupling to turn distributional properties and relations into their pointwise counterparts. The distributional versions of both exact coupling and coupling events are steps in the reverse direction: they loosen up pointwise relations, turning them into distributional relations. In fact, to obtain a distributional exact coupling we no longer need to couple Z and Z', we only need the times T and X" [Theorem 3.1]. Similarly, to obtain distributional coupling events we need not couple Y and 1", we only need the events C and C [Theorem 4.1]. We shall see in the upcoming sections and chapters that this can be quite convenient. Distributional exact coupling assumes nothing about the joint distribution of the pairs (Z,T) and (Z1, T"). They could in principle be defined on different probability spaces: the pairs are only linked distributionally, that is, through the distributional relation PtZ = (3t'Z' rather than through the pointwise and therefore, necessarily, defined-on-a-common-probability- space relation firZ = p\Z'. Thus Z and Z' need not be a coupling of Z and Z', that is, although Z and Z' should have the same distributions as Z and Z', respectively, they need not really be defined on a common probability space as in the formal definition of coupling. A similar comment applies to a coupling (Y, Y') with distributional coupling events C and C": we assume nothing about the joint distribution of (Y,C) and (Y',C). We could formalize this as follows: Let Y and Y' be random elements in an arbitrary space (E, £) defined on the probability spaces (0, T', P) and (0', J7', P'), respectively. Call two random elements Y and Y' defined on some probability spaces (Ct,J-,P) and (fi',.F',P'), respectively, a distributional coupling ofY and Y' if Y is a copy of Y and Y' is a copy of Y'. According to this definition we should write (Z,T) and (Z',T") for a distributional exact coupling rather than (Z, Z', T, X"). In fact, writing (Z, T) and (Z',T') indicates nicely that we assume nothing about the joint distribution of the pairs. We shall not use this definition, however, due to the convention of the common probability space (Chapter 3, Section 3.1). This is mentioned here only because it may be an illuminating observation. 5 Exact Coupling - Inequality and Asymptotics Section 3 was devoted to the definition of exact coupling and its distributional version. We shall now go on to the limit implications.
Section 5. Exact Coupling - Inequality and Asymptotics 143 5.1 Coupling Time Inequality The following inequality (encountered repeatedly in Chapter 2) explains much of the interest in exact coupling. Theorem 5.1. Let Z and Z' be one-sided discrete- or continuous-time stochastic processes with a general state space (E, S) and an arbitrary path space [H,T-L). If there is an exact coupling (nondistributional or distributional) of Z and Z' with time T, then for 0 ^ t < oo, \\P{OtZ £■)- P(6tZ' € -)ll COUPLING TIME ^ 2P(T > t). INEQUALITY Proof. In the nondistributional case {T ^ t} is clearly a coupling event of the coupling (9tZ,0tZ') oiOtZ and 9tZ', and the coupling event inequality (Section 8.3 in Chapter 3) yields the coupling time inequality. In the distributional case we lean on the distributional version of the coupling event inequality (Theorem 4.3). Clearly, 9tZ = 9tj3TZ on {T ^ t} and 9tZ' = 8tj3T>Z' on {T1 ^ t], and thus we obtain from $tZ — fir1 Z1 that P(9tz eA,T^t) = P(6tz' eA,T'^ t), A en. In Theorem 4.3 take Y = 9tZ, Y'=9tZ', C = {T^t}, C' = {T'^t}, to obtain the coupling time inequality. □ 5.2 Finite T — Plain Total Variation Convergence The coupling time inequality is of basic importance for total variation asymptotics. We first note that if there exists a successful exact coupling (nondistributional or distributional) of Z and Z', then P(T > t) -> 0 as t —> oo, which yields \\P(9tZe-)-P{9tZ'e-)\\^0, f->oo. (5.1) If Z' is stationary, that is, 9tZ' = Z', t> 0, then (5.1) can be rewritten as 8tZ % Z', t -> oo, (asymptotic stationarity). (5.2)
144 Chapter 4. STOCHASTIC PROCESSES 5.3 Finite Moments of T — Rates of Convergence Results on rates of convergence can be obtained from the coupling time inequality if we know how fast P(T > t) goes to zero (examples were given in Chapter 2, Section 4). Let ip be a nondecreasing function from [0, oo) to [0, oo). If <p{t)P{T > t) -> 0, t -> oo, (5.3) then clearly the total variation convergence is of order ip, that is, <p(t)\\P{8tZe-)--p(8tZ'G-)\\^0, t^oo. (5.4) Common functions to consider are <p(t) — ta where a > 0 (power order a), <p(t) = pl where p > 1 (exponential or geometric order p). Also, logarithmic order comes to mind and mixtures of these orders. More general classes of functions will be considered in Chapter 10 (Section 7), where the observations in this section are applied to regenerative processes. Often a finite (p-moment of T, E[<p(T)] < oo, is what we have rather than the rate condition (5.3). Note that <p(t)P(T > t) < E[ip(T); T >t) for nondecreasing <p -> 0 as t -> oo if E[ip(T)] < oo by dominated convergence (since ip(T)l{T>t} ^ V(^) and goes to zero pointwise as t —> oo). Thus a finite (^-moment implies (5.3), and we obtain E[<p(T)] < oo (5.5) => <p(t)\\P(6tze-)-P(6tZ'e-)\\^o, i^oo, for nondecreasing functions ip. 5.4 Finite Moments of T — Moment Rates of Convergence Now E[(/?(T)] < oo is stronger than the rate condition (5.3) and should yield a stronger rate result. Consider first continuous time and suppose ip(Q) = 0 and that <p has a density, that is, there is a nonnegative measurable function ip such that tp(t) = / ip(s) ds, 0 ^ t < oo. Jo Clearly f* <p(s) ds = J™ v(s)l{t>s} ds and thus <p(T) = f™ v(s)l{T>s} ds,
Section 5. Exact Coupling - Inequality and Asymptotics 145 which yields /•OO E[<p(T)] = / (p(t)P(T > t) dt [by Fubini]. Jo Combine this and the coupling time inequality to obtain that the total variation convergence is of moment-order ip, that is, E[<p(T)] < oo (5.6) / Jo <p(t)WP(0tz e •) - p(<?tz' e Oil dt < oo, (5.7) for nondecreasing ip having a density 0. When ip(t) = ta where a > 0, then we have convergence of power moment-order a — 1. When tp(t) = pl where p > 1, then we have convergence of exponential (or geometric) moment- order p. In the discrete-time case let A<p denote the difference function (here A is not the cemetery state!) A<p(0) = <p(0) and A<p(k) = <p(k) - <p(k - 1), k^ 1. Then, instead of (5.5) we clearly have E[<p(T)] < oo => Y, AvWWkZ e •) - p(^^' e Oil < °°- 0 Remark 5.1. The integrand in (5.6) is measurable because \\P(9tze-)-P{9tZ'e-)\\ = 2 sup |P(Z e El0'*' x A) - P(Z' e El0-" x A)\, AeH which clearly is nonincreasing (and thus measurable). 5.5 Stochastically Dominated T — Uniform Convergence Let Z be a class of discrete- or continuous-time stochastic processes on a general state space. An example of such a class is the collection of all differently started Markov chains having the same transition probabilities. Suppose there exists a finite random variable T such that for all pairs of processes Z, Z' e Z there is an exact coupling (distributional or not) of Z and Z' with time T such that T < T [that is, P(T > t) < P(T > t) for 0 < t < oo].
146 Chapter 4. STOCHASTIC PROCESSES Then the coupling time inequality yields that for 0 ^ t < oo, ||P(0tz e •) - P{Btz' e OIK 2P(T > i), z, z' e 2, and we obtain uniform convergence over the class Z: sup ||P(0(Ze-)-p(0tz'<E-)ll ->0, £->oo. (5.8) z,Z'ez Rates of convergence are obtained in the same way as in the previous subsection under conditions on T rather than T: for nondecreasing functions <p it holds that E[yj(f)] < 00 => <p(t) sup ||P(0tze-)-P(0tZ'e-)ll->o, ^^ (5.9) z,z'ez in the continuous-time case the stronger result E[<p(f)} < oo r°° (5.10) => / <p{t) sup ||P(6»(ze-)-p(^z'e-)ll*<oo 70 Z,Z'£Z holds, provided that ip has a density 0, while in the discrete-time case E[ip(f)} < oo ~ (5.11) => V^A) sup ||P(flfcZG-)-P(«fc^'G-)ll<oo- o z,z'ez In Chapter 10 (Section 7) we make a thorough use of the results of this section in the case of regenerative processes. 6 Exact Coupling - Maximality We now turn to the task of reversing the implications in the previous section: we shall show that there is always an exact coupling that is good enough. In this section we prove this by a direct measure-theoretic construction, but in Section 8 we shall reformulate the proof after introducing the concept of maximal coupling with respect to a sub-cr-algebra in Section 7. 6.1 The Maximality Theorem Call an exact coupling (distributional or not) of Z and Z' with time T maximal at time t if the coupling time inequality is an equality at t: \\P(6tZ e ■) ~ 'PifitZ' G -)l| = 2P(T > t).
Section 6. Exact Coupling - Maximality 147 Call an exact coupling (distributional or not) of Z and Z' with time T maximal if it is maximal at all times: \\P(9tz e •) - F(°tZ' e -)ll = 2P(T > *), o ^ i < oo. A maximal exact coupling brings the processes together maximally fast (has a minimal coupling time). We shall now show that a maximal distributional exact coupling always exists in discrete time. In continuous time, however, the left-hand side of the coupling time inequality need not be right-continuous in t, whereas the right-hand side is. Thus in continuous time, equality cannot be achieved in general and we shall content ourselves here with showing that equality can be achieved at a sequence of times increasing to infinity. Theorem 6.1. (a) Let Z and Z' be one-sided discrete-time stochastic processes with a general state space (E,£). Then there exists a maximal distributional exact coupling (Z, Z', T, T') of Z and Z'. Moreover, there exists a maximal nondistributional exact coupling of Z and Z' if there exists a weak-sense-regular conditional distribution of Z given 9tZ [this holds when {E,£) is Polish}. (b) Let Z and Z' be one-sided continuous-time stochastic processes with a general state space (E,£) and an arbitrary path space (H,T{). Let to < t\ < ■ ■ ■ be a sequence of nonnegative real numbers increasing to infinity. Then there exists a distributional exact coupling (Z,Z',T,T') of Z and Z' (with to, t\,..., oo valued times) which is maximal at the times tn, that is, ||P(0t„ Z e •) - P(0t„ Z' e -)ll = 2P(T > tn), 0 ^ n < oo. (6.1) Moreover, there exists a nondistributional exact coupling of Z and Z' with this property if there exists a weak-sense-regular conditional distribution of Z given (3tZ [this holds when (E,£) is Polish and the paths are right- continuous]. Remark 6.1. Thus in discrete time, total variation convergence implies the existence of a successful distributional coupling, and the same holds in continuous time because ||P(0tZ G •) — P(6tZ' G -)II is nonincreasing (see Remark 5.1). 6.2 Preparation for the Proof of Theorem 6.1 The following lemma is the key part of the proof of Theorem 6.1. Lemma 6.1. Let n be a bounded measure on a measurable space (£,£). Let A be a sub-a-algebra of C and A a component of the restriction h[a of H from C to A- Then there exists a component v of ^ such that v\A = A.
148 Chapter 4. STOCHASTIC PROCESSES PROOF. Define a nonnegative set function v on £ by v{A) = Jn(A\A)d\, AeC, [here p(A\A) := (l*/M)(A\A)]. For A e A we have p.(A|yl) = lA ^ a.e. and thus v\a = A. Since A ^ /j,\a, we have v ^ J" ju(-|^4)d/u, that is, v ^ ^u. Since the null sets of ju|^ are null sets of A, it follows that v does not depend on the version of ^-\A). Thus for a given sequence of sets in £ we can choose a version that is a- additive for that particular sequence. Hence v is u-additive, and the lemma is established. □ 6.3 Proof of Theorem 6.1 We shall prove (a) and (b) simultaneously. To simplify notation in the continuous case we carry out the proof for tn = n; the proof for general tn is analogous (replace 0,1,2,... by t0,ti, £2, • • • throughout). Due to Theorem 3.2 we only need to find a distributional coupling such that (6.1) holds. Put tt:=P(ZG-) and V := P(Z' e •)• If there are measures vo,..., v^ and i/J,...,^oiiW such that it = v0-\ h j/oo and n' = i/'0-\ V v'^, (6.2) then we can use the splitting extension (Section 5.2 in Chapter 3) to obtain integer-valued random times T and X" such that V{Z e-,T = n) = vn and P(Z' e -,T' = n) = v'n, 0 < n ^ 00. Then (Z, Z', T, T') would be a distributional exact coupling of Z and Z' if P(0nZ € A,T = n) = P(9nZ' e A,T' = n), 0 ^ n < 00, A e H, which is equivalent to vn\rn = ^nlr„, 0 ^ n < 00, where Tn = 0~l1-L. (6.3) And (Z,Z',T,T') would be maximal at integer times, that is, (6.1) would hold, if we establish ||7r|r„ ^tt'\t„ II = P(^ ^ n) (see Section 8 in Chapter 3, in particular display (8.12), and recall that A denotes greatest common component), which is equivalent to (i>0-\ \-vn)\rn = t|t„ A7r'|r„. 0 ^ n < 00. (6.4) Thus all we have to do is find subprobability measures vq,...,Voo and (/0,...,i/>nW such that (6.2), (6.3), and (6.4) hold. Since the measure 7r|r„-i A 7r'|T^_i is a component of both ir\j-n_1 and tt'\t„-i and since 7^ is a sub-u-algebraof Tn-\, it is clear that the restriction
Section 7. Coupling with Respect to a Sub-cr-Algebra 149 of 7r|r„-i A ir'\r„-i to Tn is a component of both -k\t„ and ir'\r„, that IS> (^iTn-i A ^'iTn-i)!^ is a component of 7r|r„ A 7r'(7-^. Thus we can define subprobability measures An on Tn by Ao = 7T A 7r', A« =7r|r„ Att'It,, -(t|t„-i A7r'|r„-i)lr„, 1 <« < 00. Make the induction assumption that there are subprobability measures vo,... ,vn on % such that ^|rfc=Afc, 0 ^ fc ^ n, and f0 H h t-„ ^ 7r; (6.5) this certainly holds for n = 0, since % = H. In Lemma 6.1 put n = 7r — (i/0H hi>„) and A = An+i to obtain that there is a subprobability measure vn+i on % such that vn+i\rn+i = ^n+i and vn+\ ^ it - (vQ + ■ ■ ■ + vn). Thus (6.5) holds with n replaced by n + 1. By induction we have proved that for all n < 00 there are subprobability measures v0,... ,vn on % such that (6.5) holds, that is, there are subprobability measures u0,u\,... on H such that vn\fn = An, 0 ^ n < 00, and 1/0 + v\ + ■ ■ ■ ^ it. In the same way we obtain subprobability measures v'Q, v\,... on % such that v'n\rn = A„, 0 ^ n < 00, and f0 + ^i H < 7I''- Thus (6.3) and (6.4) hold, and defining v^ and u'^ by ^00 = 7T - (^0 + v\ H ) and 1/^ = 7r' - (i/q + v\ -\ ) yields (6.2) and completes the proof of Theorem 6.1. 7 Coupling with Respect to a Sub-cr-Algebra In this section we extract a concept hidden in the argument of the last section; it will be used in Section 8 to reformulate the proof of Theorem 6.1. We have noted earlier that if (Z, Z', T) is an exact coupling, then {T ^ t) is a coupling event of (9tZ, 9tZ') for each finite t. On the other hand, {T ^ t} is in general not a coupling event of (Z, Z1). We remedy this by extending the concept of a coupling event. 7.1 Coupling Event with Respect to a Sub-cr-Algebra Let (Y, Y') be a coupling of two random elements Y and Y' in a measurable space (E^S). Let A be a sub-cr-algebra of £. Call an event C an A-coupling event if Y and Y' are A-identical (or A-indistinguishable) on C, that is, if {Y e A} n c = {Y1 e A} n c, AeA.
150 Chapter 4. STOCHASTIC PROCESSES This is equivalent to C being a coupling event for the coupling (1a(Y), 1a(Y')) of 1a(Y) and l/t(F') for each A £ A. This in turn is equivalent to C being a coupling event for the coupling (f(Z), f(Z')) of f(Z) and f(Z') for all real-valued A/B measurable functions /. Call two events C and C" distributional A- coupling events if Y and Y' have the same distribution on C and C", respectively, when considered as random elements in (E,A): P(F eA,C) = P{Y'eA,C), AeA. (7.1) If we regard the F's as random elements in (E,A) rather than in {E,£), then (Y,Y') is still a coupling of Y and Y', and C and C" are ordinary distributional coupling events of this coupling. The converse of this does not hold: although F and Y have the same distribution when regarded as random elements in (E, A), they need not have the same distribution when regarded as random elements in (E,£). We shall first show that a coupling with distributional ^4-coupling events can always be unhatted, that is, we can take F and F' to be the original F and Y'. Theorem 7.1. Let (Y,Y') be a coupling of Y and Y' with distributional A-coupling events C and C. Then the underlying probability space can be extended to support events C and C such that (F,1C) = (F,16) and (Y',lc>) = (Y',lGl). In particular, p(ye-,c)U = P(r'e-,c")U. (7.2) PROOF. This follows immediately from Theorem 4.1 by regarding the F's as random elements in (E,A) rather than (E,£). D Say that a coupling has a single distributional ^4-coupling event if (7.1) holds with C" = C. Certainly, a nondistributional ^4-coupling event C is a single distributional ^4-coupling event. We next show that a coupling with two distributional ^4-coupling events can always be turned into a coupling with a single distributional y4-coupling event (the question of when such a coupling can be made nondistributional is discussed in Remark 7.1 below). Theorem 7.2. Let (Y,Y'j be a coupling of Y and Y' with distributional A-coupling events C and C'. Then the underlying probability space can be extended to support an event C and a Y" such that (Y,lc) = (Y,ld) and (Y",lc) = (Y',ld,) and (F, F") is a coupling of Y and Y' with C a single distributional A- coupling event.
Section 7. Coupling with Respect to a Sub-cr-Algebra 151 Proof. Let C be as in Theorem 7.1. Let V and W be random elements in (E,£) that are independent of the event C and have the distributions P(Y' G -\C') and P(Y' G -|C"C), respectively. Define Y" by Y" := V on C and Y" := W on Cc. D 7.2 ^4-Coupling Event Inequality — Distributional Maximality The ^4-coupling event inequality is a corollary to the distributional coupling event inequality. Theorem 7.3. // C and C are A-coupling events (distributional or not) of a coupling (Y,Y') ofY andY', then \\P(Y G -)U - P(^' e -)UII ^-COUPLING EVENT < 2P(CC). INEQUALITY Proof. This follows immediately from Theorem 4.3 by regarding the Y's as random elements in (E,A) rather than (E,£). D Call a coupling (Y,Y') maximal with respect to A if there is an ^4-coupling event C such that the ^-coupling event inequality is an equality. Call (Y, Y') distributionally maximal with respect to A if there are distributional ^4-coupling events C and C such that the inequality is an equality. Call the (distributional) ^4-coupling event(s) C (and C") maximal if this is the case. A coupling (Y, Y') with distributional ^4-coupling events C and C" is maximal in this sense if and only if [recall that _L denotes mutual singularity] P(Ye-,Cc)\A ± P(Y'e-,C'c)\A (7.3) and if and only if ||P(ye-)UAP(F'e-)UII=P(C); (7.4) see Section 8 in Chapter 3. Theorem 7.4. There always exists a coupling with a single distributionally maximal A-coupling event. Proof. Use the splitting extension of Section 5.1 in Chapter 3 as follows. Regard Y as a random element in (E, A) and take i/:=P(FGOUAP(K'G-)U to obtain a 0-1 variable / such that P(Y e -,I = i)U = v<Y e -)U aP(F' g -)U-
152 Chapter 4. STOCHASTIC PROCESSES In the same way we obtain a 0-1 variable I' such that P(F' e -,!' = 1)U = P(Y £ -)U A P(F' G -)U- Thus C = {/ = 1} and C" = {/' = 1} are maximal distributional A- coupling events of the (unhatted) coupling (Y,Y'). Apply Theorem 7.2 to complete the proof. □ Remark 7.1. Theorem 7.4 claims only the existence of a coupling that is distributionally maximal with respect to A. The existence of a coupling that is nondistributionally maximal with respect to A would follow if we found a way to turn a coupling with distributional ^-coupling events into a coupling with a nondistributional ^.-coupling event. When there exists a regular version of the conditional distribution of Y' given Y' regarded as a random element in (E,A), then the conditioning extension can be used to turn a coupling with distributional ^-coupling events into a coupling with an almost sure ^4-coupling event, that is, with an event C such that for AeA, P(f e A,Y" e A,C) = P(Y eA,C) = P(Y' e A,C). So the question is when the a.s. can be removed. According to the next theorem this can be done, in particular, when A is generated by a measurable mapping g taking values in a separable metric space equipped with its Borel subsets. 7.3 Gluing Together on a Function Value Note that if E is a separable metric space and £ its Borel subsets, then . £ <S> £ contains the diagonal {(y, y) : y € E}. (7-5) This can be seen as follows. By separability E has a countable dense subset A. For each e > 0 and a € A, let BE(a) be the open e-ball around a. Since £ is generated by the open sets, Be{a) € £, and thus BE(a) xBE(a) € £®£. Let A£ denote the union of Be(a) x Be{a) over a £ A. Since A is countable, Ae G £ ® £■ Since A is dense, the diagonal is contained in each Ae, and thus the diagonal is the decreasing limit of the Ae, which yields the desired result (7.5). We shall need the following result in the proof of Theorem 3.2 in Chapter 7. Theorem 7.5. Let Y and Y' be random elements in some measurable spaces (K,K.) and {K1 ,K,'), respectively. Let g and g' be measurable mappings from (K,K.) and (K',K.'), respectively, to a measurable space {E,£). Suppose there exists a weak-sense-regular conditional distribution of Y'
(7.6) Section 8. Exact Coupling - Another Proof of Theorem 6.1 153 given g'(Y') [holds when (K',IC') is Polish] and £ ® £ contains the diagonal {(y,y) : y € E} [holds if (E,£) is separable metric and £ its Borel subsets]. If 9(Y) = g'{Y'), then there exists a coupling (Y,Y') of Y and Y' such that g(Y)=g'(Y') and Y can be identified with Y. Proof. Take Y : — Y and obtain Y' by applying the transfer extension in Section 2.12 as follows. Take 7, := g{Y) and (Y{,Y£) := (g'(Y'),Y') and define Y' := Yi to obtain (g(Y),Y')^(g>(Y'),Y'Y This implies (g(Y),g'(Y'))^(g'(Y'),g'(Y')). (7.7) Since £®£ contains the diagonal {(y,y) : y e E}, the set {g{Y) = g'(Y')} is an event, and we obtain from (7.7) the second equality in F(g(Y) = g'(Y')) = P((g(Y), g'(Y')) €{(y,y):ye E}) = V{{g\Y'),g\Y1)) e {(y,y) : y G E}). Thus P(g(Y) — g'{Y')) = 1, and the desired result follows by deleting a null event. □ 8 Exact Coupling - Another Proof of Theorem 6.1 We now return to stochastic processes and use Theorem 7.4 to rephrase the proof of Theorem 6.1. 8.1 The Post-i a-Algebra Again consider continuous- or discrete-time stochastic processes Z and Z' having a general state space (E, £) and an arbitrary path space (H, %). We shall apply the above theory to the post-t a-algebra, the sub-u-algebra of H defined by Tt :=6>r1H = {£[0,t) xA:Aen}, 0^t<oo.
154 Chapter 4. STOCHASTIC PROCESSES Theorem 8.1. Let (Z, Z') be a coupling of Z and Z' and let 0 ^ t < oo. // T is a coupling time, then {T ^ t} is a Tt-coupling event. If T and T' are distributional coupling times, then {T ^ t} and {T' ^ t} are distributional Tt-coupling events. Proof. Take an arbitrary A G Tt and note that A = E^') x 6tA. Thus {Z e A} = {8tZ g 6tA} and {Z' G A} = {8tZ' G 6tA}, which yields (with T" = T in the nondistributional case) {ZeA,T^t} = {0tZ G M,T ^ t] = {6tpTZ G 9tA,T ^ t}, {Z1 eA,T'^t} = {8tZ' G 8tA,T' ^t} = {6t(3T>Z' G 8tA,T' ^ t}. In the nondistributional case the right-hand sides are identical, and in the distributional case they have the same probability. □ From Theorem 8.1 and the 7f-coupling event inequality we obtain ||P(ZG-)h-P(^'e-)klU2P(r>i), o ^ t < oo. (8.1) This inequality also follows from the coupling time inequality and the observation [Remark 5.1] that for 0 ^ t < oo, ||P(Z G 01-73 - P(Z' G Ok II = \\P(8tZ G •) - PtftZ' G -)||. (8-2) 8.2 Another Proof of Theorem 6.1 Due to Theorem 3.2, it suffices to establish the distributional part. We shall prove (a) and (&) simultaneously. To simplify notation in the continuous case we carry out the proof for tn = n; the proof for general tn is analogous (replace 0,1, 2,... by to, t\, £2, • • • throughout). From Theorem 8.1 and (8.1) and (8.2) we see that a distributional exact coupling is maximal at t if and only if the distributional 7f-coupling event {T ^ t} is maximal. This suggests using Theorem 7.4 (the existence of a maximal coupling with respect to a sub-cr-algebra) recursively to obtain a distributional exact coupling maximal at the integers. Let (Z(n),Z'("),Cn), 0 ^ n ^ 00, be independent triples with the following properties. Let (Z(°>, Z'(°>) be a coupling of Z and Z'with a single maximal distributional 7o-coupling event Cq. Recursively, for 0 < n < 00, let (Z(™\ Z'(n') be a coupling of processes with distributions P(Z<"-1) G -|C^-i) and P(Z'("-1) G -|C£-i) with a single (8.4) maximal distributional 7^-coupling event Cn.
Section 8. Exact Coupling - Another Proof of Theorem 6.1 155 Put T = inf{0 ^ n < oo : C„ occurs} [inf 0 := oo] and note that [due to the independence of the triples (Z("', Z'(n\Cn)] P(T>n)=P(C0c)---P(C£), (Kn<oo, and that [since P(Z<n+1) e •) = P(Z<n) G -|C£)], for 0 ^ n < oo, P(Z<n> e-) = P(^(n) g •,<?„)+ P(Z<n+1> g-)P(0 This yields [P(Z(°) G •) = P(Z G •) and Z<°) = Z<T) on C0 = {T ^ 0}] that the following holds for n = 0, P(Ze-)=P(#) e-,T^n)+P(Z("+1) e-)P(r>n), (8.5) and that if it holds for n, then it holds with n replaced by n + 1, since P(Z("+1» G-)p(T>«) = P(Z("+1> G -,C„+i)P(r > n) + P(Z<n+2> 6 -)P(C^+1)P(T > n) = P(Z<T> G-,T = n + l)+P(Z("+2) G-)P(r>n + l), where we have used the independence of (Z^n+1\Cn+\) and {T > n} = Cq Pi • • • n C^ for the second identity. Thus by induction (8.5) holds for all 0 ^ n < oo. Dropping the last term and sending n to infinity yields P(Ze-) ^P(Z(T) e-,r < oo). Similarly, P(Z' G •) ^ P(^'(T) G -,T < oo). Let Z(°°) and Z'(co) be independent with arbitrary distributions when P(T = oo) = 0 and with the following distributions when P(T = oo) > 0 : P(Z(co' G •) := (P(Z G •) - P(Z(T) G -.T < oo))/P(T = oo), P(Z'(oo) G •) := (P(Z' G ■) - P(Z'(T» G -,r < oo))/P(T = oo). Then (Z(T),Z'(T)) is a well-defined coupling of Z and Z'. Further, for 0 ^ n < oo and AeTi, P(0TZ<T) GA,T = n) = P(Z(") G 0-xA,C„)P(r £ n) [independence] = P(Z'(n) G 9-lA,Cn)Y(T > n) [Cn single T„-event] = P(6»TZ'(T) € A,T = n) [independence]
156 Chapter 4. STOCHASTIC PROCESSES and thus T is a single distributional coupling time. Finally, due to (8.5), P(Z(T> e -,T > n) = P(Z("+1> G -)P(T > n) and similarly P(Z'(T) G -,T > n) = P(Z'(n+1) G -)P(T > "). Since P(Z("+1) g -)k -L P(Z'(n+1) G -)k, this yields that P(Z<T> G-,r>n)|r„ ±P(Z'(T) G-,r>n)|rn, that is, {T > n} is a maximal distributional 7^-coupling event. This completes our second proof of Theorem 6.1. Comment. Lemma 6.1 is implicitly re-proved in the second sentence of the proof of Theorem 7.4. 9 Exact Coupling — Tail cr-Algebra - Equivalences In this section we introduce the tail u-algebra, which is intimately linked to exact coupling, and establish a basic set of equivalences. 9.1 The Tail a-Algebra The tail a-algebra is the decreasing limit of the post-i u-algebras T:= |-| 71. For an example of a set in T let B be some set in £ and put A\ = {z G H : Zk G B for infinitely many integers k}. More generally, if B0, B\,... is a sequence of sets in £, then A2 = {z G H : Zk G Bk for infinitely many integers k ^ 0} is in T. For real-valued processes we can, for instance, take Bk = {&}• Then in discrete time A2 is the set where the space-time diagonal is visited infinitely often. Restrict attention to the continuous-time shift-measurable case. Then, with B G £, A3 = {z G H : {s ^ 0 : zs G B} has infinite Lebesgue measure} is in T. And for real-valued processes, for instance, A4 = {z G H : {s ^ 0 : zs = s} has infinite Lebesgue measure} is in T.
Section 9. Exact Coupling - Tail a-Algebra - Equivalences 157 9.2 The Inequality The following theorem explains what the tail u-algebra has to do with exact coupling. (Recall that \x\a denotes the restriction of a measure /j, to a sub-u-algebra A.) Theorem 9.1. Let (Z,Z') be a coupling of the discrete- or continuous- time stochastic processes Z and Z' with a general state space (E, £) and a general path space (H,H). If T is a coupling time, then {T < oo} is a T- coupling event. IfT andT' are distributional coupling times, then {T < oo} and {T' < oo} are distributional 7-coupling events. In both cases ||P(Z G -)\t ~ P(Z' G -)lrll ^ 2P(T = oo). (9.1) Proof. Consider first the nondistributional case. Since T is contained in each 7f, we have by Theorem 8.1 that for 0 ^ t < oo, {Z eB,T ^t} = {Z'eB,T^t}, BeT, and thus sending t —> oo renders the desired result {Z e B,T < oo} = {Z'e B,T < oo}, B eT. In the distributional case the first of these identities (with T" instead of T on the right) holds in distribution due to Theorem 8.1 and thus so does the second. The inequality follows from Theorem 7.3. □ Note that we cannot expect a coupling with coupling time T to have coupling event {T < oo}, since this would imply ||P(Z G •) - P(Z' G -)|| ^ 2P(T = oo), while it is even possible that ||P(Z G •) - P(Z' G -)ll = 2 and P(T = oo) = 0. 9.3 Maximally Successful Exact Coupling Clearly, the exact coupling in Theorem 6.1 that is maximal at, for instance, integer times is also maximally successful, that is, attains the supremum of the success probabilities over all exact couplings. The converse is not true, since if T is the time of an exact coupling'that is maximal at integer times, then replacing T by T+1, for instance, yields an exact coupling that is not maximal at integer times; however, it is still maximally successful because P(T+1 = oo) = P(T = oo). We shall now establish that this maximally successful exact coupling in fact yields a maximal T-coupling event (attains identity in (9.1)), which in particular shows that maximal success probability = ||P(Z G -)lr A P(Z' G -)lr||-
158 Chapter 4. STOCHASTIC PROCESSES Theorem 9.2. Let Z and Z' be one-sided discrete- or continuous-time stochastic processes with a general state space (E, £) and a general path space (H,H). The distributional exact coupling (Z, Z',T,T') of Z and Z' in Theorem 6.1, which is maximal at the integers, is such that {T < oo} and {T" < oo} are maximal distributional T-coupling events, that is, ||P(Z G -)\T - P(Z' G Olrll = 2P(T = oo). (9.2) Proof. Since (Z, Z',T,T') is maximal at the integers, we obtain from Theorem 8.1 together with (8.2) and (7.3) that P(ZG-,T>n)|Tn ± P(Z'e-,T'>n)|r„. Since {T = oo} C {T > n}, this implies that P(Z G -,T = oo)|Tn and P(Z' G -,T" = oo)|rn are also mutually singular, that is, 3An G Tn : P(Z G An, T = oo) = 0 and P(Z' G Acn, T = oo) = 0. Put CO CO A = limsupylri := f] \J Ak "^°° n=Ok=n and note that A eT and that Ac = liminfn^oo A^ to obtain 3A G T: P(Z G A, T = oo) = 0 and P(Z' G Ac, T' = oo) = 0, that is, P(Z G -,T = 00)7- and P(Z' G -,T' = 00)7- are mutually singular, which is equivalent to (9.2). □ 9.4 A Total Variation Limit Result The following theorem explains what the tail u-algebra has to do with total variation convergence. Theorem 9.3. Let Z and Z' be one-sided discrete- or continuous-time stochastic processes with a general state space (E, £) and a general path space (11,%). Then as t —> 00, ||P(0(Z g •) - P(etZ' G Oil -+ l|P(Z G Olr - P(Z' G Olrll- Proof. Let T be as in Theorem 9.2 and send t -* 00 in the coupling time inequality (Theorem 5.1) to obtain (due to (9.2)) that limsup \\P{OtZ GO- P(0tZ' G Oil ^ ||P(Z G Olr - P(Z' G Olrll- t—>co Since T is contained in each Tt, we have ||P(Z g Olr - P(^' G Olrll ^ ||P(Z G Ok - P(Z' G Okll,
Section 9. Exact Coupling - Tail cr-Algebra - Equivalences 159 and since the right-hand side equals ||P(0tZ G ■) — P(8tZ' G -)||, we have ||P(Z G -)lr - P(^' G Olrll ^ Hminf ||P(6>tZ G ■) - P(6»tZ' G Oil- The first and last inequality yield the desired result. □ REMARK 9.1. By a similar argument we can obtain the inequality (9.1) directly: ||P(Z G -)|r - P(Z' G -)|rll ^ HP(fl^ GO- P{etZ' G Oil < 2P(T> t) -+2P(T = oo), i ->■ ex), without the concept of coupling with respect to a u-algebra. 9.5 Equivalences We can now tie together exact coupling, total variation convergence, and the tail u-algebra as follows. Theorem 9.4. Let Z and Z' be one-sided discrete- or continuous-time stochastic processes with a general state space (E,£) and a general path space (H,T-C). The following statements are equivalent. (a) There exists a successful distributional exact coupling of Z and Z'. (b) \\P(9tZ €■)- P(0tZ' G Oil -> 0 as t ->■ oo. (c) P(ZGOIr = P(Z'GO|r- Moreover, these statements are equivalent to the existence of a successful nondistributional exact coupling if there exists a weak-sense-regular conditional distribution of Z given firZ for any random time T [this holds in discrete time when (E, £) is Polish and in continuous time when (E, £) is Polish and the paths are right-continuous}. PROOF. By the coupling time inequality, (a) implies (&), see (5.1). By Theorem 9.3, (6) implies (c). By Theorem 9.2, (c) implies (a). The final claim of the theorem follows from Theorems 3.1 and 3.2. □
o
Chapter 5 SHIFT-COUPLING 1 Introduction The previous chapter dealt with coupling one-sided stochastic processes in such a way that their paths eventually merge. This we called exact coupling to distinguish it from the more general shift-coupling to be considered in this chapter. Shift-coupling means that the paths eventually do not merge 'exactly' but only modulo a random time shift. In this chapter we shall also consider an issue that arises only in continuous time: what happens when the random time shift can be made arbitrarily small, that is, when epsilon couplings exist. It turns out that both shift-coupling and epsilon-couplings have a theory paralleling that of exact coupling: they can be linked to a mode of convergence (Cesaro and smooth total variation convergence, respectively) and to a (7-algebra (the invariant and the smooth tail u-algebra, respectively) in the same way as exact coupling is linked to plain total variation convergence and to the tail cr-algebra. For both shift-coupling and epsilon-coupling we introduce inequalities that play the same- key role as the coupling time inequality in the exact coupling case. In order to stress the similarities (and the dissimilarities) between these three types of coupling and to make comparison easier, the treatment of first shift-coupling (Sections 2 through 5) and then epsilon-couplings (Sections 6 through 9) is organized in the same way as that of exact coupling (Sections 3, 5, 6, and 9 in Chapter 4): the sections have analogous titles, and the subsections and theorems are enumerated in the same way (when possible). We start with a section defining the concept and its distribu- 161
162 Chapter 5. SHIFT-COUPLING tional version, continue with a section presenting the inequality and the resulting limit theory, then move on to a section discussing the question of maximality, and finish with a section introducing the u-algebra and the basic set of equivalences between the coupling, the total variation result, and the u-algebra. Throughout this chapter U is a random variable that is uniform on [0, l] and independent of the processes and the shift-coupling (epsilon couplings). And note that in the continuous-time case we now impose the shift-measurability condition throughout. 2 Shift-Coupling - Distributional Shift-Coupling This section introduces shift-coupling and its distributional version. 2.1 Shift-Coupling - Definition Let Z and Z' be one-sided discrete-time or continuous-time shift-measurable stochastic processes with general state space (E,£) and path space (H,l-t); see Section 2 in Chapter 4. We shall use notation in accordance with continuous time, but all we need to switch to discrete time is to substitute, for instance, t by n and s by k and to introduce the following convention: in the discrete-time case extend the definition of the shift-maps to noninteger times by etz = e[t]z, te[0,oo), z = (z0,z1,...)eE^1'-K A shift-coupling of Z and Z' is a quadruple (Z, Z',T,T') where (Z, Z') is a coupling of Z and Z' and T and T" are two random times (integer-valued in the discrete-time case) such that 8TZ = 8T'Z' on {T < oo} (2.1) and {T <oo} = {T' < oo}. Using the convention that a process is absorbed in the cemetery state when shifted to infinity we can rewrite (2.1) simply as 8TZ = 6T'Z' (see Figure 2.1). The times T and T" are the shift-coupling times, P(T < oo) is the success probability, and the shift-coupling is successful if P(T < oo) = 1. When T < oo, then T - V is the shift. There is no shift if T = T', and then the shift-coupling is an exact coupling.
Section 2. Shift-Coupling - Distributional Shift-Coupling 163 The bold parts are identical. W^ 0 T FIGURE 2.1. Nondistributional shift-coupling: merging modulo a time shift. 2.2 Distributional Shift-Coupling — Definition Say that (Z, Z', T, T') is a distributional shift-coupling of Z and Z' if (Z, Z') is a coupling of Z and Z', and T and T" are nonnegative random times such that j*yZ — ut' Z ; (2.2) here we again use the convention that the shifted processes are absorbed in the cemetery state when T and T' are infinite. Certainly a shift-coupling is also a distributional shift-coupling. We shall use the word nondistributional to distinguish a shift-coupling from a distributional one. Otherwise, we use the same terminology in both cases. For an example of a successful distributional shift-coupling consider two independent differently started versions, Z and Z', of a countable state space irreducible recurrent Markov chain and let T and T' be the times when Z and Z', respectively, first hit a fixed state. Then (Z, Z',T,T') is a distributional shift-coupling of Z and Z'. A nondistributional shift-coupling is obtained by letting the chains continue in the same way after hitting the state. Note that (2.2) implies nothing about T and T" except that P(T < oo) = P(T' < oo). It is an interesting observation, however, that a distributional shift-coupling of the space-time processes (-Zs, s)sg[o,co) and (Z's, s)s6[o,co) is a distributional exact coupling of Z and Z', since 9t(Zs, s)se[o,co) — #T'(Zs,s)se[o,co) is equivalent to (6tZ,T) = (8t'Z',T'). And in the nondistributional case a shift-coupling of the space-time processes is equivalent to T — T' and eTz = eTz'.
164 Chapter 5. SHIFT-COUPLING 2.3 The Hats May Be Dropped in the Distributional Case If we have a distributional shift-coupling of Z and Z', then we can take Z and Z1 to be the original processes Z and Z1. Theorem 2.1. Suppose (Z, Z',T,T") is a distributional shift-coupling of Z and Z'. Then the underlying probability space can be extended to support random times T and T' such that {Z, T) = (Z, f) and (Z', T') = (Z', f'). In particular, 6tZ = 6t>Z'. (2.3) PROOF. This follows from the transfer extension in Section 4.5 of Chapter 3. In order to obtain T take Fx := Z and (F/,F2') := (Z,T) and define T : = Y2. Similarly, in order to obtain T' take Yx := Z' and {Y{,Y2') : = (Z',f") and define T':=y2. D This theorem motivates again dropping the hats when discussing distributional shift-coupling, when there is no danger of confusion. Say that T and T' are distributional shift-coupling times of Z and Z' if (2.3) holds. 2.4 Turning Distributional into Nondistributional In the standard settings a distributional shift-coupling can always be turned into a nondistributional one. Theorem 2.2. Let (Z, Z', T, T') be a distributional shift-coupling of Z and Z'. Suppose there exists a weak-sense-regular conditional distribution of Z' given 9f,Z' [this holds in discrete time when the state space is Polish and in continuous time when the state space is Polish and the paths are right-continuous]. Then the underlying probability space (ft, J7,P) can be extended to support T, Z", and T" such that {Z,T) = (Z,f) and (Z",T") = {Z',f>) and (Z, Z",T,T") is a nondistributional shift-coupling of Z and Z'. Proof. Let T be as in Theorem 2.1. To obtain (Z",T") use the transfer extension in Section 2.12 of Chapter 4 as follows. Take Y\ := 8tZ and (Y{,Y^) := (6f,Z',Kf,Z') to obtain Y2 such that [see Section 2.9 of Chapter 4 for the definition of the killing maps nt] (eTz,Y2)B{et,z',Kt,z'). Define {Z",T") by (6t»Z",kt»Z") := {9TZ,Y2) (thus 9TZ = 0T»Z").
Section 3. Shift-Coupling - Inequality and Asymptotics 165 Since {eT„Z",KT»Z") is a copy of (eflZ',KflZ'), it follows that (Z",T") is a copy of (Z',T') because (Z", T") is determined in the same measurable way by {eT„Z" ,kt»Z") as {Z',f') is by (9f,Z',Kf,Z'). D 3 Shift-Coupling — Inequality and Asymptotics The last section was devoted to the definition of shift-coupling and its distributional version. We shall now go on to the limit implications. 3.1 Shift-Coupling Inequality — And Its Reformulations Rather than shifting Z to a nonrandom t as in the coupling time inequality we now shift to a point picked uniformly at random in [0, t\. Theorem 3.1. Let Z and Z1 be one-sided discrete-time or continuous-time shift-measurable stochastic processes with a general state space (E,£) and path space (H,"H). If there is a distributional shift-coupling of Z and Z' with times T and T', then the underlying probability space can be extended to support a copy R ofT' such that {T < oo} = {R < oo} and, forO ^ t < oo, \\P(9UtZ 6 •) - P(QutZ' 6 -)ll SHIFT-COUPLING ^ 2P(T V R > Ut), INEQUALITY where U is uniform on [0,1] and independent of Z, Z', T, and R. In the nondistributional case we can take R := T' and the shift-coupling inequality becomes \\P{6UtZ e •) - P(0utZ' € OIK 2P(T V T' > Ut). PROOF. With {/independent of the shift-coupling (Z,Z',T,T'), note that the remainder when T + Ut is divided by t, (T + Ut) mod t := {T/t + U- [T/t + U])t, is uniform on [0,t] and independent of Z. Therefore, 9(r+ut) mod t% 1S a copy of 6UtZ. Similarly, 0(T'+ut) mod tZ' is a copy of 9UtZ'. Thus {9(T+Ut) mod tZ,6(T' + Ut) mod t% ) \^ ) is a coupling of dutZ and dutZ'. In the nondistributional case 9TZ = 9t'Z' yields the second identity in 0(T+ut) mod tZ = Out^rZ — 9ut9r'Z' = 0(T'+Ut) mod tZ' On {Ut^t-TVT},
166 Chapter 5. SHIFT-COUPLING while the other two follow from the fact that 'mod V can be removed on both sides when Ut ^t-TVT'. Thus {Ut ^ t - T V T'} is a coupling event of the coupling at (3.1), and the coupling event inequality [see (8.16) in Chapter 3] yields the desired result, since ¥{Ut >t-TVT') = P(T V T' > (1 - U)t) = P(T V T' > Ut). In the distributional case, 6tZ = 8t'Z' allows us to apply the conditioning extension in Section 4.5 of Chapter 3 as follows. First take (lo,^i) '■ = {{Z,T),eTZ) and (Y{,Y{) := {0T>Z',T') and define R := Y2. Then take {YcYi) := {{Z',T'),eT,Z') and {Y{,Y±) := {9TZ,T) and define R' := Y2. This yields (6TZ,T,R)^(8T,Z',R',V), which in turn yields the second identity in P(0(T+£/t) mod J € -, Ut < t - T V R) = P(eut0TZ £-,Ut^t-TVR) = ¥{9ut9T,Z' e-,Ut^t-T'vR') = P(9(T' + Ut) mod tZ'€;Ut^t-T'\/ R'). Thus {Ut ^ t - T V R} and [Ut ^ t - T'V R1} are distributional coupling events of the coupling at (3.1), and the distributional coupling event inequality yields the shift-coupling inequality. □ Reformulations. The left-hand side (l.h.s.) of the shift-coupling inequality can clearly be rewritten in the following Cesaro (time-average) form in the continuous-time case, l.h.s. = \ f P{9SZ e •) *» - 7 / P{9SZ' €-)ds , (3.2) t Jo Wo and in the discrete-time case (recalling that then t = n and 9unZ = 9{Un}Z), II 1 n 1 ™ l.h.s. =\\-'Vp{0kZe-)--'yiP{8kz'e-) \\n t-*1 n ^—' o o The right-hand side (r.h.s.) can be rewritten in several ways: 'TV R r.h.s. = P ( ?-~ >*)= YEKT V R) A t) = E t Al (3.3) (3.4)
Section 3. Shift-Coupling - Inequality and Asymptotics 167 here the first and last equalities are obvious, and the one in the middle follows from tP{TvR> Ut) = E[ / 1{Tvr>s} ds) Jo /•CO = E[/ l{{TvR)At>s]ds]=E[(TvR)At}. Jo Since (T V R) A t ^ T + R, we have in particular, (since E[R] = E[T']) / V{9SZ £-)ds- f P{9SZ' e-)ds ^ 2(E[T] + E[T'\). (3.5) Jo Jo The analogy between the coupling time inequality and the shift-coupling inequality is stressed in a different way by the following reformulation of the latter: for 0 «C t < oo, / P(0sZ e •) ds - ( P(8SZ' e-)ds ^ 2 / P(T V R > s) ds. Jo Jo Jo 3.2 Finite T — Cesaro Total Variation Convergence In the same way as the coupling time inequality is basic for plain total variation asymptotics, the shift-coupling inequality is basic for Cesaro (or time-average) total variation asymptotics. If there exists a successful shift-coupling, then clearly P l—jj- > t) -*0, *-Noo and thus WP{OutZe-)--p{eutZ'e-)\\-+o, t^oo. (3.6) In particular, if Z' is stationary, then OutZ' has the same distribution as Z', and (3.6) can be rewritten as dutZ —> Z', t —> oo, (Cesaro asymptotic stationarity). 3.3 Finite Moments of T and T" — Rates of Convergence If a 6 (0,1) and both E[Ta] and E [T'a] are finite, then, since U and (T, R) are independent and (T V R)a = Ta V Ra ^ Ta + Ra, E TVfl U E[[/-a]E[(T V R)a] ^ E[U-a]E[Ta + Ra],
168 Chapter 5. SHIFT-COUPLING which yields, since E[[/-a] < oo for a < 1 and E[Ra] = E[T'a], that TViT " E U < oo. Thus (see Section 5.3 in Chapter 4) taP (^^ > t) ->■ 0 as £ -> oo, and the shift-coupling inequality yields convergence of power order a 0 < a < 1 and E[Ta] and E[T'a] < oo => ta||P(0t/tZe-)-P(0t/^'G-)ll->o, *^oo. (3.7) From the inequality we cannot obtain rates of order a = 1 or higher. What can be deduced from (3.5), however, is the following boundedness result: E[T] and E[T'] < oo sup 0<t<co / P{9SZ e-)ds- f P{9SZ' Jo Jo ds < oo. 3.4 Finite Moments of T and T" — Moment Rates of Convergence If a 6 (0,1) and both E[Ta] and E[T'a] are finite, then we have just shown that E[(-^j^) ] < oo, which yields a stronger result than (3.7), namely a rate of convergence of power moment-order a — 1: 0 < a < 1 and E[Ta] and E[T'a] < oo /•OO => / t^WPidutZe^-PieutZ'e-)\\dt< Jo oo; see Section 5.4 in Chapter 4. (In discrete time the integral is replaced by a sum.) 3.5 Stochastically Dominated T and T' — Uniform Convergence Let Z be a class of discrete- or continuous-time shift-measurable stochastic processes on a general state space. Suppose there exist finite random variables T and T' such that for all pairs of processes Z, Z' € Z there is a shift-coupling (distributional or not) of Z and Z' with times T and T" such that D _ D _ T ^T and T' < T'. Since P(T V R> Ut) ^ P(T > E/£) + P(# > C/t) and since P(i? > f/i) = P(T' > C/0, we have P(T V i? > C/i) < P(f > C/t) + P(f" > C/t)- ThuS the shift-coupling inequality yields that for 0 < t < oo and Z,Z' & Z, \\P(dutZ € •) - P(0t/tZ' € -)ll < 2P(f > t/i) + 2P(f' > t/i),
Section 4. Shift-Coupling - Maximality 169 and we obtain uniform convergence over the class Z: sup ||P(9[/tZ6-)-P(W G-)l| -+0, £->oo. Z,Z'£Z Rates of convergence are obtained in the same way as in the previous subsection under conditions on T and T" rather than on T and T'\ 0 < a < 1 and E[Ta] and E[T'a] < oo => ta sup ||P(^Ze-)-P(^c/^'e Z,Z'£Z 0, t -^ oo, and in the continuous-time case we have the stronger result 0 < a < 1 and E[fa] and E[f"a] < oo /•OO => / T"1 sup \\P{eUtZ e-)-P{9UtZ'e-)\\dt<oo, JO Z,Z'£Z while in the discrete-time case 0 < a < 1 and E[fa] and E[f"a] < oo CO => Vfc""1 sup ||P(6>[l/fc]Ze-)-P(0[t/*]^'G- j Z,Z'£Z Also, due to (3.5) and E[T] < E[f] and E[T'] < E[f'], E[f] and E[f'] < oo / P(8sZe-)ds- [ P(8sz'e-)ds Jo Jo < oo. sup 0<t<oo Z,Z'£Z < oo. 4 Shift-Coupling - Maximality In this section we shall not establish a shift-coupling analogue of maximal exact coupling. We only establish a result that enables us to show in the next section that there is a shift-coupling that is both maximally successful and also successful when the Cesaro total variation convergence (3.6) holds. But we will not be able to reverse the rate results in the previous section. The maximality question is further discussed in Section 4.5 below. 4.1 The Maximality Theorem The right-hand side of the shift-coupling inequality is nonincreasing, but the left-hand side need not be. For a simple counterexample let Z be a nonrandom periodic function with period d > 1 and put Z' — 9\Z; then
170 Chapter 5. SHIFT-COUPLING \\P(8utZ € •) - ¥(9utZ' € -)|| = 0 or > 0 according as t/d is an integer or not. Thus defining maximal shift-coupling by demanding identity at all times, even at integer times, is not without complications, and we shall not proceed further along that path. Recall, however, that for exact coupling, maximality is equivalent to P{9tZ €-,T >t) and P{9tZ' € ■ ,V > t) being mutually singular for all t. The following shift-coupling analogue of this reformulation is considered by Greven (1987): in discrete time there exists a distributional shift-coupling (Z,Z',T,T') such that OO OO ^P(SnZ6-,T>n) 1 ^P(0„Z'e-,T'>n). (4.1) o o This is a strong property, which tells us that for all times n and n' the processes 9nZ and 9niZ' stay in separate parts of the path space prior to merging. Here we shall content ourselves with the following weaker maximality property, which says only that for all times t and t' the processes 9tZ and 9t'Z' stay in separate parts of the path space if they do not merge at all. This property, however, is all we need for the next section, and the result has the merit of not being restricted to discrete time. Theorem 4.1. Let Z and Z' be one-sided discrete-time or continuous- time shift-measurable stochastic processes with a general state space (E, £) and path space (H,T-L). Then there exists a distributional shift-coupling (Z,Z',T,T') of Z and Z' such that /•OO />00 / P{9tZ e-,T=oo)dt ± / P(6tZ' e-,T' = oo)dt. (4.2) Jo Jo Moreover, there exists a nondistributional shift-coupling of Z and Z' with this property if there exists a weak-sense-regular conditional distribution of Z given 9tZ [this holds in discrete time when (E,£) is Polish and in continuous time when (E,£) is Polish and the paths are right-continuous}. We prove this result in the next three subsections. 4.2 First Part of Proof — Construction of a Candidate Let V\, V2,... be i.i.d. exponentially distributed random variables that are independent of the sequence of independent quadruples (Z^k\ Z'^k\Ck, C'k), 1 < k < 00, which have the following properties. Let (Z*1), Z"^>) be a coupling of Z and Z' and let Cx and C[ be maximal distributional coupling events of (9y1Z(-1\ 9v1Z'^). This is possible because we can first let (Z^,Z'^) be a coupling of Z and Z' and then use Theorem 4.4 in Chapter 4 [with Y :— 9vxZ^ and
Section 4. Shift-Coupling - Maximality 171 Y' := (Vi-Z^1)] to obtain C\ and C[. In the same way we can recursively, for 1 < k < oo, let (Z(fc),Z'(fc)) be a coupling of processes with distributions P(Z(fc-1) £-|C^_1)andP(Z'(fc-1) e-\C'%_^ aadletCtand C'k be maximal distributional coupling events of (9ykZ^k\ 9vkZ'^). Put K = inf{1 ^. k < oo : Ck occurs} [inf 0 := oo] and note that [by the independence of the quadruples (Z(fc), Z'^k\ Ck, C'k)\ P{K>k)=P{Cc1)...P{Cck), l^fc<oo, and that [since P(Z<fc+1) € •) = P{Z^ € -\Cck)\ P(Z(fc) e-)=P(^(fc) e-,Cfc) + P(Z<fc+1) e-)P(^), l ^ fc < 00. This yields [P(Z(1) e •) = p(z € •) and Z*1) = Z^ on d = {K «C 1}] that the following holds for k = 1 P(Z e •) = P(Z<*> e -, AT < Jfc) + P(Z<fc+1> e -)P(^ > *0 (4-3) and that if it holds for some k, then it holds with k replaced by k +1, since P(z(fc+1) e-)P{K> k) = P(Z(fc+1) e -,Ck+1)P{K >k) + P(Z(fc+2) e -)P(C£+1)P(iir > k) = p(zW e-,ii" = fc + i)+P(z(fc+2) e-)P(tf> Jfc + i), where we have used the independence of (Z(fc+1), Cfc+1) and {if > k} = C{ fl • • • fl Ck for the second identity. Thus by induction (4.3) holds for all 1 «C k < oo. Drop the last term and send k —> oo to obtain P(Ze-) ^P(zW e-,ii"<oo). Similarly, with K' = inf{l ^ k < oo : C'k occurs} we obtain P(Z'e-)^P(^'w e-,if'<oo). Put ^ = 00. Let Z(°°) and Z'(°°> be independent of (V*, Z<fc), Z'(fc), Cfc, C'k), 1 ^ fc < oo, with arbitrary distributions when P(K < oo) = 1 and with
172 Chapter 5. SHIFT-COUPLING the following distributions when P{K < oo) < 1: pr7(oo) G , = P(Ze.)--p(zWe;K<oo) { j P(K = oc) (4-4) pr7,(oo) G s = P(Z'e-)-P(Z'(«)e-,K><oc) { ' P{K' = oo) We shall show that {Z,Z',T,T') := {Z(K\Z'(K),VK,Vji) [the candidate] is a distributional shift-coupling satisfying (4.2). 4.3 Middle Part of Proof - The Candidate Is a Shift-Coupling Since K is independent of Z(°°) and K' of Z'(°°\ it follows from (4.4) that (Z, Z') is a coupling of Z and Z'. For 1 «C k < oo, we have P(9TZ e-,K = k) = P{eVkZM € ;Ck)P(K > k), P(8T'Z' €-,K' = k) = P{9VkZ'W e-,C'k)P{K' >k). Since Cfc and C'k are distributional coupling events of {9vkZ^k\9vkZ'^), and since P{K ^ k) = P(K' ^ fc), the right-hand sides are identical, and summing over 1 ^ k < oo yields [since {T < oo} = {K < oo} and {V < oo} = {K1 < oo}] P(9TZ e -,T < oo) = P(6T>Z' e -,T' < oo). Thus (Z, Z',T,T') is a distributional shift-coupling of Z and Z'. 4.4 Final Part of Proof - The Candidate Satisfies (4.2) The mutual singularity of P(%Z(fc) € -,Cck) and P{9VkZ'^ € -,C"£) means that there is an Ak & % such that P(9VkZ^ e Ak) =P(9VkZ^ e Ak,Ck), P(9Vkz'^ e A£) = P(%z'(fc) e Al,C'k). From (4.3) we obtain the equality in P(Ze-,T = oo) <P(Z<*) e.iK^k) (4.6) = P{Z^ e-)P(K^k), l«Cfc<oo.
Section 4. Shift-Coupling - Maximality 173 Let V be a copy of Vk and be independent of the shift-coupling. Then P(8VZ e Ak,T = oo) ^ V{9VkZ(k) e Ak)P{K > k) [due to (4.6)] = P{8VhZW E Ak,Ck)P(K > k) [due to (4.5)] < P(Ck)P(K >k) = P(K = k), and thus CO p(evZ e (J Ak,T = ooW P{n ^ K < oo) ->■ 0 as n ->■ oo. fc=n Put A = limsupj..^ ^ to obtain P{dvZ e A,T = oo) =0. Since V has a density with respect to Lebesgue measure that is strictly positive on [0, oo) and is independent of (Z,T), we can write this as P{6tZ e A,T= oo)dt = 0. Since liminffc^oo Ack is the complement of A, we obtain similarly P{9tZ' eAc,T' = oo)dt = 0. Thus (4.2) holds, and the proof of Theorem 4.1 is complete. 4.5 Remarks on Maximality If T and T' are distributional shift-coupling times, then so are T + Y and T' + Y for any nonnegative (integer-valued in the discrete case) random variable Y that is independent of the shift-coupling (independence is not needed in the nondistributional case). Furthermore, if Y is finite, then {T + Y = oo} = {T = oo}, and thus (4.2) holds if and only if it holds with T and V replaced by T + Y and T + Y. Thus (4.2) does not tell us anything about behaviour in finite time. Greven's maximality property (4.1) is clearly stronger than (4.2), since the measures in (4.1) contain those in (4.2). But it is not strong enough to be a full-fledged shift-coupling analogue of maximal exact coupling, as can be seen from the following example. Let Z be an irreducible recurrent Markov chain starting in a fixed state x. Let Z' be a Markov chain with the same transition probabilities starting from a different state. Take an n ^ 0 and let T'n be the time of the first visit of Z' to x after time n. Then 0 and T'n are distributional shift-coupling times of Z and Z', and (4.1) certainly holds (the left-hand side is 0). Since T'n ^ n, where n can be
174 Chapter 5. SHIFT-COUPLING chosen arbitrarily large, this is not a sufficiently sharp maximality property: it should have picked Tq. In particular, we can see from the above example that (4.1) on its own is not sufficient to reverse the rate results in the last section. We can find Z and Z' such that Tq = 1, and Section 3.4 yields Cesaro total variation convergence with (for instance) moment rate of order —|. But (4.1) holds for T = 0 and T = T'N where N is independent of Z' with E[N^2] = oo which implies E[T;1/2] = oo. 5 Shift-Coupling - Invariant a-Algebra - Equivalences In this section we introduce the invariant cr-algebra, which is linked to shift-coupling in the same way as the tail cr-algebra to exact coupling, and establish an analogous set of equivalences as in the exact coupling case. 5.1 The Invariant a-Algebra The invariant a-algebra consists of path sets in % that do not depend on where the time origin is placed. It is defined as follows: 1= {A e n : e'1 A = A,0 ^ t < oo}. This is a cr-algebra because if A is the union of sets Ak satisfying O^Ak = A/., then 9^1A = A (certainly 1 is closed under complementation and contains H). Since AeV. and A = 9^1A imply A G O^H = Tt, we have 1CT. For examples of sets in 1, and not in 1, consider the sets A\, A2, A3, At in Section 9.1 of Chapter 4. In the discrete-time case A\ G X but A2 £ 1. In the continuous-time case A3 G 1 but A\, A2, and A4 $.T. Thus in general the inclusion is strict. We note at this point the following pleasant result. Lemma 5.1. Let Z be a discrete-time or continuous-time shift-measurable stochastic process with a general state space (E,£) and path space {!!,%)■ IfT is a random time, then {z e A,T< 00} = {6Tz e A,T < 00}, Aei. In particular, ifT is finite, then {Z G A} = {8tZ G A] for A el. Proof. With A el and 0 ^ t < 00 we have {Z eA,T = t} = {Ze 9^A,T= t} = {9tz e a,t = t} = {eTz eA,T = t}.
Section 5. Shift-Coupling - Invariant <T-Algebra - Equivalences 175 Take the union over 0 ^ t < oo to obtain the desired result. (Note that uncountable union is no problem here because the union itself is measurable.) □ 5.2 The Inequality The following theorem explains what the invariant cr-algebra has to do with shift-coupling. Theorem 5.1. Let (Z, Z') be a coupling of the discrete-time or continuous- time shift-measurable stochastic processes Z and Z' with a general state space (E,£) and path space (H,TL). If T is a shift-coupling time, then {T<oo} is aX-coupling event. IfT andT' are distributional shift-coupling times, then {T < oo} and {T' < oo} are distributional X-coupling events. In both cases ||P(Z g -)|i - P(Z' e -)|z|| ^ 2P(T = oo). (5.1) Proof. For A G X we have, due to Lemma 5.1, {Z G A,T < 00} = {9TZ eA,T< 00}, {Z1 G A,T < 00} = {Or,Z' G A,T' < 00}. In the nondistributional case the right-hand sides are identical, and in the distributional case they have the same probability. And (5.1) is the I- coupling event inequality (Theorem 7.3 in Chapter 4). □ 5.3 Maximally Successful Shift-Coupling We shall now show that the coupling in Theorem 4.1 yields equality in (5.1). Thus there is a maximally successful shift-coupling (attaining the supremum of the success probabilities over all shift-couplings), and maximal success probability = ||P(Z G 0|z A P(Z' G -)|ill- Theorem 5.2. Let Z and Z' be one-sided discrete-time or continuous-time shift-measurable stochastic processes with a general state space [E, £) and path space {H,%). The distributional shift-coupling (Z,Z',T,T') of Z and Z' in Theorem 4-1 is such that {T < 00} and {T1 < 00} are maximal distributional X-coupling events, that is, ||P(Z G -)|i ~ P(Z' G Olill = 2P(T = 00). (5.2) Proof. According to Theorem 4.1 there is a set A E H such that oo P(6»(ZG A,T = 00) dt = 0, CO P(0(Z'G AC,T = 00) dt = 0,
176 Chapter 5. SHIFT-COUPLING which we can rewrite as E E Put /•OO [/ 1{etzeA}dt'T=0O}=°> /•OO IJ 1{9lz'eA^}dt'T' = oo]=0. (5.3) /•OO B = {ze H : / l{etzeA} dt = oo} Jo and note that JQ l{stZ£A} dt < oo implies f™ l{stZ£Ac} dt = oo and thus /•CO Bc = {zeH: / l{dtzeA} dt < oo} ./o /•CO C {z E H : / l{etzeAc} dt = oo}. ./o This and (5.3) yields P(Z e B, T = oo) = 0 and P(Z' G Bc, T' = oo) = 0. (5.4) Now, for 0 ^ t < oo, 8rlB={zeH: J l{93zeA}ds = oo}=B. Thus Be 1, which, together with (5.4), implies that P(Z G -,T = oo)|z and P(Z' G -,T' = oo)|j are mutually singular, that is, (5.2) holds. □ 5.4 A Cesaro Total Variation Limit Result The following theorem explains what the invariant cr-algebra has to do with Cesaro total variation convergence. Theorem 5.3. Let Z and Z' be one-sided discrete time or continuous time shift-measurable stochastic processes with a general state space (E,£) and path space {H,W). Then as t -^ oo, \\P(9utz e ■) - P(6utz' e -)|| -+ ||P(Z g -)li - p(z' e Olill. where U is uniform on [0,1] and independent of Z and Z'. Proof. Let T be as in Theorem 5.2 and send t ->• oo in the shift-coupling inequality (Theorem 3.1) to obtain (due to (5.2)) limsup \\P(9utz e ■) - P(eutz' e Oil ^ l|P(Z € Ok - P(Z' g -)lill- t—>co
Section 5. Shift-Coupling - Invariant a-Algebra - Equivalences 177 By Lemma 5.1 we have, for 0 ^ t < oo, ||P(Z g -)|i - P(Z' g oiill = \\v{Butz e -)|i - P(emz' e Okll, and since the right-hand side is ^ \\P{9UtZ G •) - V{6UtZ' G -)||, we nave ||P(Z G Oil - P(Z' e Olill < Uminf ||P(0t«Z G0~ P(0t/t^' £ Oil- t—»co These two inequalities yield the desired result. □ REMARK 5.1. By a similar argument we can obtain the inequality (5.1) directly: ||P(Z G Oil - P(Z' G Olill ^ \\^(0UtZ e 0 - ?(0utZ' G Oil ^ 2P(TV.R> Ut) ->■ 2P(T = oo), £-+oo, without the concept of coupling with respect to a cr-algebra. 5.5 Equivalences We can now tie together shift-coupling, Cesaro total variation convergence, and the invariant cr-algebra as follows. Theorem 5.4. Let Z and Z' be one-sided discrete-time or continuous-time shift-measurable stochastic processes with a general state space (E,£) and path space (H, %). Let U be uniform on [0,1] and independent of Z and Z'. The following statements are equivalent. (a) There exists a successful distributional shift-coupling of Z and Z'. (6) \\P{9mZ GO- P(9utZ' G Oil ->■ 0 as t -+ oo. (c) P(Ze-)\I = P(Z'e-)\I . Moreover, these statements are equivalent to the existence of a successful nondistributional shift-coupling if there exists a weak-sense-regular conditional distribution of Z given 9tZ for any random time T [this holds in discrete time when (E,£) is Polish and in continuous time when (E,£) is Polish and the paths are right-continuous]. PROOF. By the shift-coupling inequality, (a) implies (6); see (3.6). By Theorem 5.3, (6) implies (c). By Theorem 5.2, (c) implies (a). The final claim follows from Theorems 2.1 and 2.2. □ Corollary 5.1. Suppose Z and Z' are both stationary. Then Z and Z' have the same distribution if and only ifP(Z € -)j = P(Z' G Or-
178 Chapter 5. SHIFT-COUPLING Proof. IfP(Ze ■) = P(Z'G • ),then in particular P(ZG -)x = T{Z' G -)i, since 1 C U. Conversely, iiP(Ze-)i = P{Z'e -)i, then (6) in Theorem 5.4 holds, which together with the stationarity of Z and Z' yields ||P(z e •) - P(Z' e -)|| = \\P(emz e ■) - ?(emz' e oil -+ o as i ->• oo. Thus P(Z G 0 = p(^' 6 0 as desired. □ Corollary 5.2. Suppose for all A EH, P(eutz ei)4 P(z' eA), * ->■ oo. (5.5) Then dutZ 4Z' as ^ ->• oo. Proof. Due to Lemma 5.1, we have V{QUtZ £ A) = P(Z G A) for iel Thus, (5.5) implies that P(Z G Oil = p(z' ^ Oil- Thus (c) in Theorem 5.4 holds and the desired result follows from (6). □ Chapter 7 will mainly be devoted to extending the above shift-coupling theory to general random elements under a semigroup of transformations. We shall show there that the limit result (6) in Theorem 5.4 can be much generalized. We can, for instance, replace Ut by a random variable that is uniform on tB for any B G B[0, oo) with positive finite Lebesgue measure. This follows from a corresponding generalization of the shift-coupling inequality. 6 e-Coupling - Distributional e-Coupling In the next four sections we shall only be concerned with continuous time shift-measurable processes and treat an issue that does not arise in discrete time: what happens when the shift of a shift-coupling can be made arbitrarily small? These four sections mimic the pattern from the exact coupling and shift-coupling cases. This section introduces £-coupling and its distributional version. 6.1 £-Coupling — Definition Let Z and Z' be one-sided continuous-time shift-measurable stochastic processes with a general state space (E,£) and path space {H,%). For e > 0, an e-coupling of Z and Z' is a shift-coupling (Z,Z',T,T') such that \T-T'\^e on{T<oo}. (6.1) Certainly an exact coupling is an £-coupling with V — T, for each e > 0. We shall refer to a collection (Z^, Z'(e\ Te, T'e), e > 0, of £-couplings simply as epsilon-couplings. Call limsup£|0P(^ < oo) tne success probability. Say that the epsilon-couplings are successful if limsupP(T£ <oo) = 1. ei.0
Section 6. e-Coupling - Distributional e-Coupling 179 Remark 6.1. In Chapter 2 we proved BlackwelFs renewal theorem by ep- silon coupling random walks (the discrete-time processes formed by the renewal times) in space: the random walks got £-close in the state space (at different times). In this chapter we use the term epsilon-couplings for getting close in time, not in space. 6.2 Distributional £-Coupling - Definition The distributional version of £-coupling is not as obvious as that of exact coupling and shift-coupling. The definition (which is best motivated by Theorems 6.2 and 7.1 below) goes as follows. For e > 0, say that (Z, Z', T, T', R, R') is a distributional e-coupling of Z and Z' if (Z, Z', T, V) is a distributional shift-coupling of Z and Z', and R and R! are nonnegative random times such that {T < oo} = {R < oo} and {T' < oo} = {R' < oo}, {9TZ,T,R) = {9T'Z',R',T'), (6.2) \T-R\ <e on {T<oo} and \T' - R'\ < e on {T' < oo}. It can be helpful to think of R as a substitute for T" and of R' as a substitute for T (such R and R! have already appeared in Theorem 3.1 and its proof). Note that an £-coupling (Z,Z',T,T') is a distributional £-coupling in the sense that (Z,Z',T,T',T',T) is a distributional £-coupling. We shall use the word nondistributional to distinguish an £-coupling from a distributional e-coupling. Otherwise, we use the same terminology in both cases. 6.3 The Hats May Be Dropped in the Distributional Case If we have a distributional £-coupling of Z and Z', then we can take Z and Z' to be the original processes Z and Z'. Theorem 6.1. Let e > 0 and suppose (Z,Z',T,T',R,R') is a distributional e-coupling of Z and Z1. Then the underlying probability space can be extended to support random times T, T', R, and R! such that (Z,T,R) = (Z,f,R) and (Z',T',R') = (Z',f',R'). In particular, (eTZ,T,R) = (9T>Z,,R,,T'). (6.3) Proof. This follows from the transfer extension in Section 4.5 of Chapter 3. In order to obtain T and R take Yi := Z and (Y{, Y2') := (Z, (f, R)) and define (T',R') := Y2. Similarly, in order to obtain T' and R' take Yi := Z' and (Y{,YJ) := {Z1, (f',R')) and define (T',R') := Y2. □
180 Chapter 5. SHIFT-COUPLING This theorem motivates once more dropping the hats when discussing a (single) distributional £-coupling, when there is no danger of confusion. Since the transfer extension can be applied countably many times, we can drop the hats simultaneously when considering, for instance, distributional ^-couplings, 1 ^ k < oo. In fact, the hats can be dropped simultaneously even when we have an uncountable collection of £-couplings, e > 0: see the proof of Theorem 2.2 in Chapter 7. 6.4 Turning Distributional into Nondistributional In the standard settings a distributional £-coupling can always be turned into a nondistributional one. Theorem 6.2. Let e > 0 and let (Z,Z',f,f',R,R') be a distributional e- coupling of Z and Z'. Suppose there exists a weak-sense-regular conditional distribution of Z' given 9f,Z' [this holds when the state space is Polish and the paths are right-continuous}. Then the underlying probability space can be extended to support T, Z"', and T" such that {Z, T, T") = [Z, f, R) and (Z", T", T) = (Z',f, R') and (Z,Z",T,T") is a nondistributional e-coupling of Z and Z'. Proof. Let T and R be as in Theorem 6.1 and put T" := R. To obtain Z" use the transfer extension in Section 2.12 of Chapter 4 as follows. Take Yi := (8TZ,T,T") and (Y{,Y{) := ((6»t,Z', R',f'),KflZ') to obtain Y2 such that (6TZ,T,T",Y2) = (6f,Z',R',f',Kf,Z'). Define Z" by (eT"Z",KTnZ") := {eTZ,Y2) (thus 9TZ = 9T»Z"). Now (0T» Z", T, T", kT" Z") is a copy ot(0flZ',R',f', Kfl Z'), and it follows that (Z",T",T) is a copy of (Z',f',R') because (Z",T",T) is determined in the same measurable way by (6T»Z", T, T", kt»Z") as (Z\ f', R') is by (9t,Z',R',f',Kf,Z'). D 7 e-Coupling — Inequality and Asymptotics The last section was devoted to the definition of £-coupling and its distributional version. We shall now go on to the limit implications. This section differs from Section 3 (and from Section 5 in Chapter 4) in two ways. Firstly, we will not be able to establish any rate results. Secondly, we shall consider two types of convergence in addition to the one (smooth total variation convergence) that turns out (Section 9) to be appropriately linked to epsilon-couplings.
Section 7. e-Coupling - Inequality and Asymptotics 181 7.1 £-Coupling Inequality Rather than shifting to a point picked at random in [0,t] as in the shift- coupling case, the appropriate thing to do here is to shift to a t as in the exact coupling case and then blur t slightly (we can also think of the time origin of the processes as blurred slightly). Theorem 7.1. Let Z and Z' be one-sided continuous-time shift-measurable stochastic processes with a general state space (E,£) and path space (H,'H). Let e > 0 and suppose (Z,Z',T,T',R, R') is an e-coupling of Z and Z' (distributional or not). Then, for all h > 0 and t £ [0, oo), \\P(8t+UhZ G •) - ?{9t+UhZ' e -)|| £-COUPLING ^ 2P(T > t) + 2-, INEQUALITY h where U is uniform on [0,1] and independent of Z and Z1. Proof. Let U be uniformly distributed on [0,1] and independent of Z, Z', and (Z, Z', T, T', R, R'). Clearly ((T - R)+ + Uh) mod h [the remainder when (T — R)+ + Uh is divided by h] is uniform on [0, h] and independent of Z. Therefore 9t<T-R)++Uh) mod h.Z is a copy of OuhZ. Similarly, 8((T>-R')++Uh) mod hZ' is a copy of 8UhZ'. Thus (QtO((T-R) + + Uh) mod hZ, 9t8((T' ~R') + + Uh) mod h.Z ) is a coupling of 6t+UhZ and 6t+UhZ'■ Since 9t8((T-R)++uh) mod h.Z = Qt+(T-R)++uhZ on {Uh ^ h - \T — R\] and (due to (T - R)+ = T - T A R) 9t+(T-R) + + UhZ = Ot-TAR+UhQrZ On {T^t}, we have 8t8({T-R)Jr + Uh) mod h^ = ^t-TAR+UhOrZ on C:= {T^«,t//i^/i-|r-.R|}. Similarly, 9t8((T'-R') + + Uh) mod /i-^ = 8t-T'AR' + Uh^T'Z' on c" = {i?'^,t//i^/i-|fl'-r'|}.
182 Chapter 5. SHIFT-COUPLING These two identities together with (9TZ, T, R, U) = (Ot>Z', R', V, U) imply P(8t8((T-R) + + Uh) mod h% S •, C) = P(9t9((T'-R') + + Uh) mod h% G">C)- Thus C and C" are distributional coupling events for the coupling at (7.1), and the distributional coupling event inequality (Theorem 4.3 in Chapter 4) yields \\P(8t+Uhz e ■) - P(9t+Uhz' e -)IK 2P(CC). (7.2) Observing that P(CC) ^P(T>t) + P(Uh>h-\T-R\) (7.3) and P{Uh >h-\T-R\)^ P(Uh > h - e) = | completes the proof. □ Remark 7.1. When we have a nondistributional £-coupling (Z,Z',T,T') then R = T" and .R' = T, and in the proof C = C, and C is a nondistributional coupling event of the coupling at (7.1). Reformulation. The left-hand side (l.h.s.) of the e-coupling inequality can be rewritten in the following smooth form: 1 rt+h -. rt+h - P(esze-)ds-- P(esz'e-)ds Thus, in the same way as the coupling time and shift-coupling inequalities- are basic for plain and Cesaro total variation asymptotics, respectively, the £-coupling inequality is basic for smooth total variation asymptotics. 7.2 Finite T£'s — Smooth Total Variation Convergence Let (Z(s\,Z'(s\Te,T£,Re,R'e) be £-couplings. The £-coupling inequality yields, for h > 0, limsup \\P(9t+UhZ e ■) - P(9t+uhZ' G -)ll < 2P(^ = oo) + 2£-. t->oo n Thus if there exist successful epsilon-couplings (distributional or not), then taking lim inf as e goes to 0 yields V/i>0: \\P(9t+UhZ e-)-P(9t+uhZ'e-)\\-+0, t->oo. (7.4) If Z' is stationary, then 9t+uhZ' has the same distribution as Z', and (7.4) can be rewritten as V/i > 0 : 9t+uhZ %Z\ t-+ oo. (7.5)
Section 7. e-Coupling - Inequality and Asymptotics 183 7.3 Stochastically Dominated T£'s - Uniform Convergence In order to obtain rate results a more sophisticated £-coupling inequality is needed, but uniform convergence follows easily. Let 2bea class of continuous-time shift-measurable stochastic processes on a general state space. Suppose there exists, for each £ > 0, a finite random variable Te such that for each pair of processes Z, Z' 6 Z there is an £-coupling (distributional or not) of Z and Z' with time Te such that D _ Te < TV Then the £-coupling inequality yields that for all h > 0 and t 6 [0, oo), sup \\P(6t+Uhz e •) - PWt+uhZ' e OIK 2P(re > t) + ie-. Send first t —> oo and then £ 4- 0 to obtain the following result on uniform convergence over the class Z: sup \\p(Ot+uhZe-)-P(Ot+Uhz'e-)\\->o, t^oo. Z,Z'6Z 7.4 Total Variation Convergence in the State Space In Section 7.2 we showed that successful epsilon couplings of Z and Z', where Z' is stationary, imply smooth total variation convergence to station- arity in the path space, namely (7.5). This certainly implies the following weaker result on smooth total variation convergence to stationarity in the state space V/i>0: Zt+uh'AZ'a, t -> oo. This latter result can be sharpened to total variation convergence if we put a severe condition on the paths (while in the path space we will still have only smooth total variation convergence; see, however, Section 7.2 in the next chapter). Theorem 7.2. Let Z and Z' be one-sided continuous-time stochastic processes with a general state space (E, £) and path space (H, %) where H consists of paths that are piecewise constant and right-continuous (in the discrete topology) with finitely many jumps in finite time intervals. Suppose further that Z' is stationary and that there are successful epsilon couplings (Z^\Z'^,Te,T^,Re,R'e), £ > 0, ofZ andZ' (distributional or not). Then t -> Zq, t -> oo. Proof. In the nondistributional case this is Theorem 10-1 in Chapter 2. The following modification of the proof is needed to cover the distributional
184 Chapter 5. SHIFT-COUPLING case. Note firstly that Zt = Z(+(T_fi) on the event C = {TvR<:t-E1 no Z jump in [(T- R) + t - e,(T- R)+ t + e]}, secondly that Zt+(T-R) is determined in the same measurable way by (9TZ, T, R) on C as Z[ is by (6T,Z', R', V) on the event C = {T V R' ^ t - e, no Z' jump in[t-e,t + e]}, and thirdly that C is determined in the same measurable way by (OrZ, T, R) as C" is by (9t'Z',R',X"). These three observations yield P(zte-,C) = P(zi'e-,C"), that is, C and C are distributional coupling events of the coupling (Zt, Z't) of Zt and Zq. The rest of the proof is now the same as in the nondistribu- tional case (using 2P(C"C) rather than 2P(CC) to bound the total variation distance). □ 7.5 Distributional Convergence in the State Space We shall now show that successful epsilon-couplings yield convergence in distribution (see Section 10 in Chapter 3) in the state space, provided that the state space is metric and the paths are right-continuous. Theorem 7.3. Let Z and Z' be one-sided continuous-time stochastic processes with state space (E,£) and path space (H,!!), where E is met- ■ ric, £ its Borel subsets, and the paths are right-continuous {that is, H = Re{[0, oo))). Let Z' be stationary and suppose there are successful epsilon- couplings of Z and Z' (distributional or not). Then Zt —> Zq, t —> oo. Proof. We must prove that lim|E[/(Zt)]-E[/(Z£)]|=0 (7.6) t—>oo for an arbitrary real-valued bounded continuous function / defined on E. It is no restriction to take |/| ^ 1. For notational convenience fix an £ > 0 and let (Z,Z',T,T',R, R') be the unhatted copy of the e-coupling (ZlE\Z'W,Te,Tl,Re,R'E); see Theorem 6.1. Then /(Zt) - f(Z't) = f(Zt)l{T>t} - f(Z't)l{R,>t] + f(Zt)^{T^t] ~ f{Z't+T'-R')l{R'^t} + f(Zt+T'-R')^{R'^t} ~ f{Z't)^{R'^t}-
Section 7. e-Coupling - Inequality and Asymptotics 185 Now, f(Zt)l{T^t] is determined in the same measurable way by (6tZ, T, R) as f(Z't+T,_R,)l{Rl^t} is by (0T,Z',R',T'). Thus, by the definition of distributional £-coupling, f(Zt)l{T^t] and f{Z't+T,_Ri)l{R'^t} have the same distribution and thus the same expectation. Hence the mid-part on the right-hand side cancels when we take expectations, and we obtain (after taking absolute value) |E[/(Zt)] - E[/(Zt')]| ^\nf(Zt)l{T>t}}\ + |E[/(Zt')l{K'>t}]| + \E[f(Zl+T,_R,)l{R,m - f(Z't)l{R%i]]\. Use |/| ^ 1 and \R' — T'\ ^ e, respectively, to dominate the terms on the right and obtain (recalling that both T and R' are copies of Te) |E[/(Zt)]-E[/(Zt')]|<2P(rE>t) + E[ sup |/(Z('+J-/(Z;)|]. —e^u^e Applying the stationarity of Z' on both sides yields |E[/(Zt)]-E[/(Zi)]|^2P(rE>t)+E[ sup \}{Z'u)-f{Z'e)\}. Send t to infinity to obtain limsup|E[/(Zt)]-E[/(Z£)]| t—»00 ^2P(T, = cx))+E[ sup \f(Z'u)-f(Z'e)\]. Now, sup |/(z:)-/(z;)K|/(z^)-/(z;)|+ sup |/(z;)-/(z^)|, 0^u^.2e 0^u^2e and thus, by the continuity of / and the right continuity of the paths, suPo<u<2e \f(Z'u) — f(Z'£)\ —> 0 pointwise as e decreases to zero, and bounded convergence yields limsup|E[/(Zi)]-E[/(Z^)]|^21immfP(re = oo). Since the epsilon-couplings are successful, this yields (7.6). □ 7.6 Distributional Convergence in the Path Space Under the conditions of Theorem 7.3 we have in fact finite-dimensional distributional convergence with respect to the product metric: for n ^ 1 and *i,... ,tn ^ 0, (Zt+il,..., Zt+tn) 4 (Z'u,..., Z'tJ, t -> oo.
186 Chapter 5. SHIFT-COUPLING This follows by applying Theorem 7.3 to the processes (Zt+ti. • • • > Zt+tn )t£[o,oo) and (Z't+t! i • • • > Z't+tn )<e[o,oo) > since they are right-continuous in the product metric on En and since an £-coupling of Z and Z' yields an £-coupling of these vector processes. Now instead of the finite-dimensional vector (Zt+tl,..., Zt+tn) consider the whole process beyond t, namely 6tZ. Then distributional convergence still holds, provided that we impose more conditions: let (E,£) be Polish and let the paths not only be right-continuous but also have left-hand limits, that is, take % = VE([0, oo)). Then [see Ethier and Kurtz (1986)] for each path z in De([0, oo)) the mapping from [0, oo) to De([0, oo)) taking t to 6tz is right-continuous in the so-called Skorohod topology. Moreover, this topology can be metrized in such a way that % — T>e([0, oo)) is generated by the open subsets of De([0, oo)). (This metric makes (De([0,co)),T>e([0,oo))) Polish, but we shall not need this fact here). By convergence in distribution in the path space we mean distributional convergence with respect to this metric. Theorem 7.4. Let Z, Z1 be one-sided continuous-time stochastic processes with a Polish state space (E,£) and path space (De([0, oo)),2?e([0, oo))). Let Z' be stationary and suppose there are successful epsilon-couplings of Z and Z'. Then 9tZ S> Z', t -> oo. Proof. This follows by applying Theorem 7.3 to the processes (0tZ)te[o,oo) and (8tZ')t<=[o,oo)- These processes have the metric state space (De([0, oo)),T>e([0, oo)) and have right-continuous paths. Moreover, an £-coupling of Z and Z' yields an £-coupling of (04Z)t6[O,oo) and (6>(Z')<e[o,oo)- □ Remark 7.2. Since successful epsilon-couplings imply convergence in the path space in both distribution and smooth total variation, the question arises of what the relation is between these two modes of convergence. Due to Theorem 9.4 below, smooth total variation convergence is equivalent to the existence of successful epsilon-couplings. The same is not true of convergence in distribution, as the following simple counterexample shows: Take Zt = 1/t and Z't=0, 0 ^ t < oo, to obtain 9tZ -> Z' as t -> oo, while obviously there are no successful epsilon-couplings. Thus convergence in distribution is in general strictly weaker than smooth total variation convergence.
Section 8. e-Coupling - Maximality 187 8 e-Coupling - Maximality In this section we only establish the following straightforward consequence of the continuous-time maximality result for exact coupling (Theorem 6.1 in Chapter 4). As in the shift-coupling case this result enables us to show in the next section that there are epsilon-couplings that are both maximally successful and also successful when the smooth total variation convergence (7.4) holds. But the question of how to define an appropriate analogue of maximal exact coupling is otherwise left open. Theorem 8.1. Let Z and Z' be one-sided continuous-time shift-measurable stochastic processes with a general state space (E, £) and path space (H, %). For eache > 0 there is a distributionale-coupling {Z^e\ Z'(e>, Te,T'e, R£,R'e) of Z and Z' such that \\P{6ne+U£Z e ■) ~ P(0ns+ueZ' € -)ll = 2P(Te > ne + e), (8.1) for 0 ^ n < oo, and \\p(6Uez e Olr - P(Ousz' e OH! = 2P(T£ = oo), (8.2) where U is uniform on [0,1] and independent of Z and Z'. Moreover, there exist nondistributional e-couplings with this property if there exist weak-sense-regular conditional distributions of Z^e> given 6tc Z^ [this holds when (E,£) is Polish and the paths are right-continuous]. Proof. Fix an e > 0. According to Theorem 6.1 in Chapter 4 there is a distributional exact coupling of 6usZ and 6usZ' maximal at the times ne, n ^ 0, and having {ne : 0 ^ n ^ oo} valued coupling times. According to Theorem 3.1 in Chapter 4 this distributional exact coupling can be unhatted, that is, the underlying probability space can be extended to support {ne : 0 ^ n ^ oo} valued random times Le and L'e such that (6Lt6UsZ,Ls) = (6L,c6UsZ',L's) and, for all integers n ^ 0, \\p{9n£eUez e o - v{0nS6Uez' e Oil = 2P(Le > ne). (8.3) Due to Theorem 9.2 in Chapter 4 it holds that \\P(0ueZ e Olr - PtfueZ' e Olrll - 2P(LS = oo). (8.4) Apply the conditioning extension in Section 4.5 of Chapter 3 twice to obtain first a random variable V [take Y\ := {0Lc9u£Z,Le) and (l7/,!^) : = {{9L.e0UeZ',L'e),U) and put V := Y2] such that (8Lc8UeZ,L£,V) £ (0K0UeZ',L't,U)
188 Chapter 5. SHIFT-COUPLING and then a random variable V [take Yx := {QL,QUeZ', L'e,U) and (Y{, K,') : = ((9LJUeZ,Le,V),U) and put V := Y2\ such'that (6Lc6UeZ,Le,V,U)^(6L,6UeZ',L'e,U,V'), which implies (6Lc+UeZ, Le + Ue, Le + Ve) = (6L,+U£Z', L'e + Ve, L'e + Ue). Since \{Le + Ue)- (Le + Ve)\ = \U-V\e^e on {Le < oo}, \(L'e + Ue)-(L'e + V'e)\ = \U-V,\e^e on {L'e < oo}, it follows that (Z,Z',Le + Ue,L'£ + Ue,Le + Ve,L'e + V'e) is a distributional £-coupling. Since Le is {ne : 0 ^ n ^ oo} valued and C/ is uniform on [0,1], it holds that P(Le > ne) = P(Le ^ ne + e) = P(Le + Ue > ne + e), and thus (8.3) yields (8.1). Finally, P(L6 = oo) = P{L6 + Ue = oo), and thus (8.4) yields (8.2). □ 9 e-Coupling — Smooth Tail cr-algebra - Equivalences It turns out that there is a cr-algebra playing the same role for epsilon- couplings as the tail cr-algebra for exact coupling and the invariant a-. algebra for shift-coupling. We shall call it the smooth tail cr-algebra. In this section we introduce it, link it to epsilon-couplings, and establish an analogous set of equivalences as in the exact coupling and shift-coupling cases. 9.1 The Smooth Tail cr-Algebra It seems there is no direct way of defining the smooth tail cr-algebra by explicitly specifying the sets it contains. We shall define it by specifying a generating class of functions. Let iS° be the class of tail functions that are right-continuous in time, that is, S° = {fET: f{0tz) -> f{z) asijO.ze H}. Define the smooth tail a-algebra by <S = cr{<S°}.
Section 9. £-Coupling - Smooth Tail cr-algebra - Equivalences 189 We first note that 1CSCT. Here <S C T is obvious, while 1 C S follows by observing that if A 6 2, then 1^(^(2;) = 1a{z) and thus trivially 1^(^(2;) —> 1a(z) as £ 4- 0, that is, I4 6 <S° and consequently A 6 <S. Secondly, we note that <S° (and thus S) contains smoothed tail functions, that is, functions f(h\ h > 0, defined by fh f{h)(z) = /T1 / f(0sz)ds, zE H, f £ T and bounded. (9.1) Jo That fW e 5° follows from |/W(0(2)-/«(2)|=ri f(6sz)ds- f(6sz)ds\ Jt Jo rt+h ft = /i"1| / }{6sz)ds- / /(e.z)ds| Jh JO ^2isup|/|//i-^0, U0. This shows in particular that the inclusion X C S is in general strict. Finally, in order to see that the inclusion S C T is in general strict, consider the following example. EXAMPLE 9.1. Consider real-valued nonnegative processes with path space H consisting of right-continuous piecewise linear paths having slope —1 and finitely many jumps in finite intervals and having left-hand limit 0 at the jumps and rational lengths between jumps (that is, the jump sizes are rational). Then the set A5 = {z 6 H : z0 is rational} equals the set {z 6 H : zt is rational} when t is rational and equals its complement when t is not rational. Thus A5 6 % for arbitrarily large t. Thus A5 6 T. But the indicator of A5 is not in <S°, since \j\5{9tz) = 1 — 1,4,5(2) when t is not rational and thus lA5(6tz) cannot go to 1a5{z) as t 4- 0. This suggests that A5 is not in <S. We shall show indirectly that A5 is not in S. The set H is the path set of the remaining life process (see Section 9.1 in Chapter 2) of a nonlattice renewal process with rational recurrence times. In Chapter 2 (Theorem 7.1) we showed that there are successful epsilon-couplings of two differently started versions of such a process. According to Theorem 9.4 below this implies that the distributions of the two processes agree on S. But if we let one of the processes have a rational delay and the other have an irrational delay, then the probabilities of the processes being in A5 are one and zero, respectively. Thus A5 cannot be in S.
190 Chapter 5. SHIFT-COUPLING 9.2 The Inequality The following theorem explains what the smooth tail cr-algebra has to do with epsilon-couplings. Theorem 9.1. Let Z and Z' be one-sided continuous-time shift-measurable stochastic processes with a general state space (E, £) and path space (H, 7i). For each s > 0 let {Z^e\Z'^,Te,T'e,Re,R'e) be an e-coupling of Z and Z' {distributional or not). Then ||P(Z G .)\s - P(Z' G .)lsll ^ 21iminf P(Te = oo). eJ.0 Proof. Apply Theorem 9.3 in Chapter 4 to the processes OuhZ and OyhZ' to obtain \\p{6teuhz e ■) -T>{6teUhz' e -)\\ -> \\P(6UhZ g -)\T - P(0UhZ' g -)lrll, t -> oo. Thus sending t —> oo in the £-coupling inequality (Theorem 7.1) yields \\P(0uhZ G -)\T - P(6uhZ' G -)\r\\ ^ 2P(T£ = oo) + 2e/h. Take lim inf as e \. 0 to obtain \\P(6UhZ G -)lr " PtfuhZ' G Olrll < 2liminf P(Te = oo), h> 0. eJ.0 A reference to the following lemma completes the proof. □ Lemma 9.1. It holds that \\P(Ze-)\s~P(Z'e-)\s\\ = sup \\P(6UhZ e -)lr - P(^fc^' e Olrll- /i>0 Proof. Put ./ = P(Ze-)lr-P(^e-)lr, i/<fc> = p(0™z g -)\T - P(oUhz' g -)lr, fc > o. For bounded / G T let /CO be defined by (9.1) and recall that /CO G 5°. Note also that if / G <S°, then /CO -> / pointwise as /i 4- 0. We must prove |H5||=sup||I/W||. h>0 For that purpose take an A G S such that |Ms|| = 2v(A) [see Theorem 8.2 in Chapter 3, the first equality in (8.11)] and fix an e > 0. It is a basic
Section 9. e-Coupling - Smooth Tail cr-algebra - Equivalences 191 fact of bounded measures [see Ash (1972), Theorem 1.3.11] that if a cr- algebra is generated by an algebra, then each set in the cr-algebra can be approximated in measure by a set in the algebra (the measure of the symmetric difference of the sets can be made arbitrarily small). Now, <S is generated by the algebra {{(/i, ...,/„) £ B} : A,..., /„ G S°, B e B(Rn), 1 ^ n < oo}, and thus there is an n ^ 1 and a B 6 B(Rn) and functions fi,.-.,fn in <S° such that / lA-lB(fi,...,fn)\d\v\^e. (9.2) Moreover, it is a basic fact of bounded measures on (En,Z?(En)) [see Ash (1972), Theorem 2.4.14] that the indicator of any set in B(Rn) can be approximated by a [0,1] valued continuous function in such a way that the integral of the absolute value of their difference can be made arbitrarily small (approximation in L\). Apply this to the measure |^|(/i, • • •, /n)_1 to find a continuous function a from En to [0,1] such that / lfl(/i,...,/„)-/|dH^e, where / = a(/,,..., /„). (9.3) Since /i,..., /n are in 5° and a is continuous, it follows that / is in 5°, which implies /W —> / pointwise as h \. 0. Hence, by bounded convergence, there is an h > 0 such that J \f — f^\ d\v\ ^ £, which together with (9.2) and (9.3) yields J\lA-fw\d\v\^3e. Combine this, u(A) = f /(fc) dv + f(lA - f{h)) dv, and / /(h) dv = J f dv^ to obtain v(A) ^ f fdvW +3e. Since \\v\s\\ = 2v(A) and ||i/(h)|| =2 sup gdv(h) [see Theorem 8.2 in Chapter 3], geT J this yields \\v\s\\ ^ suph>0 \\v^\\ + 3e. Since e > 0 is arbitrary, we obtain \v\s\\ ^ sup||i/ h>0 CO I
192 Chapter 5. SHIFT-COUPLING The converse ||i>|s|| ^ suph>0 \\v^\\ holds, since g 6 T implies g^ 6 <S and thus Hs||£2 sup /Vfc>di/=|l"(fc)ll> 0<C9<:i and the lemma is established. □ 9.3 Maximally Successful Epsilon-Couplings Once more equality can be obtained in the inequality. Thus there exist maximally successful epsilon-couplings (attaining the supremum of the success probabilities over all collections of epsilon-couplings) and maximal success probability = ||P(Z 6 -)|s A P(Z' 6 -)lsll- Theorem 9.2. Let Z and Z' be one-sided continuous-time shift-measurable stochastic processes with a general state space (E, £) and path space (H, Ti). The distributional epsilon-couplings {Z^\Z'^\Te,T'e,Re,R'e), e > 0, of Z and Z' in Theorem 8.1 are such that ||P(Z e -)\s - P(Z' e -)|sll = 2supP(T, = oo) = 2limP(T£. = oo). PROOF. The first equality follows from Theorem 8.1 and Lemma 9.1. Certainly, limsupP(Te = oo) ^ supP(Te = oo) ej.0 £>0 and thus the second equality follows from the first and Theorem 9.1. □ Remark 9.1. There are distributional epsilon-couplings such that P|{T,<oo} and p|{T;<oo} e>0 e>0 are maximal distributional <S-coupling events of Z and Z'. In fact, we may take the epsilon-couplings such that the events C — {Te < oo} and C = {T'e < oo} do not depend on e; on Cc the processes Z^6\e > 0, are identical; on C'c the processes Z'(6\e > 0, are identical.
Section 9. e-Coupling - Smooth Tail cr-algebra - Equivalences 193 In order to establish this let (Z, Z') be a coupling of Z and Z' with maximal distributional <S-coupling events C and C. Then ||P(Ze-|C)|s-P(Ze-|C")ls|l=o, and thus according to Theorem 9.2, there is for each e > 0 a successful distributional £-coupling of the processes with distributions P(Z 6 -\C) and P(Z' 6 -|C"). We may let this £-coupling be independent of C and C. On C take (Z^\Te,Re) from this e-coupling and on Cc let {Z^e\Te,Re) be (Z,oo,oo). Similarly, on C" take {Z'(e\T'e,R'e) from this £-coupling and on C"c let {Z'(e\T'e,R'e) be (Z',oo,oo). 9.4 A Smooth Total Variation Limit Result The following limit result is quite different from Theorem 9.3 in Chapter 4 and Theorem 5.3 in this chapter. Theorem 9.3. Let Z and Z' be one-sided continuous-time shift-measurable stochastic processes with general state space (E,£) and path space (H,T-L). Then as h 10, \\P(6Uhz e -)\T - P(eUhz' e -)lrll -> ||P(z e -)\s - P(Z' e -)ls||, where U is uniform on [0,1] and independent of Z and Z'. PROOF. The distributional epsilon-couplings in Theorem 8.1 satisfy \\P{0UeZ € -)|r - P(0ueZ' € -)|rll = 2P(Te = oo), e > 0, and a reference to the limit result in Theorem 9.2 completes the proof. □ 9.5 Equivalences We can now tie together epsilon-couplings, smooth total variation convergence, and the smooth tail cr-algebra as follows. Theorem 9.4. Let Z and Z' be one-sided continuous-time shift-measurable stochastic processes with general state space (E,£) and path space (H,H). Let U be uniform on [0,1] and independent of Z and Z'. The following statements are equivalent. (a) There exist successful distributional epsilon-couplings of Z and Z'. (b) For each h > 0, \\P(6t+UhZ e •) - P{6t+UhZ' e -)ll -> 0 as t -> oo. (c) P(Ze-)\s = P(Z'e-)\s. These statements are also equivalent to each of the following claims, (a!) For eache>0, there is a successful distributional e-coupling of Z, Z'.
194 Chapter 5. SHIFT-COUPLING (c') For each h>0, P(8UhZ e -)lr = P(0uhZ' £ -)lr • Finally, these statements are equivalent to the existence of a successful nondistributional e-coupling of Z and Z1 for each £ > 0, if there exists a weak-sense-regular conditional distribution of Z given &tZ for each random time T [this holds when (E, £) is Polish and the paths are right-continuous}. PROOF. By the £-coupling inequality, (o) implies (6); see (7.4). By Theorem 9.4 in Chapter 4, (6) implies (c'). By Theorem 9.3, (c') implies (c). By Theorem 9.2, (c) implies (a'). Certainly (o') implies (o). The final claim of the theorem follows from Theorems 6.1 and 6.2. □ Corollary 9.1. Suppose the equivalent statements in Theorem 9-4 hold. Then, for each h > 0, the underlying probability space can be extended to support finite times Th and T'h such that (6uhZ,6uhZ'jT^jT^) is a successful distributional exact coupling of OuhZ and OuhZ'■ Moreover, if there exists a weak-sense-regular version of the conditional distribution of Z given 6tZ for each random time T, then for each h > 0, the underlying probability space can be further extended to support a copy (Z^h\Uh) of (Z',U) such that (OuhZjOuhhZ^jTh) is a nondistributional exact coupling ofOuhZ and OuhZ' . PROOF. According to Theorem 9.4 in Chapter 4, the statement (c') implies the existence of a successful exact coupling of OuhZ and OuhZ'. This together with Theorem 3.1 in Chapter 4 yields the distributional claim. Establish the nondistributional claim by applying the transfer extension of Section 2.12 in Chapter 4 as follows. Take Y\ := (9Th&uhZ,Th) and (Yi',y2') := ((OrxOuhZ'^), {KTi'+Uh)Z',U)) to obtain a Y2 such that {{0TjuhZ,Th),Y2) = {{eT,heUhZ',Th),{KT,h+UhZ',U)). (9.4) Define (ZW,Uh) by (KTh+uhhZlh\Uh) := Y2 and 6Th6UhhZ^ := 6Th6UhZ. Then, due to (9.4), (Z^,Th,Uh) is a copy of (Z',T'h,U). □
Chapter 6 MARKOV PROCESSES 1 Introduction In this chapter we apply the three sets of coupling equivalences established in the previous two chapters (Theorem 9.4 in Chapter 4 and Theorems 5.4 and 9.4 in Chapter 5) to Markov processes. To each set of equivalences we add four more equivalent statements: on triviality, on mixing, on convergence in the state space, and on the constancy of harmonic functions. In Section 2 we start by applying the equivalences to a single process (not necessarily Markovian) adding the triviality and mixing aspects. Markov processes enter in Section 3, which contains preliminaries. Then each set of equivalences gets one section (Sections 4, 5, and 6) adding the two remaining aspects, on convergence in the state space and on harmonic functions. Section 7 concludes the chapter by considering the implication of these results in the case when there exists a stationary measure for the Markov process. As in the previous two chapters we denote the time parameter by s and t in accordance with continuous time, but all we need to switch to discrete time is to restrict s and t to be integer and replace integration (over time) by summation. 2 Mixing and Triviality of a Stochastic Process In this non-Markovian section we consider a single one-sided discrete- or continuous-time stochastic process Z with a general state space (E, £) and 195
196 Chapter 6. MARKOV PROCESSES some path space (H,H). We shall apply the three sets of coupling equivalences to the two processes obtained by conditioning Z on being in two arbitrary sets of paths A and B £ %. To each set of equivalences we add a triviality aspect and a mixing aspect: a sub-cr-algebra A of 7i is trivial with respect to Z, and Z is A-trivial, if P(Zei) = 0orl, AeA, while mixing properties have to do with asymptotic independence of events happening early and events happening late in the process. 2.1 Exact coupling: T-Triviality <£> Mixing •«•••• The word 'mixing' is used to indicate some sort of independence between what happens in a process early on and in the far future. We shall use the following definition. The process Z is mixing if as t —> oo, sup \P(6tz eA,zeB)- p(6tz e A)P(z e B)\ -» o, (2.1) Aen for each B £ H. Equivalently, Z is mixing if and only if (2.1) holds for all B of the finite-dimensional form B = {zeH:ztleAi,...,ztneAn}, (2.2) where n > 1 and 0 ^ ti < ■ ■ ■ < tn and Ai,...,An £ £. This equivalence follows from Lemma 2.3(6) below [take Yt = 6tZ\. Theorem 2.1. Let Z be a one-sided discrete- or continuous-time stochas7 tic process with a general state space (E, £) and a general path space (H, H). Then the following statements are equivalent. (a) For each B £ H such that P(Z 6 B) > 0, there exists a successful distributional exact coupling of Z and the process with distribution P{Ze-\Z eB). (b) For each B £ U such that P(Z £ B) > 0, \\P{6tz e ■) - p(etz £-\ze B)\\ -> o, t -> oo. (c) For each B £ U such that P(Z £ B) > 0, p(Ze-)\T = P(Ze-\zeB)\T. (d) The process Z is T-trivial. (e) The process Z is mixing.
Section 2. Mixing and Triviality of a Stochastic Process 197 Moreover, in (a) we may replace distributional by nondistributional if there exists a weak-sense-regular conditional distribution of Z given (5tZ for any random time T [this holds in discrete time when (E, £) is Polish and in continuous time when (E,£) is Polish and the paths are right-continuous]. Finally, in each of (a), (b), and (c) we may restrict B £ % to be finite- dimensional as at (2.2). PROOF. The equivalence of (a), (6), and (c) follows from Theorem 9.4 in Chapter 4 [take Z' with distribution P(Z £ -\Z £ B)], and so does the nondistributional claim. The equivalence of (d) and (c) follows from Lemma 2.1 below [take A = 7"]. The equivalence of (e) and (b) follows from Lemma 2.2 below [take Yt = 6tZ]. The equivalence of (a), (6), and (c) with B restricted to be finite-dimensional follows from Theorem 9.4 in Chapter 4 [take Z' with distribution P(Z £ -\Z £ B)]. The equivalence of (c) and (c) with B restricted to be finite-dimensional follows from Lemma 2.3(a) below [take A = T\. □ Lemma 2.1. Let A be a sub-a-algebra ofH. The process Z is A-trivial if and only if p(Ze-|ZeB)U = p(Ze-)U (2-3) for all B EH such that P(Z £ B) > 0. Proof. To establish the 'only-if part take A £ A and B £ % such that P(Zefl)>0 and note that .4-triviality implies that P(Z £ A\Z e B) = 0 or 1 according as P(Z 6 A) = 0 or 1. To establish the 'if part take B e A such that P(Z 6 B) > 0 and note that the distributional identity on A implies that P(Z e B\Z £ B) = P(Z £ B), that is, P(Z £ B) = 1. □ Lemma 2.2. Let Yt, 0 ^ t < oo, 6e random elements in some measurable space (G,Q). Then, for all B £ %, sup|P(rt £ A,Z eB)-P(Yt eA)P(ZeB)\ -»0, t->oo, (2.4) AGS if and only if for all B £ % such that P(Z £ B) > 0, \\P(Yt £ •) - P{Yt e-\Ze B)\\ -> 0, . t -> oo. PROOF. The first limit claim trivially holds if P(Z £ B) = 0, while if P(Z eB) > 0, then sup |P(rt £ A, Z £ B) - P(rt £ A)P(Z £ B)\ AeG = 2-xP(Z £ B)||P(yi £ •) - P(n £ -|Z £ B)||, and thus the two limit claims are equivalent. □
198 Chapter 6. MARKOV PROCESSES Lemma 2.3. (a) Let A be a sub-a-algebra ofH. If (2.3) holds for all B of the finite-dimensional form (2.2) and such that P(Z 6 B) > 0, then (2.3) holds for all B Eli such that P(Z e B) > 0. (b) Let Yt, 0 ^ t < oo, be random elements in some measurable space (G,Q). If (2.4) holds for all B of the finite-dimensional form (2.2), then (2.4) holds for all B en. Proof. It is a basic fact of bounded measures [see Ash (1972), Theorem 1.3.11] that if a cr-algebra is generated by an algebra, then each set in the cr-algebra can be approximated in measure by a set in the algebra. Thus for each Ben and e > 0 there is an n ^ 1 and 0 ^ ii < • • • < tn and A\,...,An e £ such that with B£ = {zeH:ztl eAu...,ztn e An} we have P(Z eB,Z £BS) + P(Z eBs,Z$B) <e. (2.5) In order to establish (a), suppose (2.3) holds for finite-dimensional sets. Fix B e n such that P(ZeB)>0 and take A e A. Then |p(z e A, z e B) - P(z e A)P(z e B)\ ^ \p{z eA,zeB)-p{z eA,z e Be)\ + \P(z eA,ZeBe)- p(z e A)p(z e Be)\ + \P(z e A)P(z e Be) - p(z e A)P(z e B)\. Since (2.3) holds for sets like Be, the middle term on the right is zero, and due to (2.5), the first and last are less than e. Since e > 0 is arbitrary, this means that |P(Z eA,ZeB)- P(Z e A)P(Z e B)\ = 0, as desired. In order to establish (6), suppose (2.4) holds for finite-dimensional sets. Fix BG^andAeg. Then \P{Yt eA,ZeB)- P{Yt e A)P(z e B)\ ^ \P{Yt eA,zeB)-p(Yt eA,ZeBs)\ + \P(Yt eA,ZeBe)- p(Yt e A)P(Z e Be)\ + \P(Yt e A)P(Z e Be) - p{Yt e A)P(z e B)\, which together with (2.5) yields sup |p(rt e A, z e B) - P{Yt e A)p{z e B)\ A€G ^ e + sup |p(rt e A, z e Be) - p(Yt e A)P{Z e Be)\ + e. Aeg
Section 2. Mixing and Triviality of a Stochastic Process 199 Since (2.4) holds for sets like BE, the middle term on the right tends to zero as t —> oo. Since e > 0 is arbitrary, this yields (2.4). □ 2.2 Shift-Coupling:' X-Triviality O- Cesaro Mixing O • • • Let U be uniform on [0,1] and independent of Z. In the continuous-time case assume now that Z is shift-measurable. Recall that in the discrete- time case we extend the definition of the shift-maps to noninteger times by etz = e[t]z,te[o,oo). The process Z is Cesaro mixing if as t —> oo, sup \P(0utz eA,z e B) - P(8utz e A)p(z e B)\ -> o (2.6) for each B EH. Equivalently, Z is Cesaro mixing if and only if (2.6) holds for all B of the finite-dimensional form (2.2). This equivalence follows from Lemma 2.3(6) above [take Yt = 6utZ]. Theorem 2.2. Let Z be a one-sided discrete-time or continuous-time shift- measurable stochastic process with a general state space (E, £) and path space (H,7i). Let U be uniform on [0,1] and independent of Z. Then the following statements are equivalent. (a) For each B £ 7i such that P(Z £ B) > 0, there exists a successful distributional shift-coupling of Z and the process with distribution P(z e-\z eB). (b) For each B £ U such that P(ZeB)>0, \\p{8utz e •) - PtfmZ e-\ze B)\\ -> o, t -> oo. (c) For each B eV. such that P(Z £ B) > 0, P(Ze-)\I = p(Ze-\zeB)\I. (d) The process Z is I-trivial. (e) The process Z is Cesaro mixing. Moreover, in (a) we may replace distributional by nondistributional if there exists a weak-sense-regular conditional distribution of Z given OtZ for any random time T [this holds in discrete time when (E, £) is Polish and in continuous time when {E,£) is Polish and the paths are right-continuous]. Finally, in each of (a), (b), and (c) we may restrict B £ V. to be finite- dimensional as at (2.2). PROOF. The equivalence of (a), (b), and (c) follows from Theorem 5.4 in Chapter 5 [take Z' with distribution P(Z £ -\Z £ B)], and so does
200 Chapter 6. MARKOV PROCESSES the nondistributional claim. The equivalence of (d) and (c) follows from Lemma 2.1 above [take A = I\. The equivalence of (e) and (b) follows from Lemma 2.2 above [take Yt = 8utZ]. The equivalence of (a), (b), and (c) with B restricted to be finite-dimensional follows from Theorem 5.4 in Chapter 5 [take Z' with distribution P(Z £ -\Z £ B)]. The equivalence of (c) and (c) with B restricted to be finite-dimensional follows from Lemma 2.3(a) above [take A = 1}. □ 2.3 Epsilon Couplings: <S-Triviality <£> Smooth Mixing <£>••• Finally, assume that Z is a continuous-time shift-measurable process and let U be uniform on [0,1] and independent of Z. The process Z is smoothly mixing if as t —> oo, sup\P(6t+UhZeA,ZeB)-P(6t+uhZeA)P(ZeB)\->0 (2.7) Aen for each B £% and /i > 0. Equivalently, Z is smoothly mixing if and only if (2.7) holds for all B of the finite-dimensional form (2.2). This equivalence follows from Lemma 2.3(6) above [take Yt = 6t+uhZ\- Theorem 2.3. Let Z be a one-sided continuous-time shift-measurable stochastic process with a general state space (E,£) and path space (H^H). Let U be uniform on [0,1] and independent of Z. Then the following statements are equivalent. (a) For each B £ 7i such that P(Z 6 B) > 0, there exist successful distributional epsilon couplings of Z and the process with distribution F(ze-\zeB). (b) For each B £ H such that P(Z £B)>0 and each h > 0, \\P(6t+Uhz e.) - P(ot+Uhz e-\ze B)\\ ->■ o, t -> oo. (c) For each B £ % such that P(Z eB)>0, p(Ze-)\s = P(Ze-\zeB)\s. (d) The process Z is S-trivial. (e) The process Z is smoothly mixing. These statements are also equivalent to each of the following claims. (a1) For each B £ % such that P(Z £ B) > 0 and all e > 0, there exists a successful distributional e-coupling of Z and the process with distribution P(Z £ -\Z £ B).
Section 3. Markov Processes - Preliminaries 201 (c') For each B £ % such that P(Z £ B) > 0 and each h > 0, P(0Uhz e -)lr = P(0Uhz e-\ze B)\T. Moreover, in (a') we may replace distributional by nondistributional if there exists a weak-sense-regular conditional distribution of Z given 6tZ for any random time T [this holds when (E,£) is Polish and the paths are right- continuous}. Finally, in each of (a), (b), (c), (a'), and (c') we may restrict B E Ti to be finite-dimensional as at (2.2). PROOF. The equivalence of (a), (b), (c), (a'), and (c') follows from Theorem 9.4 in Chapter 5 [take Z' with distribution P(Z £ -\Z £ B)\, and so does the nondistributional claim. The equivalence of (d) and (c) follows from Lemma 2.1 above [take A = S]. The equivalence of (e) and (6) follows from Lemma 2.2 above [take Yt = 8t+uhZ}. The equivalence of (a), (6), (c), (a'), and (c') with B restricted to be finite-dimensional follows from Theorem 9.4 in Chapter 5 [take Z' with distribution P(Z £ -\Z £ B)\. The equivalence of (c) and (c) with B restricted to be finite-dimensional follows from Lemma 2.3(a) above [take A = S\. □ 3 Markov Processes - Preliminaries In this section we recall some bare-bone basics for Markov processes and then reduce total variation convergence in the path space to total variation convergence in the state space. 3.1 Basics A discrete- or continuous-time stochastic process Z with a general state space (E, £) and a general path space (H, H) is a Markov process if the future depends on the past only through the present, that is, if for all t ^ 0, 6tZ is conditionally independent of KtZ given Zt. A Markov process Z is time-homogeneous if the conditional distribution of 8tZ given the value of Zt does not depend on t. If Z is a time-homogeneous Markov process and there is, for 0 ^ s, t < oo, a regular version Ps of the conditional distribution of Zt+S given the value ofZt, P{Zt+s eA\zt = x) = ps(x,A), xeE, Ae£, then the family of probability kernels Pl, 0 ^ t < oo, is called the semigroup of transition probabilities, semigroup because Ps+t = PsPl, that is, Ps+t(x,A)= f Pt(y,A)Ps{x,dy), 0 ^ s,t < oo, x £ E, Ae£.
202 Chapter 6. MARKOV PROCESSES In the discrete-time case Pn is simply the nth power of the one-step transition probabilities P = P1. The distribution of a Markov process Z with transition semigroup Pl, 0 ^ t < oo, is determined by the semigroup and the initial distribution A = P(Zo 6 0: tne finite-dimensional distributions are p(ztoeA0,...,ztk eAk) = [■■[ Ptk-tk-1(xk-1,dxk)---Ph{x0,dxl)\(dx0), JA0 JAk 0 = t0 < ■■■ <tk, A0,...,Ak £ £, l^k<oo. Conversely, the following holds. In discrete time, for each probability kernel P and each probability measure A on (E, £) there exists a time-homogeneous Markov process Z with P as one-step transition probabilities and A as initial distribution (Ionescu Tulcea theorem, Fact 4.3 in Chapter 3). And in continuous time, if (E,£) is Polish and H = £t0.°°); then for each semigroup of probability kernels Pl, 0 ^ t < oo, and each probability measure A on (E,£) there exists a time-homogeneous Markov process Z with P(, 0 ^ t < oo, as transition probabilities and A as initial distribution (Kol- mogorov extension theorem, Fact 3.2 in Chapter 3). This need not be the case when F/El0-00). Another Markov process is a version or a differently started version of Z if it has the same semigroup of transition probabilities. We shall denote by Zx a version with initial distribution A. Thus \Pl is the distribution of P(ZX e A) = \Pl(A) := ( P\x, A)\{dx), Ae£. When Zq = x we write Zx. 3.2 Total Variation Reduced from Path Space to State Space For Markov processes total variation convergence in the path space reduces to total variation convergence in the state space, since ||P(6»tzA e ■) - P{8tZx' e -)|| = ||AP( - A'P'H (3.1) due to the following lemma [take Y = 6tZx, Y' = 6tZx', g(Y) = Z?, and 9{Y') = Zfl Lemma 3.1. Let Y and Y' be random elements in a measurable space (E, £) and let g be a measurable mapping from (E, £) to a measurable space (G,g). Then \\P(g(Y) e •) - P(s(n e Oil ^ IIP(^ e 0 - p(*" e Oil- (3.2)
Section 3. Markov Processes - Preliminaries 203 Moreover, if the conditional distribution ofY given the value of g(Y) is the same as that ofY' given the value of g{Y'), then ||P(5(Y) 6 •) - P(3(n 6 Oil = \\P<X GO- P(l" G Oil- (3-3) Proof. For B e Q we have g~lB e £. Thus \P(g(Y) eB)- P(g(Y') G B)| ^ sup \P(Y e A) - P(Y' 6 A)\. Take the supremum in B e G and multiply by 2 to obtain (3.2). In order to establish (3.3), note that by assumption there is, for each ie^.a function Q{-,A) from (G,G) to ([0,1],B[0,1]) such that P(Y eA)- P(Y' 6 A) = E[Q(g(Y), A)} - E[Q(g(Y'),A)}. Thus P{Y EA)- P(r € A) < sup E[f(g(Y))} - E[f{g(Y'))]. /ee Take the supremum in A £ £ and multiply by 2 to obtain the reverse of (3.2), that is, (3.3) holds. □ 3.3 Cesaro and Smooth Total Variation Can Also Be Reduced In order to reduce Cesaro and smooth total variation convergence from the path space to the state space we need to be able to replace t in (3.1) by a random variable that is independent of the processes. The semigroup of transition probabilities is jointly measurable if for each Ae £, the mapping taking (x,t) to Pl{x,A) is £ x B[0,oo)/B[0,l]- measurable. Lemma 3.2. Let Pl, 0 ^ t < oo, be a discrete-time or continuous-time jointly measurable semigroup of probability kernels on a measurable space (E, £) and suppose that for each probability measure A on (E, £) there is a Markov process Zx with state space (E,£), transition probabilities Pl, 0 ^ t < oo, and initial distribution A. Suppose further in the continuous- time case that these processes Z are shift-measurable with a common path space (H, %). Let V be a nonnegative random variable that is independent of the processes Zx and denote its distribution by F. Then 6yZx is a version of Zx with initial distribution P(Z£ € A) = f XPt(A)F{dt), Ae£, and for all initial distributions A and A', \\P(0vZxe-)--p(6vZx' e-)\\ (3.4) I\PlF{dt)- f \'PtF{dt)\\.
204 Chapter 6. MARKOV PROCESSES Proof. Due to Fact 3.1 below [take Y = V, Y' = Zx, and g{V,Zx) = U{9vZx)], P(0vZx 6 A\V = t) = P{6tZx eA\V = t), 0 <^t < oo, AeV.. Due to the independence of V and Zx this yields ~P{6VZX e A\V = t) = P{6tZx €A), 0^t<oo, Aen. Now P{6tZx 6 A) = JP{ZX € A)APe(da;), and thus P(<VZA 6 A) = ffp{Zx 6 A)APe(da;)F(dO, 4eE This means that flyZ* is a version of Zx with J \PtF(dt) as initial distribution, and (3.4) follows from (3.1). □ 3.4 Two Useful Facts The following fact formalizes the intuitively reasonable idea that when we condition on the value of a random element Y = y, then we may replace random elements of the form g(Y, Y') by g{y, Y'). Fact 3.1. Let Y and Y' be random elements in the measurable spaces {E,£) and (E',£') and suppose there is a regular conditional distribution Q{-,-) of Y' given the value of Y. If g is a measurable mapping from (E,£) <g> (E',£') to (ffi,£(ffi)) such that E[g(Y,Y')] exists, then E[g{Y,Y'W = y} = J9{y,y')Q{y,dy') = E[g{y,Y')\Y = y], forP{Y £■) a.e. y &E. For a proof, see Ash (1972), Problem 1 in Section 6.6 and the solution on' page 450. Below we shall need the following fact, the earliest and simplest of the martingale convergence theorems. Fact 3.2. Let X be a bounded random variable defined on a probability space (n,T,P) and Tn, 1 ^ n < oo, an increasing sequence of sub-a- algebras of T. Let T^ be generated by the J-n. Then E[X\Fn) -> EfXI^oo] a.s., t -> oo. For a proof, see Ash (1972), Theorem 7.6.2. 4 Exact Coupling In this section we show that the exact coupling equivalences established so far hold for the whole family of all differently started versions of a Markov process. We also add two more equivalent statements, one on convergence in the state space and one on the constancy of space-time harmonic functions.
Section 4. Exact Coupling 205 4.1 Space-Time Harmonic Functions A measurable function / from ((£,£) ® ([0,oo), B{[0,oo))) to (K,E(R)) is called space-time harmonic with respect to a semigroup of probability kernels P(, 0 ^ t < oo, if f{x,s)= I f{y,8 + t)Pt{x,dy), 0^s,t<<x, xeE. (4.1) If Zx, a; £ £?, is a family of differently started Markov processes with transition probabilities Pl, 0 ^ t < oo, then (4.1) can be written as f{x,s)=E[f{Zx,s + t)], 0^s,t<oo, xeE. For an example of a space-time harmonic function take A &T and put fA{x,t) :=P{ZX €M), 0^i<oo, i eE. (4.2) Then /^ is space-time harmonic, since A € T implies that #SA 6 T, which implies that 6^l6t6sA = 9SA, which yields the first equality in fA(x,s) = p(etzxeeteaA) = E[P(etz* e etetA\zf)] = E[fA{zx,s +t)}. Note that {9nZx e 0nA} = {Zx e A} ior A e T- This yields the second equality in fA(Zx,n) = P(6nZx € e„A|K„Z*,Z*) [Markov property] = P(Z*eA|KnZ*,^) -> P(Z* e A|Z*) a.s., in oo, [by Fact 3.2]. Since P(ZX 6 A|ZX) = 1A{ZX) a.s., we obtain fA(Zx,n) ^ 1A(ZX) a,.s., moo, (4.3) for A &T- This we shall use in the proof of the next theorem. 4.2 The Equivalences We are now ready for the exact coupling equivalences. Theorem 4.1. Let Pl, 0 ^ t < oo, be a discrete- or continuous-time semigroup of probability kernels on a measurable space (E, £) and suppose that for each probability measure X on (E,£) there is a Markov process Zx with state space (E,£), transition probabilities Pl, 0 ^ t < oo, and initial distribution X. Let {H,H) be some common path space. Then the following statements are equivalent.
206 Chapter 6. MARKOV PROCESSES (a) For all initial distributions A and A', there exists a successful distributional exact coupling of Zx and Zx . (b) For all initial distributions A and A', \\P{6tZx £-)-P(9tZx' e-)H-K), i^oo. (c) For all initial distributions A and A', P(ZAe-)lr = P(ZA'e-)lr- (d) For each initial distribution A, Z is T-trivial. (e) For each initial distribution X, Zx is mixing. (/) For all initial distributions A and A', IIAP'-A'P'H-^O, t^oo. (<?) All bounded space-time harmonic functions are constant. Moreover, these statements are equivalent to the existence of a successful nondistributional exact coupling if there exists a weak-sense-regular conditional distribution of Zx given (3tZx for any random time T [this holds in discrete time when {E,£) is Polish and in continuous time when {E,£) is Polish and the paths are right-continuous]. 4.3 Proof of Theorem 4.1 The equivalence of (a), (b), and (c) follows from Theorem 9.4 in Chapter 4, and so does the final claim. The equivalence of (d) and (e) follows from the equivalence of (d) and (e) in Theorem 2.1. The equivalence of (/) and (b) follows from (3.1). Thus the theorem is established if we can show that (d) implies (c), that (/) implies (<?), and that (<?) implies (d). Suppose (d) holds. Fix A and A' and put A" = (A 4- A')/2. Then, for AeT, {P{ZX eA) + P(ZX' € A))/2 = P(ZX" € A) = 0 or 1. Thus either P{ZX 6 A) and P(ZV 6 A) are both 0 or both 1. Thus (d) implies (c). Suppose (/) holds. Then, with / a bounded space-time harmonic function, x,y 6 E, and 0 ^ s < oo, |/(i, s) - f(y, s)\ = \J f(z, s + t)P\x, dz) - j f(z, s + t)Pl{y, dz)\ <sup\f\\\P\x,.)-P\y,-)\\^0, t^cxi.
Section 5. Shift-Coupling 207 Thus f(x,t) does not depend on x. This together with (4.1) implies that f{x,t) does not depend on t either. Thus (/) implies (g). Suppose (g) holds. Then the function fA defined at (4.2) is constant. Thus fA{Zx,n) = fA{x,0) = P(ZA £ A) for an arbitrary A, and from (4.3) we obtain that P(ZA £ A) = 0 or 1. Thus (g) implies (d), and the proof of Theorem 4.1 is complete. 5 Shift-Coupling We now show that the shift-coupling equivalences established so far hold for the whole family of all differently started versions of a Markov process. We also add two more equivalent statements, one on Cesaro convergence in the state space and one on the constancy of harmonic functions. 5.1 Harmonic Functions A measurable function / from ((£,£), B([0, oo))) to (ffi, B(R)) is harmonic with respect to a semigroup of probability kernels Pl, 0 ^ t < oo, if f{x) = J f(y)Pt(x,dy), 0^*<oo, x 6 E. (5.1) A harmonic function can be viewed as a space-time harmonic function that is constant in the time parameter. Let Zx, x € E, be a family of differently started Markov processes with transition probabilities Pl, 0 ^ t < oo, shift-measurable in the continuous- time case with a common path space {H,H). Then (5.1) can be written as f[x) = E[/(Zf)], 0^i<oo, xeE. For an example of a harmonic function take A 6 X and put fA{x) := P{ZX € A), 0 ^ t < oo, x 6 E. (5.2) Then fA is harmonic, since fA{x) = P{6tZx eA) [AeX means 0"1 A = A] = E[P{6tZx 6 A\ZX)) = nfA(zn}. From (4.3) we obtain fA(Zx) -»■ lA(Zx) a.s., n ^ oo, (5.3) for A £l. This we shall use in the proof of the next theorem.
208 Chapter 6. MARKOV PROCESSES 5.2 The Equivalences We are now ready for the shift-coupling equivalences. Theorem 5.1. Let Pl, 0 ^ t < oo, be a discrete-time or continuous-time jointly measurable semigroup of probability kernels on a measurable space (E, £) and suppose that for each probability measure A on (E, £) there is a Markov process Zx with state space {E,£), transition probabilities Pl, 0 ^ t < oo, and initial distribution A. Suppose further in the continuous- time case that these processes Zx are shift-measurable with a common path space {H,7i). Let U be a random variable that is uniform on [0,1] and independent of the processes Zx. Then the following statements are equivalent. (a) For all initial distributions A and A', there exists a successful distributional shift-coupling of Zx and Zx . {b) For all initial distributions A and A', \\P{6UtZx € ■) - V{9utZx' € -)ll -► 0, t ->■ oo. (c) For all initial distributions A and A', P(ZAeO|i = P(zv eOIz- (d) For each initial distribution A, Zx is X-trivial. (e) For each initial distribution A, Zx is Cesaro mixing. (/) For all initial distributions A and A', 0 / \Psds- / \'Psds " Jo Jo t —► 00. (<?) All bounded harmonic functions are constant. Moreover, these statements are equivalent to the existence of a successful nondistributional shift-coupling if there exists a weak-sense-regular conditional distribution of Zx given 9tZx for any random time T [this holds in discrete time when (E, £) is Polish and in continuous time when (E, £) is Polish and the paths are right-continuous]. 5.3 Proof of Theorem 5.1 The equivalence of (a), (b), and (c) follows from Theorem 5.4 in Chapter 5, and so does the final claim. The equivalence of (d) and (e) follows from the equivalence of (d) and (e) in Theorem 2.2. The equivalence of (/) and (b) follows from (3.4) with V — Ut and F the uniform distribution on [0,t].
Section 6. Epsilon-Coupling 209 Thus the theorem is established if we can show that (d) implies (c), that (/) implies (<?), and that (g) implies (d). Suppose (d) holds. Fix A and A' and put A" = (A + A')/2. Then, for A el, (P(ZA e A) + P(ZV e A))/2 = P{ZX" 6 A) = 0 or 1. Thus either P(ZA 6 A) and P(ZV 6 A) are both 0 or both 1. Thus (d) implies (c). Suppose (/) holds. Then, with / any bounded harmonic function and x,y € E, we obtain by averaging over the time parameter in (5.1) \m-m\ = t-11 J J f(z)Ps(x, dz) ds- J f f(z)Ps(y, dz) ds\ ^ sup|/| HI / Ps{x,-)ds- [ Ps{y,-)ds 11 Jo Jo -> 0, t -^ oo. Thus f(x) does not depend on x. Thus (/) implies (<?). Suppose (<?) holds. Then the function /a defined at (5.2) is constant. Thus fA{Z^) = P(ZA 6 A) for an arbitrary A, and from (5.3) we obtain that P{ZX 6 A) = 0 or 1. Thus (g) implies (d), and the proof of Theorem 5.1 is complete. 6 Epsilon-Coupling In this section we show that the epsilon-coupling equivalences established so far hold for the whole family of all differently started versions of a Markov process. We also add two more equivalent statements, one on smooth convergence in the state space and one on the constancy of certain space-time harmonic functions which we shall call smooth. 6.1 Smooth Space-Time Harmonic Functions We now restrict attention to the continuous-time case. Call a space-time harmonic function / smooth if for all 0 ^ t < oo and x € E, I f{y,t)P'{x,dy)->f{x,t), siO. (6.1) If Pl, 0 ^ * < oo, is jointly measurable and / is bounded space-time harmonic, then the smoothed version /(ft) defined, for h > 0, by f^h\x,t) := h~l J ff{y,t)Pu{x,dy)du, 0^t<oo, x € E, (6.2)
210 Chapter 6. MARKOV PROCESSES is smooth space-time harmonic: smoothing (4.1) yields that /(h) is space- time harmonic, and the smoothness follows from J fih)(y,t)Ps(x,dy)-f^(x,t) . rs+h r rh r = h~l\ f(z,t)Pu(x,dz)du- f(y,t)Pu(x,dy)du = h~l | J' J f(z, t)Pu(x, dz) du- J J f(y, t)Pu(x, dy) du ^ 2/T1sup|/| s ->0, slO. Note that if / is smooth space-time harmonic, then /(h) —y f pointwise as hlO. If Zx, x 6 E, is a family of differently started Markov processes that are shift-measurable and have a common path space (H, %) and jointly measurable transition probabilities P4, 0 ^ t < oo, then (6.1) can be written as E[/(Z.V)1-►/(*,0, no, and (6.2) as fW(x,t)=E[f(Z*Uh,t)}, where U is uniform on [0,1] and independent of Zx. 6.2 The Equivalences We are now ready for the epsilon-coupling equivalences. Theorem 6.1. Let Pl, 0 ^ t < oo, be a continuous-time jointly measurable semigroup of probability kernels on a measurable space {E, £) and suppose that for each probability measure A on (E, £) there is a Markov process Zx with state space (E,£), transition probabilities Pl, 0 ^ t < oo, and initial distribution A. Suppose further that these processes Zx are shift- measurable with a common path space (H,7{). Let U be a random variable that is uniform on [0,1] and independent of the processes Zx. Then the following statements are equivalent. (a) For all initial distributions A and A', there exist successful distributional epsilon-couplings of Zx and Zx . (b) For all initial distributions A and A' and all h > 0, \\P(9t+UhZx G •) - ?(0t+UhZx' G -)ll -► 0, t ->■ oo.
Section 6. Epsilon-Coupling 211 (c) For all initial distributions A and \', P{Zxe-)\s = P{Zy e-)|s. {d) For each initial distribution A, Z is S-trivial. (e) For each initial distribution X, Zx is smoothly mixing. (/) For all initial distributions A and A' and each h > 0, / \Pt+sds- [ \'Pt+sds Jo Jo -t 0, t -> oo. (<?) All bounded smooth time-harmonic functions are constant. These statements are also equivalent to each of the following claims. (a') For all initial distributions A and A' and each e > 0, there exists a successful distributional e-coupling of Zx and Zx . (c') For all initial distributions A and A' and each h > 0, P{eUhzxe-)\T = P{8uhZx' e-)lr- Finally, these statements are equivalent to the existence of a successful nondistributional e-coupling of Zx and Zx for all initial distributions A and A' and each e > 0 if there exists a weak-sense-regular conditional distribution of Zx given 8tZx for any random time T [this holds when (E, S) is Polish and the paths are right-continuous]. 6.3 Proof of Theorem 6.1 The equivalence of (a), (b), (c), (a'), and (c') follows from Theorem 9.4 in Chapter 5, and so does the final claim. The equivalence of (d) and (e) follows from the equivalence of (d) and (e) in Theorem 2.3. The equivalence of (/) and (b) follows from (3.4) with V = t + Uh and F the uniform distribution on [t,t + h]. Thus the theorem is established if we can show that (d) is equivalent to (c), that (/) implies (g), and that (g) implies (c'). Suppose (d) holds. Fix A and A' and put A" = (A + A')/2. Then, for A 6 S, {P{ZX eA) + P{ZX' 6 A))/2 = P(ZX" 6 A) = 0 or 1. Thus either P(ZX £ A) and P(ZX' e A) are both 0 or both 1. Thus (d) implies (c). Suppose (c) holds. Then, for each time t and initial distribution A, P{6tZx eA)= P{9tZx e A\KtZx, Zx) a.s, AGS, (6.3)
212 Chapter 6. MARKOV PROCESSES since 9tZx is a version of Zx (Markov property and time-homogeneity) and since, conditionally on its initial value and past, 6tZx is a differently started version of itself (Markov property). With A £ S, this yields the second equality in P(ZA 6 A) = P(9nZx £ 6nA) [holds for any tail set A] = P{6nZx 6 enA\nnZx, Zx) a.s. [due to (6.3)] = P(ZA 6 A\nnZx, Zx) a.s. [holds for any tail set A] -> P(ZA e A\ZX) a.s., ri -> oo, [due to Fact 3.2]. Thus P(ZA 6 A) -> U(ZA) a.s. as n -> oo, that is, P(ZA 6 A) = 0 or 1. Thus (c) implies (d). Suppose (/) holds. Then, with / a bounded smooth space-time harmonic function and /(h) the smoothed version defined at (6.2), we have, for x,y € E and 0 ^ t < oo, \fW{x,t)-fW{y,t)\ = j f(h\z,t + s)P'(x,dz) - J f^{z,t + s)Ps{y,dz)\ [by (4.1) applied to f^] = h~1\ f f{z,t + s)( f Ps+U{x,dz)du^j -Jf(z,t+s)(J Ps+U(y,dz)du)\ [by (6.2)] / Pu+s{x,-)du- / Pu+s{y,-)du Jo Jo —> 0, s —> oo. Thus f(h\x,i) does not depend on x. This together with (4.1) implies that f(h\x,t) does not depend on t either. Since /(h) —► / pointwise as h J. 0, f{x,t) depends on neither a; nor t. Thus (/) implies (<?). Suppose (g) holds. With A € T and /^ the bounded space-time harmonic function defined at (4.2), we have that the smoothed version fA ' satisfies fAh){x,t)=P{eUhZx eM), h>0, 0^t<oo, xeE. Since }{A\x,t) is constant in x (and £), we obtain that P(6UhZx e •) does not depend on the initial state x. Thus (5) implies (c'), and the proof of Theorem 6.1 is complete.
Section 7. Stationary Measure 213 7 Stationary Measure Consider a discrete- or continuous-time Markov process Z with state space (E, S) and a semigroup of transition probabilities Pl, 0 ^ t < oo. A measure it on (E, S) is a stationary measure for Z if WP1 = 7T, 0 ^ * < OO. A stationary measure with mass one is a stationary distribution. In Chapter 2 (Section 3) we established the existence of a unique stationary distribution for irreducible aperiodic positive recurrent Markov chains and the existence of a <r-finite stationary measure in the null recurrent case. (See Chapter 10, Theorems 3.1 and 4.1 and Section 4.5, for stationarity in the case of regenerative Markov processes.) 7.1 Stationary Distribution — Asymptotic Stationarity If there exists a stationary distribution it, then the equivalent statements in Theorems 4.1, 5.1, and 6.1, respectively, are clearly equivalent to each of the following claims: VA : XPl 4 7T, t ->• oo, [add to Theorem 4.1] i-t VA : / XPS ds % 7T, t -> oo, [add to Theorem 5.1] Jo VA, V/i >0:h~1 \Pt+s ds 4 tt, t -> oo, [add to Theorem 6.1]. Jo When there exists a stationary measure with finite mass, then dividing by the mass yields the existence of a stationary distribution. Thus if the equivalent statements in Theorem 5.1 hold, then a finite stationary measure is unique up to multiplication by a constant. 7.2 Stationary Distribution — Piecewise Constant Paths Taking a stationary distribution as initial distribution clearly yields (due to the time-homogeneity) a stationary version of the Markov process. When further the paths are piecewise constant with finitely many jumps in finite intervals, then according to Theorem 7.2 in Chapter 5, the existence of successful epsilon-couplings implies total variation convergence to stationarity in the state space. Thus, if there exists a stationary distribution and the paths are piecewise constant with finitely many jumps in finite intervals, then the equivalent statements of Theorem 6.1 imply the stronger statements in Theorem 4.1, that is, the equivalent statements of Theorem 6.1 are equivalent to those in Theorem 4.1.
214 Chapter 6. MARKOV PROCESSES In particular, the existence of successful epsilon-couplings implies the existence of a successful exact coupling. This is surprising, but not too surprising because the sojourn times in the states visited are exponential. We leave open the question whether this holds without the assumption that there exists a stationary distribution. 7.3 A cr-Finite Stationary Measure — Uniform Nullity Suppose there exists a a-finite stationary measure w with infinite mass, that is, tt(E) = co, and there is a sequence of sets A\,A2,--- £ £ such that ir(An) < co and An t E as n —>• oo. Let ir(-\An) be the conditional stationary measure, that is, ^(-^n) is the probability measure on (E, £) defined by ir(A\An) = ir(AnAn)/ir(An), Ae£. Clearly, n(-\A„) ^ w/-ir(An) and thus w(-\An)P\A) 4: vP\A)MAn) = ir(A)/ir(An). Thus A = tt(-|A„) =► P(ZeA G •) ^ ir/ir(An), 0 < * < oo. (7.1) This can be used to deduce the following result on 'uniform nullity'. Theorem 7.1. (i) Suppose the equivalent statements of Theorem 4-1 hold. If there exists a a-finite stationary measure it with infinite mass, then for each initial distribution X and each c < oo, as t —> oo, XPl(A) ->■ 0 uniformly in A £ £ such that it {A) ^ c, (7.2) or equivalently, for each initial distribution X and each e > 0, as t —> oo, XPt(A) Vtt ->■ 0 uniformly in Ae £. (7.3) e + ir(A) " v ; (ii) Suppose the equivalent statements of Theorem 5.1 hold. If there exists a a-finite stationary measure it with infinite mass, then for each initial distribution X and for each c < oo, as t —► oo, t~l I XPS{A) ds^O uniformly in A £ £ such that n(A) ^ c Jo
Section 7. Stationary Measure 215 or equivalently, for each initial distribution A and each e > 0, as t —>• oo, f1 /' XPs(A)ds A „ ttt > 0 uniformly in A G £. e + ir(A) (Hi) Suppose the equivalent statements of Theorem 6.1 hold. If there exists a a-finite stationary measure it with infinite mass, then for each initial distribution A, each h > 0, and each c < oo, as t —> oo, jt+s/ / \Pt+s(A) -> 0 uniformly in A£ £ such that n(A) ^ c, Jo or equivalently, for each initial distribution A, each h > 0, and each e > 0, as t —> oo, tfxpt+'iA) — r~r, ► 0 uniformly in A G £. e + n(A) 7.4 Proof of Theorem 7.1 Let Z have an arbitrary initial distribution A and let Z' have the initial distribution 7r(-|v4„). Since P(Zt eA)< \\P(Zt e •) - P(Z't e Oil + P(3 e A) and since [due to (7.1)] P(Z't G •) ^ 7r/7r(An), (7.4) we obtain sup P(Ze G A) ^ \\P(Zt g •) - P(Z(' G -)ll + c/tt(A„). Since one of the equivalent statements in Theorem 4.1 claims that ||P(ZtG-)-P(3e-)ll->o, ^0O; this yields limsup sup P(Zt Ei)^ c/ir(An). Send n —>• oo to obtain (7.2).
216 Chapter 6. MARKOV PROCESSES In order to see that (7.2) implies (7.3) suppose (7.2) holds. Then \Pf(A) ^ suPw(AKcAP'(A) l Ae£e + 7r(^4) e e + c , t -> co, [due to (7.2)] e + c —> 0, c —>• oo. In order to establish the converse suppose (7.3) holds. Then XPt(A) sup XPt(A)^(e + c) sup i—f- -> 0, * -> oo. Aef Aef £ + Tr(A) Thus (i) is established. To establish (ii) and (Hi), let [/ be uniform on [0,1] and independent of Z and Z' and note that (7.1) yields both P(Z'ut G •) ^ 7r/7r(An), 0 ^ t < oo, and P(Z't+Uh G •) ^ 7r/7r(An), 0 ^ t < oo. In'The above argument use these inequalities rather than (7.4) and replace P(Zt £ A) by P(Zut e A) and P(Zt+uh € A), respectively, to obtain (ii) and (m). Theorem 7.1 is established. For more on Markov processes, see Section 4.5 in Chapter 10 (in particular, Remark 4.2).
Chapter 7 TRANSFORMATION COUPLING 1 Introduction The last three chapters were concerned with shifting one-sided stochastic processes, 6fZ = (Zt+S)o^s<00. This chapter extends the view to an abstract setup where general random elements replace stochastic processes and a semigroup of transformations G replaces the shift-maps 6t, 0 ^ t < oo. As mental preparation for this leap, we start off in Section 2 by observing that the shift-coupling theory of Chapter 5 (Sections 2 through 5) extends from one-sided processes to two-sided processes, Z = (Zs)_00<s<00. The two-sided case is even easier to deal with, since, while the one-sided shifts do not form a group, the two-sided shifts, 6tZ := (Zt+s)-oo<s<oo, do: if we shift the origin to t, we have not lost what happened before time t and can shift back, 8-t8tZ = Z. The same observation applies to random fields with the index set B.d (processes in d-dimensional time). The main part of the chapter, Sections 3 through 6, then deals with transformation coupling: the generalization of shift-coupling. In order to stress similarities (and dissimilarities), the treatment parallels that of shift- coupling presented in Sections 2 through 5 of Chapter 5: we use analogous section titles and enumerate the theorems in the same way. Several proofs are more or less replicas of the proofs in the shift-coupling case, but we go through all details again to explicate where the abstract conditions enter. One of these conditions is the existence of an invariant measure (an analogue of the Lebesgue measure), which is essential in this theory and is simply assumed. This semigroup theory applies, for instance, to random fields with index set [0, oo)d. 217
218 Chapter 7. TRANSFORMATION COUPLING In Section 7 we spell out the implications of transformation coupling in the special case when G is a locally compact second countable topological group. This has similarities with the step from one-sided processes to two- sided, but this step is even more pleasant because it hands us the existence of an invariant measure, the Haar measure. Section 8 indicates applications: selfsimilarity, exchangeability, rotational invariance, ... Section 9 rounds off by considering briefly a possible generalization of exact coupling, taking random fields as a specific example. 2 Shift-Coupling Random Fields In this section we consider shift-coupling of random fields in d dimensions, highlighting aspects that distinguish this case from the case of one-sided stochastic processes. This is in part a preview of what is to come because for several claims we refer to the theory of transformation coupling to be developed in the subsequent sections. 2.1 Preliminaries Call a stochastic process (see Section 2 of Chapter 4) with the index set B.d(d ^ 1) a random field in d dimensions. Thus, in this terminology, a two- sided continuous-time stochastic process is a random field in one dimension. Call B.d the site set and a random element in (B.d,B(B.d)) a random site. Let Z - (Zs)s€Rd and Z' = (Z's)s€Rd be two shift-measurable random fields with a general state space (E,£) and path space {H.'H). Define the shift-maps 6t, t £ B.d, by 6tz = (zt+s)seRd, z £ H. Shift-measurability means that 8tH = H,t £ B.d, and that the mapping taking (z,t) in Hx Rd to 6tz in H is n®B(Rd)/'H measurable [that is, the mapping taking (z,t) in H x Rd to zt in E is 7i <g> B(Rd)/£ measurable]. Shift-measurability holds in the standard settings [when (E,£) is Polish and the processes right-continuous; see Section 2.8 in Chapter 4]. Unlike what was the case in Chapter 5, we need never impose any restrictions beyond shift-measurability in this section. 2.2 Shift-Coupling — Distributional Shift-Coupling Say that (Z,Z',T,T',C) is a (nondistributional) shift-coupling of Z and Z' if (Z, Z') is a coupling of Z and Z', T and T' are random sites, and C is an event such that 6TZ = 6T'Z' on C. (2.1)
Section 2. Shift-Coupling Random Fields 219 Say that (Z,Z',T,T',C,C) is a distributional shift-coupling of Z and Z' if (Z, Z') is a coupling of Z and Z', T and X" are random sites, and C and C are events such that P(0r£e-,C)=P(0T'£'e-,C). [thus P(C) = P(C")] (2.2) The shift-coupling (distributional or not) is successful if P(C) = 1. In this case we somtimes leave out the events and only write (Z,Z',T) or (Z, Z',T,T') for the shift-coupling. Observe that for one-sided processes we obtain the shift-couplings of Chapter 5 (Section 2) from (2.1) and (2.2) by taking C = {T < oo} and C" = {X" < oo}. In the present case it is no longer natural to use {T < oo} and {T' < oo} for the shift-coupling events C and C". With these definitions the shift-coupling results in Chapter 5 (Sections 2 through 5) still hold. This can be seen either by repeating the arguments in Chapter 5 with straightforward modifications, or by referring to the abstract group theory in Section 7 below. In fact, the present case is easier to deal with, since [unlike the one-sided shifts in Chapter 5] the shift-maps now form a group: if we shift the origin to t, we have not lost a part of the process and can shift back to the initial origin. In particular, this yields [see Theorem 3.2 below] that now distributional shift-coupling can always be made nondistributional without assuming [as we had to do in Theorem 2.2 of Chapter 5] that (E, S) is Polish and the processes right-continuous. 2.3 Shift-Coupled Fields Identical, Only with Different Origins The group property allows us also to simplify the definition of nondistributional shift-coupling. With S:=T-T' definition (2.1) can be rewritten as 6SZ = Z' on C. (2.3) Thus on C the two random fields are really the same, only with different origins. Call (Z, Z', S, C) a shift-coupling of Z and Z' with shift S if (2.3) holds. The distributional version of this is as follows: call (Z,Z',S,C,C) a distributional shift-coupling of Z and Z' with shift S if P(0sze-,C) = P(z'e-,C"). In the successful case this becomes 6sZ = Z', and we can immediately turn the distributional shift-coupling (Z, Z1, S) into the nondistributional shift-coupling (Z,6sZ,S).
220 Chapter 7. TRANSFORMATION COUPLING 2.4 Shift-Coupling Inequality - F0lner Averaging Sets The shift-coupling inequality and the associated Cesaro total variation convergence over intervals [0, t] in Chapter 5 (Section 3) extend naturally to the sets [0,t]d, but also to [-£,0]d and to {-t,t]d. In fact the sets can be quite general. Let A be the Lebesgue measure on B(Rd) and for all B £ B(Rd) such that 0 < X(B) < co, let A(-|£) be the uniform distribution on B, that is, X(A\B) := X(A n B)/X(B), A £ B(Rd). The following shift-coupling inequality holds (see Section 7.3 below): for B £ B(Rd) such that 0 < X(B) < oo, ||P(0t,B Z £ •) - P(0Ub Z' € -)||< 2 - 2E[X(S + £|£); C], (2.4) where [/^ is uniform on B [that is, Ub has the distribution A(-|£)[ and independent of Z and Z'. Thus the Cesaro total variation convergence extends to the following general class of averaging sets. Call a family Bh £ B(Rd), 0 < h < oo, F0lner averaging sets if 0 < X{Bh) < oo, (2.5) X{t + Bh\Bh) -> 1 as /i-s-co, tel"1. When P(C) = 1, we obtain from (2.5) [take B = Bh m (2.4)] the Cesaro total variation convergence l|P(0t/BhZeO-P(0£/BhZ'eOII->o, ft ^oo. (2.6) This generalization of the Cesaro total variation convergence is not restricted to random fields with index set B.d. It works for one-sided processes and for random fields with index set [0, oo)d; see Section 4 below. 2.5 The Sets hB Are F0lner We shall now give a nice example of F0lner averaging sets, which shows clearly how general they are. Theorem 2.1. If B £ B(Rd) and 0 < X(B) < oo, then the family hB := {hs € Ud : s £ B}, 0 < h< oo, ore F0lner averaging sets. Proof. Note first that X{hBf)(t + hB))/X(hB) = X(B<l(t/h + B))/X(B), t€Rd, (2.7)
Section 2. Shift-Coupling Random Fields 221 and that [with || • H2 denoting the L2 norm with respect to A ] X(B) - \(B n (t/h + £?)) = 2"1 [(1b - It/h+B? d\ J (2.8) = 2-1(||lB-le/h+B||2)2. Let /„, n ^ 1, be a sequence of bounded continuous functions such that His — /„j|2 —>■ 0, n-+00, [see Ash (1972), Theorem 2.4.14] to obtain [use \\lt/h+B - fn(- ~ t/h)h = ||ls - fnh in tlie first steP] Ills - h/h+Bh < 2||1B - /„||2 + ||/„ - /„(■ - t/h)h ->2||lB-/n||2 as/n-oo —> 0 asn-> 00. From this and (2.8) we obtain X(B) - \(B n (t/h + B)) -+ 0 as h -+ co, and a reference to (2.7) completes the proof. □ 2.6 The Invariant cr-Algebra — Equivalences Define the invariant a-algebra by l={Aen:8tA = A,te Rd}. Note that I also equals {A £ 7i : 6~[lA = A,t £Rd}, since 0t-1 = 0_e for (el'i. The following claims are equivalent [see the end of Section 7 below]: (a) There exists a successful distributional shift-coupling of Z and Z'. (a1) There exists a random site T such that 8tZ = Z'. (b) For some F0lner averaging sets Bh, 0 < h < 00, (2.6) holds. (6') For all F0lner averaging sets Bh, 0 < h < 00, (2.6) holds. (c) P(Ze-)z = P(Z'e-)i- When Z' is stationary [that is, Z' = 6tZ' for all t£Rd], then (2.6) becomes eUBhziAzl, h^oo. It follows from the equivalence of (c) and (6) that two stationary random fields agree in distribution on I if and only if they are identically distributed.
222 Chapter 7. TRANSFORMATION COUPLING 2.7 Shifting a Single Random Field to Obtain Them All We shall end this section by showing that all shift-measurable random fields that agree in distribution on the invariant u-algebra can be represented as a single random field with the origin at different random sites. Theorem 2.2. Let Z be a shift-measurable random field with site set Rd defined on a probability space (fi, T, P). All random fields Z' agreeing with Z in distribution on I can be represented in the form 9tZ on the same extension of (fi, T, P). Proof. Extend (fi, T, P) to support independent variables U\,.. ■ ,Ud that are uniform on [0,1] and independent of Z. For an arbitrary Z' agreeing with Z in distribution on I let (Z,Z',T) be a successful nondistribu- tional shift-coupling [exists because (c) implies (a1)]. Let T, denote the ith component of T, that is, T = (T\,... ,Td)- Let Pj(-|-) be a regular version of P(fj e •|(Z',fi,...,f'i_1) = •), and let Fj(-|-) be the associated distribution function and F~1(-\-) its generalized inverse [see Chapter 1, Section 3]. Define recursively, for i — l,...,d, the components of T by Ti = Fr\Ui\{Z,Tu...,Ti-1)). □ 3 Transformation Coupling We now turn to the abstract semigroup setup. This section introduces transformation coupling and its distributional version. Let G be a class of measurable mappings [transformations) from a measurable space (E, £) to itself. With 7 £ G and y £ E we write 72/ rather than 7(2/) in the same way as we wrote 6fZ when shifting paths in the stochastic processes case. With 7 and r/ £ G let 777 denote the mapping taking y € E to ■yqy. Assume that G is a semigroup, that is, 7 and 7] e G => 77? £ G, (3.1) and say that G is a transformation semigroup acting on (E,£). Further, let Q be a u-algebra of subsets of G and assume that the measurable semigroup (G, Q) is jointly measurable, that is, the mapping from G x G to G taking (7, if) to 777 is Q ® G/G measurable, and that (G, G) acts jointly measurably on (E,£), that is, the mapping from G x E to E taking (7, y) to -yy is G <8> £/£ measurable. The class G is a group if in addition to the semigroup property (3.1), it holds that 7 £ G => 7 has an inverse 7 : and 7 'eG.
Section 3. Transformation Coupling 223 If G is a group, call G inverse-measurable if the mapping from G to G taking 7 to 7-1 is Q/Q measurable. For an example, consider a random field with state space (E,£), site set [0,oo)d, and an internally shift-invariant path set H C £;[°>°°) . Then the shift-maps G = {8t : t £ [0, oo)d} form a transformation semigroup acting on the path space (//,%). We can identify G with [0,oo)d under addition and let Q be the Borel subsets of [0, oo)d. If we replace the site set [0, oo)d by Rd, then the shift-maps G = {8t : t £ Rd } form a transformation group acting on the path space (//,%). In this case we can identify G with Rd under addition and let Q be the Borel subsets of Rd. In both cases (G, Q) is jointly measurable. And in both cases (G, Q) acts jointly measurably if and only if the random field is canonically jointly measurable (that is, shift-measurable). More examples of transformation groups are given in Section 8: permuting the index set of a random field with a countable index set, rescaling time and space of a real-valued stochastic process, rotating a random field with site set Rd. 3.1 Transformation Coupling — Definition Let Y and Y' be random elements in (E, £) defined on a probability space (fi, T, P). A random transformation is a random element r in (G,Q). The expression rY denotes the random element in (E, £) defined by (rY)(u) := r(u)Y(u), wefi. Call (y, y', r, r1, C) a transformation coupling of Y and Y' if (Y, Y') is a coupling of y and Y', r and i~" are random transformations, and C is an event such that rY = r'Y' on C. (3.2) The transformations r and i~" are the coupling transformations, P(C) is the success probability, and (Y,Y',r,r',C) is successful if P(C) = 1. 3.2 Distributional Transformation Coupling — Definition Call (Y,Y',r,r',C,C) a distributional transformation coupling oiY and y' if (Y,Y') is a coupling of Y and Y', r and P are random transformations, and C and C" are events such that p(ry e-,C) = p(r'y'e-,c") [thus P(C) = p(C)]. (3.3) When P(C) = 1, this becomes rt = r'Y'.
224 Chapter 7. TRANSFORMATION COUPLING A transformation coupling can be seen as a distributional transformation coupling by identifying (Y,Y',r,r',C) with (Y,Y',r,r',C,C). Use the word nondistributional to distinguish a transformation coupling from a distributional one. Let A be a fixed element not in E (the censoring state). Let Aq be the constant mapping Aoy = A for all y £ E. Let A\ be the identity mapping A\y — y for all y € i?. Then we can rewrite (3.3) as AlcrY^Alc,r'Y'. 3.3 Dropping the Hats in the Distributional Case Unlike in the shift-coupling case, we now need conditions for dropping the hats: when we have a distributional transformation coupling of Y and Y', then we can take Y and Y' to be the original random elements Y and Y' if, for instance, (G, Q) is Polish. Theorem 3.1. Suppose (Y,Y',r,r',C,C') is a distributional transformation coupling of Y and Y'. If there exist weak-sense-regular versions of t given Y and of f" given Y' [this holds when (G,Q) is Polish], then the underlying probability space can be extended to support random transformations r and jT' and events C and C such that (Y,r,lc) = (Y,f,lG) and (Y',r',lc-) = (Y',f',l6,). In particular, P(rY e-,C) = P(r'r'e-,c"). Proof. This follows from the conditioning extension in Section 2.12 of Chapter 4. In order to obtain r take Yx := Y and (Y{, Y2') := (Y, (f, ld)) and define (P, lc) :=Y2.ln order to obtain P take YX:=Y' and (Y{, y2') : = (Y', (/", 1&)) and define (J", lc-) := ^2- □ 3.4 Turning Distributional into Nondistributional When either (E, £) is Polish or (G, Q) is a Polish inverse-measurable group, then a distributional transformation coupling can be turned into a nondistributional one. Theorem 3.2. Let (Y,Y',r,r",C,C') be a distributional transformation coupling ofY andY'. Suppose either there exists a weak-sense-regular conditional distribution of Y' given t'Y' and £ ® £ contains the diagonal (3.4) {(y,y) '■ V £ E] [both conditions hold when (E,£) is Polish]
Section 4. Inequality and Asymptotics 225 or (G, Q) is an inverse-measurable group and there exists a weak-sense-regular conditional distribution (3-5) of r' given rY' [this holds when {G,Q) is Polish]. Then there is a nondistributional transformation coupling (Y,Y',r,r',C) of Y and Y' such that (Y,r,ic) = (Y,r,ic) and (Y',r',ic) = (Y',f',i6,) and (Y,r,C) can be identified with (Y,r,C). Proof. If (3.4) holds, apply Theorem 7.5 in Chapter 4 as follows. Let both g and g' be the mapping taking (7, y, i) in G x E x {0,1} to A^y. Put V:=(Y,t,ld) and V := (?',/", l&). Then g(V) = AletY ^ Ale,t'Y' = g'(V), and we obtain the desired result [since a(£ U {0, {^\}}) <8> cr(£ U {0, {^}}) contains the diagonal of E U {A} x E U {A}]. If (G, 5) is an inverse-measurable group and (3.5) holds, take (Y, r, lc) '■ = (Y,r, 1q) and obtain (Y',r') by applying the transfer extension in Section 2.12 of Chapter 4 as follows. Take Yi := AicTY and (Y{,Y^) : = (Aic,r'Y', ?') and define T' := Y2 to obtain (z\lcrf,r') = (z\lc,r'f,f')- Since G is an inverse-measurable group, this implies that A\cr'~lrY = A\ Y\ where r"_1 denotes the group inverse of J1'-1 [that is, for each fixed outcome u, r"_1(o;) is the group inverse of r'(u); thus here T'-1 is not the inverse of i~" as a mapping of w ]. Let W be a random element in (£, £) that is independent of C and has the distribution P(Y' € -|C"C). Define Y' by f' := ^icT'-^y on C and f" := W on Cc. D 4 Inequality and Asymptotics The last section was devoted to the definition of transformation coupling and its distributional version. We shall now go on to the associated inequality and to the limit implications. For this purpose we need an analogue of the Lebesgue measure. So let us assume now that there exists a finite or cr-finite measure A on (G,Q) that is right-invariant, that is, A is nontrivial [A(G) > 0] and A<y G Q and X(Aj) = X(A), A&g, 7 g G. (4.1)
226 Chapter 7. TRANSFORMATION COUPLING Recall that A is a-finite if X(G) = oo and G is the union of disjoint sets in G each with finite mass. For B G G such that 0 < X(B) < oo, let X(-\B) be the probability measure obtained by conditioning on B, that is, X(A\B) = X(AnB)/X(B), A£G. (4.2) 4.1 Transformation Coupling Inequality Here is a generalization of the shift-coupling inequality. Theorem 4.1. Let Y and Y' be random elements in a general space (E, £). Let (G, G) be a jointly measurable semigroup acting jointly measurably on (E,£). Let (Y,Y',r,r',C,C) be a transformation coupling ofY andY', distributional or not. Suppose there exists a finite or a-finite right-invariant measure X on (G,G)- Then, for B G G such that 0 < X(B) < oo, WP(UBY G •) - P(UBY' G -)ll TRANSFORMATION ^ 2E[1 - X(Br\B);C] COUPLING + 2E[1 - X(Br'\B); C] + 2P(Ce), INEQUALITY where Ub has distribution X(-\B) and is independent ofY andY'. Comment. This inequality can clearly be rewritten in the following form: / p(7r g -)A(d7) - / p(7y g -)A(d7) Jb Jb ^2F,[X(B)-X(BrnB);C] + 2E[A(B) - X(Br' n B);C'] + 2X(B)P(CC). Proof. Since A is right-invariant, we have for A G £ and B G G, J l{7y6j4}A(d7) - J l{7rye^}A(d7) = J hrteAyWl) ~ J^ l{7ye^}A(rf7) ib JBr ^x(B)-x(BnBr). Dividing by X(B) and taking expectations over C yields P(UBY £A,C)~ V{UBrt £A,C)^ E[l - X(Br\B);C]. Similarly, P{UBr'Y' G A, C) - P{UBY' G A, C') ^ E[l - X{Br'\B); C'}.
Section 4. Inequality and Asymptotics 227 Since P((UB,rY) G -,C) = P{(UB,r'Y') G -,C"), we have P(UBTY £A,C)- P(UBr'Y' G A,C") = 0. Summing the last two inequalities and this equality yields P(UBY £A,C)- P(UBY' G A,C) < E[l - \{Br\B)\ C] + E[l - \(Br'\B); C'}. Certainly, P(UBY £ A,CC) - P(UBY' g ^,G'e) ^ P(Ce), and thus P(UBY £A)- P(UBY' G A) ^ E[l - A(Br|B); C] - E[l - A(Br'|B); C] + P(Ce). Taking the supremum in A and multiplying by 2 completes the proof. □ 4.2 Successful Transformation Coupling — Finite A Suppose there exists a successful transformation coupling (distributional or not). Then the transformation coupling inequality simplifies to \\P(UBYe-)-P(UBY'e-)\\ ^ 2E[1 - X(Br\B)} + 2E[1 - \(Br'\B)}. If A(G) < co, then A(GDG7) = A(G), 7eG, since, due to the right-invariance, A(G7) = A(G) = the full measure, and the intersection of two sets of full (finite) measure has full measure. Dividing by A(G) yields A(G) < oo => A(G7|G) = 1, 7 G G. (4.4) Take B = G in (4.3) to obtain that the right-hand side is zero and thus UGY = UGY', (4.5) that is, we need not take a limit to obtain a Cesaro result. 4.3 Successful Transformation Coupling — F0lner Sets If A is not finite but only cr-finite, then there need not even be a Cesaro limit result at all. In order to obtain one we must assume the existence of F0lner averaging sets, that is, a family of sets Bh G Q, 0 < h < oo, such that 0 < X(Bh) < co,
228 Chapter 7. TRANSFORMATION COUPLING and, for all 7 £ G, \{Bhl\Bh) -► 1, h -> 00; (4.6) see Theorem 2.1 for an example of such sets in the random field case. If there exists a successful transformation coupling, then (4.3), (4.6), and dominated convergence yield the Cesaro limit result ||P([/Bhye-)-P(^hre-)II^O, ft->oo. (4.7) In particular, when Y' is distributionally invariant under G, that is, 7r ^ r, 7 e g, then (4.7) can be rewritten as UBhY%Y', h-+oo. Results on rates of convergence and uniform convergence can of course be obtained if more is known about the behaviour of E[l — A(.B/j.r|£?/»)] and E[l - X(Bhr'\Bh)} as functions of h. 5 Maximality In this section we shall establish a transformation coupling analogue of the maximality result for shift-coupling, Theorem 4.1 in Chapter 5. The proof is basically the same, but we repeat it in the present general framework for the sake of completeness. 5.1 The Maximality Theorem Note that in the following theorem we do not assume the existence of F0lner averaging sets, not even the existence of an invariant measure (although we shall apply the theorem in the next section with n = A). Theorem 5.1. LetY andY' be random elements in a general space (E,£). Let (G, G) be a jointly measurable semigroup acting jointly measurably on (E,£). Let /j, be a finite or a-finite measure on (G,Q). Then there exists a distributional transformation coupling (Y,Y',r,r',C,C') of Y andY' such that f P(jY € -,Cc)Md7) ± / P(lY' e ;C,c)»{drt). (5.1) JG JG Moreover, there exists a nondistrihutional transformation coupling of Y and Y' with this property if either there exists a weak-sense-regular conditional distribution ofY' given FY' and 8®8 contains the diagonal {(y,y) ■ y £ E} [this holds when (E,£) is Polish] or (G,G) is an inverse-measurable group and there exists a weak-sense-regular conditional distribution of f' given FY' [this holds when (G,G) is Polish]. We prove this result in the next three subsections.
Section 5. Maximality 229 5.2 First Part of Proof- Construction of a Candidate Since /j, is finite or er-finite, there are disjoint sets £?i,i?2,-'" G G with union G and such that n(Bn) < oo, n ^ 1. Let i~i, i~2,. ■ ■ be i.i.d. random transformations in (G, 5) with common distribution 1 The important property of this distribution is that it has the same null sets as ix. Let F\, i~2, • • • be independent of a sequence of independent quadruples (Yk,Yl,Ck,C'k), 10<oo, which have the following properties. Let (Yi, Y{) be a coupling of V and Y' and let Ci and C[ be maximal distributional coupling events of (-TiYi, AYi'). This is possible because we can first let (Yi,Y{) be a coupling of V and Y' and then use Theorem 4.4 in Chapter 4 to obtain C\ and C{. In the same way we can recursively, for 1 < k < 00, let (Yk, YD be a coupling of random elements with distributions P(yfc_i G -|Cfc_i) and P(^'_! G -|Cfc-i) and let Ck and q be maximal distributional coupling events of (A^j A^')- Now put AT = inf {1 < fc < 00 : Cfc occurs} [inf 0 = 00] and note that [due to the independence of the quadruples (Yjt, Yfc', C^, Cj(.)] P(K>fc)=P(C,1C)...P(C^), 10<00, and that [since P(Yfc+i £ •) = p(*fc G "1^)] p(n e •) = p(n e ;Ck) + p(n+i e -)p(^), 1 ^ * < oo. This implies [since P(Yi e •) = P(Y~ £ •) and Yi = Yk on Ci = {K ^ 1}] that the following holds for k = 1, P(y e-) = P(Yir e-.#«;*)+ P(n+i e-)P(tf>*), (5-2) and that if it holds for some k, then it holds with k replaced by k + 1, since p(n+i e -)p(k > k) = p(n+i e -,cfc+1)p(K > t) + p(n+2 g -)p(C£+1)p(a" > *) = P(Yk e •, A" = * + !)+P(yfc+2 G-)P(if>* + l),
230 Chapter 7. TRANSFORMATION COUPLING where we have used the independence of (Yk+\,Ck+i) and {K > k} — C\ D • • • D Ck for the second identity. Thus by induction (5.2) holds for all 1 ^ k < oo. Drop the last term in (5.2) and send k —► oo to obtain P(Ye-)^P(YKe-,K<oo). (5.3) Similarly, with K' = inf{l ^ k < co : C'k occurs} we obtain P(l" € •) ^ P(*K' £;K'<oo). (5.4) Note that K' is a copy of K and, in particular, ~P{K < oo) = P(2f' < oo). Let .Too be some fixed element of G and let Yx, and Y^ be independent of (A, Yfc,Yk\Ck,C'k), 1 ^. k < oo, with arbitrary distributions when P(K < oo) = 1 and with the following distributions when ~P{K < oo) < 1: P(Yoo e •) = (P(Y e •) - P(YK e-,K< oo))/p(k = oo), P(K4 e •) = (P(Y' e •) - P(i^. €-,#'< oo))/P(if' = oo). (5'5) These distributions are well-defined due to (5.3) and (5.4). We shall show that the candidate (Y,Y',r,r',c,C) := (YK,Yk,,rK,rK,,{K < oo},{#' < oo}) is a distributional transformation coupling satisfying (5.1). 5.3 Mid-Part of Proof - Candidate Is Transformation Coupling If ~P(K < oo) = 1, we obtain from (5.3) and (5.4) that p(y e •) = P(y e •) and P(Y' e-) = P(Y' e-). If P(K < oo) < 1, we obtain this same result from (5.5), since K is independent of Yoo and K' of Y^. Thus (Y, Y') is a coupling of Y and Y'. For 1 ^ k < oo, we have p(rY e -,# = *) = P(r*n e -,ck)P(K > A), p(rf e-,K' = k) = P(rkYk' e -,c;)P(K' > *). Since C^ and Cjj. are distributional coupling events of (AYt, rkYk) jjind since P(ii' ^ k) = P(K' > fc), the right-hand sides are identical, and summing over 1 ^ k < oo yields p(rYe-,C) = P(rY'e-,c"). Thus (Y,Y',r,r',C,C) is a distributional transformation coupling of Y and Y'.
Section 6. Invariant cr-Algebra and Equivalences 231 5.4 Final Part of Proof- The Candidate Satisfies (5.1) The maximality of Ck and C'k means that the subprobability measures P(A1* £ -,Cck) and P(rkYk' £ -,C"£) are mutually singular, that is, there is an Ak £ E such that P(r*n e Ak) = P(An e ^,c,), (5.6) p(rkYk' e ^) = P(W G 4£, C£). (5.7) From (5.2) we obtain the equality in P(Ye-,Cc)^P(YKe-,K^k) (5.8) = P(Yk £ -)P(K > A), 0 s$ A < oo. Let T0 be a copy of rfc and be independent of (Y,Y',r,r',C,C). Then p(r0y e ^,ce) ^ P(rkYk e ^)P(# > A) [due to (5.8)] = P(An e Ak,Ck)P(K > A) [due to (5.6)] ^P(Ck)P(K^k) = P(K = k), and thus P(r0Y £ |J ^,CeWp(n ^if <oo) ->0 asn->oo. ra^fc<oo Put A = limsup^oo Aj; to obtain P(r0Y £A,Cc)=0. Since Jo is independent of (Y, Cc) and has a distribution that has the same null sets as n, we can write this as [ P(1YeA,Cc)»(d1) = 0. Jg Since liminfjfc-^oo Ak is the complement of limsupj._>00 Ak, we obtain similarly [using (5.7) rather than (5.6)] that [ P(1Y'eAc,Cc)»(d1) = 0. Jg Thus (5.1) holds, and the proof of Theorem 5.1 is complete. 6 Invariant a-Algebra and Equivalences In this section we introduce the invariant cr-algebra and extend the set of equivalences for shift-coupling (Chapter 5, Section 5) to the transformation coupling case.
232 Chapter 7. TRANSFORMATION COUPLING 6.1 The Invariant a-Algebra The invariant a-algebra is defined as follows: 1={A££:1-1A = A,1£G}. The following observation is useful [note that here we really do not need r to be measurable, nor the transformation class G to be a semigroup]. Lemma 6.1. Let Y be a random element in (E,£). If T is a random transformation, then {ry ei} = {ye4 At I. Proof. With iel and 7 £ G we have {rYzA,r = 1} = {1Y£A,r = 1} = {Y£1-lA,r = 1} = {Y£A,r = 1}. Taking the union over 7 £ G yields the desired result. □ 6.2 The Inequality The following result explains what the invariant er-algebra has to do with transformation coupling. Theorem 6.1. Let (Y,Y') be a coupling of random elements Y and Y' in a general space (E,£). Suppose (G,Q) is a measurable semigroup acting jointly measurably on (E,£). If C is a transformation coupling event, then it is an I-coupling event. If C and C' are distributional transformation coupling events, then they are distributional I-coupling events. In both cases ||P(y e -)li - P(l" e -)lzll *S 2P(CC). (6.1) Proof. For A £ I we have, due to Lemma 6.1, {Y eA}nC = {rYeA}nc, {Y' ei}nC" = {r'Y1 e A] nC". In the nondistributional case the right-hand sides are identical, and in the distributional case they have the same probability. And (6.1) is the I- coupling event inequality (Theorem 7.3 in Chapter 4). □ 6.3 Maximally Successful Transformation Coupling We shall now give conditions under which the coupling in Theorem 5.1 yields equality in (6.1). Thus there is, under these conditions, a maximally successful transformation coupling (attaining the supremum of the success probabilities over all transformation couplings), and maximal success probability = ||P(y G -)|i A p(^' e Olill-
Section 6. Invariant tr-Algebra and Equivalences 233 Theorem 6.2. Let Y and Y' be random elements in a general space (E, £). Let (G, Q) be a jointly measurable semigroup acting jointly measurably on (E,£). Suppose there is a finite or a-finite right-invariant measure A on (G,G) and suppose one of the following conditions holds: (i) G is a group, (ii) X(G) < oo, (Hi) G is normal [that is, G7 = -yG for all 7 € G], (iv) Gr]ip C Grj for all rj and ipgG. Then the distributional transformation coupling (Y,Y',r,r',C,C) of Y and Y' in Theorem 5.1 with fi = A is such that C and C are maximal distributional I-coupling events, that is, ||P(yG-)|i-P(ir'G-)lill = 2P(Cc). (6.2) Comment. Note that (Hi) holds when G is a group and when G is Abelian [that is, 777 = 777 for all 7 and 77 € G\. Also note that (Hi) implies (iv) since [due to the semigroup property] <pG C G and thus r^pG C r\G, which [together with (Hi)] yields Gr]ip C Gr\. Proof. Let (Y,Y',r,r',C,C) be as in Theorem 5.1 with \i = A. Thus there is a set A € S such that / P(7F G A,Cc)X(dj) = 0 = / P(7F' G Ae,C'c)\(d~t), Jg Jg which we can rewrite as v[JGlhyeA}\(dj);Cc] =0 = -E[JGl{yyleAc}\(d~t);C'c]. (6.3) Case (i): suppose G is a group. Put B:={y£E: J l{yyeA}X(d7) > o}. Then Bc C{y £E : JG l{yyeAc}X(d-y) > 0}, and (6.3) yields P(Y &B,CC) = Q and P(f e BC,C'C) = 0. (6.4) Further, for y> € G, y G <p_1S » W£B O / lhvyeA}X(dy) > 0 J G <=> / l{7yeJ4}A(d7) > 0 [since A is right-invariant] JG<p (6.5)
234 Chapter 7. TRANSFORMATION COUPLING Since G is a group, we have G<p = G, which yields, together with (6.5), the first equivalence in yev~lB «. / l^y€A}X(dj) > 0 o y£B. (6.6) Jg Thus B € I, which, together with (6.4), implies mutual singularity on I, that is, (6.2) holds. Case (ii): suppose X(G) < oo. First note that (6.5) and (6.4) still hold. Then note that [due to the right-invariance] X(G<p) = \{G), and thus [due to X(G) < oo] we have X(G\Gip) = 0 [since the difference set of two sets of full finite measure has measure zero], which together with (6.5) yields the first equivalence in (6.6). Thus B £ 1, which, together with (6.4), implies mutual singularity of P(Y € -,Cc)\i and P(Y' € -,C"e)|i, that is, (6.2) holds. Case (iii): suppose G is normal. We observed in the comment immediately after the theorem that (iii) implies (iv), and thus Case (iii) is a special case of Case (iv). Case (iv): suppose Grjtp ^ Grj for all rj and <p £ G. Redefine B:={yGE: jf 1^ 1{^A}Hdy)=0}X(dr]) = o}, and observe that y £ B implies that there is at least one rj such that Jg?) ^{tyeA}^^) > 0 and thus [since Gr\ C G] BC {y&E:J l{^yeA}X(d7) > o}. (6.7) Note also that y £ Bc implies that there is at least one rj such that Igt, ^{-ryeA}^(d7) = 0 which in turn implies J*G l^yeAcyX(d'j) > 0 and thus [since Grj C G] Bcc{y&E: J lhyeAc}X(dy)>0}. (6.8) The two inclusions (6.7) and (6.8) together with (6.3) yield that (6.4) also holds with this definition of B. Hence in order to complete the proof it only remains to show that B £ 1. For that purpose take ip € G. Since [by assumption] Gr]ip C Gr], we have JG lUa„ i{„,c=AjA(d7)=o>A(*7) > JG lUG„ i{lv,A}x(dy)=o}Hdr,). (6.9) Conversely, since A is right-invariant, we have
Section 6. Invariant tr-Algebra and Equivalences 235 which together with G<p Q G and (6.9) implies JG l{So,v U-,yzA}Hd-r)=o}x(dv) = Jg l{fan i{„eA}A(d7)=o}A(dij). This yields the final equivalence in y G <p~lB <=> <py G B ** JG hfo, i{,.»eA}A(d7)=o}A(d»7) = 0 ** / ^-W i{„€A,A(d7)=o}A(^) = 0 [A is right-invariant] <=> y & B. Hence B G I, which, together with (6.4), implies mutual singularity of P(y G -,Cc)\i and P(y' G -,C'c)\i, that is, (6.2) holds. D 6.4 The Cesaro Total Variation Result In the following theorem we impose a maximal set of conditions (all the conditions encountered up to now). Theorem 6.3. Let Y and Y' be random elements in a general space (E, £). Let (G, Q) be a jointly measurable semigroup acting jointly measurably on (E,£). Suppose there exists a finite or a-finite right-invariant measure A on (G, Q) and let, for each B G Q such that 0 < \{B) < oo, Ub have distribution A(-|J3) and be independent ofY and Y'. If X(G) < oo, then \\P(UGY G •) - P(UGY' G -)|| = \\P(Y G -)|z - P(l" € Olzll- // A(G) = oo and there exist F0lner averaging sets Bh G Q, 0 < h < oo, then as h —^ oo, \\P(UBhY G •) - P(UBhY' G -)ll -+ \\P(Y G -)|z - P(Y' G Olzll, provided that one of the conditions G is a group, G is normal [that is, G-y — ■yG for all 7 G G], Gr)ip C Gr] for all 77 and <p G G, holds.
236 Chapter 7. TRANSFORMATION COUPLING Proof. If X(G) < oo, apply the transformation coupling inequality (Theorem 4.1) with B = G to the transformation coupling in Theorem 6.2 to obtain [see (4.4)] \\P(UGY G •) - P(UGY' G OK \\P(X G Oil - P(3" G Olill- By Lemma 6.1 we have ||P(Y G Oil - P(X' G Olill = \\P(UGY G Oil - P(UGY' G OHI, and since the right-hand side is at most ||P([/gY G 0 - P(UgY' G Oil, we obtain the reversed inequality ||P(Y g oii - P(r g oiill ^ \\p(ugy go- p(ugy' g oil- If A(G) = oo, apply the transformation coupling inequality with B = Bh to the transformation coupling in Theorem 6.2 and send h —> oo to obtain limsup \\P(UBkY GO- P(UBhY' G Oil h—)-oo ^||P(yG0li-P(Y'G0lill- By Lemma 6.1 we have, for 0 ^ h < oo, ||P(Y G Oil - P(Y' G Olill = \\P(UBhY G Oil - P(UBhY' G OHI, and since the right-hand side is at most ||P([/BhY GO- P(UBhY' G Oil, we have ||P(yG Oli-P(y'e 01x11 ^ liminf \\P(UBhY G 0 - P(UBhY' G Oil- These two inequalities yield the desired result. □ Remark 6.1. By a similar argument we can obtain the inequality (6.1) directly: ||P(Y g ok - P(r g OHI < HP^r eO- P(uBhY' g oil ^ 2E[1 - X(Bhr\Bh); C] + 2E[X{Bhr'\Bh); C] + 2P(Ce) -> 2P(CC), h -> oo, without the concept of coupling with respect to a cr-algebra, but under unnecessarily strong conditions.
Section 6. Invariant a-Algebra and Equivalences 237 6.5 Equivalences We can now tie together transformation coupling, Cesaro total variation convergence, and the invariant cr-algebra as follows. Theorem 6.4. Let Y and Y' be random elements in a general space (E, £). Let (G, Q) be a jointly measurable semigroup acting jointly measurably on (E,£). Suppose there exists a finite or a-finite right-invariant measure A on (G, Q) and let, for each B € Q such that 0 < \{B) < oo, Ub have distribution A(-|.B) and be independent of Y and Y'. Suppose further that one of the following conditions holds: G is a group, \(G) < oo, G is normal [that is, G~f = jG for all 7 € G], Gr]ip C Gr\ for all 77 and ip €. G. Then the following two statements are equivalent: (a) There is a successful distributional transformation coupling ofY, Y'; (c) P(ye-)|z = P(i"e-)lz- If X(G) < 00, then the equivalent statements (a) and (c) are equivalent to (b - finite case) UGY = UGY'. If A(G) = 00 and there exist F0lner averaging sets Bh € G, 0 < h < 00, then the equivalent statements (a) and (c) are equivalent to (b - infinite case) \\P{UBhY € ■) - T>{UBhY' € ■)\\ ->■ 0 as h ->■ 00. Moreover, these equivalent statements are equivalent to the existence of a successful nondistributional transformation coupling if either there exists a weak-sense-regular conditional distribution of Y' given TY' and £ <g> £ contains the diagonal {(y,y) ■ y G E} [this holds when (E,£) is Polish] or (G, Q) is an inverse-measurable group and there exists a weak-sense-regular conditional distribution of T' given r'Y1 [this holds when {G,Q) is Polish]. Proof. By Theorem 6.1, (a) implies (c). By Theorem 6.2, (c) implies (a). By the transformation coupling inequality [see (4.6) when \(G) < 00 and (4.7) when A(G) = 00], (a) implies (6). By Theorem 6.3, (b) implies (c). The nondistributional coupling claim is due to the final statement of Theorem 6.2. □
238 Chapter 7. TRANSFORMATION COUPLING The (almost) omnipresent condition that there exists a right-invariant measure on (G,Q) is annoying in its abstractness. It holds, however, when G is a locally compact second countable topological group (and Q its Borel subsets) or a subsemigroup of such a group. In the next section we spell out the streamlined theory that results from this fact in the group case. 7 Topological Transformation Groups This section collects the transformation coupling results of Sections 3-6 in the special case when G is a locally compact second countable topological group. In this case all the conditions needed in Sections 3-6 hold, except the existence of F0lner averaging sets. 7.1 Preliminaries Let G be a class of measurable mappings (transformations) from a measurable space {E,£) to itself. Assume that G is a topological group, that is, in addition to the group properties 7 and rj € G => 777 £ G, 7 € G => 7 has an inverse 7-1 £ G, let G have a topology with respect to which the mapping from G x G (with the product topology) to G taking (7, rf) to 777 and the mapping from G to G taking 7 to 7-1 are both continuous. Let Q be the Borel subsets of G and note that (G, Q) is both jointly measurable and inverse-measurable. Assume further that G is locally compact and second countable. Then, according to the following two theorems, there exists a finite or cr-finite right-invariant measure A on (G, Q) and (G, Q) is Polish. Fact 7.1. A locally compact topological group possesses right- and left- invariant measures, the right and left Haar measures. Further, a locally compact second countable topological group is either compact, in which case the Haar measures are finite, or a-compact, in which case the Haar measures are cr-finite. For a proof, see Halmos (1950), pages 254 and 256. Fact 7.2. A locally compact first countable topological group has an invariant metric inducing the topology. Further, a locally compact second countable metric space is separable and topologically complete (that is, has a topologically equivalent metric that is complete).
Section 7. Topological Transformation Groups 239 For a proof, see Montgomery and Zippin (1955), page 34; Bourbaki (1948), page 25; and Bourbaki (1951), page 27. (In fact, any locally compact second countable topological space has a metric inducing the topology and is either compact or cr-compact; see Ash (1972).) Finally, assume that (G, Q) acts jointly measurably on (E, £), let Y and Y' be random elements in (£,£), and for B € Q such that 0 < \{B) < oo, let Ub have distribution \{-\B) and be independent of Y and Y'. 7.2 Transforming Y into a Copy of Y' and Vice Versa Recall that (Y,Y',r,r',C) is a transformation coupling oiY and Y' if fY = f'Y' onC. Since G is a group, this can be written as <PY = Y" on C, (7.1) where # = f'~1f [here T'-1 denotes the group inverse of f'\. Also, we can write this as Y = S'Y' on C, where <£' = #-1. Thus transformation coupling means that on C each random element can be transformed into the other and vice versa. Recall that (Y,Y',r,r",C,C') is a distributional transformation coupling of Y and Y' if P(fY€-,C') = P(f'y-'e ■,(?'). Since (G,G) is Polish, Theorem 3.1 gives us the existence of an unhatted version of (Y,Y',t,r',C:C'): and Theorem 3.2 gives us the existence of a nondistributional version. This means that we can first turn a distributional transformation coupling into a nondistributional one, then write it in the form (7.1), and finally unhat (7.1). That is, any transformation coupling (Y,Y',f,f',C,C'), distributional or not, has an unhatted version (Y,Y',$,C,C) where # is a random transformation and C and C" are events such that {Y,1C) = {Y,16) and (Y1, lc-) = (Y', 1&) and p($y e-,c) = P(re-,c").
240 Chapter 7. TRANSFORMATION COUPLING In particular, any successful transformation coupling (Y,Y',f,f'), distributional or not, has an unhatted version (Y,Y',$) such that #y = Y'. (7.2) Thus a successful transformation coupling means that the original Y can be transformed into a copy of the original Y' and vice versa. 7.3 Transformation Coupling Inequality A transformation coupling (distributional or not) can always be transformed to the form (7.2), that is, one of the coupling transformations can be taken to be the identity mapping. Thus one of the terms in the transformation coupling inequality (Theorem 4.1) disappears and we obtain the following simplification: for B G Q such that 0 < X(B) < oo, it holds that \\P(UBY G •) - P(UBY' G -)ll TRANSFORMATION ^ 2E[1 - \{B$\B); C] + 2P(CC). coupling inequality If there exists a successful transformation coupling (distributional or not), then this transformation coupling inequality simplifies further to \\P(UBY G •) - P(UBY' G -)|| «S 2E[1 - \{B$\B)]. If G is compact (that is, \{G) < oo), then the inequality yields [take B = G and use G# = G to obtain E[l - \(G$\Gj\ = 0] UGY = UGY'. If G is cr-compact (that is, X(G) = oo) and there exist F0lner averaging sets Bh G Q, 0 < h < oo, then \\P{UBhY£-)-P{UBhY'£-)\\-+0, A->oo. Groups possessing F0lner averaging sets are called amenable. 7.4 Maximality — Invariant a-Algebra — Equivalences According to Theorem 5.1, for any finite or cr-finite measure (i on (G,Q), in particular for \i = A, there exists a nondistributional transformation coupling (Y, Y', f, /", C) of Y and Y' such that / P(7F G -,Cc)M(d7) -L [ P(7Y' G -,Cc)Kdj). (7.3) Jg Jg According to Theorem 6.1, a (distributional) transformation coupling event C is a (distributional) J-coupling event, and ||P(ye0li-P(^'G-)lill^2P(cc).
Section 8. Self-Similarity - Exchangeability - Rotation 241 According to Theorem 6.2 and Theorem 3.2, there exists a nondistribu- tional transformation coupling with event C such that ||P(ye-)|i-P(i"e-)lill = 2P(Cc). According to Theorem 6.3, if G is compact (that is, X(G) < oo), then \\P(UGY G •) - V(UgY' G -)H = \\P(Y G Oil - PG" G Olzll. and if G is cr-compact (that is, X(G) — oo) and there exist F0lner averaging sets Bh G Q, 0 < h < oo, then as h —> oo, \\P(UBhY GO- P(UBhY' G Oil -+ l|P(^ G Ok - PO" G OMI- According to Theorem 6.4 and Section 7.2, the following statements are equivalent: (a) There is a successful distributional transformation coupling of Y, Y'. (a1) There exists a random transformation # such that $Y = Y'. (c) P(re0lz = PO"e0lz- If G is compact (that is, X(G) < oo), then the equivalent statements (a), (a1), and (c) are equivalent to (6 - finite case) UGY = C/Gy'. If G is cr-compact (that is, X(G) = oo) and there exist F0lner averaging sets Bh G Q, 0 < h < oo, then the equivalent statements (a), (a'), and (c) are equivalent to (6 - infinite case) \\P{UBhY G 0 - P([/b„1" G Oil -> 0 as /i ->■ oo. Finally, if the distribution of 1" is invariant under G [jY' = Y' for 7 G £], then (6) can be rewritten as C/Gy = y' when G is compact |A(G) < 00], UbhY —> Y' as /i —> 00 when G is cr-compact \X{G) = 00]. It follows from the equivalence of (c) and (b) that two random elements with distributions that are invariant under G agree in distribution on I if and only if they have the same distribution. 8 Self-Similarity - Exchangeability - Rotation The streamlined group theory of the previous section has many potential applications that are largely unexplored. In this section we shall indicate applications in a few fields, but these are only stumbling first steps.
242 Chapter 7. TRANSFORMATION COUPLING 8.1 Application to Brownian Motion — Self-Similarity Let W = (Wt)te[o,oo) be a standard Brownian motion (or standard Wiener process). This means that W is a one-sided continuous-time real-valued stochastic process with continuous paths and independent increments [that is, the increments Wtj — Wt2,..., Wtn_1 —Wtn are independent for all n > 0 and 0 = t\ < ■ ■ ■ < tn] and Wt is normal with E[W4] = 0 and Var[Wt] = t, t£ [0, oo). Then W is self-similar in the sense that lrW = W, 0 < r < oo, where 7r is the rescaling defined for a path z = {zt)te[o,oo) by 7rZ= {r1/2zt/r)te[0tOO). Clearly, 7,.7,, = ^rs and 7"* = 7i/r, 0 < r < 00, that is, G = {7,. : 0 < r < 00} is a group that can be identified with (0, 00) under multiplication, which in turn can be identified with K under addition. Thus we are in the framework of the previous section. Let Z be another one-sided continuous-time real-valued process with continuous paths. According to Section 7.4 [the equivalence of (a1) and (c)], Z and W have the same distribution on measurable sets that are invariant under the rescalings 7,-, 0 < r < 00, if and only if there exists a strictly positive finite random variable R such that jRZ = W. Further, according to Section 7.4 [the final claim] and since W is self-similar, the above equivalent claims hold if and only if ■yRhZ%W, h-+oo, for Rh = eUBh, where Bh € B, 0 < h < 00, is any family of F0lner averaging sets with respect to the Lebesgue measure A (for instance, Bh = hB where B € B is such that 0 < X(B) < 00; see Theorem 2.1) and Ush is uniform on Bh and independent of Z. 8.2 Application in Exchangeability Let Z and Z' be one-sided discrete-time stochastic processes on a general state space. For a path z — (2fc)o° and with p a finite permutation of {0,1,... } define the exchange tt by ■KZ = (Zp(fc) )S^o-
Section 8. Self-Similarity - Exchangeability - Rotation 243 The exchangeable a-algebra consists of measurable sets invariant under such finite exchanges. A stochastic process is called exchangeable if its distribution is invariant under finite exchanges [that is, ttZ' = Z1 for finite exchanges tt]. According to Section 7.4 [the equivalence of (a1) and (c)], Z and Z' have the same distribution on the exchangeable cr-algebra if and only if there exists a finite random exchange II such that IIZ = Z' (permutation coupling). Further, if Z' is exchangeable, then according to Section 7.4 [the final claim], the above equivalent claims hold if and only if UnZ 4Z', n —> oo, where Un is the random exchange associated with a uniformly distributed random permutation of {0,..., n} that is independent of Z. It should be possible to extend this result to one-sided continuous-time real-valued stochastic processes with right-continuous paths having left- hand limits: replace finite permutations by splitting a finite interval into finitely many subintervals (open to the right and closed to the left) and permuting them. A similar comment applies to random fields. 8.3 Rotational Invariance Let Z = (Zs)sERd and Z' = {Z's)seRd be random fields with a general state space (E, £) and path space (H, %) and site set Rd, where d ^ 1. Let § be the rotation group, the group of orthogonal real dxd matrices with determinant 1 (if we allow determinant — 1, then we get reflections also). This is a compact topological group. Here let rt denote a rotation of t £ M.d by r € S. Define the rotation maps pr, r e §, by prz = {zrt)te^d, z € H. Assume that Z and Z' are rotation measurable, that is, prH= H,r£§, and the mapping taking (z, r) in H x § to prz in H is V. <g> £?(§)/% measurable. The rotation maps G = {pr : r £ §} form a group, which can be identified with §. According to Section 7.4 [the equivalence of (a') and (c)], Z and Z' have the same distribution on measurable sets that are invariant under the rotation maps pr, r € S, if and only if there exists a random rotation R in (§,#(§)) such that PrZ = Z' (rotation coupling).
244 Chapter 7. TRANSFORMATION COUPLING Further, if the distribution of Z' is invariant under rotations \prZ' = Z' for r G §], then according to Section 7.4 [the final claim], the above equivalent claims hold if and only if PuZ = Z' where U is uniformly distributed on (§,£(§)). More generally, the theory of Section 7 applies to finite combinations of rotations and shifts, and to Lorentz transformations and Poincare transformations (relativity). 9 Exact Transformation Coupling This chapter has up to now been concerned with the generalization of shift- coupling. In this final section we shall comment briefly on the possible generalization of exact coupling without going into much detail. 9.1 Exact Transformation Coupling In the same way as exact coupling is the special case of shift-coupling when the times are identical [T = T'\ we can define exact transformation coupling to be a transformation coupling with identical transformations. In this case the framework can be made more general, for instance, we need not necessarily assume that the transformations form a semigroup nor even that they take values in the same space as the random elements. Let Y and Y' be random elements in a general space (E,E). Let G be a class of measurable mappings (transformations) from (E, S) to some measurable space (E',81) and let Q be a cr-algebra of subsets of G. Call (Y, Y1, r, C) an exact transformation coupling of Y and Y' if (Y, Y') is a coupling of Y and Y', r is a random transformation in (G, Q), and C is an event such that rY = TY' on C. (9.1) Call (Y,Y',r,r',C,C) a distributional exact transformation coupling of Y and Y' if (Y,Y') is a coupling of Y and Y', T and J1' are two random transformations, and C and C" are two events such that p((ry,r) e -,c) = p((r'y',r') e -,c'). (9.2) Note that if G is a group, then rY = rY' is equivalent to Y = Y', and thus (9.1) is only a complicated way of writing Y — Y' on C, that is, C is just a coupling event of Y and Y'. Similarly, when G is a group acting jointly measurably on {E,£) then (9.2) means only that C and C" are distributional coupling events of (Y, T) and (Y',r'), nothing more, nor
Section 9. Exact Transformation Coupling 245 less. Thus in order to obtain a nontrivial theory we must now stay away from groups. Rather than attempting to build a general theory around this concept we content ourselves here with considering two ways of applying it to random fields in d dimensions. 9.2 Random Fields - Exact Coupling - Tail a-Algebra Let z = {Zs)se[o,ooy and Z' = {Z's)se[0ooy be random fields with a general state space {E,£) and a general path space {H,7i) and site set [0, oo)d where d ^ 2. Define the shift maps 9t, *€[0,oo)d, by 9tz = {zt+s)se[a,°c)di z £ H. Call (Z, Z', T) an exact coupling of Z and Z' if (Z, Z') is a coupling of Z and Z' and T is a [0,oo]d valued random site (the coupling site) such that 6TZ = 6TZ' if T G [0, oo)d. (9.3) Let A be a state external to E (the censoring or cemetery state or womb). For t € [0,oo]d \ [0,oo)d define 6t by 6tz = {A)s€[0tOOy, z£ H. Then (9.3) can be rewritten as 6fZ = Oj-Z . For a set of sites A C [0, oo)d and z € H, define (3az {z born or observed inside A) by W^' = {A iis? A. Then (9.3) can also be rewritten as (3t+[0,oc)<<Z = 0T+[O,co)dZ ■ Call (Z,Z',T,T') a distributional exact coupling of Z and Z' if (Z, Z') is a coupling of Z and Z' and T and T' are [0, oo]d valued random sites such that f3T+[o,oo)dZ = 0T'+[o,ooyZ'. (9.4) When Z and Z' are shift-measurable, then (9.4) can be rewritten as {0TZ,T) = {6T'Z',T').
246 Chapter 7. TRANSFORMATION COUPLING The theory of exact coupling in Chapter 4 is easily redone in this d dimensional setting. Thus we obtain (Theorem 5.1 in Chapter 4) a coupling site inequality \\P(9tZ G-)- PWtZ' G Oil «S 2(1 - P(T «£ t)), t G [0, oo)d, and that (Theorem 6.1 in Chapter 4) there always exists a distributional exact coupling maximal at integer diagonal sites, that is, such that for integers n ^ 0, ||P(0(„,...,„)Z GO- P(0(n,...,„)Z' G OH = 2(1 - P(T ^ (n,... ,n))). Extend the tail cr-algebra to d-dimensions as follows: «e[o,oo)<' and let t -> oo denote that all the coordinates of t go to infinity. Then the following statements are equivalent: (a) There exists a successful distributional exact coupling of Z and Z'. (b) \\P(6tZ G 0 - P{0tZ' G Oil -► 0 as t -> oo. (c) P(Z€0lr = P(^'G0lr- It should also be possible to work out in a similar way a theory of epsilon- coupling for random fields with site set [0, oo)d. (A theory of shift-coupling for random fields with site set [0, oo)d is already implicit in Sections 3 through 6.) 9.3 Random Fields — Remote Coupling — Remote a-Algebra Another way of extending the exact coupling of one-sided stochastic processes to random fields is to drop the semigroup of shift maps and rather assume that the fields coincide outside a bounded ball. In some sense this is a more natural extension. For instance, it may fit better in the case of Markov random fields [if the Markov property is taken to be conditional independence of a field outside and inside certain sets (like bounded convex sets) given its values on the boundary]. Let Z and Z' be random fields with a general state space (E, £), a general path space (H,'H), and site set Rd where d ^ 2: Z = (Zs)seRd and z' = iz's)seRd- For a set of sites A C [0, oo)d and z G H, define kaz (z killed or censored inside A) by (a Use A, («AZ)' = \z. its * A.
Section 9. Exact Transformation Coupling 247 Let B be the unit ball around the origin. Call {Z, Z',R) a remote coupling of Z and Z' if (Z, Z') is a coupling of Z and Z' and R is a [0, oo] valued random variable (the coupling radius) such that krbZ = krbZ'. (9.5) Call (Z,Z',R,R') a distributional remote coupling of Z and Z' if (Z,Z') is a coupling of Z and Z' and i? and R' are [0, oo] valued random variables such that krbZ = kriBZ'. (9.6) The following theory is easily obtained from the theory of exact coupling in Chapter 4 by noting that (/trs2')rg[o oo) is simply a one-sided stochastic process and that R at (9.5) is then a coupling time. Thus (Theorem 5.1 in Chapter 4) we obtain a coupling radius inequality \\P(KrBZ G ■) - PKbZ' e -)|| ^ 2P(R >r), r€ [0, oo), and that (Theorem 6.1 in Chapter 4) there always exists a distributional remote coupling maximal at integer radii, that is, such that \\P(KrBZ e ■) ~ V{ktbZ' G -)ll = 2P(i? >r), r = 0,1, 2,.... For a set of sites A, let TZa be the sub-cr-algebra of % generated by the projection mappings taking z to zt: t $ A. Define the remote a-algebra by K-.= f| nrB= f) nA. 0^r<oo A bounded Then the following statements are equivalent: (a) There exists a successful distributional remote coupling of Z and Z'. (6) \\P{KrBZ e •) - P{KrBZ' G -)ll -> 0 as r h oo. (c) p(ZG-)k = P(Z'e-)k- In the above discussion the site set Kd can clearly be replaced by [0, oo)d or some other subset of Rd. * * *
248 Chapter 7. TRANSFORMATION COUPLING This chapter ends our general treatment of coupling. In the second half of the book (the remaining three chapters) the focus will be on other topics, first stationarity and then regeneration, with coupling entering only as a tool. We therefore conclude at this point with some general comments on coupling. There are many aspects of coupling that have not been treated here, like the domination coupling in partially ordered Polish spaces (see Section 3 in Chapter 1) and the many ingenious coupling tricks that have been devised in particular models. Also, as the last two subsections have demonstrated, there is much yet to be done along the above lines, both in applying the theory to specific problems and in developing new theory. Finally, the many coupling equivalences encountered in Chapters 1-7 suggest that the following might be a useful guideline: Working hypothesis. Any meaningful distributional relation should have a coupling counterpart.
Chapter 8 STATIONARITY, THE PALM DUALITIES 1 Introduction In this relatively self-contained chapter we shift the focus from coupling to stationarity. [There is, however, an obvious link to coupling because in the coupling inequalities we would like one of the processes to be a stationary version of the other. And it turns out that there are coupling applications in the end, a shift-coupling application in this chapter, exact and epsilon- coupling applications in Chapter 10.] The aspect of stationarity under consideration here is the relation between stationarity and cycle-stationarity. A stochastic process is stationary if it is distributionally invariant under (nonrandom) time shifts and cycle-stationary if it consists of cycles forming a stationary sequence (are distributionally invariant under shifts from one cycle to the next). We have already encountered examples of this relationship in Chapter 2. A recurrent Markov chain starting from a fixed state is split into cycles by the times of successive visits to this state. These cycles are i.i.d. and thus form a stationary sequence. In the positive recurrent case we showed (Sections 2.5 and 2.6 in Chapter 2) that in addition to this obvious cycle- stationary version the Markov chain has also a stationary version. A similar result was established for renewal processes (Section 9 of Chapter 2). A zero-delayed renewal process consists of i.i.d. intervals and thus is cycle- stationary. When the interval lengths have finite mean, we showed that in addition to this trivial cycle-stationary version the renewal process also has a stationary version. 249
250 Chapter 8. STATIONARITY, THE PALM DUALITIES In this chapter we consider two-sided stochastic processes split into cycles by a sequence of random times (called points) and use the simple approach of Section 9 in Chapter 2 to develop from scratch a general theory on the relation between stationarity and cycle-stationarity. The same ideas will be applied to processes in d-dimensional time (random fields) in the next chapter. The intuitive motivation for this approach is explained in the renewal case in Section 9.1 of Chapter 2. We establish two dualities between stationarity and cycle-stationarity. In the first duality the stationary process is obtained from the cycle-stationary one by placing the origin uniformly at random in a cycle after 'length- biasing' the cycle-length. Conversely, the cycle-stationary process is obtained from the stationary one by shifting the origin to the right endpoint of the cycle straddling the origin after 'length-debiasing' the cycle-length. This duality has the following -point-at-zero interpretation: The cycle-stationary dual behaves like the stationary process (1.1) conditioned on having a point at the origin. The second duality is produced in the same way as the first with the modification that the length-biasing (length-debiasing) is done under conditioning on the invariant cr-algebra. This duality has the following randomized-origin interpretation: The cycle-stationary dual behaves like the stationary process (1.2) with origin shifted to a uniformly chosen point; and conversely: The stationary dual behaves like the cycle-stationary process (1.2°) with origin shifted to a time chosen uniformly in R. This is a version of so-called Palm theory of stationary point-processes, named after the Swedish engineer Conny Palm, who pioneered this field in the early forties. Palm theory is used, for instance, in queueing theory to derive characteristics of a queue observed at particular points (like arrival or departure instants) from the stationary characteristics, and vice versa. In Section 2 we establish notation and present the trivial measure-free part of the dualities (shifting to and from a point), and in Section 3 we prove the key result for the change-of-measure part (length-biasing and length- debiasing). In Section 4 we present the point-at-zero duality and then motivate the point-at-zero interpretation by conditioning and limit results in Section 5, while Section 6 contains simulation applications. After introducing the invariant cr-algebra in Section 7, we present the randomized-origin duality in Section 8 and then motivate the randomized-origin interpretation by shift-coupling and Cesaro limit results in Section 9. Section 10 concludes with comments on the two Palm dualities.
Section 2. Preliminaries - Measure-Free Part of the Dualities 251 2 Preliminaries - Measure-Free Part of the Dualities In this section we shall establish the measure-free framework of the chapter. Although we use the words stochastic process and random times, no probability measure is present in this section. 2.1 Process and Points Let (O,^7) be a measurable space supporting Z = (Zs)seR and 5 = (5,)!°00 where Z is a two-sided continuous-time stochastic process with a general state space (E, S) and path space (H, %) and 5 is a two-sided sequence of random times satisfying -oo <-■••< 5_2 < S-i < So < Si < > oo and 5_i < 0 < S0. Refer to the Sn as points. We shall call nonrandom elements of IR times (and not points) to distinguish them from these points. Regard 5 as a measurable mapping from (fi, J7) to the sequence space {L,C) where L = {{sk^oo e Kz : -oo < < s_i < 0 ^ so < si < > oo} and C are the Borel subsets of L, that is, C = L n Bz. Thus the pair (Z,S) is a measurable mapping from (fi, J7) to (HxL,7i®£). Let ~H®C+ denote the class of all measurable functions from [H y.L,~H®C) to ([0,oo), B[0,oo)). 2.2 The Two-Sided Joint Shift - Shift-Measurability For (61, define the (joint) shift-map 9t from H x L to H x L by Ot((z,),€R, (Sfc)-oo) = ((zt+s)seR, (Snt.+k ~ 0-co), (2-1) where nt_ is determined by (s^)^ as follows: nt- = n if and only if t e{sn^i,sn}. (2.2) Note that 6t is a time shift and shifts the points (sjb)??oo regarded as a sequence of times: 6t shifts (sk)?oo by subtracting t from the times s^ and
252 Chapter 8. STATIONARITY, THE PALM DUALITIES only shifts the index k of (sjt)??oo to observe the convention that zero (the time origin) lies between the points indexed by —1 and 0 [in accordance with this we call k index and not time]. In order to be able to shift at will, assume that Z is shift-measurable, that is, let the path set H be invariant under time shifts and the mapping taking (z,t) G H x E to zt G E be H^B/E measurable (which is equivalent to the mapping taking (z,t) G H x R to (zt+s)seR £ H being % <8> B/H measurable; see Section 2 of Chapter 4). Shift-measurability is all we need assume about Z in this chapter. It covers, for instance, processes with a Polish state space (in fact, separable metric suffices) and right-continuous paths. When Z is shift-measurable, then the mapping taking (((2,),eH,(a*)~00),*) eifxlxl toflf(W,6R)(st)-Jeifxi is % <8> C <8> B/U <8> C measurable. 2.3 Cycles and Cycle-Lengths — Relative Position Think of the points S as splitting Z into cycles Cn '■= {Zs„-1 + s)s€[0,Xn), n G Z, where Xn is the nth cycle length, Thus Xq is the length of the cycle Cq straddling the origin: -^o = So — S-\. For t G E, put Nt = n if and only if t G [5n_i, S„). Note that for s < t, Nt — Ns is the number of points in (s,i\, and that t ^ 0 => Nt = number of points in [0, £]. Denote the relative position of t in [5jv(-i, 5jvt) by Ut = (t — SN,-i)/XNt. Note that the cycle C„ is a one-sided stochastic process vanishing at the random time Xn. One way of making sense of Cn as a random element is to place it in the cemetery state A from time Xn onward (see Section 2.9 in Chapter 4), that is, to identify it with a one-sided stochastic process killed at time Xn, Cn := Kx„(Zs„_i+s)se[o,oo); n G Z. The pair (Z, 5) is determined measurably by (S0, (Cn)^°oo) and vice versa.
Section 2. Preliminaries - Measure-Free Part of the Dualities 253 2.4 The Measure-Free Duality Between (Z,S) and ((Z°,S°),U) Call Z the process associated with S, and S the points associated with Z. Observe that we do not postulate any functional link between Z and S. In applications, however, S is often even determined by Z. For instance, in the Markov chain example, S is formed by the times of the successive visits of Z to a fixed state. We shall write 5° to indicate a sequence of times with a point at zero, that is, S0° = 0. In this case we also write Z° for the associated process although the ° does not indicate anything about the process except its association with S°. Let U be a (0,1] valued random variable. Throughout this chapter we assume that (Z,S) and ((Z°,S°),U) are linked functionally as follows. When {Z,S) is given, define (Z°,S°):=6So(Z,S) [thus5g=0], U := Uq- = —S-i/Xq [Uis the relative position of 0 in (5_i,5o]]. Conversely, when ((Z°,S°),U) is given, define (Z,S) : = 6_{l_u)xs(Z°,S°) [thus X0 = XS]. Note that {Z,S) and (Z°,S°) have the same cycles, Cn=C°n, which we can write 6Sn {Z, S) = 6»s° (Z°,S°), while Sn = (1 - U)X$ + 5°; see Figure 2.1. Realization of (Z, S) (to make the illustration easier we let Z be real-valued with continuous paths and S be the times of visits to 0) (Z, S) (Z°, S°) The gray axis is at the origin of (Z°, S°). X, —►|*X3 »-|<- FIGURE 2.1. The functional duality between (Z,S) and {(Z°,S°),U).
254 Chapter 8. STATIONARITY, THE PALM DUALITIES 3 Key Stationarity Theorem The last section was measure-free. We now introduce a probability measure P on (Cl,!F), that is, assume that (Z,S) is supported by the probability space (fi,^7, P). Call (Z,S) stationary (under P) if it is distributionally invariant under time shifts: under P, 6t(Z,S) = (Z,S), ieR Let P° be another probability measure on (Q,J-) and regard (Z°,S°) as supported by (fi,.F,P0). Call (Z°,S°) cycle-stationary (under P°) if the sequence of cycles is stationary: under P°, (...,Cn_i,C„,Cn+i, ...) = (...,C_i,Co,Ci, • • •), n € Z. Since 6gn (Z, S) is determined measurably by (..., Cn_i, C„, Cn+i,...) in the same way for all n, and vice versa, it follows that (Z°,S°) is cycle- stationary if and only if 6Sn(Z,S) = (Z°,S°), neZ. 3.1 The Basic Equivalences The following theorem characterizes stationarity in several ways. The link to cycle-stationarity is indicated by the last characterization, which is the key to the Palm dualities to be studied in the subsequent sections. Theorem 3.1. Let (Z,S) be supported by the probability space (Q,,J-,T). The following statements are equivalent: (a) (Z,S) is stationary under P. (6) For /e?i®£+ and t G [0, oo), it holds that E[J N' f(6s(Z,S))/Xff. ds\ = tE[f(Z,S)/X0}. (3.1) (c) The variable U is uniform on (0,1] and independent of (Z°,S°), and Nt E[£/(05*(Z,S))] =*E[/(Z0,5°)/Xo] (3.2) fc=i for f &n<Si£+ and t G [0,oo). (d) The variable U is uniform on (0,1] and independent of (Z°,S°), and E[f(0sn (Z, S))/X0] = E[f(Z°, S°)/X0] (3.3) for f &U®C+ andn G Z.
Section 3. Key Stationarity Theorem 255 We prove this result in the next four subsections, but let us first note several interesting consequences. Observe first that taking / = 1 in (3.2) yields (since by stationarity E[N-t] = -E[Nt]) (Z, 5) stationary => E[Nt] = E[l/X0]t, t G E. (3.4) In particular, (3.4) yields [take t = 1] the following result for the intensity E[iVi] of the stationary point-stream 5: (Z, 5) stationary =>■ E[Ni] = E[1/X0). (3.5) Also, we see from (3.4) that if E[Ni] < oo then (since P(50 = 0) < E[N0]) we have P(5o = 0) = 0. This is in fact also true when E[iVi] = oo as can be seen as follows. Let V be uniform on [0,1] and independent of S. Then, by stationarity, P(So = 0) = P(5n = V for some n). Since the 5n are countably many, P(5n = V for some n) = 0. Thus we obtain that a stationary point-stream cannot have a point at the origin: (Z, 5) stationary ^ P(50 = 0) = 0. Finally, due to (c), the origin of a stationary point-stream is placed uniformly at random in the cycle where it lies and independently of the process seen from one of the endpoints of the cycle: tr, ™ ■ \U uniform on (0,1] (Z,S) stationary => { v J I and independent of (Z°,S°). This beautiful fact has the following intuitive explanation. One can think of the origin of a stationary (Z, 5) as chosen uniformly at random in R. The relative position of 0 in (5_i,5o] should therefore be uniform and independent of (Z°,5°). 3.2 Proof: (a) Implies (6) Assume that (a) holds. First suppose / ^ a. Since iVs = 0 for 0 ^ s < So and So ^ -^o, we have / ° f{0s(Z,S))/XN. ds < {S0/X0)a < a. Jo
256 Chapter 8. STATIONARITY, THE PALM DUALITIES Thus the expectation of the left-hand side is finite, which allows us to take the final step in tE[f(Z,S)/X0]= f E[f(6s(Z,S))/XN3}ds (stationarity) Jo = E[J f(0a(Z,S))/XN,ds] = E[J ° f(6s(Z,S))/XN,ds\+E[J f{Ba(Z,S))/XN.ds\. By stationarity, E J ° f{6t(Z,S))/XN,ds] =E[[ N' f{0,(Z,S))/XN,ds], and thus (3.1) is established for / bounded. In order to remove the bounded- ness restriction replace / by /Aa in (3.1) and apply monotone convergence once on the left-hand side and twice on the right-hand side to obtain that (a) implies (&). 3.3 Proof: (b) Implies (c) Assume that (&) holds. The statement (c) is equivalent to the following: for all g G B+ and / G U ® £+ it holds that g(x)dx)E[Y,f(8sk(Z,S))}. (3.6) k=\ In order to establish (3.6), apply (6) to obtain the first equality in fSNt tE[g(U)f(Z°,S°)/X0] = E[J ^ g(U,)f(Os„,(Z,S))/XN,ds] = E[£f(es,(Z,S)) f k g(Ua)/Xkds] k=\ Jsk--i and then note that [ " g(Us)/Xkds= f " g(USk_1+s)/Xkds JSk-.! JO = / g{s/Xk)/Xkds= / g{x)dx. Jo Jo Thus (3.6) holds, that is, (6) implies (c).
Section 3. Key Stationarity Theorem 257 3.4 Proof: (c) Implies (d) We obtain that (c) implies (d) if we can show that (3.2) implies (3.3). For that purpose assume that (3.2) holds and let / be bounded, say / ^ a. Apply (3.2) with / replaced by the function taking ((z«)»gR, (s/b)^) to /(^sn((2s)s6Ki (s/t)^oo)) to obtain the first equality in tE[f(6Sn (Z, S))/X0] = E [ £ f(6Sk+n (Z, S)j\ fc=i N, n = E[Y,f(0sk(Z,S))]-E[J2f(0sk(Z,S))] + V[YJ f(0sk(Z,S))]. Apply (3.2) and / ^ a and divide by t to obtain -an/t < E[f{0Sn (Z, S))/X0] - E[f{Z°,S°)/X0) < an/t. Send t —> oo to obtain that (3.3) holds, that is, (c) implies (d). 3.5 Proof: (gQ Implies (a) Assume that (g?) holds. Take an / 6 H ® £+ to obtain E[/(0t(Z,5))] = E[/(0t_(1_l/)Xo(Z°,5°))] = E[/ /(0.(Z°,So))ds/xo], where the second step is due to U being uniform on [0,1) and independent of (Z°,S°). For any a ^ b and x ^ y it holds that [a, b) n[x,y)-x = [(a V x) A y, {b Ay) Vi)-i (3.7) = [(a - x)+ A (j, - x), (b - x)+ A (y - x)). (3.8) Taking [a, b) = [t- X0, t) and [x,y) = [Xx + ■ ■ ■ + Xk, Xj + ■ • ■ + Xk+i) and applying (3.8) yields ft °° rXk + 1A(t-X! Xk) + / f(6s(Z°,S°))ds = Y, f(0s0Sk(Z,S))ds. Jt-Xo ^^JXk + iAit-Xo Xk) +
258 Chapter 8. STATIONARITY, THE PALM DUALITIES Due to (3.3), rXkJri/\(t-Xi Xk) + E / f(6s6Sk(Z,S))ds/Xo\ LJXH1A(l-Xo Xk)+ ' J U/-XiA(t-X_t + i X0) + ' f(6s(Z°,S°))ds/x0\. Xi/\(t-X_k X0)+ ' J Applying (3.7) with [a, b) = [t- X_k X0, t - X_k+i X0) and \x,y) = [0, X\) yields the second equality in oo fXiAit-X-k + i X0) + E[/(0t(Z,S))] = £> / f(6s(Z°,S°))ds X0\ _o0 lJXlA(t-X_k X0)+ ' J = E[J ' f(6s(Z°,S°))ds/Xo\. Hence E[f(8t{Z, S))] does not depend on t for any /eH® C+, that is, (Z, S) is stationary. Thus (d) implies (a), and the proof of Theorem 3.1 is complete. 4 The Point-at-Zero Duality We are now ready for the first Palm duality between stationarity and cycle-stationarity. This duality has the informal point-at-zero interpretation stated at (1.1), namely, the cycle-stationary dual behaves like the stationary process conditioned on having a point at the origin. We motivate this interpretation in the next section. It is informal because in the stationary case the probability of having a point at the origin is zero. In order to see at this point why a duality with this interpretation is reasonable, consider a stationary recurrent Markov chain in two-sided discrete time. According to the Markov property, future and past are independent given the present. Thus if we condition the stationary Markov chain on being in a particular fixed reference state at time zero (on having a point at the origin), then both future and past consist of the i.i.d. cycles between visits to this reference state. That is, the conditioning makes the stationary Markov chain cycle-stationary. The duality is obtained in two separate steps, one measure-free (shifting to and from a point), the other involving only the measure (length-biasing and length-debiasing the cycle straddling the origin). The order in which the steps are taken does not matter. The measure-free step was taken in Section 2.4, and the biasing (change of measure, Radon Nikodym) step we take now.
Section 4. The Point-at-Zero Duality 259 4.1 Length-Biasing -f-» Length-Debiasing Recall that X0 is the length of the cycle straddling the origin. Suppose we are given a probability measure P on (Q,J-) satisfying E[l/X0] < oo. (4.1) Then we can define a new probability measure P° on (fi, T) by letting it have the density (Radon Nikodym derivative) dP°/dP := l/(X0E[l/X0]) with respect to P, that is, dP° = Jrifv idP {length-debiasing P). (4.2) E[l/A0] From (4.2) we obtain E™ = Eiikr («> Since 0 < X0 < oo implies E[l/X0] > 0, we obtain from (4.3) that E°[X0] < oo. (4.1°) Thus (4.2) can be rewritten as dP = X° .dP° (length-biasing P°). (4.2°) E°[a0J Conversely, suppose we are given a probability measure P° on (fi, T) satisfying (4.1°). Then we can define a new probability measure P on (fi,T) by (4.2°). From (4.2°) we obtain E^ = E^oJ- (4"30) Since X0 > 0 implies E°[X0] > 0, we obtain from (4.3°) that (4.1) holds. Thus (4.2°) can be rewritten as (4.2). [Note that (4.2°) is a reformulation of (4.2), and (4.3°) is a reformulation of (4.3), but (4.1°) is not a reformulation of (4.1): (4.1°) follows from E[l/X0] > 0, not from E[l/X0] < oo, and (4.1°) follows from E°[X0] > 0, not from E°[X0] < oo.] We have established that the length-debiasing at (4.2) is equivalent to the length-biasing at (4.2°). This yields a duality (one-to-one correspondence) between probability measures P on (fi, J7) satisfying (4.1) and probability measures P° on (ft,^) satisfying (4.1°).
260 Chapter 8. STATIONARITY, THE PALM DUALITIES 4.2 Stationarity «-* Cycle-Stationarity Combining this measure duality between P and P° and the measure-free duality [Section 2.4] between (Z,S) and ((Z°,S°),U) yields the following duality between stationarity and cycle-stationarity. Theorem 4.1. Let {Q,J-) be a measurable space supporting {Z,S) and ((Z°,S°),U) where Z and Z° are two-sided shift-measurable processes, S and S° are two-sided sequences of times increasing strictly from — oo to oo with 5_i < 0 ^ So and Sq = 0, and U is a (0,1] valued variable. Let (Z, S) and ((Z°,S°),U) be linked by (Zo,S°)=0So{Z,S) and U = -S-1/X0 or, equivalently, by (Z, S) = ^-(i-t/)*. (Z°, 5°) [thus X0 = XS]. LetT and P° be probability measures on (il,J-) satisfying (4.1) and (4.1°) and linked by (4.2) or, equivalently, by (4.2°). Then (Z, S) is stationary under P (4-4) if and only if (Z°,S°) is cycle-stationary under P° (4.4°) and U is uniform on (0,1] and independent of (Z°,S°). Comment. Note that U is uniform on (0,1] and independent of (Z°,S°) under P if and only if it is so under P° (this follows, for instance, from the equivalence of (4.5) and (4.5°) below). Proof. Due to the equivalence of (a) and (d) in Theorem 3.1, (4.4) holds if and only if for each g 6 B+, / G % ® C+, and n G Z, E[g(U)f(6Sn(Z,S))/X0] = [J g(x)dx)E[f(Z°,S°)/X0]. (4.5) Due to (4.2), (4.5) holds if and only if for each g G B+, f G Ti <2) C+, and n £ Z, V°[g(U)f(6sAZ,S))}=(J g(x)dx)E°[f(Z°,S°)}, (4.5°) which is a reformulation of (4.4°). Thus (4.4) and (4.4°) are equivalent. □
Section 4. The Point-at-Zero Duality 261 4.3 Stationary Intensity f* Mean Cycle-Stationary Cycle-Length Suppose the equivalent statements (4.4) and (4.4°) hold. Then, see (3.5), E[l/X0] = E[iVi] = intensity of the stationary point-stream. Thus (4.1) simply means that the intensity of the stationary point-stream is finite. And (4.3) becomes E^ = eW (4"6) In other words, under the duality established in Theorem 4.1, the intensity of the stationary point-stream is the reciprocal of the mean cycle-length of the cycle-stationary point-stream. This relation is familiar from renewal theory; see Chapter 2, Section 9. 4.4 The Stationary Delay Time Suppose the equivalent statements (4.4) and (4.4°) hold. Then, under P, both —5_i and the stationary delay time So have the density P°(Xo>s)/E0[X0], 0<s<oo, (4.7) that is, both have the distribution function Goo defined (as in the renewal case in Chapter 2) by G (g).= E'[*oAs]_J0xP°(Xo>,)cfa Q<x<oQ Uoo(X)- E°[X0] _ E°[X0] ' u^x<0°- This can be seen as follows. Under P both U and 1 — U are uniform on [0,1] and independent of Xq, and thus So = (1 — U)Xq and —5_i = UXq have the same distribution. For 0 ^ x < oo, and thus the common distribution function is Goo- 4.5 Conditional Distributions Given Xq Are Identical The change of measure at (4.2) length-debiases the distribution of Xq: F°(X0Gdx)= * P(X0&dx), 0^x<<x>; (4.8) xni[\./Ao\ and conversely, (4.2°) length-biases the distribution of Xq: P(X0 6 dx) = *x P°(I0 € dx), 0^x<oo. (4.9)
262 Chapter 8. STATIONARITY, THE PALM DUALITIES On the other hand, conditional distributions given Xq remain the same: P(-|X0) = P°(-|X0) a.s. P and a.s. P°. (4.10) In particular, a.s. P(X0 G dx)) and a.s. P°(X0 G dx), it holds that P((Z°,5°) G -\XQ = x) = P°((Z°,S°) G -\X0 = x), (4.11) P((Z,S) € -\X0 = x) =P°((Z,S) € -\X0 = x). (4.12) These claims are direct consequences of the following lemma. Lemma 4.1. Let (fi, T, P) be a probability space supporting a random element Y in a measurable space (E,£). Let P be the distribution ofY and Q be a probability measure on {E,£) having a density g with respect to P. If we define a new probability measure Q on (Q,J-) by dQ = g(Y)dP (change of measure), then Y has the distribution Q under Q and, for V G T+, EQ[V\Y] = E[V\Y] a.s. Q, (4.13) where Eq denotes expectation under Q, and E expectation under P. PROOF. The random element Y has the distribution Q under Q, since for Ae£, Q(Y eA) = E[l{Y€A}g(Y)} = [ gdP= [ dQ. J A J A Further, for V G T+ and / G £ +, EQ[E[V\Y}f{Y)} = E[E[V\Y]f{Y)g(Y)] (by definition of Q) = E[Vf{Y)g{Y)} (by definition of E[V\Y]) = EQ[Vf{Y)} (by definition of Q) and thus, by definition of EQ[F|y], (4.13) holds. □ Remark 4.1. The pair (Z°,S°) is regenerative if the cycles are i.i.d., that is, if in addition to being cycle-stationary, (Z°,S°) also has independent cycles. The stationary dual {Z,S) of a regenerative (Z°,S°) does not have i.i.d. cycles, since the cycle Co straddling zero is length-biased. However, the cycles of the stationary dual (Z, 5) are still independent, and if we leave out Co, then the remaining cycles ..., C_2, C_i, C\, C2, ■ ■ ■ are i.i.d. copies of the cycles of (Z°,S°). [This follows by noting that (4.10) implies P(.|C0) = P°(-|Co) and thus, for n ^ 1, P(C_n G -,..., C_i G-,d G-,...,C„G-|C0) = P°(C_nG-)---P°(C_1 e.jp^d g-)---P0(C„g-) if (Z°, 5°) is regenerative.]
Section 4. The Point-at-Zero Duality 263 4.6 The Palm Duality in Terms of Distributions Theorem 4.1 gives us a one-to-one correspondence between particular copies of a stationary (Z, 5) and a cycle-stationary (Z°,S°). In the cycle-stationary case the independent uniform (0,1] variable U is redundant, while in the stationary case (Z, 5) is obtained from (Z°, S°) with the aid of U, and if we replaced U by another independent uniform (0,1] variable, then we would obtain another stationary dual having the same distribution as (Z, 5). Thus it is in some sense more natural to think of the duality as a one-to- one correspondence between stationary and cycle-stationary distributions, say P and P°, rather than between particular copies having these distributions. A distributional form of the duality can be obtained from Theorem 4.1 as follows. If a stationary distribution P is given, apply Theorem 4.1 to some (Z, 5) with the distribution P to obtain P° as the distribution of the cycle-stationary dual. Conversely, if a cycle-stationary distribution P° is given, apply Theorem 4.1 to some ((Z°,S°),U), where (Z°,S°) has the distribution P° and U is uniform on (0,1] and independent of (Z°,S°), to obtain P as the distribution of the stationary dual. For more details, see the next subsection. A nondistributional way of getting around the pluralism of Theorem 4.1 would be to assume that ((Z°,S°),U) is canonical. Note that then {Z,S) is not canonical: it is obtained from (Z°,S°) by placing the origin at random in a cycle. [If we assume conversely that (Z, S) is canonical, then ((Z°,S°),U) is not canonical.] We shall use neither the distributional approach here nor the canonical one. Theorem 4.1 is clean-cut, highlights the simple two-step duality construction, and is easy to apply. We shall stick to the Palm duality in this form. Keeping (Q,J-) unspecified also allows us the freedom of adding new random elements when needed. 4.7 Collapsing the Two-Step Construction into a Single Step Suppose the equivalent statements (4.4) and (4.4°) hold. Combining (4.2), (3.2) in Theorem 3.1, and the observation (3.5) yields E°[/(Z°,S°)]=EE^^(Z'S))], f&n®£\ (4.14) thus deriving the distribution of the cycle-stationary (Z°,S°) from the stationary (Z, S) in a single step. This is the most common definition of the distribution P° of the so-called Palm version of (Z,S), called cycle- stationary Palm dual here.
264 Chapter 8. STATIONARITY, THE PALM DUALITIES Conversely, combining (4.2°), (Z, S) = 6_{1_u)Xo(Z0,5°), and the latter claim at (4.4°) yields nnz,s)} = ^^^p^, fzn®c+, (4.i4°) which gives us back, in a single step, the distribution P of the stationary (Z,S). This formula is known as the inversion formula, indicating that the Palm version is thought of as derived from the stationary version and just happens to have the cycle-stationarity property. In our treatment the stationary and cycle-stationary duals have equal status. 5 Interpretation — Point-Conditioning The point-at-zero interpretation of the duality established in Theorem 4.1 [stated in words at (1.1)] can now be formulated as follows: P((Z,5) G -\S0 = 0) = P°((Z°,5°) G •)■ (5-1) This of course does not have an immediate meaning because P(5o =0) = 0. In this section we present several results motivating (5.1). 5.1 The Basic Point-Conditioning Theorem The following theorem is the key result on which the rest of this section relies. It also provides an immediate motivation for the point-at-zero interpretation: put s = 0 in the third identity to obtain (5.1). Since the identity holds only for P(5o G •) a.e. s, the motivation is still informal. However, sending s to 0 provides a formal limit motivation of (5.1). Theorem 5.1. Suppose the equivalent claims (4.4) and (4.4°) hold. Then for s > 0, P((Z°,5°) G -\S0 = s) = P°((Z°,5°) G -|X0 > s), P((Z°,S°) G -I5-, = -s) = P°((Z°,S°) e-\X0 > s), P((Z,S) G -\S0 = s) = P°(0_s(Z°,S°) G -\X0 > s), P({Z,S) G -|5_i = -s) = P°(0s(Z°,S°) g -|X, > s), in the sense that the right-hand sides are versions of the left-hand sides as functions of s.
Section 5. Interpretation - Point-Conditioning 265 PROOF. We start by proving the following reformulation of the first identity: for all h£B+ and / 6 % <8> £+ it holds that E[h(So)9f{S0)) = E{h(S0)f(Z°,S°)}, (5.2) where gj is defined by gf(s) = E°[f(Z°, S°)\X0 >s), s> 0. (5.3) Use (5.3) and (4.7) to take the first step in E[h(S0)gf(S0)] /■OO = / h{s)E°[f(Zo,S°)\X0 > s]P°(X0 > s)/E°[X0] ds Jo /•OO = / h(s)E°[f(Z°,S°)l{Xo>s}}/E°[X0}ds Jo = E° [f(Z°,S°) J°° h(s)l{Xo>s} ds] /E°[X0] = e[/(Z°,5°) J°° h(s)l{Xo>s]ds/x0] [by (4.2)] = E[f(Z°,S°)J ° h{s)ds/x0 = E[h(S0)f(Z°,S°)}, while the last is due to So = (1 — U)Xq, the uniformity of U, and its independence of (Z°,S°). Thus (5.2) holds, that is, the first identity in Theorem 5.1 is established. The second identity in Theorem 5.1 follows from the first, since [due to So = (1 — U)Xo and —5_i = UX0 and the uniformity of U and its independence of (Z°,S°)] ((Z°,50),5o) = ((Z0,5°),-5_1). der to estab the third step in In order to establish the third identity in Theorem 5.1, use the first to take t.hp tVlirH Qtpn in P((Z,5) G -\S0 = s) = P(6_So(Z°,S°) e -|50 = s) = P(6-S(Z°,S°) G -\S0 = s) [cf. Fact 3.1 in Chapter 6] = P0(6LS(Z°,S°) G -\Xq > s) [due to the first identity].
266 Chapter 8. STATIONARITY, THE PALM DUALITIES Finally, in order to establish the fourth identity in Theorem 5.1, use the second to take the third step in P((Z,S) G -IS-! = -s) = P(e_s_^-x0(^o,5o) G -|5-i = -s) = P(6s6-x0(Z°,S°) G -|S_i = -s) [cf. Fact 3.1 in Chapter 6] = P°(0s0-xo(Z°,S°) G -\X0 > s) [due to the second identity] = P°(0.(Zo,So)e-|Xi >s), while the last step is due to cycle-stationarity. □ 5.2 Total Variation Motivation of (5.1) The following theorem gives a strong motivation for the point-at-zero interpretation (5.1): if the stationary process has a point in a small interval [0, t] and its origin is moved to that point, then it is close to its cycle-stationary dual in total variation. In fact, the theorem gives explicit bounds that yield an even stronger limit result: common component convergence. Recall that A denotes greatest common component of measures (or common part; see Section 7.1 in Chapter 3) and that || • || denotes the total variation norm (see Section 8.2 in Chapter 3). Theorem 5.2. Suppose the equivalent claims (4.4) and (4.4°) hold. Then the following bounds hold for t > 0: P((Z°,5°) G -\S0 < t) 2 P°((Z°,S°) G -,X0 > t), (5.4) P((Z°,S°) G -\S0 < t) < P™^. (5.5) This implies that as t \.Q, /\ P((Z°,S0)G-|So^u)tP0((Z°,S0)G-)- (5.6) Moreover, for t > 0, ||P((Z°,5°) G -|50 < t) - P°((Z°,S°) G -)|| < 2P°(X0 < t), (5.7) which implies P{{Z°,S°) G -\S0 <*)->• P°((Z°,5°) G 0 (5-8) in total variation as t 4- 0. Proof. With A G U ® C put h(s) = P°((Z°,S°) G A\X0 >s)= po((z°,s°)£A'Xo>s)_ P°(A'o > s)
Section 5. Interpretation - Point-Conditioning 267 Applying the first identity in Theorem 5.1 yields P((Z°,5°) G A\S0) = h(So) and thus P((Z°, 5°) G A\So < *) = E[/l(5o)|50 < t]. (5.9) For 0 ^ s ^ t, we have P°((Z°,S°) &A) P°((Z°,S°)eA,X0>t)^h(s)t: P°(X0>t) Combining this and (5.9) yields (5.4) and (5.5). The common part result (5.6) follows by noting that the lower bound increases to P°((Z°,S°) G •) and the upper bound decreases to P°((Z°,S°) G •) as t J, 0. In order to obtain (5.7), first deduce from (5.4) that P°((Z°,S°) G A) - P((Z°,S°) G A\S0 < t) < P°(X0 < t) and then take the supremum in A G H <S> C and multiply by 2 [see the second equality at (8.12) in Chapter 3]. The total variation limit result (5.8) now follows by noting that P°(X0 < t) ->■ 0 as 11 0. D 5.3 Smooth Total Variation Motivation of (5.1) The following theorem involves the stationary process in a direct way (without shifting its origin to a point as in Theorem 5.2): if the stationary process has a point in a small interval [0,t], then it is close to its cycle-stationary dual in smooth total variation and also in the stronger sense of smooth common component convergence. Theorem 5.3. Suppose the equivalent claims (4.4) and (4.4°) hold. Then the following bounds hold for t > 0 and h > 0: I h P(6s(Z,S)€-\S0^t)ds (5.10) fh Ih tP°{6s{Z°,S°)&-)ds 2 p°(es(z°,s°)e-,x0>t)ds-J-±-t Jo P°{X0>t) and L h o P°(X0>t) h fn.P°(6s{Z°,S°)&-)ds p(es(z,s)&-\s0^t)ds^J-^—' ' ; (5.ii) This implies that as t J, 0, A / P(0,(Z,S)e-|So<u)dsT / P°(0s(Z°,S°)€-)ds. (5.12) ^..sJo Jo 0<u^t
268 Chapter 8. STATIONARITY, THE PALM DUALITIES Moreover, for t > 0, rh r/l r /l / P(6s{Z,S)G-\S0^t)ds- P°{6s{Z°,S°)€-)ds Jo Jo ^ 2hP°{X0 ^t) + 2t (5.13) P°(Xo>*)' which implies rh r/l rh I P(6s(Z,S)€-\S0^t)ds^ P°(6s{Z°,S°)€-)ds (5.14) Jo Jo in total variation as t J, 0. Comment. The above convergence after time-smoothing has the following randomized-origin formulation: with V uniform on (0,1) and independent of (Z, 5) under both P and P° we have, for h > 0, /\ P(6vh(Z,S)€-\So^s)tP0(6Vh(Z°,S0)€-), t|0, 0<s^t and P(0y„(Z,S) G -|50 <*)->• P°(0vn(Z°,S°) G •) in total variation t J, 0. Proof. We obtain the lower bound (5.10) as follows: for A G % <8> £, -So 1 {0,(Z,S)6/1} ds So<«] = E[/ Ho,(z°,s°)£A}ds So ^ ij ~E / l{es(z°,s°)6/i} ^s 5o ^ * lJh-S0 J ^ / P{ds(Z°,S°)eA\So^t)ds Jo -f P(6s(Z°,S°)€A\S0^t)ds, Jh-t and applying (5.4) and (5.5) in Theorem 5.2 yields (5.10). The upper bound (5.11) is obtained in a similar manner. The common component result (5.12) follows by noting that the lower bound increases to J0 P(«,(2°,5°) e -)ds and the upper bound decreases to
Section 5. Interpretation - Point-Conditioning 269 JgP°{ds(Z°,S°) G -)ds as t I 0. In order to obtain (5.13), first deduce from (5.10) that rh rh \ P°{6s(Z°,S°)€A)ds- P{6s{Z,S)€A\S0^t)ds Jo Jo ^hP°{X0 ^t) + t/P°{X0>t) and then take the supremum in A G H <8> C and multiply by 2 [see the second equality at (8.12) in Chapter 3]. The total variation limit result (5.14) now follows by noting that P°(X0 < t) ->■ 0 and P°(X0 > t) ->■ 1 as Ho. a 5.4 Weak Convergence Motivation of (5.1) The following theorem also involves the stationary process in a direct way (without shifting its origin to a point as in Theorem 5.2): if the stationary process has a point in a small interval [0, £], then it is close to its cycle-stationary dual in the sense of weak convergence (convergence in distribution; see Section 10 of Chapter 3). For this result we need a metric path space: if {E, £) is Polish and D = De{R) is the set of paths that are right-continuous with left-hand limits, then the path space (D, V) is Polish; see Ethier and Kurtz (1986). Thus [see Theorem 2.2 in Chapter 4], {D,V)® (K, B)z is Polish. By weak convergence in (D,T>) <g> (L,£) we mean convergence with respect to the metric that {D,T>) <g> (L,C) inherits as a subspace of {D,V) <g> (K,#)z. Theorem 5.4. Suppose the equivalent claims (4.4) and (4.4°) hold. If (E,£) is Polish and the path space is (D,D), then P((Z,S) G -\S0 < t) -+ P°((Z°,S°) G ■) (5.15) weakly as t \. 0. PROOF. Weak convergence means that for all bounded continuous functions /eP®£+, E[/(Z, S)\So ^t]^ E°[/(Z°, 5°))], 11 0. So let /eP® £+ be bounded and continuous. Due to the total variation limit result (5.8) in Theorem 5.2 [see the last identity at (8.11) in Chapter 3] we have E[f(Z°,S°)\So^t}^E°[f(Z°,S°))}, HO, and thus it only remains to prove \E[f{Z°,S°)\So^t}-E[f{Z,S)\So^t]\^0, HO. (5.16)
270 Chapter 8. STATIONARITY, THE PALM DUALITIES For that purpose define a bounded function //, G V <g> C+ by //,(•)••= sup |/(.)-/(0_„(-))l- For each fixed (.zu)ugR G D the map taking ( e Rto (^t+«)«eR G £> is continuous, and for each fixed (sfc)ff00 G L the map taking £ G K to (s„t_+t — i^oo G L is left-continuous (recall the definition of dt and nt_ at (2.1) and (2.2)). Thus fh \. 0 pointwise as h \. 0. This (together with the boundedness of //,) yields the final step in \E[f{Z°,S°)\So^t)-E[f(Z,S)\S0^t}\ ^E[\f(Z°,S°)-f(Z,S)\\S0^t] < E[A(Z°,5°)|50 < t) for t < ft [(Z,5) = 6-So(Z°,S°)} ->E°[fh{Z°,S°)] ast|0 [due to (5.8)] -» 0 as ft 4- 0. Thus (5.16) holds, and the proof is complete. □ 5.5 Coupling Motivations of (5.1) The convergence modes (5.6), (5.12), and (5.15) have coupling counterparts established in Sections 9 and 10 of Chapter 3. Theorem 5.5. Suppose the equivalent claims (4.4) and (4.4°) hold. Then the following statements hold. (a) There is a probability space supporting pairs (Z^l\S^), 0 ^ t < oo, and a strictly positive random variable R such that (Z<f>,S<f>) has distribution P((Z°,5°) G -\SQ < *). t > °> (ZW,5(0)) has distribution P°((Z°,5°) G •), (ZW,S^) = (Z<°\S<°>) for 0 < i< R. (b) For ft > 0, there is a probability space supporting pairs (Z(h'l\ S^h'^), 0 ^ t < oo, and a strictly positive random variable R(h) such that 1 fh ^Z(h,t)^s(h,t^ has distribution- / P(0s(Z, 5) G -j^o ^i)cfs, t>0, (Z(/l'0),S(/l'0)) /ms distribution \ f P°(0s(Z°, 5°) G ■) ds, " Jo (Z(M))5(ft,0) = (z(ft.»)>S(M)) /or 0 < 4 ^ #00.
Section 6. Application - Perfect Simulation 271 (c) // (E,£) is Polish and the path space is (D,T>), then for all strictly positive sequences tn, 0 ^ n < oo, decreasing strictly to 0 as n —» oo, there is a probability space supporting pairs [Z^tn\ S"n'), 0 < n < oo, and (Z(°),5(°)) such that (Z(*»))5(tn)) has distribution P((Z,S) G -\S0 ^ tn), n > 0, (Z(0),S(0)) has distribution P°{{Z°,S°) G ■), pointwise as n —} oo. Proof, (a) By Theorem 9.4 in Chapter 3 the result (5.6) in Theorem 5.2 above is equivalent to (a). In order to see this let Yt have distribution P((Z,S) G -|50 ^ l/t) and Y^ have distribution P°((Z°,5°) G •) and put (Z^.SW) := y1/t, (Z(°),S(°)) := Yoo and i? := 1/K. (b) By Theorem 9.4 in Chapter 3 the result (5.12) in Theorem 5.3 above is equivalent to (&). In order to see this let Yt have the distribution h~l JQ P(8S(Z,S) G |5o ^ l/t)ds and let Y^ have the distribution h'1 JoP°{{Z°,S°) G-)^andput(Z(M),S(/a>):=y1/t,(Z(/,'0),S(','0>) : = Yoo and R^ := 1/K. (c) The metric, that the subspace {D,V)®{L, C) inherits from the Polish {D,T>) <S) (K, B)z, is separable. Therefore, according to Theorem 10.1 in Chapter 3, (5.15) in Theorem 5.4 above is equivalent to (c). In order to see this take Pn = P((Z,5) G -\S0 ^ tn) and P = P°((Z°,5°) G •) and put (Z('»),S('")) := Y"("> and (Z(°',5(0)) := Y. D 6 Application - Perfect Simulation In this section we show how the two-step duality construction in Theorem 4.1 yields perfect solutions of a classical simulation problem, the so- called initial transient problem. The solutions are based on the so-called acceptance-rejection algorithm. We start by explaining the problem in general terms. 6.1 The Initial Transient Problem Stochastic simulation is concerned with generating realizations of random variables and stochastic processes, usually in order to estimate properties that cannot for some reason be calculated mathematically (for instance, because of their complicated structure or because the input in the stochastic model under consideration is not completely known). An old problem in this context is the following: how can we generate a stationary version of a given process. Suppose, for instance, we know the transition probabilities of an irreducible positive recurrent Markov chain but cannot calculate the stationary distribution. Then we could start a
272 Chapter 8. STATIONARITY, THE PALM DUALITIES chain in a fixed state, let it run long enough so that it gets close to stationarity, and then use the realization from that time onward to estimate the unknown stationary distribution. The obvious problem is, what is long enough? If we do not wait long enough, a bias will be introduced. Consider, for instance, a queueing model starting with an empty system. The system will take some time to fill up, and if we do not wait long enough, we will underestimate the stationary queue length or work load. This problem goes under the name the initial transient problem. Our solutions to the initial transient problem are not obtained by waiting until the process is 'close enough' to stationarity. We shall generate a process in perfect stationarity. This is called perfect simulation. And we shall not do this by waiting for a single process to be in perfect stationarity but by generating several processes until the right one is found: we shall use acceptance-rejection. 6.2 Acceptance—Rejection Acceptance-rejection is a method for producing a random element with a desired distribution Q by selecting it from a sequence of i.i.d. random elements Y(l\Y(2\ ... from another distribution P. This P must be such that Q has a bounded density g with respect to P, say g ^ c. Sequentially, for each n, accept y(n) with probability g(y)/c, where y is the realized value of y(n). According to the following theorem, an accepted random element has the desired distribution Q. Theorem 6.1. Let (Y^\I^l)), (y(2\/(2)),... be i.i.d. copies of a pair (Y,I), where Y is a random element in some measurable space (E,£) with distribution P and I is a 0-1 variable taking the value 1 with probability p > 0. Let these pairs be defined on a probability space (fls\m,^simjPsim)- Then the following claims hold. (a) Define a geometric random variable M with parameter p and mean 1/p by M = inf{n > 1 : I<-n) = 1} = number of acceptance-rejection trials. Then y(M) has the distribution Psim(y £ -\I = 1). This distribution has a density with respect to P that is bounded by 1/p. (b) Let Q be a probability measure on (E,£) having a density g with respect to P and suppose there is a finite constant c such that g ^ c. If Ps[m(I=l\Y) = g(Y)/c, (6.1)
Section 6. Application - Perfect Simulation 273 then M has parameter p = 1/c and Esim[M] = c and Esim[/(^(M))] = Esim[f(Y)g(Y)} = J fdQ, f G £+ , (6.2) that is, y(M) has the distribution Q. (c) Suppose g ^ c and I = l{u^.g<Y)/c}> where U is independent ofY and uniformly distributed on (0,1). Then (6.1) holds. Proof, (a) Clearly, M has geometric distribution with parameter p, and thus Esim[M] = IJ p. Further, for / G £+, OO Esim[/Cr(M))] = ^Esim[/(y("))l{M=n}] n-\ oo = ^Esim[/(y("))/(")i{M^n}] n=\ oo = Y, Esim[/(y(n))/(n)]Psim(M ^ n) (independence) n=l oo = Esim[/(y)J]^Psim(AOn) n=l = Esim[/(y)/]Esim[M] (Lemma 10.1 in Chapter 2) = E8im[/(y)i]/p = Esim[/(y)|/ = l], and thus y(M) has the distribution Psim(y G -|J = 1). Since Psim(^ G -|/ = 1) < Psim(^ G -)/Psim(/ = 1) = P/p, it follows that PSim(y G -|I = 1) has a density with respect to P and that the density is bounded by 1/p. (6) If (6.1) holds, then P = Psim(/ = 1) = Esim[g(Y)]/c = 1/c, and thus Es;m[M] = c. Further, due to (a) and p = 1/c, we have Esim[/(y(M))] = Esim[/(y)|/ = i] = Esim[/(y)i]c = E8im[/(y)Esim[/|y]]C) and since Esim[J|y] = Psim(7 = l|y) = g(Y)/c, this yields Esim[/(y(M))] = Esim[f(Y)g(Y)} = JfgdP = jfdQ.
274 Chapter 8. STATIONARITY, THE PALM DUALITIES (c) Since U is uniform on (0,1) and independent of Y, we have Psim(I = 1\Y) = Psim(U ^ g(Y)/c\Y) = g(Y)/c as desired. □ Remark 6.1. An interesting feature of the acceptance-rejection method is that we need to know neither P nor Q. We only need to know g/c and to be able to obtain realizations of the i.i.d. Y^ ,Y(2\ These realizations need not be produced by an explicit use of P, that is, the characteristics of the input in the stochastic model under consideration need not be completely known. It can, for instance, be the output of another simulation. Remark 6.2. Let Q be a probability measure on (E,£) having a density g with respect to P. If we define a new probability measure QSim on (ftsirru^sim) by dQsim = g(Y)dPsim, then by Lemma 4.1, Y has the distribution Q under Q5im. Thus y(M) has the same distribution under Psim as Y has under QSim, that is, the acceptance-rejection algorithm has the same effect as a change of measure. The simulation relevance of Theorem 4.1 should now be getting clearer. 6.3 Generating Stationary Renewals — Bounded Recurrence Before applying the full strength of Theorem 4.1, let us consider an elementary special case, namely the problem of generating a stationary renewal process when it is known how to generate the i.i.d. recurrence times A"i,X2, ■ • • but their distribution function F is not explicitly known. In Section 9 of Chapter 2 (and Theorem 4.1 above) we showed that if F has a finite mean m, then a stationary renewal process is obtained by placing the origin uniformly at random in an interval of length Xq, where Xq has the density x/m, 0 < x < oo, with respect to F. Thus the simulation problem is solved if we can generate such an Xq. This can be done by acceptance-rejection if the recurrence times are bounded with probability one by a known finite constant, say Xx ^ a. Then g(x) = x/m, 0 < x ^ a, is a density of X0 with respect to F, and g ^ a/m. Theorem 6.1(6 and c) thus yields the following procedure for generating X0: let U(l\U(2\... be i.i.d., uniform on (0,1) and independent of the Xn and accept the first Xn satisfying [/(") ^ Xn/a. This approach is extended beyond renewal processes in the next subsection.
Section 6. Application - Perfect Simulation 275 6.4 Generating the Stationary Dual When X0 ^ a < oo Now consider the duality established in Theorem 4.1 between a stationary (Z,S) under P and a cycle-stationary (Z°,S°) under P°, where dP/dP° = X0/E[X0] and (Z,S) = 0_(1_U)Xo(Z°,S°). Suppose we wish to generate the stationary (Z, S) when it is known how to generate its cycle-stationary dual. Here is an acceptance-rejection solution in the bounded cycle-length case, that is, when there is a known constant a < oo such that P°(X0^a) = l. (6.3) Recursively, for n ^ 1: 1. Generate (Z(n\S^) with distribution P°((Z°,S°) e •) until X{0n) has been realized. 2. Generate an independent [/(n) uniformly distributed on (0,1). 3. Repeat steps 1 and 2 independently for n ^ 1 until {[/(") ^ Xq /a} occurs and put M = inf{n ^ 1 : U^ «C X{0n)/a}. 4. Now generate as much of (Z(M',S(M') as desired. 5. Generate an independent V uniformly distributed on (0,1). According to the following theorem, 8_v (M)(Z'M\S^M)) is a (perfect) copy of the stationary dual, and the expected number of acceptance-rejection trials is a/E°[Xo]. Theorem 6.2. Let (fiSim,-^simjPsim) be the probability space supporting the random elements generated in steps 1 through 5. If (6.3) holds, then Psim(^vxr>(i?(M),S(M)) e •) = P((Z,S) e •), (6.4) Esim[M] = a/E°[X0]. (6.5) Proof. Apply Theorem 6.1(6 and c) [see Remark 6.2] with P = P°((Z°,S°)e-) and Q = P((Z°,S°)e-), g(Z°,S°) = Xo/E°[X0} and c = a/E°[X0], to obtain Psim((Z(M),S(M>) G ■) = P((Z°,S°) £ ■). Now (6.4) follows from the fact that both 9_vxiM)(Z<-M\S^M'>) and (Z,S) are obtained by placing the origin uniformly at random in the interval straddling zero. By Theorem 6.1, Esim[M] = c, and (6.5) follows. □
276 Chapter 8. STATIONARITY, THE PALM DUALITIES 6.5 Example — The S-s-Inventory System An example of a process with bounded cycle-lengths is the supply process in the so-called S-s-inventory system with a deterministic demand component. This system is as follows. Some material is stored in a storage with maximal capacity S. Demand is the sum of a linear deterministic component and a random stationary component. The deterministic demand has rate d > 0, that is, during a period of length t the quantity demanded is dt. The random demand is compound Poisson, that is, i.i.d. nonnegative quantities are demanded at the times of a renewal process with exponential recurrence times (more generally, the compound Poisson demand can be replaced by any stationary stochastic process with nonnegative independent increments). When the supply drops below a minimal level s, the storage is filled up again to its maximal capacity S. Call the process formed by the supply in store at time i, 0 ^ t < oo, the supply process. The times between successive jumps to the maximal level S split the supply process into an i.i.d. sequence of cycles. Thus, if it is known how to generate the demand, then we can generate a cycle- stationary supply process by starting at time 0 with maximal supply S. The cycle-lengths are bounded by a = (S — s)/d, and thus the procedure in Section 6.4 gives us a way to generate the stationary version of the supply process. 6.6 Generating the Cycle-Stationary Dual When X0 ^ 6 > 0 Consider again the duality established in Theorem 4.1 between a stationary (Z,S) under P and a cycle-stationary (Z°,S°) under P°, where dP°/dP = 1/(E[1/X0]X0) and (Z°,S°) = 6So(Z,S). This time suppose we wish to generate the cycle-stationary (Z°,S°) when it is known how to generate its stationary dual. Here is an acceptance- rejection solution in the case when the cycle-lengths are bounded away from zero, that is, when there is a known constant 6 > 0 such that P(X0 ^ 6) = 1. (6.6) Recursively, for n ^ 1: 1. Generate {Z<-n\S^) with distribution P((Z°,S°) G ■) until X^n) has been realized. 2. Generate an independent f7(n) uniformly distributed on (0,1).
Section 6. Application - Perfect Simulation 277 3. Repeat steps 1 and 2 independently for n ^ 1 until {£/(") ^ b/X^ } occurs and put M = inf{n^ 1 :!/(") «C b/X{0n)}. 4. Now generate as much of {Z^M\ S^M)) as desired. According to the following theorem is a (perfect) copy of the cycle-stationary dual, and the expected number of acceptance-rejection trials is 1/(E[1/X0]b). Theorem 6.3. Let (fiSim,-^simjPsim) be the probability space supporting the random elements generated in steps 1 through 4- If (6-6) holds, then ps]m((z(M\sW)e-) = p°((Z°,s°)e-), Esim[M] = 1/(E[1/X0]6). Proof. Apply Theorem 6.1(6 and c) [see Remark 6.2] with P = P((Z°,S°)G-) and Q = P°((Z°,S°) e •), g(Z°,S°) = 1/(E[1/X0]X0) and c = 1/(E[1/X0]6), to obtain the desired results. □ 6.7 Generating the Stationary Dual — Delay Time Given Once more consider the duality established in Theorem 4.1 between a stationary (Z,S) under P and a cycle-stationary (Z°,S°) under P°. We shall now show that the problem of generating the stationary (Z,S), when it is known how to generate its cycle-stationary dual, can be reduced to that of generating the stationary delay time, that is, a random variable having the distribution Goo with density P°(X0 > x)/E°[X0], 0 < x < oo, [see (4.7)]. We shall use acceptance-rejection and the following result from Theorem 5.1: P((Z°,S°)e-\S0=s) = P°((Z°,S°)e-\X0>s), s>0. (6.7) Proceed as follows: 1. Generate W with distribution Goo and let W be the stationary delay. 2. Generate independent (Z(n\S(n)) with distribution P°((Z°,S°) <= •) until X^1' has been realized.
278 Chapter 8. STATIONARITY, THE PALM DUALITIES 3. Repeat step 2 independently for n ^ 1 until {X^n' > W} occurs and put M = inf{n ^ 1 : X^n) > W}. According to the following theorem, 9-w{Z^M\ S^M)) is a (perfect) copy of the stationary dual, and the expected number of acceptance-rejection trials is infinite! Theorem 6.4. Let (fiSim,•T'simjPsim) be the probability space supporting the random elements generated in steps 1 through 3. Then Psim(0.w(Z^M\S^) G •) = P((Z,S) G ■), (6.8) Esim[M] = oo. (6.9) Proof. We shall use Theorem 6.1(a) with Psim replaced by Psim(-| W = s). Since W is independent of the (Z^n\ S^) this is the same as applying Theorem 6.1(a) with y(n) = (Z(n),S(n>) and /(") = {X^n) > s}. Thus, for s >0, Psim((£(M),S(M)) G -\W = s)= P°((Z°,S°) G -\X0 > s), (6.10) E8im[M|W = s} = 1/P°(X0 > s). (6.11) Comparing (6.10) and (6.7) yields Psim((Z(M\S(M)) G -\W = s)= P((Z°,S°) G -\S0 = s), s> 0. This and the fact that Psim(W G •) = P(-So £ •) shows that Psim(((^(M),S(M)),^) G •) = P(((Z°,S°),So) G ■)■ Now (6.8) follows from the fact that d-W(Z^M\ S(M>) is the same measurable mapping of ((Z(M), S<M>), W) as (Z,S) is of ((Z°,S°),S0). Further, recall that W has density P°(X0 > s)/E°[X0]. This yields the first equality in Esim[M] = r-Esim[M\W = s]P°(X0 > s)ds/e°[X0] = J ds/E°[X0] (due to (6.11)), and noting that J"0°° ds = oo yields (6.9). □
Section 6. Application - Perfect Simulation 279 6.8 Imperfect Simulation — Unbounded Cycle-Lengths Finally, let us see what happens if we apply the method of Section 6.4 without the assumption (6.3) that the cycle-lengths are bounded. So fix an a < oo, carry out steps 1 through 5 in Section 6.4, and denote the number of acceptance-rejection trials by Ma. According to the following theorem, 9_ {Ma)(Z^Ma\ S^Ma^) is an imperfect copy of the stationary dual V Aq with perfection probability Goo(o/Goo(a))- Note that Goo (a/Goo (a)) ^ Goo(a) -> 1 as a ->oo. Theorem 6.5. Let a > 0 be a finite constant and let (0,s\m, Jrs\m,Ps\m ) be the probability space supporting the random elements generated in steps 1 through 5 of Section 6-4- Denote the number of acceptance-rejection trials by Ma. Then \psim{e_vx(oMa)(z(M°\s^)£-)-p((z,s)£ = 2(1-Goo(a/Goo(a))), ||psim((z(M«\s(M°>) g •) - P((z°,s°) e -)ll = 2(1-000(0/000(0))), (zW,s(M-))e-)AP((z,S) e- = Goo(a/Goo(a)), ||Psim((Z(M°\S(M°)) G 0 A P((Z°,S°) G Oil = Goo (a/Goo(a)), (6.12) (6.13) (6.14) (6.15) E8im[Ma]=o/Eo[oAX0]. (6.16) Proof. Write M = Ma. Apply Theorem 6.1(6 and c) [see Remark 6.2] with P = P°((Z°,S°) GO, g(Z°,S°) = (aAl0)/E°[oMo] and c = a/E°[a A X0], to obtain (6.16) and Esim[/(Z(M\ S(M>)] = E°[/(Z°, S°)(a A X0)]/E> A X0]. (6.17)
280 Chapter 8. STATIONARITY, THE PALM DUALITIES Due to Lemma 4.1, it follows from (6.17) that Psim(^M) e dx) = (aAx)P°(X0 G dx)/F,°-[aAX0], (6.18) Psim((Z(M\S(M)) G -\X{0M) = x)=P°((Z°,S°) G -|*o= x). (6.19) By (4.11), we have P°((Z°,S°) G .\X0=x) = P((Z°,S°) G -\X0 = x). This and (6.19), together with Lemma 3.1 of Chapter 6, yields the first equality in ||Psim((Z(M\ S(M>) G •) - P((Z°,S°) G -)|| = ||Psim(x(M)G0-P(*oG0ll (6-20) = 2(l-||Psim(X<iM)e0AP(XoG0ll), while the second identity follows from (8.12) of Chapter 3. By (4.9) we have P(X0 G dx) = xP°(X0 G dx)/E°[X0], 0 < x < oo. This and (6.18), together with (8.5) of Chapter 3, yield the first equality in ||Psim(^M)GOAP(X0GOII /nAi x „.„,,, W^]AnMp{Xoedx) = ^]InaAxa0}/nx0]Axpa{Xoedx) = i^Fl / Tr^T\ A x P°(Xo G dx) [by definition of G^] = Groo(a/Groo(a)) [by definition of Goo]. This and (6.20) yield (6.13). Since V and (1 — U) are identically distributed and independent of (Z°, S°) and (Z(M\ S(M)), respectively, we obtain from (3.3) of Lemma 3.1 in Chapter 6 that ||Psim(((Z(M\S(M>), V) GO- P{{(Z°,S°), (1 - U)) G Oil (6.21) = ||Psim((Z(M),S(M>) GO- P((Z°,S°) G Oil- Since, moreover, 0_vx(m)(Z(m\ S^m^) is the same measurable mapping of ((Z<M>, S(M0, V) as {Z, S) is of ((Z°, S°), {1-U)), and since this mapping
Section 7. The Invariant cr-Algebras I and J 281 has a measurable inverse, we obtain from (3.2) of Lemma 3.1 in Chapter 6 that the left-hand sides of (6.21) and (6.12) are identical. Thus (6.12) follows from (6.13). Finally, due to (8.12) in Theorem 8.2 of Chapter 3, (6.12) and (6.14) are equivalent and (6.13) and (6.15) are equivalent. □ 7 The Invariant cr-Algebras X and J In the previous three sections we have been concerned with the point-at- zero Palm duality. We now start preparing for the other Palm duality, the randomized-origin duality, which will be established in the next section. This latter Palm duality is obtained in the same way as the first, except that the length-debiasing and length-biasing are done conditionally on the invariant a-algebra J of the process and points. In this section we show that stationarity and cycle-stationarity properties are preserved under conditioning on J. 7.1 Definitions — Observations The pair (Z,S) is a measurable mapping from (0, T) to (H x L,1-L ® C). Define the invariant a-algebra on (H x L,H <g> C) by I:= {B eU®C:6^B = Biox te E} (7.1) and the invariant a-algebra of (Z,S) by J:={Z,S)~ll [that \s,J = {{(Z,S)£B} : B e 1}]. (7.2) Thus I is a sub-er-algebra of H <8> C, while J is a sub-er-algebra of T. According to the following lemma, J is also the invariant cr-algebra of 9t(Z,S) for any finite time T supported by (0, J7). Since (Z°,S°) = 9s0(Z,S), this means in particular that J is the invariant cr-algebra of (Z°,S°): J=(Z°,S°)-ll [that is, J = {{(Z°,S°)eB}: Bel}]. (7.3) Lemma 7.1. For any finite time T supported by (Q,J-) it holds that {8T(Z,S)GB} = {(Z,S) GB}, Be 1. (7.4) Proof. For B el we have {8T(Z,S)eB}= \J{8t(Z,S)eB,T = t} tew. = \J{(Z,S) e b,t = t} = {{z,s) e B} tew. as desired. □
282 Chapter 8. STATIONARITY, THE PALM DUALITIES 7.2 Conditioning on J We now show that stationarity and cycle-stationarity properties are preserved under conditioning on J. Theorem 7.1. Let (Z, S) and ((Z°,S°),U) be linked as in Section 2.4 and let P and P° be two probability measures on (fl, F). Then the following claims hold. (a) The pair (Z,S) is stationary under P if and only if it is so conditionally on J, that is, if and only if E[f(0t(Z,S))\J]=E[f(Z,S)\J] (7.5) for f GH&C+ and t e E. (6) The pair (Z°, S°) is cycle-stationary under P° if and only if it is so conditionally on J, that is, if and only if E°[f(0sAZ,S))\J] = E°[f{Z°,S°)\J] (7.6) for f £%®C+ andneZ. (c) The formula (3.3) holds if and only if it holds conditionally on J, that is, if and only if E[f(0Sn(Z,S))/Xo\J] = E[f(Z°,S°)/X0\J] (7.7) for f eH®C+ andne 1. (d) The formula (3.2) holds if and only if it holds conditionally on J, that is, if and only if N, E[YJf{eSk{Z,S))\j}=tE[f{Z°,So)IX0\J] (7.8) k=\ for f GH&C+ and t G K. (e) Suppose (Z,S) is stationary under P. Then U is independent of J and E[JVi|J] = E[1/X0|J] {conditional intensity). (7.9) Proof, (a) Clearly, the 'if part holds [take expectations in (7.5)]. In order to prove the converse, suppose (Z, S) is stationary under P. Then for B £ U <g> C, f G U <g> £+, and t € E, we have E[f(8t(Z,S))l{el{z,s)eB}] = E[f{Z,S)l{{Z,s)€B}].
Section 7. The Invariant u-Algebras I and J 283 By the definition of X, this yields that for B G X, f G U ® £+, and i G R, E[/(6»t(Z, S))l{(z,s)eB}] = E[/(Z, S)l{(z,S)eB}], which is a reformulation of (7.5). (6) Clearly, the 'if part holds [take expectations in (7.6)]. In order to prove the converse, suppose (Z,S) is cycle-stationary under P°. Then for B£?{®£,/e^®£+,andnGZ,we have E°[/(0s„(-£, S))l{eSn(Z,S)eB}} = E°[/(Z°, S°)1{(Z° ,s°)eB}]- Apply Lemma 7.1 to obtain that for B G X, / G W <8> £+, and (£i, E°[/(6»s„(^5))l{(Z,S)eB}] = E°[/(Z°,S°)l{(Z,S)eB}], which is a reformulation of (7.6). (c) This follows by replacing E°[-] by E[-/X0] in the proof of (6). (d) Clearly, the 'if part holds [take expectations in (7.8)]. In order to prove the converse, suppose (3.2) holds. Then for B G W<8> C, f € % <8> C-+, and n € Z, we have [apply (3.2) with / replaced by /1b] jv, E[^/(^(^5))l{eSt(Z,s)eB}]=tE[/(Z0,S0)l{(ZoiSo)eB}]. Apply Lemma 7.1 to obtain that for B G 1, f G U <8> £+, and t G K, E[^/(6»st(^S))l{(Z,s)€B}]=tE[/(Z0,S0)l{(Z,s)eB}], which is a reformulation of (7.8). (e) Suppose (Z, S) is stationary. Then, by Theorem 3.1, U is independent of {Z°,S°). Due to (7.3), J C ^{(Z0,^0)}. Thus U is independent of J. Also, by Theorem 3.1, (3.2) holds, and thus (7.8) holds. Take t = 1 and / = 1 in (7.8) to obtain (7.9). □ 7.3 The Point-Shift Invariant a-Algebra Coincides with J We now establish a curious result, namely, that the cr-algebra invariant under shifts to the points in fact coincides with J. Since J is invariant under all shifts, one would expect it to be strictly smaller. For igZ, define the point-shift r^ from H x L to H x L by Tk(z,s)=eth(z,8). (7.10) Theorem 7.2. It holds that B G I if and only if B = t~1B, n£Z. (7.11) Thus 1 = {B G H ® C : t~1 B = B for n G Z}.
284 Chapter 8. STATIONARITY, THE PALM DUALITIES Proof. In order to show that B£l implies (7.11), apply Lemma 7.1 with the general (Q, T) replaced by the canonical {Hy.L,1-L®C) and T replaced by sn and note that we can write (7.11) as B = {dSn{z,s)eB}, n<=Z. In order to establish the converse, suppose (7.11) holds. Thus 6tlB = 0^TolB [due to (7.11)] = {Todt(z,s)eB} nSZ = U {(z's) € ■B's"-1 < * ^ s«) Idue t0 (7-11)] nSZ = {(z,a)GB}=B. Thus (7.11) implies that G^1 B = B for all t e R, that is, Bel. □ 8 The Randomized-Origin Duality We are now ready for the latter Palm duality between stationarity and cycle-stationarity. This duality has the informal randomized-origin interpretations stated at (1.2) and (1.2°), namely, the cycle-stationary dual behaves like the stationary process with origin shifted to a point chosen uniformly at random among all the points, and conversely, the stationary dual behaves like the cycle-stationary process with origin shifted to a time chosen uniformly at random in R. We motivate these interpretations in the next section. They are informal because there is neither a uniform distribution on a countable set of points nor on R. In order to see at this point why a duality with these interpretations is reasonable, consider a stationary recurrent Markov chain in two-sided continuous time. If we leave out the cycle straddling the origin, then the cycles between entrances to a particular fixed reference state are i.i.d. Thus if we allow ourselves to pick a cycle uniformly at random among all the cycles, then the one straddling the origin should be lost (should disappear to plus or minus infinity), and thus the cycles seen from the selected cycle should form an i.i.d. sequence. Conversely, if we have a process formed by such i.i.d. cycles, then selecting a new origin uniformly at random in R should result in a stationary process. In fact, selecting a time uniformly at random in R should result in ending up in a cycle that is stochastically longer than a typical cycle because our uniform time is more likely to end up in a long interval than a short one. The longer the cycle, the likelier it is to be selected. Thus the length-biasing below. (Why the conditioning on
Section 8. The Randomized-Origin Duality 285 the invariant u-algebra J is needed might be clarified by the example in Section 10.2 below.) Like the point-at-zero duality, the randomized-origin duality is obtained in two separate steps, one measure-free (shifting to and from a point), the other involving only the measure (length-biasing and length-debiasing the cycle straddling the origin, this time conditionally on the invariant cr-algebra J). The order in which the steps are taken does not matter. The measure-free step was taken in Section 2.4, and the biasing (change of measure, Radon Nikodym) step we take now. 8.1 Length-Biasing -B- Length-Debiasing Recall that Xq is the length of the cycle straddling the origin. Suppose we are given a probability measure P on (0, T) satisfying E[1/X0|J] < oo. (8.1) Then we can define a new probability measure P° on (fi, J7) by letting it have the density dP°/dP := 1/(X0E[1/X0|J]) with respect to P, that is, dP° = i^T^7TdP (length-debiasing P given J). (8.2) Lemma 8.1. // (8.1) and (8.2) hold, then P° = P on J, (8.3a) E°[Y\J] = W2QQt Y&T+, (8.36) E[1/X0|J] 1 E[1/X0|J]' Proof. We obtain (8.3a) as follows: for A G J E°[*o|J] = ^ * ,„,• (8.3c) P°(A)=E°[ly4] = E[ly4^^] (due to (8.2)) = E [E [1A e[11//^°|j] I J] ] (conditioning on J) - E[l^ . , ^[1/XolJ]] (moving out functions in J) = E[U]=P(A).
286 Chapter 8. STATIONARITY, THE PALM DUALITIES We obtain (8.36) as follows: for Y G T+ and A <= J, l/*o E°[1AY] = E = E UY E[1/X0|J]J ^y^h-p] = E = E° h E[1/X0|J] 1 E[1/X0|J] 1 U; (due to (8.2)) (conditioning on J) E[yyX0| J] (moving out functions in J) E[y/X0|J]| (due to (8.3a)). E[1/X0|J]' Take Y = X0 in (8.36) to obtain (8.3c). □ Since 0 < X0 < oo implies E[l/X0| J] > 0, we obtain from (8.3c) that E°[X0|J] < oo. (8.1°) Thus (8.2) can be rewritten as X0 dP E°[X0|J] dP° (length-biasing P° given J). (8.2°) Conversely, suppose we are given a probability measure P° on (0, J7) satisfying (8.1°). Then we can define a new probability measure P on (0,T) by (8.2°). From (8.2°) we obtain (mimicking the proof of Lemma 8.1) P = P° on J, E°[yx0|j] E[T|J] E[1/X„|J] = i^ E°[X0|J] ' 1 Y eT+, e°[x0ij]' (8.3a°) (8.36°) (8.3c°) Since X0 > 0 implies E°[X0|J] > 0, we obtain from (8.3c°) that (8.1°) holds. Thus (8.2°) can be rewritten as (8.2). We have established that the length-debiasing at (8.2) is equivalent to the length-biasing at (8.2°). This yields a duality (one-to-one correspondence) between probability measures P on (0, T) satisfying (8.1) and probability measures P° on (0,T) satisfying (8.1°). 8.2 Stationarity *+ Cycle-Stationarity Combining this measure duality between P and P° and the measure-free duality [Section 2.4] between (Z,S) and ((Z°,S°),U) yields the following duality between stationarity and cycle-stationarity.
Section 8. The Randomized-Origin Duality 287 Theorem 8.1. Let {^i,T) be a measurable space supporting (Z,S) and ((Z°, S°),U), where Z and Z° are two-sided shift-measurable processes, S and S° are two-sided sequences of times increasing strictly from — oo to oo with S_i < 0 ^ So and Sq = 0, and U is a (0,1] valued variable. Let (Z, S) and {(Z°,S°),U) be linked by (Z°,S°) = 9So(Z,S) and U=-S^IX0 or, equivalently, by (Z,S) = dHl_U)XS{Z°,S°) [thus X0 = X°]. LetP andP° be probability measures on (£l,F) satisfying (8.1) and (8.2), that is, E[l/X0|J]<oo and dP° = ^ dP or, equivalently, satisfying (8.2°) and (8.2°), that is, E°[X0|J]<<X) and dP= *° dP°. Then (Z, S) is stationary under P (8-4) if and only if (Z°,S°) is cycle-stationary under P° and U is uniform on (0,1] and independent of (Z°, S°). Comment. Note that U is uniform on (0,1] and independent of (Z°,S°) under P if and only if it is so under P°. This follows from Lemma 4.1: first take g(Z°, S°) = 1/(X0E[1/X0|J]) and then g(Z°, S°) = X0/E°[X0|J] to obtain that the conditional distribution of U given (Z°, S°) is uniform on (0,1] under one of the measures if and only it is so under the other. Proof. Due to the equivalence of (a) and (d) in Theorem 3.1 and due to Theorem 7.1(c), (8.4) holds if and only if U is uniform on (0,1] and independent of (Z°, S°) under P and, for / G U <g> C+ and n G Z, E[/(6»Sn (Z, S))/X0\J] = E[/(Z°, S°)/X0\J]. (8.5) Thus [according to the above comment] the equivalence of (8.4) and (8.4°) follows if we can establish that (8.5) is equivalent to (Z°,S°) being cycle- stationary under P°. For that purpose, divide by E[l/X0| J] on both sides of (8.5) and then apply (8.36), on the left with Y = f(8s„(Z,S)) and on the right with Y — f(Z°,S°), to obtain that (8.5) is equivalent to E°[f(0sn(Z,S))\J] = E°[/(Z°,S°)|J], /£Ji8£+,»GZ. Due to Theorem 7.1(6), this holds if and only if (Z°, S°) is cycle-stationary under P°. □ (8.4°)
288 Chapter 8. STATIONARITY, THE PALM DUALITIES 9 Interpretation - Cesaro Limits and Shift-Coupling The randomized-origin interpretations of the duality established in Theorem 8.1 [stated in words at (1.2) and (1.2°)] can now be formulated as follows: P(0uniform point of S(Z, S) G •) = P°((Z°, S°) G •), (9-1) P° (Uniform time in R(Z°, S°) G •) = P((Z, S) G •)• (9-1°) This of course does not have an immediate meaning, because such uniform random variables do not exist. In this section we present two results motivating (9.1) and (9.1°). Both results are straightforward consequences of the coupling equivalences in Section 7.4 of Chapter 7. 9.1 Cesaro Total Variation Motivation of (9.1) and (9.1°) The following theorem gives a Cesaro total variation meaning to randomized- origin interpretations (9.1) and (9.1°). Theorem 9.1. Suppose the equivalent claims (8.4) and (8.4°) hold. Let A be the Lebesgue measure on (R, B), # denote number of elements in a set, tv and —> denote convergence in total variation. Then as n —> oo, -r±- £ P(eSk(Z,S) G •) 4 P°((Z°,S°) G •) (9.2) W*n keBn for all integer subsets BnCZ,0<n<oo, satisfying 0 < #i?ra < oo and such that for all k G Z, #((& + Bn) fl Bn)/#Bn —> 1, n —> oo, [F0lner averaging sets}. Examples of such sets are Bn = {—n,..., n} and Bn = {0,..., n}. Conversely, as h —» oo; tt^t / P°(9,(Z0,s°)e-)d8tAp((z,S)e-) (9.2°) -H-D/iJ JBh for all Borel sets Bh G B, 0 < h < oo, satisfying 0 < X(Bh) < oo and such that for all t G E, X((t + i?h) n Bh)/X{Bh) -» 1, ft -> oo, [F0lner averaging sets}. Examples of such sets are Bh = [—h, h} and Bh = [0, ft] and, more generally, Bh = hB where B is any Borel set such that 0 < \(B) < oo.
Section 9. Interpretation - Cesaro Limits and Shift-Coupling 289 Comment. The Cesaro results (9.2) and (9.2°) can be rewritten in the following randomized-origin form. With Kn uniform on Bn and independent of (Z, S) under P we have J>(9sKJZ,S)e-)t4P°((Z°,S°)e-), n^<x>. (9.3) With Vh uniform on Bh and independent of (Z, S) under P° we have P°(0vh(Z°,S°)e-)^P((£,S)e-), a-k». (9.3°) Proof. Note that (8.3a) in Lemma 8.1 can be written as P((Z, S) G •) = P°((Z°, S°) G •) on I. (9.4) In order to establish (9.2) let rk, k G Z, be the point shifts defined at (7.10) and note that according to Theorem 7.2, I = {B G U ® C : t~1B = B for k G Z}. Apply the results in Section 7.4 of Chapter 7 with Y having the distribution P((Z, S) £ •), Y' having the distribution P°((Z°, S°) G •), (9.5) G = {rk :k G Z}. Then (9.4) is the condition (c) in Section 7.4 of Chapter 7, and we obtain (9.2) from the final display in that subsection. In order to establish (9.2°) apply the results in Section 7.4 of Chapter 7 with Y having the distribution P°((Z°,S°) G •), Y' having the distribution P((Z, S) G •), (9.5°) G = {9t : t G R}. Then (9.4) is the condition (c) in Section 7.4 of Chapter 7, and we obtain (9.2°) from the final display in that subsection. For the fact that Bh = hB are F0lner averaging sets, see Theorem 2.1 in Chapter 7. □ 9.2 Shift-Coupling Motivation of (9.1) and (9.1°) The following shift-coupling result gives a surprisingly strong motivation of (9.1) and (9.1°): the two processes can be represented as a single process with different origins.
290 Chapter 8. STATIONARITY, THE PALM DUALITIES Theorem 9.2. Suppose the equivalent claims (8.4) and (8.4°) hold. Then the probability space (0, T, P) can be extended to support a random integer K such that P(dsK (Z, S) G •) = P°((Z°, S°) G •)• (9.6) Conversely, the probability space (0, T, P°) can be extended to support a random time T such that P°{dT{Z°,S°) G •) = P((£, S) G •)• (9-6°) Proof. In order to establish (9.6) apply the results in Section 7.4 of Chapter 7 with Y,Y' and G as at (9.5). Then (9.4) implies (c) in Section 7.4 of Chapter 7, and we obtain (9.6) from (a') in that subsection. In order to establish (9.6°) apply the results in Section 7.4 of Chapter 7 with Y,Y' and G as at (9.5°). Then (9.4) implies (c) in Section 7.4 of Chapter 7, and we obtain (9.6°) from (a') in that subsection. □ 10 Comments on the Two Palm Dualities We end this chapter with a few comments on the relation between the point-at-zero duality of Theorem 4.1 and the randomized-origin duality of Theorem 8.1. 10.1 When Do the Two Dualities Coincide? Under what conditions is it true that standing at the origin of a stationary point-stream, and happening to find a point there, is equivalent to standing at a point selected uniformly at random from the point-stream? That is, when do the two Palm dualities coincide? We shall now specify the exact condition. In order to distinguish between the two dualities let Pj and P° be two probability measures on (0, T) linked as in Theorem 4.1: E1[l/X0]<oo and dP° = -L^L-jdP,, (10.1) or equivalently, E°[X0]<<X) and dPi = -^-dP°; (10.1°) and let P2 and P£ be two probability measures on (fi,.?7) linked as in Theorem 8.1: E2[1/X0|J] < 00 and dP° = ^^"^dPa, (10.2)
Section 10. Comments on the Two Palm Dualities 291 or equivalently, E2°[X0|J]<<X) and dP2 = *° dP2°. (10.2°) t^2[X0\J\ The two Palm dualities coincide when p1=p2 o P°=P2. Consequently, if (Z, S) is stationary under a probability measure P and E[l/X0] < oo, then the two cycle-stationary duals coincide if and only if E[1/X0|J] = E[l/X0] a.s. P. (10.3) Conversely, if (Z°,S°) is cycle-stationary under a probability measure P° and E°[Xo] < oo, then the two stationary duals coincide if and only if E°[X0|J]=E°[Xo] a.s. P°. (10.3°) Note that the dualities coincide in the ergodic case, that is, when P(A) = 0 or 1 for A <= J [J is trivial under P] and, equivalently, P°{A) = 0 or 1 for A <= J [J is trivial under P°], Note also that the exact coincidence conditions (10.3) and (10.3°) are weaker than ergodicity, that is, may hold even in a nonergodic case. 10.2 Can the Two Dualities Differ? Here is a simple example showing that the two dualities can differ. Let Y be a strictly positive random variable supported by a probability space (fi,J",P). Define (Z°,S°) by Z°t = 0, t € M, and S° = nY, n G Z, that is, Z° is a nonrandom constant, S° is a lattice with a random span Y, and ■•• = X-l=Xo=X1=--- = Y. Let U be uniform on (0,1] and independent of Y under P. Put (z,s) = e.{1.u)Xo(z°,s°). Then (Z,S) is stationary under P. (Note also that the pair (Z°,S°) is cycle-stationary under P, in fact, under any measure.)
292 Chapter 8. STATIONARITY, THE PALM DUALITIES The cycle-stationary dual according to the point-at-zero duality exists if E[l/y] < oo. The length-debiasing of P then yields the measure dpo 1/y dP This results in a length-debiasing of the span Y of the random lattice 5°. The cycle-stationary dual according to the randomized-origin duality is even simpler. Note that the random span Y of the lattice S° is the same measurable function of #t(Z, S) for all t £ E, namely, Y is the length of the interval straddling t. This means that Y € J+, and thus E[1/Y\J] = 1/Y. Thus the length-debiasing conditionally on J does not change P. Thus the cycle-stationary dual according to the randomized-origin duality is simply (Z°, S°) under P itself. Since P and P° differ, the two dualities differ. The interpretation of the point-at-zero duality says the following in this case. If you observe a stationary random lattice from the origin and happen to find a point there, then the span of the random lattice shrinks. This is not too strange, because a stationary lattice is more likely to have a point close to the origin when the random span happens to be short. The interpretation of the randomized-origin duality, on the other hand, says the following. If you observe a stationary random lattice from a uniformly chosen point, then the span of the random lattice remains the same. This is not strange at all, because, obviously, choosing some point to view the lattice from will not alter its span. Remark 10.1. If we take Y such that E[l/F] = oo, then (Z,S) under P is an example of a stationary pair with infinite intensity and thus no cycle-stationary point-at-zero dual. 10.3 Random Time Change Hides the Gap Between the Dualities We can make the two Palm dualities coincide by a simple random time change. Let (Z, 5) be stationary under a probability measure P such that E[l/Xo|»7] < oo. Let Z be measurable under change of time scale and change the time scale by R := E[1/X0|J] to obtain {(Zs/r)s€R,RS). This new pair ((Zs/fi)seR, RS) is stationary, and the length of the cycle straddling the origin is RX$. Since R is J measurable, we have E[1/(RX0)\J] = E[1/X0\J]/R = 1 [thus E[1/(RX0)\ = 1].
Section 10. Comments on the Two Palm Dualities 293 The invariant cr-algebra of ({Zs/n)se^, RS) is contained in J, and thus E[l/(RX0)\((Zs/R)seU,RS)-lI\ = 1. Since also E[1/(RX0)] = 1, the coincidence condition holds [see (10.3)]. Consequently, the two cycle-stationary Palm duals of ((Zs/^)sGr, RS) coincide, that is, ((Zs/r)s£u,RS) has only one cycle-stationary Palm dual. Note that 1/(RX0) = 1 / (X0E[1 / X0\J}), and thus the change of measure used to obtain this common cycle-stationary dual of ((Zs/R)seu,RS) is the same as the change of measure used to obtain the cycle-stationary randomized-origin dual of (Z,S). Therefore, this procedure preserves the randomized-origin duality and not the point-at-zero duality. In fact, we lose the point-at-zero duality by this procedure: the point-at- zero duality merges with the randomized-origin duality by the time change and does not reappear when we return to the original time scale after changing the measure (as the randomized-origin duality does). Thus the time change is not a way to bridge the gap between the two dualities; it only hides it. To bridge the gap we cannot avoid a change of measure: with Pj and P2 the two dual measures of P denned at (10.1) and (10.2) with Pj = P2 = P we have (provided E[l/X0] < 00) E[1/X0|J] E[l/X0] dPl - E[i/xo] dP2 and dpi = W7w]dPl- There is an important distinction between the two Palm dualities. Using the first when the second is appropriate (for instance when averaging over the points) can lead to wrong results. 10.4 On Marked Point Processes The sequence of times S = (S/O^oo is sometimes called a simple point process. If to each point Sn, there is associated a random element Yn then the joint sequence (5^,1^)^ is a marked point process and Yn is the mark of the point Sn. In this chapter we have considered S in association with a stochastic process Z — (Zs)sGr. This is equivalent to considering a marked point process in the following sense. When (Z, S) is given, we could define the mark of the point Sn to be Yn :— 9s„Z- Conversely, when a marked point process (S/t, Y/i)^ is given, we could define Z by letting Zs be the marked point process with origin shifted to s. A similar comment applies in the next chapter.
o
Chapter 9 THE PALM DUALITIES IN HIGHER DIMENSIONS 1 Introduction In the previous chapter we considered stochastic processes split into cycles by a sequence of random times (called points) and established two Palm dualities between stationary processes and cycle-stationary processes. We shall now extend this theory to d > 1 dimensions: to random fields 'punctuated' by a countable set of isolated points scattered over Rd in some random manner (a simple point process). This extension is basically straightforward using so-called Voronoi cells instead of intervals, that is, associating to each point the set of sites that are closer to that point than to any other point. There is, however, one major complication, namely the apparent lack of a higher-dimensional analogue of cycle-stationarity. There are no cycles in higher dimensions, so what does cycle-stationarity mean there? In one dimension cycle-stationarity means that the cycles of the process form a stationary sequence. This definition can be rephrased as point- stationarity: the behaviour relative to a given point is independent of the point selected as origin; the process looks the same from all the points. Note that point-stationarity is different from stationarity: stationarity means that the behaviour of the process relative to any given nonrandom time is independent of the time selected as origin; the process looks the same from all nonrandom times. Point-stationarity, the property that the process looks the same from all the points, should make sense also in higher dimensions. But what does it mean, exactly? How should point-stationarity be formally defined when 295
296 Chapter 9. THE PALM DUALITIES IN HIGHER DIMENSIONS d > 1? The answer to this question needs some motivation. Since the point- stationarity problem is what really separates the higher-dimensional case from the one-dimensional case, we shall highlight it in the structure of the chapter. The point-stationarity problem is presented in Section 2 and solved in Section 3. After defining point-stationarity in Section 3, we further characterize the concept in Sections 4, 5, and 6. These characterizations are then used to extend to d > 1 dimensions the theory of the previous chapter: the point-at-zero duality is presented in Section 7 and the randomized-origin duality in Section 8. Section 9 concludes with comments on the two Palm dualities and on possible extensions of the point-stationarity concept, for instance to the zero set of Brownian motion. 2 The Point-Stationarity Problem This section explains the point-stationarity problem in full detail, moving from the obvious one-dimensional case to the not-so-obvious higher- dimensional case. Necessary notation is introduced along the way. In order to highlight the problem we consider it first in the context of simple point processes only, not introducing the associated random field until the next section. 2.1 The Simple Point Processes N and N° Intuitively, a simple point process in d dimensions (d ^ 1) is a countable set of isolated points scattered over the d-dimensional Euclidean space M.d in some random manner (like planets scattered over space). In the one- dimensional case in Chapter 8 this random set of points was written as an increasing sequence of random times. There is no natural analogue of this procedure in higher dimensions. Instead, we shall represent the random set of points (in the standard way) by a collection of random variables iV = (N{B) : B € Bd), where Bd are the Borel subsets of Rd and N(B) = the number of points in the set B. More precisely, a simple point process in d dimensions is a random element N in the measurable space (M, M), where M is the set of all simple counting measures and M. is the product a-algebra on M, that is, M = set of integer-valued measures // on (Rd,Bd) with fi(B) < oo, for all bounded B e Bd, and //({£}) = 0 or 1, for all t e Rd,
Section 2. The Point-Stationarity Problem 297 and M — smallest cr-algebra such that the projection from M to [0, oo] taking // to fi(B) is M/B([0, oo]) measurable for each B e Bd. We shall write iV0 to indicate that one of the points is placed at 0 (the origin of Rd), that is, N°({0}) = 1. We shall regard iV° as a random element in (M°,M°), where M° is the subset of M containing the simple counting measures having mass one at the origin, M° = {/x € M : /x({0}) = 1}, and M° is the trace of M° on M, M° = Mf)M°. Let (il,J-) be the measurable space supporting N and iV0 and let P and P° be two probability measures on (ft, J7). In this section, and the next, we shall not postulate any link between N and iV0, nor between P and P°. We shall think of N as governed by P and iV° as governed by P°. 2.2 Sites and Points Call a nonrandom element t of Rd a site (and not a point) to distinguish it from the points of the point process. Call a random element T in (Ed, Bd) a random site. Call a random site H a point or a random point of iV only if the point process iV has a point at II, that is, only if N({n}) = 1 [short for N({II(lj)})(lj) = 1, w € ft ]. Similarly, call a random site 11° a point or a random point of iV0 only if the point process iV° has a point at 11°, that is, only if N°({n°}) = l. A point process typically does not look the same seen from a nonrandom observation site as seen from an observation point (the universe does not look the same seen from space as seen from a planet). 2.3 Site-Shifts For (i e M, let s(/x) denote the set of //-points (the point pattern): s(fJ-) ={pef: fJ-({p}) ~ 1} = the support of [i.
298 Chapter 9. THE PALM DUALITIES IN HIGHER DIMENSIONS For t eRd, define the shift or site-shift 0t taking // G M to 6tfi G M by Otn(B)= n(t + B), B&B-, [hevet + B = {t + s:s € B}} In order to work with expressions like iV({T}) and #xiV we need the following joint measurability or shift-measurability result. Theorem 2.1. For each B G Bd, the mapping taking (fi,t) G M x Rd to [i(t + B) G [0, oo] is M <& Bd/B([0,oo]) measurable. Equivalently, the mapping taking (fi, t) G M x Rd to 9tfi G M is M ® Bd/M measurable. Proof. The equivalence follows from the definition of M. We shall prove the former claim, namely, the M ® Bd/B([0,oo}) measurability of Consider first B = [a, 6) where a = (ai,..., aj) and b = (&i,..., bd) are in Rd and a,\ < bi,..., aj < bj and [a,b) = [ai, &i) x • • • x [a^, &d) • Take a real number h > 0 and put [t\h = sup{s G hZd : s ^t}. Define 9h ■ (M) >-> n{[t]h + [a,b)) and note that (//, t) >-» (//, [*]h) is AI <8> Bd/AI (8) B(/«Zd) measurable, (//, r) >-> /x(r + [a, 6)) is .M (8) B(hZd)/B([0, oo]) measurable, and that gh is the composition of these two mappings. Thus gh is M. (8> $d/$([0, oo]) measurable. Now, <?/! goes to /[Qi6) pointwise as ii | 0. Hence /[Qib) is M (8> Bd/B([0, oo]) measurable. Hence so is /b for all B in the algebra generated by sets of the form [a,b). Thus the class of all B G Bd such that /b is M (8> Bd/B([0, oo]) measurable contains an algebra generating Bd. Moreover, this class is monotone [since if Bn increases/decreases to B as n —>• oo, then /en increases/decreases to fs pointwise as n —>• oo, and thus if fsn is M ® Bd/B([0,oo]) measurable, then so is /#]. This and the monotone class theorem (see Ash (1972), Theorem 1.3.9) imply that the class coincides with Bd. Thus fs is M (8> Bd/B([0, oo]) measurable for each B &Bd. D 2.4 The Point-Stationarity Problem for N° The point process N is stationary if N = N, (ef, [ = denotes identity in distribution]. Note that #( shifts the point pattern by —t, that is, 6t shifts the origin (the observation site) to t. Thus stationarity of the point process N means that it looks the same from all nonrandom observation sites.
Section 2. The Point-Stationarity Problem 299 Similarly, it would be natural to say that the point process iV° is point- stationary if it looks the same from all observation points. What this means is not clear, except in one dimension. When d = 1, point-stationarity means that iV0 is interval-stationary, that is, the intervals between the points form a stationary sequence: (^"«+fc)^°oo = (^"fc)^°oo) n £ Z; here (as in Chapter 8) X% = S% — S£_1; where • • ■ < 5°2 < S°_ 1 < 50° = 0 < S° < 52° < • • • are the points of iV° written as an increasing sequence. This definition of point-stationarity when d = 1 can be rewritten as es°N°=N°, n€Z. In other words, if the observer moves from the point at the origin to the nth point to the right of the origin (or left of the origin), then the probability distribution of the point pattern that he sees does not change: the point process N° looks the same from all observation points. When d > 1, this definition of point-stationarity does not work, since then there are no intervals between the points of iV° to form a stationary sequence, and the observer cannot use the simple point selection rule move- from-the-point-at-the-origin-to-the-nth-point-to-the-right (or -to-the-left), since there is typically no nth point to the right (left). But is there some similar way of moving between points in higher dimensions? 2.5 The Point-Stationarity Problem in the Poisson Case Note that even the Poisson process seems to present a problem. A point process N is a Poisson process (with constant intensity) if the number of points in disjoint Borel sets form independent random variables and the expected number of points in each Borel set is proportional to the Lebesgue measure of the set. This can be thought of as saying that the points are scattered around completely at random. If we define iV0 by adding a point at the origin to N, N° := N + S0, where S0 is the measure with mass one at 0, (2.1) then it is intuitively reasonable that all the points of iV° are equivalent as observation points (you are standing at one of them, and the others are scattered around completely at random). When d = 1, this is indeed the case: it is well known that the intervals between points are i.i.d. exponential, and thus iV° is point-stationary. But when d > 1 (in the plane, for instance), how can we shift the origin to another point of N° without spoiling the distribution of the point pattern that we see?
300 Chapter 9. THE PALM DUALITIES IN HIGHER DIMENSIONS "Why not shift the origin to the closest point?" is a natural first reaction to this question. In order to indicate why this does not work let us consider the following example. Example 2.1. Consider the case when d = 1 and let iV° be the Poisson process with a point at the origin defined at (2.1). We know that iV0 is point-stationary, so if shifting to the closest point is to be the way to shift in higher dimensions, it ought to work in one dimension also, that is, it should not change the distribution of JV°. Shift the origin of iV° to the closest point to obtain 9n° N°, where closest j^ if5°>-5°j. Then we are sure to see, either to the right or to the left of the new origin, an interval followed by a longer interval. This is definitely not a property of iV°, which in both directions from the origin has an exponential interval followed by an independent exponential interval. Thus @n° N° does not have the same distribution as iV0, closest ' that is, shifting the origin to the closest point does not preserve the distribution of this point-stationary iV°. 2.6 Point-Maps and Point-Shifts Example 2.1 illustrates that the selection of a point to shift the origin to is the key issue in defining point-stationarity. We cannot select points in any old way. In order to discuss this problem we need the following terminology. Call an M°/Bd measurable mapping it from M° to Rd an N°-point-map if it selects a point, that is, if MW/*)}) = i, c^°. Call the mapping 9n from M° to M° defined by QnV '■= #7r(M)M, M G M°, an N°-point-shift. Note that 8^ shifts the origin from a point to a point. Call 9n the point-shift associated with n. It follows from Theorem 2.1 that 9n is M°IM° measurable [since 9V, seen as an M° valued mapping from M° to M, is the composition of the M°/M ® Bd measurable mapping taking (i € M° to (fj,,ir(fj)) G M x Rd and the M ® Bd/M measurable mapping taking (fi,t) G M x Rd to 9tfi G M\.
Section 2. The Point-Stationarity Problem 301 When d = 1, examples of iV°-point-maps are the irn defined for // € M° as follows nth //-point to the right of 0, n > 0, ?r„(/i) = < 0, n = 0, —nth //-point to the left of 0, n < 0. The associated iV°-point-shifts translate the origin to the nth point to the right or nth to the left. Note that the random points 5° in Section 2.4 can be written as 5° = 7rn(JV°). When d ^ 1, an example of an iV°-point-map is the shift to the closest point, 7rciosest, defined for // £ M° by ^closest (//) = the //-point having the lexicographically highest order among the nonzero //-points being at shortest distance from the origin. The lexicographic rule is just to make sure that 7rciosest (//) is uniquely defined. The random point in Example 2.1 can be written as n°losest = 7I"closest(-/V°)- 2.7 What Is Wrong with Shifting to the Closest Point? In Example 2.1, shifting the origin to the closest point changed the distribution of an interval-stationary (that is, point-stationary) JV°. So what is wrong with this point-shift? In order to answer this question let us first consider another one: when d = 1, what is so special about shifting the origin of iV° to the nth point to the right? The essential property of this point-shift is the following. By knowing the point-selection rule (select-the-nth-point-to-the-ri<?/i£) and by looking at the point pattern from the new origin, you can always tell from what point you came and shift the origin back to the nth point to the left of the new origin: we first shift the origin of // to 7rn(//) to obtain 6Vnn and then the origin of #,,•„// to n-n{Q*n(j) = -Tn(^) to obtain Thus // is the only element of M° that 6-Kn shifts to //' := 9nnfi. Also, any element //' of M° can arise from this point-shift, since taking // := 9n_n[i' yields
302 Chapter 9. THE PALM DUALITIES IN HIGHER DIMENSIONS Thus, shifting the origin to the nth point to the right is a bijective point- shift. When d = 1, it can be shown (see Remark 3.1 below) that an interval- stationary iV° is distributionally invariant under the group of all bijective iV°-point-shifts. We can now guess the answer to the first question: shifting the origin to the closest point is wrong because this point-shift is not bijective. There can be more than one point of a point pattern having a particular point p as the closest point. 2.8 Bijective Point-Shifts? Bijective point-shifts are natural to apply in defining point-stationarity, since under a bijective point-shift 9V all points are equally important, or get equal attention, in the following sense: with // £ M° fixed, the mapping is a bijection from s(/x) to s(/x), that is, under this mapping each point is the image of a unique point. All this suggests that we should define point-stationarity in higher dimensions by requiring that iV° be distributionally invariant under bijective iV°-point-shifts. For this definition to make sense we must find some bijective point-shifts. A simple example is the following (devised by Olle Haggstrom): shift the origin to the closest point if that point has the point at the origin as its closest point; otherwise, stay where you are. But as this is being written it is still not known whether the class of bijective iV°-point-shifts is rich enough to characterize point-stationarity appropriately when d > 1 (appropriately in the sense that the Palm dualities hold). It is known, however, that the class of bijective point-shifts can at least be made rich enough to define point-stationarity when d > 1. The trick is to consider iV° against an independent stationary background. We explain what this means in the next section. 3 Definition of Point-Stationarity In order to highlight the point-stationarity problem we have up to now suppressed that the point process iV° will be regarded in association with a random field Z°. Considering iV° jointly with Z° does not solve our problem but is conceptually a step in the right direction.
Section 3. Definition of Point-Stationarity 303 3.1 The Associated Random Fields Z and Z° We shall from now on consider a pair (N, Z), where JV is a simple counting process and Z = (■^s)s€R'f is a random field with an arbitrary state space (E,£) and path space (H,H), where H is a shift-invariant subset of ER and H is the cr-algebra on H generated by the projection mappings taking z — (zs)seR<f in H to zt in E, te Rd. In order to be able to apply random shifts, we need the minimal regularity condition [satisfied in the standard settings, for instance when (E,£) is Polish and the paths right-continuous; see Section 2 in Chapter 4] that Z is canonically jointly measurable, that is, the mapping from H x Rd to E taking (z,t) to zt is H ® Bd/£ measurable. For t e Rd, let 9t denote the shift or site-shift from H to H defined by 9tz = (zt+s)send, z £ H. Canonical joint measurability is equivalent to the mapping from H xRd to H taking (z,t) to 9tz being H ®Bdj'H. measurable (shift-measurability). We shall not assume any functional connection between N and Z. At one extreme, N could be determined by Z. For instance, when d = 1, the points could be the times when a stochastic process enters a given state. At the other extreme, Z could be identically constant, which boils down to regarding N alone (not in association with any random field) as we did in Section 2. When we consider a random field in association with iV° we shall denote it by Z°. Thus the ° on Z° is not to indicate a property of Z°. It only indicates that Z° is considered jointly with iV° (while as before, the ° on iV0 is to indicate that iV° has a point at the origin). 3.2 Extended Point-Maps and Point-Shifts Let 9t denote the shift or site-shift from M x H to M x H defined, for ieEd, by 9t(fi,z) = (9tn,9tz), (n,z)GMxH. Call an M°®n/Bd measurable mapping tt from M° x H to Rd an (N°,Z°)- point-map if it selects a point, that is, if MW/i,z)}) = l, (fi,z)GM°xH. Call the mapping 9n from M° x H to M° x H defined by 9n(v, z) = Sn{^z)(fi, z), {fi, z) e M° x H,
304 Chapter 9. THE PALM DUALITIES IN HIGHER DIMENSIONS an (N°, Z°)-point-shift. Note that 9n shifts the origin from a point to a point. Call 9n the point-shift associated with tv. The M° ® H/M° ® H measurability of 9n follows from Theorem 2.1 and the shift-measurability of Z°. In order to distinguish (iV°, Z°)-point-maps and -shifts from iV0-point- maps and -shifts we shall sometimes call them extended point-maps and extended point-shifts. 3.3 The Point-Stationarity Problem for (N°,Z°) The pair (N, Z) is stationary if it looks the same from all observation sites, that is, 9t(N,Z)^(N,Z), t&W. Similarly (as in the case of iV0 alone), it would be natural to say that the pair (iV°, Z°) is point-stationary if it looks the same from all observation points. When d = 1, point-stationarity means that (N°,Z°) is cycle-stationary, that is, the points of iV0 split Z° into a stationary sequence of cycles. This definition of point-stationarity, when d = 1, can be rewritten as 9S°(N°,Z°)^(N°,Z°), n&Z, and it can be shown that this is equivalent to (N°, Z°) being distribution- ally invariant under the group of all bijective (N°, Z°)-point-shifts (again see Remark 3.1). Under a bijective (iV0, Z'0)-point-shift 9W all points are equally important, or get equal attention, in the same sense as in the iV0-case. Namely, with (//, z) £ M° x H fixed, the mapping p^p + ir(9p(n,z)) is a bijection from s(/x) to s(/x), that is, under this mapping each point is the image of a unique point. This observation again suggests that we should define point-stationarity for d > 1 by requiring that (N°,Z°) be distributionally invariant under bijective (iV°, Z°)-point-shifts. And again, as this is being written, it is not known whether the class of bijective (N°, .^-point-shifts is always rich enough to characterize point-stationarity appropriately (appropriately in the sense that the Palm dualities hold). But we are now closing in on a definition of point-stationarity that works at both the intuitive and formal levels even if the class of bijective (./V°, Z'0)-point-shifts turns out to be too meager in general to characterize the concept.
Section 3. Definition of Point-Stationarity 305 3.4 Intuitive Motivation of the Solution The key trick in our solution of the point-stationarity problem is to consider (N°,Z°) against any independent stationary background, that is, to consider (N°, Z°) jointly with an arbitrary independent stationary (shift- measurable), random field Let (L,£) be the path space of Y°. Note that the ° on Y° (like ° on Z°) is only to indicate that Y° is considered in association with iV0 and does not imply that Y° and iV° are functionally connected (while the ° on the point process iV0 itself indicates that iV° has a point at the origin). Intuitively, if the triple (N°, Z°,Y°) looks the same from all the points of JV°, then so in particular does the pair (N°, Z°). Conversely, if (N°,Z°) looks the same from all the points of iV°, then so will (iV°, Z°, Y°) because [due to the stationarity of Y°] Y° looks the same from all random sites that are independent of Y°, and thus [due to the independence of Y° and (N°, Z°) and the fact that (iV°, Z°) looks the same from all points of N°] the triple (JV°, Z°, Y°) should look the same from all points of N°. That is, (N°, Z°) should be point-stationary if and only if (N°, Z°,Y°) is point-stationary. 3.5 Solution of the Point-Stationarity Problem This suggests that we call the pair (N°,Z°) point-stationary if the triple (N°,Z°,Y°) is distributionally invariant under all bijective (N°,Z°,Y°)- point-shifts for all shift-measurable random fields Y° that are stationary and independent of (iV0, Z°). The above discussion motivates this definition intuitively, while the theory established in the upcoming sections motivates it practically. Here is the definition stated in full detail. Definition 3.1. Let iV° be a simple point process and Z° a random field defined on a probability space (ft, J7, P°). Call (N°,Z°) point-stationary if for each shift-measurable random field Y° that is stationary and independent of (iV0, Z°) and possibly defined on an extension of (fi, T', P°), it holds that 6*770 (N°, Z°, Y°) = (N°,Z°,Y°) ' (3.1) for all random points 11° of the form n° = Tr(N°,Z°,Y°), (3.2) where it is any (JV°, Z°,F°)-point-map [that is, it is an M° ®U® £/Bd measurable mapping from M° x H x L to Rd that selects a point: H({ir(fi, z,y)}) = 1, (/i, z,y) € M° x H x L]
306 Chapter 9. THE PALM DUALITIES IN HIGHER DIMENSIONS such that the associated point-shift [that is, the M.° ®1i® jC/M° ®1-L® jC measurable mapping 0^ from M° x H x L to M° x H x L defined by Mm, z>2/) =07r(M,z,y)(M, z,Z/), (v,z,y) £ M° x H x L] is a bijection. REMARK 3.1. When d = 1, this point-stationarity definition is equivalent to the apparently weaker property of cycle-stationarity, since both properties are equivalent to (4.1) in Theorem 4.1 below. REMARK 3.2. Definition 3.1 is equivalent (see Lemma 4.1 below) to the apparently weaker condition eno(N°,Z°) = (N°,Z°) (3.3) for all random points n° = Kn(N°,Y°), where Y° and k„, n £ Z, are from the family of random fields and point- shifts (indexed by h > 0) defined in the next subsection. 3.6 The Key Example of Extended Bijective Point-Shifts We shall now construct a random field Y° and a two-sided sequence of (Ar°, y°)-point-maps k„, n £ Z, such that the associated (N°, y°)-point- shifts are bijections. Both Y° and the point-maps will depend on a fixed constant h > 0. Thus we are really defining a family of random fields and point-shifts indexed by h > 0, although we suppress the parameter h in the notation. We start by constructing Y°, which will simply represent the stationary point pattern KLd — hU, where U is uniformly distributed on [—|, |)d. Let b = (bs)seud be the fixed function from Rd to Rd defined by bs = the vector from the site s to the closest element of /iZ . If there are more than one such /iZd-elements let 6S be the vector to the element of highest lexicographic order (thus bs is right-continuous in all the d coordinates of s). Let Y° have state space (Rd,Bd) and have paths in L = {6tb : t € [-h/2, h/2)d} = {9tb : t £ Rd}. Let U be a random site that is uniformly distributed on [— |, |)d and independent of (iV0,X0) and define Y°:=6uhb (see Figure 3.1)
Section 3. Definition of Point-Stationarity 307 Clearly, Y° is stationary and independent of (N°, Z°), and shift-measurability follows from right-continuity of the paths. Now turn to constructing the bijective (N°, y°)-point-maps k„, n e Z. Fix (fi,y) e M° x L. Call a site t e Rd such that j(=0a y-center and the associated set t + [—h/2,h/2)d a y-box. Note that yo is the center of the y-box containing the origin. Let k be the number of ^-points in that box, k = fi(y0 + [-h/2,h/2)d). Note that k ^ 1, since 0 e s(fi) and y0 e (-h/2,h/2]d. Let p0,... ,Pk-i be the ^-points in s(fi) (1 (yo + [—h/2,h/2)d) ordered lexicographically. Let m denote the index of the /i-point at the origin, that is, pm = 0. For n € Z, put Kn(fi, y) = P(m+n) mod k (see Figure 3.1) where (m + n) mod k = inf(m + n — kZ) D [0, oo). That is, Kn(fi,y) is the nth point after the point at the origin (if n < 0, this is to be interpreted as the —nth point before the point at the origin) in a circular enumeration of the points in the box containing the origin. Thus the point at the origin is the nth point before Kn(fi,y) in that same circular enumeration. Thus 6K_JKn(ii,y) = (ii,y), (n,y)€M°xL. Thus (fi, y) is the only element of M° x L that 9Kn shifts to (fi',y') : = dKn(fi,y). Also, any element (fi',y') of M° x L can arise from 9Kn, since taking (fi, y) := 0K_„ (fj,',y') yields 9Kn (fi, y) = {fi',y'). Thus 6Kn is bijective. The box containing the point at the origin 2 I I r i Center of the box \___/ containing the site s FIGURE 3.1. Definition of Y° and /t„ (d = 2 and A: = 5). Origin placed uniformly at random in the box ""
308 Chapter 9. THE PALM DUALITIES IN HIGHER DIMENSIONS 4 Palm Characterization of Point-Stationarity In this section we establish a pleasant characterization of point-stationarity in terms of nonrandom site shifts. We shall call it Palm characterization because it is the key to the Palm dualities presented in Sections 7 and 8 below. At the end of the section we state a dual Palm characterization of stationarity. This is analogous to the equivalence of (a) and (d) in Theorem 3.1 of Chapter 8. We start by introducing an important concept, so-called Voronoi cells, which play in some respect the same role in higher dimensions as the intervals between points in one dimension. 4.1 Voronoi Cells Consider a point pattern in Rd represented by a simple counting measure fi £ M. Note that the point pattern s(fi) need not have a point at the origin. To each point p £ s(fi) associate the set of sites t £ Rd that are strictly closer to p than to any other point. This set is the open Voronoi cell with point p. Define the Voronoi cells themselves by extending the open Voronoi cells in such a way that a site t £ Rd that is at equal minimal distance to two or more points belongs to the cell with point p having the highest lexicographic order. Thus to each point p £ s(fi) there is associated a Voronoi cell. These Voronoi cells are finitely or countably many, they are disjoint, and their union is Rd. In other words, the Voronoi cells form a finite or countable partition of Rd. [Note that when d = 1, the Voronoi cells are intervals with the points in the interior, while in Chapter 8 we considered intervals with the points at the ends. This means that in the one-dimensional case we will now arrive at results in a slightly different way from that of Chapter 8.] In what follows, the Voronoi cell containing the origin is of key importance. For N put Co = the Voronoi cell of N that contains the origin and iTo = the iV-point of Co- For iV° put Cq = the Voronoi cell of iV° that contains the point at the origin. 4.2 Shifting the Origin to and from iT0 Let S be a Cg valued random site. From now on, throughout this chapter, (N,Z) and ((N°,Z°),S) will be functionally linked as follows (see Fig-
Section 4. Palm Characterization of Point-Stationarity 309 ure 4.1). When (N, Z) is given, define (N°, Z°) := 9no(N, Z) [this implies that 11% = 0], s -.= -n0. Conversely, when ((Ar°, Z°),S) is given, define (N, Z) := 9S(N°, Z°) [this implies that iT0 = -S]. Thus ((N°,Z°),S) is ((N,Z),0) seen from iT0, (N, Z) is (N°, Z°) seen from S. Under this one-to-one correspondence between (N, Z) and ((N°, Z°),S) we have Cq = Co — iTo, or equivalently, Co = Cq — S. In particular, the Voronoi cells Co and Cq have the same volume: A(C0°) = A(C0), where A is Lebesgue measure on (Ed, Bd). FIGURE 4.1. The Voronoi cell containing the origin.
310 Chapter 9. THE PALM DUALITIES IN HIGHER DIMENSIONS 4.3 The Palm Characterization of Point-Stationarity We shall now show that point-stationarity of the pair (iV0, Z°) means that if we shift the origin to a site selected uniformly at random in Cq, then the pair will look the same from all observation sites, provided that we volume-bias by the volume of Cq. This is analogous to the equivalence of (a) and (d) in Theorem 3.1 in Chapter 8, except that we now highlight point-stationarity rather than stationarity. Theorem 4.1. Let (N°,Z°), S, and (N,Z) be supported by a probability space (Q,J-,P°) and linked as in Section ^.2. Suppose A(Co) is finite with probability one. Then (N°, Z°) is point-stationary and the conditional distribution of S given (N°, Z°) is uniform on Cq if and only if E°[\(C0)f(et(N, Z))\ = E°[X(C0)f(N, Z)\ (4.1) for all f £M®H+ and t£Rd. We prove this theorem in the next three subsections. 4.4 First Step in Proof of Theorem 4.1 - The 'Only-If' Part Assume that (iV°, Z°) is point-stationary and that the conditional distribution of S given (iV0, Z°) is uniform on Cq. We shall show that this implies that (4.1) holds. Take t € Rd and / e M ®U+ and use the conditional uniformity [and X(Cq) = X(Cq)] to obtain the second equality in E°[\(C0)f(9t(N, Z))\ = E°[\(C0)f(9t+s(N°, Z°))] f 1 (4-2) / f(9s(N°,Z°))ds . Let Kn and Y° be as in Section 3.6 and put = EC Cn = the Voronoi cell of N° associated with the point Kn(N°,Y°), K = N°(Yo° + [-h/2,h/2)d) = number of iV0-points in the y°-box containing the origin, A = union of C" over 0 ^ n < K = union of cells with points in the y°-box containing the origin.
Section 4. Palm Characterization of Point-Stationarity 311 Note that the number of iV°-points in the box containing the origin remains the same (since the point-map Kn selects a point in that box) after the shift by 6Kn, namely K. Thus =*} / by the same function in M° ® % <8> C+ as 1{K=A} / f(9,(N°, Z°)) ds is obtained from (N°, Z°, Y° J(t f(9s(N°,Z°))ds from 9Kn(N°,Z°,Y C+c-»)nc; Due to point-stationarity, (N°, Z°, Y°) and 9Kn (N°, Z°,Y°) have the same distribution. Thus E° {K=k} j J a (<+C°)nO f(es(N°,z°))ds = E°\l{K=k} [ f(es(N°,Z°))ds\. L J(t+C-»)ncs Recall that A is the union of Cn over 0 ^ n < K and note that A is also the union of C~n over 0 ^ n < K. Thus summing first over 0 ^ n < k and then over 1 ^ k < oo yields lJ(t- E°| / f(6s(N°,Z°))ds\ =E°f / l(t+cs)r\A J LJf* (t+A)nCS f(9s(N°,Z°))ds Now, A in fact depends on the parameter h and expands to the union of all the Voronoi cells as h —>• oo. Thus both A and t + A expand to Rd as h —► oo. Thus by monotone convergence, E° / /(0s(iv°,;n)d5 = E° f f(9s(N°,Z°))ds Combine this with (4.2) to obtain that E°[X(C0)f(9t(N, Z))} does not depend on t, that is, (4.1) holds. Thus the proof of the 'only-if part of Theorem 4.1 is complete. 4.5 Mid-Step in Proof of Theorem 4.1 - Preparing for the 'If The following result is needed in the proof of the 'if part of Theorem 4.1. [By some additional effort this theorem can be strengthened to become an equivalence result, similar to (6) and (c) in Theorem 3.1 of Chapter 8, but we shall not do so here, since we do not need it for the Palm dualities.]
312 Chapter 9. THE PALM DUALITIES IN HIGHER DIMENSIONS Theorem 4.2. Let (N°, Z°), S, and (N, Z) be supported by the probability space (ft, T, P°) and linked as in Section 1^.2. For t G Rd, put Ct = the Voronoi cell of N containing the site t, LTt = the N-point of Ct- Let Y be a stationary shift-measurable random field that is independent of (N, Z) and possibly supported by an extension of (ft, T, P°). Let (L,C) be the path space of Y and put Y° = 6-sY. Suppose A(Co) is finite with probability one. If (4.1) holds, then for all f G M. ® H ® C+, E°l\(Co)f(et(N,Z,Y))]=E°[\(Co)f(N,Z,Y)}, t£Rd, (4.3) A(Co) / l{77.e[o,i)'} f{et{x{CZt) Y)) dt\ = E°[/(iV' Z'Y)]' (4'4) E E° A(C0) J2 f(eP(N,Z,Y))] =E°[f(N°,Z°,Y°)}, (4.5) pes(N)n[o,i)d and the conditional distribution of S given (N°, Z°,Y°) is uniform on Cq. Proof. Assume that (4.1) holds. Since Y is stationary and independent of (N,Z) we obtain from (4.1) that (4.3) holds for all / = /i/2, where /i€M®H+ and f2 G C+, since then E°[\(C0)f(9t(N,Z,Y))] = E°[\(C0)fl(9t(N,Z))}E°[f2(etY)} = E°[\(Co)fi(N,Z)]E°[f2(Y)} =E°[\{CQ)f(N,Z,Y))]. Thus (4.3) holds for all / G M ® U ® £+. To obtain (4.4), apply (4.3) with f(N, Z, Y) replaced by f(N, Z, Y)/X(C0) and f(0t(N,Z,Y)) by f(9t(N,Z,Y))/X(Ct) to get E°[\{C0)M(N, Z, Y))/X(Ct)} = E°[f(N, Z, Y)]. Integrating over t G [0, l)d and interchanging integration and expectation yields E°[a(C0)/ f{pt(N,Z,Y))l\{Ct)dt\=E°[f{N,Z,Y)\. (4.6) L Jte[o,i)d J Take i G Zd and note that / l{Ijtei+loAy}f(et(N,Z,Y))/X(Ct)dt Jte[o,i)d
Section 4. Palm Characterization of Point-Stationarity 313 is obtained from (iV, Z, Y) by the same function in M ® H ® £+ as / l{77«e[o,i)<}/(WZ,K))/A(Ct)di ■/te-t+[o,i)d from #_j(iV, Z, F). Apply (4.3) [with / replaced by this function and t replaced by —i] to obtain E°[a(C0) / l{ntei+[0Ay}f(et(N,Z,Y))/X(Ct)dt L Jte[o,i)d J = E°[a(C0) f l{ntelo,i)'}M{N,Z,Y))/\(Ct)dt L Jte-i+[o,i)d Sum over i £ Zd to obtain E A(Co) / f(9t(N,Z,Y))/X(Ct)dt Jte[o,i)d A(Co) / l{77,€[o,i)-}/(»tW^y))/A(Ct)dt JteRd This and (4.6) yield (4.4). In order to establish (4.5), take g e Bd+ and f £ M®U®C+ and note that teC ^ if(0nt(N,Z,Y))/\(Ct) = f(0p(N,Z,Y))/\{Cp), p \g(t-nt)=g(t-p). Apply this and (4.4), with f(N,Z,Y) replaced by f(N°, Z°,Y°)g(S) and f(6t(N, Z, F)) by f(0„t (N, Z, F))<?(* - nt), to obtain A(Co) Yl f{0P{N,Z,Y)) I g(t-p)dt/\(Cp) Pes(N)n[o,i)d Jtecp ^j^ = E°[f(N°,Z°,Y°)g(S)}. Taking g = 1 yields (4.5). In order to establish the conditional uniformity, replace f(9p(N,Z,Y)) by f(0P(N, Z, F)) JteC g(t - p) dt/X(Cp) in (4.5) to obtain E°[a(C0) J2 f(0P(N,Z,Y)) f g(t-p)dt/\(Cp) = E° \f(N°, Z°,Y°) [ g(t) dt/\(Co)].
314 Chapter 9. THE PALM DUALITIES IN HIGHER DIMENSIONS Compare this and (4.7) to obtain that for g G Bd+ and / G M®H® £+, E°[f(N°, Z°, Y°)g(S)} = E° \f(N°, Z°,Y°) [ g(t) dt/\(C0) This means that the conditional distribution of 5 given (N°, Z°,Y°) is uniform on Cq as desired, and the proof of Theorem 4.2 is complete. □ 4.6 Final Step in Proof of Theorem 4.1 - The 'If Part Assume that (4.1) holds. Then, due to Theorem 4.2, the conditional distribution of S given (A7^, Z°, Y°) is uniform on Cq, and thus it only remains to establish that (N°, Z°) is point-stationary. Let Y° and ir be as in Definition 3.1. Assume also that Y° is independent of (N, Z), that is, not only independent of (N°,Z°) but also of ((N°,Z°),S). [This is no restriction, since if this were not the case, we could replace Y° by a random field that is independent of (N, Z) and has the same distribution as Y°: (3.1) holds for Y° if and only if it holds with Y° replaced by this copy of Y°.] Put y = osy° and note that since Y° is stationary and independent of (N,.X), so is Y. Thus, due to Theorem 4.2, (4.5) holds. Fix / G M ® H ® C+ and apply (4.5) with / replaced by f(0■*(■)) to obtain E°[A(Co) £ f(0„Op(N,Z,Y))] Pes{N)n[o,iy (4.8) = E0[f{e*{N°,Z0,Y°))]. Take i G Zd and note that / , 1{p+7r(ep(N,z,y))ei+[o,i)'i}/(^^p(Ar, Z, Y)) Pes(N)n[o,i)d is obtained from (N, Z, Y) by the same function in M ® H ® C+ as / , 1{p+7r(ep(N,z,y))e[o,i)'i}/(^^p(ArI ^> *0) p€»(N)n([0,l)d-t) from 9-i(N, Z,Y). Applying (4.3) [with / replaced by this function and t by —i] yields E°^A(Cb) 22 1{p+n(eT{N,z,Y))ei+[o,i)d}f(0^P(N,Z,Y))^ Pes(N)n[o,i)d = E0|^A(Co) 2^ 1{p+7r(ep(N,z,y))e[o,i)'i}/(^^p(-^'^'^r)) pGs(N)n([0,l)d-i)
Section 4. Palm Characterization of Point-Stationarity 315 Sum over i £ Zd and compare with (4.8) to obtain E°^A(Cb) 22 ^{p+-K(ep(N,z,Y))€[o,i)d}f{^7r0p{N,Z,Y)) pEs(N) = E°[f(e7r(N°,Z°,Y°))]. Since 6V is bijective, it holds that for each point q G s(iV) fl[0, l)d there is a unique point p G s(N) such that q = p+ ir(6p(N, Z, Y)). Applying this on the right-hand side yields [note 07r0p(N, Z, Y) = Op+Tv(ep(N,z,Y)){N, Z, Y)] Ec A(Co) £ f(eq(N,Z,Y))]=E°[f(8AN°,Z°,Y°))}. g6s(N)n[0,l)d Compare this and (4.5) to obtain that E°[f(9n(N°,Z°,Y°))] = E°[f(N°,Z°,Y°)}, f zM®U®C+, holds for Y° and 7r as in Definition 3.1. Thus (N°,Z°) is point-stationary, and the proof of Theorem 4.1 is complete. 4.7 The Backgrounds and Point-Maps in Section 3.6 Suffice The Palm characterization (4.1) was established in Section 4.4 using only distributional invariance under the point-shifts associated with the family of background fields Y° and point-maps k„, n e Z, defined in Section 3.6 (indexed by a parameter h suppressed in the notation). This family thus suffices to characterize point-stationarity. Below (in the proof of Theorem 5.1) we shall need the following slightly modified result. Lemma 4.1. Let V be a [0,1) valued random variable that is independent of (N°, Z°) and the family of random fields Y° in Section 3.6. Let V° be the stationary random field defined by V° = V, t G Rd. The pair (N°,Z°) is point-stationary if and only if 6n°n(N°,Z°) = (N°,Z°) (4.9) for all random points 77° of the form II°n = an(N°,Z°,Y°,V°), where an, n G Z, are the (N°, Z° ,Y° ,V°)-point-maps defined by an(/l, Z, V, v) = K[v0k}+n(H, V) with k = n(yo + [-h/2,h/2)d) and [■] denoting the integer part.
316 Chapter 9. THE PALM DUALITIES IN HIGHER DIMENSIONS PROOF. The 'only-if holds because the condition in Definition 3.1 is satisfied, namely (y0,^0) regarded as a bivariate random field (Y° ,V°)teRd is stationary and independent of (N°, Z°), and 0a„ is bijective for the same reason as 6Kn is: see the end of Section 3.6 and replace the origin in that argument by ao(fi,z,y,v). To establish the 'if part, let 5 be a random site such that the conditional distribution of S given (N°, Z°) is uniform on C£. Replace Y° by (Y°,V°) and Kn by an in the argument in Section 4.4 to obtain that (4.1) holds. By Theorem 4.1 this implies that (N°,Z°) is point-stationary. □ 4.8 Palm Characterization of Stationarity Modifying the proof of Theorem 4.1 in an obvious way yields the following dual Palm characterization of stationarity (the analogue of the equivalence of (a) and (d) in Theorem 3.1 of Chapter 8). Theorem 4.3. Let (N, Z) and ((N°, Z°),S) be supported by the probability space (n,^7, P) and linked as in Section 4-2. Suppose A(Co) < oo with probability one. Then (N, Z) is stationary if and only if the conditional distribution of S given (N°,Z°) is uniform on Cq and E[/(077- (N°, Z°))/X(Co)] = E[/(JV°, Z°)/A(C0)], / €M° ® H+, for all random points LJ° as in Definition 3.1. PROOF. Simply replace E°[-] by E[-/A(C0)] throughout the proof of Theorem 4.1. □ Thus stationarity of the pair (N, Z) means that if we view it from the point of the Voronoi cell where the origin lies, then the origin is located uniformly at random in the cell, and moreover, the pair looks the same from all observation points, provided that we volume-debias by the volume of the cell. The analogue of Theorem 4.2 is obtained by replacing E° [■] by E[-/A(Co)]- Rather than stating that result we let the following suffice. Lemma 4.2. // (N, Z) is stationary, then for all f G M ® 7i+, E[ £ f(ep(N,Z))]=E[f(N°,Z°)/\(C0)}. (4.10) p€s(N)n[0,l)d In particular, E[7V([0,l)d)] = E[l/A(C0)]. (4.11) Proof. In the proof of Theorem 4.2 replace E°[-] by E[-/A(Co)] and leave out Y to obtain (4.10) instead of (4.5). Take / = 1 to obtain (4.11). O
Section 5. Point-Stationarity Characterized by Randomization 317 The expected number of points in a unit box E[iV([0, l)d)] is called the intensity of the stationary point process TV. According to (4.11) the intensity can be calculated by taking the expectation of the reciprocal of the volume of the Voronoi cell containing the origin. 5 Point-Stationarity Characterized by Randomization In this section we shall show that point-stationarity means distributional invariance under doubly randomized point-shifts: shift to a uniformly selected site followed by a shift to a uniformly selected point. 5.1 The Characterization Result A point-stationary (TV°, Z°) turns out to be characterized by the following property: If (TV°, Z°) is first shifted by an independent site U selected uniformly at random in any bounded Borel set B of positive volume and the origin then shifted to a #_i/TV0-point 77 picked uniformly at random among the points s(6-uN°) C\B that ended up in B, then the distribution of (N°,Z°) does not change. There is at least one 9-uN°-point in B, the one at U which initially was at the origin: 0-uN°{{U}) = N°({O}) = l. This characterization is the key to the interpretation of the randomized- origin Palm duality in Section 8. Note that the first randomized shift would not change the distribution of a stationary (TV, Z): if (TV, Z) is stationary, then, since U is independent of (TV, Z), 0-u(N,Z)2(N,Z). The first randomized shift does, however, change the distribution of a point- stationary (N°,Z°), since 0-u{N°,Z°) has no point at the origin, unlike (N°,Z°). But when the first randomized shift is followed by the second, then the distribution is restored: if (TV°, Z°) is point-stationary, then en0-u(No,Z°) = (No,Zo). Observe that 77° := 77 - U is uniform on s(N°) n (B - U), so we can describe this characterization of a point-stationary (N°,Z°) alternatively as follows: If we place any bounded Borel set of positive volume uniformly at random around the TV°-point at the origin and shift the origin to an TV°-point 77° selected uniformly at random among the TV°-points in that set, then the distribution of (N°,Z°) does not change.
318 Chapter 9. THE PALM DUALITIES IN HIGHER DIMENSIONS Here is a more formal statement. Theorem 5.1. Let B £ Bd be bounded and such that X(B) > 0. Let U be a random site that is uniform on B and independent of (N°,Z°). Let LT° be a random point of N° such that the conditional distribution of 11° given ((N°, Z°), U) is uniform on the finite set of points of N° lying in B — U, that is, uniform on s(N°)n(B-U). Then (N°, Z°) is point-stationary if and only if for each such B, {6no (N°, Z°),LT° + U)^ ((N°, Z°), U). (5.1) In fact, (N°, Z°) is point-stationary if (5.1) holds for all B of the form B = [-h/2,h/2)d, h>0. Note that with 77 := 77° + U we have that U = 77 and that the conditional distribution of 77 given ((N°,Z°),U) is uniform on s(N°) n B. Thus it follows from Theorem 5.1 that if we have a point-stationary point process and place a bounded Borel set of positive volume uniformly at random around the point at the origin, then the point at the origin is located uniformly at random among the points in the set. 5.2 First Step in the Proof of Theorem 5.1 — Preparations Let Y° be as in Section 3.6 and choose h large enough for B to be contained in [-h/2,h/2)d. Put B' = [-h/2,h/2)d\B. Let /3n, n £ Z, be the following modification of the point-shifts Kn in Section 3.6: if 0 £ y0 + B, let /3n(fi,y) be the nth point after the point at the origin in the circular lexicographic enumeration of the /i-points in y0 + B, and if 0 £ y0 +B', let Pn((J-,y) be the nth point after the point at the origin in the circular lexicographic enumeration of the /i-points in yo + B'. Then 9/3„ is bijective for the same reason as 0Kn is bijective (see Section 3.6). Note that conditionally on —Yq £ B, the set Y$ + B is the set B placed uniformly at random around the origin (like B — U). In order to select an A^-point placed uniformly at random among the iV°-points in 1^,° + B (like 77° among the Appoints in B — U) proceed as follows. Let V be uniform
Section 5. Point-Stationarity Characterized by Randomization 319 on [0,1) and independent of (N°,Z°, Y°), and define a stationary random field V° by Vt° = V, ie Rd, and for each nEZ, define (N°, Z°, Y°, F°)-point-maps an as follows (with [■] denoting the integer part): if 0 e ?/o +B, let an(n, z, y, v) = h vok]+n (H,y), where k = n(y0 + B); if 0 e yo + B', let an(n,z,y,v) - fin(n,y). Then 6a„ is bijective, and (y0,^0), regarded as a bivariate random field {Yt°,V°)seRd, is stationary and independent of (N°,Z°). Since [Vk] is uniform on {0,1,..., k — 1} we can now select an iV°-point placed uniformly at random among the iV°-points in yo° + B as follows: II°=an(N°,Z°,Y°,V°). Thus p°(((iv°,z°),77° - r0°,-r0°) e -| - y° e B) = p°(((n°, z°), n° + u,u)€-). 5.3 Mid-Step in the Proof of Theorem 5.1 - The 'Only-If' Part Suppose (N°,Z°) is point-stationary. Then en°{N°,Z°,Y°,V°) = (N°,Z°,Y0,V°), which implies (Oni(N°,Z°),-YZ.) = ((N°,Z°),-Y°). (5.3) Now, {-y0°eB} = K^B} and -Y^.=n°-Y0°, (5.4) which yields the second identity in p°((en°(N°,z°),n° + u)£-) = P°((0,7o(Ar°,Z°),77° - Y°) e -| - Y° e B) (due to (5.2)) = P°((077;(JVo)Z°),-y£.)e-|-yi|o €B) (due to (5.4)) = P°(((N°,Z°),-Y0°) e -| - r0° e B) (due to (5.3)) = P°(((N°,Z°),U)£-) (due to (5.2)). Thus point-stationarity implies (5.1) for all bounded B of positive Lebesgue measure.
320 Chapter 9. THE PALM DUALITIES IN HIGHER DIMENSIONS 5.4 Final Step in the Proof of Theorem 5.1 - The 'If Part Suppose (5.1) holds for B = [-h/2,h/2)d. With this B, (5.2) becomes p°(((i\r,z°),n° - y°,-y°) e ■) = P°(((N°,z°),n° + u,u)c •)• Thus 0nz{N°,Z°) = 6n°{N°,Z°), which together with (5.1) yields ^o(AT0,Z°) = (iV0,Z0). Thus (4.9) in Lemma 4.1 holds. Also, when B = [-h/2,h/2)d, the an satisfy the condition in Lemma 4.1. Thus, by Lemma 4.1, (N°, Z°) is point- stationary, and the proof of Theorem 5.1 is complete. 6 Point-Stationarity and the Invariant a-Algebras In this section we first define the invariant <r-algebras X and J, then extend Theorem 5.1 slightly, and finally show that several properties are preserved under conditioning on J, the invariant <r-algebra of both (N, Z) and (N°, Z°) = 6nQ{N, Z). This we need for the randomized-origin duality in Section 8. 6.1 The Invariant a-Algebras I and J The pair (N, Z) is a measurable mapping from (fi, J7) to (M x H,M ®Ti). Define the invariant a-algebra on M. ® H by I:={B€M®H :6»t-1B = -B,teEd} (6.1) and the invariant a-algebra of (N, Z) by J:=(N,Z)-ll [tha.tis,J={{(N,Z)€B}:BEl}]. (6.2) Thus I is a sub-u-algebra of M. ® H, while J" is a sub-u-algebra of T. Lemma 6.1. (a) For any random site T supported by (f2, T) it holds that {eT(N,Z)€B} = {(N,Z)€B}, Bel. (6.3) (b) It holds that g e X if and only if g = g0t for all t € Rd. Proof, (a) For B e X we have {9T(N, Z)eB}= |J {0t(N, Z)€B,T = t} teRd = \J{(N,Z)€B,T = t} = {(N,Z)eB}. teRd
Section 6. Point-Stationarity and the Invariant a-Algebras 321 (b) If g = 1b where B e X, then g9t = le-iB = Is- It follows that g = gOt holds for all simple g G X and thus for all g G X. Conversely, if 3 = g0t for i e Ed, then for each A £ B, O^1g~lA = (g6t)-lA = g~lA. Thus g~lA e X, that is, g £l. O According to Lemma 6.1(a), J is the invariant u-algebra of 6t(N, Z) for any random site T supported by (f2, F). Since (N°, Z°) = 6n0(N, Z) [assumed from Section 4.2 onward], this means, in particular, that J is also the invariant u-algebraof (N°,Z°): J=(iV°,Z°)-1X [that is, J = {{(N°,Z°)EB}:BEX}]. (6.4) [Note that although we have chosen to regard iV° as a random element in {M°,M°), and not in (M,M), the invariant cr-algebra of (N°,Z°) is still J, since X° = the invariant u-algebra on (M°,M°) = the trace of M° x H on X, and thus J = (N^Z0)-1! = (N°,Z°)-ll°.} 6.2 Extension of Theorem 5.1 We shall now extend Theorem 5.1 by allowing the set B to be expanded by an invariant random variable. This result will be used in the proof of Theorem 8.4. Theorem 6.1. Let g S X be a strictly positive and finite function and put G = g(N°,Z°). Let B S Bd be bounded and such that \{B) > 0. Let U be a random site that is uniform on B and independent of (N°,Z°). Let 11° be a random point of N° such that the conditional distribution of 11° given ((N°,Z°),U) is uniform on the finite set of points of N° lying in G(B — U), that is, let 11° be uniform on s(N°)nG(B-U). Then (N°,Z°) is point-stationary if and only if for each such B (en° (N°,Z°), n° + GU) = ((N°, Z°), GU). (6.5) PROOF. First assume that g is bounded and repeat the proof of Theorem 5.1 with B replaced by g(fi,z)B. By Lemma 6.1(6), gel means that g{dt(n,z))B = g{v,z)B for all t e Rd. This fact [that the set g{n,z)B does
322 Chapter 9. THE PALM DUALITIES IN HIGHER DIMENSIONS not change by shifting (/i, z)\ is needed to deduce that 6pn is still bijective. Thus (6.5) holds for bounded g. If g is not bounded, take a finite constant a > 0 and apply (6.5) with g replaced by g A a to obtain E[f(6„. (N°, Z°),LT° + GU)l{g(6no lN.,z>))<a}] = E[f((N°,Z°), GU)l{g{N.iZ.)<a}], f€M®H®Bd+. Send a to infinity to obtain (6.5). □ 6.3 Conditioning on J We now show that all characterizations still hold when we condition on J. This result will be used in Section 8. Theorem 6.2. Let ((N°,Z°),S) and (N,Z) be linked as in Section 4.2 and let P and P° be two probability measures on (Q,,T). Then the following claims hold. (a) The pair (N°, Z°) is point-stationary under P° if and only if it is so conditionally on J. (b) The pair (N,Z) is stationary under P if and only if it is so conditionally on J. (c) The formula (4.1) holds if and only if it holds with E° replaced by E°[-|J]. (d) The formulas (5.1) and (6.5) hold under P° if and only if they hold conditionally on J. (e) If(N,Z) is stationary under P, then E[N([0, l)d)\J] = E[1/A(C0)|J] a.s. P. PROOF. The formula (3.1) in the definition of point-stationarity is equivalent to E°[Wn>{N°,Z°,Y°))l{en.lNotZo)€B}] = E°{f(N°,Z°,Y°)l{iN,tZ,)eB}}, Bel, fzM®U®C+. Due to (6.3), it holds that l{6»r,o(N°,Z°)6B} = l{(N°,.Z°)eB}, B £ 1, and thus (3.1) is equivalent to E°[f{0„.{N°,Z°,Yo))\J] = P°[f(N°,Z°,Y°)\J], f£M®U®C+,
Section 7. The Point-at-Zero Duality 323 that is, (a) holds. We obtain (6), (c), and (d) in a similar way (see the proof of Theorem 7.1 in Chapter 8 if more details are needed). In order to obtain (e), take / e X in Lemma 4.2. Then, by Lemma 6.1(6), f(Op(N,Z)) = f{N°,Z°), and we obtain from (4.10) that E[f(N°, Z°)N({0, l)d)} = E[f(N°,Z°)/\(C0)}, f e X, that is, (e) holds. □ 7 The Point-at-Zero Duality Up to now this chapter has been concerned with point-stationarity, the extension to higher dimensions of the one-dimensional concept of cycle- stationarity. We now turn to the extension of the two Palm dualities considered in Chapter 8 in the one-dimensional case. This section deals with the point-at-zero duality between stationarity and point-stationarity and the next section with the randomized origin duality. In fact, we were ready for the point-at-zero duality after Section 4; the aspects of point-stationarity studied in Sections 5 and 6 will be used for the randomized-origin duality. The point-at-zero duality has (as in Chapter 8) the following informal interpretation: The point-stationary dual behaves as the stationary process (7.1) conditioned on having a point at the origin. We start by establishing the duality, then motivate the interpretation, and finally skim through a simulation application. We shall spend minimal effort on proofs, since they are similar to those in Chapter 8, Sections 4, 5, and 6. Also, many comments from Chapter 8 apply here as well but will not be repeated. 7.1 Stationarity O Point-Stationarity Let (N,Z) and ((N°,Z°),S) be defined on some measurable space (£1, J7) and linked as in Section 4.2, namely, (N°,Z°)=6„0(N,Z) and 5 = -J7„, or equivalently, (N,Z) = es(N°,Z°). Recall that C0 and Cq, respectively, are the Voronoi cells of (N,Z) and (N°,Z°) containing the origin and that they have the same Lebesgue measure A(C0°) = A(Cb).
324 Chapter 9. THE PALM DUALITIES IN HIGHER DIMENSIONS Let P and P° be two probability measures on (Q,T) satisfying E[1/A(C0)] < oo and 1 dP° = A(Co)El[l/A(C0)] or equivalently dP (yolume-debiasing P) E°[A(C0)] < oo and dP A(Cp) E°[A(C0; rdPc (volume-biasing P°). Theorem 7.1. With (N,Z), ((N°,Z°),S), P, and P° as above it holds that (N, Z) is stationary under P if and only if (N°, Z°) is point-stationary under P° and S is conditionally uniform on Cq given (N°,Z°). Proof. This is an immediate consequence of Theorem 4.1. 7.2 Intensity - The Distribution of S Let P and P° be linked as above. Note that 1 □ E[l/A(C0)] = -[A(Co)]. When N is stationary under P, then these two quantities equal the intensity E[N[0, l)d] of N; see Lemma 4.2. Also, note that the conditional distribution of 5 given (N°,Z°) is the same under P as under P° [since A(Cb) is determined by (N°,Z°); see Lemma 4.1 in Chapter 8]. The distribution of 5 under P is given by the following theorem. Theorem 7.2. Suppose the conditional distribution of S given (N°,Z°) is uniform on Cq under Y. Then S is continuous under P and has the density P(5 E ds) _ P°(s E C§) seEd_ ds E°[A(C0)] (7.2)
Section 7. The Point-at-Zero Duality 325 Proof. For B e Bd, I>(SeB) = E{E[l{SeB}\(N°,Z°)}} = E[X(B n Cq )/A(C0)] (by conditional uniformity) = E°[\{B n C0°)]/E°[A(Co)] (by the definition of P°) = /"p0(SGC00)ds/E0[A(Co)], as desired. D 7.3 The Point-at-Zero Interpretation — Limit Motivation The point-at-zero interpretation (7.1) of the duality established in Theorem 7.1 can now be formulated as follows: P((N,Z) e-|770 = 0) = P°((iVo,Zo) e-)- (7.3) This expression is informal because P(i7o = 0) = 0 when N is stationary, since P(770 = 0) < P(770 g [-h/2,h/2)) < E[N([-h/2,h/2)d)} = /idE[l/A(C0)]4-0, hiO. The following theorem yields a strong limit motivation of (7.3): put 77o = t = 0. Theorem 7.3. Suppose the conditional distribution of S given (N°, Z°) is uniform on Cq under P. Then, for each A £ M. ® H and s G M.d, P((N°, Z") 6 A\S = s)= P°((7V°, Z°) e A\s € Cg), (7.4) and thus there is a version ofP(9n0(N, Z) G A\IIo = •) such that P(0„o(N,Z) e A\n0 = i) -► P°((N°,Z°) e A), \t\ I 0. Proof. With f e M®H+ put 5/(s) = E°[/(iV°,z°)|seC0o], seW. Let h G Bd be a nonnegative function and apply (7.2) for the first step in E[h(S)9f(S)} = [ h(s)E°[f(N°,Z°)\s£CS]P°(s&C°0)/E°[\(C0)]ds = E° [f{N°, Z°) J h(s)l{secs} ds] /e°[A(C0)] = E[f{N°,Z°)Jh(8)l{s€cS}ds/\(C0j\.
326 Chapter 9. THE PALM DUALITIES IN HIGHER DIMENSIONS Apply the conditional uniformity of S to obtain (7.4) in the following form: E[h(S)gf(S)} = E[h(S)f(N°, Z")], feM® H+,h e Bd+. The limit claim now follows by noting that 77o = —5 and that l^tec°] —* 1 pointwise as \t\ I 0. D 7.4 Application — Perfect Simulation The two-step duality construction in Theorem 7.1 yields perfect solutions of the problem of simulating a point-stationary (N°,Z°), or a stationary (N,Z), when it is known how to generate the dual. The arguments are analogous to those in Section 6 of Chapter 8, and thus we only state the results. Suppose (cf. Section 6.3 in Chapter 8) we wish to generate the stationary (N,Z) when it is known how to generate its point-stationary dual. Here is a solution when there is a known constant a < oo such that P°(A(C0) < o) = 1. (7.5) Recursively, for n ^ 1: 1. Generate (AT<n), Z<n>) with distribution P°((iV°, Z°) € •) until X(C{0n}) has been realized. 2. Generate an independent E/(") uniformly distributed on (0,1). 3. Repeat steps 1 and 2 independently for n ^ 1 until {[/<") ^ X(C{0")/a} occurs and put K = inf{n ^ 1 : [/<"' ^ X(C{0n))/a}. 4. Now generate as much of (N^K\ Z^) as desired. 5. Generate a random site V uniformly distributed on Q '. Then ev{N{K\Z^K)) is a perfect copy of the stationary (N, Z), and the expected number of acceptance-rejection trials is a/E°[A(Co)]- Apply (cf. Section 6.7 in Chapter 8) the above method without the assumption (7.5). Fix a < oo, carry out steps 1 through 5, and denote the number of acceptance-rejection trials by Ka and the uniform site in Cq by Va. Then 0v<,{N{Ka]', Z^"') is an imperfect copy of the stationary (N,Z)
Section 7. The Point-at-Zero Duality 327 with perfection probability G(a/G(a)), where G is the distribution function G{x) = E°[x A A(C0)]/E°[A(Co)], 0 ^ x < oo; that is, 9Va (N<-K"\ Z(A"°>) coincides with a copy of (N, Z) with probability G(a/G(a)). Suppose (cf. Section 6.5 in Chapter 8) we wish to generate the point- stationary (N°,Z°) when it is known how to generate its stationary dual. Here is a solution in the case when there is a known constant b > 0 such that P(A(C0) > b) = 1. (7.6) Recursively, for n ^ 1: 1. Generate (iV("),Z<")) with distribution P((iV°,Z°) e •) until A(C^n)) has been realized. 2. Generate an independent [/'"' uniformly distributed on (0,1). 3. Repeat steps 1 and 2 independently for n ^ 1 until {£/(") ^ 6/A(Cq"0} occurs and put # = inf{n ^ 1 : [/<"> ^ 6/A(c£n))}. 4. Now generate as much of (N^K\Z^K^) as desired. Then (N{K),Z{K)) is a perfect copy of the point-stationary (N°,Z°), and the expected number of acceptance-rejection trials is l/(E[l/A(Co)]6). Applying this method without the assumption (7.6) yields an imperfect solution with perfection probability R(l/(bR(l/b))), where R is the distribution function defined by R(x) = E[x A (l/A(Co))]/E[l/A(Co)], 0 ^ x < oo. Finally (cf. Section 6.6 in Chapter 8), the problem of generating the stationary (N, Z), when it is known how to generate its point-stationary dual, can be reduced to that of generating the location of the stationary origin seen from the closest point, that is, to the problem of generating a random site W with density P°(s € C0°)/E°[A(C0)], seRd, [see Theorem 7.2]. Proceed as follows:
328 Chapter 9. THE PALM DUALITIES IN HIGHER DIMENSIONS 1. Generate W with the density P°(s € C00)/E°[A(Co)], s € Rd. 2. Generate (AT(n),Z<n)) with distribution P°((N°,Z°) € •) until C(Qn) has been realized. 3. Repeat step 2 independently for n ^ 1 until {W G Cq } occurs and put # = inf{n^ l:lfeCin)}. Then 0w(Z(if),S(Ar)) is a perfect copy of the stationary (N,Z), and the expected number of acceptance-rejection trials is infinite. 8 The Randomized-Origin Duality We now extend the randomized-origin duality from d = 1 to d > 1. This duality is obtained in the same way as the point-at-zero duality except that we condition on the invariant cr-algebra before biasing. It has (as in Chapter 8) the following informal randomized-origin interpretation: The point-stationary dual behaves like the stationary process (8.1) with origin shifted to a uniformly chosen point; and conversely: The stationary dual behaves like the point-stationary process (8.1°) with origin shifted to a site chosen uniformly in Rd. These interpretations are informal because there is neither a uniform distribution on a countable set of points nor on Rd. We start by establishing the duality and then motivate the interpretations. 8.1 Stationarity O Point-Stationarity Again let (N,Z) and ((N°,Z°),S) be defined on some measurable space (fi, T) and linked as in Section 4.2, namely, (N°,Z°) = 6„0(N,Z) and S = -770, or equivalently, (N,Z)=6s(N°,Z°).
Section 8. The Randomized-Origin Duality 329 Recall that Co and Cq, respectively, are the Voronoi cells of (N,Z) and (N°,Z°) containing the origin and that they have the same Lebesgue measure, A(C£) = A(C0). Recall from Section 6.1 that (N,Z) and (N°,Z°) have the same invariant cr-algebra, namely J = (N, Z)-1! = {N°, Z°)~ll, where 1={B£M®%+ :etB = B,teM.d}. Let P and P° be two probability measures on (fi, T) satisfying E[1/A(C0)|J] < oo a.s. P and dP° = A(Co)E[l1/A(C0)|J]dP (volume-debiasinS P ^en J), or equivalent ly, E°[A(Co)|J] <oo a.s. P° and AfC ^ dP = Eorwg°N| 7ldP° (volume-biasing P° given J). Note that (see the proof of (8.3a) in Chapter 8) P = P° on J, (8.2) which we can write as P{{N, Z) e •) = P°((iV°, Z°) € •) on I. (8.3) Note also that (see the proof of (8.36) in Chapter 8) and, in particular, E°MC^J] = E[l/A(Co)|JI- (8-5)
330 Chapter 9. THE PALM DUALITIES IN HIGHER DIMENSIONS Theorem 8.1. With (N,Z), ((N°,Z°),S), P, and P° as above it holds that (N, Z) is stationary under P if and only if (N°,Z°) is point-stationary under P° and S is conditionally uniform on Cq given (Na,Z°). Proof. By Theorem 6.2(6), (N,Z) is stationary under P if and only if E[f(6t(N,Z))\J] = -E[f(N,Z)\J], feM®H+, teRd. By (8.4), this is equivalent to E°{f(6t(N,Z))\(C0)\J] = E°[f(N,Z)\(C0)\J], feM®H+, teRd. By Theorem 6.2(c), this is equivalent to E°[f(6t(N,Z))\(C0)} = E°[f(N,Z)\(C0)}, feM®H+, teRd. A reference to Theorem 4.1 completes the proof. □ The randomized-origin interpretations (8.1) and (8.1°) of the duality established in Theorem 8.1 can now be formulated as follows: P(0uniform point of n(N, Z) G •) = P°((iV°, Z°) G ■). (8"6) and conversely, P°(0unifOrm site in R-(^°, ^°) € ■) = P((N, Z) G •)• (8-6°) This does not have an immediate meaning, because such uniform sites and points do not exist. Below we motivate (8.6) and (8.6°) by shift-coupling and by Cesaro limit results. These results rely on the coupling equivalences in Section 7.4 of Chapter 7. 8.2 Shift-Coupling the Stationary and Point-Stationary Duals According to the following theorem the stationary and point-stationary duals in the duality established in Theorem 8.1 are really the same, only seen from different sites. Theorem 8.2. Suppose the equivalent statements in Theorem 8.1 hold. Then the probability space (fi, J-, P) can be extended to support a random point II of N such that P(6n(N,Z)£-)=P°((N°,Z°)e-). (8.7)
Section 8. The Randomized-Origin Duality 331 Conversely, the probability space (Q, T, P°) can be extended to support a random site T such that P°(6T{N°, Z°) g ■) = P((N, Z) G •)• (8-7°) Proof. We shall first establish (8.7°). Apply the results in Section 7.4 of Chapter 7 with Y having the distribution P°((iV0,Z0) G •) and Y' the distribution P((iV, Z) G •) and with {8t : t G Rd} the transformation group. Then (8.3) above is the condition (c) in Section 7.4 of Chapter 7, which is equivalent to (a') in that subsection, which is (8.7°). In order to establish (8.7) use the transfer extension in Section 4.5 of Chapter 3: use (8.7°) to extend (fi, T, P) to obtain a random point 77 of N such that P(((iV, Z), 77) G •) = P°((8t(N°, Z°), -T) G •)• This implies P(6n(N,Z) G •) = P°(0-T0t(N°,Z°) G ■), that is, (8.7) holds. □ 8.3 Cesaro Total Variation Motivation of (8.6°) The next theorem gives a Cesaro total variation meaning to the randomized- origin interpretation (8.6°). Theorem 8.3. Suppose the equivalent statements in Theorem 8.1 hold. Let Bh G Bd,0 < h < oo, be F0lner averaging sets, that is, a family of sets satisfying 0 < X(Bh) < oo and, for all t £ Rd, X(Bh H(t + Bh))/\{Bh) -> 1, h->oo, (this holds, for instance, when Bh = hB\; see Theorem 2.1 in Chapter 7). Let g G X fee strictly positive and finite and put G = g(N°,Z°). Let Uh be uniform on Bh and independent of (N°, Z°) under P°. Then P°(6Guh(N°, Z°) G •) -> P((N,Z) G •) in total variation as h —>■ oo. PROOF. Apply the results in Section 7.4 of Chapter 7 with Y having the distribution P°((A^°,Z°) G ■) and Y' the distribution P((N,Z) G ■) and with {6t : t G W} the transformation group. Then (8.3) above is the condition (c) in Section 7.4 of Chapter 7, and we obtain the desired limit result from the final display in that subsection. □
332 Chapter 9. THE PALM DUALITIES IN HIGHER DIMENSIONS 8.4 Cesaro Total Variation Motivation of (8.6) The next theorem gives a Cesaro total variation meaning to the randomized- origin interpretation (8.6). Theorem 8.4. Suppose the equivalent statements in Theorem 8.1 hold. Let Bh, 0 < h < oo, and G be as in Theorem 8.3. Let each Bh be bounded. Let Th be a random site such that under P the conditional distribution of Th given (N, Z) is uniform on s(N) (~) GBh when s(N) fl GBh / 0 flfirf Th - 0 when s(N) n GBh = 0. Then p(6Th(N,Z)e-)^p°((N°,z°)e.) in total variation as h —> oo. Proof. Let E/h be as in Theorem 8.3. Let 77£ be a random point of N° such that the conditional distribution of 77£ given ((N°, Z°),Uh) is uniform on s(N°) n G{Bh - Uh) under P°. Apply Theorem 6.1 to obtain P°(enh0-auh(N°,Z°) 6 •) = P°((N°,Z°) € ■), (8.8) where nh = n° + Guh. Note that the conditional distribution of 77/j given ((iV°, Z°),Uh) is uniform on s(6-GUhN°)r\GBh under P°. By Lemma6.1(6), G = g(6-GUh(N°, Z°)), and thus the conditional distribution of 77^ given 6-avh (N°, Z°) is uniform on s(9-GUhN°)f\GBh under P°. It follows that, P°(9-GUh(N°,Z°) e •) almost surely, we have P(Th G -\(N,Z) = ■) = P°(IIh G -le.Gu^N^Z0) = •). This, and (3.3) in Lemma 3.1 of Chapter 6, yields ||P((iV, Z),Th) G •) - P°((0-GUh(N°,Z°),nh) 6 -)II = ||P((iv, z) g 0 - P°(0-GUh(N°, z°) e oil, where || • || denotes the total variation norm. Due to (3.2) in Lemma 3.1 of Chapter 6, \\P(0Th(N,Z) G 0 -P°(0nhe-Guh(N°,Z°) G Oil ^ \\P(((N,Z),Th) G 0 -Po((0_GUh(iV°,Z°),/7,O G Oil- Thus ||P(^(iV,Z)G0-Po((Aro^O)G0ll = ||P((9Tfc(iV,Z) G 0 -P°(0nhe-GUh(N°,Z°) G Oil [by (8.8)] ^ \\p((N,z),Th) e o -P°((0-Guh(N°,z°),nh) e oil [by (8.10)] = ||P((iV,Z) GO- P°(0-GUh(N°,Z°) G Oil- [by (8.9)] Apply Theorem 8.3 to obtain the desired limit result. □
Section 8. The Randomized-Origin Duality 333 8.5 Another Cesaro Total Variation Motivation of (8.6) The reader may have noted that Theorem 8.4 is not an immediate counterpart of (9.2) in Theorem 9.1 of Chapter 8: we average over a random number of points in a set and not over a deterministic number of points. In order to average over a deterministic number of points we need the following fact from ergodic theory. Fact 8.1. Suppose (N,Z) is stationary under P and E[N([0,l)d)] < oo. Let Bn G Bd, 1 ^ n < oo, be convex and compact and increase to Rd as n —> oo. Then N{Bn)/\{Bn) ->. E[N([0, l)d\J] a.s. P, n ^ oo. For a proof, see Daley and Vere-Jones (1988), Proposition 10.2.II and Theorem 10.2.IV. The proof relies on the ergodic theory of Tempel'man (1972). We also need the following natural lemma (natural because a G G J+ should behave as a constant when J is given). Lemma 8.1. Suppose the equivalent statements in Theorem 8.1 hold. Let G G J+, that is, let G = g(N,Z) for some g G 1+■ Then under P the point process N(G-) is stationary and has conditional intensity E[N(G[0, l)d)\J] = GdE[iV([0, l)d)\J] a.s. P. (8.11) If further G = E[N([0, l)d)|J]_1/d, then E[AT(G[0, l)d)|J] = 1 a.s. P. Proof. Stationarity of N(G-) follows from the stationarity of (N, Z) and Lemma 6.1(6). In order to establish (8.11), note that for all k > 0, all ai,..., afc > 0, and all disjoint A\,...,Ak G T we have N((ailAl+--- + aklAk)[0,l)d) = lAlN(ai[0, l)d) + ■■■ + lAkN(ak{0, l)d) and that by Theorem 6.2(6), N is stationary conditionally on J, which yields E[JV(a[0, l)d)|J] = adE[iV([0, l)d)|J] a.s. P for all a > 0. Thus (8.11) holds for all simple functions G G J+■ Any bounded G G J+ can be approximated both from below and above by simple functions in J+, and thus, for bounded G € J+, we have that (8.11) holds with = replaced by both ^ and >. Thus (8.11) holds for bounded G G J+. Now, for any G G J+, N(G[0, l)d) = £ N(Gl{i^G<i+l}[0,1)"),
334 Chapter 9. THE PALM DUALITIES IN HIGHER DIMENSIONS and since (8.11) holds for each GI^oki+i}, it holds for G. The final claim follows from (8.11) by noting that due to Theorem 6.2(e), E[iV([0, l)d)\J] = E[1/A(C0)|J] and that by assumption, E[1/A(C0)|J] < oo. □ Here finally is the other Cesaro total variation motivation of (8.6). Theorem 8.5. Suppose the equivalent statements in Theorem 8.1 hold. Let l?/j,0 < h < oo, be as in Theorem 8.3. Let the Bh be convex and compact and increase continuously from {0} to Rd as h increases from 0 to oo. Let LTn,n ^ 0, be the points of N enumerated in the order they are hit by B^ as h increases and lexicographically if two or more are hit simultaneously. Let U be uniform on [0,1) and independent of (N, Z) under P. Then p(en[Un](N,z) & ■) ^p°((N°,z°) e ■) in total variation as n —>■ oo. Proof. Put G = E[N([0, l)d)\J]-lld. By Lemma 8.1, N(G-) is stationary and E[iV(G[0,l)d)|J] = l a.s. P. Let h(n) be such that X(Bh^) = n and apply Fact 8.1 to N(G-) to obtain N(GBh{n))/n ->■ 1 a.s. P, n ->■ oo. (8.12) Let T/j be as in Theorem 8.4, interpret 1 ° 0 5Z 1{Dnk_1{N,Z)e-} = l{(JV,Z)e-}, fc=i and note that for n,m ^ 1 and 0 ^ ai ^ 1, 0 ^ a% ^ 1,..., 1 v-> 1 v~> I n(- —-) = 1 — n/m, n ^ m, n£r{ mkT1 l("-m)^ = l- m/n> m < n, to obtain P(9„lUn](N,Z)€-)-P(0Thln)(N,Z)€-) ■1^. 1 nAN{GBh{n)) nVN(GBh(n)) rnAN(GBh(n)h ^ ^InV N(GBhln,)\
Section 9. Comments 335 Thus (see the second identity in (8.11) of Chapter 3) \\p(en[Un](N,z) e -)-P(eTHn)(N,z) e Oil <2 2E\nAN{GBh{n))] InV N(GBhln))\- By (8.12) and bounded convergence the expectation goes to 1 as n -4 oo, and thus \\P(6n[Un](N:Z) e •) - P(0Th(n)(N, Z) e -)II -> 0, n -> cx>. This, together with Theorem 8.4, yields the final step in \\P(0„lUn](N,Z)e-)--po((No,z°)e-)\\ < \\p(0„lUn](N,Z) e .)-P(6Thin)(N,Z) e -)ll + \\P(eTh(n)(N,Z)&.)-P°((N°,Z°)&-)\\ —> 0, n —> oo, and the proof is complete. □ 9 Comments We conclude this chapter with comments on the two Palm dualities and on a possible extension of the point-stationarity concept to more general random phenomena. 9.1 When Do the Two Palm Dualities Coincide? Under what conditions is it true that standing at the origin of a stationary point pattern, and happening to find a point there, is equivalent to standing at a point selected uniformly at random from the point pattern? That is, when do the two Palm dualities coincide? We now specify the exact condition. Let (JV, Z) be stationary under a probability measure P with finite intensity: E[iV([0,l)d)] = E[l/A(Co)]<oo. Then the two point-stationary duals of (JV, Z) coincide if and only if E[1/A(C0)|J] = E[1/A(C0)] a.s. P. In particular, this holds in the ergodic case, that is, when P= 0 or 1 on J.
336 Chapter 9. THE PALM DUALITIES IN HIGHER DIMENSIONS Conversely, let (N°,Z°) be point-stationary under a probability measure P° with finite Voronoi cell volume: Eo[A(C0)]<oo. Then the two stationary duals of (N°,Z°) coincide if and only if E[A(Co)|J] = E[A(C0)] a.s. P°. In particular, this holds in the ergodic case, that is, when P° = 0 or 1 on J. 9.2 Random Site Change Hides the Gap Between the Dualities As in the one-dimensional case we can make the two Palm dualities coincide by a simple random site change. Let (N, Z) be stationary under a probability measure P with E[l/A(Co)] < oo. Let Z be measurable under change of site-scale and change the site-scale by G = E[l/A(Co)|J]-1/d to obtain (N(G-), (ZGs)se^). This new pair is stationary, and its two point-stationary Palm duals coincide. This procedure preserves the randomized-origin duality and not the point-at-zero duality. In fact, we lose the point-at-zero duality: the point- at-zero duality merges with the randomized-origin duality by the change of site-scale and does not reappear when we return to the original site-scale after change of measure (as the randomized-origin duality does). Thus the site change is not a way to bridge the gap between the two dualities; it only hides it. To bridge the gap we cannot avoid a change of measure. 9.3 Extending Point-Stationarity to General Random Sets? There are other random sets that should look the same from all the points. An obvious example is the set of times where a Brownian motion takes the value zero. The solution in the present chapter of the point-stationarity problem in the case of point processes in d > 1 dimensions (Definition 3.1) suggests that point-stationarity could be defined in these models as follows. Proposed definition. A random set is point-stationary if it is distri- butionally invariant under bijective point-shifts against any independent stationary background. It remains to find such backgrounds and point-shifts. With this open problem we end the stationarity part of the book.
Chapter 10 REGENERATION 1 Introduction In this last chapter we finally focus on the third topic of the book, regeneration. Regenerative processes are generalizations of Markov chains and renewal processes, which we considered in Chapter 2. We shall look at several kinds of regeneration, and as in Chapter 2, the aspects we concentrate on are coupling, stationarity (and its generalizations), and total variation asymptotics. In Section 2 we establish notation and then consider briefly the one-sided counterpart of the two-sided stationarity theory of Chapter 8. In Section 3 we consider classical regeneration. A stochastic process is regenerative in the classical sense if there are random times where it starts anew independently of the past, like a recurrent Markov chain at the times of visits to a fixed reference state. The regeneration times form a renewal process and split the stochastic process into a sequence of cycles that are i.i.d. and independent of a possible initial delay. In Section 4 we consider wide-sense regeneration. Wide-sense regeneration allows the future after regeneration to depend on the past as long as the future is independent of the past regeneration times. This is the type of regeneration occurring in so-called Harris chains, but for a simple example consider a recurrent Markov chain and let J > 0 be fixed: / time units after visiting a fixed reference state the chain regenerates in the wide sense (but typically not in the classical sense). The regeneration times still form a renewal process, but the cycles are only stationary and need not be independent. 337
338 Chapter 10. REGENERATION In Section 5 we move on to consider time-inhomogeneous regeneration. Time-inhomogeneous regeneration allows the future after regeneration to depend on the time of regeneration. This is the type of regeneration occurring in time-inhomogeneous Markov chains with a recurrent state: if such a Markov chain visits this state at time t, then the future after time t is independent of the past before time t but has a distribution that depends on t. In this case the sequence of cycles need no longer be stationary and the regeneration times need not form a renewal process, they only form an increasing discrete-time Markov process (note that a renewal process is an example of an increasing discrete-time Markov process). Section 6 contains a coupling construction for time-inhomogeneous regenerative processes. This construction is an elaboration on the classical coupling (Chapter 2). In Section 7 we investigate the coupling time thoroughly to obtain results on uniform convergence and rates of convergence. In Section 8 we introduce asymptotics from-the-past. Ordinary asymp- totics are to-the-future: we start a process at time zero and observe it in the far future to obtain a stationary limit process. Asymptotics from-the-past is the reversal of this procedure: we start a process in the remote past and observe it from any fixed time t onwards to obtain a (typically) nonstationary limit process. In the time-inhomogeneous case we cannot expect to obtain a limit process by going to-the-future (unless the time-inhomogeneity disappears asymptotically), but coming in from-the-past turns out to yield a limit process, a nonstationary one because of the time-inhomogeneity. In Section 9 we consider taboo regeneration. Taboo regeneration means basically that the process regenerates in the classical sense while not entering a fixed region of the state space (taboo region), like a transient Markov chain while it stays in a finite irreducible set of states. In this case the regeneration times form a possibly terminating renewal process. We shall consider the existence of taboo limits: we start a process at time zero and observe it in the far future, conditionally on not yet having entered the taboo region, to obtain a limit process. A taboo regenerative process becomes a time-inhomogenous regenerative process under this conditioning, and thus the limit theory from the time-inhomogeneous case applies. In Section 10 we consider taboo stationarity, the characterizing property of a taboo limit process, and work out the structure of the taboo limit process in the taboo regenerative case. This structure is quite different from, but analogous to, the structure of the stationary version of a cycle- stationary process (Chapter 8). Section 11 rounds off with coupling from-the-past, a perfect simulation method for generating observations from the stationary, nonstationary, and taboo stationary (quasi-stationary) limits of finite state space Markov chains. As the above description indicates, many themes from the previous chapters converge in the next two sections, to be further developed and extended in the remaining sections.
Section 2. Preliminaries - Stationarity 339 2 Preliminaries — Stationarity This section lays down the framework to be used in Sections 3 through 6 (and also with certain modifications in Sections 7 through 9) and then considers briefly the one-sided counterpart of the two-sided stationarity theory in Chapter 8. 2.1 The One-Sided Process and Points Let (fl, T, P) be a probability space supporting Z = (^*)se[o,oo) and S = (Sk)o>, where Z is a one-sided continuous-time stochastic process with a general state space (E, £) and path space (H, V.) and S is a one-sided sequence of random times (points) satisfying 0 ^ S0 < Si < > oo. Regard S as a measurable mapping from (fi, J-) to the sequence space (L,£), where L = {(sfc)S° € [0, oo)*0'1-- > : so < «i < ► <x] and £ are the Borel subsets of L, that is, £ = Lfl#'-'. Thus the pair (Z, S) is a measurable mapping from (fi, F) to (HxL, H®C). Let %®C+ denote the class of all measurable functions from (H xi,7{®£) to ([0,oo),B[0,oo)). We shall not assume any functional connection between Z and S. At one extreme, Z and S could be independent. At another extreme, S could be determined by Z: for instance, S could be the times when Z enters a given state or set. At a third extreme, Z could be determined by S, as is the case if Z is one of the following processes. Let 5_i be a strictly negative random variable and put, for t € [0, oo), Nt At Bt Dt ut = inf {n ^ 1 : S„ > = t — 5jv,-i = Sn, — t = XNt =At + Bt = AtjDt t} number of points in [0, t], age at time t, residual life at time t, total life at time t, relative age at time t; see Figures 8.1 and 9.1 in Chapter 2.
340 Chapter 10. REGENERATION 2.2 The One-Sided Shift - Shift-Measurability For t € [0, oo), let 8t be the shift-map from H to H: &tZ = (^t+s)se[0,oo)- Let 8t also denote the joint shift-map from H x L to H x L: et(z,(sk)™) = (9tz,(snt_+k-t)^), where nt- = inf{n ^ 1 : sn ^ t}. Note that f?t is a time shift and shifts (sfc)o° regarded as a sequence of times, that is, 8t shifts (sfc)§° by subtracting t from the times sfc and only shifts the index k of (sfc)o° to observe the convention that the first time is indexed by 0. In order to be able to shift at will without measurability complications, assume that Z is shift-measurable, that is, let the path set H be invariant under time-shifts and the mapping taking (z,t) € H x [0, oo) to zt e E be % ® B[0, oo)/£ measurable. This is equivalent to the mapping taking (z,t) e H x [0, oo) to 0tz € H being % ® B/U measurable. Shift- measurability covers, for instance, processes with a Polish state space (in fact separable metric suffices) and right-continuous paths (left-hand continuity is not needed). See Section 2 of Chapter 4 for more details. 2.3 Cycles and Cycle-Lengths — Delay and Delay-Length The random times Sn split Z into a delay D = (zs)s£[o,sQ) (see Figure 2.1) and a sequence of cycles Cn = (•^sn_i+s)se[o,xn)> n^l, where Xn are the cycle-lengths Xn = Sn — Sn-i, n ^ 1. Realization of (Z, S) (to make the illustration easier we let Z be real- valued with continuous paths and S be the times of visits to 0) (Z,S) (Z°,5°) C. c2 The gray axis is at the origin of (Z°, 5°). C4 So S, s4 -v -x2^\*x3*\*~xA FIGURE 2.1. The points S split Z into a delay D and cycles C„.
Section 2. Preliminaries - Stationarity 341 The delay D and the cycles Cn are stochastic processes vanishing at the random times So and Xn, respectively. The easiest way to make sense of such processes as random elements is to think of them as entering an absorbing state A when vanishing, where A (the cemetery) is external to the state space; see Section 2.9 of Chapter 4 for technical details. The cycle- lengths Xi,X2,-.- and the delay-length So are all obtained by the same measurable mapping from their respective cycles C\, Ci, ■. ■ and delay D. They are simply the hitting times of the absorbing cemetery state A. The pair (Z, S) is a measurable mapping of the delay and cycles (string them together), and vice versa. Say that (Z, S) is zero-delayed if So = 0. Define a zero-delayed pair by (Z°,S°) := 6So{Z, S) (see Figure 2.1). Thus S^ = 0 and 5,° = Xf, while for n ^ 1, X° = Xn and C°n = Cn. 2.4 Cycle-Stationarity — Stationarity Call (Z, S) cycle-stationary if the cycles form a stationary sequence, that is, with = denoting identity in distribution: (C,n+1,Cn+2,...) = (C1,C2,...), n^O. Cycle-stationarity is equivalent to eSn(z,s) = (z°,s°), too, since (Cn+i, Cn+2, • • •) and 8sn(Z,S) are measurable mappings of each other, and since these mappings do not depend on n. When (Z, S) is cycle- stationary, put F(x) = P(X1^x), 0s$:r<oo, that is, F is the common distribution function of the cycle lengths. A pair (Z*,S*) is stationary if 0t{Z*,S*) = {Z*,S*), t>0. We now construct a stationary (Z*,S*) from a cycle-stationary (Z°,S°) when E[Xi] < oo. The proof is based on the same idea as in Section 9 of Chapter 2 and Section 4 of Chapter 8; namely, a stationary version should be obtained by length-biasing the first cycle of (Z°,S°) and then placing the time origin at random in that cycle. Theorem 2.1. Suppose (Z,S) is cycle-stationary with E[XY] < oo. Let U be uniformly distributed on [0,1) and independent of (Z°,S°) and let P* be the probability measure on (fi, T) defined by dP* = r ,dP (length-biasing). E[AiJ
342 Chapter 10. REGENERATION Let (Z*,S*) have the distribution V*{6UXl (Z°,S°) € ■)■ Then (Z*,S*) is stationary, E[f(Z',Sr)]=E[[ VcW^^sj/EpG], fzU®C+, (2.1) Jo and Sq is continuous with distribution function Goo defined by and density P(X] > x)/E[Xi], x ^ 0. Comment. In this chapter we return to the convention of the common probability space (see Section 3.1 in Chapter 3) abandoned for a while in Chapters 8 and 9. This means that we let all random elements under consideration [like (Z,S) and (Z*,S*) in the above theorem] be defined on a single probability space (fi, T, P). However, we sometimes [as in the above theorem and later in Section 9] establish existence and structure through a change of measure [by replacing P by P*]. Proof. The definition of P* yields the first equality in the following calculation: for t ^ 0 and bounded / e % ® C+, E*[f(eteUXl(z°,s°))} = Eifieteux^z^s^x.yEix,} = E[J ' f{es{Z\Sa))ds]/E[X,] = (E[[ 1/(^(2°,5°))rfs]+E[/ 1+/(fl.(Z°,S°))ds])/E[A:1] Jt JXi = (E[ / * f(0,(Zo,S°))d8] +E[ f fiesiZ^S^ds^/EiX,} Jt Jo = E[[ l fie^Z^S^dsj/EiX,}, Jo while the second equality follows from the fact that the conditional distribution of t + UX\ given (Z°,S°) is uniform on [t,t + Xi), the third equality holds since / is bounded (note that the integral from t to X\ can be negative, since t can be greater than X\), and the fourth equality follows from the fact that 0Xi(-Z°,S°) has the same distribution as (Z°,S°). The last term does not depend on t, and stationarity is established. This also establishes (2.1) for bounded / and thus, by monotone convergence, for all /. For the distribution of Sq, see Section 4.4 in Chapter 8. □
Section 2. Preliminaries - Stationarity 343 2.5 Lattice X\ — Periodic Stationarity Call X\ lattice with span d if d > 0 and V{Xl£dL)=\ and ¥(XY € al) < 1 for all a > d. Call a pair (Z**, S**) periodically stationary with period d if d > 0 and 6nd(Z**,S**) = (Z**,S**), TOO. (2.2) Recall that x mod d := x — [x/d]d. Theorem 2.2. Suppose {Z,S) is cycle-stationary with E[Xi] < oo and let (Z*,S*) be as in Theorem 2.1. If X\ is lattice with span d, then \Zj , O ) .— os*modd(^ ,3 ) is periodically stationary with period d, Xi/d E[/(Z**,S")] =dE[ Y, f{Okd{Z°,S°)j\IV[Xil (2-3) fc=i for f € 7i ® C+, and Sq* is d!L valued with probability mass function Proof. Since the cycle-lengths of (Z*,S*) are dZ valued, we have Sq mod d — B*nd mod d, n ^ 0. Since (Z*,S*) is stationary, this implies that (0„d(Z*,S*),S* modd) ~ ((Z*,S*),S0* modd), n>0. Since 9nd(Z**,S**) is the same measurable mapping of the left-hand side as (Z**,S**) is of the right-hand side, we obtain (2.2), that is, (Z**,5**) is periodically stationary with period d. Apply (2.1), with f{Z",S*) replaced by f{eBimoid{Z*,S*)) and with f(6s(Z°,S°)) replaced by f(eB°modd0s(Zo,S°)), to obtain nnoBimodd(Z;S*))] = E[ / ' /(flB.m0dA(^0,S0))ds]/E[A:1]. ./o Since 5q mod d = Bq mod rf, the left-hand side equals the left-hand side of (2.3), and since 0Bomodd0s{Z°,So) = 6kd{Z°,S°) for s € (kd-d,kd\, the right-hand side equals the right-hand side of (2.3). Thus (2.3) holds. Finally, the density claim in Theorem 2.1 yields the second equality in P(S0** = kd) = P(kd ^ S0* <kd + d) [S** = S* - (S0* mod d)] pkd+d = / P(X, > x) dar/Epd] = dP(X, > fc^/E^], while the last equality follows from the fact that X\ is dZ valued. O
344 Chapter 10. REGENERATION 2.6 On Discrete Time To simplify the presentation we do not treat discrete-time processes separately in this chapter. This is no restriction, since a discrete-time process (Zfc)g° can be embedded into a continuous-time shift-measurable process Z = (Z5)se[0,oo) by defining Zs = Zw, s E [0, oo). If there is an integer-valued sequence of times S associated with (Zfc)g°, then the cycle-lengths of {Z, S) will be lattice with an integer span d ^ 1. If d = 1, then the cycle lengths of (Zfc)g° are called aperiodic. In this case Theorem 2.2 yields a discrete-time pair ((Z^*)g°, S**) that is stationary (in discrete time), that is, 0„((Zr)oo,S**) = ((Zr)5°,S**), n^O, and the delay length Sq* has probability mass function If d > 1, then ((Z**)^, S**) is periodically stationary (in discrete time) with period d, and the delay-length S** has the probability mass function in Theorem 2.2. 2.7 Comparison with the Two-Sided Case in Chapter 8 An important distinction between the above one-sided framework and the two-sided framework in Section 2 of Chapter 8 is the delay concept. One could think of the delay as an initial cycle, but it is more appropriate to separate it from the cycles. For instance, the delay of a one-sided stationary process (see Theorem 2.1 above) is in fact only the latter part of a full cycle. For that reason we do not use Xq (here a symbol for a length of a cycle) for the delay length Sq. The essentials of the two-sided Palm duality theory in Chapter 8 can be established in the one-sided setting by a modification of the proofs. In fact, when the state space is Polish and the paths right-continuous with left-hand limits, then we need not even modify the proofs because then we can carry results immediately over to the one-sided setting by extending a one-sided stationary (Z*,S*) and a one-sided cycle-stationary (Z°,S°) to two-sided pairs [use the Kolmogorov extension theorem, Fact 3.2 in Chapter 3]. Here we shall not go through the details of the one-sided counterpart of the theory in Chapter 8 but only sketch briefly those results that are particularly illuminating to contrast with the results in the next section.
Section 2. Preliminaries - Stationarity 345 Theorem 2.1 above establishes half of the point-at-zero duality between stationarity and cycle-stationarity (for the two-sided version, see Theorem 4.1 in Chapter 8). The informal point-at-zero interpretation Z° behaves as Z* conditioned on the null event 5g = 0 can be established formally. For instance, when the state space is Polish and the paths right-continuous with left-hand limits, then Z* conditioned on {Sq ^ t} goes in distribution to Z° as t goes to zero. In the same way we can establish half of the randomized-origin Palm duality between stationarity and cycle-stationarity (for the two-sided version, see Theorem 8.1 in Chapter 8). Namely, suppose (Z°,S°) is cycle-stationary and that E[X°U°] < oo, where J° is the invariant a-algebra of (Z°,S°). Let U be uniformly distributed on [0,1) and independent of (Z°,S°) and define a probability measure P on (ft, J-) as follows: X° d^ = prvoiTd dP (length-biasing given J°). ViM J Let (Z,S) have the distribution P(0UXl(z°>S°) <E •)• Then (Z,S) is stationary, and the distributions of Z° and Z agree on invariant sets. Thus, by Theorem 5.4 in Chapter 5, the randomized-origin interpretation holds: Z behaves as Z° with origin shifted to a uniform time in [0, oo); or formally in terms of Cesaro total variation convergence: with V uniform on [0,1) and independent of Z°, it holds that 9vtZ° 4 Z, t -> oo. Also, there exists a successful distributional shift-coupling of Z° and Z (see Theorems 5.4 and 2.1 in Chapter 5); namely, the probability space (Q., J-, P) can be extended to support finite random times T° and T such that 0T°Z° =6fZ. When there exists a weak-sense-regular conditional distribution of Z given 8fZ (for instance, when the state space' is Polish and the paths right- continuous), then we may choose Z such that the shift-coupling is nondis- tributional (see Theorem 2.2 in Chapter 5): C7J-0 Z =■ Uj>Z. This is the shift-coupling (and Cesaro total variation) counterpart of the stronger exact coupling and epsilon-coupling (and plain and smooth total variation) results established for classical and wide-sense regenerative processes in the next two sections.
346 Chapter 10. REGENERATION Remark 2.1. The two stationary versions (Z*,S*) and (Z,S) of a cycle- stationary (Z°,S°) turn out to coincide in the regenerative case. This can be seen, for instance, by noting that there is a trivial successful distributional shift-coupling of a regenerative (Z°,S°) and the stationary version (Z*,S*) (see Theorem 3.1 below: the shift-coupling times are 5q = 0 and So). Since there is also successful distributional shift-coupling of a regenerative (Z°,S°) and the stationary version (Z,S), the two versions are both Cesaro total variation limits of (Z°, 5°) and thus must have the same distribution. Remark 2.2. The reader may wonder why we state the Cesaro total variation and shift-coupling results only for Z° and Z and not for (Z°,S°) and (Z, S) as in Chapter 8. This is just to be in accordance with the rest of this chapter. To simplify notation in this chapter we shall state many results for the process only and not for the joint process and points. This is no restriction because we can always embed 5 in the process by replacing Z by (Zs,As)se[0yOOy Then 5 is simply formed by the times when the age process (As)s6[o>0o) Enters the state zero. 3 Classical Regeneration In this section we shall consider processes regenerative in the sense commonly associated with that term. In order to distinguish this regeneration concept from the generalizations studied in Sections 4 through 10 we shall use the term classical regeneration. 3.1 Definition Call a one-sided-shift-measurable stochastic process Z classical regenerative with regeneration times S if dSn(Z,S) = (Z°,S°), n^O, (3.1) and 8sn(Z,S) is independent of ((Zs)s€[0tS„),So,---,Sn), n ^ 0. (3.2) Call the pair (Z, S) classical regenerative if this holds. This definition can be reformulated as follows: (Z, S) is classical regenerative if and only if C\, C2,... are i.i.d. and independent of D. (3.3) In order to establish this equivalence, first note that (3.3) can be reformulated as the following two claims (Cn+1,Cn+2,...) = (CuC2,...), n>0, (3.4) (C„+i,C„+2,...) is independent of (D,Ci,...,C„), n ^ 0; (3.5)
Section 3. Classical Regeneration 347 then recall that (3.4) is equivalent to (3.1); and finally note that (3.5) is equivalent to (3.2), since (Cn+i,C„+2, • • ■) and 6s„(Z,S) are measurable maps of each other and so are (D, C\,..., Cn) and ((Zs)s6[oj5?i),5o,.. .,5n). It follows from (3.3) that if (Z, S) is classically regenerative, then the cycle-lengths X1,X2,... are i.i.d. and independent of the delay-length So, that is, 5 is a renewal process. The cycle-lengths are also called inter- regeneration times and recurrence times. Let the nonnegative random variable 5-i be such that (S-i,D) is independent of (Z°,S°). 3.2 Examples Here are a few standard examples of processes that are regenerative in the classical sense. Let 5 be a renewal process. Then the age process (-*4s)se[o,oo)> the residual life process (i?s)s€[oj0o), the total life process (£,s)se[o,oo); and the relative age process (Us)se[o,oo) are all classical regenerative with S as regeneration times. In fact, these processes viewed jointly as (As, Bs, Ds, Us)se[o,oo) form a four-dimensional classical regenerative process. Also, if Z is classical regenerative with regeneration times 5, then so is (Zs, As, BS,DS, Us)se[o,oo)- Let Z be an irreducible recurrent Markov chain (as in Chapter 2). Then Z is classical regenerative with regeneration times S formed by the successive entrances to a fixed reference state. Let Z be a general state space shift-measurable Markov process (as in Chapter 6). Let A be a set of states such that Z enters A infinitely often and finitely many times in finite intervals, and satisfies the Markov property at the entrance times. If the transition probabilities are the same from all states in A, then Z is classical regenerative with regeneration times S formed by the successive entrances to A. Conversely, any classical regenerative (Z, S) can be embedded into a Markov process. For instance, the process with value (Zt+s)se[o,Bt) at time t is Markovian, and so is the process with value (Zt+s)s€[-At,o\ &*■ time t. Regeneration is usually not a primary assumption in applications. Rather regeneration is the key property of many processes that come out of stochastic models, the property that makes the processes amenable to analysis. In particular, regenerative processes abound in queueing theory. As an example, let us consider the GI/GI/1 queueing model, namely the single-server queueing system where customers arrive to a service station at times forming a renewal process and line up to be served under the first-come-first- served discipline with i.i.d. service times that are independent of the arrival process (GI stands for general independent). Let a denote an inter-arrival time and /? a services time. Let Qt denote the queue length at time t and Rt the remaining service time of the customer being served at time t. If E[/3] < E[q], then (according to the law of large numbers) the system will empty infinitely often and the bivariate process (Qs,.Rs)se[o,oo) is classical regenerative with the times of arrivals to an empty system as regeneration
348 Chapter 10. REGENERATION times. If further E[/3] < E[a] < oo, then it can be shown that the expected value of the inter-regeneration times is finite, and thus (according to Theorem 3.1 below) there exists a stationary version of (Qs, Rs)se[o,co)- Moreover, it is readily checked that if the inter-arrival times are nonlattice (or spread out), then so are the inter-regeneration times and thus (according to Theorem 3.3 below) the process (Qs, Rs)se[o,<x>) tends to its stationary version in smooth (or plain) total variation. 3.3 Stationary Version — Periodically Stationary Version A pair (Z',S') is a version of a classical regenerative (Z,S) if (Z',S') is also classical regenerative and 9S-0(Z',S') = (Z°,S°). (3.6) Note that a pair (Z\ S') is a version of a classical regenerative (Z, S) if and only if (3.6) holds and the delay D' of (Z', 5') is independent of 6S-0(Z',S'). In particular, (Z°,S°) is a zero-delayed version of (Z, S). Theorem 3.1. Suppose (Z,S) is classical regenerative with E[X\] < oo. Then (Z*,S*) in Theorem 2.1 is a stationary version of (Z,S). Further, if X\ is lattice with span d, then (Z**,S**) in Theorem 2.2 is a periodically stationary version of (Z, S) with period d. Proof. According to Theorem 2.1, (Z*,S*) is stationary. According to Theorem 2.2, (Z**,S**) is periodically stationary. Since (Z** ,5**) := 0s*modd{Z* ,5*), it follows that 0Sj. (Z**, S**) = 0S* (Z*,S*) and that the delay of (Z**,S**) is a measurable mapping of the delay of (Z*,S*). Thus (Z**,S**) is a version of (Z,S) if (Z*,S*) is a version of (Z,S). Hence it only remains to show that (Z*,S*) is a version of (Z,S), that is, with U and P* as in Theorem 2.1 we must show that P*(6Xl(Z°,S°) e •) = P((Z°,5°) 6 ■), and under P* the delay of 6UXl(z°,S°) is independent of 0Xl(Z°, S°). Basically, (3.7) holds because the change of measure dP* = Xj/EpGjdP only affects the first cycle of (Z°,S°), which is independent of 6Xl (Z°,S°), and because the delay of 8uXl(Z°,S°) is obtained by placing the origin at random in that cycle. Here is a more detailed proof of (3.7). The density A"i/E[Xi] is a measurable function of C\ and (thus) is independent of 6Xl(Z°,S°) under P. This implies that replacing P by P* does not change the distribution of eXl{Z°,S°) nor the fact that 6Xl(Z°,S°) is independent of Ci [use Lemma 4.1 in Chapter 8 with Y = C\ and with V any nonnegative
Section 3. Classical Regeneration 349 measurable function of 8Xl(Z°,S0)]. Neither does it change the fact that U is uniformly distributed on [0,1) and independent of (Z°,S°) [apply Lemma 4.1 in Chapter 8 with Y = (Z°,S°) and with V any nonnegative measurable function of U]. Thus P*(6Xl(Z°,S°) €•) = P(6Xi(Z°,S°) e-), and under P* the triple (U,Xi,C\) is independent of 9Xl (Z°, 5°). Combine this and P(0Xl(Z°,S°) e •) = f((Z°,S°) e •) and the observation that 9UXl Cx is the delay of 0UXl (Z°,S°) to obtain (3.7). D 3.4 The Key Coupling Result In order to understand why Theorem 3.2 below is the key coupling result for classical regenerative processes it may be useful to consider briefly the exact coupling (Section 3 of Chapter 4) problem in this case. If two independent classical regenerative processes happen to regenerate at the same time (say T = Sk = S'k1), then we can switch from one to the other at that time without changing the distributions of the processes, that is, we would have created an exact coupling. Now, although simultaneous regeneration may take place if the regeneration times are lattice valued, it almost surely does not take place when the regeneration times are continuous, unless we construct dependence between the processes that forces them to regenerate simultaneously. Such construction can be quite cumbersome. It might be easier to create a distributional exact coupling: if we could stop the two sequences of regeneration times in such a way that the two stopped regeneration times (say T = Sk and T" = S'k1) have the same distribution, then we would have created a distributional exact coupling. For this purpose we shall need the following generalization of the stopping time concept: a nonnegative integer-valued random variable if is a randomized stopping time with respect to a sequence of random elements (Yjt)g° if for each n ^ 0, the event {K = n} depends on (Yjfe)o° only through (Yo, • ■ •, Yn). A stopping time is the special case when for each n ^ 0, the event {K = n} is in the cr-algebra generated by (Yo,..., Yn). A randomized stopping time allows the stopping event {K = n} to be determined not only by (Yq,. .. ,Yn) but also by some additional randomization, as long as it is conditionally independent of (Yn+i,Yn+2, ■ ■ ■) given (Yo,.. ■ ,Yn). More generally, we say that K is a randomized stopping time with respect to (Yjt)o° in the presence of a random element Y if for each n ^ 0, the event {K — n} and the random element Y depend on (Yjfc)o0 Only through (Y0,..., Y"n). For properties of conditional independence, see Section 4.4 of Chapter 3. The following theorem reduces the distributional coupling problem for classical regenerative processes to a distributional coupling problem for
350 Chapter 10. REGENERATION the regeneration times (the random variable R will be needed for epsilon coupling and can be dropped in the exact coupling case). Theorem 3.2. Suppose (Z,S) is classical regenerative. Let (S,K,R) be such that R is a random variable and S = S, (3.8a) K is a randomized stopping time w.r.t. S in the presence of R. (3.86) Then the probability space (Q.,!F,P) on which (Z,S) is defined can be extended to support (K: R) such that (S,K,R) = (S,K,R), (3.9a) K is a randomized stopping time w.r.t. S in the presence of R, (3.96) (K, R) is conditionally independent of Z given S. (3.9c) Moreover, with T = Sk it holds that (T,R)£(Sk,R), (3.10a) (T,R) is independent of6TZ, (3.106) eTZ = Z°. (3.10c) PROOF. Apply the transfer extension in Section 4.5 of Chapter 3 to obtain (3.9a) and (3.9c) from (3.8a). From (3.86) and (3.9a) it follows that (3.96) holds. From (3.9a) it follows that (3.10a) holds. In order to establish (3.106) and (3.10c), note that [due to (3.9c)] the event {K = n} and the random variable R depend on (Z, S) only through S and [due to (3-96)] on S only through (So, ■ ■ ■, Sn). Thus {K = n} and R depend on (Z,S) only through (So,.. .,Sn). Thus {K = n} and R depend on 9snZ only through (So, • • •, £„). Since 9snZ and (So,- ■ ■ ,S„) are independent, this means that 0$nZ is independent of (Sn,R) and {K = n}. This and P(0SnZ e ■) = P(Z° e ■) yield the second identity in P(9sKZe-,(SK,R)e-,K^n) = P(6SnZe-,(Sn,R)e-,K = n) = P(Z°e-)P((Sn,R)E-,K = n) = P(Z° e-)P((SK,R)e-,K = n). Sum over n to obtain (3.106) and (3.10c). D This is a convenient place for the following analogue of the strong Markov property.
Section 3. Classical Regeneration 351 Lemma 3.1. Let (Z,S) be classical regenerative. Suppose K is a stopping time with respect to S or, more generally, K is a randomized stopping time with respect to S and conditionally independent of Z given S. Then 8Sk(Z,S) = (Z°,S°), 9Sk(Z,S) is independent of ((Zs)s€[0tsK),S0,... ,Sk)- Proof. In the above proof replace R by ((Zs)s€[0 sK),So, ■ ■ ■, Sk) and Z by (Z,S). ' □ 3.5 Lattice or Spread-Out Xi - Exact Coupling In this subsection and the next many ideas developed in the previous chapters converge through Theorem 3.2 above. Recall that the random variable X\ is spread out if there exists ann^l and an / e B+ such that fR f{x) dx > 0 and, with X2, ...,Xn i.i.d. copies ofXj, P(X1+---+X„e£) ^ / f{x)dx, BeB. Jb In particular, a continuous Xx is spread out (simply take n = 1 and / the density of X\ with respect to Lebesgue measure). On the other hand, a discrete X\ is not (since then X\ + ■ ■ ■ + Xn is discrete for each n ^ 1). Theorem 3.3. Let (Z, S) be classical regenerative and (Z1, S') be a version of(Z,S). Suppose either Xi is spread out, or X\ is lattice with span d, and So and S'0 are both d!L valued. Then the following claims hold. (a) The underlying probability space (Q.,Jr,P) can be extended to support finite random times T and T" such that (Z, Z',T,T') is a successful distributional exact coupling, that is, (0tZ,T) = (9T'Z',T')- (3-H) Moreover, if there exists a weak-sense-regular conditional distribution of Z given 9tZ [this holds when (E,£) is Polish and the paths are right-continuous], then (ft,Jr,'P) can be further extended to support a copy Z" of Z' such that (Z,Z",T) is a successful nondistributional exact coupling of Z and Z', that is, 0TZ = eTZ" where Z" = Z'. (3-12)
352 Chapter 10. REGENERATION (b) With || • || denoting total variation (see Section 8.2 of Chapter 3) we have \\P(6tZ e-)-P(6tZ'e-)\\-+0, i->oo. (3.13) Moreover, ifE[Xx] < oo and X\ is spread out, then 9tZ%Z\ i-»oo, (3.14) while i/E[Xi] < oo and X\ is lattice with span d and So is d!L valued, then BndZ%Z**, n->oo. (3.15) (c) With T the tail a-algebra on 7i (Section 9.1 of Chapter 4) we have P(Ze-)|r = P(^'G-)|r- (d) The process Z is T-trivial and mixing (Section 2.1 in Chapter 6). Proof, (a) If Xx is lattice with span d, and So and S'0 are both dZ valued, then S/d and S'/d are integer-valued random walks with aperiodic step- lengths, and Theorem 7.2 in Chapter 2 yields the existence of (5, S", K, K') such that S = S and K is a randomized stopping time w.r.t. S, (3.16) 5' = 5' and K' is a randomized stopping time w.r.t. S", (3-17) If Xi is spread out, then we obtain (S,S',K,K') with these properties from Theorem 6.1 in Chapter 3. Due to (3.16) and Theorem 3.2 above, (fi, T, P) can be extended to support a T such that T = 5^-, T is independent of 9tZ, 9tZ = Z°. Due to (3.17) and Theorem 3.2, (fl, T, P) can be further extended to support a T" such that T' = S'k,, T' is independent of 0T<Z', BT,Z' ~Z°. Combine Sk = S'k, and T = Sk and T' = S'k, to obtain
Section 3. Classical Regeneration 353 and combine Q^Z = Z° and Ot'Z' = Z° to obtain uj-Z = vj-i Z . Since in addition, T is independent oiOrZ, and T" is independent of 8t'Z', this yields (3.11). Apply Theorem 3.2 in Chapter 4 to obtain Z" such that (3.12) holds. (b) To obtain (3.13) use (a) and Theorem 9.4 in Chapter 4. To obtain (3.14) from (3.13) take Z' = Z* and use P(0tZ* £ •) = P(Z* <E •)• To obtain (3.15) from (3.13) take Z' = Z**and use P(6ndZ** e-) = P(Z** e •)■ (c) To obtain (c) use (a) and Theorem 9.4 in Chapter 4. (d) Due to Theorem 2.1 in Chapter 6, Z is T-trivial and mixing if we can establish that \\p(PtZe-\zeB)-P(ptZ€-)\\-+o, t-x», (3.18) for all £? of the form B = {zeH:ztl eAlt...,ztn £ An}, (3.19) where n^l,0^£i <••■<£„ and A\,..., An € f. In order to prove (3.18), note that (Z, (Sjv, +*)o°) ls a version of (Z, 5) [see Lemma 3.1], and that the event {Z E B} is in the cr-algebra generated by the delay of (Z, (S'jv,^ +fc)o°) because Swtn is greater than tn. The delay of (Z, (Sn( +*)o°) ls independent of the cycles, and thus so is {Z E B}. It follows that a pair (Z1, S') with distribution P((Z, (SNtn+k)(?) G -\Z £ B) is a version of (Z,S). Thus (6) yields (3.18). " D Remark 3.1. Let 5 be a renewal process and 5° its zero-delayed version. Consider this special case of (3.13): ||P(Bt E-)-P(B° e-)ll ->0, t->oo. (3.20) Theorem 3.3 has the following converse. If So is continuous and (3.20) holds, then X\ is spread out, since otherwise B°t would be singular with respect to Lebesgue measure for all t, which together with the observation that Bt is continuous for all t would imply \\P(Bt E ■) - P(jB(0 e -)|| = 2 for all t, thus contradicting (3.20). If Xi is dTL valued and S0 = d and (3.20) holds, then Xi is lattice with span d, since otherwise there would be a k > 1 such that P{B^kd € kdL) = 1 and P{Bnkd e d+kdZ) = 1 for alln, implying ||P(Bnfcd <E -)-P(B°nkd E -)|| = 2 for all n and thus contradicting (3.20).
354 Chapter 10. REGENERATION Remark 3.2. Let (Z,S) be classical regenerative and (Z',S') be a version of (Z, S). If Xi is lattice with span d, then 0somodd(Z,S) and 0s^modd(Z', S') are versions of (Z, S) with dZ valued delay-lengths, and Theorem 3.3(a) yields the existence of finite times T and T" such that (SrOsomoddZ, T) = (0T'8s'omoddZ',T'). Thus if Xi is lattice with span d, then for all delay-lengths there is a successful distributional exact coupling modulo a time shift that is strictly less than d. In particular, when this coupling can be made nondistributional, we have OrOsomoddZ = OrOs^'moddZ", where (Z",S'd) = (Z',S'0). Since \(T + (S0 mod d)) - (T + (S'd mod d))\ < d, this means that (Z, Z", T + (So mod d), T + (S'd mod d)) is a successful nondistributional d-coupling of Z and Z'. Remark 3.3. Classical regeneration is a special case of time-inhomogeneous regeneration, which we shall consider in Sections 5 through 8. Thus the results on uniform convergence and rates of convergence established there also apply to classical regenerative processes; see Section 7.5 below. 3.6 Nonlattice X\ — Epsilon-Couplings Recall that the random variable X\ is nonlattice if P(Xj e dZ) < 1 for all d > 0. Any spread out X\ is nonlattice, and so is, for instance, a discrete X\ taking the values 1 and y/2 with strictly positive probabilities. Theorem 3.4. Let (Z, S) be classical regenerative and (Z', 5') be a version of(Z,S). Let U be uniform on [0,1) and independent of Z and Z'. Suppose X\ is nonlattice. Then the following claims hold. (a) For each e > 0, the underlying probability space (Cl,!F, P) can be extended to support finite random times Ts, T'E, Re, and R'E such that (Z,Z',T£,T'£,R£,R'£) is a successful distributional e-coupling of'Z and Z', that is, \Te-Re\<e and \T'E-R'E\<e, (3.21) (6TcZ,Te,Re) i (0TiZ',R'£,Te)- (3.22)
Section 3. Classical Regeneration 355 Moreover, if there exists a weak-sense-regular conditional distribution of Z given OtZ for each random time T [this holds when (E, £) is Polish and the paths are right-continuous], then for each e > 0, (Cl,!F,P) can be further extended to support a copy Z^> of Z' such that (Z,Z(e\Te,Re) is a successful nondistributional e-coupling of Z and Z', that is, \T£ - Re\ < e and 6TtZ = 9RsZ{e), where Z{e) = Z'. (3.23) (a!) For each h > 0, (CI, T, P) can be extended to support finite random times Th and T'h such that (9uhZ,0uh.Z'',Th,T'h) is a successful distributional exact coupling of 8uh.Z and 6uh.Z''> that is, (0Th0uhZ,Th) £ (6T,h6UhZ',T'h). (3.24) Moreover, if there exists a weak-sense-regular conditional distribution of Z given 8tZ for each random time T [this holds when (E, £) is Polish and the paths are right-continuous], then for each h > 0, (Cl, T, P) can be further extended to support a copy (Z^h\Uh) of (Z',U) such that (6uh.Z,9uhhZ(h>,Th) is a successful nondistributional exact coupling of Q\jhZ and 9uh,Z', that is, OtJuhZ = 6Th0uhhZ(h), where (Z<"»,Uh) = (Z',U). (3.25) (b) For each h > 0, \\P(6t+uhZe-)-P(6t+UhZ'e-)\\^0, i^oo. (3.26) Moreover, if F,[Xi] < oo, then the following claims hold. (i) For each h > 0, Ot+uhZ lAZ*, t ->• oo. (3.27) (ii) If the paths are piecewise constant with finitely many jumps in finite intervals, then Zt A Zq, t -> oo. (m) With P* as in Theorem 2.1 we have P(CNte-)%P*(Cxe-), t^oo, (3.28) and p(eSNt^ze-)tAp*(z°e-), t^oo. (3.29)
356 Chapter 10. REGENERATION (iv) If E is metric, £ its Borel subsets, and the paths right-continuous, then Zt —> Zq, t —> oo. (v) If (E,£) is Polish and the paths are right-continuous with left- hand limits, then OtZ^Z*, t^t oo, where —> means weak convergence in the Skorohod topology. (c) With S the smooth tail a-algebra on 7i {Section 9.1 in Chapter 3) we have p(Ze-)\s = P(z'e-)\s, and with T the tail a-algebra on 7i we have P(9UhZ e -)|r = P(OuhZ' e -)|r for each h > 0. (d) The process Z is S-trivial and smoothly mixing {Section 2.3 in Chapter 6). PROOF, (a) Fix e > 0. Theorem 7.1 in Chapter 2 yields the existence of (S,S', K,K') such that S — S and K is a randomized stopping time with respect to 5 in the presence of S'k,, 5' = 5' and K' is a randomized stopping time with respect to 5' in the presence of Sk, (3.30a) (3.306) \Sk-S'k,\<e. (3.30c) Due to (3.30a) and Theorem 3.2 above, (fi, T, P) can be extended to support (Te,Re) such that (TE,RE)^(Sk,S'k,), (3.31a) (TE,RE) is independent of 6Te Z, (3.316) 8T,Z = Z°. (3.31c)
Section 3. Classical Regeneration 357 Due to (3.306) and Theorem 3.2, (Cl,!F, P) can be further extended to support (TE,R'E) such that (ft,R'e)2(S'k„Sk), (3.32a) (Te', R'E) is independent of 6T, Z', (3.326) 6T-CZ' = Z°. (3.32c) From (3.30c), (3.31a), and (3.32a) we obtain (3.21) and (Te,RE)^(R'EX)- (3-33) From (3.31c) and (3.32c) we find that BTcZ = 6T<Z', which together with (3.33), (3.316), and (3.326) yields (3.22). Apply Theorem 6.2 in Chapter 5 to obtain Z^ such that (3.23) holds. (a') To obtain (3.24) and (3.25), use (a) and Corollary 9.1 in Chapter 5. (6) To obtain (3.26), use (a) and Theorem 9.4 in Chapter 5. To obtain (3.27) from (3.26), take Z' = Z* and use P(8t+uhZ* e ■) = P{Z* <E •)• To obtain (ii), use (a) and Theorem 7.2 in Chapter 5. To obtain (3.28) from (n), note that the process with value Cn, at time t [the value D if Nt = 0] is piecewise constant with finitely many jumps in finite intervals, is classical regenerative with regeneration times S, and has stationary version having marginal distribution P*(Ci € •) at time zero (see Theorems 2.1 and 3.1). To obtain (3.29) from (3.28), note that HPC^.^eO-P'^eOII = \\P((CNt,8SNtz) e ■) - P*((Cu9Xlz°) e OH = ||P(CJVte-)-P*(C'1e-)ll, where the latter equality is due to the facts that Cn, and 9sN Z are independent under P, that C\ and 0XiZ° are independent under P*, and that P{0sNtZ e •) = P*{0XlZ° e •) [see (3.3) in Lemma 3.1 of Chapter 6]. To obtain (iv), use (a) and Theorem 7.3 in Chapter 5. To obtain (v), use (a) and Theorem 7.4 in Chapter 5. (c) To obtain (c), use (a) and Theorem 9.4 in Chapter 5. (d) Due to Theorem 2.3 in Chapter 6, the process Z is 5-trivial and smoothly mixing if for all B as at (3.19) and all h > 0, \\P(6t+UhZ E-\Z EB)- P(6t+UhZ e -)|| ->• 0, * ->• oo. This follows from (6) by repeating the proof of (3.18) with 9tZ replaced by 8t+UhZ. □
358 Chapter 10. REGENERATION Remark 3.4. Let 5 be a renewal process and 5° its zero-delayed version. Consider this special case of (3.26): for all h > 0, \\P(Bt+Uhe.)-P(B°+Uh)e-)\\^0, *^oo. (3.34) Theorem 3.4 has the following converse. If So is exponential and (3.34) holds, then Xi is nonlattice, since otherwise there is a d > 0 such that P(-^nd+t/d/2 e (d/2,d] + dZ) = l and P(Bnd+Ud/2 E (0,d/2] -f- dZ) > P(d/2 < 50 < d) for all n, implying \\P(Bnd+Ud/2 e •) - P(S°d+[/d/2 e -)ll £ 2P(d/2 < 50 < d) > 0 for all n, and thus contradicting (3.34). 4 Wide-Sense Regeneration - Harris Chains - GI/GI/k It turns out that all the results from the previous section (except those on mixing and triviality) still hold if we allow the future after regeneration to depend on the past, as long as the future is independent of the past regeneration times. For lack of a better term we shall call this wide-sense regeneration. If the dependence lasts only over a time interval of length I, then the regeneration is lag-l (in this case the results on mixing and triviality hold). At the end of this section we show that this kind of regeneration occurs in so-called Harris chains and in the GI/GI/k queueing system. 4.1 Definitions Call a one-sided shift-measurable stochastic process Z wide-sense regenerative with regeneration times S if 0Sn(Z,S) = (Z°,S°), n>0, (4.1) and 0Sn(Z,S) is independent of (50,...,5„), n ^ 0. (4.2) Call the pair (Z, S) wide-sense regenerative if this holds. If (Z, S) is wide- sense regenerative, then the cycles are in general not i.i.d. but S is still a renewal process. Let the nonnegative random variable 5_i be such that (S_!,So) is independent of (Z°,S°). With I ^ 0, call a wide-sense regenerative (Z,S) lag-l regenerative if (4.2) can be strengthened to: for n > 0, 9Sn(Z,S) is independent of ((2s)(,e[0i(Sn_0+],50, ■ • • ,S„); (4.3)
Section 4. Wide-Sense Regeneration - Harris Chains - GI/GI/k 359 and lag-l+ regenerative if (4.2) can only be strengthened to: for n ^ 0, 6Sn(Z,S) is independent of ((Zs)s€[Q^s„-i)+),So, ■ • ■ ,Sn). Thus lag-0+ regeneration is the same as classical regeneration, while lag-0 regeneration implies further that Zsn is nonrandom. A pair (Z1,S") is a version of a wide-sense regenerative (Z, S) if (Z1, S') is also wide-sense regenerative and 0s.(Z',S') = (Zo,So). (4.4) A pair (Z',S') is a version of a lag-/ regenerative (Z,S) if (Z',S') is also lag-/ regenerative and (4.4) holds. In both cases (Z°,S°) is a zero-delayed version of (Z,S). Note that if (Z, S) is classical regenerative, then (Z, S) is in particular wide-sense regenerative, and that if (Z', 5') is a wide-sense version of (Z, S), then (Z',S') need not be a classical version of (Z,S), since the delay of (Z',S') need not be independent of the cycles. 4.2 Observations - Examples Suppose (Z, S) is classical regenerative and / is a measurable mapping from {H,7i) into some measurable space. Then the process (/(#sZ))se[o,oo) is in general not classical regenerative. It is, however, wide-sense regenerative with regeneration times 5. In particular, the path process (#sZ)sg[o,oo) [which has state space (H, 7i)] is wide-sense regenerative with regeneration times 5 but certainly not classical regenerative (unless Z is a nonrandom constant). In fact, the following conservation properties hold: If Z is classical regenerative with regeneration times 5, then so is (/(Zs))s6[oj00) for all measurable mappings / from (E,£) into some measurable space. If Z is wide-sense regenerative with regeneration times 5, then so is (f(8sZ))se[0]0O) for all measurable mappings / from (H,H) into some measurable space. Note that a Markov process does not have these conservation properties: If Z is a Markov process, then in general (/(#s-£))se[o,oo) is n°t a Markov process and neither is (/(■^s))ss[o>oo)- On the other hand, the following holds: For any stochastic process Z the path process (0sZ)s6[o,oo) is always a Markov process.
360 Chapter 10. REGENERATION This is because the state of the path process at time t (namely 6tZ) determines its complete future (since 9t+sZ = 0s6tZ for s ^ 0), and therefore trivially the future depends on the past only through the present. If Z is any stationary process and 5 is a renewal process that is independent of Z, then (Z, S) is wide-sense regenerative but in general not classical regenerative. If 5 is a renewal process, then the process (Ns+i — iVs)sg[o,oo) is lag-' regenerative with regeneration times S for any I > 0, but in general not classical regenerative. If Z is wide-sense regenerative (or lag-/ regenerative) with regeneration times 5, then so is (ZS,AS,BS,DS, Us, Ns+t - iVs)s6[0]Oo). Consider a random walk with step-lengths that have a strictly positive expectation. Let 5 be the ladder heights renewal process, that is, let Sq be the first nonnegative state of the random walk, and recursively for n > 1 let 5„ be the first state of the random walk that is greater than 5„_i. Let Z = (Zs)sg[o,oo) be the process with Zs the number of times the random walk visits the interval (s,s + I], where I > 0. Then (Z,S) is wide-sense regenerative but in general not lag-/ regenerative. More substantial examples are given at the end of this section. 4.3 Extension of the Theorems from the Classical Case The stationarity result (Theorem 3.1) holds in the wide-sense case: Theorem 4.1. Suppose (Z, S) is wide-sense regenerative with E[Xi] < oo. Then (Z*,S*) in Theorem 2.1 is a stationary version of(Z,S). Further, if Xi is lattice with span d, then (Z**,S**) in Theorem 2.2 is a periodically stationary version of (Z, S) with period d. PROOF. In the proof of Theorem 3.1 replace the delays by the delay lengths and the cycle C\ by its length X\ to obtain the desired result. □ Distributional coupling is even more useful in the wide-sense case than in the classical case. Note that if two independent versions of a wide-sense regenerative process regenerate at the same time, then we can in general not switch from one to the other without changing the distribution of the process. This is because the future after regeneration is not independent of the past. However, after a simultaneous wide-sense regeneration the processes do continue in the same way distrihutionally (since the simultaneous regeneration only has to do with the past regeneration times and thus does not affect the future), and this is all we need to have a distributional exact coupling. More generally, in order to obtain a distributional exact coupling it is reasonable to expect that (as in the classical case) we only need to be able to stop the two sequences of regeneration times in such a way that the two stopped regeneration times have the same distribution.
Section 4. Wide-Sense Regeneration - Harris Chains - GI/GI/k 361 In fact, the key coupling result (Theorem 3.2) extends to the wide-sense case: Theorem 4.2. Theorem 3.2 holds with (Z,S) wide-sense regenerative. PROOF. The proof of Theorem 3.2 needs no modification to cover the wide- sense case. □ Also, the analogue of the strong Markov property extends to the wide-sense case. Lemma 4.1. Let (Z,S) be wide-sense regenerative. Suppose K is a stopping time with respect to S or, more generally, K is a randomized stopping time with respect to S and conditionally independent of Z given S. Then 9sK (Z, S) is a copy of (Z°, S°) and is independent of (So,..., Sk). If further (Z,S) is lag-l regenerative for some I ^ 0, then in addition, 9Sk(Z,S) is independent of ((Zs)se[0^sK~i)+],S0, ■ ■., SK)- Proof. In the proof of Theorem 3.2 replace Z by (Z,S) and replace R first by (So, ■ ■., Sk) to obtain the first statement and then replace R by ((^s)«€[o,(Sk -()+] > ^o, ■ • •, Sk) to obtain the second statement. □ The exact coupling theorem (Theorem 3.3) holds in the wide-sense case, except the mixing and T-triviality result: Theorem 4.3. Let (Z,S) be wide-sense regenerative and (Z',S') be a version of(Z,S). Suppose either X\ is spread out, or X\ is lattice with span d, and So and S'0 are both d!L valued. Then the claims (a), (b), and (c) in Theorem 3.3 hold. The claim (d) holds if (Z, S) is lag-l regenerative for some I ^ 0, but not in general. PROOF. The proof of Theorem 3.3(a, b, c) needs no modification to cover the wide-sense case. In order to establish (d) in the lag-/ case modify the proof of (3.18) by considering (Z, (SNtn+r+k)^) rather than (Z, (5Arln+A.)g°). As a counterexample to (d) in the general case take the process that is in state 1 at all times with probability \ and in state 0 at all times with probability |. This process is neither T-trivial nor mixing, but it is wide- sense regenerative with any independent renewal process as regeneration times. □
362 Chapter 10. REGENERATION The epsilon-coupling theorem (Theorem 3.4) also holds in the wide-sense case, except the smooth mixing and <S-triviality result: Theorem 4.4. Let (Z, S) be wide-sense regenerative and (Z', S') be a version of (Z,S). Let U be uniform on [0,1) and independent of Z and Z'. Suppose X\ is nonlattice. Then the claims (a), (b), and (c) in Theorem 3-4 hold. The claim (d) holds if (Z, S) is lag-l regenerative for some I ^ 0, but not in general. Proof. The proof of Theorem 4.3(a, 6, c) needs only modification in one place to cover the wide-sense case. Rather than deducing P(fl5iV,-1^e-)^P*(^0e-)» t^oo, [this is (3.29)] from (3.28), it suffices to observe that now this follows [like (3.28)] from the (n)-part of (6), since (^s^_1^)ss[o,oo) ls regenerative in the wide sense with piecewise constant paths having jumps at the regeneration times S and since the stationary version has marginal distribution P*(Z° € •) at time zero. In order to establish (d) in the lag-Z case repeat the proof of (3.18) with 6tZ replaced by 9t+uh,Z and (Z, (SNtn+k)^) by (Z, (SNtn+,+k)c?)- As a counterexample to (d) in the general case again take the process that is in state 1 at all times with probability | and in state 0 at all times with probability |. This process is neither 5-trivial nor smoothly mixing, but it is wide-sense regenerative with any independent renewal process as regeneration times. □ 4.4 Existence of Regeneration Times We shall now show that (maybe not surprisingly) we need only two regeneration times So and S\ to have the whole sequence S (see the corollaries). Theorem 4.5. Let Z be a one-sided shift-measurable stochastic process and Sq and Si random times such that 0 ^ So < S\ and 6SlZ^6SoZ. (4.5) Then the underlying probability space (ft, T, P) can be extended to support a sequence of random times S such that 6Sn(Z,S)^(Z°,S°), n^O, (4.6) (Z,So,S\) depends on (X2,X3,...) only through 0s1Z. (4.7) Comment. It can be deduced from (4.6) and (4.7) that for each n ^ 1, (Z,S0,---,Sn) depends on (Xn+l,Xn+2, ■ ■ ■) only through 0SnZ.
Section 4. Wide-Sense Regeneration - Harris Chains - GI/GI/k 363 Proof. We shall apply the transfer extension from Section 4.3 of Chapter 3 recursively. Also, the following result from Lemma 4.1 in Chapter 3 will be used repeatedly: for any random elements Yq, Yx, Y2, and Y3 it holds that Y3 depends on Y2 only through (Yi,Yq) and on Y\ only through Y0 if and only if Y3 depends on (Y2,Y\) only through Y0. Use (4.5) and the transfer extension to obtain X2 such that (Z, So, Si) depends on X2 only through Qs^Z and (9SlZ,X2) = (Za,Xl). (4.8) Use (4.8) and the transfer extension to obtain X3 such that (Z, So, Si) depends on X3 only through (6slZ,X2) and (6SlZ,X2,Xs) = (Z°,X1,X2). Repeat this countably many times to obtain a sequence X2,X3,... such that for n ^ 1, (6SlZ,X2,...,Xn+1) = (Z°,X1,...,Xn), (4.9) (Z,So,Si) depends on Xn+1 only through (6SlZ,X2,.. .,Xn). (4.10) From (4.9) we obtain ^(Z, S) = (Z°,S°), which in turn implies that 6Sn{Z,S) = (Z°,S°) holds for all n > 0, that is, (4.6) is established. In order to obtain (4.7), note that due to (4.10) the following claim holds for k = 2: (Z,S0,Xi) depends on (X2,...,Xk) only through 6SlZ. (4.11) Make the induction assumption that (4.11) holds for some k > 2. According to (4.10), (Z,SQ,Xi) depends on Xk+i only through (8SlZ,X2,... ,Xk) and, according to (4.11), on (X2,... ,Xk) only through &sxZ. Therefore {Z,So,Xi) depends on (X2,... ,Xk+i) only through 6s1Z. Thus, by induction, (4.11) holds for all k > 2, that is, (4.7) is established. □ Corollary 4.1. In addition to (4.5), suppose {0saZ,Xi) is independent of D and6s1Z is independent of(D,C\). Then (Z,S) is classical regenerative. Proof. We must show that for all n > 0, (D, d,..., C„) is independent of 0Sn {Z, S). (4.12) Due to (4.7), (D,Ci) depends on {X2,X3,...) only through 6SlZ, and by assumption, (D,Ci) does not depend on 8SlZ. Thus (D,Ci) is independent oi(0SlZ,X2,X3,.--)- Thus (4.12) holds forn = 1.
364 Chapter 10. REGENERATION Make the induction assumption that (4.12) holds for some n ^ 1. We just established that C\ is independent of 9s1(Z, S). Due to (4.6), this implies that Cn+i is independent of 9$n+1(Z, S). Due to (4.12), this implies that (D, C\,... ,Cn+i) is independent of 9sn+1(Z, S). Thus, by induction, (4.12) holds for all n > 1. It remains to establish (4.12) for n = 0. From (4.7) it follows that (X2, X3,...) depends on (D, (8s0 Z, X\)) only through 9Sl Z and thus on D only through ((9s0Z,X\),9s1Z). Since 0s1Z = 9x19s0Z, this means that (X2,X3,...) depends on D only through (9s0Z,Xi). Since by assumption D is independent of (9s0Z,Xx), this implies that D is independent of (0SoZ,X1,X2,...), that is, (4.12) holds also for n = 0. D Corollary 4.2. In addition to (4.5), suppose {9g0Z,X\) is independent of So, and 9s1Z is independent of (So, Si). Then (Z,S) is wide-sense regenerative. Proof. We must show that for all n > 0, (S0, Xi,..., Xn) is independent of 9sn {Z, S). This follows by replacing D, C\, C2, ■ ■ ■ by So, X\, X2,... in the proof of Corollary 4.1. □ 4.5 Harris Chains — Regeneration Sets — Harris Processes A discrete-time Markov process Z = (Zjt)o° with state space (E,£) and one-step transition probabilities P (see Chapter 6, Section 3.1) is a Harris chain if it has a regeneration set, that is, if there is a set A E £ such that the hitting time of A ta = inf {* > 0 : Zt E A} is finite with probability one for all initial distributions and there is an I > 0, ap£ (0,1], and a probability measure \i on (E, £) such that P(Zle-\Z0=x) = P'(x,-)^Pfi, xeA. (4.13) Note that if E is finite or countable and (^jt)o° ls irreducible and recurrent, then (Zjt)o° is a Harris chain. To see this, put A = {i} with i € E arbitrary, take some I > 0, and put p — 1 and n = T?(Z[ € -\Zq = i). More generally, we could let A be any finite set of states and take I > 0 large enough for the distributions P(Zt € -\Z0 = i), i € A, to have a nontrivial common component p[i. In order to extend regeneration sets to continuous time we need the strong Markov property (this property automatically holds in the discrete- time case). A random time r is a stopping time with respect to a continuous time process Z = (Zs)s6[oi00) if for all t > 0, the event {r ^ t} is in the
Section 4. Wide-Sense Regeneration - Harris Chains - GI/GI/k 365 cr-algebra generated by (Zs)se[0t]. In particular, a measurable hitting time is a stopping time. A shift-measurable Markov process Z = (•^s)ss[o,oo) with semigroup of transition probabilities Fs, 0 ^ s < oo, (see Chapter 6, Section 3.1) is a strong Markov process if the Markov property holds at all stopping times r, that is, if 8TZ depends on (Zs)s€[o,r] only through ZT and P(6Tz e-\zT = x) = p(z e -\z0 = x), x e E. Call a set A € £ a regeneration set for a strong Markov process Z = (Zs)sg[o,oo) if the hitting time ta is measurable and finite with probability one for all initial distributions and ZTA € A, and if there is an I > 0, a p € (0,1], and a probability measure \i on (E,£) such that (4.13) holds. Intuitively, (4.13) means that whenever Z enters A, then it lag-? regenerates I time units later with probability p. The problem is that in general we cannot tell by only observing the process itself whether regeneration is occurring or not. In order to make the lag-? regeneration observable we shall use the splitting extension from Section 5 in Chapter 5. Note that with T0 = TA and Tk+l = inf{* > Tk + I : Zt € A} for k :> 0, we have [due to the strong Markov property] P(ZTk+le-\ZTk=x)=Pl(x,-)^p^ xeA. (4.14) This allows us to extend the underlying probability space by conditional splitting. Apply Corollary 5.1 in Chapter 3 recursively [in step k take (Y0,Y1:Y2) := (ZTk, (Z,I0, ■ ■.,h-i), ZTk+i)] to obtain a sequence of 0-1 variables Iq , h,... such that for k ^ 0, (Z,I0,.. .,h-i) depends on Ik only through (ZTk, ZTk+l), (4.15a) P(/t = l\ZTk =x)=p, xeA, (4.156) P(ZTh+ie-\ZTh=x,Ik = l) = fi, xeA, (4.15c) P(h = l\ZTk = x, ZTk+l =y)= p^J., xeA,yeE. (4.15d) Put Sn = TKn+l, n > 0, (4.16) where Kn is the (n + l)th index k such that Ik — 1.
366 Chapter 10. REGENERATION We shall show that the sequence S forms lag-? regeneration times for Z. To those who possess the magical Markovian intuition this is rather obvious: At the randomized stopping time Sn the Markov process Z has distribution [i and is independent of its state at time Sn — I. Thus its future after time Sn is a version of Z with initial distribution \i and is independent of its past before time Sn — I. In spite of this we shall work through the conditional independence arguments in full detail because the Markovian intuition is delicate and can easily go wrong. Theorem 4.6. Let Z be a discrete- or continuous-time Markov process with a regeneration set A and let S be obtained by splitting as above. Then, with I, p, and pL as at (4.13), Io,h,--- are i.i.d. with P(Io = 1) = p, (4-17) 0snZ is a version of Z with initial distribution pi for n ^ 0, (4-18) and (Z, S) is lag-l regenerative and the distribution of its zero-delayed version (Z°,S°) does not depend on the initial distribution of Z. Comment. Since the cycle-lengths are greater than I, this implies that although the cycles are dependent, they are only one-dependent, that is, for each n > 0, (D, C\,..., C„) and (Cn+2, Cn+z, ■ ■ •) are independent. PROOF. Fix k > 0 and put [observe that Ik is left out] Wk = (0Tk+lZ,Ik + l,h+2,- ■ ■ ), Vk = ((Zs)s6[OTfc],io,- • • ,h-i)- Note that if the Kn are finite, then for each n > 0, @s„(Z, S) is obtained from Wk„ by the same measurable mapping as 9s0 (Z, S) from Wk0 , and ({Zs)B£[o,sn-i]i So, ■ ■ ■, Sn) is obtained by a measurable mapping from Vk„ • Thus if we can show that the Kn are finite with probability one and that Wk„ and Vk„ are independent and Wk„ = Wk0, (4-19) then it follows that (Z, S) is lag-? regenerative. To prove this we shall use the following result from Lemma 4.1 in Chapter 3 repeatedly: for any random elements y0, ^li ^2, and Y3 it holds that Y3 depends on Y2 only through (Yi,Y0) and on Y\ only through Y0 if and only if Y3 depends on (y2, Yi) only through Yq.
Section 4. Wide-Sense Regeneration - Harris Chains - GI/GI/k 367 For 0 ^ i < k, {ZTi,ZT{+i) is a measurable mapping of (■Zs)se[o,Tfc], and thus we deduce from (4.15a) that 6Tk+iZ depends on U only through ((Zs)se[QtTk],Io, ■■■, £-i)- Thus 6Tk+lZ depends on Ik^ only through {{Za)S£[o,Tk], h, ■ ■ -,h-2), and on Ik-2 only through ((Zs)se[0,Tk],Io, ■ • ■, h-z),..., and on I0 only through (Zs)se[o,Tk], and finally [due to the strong Markov property] on {Zs)se[o,Tk] only through Zj-k. Thus Ork+iZ depends on Vk only through Zj-k. (4.20) Due to (4.15a), Vk depends on Ik only through (ZTk,@Tk+iZ) and, due to (4.20), on 8rk+iZ only through Zyk. Thus (0Tk+lZ,Ik) depends on Vk only through ZTk. (4.21) Due to (4.156), Ik is independent of Z?k, and thus (4.21) implies that Ik is independent of (I0,..., h-i)- This and (4.156) yields (4.17), which in particular implies that the Kn are finite with probability one. Due to (4.21), 0Tk+lZ depends on {Vk,Ik) only through (ZTk,Ik, ZTk+l) and, due to (4.15a), on Ik only through {Zj-k,ZTk + I), and [due to the strong Markov property] on Zj-k only through Zj-k+i. Thus 0Tk+iZ depends on (Vk,Ik) only through ZTk+l. (4.22) From (4.15a) it follows that for each n ^ 1, the pair (Vjt,^it) depends on h+n only through (8Tk+iZ, Ik+l,..., Ik+n-i), and on h+n-i only through {6Tk+iZ, Ik+1,..., Ik+n-2), ■■-, and on Ik+l only through 6Tk+lZ, and finally [due to (4.22)] on 8rk+iZ only through Zxk+i- Thus Wk depends on {Vk,h) only through ZTk+l. (4.23) Due to (4.21), Zrk+i is conditionally independent of Vk given (ZTk,h), and due to (4.15c), Zrk+i is conditionally independent of (ZTk,h) given the event {Ik = 1}. Thus Zj-k+l is conditionally independent of Vk given {h = 1}. Thus, due to (4.23), Wk is conditionally independent of Vk given {/* = 1}. (4-24) This (and the fact that the event {Kn ^ k} is in the cr-algebra generated by Vk) yields the third identity in P(wKn e-,vKn e-,Kn = k) = P(Wke-,vke-,Kn^k,ik = i) = P(Wk e-,Vfc E-,Kn^k\Ik = l)P(/t = 1) (4.25) = P(Wk e-\h = i)P(Vt e -,K„ ^ *|/t = i)P(/t = l) = p(wke-\h = i)P(vKn e-,Kn = k),
368 Chapter 10. REGENERATION while for the second and fourth we have used {Kn = k} = {Kn^k, Ik = 1}- Due to (4.15c), given the event {Ik = 1} the conditional distribution of Zj-k+i is n, and due to (4.22), BTk+lZ depends on Ik only through ZTk+l. This [and the strong Markov property] yields that given {ifc = l}, 9rk+i Z is version of Z with initial distribution/^. (4.26) For n > 1, due to (4.15a), Ik+n depends on (Ik,9Tk+iZ, Ik+i,..., h+n-\) only through (Zrk+n, ZTk+n+i), and due to (4.15d), the conditional distribution of Ik+n given the value of (ZTk+n,ZTk+n+i) does not depend on k. This and (4.26) yields P(W* e-|/t = i) = P(w0e-1/0 = 1). Put this into (4.25) and sum over k > 0 to obtain P(wKn e -,vKn e •) = P(w0 e -\i0 = i)P(VKn e ■), n > o. (4.27) This identity yields (4.19), that is, (Z,S) is lag-? regenerative. Also, we obtain from (4.27) that P{0snze-) = -p(0To+iZe-\io = i), n>o, which together with (4.26) yields (4.18). Finally, due to (4.26), the distribution of 9t0+iZ does not depend on the initial distribution of Z and, due to (4.15a), neither does the conditional distribution of (7i,/2, • • •) given the value of 8t0+iZ and thus, due to (4.27), neither does the distribution of (Z°,S°); and the proof is complete. □ Remark 4.1. Note that in the discrete-time case, when I = 1, then (Z,S) is in fact classical regenerative (since in discrete time lag-1 regeneration means classical regeneration). Remark 4.2. Consider the discrete-time case and let the state space (E, £) be Polish. A discrete-time Markov process Z = (Zjt)o° wit^ a regeneration set is a generalization of the regeneration appearing in irreducible recurrent Markov chains. The following concept is a generalization of the recurrence property of such processes. Let <p be a a-finite measure on (E, £). A discrete- time Markov process Z with state space (E,£) is <p-recurrent if for B E £ such that ip{B) > 0, the hitting time rjg is finite with probability one for all initial distributions or, equivalently, the set B is visited infinitely many times with probability one. Note that a discrete-time irreducible recurrent Markov chain has this property with (p the counting measure. In fact, <p-recurrence for some (p is the original definition of a Harris chain: it can be shown (see Orey (1971)) that <p-recurrence for some <p and the existence of a regeneration set are equivalent properties. Relying on this deep theorem, we can rather easily add one more equivalence result in the
Section 4. Wide-Sense Regeneration - Harris Chains - GI/GI/k 369 discrete-time version of Theorem 5.1 in Chapter 6 (see Theorem 4.6 above and Theorem 2.2 in Chapter 5 for the 'only-if part and Glynn (1982) for the 'if part): A Markov process Z = (Zjt)o° possessing a stationary distribution is a Harris chain if and only if there is a successful shift-coupling of each pair of differently started versions of Z. A Harris chain is aperiodic if the inter-regeneration times are aperiodic (this can be shown not to depend on the choice of A, I, and p at (4.1)). We can add one more equivalence result in the discrete-time versions of Theorem 4.1 in Chapter 6 (see Theorem 4.6 and Theorem 3.3(a) for the 'only-if part and Proposition 3.1.3 in Asmussen (1987) for the 'if part): A Markov process Z = (Zjt)o° possessing a stationary distribution is an aperiodic Harris chain if and only if there is a successful exact coupling of each pair of differently started versions of Z. Remark 4.3. Consider a continuous-time strong Markov process Z = (Zs)s6[0jOO) with a Polish state space and right-continuous paths having left-hand limits. Then Z is a Harris process if it is ip-recurrent for some (p, that is, if there exists some cr-finite measure (p on (E,£) such that for B € £ with tp(B) > 0, the total time spent by Z in the set B is infinite with probability one for all initial distributions. Glynn (1994) shows that if Z has a stationary distribution, then Z is a Harris process if and only if for all initial distributions and all A € "H, P(6UtZ e A) -+P(Z* e A), t-too, where Z* is the stationary version of Z and U is uniformly distributed on [0,1] and independent of Z and Z*. Thus in Theorem 5.1 of Chapter 6 we can add that if Z has a stationary distribution, then the equivalent claims (a) through (<?) hold if and only if Z is a Harris process. In continuous time the relation between (^-recurrence for some (p (the Harris process property) and the existence of a regeneration set is not clear at this point, which is why we content ourselves with proving only Theorem 4.6 here in the hope that this theory will be more fully developed in the future. 4.6 The Queue GI/GI/k We shall end this section by sketching an interesting example of lag-/ regeneration in queueing theory. Consider the multiserver queueing model GI/GI/k, namely the fc-server queueing system, 2 ^ k ^ oo, where customers arrive at times r0 < T\ < r2 < ■ ■ • forming a renewal process and line up to be served under the first-come-first-served discipline with i.i.d. service times /3i, /32, ■ ■ ■ that are
370 Chapter 10. REGENERATION independent of the arrival process. Let Qi,q2,--- denote the i.i.d. inter- arrival times, that is, an = rn — rn_i. Let Qt denote the queue length (that is, the number of customers in the system) at time t and Rt the k- dimensional vector of remaining service times of the customers being served at time t ordered in decreasing manner (with zeros when some servers are idle). Let (Qs,Rs)se[o,oo) be right-continuous. Let the initial conditions (<2o-,-Ro-,T0) be independent of (Qi,a2,... ,/?i,/?2, ■ ■ ■ )■ In Section 3.1 above we pointed out that in the single-server case GI/GI/l under the traffic intensity condition E[^] < E[qi] (negative drift) the process (Qs,Rs)se[o,oo) IS classical regenerative with the times of arrivals to an idle system [the subsequence of r„ such that QTn _ = 0] as regeneration times. This is, in fact, also true in the multiserver case GI/GI/k, 2 $C k ^ oo, under the natural extension of the traffic intensity condition from the single-server case: E[/3X] < feE[ai] (negative drift), provided that all servers can be idle at the same time, that is, provided that the following additional condition holds: or equivalently, there is a y > 0 such that P{/3\ ^ y) > 0 and P(qi > y) > 0. Note that if P(/3i ^ oti) = 0, then whenever the queue length gets down to one, the next arrival will take place before the service of the remaining customer is completed, that is, the queue cannot empty. In the single-server case the possibility of P(/?i ^ Qi) = 0 is ruled out by E[/?i] < Efaj], but when only Ef/^] < /cE[qi] holds, then T?{f3\ ^ Qj) = 0 is a possibility. Although the arrivals to an idle system are not regeneration times in the multi-server case when P(/3i ^ ax) = 0, it turns out (maybe surprisingly) that there are other times where lag-? regeneration occurs. Consider the queue GI/GI/k with 2 ^ k ^ oo and assume that E[/?1]<fcE[a1] and P(ft ^ ai) = 0. Note that P(^ ^ ax) = 0 holds if and only if there is a y > 0 such that P{/3i > y) = 1 and P(qi < y) = 1. Due to E[/?i] < A;E[qi], we have P(/81 < on + ha*) > 0. This and P(/?i ^ e*i) = 0 imply that there is 1 ^ i < k such that P(/?i < Qi -h • ■ • + Qi) = 0 and P(/3! <<*!+••■+ ai+1) > 0.
Section 4. Wide-Sense Regeneration - Harris Chains - GI/GI/k 371 Due to P(/?i < Qi + • ■ ■ + Qi) = 0, we can find an / > 0 such that P(ai + ■ • • + Qi ^ /) = 1. (4.28) Due to P(/?i < Qi + • ■ • + Q«+i) > 0, we can find an x > 0 such that P(/?i < x) > 0 and P(ai > x/(i + 1)) > 0. (4.29) It is not hard to show that the negative drift condition E[/?i] < fcE[ai] implies that the queue length process (Qs)se[o,oo) enters the state {infinitely often. Due to P(/?i < Qi + • • • + Qi) = 0, however, it never enters the set {0,... ,i — 1}. Let T\ < T2 < ... be the successive entrance times of {Qs)se[o,oo) mto the state i. At time Tm there are i customers in the system, and since i < k, they are all being served. Denote their (total) service times by /3™i, ■ ■ ■,/3™i ■ These i customers arrived at the latest i arrival times, since P(/3i < Qi + ■ ■ ■ + Qi) = 0 implies that the latest i arrivals are still in the system. Denote these arrival times by r™ < ■ ■ ■ < r^\ and the next i + 1 arrival times by r™ < ■ ■ • < r™. Let a™i+1,..., a™ be the associated inter-arrival times. Clearly, a™,... ,a™ are i.i.d., distributed as Qi, and independent of the past before time r™. Note that P (/?i <QiH hQj)=0 implies that the service times p^,... ,0™i are not affected by the fact that (<3s)se[o,oo) enters the state i at time Tm, that is, (ir^i,..., p^ are i.i.d., distributed as /3i, and independent of (a™i+1,..., aj1). The inter-arrival times a™i+1,..., Qq1 are, however, affected by this fact, but in spite of that it is again not hard to show that, due to (4.29), the event {/?™ <x, ...,P^<x, Q^+1 >x/(i + l),.. .,a?>x/(i + l)} (4.30) occurs for infinitely many m with probability one. Let Mo be the first m such that it occurs and put So = r0Mo + /, where / is as at (4.28). Recursively for n > 1, let Mn be the first integer m such that the event {Tm > 5„_i} and the event at (4.30) both occur and put Immediately before time t^" there are i customers being served, and since i < k, the service of the customer arriving at time t^" starts without delay. Since P"? < x and a%, + ■ ■ ■ + of - > x, the customer that arrived at time r™ will have left before time r^", and thus the service of the customer arriving at time rx " starts without delay. Repeat this argument to obtain that since P-i+j<x and a%i+j + ■ ■ ■ + a^+j > x for 0^3 <i,
372 Chapter 10. REGENERATION the service of the i customers arriving at the times r0 ",...,Tt_1 starts without delay, and immediately before time rt " all customers that arrived before time r0 " have left. Thus immediately before time rt " there are [due to P(/3i < Qi + ■ • • + Qj) = 0] i customers in the system, they all arrived at or after time r0 ", and their service started without delay. Thus the future of the system after time TjM" is not affected by its past before time t™" and behaves distributionally in the same way for all n ^ 0. Now note that due to P(qi + • • • + q« ^ /) = 1, we have r^n ^ r^" + /. Thus the future of the system after time Sn = r^n +1 is not affected by its past before time Sn — I = t0 " and behaves distributionally in the same way for all n ^ 0. Thus (Qs, Rs)se[o,oo) 1S lag-' regenerative with regeneration times 5„, n ^ 0. 5 Time-Inhomogeneous Regeneration In this section we extend the classical regeneration concept by allowing the regeneration to depend on the time when it occurs. This is the kind of regeneration found in time-inhomogeneous Markov chains (Markov chains with transition probabilities that depend on the time of transition). When such a chain visits a recurrent reference state, it only starts anew conditionally on the time of visit, that is, time-inhomogeneous regeneration takes place. We shall also extend wide-sense regeneration in the same way by allowing the future, given the time of regeneration, to be conditionally independent not necessarily of the full past but only of the past regeneration times. Time-inhomogeneity allows the environment in which the process develops to change deterministically with time, as is often the case in real world situations. For instance, the traffic intensity in actual queueing systems can vary drastically with time of day. Adapting a periodic model with period the day is not very helpful, since the relevant time scale is often minutes rather than days. A time-inhomogeneous model thus seems more appropriate. In spite of this, the mathematical theory of time-inhomogeneous models is poorly developed. Hopefully, this section and the next three (though abstract) will be a small contribution to such a theory. Note that if two independent versions of a time-inhomogeneous Markov chain enter a fixed reference state at the same time, then we can let the two chains run together from that time on without affecting their distributions, that is, we would have created an exact coupling. Note also that it is not natural to look for shift-coupling or epsilon-couplings of time- inhomogeneous Markov chains because the behaviour after regeneration can differ drastically depending on the time of regeneration. The same observations apply to time-inhomogeneous regenerative processes. Therefore, our main task here is to find conditions under which two versions of such a process can be forced to regenerate simultaneously (or simultaneously in the distributional sense). We shall carry out the main
Section 5. Time-inhomogeneous Regeneration 373 part of the construction of a successful (distributional) exact coupling in the next section, but the resulting analogue of Theorems 3.3 and 4.3 is stated in this section as Theorem 5.3. 5.1 Definitions Call a one-sided shift-measurable stochastic process Z = (Zs)s€[0,oo) time- inhomogeneous regenerative with regeneration times S if the future after regeneration 6sn (Z, S) depends on the past ((Zs)s€[o,sn), So, ■ ■ ■, Sn) only through the time of regeneration S„, and the conditional distribution of 6s„{Z,S) given the value of S„ is regular and does not depend on n ^ 0. In other words, Z is time-inhomogeneous regenerative with regeneration times S if there is a (([0, oo), B[0, oo)), (H x L, H ® £)) probability kernel p(-\-) such that for n ^ 0 and A G W ® £, P(6Sn(Z,S)eA\(Z.)te[0iSn),S0,...,Sn)=p(A\Sn) a.s. (5.1) Call the pair (Z,S) time-inhomogeneous regenerative of type p(-\-) if this holds. Let the negative random variable S_i be such that (Z°,S°) depends on (D,S-i) only through So- Call a one-sided shift-measurable process Z time-inhomogeneous wide- sense regenerative with regeneration times S if the future after regeneration 6sn (Z, S) depends on the past regeneration times (So, • • •, Sn) only through the time of regeneration S„, and the conditional distribution of 6sn(Z,S) given the value of Sn is regular and does not depend on n ^ 0. In other words, Z is time-inhomogeneous wide-sense regenerative with regeneration times S if there is a (([0,oo),$[0,oo)), (H x L,H ® £)) probability kernel p(\) such that for n ^ 0 and A G U ® £, P(0Sn(Z,S)GA|So,...,Sn)=p(A|Sn) a.s. (5.2) Call the pair (Z, 5) time-inhomogeneous wide-sense regenerative of it/pe p(-|-) if this holds. Let the negative random variable S-\ be such that (Z°,S°) depends on S_i only through So- With / ^ 0, call a time-inhomogeneous wide-sense regenerative (Z, S) time-inhomogeneous lag-l regenerative if (5.2) can be strengthened to: for n ^ 0 and A G % ® £, P(0Sn(^5)GA|(Za)a€[oi(5n-r)+],5o")...,5n)=p(A|5n) a.s.; (5.3) and time-inhomogeneous lag-l+ regenerative if (5.2) can only be strengthened to: for n ^ 0 and A G % ® £, P(0S„ (Z, 5) G A|(Zs)8€[0,(sn-/)+), So,..., 5„) = p(A|5„) a.s. Therefore, time-inhomogeneous lag-0+ regeneration is the same as time- inhomogeneous regeneration, while lag-0 regeneration implies further that Zsn is a measurable mapping of 5„.
374 Chapter 10. REGENERATION A pair (Z',S') is a version of a time-inhomogeneous regenerative (Z,S) if (Z', S') is time-inhomogeneous regenerative of the same type as (Z, S). A pair (Z', S') is a version of a time-inhomogeneous wide-sense (or lag-/, or lag-/+) regenerative {Z,S) if (Z',S') is time-inhomogeneous wide-sense (or lag-/, or lag-/+) regenerative of the same type as (Z, S). Note that in these cases the zero-delayed (Z°,S°) is in general not a version of (Z, S). Call a wide-sense time-inhomogeneous regenerative (Z,S) of type p(-\-) time-homogeneous if p(-\s) does not depend on s, that is, if p(-|S)=p(-):=P((Z°,5°)G-), se[0,oo). Thus if a time-inhomogeneous regenerative (Z, S) is time-homogeneous, then it is classical regenerative. And if a time-inhomogeneous wide-sense (lag-/, lag-/+) regenerative (Z, 5) is time-homogeneous, then it is wide- sense (lag-/, lag-Z+) regenerative. 5.2 The Regeneration Times S Are Time-Homogeneous Markov If (Z,S) is time-inhomogeneous regenerative (wide-sense or not), then in general it is neither true that the cycles are i.i.d. nor that S forms a renewal process. However, the following holds. Theorem 5.1. // (Z,S) is time-inhomogeneous wide-sense regenerative, then the sequence S is a time-homogeneous Markov process. PROOF. According to (5.2), for each n ^ 0, (So, ■ ■ ■ ,Sn) depends only on ) through Sn and thus only on (5„+i, 5n+2,.. ■) through Sn, that is, 5 is a Markov process. Also, according to (5.2), the conditional distribution of (S„+i — S„, 5„+2 — 5„,...) given the value of 5„ does not depend on n, and therefore the conditional distribution of (S„+i, S„+2, ■ ■ ■) given the value of 5„ does not depend on n, that is, S is time-homogeneous. □ Let Fs be the conditional distribution of Xk+i given Sk = s, that is, for s G [0, oo) and A G B[0,oo), FS(A) :=p{H x{Ax [0,oo)°°) DL\s) = P(Xk+1 e A\Sk = s). We shall view 5 as a 'renewal process' that is time-inhomogeneous in the sense that if a 'renewal' occurs at time Sk = s, then the next recurrence time Xk+i is governed by a distribution that may depend on s, namely Fs. Call Fs the recurrence distribution at s and define, for 1 ^ n < oo, the n-step recurrence distribution at s by F?(A) := p(H x ([0, oo)""1 x A x [0, oo)°°) n L\s) = P(Sk+n -Ske A\Sk = s), s£ [0, oo), A e B[0, oo).
Section 5. Time-Inhomogeneous Regeneration 375 Let Fs and F" also denote the conditional distribution functions of Xk+i and Xk+i + • • • + Xk+n, respectively, given 5„ = s, that is, for s G [0, oo) and x G [0,oo), Fs(x) := Fs([0,x]) = P(Xk+1 ^ x\Sk = s), F?(x) := Ff{[0,x]) = P(Xk+1 + ■■■ + Xk+n ^ x\Sk = s). The n-step transition probabilities of S are P(Sfc+nG A\Sk = s) = F?(s + A), sG[0,oo), AeB[0,oo). If (Z, S) is time-homogeneous, then 5 is a renewal process, and we have Fs = F independently of s, where F is the common distribution of the i.i.d. recurrence times, and F™ = Fn, where Fn is the distribution of the sum Xi + ■ ■ ■ + Xn (Fn is the nth convolution power of F). 5.3 Examples Let S = (Sjt)o° be a discrete-time Markov process with state space [0, oo), increasing strictly to infinity. Put Ns- = lim^s Nt- Then (S/v.,_)se[o,oo) is time-inhomogeneous lag-0 regenerative with regeneration times 5, but certainly (£Vs_)se[o,oo) is n°t time-homogeneous, not even when 5 is a renewal process. Also, (Sn3_ , Aa, Bs, Ds, C/s)se[0iOO) is time-inhomogeneous lag-0+ (but not lag-0) regenerative with regeneration times S. Moreover, for any / > 0, the process (SNs_,As,Bs,Ds,Us,Ns+i - ATs)se[0]Oo) is time- inhomogeneous lag-/ regenerative. If Z is wide-sense regenerative (or lag-/ regenerative) with regeneration times 5, then so is the stochastic process (ZSiSn,„, As,Bs,Ds, Us,Ns+i - Ars)s6[0jOo). Consider a discrete-time Markov process Y = (Yfc)o° with state space R and with Hindoo Yk = oo. Put 5„ = Yk„ , where Kq = inf {k ^ 0 : Yk ^ 0} and, recursively for n ^ 1, Kn = inf {A; ^ 0 : Yk > 5„_i}. Take I > 0 and let Z — (Zs)s6[oi00) be the process with Zs the number of times the Markov process Y visits the interval (s,s + I}. Then (Z,S) is time-inhomogeneous wide-sense regenerative, but in general not lag-/. A time-inhomogeneous Markov chain Z = (^s)S£[o,oo) with a recurrent state is time-inhomogeneous regenerative with regeneration times S formed by the successive entrances to this state. More generally, let Z be a time-inhomogeneous continuous-time general state space shift-measurable Markov process, let A be a recurrent set and let the time-homogeneous space-time process (Zs, s)s6[o,oo) be strong Markov. If the transition probabilities of Z are the same from all states in A, then Z is time-inhomogeneous regenerative with regeneration times S formed by the successive entrances to A. Still more generally, Theorem 4.6 has a time-inhomogeneous counterpart. Let Z = (Zs)se[o,oo) be a time-inhomogeneous general state space shift- measurable Markov process such that the space-time process (Zs, s)se[0iOO)
376 Chapter 10. REGENERATION is strong Markov. Suppose Z has a set of states A such that ta (the hitting time of A) is measurable and finite with probability one for all initial distributions and ZTA £ A, and such that for some / > 0, p G (0,1] and a probability kernel n(-, •), P(zt+ie-\zt=x)^pfi{t,-), xeA, te[o,oo). Then, with T0 = ta and Tk+l = inf{t ^ Tk + I : Zt e A} for k ^ 0, we have P{ZTh+,e-\ZTk=x,Tk = t)2pn{t,-), xeA, *G[0,oo). This allows us to extend the underlying probability space by conditional splitting: apply Theorem 5.1 in Chapter 3 recursively to obtain i.i.d. 0-1 variables Iq, I\,... such that for k ^ 0, (ZJo, ■ ■ -,h-i) depends on Ik only through (ZTk,Tk, ZTh+i), P(Ik = l\ZTk,Tk)=p and P(ZTk+l e-\ZTk,Tk,Ik) = n(Tkr). Let Kn be the (n + l)th index k such that Ik = 1. Conditionally on the randomized stopping time 5„ = T^n + I, the time-inhomogeneous Markov process Z has conditional distribution n{Sn, •) at time 5„ and is conditionally independent of its state at time 5„ — /. Thus, conditionally on 5„, the future of Z after time 5„ is independent of the past before time Sn — I, and the conditional distribution of the future given the value of Sn does not depend on n. This argument can be sharpened along the lines of the proof of Theorem 4.6. Thus Z is time-inhomogeneous lag-/ regenerative. Finally, consider the following time-inhomogeneous version of the GI/GI/k queueing model, 1 ^ k $C oo: customers arrive to a fc-server station at times forming a Markov sequence with state space [0, oo) and increasing strictly to infinity, and line up to be served under the first-come-first-served discipline with service times that depend on the time of arrival and/or the time when the service starts. A simple special case is the time-inhomogeneous version of the M/GI/k queue obtained by allowing the Poisson arrivals (M stands for memoryless, the Poisson process property) to be nonstation- ary. If the system empties infinitely often with probability one, then the queue length and ordered remaining service times process {Qs,Rs)se[o,oo) is time-inhomogeneous regenerative with the times of arrivals to an idle system as regeneration times. If the system cannot empty but (<3s)s€[o,oo) has a recurrent state i < k and the event at (4.30) occurs for infinitely many m, then the argument in Section 4.6 shows that (Qs,Rs)se[o,oo) 1S time-inhomogeneous lag-/ regenerative. 5.4 The Key Coupling Result The key coupling result from the time-homogeneous case (Theorems 3.2 and 4.2) extends as follows to the time-inhomogeneous case. We leave out
Section 5. Time-inhomogeneous Regeneration 377 the random variable R this time, since we are not going to need it for epsilon-coupling. Theorem 5.2. Let (Z,S) be time-inhomogeneous wide-sense regenerative of type p(-\-). Let (S,K) be such that S = S, (5.4a) K is a randomized stopping time with respect to S. (5.46) Then the probability space (£l,!F, P) on which (Z,S) is defined can be extended to support a K such that (S,K)^(S,K), (5.5a) K is a randomized stopping time with respect to S, (5.56) K is conditionally independent of Z given S. (5.5c) Moreover, with T = Sa, T = Sj<, (5.6a) P{0TZe-\T)=p{-\T) a.s. (5.66) Proof. Apply the transfer extension in Section 4.5 of Chapter 3 to obtain (5.5a) and (5.5c) from (5.4a). From (5.46) and (5.5a) it follows that (5.56) holds. From (5.5a) it follows that (5.6a) holds. Finally, (5.66) holds due to (5.56), (5.5c), and the following lemma. □ Lemma 5.1. Let (Z,S) be time-inhomogeneous wide-sense regenerative of typep(-\-). Suppose K is a stopping time with respect to S or, more generally, K is a randomized stopping time with respect to S and conditionally independent of Z given S. Then P(6Sk(Z,S)e-\S0,...,Sk)=p(-\Sk) a.s. If further (Z, S) is lag-l regenerative for some I ^ 0, then p(8sk{Z,S) G •|(^)*€[o,(Sfc-o+].s'o,...,S'fe) =p(-\Sk) a.s. Proof. The event {K = n} depends on (Z, S) only through 5 [since K is conditionally independent of Z given S] and on S only through (So,... ,Sn) [since K is a randomized stopping time with respect to S]. Thus {K = n} depends on (Z, S) only through (S0,..., Sn). Thus {K = n} depends on 6sn (Z, S) only through (So, • • •, 5„). Since (So,..., S„) depends on 6sn (Z, S) only through S„, this implies that {K = n} and (So,..., Sn)
378 Chapter 10. REGENERATION depend on 6sn(Z,S) only through Sn. This yields the second equality in the following calculation: with A G U ® C, B G Uf=lB[Q, oo)* and n ^ 0, P(0Sfc (Z, 5) G A, (So, • • •, Sk) G B, K = n) = P(6Sn (Z, S)eA,(S0,...,Sn)eB,K = n) = E[P(6Sn(Z,S) G A|5n)P((50,...,5n) G B,K = n\Sn)} = E[p(A|5n)E[l{(Soi...,Sn)eB,K=n}|5n]] (due to (5.2)) = Eb(j4lS'n)l{(So,...,S„)eB,K=n}] = E\p(A\SK)l{{So,...,SK)€B,K=n}\- Sum over n to obtain that for A G U ® £ and I? G Ug^/SfO, oo)*, P(0Sfc (Z, 5) G A, (So, ...,Sk)eB)= E[p(A|5fc)l{(s0,...,sfc)€B}], that is, the first claim of the lemma holds. In order to obtain the second claim replace (S0,...,Sk) by {{Zs)se[0t{Sk _,)+], 50,... ,5fe) in the above argument. □ 5.5 Exact Coupling In the time-homogeneous case we established the existence of a successful exact coupling (Theorems 3.3 and 4.3) assuming only that the cycle-lengths are either spread out or lattice with delay lengths supported by that same lattice. The proof was based on coupling results for renewal processes from Chapter 3 (Theorem 6.1) and Chapter 2 (Theorem 7.2), which in turn relied on the Ornstein coupling, that is, on the idea of coupling two versions of a random walk in such a way that the pairwise difference of their step-lengths is bounded, symmetric, and 'strongly aperiodic'. The random walk formed by the difference of the coupled random walks is then recurrent, and hence the two coupled random walks eventually meet. The Ornstein coupling method does not apply naturally in the time- inhomogeneous case: if the two versions of a random walk are replaced by two versions of a discrete-time Markov process, then a pair of step- lengths (increments) in general cannot be coupled in such a way that their difference is bounded, and even if this could be accomplished, it still is far from implying that the difference process is recurrent. The classical coupling method, however, works smoothly in the time- inhomogeneous case: if two independent time-inhomogeneous regenerative processes (wide-sense or not) regenerate at the same time, then they behave in the same way distributionally from that time onward. Now recall that in order to establish that the classical coupling was successful for irreducible aperiodic Markov chains (Chapter 2, Section 3) we needed not only aperiodicity but also positive recurrence, that is, we needed
Section 5. Time-inhomogeneous Regeneration 379 the condition that the cycle-lengths have finite expectation. Analogously for time-inhomogeneous regenerative processes we shall need not only a generalization of the conditions in Theorems 3.3 and 4.3 (that the cycle-lengths are either spread out or lattice with delay times supported by that same lattice) but also a condition that generalizes this finite mean cycle-length condition. The coupling construction will be carried out in the next section, but we shall state the resulting (partial) extension of Theorems 3.3 and 4.3 already at this point. More detailed coupling and convergence results are presented in the next section. Theorem 5.3. Let (Z, S) be time-inhomogeneous wide-sense regenerative and (Z', S') be a version of (Z, S). Suppose either there is an n ^ 1 and a subprobability density f on [0, oo) such that J f(x) dx > 0 and inf F?{B)^ [ f(x)dx, BeB, (5.6) se[o,oo) JB or there is a d> 0 and a subprobability mass function f on {d, 2d, 3d,...} such that f is aperiodic [that is, gcd{fc ^ 1 : f(kd) > 0} = 1] and inf Fs({kd}) ^ f(kd),k ^ 1, and S and S' are dZ, valued. (5.7) s£[0,oo) Further, suppose there is a probability distribution F on [0, oo) such that inf Fs{x) ^ F{x) for x G [0, oo) and I xF{dx) < oo. (5.8) se[o,oo) J Then the following claims hold. (a) The underlying probability space (f2, T, P) can be extended to support finite random times T and T" such that (Z, Z',T,T') is a successful distributional exact coupling, that is, (6TZ,T)^(8T,Z',T'). (5.9) Moreover, if there exists a weak-sense-regular conditional distribution of Z given Oj-Z [this holds when (E,£) is Polish and the paths are right-continuous], then {£l,T, P) cafi be further extended to support a copy Z" of Z' such that (Z,Z",T) is a successful nondistributional exact coupling of Z and Z', that is, Z" = Z' and 6TZ = 6TZ". (5.10) (6) With || • || denoting total variation we have -p{6tZe-)--p{etZ' £-)\\ ->0, *^oo.
380 Chapter 10. REGENERATION (c) With T the tail a-algebra on W we have P(Z G A) = P(Z' G A), Ae T. (d) If (Z,S) is time-inhomogeneous lag-l regenerative for some I > 0, then Z is T-trivial and mixing, but not in general. Comment. The conditions (5.6), (5.7), and (5.8) are only simple sufficient conditions and far from being necessary. In the time-homogeneous case, however, (5.6) means that X\ is spread out, and (5.7) means that So and S0 are both dZ valued and X\ is lattice with span d, that is, the conditions in Theorems 3.3 and 4.3 hold and these are necessary (see Remark 3.1). Moreover, (5.8) means in the time-homogeneous case that E[Xi] < oo, which was the condition for asymptotic stationarity. Proof, (a) Theorems 6.1 and 6.2 in the next section yield the existence oi{S,S',k,k') such that S — S and k is a randomized stopping time w.r.t. 5, (5-11) S" = S' and k' is a randomized stopping time w.r.t. S", (5-12) Due to (5.11) and Theorem 5.2 above, (fi, J7, P) can be extended to support a T such that T=Sk and P(0TZ G -\T) = p(-\T) a.s. Due to (5.12) and Theorem 5.2, (fi, T, P) can be further extended to support a T" such that T' = S'k, and P(02vZ'e-|T')=p(-|T')a.s. Combine Sk = S'k, and T = Sk and V = S'k, to obtain T = T. Combine this and P(6TZ G -\T) = p(-\T) and P(0T,Z' G -|T') = p{-\T') to obtain (5.9). Apply Theorem 3.2 in Chapter 4 to obtain Z" such that (5.10) holds. (b) Use (a) and Theorem 9.4 in Chapter 4 to obtain (b). (c) Use (a) and Theorem 9.4 in Chapter 4 to obtain (c). (d) Assume that (Z, 5) is time-inhomogeneous lag-/ regenerative for some / > 0. Due to Theorem 2.1 in Chapter 6, Z is T-trivial and mixing if we can establish that \\P(Otze-\zeB)-P{Otz'e-)\\->o, t -> oo, (5.13) for all B of the form B = {z G H : ztl Gli,...,zt, G An},
Section 5. Time-Inhomogeneous Regeneration 381 where n ^ 1, 0 ^ t\ < ■ ■ ■ < tn, and Ai,..., An G £. In order to prove (5.13), note that (Z, (Sjv(n+,+jt)o°) is a version of the lag-/ regenerative (Z, S) [see Lemma 5.1] and that the event {Z G B} is in the cr-algebra generated by (Zs)se[o,t„]- Since the mth regeneration time SjvtTi+,+m is greater than tn + /, it follows from this and the definition of time-inhomogeneous lag-/ regeneration [see (5.3)] that the event {Z G B} and the past of (Z, (S,jvtri+,+jb)o°) UP to time SN,n+,+m — / depend on the future of (Z, (Sn, +,+jt)o°) after time 5jv, +, only through Sjv,n+,+TO- Thus a pair (Z1, S') with distribution P((Z, (SivfTi+,+fc)o°) S -|Z G B) is a version of (Z, S). Thus (6) yields (5.13). Thus Z is T-trivial and mixing in the lag-/ case. According to Theorem 4.3(d), this need not hold in the general wide-sense case. □ 5.6 Condition (5.8) Is Stronger Than Uniform Integrability A family of random variables Ys, s £ [0, oo), is uniformly integrable if sup E[Ysl{Ys>x]] -> 0, a;->oo. (5.14) s£[0,oo) D Recall that ^ denotes stochastic domination [see Section 3 in Chapter 1]. If Ys has the distribution Fs, then (5.8) can be rewritten as follows: there is a random variable Y such that Ys ^ Y for s G [0, oo) and E[Y] < oo. (5.15) D _ If (5.15) holds, then Ys1{ys>x} ^ ^l{y>x} f°r a^ s ^ [0, oo), and thus sup E[Ysl{Ys>x}] ^ E[yi{y>x}] ->• 0, a;->oo. s£[0,oo) Thus (5.15) implies uniform integrability. The converse is not true, however, as the following counterexample shows. Thus the condition (5.8) is strictly stronger than uniform integrability. Example 5.1. Let Y be a random variable on [2,oo) with distribution function Fix) = 1 - 2l£g£ 2 < a; < oo. ■ a; log a; Note that |0°° P(Y > x) dx = |2°° ||2|1 dx = oo, and thus [see Lemma 5.2 below] E[Y) = oo. For s G [0, oo), define Ys by Ys = s if Y > s and Ys = 2 if Y ^ s.
382 Chapter 10. REGENERATION Then E[ysl{y>>x}] = sP(Y > s) for s > x, and E[ysl{y3>x}] = 0 for s ^ x, and thus, for x ^ 2, 2 log 2 sup E[ysl{ys>a.}] = xP(Y > x) = -— >• 0, x -»• co. se[o,oo) * log a; Thus the family Ys, s £ [0, oo), is uniformly integrable. On the other hand, P(ys > x) - P(y > s) for s > x, and P(y > x) = 0 for s ^ x, and thus sup P(y > x) = P(y > x), x e [2,00). s£[0,oo) Since E[Y] = oo, this shows that there can be no finite-mean random variable Y dominating all the Ys stochastically, that is, (5.15) cannot hold. Lemma 5.2. For any nonnegative random variable Y it holds that /•OO E[y]= / p(y>x)dx. Jo Proof. We have Y = /0°° l{Y>x}dx, and taking expectation [and interchanging expectation and integration] yields the desired result. □ 5.7 Stationarity => Time-Homogeneity We shall now show that under proper time-inhomogeneity there is no stationary version. Theorem 5.4. Let (Z, S) be time-inhomogeneous wide-sense regenerative. If (Z, S) is stationary, then (Z, S) is time-homogeneous wide-sense regenerative. Proof. Take t G [0, oo), n)0,A6?{®£, and let p(-|-) be the type of (Z, S). We must show that 9s„(Z,S) does not depend on Sn, that is, we must show that p(A\Sn) is a.s. constant. Since Nt- + n is a stopping time with respect to 5, we have [see Lemma 5.1] P(0sNt_+n(Z,S)e A\SNt_+n) = p(A\SNt_+n) a.s. (5.16) By stationarity, (6>SjV(_+7i (Z, 5), SNt_+n - t) = (9Sn{Z,S),Sn), and thus P(OsNt__+n(Z,S)eA\SNt_+n-t)=p(A\SNt_+n-t) a.s. (5.17) Since the left-hand sides of (5.16) and (5.17) are a.s. equal, so are the right-hand sides, and since SNt_+n — t = Sn, we obtain p(A\t + Sn) =p(A\Sn) a.s., «G[0,oo),n^0,AGH®£. (5.18)
Section 5. Time-inhomogeneous Regeneration 383 This implies E[/0°° \p(A\t + Sn) - p(A\Sn)\P(Sn G dt)\ = 0. Thus we can find an s G [0, oo) such that oo \p(A\t + s)-p(A\s)\P(Snedt)=0. Thus p(A\s + Sn) = p(A\s) a.s. This and (5.18) with t = s yield p(A\Sn) = p(A\s) a.s., that is, p(A\Sn) is a.s. constant as desired. □ 5.8 Asymptotic Stationarity => Asymptotic Time-Homogeneity A comparison of Theorem 5.3 with Theorems 3.3 and 4.3 shows that the only difference (apart from the finite moment condition) is that Theorem 5.3 contains no claim about asymptotic stationarity or about asymptotic periodic stationarity. In the light of Theorem 5.4 this is rather natural. We shall now show further that if (Z, S) is asymptotically stationary in total variation, then it is asymptotically time-homogeneous in the sense that p{A\t + ■) is an L\ constant in the limit as t —> oo for each A G Ti CS> C and the convergence is uniform in A. Theorem 5.5. Let (Z,S) be time-inhomogeneous wide-sense regenerative of type p(-\-). Suppose there is a pair (Z*, S*) such that 6t{Z,S)tA{Z\S*), t->oo. (5.19) Then (Z*, 5*) is stationary and time-homogeneous wide-sense regenerative, and sup / \p{A\t + s)-P(9s*n(Z*,S*)eA)\ds->0, t -> oo, (5.20) AeH®CJo for all h > 0. PROOF. First note that (Z*,S*) is stationary, since (5.19) implies that as t —> oo, 9S(Z*,S*) £ 0s0t(Z,S)=9t+s(Z,S) % (Z*,5*), s G [0,oo). Now fix an arbitrary AeV.® C. Due to (5.19), we have, for each n ^ 0, (QsNi_+n(Z, S),Sn,_ —t,.. .,SNt_+n - t) n(8s.(Z*,S*),S°0,...,S*n), t->oo. Since Nt- +n is a stopping time with respect to 5, we have [see Lemma 5.1] P(^sJv,_+„ {Z, S) G A\SNt_ -t,..., SNt_+n -t)= p{A\SNt_+n) a.s.
384 Chapter 10. REGENERATION Applying Lemma 5.3 below yields that as t —> oo, E[\p(A\t + S*n)-P(9s*(Z*,S*)eA\S*0,...,S*n)\}^0. (5.21) This implies that p(A\t + 5*) converges in probability as t —> oo for all n ^ 0. Therefore (see Ash (1972), Theorem 2.5.3) there is a sequence too < toi <■■■—> oo such that p(A\t0k + Sq) converges a.s. as k -> oo, and a subsequence £10 < t\\ < . ■ ■ of (£0o, £oi, • • ■) sucn that jo(A|ti/t + 5J) converges a.s. as k —> oo, and so on. Thus p(A\tkk + S'*) converges a.s. as k —> oo for all n ^ 0. In other words, for each n ^ 0 there is a Borel set _Bn such that P(5* G -Bn) = 1 and p*(A\s) := limk-+oo p(A\tkk + s) exists for all s G -B„. Put p*(A\s) := P((Z*,5*) G A) for s £ US° B" to obtain p{A\tkk + S^) -> p*(A|5^) a.s. as fc -> oo for all n > 0. This and (5.21) yield P(0S. {Z\ S*) G A|S',..., S*n) = p*(A\S*n) a.s. for all n > 0. Therefore, (Z*,5*) is time-inhomogeneous wide-sense regenerative, and since (Z*,S*) is stationary, Theorem 5.4 yields that (Z*,S*) is time- homogeneous wide-sense regenerative. To establish (5.20) use Lemma 5.3 below and the time-homogeneity of (Z*, S*) to obtain that as t -> oo, /•oo sup / \p(A\t + s)-P(0s*o(Z*,S*)eA)\P(S*oeds)^O. (5.22) Now, 5q is the delay length of the stationary renewal process S* and thus has a nonincreasing density (see Theorem 2.1). Thus there are a and b > 0 such that |p(A|t + s)- P(9s; (Z*,S*) G A)|P(S0* G ds) ^af \p(A\t + s)-P(9s>(Z*,S*)eA)\ds, AeU®C. Jo This and (5.22) show that (5.20) holds for h = b. Thus (5.20) holds for h = kb for all k ^ 1. Since the left-hand side of (5.20) in nondecreasing in h, this implies (5.20) for all h > 0. D Lemma 5.3. For each t G [0, oo), let (Vt,Wt) be a random pair in some measurable product space (Ei,£i) ® (E2, £2)- Suppose there exists a regular version qt{-\-) of P(Vt G -\Wt = •)■ V there is a pair (V,W) such that (Vt,Wt) 4 (V, W) ast-> 00, then aupE[\qt{A\W)-P{VeA\W)\]->0, t -► 00. (5.23) Ae£i In particular, if qt{\) does not depend on t, <fc(-|-) — <z(-|') saV' then q(-\-) is a version ofPiV G -\W — ■).
Section 6. Classical Coupling 385 Proof. Take A G £1, let q-oo(^l-) be a version of P(V G A|W = ■), and put £+ = {w G £2 : 9oo(^|w) ^ q-t(A|w)}. Then E[l{W€B+}l9<x,(A|W)-gt(A|W)|] = P(V G A, W G B+) - P(Vt G A, Wt G B+) + / qt(A\w)P(Wt edw)- f qt(A\w)P(W G du>). Jb+ Jb+ The first difference on the right-hand side of this identity is dominated by i||P((V, W) G -)-P{{Vt, Wt) G -)H, and the second difference is dominated by \\\P{Wt G ■) - P(W G -)||- Thus E[l{W€B+}\qx(A\W) - qt(A\w)\] <\\P((Vt,Wt)e-)-P((V,W)e-)\\. With B~ = {w G E2 '■ qoc(A\w) < qt(A\w)} we obtain in the same way ni{WeB-}\q°o(A\W)-qt(A\w)\] <\\P((Vt,Wt)e-)-P((V,W)e-)\\. Add these two inequalities and take the supremum in A G £\ to get (5.23). In particular, if qt(A\-) = q(A\-) for t < 00, then it follows from (5.23) that E[|g(A|W) - P(V G A\W)\] = 0, that is, P(V G A\W) = q(A\W) a.s. as desired. □ 6 Classical Coupling In this section we shall use the classical coupling idea (Chapter 2) to construct a successful distributional exact coupling under the conditions of Theorem 5.3, that is, we shall complete the proof of Theorem 5.3. Classical coupling in the context of time-inhomogeneous regenerative processes means simply using the time of first simultaneous regeneration of two independent versions of the process as a distributional coupling time. This procedure is successful under the lattice conditions of Theorem 5.3 and can be modified to be successful under the nonlattice conditions. Throughout this section and the next we shall write Ps to indicate that S0 = s with probability one. 6.1 Classical Coupling — The Lattice Case We start by showing that the classical coupling is successful under the lattice condition of Theorem 5.3. Theorem 6.1. Let S and S' be independent nonnegative Markov sequences increasing strictly to infinity with common time recurrence distributions Fs,
386 Chapter 10. REGENERATION s G [0,oo). Suppose there is a d > 0 and an aperiodic subprobability mass function f on {d, 2d, 3d,...} such that S and S' are dl* valued and f is a mass function component of Fs for all s G [0, 00), that is, Fs({kd})>f(kd), k^O. Suppose further there is a probability distribution function F on [0, 00) such that J xF(dx) < 00 and F dominates Fs stochastically for all s G [0, 00), that is, Fs(x)^F(x), iG[0,oo). Then the time of first simultaneous regeneration is finite with probability one: T := ini{t ^ 0 : Bt- = B't_ = 0} < 00 a.s. Moreover, with K and K' the indices of S and S' such that Sk = S'K, = T, it holds that K is a randomized stopping time with respect to S, and K' is a randomized stopping time with respect to S'. Proof. Let us start with the randomized stopping time claims. For each n ^ 0, S depends on the event {K = n} only through (50,..., Sn, S') [since {K — n} is in the cr-algebra generated by (So,... ,Sn,S')] and on (So, ■ ■ ■, Sn, S') only through (So, ■ ■ ■ ,Sn) [since S is independent of 5']. Thus S depends on {K = n} only through (So,... ,Sn), that is, K is a randomized stopping time with respect to 5. In the same way we obtain that K' is a randomized stopping time with respect to 5'. Thus it only remains to establish that P(T < 00) = 1. For that purpose, define a Markov process (Tfc,T[)§° with state space [0,oo)2 as follows: fix an integer no > 0 and put (see Figure 6.1 in Section 6.3 below) (To,To) := (So, So) and, for k ^ 0, (Tk+1,T'k+1):=(SNLk_,S'N,; ), where Lk := rk V r'k + n0d. Put, for k ^ 0, (0k+i,/3'k+i) ■= (7-fc+i - Lk,r'k+1 -Lk). When M:=inf{*£ !:(&,&) = (0,0)}
Section 6. Classical Coupling 387 is finite, we have tm = t'm and thus T ^ tm. (6.1) In order to establish that P(M < oo) = 1, note that conditionally on (Tfc> T'k) = (s>s') tne random variables /3fc+i and /3'k+1 behave as the residual lives at time (s V s' + nod)— of two independent sequences of regeneration times with delay lengths s and s', respectively. Thus Pm+l,P'k+1) = (OM(Tk,T'k) = (s,s')) = Ps(-B(sVs'+n0d)- = 0)Ps'(-B(sVs'+nod)- = 0). Due to Lemma 6.1 below, we can choose no such that for some p > 0, P((A+iX+i) = (0,0)|(Tfc,r[))^2a.s, fc^O. (6.2) For m ^ 0, the event {M>m} = {(&,#) ^ (0,0),...,(/3m,/4) ^ (0,0)} is in the cr-algebra generated by ((t0,Tq), ..., (rm,rj). Since (Tjt,T[)g° is Markovian, this means that the event {(/3m+i,/3m+i) 7^ (0,0)} is conditionally independent of the event {M > m) given (Tm,T'm). This and (6.2) yield the inequality in P(M >m+l\M > m) = P((/Wi,/4+i) 7^ (0,0)|M ^ m) Thus P(M > k) ^ (l-p2)k ->0, fc->oo, that is, P(M = 00) = 0, which implies P(T < 00) = 1 due to (6.1). □ Lemma 6.1. Under the conditions of Theorem 6.1 there exists an integer n0 ^ 1 and a p > 0 determined by f and F such that for all s £ [0, 00), Ps{Bs+nd- = 0) ^ p, n ^ n0. (6.3) PROOF. Put q = Y^\ /(lc0 and let R = (-Rjt)o° t>e a zero-delayed renewal process with dZ valued recurrence times having probability mass function f/q. Then, for n ^ 1, Ps(£s+mi_=0) = ]r>;({nd}) i=l 00 ^ J2 qiP(Rl = nd) =: b(n), say. (6-4) »=i
388 Chapter 10. REGENERATION The set {n ^ 1 : 6(n) > 0} is both additive [since P(Ri = nd) > 0 and P(Rj = md) > 0 imply P(Ri+j = (n + m)d) > 0] and aperiodic [since Hn) ^ f{n<i) and / is aperiodic]. Thus, by Lemma 3.1(6) in Chapter 2, there is an n0 ^ 1 such that 6(n) > 0 for n ^ no- Take m ^ 1 such that F{{nid}) > 0, note that 6 := min{6(n0),..., 6(n0 + ni - 1)} > 0, (6.5) and put , , J 6, no ^ n < n0 +ni, a(n) := < - I bF(md)... F((n - n0)d), n0 + ni ^ n < oo. We shall show by induction that for n ^ no, inf Ps{Bs+nd_ = 0) ^ a(n). (6.6) s£[0,oo) Due to (6.4) and (6.5), this is true for each n G {no,..-,no + ni — 1}. Suppose (6.6) holds for all n G {no,. - - , m}, where m is some integer such that m ^ n0 + ni — 1. Since a(-) is nonincreasing, this induction hypothesis yields inf Ps(Bs+nd- =0) ^ a(m), n G {n0, ■ ■ ■ ,m}. (6.7) s£[0,oo) For each s G [0, oo), m+l Ps(-Bs+(m+1)d_ = 0) = 2J -F1s({ic?})Ps+id(-Bs+(TO+i)d_ = 0) m+l —no ^ £) Fs({id})Ps+id{Bs+{m+1)d_=0) i=l m+l —no ^ ]C Fs{{id})a(m) [due to (6.7)] i=l ^ 0(771)^ ((m + 1 - n0)d) ^ a(m)F((m + 1 — no)d) [F dominates Fs stochastically] = a(m + 1). Thus by induction (6.6) holds for all n ^ no- Since a(-) is nonincreasing, (6.6) yields that (6.3) holds with oo p := lim a(n) = 6TTc;, where Ci := F(md + id).
Section 6. Classical Coupling 389 Lemma 5.2 yields £"^(1 - c,) < / xF(dx). Thus J xF(dx) < oo implies J2°l0(l - ct) < oo, which in turn implies Yl°l0 c, > 0 due to Lemma 6.2 below. This and b > 0 yield p > 0 as desired. □ The following result was needed in the above proof. Lemma 6.2. 7/c0,ci,---G (0,1], then oo oo TT Cj > 0 if and only if /J(l - *) < oo. o o Proof. For all x e K it holds that ex ^ x + 1. Thus OO o and thus FJ^° cj > 0 implies 5Z^°(1 — <k) < oo. In order to establish the converse let Ao,Ai,... be independent events with P(Ai) = c,, i ^ 0. For k ^ 0, OO OO OO OO 11* = P( f| A,) = 1 - P( (J Af) > 1 - £(1 - cO. i=jfc i=fc i=fc i=k If 5Z^°(1 - Cj) < oo, then 5Z^fe(l — Cj) < 1 for some k ^ 1, and thus Ilfcfc Ci > 0. Thus £~(1 - Ci) < oo implies n~ c* > °- D 6.2 The Lemma for the Nonlattice Case We now turn to preparation for the coupling in the nonlattice case. The following result is the counterpart of Lemma 6.1 [to see the similarity replace the uniform distribution \x in (6.8) by the distribution with unit mass at zero to obtain (6.3)]. Lemma 6.3. Let S be a nonnegative Markov sequence increasing strictly to infinity with recurrence distributions Fs,s £ [0, oo). Suppose there is a subprobability density f on [0, oo) such that f f(x) dx > 0 and f is a density component of Fs for all s £ [0, oo), that is,. FS(B)^ [ f{x)dx, BeB[0,oo). JB Suppose further there is a probability distribution function F on [0, oo) such that f xF(dx) < oo and F dominates Fs stochastically for all s € [0, oo), that is, Fs(x)^F(x), iG[0,oo).
390 Chapter 10. REGENERATION Then there is a to ^ 0, a p £ (0,1], and a c > 0 determined by f and F such that for all s € [0, oo), P,(Ba+t-e-)^pii, t^t0, (6.8) where \x is the uniform distribution on [0, c]. We prove this lemma in six steps. First step of proof. We shall start by showing that for each h > 0 there is a th and a nonincreasing function bh(-) determined by / such that inf Es[Ns+t+h - Ns+t] ^ bh(t) > 0, t>th. (6.9) s£[0,oo) For that purpose let a be such that q := J"Qa f{x) dx > 0 and let R — (-Rfc)o° be a zero-delayed renewal process with recurrence times having density f(x)/q, O^x^a. Then, for t and h ^ 0, OO Es[Ns+t+h - Ns+t] = J2 ps(t <X1 + --- + Xt^t + h) i=\ oo ^^giP(t<ifc^t + /i)=:ffh(')> say. i=l Note that gh(t) > 0 if and only if £~x P(t < R{ ^ t + h) > 0. Since the recurrence times of R are bounded and nonlattice, Blackwell's renewal theorem (Theorem 8.1 in Chapter 2) says that the expected number of renewals in (t, t + h], namely J^'tli P(* < Ri ^ t + h), has a strictly positive limit as t —> oo. Thus there is a t'h such that gh(t) > 0 for t ^ £'h. For each /i > 0 and £> t'h, take nh(t) such that [nh(t)h/2,nh(t)h/2 + h/2) ^ [*,* + /i) and note that gh(t) ^ gh/2(nh(t)h/2). Put &h(*) := inf^^j^^^) gh/2(kh/2) and th := £j,2 to obtain (6.9). Second step of proof. We shall next show that with a > 0 such that Ja°° f(x^ dx > 0 and b = /a°° f(x) dx we have sup Es[Ns+t+h - Ns+t-} ^ - + —, t^0, /i^0. (6.10) se[o,oo) ° a0 For that purpose, consider a random walk starting at 0 with {0, a} valued step-lengths taking the value a with probability b. For k ^ 0, let Mfc be the number of times this random walk visits the state kb. Then M0, Mi,... are i.i.d. and geometric with parameter b. Now, Ns+t - JVS_ is stochastically dominated by M0-\ \-M[h/ay Thus Es[Ns+t+h — Ns+t-] ^ (1 + h/a)E[M0], and (6.10) follows by noting that E[M0] = 1/6.
Section 6. Classical Coupling 391 Third step of proof. We shall now show that with Y ~ sNSo+\ - So = 1 + Bs0+i there exists a distribution function F determined by / and F such that f xF(dx) < oo and, for all s G [0, oo), Ps(YKy)>F(y), ye[0,oo). (6.11) In order to establish this, let a and b be as at (6.10) and put d := (l + l/a)/b. Then, for y > 1 and s e [0, oo), PS(Y>y) = £P.(Xi + • • • + Xi-i ^ 1,Xj. + ■ ■ ■ + Xi > y) i=\ oo « = V / (1 - Fs+X(y - x))Ps(Xl +■■■ + X^ E dx) OO » i=i •W (6.12) ^ (1 - F(y - 1)) > ' / Ps(Xi + • • ■ + Xi_! e da;) = (l-F(j/-l))E,[JV,+1-JVg_] sC d(l - F(y - 1)) [due to (6.10)]. Thus (6.11) holds if we take y0 > 1 such that d{\ — F(y0 — 1)) $C 1 and define F by F( ) ■= /°' _ ^ < 2/0' W' \l-d(l-%-l)), y^y0. We obtain f xF(dx) < oo from f xF(dx) < oo and JxF{dx)=J (l-F(y))dy ^y0 + df {l-F{y-l))dy = y0+dfxF{dx), where the two identities are due to Lemma 5.2. Fourth step of proof. We shall next show that for each h > 0 there is an ah > 0 determined by / and F such that with th as at (6.9), inf Es[Ns+t+h - Ns+t] ^ah, t^ th. (6.13) s6[0,oo)
392 Chapter 10. REGENERATION Put n0 = \th] + 2 and take n\ > 1 such that F{nx) > 0. With bh(-) as at (6.9) and F as at (6.11) put , ,. J t>h ■= bh(n0 +nl - 1) > 0, n0 ^ n< n0 + ni, o-h{n) ■ = < I bhF(ni).. .F(n — n0), n0 + «i < n < oo. We shall show by induction that for t £ [th, oo), inf E.[Ws+t+h-iV.+t]>afc([i] + l). (6.14) s6[0,oo) By (6.9) and since bh(-) is nonincreasing, this is true for all t G [th, n0+ni—1). Suppose (6.15) holds for all t G [i/,,m) where m is some integer such that m ^ no + ni — 1. Since <!/,(•) is nonincreasing, this induction hypothesis yields inf Es[Ns+t+h - Ns+t] > a/l(m), t G [ih,m). (6.15) «6[0,oo) For each t G [m,m + 1), we now have Es[Ns+t+h - N.+t] = / Es+x[Ns+t+h - Ns+t}Fs{dx) J[0,t+h] > / Es+it[Ws+t+h-JVs+t]F,(da:) [1 < m + 1 - n0 < t] J[l,m+\-no) }t / ah(m)Fs(dx) [due to (6.15)] V[l,m+i-„0) > ah(m)Fs(m + 1 - n0) > ah(m)F(m + 1 — n0) [F dominates Fs stochastically] = ah(m +1), s G [0,oo). Thus (6.14) holds also for all t G [m,m + l). Thus by induction (6.14) holds for all t G ft/,, oo). Since a/,(-) is nonincreasing, (6.15) yields that (6.13) holds with a/. := lim ah(n) = bhTTcj, where Cj := F(ni + i). Lemma 5.2 yields 5Z^X(1 — c*) ^ f xF(dx). Thus f xF(dx) < oo implies £]°^0(1 — Ci) < oo, which in turn implies YltLo ci > 0 due to Lemma 6.2 above. This and b^ > 0 yield a/j > 0 as desired.
Section 6. Classical Coupling 393 Fifth step of proof. We shall next show that there are constants a and b > 0 determined by / and F only such that oo YJ F1™ > aX([b, oo) fl •), where A is Lebesgue measure. (6.16) n=0 Note that Jj0 . /(• — x)f(x) dx is a density component of F^. Thus, due to Lemma 6.1 in Chapter 3, there is an interval [xo,Xq + h] and a constant Co > 0 determined by / only such that for all s G [0, oo), Fs2 }tco\{[x0,Xo + h]n-). This yields the last step in the following: for A G 23[0, oo) and s G [0, oo), OO CO n=2 oo n=2>. F?+y(A-y)F?-2(dy) +y OO r oo = / F?+y(A-y)Y,K(dy) Jlo,°°) n=0 * OO > / coX([x0,x0 + h]n(A-y))y2Fp(dy). J[0,oo) n=0 The last term has a density gs(-), « oo g,{x) = / Col[x0tXo+h](x - j/) ]T F"(dv) Jl°,°°) n=0 * OO J[x — xo — h,x-xo] n=o oo = c° E ^"([x -x0 ~h,x- x0]) n=0 = coEs[JVs+a. -x0- Ns+X -x0-h] > c0aft for x ^ th + x0 + h [due to (6.13)]. Put a = c0a/j and b = th + x0 + h to obtain (6.16).
394 Chapter 10. REGENERATION Sixth, and last step of proof. We are now ready to establish (6.8). For A £ B[0, oo) and t > b, we have Ps(Bs+t- €A) OO = J2Ps(xl +■■■ + x4_i <t,X!+--- + Xi-teA) i=l oo = jr/ Fs+y{A + t-y)Fl-l{dy) i=i J[°'t) » OO = / Fs+y(A + t-y)Y,Frl(dy), J\o.t) ~T and applying (6.16) yields the first inequality in Ps(Bs+t- e A) > a / Fs+y{A + t - y) dy J[b,t) > a / / /(a;) dxdy. J[b,t) JA+t — y The last term has a density gt(-), gt(x) = a f(x + t-y)dy = a f(u)du, x€[0,oo). J[b,t) J[x,x + t-b) Take c > 0 and t0 ^ b + c such that /, t _b) /(«) du > 0 to get gt{x) ^ a f(u) du>0, x G [0, c], t > i0. •/[c,t0-f>) Put p = ca f, t _b) f(u) du to obtain the desired result (6.8). □ 6.3 Modified Classical Coupling — Nonlattice Case We shall now modify the proof of Theorem 6.1 to obtain finite distributional coupling times under the nonlattice condition of Theorem 5.3. Theorem 6.2. Let S and S' be independent nonnegative Markov sequences increasing strictly to infinity with common recurrence time distributions Fs, s G [0, oo). Suppose there is a subprobability density f on [0,oo) such that f f(x)dx > 0 and f is a density component of Fs for all s G [0, oo), that is, FS{B) ^ J f{x) dx, B <= B[0, oo). Jb
Section 6. Classical Coupling 395 Suppose further there is a probability distribution function F on [0, oo) such that f xF(dx) < oo and F dominates Fs stochastically for all s G [0, oo), that is, Fs(x)^F{x), xe[0,oo). Then the underlying probability space can be extended to support a randomized stopping time K with respect to S and a randomized stopping time K' with respect to S' such that Sk = SKi. Proof. Define a Markov process (Tk,Tk)0Xi with state space [0, oo)2 as follows: let to be as in Lemma 6.3 above and put (see Figure 6.1) (rO,To) := {So,S'0) and, for k ^ 0, (Tk+i,T'k+l) ■■= (SNLk-,S'N, ), where Lk = TkV r'k +t0. Lo U • > -•->• FIGURE 6.1. The coupling construction. Conditionally on (Tk,Tk) = (s,s'), the random variables Pk+i := Tk+i — Lk and f3'k+l := rk+1 — Lk behave as the residual lives at time (s V s' + t0)— of two independent sequences of regeneration times with delay lengths s and s', respectively. Thus, for A and A' eB[0,oo), P((3k+l € A,p'k+l e A'\(rk,r'k) = (s,s')) = Ps(B{sWs,+to)_ e A)ps,{B{sVs,+to)_ e A') Thus, due to Lemma 6.3 above, P((/9k+i./9fc+i) £-\Tk,T'k) >p2fi®ii, k^O, (6.17)
396 Chapter 10. REGENERATION where p G (0,1] and ^ is the uniform distribution on [0, c] for some c > 0. Intuitively, this means that with probability p2 the random variables pk+i and (3'k+l are i-i-d- witn distribution /z and independent of (Tk,T'k) and thus of Lk. Thus with probability p2 the regeneration times Tk+\ = Lk + (3k+\ and r'k+l = Lk + f}'k+l are identically distributed. Since /z ® /z governs the /3-pairs independently of whether it has done so before or not, this means that we obtain distributional coupling times in a geometric number of trials. This idea can be made precise by conditional splitting. Apply Corollary 5.1 in Chapter 3 recursively [in step k take (Yo,Yi,Y2) := ((Tk,T'k), (S,S',Io, ■ ■ ■ ,Ik), (T*+i>TJk+i))] to obtain 0-1 variables I0,h,- ■ ■ such that for k > 0, (S,S',I0,...,h-i) depends on Ik (6.18a) only through {{Tk,Tk), (Tk+i,Tk+l)), and P(Ik = l\Tk,T'k)=p2, (6.186) P((/9fc+i,/9i+i) G -\(Tk,r'k) = (s,s'),Ik = 1) =/i®/i. (6.18c) We shall use the following result from Lemma 4.1 in Chapter 3 repeatedly: for any random elements Y0, Y\, Y2 and Y3 it holds that I3 depends on Y2 only through (Yi,Y0) and on Y\ only through Y0 if and only if Yz depends on {Y2,Y\) only through Y0. Fix an m ^ 1. For 0 < i < m, ((tj, t/), (tj+1 , t,'+1)) is a measurable mapping of (Tfc,T^)^L0, and thus we deduce the following from (6.18a): ForrO ^ i < m,(Tm+k,T,m+k)k*Ll depends on Ii only through ((Tfc,T^)^=0,10, ■ ■. ,h-i). Thus (rm+k, T'm+k^=l depends on 7m_1only through ((rfc, t^=0, io, ■ ■ •, Im-2), and on 7m_2 only through ((Tfc,T£)£L0, I0,... ,Im-2), • • •, and on I0 only through {Tk,Tk)^-=0, and [since (Tfc,r£)£L0 is Markovian] on (ta:,t{.)£L0 only through (Tm,T'm). Thus (Tm+fc,T^+t)211 depends on (6.19) ((^>rfc)r=o,/oI---./m-i) only through {Tm,Tm). Due to (6.18a), {{Tk,T'k)rk=0,Io, ■ ■ ■ Jm-i) depends on Im only through ((rm,T^),(Tm+k,T^+k)f=l) and, due to (6.19), on {Tm+k,T'm+k)f=l only
Section 6. Classical Coupling 397 through (Tm,T'm). Thus {{Tk,T,k)rk=0'Io,---,Im-i) depends on (6.20) Tm+k, Tm+k)k^=l)) onlv through (Tm,T^j). Due to (6.18a), for each n > 1, (t^,-^, /fc)J?L0 depends on 7m+n only through ((rm+fc,r^+A.)^i1,/m+i,...,/m+„_i), depends on Im+n-\ only through ((rm+fc,T^+fc)^=1,/m+1,...,/m+„_2),..., depends on Im+l only through (Tm+i,T^+fc)^1, and finally [by (6.20)] depends on {Tm+k,T'm+k)f=l only through (rm,T^,Im). Thus (rm+fc,T^+fc,/m+fc)^:1 depends on (Tfc,r^,4)^ only through (rm,r^,/m), that is, {Tk,Tlk,Ik)'^' is a Markov process. (6-21) From (6.20) it also follows that Im depends on I\,..., Im—\ only through {Tm,T'm). By (6.186), Im is independent of (rm,r^,). Thus 7m does not depend on /i,..., 7m—i. Thus the 7i,/2,-. are independent, and (6.186) yields that the Ii,I2,- ■ ■ are i.i.d. and P(Ii = I) = p2. Thus M = inf{fc > 1 : Ik-\ = 1} is finite with probability one. Put K-^Nl^- and K':= Ni^.. Since M — 1 is a stopping time with respect to (Tk,T'k, ifc)§° and since -T/V/-1 = 1, we obtain from (6.21), due to the strong Markov property, that P((0M,0'M)e-\(TM-l,T'M-l)=(s,s')) = P((p1,p'1)e-\(T0,T^ = (s,s'),i0 = i). This and (6.18c) yield P((/3m,/3'm) G ■\t~m-i,t'm_1) > /i®/x, that is, /3m and /3M are i.i.d. with distribution /z, (6.22a) (/3M,/?M)is independent of (tm-i,t'm_1) and thus ofl/M-i- (6.226) This yields the distributional identity in Sk —tm— Lm-i + Pm = Lm-i + P'm = t'm = S'k' ■ Thus Sk — S'K, as desired. It remains to establish the randomized stopping time claims. Fix an m > 0. For 0 ^ i ^ m, ((tj,t/), (tj+i,t,'+1)) is a measurable mapping of
398 Chapter 10. REGENERATION (S', (So,.. •, «5jvlm_)), and thus we deduce the following from (6.18a): for 0 ^ i ^ m, S depends on U only through (S", (S0, - - -,SjvLm_),io, ■ ■ ■ ,-^i-i)- Thus S depends on 7m only through (S', (S0, ■ ■ ■, SNLm_), I0, ■ ■ ■, Im-i), and on 7m_! only through (S', (S0,..., SNLm_), I0,...,Im-2), ■ ■ ■, and on I0 only through (S", (S0, - - -, £Vtm_)). Thus S depends on I0, . ■ ■, /m only through (S', (S0,... ,5ivtm_)). Thus {M = m + 1} depends on S only through (S', (So, ■ ■ ■, Sjvtm_)). This yields the third equality in P(A" = n, M = m + 1|S, S') = ?(NLm =n,M = m + 1|S, S') = l{Ntm_=„}P(M = m + l|S,S') = l{Ntm_=„}P(M = m + 1|(S0,...,SNLm_),S') = l{NLm_=n}P(M = m + 1|(S0,..., Sn),S'), while the last equality follows from Lemma 6.4 below (take J = Nim-, Y0 = (S',So), and Yk = Sk for k > 1). Since the event {Nim_ = n} is in the cr-algebra generated by (S', (So,..., Sjvtm_)), we obtain further the first equality in P(K = n,M = m + l\S,S') = P(NLm_=n,M = m + l\{S0,...7Sn),S') = P(K = n,M = m + 1|(S0,...,Sn),S'). Sum over m > 0 to obtain P(A" = n\S,S') = P(K = n|(S0,. - -, Sn),S'). Thus S depends on {K = n} only through ((So, ■ ■ ■, Sn),S'). Since S does not depend on S', this means that S depends on {K = n} only through (S0, - - -, Sn), that is, K is a randomized stopping time with respect to S. In the same way we obtain that K' is a randomized stopping time with respect to S'. □ The following well-known stopping time result was used in the above proof. Lemma 6.4. Let J be a stopping time with respect to (Yq, Y\,...), where for each k ^ 0, Yk is a random element in some measurable space (Ek,£k)- Then P(A\Y0,...,Yj)=P(A\Y0,...,Yn) on {J = n}, {or all events A and all n ^ 0.
Section 7. The Coupling Time - Rates and Uniformity 399 Proof. For B € Ufcto £o ® • ■ • ® £* we have P(An{(Y0,...,Yj)eB}) CO = £ P(4 n {(Y0, ...,Yn)eB,J = n}) n=0 ex) = 2_^ E[1{(>'o,.-.,yn)6B}1{J=n}P(^li/0'-- ■ ,^n)] n=0 CXJ = E[1{(V0,-,V»€B} /, l{J=n}P(^l*0. • • ■ . Vn)]- n=0 Therefore £~ l{J=n}P(.4|Yb, ■ • • ,Yn) is a version of P{A\Y0, desired. 7 The Coupling Time - Rates and Uniformity In this section we shall take a closer look at the coupling time T constructed in the last section in order to obtain results on rates of convergence and uniform convergence along the lines of Section 6 in Chapter 4. After establishing a useful lemma we show that T can be stochastically dominated by a manageable random variable T. This yields uniform total variation convergence over a class of processes (Theorem 7.1). We then establish finite moment results for T, which yields rate results for the uniform convergence (Theorem 7.2). Finally, we establish sharper moment results for T itself, which yields improved (but not uniform) rate results (Theorem 7.3). At the end of the section we consider some consequences of this for classical and wide-sense regenerative processes, and improve Blackwell's renewal theorem in the spread-out case. Throughout this section we assume that the conditions of Theorem 5.3 hold and write Ps to indicate that So = s with probability one. 7.1 Preliminaries In the nonlattice case let (Tfc,r^,/jb)g°, {Pk,P'k)Ti and M be as in the proof of Theorem 6.2 above. In order to treat the lattice case and the nonlattice case simultaneously we shall not base our argument in the lattice case on the proof of Theorem 6.1 but rather define (Tk,T'k, ifc)§°, (0k,/3'k)T' and M as in the proof of Theorem 6.2, replacing to and p from Lemma 6.3 by to '■= nod and p from Lemma 6.1 and with c = 0 and ^ the distribution having mass 1 at zero, /x({0}) = 1. The proof of Theorem 6.2 works in the lattice case after this modification. Thus in both cases there are distributional coupling times T and T" for Z and Z' such that T = TM- (7.1) .,Yj) as □
400 Chapter 10. REGENERATION Let (Tfe,f>fc)o° be a Markov process with state space [0, oo)2, initial distribution G®G', and transition probabilities P((fk+1,fk+1)e-\(fk^) = (s,s')) := P((Tk + l,T'k+1) £ -\(Tk,T'k) = {8,8'), Ik = 0), k > 0. Thus, with (Pk+i,P'k+1) '■= (n+i -Lk,fk+1 -Lk), where Lk := fkV f'k+t0, we have, for k > 0, P(^+ie-,^+1e-|(ffc,fi) = (*,*')) = P(/Jfc+1 e -,p'k+1 e -|(n,^) = (*,*'),/*=()) = P(fo+1 6-,/^+1 6-, 4 =0\(rk,r'k) = (s,s')) l-p2 _P(ft+i£-,ft+i 6-|(rfc,^) = (g,g'))-p2/x®/x l-p2 where the last identity is due to (6.18ft, c). This and (6.17) yield, for k > 0, P0k+1 e -J'k+1 e -\(fk,fk) = (s,s')) = Ps(B{sVa,+t0)- e -)P.> (B(,v,>+to)- 6 ■) - p2/x ® /i (7-2) i-p2 Lemma 7.1. £et V have the distribution \i and let V, M, and {fk,f'k)o' be independent. Then rj, D _ 1 = TM = LM-i+V = (f0 V f0 + t0) + 0i V ~p[ + t0) + • • • + 0M-1 V P'M-1 + *o) + V. Proof. The nondistributional identities are obvious. We shall prove that T = Lm-i + V. From (7.1) and tm = Lm-i + Pm we obtain T = Lm-i + Pm, and according to (6.22a,b), Pm has the same distribution as V and is independent of Lm-i- Since V is independent of Lm-i, it only remains to establish that Lm-i = Lm-\- We shall do so below.
Section 7. The Coupling Time - Rates and Uniformity 401 Due to (6.18ft) and since (Tk,T'kJk)T is Markovian, we have, for k > 0 and i £ {0,1}, P((to,^) e -,/0 = 0 = P(/o = i)P((f0,fi) e •) and P((Tfe+i,Tfc+1) e-,Ik+l =i\(Tkyk) = (S,s')jk =0) = P(/fc+1 = i)P((f,+1,f^+1) e -|(f*,-ri) = (*,*')). This and {M = m + 1} = {I0 = 0,..., Im = 0, Im+\ = 1} yield P{(t0,t0) £ ■,..., (Tm,T'm) e-,Af = m + l) = P(Af = m + l)P((7b,fi) e •,..., (fm,0 6 ■), m ^ 0. This and the independence of M and (ffc,Tfc)o° yield P((rM-i,r^_1) e-,Af = m + l) = P((fM-i,r^_1) e-,M = m + l). Sum over m > 0 to obtain P((tm-i,t'm_1) £ •) = P((fAf-i,f^f_1) £ •)• Since LM-i = tm-i Vt'm_1 +t0 and LM-\ = tm-\ Vt^i +*o, this yields Lm-i = Lm-i as desired. □ 7.2 Dominating T Stochastically — Uniform Convergence For a probability distribution function F on [0, oo) with J xF(dx) < oo, let Gp denote the probability distribution function on [0, oo) having density (1-F)/ JxF(dx), that is, f*(l-F(y))dy GP{X)= fyF(dy) ' x^°°)- Note that if F is a recurrence distribution for a renewal process, then Gp is the stationary delay-length distribution. We shall now establish uniform total variation convergence under the conditions of Theorem 5.3. Theorem 7.1. Let (Z,S) be time-inhomogeneous wide-sense regenerative and (Z',S') be a version of (Z,S). Let G be the distribution of So, G' the distribution of S'0, and Fs, s 6 [0, oo), the recurrence distributions. Let G be a probability distribution on [0, oo) such that G(x) > G(x) and G'(x) > G(x), x e [0, oo). (7.3)
402 Chapter 10. REGENERATION Suppose either there is a subprobability density f on [0, oo) such that Jf>0 and, for s G [0,oo), FS(B)> f f, BeB[0,oo), (7.4) Jb or there is a d > 0 and an aperiodic subprobability mass function f on {d, 2d, 3d,...} such that for s G [0, oo), Fs({kd}) > f(kd),k^ 1, and S and S' are dZ valued. (7.5) Further, suppose there is a probability distribution function F on [0, oo) such that J xF(dx) < oo and, for s G [0, oo), Fs(x)^F(x), ze[0,oo). (7.6) Then there are distributional coupling times T and T' for Z and Z' and independent random variables M, Yo,Y\,... such that M is geometric with parameter determined by f and F, Yq and Y\ have the distribution G, Y2,Y$,... are i.i.d with distribution determined by f and F and there are finite a, b such that~P(Y2>x)^a(l — G-p{x)), x & [b, oo), D and, with ^ denoting stochastic domination, T^T:=Y0 + --- + Y3M. (7.7) Moreover, with \\ ■ || denoting total variation, we have \\P(8tZ G •) - P{6tZ' G -)|| < 2P(f > 0, t > 0, and thus sup* \\P(0tZ G •) - P(StZ' G -)l| -> 0, t-+oo, where sup* means the supremum over all pairs (Z, Z1) of time-inhomogeneous wide-sense regenerative processes of the same type satisfying the conditions (7.3) through (7.6) with G, f, and F fixed. The total variation part of this theorem follows from (7.7); see Section 5.4 in Chapter 4. Thus it only remains to establish (7.7). We shall do so in four steps below.
Section 7. The Coupling Time - Rates and Uniformity 403 First step of proof. We shall start by showing that there is a constant ao determined by / and F such that sup Ps(Bs+t--1> x) ^a0(l-Gp(x)), ze[0,oo). (7.8) «,te[o,oo) To that end, note first that according to (6.10) there is a constant b0 < oo determined by / such that sup Es[7Vs+t+1 - Ns+t_] ^ b0- s,te[o,oo) This yields the next-to-last inequality in the following: for s,t,x 6 [0, oo), Ps(Bs+t_-l>x) OO = ^ Ps(Xl + • • • + Xi-! <t,X1 + --- + Xi>X+l + t) »=1 oo « = J2 (1 - Fs+y(x + l + t- y))Ps(Xi +■■■ + Xi-i £ dy) i=i Jlo,t) OO „ ^ Y, / Q-F(x + l + t- y))P,(X! +■■■ + AVi edy) i=i -A0-') oo oo 5$ J] I](1 - ^ + n))ps(* -n<X1 + --- + Xt-! ^t-n + 1) i=l n=l oo = ^2(l-F(x + n))Es[Ns+t-n - Na+t-n-i] n=l oo ^feo^(l-^ + ")) n=l /•oo Oo/ (1-F(y))dy. Put a0 = b0JyF(dy) to obtain (7.8). Second step of proof. We next show that if (V0, V\) and (W0, Wi) are pairs of nonnegative random variables such that Vo^W0 and, for s and x 6 [0, oo), P(V| ^ x\V0 = s) =: Gs(x) ^ H,(x) := P{WX ^ x\W0 = a), (7.9)
404 Chapter 10. REGENERATION then Vo + Vi^Wo + Wi. (7.10) This is an extension of Corollary 3.1 in Chapter 1 and we shall use the same method of proof, the quantile coupling. Let Uq and U\ be independent random variables that are uniform on [0,1], and for s 6 [0, oo), let Gj1 and H~l be the generalized inverses of Gs and Hs- Then, for s 6 [0, oo), P(G;1(Ul)e-) = P(vle-\Vo = s), PiH-'iU,) e •) = P(Wi e -\Wo = s), G^O/iK^-1^) [due to (7.9)]. Let g and h be the generalized inverses of the distribution functions of Vq and W0. Then 9(Uo) = Vo, h(U0) = Wo, g(U0) ^ h(U0) [due to V0 J W0}. Due to Fact 3.1 in Chapter 6 and since Uo and U\ are independent, we have, for s 6 [0, oo), P(G;(1[/o)([/1) e -\g(Uo) = s) = P(G;1([/1) e •), P(H-1C/0)(f/i) e -\h(U0) = s)= PiH^iU,) e •)• Combining all this implies g(U0) + G^Uo)(Ul)^Vo + Vl, h{U0) + H-lUo)(Ul)^W0 + W,, g(U0) + G-lUo){JJ{) ^ h(U0) + H^Uo){U{). This yields (7.10).
Section 7. The Coupling Time - Rates and Uniformity 405 Third step of proof. Let Y0,Yi,... be independent and independent of M. Let Y0 and Yi have the distribution function G. Let Y2,Y3,... be i.i.d. with distribution defined as follows: with a0 as at (7.8) put a : = ao(l - p2)~1/2 and b := c V (t0 + 1) and let P(Y2>X) = i1> X<b> V ' {(fl(l-Gf(i)))Al, x^b. We shall in this third step show by induction that for all n > 0, Ln J (Y0 V Yi + Y2) + ■ ■ ■ + (Y3n V Y3n+1 + Y3n+2). (7.11) First note that (7.11) holds for n = 0, since L0 = t0 Vf0 + to, since GG (the distribution function of Y0 V Y\) dominates GG' (the distribution function of T0 V t0) stochastically (that is, GG' > GG), and since Y2 ^ *o- Now suppose (7.11) holds for some n ^ 0. Observe that Ln+1 = Ln + 0n+l - 1) V (&+1 - 1) + (t0 + 1). (7.12) From (7.2) we obtain that for x G [0, oo), P((0n+1 - 1) V (#,+1 - 1) ^ a:|(Tn,<) = (a, a')) (1-P)2 1-p2 (1 ~ P)P 1 — p Ps(B{sVs,+to)_ - 1 s$ x)Ps,(B{sVs,+to)_ -l^x) + \ _P,2PPs(B{sVs,+to)- - 1 ^ x)(i([0,1 + x}) + (i_P2Pp«'(B(«v»'+«o)- - 1 < *)M[0,1 + a:])- Due to (7.8), Ps(5(sVs,+fo)_ -1 ^ x) and P's(5(sVs,+to)_ -1 ^ x) are both greater than or equal to P(Y"2 ^ x), and so is /z([0,1 4- x]) since Y2 ^ c. Thus, for x G [0, 00), P(0n+i - 1) V (#,+1 - 1) ^ x\(fn,f^) = (s,s')) > P(Y3{n+l) ^ x)P(Y3(n+1)+l^ X) = P(Y3{n+i)VY3{n+1)+l^x). Since {fk,f'k)^' is Markov, we have that (/3n+i - 1) V (/3'n+1 - 1) depends on fn only through (fn,f'n). Thus, for x G [0, 00), P((/3n+1 - 1) V (p'n+1 - 1) ^ i|Ln) ^ P(Y3(n+1) V y3(n+1) + 1 < 1).
406 Chapter 10. REGENERATION Due to this and the induction hypothesis (7.11), (7.9) holds with Vo = Ln, V1 = (&+! - 1) V 0'n+1 - 1), W0 = (Y0VY1+Y2) + --- + (Y3n V Y3n+1 + Y3n+2), Wi = ^3(n+l) V *3(n+l) + 1; and (7.10) yields Ln + (Pn+l - 1) V (&+1 - 1) ^ (Y0\/Yl+Y2) + .. .+ (Y3n\ZY3n+1+Y3n+2)+Y3{n+1)\/Y3in+1) + l. This, (7.12), and Y3^n+l^+2 ^ to + 1 yield that (7.11) holds with n replaced by n + 1. Thus by induction, (7.11) holds for all n > 0. Fourth, and final step of proof. Since Mis independent of Yo,Yi,... and also of Lo, L\,..., we obtain from (7.11) that D Lm-\ ^ (*0 V YY + Y2) H h (l3(M-l) V ^3(Af-l) + l + *3(Af-l) + 2)- By Lemma 7.1 we can take T = Lm-\ + V and since V ^ c ^ Y3m, this yields D T ^ (Y0\/Yi+Y2) + . . . + (l3(Af-l)Vy3(Af-l) + l+^3(Af-l)+2)+^3Af- The right-hand side is dominated by the right-hand side of (7.7). Thus (7.7) holds, and a reference to Section 5.4 in Chapter 4 yields the total variation claims. □ 7.3 Moment Results for T — Uniform Rates of Convergence Let X be a nonnegative random variable with distribution F. Say that X and F have a finite geometric moment (or finite exponential moment) if there is a p > 1 such that E[px] < oo. Let ip be a nondecreasing function from [0, oo) to [0, oo). Say that X and F have a finite ip moment if E[<p(X)} < oo. Define a function # from [0, oo) to [0, oo) by $(x) = / (f(y)dy, ze[0,oo). Jo
Section 7. The Coupling Time - Rates and Uniformity 407 If ip has a density with respect by Lebesgue measure, denote it by ip, that is, ip(x)=ip(0) + <p(y)dy, ze[0,oo). Jo Let A be the class of nondecreasing functions <p from [0, oo) to [0, oo) having a density ip and satisfying log<z>(:r) ... , _. is nonmcreasmg in x and goes to 0 as x —> oo. a; If ip £ /l, then y> increases more slowly than any increasing geometric function (that is, for all p > 1 it holds that ip(x)/px -> 0 as a; -> oo). Also, if ip(x) = px for some p > 1, then <£> ^ /l. Otherwise, /l is quite general. For instance, if ip{x) = px for some p > 1 and 0 < (3 < 1, then ip£/l; if <p(x) = ea \/xa for some a > 0, then y> 6 /l; if <fi(x) = eVloga;, then <p G /l; if y £ yl and a > 0, then ya £ /l; if ip 6 /l and i/> 6 /l, then yi/) 6 A. The stochastic domination result (7.7) yields the following powerful rate results for the uniform convergence in Theorem 7.1. Theorem 7.2. The following claims hold with sup* denoting the supre- mum over all pairs (Z, Z') of time-inhomogeneous wide-sense regenerative processes of the same type satisfying the conditions (7.3) through (7.6) in Theorem 7.1 with G, ip, and F fixed. (a) If G and F have finite geometric moments, then so has T, and thus the uniform convergence is of geometric order: there is a p > 1 such that pl sup* \\P(0tz e •) - PtftZ' e -)II -+ o, t -+ oo. (6) Let if £ A and suppose G has finite <p moment and F finite 4> moment. Then T has finite ip moment, and thus the uniform convergence is of order ip: <p(t) sup* \\P(0tZ G 0 - PtftZ' e OH ->• 0, t -»• oo; and of moment-order ip: /•OO / <p(t) sup* ||P(04z e 0 - v(°tZ' e oil* < oo. Jo
408 Chapter 10. REGENERATION Proof. Let M,Y0,Yi,...,T be as in Theorem 7.1. We shall need the following observation: if p > 1, p is a nondecreasing function from [0, oo) to [0, oo) with density <p, and F is a distribution function on [0, oo), then I pxF(dx) <oo O I px(l-F(x))dx < oo, (7.13a) ip(x)F(dx) <oo O / <p(a;)(l-F(a;))da;<oo. (7.136) See Section 5.4 in Chapter 4 for (7.136) and note that (7.13a) follows from (7.136), since if <p(x) = px, then <p(x) = px\ogp. To establish (a), suppose G and F have finite geometric moments. Then J pxF(dx) < oo for some p > 1, and (7.13a) yields /0°° px(l-F(a;)) dx < oo. Thus JpxGp(dx) < oo, and (7.13a) yields J*0°° px(l-Gp(x)) dx < oo. Since P(F2 > z) ^ a(l - Gp(i)) for a; > 6 this yields /0°° pxP(Y2 > x) dx < oo, and (7.13a) yields E[py2] < oo. Since pY2 decreases to 1 as p decreases to 1, this and dominated convergence yield that E[py2] decreases to 1 as p decreases to 1. Since M has a finite geometric moment, so has 3M — 1. Thus we may take p close enough to 1 for E[E[py2]3M_1] < oo. Since lo, Y\,..., M are independent and Y2, Y3, ■ ■ ■ are i.i.d., we have E[pT] = E^E^EtE^]3**-1]. Take p close enough to 1 for the three factors on the right-hand side to be finite. Thus E[pT < 00 for some p > 1. This and a reference to Section 6 in Chapter 4 completes the proof of (a). In order to establish (6), take ip 6 A and suppose G has finite ip moment and F finite # moment. Then (7.136) yields f^° ip(x)(l - F{x))dx < 00. Thus J(p(x)Gp(dx) < 00, and (7.136) yields J0°° <p(x){l - GP(x)) dx < 00. Since P(Yz > x) ^ a(l — Gp(x)) for x > 6, we obtain from this that J0°° ip(x)P(Y2 > x)dx < 00, and (7.136) yields E[^(F2)] < 00. Due to Lemma 7.2(a) below, Efo>(f)] ^ E[p(Y0)]E[p(Y1)}E[p(Y2) + ■■■ + ip(YM)]. The first two factors on the right-hand side are finite by assumption and, due to E[<£>(Y2)] < 00 and Lemma 7.2(6) below, so is the third. Thus E[<£>(?)] < 00. This and a reference to Section 6 in Chapter 4 completes the proof of (6). □ In the above proof we needed the following key properties of the class A. Lemma 7.2. (a) If p £ A, then p(x + y) ^ <p(x)<p(y), x, y e [0,oo).
Section 7. The Coupling Time - Rates and Uniformity 409 (6) Let Wi,W2, ■ ■ ■ be i.i.d. nonnegative random variables that are independent of a nonnegative integer-valued random variable K with finite geometric moment. If ip 6 A and ~E[<p(Wi)] < oo, then E[p{Wi + ■■■+ WK)} < oo. Proof. Take ip £ A and recall that log* (n(x) a(x) : = is nonincreasing in x and goes to 0 as x —> oo. x We obtain (a) as follows: for x, y 6 [0, oo), <p(x + y) = e<x+v)x e<x+v^ < e<x)x ea^y = ip{x)ip{y). In order to establish (b), note that for x 6 [0, oo), E[tp(Wi +■■■ + WK)) s$ E[ip(x + W1+--- + WK)} — E\ea(x+Wl~{—^wk)(x+w1+---+wk)-i < e*(x)xE[ea(-x+Wl+-+WK){Wl+-+WK)] (a(-) nonincreasing) _ ea(x)x grea(a:+WH \-WK)Wi e<i(x+Wi-\ \-WK)WKl ^ ea{x)x E[e<x+Wl)Wl ... ea(x+WK">WK] (o(-) nonincreasing) = e<x)x E[E[ea(x+H/l>H/l]x] (K independent of the i.i.d. Wk). Since a(-) decreases to 0, we have that ea(x+w^)w^ _>. i as x ->• oo and also that ea(x+^iWi ^ ip(Wi). Thus, by dominated convergence, E[v3(Wri)]<oo implies E[ea^x+Wl)Wl] -»• 1 as x -+ oo. Thus if K has geometric moment, then we can take x large enough for E[ea(-x+w^Wi] to be close enough to 1 for E[E[ea^x+w^w']K] < oo. Thus (b) holds. □ 7.4 Sharper Moment Results for T - Nonuniform Rates In Theorem 7.2 we needed a finite # moment of F to obtain a finite ip moment of T for ip e /l. Recall that the functions ip 6 /l increase more slowly than any increasing geometric function. We shall now consider increasing power functions ip(x) = xa and, more generally, certain functions ip that increase more slowly than some power function, such as p(x) = :ra(e Vloga;), for instance. For these functions we relax the finite # moment condition for F to a bounded <p moment condition directly on the recurrence distributions Fs, s £ [0, oo), themselves (not on the stochastically dominating F) to obtain a finite p moment of the coupling time T itself (but not of
410 Chapter 10. REGENERATION the stochastically dominating T). This yields sharper rate results for these functions at the expense of losing the uniformity. Let a > 0 and let X be a nonnegative random variable with distribution F. Say that X and F have a finite a moment if E[Xa] < oo. Theorem 7.3. Under the conditions of Theorem 7.1 the following claims hold. (a) Let a > 0 and suppose So and S'0 have finite a moments and sup / xaFs(dx) < oo {bounded a moments). s6[0,oo) J Then T has a finite a moment, and thus the convergence is of power order a: ta\\P(0tZe-)--P(0tZ'e-)\\->O, t^oo; and of power moment-order a — 1: /•oo / r-^lP^z e ■) - PtftZ' e -)\\dt < oo. (b) Let ip be a nondecreasing function from [0, oo) to [0, oo), with ip(0) = 0 and lirriz-xx, <p(x) = oo, and having density ip with respect to Lebesgue measure. Suppose either (f is concave, E[<p(So)] < oo, and E[(f(S'Q)] < oo (7-14) ip is convex, ip is strictly increasing, ip(0) = 0, and lim <p(x) = oo, x—>oo there is a c < oo such that <£>(2x) ^ap(x) for x6 [0, oo), <■'• ' E[ip(S0)]<oo,E[ip(S'0)]<oo, and sup hpdF3 <oo. se[o,oo)J Then E[ip(T)] < oo, and thus the convergence is of order <p: <p(t)\\P(0tZ e •) - PtftZ' e i-)\\ -► 0, t -»• oo; and of moment-order <p: /•OO / <p{t)\\P(0tZ€-)-V(0tZ'€-)\\dt< Jo
Section 7. The Coupling Time - Rates and Uniformity 411 Comment. Since (a) is a special case of (ft), it suffices to establish (b). We shall do so in six steps below. The proof relies on certain facts about the convex functions at (7.15) not proved here. Readers not content with that, or who only want (a) and do not wish to go into the generality of (6), can obtain (a) by working through the proof below with ip(x) = xa and with the so-called Orlicz norm || • \\v replaced by the better-known La norm || • ||a. First step of proof. We shall first show that if <p is a nondecreasing function and sups6r0 ^ J ipdFs < oo, then there are finite constants a\ and b\ such that sup Es[y(5s+(_)] s$ai +M, *e[0,oo). (7.16) s6[0,oo) For that purpose, fix s, t 6 [0, oo) and note that Bs+t- ^XNs+t_ ^ max{X0, ...,Xjvj+(_}, and thus (since <p is nondecreasing) ip(Bs+t-) ^ max{y(X1),.. .,<p{XNs+t_)} ^ ip(X1) + .. . + ip(XNa+t_). The sum on the right-hand side equals Xl^=i <p(Xk)l{N3+t-^k}! and thus taking expectations and interchanging sum and expectation yields EMBs+t-)] ^ J2EMXk)l{Nl+t^k}]- (7-17) /t=i Now, l{Na+t_^k} = l{sjt_1<s+t}, which yields the equality in Es[ip(Xk)l{N3+t_2k}\Sk-i] = Es[<p(Xk)\Sk-i]l{N,+t->k} ^ cil{jv,+(_^fc}, where ci := sup ipdFr. r6[0,oo) J Take expectation to obtain from this and (7.17) the first inequality in oo EMBs+t-)} ^ ci£e,[1{JVi+,_^}] fc=i = ciESE V^,-^}] = ciE,[JV.+t_] ^ ci ( - H—- ), where a and b are as at (6.10). \b ab) This yields (7.16).
412 Chapter 10. REGENERATION Second step of proof. We next show that if <p is a nondecreasing function and sups6r0]OO) J <pdFs < oo, then there are finite constants a2 and fe2 such that E[¥.()9nV)9;)|rn-i,f;_1]<a2+62()9n-iV)9;-1), n^l, (7.18) where 0k,^'k)f and (ffe,f^)o° are from Lemma 7.1 and (/3o,J30) '■= (to,t0). In order to establish (7.16), fix n > 1 and note that ip(xVy)=ip(x)V <p(y) ^ip(x)+ip(y), x,ye[0,oo), (7.19) to obtain Eb09nV#l)|fn_1,f;_1] <E[¥.(4„)|fn_i,^_1]+E[¥.()9„)|f„-1)f;_1]. Due to (7.2), we have P(4ne-|(f„_1)f;_1) = (s,0) <P.(B(.v..+4o)_e ■)/(!-p2)- (7.20) (7.21) Combine this and (7.16) with t replaced by s V s' + to — s [and note that s V s' + to — s = (s' — s)+ + to] to obtain n<p(0n)\(fn-1X_1) = {s,s')]^(a1+b1((a' -s)+ + t0))/(l-pi). In the same way we get E[^;)i(fn_1,f;_1) = (S,s')]^(ai + foi((*-*')+ + *o))/(i-P2). Add these two inequalities [and note that (s' — s)+ + (s — s')+ = \s — s'|] to obtain, due to (7.20), that E[^(/3n V &n)\fn^,f'n^} ^ (20! + 2Me> + feil^n-i - r;_1|)/(l - p2). Now, |f„_i - f;_x| = |0n_, - #,_,! ^ /?„_! V #,_!, and (7.18) follows. Third step of proof. We now show that if ip is a nondecreasing function and lim ip(x)/x = oo and sup / <pdFs < oo, (7.22) x^°° se[o,oc)J then there are finite constants a^ and &3 such that E[V0nV0'n)\Tt>,TJo]^a3+b3{TO\/TJo), n>l. (7.23)
Section 7. The Coupling Time - Rates and Uniformity 413 To this end, fix n > 1 and note that for each e > 0 there is a ce such that x ^ c£ + e<p(x) for all x £ [0, oo). This and (7.18) yield E[^(/3n V #,)|f„_i, f;_i] ^ o2 + 62(ce + e^-i V &„_{)). Put e := (2&2)-1 and d3 := a2 + b2ce to obtain EM^nV^)|fn-1,f;_1]^d3 + 2-V(^n-lV^_1). Take conditional expectations with respect to (f„_2,T^_2), ■ ■., (ti,t{), recursively, to obtain n<P{0n V #,)|fi, f[] ^ d3(l + 2-1 + • • • + 2"-2) + 2n"V(A V #)). Applying (7.18) and 1 + 2_1 + • • • + 2n~2 ^ 2 and 2"-1 ^ 1 yields EM/3n V ^)|ro,ro] ^ 2d3 + a2 + b2(f0 V f'Q). Thus (7.23) holds. Fourth step of proof. We now show that if 93 is a nondecreasing function and (7.22) holds and also E[<£>(So)] < 00 and E[<£>(Sq)] < 00, then there is a constant a4 such that E[tp(f0 V fj)] s$ 04 and E[^(/3„ V #,)] s$ o4 for n > 1. (7.24) For this purpose, note that E[y>(So)] < 00 and E[^(5q)] < 00 implies E[So] < 00 and E[5q] < 00 due to lima._HX) ip(x)/x = 00. Put a4 = (Efe>(S0)] + E[<p{S'Q)]) V (03 + fe3E[S0] + b3E[S'0}) to obtain from (7.19) that EMf0 V ?0)] < Efc>(S0)] + E[<p(S'0)] ^ a4 and from (7.23) that EM/3n V ^)] ^ a3 + b3E[S0] + fo3E[S^] ^ a4, that is, (7.24) holds. Fifth step of proof. We shall now establish (6) in the convex case (7.15). So assume (7.15). See Garsia (1973), Krasnoselskii and Rutickii (1961), and the appendix of Neveu (1972) for the following facts about convex functions ip as at (7.15): f(x/a) ^ ip(x)/a, a £ [l,oo), x £ [0, 00), (7.25a) a := sup xip(x)/ip(2x) e [l,oo), (7.25b) x6[0,oo) ip(x/a)^<p{x)/aa, a 6 [l,oo), iG[0,oo), (7.25c)
414 Chapter 10. REGENERATION and the Orlicz norm || • \\v (an extension of the La norm || • ||a) is defined by \\X\\v:=mf{a>0:E[ip{X/a)]^l} for nonnegative random variables X such that the set on the left-hand side is nonempty. From (7.25a) we obtain ||X||V<1VE[^(X)], (7.26) and from (7.25c) we obtain Eb(X)UlV(||XyQ. (7.27) Due to Lemma 7.1, oo E[<p{T)] = Y, EM^n-i + V)]P{M = n). (7.28) n=l From (7.24) and (7.26) we obtain, with a5 : = 1 V a4 V ||£0||^ V H^l^, maX{||T0VTo||v, \\t0\\v, ||A VftlU lift Vft||„, ••-, \\V\\V} ^ aB. (7.29) Now (using Ln_! = f0 Vf0+n£0 + A V/3[ H \-Pn-iV0'n-i f°r the second inequality), E[¥)(Ln_1 + V)] ^ 1 V ||Ln_, + V||V)Q [due to (7.27)] n-l ^ 1 V (||f0 V f0\\v + n\\t0\\v + J2 \\h V All <p + 11* lip) k = \ s$ (a5)a(2n+l)a [due to (7.29)]. From (7.28) we now obtain oo E[¥>(T)] ^ (o5)Q £(2n + 1)QP(M = n), n=l which is finite since M is geometric. This and a reference to Section 6 in Chapter 4 completes the proof of (b) in the convex case. Sixth, and last step of proof. We now establish (ft) in the concave case (7.14). So let <p be concave and assume that E[y>(So)] < oo and E[<£>(Sq)] < oo. Due to Lemma 7.3 below, it follows from the condition f xF(dx) < oo that there is an increasing function ip with limx_>oo ip(x) =
Section 7. The Coupling Time - Rates and Uniformity 415 oo such that J%pdF < oo. Since (7.22) is satisfied with ip replaced by ip, there are constants a§ and be such that E[ip(pnV p'n)\f0,f£^a6 + b6(f0Vf^, n>l. (7.30) It is no restriction to assume that ip(x) > x for all x G [0, oo). This yields the second inequality in E[<p{0n V p'n)\f0, f0] ^ tpCElpn V #,|f0) fj]) (Jensen's inequality) ^ ip(E[fl>(fin V4;)|7b,fi]) 0/>(x) > x for all x G [0,oo)) ^^(o6 + fo6(roV^)) (due to (7.30)) ^ y(o6) + be(p(fo V Tq) (since 93 is concave). Take expectation to obtain E[ip(/3n V /3'n)] ^ y(a6) + &6E[<£(fo V 7q)] for n ^ 1. Due to (7.19), E[^(f0 V f^)] ^ E[^(50)] + E[^(5^)], which is finite by assumption. Thus there is a finite constant c$ such that Efe>(f0 V fj)] ^ c6 and E[^(/3n V #,)] ^ c6 for n > 1. (7.31) Thus (using Ln_! = f0 Vf0 + nt0 + A V^+•■•+ /?„_! V $'n_x and the concavity of ip for the first inequality) Efc>(Ln_i + V)] n-l ^ E[^(f0 V fo)] + «¥>(*„) + Y. E^n V #•)] + E[^)] fc = l ^ n(^(£0) 4- c6) + y)(c) (due to (7.31) and V ^ c). This and (7.28) yield OO E^(T)] ^ £>(¥>(*(,) + c6) + v>(c))P(M = n), n=l which is finite since M is geometric. This and a reference to Section 6 in Chapter 4 completes the proof of (b) in the concave case; and Theorem 7.3 is established. □ The following lemma was used in the last step of the above proof. Lemma 7.3. If X is a nonnegative random variable and E[X] < 00, then there exists an increasing function tp with limx_>0O ip(x)/x = 00 and such that E[ip(X)] < 00. PROOF. The result is trivial if there is an x < 00 such that P(X > x) = 0. So suppose P(X > x) > 0 for all x G [0,00) and put tn = ini{x > 0 : E[Xl{x>x}] <: 1/2"}, n > 0.
416 Chapter 10. REGENERATION By dominated convergence E[Xl{x>x}] —> 0 as x —> oo, and thus tn increases strictly to infinity as n —>• oo. Define ip by ip(0) = 0 and, for x > 0, i/>(i) = Hji, where nx is such that tUl < x ^ t„I+i. Then i/> is increasing and ip(x)/x = nx —>• oo as a; —>• oo. Moreover, i/>(X) ^ X^n-X^xx,,}. and thus oo oo E[^(X)] ^ J>E[X1{X>M] ^ J>/2" < °° 0 0 as desired. □ 7.5 The Time-Homogeneous Case We shall end this overlong section by considering briefly some consequences of the above theory in the time-homogeneous case, that is, in the special case of classical regenerative processes (Section 3) and wide-sense regenerative processes (Section 4). There are two aspects of the time-homogeneous case that make this worthwhile. Firstly, the conditions simplify, since the recurrence distribution F does not depend on the time of regeneration. Secondly, there exists a stationary version and the results on asymptotic stationarity (Theorem 3.3 and Theorem 4.3) are improved by the above theory. Theorem 7.4. Let (Z*,S*) be stationary and classical regenerative, or stationary and wide-sense regenerative. Let F be the distribution of X^ and let G be a fixed probability distribution function on [0,oo). Then the following claims hold. (a) Suppose X{ is spread out. Then 6tZ%Z\ t-^oo, uniformly in versions (Z, S) of (Z*, S*) with delay length Sq satisfying P(S0 ^ x) > G(x), x £ [0, oo). (7.32) Further, this uniform convergence holds uniformly in (Z*,S*) satisfying P(XZ s$ x) > F(x), x £ [0, oo), (7.33) P(X; + ---+X*£B)>[f(x)dx, BeB[0,oo), (7.34) Jb with F, f, and n fixed and such that f xF(dx) < oo and j0 / > 0.
Section 7. The Coupling Time - Rates and Uniformity 417 (b) Suppose X* is lattice with span d and let (Z**,S**) be the periodically stationary version of (Z* ,S*). Then 6ndZ % Z*\ n ^ oo, uniformly in versions (Z,S) of (Z*,S*) with dli valued delay length So satisfying (7.32). Further, this uniform convergence holds uniformly in (Z*,S*) satisfying (7.33) and P(X; = kd) > f(kd), k > 1, (7.35) with F and f fixed and such that J xF(dx) < oo and f is aperiodic. In both cases [(a) and (&)] the following rate results hold: If G and F have finite geometric moments, then the uniform convergence is of geometric order. If <p G A and G has finite p moment and F has finite <£ moment, then the uniform convergence is of order p and of moment order (p. PROOF. To obtain (a) from Theorem 7.1, take (Z',S') := (Z*, (S*kn)^=0) and note that (7.33) implies (see Section 3.3 in Chapter 1) P(X*1+--- + X*n^x)>Fn(x), ie[0,oo), where Fn is the distribution of the sum of n independent random variables with distribution F (the nth convolution power of F). Thus the condition (7.6) in Theorem 7.1 holds with F replaced by Fn. The condition (7.4) follows from (7.34). Further, with a := 1/ J0°° yf(y)dy we have [using F(y) ^ F(y) and 1/ J yF(dy) ^ a for the inequality] that for x £ [0, oo), P(S0* ^x) = l-J°°(l- F{y))dy/ j yF(dy) >(l-a|0°(l-F(2/))d2/)+ (7.36) = (l - aGp(x) j yF(dy))+ =: G(x) (say). Since G(x) ^ 1 and G(x) ^ 1, this yields P(S£ ^ x) > G(x)G(x), while (7.32) yields P(S0 ^ x) > G(x)G(x) for x G [0,oo). Thus the condition (7.3) in Theorem 7.1 holds with the distribution function G replaced by the distribution function GG. Thus we obtain (o) from Theorem 7.1. In order to obtain (ft) from Theorem 7.1, take (Z',S') := (Z**,S**) and note that the condition (7.6) in Theorem 7.1 follows from (7.33) and that the condition (7.5) in Theorem 7.1 follows from (7.35). Furthermore,
418 Chapter 10. REGENERATION P(Sq* ^ x) > P(Sq ^ x) for x G [0,oo), and thus (7.36) [this time with a := 1/ ^^° kdf(kd)\ again yields that the condition (7.3) in Theorem 7.1 holds with the distribution function G replaced by the distribution function GG. Thus we obtain (b) from Theorem 7.1. It order to obtain the rate results from Theorem 7.2, we shall first show that if ip G A and X and Y are nonnegative random variables, then E[#(X)] < 00 and E[#(F)] < 00 implies E[#(X + Y)} < 00. (7.37) To this end, note that for i£[l, 00) and y G [0, 00), rv <$>(x + y) = <P(x) + / <p(x + s) ds Jo ^ #(z) + ip(x)$(y) [due to Lemma 7.2(a)] ^ #(z) -f <p(l)ip(x - l)^(y) [due to Lemma 7.2(a)] ^#(x)(l+^(l)#(2/)) [^(1-1)^/ ¥>(s) ds «; *(a:)]. This yields the second step in E[#(X + Y)] ^ E[#((l V X) + y)] ^ E[#(l V X)](l + p(l)E[#(y)]) [X and y independent] ^(#(l)+E[#(X)])(l + ^(l)E[#(y)]). This yields (7.37). From (7.37) it follows (recursively) that if <p 6 A and F has finite # moment, then so has Fn. Note also that if p > 1 and J pyF(dy) < 00 then J pyFn(dy) = (Jpyp(dy))n < 00. Thus in the spread-out case (a) the moment conditions on F transfer to Fn. Thus in both cases (a) and (ft) the moment conditions in Theorem 7.2 on the distribution function dominating the recurrence distributions stochastically are satisfied. Thus it only remains to establish that this is also true for the distribution function dominating the delay-length distributions stochastically. In order to establish this, note first that GG is the distribution function of the maximum of two independent random variables with distribution functions G and G, respectively, and thus / ipd(GG) ^ / ipdG + / ipdG for nonnegative ip. (7.38) Note next [see (7.13a)] that if F has a finite geometric moment, then so has Gp, and thus so has G. Note finally [see (7.13ft)] that if if G A and F has a finite # moment, then Gp has a finite ip moment, and thus so has G. This and (7.38) yield that if G and F have finite geometric moments, then
Section 7. The Coupling Time - Rates and Uniformity 419 so has GG, and if ip 6 A and G has finite <p moment and F has finite # moment, then GG has finite ip moment. Thus Theorem 7.2 yields the last claim of the theorem. □ Two processes, Z and Z', both converging in total variation to the same stationary limit process Z*, can converge to each other at a faster rate. The following theorem is an example of this: the a and <p moment conditions on X\ show up directly as convergence of power order a and of order <p. This was also the order of the convergence to stationarity in Theorem 7.4, but the moment conditions on X\ were one order higher, namely a + 1 and #. Theorem 7.5. Let (Z,S) be classical regenerative or wide-sense regenerative. Let (Z',S') be a version of(Z,S). Suppose either X\ is spread out, or Xi is lattice with span d, and So and S'0 are d7L valued. Suppose further E[Xi] < oo. Let a > 0 and suppose So, S'Q, and X\ all have finite a moments. Then there are distributional coupling times for Z and Z' with finite a moments, and thus the convergence is of power order a : ta\\P(6tZ E-)-P(6tZ' e-)ll -+0, t-+oo; and of power moment-order a — 1: oo ta-l\\P{6tz e ■) - *(6tZ' e -)\\dt < oo. More generally, let <p be as in Theorem 7.3(b) and suppose So, S0, and X\ all have finite ip moments. Then there are distributional coupling times for Z and Z' with finite <p moments, and thus the convergence is of order <p : <p(t)\\P(0tz e ■) - PtftZ' e -)II -»• o, t -+ oo; and of moment-order ip: oo <f>(t)\\P(0tze-)-P{0tz'e-)\\dt <oo. Proof. This theorem is an immediate corollary of Theorem 7.3. □ 7.6 The Renewal Theorem — SpreaclrOut Case Blackwell's renewal theorem was established in Chapter 2 (Theorem 8.1) using epsilon-couplings, and improved in Chapter 3 (Theorem 6.2) using an exact coupling based on the Ornstein idea. The improved version states that when the recurrence times are spread out and have a finite first moment, then the limit result holds in total variation on bounded intervals. The coupling results of this section enable us to sharpen this further to hold in total variation on the whole half-line, provided that the recurrence times have a finite second moment and the delay time has a finite first moment.
420 Chapter 10. REGENERATION Theorem 7.6. Let S be a renewal process. For B G B([0, oo)), let N(B) be the number of renewals in B, that is, oo k=0 Let E[JV] 6e i/ie intensity measure, that is, the measure with mass E[N(B)] at B G B([0,oo)). Let E[JV(£ + ■)] be the measure on [0, oo) with mass E[JV(£ + B)] at B G S([0, oo)). Lei A fee i/ie Lebesgue measure on [0, oo). //Xi is spread out, E[X^] < oo, and E[5o] < oo, then the signed measure E[JV] - A/EfA-!2] is bounded and ||E[JV(t + ■)] - A/E[X,]|| ->0, £->oc. (7.39) Proof. Let 5' have the same recurrence time distribution as 5 and the delay time distribution Goo from Corollary 8.1 in Chapter 2 (note that Goo = GF). Then E[N'] = A/E[A"i] (see the proof of Theorem 6.2 in Chapter 3). With t G [0, oo) and B G B([0,i\), this yields the first step in |E[JV(B)] - A(B)/E[X,]| = |E[JV(B)] - E[JV'(B)]| A (7.40) *s£|E[JV(Bn)]-E[JV'(B„)]| n=0 where Bn:= BD [n,n + 1). Due to Theorem 6.2, there are randomized stopping times K and K' with respect to 5 and 5', respectively, such that with T = Sk and T' = S'K,, (N(T + -),T) = (N'(V + -),T'), which implies N(Bn)l{T^ny = N'(Bn)l{T,^ny. Apply this to (7.40) to obtain |E[JV(B)]-A(B)/E[X,]| *S £ |E[iV(i?n)l{T>n}] - E[JV'(B„)l{T,>n}]|. (7.41) n=0 Now N(Bn) s$ N{[SNn_,SNn_ + 1]), and thus E[JV(B„)l{T>n}] ^ E[N([SNn_,SNn_ + 1])1{T>„}]. The right-hand side equals E[N°]P(T > n) because the event {T ^ n} is independent of N([SNn_,SNn- + 1]) which is a copy of N? [since K is a. randomized stopping time]. Thus E[N(Bn)l{T>n}] ^ E[iY°]P(T > n).
Section 7. The Coupling Time - Rates and Uniformity 421 Similarly, E[JV'(Bn)l{T,>n}] ^ E[iV°]P(r' > n). Since P(T' > n) = P(T > n), this and (7.41) yield OO |E[JV(B)] - A(B)/E[X1]| ^ E[N!] £ P(T > n). n=0 Take the supremum in B G B([0, t]) and £ G [0, oo), multiply by 2, and use [see Lemma 5.2] ^2^=0 P(T > n) ^ E[T + 1] to obtain the following coupling inequality ||E[JV] - A/E[X!]|| ^ 2E[iVI0]E[r + 1]. (7.42) Since EfXj2] < oo, Goo nas a finite first moment (see (7.136). Thus So and S0 are both dominated stochastically by the distribution function GGoo, where G is the distribution function of S0- Since both G and Goo have finite first moments, so has GG oo- This and EfXj2] < oo imply, in view of Theorem 7.2(6), that E[T] < oo. This and (7.42) yield that E[iV]-A/E[X,] is bounded, which in turn implies (7.39). □ The following results on uniform convergence and rates of convergence can also be established along the lines of the proof of Theorem 7.4 Section 6. (This can be extended to regenerative random measures; see Thorisson (1983).) Let G be the delay time distribution of 5 and F the recurrence time distribution. Let G be a distribution function with a finite first moment, F a distribution function with a finite second moment, n a positive integer, and / a nontrivial subprobability density. Let sup* denote the supremum over all G that are stochastically dominated by G, and over all F that are stochastically dominated by F and such that the nth. convolution power of F has a density component /. Then sup* \\E[N] - A/E[X,]|| < oo (7.43) and sup* ||E[JV(t + ■)] - A/E[X,]|| -> 0, t ->■ oo. (7.44) Moreover, the same argument as the one leading to (7.42) yields ||E[JV(t + •)] - A/E[X,]|| ^ 2E[iVI°]E[(T - t)+ + 1]. From this it follows easily that if G and F have finite geometric moments then the uniform convergence at (7.44) is of geometric order. And if ip G A has a density ip which in turn has a density (p, and G has finite ip moment and F has finite # moment, then the uniform convergence at (7.44) is of order ip and of moment order (p.
422 Chapter 10. REGENERATION 8 Asymptotics From-the-Past How are things now (and from now on) if they started long ago? The traditional probabilistic way to answer this loosely formulated question is to start a stochastic process at time 0, consider its distribution in a time interval [t,co), and check whether it stabilizes as t —> oo (asymptotic stationarity); see Figure 8.1. ASYMPTOTICS TO-THE-FUTURE * x s' » ' J FIGURE 8.1. Realization of a process starting at time 0. This we have done repeatedly up to now, obtaining asymptotic stationarity for Markov chains in Chapter 2 and for classical regenerative and wide-sense regenerative processes in this chapter. This approach did not work, however, for (truly) time-inhomogeneous regenerative processes: according to Theorem 5.5, asymptotic stationarity forces a time-inhomogeneous regenerative process to be time-homogeneous in the long run. In this section we shall reverse this taking limits to-the-future approach as follows. We start a stochastic process at an arbitrary time r, consider its distribution in a fixed time interval [i, oo), and check whether it stabilizes as the starting time r goes backward to —oo, r I —oo; see Figure 8.2. ASYMPTOTICS FROM-THE-PAST ■ 1 ■ —oo <— y t fixed FIGURE 8.2. Realization of a process starting at time r. As an answer to the above question, taking limits from-the-past in this way is even more natural than taking limits to-the-future. Of course, for time-homogeneous processes the two approaches are equivalent. The point is that unlike taking limits to-the-future, this taking limits from-the-past approach also works for time-inhomogeneous processes and thus widely extends the class of processes admitting a limit law.
Section 8. Asymptotics From-the-Past 423 In this section we establish that there is a limit from-the-past of time- inhomogeneous regenerative processes (wide-sense or not) satisfying the conditions from Theorem 5.3. The proof is based on the stochastic domination result in Theorem 7.1. At the end of the section we extend this result to processes that are time-inhomogeneous regenerative only up-to-time-zero. 8.1 Preliminaries In order to take limits from-the-past we must consider processes with time set [r, oo). So fix an arbitrary r G (—oo,0] and add the following to the framework from Section 2. Let Z{T) = (^r))s6[r,oo) be a one-sided stochastic process with time set [r, oo), state space (E,£), and path set H^ obtained from the internally shift-invariant subset H of £[0,°o) by H(r> := {(zs-r)se[r,oo) : (zs)se[o,oo) £ H}. Let %^ be the trace of H^ on £lr'°°). For t G [r, oo), define the shift map 8t on H^ to be the map taking z = (.zs)s6[r)00) G H^ to 8tz := (zt+s)sE[o,co) £ H. Note that the process 6rZ^ has time set [0, oo) and that the process Z^ is shift-measurable if and only if 8rZ^ is shift-measurable. Let S<r> = (S£r))§° be a one-sided sequence of random times satisfying r ^ S{0r) < S[r) < > oo. Regard S^r' as a measurable mapping from (fl,J-) to the sequence space (Z,W,£(»•)), where i(r) = {(**)o° e [r, oo)*0'1'- > : so < si < • • ■ -► oo} = r + L, £(r) = L(r)nB{0,h...} = the Bord Subsets 0f £,(»•). For t G [r, oo), define the joint shift-map 8t on H^ x l/r) to be the map taking 0, (sfc)g°) G H^ x Z/r) to 0t(z,(s*)S°) := (etz,(snt_+k-t)%>) e'HxL, where n<_ = inf {n ^ 1 : sn ^ £}. For t G R, define the shift-map 8t on i?18 to be the map taking z = {zs)seR G ER to Otz := (zt+s)se[0>oo) G £[°'°°) and note that although z = (zs)ses. G £* is two-sided, the shift 8t is one-sided.
424 Chapter 10. REGENERATION 8.2 Time-inhomogeneous Regeneration in [r, oo) The definition of time-inhomogeneous regeneration for a process with time set [r, oo) is analogous to the definition for a process with time set [0, oo). Here we shall only focus on the essentials needed for the asymptotics from- the-past. Let Z^ and S^ be as above and Z^ be shift-measurable. Call the family {Z^r\ S^), r G (—oo,0], time-inhomogeneous regenerative of type p{\) if p{\) is an ((R,B(R)),(H x L,H® £)) probability kernel and, for r G (-oo, 0], n ^ 0, and A G % ® C, P(esM(z^,s^)eA\(z^)s M sir),...,sP) 6l°"b" > (8.1) = p(A|5W)a.s. Call the family (Z^r\S^), r £ (—oo,0], time-inhomogeneous wide-sense regenerative of type p(-\-) if instead of (8.1) it holds only that P(0sir)(ZW,SW) G A\S£\...,SP) =p(A\SP) a.s. (8.2) In both cases S^r' is a discrete -time strictly increasing Markov process with state space ([r, oo), B([r, oo))). Let be the recurrence times and let Fs, s G [r, oo), be the recurrence distributions: for all r G (—oo,0] and n ^ 1, FS(A) = P(lW G ^|5n_! =s), se [r, oo), A G S([r, oo)). For examples of processes that are time-inhomogeneous regenerative in [r, oo), replace [0, oo) by [r, oo) in Section 5.3. 8.3 Total Variation Convergence From-the-Past We are now ready to establish the asymptotics from-the-past under the familiar conditions from Theorem 5.3. Theorem 8.1. Let (Z^r\ S^), r G (—oo,0], be a time-inhomogeneous wide-sense regenerative family of type p(-\-)- Let G^ be the distribution (r) of Sq — r and Fs, s G IK, the recurrence distributions. Firstly, suppose there is a distribution function G on [0, oo) such that for r G (—oo, 0], G(r)(x) ^G(x), a:e[0,oo). (8.3) Secondly, suppose either there is a subprobability density f on [0, oo) such that f f > 0 and, for s G IK, FS(B)> f f, BeB[0,co), (8.4) Jb
Section 8. Asymptotics From-the-Past 425 or there is a d > 0 and an aperiodic subprobability mass function f on {d, 2d, 3d,... } such that for s £ R, Fs({kd}) ^ f{kd),k ^ 1, and S and S' are dZ valued. (8.5) Thirdly, suppose there is a distribution function F on [0, oo) such that f xF(dx) < oo and, for s £ E, Fs(x)^F(x), x£[0,oo). (8.6) Then there exists, for each t £ R, a stochastic process /?(*•') = (Z« )sz[t,oo) such that QtZ(r) 4 6tZ{*'V asri -oo, when (8.4) holds, (8.7a) 6tZ{md) 4 ^Z^'*) asm| -oo, when (8.5) ftoirfs, (8.76) and the distribution of Z^*'^ is determined by the typep(-\-). Proof. Take r' <r < £A0. In the lattice case [(8.5)] take further r, r' £ dZ. We are going to apply Theorem (7.1) to 0r(Z(r),S<r)) and 6>r(Z(r'>,S(r')). Note that 6>r(Z(r),S(r>) and 6»r(Z(r'),S(r')) are both time-inhomogeneous wide-sense regenerative of type p(-\r + •)• The recurrence distributions are Fr+S, s £ [0,oo). Due to (8.4), (8.5), and (8.6), the conditions (7.4), (7.5), and (7.6) in Theorem 7.1 are satisfied. In order to establish also the condition (7.3) in Theorem 7.1, note first that the delay length of 6>r(Z(r'),S(r')) is Br_ = the residual life immediately before time r. Now, 6ri (Z(r ), S(r )) is time-inhomogeneous wide-sense regenerative of type p(-\r' + •) with recurrence distributions Frt+S, s £ [0, oo), satisfying [again due to (8.4), (8.5), and (8.6)] the conditions (7.4), (7.5), and (7.6) in Theorem 7.1. Thus the result at (7.8) yields the existence of a finite constant ao determined by / and F such that for x £ [0, oo), sup Y{B(r0 - 1 > x\S{0r,) = s) ^ o0(l - Gp(x)). (8.8) sE.[r' ,r) On the event {S{0r'] > r} we have B{/J = S^ - r < S^0 - r'. This and (8.8) yield the inequality in P{B£)>x) = P(f£!> > x,S^ * r) +E[p(f#:> > x\S^)l{sr<r}] ^P(S{0r,) -r' >x) +a0(l-GF(x-l)), x€ [0,oo).
426 Chapter 10. REGENERATION Define a distribution function G on [0, oo) by G(x) = l-(l-G(x) + a0(l-Gp(x-l))Al, zG[0,oc), (8.9) to obtain from this and (8.3) that P(B{rrJ <^x)^G(x), zetO.oo). (8.10) The delay-length of 6r(Z(r\S^) is S<r) - r, and since G(x) < G(x) for x G [0, oo), it follows from (8.3) that P(S{Qr) -r^x)^G(x), a:e[0,oo). Thus the distribution functions of the delay lengths of both 8r(Z^r\S^) and er{Z{r'\S^"i) satisfy the condition (7.3) in Theorem 7.1 with G replaced by G. Since 9r(Z^,S^) and 6r{Z(r'\Sl-r')) satisfy the conditions of Theorem 7.1, it follows from Theorem 7.1 that there exists a finite random variable T with distribution determined by G, f, and F and such that ||P(0tz(r> e •) - P(0tz{rl) e -)|| < P(r > t - r). For each £ > 0 there is an rE G (—oo, i] such that P(T > t — rE) ^ e, and thus ||P(0tZ<r> G A) -P(0tZ<r'> G A)|| <e, r' <r<r£. Due to Lemma 8.1 below (applied to the family of probability measures P(6tZ^ G 0, r G (-oo,0], with t fixed), the re is a probability measure /i(') on (H,H) such that P(94ZW 6 ■) 4/J'1', r|-oo. (8.11) Let W = (W/s)s6[0jOO) be a stochastic process with the distribution /xW and define £(*■') := (Jys-t)s6[t,oo)- Then 0* £<*•') = W, and therefore we have P(6tZ(-*'t) G 0 = A*(/), and (8.7a,b) follows from (8.11). In order to establish that the distribution of /?(*'') is determined by the type p(-|0, let (Z'W.S'W), r G (-oo,0], be another family of the type p(-|0 and having uniformly dominated delay lengths [that is, satisfying a condition like (8.3)]. Let £'(*•') denote the total variation limit of 9tZ'^ as r I -oo. Now 8r(Z(r\S^) and 0r(Z'(r),S'(r>) satisfy the conditions of Theorem 7.1, and it follows from Theorem 7.1 that [since t — r —¥ oo as r I — oo] \\P(6tZ^ G 0 -P{OtZ'{r) G Oil -> 0, r | -oo. It follows that the two limit processes Z(*'*) and £'(*-') must have the same distribution, that is, the distribution of £(*■') is determined by the type. □
Section 8. Asymptotics From-the-Past 427 The following result was used in the above proof. (It implies that the space of all probability measures on a given measurable space is complete with respect to total variation.) Lemma 8.1. Let Ht, t G [0, oo), be a family of probability measures on some measurable space (G,G). Suppose for each e > 0 there is a te such that WlJ-t — Mt'll ^ £' t' > t > te, {the nt are Cauchy convergent). Then there exists a probability measure \x on (G, G) such that Ht % n, Moo. (8-12) Proof. For each A e Q and e > 0, we have \fit(A)-fit'(A)\^e, f>t>te. (8.13) Thus, for each A € Q, there is a number n(A) G [0,1] such that Ht(A) -> n(A), t->oo. (8.14) Send t' —> oo in (8.13) to obtain that for each A G Q and e > 0, we have MA)-/*(A)|sSe, t>te. (8.15) Let n be the set function taking A G Q to fi(A) G [0,1]. Since nt(E) = 1, it follows from (8.15) that n(E) = 1. In order to establish that \x is additive, take disjoint A and B G Q to obtain fi(A UB)= lim m(A U B) [due to (8.14)] t—>-oo = lim (fJ>t(A) + fit(B)) [fit is additive] t—>-oo = lim fit(A) + lim nt(B) t—J-oo (—+oo = /x(>l) + /x(B) [due to (8.14)]. In order to establish that n is continuous at 0, let Ai,A2,--- G Q be a sequence of sets decreasing to 0 to obtain, with e > 0 and t > te, KAn) *S /ut(^n) + e [due to (8.15)] -> £ as n -> oo [/it is continuous at 0]. Thus lim sup^,^ fj.(An) ^ e for all e > 0, that is, lim^oo /u(^l„) = 0. Thus /U is an additive set function with fi(E) = 1 and continuous at 0, that is, H is a probability measure. Take the supremum in (8.15) over A G Q and multiply by 2 to obtain \\nt - y\\ ^ e for all e > 0 and t> tE, that is, (8.12) holds. □
428 Chapter 10. REGENERATION 8.4 Coupling From-the-Past We shall now show that the processes coming in from-the-past can be made to merge with the limit process (in the distributional sense, unless (E,£) is Polish and the paths right-continuous). Theorem 8.2. Suppose the conditions of Theorem 8.1 hold. Then, for each t G IK, there is a sequence of random times S^*'1' such that the family (Z^*'1', S^*'*)), t G IK, is time-inhomogeneous wide-sense regenerative of type p(-|-)- Moreover, for each t G IK, there exists a distributional exact coupling of 8tZ^ and 9tZ(*'V with finite times T^ andT^*^ such that T(t) g f := y0 + y, + Y2 + ■ ■ ■ + Y3M, where Y0 and Y\ are independent random variables with distribution function G defined at (8.9) and independent of the independent random variables M, F2,F3,... from Theorem 1.1. Finally, if {E,£) is Polish and the paths right-continuous, then there is a nondistributional exact coupling of 6tZW and 0t£(*•') with a finite time TW such that T^ ^ f. PROOF. Fix t G IK. Apply Theorem 8.1 to the family (Zir\ Bir))se[rtQo), r G (—oo,0], to obtain the existence of a process (Zs*' ,BS*' )se[t,oo) such that [with r = md when (8.5) holds] 9((^)>Bi:)).€M ^ Wi^.-B^.elt.oo), rl-oo. Let S(r'^ be the sequence of times s G [t,oo) such that B^_ = 0, and 5^*'^ the sequence of times s G [t, oo) such that B],*2 = 0, to obtain from this that [see (3.2) of Lemma 3.1 in Chapter 6] for n ^ 0, (^"(2(r).s(r)).(So''".->^")) lA {6^{Z^,S^%(S^\...,S^), ri-oo. Now, for r G (-oo, 0], n ^ 0, and A£W.®£, P(0sc,.o(Z<r>,S<r>) EAIS^,...^^) = p(A\S™) a.s., and thus [see the final statement of Lemma 5.3] the same holds for the limit, namely for n ^ 0 and A £1-L® C, P(0s(„„(Z(*''>,S(*-'>) G A\S^l\...,S^) = p(A\S^) a.s. Thus the family (£(*■'>,S**'')), t G E, is time-inhomo geneous wide-sense regenerative of type p(-|-)-
Section 8. Asymptotics From-the-Past 429 To establish the coupling claims, note first that due to (8.10) we have, for r G (-oo, 0], that P(5^r'*) -t^x)^ G(x), x G [0, oo). Sending r | -oo we see that the same holds for the limit, namely P(Sq — t < x) ^ G(x) for x G [0, oo). Since S^r't] - t is the delay-length of 0tZ« and S^*'0 - t is the delay-length of 6»tZ(*-'), and since both 8tZ^ and 8tZ(*^ are time- inhomogeneous wide-sense regenerative of type p(-\t + ■) with recurrence distributions Ft+S, s G [0, oo), it follows that the conditions (7.3) through (7.6) in Theorem 7.1 are satisfied with G replaced by G. This yields the distributional coupling claim. The nondistributional coupling claim now follows from Theorem 3.2 in Chapter 4. □ 8.5 Two-Sided Limit Process Theorem 8.1 yields a limit process Z^*'^ with time set [t, oo) for each t G IK. We shall now show that if (E, £) is Polish, then this family of processes can be obtained by restricting a single two-sided process Z* = (Z*)s^s_ to one-sided time sets [t, oo). Note that this two-sided process Z* need not be stationary. Theorem 8.3. Suppose the conditions of Theorem 8.1 hold. If (E,£) is Polish, then there exists a two-sided stochastic process Z* — (Z*)s6k, with path space (ER,£R), such that for each t G K, 9tZ^ % etZ* asrl -oo, when (8.4) holds, (8.16a) 6tZ(md) ^ QtZ* as m | _00] when (85) holds (8.16b) If further the paths of Z^r\ r G [t, oo), are right-continuous with left-hand limits, then so are the paths of Z*, and for each t £ I, there exists a nondistributional exact coupling of 8tZ^ and QtZ* with a finite coupling time TW such that T(t) ^ f, where f is from Theorem 8.2. Comment. If the paths are right-continuous with left-hand limits (that is, if the path set H consists of such paths), then we may take 8tZ^*'^ := 8tZ* for all t G IK. On the other hand, if the paths are not right-continuous with left-hand limits, then Z* has only path space (ER,£R). Thus 8tZ* has path space (£[o,°o);(r[o,oo)) and not the path space (h,%) of 6tZ* (and of the 8tZ^). However, since 8tZ* has the same distribution as OtZ^*'^ regarded as an H valued random element in (El0-00),ft0-00)), the total variation convergence at (8.16a) and (8.166) makes formal sense by interpreting -4 to mean that total variation convergence holds with the distribution of 8tZ* restricted to {H,H). Proof. Put ^E(-oo,t) x 4) := P(Z(.,t) eA^ teR, A€ £[t'co).
430 Chapter 10. REGENERATION Take t' < t (and r € dZ when (8.5) holds). Due to (8.7a) and (8.7b), we have [with r = md when (8.5) holds], as r 4- — oo, 0tZW £■ 0tZW = 6t-tl{6t,Z^) 4 et-v{9t>ZW>) = OtZW. Thus 9tZ^*'^ and 6tZ^*,t' have the same distribution, that is, P(Z(«.0 G A) = P(Z<*1*') G jE7[*''*> x A), t' <t,Ae £lt'oo). From this we see that the set function [i is well-defined on the subalgebra {£(-«>,*) x A : t € K, A € ft*'00} of £R. Due to the Kolmogorov extension theorem [see Fact 3.2 in Chapter 3], // extends uniquely to a probability measure on £R. Let Z* be a process with the distribution // to obtain (8.16a) and (8.166) from (8.7a) and (8.76). In order to obtain that the right continuity and left-hand limits transfer to the paths of Z*, note that for all m € Z the state space of the sequences ((^+^))s6[o,i])~oo is tne Polish space (DE[0, l],VE[0,1]), namely the set of all right-continuous mappings from [0, l] to E having left-hand limits, equipped with its Borel subsets. Thus the state space of the finite- distributional distributions of the sequence ((^+s)se[01])^°oo can be restricted to (De[0, 1],T>e[0, 1]). Apply the Kolmogorov extension theorem to obtain a sequence (Wi-)??^ with state space (De[0, 1],T>e[0, 1]) and having these finite-dimensional distributions. Remove the null event that there is a k such that the right endpoint of Wk does not agree with the left end- point of Wk+i- Now redefine Z* by putting (^+s)se[o,i) := Wk for each k € Z, to obtain a process with right-continuous paths having left-hand limits. The nondistributional coupling claim follows from Theorem 8.2, since (E, £) is Polish and the paths right-continuous. □ Remark 8.1. It follows from Theorem 8.2 that Z* is time-inhomogeneous wide-sense regenerative in the sense that the one-sided (Z*)s€[t)00) restricted to the path-space (H^\ri^) is so for all t € K. It can be shown [at least when the paths of Z* are right-continuous with left-hand limits; see Thorisson (1988)] that Z* is time-inhomogeneous wide-sense regenerative in a proper two-sided sense. 8.6 Moments of T — Convergence Rates — Uniform Convergence From Theorems 8.2 and 7.2 we obtain the following result. (The function class A is defined just before Theorem 7.2.) Theorem 8.4. Suppose the conditions of Theorem 8.1 hold. Then the convergence at (8.7a) and (8.76) holds uniformly over families {Z^T\S^), r € (—oo,0], satisfying (8.3) through (8.6) with G, f, and F fixed. If G and F have finite geometric moments, then so has T, and the uniform convergence is of geometric order. If ip £ A and G has finite ip moment and
Section 8. Asymptotics From-the-Past 431 F has finite # moment, then T has finite <p moment, and the uniform convergence is of order <p and of moment order tp. If (E, £) is Polish, then the same holds for the convergence at (8.16a) and (8.166). Proof. Take r < tAO (with r G dl if (8.5) holds). From Theorem 8.2 and the coupling time inequality (Theorem 6.1 in Chapter 4) we obtain ||P(f9tZ(r> G ■) - P(6itZ(*'t) G -)IK 2P(T > t - r). Let sup* denote the supremum over families (Z^r\S^), r Gj-oo,0], satisfying (8.3) through (8.6) to obtain [since the distribution of T is determined by G, f, and F] sup* ||P(0tZ<r> G ■) - P^t^*'0 G OIK 2P(T > t - r). (8.17) This yields the uniformity of the convergence at (8.7a) and (8.76). The moment claims for T follow from Theorem 7.2 if we can establish the same claims for the distribution function G defined at (8.9). From (8.9) we obtain l-G{x)^(l-G(x)) + a0{l-Gp(x-l)), zG[0,oo). (8.18) Suppose G and F have finite geometric moments. Since F has a finite geometric moment, so [see (7.13a)] has Gp, and thus [see (7.13a)] there is a p > 1 such that J™ py(l-Gp(y)) dy < oo. Since J™ px(l-Gp{x-l)) dx = /»Jo°° pv(1 - gf(v)) dv,this yields pxa0{l-Gp{x-l))dx < oo. (8.19) Since G has a finite geometric moment, we can [see (7.13a)] take p close enough to 1 for J0°°px(l — G(x)) dx to be finite. This, together with (8.19) and (8.18), yields J0°°/5X(1 - G{x)) dx < oo, and thus [see (7.13a)] G has a finite geometric moment. Thus [due to Theorem 7.2] so has T, and a reference to Section 6 in Chapter 4 yields that the uniform convergence is of geometric order. Take <p G A and suppose G has finite <p moment and F has finite # moment. Then [see (7.136)] Gp has finite <p moment, and thus Gp(- - 1) has finite ip(- — 1) moment. Since [see Lemma 7.2(a)] ip ^ tp(l)tp(- — 1), this means that Gp(- — 1) has finite ip moment, and thus [see (7.136)] /•OO / (p{x)a0{l-Gp(x-l))dx < oo. (8.20) Since G has finite <p moment, we have [see (7.136)] f^°ip(x)(l — G(x))dx <oo. This, together with (8.20) and (8.18), yields /0°°<p(i)(l - G{x))dx < oo,
432 Chapter 10. REGENERATION and thus [see (7.136)] G has finite ip moment. Thus [due to Theorem 7.2] so has T, and a reference to Section 6 in Chapter 4 yields that the uniform convergence is of order <p and of moment order <p. The final claim of the theorem follows from the fact that OtZ* [restricted to (H,U)\ has the same distribution as Z^^ for t € R □ 8.7 Time-inhomogeneous Regeneration in [r, 0] The limit result in Theorem 8.1 should only depend on the behaviour of the regeneration times in the far past. It is reasonable to expect that it holds for processes regenerating only up to some fixed time. Without loss of generality we can take this time to be zero. The following modification of the framework in Section 2 and in Section 8.1 is needed for this purpose. For r G (-oo,0], let S<r) = (s£r))§° be a one-sided nondecreasing sequence of [r, oo] valued random times such that for each n ^ 0, r ^ S<r) < S[r) < ■ ■ ■ < s£r> on {Stf < oo}. Thus 5<r) is strictly increasing as long as it stays in [r, oo), and is absorbed in oo when leaving [r, oo). Regard S^ as a measurable mapping from (£l,!F) to the sequence space (Loo ,£«?), where LQ = {(st)o° € [r,oo]{0,1'-} : sk-i < sk < co or sk = sk+i = co}, £(£) = £,£) n B[r, oo]*0'1-* (the Borel subsets of L<r)). Put (Loo,-Coo) == (L&\C(£>). Note that L& = r + L^. For t € [r, co), define the joint shift-map 9t on H^ x Loo to be the map taking (z, (sfc)g°) g H^ x L&> to 0t(z, (st)S°) := (dtz, (snt_+k - t)8°) G if x Loo, where nt_ = inf{fc > 0 : sk > £}. For t = oo, define #t(z,(sfc)§°) := Zi, where Zi is an external nonrandom constant (see Section 2.9 in Chapter 4). Let Z^ = (Zir')se[roo) be as in Section 8.1 and let it be shift-measurable. Call the family [Z^r\S^), r £ (—oo,0], time-inhomogeneous regenerative up to time zero and of type p(-\-) if p(-|-) is a (((roo,0],B(-oo,0]),(HxL00,'H(g>C00)) probability kernel
Section 8. Asymptotics From-the-Past 433 and, for r G (-00,0], n > 0, and A £ H^Coo, the following holds a.s. on {S(nr) < 0}: P(0sW (ZW, 5«) G AK^)) w S<r),..., sP) e[ ' " ' (8.21) = p(A|5ir)) a.s. on {S^ < 0}. Call the family (Z^r\ S(r)), r G (—oo,0], time-inhomogeneous wide-sense regenerative up to time zero and of type p(-\-) if instead of (8.21) it holds only that P(V)(^(r)>5(r))eA|5(r),...,5('-)) bn (8.22) = P(A\S^) a.s. on {S^ < 0}. In both cases S^ is a strictly increasing discrete-time Markov process as long as it stays in [r, 0]. For r G ( — 00, 0] and n > 1, let y(r) = )Dn Dn-l U Dn-l ^ °°i \oo ifS^1=oo, be the recurrence times and let Fs be the recurrence distribution at s, that is, for all r G (—00,0] and n > 1, FS(A) = P(X(r> G A|5n_! =s), s G [r,0],' A € B([r, 00)). 8.8 Asymptotics From-the-Past in the [r, 0] Case We shall use Theorems 8.1 through 8.4 to establish the following generalization. This result will find an application in the next section. Theorem 8.5. Let (Z^r\S^), r G (—00, 0], be time-inhomogeneous wide- sense regenerative up to time zero of type p{-\-). Let G^ be the distribution function of Sq — r and Fs, s € ( —oo,0], be the recurrence distributions. Firstly, suppose there is a distribution function G on [0, 00) such that for re (-oo,0], G^{x) > G(x), x G [0, -r). (8.23) Secondly, suppose that either there is a c > 0 and a subprobability density f on [0, c] such that J f > 0 and, for s G ( — 00, —c], FS{B)> [ f, BGB[0,oo), (8.24) Jb or there is an integer m > 0, a d > 0, and an aperiodic subprobability mass function f on {d, 2d,..., md} such that for s G (—00, —md], Fs({kd}) ^ f{kd),l ^ k ^ m, and S and S' are d!L valued. (8.25)
434 Chapter 10. REGENERATION Thirdly, suppose there is a distribution function F on [0, oo) such that J xF(dx) < oo and, for s € ( — oo,0], F,{x)^F(x), i€[0,-j). (8.26) Then there exists, for each t € M., a stochastic process .£(**') = [Zl*' )se[t,oo) that is time-inhomogeneous wide-sense regenerative up to time zero, of type p(-|-), with distribution determined by the type, and such that 0tZ(r) % OtZ^M asrl -oo, when (8.24) holds, (8.27a) fftZ(md) tv fftZ(*,t) asm± _00) when (8.25) holds. (8.276) For each t € (—oo, —c], there are (not necessarily finite) distributional exact coupling times jW and TW) for 6tZ^ and GtZ(*V: (9noZ(t),r(J)) = (^T(.,()^(*'t),T(*'t)), (8.28a) such that T^ A (-t) ^ f, where f is as in Theorem 8.2. (8.286) If (E, £) is Polish and the paths right-continuous, then there exists a nondis- tributional exact coupling ofOtZ^ andOtZ^*'1^ with (not necessarily finite) coupling time T^> satisfying (8.286). Moreover, if (E,£) is Polish, then there exists a two-sided stochastic process Z* = (Z*)s6k, with path space (ER,£R), such that for each t € M., OtZ(r) % OtZ* asr i -oo, when (8.24) holds, (8.29a) QtZ(md) *« QtZ. asm^ _QQ^ when (8 25) holds {s,.29b) If the paths of Z^r\ r € [t, oo) are right-continuous with left-hand limits, then so are the paths of Z*, and for each t € R, there exists a nondis- tributional exact coupling of 9tZ^ and 6tZ* with (not necessarily finite) coupling time T^ satisfying (8.286). Finally, the convergence at (8.27a) and (8.276), and at (8.28a) and (8.286), holds uniformly over families (Z^r\S^), r € (-oo,0], satisfying (8.23) through (8.26) with G, f, and F fixed. If G and F have finite geometric moments, then so has T, and the uniform convergence is of geometric order. If ip € A and G has finite <p moment and F has finite $ moment, then T has finite ip moment, and the uniform convergence is of order <p and of moment order ip. Comment. The comment to Theorem 8.3 also applies here.
Section 8. Asymptotics From-the-Past 435 Proof. Put c = md when (8.5) holds. For r € (-oo,-c], let G^ be the distribution function of (5g — r) A (—r), that is, G<r)(x) = G<r)(x) for x G [0, -r) and G<r)(x) = 1 for x € [-r, oo). For s € ( —oo,—c], let Fs be the conditional distribution function of the random variable X„ A (—s) given 5„_i = s, that is, Fs(x) = Fs(x) for x € [0, -s) and Fs(x) = 1 for x € [-s,oo). For s £ [—c, oo) put F = F ± s — c — c- Note that due to (8.23) through (8.26), the conditions (8.3) through (8.6) in Theorem 8.1 are satisfied with G^ and Fs, s € [r, oo), replaced by G(r) and Fs, s 6 [r, oo). We shall apply Theorem 8.1 to the time-inhomogeneous wide-sense regenerative family (W^r\R^), r € (—oo,0], defined as follows. Let A be some nonrandom constant and V\, V2, ■ ■ ■ i-i.d. random variables with the distribution F_c and independent of the family {Z^r\ S^), r € (—00,0]. For each r € (-00,0], put W^r) := 6sZ^r) for s 6 [r, -c] and W^r) := A for s € (-c, 00) and (with 5^ := -00), for k > 0, ** (r) ._ S^AO, if5tU-c<5W, Ij£>1 + v4> if-c<si^. Then i?(r) is Markovian with state space ([r, 00), Z?[r, 00)), increases strictly to 00, and has recurrence distributions Fs, s € [r, 00); and Rq — r has the distribution function &rh Since the family (W^r\R^r>>), r £ (—00, 0], satisfies the conditions of Theorem 8.1, it follows from Theorem 8.1 that for each t€M, there exists a stochastic process py(*>') = (Wj*' )«e[t)0O) such that 0tWW % 0twW as r 4. -00, when (8.24) holds, (8.30a) QtW(md) £3. 0tW{.,t) as m 1 _00) when (8_25j holds_ (8.306) Forr€ (-00,-c], define Z(*-r) by 6lrZ(*-r):=Pyr(*'r) and recall that Pyr(r): = 6>rZ(r). Thus [see (3.2) of Lemma 3.1 in Chapter 6] (8.30a) and (8.306) yield that (8.27a) and (8.276) hold for t € (-00,-c]. For t £ (-c, 00),
436 Chapter 10. REGENERATION define £(*•') by OtZ(*<V := BtZ^*~^ to obtain (8.27a) and (8.276) from the observation that when t € (—c, oo), the left-hand side, 8tZ^r\ is obtained from d-cZ^ by the same shift as the right-hand side, dtZ(*^\ from 6»_CZ(*'_C) [see (3.2) of Lemma 3.1 in Chapter 6]. In order to obtain the regeneration claim, proceed as in the first part of the proof of Theorem 8.2 to obtain the existence of a sequence of random times S^*'') such that for n > 0 and A £ 7i <g> £oo, the following holds a.s. on {SiM) < 0}: p(0s<..„(;?<*•'>,st*-')) g a\s^\...,s^) =P(A\s^% In order to obtain the distributional coupling claim, apply Theorem 8.2 to the family (W^r\R^), r € (—oo,0], to obtain, for each t € (—oo,— c], finite random times t'1' and r'*'*' such that (JrWW(Vi)) = (»r(.,#(,'VM) and r^^f. (8.31) Define TW :=t<') if rW ^ -t-c and T^ := oo if r<f» > -t - c, r(.,t).= r(*,t) if r(*,t)^_^_c and T(*.«):=00ifT(*.*)> _i_C; and recall that Wt(t) = dtZ^ and W^*'^ = 6»tZ<*>') for t € (-oo,-c] to obtain from (8.31) that (8.28a) and (8.286) hold. The nondistributional coupling claim now follows from Theorem 3.2 in Chapter 4. If (E,£) is Polish, repeat the proof of Theorem 8.3 to obtain from (8.27a) and (8.276) that (8.28a) and (8.286) hold for a two-sided Z*, and that right continuity and left-hand limits of the paths transfer to Z*. The nondistributional coupling claim follows from the fact that (E, £) is Polish and the paths right-continuous. Theorem 8.4 yields the moment results for T and the rate and uniformity results for the convergence at (8.30a) and (8.306). This yields the rate and uniformity results for the convergence at (8.27a) and (8.276) [since the left- and right-hand sides at (8.27a) and (8.276) are measurable mappings of the left- and right-hand sides at (8.30a) and (8.306)]. The rate and uniformity results for the convergence at (8.28a) and (8.286) follows immediately from this. □ 9 Taboo Regeneration Suppose we are studying a fish population that has lived a long time in an isolated lake. This fish population will eventually become extinct, but suppose it is still there at the time of observation. Then it is not appropriate to use asymptotic stationarity to motivate a stationary process as a
Section 9. Taboo Regeneration 437 model for the present state of the population. We should rather consider the asymptotic behaviour of the population under an extinction taboo, that is, conditionally on the observed fact that the population is still nonextinct at the time of observation. We should look for a taboo limit. In this section we shall introduce taboo regenerative processes, processes that regenerate as long as some specific event (like extinction) has not occurred. This is the generalization of regeneration appropriate for obtaining a taboo limit. A key ingredient in our analyses will be the fact (Theorem 9.1 below) that the taboo conditioning turns taboo regeneration into time-inhomogeneous regeneration up-to-time-zero (up to the observation time). Therefore the asymptotics from-the-past in the previous section apply to yield a taboo limit (Theorem 9.4 below). 9.1 Preliminaries For taboo purposes we need to modify the framework in Section 2 by allowing the sequence of times S to be terminating (to be absorbed at infinity). Let S = (5fc)§° be a nondecreasing sequence of random times that is strictly increasing as long as it is finite, that is, for each n > 0, 0 ^ So < Si < • ■ • < S„ on {Sn < oo}. Regard S as a measurable mapping from (ft, J7) to the sequence space (Loo,Coo), where (with s_i = -oo) £<x> = {(sfc)o° G[0, oo]*0'1'■■■}:sfc_i <sk<cooisk = sk+i=cc}, -Coo = £<x> n B[0, oo]*0'1'-* (the Borel subsets of L^). Let Z = (Zs)se[o,oo) be as in Section 2 and let r be a finite nonnegative random time. Let (f2, T, P) be the probability space supporting (Z,S,F) and assume that P(r > Sn) > 0, n > 0. The triple (Z, S, F) is a measurable mapping from the measurable space (f>, T) to (HxLooXp, oo), n^Cav^BlO, oo)). As in Section 2, for t e [0, oo), let 9t be the shift-map from H to H 9tz := (zt+s)se[o,oo) and also the joint shift-map from H x L^ toflx L^: et{z,{sk)™):={dtz,{snt_+k-t)™), where nt- — inf{n ^ 1 : sn ^ t}.
438 Chapter 10. REGENERATION Further, for t € [0, oo), let 9t be the joint shift-map from H x Loo x [0, oo) toflx Lqo x [0, oo) defined as follows: 6t(z, (8k)?,x) := (6t(z, (St)g°), (x - t)+). In order to be able to shift with t replaced by a [0, oo] valued random time, let A be a fixed nonrandom constant (the cemetery; see Section 2.9 in Chapter 4) and define floo«:=floo(«,(st)o0):=floo(«,(*t)o°,a:) == (4).€[o,oo)- The random times Sn split Z into a delay D := (zs)se{o,s0) and a (this time possibly terminating) sequence of cycles: for n ^ 1, C„ := (^s„_i+*)s€[o,x„) on {S„_i < oo}, where Xn are the cycle-lengths Xn := S„ - 5n_i on {5n_i < oo}. In order to have nonterminating sequences of cycles and cycle-lengths put, for n > 1, Cn := (zi)se[0,oo) and Xn := oo on {5n_i = oo}. Put (Z°,S°,F°):=eSo(Z,S,F) and regard (Z°,S°,F°) as supported by the probability space (Q, T, P°), where P°:=P(-|r>50). 9.2 Taboo Regeneration — Definition Call a one-sided shift-measurable stochastic process Z taboo regenerative with regeneration times S and taboo time F if for all n ^ 0, P(6Sn(Z,S,F) £-\F > S„) = P0((Z°,S°,r°) G •) (9.1) and 6sn (Z, S, F), given the event {F > 5n}, is conditionally independent of ((■Zs)se[o,s„), So, • ■ • ,Sn). These two conditions can be written as a single condition: P(9sn(Z,S,r)e-\(Zs)se[0tsn),So,...,Sn;F^Sn) = P°((Z°,S°,r°) €■), n>0.
Section 9. Taboo Regeneration 439 Call the triple (Z,S,F) taboo regenerative if this holds. This definition can be reformulated as follows: (Z,S,r) is taboo regenerative if and only if for each n ^ 1, given {r>Sn}, £>,<?!,...,<?„_!,0s„(Z,S,r) are conditionally independent and C\,..., Cn-\ are i.i.d. We shall refer to the conditioning on the events {r > Sn} by saying under taboo. Thus taboo regeneration means, loosely speaking, that under taboo the future is independent of the past, and the past cycles are i.i.d. Call a triple (Z',S',r') a version of a taboo regenerative (Z,S,r) if (Z',S',r') is also taboo regenerative and p(0Sj(W,r') e -\r' > si) = p°((z°,s°,r°) e •)• (9-3) In particular, (Z°,S°,r°) under P° is a zero-delayed version of a taboo regenerative (Z,S,T). 9.3 Taboo Wide-Sense Regeneration — Definition Call a one-sided shift-measurable stochastic process Z taboo wide-sense regenerative with regeneration times S and taboo time F if instead of (9.2) we have only: p(esn(z,s,r)e-\s0,...,sn;r^sn) 9.4 = P0((Z°,50,r°) €•)> n>0. Call the triple (Z,S,F) taboo wide-sense regenerative if this holds. Taboo wide-sense regeneration differs from taboo regeneration in that under the taboo {F > Sn}, the future 9s„{Z,S,F) is no longer independent of the full past but only of the past regeneration times (S0, ■ ■ ■ ,Sn). However, (9.1) still holds. Call a triple (Z',S',F') a version of a taboo wide-sense regenerative (Z, 5, F) if (Z\ S', r") is also taboo wide-sense regenerative and (9.3) holds. In particular, (Z°,S°,r°) under P° is a zero-delayed version of a taboo wide-sense regenerative (Z,S,r). Taboo lag-l regeneration is defined analogously (see Section 4.1), but we shall refrain from discussing the lag-/ case here. The discussion is sufficiently inflated without it. 9.4 Examples Suppose (Z, S) is classical regenerative (Section 3) and J1 is a first exit time, r = mi{t > 0 : Zt & B} (for some specific B € £)
440 Chapter 10. REGENERATION that is finite, measurable, and such that P(.T > 5n) > 0 for n > 0. Then (Z, S, r) is taboo regenerative. On the other hand, if (Z,S) is only wide-sense regenerative (Section 4) and r is a finite measurable first exit time such that P(.T ^ 5n) > 0 for n > 0, then [Z, S, r) need not be taboo wide-sense regenerative. The dependence between future and past at the times of regeneration may, under the taboo, destroy the independence between the future and the past regeneration times, since the taboo event {r ^ 5n} carries information about the past. For an example of a process that is not classical regenerative but taboo regenerative, consider a transient Markov chain Z. Let B be a transient set of states that is irreducible, that is, the chain can go from any state i € B to any other state j € B through a sequence of states in B. If r is the first exit time from B, and 5 the times of successive entrances into a fixed reference state j € £?, then (Z, S, r) is taboo regenerative. For an example of a process that is not wide-sense regenerative but taboo wide-sense regenerative, modify Section 4.5 as follows. Let Z be a strong Markov process and r a measurable first exit time from a transient set of states B. Say that a subset A of B is a taboo regeneration set if A can be revisited without leaving B, if the first exit time from B is measurable, and if (4.13) holds with fi(B) — 1. Note that a taboo regeneration set A is not a regeneration set, since a part of our definition of a regeneration set was that A be recurrent, but A is a subset of the transient set B and thus is transient itself. An argument analogous to the one in Section 4.5 yields a sequence of taboo lag-/ regeneration times for Z. Finally, consider the GI/GI/k queueing system (see Sections 3.2 and 4.2) in the transient case, that is, with the mean inter-arrival time less than the mean service time divided by k. Let r be the first time that the queue length exceeds some fixed level. If the system can empty, then the successive entrances to an idle system form taboo regeneration times for the queue length and the ordered remaining service times. If the system cannot, empty then a modification of the argument in Section 4.3 yields taboo wide-sense regeneration times for these processes. 9.5 Time-Inhomogeneous Regeneration Under Taboo in [0, t] Consider a Markov chain Z forbidden to leave an irreducible finite set of states B up to time t. If the chain is further conditioned on visiting a fixed reference state j € B at a time s where s ^ t, then its behaviour from time s onward is, firstly, independent of its past before time s and, secondly, like the behaviour of a chain starting in state j and forbidden to leave B in a time interval of length t — s. That is, Z regenerates at time s but the regeneration is time-inhomogeneous; the future after time s depends on s. This shows that under taboo in [0,t], the Markov chain Z is time- inhomogeneous regenerative in [0, t]. Now note that the distribution of the
Section 9. Taboo Regeneration 441 future after regeneration at time s does not only depend on s but also on t. However, the dependence on t and s is only through t — s, the length of period from regeneration to the end of the taboo interval. Thus if we instead of starting Z at time 0 start it at the time —t and forbid it to leave B up to time 0, then the distribution of the future after regeneration at a time s ^ 0 depends only on s and not on t. The chain becomes time-inhomogeneous regenerative in [—1,0], and the type is the same for all t. The above example suggests that if Z is taboo regenerative and if we start Z at time —t instead of at time 0, then taboo in [—1,0] yields a process that is time-inhomogeneous regenerative up to time zero and of a type that does not depend on t. We shall now show that this is indeed the case. (This result makes Theorem 8.5 available to establish taboo limits in Theorem 9.4 below.) Theorem 9.1. For each t € [0, oo), let (Z^~l\ S(~'*) be a pair with distribution P((Z(-'),s(-t)) g •) := P(((Z,+t).e[-t,oo), (S* - t)o°) e -|^ > t) and define a probability kernelp(-\-) by p(A\s) :=P°((Z°,S°) € A\r° > -s), A£-H®Coo, s £ (-oo,0]. If (Z, S,T) is taboo regenerative, then the family (Z^r\S^), r € (—oo,0], is time-inhomogeneous regenerative up to time zero and of type p(-\-). If (Z,S,T) is taboo wide-sense regenerative, then (Z^r\S^), r€( —oo,0], is time-inhomogeneous wide-sense regenerative up to time zero and of type P(-l-)- Proof. Consider first the case when (Z, S, r) is taboo wide-sense regenerative. Fix arbitrary t € [0, oo), x £ [0,t], and n > 0. Apply (9.4) to obtain P(9Sn(Z,S)£-,r-Sn>t-x\So,...,Sn;r> Sn) = p°((z°,s°) £-,r° >t-x). But [use Fact 3.1 in Chapter 6] p{9Sn {z,s)e-,r-sn>t- x\{So,..., s„_i) = -, s„ = x- r > sn) = P{eSn(z,S)e-,r>t\(s0,...,sn-1) = ;Sn = x;r^sn), and thus P{esjz,s)e;r>t\(s0,...,sn.1) = ;Sn = x;r^sn) (9.5) = p°((z°,s°) £-,r°>t-i).
442 Chapter 10. REGENERATION In particular, p(r > t\(s0,.. .,s„_i) = -,sn = x-r > sn) = p°(r° >t-x). Divide (9.5) by this to obtain P(9Sn (Z, S) G • |(S0) • • •, S„_i) = •, S„ = i, T > 0 = P°((Z°,5°) G-|r°> t-x). Thus, with s = x — t, p(0sJ-„ (£<-*>, s<-*>) G-Ksr0,...^^) - -^i-r) = *) =p(-W- Since s £ [—t,0] is arbitrary, this means that the pair (Z^~l\ S'-')) is time-inhomogeneous wide-sense regenerative up to time zero and of type P(-l-)- When the triple (Z,S,T) is taboo regenerative, replace (5o, ■. ■ ,5„_i) by ((Zs)se[o,s„),5o,... ,Sn-i) in the above argument to obtain the desired result. □ 9.6 Change of Measure - Exponential Biasing We shall now make an exponential change of measure that is hard to motivate intuitively. It is, however, motivated (mathematically at least) by its use in the proof of Theorem 9.2 below and will be further motivated by its use in the next section. It turns out to be the taboo counterpart of the length-biasing of cycle-stationary processes in Chapter 8. Note that if (Z,S,T) is taboo regenerative (in the wide-sense or not), then P((Xn+i, Xn+2, ■ ■ ■) 6 -|5o,... ,Sn;r ^ Sn) = P°((X1,X2,...)€-), n>0. If this holds, call 5 a taboo renewal process with taboo time T and the pair (5, -T) taboo regenerative. Under the taboo {J1 > 5n}, the recurrence times Xi,..., Xn_i are i.i.d. and independent of the delay length So. Make the following basic assumption: There is an a > 0 such that E°[eaXl l{r°^Xi}] = *! and define a probability measure P£aboo on (fi, !F) by dPtaboo == eaXl l{r^xl}dP°. (9-6) Further, note that for n ^ 0, ear l{w<s„+l} = ( eaS° l{Os0>) ( eaXl l{rs0>xl}) • • • ( e°*-i l{r-s„_lW) ( ^{r-s-l) l{r-s.<x.+l}) •
Section 9. Taboo Regeneration 443 Take conditional expectations E[-|So,..., Sn; -T > Sn],..., E[-|So; -T > So] and E[-] recursively and apply taboo regeneration to obtain E[ea l{Sn^r<s„+1}] = E[eaS° l{S0^r}]Eo[eaXl l{r^xl}]nE°[ear° l{r-<xl}]- Since E°[eaXl l{r°^xa}] = 1. this yields E[ear l{w<s„+l}] = E[eaS° l{W}]E°[ear° l{r.<Xj}]. Hence if we assume that E[eaS° l{Os0}] < oo and E°[ear° l{r°<xl}] < oo, then we can define probability measures Po,Pi,... on (fi,.F) by dP„ := ^r g 6 {^^<5;.+l} rdP, n > 0. (9.8) E[e«s°l{OM]E°[e^°l{ro<Xl}] " ^ ; The following lemma is the key to the proof of the next theorem. Lemma 9.1. Let R = (i?fc)o° be a renewal process with recurrence times Yn = Rn — Rn-i having distribution P(Yn G •) = Pt°aboo(*i G ■), n > 1, and de/aj/ time Ro having distribution P(R0 G •) = P0(r G •)• TTien P(i?nG-) = Pn(rG-), n^O. (9.9) Proof. Fix n > 0 and let /0, /i,..., fn,g € B be bounded. Due to (9.7) and taboo regeneration [take conditional expectations recursively], we have E[/o(50)/i(X1)... fn(Xn)g(r - Sn) ear l{sn<r<s„+1}] = E[/o(S0) eaS° l{r^0}]E°[/i(X1) eaXl l{r^xl}] ■ • • E°[/n(X0 eaX> l{r^Xl}]E°[g(n ear° l{r.<Xl}]. The special case n = 0 yields that the product of the first term on the left and last term on the left equals E[f0(S0)g{r - So) ear \{s0^.r<Si}\- Thus dividing by E[eaS° l{r^So}}E°[ear° l{r°<Xi}] on both sides yields (due to the definition of Pn, P0, and Ptaboo) E„[/o(50)/1(X1)... fn(Xn)g(r - Sn)} = E0[f0(S0)g(r - S0)]E°aboo[/i(*i)] • • ■^boo[fn(Xl)}.
444 Chapter 10. REGENERATION Thus under Pn, Xi,.. .,Xn arei.i.d. and independent of (5o,r"—5n), (9.10a) Pn(Xk G •) = Pt°aboo(^i G ■) = P(Yn G ■). 1 < * < ». (9-106) Pn((S0,r-S„)G-) = Po((5o,r-So)G-)- (9-lOc) From (9.10c) we obtain Pn(50 + r - Sn G •) = P0(r G •) = p(#o G •)• Since r = {s0 + r-sn) + x1 + --- + xn, this, together with (9.10a) and (9.106), yields (9.9). □ 9.7 Exponential Taboo Asymptotics for r It is natural to start the study of taboo asymptotics by considering the taboo time r itself. Theorem 9.2. Let (S,T) be taboo regenerative. Suppose there is an a > 0 such that E,°[eaXl l{r°>Xi}] = 1; (9.11a) E[eaSo l{r>So}] < oo, (9-116) E°[XX eaX> l{r^Xl}] < oo, (9.11c) e°" P(S0 > r >t)-+ 0 as t-+ oo, (9.lid) E°[ear° l{r°<xl}] < oo. (9.11e) If P°(Xi G -\r° ^ Xi) is nonlattice, then as t —> oo, IfP°(Xi £ -\r° ^ X\) is lattice with span d and P°(50GdZ)=l and P°(r° G dZ\r° < X,) = 1, £/ien as n —> oo, e^P(r > nd) -* E[e«s° l{r>So}], /f °iC ^xT^ r V ' L {°6o}J(e«d-l)E°[Xie«^l{ro>Xl}] Comment. In the nonlattice case the theorem implies that conditionally on r > t, the remaining taboo time r — Ms asymptotically exponential with parameter a, namely, for x G [0, oo), P(r-t > x\r > t) ->• e~ax, t^oo.
Section 9. Taboo Regeneration 445 In the lattice case the theorem implies that conditionally on T > nd, the random variable (r — nd)/d is asymptotically geometric with parameter p = 1 — e~ad, namely, for k > 1, P((r -nd)/d> k\r>nd) -* (1 - p)k, n-> oo. Proof. Consider first the nonlattice case. Put C = E[eaSM{OSo}]E°[ear°l{ro<Xl}]. Let R, Y\, Yii ■ ■ ■ be as in Lemma 9.1 and note that E[Vi] = E^^JXi] = E[X, eaX> l{r->Xl}]- (9-12) Due to this and (9.lid), we must prove that eatP(r>t,r>5o)^^r|, t^cx>. (9.13) In order to establish (9.13), note that eat P(r > t, r > So) = E[eat l{r>t}l{So^r}] OO = XI E[eat 1{/,>01{Sn^r<5„+1})] n=0 oo = c^En[e-Q(r-*)l{r>t})] [by (9.8)] n=0 = c£E[e-°(fl--t)l{fln>t})] [by (9.9)]. n=0 Let Mt be the first n such that Rn > t and let Vt = Rm, — t be the residual life at time t. Then eat P(r > t, r £ So) = cE[£ e"0^--') l{fl„M}] n=0 OO = cE[^ e-Q(Vt + YMt+1 + ■■■ + YMt+kj\ Jt-O oo = c^2 Wa{Vt + YMt+l +■■■+ YMt+k)\ k=0 oo = c^E[e-°v<]E[e-°y Mt+ !}■■■ E[e~aY Mt + k] k-0 cE[e-aV«]^E[e-Q k=0
446 Chapter 10. REGENERATION Since E[e~aFl] < 1, this yields eat P(r > t, r > So) = cE[e-aV']/(l - E[e~aYl]). (9.14) Since P°(Xi € -\r° > Xx) is nonlattice, so is Pt°aboo(Xi 6 •), that is, Y\ is nonlattice. Due to (9.12) and (9.11c) we have E[Yi] < oo. Due to Theorem 10.1 in Chapter 2, the residual life Vt tends in distribution to a continuous random variable W with density P(Yi > x)/E[Yi], x > 0. Thus /•OO lim E[e-aVt] = E[e~aW] = / e~ax P(Yi > x) dx/E[Yi] t->oo _/0 /•OO = E[ / exp-ox l{y1>x} dz]/E[Yi] •/o (9.15) = E[ / e-ax di]/E[yi] Jo = (l-E[e-oy'])/(aE[r1]). This together with (9.14) yields (9.13), and the theorem is established in the nonlattice case. In the lattice case carry out the above argument with t = nd and with the following modification. Use Theorem 10.2 in Chapter 10 (instead of Theorem 10.1 in that chapter) to obtain a lattice-valued limit variable W with probability mass function dP(Yi > fcd)/E[Yi], k > 1. In (9.15) replace the integral by a sum and the density by this probability mass function to obtain lim E[e-aVt] = d(l -E[e-aFl])/((ead-1)E[y1]). This together with (9.14) yields (9.13) with a replaced by (ead -l)/d, and a reference to (9-12) and (9.lid) completes the proof in the lattice case. □ 9.8 Stochastic Domination We shall now use Theorem 9.2 to establish the stochastic domination result needed to apply Theorem 8.5 for taboo limit purposes. Theorem 9.3. Suppose the conditions in Theorem 9.2 hold. Then there are finite constants a and b such that for t € [0, oo) and x € [0, t], P(50 > x\r > t) < aE[eaS° l{OS0>x}] + a eat P(50 > T > t), and P°(Xi >x\r° >t) < 6E°[eaXl l{r.£x1>x}] + b eat Pa(X, > T° > t). (9.16a) (9.166)
Section 9. Taboo Regeneration 447 Proof. By Theorem 9.2, there are t0 > 0, oo > 0, and ai < oo such that for t > t0, we have eat P(r > t) > a0 and P0(f0 > t) 4 ax. For 0 ^ t4 t0, we have eatP(r > t) ^ P(F > t0) > 0 and eaiP°(r° > t) 4 eat° < oo. Thus we may take ao > 0 and a,\ < oo such that e«t p^r > ^ ^ ao and eat P°(r° > *) ^ ax for £ € [0, oo). (9.17) For x € [0, t] we have (using taboo regeneration for the third identity) P(50 > x,r > t) = p(x < So 4: t,r > t) + P(S0 >t,r >t) = I P(S0 £dy,r>t) + P(S0 >t,T>t) J(x,t] = [ p°(r° > t - y)P(S0 g dy,r > So) + P(50 > t,r > t) J(x,t] 4 ai e~at [ eay P(50 € dj/, T > S0) + P(50 > t, T > t), J{x,t] where the inequality follows from (9.17). Divide by P(.T > t) and apply (9.17) again to obtain, with a the maximum of a^/ao and l/«o, P(50 > x\r > t) 4a f eayP(S0 £dy,r^S0)+aeatP(So >t,T>t). J{x,t] Now / e«y P(50 £dy,r> S0) = E[eaS° l{Os0}l{x<s0^}] J{x,t] and a eat P(r >S0>t)4 E[eaS° l{r>s0}l{t<sQ}], and thus P(50 >x\r >t)4 aE[eaS° l{r>s0}l{x<So&}] + aE[eaS° l{r>s0}l{t<s0}} + a eat P(S0 > T > t). Add the first two terms on the right to obtain (9.16a). In order to obtain (9.166), repeat the above argument with So, -T, P, and E replaced by X?, T°, P°, and E°. D 9.9 The Taboo Limit Theorem We are now ready to give conditions under which a taboo regenerative process considered in a time interval [t — h, oo) tends, under taboo in [t, oo), to a limit process with time set [—h, oo) as t —t oo.
448 Chapter 10. REGENERATION Theorem 9.4. Let (Z,S,F) be taboo regenerative (in the wide sense or not). Suppose there is an a > 0 such that E°[eaXl l{r°^Xi}] = 1, (9.18a) E[eaS° l{r>So}] < ~. (9-18fc) E°[XieaXll{r=^l}]<oo, (9.18c) eat P(S0 > r > t) -> 0 as t ->• oo, (9.18d) /■CO / sup eat P°{XX >T° >t)dx< oo. (9.18e) JO t€[x,oo) If P°(Xi G -l-T0 ^ Xi) is spread out, then for each h G [0, oo), there exists a stochastic process Z(*'_/l) = (Zs*' )se[_/,]00) such that P{9t-hZ G -|r >t)% PiO-hZ^-V G •), t ->. oo. IfP°(X\ G -li-"0 > Xi) is lattice with span d and P°(50GdZ)=l and P°(r° G dZ|r° < Xi) = 1, (9.19) £/jen /or eac/i h G [0, oo), £/iere exists a process Z(*'~h) = (Zi*'-/,))se[_/l,00) suc/i £/ia£ P^d-ftZ G -\r > nd) lA V{e-hZ^-V G ■). n ->• oo. Comment. The conditions (9.18a) through (9.18d) are the same as the conditions (9.11a) through (9.lid) in Theorem 9.2, but (9.18e) is stronger than (9.lie). On the other hand, (9.18e) is weaker than a first-moment version of (9.lie): we have E°[re«rl{r<Xl)]<co =» (9.18e) => E°[e°rl{r.<Xl}] <oo. In order to establish the former implication, note that eat P°(X! >r°>t)^ E°[eaX> l{r°<xl}l{r°M}] ^E°[eaXll{ro<Xl}l{ro>x}], t^x, take the supremum over t > x, and integrate over x to obtain /•OO / sup eatP°{Xl >T° >t)dx Jo te[x,oo) /•OO ^E0[eoX'l{r<Xl} / l{r°>x}dx] Jo = E°[reaX'l{r<Xl}].
Section 9. Taboo Regeneration 449 The latter implication follows from /•oo fT° / eax P°(X1 > r° > x) dx = E°[l{ro<Xl} / eax dx] Jo Jo >(E0[earl{r<x,}]-l)/a. Here we need (9.18e) to be able to apply Theorem 8.5, but let us state as a conjecture that Theorem 9.4 holds with (9.18e) replaced by (9.lie). This is suggested by Theorem 9.2 and also by the structure of the limit process in the next section, which relies only on (9.lie). Proof. Theorem 9.4 follows from Theorem 8.5 if we can establish that the family (Z^r\S^), r G ( —oo,0], in Theorem 9.1 satisfies the conditions (8.23) through (8.26) in Theorem 8.5. We shall establish this in the lattice case and with a modification in the spread-out case. According to (9.16a) in Theorem 9.3, we have P(50 *£ x\r >t)> G(x), t G [0, oo), x G [0, t], (9.20) where G is the nondecreasing right-continuous function denned at its continuity points x G [0, oo) by G(x) = 1 - (aV[eaS°l{r^s„>x}] + a sup eat P(S0 > T > t)) M. £G[x,oo) Due to the conditions (9.186) and (9.18d), G(x) —> 1 as x —> oo, and thus G is a distribution function. Since the distribution function of SQ + t is P(50 ^ -\r > t), we obtain the condition (8.23) from (9.20). According to (9.166) in Theorem 9.3, we have P0(Xi ^ x\r° >t)> F{x), t G [0, oo), x G [0, t], (9.21) where F is the nonincreasing right-continuous function defined at its continuity points x G [0, oo) by F(x) = l-(bE°[eaX>l{r^Xl>x]} + b sup eatP°(X1>ro>0)Al. t£[x,co) Since /•OO /.OO / E°[eaX>l{ro^Xi>x}}dx = E° eaX>l{r^Xl} l{Xl>x}dx Jo l ' Jo ■ = E°[Xiea^l{r^Xl}dx}, we obtain f xF(dx) = [ (1 — F(x)) dx [see Lemma 5.2] /•OO ^bE°[XieaXll{r^Xl}dx}+b sup eaiP0(Xi>r°>t)da: ' Jo te[x,oo)
450 Chapter 10. REGENERATION Thus, due to the conditions (9.18c) and (9.18e), J xF(dx) < oo. For t 6 [0, oo), the recurrence distribution of the family {Z^r\S^), r 6 (—oo,0], at the time —t is P°(Xi ^ -\r° > t), and thus we obtain the condition (8.26) from (9.21). Now suppose P°(Xi 6 -\r° ^ Xi) is lattice with span d and (9.19) holds. Let m be such that P°(Xi 6 • D [0,md]-,r° ^ Xi) is aperiodic. Take n~^ m and 1 ^ k ^ m, and use taboo regeneration to obtain the second equality in P°(Xi = kd\r° > nd) = P°(Xi = kd,r° > nd)/P°(r° > nd) = P°(Xi = kd, r° 2 Xi)P°(r° > (n - k)d)/P°(r° > nd) ^P°(Xi =kd,T° ^Xx), k = l,...,m. This yields the condition (8.25), and the theorem is established in the lattice case. In the spread-out case there is an integer n and a subprobability density / on [0, oo) such that J / > 0 and P°(S°n EB,r°>S°n)> I f, Be B[0,oo). JB Let c be such that JQC / > 0. Take t ^ c and B 6 #[0,c]. Use taboo regeneration and t — 5° > t to obtain p°(s° 6 B,r° ^ s°,r° > t) > p°(s° e s,r° > s°)P°(r° > t). Dividing by P°(r° > f) yields p°(S°n eB,r°> s°n\r° > t) > / /, Be B[o,c]. ,/B Thus condition (8.24) holds with x[r) replaced by x[r) +■■■+ x£r). Since condition (8.26) holds, it also holds with x[r) replaced by x[r) + .. .+X^] and F replaced by the nth convolution power of F (see the second step of the proof of Theorem 7.1). Thus, in the spread-out case, the conditions of Theorem 8.5 hold with the family (Z<r), S^), r 6 (-oo,0], replaced by the family (Z^r\ (S^)^), r 6 (—oo,0], which is also taboo regenerative up to time zero. The limit result now follows from Theorem 8.5. □ 9.10 Comments on Uniformity, Rates, and Coupling The reader may have noted that Theorem 9.4 does not use the full power of Theorem 8.5. The uniformity results and the associated rate and coupling results are left out. We leave this out to focus better on the taboo limit phenomenon itself. Here are only a few comments.
Section 10. Taboo Stationarity 451 Rate and coupling results follow easily from Theorem 8.5 using the distribution functions G and F introduced in the proof of Theorem 9.4. We obtain the moment conditions for G and F by placing moment conditions on the ingredients in their definition. For the uniformity results the common density and mass function part is easy. However, some care must be taken with the constants a and b in Theorem 9.3. They must be traced through the proof of Theorem 9.2 (using the uniform convergence in Theorem 3.3 rather than using Theorems 10.1 and 10.2 from Chapter 2). The reader may also have noted that we left out the statements on existence of a two-sided limit process. We did this because the issue will be considered in great detail in the next section, where we shall establish the explicit structure of the two-sided limit process. 10 Taboo Stationarity In this section we shall consider the taboo counterpart of stationarity. Stationarity means that the distribution of a process does not change by non- random time shifts. This is the characterizing property of any two-sided limit process obtained by shifting the time origin of a one-sided process to the far future. Similarly, taboo stationarity means that the distribution of a process does not change by nonrandom time shifts under taboo. This is the characterizing property of any two-sided limit process obtained by shifting the origin of a one-sided process to the far future under taboo up to the new time origin. We begin by defining taboo stationarity for general two-sided processes and show that it is the characterizing property of a taboo limit (Theorem 10.1). We then establish a basic but amazingly simple structural characterization of taboo stationary processes (independent-exponential-shift- to-the-past, Theorem 10.2). After this we return to taboo regeneration and explicitly construct a taboo stationary version of a taboo regenerative process (Theorems 10.3 through 10.6). This is the taboo counterpart of the construction of a stationary process in Chapter 8. We finally show (Theorem 10.7) that this taboo stationary version is indeed the limit process in Theorem 9.4 above. 10.1 Taboo Stationary Stochastic Processes — Definition Consider a pair (Z*,T*), where T* is a nonnegative finite random time and Z* — (zs)seu is a two-sided shift-measurable stochastic process. Recall that 6t in this chapter (see Section 2 and Section 9.1 for formal details) denotes the one-
452 Chapter 10. REGENERATION sided shift-map: 6tZ* = (^*+«)«€[o,oo), t £ R, (one-sided shift). Let 6t denote the two-sided shift-map: 6tZ* = (Z;+s)seR, t £ M, (two-sided shift). Call Z* taboo stationary with taboo time r* if shift under taboo does not change the distribution of the pair (Z*, i-1*), that is, if p((etz*,r-t)£-\r*>t) = p((z*,r*)£-), te[o,oo). (10.1) Call (Z*, r*) taboo stationary if this holds. Note that if we shift the origin back, then (10.1) yields P((Z*,T*) £ -\r* > t) = P((0_tZ*,r* + t) £■) for t £ [0, oo). Thus (10.1) is equivalent to the following condition: p((etz*,r-t) e-\r* >t) = p({z*,r*) e-\r* > -t), *gr We shall now show that taboo stationarity is the characterizing property of a total variation taboo limit. Theorem 10.1. A pair (Z*, i"1*) is taboo stationary if and only if there is a pair (Z, r), where Z = (Zs)s€i0(x>\ is a one-sided shift-measurable process and r is a nonnegative finite random time, such that p((et-hz,r-t)e-\r>t)%p({0-hz*,r*)e-), *-><», (10.2) for all h £ [0, oo). Proof. If (10.1) holds, then so does (10.2) with (Z,T) := (90Z*,r*). In order to establish the converse [that (10.2) implies (10.1)], assume that (10.2) holds. Take x € [0,oo) and h £ [x,oo) and note that (10.2) implies [with h replaced by h — x\ that p((et_{h_x)z, r-t-x)£-,r-t>x\r>t) 4 P((e_(h_x)z*,r* -X)£-,r*>x), t -> oo. Divide by P(r~t > x\r > t) on the left and by the limit P(r* > x) on the right [and note that 6t-(h-x)Z = 6{t+x)_hZ and 6_(h_x)Z* = 9x0-hZ*] to obtain that as t —> oo, p{{e{t+x)_hz, r-t-x)£-\r>t + x) % p((ox6-hz*,r - x) £ -\r* > x). According to (10.2), the left-hand side tends also to P((^_/lZ*,i~"*) £ ■). Since the two limits must be identical, we have [replace x by t] that p((ete-hz*,r*) £ -\r* >t) = p((e_hz*,r*) e •), o < t < h. Since h is arbitrary, this yields (10.1). □
Section 10. Taboo Stationarity 453 10.2 Basic Structural Characterization According to the following theorem, taboo stationarity is characterized by an independent-exponential-shift-to-the-past. That is, if Z' is some two- sided shift-measurable process and V is exponential and independent of Z', then (6-vZ',V) is always taboo stationary; and conversely, all taboo stationary processes are of this form. Theorem 10.2. The pair (Z*,T*) is taboo stationary if and only if T* is exponential and independent of 6r*Z*. Proof. Suppose (Z*,T*) is taboo stationary. From (10.1) we obtain p(r*-te-|r* > t) = p(r* g-)> te[o,oo), which is the standard characterization of exponentiality. Moreover, 0r-Z* = 0r--t0tZm, *G[0,oo), that is, Of'Z* is the same measurable mapping of (6tZ*,r* — t) for all t G [0,oo). This together with (10.1) yields P{0r'Z* G -\r* >t)= P(6r*Z* G •), t€ [0,oo). Multiply by P(.T* > t) to obtain P(6r'Z* G -,r* > t) = P(0r«Z* G -)P(^* > 0, l € [0,oo), that is, 6r* Z* and i"1* are independent. Conversely, suppose .T* is exponential and independent of 6r*Z*. Since r* is exponential, we have, for all z in the path set H, P{{6t-r-z,r - 0 G -|r* > 0 = P('(fl-r.z,r*) G •), t G [0,oo). Since J1* and 6r-Z* are independent, we may replace z by 6r-Z* to obtain [since 0t_r,0r.Z* = 0tZ* and 0_r.0r.Z* = Z*] p((0tz*,r* -1) g -|r* > o = P((^*,r*) e •), t g [o,oo), that is, (Z*:F*) is taboo stationary. □
454 Chapter 10. REGENERATION Remark 10.1. In the above we could consider Z* jointly with a nonde- creasing sequence of (—00,00] valued random times S* = (S^)^^ satisfying, for all n ^ 0, -00 « < S*_2 <S*! <0^S* <---<S; on {5;<oo}. For i£t, let 6t denote the following two-sided joint shift-maps: 0t(Z*,S*) = (9tZ*, (5^_+,)-00) where N?_ = inf{* : S*k > t}, et(z*,s*,r*) = (et{z*,s*),r*-t). The triple (Z*,S*,T*) is taboo stationary if for t 6 [0, 00), p(et(Z*,s*,r*) e -\r* >t) = p({z*,s*,r*) e ■). Both Theorem 10.1 and Theorem 10.2 hold with Z* replaced by (Z*,S*). 10.3 Back to Taboo Regeneration — Intuitive Motivation We now return to the topic of the last section, taboo regeneration. The task for the rest of this section is to construct a taboo stationary version (Z*, S*,r*) of a taboo regenerative (Z, S, F). Here is an attempt at motivating this construction intuitively in the proper taboo regenerative case, that is, not in the wide-sense case but in the case when (Z: S, r) satisfies (9.1); see Figure 10.1. Think of (Z*,S*,T*) as a taboo limit of (Z,S,T). Then the following guesses seem reasonable. The cycles of (Z*,S*) coming in from the past, ..., C^, C!li, should be i.i.d. and independent of the cycle Cq straddling zero. Moreover, conditionally on {Sq < 00}, the cycles ... ,CI2)C-i>Co should be independent of the future 9s*(Z*,S*,r*), which should behave as the zero-delayed version of (Z,S,r). In order to have a complete description of (Z*,5*,T*) there are still three guesses missing: the distribution of (711, the distribution of Cq, and the position of zero in the cycle Cq . But we shall not proceed further along this path [for the complete description of (Z*,S*,T*), see the comment following Theorem 10.4] because it turns out to be easier to consider another triple (Z, S, t) defined as follows: (Z, S, f) = 6R. (Z*,S*,r*) (see Figure 10.1), where R* is the last SI in (—00, i"1*], that is, R* := S^,-i = snp{S*k : k £ Z and S*k ^ T*}. In the light of Theorem 10.2, r* should be exponential and independent of <9r'{Z*,S*). But 6r*(Z*,S*) = *0t(Z,S), and -f is the initial point
Section 10. Taboo Stationarity 455 of the ^r*(^*,5'*)-cycle straddling zero. Thus (Z,S,f) is a measurable mapping of 6r*{Z*,S*) and should therefore be independent of T*. Thus we should obtain (Z*,S*) as follows: (Z*,S*) = 6-r,6t(Z,S) where r* is exponential and independent of (Z, S, P). Now let us guess at the structure of (Z, S, t). We have already guessed that the cycles of (Z*,S*) are i.i.d. before time zero. Since (Z*,S*) is obtained by shifting the origin of 6f(Z,S) independently to the past, this suggests that the same should hold for 0p(Z,S), that is, ..., C-i, Co should be i.i.d. copies of Cl1. The naive guess would be that CL\ is like the first cycle C\ of the zero- delayed version, conditioned on the event {r° ^ X{\. But this is not the case. So now let us give up guessing and simply state the upcoming results (in the proper taboo regenerative case). 8.2 Realization of a classical regenerative (Z, S) under taboo in [0, f] The taboo stationary (Z?,S',r) Realization of {z,s,n r* is exponential a and independent of (Z, S,t). FIGURE 10.1. The structure of the taboo stationary version (Z*,S*,r*
456 Chapter 10. REGENERATION It turns out that an exponential biasing of the cycle-length X\ (and not conditioning on {r° ^ X\}) is the appropriate way to change the subprobability distribution P°(Ci £ -,T° > X\) into the probability distribution of the i.i.d. cycles ...,C-\,Cq. Further, it turns out that these cycles should be independent of the future, 60(Z, S, t). Finally, it turns out that an exponential biasing of the taboo time F° is the appropriate way to change the subprobability distribution P°((Z°, 5°, f°) € -,F° < Xx) into the probability distribution oi0o(Z,S,r). 10.4 Construction of (Z, S, F) We shall now turn to the construction of (Z, S, P) motivated intuitively above. Theorem 10.3. (a) Let (Z, S, r) be taboo regenerative, that is, let it satisfy (9.1). Suppose there is an a > 0 such that E°[eaXl l{r°>x-i>] = 1 (10-3) and E°[ear° l{r"<x1}} < °°- Define probability measures Pg, P°,... on (fi,^") 6?/ HP° — ^ "^ <an + l>dpo > q dF"- E°[e^l{ro<Xl}]dF' n^°- Tften tftere exists a two-sided process Z — (Zs)sSr; a nondecreasing two-sided sequence of ( — 00,00] valued random times S — (Sk)0?^ satisfying, for each n ^ 1, -00 <-•■•< S-2 < S-i < 0 = S0 < ■ ■ ■ < Sn on {Sn < 00}, and a finite nonnegative random time F such that p(6s_n(z,s,f)e-) = p°n((z°,s°,r°)e-), n^o. (io.4) Moreover, the cycles of (Z, S) up to time zero, ^■=(Zsk_1+s)se[0^xky k^O, {here Xk = Sk - Sk-i) are i.i.d. with distribution P°ab 00(^1 € •), where P°aboo is the probability measure on (fl,^) defined (as in Section 9.6) by dP°ab00:=e^l{ro>.Yl}dP°. Finally, these cycles are independent of 6o(Z,S,F), which has the distribution Pg((Z°, S°,r°) £ ■)•
Section 10. Taboo Stationarity 457 (b) Let (Z,S,T) be taboo wide-sense regenerative. Suppose (10.3) holds. If Z has a Polish state space and right-continuous paths with left-hand limits, then there exists a triple (Z, S, t) with the above properties except that the cycles ..., C-i, Co need neither be i.i.d. nor independent of 9o(Z:S,r). However, the cycle-lengths ... ,X-\, Xo are still i.i.d. and independent of 80(Z,S,r). Their distribution is Ptaboo(^i € ■)• Comment. The triple (Z,S,t) is obtained as follows: first string out tabooed cycles in (—oo,0) and break the taboo in the first cycle in [0,oo); then bias exponentially the cycle-lengths in ( —oo, 0) and the taboo time in [0, oo). This can be seen (informally) as an exponential biasing of the total (infinite) taboo time from — oo to t. Proof, (a) The P° are a special case of the probability measures Pn from Section 9.6 [with P replaced by P° and (Z, S, T) by (Z°,S°,r°)]. Let (Y,R,t) be a triple with distribution Pg((Z°,5°,r°) e • ). According to Fact 3.1 in Chapter 3, we may assume the existence of i.i.d. cycles ... ,C-i,Co that are independent of (Y,R,T) and have the distribution pt°aboo(c'i e ■)• Define (Z,S) by putting 00{Z, S) := (Y, R) [thus50=0] and by letting ..., C-\, (% be the cycles of (Z, S, t) up to time So = 0. In order to complete the proof of (a), it only remains to establish (10.4). The same argument as in the proof of Lemma 9.1 [leave out So, and replace the cycle lengths X^ by the cycles Ck,F — Sn by 8sn(Z, S, r), and r — So by {Z°,S°,r°)\ shows that taboo regeneration implies [instead of (9.10a), (9.106), and (9.10c)] that for n ^ 1, under P°, C\,...,Cn are i.i.d. and independent of 6s„ (Z, S, r), P°n(Ck G ■) = Pt°aboo(Ci G •), 1 < * < n, p°n(esjz,s,r)£-) = p°((z°,s°,r°)&-), that is, p^c, €■,...,cn€-,eSn(z,s,r)G-) = Pt°aboo(Ci e-)---P°aboo(Ci e-)PS((2°,5°,r°)e-)- Due to the definition of (Z,S,r), this implies that P(C_n+1 e •,...,(% e-A(z,s,f)e-) = P°n(d £■,...,Cn€-,6Sn(Z,S,r)£-), n^0, which is a reformulation of (10.4).
458 Chapter 10. REGENERATION (6) Again the P° are a special case of the probability measures Pn from Section 9.6 [with P replaced by P° and (Z,S,T) by (Z°,S°,r°)]. For bounded f £71® C^ <g> B[0, oo) and n ^ 0, we have ear° l{s:+1^ro<ss+2}/(^1 (Z°, S°, r°)) = (eaXn{r^xl})(ea(r°-Xih{s°+^r°<si+2}f(Oxl(Z°,S°,n)), and thus, due to taboo wide-sense regeneration, E°[e»r l{So+^ro<So+2}/(0Xl(Z°,S°,n| eaXl l{r->Xl}] = eaXl l{r^Xl}E°[ear l{s^r.<s.+i}/(Z0,S0,r°)]. Take expectation and use E°[eaXl l{r°^x1}] = 1 and the definition of P°+1 and P° to obtain p°n+1(eXl(z°,s°,n&-) = p°n((z°,s°,r°)&-), oo. (io.5) The same argument as in the proof of Lemma 9.1 [leave out So, and replace r - Sn by 0Sn (Z, S, f) and r - So by (Z°,S°, f°)] shows that taboo wide- sense regeneration implies [instead of (9.10a), (9.106), and (9.10c)] that for n ^ 1, under P°,Xi,.. .,Xnarei.i.d. and independent of OsJZ, S, T), (10.6a) P°(X* G-)=Pt°aboo(^ieO, lO^n, (10.66) P° (0Sn (Z, S, r) G •) = PS((Z°, 5°, r°) G •)■ (10.6c) Define the distribution of (Z,S,t) recursively by (10.4). Due to (10.5), this definition is consistent, and thus the Kolmogorov extension theorem (Fact 3.2 in Chapter 3) yields the existence of such a random element (Z, S, t). In order to see that the infinite sequence of cycles ..., C-\, Co stretches backward to —oo, note that according to (10.6a), the cycle-lengths ... ,X-i,X0 are i.i.d., and thus their sum is infinite. Thus there exists a triple (Z,S,t) satisfying (10.4). Now (10.6a), (10.66), and (10.6c) yield what remains of (6). □ 10.5 The Taboo Stationary {Z*,S*,T*) Let (Z, S, f) be as in Theorem 10.3, let r* be exponential with parameter a, (10.7a) let r* be independent of (Z, S, t), and put (Z*,S*):=fl-r«MZ,S). (10.76)
Section 10. Taboo Stationarity 459 Then (Z*, S*, T*) is taboo stationary according to Theorem 10.2 (see Remark 10.1). In the next subsection we shall motivate calling (Z*,S*,T*) a version of (Z,S,r). But first we establish the following structure of (Z*,S*,T*). Theorem 10.4. Suppose the conditions of Theorem 10.3 hold. Let the triple (Z*,S*,r*) be as above and, in addition, let the exponential T* be independent of (Z, S, T). Then, for n ^ 1, p(6s*_n(z*,s*,r*) e-,s0*Ar e-) = p*n((Z°,s°,r°) e •, (r* mod xn a (r° - s°_,)) e •), where P£ is the probability measure on (£1,^) defined by e ,aS° ,/0a(XnA(r°-S° ,)) i\i HP* •= - L "-1 —-dP° E°[e^°l{ro<Xl}] Comment. From this we can read that the independent-exponential-shift- backward-from-r1 works as acceptance-rejection (see Section 6 in Chapter 8): it gives the distribution P*(Ci € •) to the 'accepted' cycle QJ that straddles the new origin, and it unbiases the 'rejected' cycles of (Z, S), the cycles that end up in [0, oo) and do not straddle the new origin. It also unbiases the taboo time t or the part of P that becomes positive by the shift. Finally, this shift places the origin at a truncated exponential distance from the right endpoint of the cycle where it happens to fall, or to the right of the taboo time f if t happens to be in that cycle. Proof. Fix n ^ 1. We shall first show that P* is a probability measure. Due to taboo wide-sense regeneration, E°[eaS"- l{S;_1<r-}| eaS""2 l{s;_a<r-}] By assumption, ~E°[eaXl ^r^xa] = 1, and thus, recursively, we obtain E°[eaS--i ^sj.^r-}] = 1- Again due to taboo wide-sense regeneration, H, [e " \e ^ ^ » 1" -l)l{s^_l^r'}\e "-1 l{s°_1<r°}J — e " 1 l{S°_iSjro}H, [e *■ ' -1J. Take expectation and apply E°[eaS"-1 l{s°_1^r°}] = 1 to obtain the first identity in E°[eaS"-i(ea((s»Ar°)-S"-i) -l)l{So_^ro}] = E°[ea(^Ar°) -1] = E°[e°r l{ro<Xl}] (since E°[eaX' l{r->Xl}] = 1).
460 Chapter 10. REGENERATION Thus P* is a probability measure. Now take bounded / € % ® £00 and g € B[0,oo). Then, due to (10.76) and the fact that t < S\, we have E[/(«sljz*,s',r))5(s0*Ar)] (10.8) 00 = T,^f(es.k.SZ,s,r)g(r*-(r-s.k)+)i{§_ki<f,_r,^_k}}- Jfc=-1 Due to (10.4) in Theorem 10.3 [and since r* is exponential a and independent of both (Z, S, t) and (Z, S, T) under both P and P£+n; see Lemma 4.1 in Chapter 8] we have, for k ^ — 1, nWs.k.SZ,s,r))g(r*-(r-s.k)+)i{s_k_1<r_r^s.k}} =Ey/(z",s°r*)9(r-(r-sy)iK_i<p.r.N<s.}]. Put c:=l/E"[e«rl{r<Xl}]. By the definition of P£+n in Theorem 10.3, for all bounded W € T, E°k+n[W] = cE°[W e«r° Ms:+k^<s:+k+l}} and thus 00 Y, E°k+n[W] = cE°[Wear° l{Son_^ro}]. k = -l Combine this, (10.8), and (10.9) to obtain E[/(fls.n(z,,st,r)),(s0*Ar)] = cE°[/(z°,5°,r°)g(r* - (r° - s°n)+) (10.10) 1{5°_1<r°-r*^s°} ea 1{s°_1^r°}]- Since r* is exponential a and independent of (Z°,5°,r°) under P°, we have E0[s(r*-(r0-s°)+)i{ro_So<r.<ro_s._i}|(z0)s0,r0)] = E°[3(r*mod((r°-5°_1)+-(r°-5°)+))|(z°,5°,n] (10.11) ( e-a(r°-S°_l)+ _ e-a(r°-S°)+ J_
Section 10. Taboo Stationarity 461 Now l-{r°-sz^r»<r°-s°_l} = l{s°_1<r°-r*^s°}> (r° - s;u)+ - (r° - s°)+ = xn a (r° - s^), e-a(r°-S°_1)+ _ e-a(r°-S°n)+= eaS°_1(ea(XnA(rt'-S°_1) + )_1)e-ar° Combine this and (10.11) to obtain E°[3(r*-(r0-5:)+)i{So_i<ro_r.^}|(z0,50,r°)] = E°[g(r* mod Xn A (r° - 5°_1))|(Z°, 5°, T°)] Multiply by f(Z°,S°,r°) and by ear° l{s°_ <;r°}> take expectation, and compare with (10.10) to obtain nf(6Sln(z*,s*,r))g(s*0An} = cE°[/(Z°,5°,r°)3(r*modXnA(r°-5°_1)) eQ5;_l(6a(X„A(7--5»_1) + )_1)1{s;,_i^0}]. By the definition of PJj, this yields E[f(Os._n(Z',S;r*))g(SS AT')] = E;[/(z°,5°,r°)3(r* mod xn a (r° - s°_,))], which is the desired result. D 10.6 The Taboo Stationary (Z*, S*, T*) Is a Version of (Z, 5, T) We shall now establish two theorems that motivate calling the taboo stationary (Z* ,S*,r*) a version of the taboo regenerative (Z,S,r), a taboo stationary version. Theorem 10.5 deals with the behaviour of (Z*,S*, T*) after time zero and Theorem 10.6 with the behaviour before time zero. We shall now show that (Z*,S*,r*) taboo (wide-sense) regenerates at Sq and continues after the regeneration like the zero-delayed version (Z°,S°,r°). Theorem 10.5. If (Z,S,T) is taboo regenerative, then the following holds under the conditions of Theorem 10.3(a) and with (Z* ,5* ,T*) as at (10.76): p(es.(Z'I5',ne-|(^Voo,s;),-,S!1,s0t;r^so') (10.12a) = P°((Z°,5°,r°) €•)•
462 Chapter 10. REGENERATION // (Z, S, r) is taboo wide-sense regenerative, then the following holds under the conditions of Theorem 10.3(b) and with (Z*,S*,r*) as at (10.76): p(es.0(z*,s*,r*)€-\...,s*_1,s*o]r*>s*0) (10.126) = p0((z°,50,r°)e-)- Proof. We shall first consider the latter claim. Assume the conditions of Theorem 10.3(6). Take n ^ 1 and bounded / G U ® C^ and g G Bn+1. Let P*n be as in Theorem 10.4 and put c = 1/E°[ear° l{r<.<x1}]- The density of p; with respect to P° is ce^-'fe"1- -1) on {r° ^ 5°}, and thus, by taboo wide-sense regeneration and since we may let r* be independent of (Z°,S°,ro) under P° and thus under P;, E;[/(6»s. (Z°, S°, r°))g(X1 ,...,Xn, (r* mod Xn))l{r^s°n}] = e°[/(z°,5°,r°))]K[g(Xi,...,xn,(r* modxn))i{ro>s.}]. Now r* mod Xn = r* mod Xn A (r° - 5°_i) on {r° ^ S°}, and thus, due to Theorem 10.4, this identity can be rewritten as nf(0s;(z*,s*,r*)g(xin+1,...,xz,s*0)i{r.^}] = v°[f(z°,s°,r°)]ng(x*_n+1,...,xz,s*0)i{r^s,}}. Since n, /, and g are arbitrary, this yields (10.126). The claim (10.12a) is established in the same way with (Xi,... ,Xn) replaced by ((Z°,S^+s))s€[_si7l,o),Xu... ,Xn) and (X*n+1,... ,X£) replaced by ((^+a)se[Sin-s0*,o),^-„+i, ■ ■ ■ ,*o)- D Theorem 10.5 shows that (Z*, 5* r*) behaves in [0, oo) as a version of (Z, S, r). The next theorem completes the motivation for calling (Z*, S*, r*) a version of (Z, S, r) by showing that (Z* ,S* ,T*) behaves in the taboo period ( — oo, 0] as (Z, S, r) does under taboo in any nonrandom interval [0, t]. Theorem 10.6. If(Z,S,r) is taboo regenerative, then the following holds under the conditions of Theorem 10.3(a) and with (Z*, S*, r*) as at (10.76): for t G [0, oo) and s G [0, t], p(^. jz*,s*, n g -k^W^, (s*Nlt_+k)zl-, s*Nlt_ = -8) = P0((Z°,5°,r°)G-|r° >«). (10.13a) If (Z,S,r) is taboo wide-sense regenerative, then the following holds under the conditions of Theorem 10.3(b) and with (Z*,S*,F*) as at (10.76): for
Section 10. Taboo Stationarity 463 t € [0,oo) and s € [Q,t], p(es%. (z*,s*,ne-\(S*N.t +k)zl;S"N,t =-s) (10.136) = p°((z°,50,r0) e-|r° > s). Proof. We shall first consider the latter claim (10.136). Assume the conditions of Theorem 10.3(6) and fix t € [0,00) and s € [0, i\. Since (Z*,S*,T*) is taboo stationary, we have p(«s.,jr,s4,r)g,((i;.t_+t)»co,s^_)e-) = p(es.0(z*,s*,r*)&-,((x*k)0_oo,s*0-t)&-\r*>t) and thus P(Js..(_(r)st,ne-|(n:1.XiS^.=-») (10-14) = p(6si(z*,s*,r*)£;r*>t\(Xj)0_oo;SS-t = -8) p(r*>t\(x*k)°_00;S5-t = -s) On {Sq - t = —s} we can rewrite r* > t as r* — Sq > s, and thus [see Fact 3.1 in Chapter 6] we have p(6ss(z*,s;n g -,r* > wxtt^ss -t = -s) = p(6»s.(z*,s*,r*) e -,r* - s0* > sITO0^;^ -1 = -s). Since r* - Sq > s implies r* ^ Sq, this yields p(es-(z;s;n g -,r* > tK^^sgg -1 = -s) P(r*2SS\(X*k)0_oo;SS-t = -8) = P(6s.0(Z*,S*,n€-,r*-S*0>s\(X*k)0_oo;S*0-t=-s,r>S*0). By Theorem 10.5, the right-hand side equals P°{{Z°,S°,ro) £ -,r° > s). Thus p(0ss(z*,S',n e -,r* > tim^g* -t = -8) p(r* ^ ssk^j^^o* -1 = -s) (10.i5) = p°((zo,50,r°) e-,r° >s). . In particular, p(r>t\(x*k)0_oo;S5-t = -8) P(r*^S*0\(X*r_oo;S*0-t = -s) p°(r° >«). Divide (10.15) by this and compare with (10.14) to obtain (10.136). The claim (10.6) is established in the same way with (X^* )?_00 replaced by ((z;).<s;,._ _, (*£.,_+*)» oo)- ~" n
464 Chapter 10. REGENERATION As in Theorem 9.1, define a probability kernel p(-\-) by p(-\s) = P°((Z°,S°) G -\r° > -s), s G (-oo,0]. We shall now show that (Z*,S*) is time-inhomogeneous (wide-sense) regenerative up to time zero of type p(-\-) in the following two-sided sense. Corollary 10.1. If (Z,S,T) is taboo regenerative, then for t G [0, oo) and P(^.i_+n(^,5*)G-|(Z;)s<5^_+n,(S^.(_+fc)^00) = P(-\S*Nlt_+n) on {^,(_+n < 0}. (10.16a) // (Z, S, r) is taboo wide-sense regenerative, then for t G [0, oo) and n ^ 0, = Pi-\S*Nlt_+n) on{S*N,_t+n<0}. (10.166) Proof. First consider the latter claim (10.166). Assume that (Z,S,T) is taboo wide-sense regenerative and fix t G [0, oo) and n ^ 0. According to Theorem 9.1 there is a pair (Z^~^, 5(~^) such that for so,... ,sn G [0, t], P(es(-„(Z(-t))S<-i))G-|S(-t) = -So,...,5^t) = -Sn) (10.17) = P(-\ ~Sn). According to (10.136) in Theorem 10.6 and Theorem 9.1, for Sq G [0,t], p(es-Nit_jz*,s*) g -,(Shlt_+k)i e ■\(s*Nlt„+k)-1oo;S*Nlt_ = so) = P{9si-t)(Z^\S^) G -.(S^)? G .|S(-4> = -So). This implies that for so, ■.., sn G [0, t], P(^, ^+n(Z*,S*) G •|(5'^.(_+j.)lJ<); Sjv-(_ = -so,- ■ ■, 5'^(_+n= -sn) = P(0S,_„ (Z(-*), S(-')) G • \Sil) = -5o,. • •, S^-') = -sn). This and (10.17) yield (10.166). The former claim (10.16a) is established in the same way with replaced by ((Z*)S<S*N, AS*N-_t_ (S*Nlt_+k)? replaced by (C*Nli_+k,S*N._t_+k)^, and (S^_t))r replaced by (C^_t),S^-0)?. D
Section 10. Taboo Stationarity 465 10.7 The Taboo Stationary (Z*,S*,r*) Is the Taboo Limit We shall now show that in the spread-out case Z* is indeed a two-sided extension of the family Z^*'~h\ h £ [0, oo), of one-sided taboo limit processes in Theorem 9.4. Theorem 10.7. Let (Z,S,r) be taboo regenerative or taboo wide-sense regenerative. In the wide-sense case, let Z have Polish state space and right-continuous paths with left-hand limits. Suppose the conditions (9.18a) through (9.18e) in Theorem 9.4 hold and let (Z*, S1*, r*) be as at (10.7a). If P°(Xi £ -\r° ^ Xi) is spread out, then for each h £ [0, oo), P(9t-hZ £-\r > t) %V{6-hZ* £■), <->oo. (10.18) In fact, for each h £ [0, oo), p(et-h{z,s,r)e-\r>t)%p(0-h(z*,s;r*)€-), *-><». (10.19) Comment. The taboo stationary (Z*,S*,T*) exists without the first moment condition (9.18c), that is, without E°[Xi eaXl l{r°^Xi}] < °°- Let us state as a conjecture that this condition should not be needed for Theorem 10.7. See also the comment to Theorem 9.4. Proof. Fix h £ [0, oo). We obtain (10.18) from Theorem 9.4 if we can establish that (Z*)sG[_/jj00) is a copy of the limit process Z(*'~h"> in (8.28a). For that purpose let (Z^r\S^), r £ ( —oo,0], be as in Theorem 9.1 and let n ^ 1 be such that P°(5° £ -\r° ^ 5°) has a density component. Recall from the proof of Theorem 9.4 that Z(*'~h^ is the limit from-the- past obtained by applying Theorem 8.5 to the family (Z^r\ (S^)fL0), r £ (-oo, 0]. Due to Corollary 10.1, the family {{Z*a)a&[r<oo),{S*N._+kn)f=0), r £ ( — 00,0], is time-inhomogeneous (wide-sense) regenerative up to time zero and of the same type as (Z(r), (5^)^0), r £ ( —oo,0]. According to Theorem 8.5, the limit Z(*'~'1) is determined by the type, and thus, if ((Zs)se[r,00),(s*N*_+kn)'kLo),r 6 (-°o,0], satisfies the (10.20) conditions (8.23), (8.24), and (8.26) in Theorem 8.5, then Theorem 8.5 yields that (Z*)sG[roo, r e (-oo,0], has also the limit Z(*~hl Since (Z;)se[r^h r £ (-oo,0], trivially has the limit (Z*)s€[_hi0o), it follows that (Z*)s€[_/ii00) is a copy of Z(*-~h) as desired. Thus (10.18) follows if we can establish (10.20). Now, in the proof of Theorem 9.4 we showed that (Z^, (S^)f=0), r e (-oo, 0], satisfies (8.23), (8.24) and (8.26). Since the conditions (8.24) and (8.26) only have to do with the type, it follows that ((Z*)ge[ritX)), (S£f._+fc„)£l0), r £ (-oo,0],
466 Chapter 10. REGENERATION also satisfies (8.24) and (8.26). Thus it only remains to establish (8.23), namely, that there is a distribution function G on [0, oo) such that P(S*N._-r^x)^G(x), r€ (-oo,0], x€ [0,-r]. (10.21) By Theorem 10.3, ...,X_i,Xo are i.i.d. with distribution P°aboo(^i € •) and independent of r. Thus [due to (10.7a)] ... ,X^2)^Ii are i.i.d. with distribution Ptaboo(^1 e ') an<^ independent of St1. By the assumption (9.18c), E^!^] < oo. Thus (—S^_l_k)(f) is a renewal process with finite mean recurrence time. Note that for r € (—oo,0] and x € [0, — r], P(S*N,_ -r^x)= P(y(_r_x)_ < a;), (10.22) where y(_r_x)_ is the residual life of (—S*_1_k)(f immediately before time —r — x. Due to the domination result at (7.8), there is a nonincreasing right-continuous function g such that g(x) —> 0 as x —» oo and P(F(_r_x)_ >x\- SI, = s) < g(x) for r £ (-oo,0], x 6 [0, —r], and s € [0, -r - x]. This and P(F(_r_x)_ > x, -SI, > -r) < P(-S*! > x) yield the inequality in P(y(_r_x)_ > a;) = P(y(_r_x)_ >x, -S*_! <-r - a;)+P(y(_r_x)_ >x, -S*, > -r) <fl(a;) + P(-S*1>a;), r € (-oo,0], are [0,-r]. Put G(ar) = 1 - (g(x) + P(-5*! >i))Al to obtain (10.21) from this and (10.22). Thus (10.18) is established. In order to obtain (10.19), apply the above argument to the taboo wide- sense regenerative triple ((Os(Z*,S*,r*))seR,S*,r*). □ 10.8 The Lattice Case — Periodic Taboo Stationarity We shall end this section by looking briefly at the taboo counterpart of periodic stationarity. Note that this discussion also covers discrete time (see Section 2.6). Consider a pair (Z**,T**), where r** is a nonnegative finite random time and Z** = (Z*s*)seu is a two-sided shift-measurable stochastic process (possibly the extension to continuous time of a discrete-time process). Call (Z**,r**) periodically - taboo stationary with period d if d > 0, r** is dZ valued, and p((endz**,r** -nd) €-\r** >nd) = p((z**,r**) e ■), « ^ o.
Section 11. Perfect Simulation - Coupling From-the-Past 467 It is readily checked that the analogue of Theorem 10.1 holds: (Z**,T**) is periodically taboo stationary with period d if and only if there is a pair (Z,T) such that p((end-hZ,r-nd)e-\r>nd)%P((9-hz*,r*) g-)> « -> oo. Also, it is readily checked that the analogue of Theorem 10.2 holds: (Z**, r**) is periodically taboo stationary with period d if and only if r** /d is geometric and independent of 6T**Z**. In the above we can replace the pair (Z**,T**) by a triple (Z**, S**, T**); see Remark 10.1. Let (Z, S, r) be taboo regenerative (in the wide sense or not) and assume that P°(Xi £-\r° ^ Xx) is lattice with span d and that P°{S0€dZ)=l and P°(r° G dZ\r° < X,) = 1. Let (Z,S,t) be as in Theorem 10.3 and let (Z*,S*,T*) be as at (10.7a). Put r** =d+[r*/d\d and (Z**,S**) := 0-r..9f,(Z,S). Then r**/d is geometric and independent of (Z,S,t), and thus we have that (Z**,S**,r**) is periodically taboo stationary with period d. Observe that r**-T* is [0,d) valued and independent of (Z**,S**,T**) and that (z**,s**,r**) = 9r.-r,.(z*,s*:r*). From this, the lattice assumption, and Theorems 10.5 and 10.6 we obtain easily that Theorems 10.5 and 10.6 hold with (Z*,S*,T*) replaced by (Z**,5**,r**), that is, (Z**,5**,T**) is a periodically taboo stationary version of (Z, S, r). Now the proof of Theorem 10.7 (with obvious modifications) yields the following result: if the conditions (9.18a) through (9.18e) in Theorem 9.4 hold, then for h £ [0, oo), P(6nd-h(Z,S,r) G -\r > nd) 4 P(6.h(Z**,S**,r**) G •) as n -» oo. 11 Perfect Simulation - Coupling From-the-Past We shall end this final chapter by considering the simulation aspects of the above theory. Coupling from-the-past (Section 8) can be applied to finite state space Markov chains to generate the stationary version of a time-homogeneous chain, the two-sided version of a time-inhomogeneous chain, and the taboo stationary version of a time-homogeneous chain (in particular, the so-called quasi-stationary distribution).
468 Chapter 10. REGENERATION In Section 6.1 of Chapter 8 we discussed briefly the general problem of generating a stationary version of a given stochastic process, and the same discussion applies with obvious modification to the two-sided time- inhomogeneous case and the taboo case. We then gave a solution for Palm duals with bounded cycle-lengths using the acceptance-rejection algorithm. At the end of this section this algorithm is applied together with the structural results of Section 10 to generate the taboo stationary version of a taboo regenerative process when the minimum of the cycle-length and the taboo time is bounded and the exponential parameter a is known. An important distinction between the acceptance-rejection algorithm and the coupling from-the-past algorithm is that the coupling algorithm works without knowledge of a. An interesting common feature of the algorithms is that the transition probabilities (or cycle distribution) of the processes need not be known. The processes could, for instance, be the output of another simulation. 11.1 Generating a Stationary Finite-State Markov Chain Consider a Markov chain in discrete or continuous time Z = (Zk)™ or Z = (Zs)se[o,oo) with a finite state space E. Assume that Z is irreducible and, in the discrete- time case, that Z is aperiodic. Suppose it is known how to generate Z starting from any given initial state i. Note that the problem of generating the stationary version Z* of Z can be reduced to that of generating the stationaTy4nitial state Zq. The stationary initial state Zq can be generated by coupling from-the-past as follows (see Figure 11.1). Initial step. Start a family of independent versions of Z in all states at time —1 and run them up to time 0 (that is, one transition in the discrete-time case). Recursive steps. For each n ^ 2, start a new family of independent versions of Z in all states at time —n and run them up to time —n + 1. From time — n + 1 let the chains continue up to time 0 as follows: if the chain starting in state i at time — n is in state j at time — n + 1, let it continue up to time 0 along the path of the chain that starts at time — n + 1 in state j. Termination condition. Terminate the recursion at the first n ^ 1 such that all the chains that start at time — n are in the same state at time 0. This state is a realization of the stationary state Zq, according to the following theorem.
Section 11. Perfect Simulation - Coupling From-the-Past 469 -M ... -4 -3 -2 -1 0 FIGURE 11.1. Coupling from-the-past. Theorem 11.1. Let M be the first n ^ 1 such that all the chains that start at time —n are in the same state at time 0. Call this state Y. Then P(M < oo) = 1 and Y is a copy of Zq. Proof. Let P; indicate that Z starts in the state i, that is, Pi(Z0 = i) = 1. According to Theorems 3.1 and 3.2 in Chapter 2, limn^f00Pi(Zn = j) = P(Zq = j) > 0 for all states i and j. Thus there are jo, no, and p > 0 such that Pi(Zno = j0) > p, i£E. For each n ^ no the probability that all the chains starting at time — n are in the same state at time — n + no is no less than the probability that independent chains starting from all states at time — n are all in the state jo at time —n + n0. Thus P(M > {k + l)n0\M> km) ^ (l-p*E), k^O. This implies P(M > kno) ^ (1 - p*E)k -> 0 as k -> oo, that is, P(M < oo) = 1. Now fix an i € E. Let Zq"''' be the state at time 0 of the chain that starts in i at time —n and note that Z^~n''' =7on {M < n}. Thus, for each j £ E, P{Z(~n'l) = j) = P(zin'l) =j,M>n)+ P(Y =j,M^ n) ->P(V=j), n ->• oo. But for each j € £7, P(Z(-"'l) = j) = Pt(Zn = j) -> P(Z0* = j) as n ^ oo, and thus P(Y = j) = P(Z^ = j) as desired. □
470 Chapter 10. REGENERATION Remark 11.1. Why use coupling from-the-past and not ordinary coupling to-the-future? Simply because the latter does not work. The coupling time T, when the chains starting from all states at time zero merge, is also the state of the stationary chain. But T is random, and thus the stationary chain need not have the stationary distribution at time T. (In order to see this, consider a chain with state space E = {1,2,3} which goes from 1 to 2 with probability |, from 1 to 3 with probability |, from 2 to 3 with probability 1, and from 3 to 1 with probability 1. The versions starting from all states at time zero merge in state 3. The nonrandom state 3 is not the stationary state.) 11.2 More Efficient Algorithm for Birth and Death Chains The above algorithm shows that perfect simulation is theoretically possible, but the algorithm is not very efficient. Typically, in practical simulations the number of states is several thousands or millions, even trillions or more, and thus having to generate independent transitions from all states is astronomically expensive and time-consuming. However, in special cases there are efficient versions of the above algorithm. For instance, if Z is a birth and death chain in continuous time, then we can (recursively for n ^ 1) generate two chains starting at time —n, one starting from top (from the highest state) and the other from bottom (from zero), and run each of them until time zero or until it merges with a chain starting at time — k for some k < n. Repeat this until the first n such that the two chains starting at time — n are in the same state at time zero._This- common state is a realization of the stationary state because airdiains coming in from the past (in particular the stationary chain) are captured by these chains and have to merge with them. The same trick can be used for more complicated monotone chains, that is, chains having a partially ordered state space and transition probabilities that preserve this partial ordering. An example is the (finite) Ising model, a Markov chain with state space { — 1, l}{0----'fe} (a state is a configuration of minus ones and ones indexed by a location in {0,..., k}d) and with the property that a state changes either by switching a single —1 to 1 or a single 1 to — 1. The chains starting from top (with ones at all locations) and the chains starting from bottom (with zeros at all locations) will then capture all chains that come in from the past. 11.3 Generating a Two-Sided Time-Inhomogeneous Chain The above coupling algorithm also works for time-inhomogeneous chains. To simplify the presentation we shall only consider the discrete-time case. Analogous results hold in the continuous-time case.
Section 11. Perfect Simulation - Coupling From-the-Past 471 Let E be a finite or countable set and, for each k G Z, let Pk = (Pkij :i,j€E) be transition probabilities on E. For n ^ 0, consider a time-inhomogeneous Markov chain in discrete time z(-„) = (Z(-"))r=_n starting at time — n with transition probabilities Pk, k ^ —n, that is, P(Z^)=j|4-n)=i) = PWi, i,jeE, k^-n. Assume that there is a finite subset B of E and jo G B, n\ > n0 ^ 1, and p > 0 such that Pkij =0, k<0, ieB, j$B, P(Z{_-n%0=j0\Z(_-nn)=i)>p, n>nu ieB. (11.1) Suppose it is known how to generate, for each n ^ 0, versions of Z'_n' starting from any given initial state i £ B and suppose we wish to generate a two-sided version, Z* = (Z*k) OO OO" Then the key task is to generate, for each fixed integer m 4, 0, a realization of Z^. If this can be done, we can, for any k Js 1, run a version of Z(_m) for k steps starting from the realized value of Z^ and obtain a realization of {Z*n,..., Z^+k). So fix m ^ 0 and apply coupling from-the-past as follows to obtain a realization of Z^. Initial step. For each i€B, generate an independent copy Z\^~ 'l' of Zm ~ ' starting at time m — 1 in state i. Recursive steps. With n ^ 2 generate, for each i G B, an independent copy Z^™Sn+\ °f Zm-^n+i starting at time m-n in state i. From time m — n + 1 define thechain z(m-rM) up to time m as follows: Tf 7i(m-n'i) _ -■ nl]t 7{m-n,i) _ 7(m-n+l,j) „ < , < Termination condition. Terminate the recursion at the first n ^ 1 such that all the chains that start at time m — n are in the same state at time m. This common state is a realization of Z^, according to the following theorem.
472 Chapter 10. REGENERATION Theorem 11.2. Let Pk, k G Z, be a family of transition probabilities satisfying (11-1). Then there exists a two-sided time-inhomogeneous Markov chain Z* = (ZD^^ with one-step transition probabilities Pk, k G Z. Moreover, for all families Z^~n> of time-inhomogeneous Markov chains starting at time —n in the set B with one-step transition probabilities Pk, k ^ — n, it holds that as n —> oo; P(Z<-n) = j) -» P(Z*m=j) (limit from-the-past), (11.2) for all m G Z and j G E. Finally, take m ^ 0, fix an arbitrary state Iq G B, and in the above algorithm put M = inf{n > 1 : Z^~nA = Z^'71'^ for all i G B}. Then P(M < oo) = 1, and Z(^M'io) is a copy of Z*m. PROOF. Due to (11.1), for each n > n\, the probability that the chains Z(m~n'l\ i0 G B, starting at time m — n are all in the same state at time m — n + no is no less than the probability that independent chains starting from all states at time m — n are all in the state jo at time m — n + Uq. Thus P(M > (k + l)n0 + k0\M > kn0 + k0) < (l-p*B), k > 0, (11.3) that is, P(M < oo) = 1. Note that for i,j G B, p(Z(m-n,i) = j} = P(Z^~n^ =j,M>n)+ P{Z^-M^ = j, M < n) (11.4) _>P(Z(j»-Af,io)=j)) n_>00. In order to establish the (mathematical) existence of Z* put, for k ^ m, Mk = inf{n > 1 : ^m_n,i) = ^m""'io) for all i G B}. Then (11.3) holds with M replaced by Mk, and thus P(Mk < oo) = 1. Define Z* up to time m by From time m onward, let Z* run according to the transition probabilities Pk, k > m. Then by definition, Zm~ 'l°> is a copy of Z£, and (11.2) follows from p(z(l-)=j)=x: p(^-n,°=j)P(zi-nn)=i) ->• P(^m = j) asn -» oo [due to (11.4) and #B < oo].
Section 11. Perfect Simulation - Coupling From-the-Past 473 It only remains to establish that Z* is Markov with transition probabilities Pk, k G Z. Take k ^ m and jk,... jm £ B. A calculation like (11-4) yields p(z(m-„.io) = jfc> . . . , ^"".io) = jm) _> p(Z* =jki^_jZ*m= jm) and P(4m-",io) = jfc) -> P(Z*k = jk) as n -> oo. Combine this and p/7(m-n,i0) _ • 7(m-n,i0) _ • \ ^l^fc ~ Jk,- ■ ■ ,Am — Jm) = P(Zim-n^) = Jfc)P(^ffi° = J*+l , ■ ■ ■ , ^ = Jm) to obtain = P(z* = i,)P(zf+f > = jk+1,..., z&m = jm), that is, Z* is Markov with transition probabilities Pk, k G Z, up to time m. From time m onward, Z* is Markov with transition probabilities Pk, k G Z, by definition. □ 11.4 Generating a Taboo-Stationary Markov Chain Consider a Markov chain in discrete time Z = (W with state space E. Let B be a finite subset of E and let r be the first exit time out of B, r = inf{/b> \:Zk & B}. Assume that r is a.s. finite for all initial states and that B is irreducible: Vi,j &B 3k > 1 : P{Zk =j,T> k\Z0 = i) > 0; and aperiodic: gcd{k > 1 : P{Zk =i,T> k\Z0 = i) > 0} = 1, i G B. Assume that it is known how to generate Z starting from any given initial state i and suppose we wish to generate the taboo limit of Z, that is, a two-sided taboo stationary chain Z* = {Zl)°?00 such that for all integers h, P(Zn^h = j\r > n) ->P{Z*_h=j), n^oo. (11.5) In the special case when h = 0, the distribution of Zq is called the quasi- stationary distribution.
474 Chapter 10. REGENERATION The chain Z conditioned on {F > n} is a time-inhomogeneous Markov chain, and the simulation algorithm from the previous subsection can be applied to generate Z*. We shall show how to generate the taboo segment {Z*_h,..., Zq) of Z* ending at time zero. In order to generate {Z*_h,..., Z^) for an m > 0 we can then continue from time 0 up to time m with a version of Z starting from the realized value of Zq. Fix an integer h ^ 0 and generate a realization of {ZZh, ■ ■ ■, Zq) as follows. Recursive steps. For n ^ 1, fix i € B and generate i.i.d. versions Z(-h-n,i,i) Z(-h-n,i,2) ^ of z starting at time -h - n in state i and ending at time 0. Let r(>-h-n'i>1\r(-h-n'i<2\... be the first exit times out of B and continue generating until K(-h-n,i) = jnf {fc ^ x . r{-h-n,i,k) > 0} Define a chain Z^~h~n^ starting at time —h — n in stat^i, ending at time 0, and tabooed in the time set {—h — n,... ,0}, by Z(-h-n,i) = ^z(-h-n+ltj)) if Z(_-™.Mfi(-h-"(°) = j. Do this for each i G B. Termination condition. Terminate the recursion at the first n ^ 1 such that all the chains Z(~h~n'l\ i € B, that start at time —h — n are in the same state at time —h. These chains will then run together from this common state up to time 0, and this common segment is a realization of {Z^_h,..., Zq), according to the following theorem. Theorem 11.3. Let Z = (Zfc)o° be a Markov chain in discrete time with state space E. Let B be a finite irreducible aperiodic subset of E with an a.s. finite first exit time r. Then there exists a two-sided time-inhomogeneous Markov chain Z* = {Z^)0^^ such that for all h ^ 0, P((Z„_fc,..., Zn) = -\r > n) -> P((Z* h, ...,Z*) = ■), n -> oo. Further, take h > 0, fix an arbitrary state io, and in the above algorithm put M = inf{n > 1 -. Z^~n^ = Z^~nM) for all i&B). Then P(M < oo) = 1 and {Z{Shh~M'io),... ,Z(0~h'MM)) is a copy of {zih,...,zs). Proof. Let the chains Z^~h~n^ run from time —h — n up to infinity (rather than end at time zero). The acceptance-rejection used to obtain these chains renders (see Theorem 6.1(a) in Chapter 8) p(z(-fc-n,i) G .) = P((Zfc+n+fc)2L_h_n € -W > n + h).
Section 11. Perfect Simulation - Coupling From-the-Past 475 This in turn implies that the chains Z^~h"n'^ form a family of time- inhomogeneous Markov chains with common transition probabilities. Thus Theorem 11.2 applies to yield the desired results if we can establish (11.1), that is, if we can show that there are jo G B, n\ > no #s 1, and p > 0 such that Pi{Zno = j0\r > n) > P, n^ni, i€B, (11.6) where Pi indicates that Z starts in the state i. In order to establish (11.6), we shall begin by applying Lemma 3.1(6) in Chapter 2, which states that an aperiodic additive set A of nonnegative integers contains all integers from some k0 onward. Fix a state jo € B and put A = {k 2 1 : Pjo{Zk = jo,T> k) > 0}. By assumption, A is aperiodic, and A is additive, since for all k and k' ^ 1, pjo{zk+k. =j0,r>k + k') > vjo{zk = jo,zk+k. =j0,r>k + k') = pjo{zk = jo, r > k)pj0{zk, = j0, r > k'). Thus Lemma 3.1(6) in Chapter 2 yields the existence of an integer ko such that Pjo{Zk=jo,r>k)>0, k^k0. (11.7) Since B is irreducible, there is, for each i € B, an integer m; such that Pi{Zmi=jo,r>mi)>Q. (11.8) Put no — ko + maxj£B mi- In (H-7) take k = no — m; and multiply by (10.8) to obtain Pi{Zno=jo,r>no)>0, iEB. (11.9) Since B is irreducible there is, for each i e B, an integer ki such that pj0{zki =i,r>ki)>o. Multiply (11.9) by this to obtain Pi{Zno = j0,Zno+ki =i,r>n0 + ki) >0, ieB. (11.10) Put m = no + maxjes &»• F°r i G B and n ^ m, Pi{Z„0 = jo, r > n) > Pi{Zno = jo, Zno+k. =i,T> n) = Pi{Z„0 = jo, Zno+ki =i,T > n0 + h)Pi(r > n- (n0 + hi)) > Pi(^„0 = io, ^no+fci = *, £ > "o + h)Pi(r > n).
476 Chapter 10. REGENERATION This and p := inf Pi{Zno = j0, Zno+ki =i,r>n0 + k{) > 0 [due to (11.10)] yield Vi(Zno = jo, r >n)> PPi(r > n). Divide by Pj(.T > n) to obtain (11.6) and complete the proof. □ 11.5 Generating a Taboo Stationary Regenerative Process Finally, let us consider how to generate the taboo stationary version of a taboo regenerative process by acceptance-rejection (Section 6 in Chapter 8) using the structural results of Section 10. Let (Z, S, r) b, e properly taboo regenerative, that is, let [Z, S, r) satisfy (9.1). Assume th&t there are known finite constants a and b such that P°(A"i < a\r° ^X1) = l and P°(r° < b\r° < AV)_ = 1, and that there is a known a > 0 such that V°[eaX> l{r^xl}] = 1. Recall from Section 10 that the taboo stationary version (Z*,S*,T*) of (Z, S, r) has the following structure: r* is exponential with parameter a and (r,s') = I.r.Mz,s), (ii.il) where (Z, S, f) is as in Theorem 10.3 and independent of r*. Suppose it is known how to generate the zero-delayed version of (Z, S, r). Then the taboo stationary version (Z*, S*, r*) can be generated as follows. 1. Generate i.i.d. copies of the cycle C\. Let C^\C^2\ ... be the cycles that obey the taboo. Let {R^, T^), (i?(2\ T^2'),... be the cycles and the taboo time of the cycles that break the taboo. Generate as many of these cycles as needed for the remaining steps. 2. Recursively for n ^ 1, generate an independent [/(") uniformly distributed on (0,1). Accept the cycle C^ if {[/<") ^ eQ(xin)-a'} occurs. Let Co, C-1, C-2, ■ ■ ■ be the subsequence of accepted cycles and generate as many of them as desired. 3. Recursively for n ^ 1, generate an independent W^ with uniform distribution on (0,1). Accept the first (R(n\ r(n>) such that the event occurs. Let (Ci,r) be the accepted pair.
Section 11. Perfect Simulation - Coupling From-the-Past 477 4. Generate independent i.i.d. copies €2,03,-■■ of the cycle C\, as many as desired, and let (Z, S) be the pair obtained by stringing out the cycles C\, C2, ■ ■ • forward from time zero and the cycles Co, C-i, C-2, ■ ■ ■ backward from time zero. 5. Generate an independent exponential r* with parameter a and put (Z',S*):=0-r.9f(Z,S). By acceptance-rejection [Theorem 6.1 in Chapter 8], (Z, S, f) is as in Theorem 10.3(a), and thus (Z*,S*,T*) is the taboo stationary version of (Z, S, T). A modification of the above algorithm works in the wide-sense case but is less efficient, since all the generated cycles of (Z,S) must be obtained in the right order from a single process to preserve the dependence structure. 11.6 Remarks The boundedness condition in the above acceptance-rejection algorithm is automatically satisfied in the case when the taboo time is the initial point of the first cycle with length that exceeds a fixed level (or the time when a cycle-length exceeds that level). But in general the boundedness is a severe restriction. However, carrying out the above algorithm with some fixed a and b yields an imperfect simulation as in Section 6.8 of Chapter 8. A more serious drawback of the acceptance-rejection algorithm is that we must know a. The coupling from-the-past method does not have these drawbacks. It seems to be a method with much potential. It can even be extended beyond finite state space as in the following domination example. Consider a discrete-time birth and death process with all birth probabilities pi, i > 0, less than or equal to some known constant p < |. The stationary distribution is well known but hard to calculate, so proceed as follows. Dominate the birth and death process by a random walk with { — 1,1} valued step-lengths, reflection at 0, and taking the upward step 1 with probability p. Run the (known) stationary version of this random walk backward from time zero until it hits the state 0. Now generate the birth and death process (coming in from the past) forward from this time as follows: let it always take the step —1 when the random walk takes the step —1; let it also take the step —1 with probability 1 — Pi/p when the random walk takes the step 1; let it take the step 1 with probability pt/p when the random walk takes the step 1. The state at time zero of this process is a realization of the desired stationary state.
An observation by King Crimson: Said the straight man to the late man Where have you been I've been here and I've been there And I've been in between.
Notes The following notes reflect the author's desire to get this book into print without further delay. Where nothing is known, nothing is claimed. Chapter 1 RANDOM VARIABLES The quantile coupling can be traced back to the fifties, and had probably been around for awhile. It is used in Hodges and Rosenblatt (1953), Harris (1955), Skorohod (1956), and Lehmann (1959). The natural term 'quantile coupling' was suggested to me by Richard Gill. The generalization of Theorem 3.1 to partially ordered Polish spaces is called Strassen's theorem. It is a special case of Theorem 11 in Strassen (1965). For outlines of proof, see Liggett (1985) and Lindvall (19926). See also Kamae, Krengel, and O'Brien (1977). Fill and Machida (1999) consider how Strassen's theorem can (and cannot) be extended. For coupling, Poisson approximation, and the so-called Stein's method, see Barbour, Hoist, and Jansson (1992). Erhardsson (1999) combines the coupling version of Stein's method with properties of regenerative processes. Theorem 7.1 is from Thorisson (19956). Theorem 8.1 is the elementary one-dimensional version of the Skorohod coupling; see Skorohod (1956) and also Skorohod (1965). The problem described in Section 10 is considered by many scientists to be the problem in modern physics. It is often referred to by the key phrases nonlocality, Bell inequality, and EPR (Einstein, Podolsky, and Rosen). The term 'impossible coupling' seemed appropriate in the context of this book. 479
480 Notes The key historical papers are the following: Einstein, Podolsky, and Rosen (1935) pinpoint that quantum physics implies nonlocality; Bell (1964,1966) establishes the Bell inequality, an inequality like (10.4) [Boole's inequality is in fact a Bell-type inequality]; and Aspect, Dalibard, and Roger (1982) report the results of an actual experiment like the one described in Section 10, confirming the predictions of quantum physics. In addition to the non- Kolmogorovian views of Kiimmerer and Maassen (1998) and Accardi (1984, 1995, 1998) and the nonlocality view of Maudlin (1994) and Gill (1998, 1999) mentioned in Section 10.6, there is, for instance, the view of Pitowski (1989) that the problem has to do with measurability. For a short and excellent common-sense survey, see Mermin (1985). Chapter 2 MARKOV CHAINS AND RANDOM WALKS It is generally agreed that the coupling idea dates back to Doeblin (1938), where the classical coupling is presented in the context of regular finite- state Markov chains. For a survey of Doeblin's life and work, see Lindvall (1991). The classical coupling appears in Harris (1955), but otherwise seems to have disappeared for a long time. It finally surfaced in the elementary books Breiman (1969) and Hoel, Port, and Stone (1972). In Pitman (1974) the classical coupling is used to establish rates of convergence for irreducible aperiodic positive recurrent Markov chains. This idea is further explored in the context of discrete-time renewal process in Kalashnikov (1977) and Lindvall (1979a). Lindvall (19796) considered the classical coupling of birth and death processes. The Ornstein coupling was introduced in Ornstein (1969), and epsilon- coupling in Lindvall (1977). Blackwell's renewal theorem (Theorem 8.1) was first proved by Black- well (1948) although special cases had been treated by Tacklind (1945) and Doob (1948). Several proofs have been proposed since then, the least complicated analytic proof probably being the one based on Choquet's theorem, see Feller (1971). The first probabilistic proof is presented in Lindvall (1977). It covered the finite-mean case, m < oo, and was based on an epsilon-coupling version of the classical coupling relying on the Hewitt- Savage 0-1 law to establish a successful epsilon-coupling. Athreya, McDonald, and Ney (1978) considered the two-sided case [which was first treated in Blackwell (1953)] and proposed establishing successful epsilon-coupling by applying the epsilon-recurrence of zero-mean nonlattice random walks to the difference walk of two independent nonlattice random walks; the problem is that this difference walk need not be nonlattice. Berbee (1979) added a geometric number of 0 step-lengths at each step to make the difference walk nonlattice. Thorisson (1987a) extended Lindvall's approach to the infinite-mean case, m = oo, and also removed the reliance on the 0-1 law (at the cost of having to treat an epsilon-transient case). This paper also suggested as an alternative approach the Ornstein-type construction
Notes 481 to obtain epsilon-recurrence of the difference walk (which is quite hard to establish in the unbounded finite-mean case, and need not even hold when m = oo). Lindvall and Rogers (1996) gave the proof an elegant finishing touch by introducing the geometric-sum idea, which allows the use of not only bounded but actually epsilon-bounded step-lengths. A relatively simple analytic consequence of Blackwell's renewal theorem is the so-called key renewal theorem [see, for instance, Feller (1971)], which is commonly used to derive the results in Theorem 10.2 on convergence in distribution. It does not, however, yield the total variation result for the total life. This result is from Thorisson (1997a), and so is Theorem 10.1. The first complete extension (in the one-sided case) of Blackwell's theorem to Markov renewal processes is in Shurenkov (1984); it is based on Fourier analysis. Alsmeyer (19946, 1997) presents two probabilistic proofs with coupling as an ingredient (and covers the two-sided case). Comment on Strong Stationary Times Consider a deck of cards. Take the card at the top and put it into the deck uniformly at random, possibly on top again and possibly below the card at the bottom. Repeat this until the card that was originally at the bottom is at the top. When this card is put into the deck uniformly at random, then the deck is uniform (is at stationarity) and independent of how many rounds (say T) this took. This example is due to Aldous and Diaconis; see Aldous (1983), Aldous and Diaconis (1986, 1987), and Diaconis (1988). They call a time like T a strong uniform time, and later a strong stationary time. The definition is as follows: T is a strong stationary time for a Markov chain Z if the future of the chain from time T onward is stationary and independent of T. Such times are, for instance, used to establish 'nonasymptotics' for certain random walks on finite groups, in particular sudden transitions in real time from nonstationarity to stationarity (so-called threshold phenomena). For strong stationary times in simulation, see Fill (1998). In Thorisson (19886) a time T is called future-independent if it is independent of the future of the chain Z from time T onward [observe that this is exactly the defining property of regeneration times in the wide sense; see Chapter 10 (Section 4.1)]. Thorisson (19886) calls a strong stationary time a future-independent stationarity time and notes the following relation between this concept and coupling: since the future of Z after time T is stationary and independent of T, we can run a time-reversed stationary version of Z from the state Zt backward to time zero to obtain a stationary chain Z' such that Z and Z' coincide from time T onward. Thus, in fact, strong stationary times are coupling times. The converse is not true. Neither is this true for future-independent times in general.
482 Notes Chapter 3 RANDOM ELEMENTS Extension techniques are just used, but rarely explained. Extension, as defined in Section 3.1, is in Berbee (1979). The special case of a product space extension is more common; see Kallenberg (1997). The transfer idea must have been around in some form for a long time, but as far as I am aware it was first presented explicitly in Section 5.1 of Thorisson (1981); see also Construction 1.1 in Thorisson (1983). The perfect term 'transfer' is from Kallenberg (1997). A version of transfer [basically Theorem 7.5 in Chapter 4 of this book] was presented in Kallenberg (1988). Transfer under weak-sense-regularity (Theorem 2.4 of Chapter 4) seems to be new. This rather obvious observation can be quite useful, since weak- sense-regularity does not imply regularity [as one might think; see Bogachev (1998) for measure theory relevant in general setting^]. The splitting idea is from Nummelin (1978) and Athreya and Ney (1978), the eye-opening papers on regeneration in Harris chains. Theorem 5.1 (the extension to general conditional splitting) seems to\be new. Berbee (1979) presents an exact coupling of random walks with spread- out step-lengths. It is based on the Ornstein idea, as the coupling in Section 6. Ney (1981) and Lindvall (1982) elaborate on the classical coupling idea to obtain rate results. See also Kalashnikov (1980) and Silvestrov (1979, 1994). Renewal theory in the spread-out case is in Smith (1954); see also Stone (1966) and Arjas, Nummelin, and Tweedie (1978). In particular, Theorem 6.2 has an associated key renewal theorem that does not require direct Rieman integrability (as does the one based on BlackwelFs renewal theorem) and holds uniformly over a class of functions. Markov renewal theory in the spread-out case is treated in Niemi and Nummelin (1986). The material in Section 9 is from Thorisson (1994a). Theorem 10.1 is from Dudley (1968); the proof is new. Theorem 10.3 is from Skorohod (1956); the proof follows Billingsley (1971). Chapter 4 STOCHASTIC PROCESSES The term 'coupling' was earlier (and still is occasionally) used for what we have called exact coupling (following Lindvall (1992)). Distributional exact coupling was introduced (under the name 'distributional coupling', or just 'coupling') in Thorisson (1981, 1983), inspired by the approach in Ney (1981). It is called 'weak coupling' in Lindvall (1992). Maximal exact coupling of discrete-time Markov chains is presented in Griffeath (1975); see also Pitman (1976). Griffeath (1978) extended the result to discrete-time Markov processes on a Polish state space, and Goldstein (1979) to general discrete-time stochastic processes on a Polish state space. Berbee (1979) proved Goldstein's result by applying Grif- feath's result to the path process (which is Markovian with a Polish state space). Thorisson (19866) established a maximal distributional exact cou-
Notes 483 pling without any restriction on the state space, and obtained the maximal nondistributional result in the Polish case as a corollary (by transfer). See also Greven (1987) and Harison and Smirnov (1990). For maximal exact coupling in continuous time, see Sverchkov and Smirnov (1990). The equivalences in Theorem 9.4 are from Goldstein (1979). Goldstein's proof contained a Hilbert space argument, which is not needed here because of the use of coupling with respect to a sub-cr-algebra. This concept was introduced in Aldous and Thorisson (1993). Michail Sverchkov pointed out to me the need to consider canonical joint measurability rather than joint measurability. Lemma 2.1 is due to Walter Rudin (personal communication). Chapter 5 SHIFT-COUPLING Shift-coupling dates back to the amazing monograph by Berbee (1979), where the link to Cesaro total variation convergence is established (see his Theorem 4.3.3). Greven (1987) considers distributional shift-coupling in the context of discrete-time Markov processes, and introduces the maximality property (4.1). Aldous and Thorisson (1993) provide the link to the invariant cr-algebra. The term 'shift-coupling' was introduced in that paper (coined by David Aldous). Thorisson (19946) considers shift-coupling in continuous-time and presents the shift-coupling inequality. The proof of the maximality result (Theorem 4.1) is a shift-coupling version of the proof of Theorem 1 in Thorisson (1996). The epsilon-coupling material in Sections 6-9 is mostly from Thorisson (19946). Theorem 7.2 is from Thorisson (1997a), and Theorem 7.3 is from Asmussen (1992). Chapter 6 MARKOV PROCESSES The exact coupling equivalences have been around for awhile. For instance, the equivalence of mixing and triviality in Theorem 2.1 is Theorem 4.1 in Orey (1971). The whole set of exact coupling equivalences follows basically from the maximal exact couplings in Griffeath (1978), Goldstein (1979), and Berbee (1979). The shift-coupling equivalences have also been around for awhile; the whole set of them follows basically from Berbee (1979), Aldous and Thorisson (1993), and Thorisson (19946). The epsilon- coupling material follows basically from Thorisson (19946). The smooth tail cr-algebra is introduced there, and smooth space-time harmonic functions in Thorisson (19976). The total variation limit claim in Theorem 4.1(6) was established for aperiodic positive recurrent Harris chains in Orey (1959), and the extension to the null recurrent case in Jamison and Orey (1967). The limit claim (7.3) was established for null recurrent Harris chains in Jain (1966); the alternative formulation (7.2) is from Thorisson (19956).
484 Notes Chapter 7 TRANSFORMATION COUPLING The group material in Section 7 is from Thorisson (1996). The generalization to semigroups is from Thorisson (2000). Georgii (1997) uses the term 'orbit coupling' for transformation coupling. He considers a semigroup acting measurably on a standard space and assumes that either the semigroup is countable normal, or a compact metric group, or composed of finitely many such building blocks. A permutation coupling (as in Section 8.2) is in Aldous and Pitman (1979). Further References on Coupling For coupling and chains with infinite connections, see Harris (1955); see also Berbee (19876). For applications in interacting particle systems, see Liggett (1985). Decoupling is used to analyze; problems involving dependent random variables as if they were (conditionally) independent; see de la Pena and Gine (1999). For the use of coupling in improving the efficiency of simulations, see Glynn and Wong (1996), and Glynn, Iglehart and Wong (1999). For shift-coupling and convergence rates in simulation, see Roberts and Rosenthal (1997). For coupling from-the-past, see the notes on perfect simulation below. For card shuffling couplings, see Aldous and Diaco- nis (1987) and Pemantle (1989). For coupling of recursive sequences, see Borovkov and Foss (1994). For coupling in branching, see Jagers (1997). For some aspects of coupling, see Schweizer and Sklar (1983), Scarsini (1989), and Cuesta and Matran (1994). The TES process approach to modelling and forecasting yields a self-coupling of the histogram of the empirical time series, see Melamed (1993). For application of coupling to the estimation of the spectral gap, see Chen (1998). For coupling and harmonic functions on manifolds, see Cranston (1991, 1993). For coupling and the strong law of large numbers for a Brownian polymer, see Cranston and Mountford (1996). Cranston and Wang (1999) establishes the equivalence of coupling and shift-coupling for a certain class of Markov processes. A surprising behaviour of random walks on groups is established by coupling in Lyons, Pemantle, and Peres (1996). For coupling of Markov and Gibbs fields, see Haggstrom (1998) and Georgii, Haggstrom, and Maes (1999). For the canonical coupling of percolation processes, see Haggstrom and Peres (1999) and HaggstrOm, Peres, and Schonmann (1999). In Evans, Kenyon, Peres, and Schulman (1999) a noisy network is coupled to a simpler network to obtain sharp bounds in a reconstruction problem that arose independently in computer science, mathematical biology, and statistical physics. In Dembo, Peres, Rosen, and Zeitouni (1999) coupling of random walks and Brownian motion is used to prove a conjecture of Erdos and Taylor from 1960 on simple random walk. Coupling is used in the textbooks Grimmett and Stirzaker (1992) and Durrett (1991). For applications of coupling in various fields, see the collection of articles edited by Kalashnikov
Notes 485 and Thorisson (1994). Lindvall (19926) has many topics and references not mentioned here; see also his papers in the list of references. The wealth of topics in the monograph by Berbee (1979) never ceases to surprise. Chapter 8 STATIONARITY, THE PALM DUALITIES Palm theory dates back to the early forties. It was pioneered by the Swedish engineer Conny Palm, who worked at the Royal Institute of Technology and at the Swedish Telephone Company. The point-at-zero duality in Theorem 4.1 is a process version of the standard Palm duality; see Palm (1943); Kinchine (1955); Ryll-Nardzewski (1961); Neveu (1976); Kallenberg (1986); Matthes, Kerstan, and Mecke (1978); Franken, Konig, Arndt, and Schmidt (1981); Rolski (1981); Daley and Vere-Jones (1988); Brandt, Franken and Lisek (1990); Baccelli and Bre- maud (1994); and Brandt and Last (1995). This duality is usually obtained in one step through a single formula (see Section 4.7). The present two-step approach (Theorem 4.1) goes back to Thorisson (1981); see also Thorisson (1992, 1995a). For a similar two-step approach in a canonical point process setting, see Nieuwenhuis (1989a, 19896, 1994). Cycle-stationary processes are also called 'syncronous'. The length-biasing is known as the inspection (or waiting time) paradox. The uniform position of the origin in its interval under stationarity (Theorem 3.1) has been known [for an early reference, see McFadden (1962)], but has not been highlighted. Here it is (together with the length-biasing) the basic fact on which the theory is built. The randomized-origin duality in Theorem 8.1 is not as well known; see Nawrotzki (1978); Glynn and Sigman (1992); and Nieuwenhuis (1994). The path of the author from the point-at-zero duality to the randomized- origin duality went through work on shift-coupling. The resulting two-step approach (Theorem 8.1), and the Cesaro limit result (Theorem 9.1) and shift-coupling result (Theorem 9.2), are presented in Thorisson (1995a). The Cesaro limit result is from Glynn and Sigman (1992). In the past there has been some confusion regarding these two dualities and their interpretations. It seems that the distinction between them was first noted by Nawrotzki (1978). When Palm introduced his theory in 1943 he was aiming at the randomized-origin interpretation, see his Chapter 2. However, the point-at-zero interpretation became dominant, which is natural in a way since it is the correct interpretation of the standard Palm duality. But most work in this field has been done assuming ergodicity, in which case the two dualities coincide (see Section 10.1). For more discussion on history and applications, see Sigman (1995), where the shift-coupling result is used regularly throughout the text. This chapter (Chapter 8) is based on Thorisson (1995a), except the perfect simulation, which is from Asmussen, Glynn, and Thorisson (1992). Acceptance-rejection dates back to von Neumann (1951).
486 Notes Chapter 9 PALM DUALITIES IN HIGHER DIMENSIONS The material in this chapter is from Thorisson (1999). For more information on Voronoi cells, see Okabe, Boots, and Sugihara (1992). For the point-at-zero Palm version of a stationary point process when d > 1, see Matthes, Kerstan, and Mecke (1978), Daley and Vere-Jones (1988), and Stoyan, Kendall, and Mecke (1987). I am not aware of any references on the randomized-origin Palm version for d > 1. Neither am I aware of references on full-fledged Palm dualities when d > 1. The one-to-one correspondence between a stationary point process and its Palm version is a rather unsatisfactory duality because the Palm version is derived from the stationary point process, that is, the Palm version does not have a defining property without reference to its stationary dual. Here (Theorems 7.1 and 8.1) we present Palm dualities between two classes of processes of equal status: stationary and point-stationary. j The only references that the authorhas found where the point-stationarity problem is mentioned are Mandelbrot (1983) and a paper by Kagan and Vere-Jones (1988). The following is taken from that paper: Mandelbrot (1983) [suggests] that point process models for self-similar behaviour should be sought not within the class of homogeneous processes [called stationary in this book] but within the class of processes for which the behaviour relative to a given point of the process is independent of the point selected as origin. ... What hampers us in pursuing this discussion, is that we are not aware of a well established theory for such Palm-stationary point processes [called point-stationary in this book]. Chapter 9 presents such a theory. Kagan and Vere-Jones (1988) continue: Further examples of Palm-stationary processes suggested by Mandelbrot, such as the Levy dust model and zeros of Brownian motion, have a very complex point set structure, including finite accumulation points, and cannot be modelled within the standard point process framework. The proposed definition at the end of Chapter 9 suggests a solution to this general point-stationarity problem. Chapter 10 (Sections 2-4) REGENERATION Regenerative processes (called classical regenerative here) were introduced in Smith (1955). The i.i.d. cycle characterization is from Smith (1958). He uses the term 'tour' for cycle. Regeneration in the wide sense was also introduced in Smith (1955); he uses the term 'equilibrium process'. Classical regeneration became heavily used, but wide-sense regeneration passed more or less unnoticed, to be
Notes 487 rediscovered only at the end of the seventies independently by Asmussen and the author. Thorisson (1981, 1983) uses the term 'time-homogeneous regeneration' [treating wide-sense regeneration as a special case of time- inhomogenous wide-sense regeneration which is called simply 'regeneration' in Thorisson (1981, 1983)]. Asmussen (1987) uses the term 'regeneration' for wide-sense regeneration. Wide-sense regeneration turned out to be the appropriate characterization of the regeneration that Nummelin (1978) and Athreya and Ney (1978) had found in Harris chains. The clumsy term 'wide-sense regenerative' is simply the phrase that is commonly used to indicate this type of regeneration. The excellent term 'lag-? regenerative" was suggested to me by Peter Glynn. The related phenomenon of a renovating event was discovered in the multi-server queue by Akhmarov and Leont'eva (1976), and the general concept for recursive sequences was formulated in Borovkov (1978). See also Borovkov (1984), Borovkov and Foss (1992, 1994), and Baccelli and Bremaud (1994). A stationary version of a classical regenerative process with finite-mean cycle-lengths was constructed in Miller (1972); his method is basically the simulation algorithm in Section 6.7 of Chapter 8. Here we use the length- biasing and uniform-shifting from Thorisson (1981): Theorems 2.1, 2.2, 3.1, and 4.1 are Propositions 2.1 and 3.1 in that work. The key coupling results, Theorems 3.2, 4.2 and 5.2, are from Thorisson (1981, 1983). Theorems 2.3, 2.4, 3.4, and 4.4 have been around for years scattered throughout the literature; the limit parts often established by an application of the key renewal theorems. The result (3.27) on smooth asymptotic stationarity in Theorems 3.3 and 4.3 was established in Glynn and Iglehart (1989). The result on Cesaro asymptotic stationarity in Section 2.7 was established in Glynn and Sigman (1992). The material in Section 4.4 is from Sigman, Thorisson, and Wolff (1994). Theorem 4.6 is an elaboration on the approach in Nummelin (1978) and Athreya and Ney (1978). These papers considered discrete-time Harris chains with I = 1 and established classical regeneration. The extension to / ^ 2 is not trivial, and was not well understood for some time. However, the one-dependence of the cycles is in Glynn (1982), called one-dependent regeneration; and the wide-sense regeneration is in Asmussen (1987), the lag-? regeneration is also noted there; see also Asmussen and Thorisson (1987) and Sigman (1990). Glynn (1982) considers simulation of Harris chains. This work also uses the one-dependence of the cycles to establish the strong law of large numbers and the central limit theorem in the Harris chain context. Glynn (1994) extends these ideas to continuous-time Harris processes, and Andradottir, Calvin, and Glynn (1995) shows how to use the splitting idea to increase the frequency of regeneration points in a given simulated realization. The fact that the renovation in multi-server queues is actually lag-? regeneration for the continuous-time process (Section 4.6) was realized in a
488 Notes discussion with S0ren Asmussen, Serguei Foss, and Vladimir Kalashnikov at the Second Symposium on Queueing Theory and Related Topics in Poland, January 1990. This resulted in two papers, Foss and Kalashnikov (1991), and Asmussen and Foss (1993). Asmussen (1987) treats regenerative processes and Harris chains, highlighting them as basic mathematical tools in applied probability. See Sig- man and Wolff (1993) for a review of regenerative processes with many references. See Kalashnikov (1994) for topics in regenerative processes. See Nummelin (1984) and Meyn and Tweedie (1993) for more aspects of general Markov chain theory. For other regenerative phenomena such as regenerative random sets, see Kingman (1972) and Kallenberg (1997) and the references therein. These phenomena are mo/e general than classical regeneration, and orthogonal to wide-sense regeneration and renovation. However, all these concepts are still time-homogeneous. Chapter 10 (Sections 5-10) REGENERATION Time-inhomogeneous regeneration was introduced in Thorisson (1981,1983) and the material in Sections 5-7 is (mostly) from there. The coupling approach (Section 6) and the study of the coupling time (Section 7) were inspired by the treatment of renewal processes in Ney (1981) and Lind- vall (1982). The function class A is in Stone and Wainger (1967) and was considered in Ney (1981). The concave functions and the Orlicz space functions (growing slower than some power function) are generalizations of the power functions considered in Lindvall (1982). The convergence results in Section 7 were applied in queueing theory in Thorisson (1985a, 6, c). Conditions related to the stochastic domination condition (5.8) can be found in Alsmeyer (1991, 1994a, 1995). The asymptotics from-the-past in Section 8 [and the material in Sections 5.7 and 5.8] are from Thorisson (1988a); see also Thorisson (1985d, 1986a, 1990). Coupling from-the-past was introduced in these papers (under the name 'backward-successful couplings') to establish the existence of a two- sided (possibly nonstationary) limit process coming in from-the-past (called 'backward limit'). The idea of asymptotics from-the-past seemed to be the natural way to obtain limit results in a time-inhomogeneous context. Asymptotics from-the-past date back to (surprise?) Kolmogorov (1936). Kolmogorov studied finite-state time-inhomogeneous Markov chains with transition matrices p(m>") from time m to time n and showed (by simply sending m backward to minus infinity along a subsequence) that there is a probability vector solution (7r(n); —oo < n < oo) to the equations v(n) =7r(m)p(m,n)) -oo < m < n< oo; thus there always exists a two-sided version of a time-inhomogeneous finite- state Markov chain. Kolmogorov's results were elaborated on by Black- well (1945); see also Cohn (1974, 1982), Seneta (1981), Sonin (1987), and
Notes 489 Brandt, Lisek, and Nerman (1990). Asymptotics from-the-past showed up in a queueing context in Loynes (1962). Coupling from-the-past can be found in the renovation literature; see Borovkov and Foss (1994). The beautiful coupling from-the-past simulation algorithm (Section 11) was invented by Propp and Wilson (1996). Taboo regeneration (Section 9) and taboo stationarity (Section 10) are from Glynn and Thorisson (19996), where renewal theory is used to derive the limit results; see also Glynn and Thorisson (1999a) on Markov processes. The approach in this book (based on time-inhomogeneous regeneration and asymptotics from the-past) has not been published elsewhere. Taboo stationarity generalizes so-called quasi-stationarity of Markov chains. This topic was introduced in the sixties; see Seneta and Vere- Jones (1966); Tweedie (1974); Arjas and Nummelin (1976); Nummelin and Tweedie (1978); Nummelin (1984); and Ferrari, Kesten, and Martinez (1996). The excellent descriptive term 'taboo' is in Chung (1967). Chapter 10 (Section 11) Perfect Simulation The now accepted term 'perfect simulation' is coined by Kendall (1998); 'exact sampling' is also used. The earliest positive result on perfect simulation seems to be the method presented in Chapter 8 (Section 6) and Chapter 9 (Section 7), which is from Asmussen, Glynn, and Thorisson (1992) [see also the preliminary abstract Thorisson (1987-6)]. This paper considered the general question of when perfect simulation (called stationarity detection) is possible and when not. It also presents a perfect simulation algorithm for finite-state Markov chains; but this algorithm was hampered by tremendous use of computer time. The powerful idea of using coupling from-the-past for perfect simulation was introduced in the highly influential paper by Propp and Wilson (1996). The concluding example in Section 11.6 was communicated by Serguei Foss; see Foss and Tweedie (1999). A related domination idea was presented in Kendall (1998) in the context of spatial point processes. For more of the subsequential work, see Murdoch and Green (1998), Hag- gstrom, Lieshout, and M0ller (1999), Haggstrom and Nelander (1999), Wilson (1998, 1999), and Fill (1998). For further references, see David Wilson's homepage (dimacs. rutgers . edu/~dbwilson/exact). The example in the second paragraph of the introduction to Section 5 was suggested to me by Peter Glynn as a situation where coupling from- the-past might be useful for simulation in a time-inhomogeneous context. * * *
490 Notes The poem at the front of the book is from Solarljod, the ethereal Sun Poems of 13th century Iceland. It has been set to music by Jon Nordal in Ottusongvar a vori (Matins in Spring, 1993). The following is an English interpretation by Alan Boucher: From the South I saw the Sun-Hart step; leading him, two together; light his hooves on the hills below, his horns reached high to the heavens. The words at the end of Chapter 2 are the opening lines of Strawberry Fields Forever by the Beatles (1967). The words at the end of Chapter 10 are the opening lines of I Talk to the Wind from the album IN THE COURT OF THE CRIMSON KING, AN OBSERVATION BY KING CRIMSON (1969). They are followed by: I talk to the wind My words are all carried away I talk to the wind The wind does not hear The wind cannot hear.
References ACCARDI, L. [1984] Some trends and problems in quantum probability. Springer Lecture Notes in Mathematics 1055, 1-19. [1995] Can mathematics help solving the interpretational problems of quantum theory? // Nuovo Cimento 110 B, 685-721. [1998] On the EPR paradox and the Bell inequality. Preprint 352, Centra V. Volterra, Universita degli Studi di Roma Tor Vergata. AKHMAROV, I. and LEONT'EVA, N. [1976] Conditions for the convergence to the limit laws and law of large numbers for queueing systems. Theo. Probab. Appl. 21, 559-570. ALDOUS, D. [1983] Random walks on finite groups and rapidlymixing Markov chains. Springer Lecture Notes in Mathematics 986. ALDOUS, D. and DIACONIS, P. [1986] Shuffling cards and stopping times. Amer. Math. Monthly 93, 333-348. [1987] Strong uniform times and finite random walks. Adv. Appl. Math. 8, 69-97. ALDOUS, D. and PITMAN, J.W. [1979] On the zero-one law for exchangeable events. Ann. Probab. 7, 704-723. 491
492 References ALDOUS, D. and THORISSON, H. [1993] Shift-coupling. Stock. Proc. Appl. 44, 1-14. ALSMEYER, G. [1991] Random walks with stochastically bounded increments: Foundations and characterization results. Result, der Math. 19, 22-45. [1994a] Random walks with stochastically bounded increments: Renewal theory via Fourier Analysis. Yokohama Math. J. 42, 1-21. [19946] On the Markov renewal theorem. Stoch. Proc. Appl. 50, 37-56. [1995] Random walks with stochastically bounded increments: Renewal theory. Math. Nachr. 175, 13-31. [1997] The Markov renewal theorem and related results. Markov Proc. Rel. Fields 3, 103-127. ANDRADOTTIR, S., CALVIN, J.M., and GLYNN, P.W. [1995] Accelerated regeneration for Markov chain simulations. Probab. Eng. Inf. Sc. 9, 497-523. ARJAS, E. and NUMMELIN, E. [1976] A direct construction of the R-invariant measure for a Markov chain on a general state space. Ann. Probab. 4, 674-679. ARJAS, E., NUMMELIN, E. and TWEEDIE, R.L. [1978] Uniform limit theorems for non-singular renewal and Markov renewal processes. J. Appl. Probab. 15, 112-125. ASH, R.B. [1972] Real Analysis and Probability. Academic Press, New York. ASMUSSEN, S. [1987] Applied Probability and Queues. Wiley, New York. [1992] On coupling and weak convergence to stationarity. Ann. Appl. Probab. 2, 739-751. ASMUSSEN, S. and FOSS, S.G. [1993] Renovation, regeneration, and coupling in multiple-server queues in continuous time. Frontiers in Pure and Appl. Probab. 1, 1-6. ASMUSSEN, S., GLYNN, RW. and THORISSON, H. [1992] Stationarity detection in the initial transient problem. ACM Trans. Modelling Comput. Simulation 2, 130-157. ASMUSSEN, S. and THORISSON, H. [1987] A Markov chain approach to periodic queues. J. Appl. Probab. 24, 215-225.
References 493 ASPECT, A, DALIBARD, J., and ROGER, G. [1982] Experimental test of Bell's inequalities using time-varying analyzers. Physical Review Letters 49, 1804-1807. ATHREYA, K.B. and NEY, P. [1978] A new approach to the limit theory of recurrent Markov chains. Trans. Amer. Math. Soc. 245, 493-501. ATHREYA, K.B., MCDONALD, D., and NEY, P. [1978] Coupling and the renewal theorem. Amer. Math. Monthly 85, 809-814. BACCELLI, F. and BREMAUD, P. [1994] Elements of Queueing Theory. Springer, Berlin. BARBOUR, A., HOLST, L., and JANSON, S. [1992] Poisson Approximation. Oxford University Press, Oxford. BARBOUR, A., LINDVALL, T., and ROGERS, L.C.G. [1991] Stochastic ordering of order statistics. J. Appl. Prob. 28, 278-286. BELL, J.S. [1964] On the Einstein Podolsky Rosen Paradox. Physics 1:3, 195-200. [1966] On the problem of hidden variables in quantum mechanics. Reviews of Modern Physics 38:3, 447-453. BERBEE, H.C.P. [1979] Random walks with stationary increments and renewal theory. Math. Centre Tract 112 (Mathematisch Centrum, Amsterdam). [1986] Periodicity and absolute regularity. Israel J. Math. 55, 289-304. [1987a] Convergence rates in the strong law for bounded mixing sequences. Probab. Th. Rel. Fields 74, 255-270. [19876] Chains with infinite connections: uniqueness and Markov representation. Probab. Th. Rel. Fields 76, 243-253. BILLINGSLEY, P. [1971] Weak convergence of Measures: Applications in Probability. SIAM, Philadelphia. [1986] Probability and Measure. 2nd ed. Wiley, New York. BLACKWELL, D. [1945] Finite non-homogeneous chains. Ann. Math. 46, 594-599. [1948] A renewal theorem. Duke Math. J. 15, 145-150. [1953] Extension of a renewal theorem. Pacific J. Math. 3, 315-320.
494 References BOGACHEV, V. [1998] Measures on topological spaces. J. Math. Sci. (New York) 91, 3033-3156. BOROVKOV, A.A. [1978] Theorems of ergodicity and stability for one class of stochastic equations. Theo. Probab. Appl. 23, 241-262. [1984] Asymptotic Methods in Queueing Theory. Wiley, New York. BOROVKOV, A.A. and FOSS, S.G. [1992] Stochastically recursive sequences and their generalizations. Siberian Advances in Mathematics 2, 16-81. [1994] Two ergodicity criteria for stochastically recursive sequences. Acta Applicandae Mathematicae 34, 125-134. \ BOURBAKI, N. [1948] Elements de Mathematiques. Topologie Generale. Chapitre IX. Hermann, Paris. [1951] Elements de Mathematiques. Topologie Generale. Chapitre III-IV. Hermann, Paris. BRANDT, A., FRANKEN, P. and LISEK, B. [1990] Stationary Stochastic Models. Wiley. BRANDT, A. and LAST, G. [1995] Marked Point Processes on the Real Line. Springer, Berlin. BRANDT, A., LISEK, B., and NERMAN, 0. [1990] On stationary Markov chains and independent random variables. Stoch. Proc. Appl. 34, 19-24. BREIMAN, L. [1969] Probability and Stochastic Processes. Houghton Mifflin, Boston. CHEN, M.F. [1997] Trilogy of couplings - new variational formula of spectral gap. Probability Towards 2000, Springer Lecture Notes in Statistics 128,123-136. CHEN, M.F. and LI, S.F. [1989] Coupling methods for multidimensional diffusion processes. Ann. Probab. 17, 151-177. CHUNG, K.L. [1967] Markov Chains with Stationary Transition Probabilities. 2nd ed. Springer, Berlin.
References 495 CINLAR, E. [1975] Introduction to Stochastic Processes. Prentice-Hall, Englewood Cliffs, New Jersey. COHN, H. [1974] On the tail events of a Markov chain. Z. Wahrscheinlichkeitsth. 29, 65-72. [1982] On a class of non-homogeneous Markov chains. Math. Proc. Cam. Phil. Soc. 92, 527-534. CRANSTON, M. [1991] Gradient estimates on manifolds using coupling. J. Functional Analysis 99, 110-124. [1993] A probabilistic approach to Martin boundaries for manifolds with ends. Prob. Theo. Rel. Fields 96, 319-334. CRANSTON, M. and MOUNTFORD, T.S. [1996] The Strong Law of Large Numbers for a Brownian polymer. Ann. Probab. 24, 1300-1323. CRANSTON, M. and WANG, F. [1999] Equivalence of coupling and shift coupling. (Submitted). CUESTA, J.A. and MATRAN, C. [1994] Stochastic convergence through Skorohod representation theorems and Wasserstein distances. First Intern. Conf. on Stoch. Geometry, Convex Bodies and Empirical Measures. Suppl. Ren- diconti del Circolo Matematico Palermo, Serie II, 35, 89-113. DALEY, D.J. and VERE-JONES, D. [1988] An Introduction to the Theory of Point Processes. Springer, New York. DE LA PENA, V.H. and GINE, E. [1999] Decoupling. Springer, New York. DEMBO, A., PERES, Y., ROSEN, J., and ZEITOUNI, O. [1999] Thick points for planar Brownian motion and the Erdos-Taylor conjecture on random walk. (Preprint) DIACONIS, P. [1988] Group Representation in Probability and Statistics. IMS Lecture Notes - Monograph Series 11, IMS, Hay ward. DOEBLIN, W. [1938] Expose de la theorie des chaines simple constantes de Markov a un nombre fini d'etats. Rev. Math. Union Interbalkan. 2, 77-105.
496 References DOOB, J.L. [1948] Renewal theory from the point of view of the theory of probability. Trans. Amer. Math. Soc. 63, 422-438. DUDLEY, R.M. [1968] Distances of probability measures and random variables. Ann. Math. Statist. 39, 1563-1572. DURRETT, R. [1991] Probability: Theory and Examples. Brooks/Cole, Pacific Grove, California. EINSTEIN, A., PODOLSKY, B., and ROSENj'N. [1935] Can quantum-mechanical description of physical reality be considered complete? Physical Review 47^,777-780. ERHARDSSON, T. [1999] Compound Poisson approximation for Markov chains using Stein's method. Ann. Probab. 27, 565-596. ETHIER, S.N. and KURTZ, T.G. [1986] Markov Processes. Wiley, New York. EVANS, W., KENYON, C, PERES, Y., and SCHULMAN, L. [1999] Broadcasting on trees and the Ising model. Ann. Appl. Probab. (to appear). FELLER, W. [1971] An Introduction to Probability Theory and Its Applications, Vol. 2. 2nd ed. Wiley, New York. FERRARI, P., KESTEN, H., and MARTINEZ, S. [1996] i?-positivity, quasi-stationary distributions, and ratio limit theorems for a class of probabilistic automata. Ann. Appl. Probab. 6, 577-616. FILL, J.A. [1998] An interruptible algorithm for perfect sampling via Markov chains. Ann. Appl. Probab. 8, 131-162. FILL, J.A. and MACHIDA, M. [1999] Stochastic monotonicity and realizable monotonicity. Ann. Appl. Probab. (to appear). FOSS, S.G. and KALASHNIKOV, V. [1991] Regeneration and renovation in queues. Queueing Systems Theory Appl. 8, 211-224.
References 497 FOSS, S.G. and TWEEDIE, R. [1999] Unifying approaches to backward coupling. (In preparation). FRANKEN, P., KONIG, D., ARNDT, U. and SCHMIDT, V. [1981] Queues and Point Processes. Akademie-Verlag. GARSIA, A.M. [1973] On a convex function inequality for martingales. Ann. Probab. 1, 171-174. GEORGII, H.-O. [1997] Orbit coupling. Ann. Inst. H. Poincare, Prob. et Statist. 33 253- 268. GEORGII, H.-O., HAGGSTROM, 0., and MAES, C. [1999] The random geometry of equilibrium phases. Phase Transitions and Critical Phenomena (C. Domb and J.L. Lebowitz, editors), Academic Press, London (to appear). GILL, R.D. [1998] Critique of 'Elements of Quantum Probability'. Quantum Probability Communications 10, 351-361. [1999] Quantum probability and the impossible coupling (in preparation) . www.math.uu.nl/people/gill/Preprints/impossible. ps.gz GLYNN, P.W. [1982] Simulation Output Analysis for General State Space Markov Chains. Dissertation. Department of Operations Research, Stanford University. [1994] Some topics in regenerative steady-state simulation. Acta Appli- candae Mathematicae 34, 225-236. GLYNN, P.W. and IGLEHART, D.L. [1989] Smoothed limit theorems for equilibrium processes. Probability, Statistics and Mathematics. Papers in Honor of S. Karlin. Academic Press, New York, 89-102. GLYNN, P.W., IGLEHART, D. L., and WONG, E.W. [1999] Transient simulation via empirically based coupling. Probab. Eng. Inf. Sc. 13, 147-167. GLYNN, P.W. and SIGMAN, K. [1992] Uniform Cesaro limit theorems for synchronous processes with applications to queues. Stoch. Proc. Appl. 40, 29-43.
498 References GLYNN, P.W. and THORISSON, H. [1999a] Two-sided taboo limits for Markov processes and associated perfect simulation. (Submitted) [19996] Taboo stationarity and limit theory for taboo regenerative processes. (Preprint) GLYNN, P.W. and WONG, E.W. [1996] Efficient simulation via coupling. Probab. Eng. Inf. Sc. 10, 165- 186. GOLDSTEIN, S. [1979] Maximal coupling. Z. Wahr-scheinliefikeitsth. 46, 193-204. GREVEN, A. / [1987] Coupling of Markov chains and randomized stopping times. Part I and II. Probab. Th. Rel. Fields 75, 195-212 and 431-458. GRIFFEATH, D. [1975] A maximal coupling for Markov chains. Z. Wahrscheinlichkeit- sth. 31, 95-106. [1978] Coupling methods for Markov processes. Studies in Probability and Ergodic Theory. Advances in Mathematics. Supplementary Studies 2. GRIMMETT, G.R. and STIRZAKER, D.R. [1992] Probability and Random Processes. 2nd ed. Oxford University Press, Oxford. HAGGSTROM, 0. [1998] Random-cluster representations in the study of phase transitions. Markov Proc. Rel. Fields 4, 275-321. HAGGSTROM, 0., VAN LIESHOUT, M.N.M., and M0LLER, J. [1999] Characterization results and Markov chain Monte Carlo algorithms including exact simulation for some spatial point processes. Bernoulli 5, 641-658. HAGGSTROM, O. and NELANDER, K. [1999] On exact simulation of Markov random fields using coupling from the past. Scand. J. Stat. 26, 395-411. HAGGSTROM, 0. and PERES, Y. [1999] Monotonicity of uniqueness for percolation on Cayley graphs: all infinite clusters are born simultaneously. Probab. Theo. Rel. Fields 113, 273-285.
References 499 HAGGSTROM, 0., PERES, Y., and SCHONMANN, R. [1999] Percolation on transitive graphs as a coalescent process: relentless merging followed by simultaneous uniqueness. Perplexing Probability Problems: Papers in Honor of Harry Kesten, Birk- hauser, 69-90. HALMOS, P.R. [1950] Measure Theory. Van Nostrand, New York. HARISON, V. and SMIRNOV, S.N. [1990] Jonction maximale en distribution dans le cas markovien. Probab. Th. Rel. Fields 84, 491-503. HARRIS, T.E. [1955] On chains of infinite order. Pacific J. Math. 5, 713-724. HODGES, J.L. and ROSENBLATT, M. [1953] Recurrence-time moments in random walks. Pacific J. Math. 3, 127-136. HOEL, P.G., PORT, S.C., and STONE, C.J. [1972] Introduction to Stochastic Processes. Houghton Mifflin, Boston. JAGERS, P. [1997] Coupling and Population Dependence in Branching Processes. Ann. Appl. Probab. 7, 281-298. JAIN, N.C. [1966] Some limit theorems for the general Markov Process. Z. Wahr- scheinlichkeitsth. verw. Geb. 6, 206-223. JAMISON, B. and OREY, S. [1967] Markov chains recurrent in the sense of Harris. Z. Wahrschein- lichkeitsth. verw. Geb. 8, 41-48. KAGAN, Y.Y. and VERE-JONES, D. [1988] Statistical Models of Earthquake Occurrence. Springer Lecture Notes in Statistics 114, 398-425. KALASHNIKOV, V. [1977] A uniform estimate of the rate of convergence in the discrete time renewal theorem. Theo. Probab. Appl. 22, 399-403. [1980] Estimation of duration of transition regime for complex stochastic systems. Trans. Seminar, VNIISI, Moscow, 63-71 (in Russian). [1994] Topics on Regenerative Processes. CRC Press, Boca Raton.
500 References KALASHNIKOV, V. and THORISSON, H. (editors) [1994] Applications of Coupling and Regeneration. Acta Applicandae Mathematicae 34 (Special Issue). KALLENBERG, 0. [1986] Random Measures. 4th ed. Academie-Verlag and AcademicPress. Berlin and London. [1988] Spreading and predictable sampling in exchangeable sequences and processes. Ann. Probab. 16, 508-534. [1997] Foundations of Modern Probability. Springer, New York. / KAMAE, T., KRENGEL, U., and O'BRIEN/G.L. [1977] Stochastic inequalities on partially ordered spaces. Ann. Probab. 5, 899-912. I KARLIN, S. and TAYLOR, H.M. [1975] A First Course in Stochastic Processes. 2nd ed. Academic Press, New York. KENDALL, W.S. [1998] Perfect simulation for the area-interaction point process. Probability Towards 2000, Springer Lecture Notes in Statistics 128, 218-234. KINCHINE, Y.A. [1960] Mathematical Methods in the Theory of Queues. Griffin, London. (Russian ed. 1955.) KINGMAN, J.F.C. [1972] Regenerative Phenomena. Wiley, New York. KOLMOGOROV, A. [1936] Zur Theorie der Markoffschen Ketten. Math. Ann. 112,155-160. KRASNOSELSKII, M.A. and RUTICKII, Ya.B. [1961] Convex Functions and Orlicz Spaces. Noordhoff, Groningen. KUMMERER, B. and MAASSEN, H. [1998] Elements of quantum probability. Quantum Probability Communications 10, 73-100. LEHMANN, E.L. [1959] Testing Statistical Hypothesis. Wiley, New York. LIGGETT, T.M. [1985] Interacting Particle Systems. Springer, New York.
References 501 LINDVALL, T. [1977] A probabilistic proof of Blackwell's renewal theorem. Ann. Prob. 5, 57-70. [1979a] On coupling of discrete renewal processes. Z. Wahrscheinlichkeit- sth. 48, 57-70. [19796] A note on coupling of birth and death processes. J. Appl. Probab. 16,505-512. [1982] On coupling of continuous-time renewal processes. J. Appl. Prob. 19, 82-89. [1983] On coupling of diffusion processes. J. Appl. Probab. 20, 82-93. [1986] On coupling of renewal processes with use of failure rates. Stoch. Proc. Appl. 22, 1-15. [1988] Ergodicity and inequalities in a class of point processes. Stoch. Proc. Appl. 30, 121-131. [1991] W. Doeblin 1915-1940. Ann. Probab. 19, 929-934. [1992a] A simple coupling of renewal processes. Adv. Appl. Probab. 24, 1010-1011. [19926] Lectures on the Coupling Method. Wiley, New York. [1997] Stochastic monotonicities in Jackson queueing networks. Probab. Eng. Inf. Sc. 11, 1-9. [1999] On Strassen's theorem on stochastic domination. Elect. Comm. in Probab. 4, 51-59. [2000] On simulation of stochastically ordered life length variables. Prob. Eng. Inf. Sc. (to appear). LINDVALL, T. and ROGERS, L.C.G. [1986] Coupling of multidimensional diffusions by reflection. Ann. Prob. 14, 860-872. [1996] On coupling of random walks and renewal processes. J. Appl. Probab. 33, 122-126. LOYNES, R.M. [1962] The stability of a queue with non-independent inter-arrival and service time. Proc. Camb. Phil. Soc. 58, 494-520. LYONS, R., PEMANTLE, R., and PERES, Y. [1996] Random walks on the lamplighter group. Ann. Probab. 24, 1993- 2006. MANDELBROT, B.B. [1983] The Fractal Geometry of Nature. W. H. Freeman, San Francisco.
502 References MATTHES, K., KERSTAN, J., and MECKE J. [1978] Infinitely Divisible Point Processes. Wiley, Chichester. MAUDLIN, T. [1994] Quantum Non-locality and Relativity. Blackwell, Oxford. MCFADDEN, J.A. [1962] On the lengths of intervals in a stationary point process. Journ. Royal Stat. Soc. Ser. B 24, 364-382. MELAMED, B. [1993] An Overview of TES Processes and Modeling Methodology. Performance Evaluation of Computer and Communications Systems, Springer Lecture Notes in Cdmputer Science, 359-393. MERMIN, N.D. [1985] Is the moon there when nobody looks? Reality and quantum theory. Physics Today, 38-47. MEYN, S.P. and TWEEDIE, R.L. [1993] Markov Chains and Stochastic Stability. Springer, New York. MILLER, D.R. [1972] Existence of limits in regenerative processes. Ann. Math. Statist. 43,1275-1282. MONTGOMERY, D. and ZIPPIN, L. [1955] Topological Transformation Groups. Wiley (Interscience), New York. MURDOCH, D.J. and GREEN, P.J. [1998] Exact sampling from a continuous state space. Scand. J. Stat. 25, 483-502. NAWROTZKI, K. [1978] Einige Bemerkung zur Verwendung der Palmschen Verteilung in der Bedienungstheorie. Mathematische Operationsforschung und Statistik, Series Optimization 9, 241-253. NEVEU, J. [1975] Discrete-Parameter Martingales. North-Holland, Amsterdam. t [1976] Processus ponctuels. Springer Lecture Notes in Mathematics 598,249-445. NEY, P. [1981] A refinement of the coupling method in renewal theory. Stoch. Proc. Appl. 11, 11-26.
References 503 NIEMI, S. and NUMMELIN, E. [1986] On non-singular renewal kernels with an application to a semigroup of transition kernels. Stock. Proc. Appl. 22, 177-202. NIEUWENHUIS, G. [1989a] Asymptotics for Point Processes and General Linear Processes. Thesis. Catholic University of Nijmegen. [19896] Equivalence of functional limit theorems for stationary point processes and their Palm distributions. Probab. Th. Rel. Fields 81, 593-608. [1994] Bridging the gap between a stationary point process and its Palm distribution. Statistica Neerlandica 48, 37-62. NUMMELIN, E. [1978] A splitting technique for Harris recurrent Markov chains. Z. Wahrscheinlichkeitsth. 43, 309-318. [1984] General Irreducible Markov Chains and Non-Negative Operators. Cambridge University Press, Cambridge. NUMMELIN, E. and TWEEDIE, R.L. [1978] Geometric ergodicity and .R-positivity for general Markov chains. Ann. Probab. 6, 404-420. OKABE, A., BOOTS, B., and SUGIHARA, K. [1992] Spatial Tessellations - Concepts and Applications of Voronoi Diagrams. Wiley, New York. OREY, S. [1959] Recurrent Markov Chains. Pacific J. Math. 9, 805-827. [1971] Limit Theorems for Markov Chain Transition Probabilities. Van Nostrand, London. ORNSTEIN, D. [1969] Random walks I, II. Trans. Am. Math. Soc. 138,1-42 and 45-60. PALM, C. [1943] Intensitatsshwankungen in Fernsprechverkehr. Ericssons Technics 44, 1-189. (English translation: (1988), Intensity Variations in Telephone Traffic. North-Holland Studies in Telecommunication 10. Elsevier.) PEMANTLE, R. [1989] Randomization time for the overhand shuffle. Journ. Theo. Prob. 2, No. 1.
504 References PITMAN, J.W. [1974] Uniform rates of convergence for Markov chain transition probabilities. Z. Wahrscheinlichkeitsth. 29, 193-227. [1976] On coupling of Markov chains. Z. Wahrscheinlichkeitsth. 35, 315-322. PITOWSKI, I. [1989] Quantum probability, quantum logic. Springer Lecture Notes in Physics. PROPP, J.G. and WILSON, D.B. / [1996] Exact sampling with coupled Markov chains and applications to statistical mechanics. Random'Structures and Algorithms 9, 223-252. i ROBERTS, G.O. and ROSENTHAL, J.S. [1997] Shift-coupling and convergence rates of ergodic averages. Stoch. Models 13, 147-165. ROLSKI, T. [1981] Stationary Random Processes Associated with Point Processes. Lecture Notes in Statistics 5. Springer. RYLL-NARDZEWSKI, C. [1961] Remarks on processes of calls. Proc. 4th Berkeley Symp. Math. Stat. Probab. 2, 455-465. SCARSINI, M. [1989] Copulae of probability measures on product spaces. Journal of Multivariate Analysis 31, 201-219. SCHWEIZER, B. and SKLAR, A. [1983] Probabilistic Metric Spaces. Elsevier, New York. SENETA, E. [1981] Non-Negative Matrices and Markov Chains. Springer, New York. SENETA, E. and VERE-JONES, D. [1966] On quasi-stationary distributions in discrete-time Markov chains with a denumerable infinity of states. J. Appl. Prob. 3, 403-434. SHURENKOV, V.M. [1984] On the theory of Markov renewal. Theo. Prob. Appl. 29, 247-265. SIGMAN, K. [1990] One-dependent regenerative processes and queues in continuous time. Math. Oper. Res. 15, 175-189.
References 505 [1995] Stationary Marked Point Processes: An Intuitive Approach. Chapman and Hall, New York. SIGMAN, K., THORISSON, H., and WOLFF, R.W. [1994] A note on the existence of regeneration times. J. Appl. Probab. 31, 1116-1122. SIGMAN, K. and WOLFF, R.W. [1993] A review of regenerative processes. SIAM Review 35, 269-288. SILVESTROV, D.S. [1979] The method of a single probability space in renewal theory. Theo. Probab. Appl. 24, 655-656. [1994] Coupling for Markov renewal processes and the rate of convergence in ergodic theorems for processes with semi-Markov switchings. Acta Appl. Math. 34, 109-124. SKOROHOD, A.V. [1956] Limit theorems for stochastic processes. Theo. Probab. Appl. 1, 261-290. [1965] Studies in the Theory of Random Processes. Addison-Wesley, Reading, Mass. SMITH, W.L. [1954] Asymptotic renewal theorems. Proc. Roy. Soc. Edinburgh Ser. A 64, 9-48. [1955] Regenerative stochastic processes. Proc. Roy. Soc. London Ser. A 232, 6-31. [1958] Renewal theory and its ramifications. J. Roy. Statist. Soc. Ser. B 20, 243-302. SONIN, I.M. [1987] A theorem on separation of jets and some properties of random sequences. Stochastics 21, 231-250. STONE, C.R. [1966] On absolutely continuous components and renewal theory. Ann. Math. Statist. 37, 271-275. STONE, C.R. and WAINGER, S. [1967] One-sided error estimates in renewal theory. Journal d Analyse Mathematique XX, 325-352. STOYAN, D., KENDALL, W.S., and MECKE, J. [1987] Stochastic Geometry and its Applications. Wiley, New York.
506 References STRASSEN, V. [1965] The existence of probability measures with given marginals. Ann. Math. Statist. 36, 423-439. SVERCHKOV, M.Yu. and SMIRNOV, S.N. [1990] Maximal coupling of D-valued processes. Soviet Math. Dokl. 41, 352-354. TACKLIND, S. [1945] Fourieranalytische Behandlung vom Erneuerungsproblem. Skan- dinavisk Aktuarietidskrift 28, 68-105^ TEMPEL'MAN, A.A. [1972] Ergodic theorems for general dynarnical systems. Trudy Moskov Mat. Obsc. 26, 95-132. [Translation in Trans. Moscow Math. Soc. 26, 94-132.] [1992] Ergodic Theorems for Group Actions. Kluwer. Dordrecht. THORISSON, H. [1981] The Coupling of Regenerative Processes. Thesis. Department of Mathematics, University of Goteborg. [1983] The coupling of regenerative processes. Adv. Appl. Probab. 15, 531-561. [1985a] The queue GI/G/l: Finite moments of the cycle variables and uniform rates of convergence. Stoch. Proc. Appl. 19, 85-99. [19856] The queue GI/GI/k: Finite moments of the cycle variables and uniform rates of convergence. Stoch. Models 1, 221-238. [1985c] On regenerative and ergodic properties of the A;-server queue with nonstationary Poisson arrivals. J. Appl. Probab. 22, 893-902. [1985rf] Backward limits of non-time-homogeneous Markov transition probabilities. Stoch. Proc. Appl. 19, 20 [1986a] On non-time-homogeneity. Semi-Markov Models: Theory and Applications, Plenum Press, New York and London, 351-368. [19866] On maximal and distributional coupling. Ann. Probab. 14, 873- 876. [1987a] A complete coupling proof of Blackwell's renewal theorem. Stoch. Proc. Appl. 26, 87-97. [19876] Construction of a stationary regenerative process with a view towards simulation. Stoch. Proc. Appl. 26, 191. [1988a] Backward limits. Ann. Probab. 16, 914-924. [19886] Future independent times and Markov chains. Probab. Theo. Rel. Fields 78, 143-148.
References 507 [1990] Backward limits and inhomogeneous regeneration. Probability Theory and Mathematical Statistics. Proceedings of the 5th International Vilnius Conference, 474-481. [1992] Construction of a stationary regenerative process. Stoch. Proc. Appl. 42, 237-253. [1994a] Coupling and convergence of random elements, processes and regenerative processes. Acta Appl. Math. 34, 85-107. [19946] Shift-coupling in continuous time. Prob. Theo. Rel. Fields. 99, 477-483. [1995a] On time- and cycle-stationarity. Stoch. Proc. Appl. 55, 183-209. [19956] Coupling methods in probability theory. Scand. J. Stat. 22, 159- 182. [1996] Transforming random elements and shifting random fields. Ann. Probab. 24, 2057-2064. [1997a] On e-coupling and piecewise constant processes. Stoch. Models 13, 27-38. [19976] Markov processes and coupling. Theo. Stoch. Proc. 3, 424-438. [1998] Coupling. Probability Towards 2000, Springer Lecture Notes in Statistics 128, 319-339. [1999] Point-stationarity in d dimensions and Palm theory. Bernoulli 5, 797-831. [2000] Transformation coupling. To appear in Proceedings of the International Conference on Stochastic Processes and Their Applications, (A. Krishnamoorthy and P.V. Ushakumari, editors). Springer. TWEEDIE, R.L. [1974] Quasi-stationary distributions for Markov chains on a general state space. J. Appl. Probab. 11, 726-741. VON NEUMANN, J. [1951] Various techniques in connection with random digits. NBS Appl. Math. Ser. 12, 36-38. WILSON, D.B. [1998] Annotated bibliography of perfectly random sampling with Markov chains. Microsurveys in Discrete Probability, DIM ACS Series in Discrete Mathematics and Theoretical Computer Science 41, 209-220. American Mathematical Society. Updated versions to appear at dimacs.rutgers.edu/~dbwilson/exact. [1999] How to couple from the past using a read-once source of randomness. Preprint.
o
Index .4-coupling event, 149 ^-identical, 149 e-coupling, 178, see Epsilon-couplings distributional, 179 inequality, 181 maximality, 187 nondistributional, 179 successful, 72 CT-algebra exchangeable, 243 generated by, 78, 79 induced by, 78, 79 invariant, 161, 174, 221, 232 post-£, 153 remote, 247 smooth tail, 161, 188 tail, 156, 161, 246 trivial, 196 CT-finite, 214 Absolutely continuous, 82 Age process, 69, 339 Almost sure (a.s.), 81, 87 Aperiodic, 42, 47, 344 strongly, 47 Bell inequality, 27, 479 Biasing exponential-biasing, 442 length-biasing, 70, 250, 259, 284, 341, 345 length-debiasing, 250, 259, 284 volume-biasing, 310, 323 volume-debiasing, 316 Birth and death process, 33 Borel equivalence, 88 equivalent, isomorphic, 88 set, 78 Brownian motion, 97, 128, 242, 336 Can be close to, 57 Canonical, 78 Cartesian product, 79 Cemetery, 132 Censoring state, 132 Classical coupling, 34, 385, 399 Common probability space, 81 Complete metric space, 86 Component, 94
510 Index common component, 104 Condition in, 88 Conditional -ly i.i.d., 91 distribution, 87 expectation, 87 independence, 87, 90,91 probability, 87 Conditional distribution, 87 regular, 88 version of, 87 weak-sense-regular, 135 Convergence, 35 Cesaro total variation, 161, 167, 288, 331, 345 dominated, 26 from-the-past, 338, 422 geometric rate, 46 in distribution, 23, 118, 184, 185, 269 in the path space, 183-185 in the state space, 183-185, 195 plain total variation, 143, 266, 351, 361, 378 pointwise, 21, 118 smooth total variation, 161, 182, 267, 354, 362 taboo limit, 436 time-average total variation, 167 to-the-future, 338, 422 uniform, 46, 146, 168, 183, 401 weakly, 118 Convergence rate, 409 exponential order p, 144 geometric order p, 144 moment-order ip, 145 order if, 144 power moment-order a — 1, 168 power order a, 144, 168 uniform, 406 Copy, 1, 78 Coupling, 1, 79, 125, 161, see e- coupling, see Epsilon couplings, see Exact coupling, see Shift coupling, see Transformation coupling canonical version, 80 classical, 34, 338 decoupling, 484 distributional, 140, 142 epoch, 34, 137 .event, 7, 106 event inequality, 7, 9,14,112, 141 ' from-the-past, 338, 428 i.i.d., 2 impossible, 27, 479 independence, 2, 79 index, 113, 115, 116 index inequality, 113,115,116 indicator, 106 maximal, 7, 9, 10, 106, 141 maximal w.r.t. A, 151 Ornstein, 48 permutation, 243 quantile, 3 radius inequality, 247 regenerative processes, 349,376 remote, 247 rotation, 243 self-, 2 site inequality, 246 strong stationary time, 481 successful, 35 time, 34, 137 time inequality, 35 Cycle, 53, 252, 340, 438 one-dependent cycles, 366 Cycle-length, 252, 340, 438 Cycle-stationary, 249, 250, 254, 259, 284, 295, 341 Defined on, 78 Delay, 53, 340, 438 Delay time, 62 Delay-length, 341
Index 511 Deleting a null event, 82 an inner null set, 82, 84 Depends on ... only through, 91 Diagonal, 44 Distribution, 78 initial, 34 joint, 80 marginal, 79 stationary, 37 Distributionally unique, 118 Domination in distribution, 4 pointwise, 4 stochastic, 4, 146, 168, 183, 401 Drift, 67 EPR, 27, 479 Epsilon-couplings, 125, 161, 178 maximally successful, 192 regeneration, 354, 362 success probability, 178 successful, 178 Event, 81 coupling, 7 coupling event, 106 null, 81 shift-coupling, 219 Exact coupling, 125, 137, 161 coupling epoch, 137 coupling time, 137 coupling time inequality, 35 distributional, 125, 137 maximal, 147 maximal at time t, 146 maximally successful, 157 random fields, 245 regeneration, 351, 361 success probability, 137 successful, 137 time inequality, 143 time-inhomogeneous reg., 378 Exact transformation coupling, 244 Exchangeability, 243 Expectation, mean, 86 conditional, 87 Extension, 80 conditioning, 89 consistency, 85 independence, 85 product space, 83 proper, 83 reduction, 82, 84 splitting, 94 transfer, 92, 135 F0lner averaging sets, 220, 227, 240 Future-independent time, 481 Generalized inverse, 3 Greatest common divisor, 42 Harmonic function, 195 harmonic, 207 smooth space-time, 209 smoothed version, 209 space-time harmonic, 205 Harris chain, 364 i.i.d. coupling, 2 Independence coupling, 2, 79 Independent stationary background, 302, 305 Induced by, 78 Initial position, 47 Inspection paradox, 70 Invariant cr-algebra, 161,174, 221, 232, 281, 320 Ionescu Tulcea theorem, 89 Irreducible, 33, 39 Ising model, 470 Jordan Hahn decomposition, 112 Kolmogorov extension theorem, 86 Lattice, 57, 343, 351, 361 Lebesgue interval, 122 Lorentz transformations, 244
512 Index Markov chain continuous-time, 39 coupling from-the-past, 467 discrete-time, 42 regular, 45 simulation, 467 time-homogeneous, 39 Markov jump process, 39 Markov process, 201 conditional stationary, 214 initial distribution, 202 regeneration set, 364, 365 space-time process, 375 stationary distribution, 213 stationary measure, 213 strong, 44, 365 taboo regenerative, 440 time-homogeneous, 201 transition probabilities, 201 uniform nullity, 214 version, 202 Markov property, 39 strong, 44, 364 Maximal coupling, 7, 9, 10 coupling event, 7, 9 Measure dependent, 82 Measure-free, 251 Mixing, 195 Cesaro mixing, 199 mixing, 196, 351, 361, 378 smoothly mixing, 200, 354, 362 Moment a-moment, 45, 167 (^-moment, 144 classical coupling time, 406, 409 Mutually singular, 107 Nonexplosion, 33 Nonlattice, 57, 354, 362 strongly, 61 Nonlocality, 27, 479 Ornstein coupling, 48 Outcome, 81 Palm theory, 250 associated Z and 5, 253 coincidence of duals, 290, 335 cycle-stationary, 254, 259, 284 duals, 263 ergodic case, 291 inversion formula, 264 length-biasing, 250, 284 xlength-debiasing, 250, 284 Palm characterization, 308, 316 point process, 293, 296 point-at-zero duality, 250, 259, 323 point-stationary, 295, 299, 304, 305, 323 point/site, 297 point/time, 251 process and points, 251 randomization, 317 randomized-origin duality, 250, 284 regenerative, 262 sequence space (£,£), 251 stationary, 254, 259, 284, 295, 298, 304, 323, 337 version, 263 volume-biasing, 310, 323 volume-debiasing, 316, 323 Voronoi cells, 308 Part, 94 Poincare transformations, 244 Point of increase, 57 Point-map, 300 extended, 303 Point-shift, 283, 300 bijective, 302, 304 extended, 304 Point-stationary, 295, 299, 304,'305, 323 Palm characterization, 308 randomization, 317 Poisson process, 299, 376 Polish space, 85
Index 513 Post-i CT-algebra, 153 Power set, 88 Probability conditional, 87 kernel, 88 mass function, 7 Product CT-algebra, 79 Product probability space, 85 Product space, 79 Projection mapping, 79 Quantile coupling, 3 function, 3 Quantum physics, 27, 479 Quasi-stationary, 467 Queue G//G//1, 347 Queue GI/GI/k, 369, 376 Queue M/GI/k, 376 Random point, 251, 297 site, 218, 297 time, 78, 251 transformation, 223 variable, 78 Random element, 78 conditional, 89 external, 81 independent, 85 orginal, 81 transferred, 92 Random field, 218 exact coupling, 245 random site, 218 rotation invariant, 244 shift-coupling, 218, 219 shift-map, 218 site set, 218 Random walk, 47 ladder heights, 360 Randomized stopping time, 349 Recurrence time, 62, 347 Recurrent, 35 null, 37 positive, 37 Regenerative process, see Taboo regenerative, see Time- inhomogeneous regenerative classical, 337, 346 discrete time, 344 epsilon-couplings, 354, 362 equilibrium process, 486 exact coupling, 349, 351, 361, 376 inter-regeneration times, 347 lag-Z, 358 lag-Z+, 359 regeneration times, 346, 358 renovating event, 487 sequence space, 339 stationary, 348 two-sided, 262 version, 348, 359 wide-sense, 337, 358 zero-delayed, 341, 348 Relative age process, 69, 339 Relative position U, 252, 253 Relativity, 244 Renewal process, 62, 347 epsilon-couplings, 64 exact coupling, 103, 419 intensity measure, 104 taboo, 442 Renewal theorem Blackwell's, 64 total variation on [0, h], 103 total variation on [0, oo), 419 Representation, 1, 78 splitting, 11 Residual life process, 69, 339 Scheffe's theorem, 111 Self-coupling, 2 Self-similarity, 242 Separable metric space, 86 Set inner probability one, 82 outer null, 81
514 Index outer probability one, 82 Shift-coupling, 57, 125, 161, 162, 345 distributional, 163 events, 219 in Palm theory, 289, 330 inequality, 165, 220 maximality, 170 maximally successful, 175 nondistributional, 163 random fields, 218, 219 success probability, 162 successful, 162 times, 162 Shift-map, 130, 218, 251, 340 joint, 340 Signed measure, 109 Simulation, 272, 326 acceptance-rejection, 272,326 birth and death process, 477 coupling from-the-past, 338, 467 imperfect, 279, 326 initial transient problem, 272, 326 perfect, 272, 326, 338, 467 perfection probability, 279 quasi-stationarity, 467 S-s-inventory system, 276 stationarity detection, 489 Site-shift, 298, 303 Smooth tail cr-algebra, 161, 188 Span of a lattice, 74, 343 Splitting, 99, 365 element, 95 extension, 94 indicator, 94 representation, 11, 94 variable, 95 Spread out, 98, 351 Standard space, 88 Starts anew, 44 State space, 33 Stationary, 70, 72, 249, 250, 254, 259, 284, 295, 298, 304, 323, 337, 341, 348, see Cycle-stationary, see Point- stationary, see Taboo stationary asymptotically, 37, 383 conditional, 214 distribution, 37, 213 intensity, 255, 317 measure, 213 nonstationary, 429 Palm characterization, 316 periodically, 343 simulation, 467 stochastic process, 143 vector, 36 Stein's method, 479 Step-lengths, 47 Stochastic process, 33, 126 birth, 132 canonically jointly measurable, 130 cemetery, 132 censoring state, 132 Cesaro mixing, 199 distribution, 127, 128 exchangeable, 243 index set, 126 jointly measurable, 129 killing, 132 mixing, 196 path process, 359 path set, 128 path space, 128 paths, 127 real valued, 127 self-similar, 242 shift-map, 130 shift-measurable, 130, 252 smoothly mixing, 200 space-time process, 163 standard settings, 128 state space, 126 stationary, 143 trivial, 196 Strassen's theorem, 479
Index 515 Strong stationary time, 481 Strong uniform time, 481 Successful coupling, 35 Supported by, 78 Taboo regenerative, 338, 436, 438 acceptance-rejection, 468 lag-?, 439 regeneration times, 438, 439 simulation, 468 taboo limits, 338 taboo region, 338 taboo stationary, 338, 454 taboo time, 438, 439 time-inhomogeneous reg., 441 under taboo, 439 version, 439 wide-sense, 439 zero-delayed, 439 Taboo stationary, 338, 451, 452 characterization, 453 coupling from-the-past, 467 periodically, 466 simulation, 467 taboo regenerative, 454 taboo time, 452 under taboo, 451 version, 459, 461 Tail er-algebra, 156, 161, 246 Time coupling, 34 parameter, 33 random, 78 randomized stopping, 349 shift-coupling times, 162 sojourn, 33 stopping, 44, 349 Time-inhomogeneous regenerative, 337, 373 classical coupling, 338, 385, 399 exact coupling, 378 lag-J, 373 lag-f+, 373 nonstationary, 338 oftypep(-|-),373 recurrence distribution Fs, 374 regeneration times, 373 taboo renewal process, 442 time-homogeneous, 374, 383 up to time zero, 432, 433, 441 version, 373 wide-sense, 373 Topological transformation group, 238 amenable, 240 Haar measure, 238 locally compact, 238 second countable, 238 Total life process, 69, 339 Total variation, 12, 109 measure, 112 Trace, 82, 128 Transfer, 92, 100, 120, 135 Transformation coupling, 223, 239 coupling transformations, 223 distributional, 223, 239 exact, 244 inequality, 226, 240 maximality, 228 maximally successful, 232 nondistributional, 224, 239 success probability, 223 successful, 223, 240 Transformation semigroup, 222 acts jointly measurably, 222 F0lner averaging sets, 227 group,222 invariant measure, 225 inverse-measurable group, 223 jointly measurable, 222 measurable, 222 random transformation, 223 Transient, 40 Transition jointly measurable, 203 matrix, 34 probabilities, 201 semigroup, 34 Triviality, 195, 196
516 Index Unconscious probabilist, 85 Uniformly integrable, 381 Version, 63, 202 differently started, 34, 202 independent, 34 Voronoi cells, 308 Waiting time paradox, 70 Wiener process, 97, 128, 242 Zero-delayed, 63, 341
Notation R = (-00,00) and 1+ = [0, 00) Z = {...,-1,0,1,...} and Z+={0,1,...} X = Y = X and Y have the same distribution a := b = a is defined to be b f = g = for all x it holds that }{x) = g{x) \a — the indicator function of the set A #A = the number of elements in the set A a V b, a A 6 = maximum and minimum of a and b a+, a~ = a V 0 and -a A 0 [x] = inf {n 6Z:n^i} = integer part of x x mod /i = x — [x/h]h = the remainder when a; is divided by h f(x-) = limy-t-j f(y) = left-hand limit \i A A = the greatest common component of the measures \i and A (n — A)+, (/x — A)~ = positive and negative parts of /j — A /j _L A = the measures /j and A are mutually singular mU = restriction of the measure /j to the sub-er-algebra A —> = convergence in distribution A- = convergence in total variation || • || = the total variation norm dot-convention: /(•) = / = the function with value f(x) at x, for all x er(-) = the CT-algebra generated by • i.i.d. = independent and identically distributed 517