Markov Chains

Author: D. Revuz
Publisher: Elsevier
ISBN: 9780080880228
Format: PDF, Docs
Download Now
This is the revised and augmented edition of a now classic book which is an introduction to sub-Markovian kernels on general measurable spaces and their associated homogeneous Markov chains. The first part, an expository text on the foundations of the subject, is intended for post-graduate students. A study of potential theory, the basic classification of chains according to their asymptotic behaviour and the celebrated Chacon-Ornstein theorem are examined in detail. The second part of the book is at a more advanced level and includes a treatment of random walks on general locally compact abelian groups. Further chapters develop renewal theory, an introduction to Martin boundary and the study of chains recurrent in the Harris sense. Finally, the last chapter deals with the construction of chains starting from a kernel satisfying some kind of maximum principle.

Markov Chains

Author: J. R. Norris
Publisher: Cambridge University Press
ISBN: 9780521633963
Format: PDF
Download Now
In this rigorous account the author studies both discrete-time and continuous-time chains. A distinguishing feature is an introduction to more advanced topics such as martingales and potentials, in the established context of Markov chains. There are applications to simulation, economics, optimal control, genetics, queues and many other topics, and a careful selection of exercises and examples drawn both from theory and practice. This is an ideal text for seminars on random processes or for those that are more oriented towards applications, for advanced undergraduates or graduate students with some background in basic probability theory.

Markov Chains

Author: Pierre Bremaud
Publisher: Springer Science & Business Media
ISBN: 1475731248
Format: PDF, ePub, Docs
Download Now
Primarily an introduction to the theory of stochastic processes at the undergraduate or beginning graduate level, the primary objective of this book is to initiate students in the art of stochastic modelling. However it is motivated by significant applications and progressively brings the student to the borders of contemporary research. Examples are from a wide range of domains, including operations research and electrical engineering. Researchers and students in these areas as well as in physics, biology and the social sciences will find this book of interest.

Understanding Markov Chains

Author: Nicolas Privault
Publisher: Springer
ISBN: 9811306591
Format: PDF, ePub, Mobi
Download Now
This book provides an undergraduate-level introduction to discrete and continuous-time Markov chains and their applications, with a particular focus on the first step analysis technique and its applications to average hitting times and ruin probabilities. It also discusses classical topics such as recurrence and transience, stationary and limiting distributions, as well as branching processes. It first examines in detail two important examples (gambling processes and random walks) before presenting the general theory itself in the subsequent chapters. It also provides an introduction to discrete-time martingales and their relation to ruin probabilities and mean exit times, together with a chapter on spatial Poisson processes. The concepts presented are illustrated by examples, 138 exercises and 9 problems with their solutions.

Markov Chains and Mixing Times

Author: David Asher Levin
Publisher: American Mathematical Soc.
ISBN: 9780821886274
Format: PDF
Download Now
This book is an introduction to the modern approach to the theory of Markov chains. The main goal of this approach is to determine the rate of convergence of a Markov chain to the stationary distribution as a function of the size and geometry of the state space. The authors develop the key tools for estimating convergence times, including coupling, strong stationary times, and spectral methods. Whenever possible, probabilistic methods are emphasized. The book includes many examples and provides brief introductions to some central models of statistical mechanics. Also provided are accounts of random walks on networks, including hitting and cover times, and analyses of several methods of shuffling cards. As a prerequisite, the authors assume a modest understanding of probability theory and linear algebra at an undergraduate level. Markov Chains and Mixing Times is meant to bring the excitement of this active area of research to a wide audience.

Markov Chains

Author: David Freedman
Publisher: Springer Science & Business Media
ISBN: 1461255007
Format: PDF
Download Now
A long time ago I started writing a book about Markov chains, Brownian motion, and diffusion. I soon had two hundred pages of manuscript and my publisher was enthusiastic. Some years and several drafts later, I had a thousand pages of manuscript, and my publisher was less enthusiastic. So we made it a trilogy: Markov Chains Brownian Motion and Diffusion Approximating Countable Markov Chains familiarly - MC, B & D, and ACM. I wrote the first two books for beginning graduate students with some knowledge of probability; if you can follow Sections 10.4 to 10.9 of Markov Chains you're in. The first two books are quite independent of one another, and completely independent of the third. This last book is a monograph which explains one way to think about chains with instantaneous states. The results in it are supposed to be new, except where there are specific disclaim ers; it's written in the framework of Markov Chains. Most of the proofs in the trilogy are new, and I tried hard to make them explicit. The old ones were often elegant, but I seldom saw what made them go. With my own, I can sometimes show you why things work. And, as I will VB1 PREFACE argue in a minute, my demonstrations are easier technically. If I wrote them down well enough, you may come to agree.

Passage Times for Markov Chains

Author: R. Syski
Publisher: IOS Press
ISBN: 9789051990607
Format: PDF, Mobi
Download Now
The main theme of this book is that the appropriate unifying framework for discussion of passage times is provided by the probabilistic potential theory. Results of both theoretical and practical importance are discussed. Concepts of potentials, excessive functions and measures, balayage and duality are used. The crucial role of the Dirichlet problem and the Poisson equation, and various decomposition theorems is stressed. The four chapters cover analytic and measure theory, and applications. Annotation copyright by Book News, Inc., Portland, OR

Markov Chains

Author: Paul A. Gagniuc
Publisher: John Wiley & Sons
ISBN: 1119387558
Format: PDF, Mobi
Download Now
From observation to simulation -- Building the stochastic matrix -- Predictions by using 2-state Markov chains -- Predictions by using N-state Markov chains -- Absorbing Markov chains -- The average time spent in each state -- Discussions on different configurations of chains -- The simulation of an N-state Markov chain

Strong Stable Markov Chains

Author: N. V. Kartašov
Publisher: VSP
ISBN: 9789067642057
Format: PDF, Mobi
Download Now
This monograph presents a new approach to the investigation of ergodicity and stability problems for homogeneous Markov chains with a discrete-time and with values in a measurable space. The main purpose of this book is to highlight various methods for the explicit evaluation of estimates for convergence rates in ergodic theorems and in stability theorems for wide classes of chains. These methods are based on the classical perturbation theory of linear operators in Banach spaces and give new results even for finite chains. In the first part of the book, the theory of uniform ergodic chains with respect to a given norm is developed. In the second part of the book the condition of the uniform ergodicity is removed.