Continuous Time Markov Chains and Applications

Author: G. George Yin
Publisher: Springer Science & Business Media
ISBN: 1461443466
Format: PDF
Download Now
This book gives a systematic treatment of singularly perturbed systems that naturally arise in control and optimization, queueing networks, manufacturing systems, and financial engineering. It presents results on asymptotic expansions of solutions of Komogorov forward and backward equations, properties of functional occupation measures, exponential upper bounds, and functional limit results for Markov chains with weak and strong interactions. To bridge the gap between theory and applications, a large portion of the book is devoted to applications in controlled dynamic systems, production planning, and numerical methods for controlled Markovian systems with large-scale and complex structures in the real-world problems. This second edition has been updated throughout and includes two new chapters on asymptotic expansions of solutions for backward equations and hybrid LQG problems. The chapters on analytic and probabilistic properties of two-time-scale Markov chains have been almost completely rewritten and the notation has been streamlined and simplified. This book is written for applied mathematicians, engineers, operations researchers, and applied scientists. Selected material from the book can also be used for a one semester advanced graduate-level course in applied probability and stochastic processes.

Engineering Mathematics II

Author: Sergei Silvestrov
Publisher: Springer
ISBN: 3319421050
Format: PDF
Download Now
This book highlights the latest advances in engineering mathematics with a main focus on the mathematical models, structures, concepts, problems and computational methods and algorithms most relevant for applications in modern technologies and engineering. It addresses mathematical methods of algebra, applied matrix analysis, operator analysis, probability theory and stochastic processes, geometry and computational methods in network analysis, data classification, ranking and optimisation. The individual chapters cover both theory and applications, and include a wealth of figures, schemes, algorithms, tables and results of data analysis and simulation. Presenting new methods and results, reviews of cutting-edge research, and open problems for future research, they equip readers to develop new mathematical methods and concepts of their own, and to further compare and analyse the methods and results discussed. The book consists of contributed chapters covering research developed as a result of a focused international seminar series on mathematics and applied mathematics and a series of three focused international research workshops on engineering mathematics organised by the Research Environment in Mathematics and Applied Mathematics at Mälardalen University from autumn 2014 to autumn 2015: the International Workshop on Engineering Mathematics for Electromagnetics and Health Technology; the International Workshop on Engineering Mathematics, Algebra, Analysis and Electromagnetics; and the 1st Swedish-Estonian International Workshop on Engineering Mathematics, Algebra, Analysis and Applications. It serves as a source of inspiration for a broad spectrum of researchers and research students in applied mathematics, as well as in the areas of applications of mathematics considered in the book.

Discrete Time Markov Chains

Author: George Yin
Publisher: Springer Science & Business Media
ISBN: 9780387219486
Format: PDF, Mobi
Download Now
Focusing on discrete-time-scale Markov chains, the contents of this book are an outgrowth of some of the authors' recent research. The motivation stems from existing and emerging applications in optimization and control of complex hybrid Markovian systems in manufacturing, wireless communication, and financial engineering. Much effort in this book is devoted to designing system models arising from these applications, analyzing them via analytic and probabilistic techniques, and developing feasible computational algorithms so as to reduce the inherent complexity. This book presents results including asymptotic expansions of probability vectors, structural properties of occupation measures, exponential bounds, aggregation and decomposition and associated limit processes, and interface of discrete-time and continuous-time systems. One of the salient features is that it contains a diverse range of applications on filtering, estimation, control, optimization, and Markov decision processes, and financial engineering. This book will be an important reference for researchers in the areas of applied probability, control theory, operations research, as well as for practitioners who use optimization techniques. Part of the book can also be used in a graduate course of applied probability, stochastic processes, and applications.

Continuous Time Markov Decision Processes

Author: Xianping Guo
Publisher: Springer Science & Business Media
ISBN: 3642025471
Format: PDF, Docs
Download Now
Continuous-time Markov decision processes (MDPs), also known as controlled Markov chains, are used for modeling decision-making problems that arise in operations research (for instance, inventory, manufacturing, and queueing systems), computer science, communications engineering, control of populations (such as fisheries and epidemics), and management science, among many other fields. This volume provides a unified, systematic, self-contained presentation of recent developments on the theory and applications of continuous-time MDPs. The MDPs in this volume include most of the cases that arise in applications, because they allow unbounded transition and reward/cost rates. Much of the material appears for the first time in book form.

Markov Processes for Stochastic Modeling

Author: Masaaki Kijima
Publisher: CRC Press
ISBN: 9780412606601
Format: PDF, Mobi
Download Now
This book presents an algebraic development of the theory of countable state space Markov chains with discrete and continuous time parameters.

Markov Chains and Stochastic Stability

Author: Sean P. Meyn
Publisher: Springer Science & Business Media
ISBN: 144713267X
Format: PDF, Kindle
Download Now
Markov Chains and Stochastic Stability is part of the Communications and Control Engineering Series (CCES) edited by Professors B.W. Dickinson, E.D. Sontag, M. Thoma, A. Fettweis, J.L. Massey and J.W. Modestino. The area of Markov chain theory and application has matured over the past 20 years into something more accessible and complete. It is of increasing interest and importance. This publication deals with the action of Markov chains on general state spaces. It discusses the theories and the use to be gained, concentrating on the areas of engineering, operations research and control theory. Throughout, the theme of stochastic stability and the search for practical methods of verifying such stability, provide a new and powerful technique. This does not only affect applications but also the development of the theory itself. The impact of the theory on specific models is discussed in detail, in order to provide examples as well as to demonstrate the importance of these models. Markov Chains and Stochastic Stability can be used as a textbook on applied Markov chain theory, provided that one concentrates on the main aspects only. It is also of benefit to graduate students with a standard background in countable space stochastic models. Finally, the book can serve as a research resource and active tool for practitioners.

Adventures in Stochastic Processes

Author: Sidney I. Resnick
Publisher: Springer Science & Business Media
ISBN: 9780817635916
Format: PDF, ePub, Docs
Download Now
Stochastic processes are necessary ingredients for building models of a wide variety of phenomena exhibiting time varying randomness. This text offers easy access to this fundamental topic for many students of applied sciences at many levels. It includes examples, exercises, applications, and computational procedures. It is uniquely useful for beginners and non-beginners in the field. No knowledge of measure theory is presumed.

Basics of Applied Stochastic Processes

Author: Richard Serfozo
Publisher: Springer Science & Business Media
ISBN: 3540893326
Format: PDF, Docs
Download Now
Stochastic processes are mathematical models of random phenomena that evolve according to prescribed dynamics. Processes commonly used in applications are Markov chains in discrete and continuous time, renewal and regenerative processes, Poisson processes, and Brownian motion. This volume gives an in-depth description of the structure and basic properties of these stochastic processes. A main focus is on equilibrium distributions, strong laws of large numbers, and ordinary and functional central limit theorems for cost and performance parameters. Although these results differ for various processes, they have a common trait of being limit theorems for processes with regenerative increments. Extensive examples and exercises show how to formulate stochastic models of systems as functions of a system’s data and dynamics, and how to represent and analyze cost and performance measures. Topics include stochastic networks, spatial and space-time Poisson processes, queueing, reversible processes, simulation, Brownian approximations, and varied Markovian models. The technical level of the volume is between that of introductory texts that focus on highlights of applied stochastic processes, and advanced texts that focus on theoretical aspects of processes.

Stochastic Modelling of Social Processes

Author: Andreas Diekmann
Publisher: Academic Press
ISBN: 1483266567
Format: PDF, ePub
Download Now
Stochastic Modelling of Social Processes provides information pertinent to the development in the field of stochastic modeling and its applications in the social sciences. This book demonstrates that stochastic models can fulfill the goals of explanation and prediction. Organized into nine chapters, this book begins with an overview of stochastic models that fulfill normative, predictive, and structural–analytic roles with the aid of the theory of probability. This text then examines the study of labor market structures using analysis of job and career mobility, which is one of the approaches taken by sociologists in research on the labor market. Other chapters consider the characteristic trends and patterns from data on divorces. This book discusses as well the two approaches of stochastic modeling of social processes, namely competing risk models and semi-Markov processes. The final chapter deals with the practical application of regression models of survival data. This book is a valuable resource for social scientists and statisticians.

Hidden Markov Models

Author: Robert J. Elliott
Publisher: Springer Science & Business Media
ISBN: 0387943641
Format: PDF, ePub, Mobi
Download Now
The aim of this book is to present graduate students with a thorough survey of reference probability models and their applications to optimal estimation and control. These new and powerful methods are particularly useful in signal processing applications where signal models are only partially known and are in noisy environments. Well-known results, including Kalman filters and the Wonham filter, emerge as special cases. The authors begin with discrete time and discrete state spaces. From there, they proceed to cover continuous time, and progress from linear models to nonlinear models, and from completely known models to only partially known models. Readers are assumed to have a basic grounding in probability and systems theory, such as might be gained from the first year of graduate study, but otherwise this account is self-contained. Throughout, the authors have taken care to demonstrate engineering applications which show the usefulness of these methods.