Self-contained presentation: from elementary material to state-of-the-art research; Much of the theory in book-form for the first time; Connections are made between probability and other areas of mathematics, engineering and mathematical physics

A unified treatment of the theory of 'stopping times' for probability theorists and statisticians.

Multi-armed bandit problems pertain to optimal sequential decision making and learning in unknown environments. Since the first bandit problem posed by Thompson in 1933 for the application of clinical trials, bandit problems have enjoyed lasting attention from multiple research communities and have found a wide range of applications across diverse domains. This book covers classic results and recent development on both Bayesian and frequentist bandit problems. We start in Chapter 1 with a brief overview on the history of bandit problems, contrasting the two schools—Bayesian and frequentist—of approaches and highlighting foundational results and key applications. Chapters 2 and 4 cover, respectively, the canonical Bayesian and frequentist bandit models. In Chapters 3 and 5, we discuss major variants of the canonical bandit models that lead to new directions, bring in new techniques, and broaden the applications of this classical problem. In Chapter 6, we present several representative application examples in communication networks and social-economic systems, aiming to illuminate the connections between the Bayesian and the frequentist formulations of bandit problems and how structural results pertaining to one may be leveraged to obtain solutions under the other.

They then examine the Bernoulli, Poisson, and Normal (univariate and multivariate) data generating processes.

This volume is devoted to the study of asymptotic properties of wide classes of stochastic systems arising in mathematical statistics, percolation theory, statistical physics and reliability theory. Attention is paid not only to positive and negative associations introduced in the pioneering papers by Harris, Lehmann, Esary, Proschan, Walkup, Fortuin, Kasteleyn and Ginibre, but also to new and more general dependence conditions. Naturally, this scope comprises families of independent real-valued random variables. A variety of important results and examples of Markov processes, random measures, stable distributions, Ising ferromagnets, interacting particle systems, stochastic differential equations, random graphs and other models are provided. For such random systems, it is worthwhile to establish principal limit theorems of the modern probability theory (central limit theorem for random fields, weak and strong invariance principles, functional law of the iterated logarithm etc.) and discuss their applications.There are 434 items in the bibliography.The book is self-contained, provides detailed proofs, for reader's convenience some auxiliary results are included in the Appendix (e.g. the classical Hoeffding lemma, basic electric current theory etc.).