Syllabus Math 606, Spring 2024

Stochastic Processes

Author
Affiliation

Luc Rey-Bellet

University of Massachusetts Amherst

Instructor

Luc Rey-Bellet
LGRT 1423K

Office hours:

  • Tuesday 3:00 PM-4:00 PM in LGRT 1423 K or on ZOOM
  • Friday 1:00 PM-2:00 in LGRT 1423 K or on ZOOM
  • By appointment is always possible, and/or ask your questions be email.

Class Meeting

Tu-Th, 10:00AM–11:15AM in LGRT 171

Class slides

Part 1: Markov chains

Part 2: Poisson processes and continuous time Markov chains

Part 3: Martingales

Class homepage

On Canvas at https://umamherst.instructure.com/courses/9591

Syllabus

This is the second part of a 2-semester graduate sequence Math605-Math606 which leads to the Stochastics qualifying exam. Prerequisites are

  1. a solid working knowledge of probablilty (e.g. Math 605, STAT 607 or equivalent).
  2. a solid working knowedge of analysis (at least Math 523 or equivalent) and some basic linear algebra.
  3. some mathematical maturity

In Math 606 we will cover various aspects of stochastic processes. One of the main goal in the class is to develop a “probabilist intuition and way of thinking”. We will present some proofs and we will skip some others in order to provide a reasonably broad range of topics, concepts and techniques. We emphasize examples both in discrete and continuous time from a wide range of disciplines and with an eye on numerical implementation. Among the topics treated in the class are

  1. Discrete time Markov chains on discrete spaces. Definition and basic properties, classification of states (positive recurrence, recurrence and transience), stationary distribution and limit theorems, analysis of transient behavior, optimal stopping, Monte-Carlo Markov chains.
  2. Continuous-Time Markov chains. Poisson Process, Birth and Death Process, and Queueing models. Renewal processes and semi-Markov processes.
  3. Martingales and applications
  4. Brownian motion and applications.

Learning Objectives

The first basic objective is to introduce the basic stochastic processes, learn how to analyze them and apply them, in particular via Monte-Carlo methods. Our goal is to find a good balance between theory, modeling, and implementation and to devlop probabilistic intuition.

Grade/assignment

Weekly homework, one midterm and one final exam, each valued 1/3 of your grade.

The homework can be found at the end of each section in the slides. It will be submitted via gradescope https://www.gradescope.com/courses/712199

Textbooks

There is no official text book for the class. Class slides will be posted on and serve also as a form of textbook, at least as a skeleton thereof. To get started here are the slides for Markov chain chapter from last year

Markov chain slides.

I will post a new updated version as the semester moves on.

I have been inspired by many textbooks when preparing my class, see the list below. I strongly suggest you pick a textbook and spend time regularly reading from it, in parallel to the class. The book by Lawler is a marvelous short introduction which covers most topic in the class and is highly recommended as a first read. The books by Resnick and Bremaud are a bit more advanced and are both excellent as well. The topics of simulation and Markov chain is well covered in the books by Ross, Madras, and Rubinstein and Kroese. The book by David A. Levin, Yuval Peres, and Elizabeth L. Wilmer is a great book to learn modern Markov chain techniques.

  • Markov Chains Gibbs Fields, Monte Carlo Simulation, and Queues, 2nd edition (2007) by Pierre Bremaud, Springer.

  • Introduction to Stochastic Processes, 2nd edition (2007) by Gregory F. Lawler, Chapman&Hall.

  • Adventures in Stochastic processes, by Sidney I. Resnick, Birkhauser.

  • Essentials of Stochastic Processes, by Rick Durrett, Springer.

  • Introduction to Probability Models, , by Sheldon M. Ross, Academic Press

  • Lectures on Monte-Carlo Methods, by Neal N. Madras, American Mathematical Society

  • Simulation and the Monte Carlo Method by Reuven Y. Rubinstein and Dirk P. Kroese , Wiley

  • Markov chain and mixing times, by David A. Levin, Yuval Peres, and Elizabeth L. Wilmer, American Mathematical Society

  • Introduction to Stochastic Processes, by Paul G. Hoel, Sidney C. Port and Charles J. Stone, Waveland Press.

  • Stochastic Processes, by Sheldon M. Ross, Wiley.

  • A first course in Stochastic Processes, by Samuel Karlin and Howard M. Taylor, Academic Press.