This course will be in English.

We will study Markov Chains including Birth-, Death processes, Poisson process, Phase-type distributions, Markov decision processes, and queueing models.

Lecture Materials:

This lecture will be on site and in presence. However, videos and materials from the past exist and can be used, if you miss a lecture. There is no guarantee that we will cover exactly and only this material.

All existing videos can be found in the Markov Chains folder in the FU Box: https://box.fu-berlin.de/s/nE2ieRKPYkW4c8e . The following plan may still change.

Week 1: (15.04.-17.04.2025) Introduction  (slides in resources),

Lecture 1: Probability theory primer part 1  (lecture notes in resources).

Lecture 2: Probability theory primer part 2, Moments.

There will be no lectures on 24.04. and 26.04.2025

Week 2 (29.04.2025):

Lecture 3: Generating Functions, Minimum and Maximum of RV, Discrete probability distributions.

Week 3 (06.05.2025):

Lecture 4: Continuous probability distributions, reliability theory.

Week 4 (13.05.-15.05.2025):

Part b: Phase-type distributions

Lecture 5: Bounds and limit theorems:

Lecture 6: Discrete time Markov chains 

Week 5 (20.05-22.05.2025):

Lecture 7: DTMCs, Sojourn times and embedded MCs  

Lecture 8: DTMCs Classification of states

Lecture 9: DTMCs, Irreducibility, Potential and Fundamental matrix  

Lecture 10: Random Walk

Week 6 (27.05.2025):

No lecture on 29.5.2025.

Lecture 11: Limiting and stationary distributions 

Lecture 12: Reversibility  

Lecture 13: Page rank 

Markov Decision Processes (MDPs) Material by David Silver.

Week 7 (03.06.-05.06.2025):

Lecture 14: CTMCs

Lecture 15: Renewal processes, PP, uniformisation, stochastic Petri nets

Week 8 (10.6.-12.06.2025):

Lecture 16:

Petri net tool PIPE2:   

Lecture 17: Basic queueing theory 

Week 9 (17.6.-19.06.2025):

Lecture 18: Basic queueing theory (Part 2) 

Lecture 19: The M/M/1 queue 

Week 10 (24.06.-26.06.2025):

Lecture 20: The M/M/m queue 

Lecture 21: The M/M/m/K queue

Week 11 (01.07.-03.07.2025):

Lecture 22: The M/G/1 queue

Lecture 23: Open queueing networks 

Week 12 (08.07.-10.07.2025):

Lecture 24 Closed Queueing Networks

Week 13 (15.07.-17.07.2025):

Lecture 25 Mean Value Analysis 

17.07.2025: 10-12  Exam

Literatur

William Stewart. Probability, Markov Chains, Queues and Simulation. Princeton University Press 2009.