This course will be held in English.

Dieser Kurs wird auf englisch gehalten.

Wir beschäftigen uns mit den grundlegenden stochastischen Modellen, die zur Untersuchung der Leistung von Computersystemen häufig benutzt werden. Markov modelle und Warteschlangen werden gerne für die Untersuchung dynamischer Systeme verwendet, z.B. Computer Hardware, Kommunicationsprotokolle, biologische Systeme, Epidemien, Verkehr und digitale Währungen.  Wir werden uns einen raschen Überblick verschaffen.  Betrachtete Themen sind der Geburts- und Todesprozess, der Poissonprozess, verallgemeinerte Markov und semi-Markov prozesse sowie deren Lösungsmethoden. Soweit die Zeit es erlaubt werden wir auch die Hintergründe der diskreten Ereignissimulation ansehen.

Literature

William Stewart. Probability, Markov Chains, Queues and Simulation. Princeton University Press 2009.

Lecture Materials:

This lecture will be on site and in presence. However, videos and materials from the past exist and can be used, if you miss a lecture. There is no guarantee that we will cover exactly and only this material.

All existing videos can be found in the Markov Chains folder in the FU Box: https://box.fu-berlin.de/s/nE2ieRKPYkW4c8e and in the vbrick server https://fu-berlin.eu.vbrickrev.com/#/ (search for Markov Chains). The following plan may still change.

Week 1: (18.04.-20.04.2023) Introduction  (slides in resources),

Lecture 1: Probability theory primer part 1 (video: https://fu-berlin.eu.vbrickrev.com/sharevideo/605ffac7-3de7-484f-a5dc-7500fd4cf58d, lecture notes in resources).

Week 2 (25.04.-27.04.2023):

Lecture 2: Probability theory primer part 2, Moments, Generating Functions, Minimum and Maximum of RV (video1:  https://fu-berlin.eu.vbrickrev.com/sharevideo/0e089345-ba47-400e-8ac5-f3e35662941a, video2: https://fu-berlin.eu.vbrickrev.com/sharevideo/2493e910-cb71-4ef4-b765-fd7df2c527f3.

Lecture 3: Discrete probability distributions  https://fu-berlin.eu.vbrickrev.com/sharevideo/1d2133bb-ca5d-42c7-8427-c9e52dde7508.

Week 3 (02.05.-04.05.2023):

There will be no lecture on 2nd of May 2023.

Lecture 4: Continuous probability distributions, reliability theory https://fu-berlin.eu.vbrickrev.com/sharevideo/76842793-36f8-46b1-a1a3-ac5cb5211056.

Week 4 (09.05.-11.05.2023):

Part b: Phase-type distributions https://fu-berlin.eu.vbrickrev.com/sharevideo/7654dbd2-25a8-4d61-b9d3-7afc4b361ea7.

Lecture 5: Bounds and limit theorems:  https://fu-berlin.eu.vbrickrev.com/sharevideo/8b37bbfd-993c-4d6d-90a7-31d5cd534583.

Lecture 6: Discrete time Markov chains https://fu-berlin.eu.vbrickrev.com/sharevideo/491d7c88-4eb9-43ce-ac5a-0c465bc67cc5.

Week 5 (16.05-18.05.2023):

No lecture on 18.05.2023

Lecture 7: DTMCs, Sojourn times and embedded MCs  https://fu-berlin.eu.vbrickrev.com/sharevideo/305636f9-7698-4994-b91e-cf5b238d9331.

Lecture 8: DTMCs Classification of states https://fu-berlin.eu.vbrickrev.com/sharevideo/a74301da-bb72-4781-8bd7-076b60987e0f.

Lecture 9: DTMCs, Irreducibility, Potential and Fundamental matrix   https://fu-berlin.eu.vbrickrev.com/sharevideo/fafba7ad-2061-4794-a743-387568f8620f.

Lecture 10: Random Walk  https://fu-berlin.eu.vbrickrev.com/sharevideo/080d549b-5b74-449a-90a6-a7aa4c9a1a23.

Week 6 (23.05.-25.05.2023):

Lecture 11: Limiting and stationary distributions https://fu-berlin.eu.vbrickrev.com/sharevideo/4ca5c416-0719-4f42-b03a-280b0bdc9f03.

Lecture 12: Reversibility    https://fu-berlin.eu.vbrickrev.com/sharevideo/b81bcc98-97ad-4b8e-bc3f-d4d124e55603.

Lecture 13: Page rank  https://fu-berlin.eu.vbrickrev.com/sharevideo/e65fb743-fcfd-4f27-9098-f2624556993f.

Markov Decision Processes (MDPs) Material by David Silver.

Week 7 (30.05.-01.06.2023):

Lecture 14: CTMCs  https://fu-berlin.eu.vbrickrev.com/sharevideo/f4e314ae-26fe-47e0-b841-69c97b2ad23f

Lecture 15: Renewal processes, PP, uniformisation, stochastic Petri nets https://fu-berlin.eu.vbrickrev.com/sharevideo/6d1cbc1b-9c6c-4791-a6e8-f91454b4e4e0.

Week 8 (06.6.-08.06.2023):

Lecture 16:

Petri net tool PIPE2:    https://fu-berlin.eu.vbrickrev.com/sharevideo/6ee2f5c1-4714-4bce-be27-2fe798cb5e03.

Lecture 17: Basic queueing theory   https://fu-berlin.eu.vbrickrev.com/sharevideo/bb1a2e24-6f6a-4497-82be-28a3d5d101ed.

Week 9 (13.6.-15.06.2023):

Lecture 18: Basic queueing theory (Part 2) https://fu-berlin.eu.vbrickrev.com/sharevideo/dacb3026-0717-40e6-96e3-12c6f7aa4e37

Lecture 19: The M/M/1 queue   https://fu-berlin.eu.vbrickrev.com/sharevideo/0a5911f9-0358-41e1-976b-d74e46a99525 

Week 10 (20.06.-22.06.2023):

Lecture 20: The M/M/m queue https://fu-berlin.eu.vbrickrev.com/sharevideo/a0b7ef7b-19ec-4997-a8d3-5e5a06ce114c  

Lecture 21: The M/M/m/K queue  https://fu-berlin.eu.vbrickrev.com/sharevideo/c7495d8e-6c26-4c81-823c-08cc8aa8f799

Week 11 (27.06.-29.06.2023):

Lecture 22: The M/G/1 queue  https://fu-berlin.eu.vbrickrev.com/sharevideo/4a625125-ec14-40a2-9c59-b216c06b0e39

Lecture 23: Open queueing networks https://fu-berlin.eu.vbrickrev.com/sharevideo/c0722e9e-6cef-43e4-8021-e9dd1fd50cc8

Week 12 (04.07.-06.07.2023):

Lecture 24 Closed Queueing Networks https://fu-berlin.eu.vbrickrev.com/sharevideo/0ce476d9-e232-46aa-ae72-ab444bd6a876

Week 13 (11.07.-13.07.2023):

Lecture 25 Mean Value Analysis https://fu-berlin.eu.vbrickrev.com/sharevideo/85154b64-1e4d-4f24-b63e-2a108ac3948c

Week 14 (18.07.-20.07.2023):

Lecture 26  TBA

20.07.2023: 10-12  Exam