TW2570代写、代做Python编程设计、代写c/c++，Java语言

日期：2020-04-08 09:52

Exam Markov Processen

TW2570

April 9, 2019, 13:30-16:30

With this exam you are allowed to use one page A4 cheat sheet. Every part of every question has the

same weight. There are 10 parts, so there are 50 points in total.

Just an answer is not sufficient: you need to offer an explanation or calculation and/or motivation how

you obtained you answer. Everything should be eligible in in Dutch or English.

English version (you can find the Dutch version on p. 1)

1. Consider the branching process (Xn)n≥0 with X0 = 1, where the number of offspring of an individual

Z is given bu the following probability distribution:

P(Z = 0) = P(Z = 1) = 14, en P(Z = 2) = 12.

a. Determine the expected value E[X3] of X3, and the probabiluty e that this branching process

eventually will become extinct.

b. Determine the conditional probability of eventual extinction of this branching process if it is

given that X10 = 5.

2. Consider a Poisson proicess (Nt)t≥0 with parameter λ > 0. One of the properties of the Poisson

process is, that it has independent increments. Furthermore, in Theorem 11.6 of G&W we saw that

for t > 0 the discrrete random variable Nt has an Poisson distribution with parameter λt.

a. Show that for t > 0 the transition matrix Pt = (pi,j (t)) satisfies that:?????

b. The matrix Q = (qi,j ) of a continuous time Markov chain was defined in chapter 5 of the

reader as qi,j = p0i,j (0), for all i, j ∈ S.

Use the expression of Pt from part a. of this exercise to determine Q for the Poisson process

(Nt)t≥0 with parameter λ > 0.

3. Consider a discrete, irreducible and aperiodic Markov chain (Xn)n≥0 on a finite state space S with

n-step transition probabilities pi,j (n) and stationairy distribution π = (πi)i∈S. Show that for all

i, j ∈ S we have that:limn→∞P(Xn = i, X2n = j) = πiπj .

Briefly indicate which results you use.

4. Consider the situation where a company has 3 identical machines. Each of these machines contains

a vital, precious and expensive part, which breaks down according to some probability distribution.

Broken parts are replaced according to the following scheme: at the beginning of the week one

orders as many of these parts as there are broken ones at that moment. Delivery time is exactly

one week, so the next week on Monday-morning the broken machines can work again.

Let Xn be the number of operational machines at the beginning of week n, and let Un be the number

of machines breaking down in week n (because the above mentioned precious part breaks down).

Suppose that machines become out-of-orrder according to the following probability distribution:

P(Un = j | Xn = i)| =1i + 1, j = 0, 1, . . . , i, voor i = 0, 1, 2, 3.

1

Check for yourself1

that the above replacement scheme yields that:

Xn+1 = 3 ? Un,

and that (Xn)n≥1 is a Markov chain.

a. Determine the matrix P of transition probabilities of the Markov chain (Xn)n≥1.

b. Determine the stationary distribution π of the Markov chain (Xn)n≥1. Briefly indicate why π

exists.

c. In the long run, what is the average number of operating machines?

5. Consider two machines, operating simultaneously and independently, where both machines have an

exponentially distributed time to failure with mean 1/μ. There is a single repair facility, and the

repair times are exponentially distributed with rate λ.

a. In the long run, what is the probability that no machines are operating when λ = μ = 1?

b. We now assume that at most one machine can operate at any time. Namely, while one machine

is working, the other one may be either under repair or already fixed and waiting to take over.

How does this modify your answer to question a. of this exercise?

1So this is not part of this exam!

2

Answers and partial solutions

By Theorem 9.19, e is the smallest nonnegative root of G(s) = s,

G(s)?s always has a factor s?1, because G(1) = 1, from which easily follows G(s)?s = (1?s)(1?2s).

1b There are several equivalent formulation of this question, for example: if there are five independent copies

of the branching process X, what is the probability that all become extinct? P(all become extinct)

2a This is QE 5.1 from the reader.

2b This is QE 5.4 from the reader.

3 By the Markov property and time homogeneity, we have

P(Xn = i, X2n = j) = P(Xn = i) P(X2n = j | Xn = i) = P(Xn = i) P(Xn = j | X0 = i);

the last term equals pi,j (n) by definition. From what is given, it follows that the Markov chain is positive

recurrent (Theorem 12.83), and therefore (Theorem 12.98), that for all i, j ∈ S:limn→∞pi,j (n) = πj .

This also implies that limn→∞ P(Xn = i) = πi, regardless the initial distribution (we showed this in

Exercise 12.108; on the exam you would have to provide the details).

Using that the limit of a product is the product of the limits (if they exist), the result follows from the

first equation, because the RHS converges to πiπj .

4a Applying the given relationship for n = 0, we find:

pik = P(X1 = k | X0 = i) = P(3 ? U0 = k | X0 = i) = 1

i + 1

for 0 ≤ 3 ? k ≤ i and i = 0, 1, 2, 3,

and the condition on k is equivalent to 3 ? i ≤ k ≤ 3. Hence, we find:

4b Solving π = πP yields π = (1, 2, 3, 4)/10; this shows that it exists, not why. . . . So:

Note that all states communicate with state 3 (the last row and the last column have only positive

entries), so the chain is irreducible. Since X has a finite state space, there must be a positive recurrent

state, which (by Theorem 12.83) implies the existence of the (unique) stationary distribution.

4c By the convergence theorem, P(Xn = i) → πi as n → ∞, for i = 0, 1, 2, 3 and because the distribution

of Xn converges to π, E[Xn] converges to the expectation P3

i=0 iπi of π. Numerical answer: 2.

5a This is a birth-death process on S = {0, 1, 2}. Draw a rate diagram; from it, one finds:

Solving πQ = 0 for λ = μ = 1, one finds π = (1, 2, 2)/5.

5b The only change in the model is in state 0: even if no machine is broken, still only one of them is

working, so q01 = μ = ?q00. Solving πQ = 0 again, the answer now is: π = (1, 1, 1)/3.

3