Transcript Markov
Markov Analysis
Jørn Vatn
NTNU
1
Introduction
Markov analysis is used to model systems which have
many different states
These states range from “perfect function” to a total fault
state
The migration between the different states may often be
described by a so-called Markov-model
The possible transitions between the states may further be
described by a Markov diagram
2
Purpose
Markov analysis is well suited for deciding reliability
characteristics of a system
Especially the method is well suited for small systems with
complicated maintenance strategies
In a Markov analysis the following topics will be of interest
Estimating the average time the system is in each state. These
numbers might further form a basis for economic considerations.
Estimating how frequent the system in average “visits” the various
states. This information might further be used to estimate the need
for spare parts, and maintenance personnel.
Estimate the mean time until the system enters one specific state,
for example a critical state.
3
Markov Analysis procedure
1.
2.
3.
4.
5.
Make a sketch of the system
Define the system states
Draw the Markov diagram with the transition rates
Quantitative assessment
Compilation and presentation of the result from the
analysis
4
Make a sketch of the system
Pump system wit active pump and a spare pump in
standby
Active
pump
Standby
pump
5
Definition of system states
x1 = state of active pump
x2 = state of standby pump
1 if component i is functioning
xi
0 if component i is in a fault state
System state
xS
Component state Comments
x1
x2
2
1
1
Both pumps functioning
1
0
1
The active pump is in a fault state, the
standby pump is functioning
0
0
0
Both pumps in a fault state
6
State transitions
1 =
2 =
1 =
B =
For this system we have assumed that if the active pump fails, the
standby pump could always be started
Further we assume that if both pumps have failed, they will both be
repaired before the system is put into service again
The following transition rates are defined
failure rate of the active pump
failure rate of the standby pump (while running, 2 = 0 in standby
position)
repair rate of the active pump (1/1 = Mean Down Time when the
active pump has failed)
repair rate when both pumps are in a fault state. I.e. we assume
that if the active pump has failed, and a repair with repair rate 1 is
started, one will ”start over again” with repair rate B, if the standby
pump also fails, independent of “how much” have been repaired on
the active pump.
7
Markov state space diagram
The circles represent the system states, and the arrows
represent the transition rates between the different system
states
The Markov diagram and the description of states
represent the total qualitative description of the system
1
2
1
2
1
0
B
8
Quantitative assessment
We want to assess the following quantities
Average time the system remain in the various
system states
The visiting frequencies to each system state
1
2
9
1
2
1
0
B
a00
a
Transition matrix
10
A
The indexing starts on 0, and moves
to r, e.g. there are r +1 system states
ar 0
Each cell in the matrix has two indexes,
a01
a11
aij
a r1
where the first (row index) represent the
”from” state, whereas the second
(column index) represent the “to” state.
The cells represent transition rates from one state to
another
aij is thus the transition rate from state i to state j
The diagonal elements are a kind of ”dummy”-elements,
which are filled in at the end, and shall fulfil the condition
that all cells in a row adds up to zero
10
a0 r
a1r
arr
Example transition matrix: (From , To )
0
0 B
A
1 2
2 0
1
2
0
B
1
1
2 1
1
1
2
11
1
2
1
0
B
State probabilities
Let Pi(t) represent the probability that the system is in
state i at time t
Now introduce vector notation, i.e.
P(t) = [P0(t), P1(t),…,Pr(t)]
From the definition of the matrix diagram it might be
shown that the Markov state equations are given by:
P(t) A = d P(t)/d t
These equations may be used to establish both the steady
state probabilities, and the time dependent solution
12
Steady state probabilities
Let the vector P = [P0, P1,…,Pr] represent the average
time the system is in the various system states in the long
time run
For example, P0 is average fraction of the time the system
is in state 0, P1 is average fraction of the time the system
is in state 1
The elements P = [P0, P1,…,Pr] are also denoted steady
state probabilities to indicate that in the stationary situation
Pi represents the probability that the system is in state i.
13
The steady state solution
In the long run when the system has stabilized we must
have that d P(t)/d t = 0, hence
PA = 0
This system of equations is over-determined, hence we
may delete one column, and replace it with the fact that
P0+ P1+…+Pr = 1
Hence, we have
14
The steady state solution
P A1 = b
where
a00
a
A1 10
ar 0
a01
a11
a r1
1
1
1
and
b = [0,0, …,0,1]
15
Example
P0
P1
B
P2 2
0
0
2 1
1
1
1 0 0 1
1
which gives
12
P0
(2 B )1 (2 1 ) B
B 1
P1
( 2 B )1 ( 2 1 ) B
B ( 2 1 )
P2
( 2 B )1 ( 2 1 ) B
16
Numerical solution
To solve the steady state equations P A1 = b is a tedious
task
Often we therefore solve these equations by numerical
methods
The Markov.xls program does this, where we have to:
Define the transition rates
Assign numerical values to the transition rates
Specify the Markov state space matrix
17
Program for simple Markov analysis
Transition matrix
Parameter Value
Dim
Init
SystFail
3
2
0
1
1.00E-03
2
1
B
5.00E-03
Steady state pr.
P0
0.000915
P1
0.007627
P2
0.991458
Visit frequencies
v0
3.81317E-05
v1
0.000991458
v2
0.000991458
From 0
From 1
From 2
To 0
To 1
To 2
-0.04167
0 0.041667
0.005
-0.13
0.125
0
0.001
-0.001
-0
0.125
0.0416667
0
MTTFS 26200.02
Parameter Numeric values
names
of the parameters
(Give the cells
names)
1
2
1
2
1
0
B
0
A
1
2
18
B
2
0
1
0
2 1
1
2
B
1
1
Visiting frequencies
Often we are interested in evaluating how many times the
system enters the various states, i.e. the visiting
frequencies
The visiting frequency for state j is denoted j, and could
be obtained by:
j = -Pjajj
From our example we obtain the “system failure rate”
0 P0a00
B12
(2 B )1 (2 1 ) B
19
Time dependent solution
Up to now we have investigated the steady state situation
In some situations we also want to investigate the time
dependent solution, i.e. the probability that the system is
in e.g. state 0 at time t
We now let Pi(t) be the probability that the system is in
state i at time t
The time dependent solution may be found by:
P(t) A = d P(t)/d t
Which could be solved by Laplace methods, or numerical
methods
For numerical methods we apply Markov.xls
20
Spare parts – Simple model
Assume a spare part regime where there is only one
spare in the stock
Upon a demand (with demand rate ) a new spare is
ordered
The intensity of arrival of a new spare is = 1/ MTA
(Mean Time to Arrival)
A
1
0
21
Spare parts 2 spares in stock
Assume a spare part regime where there are two
spares in the stock
Upon a demand (with demand rate ) a new spare is
ordered
The intensity of arrival of a new spares is = 1/ MTA
(Mean Time to Arrival) independent of how many in
order
0
1
2
2
1
A
0
22
0
1
2
0
0