Transcript IT-488

EC 723
Satellite Communication
Systems
Mohamed Khedr
http://webmail.aast.edu/~khedr
1
Syllabus
Week 1
Overview
Week 2
Orbits and constellations: GEO, MEO and LEO
Week 3
Satellite space segment, Propagation and
satellite links , channel modelling
Tentatively
Week 4
Satellite Communications Techniques
Week 5
Satellite Communications Techniques II
Week 6
Satellite error correction Techniques
Week 7
Multiple Access I
Week 8
Multiple access II
Week 9
Satellite in networks I, INTELSAT systems ,
VSAT networks, GPS
Week 10
GEO, MEO and LEO mobile communications
INMARSAT systems, Iridium , Globalstar,
Odyssey
Week 11
Presentations
Week 12
Presentations
Week 13
Presentations
Week 14
Presentations
Week 15
Presentations
2
Agenda
• Modulation Concept
• Analog Communication
• Digital Communication
• Digital Modulation Schemes
3
EFFECT OF FILTERING - 1
Fig. 5.8 in text
4
EFFECT OF FILTERING - 2
Rectangular pulses (i.e. infinite rise and fall
times of the pulse edges) need an infinite
bandwidth to retain the rectangular shape
Communications systems are always bandlimited, so
send a SHAPED PULSE
Attempt to MATCH the filter to the spectrum
of the energy transmitted
Before FILTERS, let’s look at Inter-Symbol Interference
5
INTER-SYMBOL INTERFERENCE
Sending pulses through a band-limited channel
causes “smearing” of the pulse in time
“Smearing” causes the tail of one pulse to extend
into the next (later) pulse period
Parts of two pulses existing in the same pulse
period causes Inter-Symbol Interference (ISI)
ISI reduces the amplitude of the wanted pulse
and reduces noise immunity
Example of ISI
6
ISI - contd. - 1
Form Couch, Fig. 3-23
7
ISI - contd. - 2
To avoid ISI, you can SHAPE the pulse so
that there is zero energy in adjacent pulses
Use NRZ; pulse lasts the full bit period
Use Polar Signaling (+V & -V); average value is
zero if equal number of 1’s and 0’s
Communications links are usually AC coupled so
you should avoid a DC voltage component
Then use a NYQUIST filter
Nyquist Filter???
8
NYQUIST FILTER - 1
Bit Period is Tb
Sampling of the signal is usually at intervals
of Tb
Thus, if we could generate pulses that are at
a one-time maximum at t = Tb and zero at
each succeeding interval of Tb (i.e. t = 2Tb,
3Tb, ….. , NTb then we would have no ISI
This is called a NYQUIST filter
9
NYQUIST FILTER - 2
Sampling
instant is
CRITICAL
Impulse at
this point
0
Tb
2Tb
3Tb
4Tb
t
10
NYQUIST FILTER - 3
NOTE: At each
sampling interval,
there is only one
pulse contribution
- the others being
at zero level
Fig. 5.9 in text
11
NYQUIST FILTER - 4
Arranging to sample at EXACTLY the
right instant is the “Zero ISI” technique,
first proposed by Nyquist in 1928
Networks which produce the required
time waveforms are called “Nyquist
Filters”. None exist in practice, but you
can get reasonably close
12
NYQUIST FILTER - 5
Noise into receiver must be held to a minimum
Place half of Nyquist filter at transmit end of link,
half at receive end, so that the individual filter
transfer function H(f) is given by H(f) matches pulse
Vr(f)NYQUIST = H(f)  H(f)
characteristic, hence
H  f   V r ( f ) NYQUIST
it is called a
“matched filter”
Filter is a “Square Root Raised Cosine Filter”
Matched Filter
13
MATCHED FILTER - 1
f
f
Roll-off factor =  = (f / f0 )
6 dB
where f0 = 6 dB bandwidth
B = absolute bandwidth (here
shown for  = 0.5) and
f1
f0
B
B = f
+
f0
f1 = start of ‘roll-off’ of the
filter characteristic
14
MATCHED FILTER - 2
A Raised Cosine Filter gives a Matched
Filter response
The “Roll-Off Factor”, , determines
bandwidth of Raised Cosine Low Pass Filter
(LPF)
Gives zero ISI when the output is sampled
at correct time, with sampling rate of Rb
(i.e. at a sampling interval of Tb)
BUT how much bandwidth is required
for a given transmission rate???
15
BANDWIDTH REQUIRED - 1
Bandwidth required depends on
whether the signal is at BASEBAND or
at PASSBAND
Bandwidth needed to send baseband
digital signal using a Nyquist LPF is
Bandwidth = (1/2)Rb(1 + )
Bandwidth needed to send passband digital
signal using a Nyquist Bandpass filter is
bandwidth = Rb(1 + )
NOTE: It is the Symbol Rate that is
key to bandwidth, not the Bit Rate
16
BANDWIDTH REQUIRED - 2
SYMBOL RATE is the number of digital
symbols sent per second
BIT RATE is the number of digital bits sent
per second
Different modulation schemes will “pack”
different numbers of Bits in a single Symbol
BPSK has 1 bit per symbol
QPSK has 2 bits per symbol
17
BANDWIDTH REQUIRED - 3
OCCUPIED BANDWIDTH, B, for a
signal is given by B = Rs ( 1 +  )
where Rs is the symbol rate and  is
the filter roll-off factor
NOISE BANDWIDTH, BN, for a
channel will not be affected by the rolloff factor of filter. Thus BN = Rs
18
BANDWIDTH EXAMPLE - 1
GIVEN:
Bit rate 512 kbit/s
QPSK modulation
Filter roll-off, , is  = 0.3
FIND: Occupied Bandwidth, B, and
Noise Bandwidth, BN 2 bits per
Number
symbol
of bits/s
SOLUTION:
Symbol Rate = Rs = (1/2)  (512  103)
= 256  103
19
BANDWIDTH EXAMPLE - 2
Occupied Bandwidth, B, is
B = Rs (1 +  )
= 256  103 ( 1 + 0.3)
= 332.8 kHz
Noise Bandwidth, BN, is
BN = Rs = 256 kHz
Now what happens if you have FEC?
Example with FEC
20
BANDWIDTH EXAMPLE - 3
SAME Example, but 1/2-rate FEC is now used
2 bits per
symbol
1/2-rate
FEC used
Number
of bits/s
SOLUTION
Symbol Rate, Rs = (1/2)  (2)  (512  103)
= 512  103 symbols/s
Occupied Bandwidth, B, is
B = Rs ( 1 +  )
= 665.6 kHz
21
BANDWIDTH EXAMPLE - 3
Noise Bandwidth, BN, is
BN = Rs = 512  103 = 512 kHz
Summary:
High Modulation Index  More Bandwidth
Efficient
FEC (Block or Convolutional)  Increases
bandwidth required
22
Digital Modulations
Functional model of passband data transmission system
23
Digital Modulations
In digital communications, the modulating signal is
a binary or M-ary data.
The carrier is usually a sinusoidal wave.
Change in Amplitude: Amplitude-Shift-Keying (ASK)
Change in Frequency: Frequency-Shift-Keying (FSK)
Change in Phase: Phase-Shift-Keying (PSK)
Hybrid changes (more than one parameter).
Ex. Phase and Amplitude change: Quadrature
Amplitude Modulation (QAM)
24
Binary Modulations – Basic Types
These two have
constant envelope
(important for
amplitude sensitive
channels)
25
Coherent and Non-coherent Detection
Coherent Detection (most PSK, some FSK):
Exact replicas of the possible arriving signals are available at
the receiver.
This means knowledge of the phase reference (phasedlocked).
Detection by cross-correlating the received signal with each
one of the replicas, and then making a decision based on
comparisons with pre-selected thresholds.
Non-coherent Detection (some FSK, DPSK):
Knowledge of the carrier’s wave phase not required.
Less complexity.
Inferior error performance.
26
27
Design Trade-offs
Primary resources:
Transmitted Power.
Channel Bandwidth.
Design goals:
Maximum data rate.
Minimum probability of symbol error.
Minimum transmitted power.
Minimum channel bandiwdth.
Maximum resistance to interfering signals.
Minimum circuit complexity.
28
Coherent Binary PSK (BPSK)
Two signals, one representing 0, the other 1.
s1 t   A cos  2 f c t 
s 2 t   A cos  2 f c t      A cos  2 f c t 
  s 1 t 
Each of the two signals represents a single bit of information.
Each signal persists for a single bit period (T) and then may be
replaced by either state.
Signal energy (ES) = Bit Energy (Eb), given by:
2
E S  Eb 
A T
2
Therefore 
A
2Eb
Tb
29
Orthonormal basis representation
Gram-Schmidt Orthogonalization: basis of signals that
are both ortogornal between them and normalized to
have unit energy.
T
 1 if i  j
  i ( t ) j ( t ) dt  
0
0
if i  j
Allows representation of M energy signals {si(t)} as
linear combinations of N orthonormal basis functions,
where N<=M.
Ex.: N=2
2
1 (t ) 
 2 (t ) 
T
2
T
cos( 2 f c t )
0tT
sin( 2 f c t )
0tT
30
BPSK representation
Let’s consider the unidimensional base
(N=1) where:
2
1 (t ) 
Tb
cos( 2 f c t )
0 tT
Let’s also rewrite the signal amplitudes
as a function of their energy:
s 1 t  
s 2 t   
2 Eb
Tb
cos  2 f c t 
2 Eb
Tb
cos  2 f c t 
31
BPSK representation
Therefore, we can write the signals s1(t) and
s2(t) in terms of 1(t):
s1 ( t ) 
E b 1 (t )
s 2 (t )   E b 1 (t )
0  t  Tb
0  t  Tb
• Which can be graphically represented as:
32
BPSK Physical Implementation
+A
1
t
t
0
0
1
0
0
1
1
1
0
-A
cos(2  f c t)
33
Detection of BPSK
Actual BPSK signal is
received with noise
We assume AWGN in
this class
Other noise properties are possible
AWGN is a good approximation
Other noise models are more complex
Constellation becomes a distribution because
of noise variations to signal
34
Recall Gaussian Distribution
Area to the right of this
line represents
Probability (x>x0)
= mean
=standard deviation

erfc  y  
x
 x0 -  b 
 x0 -  b  1
(x  x 0 )  Q 

  erfc 
2 
 
 2

Probabiliy
Where:
x0
2


e
z
2
dz
Approximation for large
positive values of y 
erfc  y  
y
e
y
2
y 
Both Q(.) and erfc(.) functions are integrals widely tabled
and available as functions in Excel and calculators
35
Calculating Error Probability
A W G N on S ignal
Noise Spectral Density = N0
Noise Variance:

-A
0

N0
2
i
+A
2
 
N0
2
P (-A /+ A ) = P (+ A /-A )

P( 0 /1)  P( 1 /0)  P x 


1
 erfc 

2


Eb




Eb  1
 erfc 

N0  2

2

2 
 Eb
 Q
 

Eb 

N 0 
 1

  erfc 
 2



Eb 

2  
BPSK error probability
36
Bit Error Rate (BER) for BPSK
BER is therefore given by

BER  erfc 

2

Eb 

N o 
1
BER 
0 . 2821
Eb
No

e
Eb
No
Approximation
valid for Eb/No
greater than ~4 dB
Eb/No
(dB)
0
2
4
6
8
10
10.543
BER
0.08
0.04
0.014
0.0027
2*10-4
4*10-6
10-6
Note that these calculations are for synchronous detection
37
Ambiguity Resolution
We haven’t discussed yet how to tell
which signal state is a 1 and which a 0
Because of variations in the signal path,
its impossible to tell a priori
Two common approaches resolutions:
Unique Word
Differential Encoding
38
Unique Word Ambiguity Resolution
A specific, known unique word is sent
The unique word is sent at a known time in
the data
The correct signal state is chosen as 1 to
correctly decode the unique word
Usually implemented with two detectors - the
output of the correct one is simply used
Could lead to problems until a new UW is RX
if a phase slip occurs
All bits after slip will be received wrong!
39
Differential Encoding Ambiguity Resolution
Data is not transmitted directly
Each bit is represented by:
0 => phase shift of  radians
1 => no phase shift
in the carrier
This results in ~ doubling the BER since
any error will tend to corrupt 2 bits
BER is then
BER
DBPSK
 2 BER
BPSK
Valid for BER<~0.01

 erfc 


E
b
E b  0 . 5642  N o

e
N o 
Eb
No
40
Coherent Quaternary PSK (QPSK)
Four signals are used to convey information
s1 t   A cos  2 f c t 
Constant Modulus =>
s 2 t   A cos  2 f c t   / 2 
s 3 t   A cos  2 f c t   
s 4 t   A cos  2 f c t   / 2 
This leads to a constellation of:
when shown as a phasor
referenced to the signal phase, q
Each of the two states represents
a two bits of information
q
+ jA
-A
+A
i
-jA
41
QPSK Constellation Representation
In this case we use the following orthonormal
basis:
1 (t ) 
 2 (t ) 
2
T
2
T
cos( 2 f c t )
0tT
sin( 2 f c t )
0tT
Which gives, after application of some
trigonometric identities, the following
constellation representation:
42
QPSK Constellation
43
QPSK Waveform
44
QPSK Physical Implementation
1/2 rate data
cos(2  f c t)
full rate
data, R b

D em ux
Q P S K sym bols
R s = R b /2
90 o
1/2 rate data
Note that the QPSK
signal can be seen to
be two BPSK signals
in phase quadrature
45
Block diagrams
of (a) QPSK
transmitter and
(b) coherent
QPSK receiver.
46
The two QPSK constellations. Note that they differ by п /4. When going from (1,1)
to (-1, -1), the phase is shifted by п. When going from (1, -1) to (1,1), the phase
shifts by п /2. Thus, depending on the incoming symbol, transitions from (1,1) can
occur to (1,1), (1,-1), (-1, 1), or (-1, -1) or vice versa, leading to phase shifts of 0,
± п /2, or ± п in QPSK. I and Q represent the in-phase and quadrature bits,
respectively. Arrows show all possible transitions.
47
Bit Error Rate (BER) for QPSK
The BER is still the probability of choosing the
wrong signal state (symbol now)
Because the signal is Gray coded (00 is next
to 01 and 10 for instance but not 11) the BER
for QPSK is that for BPSK:
BER (after a lot of derivation) is given by:
BER
QPSK

1
 erfc 

2

E
b
E b  0 . 2821  N o

e

No 
Eb
No
Approximation
valid for Eb/No
greater than ~4 dB
Note that Eb is here, not Es!
48
Possible paths for switching between the
message points in (a) QPSK and (b)
offset QPSK.
49
Block diagram of the OQPSK modulator.
50
Explanation of the phase transitions in OQPSK.
51
Two commonly used signal constellations of
QPSK; the arrows indicate the paths along which
the QPSK modulator can change its state.
52
Eight possible phase states for the /4shifted QPSK modulator.
53
Phase encoding for  /4-QPSK. The brackets [ ] and { } correspond
to the two respective constellations.
54
Details of the phase constellation
associated with п/4-QPSK. For
every alternate symbol, the carrier
waves are changed. From (1,1) to (1, -1), we go from [1 1] to {-1, -1},
or from {1,1} to [-1, -1], resulting in
a phase change of п /4, as opposed
to п/2 in QPSK. In QPSK we can go
from [1, 1] to [-1, -1] or from {1 1}

to {-1, -1}, resulting in a phase
change of п. Also, when we go
from (1, 1) to (1, 1) in п/4-QPSK, we
go from [1, 1] to {1, 1}, resulting in
a phase change of п/4. The phase

changes in п/4-QPSK are limited to

3п/4 or п/4. There are no phase


changes of 0, п/2, or п. All the
possible transitions are shown by
arrows.
55
Block diagram of the п /4-DQPSK transmitter.
Input dibit
Phase change
00
Pi/4
01
3pi/4
11
-3pi/4
10
-pi/4
 k   k 1  
56
Block diagram of the /4-shifted DQPSK
detector.
57
Frequency Shift Keying
Two signals are used to convey information
s1 t   A cos  2 f 1t  q 1 
Constant Modulus =>
s 2 t   A cos  2 f 2 t  q 2 
In principle, the transmitted signal appears as
2 sinx/x functions at carrier frequencies
Each of the two states represents
a single bit of information
Each state persists for a single bit period and
then may be replaced either state
BER is: 2x BPSK BER for coherent
1
for non-coherent
BER  e
 Eb
2No
2
58
Frequency Shift Keying
59
Other Modulations (cont.)
M-ary PSK
PSK with 2n states where n>2
Incr. spectral eff. - (More bits per Hertz)
Degraded BER compared to BPSK or QPSK
QAM - Quadrature Amplitude Modulation
Not constant envelope
Allows higher spectral eff.
Degraded BER compared to BPSK or QPSK
60
M-ary PSK
61
M-ary QAM
62
Other Modulations
OQPSK
QPSK
One of the bit streams delayed by Tb/2
Same BER performance as QPSK
MSK
QPSK - also constant envelope, continuous phase
FSK
1/2-cycle sine symbol rather than rectangular
Same BER performance as QPSK
63
Noncoherent
receivers.
(a) Quadrature
receiver using
correlators.
(b) Quadrature
receiver using
matched filters.
(c) Noncoherent
matched filter.
64
Output of
matched filter for
a rectangular RF
wave: (a) q  0,
and (b) q  180
degrees.
65
(a) Generalized
binary receiver for
noncoherent
orthogonal
modulation.
(b) Quadrature
receiver equivalent to
either one of the two
matched filters in
part (a); the index i 
1, 2.
66
Noncoherent receiver for the detection of
binary FSK signals.
67
Block
diagrams of
(a) DPSK
transmitter
and (b)
DPSK
receiver.
68
Signal-space diagram of received DPSK
signal.
69
Shannon Bound
1948 Shannon demonstrated that, with
proper coding a channel capacity of

E b  BW N o
Rb E b  
S 


Capacity  BW log 2  1 
log 2  1 
  Rb

N 
N o  Rb E b
BW N o  


since
S
N

Rb
Eb
therefor e
BW N o
lim Capacity  1 . 443 R b
B
since
Eb
No
R b  lim Capacity
B
R b  1 . 443 R b
Eb
No
or
Eb
No
  1 . 6 dB
Required channel quality
for error free communications
=>we’re doing much worse
70
Modulation Schemes Error Performance
71
M-ary PSK Error Performance
72
Operation Point Comparison
73
Summary of Useful Formulas
74
Summary of Digital Communications -1
Legend of variables mentioned in this section:
M = modulation size. (Ex: 2, 4, 16, 64)
Bw = Bandwidth in Hertz
 = Roll-off factor (from 0 to 1)
Gc = Coding Gain (convert from dB to linear to use in formulas)
Ov = Channel Overhead (convert from % to fraction : 0 to1)
BER = Bit Error Rate
75
Summary of Digital Communications - 2
•
Bits per Symbol:
B s  Log 2 M
•
Symbol Rate [symbol/second]:
•
Gross Bit Rate [bps]:
•
Net Data Rate [bps]:
RG
Rs 
1
1
BW
 1 
 B s R s  Log 2 M 
 BW
1  
 1
R i  R G (1  Ov )  Log 2 M 
1 

 B W (1  Ov )

76
Summary of Digital Communications - 3
•
Required Eb/No (assuming no coding) [adimensional]:
(function of modulation scheme and required bit error rate – see table later)
 Eb 


N 
 0  Req
•
 Table function
(Modulatio
n Scheme, BER)
from theory
Required Eb/No (using coding gain):
 Eb 
1  Eb 





N 


 0  Re q G c  N 0  Req
•
1
from theory
Required C/N :
 Eb
C 
 
 
 N  Re q  N 0

RG

*

 Re q B W
77
Summary of Digital Communications - 4
•
Required Signal Strength [Watts]:
Where
C Re q
k = Boltzman constant = 1.38e-23 J/Hz
TS = System Noise Temperature
T0 = ambient temperature (usually 290 K)
F = System Noise figure in linear scale (not in dB)
C 
 
N
 N  Re q
C 
C 
 
kT s B W   
kT 0 BW F
 N  Re q
 N  Re q
78
BER Calculation as a Function of Modulation
Scheme and Eb/No Available
• Equations given on next slide are used to calculate the bit error
rate (BER) given the bit energy by spectral noise ratio (Eb/No) as
input.
• These functions are used in their direct form for the bit error rate
calculations. Excel and some scientific calculators provide the
solution for the “erfc” function.
• The formulas provided can be inverted by numerical methods to
obtain the Eb/No required as a function of the BER.
• Also possible to draw the graphic and obtain the “inverse” by
graphical inspection.
79
BER Calculation as a Function of Modulation
Scheme and Eb/No Available - 2
Modulation
Schem e
Coh-PSK
Coh-DPSK
Coh-QPSK
Ncoh-QPSK(Dif)
Coh-8-PSK
Ncoh-8PSK(Dif)
16-QAM
32-QAM
64-QAM
256-QAM
Coh-4FSK
Theoretical BER Calculation
BER = 0.5*ERFC(SQRT((Eb/No)))
BER = ERFC(SQRT((Eb/No)))-0.5*(ERFC(SQRT((Eb/No))))^2
BER = ERFC(SQRT((Eb/No)))-0.25*(ERFC(SQRT((Eb/No))))^2
BER = ERFC(SQRT(2*(Eb/No))*SIN(PI()/4))
BER = ERFC(SQRT(3*(Eb/No))*SIN(PI()/8))
BER = ERFC(SQRT(2*3*(Eb/No))*SIN(PI()/(2*8)))
BER = ((1-1/K)/(LOG(K)/LOG(2)))*ERFC(SQRT(3*(LOG(K)/LOG(2))/(K^2-1)*(Eb/No)))
Where K = 4
BER = ((1-1/K)/(LOG(K)/LOG(2)))*ERFC(SQRT(3*(LOG(K)/LOG(2))/(K^2-1)*(Eb/No)))
Where K = 6
BER = ((1-1/K)/(LOG(K)/LOG(2)))*ERFC(SQRT(3*(LOG(K)/LOG(2))/(K^2-1)*(Eb/No)))
Where K = 8
BER = ((1-1/K)/(LOG(K)/LOG(2)))*ERFC(SQRT(3*(LOG(K)/LOG(2))/(K^2-1)*(Eb/No)))
Where K = 16
BER = 0.5*ERFC(SQRT((Eb/No)/2))
80
Maximum Likelihood (ML) Detection:
Concepts
81
Likelihood Principle
Experiment:
Pick Urn A or Urn B at random
Select a ball from that Urn.
The ball is black.
What is the probability that the selected
Urn is A?
82
Likelihood Principle (Contd)
Write out what you know!
P(Black | UrnA) = 1/3
P(Black | UrnB) = 2/3
P(Urn A) = P(Urn B) = 1/2
We want P(Urn A | Black).
Gut feeling: Urn B is more likely than Urn A (given
that the ball is black). But by how much?
This is an inverse probability problem.
Make sure you understand the inverse nature of the
conditional probabilities!
Solution technique: Use Bayes Theorem.
83
Likelihood Principle (Contd)
Bayes manipulations:
P(Urn A | Black) =
P(Urn A and Black) /P(Black)
Decompose the numerator and denomenator in terms of the
probabilities we know.
P(Urn A and Black) = P(Black | UrnA)*P(Urn A)
P(Black) = P(Black| Urn A)*P(Urn A) + P(Black|
UrnB)*P(UrnB)
We know all these values Plug in and crank.
P(Urn A and Black) = 1/3 * 1/2
P(Black) = 1/3 * 1/2 + 2/3 * 1/2 = 1/2
P(Urn A and Black) /P(Black) = 1/3 = 0.333
Notice that it matches our gut feeling that Urn A is less likely, once we
84
have seen black.
Likelihood Principle
Way of thinking…
Hypotheses: Urn A or Urn B ?
Observation: “Black”
Prior probabilities: P(Urn A) and P(Urn B)
Likelihood of Black given choice of Urn: {aka forward probability}
P(Black | Urn A) and P(Black | Urn B)
Posterior Probability: of each hypothesis given evidence
P(Urn A | Black)
{aka inverse probability}
Likelihood Principle (informal): All inferences depend ONLY on
The likelihoods P(Black | Urn A) and P(Black | Urn B), and
The priors P(Urn A) and P(Urn B)
Result is a probability (or distribution) model over the space of
possible hypotheses.
85
Maximum Likelihood (intuition)
Recall:
P(Urn A | Black) = P(Urn A and Black)
/P(Black) =
P(Black | UrnA)*P(Urn A) / P(Black)
P(Urn? | Black) is maximized when P(Black |
Urn?) is maximized.
Maximization over the hypotheses space (Urn A or Urn B)
P(Black | Urn?) = “likelihood”
=> “Maximum Likelihood” approach to maximizing
posterior probability
86
Maximum Likelihood (ML): mechanics
Independent Observations (like Black): X1, …, Xn
Hypothesis q
Likelihood Function: L(q) = P(X1, …, Xn | q) = i P(Xi | q)
{Independence => multiply individual likelihoods}
Log Likelihood LL(q) = i log P(Xi | q)
Maximum likelihood: by taking derivative and setting to zero
and solving for q
P
Maximum A Posteriori (MAP): if non-uniform prior
probabilities/distributions
Optimization function
87