Transcript Dynamic SEM
A Proximal Gradient Algorithm for Tracking
Cascades over Networks
Brian Baingana, Gonzalo Mateos and Georgios B. Giannakis
Acknowledgments: NSF ECCS Grant No. 1202135 and NSF AST Grant No. 1247885
May 8, 2014
Florence, Italy
Context and motivation
Contagions
Infectious diseases
Buying patterns
Popular news stories
Network topologies:
Unobservable, dynamic, sparse
Propagate in cascades
over social networks
Topology inference vital:
Viral advertising, healthcare policy
Goal: track unobservable time-varying network topology from cascade traces
B. Baingana, G. Mateos, and G. B. Giannakis, ``A proximal-gradient algorithm for tracking cascades over
2
social networks,'' IEEE J. of Selected Topics in Signal Processing, Aug. 2014 (arXiv:1309.6683 [cs.SI]).
Contributions in context
Structural equation models (SEM) [Goldberger’72]
Statistical framework for modeling relational interactions (endo/exogenous effects)
Used in economics, psychometrics, social sciences, genetics… [Pearl’09]
Related work
Static, undirected networks e.g., [Meinshausen-Buhlmann’06], [Friedman et al’07]
MLE-based dynamic network inference [Rodriguez-Leskovec’13]
Time-invariant sparse SEM for gene network inference [Cai-Bazerque-GG’13]
Contributions
Dynamic SEM for tracking slowly-varying sparse networks
Accounting for external influences – Identifiability [Bazerque-Baingana-GG’13]
First-order topology inference algorithm
D. Kaplan, Structural Equation Modeling: Foundations and Extensions, 2nd Ed., Sage, 2009.
3
Cascades over dynamic networks
N-node directed, dynamic network, C cascades observed over
Unknown (asymmetric) adjacency matrices
Example: N = 16 websites, C = 2 news events, T = 2 days
Event #1
Event #2
Node infection times depend on:
Causal interactions among nodes (topological influences)
Susceptibility to infection (non-topological influences)
4
Model and problem statement
Data: Infection time of node i by contagion c during interval t:
un-modeled dynamics
external influence
Dynamic SEM
Captures (directed) topological
and external influences
Problem statement:
5
Exponentially-weighted LS criterion
Structural spatio-temporal properties
Slowly time-varying topology
Sparse edge connectivity,
Sparsity-promoting exponentially-weighted least-squares (LS) estimator
(P1)
Edge sparsity encouraged by
-norm regularization with
Tracking dynamic topologies possible if
6
Topology-tracking algorithm
Iterative shrinkage-thresholding algorithm (ISTA) [Parikh-Boyd’13]
Ideal for composite convex + non-smooth cost
Let
gradient descent
(P2)
Solvable by soft-thresholding operator [cf. Lasso]
-γ
γ
Attractive features
Provably convergent, closed-form updates (unconstrained LS and soft-thresholding)
Fixed computational cost and memory storage requirement per
Scales to large datasets
7
Recursive updates
Sequential data terms in
recursive updates
: row i of
Each time interval
Recursively update
Acquire new data
Solve (P2) using (F)ISTA
8
Simulation setup
Kronecker graph [Leskovec et al’10]: N = 64, seed graph
Non-zero edge weights varied for
1
0.5
Uniform random selection from
0
0
20
40
60
80
100
120
140
160
180
200
0
20
40
60
80
100
120
140
160
180
200
0
20
40
60
80
100
120
140
160
180
200
0
20
40
60
80
100
120
140
160
180
200
1
edge weight
0.5
0
1
0.5
0
1
0
Non-smooth edge weight variation
cascades,
−1
time
,
9
Simulation results
Algorithm parameters
20
20
40
40
60
60
20
40
actual, t=20
60
20
40
inferred, t=20
20
20
40
40
60
60
20
40
actual, t=180
60
60
20
40
60
inferred, t=180
Error performance
10
The rise of Kim Jong-un
Web mentions of “Kim Jong-un” tracked from March’11 to Feb.’12
Kim Jong-un – Supreme leader of N. Korea
N = 360 websites, C = 466 cascades, T = 45 weeks
Increased media frenzy following Kim
Jong-un’s ascent to power in 2011
t = 10 weeks
t = 40 weeks
Data: SNAP’s “Web and blog datasets” http://snap.stanford.edu/infopath/data.html
11
LinkedIn goes public
Tracking phrase “Reid Hoffman” between March’11 and Feb.’12
N = 125 websites, C = 85 cascades, T = 41 weeks
US sites
t = 5 weeks
t = 30 weeks
Datasets include other interesting “memes”: “Amy Winehouse”, “Syria”, “Wikileaks”,….
Data: SNAP’s “Web and blog datasets” http://snap.stanford.edu/infopath/data.html
12
Conclusions
Dynamic SEM for modeling node infection times due to cascades
Topological influences and external sources of information diffusion
Accounts for edge sparsity typical of social networks
Proximal gradient algorithm for tracking slowly-varying network topologies
Corroborating tests with synthetic and real cascades of online social media
Key events manifested as network connectivity changes
Ongoing and future research
Dynamical models with memory
Identifiabiality of sparse and dynamic SEMs
Statistical model consistency tied to
Large-scale MapReduce/GraphLab implementations
Kernel extensions for network topology forecasting
Thank You!
13
ISTA
iterations
PG iterations
with equality constraints yield the (pseudo) real-time tracking algorithm:
T
als.
R equir e: Y t t= 1, X , β.
ˆ 0 = 0N × N , Bˆ 0 = Σ 0 = I N , Y¯ 0 = 0N × C , λ 0.
1: Initialize A
2: for t = 1, . . . , T do
3:
Σ t = βΣ t− 1 + Y t (Y t )⊤
Recursive Updates
4:
Y¯ t = β Y¯ t− 1 + Y t
5:
Initialize A [0] = Aˆ t− 1, B [0] = Bˆ t− 1, and set k = 0.
6:
while not converged do
7:
for i = 1. . . N (in parallel) do
8:
a− i [k + 1] = Sλ t / L f a− i [k] − (1/ L f )∇ a− i f [k]
Parallelizable
9:
bi i [k + 1] = bi i [k] − (1/ L f )∇ bi i f [k]
10:
a⊤
i [k + 1] = [a− i ,1[k + 1] . . . a− i ,i − 1[k + 1] 0 a− i ,i [k + 1] . . . a− i ,N [k + 1]]
11:
end for
12:
k = k + 1.
13:
end while
14:
r et ur n Aˆ t = A [k], Bˆ t = B [k].
15: end for
A t t r act ive feat ur es of t he algor it hm :
for
1. Provably guaranteed convergence
14
ADMM iterations
Sequential data terms:
,
,
can be updated recursively:
denotes row i of
15
ADMM closed-form updates
Update
with equality constraints:
,
:
Update
by soft-thresholding operator
16
Outlook: Indentifiability of DSEMs
a1) edge sparsity:
a2) sparse changes:
a3) error-free DSEM:
Goal: under a1)-a3), establish conditions on
to uniquely identify
Preliminary result (static SEM)
If
, with
and diagonal matrix
and i)
, ii)
non-zero entries of
are drawn from a continuous distribution, and iii) Kruskal
rank
, then
and
can be uniquely determined.
J. A. Bazerque, B. Baingana, and G. B. Giannakis, "Identifiability of sparse structural equation models for directed, cyclic, and time-varying
networks," Proc. of Global Conf. on Signal and Info. Processing, Austin, TX, December 3-5, 2013.
17