Transcript pptx

• Real-time
raise alerts
• Real-time with historical
• Correlate
• Offline
Real-Time
Dashboard
Engine
+ Fabric
• Develop initial monitoring query
• Back-test
• Progressive
Non-temporal analysis
Interactive Query Authoring
• Performance
• Fabric & language integration
• Query model
Scenarios
• monitor
telemetry &
raise alerts
• correlate realtime with logs
• develop initial
monitoring
query
• back-test over
historical logs
• offline analysis
(BI) with early
results
• Performance
• Fabric & language integration
• Query model
• Key enabler: performance +
fabric & language integration +
query model
struct ClickEvent { long ClickTime; long User; long AdId; }
var str = Network.ToStream(e => e.ClickTime, Latency(10secs));
var query =
str.Where(e => e.User % 100 < 5)
.Select(e => { e.AdId })
.GroupApply( e => e.AdId,
s => s.Window(5min).Aggregate(w => w.Count()));
query.Subscribe(e => Console.Write(e)); // write results to console
…
stream of batches
𝑜𝑝2
𝑜𝑝1
…
• More load  larger batches  better throughput
…
…
𝑜𝑝2
class DataBatch {
long[] SyncTime;
...
Bitvector BV;
}
𝑜𝑝1
…
…
class UserData_Gen : DataBatch {
long[] c_ClickTime;
long[] c_User;
long[] c_AdId;
}
timestamp
payload columns
bitvector
Application
Receive(results)
Trill
On(Batch b) {
for i = 0 to b.Size {
if !(b.c_User[i]%100 < 5)
set b.bitvector[i]
}
next-operator.On(b)
}
Send(events)
...
str.Where(e => e.User % 100<5);
session windows
• Powerful high-perf expression-based user-defined aggregate framework
http://aka.ms/trill