Transcript Cassandra in a real time bidding infrastructure

Keepin it real (time)
Cassandra in a real
time bidding
infrastructure
What is real time bidding (RTB)?
•
•
•
a dynamic auction process where each impression
is bid for in (near) real time
Advantages are cost efficiency, higher performance
and greater granularity with targeting and
measurement.
http://www.advertisingperspectives.com/adblog/media-technology/are-ad-exchanges-and-real-timebidding-the-next-big-thing/
I'm more of a visual person...
Where Cassandra lives
•
•
•
•
True 2-tier
Cassandra does
caching
Cassandra does
storage
AdServers talk to any
node
Requirements
•
Low latency reads
(exchanges have max request time)
•
Low latency writes
(record win for frequency capping)
•
Other Information (bulk/back loaded)
•
Large volumes of data ~ 10 TB (RF=3)
•
NOT having 1 to 1 Disk/RAM Ratio
•
Uptime (surviving node failures) :)
•
Manageability/Usability
Cassandra and writes
•
Writes to sorted memtable (in memory) + commit
•
Memtables flush periodically to Sstables (thresholds)
•
Compacts SStables (thresholds)
Results in
•
1-3 ms writes typical !Sweet!
So all good right?
•
Bulk load can pollute caches
•
Model requires compaction (vs write in place)
Cassandra and Reads
•
SSTables are sorted by key (>1SStables Bloom
filters)
•
Two Cassandra caches KeyCache and RowCache
•
VFS Cache + mmap (efficient for non cached items)
Results in
•
3-10ms(avg) Reads !!That's nice!! (Borat voice)
So its all good right ?
•
more data = more ram | faster disk | more nodes
•
Cache tuning for multiple Column Families is a
moving problem
Cassandra and Uptime
•
•
•
Cassandra handles replication for you
Multiple active nodes to serve reads and writes!
Nodes get themselves back into sync HH and Read
Repair
Results in
•
Minor failures may not even be visible on the client
•
Restart without dealing with replication logs etc.
So its all good right?
•
Do see the occasional OOM
•
Gossip protocol has turned into tabloid protocol
NoSQL Adventure time