Transcript Memcached - UBC Computer Science

Scaling for Web
Architecting HootSuite for 5 Million+ Users
About Us
Beier Cai
Chief Architect
[email protected]
@beiercai
Bill Monkman
Lead Developer, Web
[email protected]
@bmonkman
Hoo’s HootSuite
Some Numbers
•
5+ million Members
•
10+ million Social Networks
•
25+ million Streams
•
350+ million Messages Scheduled
•
1.3+ Billion Messages Sent
•
2+ Billion Page Views per month
•
2,000 Web Requests/sec
Technologies
PHP
MySQL
Technologies
PHP
Memcached
MySQL
Technologies
PHP
Gearman
Memcached
MySQL
Technologies
PHP
Gearman
Memcached
MySQL
MongoDB
Technologies
PHP
Gearman
ZeroMQ
Memcached
MySQL
MongoDB
Technology Stack
•
LAMP (Linux, Apache, MySQL, PHP)
•
jQuery/Backbone
•
Scala
•
EC2 Cloud / CloudStack
•
Akka
•
Jenkins / CruiseControl
•
Python
•
CloudFront CDN
•
Clojure
•
HAProxy/Nginx/Varnish
•
MongoDB
•
Zend
•
Redis
•
Statsd/Graphite
•
Memcached
•
LogStash
•
Gearman
•
ElasticSearch/Lucene
•
ZeroMQ
•
Nagios/Munin/Sensu
•
RabbitMQ/AMQP
•
ZooKeeper
Optimization
• Push processing up to the front-end, make your users’ computers into a
distributed processing network. e.g. stream rendering
• Cache as much as possible, invalidate only when necessary.
• Sometimes features have to take a back-seat to performance/scalability.
• Look at profiler data.
• Watch query execution time, optimize when necessary with indexes,
partitions, de-normalization, etc.
• Know what your weak spots are and keep a close eye on them.
Optimization
Optimization Case Study
Optimization Case Study
Scaling Dev Team
Besides technical competency, we also value
• Passion
• Commitment
• Dedication
• Responsibility
• General Awesomeness
Scaling Dev Team
Besides technical competency, we also value
•
Passion
•
Commitment
•
Dedication
•
Responsibility
•
General Awesomeness
Scaling Dev Team
Team Work
•
Entrepreneurial
•
Communication
•
Respect
•
Trust
Coding Practice
Have a well defined coding standard within your dev team, and
follow it!
• Clean and Readable
• Maintainable
• Reusable
• Scalable
• Commented
Monitoring, Profiling & Logging
Monitoring, Profiling & Logging
Case Study
Case Study
Thank You!
Beier Cai
[email protected]
twitter.com/beiercai
Bill Monkman
[email protected]
twitter.com/bmonkman