Transcript Open Source in the Cloud

November 12th, 2012
5:30 – 7:30pm
“Open Source in the
Cloud”
Sponsored by
Sonian, Open Source and Sensu
November 12, 2012
Sonian’s Contributions
 Fog
- https://github.com/fog
 Elasticsearch
 Openstack
 Opscode
 Various




- https://github.com/elasticsearch
Swift - https://github.com/openstack/swift
Chef – https://github.com/opscode/chef/
Chef Tools:
https://github.com/portertech/chef-metrics
https://github.com/portertech/chef-journalist
https://github.com/portertech/recognizer
https://github.com/portertech/chef-irc-snitch
Home Built and Released
 SCLI

(Smart Cloud Command Line interface)
https://github.com/sonian/scli (MIT)
 Amazon-Pricing

https://github.com/sonian/amazon-pricing (Ruby)
 ElasticSearch

Jetty Plugin
https://github.com/sonian/elasticsearch-jetty (Apache 2)
 Sensu

(Pricing Gem)
– Monitoring Framework
https://github.com/sensu (MIT)
Sensu – “The Monitoring Router”
Framework – Built for the cloud (Dynamic
Environments)
 Monitoring






Ruby (EventMachine, Sinatra, AMQP), RabbitMQ, Redis
Messaging oriented architecture. Messages are JSON
objects. (Pub/Sub)
Ability to re-use existing Nagios plugins
Plugins and handlers (think notifications) can be written in
any language
Designed with modern configuration management systems
such as Chef or Puppet in mind
Lightweight, less than 1200 lines of code
Why We Built It
 Highly




Elastic Infrastructure
Nodes are created (Spot Nodes)
Bootstrapped (With Chef)
Take and process work
Terminated (when prices increase)

All before they are discovered and monitored by Nagios

Nagios is:

Difficult to Extend
 Can not discover new services on its own
 Generally Unpleasant
Keep It Simple™
• The Idea:
• Schedule the execution of
remote checks
• Collect their Results
• “Checks” are:
• Is the server up?
• How hard is it working?
• Tied into Modern CM
• Chef
• Puppet
• Message Oriented Middleware
• RabbitMQ
• Securely Routing
Checks/Results
• Redis: Fast In-Mem K/V Store
Open Source = Community
• Early Development – Recruit Community Experts
• Help Test – Drive Early Roadmap
• Develop Puppet and Chef modules
• Release Day (Nov 1st 2011)
• Make Sensu Github Repo Public
• Open IRC channel on Freenode (#sensu)
• Blog posting and Twitter for marketing
• Community, Community, Community
• Adoption – Documentation
• “Omnibus” Style Packaging for Quick Deployments
Contact
 Pete
Cheslock
 Director
of Technical & Cloud Operations @ Sonian
 @petecheslock
 http://about.me/petecheslock
 We’re
Hiring!

Please contact Sonian’s VP of Product Development Glenn
Snyder

[email protected]
Outline
• Level Set on Current Cloud Trends, pain
points, and markets
• Open Source Trends
• OpenStack Introduction
–What it is
–Historical Evolution
–Use Cases
–Conceptual Architecture
–Customer Adoption
–Cloud Taxonomy
11
Global Marketing
Level Set On
Cloud trends
12
Global Marketing
Cloud is about Services, Not Systems: Consumer
Market Driving Trends
24/7 CLOUD
SERVICES
Interconnected
mobile devices
13
Anywhere,
Anytime,
Internet Access
Global Marketing
Enterprises: Same Cloud Services Goals But
Different Market Segments and Pain Points
•
•
•
•
•
•
Burdened by “legacy”
infrastructure and
applications
Too complex to manage
Expensive to implement and
maintain
Takes too long to realize
value
Not easy to innovate
Security, compliance, and
privacy concerns
Mega Datacenters
Datacenter
 Mega
– Homogeneous
– Dense/Low Cost
 Small
Small Site
Site (SMB/Branch)
(SMB/Branch)
– Low Cost
– Ease of deployment
– Ease of management
Enterprise Datacenter
Datacenter
 Enterprise
– Heterogeneous
– Legacy / Complex
– Consolidating Quickly
• Source: IDC
14
Global Marketing
Virtual Machines Already Outstripping
Physical Machines
“2012: RATE(of VMs
launched per sec
6/sec) > RATE (at
which babies are
born in the US per
sec 8/sec) (from
#VMW) #cloud”
Global Marketing
But what about storage and networks?
Extensive
Use of
SAN
Complex
Networks
Dense
Blades
>$2000 /
VM
Specialized
Skillset
Fixed
Capacity
Global Marketing
Virtualization 1.0: Not up to the challenge
HOST 1
HOST 2
HOST 3 HOST 4, ETC.
VM
Hypervisor
(VMWare ESX, Citrix XEN Server, KVM, Etc.)
Hardware abstraction for each server
Better resource utilization for each server
1. Server Virtualization
Automation & Efficiency
2. Cloud Data Center
3. Cloud Federation
Global Marketing
But questions arise as the environment grows...
“VM SPRAWL” CAN MAKE THINGS UNMANAGEABLE VERY QUICKLY
USERS
APPS
ADMINS
How do you make your apps cloud aware?
How do you empower people to self-service?
Where should you provision new VMs?
How do you keep track of it all?
+
1. Server
Server Virtualization
Virtualization
Automation & Efficiency
2. Cloud Data Center
3. Cloud Federation
Global Marketing
But questions arise as the environment grows...
“VM SPRAWL” CAN MAKE THINGS UNMANAGEABLE VERY QUICKLY
USERS
APPS
ADMINS
A Cloud Management Layer Is Missing
1. Server
Server Virtualization
Virtualization
Automation & Efficiency
2. Cloud Data Center
3. Cloud Federation
Global Marketing
What is needed is a cloud Operating System that
adds automation and control at scale
Connects to apps via APIs
Self-service for users
USERS
ADMINS
APPS
CLOUD OPERATING SYSTEM
Creates Pools of Resources
1. Server
Server Virtualization
Virtualization
Automation & Efficiency
Automates The Network
2. Cloud Data Center
3. Cloud Federation
Global Marketing
Types of pools managed by the Cloud O.S.
COMPUTE, NETWORK, & STORAGE
Compute Pool
Network Pool
Storage Pool
1. Server
Server Virtualization
Virtualization
Automation & Efficiency
2. Cloud Data Center
Load Balancing Pool
Image Service Pool
3. Cloud Federation
Global Marketing
Assembly Line IT
Global Marketing
Robotics Factory IT
Global Marketing
Open Source,
OpenStack
Evolution and
Trends
24
Global Marketing
Innovation + Open Source = Virtuous Cycle
Open Source
Innovation
600,000+ OSS projects
100+ billion lines of code
10 million person-years of
work
Source: Blackduck
• Open source is leading, not following, in
important areas including cloud, big
data, mobile apps and enterprise
mobility.
• More than 50 percent of software
acquired in the next five years will be
open source software.
• Innovation, flexibility, cost, quality of
open source are some of the top reasons
that make it attractive for use
• Companies most likely to be impacted by
OSS have these characteristics: Business
drivers to invest in IT, software
development is an essential strategic
process, technology centric
• When asked about revenue generating
strategies likely to create value for
vendors, 52 percent of respondents said
an annual, repeatable support and service
agreement was the most likely value
strategy
• http://northbridge.com/2012-opensource-survey
Global Marketing
OpenStack is the Open Source Software Powering
Public and Private Clouds
Private Cloud:
Run OpenStack software
in your own
data centers
1. Server Virtualization
Automation & Efficiency
Public Cloud:
OpenStack powers some
of the worlds largest public
cloud deployments.
2. Cloud Data Center
3. Cloud Federation
Global Marketing
• What is it: An open source cloud operating system
– 30,000 lines of code to 600,000 in under 18
months
• Who’s building it: A worldwide community of
developers
– 600+ developers, 250+ contributed in the last 12
months
– 6000+ individual members in 87 countries
• Who is govering it: The OpenStack Foundation,
backed by AT&T, Canonical, Cisco, ClearPath,
Cloudscaling, Dell, DreamHost, HP, IBM, ITRI,
Mirantis, Morphlabs, Nebula, NetApp, Piston,
Rackspace, Red Hat, SUSE, and Yahoo! (so far)
Global Marketing
Wide-Ranging Community Support….
28
Confidential
5/6/2020
Global Marketing
OpenStack Timeline
Folsom
“Platform for
Innovation”
Essex
Diablo
Workable
Foundation
Cactus
Bexar
Austin
Formation
First
Public
Code
Community
Development
Forming
Working
Prototypes
Exposes Gaps
Solidify
Community
Loses VMware
& HyperV
“Production
Ready”
Stable Foundation
Included in Ubuntu
12.04
Network as a
Service
Block Storage
Incubated:
Network & Block
Storage
Public Adoptoin
Multiple Scale
Deployments
2012
2011
Nov 2010
Dec
Nov 2010:
Austin
Release
Feb
Feb 2011:
Bexar
Release
Oct 2010:
Design
Summit
Core Platform for
Innovation
Apr
Jun
Apr 2011:
Cactus
Release
Apr 2011:
Design
Summit
Aug
Oct
Sep 2011:
Diablo
Release
Oct 2011:
Design
Summit
Dec
Feb
Apr
Jun
Mar 2012:
Essex
Release
Aug
Oct 2012:
Folsom
Release
Apr 2012:
Design
Summit
Global Marketing
OpenStack Value Proposition
• Addresses Real Market Pains
Reduces Provides
Lock-in & Open APIs
Licenses & Support
– Limits costly software licenses
– Limits lock-in by vendors (VMware) & by providers
(Amazon)
– Allows for massive scalability
– Extensible hypervisor support (Xen, KVM, Hyper-V,
etc.)
– Offers standard APIs enabling growing cloud ecosystem
OpenStack is commoditizing the IaaS market
from single provider (Amazon) to many small
copy cats (startups).
Global Marketing
Popular OpenStack Use Cases
• Service providers offering an IaaS compute
platform
• IT departments provisioning compute
resources to teams and projects
• Processing big data with tools like Hadoop
• Scaling compute up and down to meet
demand for web resources and applications
31
Global Marketing
OpenStack Conceptual Architecture
32
Confidential
5/6/2020
Global Marketing
There is a broad adoption of
OpenStack across many markets.
The most common markets we have
seen so far have been:
Universities
Healthcare
CDN providers
Hosting providers
Start-ups
SaaS companies
Government
33 Confidential
Global Marketing
Vision for Complete
OpenStack Solution
Global Marketing
Complete Cloud Taxonomy
Software as a Service
IT as a Service
Platform as a Service
Infrastructure as a Service
Everything as a Service
Admin Software
SSO
Web Services & APIs
Customer Management
Information Service Management
Legacy Management
Security
IPS
Reporting
LDAP/AD
Entitlement, rights
Billing
Metering
Infrastructure Software
Ser Gov/Workflow
Automation
Monitoring
Workload Lifecycle
Management
Orchestration
Intelligent Resource Manager
Platform Provisioning
Abstraction Software
OS
Data Store
Hardware Virtualization
Operating System
Virtualization
Application Run-Time
Virtualization
Physical
Firewall
Analytics
Self Service Portal
Overarching
Systems
Compute
Switch
Storage
Network
HVAC
Power
Facility
Environmentals
Global Marketing
Disruption
Global Marketing
Discussion
Global Marketing