Slide 1

Transcript Slide 1

OpenStack CE

Technology Review & Demo

Egan Ford IBM Distinguished Engineer [email protected]

3 Agenda

• IBM SmartCloud and OpenStack • Cloud Taxonomy • Some OpenStack Public Use Cases • What is OpenStack • OpenStack Resources • IBM Resources/Solutions for OpenStack Available Today • OpenStack (Video) Demo

Open architectures enable real innovation through interoperability

Evolve existing infrastructure to Cloud Accelerate adoption with expert integrated systems Immediate access to a managed platform Common Open Standards-based Cloud Management Services

IBM embraces & invests in open source to foster innovation Application Servers Service Oriented Architecture Cloud Computing Systems of Interaction Service Orientation

June 1998:

IBM enters into an engineering agreement with The Apache Group for development of the open-source Apache HTTP server software eventually becoming the leader of the new Application Server market

September 1999

: IBM capitalizes on an untapped market trend and begins participating in the community development of Linux with a $60M annual investment 5 Social Business

November 2001

: IBM rallies 150 influential vendors and the development community around a new tools environment with a $40 Million software donation disrupting the leadership of the software development ecosystem Open Cloud Architecture

September 2012

An open cloud architecture is emerging…built on open technologies OAuth cloudfoundry.org

TOSCA OSLC 6

Hardware

7 The OpenStack Foundation – IBM is a driving force in it’s success Software Defined Environmen t OpenStack is a global collaboration of developers & cloud computing technologists working to produce an

ubiquitous Infrastructure as a Service (IaaS) open source

cloud computing platform for public & private clouds.

Platinum Sponsors

APR 2012

Exponential growth in 1+ YR

150 Contributors 2600 Individuals 1021 Contributors Sep 2013 11,800+ Individuals

Gold Sponsors

IBM is working to accelerate OpenStack Foundation success LinkedData, OSLC TOSCA

OpenStack Open Source Reference

Because an open interoperable Cloud is critical for flexible cloud deployment and customer success…

12 IBM has 12 core contributors Cloud Standards Customer Council (CSCC) Cloud Computing Reference Architecture (CCRA)

IBM ’s ecosystem approach to an Open Cloud Architecture

IBM contributions to OpenStack are wide-ranging, contributing to OpenStack success and delivering real value

OpenStack Dashboard

Enables administrators and users to access & provision cloud based resources through a self-service portal.

• Globalization and crowd-sourced translation integration ( SOS ) • Cross hypervisor testing and validation ( CCS ) • • • • • • • •

OpenStack Compute

Provision and manage large networks of virtual machines Platform integration ( CCS ) High Availability enhancements ( RES ) Resource optimization ( RES ) Live upgrade contributions ( LTC ) Enablement for P & Z Systems, DB2 ( CCS, SOS ) ESXi support ( SOS w/ VMWare) VM group enablement in scheduler ( RES ) CPU allocation for vCPUs ( RES ) • • •

OpenStack Networking

Create petabytes of secure, reliable storage using standard HW Support for key emerging networking standards ( RES ) Quantum blueprints & migration from Nova ( LTC ) FibreChannel support ( RES )

OpenStack Object Store

Create petabytes of secure, reliable storage using standard HW • Block & object storage enablement for IBM capability ( RES ) • Nova blueprints ( LTC ) • Cinder local storage & local instance clone ( CCS ) • Efficient clone image in Cinder SVC driver for cFlex ( RES ) • Nova & Cinder storage blueprints ( CCS, SOS ) • Storwise/SVC driver update – support iSCSI CHAP auth ( SOS, RES ) • Wsgi application interface enabling external web server ( RES ) • Swift / Keystone interface for Keystone v3 API ( RES ) IBM CONFIDENTIAL • • • • •

OpenStack Shared Services

Libraries that provide image management, authentication & security across all OpenStack projects • Security & authentication enhancements ( CCS, SOS ) Image activation for OVF ( CCS ) Guest level metric collection ( CCS ) APIs: Enablement for key emerging standards ( SOS ) Membership services enhancements ( CCS, RES ) Glance: multiple image locations ( CCS )

General OpenStack contributions

• Drive IBM value-add capability from SCP ( CCS ) • Community facing contributions – bug fixing, community building & promotion ( LTC, SOS ) • QA items ( LTC ) CCS LTC RES – Common Cloud Stack (STG & SWG) – Linux Technology Center (STG) SOS – Standards & Open Source (SWG) – Research © 2011 IBM Corporation

IBM SmartCloud solutions are moving to an OpenStack-based infrastructure layer, enabling smooth migration and upgrade    

Simple 3 tier structure

, with increased Client Value at each tier Using

open, common, standards based architecture

providing choice, flexibility, interoperability, portability

Clean upgrade paths

with

progression

to fully integrated and factory optimized PureApplication System

Significant customer benefits above and beyond base OpenStack

Related Standards & Organizations CCRA CIMI & OVF OSLC TOSCA

SmartCloud Orchestrator

Orchestrate Services across multiple environments and domains

SmartCloud Provisioning

Automate Optimized Workloads

SmartCloud Provisioning

Automate Optimized Workloads Common Cloud Stack Factory Integrated Bundle Option

PureApp Server

Automate Optimized Workloads

SmartCloud Entry

Automate IT Delivery

SmartCloud Entry

Automate IT Delivery

PureApplication System Customer integrated hardware PureFlex System

Cloud Taxonomy

Source: http://it20.info/2012/02/the-cloud-magic-rectangle-tm /

Cloud Value Proposition and Positioning

Source: http://it20.info/2012/02/the-cloud-magic-rectangle-tm /

How You (Provider) Build These Clouds

Source: http://it20.info/2012/02/the-cloud-magic-rectangle-tm /

What You (Consumer) Get with These Clouds:

Source: http://it20.info/2012/02/the-cloud-magic-rectangle-tm/

Policy-based Clouds and Design-for-fail Clouds are purpose optimized Infrastructure Management solutions Policy-based Clouds • Purpose optimized for longer-lived virtual machines managed by Server Administrator • Centralizes enterprise server virtualization administration tasks • High degree of flexibility designed to accommodate virtualization all workloads • Significant focus on managing availability and QoS for long-lived workloads with level of isolation • Characteristics derived from exploiting enterprise class hardware • Legacy applications Design-for-fail Clouds • Purpose optimized for shorter-term virtual machines managed via end-user or automated process • Decentralized control, embraces eventual consistency, focus on making “good enough ” decisions • High degree of standardization • Significant focus on ensuring availability of control plane • Characteristics driven by software • New applications 15 © 2012 IBM Corporation

Some OpenStack Public Use Cases • • • • • • • • Internap http://www.internap.com/press-release/internap-announces-world%E2%80%99s-first commercially-available-openstack-cloud-compute-service/ • Rackspace Cloud Servers, Powered by OpenStack http://www.rackspace.com/blog/rackspace-cloud-servers-powered-by-openstack-beta/ • Deutsche Telekom http://www.telekom.com/media/media-kits/104982 • AT&T http://arstechnica.com/business/news/2012/01/att-joins-openstack-as-it-launches-cloud for-developers.ars

• MercadoLibre http://openstack.org/user-stories/mercadolibre-inc/mercadolibre-s-bid-for-cloud automation/ • NeCTAR http://nectar.org.au/ • San Diego Supercomputing Center http://openstack.org/user-stories/sdsc/ 16 © 2012 IBM Corporation

OpenStack design tenets focus on delivering essential infrastructure on an available, scalable, elastic control plane OpenStack Leadership's vision statement “essential Infrastructure, support platform” Basic Design Tenets 1) Scalability and elasticity are our main goals 2) Any feature that limits our main goals must be optional 3) Everything should be asynchronous. If you can't do something asynchronously, see #2 4) All required components must be horizontally scalable 5) Always use shared nothing architecture (SN) or sharding. If you can't Share nothing/shard, see #2 6) Distribute everything. Especially logic. Move logic to where state naturally exists.

7) Accept eventual consistency and use it where it is appropriate.

8) Test everything. We require tests with submitted code. (We will help you if you need it) Sources: http://www.openstack.org/downloads/openstack-compute-datasheet.pdf

OpenStack 18

Source: http://ken.pepple.info/openstack/2012/09/25/openstack-folsom-architecture/

OpenStack is comprised of seven core projects that form a complete IaaS solution IaaS IaaS

Compute (Nova) Storage (Cinder) Network (Quantum)

Provision and manage virtual resources

Dashboard (Horizon)

Self-service portal

Image (Glance)

Catalog and manage server images

Identity (Keystone)

Unified authentication, integrates with existing systems

Object Storage (Swift)

Compute delivers a fully featured, redundant, and scalable cloud computing platform Architecture

Key Capabilities:

•Manage virtualized server resources • CPU/Memory/Disk/Network Interfaces •API with rate limiting and authentication •Distributed and asynchronous architecture • Massively scalable and highly available system •Live guest migration • Move running guests between physical hosts •Live VM management (Instance) • Run, reboot, suspend, resize, terminate instances •Security Groups •Role Based Access Control (RBAC) • Ensure security by user, role and project •Projects & Quotas •VNC Proxy through web browser Sources: http://ken.pepple.info/openstack/2012/09/25/openstack-folsom-architecture/ http://openstack.org/projects/compute/ 20 © 2012 IBM Corporation

Compute management stack control plane is built on queue and database

Key Capabilities:

• Responsible for providing communications hub and managing data persistence • RabbitMQ is default queue, MySQL DB • Documented HA methods • ZeroMQ implementation available to decentralize queue • Single “cell” (1 Queue, 1 Database) typically scales from 500 – 1000 physical machines • Cells can be rolled up to support larger deployments • Communications route through queue • API requests are validated and placed on queue • Workers listen to queues based on role or role + hostname • Responses are dispatched back through queue 21 © 2012 IBM Corporation

nova-compute manages individual hypervisors and compute nodes 22

Key Capabilities:

• Responsible for managing all interactions with individual endpoints providing compute resource, e.g.

-- Attach iSCSI volume to phsyical host, map to guest as additional HDD • Implementations direct to native hypervisor APIs – Avoids abstraction layers that bring least common denomination support – Enables easier exploitation of hypervisor differentiators • Service instance runs on every physical compute node, helps to minimize failure domain • Support for security groups that define firewall rules • Support for – KVM – LXC – VMware ESX/ESXi (4.1 update 1) – Xen (XenServer 5.5, Xen Cloud Platform) – Hyper V © 2012 IBM Corporation

nova-scheduler allocates virtual resources to physical hardware

Key Capabilities:

• Determines which physical hardware to allocate to a virtual resource • Default scheduler uses a series of filters to reduce set of applicable hosts and uses costing functions to provide Weight • Not a focus point for OpenStack – Default implementation finds first fit – Shorter the workload lifespan, less critical the placement decision • If default does not work, often deployers have specific requirements and develop custom 23 © 2012 IBM Corporation

nova-api supports multiple API implementations and is the entry point into the cloud

Key Capabilities:

• APIs supported – OpenStack Compute API (REST-based) – Similar to RackSpace APIs – EC2 API (subset) – Can be excluded – Admin API (nova-manage) • Robust extensions mechanism to add new capabilities 24 © 2012 IBM Corporation

Network automates management of networks and attachments (network connectivity as a service) Architecture • • • • • • •

Key Capabilities:

Responsible for managing networks, ports, and attachments on infrastructure for virtual resources Create/delete tenant-specific L2 networks L3 support (Floating IPs, DHCP, routing) Moving to L4 and above in Grizzly Attach / Detach host to network Similar to dynamic VLAN support Support for • Open vSwitch • OpenFlow (NEC & Floodlight controllers) • Cisco Nexus • Niciria 25 © 2012 IBM Corporation

Cinder manages block-based storage, enables persistent storage 26 Architecture • • • • • •

Key Capabilities:

Responsible for managing lifecycle of volumes and exposing for attachment Structure is a copy of Compute (Nova), sharing same characteristics and structure in API server, scheduler, etc.

Enables additional attached persistent block storage to virtual machines Support for booting virtual machines from nova-volume backed storage Allows multiple volumes to be attached per virtual machine Supports following – ISCSI – RADOS block devices (e.g. Ceph distributed file system) – Sheepdog – Zadara © 2012 IBM Corporation

Identity service offers unified, project-wide identity, token, service catalog, and policy service designed to integrate with existing systems

Key Capabilities:

• Identity service provides auth credential validation and data about Users, Tenants and Roles • Token service validates and manages tokens used to authenticate requests after initial credential verification • Catalog service provides an endpoint registry used for endpoint discovery.

• Policy service provides a rule-based authorization engine and the associated rule management interface.

Image service provides basic discovery, registration, and delivery services for virtual disk images 28 References http://openstack.org/projects/image-service/

Key Capabilities:

• Think Image Registry, not Image Repository • REST-based APIs • Query for information on public and private disk images • Register new disk images • Disk images can be stored in and delivered from a variety of stores (e.g. SoNFS, Swift) • Supported formats – Raw – Machine (a.k.a. AMI) – VHD (Hyper-V) – VDI (VirtualBox) – qcow2 (Qemu/KVM) – VMDK (VMWare) – OVF (VMWare, others) © 2012 IBM Corporation

Dashboard enables administrators and users to access and provision cloud-based resources through a self-service portal 29 References http://horizon.openstack.org/intro.html

Key Capabilities:

• Thin wrapper over APIs, no local state • Registration pattern for applications to hook into • Ships with three central dashboards, a “User Dashboard”, a “System Dashboard”, and a “Settings • Out-of-the-box support for all core OpenStack projects • Nova, Glace, Switch, Quantum • Anyone can add a new component as a “first-class citizen ”.

• Follow design and style guide.

• Visual and interaction paradigms are maintained throughout.

OpenStack Resources • • • • • • • • • • Forums http://forums.openstack.org/ • Wiki http://wiki.openstack.org/ • Documentation http://docs.openstack.org/ • Mailing Lists http://wiki.openstack.org/MailingLists • OpenStack Project Management https://launchpad.net/openstack • Blogs http://planet.openstack.org

Real-time chat room • #openstack and #openstack-dev on irc://freenode.net (443 users currently logged in) • Rackspace Reference Architectures http://www.referencearchitecture.org/ • Easy Install http://www.hastexo.com/resources/docs/installing-openstack-grizzly-20131-ubuntu-1204-precise pangolin 30 © 2012 IBM Corporation

IBM Resources/Solutions for OpenStack CE Available Today • developerWorks • Google: openstack IBM developerworks • xCAT (FOSS) for 0-day deployment • xCAT OpenStack Paper (CATStack) • Automated qcow2 image creation for Glance • HW control • Bare-metal discovery and bring up •Firmware, Base OS, etc… • IBM Intelligent Cluster Solutions (see Matt Ziegler's PPT) • Preconfigured Switches • Rack and stacked and ready to go • Lab Services for 0-day 31 © 2012 IBM Corporation

IBM Resources/Solutions for OpenStack Available Today • All IBM System Software and Tools can coexist with OpenStack.

• Director, ASU, lflash, etc… • SoNAS for shared file (NFS, SMB) • XIV, v7000 for block storage (Cinder) • iDPX for scale-out Nova Compute and Swift • BNT switches for OpenFlow and Quantum • GPFS for iSCSI/block (Cinder) or file.

OpenStack Demo Setup Private Networks: eth0: 172.20.249/24 vm: 172.20.250/24 Control Nodes Compute Nodes 172.20.249.10

compute network scheduler volume console glance api os-essex0 10.0.9.10

compute network scheduler volume console glance api os-essex1 172.20.249.11

VM compute network VM 10.0.9.11

os-essex2 172.20.249.12

VM VM compute network 172.20.249.13

10.0.9.12

os-essex3 10.0.9.13

VM Firewall compute network 172.20.249.X

os-essexX 10.0.9.X

Slide 1

Transcript Slide 1

OpenStack CE

Technology Review & Demo

• IBM SmartCloud and OpenStack • Cloud Taxonomy • Some OpenStack Public Use Cases • What is OpenStack • OpenStack Resources • IBM Resources/Solutions for OpenStack Available Today • OpenStack (Video) Demo

Open architectures enable real innovation through interoperability

Cloud Taxonomy

Cloud Value Proposition and Positioning

How You (Provider) Build These Clouds

What You (Consumer) Get with These Clouds:

Directory