Connecting to the Clouds

Cloud Brokers and OCCI.

Raffaele Di Fazio

Overview

• • • • The Broker, components and architecture jOCCI: an OCCI Java implementation The “Connector Project” A full overview

Raffaele Di Fazio

November 2012 - 2

Disclaimer

• • • My visibility on the projects was limited. I was just asked to design and develop with a general overview of the goals.

I do NOT have any details about European projects involved with the cloud broker nor any of the other parts I will show to you.

Raffaele Di Fazio

November 2012 - 3

The Broker

Raffaele Di Fazio

November 2012 - 4

The Broker: a cloud perspective

• • A cloud broker is an intermediary between the customer and a cloud provider.

• • • The goal of a Cloud Broker is: To be able to provide a unified interface to manage multiple cloud providers.

To simplify the choice for the customer.

To try to understand the

need

of the customer and to find the provider that best suits its needs.

Raffaele Di Fazio

November 2012 - 5

The Broker: a cloud perspective

• • • • Can have different goals based on the user request: Try to minimize the costs.

Try to give the fastest service (i.e. deploying in EU instead of US) The Broker, essentially: propose the user a service.

Raffaele Di Fazio

November 2012 - 6

Cloud Platforms

• • • At the beginning: one player (Amazon) Now: multiple players, variegated offer.

• • • • • Several cloud providers: Amazon VCloud HP Cloud Eucalyptus …

Raffaele Di Fazio

November 2012 - 7

Amazon Web Services

Raffaele Di Fazio

November 2012 - 8

HP Cloud

Raffaele Di Fazio

November 2012 - 9

OpenStack

• • • • • A scalable “operating system” for cloud computing.

It is used to control “compute, network, storage” resources. A “compute” is any information processing resource (i.e. a VM).

It can be used to control cloud infrastructures. It is a

software

suite.

This leads to multiple custom clouds ( see http://www.openstack.org/user-stories/)

Raffaele Di Fazio

November 2012 - 10

Differences btw cloud providers

• • • • • Interfaces APIs Costs Way of managing the VMs (i.e. different functionalities) Even the terminology used can be different.

• GOAL : simplify this mess!

Raffaele Di Fazio

November 2012 - 11

Broker + OCCI + Proactive

OCCI Server ProActive Cloud Provider Broker

Raffaele Di Fazio

November 2012 - 12

Broker + OCCI + Proactive: 2

nd

architecture

Cloud Provider OCCI Server Broker ProActive

Raffaele Di Fazio

November 2012 - 13

Broker + OCCI + Proactive: expanded

jOCCI OCCI Server PROCCI ProActive Broker Cloud connectors

Raffaele Di Fazio

November 2012 - 14

JOCCI: AN OCCI JAVA LIBRARY

Raffaele Di Fazio

November 2012 - 15

Dictionary… pay attention!

• • • • • OCCI has a particular naming convention we must follow.

There are very common words that are keywords when speaking about OCCI.

Kind, Link, Resource, Attributes, Category are some of them.

Other classes are: Parser, Server, Model.

Please be careful!

Raffaele Di Fazio

November 2012 - 16

OCCI

• • • OCCI: Open Cloud Computing Interface It is a standard for interoperability between cloud platforms.

• • • • GOALS: Interoperability: between cloud providers Portability: avoid vendor lock-in Integration: possibility to support multiple platforms Extensibility

Raffaele Di Fazio

November 2012 - 17

OCCI continued

• • • • Currently focus on IaaS.

An extension to support PaaS, SaaS is possible.

• • • It is a set of specifications: Core: Model specification Rendering: How to structure the OCCI requests … IN PRACTICE: RESTful protocol for cloud management tasks

Raffaele Di Fazio

November 2012 - 18

OCCI Core Model

Raffaele Di Fazio

November 2012 - 19

OCCI implementation – core

• • • Main classes reflect the OCCI core model: • • • Category: basic classes to identify OCCI types Kind: type identification system for all the Entities. Mixin: extension mechanism for a Kind.

Action: … • • Entity: abstract type of resource and link types Resource: represent a concrete resource and represents real world instances of VMs, etc.

Link: association between two resources.

Raffaele Di Fazio

November 2012 - 20

jOCCI

• • • It is a OCCI server written in Java.

• Born because of the necessity to replace occi4java Unsupported, unmaintainable, ugly code.

Inspired by the design of rOCCI, a ruby based OCCI implementation.

Raffaele Di Fazio

November 2012 - 21

jOCCI design goals

• • • • • Be extensible Model expressed by means of JSONs, defined at runtime.

Extendible, following OCCI principles.

No assumption on the backend (more about this later) Only supporting OCCI core model by default

Raffaele Di Fazio

November 2012 - 22

jOCCI implementation

• • • • Made up of two components: OCCI Core: contains the definitions of all the classes representing the OCCI core specifications OCCI Server: contains the server, the parser and all the other classes with functionalities to handle OCCI requests/responses. This makes the development of the specification (CORE) and the server, decoupled.

Raffaele Di Fazio

November 2012 - 23

jOCCI core – features

• • • jOCCI implements only the core classes of the OCCI Model.

• • Rough example: It defines what are the fields of the Kind class It does NOT define which types of Kind we can use (i.e. Compute) This approach makes jOCCI independent from a specific definition of the instances.

Raffaele Di Fazio

November 2012 - 24

jOCCI – Server

• • • • • Developed using the following technologies: Apache Tomcat RESTEasy Jackson (JSON parsing) Completely independent of the backend implementation. This is loaded at runtime (more about this later).

Raffaele Di Fazio

November 2012 - 25

jOCCI - Server

• • • • REST interfaces supporting HEAD, GET, POST, PUT, DELETE HTTP methods.

OCCI requests are HTTP requests that can be expressed in different formats.

Request formats: text/occi, text/plain, application/json, etc.

The server should be flexible enough to support these different types.

Raffaele Di Fazio

November 2012 - 26

Example of OCCI request

• curl -X POST --header 'Content-type:text/occi' --header 'X OCCI-Attribute: method=start' http://localhost:8080/jocci/compute/8432c343-15ec-40ff-bb19 1a7062a1aa5b?action=start --header 'Category: start; scheme=" http://schemas.ogf.org/occi/infrastructure/compute/acti on#"; class="action"' -u demo:demo

Raffaele Di Fazio

November 2012 - 27

Broker + OCCI + Proactive: an Expanded view

OCCI Server ProActive Broker jOCCI PROCCI Cloud connectors

Raffaele Di Fazio

November 2012 - 28

Connecting to ProActive:

PROCCI

Raffaele Di Fazio

November 2012 - 29

PROCCI

• • • Procci was created to be integrated with occi4java to allow ProActive to process OCCI requests.

These requests are mainly intended to be used to manage virtual machines. The final goal of this interconnection is to be able to speak with the broker.

Raffaele Di Fazio

November 2012 - 30

PROCCI

• • • • jOCCI parses OCCI requests.

Methods exposed by PROCCI are called directly by jOCCI.

Under the hood, a job is created and it is submitted to the scheduler. After the job is executed, a response is shown to the user through jOCCI by means of HTTP response.

Raffaele Di Fazio

November 2012 - 31

PROCCI Connection

• • • When jOCCI is started, it executes the setup phase. In each Action class attach the method implemented by the backend.

• This allow us to plug the actions to be executed.

Does not force jOCCI to know the backend.

Raffaele Di Fazio

November 2012 - 32

PROCCI connection

Raffaele Di Fazio

November 2012 - 33

THE CONNECTOR PROJECT

Raffaele Di Fazio

November 2012 - 34

Broker + OCCI + Proactive: an Expanded view

OCCI Server ProActive Broker jOCCI PROCCI Cloud connectors

Raffaele Di Fazio

November 2012 - 35

The connector project

• • • • We wanted to write several connectors to be able to connect to multiple cloud providers.

• Connectors are fundamental for the broker.

It must talk with the cloud service providers.

Technically they are built on top of APIs to connect to the cloud providers.

Not much more to say here! 

Raffaele Di Fazio

November 2012 - 36

A complete example

• • • • A user submit a OCCI request to

start

an already created virtual machine.

jOCCI receives the request. Once processed, it calls PROCCI.

PROCCI creates a job. The job is submitted to ProActive scheduler.

The job is executed. A remote cloud provider is called using a connector.

Raffaele Di Fazio

November 2012 - 37

That’s it! Thank you!

Raffaele Di Fazio

November 2012 - 38