Transcript perfSONAR Installation and Configuration

Connect. Communicate. Collaborate
MODULE 1: perfSONAR TECHNICAL
Click to edit
Master title style
OVERVIEW
WHAT IS PERFSONAR?
perfSONAR is:
• A project consisting of a variety of organisations and
individuals
• A set of protocols that:
• Assume a set of services based on defined roles.
• Define their communication syntax and semantics.
• Allow anyone to develop an implementation of a service.
• A set of code
• Service implementations
2
WHAT ARE THE PERFSONAR SERVICES?
The perfSONAR services form an interoperable, distributed
performance measurement middleware framework.
perfSONAR stands for PERformance Service Oriented
Network monitoring Architecture.
3
DESIGN OBJECTIVES
perfSONAR is designed to be:
• Decentralised and Scaleable
• Large number of networks and services, large volume of data
• Each domain can set its own security policy
• Dynamic and ‘Self-Discovering’
• Add and remove components during operation
• Components ‘automatically’ become aware of one another
• Secure
• Will not put participating networks at risk of attack or congest them
• Modular
• Allows discrete module development
4
THREE-TIER ARCHITECTURE
The perfSONAR framework:
• Is middleware.
• Is distributed between domains.
• Facilitates inter-domain performance information sharing.
perfSONAR services ‘wrap’ existing measurement tools.
perfSONAR Visualization Tools
Domain D
perfSONAR
service
perfSONAR
service
perfSONAR
service
perfSONAR
service
perfSONAR
service
perfSONAR
service
Measurement
Tool
Measurement
Tool
Measurement
Store
Measurement
Tool
Measurement
Tool
Measurement
Store
Domain C
Key
Domain A
Domain B
= perfSONAR protocols
5
WHAT IS A SERVICE?
A Service is a tightly defined, independent entity that has a
well defined interface and can be accessed directly.
6
ARCHITECTURAL PRINCIPLES
Postulate: all measurement systems
contain a combination of:
• Measurement tools
• Data Storage
• Security and Policy implementation
• Topology information
• Visualization
Services have been identified that are:
• Based on these ‘roles’
• Based on requirement for other
functionality such as
Command line
tools
Measurements
in file system
Router
Measurement
Database
BWCTL
Router
• Service discovery, resource protection,
data formatting
7
THE PERFSONAR SERVICES FRAMEWORK (1)
perfSONAR divides measurement system tools into generic
‘families’ or ‘services’. Each service has a protocol.
Key
Enabling Services
= perfSONAR
service
Authentication
Services
Lookup
Services
Resource
protectors
Measurement
Points
Transformation
Services
Measurement
Archives
Measurement
Tools
= existing
measurement
tool or
measurement
data store
= Registration
Measurement
Stores
Performance Data Services
Domain
8
THE PERFSONAR SERVICES FRAMEWORK (2)
Client
Authentication
Service
Lookup
Service
Transformation
Service
Measurement
Point Service
Measurement
Archive Service
Resource
Protector Service
9
THE CLIENT
Client
Authentication
Service
Lookup
Service
Transformation
For example the perfSONAR
Service
Visualisation User Interface
Measurement
Point Service
Measurement
Archive Service
Resource
Protector Service
10
THE LOOKUP SERVICE
Client
Authentication
Service
Transformation
Registers services including their
Service
capabilities.
Lookup
Service
Facilitates complex searches.
Participates
in network of Lookup
Measurement
Services
Point Service
Measurement
Archive Service
Resource
Protector Service
11
THE LOOKUP SERVICE (1)
Purpose: all other services must register with the lookup
service in order to participate in the framework.
12
THE LOOKUP SERVICE (2)
Other services (including measurement points) register their
existence with a lookup service, by delivering ‘lookup
information’:
• Location
• Type of Service
Each domain has an instance of the lookup service
• These instances (will) communicate with one another
Clients find other services by querying the lookup service.
• All the client needs to know is the URL of a Lookup Service
13
THE AUTHENTICATION SERVICE
Client
Authentication
Service
Lookup
Service
Transformation
Service
Provides authentication for clients
and protects privacy.
Measurement
Point Service
Can be federated.
Measurement
Archive Service
Resource
Protector Service
14
THE AUTHENTICATION SERVICE
Purpose: ensures client-privacy and domain security by using
role-based authentication and authorisation.
15
THE MEASUREMENT POINT SERVICE
Client
Exposes measurement tools and
Authentication
publishes
their data.
Service
Transformation
Service
Measurement
Point Service
Lookup
Service
Measurement
Archive Service
Resource
Protector Service
16
MEASUREMENT POINTS
Measurement Points:
• Belong to domains (domain = a network)
• Each measurement point implementation maps to a tool that
provides one or several metrics
• Examples:
– One-way-loss
– Jitter
– TCP throughput
– Show commands on routers
17
THE MEASUREMENT ARCHIVE SERVICE
Client
Authentication
Exposes
measurement
databasesService
and file stores.
Publishes measurement data
Lookup
Service
Transformation
Service
Avoids queries to multiple
Measurement Point Services
Measurement
Point Service
Measurement
Archive Service
Resource
Protector Service
18
MEASUREMENT ARCHIVES
Purpose: expose measurement data held in databases or file
systems.
• Wrapper for any type of storage mechanism (SQL
Databases, RRD files, etc)
• Provides access to recent and stored data
• Collects information from Measurement Points, Transformation
Services or other Measurement Archives.
– i.e. it ‘subscribes’ to these other services
• Can also write to databases and file systems
19
THE TRANSFORMATION SERVICE
Transforms data in a
variety of ways (e.g.,
aggregation, filtering,
Authentication
correlation).
Service
For future
development. Precise
role needs to be
defined.
Client
Lookup
Service
Transformation
Service
Measurement
Point Service
Measurement
Archive Service
Resource
Protector Service
20
THE RESOURCE PROTECTOR SERVICE
Controls the comsumption of limited resources (e.g. network bandwidth).
Client
Authentication
Service
Lookup
Service
Transformation
Service
Measurement
Point Service
Measurement
Archive Service
Resource
Protector Service
21
THE PERFSONAR SERVICES FRAMEWORK (3)
Each service has a specific function.
Each instance of a service belongs to an administrative
domain.
22
PROTOCOLS
perfSONAR has developed a set of protocols for sharing performance
data. These:
• Assume the services set out in the framework.
• Define their communication syntax (schema) and semantics
(business logic).
• Allow anyone to develop an implementation of a defined service.
• Are compliant with the Global Grid Forum’s Network Measurement
Working Group (NM-WG) schema specification.
• Are based on XML over SOAP.
You can see the protocols as ‘rules and tools’ for participating in the
perfSONAR framework.
23
GENERIC SERVICES AND SERVICE IMPLEMENTATIONS
Imple
Measurement
Archive Service
ic fu
specif
lo
o
t
tion:
menta
added
nction
ality
Implementation: toolsp
ecific functionality
added
Common functionality,
common protocol
RRD
Measurement
Archive service
implementation
Round Robin
Database files
SQL
Measurement
Archive service
implementation
SQL database
Imple
Measurement
Point Service
Common functionality,
common protocol
ty
tionali
c
n
u
f
pecific
tool-s
:
n
io
t
menta
added
Implementation: toolsp
ecific functionality
added
BWCTL
Measurement
Point service
implementation
BWCTL
measurement
tool
SSH / Telnet
Measurement
Point service
implementation
SSH / Telnet
measurement
tool
24
SERVICE IMPLENTATIONS
The perfSONAR project has also developed a set of service
implementations that use the defined protocols.
• Some of these have been developed by JRA1, within the
GEANT2 project for the perfSONAR pilot:
• The Lookup Service
• The BWCTL Measurement Point
• The SSH / Telnet Measurement Point
• The RRD Measurement Archive
• The SQL Measurement Archive
25
DATA COLLECTION, NORMALISATION AND SHARING VIA
THE FRAMEWORK
Sy
da
BWCTL Tool
perfSONAR
SSH / Telnet
Measurement
Point (Web)
Service
Existing
Measurement
Tools
Measurement Data
SSH / Telnetspecific
Code
Tool-specific
commands
Measurement Data
BWCTL Toolspecific
Code
Standardised
Schema
Data normalization /
conversion to XML
Data normalization /
conversion to XML
Data normalization /
conversion to XML
perfSONAR
BWCTL
Measurement
Point (Web)
Service
Tool-specific
commands
Data normalization /
conversion to XML
St
an
ax
nt
Sy
da
rd
ed
is
is
ed
rd
Standardised
Schema
perfSONAR
Measurement
Framework
an
St
nt
ax
Other
perfSONAR
Services (e.g.
measurement
archive)
SSH / TELNET
26
FRAMEWORK FACILITATES NORMALISED END-TO-END
PERFORMANCE DATA (SIMPLIFIED DEPICTION)
Client (e.g. Visualisations Tool)
Network 3
Lookup
Service
RRD
Measurement
Archive
Authentication
Service
SSH / Telnet
Measurement
Point
Lookup
Service
SQL
Measurement
Archive
Authentication
Service
Network 4
SSH / Telnet
Measurement
Point
Data
Data
Network 1
Router
Network 2
Router
27
THE PERFSONAR PILOT: MEASUREMENT AND
ARCHIVING SERVICES
Network 2
RRD
Measurement
Archive service
implementation
SQL
Measurement
Archive service
implementation
BWCTL
Measurement
Point service
implementation
SSH / Telnet
Measurement
Archive service
implementation
Round Robin
Database files link utilisation
data
SQL database utilisation data
and path status
BWCTL
measurement
tool - available
bandwidth
SSH / Telnet
measurement
tool - router
commands
Network 4
Network 3
Network 1
Network 5
28
THE PERFSONAR PILOT
Key
Enabling Services
Authentication
Services
Lookup
Services
Service
implementations
included in pilot
Resource
protectors
Partial service
implementation
included in pilot
BWCTL & SSH
/ Telnet
Measurement
Points
Transformation
Services
Measurement
Tools
RRD & SQL
Measurement
Archives
Measurement
Stores
Performance Data Services
Service
implementation not
included in pilot
Existing
measurement tools
/ data stores
Domain
29
THE ROADMAP
Enhanced Lookup Service:
• Lookup Services (ideally one per domain) peer with one
another for increased ease of data discovery.
Authentication Service:
• Collaborating with JRA5 and Edugain to produce an
implementation of the service.
• A data subscriber (client) and a data producer (server) can
communicate directly across domains provided that they are
authorised to do so.
Further service implementations using Netflow information or
packet capture features
30
TECHNICAL OVERVIEW: SUMMARY
perfSONAR will provide an infrastructure to:
• Locate data sources
• Authenticate and authorise clients
• Protect resources and ration their usage
• Retrieve, normalise, transform and share data
• Only a partial infrastructure is implemented in the pilot
perfSONAR is a flexible and open framework:
• perfSONAR services can ‘wrap around’ existing data
collection tools
31
THE PERFSONAR SERVICES FRAMEWORK - RECAP
How do we expose measurement tools?
• Measurement point services
How do we expose measurement databases and file stores?
• Measurement archive services
How do we transform data (aggregate, correlate, filter etc.)?
• Transformation services
How do we locate all these services and their capabilities?
• Lookup Services
How do we protect resources?
• Resource Protection Services
How do we ensure a client is allowed to access a service?
• Authorisation and Authentication Services
32