Transcript Social Cloud: Cloud Computing in Social Networks

4th GEOSS Science and Technology
Stakeholder Workshop
Social Cloud:
Facilitating “Trustworthy” Compute
& Data Resource Sharing
Presented by Daniel S. Katz*
*Work by Katz was supported by the National Science Foundation while working at the Foundation. Any opinion, finding, and conclusions or
recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.
Most work performed by:
Kyle Chard, Simon Caton, Omer Rana, Kris Bubendorfer,
Ioan Petri, Ivan Rodero, Magdalena Punceva, Manish Parashar
http://www.facebook.com/SocialCloudComputing
How does resource sharing happen today
Scenario:
A user wants to backup data, but doesn’t have the required amount of computational
resources/storage capacity
Required:
Where to get these
resources?
Buy
servers
Volunteer1,2
Available:
Grid
Cloud
1: Anderson 2002; 2: Stainforth 2002
Existing approaches all have certain shortcomings
Buy
servers
Data control
Costs, inflexibility
Volunteer
Willingness to provide
resources
Exchange not necessarily
bilateral
Grid
Access to (heterogeneous)
resources
QoS, user-friendliness, access
Cloud
Resources available on
demand
Trust assumptions
Use the (spare) capacity of friends and extend the Volunteer Computing approach
to bilateral and multilateral exchanges?
Many years of Cloud computing … and the same old hurdles
• Security (still a major limitation)
• Lack of Customisability
• Economics
– Small scale consumers have
requirements that may not match
providers offerings
– Providers have explicit incentives
to lock in consumers
– Still no true open cloud market
• Trust
– Always assumed at some level
– Can have anonymity (Marketbased/broker allocation)
• But has new levels of trust needed
– Many models fall apart when trust
is not assumed
The vision of a social cloud
• Definition: (based on Chard et al. 2011)
“Social Clouds are a scalable, dynamic and user-centric resource sharing
framework in which computational resources, services and information are
shared amongst members on the premise of the relationships encoded in a
social network.”
Socially-oriented Sharing Platform
Social Clouds enable the sharing of (heterogeneous) resources in a framework
where the social structures infer an implicit level of trust
Social Cloud – building on existing Social Network
Platforms
1,000,000,000 Users
On average 190
friends
Resources are idle 40-95%
Users contribute to “good” causes
Ubiquitous: Facebook ~ 1.3B users (Q2, 2014)
Represent mostly pre-existing real world relationships
Have notions of pre-existent trust fabric inherently interwoven into the network structure
Many applications now use social networks as a platform for:
Authentication e.g. Facebook Connect
Online Presence e.g. fb.com/your_page, Google Places (API)
Application Portals e.g. progress thru processors, ASPEN and PolarGrid project
Social Clouds ... Via Edge Devices, not Data Centers
• Cloud computing is built
around large datacenters
– Distributed, but still built
with capacity limits
• Expanding the boundaries
of clouds across edge
devices
• Devices can vary in scope:
– Set-top boxes
(from 500 GB to 1 TB of storage)
– Media Centres (Home PCs)
– Can use content distribution
networks (Netflix, HTTP, YouTube)
21.07.2015
7
Social Clouds: two aspects/research challenges
• Resource management
– Edge Devices – with
Peer-2-Peer-like
interaction
– Means of content
distribution
– Provides basis for a
“credits” and revenue
model
– Could be via social group
or facilitated by content
provider (e.g. Virgin
Media)
• Access
– Use of Social Media (e.g.
Facebook) app. to utilise
friends network
– Use trust relationships
within an existing social
network to discover
partners for interaction
– Use of a “virtual
currency” (credit tokens)
that facilitates trading
within a local community
Social Clouds: types of data
(e.g. podcasts, blogs, photos, music, documents)
• User generated content may be:
– Intermittent - often required at different time
intervals but not continuously
– Temporary - required for a short period of time
(e.g. processing memory for running an
experiment) and often only once
– Backup - required with the highest security
implications and privacy
– Working - can be accessed in real time and
continuously
Incentives
Volunteer
Reciprocal
Family, close friends
Trophy
Posted Price
Auction
Friends, Colleagues
Other People
Example:
• Non-monetary incentives
for close friends
• Monetary incentives for
other people
Different groups have different levels of perceived trust
Choice of interaction mechanism could depend on relationship type
Relationship to GEOSS
(http://www.earthobservations.org/geoss.php)
• Variety of data sets
– at different locations (“a network of content providers”)
• Linking “system of systems”
• Enable multiple types of specialist, application
specific, decision support
– Community provision of algorithms + “shim” (translation
services)
• Trust issues seem to be key
– Who: (i) generates, (ii) hosts; (iii) caches the data set
– Provenance issues associated with data
– Hosted on devices with varying capability (and reliability/
availability profiles) Similar requirements to Social Clouds
Investigations/Talk Structure
• Social Cloud platform
– Data Storage: Facebook app.
– Computational: Seattle-based VMs
– Use of Social Clouds to support Content Distribution (userside version of the “Cloud Files” implementation)
• Resource trading scenarios
– Eigentrust (direct interaction + recommendations and
feedback)
– Complementary (“virtual”) currency
• Trust relationships (incentive models)
– Game theoretic models (incentives to provide incorrect
feedback) + number of malicious peers
The Platform
Data Storage: Facebook app.
Platform Architecture
Resource Fabrics
Virtualisation & Sandboxing mechanism
Integration with Globus end points, Seattle VM, etc
Social
Marketplace
Socio-Technical
Adapter
Matching
with demand
Identity supply
verification
(e.g. OAuth)
Protocols
for
allocation,
Platform
Manager
Connect
to resource
various
“types”
of social
rules
of Overall
exchange,
information
store/registry
system
coordination
networks
(DBLP,
Facebook,
Foursquare,
etc)
Social Cloud: Platform for Data Storage (Demonstrator)
• Simple Storage Service Implemented as a Facebook application
• Use Case: a back up facility
Agreement
Kyle Chard, Kris Bubendorfer, Simon Caton, Omer F. Rana, “Social Cloud Computing: A
Vision for Socially Motivated Resource Sharing”. IEEE Transactions on Services
Computing 5(4): 551-563 (2012)
Posted Price
– Enables interactions based upon active trading/collaborative
decisions
– Intuitively facilitates reciprocal collaboration
– Current “norm” in industry solutions
Social Cloud
MDS
User ID URL Capacity
Price
User1
5
100 MB
Storage
Storage
User2
500 MB
10
User3
5 GB
7
Storage
Dynamic Auctions
• Auction:
– Enables dynamic participant pairing
– Sealed bid second price reverse auction
• Could be extended to any other auction mechanism
Application
Posted Price Scalability
• Varying the size of the MDS and number of matches
• With a size of 2000, 100 matches can be discovered
in ~ 2 seconds
The Platform
Computational: Seattle-based
VMs
Processor Sharing – via extended Seattle VMs
• Matching between users & owners
• Seattle (https://seattle.poly.edu/html/) – Open P2P platform
– Seattle “Clearing house” mechanism. 10 “vessels” (VMs) for each new install
– Node Manager: gatekeeper for resources deployed on every contributed
resource (credential checking for VM interaction)
– Host machine location (in a lookup service) + Public/Private keys generated
– Repy (Reduced Python for sandboxed environments)
Processor Sharing – via extended Seattle VMs
• Identify list of donation nodes
• Filter list based on “friends list” for a particular user
• Match mechanism
– Select consumer preferences for each friend
– Select preferences for each friend for requesting user
• Extends Seattle’s implementation of (pseudo)
random allocation to reduce user/donation
permutations
Simon Caton, Christian Haas, Kyle Chard, Kris Bubendorfer, Omer F. Rana, “A Social
Compute Cloud: Allocating and Sharing Infrastructure Resources via Social
Networks”. IEEE Transactions on Services Computing 7(3): 359-372 (2014)
The Platform
Use of Social Clouds to support
Content Distribution (user-side
version of the “Cloud Files”
implementation)
Social CDN – Use Case
• Enable content distribution through user-supplied
storage
– Home users (DSL-based)
– National Labs (dedicated networks) – often related to a
particular project (e.g. D0)
• Variable availability profiles
– Bandwidth throttling with DSL-based set up
– Establish a VPN between contributing sites (use of
SocialVPN)
• Identify:
– Number of replicas needed to ensure availability
• Served-based approach
– “CloudFiles” in RackSpace (OpenStack) – use of Akamai
CDN
A Social Content Delivery Network for Scientific Cooperation
Replica Placement:
•Random
•Node Degree:
highest no. of edges
•Community Node
Degree (highest
degree within a
community, i.e. no
adjacent placement)
•Clustering Coefficient
(similar to highest
betweenness scores)
Scenario and Community Representation
• Baseline Graph: DBLP publications graph (Kyle): 3 degrees (2009-10)
– Nodes: authors, Edges: coauthorship of 1 or more papers
•
•
•
•
26
Double co-authorship: at least 2 publications
No. of Authors: < 6 authors on the paper
Trust: captured through prior collaborative work
Having constructed a network, we assign replicas, and then test with
publications from 2011
Kai Kugler, Simon Caton, Kyle Chard, Daniel S. Katz, "On Replica Placement in a
Social CDN for e-Science,” 10th IEEE International Conference on eScience, 2014.
Results (at least 60 repetitions)
Double Coauthorship
40
No. of Coauthors
70
Random
Node Degree
35
Random
Node Degree
60
Community Node Degree
Community Node Degree
Clustering Coefficient
25
20
15
40
30
20
10
10
5
0
0
1
2
3
4
5
6
7
Number of Replicas
27
Clustering Coefficient
50
Replica Hit Rate (%)
Replica Hit Rate (%)
30
8
9
10
1
2
3
4
5
6
7
8
9
10
Number of Replicas
Kyle Chard, Simon Caton, Omer Rana and Daniel S. Katz, “A Social Content Delivery
Network for Scientific Cooperation: Vision, Design, and Architecture”, 3rd Int.
Workshop on Data Intensive Computation in the Cloud (DataCloud), in conjunction
with ACM/IEEE SC12 conference, Salt Lake City, November 2012
Data “Followers” & “Survivability” dynamics
(The “Data Wildfire”)
• Register interest in a data set
– Equivalent to “Like” (Facebook) and “Favorite”
(Twitter)
– Event generated on subsequent update on a data set
• Enable “interesting” data set to be propagated
– Equivalent to a “Share” (Facebook) and “Retweet”
(Twitter)
– Enables data sets with community interest to become
popular over time
• Can be useful as a basis to support resource allocation
Finding Brokers
Broker Emergence
• Finding suitable providers
– Centralized (registry/index)
– Distributed (flooding, gossip protocol, federated registry
services)
• Brokers can utilise both centralized and distributed
capability
– Brokers influence interaction dynamics in the network
– Predefine broker nodes at start up
• How can trading within a Social Cloud be enhanced
by dynamic emergence of “brokers”
– Which nodes could be more useful Brokers?
Take away message
• Dynamic selection of brokers based on their position in the
network
– Incentivise nodes to become broker based on increase in revenue
– Role adaptation in the network – buyers/seller  broker
– Use of “influence” metrics based on a social score
• Dynamic network properties may lead to limited benefit with
pre-defined brokers
• the social score algorithm generates a higher volume of trades
than the dominating set algorithm.
• The performance differences of Social Score vs Dominating set
are determined by two aspects:
– Graph properties and
– Choice of evaluation metrics.
Ioan Petri, Magdalena Punceva, Omer F. Rana, George Theodorakopoulos: Broker
Emergence in Social Clouds. IEEE CLOUD 2013, San Jose, CA, USA, pp 669-676
Incentive Models
Incentives & Trading
In Conclusion
• Social Clouds provide an important user-driven
alternative to data-center based Clouds
– E.g., Wuala networks (NL), AmazingStore (China), etc
• Issues of Trust, Reputation and Economic incentives
are key
– Include other factors: availability, reliability, uptime, power
usage, etc.
– Softer than traditional Service Level Agreement model
• Current focus: Broker “emergence” in Social Clouds
– Identify dominating sets in a social graph
– Implementation using CometCloud
Further Reading
D. Neumann, C. Bodenstein, O. F. Rana, R. Krishnaswamy, ”STACEE: Enhancing Storage Clouds
using Edge Devices”.IEEE/ACM Workshop on Autonomic Computing for Economics (ACE 2011)
alongside ICAC 2011, Karlsruhe, Germany, 14 June 2011. ACM Press.
Kyle Chard, Kris Bubendorfer, Simon Caton, Omer Rana, “Social Cloud Computing: A Vision for
Socially Motivated Resource Sharing,” IEEE Transactions on Services Computing, 2011.
Ioan Petri, Omer Rana, Gheorghe Cosmin Silaghi: “SLA as a Complementary Currency in Peer-2Peer Markets”, Proceedings of GECON 2010: 141-152, Springer Verlag.
“Trust Modelling and Analysis in Peer-to-Peer Clouds” Ioan Petri, Omer Rana, Yacine Rezgui, and
Gheorghe Cosmin Silaghi, Journal of Cloud Computing, Inderscience, 2012
Omer Rana and Simon Caton, ”Business Models for On-line Social Networks: Challenges and
Opportunities”, International Journal of Virtual Communities and Social Networking, JulySeptember 2010, Vol.2, No.3, pp 31-42
Ioan Petri, Omer F. Rana, Gheorghe Cosmin Silaghi: Service level agreement as a complementary
currency in peer-to-peer markets. Future Generation Comp. Syst. 28(8): 1316-1327 (2012)