Transcript A P2P Distributed Computing Platform: Korea@Home 2004.1.28(Wed)

A P2P Distributed
Computing Platform:
Korea@Home
2004.1.28(Wed)
C.Y. ‘Connor’ Park
[email protected]
KISTI Supercomputing Center
Overview
 Korea@Home (KatH) : A Software
Platform for Distributed Computing
using Volunteer Computer Resources
 Features
 Hybrid Peer-to-Peer architecture
 Secure PKI-based Security and Digital
Signature
 Web Services with SOAP messaging
2
Motivation
 To take advantage of and stimulate
‘Broadband Network Infrastructure in
Korea’
 To meet the growing needs of
computing power for applications
 Funded by ‘Ministry of Information and
Communication (MIC)’
3
Broadband Network Infrastructure
in Korea
 Over 10M Broadband network (xDSL, Cable)
households (over 70% penetration)
 Over 58.4% of PCs with Pentium3+
Own PCs
Not own PCs
100%
80%
23.1
21.4
100%
Internet-unable
17.7
10.7
82.3
89.3
2001.12
2002.12
80%
60%
40%
Internet-enable
– 2002.11
60%
76.9
78.6
40%
20%
20%
0%
0%
2001.12
2002.12
4
Similar Projects
 Science
 Internet
 SETI@Home,
climateprediction.net
 Life Science
 Grub, CycleTrader
 Financial
 MoneyBee
 Folding@Home,
FightAIDS@Home
 Cryptography
 Distributed.net,
RSAttack576
 Games
 ChessBrain
 Miscellaneous
 DALiWorld
 Mathematics
 Mersenne Prime Search,
Pi(x) Project
from http://www.aspenleaf.com/distributed/index.html
5
Project History
 2002. 4.
 MIC launched the pilot project ‘Korea@Home’
 Application : Virtual Screening for New Drug Design
 2002. 8.
 First pilot production and test
 Client/Server Architecture Model
 2002. 12.
 Second pilot production and test
 2003. 4.
 Regularize the project “P2P Internet-based Distributed
Computing Platform Development”
 2003. 9.
 Announcement of Hybrid P2P-style Distributed Computing
Platform
6
Architecture
Agent
Agent
Cooperation
Agent
Cooperation
Distribution
Distribution
Cooperation
Distribution
Agent
Internet
Distribution
Subscription
Job Request
Resource
Management
Large-scale
Application
7
Job Distribution
Agent
Agent
Agent
 Authentication
Internet
 Working Specification
Request
• Is there any job for
me to work?
• Who has the job
Agent
data?
 Getting data, working,
and returning result to
server
KatH Server
8
2002 Pilot Projects
 Application : Virtual Screening for New Drug
Design
 Client-Server Architecture
 An Agent
 Receives about 400KB-size file
 Executes during 20 minutes per working unit
 Returns about 190KB-size file
 Total number of working job units : 50,000
9
Virtual Screening
Target Protein
Chemical DB
Share
Analysis
Disease Protein
Clinical Demo New Drug Mass-produce
Ligand DB
Virtual High-Throughput Screening
+
+
Client 1
+
+
Client 2
Server
+
Calculating Active Site PMI
Calculating Ligand PMI
+
Looking for
Drug Candidate
Client 3
Internet
Structure Analysis
10
Experimental Results
 1st Test
 1,217 installed agents
 2002.8.22-9.2
 Max 53.5 Gflops / Avg. 31.78 Gflops
 2nd Test
 6,472 installed agents
 2003.2.6-2.14
 Max 644 Gflops / Avg. 470.6 Gflops
※ World Top Rank 124 - 629.7 Gflops:
IBM xSeries Cluster Xeon 2.8 GHz - Myrinet/ 192
(from http://www.top500.org, 2003.6)
11
Intention of Participation
Korea@Home Understanding
Don’t Know (33.3%)
Intention of Providing Resources
Providing with
Don’t (2.9%)
Compensation (7.2%)
Well-Understanding
(44.9%)
Keep up
Providing (59.5%)
Providing later (30.4%)
Understanding
roughly (21.8%)
• 67.7% of Netizen already know what is Distributed Computing and
Korea@Home.
• 89.9% of Netizen have intention of providing their PC resources.
12
2003 Progresses
 P2P data distribution
 New Applications
 Virtual Screening for New Drug Design
 Global Risk Management for financial
investment
 3D image rendering
 Supplementary plug-in
 Remote backup service
13
Futurework
 Make P2P agents more intelligent and





cooperative with each other.
A set of APIs for application developers
Fault-tolerance : checkpointing &
rollback recovery
IPv6 version
Open-Source (?)
Windows-only  Unix version
14
Thanks a lot! ^^
[email protected], http://www.koreaathome.org
15
Design Goals
 Application Developers (Job Requester)
 Convenience
 They should be able to use easily.
 Flexibility
 The platform is general enough to be used by
various applications.
 Correctness
 The platform should return correct results.
16
Design Goals
 PC agents (Resource Providers)
 Transparency
 PC users are not disturbed in their own works.
 Reliability
 It’s reliable not to make any security and privacy
problem.
 Reward
 There is any kind of compensation, even just in
mind.
17
Design Goals
 Platform
 Scalability
 Performance should be improved according to
the growth of agents.
 Adaptability
 It should put up with the dynamic features of
Internet.
18
Implementation Choices
Make server as simple as possible,
and Make PC agents more intelligent
 Web Services
 SOAP messaging over https
 Request-Response Model
 It is Agents’ duty to make most decisions.
 PKI-based certification and Digital
Signature for files
19
Functional Classification
 Resource Management
 Authentication, Certification
 Simple accounting
 Scheduling and Job distribution
 Job distribution with hybrid P2P style
 Result gathering
 Client Management




Client authentication
Application preparation to work
Digital signing
Result returning to client
20
Networking Issues
 Network Security
 DoS, Spoofing, Sniffing, …
 Variable Connection
 Dial-up, cable, xDSL, T1/E1, …
 IPv6, WiFi, Bluetooth, …
 Network Unobtrusiveness
 Background/shared communication
 Transparency (Firewall/NAT)
21