Transcript Big data
Evolution in Coming 10 Years:
What's the Future of Network?
- Big Data- Big Changes in the Network
WangHuan
Waseda GITS Nakazato lab
2013 2nd Oct
Outline
Big Data Introduction
Characteristics of Big Data
Challenges
Big Data Transmission Network
Big Data Network Future
Q/A
Who’s Generating Big Data
Mobile devices
(tracking all objects all the time)
Social media and networks
(all of us are generating data)
Scientific instruments
(collecting all sorts of data)
Sensor technology and
networks
(measuring all kinds of data)
The progress and innovation is no longer hindered by the ability to collect data
But, by the ability to manage, analyze, summarize, visualize, and discover knowledge
from the collected data in a timely manner and in a scalable fashion
3
How much data?
Google processes 20 PB a day (2008)
Wayback Machine has 3 PB + 100 TB/month (3/2009)
Facebook has 2.5 PB of user data + 15 TB/day
(4/2009)
eBay has 6.5 PB of user data + 50 TB/day (5/2009)
CERN’s Large Hydron Collider (LHC) generates 15
PB a year
640K ought to be
enough for anybody.
Characteristics of Big Data:
1-Scale (Volume)
Data Volume
44x increase from 2009 2020
From 0.8 zettabytes to 35zb
Data volume is increasing exponentially
Exponential increase in
collected/generated data
5
Characteristics of Big Data:
2-Complexity (Varity)
Various formats, types, and
structures
Text, numerical, images, audio,
video, sequences, time series, social
media data, multi-dim arrays, etc…
Static data vs. streaming data
A single application can be
generating/collecting many types of
data
To extract knowledge all these types of
data need to linked together
6
Characteristics of Big Data:
3-Speed (Velocity)
Data is begin generated fast and need to be
processed fast
Online Data Analytics
Late decisions missing opportunities
Examples
E-Promotions: Based on your current location, your purchase history,
what you like send promotions right now for store next to you
Healthcare monitoring: sensors monitoring your activities and body
any abnormal measurements require immediate reaction
7
Challenges
How to transfer Big Data?
Big Data Transmission
Data transmission with high-value-per-bit
SONET/SDH (Synchronous Optical Network)
Lower-cost-per bit technologies
Carrier Ethernet.
Networking switching is moving rapidly to
10-40 Gbps
Big data requires big networks.
Big Data- Network
However Ethernet on its own is not
sufficient to handle the torrent of Big Data.
Network architectures also need to change
to become flatter and more flexible.
Basic idea:
Push computing to data, not pushing
data to computing.
Limitations of Current Networks
Switches
11
Limitations of Current Networks
Old ways to configure a network
App
App
App
Operating
System
App
Specialized Packet
Forwarding Hardware
App
App
App
App
Operating
System
Specialized Packet
Forwarding Hardware
App
Operating
System
App
Specialized Packet
Forwarding Hardware
App
App
Operating
System
App
App
Specialized Packet
Forwarding Hardware
App
Operating
System
Specialized Packet
Forwarding Hardware
12
An OS for Networks
Towards an Operating System for
Networks
Software-Defined Networking (SDN)
Control Programs
Global Network View
Network Operating System
Control via
forwarding
interface
Protocols
Protocols
13
Q & A
Thank You for your attention!
Q/A