MapReduce Programming in Clusters

Transcript MapReduce Programming in Clusters

Introduction to MapReduce

ECE7610

The Age of Big-Data

   Big-data age  Facebook collects 500 terabytes a day(2011)  Google collects 20000PB a day (2011) Data is an important asset to any organization  Finance company; insurance company; internet company We need new  Algorithms/data structures/programming model 2

What to do ? (Word Count)

 Consider a large data collection and count the occurrences of the different words

{web, weed, green, sun, moon, land, part, web, green,…}

Data collection Main WordCounter parse( ) count( )

web weed green sun moon land part 1 1 1 2 1 2 1

ResultTable DataCollection 3

What to do ?(Word Count)

Multi-thread Lock on shared data Main Thread Data collection WordCounter parse( ) count( ) DataCollection ResultTable

web weed green sun moon land part 2 1 2 1 1 1 1

What to do?(Word Count)

Data collection      Single machine cannot serve all the (file) system Large number of commodity hardware disks: say, 1000 disks 1TB each Critical aspects: fault tolerance + replication + load balancing, monitoring Exploit parallelism afforded by splitting parsing and counting Provision and locate computing at data locations 5

What to do? (Word Count)

Data collection Data collection Data collection Data collection Parser 1..* Main Thread Counter Data collection DataCollection WordList ResultTable

web weed green sun moon land part 2 1 2 1 1 1 1

Separate counters Separate data

KEY VALUE web weed green sun moon land part web green …….

It is not easy to parallel….

Different programming models Message Passing Shared Memory Fundamental issues Scheduling, data distribution, synchronization, inter process communication, robustness, fault tolerance, … Architectural issues Flynn ’ s taxonomy (SIMD, MIMD, etc.), network topology, bisection bandwidth, cache coherence, … Different programming constructs Mutexes, conditional variables, barriers, … masters/slaves, producers/consumers, work queues,. … Common problems Livelock, deadlock, data starvation, priority inversion, …dining philosophers, sleeping barbers, cigarette smokers, …

Actually, Programmer

’

s Nightmare….

MapReduce: Automate for you

    Important distributed parallel programming paradigm for large-scale applications.

Becomes one of the core technologies powering big IT companies, like Google, IBM, Yahoo and Facebook.

The framework runs on a cluster of machines and automatically partitions jobs into number of small tasks and processes them in parallel. Features: fairness, task data locality, fault-tolerance. 8

MapReduce

MAP: Input data  pair Data Collection: split1 Data Collection: split 2 Data Collection: split n Split the data to Supply multiple processors Map Map

web

weed green sun moon land part web green

web

… weed KEY green sun moon land part web green … KEY

1 1 1 part web 1 green 1 … 1 KEY 1 1 1 VALUE

1 1 1 1

web

weed green web green … 1 moon

1 1 1 1 1 1 1 1 1

1 1 1 1 1 1 part web … KEY 1 1 1 1 VALUE 9

MapReduce

MAP: Input data  pair REDUCE: pair  Data Collection: split1 Data Collection: split 2 Data Collection: split n Split the data to Supply multiple processors Map Map Map Reduce Reduce Reduce 10

Large scale data splits Map Parse hash Reducers (say, Count) Parse hash Parse hash Parse hash C. Xu @ Wayne State 11

MapReduce

How to store the data ?

Compute Nodes What ’ s the problem here?

Distributed File System

   Don ’ t move data to workers… Move workers to the data!

  Store data on the local disks for nodes in the cluster Start up the workers on the node that has the data local Why?

 Not enough RAM to hold all the data in memory  Network is the bottleneck, disk throughput is good A distributed file system is the answer   GFS (Google File System) HDFS for Hadoop 14

GFS/HDFS Design

       Commodity hardware over “exotic” hardware High component failure rates Files stored as chunks  Fixed size (64MB) Reliability through replication  Each chunk replicated across 3+ chunkservers Single master to coordinate access, keep metadata  Simple centralized management No data caching  Little benefit due to large data sets, streaming reads Simplify the API  Push some of the issues onto the client 15

GFS/HDFS

MapReduce Data Locality

  Master scheduling policy   Asks HDFS for locations of replicas of input file blocks Map tasks typically split into 64MB (== GFS block size)   Locality levels: node locality/rack locality/off-rack Map tasks scheduled as close to its input data as possible Effect  Thousands of machines read input at local disk speed. Without this, rack switches limit read rate and network bandwidth becomes the bottleneck.

MapReduce Fault-tolerance

 Reactive way  Worker failure • Heartbeat, Workers are periodically pinged by master – NO response = failed worker • If the processor of a worker fails, the tasks of that worker are reassigned to another worker.

 Master failure • Master writes periodic checkpoints • Another master can be started from the last checkpointed state • If eventually the master dies, the job will be aborted 18

MapReduce Fault-tolerance

 Proactive way (Speculative Execution)  The problem of “ stragglers ” (slow workers) • Other jobs consuming resources on machine • Bad disks with soft errors transfer data very slowly • Weird things: processor caches disabled (!!)   When computation almost done, reschedule in progress tasks Whenever either the primary or the backup executions finishes, mark it as completed 19

MapReduce Scheduling

   

Fair Sharing

 conducts fair scheduling using greedy method to maintain data locality

Delay

 uses delay scheduling algorithm to achieve good data locality by slightly compromising fairness restriction LATE(Longest Approximate Time to End)  improves MapReduce applications' performance in heterogenous environment, like virtualized environment, through accurate speculative execution

Capacity

 introduced by Yahoo, supports multiple queues for shared users and guarantees each queue a fraction of the capacity of the cluster 20

MapReduce Cloud Service

• • • Providing MapReduce frameworks as a service in clouds becomes an attractive usage model for enterprises. A MapReduce cloud service allows users to cost-effectively access a large amount of computing resources with creating own cluster. Users are able to adjust the scale of MapReduce clusters in response to the change of the resource demand of applications.

Amazon Elastic MR

0. Allocate Hadoop cluster 1. Scp data to cluster 2. Move data into HDFS EC2 3. Develop code locally You 4. Submit MapReduce job 4a. Go back to Step 3 Your Hadoop Cluster 5. Move data out of HDFS 6. Scp data from cluster 7. Clean up!

New Challenges

   Interference between co-hosted VMs  Slow down the job 1.5-7 times Locality preserving policy no long effective  Lose more than 20% locality (depends) Need specifically designed scheduler for virtual MapReduce cluster   Interference-aware Locality-aware 23