Transcript Inference
Look Who’s Talking: Discovering Dependencies between Virtual Machines Using CPU Utilization HotCloud 10 Presented by Xin Problem with current cloud • Hard to monitor, manage and debug • Complex dependency causes butterfly effect We need to know VM Interdependencies • Better VM placement and migration decisions – Being not restricted to the physical machine, isn’t it what we want at the beginning? • Better resource allocation – Reduce intra cloud traffic. • Better disaster recovery automation – What can we do about this? • Better troubleshooting How to infer VM Interdependencies? • In a multi-tier application, VMs have request-response interactions • The server’s workload is determined by the clients workload How to infer VM Interdependencies? • Monitor – Sample ‘per VM’ CPU utilization • Model – Estimate an model for CPU utilization of each VM • Cluster – K-means clusters VMs with similar models VMs together Monitoring • CPU utilization sampled per VM • Sampling Period – Too small : increases computation Too large : Might miss relevant spikes • Sample size – 300 seconds Modeling • Auto Regressive modeling Modeling • If VM1 and VM2 have the same coefficients, we say they are interdependent • Why do we need this model? – Can we just compare the time series using a scale factor? Clustering Evaluation • 31 VMs spread over 5 physical servers • Applications – RUBiS : eBay like benchmark • 4VMs – Apache, Tomcat, MySQL and RUBiS client – Hadoop MapReduce Framework • 3 VMs – 1 master and 3 slave nodes – Iperf : Network testing tool • 2 VMs – sender and receiver Evaluation – 91.67% true positives – 99.08% true negatives Why it works ? • RUBiS – 100% accuracy – Lot of request-response interaction between the VMs – ‘n-tier’ application model Why it fails ? • Hadoop – false positive and false negative. – Mappers and reducers communicate intermediate results via files How does the system work? Discussion • Security concerns – Who can monitor the CPU utilization of all VMs? • The provider can ues “xentop” command to display realtime information about a Xen system – Can one VM infer the CPU utilization of another VM located at the same physical machine? • Infer from CPU cache utilization? – What can an attacker do when it detects the interdependencies of other VMs? • Attack the bottleneck • Estimate the workload Discussion • Is identifying VM interdependency important? – Yes, debugging, resource allocation – Is locating interdependent VMs on the same machine a good idea? • For provider: yes. For user: no – What can we do about disaster recovery? • What kind of disaster? • What causes it? • Can we rank the top 10 reasons? Is VM interdependence on the list? Discussion • Is CPU utilization a good metric? – No. What is a VM runs multiple applications? – No. We cannot infer who depends on whom. • What metrics can we use?