Theory of Complex Networks John Doyle Control and Dynamical Systems Caltech Transportation Finance Health Commerce Our lives are run by/with networks Energy Consumer Emergency Manufacturing Information Utilities.
Download ReportTranscript Theory of Complex Networks John Doyle Control and Dynamical Systems Caltech Transportation Finance Health Commerce Our lives are run by/with networks Energy Consumer Emergency Manufacturing Information Utilities.
Theory of Complex Networks John Doyle Control and Dynamical Systems Caltech Transportation Finance Health Commerce Our lives are run by/with networks Energy Consumer Emergency Manufacturing Information Utilities Environment Health Transportation Finance Commerce Convergent Energy Emergency Networks Consumer Manufacturing Utilities Information Convergent networking: the promise Ubiquitous computing, communications, and control • that is embedded and intertwined • via sensors and actuators • in complex networks of networks, with layers of protocols and feedback. Resulting in: • Seamless integration and automation of everything • Efficient and economic operation • Robust and reliable services Environment Health Transportation Finance Commerce Energy Convergent Emergency Networks Consumer Manufacturing Information Utilities Convergent networking: the reality • Right now, back in Los Angeles, we can experience (in addition to smog, earthquakes, fires, floods, riots, lawyers,…) – Widespread and prolonged power outages from lightning strikes in Washington (or just “nonequilibrium market fluctuations”). – Widespread and prolonged flight delays from weather or ATC software glitches in Chicago or Atlanta. – Internet meltdowns caused by hackers in Moscow. – Financial meltdowns caused by brokers in Singapore. • What can we expect? – Widespread and prolonged meltdowns of integrated power, transportation, communication, and financial networks caused by lightning strikes in Singapore or a new release of MS Windows 2020? Elements of systems • Sense the environment and internal state • Extract what’s novel • Communicate or store what’s novel • Extract what’s useful • Compute decisions based on what’s useful • Take action • Evaluate consequences • Repeat We want results H A R D E R Data Is not novel information Is not useful Information Is not knowledge Is not understanding Is not wisdom Is not action Is not results Two great abstractions of the 20th Century 1. Separate systems engineering into control, communications, and computing – – Theory Applications 2. Separate systems from physical substrate • Facilitated massive, wildly successful, and explosive growth in both mathematical theory and technology… • …but creating a new Tower of Babel where even the experts do not read papers or understand systems outside their subspecialty. Tower of Babel • Issues for theory – Rigor – Relevance – Accessibility • Spectacular success on the first two • Little success on the last one, which is critical for a multidisciplinary approach to systems biology • Perhaps all three is impossible? • (In contrast, there are whole research programs in “complex systems” devoted exclusively to accessibility. They have been relatively “popular,” but can be safely ignored in biology.) Biology and advanced technology • Biology – Integrates control, communications, computing – Into distributed control systems – Built at the molecular level • Advanced technologies will do the same • We need new theory and math, plus unprecedented connection between systems and devices • Two challenges for greater integration: – Unified theory of systems – Multiscale: from devices to systems Compute Communications and computing Compute Act Sense Environment Computation Devices Devices Control Dynamical Systems From • Software to/from human • Human in the loop Compute To • Software to Software • Full automation • Integrated control, comms, computing • Closer to physical substrate Computation • New capabilities & robustness • New fragilities & vulnerabilities Devices Devices Control Dynamical Systems Theoretical foundations • • • • • Computational complexity: decidability, P-NP-coNP Information theory: source and channel coding Control theory: feedback, optimization, games Dynamical systems: dynamics, bifurcation, chaos Statistical physics: phase transitions, critical phenomena • Unifying theme: uncertainty management • Different abstractions and relaxations • Integrating these theories involves new math, much not traditionally be viewed as “applied,” e.g.. – Perturbation theory of operator Banach algebras – Semi-algebraic geometry Uncertainty management • Each domain faces similar abstract issues and tradeoffs, but with differing details: • Sources of uncertainty • Limited resources • Robust strategies • Fundamental tradeoffs • Ignored issues Control theory • Sources of uncertainty: plant uncertainty and sensor noise • Limited resources: sensing, actuation, and computation • Robust strategies: feedback control and related methods • Fundamental tradeoffs: Bode’s integral formula, RHP zeros, saturations, … • Ignored issues: communications in distributed control, software reliability Information theory • Sources of uncertainty: source and channel • Limited resources: storage, bandwidth, and computation • Robust strategies: coding • Fundamental tradeoffs: capacity, rate-distortion • Ignored issues: feedback and dynamics Computation complexity • Sources of uncertainty: intractability, problem instance • Limited resources: computer time and space • Robust strategies: algorithms • Fundamental tradeoffs: P/NP/Pspace/undecidable • Ignored issues: real-time, uncertainty in physical systems Software correctness • • • • • Sources of uncertainty: bugs, user inputs Limited resources: computer time and space Robust strategies: formal verification Fundamental tradeoffs: computational complexity Ignored issues: real-time, uncertainty in physical systems Multiscale physics • Sources of uncertainty: initial conditions, unmodeled dynamics, quantum mechanics • Limited resources: computer time and space, measurements • Robust strategies: coarse graining, renormalization?? • Fundamental tradeoffs: energy/matter, entropy, quantum, etc… • Ignored issues: robustness, rigor, computation, etc • (This looks mostly fixable.) Unified theory of uncertainty management • Sources of uncertainty: plant, multiscale physics, sensors, channels, bugs, user inputs • Limited resources: computer time and space, energy, materials, bandwidth, actuation • Robust strategies: ?? • Fundamental tradeoffs: ?? • Ignored issues: human factors Progress • Unified view of web and internet protocols – – – – – Good place to start Add feedback and dynamics to communications Observations: fat tails (Willinger) Theory: Source coding and web layout (Doyle) Theory: Channel coding and congestion control (Low) • Unified view of robustness and computation – Anecdotes from engineering and biology – New theory (especially Parrilo) – Not enough time today… Bonus! • “Unified systems” theory helps resolve fundamental unresolved problems at the foundations of physics • Ubiquity of power laws (statistical mechanics) • Shear flow turbulence (fluid dynamics) • Macro dissipation and thermodynamics from micro reversible dynamics (statistical mechanics) • Quantum-classical transition • Quantum measurement • Thus the new mathematics for a unified theory of systems is directly relevant to multiscale physics • The two challenges (unify and multiscale) are connected. Network protocols. HTTP Files TCP IP packets packets packets packets packets packets Routers web traffic Web/internet traffic Is streamed out on the net. Web servers Creating internet traffic Web client web traffic Is streamed out on the net. Web servers Creating internet traffic Let’s look at some web traffic Web client 6 5 Frequency (Huffman) (Crovella) 4 Cumulative Data compression WWW files Mbytes 3 Forest fires 1000 km2 2 (Malamud) 1 Los Alamos fire 0 -1 -6 -5 Decimated data Log (base 10) -4 -3 -2 -1 0 1 Size of events 2 6 Web files 5 Codewords 4 Cumulative Frequency -1 3 Fires 2 -1/2 1 0 -1 -6 -5 -4 -3 -2 -1 0 1 Size of events Log (base 10) 2 6 5 Frequency (Huffman) (Crovella) 4 Cumulative Data compression WWW files Mbytes 3 Forest fires 1000 km2 2 (Malamud) 1 Los Alamos fire 0 -1 -6 -5 Decimated data Log (base 10) -4 -3 -2 -1 0 1 Size of events 2 20th Century’s 100 largest disasters worldwide 2 10 Technological ($10B) Natural ($100B) 1 10 US Power outages (10M of customers) 0 10 -2 10 -1 10 0 10 2 10 Log(Cumulative frequency) 1 10 = Log(rank) 0 10 -2 10 -1 10 Log(size) 0 10 100 80 Technological ($10B) rank 60 Natural ($100B) 40 20 0 0 2 4 6 8 size 10 12 14 2 100 10 Log(rank) 1 10 10 3 2 0 1 10 -2 10 -1 0 10 10 Log(size) 20th Century’s 100 largest disasters worldwide 2 10 Technological ($10B) Natural ($100B) 1 10 US Power outages (10M of customers) Slope = -1 (=1) 0 10 -2 10 -1 10 0 10 6 Data compression WWW files Mbytes 5 4 Cumulative Frequency -1 3 Forest fires 1000 km2 2 -1/2 1 0 -1 -6 -5 Decimated data Log (base 10) -4 -3 -2 -1 0 1 Size of events 2 6 5 4 Cumulative Frequency Data compression WWW files Mbytes exponential -1 3 Forest fires 1000 km2 2 -1/2 1 0 -1 -6 -5 -4 -3 -2 -1 0 1 Size of events 2 0 0 10 1 10 2 3 10 10 10 .5 -1 10 -2 10 loglog 1 semilogy -3 10 exp -4 10 1 Plotting power laws and exponentials linear 0.6 0.2 20 40 60 80 100 6 Data compression WWW files Mbytes 5 exponential 4 Cumulative Frequency 3 Forest fires 1000 km2 2 All events are close in size. 1 0 -1 -6 -5 -4 -3 -2 -1 0 1 Size of events 2 6 5 4 Cumulative Frequency Data compression WWW files Mbytes -1 3 Forest fires Most2events 1000 km2 are small 1 0 -1/2 But the large events are huge -1 -6 -5 -4 -3 -2 -1 0 1 Size of events 2 6 5 4 Cumulative Frequency Data compression WWW files Robust Mbytes -1 3 Forest fires Most2events 1000 km2 are small 1 0 -1/2 Yet Fragile But the large events are huge -1 -6 -5 -4 -3 -2 -1 0 1 Size of events 2 Robustness of HOT systems Fragile Robust (to known and designed-for uncertainties) Fragile (to unknown or rare perturbations) Robust Uncertainties Large scale phenomena is extremely non-Gaussian • The microscopic world is largely exponential • The laboratory world is largely Gaussian because of the central limit theorem • The large scale phenomena has heavy tails (fat tails) and power laws Size of events x vs. frequency dP ( 1) p( x) x dx log(probability) log(Prob > size) log(rank) Px log(size) 0 =1 -1 1e3 samples from a known distribution: log10(P) 10 P( X x) 10 x -2 -3 Px -4 -1 0 1 10 10 x 2 x integer 3 4 log10(x) 5 =1 P( X x) Cumulative Distributions =0 Slope = - Noncumulative dP Densities =0 p( x) dx =1 Slope = -(+1) =1 Correct Cumulative Distributions Noncumulative Densities =0 =0 Wrong The physics view • Power laws are “suggestive of criticality” • Self-organized criticality (SOC) • Examples where this holds: – Phase transitions in lab experiments – Percolation models – Rice pile experiments • No convincing examples in technology, biology, ecology, geophysical, or socio-economic systems • Special case of “new science of complexity” • Complexity “emerges” at a phase transition or bifurcation “between order and disorder.” • Doesn’t work outside the lab. Data + Model/Theory 6 DC 5 WWW 4 3 2 1 SOC = .15 Forest fire 0 -1 -6 -5 -4 -3 -2 -1 0 1 2 SOC = .15 Cumulative distributions Noncumulative densities, logarithmic binning = .15 =.15 6 Web files 5 Codewords 4 Cumulative Frequency -1 3 Fires 2 -1/2 1 0 -1 -6 -5 -4 -3 -2 -1 0 1 Size of events Log (base 10) 2 The HOT view of power laws (w/ Jean Carlson, UCSB) • The central limit theorem gives power laws as well as Gaussians • Many other mechanisms (eg multiplication noise) yield power laws • A model producing a power law is per se uninteresting • A model should say much more, and lead to new experiments and improved designs, policies, therapies, treatments, etc. The HOT view of power laws • Engineers design (and evolution selects) for systems with certain typical properties: • Optimized for average (mean) behavior • Optimizing the mean often (but not always) yields high variance and heavy tails • Power laws arise from heavy tails when there is enough aggregate data • One symptom of “robust, yet fragile” HOT and fat tails? • Surprisingly good explanation of statistics (given the severity of the abstraction) • But statistics are of secondary importance • Not mere curve fitting, insights lead to new designs • Understanding design Examples of HOT fat tails? • • • • • • Power outages Detailed Web/Internet file traffic simulations Forest fires Commercial aviation delays/cancellations Disk files, CPU utilization, … Deaths or dollars lost due to man-made or natural disasters? • Financial market volatility? • Ecosystem and specie extinction events? • Other mechanisms, examples? Examples with additional mechanisms? • • • • • • • Word rank (Zipf’s law) Income and wealth of individuals and companies Citations, papers Social and professional networks City sizes Many others…. (Simon, Mandelbrot, …) Data 6 DC 5 WWW 4 3 2 1 0 -1 -6 -5 -4 -3 -2 -1 0 1 2 Data + Model/Theory 6 DC 5 WWW 4 3 2 1 0 -1 -6 -5 -4 -3 -2 -1 0 1 2 6 Cumulative Frequency 5 WWW files Mbytes 4 (Crovella) Most files are small (mice) 3 2 Most packets are in large files (elephants) 1 0 -1 -6 -5 Decimated data Log (base 10) -4 -3 -2 -1 0 1 Size of events 2 Router queues Mice Sources Network Elephants Router queues Mice Delay sensitive Sources Network Bandwidth sensitive Elephants BW = Bandwidth sensitive traffic Delay = Delay sensitive traffic Log(bandwidth) BW cheap Delay Expensive Log(delay) • We’ll focus to begin with on similar tradeoffs in internetworking between bandwidth and delay. • We’ll assume TCP (via retransmission) eliminates loss, and will return to this issue later. Bulk transfers BW (most packets) Log(bandwidth) Web navigation, voice (most files) Delay Log(delay) • Mice: many small files of few packets which the user presumably wants ASAP • Elephants: few large files of many packets for which average bandwidth will be more important than individual packet delay • Most files are mice but most packets are in elephants… •…which is the manifestation of fat tails in the web and internet. Bulk transfers BW (most packets) Log(bandwidth) Web navigation, voice (most files) Delay Log(delay) Claim I: Current traffic dominated by these two types of flows Claim II: Intrinsic feature of many future network applications Data 6 DC 5 WWW 4 3 2 1 0 -1 -6 -5 -4 -3 -2 -1 0 1 2 Data + Model/Theory 6 DC 5 WWW 4 3 2 1 0 -1 -6 -5 -4 -3 -2 -1 0 1 2 6 Cumulative Frequency 5 WWW files Mbytes 4 (Crovella) Most files are small (mice) 3 2 Most packets are in large files (elephants) 1 0 -1 -6 -5 Decimated data Log (base 10) -4 -3 -2 -1 0 1 Size of events 2 6 Data compression WWW files Mbytes 5 exponential 4 Cumulative Frequency All events are close in size. 3 2 1 0 -1 -6 -5 -4 -3 -2 -1 0 1 Size of events 2 Based on frequencies of source word occurrences, Select code words. To minimize message length. Source coding for data compression Source coding for data compression Objectives: • Optimally compress file • Tractable compression • Tractable decompression Shannon: • Optimally compress ensemble • Tractable compression • Tractable decompression Kolmogorov: • Optimally compress file • Undecidable compression • Intractable decompression • Surprise: natural and practical • Stochastic relaxation • Philosophically important • Turing, Godel, Chaitin, … Shannon coding Data Compression • Ignore value of information, consider only “surprise” • Compress average codeword length (over stochastic ensembles of source words rather than actual files) • Constraint on codewords of unique decodability • Equivalent to building barriers in a zero dimensional tree • Optimal distribution (exponential) and optimal cost are: length li log( pi ) pi exp(cli ) Avg. length = pl pi log( pi ) i i Shannon source coding Minimize expected length J source words with probabilities pi p l i i r 1 i length of codewords li 2 1 ri 1 l log(r ) i i li unique decodability Kraft’s inequality Codewords 0 100 10 101 1 110 11 11100 1110 2 li 1 11101 11110 111 1111 0 100 101 110 11100 11101 11110 11111 11111 2 li 1 ri 1 l log(r ) i i Kraft’s inequality = Prefix-less code Codewords 0 0 dimensional (discrete) tree 100 10 101 1 110 11 11100 1110 2 li 1 11101 11110 111 1111 cut in a 0-dim tree 2 li 1 ri 1 l log(r ) i i 0 100 101 110 11100 11101 11110 11111 11111 Kraft’s inequality = Prefix-less code Coding = building barriers Source coding 2 li Channel coding 1 Kraft’s inequality = Prefix-less code Channel noise Control = building barriers Minimize J pi li r 1 i Leads to optimal solutions for codeword lengths. With optimal cost l (r ) log(r ) li log( pi ) J pi log( pi ) Equivalent to optimal barriers on a discrete tree (zero dimensional). J pi li r 1 i J pi log( pi ) l (r ) log(r ) li log( pi ) • Compressed files look like white noise. • Compression improves robustness to limitations in resources of bandwidth and memory. • Compression makes everything else much more fragile: – Loss or errors in compressed file – Statistics of source file • Information theory also addresses these issues at the expense of (much) greater complexity length li log( pi ) pi exp(cli ) Data 6 5 How well does the model predict the data? DC 4 3 2 1 0 Avg. length = pl pi log( pi ) i i -1 0 1 2 length li log( pi ) pi exp(cli ) Data + Model 6 5 How well does the model predict the data? DC 4 3 Not surprising, because the file was compressed using Shannon theory. 2 1 0 Avg. length = pl pi log( pi ) i i -1 0 1 2 Small discrepancy due to integer lengths. Why is this a good model? • Lots of models will reproduce an exponential distribution • Shannon source coding lets us systematically produce optimal and easily decodable compressed files • Fitting the data is necessary but far from sufficient for a good model Web layout as generalized “source coding” • Keep parts of Shannon abstraction: – Minimize downloaded file size – Averaged over an ensemble of user access • Equivalent to building 0-dimensional barriers in a 1- dimensional tree of content document split into N files to minimize download time A toy website model (= 1-d grid HOT design) Optimize 0-dimensional cuts in a 1-dimensional document # links = # files More complete website models (Zhu, Yu, Effros) • Necessary for web layout design • Statistics consistent with simpler models • Improved protocol design (TCP) • Commercial implications still unclear Generalized “coding” problems • Optimizing d-1 dimensional cuts in d dimensional spaces… • To minimize average size of files • Models of greatly varying detail all give a consistent story. • Power laws have 1/d. • Completely unlike criticality. Web Data compression PLR optimization Minimize expected loss J pili ri R P: uncertain events with probabilities pi R: limited resources ri L: with loss li P DC source WWW user access L R codewords decodability files web layout document split into N files to minimize download time r = density of links or files l = size of files lr 1 d-dimensional li = volume enclosed ri = barrier density li , ri d pi = Probability of event d 1 li Resource/loss relationship: lr d PLR optimization J =0 =1 = “dimension” pili ri R data compression web layout l (r ) r c 1 d PLR optimization J =0 =0 is Shannon source coding pili ri R data compression log(r ) l (r ) c r 1 0 0 Minimize average cost using standard Lagrange multipliers J pili ri R Leads to optimal solutions for resource allocations and the relationship between the event probabilities and sizes. With optimal cost R c 1 pi 1 log(r ) l (r ) c r 1 0 0 c Rpi1 /(1 ) 1 li 1 /(1 ) p j j pi log(Rpi ) pi log( pi ) J 1 c 1 R p 1 p i i 0 0 Minimize average cost using standard Lagrange multipliers J pili ri R Leads to optimal solutions for resource allocations and the relationship between the event probabilities and sizes. With optimal cost R c 1 pi 1 log(r ) l (r ) c r 1 0 0 c Rpi1 /(1 ) 1 li 1 /(1 ) p j j pi log( pi ) 0 1 J 1 1 pi 1 1 0 To compare with data. c Rpi1 /(1 ) 1 li 1 /(1 ) p j j Forward engineering pi ri Reverse engineering li To compare with data. pi ri pˆ i c1 li c2 c Rpi1 /(1 ) 1 li 1 /(1 ) p j j (11/ ) Reverse engineering li plot l , Pˆ i sizes from data i compute using model Cumulative c Rpi1 /(1 ) 1 li 1 /(1 ) p j j Pˆi pˆ l k i pˆ i c1 li c2 (11/ ) i i li 1 Data 6 DC 5 WWW 4 3 2 1 0 -1 -6 -5 -4 -3 -2 -1 0 1 2 Data + Model/Theory 6 DC 5 WWW 4 3 2 1 0 -1 -6 -5 -4 -3 -2 -1 0 1 2 Typical web traffic Heavy tailed web traffic > 1.0 log(freq > size) p s- Is streamed out on the net. Web servers Creating fractal Gaussian internet traffic (Willinger,…) log(file size) 3 H 2 Fat tail web traffic time Is streamed onto the Internet creating long-range correlations with 3 H 2 Data + Model/Theory 6 DC 5 WWW 4 Are individual websites 3 distributed like this? 2 1 Roughly, yes. 0 -1 -6 -5 -4 -3 -2 -1 0 1 2 Data + Model/Theory 6 DC 5 WWW 4 How has the data changed3 since 1995? 2 Steeper. Consistent with more use of cross hyperlinks. 1 0 -1 -6 -5 -4 -3 -2 -1 0 1 2 More complete website models (Zhu, Yu, Effros) • More complex hyperlinks leads to steeper distributions with 1< < 2 • Optimize file sizes within a fixed topology: • Tree: 1 • Random graph: 2 • No analytic solutions The broader Shannon abstraction • Information = surprise… and therefore ignoring – Value or timeliness of information – Topology of information • Separate source and channel coding – Data compression – Error-correcting codes (expansion) • Eliminate time and space – Stochastic relaxation (ensembles) – Asymptopia • Brilliantly elegant and applicable, but brittle • Better departure point than Kolmogorov, et al What can we keep? • Separation: – Source and channel – Congestion control and error correction – Estimation and control • Tractable relaxations – Stochastic embeddings – Convex relaxations What must we change? • Add to information: – – – – Value Time and dynamics Topology Feedback • More subtle treatment of computational complexity • Naïve formulations intractable Log(bandwidth) achievable not Distortion Rate distortion theory studies tradeoffs between bandwidth and distortion from lossy coding. BW = Bandwidth sensitive traffic Delay = Delay sensitive traffic Log(bandwidth) BW cheap Delay Expensive Log(delay) • We’ll focus to begin with on similar tradeoffs in internetworking between bandwidth and delay. • We’ll assume TCP (via retransmission) eliminates loss, and will return to this issue later. Bulk transfers BW (most packets) Log(bandwidth) Web navigation, voice (most files) Delay Log(delay) • Mice: many small files of few packets which the user presumably wants ASAP • Elephants: few large files of many packets for which average bandwidth will be more important than individual packet delay • Most files are mice but most packets are in elephants… •…which is the manifestation of fat tails in the web and internet. Bulk transfers BW (most packets) Log(bandwidth) Web navigation, voice (most files) Delay Log(delay) Claim I: Current traffic dominated by these two types of flows Claim II: Intrinsic feature of many future network applications Router queues Mice Sources Network Elephants Router queues Mice Delay sensitive Sources Network Bandwidth sensitive Elephants Bulk transfers BW (most packets) Web navigation, voice (most files) Log(bandwidth) Claim (channel): We can tweak TCP using ECN and REM to make these flows co-exist. Delay Log(delay) Currently: Delays are aggravated by queuing delay and packet drops from congestion caused by BW traffic? Specifically: • Keep queues empty (ECN/REM). • BW slightly improved (packet loss) • Delay greatly improved (queuing) • Provision network for BW • “Free” QOS for Delay • Network level stays simple BW Log(bandwidth) The rare traffic that can’t or won’t will be expensive, and essentially pay for the rest. Delay Expensive Log(delay) Claim (source): Many (future) applications are natural and intrinsically coded into exactly this kind of fat-tailed traffic. BW Delay Expensive Log(bandwidth) Fat tailed traffic is “intrinsic” Log(delay) • Two types of application traffic are important: communications and control • Communication to and/or from humans (from web to virtual reality) • Sensing and/or control of dynamical systems • Claim: both can be naturally “coded” into fat-tailed BW + delay traffic • This claim needs more research BW Log(bandwidth) Abstraction I Delay Expensive Log(delay) • Separate source and channel coding • Source is coded into – Delay sensitive mice – Bandwidth sensitive elephants • “Channel coding” = congestion control Log(BW) Loss? Putting loss back into the picture Log(d) • Packet loss can be handled by coding (application) or retransmission (transport) • Need coherent theory to perform tradeoffs • Currently, congestion control and reliable transport are intertwined • What benefits would derive from some decoupling, enabled by ECN or other explicit congestion control strategies? Optimization/control framework • Application specific cost functions J(app,delay,loss,BW) (assume to be minimized) • Network resources:lines, routers, queues (energy, spectrum, deployment, repair, stealth, security, etc) • Comm/control network is embedded in other networks (transportation, energy, military action, …) • Robustness to uncertainties in users and resources • Need to flesh out details for future scenarios Optimization/control framework • Global optimal allocation sets lower bound on achievable performance • Control problem is to find decentralized strategies (eg TCP/IP) with (provably) near optimal performance and robustness in dynamical setting • Duality theory key to using network • Coding and control interact in unfamiliar ways • Naïve formulations intractable: – Computation intractable – Requires too much information not available to decentralized agents • Key is to find tractable relaxations Optimization/control framework • Pioneered by Kelly et al and extended by Low et al. • Ambitious goal: foundation for (much?) more unified theory of computation, control, and communications • Hoped for outcome: – – – – Rich theoretical framework Motivated by practical problems Yielding principled design of new protocols And methods for deploying and managing complex networks Scalable Congestion Control (Paganini, Doyle & Low ’01) ROUTING + DELAY x : source rates R f ( s) y : aggregate link flows LINKS SOURCES T q : aggregate prices per source Rb ( s) p : link prices Robustness, evolvability/scalability, verifiability Ideal performance Typical design IP Robustness Evolvability Verifiability Robustness of HOT systems Fragile Robust (to known and designed-for uncertainties) Fragile (to unknown or rare perturbations) Robust Uncertainties Feedback and robustness • Negative feedback is both the most powerful and most dangerous mechanism for robustness. • It is everywhere in engineering, but appears hidden as long as it works. • Biology seems to use it even more aggressively, but also uses other familiar engineering strategies: – – – – – Positive feedback to create switches (digital systems) Protocol stacks Feedforward control Randomized strategies Coding The Internet hourglass Applications Web FTP Mail News Video Audio ping napster Transport protocols TCP SCTP UDP ICMP IP Ethernet 802.11 Power lines ATM Optical Satellite Bluetooth Link technologies From Hari Balakrishnan The Internet hourglass Applications Web FTP Mail TCP News Video Audio ping napster Everything Transport protocols on IP SCTP UDP ICMP IP Ethernet 802.11 IP on Power lines ATM Optical everything Satellite Bluetooth Link technologies From Hari Balakrishnan Consumers, Applications Applications TCP/ IP Hardware Robust, yet fragile Robust Mesoscale Commodities, Hardware Uncertainty Robust Consumers, Applications Robust Mesoscale Uncertainty Commodities, Hardware Consumers, Applications Yet fragile Difficult to change Robust Mesoscale Commodities, Hardware Yet fragile Protocols allow for the creation of large complex networks, with rare but catastrophic cascading failures. Early computing Various functionality Software Digital Hardware Analog substrate Applications Software Modern Computing Operating System Hardware Hardware Varied functionality Robust mesoscale Uncertain substrate Various functionality Robust, yet fragile Digital Analog electronics Consumers Consumers Barter Money Commodities Commodities Investors Consumers Markets, Insitutions Money Investments Commodities The hourglass Garments Dress Shirt Slacks Lingerie Coat Scarf Sewing Cloth Wool Cotton Rayon Polyester Material technologies Nylon Tie Consumers Energy • • • • 110 V, 60 Hz AC Gasoline ATP, glucose, etc Proton motive force Energy Producers • Decentralized • Asynchronous Robust to: • Network topology • Application traffic • Delays, link speeds High performance Applications TCP/ IP Hardware Necessity: Essentially only one design is possible • Decentralized • Asynchronous Robust to: • Network topology • Application traffic • Delays, link speeds Applications TCP/ The existing design is incredible,IPbut… Hardware It’s a product of evolution, and is not optimal. High performance Necessity: Essentially only one design is possible Control Theory Computational Information Theory All design Theory of Complex systems? Complexity None Statistical Physics Dynamical Systems 1 dimension Control Theory All Computational Biology Information Theory design • Non-equilibrium • Highly tuned or optimized • Finite but large dimension Complexity None Statistical Physics Dynamical Systems 1 dimension Control Theory All • Integrated horizontally and vertically • Horizontal: control, communications, computing • Vertical: multiscale physics design None Computational Theory needs Information Theory • Status: nascent but promise results Complexity • Bonus: unexpected synergy Statistical Dynamical Physics Systems 1 dimension Control Theory Computational Information Theory All design • Ubiquity of power laws • High shear turbulence • Dissipation • Quantum/classical transition • Quantum measurement Complexity None Statistical Physics Dynamical Systems 1 dimension