I N D I A N A U N I V E R S I T Y

TransPAC HPIIS Performance Review

Michael A. McRobbie

Vice President for Information Technology and CIO Indiana University October 25, 2000

I N D I A N A U N I V E R S I T Y

Overview

• HPIIS, TransPAC and APAN • The TransPAC Cooperative Agreement • Progress toward objectives • Science enabled by TransPAC

I N D I A N A U N I V E R S I T Y

Goals of the HPIIS program

• US NSF Program for High Performance International Internet Services (HPIIS), solicitation NSF97-106.

• “The goal is to provide the basis for the development of next generation applications supporting international collaborations and enabling interaction with global information and experimental resources.” • HPIIS connections link international high-performance production networks for international scientific, research and educational collaborations. Additionally, HPIIS networks enable global testbeds for new networking technologies.

– e.g. multicast, IPv6, QoS and web cache

I N D I A N A U N I V E R S I T Y

Historical Overview

• April 1997 HPIIS solicitation • August 1997 IU’s TransPAC response • February 1998 TransPAC Proposal revised • August 1998 Cooperative Agreement signed • August 1998 TransPAC operational

I N D I A N A U N I V E R S I T Y

Asia-Pacific Advanced Network (APAN)

• 14 member-nations cooperating to build a high performance research and education network across the AP region • 5 primary members – Australia, Japan, Korea, Singapore, USA • 2 associate members – China, Malaysia • 7 affiliates, liaison and other members – Hong Kong, Indonesia, Thailand, Philippines, Canada, Europe (Dante), Consultative Group on International Agricultural Research (CGIAR)

I N D I A N A U N I V E R S I T Y

TransPAC: Connecting APAN to the US

I N D I A N A U N I V E R S I T Y

STAR TAP Logical Map

TransPAC (OC-3)

I N D I A N A U N I V E R S I T Y

TransPAC Objectives • Provide leading edge connectivity between vBNS and APAN • High throughput production network for research and education • Testbed for new protocols, network services and applications

I N D I A N A U N I V E R S I T Y

TransPAC Consortium

• Indiana University (lead US organization) • APAN (APAN-JP lead organization) • National Science Foundation HPIIS program • Japan Science and Technology Agency (JST) • Kokusai Denshin Denwa, Co. Ltd (KDD) • AT&T • Korea Telecom • STAR TAP • NCSA and NPACI

I N D I A N A U N I V E R S I T Y

TransPAC Cooperative Agreement

I N D I A N A U N I V E R S I T Y

Cooperative Agreement

• TransPAC operates with funding from – The NSF – the Japan Science and Technology Agency – additional contributions from Indiana University • Operation is controlled by NSF CA ANI 9730201

I N D I A N A U N I V E R S I T Y

Terms of the Cooperative Agreement

• Primarily an agreement for services • 5 year term, renewable annually • Specific deliverables based on HPIIS program objectives • General notion of cooperating with other HPIIS funded projects • Review at the beginning of the third year – Changes in R&E networks – Impact on science

I N D I A N A U N I V E R S I T Y

Cooperative Agreement Deliverables

1. Provide and manage a direct, high-speed network between the vBNS and APAN 2. Allow access only to HPIIS approved institutions 3. Monitor the performance and use of the TransPAC connection 4. Cooperate with NLANR to develop and test new Internet protocols 5. Provide publicly-accessible information about TransPAC and the projects it enables 6. (a) In concert with NLANR provide consultative user services and (b) front-end tools for direct access to differentiated network services

U N I V E R S I T Y I N D I A N A

1. Provide and manage a direct, high-speed network between the vBNS and APAN

• Network infrastructure – Chicago-Tokyo link provided through annually renewable contract with AT&T and KDD – Peering at STAR TAP and the APAN Tokyo Exchange Point – Infrastructure jointly funded by the NSF, JST • Network engineering – Provided by IU, STAR TAP, and KDD staff – Funded by NSF, IU and KDD (in Tokyo) • Network operation and information centers – US side: Indiana University (Global NOC) – APAN side: APAN/KDD Tokyo XP • User support – US side: NSF and IU funded (IU and NLANR DAST staff) – APAN side: provided by the APAN community

I N D I A N A U N I V E R S I T Y

155

I N D I A N A U N I V E R S I T Y

I N D I A N A U N I V E R S I T Y

I N D I A N A U N I V E R S I T Y

Highlights

• Network and support staff jointly funded by NSF, JST and IU • TransPAC was up and on-line before the cooperative agreement was signed • Partnership with JST, AT&T and KDD have led to continuous improvements in service levels without additional NSF funding • Synergies within the HPIIS program and with Abilene/Internet2 have led to the Global NOC, run by Indiana University

I N D I A N A U N I V E R S I T Y

300 250 200 150 100 50 0

TransPAC “No-Cost” Bandwidth Improvements

OC-3 “southern route” (Q2/2001) AT&T consolidation upgrade (11/99) 73 Mbps JST upgrade (5/99) Initial 35Mbps 155 Mbps upgrade (10/00)

I N D I A N A U N I V E R S I T Y

I N D I A N A U N I V E R S I T Y

Leveraging the HPIIS TransPAC CA

• “First” circuit – NSF: $2M annually for 5 years (US half circuit) – JST: $3M annually (est.) for 5 years (JP half circuit) • “Second” circuit – JST: $2M annually through IU for 5 years based on MOU (US half circuit) – JST: $3M annually (est.) for 5 years (JP half circuit) • IU: $300K annually for 5 years • TOTAL: $51.5M over 5 years for $10M NSF investment

I N D I A N A U N I V E R S I T Y

TransPAC Peering (STAR TAP)

• vBNS • Abilene • ESNet • NREN/NISN • EuroLink • MirNet

I N D I A N A U N I V E R S I T Y

I N D I A N A U N I V E R S I T Y

2. Allow access only to HPIIS approved institutions

• Policy based routing restricts TransPAC traffic to HPIIS authorized institutions only • AP traffic segregation done at Tokyo XP • At STAR TAP only authorized networks can peer with TransPAC • Traffic segregation is effective

I N D I A N A U N I V E R S I T Y

HPIIS Networks Acceptable Use Policy

• HPIIS networks are provided for non commercial advanced research and education applications.

• Limited extensions are granted to the research arms of commercial organizations that participate in collaborative projects with non-commercial research and education institutions.

I N D I A N A U N I V E R S I T Y

I N D I A N A U N I V E R S I T Y

3. Monitor the performance and use of the TransPAC connection

• General traffic monitoring done at STAR TAP and Tokyo XP with MRTG and OC3Mon with graphs available on the TransPAC web site • BGP session monitoring at STAR TAP • Sysmon router exception logs available on-line • “Top talkers” display (from MirNet MADAS, soon) • Looking Glass servers available at both ends of the network • Test workstations available in Chicago and Tokyo for special monitoring applications

I N D I A N A U N I V E R S I T Y

4. Cooperate with NLANR to develop and test new Internet protocols

• TransPAC participates in MBone, 6Bone and web cache projects • TransPAC supports – native IPv6 (application example: telemicroscopy) – native multicast (many events and meetings are broadcast annually) – APAN web cache • Production diffserv QoS in the works – Several experiments and demonstrations in the US this year will help define production parameters

I N D I A N A U N I V E R S I T Y

5. Provide publicly-accessible information about TransPAC and the projects it enables

• Information avaiable through the TransPAC web site ( www.transpac.org

– Network status ): – Monthly and annual reports – Utilization, current and historical record – Descriptions of major projects and applications – AUP and application procedures – Problem reporting and tracking – User support contact information – Application development and tuning information – Engineering documents

I N D I A N A U N I V E R S I T Y

I N D I A N A U N I V E R S I T Y

6. In concert with NLANR provide:

• Consultative user services – DAST provides application tuning workshops for US TransPAC users and consulting to resolve performance problems. – TransPAC engineering staff actively engaged in NLANR activities including participation in 1999 and 2000 NLANR Joint Techs meetings – NLANR operates TransPAC’s OC3mon at the STAR TAP and makes traffic data available in the DataCube (moat.nlanr.net/Datacube) • Front-end tools for direct access to differentiated network services – Currently not a big issue due to bandwidth expansions (QoS by over-provisioning) – TransPAC is a focal point for coordination between I2 QBone effort and APAN QoS Working Group

I N D I A N A U N I V E R S I T Y

Enabling Science

I N D I A N A U N I V E R S I T Y

HPIIS and TransPAC Expected Outcomes

• Better infrastructure for international scientific collaborations • Qualitative and quantitative changes in research methodologies • New modes of communication between individuals and groups

I N D I A N A U N I V E R S I T Y

Unexpected outcomes

• Global standardization in data handling, aggregation and distribution, e.g.

– HEP: GriPhyN – Genomics: BioMirror and AP-BioNet – Astronomy: Sloane DSS • Much lower barriers to the use of “world class” instruments (e.g. Osaka electron microscope) • Greater interest in forming international collaborations

I N D I A N A U N I V E R S I T Y

Earth Observation

Data and Information Access Link (DIAL) DIAL is a web-based distributed system to search, access and visualize satellite remote sensing data for Global Change research. In collaboration with NASDA and other institutions, NASA has DIAL servers set up to distribute satellite remote sensing data. NASA and NASDA also collaborate on the Tropical Rainfall Measurement Mission (TRMM); 3D data is transferred from NASA to NASDA using TransPAC/APAN, processed and visualized for the web.

U N I V E R S I T Y I N D I A N A

High Energy Physics

Japan-US-Australia Collaboration in the Silicon Vertex Detector Project for the BELLE High Energy Physics Experiment at KEK

The BELLE detector is the state-of the-art detector to investigate CP violating phenomena with unprecedented precision at the KEK B meson factory. The CP (C=Charge conjugation, P=Parity) violation is a key to explain why the universe is dominated by the matter, not by the anti-matter. The primary goal of the BELLE detector is to identify the origin of the CP violation. The BELLE collaboration consists of more than 40 institutions from Japan, Korea, China, Taiwan, India, Russia, USA, Australia, and Europe.

I N D I A N A U N I V E R S I T Y

Scientific Instruments

Trans-Pacific Telemicroscopy Scientists at the Osaka University Research Center for Ultra High Voltage Electron Microscopy (UHVEM) and University of California San Diego National Center for Microscopy and Imaging Research (NCMIR) successfully use international advanced research networks to couple the world's largest and most powerful (3 million volt) transmission electron microscope at UHVEM to a remote-use computer pavilion set up at NCMIR.

http://www.npaci.edu/online/v3.10/telemicroscopy.html

I N D I A N A U N I V E R S I T Y

Scientific Instruments

Japan-US Collaboration in the Sloan Digital Sky Survey http://www.sdss.org/ Sloan Digital Sky Survey (SDSS) is a project to carry out imaging and spectroscopic surveys of half the northern sky using a dedicated, wide-field, 2.5-m telescope. The imaging survey with a large mosaic CCD camera will produce digital photometric maps of the sky in five color bands. These maps will be used to extract the position and various photometric parameters of about 100 million galaxies and close to the same number of stars. The SDSS is a collaborative project between the US and Japan involving seven US institutions and the Japan Promotion group (JPG). The JPG will produce merged pixel maps from flat-fielded data.

I N D I A N A U N I V E R S I T Y

Distributed Computing and Data Resources Maximum Likelihood Analysis Of Phylogenetic Data DNA data has accumulated more rapidly than compute power so researchers must often exclude potentially infor mative data to make statistical analysis practical. Utilizing the computationally intensive maximum-likelihood method of phylogenetic inference in a globally distributed collection of computational nodes, Indiana University, National University of Singapore and ACSys CRC in Australia have analyzed the DNA of cytoplasmic coat proteins, micro sporidia, and cyanobacteria.

http://www.indiana.edu/~rac/hpc/cp.html

I N D I A N A U N I V E R S I T Y

Global Distribution of Research Data Bio-Mirror: Sequence & Bioinformatic data Bio-Mirror is a world bioinformatic public service for high-speed access to up-to date DNA & protein biological sequence databanks. Mirror sites have been established in Australia, China, Japan, Korea, Singapore, Thailand, and the US. New data are propagated to all mirror sites as soon as they are made available, creating a uniform, universal and reliable data base for biological and medical research.

http://www.bio-mirror.net/

I N D I A N A U N I V E R S I T Y

Advanced Collaboration

Space Physics and Aeronomy Research Collaboratory The Space Physics and Aeronomy Research Collaboratory (SPARC) is an NSF-sponsored community resource for the upper atmospheric and space sciences; operating 24 hours a day for scientific collaboration and access to real-time and archival data.

http://sparc-1.si.umich.edu/sparc/central

I N D I A N A U N I V E R S I T Y

Conclusions

• HPIIS and TransPAC are excellent investments for US science that allow access to global facilities and expertise • The HPIIS program has set new standards for infrastructure in network-enabled international scientific collaboration • For many new projects this is critical infrastructure • TransPAC has been able to deliver 4-5 times the capabilities expected for the NSF’s original investment • Synergies between HPIIS projects built on the global network crossroads, the STAR TAP, have led to the establishment of an integrated global NOC at IU