A Linux-based Software Environment for the Reconfigurable Scalable Computing Project

John A. Williams 1 ([email protected]) Neil W. Bergmann 1 ([email protected]) Robert F. Hodson 2 ([email protected]) 1 The University Of Queensland, Australia 2 NASA Langley Research Center RSC MAPLD 2005/BOF-S Williams 1

RSC Platform Architectural Overview

• Collection of one or more modular stacks of computing elements • RPM is core reconfigurable component hosting reconfigurable FPGA fabric Williams 2 RSC MAPLD 2005/BOF-S

RSC Embedded Processing

• Primary target microprocessor is the MicroBlaze TM soft processor.

– Design mitigated with XTMR tool (or manually) • Embedded Linux – No MMU -> uClinux – Provides easy path to high level development for instrument applications (C, sockets, file systems, etc) – Development environment similar (if not identical) to typical Linux desktop Williams 3 RSC MAPLD 2005/BOF-S

RSC Software Environment

• Why Linux?

– Path for existing applications onto RSC – Standard platform improves design efficiency • Application development/debug • Multiprocessing/clustering • Software infrastructure – Interoperability • Networking • File systems – Desktop application prototyping “Linux is the C runtime” –

D. Jeff Dionne

Williams 4 RSC MAPLD 2005/BOF-S

Software Multiprocessing Model

• Message Passing Interface (MPI, MPI2) – Standardised protocol for message passing parallel computation – Strong uptake in terrestrial cluster computing community – Supports distributed (networked) clusters as well as shared memory machines • MPI on MicroBlaze and uClinux – Based on Argonne National Labs’ MPICH2 implementation – Start with MPICH on Linux TCP/IP stack – Migrate to higher performance implementation as RSC network architecture evolves Williams 5 RSC MAPLD 2005/BOF-S

Hardware Multiprocessing Model

• Reconfigurable Processing Module (RPM) – Application FPGA logic capacity (after TMR) • Two CPUs, support HW, system interconnect • Custom processing HW and IO cores – 512MB shared EDAC DRAM • Multiprocessing options – SMP Linux – Dual UP Linux (shared memory) – UP Linux + custom coprocessor – UP Linux + I/O processor – … Williams 6 RSC MAPLD 2005/BOF-S

Hardware Multiprocessing Model

Application FPGA (Xilinx) Timer / INTC/ … On-Chip Peripheral Bus CPU0 I/O core(s) Custom core Custom core Interface FPGA (Actel) Caches Bus I/F SLiP I/F On Chip Bus (Wishbone)

NIC PCI I/F

3.3V PCI 33MHz 32/64 bit CPU1 Timer / INTC/ … On-Chip Peripheral Bus Custom core Custom core I/O core(s) SDRAM Flash

Williams 7 RSC MAPLD 2005/BOF-S

Status and outlook

• OS and multiprocessing prototyping – COTS FPGA eval board • Insight-Memec V4LX25 + comms module • Dual ethernet, uart • 64MB DDR • UP Linux reference design completed • SMP feasibility study underway • Dual UP Linux – Dual MicroBlaze HW system built – Dual kernel bringup underway • MPICH2 port in progress – MPICH libraries integrated into uClinux build – Preliminary port of cluster process manager daemon Williams 8 RSC MAPLD 2005/BOF-S

Status and outlook

•COTS prototype cluster •4 x dual CPU subsystems Williams 9 RSC MAPLD 2005/BOF-S

Research questions

• Impact of TMR on performance • How to represent custom HW in an MPI cluster – Coprocessor to CPU nodes?

– Fully fledged MPI nodes / peers?

• Application of standard Linux technologies for reliability and survivability – RAID ramdisks – Cluster node failover • Performance modeling and analysis – Rob Jones, RSC Co-I Williams 10 RSC MAPLD 2005/BOF-S