Transcript Charm++ Machine Layer

Introduction to
Charm++ Machine Layer
Gengbin Zheng
Parallel Programming Lab
4/3/2002
Machine Layer Overview


Startup and initialization;
Low level communication



CmiFreeSendFn, etc
CmiSyncBroadcastAllFn(), etc
SMP support


Multiple worker threads and one comm
thread;
Thread creation and state initialization;
Supported Platforms
Charm version
------------------net-linux
net-linux-ia64
net-axp
net-cygwin
net-win32
net-sol
net-irix
net-hp
net-ppc-darwin
mpi-linux
mpi-linux-ia64
mpi-axp
mpi-linux-axp
mpi-origin
mpi-sp
origin2000
t3e
OS
Communication Compiler
--------- ------------------- -------------------PC Linux
UDP/Myrinet
GNU compiler
IA64 Linux
UDP/Myrinet
GNU compiler
Alpha
UDP
GNU compiler
Win32/cygwin UDP
GNU compiler
Win32
UDP
MS Visual C++
Solaris
UDP
GNU compiler
IRIX
UDP
GNU compiler
HP-UX
UDP
GNU compiler
MacOS X
UDP
GNU C++ compiler
PC Linux
MPI
GNU compiler
IA64 Linux
MPI
GNU compiler
Alpha
MPI
GNU compiler
Alpha Linux
MPI
GNU compiler
Origin2000
MPI
C++ compiler
AIX
MPI
AIX xlC Compiler
Origin2000 shared-mem
SGI C++ compiler
Cray T3E
shared-mem
Cray C++ compiler
Communication Architecture
Communication
API (Send/Recv)
Net
MPI
UDP
TCP
Myrinet
(machine-eth.c)
(machine-tcp.c)
(machine-gm.c)
Shmem
Machine Layer Files Layout

Configuration files for each main platform:



conv-mach.h
conv-common.h
conv-mach-opt.h from:



smp, scyld, bluegene, gm, vmi, tcp …
cc, cxx, kcc, cc64, acc, ecc …
Implementations:


common.net, common.mpi, common.shmem
machine.c
How ‘build’ works

build charm++ net-linux gm smp kcc bluegene






Sort gm, smp and bluegene
Mkdir net-linux-bluegene-gm-smp-kcc
Cat conv-mach-[kcc|bluegene|gm|smp].h to
conv-mach-opt.h
Cat conv-mach-[kcc|bluegene|gm|smp].sh to
conv-mach-opt.sh
Gather files from common.net, etc (Makefile)
Make charm++ under

net-linux-bluegene-gm-smp-kcc/tmp
Parallel Program Startup

Net version - nodelist
Charmrun node
Rsh/ssh
(IP, port)
Rsh/ssh
(IP, port)
compute node
my node
(IP, port)
Broadcast all nodes
(IP, port)
compute node
Converse life

ConverseInit



Global variables initialization;
Start worker threads;
ConverseRunPE for each Charm PE


Per thread initialization;
Loop into scheduler: CsdScheduler()
Message formats

Net version
#define CMK_MSG_HEADER_BASIC { CmiUInt2 d0,d1,d2,d3,d4,d5,hdl,d7; }
Dgram Header

length
handler xhandler
MPI version
#define CMK_MSG_HEADER_BASIC
rank
root
handler xhandler
{ CmiUInt2 rank, root, hdl,xhdl,info,d3; }
info
d3
SMP support

MPI-smp as an example


Create threads: CmiStartThreads
Worker threads work cycle


See code in mahcine-smp.c
Communication thread work cycle

See code in machine-smp.c