Transcript Document

Cluster Components
• Compute Server
• Disk Storage
• Image Server
Compute Server:
Stanford Research Computing Facility (RCF)
• Accessible only to Stanford faculty and authorized
sub users for their research
• 128 nodes provided by the provost
– 8 cores each (16 threads), 64GB RAM, Infiniband card
– Ivy Bridge V2 with 1866GHz memory
– 1024 cores, 2048 threads, 8TB RAM total in the initial
cluster
– Cluster re-evaluated every 2-3 years for
upgrade/expansion
Compute Server:
Stanford Research Computing Facility (RCF)
• 1PB of scratch space (not backed up)
– Must push data to the local storage prior to running jobs
– Home directories allocated to individual users (~1015GB/user), but may be combined among users within a
group
– Expansion units will be available for purchase later on
• Two additional nodes with 1.5TB RAM each
– Jobs running on these machines must meet minimum RAM
requirements or else they will be terminated and sent to
the general cluster
• Infiniband 56Gbits/sec Ethernet lines
Purchasing into the RCF: Hardware
• Purchasers will always have access to the main cluster
• Base system purchase
– 8 cores (16 threads), 64GB RAM (4GB/thread), Infiniband
card, $5000
• Custom systems can be purchased
– i.e., 8 cores (16 threads), 128GB RAM (8GB/thread),
~$7000
• Flexible configurations can be made to purchased
machines
– i.e., 4 active threads (12 “wasted” threads, 128GB RAM
with 32GB/thread)
– Can be changed as long as DIMM configuration remains
optimal
Purchasing into the RCF: Administration
• Can assign priority of jobs among authorized
users that have access to a purchased machine.
• Server administration is included in purchase
price. No annual fees.
• Purchase announcements made ~3 times per
year.
– May have to wait a couple months before purchased
machine is available.
• Other users’ access to purchased machines that
are idle is TBD
– Compliance/audit issues remain
Loading Software on the RCF cluster
• Can request to have software packages on the
general cluster
– More likely to be installed if widely used
– Matlab is on the list to be added, but is currently a
source of frustration
• Can install software on a purchased machine
– Recommended for software that is not widely used
(i.e., offline.recon)
– Need to evaluate current licenses for transfer to
purchased machines (i.e., COMSOL, SEMCAD, etc.)
RCF Timeline
• February
– Designated “friendly” users to begin testing
– One each from Rutt group and McNab group
• March
– Cluster becomes available for general use
• April-June?
– 1st round of purchased machines available
RCF Considerations
• May be advantageous to purchase early due
to our high RAM requirements
• Purchase costs may increase next year
• Unclear if purchased machines are available to
other users
• Job priority rules are still being decided
Compute Server Action Items
• Leaning towards RCF purchase of 2 x 128GB
systems to be used as a mini cluster.
– Jennifer’s group gets top priority on one machine and
Brian’s group gets top priority on other.
– Users from both groups are authorized to use both
machines.
• ASAP: Designate friendly users for RCF testing.
One each from Brian’s group and Jennifer’s
group.
– Jason/Simone + Grant Yang
Disk Storage
• Synology desktop NAS or rack-mounted
• RAID6 configuration
– Main Unit: 48TB capacity  40TB actual storage
– Expansion Units: 48TB capacity  48TB actual storage
• Must decide maximum expansion capacity prior
to purchasing main unit
– Main + 2xExpansion = 144TB system  136TB RAID6
– Main + 1xExpansion = 96TB system  88TB RAID6
Desktop vs. Rack Storage vs. Max Capacity
DESKTOP NAS SYSTEM
Main Unit
44TB Capacity (36TB RAID 6)
88TB RAID6 136TB RAID6
DS2413+
$1,900.00
DS3612xs
$3,000.00
WD Red 4TB
$2,268.00
WD Red 4TB
$2,268.00
$4,168.00
$5,268.00
DX1211
$1,300
DX1211
$1,300
WD Red 4TB
$2,268.00
WD Red 4TB
$2,268.00
EXPANSION UNIT COST
$3,568.00
$3,568.00
Maximum Expansion Units
MAX EXPANSION TOTAL
COST
COST PER TB
1
2
$7,736.00
$87.91
$12,404.00
$91.21
Main Unit Drives
12 Drives @ $189 each
MAIN UNIT INITIAL COST
Expansion Unit
44TB Capacity
Expansion Drives
12 Drives @ $189 each
RACK MOUNTED SYSTEM
Main Unit
44TB Capacity (36TB RAID 6)
88TB RAID6 136TB RAID6
RS2212+
$1,900.00
RS3614xs+
$5,000.00
WD Red 4TB
$2,268.00
WD Red 4TB
$2,268.00
$4,168.00
$7,268.00
RX1214
$1,700
RX1214
$1,700
WD Red 4TB
$2,268.00
WD Red 4TB
$2,268.00
EXPANSION UNIT COST
$3,968.00
$3,968.00
Maximum Expansion Units
MAX EXPANSION TOTAL
COST
COST PER TB
1
2
$8,136.00
$92.45
$15,204.00
$111.79
Main Unit Drives
12 Drives @ $189 each
MAIN UNIT INITIAL COST
Expansion Unit
44TB Capacity
Expansion Drives
12 Drives @ $189 each
Image Server
• Want a multi-user DICOM server and database
– Need a way for people to be able to view and pull
DICOM images remotely
– Adding all these users as recognized DICOM nodes
is cumbersome
– A web interface is an effective way to accomplish
this
OsiriX
• Has a fairly basic web layout
– Rudimentary search
– Organized by albums (which cannot be managed
through the website, I think)
– Excellent image viewer
– Can be customized to our heart’s content though
• Advanced user management with customized
access rights for each individual
• Try it out: http://171.65.72.243:3333/
– Just press login
OsiriX – Main Page
OsiriX – Album Page
• Download options
– “OsiriX remote transfer”
– Zip of DICOMs
– Weasis remote view
OsiriX – Exam Page
OsiriX – Series View