Transcript NOAA – DFC Interoperability

Solving IT Security Problems with iRODS
Alan Hall – NOAA’s National Climatic Data Center
July 8, 2015
NOAA – DFC Interoperability
• Ocean Observatory Initiative is tasked with
depositing climate data records in NOAA
archive
– OOI has installed an iRODS data grid
– NOAA National Climatic Data Center has installed
an iRODS data grid
• Federation of the two systems will simplify
ingestion of climate data records
– Provided security requirements can be met
DFC April 2013 NSF Review—5-5—1
iRODS Secure Ingest
DMZ Landing Zone: Open for data delivery
NCDC Internal Network
ftp1
ftp2
ftp4
ftp3
ftp5
DMZ Firewall
ftp
Tape
ingest1
ingest2
HDSS
Disk
Cache
FTP Load Balance
FTP PUSH/PULL
NCDC External Firewall
iRODS
FTP/FTPS
External Providers
Anonymous FTP is:
• Not Secure
• Not a management tool (clean-up)
• Limited in scope to one-to-one relationship
• Pushes data into the NCDC archive
DFC April 2013 NSF Review—5-4—2
iRODS Secure Ingest
NCDC Internal Network
DMZ Landing Zone: Open for data delivery
iRODS DMZ
ftp1
ftp2
ftp3
Grid
/DMZ
ftp4
ftp5
/Archive
/NR2
FTP /NR3
Load Balance
DMZ Firewall
ftp
iRODS NCDC
ingest1 Grid
/NCDC
HDSS
/Ingest
/Archive
ingest2
/NR2
/NR2
/NR3
/NR3
Tape
Disk
Cache
FTP PUSH/PULL
NCDC External Firewall
iRODS
FTP/FTPS
External Providers
iRODS is:
• Secure authentication
• Security via Obscurity (one to bind them)
• Uses a pull mechanism to move data into NCDC grid
• A virtual management tool (clean-up)
• Scope is entire grid
DFC April 2013 NSF Review—5-4—3
NCDC Cloud Pilot
NCDC Internal Network
DMZ Landing Zone: Open for data delivery
iRODS DMZ
ftp1
ftp2
ftp3
Grid
/DMZ
ftp4
ftp5
/Archive
/AWS-s3
/NR2
/NR2
FTP Load Balance
/NR3
DMZ Firewall
ftp
iRODS NCDC
ingest1 Grid
/NCDC
HDSS
/Ingest
/Archive
ingest2
/NR2
/NR2
/NR3
/NR3
Tape
Disk
Cache
FTP PUSH/PULL
NCDC External Firewall
iRODS
AWS S3
FTP/
FTPS
External
Providers
DFC April
2013 NSF
Review—
5-4—4
Cloud made easy:
• Easy set up with iRODS
• Connection to “cloud” is from the DMZ (Secure)
• Can synchronize from either DMZ or NCDC Grids
• End to End Data Management
• Initial copy to “cloud” resource
• Re-sync to “cloud” resource for failures
• Copy to Amazon Web Services (AWS) S3
National Science Foundation Cooperative Agreement: OCI-0940841