Transcript USGS/EROS Archive Preservation Status Update

USGS/EROS
Archive Preservation
Status
Update
Stuart Doescher, USGS
(Cheryl Greenhagen)
WGISS – 22
Sept 11- 15, 2006
Annapolis, MD USA
TM and MSS Archive Conversion
System (TMACS)
• Operations: 1992 to 2002.
• Number of High Density Tapes (HDT)
transcribed: 55,503
– Landsat MSS, TM
• Number of DCRSi Cassette Tapes (DCT)s
output: 3,827
• Mostly in Telemetry stream format
(Frame sync required to read)
WBVT summary
• Operated: April 1999 - May 2002
• Number of HDTs converted: 21,300
• Number of Digital Linear Tape (DLT 7000)
generated: 143 and an additional 143 as backup
• Format: BSQ ( MSS-X)
• "orphan" data (no DIAT / SLAT) save on 172 DLTs
• Bad input tapes: 476 (even with cleaning &
baking).
SPOT summary
• Operated: late April 2000 - February 2002
• Number of HDTs converted: 6,100
• Number of DLT 7000 tapes generated: 901
• Format: MDA's FRED
Landsat Archive Conversion
System (LACS)
• Planned Operations: June 2004 – December 2006
• Input: About 5500 HDTs, 4000 DCTs,
plus current Landsat 5 data
• Output: ~ 1100 tapes of "formatted" data
and ~ 1200 tapes of "raw" data
on 9940B. (~180G/tape)
• Format: BSQ
LACS Description
• Ingest current Landsat 5 data from disk files
• Ingest data from DCRSi Cassette Tapes (DCTs)
–
–
–
–
Multispectral Scanner Archive format (MSS-A) data
Multispectral Scanner Product format (MSS-P) data
Thematic Mapper Archive format (TM-A) data
Thematic Mapper raw (TM-R) data
• Ingest TM-R data from High Density Tapes (HDTs)
• Archive the data to a high-density computer-compatible
digital tape: 9940B
• Generate and archive browse imagery
• Generate inventory information for the new archive.
Data to be converted by LACS
Data Set
Number of
Scenes
Number of
Tapes
Data Volume
MSS-P
65,128
118 DCTs
3.2 terabytes
MSS-A
262,088
277 DCTs
9.5 terabytes
17,000
108 DCTs
3.6 terabytes
TM-R (1982 – 2005)
550,000
3,320 DCTs
140 terabytes
TM-R (1995 – 2002)
97,250
5,444 HDTs
27.0 terabytes
TM-A
TM-R (new)
~ 14 terabytes
1,031,762
Total
~200 terabytes
LACS Status
• Started Operations: June 2004
• Progress to Date: (July 31, 2006)
–
–
–
–
–
TM-R DCTs complete (3,320 DCTs, 555,000 scenes)
TM-R HDTs complete (5,444 HDTs, 112,000 scenes)
TM-A DCTs complete (108 DCTs, 17,000 scenes)
TM-R New Landsat 5 Data (86,000 scenes)
MSS-A started (21 DCTs, 17,000 scenes)
• Transcription rate will decrease as we handle
more data anomalies and problem tapes
• Estimated completion: December 2006
LACS Configuration
LCDS
DCS
LACS DB
DCS DB
LIS1
LACS1
LTS1: RCC Ingest
LIS1
Disk
LIS1: DB,
Browse / ACCA
LACS2
LIS2
LTS2: DCT Ingest
LIS2: Browse /
ACCA
LACS3
LACS3
Disk
LTS3: DCT Ingest
LIS3
LIS3: Browse /
ACCA
LACS4
LTS4: DCT Ingest
SAN
LAM
Local
nfs
SAN
DB link
SILO
LACS Status
• LTS (LACS Transcription Subsystem)
4 Strings:
– LTS1: Current Landsat 5 data
(acquired at EDC or GLC, or received on DLT from Australia)
– LTS2: DCT Input (2 shifts per day, 5 days a week)
– LTS3: DCT Input (2 shifts per day, 5 days a week)
– LTS4: DCT Input (2 shifts per day, 7 days a week)
LACS Status
• LIS (LACS Integration Subsystem)
Performs Automated Cloud Cover Assessment,
Browse Generation, Quality Assignment
Provides Interactive Assessment Capability
Gives priority to current Landsat 5 data
LIS work is distributed across 3 strings
LACS Status
• Issues / Problems:
– Data anomalies, tape degradation, cross-play issues
– DCRSi tape drives require frequent cleaning and
maintenance
• Plans:
– Complete tape transcription
– Port LTS from SGI to Intel (Linux)