Transcript Hubble Legacy Archive Lee Quick - TIPS meeting July 19, 2012 • • • • • Goals Data History Current Work Demo.

Hubble Legacy Archive
Lee Quick - TIPS meeting July 19, 2012
•
•
•
•
•
Goals
Data
History
Current Work
Demo
Goals of the HLA Project
–
–
–
Provide intuitive searching, browsing and download of Hubble data
Create online, enhanced data products for all Hubble data
Develop enhanced user interface capabilities
– hla.stsci.edu
Hierarchy of HLA Image Products
•
Level 1 data are the individual exposures, projected
("drizzled") onto a common frame for each visit.
•
Level 2 data are the combined images (exposures with the
same filter, same camera, and within the same visit).
•
Level 3 images are deep combined images or mosaics
creating by combining data taken in different visits to the
target.
•
Level 4 data are false color images (a combination of two or
three colors, depending on what is available).
•
Level 5 data are the community-contributed high-level
science products (HLSP).
•
Level 0 (calibrated, unprocessed) are not available for direct
searches but may often by found using the More... link.
HLA - Data Holdings
HLA (a brief) History
EDR
82007
DR2
DR4
DR6
92008
32010
12012
DR1
DR3
DR5
22008
52009
32011
HLA Early Data Release - August 2007
EDR 8-2007
DR2
DR4
DR6
9-2008
3-2010
1-2012
DR1
DR3
DR5
2-2008
5-2009
3-2011
• Data online for immediate
access
• Footprint service to make it
easier to browse and
download
• Interactive image display
• HLA combined images for
~25% of ACS data
• Prototype ACS source lists
• NICMOS grism extractions
for ~10% of G141 data
(produced at ST-ECF)
HLA Data Release 1 – February 2008
DR2
9-2008
EDR 8-2007
DR1
2-2008
•
•
•
•
•
•
•
•
•
•
•
•
•
•
DR4
3-2010
DR3
5-2009
NEW DATA PRODUCTS
HLA combined images for ~90% of ACS data
NICMOS grism extractions for ~80% of available data
(produced at ST-ECF)
DAOphot source lists for ~50% of ACS images
Prototype SExtractor source lists for ~10% of ACS images
A few prototype combined WFPC2 images (produced at
CADC
USER INTERFACE ENHANCEMENTS
Footprints for ACS, WFPC2, STIS, NICMOS, FOS, and GHRS
STIS images accessible in HLA interactive display
Non-HLA data products included in inventory and images,
accessible through DADS
Ability to filter inventory table to select observations of
interest
Readout of mouse RA/Dec position in interactive display
Plot cuts of rows or columns through images (or 2-D
spectra) in interactive display
Introductory training movies on how to use the HLA
DR6
1-2012
DR5
3-2011
HLA Data Release 2- September 2008
DR2
9-2008
EDR 8-2007
DR1
2-2008
•
•
•
•
•
•
•
•
•
•
•
DR4
3-2010
DR3
5-2009
NEW DATA PRODUCTS
Combined HLA images for ~95% of WFPC2 data
(produced at CADC)
Combined HLA images for ~95% of ACS
observations
Both DAOphot and SExtractor source lists for
70% of ACS observations
USER INTERFACE ENHANCEMENTS
(Shopping) Cart to allow users to download
multiple files at once
Enhanced plotting capabilities
Additional catalog overlays in interactive display
(SDSS, 2MASS, GSC2, FIRST)
User defined search lists
Much faster footprint interface
Additional advanced search options (Proposal
ID, Moving Targets, Filter)
DR6
1-2012
DR5
3-2011
HLA Data Release 3 – May 2009
DR2
9-2008
EDR 8-2007
DR1
2-2008
•
•
•
•
•
•
•
•
•
•
•
•
•
DR4
3-2010
DR3
5-2009
NEW DATA PRODUCTS
HLA drizzled combined images for ~30% of NICMOS data
Sample of 1235 ACS extracted grism spectra (provided by STECF)
Preliminary DAOphot and SExtractor source lists for ~80% of
WFPC2 images
Sample prototypes of ACS multi-visit mosaics
USER INTERFACE ENHANCEMENTS
User search list positions are accepted in a variety of input
formats
Selected datasets (e.g., from clicking on footprints) can be
added to cart
Selection of datasets can be used for sorting and filtering
(e.g., to show selected data at the top of the table)
Cart can be used to retrieve both DADS and HLA data
products
Cart includes estimated download data volume
GALEX catalog is available for overlay in interactive display
Advanced contrast controls in interactive display can be used
to change color balance
DR6
1-2012
DR5
3-2011
HLA Data Release 4 – March 2010
DR2
9-2008
EDR 8-2007
DR1
2-2008
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
DR4
3-2010
DR3
5-2009
NEW DATA PRODUCTS
Fully reprocessed NICMOS data
Fully reprocessed WFPC2 data (produced at CADC) with new source
lists
Access to WFC3 Early Science release data
New high-level science products
Additional ACS multi-visit mosaics
Updated ACS combined images
USER INTERFACE ENHANCEMENTS
Plotting tool allows for more dynamic user interaction, including
zooming and panning
Spectra from FOS and GHRS are displayed in the plotting tool using the
"Interactive Display" links in the inventory and images views.
COS and WFC3 observations are included in the search results and
footprints
Proprietary data are included in the search results plus footprints and
can be retrieved from DADS (by privileged users) via the HLA cart
Search by IPPPSOOT exposures and HST visit names (propid_visit)
Inventory table updates
Consolidated Help Center holds HLA resources such as FAQ, Help,
Movies etc., and includes a new HLA documentation search tool.
Source lists are dynamically generated from our databases, which
improves consistency between the various catalog sources and makes
it possible to download them from the inventory
Search results get automatically gzip-compressed when the browser
permits it (which is usually the case).
The interactive display shows the original image pixel value in the
coordinate box when the mouse cursor stops moving.
DR5
3-2011
DR6
1-2012
HLA Data Release 5 – March 2011
DR2
9-2008
EDR 8-2007
DR1
2-2008
•
•
–
–
•
•
•
•
•
Zooming in/out capability
Panning
Fully integrated table, so that selections in one screen
remain valid in the others
Selection across instruments possible
Option to search through proprietary and/or nonproprietary data
Improved download flow from shopping cart
–
–
•
•
DR3
5-2009
USER INTERFACE ENHANCEMENTS
A completely new footprint interface, featuring:
–
–
–
•
DR4
3-2010
Single or sequential download
Partial downloads can be restarted
Sources clickable in interactive display
When filtering RA or Dec columns, the values may
be entered either in decimal degrees or in
"sexagesimal" format (hh:mm:ss, dd:mm:ss)
Multiband source list (ACS and WFPC2)
NEW DATA PRODUCTS
Completed WFPC2 processing (over 300 new visits
with source lists)
New High Level Science Products
ECF-generated ACS and NICMOS extracted GRISM
data hosted at STScI
DR5
3-2011
DR6
1-2012
HLA Data Release 6 – January 2012
DR2
9-2008
EDR 8-2007
DR1
2-2008
•
•
•
•
•
•
•
•
•
DR4
3-2010
DR3
5-2009
NEW DATA PRODUCTS
WFC3 image data: includes ~62% of data from
observations that have a release date up to 1130-2011
New High Level Science Products
Additional ACS (updated to release date 2-162011), WPFC2 and NICMOS data
USER INTERFACE ENHANCEMENTS
Interactive Display enhancements.
The inventory and images interfaces now
include a "Select All" button that can be used
to select all the datasets in the current list.
The HLA Simple Image Access Protocol server is
now fully compliant with the Virtual
Observatory standards. The HLA SIAP server
will be added to the VO registry for easy access
to HLA data from scripts and other VO tools.
Searches using lists of positions now show the
number of lines in the list.
DR6
1-2012
DR5
3-2011
Current work – WFC3 Source lists
•
•
Based on existing HLA pipelines (ACS, NICMOS) with structural changes
Some algorithmic changes
–
–
–
•
New alignment module performs very well on crowded fields
Sky equalization included
More rigorous handling of header information
Source lists are undergoing quality review, will be released later
Current work – Spectral Data
•
•
•
•
•
Strong desire to better integrate spectral data in the HLA
General purpose spectral pipeline not yet available - focus on High
Level Science Products
First release will include collections of processed spectra (e.g.,
STARCAT)
• Interface modifications to handle spectral products
• Preview code replaces image previews and cutouts
• New Spectral Plotting tool with better functionality to view and
interact with spectra
Difficulties:
• Spectral data products inhomogeneous in properties,
combination process
• Naming conventions in development
• Large number of data points in some spectra can challenge
some browsers
• User preferences not yet established - may need to iterate on
specifics
Expect ~200 products at first release