Transcript Digital Archiving and Processing with MIDAS Kitware Inc. Motivation Scientific datasets are becoming larger and larger (increasing resolution, new modalities, …) •Storing datasets is the.

Digital Archiving and Processing with
MIDAS
Kitware Inc.
Motivation
Scientific datasets are becoming larger and larger (increasing
resolution, new modalities, …)
•Storing datasets is the first step but querying and retrieving them
is even more important
•Related documents should be stored along with the corresponding
datasets
•Data without metadata are useless
•Distributed and remote computing is becoming a necessity
•Distributed visualization is an emerging technology
•Goal is to increase collaboration between research teams
•
What is MIDAS?
Web-based Multimedia Digital Archiving System
•Store, search and manage digital media
•Open Source (BSD)
•Started in 2005, based on DSpace
•Modular and highly customizable Framework
•Provides an external API to access media: REST, C++
•Provides server side processing via distributed computing
•Provides online visualization
•MIDAS features
•- Large datasets
•- Image Gallery
•- Download/Processing carts
•- Handle system
•- OAI-PMH
•- MIDAS-FS
•
MIDAS Data Server
/assetstore/01/10/19/01101909874657394
MIDAS Technology
Server for storing and managing digital content
- Apache (www)
•- PostgreSQL/MySQL (database)
•Web 2.0 technology
•- PHP
•- Ajax
•- Flash
•- Unit testing (integrated with CDash)
•Clients
•- Web API (REST API)
•- Standalone Client (MidasDesktop)
•- C++ API
•- WebDAV
MIDAS Modules
MIDAS Client
MIDAS C++ API
MIDAS Web API
MIDAS Visualization
MIDAS Compute Server
MIDAS e-journal
Apache
Publication DB MIDAS Data Server
MIDAS Core
PostGreSQL
File System
Server Side Processing
Server Side Processing
Distributed computing using BatchMake (www.batchmake.org)
•Support for Condor Grid
•Online selection of datasets and tasks
•Online monitoring of grid processing jobs
•Online reporting
•
Ongoing Work
Phase II NIH STTR: Murine Imaging with Martin Styner at UNC
•
Server-side processing on neural imagery from rodents
•
NIH NLM A2D2 SCORE
•
Segmentation validation framework within built on MIDAS
•
Phase I NIH STTR: COVALIC
•
Fully featured segmentation validation and method repository
built on MIDAS and IJ technologies
•
Kitware Deliverables
Aim 2.1 : Y1/Q3 – Y2/Q1
•
•
Open-source Software: Demonstrator
Aim 2.2 : Y2/Q1 – Y4/Q3
•
•
•
•
•
•
Open-source Software: Profile Editor
Open-source Software: MIDAS interfaced with the NBIA
database
Open-source Software: Batch processing workflow interfaced
with the Algorithm Validation Toolkit
Open-source Software: “Dashboard” in MIDAS
Open-source Software: Integrate the “R” statistical package
Aim 2.2 : Y4/Q3 – Y5/Q4
•
•
Active Dissemination
Project Demonstrator
Import a dataset from NBIA into MIDAS
•
Done using NBIAAdapter, a Java CLI Application
•Set up XML descriptions of seed points and ROIs (AIM)
•Run the lesion sizing toolkit algorithm on the dataset using
Batchmake
•Create a performance report (AVT)
•