Transcript CZAR: A New Archiving System

My Week in Berlin
Ron Chestnut
May, 2002
EPICS Collaboration
ESD Controls
Outline

Channel Archiver
 EDM
 E-Log
 Serial Support
 CALCString record
 RDB Support
 A-beans
 JOIMINT
May, 2002
EPICS Collaboration
ESD Controls
JLAB CZAR System Overview
Database
(MySQL)
Embedded SQL
JDBC
DAQ Engine
History Server
CORBA
Manager GUI
XARR
CORBA
StripTool
Filesystem
DARTH
May, 2002
EPICS Collaboration
ESD Controls
CZAR Data Store

Engine


Converter



streams raw data to disk
runs periodically, or on-demand
“cooks” raw data (convert, compress, index)
Database



connection history
data directory
file status
May, 2002
EPICS Collaboration
ESD Controls
CZAR History Server Remote API

Supplies archive accessor objects to clients
 Client initializes range in archive accessor

data summary
• first, last point
• before, after point
• num points

range accessor object
• provides data pull interface
• serves sequences of data in large chunks
• timestamp, value, status, severity, control info
May, 2002
EPICS Collaboration
ESD Controls
CZAR Status

Engine up since January 2002




~ 21,000 signals
~ 2Gb / day raw; ~ 600Mb processed
< 20% CPU (HP B2000, 1GB RAM)
XARR

retrieve 100,000 points: ~ 3 sec.

Management GUI 1.0 99% done
 Port to Solaris, Oracle: June
May, 2002
EPICS Collaboration
ESD Controls
Channel Archiver Overview
Config
CA
CGIExport
ArchiveExport
Online Access
to Excel/GNUplot…
WinBrowser
CAManager
ArchiveEngine
LibIO (C++)
StripTool
Archive
xarr
LibIO
(C++)
Graphic Retrieval
ArchiveManager
casi
(swig)
extract, merge, delete
Python
Perl
some examples provided
Tcl
May, 2002
EPICS Collaboration
ESD Controls
JLAB CZAR System Overview
Database
(MySQL)
Embedded SQL
JDBC
DAQ Engine
History Server
CORBA
Manager GUI
XARR
CORBA
StripTool
Filesystem
DARTH
May, 2002
EPICS Collaboration
ESD Controls
(non) Decisions

Chris to continue ; Steve Hunte will test
July target
 Thomas and LeeAnn/Bob to continue
July target
 Comparison of engine capabilities needed
 Channel Archiver should not require Oracle or
CORBA, or CORBA should be supplied too
 Look again at performance at next meeting
May, 2002
EPICS Collaboration
ESD Controls
EDM Status

New Features







Select colors from palette or by function
Blinking colors for all display depths
Select color schemes by menu
EDMDATAFILES is now a colon separated list
Various new edit mode operations
New drag & drop behavior (Thomas Birke)
Online help
May, 2002
EPICS Collaboration
ESD Controls
EDM Status

New widgets










text entry (Kay Kasemir)
text update (Kay Kasemir)
strip chart (Kay Kasemir)
text update with reg expr (Pam Gurd)
static text with reg expr (Pam Gurd)
Byte (Carl Lionberger)
X-Y Graph
Motif slider
Radio Box
Up/Down button
May, 2002
EPICS Collaboration
ESD Controls
Widgets
May, 2002
EPICS Collaboration
ESD Controls
X-Y Plot Widget
May, 2002
EPICS Collaboration
ESD Controls
Future

Complete documentation


User
Widget developer

Runtime shutdown/restart
 Modify all widgets to utilize pv factory
 Implement remaining undo operations
 New file format (xml?)
 Enhance compatibility with medm
 Increment local PV class
 New scheme for printing displays
May, 2002
EPICS Collaboration
ESD Controls
History

First instituted at the lab in 1996.
 Variously used




Official Record for DOE
Knowledge repository
Bulletin board to pass info among shifts
Heavily Used



~35,000 entries/year in main ops logbook
400 MB/year in main ops logbook
Other logbooks for Software, FEL, etc.
May, 2002
EPICS Collaboration
ESD Controls
Shortcomings

Reliance on static html files – 100,000’s



Sheer number a headache to manage
The html mingling of content and formatting
makes data searching and extraction difficult
Every program (60+ of them) responsible for
number assignment, content formatting, not
clobbering other entries. Little consistency.
May, 2002
EPICS Collaboration
ESD Controls
Components of an Entry

Required


Keywords/Title, Timestamps (create, commit),
Logbook(s), Entry-type
Optional




Attachment(s) (screenshots, documents, etc.)
Downtime information
References to other logs (backward/forward)
Problem-Tracking History
May, 2002
EPICS Collaboration
ESD Controls
New Version

Summer 2002.
 Addresses shortcomings




RDMS replaces files.
Dynamic Scripting replaces static HTML
APIs shield apps from entry-making details
New Features


Integrated problem tracking
Threaded/linked entries
May, 2002
EPICS Collaboration
ESD Controls
Underpinnings

Relational Database (Oracle)
 Web Application Server (Apache + PHP)
 Libraries for programs that make entries




Tcl Package
Perl Module
C++ Library
HTTP API
May, 2002
EPICS Collaboration
ESD Controls
New logbook “home page”
May, 2002
EPICS Collaboration
ESD Controls
Alternative Look
May, 2002
EPICS Collaboration
ESD Controls
New trouble report entry
May, 2002
EPICS Collaboration
ESD Controls
Browser is not only interface
May, 2002
EPICS Collaboration
ESD Controls
Logbook Comparison
Old Logbook
New Logbook
Files (one per record)
RDBMS (Oracle)
Separate downtime &
problem tracking
Unauthenticated entries
Integrated downtime &
problem tracking
Authenticated entries
Web browser only
interface
Accessible from scripts,
browsers,spreadsheets.
May, 2002
EPICS Collaboration
ESD Controls
Future Direction

Enhance Problem-Tracking



Track problems with specific hardware in
addition to general hardware types.
Integrate with future Plant Engineering Work
Order system
Enhance Query/Search Tools

Oracle Intermedia to search attachments such
as PDF, Word documents.
May, 2002
EPICS Collaboration
ESD Controls
Serial Interfaces
Sharp intake
of breath!
May, 2002
EPICS Collaboration
ESD Controls
Vacuum Equipment Test Rack

VME 64x Crate
 Processors:


IP Carriers:



MVME 167 & PPC 604
Hytec VICB8002
Greenspring VIPC601
Serial Interface Card:

Greenspring IPOctal - 8 channel IP card
May, 2002
EPICS Collaboration
ESD Controls
EPICS Devices
 devAscii
- Allan Honey/Jeff Hill - KECK
 Stream Device - Dirk Zimoch - DELTA
 MPF - Mohan Ramanathan - APS
 ornlSerial - John Sinclair - Oak Ridge
 tyGSOctal - Peregrine McGehee - Hawaii
May, 2002
EPICS Collaboration
ESD Controls
Summary

All software built and worked

minor glitches 68k / PPC

Where existing support exists - use it
 For new development we like:


stream device for most cases
ornlSerial for complex
protocols/checksums
May, 2002
EPICS Collaboration
ESD Controls
Basic record/device support








Motors
Scalers
Multichannel analyzer
Multichannel scaler
Serial (RS-232)
GPIB
ADC’s
DAC’s








Encoders
Optical table
String calc, sequence
Complex expressions
Enhanced PID
Scan
Scan parameter
Generic VME
May, 2002
EPICS Collaboration
ESD Controls
Layered devices, techniques








Optical tables
Slits
Mirrors
Monochromators
Piezo controller
Digital Multimeter
Current preamplifier
Interpolation








N-step measurement
Serial I/O block
GPIB I/O block
Autocollimator
Temperature controller
X-ray microscope
Insertion device
Filter/shutter
May, 2002
EPICS Collaboration
ESD Controls
String Expressions



Extends CALC record
to combine string and
numeric expressions
Useful for run-time
programming of serial
devices
EPICS links are
programmable at run
time
May, 2002
EPICS Collaboration
ESD Controls
Serial O/I Block



Combination of string
calc to construct output
message, serial record to
send and/or receive, and
string calc to parse reply
Intended for run-time
programming
Similar GPIB support
May, 2002
EPICS Collaboration
ESD Controls
CCD support
CA server on CCD’s native platform
 Calls vendor-supplied software
 Stores images to local disk (HDF format)
 Some simple image analysis
 Hosts EPICS PV’s for control

May, 2002
EPICS Collaboration
ESD Controls
Goal: DB Based Configuration
Maintenance
Add/Delete
Devices in DB
Procedures and
Scripts
propagate
Changes to
Configuration
Files
May, 2002
EPICS Collaboration
Consistency
Guaranteed
ESD Controls
Initial BESSY Database System
Device Oriented Magnets
device
specs
Power Supplies
family
I/O
params
PS name
magnet
name
device
specs
measure
family
m:n
EPICS
specs
node
name
CAN
segment
node
specs
IOC
Controls
May, 2002
EPICS Collaboration
ESD Controls
Worked Sufficiently Well for
Creation of RTDB
Standard mechanism
power supplies, vacuum system, RF PLC-interface, triggers/timings, …
Graphical Editor
Text Editor
Many instances (substitutions)
of few different types (templates)
RTDB Template
DCT
RTDB
Script
DB
Substitutions
Text Editor
May, 2002
EPICS Collaboration
ESD Controls
New Structural Overview
May, 2002
EPICS Collaboration
ESD Controls
Abeans (or xal at SNS)

Application framework
Applications
Applications
Abeans
IOC
IOC
Hardware
Hardware
:
May, 2002
EPICS Collaboration
Hide comm
layer
Provide
application
services
Provide CS
models (device,
channel, etc.)
RAD, easier
maintenance
ESD Controls
Abeans Features

Hide details of Control System - just one of many:






Same core code is used everywhere


launch application/applet, remote install
Logging, alarm, and communication error reporting
Resource initialization and destruction, provide defaults
find, choose and connect to selected device
supports different protocol via plugs (e.g. CORBA, EPICS,…)
The code is tested very well
Visual composition and normal programming together


choose device type and command through menu!
almost all errors detected by compiler not during operation
May, 2002
EPICS Collaboration
ESD Controls
Abeans and EPICS ?

MEDM is faster for “expert” panels, TCL is simpler for prototyping, but:

Abeans application development is scalable!

profit from standard Java features
• GUI layout, file I/O, XML, WebStart, resources, logging,…

Abeans fight Java deficiencies
• optimize graphic performance and reduce memory usage

Use Abeans for complex applications…



client-side algorithms (machine physics)
display of multiple data (tables, tree, bird-view)
…and when long-term maintenance matters
May, 2002
EPICS Collaboration
ESD Controls
JoiMint
Where to use it?



Create graphics ‘on the fly’ -> JoiMint does not
distinguish between runtime and edit mode.
All changes (adding/ removing objects; save/ restore
displays) is done in the online mode.
Remote access to control system’s data
Diagnostic data from ‚smart devices‘




PLC‘s or power supplies
Get graphic config file from default http port
Quick diagnostic using a PDA version (planned)
An example of a ‘light weight’ application for Global
Accelerator Networks
May, 2002
ESD Controls
EPICS Collaboration
Basic Structure
Any Java Object
Graphic Objects
?
Timer
The Core Registry
Scan
Engine
Data Source Connector
(monitor)
http client
Configuration &
Commands
Data
http server
Databases / Control Systems Applications
May, 2002
EPICS Collaboration
Cfg
files
ESD Controls
local
thread
XML
Is used for:

Configuration Data:




Graphic Object Tab
(Http) Data source definition
Synoptic Displays (i.e. translated dm2k screens)
Data/ Commands



Value Updates
Archived Data
Commands to/ from Applications
May, 2002
EPICS Collaboration
ESD Controls
Diagnostic Tools:
EPICS property table





Activate by MB3 on graphic
device
Select: Device -> Device
Property
Device Properties ‚known‘
by http (DIXI) server
No special knowledge
programmed into JoiMint
DIXI server creates JoiMint
graphic (as XML file)
Display of graphic and
dynamic updates in JoiMint
May, 2002
EPICS Collaboration
ESD Controls
Registration
Register a Graphic text Object
Class: JoiTextField
Object: JoiTextField-1
Property: Value
register object Property.deviceName EPICS|WLST1D32_temp
Graphic Text Object
actual value
register devicename /object
Name
Object
JoiTextField-1 instance of JoiTextField
Registry
EPICS|WLST1D32_temp
Send
Command
„JoiTextField-1“ „setWidth“ „200“
Name
Object
Method
JoiTextField-1 instance of JoiTextField setValue
...
Send
Value
„ EPICS|WLST1D32_temp“
May, 2002
EPICS Collaboration
„actual value“
ESD Controls
Future Plans (I)
Command Record and Playback

Record all operator commands

Allow command editing

Store commands as XML file
 Load command-set from XML file
 Replay commands
 Implementation of sequencing features:


Loops
If – then – else structures
(based on live data from the underlying control
system)
May, 2002
EPICS Collaboration
ESD Controls