Transcript DigiTool 3.0 System and Database Structure

DigiTool Database Structure
DigiTool Version 3.0
Session Agenda
Overview and Terms
Databases/Units
Directory Tree structure
DigiTool System Architecture
2
DigiTool Terms - General
Meditor client = Staff module (tool) for editing and management of
metadata, objects and users.
Web-based Resource Discovery = Web interface (tool) for end-user
collection searching, viewing and use.
Repository = Where DigiTool resources are stored.
Deposit Module = Web interface (tool) for end-user material deposits and
staff review.
Ingest Module = Web interface (tool) for staff loading of material.
Collection Management Module= Web interface (tool) for managing
hierarchical collections.
Staff Management Module= Web interface (tool) for staff management
of repository, delivery and deposit.
DigiTool System Architecture
3
Session Agenda
Overview and Terms
Databases/Units
Directory Tree structure
DigiTool System Architecture
4
Databases/Units
Three types of units exist:
1. Admin Unit is a database that controls the work
activity of cataloging, deposit, ingest, etc. for that
specific unit only. e.g. DTL01, DTL02
2. Silo Unit is a database that holds the data and
configuration for the Web-based Resource
Discovery e.g. GEN01
3. System Unit is a database that controls systemrelated activities such as users, repository, storage
and cache e.g. DAT01, REP00, EXT01, VIR01
DigiTool System Architecture
5
Session Agenda
Overview and Terms
Databases/Units
Directory Tree structure
DigiTool System Architecture
6
Overview
Two main types of directories exist:
1. Local directories of databases (units) contain
files/tables with definitions that are local to specific
databases (e.g., dtl01, gen01, etc.).
2. Global directories such as dtle or system include
global configuration files used across local
units/databases.
DigiTool System Architecture
7
Overview II
Each directory contains sub-directories, which
include configuration files and tables.
There are aliases (shortcuts) for some subdirectories (e.g., dt for local unit’s tab/ ),
which will be noted in italicized font
throughout this presentation.
DigiTool System Architecture
8
DigiTool Directory Structure
exlibris
aleph
Install - Default
sfx
dtl
j3_1
metalib
d3_1
verde
u3_1
DigiTool System Architecture
9
The d tree– DigiTool Default
dtl
d3_1
Version
dtle*
Directories
dtlm
dtl
pds
dtl01
dtl02
vir01
tmp
log
Subdirectories
DigiTool System Architecture
10
d3_1 directory & aliases
$dtl_dev
dm, dpm
dp, de
d3_1
dtlm
dtl
vir01
dtl01 & dtl02
$TMPDIR
Software - development &
maintenance programs
tmp
log
$LOGDIR
pds
xyz_media
service pack
Software executables, scripts
Temp system unit
Demo admin units
Temporary directory
Server logs
Patron Directory Services
Temp Media – web use
Service Pack installations
DigiTool System Architecture
11
The u tree– DigiTool User
dtl
u3_1
Version
Directories
dtle
rep00
xxx01
yyy01
zzz01
gen01
dat01
ext01
Subdirectories
DigiTool System Architecture
12
u3_1 directory
$dtle_root
u3_1
dtle
Node management – global
configuration - active
dat01
Local users unit
gen01
Local silo unit
ext01
Local external unit
xxx01
yyy01
Local admin units
DigiTool System Architecture
13
The j tree– DigiTool
dtl
j3_1
Version
digitool
home
Directories
system
profile
work
Subdirectories
DigiTool System Architecture
14
j3_1/digitool/home/ directory
j_home
j3_1
System configuration
system
profile
Local configuration
work
DigiTool System Architecture
Work and temp directory
15
Unit Structure
• Each unit has a separate root directory.
• Each unit contains information relevant to the
configuration of each unit defined in the
system.
• Each unit is implemented as its own ORACLE
user.
DigiTool System Architecture
16
The Tree Structure - Unit
dtl
u3_1
Version
Directories
dtle
rep00
xxx01
yyy01
zzz01…
gen01
dat01
ext01
Subdirectories
DigiTool System Architecture
17
The Local d/u Unit Root Directory
To get to the root directory of the collection you are
in, type “dr” . dr functions based on the dlib
environment which you are located.
dtl01/
- is the $data_root of the dtl01 unit tree/database.
DigiTool System Architecture
18
Unit Root – d/u tree
dr
xxx01
file_list
prof_library
tab/
dt
files/
df1
print/
dpr
md/
dmd
pers_coll_forms/
scratch/
ds
form_eng/
object_xyz/
DigiTool System Architecture
Silo only
19
Local Sub-directories I
Local Directory
tab/ includes configuration tables for defining the unit’s
behavior. (dt or cd $data_tab)
md/ includes tables for defining the unit‘s behavior
during Meditor metadata cataloging. (dmd or cd
$data_md )
files/ Dump files (znn.seqaa) and batch queue
management (df1or cd $data_files )
print/ includes printouts. It can be accessed using the
Meditor’s Task Manager. (dp or cd $data_print)
DigiTool System Architecture
20
Local Sub-directories II
Local Directory
form_eng/ includes email forms, templates
for printing, batch services and reports
(e.g., registration approval)
pers_coll_forms/ includes forms for the
wizard steps for adding personal objects
to the e-Shelf from the Resource
Discovery.
scratch/ includes temporary files and unitspecific log files. (ds, cd $data_ scratch)
DigiTool System Architecture
21
Local Sub-directories III (Silo ONLY)
Local Directory
object_view/ includes harvested XML files that may be
used for “Search in XML” advanced search from the
RD.
object_thumbnail/ includes any thumbnail harvested
into the silo and for use in the RD.
object_index/ includes all full text indexing files that
are harvested into the silo.
object_ctx/ includes all indexed data (full text and
otherwise) that is loaded into the DB. Indexes can
be built based on the data here or from scratch.
DigiTool System Architecture
22
Local Files
file_list defines all the Oracle tables required for that unit,
their maximum sizes and their indexes.
Local Directory
TAB
IND
TAB
IND
TAB
TAB
IND
TAB
IND
TAB
IND
IND
TAB
IND
IND
z52
z52_id
z00
z00_id
z01
z00h
z00h_id
z101
z101_id
z104
z104_id
z104_id1
z124
z124_id
z124_id2
10K
10K
2M
100K
2M
100K
100K
400K
100K
100K
100K
100k
400K
100K
100K
10K
10K
1M
100K
1M
100K
100K
100K
100K
100K
100K
100k
100K
100K
100K
DigiTool System Architecture
ts0
ts1
ts0
ts1
ts0
ts0
ts1
ts0
ts1
ts0
ts1
ts1
ts0
ts1
ts1
23
Unit Root – j tree
jdr
xxx01
jdc
conf
jdl
load
jdd
deposit
DigiTool System Architecture
24
Local Sub-directories I
Local Directory
conf/ Unit configuration including access rights, task
chains and template settings.
(jdc or jdata_conf)
deposit/ Unit deposits are stored in this directory until
staff approval or deletion.
(jdd or jdata_deposit)
load/ Unit-owned ingests originate from this file system
location.
(jdl or jdata_load)
DigiTool System Architecture
25
Navigating – Unit
Unit = databases (dtl01, dtl02, gen01, etc.)
root directory of local database
dr
cd $data_root/
(u3_1/xxx01/)
jdr
tab directory of local database
dt
cd $data_tab
(u3_1/xxx01/tab)
cd $jdata_root
(j3_1/digitool/home/profile/units/DTLXX/)
scratch directory of local database
ds
jdc
cd $jdata_root/conf
jdd
cd $jdata_root/deposit
jdl
cd $jdata_root/load
j tree
cd $data_scratch
(u3_1/xxx01/scratch/)
metadata directory of local database
dmd
cd $data_md
(u3_1/xxx01/md/)
d/u tree admin unit
d/u tree
DigiTool System Architecture
26
The Tree Structure - dtle
dtl
u3_1
Version
Directories
dtlm
dtle
rep00
dtl01
d3_1
dtl02
gen01
dat01
vir01
Subdirectories
DigiTool System Architecture
27
dtle Sub-directories
Directory
Subdirectories
wr
pcb
apc
dtle
tab/
scratch/
error_lng/
www_r_lng/
gate/
pc_b_lng/
coll_mng/
apache/
DigiTool System Architecture
28
dtle Sub-directories I
dtle
The dtle directory includes files that define
behavior on the node level, or across unit
databases.
Some files that sit under the root (base) directory
(cd $dtle_root): e.g.
dtl_start
dtl_start.private
dtl_shutdown_all
dtl_startup_all
pc_server_defaults
www_server.conf
DigiTool System Architecture
29
dtle Sub-directories II
dtle
tab/ includes tables for defining active bases,
logical-bases and general configuration such as
Resource Discovery behavior. (cd $dtle_tab).
Some important files are:
tab_base.eng
www_r_conf.xml
tab_dc_to_marc
DigiTool System Architecture
30
dtle Sub-directories III
dtle
scratch/ includes log files (detailed files
with information on the programs that
run in all units). (cd $dtle_scratch)
www_r_eng/ includes HTML and .css files
for the Web-based Resource Discovery.
(alias is wr)
DigiTool System Architecture
31
dtle Sub-directories IV
dtle
apache/ defines apache settings in
regards to DigiTool Resource Discovery.
(alias apc)
error_lng/ includes error and information
messages displayed online.
coll_mng/ includes configuration and
relevant files for collection management
DigiTool System Architecture
32
dtle Sub-directories V
dtle
gate/ z39.50 Universal Gateway
configuration.
pc_b_lng/ Meditor services related
configuration.
(alias pcb)
DigiTool System Architecture
33
j3_1/digitool/home/ profile directory
j_home
j3_1
System configuration
system/
profile/
Local configuration
work/
Work and temp directory
Note: j3_1/digitool/home.orig is used for service
pack mechanism comparisons and should not be
moved or changed.
DigiTool System Architecture
34
J-tree profile global directories
Directory
global.properties
Subdirectories
profile
General local configuration
conf/
Repository report output
reports/
Export of digital entities
export/
units/
collection/
Admin units
Collection staging location
overwrites/
work/
DigiTool System Architecture
ALL system/conf changes
made in this directory.
Work and temp directory
35
Profile Sub-directories I
profile
conf/ Global configuration across admin
units including copyrights.
reports/ Output of Repository-related
services.
export/ Repository exports output in this
directory.
DigiTool System Architecture
36
Profile Sub-directories II
profile
collection/ Collections and related objects marked
for export are stored here until p-media-35 service
run.
overwrites/ Any home/system changes are made
here and implemented by running set_globals.sh
Note: .tmpl files are used for specific changes to
repository_configuration.xml and conn_conf.xml.
work/ Index and temporary file staging area.
units/ All active admin unit directories are stored
here.
DigiTool System Architecture
37
Profile root Configuration File
profile
global.properties – general properties of
the DigiTool version
dbconnection.url=jdbc:oracle:thin:@server:1521:dtl99
dbconnection.username=d31_rep00
dbconnection.password=d31_rep00
server.name=server
http.port=1801
jndi.port=2801
rmi.port=3801
rmi.object.port=3901
webservice.port=4801
server.pool.port=6801
pds.port=8881
pds.server=server
index.directory=/exlibris/dtl/j3_1/digitool/home/profile/work/indexes
/d31_rep00
oracle.index.directory=/exlibris/dtl/j3_1/digitool/home/profile/work/
indexes/d31_rep00
index.files.per.directory=1000
DigiTool System Architecture
38
j3_1/digitool/home/system directory
j_home
j_sys
j3_1
system/
profile/
j_work
Active System configuration
Local configuration
work/
DigiTool System Architecture
Work and temp directory
39
J-tree system global directories
Directory
j_sys
system
j_bin
j_conf
Subdirectories
bin/
client/
conf/
sql/
thirdparty/
xsl/
Scripts and programs
Java client
General system config
SQL schema
TP applications and web log
XSL schema and mapping
Note: j3_1/digitool/home/system is the active system,
but should not be changed directly. Any changes to the
system are placed in the profile/overwrites directory and
initiated by running the script (system/bin) set_globals.sh.
DigiTool System Architecture
40
System Sub-directories I
system
bin/ Scripts and programs. (e.g. starting and
shutting down JBOSS)
(j_bin)
conf/ Global configuration e.g. repository,
ingest, e-Shelf, internationalization. (j_conf)
xsl/ XSL configuration and mappings.
DigiTool System Architecture
41
System Sub-directories II
sql/ SQL schema for J-tree
system
thirdparty/ Third-party tools including
open server and log files.
(jb_log – server.log)
client/ Java system clients.
DigiTool System Architecture
42
Navigating – Global directories -Summary
$dtle_root (dtle) = root directory of dtle (u3_1/dtle)
$dtle_tab = tab directory of dtle (u3_1/dtle/tab)
j_home
j_bin
j_conf
j_sys
jb_log
cd
cd
cd
cd
cd
$jdtlh
$jdtlh_bin
$jdtlh_conf
$jdtlh_system
$jdtl_jb_def_log
DigiTool System Architecture
43
Log Files and Monitoring Servers
Web Server Log
>>cd $LOGDIR
www_server_4881.log
UTIL-W
PC Server Log
>>cd $LOGDIR
pc_server_6881.log
UTIL-W
JBOSS Application Server Log
>> jb_log
server.log
Apache Server Log
>> apcl
access_log
comlog – soft links to the log directories
/exlibris/dtl/logs/log_3_00_1
DigiTool System Architecture
44
Important Server Commands
Re-start Web Server
>>start_w
Re-start PC Server
>>start_pc
Re-start JBOSS Application Server
>> j_bin
>> jboss_shutdown.sh
>> jboss_startup.sh
Re-start all DigiTool Servers
>> dtle
>>dtl_shutdown_all
>>dtl_startup_all
Re-start Apache
>> apcb
>>./apachectl restart (or ./apachectls_auto for SSL)
DigiTool System Architecture
45
Thank you!
www.exlibrisgroup.com
DigiTool System Architecture
46