Transcript Cultivating IT skills at Cornell’s Library Technical

Cornell's Innovative Batch Processing Techniques Put
Into User-Friendly Practice
Boaz Nadav-Manes
Director, Acquisitions and Automated Technical Services
Philosophy Selector
Cornell University Library
Library Technical Services
Background: Cornell Technical Services and IT
• "Technical Services" vs. "Automated Technical Services“ (04/05
onwards).
• History of innovative co-development.
• Reporting lines (not much "red tape").
• Staff reductions through attrition, early retirements and some
layoffs.
• Transition to electronic.
• 2CUL (combining Technical Services with Columbia’s – 2015)
TS Staff and Automation
•
Not everything in TS can be automated.
•
Ask: what is big enough and/or what will happen a lot?
•
TS Staff are enthusiastic about automatic tools if they are the ones to contribute in
developing them and they are the ones to operate them.
•
Institutional history and “traditional” experience count! As you move forward
emphasize more your core values of service and train (not necessarily hire) for higher
level positions.
•
You don’t need to be a code expert as long as you know how to logically describe
and write a scope for the solution.
•
Scalability and Flexibility are keys for good solutions.
•
Work collaboratively – invite others to contribute and to lead projects.
•
Most of the automatic solutions are not "ground breaking" from the start - they
relieve some marginal FTE counts that add up later on.
Batch Your Mind!
General Training for all staff on how to use and design
MS-Access reports to streamline TS processes:
•
•
•
•
Automatic claims (is there a note in PO?)
Open purchase orders
How much you spent with whom?
Who has done what and when?
Have a users’ group - teach each other = BPEG (Batch
Processing Exploratory Group)
Introduction to Web Service API's using PHP and HTML
In the ecosystems business...
Work on solutions that have value beyond TS:
http://poof.library.cornell.edu/content/about
http://www.oclc.org/worldcat-selection.en.html
Patron Driven Acquisitions
in electronic and in print
http://2cul.org/
http://www.slideshare.net/boaz22/ala-2011-pda-preconference
Maximize simple building blocks (metadata)
1. SSH-Secure Shell using Pre-written Scripts:
Commands perform tasks quickly, but offer
limited flexibility for record quality and script
customization.
2. LS-Tools Web Interface
Greater flexibility for accommodating new
vendor plans and modifications of records.
3. Combinations of above, adding Perl scripts,
OCLC Batch-searching, Z39.50 protocol;
and/or MARC-Edit Tool
More complex operation, but results in better
quality of loaded records.
Friendly Interfaces allow non-experts to participate:
Locally designed “LS-Tools” Web Interface
Consists of 3 main steps:
1.
2.
3.
•
General Import of record files to LS-Tools
Modification of files to suit our needs (adding appropriate 948 fields, deleting
unneeded fields, etc.)
Bulk Import of newly modified records, to be loaded into Voyager in SSH.
Benefits over basic SSH commands: not programmer dependent for new
vendor’s files; greater control with a “do-it-yourself” approach.
Functional Approach to Tasks
Assessment Incorporated: Variety of Users
Heaviest User (non specialist!)
Power User
Structure and Permissions
Case Study: ISBN extract from YBP approvals
If/then Logic Runs In The Background
(invisible and flexible)
Checks and Balances:
**********************************************************************
Cornell University Library Batch Processing
Yankee Approval Metadata run on Mon Oct 7 17:36:04 EDT 2013
Who and Where: apache on lstools01.library.cornell.edu
Home directory: /libweb/sites/lstools.library.cornell.edu/htdocs
Working directory: /lstools/batch_data/YankAppr.280.1736
**********************************************************************
Netid: btn3
Voyager Site: Test
Output Log File: /lstools/users/btn3/Jobs/vendor.yankee.1381181763.log
E-mail notification: [email protected] [email protected]
**********************************************************************
ACTION: Misc FTP
BULK: YANKEE
CATNAME: Yankee
EMAIL: ctsbulk-l
FTPGET: yankee
INFILE: 13300927.mrc
OUTF: /lstools/users/btn3/Marc/13300927.mrc
PATH:
POFLAG: no
SCRIPT: yankee.pl
TITLE: Yankee Approvals
VENDOR: yankee
VNAME: Yankee_Appr
**********************************************************************
STEP 0.1: 183 initial load records
STEP 1.2: 183 input marc records with ISBNs.
STEP 1.2: 183 unique ISBNs.
STEP 2.1: 183 records written
FINAL: 1 vendor specific file prepared for load into Voyager
**********************************************************************
SUMMARY FOR PROJECT MANAGEMENT
**********************************************************************
Packaged for bulkimport using YANKEE: 183 records
filename: /lstools/users/btn3/Marc/13300927.mrc
**********************************************************************
Job completed on 2069-12-31 at 19:00:00
RC=0
Usage Statistics
Use pre-existing tools:
Opens MARC files into temporary
“MRK” files that can be edited (then
compiled back into MARC)
Using MarcEdit for a more ‘surgical’ approach
http://people.oregonstate.edu/~reeset/marcedit/html/about.html
Mix and match tools and approaches:
Using ISBN Extract with Batch Searching in OCLC
After retrieving a MARC file from vednor’s FTP, ISBN-Extract command
creates a “.txt” file of ISBNs in a list, and sends it to a designated folder
ISBN File is batch-loaded into OCLC to be searched & exported to Local files.
Cultivate outside the box thinking:
Using PERL Scripts when MARC files are unavailable
Script is designed to navigate to Website, enter password, and
extract lists of LCCNs from a set of framed pages.
Extracting LCCN lists as “.txt” for all un-received LC Jakarta
shipments…
SHARE (FOLDERS)
LCCN Text files are sent directly to relevant vendor folder,
pre sorted by country (IO, MY, PH, BR, TH).
Allow experts to review exceptions in rotation.
Communal deposit of itterative procedures.
smile
Contact information: [email protected]