Transcript License Servers Overview and Maintenance

Teradyne License Service and Monitoring
Bob Van der Kloot
Teradyne, Inc
978-370-2014
[email protected]
License Service Goals
•
•
High Availability
Minimize effort for maintenance/support
– Standardization of operation
– Common code across platforms
•
•
Compliance with license agreements
Provide information necessary to manage license portfolio
– Easy access to metrics
– Standard access
License Server Technologies
•
Acresso (Macrovision) FlexLM
–
–
–
–
•
Reprise License Manager (RLM)
–
–
–
–
•
A new product by the creators of FlexLM, and very similar to FlexLM
We support on Solaris servers
Currently only used by RTDA
We support only daily peak logging, though full logging should be trivial
LUM
–
–
–
–
•
The vast majority of our products are licensed with FlexLM.
We support on Solaris and Windows servers
We support complete logging (1 sec resolution)
No LMAdmin!!!
Now owned by IBM
We support on Solaris servers
Currently only used by SmarTeam
We support only daily peak logging, more may be available via FNM
Vendor Proprietary Licenses
–
–
–
–
MicroFocus/Borland SilkTest License
Rational Atria License (deprecated, but still in use)
Compuware DLM license (not currently in production)
We support only daily peak logging
Support Technologies
•
Veritas Cluster Server
– Windows and Solaris
•
Veritas Volume Replicator
– Windows and Solaris
•
Acresso Flex Net Manager
– Linux Master
– Windows and Solaris Clients
– Oracle Database (Linux Server)
Types of License Servers
•
Solaris vs Windows Architecture
–
•
Driven by availability of license daemons
Global vs North America Service
–
•
•
Driven by Contractual agreements
All are Veritas VCS Clusters
Agoura Hills, California
–
“means”
•
•
–
–
–
•
Solaris 8 cluster
Global license service
“adriatic”
•
•
Solaris 8 cluster
North America license service
Solaris 8 cluster
Disaster recovery / development
Clone of “ways”
“ways”
•
•
Solaris 8 cluster
Global license service
“devways”
•
•
•
North Reading, MA
–
“cadet”
•
•
–
•
Solaris 8 cluster
North America license service
“wpcluster”
•
•
Windows Server 2009 cluster
Global license service
Achieve High Availability by:
–
–
High availability cluster
Distribute pool between locations when possible
•
50/50 distribution for vendors that support multiple lic servers
License Server Principles
•
Management of licenses is as regular as possible across:
–
License Server Platform
•
•
•
–
License Technology
•
•
•
–
FlexLM
LUM
etc…
Cluster or non-cluster
•
•
Solaris
Windows
Linux (when we choose to implement it)
All production licenses are on clusters
Consistent Abstraction layer provided for automation… same for all licenses
–
Cluster control
•
•
•
•
–
Start
Stop
Health Check
Clean
Monitoring
•
List of licenses in consistent format
–
–
–
•
Quantity
Expiration
Characteristics (UHD, etc)
Current Status
–
–
Current checkouts
Duration of each checkout
License Cluster Basics – Veritas Cluster Server
•
Keep it Simple
– Two cluster members
• Active member:
– Hostid is faked
– Licenses are served
• Passive member:
– Native hostid of the server
– Licenses are disabled
– Veritas Volume Replicator for volume containing licenses
• /data/vol/licenses/<vendor>
/keys
/bin
/logs
• Exact same path on windows server (via links in cygwin)
– Standardized control of licenses:
• Cluster needs to do:
– Start
– Stop
– Health Check
• User needs to do:
– Request cluster to start license
– Request cluster to stop license
License Daemon Control
lmgrdFOO
Administrator
(human)
Customization of
control of FlexLM
daemons
Custom each Vendor
lmgrd_generic
Standardized
control of FlexLM
daemons
cluster_control
Encapsulate Control
And Status Check
Of Cluster Environment
lmutil
hares
HA software
event detection
or
Administrator
VCS cluster
software control
interface
lmgrdFOO
Customization of
control of FlexLM
daemons
Custom each Vendor
lmgrd_generic
Standardized
control of FlexLM
daemons
Acresso
FlexLM daemon
control
lmgrd
Acresso
FlexLM daemon
control
The VCS cluster software uses
the same control programs that
the human administrator uses
(but with different arguments)
Vendor Daemon
Supplies
Licenses
Custom each Vendor
Typical customized calling module
#!/bin/sh
# This is lmgrdCADENCE
# control script for Cadence daemons
# this script must be run by "cds“
VENDOR='cadence'
/etc/init.d/lmgrd_generic -o $1 -v $VENDOR
Command interface of lmgrd_generic
-------------------------------lmgrd_generic version: 1.60
-------------------------------Usage: lmgrd_generic
-o operation
-v vendor
# vendor name (directory in /data_vol/licenses )
[-k key_spec ]
# license file name (.lic implies all .lic files in directory)
# (default to license.dat)
[-l lmgrd_name ] # lmgrd name override (default to lmgrd )
[-d daemon]
# vendor daemon executable name (can replace one only)
[-e effective ]
# effective daemon name (when daemon runs)
[-h hostid]
# hostid override
[-s sequence]
# daemon start sequence ( 0 to 99 allowed )
[-p prefix]
# prefix of vendor for control script name
[-n]
# dongle ( license file will be license_<host>.dat )
[-g]
# enable logging
[-rev]
# print revision and exit
[-rlm]
# reprise license server (not macrovision)
[-w]
# windows
operation is one of:
check
- examine files for correct settings
start
- start license daemon
stop
- stop license daemon
restart - restart daemons (use when license file is changed)
clean
- clean up from crashed daemon
info
- report PID numbers of running processes
for use only by VCS
vcs_start
vcs_stop
vcs_clean
vcs_monitor -
cluster
USED BY
USED BY
USED BY
USED BY
control:
CLUSTER CONTROL
CLUSTER CONTROL
CLUSTER CONTROL
CLUSTER CONTROL
Written in perl, same executable runs
on solaris, windows (cygwin) and linux
License Monitoring Goals
•
•
•
Standard Interface across all products
Maintainable
Standard WEB interface useful for beginning and advanced users.
Two license monitoring methods used
•
Sample use every 15 minutes
– Program “license_use” run remotely on each license server
• Returns record for each license in use, across all used license technologies
• vendor|product|version|qty|user|host|display|duration
– Data stored in Oracle, plots produced on demand
•
FlexNetManager
– Records to-the-second license use, but only for FlexLM
• Possibly also for LUM, have not tried
– Output in “raw” format done once per day, read into Oracle
– Plots produced on demand.
Demo of license monitor
•
•
I can’t include slides without violation of vendor license agreement
(because it would have real usage data)
I’ll show a live demo
LSF Batch Environment
•
•
•
6 sites
200+ machines
Monitoring by homemade programs
– Read LSF log into Oracle
What’s Next?
•
Major effort to continue automation and standardization
– Reduce Tool and License Administration from 2.5 FTE to ~.6 FTE
•
Significant improvement in prediction tools
– Give better access to historic data, so project teams can forecast their
future demand.