QAD Monitor - Midwest User Group

Download Report

Transcript QAD Monitor - Midwest User Group

FNST’s QAD Monitoring

One part of the monitoring/management initiative for FNST/FST

© Freudenberg-NOK General Partnership

1

Agenda

• Intro – About FNST • Background/History • QAD Implementation • Present State • Benefits Realized • Future Directions • Quick Review (screenshots) • Lessons Learned • Q&A

© Freudenberg-NOK General Partnership

2

Freudenberg-NOK Sealing Technologies Established 1989 Headquartered in Plymouth, Mich., USA

(Privately Held)

~23% Established 1849

Headquartered in Weinheim, Germany (Privately Held) © Freudenberg-NOK General Partnership

Established 1939

Headquartered in Tokyo, Japan (Publically Traded on the NSS)

3

Diverse and Established Partners

Headquarters Established 2008 Sales Ownership Products Plymouth, Michigan

1989 $1 Billion Privately Held Radial Shaft Seals O-Rings Gaskets Fluid Power Products Diaphragms Specialty Sealing Products Weinheim, Germany 1849 $6.5 Billion Privately Held Radial Shaft Seals O-Rings Gaskets Fluid Power Products Oil and Gas Products Diaphragms Specialty Sealing Products Vibration Control Devices Medical Products Tokyo, Japan 1939 $4.5 Billion Publicly Held Radial Shaft Seals O-Rings Gaskets Fluid Power Products Diaphragms Specialty Sealing Products Vibration Control Devices © Freudenberg-NOK General Partnership

4

Freudenberg Global locations

Production Sales Production and Sales © Freudenberg-NOK General Partnership

5

Production Locations in the Americas

Elgin, IL

Simrit Sales Office

Plymouth, MI

Technology Center

Morristown, IN

Custom Molded Rubber Products

Shelbyville, IN

Boots & Bellows

Necedah, WI

Gaskets

Santa Ana, CA

O-rings

Quer étaro, Mexico

O-rings

Cuautla, Mexico

Radial Shaft Seals and Custom Molded Rubber Products and Vibration Control Devices

New Hampshire Facilities: Ashland

Aerospace Radial Shaft Seals Industrial Radial Shaft Seals

Tillsonburg, ON

Silicone and Fluoro-silicone Products

Bristol

Mechanical Face Seals, Bonded Piston Seals

Northfield

Axle Seals

Manchester

Gaskets

Manchester

Microcellular Polyurethane

Cleveland, GA

Valve Stem Seals and Powertrain Seals

LaGrange, GA

O-rings

Milan, OH

Heavy Industry Lead Center, Imported Products

Findlay, OH

PTFE Products

Troy, OH

PTFE Products

Sao Paulo, Brazil

Radial Shaft Seals

© Freudenberg-NOK General Partnership

6

Freudenberg-NOK Sealing Technologies Today Responsible for sealing technology business in the Americas Total sales in 2010: US$ 798.7 million Workforce in 2010: 4,224 Locations in: U.S., Canada, Mexico, Brazil, Malaysia, China Sales exceeded market recovery in 13 of 15 principal market segments Sites: 35 production, sales and technical facilities

© Freudenberg-NOK General Partnership

7

• • • • • • •

Background/History

Agenda © Freudenberg Sealing Technologies GmbH & Co. KG

8

Background/History

HP OpenView implemented in 1994 Dedicated a great deal of effort to include Progress and QAD in our monitoring/management Versions not kept current (budget constraints) to support our Linux environment

© Freudenberg Sealing Technologies GmbH & Co. KG

9

• • • • • • •

QAD Implementation

Agenda © Freudenberg Sealing Technologies GmbH & Co. KG

10

QAD Implementation -

Contracted QAD Services.

“Canned” product, virtual Linux server (recover image and change IP), Icinga and Cacti for alerting and graphing.

“Base” configuration file created. Deployment simplified, copy base and change host info.

-

Scripts built for deploying: Database monitors based on our “/node/info/databases” file (list of databases per host with environment, used for many purposes including start-up).

Disk statistics.

Graphs too.

© Freudenberg Sealing Technologies GmbH & Co. KG

11

Our Environment

Centralized 2010SE Host (co-lo in NC) Supporting 30 sites from Canada to Mexico Distribution Site Linux eB2 (our facility in OH) Brazil Site Linux 2007

© Freudenberg Sealing Technologies GmbH & Co. KG

12

Our Monitoring Environment

Each host has a system “control” file containing all databases and environments This allows easy scripting of status checks, up/down, and query from other applications and scripts to invoke This file allowed us to automate our monitoring implementation

© Freudenberg Sealing Technologies GmbH & Co. KG

13

Database Control File

###################### /prod/prddbhe10/prddbhe10 DEFAULT hlpea PRODEA10 QAD_add-on /prod/prddbqxo17/prddbqxo17 DEFAULT qxodb PRODEA10 QAD_add-on ####################### /prod/prdcmbme10/prdcmbme10 DEFAULT mfgdb PRODEA10 Prod_QAD /prod/prdcmbae10/prdcmbae10 DEFAULT admea PRODEA10 /prod/prdcmbfe10/prdcmbfe10 DEFAULT fngpea PRODEA10 FNGP_QAD_add-on /prod/prdcmbte10/prdcmbte10 DEFAULT tpea PRODEA10 TailorPro /prod/prdcmbev17/prdcmbev17 DEFAULT qxevents PRODEA10 Qxtend_events ####################### ## Corteco Mexico ####################### /prod/prd73me10/prd73me10 DEFAULT mfg73 PRODEA10 Prod_QAD mexico /prod/prd73ae10/prd73ae10 DEFAULT admea PRODEA10 /prod/prd73fe10/prd73fe10 DEFAULT fngpea PRODEA10 FNGP_QAD_add-on /prod/prd73te10/prd73te10 DEFAULT tpea PRODEA10 TailorPro /prod/prd73ev17/prd73ev17 DEFAULT qxevents PRODEA10 Qxtend_events #######################################################

© Freudenberg Sealing Technologies GmbH & Co. KG

14

Site Monitoring

Ping a host in each supported site Good method of determining how a site sees performance

© Freudenberg Sealing Technologies GmbH & Co. KG

15

Scripts for Automation of Monitoring Services

./build_server_cfg.sh [remote host] [DLC] Builds a file with the Icinga components for Progress database monitors based on the control file. This generated file gets inserted into the host’s configuration file in Icinga.

Creates Icinga monitors for DB check, extents, log entries, last backup, buffer hits, read rates, and locks ./build_server_graph.sh [remote host] [DLC] Queries for host ID, then creates Cacti graphs for each database in the control file ./build_server_disk.sh [remote host] Ran a “df” on the remote host, builds Icinga space monitor for each local disk, insert into host’s configuration file.

© Freudenberg Sealing Technologies GmbH & Co. KG

16

Script to Build Configuration File

copy the template to [servername]-server.cfg

cp fnst-server.cfg.template [servername]-server.cfg

find and replace ral2lnxqad with the server name edit the host IP correct the webspeed entries correct DLC if needed build the databases section: ./build_server_cfg.sh [remote host] [DLC]

© Freudenberg Sealing Technologies GmbH & Co. KG

17

• • • • • • • • • Present State

Agenda © Freudenberg Sealing Technologies GmbH & Co. KG

18

Present State

All Linux hosts monitored (ping, OS) All sites monitored (ping to a host in all sites) All Linux QAD hosts monitored (Progress databases, NetUI performance)

© Freudenberg Sealing Technologies GmbH & Co. KG

19

• • • • • • • • • Benefits Realized

Agenda © Freudenberg Sealing Technologies GmbH & Co. KG

20

Benefits Realized

Network issues identified “stair step” graph noticed, turns out cable disconnect at site Qxtend performance issue Alert on CPU utilization, worked w/ EA Team Cache Hit Rate Alert, increased “Big B” DB Writes/Min Worked w/ ERP Team, identified code that was not “domainized”

© Freudenberg Sealing Technologies GmbH & Co. KG

21

• • • • • • • • • Future Directions

Agenda © Freudenberg Sealing Technologies GmbH & Co. KG

22

Future Direction

Regular internal review.

Regular review with ERP Team.

Review what else would benefit us.

Remote point(s) for monitoring. NetUI testing for one.

With the implementation of Vmware in remote sites we may just deploy VM Include HPUX (retire OpenView).

Visibility to others outside the Server Team.

Migrate additional legacy performance and monitoring scripts Review how to play with SCOM

© Freudenberg Sealing Technologies GmbH & Co. KG

23

Agenda

• • • • • • • • • Quick Review (screenshots)

© Freudenberg Sealing Technologies GmbH & Co. KG

24

Alerts Graphs

Quick Review © Freudenberg Sealing Technologies GmbH & Co. KG

25

Graphical, Emails

Alerts © Freudenberg Sealing Technologies GmbH & Co. KG

26

Status/Alerts by Location/Service © Freudenberg Sealing Technologies GmbH & Co. KG

27

Alerts by Service © Freudenberg Sealing Technologies GmbH & Co. KG

28

Alerts by Host © Freudenberg Sealing Technologies GmbH & Co. KG

29

Email Alerts © Freudenberg Sealing Technologies GmbH & Co. KG

30

Graphs

Hourly (min avg), Daily (5 min avg), Weekly (30 min avg), Monthly (2 hr avg), Yearly (1 day avg) Data points include: Ping Response Host Statistics QAD Users DB Reads Time to Read 10K Records Database Performance Statistics NetUI View

© Freudenberg Sealing Technologies GmbH & Co. KG

31

Site Overview

We included (at least) one host in each of the sites accessing our central QAD servers This gives us up/down and response times by site, assisting in troubleshooting issues

© Freudenberg Sealing Technologies GmbH & Co. KG

32

Site Overview © Freudenberg Sealing Technologies GmbH & Co. KG

33

Ping Response Time © Freudenberg Sealing Technologies GmbH & Co. KG

34

Statistics Specific to QAD Hosts

Processor statistics QAD users Database statistics NetUI performance

© Freudenberg Sealing Technologies GmbH & Co. KG

35

Host Performance Statistics © Freudenberg Sealing Technologies GmbH & Co. KG

36

QAD Users © Freudenberg Sealing Technologies GmbH & Co. KG

37

QAD DB Reads © Freudenberg Sealing Technologies GmbH & Co. KG

38

Sidecar DB Reads © Freudenberg Sealing Technologies GmbH & Co. KG

39

Admin DB Reads © Freudenberg Sealing Technologies GmbH & Co. KG

40

Time to Read 10,000 Records © Freudenberg Sealing Technologies GmbH & Co. KG

41

tr_hist Records Added (good indicator of activity) © Freudenberg Sealing Technologies GmbH & Co. KG

42

NetUI Response © Freudenberg Sealing Technologies GmbH & Co. KG

43

Point browser to http://ra2sqadmon

Live © Freudenberg Sealing Technologies GmbH & Co. KG

44

Questions, Suggestions, Comments,

Questions?

© Freudenberg Sealing Technologies GmbH & Co. KG

45