www.mague.com
Download
Report
Transcript www.mague.com
Where I Want to be.....
+
2
What I'll Settle For.....
3
Requirements
Rock solid stability
Automated node addition (discovery)
Scales horizontally
Service dependency models
Easy to write plugins
Promotes sane workflows
Unified front end view
Flexible configuration
4
Tool Stack
5
What????
but #monitoringsucks and #ihatenagios
How could you?
6
In defense of Nagios
Been around since 1996
Has Service dependencies
Easy to write plugins
Easy-ish to troubleshoot
ROCK SOLID
7
Valid attacks on Nagios
No automated discovery
It's complicated to setup
Text files – really?
Front end won't win any beauty contests
Development is slow
Stats collection is a PITA
8
Solutions
Use Icinga!
Use Puppet to auto configure
Stats – leave it to graphite. It's really good at that
Big boys and girls learn their tools
9
Icinga
Fork of Nagios
Configurations are compatible
More solid architecture ( core, API, Web, IDODB )
Nice front end, nice mobile front end
Can use NRPE
10
High Level View
11
Configure Icinga Servers
using Puppet Standard Types
12
Things to configure with Standard Types
icinga.cfg (file)
=> icinga main config file
Apache icinga.conf (file)
=> http access to each server
cgiauth.cfg (file)
=> cgi access
cgi.cfg (file)
=> options, users
templates.cfg (file) got lazy => use for basic classes
idomod.cfg (template)
=> template for hostname to DB
13
Configure Icinga using Nagios Types
14
Puppet Nagios Types
nagios_command
nagios_hostgroup
nagios_contact
nagios_service
nagios_contactgroup
nagios_servicedependency
nagios_host
nagios_serviceescalation
nagios_hostdependency
nagios_serviceextinfo
nagios_hostescalation
nagios_servicegroup
nagios_hostextinfo
nagios_timeperiod
15
Configuring Hosts
16
Overview
17
Detailed Overview
18
Store Configs
Store puppet info in a DB
Retrieve information from
DB
Share info across nodes
Use thin_storeconfigs
Set up on puppet master
19
Exporting Nagios_host Resources
Export = Save to DB
Use facter for dynamic data
PRO TIP: use ENC
PRO TIP: use targets
PRO TIP: hostgroups
PRO TIP: use tags
20
PRO TIP: Use your ENC
21
PRO TIP: use targets
Use cfg_dir in icinga.cfg
Create a unique file per host
or service
Addition and removal are
now super easy
Also default dirs are in a
horrible place /etc/nagios
22
PRO TIP: hostgroups
Add machines to a
hostgroup
Add services to a hostgroup
New machines inherit all of
the services associated with
a hostgroup
23
PRO TIP: use tags
Tags allow you to filter
resources so that you only
realize those resources that
you need
24
Configuring Services/Commands
25
Icinga Services
OR 'Stuff I want to monitor'
Associate with a hostgroup
Use a target
26
Icinga Commands
OR 'What actually gets run'
Use Macros to set paths in resource.cfg
27
Dependencies
28
PRO TIP: Dependencies
Unreliable services
Cut down on the number of alerts
Tell me what's really wrong
Route alerts accordingly
29
Nagios_servicedependencies
30
NRPE
31
NRPE
Runs on client
Secured via SSL
Has ACLs
Runs as nobody
Can run commands
Useful for other things...
32
Configuring NRPE
33
NRPE Checks
34
Plugins
35
exchange.nagios.org
36
Writing Plugins
Write in any language
Output 1 line to stdout
NRPE/Icinga/Nagios all use exit
codes to determine status
Run by hand to check
37
Workflows
38
Watching Monitoring
39
Scheduling Downtime
40
Filtering
41
Alerting
42
#monitoringisawesome
REMOVE unreliable checks
Just MONITOR – don't bolt on - especially stats
TIER your monitoring
Use timeperiods for sanity
Delegate responses
Use dependencies to pin down problems quickly
Work smart
43
44
Resources
Icinga
http://icinga.org
Puppet
http://docs.puppetlabs.com/references/latest/type.html#nagioscommand
NRPE
http://nagios.sourceforge.net/docs/3_0/addons.html
IRC
##infra-talk, #icinga, #puppet
Contact
[email protected], @maguec, #gaijin (freenode), http://blog.mague.com
Thanks
Yvonne Kong, Michael Catlin, Juan Ortega, Anthony Kong, Puppet Labs, Icinga Team
45