Transcript Apache Performance Tuning

Apache Performance
Tuning
Part 1: Scaling Up
Sander Temme <[email protected]>
1
Who am I?

Member, Apache Software Foundation



Infrastructure Team Member
Contractor




Lurker on dev@httpd
Walt Disney Imagineering
ADP
University of California
Currently unemployed…
2
http://httpd.apache.org/docs/1.3/misc/perf-tuning.html
says:
“Apache is a general webserver, which is designed to
be correct first, and fast second. Even so, its
performance is quite satisfactory. Most sites have less
than 10Mbits of outgoing bandwidth, which Apache can
fill using only a low end Pentium-based webserver.”
3
4
Goals of Performance Tuning


Serve as Many Requests as Possible
Graceful Degradation

(Don’t crash the box)
5
Assumptions





Limited Resources
Work With What You Have
You’re probably running Linux
No Compiling or Recompiling!
You can find httpd.conf
6
Monitoring Your Server
7
Purposes of Monitoring




Observation
Extrapolation
Signals/Alerts
Testing
8
Monitoring Apache
9
Monitoring Solaris: SE Toolkit



Lightweight
Programmable
Alive
http://www.sunfreeware.com/setoolkit.html
10
Other Monitoring Tools




vmstat
iostat
top
free
11
Web Server Logs

ErrorLog


LogLevel: debug, info, notice, warn, error, crit
Access Log: TransferLog or CustomLog

Common Log Format
12
Common Log Format
172.16.2.91
172.16.2.91
172.16.2.91
172.16.2.91
172.16.2.91
-
-
[16/Nov/2003:15:23:27
[16/Nov/2003:15:23:28
[16/Nov/2003:15:23:28
[16/Nov/2003:15:23:29
[16/Nov/2003:15:23:34
-0800]
-0800]
-0800]
-0800]
-0800]
"GET
"GET
"GET
"GET
"GET
/~sctemme/ HTTP/1.1" 200 1446
/~sctemme/css/doc.css HTTP/1.1" 200 167
/~sctemme/css/menu.css HTTP/1.1" 200 623
/favicon.ico HTTP/1.1" 404 283
/~sctemme/index.html HTTP/1.1" 200 1446
Client IP
172.16.2.91
RFC 1413 ident
-
username
-
timestamp
[16/Nov/2003:15:23:29 -0800]
Request
"GET /favicon.ico HTTP/1.1"
Status Code
404
Content Bytes
283
13
Log File Analysis

Analog



Fast
Cryptic output
Webalizer



Not as fast
Nicer output
Patched version available w/ GeoIP, DNS, 404
support (Google for it)
http://www.mrunix.net/webalizer/
http://www.analog.cx/
14
Webalizer Output
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
15
Webalizer Output Detail
QuickTime™ and a
TIFF (LZW) decompressor
are needed to see this picture.
16
Configuring for Performance


Configuring Apache
Tuning the Operating System
17
Apache Configuration





Process/Thread Management
DNS Lookups
Avoid .htaccess Files
Disable unused modules
Tune your App Tier
18
Selecting Your MPM




Apache 2.x only!
Processes and Threads
Differences between platforms
Thread-safety issues
19
Processes and Threads

Process:




Own copy of data structures
Shares: program code, shared memory
Context switches expensive
Thread:



Runs within process
Shares process environment
No context switch
20
Platforms and Threading





Context switches expensive on Solaris, AIX
Context switches cheaper on Linux
Solaris uses M:N threading
Linux uses 1 process per thread
LinuxThreads implementation is old


Replaced by Native Posix Thread Library (NPTL)
in 2.6
NTPL already in RH 9, RHAS 3
21
Thread-safety

Third-party modules and libraries



mod_perl: experimental threading in Perl 5.6;
more mature in Perl 5.8
PHP: uses many third-party libraries
FreeBSD: threading not reliable until 5.x

Apache 2.2 Worker MPM runs on FreeBSD
22
MaxClients





Configuration file directive
Maximum number of workers
Apache 1.3, 2.0 Prefork: processes
Apache 2.0 Worker: threads * processes
Limit according to resources (memory)
23
Sizing MaxClients


Take total RAM
Subtract OS allowance


Subtract external program allowance


look at free value without Apache, etc.
JVM, cgi programs, MySQL?
Divide by httpd process size

Read process size from top
24
Top
25
DNS Lookups


HostnameLookups
Access Control


Bad: Deny from example.com
Good: Deny from 172.160.234.5
26
.htaccess Files



Per-directory configuration files
Accessed for every request
Best performance: AllowOverride none
GET /dir1/dir2/restricted.html HTTP/1.0
27
Server-side Includes

We Love Them!



We Hate Them!



Easy to implement
Easy to manage
They break sendfile()
They break cacheability
Use XBitHack Directive


Only parse executable text/html type files
Special treatment of Last-modified date
28
SSI on www.apache.orgSSI Enabled
29
Tune your App Tier

Tomcat




Edit server.xml, tune minProcessors,
maxProcessors
Use APR -> Persistent connections
Tune JVM (Heap, Garbage Collection)
MySQL

Ships with various scenarios in support-files:



my-{small,medium,large,huge}.conf
PHP & prefork: every child makes a connection
Tune max_connections variable in my.cnf
30
System Tuning Tips



RAM and swap space
ulimit: files and processes
Turn off unused services and modules
31
RAM and Swap




Swap is disk-based Extension of RAM
Excessive swapping kills performance
Tune MaxClients
Never have more memory than swap

Upgrade RAM -> add more swap space
32
ulimit



Per-process resource limits
Built-in command of sh, bash
Important limits:




Set in invoking shell
Code in Apache 2 startup script


processes (-u)
open files (-n)
ulimit -S -n `ulimit -H -n`
Linux: /etc/security/limits.conf
33
Caching Content



Dynamic Content is Expensive
Static Content Relatively Cheap
Several Approaches:


Dynamic caching
Pre-rendering popular pages (index.rss…)
34
mod_cache Configuration
<IfModule mod_cache.c>
<IfModule mod_disk_cache.c>
CacheRoot /raid1/cacheroot
CacheEnable disk /
# A page modified 100 min. ago will expire in 10 min.
CacheLastModifiedFactor .1
# Always check again after 6 hours
CacheMaxExpire 21600
</IfModule>
</IfModule>
35
Make Popular Pages Static



RSS Feeds
Popular catalog queries
… (Check your access log)
36
Static Page Substitution
<Directory "/home/sctemme/inst/blog/httpd/htdocs">
Options +Indexes
Order allow,deny
Allow from all
RewriteEngine on
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ /cgi-bin/blosxom.cgi/$1 [L,QSA]
</Directory>
37
Tips



Observe Before You Act
Act on Monitoring Results
Don’t Overload Your System
38
Q&A
39
Conference Roadmap








TU1
WE3
30 Top PHP Scalability Mistakes
CNN.com Performance and Lessons
Learned
WE8 Hacking the Apache HTTP Server at
Yahoo!
WE24 Essentials on Performant JavaServer™
Faces Apps: Client Versus Server Side
TH1
Httpd Power Tools
TH4
mod_perl for Speed Freaks!
TH7
2.2 Rocks! Quit using 1.3 already!
TH9
mod_perl-2.0: Advanced Profiling &
Instrumenting Techniques
40
Current Version
http://people.apache.org/~sctemme/ApconUS2006/WE18/
41
Thank You
42