Transcript Mass Storage 성능 분석 2002. 1. 17 강사 : 이 경근 대리 HPCS/SDO/MC

NDD Parameter tuning
GSC/Backline
최원규
© 2006 Hewlett-Packard Development Company, L.P.
The information contained herein is subject to change without notice
Page 1
NDD Parameter
The command ndd is a networking configuration tool used to
customize the behavior of the networking kernel. To make
the 11i system more internet friendly and easier to get
running "out of the box", some of the ndd tunables have
been enhanced.
Network parameter를 “tune”할 수 있는 tool
Page 2
NDD / IP
ip_forwarding
HP hosts가 gateway로 사용시 IP을 forward할 것인가 말것인가.
0 -> forward안함.
1 -> forward함.
2 -> Interface가 2이상인 경우에만 forward함.
Ver
Min
Max
Default
11.0
0
2
2
11.i
0
2
2
Page 3
NDD / IP
ip_ire_gw_probe
Turns the Dead Gateway Detection on and off.
HP-UX 11.0에서는 default gateway를 ICMP을 이용하여 주기적으로
check하는데 일부 router의 경우 ICMP에 응답을 하지 않는 경우가 있는데
대개의 경우 Firewall에서 default router로 ping이 않되게 setting 하는 경우가
대부분. 그때 이 시간을 넘어서면 router넘어로 network connection fail.
ndd -get /dev/ip ip_ire_status | grep -e IRE_GATEWAY -e flag
Ver
Min
Max
Default
11.0
0
1
1
11.i
0
1
1
Page 4
NDD / IP
ip_pmtu_strategy
0 -> any non-local networks have a maximum MTU of 576
1 -> All outbout packet에 DF(Don't Fragment)를 set한다.
의미는 HP machine의 outbound MTU size를 그대로 두라는 것이다.
destination과 HP machine사이의 MTU size값이 틀려 fragmenetion이
필요한 필요한 경우 router로 부터 ICMP Fragmentation Needed를
받으면 IP는 MTU size을 수정한다. 이를 지원하지 않는
router의 경우는 순차적으로 작은 MTU을 사용한다.
2 -> Disable
3 -> pmtu option "-p“
Ver
Min
Max
Default
11.0
0
3
1
11.i
0
3
1
Page 5
NDD / IP
ip_ire_gw_probe_interval
HP-UX 11.x에서는 default gateway를 ICMP을 이용하여 주기적으로 check하는데
일부 router의 경우 ICMP에 응답을 하지 않는 경우가 있는데 대개의 경우
Firewall에서 default router로 ping이 않되게 setting 하는 경우가 대부분.
그때 이 시간을 넘어서면 router넘어로 network connection fail.
IBM 2212 router에 VRRP(Virtaul Router Redundancy Protocol RFC 2338)을
enable한 경우 primary router에 문제가 발생하여 backup으로
넘어가는 경우 이 backup router에서 ICMP ECHO reply을 하지 않음으로 HP
machin에서 gateway를 인식하지 못함.
CISCO의 HSRP(Hot Standby Routing Protocol)에서 firewall로 ICMP Reply을
disable한 경우.
Ver
Min
Max
Default
11.0
15000
X
180000
11.i
15000
X
180000
Page 6
NDD / IP
ip_send_source_quench
IP packets이 처리할 수 있는 용량보다 빨리 node에 도착하는것인데 ICMP
packet을 받아서 처리 할때 만일 ICMP을 처리하는 Process가 system에
running중인 경우 ICMP packet을 해당 process의 socket에 넣는데 socket full이
발생시 ICMP Source Quench Message가 상대방에게 보내진다.
이를 0로 하면 Source Quench을 check하지 않는다.
Ver
Min
Max
Default
11.0
0
1
1
11.i
0
1
1
Page 7
NDD / IP
ip_max_bcast_ttl
Hop count가 1로 bcast packet이 router을 넘어가지 못함.
C class로 subnet하여 사용시.
199.3.20
199.3.201
199.3.208
20 subnet에서 braodcast가 208에 도달하게 하기위한 방법.
Broadcast 하는 program에서 조정하는 방법.
System gloval변수 조정.
ndd -set /dev/ip ip_max_bcast_ttl 3
보통 16정도 주면 됨. 증권사에서 자주 사용함.
Ver
Min
Max
Default
11.0
0
255
1
11.i
0
255
1
Page 8
NDD / IP
ip_check_subnet_addr
Controls the subnet portion of a host address
RFC 1122을 check,즉 subnet mask 255.255.255.192을 허용하지 않음.
0 -> RFC 1122 check을 bypass함.
Host IP 205.185.219.252 subnet mask 255.255.255.192.
The error he gets is: ifconfig: ioctl (SIOCSIFADDR).
RFC 1122에 따른 Guide을 HP에서 준수하기 때문에 발생한 문제.
Ver
Min
Max
Default
11.0
0
1
0
11.i
0
1
1
Page 9
NDD / IP
ip_strong_es_model
•
•
•
This tunable controls the support for the "Strong End-System Model" described in the RFC1122,
Section 3.3.4.2.
This means that the source address of a packet (that the host wants to send out) affects the selection of
the gateway for the outbound packet.
It is meaningful in the case that the host is multihomed (means several IP addresses for the interfaces).
Here are the key requirements from the RFC regarding the multihomed environments for the ES model:
A host MUST silently discard an incoming datagram whose destination address does not correspond to
the physical interface through which it is received. A host MUST restrict itself to sending (non-sourcerouted) IP datagrams only through the physical interface that corresponds to the IP source address of
the datagrams.
In restricted network environments it may be a wanted behavior to know exactly which of the addresses are valid and which
not. in most grown environments it is not very useful to enable this, since it will be not too easy to fulfill the restrictions
always.
Ver
Min
Max
Default
11.0
0
1
0
11.i
0
1
0
Page 10
NDD / TCP
tcp_conn_request_max
Max number of outstanding connection request
동시에 Connection을 맺을 수 있는 각 socket에 대한 listen queue limit.
socket program에서 listen()에서 define 할 수 있는 최대크기.
Application에서의 Create한 socket port에대한 listen queue의 제한은 실제로
Application program내부의 listen()에서 사용된 size에의해서 제한을 받는다.
/usr/include/sys/socket.h: #define
SOMAXCONN
20
netstat –s : connect requests dropped due to full queue
For a web server it might be a good idea to set this value to 1024 or higher.
Ver
Min
Max
Default
11.0
1
undefined
20
11.i
1
undefined
4096
Page 11
NDD / TCP
tcp_recv_hiwater_def : receive buffer window size
tcp_xmit_hiwater_def : send buffer window size
개별 TCP session에 대해 buffer size할당
Ver
Min
Max
Default
11.0
4096 Bytes
undefined
32768(32KB)
11.i
4096 Bytes
undefined
32768(32KB)
Page 12
NDD / TCP
tcp_fin_wait_2_timeout
Ver
Min
Max
Default
11.0
0
undefined
0
11.i
0
undefined
0
Page 13
NDD / TCP
tcp_syn_rcvd_max
Controls the SYN attack defense of TCP effective.
SYN packet을(SYN_RECV in netstat -a) 보관하는 table space을 지정한다.
여기을 거쳐서 각 socket별로 listen queue에 들어 간다.
Ver
Min
Max
Default
11.0
1
1000
500 connections
11.i
1
1000
500
Page 14
NDD / TCP
tcp_time_wait_interval
How long stream persists in TIME_WAIT
Amount of time TCP endpoints persist in TCPS_TIME_WAIT state.
netstat -an으로 확인시 모든 TCP session은 정상적으로 Session이 종료시
TIME_WAIT 상태을 거치게 된다. 이 값을 조정하는 Parameter임.
10.X에서는 default로 2MSL을 사용하며 MSL이 60 Second이기에 2분을 Waiting
하다 Session이 netstat -an시 보이지 않는다. 10.x OS에서는 조정이 불가능함.
/usr/conf/h/protosw.h:
#define PR_SLOWHZ 2
/usr/conf/netinet/tcp_timer.h: #define TCPTV_MSL ( 30*PR_SLOWHZ)
Ver
Min
Max
Default
11.0
1000 (1sec)
600000(10min)
60000 (1min)
11.i
1000 (1sec)
600000(10min)
60000 (1min)
Page 15
NDD / TCP
tcp_status
TCP
dst
rto mss [lport,fport] state
snxt
suna
swnd
cwnd
rnxt
rack
rwnd
00000001015d1468 015.043.232.202 361e667e 361e667e 00008000 000005b4 06dda013 06dda013 00008000
24000 01460
ca16,9e9
TCP
TCP instance number
dest
destination IP-address
snxt
Senders next sequence number
suna
Data the sender has not acknowledged yet
swnd
Senders window (relative to suna)
cwnd
Congestion window
rnxt
Sequence number we expect to receive next
rack
Sequence number we have acknowledged
rwnd
Current receive window
rto
Round trip timeout
mss
Max segment size
lport
Source port
fport
Destination port
state
TCP state this connection is in
Page 16
NDD / TCP
tcp_discon
ndd -get /dev/tcp tcp_status | grep -e state -e TCP_FIN_WAIT_2
tcp_discon
- Terminate a TCP connection
-> TCP session을 disconnection하기 위한 ndd option
$ ndd -get /dev/tcp tcp_status
TCP
dest
snxt suna swnd cwnd rnxt
rack rwnd rto mss [lport,fport] state
0183b8b4 015.043.233.086 533cb8ce 533cb8ce 00008000 00003000 533bc583
533bc583 00000000 02812 04096 [c00a,cea9] TCP_FIN_WAIT_2
$ ndd -set /dev/tcp tcp_discon 0x0183b8b4
Page 17
NDD / TCP
tcp_discon_by_addr
TCP session disconnection by address.
Local IP: 192.1.2.3 (0xc0010203)
Local Port: 1024
(0x0400)
Remote IP : 192.4.5.6 (0xc0040506)
Remote Port: 2049
(0x0801)
$ ndd -set /dev/tcp tcp_discon_by_addr "c00102030400c00405060801"
Page 18
NDD / TCP retransmission(접속할때)
tcp_ip_notify_cinterval [10,undefined] : R1 timer
tcp_ip_abort_cinterval [75, undefined] : R2 timer
Cinterval 같이 앞에 c가 붙는것은 보내는 것이다.
Page 19
NDD / TCP retransmission(접속중일때)
tcp_rexmit_interval_initial [3, 20] [0.5,20 :11.0] [1.5,20 :< PHNE_ 27730]
tcp_rexmit_interval_min [0.5, 20]
tcp_rexmit_interval_max [1m, 1h]
tcp_ip_notify_interval [10,undefined] : R1 timer
tcp_ip_abort_interval [10m, undefined] : R2 timer
tcp_time_wait_inteval [1m, 10m]
Page 20
NDD / TCP
tcp_keepalive_interval [2h]
Idle connection에도 TCP상태를 가지고 있는 timer
Established connection - tcp_ip_abort_interval [10m, -]
Connection establishment - tcp_ip_abort_cinterval [75s, -]
Connection terminating - tcp_keepalive_detached_interval [2m, 10d]
setsockopt() 에서 enable해야 적용됨
Page 21
NDD / Sockets
socket_buf_max
Specifies the maximum socket buffer size for AF_UNIX sockets.
Ver
Min
Max
Default
11.0
X
X
X
11.i
1024
2147483647
256000
Page 22
NDD / Sockets
socket_caching_tcp (웹서버 용도로 사용될시 효과 만점)
Used to enable socket caching on TCP AF_INET sockets. This value determines
how many cached data structures for TCP sockets the system keeps. This
could cause the system to speed up considerably if there are many short-lived
connections on the system.
A value between 1 and 512 will set a minimum of 512. Any number above
512 will set that value.
Ver
Min
Max
Default
11.0
X
X
X
11.i
0
2147483647
0
Page 23
Oracle DB Server Tuning
Parameter
ip_send_source_quench
tcp_conn_request_max
tcp_rexmit_interval_max
tcp_rexmit_interval_min
tcp_xmit_lowater_def
tcp_syn_rcvd_max
tcp_xmit_hiwater_def
Default value
1
20
60000
500
8192
500
32768
Suggested value
0
20000
10000
1500
24576
1024
1048576
Page 24
Apache 1.3.x Web Server Tuning
Parameter
Default value
tcp_conn_request_max
20
tcp_fin_wait_2_timeout
0 (infinite)
tcp_ip_abort_interval
600000
tcp_keepalive_interval
72000000
tcp_rexmit_interval_initial
500
Suggested value
1024
900000
60000
900000
Site-specific (see note)
Page 25
Netscape & Iplanet Web Server Tining
Parameter
Default value
tcp_time_wait_interval
60000
tcp_conn_request_max
20
tcp_ip_abort_interval
600000
tcp_keepalive_interval
72000000
tcp_rexmit_interval_initial
1500
tcp_rexmit_interval_max
60000
tcp_rexmit_interval_min
500
tcp_xmit_hiwater_def
32768
tcp_recv_hiwater_def
32768
Suggested value
60000
1024 --> 4096
60000
900000
1500
60000
500
32768
32768
Page 26
ndd parameter for security
Parameter
Default value Suggested value
Comment
ip_forward_directed_broadcasts 1 0 Don't forward directed broadcasts
ip_forward_src_routed 1 0 Don't forward packets with source route options
ip_forwarding 2 0 Disable IP forwarding
ip_ire_gw_probe 1 0 Disable dead gateway detection (currently no ndd help text;
echo-requests interact badly with firewalls)
ip_pmtu_strategy 1 1 Don't use echo-request PMTU strategy (can be used for amplification attacks
and we don't want to send echo-requests anyway)
ip_send_redirects 1 0 Don't send ICMP redirect messages (if we have no need to send redirects)
ip_send_source_quench 1 0 Don't send ICMP source quench messages (deprecated)
tcp_conn_request_max 20 500 Increase TCP listen queue maximum (performance)
tcp_syn_rcvd_max 500 500 HP SYN flood defense
ip_check_subnet_addr 1 0 Permit 0 in local network part (should be the default)
ip_respond_to_address_mask_broadcast 0 0 Don't respond to ICMP address mask request broadcasts
ip_respond_to_echo_broadcast 1 0 Don't respond to ICMP echo request broadcasts
ip_respond_to_timestamp_broadcast 0 0 Don't respond to ICMP timestamp request broadcasts
ip_respond_to_timestamp 0 0 Don't respond to ICMP timestamp requests
tcp_text_in_resets 1 0 Don't send text messages in TCP RST segments (should be the default)
Page 27
100BT configuration file
Configuration File
in /etc/rc.config.d Variables
btlan hpbtlanconf xxx_INTERFACE_NAME : Name of interface (lan0, lan1...)
xxx_STATION_ADDRESS : Station address of interface. This will be the MAC
address of your interface.
xxx_SPEED : set the card speed. Value are : 10HD, 10FD,100HD, 100FD,
auto_on. For EISA see Note1.
11.0
btlan0 -> hpeisabtconf
btlan1 -> hpbasetconf
btlan3 -> hpbase100conf
btlan4 -> hpgsc100conf
btlan5 -> hppci100conf
btlan6 -> hpsppci100conf
11.I
hpbtlanconf
Page 28
Nettl trace
packet 분석 tool
#nettl -start
#nettl -tn pduin pduout -e ns_ls_ip -s 1024 –tm 99999 -f /tmp/trace
-> 이러면 /tmp/trace.TRC00 와 trace1.TRC01 의 file이 생깁니다.
필요한 action 수행
#nettl -tf -e all
#vi /tmp/filter
filter ip_saddr 10.1.61.212
filter ip_daddr 10.1.61.212
-> 대상 장비의 IP
#netfmt -c /tmp/filter -Nlf /tmp/trace.TRC00 > /tmp/trace.txt0
Page 29
APA(Auto Port Aggregation)
1. Cisco’s Protocol for Automatic Trunk Discovery and
Automatic Link Aggregate Configuration
2. High Availability via Multiple Links with Failover Capability
3. High Availability Servers
4. Load Balancing
5. High Throughput
6. Advanced Interoperability
7. Basic Interoperability
8. Single IP Address Capability
9. Flexibility
10. Application Transparency
Page 30
APA(Example)
/etc/rc.config.d/hp_apaconf
HP_APA_INTERFACE_NAME[0]=lan100
HP_APA_LOAD_BALANCE_MODE[0]=LB_MAC
HP_APA_MANUAL_LA[0]=1,2
HP_APA_HOT_STANDBY[0]=on
/etc/rc.config.d/hp_apaportconf
HP_APAPORT_INTERFACE_NAME[0]=lan1
HP_APAPORT_GROUP_CAPABILITY[0]=5
HP_APAPORT_PRIORITY[0]=0
HP_APAPORT_CONFIG_MODE[0]=MANUAL
#
HP_APAPORT_INTERFACE_NAME[1]=lan2
HP_APAPORT_GROUP_CAPABILITY[1]=5
HP_APAPORT_PRIORITY[1]=0
HP_APAPORT_CONFIG_MODE[1]=MANUAL
/etc/rc.config.d/netconf
INTERFACE_NAME[1]=lan100
IP_ADDRESS[1]=150.100.1.3
SUBNET_MASK[1]="255.255.0.0"
BROADCAST_ADDRESS[1]=""
INTERFACE_STATE[1]=up
DHCP_ENABLE[1]=0
Page 31
Utilization 측정
FTP 는 performance 측정도구로 적절하지 않다.
System 에서 제공하는 command 는 lanadmin, netstat, glance 등이 있다
정확한 측정은 NMS나 analyze가 있어야 한다.
정확한 계산방법은 packet의 수가 아니라 bit 수이다.
(IfInOctet + IfOutOctet) * 8 /IfSpeed * 1/100
Page 32
Page 33