Transcript Unix Linux Administration II
Unix Linux Administration II
Class 8: Scripting loops. Introduction to sendmail. Reading and printing data.
Agenda
discuss Homework.
Unit 1: Scripting loops.
Unit 2: Introduction to sendmail.
Unit 3: Reading and printing data.
Homework review
DNS configs slave and master updates Configuring views.
Scripting – file management script.
Intermediate certificate, new chained www certificate.
Review: conditionals
Exit status, 0 = success, !0 = fail.
if test "$user" == “
[ -e /etc/nsswitch.conf ] logical operators -a -o || && You can use parentheses to alter the order of evaluations.
if cmd; then do; else do; fi if [ "$HOME" ]; then echo "Found home!"; else echo "shucks we are homeless!"; fi
Review: PKI
Private keys, Public certificates and CSR public CA Chain of Trust Chain certificates PKI setup private key, csr signed cert.
sign other requests (CSR).
Class 8, Unit 1
What we are going to cover: Scripting and loops What you should leave this session with: Basics to creating loops within your scripts.
How to enable debug in your scripts.
Loops.
Loops are blocks of code that run until complete (they can be infinite loops) The first example is the for loop. for f in value1 value2 value3 do cmd done
For loops - body.
for letter in a b c do echo “found: $letter” done.
The “Body” is the content between “do” and “done”.
When the script is executed the value for “letter” is assigned to the first value provided after “in” and then the body of the loop is executed. When complete the second value is assigned to the variable $letter and the process is repeated.
? What happens if you enclose a b c in quotes?
for loops cont.
You can leverage the shells ability for filename substitution in loops. The shell provides for filename substitution in the list provided to the body of the loop.
for f in [1-3].txt
do echo $f done.
Just as in the other examples, echo is executed 3 times in this example
for loops cont.
you can also read in file values and feed those to the for loop.
cat filelist.txt
1.txt
2.txt
3.Txt
for files in $(cat filelist.txt) ; do echo $files; done or for files in $(cat filelist.txt) ; do cat $files; done *example of command substitutions.
Using $* in loops
$* = all arguments echo “Number of arguments passed in $#“ for variables in $* do echo "$variables" done
Replacing $* with $@
You know that $* returns all the values provided at the command line. However if you use $@ this is actually a comma separated list of values for f in “$@” do echo $f done *Best practice to place double quotes around $@
while loops
Another looping function is "while". while cmd do cmd done “cmd” is executed and its exit status is tested. if the exit status is zero the commands between do and done are competed otherwise the script exits with a non zero status code
while script
Similar to saying “while true do” sample “while” script counting to 10 num=1 while [ "$num" -le 10 ] do echo $num done num=$(( num+1 ))
until
until - the inverse of while, meaning it will run so long as the return code is not 0, or not successful.
Similar to the while blocks, commands between the do and done functions may never be executed if the initial command returns a successful response (zero).
Useful when checking for a status change
until cont.
# if NOT successful enter the body until ps -ef | grep i "named“ | grep –v grep > /dev/null do echo "bind is not running" sleep 5 done echo "bind is running“
Break out!
Sometimes in a logic loop you want to break out based on user input such as the user asking to quit. Enter “break” while true do read cmd if [ "$cmd" = "quit" ] then break else echo "$cmd" fi done
Continue on…
The opposite of break is to continue. Sometimes you want the loop to simply leave the current loop and continue working through the script. This is where you might use continue for file do if [ ! –e “$file” ] then echo “file not found” continue fi process rest of file/data done
Sending the process to background
You can background a process using the & after the done statement. Just as we have done at the command line.
for file in data[1-4] do run $file done &
redirection
I/O redirection on a loop can be obtained using the < or > based on your need.
Write to file: for i in 1 2 3 4 do echo $i done > data.out
Sleep and background
sleep n - where n is a numeric value. Sleep will pause the system for the time specified on the command line.
You can run programs in the background using ampersand "&" script & output from this command will tell you the process associated with your process.
Use fg to foreground a background process.
options
You can define options in your scripts using syntax similar to this: if [ "$1" = "-a" ] then option=TRUE shift else option=FALSE fi echo "value for option is: $option"
getopts
The previous example is fine for simple options but if you want more flexibility it can become tedious to script. However
getopts
is available for this purpose.
getopts works within a loop and examines each argument to determine if it is an option based on the existence or absence – before the value.
getopts
The syntax of the getopts command is: getopts optstring option opstring – is the list of options expected from the command line. option - value used to iterate over the command line options provided.
getopts cont.
You can stack your options or pass them individually. Meaning –abc or –a –b -c If your option needs an argument add “:” getopts a:bc name Now a valid command line looks like: script.sh –a braeburn –b –c script.sh –a braeburn script.sh –b –c
getopts cont.
OPTARG used when an option requires an argument, e.g. –a braeburn OPTIND is a special variable used by getops which is set to 1 by default and is updated each time getopts complete a loop.
If you reset $OPTIND to 1 at the end of the loop it is possible to use getops again in the same script.
Impact of “:”
When an option character not contained in optstring is found, or an option found does not have the required option-argument: If optstring does NOT begin with a :
(colon
) 1.
Option will be set to a ?
2.
3.
OPTARG. will be unset A diagnostic message WILL be written to standard error.
Impact of “:”
Alternatively if optstring DOES begin with a : (
colon
) 1.
2.
3.
option will be set to a ? character for an unknown option or to a : (colon) character for a missing required option.
OPTARG. will be set to the option character found. no output will be written to standard error.
getopts sample
while getopts ":ab:c" option; do case $option in a) echo received -a ;; b) echo received -b with $OPTARG ;; c) echo received -c ;; :) echo "option -$OPTARG needs and an ARG" ;; *) echo "invalid option -$OPTARG" ;; esac done
Review: loops and breaks
For loops:
for f in a b c; do echo "found: $f"; done for f in $(cat filelist.txt); do echo $f; done for f in $(cat filelist.txt); do cat $f; done $* vs $@, $@ provides a comma separated list
Until and While:
while loops, if the exit status is zero the loop is entered.
until, if the exit status is NOT zero the loop is entered.
Break and continue are used to manipulate the loop behavior.
Review: Options and GETOPTS
Passing options to your script manually.
if [ "$1" = "-a" ] then option=TRUE shift GETOPTS is a built-in shell function. GETOPTS loops through arguments looking for a “-” before any arguments and determines if it is a valid option.
If arguments are required with the options then you simple add a “:” after the option in your script the GETOPTS will require one.
In class lab 8a
Lab notes for this session can be found here: http://www.ulcert.uw.edu
-> Class Content -> InClass labs ->
Class 8, Unit 2
What we are going to cover: Sendmail What you should leave this session with: DNS mail configuration Basic Sendmail message flow and configuration.
DNS and mail
In order for mail to routed to your server there must be a valid MX or mail server record in the DNS domain.
MX records are another type of Resource Record (RR) just as Name Servers are of type NS. Once we add MX records we should have at least four RR types defined in our domain zone files.
Just as CNAMES and NS RR always need to eventually point to A records, so do MX records.
DNS and mail cont.
Mail servers have priority ratings which are different from other DNS records. The values are somewhat arbitrary but tend to run from 10 to 90 The lower value the higher the priority. If you have two mail servers one set to 10 and the other to 20 mail will be routed to the lower value unless it is unavailable.
If both had the same value it would be a round robin configuration.
Sample DNS MX configuration
books.ulcert.uw.edu MX 10 mail.books.ulcert.uw.edu
mail.books.ulcert.uw.edu CNAME ns1.books.ulcert.uw.edu
---------------------------------------------------------------------- Or ---------------------------------------------------------------------- mail MX 10 CNAME mail ns1
How mail servers work.
A client generates a message using one of many mail clients. This client will either include a built-in SMTP client or it will hand it off to /usr/sbin/sendmail interface. This client then opens a session on port 25 with the SMTP server and begins to send SMTP commands: HELO, MAIL FROM, RCPT TO, DATA The message is completed with dot . on a single line. And the message is delivered.
Sendmail history
Written by Eric Allman who was working and studying at UC Berkley. The first version was called delivermail and shipped with BSD 4.0 and 4.1. Sendmail came about as a result of move from NCP (Network Control Protocol) to TCP. Also namespaces changed from a flat design to a hierarchical namespace (think DNS). Sendmail first shipped with BSD 4.1c which happened to be the first tcp based version of BSD.
Sendmail success
As Allman has been quoted saying “sendmail is complex because the world is complex. It is dynamic because the world is dynamic”.
Sendmail strives to accommodate all types of messages. This inclusive goal means rather than denying or rejecting messages that lack the correct header or syntax sendmail tries to compensate for them. The low cost entry along with a high delivery percentage many consider the primary reason sendmail is so popular today.
Sendmail version info
Postfix is the default MTA but sendmail is simple to install sudo yum install sendmail sendmail-cf YUM will install sendmail 8.14.x
We can switch between sendmail and postfix using /usr/sbin/alternatives and or enabling services using /sbin/chkconfig Current stable sendmail version available from sendmail.org is *8.14.8
* now purchased by Proofpoint
Email and Sendmail
There are three primary roles to consider when reviewing mail: MUA – message user agent, examples?
MTA – message transfer agent, delivers mail and transports mail between machines, examples?
MSA – Mail submission agent, capable of altering mail messages such as confirming hostnames are fully qualified, examples?
What are Sendmail, postfix and Exchange?
Basic parts of Sendmail
The basic parts to Sendmail The configuration file /etc/mail/sendmail.cf
A queue directory /var/spool/mqueue Aliases Sendmail can and will redirect mail destined for one account to another based on defined aliases.
Addresses and Rules
Sendmail is based primarily on rules.
rules are used to rewrite (modify) mail addresses, to detect errors in addressing and to select mail delivery agents.
rules are used to detect and reject errors, such as mail with no username rules examine the address of each envelope recipient and select the appropriate delivery agent.
Rule Sets
a sequence of rules are grouped together into rule sets, each set is similar to a subroutine a rule set is declared with the S command rule sets are numbered or named rule sets such as 0, 3, 4 and 5 are internally defined by Sendmail 0 resolve mail delivery agent 3 preprocess all addresses 4 post process all address 5 rewrite un-aliased local user
The three parts to a message
All messages have three primary components Header Body Envelope
The Header
Most header lines start with a word followed by a colon.
Received: Date: From: To: Each word indicates the expected value.
Not all headers are required.
The Body
The body of a message consists of everything following the first blank line To: user@domain Subject: Test message, blank line next!
The body start here. Message content here.
Is the subject line required?
The Envelope
Because of the diverse recipients, the sendmail program uses the concept of an envelope.
Content that describes the sender or recipient but is not part of the header is considered envelope information.
Envelope data is used to tell remote machines that mail is ready from a given user.
Before sendmail sends the data to a remote MTA it will send just the envelope-sender address and recipient list to the remote MTA. If
ANY
of the recipients are accepted the message is sent over otherwise it is not.
Aliases file functions
Aliasing is the process of converting one address into another address.
Convert root to mailer-daemon Convert name to list as in mailing list Sample conversions Bob Barker geeks Nobody app bbarker allman, schmidt, wall, joy /dev/null |/usr/local/bin/myapp When mail is bounced (returned because it could not be delivered), it is always sent from MAILER DAEMON. That alias is needed because users might reply to bounced mail without it, replies to bounced mail would themselves bounce.
Queue Management
Messages can be temporarily undeliverable for a variety of reasons. As a result sendmail will queue up messages that are delayed.
These messages are stored in the QueueDirectory which is defined in the sendmail.cf file
Local delivery
Sendmail will deliver messages to local user, meaning a user with a mailbox on the host where sendmail is running.
Local mail is appended to a users mailbox file.
The local file is often ~/mbox
Remote delivery
Of course sendmail will also deliver mail to other machines. This happens when sendmail determines the user is not local.
By default Sendmail only supports TCP/IP enabled networks though other options are available (uucp, mfax)
Sendmail modes
Usually sendmail runs in Daemon mode –bd, listening for mail but it can be run in: Test mode –bt Just resolve addresses Verify mode –bv Don’t collect or deliver mail Mail sender –bm Just send mail Many others possible, verbose –v…
Sendmail Macros
Sendmail macros allow you to reference text symbolically within the config file. This means you can centrally define values. Some macros are defined by Sendmail for you such as $u, $h enter the following to see some of the macros used by sendmail /usr/lib/sendmail –C/etc/mail/sendmail.cf -bt -d0 *ctrl-+d to exit and no space between –C and /etc…
Sendmail options cont.
Sendmail options are defined in sendmail.cf. Options are declared with an
O
O QueueDirectory=/var/spool/mqueue Other sample variables are: Timeout Timeout.queuewarn=4h Timeout.queuereturn=5d DeliveryMode Background most common TempFileMode DefaultUser LogLevel
Review:
Default MTA in CentOS 6.x is postfix.
Installing sendmail provides two MTA options.
Mail delivery requires DNS support. MX records are defined in DNS similar to how we setup NS records. Three primary roles for mail include: MUA MTA MSA primary sendmail configuration file /etc/mail/sendmail.cf. This file is not typically edited directly.
Review:
Mail is store in the Queue directory before/until delivered Aliases allow mail to be redirected between accounts or services as required.
sendmail is based on rules and rulesets.
messages are processed by these rulesets before being accepted or denied.
The three primary parts of a message are: header: received, date body: everything after the first blank line.
envelope: meta data about the message
In class lab 8b
Lab notes for this session can be found here: http://www.ulcert.uw.edu
-> Class Content -> InClass labs ->
Class 8, Unit 3
What we are going to cover: Reading and printing data What you should leave this session with: How to read data in at the cmd line How to format data for printing
Reading in data
To read in data use read variable eg.
read userinput echo $userinput Or for multiple variables read value1 value2 value3
Read cont.
If more arguments are entered than variables the last variable will store the overflow.
echo -n "enter names: " read name names echo "you entered \$name $name" echo "then you entered \"$names\" to be stored in \ $names “ enter names: TOM SAM JOHN BILL you entered $name TOM then you entered "SAM JOHN BILL" to be stored in $names
Read exit code.
Read always returns an exit status of zero unless the end of file condition is detected from input. This usually means Ctrl+d Knowing this we can use a while loop to read in data at the command line.
while read num1 num2 do echo $(( $num1 + $num2 )) done
User input, yes/no
Using the read function and if/then statements we can check for user acceptance.
echo -n "enter yes/no" read answer if [ "$answer" = yes ]; then echo "you agree!" elif [ "$answer" = no ] ; then echo "you disagree" else echo "I did not understand your answer" fi
Using $$ for uniqueness
The value for $$ is set to the process id for a given process.
Each process ID on Unix or Linux system is unique for that host. So using this value you can create objects that are very unlikely to conflict with other files on the same system.
grep $USER /etc/passwd >> /tmp/userinfo.$$.tmp
printf: print formatted output
Syntax is printf “format” arg1 arg2 e.g.
printf “this is a number: %d\n” 10 printf scans the input, sees %d substitutes the first variable with an argument 10
printf conversions.
printf “octal for %d is %o\n” 20 20
Format characters that are NOT preceded by a percent sign are written to stdout.
octal for 20 is 24
Characters that ARE preceded by a percent sign are called “conversion specifications” and will be converted based on the display command.
printf cont.
Common printf conversion characters d integer c single character s literal characters b literal strings with backslash escape char % percent sign
printf output samples.
printf "string contains backslash: %s\n" "test\string" string contains backslash: test\string printf "string %s and character %c\n" hello A string hello and character A printf “print just the first character: %c\n” QAZW print just the first character: Q
printf general format
%[flags][width][.precision]type Only the % and type are required the others are modifiers.
Flags include - Left justify + precedes integers with -/+ # printf precedes hex integers with 0x or 0X
printf formatting
Printf is typically used to format output. Printf can align output, set columns and justify content as required.
printf "%+d\n%+d\n%+d\n" 10 -10 20 +10 -10 +20 printf "%-20s%-20s\n" Firstname Lastname Firstname Lastname
Review
read variable echo $variable read var0 var1 var2 read exit code zero or true unless end of file detected while true or while read input; do ...
read answer if [ "$answer" = X ] ; then using process id for file names: file.txt.$$ printf used for formatting output.
printf "%-20s%-20s\n" ColumnA ColumnB
Homework
homework for this week posted later tonight.