Perl, Webservers (apache) and web applications (cgi-bin)
Download
Report
Transcript Perl, Webservers (apache) and web applications (cgi-bin)
Perl, Webservers (apache)
and web applications (cgi-bin)
1
Corrections
• "Elements to be compared are passed into the
subroutine as the package global variables $a
and $b" (I said any variables could be used -which is wrong). $a and $b are references (the
equivalent of pointers in C) They are exempt
from the usual restrictions on global variables
when using "strict".
• In chr1 -- lowercase letters are "regions of low
complexity" which have been masked by
repeatMasker
2
The final word on &func
PG 114-115
The official name of a subroutine includes the & prefix. A subroutine
may be called using the prefix, but the & is usually optional, so are
the parentheses if the subroutine has been predeclared. (Note,
however, that the & is not optional when you’re just naming the
subroutine, such as when its used as an argument to defined or
undef, or when you want to generate a reference to a named
subroutine by saying $subref = \&name. Nor is the & optional when
you want to do an indirect subroutine call with a subroutine name or
reference using the &$subreff() or &{$subref}() constructs (see chap
4).
Subroutines may be called recursively. If a subroutine is called using
the & form, the argument list is optional, and if omitted, no @_ array
is set up for the subroutine: the @_ array of the calling routine at
the time of the call is visible to called subroutine instead. This is an
efficiency mechanism that new users may wish to avoid.
3
Sorting by Multiple Fields
#!/usr/bin/perl
use strict;
my $i = 0;
my %score = ("tim" => 195, "tracy" => 205, "indy" => 30, "tina" => 195);
# sort keys of hash, based on values of hash
my @keys = keys %score;
# Sorting just the "keys" descending by numerical score, then by key, ascending alphabetical
my @winners = sort by_score_and_name @keys;
foreach $i (@winners) {
print "$i $score{$i}\n";
}
sub by_score_and_name {
$score{$b} <=> $score{$a} #can do multiple levels of sort -- if comparison
or
#result is 0 -- it will "short circuit" to the next one
$a cmp $b;
# and perl returns last evaluated expression
} #note descending order AND ascending order
#tracy 205
#tim 195
#tina 195
#indy 30
4
Hash of Hash
#!/usr/bin/perl
#simple example of hash of hash
# m people X n genotypes
$record{1}{SNP1}
$record{1}{SNP2}
$record{1}{SNP3}
$record{1}{SNP4}
$record{1}{SNP5}
= "AA";
= "AA";
= "BB";
= "AB";
= "AA";
$record{2}{SNP1}
$record{2}{SNP2}
$record{2}{SNP3}
$record{2}{SNP4}
$record{2}{SNP5}
= "BB";
= "AB";
= "BB";
= "AB";
= "AB";
5
Webserver
• A webserver (apache) is a computer
application that provides a service
– the service is to provide text, images, and
programs to clients (browsers IE, Safari,
Firefox, Mozilla, Netscape, etc.)
6
clients
server
address: pdb.eng.uiowa.edu/~tabraun/biotech
text
images
applications
7
Protocols
•
Needed so that the client/server may communicate
–
–
–
–
HTML – hypertext markup language
JPEG (jpg) – image format
others
cgi-bin (perl), Java, Java-script, PHP, Ruby, others
• programming languages that are capable of generating information within these
protocols
• "server-side scripting languages"
– Perl was never designed to be SSSL, however power and flexibility of Perl makes this possible
• cgi – common gateway interface – a protocol for specifying how a server may "serve"
perl to a client
• Instead of static HTML, a perl script runs, and generates the HTML-equivalent "on the
fly", or dynamically.
• client accesses server
• server executes program
• output of program is sent by server/read by client
• Thus, the perl script may provide different results depending on input and the state of
the server (current data available, date, user specified inputs, etc)
• Note that in many cases, the execution time of a "program" is much faster than the
network delays – so output "appears" instantaneous
8
HTML example:
http://pdb.eng.uiowa.edu/~tabraun/biotech/
<html>
<!-- <BODY BGCOLOR="#000000" TEXT="#FFFFFF" link="0099ff" vlink="#00ff99"> -->
<title>51:123 Bioinformatics Techniques</title>
<font SIZE=+4>
51:123 Bioinformatics Techniques<br>
<font SIZE=+2>
Course Web Page
<br>
<hr>
<br>
<br>
<font SIZE=+2>
<b> <a href=BioinfoTechsSyllabus2_51_123.pdf>Syllabus</a></font></b><br>
<br>
<br>
<font SIZE=-1>
Announcements: <br><br>
9/8/2003 The text book is at the IMU.<br><br>
9/15/2003 For Assignment 3, try the <a href=upload.html> submission cgi</a> <br><br>
9/16/2003 Pace of the course material. I have received multiple statements of
concern about the rate at which we are covering course material in the class.
Due to the varied background of students in this class, this feedback is very
valuable to me for trying to adjust the pace of the course. I had already planned
on slowing the pace and including more examples over the next lectures.
However, based on this feedback, it would appear that multiple students would
have preferred this to occur sooner.
If you have an opinion -- send me an e-mail.
Please feel free to send me your feedback. This is the first
time this course has been offered, so I (and other faculty in this area) am very
interested in developing this course to fit the needs of students in this field.
<br><br>
9
What does it mean?
• What does it mean to be able to put a perl
application (cgi-bin) "in" a web page?
• Content of the page may be generated
programatically or dynamically
• Users can interact with the web page
– content displayed depends on state of the
program (day of the week, user input, etc.)
10
Simple Example
http://texas.eng.uiowa.edu:8888/~tabraun/biotech/sequence.html
11
sequence.html
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso8859-1">
</HEAD>
<form action="http://texas.eng.uiowa.edu:8888/~tabraun/cgibin/translate.cgi" method="POST" enctype="multipart/form-data">
<P>Enter filename to upload:
<input type="file" name="file">
<input type="submit">
</form>
</HTML>
12
#!/usr/bin/perl -wT
use strict;
use CGI;
use Fcntl qw( :DEFAULT :flock );
use constant BUFFER_SIZE => 16_000;
use constant MAX_FILE_SIZE => 1_000_000;
#
my $seq = "";
my %aminos = (
"TTT", "F", "TTC", "F", "TTA", "L", "TTG", "L",
"CTT", "L", "CTC", "L", "CTA", "L", "CTG", "L",
"ATT", "I", "ATC", "I", "ATA", "I", "ATG", "M",
"GTT", "V", "GTC", "V", "GTA", "V", "GTG", "V",
"TCT", "S", "TCC", "S", "TCA", "S", "TCG", "S",
"CCT", "P", "CCC", "P", "CCA", "P", "CCG", "P",
"ACT", "T", "ACC", "T", "ACA", "T", "ACG", "T",
"GCT", "A", "GCC", "A", "GCA", "A", "GCG", "A",
"TAT", "Y", "TAC", "Y", "TAA", ".", "TAG", ".",
"CAT", "H", "CAC", "H", "CAA", "Q", "CAG", "Q",
"AAT", "N", "AAC", "N", "AAA", "K", "AAG", "K",
"GAT", "D", "GAC", "D", "GAA", "E", "GAG", "E",
"TGT", "C", "TGC", "C", "TGA", ".", "TGG", "W",
"CGT", "R", "CGC", "R", "CGA", "R", "CGG", "R",
"AGT", "S", "AGC", "S", "AGA", "R", "AGG", "R",
"GGT", "G", "GGC", "G", "GGA", "G", "GGG", "G",);
$CGI::DISABLE_UPLOADS = 0;
$CGI::POST_MAX
= MAX_FILE_SIZE;
my $q = new CGI;
$q->cgi_error and error( $q, "Error transferring file: " . $q->cgi_error );
my $file
= $q->param( "file" ) || error( $q, "No file received." );
my $fh
= $q->upload( "file" );
my $buffer = "";
13
binmode $fh;
#binmode OUTPUT;
print $q->header( "text/plain" ), "File
received.";
print "\n";
# process contents of file
while ( read( $fh, $buffer, BUFFER_SIZE ) ) {
print "$buffer";
$seq = $seq.$buffer;
}
&printOutput($seq);
############ Subroutines
sub printOutput #pass in the sequence
{
my $sequence = "";
my $codon = "";
$sequence = $_[0];
$sequence =~s/\s+//g; #kill newlines?
print "Amino acids:\n";
while(length($sequence) >0)
{
$sequence =~s/(...)|(..)|(.)//;
$codon = $1;
#print $q->print("|$codon| :$aminos{$codon}:");
print "$aminos{$codon}";
}
print "\n";
}
sub error {
my( $q, $reason ) = @_;
print $q->header( "text/html" ),
$q->start_html( "Error" ),
$q->h1( "Error" ),
$q->p( "Your upload was not procesed because
the following error ",
"occured: " ),
$q->p( $q->i( $reason ) ),
$q->end_html;
exit;
14
}
HTML form
http://pdb.eng.uiowa.edu/~tabraun/biotech/upload.html
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<html>
<head>
<meta HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
</head>
<form action="http://pdb.eng.uiowa.edu/~tabraun/cgi-bin/upload.cgi" method="POST"
enctype="multipart/form-data">
<p>Enter filename to upload:
<input type="file" name="file">
<p>Enter Assignment number (1, 2, 3, ...):
<input type="text" name="assignment">
<p>Enter lastname:
<input type="text" name="login">
<p>Enter name of file to upload (can re-name, or leave the same):
<input type="text" name="filename">
<input type="submit">
</form>
15
</html>
Modified Simple cgi-bin:
CGI Programming with Perl, O'Reilly
#!/usr/bin/perl -wT
use strict;
use CGI;
use Fcntl qw( :DEFAULT :flock );
use constant UPLOAD_DIR =>
"/usr/local/apache/data/uploads";
use constant BUFFER_SIZE => 16_384;
use constant MAX_FILE_SIZE => 1_048_576;
# Limit each upload to 1 MB
use constant MAX_DIR_SIZE => 100 *
1_048_576; # Limit total uploads to 100 MB
use constant MAX_OPEN_TRIES => 100;
$CGI::DISABLE_UPLOADS = 0;
$CGI::POST_MAX
= MAX_FILE_SIZE;
my $q = new CGI;
$q->cgi_error and error( $q, "Error transferring
file: " . $q->cgi_error );
my $file
= $q->param( "file" ) || error( $q,
"No file received." );
my $filename = $q->param( "filename" ) || error(
$q, "No filename entered." );
my $fh
= $q->upload( "file" );
my $buffer = "";
if ( dir_size( UPLOAD_DIR ) +
$ENV{CONTENT_LENGTH} > MAX_DIR_SIZE )
{
error( $q, "Upload directory is full." );
}
# Allow letters, digits, periods, underscores,
dashes
# Convert anything else to an underscore
$filename =~ s/[^\w.-]/_/g;
if ( $filename =~ /^(\w[\w.-]*)/ ) {
$filename = $1;
}
else {
error( $q, "Invalid file name; files must start with
a letter or number." );
16
}
# Open output file, making sure the name is unique
until ( sysopen OUTPUT, UPLOAD_DIR . "/$filename",
O_CREAT | O_RDWR | O_EXCL ) {
$filename =~ s/(\d*)(\.\w+)$/($1||0) + 1 . $2/e;
$1 >= MAX_OPEN_TRIES and error( $q, "Unable to
save your file." );
}
# This is necessary for non-Unix systems; does nothing
on Unix
binmode $fh;
binmode OUTPUT;
# Write contents to output file
while ( read( $fh, $buffer, BUFFER_SIZE ) ) {
print OUTPUT $buffer;
}
sub dir_size {
my $dir = shift;
my $dir_size = 0;
# Loop through files and sum the sizes; doesn't
descend down subdirs
opendir DIR, $dir or die "Unable to open $dir: $!";
while ( readdir DIR ) {
$dir_size += -s "$dir/$_";
}
return $dir_size;
}
sub error {
my( $q, $reason ) = @_;
print $q->header( "text/html" ),
$q->start_html( "Error" ),
$q->h1( "Error" ),
$q->p( "Your upload was not procesed because
the following error ",
"occured: " ),
$q->p( $q->i( $reason ) ),
$q->end_html;
exit;
close OUTPUT;
print $q->header( "text/plain" ), "File received.";
}
17
cgi-bin (Common Gateway
Interface – Binary) – upload.cgi
#!/usr/bin/perl -wT
use strict;
use CGI;
use Fcntl qw( :DEFAULT :flock );
use constant UPLOAD_DIR =>
"/home/tabraun/public_html/biotech/hw";
use constant BUFFER_SIZE => 16_384;
use constant MAX_FILE_SIZE => 1_048_576;
#
Limit each upload to 1 MB
use constant MAX_DIR_SIZE => 10 * 1_048_576; #
Limit total uploads to 10 MB
use constant MAX_OPEN_TRIES => 100;
$CGI::DISABLE_UPLOADS = 0;
$CGI::POST_MAX
= MAX_FILE_SIZE;
my $q = new CGI;
$q->cgi_error and error( $q, "Error transferring file: " .
$q->cgi_error );
my $file
= $q->param( "file" ) || error( $q, "No
file received." );
my $login = $q->param( "login" ) || error( $q, "No
login entered." );
my $assignment = $q->param( "assignment" ) ||
error( $q, "No assignment entered." );
my $filename = $q->param( "filename" ) || error(
$q, "No filename entered." );
my $fh
= $q->upload( "file" );
my $buffer = "";
if ( dir_size( UPLOAD_DIR ) +
$ENV{CONTENT_LENGTH} > MAX_DIR_SIZE ) {
error( $q, "Upload directory is full." );
}
# Allow letters, digits, periods, underscores, dashes
# Convert anything else to an underscore
$filename =~ s/[^\w.-]/_/g;
if ( $filename =~ /^(\w[\w.-]*)/ ) {
$filename = $1;
}
else {
error( $q, "Invalid file name; files must start with a
letter or number." );
}
$login =~ s/[^\w.-]/_/g;
if ( $login =~ /^(\w[\w.-]*)/ ) {
$login = $1;
}
else {
error( $q, "Invalid login name; files must start with
a letter or number." );
}
if ( $assignment =~ /^(\d+)/ ) {
$assignment = $1;
}
else {
error( $q, "Invalid login name; files must start with
a letter or number." );
}
18
upload.cgi continued
mkdir
"/home/tabraun/public_html/biotech/hw/$logi
n";
mkdir
"/home/tabraun/public_html/biotech/hw/$logi
n/$assignment";
# Open output file, making sure the name is
unique
until ( sysopen OUTPUT, UPLOAD_DIR .
"/$login/$assignment/$filename", O_CREAT |
O_RDWR | O_EXCL ) {
$filename =~ s/(\d*)(\.\w+)$/($1||0) + 1 . $2/e;
$1 >= MAX_OPEN_TRIES and error( $q,
"Unable to save your file $filename." );
}
# This is necessary for non-Unix systems; does
nothing on Unix
binmode $fh;
binmode OUTPUT;
# Write contents to output file
while ( read( $fh, $buffer, BUFFER_SIZE ) ) {
print OUTPUT $buffer;
}
close OUTPUT;
print $q->header( "text/plain" ), "File received.";
sub dir_size {
my $dir = shift;
my $dir_size = 0;
# Loop through files and sum the sizes;
doesn't descend down subdirs
opendir DIR, $dir or die "Unable to open
$dir: $!";
while ( readdir DIR ) {
$dir_size += -s "$dir/$_";
}
return $dir_size;
}
sub error {
my( $q, $reason ) = @_;
print $q->header( "text/html" ),
$q->start_html( "Error" ),
$q->h1( "Error" ),
$q->p( "Your upload was not procesed
because the following error ",
"occured: " ),
$q->p( $q->i( $reason ) ),
$q->end_html;
exit;
}
19
Server Configuration
• Need a web server (apache)
• Need perl
– specify permissions (writing/reading/access)
– location of scripts so web-server knows where to find
them – apache configuration files
• Other system issues
– access through the firewall – web server is on port 80
• a firewall may be thought of as a set of rules that specifies
the remote address space that may access the (local) server
• and "ports" that may be accessed
20
Notes
• Note that the cgi-bin generated HTMLformatted text was sent by the apache
web server to the web-browser (client)
• Used some "objects" – CGI.pm – that we
don't know about yet.
• Leads us to perl modules, objects, and
eventually bioperl
21
Take Home Messages
• Perl was originally designed for "text" processing
• Powerful pattern matching, regular expressions,
and flexibility of the language make it adaptable
to other tasks (server-side scripting language)
• You should know about Apache, cgi-bin
programming, and Perl (but we will not do any
cgi programming in this class)
• Making tools and techniques available on the
web is very common
– (examples: Primer3, BLAST, Ensmart, local blast)
22
Another example
http://genome.uiowa.edu/cgibin/distribution_info/index.cgi
At: pdb: /home/httpd/cgi-bin/distribution_info
Look at code VS "page-source"
Why does code look like perl -- and not page-source?
23