Transcript Makefile - VAST Lab

The Makefile utility
Motivation
 Small programs
single file
 “Not so small” programs :
– Many lines of code
– Multiple components
– More than one programmer
Mozilla
Mozilla Directory Structure
One very large directory tree
26,000 files
2,500 subdirectories
240 file types
Mozilla Directory References
1,521 directory references
Avg. fan-in/fan-out 12
Max fan-in=100
Max fan-out=42
Median fan-in=5
Median fan-out=12
Build-level
File (atomic entity)
–Source
–Documentation
Directory tree (container) = source tree
Build process
–To build/install software
–Driven by make/ANT/…
Configuration process
–To control build process
–Driven by configure/configuration files/…
Mozilla Build Level
1,350 Makefiles
40,000 LOC build instructions
16,000 LOC configuration
Cyclic dependencies
Two-phase build process
Centralized build/configuration knowledge
Code duplication
Component implementations scattered
Motivation – continued
 Problems:
– Long files are harder to manage
(for both programmers and machines)
– Every change requires long compilation
– Many programmers can not modify the
same file simultaneously
– Division to components is desired
Motivation – continued
 Solution : divide project to multiple files
 Targets:
– Good division to components
– Minimum compilation when something is
changed
– Easy maintenance of project structure,
dependencies and creation
Project maintenance
 Done in Unix by the Makefile mechanism
 A makefile is a file (script) containing :
– Project structure (files, dependencies)
– Instructions for files creation
 The make command reads a makefile,
understands the project structure and
makes up the executable
 Note that the Makefile mechanism is not
limited to C programs
Project structure
 Project structure and dependencies can be
represented as a DAG (= Directed Acyclic
Graph)
 Example :
– Program contains 3 files
– main.c., sum.c, sum.h
– sum.h included in both .c files
– Executable should be the file sum
sum (exe)
sum.o
main.o
main.c
sum.h
sum.c
sum.h
Make: Header Dependencies

What if …
– Sensor.cc changes?
– Sensor.h changes?
– Robot.h changes?




Pattern rule ignores header
dependencies!
Requires unnecessary “make
clean; make”
Let gcc figure out the header
dependencies for us!
The –MM option produces make
rules in a .d file which we can
then include in the Makefile
Sensor.h
Sensor.cc
Robot.h
Robot.cc
robotest.cc
makefile
sum: main.o sum.o
gcc –o sum main.o sum.o
main.o: main.c sum.h
gcc –c main.c
sum.o: sum.c sum.h
gcc –c sum.c
Rule syntax
main.o: main.c sum.h
gcc –c main.c
Rule
tab
dependency
action
Equivalent makefiles
 .o depends (by default) on corresponding
.c file. Therefore, equivalent makefile is:
sum: main.o sum.o
gcc –o sum main.o sum.o
main.o: sum.h
gcc –c main.c
sum.o: sum.h
gcc –c sum.c
Equivalent makefiles - continued
 We can compress identical dependencies
and use built-in macros to get another
(shorter) equivalent makefile :
sum: main.o sum.o
gcc –o $@ main.o sum.o
main.o sum.o: sum.h
gcc –c $*.c
make operation
 Project dependencies tree is constructed
 Target of first rule should be created
 We go down the tree to see if there is a target
that should be recreated. This is the case when
the target file is older than one of its
dependencies
 In this case we recreate the target file according
to the action specified, on our way up the tree.
Consequently, more files may need to be
recreated
 If something is changed, linking is usually
necessary
make operation - continued
 make operation ensures minimum
compilation, when the project structure is
written properly
 Do not write something like:
prog: main.c sum1.c sum2.c
gcc –o prog main.c sum1.c sum2.c
which requires compilation of all project when
something is changed
Make operation - example
File
sum
main.o
sum.o
main.c
sum.c
sum.h
Last Modified
10:03
09:56
09:35
10:45
09:14
08:39
Make operation - example
 Operations performed:
gcc –c main.c
gcc –o sum main.o sum.o
 main.o should be recompiled (main.c is
newer).
 Consequently, main.o is newer than sum and
therefore sum should be recreated (by relinking).
Another makefile example
# Makefile to compare sorting routines
BASE = /home/blufox/base
CC
= gcc
CFLAGS = -O –Wall
EFILE
= $(BASE)/bin/compare_sorts
INCLS = -I$(LOC)/include
LIBS
= $(LOC)/lib/g_lib.a \
$(LOC)/lib/h_lib.a
LOC
= /usr/local
OBJS = main.o another_qsort.o
compare.o quicksort.o
chk_order.o \
$(EFILE): $(OBJS)
@echo “linking …”
@$(CC) $(CFLAGS) –o $@ $(OBJS) $(LIBS)
$(OBJS): compare_sorts.h
$(CC) $(CFLAGS) $(INCLS) –c $*.c
# Clean intermediate files
clean:
rm *~ $(OBJS)
Example - continued
 We can define multiple targets in a makefile
 Target clean – has an empty set of
dependencies. Used to clean intermediate
files.
 make
– Will create the compare_sorts executable
 make clean
– Will remove intermediate files
Passing parameters to makefile
 We can pass parameters to a makefile by
specifying them along with their values in
the command line.
 For example:
make PAR1=1 PAR2=soft1
will call the makefile with 2 parameters:
PAR1 is assigned the value “1” and PAR2
is assigned the value “soft1”. The same
names should be used within the makefile
to access these variables (using the usual
“$(VAR_NAME)” syntax)
Passing parameters - continued
 Note that assigning a value to a variable
within the makefile overrides any value
passed from the command line.
 For example:
command line : make PAR=1
in the makefile:
PAR = 2
 PAR value within the makefile will be 2,
overriding the value sent from the command
line
Conditional statements
 Simple conditional statements can be
included in a makefile.
 Usual syntax is:
ifeq (value1, value2)
body of if
else
body of else
endif
Conditional statements - example
sum: main.o sum.o
gcc –o sum main.o sum.o
main.o: main.c sum.h
gcc –c main.c
#deciding which file to compile to create sum.o
ifeq ($(USE_SUM), 1)
sum.o: sum1.c sum.h
gcc –c sum1.c –o $@
else
sum.o: sum2.c sum.h
gcc –c sum2.c –o $@
endif
Make: Advanced Options

Text manipulation functions
– $(patsubst pattern,replacement,text)
– $(patsubst %.o,%.cc,<list of objfiles>)

Pattern rules
– Uses a pattern in the target with % as wildcard
– Matched % can be used in dependencies as well
– Simple Example:
%.o : %.cc
<tab>command …

Pattern rules with automatic variables
–
–
–
–
$@ full target name
$< first dependency
$* string which matched % wildcard
Advance Example:
%.o : %.cc
<tab>$(CC) $(CCFLAGS) –c $< $(INCPATHS)
Make: A Simple Example
CC=g++
FLAGS=-g
MASLAB_ROOT=maslab-software
LIB_DIR=$(MASLAB_ROOT)/liborc
INC_DIR=$(MASLAB_ROOT)/liborc
LIBS=-lm –lpthread -lorc
#
#
#
#
#
#
Compiler to use
Compile flags
Maslab software root directory
orc-related library directory
orc-related include directory
Library files
all : helloworld
helloworld.o : helloworld.cc
$(CC) $(FLAGS) –c $*.cc –o $@
helloworld: helloworld.o
$(CC) -o helloworld helloworld.o $(LIBS)
clean:
rm -f *.o helloworld
Make: Example Makefile
MASLABROOT = /mnt/maslab/software/maslab-software-current
#This is the list of places to look for include files
INCPATHS = -I$(MASLABROOT)/libs/liborc \
-I$(MASLABROOT)/libs/libim \
-I$(MASLABROOT)/libs/libbotserver \
-I/opt/intel/ipp/include
#This is the list of places to look for libraries
LIBPATHS = -L$(MASLABROOT)/libs/liborc \
-L$(MASLABROOT)/libs/libim \
-L$(MASLABROOT)/libs/libbotserver \
-L/opt/intel/ipp/sharedlib
#This is the names of the libraries
LIBS = -lippipx -lippcvpx -lim -lorc -lbotserver -lm -lpthread -ljpeg -lpng
#This is the compiler to use
CC = g++
# This is your c++ file extension (usually cc or cpp)
CCEXT = cc
Make: Example Makefile (cont)
CCFLAGS = -g -Wall
# This rule builds everything.
# programs in this list.
all : robotest
You should put the names of all your
# This rule says how to turn any .cpp file into a .o file.
%.o : %.$(CCEXT)
$(CC) $(CCFLAGS) -c $< $(INCPATHS)
#---------------------------------------------------------------------# robotest
PROGRAM_NAME = robotest
PROGRAM_OBJS = robotest.o BumpSensor.o Robot.o
GLOBAL_OBJS += $(PROGRAM_OBJS)
JUNK += $(PROGRAM_NAME)
$(PROGRAM_NAME): $(PROGRAM_OBJS)
$(CC) $(PROGRAM_OBJS) -o $(PROGRAM_NAME) $(LIBS) $(LIBPATHS)
chmod a+x $(PROGRAM_NAME)
Extending Example Makefile
So to add a new executable program, simply cut and
paste the robotest section and enter your own
PROGRAM_NAME and PROGRAM_OBJS
#---------------------------------------------------------------------# Sensor Incremental Test
PROGRAM_NAME = sensor-test
PROGRAM_OBJS = sensor.t.o BumpSensor.o
GLOBAL_OBJS += $(PROGRAM_OBJS)
JUNK += $(PROGRAM_NAME)
$(PROGRAM_NAME): $(PROGRAM_OBJS)
$(CC) $(PROGRAM_OBJS) -o $(PROGRAM_NAME) $(LIBS) $(LIBPATHS)
chmod a+x $(PROGRAM_NAME)
Extending example Makefile
DEPENDS += $(patsubst %.o, %.d, $(GLOBAL_OBJS))
JUNK += $(DEPENDS)
include $(DEPENDS)
depend : $(DEPENDS)
depend.$(CCEXT) = set -e; $(CC) $(CCFLAGS) -MM $(DEFS) $(INCPATHS)
depend.filt = sed 's/\($*\)\.o[ :]*/\1.o $@ : /g' > $@; [ -s $@ ] || rm -f $@
%.d: %.$(CCEXT)
$(depend.cc) $< | $(depend.filt)
#======================================================================
# Miscellaneous Rules
#---------------------------------------------------------------------# This rule cleans your directory. You could also add commands here
# to remove program files and any other files that should be cleaned
clean:
rm -f $(JUNK) *~ *.o core.[0-9]* core
ANT:
Another Nice Tool
What is Ant?
 Java-based Build tool from Apache
 De facto standard for building, packaging, and
installing Java applications
 Accomplishes same objectives that make does on
Unix based systems
 Files are written in XML
Why Ant?
 Many claim.. That
– Unlike makefiles, Ant files work cross platform
- No need for multiple, complex makefiles
depending on the operating system.
- Tasks declared in platform independent way; Ant
engine translates to OS specific commands.
But still need to know (potentially) complex
path/install data. So not as general as they claim
 Easy to create own Ant “tasks”, in addition to core
tasks
Installing Ant
 Download Ant binary distribution from:
http://ant.apache.org/bindownload.cgi
 Set ANT_HOME to where you installed Ant
 Include $ANT_HOME/bin in PATH
 Make sure JAVA_HOME is set to point to JDK
Running Ant
 Type “ant” at the command line
 Automatically looks for build.xml file in current
directory to run
 Type “ant –buildfile buildfile.xml” to specify
another build file to run.
Ant Output
Sample build.xml
<project name="MyProject" default="dist" basedir=".">
<property name="src" location="src"/>
<property name="build" location="build"/>
<property name="dist" location="dist"/>
<target name="init">
<tstamp/>
<mkdir dir="${build}"/>
</target>
<target name="compile" depends="init" >
<javac srcdir="${src}" destdir="${build}"/>
</target>
<target name="dist" depends="compile" >
<mkdir dir="${dist}/lib"/>
<jar jarfile="${dist}/lib/MyProject-${DSTAMP}.jar" basedir="${build}"/>
</target>
<target name="clean" >
<delete dir="${build}"/>
<delete dir="${dist}"/>
</target>
</project>
Ant Overview: Project
 Each build file contains exactly one project and at
least one target
 Project tags specify the basic project attributes
and have 3 properties:
- name
- default target
- basedir
 Example: <project name=“MyProject” default=“build” basedir=“.”>
Ant Overview: Targets




Target is a build module in Ant
Each target contains task(s) for Ant to do
One must be a project default
Overall structure of targets:
<target name="A"/>
<target name="B" depends="A"/>
<target name="C" depends="B"/>
<target name="D" depends="C,B,A"/>
Ant Overview: Tasks
 Each target comprises one or more tasks
 Task is a piece of executable Java code (e.g.
javac, jar, etc)
 Tasks do the actual “build” work in Ant
 Ant has core (built in) tasks and the ability to
create own tasks
Ant Overview: Tasks
Example:
<target name="prepare" depends="init“ >
<mkdir dir="${build}" />
</target>
<target name="build" depends="copy" >
<javac srcdir="src" destdir="${build}">
<include name="**/*.java" />
</javac>
</target>
Ant Overview: Core Tasks







javac – Runs the Java Compiler
java – Runs the Java Virtual Machine
jar (and war) – Create JAR files
mkdir – Makes a directory
copy – Copies files to specified location
delete – Deletes specified files
cvs – Invokes CVS commands from Ant
Ant Overview: Writing Own
Task
 Create a Java class that extends org.apache.tools.ant.Task
 For each attribute, write a setter method that is
public void and takes a single argument
 Write a public void execute() method, with no
arguments, that throws a BuildException -- this
method implements the task itself
Ant Overview: Properties
 Special task for setting up build file properties:
 Example:
<property name=“src” value=“/home/src”/>
 Can use ${src} anywhere in build file to denote
/home/src
 Ant provides access to all system properties as if
defined by the <property> task
Ant Overview: Path Structures
 Ant provides means to set various environment
variables like PATH and CLASSPATH.
 Example of setting CLASSPATH:
<classpath>
<pathelement path="${classpath}"/>
<pathelement location="lib/helper.jar"/>
</classpath>
Command Line Arguments
 -buildfile buildfile – specify build file to use
 targetname – specify target to run (instead of
running default)
 -verbose, -quiet, -debug – Allows control over
the logging information Ant outputs
 -logger classname – Allows user to specify their
own classes for logging Ant events
IDE Integration
 Eclipse, NetBeans, JBuilder, VisualAge, and
almost any other Java IDE has Ant integration
built-in to the system
 Refer to each IDE’s documentation for how to use
Ant with that IDE
Web Development with Ant
 Tomcat comes with special Ant tasks to ease Web
application development and deployment
 Copy $TOMCAT_HOME/server/lib/catalina-ant.jar to
$ANT_HOME/lib
 Ant tasks for Tomcat:
- install
- reload
- deploy
- remove
Documentation/References
 Good tutorial for makefiles
http://www.gnu.org/manual/make-3.80/make.html
 Good tutorial for cmakef
http://www.cmake.org/HTML/Documentation.html
 ANT User Manual:
http://ant.apache.org/manual/index.html
 Sun’s Web development tutorial (Ant and JSPs):
http://java.sun.com/webservices/docs/1.2/tutorial/doc/GettingStarted3.html
Standardized Build Interface
Different build systems exist, e.g.,
–make, ANT, shell scripts, IDE
Different software systems require different build
actions, e.g.,
–make
–make bootstrap, make, make install
make
make
make
make
make
clean
all
check
(un)install
dist
The real Problem
 How do we handle platform specific issues?
– Providing a different Makefile for each architecture
– Using Autoconf, Automake and Libtool
 The installer needs only
– Bourne shell
– C compilers
– Make program
Standardized Configuration
Interface
Different mechanisms exist to control software
construction, e.g.,
–configuration files, configuration tools, Makefile
editing
Standardized configuration interface to enable
uniform compile-time configuration:
configure
configure
configure
configure
configure
--help
--prefix=/usr
--with-aterm=/usr/lib
--with-optimization=true
--with-debug=false
Explicit Context Dependencies
Dependencies on build-level components are
declared in configuration interfaces
> configure --help
…
--with-aterm=DIR
--with-sglr=DIR
…
use ATerm Library at DIR
use SGLR Parser at DIR
Independent Deployment
Build-level components are deployed as packages
A package is a versioned release of a build-level
component, e.g.,
firefox-1.0.tar.gz
Packages are published on web/ftp sites
Third-party Composition
A configuration interface enables late-binding of
dependencies
> configure --with-aterm=/usr/local/aterm
--with-sglr=/usr/local/sglr
--with-…=
Compositions are thus not predefined
and can be defined by a third party
Some advantages when using GNU
autotools
 The installation of a program is straightforward:
./configure; make; make install
 This procedure checks for system parameters, libraries,
location of programs, availability of functions and writes
a Makefile
 ./configure supports many options to overwrite
defaults settings
GNU autoconf
Source Code
configure.ac
(configure.in)
aclocal
autoscan
autoconf
configure.scan
configure
GNU automake
Makefile.am
automake
Makefile.in
configure
Makefile
configure.ac

dnl Comment … …

AC_INIT(main.c)

AM_INIT_AUTOMAKE(project_name, 1.2.8)

AM_PROG_LIBTOOL
it supports libtool and shared libraries

AC_PROG_CC
it locates the C (C++) compiler

AC_HEADER_STDC
it checks for standard headers

AC_CHECK_HEADERS(sys/time.h /header.h)
it checks for headers availability

AC_CHECK_LIB(crypto SSLeay_version)
it checks for libraries availability

AC_CHECK_FUNCS(ctime)
it checks for functions availability

AC_PROG_INSTALL
it checks for BSD compatible install utility

AC_OUTPUT([Makefile])
(or AC_PROG_CXX)
Makefile.am
 bin_PROGRAMS = foo
/configure --prefix=… (default /usr/local)
 foo_PROGRAMS=foo.c foo.h
 noist_PROGRAMS=test
(make compiles, make install does nothing)
 EXTRA_DIST=disclaimer.txt
Example

foo.c :
#include <stdio.h>
main()
{
printf(“Cum grano salis\n");
}

Makefile.am :
bin_PROGRAMS = foo
foo_SOURCES = foo.c

configure.ac :
AC_INIT(foo.c)
AM_INIT_AUTOMAKE(latin_words, 0.9)
AC_PROG_CC
AC_HEADER_STDC
AC_PROG_INSTALL
AC_OUTPUT([Makefile])
Summary
 Source Code, configure.ac, Makefile.am

autoscan; aclocal; autoconf
 Create NEWS README AUTHORS ChangeLog

automake –add-missing
 ./configure; make; make dist
 Result: project_name-2.10.tar.gz
aclocal.m4
autom4te-2.53.cache
ChangeLog
config.status
configure.in
COPYING
install-sh Makefile.am
missing
NEWS
README
AUTHORS
autoscan.log
config.log configure
configure.scan
INSTALL Makefile.in mkinstalldirs code.c
References

GNU Autoconf, Automake, and Libtool
http://sources.redhat.com/autobook/autobook/autobook_toc.html

GNU Autoconf Manual
http://www.gnu.org/manual/autoconf

GNU Automake Manual
http://www.gnu.org/manual/automake

GNU Libtool Manual
http://www.gnu.org/manual/libtool

Learning the GNU development tools
http://autotoolset.sourceforge.net/tutorial.html

The GNU configure and build system
http://www.airs.com/ian/configure/configure_toc.html

GNU macro processor (GNU m4)
http://www.gnu.org/manual/m4-1.4/m4.html