Transcript Novell Corporate Presentation Template 2007

Open Source Best Practices
Experiences from within the Apache Software Foundation
Brad Nicholes
Senior Software Engineer, Novell Inc.
Member, Apache Software Foundation
[email protected]
Agenda
•
•
Introduction to Open Source
Open Source Projects
–
•
Rules of Development
–
•
•
2
Apache Software Foundation
Getting Involved
Best Practices
–
Patching
–
Development
–
Distribution
–
Communication
Development Process
© Novell Inc. All rights reserved
Open Source Introduction
“Open Source” is the concept of taking an
idea, opening it up to public discussion
and developing it into something that can
have a broad benefit to an entire
community. In its purest sense, it is an
entirely free and open process whose
contributions are made on a voluntary
basis and whose result benefits all.
3
© Novell Inc. All rights reserved
Open Source Introduction
•
4
What is “Open Source”?
–
Free software that ships with the source code
–
Movement that has had a huge impact on the way software
is developed
–
Collaborative software development process (even chaotic at
times)
–
A driving factor in how companies do business whether they
develop software or simply use it
© Novell Inc. All rights reserved
Open Source – Basic Idea
“The basic idea behind open source is very
simple: When programmers can read,
redistribute, and modify the source code for a
piece of software, the software evolves. People
improve it, people adapt it, people fix bugs. And
this can happen at a speed that, if one is used
to the slow pace of conventional software
development, seems astonishing.”
OSI – http://www.opensource.org
5
© Novell Inc. All rights reserved
Open Source Misconceptions
•
•
6
User point of view
–
A community of hackers with nothing else better to do
–
Only programmers can participate in the Open Source
community
–
Open Source software is typically beta quality
–
It's free, I can do anything I want with the source code
Corporate point of view
–
It’s free, we can’t make money with Open Source
–
The only good software is proprietary software
–
NIH (“Not Invented Here”) Syndrome
© Novell Inc. All rights reserved
Open Source Reality
7
•
All Open Source projects have an owner
•
Anyone can participate in Open Source regardless of
their skill level or technical knowledge
•
Volunteers may actually be paid by a third party to
participate in an Open Source project
•
Results can be freely used by the entire community
•
Results may be used to generate revenue directly or
indirectly
© Novell Inc. All rights reserved
Open Source Legal Issues
•
There are legal and business issues involved with
participating in Open Source Projects:
–
8
Avoid contamination
>
An engineer working on a given proprietary project may not be allowed to
contribute to a specific OSS project
>
Importing GPL code into a proprietary project may force the project to be
released under the GPL
–
Understand copyright issues
–
Committers to projects must do due diligence to resolve
source code ownership
–
Comments and opinions made through your company's
association ([email protected]) must be consistent with
your company's policies and views
© Novell Inc. All rights reserved
Communities and Projects
9
•
A community is an organization made up of ideas and
concepts around a particular discipline, product,
vertical market, technology, development
methodology, etc.
•
A project is centered around the creation,
development, and distribution of a software product.
Projects are often associated with communities.
© Novell Inc. All rights reserved
What is an Open source Project?
•
Collaborative development environment
–
–
–
•
License agreement
–
•
All Open Source code is protected by some type of license
agreement; the license defines the conditions under which the
source code may be used and the obligations with respect to
enhancements and derivative works
Project management
–
–
10
Project contributors are often geographically dispersed and usually
collaborate via email or IRC
Email and email lists are the most popular means of communication.
Thousands of emails are sent and received everyday discussing
various projects
Open Source projects are hosted at sites such as SourceForge.org,
Savannah.gnu.org and Forge.Novell.com
Most Open Source projects are managed by a single individual
Larger projects such as Linux, Apache, OpenLDAP are managed by
a group or organization with assigned roles such as “Release
Manager” or “Sub-system Maintainer”
© Novell Inc. All rights reserved
The Apache Software Foundation
What is the Apache Software
Foundation
•
Membership based non-profit organization
incorporated in the state of Delaware, USA
•
Provides organizational, legal and financial support for
a broad range of open source software projects
•
Provides a hardware, communication and business
infrastructure
•
All volunteer organization
•
Governed in accordance to bylaws
–
12
http://www.apache.org/foundation/bylaws.html
© Novell Inc. All rights reserved
ASF History
13
•
Founded in 1995 as the “Apache Group”
•
Incorporated in 1999 as the Apache Software
Foundation
•
Purpose was to maintain and support the original
HTTPD web server written by the NCSA
•
Became the #1 web server within a year after the
organization of the Apache Group
© Novell Inc. All rights reserved
Apache Name
“The name 'Apache' was chosen from respect for
the Native American Indian tribe of Apache, wellknown for their superior skills in warfare strategy
and their inexhaustible endurance.”
“It also made a cute pun on “a patchy web server”
-- a server made from a series of patches – but
this was not its origin.”
http://www.apache.org/foundation/how-it-works.html#history
14
© Novell Inc. All rights reserved
Philosophy – “The Apache Way”
15
•
Collaborative Software Development
•
Commercial-friendly Standard License
•
Consistently High Quality Software
•
Respectful, Honest, Technical-based Interaction
•
Faithful Implementation of Standards
•
Security as a Mandatory Feature
© Novell Inc. All rights reserved
Meritocracy
16
•
Govern of Merit
•
The more you participate the more responsibility you
merit
•
Easily able to weed out those that were just passing
through from those that really have an interest
•
Allows for all levels of participation
•
The ASF only recognizes individuals rather than
corporations
© Novell Inc. All rights reserved
ASF People
•
Member
–
230+ Members – 13+ Emeriti Members
Nominated by current members and elected on merit
–
Holds the right to vote on ASF related issues
–
All members are also committers
–
•
•
Committer
–
1400+ Committers ... and growing
–
Given 'WRITE' rights to the source code repository
–
Assigned an “apache.org” mail address
Contributor
–
–
•
17
Contributes code and/or documentation in the form of patches
Actively participates on the developer mailing lists
User
–
Uses the Apache software
–
May provide feedback in the form of bug reports or feature suggestions
© Novell Inc. All rights reserved
Infrastructure
•
The ASF is a virtual entity that only exists on the
Internet
•
The only physical existence of the ASF is the server
hardware itself which provides:
•
18
–
Web Services (Sites and Wikis)
–
Source code repositories
–
Mailing list environment
–
Bug tracking
–
Distribution mirroring
All services and hardware are managed by the
Infrastructure Team which is also an ASF project
© Novell Inc. All rights reserved
Getting Involved
My Involvement with the Apache
Developer Community
•
•
•
Project Committer (since March 2001)
–
HTTPD (1.3, 2.0, 2.2)
–
APR
–
APR-Util
Maintaining Apache 1.3 and 2.x for NetWare
Writing/Maintaining Various Apache Modules
–
NetWare
>
–
Cross Platform
>
•
20
Mod_LCGI, Mod_NDS, Mod_HDirs, Mod_RDirs, Mod_EDir, Mod_NW_SSL
Mod_Auth_LDAP, Mod_Authnz_LDAP, Mod_LDAP, Mod_Authn_Alias
Authentication and Authorization
© Novell Inc. All rights reserved
My Involvement with the Apache
Developer Community
21
•
Member of Apache Software Foundation (since May
2002)
•
Member of Various Project Management Committees
•
Conference Presentations & Sponsor
–
BrainShare (U.S. & Europe)
–
ApacheCon (U.S. & Europe)
© Novell Inc. All rights reserved
Becoming an Apache Committer
•
•
22
Making enough noise to get noticed
–
Asking questions
–
Posting patches
–
Doing the work that nobody else wants to
Finding a Niche
–
Apache for NetWare maintainer
–
LDAP authentication
–
Authentication & Authorization
© Novell Inc. All rights reserved
Becoming an ASF Member
•
•
23
Member privileges
–
Nominate others for membership
–
Access to all foundation records
–
Access to Members', Board & Security mailing lists
–
Annual members' meeting
Many opinions about what makes a good ASF
member candidate
–
Active in multiple projects
–
Active in foundation affairs
–
Members must exercise their right to vote
–
Attend the annual members meeting
© Novell Inc. All rights reserved
Best Practices
Rules of Open Source Development
1. Let the source be open
2. Release early, release often
3. Reward contribution with praise
4. Never “fork” a project and continue development
behind closed doors
5. Adapt to the coding style or other standard practices
outlined by the project
25
© Novell Inc. All rights reserved
Rule #1: Let The Source Be Open
•
•
•
•
•
Have no secrets
Make the code and the process that produces it,
public
Encourage third-party peer review
Make sure that others can modify and redistribute the
code freely
Grow the developer, tester, documentation
communities as large as possible
–
•
26
Encourage users and testers to become developers
An efficient feedback channel from users/testers to
developers is a major factor in producing high quality
open source software
© Novell Inc. All rights reserved
Rule #2: Release Early, Release Often
•
•
•
•
27
Rapid release means quick and effective feedback
When incremental releases are small, changing
course to respond to feedback is easier
Make sure that your first release demonstrates
promise
–
It doesn't have to be fully functional
–
It must at least build and show elements of base functionality
Don't make releases a momentous events
–
It is important to streamline your release process so that
releases can be made often and painlessly
–
Resist thinking that the next release must be a polished jewel
that can not ship until everything is perfect
© Novell Inc. All rights reserved
Making Source Code “Open” Source
code
•
Have a clear idea of what problem you are trying to solve and
how the project will be useful to the community
•
Resist the temptation to make the project “Code Complete” or
“Perfect” before releasing it
•
28
–
A new large code base is difficult to review and understand
–
A design that is already set in stone leaves little room for innovation
–
Opportunities to join forces with similar projects are lost which could
have resulted in rapid community growth and acceptance
GO PUBLIC EARLY!
© Novell Inc. All rights reserved
Release Methods
1. Beta, Beta, Beta..., Release and Repeat
–
Avoid thinking that all must be perfect
–
During development, produce as many beta releases as necessary
–
Before a release, produce “Release Candidates”
–
Even if you are not ready to release a beta yet, make any and all
source code available to the community
2. Stable Release Branch vs. Unstable Development Branch
–
Usually implemented by more mature projects
–
Allows maintainability of a stable product while fostering innovation
and creativity for the next major release
3. All of the above...
29
© Novell Inc. All rights reserved
Rule #3: Reward Contribution with
Praise
•
If you can't give co-developers material rewards, give
them psychological ones
–
•
•
30
Being able to attach one's name to a well accepted piece of
software is in itself a reward
Most Open Source developers are motivated by two
things
–
Increased usefulness of the software to them or to their
employer
–
Honor, reputation and exposure that one can earn through
open source software development
Software creation is especially satisfying if a large
audience welcomes it and finds it useful
© Novell Inc. All rights reserved
Rule #4: Never “Fork” a Project
Behind Closed Doors
31
•
Denies the Open Source community the opportunity to
participate
•
Eliminates valuable feedback that would have
increased the quality of the software
•
Eliminates development and testing resources that
could have helped speed up the development process
•
Demonstrates that we really aren't Open Source
community players
•
May violate the license agreement that governs the
source code and all derivative works
© Novell Inc. All rights reserved
Avoid Forking A Project At All
•
•
•
•
32
Your goal should be to participate with and contribute
your work back to the project
If you are porting the source to a new platform, forking
the project will require you to continually re-port the
code
Forking a project dramatically increases the probability
of having to debug and fix issues that have already
been resolved by the community
Forking a project makes it much more difficult to have
your changes accepted back by the project
–
Source code baseline may be very different
–
No relationship established with the project maintainers
© Novell Inc. All rights reserved
Rule #5: Adapt to Project Processes
and Standards
•
33
Use the coding standards and styles that are outlined
by the project
–
Put aside your own style beliefs
–
If no coding standards are specified, use the existing source
code as your guide
–
Same goes for documentation
•
The goal is to establish a relationship with the project,
not to reform it
•
There is no problem with suggesting a change in
process, just be prepared to accept the project's
decision
© Novell Inc. All rights reserved
Best Practices When Working With
Open Source Developers
34
•
Good Patching Practice
•
Good Project and Archive Naming Practice
•
Good Development Practice
•
Good Distribution Making Practice
•
Good Communication Practice
© Novell Inc. All rights reserved
Good Patching Practice
•
•
35
Make sure your patch is easy to read and understand
–
Others need to review the patch and merge it, therefore a sloppy
patch will likely be rejected
–
Sloppy patch creation and submission is usually a sign of sloppy
code
One fix, One patch
–
Don't submit multiple fixes or enhancement in a single patch file
–
If the fixes can be logically separated, submit separate patches with
their own descriptions
–
The more complex the patch, the harder it is to review
–
One patch may span multiple source files but it is still one patch
© Novell Inc. All rights reserved
Good Patching Practice
•
•
•
36
Send patches not files
–
A patch is a unified diff between the original and changed source
–
A patch can include changes to multiple files
–
Send an entire file only if your patch includes a new file
Send patches against the current code base
–
Extremely difficult to determine your changes from changes made
“nn” revisions ago
–
Your patch may actually be obsolete if based on old code
Patching tools
–
Besides a compiler, “diff” and “patch” are the basic tools of Open
Source development
–
CVS and SVN (Subversion) are popular revision control systems
© Novell Inc. All rights reserved
Good Patching Practice
•
37
Send a minimal patch
–
The patch should only contain relevant code changes,
documentation and comments
–
Use the unified diff format not the default
>
“diff -uNrp” is normally used to diff trees
>
“diff -up” for a single file
–
Don't mix code “style” changes with code “logic” changes
–
Include an explanation or description along with the patch
submission of what your patch does or fixes and why
–
Don't be afraid to point out weaknesses
© Novell Inc. All rights reserved
Good Patching Practice
•
•
38
Don't forget the documentation
–
If your patch includes user-visible changes, make sure you
also update the man-pages or any other documentation
–
Comment your code so that it is easier for others to review as
well as to understand later
–
Make note of any user-visible changes or significant changes
or features in the project's CHANGES log
DON'T TAKE IT PERSONALLY IF YOUR PATCH IS
REJECTED
© Novell Inc. All rights reserved
Good Development Practice
•
Don't rely on proprietary tools or utilities
–
The farther you stray from accepted Open Source tools, the less
likely it is that your project or patches will be accepted
–
Keep the build procedure as simple as possible
>
•
39
Configure; make; make install
Stick to standard libraries and API's that are widely available
–
Keep portability in mind
–
Usage of cross-platform libraries will make it easier for others to port
your project to other operating systems
–
It is more likely that your code will be reviewed and accepted by the
project if you keep with accepted standards and RFC's
© Novell Inc. All rights reserved
Good Development Practice
•
•
40
Keep #ifdef's to a minimum if possible
–
Stay away from platform specific code unless absolutely necessary
–
Stay away from compiler or linker specific options
–
Abstract away platform specific code if possible
–
Use common programming practices
Test before you release
–
Provide your own test suite
–
Test or ask for testing results from as many platforms or platform
variations as practical
–
Build with different compilers and test
–
Use testing tools where possible
© Novell Inc. All rights reserved
Good Distribution Build Practice
•
41
Include various description and information files
–
README – new and important information
–
INSTALL – configuration and build instructions
–
CHANGES – significant version-by-version changes
–
COPYING/LICENSE – project license terms
–
FAQ – common questions and answers
–
HISTORY – project history
–
NEWS – high level list of enhancements or features
© Novell Inc. All rights reserved
Good Distribution Build Practice
42
•
Make sure tarballs expand into a single new directory
•
Respect and follow standard file naming practices
•
Provide checksums so that others are able to verify
origins of the tarball, RPM or other types of packages
•
Larger projects will take advantage of mirror sites to
decrease the number of direct downloads and
required bandwidth
© Novell Inc. All rights reserved
Good Communication Practice
•
Host your project on a well known Open Source
hosting site (Forge, SourceForge, Savannah, etc.)
•
Provide various mailing lists that interested parties can
subscribe to
•
43
–
DEV – developers' list
–
HELP – user help list
–
ANNOUNCE – announce new releases or significant changes
–
DOC – documentation development list
Announce new releases or other significant events on
the project's mailing lists as well as other relevant
news groups or community lists
© Novell Inc. All rights reserved
Good Communication Practice - Email
•
44
Because email is the prominent form of collaboration
for OSS projects, good email habits are important
–
Use text mode email clients rather than HTML based. Plain
text has the advantage of being easily quoted and referred to
in replies
–
Retain minimal context when replying to an email by
duplicating the referenced paragraph or quote with indented
“>” symbols rather than including the entire message
–
Stick with short and clean statements rather than a “longwinded” email
–
Long attachments or binary data should be referenced on a
web or ftp site for those that are interested
© Novell Inc. All rights reserved
Good Communication Practice - Email
45
–
Vacation messages or bounces should never to directed to a
public mailing list
–
Large communities tend to attract people who enjoy
controversial, insulting or off-topic discussions. RESIST THE
TEMPTATION TO PARTICIPATE
–
Project participates are expected to speak for the project
rather than their company or personal interest
–
If a project provides several lists with different topics, try to
comply with their policies and avoid posting off-topic
messages
–
Avoid cross-posting messages to a large number of email lists
© Novell Inc. All rights reserved
Good Communication Practice
•
46
Build a web-site that includes
–
Project goals, guidelines and standards
–
Quick link to all downloads and archives
–
HTML-ized version of all documentation
–
Links to source code control system, bug tracking and other
online utilities
–
Information about how to participate
–
Project roadmap and/or schedule
–
Who's who and history
© Novell Inc. All rights reserved
Good Communication Practice
•
47
Always stay positive and constructive
–
All communication should work towards moving the project
forward
–
Suggest ways to make things better rather than different
–
Don't be afraid to point out a problem but be prepared to
accept and correct your own problems
–
Use criticism or rejection as a learning tool for next time
–
Be prepared to accept the decisions of the project maintainers
–
DON'T flame the mailing list
© Novell Inc. All rights reserved
Project Development Cycle
•
Allow users and developers to download the source code
•
Accept changes and enhancements to the project
•
Determine when a release of the software is appropriate
•
Select a Release Manager
•
Tag a snapshot of the source code (optionally provide binaries)
•
Request that the community test the software on the various
platforms and provide feedback
•
Package and post the source and binaries on the web site as a
formal release
•
Announce the release via the email lists and web site
*There is no set time limit on the development cycle. A complete
cycle could happen in a matter of days or months
48
© Novell Inc. All rights reserved
Development Process
•
Download the source code
–
–
–
–
•
Collaborate
–
–
–
49
Most Open Source projects allow anonymous
access to the source code repository
CVS and Subversion are two of the more popular revision control
systems
Variety of client tools (WinCVS, TortoiseCVS, SVN TortoiseSVN)
HTTP view access is also available for most projects
Post messages to various email lists depending on the topic
(Bugs, Development, Support, etc.)
Monitor projects activity and progress by subscribing
to a project’s mailing lists
Easiest way to contribute to an Open Source project
© Novell Inc. All rights reserved
Development Process
•
Report bugs or suggest enhancements
–
–
–
–
–
•
Submit a patch
–
–
–
50
Submit a bug report through the tracking system
or via project’s mailing list
Bugzilla is the most popular bug tracking system for OSS
Include as much detailed information as possible
If possible include test cases that demonstrate the problem
Requests for enhancements can also be submitted in the same
manner
Create a patch file according to the instructions outlined
by the project (usually a simple DIFF listing)
Submit the patch file through the project’s email list
A patch can be submitted against source code or documentation
© Novell Inc. All rights reserved
Open Source Community Etiquette
•
Open source is a UNIX/Linux world
–
•
Stay positive
–
•
These are volunteers, they aren’t paid enough to be abused
by anybody; “You’ll catch more flies with honey than vinegar”
Contribute
–
51
If an implementation doesn’t make sense in the nonUNIX/Linux world, suggest a way to enhance it rather than
criticize it
The community is interested in what you can contribute no
matter what that contribution may be
© Novell Inc. All rights reserved
Secret to Success…
•
52
How to succeed in the Open source community
–
Relationships
–
Relationships
–
Relationships
•
The key to success in the Open source community is
forming good relationships through consistent participation,
contributions and acknowledgements
•
Relationships are built on trust. Trust in openness and honesty,
trust in the abilities, trust in reaching the common goal
© Novell Inc. All rights reserved
Resources
53
•
http://developer.novell.com
•
http://forge.novell.com
•
http://www.oreillynet.com/pub/a/oreilly/opensource/ne
ws/myths_1199.html
•
http://www.linux-foundation.org
•
http://www.catb.org/~esr/writings/cathedral-bazaar
•
http://www.nswscl.org.au/journal/51/Mark_H_Webbink.
html
© Novell Inc. All rights reserved
References
•
The Art of Unix Programming by Eric S. Raymond
–
•
The Cathedral and the Bazaar
–
•
http://www.opensource.org/docs/definition.php
Open Source Licenses
–
54
http://www.catb.org/~esr/writings/cathedral-bazaar/
The Open Source Definition
–
•
http://www.faqs.org/docs/artu
http://www.opensource.org/licenses/
© Novell Inc. All rights reserved
Questions
Unpublished Work of Novell, Inc. All Rights Reserved.
This work is an unpublished work and contains confidential, proprietary, and trade secret information of Novell, Inc.
Access to this work is restricted to Novell employees who have a need to know to perform tasks within the scope of
their assignments. No part of this work may be practiced, performed, copied, distributed, revised, modified,
translated, abridged, condensed, expanded, collected, or adapted without the prior written consent of Novell, Inc.
Any use or exploitation of this work without authorization could subject the perpetrator to criminal and civil liability.
General Disclaimer
This document is not to be construed as a promise by any participating company to develop, deliver, or market a
product. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in
making purchasing decisions. Novell, Inc. makes no representations or warranties with respect to the contents
of this document, and specifically disclaims any express or implied warranties of merchantability or fitness for any
particular purpose. The development, release, and timing of features or functionality described for Novell products
remains at the sole discretion of Novell. Further, Novell, Inc. reserves the right to revise this document and to make
changes to its content, at any time, without obligation to notify any person or entity of such revisions or changes. All
Novell marks referenced in this presentation are trademarks or registered trademarks of Novell, Inc. in the United
States and other countries. All third-party trademarks are the property of their respective owners.