Transcript Perforce@SAP

Perforce @ SAP
Vladimir Velinov
Build Engineering Team
SAP Labs Bulgaria
Perforce SCM
Perforce@SAP
Demo
Q&A
Perforce SCM
Perforce@SAP
Demo
Q&A
Glossary of terms
Perforce (P4) calls it…
Other SCMs…
add
check in, extract part
branch
codeline, project, stream
changelist
changeset
client workspace
project, view, sandbox, family +
release + component + work area
depot
repository, archive
edit
check out (for edit)
head revision
tip revision, head
integrate
merge, branch
resolve
revert
merge
uncheckout, undo checkout,
unlock revision or part
submit
checkin, put
sync
check out tip, refresh, extract
part, get version
 SAP Labs Bulgaria 2006, Perforce@SAP / Vladimir Velinov / 4
How Perforce works
Master file repository
 SAP Labs Bulgaria 2006, Perforce@SAP / Vladimir Velinov / 5
Perforce server
 Changeslists
 Integration history
 Labels
 Client workspaces
…
 SAP Labs Bulgaria 2006, Perforce@SAP / Vladimir Velinov / 6
P4 users and clients
P4 users
 Unique name (case sensitive)
 Can be organized in groups
 Security is managed by protection tables (admins only)
P4 Clients (client workspace)
 Mapping P4 server depot -> local FS
 Have a local root
 Client view of depot
 Stored in Server DB and keeps the file versions on developers FS
 Write permission is enabled when you edit file and disabled when you
submit changes
 Line ending (CR/LF) is determined automatically
 SAP Labs Bulgaria 2006, Perforce@SAP / Vladimir Velinov / 7
P4 File management
File structure - herarchical
 Example: //depot/project/release/…
File operations
 Add - add new file to depot
 Edit - modifies existing file
 Delete – marks file as deleted
 Obliterate – physically delete file (admins only)
 Branch/Integrate – merge changes from another branch
File types
 Base types – text, binary, symlink, unicode, apple
 File type modifiers
 Storage method
 SAP Labs Bulgaria 2006, Perforce@SAP / Vladimir Velinov / 8
P4 Changelists
What are changelists?




Logical units to organize your changes
Atomic change transactions
Ensure consistency of your work – related changes in files are checked in together
Unique numbers
Creating changelists
 Default changelist
 Lock/unlock files or shared locks
Submiting changelists
 Enter some description
Resolving conflicts
 Manual – merge tool
 Auto resolve
Properties





Number (unique)
Description – text info
List of affected files
user@client
Timestamp
 SAP Labs Bulgaria 2006, Perforce@SAP / Vladimir Velinov / 9
P4 Jobs
Basic defect tracking mechanism
 Description
 Status
 Link to a changelist(s)
 SAP Labs Bulgaria 2006, Perforce@SAP / Vladimir Velinov / 10
P4 Labels
Snapshot of current state of set of files
 Use names instead of numbers
 Can be used when creating releases
 Can be used to restore older releases
 SAP Labs Bulgaria 2006, Perforce@SAP / Vladimir Velinov / 11
Branching
Branching
 Managing changes between two or more sets of related files
 Inter-File branching
 Uses codelines (file sets)
 Source -> Target codeline
 Low overhead - Lazy copy
 SAP Labs Bulgaria 2006, Perforce@SAP / Vladimir Velinov / 12
Integration
Integration
 Propagate changes between two or more codelines
 Keep track of fixes
 Integration history
Next: Revision graph
 SAP Labs Bulgaria 2006, Perforce@SAP / Vladimir Velinov / 13
Revision graph
New feature codeline
Dev codeline
Released codeline
 SAP Labs Bulgaria 2006, Perforce@SAP / Vladimir Velinov / 14
Revision graph – Real life example
Dev codelines
Milestone codelines
Delivery codelines
New feature codelines
Special customer codelines
 SAP Labs Bulgaria 2006, Perforce@SAP / Vladimir Velinov / 15
Perforce vs. CVS
Attribute
CVS
Perforce
Branching and merging
CVS offers branching, but
requires manual tracking of
merge history.
Perforce branching automatically
tracks the history of all branching
operations.
Distributed development
No performance solutions for
remote development are
available.
The Perforce Proxy offers
caching for remote users.
Atomic transactions
Doesn’t have an atomic change
mechanism and cannot group
changes to related files.
Changelists - enabling users to
track file versions associated with
a feature addition or issue
resolution.
Scalability and
performance
Metadata is appended and
maintained in the content of each
file.
Centralized metadata records all
user and file activity.
Defect tracking
Not part of CVS; need to use a
separate defect tracking solution.
Perforce provides a basic defect
tracking - jobs, and integrates
with several 3rd party defect
tracking systems.
Integration with related
tools
Integrations are available via the
open source community.
Many third-party tools are
designed specifically to work with
Perforce.
 SAP Labs Bulgaria 2006, Perforce@SAP / Vladimir Velinov / 17
Perforce vs. Subversion
Attribute
Subversion
Perforce
Branching and merging
Subversion offers branching, but
requires manual tracking of
merge history.
Perforce branching automatically
tracks the history of all branching
operations.
Distributed development
No performance solutions for
remote development are
available.
The Perforce Proxy offers
caching for remote users.
Platform dependencies
Compatibility issues exist with
line ending of files in Unix and
Windows environments.
Line ending issues are solved
automatically.
Scalability and
performance
Limited due to file binary diff
versioning process.
Architected for speed, which
scales linearly.
Defect tracking
Not part of Subversion; need to
use a separate defect tracking
solution.
Perforce provides a basic defect
tracking - jobs, and integrates
with several 3rd party defect
tracking systems.
Integration with related
tools
Integrations are available via the
open source community.
Perforce integrates with most
leading IDEs, build tools,
Microsoft Office, Adobe
Photoshop, and many others.
 SAP Labs Bulgaria 2006, Perforce@SAP / Vladimir Velinov / 18
Perforce vs. Microsoft VSS
Attribute
VSS
Perforce
Platform dependencies
Works over network shares slowing response times. Limited
to Microsoft platforms only.
Client/server architecture.
Response times are much faster
for most operations.
Scalability and
performance
MS recommends that VSS DB
doesn’t exceed 3 to 5 GB, for
perf. and stability reasons.
The Perforce repository can
scale to the multiterabyte range
easily.
Atomic transactions
Doesn’t have an atomic change
mechanism, and cannot group
changes to related files.
Changelists - enabling users to
track file versions with a new
feature or issue resolution.
Branching and merging
Limited support. Branching
relationships aren’t maintained.
Perforce branching automatically
tracks the history of all branching
operations.
Distributed development
Doesn’t offer a scalable
distributed development solution.
The Perforce Proxy offers
caching for remote users.
Integration with related
tools
Only available for IDEs that
support the SCC interface.
Many available tools are
designed specifically to work with
Perforce, including defect
tracking tools.
 SAP Labs Bulgaria 2006, Perforce@SAP / Vladimir Velinov / 19
P4 Installation & Toolset
Installation
 Download from www.perforce.com
 Trial version supports 2 users/clients
 Support for most major platforms - Win/UNIX/MacOS
Toolset





P4D – Perforce server
P4S – Perforce server service (Windows only)
P4 – command line client
P4Win – GUI for Windows
P4V – cross platform GUI
 P4Web – Web GUI for remote access
 P4Proxy – Caching for distributed development
 P4Merge – Perforce merge tool
 P4Diff – Perforce diff tool
 SAP Labs Bulgaria 2006, Perforce@SAP / Vladimir Velinov / 20
Perforce SCM
Perforce@SAP
Demo
Q&A
History of Perforce within SAP
1998


1999

Productive move from ClearCase, SCCS into Perforce
’00-’04

Tool infrastructure development & maintenance

2005
+
Evaluation of SCM systems
First Perforce server at SAP
Major Hardware investments & performance improvements
P4MS – PerforceManagementSystem running on SAP
NetWeaver
80+
2006.1
4,600+
Perforce server
instances
Upgrade of all
instances
Registered
Perforce users
 SAP Labs Bulgaria 2006, Perforce@SAP / Vladimir Velinov / 22
SAP's Global R&D Organization
Canada
Montréal
Germany
45.79%
Walldorf and satellites
Hungary
3.69%
China
Budapest
Shanghai
Bulgaria
4.57%
Sofia
North America
Israel
Palo Alto and satellites
Ra’anana
13.52%
17.74%
India
6.55%
Bangalore
Location
Outbound Inbound
Bulgaria
31.45 GB
0.68 GB
India
11.70 GB
0.18 GB
Israel
954.55 GB
13.72 GB
9.31 TB
2.74 TB
Overall
Monthly transfered volume via P4 protocol
 SAP Labs Bulgaria 2006, Perforce@SAP / Vladimir Velinov / 23
 SAP Labs Bulgaria 2006, Perforce@SAP / Vladimir Velinov / 24
02
02
02
02
01
/1
2
/0
9
/0
6
/0
3
/1
2
/0
2
/0
2
/0
2
/0
2
/0
2
/0
2
/0
2
/0
2
/0
2
/0
2
/0
2
20
/
03 02
/0
3/
0
03 2
/0
6
20 /0
03 2
/0
20 9/0
03 2
/1
20 2/0
04 2
/0
20 3/0
04 2
/0
20 6/0
04 2
/0
20 9/0
04 2
/1
20 2/0
05 2
/0
20 3/0
05 2
/0
20 6/0
05 2
/0
20 9/0
05 2
/1
20 2/0
06 2
/0
20 3/0
06 2
/0
20 6/0
06 2
/0
20 9/0
06 2
/1
2/
02
20
20
20
20
20
/0
9
/0
6
/0
3
/1
2
/0
9
/0
6
/0
2
+11.6% users
last year
20
01
01
01
00
00
00
/0
3
/1
2
SERVERS
20
20
20
20
20
20
00
99
USERS
20
19
Key figures – User/License growth
LEVELS
FILES/REV
CURRENT
Perforce users & licenses
4,700
4,200
3,700
3,200
2,700
2,200
1,700
1,200
700
Key figures – Server growth
USERS
SERVERS
+11,6% users
last year
~8 new
servers
per year
LEVELS
FILES/REV
CURRENT
Perforce servers
100
90
80
70
60
50
40
30
20
10
0
by 31.12.2000
 SAP Labs Bulgaria 2006, Perforce@SAP / Vladimir Velinov / 25
by 31.12.2001
by 31.12.2002
by 31.12.2003
by 31.12.2004
by 31.12.2005*
by 12.06.2006
Key figures – Project & codeline growth
USERS
SERVERS
+11,6% users
last year
~8 new
servers
per year
~2,000 new
projects
last year
LEVELS
FILES/REV
CURRENT
Perforce projects (2nd) & codelines (3rd)
7.000
6.000
5.000
4.000
Projects
Codelines
3.000
2.000
1.000
0
by 31.12.2000
 SAP Labs Bulgaria 2006, Perforce@SAP / Vladimir Velinov / 26
by 31.12.2001
by 31.12.2002
by 31.12.2003
by 31.12.2004
by 17.11.05
by 12.06.2006
Key figures – File & revision growth
USERS
SERVERS
+11,6% users
last year
~8 new
servers
per year
~2.000 new
projects
last year
>10 Mio files
added
last year
LEVELS
FILES/REV
CURRENT
Perforce files & revisions
140.000.000
120.000.000
100.000.000
80.000.000
Files
Revisions/Activity
60.000.000
40.000.000
20.000.000
0
by 31.12.2000
 SAP Labs Bulgaria 2006, Perforce@SAP / Vladimir Velinov / 27
by 31.12.2001
by 31.12.2002
by 31.12.2003
by 31.12.2004
by 17.11.05
by 12.06.2006
Key figures – Current overview
USERS
SERVERS
+11,6% users
last year
~8 new
servers
per year
~2.000 new
projects
last year
>10 Mio files
added
last year
~27 p4 calls
per second
LEVELS
FILES/REV
CURRENT
Current overview
 Active Users / Licenses
4,399
 Server instances
80
 Depots (1st level)
333

Projects (2nd level)
6,143

Codelines unique
6,498

Codelines overall (3rd level)
 Files
 Revisions
 Filesystem consumption
 Average number of p4 calls per day
 SAP Labs Bulgaria 2006, Perforce@SAP / Vladimir Velinov / 28
55,787
46,972,123
116,482,290
1.80 TB
2,303,174
Hardware setup
 Cluster node 1 & 2:
 FSC PrimePower 450N
– CPU:
4 x 1320 MHz
– Memory:
32GB
– OS:
SunOS 9
 Cluster node 3 & 4:
 FSC PrimePower 650
HA ensured by
PRIMECLUSTER
– CPU:
8 x 675 MHz
– Memory:
32GB
– OS:
SunOS 9
 Shared Storage (SAN):
 2 x EMC Clariion CX500 (mirrored)
– Size: 3TB each
 SAP Labs Bulgaria 2006, Perforce@SAP / Vladimir Velinov / 29
HA Setup
Location II
Location I
Cluster Node 1
Cluster Node 2
Cluster Node 3
Cluster Node 4
2 Fibre Channel
SAN switches
2 Fibre Channel
SAN switches
EMC I
Redundant 2 Gbit/s Fibre Channel connections
Redundant Gbit/s Ethernet connections
 SAP Labs Bulgaria 2006, Perforce@SAP / Vladimir Velinov / 30
Mirroring filesystem per service
EMC II
Production process (development phase)
Dev
 Submit changes to ‚dev‘ after local test
 nightly build, central deployment to test
system and test
 Release change for transport to ‚export‘
cons
Maturity
delivery
Export
 Collect changes of ‚dev‘ (gate to patch)
patch
 Transport (copy) content to ‚patch‘ in
regular intervals (e.g. weekly cycle) using a
timestamp
export
last call
collect, synchronize
Patch
 Small group of developers is allowed to
apply fixes for stabilization
 approval by Quality Manager(s)
 Transport (copy) content to ‚cons‘
Cons (consolidated code line)
 ready to be used by dependent
components/products
 SAP Labs Bulgaria 2006, Perforce@SAP / Vladimir Velinov / 31
dev
develop
t
Dependencies – single project
 SAP Labs Bulgaria 2006, Perforce@SAP / Vladimir Velinov / 32
Java projects dependencies – real life example
79 projects
 SAP Labs Bulgaria 2006, Perforce@SAP / Vladimir Velinov / 33
Depot/Directory Structure
depot
project
codeline
 SAP Labs Bulgaria 2006, Perforce@SAP / Vladimir Velinov / 34
Real life example for a change list dependency graph
 SAP Labs Bulgaria 2006, Perforce@SAP / Vladimir Velinov / 35
Source transports between the code lines
Integration




Manual transport (p4 integrate)
Tools provided by Perforce (p4.exe, P4Win and P4V)
Ignores dependencies between changes
Used to create identical codelines = full copy e.g. initial creation of (remote)
maintenance codelines out of development codelines (P4MS)
Perforce@SAP
Consolidation




Controlled integration
Tools developed within SAP based on „p4 integrate“
Takes dependencies between changes into account
Allows bundling of changes
– bundle = set of changes that have to be transported together (e.g. c1 and c2)
 Shows status, user and timestamp of a transport
– N (new), P (consolidation in process), E (error),
W (waiting for dependent change), C (consolidated)
 No multi-server so far
 SAP Labs Bulgaria 2006, Perforce@SAP / Vladimir Velinov / 36
Perforce SCM
Perforce@SAP
Demo
Q&A
Perforce SCM
Perforce@SAP
Demo
Q&A
Questions?
Q&A
 SAP Labs Bulgaria 2006, Perforce@SAP / Vladimir Velinov / 39