Document

Transcript Document

Using Atlassian Products in an Enterprise Environment

Mark McCormack British Telecom Inlife Support Lead Development Services Organisation

Outline

• • • • • • • • • • • Using Atlassian products in an Enterprise Atlassian Product adoption Size of Confluence and Jira in our Enterprise Some Atlassian Use Cases Customisations Application Monitoring Platform What’s Next Product Wishlist Some Lessons Learnt Q & A © British Telecommunications plc

About

• • • • • • • Network is available in 173 countries. Around 3,000 new customer sites every month Carry 80% of all credit card transactions in the UK. More than 90% of the UK’s financial institutions use our secure data networks Rolling out super-fast broadband to 40% of the UK by 2012. FTTC downloads of 40Mb/s and uploads of 2 Mb/s. FTTP downloads of 100 Mb/s

London 2012 Olympics

Development Services Organisation

• • We advise on best practices for all tools Inlife team looks after all hosted applications: – We are an Atlassian shop – Subversion , Jive Forums , Nexus – Other in-house tools • Authz , Teams , IPPI , Provisioning , Static Data • SDK Catalogue & PALMS (will soon become BT Enterprise Repository) © British Telecommunications plc

Product History

• •

Dec 2003

5 people created a team (called code red) but there was no way of coordinating their efforts Bought Confluence and Jira and installed it on a PC and put it in a meeting room • •

June 2004

Usage spread to a few development teams (200 users) Bought new hardware and installed it in a server room (engineering.intra.bt.com) •

April 2006

Another group installed a separate instance of Jira and Confluence to manage service improvements • • •

February 2007

Management Services and Engineering decided joining forces would be a good idea Userbase reached 300 and 2000 respectively http://collaborate.intra.bt.com was born and hosted by VS Group on VMWare.

• •

July 2007

A Virtual Administrator crew formed out of 35 volunteers from the userbase to perform end user support and administer the platform Two users in particular did most of the Admin work. Userbase grows to over 9000 people.

Product Maturity

• •

August 2007

Sponsors found to pay for new hardware & uplift plan created Team split started to disrupt progress to achieve this and ownership was contested •

September 2007

Userbase grows to 10,000 • •

October 2007

I/O Capacity of the disks in the VSP environment started to be exceeded.

Platform started to have outages with huge traffic increase • •

January 2009:

Moved to an enterprise hardware infrastructure, Oracle 10G RAC & 24x7 support Collaborate Wiki was formed, Confluence 2.10 and integrated with Crowd •

February 2009:

Introduced Storm & Jira Issues •

July 2009:

Started to assimilate other wikis (like twiki, rdrwiki, ProfComms) © British Telecommunications plc

Confluence Numbers

• • • • • • 800,000 pages 102,000 attachments 3000 spaces 25,000 display requests per day 30,000 actions per day (eg login, edit, comments, rss feeds etc)

Pages

Current, 100,000 Old; 700 000

44 000

Old, 27,000

Attachments

Current; 75 000

13 000 Users

Contributers

• • • • •

Numbers

3 flavours of Jira ACF – Architecture Conformance Framework Issue Tracking (100 projects) Story Management Operating Model BT is adding > 20,000 stories every month This Week 1,000

5,500 Users

4,500

110,000 Stories 400 Spaces

Confluence Use Cases

• • • • • • • • • Tool Guides on how to get started Tool Access Request Forms – We have a teams application we encourage before requesting a Space – Early stages of a Provision Application Application Deployment Schedule Collaboration between Dev and Inlife Teams (geographically dispersed) Don’t send emails but rather send Confluence links Trying to have an FAQ before raising an issue for help Application Usage Statistics MyBT has a dashboard interface and makes use of the Confluence remote API and render as html inside a portlet.

Most of the Ajax related actions don’t work © British Telecommunications plc

Confluence Customisations

• • • • Implemented the functionality of security classification requirements in the scaffolding plugin.

To implement this functionality the edit page / do edit page actions of the confluence needs to be overridden. Scaffolding plugin was already overriding the edit actions of confluence.

Confluence doesn't allow two plugins to override a particular action.

Extended the edit actions to include classification © British Telecommunications plc

More Confluence Customisations

• • • • https://collaborate.bt.com/staticdata plugin to provide a LOB picker from LDAP Static Data Macro that retrieves platforms BT Space Admin Confluence plugin makes use of our eorg DB in order to get the Line of Business functionality associated with a Space Modified the Excel Plugin as it was hanging Confluence a lot – We had one user with a page that had 48 references to the excel macro – Every reference was an API call that forced a GC, so we disabled explicit GC – Page went from 14 minutes to load to about 4 seconds © British Telecommunications plc

Customisations

• • • • • • Hierarchy plugin Static Data plugin to strip LOB info out from LDAP Loads of constraints on Story creation and workflow – All Stories must have at least one child and at least on Acceptance Criteria – Organogram Added Active User Report under Users, Groups and Roles Roles for Storm Loads of different workflows (put issues On Hold whilst with Dev team or Vendor) © British Telecommunications plc

More Customisations

• • • • https://collaborate.bt.com/fisheye plugin to allow the authorisation of SVN repositories (handled by hasPermissionToAccess method which is overrided to authorise against our Authz).

Use https://collaborate.bt.com/crowd https://collaborate.bt.com/forum client libraries for apps like We have a Jira Project that helps us keeps track of Subversion Projects ensuring everyone has a Salsa ID.

Unified Rolling Calendar Plugin © British Telecommunications plc

Application Monitoring

• ZXTM • Nagios alerts via email for slow (or dead) apps.

• JMX is enabled for most of our java apps • Disk space (e.g shell script to alert > 80%) • Backup successes or failures uploaded to our wiki • Access logs with IP tracking • Usage statistics to figure out what our users are doing • Perl script we run for snapshots of ugly apps • YourKit Profiling from within IntelliJ • kSar © British Telecommunications plc

• • • • • • • •

Platform

Currently 18 HP Blades mostly BL460s with 2 Quad cores and 32GB of RAM for our existing hosted application servers RHEL 5 (soon OEL) Were using Weblogic 10.3 but now Tomcat 6 Java 6.0.18

ZXTM 5.0

Oracle 10g in a Cluster hosted by another team on HP Blade 680s with 2 Quad cores and 64GB of RAM Crowd is our only clustered application (currently) We use Capistrano to help automate our deployments and allows easy rollback © British Telecommunications plc

What’s Next?

• • • • • • • Clustering of Confluence, Jira and SVN Triple our hardware for Disaster Recovery Archiving & stats analysis plugins (customised) Improve provisioning All the Atlassian 2s (Bamboo, Crowd, Fisheye, Crucible) although upgrading to Fisheye 2.0 will be expensive on disk space.

Start charting long-term trends with better monitoring Assimilate all wikis (like LimsWiki for one) © British Telecommunications plc

Wishlist

• • • • • • Confluence Wishlist Tighter integration with Jira Allow Jira Issues to be raised via Confluence and vice versa Allow profiles to have attributes (like team, group or project) Confluence attachment storage API. We want to store attachments away from Confluence and write our own adaptors to do this.

Improve the edit permissions cascading security Customisations of actions (eg. Overrides to allow > 1 plugin) • • • • • Jira Wishlist Allow two columns for attributes Allow picklists to show more values without scrolling Smart forms that can include wiki macros How can we hide Administrators from assignee list?

Can Greenhopper scale? How compatible is it with existing 3.13.2 plugins/customisations?

More Wishes

• • • Jira … Would like to use our Issues as a source for FAQ before users raise an issue? Ought to be Confluence tightly coupled with Jira Issue labels Timebooking to allow resource Management?

• • • Others Fisheye 2 IE6 browser support Make Crucible a viable app for the enterprise (we have some open Atlassian issues for 10k+ users) Can we turn on “remember my login on this computer” checkbox during login?

• • • • •

Lessons Learnt

Be wary of any plugin – Excel Macro plugin – We turned off the usage tracking plugin – We trialled the stats analysis plugin but its currently off since we got hammered two lunchtimes in a row shortly after going live WebLogic seems to be a lot slower than tomcat Confluence 3.x is much better than 2.x

We had problems with Crowd 1.6 client and currently use Crowd 1.3 client Tune the JVM heap –

Confluence

: we use java –server –Xms:8G –Xmx:8G PermSize=512m MaxPermSize=512m +UseParallelGC +AggressiveHeap LargePageSizeInBytes=4M -DisableExplicitGC –

Jira

: we use –server –Xms4G –Xmx4G PermSize=1G MaxPermSize=1G +UseParallelOldGC +AggressiveOpts –

Crowd

: we use java –server –Xms:4G –Xmx:4G PermSize=512m MaxPermSize=512m –

Fisheye

Questions

[email protected]

British Telecom Inlife Support Lead Development Services Organisation