Transcript Operational procedures for Controls-related problems

Operational procedures
for Controls-related
problems
Kris. Kostro BE/CO
Content
► Creation
and follow-up of operational issues
► How to find support for a controls-related
issue (Front-End, Application,
Infrastructure)
► Discussion
Reporting and follow-up of
operational issues
►
All operational issues shall be followed up in the
Operational Issues Jira project (APS)
 Only registered issues have the guarantee for a transparent
follow-up and wide visibility.
 Most projects have their own Jira projects, they are for
internal development issues, not for operational issues.
Reported OP issues which require additional development are
closed as OP issue and moved to internal projects.
 All issues shall be reported to JIRA, including these where CO
expert or piquet was called => Logbook entry concerning CO
shall have a corresponding JIRA issue
► Send
OP issue from logbook event entry
 This allows BE/CO to spot areas of concern
Operational Jira workflow
1.
2.
3.
4.
5.
Operational issue was created (from logbook, mail to
[email protected] or Jira API (type issues in the browser)
Issue is first assigned, typically to a project mailing list i.e.
[email protected]. This is typically done within 10 min
during the working hours. This way everybody in the project is
aware of the issue as opposed to a personal call or mail. Also
temporary absences and holidays are covered.
Typically one person in the team is responsible for assigning
the issue to a person within the team which will become
responsible for the resolution.
As the issue is commented, reassigned, etc the person which
created the issue and all people involved in the lifetime of the
issue are automatically informed by mail.
Finally issue is resolved, refused or, if it requires additional
development it is moved to the internal project Jira.
Operational issues in 2010
►
►
►
The Exploitation Manager and the MAM meeting are
regularly reviewing pending issues:
http://wikis/display/ABCO/Exploitation contain links to
issue lists (last week, previous week, older issue)
316 issues in 2010 (360 in 2009)
Jira statistics give some insight into the operational
performance of components
 Only if everybody is playing the game
 Operational issues statistics at
http://issues.cern.ch/secure/Dashboard.jspa?selectPageId=10551
►
Please send ALL controls-related logbook entries to JIRA
APS issues by component (2010)
APS issues by accelerator (2010)
APS issues by resolution (2010)
How to find a support team?
Equipment Server or Front-End
► Almost
all LHC and SPS Front-Ends are supported by
the equipment groups
► In the PS complex most F-E are supported by the
controls piquet but the picture will be moving as the
renovation progress
 Ultimately only CO proper F-E (Timing, OASIS) will be
supported by CO
► Information
on who supports a Front-End will be
available in DIAMON soon
 In PS complex the CO piquet shall not be called for
issues related to the renovated Front-Ends. The
Equipment responsible shall be called first (F-E down,
configuration and server issues)
How to find a support team?
Specific application
► All
controls applications shall be listed in the
configuration database https://espace.cern.ch/bedep/CO/DM/default.aspx
 2 ways to find responsible
►Application
Programs
►Operational Configuration (CCM configuration)
► Zory’s
talk Thursday 14h is dedicated to
controls configuration database
Application program search
Responsible Search (console
menu)
Console menu search
Details of application from menu tree
How to find a support team?
Infrastructure
►
►
This is sometimes obvious (i.e. OASIS) but not always
The whole controls infrastructure is partitioned into
Controls Components
http://wikis/display/ABCO/Support+of+Controls+Components
►
Each component
 Has an entry in the webpiquet CO specialist list http://aboppiquets.web.cern.ch/ABOP-piquets/
 Has a support team and a mailing list
 Has a JIRA component
►
For the PS complex infrastructure issues it is preferable
to call the expert rather than CO piquet first
CO Specialists in webpiquet
Discussion?