Transcript Helix - ISACA Mumbai Chapter

Business Resumption Planning
with Case Studies
By Priti Sikdar (F.C.A., C.I.S.A., C.I.S.M.,
I.S.O.27001 L.A., BS 25999 L.A.)
Email:[email protected]
Business Resumption is…
The process of recovery of all systems and related processes after a
disaster to return to Business-as-Usual.
It involves re-opening each of the institution's components -- and testing
and revising the process based upon the results.
No matter which sector an organization occupies, failing to prepare to
manage disruptive events and allowing them to progress from event to
catastrophe can have major impacts – even putting the organization’s
survival at risk.
28-Jun-08
Business Resumption Planning  Page 2
Business Continuity Cycle
Normal
Business
Business
Interruption
Event
Implement
Improvements and
Updates
Depending on
the nature of the
interruption, one
or both of these
activities may
occur
Emergency
Response
Lessons
Learned
CriticalBomb
– Services
that must be provided
Threat
immediately or will definitely result in loss of life,
Medical
Emergencies
infrastructure destruction, loss of confidence
in the
government, or significant loss of revenue. These
Cyber
Gas Leak
services normallyRecovery
require
resumption within 24 hours
of interruption.
Desired Services
Civil Disorder
Vital – Power
ServicesOutage
that must be provided within 72
Restoration
hours or will likely result in loss of life, infrastructure
Necessaryor
Services
destruction, loss of confidence
in the government,
Severe
Weather
significant loss of revenue or disproportionate
recovery costs.
28-Jun-08
Terrorist Incident
Business
Resumption
Crime
Critical & Vital
Services
Chemical
Spill
Flood
Business Resumption Planning  Page 3
Failure Mode Effects Analysis (FMEA)
The CIO of a $25 billion organization, who was also an ex-Air Force general and
telecommunication officer, once commented
“ When we send up a military satellite, everything has to be perfect. This is because
no one has yet invented a 23,000 mile long screwdriver to fix it if it is not [perfect].”
28-Jun-08
Business Resumption Planning  Page 4
Failure Mode Effects Analysis (FMEA)
Understanding the mission of the technical systems that support your
organization and their vulnerabilities is critical to developing a sound Failure
Mode Effects Analysis, the second step in creating a Business Resumption
Plan
There are 3 components of FMEA1.) The first step is Problem Identification. When designing or evaluating
a network both in hardware and software, the rule-of-thumb is to ask, “What
can possibly go wrong? ” This includes both failure rates of the equipment
itself as well as the external factors.
Ex. heat, water, air, people, etc.
28-Jun-08
Business Resumption Planning  Page 5
Components of FMEA-Assign Risk Priority Number
2.) The second component is the assignment of a Risk Priority Number
(RPN) to each of the probabilities.
Pick a number from one to 10. The higher the number, the greater the
associated risk. In the case of problem resolution, the higher number
indicates a longer time to detect and correct the problem.
Identification and prioritization of risks after ranking them is vital for the
success of a risk mitigation program.
28-Jun-08
Business Resumption Planning  Page 6
Components of FMEA-Standards Refinement Component
3) The final component is the Standards Refinement step or the “What
are we going to do about it?” step.
Ongoing maintenance and repair procedures should be designed to prevent
disasters before they happen in the first place. Change control and
management control over a system are even more important.
When properly orchestrated, however, standards pay off handsomely in
reduced operating costs as well as in greater peace of mind which comes
with a more robust operating environment.
28-Jun-08
Business Resumption Planning  Page 7
Developing a Business Continuity Culture
 Developing a business continuity culture in any organization is a significant
undertaking, particularly if the organization has traditionally seen business
continuity as an information technology risk and not an organization-wide issue.
Information technology is only one of many dependencies the organization has in
the delivery of its products and services.
 Many organizations fail to develop a business continuity culture because there is a
perception that the process is too costly, time-consuming and/or requires a large
amount of resources
 Management needs continually to be reminded that the aim of business continuity
is to keep the organisation in business in the event of a disaster by maintaining its
critical core processes in the delivery of products and services to its internal and
external customers.
The key to developing a business continuity culture is the ongoing commitment of
executive management. This is supported by:
 A business continuity policy
 Resourcing
 Business continuity investment
 An education program
28-Jun-08
Business Resumption Planning  Page 8
Prepare Business Resumption Plan according to the type and
impact of the disaster.
28-Jun-08
Business Resumption Planning  Page 9
Planning For Business Resumption
28-Jun-08
Business Resumption Planning  Page 10
Business Resumption Planning Life Cycle
Prevention includes:
Threat Analysis
Physical Security & Protection Program
Data Security & Protection Program
The first steps in a the BRP program, is to determine the possible extent of
exposure to a disaster and then to minimize the probability of a disaster
occurring. Purpose is to get the substantial funding normally required. This
requires selling the Board of Directors on the reality of a possible disaster and
the impact on the ability of the organization to survive.
TESTING
PREVENTION
28-Jun-08
PLANNING
Business Resumption Planning  Page 11
Planning
Planning is important for success of the resumption program.
It includesCritical Function/Application Analysis
Design of Normal & Emergency
Processing Architectures for computer & telecommunications, manual
processing and record storage
Obtain Backup Resources for:
- off-site storage
-
computer processing
manual processing
data and voice communications
management control
Arrange Disaster Response Team Staffing for:
- damage assessment and recovery planning
- emergency operations
- disaster response management
28-Jun-08
Business Resumption Planning  Page 12
Steps involved in building an effective Business Resumption Plan
1) Establish a Business
Resumption Planning
Committee
 Project Leader
 Project Plan/Control
 Committee Selection
 Assign Responsibilities
 Regular Committee Meetings
 Periodic Management Briefings
28-Jun-08
Business Resumption Planning  Page 13
Steps involved in building an effective Business Resumption
Plan
2) Perform a Business Resumption Capability Assessment
Assess how quickly and fully you need to resume if a disruption were to
occur today. What are your critical business needs?
Security Check List
Recovery Analysis
Task Assignments
3) Perform a Risk Analysis
Risk Assessment
Risk Management
Evaluate Threats
Establish Controls
Review Security Measures
28-Jun-08
Business Resumption Planning  Page 14
Study the business impact factors
High Impact / Terrorist Attack
Low Probability
Earthquake
10
9
Hurricane
8
Computer Failure
10
20
30
High Impact /
High Probability
40
Tornado
7
6
60
70
80
90
4
Workplace Violence
Staffing Issues
Low Impact /
Low Probability
28-Jun-08
3
2
100
Probability
Factor
Scale
Virus Attack
Snow Storm
1
Business Resumption Planning  Page 15
Low Impact /
High Probability
Steps involved in building an effective Business Resumption
Plan
4) Analyze and Define Requirements for Recovery
Hardware
Software - system and application
software
Communications
Back-up Data
Physical Facility
Vendor Support
Inter-Campus Support
28-Jun-08
 Office Equipment
 Personnel
 Security
 Forms/Paper Supplies
 Logistics
 Storage
 Funding/Purchase Orders
Business Resumption Planning  Page 16
Steps involved in building an effective Business Resumption
Plan
5)Design and Document the BRP for Recovery Operations
Damage Assessment Team
User Liaison Team (if needed)
Communications Team
Operations Team
Security/Back-up Team
System Software Team
Procurement Team
Facilities Team
Identify Processes Required
Develop Procedures (by team)
Risk Manager or initiate an Audit Review and Approval team.
28-Jun-08
Business Resumption Planning  Page 17
Steps involved in building an effective Business Resumption
Plan
6)Test the BRP
7). Maintain and Update the BRP
Frequency - at least annually
Develop a Test Plan/Script
Test Scenario
Evaluation and Reporting
Follow-up
Follow-up BRP Test
Report Test Results to Risk
Manager
Institute Controls/Changes environmental, procedural,
personnel, training, etc.
28-Jun-08
Business Resumption Planning  Page 18
Testing
28-Jun-08



Desk Top Walk Through
backup resource utilization
team responsibilities
emergency operations approaches


Operations Testing
computer processing based applications
manual processing based applications


Simulation Testing
emergency response teams
disaster management team
Business Resumption Planning  Page 19
Steps involved in building a Business Resumption Plan
8) Training for business resumption
Select Training Topics
- emergency procedures,
- use of fire extinguishers,
- backup retrieval, etc.
Select Instructors
Develop Training Material
Risk Management
Procedures
Select Personnel for Training
Train Personnel
28-Jun-08
Business Resumption Planning  Page 20
Goals Of The Disaster Recovery & Business Resumption
Plan
 Eliminate or reduce the potential for injuries or the loss of human
life, damage to facilities, and loss of assets and records:
This requires a comprehensive assessment of each department within
the institution, to insure that appropriate steps have been taken to-
Minimize disruptions of services to the institution and its customers;
Minimize financial loss;
Provide for a timely resumption of operations in case of a disaster; and
Reduce or limit exposure to potential liability claims filed against the institution,
and its directors, officers and other personnel.
 Immediately invoke the emergency provisions of Disaster Recovery
& Business Resumption Plan: For stabilizing the effects of the disaster,
allowing for appropriate assessment and the beginning of recovery
efforts. We then minimize the effects of the disaster and provide for the
fastest possible recovery.
 Implement the procedures contained in the Disaster Recovery &
Business Resumption Plan: Care to be taken to gauge the disaster
and measure the likely impact from the disaster.
28-Jun-08
Business Resumption Planning  Page 21
MTPoD Concept - MAXIMUM Tolerable Period of
Disruption
MTPoD Concept - MAXIMUM Tolerable Period of Disruption
MTPoD
Incident, e.g. Flood
Duration after which an
organization's visibility will
be irrevocably threatened if
product & service delivery
cannot be resumed
RTO
28-Jun-08
Business Resumption Planning  Page 22
Essential Services
N
IMPACT
CONTINUUM
M
Desired Services
Determination
of highest
factors
should be
adjusted
according to
departmental
expectations
and priorities
L
K
J
TIME CONTINUUM
I
Determine the
significance and
length of the time
continuum based
on their services
and time
expectations.
H
Necessary Services
G
F
E
D
Vital Services
C
B
Critical
Services
A
28-Jun-08
Less than
One day
2–3
4–7
7 – 14
days Resumptiondays
Business
Planning  Page days
23
more than
14 days
How Do Businesses Survive Disaster?
Businesses that survive disaster are those with a cohesive business
resumption plan. What are we planning for?
1) Crisis
Localized to a system or resource- "Half of U.S. corporations rate
their internet downtime costs at more than $1,000 per hour."
Communication failure and link failure leads to loss of data.
Minor interruption to business due to virus infestation, computer
crime and the like.
2) Disaster
Contained within an area due to economic sanctions, human
error,
Damage of property due to terrorism and sabotage
3) Catastrophe
Regional or larger
Infrastructure disrupted
28-Jun-08
Business Resumption Planning  Page 24
Characteristics of a good BRP
A good Business Resumption Plan
Identifies the pre-set arrangements you need to have on "stand-by" in
order to get vital functions operating again with as little delay as possible
Ensures the availability of necessary resources including personnel,
information, equipment, financial arrangements, services and
accommodations
Helps an operation to survive an unplanned interruption by making sure
essential clients needs can be met until normal operations are resumed.
28-Jun-08
Business Resumption Planning  Page 25
Two Major Factors while implementing BRP
 Business Factors:
1. Insurance of
- Equipment and Facility insurance
- Business interruption insurance
- Extra Expense
- Professional Liability
- Extra Equipment Coverage
- Data Reconstruction
- Specialized Equipment Coverage
- Valuable Papers and Records
2. Business Risk (dependency on Information
Technology)
 Driving Factor:
Legal/Regulatory Compliance ( SOX 404, MI
52-109)
28-Jun-08
Business Resumption Planning  Page 26
Components of Business Resumption Plan
People
Process
28-Jun-08
Technology
Business Resumption Planning  Page 27
Baseline Requirements
Before you can begin to design a Business Resumption Plan there are
some primary Disaster Recovery activities that must be implemented.
Without these procedures in place, no plan will ever be successful.
Management buy-in for disaster recovery and resumption should be
existing right from beginning.
Your mission critical data must be backed up, with a defined schedule,
and fully documented. This includes which server is backed up onto which
tape, where key data is located, type of backup device, and even backup
type (differential, incremental etc).
At least one set of backups must be in secured offsite storage. This set
should be rotated back onsite, with a more recent backup sent offsite.
Rotation should occur at a minimum of once per week. You should also
maintain a full month end backup and a set of current emergency repair
disks offsite.
28-Jun-08
Business Resumption Planning  Page 28
Recovery Time Objectives (RTO) & Recovery Point Objectives
(RPO)
RTO (recovery-time objective) indicates allowable downtime, or the
earliest point in time at which the business operations must resume after
disaster.
RPO (recovery-point objective) signifies the amount of data that is
acceptable to have been lost and subsequently recovered once the service
is restored.
28-Jun-08
Business Resumption Planning  Page 29
Determining Recovery Objectives
“Freshness”
mths
wks
I’m up and running in
seconds, but I’ve
lost a day’s data
Recovery
Point
Objective
I lost no data but it
took me a week to
get back up and
running
days
What are my
disaster recovery
needs?
hrs
mins
Recovery
Time
Objective
secs
Zero
28-Jun-08
secs
mins
hrs
days
wks
Business Resumption Planning  Page 30
mths
“Downtime”
Develop Recovery Time Objective
Once you have completed the identification and prioritization of the
business functions it is time to outline your planning objective, or basically
what gets fixed, how quickly and to what level of service. It may help to
structure this in the form of a table such as that shown below.
Essential Function
Resumption Objective
(priority)
Resumption
Alternative
Telephone Service
Email Connectivity
0 - Immediately
0 - Immediately
Firewall Protection
1 - First Day
Cellular Telephones
Free service –
temporary solution
Co-Location
28-Jun-08
Business Resumption Planning  Page 31
Set your priorities
When we implement these procedures, we must prioritize all recovery efforts as
follows:
- Employees: Not only must we help to ensure their survival as a basic human
concern, but because of their anticipated performance in helping other
persons on the institution's premises when the disaster strikes
- Customers: As we do with employees, we must help to ensure the survival
of or care for customers affected by the disaster: physically, mentally,
emotionally and financially;
- Facilities: After ensuring the safety of employees and customers, we then
secure each facility as shelter for both people and assets;
- Assets: Conducting a damage assessment will determine which assets have
been destroyed, which ones are at risk and what resources that we have left;
and
- Records: Documenting the disaster and the actions taken by the institution's
personnel -- when combined with comprehensive videotapes of facilities that
are obtained during routine facility inspections -- reduce the likelihood of legal
actions while helping to assess the responsibility for losses.
28-Jun-08
Business Resumption Planning  Page 32
Put thrust on training and updating of resumption plan
 A comprehensive training program for all personnel at all facilities,
conducted at specified intervals -- at least annually;
that may also include the:
-
Identification and operation of utility shut-off devices;
Location of emergency staging areas;
Basic first aid and survival techniques; and
Emergency responsibilities and re-assignment plans for all positions; and
Written copies of the final Disaster Recovery & Business Resumption Plan
distributed to branch and department leaders -- including a complete list of
appropriate emergency response agencies and facilities.
28-Jun-08
Business Resumption Planning  Page 33
Prioritizing resumption requirements
 Prioritization is the process of understanding what will be needed, when,
and how long you have to get things rolling again.
The one consistent activity is the establishment of basic telephone
communication and should always be first on your list.
List the major functions or activities of your business or organization. (in a
large organization, list the "time-critical" functions or activities of each unit,
division, department, branch etc.)
28-Jun-08
Business Resumption Planning  Page 34
Recovery of Documents
 Developed, maintained and implemented an effective storage and
recovery plan for the institution's original documents and vital records?
Recovering business operations after a disaster often requires the use of
original documents and vital records not stored as electronic data. The
contingency plan should include plans for the consolidation and storage of
appropriate original documents and vital records in a central fireproofed
location, including:
-
Contracts;
Insurance policies;
Corporate papers;
An inventory list of stored items, stored in two (2) locations; and
Annual review for applicability, currency and legality
28-Jun-08
Business Resumption Planning  Page 35
Case Study 1 - The Katrina Disaster
Hurricane Katrina left behind nearly a million displaced people and
destroyed paper medical records, underscoring the critical need for a digital
health system.
Hurricane Katrina pounded the Gulf Coast as a Category 4 storm at 7
a.m.Monday, August 29, 2005. Raging winds sustained at 140 mph and
nearly 13 inches of torrential rain inundated the city for 48 straight hours.
While the rest of the city went dark, redundant generator power kept St.
Tammany alive with light, ensuring that computer operations, internal
communication, and critical equipment including air conditioning and
elevators never faltered.
28-Jun-08
Business Resumption Planning  Page 36
Model instance of coping with a disaster
Overview:
Merrill Lynch's Director of Global Contingency Planning, was in the
company's world-wide headquarters in the World Financial Center, across
the street from the World Trade Center, when the 9/11 attacks occurred.
Within three to five minutes Merrill Lynch had its command center up and
running. In the hour following the attacks, obtaining accurate information
was a challenge.
With the condition of the surrounding buildings becoming increasingly
uncertain, they relied on media reports to keep them up to date.
Within a few hours, they were able to go from an employee evacuation
and accounting mode to a standard business recovery mode, prioritizing
resumption as dictated by the continuity plan.
Merrill Lynch mandated the use of LDRPS for all business units worldwide
after Y2K.
28-Jun-08
Business Resumption Planning  Page 37
Building the Foundation for BCP & DC
Best Results
Come From
Alignment &
Optimization
Business Strategy
To unravel the complexity
associated with Business Continuity,
while maintaining an operational
business, we advocate a
comprehensive structural approach
utilizing building blocks...
Organization
Resource
Management
Processes
Technology
Landscape
Architecture
Deployment
Planning
Process
Optimization
Local
Planning
Activity
Prioritization
…..Enabling your company to ensure organizational, business process and technological
readiness, while limiting overall business impact to its Information Technology, Business
Processes, the Supply Chain and its client base
28-Jun-08
Business Resumption Planning  Page 38
Agility Recovery
A cohesive business
resumption plan can prepare
your business for nearly any
contingency. An integral part
of any business resumption
plan is a fully-functional
mobile command center.
28-Jun-08
Business Resumption Planning  Page 39
CASE Study 1: Morphy Richards
 December bad weather left Morphy Richards' South Yorkshire
headquarters flooded, covering the Mexborough site with around two feet
of water, affecting its IT systems and rendering its premises unusable.
 Fortunately, Morphy Richards had a Business Continuity Plan in place
- Due to the extensive coverage of the flooding, suitable temporary premises were
found in Rotherham, where a backup IT system was installed to run Morphy
Richard’s business systems.
- Technicians, working with Morphy Richards IT staff were able to transfer all of
Morphy Richards’s information from its own computers to the backup system,
enabling business to continue.
 Almost three weeks later Morphy Richards was still based at the
alternative site and utilising the back-up IT facilities whilst its Mexborough
site was being prepared for the return.
28-Jun-08
Business Resumption Planning  Page 40
CASE Study 1: Morphy Richards
Trevor Burrows, Head of IT at Morphy Richards, comments:
“The response of our business continuity provider was quick and very
professional. The run up to Christmas is our busiest time of year, and BCP
team's immediate response meant we were able to keep essential
business systems up and running during this extremely demanding
period. After losing everything at our Mexborough site, with BCP team's
help all key business systems were available again within 24 hours.
Having a business continuity plan in place is not an option, it is essential.”
28-Jun-08
Business Resumption Planning  Page 41
CASE Study 2: Norwich Union
 Incident: discovery of an envelope containing a suspicious substance was
detected in the post room in Norwich.
 Once the substance was discovered a preliminary investigation was completed
and the police contacted.
 Resulted in 17 members of staff going through decontamination.
 Two business continuity plans were invoked to continue critical processes. The
substance was declared non-hazardous.
 This incident was positive proof that the time and effort which have been invested
in the Incident Management Process and the training and education of the Silver
Team has been worthwhile.
 Despite moving into uncharted waters with regard to the nature of the incident, the
fundamental process was robust, withstood the challenge and was well-executed.
The priority was clearly understood and full co-operation was received from all
operational areas.
28-Jun-08
Business Resumption Planning  Page 42
CASE Study 2: Norwich Union
 Lessons / Messages
 The 17 staff were safe with due care and attention given to their welfare. This
meant that they all returned to work the following day.
 No business impact was experienced, due to the successful implementation of the
Business Continuity Plans.
 Prior exercising ensured that all members of the Silver team worked well together.
 There was effective liaison with the Police, Fire and Ambulance services
throughout the incident, which ensured that information they requested was
available in a timely manner.
 Various communication methods were used throughout the incident to staff,
directors and the media. Various methods were employed: one-to-one briefings;
press releases; intranet messages; SMS texts; runners; telephone incident lines
and a notice board at rendezvous point for evacuated staff.
28-Jun-08
Business Resumption Planning  Page 43
Thank You for your time…
28-Jun-08
Business Resumption Planning  Page 44