This chapter shows how to design, run and document scenario tests in five steps. After reading it you should be able to choose scenarios from past incidents and near misses, select a test type, evaluate the response and give the board the information it needs to decide on investment. The scenarios are illustrative.
Business continuity testing often targets short outages of a site or system. Resilience testing takes an end-to-end service view and asks where intolerable harm would occur, especially harm to clients. It covers gradual change as well as sudden shock, and it includes third parties.
Draw on past incidents, near misses and events in other sectors and jurisdictions. Reuse the scenario libraries of operational risk, stress testing and business continuity, and tailor them to each critical service.
Illustrative scenarios for BJB Singapore: a ransomware attack on a payment messaging provider; loss of a primary data centre; unavailability of a large share of operations staff; a sustained failure of a market data or custody provider.
Evaluate effects on services, systems, processes and stakeholders across financial, operational, reputational and client dimensions.
Choose the format: tabletop exercise, simulation or live test. Start with individual components, then combine them into integrated tests.
Illustrative example: in year one BJB runs a tabletop for CBS-2 Client Payments and Cash Transfers. In year two it runs an integrated test that involves the payment provider and the hosting partner [subject to contract terms].
Review incident management, communication and recovery. Measure results against the tolerance, not only against internal targets.
Illustrative finding: the team switches to a manual payment process in five hours, but the process cannot handle more than a fraction of normal volumes, so the same-day tolerance would be breached for a number of clients.
Record scenario design and assumptions, results, risks to tolerance and recommendations. These records support the annual self-assessment and supervisory discussions.
The board should be told about scenarios that could breach tolerance and should decide on prioritised investment. Keep a test plan covering all critical services over a multi-year cycle. Record who observed, who signed off and which actions came out of each test.
A good scenario is specific, uncomfortable and believable. It states what fails, when, for how long, and what else is going wrong at the same time. For BJB Singapore, a scenario for CBS-2 might read: at 10:00 on the last business day of the month, the bank is informed that its payment messaging provider has suffered a ransomware attack. The provider expects to take at least 48 hours to restore service. Email is unavailable for the operations team, and two senior approvers are travelling. The scenario also states the injects, such as a call from a large client with an urgent payment, and a query from the regulator.
Plan the tests over several years so that every critical service is covered and the severity increases. A typical sequence starts with a walkthrough of the mapping, moves to a tabletop exercise with the crisis team, then to a simulation of a component, and finally to an integrated test with the relevant third parties. Vary the scenario type across the cycle: cyber attack, technology failure, loss of premises, loss of staff, third-party failure. Rotate the participants so that deputies as well as principals gain experience.
A short report for each test should say what was tested, whether the bank stayed within tolerance, where it did not, and what has been decided. Where a test shows that a tolerance cannot be met, the report should ask for a decision on investment or on a change in risk acceptance. The board should see the pattern across tests, not only individual results.
| Step | Activity | Output |
|---|---|---|
| 1 | Define scenarios | Scenario library per service |
| 2 | Assess impact | Impact view |
| 3 | Conduct testing | Test records |
| 4 | Evaluate responses | Results against tolerance |
| 5 | Document lessons learned | Findings and recommendations |
Testing is where plans meet reality. A test that exposes a weakness is a success, because the weakness can be fixed before it matters to a client. The next chapter explains how BJB turns findings into improvements.
Continue reading: Chapter 13, Improving Lessons Learned of OR Implement Phase for Bank Julius Baer, converts findings into action.
| eBook 2: Implementing Operational Resilience for Bank Julius Baer |
|||
| C1 | C2 | C8 | C14 |
| "Implement" Phase of the Operational Resilience Planning Methodology | ||||||
| C8 | C9 | C10 | C11 | C12 | C13 | |