[OR] [MAS] [E3] [C1] Importance of Testing and Exercising

eBook 3: Chapter 1

Importance of Testing and Exercising

Introduction

Testing and exercising form the backbone of any effective operational resilience programme.

While organisations may design robust frameworks, identify Critical Business Services (CBS), and establish impact tolerances, these elements remain theoretical until they are validated through structured testing.

In the context of Singapore’s financial sector, the Monetary Authority of Singapore (MAS) places strong emphasis on testing and exercising as a regulatory expectation to ensure that resilience capabilities are not only documented but demonstrably effective.

Aligned with MAS guidance in “Achieving Operational Resilience for Financial Institutions in Singapore,” testing is not a one-off activity but a continuous process embedded in the resilience lifecycle.

It ensures that institutions can respond, adapt, and recover from disruptions while maintaining the delivery of critical services.

Complementing this, BCM Institute’s perspective on scenario testing reinforces that organisations must move beyond static plans and adopt dynamic, scenario-driven validation approaches.

This chapter explores the importance of testing and exercising, highlights MAS expectations, and outlines the primary types of testing used in operational resilience.

MAS Emphasis on Testing and Validation

MAS requires financial institutions to demonstrate that their operational resilience frameworks are effective under severe but plausible scenarios.

This shifts the focus from compliance-driven documentation to evidence-based assurance.

Key regulatory expectations include:

• Validation of Impact Tolerances
  Financial institutions must prove that their CBS can remain within defined impact tolerances during disruptions.
  Testing provides measurable evidence of whether tolerances are realistic and achievable.
  
  
• End-to-End Scenario Testing
  MAS emphasises testing across the full service delivery chain, including dependencies on people, processes, technology, and third parties.
  This aligns with the service-centric approach of operational resilience.
  
  
• Inclusion of Severe but Plausible Scenarios
  Institutions are expected to design scenarios that reflect realistic but extreme conditions, such as cyberattacks, system outages, or third-party failures.
  
  
• Regular and Progressive Testing
  Testing should evolve in complexity and scope over time, moving from simple tabletop exercises to integrated, cross-functional simulations.
  
  
• Board and Senior Management Oversight
  Testing outcomes must be reported to senior management to ensure accountability and drive continuous improvement.

From a regulatory standpoint, testing is not merely an operational activity—it is a governance and assurance mechanism that validates the institution’s resilience posture.

Role of Scenario Testing in Operational Resilience

Scenario testing is central to operational resilience, as highlighted in BCM Institute’s guidance. It is the process of evaluating how an organisation responds to disruption scenarios that threaten the delivery of critical services.

Key Principles of Scenario Testing

• Service-Centric Focus
  Testing is anchored on the ability to deliver CBS, rather than isolated system or process performance.
• Integration Across Functions
  Scenario testing involves multiple business units and ensures coordination among operational teams, IT, risk management, and crisis management.
• Dynamic and Realistic Conditions
  Scenarios must simulate real-world complexities, including cascading failures and interdependencies.
• Learning-Oriented Approach
  The objective is not to “pass” the test but to identify weaknesses, gaps, and opportunities for improvement.

Outcomes of Effective Scenario Testing

• Identification of hidden vulnerabilities in processes and dependencies
• Validation (or recalibration) of impact tolerances
• Strengthening of incident and crisis response capabilities
• Enhanced cross-functional coordination and decision-making

Scenario testing transforms resilience from a theoretical construct into a practical, measurable capability.

Types of Testing and Exercising

Operational resilience programmes typically employ a spectrum of testing approaches, each serving a distinct purpose. These range from discussion-based exercises to fully operational simulations.

Tabletop Exercises

Tabletop exercises are discussion-based sessions where participants walk through a disruption scenario in a controlled environment.

Characteristics:

• Facilitated discussions involving key stakeholders
• Focus on decision-making, roles, and communication
• No actual system or operational disruption

Purpose:

• Validate understanding of plans and procedures
• Test escalation protocols and governance structures
• Build awareness among senior management

Strengths:

• Low cost and easy to organise
• Effective for identifying gaps in policies and coordination

Limitations:

• Does not test real operational capabilities
• Limited ability to simulate technical failures

Scenario Simulations

Scenario simulations introduce greater realism by testing how systems and teams respond to simulated disruptions.

Characteristics:

• May involve partial system testing or controlled disruptions
• Includes cross-functional participation
• Often aligned with severe but plausible scenarios

Purpose:

• Validate interdependencies across people, process, and technology
• Assess response times and operational effectiveness
• Test coordination between business units and external stakeholders

Strengths:

• Provides more realistic insights than tabletop exercises
• Identifies operational and technical gaps

Limitations:

• Requires more planning and resources
• May still not fully replicate real-world complexity

Full-Scale Recovery Testing

Full-scale recovery testing represents the highest level of testing maturity, involving actual execution of recovery strategies.

Characteristics:

• Activation of disaster recovery sites or alternate processes
• Real-time system failover and recovery
• End-to-end validation of CBS delivery

Purpose:

• Demonstrate that recovery strategies meet defined RTOs and RPOs
• Validate the organisation’s ability to operate under disruption
• Provide assurance to regulators and stakeholders

Strengths:

• Offers the highest level of confidence in resilience capabilities
• Produces tangible, evidence-based outcomes

Limitations:

• Resource-intensive and complex to execute
• Potential operational risks if not carefully controlled

Integrating Testing into the Resilience Lifecycle

Testing and exercising should not be treated as standalone activities. Instead, they must be embedded within a continuous improvement cycle:

• Plan – Define scenarios, objectives, and success criteria

• Execute – Conduct exercises across varying levels of complexity

• Evaluate – Assess performance against impact tolerances

• Improve – Address gaps and enhance resilience capabilities

This iterative approach ensures that operational resilience evolves alongside emerging risks, technological changes, and regulatory expectations.

Testing and exercising are critical to transforming operational resilience from theory into practice.

Under the Monetary Authority of Singapore's expectations, financial institutions must demonstrate that their critical business services can withstand and recover from severe disruptions through structured, continuous testing.

By leveraging a combination of tabletop exercises, scenario simulations, and full-scale recovery testing—anchored by robust scenario testing methodologies—organisations can validate their resilience capabilities, uncover hidden vulnerabilities, and drive continuous improvement.

Ultimately, a well-designed testing programme provides not just compliance assurance, but the confidence that the organisation can deliver its critical services when it matters most.

Gain Competency: For organisations looking to accelerate their journey, BCM Institute’s training and certification programs, including the OR-5000 Operational Resilience Expert Implementer course, provide in-depth insights and practical toolkits for effectively embedding this model.