Operational Resilience Audit

What is Self-assessment?

Self-Assessment in Operational Resilience ensures that the regulated organisation captures and documents the steps taken towards operational resilience and provides a comprehensive and objective evaluation of the organisation's strategy and overall ability to respond to disruptions.

• Are operational resilience policies and procedures well-documented and readily accessible?
• Are the policies and procedures aligned with industry best practices and regulatory requirements?
• Do the documented policies clearly define roles, responsibilities, and accountability for operational resilience?
• Is there evidence of regular reviews and updates to the operational resilience documentation?

• Review the documentation of operational resilience policies and procedures.
• Assess the alignment of policies with industry best practices and regulations.
• Evaluate the clarity and completeness of roles, responsibilities, and accountability definitions.
• Verify the existence of a process for regular reviews and updates to the documentation.

• Has a comprehensive risk assessment been conducted to identify and assess potential risks?
• Are risks prioritized based on their potential impact and likelihood?
• Are mitigation strategies and controls in place to address identified risks?
• Is there a process for regularly monitoring and updating risk assessments?

• Evaluate the documentation of the risk assessment process.
• Assess the comprehensiveness of the risk assessment, including identification and assessment of risks.
• Verify the prioritization of risks based on impact and likelihood.
• Review the documented mitigation strategies and controls.
• Determine if there is a process for regularly monitoring and updating risk assessments

• Has a thorough business impact analysis (BIA) been conducted to identify critical processes and systems?
• Have the potential impacts of disruptions to critical processes and systems been assessed?
• Are recovery time objectives (RTOs) and recovery point objectives (RPOs) defined for critical processes?
• Are mitigation strategies and plans in place to ensure the timely recovery of critical processes?

• Review the business impact analysis (BIA) process documentation.
• Evaluate the completeness and accuracy of the identification of critical processes and systems.
• Assess the thoroughness of the assessment of potential impacts.
• Verify the definition of recovery time objectives (RTOs) and recovery point objectives (RPOs) for critical processes.
• Review the mitigation strategies and plans to ensure timely recovery.

• Is there a training program in place to educate employees on operational resilience?
• Are employees aware of their roles and responsibilities regarding operational resilience?
• Are there mechanisms to track and monitor employee completion of operational resilience training?
• Are there regular communication and awareness campaigns to promote a culture of operational resilience?

• Review the training program documentation for operational resilience.
• Evaluate the effectiveness of the training in educating employees.
• Assess the mechanisms in place to track and monitor employee completion of training.
• Verify the existence of regular communication and awareness campaigns.
• Determine the extent of the culture of operational resilience within the organization.

• Have operational resilience plans and procedures been tested through exercises and simulations?
• Is there a documented schedule for testing and exercising operational resilience capabilities?
• Are different scenarios and levels of disruptions considered during testing?
• Are testing results analyzed to identify areas for improvement and corrective actions?
• Are there mechanisms to track and follow up on implementing corrective actions identified during testing?

• Review the operational resilience testing and exercise plan documentation. 
• Evaluate the adequacy of the testing schedule and the consideration of various scenarios.
• Assess the testing results analysis to identify improvement areas.
• Determine if lessons learned from testing and exercises are documented and incorporated into improvements.

• Is there an incident response plan for operational resilience incidents?
• Has the incident response plan been tested and validated?
• Are roles, responsibilities, and communication channels clearly defined within the incident response plan?
• Is there a designated incident response team and a straightforward escalation process?
• Is there a process for post-incident analysis and continuous improvement of the incident response capabilities?

• Review the incident response plan documentation for operational resilience incidents.
• Evaluate the testing and validation activities conducted on the incident response plan.
• Assess the clarity and accuracy of roles, responsibilities, and communication channels.
• Verify the incident response team's existence and composition and escalation process.
• Determine if there is a process for post-incident analysis and continuous improvement.

• Is there a process in place to monitor and review the effectiveness of the operational resilience program?
• Are lessons learned from incidents, tests, and exercises incorporated into improvements?
• Is there a mechanism to capture and address feedback and suggestions for operational resilience?
• Are there metrics and performance indicators to measure the effectiveness of the operational resilience program?
• Is there a culture of continuous improvement and learning within the organization?

• Evaluate the process for monitoring and reviewing the effectiveness of the operational resilience program.
• Assess the incorporation of lessons learned from incidents, tests, and exercises into improvements.
• Verify the existence of a mechanism to capture and address feedback and suggestions.
• Review the metrics and performance indicators for measuring program effectiveness.
• Determine the extent of the organization's continuous improvement and learning culture.

Questionnaires and Checklist "Sustain" Phase

Implement Training and Awareness

Provide Self-assessment

Conduct Independent Quality Review

What is a Communication Strategy?

A communication strategy is a guide or a plan that helps an organization share information and achieve its communication and business objectives.  In this case, the operational resilience initiative.

Ensuring effective and timely communication of the internal and external communication plans is essential to help the organization keep customers and other stakeholders informed.

• Have key stakeholders for operational resilience been identified?
• Is there a clear understanding of each stakeholder group's communication needs and expectations?
• Have internal and external stakeholders, including employees, customers, suppliers, and regulators, been considered?
• Is there a process to regularly review and update the stakeholder list and their communication requirements?

• Review the documentation of stakeholder identification for operational resilience.
• Assess the clarity and completeness of understanding each stakeholder group's communication needs and expectations.
• Verify that both internal and external stakeholders have been considered.
• Determine the existence of a process for regular review and update of the stakeholder list and their communication requirements.

• Have communication objectives for operational resilience been defined?
• Are there clear and concise key messages that must be communicated to stakeholders?
• Do the key messages align with the operational resilience goals and priorities?
• Is there a process to regularly review and update the communication objectives and key messages?

• Evaluate the documentation of communication objectives for operational resilience.

• Assess the clarity and alignment of key messages with operational resilience goals and priorities.
• Verify the existence of a process for regular review and update of the communication objectives and key messages.

• Are there appropriate communication channels to reach each stakeholder group effectively?
• Have the advantages and limitations of different communication channels been considered?
• Is there a mix of channels, including both traditional and digital, to ensure comprehensive communication?
• Are there tools and platforms in place to facilitate efficient and secure communication?

• Is there a documented communication plan for operational resilience?
• Does the communication plan include a timeline, responsibilities, and deliverables?
• Are there mechanisms to ensure timely and consistent communication?
• Are there processes in place to handle urgent and sensitive communications?

• Is there a process to measure and evaluate the effectiveness of communication activities?
• Are there metrics and performance indicators to assess the impact of communication efforts?
• Is feedback from stakeholders collected and analyzed to identify areas for improvement?
• Are there mechanisms to monitor and address misconceptions or misinformation?

Questionnaires and Checklist "Sustain" Phase

Implement Training and Awareness

Provide Self-assessment

Conduct Independent Quality Review

What is Scenario Testing?

Scenario Testing aims to test the organisation's ability to remain within impact tolerances in severe but plausible disruption scenarios, focusing on recovery and response arrangements rather than preventative measures.

1. Scenario Testing Planning

• Has a scenario testing plan been developed outlining the objectives, scope, and methodology?
• Are the scenarios relevant to the organization's critical business services and potential threats?
• Has the testing plan considered various disruption scenarios, including natural disasters, cyberattacks, and system failures?

• Verify the existence of a scenario testing plan that outlines objectives, scope, and methodology.
• Assess the relevance of the scenarios to the organization's critical business services and potential threats.
• Ensure the testing plan considers various disruption scenarios, including natural disasters, cyberattacks, and system failures.

2. Scenario Development

• Has a range of realistic scenarios been identified for testing operational resilience?
• Do the selected scenarios cover a variety of potential disruptions and stress events?
• Have scenarios been designed to test different aspects of operational resilience, including people, processes, technology, and facilities?
• Are the selected scenarios aligned with the organisation's risk profile and potential impact on critical business services?
• How were the scenarios developed? Were they based on historical incidents, industry best practices, or internal risk assessments?
• Are the scenarios realistic and representative of the organisation's potential threats and disruptions?
• Have relevant stakeholders, including management and subject matter experts, reviewed and approved the scenarios?

• Review the scenario development process and ensure it is based on historical incidents, industry best practices, or internal risk assessments.
• Evaluate the realism and representativeness of the scenarios concerning potential threats and disruptions.
• Confirm that relevant stakeholders, including management and subject matter experts, have reviewed and approved the scenarios.

What is Lesson Learnt?

The key to improving "Lesson Learnt" when implementing Operational Resilience or OR is for an organisation to promote a continuous learning and improvement culture.   It is essential to improve and communicate remediation and vulnerabilities after scenario testing.

Leadership Commitment

• Is there a visible leadership commitment to promoting a culture of continuous learning and improvement?
• Do leaders actively support and participate in scenario testing and incident review processes?
• Are leaders accountable for implementing recommendations and lessons learned from scenario testing and incidents?
• Is there a communication strategy emphasising the importance of continuous learning and improvement for all employees?

Learning Framework

• Is there a documented framework or process for capturing and analysing lessons learned from scenario testing and incidents?
• Does the framework include mechanisms for identifying and documenting root causes and contributing factors?
• Are there standardised templates or tools for collecting and organising lessons learned information?
• Is there a designated team or individual responsible for managing the lessons-learned process?

Incident Review and Analysis

• Is there a structured process for reviewing and analysing actual incidents?
• Are incidents thoroughly investigated to identify root causes and contributing factors?
• Are incident review findings documented and shared with relevant stakeholders?
• Is there a mechanism to track and monitor the implementation of corrective actions resulting from incident reviews?

Scenario Testing Evaluation

• Is there a process for evaluating the effectiveness and impact of scenario testing exercises?
• Are scenario testing results analyzed to identify areas for improvement and enhancement?
• Are there mechanisms to capture feedback from participants and stakeholders on the scenario testing process?
• Is there a feedback loop to incorporate insights from scenario testing into future exercises?

Knowledge Sharing and Communication

• Is there a platform or mechanism for sharing lessons learned and best practices across the organisation?
• Are lessons learned and best practices communicated to relevant teams and departments?
• Are there regular communication channels, such as newsletters or internal portals, to disseminate information on operational resilience and continuous learning?
• Is there a process for capturing and sharing success stories and examples of continuous learning and improvement?

Training and Development

• Is there a training program in place to enhance employees' knowledge and skills related to operational resilience?
• Are employees trained on incident response, scenario testing, and lessons learned?
• Are there opportunities for employees to participate in specialised training or workshops related to operational resilience?
• Is there a process to evaluate the effectiveness of training programs and incorporate feedback for improvement?

Metrics and Performance Monitoring

• Are there defined metrics and indicators to measure the effectiveness of the continuous learning and improvement initiatives?
• Is there a process to track and monitor the organization's performance in implementing lessons learned and recommendations?
• Are performance metrics used to identify areas of success and areas that require further attention?
• Is there a mechanism for reporting and communicating performance metrics related to operational resilience readiness?

Continuous Improvement Culture

• Is there a culture of continuous improvement embedded in the organisation's values and behaviours?
• Are employees encouraged and empowered to share insights, ideas, and suggestions for improving operational resilience?
• Are there mechanisms to capture and evaluate employee suggestions, such as suggestion boxes or innovation platforms?
• Are there recognition and reward mechanisms for individuals or teams that contribute to continuous learning and improvement?

External Benchmarking

• Does the organisation seek opportunities for external benchmarking and learning from other organisations?
• Are there partnerships or networks established to share experiences and best practices in operational resilience?
• Is there a process to review and incorporate relevant industry standards and guidelines into the organisation's practices?
• Are there mechanisms to learn from regulatory changes, industry trends, and emerging risks?

Governance and Oversight

• Is there a designated governance body or committee responsible for overseeing and promoting continuous learning and improvement?
• Are there regular reporting and updates provided to senior management or the board of directors on the organisation's operational resilience readiness and continuous improvement efforts?
• Are clear accountability and responsibilities assigned for implementing and monitoring continuous learning initiatives?
• Is there a process to review and assess the effectiveness of the organisation's continuous learning and improvement initiatives?

Questionnaires and Checklist "Implement" Phase

Set Impact Tolerance

Conduct Scenario Testing

Improve Lesson Learnt

What is Gap Analysis in OR?

A gap analysis is a method of assessing the performance of a business unit to determine whether operational resilience requirements or objectives are being met and, if not, what steps should be taken to meet them.

A gap analysis is called a needs analysis, needs assessment or need-gap analysis.

• Has a structured process been defined for conducting the gap analysis?
• Are the objectives and scope of the gap analysis clearly defined?
• Is there a designated team responsible for conducting the gap analysis?
• Are the necessary resources allocated for conducting a thorough analysis?
• Has a timeline or schedule been established for completing the gap analysis?
  
  

• Has the current state of the operational resilience program been accurately assessed?
• Are the program's key components, processes, and controls identified and documented?
• Has the maturity level of each component been evaluated?
• Are there any gaps or deficiencies identified in the current state?
• Have relevant stakeholders been involved in the identification process?
  
  

• Verify the accuracy and comprehensiveness of the assessment of the current state of the operational resilience program.
• Evaluate the documentation of key components, processes, and controls.
• Assess the methodology used for evaluating the maturity level of each component.
• Identify and document any identified gaps or deficiencies in the current state.
• Confirm the involvement of relevant stakeholders in the identification process.
  
  

• Has a risk assessment been conducted to identify the risks of closing the gap?
• Are the identified risks prioritized based on their potential impact and likelihood?
• Has a mitigation strategy been developed for each identified risk?
• Are the resources and efforts allocated appropriately based on risk prioritization?
• Have appropriate stakeholders reviewed and approved the risk assessment and prioritization?
  
  

• Is there a well-defined governance framework and structure for operational resilience?
• Are individuals or teams responsible for operational resilience assigned clear roles, responsibilities, and accountabilities?
• Is there a mechanism to ensure oversight and monitoring of operational resilience activities?
• Are there regular reporting and escalation processes to senior management or the board of directors?
• Are there mechanisms to review and update the governance framework and structure as needed?
  
  

• Review the documented governance framework and structure for operational resilience.
• Evaluate the clarity and effectiveness of assigned roles, responsibilities, and accountabilities.
• Assess the mechanisms in place for oversight and monitoring of operational resilience activities.
• Verify the existence of regular reporting and escalation processes to senior management or the board.
• Determine if there are mechanisms to review and update the governance framework and structure.
  
  

• Are there documented plans and procedures for testing the effectiveness of business continuity and resilience measures?
• Is there a schedule for conducting regular testing and exercises?
• Are different scenarios and levels of disruptions considered during testing?
• Are testing results analyzed and used to identify areas for improvement and corrective actions?
• Are there mechanisms to track and follow up on implementing corrective actions identified during testing?

• Is there a process to identify and address gaps and deficiencies in the operational resilience program?
• Are there mechanisms to capture and document lessons learned from incidents, tests, and exercises?
• Is there a feedback loop to ensure that identified improvements are implemented?
• Are there metrics and performance indicators to measure the effectiveness of the operational resilience program?
• Is there a culture of continuous improvement and learning within the organization?

Questionnaires and Checklist "Plan" Phase

Develop Strategy Roadmap

Confirm Risk Appetite

Develop and Embed Governance