Operational Resilience Audit

What is an Independent Quality Review?

A significant part of independent quality review revolves around audit and assurance.  It significantly contributes to achieving organisational objectives and value creation for shareholders and stakeholders, especially when implementing operational resilience.

• Are operational resilience policies and procedures well-documented and up to date?
• Is there evidence of a comprehensive operational resilience framework?
• Are the policies and procedures aligned with industry best practices and regulatory requirements?
• Are there clear guidelines and standards for operational resilience practices?
• Is there evidence of senior management endorsement and approval of operational resilience policies?

• Review operational resilience policies and procedures documentation.
• Assess the comprehensiveness and currency of the operational resilience framework.
• Evaluate the alignment of policies and procedures with industry best practices and regulations.
• Verify the presence of clear guidelines and standards for operational resilience practices.
• Determine if senior management has endorsed and approved the operational resilience policies.

• Has training on operational resilience been provided to employees at all levels?
• Is there evidence of awareness campaigns and communication initiatives related to operational resilience?
• Are training materials comprehensive and effectively communicated to employees?
• Is there a mechanism to track and monitor employee completion of operational resilience training?
• Are training programs periodically updated to reflect changes in operational resilience requirements?

• Verify the provision of operational resilience training to employees at all levels.
• Assess the effectiveness of awareness campaigns and communication initiatives.
• Evaluate the comprehensiveness and clarity of training materials.
• Determine if there is a mechanism to track and monitor employee completion of training.
• Review the process for updating training programs based on changes in requirements.

• Have operational resilience plans and procedures been tested through exercises and simulations?
• Is there a documented schedule for testing and exercising operational resilience capabilities?
• Are different scenarios and levels of disruptions considered during testing?
• Are testing results analyzed to identify areas for improvement and corrective actions?
• Are there mechanisms to track and follow up on implementing corrective actions identified during testing?

• Review documentation of operational resilience testing and exercise plans.
• Evaluate the adequacy of the testing schedule and the consideration of various scenarios.
• Assess the analysis of testing results to identify areas for improvement.
• Verify the existence of mechanisms to track and follow up on corrective actions.
• Determine if lessons learned from testing and exercises are documented and incorporated into improvements.

• Is there an incident response plan in place for operational resilience incidents?
• Has the incident response plan been tested and validated?
• Are roles, responsibilities, and communication channels clearly defined within the incident response plan?
• Is there a designated incident response team and a straightforward escalation process?
• Is there a process for post-incident analysis and continuous improvement of the incident response capabilities?

• Review the incident response plan documentation for operational resilience incidents.
• Evaluate the testing and validation activities conducted on the incident response plan.
• Assess the clarity and accuracy of roles, responsibilities, and communication channels.
• Verify the incident response team's existence and composition and escalation process.
• Determine if there is a process for post-incident analysis and continuous improvement.

• Are there mechanisms to monitor and ensure compliance with operational resilience regulations?
• Is there evidence of regular assessments and audits to evaluate compliance?
• Are compliance gaps and deficiencies promptly addressed and remediated?
• Are there documented processes to stay updated with evolving regulatory requirements?
• Are there precise mechanisms for reporting and escalating non-compliance issues?

• Evaluate the mechanisms to monitor and ensure compliance with operational resilience regulations.
• Review evidence of regular assessments and audits to evaluate compliance.
• Assess the effectiveness of processes for addressing compliance gaps and deficiencies.
• Verify the existence of processes to stay updated with evolving regulatory requirements.
• Determine the clarity and effectiveness of mechanisms for reporting and escalating non-compliance issues.

Questionnaires and Checklist "Sustain" Phase

Implement Training and Awareness

Provide Self-assessment

Conduct Independent Quality Review

What is Self-assessment?

Self-Assessment in Operational Resilience ensures that the regulated organisation captures and documents the steps taken towards operational resilience and provides a comprehensive and objective evaluation of the organisation's strategy and overall ability to respond to disruptions.

• Are operational resilience policies and procedures well-documented and readily accessible?
• Are the policies and procedures aligned with industry best practices and regulatory requirements?
• Do the documented policies clearly define roles, responsibilities, and accountability for operational resilience?
• Is there evidence of regular reviews and updates to the operational resilience documentation?

• Review the documentation of operational resilience policies and procedures.
• Assess the alignment of policies with industry best practices and regulations.
• Evaluate the clarity and completeness of roles, responsibilities, and accountability definitions.
• Verify the existence of a process for regular reviews and updates to the documentation.

• Has a comprehensive risk assessment been conducted to identify and assess potential risks?
• Are risks prioritized based on their potential impact and likelihood?
• Are mitigation strategies and controls in place to address identified risks?
• Is there a process for regularly monitoring and updating risk assessments?

• Evaluate the documentation of the risk assessment process.
• Assess the comprehensiveness of the risk assessment, including identification and assessment of risks.
• Verify the prioritization of risks based on impact and likelihood.
• Review the documented mitigation strategies and controls.
• Determine if there is a process for regularly monitoring and updating risk assessments

• Has a thorough business impact analysis (BIA) been conducted to identify critical processes and systems?
• Have the potential impacts of disruptions to critical processes and systems been assessed?
• Are recovery time objectives (RTOs) and recovery point objectives (RPOs) defined for critical processes?
• Are mitigation strategies and plans in place to ensure the timely recovery of critical processes?

• Review the business impact analysis (BIA) process documentation.
• Evaluate the completeness and accuracy of the identification of critical processes and systems.
• Assess the thoroughness of the assessment of potential impacts.
• Verify the definition of recovery time objectives (RTOs) and recovery point objectives (RPOs) for critical processes.
• Review the mitigation strategies and plans to ensure timely recovery.

• Is there a training program in place to educate employees on operational resilience?
• Are employees aware of their roles and responsibilities regarding operational resilience?
• Are there mechanisms to track and monitor employee completion of operational resilience training?
• Are there regular communication and awareness campaigns to promote a culture of operational resilience?

• Review the training program documentation for operational resilience.
• Evaluate the effectiveness of the training in educating employees.
• Assess the mechanisms in place to track and monitor employee completion of training.
• Verify the existence of regular communication and awareness campaigns.
• Determine the extent of the culture of operational resilience within the organization.

• Have operational resilience plans and procedures been tested through exercises and simulations?
• Is there a documented schedule for testing and exercising operational resilience capabilities?
• Are different scenarios and levels of disruptions considered during testing?
• Are testing results analyzed to identify areas for improvement and corrective actions?
• Are there mechanisms to track and follow up on implementing corrective actions identified during testing?

• Review the operational resilience testing and exercise plan documentation. 
• Evaluate the adequacy of the testing schedule and the consideration of various scenarios.
• Assess the testing results analysis to identify improvement areas.
• Determine if lessons learned from testing and exercises are documented and incorporated into improvements.

• Is there an incident response plan for operational resilience incidents?
• Has the incident response plan been tested and validated?
• Are roles, responsibilities, and communication channels clearly defined within the incident response plan?
• Is there a designated incident response team and a straightforward escalation process?
• Is there a process for post-incident analysis and continuous improvement of the incident response capabilities?

• Review the incident response plan documentation for operational resilience incidents.
• Evaluate the testing and validation activities conducted on the incident response plan.
• Assess the clarity and accuracy of roles, responsibilities, and communication channels.
• Verify the incident response team's existence and composition and escalation process.
• Determine if there is a process for post-incident analysis and continuous improvement.

• Is there a process in place to monitor and review the effectiveness of the operational resilience program?
• Are lessons learned from incidents, tests, and exercises incorporated into improvements?
• Is there a mechanism to capture and address feedback and suggestions for operational resilience?
• Are there metrics and performance indicators to measure the effectiveness of the operational resilience program?
• Is there a culture of continuous improvement and learning within the organization?

• Evaluate the process for monitoring and reviewing the effectiveness of the operational resilience program.
• Assess the incorporation of lessons learned from incidents, tests, and exercises into improvements.
• Verify the existence of a mechanism to capture and address feedback and suggestions.
• Review the metrics and performance indicators for measuring program effectiveness.
• Determine the extent of the organization's continuous improvement and learning culture.

Questionnaires and Checklist "Sustain" Phase

Implement Training and Awareness

Provide Self-assessment

Conduct Independent Quality Review

What is a Communication Strategy?

A communication strategy is a guide or a plan that helps an organization share information and achieve its communication and business objectives.  In this case, the operational resilience initiative.

Ensuring effective and timely communication of the internal and external communication plans is essential to help the organization keep customers and other stakeholders informed.

• Have key stakeholders for operational resilience been identified?
• Is there a clear understanding of each stakeholder group's communication needs and expectations?
• Have internal and external stakeholders, including employees, customers, suppliers, and regulators, been considered?
• Is there a process to regularly review and update the stakeholder list and their communication requirements?

• Review the documentation of stakeholder identification for operational resilience.
• Assess the clarity and completeness of understanding each stakeholder group's communication needs and expectations.
• Verify that both internal and external stakeholders have been considered.
• Determine the existence of a process for regular review and update of the stakeholder list and their communication requirements.

• Have communication objectives for operational resilience been defined?
• Are there clear and concise key messages that must be communicated to stakeholders?
• Do the key messages align with the operational resilience goals and priorities?
• Is there a process to regularly review and update the communication objectives and key messages?

• Evaluate the documentation of communication objectives for operational resilience.

• Assess the clarity and alignment of key messages with operational resilience goals and priorities.
• Verify the existence of a process for regular review and update of the communication objectives and key messages.

• Are there appropriate communication channels to reach each stakeholder group effectively?
• Have the advantages and limitations of different communication channels been considered?
• Is there a mix of channels, including both traditional and digital, to ensure comprehensive communication?
• Are there tools and platforms in place to facilitate efficient and secure communication?

• Is there a documented communication plan for operational resilience?
• Does the communication plan include a timeline, responsibilities, and deliverables?
• Are there mechanisms to ensure timely and consistent communication?
• Are there processes in place to handle urgent and sensitive communications?

• Is there a process to measure and evaluate the effectiveness of communication activities?
• Are there metrics and performance indicators to assess the impact of communication efforts?
• Is feedback from stakeholders collected and analyzed to identify areas for improvement?
• Are there mechanisms to monitor and address misconceptions or misinformation?

Questionnaires and Checklist "Sustain" Phase

Implement Training and Awareness

Provide Self-assessment

Conduct Independent Quality Review

What is Scenario Testing?

Scenario Testing aims to test the organisation's ability to remain within impact tolerances in severe but plausible disruption scenarios, focusing on recovery and response arrangements rather than preventative measures.

1. Scenario Testing Planning

• Has a scenario testing plan been developed outlining the objectives, scope, and methodology?
• Are the scenarios relevant to the organization's critical business services and potential threats?
• Has the testing plan considered various disruption scenarios, including natural disasters, cyberattacks, and system failures?

• Verify the existence of a scenario testing plan that outlines objectives, scope, and methodology.
• Assess the relevance of the scenarios to the organization's critical business services and potential threats.
• Ensure the testing plan considers various disruption scenarios, including natural disasters, cyberattacks, and system failures.

2. Scenario Development

• Has a range of realistic scenarios been identified for testing operational resilience?
• Do the selected scenarios cover a variety of potential disruptions and stress events?
• Have scenarios been designed to test different aspects of operational resilience, including people, processes, technology, and facilities?
• Are the selected scenarios aligned with the organisation's risk profile and potential impact on critical business services?
• How were the scenarios developed? Were they based on historical incidents, industry best practices, or internal risk assessments?
• Are the scenarios realistic and representative of the organisation's potential threats and disruptions?
• Have relevant stakeholders, including management and subject matter experts, reviewed and approved the scenarios?

• Review the scenario development process and ensure it is based on historical incidents, industry best practices, or internal risk assessments.
• Evaluate the realism and representativeness of the scenarios concerning potential threats and disruptions.
• Confirm that relevant stakeholders, including management and subject matter experts, have reviewed and approved the scenarios.

What is Lesson Learnt?

The key to improving "Lesson Learnt" when implementing Operational Resilience or OR is for an organisation to promote a continuous learning and improvement culture.   It is essential to improve and communicate remediation and vulnerabilities after scenario testing.

Leadership Commitment

• Is there a visible leadership commitment to promoting a culture of continuous learning and improvement?
• Do leaders actively support and participate in scenario testing and incident review processes?
• Are leaders accountable for implementing recommendations and lessons learned from scenario testing and incidents?
• Is there a communication strategy emphasising the importance of continuous learning and improvement for all employees?

Learning Framework

• Is there a documented framework or process for capturing and analysing lessons learned from scenario testing and incidents?
• Does the framework include mechanisms for identifying and documenting root causes and contributing factors?
• Are there standardised templates or tools for collecting and organising lessons learned information?
• Is there a designated team or individual responsible for managing the lessons-learned process?

Incident Review and Analysis

• Is there a structured process for reviewing and analysing actual incidents?
• Are incidents thoroughly investigated to identify root causes and contributing factors?
• Are incident review findings documented and shared with relevant stakeholders?
• Is there a mechanism to track and monitor the implementation of corrective actions resulting from incident reviews?

Scenario Testing Evaluation

• Is there a process for evaluating the effectiveness and impact of scenario testing exercises?
• Are scenario testing results analyzed to identify areas for improvement and enhancement?
• Are there mechanisms to capture feedback from participants and stakeholders on the scenario testing process?
• Is there a feedback loop to incorporate insights from scenario testing into future exercises?

Knowledge Sharing and Communication

• Is there a platform or mechanism for sharing lessons learned and best practices across the organisation?
• Are lessons learned and best practices communicated to relevant teams and departments?
• Are there regular communication channels, such as newsletters or internal portals, to disseminate information on operational resilience and continuous learning?
• Is there a process for capturing and sharing success stories and examples of continuous learning and improvement?

Training and Development

• Is there a training program in place to enhance employees' knowledge and skills related to operational resilience?
• Are employees trained on incident response, scenario testing, and lessons learned?
• Are there opportunities for employees to participate in specialised training or workshops related to operational resilience?
• Is there a process to evaluate the effectiveness of training programs and incorporate feedback for improvement?

Metrics and Performance Monitoring

• Are there defined metrics and indicators to measure the effectiveness of the continuous learning and improvement initiatives?
• Is there a process to track and monitor the organization's performance in implementing lessons learned and recommendations?
• Are performance metrics used to identify areas of success and areas that require further attention?
• Is there a mechanism for reporting and communicating performance metrics related to operational resilience readiness?

Continuous Improvement Culture

• Is there a culture of continuous improvement embedded in the organisation's values and behaviours?
• Are employees encouraged and empowered to share insights, ideas, and suggestions for improving operational resilience?
• Are there mechanisms to capture and evaluate employee suggestions, such as suggestion boxes or innovation platforms?
• Are there recognition and reward mechanisms for individuals or teams that contribute to continuous learning and improvement?

External Benchmarking

• Does the organisation seek opportunities for external benchmarking and learning from other organisations?
• Are there partnerships or networks established to share experiences and best practices in operational resilience?
• Is there a process to review and incorporate relevant industry standards and guidelines into the organisation's practices?
• Are there mechanisms to learn from regulatory changes, industry trends, and emerging risks?

Governance and Oversight

• Is there a designated governance body or committee responsible for overseeing and promoting continuous learning and improvement?
• Are there regular reporting and updates provided to senior management or the board of directors on the organisation's operational resilience readiness and continuous improvement efforts?
• Are clear accountability and responsibilities assigned for implementing and monitoring continuous learning initiatives?
• Is there a process to review and assess the effectiveness of the organisation's continuous learning and improvement initiatives?

Questionnaires and Checklist "Implement" Phase

Set Impact Tolerance

Conduct Scenario Testing

Improve Lesson Learnt