CIR TE Types of Tests

CIR Testing and Exercising

Types of Tests

This section will cover the various types of tests organisations can conduct to evaluate the effectiveness of their CIR plan.

1. Evaluate the Availability and Relevancy of the Plan

Most plans are documented with the assistance of automation, meaning that the copies of the plan are in soft copy. Being in the soft copy also facilitates the distribution (Goh, 2006) of the plan to the respective team members.

However, as cyber security incidents target the information assets within an organisation, there is a high probability that the information/data found in the organisation’s systems are affected. The CIR plan document would not be available to the team members during a cyber security attack.

Hence, there has to be a hard copy available in a location that is the easiest for the team members to go and retrieve so that they can execute the appropriate response procedures. Additionally, the organisation should develop a program that reviews and updates the plan regularly to ensure that the content of the plan is up to date. An up-to-date plan will ensure it remains relevant to tackle the evolving cyber security threats.

2. Retrieve information/ Data from Virtual Servers

Previously, information or data produced or utilised were stored on tapes in an offsite location (Goh, 2006) or within the headquarters with rigorous and stringent security management. These tapes, which are physical objects, would assure organisations with a guaranteed recovery of the lost information unless the tapes were destroyed.

Now, organisations have begun to store their information or data in data centres or the cloud. The storage of information has shifted from physical servers to virtual servers. Therefore, this test is conducted to determine if the organisation can retrieve their information or data from the virtual servers when it is under a cyber security attack so that the data that was backed up can be used for the resumption of CBFs or as a starting point to key in the data that was lost from the cyber security incident.

3. Contact Relevant Parties

A list of contact details of the staff, suppliers and vendors has been developed during the previous processes. The purpose of conducting this test (Goh, 2006) is to ensure that the contact information found on the list is up to date. During a cyber security incident, the relevant parties, especially those with key roles and responsibilities, need to be easily contactable. If the contact information is outdated, the exchange of information to facilitate efficient incident management is disrupted.

4. Check Conditions of Information Assets

An inventory of the information assets within the organisation should have been developed and attached as supplementary documents for the CIR plan. These information assets should be available and ready for deployment (Goh, 2006) when the primary information assets within the organisation are affected by the cyber security attack.

Therefore, by conducting this test, the organisation ensures that its backup information assets are in good working conditions and not susceptible to the second round of cyber security attacks.

5. Determine the Readiness of the Alternate Site

Specific organisations would have chosen an alternate site (Goh, 2006) as part of their response strategy. Depending on the nature of the alternate site, the organisation will have to verify if it is ready to shift operations over temporarily.

For example, a hot site is almost a replicated copy of the primary location, so most of the facilities found in the primary location will be at the alternate site. Therefore, the organisation has to check to ensure that every facility is functioning as it should be, so there will be no issues when operations are transferred over during a cyber security incident.

The same steps are taken when the organisation uses a cold site instead. The only difference is that a cold site only has the bare minimum of facilities. Hence, the time and resources spent determining the alternate site's readiness are significantly lesser.

Additionally, some facilities are not at the alternate site during peace times. Contracts have been established with vendors where that will provide the necessary facilities during a cyber security incident. The organisation has to engage with the vendors to confirm the facilities’ availability. Hence, they are ready at the alternate site for the transference of operations in the event of a cyber security incident.

6. Test Team Members’ Knowledge of the Plan

Although the CIR plan has been documented and distributed to every team member, some members do not read the content. As the management of cyber security incidents shifts to the organisation’s responsibility instead of just being the IT department’s, every employee within the organisation has a role to play (Goh, 2006), ensuring effective management of cyber security incidents. This test ensures that all employees have read through the plan and are aware of the procedures documented. At the same time, it highlights the enormity and severity of cyber security threats the organisation faces. It builds a culture so security practices can be incorporated into daily operations.

7. Notification Call Tree

The notification call tree structure, as shown in Figure 9-1, is developed to facilitate the exchange of information (Goh, 2006) during a cyber security attack.

Figure 9-1: Example of Notification Call Tree Structure

When a cyber security attack occurs, not all the team members may be at the affected site at the time of the attack. However, during this crucial point, it is essential that every member is aware and informed of the status of the attack so that the appropriate response procedures can be executed.

Therefore, two-way communication with the Organisation BCM Coordinator as the middle-man must communicate the correct messages to the relevant parties. By conducting this test, the different members are aware of who they are responsible for informing, so the flow of information during a cyber security incident is smooth and not disrupted.

8. Walk-through/ Table Top

Relevant parties gather in a location and discuss the execution of appropriate procedures for a given cyber security scenario. The environment created from this test facilitates communications between the involved parties (Kick, 2014).

Different viewpoints from the respective parties are voiced, allowing the organisation to align cyber security incident processes to address their concerns. Internal and external relationships are established, developing communication channels for information exchange.

At the same time, the readiness of response capabilities by the respective teams and the effectiveness of the documented procedures are evaluated to see if they can achieve the recovery objectives. In Appendix 20, the steps to design an effective tabletop exercise are elaborated on in detail.

9. Penetration Testing

Penetration tests are designed to identify vulnerabilities that cybercriminals can exploit. An increased understanding of the weaknesses in cyber security within the information assets in the organisation can be achieved (Redscan, 2017) with penetration tests. Therefore, the organisation can allocate resources more efficiently to implement cyber security controls depending on the test results.

10. Red Team versus Blue Team

Live events can be included to increase the realism and training opportunities for the audience through conducting the red and blue team exercise. The red team assumes the role of the cybercriminal while the blue team assumes the role of the organisation defending against the cyber security attack. Using the red and blue team exercise, a mixture of actual and fictitious scenarios (Kick, 2014) are simulated to provide realistic training for the team members.

As the team constitution comprises various employees from different aspects of the organisation, communication and interaction between them improve as they work together to limit the impacts of the simulated cyber security attack.

11. Black Box, White Box and Grey Box Testing

These three tests (Saunois, 2016) (Das, 2016) are conducted to ensure that a newly developed software or application is complete, secure and efficient.

12. Black Box Testing

During the reconnaissance phase of the cyber security kill chain, where the cybercriminal gathers intelligence regarding their target organisation, he will probably not know all the details about the information assets within the organisation. Hence, the cybercriminal will launch an all-out attack to detect vulnerabilities or weaknesses within the organisation to exploit.

Contextualising this to penetration testing, only the functionalities of the software or application are reviewed. Information regarding the internal mechanism of the software or application is not given. This type of test, commonly referred to as ‘trial and error’, takes a long time to complete due to the number of scenarios required to test all of the functionalities of the software or application to determine if it can serve the user efficiently and not be exploited by cybercriminals to launch a cyber security attack.

13. White Box Testing

White box testing is the complete opposite of black box testing, where information about the internal mechanism of the software or application is given. The functioning, processes and internal structure of the software or application are reviewed instead of the functionalities. Although a quicker and much more thorough test is conducted, more resources regarding time to decide on the scope of the test and sophisticated tools are required.

With the knowledge and access to the internal mechanism of the software or application, the organisation can understand the attack pattern and mindset of the cybercriminal. Functionalities of the software or application affected by the cyber security attack can be identified. With this information, the organisation can develop and implement appropriate cyber security controls to prevent the cyber security attack from affecting the functionality of the software or application.

14. Grey Box Testing

Grey box tests incorporate elements of both the black and white box tests. With partial knowledge of the internal mechanism, both the functionalities and functioning of the software and application are reviewed.

15. Bug Bounty

Some organisations, huge corporate ones, are under assault from cyber security attacks daily. Therefore, a developing trend among these organisations is utilising the bug bounty program. A bug bounty program refers to crowdsourcing hackers and information security experts to pinpoint vulnerabilities (HSNW, 2017) within an organisation's information assets.

Monetary rewards or recognition is given to those (Siwicki, 2017) who have responsibly disclosed the identified vulnerabilities before potential cyber criminals can exploit them to launch a successful cyber security attack on the organisation.

16. Incident Simulation/ Scenario Exercise

The information assets within the organisation utilised for the operation of CBFs have been identified from the previous process of Risk Analysis and Review and Business Impact Analysis. With this information, the organisation develops a scenario to simulate a cybercriminal's actions and thought processes.

The attack vectors that a cybercriminal can utilise are numerous an organisation can't have the necessary resources to protect every aspect. Additionally, the resources required to perform this exercise are incredibly strenuous on the organisation’s limited availability of resources.

Through simulation of a cyber security attack (F. Mills, R. Grimaila, L. Peterson, & W. Butts, 2011), it focuses the organisation’s attention on particular behaviours of the cyber criminals to prepare themselves for a fraction of the numerous forms of cyber security attacks. Over time, as the organisation develops and grows, more refined scenarios can be developed to test its capability to manage cybersecurity incidents.

The scenarios should be developed with the mindset of a cybercriminal. Based on the multiple information assets within the organisation, the organisation can pinpoint the cyber criminal's goal (stealing information, denial of service).

Referencing the cyber security kill chain from the "Mitigation and Response Strategy" blog (Click the right icon), cybercriminals must execute specific steps to launch a successful cyber security attack. These compulsory events can be identified and assessed by the organisation.

The development of attack scenarios depends on the organisation's nature; no fixed formula exists. The organisation can adopt the attack tree concept where the cyber criminal’s goals are identified first, then specify how the goals can be achieved. However, a disadvantage of the attack tree concept is the inability to differentiate between authorised and malicious activity when the cybercriminal gains authorised access to the organisation’s system.

Related Topics for CIR Testing and Exercising

Overview of Testing and Exercising	Test Design	Types of Tests
Scheduling	9.7 Baseline for Success Criteria	Back To: Table of Content

Do You Want to Continue BCM Training onsite or online?

Competency-based Course		Certification Course

Goh, M. H. (2017). A Manager's Guide to Business Continuity Management for Cyber Security Incidents, 2nd Edition. GMH Pte Ltd.

Reference: Chapter 9 Testing and Exercising 9.5 Types of Tests

Note:  This version was the draft 2nd Edition being updated in 2023. The numeric in the square bracket [X-X] cross-refers to the actual chapter and section in the 1st Edition.