Closure phase

Overview

The closure phase (which includes remediation planning and result sharing) allows all relevant stakeholders to reflect on the outcome of the test and make improvements to further enhance the cyber resilience of the entity. In this phase, the RT provider will draft a Red Team Test Report, which will include details of the approach taken to the testing and the findings and observations from the test.

Where necessary, the report will include advice on areas for improvement in terms of technical controls, policies and procedures, and education and awareness; the relevant stakeholders will replay the executed scenarios and discuss the issues uncovered during the test; the entity will take on board the findings, and agree and finalise a Remediation Plan (including planning for follow-up testing) with the authorities; the process will be reviewed, and the entity’s detection and response capabilities assessed and discussed; and the key findings from the test will be shared with other relevant authorities. The duration of the close-down activities in this final phase is approximately four weeks.

Red Team Test Report and Blue Team Report

The output of this activity is a draft version of the Red Team Test Report produced by the RT provider for delivery to the entity, which then forwards the document to the TCT. The draft report must be issued within two weeks of test completion. The key members of the entity’s BT are informed of the test and will use the Red Team Test Report to deliver their own Blue Team Report. In the Blue Team Report, the BT maps its actions alongside the Red Team’s actions. The Blue Team Report should be completed ahead of the replay workshop (see Section 10.3 below) to maximise the learnings from the replay.

Red Team and Blue Team replay

After the RT provider and BT deliver their reports, the entity must arrange a replay workshop. The goal of this workshop is to learn from the testing experience in collaboration with the RT provider. During the workshop, a replay is organised in which the BT and the RT provider review the steps taken by both parties during the test.

Additionally, a purple teaming element can be added, in which the BT and the RT provider work together to see which other steps could have been taken by the RT provider and how the BT could have responded to those steps.

When conducting the replay, the RT provider should state how well the testing team managed to progress through the targeted attack life cycle stages of each scenario. The RT provider should also offer an opinion as to what else could have been achieved with more time and resources given that genuine threat actors are not constrained by the time and resource limitations of TIBER-EU.

The TTM and TI provider can also be present during these replay workshops.

360 -degree feedback

During the 360-degree feedback meeting, the entity, TCT, and (TI and) RT providers should come together to review the TIBER-EU test. The TCT should arrange and facilitate the workshop. In the 360-degree feedback meeting, all parties should deliver feedback on each other and on the overall process. The goal is to further facilitate the learning experience of all those involved in the process for future exercises. The key topics to be covered, from all parties’ perspectives, are:

which activities/deliverables progressed well;
which activities/deliverables could have been improved;
which aspects of the TIBER-EU process worked well;
which aspects of the TIBER-EU process could be improved;
any other feedback.

In this way, the TI and RT providers will obtain feedback on their performance, and the relevant authorities will have opportunities to identify and improve the TIBER-EU process.

The TCT may share the output from the 360-degree feedback on an anonymous basis with the TKC so that all lessons learned can be reflected on and improvements can be made to the TIBER-EU framework. This is a key part of the “learning and evolving” principle that underlies the TIBER-EU framework.

Remediation Plan and Test Summary Report

After the BT and RT provider replay and the 360-degree feedback workshop, the entity should draft its Remediation Plan and Test Summary Report.

The R emediation Plan is based on the test results, which should be used in turn to support the business case for implementing improvements to mitigate the vulnerabilities identified during the TIBER-EU test.

The Test Summary Report summarises the overall test process and results (including the Remediation Plan) and should draw on the test documentation, such as the Red Team Test Report, the Blue Team Report, the TTI Report, the Red Team

Test Plan and the Remediation Plan. The Test Summary Report should not contain detailed technical information and findings regarding weaknesses and vulnerabilities, as information at that level of detail is highly sensitive and for the entity only. The entity must share the Test Summary Report with the lead authority. The lead authority may also review the more detailed findings from the test if this is deemed necessary.

At the end of the test, once the reports and Remediation Plan have been agreed, the entity, TI/RT providers and lead authority should provide an attestation confirming that the test was conducted in accordance with the core requirements of the TIBER- EU framework. The attestation should be signed by the board of the entity and TI/RT providers, and can serve as a means of qualifying the test for mutual recognition among other relevant authorities.

In cases where the other relevant authorities did not participate in the test but there was mutual agreement to share the test results, the lead authority and entity should share the Test Summary Report and attestation. The Test Summary Report serves as a form of assurance to other relevant authorities, and the attestation qualifies the test as a legitimate TIBER-EU test.

As one of the key objectives of the TIBER-EU f ramework is to enhance sector resilience, the TCT of each jurisdiction should analyse the results of all the tests to identify the key findings, common threats and vulnerabilities, and to disseminate these in the appropriate form to relevant stakeholders. The TCT may also share anonymised findings or lessons from their respective TIBER tests with the TKC. This information will allow the TKC to aggregate the key findings, common threats and vulnerabilities, to form a picture of the resilience of the European financial sector, and to bring about improvements where feasible. In all cases, any exchange of information should be conducted in a safe and secure manner.

Oversight and supervision

From the outset, it is up to the relevant authority to determine the role of the overseer and supervisor in the TIBER-XX implementation. In some cases, the authority may opt to include the overseer and supervisor throughout the entire testing process, while in some jurisdictions the authority may opt to formally exclude the involvement of the overseer and supervisor.

In cases where the overseer and supervisor have not been involved during the testing phase, the TCT should notify the oversight and supervisory functions once the test has ended. At this stage, it is recommended that the overseer and supervisor work with the entity to implement the remediation measures.