Skip to content

Purple teaming in the testing phase

Rationale

While the TIBER-EU methodology ensures thorough planning, circumstances may arise during a live test that force the stakeholders to act pragmatically to balance the objective of maximising the learning outcome against maintaining a strict interpretation of the framework.

A considerable amount of time, cost and effort goes into planning and executing a TIBER test. So, the invalidation of a TIBER test is not desirable, unless the test fails to meet the requirements and spirit of the framework. Hence, it is reasonable that in certain circumstances, it is possible to carry out PT during the testing phase in order to continue the TIBER test and to maximise the return on investment.

It is recommended that a scenario-based approach be employed for PT within a TIBER test. When planning for or transitioning to PT during the testing phase, it is always advisable to re-evaluate the attack scenarios to ensure they fit the PT setting while still remaining close to the TTPs of the simulated threat actor (see Figure 1). The PT activity may cover one or more of the attack scenarios, depending on the situation; however, it is conceivable that PT will be applied to a specific attack scenario whereas other attack scenarios will continue normally under red teaming.

Indicative PT timeline in the testing phase

Indicative PT timeline in the testing phase

Circumstances leading to purple teaming

Alternative ways of progressing are thoroughly examined before the WT proposes moving to PT. For example, pausing the test should be considered to see if this would be an equally suitable measure to maximise the lessons learnt. It is then up to the TCT to evaluate each case individually, in close dialogue with the WT, RT and TI providers, and assess whether PT is an option.

The WT is also encouraged to consider, within the risk management controls of the test, any circumstances that may lead to PT. For this reason, it is advisable for the WT to plan to execute the most daring or noisiest attack scenarios last in order to avoid the BT detecting RT activities during the early stages of the test.

Some potential circumstances that may lead to PT during the testing phase are described below.

  • When the BT has detected the RT in such a way that the secrecy of the test is irreparably compromised. Note that it is possible that during a test, the BT may detect some RT actions; however, this alone does not necessarily mean that PT is the right way forward, and it is possible to still continue the test in its original RT manner using a cover story (e.g. a local penetration test) to explain certain detections to the BT or to only introduce PT for the detected attack scenarios. In addition, it may be possible that a test is partially detected by the BT and the WT can then instruct a freeze on RT activities to allow the elevated threat level to subside. In such cases, it is crucial to have alternative approaches and techniques at hand, as it is a common mistake to pause only to reuse the same attack vectors that have already been detected.
  • In difficult to foresee situations where there is a high degree of confidence that the emulated attack on specific systems that underpin critical functions could lead to a substantial disruption. In these cases, it is advisable to discontinue testing and to introduce PT for these systems instead. This would enable the BT, once informed, to take timely action to prevent and minimise any impact.
  • In the case of a parallel real cyberattack (i.e. outside of the TIBER test) where the BT has to fully shift its focus to disruption prevention and containment. This may result in the TIBER test being revealed in order to transparently help the BT differentiate TIBER activities from the genuine attack. Among other possibilities, the test may be postponed to a later date, possibly utilising PT.
  • When there is a high probability (e.g. derived from clear signs) that the response of the uninformed BT to contain the detected emulated attack will have a critical impact on systems underpinning critical functions. This potential overreaction might be appropriate in the event of a real attack, but not in the context of a TIBER test, given that the RT will never deliberately cause disruption. If the BT is not aware of the TIBER test, they have no way of knowing if their response is adequate.
  • To prevent situations that can lead to the BT straying from normal response procedures. This would be counterproductive and both reduce the realism of the test and hamper its learning outcome. This can happen when the BT, suspecting that the attack is not genuine, changes its attitude and response mechanisms.
  • When the WT is unable to stop escalation by the BT and the BT has involved external parties such as the police, intelligence services, government authorities, industry bodies or financial institutions, for example, due to the perceived severity of the incident. Involving these parties will put an unnecessary strain on those authorities and could have a severe impact on current and future testing activities. The test should be halted immediately and PT may be considered.

Minimum requirements in the testing phase

It is not possible to provide an exhaustive list of circumstances that could result in shifting to PT during the testing phase. However, one of the main criteria should be that the testing phase cannot continue in a secret and/or secure manner due to an event outside of the control of the WT, RT or TCT.

Including PT in the testing phase needs to be discussed and agreed. This should involve:

  • the WT formally proposing PT, detailing specific scope and objectives;
  • the TCT agreeing to PT and not raising an objection;
  • the test still being conducted in accordance with the spirit of the framework (i.e. PT should be considered an option of last resort rather than a relaxation of the TIBER-EU requirements), focusing on maximising the learning experience and outcome;
  • the WT liaising with the TI and RT providers as necessary to adapt existing scenarios or implement alternative scenarios so as to maximise the value of the test for the tested entity;
  • agreeing in advance on expectations regarding the outcome, communication channels, response and recovery activities, confidentiality boundaries, start and end, escalation paths, allocated resources (including budget) and reporting formats;
  • agreeing that the outcomes of PT be clearly documented and form an integral part of the remediation plan.