Annexes

List of qualifications and certifications

The following is a detailed, though not exhaustive, list of qualifications and certifications available in the market for TI and RT practitioners and organisations. Providers should demonstrate that their staff possess a broad range of such qualifications and certifications, and at an organisational level that they are suitably certified in security management. It should be noted that this list provides indicators and benchmarks that entities may leverage for their own assessment of procuring these security services, and entities should apply their professional judgement during the due diligence process and liaise with the respective TCT for further guidance (if necessary). The list provides a snapshot of the most well-known qualifications and certifications at international level and may be updated when there are major changes to Guidelines.

TI and RT managers

Certification body/Standard setting body	Qualification
CREST	CREST Certified Threat Intelligence Manager (CCTIM)
CREST	CREST Certified Simulated Attack Manager (CCSAM)
Offensive Security	Offensive Security Certified Expert (OSCE)
eLearnSecurity	eLearnSecurity Certified Penetration Tester eXtreme (eCPTX)

TI and RT teams

Certification body/Standard setting body	Qualification
CREST	CREST Certified Simulated Attack Specialist (CCSAS)
ISACA	CSX Penetration & Vulnerability Tester Pathway
	CSX-P - Cybersecurity Practitioner Certification
(ISC)²	Certified Information Systems Security Professional (CISSP)
	Systems Security Certified Practitioner (SSCP)
SANS Institute - GIAC	GIAC Penetration Tester (GPEN)
	GIAC Web Application Penetration Tester (GWAPT)
	GIAC Exploit Researcher and Advanced Penetration Tester (GXPN)
	GIAC Mobile Device Security Analyst (GMOB)
	GIAC Assessing and Auditing Wireless Networks (GAWN)
Offensive Security	Offensive Security Certified Professional (OSCP)
	Offensive Security Wireless Professional (OSWP)
	Offensive Security Exploitation Expert (OSEE)
	Offensive Security Web Expert (OSWE)
eLearnSecurity	eCPPT e-learn Security Certified Professional Penetration Tester (eCPPT)
	eLearnSecurity Web Application Penetration Tester (eWPT)
	eLearnSecurity Web Application Penetration Tester eXtreme (eWPTX)
	eLearnSecurity Mobile Application Penetration Tester (eMAPT)
	eLearnSecurity Certified eXploit Developer (eCXD)
Others	EC-Council Certified Security Analyst (ECSA)
	Licensed Penetration Tester (LPT)
	Certified Ethical Hacker (CEH)

TI and RT providers

Certification body/Standard setting body	Qualification
ISO	ISO/IEC 27001, ISO/IEC 29147, ISO 30111
NIST	NIST 800-115 for Information Security
FedRAMP	FedRAMP-Compliant data centres

List of questions to facilitate the procurement process

When the entity undertakes its procurement process and engages with potential providers, there are a number of questions it can pose to the prospective providers to gauge their levels of competence and suitability to deliver a TIBER-EU test. Although the entity (or an accreditation and certification provider) is responsible for validating the core requirements of the providers, as set out in the Guidelines, there are a number of questions of a more qualitative nature that the entity should pose to determine the provider’s eligibility. These questions are largely based on the guiding principles and criteria set out in the Guidelines.

The entity may use the questions below in its request for proposals, in the form of a self-assessment for the prospective provider to complete, or integrate them into its existing procurement processes and documents. Responses to the below questions will provide useful input to the entity (or accreditation and certification provider) in carrying out due diligence.

TI Providers

Reputation, history and ethics

• Can the TI provider provide evidence of a solid reputation, history and ethics (e.g. a full trading history, a strong history of performance, good feedback from both clients and providers and a reliable financial record)?
• More specifically, can the TI provider provide at least three reference from previous assignments related to red team tests?
• Is the TI provider accredited by an accreditation/industry body in the European Union?
• Does the TI provider adhere to a formal Code of Conduct and Ethical Framework?
• Does the TI provider contribute to specialised industry events (such as those run by BlackHat or RSA Conferences, etc.)?
• Is the TI provider sufficiently insured to conduct TIBER-EU tests? More specifically, does the TI provider have adequate indemnity insurance in place to cover activities which were not agreed upon in the contract and/or which stem from misconduct, negligence, etc?
• What is the TI provider’s recruitment policy and process?
• Does the TI provider ensure that its staff members are adequately vetted?
• Does the TI provider have adequate knowledge of the different jurisdictional regulations and requirements to conduct red team tests?

Governance, security and risk management

• Are the TI provider’s ISMS, and its implementation, independently audited? Can the TI provider share these audits with the entity?
• Does the TI provider hold any international certifications, with specific regard to security and risk management?
• Can the TI provider offer independent assurances that the risks associated with the red team test (including the entity’s confidential information and any other business risks) will be adequately addressed and protected and compliance requirements met?
• How does the TI provider ensure that the results of test are generated, reported, stored, and communicated, redacted (if necessary) and destroyed in a manner that does not put the entity at risk?
• How does the TI provider ensure that no data leakage occurs from its staff’s devices and teams?

Methodology

• Does the TI provider have a clearly documented methodology for conducting threat intelligence and reconnaissance?
• Does the TI provider have a comprehensive threat intelligence collection process and function, which provides the raw materials for conducting threat intelligence analysis?
• Does the TI provider have access to threat intelligence from a broad range of sources?
• Does the TI provider have the capability to analyse threat intelligence in multiple languages?
• Does the TI provider have the capability to analyse different types of threat intelligence (e.g. OSINT, HUMINT, SIGINT, etc.)?
• What mechanisms does the TI provider have in place to ensure that it can keep up to date with the latest tactics, techniques and procedures of advanced real-life attackers, and how are these transmitted to its staff?
• Does the TI provider take into account public data about previous incidents that would be relevant to the threats today?
• Does the TI provider take into account, and keep confidential, private data about previous incidents that would be relevant to the threats today?
• Does the TI provider look at the short, medium and longer-term goals of the business that might provide information on the likely interests of a potentially hostile party?
• Does the TI provider ask for previous risk assessments or risk models exercises?
• Does the TI provider have a comprehensive view of the financial sector and does it understand the current geopolitical context the entity is operating in?

Staff competence

• Does the TI provider employ a broad range of staff with varying expertise? Specifically, can the TI provider deliver services for TIBER-EU tests with teams, led by Threat Intelligence Managers?
• Does the TI provider have Threat Intelligence Managers with at least five years of experience in threat intelligence, including three years producing threat intelligence in the financial services industry?
• Do the Threat Intelligence Team members each have at least two years of experience in threat intelligence?
• Can the TI provider demonstrate that its Threat Intelligence Team has experience in delivering threat intelligence for red team tests?
• Can the TI provider specify named individuals who will be responsible for managing and conducting the test, their experience of the environment within the scope, their qualifications and the exact role each individual will perform?
• Can the TI provider demonstrate that its Threat Intelligence Team is multi- disciplinary, with a broad range of skills including OSINT, HUMINT and geopolitical knowledge?
• Can the Threat Intelligence Manager provide an up-to-date CV and at least three references from previous assignments, specifically in delivering threat intelligence for red team testing activities?
• Can the TI provider provide an up-to -date CV for each member of the Threat Intelligence Team?
• What qualifications do the TI provider’s staff hold in the various areas in which tests may be required?
• What continuous professional development programme does the provider have in place to ensure that its staff continuously enhance their skills?
• Are all staff experienced in the specific dynamics of the financial services industry?

RT providers

Reputation, history and ethics

• Can the RT provider provide evidence of a solid reputation, history and ethics (e.g. a full trading history, a strong history of performance, good feedback from both clients and providers and a reliable financial record)?
• More specifically, can the RT provider provide at least five references from previous assignments related to red team tests?
• Is the RT provider accredited by an accreditation/industry body in the European Union?
• Does the RT provider adhere to a formal Code of Conduct and Ethical Framework?
• Does the RT provider contribute to specialised industry events (such as those run by BlackHat or RSA Conferences, etc.)?
• Is the RT provider sufficiently insured to conduct TIBER-EU tests? More specifically, does the RT provider have adequate indemnity insurance in place to cover activities which were not agreed upon in the contract and/or which stem from misconduct, negligence, etc?
• What is the RT provider’s recruitment policy and process?
• Does the RT provider ensure that its staff members are adequately vetted?
• Does the RT provider have adequate knowledge of the different jurisdictional regulations and requirements to conduct red team tests?

Governance, security and risk management

• Are the RT provider’s ISMS, and its implementation, independently audited? Can the RT provider share these audits with the entity?
• Does the RT provider hold any international certifications, with specific regard to security and risk management?
• Does the RT provider apply independently validated security and risk management controls during the red team testing process?
• Can the RT provider offer independent assurances that the risks associated with the red team test (including the entity’s confidential information and any other business risks) will be adequately addressed and protected and compliance requirements met?
• How does the RT provider ensure that the results of test are generated, reported, stored, communicated, redacted (if necessary) and destroyed in a manner that does not put the entity at risk?
• Does the RT provider record and log all tests carried out by its testers, and what is the retention period of these records and logs?
• How does the RT provider ensure that no data leakage occurs from its staff’s devices and systems?

Methodology

• Has the RT provider ever performed testing that emulates the most advanced attackers involving people, processes and technical weaknesses? Can the RT provider give examples and references?
• Is the RT provider able to demonstrate exploits or vulnerabilities it has found in other similar environments?
• Is the RT provider adequately capable of collecting threat intelligence concerning its (potential) trargets?
• Does the RT provider have experience emulating advanced attacks on live critical core financial systems? If yes, the entity should request evidence.
• Is the RT provider mature and capable enough to adapt its attack scenarios and techniques during the test, dependent on the behaviour of the target?
• Can the RT provider provide evidence that it can provide high-quality services, including the methodologies, tools, techniques and sources of information it will use as part of the testing process?
• Is the RT provider mature and creative enough to develop high-end scenarios using cutting-edge techniques available on the market? Does the RT provider have knowledge of the latest vulnerabilities and can it develop its own tools?
• Does the RT provider have knowledge and experience of the financial sector and the functioning of its systems?
• How does the RT provider perform rigorous and effective team tests to ensure that a wide range of system attacks is simulated?
• Can the RT provider describe its proven testing methodology that is tailored for particular types of environment (e.g. infrastructure, web applications and mobile computing)?
• Can the RT provider demonstrate its red team testing capabilities (e.g. by making a presentation, showing examples of similar projects it has undertaken) and provide a sample report?
• Does the RT provider have independently reviewed quality assurance processes that it applies to each test, in order to ensure client requirements are being met in a secure and productive manner?
• What is the exploitation process used by the RT provider? How does the RT provider ensure that it is safe?
• Can the RT provider support out of business hours testing?
• What is the RT provider’s peak testing capacity?
• Can the RT provider’s infrastructure and team support the peak requirement of the entity?

Staff competence

Does the RT provider employ a broad range of staff with varying expertise? Specifically, can the RT provider deliver services for TIBER-EU tests with teams, led by Red Team Test Managers? - Does the RT provider have Red Team Test Managers with at least five years of experience in red team testing, including three years managing red team tests in the financial services industry? - Do the red team members each have at least two years of experience in red team testing? - Can the RT provider specify named individuals who will be responsible for managing and conducting the test, their experience of the environment within the scope, their qualifications and the exact role each individual will perform? - Can the RT provider demonstrate that the composition of its red teams is multi- disciplinary, with a broad range of knowledge and skills, such as: business knowledge, red team testing, penetration testing, reconnaissance, threat intelligence, risk management, exploit development, physical penetration, social engineering, vulnerability analysis and combinations thereof? - Can the Red Team Test Manager provide an up-to-date CV and at least three references from previous assignments, specifically in red team testing activities? - Can the TI provider provide an up-to -date CV for each member of the red team that will conduct the TIBER-EU test? - What qualifications do the RT provider’s staff hold in the various areas in which tests may be required? - What continuous professional development programme does the provider have in place to ensure that its staff continuously enhance their skills? - Are all staff experienced in the specific dynamics of the financial services industry? - How do the RT provider’s testers identify “root cause” findings, strategically analyse findings in business terms, help to develop security improvement strategies and recommend counter measures to both address vulnerabilities and prevent them from recurring?

R&D capability

• Does the RT provider have an active, continuous and relevant R&D capability?
• Has the RT provider produced research papers, published vulnerabilities or won awards in the industry?
• Does the RT provider perform sufficient R&D to be able to identify all significant vulnerabilities?
• How does the RT provider carry out specially tailored, manual tests to help detect unknown vulnerabilities, rather than simply using a standard of set tools?
• Does the RT provider have proprietary tools and technology?

TI provider agreement checklist

When formalising the arrangements for a TIBER-EU test, the entity and TI provider could agree on the following clauses as part of their agreement. The agreement must be shared with the TIBER Test Manager.

Table 4 - Example of TI provider agreement checklist

Intellectual property

• The contract includes agreements on intellectual property (IP), stating that IP remains with the entitled party.

Non-disclosure agreement

The contract has a non-disclosure agreement, stating as a minimum that:

• information will not be used outside of the context of TIBER-EU
• information will only be used for the purpose for which it was provided/collected; and
• the TI provider must ensure that all staff involved in the service (including staff provided by external parties) adhere to the agreements made concerning security and confidentiality.

Sharing threat intelligence information

The threat intelligence report that the entity receives is the property of the entity. The entity therefore has the right to share this information with other relevant parties. The TI provider cannot share this information with any other party without the prior approval of the entity.

Information security

The TI provider demonstrates its security measures and procedures and how these operate. For example:

• the TI provider has a security policy, approved by its Board of Directors;
• the TI provider has a demonstrable effective Information Security Management System;
• information security is an integral part of the TI provider’s risk management processes;
• every risk-mitigating measure, including those regarding information security, is documented and reviewed regularly;
• information systems used for storing and processing information regarding the entity are adequately protected and secured using state of the art methods, including periodical penetration tests and vulnerability assessments;
• information asset management is in place including inventories, retention and secure deletion and destruction; and
• all information related to TIBER-EU is accessed on a need to know basis. This is controlled by a combination of pro-cedural and technical measures, and all access to this information is logged and monitored.

Acceptance of provided services

The entity and the TI provider define criteria and validation methods according to which the delivered services will be accepted by the entity.

Pricing

Pricing is part of the agreement. The TI provider is transparent in the pricing of its services, including any additional or value added services.

Continuity

The TI provider has implemented policies and procedures to ensure continuity of its services during the term of the contract.

Audit

The TI provider gives the entity permission to verify the process and the results of the agreement by means of an (external) audit. The agreement specifies by which party, at what time, at what cost (including the distribution of costs amongst the contracting parties) and against which audit standards.

Assurance

The TI provider can provide assurance, via its second and third lines of defence or via external assurance providers, that its risk management objectives related to the service are met. The TI provider shows that processes crucial to the service and continuity are effective.

Legal privacy requirement (data transfer agreement)

If processing (including storing) of data takes place outside the legal premises of the European Economic Area (EU + Switzerland, Norway and Iceland), a data transfer agreement confirming compliance with EU standards is required. This requirement is also effective when data are processed in the European Economic Area but are accessible for e.g. technical support from within countries outside the European Economic Area.

Legal privacy requirement (personally identifiable information)

If the TI provider processes personally identifiable information, it will do so according to the GDPR.

Service quality

The TI provider provides services in accordance with the quality associated with the TIBER-EU standards. The agreed quality standards, as well as related technical and operational security requirements, are defined in a service level agreement between the TI provider and the entity. Additional requirements (e.g. SLAs and KPIs) can be added by the procurement staff of the entity.

Security incidents and risks

Security incidents regarding the agreed upon services are always reported immediately to the entity. The TI provider has implemented an efficient process to ensure the timely notification of security incidents and risks related to the services provided to the entity. When asked, the TI provider is willing to provide the entity with the details of this process.

Responsible disclosure procedure (RDP)

The TI provider and entity should agree that:

• if the TI provider finds vulnerabilities or other weaknesses during the research on an entity, it will disclose these to the white team of that entity; and
• if the TI provider finds vulnerabilities or other weaknesses during the research on an entity that relate to a product that is generally used, e.g. in operating systems, it will disclose these vulnerabilities or weaknesses to the vendor of that particular product.

Screening of employees

The TI provider has an adequate process for assuring that its employees are of outstanding reputation, are not and were never involved in criminal activity relevant to his or her current occupation and have sufficient skills to perform TI tasks for entities. The TI provider is willing to demonstrate the existence and the operation of this process.

Change of services

The entity or its affiliates are always entitled to ask for a change in the way the TI provider provides its services to the entity.

Exit clause (general)

The TI provider provides formal procedures to assure the destruction of any threat intelligence regarding the entity after the end of the contract and relationship between the TI provider and the entity.

Exit clause (confidentiality)

Arrangements regarding confidentiality are still valid after the end of the contract and relationship between the TI provider and the entity.

RT provider agreement checklist

When formalising the arrangements for a TIBER-EU test, the entity and RT provider could agree on the following clauses as part of their agreement. The agreement must be shared with the TIBER Test Manager.

Table 5 - Example of RT provider agreement checklist

Intellectual property

The contract includes agreements on intellectual property (IP), stating that IP remains with the entitled party.

Non-disclosure agreement

The contract has a non-disclosure agreement, stating as a minimum that:

• information will not be used outside of the context of TIBER-EU;
• information will only be used for the purpose for which it was provided/collected; and
• the RT provider must ensure that all staff involved in the service (including staff provided by external parties) adhere to the agreements made concerning security and confidentiality.

Sharing threat-intelligence information

The threat intelligence report that the entity receives is the property of the entity. The entity therefore has the right to share this information with other relevant parties. The RT provider cannot share this information with any other party without the prior approval of the entity.

Roles and responsibilities

The contract defines and states roles and responsibilities to avoid confusion, misunderstanding or abuses. One person within the RT provider should be accountable during the whole contract life cycle to ensure that: - security risks and requirements are fully understood; - appropriate processes are in place and a minimum acceptable level of residual risk is agreed with the entity and duly accepted by each party; - security risks are managed and appropriate processes are in place and communicated to the entity; - appropriate support is provided to the entity; and - contractual clauses are respected.

Information security

The RT provider demonstrates its security measures and procedures and how these operate. For example: - the RT provider has a security policy, approved by its Board of Directors; - the RT provider has a demonstrable effective Information Security Management System; - information security is an integral part of the RT provider’s risk management processes; - the RT provider should provide evidence of its relevant internal information security policies ensuring the security and resilience of its products and services; - every risk-mitigating measure, including those regarding information security, is documented and reviewed regularly; - information systems used for storing and processing information regarding the entity are adequately protected and secured using state of the art methods, including periodical penetration tests and vulnerability assessments; - information asset management is in place including inventories, retention and secure deletion and destruction; and - all information related to TIBER-EU is accessed on a need to know basis. This is controlled by a combination of procedural and technical measures, and all access to this information is logged and monitored.

Service quality

The RT provider ensures services in accordance with the quality associated with the TIBER-EU standards. The agreed quality standards, as well as related technical and operational security requirements, are defined in a service level agreement between the RT provider and entity. These high-level requirements aim to provide the control required to mimic the most advanced attacks on live critical systems. Additional requirements (e.g. SLAs and KPIs) can be added by the procurement staff of the entity.

Acceptance of provided services

The entity and the RT provider define criteria and validation methods according to which the delivered services will be accepted by the entity.

Pricing

Pricing is part of the agreement. The RT provider is transparent in the pricing of its services, including any additional or value added services.

Continuity

The RT provider has implemented policies and procedures to ensure continuity of its services during the term of the contract.

Audit

The RT provider gives the entity permission to verify the process and the results of the agreement by means of an (external) audit. The agreement specifies by which party, at what time, at what cost (including the distribution of costs amongst the contracting parties) and against which audit standards.

Assurance

The RT provider can provide assurance, via its second and third lines of defence or via external assurance providers, that its risk management objectives related to the service are met. The RT provider shows that processes crucial to the service and continuity are effective.

Legal privacy requirements (data transfer agreement)

If processing (including storing) of data takes place outside the legal premises of the European Economic Area (EU + Switzerland, Norway and Iceland), a data transfer agreement confirming compliance with EU standards is required. This requirement is also effective when data are processed in the European Economic Area but are accessible for e.g. technical support from within countries outside the European Economic Area.

Legal privacy requirement (personally identifiable information)

If the RT provider processes personally identifiable information, it will do so according to the GDPR. Security incidents and risks Security incidents regarding the agreed upon services are always reported immediately to the entity.

The RT provider has implemented an efficient process to ensure the timely notification of security incidents and risks related to the services provided to the entity. When asked, the RT provider is willing to provide the entity with the details of this process.

Reporting of vulnerabilities and weaknesses

The RT provider and the entity agree that:

• if the RT provider finds vulnerabilities or other weaknesses during the research on an entity, it will disclose these to the white team of that entity; and
• if the RT provider finds vulnerabilities or other weaknesses during the research on an entity that relate to a product that is generally used, e.g. in operating systems, it will disclose these vulnerabilities or weaknesses to the vendor of that particular product.

Screening of employees

The RT provider has an adequate process for assuring that its employees are of outstanding reputation, are not and were never involved in criminal activity relevant to his or her current occupation and have sufficient skills to perform intelligence tasks for entities. The RT provider is willing to demonstrate the existence and the operation of this process.

Credentials of the RT provider’s employees should be provided to demonstrate their relevant experience.

Employees’ security knowledge and training

The RT provider should provide sufficient evidence regarding the training programme of its employees. The entity should request the RT provider to perform and provide its due diligence to ensure that its employees have sufficient security and technical knowledge, skills and qualifications to avoid any unintentional alterations of the entity’s systems. This due diligence should demonstrate that its red team meets the requirements set out in the Services Procurement Guidelines.

Personnel changes

All personnel from the RT provider or downstream sub-contractors, who are involved in the entity’s TIBER-EU test, should sign a confidentiality and non-disclosure agreement. The contract between the entity and the RT provider should include clauses regarding confidentiality and personal data protection.

Any change or replacement of personnel in the scope of the test should be agreed with the entity.

Change of services

The entity or its affiliates are always entitled to ask for a change in the way the RT provider provides its services to the entity.

Exit clause (confidentiality)

Arrangements regarding confidentiality are still valid after the end of the contract and relationship between the RT provider and the entity.

Exit clause (general)

The RT provider provides formal procedures to assure the destruction of any information security-related information regarding the entity after the end of the contract and relationship between the RT provider and the entity.