After performing many Purple Team engagements with organizations ranging from large enterprise networks to small-to-medium-businesses, we've found that the most effective and productive experiences were due in large part to the preparation and the readiness of our clients. This post will explore some of the factors that we’ve noticed help clients and our team conduct the most organized, effective, and successful engagements. We'll also cover how TrustedSec can help you get ready or assess your present security policy in preparation for future Purple Team exercises.

1    Pre-Purple Team Engagement Scoping

1.1      Pre-Scoping Questionnaire

To help evaluate whether your organization is ready for a Purple Team engagement, TrustedSec has designed a list of questions to help identify how to best align organizations with our various offerings. Below is a sample of questions we ask prospective clients to accurately identify their current overall security posture:

1. What security tools do you have deployed and in use? Please share the vendors of these tools.
2. Do you have an internal team to manage these tools or do you contract with a third party?
3. You mentioned you have [SIEM, EDR, AV, Behavior Analytics, select any that apply]. Does your internal team or third-party management have a single source to view this data? 
4. Does your team have administrative access to your security tools to edit rules and alerts and to provision access to TrustedSec? 
5. Are you ingesting logs from workstations/servers/EDR into the SIEM? If not, is this something a technical resource can create dynamically for the engagement?

If it is determined that an organization is not ready for more advanced Purple Team engagements, TrustedSec has multiple offerings designed to help identify gaps in security coverage, logging, and tooling as well as challenges with the existing security posture and policy. The following sections will provide overviews of these offerings, and how they can help prepare clients for future advanced Purple Team engagements.

1.2      Detection & Alerting Workshop (DAW):

TrustedSec's Detection & Alerting Workshop is a pre-scoping discovery call that engages the client in interview-style questioning (i.e., inquiry-based testing) with security leadership and the teams responsible for responding to security alerts. The goal is to create a roadmap for the creation of alerting to cover existing detection gaps within the existing SIEM. Broadly speaking, a Detection & Alerting Workshop engagement measures the existing and future capability of an organization's security policy.

The Detection & Alerting Workshop is broken up into the following phases:

1.2.1     Primary Areas of Concern

TrustedSec works with the client to identify where the critical assets and data exist within the organization. The generated discussion is then used to develop insights into possible attack paths based on current trends and historical events.

1.2.2     Analysis of TTPs

Using the data gathered and current security incident response procedures, the triggers that would initiate a security event at the organization are investigated. Upon further review and analysis, the data sources required to observe these behaviors are also discussed.

1.2.3     Detection Prioritization

• Immediate (1-3 months)
• Prospective (3 months to 1 year)
• Strategic (>year)

1.2.4     Reporting

The deliverable for the engagement focuses on documenting the findings materialized from the workshop.

1.2.5     Use Case: Azure Data Backup

A client was concerned about five (5) different styles of attacks ranging from network to physical. Through inquiry-based testing, it was identified that one (1) of the client's software as a service (SaaS) products, which contained business critical data, was backed up every few days to an Azure SQL database, blobs, and containers.

For the client, this was a revelation. The Information Security team was unaware of this process, and therefore had no detection (telemetry or alerting), deflection (EDR), deterrence (controls or best practices), or digital forensics and incident response (DFIR) processes to cover the compromise, exfiltration, or destruction of this data.

This led to the gamification and threat modeling of a variety of attack scenarios, which ultimately drove building out the requirements to ensure this previously unidentified use case was addressed, and the necessary telemetry could be captured to provide coverage of this significant security gap. 

These findings were then prioritized against the overall initial attack scenarios that the team was presented with and were addressed according to that priority.

1.3      Defense Validation (DV):

TrustedSec's Defense Validation Assessment is a collection of assessments broken up into multiple phases that quantify the effectiveness of defensive controls implemented by an organization. Broadly speaking, a Defense Validation Assessment measures the existing implementation of an organization's security policy.

The phases are:

1.3.1     Detection and Alerting Interview

This scheduled call identifies the primary goals of the organization and how TrustedSec can provide the most value toward improving the security posture of the environment. TrustedSec specifically identifies:

• Primary areas of concern
• High-priority systems
• Business-critical data
• Current and historic TTPs

1.3.2     Security Posture Assessment

Utilizing information from previous assessments (e.g., External/Internal Penetration Test, Adversarial Attack Simulation, Adversarial Detection & Countermeasures), the past and present security posture of the organization is evaluated by identifying the defensive controls around the following abilities:

1.3.3     SIEM Configuration Review

When organizations invest money and resources in detection capabilities, tools should be leveraged to the greatest extent possible. To utilize the SIEM more efficiently, a preliminary audit is conducted to investigate whether the organization’s SIEM has effective visibility and high-fidelity detections that decrease the threat to the organization while maximizing process efficiency and reducing alert fatigue for the analyst.

This phase focuses on leveraging and maximizing capabilities for technologies that directly support detection, Incident Response, and Threat Hunting efforts.

TrustedSec may conduct the following exercises to determine SIEM efficacy and efficiency:

• Review SIEM log ingestion levels
• Log sources
• Log source utilization
• Event types
• Events per second (EPS)

TrustedSec also:

• Identifies visibility gaps/blind spots
• Inspects excessive alerts that might saturate analyst backlog and cause alert fatigue
• Provides SIEM tuning and improvement strategies based on the organization’s acceptable level of exposure
• Provides recommendations for changes to ingestion practices to increase coverage of the MITRE ATT&CK framework

1.3.4     Framework Coverage

The MITRE ATT&CK framework is the preferred matrix for TrustedSec's Adversarial Detection & Countermeasures assessment as well as the security industry as a whole. For that reason, a preliminary assessment of the organization's SIEM coverage of logs ingested are mapped to the ATT&CK framework and are used to evaluate the maturity of the organization's detection coverage and overall security posture.

2    So You're Ready for a Purple Team

In general, TrustedSec follows the below methodology for every Purple Team engagement:

However, for the sake of this post, we will only address the Planning phase of the TrustedSec Purple Team methodology.

2.1.1     Access & Requirements

The Planning phase begins with establishing Access & Requirements, which primes the engagement for success and mitigates any future potential roadblocks that might impede progress during the Execution phase.

2.1.2     Defensive Information Gathering

Defensive Information Gathering utilizes key attributes like EDR, SIEM, and logging tools to help scope and identify engagement direction. These attributes are also used to provide strategic recommendations to fine-tune existing security logging infrastructure. Key identifications during this process are used to develop and generate the simulation plan for the engagement.

2.1.3     Pre-Posture 3D Matrix

The Pre-Posture 3D Matrix gives a list of attacks that will be executed throughout the engagement. This matrix helps establish a baseline of the environment before execution and evaluates the ability to detect, deflect, and deter attacks during the Execution phase. Over the Execution phase, this matrix is updated to accurately assess the defensive capabilities, and this data is reflected within a Post-Posture 3D Security Matrix.

Ultimately, the client will be able to use the provided matrices to create a complete picture of the direct changes in security posture that are achieved throughout the engagement.

3    Additional Recommendations

While TrustedSec’s client onboarding process and planning phase covers a few of the items covered below, we also wanted to call out some additional specific recommendations that we’ve found to be helpful in ensuring the success of your Purple Team engagement.

3.1.1     The Right People

Whether it’s your first Purple Team or just one of many, one of the most important things we’ve observed is the appropriate allocation of (human) resources available during the engagement. Specifically, the most value will come from incorporating any resources that might interact with or have knowledge of in-house solutions or security functions as well as the ability to troubleshoot and resolve issues on the fly.

This will differ per organization, but the resources needed at a minimum are:

• The SIEM administrator and/or somebody with SIEM-specific knowledge that includes internal detection building processes and procedures should be involved. If access is not given to TrustedSec, someone with the ability to make SIEM modifications and to create and edit detections saved in the SIEM should be present.
• IT personnel with the ability to troubleshoot issues with AD accounts, firewall blocks, and workstation access (e.g., RDP, EDR)
  • If this person cannot be directly on the call, a representative should be appointed who can quickly contact the appropriate administrators.

People who may be beneficial but are not required to be available are:

• SOC Analysts/Defensive Security Team
• Internal Red Team (if applicable)
• Other IT support staff who may have security-related functions or be able to assist with other troubleshooting issues

3.1.2     Logging Setup

One of the most important components of a TrustedSec Purple Team engagement is ensuring adequate logging is in place across the environment. In general, we recommend following best practices for the following sources as well as any additional logging sources relevant to the devices present.

Windows Auditing:

• https://learn.microsoft.com/en-us/windows-server/identity/ad-ds/plan/security-best-practices/audit-policy-recommendations

Sysmon:

• https://github.com/trustedsec/SysmonCommunityGuide
• https://www.youtube.com/watch?v=kESndPO5Fig&list=PLk-dPXV5k8SG26OTeiiF3EIEoK4ignai7
• https://github.com/SwiftOnSecurity/sysmon-config
• https://github.com/olafhartong/sysmon-modular
• https://github.com/ion-storm/sysmon-config
• https://github.com/Neo23x0/sysmon-config

Carlos Perez (@Carlos_Perez) has also provided a wealth of information on Sysmon via YouTube.

Third-Party Security Products (follow specific product best practices):

• AV/EDR
• UBEA
• Web Proxy Logs
• Email Logs

Linux:

• Follow best practices provided by the utilized Linux Distro

Cloud Services:

Ensure you are ingesting the recommended logs to your SIEM via provider standards (e.g., CloudTrail Logs for AWS)

• https://docs.aws.amazon.com/whitepapers/latest/aws-security-incident-response-guide/logging-and-events.html
• https://cloud.google.com/security/best-practices
• https://learn.microsoft.com/en-us/azure/azure-monitor/best-practices-logs

Mac Logging:

• https://www.iansresearch.com/resources/all-blogs/post/security-blog/2021/04/29/best-practices-for-macos-logging-monitoring

As most SIEM administrators already know, logging is not a 'set it and forget it' endeavor. Review your SIEM before the engagement to ensure that no changes have resulted in missing logs or broken logging policies. TrustedSec frequently encounters missing logs or incorrect parsing that can, in some cases, prevent adequate detections from being built. While we don’t expect all issues to be resolved prior to the engagement, a quick check to ensure logging ingestion is as expected can go a long way to save time troubleshooting later.

3.1.3     Understanding Your Environment

Another valuable asset that TrustedSec has identified is when the client has a detailed understanding of their organization, such as where visibility may be lacking or where specific misconfigurations or vulnerabilities may be present.

This not only gives TrustedSec context for the environment but can provide direction and focus for customizing the engagement to client-specific needs.

3.1.4     When to Involve Third-Party Service Providers (or Internal Security Providers)

Occasionally, we get asked by clients whether they should inform their MSSP or other third-party service providers of the engagement before starting. Realistically, that depends on if the client wants to test general security response in addition to executing attacks and building detections.

As a general rule, purple teams ARE NOT red team engagements, nor are they traditional threat hunting or incident response exercises (read more on what to expect from a TrustedSec purple team engagement here). Thus, we will typically recommend providers be informed in order to prevent resources (e.g., user accounts, company-provided workstations) used by TrustedSec during the engagement from being quarantined or blocked, which can impede the progress of the engagement. This also applies to internal resources who are not present on the call.

3.1.5     Understand Your Existing Alert Structure

We also recommend having a solid understanding of your existing alert structure, including at least cursory knowledge of the following:

• What alerts exist already within your SIEM
• Preferred method of alert notification (e.g., in a dashboard or sent to an email distribution list)
• Where existing alerting gaps exist

Because TrustedSec custom-builds detections for the specific client environments where we operate, we want to track and notate which alerts are already working in your environment. This can save time in the long run, allowing TrustedSec to close gaps rather than 'fixing' a detection that is already functional.

Alternatively, we can also identify if a given alert is not functional and either build a better alert or improve what already exists.

4    Conclusion

While not all inclusive, we hope this list of recommendations improves understanding and illustrates how to maximize value when preparing for a Purple Team engagement.