Skip to Main Content
April 11, 2023

On the Road to Detection Engineering

Written by Leo Bastidas
Career Development Incident Response Penetration Testing Purple Team Adversarial Detection & Countermeasures

Introduction

People have asked numerous times on Twitter, LinkedIn, Discord, and Slack, “Leo, how do I get into Detection Engineering?” In this blog, I will highlight my unique experience, some learning resources you might want to get your hands on (all free or low cost), and extras that have helped me overall.

I'm currently a Senior Detection Engineer at TrustedSec. I did some cool Information Operations stuff in the Army, along with digging trenches for LAN cables. When I finally transitioned out of the military in 2016, I did Cloud Incident Response, Threat Hunting, and joined TrustedSec in 2019 as a Senior Incident Response Consultant. I was actually fortunate enough to make Principal Incident Response Consultant but jumped at the chance of joining Ben Mauch and Andrew Schwartz on the Tactical Awareness and Countermeasures (TAC) team! This is from where I'm writing to you today.

“So, Leo, what does a typical day look like?” Plenty of PowerPoints Just kidding. No slides, I'm excited to report. A typical day when I'm not on an engagement is researching cool threats pending any client or internal needs and developing detections on those techniques and procedures. When I'm not developing signatures, I'm working on, you guessed it, documentation. That never seems to go away, and for good reason. I cannot be the all-knowing curmudgeon that hoards all wisdom. We need to be able to point to a reference that teammates across the whole organization can look at.  We also need to be able to level set our expectations when it comes to our detection development lifecycle. Unfortunately, we will save the Detection Development Lifecycle in another installment of Leo's diary. Since this blog post focuses on getting into Detection Engineering, I am skipping numerous nuances in the interest of time. Let's just say that Detection Engineers should be able to conduct Internal/External Penetration tests and Purple Team engagements to stay up-to-date on the latest attack techniques here at TrustedSec.

Resources

Follow

Podcasts

Videos

Training

Links

Incident Response

OK, now that the formalities are out of the way, let's get into some skills that helped me get into detection engineering. I spent many years before I joined Information Operations for US Army Special Operations as a Cyber Defender. It wasn't until the final half of my career that I joined Special Operations and got to do cool Information Operation stuff. Anyway, I already had the foundation of Systems Administration along with Incident Response in a primarily Widows environment. Even though I touched Linux, I considered myself proficient on Windows. Make sure you are comfortable in a domain, be it Windows, Linux, or macOS. I'm a big proponent of knowing the fundamentals of the operating system before becoming an Incident Responder. The Army deemed me fit to send me to class for CompTIA A+, Security+, and Network+. Even though I was voluntold to attend class, most of the training I did was self-study. I used YouTube and books like everyone else. I will link references at the bottom of each section for easy references. I went through training a long time ago, so my editions are most likely outdated, but I will attempt to post the updated versions. I have heard good things about the CySA+ from CompTIA based on military and non-military friends but have not gone through the material myself. Maybe this is something I will pursue in the future and blog about. My favorite study book that was not certification related for Incident Response would have to be Kevin Mandia's Incident Response & Computer Forensics. The material is older now, but the fundamentals have not changed since I started in 2009.

Resources

Books

Training

Links

Follow

System Administration

I touched on this before, but I believe knowing how to administrator a box will only make you a better defender. Believe it or not, I am actually in the new school of thought where I believe if you do not come from a prior life of sysadmin like I did, you can still become a good defender, but it certainly does help! I am fond of the Unleashed book series for Window Servers. There are multiple resources to deploy an Active Directory lab these days, so much so that it is almost hard not to. Either use one (1) of the major cloud providers, a smaller VPS provider, or even in your local homelab. Homelabs are outside the scope of this blog post, but one can sign up quickly on their local host using either Chris Long's Detection Lab or Johnny Johnson's Marvel Lab. Grab them from GitHub!

Resources

Books

Virtual Labs

Follow

Threat Hunting

While my route might have been different from most, I assisted in conceptualizing and creating the Threat Hunt service with my good buddy, Justin Vaicaro at TrustedSec, along with the guidance of Tyler Hudak and the rest of the TrustedSec leadership. Is there a lot of overlap when it comes to Threat Hunting and Detection Engineering? Well, what if I tell you that every organization is different and everyone does Threat Hunting and Detection in their own way, but the foundational steps stay the same? In an ideal world, Detection Engineering would actually feed into the Cyber Threat Intelligence team whom the Threat Hunt team would be under. This is not the case most of the time. Be familiar with the Threat Hunting loop. Shameless plug for our webinar: Threat Hunting Lessons You Won’t Learn From Guides and Whitepapers

Resources

Links

Follow

Books

Training

Offensive Mindset

Being proficient from a defensive perspective is all well and good, but you will have to get into the attacker's mindset to threat model your target detection environment. I was lucky enough to attend SANS 560 GPEN class in order to classroom learn in combination of studying my Incident Response cases while in the Army in order to have that offensive mindset, but there is no way anyone can recommend a 10,000-dollar class unless someone else is paying the bill. TCM Security by Heath Adams is the first that comes to mind. This is a great starting point, but there are others like Zero-Point Security, Sektor7, and Pentest Academy where you can take low-cost offensive training to get into that offensive mindset. When it comes to offensive books, I am a big fan of the Hacker's Playbook and the Metasploit book that our very own Dave Kennedy co-authored. The point being, learn how the adversary might think and act, which will help you with crafting hypotheses when it comes to building out your detection strategy.

Resources

Books

Training

Follow

Programming Language

There's no way around it; the Detection Development Lifecycle is incredibly similar to the Software Development Lifecycle. Sorry. That means pushing code to test, development, and production. Pick a language, it doesn't even matter which one. It could be Python, Java, Bash, PowerShell, or C. The point is, you will know how to automate processes and be able to contribute code to the detection pipeline. I'm a self-study person like countless others in our industry, so I will highlight some of my favorite books. I also suggest going to your local community college and jumping in on a programming class if you have the chance!

Resources

Courses

Books

Auxiliary

I will add a handful of other resources that have helped me through my career and really changed how I produce my work. When it comes to becoming a Detection Engineer, I am in the school of thought that it requires three (3) auxiliary or soft skills as well. Discipline, mental and physical health, and lastly, personal information management (PIM)/personal knowledge management (PKM). Why those three (3)? Well, I'm glad you asked. As a Detection Engineer, at least at TrustedSec, no one will be holding your hand. The expectations are that you will produce quality content with no supervision. Leadership is there to guide you, but it is up to you to conduct the research, validate the attack, and document everything to back up your research. Those were just some examples, but this is where discipline comes in.

In reality, discipline is simply another word for control or orderliness. It will be up to you to maintain your routine to achieve that flow state of where you are putting out cool research with all your processes from testing, validating, hypothesis building, strategy thinking and so forth. This is important because you do not want to miss a step. Not only does a misstep throw off your routine, but it also throws you off your practical research that you have already concluded. You will have to double back or worse, revalidate an important detail of your research that is necessary for your detection research output. In other words, you can lose valuable time revisiting on tasks you are already completed.

The second auxiliary tool I believe in is health. Health, left intentionally as a broad term since this is not a neuroscience blog post, will combine mental and physical health. Making sure your health is in order will assist with your focus and energy, which have a lot of benefits not only with your work output, but with your overall well-being as well. Things like making sure you stay hydrated, walking 30 minutes a day, mediating a modest amount a week and so forth. This really does affect the quality of work you produce.

Lastly, for knowledge management, I subscribe to the ideological principles of Zettelkasten. While the specifics are not relevant for this blog post, what is important is that you have a system that works for when it comes to information. Typically, there are usually three (3) stages that make up the acquisition of personal knowledge: the intake of the information, the analysis, or reflection of the data, and the output of the first two (2) steps. Of course, the steps highlighted are very broad, and that is intentional. What merits your attention is how you personally can intake the information and produce more specifically, insight for other people to read.

Resources

Books

Podcasts

I hope this blog post can get you what you are striving to achieve. I want to caution you that there is so much more in my journey to becoming a Detection Engineer that I could share. This is not an all-inclusive list and based solely on my unique experience, but I believe it could be a good starting point on your journey to becoming a Detection Engineer. While this guide may help you find the front door, it is up to you to take the next steps and make your way inside the building.