A Brief Look at Approaches to Logging and Pitfalls to Avoid

TL;DR

The Logger++ extension is a great tool for recording requests and responses across all of Burp Suite. However, it is important to ensure enough log entries are retained from the tools you expect and that logs are exported if you want to keep them (my preference is automatic export to CSV).

Detailed logging of command line tools is simple with tmux and the custom pipe-pane command:

pipe-pane -o 'exec bash -c "while IFS= read -r line; do printf  \"%%(%%Y%%m%%dT%%H%%M%%S%%z)T: %%s\n\" -1 \"\$line\"; done"\; exec cat >>./tmux-#S-#W-#I.log'

The Reason

"Last night at 2:24AM EST, the application did go down. Below are the exact times of the outage. Can we confirm if and when testing was being conducted last night?"

If you have ever received an email like that during an application security assessment, you will appreciate the need for accurate and detailed logging of penetration testing activities. Regardless of the cause of a service disruption (it was probably DNS), you can only benefit from being able to demonstrate with precision what you were doing and when throughout the engagement.

Besides attestation, there are other practical benefits to thorough logging. Additional screenshots can be retroactively taken or retaken, and steps to an attack can be recreated in detail. Fortunately for most web application assessments, this is not too difficult. This generally involves logging in two (2) basic areas: the proxy and the terminal.

The Proxy

Burp Suite

Burp Suite records requests through the browser in the Proxy HTTP History tab but does not capture requests generated from the likes of Intruder or Repeater. Repeater maintains a record of requests and responses in its own tab, but precise logging is dependent on Date header responses (assuming you received a response). Intruder attacks do not persist between Burp Suite sessions, and some extensions have no interfaces at all (looking at you, Param Miner). This is the reason for the popularity of Logger++. This extension is available in both community and professional versions of Burp Suite and allows "logging [of] requests and responses from all Burp Suite tools".

However, Logger++ in its default configuration may cause some problems. Log entries do not persist between Burp Suite sessions. So, if at some point during the engagement or reporting process you close the Burp Suite application, you will no longer have the logs you collected.

To avoid this problem, it is possible to export log entries to several formats: comma-separated values (CSV), JavaScript Object Notation (JSON), or straight to an ElasticSearch instance.

JSON: This is the obvious choice—every hip young person loves JavaScript Object Notation (JSON). However, the deal-breaker for this (in my humble opinion) is that there is no option to export logs in JSON format automatically with Logger++, meaning you will have to remember to export when you are ready to close Burp Suite. You would also have to bank on Burp Suite never crashing (YOLO). The second issue my lazy self has is there now needs to be a way to parse the JSON, which is less readily available. There also does not appear to be a way to specify which fields get exported, which may contribute to unnecessarily large exports.

ElasticSearch: This seems like an option that can provide a lot of utility, not only for storing the data but for post-processing as well. This requires some additional infrastructure, however, which is something I am trying to avoid because I am suspicious of complexity and I want something as basic as logging to always work. I did try this method, but to be honest, I do not know how to use ELK stack. If you do, and you have one set up, give it a shot and let me know what you think. There seem to be some other Burp Suite extensions that connect to ElasticSearch that might be worth checking out (https://github.com/portswigger/elastic-burp).

CSV: This is my preferred choice. The output is hideous if you try to view it as a spreadsheet, and it may create log files of such size that spreadsheet applications cannot process them. However, it is possible to automatically export log entries to CSV, making it less likely that data is lost. Automatic export is my main concern, and I can deal with any discomfort while extracting data later. But honestly, I do not have much difficulty parsing the CSV later with grep and sed to pull out what I need.

Most options allow exports to be restricted to specific fields, which may allow exports to remain as minimal as possible, while keeping the data verbose enough that questions like, "What were you doing at such-and-such time?" can still be answered.

At one point in the past, I remember a single password spray overwriting my Logger++ logs and finding out that the Maximum Log Entries value was set to something low, like 5,000. I cannot find any evidence this value was ever the default, and perhaps I misconfigured this myself. Regardless, the default value now appears to be 1 million, but double check to ensure these values are appropriate for your application.

One (1) other configuration I have had issues with in the past is In scope items only. I typically select this option to keep logging succinct and focused. However, there have been a few times that I failed to add assets to the scope within Burp Suite, and nothing was recorded. This is something to keep in mind.

When my father complained once about the pancakes on a father-son trip with church, my grandmother suggested he could cook them all the next year. Her point was that it may not be wise to critique things too harshly if you are not willing to help fix them. So, although I think Logger++ leaves some things to be desired, I am not yet ready to cook all the pancakes. It is a useful tool that I use every engagement, but I have learned a few painful things through trial and error. Hopefully, others can learn from my mistakes.

The Terminal

Although a lot of a dynamic application assessment takes place with a browser and HTTP proxy, there are still many times when command-line tools are necessary or maybe convenient. There are quite a few options to record commands and their output within terminals, and they each seem to have their own advantages and disadvantages. What works and what does not may hinge on the type of assessment you are conducting or the features you care about. I am writing from the perspective of web application assessments and have settled on approaches that I have found to be successful for me, but I would reiterate that these are certainly not the only or best options in all cases.

It is possible to use the script command within Unix systems to record a terminal session. This can be made more verbose when a terminal prompt has been modified to include additional information like the date and time. This is simple, native to Unix systems, and preserves ANSI sequences (https://en.wikipedia.org/wiki/ANSI_escape_code). Depending on what you have run in the terminal, printing the file with cat or sed will result in a nice output for later screenshots.

However, script can provide a timestamp within the prompt, but it fails to provide a timestamp for tool output, which could be significant for tools that run over time (e.g., dirsearch, Metasploit, wpscan, etc.). That may be splitting hairs in most cases, but I consider it a deficiency.

A bigger issue is that the timestamp that seems to demonstrate the time a command was run is actually the time the previous command finished, and a command prompt was returned. This can lead to inaccuracies.

This will not be so far off, in the case of, say, an Internal Penetration Test where the terminal is your primary tool. But in the case of web application assessments, I may use the terminal infrequently and rely primarily on Burp Suite, so this issue could very easily render terminal logs useless in terms of timestamp accuracy. I prefer a different approach.

NOTE ON ANSI SEQUENCES

ANSI Sequences can be a double-edged sword. They preserve the colors and formatting within the terminal, which can be useful later when trying to quickly understand tool output or capture screenshots. On the other hand, those sequences make an unreadable mess when they are not being interpreted.

Fortunately, they are easy to remove with

sed 's/\x1b\[[0-9;]*m//g' <log-file>

Tmux

But perhaps you are like me and enjoy the benefits of a terminal multiplexer like tmux. Multiplexers allow a variety of useful features like multiple terminal panes, persistence of sessions between connections, and session sharing. Search engine results for tmux logging generally returns two (2) options for logging tmux terminals: tmux logging plugin and pipe-pane.

The tmux logging module (https://github.com/tmux-plugins/tmux-logging) requires installation, although that is straight forward. ANSI sequences are not preserved, which may or may not be relevant in some situations, but the module otherwise works as expected. Similarly, the other common solution when using tmux is the pipe-pane command within the tmux session. By itself, it is simple, built-in, and does indeed preserve ANSI sequences.

Basic Usage:

pipe-pane -o 'cat >>~/tmux_output.#S:#I-#P'

This output is customizable to a degree with tmux aliases (https://man7.org/linux/man-pages/man1/tmux.1.html)

• #H – Hostname of local host
• #h – Hostname of local host (no domain name)
• #D – Unique pane ID
• #P – Index of pane
• #T – Title of pane
• #S – Name of session
• #F – Window flags
• #I – Index of window
• #W – Name of window

But ultimately, the same issues exist with the logging plugin and vanilla pipe-pane as do with the script command; timestamps are not included with each terminal line, which is less precise than I would like. For this reason, I have settled on an embellished pipe-pane command that offers some customization to get me what I want.

pipe-pane -o 'exec bash -c "while IFS= read -r line; do printf \"%%(%%Y%%m%%dT%%H%%M%%S%%z)T: %%s\n\" -1 \"\$line\"; done"\; exec cat >>./tmux-#S-#W-#I.log' \; display-message 'Started logging to ./tmux-#S-#W-#I.log'

Tmux commands can be entered or pasted with the tmux prefix (ctrl+b by default) and a colon Ctrl+b :

This produces a log file that looks like this:

Customizing tmux configurations can be a complex process (Fact check: this is an understatement). But without going down the rabbit hole too far, I have added the following two (2) lines to the bottom of my ~/.tmux.conf

bind-key H pipe-pane -o 'exec bash -c "while IFS= read -r line; do printf \"%%(%%Y%%m%%dT%%H%%M%%S%%z)T: %%s\n\" -1 \"\$line\"; done"\; exec cat >>./tmux-#S-#W-#I.log' \; display-message 'Started logging to ./tmux-#S-#W-#I.log'

bind-key h pipe-pane \; display-message 'Ended logging to ./tmux-#S-#W-#I.log'

This allows me to fire up a new tmux session with a client name, ctrl+b H, and I am ready to move out and draw fire. Terminating logging is as simple as ctrl+b h.

This way, when I forget how a tool works, I can terminate logging, print the options, fiddle with it until I get it right, then reinstate logging so there is no record that I did not know what I was doing. Just kidding, I would not do that.

Screen

Screen is another popular multiplexer, although I do not use it and do not have any experience logging terminal output with it. However, if you do look for a terminal logging solution with screen, I would suggest evaluating those solutions with similar criteria based on what is important to you:

• Is every terminal line timestamped? Do you want that?
• Are ANSI Sequences preserved? Do you want them to be?
• Is it reliable? (I get paranoid that logging will quit halfway through an engagement)
• Is it easy to implement in your workflow?

Conclusion

Take these recommendations with a grain of salt and apply them as you are able within your workflow. I recognize that I have made some critiques (complaints? Sorry.) that may not apply to you. These are simply the lessons I learned the hard way or things that bother me, and your mileage may vary. Good luck, and have fun.