Websphere, IIOP and Spectrum Scale GUI

Do the unauthenticated remote code execution vulnerabilities reported in Websphere affect IBM Spectrum Scale (GPFS)?

TL;DR – we do not think that the recent critical vulnerabilities in Websphere Application Server affect Spectrum Scale GUI. If the following command returns nothing then we believe you are fine, if it does return something then you might not be:

grep -i "ejbRemote\|iiopEndpoint" /opt/ibm/wlp/usr/servers/gpfsgui/server.xml

The detail – CVE-2020-4449 + CVE-2020-4450

On 2nd May 2020 the ZDI reported two critical vulnerabilities (as well as a medium risk information disclosure vulnerability) which affect IBM Websphere Application Server (WAS). We know that Spectrum Scale uses WAS, in fact a few days ago IBM dropped around a security bulletin informing us that Spectrum Scale was affected by CVE-2019-4720, a denial of service vulnerability in WAS. Awkwardly this vulnerability was disclosed by IBM on 17th February 2020, however, it wasn’t until 2nd May that IBM dropped around a bulletin informing Spectrum Scale users that it also affects Spectrum Scale instances. Those time frames are clearly not acceptable when vulnerabilities allow unauthenticated remote code execution as root, so naturally at HPCsec we wanted to find out whether these vulnerabilities affect Spectrum Scale users (or more specifically, Spectrum Scale GUI users).

Whilst this mini investigation is specific to Spectrum Scale parts will be relevant more generally. It is worth mentioning that this work was conducted within test labs that we have access to and so may not reflect your deployment, so do check your own environment.

These are the vulnerabilities that this relates to:

CVE-2020-4449 – IBM WebSphere Application Server IIOP Deserialization of Untrusted Data Information Disclosure Vulnerability
CVE-2020-4450 – IBM WebSphere Application Server IIOP Deserialization of Untrusted Data Remote Code Execution Vulnerability
CVE-2020-4448 – IBM WebSphere Application Server UploadFileArgument Deserialization of Untrusted Data Remote Code Execution Vulnerability

We’re ignoring #3 for now as this actually relates to “Websphere Application Server Network Deployment”, we’ll cover that later.

Spectrum Scale GUI is a web interface for accessing information relevant to your Spectrum Scale deployment. If you are running Spectrum Scale GUI then you should not allow this to be accessible by anyone other than administrators of your cluster. The vast majority of critical vulnerabilities within HPC environments can be mitigated with the use of appropriate network segmentation, this vulnerability is no exception.

The first thing to note is that in the configurations we’ve looked at make use of IBM WebSphere Application Server Liberty which has different version numbering to standard WAS. On our 5.0.3 Spectrum Scale build we have version 19.0.0.1 of WAS Liberty Server (WLS) in use (which I’d assume is mostly the Websphere 9.0 branch). You’ll find the server configuration here:

/opt/ibm/wlp/usr/servers/gpfsgui/server.xml

(the “wlp” is because WAS used to be referred to as the websphere liberty profile for WAS when it was introduced)

The vulnerability we are checking for affects the Internet Inter-ORB Protocol (IIOP) component. If it is enabled then we’d expect to see it configured with by enabling the ejbRmote feature in your server.xml file, e.g:

<featureManager>
    <feature>ejbRemote-3.2</feature>
</featureManager>

You may also have configured specific IIOP endpoints

<iiopEndpoint id="defaultIiopEndpoint" iiopPort="2809">
     <iiopsOptions iiopsPort="9402" sslRef="defaultSSLConfig"/>
</iiopEndpoint>

A grep for “ejbremote” and “iiopEndpoint” should do the trick:

grep -i ejbRemote server.xml

We’ve not seen either of these settings configured in any of the Spectrum Scale environments that we’ve looked at whilst compiling this write-up. But if you do have either of these settings in your environment then there is the possibility that you have IIOP enabled and so may be vulnerable. If you have multiple servers you may want to check for this in all of your server configurations. If you do find IIOP enabled then we’d love to hear from you.

If there was no indication in your configs that IIOP is enabled but, like us, you’re a touch paranoid then you may also want to check for IIOP related ports. The default ports are 2809/tcp and 9403/tcp but the security researcher named in the advisory (tint0) has also highlighted ports 9100/tcp and 9402/tcp in his prior research so those are worth checking too. The following should do the job:

netstat -ntlp | grep <port-number>

If neither of the checks above have returned anything on your Spectrum Scale GUI server then it’s fair to assume that you do not have IIOP enabled. It is highly likely that Websphere Liberty is affected by this issue, but, when it comes to spectrum scale the affected features are not in use, so even if though the software might be affected it is not exploitable.

We’ll obviously keep a close eye on this, in particular for any comment from IBM. If you’re aware of something that we’ve missed then do get in touch.

CVE-2020-4448 – Vulnerability #3

…We said we’d come back to vulnerability #3 (CVE-2020-4448):

A similarly critical vulnerability was also identified in Websphere Application Server Network Deployment: https://www.ibm.com/support/pages/node/6220336. This is entitled “IBM WebSphere UploadFileArgument Deserialization of Untrusted Data Remote Code Execution Vulnerability”. This looks very related to CVE-2019-8352 and CVE-2019-4279 for which there is exploit code in the wild.

It’s pretty unlikely that you’re running WASND within an HPC cluster, but if you are then you ought to be fixing this one or at the very least limiting access. Port 11006/tcp is primarily used, but you should also look out for 11002 and 11004 too.