SSH-nanigans: Busting Open the Mainframe's Iron Fortress through Unix

SSH-nanigans: Busting Open the Mainframe's Iron Fortress through Unix

Speakers: Philip Young

Conference: DEF CON 33

YouTube: https://www.youtube.com/watch?v=J3f9axtJsCw

Slides: https://media.defcon.org/DEF%20CON%2033/DEF%20CON%2033%20presentations/Philip%20Young%20-%20SSH-nanigans%20Busting%20Open%20the%20Mainframes%20Iron%20Fortress%20through%20Unix.pdf

Overview

IBM mainframes (z/OS systems) are among the most security-hardened computing platforms on earth. They run global financial transactions, process enormous volumes of government and healthcare data, and are defended by a layered security architecture that includes RACF (Resource Access Control Facility), System Authorization Facility (SAF), and a constellation of host-based security controls that have been refined over decades. Yet at DEF CON 33, Philip Young — Director of Mainframe Penetration Testing at NetSpy and one of the most experienced mainframe hackers in the world — demonstrated that the Unix System Services (USS) subsystem, the POSIX-compatible Unix environment that lives inside z/OS, provides a well-lit path for attackers who can reach SSH on a mainframe. The Unix layer, added to mainframes to provide compatibility with modern software and tooling, introduces attack vectors that the traditional z/OS security model was never designed to address — and which organizations frequently leave open.

The talk synthesizes a decade of professional mainframe penetration testing experience into a focused demonstration of how SSH access to the z/OS Unix subsystem can be leveraged to escalate privileges across the entire mainframe stack — from Unix to native z/OS datasets, to JES (Job Entry Subsystem), to eventually full system control.

Background

▶ Watch: mainly from our next speaker. (0:03)

The Mainframe's Role and Its Security Reputation

IBM mainframes — the z/Architecture family — are not legacy systems in the colloquial sense of old and obsolete. They are the world's most reliable, highest-throughput transaction processing platforms, running the backbone of global banking (processing over 30 billion transactions per day across all mainframe deployments), insurance, government benefit systems, and healthcare infrastructure. The security model around mainframes evolved over 60 years of production operations, and the platforms have a justified reputation for being extraordinarily difficult to compromise.

Classic mainframe security assumes adversaries are internal users on 3270 terminals, authenticated through RACF and operating within strictly controlled JCL (Job Control Language) workflows. The access model is explicit-permit: unless you have been granted specific resource access, you cannot touch anything. This contrasts sharply with traditional Unix systems, where many operations are permitted by default unless explicitly restricted.

Unix System Services (USS)

IBM added a POSIX-compliant Unix environment to z/OS called Unix System Services (USS) to allow mainframe shops to run software compiled for Unix environments — scripting tools, web servers, Java middleware, databases, and modern DevOps tooling — without losing the mainframe's performance and reliability advantages. USS provides a hierarchical file system (HFS/ZFS), standard POSIX APIs, shell access (via sh, bash, zsh), and crucially, an SSH daemon (OpenSSH ported to z/OS).

The SSH daemon allows users to log into the mainframe's Unix environment from a standard SSH client. From the USS shell, a user has access to a standard Unix environment — but they are still running on a z/OS system, with access to z/OS datasets (sequential files, PDS libraries), JES spools, started tasks (system services), and, if privileges can be escalated, the entire z/OS security and execution environment.

Philip Young and the Mainframe Security Community

Philip Young (aka Soldier of FORTRAN) has been hacking mainframes since the 1990s, initially as a self-described "mainframe security enthusiast" before transitioning to professional penetration testing. As Director of Mainframe Pen Testing at NetSpy, he has spent a decade conducting authorized security assessments of mainframe environments for banks, insurance companies, and government agencies. He is one of a tiny community of security researchers with both the access and expertise to do credible mainframe offensive security work, having spoken at DEF CON multiple times on mainframe topics.

The talk was intended as a co-presentation but his co-speaker was unable to attend; Young delivered it solo, with the co-speaker's contributions acknowledged throughout.

Key Findings

▶ Watch: So now that we know what's wrong, right? (21:11)

• SSH access to z/OS USS provides a significantly expanded attack surface compared to traditional 3270 terminal access, because the Unix environment enables standard penetration testing tooling and techniques that were never considered when mainframe security controls were designed.

• Privilege escalation from USS to native z/OS authority is achievable through multiple paths, including SUID/SGID binary abuse in the USS environment, z/OS-specific privilege APIs accessible from the Unix shell, and misconfiguration of the omvs (Unix-to-RACF) profile mapping.

• The BPX.SUPERUSER facility is a critical escalation target. Access to BPX.SUPERUSER is the z/OS equivalent of root on Unix: it grants full control over the USS environment, and from there, access to z/OS resources is dramatically expanded.

• JCL injection via the Unix-to-JES interface is a practical exploitation path. From a USS shell, an attacker with appropriate (but not maximum) permissions can submit JES jobs that execute z/OS programs with higher authority than the attacker's native RACF profile allows.

• Configuration errors in RACF profiles for USS are endemic. Organizations frequently grant broadly permissive omvs UID assignments (particularly UID(0), the superuser), misconfigure BPX facility access, and leave SUID binaries in place that would not be acceptable in a hardened Linux environment.

• Mainframe security teams often have limited Unix security expertise. z/OS security administrators who are expert in RACF and JCL may have limited familiarity with Unix privilege escalation, SUID/SGID abuse, symlink attacks, and other standard Unix attack techniques — creating a skills gap that attackers can exploit.

Technical Deep Dive

▶ Watch: C is not a memory safe programming language. (32:59)

The z/OS Unix Security Model: OMVS and RACF Integration

In z/OS, every USS user must have an OMVS segment in their RACF profile. The OMVS segment contains Unix UID and GID assignments, the home directory path, and the default shell. The UID/GID namespace in USS maps to Unix-style permission checking — file ownership and permission bits are evaluated using these Unix IDs.

The critical mapping: a RACF user with UID(0) in their OMVS segment is the USS superuser. UID(0) bypasses most USS permission checks — exactly as root(0) bypasses permission checks on Linux. Organizations that assign UID(0) broadly (for convenience, because the mainframe RACF team didn't understand the implications) create a straightforward path from any such user's account to full USS control.

BPX.SUPERUSER: The Ultimate z/OS Unix Privilege

The BPX.SUPERUSER RACF facility class resource is the formal mechanism for granting superuser authority to z/OS users who are not UID(0). A user with READ access to BPX.SUPERUSER can elevate to superuser using the setuid(0) system call — they can become UID(0) on demand. This is powerful but also frequently granted carelessly, to application service accounts and development users who need specific Unix capabilities but whose access was granted without full understanding of what superuser access entails.

Young's penetration testing experience has found BPX.SUPERUSER access granted to service accounts used by application middleware (Java application servers, web servers) running in USS. Compromising such a service account — through a web application vulnerability, credential theft, or other means — immediately provides superuser authority over the USS environment.

SUID/SGID Abuse in USS

Standard Unix SUID/SGID (Set User ID / Set Group ID) permission bits allow executables to run with the permissions of their owner rather than the invoking user. On Linux, SUID root binaries are a well-understood privilege escalation surface and modern Linux distributions have very few of them. On z/OS USS, the situation is less controlled: IBM ships USS with some SUID binaries, and third-party software installed in USS frequently introduces additional SUID binaries without adequate review.

Young documents specific USS SUID binary configurations that can be abused for privilege escalation, using techniques directly analogous to classical Unix SUID exploitation. The z/OS USS kernel honors SUID semantics identically to Linux in most cases, meaning decades of Unix privilege escalation research applies directly to mainframe USS environments.

From USS Superuser to z/OS Dataset Access

USS superuser authority (UID(0) or BPX.SUPERUSER) provides control over the Unix filesystem hierarchy, but z/OS native datasets (the traditional sequential files, PDS libraries, and VSAM files that hold mainframe applications and system software) are a separate namespace. The bridge between USS and native datasets is the z/OS file system mounting mechanism: native datasets can be mounted as paths in the USS hierarchical file system using MOUNT JCL or dynamically.

Additionally, from a USS shell, a user can invoke z/OS services directly using:

• tso command: Invokes TSO (Time Sharing Option) commands, which can access and manipulate z/OS datasets
• uss2mvs interfaces: Shell utilities that provide direct access to z/OS MVS APIs
• submit command: Submits JES batch jobs from the USS shell, allowing execution of JCL with whatever RACF authority the submitting user holds

With superuser authority in USS, a user can often read or modify z/OS datasets that their RACF profile would not permit under direct access, because the USS filesystem mount provides a path around the RACF dataset access control when the USS layer is not properly sandboxed from the MVS layer.

JCL Injection and Privilege Escalation via JES

Job Entry Subsystem (JES) is the z/OS batch job scheduler. JCL submitted to JES executes programs with the RACF authority of the submitting user plus any authority granted by the JCL itself (including PARM data, STEPLIB overrides, and program-controlled APF authorization). Young demonstrates a JCL injection scenario:

1. Attacker has USS shell access with limited RACF authority.
2. A shared JCL template or include library in USS HFS (or a poorly protected native dataset mounted in USS) is writable by the attacker.
3. The attacker modifies the shared JCL to include additional JCL statements that grant RACF authority to their own user ID, submit unauthorized programs, or extract sensitive data from protected datasets.
4. When the legitimate job using the template runs (perhaps a scheduled batch job), the injected JCL executes with the legitimate job's elevated RACF authority.

This attack requires no direct RACF privilege escalation — it exploits the file system access controls in USS to inject into a batch workflow that runs with higher authority.

Lateral Movement from Mainframe to Enterprise

A fully compromised mainframe is not an isolated asset. Modern mainframes are connected to enterprise networks for application integration, web services, and database connectivity. From a compromised USS environment, Young demonstrates:

• Network pivoting: The USS TCP/IP stack provides full network access. A shell on the mainframe can reach other internal systems.
• Credential extraction: The mainframe may hold RACF database exports, application configuration files with database credentials, and LDAP/Active Directory integration configurations that expose enterprise credentials.
• Data exfiltration: The mainframe's datasets may contain high-value data (financial transaction records, healthcare records, government databases) that can be extracted through the USS filesystem.

Demo / Proof of Concept

▶ Watch: so that you can generate something that looks like this. (34:54)

The DEF CON 33 demonstration uses a test mainframe environment configured to replicate common production misconfigurations:

1. Initial access via SSH: Connecting to the z/OS SSH daemon as a low-privilege user with a standard SSH client.
2. OMVS UID(0) discovery: Using USS commands to check the effective UID (trivially, id command), confirming superuser status due to UID(0) misconfiguration in the RACF OMVS profile.
3. Native dataset access: Using TSO commands from the USS shell to browse and read z/OS datasets that the user's RACF profile would not permit through a 3270 terminal session.
4. SUID escalation path: Demonstrating a SUID binary in the USS environment that can be abused to escalate to superuser on a system where the initial account did not have UID(0).
5. JCL submission: Submitting a JES batch job from the USS shell that runs with elevated authority, demonstrating the Unix-to-JES escalation path.

The demonstration is grounded in real configurations Young has encountered in production mainframe penetration tests, illustrating that these are not theoretical vulnerabilities but active findings in live environments.

Defensive Implications

▶ Watch: a local file included and then I could read a whole bunch of files. (36:55)

For mainframe security administrators:

• Audit RACF OMVS segment UID assignments across all user profiles. UID(0) should be assigned only to specifically designated superuser accounts, not to service accounts or general users. Any account with UID(0) has full USS superuser authority.
• Restrict access to the BPX.SUPERUSER facility class in RACF to the minimum set of accounts that require superuser capability.
• Enumerate all SUID/SGID binaries in the USS HFS/ZFS filesystem and audit each one for privilege escalation potential. Remove SUID bits from any binaries not explicitly required to have them.
• Review USS filesystem permissions — particularly for HFS directories and datasets mounted in USS — to ensure write access controls match the sensitivity of the data and the trust level of authorized users.
• Restrict SSH access to z/OS USS: consider disabling SSH entirely if it is not required, or restricting SSH login to specific users and source IP addresses using RACF security labels and TCP/IP filtering.

For enterprise security teams:

• Include mainframes in the scope of penetration tests. Many organizations have never had their mainframe SSH interface professionally tested from an offensive Unix perspective.
• Train z/OS security administrators in Unix privilege escalation concepts, as these are directly applicable to the USS environment.
• Include mainframe SSH access in SIEM monitoring and anomaly detection. Unexpected logins, unusual SSH source IPs, and atypical command execution patterns are actionable signals.

For the mainframe vendor and ecosystem:

• IBM USS ships with some SUID binaries that may not be necessary for most production environments. Guidance on reducing the SUID attack surface in USS post-installation would reduce default exposure.

Key Takeaways

1. SSH access to z/OS Unix System Services provides a modern attack surface that mainframe security controls were not designed to address, enabling standard Unix privilege escalation techniques against what is typically regarded as the most secure computing platform in enterprise use.
2. RACF OMVS UID(0) misconfiguration — granting superuser authority to accounts that should not have it — is endemic and immediately provides full USS control to an attacker.
3. BPX.SUPERUSER facility access is an equally powerful but less obvious privilege escalation target, frequently granted to service accounts running middleware in USS.
4. SUID/SGID binary abuse and JCL injection via the USS-to-JES interface provide additional escalation paths from limited USS access to z/OS-level authority.
5. A decade of professional mainframe penetration testing confirms that these misconfigurations are common in production environments, not theoretical edge cases — the mainframe's iron fortress has a Unix-shaped door that many organizations have left unlocked.

About the Speaker(s)

▶ Watch: but no one should just have that as a blanket access, right? (38:45)

Philip Young (aka Soldier of FORTRAN) is the Director of Mainframe Penetration Testing at NetSpy and one of a very small community of professional mainframe security researchers. He began as a mainframe security enthusiast in the 1990s and has spent the past decade conducting professional penetration tests of z/OS environments for major financial institutions, government agencies, and insurance companies. He is the leading public voice on mainframe offensive security, with multiple DEF CON appearances on mainframe exploitation topics, and a passionate advocate for treating mainframes as a first-class target in enterprise security programs rather than an afterthought. He acknowledges feeling like an outsider at DEF CON — and encourages others who feel the same way to know that is the universal experience.