7 Steps to Secure a SaaS Platform – the Worlds Biggest Companies Trust

7 Steps to Secure a SaaS Platform – the Worlds Biggest Companies Trust

Speakers: Joel Balog, Vice President of DevSecOps, Liferay

Conference: Hacktivity

YouTube: https://www.youtube.com/watch?v=hPnaMSku6nI

Overview

In this insightful Hacktivity presentation, Joel Balog, Vice President of DevSecOps at Liferay, shares seven critical steps his organization took to secure its SaaS platform, a journey undertaken while transitioning from a traditional open-source software vendor to a cloud-native service provider. The talk offers a candid look into the real-world challenges faced by a company supporting a massive, highly extensible enterprise platform for some of the world's largest corporations and governments. Balog emphasizes that securing a SaaS platform goes beyond merely implementing security tools; it requires a deep organizational shift, robust processes, and a culture of security awareness.

Balog's presentation distinguishes itself by focusing on practical, hard-earned lessons that often go unmentioned by security vendors. He delves into both the technical intricacies and the crucial human elements of security, highlighting how Liferay navigated incidents, fostered executive buy-in, and built cross-departmental alliances. The talk is particularly relevant for organizations undergoing a similar digital transformation or those grappling with securing complex, legacy-rich environments in the cloud.

The core message revolves around proactive measures, continuous learning, and an acknowledgment that security is an ongoing, imperfect process. Balog's "seven steps" are not just a checklist, but a strategic framework for building resilience, from formalizing incident response to leveraging red teaming for cultural change and making informed risk-based decisions for product launches.

Background

▶ Watch: Announcing Liferay's new private bug bounty program (2:20)

Liferay, founded in 2000, began as an open-source portal project and has evolved into a comprehensive digital experience platform offering CMS, low-code capabilities, and complex workflows tailored for enterprise needs. With a substantial codebase – a 900-megabyte zipped bundle – and a history of high commit numbers on GitHub, Liferay’s platform is renowned for its extreme extensibility. While this flexibility is a key selling point for its global clientele, which includes giants like A1, European Space Agency, Vodafone, O2, Bosch, HP, and Airbus, it also presents significant security challenges, as customers can effectively run custom code within the system.

Joel Balog's journey into leading Liferay's security initiatives is rooted in a company-wide shift towards a SaaS model. Having spent 14 years with Liferay, leading its Hungarian office and Global Technical Support, he was tasked by the board to become the Vice President of DevSecOps. This strategic move was driven by the board's understanding of Conway's Law – that an organization's communication structure influences its product architecture – and a commitment to integrating security seriously from the outset of their SaaS and SRE (Site Reliability Engineering) practices. Initially, Balog, coming from a general management and technical support background, received intensive training in security. To establish a baseline for their security posture, Liferay utilized the OWASP DevSecOps Maturity Model. However, applying this model proved challenging due to the organization's age, the complexity of its open-source environment, and the need to support software versions up to seven years old. This initial assessment highlighted the need for a pragmatic, prioritized approach to security transformation.

Key Findings

▶ Watch: How a fake invoice fishing campaign gained board buy-in (10:30)

Liferay's journey to secure its SaaS platform yielded several critical insights, presented as seven practical steps:

1. Formalized Incident Management is Paramount (Step 1): A real-world remote code execution (RCE) incident on Liferay's build system underscored the urgent need for well-defined and tested incident management processes. By formalizing their approach, drawing inspiration from firefighting protocols, Liferay was able to contain the incident swiftly, minimizing the financial impact of engineering downtime (estimated at $22,000 per hour).

1. Red Teaming Drives Cultural Transformation (Step 3): Traditional security training often struggles with engagement. Liferay discovered that a carefully executed red teaming exercise, specifically a simulated phishing campaign targeting their finance department with a fraudulent invoice, dramatically shifted executive and employee perception of security risks. This visceral demonstration of vulnerability led to an impressive 93% voluntary completion rate for subsequent security training, proving far more effective than mandatory directives.

1. Cross-Departmental Collaboration is Essential (Step 4): Security initiatives gain significant traction when aligned with the objectives of other departments. Liferay found "win-win" scenarios by collaborating with privacy, HR, and sales teams. For instance, advocating for privacy standards like GDPR and LGPD garnered legal support for security projects, while involving HR early in compliance discussions streamlined processes and enabled global policy enforcement. Sales benefited from certifications like ISO and SOC2, which reduced the burden of extensive security questionnaires from large clients.

1. Unsolved Technical Problems Persist (Step 5): Despite leveraging leading cloud providers and advanced tools like AI-based DDoS detection, Liferay encountered significant gaps in "solved problems." Sophisticated DDoS attacks with constantly changing IPs evaded detection, and securing backups against ransomware, particularly from compromised administrative accounts, revealed a lack of comprehensive solutions even from major cloud vendors, highlighting the need for custom, layered defenses.

1. Risk-Based Approach for Go-Live Decisions (Step 6): Acknowledging that perfect security is unattainable, Liferay adopted a risk-based approach to decide when to go live with new services. By involving the entire company in identifying risks and then categorizing them based on probability and an exponentially scaled severity, they could make conscious, data-driven decisions with the board about which risks to accept, ensuring transparency and shared accountability.

1. Global Talent Acquisition and Team Empowerment (Step 7): Building a strong security team requires looking beyond geographical boundaries and fostering an environment that leverages individual strengths. Liferay emphasized the importance of a diverse, globally distributed team and cultivating a mindset where everyone in the company understands their role in security, transforming the entire organization into a collective security force.

Technical Deep Dive

▶ Watch: Unsolved ransomware backup challenges with leading cloud providers (19:30)

Liferay's journey into securing its SaaS platform involved navigating several complex technical and organizational challenges, leading to the development of robust strategies.

The first critical technical lesson came from a Remote Code Execution (RCE) incident on Liferay's build systems (Step 1). A security researcher successfully exploited a vulnerability, allowing code execution during a build process. The immediate concern was the potential impact on 500 technical employees. Liferay's response was swift and decisive: they temporarily shut down engineering operations, a costly measure estimated at $22,000 per hour. While investigating, they leveraged EDR (Endpoint Detection and Response) logs but initially found no clear clues. The quick action involved moving engineers to use local caches to isolate the problem. It was later discovered that an engineer had copied parts of the build system but inadvertently omitted critical configuration files that ensured fetching from the correct, secure repositories. This incident, while ultimately contained, highlighted the immense value of Liferay's newly formalized Incident Management processes, which were modeled after firefighting principles, emphasizing clear communication and hourly updates on "conditions, actions needed, and reports." This structure enabled efficient communication across the company and a rapid return to operation within five hours.

A major breakthrough in fostering a security-conscious culture came through a daring red teaming exercise (Step 3). To convince the board and employees of the tangible risks, Liferay orchestrated a sophisticated phishing campaign. Leveraging Joel Balog's prior role as General Manager of the Hungarian office, they crafted an email to their parent company's finance department, falsely claiming a bank account change for an upcoming payment. This email included a convincing, fake invoice for €300,000. The campaign successfully tricked the finance department into accepting the change and nearly transferring funds. The stakes became even higher when a real invoice of €600,000 was also due, meaning a total of €900,000 was almost diverted. The critical vulnerability exposed was the lack of mandatory Multi-Factor Authentication (MFA) for finance accounts. This real-world demonstration of financial risk completely changed the board's perspective, leading to immediate approval for enforcing MFA and a strategic shift towards implementing more advanced security measures like FIDO2. The red teaming results, when shared with the entire company, dramatically increased security awareness, leading to a 93% automatic completion rate for subsequent security training, a stark contrast to previous struggles with compliance.

In addressing the raw technical challenges of a SaaS platform (Step 5), Liferay confronted issues like DDoS protection. While deploying AI-based DDoS detection solutions, they found these tools were not a panacea. For instance, highly evasive "Chinese crawlers" repeatedly attacked specific slow pages, constantly changing their IP addresses and exhibiting no discernible patterns that could be identified by traditional threat intelligence databases. This highlighted the limitations of even advanced automated systems against sophisticated, adaptive attackers.

Another significant technical gap was identified in ransomware protection for backups. Despite using a leading cloud provider and being a major account, Liferay found no "solved problem" for ensuring backup integrity against a compromised administrative account. The cloud provider's initial recommendation was to implement retention policies to prevent deletion or modification of backups. However, Liferay quickly pointed out that a compromised administrator could simply delete the retention policy itself. The next suggestion was to apply a "lean" (lock) on the entire project to prevent its deletion. Again, Liferay raised the critical question: "Who will make sure that the lean cannot be deleted?" The cloud provider had no definitive answer, eventually recommending an air-gapped account setup, which is notoriously complex to implement without risking self-lockout. This experience underscored that even with leading cloud providers, organizations must be prepared to develop custom, layered solutions for critical security challenges.

Finally, Liferay adopted a structured risk-based approach (Step 6) to make informed decisions about product launches, acknowledging that perfect security is an elusive goal. They engaged the entire company, not just the security team, in identifying potential risks. Each identified risk was then assessed for its probability and severity. Crucially, Balog emphasized that severity must be defined on an exponential scale to effectively differentiate and prioritize risks. This approach allowed Liferay to present a clear risk profile to the board, enabling conscious, joint decisions on which risks were acceptable to take when going live and which required further mitigation. This transparency fostered trust and shared accountability for security outcomes.

Demo / Proof of Concept

▶ Watch: Using a risk-based approach for go-live decisions (21:30)

While Joel Balog's presentation did not feature a live software demonstration or code walkthrough in the traditional sense, the talk itself centered around a highly impactful "proof of concept" – the red teaming exercise Liferay conducted (Step 3). This was a real-world demonstration of a security vulnerability and its potential consequences, designed specifically to convince the executive board and the wider employee base of the critical importance of security.

The demonstration involved a sophisticated phishing campaign where Liferay's internal security team (acting as red teamers) targeted the parent company's finance department. They successfully sent a fake email purporting to be from a legitimate vendor, requesting a bank account change for an upcoming payment and including a fraudulent invoice for €300,000. The "proof" lay in the finance department's acceptance of this request, nearly leading to the diversion of a significant sum of money, coupled with a real €600,000 payment. This exercise starkly demonstrated the real-world financial and operational risks associated with a lack of Multi-Factor Authentication (MFA) and inadequate security awareness.

The results of this internal "demo" were presented to the board, providing irrefutable evidence of the company's vulnerabilities. This tangible demonstration was far more effective than theoretical discussions or generic training, leading to immediate executive buy-in for security initiatives, including mandatory MFA implementation and a commitment to more advanced security protocols like FIDO2. Furthermore, sharing the results of this red teaming exercise with all employees acted as a powerful educational tool, directly translating into a 93% completion rate for subsequent security training, proving that a compelling, real-world demonstration can profoundly impact an organization's security posture and culture.

Defensive Implications

▶ Watch: Measuring security posture beyond human evaluation (Q&A) (28:00)

The insights shared by Joel Balog offer several crucial defensive implications for organizations securing SaaS platforms:

1. Prioritize and Test Incident Management: Organizations must establish and regularly test robust Incident Management processes. The Liferay RCE incident, though contained, highlighted the immense financial and operational costs of even a brief shutdown ($22,000/hour). Formalizing communication channels and response protocols, ideally inspired by models like firefighting, is critical for rapid containment and recovery.

1. Embrace and Share Red Teaming: Generic security awareness training often falls short. Defenders should actively pursue red teaming exercises to expose real-world vulnerabilities and demonstrate their impact. Crucially, the results of these exercises, especially those highlighting financial or reputational risks, should be transparently shared with both executive leadership and the broader employee base. This vivid demonstration of risk is far more effective in driving cultural change and gaining buy-in for security initiatives than abstract warnings.

1. Enforce Multi-Factor Authentication (MFA): The near-diversion of €900,000 due to a lack of MFA is a stark reminder of its fundamental importance. Defenders must enforce MFA across all critical systems and accounts, especially those with financial or administrative privileges. Furthermore, a strategic roadmap towards stronger authentication methods like FIDO2 should be a priority to enhance resilience against credential-based attacks.

1. Cultivate Cross-Departmental Alliances: Security teams should actively seek "win-win" scenarios by collaborating with other departments. Engaging legal teams on privacy compliance (GDPR, LGPD), HR on employee background checks and non-compliance policies, and sales on security certifications (ISO, SOC2) can transform them into powerful advocates for security initiatives, making it easier to implement and enforce policies.

1. Anticipate and Strategize for Unsolved Problems: Even with leading cloud providers and advanced security tools (e.g., AI-based DDoS detection), organizations must be prepared for scenarios where off-the-shelf solutions are inadequate. Sophisticated, adaptive attacks (like evasive crawlers) and critical infrastructure protection (like ransomware-proofing backups from compromised administrative accounts) often require custom, layered defenses and a deep understanding of cloud provider limitations. Defenders need to challenge vendor assurances and develop contingency plans.

1. Adopt a Risk-Based Approach with Exponential Scaling: Security is never perfect. Defenders must adopt a risk-based approach to prioritize efforts and make informed decisions, especially for product launches. Involving the entire organization in risk identification and using an exponential scale for severity helps to accurately differentiate and prioritize truly critical risks. This approach fosters transparency and allows executive leadership to make conscious, shared decisions about acceptable risk levels.

1. Build a Company-Wide Security Culture and Global Team: Ultimately, security is everyone's responsibility. Defenders should work to instill a culture where every employee understands their role in the security chain. Furthermore, building a diverse, globally distributed security team, focusing on individual strengths, can significantly enhance an organization's overall security posture and ability to adapt to evolving threats.

Key Takeaways

• Formalize and Test Incident Management: Establish clear, tested incident response processes (like Liferay's firefighting-inspired model) to minimize impact and cost during security incidents.
• Leverage Red Teaming for Cultural Shift: Use targeted red teaming exercises, like simulated phishing, to vividly demonstrate real-world risks, gain executive buy-in, and dramatically increase employee engagement in security training.
• Foster Cross-Functional Collaboration: Build alliances with departments like Legal, HR, and Sales to find "win-win" scenarios, transforming them into advocates for security initiatives and streamlining compliance efforts.
• Prepare for Unsolved Technical Challenges: Recognize that even leading cloud providers and advanced tools have limitations, particularly for sophisticated DDoS attacks and robust ransomware protection for backups against insider threats. Be ready to develop custom, layered solutions.
• Adopt a Risk-Based Decision Framework: Implement a company-wide risk assessment process, using an exponential scale for severity, to make transparent, informed decisions with leadership about acceptable risks for product launches and ongoing operations.
• Cultivate a "Whole Company as Security Team" Mindset: Build a diverse, global security team and empower every employee to understand and contribute to the organization's security posture, making security a collective responsibility.

About the Speaker(s)

Joel Balog is the Vice President of DevSecOps at Liferay. With a tenure of 14 years at the company, he has held various leadership roles, including leading the Hungarian office and serving as the head of Global Technical Support. His transition into the DevSecOps leadership role was a strategic move by Liferay's board to embed robust security practices during the company's shift towards a SaaS model. Balog's experience spans general management and technical operations, providing him with a holistic perspective on integrating security into business and product development.