Training TFHE-Based Neural Networks with Approximated Floating-Point Arithmetic
Emanuele Nicoletti (Paper author · Politecnico di Milano), Fabrizio Pittorino (Paper author · Politecnico di Milano), Alessandro Falcetta (Paper author · Politecnico di Milano), Luca Colombo (Paper author · Politecnico di Milano), Manuel Roveri (Paper author · Politecnico di Milano)
Privacy Enhancing Technologies Symposium (PETS) 2026 · Day 2 · Beaver
Overview
In the rapidly evolving landscape of artificial intelligence, the ability to process and train models on sensitive data while preserving user privacy has become paramount. Fully Homomorphic Encryption (FHE) offers a cryptographic solution, allowing computations to be performed directly on encrypted data without prior decryption. However, a significant hurdle in the practical application of FHE, particularly for deep learning, has been its native limitation to binary or integer arithmetic, while neural network training fundamentally relies on floating-point arithmetic. This disparity has traditionally forced researchers to either redefine complex cryptographic primitives from scratch or compromise on the complexity and accuracy of models by using integer approximations.

Key moments
- 0:00 Introduction: TFHE's integer limitation for AI training
- 1:40 Novel approach: Inheriting integer primitives for floating point
- 3:00 Defining approximated multiplication and division (LMUL, LDiv)
- 4:00 Practical results: Reduced execution time for key operations
- 5:45 Emulated tests: No accuracy drop with approximations
- 6:40 Conclusion: Integer interpretation effectiveness and broader impact
Training TFHE-Based Neural Networks with Approximated Floating-Point Arithmetic
Speakers: Emanuele Nicoletti (Paper author, Politecnico di Milano), Fabrizio Pittorino (Paper author, Politecnico di Milano), Alessandro Falcetta (Paper author, Politecnico di Milano), Luca Colombo (Paper author, Politecnico di Milano), Manuel Roveri (Paper author, Politecnico di Milano)
Conference: PETS
YouTube: https://www.youtube.com/watch?v=MaJ-bo4ljQM
Overview
In the rapidly evolving landscape of artificial intelligence, the ability to process and train models on sensitive data while preserving user privacy has become paramount. Fully Homomorphic Encryption (FHE) offers a cryptographic solution, allowing computations to be performed directly on encrypted data without prior decryption. However, a significant hurdle in the practical application of FHE, particularly for deep learning, has been its native limitation to binary or integer arithmetic, while neural network training fundamentally relies on floating-point arithmetic. This disparity has traditionally forced researchers to either redefine complex cryptographic primitives from scratch or compromise on the complexity and accuracy of models by using integer approximations.
The talk "Training TFHE-Based Neural Networks with Approximated Floating-Point Arithmetic," presented by Emanuele Nicoletti, a PhD student from Politecnico di Milano, alongside co-authors Fabrizio Pittorino, Alessandro Falcetta, Luca Colombo, and Manuel Roveri, introduces a novel and practical approach to bridge this gap. Their work focuses on TFHE (Torque FHE), a specific FHE scheme known for its "true bootstrapping" capability, which enables unbounded computation. The core contribution is the development of a set of floating-point operations—including addition, subtraction, multiplication, division, and notably, the first TFHE implementation of square root—by leveraging existing integer primitives. This approach not only simplifies the integration of floating-point arithmetic into FHE but also introduces approximations for multiplication and division that yield significant performance gains without sacrificing model accuracy, thereby making end-to-end encrypted neural network training more feasible.
This research marks a crucial step towards realizing privacy-preserving machine learning at scale. By demonstrating that complex deep learning architectures can be trained on encrypted data using approximated floating-point arithmetic with preserved or even improved accuracy, the authors pave the way for broader adoption of FHE in sensitive AI applications. The methodology bypasses the need for deep cryptographic expertise for scheme modification, instead focusing on defining the logical sequence of operations, making it more accessible to machine learning practitioners.
Background
▶ Watch: Introduction: TFHE's integer limitation for AI training (0:00)
The vision of privacy-preserving artificial intelligence is to enable machine learning models to be trained and deployed on sensitive datasets without ever exposing the raw data. This is particularly relevant in sectors like healthcare, finance, and government, where data privacy regulations are stringent. Fully Homomorphic Encryption (FHE) stands as one of the most promising cryptographic primitives to achieve this. FHE allows arbitrary computations on encrypted data, meaning a cloud service, for example, could train a machine learning model on encrypted patient data and return an encrypted model, all without ever seeing the unencrypted health records.
Among the various FHE schemes, TFHE (Torque FHE) is distinguished by its efficient bootstrapping procedure. Bootstrapping is a critical FHE technique that refreshes the "noise" accumulating in ciphertexts during homomorphic operations, preventing it from overwhelming the signal and allowing for an unbounded number of computations. While TFHE offers this powerful capability, its native arithmetic support is limited to binary values or integer values. This presents a fundamental disconnect with the demands of modern deep learning.
Neural network training, from forward propagation to backpropagation, relies heavily on floating-point arithmetic. Weights, biases, activations, and gradients are typically represented as floating-point numbers to capture the necessary precision for complex function approximation. The existing literature attempting to combine FHE with neural networks has largely been constrained by this limitation, often resorting to integer-based approximations for encrypted data, weights, and activations. While this can work for simpler tasks, it quickly becomes a bottleneck for more complex architectures and tasks, potentially leading to accuracy degradation or requiring careful quantization strategies that add complexity.
Previous attempts to introduce floating-point capabilities into FHE schemes often involved a radical overhaul of the cryptographic primitives themselves. This typically entails redefining the underlying mathematical structures and parameters of the FHE scheme to natively accommodate floating-point representations. Such an approach demands profound expertise in cryptography and the specific FHE scheme, making it a highly specialized and resource-intensive endeavor. It also creates a conceptual separation, where the "floating-point FHE" becomes a distinct entity from the "integer FHE," complicating interoperability and maintenance. The speakers highlight this as a significant barrier, requiring "strong expertise with the scheme in order to be able to modify it."
The work presented by Nicoletti and his team proposes a fundamentally different, "totally opposite" philosophy. Instead of modifying the cryptographic scheme itself, they advocate for inheriting all the existing primitives and parameters from the integer side of TFHE. The innovation lies in defining the logic—the sequence of low-level operations—that constitutes higher-level floating-point arithmetic. This means treating sequences of bits, which represent floating-point numbers, as encrypted integers and then performing operations like XOR, shifts, logical AND, and OR on these encrypted integers to emulate floating-point behavior. This approach significantly lowers the barrier to entry, as it "does not require you to have strong expertise on scheme, rather just define your logic," making FHE-based floating-point computation more accessible to a broader range of developers and researchers.
Key Findings
▶ Watch: Defining approximated multiplication and division (LMUL, LDiv) (3:00)
The research team successfully developed a suite of five essential floating-point operations within the TFHE framework, built upon their novel "integer interpretation" methodology. These operations are: addition, subtraction, square root, multiplication, and division. A critical distinction is made between exact and approximated operations. Addition and subtraction were implemented as mathematically exact operations, preserving full precision.
A standout contribution is the square root operation, which is highlighted as the "first implementation done in TFHE." The availability of a homomorphic square root is particularly significant for deep learning, as it enables the deployment of normalization layers—a key feature in modern deep neural networks that stabilizes training and improves convergence by standardizing input features.
For multiplication and division, the team introduced approximated operations. The approximate multiplication is based on LMUL, a lightweight multiplication algorithm originally designed for energy efficiency in large language models (LLMs). The researchers adapted LMUL, adding "some fancy stuff on top of it" to suit the TFHE context. For division, they designed their own approximate algorithm, dubbed Ldiv. The decision to use approximation for these operations stems from their computational intensity in FHE and their prevalence in neural network training.
The practical results of their approach are compelling:
- Performance Gains for Exact Operations: For addition and subtraction, the "integer interpretation" method led to a reduction in both memory footprint and execution time. This indicates that even for exact operations, their methodology is effective.
- Significant Speedup for Approximated Operations: The most dramatic improvements were observed in multiplication and division. Thanks to the approximation, the team was able to reduce the execution time for these operations by "30 seconds." This is particularly impactful for multiplication, which is "the most used operation" in neural network training, occurring in the range of "hundreds of thousands or even millions" of times. For division, which is used more sparingly, a single homomorphic execution could previously take "from 30 to 40 seconds," making the 30-second reduction highly valuable.
Beyond these practical performance advantages, the work yields several theoretical advancements for privacy-preserving AI:
- Enabling Floating-Point Backpropagation: The developed set of operations provides a "sufficient set of operations to deploy floating point backpropagation," which is the cornerstone algorithm for training neural networks. This capability is essential for optimizing complex models on encrypted data.
- Deployment of Normalization Layers: As mentioned, the square root computation directly enables the integration of normalization layers, which are crucial for the stability and performance of modern deep learning architectures.
- Support for Complex Architectures: With these new capabilities, researchers can now "test more complex architectures" under FHE, including prominent models like ResNets and VGGs, in addition to standard MLPs and CNNs.
To validate their approach, the team conducted empirical evaluations:
- First TFHE CNN Training: They were the "first capable of training from scratch a small scale CNN using TFHE" for a "very trivial task" of ternary classification of handwritten digits (0-2). This initial experiment demonstrated an "accuracy gain of more than 4%" using convolutional layers compared to a baseline from related literature.
- Emulated Validation on Larger Architectures: Recognizing that end-to-end training of larger networks on FHE remains computationally intensive ("still out of reach"), they developed an emulated version of their approximate arithmetic that runs on plaintext. This emulated version was extensively verified across "eight other experiments" on larger architectures (VGG, ResNets) and more complex datasets (CIFAR, "old fashion MNIST"). Crucially, these experiments showed "no accuracy drop" even with the use of approximations. In some cases, they even experienced "an improvement with respect to the exact floating point arithmetic," suggesting that the approximations might sometimes act as a form of regularization.
- Generalizability of Approximated Arithmetic: The researchers also noted that this approximate arithmetic, even outside the FHE context, could be used to "speed up your computation" in general, without altering the learning capabilities of the network, presenting a broader advantage.
In summary, the key findings demonstrate that FHE-based floating-point arithmetic is not only feasible but can be implemented efficiently and accurately through an "integer interpretation" approach, significantly advancing the state of privacy-preserving machine learning.
Technical Deep Dive
▶ Watch: Practical results: Reduced execution time for key operations (4:00)
The core technical innovation of this work lies in its methodology for implementing floating-point arithmetic within the TFHE (Torque FHE) framework without modifying the underlying cryptographic scheme. TFHE, while powerful due to its bootstrapping capability enabling unbounded computations, natively operates on binary or integer values. The challenge is that virtually all modern AI/ML applications, especially neural network training, rely on the higher precision and dynamic range offered by floating-point numbers.
Traditional approaches to bridging this gap have involved redefining the cryptographic primitives themselves to support floating-point. This is a highly complex task, requiring deep expertise in number theory, lattice cryptography, and the specific FHE scheme. Such modifications can introduce new security considerations, performance overheads, and hinder compatibility with existing FHE libraries.
The authors propose an "integer interpretation" strategy. Instead of altering TFHE's core, they treat a floating-point number as a sequence of bits, which can then be represented as an encrypted integer. For instance, an IEEE 754 standard floating-point number (e.g., single-precision float32 or double-precision float64) consists of a sign bit, exponent bits, and mantissa bits. Each of these bits, or groups of bits, can be homomorphically operated upon as encrypted integers. The task then becomes defining the logic—the sequence of elementary integer-based operations—that collectively performs a floating-point operation.
Consider the example of floating-point addition. In plaintext, this involves aligning exponents, adding mantissas, and then normalizing the result. In an FHE context, each of these steps must be broken down into operations TFHE can natively perform:
- Extract components: Separate the sign, exponent, and mantissa from the encrypted floating-point numbers. This involves bitwise operations (e.g., encrypted shifts, ANDs) on the encrypted integer representation.
- Compare exponents: Determine which number has the larger exponent. This requires homomorphic comparison, which can be constructed from basic arithmetic and conditional operations.
- Align mantissas: Shift the mantissa of the smaller number to match the exponent of the larger. This involves homomorphic bit shifts.
- Add/Subtract mantissas: Perform addition or subtraction on the aligned mantissas. This is done using homomorphic integer addition/subtraction.
- Normalize result: Adjust the exponent and mantissa to maintain the standard floating-point format. This might involve further shifts and comparisons.
- Combine components: Reassemble the sign, exponent, and mantissa into the final encrypted floating-point number.
The authors successfully implemented addition and subtraction as mathematically exact operations using this "integer interpretation." This means the results are identical to plaintext floating-point operations, albeit performed on encrypted data.
A significant technical achievement is the square root computation, noted as the "first implementation done in TFHE." Implementing square root homomorphically is non-trivial. It often involves iterative algorithms (like Newton-Raphson or digit-by-digit methods) that converge to the square root. Each iteration consists of basic arithmetic operations (multiplication, addition, division). To make this work in TFHE, these iterative steps must be translated into homomorphic operations. The ability to perform square root homomorphically is crucial for normalization layers (e.g., Batch Normalization, Layer Normalization) in deep learning. These layers often involve calculating the variance (which requires squaring and summing) and then dividing by the standard deviation (which requires a square root). Without a homomorphic square root, implementing these common and powerful layers in FHE would be exceedingly difficult or inefficient.
For multiplication and division, the authors opted for approximated algorithms. This decision was driven by the inherent computational cost of exact multiplication and division in FHE, which can be orders of magnitude slower than addition.
- Approximate Multiplication (LMUL-based): The multiplication algorithm is based on LMUL, a "lightweight version of standard multiplication." LMUL was originally developed for energy-efficient computations in Large Language Models (LLMs), implying it sacrifices some precision for speed. The authors adapted this algorithm for the FHE context, adding "some fancy stuff on top of it" to optimize its performance or suitability for TFHE's specific characteristics. The key idea behind such lightweight or approximate multiplications is often to reduce the number of carry propagation chains or to truncate intermediate products, thereby reducing the depth of the homomorphic circuit and thus the computational cost.
- Approximate Division (Ldiv): For division, they designed a custom approximate algorithm called Ldiv. Homomorphic division is notoriously expensive. Exact division typically involves iterative methods or complex circuit constructions that translate division into a series of multiplications and subtractions, each of which is costly in FHE. An approximate division algorithm likely simplifies these iterations, potentially using fewer steps or less precise intermediate calculations, leading to substantial speedups.
The performance gains achieved by these approximated operations are remarkable. Reducing the execution time of multiplication by 30 seconds per operation is highly impactful, given that neural network training involves "hundreds of thousands or even millions" of multiplications. This directly translates to making the training process significantly more feasible within FHE. Similarly, the 30-second reduction for division, which could otherwise take "from 30 to 40 seconds," drastically improves the practicality of any operation requiring homomorphic division.
Crucially, the authors' work enables floating-point backpropagation. Backpropagation is the algorithm used to train neural networks by iteratively adjusting weights based on the gradient of the loss function. This process involves numerous floating-point multiplications, additions, and subtractions, as well as more complex derivatives. By providing a sufficient set of these fundamental operations in floating-point, the researchers have laid the groundwork for end-to-end homomorphically encrypted training of deep neural networks. The ability to handle various bit widths, from 8-bit integers up to 64-bit floating points, further enhances the flexibility and applicability of their arithmetic.
Demo / Proof of Concept
▶ Watch: Emulated tests: No accuracy drop with approximations (5:45)
While the talk did not feature a live, interactive demonstration of their FHE system, the researchers provided concrete proofs of concept and empirical evidence to validate their methodology and findings. These demonstrations focused on both the functional correctness and the practical utility of their approximated floating-point arithmetic within TFHE.
The primary proof of concept involved training a small-scale Convolutional Neural Network (CNN) entirely from scratch using their TFHE-based floating-point arithmetic. This CNN was tasked with a "very trivial task": ternary classification of handwritten digits, specifically distinguishing between digits zero, one, and two. This experiment served as a foundational validation, demonstrating that their homomorphic floating-point operations were robust enough to support the complex computations involved in a full neural network training loop. The results were encouraging, showing an "accuracy gain of more than 4%" compared to a baseline from existing literature, even on this simple task. This indicated that not only was the training feasible, but the convolutional layers, powered by their FHE arithmetic, could yield superior performance.
Acknowledging the current computational limitations of training "larger networks" end-to-end with FHE (which is "still out of reach" for practical timeframes), the team developed an emulated version of their approximate arithmetic. This emulator runs on plaintext data, allowing for rapid experimentation and verification of the arithmetic's learning capabilities and accuracy preservation on significantly larger and more complex scenarios. This emulated environment was critical for testing the hypothesis that approximations for multiplication and division would not degrade model performance.
The emulated version was put through its paces in "eight other experiments," involving much larger and more common deep learning architectures such as VGG and ResNets. These experiments utilized more complex and widely recognized datasets like CIFAR and the full MNIST dataset (beyond just digits 0-2). The results from these extensive emulated trials were profoundly significant: the networks trained with their approximated floating-point arithmetic exhibited "no accuracy drop" compared to networks trained with exact floating-point arithmetic. Even more surprisingly, in "some cases," they observed "an improvement with respect to the exact floating point arithmetic." This counter-intuitive outcome suggests that the inherent noise or regularization effect introduced by the approximation might occasionally aid in generalization or prevent overfitting, a phenomenon sometimes observed with reduced precision training.
The combination of the small-scale FHE-trained CNN and the extensive emulated validation on larger models and datasets provides strong evidence for the viability and effectiveness of their approximated floating-point arithmetic. While a full-scale FHE training demonstration of a ResNet on CIFAR remains a future goal, these proofs of concept firmly establish the theoretical and practical foundations for achieving it. The mention of discussing "more stuff in the poster block" also implies that detailed visual results, architectural diagrams, and performance metrics were presented in a supplementary format, further elaborating on these demonstrations.
Defensive Implications
▶ Watch: Conclusion: Integer interpretation effectiveness and broader impact (6:40)
The research presented by Emanuele Nicoletti and his team has profound defensive implications for the field of cybersecurity and privacy-preserving AI. By enabling floating-point arithmetic within TFHE, their work directly addresses one of the most significant barriers to the widespread adoption of Fully Homomorphic Encryption (FHE) in real-world machine learning applications.
First and foremost, this work significantly advances the practical feasibility of privacy-preserving machine learning (PPML). Organizations handling highly sensitive data—such as patient medical records, financial transactions, or classified intelligence—are often constrained by strict regulatory requirements (e.g., GDPR, HIPAA) that prohibit direct processing of unencrypted information by untrusted parties (like cloud providers). FHE allows computations to be performed directly on encrypted data, meaning a neural network can be trained or inferences can be made without ever decrypting the input data. The ability to perform floating-point operations within FHE, which is essential for modern deep learning, means that complex, high-performing models can now theoretically be trained and deployed in such privacy-critical environments. This empowers defenders to leverage the power of AI for tasks like fraud detection, anomaly detection, or medical diagnostics without compromising individual privacy or violating compliance mandates.
Secondly, the "integer interpretation" methodology employed by the researchers lowers the barrier to entry for FHE development. Historically, implementing custom arithmetic or adapting FHE schemes required deep cryptographic expertise. By demonstrating that floating-point operations can be constructed from existing integer primitives, the work allows machine learning engineers and developers with less cryptographic background to define and implement complex functions on encrypted data. This democratizes FHE development, potentially accelerating the creation of new privacy-preserving applications and tools. This shift from modifying the cryptographic scheme itself to merely defining the logic for operations makes FHE more accessible and practical for a broader developer community, fostering innovation in secure computing.
The performance improvements, particularly the 30-second reduction in execution time for homomorphic multiplication and division, are critical for moving FHE from theoretical possibility to practical utility. Neural network training is computationally intensive, with millions of multiplications. While end-to-end training of very large networks is still a challenge, these speedups make incremental progress towards making such operations feasible. For defenders, this means that privacy-preserving AI models, even if currently smaller in scale, can be trained and updated more efficiently, allowing for quicker adaptation to new threats or data patterns without compromising privacy. This efficiency gain is vital for security operations centers (SOCs) or intelligence agencies that need to rapidly deploy and update models on sensitive data.
Furthermore, the crucial finding that approximated floating-point arithmetic preserves, and in some cases even improves, model accuracy is a game-changer. One of the primary concerns with using approximations in machine learning, especially in security-sensitive contexts, is the potential degradation of model performance or the introduction of vulnerabilities due to reduced precision. The empirical validation on large architectures and datasets (VGG, ResNets on CIFAR, MNIST) demonstrating no accuracy drop assuages these concerns. This gives defenders confidence that adopting FHE with these approximations will not compromise the efficacy of their AI-driven security tools. In fact, the observed accuracy improvements could even be seen as a defensive advantage, potentially leading to more robust models.
Finally, the development of a homomorphic square root operation, enabling normalization layers, is a subtle but powerful defensive implication. Normalization layers are essential for stable and efficient training of deep neural networks, especially very deep ones. Their absence in FHE-based training would severely limit the complexity and performance of privacy-preserving models. By enabling these layers, the research allows defenders to build and train state-of-the-art deep learning architectures that are both private and highly effective, capable of tackling sophisticated adversarial attacks or detecting subtle anomalies that simpler models might miss.
In essence, this research provides cryptographic defenders with practical tools and methodologies to deploy powerful AI capabilities in privacy-sensitive environments. It paves the way for a future where the benefits of AI can be fully realized without sacrificing the fundamental right to privacy, strengthening the overall security posture against data breaches and misuse.
Key Takeaways
- Accessible FHE Floating-Point: Floating-point arithmetic for TFHE can be effectively implemented by leveraging existing integer primitives ("integer interpretation"), eliminating the need to redefine complex cryptographic schemes from scratch. This significantly lowers the expertise barrier for FHE development.
- Performance Boost via Approximation: Approximate algorithms (LMUL-based for multiplication and Ldiv for division) yield substantial execution time reductions (e.g., 30 seconds per operation for multiplication), which is critical for making homomorphic neural network training more feasible given the millions of operations involved.
- Enhanced Deep Learning Capabilities: The developed operation set, including the first TFHE implementation of square root, enables crucial deep learning features such as floating-point backpropagation and the integration of normalization layers, which are vital for training complex, stable, and high-performing neural networks.
- Accuracy Preservation with Approximation: Extensive testing on large architectures (VGG, ResNets) and complex datasets (CIFAR, MNIST) demonstrates that the approximated floating-point arithmetic preserves or even improves neural network learning capabilities and accuracy, debunking concerns about precision loss.
- Broad Applicability: The proposed approximate arithmetic offers advantages beyond FHE, capable of speeding up general computation without compromising the learning capabilities of neural networks, suggesting wider utility for efficient AI.
- Memory and Time Efficiency: The "integer interpretation" approach effectively reduces memory consumption and computational time for all tested bit widths, from 8-bit integers to 64-bit floating points.
About the Speaker(s)
The talk was presented by Emanuele Nicoletti, a PhD student from Politecnico di Milano. He is one of the primary authors of the research paper discussed, alongside his colleagues Fabrizio Pittorino, Alessandro Falcetta, Luca Colombo, and Manuel Roveri, all affiliated with Politecnico di Milano. Their collaborative work focuses on advancing the practical applications of Fully Homomorphic Encryption, particularly in the domain of neural network training, by addressing fundamental challenges related to arithmetic operations and computational efficiency.
Reviews
Dr. Zero (Offensive Security Researcher) — SOLID
Nicoletti and team solved a massive bottleneck in privacy-preserving ML without inventing new crypto primitives. They mapped floating-point logic onto TFHE integer operations and delivered the first working homomorphic square root. It's still too slow to train a full ResNet natively in ciphertext, but it's a huge, practical step forward for FHE usability.
Heather Calloway (CISO) — WEAK
This is deep academic research on Fully Homomorphic Encryption (FHE), well outside a standard CISO's operational focus. However, it signals important progress toward a massive strategic capability: training AI models on encrypted data without destroying performance. It is not ready for enterprise deployment, but it tracks the underlying math that will eventually rewrite our data privacy architectures.
→ Top-rated talks at Privacy Enhancing Technologies Symposium (PETS) 2026
All talks from Privacy Enhancing Technologies Symposium (PETS) 2026