Introduction
At first glance, training language models and encrypting data seem like completely different problems: one learns patterns from examples to generate text, the other scrambles information to hide it. Yet, their underlying algorithms share a curious resemblance. In 2025, this similarity is increasingly studied to optimize both fields.
Sequence Processing: From Old to Modern
Consider the venerable recurrent neural network, feeding text token by token into a recurrent state before generating the output text. This structure is identical to the Sponge construction in SHA-3, absorbing bytes into a state before squeezing out the hash. To process variable-length input into a fixed-size state, absorbing sequentially is a natural choice.
Modern neural networks like Transformers have improved upon this sequential approach by using parallel processing. In cryptography, similar constructions are used for ultra-fast Message Authentication Codes (MACs).
The Basic Primitive: Linear and Nonlinear Layers
At the core of neural networks and ciphers are alternating linear and nonlinear layers. Linear transforms provide "mixing" between different vector positions, allowing many vector elements to influence many others. Nonlinear transforms provide complexity. These layers are repeated to form complex architectures, focusing research and optimization efforts.
Efficient Mixing: Rows and Columns
Neural networks and ciphers often organize their state as a grid, alternating between mixing rows and mixing columns. In neural nets, attention mixes across sequence positions (rows), while feed-forward layers mix within each position (columns). In the AES cipher, ShiftRows permutes across columns while MixColumns combines within them.
Implications and Future Convergences
The similarities between neural networks and cryptographic ciphers offer opportunities for innovation. Optimization techniques developed in one field can be applied to the other, paving the way for more secure and efficient systems.
Conclusion
In 2025, the convergence of neural networks and cryptographic ciphers is not merely an academic curiosity but a driver of technological innovation. Let's discuss your project in 15 minutes.