Transforming BEC Protection Against Generative AI Evasions with Deep Learning, Zero-Shot Classification, and Zero-Trust Principles

Introduction

Business Email Compromise (BEC) continues to be a significant threat to organizations, causing an estimated $2.9 billion in losses in 2023—a 7% increase from the previous year—and totaling $50 billion between October 2013 and December 2022, according to the FBI's IC3 report [3]. BEC scams target businesses or individuals involved in regular wire transfers by compromising email accounts or other communication channels using social engineering or computer intrusion techniques to conduct unauthorized fund transfers - in short BEC are scams that, when successful, cause an organization to lose money

Traditional BEC detection methods face challenges due to the sophisticated, conversation-based nature of these scams. The rise of generative AI and large language models (LLMs) allows scammers to create highly varied syntactic and semantic messages, [3] making conventional pattern-matching systems and fine-tuned neural network classifiers more susceptible to evasion. 

Besides detecting highly evasive malicious attachments and URLs (such as ransomware, phishing, etc.) that can be introduced by generative AI or threat actors, the Neural Analysis and Correlation Engine (NACE) is also designed to detect BEC messages.

In this blog, I will explore various BEC detection approaches, their limitations, and a solution within NACE that leverages zero-shot classification and is built on zero-trust principles to detect BEC emails and withstand evasion techniques introduced by generative AI or threat actors.

The problem with building a profile using a ML-based model

A machine learning algorithm can be used to build profiles of C-Level executives by analyzing emails. Some of the feature sets that can be used to create these profiles include writing style, activity-based features such as date, time, geo-location from which a person is accessing emails, a relationship graph that captures interactions with others, etc. Each incoming email is then scanned against the profile, and deviations from the profile trigger an alert for BEC.

While this method can detect BEC, it has limitations. Training the machine learning model requires data from real traffic, and the processes of data collection, model building, and training takes time. This creates a window of opportunity for threat actors to exploit. 

Based on the implementation, if the profiling feature in the algorithm requires accessing all historical records before providing its result, it will make the algorithm a reactive solution that necessitates triggering remedial actions. For example, to check if a login attempt from location X does not match a person’s profile, the system would need to wait until after the user has logged in, compute the location, compare it with past records, detect any anomalies (using an unsupervised clustering algorithm, using  rule), and then forward that result to the algorithm, which, along with other features, will decide if exploitation was possible. If the verdict is indeed potential exploitation, additional remedial actions would have to be triggered since the login has already occurred.

Additionally, the final verdict of the algorithm may still lack an understanding of the threat actor’s tactics. As shown in Figure 1.0, based on our research presented at Black Hat 2022 [1], since threat actors can impersonate anyone within an organization for BEC scams, the solution must be scalable to cover every employee. 

Figure 1.0  Non-executive Employees Impersonated by Threat Actors, presented at BlackHat 2022 [1] 

Binary and multiclass classifier based upon threat actor tactics

The approach breaks down detecting BEC into two distinct problems:

• a binary class problem, which classifies emails into a BEC message
• a multi class problem, which classifies BEC into a type of scam 

So the approach not only detects BEC but also labels it into the kind of BEC scam which can be payroll, Money Transfer, Initial Lure, Gift Card Scams, Invoice Scams, Acquisition Scam, W2 Scam, aging reports etc. The approach involves the extracting of text from an email, converting sentences to numeric vectors by encoding the meaning of the words in the sentences, using NNLM or BERT encodings and then performing detection and classification using deep neural networks, The output of the model is a probability score. The higher confidence score is used to give the verdict and the lower probability score is combined with the other analytics detection to give the verdict of BEC. 

The approach has two main limitations:

• Limitation #1: As discussed in our previous blog as well, with Generative AI, conversation payload can be written in multiple ways. Figure 2.0 shows an actual direct deposit payroll BEC scam impersonating the CEO of an organization.

Figure 2.0 Actual Exploit email of direct deposit scam Impersonating CEO 

Figure 3.0 shows some (not all) of the semantic variants generated by GPT-4. Generative models like GPT-4 can create numerous variations of text based on the same underlying intent. This variability can result in different ways of expressing the same message, complicating current detection and classification fine-tuned models. If a neural network model isn’t exposed to all possible semantic variants (the different ways the same message might be expressed), it may not generalize well to unseen examples, potentially resulting in misclassification.

Figure 3.0 A few (not all) variants of semantics used in actual attack generated by GPT

• Limitation #2: To detect BEC messages, features such as the sense of urgency, the domain from which the email was sent, the size of the message, the time it was sent, bank account details, first time sender etc. are extracted. These features combined with semantics contribute to the detection of BEC messages. If any of these features are missing due to variations introduced by the threat actor or generative AI, it may result in BEC messages being delivered to the user’s inbox.

NACE leveraging Zero-Shot Classification and Principles of Zero Trust 

Neural Analysis and Correlation Engine (NACE) uses a comprehensive multimodal, semantic-aware zero-trust approach for detecting BEC messages. This approach uses multiple deep learning models to detect anomalous BEC signals from the body and text from attachment of the email. Additional header features are extracted from the SMTP headers forming a rich header-based feature set. The system extracts text from the email body and uses it as input for various deep learning algorithms to detect the topics, tone, sentiment, tactic, call-to-action and class of the BEC. This is achieved by using a multitude of techniques including zero shot classification using LLM, to understand the semantics and leveraging specialized pre-trained and fine-tune transformer models for classification and natural language understanding. The approach employs both binary having sigmoid activation function and theme or tactics-based multi-class classifiers having softmax activation function, which have been fine-tuned to detect BEC messages and to identify threat actors’ tactics. The tactics-based classifier discerns specific themes or tactics of threat actors, such as payroll changes, money transfers, initial lures, gift card scams, etc. 

Figure 3.0  Sub-system inside NACE to detect BEC

Additionally, the system uses pre-defined natural language classifiers to extract sentiments (positive, negative, and neutral) and emotions from emails. The extracted text is further processed with zero-shot classification, leveraging Bidirectional and Auto-Regressive Transformers to identify emotions, and zero-shot semantics classification with LLMs to accurately identify threat actor tactics making it immune to variations in semantics introduced by Generative AI.

The system is also capable of parsing and interpreting the semantics within an invoice attachment, enabling it to detect invoice scams and the use of fraudulent account numbers for money laundering. This is achieved by combining the financial semantics with the contextual understanding derived from the email.

The outputs from these models—including the BEC binary classifier, sentiment, emotion, tactics classifier, zero-shot classification, and various NLP-derived features—serve as inputs to a meta-classifier to generate a BEC threat score. The output of all these models and semantics derived from them are sent to an expert system to generate a final verdict. This system determines whether an email is a BEC threat.  If the verdict is found to be true, the email is classified as malicious. If the exploit is on east-west traffic the internal account is marked as compromised. The zero-trust layer, which leverages zero-shot classification using LLMs ensures that if an email related to tactics of threat actor matches an executive’s display name but isn’t flagged as malicious, an SMS alert is sent to notify the individual of the request which has been initiated on their behalf. The zero trust layer makes the system immune to any variations in features or semantics which can be introduced by generative AI or by a threat actor. 

Table 1.0 Comparison of Different Approaches. 

Conclusions

Business Email Compromise (BEC) remains a significant threat, with substantial financial impacts on organizations. Traditional detection methods struggle to keep up with the evolving tactics of scammers, particularly with the rise of generative AI, which can produce diverse and sophisticated email variants.

Our holistic approach provides comprehensive detection of BEC messages by identifying threat actor tactics and flagging fraudulent account details. The use of multiple zero-shot analyses strengthens resilience against semantic variations introduced by Generative AI. If a message is not flagged as malicious due to missing features, an SMS alert is sent to the display name’s owner, reinforcing our zero-trust approach and ensuring additional layers of protection.

Reference 

[1]    Abhishek Singh, Fahim Abbasi, “BEC- An insight into a 43 Billion Dollar Problem” , Black Hat 2022 Presentation

[2]   BEC Attacks Surge 20% Annually Thanks to AI Tooling

https://www.infosecurity-magazine.com/news/bec-attacks-surge-20-annually-ai/

[3]   FBI, Business Email Compromise: The $50 Billion Scam

https://www.ic3.gov/Media/Y2023/PSA230609