AI КАК ИСТОЧНИК УГРОЗ (LLM, ГЕНЕРАТИВНЫЙ ИИ). РИСКИ ИСПОЛЬЗОВАНИЯ ГЕНЕРАТИВНЫХ ЯЗЫКОВЫХ МОДЕЛЕЙ В ФИШИНГОВЫХ АТАКАХ: АНАЛИЗ И МОДЕЛЬ УГРОЗ

The proliferation of large language models has transformed the nature of emerging digital threats. While early discussions of risks associated with generative AI focused primarily on adversarial attacks on models and issues of output reliability, attention has increasingly shifted toward the risk of generating malicious content using AI. From this perspective, LLMs function as a tool for preparing attack vectors, which is particularly relevant in the context of phishing, as this form of attack is inherently built around text, trust, the imitation of legitimate communication, and the exploitation of user behavioral responses. For this reason, generative AI should be considered as an independent source of new cyber threats

The aim of this study is to analyze the risks associated with the use of generative language models in phishing attacks and to develop a threat model that links the technological capabilities of LLMs with typical attack scenarios, victim vulnerabilities, and organizational consequences.

The research materials consisted of academic publications addressing the cybersecurity risks of generative AI, the human factor in phishing, AI-enhanced social engineering, tools for generating phishing content, methods for its detection, and security issues of systems incorporating large language models. The study is based on methods of comparative analysis, problem-oriented thematic classification, structured description of AI-enabled phishing scenarios, and threat modeling.

The emergence of generative language models is transforming phishing primarily by increasing its accessibility. Previously, the preparation of a plausible attack depended on the attacker’s competence (at a minimum, language proficiency), the ability to imitate corporate communication styles, and the manual crafting of messages; however, with the widespread availability of LLMs, much of this work is delegated to the model. Existing research on offensive-side scenarios demonstrates that publicly available tools can be used to rapidly generate multiple variations of similar phishing messages, implement phishing schemes involving voice modulation, and even configure chatbots capable of conducting conversations on behalf of the attacker. Notably, generative models are involved already at the attack preparation stage, as they assist in collecting and structuring information about the target, constructing a credible pretext, generating message content, selecting an appropriate tone, and maintaining communication after the initial contact

At the content level, the threat is associated with several key changes (Fig. 1).

The impact of generative models on the transformation of phishing and associated threats

DOI:10.60797/itech.2026.11.9.1

Открыть в полном размереСкачать рисунок

The first change concerns OSINT preparation, in which various data sources – such as social media, corporate websites, public appearances, and other digital traces – are used to generate messages that account for the recipient’s position, field of activity, current projects, and communication style. The second aspect relates to text quality, as AI-generated messages no longer reveal themselves through poor style, spelling errors, or templated phrasing. At the same time, scalability becomes possible, as attackers can generate dozens of message variations for different target groups, vary delivery channels, and quickly relaunch campaigns after previous templates are blocked.

In addition, contemporary threats are becoming increasingly multimodal; attackers may employ not only email, but also voice phishing (vishing), smishing, spoofed video calls, and chatbots that sustain the victim’s engagement as the attack unfolds. In the context of AI-enhanced social engineering, the academic literature identifies three key properties that provide attackers with the highest likelihood of successful phishing: plausibility, personalization, and automation. Taken together, these factors enable an unprecedented level of influence over the user during an attack

In light of this, the following classification of AI-enabled phishing can be proposed (Fig. 2).

Classification of attacks associated with AI-enabled phishing

DOI:10.60797/itech.2026.11.9.2

Открыть в полном размереСкачать рисунок

In practice, the greatest risk arises from the combination of all three levels, as it turns LLMs into a tool for conducting multi-stage social engineering. At the same time, several key directions remain predominant in the academic literature on the study of AI in phishing (Table 1):

Thus, LLMs can be used to bypass traditional filtering systems and to generate messages that account for anti-spam features already at the construction stage. In one of the most illustrative studies, the Phish-Master algorithm was proposed, combining Chain-of-Thought, MetaPrompt, and domain-specific prompts. The authors demonstrated that emails generated in this manner were successfully delivered in 99% of cases in a real network environment; accordingly, the detection model developed on their basis achieved very high performance on test data

In addition, it is widely recognized that traditional defense approaches have long relied on URL-based features, email header analysis, domain anomalies, and other formal indicators. However, these methods are no longer sufficient for detecting LLM-generated phishing, as such messages can be grammatically correct, stylistically consistent, and contextually relevant while still containing malicious intent. As a result, the importance of semantic analysis and the detection of underlying persuasion patterns is increasing. In particular, the academic literature proposes a two-stage model in which the presence of persuasion principles is first assessed, followed by binary classification of the message based on this evaluation. In one study, the authors used a dataset of 2,995 AI-generated emails, achieving a detection accuracy of 94% with an AUC of 98%. Notably, this approach analyzes the message as a mechanism of psychological influence on the recipient

Based on this, a threat model for the use of LLMs in phishing attacks can be proposed (Fig. 3), the construction of which is grounded in the vulnerabilities of LLM-based systems themselves when they are integrated into an organization’s business processes

Threat model for the use of LLMs in phishing attacks

DOI:10.60797/itech.2026.11.9.4

Открыть в полном размереСкачать рисунок

In this context, the threat model of AI-enabled phishing can be represented as a sequence of interconnected elements. At the reconnaissance stage, the generative model assists in organizing OSINT data and constructing a victim profile. At the preparation stage, a pressure scenario, delivery channel, and communication style are selected. At the delivery stage, an email, message, voice fragment, or video communication script is generated. During the attack support stage, follow-up contact may occur, including clarification of details, reduction of suspicion, and guiding the victim toward the intended action. This is followed by the impact stage, involving data disclosure, financial transfers, deployment of malicious payloads, or unauthorized access to internal systems. Compared to traditional phishing, the key distinction of this scenario lies in the ability to rapidly adapt text, tone, argumentation, and persuasion tactics in response to the victim’s behavior. Accordingly, based on this model, the development of defensive strategies and mechanisms becomes feasible.

Thus, the analysis demonstrates that the use of generative language models transforms both the nature of traditional phishing and the approaches to its execution, affecting the entire attack lifecycle. At a minimum, LLMs accelerate reconnaissance, increase the likelihood of successful message delivery, facilitate personalization, and support the scaling of attacks across multiple channels. As a result, phishing becomes less costly to prepare, more precise in targeting, and more difficult to detect. The core of the threat lies in the human factor, as well as in the limited capabilities of existing defenses and detection methods against AI-generated phishing. In this context, the most effective approach appears to be the integration of the identified threats within a unified model, followed by the development of appropriate defensive strategies and mechanisms.