Introduction

A major societal problem, the rapid dissemination of false news poses risks to public health, politics, and the financial sector1. The proliferation of visual deep fakes and textual fake news worsens these issues, especially with the advancement of sophisticated generative models used to create misleading content2. Convergence also enables multimodal forgery media to spread more extensive and deceptive material to readers. Modifications in both visual and textual modalities make it particularly challenging to detect such fabricated news3. One potential solution is incorporating cross-modal features into multimodal fake news detection (MFND) algorithms. Although significant improvements have been made, training remains limited to specific topics, making it difficult to acquire global knowledge4. Domain shift refers to large distribution differences seen in open-world fake news, characterized by various forging techniques and real-world contexts5. The presence of domain shift further complicates the detection and definition of fake news in MFND tasks.

One of the most persistent challenges in today’s digital age is the spread of fake news, defined as intentionally misleading information presented as real news6,7. The rapid proliferation of false information results in serious consequences, including declining trust in institutions, increased violent incidents, and the weakening of democratic processes8. Strong detection systems for identifying false news are more crucial than ever. Several factors contribute to the widespread dissemination of fake news online. The ease of content creation and distribution, anonymity, and lack of strict regulations allow bad actors to spread false narratives without facing repercussions9,10. Additionally, social media algorithms inadvertently amplify the spread of fake news, as they are often designed to prioritize engaging and provocative content11.

The widespread occurrence of fake news underscores the need to develop and implement efficient detection methods12. One approach involves fact-checking organizations verifying the accuracy of information. However, this process is highly labor-intensive and cannot scale to handle the massive daily content production13. Conversely, AI and ML techniques offer promising solutions for automatically detecting fake news. These algorithms analyze various factors, including language patterns, writing style, and source credibility, to identify potentially deceptive material14,15. However, AI-driven detection approaches also face challenges. AI models can be influenced by biased or inaccurate datasets, and fake news creators continuously evolve their tactics to evade detection16.

To effectively combat fake news, social media companies must integrate multiple content verification techniques within their algorithms and AI strategies, as highlighted in recent research. Given the complexity of the problem, a single approach is unlikely to be effective17. A combination of methods from various disciplines may provide a more robust solution. AI presents a paradox in this context—while it can be leveraged to detect and mitigate fake news, it also contributes to the proliferation of misleading content18. Nonetheless, AI remains a key tool in addressing this challenge.

This study proposes a hybrid approach that combines Particle Swarm Optimization (PSO) and Dandelion Optimization (DO) (PSODO), along with fine-tuning a Modified Transformer (MT)-based model, to develop an advanced system for detecting multimodal fake news. The goal of this approach is to detect distribution inconsistencies and localized forgeries earlier and more effectively than traditional detectors and Large Vision-Language Models (LVLMs) in an open-world setting. The key benefits of the proposed algorithm include:

  • Enhanced search range by integrating DO’s particle updating rules with PSO’s location and velocity updating mechanisms.

  • Implementation of a velocity decay plan to regulate particle movement, ensuring comprehensive exploration while avoiding premature convergence.

  • Improved search precision using velocity decay, allowing a balanced exploration-exploitation tradeoff for effective feature extraction.

  • Increased solution stability by combining DO and PSO while slowing PSO’s learning rate to mitigate oscillation issues.

The remainder of this paper is structured as follows. The next section reviews related literature on fake news detection methods. Then, the methodology underlying the proposed Modified Transformer (MT) and hybrid PSODO optimization approach is detailed. Subsequently, the experimental results and discussion are presented, showcasing the performance of the proposed system. Finally, the paper concludes with a summary of findings and directions for future work.

Related works

Raza et al.19 examined two large-scale language models for fake news detection: one that relies solely on autoregressive decoders and another that is comparable to BERT but relies only on encoders. To ensure the accuracy of news stories in their dataset, they used GPT-4 as an AI-labeling method and had the results verified by human experts. Long Short-Term Memory (LSTM) and BERT-like encoder-only models were fine-tuned using this dataset. Another developed approach for instruction-tuned LLM relies on a majority vote to determine labels during inference. Although LLMs are more robust against textual disruptions, their research shows that BERT-like models outperform them in most classification tasks. The classification accuracy of AI-generated labels supervised by human oversight is higher than that of weak labels trained with remote supervision. This study highlights the effectiveness of combining AI annotation with human monitoring and compares various models for fake news detection.

Cui and Shang20 proposed a Multimodal Interaction Graph Contrastive Learning Network (MIGCL) for fake news detection. This network adaptively reduces irrelevant cross-modal connections while considering both comprehensive and locally fine-grained cross-modal interactions using alignment and filtering algorithms. To further investigate complex interrelationships between intra- and intermodal representations, they developed a framework for hierarchical graph contrastive learning that employs both supervised and unsupervised contrastive learning methods. By constructing unimodal graphs at the intermodal level, a more precise authenticity assessment for specific modalities is achieved. Additionally, multimodal approaches are used to construct graphs at the intermodal level, capturing relationships between intra- and cross-modal samples. The model’s feature representation is further strengthened by applying topological perturbations. Experimental results on three benchmark datasets demonstrate that MIGCL outperforms existing methods, proving its effectiveness.

Wang et al.21 introduced a multimodal model for fake news detection that integrates data from multiple sources, such as news images and text, to address differences in source and task domains. Their approach employs a multimodal domain adaptation network to analyze social media data, mitigating domain shifts in news data while capturing links between events caused by adversarial influences. For deep feature extraction, BERT and EfficientNet are utilized within a multi-network structure. Experimental results using public datasets from Weibo and Twitter confirm that this approach significantly enhances fake news detection.

Jing et al.22 developed a multimodal model using DPSG (Dynamic Propagation Social Graphs) to detect fake news. Their approach integrates temporal penetration fusion blocks for multimodal dynamic fusion of different node types in propagation sequences. Additionally, a dynamic signed weighting method is used to enhance social network features in the propagation graph. Finally, a Transformer model integrates data from multiple sources, capturing patterns in news propagation. Experimental results on three datasets validate the effectiveness of the proposed model in detecting fake news.

Li et al.23 addressed challenges that arise when existing fake news detection strategies are used independently. Domain shift is a significant issue, characterized by anomalies such as variations in dissemination patterns, nominal polysemy, and disparities in data distribution. Their study employs multiple BERT models in a multi-extraction network to extract features, followed by a domain localization module for precise domain targeting, and a fast model to address timeliness. Their results indicate that the proposed method significantly enhances fake news detection across multiple domains.

Beseiso and Al-Zahrani24 proposed an ensemble method that improves fake news detection by integrating Long Short-Term Memory (LSTM) and CNN architectures. Their model employs LSTM layers with 2, 3, and 4 kernel sizes to handle 2-gram, 3-gram, and 4-gram token sequences. Experimental results show a significant improvement over previous studies, with the CNN-LSTM model achieving 97.3% accuracy and the LSTM model achieving 96.7% accuracy. Performance was further improved by combining embedding layers with enlarged word sequences and pre-trained embedding vectors. The ensemble model outperformed prior studies by 10.03% in prediction accuracy on the Liar dataset.

Radha et al.25 integrated blockchain technology, reinforcement learning (RL), and natural language processing (NLP) to enhance fake news detection. Their approach involved building a massive database of news stories and associated metadata, followed by text tokenization and cleaning using NLP techniques. Key features such as word frequency and readability scores were extracted to train an RL agent, which evaluates the credibility of new articles based on a reward-and-penalty system. While blockchain’s systemic consequences need further study, their model provides a novel approach to preventing fake news dissemination online.

Wang et al.26 introduced Redesigned Graph Pretraining with Contrastive Prompting (RGCP) for fake news detection. Their approach improves propagation structures by eliminating noisy interactions and including implicit connections using self-supervised contrastive learning. The restructured model enhances news transmission reliability, producing strong pre-trained news representations. Their contrastive learning-based prompt tuning module further bridges the optimization gap, reframing fake news detection as a graph contrastive pretraining task. Benchmark dataset experiments show RGCP improves few-shot classification accuracy by an average of 10.15%.

Abe et al.27 proposed a graph convolutional network (GCN) that adapts to new input data and extracts structural features from time series to detect fake news. Their time series-aware approach enhances detection effectiveness by tracking the evolution of news dissemination structures over time. Experimental results on FibVID and FakeNewsNet datasets confirm the efficacy of this method.

Murti et al.28 developed a machine learning-based fake news detection system using K-Nearest Neighbors (K-NN). Their model classifies fake news by comparing new texts to known feature space data. K-NN is advantageous for its simplicity and ability to handle non-linear data. Their approach enhances detection accuracy by leveraging linguistic features of fake news. Their model achieved Mean Absolute Error (MAE) of 0.011 and RMSE of 0.077, outperforming SVM and neural networks on the FakeNewsDetection dataset.

Lingzhi et al.29 introduced GAMED (Knowledge Adaptive Multi-Experts Decoupling), a multimodal modeling approach. GAMED improves detection by generating discriminative features through modal decoupling and optimizing cross-modal synergy. It leverages concurrent expert networks to adjust attributes and embed semantic knowledge for better information sharing. GAMED dynamically processes multimodal inputs using adaptive classification techniques, enhancing detection accuracy. Comparative results show GAMED outperforms state-of-the-art models on Fakeddit and Yang datasets.

Ding et al.30 developed EvolveDetector, a hard attention-based knowledge storage mechanism for retaining knowledge of learned events. Their memory-based approach enables the model to compare new events with previously learned ones, using multi-head self-attention for classification. EvolveDetector surpassed state-of-the-art baselines on Sina Weibo and Twitter datasets, demonstrating strong cross-event fake news detection capabilities.

Fatoni et al.31 created a deep learning model to distinguish real and fake images. They evaluated ResNet, VGG16, and CNN in conjunction with Error Level Analysis (ELA) for dataset preprocessing. Among the models, CNN achieved 92% accuracy, VGG16 reached 94%, and ResNet attained 95%. A real-world system prototype was also developed, demonstrating ResNet’s superior image classification performance.

Hashmi et al.32 developed a hybrid deep learning model for fake news detection using FastText word embeddings combined with ML and DL methods. Their model attained 99% accuracy, 0.99 F1-score, and 0.99 LTM score. They also experimented with transformer-based models such as BERT, XLNet, and RoBERTa, which performed better than traditional RNN-based models.

Mallik and Kumar33 proposed a Word2 Vec and LSTM-based hybrid framework for fake news detection. Their model extracted context-free feature vectors and applied stacked LSTM layers for classification. Comparative analysis confirmed that their proposed model outperformed state-of-the-art models across multiple datasets.

Research gap

Despite significant advancements in fake news detection, several challenges remain unaddressed. Existing approaches struggle with domain shift, limiting their ability to generalize across diverse datasets. While multimodal models enhance detection, they often fail to effectively integrate cross-modal features, reducing their robustness against manipulated content. Furthermore, large vision-language models (LVLMs) contain global knowledge but lack specificity in detecting local forgeries. Current methods also over-rely on public opinion analysis, making early-stage misinformation containment difficult. Additionally, optimization algorithms used in fake news detection suffer from slow convergence and local optima entrapment, affecting model performance. Reinforcement learning techniques remain underutilized in fine-tuning transformer-based models for fake news classification. To address these gaps, this study proposes a hybrid Modified Transformer (MT) model fine-tuned with PSODO, a multi-strategy particle swarm optimization hybrid dandelion optimization algorithm, to improve accuracy, generalization, and computational efficiency in detecting multimodal fake news.

Proposed methodology

Those who produce fake news essentially distort the truth. Theoretical studies on media bias suggest that distortion bias is best represented as a binary classification problem. For this reason, the task of detecting false news is typically formulated as a binary classification problem. A news post on social media is denoted as N. The label y is assigned a value of ‘0’ or ‘1’, depending on the truthfulness of N. The predicted label ŷ is also categorized as either ‘0’ or ‘1’, as determined by the assessment of the model M.

Fake News Detection: With the original post of news N and relevant information provided, i.e., M: N → {1,0}, the goal of this issue is to determine if news article N is fake.

$$\:M\left(N\right)=\left\{\begin{array}{c}1,\:\:\:N\:is\:fake\:\\\:0,\:\:\:\:otherwise\end{array}\right.$$
(1)

where M represents the assessment method that the researchers are working on.

Proposed method for feature extraction

The design of the Modified Transformer (MT) model is grounded in a progressive, three-stage fine-tuning strategy, crafted to optimize text representation and summarization for fake news detection. Each stage—encoder tuning, decoder adjustment, and bias parameter refinement—targets specific components of the model to incrementally improve semantic comprehension, contextual alignment, and output coherence. This deliberate separation enhances model generalization across diverse linguistic inputs. Additionally, a hybrid optimization framework, PSODO (Particle Swarm Optimization + Dandelion Optimization), is integrated to reinforce learning by balancing global exploration and local exploitation. This hybrid approach prevents premature convergence and improves convergence precision, thereby maximizing the model’s reward-guided training performance. To present a methodology that employs a dual approach, leveraging the mT5 Transformer model to enhance text summarization. This approach begins with a detailed examination of mT5 and its three-step supervised fine-tuning procedure. To further improve the accuracy and contextual relevance of the model’s summaries, reinforcement learning is incorporated using the Proximal Policy Optimization (PPO) algorithm. Based on the principles of both supervised and reinforcement learning, this methodology provides a comprehensive strategy for summarizing content from false news sources.

Modified transformer model (MT)

The Transformer model serves as the backbone of our proposed approach. In the domain of text summarization and other natural language processing (NLP) tasks, the mT5 text-to-text model is well-known for its versatility and power. Its ability to adapt to multiple languages makes it a strong candidate for summarizing articles containing fake news.

The mT5 model retains the core architecture of the original T5 model, developed by Google Research. Due to its sequence-to-sequence processing capability, T5 efficiently handles various NLP tasks, including translation, summarization, classification, and question answering.

A variant of the Transformer architecture, mT5 follows an encoder-decoder structure, as illustrated in Fig. 134. Both the encoder and decoder consist of stacked identical layers comprising fully connected feed-forward networks and multihead self-attention mechanisms. The mT5 model modifies this architecture using multilingual pre-training and fine-tuning techniques, enabling it to handle cross-linguistic tasks effectively.

A) Encoder.

Before processing the input sequence through its multiple feed-forward and self-attention layers, the mT5 encoder first tokenizes the text. The self-attention mechanism allows the model to track dependencies across the sequence, irrespective of token position, ensuring it captures semantic and syntactic nuances in different languages.

B) Decoder.

The decoder processes the encoded data to generate the output sequence, which in this case is the summarized text. It applies self-attention mechanisms to interpret contextual relationships within the encoded data. Additionally, the encoder-decoder attention mechanism enhances the decoder’s ability to extract key information from the encoded news article, enabling the generation of concise and accurate summaries.

Fig. 1
figure 1

Diagram of the Transformer Encoder-Decoder classical construction.

Full size image

C) Multilingual pre-training.

The MT model can comprehend and generate documents across various linguistic contexts due to its multilingual training. This capability is particularly beneficial for fake news detection, as fake news often involves complex structures and subtle linguistic nuances.

Adapted from the T5 model, the MT model supports multiple languages. To ensure efficient training across languages with varying data availability, a balancing strategy is implemented. This strategy selectively includes texts during training, enhancing learning for low-resource languages. The training algorithm dynamically adjusts the emphasis placed on each language, ensuring effective multilingual learning and improved model generalization.

$$\:p\left(L\right)\propto\:{\left|L\right|}^{a}$$
(2)

Where |L| is the total number of samples in a given language and p(L) is the likelihood of sampling text related to that training. During limited textual resources, the degree of reinforcement probability is controlled by the hyper-parameter α.

Importantly, this formula ensures that languages with fewer textual resources can still learn effectively from more data-rich languages, preventing the model from being skewed by dominant languages. This balancing strategy, when applied meticulously, enhances the model’s adaptability, improving its ability to capture linguistic complexities across different languages and promoting a more comprehensive and fair learning process during training.

Having established the MT Transformer model as the foundation of our approach, we now present a detailed examination of our proposed strategy. The process begins with data preparation and cleaning, followed by fine-tuning the model for fake news text summarization. To further enhance the model’s summarization capabilities, we incorporate reinforcement learning, strengthening the method’s effectiveness.

Three-Phase Fine-Tuning.

Our proposed methodology introduces a unique and robust fine-tuning strategy for text summarization by employing a three-phase training approach. Each phase enhances the model’s ability to understand context and generate high-quality summaries. Distinguishing between these phases and epochs is critical, as each step contributes to model refinement.

Phase 1: Encoder Fine-Tuning.

This phase focuses on enhancing the model’s capability to interpret and encode complex linguistic structures, ensuring rich semantic embeddings. It sets a strong contextual foundation for subsequent text generation tasks.

Phase 2: Decoder Fine-Tuning.

By isolating the decoder, this phase improves the quality of generated summaries. It enables the model to accurately translate encoded information into coherent outputs, crucial for detecting nuanced misinformation.

Phase 3: Training Bias Parameters.

This final phase fine-tunes bias terms, enabling the model to adapt to subtle data patterns often present in fake news. This targeted optimization boosts model responsiveness and interpretability.

Our three-stage fine-tuning method, applied across three epochs per phase, introduces an innovative contribution to text summarization. By systematically addressing individual model components, we equip the MT model with the flexibility and capability to generate context-aware and coherent summaries. This revolutionary methodology pushes the boundaries of automated fake news detection and summarization.

Reinforcement learning

Our efforts to enhance text summarization have led to a groundbreaking approach—integrating reinforcement learning into the training pipeline. We employ the hybrid PSODO algorithm (a combination of Particle Swarm Optimization (PSO) and Dandelion Optimization (DO)) to improve sequential decision-making processes. Integrating reinforcement learning with PSODO significantly enhances the model’s ability to generate accurate, coherent, and context-aware summaries.

A) The Role of Reinforcement Learning.

The MT model functions as an agent that interacts with its environment through reinforcement learning. The training process rewards the model for generating high-quality summaries, guiding it toward maximizing cumulative rewards. In text summarization, this ensures the model captures key points efficiently and arranges them logically, matching human-level summarization quality.

B) Hybrid PSODO Algorithm.

To facilitate reinforcement learning, we utilize PSODO35, a well-established and reliable method for policy optimization in RL contexts. Due to its stability and ability to handle high-dimensional action spaces, PSODO is particularly suited for training neural networks, including the MT model. By leveraging PSODO, the model improves its comprehension of context, fluency, and coherence in summary generation.

C) Reinforcement Learning Environment.

Transitioning from supervised learning (fine-tuning) to reinforcement learning requires a well-structured training environment. This step involves integrating the trained model into a reinforcement learning framework for further improvement. We utilize Transformer Reinforcement Learning X (TRLX), a state-of-the-art distributed training system designed for fine-tuning large language models using reinforcement learning. TRLX supports reward-labeled datasets, offering flexibility in training. Additionally, we develop a specialized reward function to optimize the summarization process. The following section details this reward function.

D) Reward Function.

Selecting an effective reward function is crucial when integrating PSODO with reinforcement learning. For this purpose, we adopt ROUGE (Recall-Oriented Understudy for Gisting Evaluation), a widely used and established metric for assessing text summarization quality.

Our reinforcement learning approach relies on ROUGE to guide the MT model’s learning process. The ROUGE score measures how well model-generated summaries align with reference summaries created by human experts. Using this quantitative evaluation, we provide real-time feedback to the model, refining its summarization quality throughout training.

By implementing ROUGE as the reward function, we ensure that reinforcement learning aligns with well-accepted evaluation standards. This method enables the model to generate summaries that capture essential information, while maintaining high linguistic quality and coherence. Our research confirms that ROUGE-1 serves as the primary reward function in our reinforcement learning framework, leading to significant improvements in fake news summarization.

This is the formula that describes ROUGE-1:

$$\:ROUGE-1=\frac{\sum\:_{i=1}^{N}{C}_{i}}{N}$$
(3)

Where:

  • N represents the total number of unigrams in the reference summary.

  • Ci denotes the number of unigrams shared between the reference summary and the generated summary, where i is an integer ranging from 1 to N.

Our methodology achieves a new level of sophistication and precision through this integration, ensuring that the MT model’s summaries effectively capture key information while maintaining linguistic and contextual accuracy. This innovative approach underscores our commitment to advancing automated text summarization for fake news detection, pushing the boundaries of what is possible and setting new benchmarks for quality and context awareness.

Fine-tuning using hybrid Model – PSODO

The model captures distribution discrepancies by learning contextual semantics through fine-tuned encoder-decoder layers, enabling it to adapt to varying language patterns and sources. For multimodal forgery, reinforcement learning with PSODO enhances the model’s sensitivity to inconsistencies between image and text modalities. PSODO’s global-local optimization ensures that forged elements—such as facial manipulations or sentiment inconsistencies—are effectively detected even in domain-shifted data.

A) Dandelion Optimization Algorithm.

The three-stage process known as the Dandelion Optimizer (DO) mimics the wind-borne long-distance flight of dandelion seeds. The four main components of the DO algorithm framework are: population initialization, fitness calculation, population updating, and global optimization strategy selection.

The seed radius (α) and the local search coefficient (K) are the key parameters of the algorithm. During the iterative process, both α (seed propagation radius) and K (local search coefficient) dynamically adjust. α controls the global search step size, while K prevents local optimization stagnation by introducing random numbers following a normal distribution. As the process evolves, it transitions from exploration to exploitation, ensuring efficient optimization and improved convergence performance.

A dandelion populace matrix seed with N seeds space is distinct, besides its ith seed can be uttered as \(\:{X}_{i}=\:[{X}_{i}^{1},\:{X}_{i}^{2},\:{X}_{i}^{3},\:.\:.\:.\:,\:{X}_{i}^{d}],\:i\:=\:1,\:2,\:.\:.\:.,\:N\). The populace is prepared as Formula (4):

$$\:{X}_{i}={L}_{B}+{r}_{i}\times\:\left({U}_{B}-{L}_{B}\right)$$
(4)

Among them, limit \(\:{r}_{i}\) is a random sum among (0, 1) distribution, \(\:{U}_{B}\) is the maximum charge space, \(\:{L}_{B}\), where d is the maximum column value and N is populace matrix, and where the same characters in the following mean the same thing, is the minimal value in space.

  1. (1)

    Rising stage.

Once dandelion seeds reach a specific height, they begin to disperse. The weather conditions determine one of two possible scenarios regarding the altitude the seeds reach, depending on factors such as humidity. On sunny days, the wind speed follows a logarithmic normal distribution, expressed as \(\:Y\:\sim\:N\:(\mu\:,\:{\sigma\:}^{2})\), where Y is the wind speed, and N(µ,σ²) represents a normal distribution with mean µ and variance σ². Under these conditions, dandelion seeds are more likely to be dispersed over greater distances. The Dandelion Optimizer (DO) prioritizes exploration at this stage. As the wind moves across the search space, dandelion seeds are scattered to various locations. The height to which the seeds can rise depends on wind speed—stronger winds enable greater dispersion. The seeds ascend in a spiral motion, with the vortex above them dynamically adjusting based on wind intensity. This process is mathematically represented in Eq. (5), corresponding to the seed ascending stage in the DO algorithm.

$$\:{X}_{t+1}={X}_{t}+{av}_{x}{v}_{y}In\text{{\rm\:Y}}({X}_{s}-{X}_{t})$$
(5)

Where \(\:{X}_{t}\) is the site of seed in t repetitions. \(\:{X}_{s}\) is the arbitrarily selected site in the search space during iteration. The random locations designated by \(\:{X}_{s}\) are exposed in Eq. (6).

$$\:{X}_{s}=rand\left(1\right)\left({U}_{B}-{L}_{B}\right)+{L}_{B}$$
(6)

Where \(\:{U}_{B}\) and \(\:{L}_{B}\) values are set to 1 besides 0, individually, and\(\:\:rand\left(1\right)\) is a random sum. The function ln Y obeys distribution of \(\:\mu\:=0\) and \(\:{\sigma\:}^{2}=\:1\), and its formula is exposed in (7).

$$\:1n\:\text{{\rm\:Y}}=\left\{\begin{array}{c}\frac{1}{y\sqrt{2\pi\:}}\text{exp}\left[-\frac{1}{{2\sigma\:}^{2}}{\left(1ny\right)}^{2}\right],\:\:\:\:\:y\ge\:0\\\:0,\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:,y<0\end{array}\right.$$
(7)

Parameter \(\:\text{{\rm\:Y}}\) is the distribution \(\:N\left(\text{0,1}\right)\). The search phase α is attuned as an adaptive limit, as exposed in Eq. (8).

$$\:\alpha\:=rand\left(\right)\left(\frac{1}{{T}^{2}}{t}^{2}-\frac{2}{T}t+1\right)$$
(8)

This random perturbation factor α can have values zero and one. The nonlinearity of α reduces and approaches zero as the increases. At the beginning of the process, when the disturbance is somewhat substantial, the global search is given higher priority. As the algorithm progresses, α drops and the local search is employed. For more precise convergence, it’s best to switch to a local search after running a global search. Lift component coefficients are affected when dandelion is subjected to the separating vortex. \(\:{v}_{\text{x}}\) and \(\:{v}_{\text{y}}\) produced. The exposed in Eq. (9).

$$\:\left\{\begin{array}{c}r={e}^{-\theta\:}\\\:{v}_{x}=r\text{cos}\theta\:\\\:{v}_{y}=r\text{sin}\theta\:\end{array}\right.$$
(9)

The parameter θ is a random quantity\(\:[-\pi\:,\:\pi\:]\). On rainy days, the ability of dandelion seeds to rise fully with the wind is influenced by various factors such as air humidity and air resistance. Additionally, these seeds are generated in certain locations. Equation (10) displays the matching formula.

$$\:{X}_{t+1}={X}_{t}k$$
(10)

The parameter k worth adjusts the dandelion. The k charge is intended as exposed in Eq. (11).

$$\:\left\{\begin{array}{c}q=\frac{1}{{T}^{2}-2T+1}{t}^{2}-\frac{1}{{T}^{2}-2T+1}t\frac{1}{{T}^{2}-2T+1}+1\\\:k=1-rand\left(\:\right)q\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\end{array}\right.$$
(11)

Equation (11) shows a convexly decreasing oscillation of k. Because of this, the algorithm can benefit by first searching the global a large step and then developing the local region with a small step. In order to guarantee that the populace eventually converges to the ideal search individual, iteration times are increased until the parameter k approaches 1. Formal analysis demonstrating the dandelion seeds is depicted in Formula (12) according to the data presented above.

$$\:{X}_{t+1}=\left\{\begin{array}{c}{X}_{t}+a{v}_{x}{v}_{y}1n{\rm\:Y}\left({X}_{s}-{X}_{t}\right),\:\:\:\:rand\left(n\right)<1.5\\\:{X}_{t}k\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:,others\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\end{array}\right.$$
(12)

The value of rand(n) is a completely random integer that follows distribution. The general trajectory of evolution is shown by Eq. (12): during periods of sunshine, the orientation of the spiral movement of seeds is adjusted by randomly selecting the location information to update. \(\:{v}_{\text{x}}\) and \(\:{v}_{\text{y}}\) and emphasized. Dandelion seeds germinate in damp environments. As the algorithm improves, it becomes more globally focused, initially exploring the full search space to point iterative optimisation in the right direction. The dynamic control process of random numbers with a normal distribution is studied and improved upon to accomplish this.

(2) Descending stage.

At this point, the search and optimisation phases of the DO algorithm take centre stage. Once dandelions reach a particular height, their seeds begin to fall slowly but surely. So that individuals can easily explore iterative phase, the DO method uses a Brownian gesture obeying normal distribution to imitate the seeds. As seen in Formula (13), the average position data after the rising stage represents the descent, which aids the populace as a whole in searching for and developing the best individual regions.

$$\:{X}_{t+1}={X}_{t}-a\beta\:t\left({X}_{t}^{mean}-a\beta\:t{X}_{t}\right)$$
(13)

Parameter \(\:\beta\:t\) is a random sum distribution. \(\:{X}_{t}^{mean}\) as demonstrated in Eq. (14), is the population’s average location information at the tth iteration.

$$\:{X}_{t}^{mean}=\frac{1}{N}\sum\:_{i=1}^{{N}_{p}}{X}_{i}$$
(14)

Parameter \(\:{N}_{\text{p}}\) is the number of populations. Important for individuals’ iterative updates, the average location data of the population dictates the direction of evolution. People iteratively updating in an irregular Brownian search are more likely to evade the local extremum, and the population is compelled to seek optimisation in a range near the global optimum.

(3) Landing phase.

Development is the main objective of the DO algorithm. The third and last step is for the dandelion seeds to settle anywhere they like. The procedure is predicted to join to global optimal solution, which represents the estimated place where seeds have the best chance of survival, as the increases. Hence, the algorithm can meet to the global local individuals after getting the estimated place where the optimal most likely to develop. The final procedure optimal key with evolution of the population; the expression is Formula (15).

$$\:{X}_{t+1}={X}_{elite}=Levy\left(\lambda\:\right)a\left({X}_{elite}-{X}_{t}\delta\:\right)$$
(15)

Parameter \(\:{X}_{elite}\) is the optimal site of seed repetition. \(\:Levy\:\left(\lambda\:\right)\) is an improved local search capability made possible by a Levy flight function. To prevent overexploitation and ensure correct convergence to the global optimum, the parameter δ is a linearly rising function with a value 0 and 2. The total number of repetitions is denoted by T.

$$\:Levy\left(\lambda\:\right)=s\omega\:\sigma\:/{\left|t\right|}^{\frac{1}{\beta\:}}$$
(16)
$$\:\delta\:=2t/T$$
(17)

This paper uses 1.5 as the value of the random parameter β, which can take on values between 0 and 2. S is assigned a value of 0.01. The parameters w and t can take values between 0 and 1.

B) Dandelion Algorithm Optimized by PSO (PSODO) Particle Swarm Optimization.

A basic algorithm model was developed by James Kennedy and Russell Eberhart, who were motivated by bird foraging behavior. Over the years, this model was refined and eventually became known as the Particle Swarm Optimization (PSO) algorithm36,37. The concept of Particle Swarm Optimization (PSO) first emerged from the study of how birds locate food. Through the exchange of collective information, birds can identify the optimal destination. Consider this hypothetical scenario:

Birds forage randomly in a forest, searching for the area with the highest concentration of food or the only available food source in a given region (analogous to the optimal solution in an optimization problem). The challenge is that none of the birds can see the food; they can only sense its approximate location. Each bird follows a unique flight path while keeping track of the location where it found the most food. Since every bird in the flock shares information about food locations and quantities, the flock as a whole can determine whether it has found the best possible food source (i.e., the optimal solution). Over time, information spreads within the flock about the most promising location, leading all birds to gradually converge around the best food source. This represents the process of solving an optimization problem—once the best solution is found, the swarm converges.

PSO relies on effective initialization to avoid getting trapped in local optima. However, it remains a simple, easy-to-implement algorithm with few parameters and fast convergence, making it highly efficient for solving high-dimensional optimization problems.

The PSO algorithm revises regulations. To begin, below is the formula (18) that updates the velocity of every particle in PSO:

$$\:{V}_{ij}=\omega\:.{V}_{ij}+{c}_{1}.{r}_{1}.\left({P}_{ij}-{X}_{ij}\right)+{c}_{2}.{r}_{2}.({G}_{j}-{X}_{ij})$$
(18)

Parameter Vij is the dimension j; \(\:{X}_{ij}\) is the site of the particle j; limit \(\:{P}_{ij}\) ϋ determines the inertia weight, which maintains the particle’s motion inertia and allows it to explore new areas; is the particle’s individual optimal site, which is its global optimal position; A couple of acceleration factors are c1 and c2.; \(\:{r}_{1}\) and \(\:{r}_{2}\) are random statistics; \(\:{r}_{1}\) and \(\:{r}_{2}\) are those that fall within the range of the rand () function, which is [0, 1]. In Formula (19), to have the following location update:

$$\:{X}_{ij}={X}_{ij}={V}_{ij}$$
(19)

C) Hybrid Strategy of PSODO Algorithm.

By combining the features of Dandelion Optimization (DO) with Particle Swarm Optimization (PSO), the PSODO algorithm effectively integrates both methods to achieve optimal results. In the PSO phase, the algorithm directs the particle search toward the global optimal position (g_Best) using the velocity update formula, which incorporates inertia variables. During the DO phase, the location update rules are applied, incorporating a random selection procedure. This phase leverages the inherent randomness of the DO algorithm to enhance search diversity and introduce varied update methods.

By utilizing random numbers, the algorithm further increases search unpredictability, enhancing the DO component’s random selection and update techniques. This randomization mechanism strengthens the algorithm’s global search capability, helping it escape local optima and explore a broader solution space. Additionally, to facilitate local search, a deceleration technique is introduced. PSO is initially applied to accelerate particle movement across the entire search space, promoting global exploration and increasing the likelihood of finding the optimal solution. In the subsequent phase, particle velocities are gradually reduced, allowing for precise fine-tuning near the optimal solution. This step improves accuracy and ensures that the final solution is highly optimized.

The proposed MT model, through its structured fine-tuning stages, is specifically designed to address distribution discrepancies caused by domain shifts across different news sources and contexts. The encoder phase captures deep semantic representations, even in text with altered sentiments or word substitutions, while the decoder focuses on generating coherent summaries aligned with the contextual meaning. By fine-tuning bias parameters, the model further adapts to dataset-specific language patterns and manipulations.

Furthermore, the integration of reinforcement learning with the PSODO algorithm plays a critical role in detecting multimodal forgery details. PSO’s global search facilitates exploration of diverse feature representations across text and image modalities, while DO’s adaptive local search enhances sensitivity to subtle modifications such as facial changes in images or sentiment flips in text. The ROUGE-1 reward function ensures that the summaries retain crucial semantic elements indicative of manipulation. Together, these mechanisms allow the model to robustly capture cross-modal inconsistencies and subtle forgery traits across varied data domains.

Results and discussion

This section presents detailed experimental results, demonstrating the superiority of our proposed approach. It begins with an introduction to the experimental setup, followed by a discussion of the findings.

Experimental setup

This study considers two distinct datasets from38to evaluate the proposed approach: the DGM4 dataset39and the Fakeddit dataset40.

Description of DGM4

The DGM4 dataset is a large-scale multimodal manipulation resource, containing approximately 77,000 pristine pairs and 152,000 manipulated pairs, totaling 230,000 image-text paired samples. It was recently made publicly available for fake news detection.

  • Image manipulation in DGM4 includes techniques such as face swapping and changing facial expressions.

  • Text manipulation involves phrase replacement and text sentiment modification.

The DGM4 dataset was constructed using data from the Visual News dataset, which aggregates news articles from various news agencies. Due to differences in regional coverage, topic focus, and language style, there are significant distribution disparities across sources (for word cloud analysis, refer to Supplementary Materials).

To model open-world domain-shift scenarios, the DGM4 dataset was divided into four subsets, based on news sources: BBC, The Guardian, USA TODAY (USA), and The Washington Post (Wash.). The distribution of data across these groups is presented in Table 1.

Table 1 The data of the four subsets within the DGM4 dataset branded by the news sources.
Full size table

Description of fakeddit

The Fakeddit dataset is curated from multiple subreddits on the Reddit platform, containing diverse content ranging from text-based posts to multimodal data. In this study, we follow the official dataset partition and only utilize multimodal samples with the 2-way categorization for classification. To improve data quality, we preprocess the dataset by removing excessively short text samples, as such texts often lack sufficient information for semantic inconsistency detection. For our task, we use only the test set for cross-dataset evaluation.

Evaluation metrics

We treat multimodal fake news detection as a classification task. Following previous works, we apply the Area Under the Characteristic Curve (AUC) and the Accuracy Score (ACC) as our primary evaluation metrics.

Implementation details

Our research relies on the encoder obtained from ImageBind-Huge41 to support feature extraction for both images and text. Additionally, we use Vicuna-7B as the linear layer-connected inferential LLM. The model is initialized using the instruction-tuned checkpoint provided by PandaGPT.

For image preprocessing:

  • Each image used for training is resized to 224 × 224 and randomly flipped horizontally.

  • Random perturbation techniques such as JPEG compression and Gaussian blur are applied42.

  • From the image encoder’s eighth, sixteenth, twenty-fourth, and thirty-second layers, the multi-level cross-modal reasoning module extracts intermediate patch features.

When training on the BBC subset, we set:

  • The number of epochs to 10.

  • The learning rate to 1.5e-5 with a batch size of 16.

  • The one-cycle cosine learning schedule and linear warm-up are implemented.

All experiments are conducted using four NVIDIA GeForce RTX 3090 GPUs running PyTorch.

Environment Setup:

  • Python version: Python 3.6.3 (installed via PyPI).

  • Virtual environment: Anaconda.

  • Deep learning framework: TensorFlow 1.13 (for complex classification tasks).

Libraries Setup:

  • Scikit-learn – for fundamental machine learning tasks.

  • OpenCV – for image recognition (IR) tasks.

  • NumPy and Pandas – for data processing and manipulation.

  • Seaborn and Matplotlib – for graphical representation of findings.

Validation analysis of proposed model on datasets

Table 2 presents the validation evaluation of the proposed model based on various metrics across two different datasets, using 60% of the data for training and 40% for testing.

Table 2 Analysis of proposed classifier for 60%−40%.
Full size table

Table 2 presents an analysis of the proposed classifier’s performance using a 60%−40% data split on the DGM4 and Fakeddit datasets. For DGM4, the model achieves a loss of 0.2957 and an accuracy of 86.74%, with a training time of 36 min and 11 s and a prediction time of 58 s. In contrast, for Fakeddit, the model outperforms DGM4, achieving a lower loss of 0.2386 and a higher accuracy of 90.42%, with a significantly faster training time of 17 min and 23 s and a quick prediction time of 11 s. These results highlight the efficiency and effectiveness of the proposed classifier, particularly on the Fakeddit dataset, where it demonstrates superior accuracy and faster computation.

Comparative analysis of projected model with existing techniques

Figures 2, 3 and 4 present a comparative analysis of the proposed model on the two selected datasets, evaluated across various metrics.

Fig. 2
figure 2

Analysis of projected model for timing analysis.

Full size image

The performance comparison of algorithms on Dataset 1 is evaluated based on multiple metrics. The proposed model outperforms other algorithms across most metrics, achieving the highest values for correctly classified instances (84.83%), Kappa statistic (0.810), Precision (0.849), Recall (0.848), F-measure (0.848), and ROC Area (0.974). Additionally, the proposed model is the fastest, with a runtime of 108.09 s. In comparison, CNN + LSTM24and ELM have runtimes of 154.02 s and 193.14 s, respectively. While CNN31 achieves a better runtime of 122.72 s, it falls behind in other performance metrics compared to the proposed model. Although the ELM model performs well in certain metrics, it has a lower ROC Area (0.888), indicating its limitations. Overall, the proposed model demonstrates superior accuracy, efficiency, and effectiveness.

Fig. 3
figure 3

No of correctly predicted samples.

Full size image

The proposed model outperforms other algorithms across all performance metrics for Dataset 2. It achieves the highest correctly classified instances (98.9%), Kappa statistic (0.986), Precision (0.989), Recall (0.989), and ROC Area (0.996). Additionally, it has the shortest running time of 92.02 s, highlighting its efficiency and speed. Among other models, CNN31and ELM perform relatively well, with correctly classified instances of 80.83% and 80.67%, respectively, along with comparable values for Precision, Recall, and F-measure. However, their ROC Area values are lower, particularly for ELM (0.879), indicating reduced classification robustness. CNN + LSTM24 has the lowest performance in this comparison, with 78.67% correctly classified cases, a Kappa statistic of 0.733, and a longer computation time of 102.01 s. Overall, the proposed model demonstrates exceptional classification accuracy and computational efficiency, making it the best choice for Dataset 2.

Fig. 4
figure 4

Graphical Comparison of proposed model.

Full size image

In addition to Accuracy and AUC, we evaluated the proposed model and baseline methods using Precision, Recall, F1-Score, and Specificity. Table 3 presents these metrics for both DGM4 and Fakeddit datasets. The proposed MT + PSODO model achieves superior performance across all metrics, highlighting its ability to detect fake news with high reliability and balanced sensitivity and specificity.

Table 3 Comprehensive evaluation metrics across datasets.
Full size table

Comparative analysis of proposed model on image and pixel based analysis

The proposed representation’s efficiency is tested based on image based and pixel-based input data on two input datasets that is in Tables 4 and 5.

Table 4 Classification accuracy besides running period for image block input representations.
Full size table

This section compares accuracy and computational performance for image block input models applied to the DGM4 and Fakeddit datasets. For DGM4, the proposed model achieves the highest Overall Accuracy (OA) of 86.11% and a Kappa value of 0.8063, outperforming CNN (79.81%, 0.7215) and CNN + LSTM (85.86%, 0.805). It has a moderate training time of 108 min and 13 s, which is slightly higher than CNN but lower than CNN + LSTM, with a testing time of 14 s. For Fakeddit, the proposed model again achieves the highest OA (91.21%) and Kappa value (0.8615), surpassing CNN + LSTM (90.76%, 0.8562) and CNN (85.93%, 0.8034). However, its training time is significantly higher (239 min and 46 s) compared to both CNN and CNN + LSTM, while its testing time (14 s) remains competitive. These results validate the proposed model’s superior accuracy and robustness, particularly for Fakeddit, albeit with a trade-off in training efficiency.

Table 5 Classification accuracy besides running time for per-pixel input representations.
Full size table

This section presents a comparison of classification accuracy and computational efficiency for per-pixel input models applied to two datasets: DGM4 and another dataset. For the DGM4 dataset, the proposed model achieves a slightly higher Overall Accuracy (OA) of 86.90% compared to CNN (86.89%), with a marginal difference in the Kappa statistic (0.8158 vs. 0.8165). However, the proposed model significantly reduces training and testing times, requiring only 140 min and 26 s for training and 11 s for testing, compared to CNN’s 226 min and 42 s for training and 15 s for testing.

For the other dataset, the proposed model again outperforms CNN in OA (85.86% vs. 84.90%) and Kappa (0.8024 vs. 0.7884). However, it requires a considerably longer training time (1205 min and 29 s) compared to CNN (136 min and 33 s) and has a slightly longer testing time (27 s vs. 9 s). These results highlight the proposed model’s improved accuracy and efficiency for DGM4, while trading off training time for better accuracy and robustness in the second dataset.

Comparative analysis of proposed model with existing techniques on combined dataset

Figure 5 presents a comparative analysis of the proposed model with existing techniques for combined datasets, evaluated across various training and testing ratios.

Fig. 5
figure 5

Visual analysis of proposed model on Combined dataset.

Full size image

The performance metrics for Dataset 1 across two splits (80 − 20 and 70 − 30) validate the efficiency of the classifiers in average correct classification. In the 80 − 20 split, the proposed model achieves the highest sensitivity (61.9%) and specificity (92.8%), demonstrating its superior ability to recognize true positives and negatives compared to CNN + LSTM (53.1% sensitivity, 83.3% specificity). The proposed model also leads in accuracy (89.4%) and average correct classification (79.4%), outperforming other methods like CNN (94.4% accuracy, 76.1% classification). In the 70 − 30 split, the proposed model again outperforms other classifiers, achieving the highest sensitivity (85.04%), specificity (99.2%), and accuracy (93.7%), while LSTM follows closely with slightly lower values (83.6% sensitivity, 97.2% specificity, 95.5% accuracy, and 91.4% classification).

These results highlight the proposed model’s ability to handle data effectively, delivering reliable and balanced performance compared to traditional classifiers such as AE, DBN, and CNN.

Discussion

The proposed MT + PSODO framework consistently outperforms conventional detectors (CNN, LSTM, CNN + LSTM) and state-of-the-art LVLMs (e.g., ImageBind, PandaGPT) across multiple metrics. Specifically, it achieves higher detection accuracy (up to 90.42%) and superior AUC (up to 0.904) on the Fakeddit dataset and maintains robustness on DGM4, which includes high domain variation. This performance is primarily attributed to three core innovations:

  • Three-Phase Fine-Tuning of the Modified Transformer (MT): The staged tuning process (encoder → decoder → bias) enables the model to deeply encode semantic features while generating coherent and contextually relevant summaries. This structure helps the model adapt to domain shifts, which are common in real-world fake news across regions and sources.

  • Reinforcement Learning with PSODO Optimization: By optimizing with a hybrid of Particle Swarm Optimization (PSO) and Dandelion Optimization (DO), the model balances global exploration and local refinement, thus avoiding convergence to suboptimal local minima and improving the policy’s ability to detect subtle forgeries and manipulated semantics.

  • Reward-Based Summarization Training: Using ROUGE-1 as a reward function aligns the model’s summarization outputs with human-like comprehension, ensuring that key manipulated or contradictory statements are captured effectively.

Compared to LVLMs that operate with generalized embeddings, our approach introduces task-specific reinforcement that captures localized anomalies and modality-specific inconsistencies. For instance, in image-text mismatches such as sentiment flips or image swaps, the MT + PSODO model demonstrates stronger contextual grounding and higher detection sensitivity. These architectural and training design choices directly contribute to the system’s improved generalization across datasets and domains, making it more suited for real-world deployment in misinformation monitoring systems.

Conclusion

This study aimed to develop a robust and advanced methodology to enhance text summarization for fake news detection. To achieve this, we leveraged the mT5 Transformer model, known for its adaptability and efficiency across various natural language processing tasks, as the foundation of our approach. Additionally, we integrated the PSODO algorithm for reinforcement learning and implemented a three-stage fine-tuning procedure, ensuring improved contextual awareness in automated Persian text summarization. Extensive experiments conducted on multiple datasets demonstrated the effectiveness of our proposed model, achieving superior cross-domain performance compared to existing methodologies. The results confirm that our approach enhances summarization quality, improves semantic consistency, and ensures higher classification accuracy in fake news detection.

Future work

Despite its effectiveness, several areas can be explored for further improvement. First, incorporating additional modalities (such as audio and video) and fully utilizing their cross-modal features can further enhance summarization quality. Second, optimizing parameter selection remains an open challenge; future work could explore reaction-diffusion neural networks, fault-tolerant iterative learning control, or cooperative-competitive neural networks to refine model tuning. Moreover, investigating how entity features contribute to feature representation and interaction presents another promising direction. Additionally, integrating prior knowledge and dissemination systems could further enhance the generalization and real-world applicability of fake news detection models. Future studies will explore these enhancements to advance the field of automated fake news summarization and detection.