2026

Fake News Detection

NLP-based fake news detection system with full pipeline: TF-IDF baseline, BERT/RoBERTa/DeBERTa fine-tuning, cross-dataset evaluation, and SHAP/LIME interpretability analysis. Trained on the LIAR dataset (12,800 political statements).

Weighted F1 of LR, XGBoost, RoBERTa and Ensemble, in-domain on LIAR then out of distribution on FNN — In-domain on LIAR, then out of distribution: transformer F1 collapses next to the baselines

Overview

This NLP specialization project offers a complete fake news detection system. The pipeline covers exploratory analysis, preprocessing, baseline models (Naive Bayes, Logistic Regression, XGBoost), transformer model fine-tuning (BERT, RoBERTa, DeBERTa), evaluation on an external dataset (out-of-distribution generalization), and interpretability analysis with SHAP and LIME. The LIAR dataset contains 12,800 political statements labeled by PolitiFact fact-checkers. The project also includes an ethical bias analysis.

Exploring the LIAR dataset

Two bar charts: the six LIAR classes (true to pants-fire) and the binary FAKE / REAL split on the training set — Distribution of the six LIAR classes and of the binary version on the training set

The LIAR dataset holds 12,800 political statements labeled by PolitiFact fact-checkers on six truth levels. The pipeline starts with exploratory analysis and preprocessing, with a six-class task and a binary version.

Baselines versus transformers

Two bar charts of weighted F1 on the test set, six-class and binary, for LR, LightGBM, BiLSTM, RoBERTa, four ensemble configurations, a soft vote and a credit-history variant — Weighted F1 on the test set, six-class and binary: baselines, RoBERTa and ensembles side by side

More than six models compared on the same test set: Naive Bayes, logistic regression and XGBoost baselines on TF-IDF, then fine-tuned BERT, RoBERTa and DeBERTa, plus ensembles. The pipeline is progressive: the TF-IDF baseline is the reference the transformers are measured against.

LIME interpretability

Two LIME horizontal bar charts in red and green listing the words influencing RoBERTa, for a correctly classified FAKE statement and a FAKE statement classified REAL — LIME on binary RoBERTa: word contributions on a correct prediction and on an error

To make model decisions readable, every prediction can be explained with SHAP and LIME. Here LIME shows the words pushing RoBERTa toward FAKE or REAL, on a correctly classified statement and on an error.

Out-of-distribution generalization

Bar chart of out-of-distribution weighted F1 on FakeNewsNet and PHEME for LR, text-only RoBERTa and Ensemble, with LR clearly ahead — Out-of-distribution weighted F1 on FakeNewsNet and PHEME: LR, RoBERTa and ensemble

Models trained on LIAR are evaluated on external datasets never seen in training, to measure out-of-distribution robustness. The result is counter-intuitive: logistic regression holds up better than RoBERTa and the ensemble.

Bias and ethics

Two charts: RoBERTa accuracy on the ten hardest topics, and per-speaker bias, red for FAKE over-detection, green for under-detection — RoBERTa bias audit: accuracy per topic and FAKE detection gap per speaker

A bias audit is built into the evaluation pipeline: accuracy per topic and the gap between predicted and actual FAKE rate per speaker, to spot where the model over- or under-detects.

Gallery

SHAP summary plot of the binary LightGBM, 25 most influential features — SHAP summary of the binary LightGBM: the 25 most influential features

Four RoBERTa confusion matrices depending on how credit history is fused, on the test set — RoBERTa and credit history: four fusion variants, confusion matrices on the test set

Challenges

Multi-class classification of political statements with contextual nuances
Model generalization on unseen external datasets
Model decision interpretability to ensure trust
Detection and analysis of prediction biases

Solutions

Progressive pipeline: TF-IDF baseline → BERT/RoBERTa/DeBERTa fine-tuning
Cross-dataset evaluation to measure out-of-distribution robustness
SHAP and LIME analysis for prediction explainability
Ethical bias audit integrated in the evaluation pipeline

Results

5 notebooks covering the complete EDA → interpretability pipeline
Comparison of 6+ models (baseline + transformers)
SHAP/LIME interpretability analysis on predictions
Generalization evaluation on external dataset

Technologies

PythonPyTorchBERTTransformersSHAPLIMEScikit-learnXGBoost

Next project