AI Text Classifier – Human vs AI Detection

Model: alanjoshua2005/AI-text-classifier

1. Project Overview

The AI Text Classifier is a deep learning–based system designed to estimate the likelihood that a given piece of text was generated by an AI model versus written by a human.

The system leverages:

A Transformer-based language model (RoBERTa)
Supervised fine-tuning on a large, diverse corpus of AI- and human-written text
GPU-accelerated training for scalability and performance

The final model outputs probabilistic predictions, making it suitable for real-world use cases such as plagiarism detection, AI content auditing, and academic integrity tools.

2. Use Cases

AI-generated content detection
Academic integrity verification
Content moderation pipelines
AI watermarking support systems
Research on AI-human text differences

3. Dataset

Dataset Used

Name: artem9k/ai-text-detection-pile Source: Hugging Face Datasets

Dataset Description

This dataset contains over 1.3 million text samples labeled as:

"human" → Human-written text
"ai" → AI-generated text

Each entry includes:

text: The content
source: Origin label (human or ai)

Dataset Processing Strategy

To ensure efficient training on limited GPU resources (e.g., Google Colab), the following steps were applied:

Shuffling the dataset for randomness
Sampling 200,000 examples
Label encoding
- human → 0
- ai → 1
Filtering
- Removed text samples shorter than 100 characters
Train–validation split
- 90% training
- 10% validation

4. Model Architecture

Base Model

RoBERTa-base
Transformer encoder with bidirectional attention
Pretrained on large-scale natural language corpora

Classification Head

Sequence classification head
Binary output (human vs ai)
Softmax probability distribution

Key Parameters

Parameter	Value
Max sequence length	256
Batch size	16
Epochs	2
Learning rate	2e-5
Optimizer	AdamW
Weight decay	0.01

5. Tokenization

The model uses AutoTokenizer from Hugging Face Transformers:

Padding: max_length
Truncation: Enabled
Maximum length: 256 tokens
Attention masks generated automatically

This ensures consistent input formatting across training and inference.

6. Training Pipeline

Frameworks Used

PyTorch
Hugging Face Transformers
Hugging Face Datasets
Scikit-learn (metrics)

Training Features

GPU acceleration via CUDA
Mixed precision training (fp16) for speed and memory efficiency
Automatic evaluation at each epoch
Best model selection based on ROC-AUC

Training Strategy

Input Text
   ↓
Tokenizer (RoBERTa)
   ↓
Transformer Encoder
   ↓
Classification Head
   ↓
Probability Output (AI vs Human)

7. Evaluation Metrics

The model is evaluated using:

Accuracy

Measures overall classification correctness.

ROC-AUC (Primary Metric)

Measures how well the model separates AI-generated and human-written text across all thresholds.

ROC-AUC was chosen because:

It is robust to class imbalance
It reflects probabilistic confidence
It is widely used in production classifiers

8. Model Performance

On the validation set:

High ROC-AUC (>0.9) indicating strong separation
Reliable probability calibration on long-form text
Better performance on text lengths > 100 characters

⚠️ Short texts may yield lower confidence and should be handled carefully in production.

9. Model Saving & Deployment

After training, both the model and tokenizer are saved:

trainer.save_model("ai_text_detector_gpu")
tokenizer.save_pretrained("ai_text_detector_gpu")

The final model is uploaded to Hugging Face Hub:

🔗 Model URL https://huggingface.co/alanjoshua2005/AI-text-classifier

Perfect — I’ll integrate this inference logic into your technical documentation properly, while:

✅ Using the Hugging Face model: alanjoshua2005/AI-text-classifier
✅ NOT changing the temperature (TEMPERATURE = 1.5)
✅ Keeping it production-accurate
✅ Writing it in a documentation-ready format

Below is a drop-in documentation section you can paste directly into your technical document or HF README.

10. Inference & AI Likelihood Estimation

This section describes how the trained model is used during inference to estimate the likelihood that a given text was written by an AI versus a human.

The inference pipeline loads the fine-tuned model from Hugging Face Hub and produces probabilistic outputs rather than hard labels. This allows downstream applications to apply custom thresholds depending on risk tolerance.

10.1 Inference Configuration

Model: alanjoshua2005/AI-text-classifier
Max sequence length: 256 tokens
Device: CUDA (GPU) if available, otherwise CPU
Temperature scaling: 1.5 (used for probability stabilization)

Temperature scaling is applied only during inference to smooth overconfident predictions while preserving class separation.

10.2 Inference Code

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

MODEL_NAME = "alanjoshua2005/AI-text-classifier"
MAX_LENGTH = 256
TEMPERATURE = 1.5   # 🔥 probability stabilization (unchanged)

# Device selection
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print("Using device:", device)

# Load tokenizer and model from Hugging Face Hub
tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)
model = AutoModelForSequenceClassification.from_pretrained(MODEL_NAME)
model.to(device)
model.eval()

def predict_ai_likelihood(text: str):
    """
    Returns AI vs Human likelihood percentages for a given input text.
    """
    inputs = tokenizer(
        text,
        return_tensors="pt",
        truncation=True,
        padding="max_length",
        max_length=MAX_LENGTH
    )
    inputs = {k: v.to(device) for k, v in inputs.items()}

    with torch.no_grad():
        logits = model(**inputs).logits
        probs = torch.softmax(logits / TEMPERATURE, dim=1)[0]

    return {
        "human_probability": round(float(probs[0]) * 100, 2),
        "ai_probability": round(float(probs[1]) * 100, 2)
    }

10.3 Example Predictions

human_text = "idk man, salt is cheap and it works. cars rust but people care more about not crashing."
ai_text = "Salt is applied to roads to lower the freezing point of water and improve winter safety."

print("Human:", predict_ai_likelihood(human_text))
print("AI:", predict_ai_likelihood(ai_text))

Sample Output

Human: {'human_probability': 99.74, 'ai_probability': 0.26}
AI:    {'human_probability': 8.55,  'ai_probability': 91.45}

10.4 Interpretation of Results

The model outputs probability scores, not absolute truth.
Higher ai_probability indicates stronger similarity to AI-generated patterns.
The temperature-scaled softmax helps prevent extreme confidence on ambiguous text.
Recommended minimum text length for reliable inference: ≥ 100 characters

10.5 Production Notes

This inference function is suitable for:
- REST APIs (FastAPI)
- Streamlit dashboards
- Batch document analysis
For long documents, chunking into overlapping segments is recommended.
Results should be interpreted as likelihood, not definitive proof.

If you want, next I can:

Merge this into your full documentation PDF
Write a Streamlit inference section
Add a confidence threshold policy for production
Prepare a FastAPI inference endpoint

Just tell me 👍

11. Production Considerations

Strengths

Transformer-based (state-of-the-art)
Probabilistic outputs (not hard labels)
Trained on large-scale data
GPU-optimized training

Limitations

Not a definitive proof of AI usage
Performance drops on very short texts
May require re-training for future AI models

Ethical Disclaimer

This tool estimates likelihood, not certainty. It should not be used as sole evidence for academic, legal, or disciplinary decisions.

12. Future Improvements

Probability calibration (Platt / Isotonic)
Multilingual support
Chunk-based long document inference
Model ensemble (classical + transformer)
Continual learning with new AI models

13. License

MIT License

14. Author

Alan Joshua Hugging Face: alanjoshua2005 Project: AI Text Classification & Detection

If you want, I can:

Convert this into a formal research paper format
Create a system architecture diagram
Write a deployment section (FastAPI / Streamlit)
Optimize this doc for college submission

Just tell me 👌

Downloads last month: 19

Safetensors

Model size

0.1B params

Tensor type

F32

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support