Agreemind ⚖️

Legal AI for everyone.
We build open-source NLP models and tools that help people find and understand potentially risky clauses in Terms of Service (ToS) and similar consumer agreements.

Disclaimer: Agreemind provides informational assistance only and does not provide legal advice.

🎯 Mission

Legal documents are dense and time-consuming. Our goal is to make them more accessible by:

highlighting clauses that commonly reduce user rights,
labeling the type of risk (e.g., unilateral changes, arbitration),
enabling downstream apps to display "risk badges" and evidence-backed highlights.

🧩 What our models do

Our models perform multi-label classification at the sentence/clause level:

Input: a clause (sentence/paragraph)
Output: probabilities over risk categories (one clause can match multiple)

This makes the models suitable for:

clause highlighting in a document viewer,
ranking "most risky" clauses first,
powering a lightweight "risk badge" in a UI.

🏷️ Risk Categories (UNFAIR-ToS)

We currently support 8 types of potentially unfair clauses:

Limitation of liability — Limits the provider's legal responsibility
Unilateral termination — Provider may terminate/suspend without clear cause
Unilateral change — Terms can change with minimal notice or constraints
Content removal — Provider may remove user content at discretion
Contract by using — Agreement is implied by browsing/using the service
Choice of law — Specifies governing law (may disadvantage the user)
Jurisdiction — Specifies dispute venue (may disadvantage the user)
Arbitration — Requires arbitration instead of court

These categories are commonly used in the UNFAIR-ToS / CLAUDETTE research line.

📦 Model Zoo

All models are trained/evaluated on the LexGLUE UNFAIR-ToS benchmark (unfair-tos subset). We report the same metric set across models whenever possible.

Model	Task	Key metric(s)
deberta-unfair-tos-augmented	ToS clause risk classification	F1: 0.96 • Accuracy: 94.12% ⭐
deberta-unfair-tos	ToS clause risk classification	F1: 0.87 • Accuracy: 78.8%
electra-large-unfair-tos	ToS clause risk classification	Accuracy: 77.3%
legalbert-unfair-tos	ToS clause risk classification	Accuracy: 74.9%
modernbert-unfair-tos	ToS clause risk classification	Accuracy: 70.6%
legalbert-large-unfair-tos	ToS clause risk classification	Accuracy: 66.3%

Notes

Accuracy = Exact Match (all 8 labels correct per sample)
F1 = Micro-F1 across all labels
For production use, we recommend tuning per-class thresholds on your domain.
The augmented model was trained with 605 additional synthetic examples for weak classes.

🚀 Quickstart (Transformers)

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_id = "Agreemind/deberta-unfair-tos-augmented"  # Best model
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)

labels = [
  "Limitation of liability",
  "Unilateral termination",
  "Unilateral change",
  "Content removal",
  "Contract by using",
  "Choice of law",
  "Jurisdiction",
  "Arbitration",
]

text = "We may modify these Terms at any time without notice."
inputs = tokenizer(text, return_tensors="pt", truncation=True, padding=True)

with torch.no_grad():
    logits = model(**inputs).logits
probs = torch.sigmoid(logits).squeeze().tolist()

top = sorted(zip(labels, probs), key=lambda x: x[1], reverse=True)[:3]
print(top)

🔒 Responsible use

These models:

may miss important risks,
may produce false positives,
should not be used as a substitute for legal counsel.

Always present outputs as informational signals, ideally with:

the highlighted clause text,
a short non-prescriptive explanation,
a suggestion to review critical terms carefully.

🔗 Links

🌐 Website: https://agreemind.vercel.app
💻 GitHub: https://github.com/agree-mind

📄 License

Models and code are released under the MIT License, unless otherwise stated in individual repositories/models.