What is Massive Multitask Language Understanding (MMLU)?

Massive Multitask Language Understanding (MMLU)

Massive Multitask Language Understanding (MMLU): A benchmark of multiple-choice questions across fifty-seven subjects, from history to law, used to gauge a model's broad knowledge.

Related terms

Contrastive Language-Image Pre-training (CLIP)
Document understanding
Holistic Evaluation of Language Models (HELM)
Large language model (LLM)
Masked language modelling (MLM)
MMLU-Pro
Program-aided language model (PAL)
Small language model (SLM)

Browse the full glossary →