Thomas Wolf

Why read Thomas Wolf?

Start with Wolf's Einstein AI essay when a difficult exam score is presented as evidence of scientific discovery. He argues that answering a known question and formulating a useful new question require different tests. It is a proposal for how to evaluate scientific AI, with the benchmark design left as an open research problem.

His coauthored DistilBERT paper gives a concrete earlier example of evaluating a tradeoff. A smaller student model learns from a larger teacher, and the authors compare task scores, parameter counts and inference time. Read the setup and ablations to see which measurements support the compression claim and where performance is lost.

Start with the original

Selected work

1. Essay

   The Einstein AI model ↗

   Does answering a hard question establish scientific discovery? Wolf argues for testing how models ask new questions and challenge assumptions. Read it as his proposed research direction; the essay leaves the evaluation method open.

2. Paper · 2 Oct 2019

   DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter ↗

   What does model compression preserve? This coauthored paper trains a smaller BERT student using a teacher model, then compares language-task scores and inference speed. Inspect the task tables and loss ablations rather than treating one aggregate score as universal capability.

Projects & roles

• Transformers

  An open source model library; Wolf's own site identifies his work on it and links its coauthored research paper.

• Hugging Face

  The company and open model platform Wolf cofounded.