SESKStuff Everyone Should Know

Games · Tech, AI & Security

Stuff Every AI Expert Should Know

Machine learning, neural networks, large language models, training, evaluation and AI safety, explained.

Start Stuff Every AI Expert Should Know ⚡ Today's sprint

Print the first 50 cards (free)Get the whole test as a printable PDF: $3 (pre-order)

Free to start, no sign-up. 170 cards so far. One Knowledge Score and day streak across all your courses.

Try 3 cards

  1. How do AI, machine learning and deep learning fit together?
    Show the answer

    Deep learning is inside machine learning, inside AI. AI is the big umbrella. Machine learning is the part of AI where systems learn from data. Deep learning is a subfield of machine learning built around neural networks.

  2. What did Alan Turing call his 1950 test of machine intelligence?
    Show the answer

    The Imitation Game. In his 1950 paper, Turing proposed a game where an interrogator chats with a person and a machine and tries to tell which is which. He called it the imitation game.

  3. Turing predicted that by about 2000, judges would ID the machine at most what percent of the time?
    Show the answer

    70 percent, after five minutes. In 1950 Turing guessed that in about fifty years, an average interrogator would have no more than a 70 percent chance of spotting the machine after five minutes of questions.

All 170 practice questions

Each one has the answer, why it's right, and the official sources. Or play them as a game and track what you've learned.

Show all 170 questions
  1. How do AI, machine learning and deep learning fit together?
  2. What did Alan Turing call his 1950 test of machine intelligence?
  3. Turing predicted that by about 2000, judges would ID the machine at most what percent of the time?
  4. What 1956 event is called the birth of AI as a field of research?
  5. Which approach represents the original top-down AI paradigm?
  6. What makes machine learning "supervised"?
  7. What kind of data does unsupervised learning usually train on?
  8. In reinforcement learning, what tells the agent how well an action went?
  9. In a model predicting house prices from size and bedrooms, what is the label?
  10. What makes a regression model different from a classification model?
  11. In machine learning, what is inference?
  12. Which data split is saved for the final check after tuning on the validation set?
  13. A model scores great on its training data but badly on new data. What is that called?
  14. A model does poorly even on its own training data. What is the likely problem?
  15. Which term measures error from a model's assumptions, even with infinite training data?
  16. Training error is far lower than test error. Which problem does that point to?
  17. What does L2 regularization penalize?
  18. Which regularization can push useless features' weights to exactly zero?
  19. With early stopping, when do you end training?
  20. Which loss function punishes big errors, like outliers, more heavily: MSE or MAE?
  21. In gradient descent, which way does each step move the weights?
  22. What happens if the learning rate is set too high?
  23. With 1,000 training examples and a batch size of 50, how many iterations make one epoch?
  24. If 1 percent of cases are positive, what accuracy does always guessing negative score?
  25. A model flags 200 emails as spam; 150 really are. What is its precision?
  26. Which metric answers: of all the real positives, how many did the model catch?
  27. The F1 score combines precision and recall using which kind of average?
  28. What does a confusion matrix show for a classifier?
  29. What AUC (area under the ROC curve) does a model that just guesses randomly get?
  30. How is a random forest built?
  31. Why are decision trees called a "white box" model?
  32. How does k-nearest neighbors classify a new point?
  33. What must you tell the k-means algorithm before it starts?
  34. In 1958, what did Rosenblatt's perceptron learn to tell apart after 50 trials?
  35. Which 1969 book by Minsky and Papert sharply criticized perceptron research?
  36. Which 1973 UK report helped cause a sharp drop in AI research funding?
  37. About how many parameters did AlexNet, the 2012 ImageNet breakthrough network, have?
  38. What did AlexNet's authors use to reduce overfitting in its fully connected layers?
  39. ImageNet's millions of labeled images are organized by which word hierarchy?
  40. What does the "deep" in deep learning refer to?
  41. Which of these is a hyperparameter you set, not something the model learns?
  42. Raising a classifier's threshold usually does what to false positives and false negatives?
  43. What calculation does a basic artificial neuron perform before activation?
  44. What can a neuron's bias change even when its inputs stay fixed?
  45. Why place nonlinear activation functions between neural network layers?
  46. What does the ReLU activation return for an input x?
  47. Into what open interval does the logistic sigmoid map real inputs?
  48. Where is a neural network's hidden layer located?
  49. Which algorithm makes gradient descent practical for multilayer neural networks?
  50. What happens to layers whose training gradients vanish?
  51. In a CNN, what does a neuron's receptive field describe?
  52. What lets a recurrent neural network carry information through a sequence?
  53. How does backpropagation through time treat a recurrent network?
  54. Why are ordinary RNN time steps difficult to parallelize within one sequence?
  55. What does a learned embedding assign to each input token?
  56. What property does cosine similarity ignore when comparing two embeddings?
  57. Which 2017 paper introduced the Transformer architecture?
  58. Which two sequence mechanisms did the original Transformer dispense with?
  59. What makes an attention mechanism self-attention?
  60. After query-key scores become weights, what does attention combine?
  61. By what quantity does scaled dot-product attention divide query-key dot products?
  62. What is a key benefit of using multiple attention heads?
  63. Why does a Transformer add positional information to token embeddings?
  64. How did the original Transformer construct its fixed positional encodings?
  65. What does a Transformer encoder produce from an input sequence?
  66. Which statement correctly distinguishes model parameters from tokens?
  67. Why is one written word not always one language-model token?
  68. Why do GPUs and TPUs fit neural network workloads especially well?
  69. What sat at the heart of Google's first published datacenter TPU?
  70. What mathematical pattern did neural language-model scaling laws observe for test loss?
  71. Under the Chinchilla study's compute-optimal rule, what should double with model size?
  72. What is the main idea of mixed-precision neural network training?
  73. Why can mixed-precision training keep an FP32 master copy of weights?
  74. What does dropout randomly remove during a training step?
  75. How is standard dropout commonly handled at inference time?
  76. Across what values does layer normalization compute its statistics?
  77. Does layer normalization compute differently at training and inference time?
  78. What does transfer learning reuse from a previously trained model?
  79. What changes when a pretrained model is fine-tuned rather than only used frozen?
  80. What two processes form the basic idea of a denoising diffusion model?
  81. Which two models compete in a generative adversarial network?
  82. Why does an autoregressive Transformer decoder mask later positions?
  83. In Transformer cross-attention, where do queries and key-value pairs come from?
  84. How does a convolutional layer reduce parameters across image locations?
  85. At its core, what is a large language model trained to do?
  86. How does byte pair encoding tokenization handle a rare word the model has never seen?
  87. What does an LLM's context window measure?
  88. In a long prompt, where is key information most likely to be missed by a language model?
  89. What kind of learning is usually used in the first, pretraining phase of an LLM?
  90. What is instruction tuning?
  91. In the InstructGPT study, which answers did people prefer: a 1.3B RLHF model's or the 175B GPT-3's?
  92. In RLHF as used for InstructGPT, what did human labelers provide for the reinforcement learning step?
  93. What does Direct Preference Optimization (DPO) replace RLHF's reinforcement learning with?
  94. In Constitutional AI, what is the only human oversight used to judge harmfulness?
  95. What happens when you raise a language model's temperature setting?
  96. What does nucleus (top-p) sampling do when picking the next token?
  97. If a language model always picks the most likely words, what kind of text tends to come out?
  98. In generative AI, what is a hallucination?
  99. What term does NIST's generative AI risk profile use for AI hallucinations?
  100. A 2025 paper by OpenAI researchers says hallucinations persist partly because most evaluations reward what?
  101. What two kinds of memory does retrieval-augmented generation (RAG) combine?
  102. To search millions of embeddings fast, what do vector indexes like HNSW usually trade away?
  103. In word2vec, what is vector(King) minus vector(Man) plus vector(Woman) closest to?
  104. When you put a few examples in a prompt (few-shot prompting), what happens to the model's weights?
  105. What is a common use of the system prompt in a chat model API?
  106. What is chain-of-thought prompting?
  107. How does the self-consistency method improve chain-of-thought answers?
  108. Toolformer taught a language model to call tools. Which weak spots of LMs did that target?
  109. With function calling, who actually runs a tool that the developer defined?
  110. What does the ReAct method have a language model do?
  111. What separates an AI agent from an AI workflow, in Anthropic's definition?
  112. What was OpenAI's CLIP trained to predict on 400 million internet images?
  113. How does a Vision Transformer (ViT) feed an image into a Transformer?
  114. In Hinton's knowledge distillation, how does the big teacher model produce soft targets for the student?
  115. DistilBERT made BERT 40 percent smaller. About how much of its language understanding did it keep?
  116. Quantizing a 16-bit model's weights to 8-bit integers does what to the memory needed for inference?
  117. QLoRA made it possible to fine-tune a 65 billion parameter model on what hardware?
  118. How does LoRA fine-tune a large model so cheaply?
  119. What makes an AI model open-weight?
  120. What is benchmark contamination for a language model?
  121. Researchers wrote GSM1k, fresh math problems like GSM8K. What happened to some models' scores?
  122. Which latency measure matters most for how fast a streaming chatbot feels?
  123. Speculative decoding speeds up a big model by drafting tokens with a smaller one. What happens to the output?
  124. When serving an LLM, what memory grows with each request's length and limits how many requests fit at once?
  125. How did BERT's pretraining differ from a left-to-right GPT-style model?
  126. LLM APIs usually charge per request based on what?
  127. If a reply hits the max_tokens limit, what happens?
  128. The MMLU benchmark tests language models across how many subjects, from math to law?
  129. How can an unrepresentative training sample bias an AI system?
  130. Where can systemic AI bias exist besides the dataset?
  131. Can every mathematical fairness condition usually be satisfied at once?
  132. How does NIST distinguish explainability from interpretability?
  133. Should one AI explanation format serve every audience?
  134. What does a membership inference attack try to learn?
  135. Why can overfitting increase training-data privacy risk?
  136. Under current US guidance, is purely AI-generated material copyrightable?
  137. Does using AI assistance automatically block copyright protection?
  138. Is every use of copyrighted work for AI training automatically fair use?
  139. What causes specification gaming in an AI system?
  140. What are the two broad targets of reward tampering?
  141. What is the main purpose of AI red teaming?
  142. What weakness does a direct prompt injection exploit?
  143. Where does an indirect prompt injection hide?
  144. What does an AI jailbreak try to get around?
  145. Can one prompt filter fully prevent every injection attack?
  146. What should a model card communicate?
  147. What does a datasheet for a dataset document?
  148. Why document a dataset's intended use and context?
  149. What makes GOVERN different from the other NIST AI RMF functions?
  150. What is the central job of MAP in the NIST AI RMF?
  151. What happens in the MEASURE function of the NIST AI RMF?
  152. What does MANAGE do with mapped and measured AI risks?
  153. Is the NIST AI RMF Core a fixed ordered checklist?
  154. How many risk levels does the EU AI Act approach define?
  155. What happens when an AI system is classified as high risk in the EU?
  156. What does the EU transparency-risk tier mainly require?
  157. What special AI Act rules apply to minimal-risk AI systems?
  158. Does adding a human reviewer automatically make an AI system safe?
  159. Under the EU AI Act, what must deployers do with covered deepfakes?
  160. What is the purpose of machine-readable marks on AI-generated content?
  161. Does a valid C2PA manifest prove that media is true?
  162. Does missing C2PA metadata prove that content is fake?
  163. Why can equal AI training compute produce different carbon emissions?
  164. Why can inference consume more total energy than one training run?
  165. What is production data drift?
  166. Why monitor model quality after deployment?
  167. What must be versioned to reproduce an ML result?
  168. What does an online A/B test compare for two model versions?
  169. When should a team prefer a simple rule over machine learning?
  170. Why does a predictive correlation not prove causation?