Contents

• 1. Introduction
  • 1.1 Why entropy?
  • 1.2 Roadmap

Part I — Foundations

• 2. Entropy and the Maximum Entropy Principle
  • 2.1 The many faces of entropy
    • 2.1.1 Information: Shannon entropy
    • 2.1.2 Heat: thermodynamic entropy
    • 2.1.3 Quantum states: von Neumann entropy
    • 2.1.4 One knob, a family: Rényi and Tsallis
    • 2.1.5 Minds: psychological entropy
    • 2.1.6 One form, many readings
  • 2.2 The maximum entropy principle
  • 2.3 Minimum description length and minimax entropy
  • 2.4 Why it matters for learning
• 3. Energy, Entropy, and Free Energy
  • 3.1 Maximum entropy with an energy constraint
  • 3.2 Free energy
  • 3.3 What entropy buys you
    • 3.3.1 Isothermal expansion
    • 3.3.2 The rubber band
    • 3.3.3 Melting
  • 3.4 From spins to networks
  • 3.5 How this reached AI
  • 3.6 The temperature dial
  • 3.7 Why it matters for learning
• 4. Energy-Based Models
  • 4.1 The energy view of inference
  • 4.2 Why learn an energy at all?
  • 4.3 Learning is shaping a landscape
  • 4.4 Strategy one: sample the pull-up
  • 4.5 Strategy two: match slopes, not heights
  • 4.6 Strategy three: learn by telling things apart
  • 4.7 One family, three estimators
  • 4.8 Latent variables, and the free energy again
  • 4.9 The modern landscape
  • 4.10 Where this sits in the book
  • 4.11 Takeaway

Part II — Methods

• 5. Maximum-Entropy Learning
  • 5.1 Graphical models as an explanatory family
  • 5.2 A worked example: energy-based cooperative games
    • 5.2.1 Why this is useful
  • 5.3 Takeaway
• 6. Minimum-Entropy Learning
  • 6.1 Two regimes, and a warning about the name
  • 6.2 From word-level to meaning-level uncertainty
  • 6.3 Entropy minimisation as an objective: a short history
  • 6.4 EMPO: reward agreement, not correctness
  • 6.5 What “reward agreement” actually optimises
  • 6.6 The free-energy view, and why entropy collapses
  • 6.7 The information ceiling
  • 6.8 Schrödinger, read carefully
  • 6.9 Prigogine, and an analogy that does not hold
  • 6.10 So why call it minimum-entropy learning?
  • 6.11 When agreement means truth, and when it does not
  • 6.12 Takeaway
• 7. Minimax-Entropy Learning
  • 7.1 Which features should a model include?
  • 7.2 The minimax entropy principle
  • 7.3 Three equivalent views
  • 7.4 Solving it
  • 7.5 Takeaway
• 8. Energy-Based Reasoning, Guidance, and Refinement
  • 8.1 Guidance
  • 8.2 Refinement
  • 8.3 Reasoning as energy minimization

Part III — Applications

• 9. Entropy-Based Learning for Science
  • 9.1 Two synergistic directions
  • 9.2 Molecular foundation models
  • 9.3 Outlook
• 10. Summary
• References