10  Summary

This book introduced entropy-based learning through three complementary principles and their shared energy-based substrate:

• Maximum entropy yields the least-biased model consistent with given constraints — grounding exponential families, Boltzmann machines, and a principled family of game-theoretic valuations (Bian et al. 2022).
• Minimum entropy sharpens a capable model’s own predictions, enabling unsupervised elicitation of reasoning (EMPO) in a latent semantic space (Zhang et al. 2025).
• Minimax entropy selects the features worth modelling at all: those whose maximum-entropy model carries the least entropy, simultaneously the shortest, most accurate, and most informative description (Carcamo et al. 2025).

Binding the three together is the free energy \(F = U - TS\): temperature sets the exchange rate between energy and entropy, so the principles are three settings of one dial rather than three separate doctrines. It is also the equation that carried statistical physics into machine learning, from the Hopfield network and the Boltzmann machine through to the evidence lower bound — a lineage recognized by the 2024 Nobel Prize in Physics (The Royal Swedish Academy of Sciences 2024).

This is an early scaffold; future revisions will add worked examples, code, and experiments. For updates, see https://yataobian.com/ and the Blue Whale Lab.

Bian, Yatao, Yu Rong, Tingyang Xu, Jiaxiang Wu, Andreas Krause, and Junzhou Huang. 2022. “Energy-Based Learning for Cooperative Games, with Applications to Valuation Problems in Machine Learning.” International Conference on Learning Representations (ICLR). https://openreview.net/forum?id=xLfAgCroImw.

Carcamo, David P., Nicholas J. Weaver, Purushottam D. Dixit, and Christopher W. Lynn. 2025. “Minimax Entropy: The Statistical Physics of Optimal Models.” Physical Review E 112 (6): 061001. https://doi.org/10.1103/kr9x-q59y.

The Royal Swedish Academy of Sciences. 2024. Scientific Background to the Nobel Prize in Physics 2024: For Foundational Discoveries and Inventions That Enable Machine Learning with Artificial Neural Networks. https://www.nobelprize.org/prizes/physics/2024/advanced-information/.

Zhang, Qingyang, Haitao Wu, Changqing Zhang, Peilin Zhao, and Yatao Bian. 2025. “Right Question Is Already Half the Answer: Fully Unsupervised LLM Reasoning Incentivization.” Advances in Neural Information Processing Systems (NeurIPS). https://arxiv.org/abs/2504.05812.