Chaitin, the inventor of algorithmic information theory, presents in this book the strongest possible version of Gödel's incompleteness theorem, using an information theoretic approach based on the size of computer programs. One half of the book is concerned with studying the halting probability of a universal computer if its program is chosen by tossing a coin. The other half is concerned with encoding the halting probability as an algebraic equation in integers, a so-called exponential diophantine equation.

This book is the first one that provides a solid bridge between algorithmic information theory and statistical mechanics. Algorithmic information theory (AIT) is a theory of program size and recently is also known as algorithmic randomness. AIT provides a framework for characterizing the notion of randomness for an individual object and for studying it closely and comprehensively. In this book, a statistical mechanical interpretation of AIT is introduced while explaining the basic notions and results of AIT to the reader who has an acquaintance with an elementary theory of computation. A simplification of the setting of AIT is the noiseless source coding in information theory. First, in the book, a statistical mechanical interpretation of the noiseless source coding scheme is introduced. It can be seen that the notions in statistical mechanics such as entropy, temperature, and thermal equilibrium are translated into the context of noiseless source coding in a natural manner. Then, the framework of AIT is introduced. On this basis, the introduction of a statistical mechanical interpretation of AIT is begun. Namely, the notion of thermodynamic quantities, such as free energy, energy, and entropy, is introduced into AIT. In the interpretation, the temperature is shown to be equal to the partial randomness of the values of all these thermodynamic quantities, where the notion of partial randomness is a stronger representation of the compression rate measured by means of program-size complexity. Additionally, it is demonstrated that this situation holds for the temperature itself as a thermodynamic quantity. That is, for each of all the thermodynamic quantities above, the computability of its value at temperature T gives a sufficient condition for T to be a fixed point on partial randomness. In this groundbreaking book, the current status of the interpretation from both mathematical and physical points of view is reported. For example, a total statistical mechanical interpretation of AIT that actualizes a perfect correspondence to normal statistical mechanics can be developed by identifying a microcanonical ensemble in the framework of AIT. As a result, the statistical mechanical meaning of the thermodynamic quantities of AIT is clarified. In the book, the close relationship of the interpretation to Landauer's principle is pointed out.

Algorithmic information theory (AIT), or Kolmogorov complexity as it is known to mathematicians, can provide a useful tool for scientists to look at natural systems, however, some critical conceptual issues need to be understood and the advances already made collated and put in a form accessible to scientists. This book has been written in the hope that readers will be able to absorb the key ideas behind AIT so that they are in a better position to access the mathematical developments and to apply the ideas to their own areas of interest. The theoretical underpinning of AIT is outlined in the earlier chapters, while later chapters focus on the applications, drawing attention to the thermodynamic commonality between ordered physical systems such as the alignment of magnetic spins, the maintenance of a laser distant from equilibrium, and ordered living systems such as bacterial systems, an ecology, and an economy. Key Features Presents a mathematically complex subject in language accessible to scientists Provides rich insights into modelling far-from-equilibrium systems Emphasises applications across range of fields, including physics, biology and econophysics Empowers scientists to apply these mathematical tools to their own research

Algorithmic Information Theory treats the mathematics of many important areas in digital information processing. It has been written as a read-and-learn book on concrete mathematics, for teachers, students and practitioners in electronic engineering, computer science and mathematics. The presentation is dense, and the examples and exercises are numerous. It is based on lectures on information technology (Data Compaction, Cryptography, Polynomial Coding) for engineers.

"Algorithmic information theory (AIT) is the result of putting Shannon's information theory and Turing's computability theory into a cocktail shaker and shaking vigorously", says G.J. Chaitin, one of the fathers of this theory of complexity and randomness, which is also known as Kolmogorov complexity. It is relevant for logic (new light is shed on Gödel's incompleteness results), physics (chaotic motion), biology (how likely is life to appear and evolve?), and metaphysics (how ordered is the universe?). This book, benefiting from the author's research and teaching experience in Algorithmic Information Theory (AIT), should help to make the detailed mathematical techniques of AIT accessible to a much wider audience.

Computability and complexity theory are two central areas of research in theoretical computer science. This book provides a systematic, technical development of "algorithmic randomness" and complexity for scientists from diverse fields.

Information theory and inference, taught together in this exciting textbook, lie at the heart of many important areas of modern technology - communication, signal processing, data mining, machine learning, pattern recognition, computational neuroscience, bioinformatics and cryptography. The book introduces theory in tandem with applications. Information theory is taught alongside practical communication systems such as arithmetic coding for data compression and sparse-graph codes for error-correction. Inference techniques, including message-passing algorithms, Monte Carlo methods and variational approximations, are developed alongside applications to clustering, convolutional codes, independent component analysis, and neural networks. Uniquely, the book covers state-of-the-art error-correcting codes, including low-density-parity-check codes, turbo codes, and digital fountain codes - the twenty-first-century standards for satellite communications, disk drives, and data broadcast. Richly illustrated, filled with worked examples and over 400 exercises, some with detailed solutions, the book is ideal for self-learning, and for undergraduate or graduate courses. It also provides an unparalleled entry point for professionals in areas as diverse as computational biology, financial engineering and machine learning.