Entropy in Ergodic Theory and Dynamical Systems

Last modified by Roland Gunesch on Sept 1, 2003. Original by Chris Hillman.

In the mid 1950's, Andrei Kolmogorov imported Shannon's probabilistic notion of entropy into the theory of dynamical systems and showed how entropy can (sometimes) be used to tell whether two dynamical systems are non-conjugate (i.e., non-isomorphic). His work inspired a whole new approach in which entropy appears as a numerical invariant of a class of dynamical systems. Kolmogorov's metric entropy is an invariant of measure theoretical dyamical systems and is closely related to Shannon's source entropy. It was a spectacular vindication of Kolmogorov's ideas when Donald Ornstein showed that metric entropy suffices to completely classify two-sided Bernoulli processes, a basic problem which for many decades appeared completely intractable. (Recently, Selim Tuncel has shown how to classify one-sided Bernoulli processes; this turns out to be quite a bit harder).

In 1961, Roy Adler et al. introduced topological entropy, which is the analogous invariant for topological dynamical systems. It soon turned out that there is a simple relationship between these quantities: maximizing the metric entropy over a suitable class of measures defined on a dynamical system gives its topological entropy.

Here are two expository papers and two on-line (graduate level) textbooks which stress the role of entropy in understanding dynamical systems:

There is a principle in dynamical systems which says that These relations show that entropy can be regarded quite generally as a measure of: Entropy can also be regarded as the fractal dimension of an appropriate compact set.

Here are two papers which discuss these ideas:

There is another general principle which says that Here are two papers which discuss the first and second parts (respectively) of this principle:

Another web resource is

Further Reading

Hundreds of books have already been published on dynamical system theory, of which several dozen at least stress the role of the various kinds of dynamical entropies. Recent offerings include the following:

Back to Entropy on the World Wide Web.