Original by Chris Hillman
(Last modified by Chris Hillman 2 Feb 2001.)
Here is a list of entropy related software, organized into three categories:
Tutorials
(Note that you will need a browser which supports forms and Java to use some of the software
listed here.)
Eric Sven Ristad (Computer Science, Princeton) has written a
Maximum Entropy applet
which computes the maximal entropy estimates of "joint events" in a simple model,
and compares results with the naive averages.
The Shannonizer,
by Josh Senyak,
will take any web page and rewrite it in the style of
any one of eight well known authors. Claude Shannon himself was
the first to play such games, which can actually be quite instructive.
Give this a try!
The results are hilarious.
Compare this genuine news article with the same article
as rewritten by Raymond Chandler.
Another instructive language game involves feeding (for example) the same news article to
the
Alta Vista automatic translation software.
For instance, translating this article from English to French and back and then from English to
German and back yields this result,
whereas translating from English to Spanish to English to Italian to English gives
this result.
1-Dimensional MomEnt Form,
by Jon Borwein (CECM and Dept. of Mathematics and Statistics, Simon Fraser University).
This is a form which allows the user to compare the result of various methods for
reconstructing an unknown function from its integrals against certain known "kernels".
In general, the given information does not determine a unique function, so one must
introduce criteria for selecting the "best guess".
The most popular reconstruction methods involve a criterion of the form "choose the function
with the greatest entropy and satisfying the given constraints".
Here the "entropy" in question may be Shannon's entropy or any of a wide variety of other
possibilities.
Such "maximal entropy" reconstruction methods have applications to spectral estimation,
spectroscopy, and tomography.
In playing around with this form you may further complicate matters by introducing some
"noise" so that the given integrals are somewhat inaccurate; an important question is to
determine how such perturbations affect the "best guess" according to various reconstruction
methods.
For more information, see the
MomEnt+ Project Home Page.
Ken White's Coin Flipping Page.
An amusing program which gives you the result of a run of coin flips.
Includes links to data from
previous experiments and to an article by Nicholas P. Maxwell
(University of Washington, Bothell) originally published
in Journal of Statistics Education v.2, n.1 (1994)
which explains how the coin flipping excercise can help students
understand the process of statistical inference.
Produced by SHAZAM Econometrics Software.
Freeware
MATLAB programs for coding theory
are available from the
Information Theory Research Group
in the Department of Electrical Engineering, University of Minnesota.
Buster
is a program for recovering missing phase information (in crystallography?)
using a maximal entropy principle,
currently under development at the MRC Laboratory of Molecular Biology
in Cambridge, England by Gérard Bricogne and John Irwin.