Entropy-Related Software

• Tutorials

  (Note that you will need a browser which supports forms and Java to use some of the software listed here.)

  • Eric Sven Ristad (Computer Science, Princeton) has written a Maximum Entropy applet which computes the maximal entropy estimates of "joint events" in a simple model, and compares results with the naive averages.
  • The Shannonizer, by Josh Senyak, will take any web page and rewrite it in the style of any one of eight well known authors. Claude Shannon himself was the first to play such games, which can actually be quite instructive. Give this a try! The results are hilarious. Compare this genuine news article with the same article as rewritten by Raymond Chandler. Another instructive language game involves feeding (for example) the same news article to the Alta Vista automatic translation software. For instance, translating this article from English to French and back and then from English to German and back yields this result, whereas translating from English to Spanish to English to Italian to English gives this result.
  • The Dialectizer, by Samuel Stoddard, a similar page.
  • 1-Dimensional MomEnt Form , by Jon Borwein (CECM and Dept. of Mathematics and Statistics, Simon Fraser University).
    This is a form which allows the user to compare the result of various methods for reconstructing an unknown function from its integrals against certain known "kernels". In general, the given information does not determine a unique function, so one must introduce criteria for selecting the "best guess". The most popular reconstruction methods involve a criterion of the form "choose the function with the greatest entropy and satisfying the given constraints". Here the "entropy" in question may be Shannon's entropy or any of a wide variety of other possibilities. Such "maximal entropy" reconstruction methods have applications to spectral estimation, spectroscopy, and tomography. In playing around with this form you may further complicate matters by introducing some "noise" so that the given integrals are somewhat inaccurate; an important question is to determine how such perturbations affect the "best guess" according to various reconstruction methods.
    For more information, see the MomEnt+ Project Home Page.
  • Ken White's Coin Flipping Page. An amusing program which gives you the result of a run of coin flips. Includes links to data from previous experiments and to an article by Nicholas P. Maxwell (University of Washington, Bothell) originally published in Journal of Statistics Education v.2, n.1 (1994) which explains how the coin flipping excercise can help students understand the process of statistical inference. Produced by SHAZAM Econometrics Software.

• Freeware

  • MATLAB programs for coding theory are available from the Information Theory Research Group in the Department of Electrical Engineering, University of Minnesota.
    
  • The Natural Language Software Registry
    
    • Monkey. "given any text as input MONKEY
      1. computes its character entropy to any order up to 120 (no smoothing)
      2. computes its word entropy to any order up to 120 (no smoothing)
      3. generates mimic text to any order of accuracy (up to 120th order), character by character, or word by word
      4. produces an on-screen scrollable concordance with text-retrieval facility."
    
    
  • Random Numbers and Monte Carlo Methods. Links to software for Markov chains, Monte Carlo simulation, and random number generation.
    
  • Buster is a program for recovering missing phase information (in crystallography?) using a maximal entropy principle, currently under development at the MRC Laboratory of Molecular Biology in Cambridge, England by Gérard Bricogne and John Irwin.
    
  
  

• Payware

  • Mnemonic, Inc. offers some maximal entropy software.
  • Entropic Research Laboratory, Inc. offers UNIX speech and signal-processing software.

Back to Entropy on the World Wide Web.