Entropy in the Humanities and Social Sciences

Original by Chris Hillman. Last modified by Roland Gunesch on May 7, 2001.

One of the most interesting (and best justified) applications of entropy to the humanities involves pattern matching and entropic measures of dissimilarity to help answer questions like

For a very nice introduction to these ideas, see Typical Sequences and All That: Entropy, Pattern Matching, and Data Compression, by the late Aaron D. Wyner.

The problem of recovering a lost language from written inscriptions is one on which entropy theory can quite possibly shed some light. An interesting problem is for example the following:

I should perhaps explain this a little further. Linear B was deciphered by the remarkable amateur Michael Ventris, who proved it was a written form of ancient Greek. The known inscriptions turn out to mostly be product inventories; not very useful for understanding the political situation when Linear B was in use, but painting a remarkably clear picture of the economy of ancient Crete. Nonetheless, the mere fact that the official language of Crete was Greek proved for the first time that, as had long been suspected, the Greeks had wrested political control of the island from the Minoans, sometime around 1070 BC.

The Linear A script is similar in appearance to the Linear B script but it is believed to represent the original Minoan language. Various eminent epigraphers have claimed to decipher the available tablets, but unfortunately, not only did their readings differ completely, they even disagreed over the fundamental issue of whether ancient Minoan belonged to the Indo-European family, the Hamito-Semitic family, or to still another language group!

I conjecture that the trouble may be that unlike the tablets written in Linear B, some of which were fairly lengthy documents, the known Linear A inscriptions are (so far as I am aware) at most a few lines long. And as Shannon showed, a sufficiently short message simply cannot be understood in a meaningful way.

Psychologists have long attemped to apply entropy concepts to define and compute "sensory capacities" or (more dubiously) "cognitive capacities"; such applications have generally been criticized by information theorists as embarrasingly naive. In a famous lament, Shannon himself cautioned that "workers in other fields should realize that the basic results of the subject are aimed in a very specific direction, a direction that is not necessarily relevant to such fields as psychology, economics, and other social sciences" (IEEE Transactions on Information Theory, December 1955). For a recent survey, see the book by Norwich below (I haven't had a chance to examine this, so I don't know if the psychophysicists have grown more sophisticated since the demise of the unfortunate Zipf's Law.) More ambitiously, psychologists have attempted to apply information theory to the study of the human brain; see for instance these abstracts from a Workshop on Information Theory and the Brain.

Sociologists have applied entropy concepts in various ways, ranging from the use of statistical entropies to help interpret statistical patterns in voting records to vague and apparently vacuous speculations concerning a sort of grand unified social theory (e.g. Bailey's book below; I have not examined the books by Forse or Geiger). For more on Coleman's work see his book and the following paper:

Further Reading

Stephen Coleman suggested the following references (but he doesn't endorse these books, other than the one he wrote himself!):

Back to Entropy on the World Wide Web.