Tricki

## Entropy vs. energy

### Quick description

Entropy vs. energy is a powerful technique in discrete probability. You begin with a probability measure which depends on some parameter. To estimate certain events under this measure, you tune the parameter so that the number of possible configurations (entropy) does not outweigh the probability of each configuration satisfying that event (energy).

### Prerequisites

Elementary probability. Borel-Cantelli_lemma

### Example 1

Consider the lattice for . Let . Independently, we consider each bond (edge) of the lattice to be open with probability and closed with probability . We write the probability measure of this model , indicating the dependence on the parameter .

A natural question to ask is if there exists a connected component of infinitely many open bonds, which we call an infinite cluster. This is a random event which depends on the parameter . Since the existence of such a cluster is translation-invariant, the Kolmogorov 0-1 Law says that a cluster either exists with -probability one or zero. If such a cluster exists, the origin may or may not belong to it. This is not a translation-invariant event, so we write

If , every bond is open so , that is 0 trivially belongs to the cluster. Conversely, if , every bond is closed so . We are interested in the behavior of the function . Let

be the highest value of such that there is no infinite cluster. It is a major early result of percolation that .

In what follows, we use the energy vs. entropy technique to demonstrate that . Fix a positive integer . Let be the event that there exists a path such that , , does not intersect itself, and passes through only open bonds. We will show that for small but positive values of , decays exponentially in . Since converges, the Borel-Cantelli lemma will imply that the event holds for only finitely many . So for such values of , 0 will belong to the infinite cluster with probability zero, hence such a cluster will almost surely not exist.

Here is the crucial observation:

where the union and outer sum are over all self-avoiding paths starting at the origin of length . We overestimate the number of such paths: at the origin, has choices of direction, but at all successive points it has no more than choices since it cannot backtrack. Thus the number of such paths is bounded above by . This is an example of an entropy statement, where we count all possible configurations, versus the energy term .

Thus, the quantity in (1) is bounded above by . This is quite an awful estimate, but we have considerable freedom in choosing the value of . In particular, if we choose , then the series is bounded above by a convergent geometric series, and the Borel-Cantelli lemma applies as stated above. Thus for such , , proving that the critical value is strictly positive.

This is not the end of the story, of course. In the case , this argument shows that . This argument was formulated in the sixties by Hammersley (I think, needs verification). However, the true value of for is , which was not proved until Kesten in 1982 (I think). Entropy vs. energy is a powerful technique, provided you do not need optimal parameters.