entropy

entropy (ĕn`trəpē), quantity specifying the amount of disorder or randomness in a system bearing energy or information. Originally defined in thermodynamics in terms of heat and temperature, entropy indicates the degree to which a given quantity of thermal energy is available for doing useful work—the greater the entropy, the less available the energy. For example, consider a system composed of a hot body and a cold body; this system is ordered because the faster, more energetic molecules of the hot body are separated from the less energetic molecules of the cold body. If the bodies are placed in contact, heat will flow from the hot body to the cold one. This heat flow can be utilized by a heat engine (device which turns thermal energy into mechanical energy, or work), but once the two bodies have reached the same temperature, no more work can be done. Furthermore, the combined lukewarm bodies cannot unmix themselves into hot and cold parts in order to repeat the process. Although no energy has been lost by the heat transfer, the energy can no longer be used to do work. Thus the entropy of the system has increased. According to the second law of thermodynamics, during any process the change in entropy of a system and its surroundings is either zero or positive. In other words the entropy of the universe as a whole tends toward a maximum. This means that although energy cannot vanish because of the law of conservation of energy (see conservation laws), it tends to be degraded from useful forms to useless ones. It should be noted that the second law of thermodynamics is statistical rather than exact; thus there is nothing to prevent the faster molecules from separating from the slow ones. However, such an occurrence is so improbable as to be impossible from a practical point of view. In information theory the term entropy is used to represent the sum of the predicted values of the data in a message.

---

entropy

Measure of a system's energy that is unavailable for work, or of the degree of a system's disorder. When heat is added to a system held at constant temperature, the change in entropy is related to the change in energy, the pressure, the temperature, and the change in volume. Its magnitude varies from zero to the total amount of energy in a system. The concept, first proposed in 1850 by the German physicist Rudolf Clausius (1822–1888), is sometimes presented as the second law of thermodynamics, which states that entropy increases during irreversible processes such as spontaneous mixing of hot and cold gases, uncontrolled expansion of a gas into a vacuum, and combustion of fuel. In popular, nontechnical use, entropy is regarded as a measure of the chaos or randomness of a system.

---

Disorder or randomness. In data compression, it is a measure of the amount of non-redundant and non-compressible data in an object (the amount that is not similar). In encryption, it is the amount of disorder or randomness that is added. In software, it is the disorder and jumble of its logic, which occurs after the program has been modified over and over. See encryption algorithm.

---

(theory)

entropy - A measure of the disorder of a system. Systems tend to go from a state of order (low entropy) to a state of maximum disorder (high entropy). The entropy of a system is related to the amount of information it contains. A highly ordered system can be described using fewer bits of information than a disordered one. For example, a string containing one million "0"s can be described using run-length encoding as [("0", 1000000)] whereas a string of random symbols (e.g. bits, or characters) will be much harder, if not impossible, to compress in this way. Shannon's formula gives the entropy H(M) of a message M in bits: H(M) = -log2 p(M) Where p(M) is the probability of message M.