| Dictionary, Encyclopedia and Thesaurus - The Free Dictionary 1,727,958,660 visitors served. |
|
Dictionary/ thesaurus | Medical dictionary | Legal dictionary | Financial dictionary | Acronyms | Idioms | Encyclopedia | Wikipedia encyclopedia | ? |
information theory |
Also found in: Dictionary/thesaurus, Medical, Legal, Acronyms, Wikipedia, Hutchinson | 0.01 sec. |
|
information theory or communication theory, mathematical theory formulated principally by the American scientist Claude E. Shannon Shannon, Claude Elwood, 1916–2001, American applied mathematician, b. Gaylord, Michigan. A student of Vannevar Bush at the Massachusetts Institute of Technology (MIT), he was the first to propose the application of symbolic logic to the design of relay ..... Click the link for more information. to explain aspects and problems of information and communication. While the theory is not specific in all respects, it proves the existence of optimum coding schemes without showing how to find them. For example, it succeeds remarkably in outlining the engineering requirements of communication systems and the limitations of such systems. In information theory, the term information is used in a special sense; it is a measure of the freedom of choice with which a message is selected from the set of all possible messages. Information is thus distinct from meaning, since it is entirely possible for a string of nonsense words and a meaningful sentence to be equivalent with respect to information content. Measurement of Information ContentNumerically, information is measured in bits (short for binary digit; see binary system binary system, numeration system based on powers of 2, in contrast to the familiar decimal system , which is based on powers of 10. In the binary system, only the digits 0 and 1 are used. Interestingly, the mathematical expression for information content closely resembles the expression for entropy entropy (ĕn`trəpē), quantity specifying the amount of disorder or randomness in a system bearing energy or information. Analysis of the Transfer of Messages through ChannelsA message proceeds along a channel from the source to the receiver; information theory defines for any given channel a limiting capacity or rate at which it can carry information, expressed in bits per second. In general, it is necessary to process, or encode, information from a source before transmitting it through a given channel. For example, a human voice must be encoded before it can be transmitted by telephone. An important theorem of information theory states that if a source with a given entropy feeds information to a channel with a given capacity, and if the source entropy is less than the channel capacity, a code exists for which the frequency of errors may be reduced as low as desired. If the channel capacity is less than the source entropy, no such code exists. The theory further shows that noise noise, any signal that does not convey useful information. Electrical noise consists of electrical currents or voltages that interfere with the operation of electronic systems. BibliographySee C. E. Shannon and W. Weaver, The Mathematical Theory of Communication (1949); M. Mansuripur, Introduction to Information Theory (1987). information theoryField of mathematics that studies the problems of signal transmission, reception, and processing. It stems from Claude E. Shannon's mathematical methods for measuring the degree of order (nonrandomness) in a signal, which drew largely on probability theory and stochastic processes and led to techniques for determining a source's rate of information production, a channel's capacity to handle information, and the average amount of information in a given type of message. Crucial to the design of communications systems, these techniques have important applications in linguistics, psychology, and even literary theory. information theoryThe study of encoding and transmitting information. From Claude Shannon's 1948 paper, "A Mathematical Theory of Communication," which proposed the use of binary digits for coding information. Shannon said that all information has a "source rate" that can be measured in bits per second and requires a transmission channel with a capacity equal to or greater than the source rate. information theory a collection of mathematical theories, based on statistics, concerned with methods of coding, transmitting, storing, retrieving, and decoding information information theory [‚in·fər′mā·shən ‚thē·ə·rē] (communications) A branch of theory which is devoted to problems in communications, and which provides criteria for comparing different communications systems on the basis of signaling rate, using a numerical measure of the amount of information gained when the content of a message is learned. (mathematics) The branch of probability theory concerned with the likelihood of the transmission of messages, accurate to within specified limits, when the bits of information composing the message are subject to possible distortion. Information theory A branch of communication theory devoted to problems in coding. A unique feature of information theory is its use of a numerical measure of the amount of information gained when the contents of a message are learned. Information theory relies heavily on the mathematical science of probability. For this reason the term information theory is often applied loosely to other probabilistic studies in communication theory, such as signal detection, random noise, and prediction. See Electrical communications In designing a one-way communication system from the standpoint of information theory, three parts are considered beyond the control of the system designer: (1) the source, which generates messages at the transmitting end of the system, (2) the destination, which ultimately receives the messages, and (3) the channel, consisting of a transmission medium or device for conveying signals from the source to the destination. The source does not usually produce messages in a form acceptable as input by the channel. The transmitting end of the system contains another device, called an encoder, which prepares the source's messages for input to the channel. Similarly the receiving end of the system will contain a decoder to convert the output of the channel into a form that is recognizable by the destination. The encoder and the decoder are the parts to be designed. In radio systems this design is essentially the choice of a modulator and a detector. A source is called discrete if its messages are sequences of elements (letters) taken from an enumerable set of possibilities (alphabet). Thus sources producing integer data or written English are discrete. Sources which are not discrete are called continuous, for example, speech and music sources. The treatment of continuous cases is sometimes simplified by noting that signal of finite bandwidth can be encoded into a discrete sequence of numbers. The output of a channel need not agree with its input. For example, a channel might, for secrecy purposes, contain a cryptographic device to scramble the message. Still, if the output of the channel can be computed knowing just the input message, then the channel is called noiseless. If, however, random agents make the output unpredictable even when the input is known, then the channel is called noisy. See Communications scrambling, Cryptography Many encoders first break the message into a sequence of elementary blocks; next they substitute for each block a representative code, or signal, suitable for input to the channel. Such encoders are called block encoders. For example, telegraph and teletype systems both use block encoders in which the blocks are individual letters. Entire words form the blocks of some commercial cablegram systems. It is generally impossible for a decoder to reconstruct with certainty a message received via a noisy channel. Suitable encoding, however, may make the noise tolerable. Even when the channel is noiseless, a variety of encoding schemes exists and there is a problem of picking a good one. Of all encodings of English letters into dots and dashes, the Continental Morse encoding is nearly the fastest possible one. It achieves its speed by associating short codes with the most common letters. A noiseless binary channel (capable of transmitting two kinds of pulse 0, 1, of the same duration) provides the following example. Suppose one had to encode English text for this channel. A simple encoding might just use 27 different five-digit codes to represent word space (denoted by #), A, B, . . . , Z; say # 00000, A 00001, B 00010, C 00011, . . . , Z 11011. The word #CAB would then be encoded into 00000000110000100010. A similar encoding is used in teletype transmission; however, it places a third kind of pulse at the beginning of each code to help the decoder stay in synchronism with the encoder. How to thank TFD for its existence? Tell a friend about us, add a link to this page, add the site to iGoogle, or visit webmaster's page for free fun content. |
|
| ? Mentioned in | ? References in periodicals archive | ||
|---|---|---|---|
| To study evolution of both organisms and manuscripts, Cisne is now using tools from information theory, which considers transmission errors in information. Our approach (Fowler and Hobbs 2002, 2003) stems from systemic thinking as a paradigm that is emergent from modern systems theory, cybernetics, and information theory from their beginnings in the late 1940s. In light of recent art-historical obsessions with technology, information theory, vision, and modes of attention--not to mention our acute cultural preoccupation with all things scientific--it is perhaps unsurprising that the ICP has devoted a number of shows to such topical themes. |
| Encyclopedia |
| Free Tools: |
For surfers:
Free toolbar & extensions |
Word of the Day |
Help
For webmasters: Free content | Linking | Lookup box | Double-click lookup | Partner with us |
|---|