Kolmogorov Complexity and Information Theory: With an Interpretation in Terms of Questions and Answers
We compare the elementary theories of Shannon information and Kolmogorov complexity, the extent to which they have a common purpose, and where they are fundamentally different. We discuss and relate the basic notions of both theories: Shannon entropy, Kolmogorov complexity, Shannon mutual informatio...
Veröffentlicht in: | Journal of Logic, Language, and Information. - Springer Science + Business Media. - 12(2003), 4, Seite 497 |
---|---|
1. Verfasser: | |
Weitere Verfasser: | |
Format: | Online-Aufsatz |
Sprache: | English |
Veröffentlicht: |
2003
|
Zugriff auf das übergeordnete Werk: | Journal of Logic, Language, and Information |
Schlagworte: | Algorithmic information theory data compression Kolmogorov complexity mutual information prefix codes rate distortion theory Shannon information theory universal codes Mathematics Physical sciences mehr... |
Zusammenfassung: | We compare the elementary theories of Shannon information and Kolmogorov complexity, the extent to which they have a common purpose, and where they are fundamentally different. We discuss and relate the basic notions of both theories: Shannon entropy, Kolmogorov complexity, Shannon mutual information and Kolmogorov ("algorithmic") mutual information. We explain how universal coding may be viewed as a middle ground between the two theories. We consider Shannon's rate distortion theory, which quantifies useful (in a certain sense) information. We use the communication of information as our guiding motif, and we explain how it relates to sequential question-answer sessions. |
---|---|
ISSN: | 15729583 |