Information theory is not dealing with the correct interpretation of the modal operator 'Knows'.It simply gives rules for selecting ‘the best model’. (Adriaans, 13)

I was recently asked by a friend and colleague what I thought information was. He said that wondered if it was always something that existed only on “organism relative terms” in the sense that 'X counts as INFORMATION for organism O'. This is a worthwhile question that pertains to to my thesis. Here are some thoughts on it.

Is DNA informational? Does it encapsulate information? Say information is something that either only exists as a subjective, epistemic or perceptual entity, or is definable only in terms of utility to and effects upon an organism. Let’s say the organism is whatever the DNA belongs to. What if the organism is not a cognitive agent? Can they perceive the information in the DNA directly? Can they perceive it or cognise it even if they are a cognitive agent? What if the organism is not a cognitive agent and not a percipient in terms of sensory perception? Perhaps the organism is a cell or some kind of eukaryote. Perhaps it is a virus (if one is happy to classify a virus as an organism). Such non-cognitive organisms respond to what we would normally call information in their environments. In cell signalling cascades, cells respond to organic molecules and biochemicals in their environment in certain regular ways. Eukaryotes also respond to heat and light as triggers or signals, and perform protein synthesis using tRNA. The tRNA is considered by scientists and information theorists as carrying information. Organic structures such as DNA and tRNA involve a kind of natural encoding. Considering such structures, it is not as straightforward to make a determination about whether or not information is only an organismic phenomenon. However, if a rise in temperature in the environment of a flagellum indicates to the flagellum that there is a nearby heat source, then we would normally think of the rise in water temperature as indicating to the flagellum that there is a heat source nearby. Theorists and folk alike would thus say that the rise in temperature of the water in the flagellum’s pond carried the information or provided the information that there was a heat source. Now, the question is, if we remove the flagellum, does the rise in water temperature still carry the information that there is a heat source? Most theorists, including Claude Shannon and Fred Dretske, would – I am fairly certain – answer yes. There is still information in the environment even if there are no organisms to advantage it.

Consider a simpler example involving a sentient organism. If a percipient organism like a bird responds to a smoke in the air as an indicator that there is fire and thus flees, we would normally say that the bird has received a warning in the form of an indication or signal giving or carrying the information that there is fire. If one removes the bird from the example, is the information gone? I don’t think so. Luciano Floridi defends a form of informational structural realism wherein information is in fact abstract structure that can exist without any concrete patterns or structures. He would I think allow (require in fact) that there is still information without the entire physical environment and its structures (Floridi, “A Defence of Informational Structural Realism” and “Information” in (Information, in Blackwell Guide to the Philosophy of Information and Computing)). I do not endorse Floridi’s Platonistic structural realism (which allows information itself to be abstract) but I do agree with him that information is essentially data where data boils down to a structural non-uniformity in the world or more specifically for the above cases, in nature. How is smoke a non-uniformity? In many ways, but most basically where it contrasts with non-smoky air in the environment.

Information is not a teleological entity. It is objective and natural. It may be used to convey meaning in communication, but it is not itself meaning in the de-dicto sense of propositional content encoded into syntactic sentences comprised of symbols. Information transmission for Ralph Hartley and Claude Shannon was not in fact directly about transmitting meaning (although they acknowledged that meaning might 'go along for the ride'.) A communication system encodes and transmits what Hartley called physical symbols (Hartley, Ralph V. L. “Transmission Of Information.” (1928), 2.). Information on Shannon's theory had something to do with both the set of possible physical symbols (corresponding to states of the physical source), the probability of those symbols (the probability distribution over the set of possible symbols) and the symbols as an indication of physical source states (Shannon, 2,5,6,7,11.). Information on Shannon’s thesis was not meaning or about teleology in the sense of utility for an organism, even though the communication system he was modelling was clearly artefactual and teleological. Information seems to be one or the other of the above things depending where one looks in The Mathematical Theory of Communication. There is a question about a physical symbol as having natural meaning in H.P. Grice's sense – being similar in some ways to C.S. Peirce's conception the signification of a reagent index and a sinsign. However, the information content of smoke as a symbol/indication of fire is not de-dicto information content, it is what Dretske calls de re information content (Dretske, KFI, 67-68).

The etymology of the word information comes from the Platonistic 'to enform'. The Airstotelian scholars in the middle ages focussed on the enforming of ideas in the mind: information was about learning. The OED lists another meaning: that which is the same as 'giving form to'. A non-cognitive kind of en-forming. It has to do with a form being conveyed from one thing to another. Floridi for one allows the Platonistic and Aristotelian varieties of informing here (the latter subsumed to the former), but I think it more likely that with information – since a source is a physical thing – the Aristotelian interpretation is necessary. Information sources produce information by virtue of having certain spatiotemporal structures. This is the metaphysics behind Shannon’s idea of the generation and production of information. The physical source originates and creates the information, not some Platonistic pattern or ethereal form.

To complicate matters, if you look at Shannon's theory of information, you find that it isn't really a theory of information. It is just a statistical theory which describes a measure said to be one of information. The measure is devised to support the development of the mathematical theory of communication, which is about getting physical symbols encoded and transmitted as a signal from point to point. No actual coherent or consistent conception of information is offered. It isn't clear whether it is possibilities, probabilities produced at a selection of a source state, or some kind of indication of what obtains. Shannon suggests that a valid measure of information at a source is simply the cardinality of the set of possible states of the source:

If the number of messages in the set is finite then this number or any monotonic function of this number can be regarded as a measure of the information produced when one message is chosen from the set, all choices being equally likely (Shannon, 1)

This conception does not directly involve and probabilities. The logarithmic measure he devises (following the lead of Hartley) involves probabilities for the possible states. Information seems to be necessarily a different thing according to each measure. Luciano Floridi thinks it is a polyform idea. He rejects the idea of an objective kind of information and favours a pluralist approach. However, as mentioned above, he also reduces information to data, and defines data as abstract relations (any nonuniformity in any kind of space - physical, noetic, abstract mathematical). Moreover, Andre Kolmogorov’s conception of a measure of information as the minimum length of program required to produce a (usually syntactic) ‘object’ tends to suggest that information is an artefact, as does the idea of minimum description length (MDL).

Dretske, Shannon, Keith Devlin and Floridi all have naturalistic conceptions of information, and none of them really favour a reductive conception, but Floridi reduces to data, Devlin to the infon, and Shannon to the set of possible states of a a physical process (sort of). Dretske may in fact get probability interpretations confused (objective frequentist with epistemic/subjective). Shannon relied upon objective frequentist and classical probability theory, and required that the possible set of source states and probability distribution over the possible source states be set a-priori for the communication system to function. Dretske takes this to mean that there is an epistemic subjective component to information. This does not follow from Shannon's objective only probability interpretation. There are other complex problems with prior probabilities and information, but for the most part frequentist priors are objective. In any case, Dretske follows Shannon, and for Shannon following Hartley, information exists in physical sources, physical states of physical sources, and physical symbols. It reduces to both physical and statistical structure in some fairly obscure way.

Basically, I have an Aristotelian realist view of information: it is whatever spatiotemporal configuration or physical structure exists at a source, and then whatever elements or aspects of that structure are conveyed from one point in space to another through causal pathways when the source interacts physically with other entities. Physical structures can be used to represent other physical structures, in which case their spatiotemporal structure and information is being used to represent the information of the other source as structured information. The original source must have been a physically real situation, state of affairs or source in some way, and when represented. Its information was not abstract, and it is represented using physical information. The representation of the information is in some sense abstract: in the sense that it is an abstraction from a real world source. It is the Aristotelian variety of abstraction and the Aristotelian kind of abstract that is in play.

On my thesis, there is information generated by a Pulsar located somewhere distant in the galaxy even if there is no organism around to act as a percipient or to advantage the information it emits for some teleological or evolutionary purpose. This kind of information is the basis of all other types of information. Information is objective, but can be processed by percipient cognitive agent subjects. There is information that has been produced by such agents. This is artefactual information. It may involve the use of syntactic and symbolic encoding, but the symbols themselves are physical, and they are used to represent the structure of some other information source(s): the information other sources encapsulate. For Floridi, information is structure in the structural realist sense. On my thesis, information reduces to structure on an Aristotelian realist basis.

You cannot have an abstract thought without a neural configuration of some kind which supports it structurally in a physical sense. There is no DNA, or RNA, mRNA, tRNA, nor protein nor any molecule without spatiotemporal structure. The information they carry is essentially their physical structures. There is no information without spatiotemporal structure. This is the crux of my thesis. It is supported by quantum information theory and computer science. It is prospectively unattractive to philosophers because both scientific pluralist liberal naturalism which minimises the role of reduction and Platonistic structural realism are both popular at the moment. However, some computer scientists and quantum information theorists may be more amenable to the idea. Plural naturalism works well for many things, but I do not think information is one of them. Information is an abstraction from physical structure, not some kind of pattern that exists prior to physical spatiotemporal structures. You just can't have any without physical structure to support it. Shannon modelled an information source as an abstrata You cannot transmit it without a causal pathway. It just can't happen. To get the physical structure to support complex information from one source or sources encoded into another source, you need causal interactions and pathways. There was no such thing as an abstract information source for Shannon. He developed a statistical model which included an abstract model element to represent a physical process as a source. The model he also called a source, He had a dual definition. However, the model of the phsycial process could cannot produce any information: only the physical process can do that. There is no such thing as an abstract information source, only a physical one.

References:

Devlin, Keith. Logic and Information. Cambridge: Cambridge University Press, 1995.

Dretske, Fred I. “Epistemology and Information.” Adriaans, Pieter and Johan van Benthem. The Philosophy of Information. Amsterdam: Elsevier, 2008. 29-48.

—. Knowledge and the Flow of Information. London: Basil Blackwell, 1981.

—. Naturalising the Mind (Jean Nicod Lectures). Massachusettes: Massachusettes Institute of Technology, 1995.

Floridi, Luciano. “A Defence of Informational Structural Realism.” Synthese (2008): 219–253.

—. “Against Digital Ontology.” Synthese (2009): 151–178.

Floridi, Luciano. “Information.” The Blackwell Guide to the Philosophy of Computing and Information. Ed. Luciano Floridi. Oxford: Blackwell, 2003. 40-61.

—. Information: A Very Short Introduction. Oxford: Oxford University Press, 2010.

Hájek, Alan. “Interpretations of Probability.” 2007. Stanford Encyclopedia of Philosophy. 24/02/10 November, February 2008, 2010 .

Hartley, Ralph V. L. “Transmission Of Information.” 1928. Lucent Technologies. 2010.

Shannon, Claude E. “A Mathematical theory of Communication: Reprinted with corrections from The Bell System Technical Journal.” 1998 (50th anniversary release of 1948 paper). Shannon Day at Bell Labs. Bell Labs. http://cm.bell-labs.com/cm/ms/what/shannonday/paper.html; http://cm.bell-labs.com/cm/ms/what/shannonday/

(This entry originally posted 06-10-10. Updates 07-10-10)

I think that it may be the case that Fred Dretske’s theory of semantic information combines two incompatible conceptions of information at a source: that which is necessarily exclusively determined by objective possibilities about what source states can obtain, and that which is somehow affected by k - the subjective epistemic content of a receiving agent w.r.t the possibilities at the source s. First of all, I think that the latter (conception of information at a source) can be seen not to be obtainable by the necessarily causal and unidirectional (from source to receiver) nature of information transmission in Shannon’s schema. The idea here is that in order for the source possibilities to be truly affected by what a receiving agent knows, there must be a causal influence along a causal signal-bearing pathway (channel) from the receiving agent to the information source such that the set of possible source states is changed by k. This is not achievable on the basis of the receiving agent “carving up the possibilities at the source” a certain way in accordance with k. The source possibilities are immutably objective on Dretske’s own assertion. Nevertheless, the combining of objective and epistemic notions of the possibilities at the source is necessary for Dretske’s semantic theory of information to work. Dretske himself first asserts that the possibility space and the probability distribution over possible source states at a source is exclusively objective, but then asserts that the knowledge k of the receiver about those possibilities and how they are carved up affects the possibilities at the source. It seems apparent that the two simply cannot obtain at once, and that deferring to the level of abstraction afforded by the mathematical/probabilistic model of the system does not mitigate the problem. The epistemic conception of information of a source where the possibilities at the source are partly or wholly receiving-agent-knowledge determined renders the source possibilities no longer objective. The objective nature of the source possibilities and probability distribution is a stipulation of the theory required to get the measure of information for a specific source state and a specific signal off the ground. A generalization from these measures is relied upon to enable a receiving agent that is ignorant of the objective source possibilities to compare the relative magnitudes of the information content of each signal/message based on their knowledge of the objective possibilities at the source. These objective possibilities are made to be epistemic and subjective as Dretske develops the semantic theory. This does not seem to be consistent.

Otherwise put, it seems there is the following problem:

1. Dretske asserts that the set of possibilities for source states and the probability distribution over that set is necessarily objective, as is the information content of a message from such a source.
2. Dretske’s semantic theory of information requires the receiving agent to be able to ascertain which of two messages from a source has greater information content without any knowledge of the actual objective possibility space – the set of possible source states – nor the probability distribution over those states at the source.
3. 2. Is achieved by considering that the agent’s knowledge of the possibilities that might obtain at a source – the way that the receiving agent carves up the possibilities at the source based on ‘existing’ knowledge k about the source – allows them to discern which of the two different messages from the source has greater information content and which results in greater equivocation at the source (equivocation being the possibilities at the source not carried by the received message).
4. The message comparison in 3. is only made possible by generalizing from formulas that give a measure of the information in a specific source state and signal. These measures rely by definition on 1 being true.
5. Dretske asserts that the k in 3. and the associated carving up of the possibilities at the source affects the value of the information measure of the source state, and this necessarily means that it affects the set of possibilities at the source.
6. Thus 2. Is incompatible with 1. via the incompatibility of 1. with 5.
7. 5. Would also require information to flow through a causal pathway from receiver to source, and the agent’s carving up of the possibilities at the source based upon k does not involve such: it is only an artifact/facet of the model – not a dynamic of the modeled system. What effectively happens is that the source in question for semantic information becomes s+(receiver with k), which is a completely different source from s.

Bruce R. Long., 17th September 2010.

I am currently wrestling with Fred Dretske's semantic theory of information. I have been a student of Dretske's work for several years now, and have completed at least two major papers that refer to his conception of information theory and information as the content of mental representations.

I will be giving this presentation again in near future (at a WIP seminar at the University of Sydney). I am attempting to get to the bottom of whether Dretske requires two conflicting conceptions of information content on the basis of two conflicting conceptions of the possibilities that exist at an information source:

Dretske’s conception of information combines two incompatible conceptions of information at a source: that which is necessarily exclusively determined by objective possibilities, and that which is somehow affected by k - the subjective epistemic content of the receiver w.r.t the possibilities at the source s. The latter (conception of information at a source) can be seen not to be obtainable by the necessarily causal and unidirectional (from source to receiver) nature of information transmission in Shannon’s schema. The combining of objective and epistemic notions of the possibilities at the source is necessary for Dretske’s semantic theory of information to work. However, Dretske himself first asserts that the possibility space and the probabiltiy distribution over possible source states at a source is exclusively objective, and then asserts that the knowledge k of the receiver about those possibilities and how they are carved up affects the possibilities at the source. The latter renders the source possibilities no longer objective, which objectiveness is a stipulation of the theory required to get the measure of information for a specific source state and a specific signal off the ground. These measures are relied upon to enable a receiving agent that is ignorant of the objective source possibilities to compare the relative magnitudes of the information content of each message based on their knowledge of the objective possiblities at the source. These objective possibilities are made to be epistemic and subjective as Dretske develops the semantic theory. This does not seem to be consistent.