Distributed Representation Revisited


If the conventional symbolic model mandates a creation of theory that is sought to address the issues pertaining to the problem, this mandatory theory construction is bypassed in case of distributed representational systems, since the latter is characterized by a large number of interactions occurring in a nonlinear fashion. No such attempts at theoretical construction are to be made in distributed representational systems for fear of high end abstraction, thereby sucking off the nutrient that is the hallmark of the model. Distributed representation is likely to encounter onerous issues if the size of the network inflates, but the issue is addressed through what is commonly known as redundancy technique, whereby, a simultaneous encoding of information generated by numerous interactions take place, thus ameliorating the adequacy of presenting the information to the network. In the words of Paul Cilliers, this is an important point, for,

the network used for the model of a complex system will have to have the same level of complexity as the system itself….However, if the system is truly complex, a network of equal complexity may be the simplest adequate model of such a system, which means that it would be just as difficult to analyze as the system itself.

Following, he also presents a caveat,

This has serious methodological implications for the scientists working with complex systems. A model which reduces the complexity may be easier to implement, and may even provide a number of economical descriptions of the system, but the price paid for this should be considered carefully.

One of the outstanding qualities of distributed representational systems is their adaptability. Adaptability, in the sense of reusing the network to be applicable to other problems to offer solutions. Exactly, what this connotes is, the learning process the network has undergone for a problem ‘A’, could be shared for problem ‘B’, since many of the input neurons are bounded by information learned through ‘A’ that could be applicable to ‘B’. In other words, the weights are the dictators for solving or resolving issues, no matter, when and for which problem the learning took place. There is a slight hitch here, and that being this quality of generalizing solutions could suffer, if the level of abstraction starts to shoot up. This itself could be arrested, if in the initial stages, the right kind of framework is decided upon, thus obscuring the hitch to almost non-affective and non-existence impacting factor. The very notion of weights is considered here by Sterelny as a problematic, and he takes it to attack distributed representation in general and connectionsim as a whole in particular. In an analogically witty paragraph, Sterelny says,

There is no distinction drawable, even in principle, between functional and non- functional connections. A positive linkage between two nodes in a distributed network might mean a constitutive link (eg. Catlike, in a network for tiger); a nomic one (carnivore, in the same network), or a merely associative one (in my case, a particular football team that play in black and orange.

It should be noted that this criticism on weights is derived, since for Sterelny, relationship between distributed representations and the micro-features that compose them is deeply problematic. If such is the criticism, then no doubt, Sterelny still seems to be ensconced within the conventional semantic/symbolic model. And since, all weights can take part in information processing, there is some sort of a democratic liberty that is accorded to the weights within a distributed representation, and hence any talk of constitutive, nomic, or even for that matter associative is mere humbug. Even if there is a disagreement prevailing that a large pattern of weights are not convincing enough for an explanation, as they tend to complicate matters, the distributed representational systems work consistently enough as compared to an alternative system that offers explanation through reasoning, and thereby, it is quite foolhardy to jettison the distributed representation by the sheer force of criticism. If the neural network can be adapted to produce the correct answer for a number of training cases that is large compared with the size of the network, it can be trusted to respond correctly to the previously unseen cases provided they are drawn from the same population using the same distribution as the training cases, thus undermining the commonly held idea that explanations are the necessary feature of the trustworthy systems (Baum and Haussler). Another objection that distributed representation faces is that, if representations are distributed, then the probability of two representations of the same thing as different from one another cannot be ruled out. So, one of them is the true representation, while the other is only an approximation of the representation.(1) This is a criticism of merit and is attributed to Fodor, in his influential book titled Psychosemantics.(2) For, if there is only one representation, Fodor would not shy from saying that this is the yucky solution, folks project believe in. But, since connectionism believes in the plausibility of indeterminate representations, the question of flexibility scores well and high over the conventional semantic/symbolic models, and is it not common sense to encounter flexibility in daily lives? The other response to this objection comes from post-structuralist theories (Baudrillard is quite important here. See the first footnote below). The objection of true representation, and which is a copy of the true representation meets its pharmacy in post-structuralism, where meaning is constituted by synchronic as well as diachronic contextualities, and thereby supplementing the distributed representation with a no-need-for concept and context, as they are inherent in the idea of such a representation itself. Sterelny, still seems to ride on his obstinacy, and in a vitriolic tone poses his demand to know as to why distributed representation should be regarded as states of the system at all. Moreover, he says,

It is not clear that a distributed representation is a representation for the connectionist system at all…given that the influence of node on node is local, given that there is no processor that looks at groups of nodes as a whole, it seems that seeing a distributed representation in a network is just an outsider’s perspective on the system.

This is moving around in circles, if nothing more. Or maybe, he was anticipating what G. F. Marcus would write and echo to some extent in his book The Algebraic Mind. In the words of Marcus,

…I agree with Stemberger(3) that connectionism can make a valuable contribution to cognitive science. The only place, we differ is that, first, he thinks that the contribution will be made by providing a way of eliminating symbols, whereas I think that connectionism will make its greatest contribution by accepting the importance of symbols, seeking ways of supplementing symbolic theories and seeking ways of explaining how symbols could be implemented in the brain. Second, Stemberger feels that symbols may play no role in cognition; I think that they do.

Whatever Sterelny claims, after most of the claims and counter-claims that have been taken into account, the only conclusion for the time being is that distributive representation has been undermined, his adamant position to be notwithstanding.

(1) This notion finds its parallel in Baudrillard’s Simulation. And subsequently, the notion would be invoked in studying the parallel nature. Of special interest is the order of simulacra in the period of post-modernity, where the simulacrum precedes the original, and the distinction between reality and representation vanishes. There is only the simulacrum and the originality becomes a totally meaningless concept.

(2) This book is known for putting folk psychology firmly on the theoretical ground by rejecting any external, holist and existential threat to its position.

(3) Joseph Paul Stemberger is a professor in the Department of Linguistics at The University of British Columbia in Vancouver, British Columbia, Canada, with primary interests in phonology, morphology, and their interactions. My theoretical orientations are towards Optimality Theory, employing our own version of the theory, and towards connectionist models.


The Differentiated Hyperreality of Baudrillard


A sense of meaning for Baudrillard connotes a totality that is called knowledge and it is here that he differs significantly from someone like Foucault. For the latter, knowledge is a product of relations employing power, whereas for the former, any attempt to reach a finality or totality as he calls fit is always a flirtation with delusion. A delusion, since the human subject would always aim at understanding the human or non-human object, and, in the process the object would always be elusive since, it being based on signifiers would be vulnerable to a shift in significations. The two key ideas of Baudrillard are simulation and hyperreality. Simulation accords to representation of things such that they become the things represented, or in other words, representations gain priority over the “real” things. There are certain orders that define simulations viz. signs get to represent objective reality, signs veil reality, signs masking the absence of reality and signs turning into simulacra, since they have relation to reality thus ending up simulating a simulation. In Hegarty‘s reading of Baudrillard, there happen to be three types of simulacra each with a distinct historical epoch. The first is the pre-modern period, where the image marks the place for an item and hence the uniqueness of objects and situations marks them as irreproducibly real. The second is the modern period characterized by industrial revolution signifying the breaking down of distinctions between images and reality because of mass reproduction of copies or proliferation of commodities thus risking the essential existence of the original. The third is the post-modern period, where simulacra precedes the original and the distinction between reality and representation vanishes implying only the existence of simulacra and relegating reality as a vacuous concept. Hyperreality defines a condition wherein “reality” as known gets substituted by simulacra. This notion of Baudrillard is influenced by Canadian communication theorist and rhetorician Marshall McLuhan. Hyperreality with its insistence of signs and simulations fit perfectly in the post-modern era and therefore highlights the inability or shortcomings of consciousness to demarcate between reality and the phantasmatic space. In a quite remarkable analysis of Disneyland, Baudrillard (166-184) clarifies the notion of hyperreality, when he says,

The Disneyland imaginary is neither true nor false: it is a deterrence machine set in order to rejuvenate in reverse the fiction of the real. Whence the debility, the infantile degeneration of this imaginary. It’s meant to be an infantile world, in order to make us believe that adults are everywhere, in the “real” world and to conceal the fact that real childishness is everywhere, particularly among those adults who go there to act the child in order to foster illusion of their real childishness.

Although his initial ideas were affiliated with those of Marxism, he differed from Marx in his epitomizing consumption as the driving force of capitalism as compared to latter’s production. Another issue that was worked out remarkably in Baudrillard was historicity. Agreeing largely with Fukuyama’s notion of the end of history after the collapse of the communist block, Baudrillard only differed by placing importance on the idea of historical progress to have ended and not history necessarily. He forcefully makes the point of ending of history as also the ending of dustbins of history. His post-modern stand differed significantly with that of Lyotard’s in one major respect, despite finding common grounds elsewhere. Despite showing growing aversion to the theory of meta-narratives, Baudrillard, unlike Lyotard, reached a point of pragmatic reality within the confines of an excuse laden notion of universality that happened to be in vogue.

Baudrillard has been at the receiving end with some very extreme, acerbic criticisms directed at him. His writings are not just obscure, but also fail in many respects like defining certain concepts he employs, totalizing insights that have no substantial claim to conjectures, and often hinting strongly at apodicticity without paying due attention to the rival positions. This extremity reaches a culmination point when he is cited as a purveyor of reality-denying irrationalism. But not everything is to be looked at critically in his case and he does enjoy an established status as a transdisciplinary theorist, who, with his provocations have put traditional issues regarding modernity and philosophy in general at stake by providing insights into a better comprehensibility of cultural studies, sociology and philosophy. Most importantly, Baudrillard provides for autonomous and differentiated spaces in cultural, socio-economic and political domains by an implosive theory that cuts across boundaries of various disciplines paving the way for a new era in philosophical and social theory at large.

Benjamin Noys, Lyotard, Baudrillard and the liquidity grid of capitalism (Notes Quotes)

For Benjamin Noys, as Lyotard put it, “desire underlies capitalism too“, then the result is that: ‘there are errant forces in the signs of capital. Not in its margins as its marginals, but dissimulated in its most essential exchanges.’ For Deleuze and Guattari, the problem of capitalism is not that it deterritorializes, but that it does not deterritorialise enough. It always runs up against its own immanent limit of deterritorialisation – the deterritorialisation of decoded flows of desire through the machine of oedipal grid. It is the figure of the schizophrenic, not to be confused with the empirical psychiatric disorder, which instantiates this radical immersion and the coming of a new porous and collective ‘subject’ of desire. The schizophrenic is the one who seeks out the very limit of capitalism: he is the inherent tendency brought to fulfilment. Contrary to Deleuze and Guattari’s faith in a subject who would incarnate a deterritorialisation in excess of capitalism, Lyotard’s Libidinal Economy denies any form of exteriority, insisting that capital itself is the unbinding of the most insane drives, which releases mutant intensities. the true form of capitalism is incarnated in the a-subjective figure of the libidinal band, a Moëbius strip of freely circulating intensities with neither beginning nor end.


Baudrillard argues that the compulsion towards liquidity, flow and accelerated circulation is only the replica or mirror of capitalist circulation. his catastrophic strategy comprises a kind of negative accelerations, in which he seeks the point of immanent reversal that inhabits and destabilises capital. In Symbolic Exchange and Death, this is the death function, which cannot be programmed and localised. against the law of value that determines market exchange, Baudrillard identifies this “death function” with the excessive and superior function of symbolic exchange which is based on the extermination of value.

Reality as Contingently Generating the Actual


If reality could be copied, the mapping or simulation of a neural network digitally is possible. This scheme has some nagging problems. Chief among them being the simulated nature of neural network, as the possibility of reality vis-a-vis natural neural network is susceptible to mismatch thus leading to what could be termed non-reductionist essentialism. The other option that could aid better apprehensibility of hyperrealism and simulation in terms of neural networks is from Bhaskar Roy’s idea of critical realism. This aspect differs significantly from Baudrillard’s in that the latter takes reality as potentially open to copying, whereas the former delves into reality as a generative mechanism. According to Bhaskar’s take, reality is not something that can be copied, but something that contingently generates the actual.