Regulating the Velocities of Dark Pools. Thought of the Day 72.0

On 22 September 2010 the SEC chair Mary Schapiro signaled US authorities were considering the introduction of regulations targeted at HFT:

…High frequency trading firms have a tremendous capacity to affect the stability and integrity of the equity markets. Currently, however, high frequency trading firms are subject to very little in the way of obligations either to protect that stability by promoting reasonable price continuity in tough times, or to refrain from exacerbating price volatility.

However regulating an industry working towards moving as fast as the speed of light is no ordinary administrative task: – Modern finance is undergoing a fundamental transformation. Artificial intelligence, mathematical models, and supercomputers have replaced human intelligence, human deliberation, and human execution…. Modern finance is becoming cyborg finance – an industry that is faster, larger, more complex, more global, more interconnected, and less human. C W Lin proposes a number of principles for regulating this cyber finance industry:

1. Update antiquated paradigms of reasonable investors and compartmentalised institutions, and confront the emerging institutional realities, and realise the old paradigms of governance of markets may be ill-suited for the new finance industry;
2. Enhance disclosure which recognises the complexity and technological capacities of the new finance industry;
3. Adopt regulations to moderate the velocities of finance realising that as these approach the speed of light they may contain more risks than rewards for the new financial industry;
4. Introduce smarter coordination harmonising financial regulation beyond traditional spaces of jurisdiction.

Electronic markets will require international coordination, surveillance and regulation. The high-frequency trading environment has the potential to generate errors and losses at a speed and magnitude far greater than that in a floor or screen-based trading environment… Moreover, issues related to risk management of these technology-dependent trading systems are numerous and complex and cannot be addressed in isolation within domestic financial markets. For example, placing limits on high-frequency algorithmic trading or restricting Un-filtered sponsored access and co-location within one jurisdiction might only drive trading firms to another jurisdiction where controls are less stringent.

In these regulatory endeavours it will be vital to remember that all innovation is not intrinsically good and might be inherently dangerous, and the objective is to make a more efficient and equitable financial system, not simply a faster system: Despite its fast computers and credit derivatives, the current financial system does not seem better at transferring funds from savers to borrowers than the financial system of 1910. Furthermore as Thomas Piketty‘s Capital in the Twenty-First Century amply demonstrates any thought of the democratisation of finance induced by the huge expansion of superannuation funds together with the increased access to finance afforded by credit cards and ATM machines, is something of a fantasy, since levels of structural inequality have endured through these technological transformations. The tragedy is that under the guise of technological advance and sophistication we could be destroying the capacity of financial markets to fulfil their essential purpose, as Haldane eloquently states:

An efficient capital market transfers savings today into investment tomorrow and growth the day after. In that way, it boosts welfare. Short-termism in capital markets could interrupt this transfer. If promised returns the day after tomorrow fail to induce saving today, there will be no investment tomorrow. If so, long-term growth and welfare would be the casualty.

Advertisement

Belief Networks “Acyclicity”. Thought of the Day 69.0

Belief networks are used to model uncertainty in a domain. The term “belief networks” encompasses a whole range of different but related techniques which deal with reasoning under uncertainty. Both quantitative (mainly using Bayesian probabilistic methods) and qualitative techniques are used. Influence diagrams are an extension to belief networks; they are used when working with decision making. Belief networks are used to develop knowledge based applications in domains which are characterised by inherent uncertainty. Increasingly, belief network techniques are being employed to deliver advanced knowledge based systems to solve real world problems. Belief networks are particularly useful for diagnostic applications and have been used in many deployed systems. The free-text help facility in the Microsoft Office product employs Bayesian belief network technology. Within a belief network the belief of each node (the node’s conditional probability) is calculated based on observed evidence. Various methods have been developed for evaluating node beliefs and for performing probabilistic inference. Influence diagrams, which are an extension of belief networks, provide facilities for structuring the goals of the diagnosis and for ascertaining the value (the influence) that given information will have when determining a diagnosis. In influence diagrams, there are three types of node: chance nodes, which correspond to the nodes in Bayesian belief networks; utility nodes, which represent the utilities of decisions; and decision nodes, which represent decisions which can be taken to influence the state of the world. Influence diagrams are useful in real world applications where there is often a cost, both in terms of time and money, in obtaining information.

The basic idea in belief networks is that the problem domain is modelled as a set of nodes interconnected with arcs to form a directed acyclic graph. Each node represents a random variable, or uncertain quantity, which can take two or more possible values. The arcs signify the existence of direct influences between the linked variables, and the strength of each influence is quantified by a forward conditional probability.

The Belief Network, which is also called the Bayesian Network, is a directed acyclic graph for probabilistic reasoning. It defines the conditional dependencies of the model by associating each node X with a conditional probability P(X|Pa(X)), where Pa(X) denotes the parents of X. Here are two of its conditional independence properties:

1. Each node is conditionally independent of its non-descendants given its parents.

2. Each node is conditionally independent of all other nodes given its Markov blanket, which consists of its parents, children, and children’s parents.

The inference of Belief Network is to compute the posterior probability distribution

P(H|V) = P(H,V)/ ∑_HP(H,V)

where H is the set of the query variables, and V is the set of the evidence variables. Approximate inference involves sampling to compute posteriors. The Sigmoid Belief Network is a type of the Belief Network such that

P(X_i = 1|Pa(X_i)) = σ( ∑_{Xj ∈ Pa(Xi)} W_jiX_j + b_i)

where W_ji is the weight assigned to the edge from X_j to X_i, and σ is the sigmoid function.

Conjuncted: Axiomatizing Artificial Intelligence. Note Quote.

Solomonoff’s work was seminal in that he has single-handedly axiomatized AI, discovering the minimal necessary conditions for any machine to attain general intelligence.

Informally, these axioms are:

AI0 AI must have in its possession a universal computer M (Universality). AI1 AI must be able to learn any solution expressed in M’s code (Learning recursive solutions).
AI2 AI must use probabilistic prediction (Bayes’ theorem).
AI3 AI must embody in its learning a principle of induction (Occam’s razor).

While it may be possible to give a more compact characterization, these are ultimately what is necessary for the kind of general learning that Solomonoff induction achieves. ALP can be seen as a complete formalization of Occam’s razor (as well as Epicurus’s principle)  and thus serve as the foundation of universal induction, capable of solving all AI problems of significance. The axioms are important because they allow us to assess whether a system is capable of general intelligence or not.

Obviously, AI1 entails AI0, therefore AI0 is redundant, and can be omitted entirely, however we stated it separately only for historical reasons, as one of the landmarks of early AI research, in retrospect, was the invention of the universal computer, which goes back to Leibniz’s idea of a universal language (characteristica universalis) that can express every statement in science and mathematics, and has found its perfect embodiment in Turing’s research. A related achievement of early AI was the development of LISP, a universal computer based on lambda calculus (which is a functional model of computation) that has shaped much of early AI research.

Minimum Message Length (MML) principle introduced in 1968 is a formalization of induction developed within the framework of classical information theory, which establishes a trade-off between model complexity and fit-to-data by finding the minimal message that encodes both the model and the data. This trade-off is quite similar to the earlier forms of induction that Solomonoff developed, however independently discovered. Dowe points out that Occam’s razor means choosing the simplest single theory when data is equally matched, which MML formalizes perfectly (and is functional otherwise in the case of inequal fits) while Solomonoff induction maintains a mixture of alternative solutions. However, it was Solomonoff who first observed the importance of universality for AI (AI0-AI1). The plurality of probabilistic approaches to induction supports the importance of AI3 (as well as hinting that diversity of solutions may be useful). AI2, however, does not require much explanation. 

Conceptual Jump Size & Solomonoff Induction. Note Quote.

Let M be a reference machine which corresponds to a universal computer with a prefix-free code. In a prefix-free code, no code is a prefix of another. This is also called a self-delimiting code, as most reasonable computer programming languages are. Ray Solomonoff inquired the probability that an output string x is generated by M considering the whole space of possible programs. By giving each program bitstring p an a priori probability of 2^−|p|, we can ensure that the space of programs meets the probability axioms by the extended Kraft inequality. An instantaneous code (prefix code, tree code) with the code word lengths l₁,…,l_N exists if and only if

∑_i=1^N 2^-L_i ≤ 1

In other words, we imagine that we toss a fair coin to generate each bit of a random program. This probability model of programs entails the following probability mass function (p.m.f.) for strings x ∈ {0, 1}∗:

P_M(x) = ∑_M(p)=x* 2^-|p| —– (1)

which is the probability that a random program will output a prefix of x. P_M(x) is called the algorithmic probability of x for it assumes the definition of program-based probability.

Using this probability model of bitstrings, one can make predictions. Intuitively, we can state that it is impossible to imagine intelligence in the absence of any prediction ability: purely random behavior is decisively non-intelligent. Since, P is a universal probability model, it can be used as the basis of universal prediction, and thus intelligence. Perhaps, Solomonoff’s most significant contributions were in the field of AI, as he envisioned a machine that can learn anything from scratch.

His main proposal for machine learning is inductive inference (Part 1, Part 2), for a variety of problems such as sequence prediction, set induction, operator induction and grammar induction. Without much loss of generality, we can discuss sequence prediction on bitstrings. Assume that there is a computable p.m.f. of bitstrings P₁. Given a bitstring x drawn from P₁, we can define the conditional probability of the next bit simply by normalizing. Algorithmically, we would have to approximate (1) by finding short programs that generate x (the shortest of which is the most probable). In more general induction, we run all models in parallel, quantifying fit-to-data, weighed by the algorithmic probability of the model, to find the best models and construct distributions; the common point being determining good models with high a priori probability. Finding the shortest program in general is undecidable, however, Levin search can be used for this purpose. There are two important results about Solomonoff induction that we shall mention here. First, Solomonoff induction converges very rapidly to the real probability distribution. The convergence theorem shows that the expected total square error is related only to the algorithmic complexity of P₁, which is independent from x. The following bound is discussed at length with a concise proof:

E_P [∑_m=1ⁿ (P(a_m+1 = 1|a₁a₂ …a_m) – P₁(a_m+1 = 1|a₁a₂…a_m))²] ≤ -1/2 ln P(P₁) —– (2)

This bound characterizes the divergence of the Algorithmic Probability (ALP) solution from the real probability distribution P₁. P(P₁) is the a priori probability of P₁ p.m.f. according to our universal distribution P_M. On the right hand side of (2), −lnP_M(P₁) is roughly kln2 where k is the Kolmogorov complexity of P₁ (the length of the shortest program that defines it), thus the total expected error is bounded by a constant, which guarantees that the error decreases very rapidly as example size increases. In algorithmic information theory, the Kolmogorov complexity of an object, such as a piece of text, is the length of the shortest computer program  that produces the object as output. It is measure of the computational resources needed to specify the object, and is also known as descriptive complexity, Kolmogorov–Chaitin complexity, algorithmic entropy, or program-size complexity. Secondly, there is an optimal search algorithm to approximate Solomonoff induction, which adopts Levin’s universal search method to solve the problem of universal induction. Universal search procedure time-shares all candidate programs according to their a priori probability with a clever watch-dog policy to avoid the practical impact of the undecidability of the halting problem. The search procedure starts with a time limit t = t₀, in its iteration tries all candidate programs c with a time limit of t.P(c), and while a solution is not found, it doubles the time limit t. The time t(s)/P (s) for a solution program s taking time t(s) is called the Conceptual Jump Size (CJS), and it is easily shown that Levin Search terminates in at most 2.CJS time. To obtain alternative solutions, one may keep running after the first solution is found, as there may be more probable solutions that need more time. The optimal solution is computable only in the limit, which turns out to be a desirable property of Solomonoff induction, as it is complete and uncomputable.

Roger Penrose and Artificial Intelligence: Revenance from the Archives and the Archaic.

Let us have a look at Penrose and his criticisms of strong AI, and does he come out as a winner. His Emperor’s New Mind: Concerning Computers, Minds, and The Laws of Physics

sets out to deal a death blow to the project of strong AI. Even while showing humility, like in saying,

My point of view is an unconventional among physicists and is consequently one which is unlikely to be adopted, at present, by computer scientists or physiologists,

he is merely stressing on his speculative musings. Penrosian arguments ala Searle, are definitely opinionated, in making assertions like a conscious mind cannot work like a computer. He grants the possibility of artificial machines coming into existence, and even superseding humans (1), but at every moment remains convinced that algorithmic machines are doomed to subservience. Penrose’s arguments proceed through showing that human intelligence cannot be implemented by any Turing machine equivalent computer, and human mind as not algorithmically based that could capture the Turing machine equivalent. He is even sympathetic to Searle’s Chinese Room argument, despite showing some reservations against its conclusions.

The speculative nature of his arguments question people as devices which compute that a Turing machine cannot, despite physical laws that allow such a construction of a device as a difficult venture. This is where his quantum theory sets in, with U and R (Unitary process and Reduction process respectively) acting on quantum states that help describe a quantum system. His U and R processes and the states they act upon are not only independent of observers, but at the same time real, thus branding him as a realist. What happens is an interpolation that occurs between Unitary Process and Reductive Process, a new procedure that essentially contains a non-algorithmic element takes shape, which effectuates a future that cannot be computable based on the present, even though it could be determined that way. This radically new concept which is applied to space-time is mapped onto the depths of brain’s structure, and for Penrose, the speculative possibility occurs in what he terms the Phenomenon of Brain Plasticity. As he says,

Somewhere within the depths of the brain, as yet unknown cells are to be found of single quantum sensitivity, such that synapses becoming activate or deactivated through the growth of contraction of dendritic spines…could be governed by something like the processes involved in quasi-crystal growth. Thus, not just one of the possible alternative arrangements is tried out, but vast numbers, all superposed in complex linear superposition.

From the above, it is deduced that the impact is only on the conscious mind, whereas the unconscious mind is left to do with algorithmic computationality. Why is this important for Penrose is, since, as a mathematician believes in the mathematical ideas as populating an ideal Platonic world, and which in turn is accessible only via the intellect. And harking back to the non-locality principle within quantum theory, it is clear that true intellect requires consciousness, and the mathematician’s conscious mind has a direct route to truth. In the meanwhile, there is a position in “many-worlds” (2) view that supersedes Penrose’s quantum realist one. This position rejects the Reduction Process in favor of Unitary Process, by terming the former as a mere illusion. Penrose shows his reservations against this view, as for him, a theory of consciousness needs to be in place prior to “many-worlds” view, and before the latter view could be squared with what one actually observes. Penrose is quite amazed at how many AI reviewers and researchers embrace the “many-worlds” hypothesis, and mocks at them, for their reasons being better supportive of validating AI project. In short, Penrose’s criticism of strong AI is based on the project’s assertion that consciousness can emerge by a complex system of algorithms, whereas for the thinker, a great many things humans involve in are intrinsically non-algorithmic in nature. For Penrose, a system can be deterministic without being algorithmic. He even uses the Turing’s halting theorem (3) to demonstrate the possibility of replication of consciousness. In a public lecture in Kolkata on the 4th of January 2011 (4), Penrose had this to say,

There are many things in Physics which are yet unknown. Unless we unravel them, we cannot think of creating real artificial intelligence. It cannot be achieved through the present system of computing which is based on mathematical algorithm. We need to be able to replicate human consciousness, which, I believe, is possible through physics and quantum mechanics. The good news is that recent experiments indicate that it is possible.

There is an apparent shift in Penrosean ideas via what he calls “correct quantum gravity”, which argues for the rational processes of the mind to be completely algorithmic and probably standing a bright chance to be duplicated by a sufficiently complex computing system. As he quoted from the same lecture in Kolkata,

A few years back, scientists at NASA had contemplated sending intelligent robots to space and sought my inputs. Even though we are still unable to create some device with genuine feelings and understanding, the question remains a disturbing one. Is it ethical to leave a machine which has consciousness in some faraway planet or in space? Honestly, we haven’t reached that stage yet. Having said that, I must add it may not be too far away, either. It is certainly a possibility.

Penrose does meet up with some sympathizers for his view, but his fellow-travelers do not tread along with him for a long distance. For example, in an interview with Sander Olson, Vernor Vinge, despite showing some reluctance to Penrose’s position, accepts that physical aspects of mind, or especially the quantum effects have not been studied in greater detail, but these quantum effects would simply be another thing to be learned with artifacts. Vinge does speculate on other paradigms that could be equally utilized for AI research hitting speed, rather than confining oneself to computer departments to bank on their progress. His speculations (5) have some parallel to what Penrose and Searle would hint at, albeit occasionally. Most of the work in AI could benefit, if AI, neural nets are closely connected to biological life. Rather than banking upon modeling and understanding of biological life with computers, if composite systems relying on biological life for guidance, or for providing features we do not understand quite well as yet to be implemented within the hardware, could be fathomed and made a reality, the program of AI would undoubtedly push the pedal to accelerate. There would probably be no disagreeing with what Aaron Saenz, Senior Editor of singularityhub.com said (6),

Artificial General Intelligence is one of the Holy Grails of science because it is almost mythical in its promise: not a system that simply learns, but one that reaches and exceeds our own kind of intelligence. A truly new form of advanced life. There are many brilliant people trying to find it. Each of these AI researchers have their own approach, their own expectations and their own history of failures and a precious few successes. The products you see on the market today are narrow AI-machines that have very limited ability to learn. As Scott Brown said, “today’s I technology is so primitive that much of the cleverness goes towards inventing business models that do not require good algorithms to succeed.” We’re in the infantile stages of AI. If that. Maybe the fetal stages.

(1) This is quite apocalyptic sounding like the singularity notion of Ray Kurzweil, which is an extremely disruptive, world-altering event that has the potentiality of forever changing the course of human history. The extermination of humanity by violent machines is not impossible, since there would be no sharp distinctions between men and machines due to the existence of cybernetically enhanced humans and uploaded humans.

(2) “Many-worlds” view was first put forward by Hugh Everett in 1957. According to this view, evolution of state vector regarded realistically, is always governed by deterministic Unitary Process, while Reduction Process remains totally absent from such an evolutionary process. The interesting ramifications of this view are putting conscious observers at the center of the considerations, thus proving the basic assumption that quantum states corresponding to distinct conscious experiences have to be orthogonal (Simon 2009). On the technical side, ‘Orthogonal’ according to quantum mechanics is: two eigenstates of a Hermitian operator, ψm and ψn, are orthogonal if they correspond to different eigenvalues. This means, in Dirac notation, that < ψm | ψn > = 0 unless ψm and ψn correspond to the same eigenvalue. This follows from the fact that Schrödinger’s equation is a Sturm–Liouville equation (in Schrödinger’s formulation) or that observables are given by hermitian operators (in Heisenberg’s formulation).

(3) Halting problem is a decisional problem in computability theory, and is stated as: Given a description of a program, decide whether the program finishes running or continues to run, and will thereby run forever. Turing proved that a general algorithm to solve the halting problem for all possible program-input pairs cannot exist. In a way, the halting problem is undecidable over Turing machines.

(4) Penrose, R. AI may soon become reality. Public lecture delivered in Kolkata on the 4th of Jan 2011. <http://timesofindia.indiatimes.com/city/kolkata-/AI-may-soon-become-reality- Penrose/articleshow/7219700.cms>

(5) Olson, S. Interview with Vernor Vinge in Nanotech.Biz <http://www.nanotech.biz/i.php?id=01_16_09&gt;

(6) Saenz, A. Will Vicarious Systems’ Silicon Valley Pedigree Help it Build AI? in singularityhub.com <http://singularityhub.com/2011/02/03/will-vicarious-systems-silicon-valley-pedigree-help-it-build-agi/&gt;

Conjectures of Capitalism and Organic Necrocracy, RN

Following a reading of Freud and Deleuze in their use of death drive, Reza Negarestani tells us that capitalism forges an inhuman model that “weds the concrete economy of human life to a cosmos where neither being nor thinking enjoys any privilege.” Taking his trajectory from the investigation of Nick Land in his “The Thirst for Annihilation“, he tells us that “what brings about this weird marriage between human praxis and inhuman emancipation is the tortuous economy of dissipation inherent to capitalism as its partially repressed desire for meltdown.” According to a quote from Land, “What appears to humanity as the history of capitalism is an invasion from the future by an artificially intelligent space that must assemble itself from an enemy’s resources.” Negarestani compares this emancipatory capitalism with HP Lovecraft‘s fantastic concept of ‘holocaust of freedom’, which celebrates the consummation of human doom with human emancipation.

Jaron Lanier gets real about AI & Acceleration

From CS: “The whole thing eats itself.” I, totally think we need to endeavor to birth new systems, not prop up the outmoded. Life feeds on life, canibalalize the old and over time use the bones to build something that’s compatible with the proper (non self defeating or borderline psychotically irresponsible) use of our technological abilities.

Thanks to Syn0