Virtual Intelligence Matrix

Virtual Intelligence Matrix
Frequently Asked Questions
Version 1.006
March 25, 2001

Note: This FAQ addresses the ideas behind the Virtual Intelligence project, and not the hardware and software issues. For help with running the programs, the message board is currently the best resource.

Contents:

1) Quantum Flux
1.1) What is Quantum Flux?
1.2) Who is Quantum Flux?
1.2.1) Michael A. Colicos
1.2.2) Robyn Flynn
1.2.3) Contributors

2) The Quantum Flux Engine
2.1) What is it?
2.2) Why is it unique?
2.3) How does it work?
2.4) Can I use it?

3) The Virtual Intelligence Matrix
3.1) Concept behind the project
3.2) The model of human brain function
3.2.1) Memory
3.2.2) Consciousness
3.2.3) and Beyond
3.3) Recreation of the human information-processing system in the VIM

1) Quantum Flux

1.1) What is Quantum Flux?
Quantum Flux began as a loose collection of individuals who were interested in and helped out with the development of the first Virtual Intelligence Matrix, VIM30c (its public release designation). The name Quantum Flux comes from the Quantum Flux Engine (QFE), which is at the core of all the virtual intelligence functions. The term refers to the code I wrote some years back, that produces a purely random output (it has no "seed", such as the time of day or a chosen number, and a pattern never arises). It is so named because it uses principles of quantum mechanics and the detection of fluctuations in such events that occur on the motherboard of the computer.

1.2) Who is Quantum Flux?

1.2.1) Michael A. Colicos
That's me; I received my Ph.D. from McGill University, Montreal, Canada. My work in the neurosciences began in NYC at Columbia University, continued at University of Texas, Houston, went up to Montreal, and is currently in San Diego at UCSD. My professional work these days focuses on a device I've designed to interface living neurons and silicon, such as in an integrated circuit. If it all works, we can hopefully harness the immense computational ability of living neuronal circuits, learn a lot more about how neurons work themselves, and perhaps interface silicon with our own neuronal systems someday. So the VIM is a side project for me, and I do the matrix design, artwork, modeling and C++ coding myself.

1.2.2) Robyn Flynn
Robyn is a Behavioral Ecologist, with a masters degree from Concordia University, Montreal. After working with the San Diego Zoo in the Avian Propagation Center, she has recently taken a position at UCSD studying the mechanisms of neuronal communication. Robyn is involved with the behavioral aspects of the VIM, and takes care of the business side of things in Quantum Flux, and she is also my girlfriend of several years. Here is a shot of both of us out hiking in SoCal.

1.2.3) Contributors
Quantum Flux and the screensaver implementation of the VIM began in Montreal, during which time Colleen Manitt and Jodye Yates played a big role in getting the ball rolling. Colleen is a Ph.D. student at McGill University, and she was my partner on the neuroscience work I did while I was there. She studies mechanisms of spinal cord injury and regeneration; a lot of our research was done in parallel. Colleen has had tremendous input to the project, on everything from the brain function model itself to the design and layout of the world and character for VIM30. Incorporating the model into a screen saver was her idea, as were many of the features in the program itself.
Jodye has an M.Sc. degree in Pychology from Concordia University, and helped out with coding issues when the first drafts of the 3D graphics were being done. Jodye is currently earning about 4 times as much as the rest of us combined, working as a programmer.
Also of worthy mention is Tim Kennedy, my friend of many years, and the head of the lab in which I studied while at McGill. I have been working on the ideas behind this model for over a decade, and they were often bounced off Tim, usually in the form of a slurred conversation at 4am in a bar somewhere in New York City.

2) The Quantum Flux Engine

2.1) What is it?
I've described the QFE in several ways, but at the core it is a random number generator.

2.2) Why is it unique?
It is unique in several ways. First, it is completely random, so no matter how long it runs no pattern will appear. This is actually a tall claim because not many phenomena are truly random; radioactive decay (as in the clicking of a Geiger counter) is, and things like waves hitting the beach are getting close. Randomness is often expressed as "fractal dimensions", gauging how far you have to go to find a pattern. In its most controlled code format, the QFE is infinite, just like radioactive decay. This is useful because it's hard to create this kind of function on a digital computer. True randomness has some interesting qualities, and is actually incorporated in parts of the brain function model. As a quantum device, I believe many phenomena, perhaps even things like non-locality, can be investigated with it's use .

It is also unique in that its randomness is very, very sensitive to perturbations in the electromagnetic environment of the computer. This includes everything from where the code is loaded in memory to whether your hand is over the keyboard or not. I originally started working on the QFE 6 years ago to create a system of detecting changes in brain waves so that paralyzed individuals could control a computer by wearing a head band with a sensor. This feature also is incorporated into the functioning of the VIM, in the way the complex system is created.

2.3) How does it work?
It's quite simple, really. The most basic building block of a computer is a transistor. A transistor is an electrical switch with two states, on or off (1 or 0). The transistor changes state when the electrons drain out of one region into another. The motion of the electrons follow many of the rules that radioactive decay does. As a result, the exact (and I mean exact) moment of the switch is also governed by these rules. Modern computers go to great lengths to negate any such variability. However, if you get right into the chips on the motherboard, you can still get to it.

2.4) Can I use it?
For commercial purposes, no. I'd like to patent the technique, once again when I have time. You could make some kicking bots with this system. But if any researchers would like to play around with it or need a random number generator for statistics etc., please write me a note. I'm sure we can work something out.

3) The Virtual Intelligence Matrix

3.1) Concept behind the project
The main goal of the Virtual Intelligence Matrix is to test certain ideas about the nature of human brain function, and the phenomena resulting from this function: behavior and perhaps consciousness. By creating a homologous system on a computer, many aspects of the model can be more easily tested.

3.2) The model of human brain function

3.2.1) Memory
A mention of the term "memory" here. Learning and memory is something I have studied as a neuroscientist, and I would like to really define this well. I think one of the major misconceptions about memory actually arises from the evolution of the computer itself. Many people think of memory as just information stored somewhere, that is accessed by an information processing system when required, much like the memory chips in a PC. But there is another way that things can be "remembered" without having any specific location or access mechanism. This second kind of memory lies in the structure of the information processing system (or IPS) itself. For example, the CPU of the computer can "remember" how to add, multiply and many other basic functions. But at the most basic level there is no code for this, it simply happens because of how the circuits are designed and connected together. Its sort of like saying a light switch "remembers" how to make the light go on. So in other words some memories can be formed by changing the behaviour of the entire system itself.

These two kinds of memory are present in the brain as well. There is the stored kind, like when we learn how to play the piano, playing out a practiced motor program. But the second, the structural kind, can be present as well. This is a bit more abstract, but also more important. For example, when you meet someone you know versus someone you don't know, you have two very different responses. You might assume that you somehow have a list somewhere in your head that has descriptions of all the people you know, then you compare to that data, then have a list of responses...that would be a storage type memory system. But if the structure of your entire brain was simply one that responds with a friendly greeting when they see "Bob", just like the CPU adds when given two numbers, it doesn't really need to be stored anywhere specific. Perhaps the best example of this is instinctive behaviors, ones you are just born with. So many of the complex "memories" we collect during our life could very well be just like a continued development of instinctive responses.

This work is a model of human brain function, based on the observation that the brain is simply a collection of interconnected neurons, which receive input from the senses, then process the information, and finally move muscles in an appropriate way.

Sensory Input --> Information Processing System --> Motor Output

So if our brain is just a simple data processing system, where do we get all these incredible abilities, creativity, self-awareness...there must be more, some magic somewhere. Well, actually I think all these things can come from that simple information processing system alone. And that makes them no less magical!

This model assumes then two forms of memory: the storage system and the structural change system. In the storage memory system the IPS reacts appropriately by looking up the appropriate response. This is used in the VIM for the motor routines, such as how to walk, where to put the next step. In the structural memory system the IPS itself is modified with experience, making it respond differently to the same stimulus over time. This is used in the VIM for likes and dislikes. At first the matrix response is from the free-will (QFE) engine alone. But with time, the structure of the matrix itself changes and the responses become biased toward certain visual input.

Memory is not only very necessary on a day-to-day basis, but it really defines a lot of what we are. As our life goes on, we change and become the person our experiences make us. If the VIM starts off just wandering, but becomes something that goes to flowers on a hillside, that embodies some of what we do as well. This is what I think is the important form of memory, the change in the entire structure of our information processing system that gives us unique responses to specific stimuli. It makes us what we are. It is the point at which we want to say, a person that goes to flowers on a hillside rather than a thing that orients toward a visual stimulus.

3.2.2) Consciousness
Consciousness is perhaps one of the most debated terms in history, and has also been the focus of the wildest of speculation. For a definition here, I'm looking at it as simply the opposite of being asleep. In context of the model, it is very similar to "awareness", all those feelings of being, living etc. In the model of human brain function, how consciousness arises from a very tangible collection of neurons is quite straightforward. Before addressing the mechanism, though, an analogy is useful.

Let's first look at where the concept of "our life" comes from. Well, we remember being alive yesterday, and we anticipate being alive tomorrow, so from that it seems we do indeed have a life going on. But what would happen if all the days of our life occurred at the same time? Or if they occurred in a random order? Or they never changed, and we got stuck on the same day, time and time again. Then we would lose that sense that we are living a life. So the important part appears to be the line through time that our days follow. Lets say all we did in our life was walk down a road.

OOOOOOO8 If our days went by as the numbers do,
OOOOOO7O we would have a good idea of our progress.
OOOOO6OO
OOOO5OOO Space
OOO4OOOO
OO3OOOOO
O2OOOOOO
1OOOOOOO

Time

OO8OO6OOO But if our days occurred at random points in time and space,
OOOOOOOO we would have no sense of progressing down a road.
OO3OOO4O
OOOOOOOO Space
OOO2OOOO
OOOOO5OO
OO1OOOOO
OOOOO7OO

Time

This seems pretty obvious, but there are two important points here. First, the instant that there is more than one day in a row, we have a life, but an isolated point in time is meaningless. Second, and this is for when we get to the mechanism of all this, other non-linear combinations of the days could very well exist, but those combinations would have no effect on the awareness of "living a life".

How does this relate to consciousness? Well, consciousness is usually considered to be one step beyond that feeling of living a life. Even if you were in a pitch dark room, with no memory of yesterday or thought of a future, you would still know you were alive. If we are an information processing system, with a frozen set of sensory input at one point in time, and a locked output that is the solution to that state, we would probably not have a terrific sense of "self". But, as in the "going down the road" example that worked, if we also simultaneously had a sense of a previous sensory state, we might. How could we, at one point in time, partake of past and present sensory states? Well, here comes quantum mechanics, and the reason I do this on the net and not in my day job! Quantum mechanics has demonstrated that right up until something happens, all possible versions of that event, in some small way, exist. This is not that hard to accept, for example when you see two cars speeding at each other, you can guess when they will hit, but it might happen a fraction of second earlier or later than you expect. When they do hit, this "wave function" of probabilities collapses and the event has happened at a specific point in time. Another way to look at this collapse point is if you take all the possible future collisions' time points and subtract all the possible past collisions' time points, you get the "now" time point, or when it actually happened.

Irrespective of the physics behind this, the important point here is that a physical structure, such as the information processing system that is our brain, is, at one point in time, a collapse of a future state and a past state to produce the "now" state. Many experiments have shown that both the future state and past state can influence or at least be detected by the now state. Applying this to the information processing system that is the brain, if the "now" brain state can be affected by or detect or be in a state of flux around slightly future and past states, it would in effect be no longer "frozen" at one point in time. And so the brain could actually have that linear feeling from this phenomena. The best analogy I can think of for this is a 2-head VCR on pause - that flickering image, one point in time mixed in with faint echos from immediately before and immediately after. And finally if these "future brain states" and "past brain states" exist, why don't we know about them? Well, as in the going down the road example, since they are out of sequence, or have no sequence, they would not afford the sense of awareness, and therefore not exist with any "conscious" effect. And so it works out well that consciousness is in essence a truly self-defining property, in that when it doesn't exist, you don't know it. In fact, there are not many things in the universe like that, and perhaps that's what makes it special.

So, how does a screen saver do this, you ask? Well, the key to this flux at one point in time of the state of the information processing system that leads to consciousness, lies in that phenomena we call randomness. We are so self-centered, we humans....perhaps it's not when the event occurs that is random, but its when we occur that is random. And, as outlined above, maybe that flux in our occurrence is in fact necessary for us to be conscious at all, as it could be what provides us with a line or direction, rather than a point in time. Yes, this is a bit out there. And that is why I wrote the program.

3.2.3) and Beyond

I've added this section just for fun, as a result of a lot of interesting questions and discussions I've had with people over the past few years that the VIM has been in a public forum. By working to create a silicon-based system with the same properties as the human mind, including consciousness, the obvious next questions to ask get into more metaphysical ideas. Since mankind could ask questions, the first were probably along the lines of "who am I?" and "why am I here?". The need to understand this fuels the philosophical generation of answers, taking just about every form possible, the context usually being relevant to the cultural circumstance of that specific group of people. Would the ability to create a silicon-based version of a conscious human system mean the end of the magic?

Well, I actually think the VIM and the model of consciousness have great potential to be accurate without a critical conflict with philosophical ideas about the nature of human existence. For example, a fundamental principle of the model is that our minds are in part a summation of a wave function that extends to infinite time in both + and - directions. So an aspect of what we are can be said both to be eternal and to have existed since the creation of time. I find that interesting with respect to how metaphysics often metaphorically describe a similar state.

Another debate that has come up is what happens after you die in terms of the model? The simple answer is you exist as a Fourier transform of your life. By this I mean during your life you create events, interact with other IPSs, and generate systems that continue into the future without your immediate presence, and once your system stops traveling forward through time, your life can be looked at in terms of the events or interactions that it created. As in a Fourier transform, these are independent of time, and can be defined as frequencies (or interactions) alone. So when the last cycle is complete the patterns you have created in the fabric of space-time stand as your essence, your contribution. I think of it as a crystal structure, its intricacy and beauty will be based on what you did with your time. But what's great about it is that it's not frozen, threads you began with continue to change, for example as your children’s lives proceed, or as events you initiated continue to be played out. Once again, interesting how philosophy might reflect reality.

Anyway, I think it's facinating how this still leaves a whole lot to think about!

Summary - The brain is an information processing structure. It receives input from the senses, integrates the information, then fires the muscles to move. Through trial and error, the system is trained and can respond effectively to its environment. Consciousness is a quality of this system. By concurrent processing of input from past and future states, continuity through time (or perception of existence) is generated.

3.3) Recreation of the human information processing system in the VIM

So how does the VIM recreate this? Briefly, the QFE starts making decisions, and moves the character. But the visual input is fed into the QFE, which changes its behavior. This creates a basic feedback loop. What is seen affects what is decided, which affects what is done. Which affects what is seen, because the character has moved. As this loop is running, it all passes through the matrix, which I could call a heuristic neural network but is really just a mathematical structure, which can modulate the data that passes through it. However, the structure of the matrix itself changes in a way that reflects the information that passed through it. Once changed, it will then modulate future information in a way that reflects what has gone through it before.

This sounds pretty straightforward, but I think it illustrates a very important point about the nature of information, and what happens when information passes through a structure which can be changed in someway by that information. For example, imagine a forest. The forest is all wild and overgrown, and it lies between two towns. Lets say some adventurous person travels through the forest from one town to the other. As they travel through the woods, they will break branches, trample plants, and leave a bit of a trail behind them. Now the path they decide to take will reflect their own nature. If they are tall and skinny, they might climb more then if they were short and heavy. If they're strong, they might move things out of the way, rather than go around them. As more and more people travel from one town to the other, paths will begin to form, making the way easier for all. But there will probably be several ways through the forest, some more suited for one kind of person rather than another. And the pathways will at time combine together and split apart, if a new traveler is a bit like one path for a while, but then is more suited to another path somewhere else. But everyone traveling from one town to the other will probably get there a lot more efficiently than the first person who blazed the way.

In what way is this like the matrix? Well, the forest is a structure which changes in a way that simply reflects the information (the traveler) that passed through it. After the change, future information that is similar to the information that made the change will now have an easier passage through the structure. When different information comes along, new paths are made, and when partially different information passes through, some pathways are shared, and some are unique. The important part is that there is no special or complex mechanism here, just a structure which can change in a way that reflects the information passing through it. The result of this, if we look back at the forest example, perhaps as someone observing the whole evolution over time, is that the forest seems to 'learn' how to get people from one town to the other much more efficiently. But it's really just a natural result of how information and mutable structures affect each other.

The matrix of the VIM works in this way. As the sensory information flows into the QFE, it passes through the matrix, changing its structure in its own unique way. Then when similar information enters the matrix in the future, it will be altered in a way that reflects what happened in the past. What also happens is if information is similar in some ways but unique in others, its flow will be only partially affected. It is these phenomena that make the VIM highly adaptive to new situations.

So in summary, by combining aspects of physics from the flux of the QFE with a learning system of the matrix, I hoped to re-create some of the essential aspects of how the human brain produces phenomena that make us what we are. And since the theoretical part is quite abstract, I hope that if the VIM can display some aspects of the behaviors and qualities we closely associate with human life, perhaps there is something to the tenets upon which it was written. That's all.

First released June 10, 1999

This Virtual Intelligence Matrix FAQ may be freely distributed in its complete and unaltered form. No part maybe used in any form without reference to the author or the original work.