6. Neural Networks - Processing Units

I have spent enough time online and books to study how the neuron or neural networks carry out their data processing and what their techniques are. Somehow everywhere what I find is the ANN (Artificial Neural Network) or DNNA (Digital Neural Network Architecture) etc. While most of them explain the techniques and algorithms what scientists and mathematicians have come up with for an artificial neural network based on the basics of how our brain works or neural network processes data.

However, I hardly find information about how exactly our human brain or neural networks work! That increased my curiosity and I sincerely wanted to figure it out. In all the ANN concepts, neurons are described to be associated with weights (biologically equivalent to amount of voltage of impulse) and that gets carried over various number of layers of neurons before finally brain concludes anything about what is observed. The following diagram is a typical example for a multilayer neural network which is widely used in Artificial Intelligence applications like pattern recognition, super computations etc.

In the above picture, X1, X2.. Xk are input points and they are linked to each and every node in the next layer (hidden layer). So, each node or neuron in the hidden layer has inputs from various input points and it determines its results by summing up all the inputs as shown in the below formula. That output is fed as input to the neurons that are in the next hidden layer or the output node or neuron.

From this model the interval activity of the neuron can be shown to be:

Here vk is not the result, the actual result is nothing but the final output of the neurons in the output layer, yk, which is determined by some activation function on the value of vk. What is activation function? It can be described as follows. Consider an example of how you consider or eliminate items from the list of items based on it's price. You have your budget and only if it falls under, then you tick it otherwise you would strike it or push it for later.

Similarly, activation function is a function which has a threshold value set and if the inputs fall under its threshold value, it returns an output value and if not, it will return a different value! There are three types of activation functions used in this field. First, there is the Threshold Function which takes on a value of 0 if the summed input is less than a certain threshold value (v), and the value 1 if the summed input is greater than or equal to the threshold value.


Secondly, there is the Piecewise-Linear function. This function again can take on the values of 0 or 1, but can also take on values between that depending on the amplification factor in a certain region of linear operation.



Thirdly, there is the sigmoid function. This function can range between 0 and 1, but it is also sometimes useful to use the -1 to 1 range. An example of the sigmoid function is the hyperbolic tangent function.




In all three types of activation function, the only attribute used is 'weights'. There is a thing to be clarified about weights. Neurons are connected with more than one other neurons. We know each of the neuron has (possibly unique) weights associated with it, I mean one weight per neuron. However, the ANN depicts that neurons can have weights per connection not as one value for all the connection! ANN models and real-time applications work with this idea and that makes it believable. So, the electric impulses vary per connection/links. It means that neuron determines the weight for each link that connects to each other neuron. In order to have a better idea about weights, we need an example. Lets explore one here.



According to me, each neuron does not only have weight but it is also tied to a feature or the attribute of the object that is tagged to that neuron or set of neurons. It's also possible that more than one neurons are allocated for same feature in order to carry out the approximation. So based on which sense sends the information, this feature is determined for each of the neurons. I also conclude that not all the neurons, in our brain, are tagged to a feature because few of them only returns the outputs. Or it may be that the neurons handle it's feature only when they are in the input or hidden layers and simply switch the feature off when the same neuron acts in the output layer.

When it becomes impossible for a neural network to understand a subject, it allocates or fires more neurons into its network or it's loop. When a subject creates confusion and the final outcome is different compared to it's knowledge, then the neurons adjust their weights based on how much each of the feature differs and try to come up the with the same overall value. Hence the neural network is knows how to approximate but it doesn't do unless the external force acts up on it!

Yes, it's like Newton's first law 'Objects remain idle until external force acts on it'. According to me, the force could be any kind of basic instinct to feed, live (ex: hunger) or teaching force (ex: teaching children). For example, lets say a child learned what is color 'Red' with the value 'X'. Now, if you show yellow to the child and child recognizes the color as 'Yellow'! Now, lets take a color 'Yellowish Red' (ex: sunset) which has more Red in it. What does happen when this is shown to a child and taught as 'Red but little yellow'?

Even though the child's neurons come up with value 'Y' since the color is not exactly 'Red', because of the authority of teaching and knowledge, when we teach the child, the child's neurons will adjust their weights in a way so that the overall value comes to X in order to accept that this color is still 'Red'! During this adjustment or approximation, if neurons finds that it can't conclude with available neurons, it will start allocating or using the unused nearby neurons into the loop to get the value 'X'.

This remains possible only with the help of other senses like hearing and vision. It gets difficult as we intend to teach the kids who are physically challenged children such as born-blind or born-deaf as the number of attributes will be limited! Same object can be realized at different limits depends on how limited the observer's senses are.

The number of attributes of an object is as same as the number of senses that the observer has; at least that's how the object is seen by the observer and Science buys only what can be seen and so is least interested in something which can't be seen or shown! Hence Science is also a subjective matter like spiritual experience; the only difference is spiritual experience can't be explained or shared while it is possible with scientific experiments!

Lets consider a 'red cubic box' that continuously makes noise. Now a person with all 5 senses would say all of it's attributes as it is described before. A blind person may see the same box just as 'a cubical box that makes noise'. A deaf person would realize it as 'a red cubical box'. A person who is blind and also deaf would realize it simply as 'a cubic box' and if that person also doesn't have sense of feel then he realizes nothing! So, according to me, we can see things with their other (non-visible) attributes if we have other senses that could recognize them! I would conclude that objects may be possessing other attributes but we simply don't see them because we have only 5 (limited) senses.

5. Neural Networks - An introduction

Neural networks are the basic communicating and processing channels for brain. Just like any other pictorial representation, neural network is also represented by dots, connection & junctions but in real, a neural network is the network of individual cells!

When we realize that a neuron is just a cell, we would also realize how tiny they are and their huge population in a human brain. A human brain weighs, approximately, 1500g! Just like any other cells, neuron's size and shape vary depends on it's location in brain. It size varies from 4 micron which has nucleus of size 3 microns in diameter up to 100 micron in diameter that has bigger nucleus. Just imagine how many neurons could possible be there in a 1500g weighing brain! Now, lets explore them more.

Neuron cells:
Basically our brain and the spinal cord are made up of many neurons. Neurons are cells that send and receive electrochemical signals to and from the brain and nervous system. There are about 100 billion neurons in the brain. Though their size says that they are tiny, those cells are not necessarily short. Their length varies from a fraction of an inch to several feet. Neurons transmit the nerve signals to and from the brain at up to 200 mph.

The neuron consists of a cell body (or Soma) with branching dendrites (signal receivers) and a projection called an axon, which conduct the nerve signal. At the other end of the axon, the axon terminals transmit the electrochemical signal across a synapse (the gap between the axon terminal and receiving cell). Below picture illustrates the structure of a neuron.

Axon:
The axon, a long extension of a nerve cell, and take information away from the cell body. Bundles of axons are known as nerves or, within the CNS (central nervous system), as nerve tracts or pathways. Dendrites bring information to the cell body.

Myelin sheaths:
Myelin coats and insulates the axon (except for periodic breaks called nodes of Ranvier), increasing transmission speed along the axon. Myelin is manufactured by Schwann's cells, and consists of 70-80% lipids (fat) and 20-30% protein.

Soma or cytoplasm of a neuron:
The body of a neuron is called as Soma. It contains the neuron's nucleus (with DNA and typical nuclear organelles). Dendrites branch from the cell body and receive messages. A typical neuron has about 1,000 to 10,000 synapses (that is, it communicates with 1,000-10,000 other neurons, muscle cells, glands, etc.).

Types of neurons:
There are different types of neurons. They all carry electro-chemical nerve signals, but differ in structure (the number of processes, or axons, emanating from the cell body) and are found in different parts of the body. Unlike most other cells, neurons cannot regrow after damage (except neurons from the hippocampus). There are three types of neurons.
    * Sensory neurons or Bipolar neurons carry messages from the body's sense receptors (eyes, ears, etc.) to the CNS. These neurons have two processes. Sensory neuron account for 0.9% of all neurons. (Examples are retinal cells, olfactory epithelium cells.)
    * Motoneurons or Multipolar neurons carry signals from the CNS to the muscles and glands. These neurons have many processes originating from the cell body. Motoneurons account for 9% of all neurons. (Examples are spinal motor neurons, pyramidal neurons, Purkinje cells.)
    * Interneurons or Pseudopolare (Spelling) cells form all the neural wiring within the CNS. These have two axons (instead of an axon and a dendrite). One axon communicates with the spinal cord; one with either the skin or muscle. These neurons have two processes. (Examples are dorsal root ganglia cells.)

Glial cells:
If such active neurons are too busy at their work, who is feeding and managing them? Like brain that manages and coordinates other organs, there must be somebody else to take care of neurons! In fact, along with neurons there are Glial cells. There are many more Glial cells; they provide support functions for the neurons, and are far more numerous than neurons in count.

Glial cells make up 90 percent of the brain's cells. Glial cells are nerve cells that don't carry nerve impulses. The various glial (meaning "glue") cells perform many important functions, including: digestion of parts of dead neurons, manufacturing myelin for neurons, providing physical and nutritional support for neurons, and more. Types of glial cells include Schwann's Cells, Satellite Cells, Microglia, Oligodendroglia, and Astroglia. There is another type of brain cell called 'Neuroglia' (meaning "nerve glue"). These cells guide neurons during fetal development.

I hope that this gives a strong idea of neurons and their network. In my next posts, lets explore how these neurons process signals and patterns to interact with the external world!

4. Brain's structure and activities

Whenever we talk about 'Brain', we mean the human brain. That doesn't mean that only human brains are complicated because even animal brains, even an ant's brain is so complicated and works just like ours. I mean in terms of electrical impulses. The average ant brain has about 250,000 neurons.Each ant neuron has thousands of dendrites which are the electrical connections that “fire” info packets to adjoining nerve cells. Neurons can fire several million times per second. So a single ant brain has a minimum capability of 1.2 trillion calculations per second. However, what makes human brain the most complicated and unique is that it is the most developed brain compared to any other animals in the world. That's why no other species could dominate or utilize the nature as much as humans do.

Brain's structure:
Though brain is just like another organ in the body, it is quite different than organs. The difference is in various terms. Brain is the only organ that controls other organs. Subsequently, that derives another difference. Unlike heart, lungs, liver, etc each of which is responsible for one activity, brain takes the responsibility of various actions like hearing, vision, touch, smell, taste, etc. So I would say that it's a collection of organs that takes the overall control of the whole nervous system that connects our whole body. If you look at the brain from outside, the different portions of brain can be seen. Please refer the below picture that indicates the portions and responsibilities.

Brain's cerebrum is actually split into two pieces; each sits side by side and their shape is like hemisphere. Hence they are termed as left hemisphere and right hemisphere. Mysteriously, the fact is left hemisphere is responsible for the whole right side of our body such as right arm, right leg, etc... and the right hemisphere is responsible for the left side of the body. Similar way, our optical nerves from eyes cross over each other and connects with the opposite portions of the occipital lobe.

Apart from these each hemisphere remains sole responsible for other activities. For example, the left brain focused on detail. This would make it the natural home for all those mental skills that need us to act in a series of discrete steps or fix on a particular fragment of what we perceive--skills such as recognising a friend's face in a crowd or "lining up" words to make a sentence. By contrast, the right brain concentrated on the broad, background picture. The researchers believed it had a panoramic focus that made it good at seeing general connections; this hemisphere was best able to represent the relative position of objects in space and to handle the emotional and metaphorical aspects of speech.

Researches say that from childhood to Adulthood, the brain under goes vast number of changes. It grows in terms of size as well as knowledge. If you look at the brains of a child and an adult side by side, you can clearly see two important differences. Please refer the below pictures.
1. A child's cerebrum has more 'grey matter' than the adult's developed brain. It has been found that the cerebrum gains as more neurons are trained and used and more knowledge is stored.
2. Since the neurons are more trained in adults, their brain shows more activity than a child's brain.

If you look inside the brain, kind of zoom in, what you see is nothing but the network of millions of neurons. As you in the below picture, there are branches that connect all the different portions of the brains such as cortex and lobes. There is also one more important portion of brain that links the whole nervous system to the endocrine system via the pituitary gland. It's called as hypothalamus and it contains a number of small nuclei with a variety of functions. The hypothalamus controls body temperature, hunger, thirst, fatigue, and circadian cycles.

Brain's activities:
We can classify the brain's activities into four basic actions such as receiving signals from organs, sending signals back to organs, analyze the situation and finally storing into memory. Now lets explore them in detail.

1. Receives signals:
Brain receives signals from the sensory organs like hands, legs, ears, eyes, nose and mouth. That's how we realize what we do, what we hear, what we see, what we smell and what we eat. Every time, you act, your organ sends signals or messages to the brain through nervous system. Specific portion which is responsible for that particular task takes up the signals for further processing.

2. Sends signals:
Once brain receives the signals from the sensory organs, it starts to process them through it's axons and neurons and sends specific commands to the organs in return. Hence we take our hands off soon after we touch the fire. These signals are basically passes through blood. Heart also participates in the brain's activity by pumping enough blood for brain to do it's job with no problems. That's why people who are tired stop moving because brain doesn't send signals to the organs to act or react. Similarly dead person doesn't move or even blink as brain stops receiving signals through blood as heart stops pumping it to the brain.

3. Analyze signals:
Brain is capable of doing various kinds of analysis such as Logical reasoning, Pattern recognition, memory, mathematical calculations, etc. However, brain does not properly analyze every signal before it sends the commands. It depends on how quick the reaction is required or command needs to be executed. Let me explain this here. For example, lets say you are shown a picture of a smiling baby and your brain tells you that it's a smiling baby and probably put a smile on your face! So, here your brain receives the signals, process them and returns the commands.

Signals come via blood and pass through the brain's neural network and fire electrical impulses at various intervals synchronously, simultaneously, periodically etc... and that creates a specific pattern. When brain detects such a pattern of action, it goes back to its memory and tries to find matching pattern of action from its previous experiences. When it finds, it goes further to find what was the desired behavior or pattern of reaction for such a pattern of action. Then it finds the pattern of reaction and returns them in terms of signals back to the respective organs through blood. When it doesn't find a desired pattern of reaction, that's when we stand still by not knowing what to do or how to react to such actions or situations. Sometimes this analysis or pattern matching process doesn't happen on time but a little later or after sending emergency signals.

Lets say that you are watching a 3D movie and suddenly there is a knife thrown into your eyes via 3D projection. You would immediately try to move away from it as it's gonna hit you for real! Why is that? Even though you know it's just a movie, your brain still thinks it would hurt you and that's why it sends signals to your body to move away from it. At this point of time, your brain doesn't do any analysis on what is going on rather it goes under defensive mode. However, after the reaction you come back to normal as your brain learns that it's just a movie 'once again'. Brain always prefer to come up with faster and average results than the perfect answer later.

4. Stores in memory:
When brain observes a situation, it records the same in to its memory but depends on the intensity of the observation on each item or detail of the situation, it records that specific detail in the active portions of the neural network. Other less-interesting details are stored but eventually lost over time as if that particular pattern is not reproduced frequently or refreshed over time. Hence, the difference comes between memorizing and just watching. Irrespective of different hemispheres acts on observing different attributes of a situation, underneath the information is stored in the form of patterns. Hence each and every bit of information is shared across various number of neurons in the form of patterns of impulses.

In my next posts, we will explore how information is processed by the neural network inside our brain.

3. Consciousness, Subconsciousness, Perspectives & Perceptions

Consciousness:
Before getting into details of how senses and brain works together, let's understand what perception and consciousness really mean. Sometimes, people get confused between these two. I would say perception is the outcome of the consciousness. If you are not conscious at all, then you don't have perception. So, does it mean that person doesn't have any perception if he is unconscious?

If you ask me, I would first define what the term 'unconscious' means. In order to do that, first we need to know what the conscious being is. When can we say that a being is conscious? 'Consciousness' means 'Self Awareness'; and so by any mean, if that being realizes its existence, then we can say that the being is conscious. So, when a being looses its consciousness, in other words, by no mean it realizes its existence, then we can say that it's unconscious!

However, there are two things about 'determining consciousness'. In older days, we all consider a person is unconscious when s/he faints. With the help of science, we have learned that even a fainted person is conscious or aware of things happening around him! Similar the case with patients who are under Coma stage. So, in order to be conscious, one has to interact with the environment by one or more means. The challenging question is that what if one doesn't interact using his fundamental senses but still aware of the existence? Is it possible?! Think through it... we will come to these questions later.

Subconsciousness:
For now, lets explore in what mental state are we while we sleep? We are not aware of what's exactly going on around us but still we are aware of our existence! Such consciousness is termed as 'Sub-consciousness'. Brain doesn't have allocated portions for conscious and subconscious mind. What brain does is that it shuts down few of the unused portions of the brain and limit its activities. One of the portion that gets shut down is the one that is responsible for logical reasoning.

When we are asleep, our conscious side of the brain still works but without the logical reasoning abilities. Hence dreams are, most of the time, illogical like magical world, height of our own imagination. That's why we feel like flying and floating, if we watch a horror movie at night then there are more chances for the ghost to come in your dreams and make you sweat before you happen to have rational and logical thoughts and analysis. You need to be awake to be logical and rational. However, sometimes you get solutions for your problems while asleep. History could tell us that there were more inventions occurred while body is at rest or sleep.

Perspectives vs. Perceptions:
Nature is the one whose beauty is admired in terms of it's complexity and mystery. Everyone, including you and me, has our own arguments on any topic with respect to any context! Why are there arguments? because people think differently. Why do people think differently? because people see things differently. Why do people see things differently? because everyone has their own perspectives and that conflicts with others! If you apply theory of relativity, it would make more sense.

Any object, which is observed, is subject to be relative to the observer based on his frame of reference.

Yes! What you see might not be seen by others who are in different side or view of the same object. Here is where perception comes. Though 'perspective' and 'perception' seems to mean the same, still they are different! Perception is the result of perspective. If two people perceives at one perspective of an object, then they would have the same perception of an object. So, in order to get the complete perception of an object, it is necessary to view the same object from all the perspectives.

2. Fundamental Senses

I would like to list the bodily senses based on their priority or how important for a living being to exist in this world. To start with, I ordered the senses based on its importance for a being to perceive the world from the worst to best possible ways and here it goes,
1. Sense of Touch
2. Sense of Smell
3. Sense of Taste
4. Sense of Sound
5. Sense of Vision

Sense of Touch:
Yes, sense of touch was the first one that was developed during evolution of species. We will leave debate about the accuracy of the theory of evolution for now! So, this is the first sense that indicates oneself about its existence. Think of 'Amoeba' which is the simplest of the protozoa. It can be found in ponds and rivers and on the surface of the leaves of water plants. It's made up of only one cell and has only sense of touch, using which it moves, finds his food, etc. Temporary pseudopods are formed for its feeding and locomotion! This also helps being to be aware of danger as well as its food.

Sense of Smell:
Though it is bit confusing to understand or decide which sense was developed first, among sense of smell and taste, my analysis tells me that sense of smell takes priority over sense of taste. Similar to touch, smell also helps the living beings to find its food and also to be aware of any danger. Of course, not all of us perceive smell in a same way. It is subjective. Even though it's subjective, animals fall under same species tend to have similar (almost) smelling abilities and perceive smells in a same way. The reason why I kept sense of smell before sense of taste is because one is informed about it's prey or any danger it only with the sense of smell even before it comes to contact with food or danger.

Sense of Taste:
What comes next is the sense of taste. Just like sense of smell, this also needs highly developed of organ. When we say 'highly developed', I don't really mean the size or shape of the organ because even a tiny ant has a great sense of taste. Researches say that ants have a well-developed sense of taste, and can distinguish sour, sweet, bitter, and salty tastes. Sense of taste is just another level of classifier to distinguish whether the food is suitable or dangerous, etc. If you see, by nature things which are not suitable for us smell and taste bad for us! Some say taste is the last sense to be developed. It may be true in case of human but if you consider other animals, they don't develop their tastes like us. Only we humans eat and wants to try different food of various tastes!

Sense of Sounds:

Sound is the another flavor of this universe. In order to perceive this we need organs that perceive sounds. Apart from just perceiving, most of the animals use this for communication. Few, such as bats, use sound for their movements. They make sound waves and based on how soon it reflects, it understands that whether its path is clear or not. For us it sounds bit difficult, but for bats its not that difficult as they grow up with this ability and they develop their senses as they grow up.

Researches tell that even tiny ants communicate with each other ants using one of the following techniques,
1. Sense of touch - by touching each other with their antennae.
2. Sense of smell - by releasing chemical on their path
3. Sense of sound - by making sound using jaw slapping, jumping, rubbing body parts, etc.
Some species of ants ingeniously combine all three communication techniques—touch, chemicals (smell), and sound to convey their messages to cohorts.

5. Sense of Vision
Finally, it's Vision! This is the most important one among all other senses because even though one can feel, smell, taste, hear, if one cannot see this world then we can say, in no time without a second thought, it is missing the most beautiful perspective of this world. Even the word 'beauty' is not justified in the absense of vision. Few animals are blind and few of them have under developed vision. Even animals like cows, goats have fully developed eyes but due to the position of their eyes they don't have unified sights like us. In simple words, they have more blind spots than humans.

If you look at the overall picture, it's so clear that all the animals pretty much comfortably do their jobs with their limited senses and have no complaints. That's because nature has given the necessary alternative senses to fulfill their needs, required for its existence. For example, mammals in the oceans use their sense of sound not only to communicate but also to find their food in the dark deep bottom of the ocean where there is minimum or no light. In my next posts, we will see how each of these senses helps one to perceive this world in terms of 'perceptions and consciousness'.

1. Welcome to Perceptions and Consciousness

Brain - It is the most mysterious, the most complex system and also the most precious part in our body!

Why is that? Why brain takes precedence over even Heart or Lungs? Simple! Even with the latest technology, if any part of the body fails, we have a substitute but not for Brain, at least not yet! Not just because of no substitute, also Brain is the boss, I mean he is the master who controls the rest!

Though most of our internal organs like lungs, liver, etc just work by themselves for which brain doesn't have big contribution. In other words, for all the activities related to the external world needs Brain's management. We need our brain's green signal for our each and every activity. Most of the times, it happens spontaneously even before we realize. Activities like breathing, eating, drinking, etc we don't realize that our parts like hands, legs, mouth, eyes, etc acting only upon successful reception of brain's green signal.

What I mean by green signal is that the signals that inform legs to stop walking or start walking. Please don't let the term 'green signal' confuse you with the traffic signals because technically there are no red or yellow kind of signal which would exhibit similar effects.

Among all the organs, internal organs are even more important than external organs for a being to live! Still, we need to define the term 'Living'. What does living mean? It means a being is aware of it's existence and it interacts with it's environment and able to fulfill it's needs. This is how I define 'Living'. If we go by this definition, what all do we need for such Living?

1. To be aware of our existence - we need sense of 'touch' or 'feel'.
2. To interact with the environment and fulfill the needs- we need sense of vision, sense of smell, sense of hearing for an effective communication and also need the control over body to fulfill the needs.

These are called 5 senses. Apart from these, we eventually develop another sense out of our intelligence, from our experience and that's called sixth sense or common sense. This sense is unique compared to other senses because of three reasons. 1. This sense is derived from all other 5 senses, 2. This sense is subjective based on the knowledge one possesses, and the most important 3. Not like other 5 senses, this sense has ability to impact the brain and impacts other senses too.

We can view this as a cycle of evolution of senses. 5 senses result in 6th sense and 6th sense impacts the basic senses back with the help of brain. The driving force for the development of 6th sense relies on how the individual perceive the external world and realize his existence. Let's see the details in next post!