Neurons

• Neurons are biological cells; there are lots of different types, and each is unique.
• None the less there are lots of ways of modelling them.
• The simplest (comes from Lapique in 1907) is the integrate and fire model.
• A neuron collects activation (energy or potential) from other neurons. If it gets enough activation, it fires. When it fires it sends activation to other neurons.
• This is the neuron used in a Hopfield net, and it's close to a neuron in a multi-layer perceptron (rate encoding).
• It's commonly extended to make a leaky integrate and fire neuron.
• What happens to the activation when an IF neuron doesn't fire. The model either keeps it all (Lapique), or throws it all away (McCulloch -Pitts). More accurately, some leaks away. This means that it's easier to fire in the next cycle, but never fires with very low input.
• I've extended it to include fatigue. This means that if a neuron fires frequently, it becomes harder to fire. Similarly, if it hasn't fired in a long time (hypo-fatigue), it fires.
• Boltzmann machine.
• Continuous value output neurons instead of spikes.
• All of these are point models. Compartmental models are much more accurate, but more expensive to simulate.
• The original compartmental model is the Hodgkin Huxley neuron. There are others, but this is still popular.