Infinite Tape

• You also want to have memory in systems. It's harder than in VonNeumann machines.
• There are lots of ways to do it, but with bits, you can just have some neurons that can be in one of two states. They don't need to persistently fire, but can be activated when necessary. You want to avoid firing as its expensive both computationally and biologically.
• Now, you need to set the state. This is done by learning, and that's Hebbian learning.
• So, turn on the appropriate bit to be learned, and turn on the bit of memory. This activates the appropriate bit in the memory, and can erase the other bit.
• When you want to change the setting, you do that again.
• You can have a tape of these memory items.
• You need an infinite number of neurons for an infinite tape, but the number of neurons only grows linearly with the number of memory items. You might need n(log(n)) synapses.