1. Teaching Recurrent NN to be Dynamical Systems

2. Homogeneous Linear Systems
(no inputs)
Example:

3. Discrete time recurrent networks
External
Input
Weights V
Inputs u
Outputs
Recurrent
Weights W f y1 y2 yn

4. Non-Linear sin wave generator example
Teacher y1 y2
w12
w11
w22 b2 b1
w21
Limit cycle attractor

5. t(n)
u(n)
y1(n)
y2(n)
t(n - k)
y1(n - k)
y2(n - k)
u(n - k)
t(2)
y1(2)
u(2)
y2(2)
t(1)
y1(1)
u(1)
y2(1)
t(0)
y1(0)
y2(0)

6. Parenthesis balancing Turing Machine
A before B A B
OUT
input
input
Parenthesis balancing Turing Machine
…( ( ) ( ) ) )…
FSM
Tape

7. Recurrent NN model of active memory Load-in Info-in Info-in Load-in1 2 n …
Load-in
Info-in
Out

8. Testing Spiking Info-in load-in Hidden-1 Hidden-2 Hidden-3 Hidden-4
Out

9. { { Color match to sample Recorded in IT Red Neuron 1 Green Red19 1
Sample
Delay
Match