Modeling Neural Networks Christopher Krycho Advisor: Dr. Eric Abraham May 14, 2009

Computational Neuroscience Young field (decades old) Modeling the brain with physical principles –Oscillatory behaviors –Large-scale networks Technique: –Create a model of neurons –Model neuron interactions (varying degrees complexity) –Compare results to EEG/MRI/etc. An EEG recording of various locations in a brain

Project Goals Build a computer program that builds and analyzes small-world networks Apply that model to neural network models created by the Zochowski group at Michigan –Reproduce neuron and network models –Run a simulation with that model and compare results Test different models and compare results –Doing new science by examining variations on their model

Outline What are neurons and neural networks? Small world networks and their relevance Modeling a set of neurons in the brain with small world networks Graph, analyze total synaptic current in system over time.

Neurons Primary components of nervous system Transmission and reception of electrical signals –“synaptic” current: current along synapses - connections between neurons Representation of a neuron Source: Wikipedia

The human nervous system Source: The Nervous System Composed of neurons throughout –Brain –Nerves in fingers –Spinal cord neurons in the brain alone –Each with connections to other neurons (and itself)

Small World Networks (SWNs) Small average “path length” –Number of steps from random node to other random node Large number of nodes –# of nodes in network Low connectivity –# of connections per node High clusteredness –Nodes with mutual connections also connect to each other A network with small world characteristics Source: Exploring Complex Networks, Steven H. Strogatz, Nature (8 March 2001)

Network vs. SWN “Small world” different from random networks Despite… –low relative connectivity –Connections primarily short-range –Connections very clustered … short average path length between any random points in network SWN architecture observed in brain –On a small scale in local neural networks –Possibly on a larger scale among neural networks

Making SWNs Initialize an array representing neurons’ connections to each other –Connections initially symmetric within networks –Two or more networks connected internally, but not to each other –Note: actual networks also connected to next- nearest neighbors Two networks of 9 nodes each: A B

From network to SWN, pt. 1 Randomly rewire the internal connections of each network –Use a random number generator –Connections can be symmetric or directional Here, symmetric In model, directional The networks after internal rewiring: A B

From network to SWN, pt. 2 Randomly add connectivity between networks –Choose some proportion of the neurons in any given network to receive connections –Choose the number in any given network sending connections –Connect - showing directional connections here The networks after adding connectivity: Key: From A to B From B to A A B

Representing Connections Matrices filled with 0s and 1s –Rows send, columns receive

A small network rewired From left to right: –The initial setup of the network - symmetric –The networks, with random rewiring of 0.3 of connections –25% of neurons on each network receiving input from 4 different neurons on the other network

The networks before any rewiring: Representative Connection Graphs 15x15 grids = 225 neurons/network 2 networks 450x450 matrix: –Rows send –Columns receive Connections to nearest and next- nearest neighbors

Rep. Conn. Graphs cont’d. The networks after internal rewiring:

Rep. Conn. Graphs cont’d. The networks after adding internetwork connections:

A Neuron Model Equation representing the state of each neuron at a given time –Numerical value representing charge (V) on the neuron and thus closeness to “spiking” (sending a signal) Equation involves 4 parameters: –  i - Leakage current on i th neuron; constant over time but differs for each neuron –  J i,j (t) - Incoming current to the i th neuron from the j th neurons (connected, in the same network) –  J i,k (t) - Incoming current to the i th neuron from the k th neurons (connected, in another network) –  (t) - White noise

Generating Current cont’d. Equation (identical form for intra/internetwork): J i,j (t): current to neuron i from neuron j at time t – J i,k (t) has the same form, but different lag  s and  f ensure correct shape of pulse Current versus time; threshold reached at t=0

Solving the Equation Integrate over 10 sec with Euler numerical method –Maximum sec step size for good resolution (some runs as low as sec) Large computation: –450 neurons –5 calculations/neuron/iteration –~ 225 billion calculations at highest resolution ~ minutes of real time for high resolution –Coded in Fortran –Running on 2007 MacBook Pro

Network Progress

Network Finished

Simulation Progress Replicating Zochowski model: in progress –Reproducing underlying phenomena –Missing the tell-tale sign of perfectly reproducing their model: “bursting” The Zochowski group’s model graph of network behavior:

Early Simulation Run Time (s) Total synaptic current (arb. Units) Network 2 Network 1

Present Simulation Run The Zochowski group’s model showing bursting: Our model showing basic behaviors but no bursting

Result Details The Zochowski model individual neurons Our model individual neurons

The Future (of the Project) Finish replicating Zochowski group’s results Open doors for the future Possibility of expanding the number and/or scale of the networks My design includes –Ability to implement and then test learning mechanisms –Ability to increase complexity of neuron model –Ability to increase complexity of network model

Acknowledgments The Zochowski group at The University of Michigan, particularly Jane Wang and Sarah Feldt, for their work on which this project is based, and for answering many questions along the way Dr. Keiran Mullen for teaching me enough Fortran to start the project Dr. Eric Abraham, my advisor