1. Computational Sensory Motor Systems Lab Johns Hopkins University Computation Sensory Motor-Systems Lab - Prof. Ralph Etienne-Cummings Modeling life in silicon

2. Computational Sensory Motor Systems Lab Johns Hopkins University The Big Picture: Lab Motivation Restoring function after limb amputation Restoring locomotion after severe spinal cord injury Developing Biomorphic Robotics Adaptive Biomorphic Circuits & Systems

3. Computational Sensory Motor Systems Lab Johns Hopkins University Computation Sensory Motor-Systems Lab Ralph Etienne-Cummings’ Lab Towards a Spinal Neural Prosthesis Device Decoding Individual Finger Movements Using Surface EMG Electrodes Normal Optical Flow Imager Integrate-and-Fire Array Transceiver Optimization of Neural Networks Design of Ultrasonic Imaging Arrays for Detection of Macular Degeneration Precision Control Microsystems

4. Computational Sensory Motor Systems Lab Johns Hopkins University Towards a Spinal Neural Prosthesis Device Jacob Vogelstein Francesco Tenore

5. Computational Sensory Motor Systems Lab Johns Hopkins University Our Approach Previous approaches ignore CPG and focus on controlling muscles to generate locomotion We propose to directly control the CPG and use it to generate locomotion Basic idea is to recreate natural neural control loop in an external artificial device (i.e. replace tonic and phasic descending inputs to the CPG with electrical stimulation) SLP RS Muscles Source: Grillner, Nat Rev Neurosci, 2003

6. Computational Sensory Motor Systems Lab Johns Hopkins University The Big Picture: Lab Motivation Restoring function after limb amputation Restoring locomotion after severe spinal cord injury Developing Biomorphic Robotics Adaptive Biomorphic Circuits & Systems

7. Computational Sensory Motor Systems Lab Johns Hopkins University Responsibilities of Locomotion Controller 1. Select Gait + specify desired motor output - phase relationships - joint angles 2. Activate CPG + tonic stimulation initiates locomotion - epidural spinal cord stimulation (ESCS) - intraspinal microstimulation (ISMS) 3. Generate “Efferent Copy” + monitor sensorimotor state - external sensors on limbs - internal afferent recordings 4. Control Output of CPG + phasic stimulation (efferent copy required for precisely-timed stimuli) - convert baseline CPG activity into functional motor output - correct deviations - adjust individual components - adapt output to environment Select gait ~ brain Activate CPG ~ brainstem (MLR) Efferent copy ~ efferent copy Enforce/adapt output ~ phasic RS

8. Computational Sensory Motor Systems Lab Johns Hopkins University Gait Control System 12 pairs of IM electrodes: 3 each for left/right hip, knee, and ankle extensors/flexors Two types of sensory data were collected for each leg Hip angle (HA) Ground reaction force (GRF) Source: Vogelstein et al., IEEE TBioCAS, (submitted) Spike processing back-end Analog signal processing front-end

9. Computational Sensory Motor Systems Lab Johns Hopkins University Results: SiCPG Chip Controls Locomotion in a Paralyzed Cat Source: Vogelstein et al., IEEE TBioCAS (submitted)

10. Computational Sensory Motor Systems Lab Johns Hopkins University Decoding Individual Finger Movements Using Surface EMG Electrodes Francesco Tenore

11. Computational Sensory Motor Systems Lab Johns Hopkins University Problem Fast pace of development of upper- limb prostheses requires a paradigm shift in EMG-based controls Traditional control schemes typically provide 2 degrees of freedom (DoF): Insufficient for dexterous control of individual fingers Surface ElectroMyoGraphy (s-EMG) electrodes placed on the forearm and upper arm of an able bodied subject and a transradial amputee

12. Computational Sensory Motor Systems Lab Johns Hopkins University Implemented Solution Neural network based approach Number of electrodes (inputs)  amputation level (I-V)  Level I: 32 electrodes, Level V: 12 electrodes

13. Computational Sensory Motor Systems Lab Johns Hopkins University Results 1. High decoding accuracy: Trained able-bodied subject, ~99% Untrained transradial amputee, ~ 90% 2. No s.s. difference in decoding accuracy between able-bodied subjects and transradial amputee 3. No s.s. difference in decoding accuracy between networks that used different number of electrodes (12-32)

14. Computational Sensory Motor Systems Lab Johns Hopkins University Current/Future Work Towards real-time control: training on rest states and movements  Implementation on Virtual Integration Environment (VIE) Independent Component Analysis (ICA) to minimize number of electrodes by choosing the ones that most contribute to the accuracy results

15. Computational Sensory Motor Systems Lab Johns Hopkins University Normal Optical Flow Imager Andre Harrison

16. Computational Sensory Motor Systems Lab Johns Hopkins University Normal Optical Flow Imager Computer Vision Neuromorphic

17. Computational Sensory Motor Systems Lab Johns Hopkins University Normal Optical Flow Imager Imager that computes 2-D dense Normal Optical Flow estimates using spatio-temporal image gradients, without interfering with the imaging process Optical Flow is the apparent motion of the image intensity

18. Computational Sensory Motor Systems Lab Johns Hopkins University Normal Optical Flow Imager

19. Computational Sensory Motor Systems Lab Johns Hopkins University Integrate-and-Fire Array Transceiver Fopefolu Folowosele

20. Computational Sensory Motor Systems Lab Johns Hopkins University Motivation The brain is capable of processing sensory information in real time, to analyze its surroundings and prescribe appropriate action Software models run slower than real time and are unable to interact with the environment Silicon designs take a few months to be fabricated, after which they are constrained by limited flexibility

21. Computational Sensory Motor Systems Lab Johns Hopkins University IFAT The IFAT combines the speed of dedicated hardware with the programmability of software for studying real-time operations of cortical, large-scale neural networks

22. Computational Sensory Motor Systems Lab Johns Hopkins University Application: Visual Processing

23. Computational Sensory Motor Systems Lab Johns Hopkins University Optimization of Neural Networks Alex Russel and Garrick Orchard

24. Computational Sensory Motor Systems Lab Johns Hopkins University Pre Evolution Architecture

25. Computational Sensory Motor Systems Lab Johns Hopkins University Evolved Hip Controller

26. Computational Sensory Motor Systems Lab Johns Hopkins University Evolved Knee Controller

27. Computational Sensory Motor Systems Lab Johns Hopkins University The Final Product

28. Computational Sensory Motor Systems Lab Johns Hopkins University Design of Ultrasonic Imaging Arrays the Detection of Macular Degeneration Clyde Clarke

29. Computational Sensory Motor Systems Lab Johns Hopkins University Design of Ultrasonic Imaging Arrays the Detection of Macular Degeneration www.seewithlasik.com/.../CO0077.jpg

30. Computational Sensory Motor Systems Lab Johns Hopkins University Tool-tip Mounted Ultrasonic Micro-Array C. Numerical Modeling 1)Finite Element Method 2)Finite Difference Method B. Derive Equations for Wave Propagation in Vitreous and Retina 1)Scattering 2)Absorption L L W W A.Create Models of Transducer array operating in Homogeneous Media [Yakub,IEEE Trans 02] D.Modify Design Parameters of Array to perform optimally in Surgical Environment

31. Computational Sensory Motor Systems Lab Johns Hopkins University Adaptive and Reconfigurable Microsystems for High Precision Control Ndubuisi Ekewe

32. Computational Sensory Motor Systems Lab Johns Hopkins University Adaptive and Reconfigurable Microsystems for High Precision Control Simaan, 2004 Ekekwe et al, US Patent (Pending)