1. Applying Category Theory to Improve the Performance of a Neural Architecture Michael J. Healy Richard D. Olinger Robert J. Young Thomas P. Caudell University of New Mexico Kurt W. Larson Sandia National Laboratories This work was supported in part by Sandia National Laboratories, Albuquerque, New Mexico, under contract no. 238984. Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy's National Nuclear Security Administration under Contract DE-AC04-94AL85000.

2. P1P1 P2P2 P3P3 m T T’ Semantic Representation Functor M Concept category Neural category Neural network M (m) M (T) M (T’)

3. Mod(m) P1P1 P2P2 P3P3 m T T’ Mod(T’) Mod(T) M(m) M(T) M(T’) Model-Space Morphisms  Reciprocal Connections Functor M Functor Mod Instances of Instances of

4. Colimits Express Specialization - Limits Express Abstraction TT TT TT TT TT Least specialization TT TT TT TT Maximally specific abstraction

5. Classifying Pixels by Spectral Similarity Multispectral camera data...... Intensities for 10 optical bands Data for pixel i ( = one input pattern) Neural network classifier Pixel class (color) Colored pixel i Multispectral image …

6. Stack Interval Network   StimVal           StimLB  StimUB  Positive stack nodesComplement stack nodes   

7. Stack Interval Patterns Represent Real Intervals 0  v   Width 1 unit Positive stackComplement 0  v   Width 2 units Intersection of stack patterns (in template patterns) 1  v   Width 1 unit Positive stackComplement

8. ART-1 with Stack Interval Inputs FF FF FF... GC  Band  Band     V bb bcbc bb bcbc Template pattern Composite input pattern (two stack Intervals) 

9. ART-1 + F 1 Colimits, Limits FF FF FF      V   FF... S L F  

10. Panchromatic Image - 1 m Resolution

11. Multispectral Image - Generic ART-1  = 0.795 Template density ordering

12. Multispectral Image - ART-1 with Limits  = 0.55 F -1 tol = 0.55 Template density ordering

14. References M. J. Healy, R. D. Olinger, R. J. Young, T. P. Caudell, and K. W. Larson, “Applying Category Theory to Improve the Performance of a Neural Architecture” (under review). M. J. Healy and T. P. Caudell (2006) “Ontologies and Worlds in Category Theory: Implications for Neural Systems”, Axiomathes, 16 (1), pp. 165-214. M. J. Healy and T, P. Caudell (2004) “Neural Networks, Knowledge, and Cognition: A Mathematical Semantic Model Based upon Category Theory”, UNM Technical Report EECE-TR-04-020, University of New Mexico, Albuquerque, NM, USA.

15. Template Patterns Band 1 Band 2 Template 1 Template 2

16. Stack Numeral Quanta v   3  v   0  v    v   0  v   2  v   Width 0 units Width 1 unit Width 2 units..................

17. Neural Network Research Objective: Associate an Evolving Knowledge Structure with Neural Structure and Activity Concept hierarchy Environment Neural network Sensors and actuators Learning and representation Modality-specific input streams Event stream Motor functions

18. Limits Express Abstraction TT TT TT TT … maximally specific abstraction

19. Colimits Express Specialization TT TT TT TT TT … least specialization