1. An Analysis of Hamptonese Using Hidden Markov Models Ethan Le and Mark Stamp Department of Computer Science San Jose State University McNair Scholars Program Summer Research Project, 2003

5. Agenda James Hampton Purpose What is Hamptonese? Hidden Markov Models (HMMs) HMMs and Hamptonese Other approach Results Questions

6. James Hampton WWII veteran Janitor in Washington D.C. Life of solitude Died in 1964 Hamptonese discovered after his death “The Throne” also found

7. “The Throne of the Third Heaven of the Nations’ Millennium General Assembly” James Hampton alongside his Throne Pieces of The Throne

8. Purpose Analyzed Hamptonese using Hidden Markov Model to determine whether Hamptonese was created using simple substitution of English letters (or other languages). Consider alternative interpretations of Hamptonese

9. What is Hamptonese?

10. What is Hamptonese? (Cont’d) Unknown origin and content Spans 167 pages Could it be:  A cipher system?  Gibberish?  Example of human randomness?  Other?

11. Hidden Markov Models (HMMs) Markov process with hidden states HMMs provide probabilistic information about the underlying state of a model, given a set of observations of the system.

12. HMMs (Cont’d) Three solvable problems 1. Find probability of observed sequence 2. Find “optimal” state sequence 3. Train the model to fit observations

13. HMMs examples Example 1: Music Information Retrieval System (MIR) Example 2: Deciphering English text (Cave and Neuwirth)  27 symbols (alphabet letters plus space)  Assume 2 hidden states  Train HMM to best fit input data  Results: separation of consonants and vowels

14. HMM Example (Cont’d) – English Text CharacterInitialFinal A0.03685 0.037930.0044447 0.1306242 B0.04007 0.039780.0241154 0.0000000 C0.03362 0.034230.0522168 0.0000000 D0.03777 0.036540.0714247 0.0003260 E0.03409 0.036080.0000000 0.2105809 F0.03685 0.039320.0374685 0.0000000 G0.03593 0.038390.0296958 0.0000000 H0.03961 0.039320.0670510 0.0085455 I0.04007 0.033770.0000000 0.1216511 J0.03501 0.035150.0065769 0.0000000 K0.03685 0.037000.0067762 0.0000000

15. HMMs and Hamptonese Transcribed 103 pages About 30,000 observations More than 40 distinct symbols Assume 2 hidden states 3 different reading techniques

16. Other Approach Hamptonese vs. 246 different languages  HMM for other languages Religious references & organizational patterns  Analysis of first 40 pages  Pattern recognition

17. Results Hamptonese is not a simple substitution for English letters Hamptonese is probably not a simple substitution for any other language First comprehensive study of Hamptonese  Transcription of entire text  Future research suggestions

18. Acknowledgments… Dr. Mark Stamp SJSU McNair Scholars Program SCCUR UC Irvine

19. Questions???

20. References Chai, Wei and Vercoe, Barry. (n.d.). Folk Music Classification Using Hidden Markov Models. MIT Media Lab. M. Stamp, A revealing introduction to Hidden Markov Models http://www.cs.sjsu.edu/faculty/stamp/Hampton/HMM.pdf http://www.cs.sjsu.edu/faculty/stamp/Hampton/HMM.pdf M. Stamp and E. Le, Hamptonese website. http://www.cs.sjsu.edu/faculty/stamp/Hampton/hampton. html http://www.cs.sjsu.edu/faculty/stamp/Hampton/hampton. html R.L. Cave and L.P. Neuwirth, Hidden Markov Models for English, in Hidden Markov Models for Speech, IDA-CRD, Princeton, NJ, 1980