Evolving Hypernetworks for Language Modeling AI Course Material Oct. 12, 2009 Byoung-Tak Zhang Biointelligence Laboratory School of Computer Science and.

Evolving Hypernetworks for Language Modeling AI Course Material Oct. 12, 2009 Byoung-Tak Zhang Biointelligence Laboratory School of Computer Science and Engineering Brain Science, Cognitive Science, Bioinformatics Programs Seoul National University Seoul 151-742, Korea btzhang@cse.snu.ac.kr http://bi.snu.ac.kr/

Outline Problem: Language Modeling  Data  Model Model: Hypernetwork  Individuals  Population Method: Evolving Hypernetworks  Variation  Selection  Amplification  Fitness Evaluation Experimental Results © 2006-2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/ 2

5 Evolutionary Hypernets for Linguistic Memory Why ? you ? come ? down ? Why are you go come on down here ? appreciate it if ? call her by ? ? I appreciate it if you call her by the way Would you ? to meet ? ? Tuesday ? Would you nice to meet you in Tuesday and ? gonna ? upstairs ? ? a shower I'm gonna go upstairs and take a shower ? have ? visit the ? room I have to visit the ladies' room ? ? ? decision to make a decision ? still ? believe ? did this I still can't believe you did this Zhang and Park, Self-assembling hypernetworks for cognitive learning of linguistic memory, International Conf. on Cognitive Science (ICCS-2008), WASET, pp. 134-138, 2008. Zhang, Cognitive learning and the multimodal memory game: Toward human-level machine learning, IEEE World Congress on Computational Intelligence (WCCI-2008), 2008.

Data and Model Data D = { S i | S i are sentences }  Eg. Dialogue sentences from “Friends” Model M = { R j | R j are grammar rules } Generator g: S x M  S Learner L: D  M Goal: to learn the grammar by evolution 6 © 2006-2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/

7 x1 =1 x2 =0 x3 =0 x4 =1 x5 =0 x6 =0 x7 =0 x8 =0 x9 =0 x10 =1 x11 =0 x12 =1 x13 =0 x14 =0 x15 =0 y = 1 x1 =0 x2 =1 x3 =1 x4 =0 x5 =0 x6 =0 x7 =0 x8 =0 x9 =1 x10 =0 x11 =0 x12 =0 x13 =0 x14 =1 x15 =0 y = 0 x1 =0 x2 =0 x3 =1 x4 =0 x5 =0 x6 =1 x7 =0 x8 =1 x9 =0 x10 =0 x11 =0 x12 =0 x13 =1 x14 =0 x15 =0 y =1 1. Sentences x4x4 x 10 y=1x1x1 x4x4 x 12 y=1x1x1 x 10 x 12 y=1x4x4 x3x3 x9x9 y=0x2x2 x3x3 x 14 y=0x2x2 x9x9 x 14 y=0x3x3 x6x6 x8x8 y=1x3x3 x6x6 x 13 y=1x3x3 x8x8 x 13 y=1x6x6 1 2 3 1 2 3 x1 =0 x2 =0 x3 =0 x4 =0 x5 =0 x6 =0 x7 =0 x8 =1 x9 =0 x10 =0 x11 =1 x12 =0 x13 =0 x14 =0 x15 =1 y =1 4 x 11 x 15 y=0x8x8 4 Round 1 Round 2 Round 3 2. Many Micro Grammar Rules 3. Hyperedges (Individuals) 4. Hypernet = Grammar (Population) © 2006-2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/

10 x1 =1 x2 =0 x3 =0 x4 =1 x5 =0 x6 =0 x7 =0 x8 =0 x9 =0 x10 =1 x11 =0 x12 =1 x13 =0 x14 =0 x15 =0 y = 1 x1 =0 x2 =1 x3 =1 x4 =0 x5 =0 x6 =0 x7 =0 x8 =0 x9 =1 x10 =0 x11 =0 x12 =0 x13 =0 x14 =1 x15 =0 y = 0 x1 =0 x2 =0 x3 =1 x4 =0 x5 =0 x6 =1 x7 =0 x8 =1 x9 =0 x10 =0 x11 =0 x12 =0 x13 =1 x14 =0 x15 =0 y =1 4 examples x4x4 x 10 y=1x1x1 x4x4 x 12 y=1x1x1 x 10 x 12 y=1x4x4 x3x3 x9x9 y=0x2x2 x3x3 x 14 y=0x2x2 x9x9 x 14 y=0x3x3 x6x6 x8x8 y=1x3x3 x6x6 x 13 y=1x3x3 x8x8 x 13 y=1x6x6 1 2 3 1 2 3 x1 =0 x2 =0 x3 =0 x4 =0 x5 =0 x6 =0 x7 =0 x8 =1 x9 =0 x10 =0 x11 =1 x12 =0 x13 =0 x14 =0 x15 =1 y =1 4 x 11 x 15 y=0x8x8 4 Round 1 Round 2 Round 3 © 2006-2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/

11 Initial Library L 0 (x 2 =1, x 3 =1, y=1) (x 2 =1, x 3 =0, y=0) AATTGGAAGGCCATGCCC AATTGGCCTTGGATGCGG (x 1 =0, x 2 =0, x 3 =1, y=0) (x 1 =0, x 2 =1, x 3 =1, y=1) AAAACCAATTGGAATTGGATGCGG (x 2 =1, y=0) AATTGGATGCCC AAAACCAATTCCAAGGGGATGCCC (x 1 =0, y=1) AAAACCATGCGG (x 1 =0, y=0) AAAACCATGCCC (x 2 =0, y=1) AATTCCATGCGG (x 2 =0, y=0) AATTCCATGCCC … (x 1 =0, x 2 =0, y=0) AAAACCAATTCCATGCCC (x 1 =0, x 2 =0, y=1) AAAACCAATTCCATGCGG (x 1 =0, x 2 =1, y=0) AAAACCAATTGGATGCCC (x 1 =0, x 2 =1, y=1) AAAACCAATTGGATGCGG … (x 1 =0, x 2 =0, x 3 =0, y=0) AAAACCAATTCCAAGGCCATGCCC (x 1 =0, x 2 =0, x 3 =0, y=1) AAAACCAATTCCAAGGCCATGCGG (x 1 =0, x 2 =0, x 3 =1, y=0) AAAACCAATTCCAAGGGGATGCCC (x 1 =0, x 2 =0, x 3 =1, y=1) AAAACCAATTCCAAGGGGATGCGG (x 1 =0, x 2 =1, x 3 =0, y=0) AAAACCAATTGGAAGGCCATGCCC (x 1 =0, x 2 =1, x 3 =0, y=1) AAAACCAATTGGAAGGCCATGCGG … x1x1 x2x2 x3x3 y 0 1 where AAGG AATT AAAA ATGC CC GG © 2006-2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/

12 + Amplify Library Example 1 (x 1 =0, x 2 =1, x 3 =0, y=0) TACGGG TTCCGGTTAACCTTTTGG AATTGGAAGGCCATGCCC AATTGGCCTTGGATGCGG AAAACCAATTCCAAGGGGATGCCC AAAACCAATTGGAATTGGATGCGG AATTGGATGCCC TTTTGG TTAACC TTCCGG GGTTGG Hybridization (x 1 =0, x 2 =1, x 3 =1, y=1) (x 1 =0, x 2 =0, x 3 =1, y=0) (x 2 =1, x 3 =1, y=1) (x 2 =1, x 3 =0, y=0) (x 2 =1, y=0) TACGGG TTCCGGTTAACCTTTTGG TACGGG TTCCGGTTAACCTTTTGG (x 2 =1, x 3 =1, y=1) (x 2 =1, x 3 =0, y=0) AATTGGAAGGCCATGCCC AATTGGCCTTGGATGCGG (x 1 =0, x 2 =0, x 3 =1, y=0) AAAACCAATTCCAAGGGGATGCCC (x 1 =0, x 2 =1, x 3 =1, y=1) AAAACCAATTGGAATTGGATGCGG (x 2 =1, y=0) AATTGGATGCCC © 2006-2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/

13 Updated Library L 1 (x 2 =1, x 3 =1, y=1) (x 2 =1, x 3 =0, y=0) AATTGGAAGGCCATGCCC AATTGGCCTTGGATGCGG (x 1 =0, x 2 =0, x 3 =1, y=0) (x 1 =0, x 2 =1, x 3 =1, y=1) AAAACCAATTGGAATTGGATGCGG (x 2 =1, y=0) AATTGGATGCCC AATTGGAAGGCCATGCCC AATTGGATGCCC AAAACCAATTCCAAGGGGATGCCC (x 1 =0, y=1) AAAACCATGCGG (x 1 =0, y=0) AAAACCATGCCC (x 2 =0, y=1) AATTCCATGCGG (x 2 =0, y=0) AATTCCATGCCC … (x 1 =0, x 2 =0, y=0) AAAACCAATTCCATGCCC (x 1 =0, x 2 =0, y=1) AAAACCAATTCCATGCGG (x 1 =0, x 2 =1, y=0) AAAACCAATTGGATGCCC (x 1 =0, x 2 =1, y=1) AAAACCAATTGGATGCGG … (x 1 =0, x 2 =0, x 3 =0, y=0) AAAACCAATTCCAAGGCCATGCCC (x 1 =0, x 2 =0, x 3 =0, y=1) AAAACCAATTCCAAGGCCATGCGG (x 1 =0, x 2 =0, x 3 =1, y=0) AAAACCAATTCCAAGGGGATGCCC (x 1 =0, x 2 =0, x 3 =1, y=1) AAAACCAATTCCAAGGGGATGCGG (x 1 =0, x 2 =1, x 3 =0, y=0) AAAACCAATTGGAAGGCCATGCCC (x 1 =0, x 2 =1, x 3 =0, y=1) AAAACCAATTGGAAGGCCATGCGG … x1x1 x2x2 x3x3 y 0 1 where AAGG AATT AAAA ATGC CC GG © 2006-2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/

14 + Amplify Library (x 2 =1, x 3 =1, y=1) (x 2 =1, x 3 =0, y=0) AATTGGAAGGCCATGCCC AATTGGCCTTGGATGCGG (x 1 =0, x 2 =0, x 3 =1, y=0) AAAACCAATTCCAAGGGGATGCCC (x 1 =0, x 2 =1, x 3 =1, y=1) AAAACCAATTGGAATTGGATGCGG Example 2 (x 1 =0, x 2 =1, x 3 =1, y=1) TTCCCCTTAACCTTTTGG TACGCC (x 2 =1, y=0) AATTGGATGCCC AATTGGAAGGCCATGCCC AATTGGCCTTGGATGCGG AAAACCAATTCCAAGGGGATGCCC AAAACCAATTGGAATTGGATGCGG AATTGGATGCCC TTTTGG TTAACC Hybridization (x 1 =0, x 2 =1, x 3 =1, y=1) (x 1 =0, x 2 =0, x 3 =1, y=0) (x 2 =1, x 3 =1, y=1) (x 2 =1, x 3 =0, y=0) (x 2 =1, y=0) TACGCC TTCCCCTTAACCTTTTGG TACGCC TTCCCCTTAACCTTTTGG (x 2 =1, x 3 =0, y=0) AATTGGAAGGCCATGCCC (x 2 =1, y=0) AATTGGATGCCC TTCCCC TACGCC TTCCCC TACGCC AATTGGAAGGCCATGCCC AATTGGATGCCC TTAACC © 2006-2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/

15 Updated Library L 2 (x 2 =1, x 3 =1, y=1) (x 2 =1, x 3 =0, y=0) AATTGGAAGGCCATGCCC AATTGGCCTTGGATGCGG (x 1 =0, x 2 =0, x 3 =1, y=0) (x 1 =0, x 2 =1, x 3 =1, y=1) AAAACCAATTGGAATTGGATGCGG (x 2 =1, y=0) AATTGGATGCCC AATTGGAAGGCCATGCCC AATTGGATGCCC AAAACCAATTCCAAGGGGATGCCC (x 1 =0, y=1) AAAACCATGCGG (x 1 =0, y=0) AAAACCATGCCC (x 2 =0, y=1) AATTCCATGCGG (x 2 =0, y=0) AATTCCATGCCC … (x 1 =0, x 2 =0, y=0) AAAACCAATTCCATGCCC (x 1 =0, x 2 =0, y=1) AAAACCAATTCCATGCGG (x 1 =0, x 2 =1, y=0) AAAACCAATTGGATGCCC (x 1 =0, x 2 =1, y=1) AAAACCAATTGGATGCGG … (x 1 =0, x 2 =0, x 3 =0, y=0) AAAACCAATTCCAAGGCCATGCCC (x 1 =0, x 2 =0, x 3 =0, y=1) AAAACCAATTCCAAGGCCATGCGG (x 1 =0, x 2 =0, x 3 =1, y=0) AAAACCAATTCCAAGGGGATGCCC (x 1 =0, x 2 =0, x 3 =1, y=1) AAAACCAATTCCAAGGGGATGCGG (x 1 =0, x 2 =1, x 3 =0, y=0) AAAACCAATTGGAAGGCCATGCCC (x 1 =0, x 2 =1, x 3 =0, y=1) AAAACCAATTGGAAGGCCATGCGG … AAAACCAATTGGAATTGGATGCGG AATTGGCCTTGGATGCGG x1x1 x2x2 x3x3 y 0 1 where AAGG AATT AAAA ATGC CC GG © 2006-2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/

16 + Library (x 2 =1, x 3 =1, y=1) (x 2 =1, x 3 =0, y=0) AATTGGAAGGCCATGCCC AATTGGCCTTGGATGCGG (x 1 =0, x 2 =0, x 3 =1, y=0) AAAACCAATTCCAAGGGGATGCCC (x 1 =0, x 2 =1, x 3 =1, y=1) AAAACCAATTGGAATTGGATGCGG Query (x 1 =1, x 2 =1, x 3 =0) TTCCGG TTAACCTTTTCC (x 2 =1, y=0) AATTGGATGCCC Hybridization TTCCGGTTAACCTTTTCC TTAACCTTTTCC AAAACCAATTGGAATTGGATGCGG AATTGGCCTTGGATGCGG AATTGGAAGGCCATGCCC AATTGGATGCCC TTCCGG AATTGGAAGGCCATGCCC AATTGGCCTTGGATGCGG AAAACCAATTCCAAGGGGATGCCC AAAACCAATTGGAATTGGATGCGG AATTGGATGCCC (x 1 =0, x 2 =1, x 3 =1, y=1) (x 1 =0, x 2 =0, x 3 =1, y=0) (x 2 =1, x 3 =1, y=1) (x 2 =1, x 3 =0, y=0) (x 2 =1, y=0) TTCCGG TTAACC AAAACCAATTGGAATTGGATGCGG TTAACC AATTGGCCTTGGATGCGG TTAACC AATTGGAAGGCCATGCCC TTCCGG TTAACC AATTGGATGCCC TTAACC Majority voting Predict the class © 2006-2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/

17 1. Let the library L represent the current distribution P(X,Y). 2. Get a training example (x,y). 3. Classify x using L as follows 3.1 Extract all molecules matching x into M. 3.2 From M separate the molecules into classes: Extract the molecules with label Y=0 into M 0 Extract the molecules with label Y=1 into M 1 3.3 Compute y * =argmax Y  {0,1} | M Y |/|M| 4. Update L If y * =y, then L n ← L n-1 +{  c(u, v)} for u=x and v=y for (u, v)  L n-1, If y * ≠y, then L n ← L n-1  {  c(u, v)} for u=x and v ≠ y for (u, v)  L n-1 5.Goto step 2 if not terminated. Molecular Programming (MP): The Evolutionary Learning Algorithm [Zhang, GECCO-2005] © 2006-2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/

18 Learning the Hypernetwork (by Evolutionary Self-assembly) Library of combinatorial molecules + LibraryExample Select the library elements matching the example Amplify the matched library elements by PCR Next generation Hybridize [Zhang, DNA11] © 2006-2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/

© 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/ 22 Text Corpus: TV Drama Series Friends, 24, House, Grey Anatomy, Gilmore Girls, Sex and the City 289,468 Sentences (Training Data) 700 Sentences with Blanks (Test Data) I don't know what happened. Take a look at this. … What ? ? ? here. ? have ? visit the ? room. …

24 Evolutionary Hypernets for Linguistic Memory Why ? you ? come ? down ? Why are you go come on down here ? appreciate it if ? call her by ? ? I appreciate it if you call her by the way Would you ? to meet ? ? Tuesday ? Would you nice to meet you in Tuesday and ? gonna ? upstairs ? ? a shower I'm gonna go upstairs and take a shower ? have ? visit the ? room I have to visit the ladies' room ? ? ? decision to make a decision ? still ? believe ? did this I still can't believe you did this Zhang and Park, Self-assembling hypernetworks for cognitive learning of linguistic memory, International Conf. on Cognitive Science (ICCS-2008), WASET, pp. 134-138, 2008. Zhang, Cognitive learning and the multimodal memory game: Toward human-level machine learning, IEEE World Congress on Computational Intelligence (WCCI-2008), 2008.

(c) 2009 SNU Biointelligence Laboratory, http://bi.snu.ac.kr/ 25 Corpus: Friends Keyword: “mother” Corpus: Prison Break Keyword: “mother” you're mother killed herself it's my mother was shot by a woman at eight we're just gonna go to your mother that i love it feeling that something's wrong with my mother and father she's the single mother i put this on my friend's mother apparently phoebe's mother killed herself thanks for pleasing my mother killed herself i'm your mother told you this is an incredible mother that's not his mother or his hunger strike holy mother of god woman i like your mother and father on their honeymoon suite with her and never called your mother really did like us is my mother was shot by a drug dealer tells his mother and his family she's the mother of my eyes speak to your mother used to be tells his mother made it pretty clear on the floor has speak to your mother never had life insurance she's the mother of lincoln's child she's the mother of my own crap to deal with you just lost his mother is fine just lost his mother and his god tells his mother and his stepfather she's the mother of my time his mother made it clear you couldn't deliver fibonacci she's the mother of my brother is facing the electric chair same guy who was it your mother before you do it they gunned my mother down Memories for Friends and Prison Break

Learning Languages from Kids Video Goal: (1) Natural language generation at sentence level based on the probabilistic graphical model, and (2) Natural language processing without the explicit grammar rules. © 2009 SNU CSE Biointelligence Lab 26  Training data Kids video scripts Sentence structure Converting sentences into graph structure  Application Sentence completion and generation Script sequenceGenerated sentence TimothyI like it too nora. Hello kittyI like it too mom. Looney toonsI like it too this time you're a diving act today. Dora I like it too this time you're a hug.

Generated Sentences and Evolved Grammar Generated sentences  (Good) On my first day of school  (Good) Yes timothy it is time to go to school  (Good) Thomas and Percy enjoy working in the spotlight  (Good) Well it is morning  (Bad) He couldn’t way to go outside and shoot  (Bad) the gas house gorillas are a lot of fun players Grammar rules analyzed from the generated sentences  G1: S = NP + VP,G2: NP = PRP  G3: S = VP, G4: PP = IN + NP  G5: NP = NN,G6: NP = DP + NN  G7: ADVP = RB,G8: NP = NP + PP  G9: SBAR = S © 2009 SNU CSE Biointelligence Lab 27

Sentence Generation Accuracy Corpus: scripts from kids video (Miffy, Looney, Caillou, Dora Dora, Macdonald, Thoams & Friends, Timothy, Pooh) Corpus: Video scripts (kids video + sitcom Friends, 120K sentences) In each phase, corpus size is incremented by addition of a video script. Learning: building a language model based on a hypernetwork. Task: Sentence completion from a partial sentence. © 2009 SNU CSE Biointelligence Lab 28 D 1 D 2 D 3 D 4 D 5 D 6 D 7 D 8 D 9 D 1 = Miffy, D 2 = D 1 + Looney, D 3 = D 2 + caillou, D 4 = D 3 + Dora Dora D 5 = D 4 + Macdoland, D 6 = D 5 + Thomas, D 7 = D 6 + Timothy, D 8 = D 7 + Pooh, D 9 = D 8 + Friends

Evolution of Grammar Rules © 2009 SNU CSE Biointelligence Lab 29 Grammar learning curve KL divergence between the distribution of training corpus (P) and the generated sentences (Q). The right curve shows occurrence number of grammar rules are increasing as training progresses. D 1 D 2 D 3 D 4 D 5 D 6 D 7 D 8 D 9 Grammar rules learning curve G* = grammar rule *

DNA Computing and DNA Nanotechnology: An Introduction

Hypernetworks: More Details

32 Hypergraphs A hypergraph is a (undirected) graph G whose edges connect a non-null number of vertices, i.e. G = (V, E), where V = {v 1, v 2, …, v n }, E = {E 1, E 2, …, E n }, and E i = {v i1, v i2, …, v im } An m-hypergraph consists of a set V of vertices and a subset E of V [m], i.e. G = (V, V [m] ) where V [m] is a set of subsets of V whose elements have precisely m members. A hypergraph G is said to be k-uniform if every edge E i in E has cardinality k. A hypergraph G is k-regular if every vertex has degree k. Rem.: An ordinary graph is a 2-uniform hypergraph. © 2006-2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/

33 An Example Hypergraph v5 v1 v3 v7 v2 v6 v4 G = (V, E) V = {v1, v2, v3, …, v7} E = {E1, E2, E3, E4, E5} E1 = {v1, v3, v4} E2 = {v1, v4} E3 = {v2, v3, v6} E4 = {v3, v4, v6, v7} E5 = {v4, v5, v7} E1 E4 E5 E2 E3 © 2006-2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/

34 Hypernetworks A hypernetwork is a hypergraph of weighted edges. It is defined as a triple H = (V, E, W), where V = {v 1, v 2, …, v n }, E = {E 1, E 2, …, E n }, and W = {w 1, w 2, …, w n }. An m-hypernetwork consists of a set V of vertices and a subset E of V [m], i.e. H = (V, V [m], W) where V [m] is a set of subsets of V whose elements have precisely m members and W is the set of weights associated with the hyperedges. A hypernetwork H is said to be k-uniform if every edge E i in E has cardinality k. A hypernetwork H is k-regular if every vertex has degree k. Rem.: An ordinary graph is a 2-uniform hypergraph with w i =1. [Zhang, 2006, in preparation] © 2006-2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/

40 Encoding Hyperedges with DNA z 1 : z 2 : z 3 : z 4 : b) x1x1 x2x2 x3x3 x4x4 x5x5 y 0 1 where z 1 : (x 1 =0, x 2 =1, x 3 =0, y=1) z 2 : (x 1 =0, x 2 =0, x 3 =1, x 4 =0, x 5 =0, y=0) z 3 : (x 2 =1, x 4 =1, y=1) z 4 : (x 2 =1, x 3 =0, x 4 =1, y=0) a) AAAACCAATTGGAAGGCCATGCGG AAAACCAATTCCAAGGGGCCTTCCCCAACCATGCCC AATTGGCCTTGGATGCGG AATTGGAAGGCCCCTTGGATGCCC GG AAAA AATT AAGG CCTT CCAA ATGC CC Collection of (labeled) hyperedges Library of DNA molecules corresponding to (a) © 2006-2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/

41 The Theory of Bayesian Evolution P0(Ai)P0(Ai) P g (A i |D)... generation 0 generation g P(A |D)P(A |D) P(A |D)P(A |D) Pg(Ai)Pg(Ai) [Zhang, CEC-99] Evolution as a Bayesian inference process Evolutionary computation (EC) is viewed as an iterative process of generating the individuals of ever higher posterior probabilities from the priors and the observed data. © 2006-2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/

42 Unconventional Computing Quantum Computing  Atoms  Superposition, quantum entanglements Chemical Computing  Chemicals  Reaction-diffusion computing Molecular Computing  Molecules  “Evolutionary Hypernetworks” © 2006-2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/

43 Molecular Computers vs. Silicon Computers Molecular ComputersSilicon Computers ProcessingBallisticHardwired MediumLiquid (wet) or Gaseous (dry)Solid (dry) Communication3D collision2D switching ConfigurationAmorphous (asynchronous)Fixed (synchronous) ParallelismMassively parallelSequential SpeedFast (millisec)Ultra-fast (nanosec) ReliabilityLowHigh DensityUltrahighVery high DevicesUnreliableReliable © 2006-2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/

44 DNA as “Programmable” Nanomatter Information Density:  10 6 Gbits per cm 2 (1 bit per nm 3 )  Semiconductor: 1 Gbits per cm 2 Massive Parallelism:  10 26 reactions per 1 mmol of DNA  Desktop: 10 9 operations / sec  Supercomputer: 10 12 operations / sec Energy Consumption:  10 19 operations per Joule  Semiconductor: 10 9 operations per Joule © 2006-2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/

Evolving Hypernetworks for Language Modeling AI Course Material Oct. 12, 2009 Byoung-Tak Zhang Biointelligence Laboratory School of Computer Science and.

Similar presentations

Presentation on theme: "Evolving Hypernetworks for Language Modeling AI Course Material Oct. 12, 2009 Byoung-Tak Zhang Biointelligence Laboratory School of Computer Science and."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Evolving Hypernetworks for Language Modeling AI Course Material Oct. 12, 2009 Byoung-Tak Zhang Biointelligence Laboratory School of Computer Science and.

Similar presentations

Presentation on theme: "Evolving Hypernetworks for Language Modeling AI Course Material Oct. 12, 2009 Byoung-Tak Zhang Biointelligence Laboratory School of Computer Science and."— Presentation transcript:

Similar presentations

About project

Feedback