Tools for Tracking Keyword-based Censorship: Character Comparison This material is based upon work supported by the National Science Foundation under Grant No. IIS/REU/ Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. Veronika Strnadova and Leif Guillermo with Jedidiah Crandall, Tony Espinoza and Ronnie Garduno

China Most advanced filtering system in the world Keyword based evidence Concept Doppler Falun Gong Tienanmen Square Political Dissident Censorship Jail

Chinese filtering system Goal:  Maintain an updated list of words that are being filtered Our part:  Find new filtered words by testing known blacklisted words in new forms “New forms”  Forms of words that users may try to avoid filtering

Cao Ni Ma cǎo ní mǎ cào nǐ mā

New Combinations of Characters Falun Gong 法轮功 Sound Permutations  功 [gong1] (meritorious deed or service/achievement/result)  龔 [Gong1] (surname Gong) Visual Permutations  功 [gong1] (meritorious deed or service/achievement/result)  劝 [quan4] (to advise/urge/persuade/exhort)

Methods – Sound Permutations Sound Permutations  Find all permutations of a word based on its pronunciation  Test these permutations for filtering  We found some new filtered words! Known filtered word: 法轮功 Found word: 法仑龔

Methods – Visual Permutations Point Feature Comparison

Feature Extraction Tamasi & Kanade  Features selected based on best fit for motion tracking Hough Transform  Features selected as endpoints of Hough lines that best fit the image Harris Corner Detector  Features selected based on image gradient in a region around each pixel

Harris Corner Detection Find points where two edges meet—i.e., high gradient in two directions Change of intensity for the shift [u,v]:  S(x,y) = Ʃ Ʃ (w(u,v) *( I(u,v) - I(u+x,v+y) )^2 u v

Harris Corner Detection For small shifts (u,v) we have a bilinear approximation:  S(x,y) ~ [u v] M [u v]' Where M is a 2x2 matrix computed from image derivatives:  M = [ Ixx Ixy Ixy Iyy]

Harris Corner Detection Measure of Corner Response  R = det(M) – kTrace(M)^2 Find points with large corner response function R (R > threshold) det(M) = ʎ 1* ʎ 2 trace(M) = ʎ 1 + ʎ 2 (Ix2*Iy2 – Ixy²) / (Ix2 + Iy2 + ɛ )

Preliminary Results (Frobenius Inner Product)

Similarity Comparison Extract feature points for each image Build feature vectors Build proximity matrix for every pair of images I, J  G(i,j) = e^( (-r(i,j)^2) / (2*(sigma^2) )

Similarity Comparison Perform SVD on G [U S V] = svd(G)

Similarity Comparison Build E, where E is same size as S and E(i,i) is 1 for every S(i,i) that is nonzero Build a matrix P which maximizes the inner product P:G  P = U*E*V'

Similarity Comparison If an element in P, P(i,j) is both the greatest element in its row and its column, then we have found a match for feature i in image I and feature j in image J

Similarity Measure Build a matrix of vectors describing matching features for each pair of images:  V = [ r1 theta1  r2 theta2 .  rN thetaN] Similarity Measure = average distances between mapped features

New Results

Dissimilar Images

Future Work Find a good information theoretic similarity measure  Specifically, one that includes angles Test visually similar permutations for filtering Test more phonetically similar words for filtering

Please hold all questions for Tony (Just Kidding) Thank you for listening! Questions?