A Human-Centered Computing Framework to Enable Personalized News Video Recommendation (Oh Jun-hyuk)

Questions How to detect news topic from video? How to measure inter-topic association? How to measure interestingness of news topic?

Contents Introduction Related Work User-Adaptive News Topic Recommendation News Topic Detection Topic Association Extraction Interestingness Scores of News Topics Hyperbolic Topic Network Visualization Personalized Topic Network Generation Personalized News Video Recommendation Algorithm Evaluation Conclusions

Contents Introduction Related Work User-Adaptive News Topic Recommendation News Topic Detection Topic Association Extraction Interestingness Scores of News Topics Hyperbolic Topic Network Visualization Personalized Topic Network Generation Personalized News Video Recommendation Algorithm Evaluation Conclusions

Introduction News video recommendation in CNN Not related to the current news topic and user profile Users need to follow(subscribe) news topics manually

Introduction Video recommendation in YouTube Related to the current video topic and user profile but not visualized

Introduction Topic Network Visualize news videos and represent inter-topic association. Hyperbolic visualization Enables interactive topic network navigation(browsing). Recommend the news topics of interest according to the personal preferences.

Contents Introduction Related Work User-Adaptive News Topic Recommendation News Topic Detection Topic Association Extraction Interestingness Scores of News Topics Hyperbolic Topic Network Visualization Personalized Topic Network Generation Personalized News Video Recommendation Algorithm Evaluation Conclusions

Related Work 1. Automatic news topic detection Identification of individual topics within a broadcast news video by detecting the boundaries where the topic of discussion changes. Special program structures and styles can be used to detect boundaries. IntroNewsIntroNewsIntroNews…… KBS news broadcast

Related Work 2. News visualization Existing visualization systems disclose all the available news topics to news seekers without considering interestingness of topics. It will be better to provide a small number of interesting news!

Contents Introduction Related Work User-Adaptive News Topic Recommendation News Topic Detection Topic Association Extraction Interestingness Scores of News Topics Hyperbolic Topic Network Visualization Personalized Topic Network Generation Personalized News Video Recommendation Algorithm Evaluation Conclusions

User-Adaptive News Topic Recommendation Goal : recommend the news topics of interest by incorporating topic network and hyperbolic visualization. To-do List News Topic Detection Topic Association Extraction Interestingness Scores of News Topics Hyperbolic Topic Network Visualization Personalized Topic Network Generation

Contents Introduction Related Work User-Adaptive News Topic Recommendation News Topic Detection Topic Association Extraction Interestingness Scores of News Topics Hyperbolic Topic Network Visualization Personalized Topic Network Generation Personalized News Video Recommendation Algorithm Evaluation Conclusions

News Topic Detection Define a set of over 4,000 elemental news topics. Three major sources are integrated Audio, Video, Closed Caption

News Topic Detection

News Topic Detection – Closed Captions Natural Language Processing(NLP) is conducted. Closed Captions are segmented into a set of keywords. Special text sentences are removed by syntax parser ex) CNNs Andrew reports from Seoul not related to the topic TreeTagger is used to extract the POS(part-of-speech) information POS : a linguistic category of words (lexical category) LingPipe is used to extract keywords.

News Topic Detection – Closed Captions News Topic K-POP, Korean Music, PSY LingPipe (Keyword Extraction) Korean, music, PSY, Gangnam Style, release, sequel TreeTagger (POS Tagging) Korean[NN] music[NN] sensation[NN] PSY[FW] will[MD] release[VD] his[PP$] much[JJ] anticipated[JJ] sequel[NN] to[TO] "Gangnam Style[FW] Friday[NN].[SENT] Original Caption Korean music sensation PSY will release his much anticipated sequel to "Gangnam Style" Friday.

News Topic Detection

News Topic Detection – Audio Automatic Speech Recognition(ASR) system is used to translate the audio channel to a text transcription. Audio Text processed in a similar way to closed caption. Hidden Markov Model for speech recognition

News Topic Detection

News Topic Detection – Video Detect video objects(text area, human face) because they provide important clues about news story. Confidence map is used to measure the importance of video objects in video.

News Topic Detection

Contents Introduction Related Work User-Adaptive News Topic Recommendation News Topic Detection Topic Association Extraction Interestingness Scores of News Topics Hyperbolic Topic Network Visualization Personalized Topic Network Generation Personalized News Video Recommendation Algorithm Evaluation Conclusions

Topic Association Extraction Inter-topic association(inter-topic contextual relationship) d(Ci, Cj ) : The length of the shortest path between the news topics by searching the relevant keywords for news topic interpretation fro m WordNet. ψ(Ci, Cj) : the co-occurrence probability between the relevant news topics obtained in news topic detection process. The frequency of co-occurrence of two news topic keywords in the same video. ex) In a news video, PSY, Music co-occurs.

Topic Association Extraction WordNet : a lexical database for the English language. Provide a graph representing semantic relationship between words.

Topic Association Extraction Topic network can be generated from topic association News topics are organized according to the strength of their association Allow news seekers to easily recognize global overview of large-scale news videos at the first glance.

Contents Introduction Related Work User-Adaptive News Topic Recommendation News Topic Detection Topic Association Extraction Interestingness Scores of News Topics Hyperbolic Topic Network Visualization Personalized Topic Network Generation Personalized News Video Recommendation Algorithm Evaluation Conclusions

Interestingness Scores of News Topics Interestingness Score m(Ci) : the number of TV channels or news programs which have di scussed the given news topic Ci. Popularity k(Ci) : the number of news topics linked with the given news topic C i on the topic network. Importance (similar to PageRank) Used to highlight the mo st interesting news topi cs and eliminate the les s interesting news topic s for reducing the visual complexity for large-scal e topic network visualiz ation.

Interestingness Scores of News Topics PageRank Algorithm

Summary & Answers How to detect news topic from video? Define a set of news topic. Integrate multi-modal sources Closed caption : Natural language processing Audio : Automatic speech recognition Video : video object extraction and classification How to measure inter-topic association? Keyword association : length of path between keywords in WordNet Co-occurrence : the probability of co-occurrence obtained in news topic detection process. How to measure interestingness of news topic? Popularity : the number of TV channels or news programs which have discussed the given news topic Importance : the number of news topics linked with the given news topic on the topic network.

THANK YOU Q&A