1. ADBOT Advertisement Recognition FROM television and radio broadcast
Computer Vision & Machine Intelligence Group
Department of Computer Science
College of Engineering
UP Diliman
Outline
Significance
Keywords and Definitions
System Architecture
Experiments and Results
The AdBot Team:
Aljon Rey P. Aniban
Anna Mae C. Yap
Emmanuel B. Cagadas
The AdBot Advisers:
Prospero Naval Jr. PhD.
Carlo Raquel MS
4/23/ :57 PM
AdBot: Advertisement Recognition from TV and Radio Broadcast

2. I. Significance
Advertising plays an important role in the marketing industry.
4/23/ :57 PM
AdBot: Advertisement Recognition from TV and Radio Broadcast

3. AdBot: Advertisement Recognition from TV and Radio Broadcast
I. Significance
4/23/ :57 PM
AdBot: Advertisement Recognition from TV and Radio Broadcast

4. AdBot: Advertisement Recognition from TV and Radio Broadcast
I. Significance
In developed countries, Advertisement monitoring has already been automated.
Mediaguide.com
TNS Media Intelligence’s Broadcast Verification Services
In the Philippines, Advertising companies monitor advertisements manually.
4/23/ :57 PM
AdBot: Advertisement Recognition from TV and Radio Broadcast

5. AdBot: Advertisement Recognition from TV and Radio Broadcast
I. Significance
AdBot can monitor advertisements utilizing only the audio feature as input.
4/23/ :57 PM
AdBot: Advertisement Recognition from TV and Radio Broadcast

6. II. Keywords and Definitions
4/23/ :57 PM
AdBot: Advertisement Recognition from TV and Radio Broadcast

7. AdBot: Advertisement Recognition from TV and Radio Broadcast
II. Keywords
Audio Landmarks
Hash Values
Fingerprint
SNR – Sound to Noise Ratio
4/23/ :57 PM
AdBot: Advertisement Recognition from TV and Radio Broadcast

8. Points in the graph with high freqeuncy concentration.
II. Keywords
Audio Landmarks
Points in the graph with high freqeuncy concentration.
4/23/ :57 PM
AdBot: Advertisement Recognition from TV and Radio Broadcast

9. Hash value(ref) = ΔFrequency
II. Keywords
Hash Values
Hash value(ref) = ΔFrequency
Δtime
*every landmark gets a chance to be a reference point.
*The number of landmarks found = the number of hash values that will be generated
Reference
landmark
Target Zone
4/23/ :57 PM
AdBot: Advertisement Recognition from TV and Radio Broadcast

10. AdBot: Advertisement Recognition from TV and Radio Broadcast
II. Keywords
Fingerprint
When these landmarks are connected, they form a constallation like figure ---- which as a whole, serves as the unique fingerprint of an audio file.
*the hash values generated collectively form the numerical fingerprint of an audio file.
4/23/ :57 PM
AdBot: Advertisement Recognition from TV and Radio Broadcast

11. AdBot: Advertisement Recognition from TV and Radio Broadcast
II. Keywords
SNR – Sound to Noise Ratio
We used:
y = awgn(x,snr) adds white Gaussian noise to the vector signal x. The scalar snr specifies the signal-to-noise ratio in decibels.
4/23/ :57 PM
AdBot: Advertisement Recognition from TV and Radio Broadcast

12. III. System Architecture
Technologies used: MATLAB, C and Java
III. System Architecture
4/23/ :57 PM
AdBot: Advertisement Recognition from TV and Radio Broadcast

13. AdBot: Advertisement Recognition from TV and Radio Broadcast
III. System Architecture
Adbot is divided into 2 modules:
Landmark extraction (Fingerprinting)
Matching
4/23/ :57 PM
AdBot: Advertisement Recognition from TV and Radio Broadcast

14. III. System Architecture
Landmark extraction (Fingerprinting)
Hash function
Landmark conversion
(high frequency concentration)
Hashes
Commercial Audio Input (.wav) 01 02 03 04 05 06 n
Database
4/23/ :57 PM
AdBot: Advertisement Recognition from TV and Radio Broadcast

15. III. System Architecture
Matching 01 02 03 04 05 06 n
Hash values of the consecutive landmarks from the segments of a commercial .
Searching of landmark within the segments
Hash function
Database ?
Three-second segmentation of the query audio
Start
Matching of segments
has a match
has no match
Previous and succeeding segments matched?
yes no
End
Audio is now verified inside the database
Audio is now verified not in the database
4/23/ :57 PM
AdBot: Advertisement Recognition from TV and Radio Broadcast

16. IV. Experiments and Results
4/23/ :57 PM
AdBot: Advertisement Recognition from TV and Radio Broadcast

17. IV. Experiments and Results
Single Commercial Query without noise:
Green circles represent landmarks detected that matched.
4/23/ :57 PM
AdBot: Advertisement Recognition from TV and Radio Broadcast

18. IV. Experiments and Results
Single Commercial Query with noise:
Green circles represent landmarks detected that matched.
Red circles represent landmarks detected that DON’T match.
4/23/ :57 PM
AdBot: Advertisement Recognition from TV and Radio Broadcast

19. AdBot: Advertisement Recognition from TV and Radio Broadcast
IV. Experiments and Results
Landmark Recognition With Five Levels of Additive White Gaussian Noise:
4/23/ :57 PM
AdBot: Advertisement Recognition from TV and Radio Broadcast

20. AdBot: Advertisement Recognition from TV and Radio Broadcast
IV. Experiments and Results
Multiple advertisement monitoring. Two hours of liveTV stream was recorded, analyzed and fed into the system.
Precision rate: 96.74%
Recall rate: 83.96%
Accuracy rate: 97.10%
4/23/ :57 PM
AdBot: Advertisement Recognition from TV and Radio Broadcast

21. AdBot: Advertisement Recognition from TV and Radio Broadcast
V. Conclusion
4/23/ :57 PM
AdBot: Advertisement Recognition from TV and Radio Broadcast

22. AdBot: Advertisement Recognition from TV and Radio Broadcast
V. Conclusion
It was able to successfully match single advertisements both with and without noise with those saved in the database.
It was also able to monitor multiple advertisements in a two hour recorded TV streaming. Adbot showed promising results as significant amount of landmarks managed to survive even in the presence of noise.
The combination of landmark audio fingerprinting and boundary detection using offsets is a good and efficient combination in developing an audio-based advertisement monitoring system.
4/23/ :57 PM
AdBot: Advertisement Recognition from TV and Radio Broadcast

23. AdBot: Advertisement Recognition from TV and Radio Broadcast
Thank you
End of presentation.
4/23/ :57 PM
AdBot: Advertisement Recognition from TV and Radio Broadcast