CS Lab Usage Logan Balow

Requirements The Camera activates based on motion Capture snippets of video System recognizes people compared to motion Collect usage times and numbers Develop Admin side that reports statistics and maintains video Implement a camera based system that monitors the CS Lab and usage Using the Foscam-fi8910w IP Camera WPF C# application in Visual Studios 2017 Detect motion from previous frames Capture video based on motion or user Person detection algorithm Collect and report the data

Image Data Flow Wireless Router Camera is wireless (minus the power) Allows you to connect remotely using its IP web address School firewall prevents p2P access so both the computer and camera have to be on the same network to access currently Program connects to the camera using a username and password, then begins querying and collecting the frames from the camera Rate at about 30 per second at 640x480 res

Haar Cascade Structures Method proposed by Paul Viola and Michael Jones Machine learning approach using positive and negative images Utilizes integral image Reduces any sized image to an operation involving just four pixels at a time Starts with Eyes, then nose/bridge of nose area etc. Must pass all “Stages” to be accepted as a face - effective object detection method proposed by Paul Viola and Michael Jones  Initially, the algorithm needs a lot of positive images (images of faces) and negative images (images without faces) to train the classifier. Each feature is a single value obtained by subtracting sum of pixels under the white rectangle from sum of pixels under the black rectangle. Not 24x24 = 160000 features To solve this, they introduced the integral image. However large your image, it reduces the calculations for a given pixel to an operation involving just four pixels  most of them are irrelevant  Pass through each stage and throws out the image if a portion doesn’t meet the stage. Must pass all stages in order to be accepted as a face

Facial Detection BitmapSouce Bitmap EMGU Image BitmapSource to Bitmap to EMGU Image, then greyscale, to face detection (rectangles) EMGU Image

Video Recording List<Images> EMGU Video Writer User Can Record based on User interaction or motion recording Motion tracking must be on first User Stops Recording EMGU Video Writer

Spreadsheet Generation Select if you would like a report by hour or by day C# has using Microsoft.Office.Interop; library Allows you to dynamically generate and save excel files

Difficulties Using EMGU_CV means no access to altering the algorithm for facial detection People must be perpendicular to the camera No depth perception Counting people only once Location Bottleneck of the lab entrance Converting the frame from the camera to EMGU_CV Image class CPU Usage EMGU DOCUMENTATION Lack of it for newer items Detection is not a perfect science No way to tune the facial detection Limited to how it works. Must Be perpendicular to camera Flat 2D image no depth Location of the lab allows for stacking up of people and may not get everyone Converting the frame from BitmapSource to EMGU_CV library CPU Usage initially 80%-90% idle Brought it down to 10% with motion detection on with spikes to 50%

DEMO TIME!

Strategies - Pretty waterfall type approach initially pretty even flow

Plan was straightforward kind of a waterfall approach where you just work down the line and test as you go Developed into this mess that you see Lots of testing and going back to the drawing board Past 3 weeks have been testing recording human data Count people once and only once Dr Pankratz Pointed me to EGMU Also gave me Kinect book which helped

Classes That Helped CSCI 110: Eased me into coding got me more interested CSCI 205: Opened me to OOP CSCI 220: Advanced knowledge of Files and data structures CSCI 350: Event programming knowledge 110: cause intro 205: Elementary Data structures 220: Advanced File and Data 350: Event Programming

Where to Go From Here Firewall Problems Allow for remote viewing and access to data Smart Home extensions Lights, Doors, Temperature, Pet interaction Machine learning project for personal person detection Gestures with person detection Kinect Camera or other Firewall Currently no P2P access within the snc network unless IT opens certain ports for a short time Perhaps find a way around this Will allow for remote viewing and accessing the data from anywhere Further the data collected and improve administrative side Smar Home extensions Smart home items such as lights, doors, temperatures Pet interractions, such as viewing, talking to, feeding remotely Machine learning project Develop a similar EMGU_CV process for detecting a person from an image or depth image Explore this project with a different camera and different extensions Kinect and gestures

Questions? Questions? Comments? Stand-up comedy?