By : Adham Suwan Mohammed Zaza Ahmed Mafarjeh

Achieving Security through Kinect using Skeleton Analysis (ASKSA)

Outline Introduction About the Kinect Contribution State Diagram & Software Security Algorithm Demo Future Work Conclusion

ASKSA Project Our project is a hardware / software solution that provides a more accurate security system. Detects humans and differentiates the home owner from an ill-intentioned intruder. Compares the skeleton dimensions of a person to those stored in the database. Works well in all lighting situations specially in the dark. Alerts the home owner in intrusion cases and reduces the false alarms as much as possible.

Cont…

About the Kinect Motion sensing input device. Invented by Microsoft. Based around a webcam style. Users interact with the Xbox360need no touch a game controller through a (NUI). Not actually hacked, but someone wrote an open- source driver for PCs that essentially opens the USB connection and read the sensor inputs

Cont… The maximum depth range is 4 meters and the minimum range is 0.5 meter. On , Microsoft announced its official release of its SDK for non–commercial use.

Kinect Components RGB camera. Tow 3D depth sensors. Multi-array microphone that is capable of separate the voices that are in front of the device from the others sounds of the environment to use voice commands.

Cont… Use an infrared laser to project a matrix of dots and then the camera detects the distortion of each respective dot, enabling the Kinect to calculate the distance of each dot at 30 frames per second. A depth matrix produces, which is the distance of each pixel.

Kinect Features Developers use the Kinect to build interesting applications C++, C# or VB.NET. Raw sensor stream: access to low-level streams from the depth sensor. Skeletal tracking: track the skeleton image of a person moving within the Kinect field. Advance audio capabilities: supports the voice recognition technique.

Contribution Biometric Authentication: ASKSA to automatically differentiate between a known person and an unknown person based on skeletal recognition technology. Enhancement of security cameras’ utility and minimization of false alarms. ASKSA works in various lighting conditions and in the dark. ASKSA security algorithm is an efficient lookup against an in-memory biometric database. ASKSA is inexpensive compared with current security solutions.

State Diagram

Software Modules Kinect Manager Alarm Manager Authentication Manager Mail Manager Twillio Manager

Security Algorithm Kinect provides a collection of 20 joint positions, each with an x, y, and z Our system relies on the distances between the joints of the skeleton of the human 9 distances are taken into account (12 joints) All the 12 joints must be detected when authenticating, undetected joint get ∞ position It is almost impossible to match the 9 distances between two different persons  Secure Algorithm

Cont… In order for a seen person to pass authentication, the sum of the differences of the lengths from a known person must be within a specific threshold

Authentication Process Measurements for the seen person joints are taken ASKSA starts searching for the closest known person in the database If the sum of the differences of the lengths for the seen person from a known person is less than 12 cm the authentication success, otherwise the authentication fails ASKSA give the person 10 seconds to pass/fail the authentication What if the person get close/far from the Kinect ?!

Equations Used

Numerical Notes The average lengths for the 9 segments we took for ordinary people is approximately 327 cm 12/327 = 3.7%  ASKSA is able to differentiate two persons who have at least 3.7% difference in their skeletal dimensions 12 cm was taken as threshold after several experiments on many persons Kinect 30 FPS  10 sec = 300 frame If delta < epsilon in one of the 300 frames the authentication immediately success Any ∞ position joint make the authentication fails

Snapshots & Demo

Future Work Adding a new skeleton to the known person database via voice commands. Adding a filter to smooth the fluctuations of Kinect’s measurements, which would prevent false matches. Adding face recognition and more skeleton joints

Conclusion Skeletal recognition technology is predicted have a bright future. Play Station 4 uses a built-in Kinect that uses facial recognition and skeletal dimensions to differentiate players. ASKSA provides a security system, which has fewer false alarms, is more secure, and inexpensive compared with current security solutions.