Bumgardner, Cody

CS612_S20 Course

University of Kentucky
CS612: Independent Work in Computer Science Summer 2020 : 2nd 8-week : June 11th - August 6th
Topic: AI-assisted verification of biometric data collection Student: Ryan Lindsey
Instructor: V. K. Cody Bumgardner Office: MS116a Willard Medical Science Building Lexington, Kentucky 40536-0298 Office Phone: (859) 323-7215 Office Hours: W 1pm-2pm Email: cody ’at’ uky.edu

Reading course for graduate students in computer science. May be repeated to a maximum of nine credits. Prereq: Overall standing of 3.0, and consent of instructor.

Course Instance
AI-assisted verification of biometric data collection This independent study course provides instructor guided instruction and research on topics relating to the use of AI image analysis in the auto verification of data collection from specific individuals. Image analysis related to object detection and facial identification is a mature, but active area of research. A more recently developed of image analysis called action recognition, is focused, as the name suggest, on classifying observed actions. Unlike many other areas of image analysis, action recognition might require more than one image in a know time series to infer classification. Action recognition is of great interest due to its broad applicability ranging from detecting falls in healthcare, to the recognition of an active shooter in public safety applications. Remote and distributed electronic biometric data collection is an increasingly active and necessary component of telemedicine. However, patient administered data collection is not always performed accurately, and in some cases patients and participants might be motivated to submit fraudulent results. For example, a breathalyzer attached to a car to prevent a person from driving while intoxicated, can be circumvented by another person taking the breathalyzer test. Likewise, participants in alcohol secession programs and studies can also submit fraudulent results. In many study protocols images are taken of participants while breathalyzer readings are being taken, to verify the participant. However, this image review is a manual process, which does not scale broadly. The application of action recognition and facial identification might be used in the auto verification of breathalyzer and other biometric result collections. Human biometric data collection and other guidance for this study will be provided by Mikhail Koffarnus, Ph.D., Associate Professor, Department of Family and Community Medicine. Dr. Koffarnus’s research focuses on self-directed alcohol-cessation through the use of breathalyzers connected to mobile phones with cameras.

Course Objectives
During this course the student will gain experience conducting independent supervised research in the ares of image analysis and action recognition using deep learning techniques. At the end of this course the student
should have demonstrated knowledge of distributed biometric data collection, data collection from mobile devices, and the use of AI models with mobile devices.

Course Deliverables and Timeline
During the course of the semester the student will read and provide five (1-2 page) written summary reports based on technology evaluation and related scientific literature. If agreed upon by the instructor and student, the five summary reports can be satisfied by five or more sections in a original paper developed as part of this course. In addition, the student will demonstrate a proof-of-concept tool, demonstrating the use of action recognition and facial identification in biometric data collection. • Literature Review: The student will provide reports on five papers, each report approximately two pages long presenting the main theses of each paper by the end of the course. Papers will be delivered via email to the instructor. • Mid Semester Review: On or before the 5th week of the semester, the student will present to the instructor a written Project Design report outlining high-level design components, data structures, and an implementation approach. This report will be delivered to the instructor via email and presented via online conference. In addition, any proof-of-concept components will be demonstrated and software code posted to a Git repository available to the student and instructor. • Final Project Review: Prior to the last day of the semester, August 6th, the student will demonstrate their proof-of-concept product via online conference. The project state will be evaluated based on the final project description outlined in the Project Design. The final project must implement action recognition and facial identification for both video and still image frames. Final software must be posted to the shared Git repository.

Course Materials
Course material includes, but is not limited to, the following sources: • Jeon, Byoungjun, et al. ”A Facial Recognition Mobile App for Patient Safety and Biometric Identification: Design, Development, and Validation.” JMIR mHealth and uHealth 7.4 (2019): e11472. • Mukherjee, Dhritiman, et al. ”Energy Efficient Face Recognition in Mobile-Fog Environment.” Procedia Computer Science 152 (2019): 274-281. • Nekhaev, Dmitry, Sergey Milyaev, and Ivan Laptev. ”Margin based knowledge distillation for mobile face recognition.” Twelfth International Conference on Machine Vision (ICMV 2019). Vol. 11433. International Society for Optics and Photonics, 2020. • Tran, Du, et al. ”A closer look at spatiotemporal convolutions for action recognition.” Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. 2018. • Ng, Joe Yue-Hei, et al. ”Actionflownet: Learning motion representation for action recognition.” 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 2018. • Ghadiyaram, Deepti, Du Tran, and Dhruv Mahajan. ”Large-scale weakly-supervised pre-training for video action recognition.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2019. • Korbar, Bruno, Du Tran, and Lorenzo Torresani. ”SCSampler: Sampling salient clips from video for efficient action recognition.” Proceedings of the IEEE International Conference on Computer Vision. 2019.

• Zhang, Xiao-Yu, et al. ”AdapNet: Adaptability decomposing encoder-decoder network for weakly supervised action recognition and localization.” IEEE Transactions on Neural Networks and Learning Systems (2020). • BACKtrack (breathalyzer) Developer API: https://developer.bactrack.com/documentation

Grading Policy
The course is letter-graded. The course grade will be based on effort described in the course objectives section.
Percentage Letter Grade 90 - 100% A

80 - 89% B

70 - 79% C

60 - 69% D

< 60% F

Assignments Percentage Paper Summaries and/or Report

40% Mid-Project Review 20% Final-Project Review 40% Total 100%

University Attendance Policy
Excused absences include documented illness, deaths in the family and other documented crises, call to active military duty or jury duty, religious holy days, and official University activities. These absences will be accommodated in a way that does not arbitrarily penalize students who have a valid excuse. Consideration will also be given to students whose dependent children experience serious illness.

Academic Honor Policy
Plagiarism and cheating are serious academic offenses, when in doubt contact the instructor. The university regulations pertaining to this matter can be found: http://www.uky.edu/StudentAffairs/Code/

Syllabus Change Policy
The course syllabus is the first indicator of the instructor’s expectations of students and provides a detailed description of both course content and assignments. It functions as an academic ”contract” between the student and the instructor (and, by extension, the department, college, and university). Thus, it should be clear, explicit, and complete; it should not contain imprecise, vague, or ambiguous language, and it should not be changed for the duration of the semester.

Students Registered CS612_S20

Lindsey, Ryan