Most biometric systems employed for human recognition require physical contact with, or close proximity to, a cooperative subject. Far more challenging is the ability to reliably recognize individuals at a distance, when viewed from an arbitrary angle under real-world environmental conditions. Gait and face data are the two biometrics that can be most easily captured from a distance using a video camera. This comprehensive and logically organized text/reference addresses the fundamental problems associated with gait and face-based human recognition, from color and infrared video data that are acquired from a distance. It examines both model-free and model-based approaches to gait-based human recognition, including newly developed techniques where the both the model and the data (obtained from multiple cameras) are in 3D. In addition, the work considers new video-based techniques for face profile recognition, and for the super-resolution of facial imagery obtained at different angles. Finally, the book investigates integrated systems that detect and fuse both gait and face biometrics from video data. Topics and features: discusses a framework for human gait analysis based on Gait Energy Image, a spatio-temporal gait representation; evaluates the discriminating power of model-based gait features using Bayesian statistical analysis; examines methods for human recognition using 3D gait biometrics, and for moving-human detection using both color and thermal image sequences; describes approaches for the integration face profile and gait biometrics, and for super-resolution of frontal and side-view face images; introduces an objective non-reference quality evaluation algorithm for super-resolved images; presents performance comparisons between different biometrics and different fusion methods for integrating gait and super-resolved face from video. This unique and authoritative text is an invaluable resource for researchers and graduate students of computer vision, pattern recognition and biometrics. The book will also be of great interest to professional engineers of biometric systems.

The recognition of humans and their activities from video sequences is currently a very active area of research because of its applications in video surveillance, design of realistic entertainment systems, multimedia communications, and medical diagnosis. In this lecture, we discuss the use of face and gait signatures for human identification and recognition of human activities from video sequences. We survey existing work and describe some of the more well-known methods in these areas. We also describe our own research and outline future possibilities. In the area of face recognition, we start with the traditional methods for image-based analysis and then describe some of the more recent developments related to the use of video sequences, 3D models, and techniques for representing variations of illumination. We note that the main challenge facing researchers in this area is the development of recognition strategies that are robust to changes due to pose, illumination, disguise, and aging. Gait recognition is a more recent area of research in video understanding, although it has been studied for a long time in psychophysics and kinesiology. The goal for video scientists working in this area is to automatically extract the parameters for representation of human gait. We describe some of the techniques that have been developed for this purpose, most of which are appearance based. We also highlight the challenges involved in dealing with changes in viewpoint and propose methods based on image synthesis, visual hull, and 3D models. In the domain of human activity recognition, we present an extensive survey of various methods that have been developed in different disciplines like artificial intelligence, image processing, pattern recognition, and computer vision. We then outline our method for modeling complex activities using 2D and 3D deformable shape theory. The wide application of automatic human identification and activity recognition methods will require the fusion of different modalities like face and gait, dealing with the problems of pose and illumination variations, and accurate computation of 3D models. The last chapter of this lecture deals with these areas of future research.

In recent years, off-the-shelf cameras became vastly available, producing a huge amount of content that can be used in various applications. Among the applications, visual surveillance receives a great deal of interest. In visual surveillance, for the purpose of advanced security and monitoring system, human recognition at a distance is of importance. Interest in finding specific individual becomes important such as the case in locating missing person and identification of terrorist from recording of video surveillance in public places. Many methods published so far for human recognition in video image analysis focused on face and gait. However, these methods have low recognition performance due to some external factors such as face are prone to fake facial images and gait is susceptible to occlusion. The presence of the occlusion introduces errors into many existing vision algorithms which have yet to be resolved. In such situations vision techniques to identify a human fails because descriptors of part of the human shape may not have any resemblance with the descriptors of the entire human shape. To resolve this problem, an intelligent system for human recognition under partial occlusion is proposed in this study. The hypothesis of the study is that special features of human body shape can be used to identify a person identity from a distance. Each person has different body features characteristics and hence recognizable using these special body shape features. The body shape features used are the head, shoulder, and trunk. The features can be recognized using fuzzy logic (FL) approach and used as inputs to a recognition system based on a multilayer neural network (MNN). For the developed human recognition algorithm, database has been implemented to provide efficiency in displaying, storing, and retrieval of data. Experimental results show that the developed human recognition system is capable of detecting and recognizing human subject from a distance with 98.1% and 77.5% accuracy respectively. For successful identity recognition, the average percentage coverage of occlusion for Single Direction Partial Occlusion Test (SDPOT) from the top, bottom, left, and right of specific human subjects are 0.41%, 10.41%, 8.44%, and 12.27% respectively. Whereas for Multiple Direction Partial Occlusion Test (MDPOT), an average of 40.15% coverage of occlusions is achieved with successful recognition while the lowest is found to be 11.92% and highest is 60.92%.

This book constitutes the refereed proceedings of the Third International Conference on Audio- and Video-Based Biometric Person Authentication, AVBPA 2001, held in Halmstad, Sweden in June 2001.The 51 revised papers presented together with three invited papers were carefully reviewed and selected for inclusion in the book. The papers are organized in topical sections on face as biometrics; face image processing; speech as biometrics and speech processing; fingerprints as biometrics; gait as biometrics; and hand, signature, and iris as biometrics.

We have compiled a database of images and videos that includes images and videos of approximately 300 human participants. Duplicate sets of images and videos taken from 1 week to 6 months after the first set are available for approximately 200 of these people. The images include 9 standard "mug shot poses". The videos include head rotations, dynamic facial expressions, facial speech clips, and 3 whole-body gait clips. Experiments have been completed comparing the effects of several types of facial motion on face recognition, the effects of face familiarity on recognition from video clips taken at a distance, and on the effects of attention on recognition of moving faces. The results of these studies provide insight into the way motion can facilitate or interfere with the encoding of the invariant face and body features that support recognition.