FAIR presented the Casual Conversations dataset, consisting of 45,186 videos with people of different ages, genders, and skin colors. The dataset will allow developers to evaluate the uniformity of recognition of these parameters by computer vision models in different subgroups of people.
Casual Conversations is the first public dataset with participants who have specified their own age and gender. In previous datasets, this data was specified by third parties or predicted using machine learning models. The dataset solves the problem of biased attitudes towards people based on erroneous predictions of their age and gender. Also, for each video, light conditions and skin color are set according to the Fitzpatrick scale (see Figure), which will allow you to analyze how artificial intelligence systems determine skin color in different light conditions. Today, most models are less accurate at recognizing certain subgroups of people due to the fact that training datasets do not fully take into account possible skin tones. This can lead to potentially harmful consequences for individuals and groups. In particular, some decision-making algorithms in healthcare, due to recognition errors, unfairly deprive people of the opportunity to receive the necessary treatment.
The dataset consists of 45,186 videos with 3,011 participants: 15 videos with each participant. FAIR allows its use only for evaluating existing models, and not for training new ones that determine gender, age, and skin color. Participants could specify their gender as “male”, “female”, and “other”. Over the next year, FAIR plans to expand the dataset to become more inclusive and include a wider range of participants ‘ ages, geographical location, type of activity, and other characteristics.
The dataset is proposed to be used as an additional tool for evaluating the effectiveness of computer vision and audio models in addition to standard recognition accuracy tests. It is designed to detect cases where the recognition performance is heterogeneous and depends on age, gender, skin color, and ambient light conditions. Another important application of the new dataset is the definition of deepfakes – a rapidly growing problem in the field of media forensics. The dataset is part of Facebook’s long-term initiative to take a responsible approach to creating machine learning technologies.
The dataset is available here.