As ChatGPT, Dall-E and many other AI algorithms have crept into our lives, awareness of their potential consequences has grown. In the first place of practice (e.g. can some algorithm take my job?), but secondly of ethics. And in this sense, more and more researchers are asking whether it is possible to develop algorithms capable of evasion most prejudices which we human beings have. In other words: can we create an AI that is just, charitable and empathetic… or are we doomed to repeat our mistakes over and over again?

Based on this assumption, two researchers from MIT, who in their paper “A Framework for Understanding Sources of Harm through the Machine Learning Life Cycle” identify six biases that are usually found in the data with which the models are trained. machine learningand this can lead them to repeat undesirable patterns of behavior such as racism, under-representation of diversity, exclusion on socio-economic grounds or cultural uniformity. Harini Suresch and John Guttag, authors of this study, identify the following biases that data scientists and AI professionals should avoid.

historical bias

An AI model is said to produce historical bias when, despite being trained with measurable and contrasting data sets, it produces incorrect results when it interprets the present in terms that may belong to the past.

This type of bias causes some AI models to reproduce stereotypes that are harmful to a segment of the population and that do not correspond to reality. In him paper For example, published by MIT, it explains how a financial institution that uses an AI algorithm to predict its customers’ ability to pay could “learn” that it’s better not to make loans to African-Americans based solely on the data (and stereotypes) the algorithm was trained with.

representation bias

Representation bias occurs when the data with which the model was trained underrepresents a portion of the population and, as a result, is unable to offer a broad and diverse picture of society.

An example of the above is ImageNet, a huge database containing millions of tagged images that is commonly used for AI algorithms to be able to recognize objects. Well, as they point out at MIT, about half of those images were taken in the United States and other Western countries, and only between 1% and 2% of the images represent cultures like Asia, various Africans, and others.

As a result, if we asked ImageNet for a wedding dress, for example, it would perfectly recognize a “Western bride” (white) dress, but it would have enormous difficulty recognizing ceremonial costumes used in weddings in South Korea or Nigeria.

Measurement bias

Measurement bias occurs when selecting, collecting, or computing features and labels to be used in the prediction process. It normally occurs when you want to analyze a characteristic or idea that is not directly observable.

For example, a person’s “creditworthiness” is an abstract concept that is often operationalized using a proxy that can be measured, such as a credit score. In this case, we can find measurement bias when the algorithm includes indirect indicators that do not adequately reflect the different groups that are analyzed.

In him paper The US judicial system’s use of the controversial COMPAS algorithm is considered a paradigmatic case of this bias. Allegedly, this algorithm predicts the likelihood that a defendant will commit a crime again, and its data is used by judges and court officials, for example, to decide whether an arrested person should remain in pretrial detention (and for how long) or whether, later, he You can grant probation. However, and given that the prison population in the United States is predominantly African-American and Latino, the algorithm estimates not so much the likelihood of future crime as of future arrest, introducing higher risk variables associated with race.

Aggregation bias

Aggregation bias arises when a single model is used for data where there are underlying groups or types of data that should be treated differently. Aggregation bias is based on the assumption that the correspondence between items and labels is consistent across all subsets of the data, when in fact it often is not.

This type of bias leads to a model that ends up being suboptimal for any of the groups it claims to represent, or in many cases ends up only representing the dominant group. This bias often occurs when data from social networks as a whole (such as a Twitter hashtag) is taken without taking into account that these are spaces in which “very different cultures and socio-demographic spaces coexist”.

Assessment bias

This bias usually occurs when the reference data used for a particular target do not adequately represent the target population that should be used for the intended target.

One of the most paradigmatic cases of evaluation bias can be found in various face recognition algorithms. As the MIT researchers point out in this case, some of these algorithms, which clearly have a commercial purpose, have no problem identifying white men, but have much more trouble performing analysis on images that feature dark-skinned women, which is confusing . is with other things. Because? Often due to the underrepresentation of these people in the data used for their training.

deployment bias

Implementation or implementation bias occurs when there is a mismatch between the problem the model is intended to solve and the way it is used in practice.

This often occurs when a system is built and evaluated as if it were completely autonomous, when in fact it operates within a complicated socio-technical system moderated by institutional structures and human managers. Thus, systems produce results that must first be interpreted by human managers and then lead to decision making. Despite their good performance in isolation, they can eventually cause harmful consequences due to phenomena such as automation or confirmation bias.

In short, if we believe that AI will promote the development of a fairer and more egalitarian society, we need to start from the ground up, identifying biases and preconceived notions to start from before implementing a training model.