The role of historical data in perpetuating biases in AI algorithms

A loving and caring mother gave £2 every week to her elder son to buy candies for his younger sister and himself.

The elder child, “I”, was bossy and always acted privileged; while the younger, “U”, was gentle and a bit scared of the elder.

“I” loved the white candy whereas “U” preferred the rainbow coloured one.

“I”, as the elder, decided he should have more than his sister. At the shop, he ordered 3 white candies and a rainbow candy. “U” felt dejected but due to fear didn’t say anything to her mother.

The shop keeper took the 2 pounds from “I” and sold him 3 white candies and a white one.

This went on for a few years.

One day, the shop keeper installed a vending machine where kids could insert coins and select the candies. The machine also kept track of the number of candies being ordered every day. This helped him to refill the machine with the most popular candies.

The kids were excited with the new machine and “U” thought that now she stood a fair chance to order her own sweets. Unfortunately, that did not happen. “I”, being the self-established custodian of the money from their mum, continued to follow the same pattern.

Years went by…

The machine got better with new features. It started collating past usage data in a database. It also sent daily analysis to the shop keeper so he could keep inventory and order candies online based on demand.

The children grew up. “U” became more aware of her freedom and consciousness of equitability. Now she didn’t have to depend on her brother to buy her candies. But where could she could go to buy her candies? The shop keeper didn’t order enough rainbow candies. Based on the trend from the database, he always ordered more of the white candies.

Next year, the owners of the vending machine announced with much fanfare that they had leveraged state of the art Artificial Intelligence by utilising the historical data from the machine. The entire supply chain behind the ordering and delivery of candies was now automated. AI constantly read data from the machine, calculated the reorder point and even placed orders directly without any manual intervention. Orders were received by the candy factory and were refilled..

But……………………………………………..

 There were still 3 white candies and 1 rainbow candy……………………………

For centuries scissors were designed by right hand people..it took someone to recognise that bias against left handers” – Fei Fei Li, Chief scientist for AI, Google Cloud Computing

Machine Learning algorithms haven’t been optimised for any definition of fairness. They have been optimised to do a task” – Deirore Mulligan, Associate Professor, UC Berkeley School of Information

Leave a comment