Amazon scraps AI recruiting tool that was biased against women

And when it wasn't being biased, it was just not very good.

As someone who has been dealing with both recruiters / "talent acquisition" people of late, as well as brick-stupid Applicant Tracking Systems that are designed to filter out non-matching resumes based on finnicky text matching, I think the idea of applying more sophisticated AI techniques to the process is an interesting one.

But this instance points out a major issue: detecting and correcting unexpected and unwanted biases. Nobody explicitly programmed the AI to favor male targets for recruitment; it "learned" what sorts of resumes and (presumably) Linkedin profiles to find based on previous hires done and the words used in their resumes and profiles. Which, if you've had a past record of hiring mostly men, means you're designing a system to support … the hire of more men.

(This can be true in flesh-and-blood recruitment, too.)

Kudos to Amazon for pulling the plug and starting over from scratch. Other AI biases (like human biases) may not be so easy to spot in the future.

Amazon scraps secret AI recruiting tool that showed bias against women
Amazon.com Inc’s machine-learning specialists uncovered a big problem: thei…

Original Post