AI - part 3 - Bias

/ by
Reading Time: 3 minutes

In the third chapter of his 5-part blog post on artificial intelligence, Richard Develyn, CloudTrade CTO, talks us through learning by example and how this results in bias, whether the learning has been done by a human or by a neural network.

If you were to teach a neural network about invoices using sample documents taken mainly from obstetricians, the term “due date” would probably end up having some ambiguity about it. This could cause problems if you were to take that neural network and applied it somewhere else.

This is bias, again, which doesn’t just exist in the human brain, but rather permeates any system which has to learn by example.

Trying to remove bias seems a bit of a fool’s errand, not just in its chance of realisation but also in its ambition. Bias can be a good thing if the bias is in your favour.

Internet-based language translation applications have run into some interesting phenomena caused by the bias of the information that they have trained their neural networks from. One infamous example was reported back in 2018 (I don’t suppose it will happen now), where the word “dog” repeated nineteen times in Maori was translated by Google Translate as: “Doomsday Clock is three minutes at twelve. We are experiencing characters and a dramatic developments in the world, which indicate that we are increasingly approaching the end times and Jesus' return.”

If I ever visit New Zealand, I shall make sure to treat their dogs with respect!

The reason for this strange behaviour came down to the bias that is present in the neural network underpinning the translator. Translators such as Google Translate learn how to do their job by trawling through the internet looking for Rosetta Stones – i.e. texts which are present in more than one language. When one of those languages is rare, the predominant Rosetta Stone tends to be the Christian Bible, and it is this mighty text which ends up giving translators their distinctly “biblical” feel.

Of course, this wouldn’t be a problem if you were principally interested in translating passages from the Bible, or at least texts from biblical times. In fact, the bias in this case would work in your favour. When bias goes hand in hand with a reduction, or a refinement, in scope, you win. The neural network trained using documents from obstetricians will work well in if you carry on using it with obstetricians. Although it will probably make terrible mistakes if used elsewhere, it will perform better in its own domain than a neural network which tries to understand the whole world of invoices without having any bias – even if such a thing were possible. The problem with narrowing the scope, however, is that you also narrow the source of your learning material, which means that whilst you might gain beneficial bias in your neural network, your blinkered view of the world might also cause you to pick up some detrimental bias as well.

The problem of bias is a difficult one to solve algorithmically. Our ability to understand where there might be bias and what sort of bias that might be is really quite refined and, dare I say it, intelligent. If we were writing deterministic rules for extracting information from invoices from obstetricians, we would know about the possibility of there being more than one “due date” because we know what that’s all about, and include this bit of “significant” bias in our work. We wouldn’t, however, assume that the correct date would always be on the bottom of the page, even if that’s where we’ve always found it with our learning set, because we’d recognise that as an insignificant piece of bias.

Unfortunately, trying to work without any sort of bias at all is hard, not only because you lose the advantage that bias gives you when you narrow the scope of your problem, but also because, like Google Translate, the chances are that you haven’t got rid of your bias at all, you’ve simply lost visibility of it.

An interesting example of bias happened with a test devised by Alan Turing in 1950 as a way of determining the presence of machine intelligence. In this test, a human being is asked to converse, via some sort of chat mechanism, with a correspondent who may either be another person or who may be what is termed these days a “chatbot”. If they cannot tell the difference, then that chat bot is considered to be “intelligent”, and we can all go home, put our feet up, and allow them to take over the country.

The test was passed in 2014 by a program called Eugene Goostman, who successfully fooled his human testers by pretending to be thirteen years old and Ukrainian – i.e. juvenile, and not speaking in his first language. The testers couldn’t expect Eugene to speak perfect English, and his conversation was always going to be limited to whatever a young lad from Odessa might have experienced in his short life. Eugene went down in history as the first program to pass the Turing Test. Here was bias and limitation of scope at its artificial best.

Incidentally, Eugene was also supposed to have had a pet guinea pig and a father who was a gynaecologist. I swear I didn’t know that when I started writing this blog.