Why is AI such a black box?
And what can be done to fix this for startup founders?
Artificial Intelligence, or AI, has become a mainstream term in recent years, a commonplace concept that companies and consumers alike talk about and use in their day-to-day lives. With the acceleration of computing power and the release of deep-learning technologies such as GPT-3, AI has become less science-fiction and more everyday reality, as we all interact with apps, games and software widgets that rely on more and more complex coding architectures.
Interested in cracking open the AI black box and learning about what Explainable AI is?
Get in touch with the founded.ai team.
Perhaps the biggest acceleration of AI usage has been witnessed since 2020 – in part, because of technological advances driven by the tech giants of the world, but also due to societal demands, namely the COVID pandemic.
In fact, in February 2020, researchers in the US used deep neural networks to look at X-rays and CT scans in order to differentiate between people with COVID-based pneumonia and people without. It was a groundbreaking use of AI, but it didn’t come without its issues. In their haste to get the AI working, researchers relied on a limited dataset, which unfortunately meant that the AI started to make assumptions that led to incorrect diagnoses. If the team had had more data, and subsequently more time, then the AI would have been more accurate and refined, with a lower probability of incorrect pattern recognition.
COVID aside, AI is still making leaps and bounds in everyday life – from the use of face ID to unlock your smartphone, to fraud detection and risk management in digital banking, AI is only set to become more and more capable and therefore more prolific.
But with AI powering so many of today’s applications, why is it still such a black box that very few understand?
And surely with it becoming more and more mainstream, shouldn’t it also become more accessible?
Because aren’t we missing out on a huge opportunity here for innovative thinkers and pioneering startups to take advantage of AI in order to solve some of the world’s biggest problems?
Why is AI a black box?
The AI black box problem is not new and it’s actually quite easily explained. It comes down to the use of artificial neural networks and deep learning in AI applications. Two concepts that are often shrouded in mystery due to complex, inaccessible language.
Why not talk to the founded.ai team about your next AI project? We promise we don’t use complex, inaccessible language!
If we were to break it down into its basic components, an artificial neural network is fundamentally a network of nodes or connection points. Each of these nodes (think of them as electricity junction boxes) process information and pass it on to the next node. The network is a mass of these nodes all connected together on different layers; deep learning takes place because the network – a bit like a human brain – is learning on its own to recognise patterns.
It’s easy to see how this can get complicated as you add more and more nodes in. Furthermore, we don’t always see what or how the nodes have actually learned, we just see the outputs. Again like the human brain – we see the output of it making your finger move or your toe point, but we don’t know how this has happened from all the connection points in your mind and body.
As Alan Winfield, a robot ethicist at the University of the West of England Bristol explains, “It’s very difficult to find out why [a neural network] made a particular decision.”
Speaking about the time when Google’s AlphaGo neural net played human champion Lee Sedol in Seoul in 2016, and the AI made a move that flummoxed everyone watching, Winfield expanded, “We still can’t explain it. […] It doesn’t matter in the game of go, but imagine the autopilot of a driverless car. If there’s a serious accident, it’s simply not acceptable to say to an investigator or a judge, ‘We just don’t understand why the car did that.’”
Which doesn’t help with the whole AI black box issue…
AI and human trust
As Winfield says, playing go is one thing, but when AI starts to spread to more and more of our everyday tools, it’s easy to see why it’s met with such scepticism. It doesn’t even have to be self-driving cars…
What about digital banking processes relying on AI to decide if you’ll get that mortgage or not?
Or medical applications using AI to diagnose a terminal illness?
This raises very serious ethical concerns about the whole AI black box problem because although humans make mistakes probably even more often than a fully trained AI, the potential scale of application that AI has and the speed at which it can work can seem frightening – and how can mistakes be fixed if we don’t understand how they’ve happened in the first place?
There’s no getting past it – AI needs to be more transparent and open for people to be comfortable with it and to trust in it.
Thankfully, there is a solution…
What is Explainable AI?
IBM provide a very clear and comprehensible answer to the question of what is Explainable AI:
“Explainable artificial intelligence (XAI) is a set of processes and methods that allows human users to comprehend and trust the results and output created by machine learning algorithms. Explainable AI is used to describe an AI model, its expected impact and potential biases.”
It also goes on to highlight the main benefit of XAI in the context of trust and the AI black box problem:
“Explainable AI is crucial for an organisation in building trust and confidence when putting AI models into production. AI explainability also helps an organisation adopt a responsible approach to AI development.”
Given what we’ve said about the need for AI to be more accessible and understandable for it to be fully accepted by humans the world over, XAI appears to be a meaningful step in the right direction.
Key global projects will only serve to support this. For example, the EU-funded Cordis project reinforces this concept of transparency:
“The EU-funded XAI project aims to produce meaningful explanations for AI/ML systems. The research focuses on how to design transparency in ML models, how to produce controlled black-box explanations, how to reveal used data and algorithms, unfairness and causal relationships in processes. The project will also formulate ethical and legal standards for AI.”
And the OpenAI manifesto puts safe and beneficial AI at the core of its mission:
“OpenAI’s mission is to ensure that artificial general intelligence (AGI)—by which we mean highly autonomous systems that outperform humans at most economically valuable work—benefits all of humanity. We will attempt to directly build safe and beneficial AGI, but will also consider our mission fulfilled if our work aids others to achieve this outcome.”
Where XAI really starts to take shape though is in the new products it powers and how accessible they are to a wider audience.
Take Google Cloud’s Explainable AI tools and products.
- BigQuery allows a wider pool of data analysts to create machine learning models in SQL.
- The What-If Tool lets business users test out hypothetical scenarios, for example, in financial forecasting or demographic modelling.
- The Language Interpretability Tool helps researchers and medical practitioners understand NLP model behaviour.
Then there are products launched recently by OpenAI and Meta.
January 2021 saw the introduction of DALL-E, an image-generating system that uses a neural network to transform natural language text into images. 2022 has seen Meta launch Make-A-Video, an AI system that generates video from text. With both of these, the key is opening up AI to an even wider pool of business users to access and use AI at a very low cost.
Now, the comeback on all of the above is that they still only address a narrow audience of business users. Many would argue that expanding data science from the engineers to the analysts isn’t a huge step, or opening up NLP models from researchers to practitioners doesn’t scream great advancement.
But the key with this progress is two-fold:
- The models are all open-source. The code isn’t hidden behind a company firewall that people can’t access and interrogate. All of the products and websites mentioned above contain comprehensive documentation on how the product has been built, and with some you can even access the code directly from the public site.
- It’s a small step in the right direction! It will still take time for Explainable AI to reach more of a mass-accessibility point, but it’s got to start somewhere.
Why Explainable AI is important for startup founders
Which of course leads us on to why Explainable AI (XAI) is important for startup founders, and why this ‘glass box’ AI as we like to call it here at founded.ai, will pave the way for innovative thinkers and makers to crack some of the bigger global challenges.
First and foremost, XAI means you no longer have to have a PhD in Machine Learning to develop an AI application.
Secondly, XAI enables startups to achieve Enterprise-grade software development processes and outcomes without the need for huge budgets or massive tech teams.
Lastly, XAI lets ambitious startup founders develop solutions for some of the bigger problems that society currently faces – namely in healthcare and climate change.
Take a look at Sense.ly as an example. Their AI-based virtual nurse, Molly, helps with prolonged patient monitoring, gathering feedback and symptoms from an individual by using NLP algorithms.
Or Artelus, an AI-powered solution that can detect diabetic retinopathy (DR) with high accuracy in under three minutes, which is faster than a human professional.
In the field of climate change, there’s Blue Sky Analytics, a geospatial intelligence platform that harnesses satellite data for environmental monitoring and climate risk assessment.
Additionally, there’s ASTERRA, who have developed AI models that analyse satellite images and save water with GPU-accelerated leak detection technology.
In actual fact, you don’t have to go far to find pioneering startups from the likes of the US, India and Israel who are taking on some of the big world challenges. And they all share the same origin of developing AI from a more accessible and beneficial starting point. Yes, AI has some way to go before it’s fully embraced and trusted by the masses, but initiatives such as Explainable AI are the baby steps needed to make this eventually happen.
It’s a mission we also share here at founded.ai. Our prebuilt AI engines are deliberately designed to be more accessible and quicker to set up for startup founders.
If you’d like to discuss the ins and outs of XAI in more detail, or are interested in exploring how pre-built AI engines can get you launched more quickly, just get in touch.