Mint Primer: Will decoding AI ‘black boxes’ make them safe?
Artificial intelligence (AI) and generative AI (GenAI) are advancing at an incredible pace, but we are unable to understand how they make decisions, so they are called ‘black boxes’. Researchers say they are now able to peek under the AI hood. Will this make AI models safer? AI algorithms, especially those involving complex machine learning (ML) models like deep neural networks, are modelled on the human brain.
They receive inputs and send outputs layer by layer, until the final output. But their internal decision-making processes are complex, opaque, and difficult to interpret and so are called ‘black boxes’. For instance, if a self-driving car hits a pedestrian instead of applying the brakes, it’s challenging to trace the system’s thought process to find out why it made this decision.
This has significant implications for trust, transparency, accountability, bias, and fixing errors in such models. The process involves improving model transparency, auditing model decisions, and introducing regulatory measures to enforce explainability, along with ongoing research and community collaboration to advance the field of explainable AI (XAI). It is focused on developing methods to make AI more interpretable with the help of researchers, ethicists, legal experts, and specialists.
Google, Microsoft, IBM, OpenAI, and credit scoring service Fair Isaac Corp are developing XAI techniques, while governments in the EU, the US, etc. are actively promoting and regulating its ethical and transparent use. Last October, Anthropic, an AI startup, said it was successful in breaking neural networks into parts that humans can understand, by applying a technique called ‘dictionary learning’ to a very small “toy" language model, and decomposing
. Read more on livemint.com
