The rise of local AI should democratize this technology

livemint.com:

Groundbreaking announcements on Generative Artificial Intelligence (GenAI) have been making news. For instance, Google and OpenAI introduced new GenAI-powered assistants that can engage in real-time conversations, even adapting when interrupted, mirroring human interaction. These assistants don’t just converse; they can also analyse your surroundings through a live video feed and translate conversations on the spot.

Google announced new tools at its I/O Conference, including enhancements to its bewildering array of products under Gemini AI, to compete with OpenAI’s new ChatGPT 4o announced the day before. Google also announced that it is building Gemini into a “do everything" model across almost all its product suites. For its part, OpenAI’s conversational ChatGPT 4o model can supposedly respond with a lag time of 320 milliseconds, which is about the same as human speech.

With humour, sarcasm and more, its responses are remarkably human-like. However, the immense computational power and energy required to train and deploy these large language model (LLM)- based systems raises concerns about their sustainability. LLMs are designed to understand and generate human language by processing vast amounts of data, literally every scrap available online.

They leverage specific learning techniques, such as transformer architectures, to create sophisticated models that can perform various language-related tasks. The most prominent examples, such as OpenAI’s and Google’s, have demonstrated remarkable proficiency in functions like text and image generation, summarization and now conversational AI. The primary advantage of LLMs lies in their ability to generate coherent and contextually relevant text.