OpenAI introduces CriticGPT to improve AI-generated code quality, shows 63% improvement in error detection
AI-generated code. OpenAI revealed that in testing, individuals who used CriticGPT to review code generated by ChatGPT performed better 60 per cent of the time compared to those who did not use the model. The research findings have been documented in a recently published paper.The RLHF framework involves human evaluators, known as AI trainers, providing feedback on the AI's performance to help adjust and enhance the model's behavior.
For CriticGPT, trainers added intentional errors to code samples containing natural mistakes and then provided example feedback for these errors. The model's performance was evaluated based on its ability to identify both the naturally occurring and the intentionally inserted errors.According to OpenAI, CriticGPT demonstrated a 63 per cent improvement over ChatGPT in catching code errors. However, the model has certain limitations.
It has primarily been trained on short code snippets and has not yet been tested on longer, more complex coding tasks. Additionally, the model still experiences issues with generating incorrect factual responses, a phenomenon known as hallucination. It has also not been tested in scenarios where multiple errors are dispersed throughout the code.OpenAI suggests that CriticGPT is primarily intended to enhance the company's understanding of training techniques to produce higher quality AI outputs.
Read more on livemint.com
