Context:
According to a study published in BMJ, state-of-the-art AI models, namely some LLM models such as ChatGPT, Claude, and Gemini, showed sustained cognitive decline with the passage of time just like human aging. The researchers evaluated the cognition of these models using the Mochata Cognitive Assessment (MoCA) test.
Key Findings:
- The older versions were scoring lower as each model was being put through the test in a manner that was virtually reminiscent of human cognitive decline. Gemini 1.0 had the worst score with sixteen points, while ChatGPT 4 scored twenty-six.
- MoCA Test-MoCA test, which is ordinarily applied to humans, has been adapted to test AI cognitive functions like those of attention, memory, and executive function.
- Chatbot Limitations
- The AI chatbots were impaired on tests relating to e.g., visuospatial skills and executive functions. None of the bots scored full marks of 30, with most scoring below the cut-off of 26 points, suggestive of a mild cognitive impairment.
- Medical Implications
- The cognitive deficits of AI models can seriously question their reliability for medical diagnostics, and claim that AI will soon take human doctors’ place.
The study demonstrates AI models’ cognitive decline with time and draws attention to their role in medical diagnostics. While AI may be useful, its cognitive impairments may ensure that it is indeed never able to replace human expertise in applied areas of neurology.
