AI systems fail to read clocks and decode calendars

Conducted by a team at Edinburgh University, the research investigated the capabilities of multimodal large language models (MLLMs) to answer time-related questions by looking at pictures of clocks or calendars. The AIs were tested using various clock designs, including some with Roman numerals, with and without second hands, and different coloured dials. It was found that the AI systems interpreted the correct clock-hand positions less than a quarter of the time.

Roman numerals or stylised clock hands induced more mistakes. According to the Edinburgh team, the AI systems did not perform any better when the second hand was removed, suggesting fundamental issues with hand detection and angle interpretation.

AI models were also tasked with a range of calendar-based questions, such as identifying holidays and working out past and future dates. It was found that even the best-performing AI model got date calculations wrong 20 per cent of the time. According to the researchers, the combination of spatial awareness, context and basic maths required to understand clocks and calendars is clearly a weakness of current AI models.

“Most people can tell the time and use calendars from an early age,” said study lead Rohit Saxena, from the Edinburgh University’s School of Informatics.

“Our findings highlight a significant gap in the ability of AI to carry out what are quite basic skills for people. These shortfalls must be addressed if AI systems are to be successfully integrated into time-sensitive, real-world applications, such as scheduling, automation and assistive technologies.”

The team’s work is reported in a peer-reviewed paper that will be presented at the Reasoning and Planning for Large Language Models workshop at The Thirteenth International Conference on Learning Representations (ICLR) in Singapore on 28 April 2025.

“AI research today often emphasises complex reasoning tasks, but ironically, many systems still struggle when it comes to simpler, everyday tasks,” said Aryo Gema, another researcher from Edinburgh’s School of Informatics.

“Our findings suggest it’s high time we addressed these fundamental gaps. Otherwise, integrating AI into real-world, time-sensitive applications might remain stuck at the eleventh hour.”