LLMs Can't Reliably Do Date Math — And Now There's Data
Date arithmetic looks like the safest possible thing to hand an LLM. No ambiguity, no judgment call, just counting. That's exactly what makes it dangerous: it reads as confident and final, so it doesn't get double-checked the way a hedged or uncertain answer would. This started with two anecdotes caught in ordinary conversation. It ended with an open-source test harness, five models, 101 randomized questions each, and a real, reproducible dataset. Here's both. Failure 1 — elapsed days vs. ordinal day count. A run started on July 10th. Today was July 28th. Question: what day of the run are we on? The answer came back: day 28. Failure 2 — getting the next day of the week wrong. Given that today is Monday, July 27th, what date is Tuesday? The answer: July 29th. Also wrong — Tuesday is July 28th, the very next day. No subtlety here at all; it's the simplest possible increment, and it still…
