A curious Naked Security reader alerted us to what they thought might be a “Y2K-like bug” in Java’s date handling.
The cause of the alarm was a Twitter thread that started with a headline tweet saying, “PSA: TIS THE SEASON TO CHECK YOUR FORMATTERS, PEOPLE.”
https://twitter.com/NmVAson/status/1207820284268597249As @NmVAson points out, the problem comes when you ask the Java DateTimeFormatter library to tell you the current YYYY, a commonly-used programmers’ abbreviation meaning “the year expressed as four digits”.
For example, when programmers abbreviate the world’s commonly used date formats, they often use a format string to denote the layout they want, something like this:
Layout Format string Example ------------------------ ------------- ---------- US style (Dec 29, 2019) MM/DD/YYYY 12/29/2019 Euro style (29 Dec 2019) DD/MM/YYYY 29/12/2019 RFC 3339 (2019-12-29) YYYY-MM-DD 2019-12-29
In fact, many programming languages provide code libraries that help you to print out dates using format strings like those above, so that you can automatically adapt the output of your software to suit each user’s personal preference.
The problem here is that there are many different date-handling functions, such as GetDateFormatEx() on Windows, strftime() on Unix and Linux systems, all the way way to Java’s all-singing, all-dancing DateTimeFormatter module.
The abovementioned Java library, amongst others, lets you conveniently format dates with the three strings shown above, giving you plausible results like this:
import java.time.LocalDate;
import java.time.LocalDateTime;
import java.time.format.DateTimeFormatter;
public class CarefulWithThatDateEugene {
private static void tryit(int Y, int M, int D, String pat) {
DateTimeFormatter fmt = DateTimeFormatter.ofPattern(pat);
LocalDate dat = LocalDate.of(Y,M,D);
String str = fmt.format(dat);
System.out.printf("Y=%04d M=%02d D=%02d " +
"formatted with " +
"\"%s\" -> %s\n",Y,M,D,pat,str);
}
public static void main(String[] args){
tryit(2020,01,20,"MM/DD/YYYY");
tryit(2020,01,21,"DD/MM/YYYY");
tryit(2020,01,22,"YYYY-MM-DD");
}
}
//---------------
Y=2020 M=01 D=20 formatted with "MM/DD/YYYY" -> 01/20/2020
Y=2020 M=01 D=21 formatted with "DD/MM/YYYY" -> 21/01/2020
Y=2020 M=01 D=22 formatted with "YYYY-MM-DD" -> 2020-01-22
So far, so good!
But if you try this in the middle of the year, you get:
Y=2020 M=05 D=17 formatted with "MM/DD/YYYY" -> 05/138/2020 Y=2020 M=05 D=18 formatted with "DD/MM/YYYY" -> 139/05/2020 Y=2020 M=05 D=19 formatted with "YYYY-MM-DD" -> 2020-05-140
An easily spotted bug
What?!?
Note the weird day numbers that are way greater than 31, even though the longest months in the year only have 31 days.
This should get you scrambling back to the documentation, or at least to your favourite search engine, where a cursory glance will reveal that the abbreviation DD actually means day of the year rather than day of the month.
So DD and dd only produce the same answer in January, after which the day of the year goes to 32 while the day of the month resets to 01 for the first of February. (To be clear, on New Year’ Eve – the last day of the year, 31 December – the day of the year is 365, or 366 in a leap year, while the day of the month is 31.)
In other words, even cursory testing of dates outside January will show up this format-string bug, so not many people make it.
What you want is the format string dd, as follows:
Y=2020 M=05 D=17 formatted with "MM/dd/YYYY" -> 05/17/2020 Y=2020 M=05 D=18 formatted with "dd/MM/YYYY" -> 18/05/2020 Y=2020 M=05 D=19 formatted with "YYYY-MM-dd" -> 2020-05-19
A bug that’s harder to spot
Except that you’re still wrong, because YYYY doesn’t mean “the Christian-era year in four digits”.
That’s denoted, in the Java library (and other full-fat date libraries, too) as the lower-case text string yyyy.
In contrast, YYYY denotes what’s known as the week-based year, something that accountants rely on to avoid splitting weeks – and thus the company payroll – between two different years.
The week-based year number and the Christian-era year number are almost always the same, so it’s easy to look at a few outputs from Java’s DateTimeFormatter module and assume that they are always the same…
…but you’d be dangerously wrong.
Inconveniently for farmers, priests, astronomers and businesspeople alike, the solar year doesn’t divide exactly into days, and therefore doesn’t divide neatly into weeks or months. (The lunar month isn’t commensurate with the solar year, either, to make things even more complicated.)
As every bookkeeper knows, there aren’t exactly 52 weeks in a year, because there are always one or two days left over at the end.
That’s a consequence of the fact that there are 365 (or 366) days in a year (or a leap year); that there are 7 days in a week; and that 365/7 = 52 remainder 1 (or 366/7 = 52 remainder 2).
For accounting convenience, therefore, it’s conventional to treat some years as having 52 full weeks, and others as having 53 weeks, which keeps things like weekly revenue plans and weekly payroll evened out in the long run.
In other words, in some years, “payroll week 01” actually starts just before New Year’s Day; in other years, it doesn’t start until a few days into the first week of the New Year.
There’s a standard for that
There’s a standard for that, defined in the ISO-8601 calendar system, described in Java’s documentation as “the modern civil calendar system used today in most of the world.”
ISO-8601 makes some assumptions, including that:
- The first day of every week is Monday.
- If a week is split at the end of the year then it is assigned to the year in which more that half of the days of that week occur.
The second assumption seems reasonable, because it means that you always have more payroll days in the correct year than in the wrong one.
For 2015, for example, the there were four days left over after week 52, so the first three days of 2016 were “sucked back” into the payroll year 2015:
Sun 2015-12-27 -> Payroll week 52 of 2015 Mon 2015-12-28 -> Payroll week 53 of 2015 Tue 2015-12-29 -> Payroll week 53 of 2015 Wed 2015-12-30 -> Payroll week 53 of 2015 Thu 2015-12-31 -> Payroll week 53 of 2015 -------------NEW YEAR--------------------- Fri 2016-01-01 -> Payroll week 53 of 2015 Sat 2016-01-02 -> Payroll week 53 of 2015 Sun 2016-01-03 -> Payroll week 53 of 2015 Mon 2016-01-04 -> Payroll week 01 of 2016
But in 2025, it’s the other way round, with just three days left over at the end of 2025 that get “shoved forwards” into the payroll year 2026:
Sun 2025-12-28 -> Payroll week 52 of 2025 Mon 2025-12-29 -> Payroll week 01 of 2026 Tue 2025-12-30 -> Payroll week 01 of 2026 Wed 2025-12-31 -> Payroll week 01 of 2026 -------------NEW YEAR--------------------- Thu 2026-01-01 -> Payroll week 01 of 2026 Fri 2026-01-02 -> Payroll week 01 of 2026 Sat 2026-01-03 -> Payroll week 01 of 2026 Sun 2026-01-04 -> Payroll week 01 of 2026 Mon 2026-01-05 -> Payroll week 02 of 2026
Big date bug ahead!
Can you see where this is going?
If you’ve got a date format string like MM/dd/YYYY or YYYY-MM-dd at any point in any software in which you are using ISO-8601 date formatting libraries…
…you will inevitably encounter bugs that print out dates with the wrong year, either at the end of one year or the start of the next, except in years when New Year’s Day is on a Monday.
(When 31 December is on a Sunday and 01 January is on a Monday, the ISO-8601 “week splitting” process works neatly, with 0 days at the end of the year left over.)
If you use YYYY where you should have written yyyy, your dates will regularly but infrequently be wrong, so your code will make mistakes, even though you might not easily notice them.
Here are the dud dates you’d see for 2018:
Y=2018 M=12 D=30 formatted with "YYYY-MM-dd" -> 2018-12-30 +correct+ Y=2018 M=12 D=31 formatted with "YYYY-MM-dd" -> 2019-12-31 *WRONG* (one year ahead) -------------------------------NEW YEAR------------------------------ Y=2019 M=01 D=01 formatted with "YYYY-MM-dd" -> 2019-01-01 +correct+
For 2019:
Y=2019 M=12 D=28 formatted with "YYYY/MM/dd" -> 2019/12/28 +correct+ Y=2019 M=12 D=29 formatted with "YYYY-MM-dd" -> 2020-12-29 *WRONG* (one year ahead) Y=2019 M=12 D=30 formatted with "YYYY-MM-dd" -> 2020-12-30 *WRONG* (one year ahead) Y=2019 M=12 D=31 formatted with "YYYY-MM-dd" -> 2020-12-31 *WRONG* (one year ahead) -------------------------------NEW YEAR------------------------------ Y=2020 M=01 D=01 formatted with "YYYY-MM-dd" -> 2020-01-01 +correct+
For 2020:
Y=2020 M=12 D=31 formatted with "YYYY-MM-dd" -> 2020-12-31 +correct+ -------------------------------NEW YEAR------------------------------ Y=2021 M=01 D=01 formatted with "YYYY-MM-dd" -> 2020-01-01 *WRONG* (one year behind) Y=2021 M=01 D=02 formatted with "YYYY-MM-dd" -> 2020-01-02 *WRONG* (one year behind) Y=2021 M=01 D=03 formatted with "YYYY-MM-dd" -> 2020-01-03 *WRONG* (one year behind) Y=2021 M=01 D=04 formatted with "YYYY-MM-dd" -> 2021-01-04 +correct+
If the Twitter thread started by @NmVAson is anything to go by, quite a few programmers still seem to be making this sort of mistake, which implies that they’re not testing their code very well.
As we mentioned above, writing DD by mistake instead of dd seems to be an unusual bug, presumably because the error shows up for about 85% of the year, and stands out dramatically for more than 70% of the year thanks to the three-digit days.
Admittedly, writing YYYY by mistake instead of yyyy produces bugs on fewer than 1% of the dates in a year, and not at all in one year every seven, but even with a lesss-than-1% error rate, there’s not really any excuse for failing to spot that you have made this blunder.
You could probably excuse not catching a one-in-232 error as bad luck; you might even get away with “bad luck” to excuse an error rate of one-in-a-million…
… but at 1% (especially when those one-percenters are right at the proverbial year-end), you really ought not to be letting this sort of bug escape your notice.
What to do?
If you are a programmer or a project manager reponsible for code that handles dates – and anything that does any sort of logging almost certainly needs to do just that – then please make sure that you:
- Don’t make assumptions. Just because upper-case
YYYYdenotes the calendar year in some places doesn’t mean it always does. - Read the full manual, or RTFM for short. Sadly, TFM for ISO-8601 is extremely complicated, but that should be your problem, not your users’ problem – with power comes responsibility.
- Review your code properly. Remember, the reviewer needs to RTFM as well.
- Test your code thoroughly. Anyone with ISO-8601
YYYYbugs really doesn’t have a good and varied test set, given that this bug shows up around the end of six years in every seven.
We take calendars for granted, but their design and use of is a mixture of art and science that has been going on for millennia, and still requires great care and attention.
Especially in software!
Pete
As fot the Y2K bug, I discovered it in 1992 but nobody paid attention. And, as I predicted, it caused havoc in accounting systems. Only that it was hushed in the media, for reasons of suppressing litigation. Don’t be naïve to think it didn’t happen.
Bryan
Pete, I wouldn’t know the litigative side, but I recall plenty of Y2K coverage† and awareness; where was it silenced? Conspiracy advocates and evangelical doomsday grifters everywhere used it as a springboard for spreading panic–much like 2012/12/12–or at least uncertainty and confusion leading to profit from donations and book and emergency kit sales.
In each case (1999 and 2012) I tired of it by September or so, and in December it was Talk Of The Town.
Y2K spent more time esteemed “non news” than as much of a secret, known for decades (1958 at least). I wouldn’t be surprised if the first programmer to truncate dates bemoaned the limitation (it’s how programmers think after all). However as humans tend to do, they (particularly those with decision-making power–and those paying for expensive RAM and hard drives) decided we’ll “go for it” now and remediate the consequences later. Their eschewing objectively beneficial long-range plans for short-term bottom line is understandable but ultimately regrettable in hindsight.
Anyone objecting to the shortcut was overruled and lost in torrential justification in “how much money we saved on two little digits.” Devils’ advocates acquiesced and shrugged, conceding that many would retire by the time “rent came due.” The collective tune changed at the last minute once those same bankrollers faced massive consulting fees or legions of “fixers” working overtime to compensate.
† Office Space (from 1999) is a great film. In one scene the protagonist describes his job, in a brief-but-salient allusion to Y2K. If it was supposed to be a secret, it wasn’t a very well-kept one.
Bryan
Re-reading my words begs of tinfoil-hat leanings, but the reality is that CEOs often heed bean counters more fervently than the daily grunts, and near universally more than an occasional harbinger of doom–who’s typically outmaneuvered by “yes men” anyway.
Behaviorally-Related Tangent:
> we’ll “go for it” now and remediate the consequences later
This should ring a contemporary bell, bearing similarities to the atmospheric CO2 accumulation we now face, twenty years later finally (albeit slowly) beginning to garner the Y2K-Signature-Last-Minute-Attention of YesterMistake.
With awareness even in the 1890s there was little regard for the matter, and–despite ever-mounting (and consistent) supporting evidence and increasing expert endorsement–it still faces widespread resistance, again fostered by those who will either lose revenue or face costly, short-term changes if proper approaches are enacted.
Doomed to repeat history…or at least paraphrase it.
† and I must correct my error:
I meant 2012/12/21 (not 12/12/12), when the Mayan “baktun” calendar was forecast to halt the rotation of the earth and release Godzilla.
David Heath
I’m not sure of your claim that it was ‘hushed.’ I’m quite sure it was very public knowledge by 1998 (probably earlier) and publicly visible remediation was well under-way by that time. Noting Bryan’s comments, I heard plenty of stories of computers that were deemed to ‘fail’ purely because the clock did not correctly roll-over to 1/1/2000… however, a simple re-boot correctly re-aligned the date… these machines were replaced anyway!
BTW… this thread does bring to mind the old (slightly) NSFW joke… “Y2K-Y jelly… for putting 4 digits where only 2 would fit before.”
daveboltuk
Did you also know that in 1993 many motherboards did not have a real problem with the rollover, but as we got closer to ‘the day’ more and more new motherboards failed to rollover correctly.
And just a side note about fixes. I received the fix for my Tandy 4P computer in 1984, to handle dates in 2000 onwards.
Bryan
> but that should be your problem, not your users’ problem – with power comes responsibility.
Excellent advice, Uncle Ben. ER… Uncle Duck!
> even though the longest months in the year only have 31 days.
I must take exception to this (after an article listing a great many of exceptions):
I am quite positive that last week alone had at least 35 or forty days…omg.
Engaging and informative, as usual. I’m annoyed I didn’t catch the “bug” in your first example. But I’ll let you blame your miscalibration on last week’s 57 long days.
Mark Stockley
Don’t worry, only 20 more days until Christmas.
David Heath
I did rather enjoy the subtle hat-tip to very early Pink Floyd!!
(for the younger readers… can you find it?)
Steve
David, I need a little help here. I’m no youngster; I’ve been a Floyd fan for nearly five decades. But I’m not seeing that “hat-tip” anywhere. What am I overlooking?
(my excuse would be that I am stuck here at work on Christmas Eve once again… sigh…)
Paul Ducklin
Check the variable names.
But to save you some time :-)
https COLON SLASH SLASH www DOT youtube DOT com SLASH watch QUESTION v EQUALS 4yI6zYT9Uo8
Enjoy Christmas Eve. Three minutes left where I am sitting…
yankeefarmer
Steve, 1968 falls out of the date range for “nearly 5 decades”. It’s alright, we know where you’ve been.
The reference is to “Eugene” the B-side of a single: Point me at the sky.
David Heath
It was also on Ummagumma… I just found the Wikipedia page you quoted from! Oddly, I thought it was on one of the other early albums… obviously a faulty memory!
Paul Ducklin
I checked the release date of Ummagumma (in reference to the “five decades” remark), where there is indeed a longer, live version of ‘CWtAE’, and IIRC that album came out 50 years *and one month* ago :-)
They performed and recorded the song in the original Live at Pompeii gig, too, I think.
David Heath
There’s a song on a very early album called “Careful with that axe Eugene.”
Dave B
I don’t program in Java directly at the moment, my main languages are currently C/C++, Delphi, PHP and JavaScript (not necessarily in that order).
Whenever I have to do something with dates I find myself checking the format strings, because they vary from one language to the next.
Just took a look at the Java docs. Much clearer than average and no real excuse for making mistakes with YYYY and yyyy, for instance. Wish more documentation came up to that standard.
Really nice explanation of the pitfalls of the different formats and why in programming you cannot take things like dates for granted even though you use them every day.
Dave
Yet another euphemism for the true F in RTFM, though consistent with your point… BTW: reviwer needs a spell check.
Paul Ducklin
Fixed “reviewer”, thanks!
li bo
Why i get “Y=2019 M=12 D=29 formatted with “YYYY/MM/dd” -> 2020/12/29″ from “tryit(2019, 12, 29, “YYYY/MM/dd”);”?
Paul Ducklin
[Note. This comment was updated at 2020-01-02T10:30Z. I mis-read the OP’s comment before.]
Oops. I made a mistake in the article. (Ironic!)
I’ve corrected it now:
Y=2019 M=12 D=28 formatted with "YYYY/MM/dd" -> 2019/12/28 +correct+
Y=2019 M=12 D=29 formatted with "YYYY/MM/dd" -> 2020/12/29 *WRONG*
Thanks for spotting that!
li bo
en… but 29 is sunday, it`s not the day of the last week in 2019 (“The first day of every week is Monday.” —— ISO-8601).
Paul Ducklin
Hmmm. You’re quite right – 29th was a Sunday. That’s weird.
IIRC, when I wrote the article my code produced the output I originally published, namely that 2019-12-29 was in 2019 according to both YYYY and yyyy. At least, I’m pretty sure that’s what happened. But trying it now, my results match yours!
So there’s something else here – because I am not using the same OS or the same Java version. (I wrote the article last year on my beloved Mac, which is no longer so beloved because the keyboard broke on NYE, so I am now using Linux with a newly-downloaded JDK 13. I don’t have my Mac handy now to compare results.)
Anyone got any ideas? Could it be that the “local” settings for my new Java runtime somehow treat *Sunday* as the first day of the week (which is the traditional setting for GBR)? If you don’t mind me asking, what JRE/JDK do you have, on what OS, and which locale settings are you using?
FYI, I have Slackware Linux 5.4.7 running:
openjdk 13.0.1 2019-10-15
OpenJDK Runtime Environment (build 13.0.1+9)
OpenJDK 64-Bit Server VM (build 13.0.1+9, mixed mode, sharing)
In case it matters, my time zone is “Europe/London”, currently at UTC+0.
li bo
1. OS: macOS Mojave 10.14.6
2. Java version: “1.8.0_201”
3. Time Zone: Asia/Shanghai, UTC + 8:00.
and this:
“`java
System.out.println(Locale.getDefault()); // zh_CN
System.out.println(WeekFields.of(Locale.getDefault()).getFirstDayOfWeek()); // SUNDAY
“`
A new date API was introduced in Java 8. The new date API is an implementation of the JSR-310 specification. Could it be that in the new specification a week starts on Sunday? Although I haven’t found a specific description.
dclaar
I kind of disagree with ” Anyone with ISO-8601 YYYY bugs really doesn’t have a good and varied test set”.
One of the primary tenants of unit testing is that you test your logic and assume that the libraries that you call have their own unit tests and can be trusted.
If I write a function that does a date-related calculation and prints the results, I’m going to test dates that are interesting *to my algorithm*. Maybe leap years matter: I’ll check them. And I’ll spot check other dates. But otherwise, I am going to assume that date formatting works as advertised. Similarly, I will only test the results of an XML parser for the bits that are of interest to my algorithm. (Wait, did I just publically state I’m a slacker at testing?) :)
It’s a bit how you mis-read a comment. No matter how much you checked your answer, it would still be wrong because you mis-read the manual, err, comment. Similarly, if I’ve read the date formatting page, and I’m “sure” that I know what it says, then so as long as my spot-checks show that it’s working, I’m going to believe that it works in general and test the stuff that I wrote. Hopefully my code reviewer will catch it, but they are probably going to think “YYYY…yeah, that’s right” and focus on other areas.
Paul Ducklin
Given that YYYY is explicitly documented as giving values different from the calendar year for 1 to 3 days in 6 out of 7 years, sometimes at the end of one year and sometimes at the start of the next, but is identical when the year ends on Sunday… I’d definitely expect that in my unit tests.
Especially because YYYY is associated with financial and accounting calculations, e.g. payroll, and it’s weird only at year-end, and year-end is a tricky time in anything involving weekly calculations.
Paul Mawdsley
“public class CarefulWithThatDateEugene” – One does not want condensation, or does one, Floyd?
Pszczółka Erina
It is not Thirteember!!!
September is the 9th month, 9-2=7, SEPTEMber is named after 7.
This works with OCTOber(10-2=8), NOVEMber(11-2=9) and DECEMber(12-2=10)
So the 13th month is named after 11, because 13-2=11.
So the 13th month is Undecember, NOT THIRTEEMBER!!!
Paul Ducklin
Who said I was referring to month #13 :-)
I was referring to month #15, which (given the insertion of the months Julius and Augustus) would be named after 13. In Latin, Tredecember. But in English, at least in the English spoken in the year AD 16 million, Thirteember.
Anders Gustafson
Really good and informative article! Thanks!
The part for 2019 with the line
Y=2019 M=12 D=29 formatted with “YYYY-MM-dd” -> 2019-12-29 *WRONG* (one year ahead)
is still a bit misleading though I think. Either the line should contain “2020-12-29” or be marked as “correct” instead.
Paul Ducklin
Thanks… and, yes, that’s a typo. It should say:
Y=2019 M=12 D=29 formatted with “YYYY-MM-dd” -> 2020-12-29 *WRONG* (one year ahead)
So the word “WRONG” was not wrong, but the date “2019-12-29” was.
Thanks for noticing and reporting it. It’s fixed now.