The industry standard is 5 years and that's to #1, sell more helmets and #2, for protection from long term liability for the manufacture. I've seen a couple of "studies" on this and I've seen no reason for every helmet to be replaced at the 5 year mark simply because the shell is "breaking down" or the "shelve life" of the polymers.
Depending on frequency of use, storage & environment (humid v. dry air) and of course how you handle your helmet determines your helmet's actual usefulness/life span. I've seen riders with scratched up helmets and how they handle them when putting them down and some of those need replacing. My oldest helmet, which sits mostly on a shelve, still looks almost new, and I got that one as a hand me down from my brother over 7 years ago. It's an XL and after a couple of uses it was replaced with a better fitting helmet (LG) and rarely used.
Like I said, that one sits on the shelve most of the time, I have 4 helmets, three different sizes (SM, 2 LGs, & XL) and I'm currently looking for 2 more (XS & MD) so when I offer a lady friend a ride I have the right size for her. I think having the right size helmet in good condition is more important than it's age, assuming it's not showing it's age, discoloration or other signs of the shell breaking down; not being used enough for the liners to be a problem yet.
But how you handle your equipment matters more, my most used helmet a Bell, is three years old and it looks brand new.