We're having an ongoing battle with at my state owned and operated marina. There's the two percenters who tie up their boats with binder twine, old shoe laces, or whatever is handy that create a problem. Management response? The legal whiz kids decided that all dock lines must be no older than 2 years. In responding to this idiocy, several of us conducted some hard research on the longevity of dock lines. Answer - we were unable to find any reliable reference (not manufacturers and distributors [except Parks], USCG, insurors, AWWO, ASTM, etc., etc.) that so much as suggested a maximum useful lifetime or replacement schedule for nylon line.
There are plenty of common sense evaluation points, as noted above, but no objective criteria.
A lot of people think "stiffness" is a indicator of the line "wearing out" - my experience is that washing and fabric softener as OC suggests will restore a more agreeable hand. I look for chafe, fraying, and consistent lay. I also retire storm lines (dedicated, that only see the light of day when needed) after a serious blow (Isaac, for instance) when they've experienced substantial stretch.