They'll get worse with age, so the more concerned you are with performance (and the more money you have invested in them) the sooner it'd be worth getting them serviced. Annually would probably be more than enough for most people though.
We strip and rebuild our dampers every 1,000 miles or every other year. Some seasons that's every season though, depends on how much running we've done.