In older men, decreased sexual activity and desire, not erectile dysfunction, may cause serum testosterone to decline, according to a recent study.