Neutron stars undergoing r-mode oscillation emit gravitational radiation that might be detected on the Earth. For known millisecond pulsars the observed spin-down rate imposes an upper limit on the possible gravitational wave signal of these sources. Taking into account the physics of r-mode evolution, we show that only sources spinning at frequencies above a few hundred Hertz can be unstable to r-modes, and we derive a more stringent universal r-mode spin-down limit on their gravitational wave signal. We find that this refined bound limits the gravitational wave strain from millisecond pulsars to values below the detection sensitivity of next generation detectors. Young sources are therefore a more promising option for the detection of gravitational waves emitted by r-modes and to probe the interior composition of compact stars in the near future.