I had a few weights that had rust on them from being in the basement and garage. You could still see some of the red paint but the rust red was way more prominent. This image has one weight with a little rust and another covered with it.
One of the better ways to remove the rust is with a wire brush. It will take some time to remove but its worth it. To speed it up, get a wire brush cup for your drill. I used a wire brush cup on my Ryobi hand drill and it made the job much easier and a bit quicker.
Take care to remove any dust or dirt that is on the weight first. Rust can be a dust magnet. Using a rag or brush should be enough to remove loose dirt. You might find it a bit easier to use a damp rag but do not be temped to use a wet rag. You want to keep the surface as dry as possible.
Take your time taking the rust out. There is no right or wrong way to remove the rust. Be patient and slow and it will make a difference. Be sure to use caution as well, wear eye protection and be in a ventilated area, wear a mask if needed.. Do not breathe in any of the produced dust that is airborne.
The temptation to begin painting right away will be great but before you do, brush and wipe off the weights again. This will ensure that you have a clean surface and will not see any loose particles mess up your paint job.
With a clean surface, it is time to paint. Krylon Fusion paint dries to a hard top layer. I have found it resists scuffs and scratches well. It can be found in many colors and Stain Black matches well with common weights.
If you wait too long, you may have to scuff the paint to get it to adhere again. This is not a project that can wait for the next day but needs to be done on the same day, so start early.