For tough jobs like removing rust or preparing surfaces for paint or varnish, there's nothing like steel wool. It's made of steel thread, loosely woven into small pads, and comes in various grades, from very coarse number three down through superfine four ought (0000). Four ought is probably the best abrasive you can use just before a final coat of varnish.
For more on tools, consider:
Varnishing Made Easy
Inexpensive Ideas and DIY Tips for Storing Sandpaper