You should be fine. Obviously the faster you go, the more it helps, but it's not like aero wheels where you can actually go slower with them if you're not going fast enough. Even then, it's only when comparing a $2k aero wheelset with a $2k non-aero wheelset since the non-aero one will be lighter. I think most people don't buy like that, though. It's more likely that they'll go from a general training wheel to something like a 404, which is both lighter AND more aero, so it'll be faster all the time. Sorry, this answer is probably way longer than you wanted, but 22 for a rolling HIM course is pretty solid and no one's going to talk bad about you for having it. I've seen guys average slower than that on a sprint who had one. Especially over something that long, you'll probably see a couple minutes difference.

Yeah, man....if you're averaging 22 for 56 miles, not many people are going to be passing you.