Brian, I understand your frustration, and it's shared by many folks until they get a few flights under their belt. There are a lot of variables in play, and some of it is systemic, too. If you've been around aviation a while (real world), you can generally anticipate what the controller is going to say and in what order; that helps a lot. Also, note that the voice codecs VATSIM uses are old and kept that way to aid in backward compatibility, although at some point you have to cut the old stuff loose. So the network, by (old) design, is not as clear as a bell as TeamSpeak. Other variables can include how fast the controller is talking, internet lag, etc.
My suggestion, though, is to keep plugging away and trying voice every time. It does get better as you get used to it. One thing I suggest you also try is to file, in your flight plan comments, something along the lines of "Old ears, please speak slowly and clearly". It's a mildly humorous way to let the controller know (if he looks at your flight strip) that you'd appreciate your instructions in a slower, clearer manner. Sometimes when controllers get busy, they rattle stuff off faster than professional auctioneers!
Another thing you might try is changing the audio input/output type in your client. Assuming you're using vPilot, it's in the Audio tab, the Microphone Device and Output Device. If you're using Wave, try Direct (and vice versa). Note that many people find it better to have the audio device set to the opposite of what their TS device setting is (i.e. if TS is set for Direct, use Wave on VATSIM).
Keep at it, and all the best.