(the highest setting it offers)
You might try manually entering a higher bitrate if it would be desirable. I have found that with the latest VLC for windows I can set video bitrates as high as 15360 even though the dialog box gives options for only up to 3072. The video quality is noticably better when output to mp2v transport stream file. And running through mpeg2repair afterwards does report a higher average video bitrate which with older VLC's it did not. I've noticed in most cases that VLC's encoder does NOT seem to just be padding to get higher bitrate as it isn't adding nulls and in most cases a setting of 15360 does not give a higher resulting file size than some lower rate if there's no data needing encoding to the higher rate, seems to just use as much as it needs up to the 15360 max.
I've been doing this with WMV-HD clips with very good results lately to get them in a format MPlay can play natively. If I could just get VLC to understand how to transcode 6-channel WMA into 6-channel AC-3 I'd be set, but alas no such luck yet. It can't even downmix a 6-channel WMA to a 2-channel mpga either, so any WMV-HD's with more than 2-channel sound I have to leave in WMV and use Nero or such to transcode on the fly (which needs MASSIVE PC horsepower) via UPnP.
Unknown if audio bitrates higher than 512 (which I believe is actually 256/channel for 2 channel output, not 512/channel) would give you better audio quality with the mpga output encoder but it might be worth a shot.