You might want to have a look at this ...
GeorgeBreese.com ... this particular 'George' is the guy who writes many of VIAs utilities & post chipset patches ;)
The latency patches found there are more to do with pci timings & dma calls, than sound latency issues, but probably they might help with this problem.
Which makes me think ... just what is the problem ... too much latency (which you just 'allow for' don't you ?) or 'unpredictable' latency this 'usb jitter' Platypus mentions - in which case the above will help.
BTW ... if you ask me any setup that depends on s/w sound processing (i.e. by the cpu) is always gonna be more suscepible to latency, the less going betwixt the s/c & cpu the better ;) .... but I see you are 'onto' that one :)