So the icons remain in place until you visit the site, then the shortcut icon changes. This is (I think) because the writers of the HTML page have put in a line
Code:
<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon" />
which forces your browser (not just IE) to recognise the favicon. Since the shortcut is actually a bookmark, it will try to put the favicon in... you go round deleting them, so they end up blank.... if you leave the favicons alone for a while, you should see the shortcut change when you visit to the favicon.