I'd have to support this. Allowing cached certificates to persist after smart card removal does not appear to have any benefits and has some serious downsides. The certificate in the SSL cache clearly needs purging when the originating credential is withdrawn. Are the IE developers are relying on a smart card authenticated Windows session to be logged out or locked?