The Senate/Congress can remove a Vice-President at will?
US Constitution, Article II, Section 4 wrote:
The President, Vice President and all civil Officers of the United States, shall be removed from Office on Impeachment for, and Conviction of, Treason, Bribery, or other high Crimes and Misdemeanors.
It does apply to Cabinet officers and
Federal judges though.
While such an expulsion is not the same as a removal from office, it effectively means that the body will no longer recognize that individual as a representative.
This is coupled with the ability in *some* states to recall their Congressmen or Senator.