For my money all the TryParse functionality should be moved into an 'advanced' class, such as System.Text.FastParser for example.
All the BCL value types that support parsing should have a Parse method that throws and an IsInvalid(String) method. (I like to test for 'invalid' because that tends to be the exceptional case, but I'd be happy enough with IsValid(..)).
Keep the common APIs simple, clean, and consistent. Having TryParse on Int32 is just ugly.