It's also important to note that in the current implementations, finalizers always run on the 'wrong' thread. They're always run on a dedicated finalizer thread. You need to be careful in order to avoid deadlocks or race conditions.

Another side-effect of an object with a finalizer is that the memory for that object cannot be freed until the finalizer has run. It takes (at least) two collections for this to occur: the first detects that the finalizable object is garbage and adds it to the finalization queue. The finalizer thread is signalled that an object has been added to the queue. If it was asleep it wakes up and finalizes the objects in the queue, blocking again once all objects are finalized.
If the object no longer has any references this will be detected by GC on the next collection and the object's memory will be freed. If a GC runs while the object is still on the finalization queue, the GC will consider the object reachable and not delete the memory.

My rules are:

For producers (class designers):

If your class owns an unmanaged resource, make that its only function. Don't add any other data. You might want to create a nested wrapper class within your main class if your main class needs more than one unmanaged resource or if it needs unmanaged and managed resources. Implement IDisposable, override the Dispose method and provide a finalizer.
 
If you implement a finalizer, in the Dispose method, call GC.SuppressFinalize(this) to tell the runtime not to call the finalizer. Otherwise, your object will end up on the finalization queue when you've got nothing to clean up.

If your class owns a managed resource which has a Dispose or Close function, implement IDisposable. In your Dispose override, call Dispose on all owned resources.

(Rare) Consider using IDisposable for resource acquisition/automatic release semantics, e.g. for monitors, managed synchronisation objects, etc. Yes, C# has the lock keyword for acquiring and automatically releasing monitors; however, you can't perform a TryEnter using the lock statement.

For consumers (class users):

If an object you're using provides a Dispose or Close method, call it as soon as you've finished with the object. Use a using statement if the class implements IDisposable. If it doesn't, but provides a Close or Dispose method, report a bug against the class.