Great discussion. I've moved away from ConcurrentQueue and ConcurrentDictionary Collections in favor of ImmutableList and ImmutableDictionary collections on a high performance a WCF service that uses TPL extensively. I noticed a nice performance increase on the larger collections when iterating through larger collections.
For everyone getting started, the only real gotcha I've found is to remember to take the return result and assign it to the collection like so:
var tmpCollection = ImmutableList<string>.Empty;
tmpCollection = tmpCollection.Add("Test");
Overall though, I am very pleased with the new Collections, kudos to your team!