@wkempf: Another interesting thing to note is that this version, even if you do it in a single statement like you did, require two lookups into the dictionary: one to find the old value, and one to put the new one in. If profiling shows that this is a performance-sensitive part of your application, it can be worth it to use a reference type so you can modify the value without performing the second lookup.

class Count
    public int Value { get; set; }

Dictionary<string, Count> runningCount = new Dictionary<string, Count>();

string key = "foo";
Count count;
if( runningCount.TryGetValue(key, out count) )
    runningCount.Add(key, new Count() { Value = 1 });

Note that this still performs a double lookup for values that aren't in the Dictionary, but that's unavoidable (ConcurrentDictionary provides a GetOrAdd method that provides this operation, but more for synchronizing this kind of operation than to avoid the extra lookup; ConcurrentDictionary is probably not the best idea to use if you don't actually need concurrency because of the overhead it adds to be thread-safe).

And as always, profile your code both before and after you make a change like this because there's no point adding unnecessary complexity if this isn't a major performance bottleneck of your code.