Also check to see how long it takes for users to get a connection object from the pool.  You may have to adjust the pool size and see if that helps.  You should be able to use SQL Profiler for that, and look for connection events.