This is totally possible. Add the contract files to the client project as a link (Add | Existing Item | Add As Link). If you're willing to create the ChannelFactory on the client yourself then you don't need any config either.
Yes I ended up adding the common files as a link to the other project. This works well and is the best way to share common files between different projects. I remember using links to files in a previous version of VS (maybe it was VS 2008), but it was very flaky at that time. Sometimes VS would get confused and even end up crashing. But it seems to work flawlessly in VS 2012.