I think it's probably possible, but it's a very bad idea to begin with. Having a fixed swap partition is also a very bad idea in Linux (at least *BSD can have swap files optionally).
That's because the system fails miserably if it runs out of partition space. Having those things on the filesystem means that the data file(s) can grow dynamically - and that in turn means you have abstraction over the actual disk hardware, so you can use nifty
stuff like, depending on the filesystem, multiple storage devices or distributed locations transparently.
I think you're right in respect to performance, but performance penalties are cheap compared to the gains in abstraction and maintenance a filesystem provides. (also think: backup, error correction, software RAID features, "defrag", etc)