My idea: file distribution reimagined

how is the authority that makes sure that a certain hash really represents a certain file

Easy. We don't care about what the file is, or says it is. That is ultimately up to the provider. It's the same trust you have with every provider right now with filenames. I trust the filenames on Microsoft's servers are indicative of the files they say they are. I trust the files on Sourcefourge are for the downloads I was looking for.

The whole reason I need hashes for this system is so that files are normalized. Their content is what's important. You can name a file any number of ways. But for distributing the file, it doesn't really matter what it's called. It matters though where it reaches its destination, which is why using magnet links you can still append a "suggested name" to the result. That has the same affect as filenames in a HTTP URL. It is suggested, but you don't always have to save it as such.

First, I would like to quote a very insightful comment on "hash collisions" on my blog.

http://www.istartedsomething.com/20090710/reimagining-file-distribution-universal-downloads/#comment-76909

Okay, but even if the odds are less than a single atom in the known universe, doesn't mean it couldn't happen. I guess there's two ways of looking at this. The first way is to simply ignore it, and that's what I believe GUID does. It since turned out pretty good for GUIDs. The other method is to try to reduce the chances even more, and with that you could concatenate another hash string generated by a different hash function. This doesn't decrease the security risk, but it does reduce the risk of a single hash collision.

Once the project gets started (which the odds are currently looking pretty good), I'll obviously have to do more real world testing and see which path to go down.

If you use anything besides a hash, it breaks the system. You need a method to identify unique files in a way that can be reproduced with 100% certainty on at least two different computers that has no communication with each other. You cannot do this with GUIDs.

Well you could, but that'd require you to query a server every time you needed to identify a file, which is not applicable in the nature of the internet today.

Actually

If I download a file from microsoft.com/download/ I'm sure that this one comes from Microsoft. There is little reasons to not believe that.

That is because you "trust" Microsoft's website, their URLs and their download servers. You'll have to utilise the same trust in the distribution system I've explained, but you no longer have to trust the servers, since as long as hashes have not been compromised (which SHA-256 has not), it can be guaranteed if you trusted what Microsoft wanted to provide to you, you'll get it.

Under the SHA-256 hashing algorithm, it is currently impossible to compromise hashes.

There is no "filing" invalid hashes or "assigning" hashes to files they don't belong to. A hash either matches or it doesn't. You're assumping a hash has some sort of relationship to files in the database. It doesn't.

This is the same way most online services check if you are the user you claim. They store your username and your password's hash. If it matches, then it's assumed you know your original password, without it ever storing your original password. If it doesn't match, then it doesn't really care what you typed, it didn't match.

You could argue nothing is definitive, except very close to 0 or 1

I don't think I said there's no possibility of collisions. Like someone said, the chances always exist, but its very small.

On the other hand however, SHA-1, SHA-256 and SHA-512 has not been compromised. By that I mean there's no known way to artificially create two files that have the same hash. Something that MD5 has recently failed against.