Securely Reading a Custom File Format

Posted By: thumbtacks2 | Jul 27th, 2007 @ 12:56 PM
page 1 of 1
Comments: 6 | Views: 1782
#Jul 27th, 2007 @ 12:56 PM
thumbtacks2
thumbtacks2
Is there a decent guide/book/series of articles anywhere for building secure code when reading data from a customized file format? I'm developing a handful of specialized animation formats for my game engine and would like my routines to gracefully handle corrupted data (should it occur). I will be using C++.

Thanks.
#Jul 27th, 2007 @ 3:05 PM
ScanIAm
ScanIAm

I don't know of any books, but I've had to do this in the past, and the solution tends to be to use fixed length records whose position in the file gives meaning to those records.

i.e. record #5 is the 5th pixel or item.  Or the 5th item is a group based off of the name of the file.

or, the number '5' is used along with the filename in a numeric function to produce meaning.  You can also place the record number in the record itself as extra validation. 

You can also run a CRC or simple checksum over the record and store it at a known location in the record (like the end, or beginning).

There are some other tricks, but these are the quickest.

If you are trying to recover/rebuild if the file happens to get corrupted, then you'll need to store the data in multiple locations or use parity bits (ugh).  I'd shy away from the rebuilding part if at all possible.

#Jul 29th, 2007 @ 3:29 PM
PerfectPhase
PerfectPhase
Don't know if it's of any use, but I seem to remember this video having a bit about testing image formats

http://channel9.msdn.com/ShowPost.aspx?PostID=170290#170290
#Jul 30th, 2007 @ 7:13 AM
Minh
Minh
I wouldn't bother with corruption recovery, I mean, with animation data, what can you do?

The most you should probably do is corruption detection.

Also, why a custom animation format? You wouldn't be able to use any of the available animation tools out there.
#Jul 30th, 2007 @ 7:38 AM
thumbtacks2
thumbtacks2
Minh wrote:
The most you should probably do is corruption detection.
That was my main concern, although it isn't a big one.
Minh wrote:
Also, why a custom animation format? You wouldn't be able to use any of the available animation tools out there.
It's only a small piece of what I'm doing; I am likely going to be using bones for most of the model animation (as opposed to something like keyframes/interpolation), and am already using Blender for the models (exporting in a wavefront format). I don't know how well Blender exports bone information, however, so that may be a bit of a challenge...although I'm sure I can always alter the relevant Python export script to get what I need.

There are going to be other, smaller bits of animation that I'm going to need, however, and in order to get the effects I want, I'm going to have to come up with at least a couple of new tools...and hence, the custom format.

Thanks for the replies. Smiley
#Jul 30th, 2007 @ 9:58 AM
Minh
Minh
The .X format is fairly well documented (if you're using DirectX). And .X is popular enough that there should be a converter (paid or free).
#Jul 30th, 2007 @ 10:24 AM
blowdart
blowdart
MSDN Mag this month had an article on OPC. Which includes signing of items in the packaged file, so if the signature fails, someone edited it.
page 1 of 1
Comments: 6 | Views: 1782