Page 12 of 16

Re: IPF and Openness

Posted: Sun Feb 13, 2011 1:47 am
by balrog
Whatever you do, please don't hardcode hashes. I'd rather provide an XML file or such with the hashes. This way if it's found that rips were done badly, they can be invalidated later.

Why can't something like OpenSSL be used for signing? This shouldn't really take that long to integrate...

Re: IPF and Openness

Posted: Sun Feb 13, 2011 1:58 am
by karadoc
The hashes would indicate it was from us, that is all. I can't see why people would want to edit them, and it saves having a file having to be placed somewhere. The only other way is to re-issue all the current IPF's... but that would be annoying for everyone, not least that there are 3600 of them.

OpenSSL. Possibly, but we still need to do the work. We're "a bit" busy. :) Actually, before I saw your post, I clarified my previous post to emphasise that we would want this functionality in the library so an emulator just has to say "who issued it and does the signature correct" - to make it far more likely they would do so.

Re: IPF and Openness

Posted: Sun Feb 13, 2011 2:10 am
by IFW
The library has no dependencies (licensing or technical/other libraries), apart from some minor issues, that were very easy to overcome when it was ported to Linux, Mac and AmigaOS.

Re: IPF and Openness

Posted: Sun Feb 13, 2011 2:38 am
by balrog
Unfortunately writing a good (read: no significant bugs) public-private key implementation from scratch is pretty tedious work. Why not use OpenSSL as a compile time option? That way if someone doesn't want the external dependency, they can leave out the crypto features (until it's reimplemented internally).

Also, couldn't IPF's be invalidated by adding the hash to a "blacklist" of sorts? Or removing it from the official list? Of course, this would only be needed for officially signed IPF's.

"The hashes would indicate it was from us" ... that's a bit overworked. Public-private key crypto would be so much more efficient here ... instead of a hash per IPF, only a public key would be needed. But the IPFs themselves would need to hold signature info — which I hope is part of the current IPF standard, since I do think it should be.

Re: IPF and Openness

Posted: Sun Feb 13, 2011 9:28 am
by karadoc
I'm not sure I understand quite what you mean. The hard coded hashes in the library would only be for the current set of IPF files that do not have a signature The library call to verify origin would check against that list if no signature was present in the IPF itself (and so transparent to library users).

That's not to say we have thought about it much beyond this. As I said, we've not managed to get time to look into it.

IPF and Openness

Posted: Sun Feb 13, 2011 10:01 am
by Interceptor
Quite a few people say 'you should do this and use this and don't do this and I'd prefer this'

But it's useful to know why you think that should or shouldn't be the case, on the basis of what's best for the format overall, it's acceptance and it's practical use as a preservation medium, not just any particular personal preference.

Re: IPF and Openness

Posted: Sun Feb 13, 2011 12:12 pm
by phi2x
It's interesting to note that there is another file preservation format that is gaining grounds these days: the HFE file format.
It was originally created for the universal HxC floppy emulator:

Software is also provided to convert from all popular file formats, including IPF, to HFE format.

What are your plans regarding this stuff?

Re: IPF and Openness

Posted: Sun Feb 13, 2011 12:32 pm
by IFW
Nothing. The format you mentioned is timing and pulse information, not content with context.

IPF is efficient, and can change its context without changing the content, ie not medium specific; be it software emulation, hardware emulation, or actually writing to the real media or a different media type. It can also change certain parameters run-time to adopt to the properties of the target media.
IPF file data content can also be compared by design, regardless of context or context like data such as gap information; ie you can compare two IPFs and can tell by absolute certainty whether they are duplicates or not, even though your dump files could be very different due to other reasons.

No other preservation format was ever designed that supports all of these things.

IPF and Openness

Posted: Sun Feb 13, 2011 9:01 pm
by mr.vince
Funny how things become a preservation format over night.

Besides other things: HFE can not store the density information (=variation) present in many protections. This means such data gets lost during conversion from IPF to HFE. I am also sure weak (flakey) bits are am issue.

Jeff has made it clear several times that the format was designed to have something that fits the HxC SD card edition.

Re: IPF and Openness

Posted: Sun Feb 13, 2011 9:03 pm
by balrog
What I meant is as follows:

IPF should have a section for metadata. In this metadata, a signature field should exist, and even a public key field. This metadata area should be extensible.
Such metadata would be used to verify the validity of an IPF.
This isn't a part of the actual IPF data, just metadata. Similar to EXIF tags in JPEG, or ID3 tags in MP3.
At least that's how I'd do it :)