IPF creation and disk writing via Kryoflux

[keir]

I've been doing some more work on my Amiga disk decoding/encoding utilities. These allow you to read disk dumps from various sources (Kryoflux, IPF, ADF, etc) analyse the contained data according to known encoding schemes (AmigaDOS, Copylock, RNC PDOS, various game publisher custom formats, ...), and write the analysed data in another format. In effect this is an all-in-one Amiga disk archiving/transcoding/reading utility.

The new feature which I just completed is support for writing IPFs. So you can now create or transcode disk dumps that can be immediately loaded into emulators, read on a real Amiga (latest version of RawDIC/WHDLoad), or written back to a real disk using the latest Kryoflux software!

For example, I was able to back up my own copy of New Zealand Story (a typical Copylocked game disk) as follows. In Linux:

Code: Select all

dtc -r6 -fnzs/ -i0 -e79 -i5
disk-analyse --format="New Zealand Story" nzs/ nzs.dsk
disk-analyse --format="New Zealand Story" nzs.dsk nzs.ipf

This dumps the game disk using Kryoflux, into a set of raw Kryoflux STREAM files. It then uses my analyser to archive in my custom format, and then again to transcode to IPF (you could also archive straight to IPF in one step).

Then, since I believe Kryoflux write support is currently only in the Windows tools, I do the writing step from a Windows VM:

Code: Select all

 DTC -d0 -fnzs.ipf -s0 -e79 -w -l15

Result, a working copy of my own original game disk -- no cracks or patches!

You can also do things like write out unprotected ADFs through Kryoflux, by transcoding them via mfmparse first:

Code: Select all

 disk-analyse my.adf my.ipf

If you want to try it out, you can grab all my code from Github: http://github.com/keirf/Disk-Utilities

The bad news: It's Linux only (also Mac OS and Windows via Cygwin/MinGW) and you have to build it yourself, something like:

Code: Select all

git clone http://github.com/keirf/Disk-Utilities
cd Disk-Utilities
make
su -c make install

Let me know if it's useful to you!

[breiztiger]

hello

great job

do you expect doing a executable for dos or windows ?

thanks

breiztiger

[keir]

do you expect doing a executable for dos or windows ?

No, I don't develop on Windows. It would be pretty easy for someone with Windows dev experience to port the code and send me patches though.

More realistically, you'd have to run these tools in a Linux VM. Which is rather like what I do to access Kryoflux write support (run a Windows VM).

[keir]

Actually someone suggested trying Cygwin, and that was easy to get basically working. If you have Cygwin installed you can do this:

Code: Select all

git clone http://github.com/keirf/Amiga-Disk-Utilities
cd Amiga-Disk-Utilities
SHARED_LIB=n make
cd mfmparse ; ./mfmparse.exe -h

Don't try 'make install'. Just run the mfmparse.exe straight out of the build directory.

[MrCheese]

I've just tried this with a Workbench disk image (the track00.0.raw, track00.1.raw etc) and booted from the resulting IPF file.

[tect]

This is really very cool!

Is there something a normal user like me can do to help you implementing more game formates for your mfmparse-tool?

[keir]

This is really very cool!

Is there something a normal user like me can do to help you implementing more game formates for your mfmparse-tool?

Yes, try out more disks and report the format! Every game I have analysed I add to the mfmparse_formats file, which describes what format we expect each track on the disk to be in. There's a flow to working out the disk format that's easy to get the hang of.

I'll give an example here, using the Graftgold classic Fire & Ice:

Code: Select all

# mfmparse fireice_disk1/ fireice_disk1.ipf
T0: AmigaDOS
T1: Unformatted*
T2-129: AmigaDOS Extended
T130-159: Unformatted
** WARNING: 1 tracks are damaged or unidentified!

So, we didn't specify a format (via the --format= command-line option), which means we default to amigados. You can see that most of the disk *is* amigados. The exceptions are tracks 1 and 130-159. Unformatted* means that the track looks like it has data but we couldn't identify it. When it's track 1 on its own, that usually means Copylock. Tracks 130-159 are Unformatted (note no asterisk) which means they really contain no data -- it's not a custom format or anything like that.

So we guess the format is amigados with a copylock protection track:

Code: Select all

# mfmparse --format=amigados_copylock fireice_disk1/ fireice_disk1.ipf
T0: AmigaDOS
T1: Copylock
T2-129: AmigaDOS Extended
T130-159: Unformatted

Success! Note no warning about damaged or misidentified tracks. We could also just have specified --format="Fire & Ice" as I already added this game to the /usr/local/share/mfmparse/mfmparse_formats file.

Some are more tricky. Here is Amnios disk 1:

Code: Select all

# mfmparse amnios_disk1/ amnios_disk1.ipf
T0: AmigaDOS
T1-154: Unformatted*
T155-159: Unformatted
** WARNING: 154 tracks are damaged or unidentified!

A few genuinely empty tracks at the end (155-159) which is quite common. But clearly the bulk of the tracks are in a custom disk format. Sometimes it is a format we can already analyse and I have a simple way to probe that:

Code: Select all

# mfmparse --format=probe amnios_disk1/ amnios_disk1.ipf
T0: AmigaDOS
T1: Unformatted*
T2-154: Psygnosis A
T155: Core Design
T156-159: Unformatted
** WARNING: 1 tracks are damaged or unidentified!

Better! Note that track 1 is still unidentified (but 'probe' does not try all things we can analyse) and track 155 has been misidentified -- this is why we only use 'probe' to scope out a new disk, it's too inaccurate to rely on it in practice. In this case you'd have to send the image to me to get some help. In this case I've already done the work:

Code: Select all

# mfmparse --format="Amnios Disk 1" amnios_disk1/ amnios_disk1.ipf
T0: AmigaDOS
T1: Protection Long Track
T2-154: Psygnosis A
T155-159: Unformatted

Note how the more specific format specifier has prevented bogus identification of track 155, and also it handles the hard-to-accurately-detect copy-protection track. Nice.

If you have 68000 and Amiga coding experience you could even have a go at adding new track formats to my library.

[tect]

Nice! Thanks for the tutorial!
I tried some of my dumps:

Here AmigaDOS (easy one ):

Code: Select all

root@midget:/media/share/Daten/KRYO# mfmparse --format=probe RoadRaider/track RoadRaider.ipf
T0-159: AmigaDOS

Here CopyLock (says mfmparse, I don't know):

Code: Select all

root@midget:/media/share/Daten/KRYO# mfmparse --format=probe Paradroid90/track Paradroid90.ipf
Copylock: Short sector is only -4.68% different
T0: AmigaDOS
T1: Copylock
T2-159: AmigaDOS Extended
root@midget:/media/share/Daten/KRYO# mfmparse --format=probe TimeMachine/track TimeMachine.ipf
Copylock: Short sector is only -4.79% different
T0: AmigaDOS
T1: Copylock
T2-159: AmigaDOS
root@midget:/media/share/Daten/KRYO# mfmparse --format=probe Voyager/track Voyager.ipf
T0: AmigaDOS
T1: Copylock
T2-159: AmigaDOS

But I have some Disks which are completely nondos. Here mfmparse has problems...

Code: Select all

root@midget:/media/share/Daten/KRYO# mfmparse --format=probe Twinworld/track Twinworld.ipf
T0-1: AmigaDOS
T2-159: Unformatted*
** WARNING: 158 tracks are damaged or unidentified!
root@midget:/media/share/Daten/KRYO# mfmparse --format=probe IK/track IKplus.ipf
T0: AmigaDOS
T1: Core Design
T2: Unformatted*
T3: Unformatted
T4: Unformatted*
T5: Core Design
T6: Unformatted*
T7: Core Design
T8: Unformatted*
T9: Core Design
T10: Unformatted*
T11: Core Design
T12: Unformatted*
T13: Core Design
T14: Unformatted*
T15: Core Design
T16: Unformatted*
T17: Core Design
T18: Unformatted*
T19: Core Design
T20: Unformatted*
T21: Core Design
T22: Unformatted*
T23: Core Design
T24: Unformatted*
T25: Unformatted
T26: Unformatted*
T27: Core Design
T28: Unformatted*
T29: Core Design
T30: Unformatted*
T31: Core Design
T32: Unformatted*
T33: Core Design
T34: Unformatted*
T35: Core Design
T36: Unformatted*
T37: Core Design
T38: Unformatted*
T39: Unformatted
T40: Unformatted*
T41: Core Design
T42: Unformatted*
T43: Core Design
T44: Unformatted*
T45: Core Design
T46: Unformatted*
T47: Unformatted
T48: Unformatted*
T49: Core Design
T50: Unformatted*
T51: Unformatted
T52: Unformatted*
T53: Core Design
T54: Unformatted*
T55: Core Design
T56: Unformatted*
T57: Core Design
T58: Unformatted*
T59: Core Design
T60: Unformatted*
T61: Core Design
T62: Unformatted*
T63: Core Design
T64: Unformatted*
T65: Unformatted
T66: Unformatted*
T67: Unformatted
T68: Unformatted*
T69: Core Design
T70: Unformatted*
T71: Core Design
T72: Unformatted*
T73: Core Design
T74: Unformatted*
T75: Unformatted
T76: Unformatted*
T77: Core Design
T78: Unformatted*
T79: Core Design
T80: Unformatted*
T81: Core Design
T82: Unformatted*
T83: Core Design
T84: Unformatted*
T85: Core Design
T86: Unformatted*
T87: Core Design
T88: Unformatted*
T89: Core Design
T90: Unformatted*
T91: Core Design
T92: Unformatted*
T93: Core Design
T94: Unformatted*
T95: Core Design
T96: Unformatted*
T97: Core Design
T98: Unformatted*
T99: Unformatted
T100: Unformatted*
T101: Core Design
T102: Unformatted*
T103: Core Design
T104: Unformatted*
T105: Core Design
T106: Unformatted*
T107: Core Design
T108: Unformatted*
T109: Unformatted
T110: Unformatted*
T111: Core Design
T112: Unformatted*
T113: Unformatted
T114: Unformatted*
T115: Core Design
T116: Unformatted*
T117: Core Design
T118: Unformatted*
T119: Core Design
T120: Unformatted*
T121: Core Design
T122-124: Unformatted
T125: Core Design
T126-127: Unformatted
T128-131: Core Design
T132: Unformatted
T133: Core Design
T134: Unformatted
T135-136: Core Design
T137: Unformatted
T138-139: Core Design
T140-142: Unformatted
T143-144: Core Design
T145: Unformatted
T146: Core Design
T147-150: Unformatted
T151: Core Design
T152-153: Unformatted
T154: Core Design
T155: Unformatted
T156: Core Design
T157-159: Unformatted
** WARNING: 60 tracks are damaged or unidentified!

Unfortunately I have not too much time before Christmas, but I'll try to write those ipfs and check the results this weekend.
Too bad I have no clue how to programm a seriuos piece of software, so I can't help you here...
By the way: The syntax of mfmparse is a litte strange, I have to use "DirOfStreamfiles/track" as source, not "DirOfStreamfiles/" if the "trackxx.x.raw" files are in the directory "DirOfStreamfiles"

[keir]

Okay, thanks for the update. I have added the three Copylocked games (Paradroid 90, Time Machine, Voyager) to the mfmparse_formats file.
You can update your version of the mfmparse tool as follows (as it happens I've also fixed some bugs in my IPF encoder ):

Code: Select all

cd Amiga-Disk-Utilities
git pull
make clean
su -c "make install"

Twinworld I can confirm has a new disk format which I will have to take a look at. I don't think I've done any Ubi Soft games before so possibly this is a format they use more widely.

IK+ I'm afraid I only have an image of the budget re-release, which is plain AmigaDOS format. PM me about this one if you want me to take look.

[keir]

By the way: The syntax of mfmparse is a litte strange, I have to use "DirOfStreamfiles/track" as source, not "DirOfStreamfiles/" if the "trackxx.x.raw" files are in the directory "DirOfStreamfiles"

Yeah, mfmparse is dumb like that. You have to specify the whole filename up to the bit that changes (the xx.x.raw part). It sort of makes sense, if you are TAB-completing the filename in Linux.