New to KryoFlux, what can (and can't) it do?

All questions about how to use KryoFlux go here.
User avatar
Modeler
Posts: 8
Joined: Sun Feb 04, 2018 9:49 pm

New to KryoFlux, what can (and can't) it do?

Post by Modeler » Mon Feb 05, 2018 1:17 pm

Hi all,

I've been enjoying dumping / remastering some of my original Commodore 64 disk games using NIBTools, which is awesome but it can't do everything I would like it to do (I cannot write later V-MAX and Vorpal protected disks no matter what I try). So I invested in a KryoFlux board in the knowledge that this is as good as it gets, plus it would work with disks from the Amiga, Atari ST, PC and beyond. I already have my board but compatible 5.25" DS/HD drives are not cheap these days, so I'm still saving for one. In the meantime I have a (supposedly) compatible 3.5" drive, so will be finding my way with some Amiga and ST disks first.

I've read the manual, help page and a few posts on this forum, but I'm still not entirely clear on how it handles different disk and image formats. I understand that DTC supports some 40+ image formats, which are used to decode the streams and store the data in specific formats (for use with emulators and such), with some containing mastering information for writing images back to real disks e.g. type 22a.

Here's the big question. I (perhaps naively) thought that as long as your drive can physically read and write the required number of tracks, you could dump a disk at such a low level (as a raw stream) that – regardless of its format – you could then write the dump back to a new disk. Is that correct? If so I can appreciate the disadvantages of this method, I read somewhere that this was akin to copying a VHS tape to another VHS tape, noise and all. I would still like to know if it's an available option and a way to duplicate a disk from e.g. a BBC Micro and have the copy (hopefully) work on a real machine.

Furthermore, the GUI shows two output formats "KryoFlux stream files (preservation)" and "KryoFlux stream files (format guided)". Does anyone know what the difference is please? I thought stream files were format-agnostic by definition, so I'm would be grateful if someone could educate me a little here. Apologies if I've missed some key information somewhere, just a tad confused and need to align my expectations before I go any further. :)

Thanks!

SomeGuy
Posts: 122
Joined: Wed Feb 18, 2015 8:18 pm

Re: New to KryoFlux, what can (and can't) it do?

Post by SomeGuy » Tue Feb 06, 2018 12:26 am

The "KryoFlux stream files (preservation)" format will save a stream file image of the disk - this includes the entire low-level "flux" signal as read by the drive.

In theory this can be written back to a disk. However, there are a few caveats: Yes, it sort of like copying VHS video tapes as it preserves "noise" and the new disk media may add "noise". You must make sure the saved image actually contains what it is supposed to - some copy protections may not be read in properly by your physical drive. Kryoflux currently has some limitations writing things back, such as the inability to "splice" tracks that are not index aligned.

Unfortunately, the supported decoding formats only include non copy-protected formats. So if your disk is copy protected, then the only real choice is Kryoflux stream file.

The "KryoFlux stream files (format guided)" is a little different as it saves both a stream file and a decoded image at the same time. Because the image decoding can retry read errors, the stream file will omit any initially bad reads if a read is ultimately successful.

Personally, I prefer to save just a "preservation" stream first. This way the disk won't be kept spinning (and potentially disintegrating) as long, and I can go back and analyze/decode the preservation steam later without the disk at all. This is especially useful if I have no idea what encoding, density, or sector size(s) the disk may contain.

Now, if the preservation stream shows a known format that the KF supports, with a few read errors that a bit of disk cleaning and retrying doesn't fix, then it is time to break out the format guided stream.

User avatar
IFW
Posts: 2858
Joined: Mon Nov 08, 2010 2:42 pm

Re: New to KryoFlux, what can (and can't) it do?

Post by IFW » Tue Feb 06, 2018 1:01 pm

As SomeGuy said it can function as an "analog copier", but currently the tracks to be copied have to be index synced. This is a limitation of the host soft software, and will be changed in future host software updates.
For Amiga and Atari ST (and CPC, Spectrum+3 etc) I'd recommend writing IPF files back to your disk, the host software natively supports this including writing any kind of copy-protection and/or custom format the program might use.
You can perfectly image your C64 original disks as well, and convert them to extended g64 images that you can write back to real disks. That includes V-Max, Vorpal and tons of other custom formats :)
Note, when writing any kind of disk data either IPF or G64-ext there is no requirement whatsoever for the data to be index synced - that is specific to the stream rewrite (aka analog copy) option.

For emulation (rather than writing a real disk) you can use a format called CT Raw which can be used by all emulators using the SPS Decoder Library for accessing disks as a disk image, e.g. Amiga, Atari ST, CPC, Spectrum+3.
There is also a PC emulator that can use stream files directly.
In C64 emulators you can use both the normal g64 as well as the host software (DTC) generated g64-ext files as disk images. I recommend only making g64-ext files as they can actually be reliably written back to real disks, since they contain additional information gained during format analysis.

If you image disks containing commercial games or applications always use stream files; you can convert your stream files to any other format and the number of formats supported increase eventually. Think of stream files as raw photos or uncompressed audio samples; you can convert them to something else, but it's always a lossy, one-way conversion - there is no way to get your original stream data back.
For professional archival work it is also advisable to use stream files; that way if there is any issue found you can always refer/decode the original sample data.
Note, that it is significantly faster to image a disk if you do not use stream files, as stream files contain 5 revolutions worth of data for each track (or more as per user setting), while normal disk reading requires 2 revolutions sampled per track at most - but we can't recommend doing this, as you'll only have decoded/converted data from the tracks and a disk that may be readable one time may become unreadable eventually if you ever want to sample the real stream data.

User avatar
Modeler
Posts: 8
Joined: Sun Feb 04, 2018 9:49 pm

Re: New to KryoFlux, what can (and can't) it do?

Post by Modeler » Wed Feb 07, 2018 12:09 am

Great answers, thank you both! It seems I made a good purchase, I'm a Linux guy and at home on the command line so I'm not afraid to experiment with DTC a little. This threw me a bit though:
SomeGuy wrote:
Tue Feb 06, 2018 12:26 am
Unfortunately, the supported decoding formats only include non copy-protected formats. So if your disk is copy protected, then the only real choice is Kryoflux stream file.
Just to be clear, you mean if the disk is copy-protected and isn't a format the DTC supports (e.g. BBC Micro), then stream file is the only option? What format would you use for a non copy-protected BBC Micro disk please? I don't think DTC can create SSD files that BeebEm uses can it?

Similarly I don't see any D64 support for C64 games but it does handle Amiga ADFs. In an ideal world, KryoFlux would be a like a universal ZoomFloppy for many different systems (rather than just Commodore 8-bit). Or am I asking way too much from this board?

In any case, I think my V-MAX and Vorpal disks can be decoded (types 19 and 18 respectively) and properly written back to new disks, which is great. It struck me as odd that many disks with different copy protections can be archived as type 22a, yet DTC needs specific image types for others. Odd because I don't understand the structure of the data on magnetic disks like you guys. :oops:
IFW wrote:
Tue Feb 06, 2018 1:01 pm
For emulation (rather than writing a real disk) you can use a format called CT Raw which can be used by all emulators using the IPF library for accessing disks as a disk image, e.g. Amiga, Atari ST, CPC, Spectrum+3.
There is also a PC emulator that can use stream files directly.
So this would be type 2 (CT Raw image, 84 tracks, DS, DD, 300, MFM)? Would I just name output files made with with this format as e.g. dump.ipf? Sorry if I need this stuff spelling out to me, but I guess I want to know what makes it an IPF file. Surely this format can't be used for anything old disk. Does it work for more obscure 3.5" disk types e.g. Acorn Archimedes (for the purpose of writing back to disk, rather than to use with emulators)?

Thanks for bearing with me, I do appreciate all the help. It's important that I learn as much of this stuff as I can now, so that I can enjoy using the board and not be sat here wasting evenings scratching my head over non-issues. I nearly titled this thread KryoFlux or Cry "OH F*CK" :D

SomeGuy
Posts: 122
Joined: Wed Feb 18, 2015 8:18 pm

Re: New to KryoFlux, what can (and can't) it do?

Post by SomeGuy » Wed Feb 07, 2018 4:27 am

Modeler wrote:
Wed Feb 07, 2018 12:09 am
Just to be clear, you mean if the disk is copy-protected and isn't a format the DTC supports (e.g. BBC Micro), then stream file is the only option? What format would you use for a non copy-protected BBC Micro disk please? I don't think DTC can create SSD files that BeebEm uses can it?

Similarly I don't see any D64 support for C64 games but it does handle Amiga ADFs. In an ideal world, KryoFlux would be a like a universal ZoomFloppy for many different systems (rather than just Commodore 8-bit). Or am I asking way too much from this board?
What I mean is that the formats I am familiar with (I am not familiar with the C64 or ADF formats) are just unstructured dumps of all decoded sectors. Any copy protection or special sector organization is lost.

For example, lets say I dump a copy protected word processor from an IBM PC 360k disk. The protection consists of a single missing sector at the end of the disk (I may not know this as I am dumping the disk). Decoding as an "MFM" format, I will get a 360k disk image containing the program and data, but if I load that image in an emulator, it won't run. Although if I just needed to get an text document off of the disk, that might be enough.

The thing is there are thousands of different disk formats, many of which were used only by a tiny number of obscure proprietary machines. On top of that there are a vast number of unique copy protection schemes. It is impossible for the Kryoflux to recognize them all, but the idea is the Kryoflux can still archive the raw bits until someone chooses to implement support.

On the one hand, it would be nice to see Kryoflux become an all-in-one solution for the more common formats, but on the other hand there is the philosophy of doing one thing and doing it well.

A number of third party programs help take up the slack, providing additional analysis, visuilization, conversion, and automated decoding tools.

A quick lookup of the BBC Micro disk format suggests that a standard disk is FM encoded, 40 tracks, single sided, with 256 byte sectors.

User avatar
IFW
Posts: 2858
Joined: Mon Nov 08, 2010 2:42 pm

Re: New to KryoFlux, what can (and can't) it do?

Post by IFW » Fri Feb 09, 2018 3:43 pm

BBC Micro disks should just work fine as generic FM formats (-i3) - if not, please post the problem image :)

There are hundreds of possible format permutations used by FM and MFM controllers - like the one in BBC Micro - and all of those are simply referred to as generic FM/MFM by DTC.
So while DTC could list you hundreds of different computing platforms using variants of these, in principle they all use a hardware based controller that can only read (we are not talking about copy-protection here though!) disk content encoded in a pretty much identical way.
While it would look really cool if we added new platforms supported every day to the DTC help, instead we just opted to say generic FM/MFM and be done with it - easier for everyone :lol:

As a base concept, DTC can create standard sector images for pretty much any kind of side/track layout; you can change the "normal" values with the -oo option.
Standard sector images is what almost all emulators expect to use when not dealing with copy-protected software.
They are simply all the sides, track and sectors of a disk fully decoded and presented in a specific order in a single file.
Examples are ADF, D64, IMG etc. - they are essentially identical in principle just use a different name for a different target platform.

You can create D64 files easily: since that happens to be a CBM DOS sector image, use the format -i6 or if you want a D64 image with error map, use -i6a.
Note, there is no requirement in DTC for using a specific extension hence image types (like -i6) are referred to by what they do/what their content is, rather than what their file extension is - feel free to use an extension you like or your emulator/tool expects.

User avatar
IFW
Posts: 2858
Joined: Mon Nov 08, 2010 2:42 pm

Re: New to KryoFlux, what can (and can't) it do?

Post by IFW » Fri Feb 09, 2018 4:35 pm

To clarify: DTC supports tons of formats, but some of the formats listed are actually sets of various formats related to the image format (-i) selected.
e.g. generic FM/MFM covers a lot of possible permutations for disk controller parameters.
Other good examples are V-MAX!, RapidLok or Vorpal images; in reality we are talking about at least 6 iterations for RapidLok alone, but DTC once knows that the image is expected to be RapidLok, autodetects the actual format variant and parameters used.
Please note, that many of the formats are intended for guiding the imaging process; check if the tracks imaged are actually error free - there is no way of getting this information without decoding the data in the first place.
You get the decoded data anyway, as a bonus or side effect. ;)
You do not have any valid use case for creating e.g. a RapidLok sector image - no emulator or other tool would ever be able to use such image, since it only contains decoded data, and the protecion format is essential part of the image.
What you can use a format like RapidLok for is, to:
- check that your stream file does not contain read errors (ie can be decoded error free)
- compare two different RapidLok images and see if they are identical, e.g. whether you have a new verison of a game or not, if you are collecting ;)
- just enjoy looking at decoded data that would be very well hidden otherwise :D

So as a practical example, let's say you have a C64 disk with RapidLok format, and you want to archive it.
1, dump the disk to stream files
2, check the stream files against RapidLok format (-i16) and redump the tracks that do not decode error free.
Alternatively, add the RapidLok format to your imaging command, so DTC can autodetect errors while imaging the disk, and retry bad tracks automatically. This is why the manual refers to the image formats as guide formats; they guide the imaging process, making it possible to retry bad track reads.
Now you should have an error free image with stream files for all the tracks on the disk. From this point you can:
3, convert your stream files to a g64-ext image (-i22a) and use it in an emulator like VICE, or write it back to a real-disk.

Hope this helps :)

User avatar
IFW
Posts: 2858
Joined: Mon Nov 08, 2010 2:42 pm

Re: New to KryoFlux, what can (and can't) it do?

Post by IFW » Fri Feb 09, 2018 5:00 pm

Another example:
You have a copy-protected Amiga game.
1, create stream files; if the disk is likely to have AmigaDOS tracks mostly, add -5 as well as a guide format.
2, convert your stream files to a CT Raw image (-i2) and use it in a recent version of UAE. You will need to add the SPS Decoder Library (open source) to your emulator to make the image readable; this is specific to your platform.
https://kryoflux.com/?page=download

Similarly, you can use CTR images in Atari ST (Hatari, Steem), CPC (Caprice), Spectrum+3 (Spectaculator) etc. emulators, not to mention using IPF images in MESS (which I think is completely merged with MAME these days).
https://www.kryoflux.com/?page=links_emulators

You can use sector images in almost all emulators, except for the sector images actually really intended for guiding the imaging process, ie verification.
Sector image formats directly usable in various emulators and tools are the image formats -i3...-i11, -i13, -i14 (including alternates ending with a letter, e.g. -i3a).
All other formats are intended to be guide formats or conversion formats.
The converson formats are: -i2 (to CT Raw), -i22 (to G64), -i22a (to G64-ext)

User avatar
IFW
Posts: 2858
Joined: Mon Nov 08, 2010 2:42 pm

Re: New to KryoFlux, what can (and can't) it do?

Post by IFW » Fri Feb 09, 2018 5:12 pm

As for IPF files: there are literally thousands of them out there that are guaranteed to be 100% identical to the original disk they were imaged from.
IPF files are also integrated to emulators via the SPS Decoder Library - which I called IPF library due to bad habit before ;)

drdanj
Posts: 39
Joined: Sat Dec 29, 2012 9:47 pm

Re: New to KryoFlux, what can (and can't) it do?

Post by drdanj » Fri Feb 09, 2018 5:31 pm

I use dtc to create bbc micro files all the time :) - ssd is single sided,FM 80 track, 256byte sector, 10 sectors/track. dsd is the same, but with the sides interleaved.

Post Reply