KryoFlux Support Forums

Thanks for the new release 2.72 of DTC.

I am particularly interested by the new graph capability. I have read carefully the documentation and I would appreciate if you could clarify some points.

First I do not understand the meaning of “Heat map” graph. I have looked for definition at a dictionary (http://www.wordreference.com/enfr/heat%20map) and I can see a lot of “heat something” (e.g. heat engine, heat cramp, heat lamp, …) but no heat map. I suspect that you have called the flux reversal scattered chart like this to indicate how “hot” are the transition?

Now about the graph content I first an easy question: in the scattered chart the flux dots are mostly in blue but some are in red (and may be some other color?), what is the meaning of the colors?

About the “consistency bars”: You indicate that “Each horizontal bar” represents one revolution. I only see one big green bar so I suspect that this is because there are 5 horizontal bars that are merged together without separation? So if inside this big bar I see some red dots or lines their locations are based on inconsistencies for horizontal position and revolutions for vertical position.

You indicate that some copy protections like weakbits and Non flux area produces inconsistent data but I believe this is not the case for NFA. About the weakbits if they are obtained by partially unformatted areas we get indeed inconsistent data and they are clearly indicated in red in your graph. But for what you call programmed/predictable weakbits (Dungeon Master) there is no indication of inconsistency. I first thought that you had programmed a special detection for DM but to better understand I did a test on “DrT protected diskette” (described in detail in my protection document page 57) that use a totally different scheme of sliding predictable weakbits (you only get pseudo randomly $00 or $01 bytes) and here again you do not detect any inconsistency. Therefore, I would be interested to know what kind of analysis you are performing to detect inconsistencies, I suspect that you use some sort of correlation analysis?

On page 28 the d) paragraph is hard to understand but I think I got it. However, the black line drawn on the graph is a bit mysterious. I guess you want to indicate that if “YOU” can trace an uninterrupted line in the consistency graph you have some chance to read correctly, but if this line is interrupted by a complete vertical line you are out of chance.

Also not related I have a question about image type. When and why you use 0a type versus 0 type?
What is the difference in the output is you use “–fstreamfile –i0 –i4” and “–fstreamfile –i0a –i4”?
What is the difference in the output is you use “–fstreamfile –i0” and “–fstreamfile –i0a”?

Aufit graph

Heat map graph

https://en.wikipedia.org/wiki/Heat_map
It's fairly common in e.g. console performance profilers.

The colours change as a specific point/area gets more and more hits, eg you have colder (less hits) to warmer areas (more hits) represented by different shades, or intensity etc.
DTC actually renders the graphs in a different colour space for this and that gets converted back to RGB.

Consistency:
Yes, horizontal is the position on the track and vertical position is the specific revolution affected by an inconsistency.
Ideally, what you see is what you posted - ie large green bar, without any hint of revolutions
Once you can see the revolutions in the bar it's probably a good idea to clean the disk and redump it...

There is no inconsistency in the data used to generate the predictable weakbits (assuming you have a good disk/image), therefore you should see that the data matches, ie all green.
There is no special detection whatsoever for things like that; it just works.
DTC is now capable of perfectly matching each flux reversal on each revolution of samples, regardless of how bad the samples are, weakbits, NFAs, index signal jitter, wow/flutter or anything else.
The consistency graph is based on the feedback from that analysis; ie areas without an acceptable match are red. Unless you really magnify the data it may not be obvious why a match is not acceptable - but rest assured if it's marked there is a problem

Jeff Del Nero uses in the HXC 2001 emulators a matching technique similar to what you describe (http://hxc2001.free.fr/floppy_drive_emulator/).

What you call predictable weakbits seems to correspond to what I call “flux reversal in ambiguous area”. The most famous case on Atari is the Dungeon Master protection. I thought that was the only one on Atari because the protection is copyrighted. But almost all programs released by DrT uses a very similar protection but with a different pattern (so it does not infringe the copyright). Basically in both cases you have slowly sliding flux transitions crossing the edge of the inspection window. As you can see in the following picture in case of the DrT protection the sliding flux transitions follows what I call a “fish bone” pattern resulting in bytes decoded as $00 or $01 (only these two characters). This pattern is then followed by fuzzy bits obtained by unformatted area. In both cases (DM & DrT) the flux transitions spacing are almost equals from revolution to revolution even though they result in different bytes. It is therefore completely normal in that case that DTC detect them as consistent even though they result in random bytes decoded.

I can understand how the matching is done when no inconsistency is detected, but how do you “resynchronize” the data from two revolutions after an inconsistency is found (e.g. unformatted area). Is this done only on transitions position in the track (of course normalized to a perfect 300 rpm timing)? Index signal jitter, wow/flutter are probably also challenging.

I may consider adding this kind of analysis in Aufit to provide extra information about the “quality” of the dump. Currently when Aufit find a bad sector on a revolution it tries to find if this same sector is good on at least one of the other revolutions.

It's very computationally intensive to do. Dtc applies logical reasoning, how a real person would do it, before dying from the boredom
It uses different techniques and keeps on trying until it can positively reason for a match or positively reason why something can't be a match.
It is also able to do several steps and backtrack if needed.
Dtc is effectively solving a (sometimes) very complex puzzle or crosswords etc. with lots of variables.
What makes it even harder is recognising pathological cases that can't be resolved as well as recognizing possible shortcuts that decrease the processing required.
I think we had a beta with this feature in development for over 2 years...

Hum does not seem like it’s worth it to add this capability in Aufit.

It also looks like you are making a tighter checking on reading the raw files in DTC. As you may know Aufit can read .scp files to produce KF .raw files. I have added this capability so I can use DTC to generate .ctr files from .scp file for further analysis with CTA.

DTC 2.51 reads these files without problem but DTC 2.72 complains with error message “Some index positions could not be found”. Did you add more checks when reading raw files?
Attached a track that reads with 2.51 but not with 2.72

Any pointer on explanation about -i0 and -i0a difference?

Yes, I had to, as some conversions generated stream files that crashed DTC at random points in the analyser pipeline... or worse, DTC did not crash but the results were incorrect/random.

From the 2.72 release notes:
- prevent using a stream file with broken index signal data (generated by
some third party tools)
- ensure that broken index data does not cause invalid calculations and
NaNs for the cell conversion and thus later crashes at random stages of the
analyser pipeline

...and here is the original bug report:
https://forum.kryoflux.com/viewtopic.php?f=3&t=1398

-i or (-i0 in its full form) means image every track regardless of whether it's part of the selected guide format(s) or not - unless restricted by other (not the format) parameters.
-i0a means image only the tracks that are part of the specifications of the guide format(s) selected. This was how originally DTC worked, but that meant you had to add another "dummy" format using all tracks (typically -i2) for preservation quality images containing all tracks and adding -i2 was easy to forget and inconvenient to add all the time, e.g. when preserving C64 disks.
So basically by using -i0a you are specifically asking for trouble