Apple ][ disks

All questions about how to use KryoFlux go here.
Keatah
Posts: 1
Joined: Fri Oct 29, 2010 2:16 am

Apple ][ disks

Post by Keatah » Fri Oct 29, 2010 2:23 am

I have an extensive collection, more than 4000 Apple ][+, //e, //c, & IIgs disks. Many of them are 5.25, with some being 3.5..
These are disks I'd like to make available to the community!

What hardware and software will I need in addition to a Kryoflux board in order to begin a preservation project?
When I say that I mean.. I have a winXP system with usb ports, tons of hard disk space.
I have several Apple ][ series computers along with several of the Shugart 5.25 drives -- the original Disk ][ drives.

Beyond that, what will I need to get in terms of cables or other disk drives or software?
And I'm looking for a simple way to do this, simple enough that I can focus on the archival aspect and worry about cataloging the disks correctly with documentation.

User avatar
mr.vince
Posts: 2119
Joined: Tue Oct 05, 2010 5:48 pm

Re: Apple ][ disks

Post by mr.vince » Sun Oct 31, 2010 9:32 am

Ok, hardware: To access the disks, all you need is KryoFlux and a standard 3.5"/5.25" drive. KryoFlux can make a forensic dump (STREAM files or, once implemented, DRAFT) without knowing about the disk format. It's like a language you don't understand - you can still make a good sounding recording and then later ask someone to translate it for you. DTC, KryoFlux' software, can also convert to most popular sector dump formats:
  • FM sector image, 40/80+ tracks, SS/DS, DD/HD, 300, FM 3a: FM XFD, Atari 8-bit
  • MFM sector image, 40/80+ tracks, SS/DS, DD/HD, 300, MFM (actually PC, Atari ST, many samplers etc.)
  • MFM XFD, Atari 8-bit
  • AmigaDOS sector image, 80+ tracks, DS, DD/HD, 300, MFM
  • CBM DOS sector image, 35+ tracks, SS, DD, 300, GCR
  • Apple DOS 3.2 sector image, 35+ tracks, SS, DD, 300, GCR
  • Apple DOS 3.3+ sector image, 35+ tracks, SS, DD, 300, GCR
  • DSK, DOS 3.3 interleave
  • Apple DOS 400K/800K sector image, 80+ tracks, SS/DS, DD, CLV, GCR
  • Emu II sector image, 80+ tracks, DS, DD, 300, FM
I must admit I know little about the Apple II and it's disk formats. The main problem is there are so many different ones for Apple and I better say "not sure" before I say something wrong. We do support Apple DOS 3.2, 3.3 and variants. Not sure if these are coming from the Apple II because I have in mind that Apple used something even more strange before, and had own "intelligent" drives (like the C64's 1541) which led to a colorful selection of really innovative copy protections. But we're eager on finding out...

In regard to a preservation project... you might know we started one some time ago. The main problem will be to differentiate the good from the bad and to describe if bad in regard to recording scheme is intentionally bad. What I've been pointing out above is getting access to the data. Actually preserving it is a completety different thing. You must make sure the data is unmodified and genuine. We have a software called the Analyser that we use for this, something that other projects lack (to my knowledge). I've therefore seen many damaged or at least modified dumps of so called preserved stuff. If this is sufficient for you, you can continue and use sector dumps (like .ADF, .D64) or "raw" dumps (e.g. .G64) but at least I'd then want to differentiate between "collected" (on the shelf), "transferred" (from floppy to image) and "preserved" (unmodified and genuine).

hkzlab
Posts: 11
Joined: Sun Oct 31, 2010 7:52 pm

Re: Apple ][ disks

Post by hkzlab » Fri Nov 05, 2010 10:19 pm

Want to chime in to say that i got my Kryoflux today here in Italy, and i'm very satisfied with it! :D

Also, i have an original copy of Leisure Suit Larry 1 for Apple IIc to dump around here somewhere, if i remember correctly it's not AppleDOS 3.2 format and i wasn't able to get anything out of it with the catweasel.
Is someone (SPS?) interested in STREAM dumps to analyze when i'll be able to dump it?

Thanks again for this great device!

User avatar
karadoc
Posts: 137
Joined: Sun Oct 31, 2010 9:12 pm

Re: Apple ][ disks

Post by karadoc » Fri Nov 05, 2010 10:27 pm

Yes, absolutely. Please contact us at the form about dump submission:
http://www.softpres.org/contact

Thanks!

TeaRex
Posts: 120
Joined: Tue Nov 30, 2010 5:36 am

Re: Apple ][ disks

Post by TeaRex » Tue Nov 30, 2010 6:20 am

Just FYI... Apple II drives are not intelligent at all, they use a low-level bitstream transfer from/to the host computer. The early and widespread "Disk ][" drive contains only four small ICs. There is one exception, the "Unidisk 3.5", which IS an intelligent drive containing a 6502 CPU that is faster than the one in the Apple to keep up with the data, but this drive was not used very much. Normal Apple 3.5" drives (mainly used with the intrinsically faster Apple IIgs or with "intelligent" controllers) and all their 5.25" drives, including the "Unidisk 5.25", are unintelligent drives.

However the standard Apple controller ("Woz Machine") does very little in hardware (for example, the software directly controls the four inputs to the stepper motor) and is thus quite a bit more flexible than the FM/MFM controllers used in IBM Compatibles. This is what makes the plethora of protection methods possible, including stuff like quarter tracks and spiral tracks that are probably beyond even KryoFlux, at least with unmodified PC drives used for dumping. Luckily most of the meaner methods were used only for very few games.

There are only two STANDARD low-level formats used by Apple II for 5.25" disks (usually called 13-sector and 16-sector, or "DOS 3.2" and "DOS 3.3"), and two for 3.5" disks (one common, very similar to Mac 800K, and one rare, identical to the well-known ubiquitous "1.44 MB" MFM format). The number of standard high-level formats is larger, but this shouldn't concern KryoFlux at all.

The 13-sector formats was only used by OS software released from 1978 to 1980, i.e. by the short-lived and buggy DOS 3.1 and by DOS 3.2, and IIRC early versions of Apple CP/M. The 16-sector format was used from 1980; by DOS 3.3, all common versions of Apple CP/M, Apple Pascal (which is its own OS), and of course all versions of ProDOS. Some commercial disks sold shortly after the switch to 16 sectors used a special track 0 that was bootable on both kinds of disk controllers (it contained both a "13-sector" sector 0 and a "16-sector" sector 0), then used 13-sector format for the rest of the disk, since the 13-sector controller is physically unable to read 16-sector data, while the reverse is not true.

Since the 13-sector format was the main format for only two years, and before the gaming industry really took off, it is not so terribly widespread. Support for the 16-sector format would cover the vast majority of unprotected 5.25" Apple II disks. As the common 3.5" low-level format is almost the same as Mac 800K (the only difference is some flag bytes IIRC), it should be supported already, I guess.

If you want me to describe the details of the Apple low-level formats just ask. Also I recommend searching the net for the book "Beneath Apple DOS" which exists as a PDF, but note that early editions of the book contain some misunderstandings of the actual flux transition structure used on the disk. To get an idea for how the controller hardware works, the drive chapter of "Understanding the Apple II" by Jim Sather is invaluable. It exists as PDF as well.

Oh and I know you guys are busy, so just ask me when/if you get time.

Edit: I re-read the second post and I see you already support DOS 3.2 and DOS 3.3, as well as 800K 3.5" and of course MFM. So, all bases are covered as far as standard formats are concerned.

User avatar
IFW
Posts: 3079
Joined: Mon Nov 08, 2010 2:42 pm

Re: Apple ][ disks

Post by IFW » Tue Nov 30, 2010 12:00 pm

All the formats you mentioned are supported which is not possible without completely understanding how the drives and software work in the first place - not just Apple ones, but many others including ones that have been completely reverse-engineered for the first time ever (E-mu Emulator I & II) by me.

The intelligent drive comes from C64 terminology: that was an euphemism for the need to do (almost) everything in software, in that case using the drive's dedicated cpu and ram.

(In theory) Apple protections could be just as inventive as C64 ones since you have direct access to the bitcell data - the only constraints are the physical properties of the drives and the disks used and of course the cpu resources that could be dedicated to the task... given that the decoding routines are cycle-counted for some Apple disk code I am sure it was a fun thing to do and some people just couldn't resist writing their own :)

Everything that can be read by a drive can be read by KryoFlux.
Stepping as far as I remember (was a while ago...) was very similar to C64 for the affected Apple system with half-tracks etc; using a 96 TPI drive for 48 TPI recording.
You can read half-tracks for C64 just like you can read half-tracks for Apple - using a 96 TPI drive.

Should you ever encounter anything that absolutely requires a different TPI resolution, you can always attach a drive that can read the disk, and then you can read the data that way using KryoFlux.

There are two unsupported things:
1, There was a proposed format with more speed zones on the disk, can't recall which machine it was for.
If it turns out it was used, it will be fairly trivial to add to the existing supported formats if a stream dump of such a disk ever gets submitted.
2, Some format has OS information stored in the sector unused area (524 bytes, 512 used normally).
Since there was no standard sector format proposed for this for emulation or otherwise it is supported (required for decoding and verifying the sector integrity anyway), but the extra data is not exposed.

Preserving the games using inventive protections etc is a whole different story... see C64.

TeaRex
Posts: 120
Joined: Tue Nov 30, 2010 5:36 am

Re: Apple ][ disks

Post by TeaRex » Tue Nov 30, 2010 6:20 pm

Yeah, I was more or less answering to the post by mr.vince above, but without carefully reading it in the first place. Obviously you must understand the formats before they can be supported. And since the disk read routines in the Apple can in fact use much more RAM than in the Commodore drives which only have 2K, you can probably do even more complicated stuff than on the Commodore if you want to. Also the Apple doesn't have the hardware SYNC detection that constrains you in some ways on the Commodore drives.

As for cycle counting, I believe it is in fact mandatory for write routines on the Apple, since the controller will only accept data from the bus at specific cycles. For reading you at least have to make sure you don't go beyond the number of cycles that pass between bits, or the time allowed between two checks of the data-in register (which leaves time just for one LDA abs,x and one BPL if I remember correctly).

As for stepping, I don't think the Commodore drives can hit quarter tracks as the Apples can do, since they don't talk directly to the stepper motor coils; I seem to remember that they use "step in" and "step out" signals only that are then converted by some 74xx junk to the stepper motor signals. Thus the step signals always step by half tracks. On the Apple you can magnetize two coils at the same time and keep them that way, thus stopping the head between two half-tracks. But as I said it was not a very common protection.

Now I'll stop writing stuff that you (IFW) already know ;-)

User avatar
IFW
Posts: 3079
Joined: Mon Nov 08, 2010 2:42 pm

Re: Apple ][ disks

Post by IFW » Tue Nov 30, 2010 7:03 pm

I am sure there are other people who don't know about these things, but are interested in the amazing world of floppy disks be at Apple or any other format - otherwise they wouldn't be on this board, it's quite a special interest one :lol:

The system uses a resync on MSB and they had to sacrifice 1 bit for that for each GCR entry while there is no need for that on a 1541 - but that has hardware assisted cell timing for reading and writing.

I think the cycle counting was probably not good enough in the write routines (although the instruction timings do add up correctly!) - I guess bus contention or something similar was not (properly) accounted for - they just got it working somehow and were happy with that ;)

This can be clearly seen when you take a look at disks written on the machine vs a professionally duplicated disk, e.g. duplicated on Trace. The latter has perfect cell timing while the former is of questionable quality timing wise - I am fairly sure the unaccounted jitter caused was the main reason they decided to resync on each MSB.

For comparison: a duplicated disk can always be decoded properly, regardless whether a resync happens or not on MSB - that's how the original KF host software did the decoding.
However, if you try to do the same with a home written disk, you'll randomly encounter decoding errors. I had a hunch that it was caused by imperfect timing during write, so I added MSB resync to the decoder - the result is the ability to decode the machine written disks...

TeaRex
Posts: 120
Joined: Tue Nov 30, 2010 5:36 am

Re: Apple ][ disks

Post by TeaRex » Wed Dec 01, 2010 2:00 am

OK, I'm breaking my promise and continuing my rants...

In the Apple controller the "MSB is always 1" rule is also used to delay the next bit-shifting by up to 3 cycles during reading, so that the fastest possible 6502 loop construct for checking whether a byte is complete (LDA abs,x + BPL, which takes 7 cycles per round, 3 more than there are in one bit time) can't miss a bit. And of course that loop itself relies on the 1 bit in the MSB, otherwise it would have to be much longer.

As for writing I'm pretty certain that the cycle counting must be correct insofar as the the controller will only accept a byte from the bus in one of every four CPU cycles (4 CPU cycles make one bit time) and will ignore writes to its data-out register that occur during the other three cycles. So you have to hit at least those 4-cycle boundaries with precision. But of course as long as it doesn't create more than two 0 bits in a row, you can let one or two of these boundaries go by unused and write the next byte four or eight cycles later, as is in fact done on purpose during the writing of the self-sync bytes. Maybe the write routines of one or more Apple OSs sometimes do that even during the writing of a block, and probably if some game uses its own write routine for high scores or whatever it is even more likely to have such "bugs". Apple DOS does contain a cosmetic bug in that it turns off writing before the last byte of the block trailer is completely written, but since this byte is never checked during reading, it is not a problem. I think ProDOS fixed this.

I doubt bus contention is a problem since the Apple is such a simple system. If you don't have special add-on hardware doing DMA, I don't think bus contention can even exist, as there is no such thing as the bad-lines or stolen cycles of the C64. Neither IRQs nor NMIs are used in the standard system; and AFAIK all OSs explicitly disable IRQ (which could be caused by add-on cards) during floppy access. Things are more complicated in the IIc and IIgs models, but I don't think they were ever as widespread as the simple II+ and IIe machines.

Some jitter will be introduced by the infamous "long cycle". Every 65th CPU cycle on the Apple is 1/7th longer than the normal cycles, and disk code doesn't "sync" to this pattern in any way, so the longer bit cells will be more or less randomly distributed. This serves to keep the phase of the NTSC color carrier signal the same in every line of video. It's contrary to the NTSC standard, but it makes color handling a bit easier.

Also the most common drive, the Apple Disk ][, is a belt driven mechanism with purely analog speed control, and thus probably has more natural jitter than the quartz controlled direct-drive mechanisms used by many other manufacturers.

But whatever the cause of the jitter, your solution sounds like the smart thing to do, since it does the same thing the original hardware does.

User avatar
IFW
Posts: 3079
Joined: Mon Nov 08, 2010 2:42 pm

Re: Apple ][ disks

Post by IFW » Wed Dec 01, 2010 12:06 pm

Thanks, very interesting - obviously I am only familiar with the disk system as that was needed.
I think that the long cycle you mentioned would explain the jitter, not to mention the drive control.

Post Reply