KryoFlux - USB Floppy Controller

[karadoc]

Hi,

We at the Software Preservation Society have been working on a system to help with our preservation goals, and now it is open to the public, we thought many people would find it interesting and useful to use themselves.

This system, which we have called "KryoFlux" is basically a way of reading your old floppy disks, no matter what the system they are for, with no regard for what disk format or copy protection they may contain.

See the YouTube trailer here: http://www.youtube.com/watch?v=kjfT-F0GUl4

KryoFlux is actually a software-programmable FDC (Floppy Disk Controller) system that runs on small and cheap ARM-based devices that connects to a floppy disk drive, and a host PC over USB connector. It reads (and in the future, will write) "flux transitions" from floppy disks at a very fine resolution. It can also read disks originally written with different (and even varying) bit cell widths and drive speeds, with a normal fixed-speed drive.

KryoFlux is available **FOR FREE** for private non-commercial use. You will however need to build or buy a board based on our open hardware design (or buy a development board which you can get right now). We are currently in negotiations with hardware manufacturers to allow you easy access to this hardware if you do not want to build it yourself.

KryoFlux supports dumping any floppy disk to “stream files”, which contain the low-level flux transition information, but it also supports output to a wide range of common “sector dump” formats to allow you to use your images right away in your favourite emulator.

More info here:
Basic Info: http://softpres.org/glossary:kryoflux
Beta 2 Release News Item: http://kryoflux.org/news:2010-02-18

 

[karadoc]

MINI FAQ:

Q. Why is it free?
A. Because we are a non-profit preservation organisation, and our ultimate goal is not financial, but to save all these wonderful games before they are lost.

Q. Do I need a special drive?
A. No. Just a standard PC drive and floppy cable is required to dump any media for that drive.

Q. Does it work with 5 1/4" drives?
A. Yes. Check 1min 10secs into the YouTube trailer.

Q. Will it work on a laptop?
A. Yes. It uses standard USB. We have it running on an EeePC.

Q. Why can't I just use the floppy drive in my PC?
A. The floppy controller in the PC can only read the very strict format PC disks, and not very much of anything from other platforms. Not only that, but any copy protection on disks is hard to extract properly without a "low level" read - which you can't do through a PC's FDC. For various reasons, we would strongly argue that images of disks read through a PC floppy controller are unsuitable for preservation.

Q. Is it Windows only?
A. The software is currently in beta, and only runs on Windows (32 or 64-bit, Windows XP or greater). The source to the host software will be available after we go final, and we are then hoping to get it ported to as many platforms as possible. The code was written with portability in mind.

Q. Is it command-line only usage?
A. For the beta, yes. However, we are working on a graphical user interface which will likely be available when we go final.

 

[donutty]

Nice. How many different (physical) disk interfaces will it have? I can see the need to read 8" disks too. Also, does it play cool chiptunes when reading a disk?

 

[karadoc]

Currently 3.5 and 5.25". Also, 3" shouldn't be a problem, as we have experience with doing that using our previous dumping solution, so I'm hoping we will get that one sorted pretty quickly. We would love to connect up an 8" drive to it, the problem is finding one to test with - the only ones I personally know of are in museums... We are also looking into Quick Disks, but again, the problem is finding a drive.

Edit: Sorry, re-read your question. The above is what it works with it currently. We are still negotiating the hardware manufacture, and our board design is still really a prototype. You can play with it now, but you need a development board (about 55 EUR), and wire your own interfaces to it.

We're also currently working on a graphical user interface that will work on top of the current command line software, so I'll take your feature request under advisement

 

[Chuck(G)]

Do you buffer the outputs of your controller and are they open-collector? Pullups on 8" drives are typically 150 ohms. At 5 volts, that's about 30 ma per output--a lot to ask from a simple microcontroller. How long can the connecting cable be?

 

[karadoc]

Hi Chuck. I hope that we have made it clear that the current design is very much a prototype, and the manual (worth a read btw ) specifies it is for modern drives only (not 8"). After referring to the hardware guys I can say that, yes, the design is open collector, and you are completely correct to bring this up, as you might fry the controller if you connect such a drive. We're working on a refined layout with Shottky stuff (e.g. 74LVC14 or similar) and level shifters that will take care of this.

Hopefully we will get this sorted out soon, because we'd love to get it working with 8" drives, again, assuming we can actually find one to test with... Most of the work, and the real magic, is in the software.

As usual, the connecting cable should be as short as possible. Since the flat ribbon cables are huge antennas, it depends on where you live and how the attached drive will behave when making it longer. If using a long USB cable, we'd recommend one with proper shielding and mount the board above, below, or just near the drive. Of course, making the USB cable longer than needed isn't recommended either.

 

[kyeakel]

Does the Software Preservation Society preserve software other than games? I see you have alot of Amiga games, but didn't find much else. Is there another archive section where one would find early PC, Kaypro, CP/M software?

Thanks,
Kipp

 

[kb2syd]

Hopefully we will get this sorted out soon, because we'd love to get it working with 8" drives, again, assuming we can actually find one to test with...

Being able to figure out WHERE development is taking place may make it easier to find an 8" drive. Setting your location in your profile would help. For instance, if you were within driving distance of me, you could borrow an 8" Tandon drive, with the understanding that it would be returned when developement was complete. I'm guessing others would be willing to help.

 

[Chuck(G)]

There have been several such projects here. During last year, I had posted a prototype that showed that it was possible to do this with just about any microcontroller that had sufficient memory and a capture mode for at leat one timer. I suspect that you've done this also. Phil Pemberton also demoed his FPGA-based model. And there's always the Catweasel.

Where you're going to have to spend your effort is to make the design robust enough to be able to drive just about any kind of drive, not just 5.25". For example, do you have sufficient sample memory to handle 2.88M or specially-formatted 3.7M floppies? Can you work with Drivetek 5.25" 6.6MB drives? Are your drivers and receivers replaceable, or do you have to replace the entire board if a driver output gets blown?

Software is still where the venerable Catweasel is lacking--and that's after, what 10 years? Writing many formats is not possible with the CW, mostly because no one's done software for it. I hope your device doesn't fall into the same trap.

All in all, perhaps the best solution will be a device that not only reads floppies but one that also provides a floppy interface for simulating them on vintage equipment. One may have a system, but not documentation or understanding of how the system works, so emulation isn't practical.

I'm not trying to disparage your efforts or discourage you, but rather to encourage you to take a wide view of the true issues of data preservation.

 

[karadoc]

Does the Software Preservation Society preserve software other than games? I see you have alot of Amiga games, but didn't find much else. Is there another archive section where one would find early PC, Kaypro, CP/M software?

You are right. Unfortunately, we have finite resources, and our focus is currently on games. That's not to say that we won't do anything else in the future of course, it's just that our preservation efforts are difficult enough just having games to do.

What you mention though really brings us back to one of the motivations behind KryoFlux. In order to do what we want to do for other platforms, we ideally needed something more that what we had. We have done a lot of work over the last couple of years, currently not public, adding support for other platforms to our preservation tech, and now with KryoFlux to aid us, very soon we will be able to ramp up our work in this area. Amiga was certainly the most difficult, technically speaking, so in retrospect, I'm glad we did that one first

 

[karadoc]

Being able to figure out WHERE development is taking place may make it easier to find an 8" drive. Setting your location in your profile would help. For instance, if you were within driving distance of me, you could borrow an 8" Tandon drive, with the understanding that it would be returned when developement was complete. I'm guessing others would be willing to help.

Well, thank you very much for your kind offer, but we are a bit too far to drive, and you might get wet. Most of us are in the UK and Germany.

 

[karadoc]

There have been several such projects here. During last year, I had posted a prototype that showed that it was possible to do this with just about any microcontroller that had sufficient memory and a capture mode for at leat one timer. I suspect that you've done this also. Phil Pemberton also demoed his FPGA-based model.

Nice. Yes, I think we have also spoken to Phil. USB controllers are becoming rather like buses lately, aren't they!

(hmm... that might be too much a British-ism, we have a saying "you wait for a bus for ages, and then several come along at once").

Where you're going to have to spend your effort is to make the design robust enough to be able to drive just about any kind of drive, not just 5.25".

As I said, we have been working on this. We have some professional hardware engineers working on it right now actually.

For example, do you have sufficient sample memory to handle 2.88M or specially-formatted 3.7M floppies? Can you work with Drivetek 5.25" 6.6MB drives?

We are currently looking at Quick Disk drives, which is an interesting one. I'm not sure about the ones you mention though since I haven't heard them mentioned in our group. We are really starting to get out of my realm of knowledge now I am afraid, and people are not around to ask. I'd be interested to know, was much commercial software available for those types of disks?

Are your drivers and receivers replaceable, or do you have to replace the entire board if a driver output gets blown?

Again, I am afraid this is outside my realm of knowledge, but thanks for making the point. As I said, we have hardware engineers now looking at it, and I'm sure they know what is best. I'll try and find out though.

Software is still where the venerable Catweasel is lacking--and that's after, what 10 years? Writing many formats is not possible with the CW, mostly because no one's done software for it. I hope your device doesn't fall into the same trap.

Well, software is exactly where our strength lies. We have spent nearly 10 years creating technology to analyse and verify disk data through flexible scripting (a bit like how Trace machines worked), checking whatever checksums and format structure there is. This includes the vast number of formats used on the Amiga, of which there are hundreds of very different ones (the Amiga's controller is almost non-existent, and formats can be specified in software).

We have a lot of experience in this area, so format support will not be a problem. We are also working on specifying an open format (DRAFT) for holding flux transition data. We currently capture the raw output of our device, but that is not exactly suitable for storage as it isn't hardware-agnostic.

All in all, perhaps the best solution will be a device that not only reads floppies but one that also provides a floppy interface for simulating them on vintage equipment. One may have a system, but not documentation or understanding of how the system works, so emulation isn't practical.

Nice idea, and this already exists: http://www.torlus.com/floppy

We do not have any intention of doing a multi-function device such as this. Apart from HxC being available, it seems to be to be a very different type of problem. I don't think people would mind using different solutions for different jobs rather than cram it all into one device. Of course, if something like that was available, I'm sure it would be popular - just not our thing.

I'm not trying to disparage your efforts or discourage you, but rather to encourage you to take a wide view of the true issues of data preservation.

Oh, I definitely appreciate your input, thanks.

The true issues of data preservation - I not sure what you mean by this, because I think that is exactly what we are trying to help solve - and we don't presume to be the end of the story or the final word on the matter. I suppose "true issues" is a bit subjective too, as it depends on what you are trying to achieve. Many librarians have told me that "meta-data is the main issue of digital preservation" for example. While I am sure that is a big problem, I think there are many other things that need solving which are equally important (like getting the data in the first place).

 

[Chuck(G)]

Nice idea, and this already exists: http://www.torlus.com/floppy

Not quite what I had in mind, although I'm familiar with the unit. What I was getting at was a unit that not only connects to a floppy, but also to a controller, as well as having either some local storage (e.g. flash) or a peripheral connection. The idea being that one could directly read or write directly from any connected drive or use the onboard images. The point being that to get the data to the emulator requires that one image the floppy somehow, which means purchasing yet another device and the accompanying software.

For an example of an "interesting" diskette drive, google "Kaypro Robie".

The true issues of data preservation - I not sure what you mean by this, because I think that is exactly what we are trying to help solve - and we don't presume to be the end of the story or the final word on the matter. I suppose "true issues" is a bit subjective too, as it depends on what you are trying to achieve. Many librarians have told me that "meta-data is the main issue of digital preservation" for example. While I am sure that is a big problem, I think there are many other things that need solving which are equally important (like getting the data in the first place).

In my experience (I've been doing data conversion work since about 1987), there are two general classes of users. The first just wants the data in some modern format; doesn't care about preserving the original format or the media. I've got stacks of old floppies that go to the landfill periodically that are of this nature.

The second is someone with equipment (usually industrial high-value gear) who is having trouble finding media or drives to replace what he has. Again, he doesn't care about the original medium, just its contents and that any replacement, whatever it is, work with his system. There are a bunch of floppy disk emulator offerings out there for mission-specific equipment, such as embroidery machines.

At least that's what I've been seeing.

 

[Raven]

YAY! A Win64-compatible floppy solution! When this can write to disks I'll be all over this like... something undefinable is all over something else! RAWR! Catweasel has failed me but this shall not!

 

[Chuck(G)]

YAY! A Win64-compatible floppy solution! When this can write to disks I'll be all over this like... something undefinable is all over something else! RAWR! Catweasel has failed me but this shall not!

Same-o, same-o. Someone's got to wirite the code--in some senses, writing for the Catweasel is easier than writing for a uC-based design such as the one on this thread. I write floppies using both CW MK1 and MK3 cards (I like MK3 better--it integrates better into PnP hardware). The latest rcopy job that I have is from hard-sector 8" diskettes running a wafer tester from the late 70s. So all this stuff about "CW doesn't write floppies" is stuff and nonsense. The MK4 looks to be even more interesting in that it has an FPGA that one can load with one's own code.

But the crux of the matter is that no one wants to write code--and the KyroFlux will probably have the same problem. Besides, an awful lot of people don't want to write floppies--they'd just like to read them and use them with emulation software.

So the moral is--if you want (fill in the blank), sharpen your pencil and get cracking instead of waiting for someone to do it for you. If you don't know how, learn--it's never been easier to learn how to do just about anything from tuning a piano to cleaning a septic tank, thanks to the web. And by and large, the tools are free.

 

[Raven]

Mind pointing me to some microcontroller-ASM (or whatever ya use) resources? I don't even know what's necessary. I think you need some kind of special development flash tool on a per-chip-you-want-to-code basis..

All I know is Python.

 

[Chuck(G)]

If you are interested in the KyroFlux (I believe it's an ARM7 design, but you should check), there are lots of GNU tools:

http://www.siwawi.arubi.uni-kl.de/avr_projects/arm_projects/#winarm

There are lots of tutorials. If you want to play with a small development system, there are lots of those. Futurlec has several for very little money.

If you want to fool with the Catweasel, take a look at the source code that comes with Tim Mann's DMK tools or the Linux drivers. I think Tim uses DJGPP C for his stuff, where the Linux stuff uses GCC. Both are free.

There are tons of C tutorials.

 

[Raven]

Ideally I'd get a Catweasel, but they are so damn expensive. The nice thing about them is that they combine an updatable floppy controller with Atari-style joystick ports and SID playback from real SID chips. Considering that I have only one free PCI slot that's a pretty good amount of function from that one slot.

The thing that draws me to the Kryoflux so much is that it's USB, so it won't take up that single PCI slot. I can mount a USB controller there instead and use the internal USB port on my USB PCI card to mount the Kryoflux on the inside of the case hooked up to floppy drives mounted in standard brackets

That dev board you linked to is pretty damn cheap - that board has the necessary CPU on it already? If so perhaps I'll pick up one of those to get my feet wet in uC programming so I'll be more confident before dropping more money on a Kryoflux or Catweasel.

 

[timofonic]

Any news about the project? I see interesting stuff in the WIP site but they seems so thechie oriented.

IPF could be cool for SPS, but to my knowledge it's a propietary format as it requires a closed source library to be readed (and I hate that A LOT, despite the reasons for it).

http://www.softpres.org/wip

 

[karadoc]

The current state of things is that we are testing boards produced by our hardware manufacturer. As you can probably imagine, this needs quite some time to avoid any unfortunate (and expensive) mistakes. We are currently waiting on the next revision boards that clear up some minor issues, which I think we're getting next week. The next thing you are likely to see is a simple KryoFlux webpage to get an idea of numbers for the first production run.

As to IPF, you are of course absolutely correct. We have known since the beginning that it was a pretty ridiculous state of affairs for a image format intended for preservation. Fortunately the reasons behind it have disappeared over the years, and it now basically comes down to finding the time to sort it out (e.g. sorting out the source code license to the IPF library). I think we have mentioned elsewhere that we planned to sort this out after we finished KryoFlux (nearly) and multi-platform support in our (logically unrelated) analyser+IPF technology (done). The source code will be made available, which is the most important thing, but I should probably state that people should not expect the source code license to GPL-compatible.

I should state here (as we have confused people on this before), that IPF is a mastering format. It is similar to how Trace prepared disk data for writing, where they had the data but also scripts describing the data format/protection - since you need to know how to write the data (especially copy protection) for it to be reliable (or work at all, for some types of protection). It is designed for writing, and that is basically what happens when an IPF tracks are loaded in an emulator (just not to a physical disk of course).

I say all this just to explain that IPF is far more complicated than an imaging format, like the one that will be produced by KryoFlux (which will also be open, of course).

I hope that helps.