Page 1 of 16

IPF and Openness

Posted: Tue Feb 08, 2011 1:07 am
by karadoc
For a long time now, and for the last year very seriously, we (at the Software Preservation Society) have been wanting to make the IPF library source code available. We have always believed in making IPF an open format, but we had some issues in actually getting there. Over the years, those issues have disappeared, and now we have the source nearly ready for public release. It is quite unacceptable that it currently is not. Just one problem remains...

The software licence.

This may be an easy decision for some projects, but we have not been able to resolve things to our satisfaction. So, we would like to post about our issues publicly, and get help from people here on what to do about it.

The benefits of being open are numerous of course. Apart from the critically import preservation aspect, we will also benefit from more eyes on the code, suggestions of improvements and features and patch submissions.

Our Requirements

The first, and major requirement is that it should be a non-commercial licence. Why? Well, we have spent 10 years, and many thousands (10's) of EURs buying and preserving games, and that is not including what our numerous contributors have spent helping us by doing the same. Our purchasing has used almost entirely our own individual private money. We do get money contributions, but they are relatively small in comparison to what has been needed (in fact, they don't even pay our hosting costs). KryoFlux helps a bit, but it is not very much money, especially after all the costs have been taken into account. We get no grants or anything like that, and it seems we probably never will, being realistic. Therefore, we strongly believe that if there is any money to be made from licensing our IPF library in commercial projects (e.g. properly licensed emulated games running original non-cracked and unmodified versions on consoles and handhelds), it should be us making it so we can properly invest that money in continuing our quest to find and preserve everything we can, while we still can.

However, this requirement sadly rules out all the OSI licences (Apache, GPL, BSD, etc.), and it may also mean people writing emulators under those licences are less likely to want to dynamically link to our library. It's perfectly possible, as I understand it, but it seems the case that many (for example) GPL emulator authors don't like it.

I should clarify what I mean by commercial restrictions. Commercial software will be able to dynamically link to our IPF library (having a separate binary that you drop into the emulator folder), but they will not be able to ship with it. They can either license it, or they can get their users to download it from our site themselves. This is currently the case, and has worked out pretty well.

It also seems to be the case that source code licences that do not allow commercial use are pretty thin on the ground. We would ideally want something well known, because it is much more likely to have had lawyer eyes cast over it. We have rolled our own in the past, but we really do not want to do that again.

So, we had a look at licences... Here is the current situation:

1) The Creative Commons Non Commercial licences would be great but for the fact that the legal wording doesn't seem to have software in mind, a fact brought home by their FAQ, which states that they do not recommend it for source code.

2) The MAME licence is well known, it's a BSD-derivative with commercial restrictions. It is probably the most suitable one we've seen so far. It's very brief though, which makes me wonder what it's missing.

3..n) Other licences we have seen are not very well known, so we would have concerns about using them.

So now we would like to ask for advice or opinions from the people on this forum. Do anyone know of any non-commercial source licences that might work for us? Can you see any potential problems in using something like the MAME licence? Do you have any other ideas or opinions that may be relevant to this?

Your help and ideas would be much appreciated.
The Softpres Team

Re: IPF and Openness

Posted: Tue Feb 08, 2011 12:26 pm
by Malvineous
Firstly, I think you will need to make a difficult decision. Do you want IPF to be popular and widely supported, or do you want to prevent others from making money off it? It is very rare and difficult to do both. You either give something away for free, lots of people use it and it becomes popular, or you hoard it desperately trying to make money from it and it remains an obscure format hardly anyone knows about. Probably because it's so obscure you then won't make much money from it either.

I would strongly recommend releasing it under a conventional open source license like the GPL. Yes this means companies can sell a product using your code, *but* it also means you can get all their other code for free to use for yourself. You can still dual-license the code, so a company who doesn't want to release all their code just pays you for a commercial license instead. (In fact this is probably the best option, because lots of companies "forget" about the GPL and don't release source code. I'm no lawyer, but I'm sure this would be sufficient grounds to sue the company for the income you lost because they did not purchase a commercial license.)

Because something like IPF has such a small potential customer base, I think you would be fighting an uphill battle trying to get commercial licenses for it. It would doom the format into obscurity. If you released it with a friendly open source license, it would at least become popular and you would not have wasted all the effort you put into designing the format.

This is why I also favour releasing both the KryoFlux software (dtc) and firmware under an open source license. Yes, there is a risk people will "steal" it, but there is a much greater chance of people doing amazing things that you never expected. You may find you have to produce many thousands of KryoFlux boards because people want to use it for things you never thought possible...

Re: IPF and Openness

Posted: Tue Feb 08, 2011 12:33 pm
by mr.vince
Pretty easy: no GPL. :) We don't want *their* code. We have the best code there is, and it's working pretty well. I know this sounds ignorant and arrogant, but we simply don't want anyone selling our work.

Some of the best tools I use are not open, but closed. They are wonderful products, made by very focussed people. They are user friendly, and I get service and support. On the other hand I do like several open source products, like open office.

I think open source stuff is fine for very popular stuff, where many people help enhancing it and you basically get some professionals that donate their time because they can use the codebase at work, too.

We never came here to make KryoFlux boards. We made them, because people weren't able to do so. But we never were a hardware company. We do preservation work and made our own tools.

Re: IPF and Openness

Posted: Tue Feb 08, 2011 12:54 pm
by karadoc
I sympathise with that view Malvineous, although I disagree. I am sure you are right about it making IPF more popular, but there are features of the GPL that make us uncomfortable using it. Not only that, but if we went GPL/etc, we cannot change our mind. At least with a non-commercial licence, we are free to license it under a more permissive licence later on. I am very much in favour of starting out slowly with this. We can reassess the sitation down the road, but right now we just want to make the information public. I think going GPL/BSD/etc. would just cause further delays, and that is the last thing we want really. It's way past time already. So... lets have that conversation another time.

One thing I didn't mention above, is that IPF is quite unlike any other disk format we are aware of. It contains the disk data, but also what the data means, using essentially a scripting language that describes each data area, which is then used to master that disk (either virtually for an emulator as it is being loaded, or for real).

There have been several IPF reverse engineering efforts in the past, probably the reason they never came to anything is because there is a lot more to it than a simple disk format.

Not only that, the library contains a cycle-exact NEC765 floppy controller emulation core, which IFW spent a long time reverse engineering from the original datasheets.

It's not a few hundred lines of code, it represents a lot of engineering effort. Hence our desire to start out slowly with this.

BTW, we don't make much money on KryoFlux boards, and that was never our objective really. We think the market there is very much in commercial software licenses for it, and an commercial-friendly license would obviously kill that business strategy.

Re: IPF and Openness

Posted: Tue Feb 08, 2011 12:57 pm
by Malvineous
Don't take this the wrong way, but yes, it does sound arrogant. Basically you're saying that you would rather nobody ever used your format, just in case someone might make some money from it. If that's the case, the best option is not to release the source code. If you're lucky, the format will disappear into obscurity. If you're unlucky, somebody will reverse engineer the format, stick their own name on it and release it for free - and you'll miss out on both the credit *and* the money. Remember even Microsoft couldn't stop the .doc format from being made free in this way. Most likely someone else will come up with a format like IPF, but not as good, and everyone still start using it because it'll be free. It will annoy you because you'll keep pointing out all the faults and explain why your version is better, but nobody will listen because your version is closed.

I see this attitude a lot among closed-source software developers - I know, I used to share it. It's like someone else selling your code would be the end of the world, all your hard work wasted. Later I found out that it isn't really a problem. The real problem is that no company cares about your closed source code. They don't want to use it, let alone pay for it.

When I switched to open source development, everything changed. Sharing was encouraged, and everyone's code was made better because of it. It turned out that I was worrying over nothing with my concerns about someone selling my code, because still no company cared about my code. It was GPL, and if they used it they'd have to release their own closed source code, which they weren't prepared to do.

So now all the code I write is GPL'd. I do it because I enjoy it, not because I want to make money. If you're doing all this because you want to make money, I think you're in it for the wrong reasons. You should do it because you want to help preserve software, and if that means releasing an important format like IPF for free, then - hopefully - you will agree that the benefits in doing so are worthwhile. I know Karadoc said you don't get much in the way of donations, so perhaps you could lead by example and donate the format to the community to show us how it should be done.

Re: IPF and Openness

Posted: Tue Feb 08, 2011 1:02 pm
by Malvineous
Ah, Karadoc beat me to it.
karadoc wrote:BTW, we don't make much money on KryoFlux boards, and that was never our objective really. We think the market there is very much in commercial software licenses for it, and an commercial-friendly license would obviously kill that business strategy.
I am curious - how many KryoFlux boards have you sold? As far as I'm aware, there were two batches (complete with pre orders) and they all sold out. I see that as a successful hardware model you could pursue further.

I am, however, not convinced that there is a viable market with commercial licenses. I can't think of any company, hardware or software, that would need to use the IPF format where a simple disk image would suffice.

IPF and Openness

Posted: Tue Feb 08, 2011 1:28 pm
by Interceptor
Bit of a ramble here from my phone...

We aren't trying to make money from IPF.

But we'd like to avoid people using the thing to make serious revenue from it, and we get nothing.

Ideally, this won't happen so no worries.

I don't personally support gpl or viral licensing, certainly not for this, and I think it's a bit crap that gpl won't let you use free code that's not under gpl. Gpl has it's downsides any anyone using it would or should have given that due consideration.

I feel that it's important that ipf is free and 'open' in that any anyone can use and understand it and access it, and dissect it and the information in it.

Adobe didn't gpl acrobat, but the format is open and seems to be an accepted standard, which I believe we are also trying to achieve here.

Arrogant or not, sps has never needed outside help to create ipf, nor did adobe to create PDF afaik. We really are experts and we really do know what we are doing. It seems people were not willing to trust us on that as we are not an open standard, well now we are pretty much finished in the design of ipf, we'd like to share the creation with you all.

Re: IPF and Openness

Posted: Tue Feb 08, 2011 1:32 pm
by karadoc
As you say, I think I answered some of that in my post.

But to your other points:

We are certainly not in this for the money. We are in this to help finance our preservation efforts, and even that money is not worth anything near the time we're putting in (it ceased being a hobby many years ago). That's fine, we'll do it anyway because we know it is important.

There are various companies that want to sell our code, or at least expressed a desire to do so in the past, so there is that difference. I also think that, what with smart phones just beginning to be capable to run games from the 16-bit era, there is some potential for commercial licences. However, you may very well be right about that. We don't really know tbh, but we'd rather not cut our noses off at this stage - not until we have actually tried to properly pursue it. I personally think it is a great selling point for an emulator, but I may very well be unusual :)

As to KryoFlux - sales are indeed going well. However, as I said before, it's really small potatoes in monetary terms, especially with all the fees and other rubbish we have to take care of. At least it is right now, and that very well may change in the future as you say, but the potential for KF commercial licences is very real, right now.

I have contributed a bit to various open source projects over the years, so I know where your coming from. I just think it is a little premature for us right now. Lets get our code out there for now, so the 1000's of IPFs already done are preserved properly.

Which of course leaves us with our problem... a licence to use...

Re: IPF and Openness

Posted: Tue Feb 08, 2011 5:21 pm
by ZXDunny
I'm glad you have opened up discussion, as heads have been butted in the past over this.

Let me make my position clear now:

1. There are now ZXSpectrum IPF files in the wild. I'd like to support those when I go back to developing ZXSpin later this year.
2. I have absolutely no wish whatsoever to use your library code - I shall write my own.
3. For (2), I will need a completely open file format for .ipf files, so that's what needs to be open. Whatever license you choose is fine.
4. If the format is not opened and you insist I use the library then I shall reverse engineer the format and publish my work. I have no need to concern myself over a complete implementation - just enough to read and write ZXSpectrum disk images. I suspect you'd prefer that disks created in IPF format adhere to your spec, not an incomplete one. There has been discussion amongst certain folks in the Speccy scene, and there are more than just me interested in reverse engineering the format. This is likely going to be done by comparing .ipf files with e-dsk files, though I myself am certainly not above disassembling your library to see how it works ("clean room", if you will).

So, in short, I have no need of your library, or its source code - just the file format.



IPF and Openness

Posted: Tue Feb 08, 2011 5:49 pm
by Interceptor
I dont see why anyone wants to create their own ipf files though?

Nor am I sure why anyone thinks they are qualified to do it?

Not ignoring the rest of your post it's all good comment, but that's the part that stands out, to me at least.

I fear that nay be off topic though, sorry.