« Support Ticket Prayer | Main | Trackback Test »
Monday, November 29, 2004
Encoding Source Code
To encode, or not to encode: Whether 'tis nobler in the mind to suffer, The zips and rars of hacked distributions. Or to take arms against a sea of pirates, And by opposing upset customers?
Bad lampooning aside, there is a real question here: To encode our source code and possibly upset customers or to continue to allow readable source code and continue to battle piracy?
Pros and Cons
Readable source code is the default method of choice for downloadable code written for the likes of PHP, Perl and ASP. This is largely because readable source code can be run without the need to install server level decoders and is how scripted files were designed to be used. You write the code, save it and run it. No need to compile it or fuss around with it in any other way.
The big downside with readable source code is that it's, well, readable. You can't add any real restrictions in the code to prevent it from being abused as the code can simply be found and deleted. Another negative is that your code is available for everyone to see and steal.
Encoded source does away with those problems. The code is compiled and encoded and then available for download. The encoding is pretty solid and is hard to "crack". The downside is that the customer has to be running the decoder on their server for it to run. Fortunately, Zend offer a free optimizer/decoder which is very simple to install and IonCube offers a server level version and local file based versions. The IonCube approach is excellent as it allows (virtually) everyone to use encoded software without the need for root level access to the server, which is a real problem on shared and virtual server accounts.
The big downside with encoded source code is that you can't modify it to suit your needs - and that's the crux of the problem.
For years, PHP and perl applications have been available as readable source and this means that, with a little knowledge, you can tailor the application to suit your own needs. This side benefit has now boomed into its own little industry with modification communities springing up for popular products. In a competitive industry, readable source code is another selling point and may be a customer's deciding factor.
Software Piracy
Piracy is a problem in every industry. Even compiled applications get cracked and put on peer-to-peer networks for free downloading so encoding PHP source code isn't going to remove it totally but it will reduce it. As the industry stands now, it takes no skill to pay for one copy of a piece of software and make it available for everyone. Most commercial software has license keys and other data to make it a little harder, but any such code can be removed quickly making it useless.
This brings us back to the original question. There is an argument to say that when you purchase an application, you purchase the application - not the right to have access to the source code to tweak it for your own use. This is certainly true for desktop software. There is an argument to say that encoding files will kill the modifications community and market, and this is where I think we can find a compromise between allowing code modification and encoding source code.
Necessity is the mother of invention
If we can rethink the concept of a modification and how the
modification is applied then we can create a new way of allowing
additional code to be run alongside our existing code. Indeed, an often
asked request is a way to automatically "add" code modifications into
the product to avoid having to search for x and replace with y.
If we can find a way to allow people to write and add modifications
without touching a line of the original code then we can kill two birds
with one stone. We, at IPS have already made a start with our modules
system and synchronization system and we plan to extend that. With
custom settings, HTML logic and a modules system, most modifications
can be rewritten to use these systems.
I definitely think the future is with encoded source. Many applications already encode their source, Modernbill is a good example. If modifications don't need to alter the existing code and it makes it easy to install modifications, then there really can't be any complaints, can there?
November 29, 2004 | Permalink
TrackBack
TrackBack URL for this entry:
http://www.typepad.com/services/trackback/6a00d83423035953ef00d8343b1dfd53ef
Listed below are links to weblogs that reference Encoding Source Code:
Comments
Yes, it is a very difficult area. Personally, I'd want to be able to access the source code to make whatever changes I want, but as a developer myself, I also want to protect my code.
The way I'm going to get around it is to encode the installer and leave the rest of the application open. That way you have to use the installer and therefore enter your license key, but can still modify the application as you like.
Of course, someone could still just buy a copy, run the installer and then dump the database into a file and distribute it...
Posted by: David Mytton | Nov 29, 2004 4:37:20 PM
That's the problem.
You could just encode a license key file and leave the rest of the source code alone but you'd be able to code around that license key file making it useless unless the license key file held vital functions that the board couldn't operate without and the license key check was actually in the license file. Even that has its flaws.
Posted by: Matt | Nov 29, 2004 4:58:08 PM
As a fellow developer, I have also had this major headache of deciding between the two and came to the conclusion that with the software I write, most people aren't going to need, nevermind want, access to the source. Many of my releases will be encoded for that reason.
Problem is, you don't have it so easy in that decision... Your software *is* something many people want to edit. I personally have no problem with encoding, as long as the modules system is robust enough to allow me to add-on what I need.
Posted by: Dan | Nov 29, 2004 8:55:37 PM
I agree with Dan, if you can rework the modules system into making it much more powerful than it currently is, and throw hooks into the software in enough places to allow people to write modules in the place of directly editing the sources files themselves, you would not only solve the problem of "to encode or not to encode?", but also make maintaining board customizations much less of a chore for the board admins, as well.
Posted by: Jay | Nov 29, 2004 9:48:18 PM
Well, I can see why a software vendor would want to encode their PHP source, but as a customer, I would go with the open source (in the unencoded sense) every time. You will probably push away a number of potential customers trying to lock down your source code.
Better to focus on licensing terms that are acceptable to the majority of your customers and make it clear that violating those licensing terms is unacceptable. Don't treat you real customers as though they can't be trusted just because you might have some that illegally use your software.
You have a major advantage that you are providing community software that is only useful if community members have web access to the installation. So, identifying the vast majority of bootleg copies of IPB is much easier than for vendors of other types of software.
I already see that with 2.0.3, you increased the spyware in IPB to send you the license key each time the admin logs into the admin control panel so you can identify improperly licensed boards. I personally hate this tactic even though I would never use software that I don't properly license. When you added this little spyware addition, it has caused me to start thinking about switching to a competing product.
Now, I read this blog entry and it makes me wonder where you are headed.
Posted by: TheWalrus | Nov 29, 2004 10:01:18 PM
Even if you make a good plugin systme theres always the case if the user wants to edit a default function.
Depending on the product you want to encode it can work or not. In the case of a BB it's not going to work very well.
Posted by: Outlaw | Nov 30, 2004 12:42:33 AM
I'm absolutely against any form of source encoding. The ability to read and modify the source is invaluable, and encoding it would have little effect on piracy. Such measures will cause more grief for the legitimate customer than the pirate, and that is (obviously) a road you never want to start down.
When upgrading my IPB 1.3 install to IPB 2.0, I ran into a bug with the MySQL driver that caused the import to fail due to improper escaping of quotes on a query. I was able to hunt down the source of the bug and fix it to get my board up and running, then report not only the problem I was experiencing, but also why it occurred and what had to be done to fix it. Had I not had access to the source, I would have had to report the bug and wait for your company to track it down and fix it before I could upgrade my board. Even today, with IPB 2.0.3, the bug still seems to be present, so I manually modify the MySQL driver to eliminate the bug every time I upgrade to a new version. (The bug was obviously quite rare, and it is likely specific to configurations with magic_quotes_gpc turned off, so I'm not overly concerned that it has not been fixed. However, I feel much better having access to the source code, as I can fix it myself with each upgrade, alleviating concerns over the chance of a SQL injection attack made possible by this bug.) Access to the source code allows the customer to make these small tweaks to tailor the product to his own needs. The other modification I routinely make is to remove the board version from the footer, so that crackers will not find my board when using search engines to look for specific vulnerable versions. Would a module system, no matter how well engineered, be so capable as to allow one to make such mundane and specific adjustments? Would I still be able to make a three-line fix to the SQL driver, or a one-line modification to hide the board version? The module system sounds as if it would make managing add-ons much easier for the customer, but it would be so much better if the source would remain open, so the customer would be free to dive into the source and modify it if the need arises.
Many people also appreciate the ability to read the source for educational purposes. For example, I read a remark by someone on the interesting way in which you handled user verification through cookies in IPB2. All it took was a few minutes of reading through the source to see how exactly you did it. In addition to satisfying my curiousity, it also gave me the knowledge necessary to integrate the other functions of my web site with your user authentication system.
No longer allowing people to read or modify the source would have little effect on piracy. Certainly, it would raise the knowledge barrier necessary to disable anti-piracy mechanisms, but it would hardly make it impossible. Almost every application (source provided or not) is available to pirates, ranging from the largest applications (with the most elaborate anti-piracy systems) to the smallest shareware apps. All it would take to render your anti-piracy measures useless is one smart person dedicated to cracking them. After that, your product will be available to anyone who knows where to look. Personally, I bought a license when you stopped offering the free version, even though it would have been just as easy to download a pirated copy. To me, access to timely upgrades was worth the cost; it takes too much effort to seek out another pirated copy every time a new version is released, and the danger that the releaser did not fully disable the anti-piracy mechanisms in the source was too great. Most people knowledgeable enough to run their own web site likely have equally easy access to pirated copies, but I imagine many make the same choice I did. Considering the time I put into development and money I put into hosting, forty dollars was more than reasonable for a product as good as IPB is. But as soon as your company institues measures that significantly impede IPB's functionality for paying customers in an attempt to deter pirates, I will be making the switch to whatever the next best non-encoded forum is. I doubt I'm the only customer thinking along these lines.
Posted by: Micand | Nov 30, 2004 12:54:29 AM
I must say I also hate software encoding, and its one of the reasons I prefer open source software where its at an equal or better level. However, I understand its need.
However, as far as encoded PHP scripts are concerned, the modules system in IPB and similar software isn't enough, especially if its encoded. The ipb_member_sync.php file shouldn't be that, it should rather have a modules folder with init files that register themselves to be called when certain functions are called. Smarty does this, and its not that hard to do. All software should do this. Its simple, the plugin developers can write a function, then tell it to run on certain events by saying register_postlogin_function(funcname);
Posted by: aent | Nov 30, 2004 4:18:33 AM
Personally I don't agree with encoding the full version of IPB, being able to read the source offers a complete understanding of how things are which makes things like site integration far easier, if this was lost I'd seriously consider switching to something which enables me to view the source code.
Posted by: Michael | Nov 30, 2004 6:51:24 AM
I can certainly recommend... implementing a proper plugin-in system. For one it's a nice challenge and second, designing is better than hacking (although not every teenager who just learned a little PHP may realise it).
About the encoding, your post about sums it up well. Expect another hell freezing over when you decide to encode the paid version of IPB too, though.
Posted by: Zef Hemel | Nov 30, 2004 7:10:56 AM
A better plugin system would be great regardless if IPB was encoded or not.
Posted by: Outlaw | Nov 30, 2004 7:33:02 AM
Firstly, my musings here do not represent policy at IPS. I've been toying with the pros and cons of encoding source for a while now and wanted to get it out onto "paper" just to see how it sounds out loud.
There are some good points raised here and I'm still not convinced that encoding the source is the way forward. As Micand stated, there are too many legitimate reasons for leaving the source open versus the piracy benefits. I'm not convinced either way, currently.
However, I do plan on creating a better modules system and allow easier manipulation of the output streams so that one doesn't have to hack class_display.php.
Posted by: Matt | Nov 30, 2004 9:46:10 AM
"I already see that with 2.0.3, you increased the spyware in IPB to send you the license key each time the admin logs into the admin control panel so you can identify improperly licensed boards."
It's not spyware. I hate it when people draw a conclusion like that. Spyware is software that's installed onto your desktop and sends sensitive information without your knowledge.
IPB hits the IPS servers with your current version number and your license key number. That's it. That's the only information we have access too - and we *should* have access to that information if you're running a legitimate board anyway. To be honest, it's not a measure to stop piracy so much as it is to record if the keys are being used correctly now we've changed systems over and the key is embedded into the code when you download.
Posted by: Matt | Nov 30, 2004 9:49:01 AM
As long as your not doing anything wrong and it doesn't slow down the loading of the page you have no reason to even care about any new checks that maybe added.
Posted by: Outlaw | Nov 30, 2004 10:30:49 AM
Okay. I will stop calling it "Spyware", but it is a strange idea that you are doing this to to "record if the keys are being used correctly".
I don't really understand why you care that my key is installed correctly if it isn't for stopping piracy.
It is a small issue. It just raised my antenna when I saw you had made the change in the last update. I don't really like my board contacting other sites without my clicking on a link to initiate the contact. Changing it to a "Check for new version" link that you would need to click on would be a better idea or have a setting that says to automatically check for updates that could be disabled.
Posted by: TheWalrus | Nov 30, 2004 11:22:49 AM
The code has always been there since IPB 1.2 if I remember correctly. The only difference is that we've tacked on the license key to the end of the URL.
It's a simple IMG tag which loads a PHP script on our server, it's that simple. It won't slow down the board as at worst case you'll simply get a broken image icon.
Posted by: Matt | Nov 30, 2004 11:31:33 AM
Walrus thats the kind of thing most peopl ehave to do now and days, unless you plan no solving all piracy it will always be an issue.
Posted by: Outlaw | Nov 30, 2004 12:45:52 PM
It's simple enough to tell your server firewall to block the request if you really feel you need to.
Regarding encoding - why does it prevent piracy at all? Surely the pirate would install exactly the same encoded files that a legit user does and then has a fully functioning forum.
Posted by: | Nov 30, 2004 1:50:56 PM
Anti-hacking and source encoding is useless. Software/source code piracy is unbeatable. Remember Microsoft, then Adobe, then Symantec, then Macromedia with their product activation systems... :-| They all failed. WPA was cracked even before the official Windows XP release. Adobe's PA was cracked a day after is was released. Symantec's took several days.
The only option I see here is to get more and more legal customers by developping a superiour product and a wise price strategy // or develop a very pricy specific product available only to a limited audience so uninteresting for pirates.
Posted by: ZYV | Nov 30, 2004 2:28:24 PM
Encoding source code? I felt it comming with the encoded trial.
While this will shurely make the code unreadable, how is this going to prevent the spreading of illegal copies?
People who use illegal versions of IPB hardly have a community, and it never lasts longer then 2 to 3 months before they take their board down.
Any one who is serious about their site/community will pay. The piracy is in most cases just some kids wanting to have it just for the sake to have it, or just to play around with it... But it never lasts long.
And the harder you make it to be copied/cracked/nullified, the more challenging and attractive you make your software to piracy.
So because of these situations custommers should be punnished by encrypting the source code?
I have been with Ikonboard and IPB because the code is good, and because I can make modifications to tailor the board to my needs.
No matter what system you will implement to allow mods without the need to touch the sourcecode, some modifications require you to modify the original IPB files, no way around that.
And what will happen? Those kiddies will still have an illegal IPB copy, and custommers will have another dissapointment.
On a post about this on the forums, I had expressed my fear that the paied download would get encoded as well and Charles had assured us that the sourcecode would remain open for customers.
Encoding the sourcecode will be the biggest and last disapointement that IPS has set on me.
I beg you: do not encode! While this will be fine for people with no technical skills or interrests, this will completely cripple all of us.
I am a developper too and theft is very frustrating but I also know that encoding will not stop theft, period!
In the end, IPS will just do what they want and that is ok.
I know I will start looking to an alternative to IPB right now, just in case because I feel something really nasty comming :)
Posted by: | Nov 30, 2004 8:24:04 PM
A few people seem to be asking questions about how it'll stop piracy because you can still pass files around.
The idea is, with encoded files, you could code in functions that stop the board running on any other URL than what it is registered to. Have the URL in some way used in the key with some other data. I've been looking into this idea for a while and I will be using that sort of system, I think. It's worth trying, even if it doesn't block everyone, it'll block some.
I still stand with the point that IPB needs a bit better modules system (because of the type of software that it is) before they should consider encoding the full version, though.
Posted by: Dan | Nov 30, 2004 10:35:22 PM
As a developer myself, I honestly have always thought that competiion could watch in IPB code (if not encoded) and copy at will here and there.
Ok, if they do not copy, they can be inspired by that ...
So ... I think Matt should encode IPB and all Invision software ...
As a user and a community admin I think having an encoded board would irritate me without the FULL possibility to modify the code as I need.
I could call IPB functions from my script, I could modify some functions as best suit me ...
... with an encoded product this won't be possible ...
... IPB and possibly Invision Products would be closed ...
In the end I just think: "Matt, here's another super test, give IPB a SUPER plug-in system and encoding could be possible; if not IPB will lose customers ... that's it".
powergen
Posted by: Andrea Di Salvo | Dec 1, 2004 9:16:58 AM
When it comes down to it, it would be bad business for IPS to release a fully encoded version no matter how better it is then vB.
Simple as that, if you want to think that IPS is going fully paid then think about that.
Posted by: Outlaw | Dec 1, 2004 9:18:06 AM
If IPB were encoded so that I could not alter the files in /sources or the /skin_cache I would move to another platform. As stated above (and I did read your reply to it) something should be done with an encoded license key that is called when a page is read. Like you said, "even this has flaws" but there are ways around EVERYTHING. Make it MORE secure, but allow it to be customized. Without that, you will loose customers.
Posted by: Coastie | Dec 1, 2004 9:41:40 PM
I agree with most people who've commented so far. If you could manage that modules system you talk of, we'd be all set.
But I'd like to share something from a PHP and Flash developer:
http://www.tufat.com/about.php#rationale
Darren (Tufat.com) claims that lower prices reduce the need for piracy. True, if the price is as low as $5 then why bother to download a pirated, non-upgradable, unsupported copy of the script?
I'm not saying IPB should be $5. But do consider making it reasonable and you may find there's a difference..
Posted by: Aric | Dec 2, 2004 12:25:06 AM
