Dear theme designers: Consider yourselves on notice

71 Comments

EDIT: The premise of this post is to specifically call attention to the nefarious practice of using base64_ and other php methods to hide links to 3rd party sites in a theme file. The average user has no idea what is going on: a base64_ function in a footer.php or sidebar.php file is a very very strong indicator that their theme designer is up to no good. It is only an indicator though. When in doubt contact your host or the theme designer and ask them to explain what’s going on. – strebel

We see a lot of themes at Page.ly. We see good themes and we see bad themes.

From our vantage point, hosting thousands of WordPress sites, we literally see everything.

MoPappy will host anything under the sun and could care less what app is running or what it is doing. At Page.ly we only manage WordPress, and we have a keen interest in the performance aspect of WordPress at scale.

Stop the base64_ shit. You are a douchebag if you don’t.

A two hundred character string of HTML in order to encode and hide two backlinks is 75k. That is a relatively big file, and also adds overhead to the PHP process when it is decoded every time the page loads.

I would like to take a moment to tell all theme designers, everywhere: stop the base64_encode shit. You are a douchebag if you do this.

If you run a WordPress site and use a theme that has base64_ anywhere in it then there is a 99% chance that one or both of the following is going on:

  1. Your theme designer is a douchebag.
  2. You have malware.

Both choices suck.

A third option is that your designer may think they have a good reason for base64_encode-ing something legitimate in the theme. I would venture to say the reason is not good enough.

In the near future we will begin calling out—by name and very publicly—any theme designers and shops that do this. I wish to also call upon the entire community to purge and publicly shame the clowns that do this.

Soon Page.ly will likely just disable base64 functions system wide. If your theme or plugin uses this function and you want our thousands of users to use your stuff, it is time to refactor.

You are on notice.

Thank you.

71 thoughts on “Dear theme designers: Consider yourselves on notice

  1. Soon Page.ly will likely just disable base64 functions system wide. If your theme or plugin uses this function and you want our thousands of users to use your stuff, it is time to refactor.

    Yes, and echo. echo is evil, so much hacking happens because you can echo strings.

    Because you know, code never need to be able to

    This encoding is designed to make binary data survive transport through transport layers that are not 8-bit clean, such as mail bodies.

    http://php.net/manual/en/function.base64-encode.php

    It’s not like it is used all over WP core, both in WP itself and bundled libraries.

    Clearly all functions in PHP are made for hacking. Burn… I mean ban them!

      • See quote from original post, it says If your theme or plugin uses this function and you want our thousands of users to use your stuff, it is time to refactor..

        So high profile plugins such as BackupBuddy, W3 Total Cache, etc just got lumped together under “douchebags” label here.

        • Backupbuddy is not a plugin that anyone should ever use. It’s a way to break your WP site every day. Places ridiculous load/processing time onto every page load.

          NEVER USE BACKUPBUDDY!

  2. As a developer I can’t begin to imagine why I would need to use that function. Anyone knows any legitimate use for it? And if yes, what would be the non douchebag way to do it?

    • Step 1 read PHP docs on what it actually does.
      Step 2 search your WordPress installation (including plugins, especialyl those that need to send juggle data around, especially to and from remote web services) for it.

    • There is no legitimate use for this sort of thing in a theme.

      There are legitimate uses in plugins. I use it myself in the latest Simple Facebook Connect, because Facebook sends base64 encoded data back in some cookies (the signed_request in particular includes base64 encoded code).

      • Otto – without bothering to look at your code, I’m pretty sure you documented inline what the use was for, though. (If not, hint hint)

        Rarst – As you well know, the problem in themes is that douche bags use this to obfuscate and hide footer crap like ‘This theme made by God’s left nipple’ or something. They’re not doing it for legit reasons. And because the problem is so prevalent, yeah, it’s a dick move. There are legit uses, and if there’s one for a theme, you should document ‘This encode will do foobar.’ Otherwise, thanks to the assholes of the world, it’s assumed you’re one too.

        • I have no problem with calling out obfuscated links. Even if I have my personal doubts how much it is of hosting’s business what customers use.

          But blaming and threaten to ban PHP functions? Calling out al themes and plugins that use them? The point of function in PHP is that it does something useful. PHP folks didn’t sit down and thought – hey, let’s include bunch of functions to make hacking easier!

  3. Thanks for letting me know about this as never know that base 64 coding do enough of work. As far as I know, free themes use base 64 coding? I only wish, Thesis and Genesis Frameworks do not use base 64 coding?

    • It’s often used by free themes for backlinks, mostly to very shady sites. Frameworks will sometimes include a link back to the site, but they are pretty much always obvious and removable.

      Is that what you meant?

  4. Agree that themes developers shouldn’t be doing this to stop people from removing link backs. Don’t agree with disabling base64 functions because of it. As Rarst has pointed out base64 functions are needed for legit reasons.

    But don’t you run a multi-site site up anyway? Don’t you have control over the themes you allow your users to install? What would be better is to ban badly coded themes. Means less wasted processing power on your servers which means smiles all round for you and your customers as page load times and memory usage drops 🙂

  5. I agree with Rarst. Banning it in themes is fine, but system-wide? As Rarst points out, there are legitimate uses in plugins (the two that I saw using it in a quick search are Gravityforms and W3 Total Cache), and in core itself! The following core classes all use base64_encode:

    Importer
    HTTP
    IXR
    PHPMailer
    Simplepie
    SMTP
    Snoopy

    The following use base64_decode in core:

    IXR
    Simplepie
    Atom Publishing Protocol

  6. Soon Page.ly will likely just disable base64 functions system wide.

    I agree that base64 functions have no place in themes but doing this would break WordPress core functions.

  7. I see a lot of people saying that they see no valid reason for having these functions in a theme, so I’ll throw my hat in the ring for a valid use.

    I’m the developer of the iThemes Builder theme, and it has one active call to base64_decode that is required for the theme to operate properly. However, this function is only run once… Ever… Unless Builder’s data storage is messed up and it needs to repair itself. Builder supports a variety of different layouts. These are stored as data structures. If no layouts exists (such as on the first load of Builder or if the data structures stored in WordPress’ Options system get mangled by some other code or database corruption), a file is read that contains serialized, base64 encoded data structures of the default layouts. This data is then decoded using the base64_decode function, unserialized, and then stored as the new set of layouts.

    Some may wonder why serializing alone isn’t enough. Well, if you try to store a serialized string in a file, some file systems will fail to read in the data properly, resulting in data that cannot be unserialized. The solution to this is to base64 encode the data so that it can be stored in a portable format. Thus, the use of base64_decode in Builder.

    Sure, there are other ways of doing this. I could have the data structure in raw PHP. This is how older versions of Builder loaded the data as I hand-coded the data structure. However, for providing new sets of layouts, this became increasingly difficult, so I opted for this route.

    People can argue with me on the merits of Builder’s approach. People can argue with me about other ways to do it. But the fact remains that this is 1) a valid use of the base64_decode function in a WordPress theme and 2) is not doing anything scammy or underhanded.

    So please. Please. Please. We need to stop blacklisting functions because people can use them for bad. If you blacklist base64_decode, people will just start using their own techniques to mask all of this. There are literally dozens of functions that people can use for this. Are all of them going to be blacklisted as well? What about the code that needs these functions in order to securely transmit data back and forth between services? What about the 2 instances of base64_decode and 14 instances of base64_encode currently found in WordPress 3.2? What about the dozens/hundreds of plugins and themes that are using these functions for the reasons that they are intended?

    Knee jerk reactions like these are extremely frustrating. I know where you are coming from in complaining about it, but this amounts to prohibition of alcohol. Sure, it sounds good on paper, but the net result of implementing such a limitation is that the people who caused the problem in the first place adapt and are unaffected while people with legitimate needs and uses are left with no recourse other than to recommend against using those that impart the limitations.

      • Thanks for the tip Otto. This is the kind of discussion I want to have. Something with meat on it.

        I’ve never seen anything in my testing, reading, or from talking to other devs about issues with using base64-encoded serialized data for data structure storage. What is it about this approach that is “bad mojo?” Is it simply because we often associate such usage with bad behavior or is there a technical reason for it?

        • Sure. Basically, base64 was intended to be used for a transport mechanism to transport data over non-8-bit-safe methods. It adds a 33% size overhead to whatever data you’re using it for, so if you’re using it for storage then it’s not the best way. Serialized PHP was also not meant to be stored in files, really. It was originally a way to encode objects and arrays, with their context, for storage in databases instead.

          Without seeing the specifics of what you’re doing, I can’t really comment on it, however serializing data, encoding it in base64, then sticking it in a file is a bit silly on the face of it. There’s a lot of processing you’re doing there for no obvious reason. If you have data in a file that isn’t going to be rewritten, then you might as well just make it a PHP file with an exported variable in it. Then you can load the data by simply including the file. This is going to be much faster, and safer, than relying on two separate encoding methods both of which were intended for different purposes. Sure, it can *work*, but it’s probably not optimal.

          • I did some performance testing comparing the two methods using the data set that I have in Builder.

            File sizes:
            serialize: 14182
            serialize + base64_encode: 18912 +28.6%
            var_export: 19895 +33.5% (19927 when PHP tags and variable assignment added)

            Not much of a difference, but the base64-encoded version is actually smaller. I could try to reduce the size of the var_export version by removing all the unneeded whitespace, but this could easily break some of the data in the data structure inadvertently. It is this that causes me to think that var_export was not meant for any kind of storage format but more as an advanced debugging tool such as being able to easily hand-craft data structure modifications.

            Benchmark results:

            test_var_export
            ==========================
            Trials: 499426
            Total Memory: 831045904
            Average Memory: 1664.0020823906
            Max Memory: 2704
            Total Time: 181.36194324493
            Average Time: 0.0003631408
            Runs per Sec: 2753.753025934

            test_serialize
            ==========================
            Trials: 500574
            Total Memory: 832955712
            Average Memory: 1664.001150679
            Max Memory: 2240
            Total Time: 101.90418386459
            Average Time: 0.0002035747
            Runs per Sec: 4912.2026301211

            The memory consumption differences are negligible at less than 500 bytes when comparing maxes. The big differences are in the execution times. Loading the var_export data is a full 56% slower than using base64_decode and unserialize. I assume that this is due to the var_export data needing to have a full lexical analysis on the var_export data while the serialized data just needs to fail when bad data is hit.

            This brings up why I really like serialize/unserialize. If the data is corrupt, PHP doesn’t crater and the site doesn’t crash with the white screen of death. If unserialize fails, my code can adapt or at the least provide an appropriate message to the user. If an include fails due to corruption, there isn’t any recovery. Not even a try/catch block will protect against this. Even if try/catch blocks could protect against syntax errors in includes, I’d rather not go down the Exception path as that has its own price to pay.

            So I’m sorry to say that I still like my approach. Not only is the resulting file smaller, the max memory consumption is smaller, and the execution time is much faster.

            You can prefer using var_export for whatever reason you have. I have no problem with that. I do have a problem with people telling me what I have to use however.

            This is what I’m getting really tired of. People through around ideas of performance and efficiency and code cost, but I never, ever see people actually trying to benchmark this stuff to put proof behind the claims. Then I do benchmarks and people ignore it because tl;dr.

            It’s time that we grew up and acted like professionals (this isn’t aimed at you Otto, it is a general statement). Opinions only matter as far as they can be backed by evidence and rationality. People need to buck up and start supporting their claims.

        • Note that I dislike the use of serialized PHP data over HTTP transports as well. Even though we do it in WordPress and with the API, it’s not the best way, IMO. I’ve had arguments about this with other core devs before. Yes, it works, but it’s limiting and non-optimal. I’d much prefer to only send XML or JSON data over HTTP instead, despite the slight decoding overhead.

  8. While I agree with your frustrations, this post is in EXTREMELY poor taste.

    There are much more professional ways to convey your frustrations.

    Also, like it or not, Rarst has good points. Outright disabling the base64_ functions is not only extremely poor practice (let’s just disable the rest of PHP while we’re at it, yes?), but also will cripple any current plugins that need these functions.

    I think you need to take a breather, stop pumping yourself up with so much caffeine, and relax. Going on tirades like this will only get you negative attention.

  9. I’m not sure about banning completely the whole function, system wide. And there are just too many ways of inserting douchebaggery that don’t involve base64.

    The thing is, a theme should not output an external link that is not searchable in plain text in the theme files.
    – Make a page template that fetches nothing from the DB, just get_header(), footer, whole page structure.
    – Grep output for all external links.
    – If find any, grep theme files for these links. Cannot find? Not good.

  10. Good to see everyone discussing it from both sides. My snarky tone was aimed specifically at the link spammers, and more generally at everyone to reconsider their coding choices.

    • I have no problem with snarky; however, I didn’t see your tone as snarky, I saw it as needlessly accusatory and divisive. I assume that many non-dev and even some dev readers would take your commentary as reality and think that anyone that used base64_ functions is doing so needlessly and possibly recklessly. Thus leaving those that use the functions responsibly to unnecessarily defend themselves against this baseless accusation.

      You didn’t start a dialog about the merits of using the functions weighed against what you see as rampant abuse of them. You didn’t say that you are getting tired of people embedding hidden links inside their themes using scammy techniques. You specifically targeted a couple of functions, literally pointed a finger at anyone that uses either of the functions, called everyone that uses them douchebag, made a purely anecdotal claim that 99% of all of the uses of the functions are for nefarious purposes, and then made an ultimatum despite how you clearly have a limited understanding of what you are talking about.

      Personally, I’m quite put off as we’ve chatted on a number of occasions, and I think you’re a cool guy. Professionally, I’m surprised.

      We are all in this together. We developers need to always be mindful of how we tax hardware and that maybe, just maybe, we have a very limited scope of the true expense of the choices we make in developing software. You and your fellow hosters need to always be mindful that blocking access to functions can severely limit what developers can do to produce good software with cool features and that maybe, just maybe, you have a very limited scope of the value of specific features and functions.

      Going forward, I would like to see a dialog between developers and hosts to come up with best practices that we can all agree on to lessen the hardware costs of running software without limiting what our code can and cannot do. This would be much better than someone on one side getting pissed off and cranking out a frustrated post that creates division between the two camps rather than a cooperative dialog. Yes, this is coming from someone who has done this in the past. Fortunately, that ended in some good dialog.

      • I dont buy it Chris. I love you guys at ithemes, and you know full well this post was not aimed at you guys. However we already ban a dozen or so php functions.. Like exec(), which buddypress uses that I have mentioned to you guys. As such buddypress is kinda useless on page.ly.

        Disallowing a php function is a tall order which we dont take lightly, but do consider as needed. Remember we see everything from the top down. If 1 or 5 or even 10 sites cannot use a theme because we chose performance considerations for the thousands of others instead of a function for a few, so be it. It’s a tradeoff we’ll make in favor of scale every time.

        • I’m not sure what you don’t buy. If you don’t buy cooperation, I’m not sure why. You told us about how your servers prevented code from using PHP’s built-in zip functions and the exec function, so BackupBuddy’s zip features literally had no way of working unless we wanted to write our own pure-PHP zip utilities (which would be drastically less performant than the features you blocked for performance reasons… a dead end). Since we weren’t going to argue with you on your server decisions and BackupBuddy has zip files as its central feature, we were left at an impasse.

          I’ve made my position very clear. What I don’t understand if why you see this as such a good idea. I have yet to see any valid justification.

          If you wanted to do this on Page.ly and not make a big fuss over it, that’s your business. You certainly did this at other times as you note, and I don’t remember ever complaining privately or publicly. But you posted this here in a public space in a way that attempts to rally the troops around a cause that I think is short-sighted and dangerous.

          I think you made a mistake. I think that you’ve made a poor argument for blocking the base64 functions outright, and now you want it justified so that you can go ahead and block it on your servers for whatever reason.

          In your post, you declare that it takes 75k of file space to store an encoded 200 character string. Actually, if you base64 encode a 200-character string of HTML, it runs 268 bytes. This is a far cry from 75k. Even if I wrap all of this up in a layer of base64_decode and eval, it only increases in size to just 280 bytes. Sure, I could keep nesting the encoding and evaling until I wound up at 75k, but at that point, we aren’t talking about the performance merits and demerits of base64_decode, we are talking about the expense of running very specific code that was very specifically written in an expensive manner at no fault of base64_decode itself.

          You declare that “[this] adds overhead to the PHP process when it is decoded every time the page loads.” This is true, but let’s find out the true impact of this has.

          I did some benchmarking and got some interesting results.

          Compared to just echoing out a string (or having it outside the PHP blocks, it doesn’t matter which), I found that echoing from base64_decode output increased execution time by an average of .000001269 seconds or 1.3 millionths of a second.

          For doing an “eval( base64_decode( … ) )”, the execution time increase is .000000489 seconds or .489 millionths of a second. This seems illogical, but in my testing, I’ve found that PHP does an amazing job of optimizing evals. Since these benchmarks were run in batches of millions of tests to get good averages, this could account for the optimization. I ran some individual tests which had widely varying times (for both the control and the eval), but the biggest difference between the control and eval that I saw when running the tests this way was around .00001 seconds or 10 millionths of a second.

          My benchmark script showed no difference in memory usage, so I used the individual tests again to look for memory differences. The memory consumption was the same in each of these tests as well (and yes, I am checking peak usage). So memory consumption is not an issue.

          I repeated these benchmarks on two other servers, and I found the same results (slightly different timing differences but with the same pattern). You can get my scripts here and try them out for yourself.

          So clearly memory consumption is not an issue when looking at these approaches critically. Nor does execution time seem to be much of an issue.

          If you don’t buy that I care about efficiency or development, you don’t know me well enough. I truly do believe that every fraction of a second is precious. Case in point, I submitted a very simple patch to core yesterday that is a rewrite of a small function that literally runs on every page that uses permalinks, and the benefit for that function (using the same system I ran these benchmarks on) shows an average improvement of .000008223 seconds or 8.223 millionths of a second. Note that this is significantly larger than some of the performance differences seen with the examples I ran based upon the information in your post. So, if you truly want to improve WP performance, start encouraging patches like these as patches like this one will improve performance across the board.

          My point is that simply blocking all base64 functions out of a concern for performance is looking in the wrong area. You can gain greater time and memory savings by having a single plugin not needless run code that won’t be used or not enqueue scripts and/or styles that aren’t needed on the current page.

          The only thing you have left in your argument against base64 functions is how some theme devs use this to hide stuff in the footer and in other locations. I can’t think of a single dev in the past few years that has done this to their theme. Each and every time it was some affiliate link spammer who took a free theme, ran it through a script that tacked on the extra link crap, and then spread the theme far and wide. So if you’re putting theme devs on notice for link spamming, you are talking to the wrong crowd.

          As I’ve already indicated before, if people start shutting off base64 functions, they’ll just do the same thing in one of a billion different ways that are harder to identify (case in point, the _verify_isactivate_widgets self-replicating theme virus, bonus points for people who can actually understand what it is really doing).

          So you say that you don’t “buy it,” but all I’m trying to sell is that your solution for performance gains and stopping link spammers is 1) poorly thought out, 2) a bad idea, 3) really won’t achieve either goal in any measurable way, and 4) is going to needlessly upset developers and your customers who will have their sites crash when they try to activate code that understandably tries to call one of the functions without doing a check to ensure that it is callable first.

          So it is my turn. I don’t buy it. I don’t buy your reasoning for constantly locking down functions. I want to know where you are getting your data that says that these two functions are adding significant load on your servers. If you are going to publicly make an ultimatum, then you need to also make public your reasons and justifications. If it all boils down to just not liking spammy links, either get used to it or make a better plan because shutting off base64_decode won’t make them go away.

          • Please read my reply at the end of this comment string. What Josh was trying to do was get people talking about the issue. It seems to have worked. We are coming from a hosting perspective however we did client services for years. We think there are better options than base64 as do many people. As Josh said earlier, you know this was not directed at iThemes.

          • You probably should do your homework before you make invalid claims about a product.

            And when I see arguments citing things like “…tradeoff…in favour of scale…” maybe it’s just my cynical side but somehow I always seem to read that as “…tradeoff…in favour of packing as many customers on one server as we can and maximizing our profit…” – still it’s refreshing not to see the tired old “security” argument rolled out and some honesty about commercial imperatives vs service to the customer.

            However, I don’t quite see your argument there – if you have thousands of customers _not_ using a function (by definition they’re not because you have them on the server with the function disabled) then if you enable the function for that 1 or 5 or 10 customers and they use it responsibly then how is that impacting your “scaleability” exactly – are the other thousands suddenly going to start using that function for some reason – sorry guy, your argument makes no sense 🙂 If your servers are really _that_ sensitive then they’re waaaayyyy overcrowded.

  11. Strebel, I completely agree with your feelings on the link spammers. I just don’t think punishing everyone for the dubious acts of some people is completely necessary or fair. In general, it is inevitable that you will run into someone somewhere who is trying to screw your system. I suppose it’s just a matter of handling it properly when it happens. 😛

    • We have been very fortunate to grow very quickly. However, one of the main reasons is because we act on potential problems when they are small. It’s good business and good for our clients.

  12. I’ve never heard of a legit theme developer obfuscating code with base64 funcions, but it’s a very common occurrence for it to turn up in “pirated” versions of themes that a user finds through a search engine. There are a lot of copies of WooThemes and similar companies’ themes circulating with added spam links (or worse: malicious code passed through exec()).

    But, no, a legitimate theme developer should not be using base64 functions to obfuscate things. It is however, a Very Bad Idea to disable it “disable base64 functions system wide.” That would certainly break a lot of plugins, and possibly even functions of the WordPress core itself. There are a lot of legitimate uses for encoding/decoding Base64 strings.

    A better solution would be to scan theme files for the function when they’re uploaded.

  13. I have a problem with this :

    In the near future we will begin calling out—by name and very publicly—any theme designers and shops that do this. I wish to also call upon the entire community to purge and publicly shame the clowns that do this.

    I agree that adding hidden links in obfuscated code is a shadey thing to do, however – that person has designed and built that theme, he is free to do what ever they like with it, if they want to include encoded footer links then it’s up to them – no one is forcing people to download and use their theme.

    People need educating about where to get good quality, secure themes from – not some stupid campaign to “out” developers who do this.

    • Right so why not make a list of the ones that do use encoding for bad and make that available to the public who doesn’t have the know-how to decode it and see what it’s putting on their site?
      If anything it’ll help weed out the bad ones.

  14. Josh has a gift of being able to get people talking about important issues. I admire him for being able to stir up passion in other people allowing them to share their viewpoints. I believe the intent of this post was to discuss misuse of code rather than base64 itself.

    We love our company, our clients, and the WordPress community. We work tirelessly to provide an awesome hosting product because that is what people deserve and don’t often receive. Because Page.ly is WordPress hosting only and we strive to provide the best to our clients, we have to watch out for nefarious attempts with base64. There are other options. Does that mean that we are going to become the base64 police? Only if we deem the use nefarious. We owe it to the WordPress community at large. Will we publicly tell people about it? Probably, if private attempts do not work. Again this is for the greater good and won’t affect most of you who read this. Many of us can read code and can stay away from questionable antics however there are millions of people who have no idea. As a community, we should strive to help each other and look out for the little guy.

    Last but not least, don’t worry about the well known theme and plugin companies. If they were doing something shady, they would have been called out already by others. However, I think we can all agree that we should stop growing companies that use questionable base64 and keep the ever growing WordPress community culture intact.

    • Maybe you should just proofread your hubby’s writing so you can tell us all what he intended… before he posts his garbage on public websites. Or maybe your time might be better used actually providing testing results to backup your hubby’s “gift” of getting people to talk.

    • yikes.

      I had page.ly bookmarked for future use, but… something’s not right here.

      Does stirring up passion = making me suddenly feel the need to take a shower because of all the feeling of ickiness I got while reading your responses throughout the comments section?

      I can handle snark but will never be a fan of shadiness.

      Kudos to the devs in this thread for nipping the misinformation in the bud via clear concise arguments(facts).

  15. while I agree with most comments here so far, from both sides of the dialog; I am most curious about why a post directly from the Page.ly founder about Page.ly was posted on WPCandy. Seems like an odd occurrence to me?

    • Hi Joachim. It’s simply an editorial, contributed from a member of the community. Obviously the opinion Josh expresses is influenced by his work, and since it’s relevant he mentioned it in the post.

      We’re always open to contributed editorials. I love seeing them go up, personally 🙂

  16. Thanks for this post Joshua. It really opened my eyes about the mystake i was about to do: start hosting my clients’ sites on page.ly.

    Inspite the come-to-the-rescue Sally post, the fact is that this wasn’t a post about making the community talk about the subject. Unless the community that reads wpcandy are a bunch of evil people using base64 function to mask links. If “we” are not, then why the use of the generalization? There’s nothing that tells that the way to be snarky has to be through generalization. If it is not aimed at iThemes, at , then only write about who you’re talking: the spammers.

    Imagine a client of mine reading this article. Some of them are not that naive for not being interested being informed about their content manager, it is not that impossible as wpcandy is the best news site around about wordpress. What would they would think without reading the comments? Who use base64 is the evil, therefore making us “the good huys” thought extend lenth explaning all. And if those clients of mine didn’t knew me? If they were just people starting their hosting site reading wpcandy to be well informed? Sure they would think that iThemes was the evil for using base64. One client lost? But who cares, after all, what i am saying is so impossible… right.

    First it was a performance issue. But then this myth felt apart with the benchmarks from the iThmes guy, and there’s not one line written with an counter-argument too based on benchmark: just a spitted (spat?) 75k overhead god knows where it came that from. And now all i see is that there’s other ways, there’s other ways… and of course, the come-to-the-rescue-Sally-it-was-just-him-being-snarking-he-is-so-talented-making-people-talk.

    How about… working? The easy route is like Franco, Salazar, Fidel Castro would do: make a rule and send to jail everyone that doesn’t comply.
    Start a campaign of prevention amoung your coustumers. Prevention, prevention, prevention. Does everyone knows that downloading a theme from themeforest but via rapidshare (piracy) may be in danger? I bet they do not know. A campaign to recommend people knowing exactly where they are getting their themes for.
    A huge, giant, list of trusted themes shops, both payed and free. I do trust wpexplorer, if that guy running it has a free theme, i do trust him.
    Make a list of plugins to help people see if their theme/plugin has injection code, etc.

    Make a list of themes that are compromised, and block them. I’m sure you could add 4 or 5 themes, sources, that you do know are “always” compromised (damn, low english skills on this one, hope it is understandable what i was trying to say).

    Of course, everything that i just said it’s stupid and not valid. The way is to ban base64 and all that happened was a snarky post with the sole purpose to make everyone talk thought a generalization, by putting in the same bag everyone: spammers and fine guys like the ones at iThemes — and everyone that complains: hey, it was not aimed at you how could you not tell even if i just put everyone in the same bag?

  17. Hehe.. Rather then the collective “Yeah! Link spammers are douchebags!” response I was expecting, It appears from some of these comments I must have unknowingly insulted your mothers.

    You angry folks all latched on to a mention in passing we may just disallow a php function. So what if we do? I doubt anyone will lose any sleep over it at the end of the day. It’s our system and we make choices everyday on how to run it, if you dont agree with that, use someone else. It’s okay with us. We know not everyone is our customer.

    Come out to WCPHX next year and I will buy you all a drink and we can have a big ol laugh about this one time someone posted something on the internet and people disagreed.

    • I really don’t want to carry this on forever like this, but you just keep trying to change your post to mean something different.

      Original post: “Soon Page.ly will likely just disable base64 functions system wide. If your theme or plugin uses this function and you want our thousands of users to use your stuff, it is time to refactor.” Emphasis is mine.

      Your reshaping: “You angry folks all latched on to a mention in passing we may just disallow a php function.”

      It was the capstone to your post. Your closing statement, and while it did have a very subtle “likely,” the “soon” with an ultimatum to theme developers certainly makes it sound like immediate action is necessary, even if it truly isn’t.

      If you want to have a discussion… If that was truly the purpose of this post, then stop trying to reshape what you wrote and join the conversation.

      People have posted ways to identify the spam link themes without banning base64. I’m sure that a handful of developers would love to take a crack at some type of script that could run on your servers and identify such themes in an automated manner. What are your thoughts on such solutions? Would you like to try them out on Page.ly? Would such a solution be sufficient enough to consider not restricting base64 functions?

      I posted information on how your claims about the base64_decode function performance doesn’t stand up to my benchmarks. Can you provide more information and raw stats that show how base64 functions are affecting your server performance? I’m sure that many people would love to see performance metrics of specific functions on high-scale server loads. This data would be invaluable for helping to increase WordPress, plugin, and theme performance overall. If you don’t have any such data, can you provide us with the information that you are using to determine whether or not these functions should be restricted on your servers? Can you provide estimates of load reduction from such actions and how you calculated these figures?

      People are pointing out that specific features in WordPress rely on these functions. Do you have information that shows that restricting these functions will not result in any site crashes or loss of WordPress features? For plugins and themes running on Page.ly sites, have you tested out the plugins and non-spammy themes that use these functions to see if they would be affected by such a restriction?

      If code starts to fail because of such function limitations, what will your response be to your users? Will you share responsibility by telling your users that you have limited one or more functions on your servers for performance and security reasons and that this is also affecting the plugin/theme? Or will you push it back to the developers directly by saying that the plugin/theme cannot run due to performance and security concerns?

      If it turns out that this is a bad idea and you decide not to go forward with it, will you do an addendum to your post saying that the functions are actually critical and that there are other ways of going after bad themes? This way people coming and seeing your post won’t think that its a good idea just because you said it was and start implementing it on their systems.

      You mentioned before that you limit a number of features on your servers. Is a comprehensive list of what you don’t allow to run on your servers available somewhere so that developers know what functions, classes, etc to avoid using to keep their code running well on your servers?

      The ball is in your court.

      • It would be really informative to see the numbers behind your claims, I think Chris raises a valid point, and from everything I’ve read here in the comments, I’m in no hurry to use page.ly for WordPress hosting.

    • Let me layout how this looks from the side. I think it is quite different from how you see it. Note that this is probably not what you intended, wanted, assumed, implied… but this is what it turned out to be for some.

      1. You got on soap box to vent about cloaked links. Perfectly fine thing to do. Maybe there are some edge cases where such links are fine, but I am yet to see anyone coming up with such case.

      2. While at it you made technically inaccurate and sweeping generalization, that turned your venting about nefarious people into wide insult towards developers and their code.

      3. You finalized it by proudly dragging function banning into spotlight, that plenty developers (both in and out of this discussion) see as incompetent. Which contradicts value of allowing your clients to run any code, that you put in bold on your front page.

      Now you seem stuck in your original idea (1), while others try to get through to you on generalizations and decisions you made while at it at stages (2) and (3).

    • “The fool doth think he is wise, but the wise man knows himself to be a fool.” – can’t argue with the Bard there.

      Stop digging yourself into a hole and just admit you made an ill-judged posting and apologize rather than trying to blame everyone else. I don’t see anyone being “angry” as you so put it, more passionate about something that they believe in and you don’t seem to understand.

      If you really don’t understand the community in that respect then you would probably be better of concentrating on your day job and making your system better so that you can actually fulfil the grand claims for your service and not restrict the freedom of your customers – or if you can’t do that then stop advertising your service as the mother of all WordPress hosting environments which, judging by all the restrictions it sounds like you have it plainly isn’t.

      You should probably be advertising as just a blogging service (and you can use whatever hobbled system you want to provide it) but please don’t make people think it’s like WordPress Plus and the best thing since sliced bread.

    • O.o … is this real life?

      Honestly, the only thing you’re insulting is everyone’s intelligence.

      Does receiving free(?) page.ly hosting mean WPCandy has to let this guy post here?

      Were all of the ‘angry’ comments deleted or is he just one of those people who automatically feels attacked by those who are more knowledgeable?

      • Hi Gina. WPCandy doesn’t receive hosting, free or otherwise, from Pagely. Even if we did, we would never trade content on the site for stuff like that, trust me.

        Josh is a community member with an opinion, which means he is a perfect candidate for authoring an editorial on WPCandy. You’re welcome to send over an editorial if you’d like as well 🙂

  18. Or next time, think twice about being so ahead of all and starting label everyone that uses something as duchebags? Maybe, but then again, we failled at teh lulz, damn we are too much serious business, must be it 🙂

  19. Not just theme designers, plugin as well. iPay88 (Malaysia) online payment gateway provider also do this s**t. You’re paying USD 170+ for the yearly cost and they provide you with:
    – Uncustomizable payment page (that is ugly)
    – 3% to 4% transaction fees.
    – Base64 encoded script.
    – and more.

    I really hope one day that some people can break this out.

  20. Well, I gotta say I clicked over here from my reader because I enjoyed the post. So thank you for the post, whether it was your intent or not, I got a chuckle! As for the comments, some of them are quite educational, so thank you for taking the time to discuss the underlying concepts a bit!

  21. Take a look at this gem of a footer.php: http://www.freewordpressthemes.com/themes/2396/
    Decoded and inflated all that nonsense you get a link to payday loans and some designer’s site: http://pastie.org/private/dj0od8nrwljey2r66c0ua

    ~69k of code (and a few valid lines of html) to obfuscate 2 spam links.

    The example above and many more like them was the basis for this post. We see this stuff all day picked up in our system scans, and have to verify each one to make sure it is merely link spam bullshit and not something more nefarious.

    I have seen a few legitimate uses of the base64_ functions in plugins across our installs. W3 Total cache (a plugin we use on every site) uses it in request signing in a few places to support services like S3 and WindowsAzure. I have seen base64 in a commercial theme by a popular shop once upon a time. These cases were not the premise for this post.

    This is a true statement: “Page.ly will likely just disable base64 functions system wide”. I will also add “we will likely add other functions to our list of a dozen or so we currently disallow”. We have not yet, and have not made plans to yet, but likely we will at some point. And we will certainly make exceptions for what we feel is appropriate. But we won’t for every use case. Hence: If you are dbag theme designer and obfuscate links, you need not apply. If use base64 functions and wish your stuff to work on our system, perhaps it is time to consider a refactor. If you could care less about our system.. that is okay too.

    In our experience it really is only 1% of the time we have seen base64 in a theme that was not obfuscating links. As Otto pointed out there are other ways to achieve the same/similar legitimate result using other methods. I have added an edit to the top of the post to make my intentions clear and apologize for not doing so sooner.

    The statement in our marketing of supporting every theme and plugin has recently been rendered inaccurate and I need to change it. We do not allow some server side analytics plugins any more as there are a wide choice of client side and free options out there. Some of those plugins log every single page request.. resulting in hundreds of thousands of rows in a database table. Taken on a single site no big deal, taken across thousands it is a performance hit on mysql, doubly so when some of them do not even use an index on said rows. Our long term goal is contribute patches back to these and other plugin/theme authors to improve their performance and security.

    Thank you all for your input. We’ll consider it as we continue to grow. Our stance will always lean heavily towards making decisions for our customers that benefit them by improving performance, maintaining security (like the system wide patch of timthumb and the new firewall rules to continue to block current and future signatures of this hack), and anything else we deem appropriate for our system. Even if it means over time a small % of plugins and/or themes are no longer supported.

    • There’s so much to say.

      I could mention how this comment essentially removes all the backtracking you and Sally have done to indicate that this was about fostering a discussion (as you clearly point out in this comment that it was meant as a warning). I could talk about how the theme you point to was modified to have the obfuscation code not by the original theme developer but by the site hosting that zip file (do a search for the theme, you’ll see what I mean as it is everywhere and each site has its own obfuscated code in the footer.php). I could point out how the code in that theme is slow and memory intensive not because of base64 but because the first set of obfuscated code is a complex set of arrays containing 482 links and keyword mixes and how the second set of obfuscated code uses 41 calls to base64_decode, 41 calls to eval, 41 calls to gzinflate, and 23 calls to str_rot13 to do their link hiding. I could talk about a lot of stuff, but I think I’ve made my case for why this is a bad idea.

      All I’m left with are questions. Many of which have already been asked yet not addressed by you.

      What research have you done to know that this won’t cause big problems for your users and how much benefit will it have on server load? I.E. Why is this such a good idea? I still haven’t seen a good answer to this one. And no, “because I see a lot of crap that uses this function” is not good enough.

      Why are you so stuck on this being the only solution while you are simultaneously so stuck on every single WP developer in the world needing to inspect and possibly rewrite their code just to satisfy your desire to block this function?

      You currently block a number of functions as you state a few times in the comments. This list is unpublished and you seem to have no willingness to start publishing it. You now want to block base64_encode and base64_decode. You have also indicated a desire to find other functions to block if you deem the value gained by blocking them to be sufficient. When will you be satisfied? Will you continue to notify us developers in this same fashion each time you decide to do so? Will you publish the list of blocked or restricted functions, classes, and features so devs know how to be Page.ly-compatible?

      When code starts to fail because of this, are you going to say that the plugin/theme your users want to run won’t work due to security/performance problems or tell your users that the code can’t function due to limitations on Page.ly’s servers?

      Who do you consult when you decide to limit these functions? I can’t imagine that you haven’t consulted with any developers on this, so I would be very interested in knowing who says what functions should be blocked.

      You mention in a response below that you will permit certain code to get around these limitations. I think we would all be very interested to know what code gets special permission. Is this limited to just WordPress core or will certain plugins and themes get exceptions? If so, how is this determined and is there a process for other devs to get whitelisted?

      Writing good code is not easy. Running solid servers is not easy. What is out of balance here though is that hosting companies can always take the easy way out, turn things off, and then lay blame on the developers. Coders can’t do the same. All we can do is complain in places like this. But at the end of the day, I think we all know that you will just turn off what you want to and that our comments amount to an annoying buzz.

      From my perspective, you and the people that distribute spammy themes that you hate so much have a lot in common. Just as those spammy theme sites don’t give a damn about anyone else as they aim to spam the net with their affiliate links, you don’t give a damn about how your heavy-handed solutions to performance and security issues make the lives of every developer harder. If you cared even a little, this post would have been a true call to an open discussion about the topic rather than a blatant notice that the decision has already been made and that devs better pony up or get banned by Page.ly.

      Many of us did the consultation work of showing all the problems you should expect to run into. Otto gave you some quotes you can send to complaining developers when they say that there isn’t another way for them to do what their code needs. Seems like everything is ready to go. Congrats.

      Do whatever you want. Hosting companies always do. Meanwhile us coders will do what we always do and spend more waking hours making sure that our code works in as many places as it possibly can while navigating an invisible minefield of limitations that hosting companies never talk about unless asked directly in private conversations covered by NDAs. Don’t worry, we probably have you covered whether we want to or not. You can sleep soundly knowing that hundreds of developers will be going without sleep tonight to solve problems created by the artificial limitations of hosting companies and are likely to do the same when you start breaking their code with this new rule.

      Now it’s time to get back to work. There’s much to do as I just found out that some hosting company will start blocking functions that we use and will do so at some arbitrary time for some arbitrary reason.

      For those interested in my code analysis for this theme, I zipped up my findings here. The zip file contains footer-analyze-code.php which decodes the recursive use of eval to hide code (I have only tested it on this code, so it is not robust enough for other applications) and counts the functions used to do so. This may prove interesting to some.

      • “Do whatever you want. Hosting companies always do. Meanwhile us coders will do what we always do and spend more waking hours making sure that our code works in as many places as it possibly can while navigating an invisible minefield of limitations that hosting companies never talk about unless asked directly in private conversations covered by NDAs. Don’t worry, we probably have you covered whether we want to or not. You can sleep soundly knowing that hundreds of developers will be going without sleep tonight to solve problems created by the artificial limitations of hosting companies and are likely to do the same when you start breaking their code with this new rule.”

        This is a problem that all hosting companies face. The people that develop code do not fully understand how things run on a processor, what implications their choices with their code has to the performance of their application, how to efficiently utilise a RDBMS (if they even need to use one!). The number of times I’ve come across a poorly performing WP site on a customer’s server only to discover that they have a very poorly planned plugin causing page generation time to be over 1s (sometimes into the minutes). At the hosting company I work at if we spot a BackupBuddy installed on a site we immediately recommend that the customer stops using it an relies on more traditional backups, BackupBuddy just doesn’t scale.

        Remember the hosting company is the one that deals with the customer complaining because their site is slow. They are first to start to point the finger towards Disk IO or Slow Network, when in 99% of the time it’s actually a poorly written application that is causing their issues. Then we have to go out of our way to prove that it is indeed the application causing the issues.

        Do developers ever think of using XDebug to generate trace data to determine how their code is running? I would guess less than 1% of them do. And I’m not targeting my comments towards that 1%.

        Spend a month working at a hosting company dealing with the shite code that is thrown on to customers machines and then you will realise the scale of the problem of poorly written code.

        Getting back around to base64_[en|de]code there is no need to have hard coded strings inside a base64 function call. It’s purely obfuscation of something, be that nefarious or legitimate. If you really want to “secure” your code/links/etc then dare I say it use ionCube, though in effect that is still only obfuscation.

  22. Josh, you now tell everyone, “Oh Otto said you can use something else.” But Chris points out that Otto’s suggestion is less efficient and causes greater overhead. So, I take it to understand that you would prefer to tax your servers to a greater extend rather than allow a legitimate use of base64. This seems to fly in the face of your desire to make your decisions based on “improving performance”.

    You are free to do whatever you want with your servers. It’s your company. But its also the right of other people to never recommend your crippled server hardware (and even go so far as to tell people to avoid) because of your choices.

    You also still haven’t responded to the fact that core WordPress uses base64. It seems you’ve inadvertently painted yourself in a corner and you don’t know how to get out of the room without stepping in the paint.

  23. Benjamin,

    Do I really need to make that clearer? You seem to need a little more clarity so I will attempt to be less opaque: “And we will certainly make exceptions for what we feel is appropriate.” So.. as the first (2006) and largest (by a wide margin) WordPress.org only host I can tell you with certainty that we would make exceptions for the core app that powers our business. Would that seem appropriate?

    Chris and Otto have divergent opinions. I respect both of their view points. On page.ly, we would rather see alternatives as Otto suggested.

    We use litespeed, others swear by nginx. Our performance benchmarks show we made the best decision for our needs at this time and others will still call us crazy. People have varied opinions, and I respect yours even if I do not agree with it.

    • Ah, seeing as you bring up transparency – you could obviously really help both your Customers, potential Customers and third party developers by publishing exactly what you prohibit now, are likely to prohibit in the next 6 months or 12 months and general limitations and constraints upon what can be done on your system.

      That way existing Customers would have a better idea of what plugins/themes they could and couldn’t use (saves frustration all round and doesn’t push the blame onto the supplier when something doesn’t work because of one of your prohibitions).

      Also, prospective Customers could make a much more informed decision about your service and whether it would suit them (saves frustration all round because I don’t set up a site, load my last plugin that I need and find it doesn’t work).

      And last but not least, developers would know either not to bother to design for your system or that they have to provide some option for compatibility with your system with appropriate warning to the user.

      Does that seem reasonable?

      You might also consider publishing developer guidelines as to what you consider good and bad practice because trying to determine what “…exceptions for what we feel is appropriate” means is, I am sure you will agree, rather tricky. So if you could details something like “we prohibit X because blah, blah but the exception is if it is used for blah, blah in which case it will be allowed”. If you expect people to abide by the rules of your game then you have to tell them what the rules are – then they can make an informed decision as to whether they want to play your game or not. Oh, and remember your Customers as well, if someone is using something that abides by the rules you can’t just go and arbitrarily change the rules.

      And I notice you mention your “performance benchmarks” – in the interests of transparency I assume you will be publishing those benchmarks here so that your decision can be put in context and understood even if not agreed with.

      Regards

Comments are closed.