[Rant] Web development is just broken

April 18, 2011 § 104 Comments

No, really, there’s something deeply flawed with web development. And I’m not (just) talking browser incompatibilities, or Ruby vs. Java vs. PHP vs. anything else or about Flash or HTML5, I’m talking about something deeper and fundamental.

Edit Added TL;DR

Edit Interesting followups on Hacker News and Reddit.

Edit Added a comparison with PC programming.

Edit Added a follow up introducing the Opa project.


The situation

  • Plenty of technologies somehow piled on top of each other.

My nightmares:

  • Dependency nightmare
  • Documentation nightmare
  • Scalability nightmare
  • Glue nightmare
  • Security nightmare

Just to clarify: None of these nightmares is about having to make choices.

My dream:

  • Single computer development had the same set of issues in the 80s-90s. They’re now essentially solved. Let’s do the same for the web.

Shop before you code

Say you want to write a web applications. If it’s not a trivial web application, chances are that you’ll need to choose:

  • a programming language;
  • a web server;
  • a database management system;
  • a server-side web framework;
  • a client-side web framework.

And you do need all the above. A programming language, because you’re about to program something, so no surprise here. You also need a web server, because you’re about to write a web application and you need to deliver it, so again, no surprise.

A database management system, because you’ll want to save data and/or to share data, and it’s just too dangerous to access the file system. Strangely, though, your programming language will give you access to the file system, and it’s somewhere else, at the operating system layer, that you’ll have to restrict this. Now, depending on your application and your DBMS, your data may fit completely with the DBMS, but often, that’s not the case, because your application manipulates object-oriented data structures, while your DBMS manipulates either records and relations or keys and values. And at this stage, you have two possibilities: either you forcefeed your data into your database – essentially reinventing (de)serialization and storage on top of an antagonistic technology – or you add to your stack a form of Object Relational Manager to do this for you.

You need a server-side framework – perhaps more than one –, too, because, let’s face it, at this stage, your (empty) application already feels so complicated that you’ll need all the help you can get to avoid having to reinvent templating, POST/GET management, sessions, etc. Oh, actually, what I wrote above, that’s not quite true: depending on your framework, you may need some access to the file system for your images, your pages, etc. and all other stuff that may or may not fit naturally in the DBMS. So back to the OS layer to configure it more finely.

And finally, you’ll certainly want to write your application for several browsers. As all developers know, there is no such thing as a pair of compatible browsers, and since you certainly don’t want to spend all of your time juggling with (largely undocumented) browser incompatibilities and/or limitations of the JavaScript standard library, it’s time to add a client-side framework, perhaps more.

So, in addition to the first list, you probably have to choose and configure:

  • an OS;
  • OS-level security layers;
  • an ORM (unless you’re reinventing yours) or an approximation thereof;

At this stage, you haven’t written “Hello, world” yet.

On the other hand, you’re about to enter dependency nightmare, because not all web servers fit with all frameworks, not all server-side frameworks with all client-side frameworks, or with all ORMs, or with all OSes, not to mention incompatibilities between OS and DBMS, etc. You also have entered documentation nightmare, because information on how to configure the security layers of your OS is marginal at best, and of course totally separate from information on how to configure your DBMS, or your ORM, or your frameworks, etc.

Note that I haven’t mentioned anything about scaling up yet, because the scaling nightmare would deserve a complete post.

Sure, you will solve all of these issues. You will handpick your tools, discard a few, and eventually, since you’re a developer (you’re a developer, right?), you’ll eventually assemble a full development platform in which every technology somehow accepts to talk to its neighbors. Heavens forbid that you make a mistake at that stage, because once you start with actual coding, there will be no coming back, but yes, you’re now ready to code.

At this stage, a few questions cross my mind:

  • You have reached that stage, because you have the time and skills to do this, but what about Joe beginner? Do they deserve this?
  • Remember that you haven’t written “Hello, world” yet. These hours of your life you have spent to get to this stage, do you have a feeling that they were well-spent?
  • What if you made a mistake, i.e. what if something is subtly incompatible but you haven’t noticed yet, or if one of the technologies you’re using is deprecated, or doesn’t match your security policy, how much time will you spend rooting out all the stuff that’s hardwired with this technology?

So, yes, for all these reasons, I decree that web development is broken. But that’s not all there is to it.

So you have started coding. Good for you.

Now, you have a set of tools that should be sufficient to develop your application – again, possibly not for scaling it up, but that’s a different story. So, you can start coding.

Welcome to the third nightmare: the glue nightmare. What’s the glue? Well, it’s that sticky stuff that you put between two technologies that don’t know about each other, that don’t really fit with each other, but that you need to get working together.

You have data on the client and you want to send it to the server. Time to encode them as JSON or XML, send them with Ajax, open an Ajax entry point on the server (temporary? permanent?), parse the received data on the server (what do you do if it doesn’t parse?), decode the parsed data to your usual data structures, validate the values in these data structures (or should you have done that before decoding?), and then use them (I really hope that you have validated everything carefully). That was the easy part. Now, say you have data on the server and you want to send it to the client. Time to encode them as JSON or XML, send them with Comet – oops, there’s no such thing as “sending with Comet”, so you should open an Ajax entry point on the server (same one? temporary? permanent?) and let the client come and fetch the data (how do you ensure it’s the right client?). Plus the usual parsing and decoding. Except the server code you wrote for parsing and decoding doesn’t work in your browser. Plus, be careful with parsing, because you can get some nasty injections at that stage, or you can just crash a number of JS engines accidentally. Add a little debugging, some more work on garbage-collection and you can send “Hello” from the client to the server or from the server to the client.

Again, the question is not: “can you get this to work?” – I’m sure that you can, many of us do this on a regular basis. The questions are more:

  • was this time well-spent?
  • are you sure that it works?
  • really, really sure?
  • even if browsers can crash?
  • even if users are malicious?
  • how can you be certain?

The client-server glue doesn’t stop here – if only we were so luck. There’s more for handling forms or uploads, or to inject user-generated contents into pages, but let’s move to server-side glue.

Storage is full of glue, too. You have data that fits your application and you’ll want to send it to your DBMS. Now, either you’re using an ORM or you’re encoding the data manually in a manner that somehow fits your database paradigm. That’s already quite a sizable layer of glue, but that’s not all. Sending your application to your DBMS means opening a connection, somehow serializing your data (I hope it’s well-validated, too, just in case someone attempts to inject bogus database requests), somehow serializing your database request, handling random disconnections and sometimes unpredictable (de)serialization errors (by the way, I hope you made sure that the database could never be in an inconsistent state, even if the browser crashes), somehow (de)serializing database responses (can you handle the database not responding at all?) and reconnecting in case of disconnection. Oh, and since your database and your application are certainly based on distinct security paradigms, you’ll have to set up both your application security and your database security, and you’ll have to ensure that they work together. Did I mention handling distinct encodings? Ensuring that temporary bindings are never stored in the database? Performing garbage-collection in the database?

Glue doesn’t stop here, of course. Do I need to get started on REST or SOAP or WSDL?

Again, you’ll solve all these issues, eventually. But if you’re like me, you’ll wonder why you had to spend so much time on so little stuff. These are not features, they are not infrastructure, they are not fun, they’re just glue, to get a shamble of technologies – some of which date back to the 1970s – to work together.

Oh, and at this stage, chances are that you have plenty of security holes. Welcome to the security nighmare. Because your programming language has no idea that this piece of XML or JSON or string will end-up being displayed on a user’s browser (XSSi, anyone?) or stored in the database (SQLi or CouchDB injection, anyone?), or that this URI is actually a command with potentially dangerous side-effects (sounds like a XSRF), etc. By default, any web application is broken, security-wise, in dozens of ways. Most of the web application tutorials I’ve seen on the web contain major security holes. And that’s just not right. If you look at these security issues, you’ll realize that most of them are actually not in the features you coded, not even in the infrastructure you assembled, but in the glue itself. Worse than that: many of these issues are actually trivial things, that could be solve once and for all, but we are stuck using tools that don’t even attempt to solve them. So, chances are that in your application, no matter how good you are, you will forget to validate and/or escape data at least once, that you will forget to authenticate before giving access to one of your resources or that you will forget something that somehow will let a malicious user get into your app and cause all sorts of chaos.

I stand my case. Web development is broken.

History of brokenness

But if we look at it closer, a few years ago, so was PC development. Any non-trivial application needed:

  • application-level code, of course;
  • memory-level hackery just to get past the 640kb barrier (or, equivalently, the handle limitation under Windows – 64kb iirc);
  • IRQ-level coding and/or Windows SDK-level coding (the first one was rather fun, the second was a complete nightmare, neither were remotely meant for anybody who was not seriously crazy);
  • (S)VGA BIOS level hackery to get anything cool to display;
  • BIOS-level fooling.

Five layers of antagonist technologies that needed to get hacked into compliance. The next generation was represented by the NeXT frameworks and attempts to bring the same amount of comfort to Windows-land (OWL, MFC, etc.). Still fragile and complex, but a huge improvement. And the next generation was Java/C#/Python programming. Nowadays, you can install/apt-get/emerge/port install your solution, start coding right away, and be certain that you have everything you need. Nightmare solved.

On the web, we’re still stuck somewhere between the first generation and the second. Why don’t we aim for the third?

A manifesto for web development that works

Time to stop the rant and start thinking positive. All the above is web development that’s broken. Now, what would not-broken web development look like?

Let’s go for the following:

I want to start coding now

        without having to learn configuration, dependencies or deployment;

I don’t want to write no glue

        the web is one platform, time to stop forcing us to treat it as a collection of heterogeneous components;

I don’t want to repeat myself

        so don’t force me to write several validators for the same data, or several libs that do the same thing in different components of my web application;

I don’t care about browser wars

        my standard toolkit must work on all browsers, end of the story;

Give me back my agility

        I want to be able to make important refactorings, to move code around the client, the server, the database component without having to rewrite everything.

Secure by default

        all the low-level security issues must be handled automatically and transparently by the platform, not by me. I’ll concentrate on the high-level application-specific issues.

All of this is definitely possible. So please give it to me this and you’ll make many much happier coders.

Disclaimer My company builds related technology. However, this blogs expresses my personal views.

Tagged: , , , , , , , ,

§ 104 Responses to [Rant] Web development is just broken

  • Alex says:

    C’est vraiment des râleurs ces français…

  • Aninymous says:

    Amen, brother!

  • Mark says:

    N + 1: it must also work on all server types / clouds.

  • edtechdev says:

    drupal is probably the closest to that ideal at the moment

  • marajit says:

    Its called Facebook Apps. Not flexible enough? Oh well, back to the ten thousand piece puzzle.

  • Mark Lemming says:

    This is why programmers get paid a lot of money. Don’t fix it.

    • yoric says:

      Ah, well 🙂

      But I’d rater get paid a lot of money to do something interesting.

    • I think the best thing we can do to make ourselves more money is to make ourselves more productive. The problem with dependency hell is that there isn’t much of an advantage to being smart. You just have to be dedicated enough to learn how to manage the kludges. So I’d love to see it fixed, because it costs me a lot of time, and the fact that it costs someone else a little more time and therefore grants a (small) competitive advantage doesn’t matter when its main effect is to hurt everyone.

      For a counter-point, the accelerating adoption of functional programming is a boon for talented programmers. It enables the best and smartest to be substantially more productive than they would be in non-FP languages, and it’s incomprehensible to the not-as-smart people who really shouldn’t be programmers anyway.

  • Vlad says:

    Have you tried Erbix.com? It’s an online-based IDE that can be used to create webapps. It includes a free plan which allows you to host the app for free, and is based on JavaScript (with some server-side extensions). Basically all the stack that you’ve mentioned about (web server, database, web framework etc) is taken care of so that you can focus on the coding.

    • yoric says:

      Interesting. What’s the security model?

      • Vlad says:

        The engine is RingoJS.org, an open-source server-side JS engine (based on Mozilla’s Rhino). Basically each account gets its own “space” where its JavaScript context is kept; there are no shared locations between them, which ensures their complete isolation.

  • Would be nice, but I don’t see it happening anytime soon. The closest I’ve seen was the LAMP (Linux/Apache/MySQL/PHP)/ WAMP(Windows) / MAMP(Mac) packages. Maybe someone will take the same track and create somehing similar with a few of the other popular frameworks.

  • llou says:

    You simply want to code like a master. You’d better start coding and stop complaining, mon ami.

    • yoric says:

      Don’t worry about me, I can code just fine. Sufficiently well, indeed, that I can afford to think about what next-generation web development platforms should look like.

  • Anhar says:

    I feel your pain, that’s probably why I started my S3ingularity project (Shame I’ve never finished it).

    It is simply a single exe, that you drop onto a server. Its a web-server/web-application/ and DBMS (SLQLite) all integrated into ONE executable.

    It takes seconds to deploy, just run it.

    To create a web application, you simply log into via a any web browser. All code is stored inside the SQLLite DBase, as well as content/Images etc.

    Zero Dependencies, Zero Configuration, and minimal files (just two, one executable + SQLLite file).

    Maybe one day I’ll release to the world 🙂

  • A web developer says:

    The only way I know how to solve these problems is to try to not have them.

    You must *target* your platform, your browser, your framework, your “glue”.

    Here is what I use (or at least try to use).

    “I want to start coding now.”

    Taken care of by Xampp (free) or WampDeveloper Pro (paid).

    “I don’t want no glue”

    XML. It’s not as trendy as JSON today, but is supported everywhere.

    “I don’t want to repeat myself”

    Depends… Pick the framework you know or one for your problem: Yii, CodeIgniter, Zend.

    “I don’t care about browser wars”

    I don’t either. Most of my clients are using IE, so that’s what I use. I’ll check in Firefox to make sure things are compatible and fix everything up.

    “Give me back my agility”
    “Secure by default”

    The rest are an emerging phenomena arising from a complex system with more than 1 part (both human and machine)… Its basically will always be this way.

    • yoric says:

      Er… coding for IE only? Sorry, but that’s not *web* development, that’s Windows-development hosted on the web.

      XML? In my experience, people who believe that XML is the solution to any problem (except parsing), just haven’t understood the problem. In that case, perhaps I was unclear as to my definition of “glue”, but I can’t think of any better formulation for the moment.

      • A web developer says:

        “Er… coding for IE only?”

        I only start with IE, and then make sure it works in Firefox as I go along. This has worked well. I’m not going to throw away / not support the browser *most* of my clients and visitors use.

        “Sorry, but that’s not *web* development, that’s Windows-development hosted on the web.”

        Don’t quite follow you on that one.

        IE is a browser just like Firefox, Chrome, Opera.

        Most of the problems I’ve seen people complain about where a result of them not truly understanding the problem and shifting blame to someone/something else.

        “XML? In my experience, people who believe that XML is the solution to any problem (except parsing), …”

        Agreed! I use XML mostly for programming / native applications and it works well here. It’s something to “standardize” on.

      • yoric says:

        I only start with IE, and then make sure it works in Firefox as I go along.

        My bad, I misunderstood your original comment.

        But in my experience (usually going the other way, FF to IE), this process is painful, error-prone and often ends with dropping support of many browsers.

  • Andy Wentworth says:

    This is why I get paid so well, I hope it never changes! 🙂

  • Jonathan says:

    Django on PostgreSQL

  • Robert S. says:

    “coding for IE only? Sorry, but that’s not *web* development, that’s Windows-development hosted on the web.”

    Couldn’t agree more. IE is _the_ worst browser in regards to respecting standards.

  • Bèr Kessels says:

    You can say this about everything anywhere anytime. I am not only a webdeloper and programmer, but builder too: building houses, laying bricks, carpenting payed my college and summer-fun.

    No-one can start of building a house without choosing the exact stack of technologies, learning them and probably failing a few times during the building process.

    There is no simple guideline to making tables or chairs. Sure, there are many preferred techonologies, but it takes not just education, but years of practice before you can build a chair that will hold my old-auntie-bertie’s fat behind.

    People are often surprised how much knowledge skill and technological knowledge is required to put two pieces of metal together: welding requires experience and theoretical knowledge; bolting requires a degree in physical-mechanics and so on.

    Really: this is nothing special for webdevelopment; software and web-development suffer of exactly the same “problems” and are just as “broken” as any craft or profession.

    Maybe we can find solutions there too: a baker becomes baker after years of being apprentice. Could work well in several regions of software development, for example.

    • yoric says:

      Interesting remark.

      On the other hand, we’re not bakers, we’re engineers. Processes are both the tools of our trade and the things we build. When a process is broken – or simply not good enough – it’s our job to locate the issues and fix them.

      Oh, and I cook bread regularly, and it’s much easier than any kind of web issue I’m faced with 🙂

  • Amen, brother.

    “Best practices” are the solution in the near term. In the long term, some cloud based (probably open or otherwise non vendor specific) solution should emerge as a result because these pain points are very real. Let’s call that a WVM – Web Virtual Machine – that major vendors (Amazon / Rackspace / Google, etc) start supporting.

    And I hate the “job security” aspect of this. Status quo always leads to radical shifts and shocked people loosing their jobs. Seems like “head out of the sand” is a better strategy.

    • yoric says:

      Indeed. If only developers actually followed these best practices… On the security front, I’ve been following OWASP’s efforts to spread the word on web security, and I have the strong impression that many developers just don’t care – often because their employers have no idea and are not willing to invest time or money on anything that doesn’t look like a feature.

  • The solution is already on it’s way:

    Bridging the client-server ‘gap’ wont instantly solve all our problems, but it’s a start.

    • white tentacle says:

      Yeah, server-side javascript, back to the netscape way of doing things ! \o/

    • yoric says:

      Indeed, it does solve a large part of “don’t make me repeat myself”. I believe that this does not quite reach the database, though, nor does it do anything for, say, security. But it’s definitely an interesting step.

      • raphael says:

        You can always use mongodb or couchdeb or several dozen other databases that use javascript for map/reduce.

        Go 100% javascript.

      • yoric says:

        Yes, server-side JS certainly seems to have gained in popularity. And it does help with “don’t make me repeat myself” and somewhat with “I don’t want no glue”. Not really all that much with security, though.

  • white tentacle says:

    webtoolkit (a c++ toolkit for the web) might be one option you may like.It does handle all the silly javascript and security layers. It comes with an orm for the database, you still have to configure your own database, but that’s not much and gives you more flexibility.

    You might as well go with ms azure, for an integrated platform.

    • yoric says:

      What about security?

      • wdu says:

        Security is actually much better handled and integrated by design in Wt (both in C++ and Java) than most other toolkits. Much of the content of your rant is the reason of why Wt exists.

        It protects against XSS attacks because it knows the semantics of a string – i.e. the toolkit knows what is supposed to be displayed text and when it is supposed to be JavaScript, and it filters out all script content from elements that are not supposed to contain them.

        It protects against unauthorized actions by filtering on the server all events except those that can be accessed by what the toolkit knows to be displayed at the screen by a session at any given time. It is impossible to click a button that is not on your screen or fill a field that is disabled – the toolkit will detect it and filter it out server-side.

        All validators have a client-side (for immediate feedback) and a server-side (for security) component.

        It protects against XSRF in the session layer of the toolkit.

        Wt can further protect sessions from each other by running each one in a separate process, so that the OS guarantees that one process cannot access another process’ data.

        Wt doesn’t require a database to run, but if you want to use it and decide to go for Wt::Dbo (Wt’s CRM), you’ll notice that it uses prepared statements everywhere.

        By default, a Wt application is secure. Security is one of the very strong (yet less obviously visible) points of Wt.

      • yoric says:

        Sounds nice. What about SQL injections and buffer overflows/underflows/format string injections/etc.?

      • wdu says:

        * sql injections

        Wt doesn’t require a database to operate. If you decide to use Wt::Dbo in your application, you’ll notice that it uses prepared statements.

        * buffer overrun/underrun/format string injection

        The concept of Wt is not about the language – use the Java version if you feel that C++ is too risky on this point.

      • yoric says:

        Interesting, I’ll take a look whenever I find some time.

  • Gront P. says:

    A very astute observation. Web development truly is a dog’s breakfast at present. I think also it is an area where the majority of lesser skilled ‘developers’ operate and this doesn’t help matters.

    If we follow the biological evolutionary model, then it is just a matter of time until good tools are available. How much time? I have no idea.

    • Np237 says:

      Well, if you like software development, don’t you often find that you don’t want to do web development, and instead find all excuses to work on other kinds of applications?

      Yoric put into words why a good developer doesn’t want to be a web developer. The direct consequence being that web developers are morons.

  • Steve Taylor says:

    Oracle Application Express solves a lot of these problems.

  • Allen says:

    Thought-provoking article!

    “Except the server code you wrote for parsing and decoding doesn’t work in your browser.” I can’t imagine server code running in the browser. Are you sure that’s what you meant?

    • yoric says:

      I meant the source code: if I wrote some source code for the browser, chances are that I can’t use the same source code in the server and even if it does (with server-side JS), it definitely won’t work in the database. This means that I need to write not one parser but two (sometimes more) and hope that they parse the exact same language.

  • woan says:

    Pretty much the same issues we’ve had in client-server and distributed computing for some time… The main differences is greater likelihood that technology stack on client is almost always different than on server in highly interactive apps. Of course you could choose node.js on the server or GWT on the client with their difficulties.

    Those slamming drupal, I think the point someone might make is that you can go back to a mainframe model of server side rendering and limited function terminals, i.e. browsers. Not very competitive but does remove some complexity.

  • No mention of Play framework yet??

  • Anonymous says:

    Ah ha! Finally an enlightened developer. I used to be an open source cheerleader (still am actually). But one day I took an _objective_ look at Microsoft’s platform. I still found isolated components (no way around that), but there was MUCH less glue. Everything ties together in a way that is by default 1) less work, 2) more secure and 3) highly scalable.

    For example, take their memcache equivalent – that’s just a configuration file change and suddenly your cache store moves from in-process to over-the-network. And all your objects are serialized for you – no XML to worry about.

  • sebastian says:

    Ok this is totally biased but you have to see Seaside
    it promises:
    – coding now
    – no glue
    – don’t repeat yourself
    – agility

  • Paul Salcido says:

    I don’t think that web development is broken, it’s the limitations of http, html, and standards bodies (or better: client side applications following standards) that are the ultimate issue. A lot of the problem that you describe are getting resolved, but ultimately you need some kind of serialization and a way to deal with it (no matter how well it is hidden by the application/development libraries that you are using).

    Within the environment that exists today, these problems cannot be easily resolved, and trying to resolve them considering the varying needs of applications (scalability, business rules, etc) is going to take someone a lot smarter than many of us. Different development tools have different ways of resolving these, but it doesn’t change the fact that web techs associated with scaling are, and will be, incredibly complex.

  • See Links: http://groups.inf.ed.ac.uk/links/ , it addresses all your concerns.

  • Martin says:

    Stop crying and make it better.

    You are talking about a high number of complex technologies that need to work together. In my wishful world I would also like to be able to build a car in one day by myself. Is car manufacturing broken? I’m happy that you had an insight into the development process of web applications and I am happy that you will see web developers with different eyes and respect.

  • Luis says:

    “Do I need to get started on REST or SOAP or WSDL?”

    SOAP or WSDL? They’re not competing technologies.

    • yoric says:

      Sure. But they are equally hated by numerous web developers.

      • Luis says:

        Fair Enough.

        What’s funny, is that I loved the technology at the time. Was great to be able to communicate between J2EE and dot net.

        These days I’d much rather use REST and JSON. And I’m leaning more to JS app servers and JSON/BSON object stores. Still have the same layers and security issues you pointed out, but at least each tier is not so different from the next.

        Good article, BTW.

  • […] this post hit the HN front page and to me, the overall gist of it came across as: “Web development is […]

  • Pedro says:

    As a person with business responsabilties, I found this article very, very interesting; finally, someone from the programmng world clarifies why the two worlds colide so often…
    Recently I found Agile Platform from Outsystems (no interests there), that from a outsider perspective solves all this issues, even given proper documentation of the code and processes…
    It’s so flexible and powerfull that we could even install the full platform in each user laptop to deploy “offline” applications.
    For some of programmers, the downside might be that makes programmes very structured and logical, at the same time that makes it visible (not idem in tons of lines of code).
    Just my positive contribution to this post.
    Kind regards, Pedro

  • arbie samong says:

    The one I use currently are Django+PostgreSQL (or mysql for the simple ones) or web2py+GoogleAppEngine. I think the server-side javascript solutions hold great promise.

  • Chess says:

    You need to use Ruby on Rails. If there is anything that can bring developer happiness its using Ruby, and developer productivity for web apps, its Rails.

    • Paul says:

      Surely you’re joking. Rails fails miserably on most of the issues listed.

      It’s actually a mess of duct-taped together solutions, each of which is individually poor, but which comes with enough default glue configuration provided to make it approachable for people new to web development.

      Rails is successful because it claims to solve about half of the problems described here, but it does that rather badly. The dev team became more interested in building their own cult of personality and getting ridiculous lecture fees as a result. (“Rails 3” is actually just a rebranding of an entirely separate framework that used to be called “Merb.” When it became clear that Merb was poised to blow Rails out of the water and deflate their personality cult bubble, they just bought out the company that makes Merb instead.)

  • Albert Bori says:

    Amen! All of these technologies (glue) are built as band-aids or stilts (if you will) to using HTTP/HTML for things that it was never intended to do. HTML was created in 1989 to share documents/texts globally.

    Today, it is used for: games, e-commerce, social networking, SERVER MANAGEMENT (e.g. AWS or VMWare), chat, Q&A, search, professional networking, marketing, and the list goes on…

    What we need is an extensible technology that was built for what we use the internet for NOW. HTML5 attempts to address some of the glue, but it still leaves most the issues stated in the article above.

    Great article, btw. You get +1 points of awesome.

  • Phil Brass says:

    As a web app security guy, I’m very interested in frameworks that allow fail-safe web app programming. I think two platforms that are definitely headed in the right direction are web2py and Ruby on Rails 3. These frameworks include template languages that perform HTML encoding by default. Older template languages make it trivial to code up some XSS unless the developer remembers to call the HTML encoding function every time they write a parameter. If they fail to remember, their app is not safe.

    Newer template languages perform HTML encoding by default, so there is no dependency on having a perfect developer who hasn’t made a single mistake.

    Newer frameworks use ORMs or DALs or LINQs or other database wrapping mechanisms that, by default, aren’t vulnerable to SQL injection. In almost every case, the developer can still go out of their way and shoot themselves in the foot, and in some cases the ORM turns out to have a SQL injection in there somewhere (RoR3’s limit function, IIRC), but that’s easily patched. The point is that we’re not counting on developers to remember to escape or parameterize every single SQL statement. We know developers are human, so we should be using frameworks that allow human developers to fail safe. By default, ORMs and DALs and such do this, at least in terms of SQL injection.

    Frameworks like RoR3 and web2py also have built-in, automatic CSRF protection mechanisms. The developer doesn’t need to declare a CSRF token somewhere. The framework takes care of it.

    Both frameworks are also big on “don’t repeat yourself”, so that’s cool as well. I think there are also components that are similar in the .Net and Java world, but most of them aren’t quite as advanced in every area.

    I would also like to see frameworks with more built-in, fail-safe support for fine-grained access control on data items and URLs/actions – that is, making data items that have a user id only accessible to the specified user (by default), ensuring that all URLs are only accessible to logged-in users by default, and having simple conventions that make it easy to know which URLs are accessible to which roles, and which data items are accessible to which groups. Eventually it may get too complicated to rely solely on conventions like db foreign keys and URL names, and you’ll need a full-on rule-based reference monitor, so the framework should have some support for that as well.

    Also, support for relative object IDs instead of using database object IDs directly in parameters and URLs would be nice to have built-in. That is, having a framework that lets a user access their 3 accounts with ?account_id=1 and ?account_id=2 and ?account_id=3, instead of using the actual account number. If that could be built in to the database access layer as the default way to get ID values for objects, it would also help prevent access-control problems where users put somebody else’s account ID into the show_accounts page (or better yet, the transfer_funds page…).

    Anyway, I like the new fail-safe style of some of these modern frameworks, and I hope they get even better going forward.

    • yoric says:

      Yes, I definitely think that safe templates and LINQ-style mechanisms are a step in the right direction.

    • wdu says:

      I posted a short writeup on the security model of Wt and JWt in response to another comment on this page, which you may find interesting. Many security issues can be dealt with by the frameworks, rather than being the responsibility of the developer.

  • Stu says:

    Hmmm… how about the new GTK HTML backend (broadway) … I guess this holds the promise of just writing apps and having them stream over the wire (I’m sure there must be some problems with this, but still – it seems silly that the web must reinvent everything).

  • […] [Rant] Web development is just broken No, really, there’s something deeply flawed with web development. And I’m not (just) talking browser incompatibilities, […] […]

  • tomfumb says:

    Unfortunately I think the closest you’ll ever see is ASP.NET, WCF (maybe) & MSSQL. It ticks many of your boxes (simple server setup, in-built security, in-built glue, *some* cross-browser good times, the ability to do much of your server & client logic in the same language, in-built validators, abstracted serialization, and very sophisticated IDE and DBMS client).

    If only it wasn’t made by Microsoft and such a massive pile of shit… Can you really imagine anyone other than a massive company like that being able to pull this off? Your one-stop-solution as an Open Source project would have at least 10 forks after the first year

  • I understand the initial architectural process is laden with these kinds of issues, but it’s really not that hard. After you’ve done it two or three times, you have a pretty good idea of what works and what doesn’t, and you can develop a standard for there. If you find yourself back at square one for every project, then you haven’t learned much from your past experiences.

    For most companies, once you’ve got your platform established, it can become a cookie-cutter operation for every similar platform after that. Granted that some will have special requirements, but even if your company does client based work, the needs of most clients within and across demographics are basically the same. Pick a standard and run with it.

    • yoric says:

      So, if I understand your answer, it’s not that hard, and you get used to it, and eventually, you learn to ignore it and/or to do it in your sleep.
      Sure. But part of my rant was about the fun of coding. What you’re describing – and what I believe many of us experience all too often – is a tiresome, repetitive process, that holds no fun, no challenge. And that process also happens to be error-prone, sometimes in subtle and scary ways, if security is an issue.

  • /V says:


    maybe this could help… you might wanna give it a look!

    • yoric says:

      Yes, I’m aware of it, and I believe it shows great promises. Truth be told, the project I’m working on shares some of the initial ideas with HOP, although with a much heavier emphasis on security.

  • rad_g says:

    You are ranting a lot but don’t have any ideas how to improve. If you do – share.

    • yoric says:

      You are ranting a lot but don’t have any ideas how to improve.

      I certainly do.

      If you do – share.

      Planning to, in another blog entry.

  • Norbert says:

    You think of these things as broken and a hinder for you to enjoy coding or just doing your work. But I think they are an opportunity to show our excellence and craftsmanship.
    And there is always the option: “If you think it’s broke, fix it”.
    For me, I like the challenges the web gives us. Bring it on.

  • vasya10 says:

    If things were simple, most of us will be out of job, and sales people will write code. i dont want that. complexity is very good. very very good. complexity keeps brain sharp.

    • yoric says:

      There are two distinct things. Complex and smart keeps brain sharp. But repetitive, boring and error-prone dulls it.
      Here, I classify as nightmares tasks that I consider repetitive, boring and error-prone. I want to concentrate on complex and smart.

  • gorlok says:

    Hilarious 😀 I have news to you: PC was/is broken too. Lets go back to mainframe, my friend!

  • KoW says:

    one thing i dont get: why exactly is the situation much better in non-web environments? you indeed have to choose an os (oh, all those nasty little differences between the different linux distros…), a orm, etc, you have to choose a presentation framework.
    if your project is succesfull, you better have thought about a scalable solution…
    security-wise: look at all those adobe pdf attacks…
    not that much better imho

    • yoric says:

      Good point, but there’s a misunderstanding about my rant: I have no issue with choice. I have issues with lack of documentation, needless, repetitive and tiresome configuration, hidden incompatibilities and writing insane amounts of glue/adapter code, especially when this glue/adapter code is itself both repetitive, tiresome and the weakest link in the security of the application. And, for most non-web applications, nowadays, I have little to nothing to do in all these domains.

      The point of pdf attacks is interesting and I have no clear answer on this, except perhaps that Acrobat Reader has grown so powerful that it has reached the same complexity as a web browser a long time ago, which by itself is basically the same complexity as an operating system.

      And if you’re asking for the fundamental reason of the difference between web and non-web, I think it’s simple: 20 years of maturity and lots of efforts in research.

  • why is no-one raising objects? says:

    Actually, I think this article is 100% bullshit.

    He’s comparing “hello world” in python to “hello world” for the web, and he claims he can fire up python and get hello world immediately. On the other hand, he claims you need a DBMS and an OS if you want to do this for the web. Wait, what?

    “Nowadays, you can install/apt-get/emerge/port install your solution, start coding right away, and be certain that you have everything you need.”

    Rails gives me that. I apt-get rails, write the damn code, then do “rake push heroku” and it’s running online. did I have to choose rails and heroku? Yes. So did the author if he wanted to write python code with a qt interface.

    I think this problem is *all over the place* the moment you’re writing non-CLI apps. This was the sweet part of visual basic. You buy into the platform, and you can throw together little gui apps in minutes. Wanna do this in C++? Not as much fun. Wanna do this while being OS-independent? Ouch. OK, so Qt can do this. Now you’re in the same position as what he’s complaining about – what if qt doesn’t do exactly what I want? what if it doesn’t scale well? oh no!

    you can buy yourself totally into the windows world and then ASP.net will give you what this guy wants. Code for IE, using ISS, and you get all this decisions made my microsoft, as others have noted.

    Here’s my counterargument. How much do you have to do to get a visual hello-world app on the web? In the days of geocities every second kid had a webpage that did this. How many had gui apps running on the desktop? none.

    what pisses me off much more is, if I want to do a “hello world” gui app I’m way way more painted into a corner than i am on the web.

    • yoric says:

      I concur that the OS-independent constraint makes things much easier. I haven’t coded for Windows in ages, so I don’t know how easy it is to write and deploy for instance a Qt or Gtk or SWT app under Windows nowadays, but at least Gtk used to be pretty hard. If anyone could contribute on the topic, this would be interesting. Otoh, between Linuxes, it used to be trivial – not a major victory, I grant you.

      Here’s my counterargument. How much do you have to do to get a visual hello-world app on the web? In the days of geocities every second kid had a webpage that did this. How many had gui apps running on the desktop? none.

      Good point. But then, that’s using technology <marketspeech>that’s so Web 1.0</marketspeech>, that is to say, technology that just doesn’t scale up to any application more complex than Hello, World. Possibilities and needs have evolved considerably since Geocities. And I have the very strong feeling that our tools haven’t evolved nearly as fast. And that we can definitely dream – and build – something better.

  • pequt의 생각…

    웹 개발은 정말 정말 정말 넘어야 할 단계가 많아서 힘든 것입니다… 하지만 예전 PC 애플리케이션 개발도 그랬죠. 안 힘든 개발이 어디 있겠어요….

  • pequt의 생각…

    웹 개발은 정말 정말 정말 넘어야 할 단계가 많아서 힘든 것입니다… 하지만 옛날에 PC 애플리케이션 개발도 그랬죠….

  • Radek says:

    The web is not one platform, it is built up from different tiers so expect having to work with different tiers.
    You have different clients as well, thus the complexity increases further more. You can use a technology that will handle your back end and front end and some glue at the same time by the way, it is just you prefer (perhaps) to use the different technologies because they are best in their area. If you are thinking otherwise, look at Naked Objects architecture etc.

    • yoric says:

      Well, 20 years ago, my PC was built from different tiers. The BIOS and the DOS were distinct – and antagonist – tiers, so were the RAM and the DOS, while Windows 3.x imposed a system-wide limit to 64kb of pointers (if my memory serves correctly), video cards were completely incompatible with each other and needed to be hacked into with direct machine code – stories tell of exploded monitors, although I haven’t witnessed any myself, etc. None of the relevant information, of course, appeared in any manual, or in any programming course, but had to be gathered from sometimes quite obscure sources.

      And somehow, platform maturity progressively crept in. Today’s development is still imperfect, but it is impressively clearer than what it was at the time.

      At the time, would you have believed that such maturity would be possible? Would I have? I have no idea. But I still think that we can dream (and build) a web developer experience orders of magnitude better than anything we have to this day.

      As for building the best possible experience depending on the domain, I’m actually a big believer of Domain-Specific Tools – can’t count how many Domain-Specific Languages I’ve built in the past 10 years. And this makes me think that we are approaching the problem at the wrong level of abstraction.

      If you are thinking otherwise, look at Naked Objects architecture etc.

      I will, thanks for the source. Should I look for something specific? I’m not familiar with this architecture at all.

  • Tom says:

    Hi. Great blog post. Wrote a response over here:


    Would be interested in your thoughts.

  • […] development has a number of big problems. I have trolled written about some of them in previous posts, and Dart was definitely designed to help, at least a […]

  • Alejandro says:

    Use ASP.NET (Webforms or MVC) + Entity Framework on IIS, most of the problems you face are solved by default. Except for being tied to Microsoft is an almost perfect solution. 😉

Leave a Reply to Tom Cancel reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

What’s this?

You are currently reading [Rant] Web development is just broken at Il y a du thé renversé au bord de la table.


%d bloggers like this: