The quickfix

You can fix this by doing the following:

1. Type about:config in the Firefox address bar
2. Look for the setting called layout.css.dpi. The default value is -1.
3. Change it to 96

There may be a deeper problem; Firefox on Linux is a GNOME application and so uses the GNOME window manager settings. Kubuntu uses the KDE window manager, so the DPI settings for it do not seem to apply to GNOME and I don’t know where/how to set GNOME ones while running Kubuntu.

Some forums suggest a few things that have not worked. While the above change in Firefox has worked, ideally I’d like to change the GNOME settings instead. Anyone have any ideas on that?

Getting Firebug working properly, too

To get Firebug to work properly (YSlow 0.9.5b1 seems to stop it working, which is a shame):

1. Uninstall YSlow if you have it
2. Uninstall any existing Firebug
3. Install DOM Inspector if not already installed
4. Get Firebug 1.2 alpha (1.1 beta did not seem to work properly for me)
5. Restart as prompted

I tried to reinstall YSlow, but no joy. That’s a bummer as YSlow is useful. Until they fix it, you can remote to a PC/Mac running it or use a virtual machine of something that runs it!

Some additional thoughts/notes

The Mozilla Knowledgebase on layout.css.dpi says that the default of -1 means to use the OS dpi or 96, whichever is *greater*, although Kubuntu/KDE reports 96! Somewhere there is a higher value, but I am a bit new to Linux to know where to find that.

While I initially suspected this to be a problem specific to Firefox on Linux when I used the vector graphic application, Inkscape, as I was exporting an svg to a bitmap, I noticed that the dpi settings it reported was something like 116dpi. Inkscape is also a GTK application, so now I think that it is a DPI setting used by GNOME.

Unless someone knows how to set those things while using Kubuntu, I have to make my change in Firefox itself (and see what happens when the final version is released). A few forums and posts suggested things like running the gnome setting daemon at start up, using the KDE GUI for GTK settings etc, but none of those seem to make any difference. Any ideas anyone?

I can live with this for now (I just need to figure out how to get those ugly close buttons replaced with something else — anyone know how to get GNOME apps to use a different icon set or theme when running Kubuntu???)

While I won’t summarize all the things that the session speakers said (there are some links towards the end for that), here are some key things I took away from each presenter:

Open Data — Keynote Presentation from Jeremy Keith

This guy is usually quite funny, but this time he decided to read something he had written. I thought this was going to be a long and dry session, but instead it was very interesting, looking at the importance of

• Open data for accessible information (to everyone) and for digital preservation
• Standards for format longevity and innovation (bred by constraints)

He was approaching accessibility from the perspective of accessible data for everyone.

Assistive Technologies and AJAX by Steve Faulkner

Steve Faulkner is a noted accessibility expert and has lots of detailed information about accessibility on his blog (and in particular, some decent research on how screen readers work in different scenarios). His examples mainly came from Twitter, the micro-blogging tool.

There were some interesting examples of how WAI ARIA live regions would help make some simple features accessible, such as the text countdown that shows how many characters you have left to type, common on many sites where you fill in text areas.

(One of his points about the problem of using the abbr element’s title attribute to put in an ISO 8601 date time format eventually led to a public argument during the final panel session between Mike Davies and Jeremy Keith!)

Fencing in the Habitat by Christian Heilmann

Christian Heilmann works for Yahoo but has long blogged about accessibility, JavaScript and more.

This talk was about how not to approach accessibility. He has already published his slides and notes for the talk, so the only thing I will add here are my main take-away points:

• There’s no need to scare people into doing accessibility anymore
• It’s generally easy (despite a lot of misconceptions) and helps so many people. He described the immense joy some elderly people had when they were given a wii without any knowledge how to use it. Yet, very quickly they were having immense fun playing on it! Accessibility is more than guidelines; it is also about usability.

Rich Media and Web Applications for People with Learning Disabilities by Antonia Hyde

This talk was really interesting about how a group that is often left behind in accessibility implementations — people with learning disabilities — are affected by the web.

A number of instructive videos showed how some people can struggle with common features on sites.

She ran out of time but put up her slides on her blog.

User Generated Content by Jonathan Hassell

Jonathan Hassell is the head of Audience Experience and Usability for BBC Future Media & Technology. He also leads the BBC Usability & Accessibility team so had quite a lot of interesting things to say.

User generated content is of course a major issue in modern web sites when it comes to concerns like standards and accessibility.

One of the strategies Hassell mentioned was to use a moderator approach, accepting it is really hard to get end users to know about how to create accessible content. This way, a moderator adds accessibility and other things once the content has been created.

This is quite an expensive way to do thing, he accepted and sometimes will not be possible.

The other problem is that as more content moves away from basic text-based formats that is the web, assistive technologies will struggle even more to provide accessible alternatives. While not impossible, the implications include the following:

• Those formats require accessibility be built in, or have alternatives
• Content producers will become increasingly responsible for accessible content

The other point he stressed, like many others, is that accessibility is a misleading word; it is really an issue of usability.

As content becomes more multi-media the additional challenge he finds is how to make such multi media accessible in innovative ways. One example was using 3D and stereo sound in a Flash-based children’s game where they had to push a train carriage from the left part of the screen to the right hand side. By using 3D sound and stereo, audio feedback to a child who is blind or has poor eyesight, would still allow them to experience more of the game than usual. Another game showed sign language being used.

Another interesting point he made was regarding JavaScript: As a number of people have noted for a long time, there is a common misconception that JavaScript is an accessibility no-no. Hassell noted that it helped make the new BBC home page more accessible to screen readers, rather than less.

A Case Study: Building a Social Network for Disabled Users by Stephen Eisden

This talk was about the user testing, accessibility audits and technology decisions that come with building a social network for disabled users called Disability Information Portal (DIP). After evaluating a few different ways to do this, they settled on WordPress as the basis of their network, currently in pilot stage, http://www.dip-online.org/.

Eisden also noted that an accessible site was generally a more usable site, too.

Tools & Technologies to Watch or to Avoid by Ian Forrester

Ian Forrester runs BBC’s Backstage, a web site for designers and developers.

This promised to be an excellent session, but for some reason they only gave him about 20 minutes and he had some 80 slides to go through! So he had to cut really short which was disappointing. I had the opportunity to talk to him a bit more after the sessions were over, and that was really interesting. He has a good interest in XSLT too

Some of the points he did manage to get across were how some Web 2.0 sites made it hard enough for normal users to do things, let alone people with additional needs. Examples included hard to understand user licenses and terms & conditions; Flash videos and captioning; using various sites to import contacts or export data, etc.

Accessibility 2.0 Panel Discussion chaired by Julie Howell

This panel was an interesting enough discussion, chaired by one of the most prominent people in the accessibility circles. Nothing to really add here that is not captured in the notes by others listed below.

Slides and notes from each session

AbilityNet, who hosted the conference will be putting up videos and transcripts of each session shortly.

Jeremy Keith, one of the speakers, and well known for his work on DOM Scripting (I think he coined that term?) was taking notes during the session and posted them as follows:

1. Open Data by Jeremy Keith.
2. Making twitter tweet by Steve Faulkner.
3. Fencing in the habit by Christian Heilmann.
4. Rich Media and Web applications for people with learning disabilities by Antonia Hyde.
5. User-generated Content by Jonathan Hassell.
6. A case study: Building a social network for disabled users by Stephen Eisden.
7. Tools and Technologies to Watch and Avoid by Ian Forrester.
8. Accessibility 2.0 panel discussion with Mike Davies, Kath Moonan, Bim Egan, Jonathan Hassell, Antonia Hyde and Panayiotis Zaphiris, moderated by Julie Howell.

Christian Heilmann also did some live-blogging and posted his notes up on the Yahoo Developer Network. He also includes a number of additional links to speakers, and other live-bloggers.

Summarising the whole day

A common theme running through the day was that web accessibility is not just for disabled people; it is about usability and ensuring as wide an audience as possible can access the content.

Jeremy Keith also had a useful summary:

All in all, it was a great day of talks with some recurring points:

• Accessibility is really a user-experience issue.
• Guidelines for authoring tools are now more relevant than guidelines for content.
• Forget about blindly following rules: nothing beats real testing with real users.

– Jeremy Keith, Open Data and Accessibility, April 26, 2008

I find it interesting that a number of interfaces are now offering “shortcuts” to mouse clicking everywhere.

While desktop search applications already provide this kind of convenience for finding files, such an interface for finding commands is quite interesting, to me.

I have noticed this more so in products used typically by technical people.

For example, on the Mac at various places, such as the System Preferences, you can type for the settings you want to adjust and the application will highlight possible matches for you:

KDE 4, a Linux window manager, has a start-like menu (called the KickOff) where you can type the program you want to find:

Vista has a similar thing from its Start menu:

Various software development tools from the excellent Resharper add-in for C# in Visual Studio, to API documentation readers, all offer some way to type in the class, file, method etc you are looking for. And such features are sometimes an efficient alternative to clicking through large trees of information:

Someone wrote a very useful jQuery API guide where the main form of navigation is to begin typing for the api feature you are looking for:

The Office add-on for finding commands, while still in early testing, is a feature that targets more than just technical users, so it will be interesting to see how this works out.

Is Microsoft’s announcement a sign of failure of the Ribbon interface? I don’t think so; it may be more reflective of the challenge they find of getting users who are so used to a well-established product to doing things dramatically differently.

As people are getting more used to searching on the web, perhaps that metaphor is making its way back into more traditional user interfaces and that typing is perhaps no longer seen as a poor usability option compared to clicking around using a mouse.

(To clarify, I don’t expect, or want to imply, that typing/command lines etc should replace window managers and pointing/clicking interfaces; instead, there seems to be some appropriate situations where typing looks more efficient than clicking, often in places where the GUI was there because of the advantages of pointing and clicking. Or, at least, typing helps augment a GUI, reducing barriers.)

This reminds of my first job (technically a placement year/internship at Northern Telecom in London but it was such a good experience I consider it a proper job!) in 1996 looking at the HP-UX box on my desk (HP’s UNIX machines) trying to get it all set up and asking the person opposite how to use the file manager GUI to do a certain thing; she just looked at me, smiled, and said, “just use the console; typing is a lot more efficient!”

Typing seems to have become the new mouse clicking…

I have to admit, I have never thought of myself as a graphic designer so this is really nice to see.

If you like it, why not head over to the Logo Design Love Awards site and add your vote — come to think of it, why not vote anyway, even if you prefer other logos

I also had a post on the implications of that meta switch.

Well, it seems that the IE team have decided to change that decision, and decided that IE8 will, by default, interpret web content in the most standards compliant way it can.

The reasons cited were to match Microsoft’s recently published interoperability principles, and due to developer community feedback.

It seems that legal concerns may have also been a factor.

Rather than go through all those issues here, have a look at these for more information:

• Microsoft Spins IE 8 Rendering Changes, Microsoft Watch
• Principles and Legality, by Eric Meyer (see also the interesting comment by Matt Wilcox in the comments thread)

It certainly seems like a welcome switch. If nothing else, Matt Wilcox made an interesting observation in the link above, that if this happens, Microsoft will be able to get a good set of stats on who actually decided to opt out and go for the IE7 rendering mode, and this may help inform a better and more accurate decision for IE9.

Are there any catches, or is this a welcome decision?

Update: Looks like a whole new raft of features coming in IE8 — some of them are playing catch up with other browsers, while other things seem new. (Interesting that they are increasing parallel HTTP connections to 6 per host, not just 2. Hope other browsers do similar things…!) Are we finally seeing innovation with IE again?

This post follows on from previous posts about the way markup is sometimes created.

In a previous post, I worried about some web frameworks that abstract away the creation of markup, thus preventing (or making it more difficult for) web developers to create what they need — the leaky abstraction.

This post explores one way to overcome this, using XSLT.

As this is a rather long post, here are the key points:

• Creating good quality markup is important for a variety of reasons (quick loading pages, cross browser compatibility, accessibility, etc).
• Some server side frameworks may either generate poor quality markup or use templating systems that are specific to that framework.
• Using XSLT to generate the output can be one way to overcome these limitations, while giving you full control of the markup that you need to be create.
• Markup generation via XSLT may be simpler to unit test as part of a developer’s daily work, which may not be the case (or not necessarily as simple) in other templating systems.
• Being platform agnostic, XSLT can be an effective part of a View in an MVC framework and be applied to different frameworks and platforms, reducing upskilling costs in the long term.
• XSLT may not always be appropriate in all cases, and many are certainly put off by its oddities or perceived limitations, but perhaps it deserves another look?

(Some forth-coming posts will provide some code-examples, in particular of unit-testing XSLT and how it might fit into MVC as well as how it can be written in a reusable manner.)

Poor quality markup from the server side.

A previous post on WYSIWYG editors described how the output quality from editors that hide content markup generation can result in markup that is of poor quality, hard to customize/correct/change, affect accessibility (which is a legal requirement in some countries), result in excess bloat, etc.

Some server-side techniques, tools, and frameworks can contribute to this poor markup. This can happen for number of reasons. For example,

• Popular IDEs may encourage the use of WYSIWYG editors to drop in and drag around HTML controls (or groups of them). As argued in that previous post, WYSIWYG as a paradigm for HTML may actually be inappropriate, (unless you treat WYSIWYG as WYSIWYG-when-css-is-not-enabled).
• Popular frameworks may abstract away the creation of HTML, to do it for you. ASP.NET and GWT are such examples. But, as argued in a previous post, such abstractions are leaky.
• Some popular scripting environments have also encouraged spaghetti code (mixing markup, presentation, and business logic, for example).

And so it seems there is a gap between web standards and server-side output generation that is web standards-friendly.

Web application frameworks help immensely

Various frameworks have been commonplace for a long time now. These help separate out the presentation, business/mid-tier and data access logic.

Most frameworks are not solely about markup generation (if that is their priority at all). Frameworks have been important to address a number of important issues from data access, scalability, performance, transaction management, object-relational mappings, code maintenance, and so on.

These are certainly of value and usually worth using as they have teams of people devoted to them. At their best, they empower the framework user (developers and designers, for example). Yet, on the other hand, some frameworks may take some things too far, thus disempowering framework users.

Some frameworks still produce poor quality markup

Those of you who have used ASP.NET 1.x will know what a mess its controls create in terms of markup output. (ASP.NET 2 is far better, but can still encourage poor markup or make full control trickier than it needs to be.)

While server side programming using the .NET framework, and C# in particular, are extremely powerful, when it comes to creating web sites, that power is used to typically generate poor quality markup (even though it may be maintainable!).

Some of this stems from the fact that although it tries to hide away the markup creation, ASP.NET is a leaky abstraction. (This isn’t just a problem with ASP.NET, but potentially anything that tries to make markup creation a hidden, dark art!)

Abstraction: a good idea but leaky

Abstraction and loose coupling are certainly important, especially between layers such as presentation, business/middle tier logic and data access. It is a common approach to writing software.

Understandably, developers often like the idea of abstracting away markup/output creation, too. In ASP.NET for example, you could create a simple site without almost writing any HTML. Instead, you instantiate/declare controls such as asp:Label, asp:Panel etc, and you trust .NET to create the markup for you(!).

Interesting attempts are even available so you don’t have to write JavaScript and you get the benefit of tool support in IDEs for .NET and Java. For example, Script# enables developers to write code in C# (or any .NET language) and then compile it into JavaScript. Google has also been doing such a thing for a long time with its Google Web Toolkit (GWT) for Java. Such ideas are certainly very interesting and appealing.

While appealing perhaps (and maybe more to back end developers), a web developer also loses the ability to control how the output is created.

Gaining control of how you create the output

Why does it matter?

Being able to control how you create the output ultimately gives you greatest flexibility, especially when you need to create a flexible, rich, yet lean web-based UI.

Using the “web standards” approach (i.e. following W3C recommendations) to achieve these things most efficiently has been well thrashed out for years.

And, just as developers take pride in the quality and readability of their server side code, web developers increasingly take pride in the sites and web apps they create from the code perspective, too.

People from anywhere can easily see your markup; less will see your server side code (though that is not an excuse to write poor server side code!).

Although this is by no means the only factor, when hiring we will often look at sites that people have mentioned in their CVs/resumes (or ask for examples to be sure to see what is theirs) and try to understand how they have coded bits.

Some frameworks give full control but may use specific templating languages

Templating languages that accompany many popular web development frameworks may be open, but use mechanisms specific to that framework.

But there are clearly advantages to this approach, too (else they wouldn’t be so common).

For example, templates based on HTML, combined with the language of that framework (e.g. PHP, or Python, or Ruby, or C#, etc.) means there is some familiarity for developers to get on with the task at hand.

Consultants and contractors moving between frameworks

If you work as a consultant or contractor for numerous clients, you may find yourself moving from project to project and then having to learn different templating languages, some with more limitations than others.

In some cases it may be that you end up limiting yourself to just advising on markup, or only providing XHTML/CSS/JavaScript template files, which other coders then have to reproduce.

I have talked to many who have been frustrated at this process, especially with .NET, it seems, as their nicely validating, clean, accessible markup becomes something completely unrecognisable, when ASP.NET gets its way! Unnecessary trade-offs then result.

At work, we have recently been hiring both permanent and contract web developers mostly for work with .NET platforms. We have found it hard to find people with exceptional skills at both the front end and .NET back-end. We tend to find either good front end skills but a different back end (PHP, Java, etc.) or good ASP.NET but little or no real front end knowledge.

In some instances, this is not a problem, but other times it has been preferable to have .NET and good client side skills.

XSLT as the templating mechanism can perhaps help achieve the best of both worlds:

XSLT — an open standard

XSLT could help achieve both abstraction and full control of the markup.

XSLT is a templating language for transforming XML documents into another format (another XML document or some other text-based format).

For web development this can be useful to output exactly what you want and thus gain full control of the markup output generation.

You can re-use XSLT, and even create a hierarchy of XSLTs, that follow a Method Template design pattern, similar to a class hierarchy in Object Oriented Programming, whereby the base XSLT is equivalent to an abstract base class, and other XSLTs that import the base one can override various templates. (A future post will demonstrate this.)

It does of course assume you can easily (or without much cost) get your data in XML format, so you can transform it.

XSLT in MVC

Web developers are familiar with separation of concerns (semantic HTML markup for good document structure, CSS for style/layout, JavaScript for behaviour) and progressive enhancement (or graceful degradation).

Back-end/mid-tier software developers are also familiar with separation of concerns (n-tier architectures, single responsibility per class, MVC, etc.)

The Model-View-Controller (MVC) pattern for developing user interfaces has been around for a while. It is a decent approach to separating concerns (the model to represent the data, the view to represent the presentation, and the controller to bind them together).

It can also be seen at a high level (a typical n-tier architecture), as well as being used within the presentation tier (the model may interact with the middle tier, the controller will be part of the presentation tier passing the data (model) to the view code to display it).

It is a well-proven pattern in graphical/desktop application development and being successfully applied by many web-based frameworks today.

MVC can also help improve testability. For example, unit testing can sometimes be easier by testing specific views, models and controllers in separation.

Microsoft has recently announced an MVC model for ASP.NET development. I understand one of the many drivers is the unit testability.

From an initial read, this sounds quite good. It encourages better separation. Interestingly it also seems not to support their post-back model which most of their controls are built from. It is therefore not clear to me at the moment if this means you write out more HTML (meaning you get more control of what you output) or if you still use packaged controls (which may risk getting HTML that is not appropriate, or hard to control).

But Microsoft is a bit behind on MVC for web development (though that can also mean it can learn from the earlier frameworks). Other popular frameworks have existed for a while that do this for the web, such as Ruby on Rails, Django, CakePHP, and many more etc. (I am quite impressed with the Python-based Django, for example, even though I have only had a cursory exposure to it — and I usually prefer curly brace type programming languages!)

While these all have their own merits, they all have different ways of constructing the markup.

XSLT, then, could be part of the View part of an MVC framework. (It need not be the only way to create the view, of course.) For example,

• A Controller can invoke a Model to get data the way it needs to
• The Controller can pass the Model to the View
• The View can internally serialize that data into XML and transform it with XSLT to the output.

In this way, the markup generation is still nicely abstracted away, but inside that abstraction itself, you have full control of what you need to output.

Advantages/Benefits of an XSLT-based approach

Here are a few more detailed reasons XSLT can be useful (though, of course, some of these reasons are by no means unique to an XSLT-based approach).

XSLT is an Open Standard

People often talk about the benefits of using open standards on the web client side of things (”web standards”, for example), so why not on the server side?

XSLT 1.0 also has reasonably universal support. (Universal 2.0 support would be even better, though we are not there yet, unfortunately.)

Better Separation of the View from the Model and Controller in MVC

A follow-up blog post will demonstrate this in PHP and, time-allowing, Mono/ASP.NET.

Unit Testable

Many framework templates have been tricky, if not impossible, to fully unit test in isolation from other layers and dependent systems.

ASP.NET’s new MVC model mentioned earlier has unit testing and test driven development as a driver for this approach, but from what I understand, while it makes controllers and models unit testable, the views are still not (please correct me if I am wrong in the comments below!).

From what I have seen of the other View templates in other frameworks, I believe the same constraint applies (but please let me know if I have missed something).

XSLT, however, requiring mostly an XML input (and possibly arguments passed in programmatically), can therefore make it easier to unit test the view, in isolation of a running web server, database, etc. It can therefore be easier to integrate into a developer’s coding process.

XSLT-based unit tests don’t necessarily need a whole web server set up to execute. Such unit tests, which are typically numerous by their nature, would be quick to run for the developer during development. This is important in the adoption of unit testing so it doesn’t make running them feel like watching paint dry after each modification!

Another follow-up blog post will attempt to demonstrate this.

Gain Full Control of the Markup Generation

As explained above, markup is important. XSLT gives you the ability to output the markup as you want, with little or no trade-off. (Some other frameworks also allow this, so this isn’t a unique advantage, by any means.)

Platform and Framework Agnostic

While the suggestion of using XSLT may introduce a problem of yet another technology, it may also enable skills-standardisation across platforms. This would be useful for say web developers that need to work on multiple projects, for different clients, using different technologies.

This of course makes the assumption that XSLT would be widely adopted in this way! If it were, migrating to desirable new frameworks where the View at least is somewhat XSLT-driven may not be as time-consuming (though of course time is still required on the middle-tier and database layers).

I have, for example, migrated an XSLT-driven site based on “classic” ASP, to PHP-driven XSLT, using the same pattern, but where the XSLT hardly changed at all, saving a fair chunk of time.

For a major client of ours at work, their extremely high-traffic .NET 1.1 web site (now moving to .NET 2.0) was architected to by-pass the .NET controls, to avoid all the problems described above, and we transformed XML with XSLT.

Doing this gave us a number of benefits all in one go:

1. We got the lean — and accessible — output needed
2. The “object-oriented” XSLT-based approach also allowed us to create reusable template hierarchies
3. The customer also wanted multiple sub-sites which was achieved with very little effort or duplication of code, while allowing each sub-site to be customised as needed — not just CSS or database-driven data differences, but template level changes such as markup and layout changes. (Had we attempted this with ASP.NET, we would have had to create Server Controls with lots of configuration and then in each sub-site we would have to wire them all up, which would lead to a lot of repetition.)
4. In one case, it wasn’t just separate sub-sites they wanted, but entirely separate web sites/applications (a public facing one, a kiosk-mode one, and a version of the site that could be used by customer support). All this ran off pretty much the same XSLTs with minor adjustments in inherited XSLTs for each web site.
5. The migration to .NET 2 involved no XSLT change (only C# changes to take advantage of things like generics, the improved XSLT processor, etc.)

And the sites comfortably take thousands of orders an hour on very few web servers (though of course performance is a bigger architectural issue; the point here is that XSLT is not a performance hindrance that some believe — see below for more on this).

Portable skill

Being both platform and framework agnostic makes this a portable skill. Of course, XSLT is not limited to just web-based scenarios, either, so it may be quite a good skill to have, generally.

Repeatable

Just as OO techniques and design patterns help with repeatability in different contexts, or just as adopting web standards/CSS-based layout helps to create a repeatable process that can be applied across any web site you wish to develop, using an XSLT-approach can apply to many different projects and in different frameworks.

Development Productivity; Help Inject Key Skills into Project Delivery

I have consulted at a few companies, and read/heard others describe similar scenarios, whereby skilled client side developers are limited to providing templates they throw over to the developers who are not often web developers.

This can result in either extra time explaining the required markup to the developers. Time may also be needed in working out alternatives (and unnecessary trade-offs) because of limitations in the framework of choice, or garbled markup coming out of the development process (ASP.NET, looking at you!).

This past year at work, we have been aggressively hiring for web development positions, looking for a mixture of strong web standards, and server side programming with .NET. We have found it incredibly hard to hire candidates who are exceptional in all these areas (many good web developers seem to avoid .NET for example, and many ASP.NET experts we find know little on web standards).

But we don’t always need everyone to have all those skills, either; the separation afforded by XSLT has allowed us to hire really strong people with parts of these and get them to work in parallel with others. Those well versed in web standards seem to find XSLT easier to adopt, in my experience, especially if full-blown, back-end programming is not their usual area.

Such developers are therefore not excluded from the development process.

Getting the key skills into the project in this way gives a number of benefits, including

• Greater productivity and adaptability: as project requirements change even during development, having more skilled, specialist staff than generalists in all these areas can pay off. (Generalists can often be extremely valuable too, not knocking that at all!)
• Parallel development. We have found it incredibly powerful when development phases are tight and mid-tier and client side development have to start somewhat in parallel. Some of our front-end guys who have not got skills in C# and ASP.NET are nonetheless able to fully contribute to the development process, and the project gains from their in-depth knowledge of the client side.

Encourage Front and Back-end Developers to Work More Closely

During day 1 of @media AJAX, interface designer Mike Stenhouse gave a talk from the perspective of a designer/front end developer having to know more about development, getting more involved in agile development, test-driven, even behaviour-driven development.

Designers understanding more software development processes, getting involved in it – not just throwing templates over the proverbial fence – would help communicate the importance of the “lowly” client side technologies.

At the same time, mid-tier developers get a chance to explain concerns such as architecture, OO, patterns, frameworks etc. This common understanding, even though they need not have to be experts on the other area, would allow them to be more productive.

Of course, Mike Stenhouse’s point isn’t unique to an XSLT-based approach (in fact, I think he was talking about Java or PHP). But one possible advantage an XSLT-based approach may have here is that it might be simpler to learn (controversial statement, maybe!) than some of the framework languages, such as Java, C#, PHP, etc. Where this is the case, XSLT can be a useful way to help bridge the gap.

Coding productivity

Using something like ASP.NET controls and simply dropping them onto a canvas seems really productive at first. I was certainly excited back in 2000/2001 as information about ASP.NET started to come out. I remember being really excited about the date control that ASP.NET had demoed years ago! Such a time saver – or so it seemed.

However, if you factor in the time you often need to sort out the mucky markup output, make things accessible, CSS-friendlier, and so on, the overall time, in my experience doesn’t seem that much better (in some cases worse, some cases better) than an XSLT-driven approach.

If you know the output you need to create, I find writing XSLT to be more liberating as you are in full control of your output destiny!

The switch I described earlier from ASP based XSLT-driven site to an PHP based XSLT-driven was also a switch from Windows/IIS to Linux/Apache (I was finally fed up of IIS’s limited configurability, in a hosted environment).

In just two weeks, a site with about 700 pages was converted, fully tested, and up and running again on a new hosting company. There was virtually no XSLT change, and the design pattern (a kind of Page Controller pattern which I might change slightly to MVC if time allows) was just reapplied to PHP. PHP 5’s object oriented capabilities meant further reduction in server side code. (Come to think of it, it wasn’t two weeks – it was a LOT less, as it was just spare time during some of the evenings.)

For work, most of our customers (one of them being amongst the largest in the world in their industry) use .NET. We often by-pass ASP.NET controls and use XSLT to generate the output instead. C#/.NET are still used on the server side, as .NET itself is quite powerful.

For the generation of the UI, XSLT has allowed us to even create multiple web sites simultaneously, as all the XSLTs can be reused. Where creating a separate sub-site in the past has take 6 months or more, our move to the XSLT-based approach has meant it has taken a fraction of that time (and included SEO, accessibility, web standards, etc.).

As another recent example: For my current project, a colleague (an excellent developer in general, though not well-versed in web-standards, yet) is creating a functionally rich and powerful page as part of the overall app we are developing. It has a complex grid of tabular data, with a folder/tree type browser on the side with lots of dynamic stuff possible on that page. Some controls created by a vendor for ASP.NET 2.0 are being used. While they are rich in functionality, the page produces just under 1MB of output!!!

The output from these controls are plagued with excess divs and unnecessary table-based layout (tables for tabular data of course is fine). Styling some of the specific features also becomes tricky (because the markup created is not very good and can’t be changed).

In addition, on a gut feel, the amount of C# used to prepare the data to bind it to the grid-based controls seems excessive. In the output, there is a lot of unnecessary (and bloated) JavaScript needed to get it to behave as needed, too.

This also (currently) runs like a dog on IE 6 because of all that JavaScript and HTML bloat! I have spent a total of 15 days just trying to get the resizer/splitter control that contains the table and tree to resize properly by battling its JavaScript (to be fair, during that time in the summer, I discovered the excellent jQuery, which helped reduce that effort considerably but I was still using jQuery to do unnecessary stuff)!

In my own opinion, it would have been much more efficient to have written the whole thing with XSLT and use high quality client-side components from people like jQuery, YUI, Dojo, etc. The trees and tabular data on that page are really well suited for XSLT processing. To be fair, the guy writing the page isn’t as well versed in XSLT/web development but an experienced GUI developer, so the control approach makes perfect sense for him and he has certainly made me appreciate that approach more than I used to. (I do still feel that the amount of code needed feels excessive and the experience so far has strengthened my view that while ASP.NET is powerful on the server side, I prefer to use XSLT to create the markup, where possible.)

Drawbacks (or perceived drawbacks) to the XSLT approach

XSLT is certainly not to everyone’s liking. The right tools may not be there in the favored IDE, or it may just look too weird for some people (it is more like functional programming than procedural or object oriented, so can sometimes take a while to get used to). Here are the disadvantages of XSLT I can think of, and where I can, some ways to overcome those problems.

Not everyone likes XSLT!

Many who are used to OO or procedural programming may find this functional programming language strange to get around. Angle bracket coding isn’t to everyone’s taste! And these personal preferences can be important for a team/project. Initially I didn’t find it too intuitive, but after a few days with it, it grew on me.

(And I certainly don’t hold the position that one must only use XSLT. Different people will prefer different technologies, and are of course free to use them.)

Yet another technology to learn

I kind of appreciate this concern, but also in some ways do not. For example, XQuery, or the more generic and powerful LINQ from Microsoft, get many developers excited, and they are willing to learn those if needed. People will invest time in learning CSS, JavaScript, etc.

So, perhaps it could be argued that just like any other technology, if XSLT is determined to be appropriate for the project at hand, then that can be learned too.

XSLT 1.0 is Verbose and Weakly Typed

Some simple things in XSLT 1.0 are a bit verbose to code (though XSLT 1.0 can often be written more cleanly than what many people write).

XSLT (and XPath) are indeed weak on typing — at least 1.0, though given the nature and purpose of XSLT this is not too limiting a factor as it would be if this were a full-blown language. (Some popular languages for the web are also weakly typed, too, e.g. PHP and JavaScript and the “limited” typing can also result in many capabilities, depending how they are used.)

XSLT 2.0 and XPath 2.0 are more powerful and overcome many of these limitations, adding many more powerful capabilities, allowing for leaner code and improved performance.

However, while XSLT 2.0 is an open standard, many have not supported it. (Microsoft, for example, has a decent implementation of XSLT 1.0 in .NET — and MSXML — but has hinted that XSLT 2.0 is not a priority for .NET at the moment — see also this discussion for more information.)

XSLT is a limited language

In a way, if compared to more general purpose languages such as Java or C#, XSLT (especially 1.0) is very limited in what you do.

That can be off-putting straight away.

However, (using extension objects etc aside), the simplicity of the language can also be considered a strength; it may help architect a better separation of data, presentation and logic. I personally have not felt too much pain from its limitation.

Poor Tool Support Compared to Other Languages?

Tool support for XSLT does not seem as good as ones for say C# or Java.

With C#, both Visual Studio in-built refactoring or JetBrain’s excellent Resharper makes refactoring a lot easier. Same with Eclipse and other such IDEs for Java and various other languages.

Need to rename a class that is used throughout your super-massive project? No problem, just right-click the definition and rename; the IDE will correctly update everything for you. Want to implement missing implementations of an interface? Right-click and get it to fill out the missing implementations. Want to move a block of code into a separate method? Highlight it, right click and select an option to extract a method. Want to find all the places where a particular method is used? Right click it and select find all instances. And so on.

In addition, many IDEs and plugins for code assistance goes beyond “intellisense”, advising on good coding practices, code formatting, etc. This is immensely powerful, especially when establishing team standards.

With XSLT, however, the level of sophistication is not as high, as far as I am aware. For example, finding instances of where a given XSLT template is used seems limited. The best editor I have come across for this purpose is oXygen (let me know if there are better ones!). It is has various refactoring and finding capabilities, but finds don’t support searches for XSLT template modes. (I find using moded templates is generally a cleaner way to code many XSLT.)

A developer is therefore reduced to global finds within their text editor of choice. To be honest, I don’t particularly find this a problem, but I know that some developers, especially when used to using tools like JetBrain’s excellent refactoring tools, will find this backward, and I can’t blame them!

Many IDE vendors, such as Microsoft, don’t have their priorities on XSLT. While somewhat understandable, it does contribute to the chicken-egg situation:

• IDE tools are not as good for XSLT;
• XSLT therefore not as widely adopted in that sphere of development;
• IDE tools remain limited for XSLT;
• XSLT development seen as backward from an IDE/tool support perspective so usually not looked at.

As to what IDEs/tools/editors are useful:

• A number of people at work like to use XML Spy for XSLT development, though I personally don’t like it’s text editor.
• I tend to use jEdit, which is a Java-based IDE, so I have used it on Linux, PC and Mac, quite nicely (it is hideous to look at initially, and I have spent a long time customising it! I will explain that in some future blog post!).
• Eclipse and the Aptana plugin looks really useful, and I am beginning to look at that more.
• Many people also swear by oXygen, so I am looking at that a bit more, too.
• But, for my current project, for the past year, at work I have been content with using Visual Studio 2005 (and now 2008) for XSLT. It has an XSLT debugger (and recently a profiler) but I have not needed to use the XSLT debugger yet, so can’t comment on it.

If you know of any other useful XSLT tools, please comment so!

Refactoring in a mixed language environment can be problematic

Suppose you have an XSLT that transforms some XML which comes from a Java or C# serialized object.

If you modify that object in a simple way (rename the class or a public member), in IDEs such as Visual Studio and Eclipse, you can very quickly update all usages of your class/method throughout your project and unit tests, and you are done.

But, imagine you are renaming class members that get serialized in such a way that the name of that class member is the name of the element/attribute in the XML representation. IDEs aren’t going to easily know that and you are likely to have to manually update the XSLTs.

Unit Tested XSLTs may help to some extent to find where changes are needed, but it may still be more work than just working in C# or Java.

Getting data as XML may not be practical

Using XSLT assumes you can easily (or without much cost) get your data in XML format, so you can transform it. This may not be possible in many cases.

In many instances, with in-built XML serialization capabilities, such costs are often minimized or minimal. (You are often doing well if this is your biggest problem in your overall system architecture!)

Even if serialization is not possible, XSLT performance can benefit from well structured XML. Is it easy to create that structure from your object model? Depends on the project at hand.

Perceived Performance Concerns

A common question I often get in discussions of XSLT is concerns of performance.

XSLT performance, amongst other things, is also affected by:

• The XML being transformed – its structure, size, etc.
• The efficiency of the XPath expressions (e.g. using ‘//’ or invoking dynamic scripting languages from within XSLT can be performance killers).
• XSLT itself (e.g. loading, interpreting and executing XSLT in general, whether or not the XSLT Processor is compiled/cached/reused or not, etc.)

As well as addressing the XML and XPath concerns, the XSLT performance concern can also be addressed in many cases.

Using cachable XSLT Processor implementations can dramatically boost performance. (Kind of like XSLT being compiled into the native code, like Java bytecode or .NET IL, etc. and then caching them in memory across sessions, for subsequent reuse). We use this approach on one of UK’s most profitable and biggest online retail web sites, for example.

Also, as a high level language, XSLT has potential to be optimized further at the implementation level resulting in better performance without changes needed to the XSLT itself.

(Extensibility through XSLT extension objects may offer some performance gains if you write extension objects in a compiled language such as Java or C#. A risk may be that you lose some portability — though how much that is a risk depends on your circumstances.)

It implies error-prone XML-based templates

Consider this concern from Django’s design philosophies behind its templating system:

XML should not be used for template languages

Using an XML engine to parse templates introduces a whole new world of human error in editing templates — and incurs an unacceptable level of overhead in template processing.

– Design philosophies, Django, last accessed January 4, 2008

While the above is not specifically geared towards XSLT, human error is certainly an issue with XML-based templates, or XSLT-authoring.

That error, if it comes at template creation time, is typically caught during the development/debug/test cycles, as malformed XML will error early.

Of course, if content comes from external sources (user generated, from third parties, etc) the well-formedness of that XML cannot be guaranteed. However, these are usually not likely to be within the template itself. It may or may not be possible to cleanse such data, depending on the project.

In addition, as addressed earlier, I do not believe the overhead of template processing is that bad. Cached XSLT processors, as well as other caching strategies can help, for example.

A colleague of mine found an ASP.NET data grid control taking some 2 minutes to bind and render a standard .NET dataset. He changed that into an XSLT transformation instead and the data transformed in around 20 seconds — clearly an exceptional case compared to most web needs (2 minutes or 20 seconds implies a LOT of data), but XSLT is not automatically slow.

In addition, depending on the application, the generation of the XML may come entirely through serialization of objects, which in languages such as Java and .NET (not sure about python, which Django is in) is reasonably fast.

In a current project I serialize a very large object (about 200K when saved as XML) and transform it into HTML. The HTML is a very complex form of over 300 form fields! (Fear not, we use a tab-like interface and various other techniques to group them!) Using the cached XSLT Processors on my developer box, the transformation takes just 5-10 milliseconds. The far bigger cost is getting the data from the database in the first place (about 50 to 120 milliseconds, but that is on a developer database on the same box as the rest of the stack). The serialization takes about 5 milliseconds. Even when the XSLT processor is not cached, the transformation times are very small (I think about 150-300 milliseconds depending on the data).

That same design philosophy’s very next point was that you should

Assume designer competence

The template system shouldn’t be designed so that templates necessarily are displayed nicely in WYSIWYG editors such as Dreamweaver. That is too severe of a limitation and wouldn’t allow the syntax to be as nice as it is. Django expects template authors are comfortable editing HTML directly.

– Design philosophies, Django, last accessed January 4, 2008

I would tend to agree with the bit about assuming designer competence, which is actually why I find XML for templates are not as bad as it may initially seem.

As I mentioned earlier, in my experience, when trying to hire front end developers, we have found they are quite capable at adopting XSLT.

It is not that Django’s is wrong in my opinion; I am sure there are many for whom XML-based templates are not appropriate. My point here is that there are also many for whom XSLT will be seen as appropriate, too.

(And this is not a criticism of Django, either. I have only looked at it very, very, very briefly and am already impressed with it!)

Coding Productivity is Perceived to be Poor; Hand writing all that code seems inefficient

A common concern I hear (usually from middle-tier developers who want to avoid writing HTML) is that it is inefficient to hand write all that code. I can certainly understand their concerns; it is something I shared when I used to do more middle-tier development than front-end.

However, I actually find the opposite can be true: After using it for many years, a few colleagues of mine and I find it is far more productive creating web sites this way, because we know what we need to output, and we don’t spend ages battling tools that generate HTML to make them do stuff they don’t easily want to. We also help minimize, or completely avoid the excessive bloat that many tools create.

In addition, past experiences suggest that many of our XSLT projects seem to have less code than would be needed in say a C#/ASP.NET template (like the tree/grid example mentioned earlier).

Unlikely to get vendor buy-in if they have vested interest in other ways

Vendor buy-in may be limited if there is an inherent interest in vendor lock-in.

This is a mixed one too. Take .NET for example. Microsoft is unlikely to do away with its control-based approach and cause more upheaval to its target developer audience by doing this. It also reduces the dependency on Microsoft!

I would think that if this XSLT idea has any large-scale merit, open source platforms/frameworks are more likely to adopt it, though that is not necessarily a given, either.

XSLT for web-based Views hasn’t taken off yet, so it must be a bad choice

I think this point can have merit. However, I think there may be a number of different reasons why XSLT has not taken off.

For example, it may not have been appropriate in the past because XSLT tools were not as mature. While the XSLT tools were maturing, other technologies came through.

In addition, not all data is always easy to represent as, or convert into, XML, and the tools built into languages these days such as serialization may not have been around sufficiently in the past.

But the chicken-egg scenario mentioned above about IDEs not supporting XSLT as much so XSLT not getting as much traction in this area, could also be a factor. And, as hinted above, some vendors may have interests in promoting their own ways of doing things as an open standard may make it easier for someone to move to another platform, if needed.

Of course, developers not liking the language may also be a factor. (It seems to be quite big in the Java world, from what I can tell, though I don’t have much hands-on experience with Java these days – if anyone has some more insights here, please let me know!)

We have used this XSLT approach very successfully on a number of projects, one of which is one of the most prominent online retail web sites in the UK, if not the world. We haven’t really had a chance to write about it. Maybe there are some (many?) others who have also not had a chance to write about similar approaches? The BBC, I understand from some colleagues/friends that have worked there in the past, use it but in a more off-line scenario to pre-generate certain pages/sub-sites. So it can’t be all bad!

In addition, valid (or invalid) reasons to reject it in the past do not always automatically apply in for now or the future, especially in a dynamically changing environment such as the Internet. Times and technology change, so maybe it is at least worth re-validating (not adopting blindly of course!).

In conclusion

Despite the concerns, or perceived concerns, I feel the potential advantages make it worth considering.

There are certainly XSLT-based frameworks out there, though not as popular. (Apache’s Cocoon project comes to mind, for example.)

I would think that just as advocates of web standards have pushed hard to get open standards adopted for client side web development, maybe there is a case for XSLT on the server side to aide in that process.

But it also depends on personalities, priorities, and specific requirements, too. Some people really don’t like XML-based things, which is fair enough as sometimes there are legitimate reasons for it.

Others may not be convinced by the need for controlling markup, thinking it is unimportant or that their favourite vendor will sort it all out for them!

I am not arguing that XSLT should be the only way of doing things. I think it can be useful in many situations, and I have certainly found it to be so. Following the ideas of using web standards on the client side, using XSLT on the server side (at least for the generation of output) appeals to me too. What I therefore hope to do in coming posts is demonstrate some possibilities.

Some examples

To explore this further, as this post is already quite long, in the next few days or weeks (whenever I get the time!) I will add posts to demonstrate

• Unit testable XSLT
• “Object-oriented” XSLT
• XSLT as the View part of MVC, or as part of a Page Controller pattern.

[You might, rightly, wonder why this is a WordPress blog, and not something I have written using say PHP with XSLT! Fair, but lack of spare time is the main problem: It has taken me two years just to get round to starting this blog! One day I will convert, time allowing!]

In the meanwhile, if you have got this far, thanks for bearing with me!

Do you have similar concerns, or agree/disagree?

Image credits

“XSL Transformers” reposted with kind permission from NaGraDim from flickr. See the original image at http://www.flickr.com/photos/nagradim/750087828/.

PHP recently announced PHP 5.3 which will include an XSLT profiler that can be invoked from within code. Current versions of PHP and the Microsoft one can invoke an XSLT profiler through the command line or against static XML/XSL files only, so being able to call it from within code is quite useful.

The run time invocation is really useful because if you are passing parameters into the XSLT or are generating the XML through DOM programmatically it is easier to profile. Otherwise, you need to capture the XML generated and save it, then invoke a profiler separately from the command line.

An example of calling it from within PHP is this (taken from a SitePoint article explaining the new features — see previous link):

$doc = new DOMDocument();
$xsl = new XSLTProcessor();

$doc->load('./lib/collection.xsl');
$xsl->importStyleSheet($doc);

$doc->load('./lib/collection.xml');
$xsl->setProfiling("/tmp/xslt-profiling.txt");
echo $xsl->transformToXML($doc);

echo '<h2>Profile report</h2>';
echo '<pre>' . file_get_contents( '/tmp/xslt-profiling.txt' ) . '</pre>';

Are there other profilers out there you recommend?

The profilers in oXygen and XML Spy look quite useful too (not used them, though heard about them).

I have not used XSLT profilers extensively. Just once in a blue moon. Some of the XSLT performance tips I have written earlier seems to have served me well, thus far, but I think more regular use of these profilers will be important.

Have you used other profilers for XSLT? What are they like?

Have you used other profilers that you can invoke from your code, rather than from an IDE or command line?

Do you find that to be useful or in the end is it just as easy to capture the XML and run the profiler in the IDE (which usually has a more powerful GUI to splice and dice the profile data)?

What about XSLT code coverage?

Is anyone aware of any good XSLT code coverage tools?

The reason this is important is that as well as getting good performance, it is important to know if you are exercising all your XSLT code or not.

This would be particularly useful when integrated with unit-tested XSLTs (which I will be writing about shortly!).

Image credits

“Stopwatch” by jamieriddell from flickr. See the original image at http://www.flickr.com/photos/jamieriddell/2183060366/.

PHP 5 has the usual things, such as classes, interfaces, abstract classes, inheritance, etc, but some useful programming constructs have been missing, though PHP 6, under development, aims to rectify that.

However, it seems that many of those features are going to be brought forward to the up-coming PHP 5.3 (which may make it more likely that it will get installed by web hosting companies sooner than they would likely go for PHP 6).

Sitepoint has an excellent summary of the features. The list of features include:

• Namespaces
• The use keyword to use namespaces!
• Namespace aliases
• Class constants
• Namespaced functions
• The Global namespace
• Autoloading namespaced classes
• Late static binding
• Variable static calls
• MySQL native driver
• Additional OpenSSL functions
• Improved command line parameter support
• XSLT profiling
• New Error Levels
• and more…

But visit the sitepoint article for more details.

If you want, there are PHP snapshot builds of PHP 5.3 (and 6.0).

(Those of you that have never coded in PHP may be surprised to see some of these features only surface now, yet, PHP has still managed to drive some of the biggest web sites out there…! I have only used it for about a year myself, in spare time, but it seems quite powerful even without these things. Can’t knock it too much then, I guess )

The native MySQL driver will be a good boost for performance, and I will be looking forward to the XSLT profiling in code, while namespaces will be an obvious winner too.

But a lot of PHP programmers are often using it for small scripts and may not often need or want the full power of an OO language. Personally, I prefer OO, so am looking forward to this. Do you use PHP? If so, do you prefer the OO approach and like what is coming?

The challenge they had was to find a way to “enable (and encourage) interoperable web development, but don’t force IE to break pages that work properly in IE today.”

They eventually settled for a <meta>-based “opt-in to the browser version I tested with” strategy.

What this means is that if you as a web developer want IE 8 to render according to their best implementation of standards then you opt in by adding a particular meta element into your HTML (or send down a similar HTTP header in the response). This is explained in an article in A List Apart.

In other words, for web developers trying to do the right thing they must pay a small “don’t-break-IE-web tax!”

Many prominent web developers and designers have been highly critical of this. But, ironically, will this actually be a positive thing in the long run?

Don’t break the Best-viewed-in-IE web

The IE team described their attempt as a “don’t break the web” approach. Judging by many reactions, it feels they are actually breaking the principles of web development.

And, in their own words they don’t want to “force IE to break pages that work properly in IE today.” In effect they are asking web standards developers to help them out so they don’t have to go and fix all their problems resulting from the IE-driven web which has slowed down web development and contributed to this current problem. (That is probably a bit harsh, as I can certainly understand Microsoft would be unable to fix all the issues in the various places where IE is used! More on that below.)

The lead of the IE team, Chris Wilson, seems to admit this meta switch is a hack but given a difficult situation for Microsoft, trying to support both standards and non-standards developers, this might be the best way to go about it.

So other browser vendors were encouraged to adopt this idea too. However, the guys working on WebKit (the rendering engine behind Safari) felt that,

We don’t see a great need to implement version targeting in Safari. We think maintaining multiple versions of the engine would have many downsides for us and little upside. The IE team is, of course, under different constraints and free to make their own choices.

– Maciej Stachowiak, Versioning, Compatibility and Standards, January 22nd, 2008

John Resig (author of the excellent jQuery JavaScript framework and an employee of Mozilla), also takes issue with

having this be portrayed as a general-purpose solution for all browsers, when it’s really just a solution to Internet Explorer’s situation. That’s not necessarily a bad thing. I completely commiserate with [IE's] situation and don’t envy it at all. It may very well be that this new meta tag addition is the only viable solution - and that’s fine. However showing it as something that is a universal issue to all browsers is definitely not the case.

– John Resig, Meta Madness, January 22nd, 2008

Resig also made an important point about how this decision was arrived at:

On a whole, I really wish that this process had been done more in the open than it was. I doubt that as many issues, that we’re seeing now, would’ve arrived. I absolutely don’t envy Internet Explorer’s situation in the matter so, for the sake of the web, let’s hope they come up with an acceptable solution.

– John Resig, Meta Madness, January 22nd, 2008

Ian Hixie, lead member of the WHATWG (the guys that started defining HTML5), and Google employee, was concerned that browsers would have to implement it, like it or not:

If Web authors actually use this feature, and if IE doesn’t keep losing market share, then eventually this will cause serious problems for IE’s competitors — instead of just having to contend with reverse-engineering IE’s quirks mode and making the specs compatible with IE’s standards mode, the other browser vendors are going to have to reverse engineer every major IE browser version, and end up implementing these same bug modes themselves. It might actually be quite an effective way of dramatically increasing the costs of entering or competing in the browser market. (This is what we call “anti-competitive”, or “evil”.)

It will also increase the complexity of authoring by an order of magnitude. Big sites will become locked in to particular IE version numbers, unable to upgrade their content for fear of it breaking. Imagine in 18 years — only twice the current lifetime of the Web! — designers will not have to learn just HTML, they’ll have to learn 4, 5, maybe 10 different versions of HTML, DOM, CSS, and JS, just to be able to maintain the various different pages that people have written, as they move from job to job.

– Ian Hixie, Mistakes, Sadness, Regret, January 23, 2008

Chris Heilmann, a lead web developer at Yahoo suggests that maybe now is the time to try and mend the broken web; a large number of sites are now created by CMSs and now might be the time to explain, help, and contribute to fixes. “As most are built with frameworks and CMS and one of the main selling points of these was that the outcome can be easily changed in the future, how about proving that now?” A garganutan task it would seem!

(Heilmann also posted a link to this hilarious explanation of how this meta-switch problem has come about.)

Microsoft is in a difficult situation

Resig noted above Microsoft’s situation is not something to take lightly.

The following is perhaps one of the best articles I have read trying to explain Microsoft’s predicament. It is not by a Microsoft employee, but a lead web developer at Yahoo! (and way before the Microsoft buy-out bid!!). Here is a lengthy quote:

[Don't just consider webservers,] but … other forms of distributing HTML content. How easy is it going to be to configure a webserver, or add a meta tag to HTML documentation on CD? It’s not just a case of editing the HTML and reburning to the CD - what about making sure everyone who has the old CD now has the new one?

Compiled HTML is another source of grief - every chm ebook ever published relies on an IE renderer library to display its content. That content is authored and then distributed to many people. Again, the HTML editing is simple, but the actual redistribution of the non-broken version is a massive problem.

Consider also the number of Windows applications that use the Internet Explorer rendering engine as a component of the application. All the content and logic for those applications would need to be updated, and the application would need to be re-shipped to all the users of that application.

The point here is that web’s solution is remarkably simple, it’s a case of fixing the problem in one place.

…

Sure, redistribution [for the non-web server-based products that use IE] is just a link to a website to download an update. But every software product, every ebook title, every software documentation would need that redistribution. It’s not a couple of web pages any more, it’s the entire Windows platform, the operating system and all applications (including third party software) that would need updating.

…

This points to a grievous mistake on Microsoft’s part: integrating the browser (well, the rendering engine) as part of the operating system.

A change in the rendering affects not only web pages, but local documentation, software applications, even so far as the file-system browser Windows Exploring. They all use the Internet Explorer renderer. The moment that renderer fails, or decides not, to render the content it used to render - that is a problem Microsoft want to avoid.

– Mike Davies, Internet Explorer Reality, January 27, 2008

It may have indeed been a mistake of integrating the browser rendering engine, but I wonder if more fundamentally, Microsoft letting IE6 stagnate and not developing it into IE 7 until just a little while ago was a root problem. Maybe continuous, smaller and incremental updates could have let the pain be less than it is now? Hard to tell.

Legal concerns if Microsoft breaks IE?

It seems that occasionally legal concerns about breaking compatibility with new IE releases comes up as another reason for not being able to move as quickly on standards implementation as ideal. However, Mike Shaver (from Mozilla) questions the legal concerns, asking, “What would be the legal basis for a suit by a customer, given the provisions of typical EULAs which explicitly disclaim pretty much all warranty they can?”

Comments on Shaver’s post also notes that changes to their operating systems (XP to Vista for example) and application compatibility problems seem not to raise such solutions, by comparison, despite concerns raised by even the likes of the Korean government and their banks who had to tell their population to delay upgrading to IE7 (because they had developed to IE6).

Shaver is hinting that legal issues may not be a hindrance as much as made out to be (though other practical concerns might still be).

IE8 passing the Acid2 CSS test won’t matter for most sites

A little while back, Microsoft announced that some development version of the IE8 code was passing the Acid 2 CSS test.

This led to a lot of praise in web development circles that maybe finally Microsoft was moving forward. But there were those expressing cautious optimism, too.

And the latter are justified; IE8 Acid 2 rendering won’t matter for the vast majority of sites that are likely not to even know about this new meta switch.

Maybe it won’t be too bad in the long run?

Many have opined that maybe IE is getting less significant now and whatever it does won’t really impact the web that much; it seems to follow these days, rather than lead.

It may be that IE 7 is just about bearable enough that developers can get on with cool things on other browsers while giving half decent support to IE7 (and with the increasingly sophisticated JavaScript frameworks hiding away those IE deficiencies and limitations in most cases, this may make it even easier to bear with!).

On the other hand, Matt Wilcox, a developer who is also on the HTML 5 working group, had a really interesting perspective:

I think the new default of “stay as IE7 when no meta tag found” will enhance the uptake of web standards. Because every ignorant developer on earth is now going to be permanently stuck with IE7 rendering and behaviour. They will never get to use any of the new-shiny [features] of IE8 and above if they don’t know to include that meta tag. They won’t know to include that meta tag until they go away and learn about web standards.

– Matt Wilcox, Goodbye DOCTYPE switching, hello META targeting, January 22, 2008

Maybe this particular tax will actually trickle down? Or will it really bad for standards-based web developers?

Maybe none of this actually matters?

John Resig, in another post on the HTML 5 DocType found out that IE will render unknown doc types (such as the HTML 5 DocType) in standards mode now.

The HTML 5 DocType is simply this:

<!DOCTYPE html>

This means authors could start using the HTML5 doc type now for their HTML 4 documents.

The W3C Validator would fail though (I think), which may be problematic for some.

So would it be better to pay that little tax, or go for this DocType now?

ASP.NET has been problematic for many web developers/designers trying to do the “right thing”, such as

• Use web standards
• Output semantically meaningful markup
• Develop to standards, rather than browsers for cross-browser development
• Use progressive enhancement concepts to cater for as many user preferences and technology limitations as possible
• Create sites that are as accessible as possible without unnecessary trade-offs with the visual design
• Create sites that are friendly for SEO
• Create lean and optimal markup for quicker download
• etc.

(ASP.NET 1 was far worse, and version 2 allows many of the above to happen, but not all, and not always easily.)

Describing the problems caused by the poor quality and bloated markup and the weird/wonderful/complex/ugly/hacky ways to work around this seems to take ages. Some fans of ASP.NET don’t like to hear it when going into these details, and react defensively. And, to be fair, these problems are not unique to ASP.NET, either.

But recently, I rediscovered a summary explanation to help explain why ASP.NET is often problematic from a web developer’s perspective:

ASP.NET is a leaky abstraction:

That term, “leaky abstraction”, was coined some time ago (end of 2002) by Joel Spolsky, a prominent writer on software development and he was noting the problems with ASP.NET way back then.

In teaching someone about ASP.NET programming, it would be nice if I could just teach them that they can double-click on things and then write code that runs on the server when the user clicks on those things. Indeed ASP.NET abstracts away the difference between writing the HTML code to handle clicking on a hyperlink (<a>) and the code to handle clicking on a button. Problem: the ASP.NET designers needed to hide the fact that in HTML, there’s no way to submit a form from a hyperlink. They do this by generating a few lines of JavaScript and attaching an onclick handler to the hyperlink. The abstraction leaks, though. If the end-user has JavaScript disabled, the ASP.NET application doesn’t work correctly, and if the programmer doesn’t understand what ASP.NET was abstracting away, they simply won’t have any clue what is wrong.

– Joel Spolsky, The Law of Leaky Abstractions, November 11, 2002

I think this is a decent summary description that can be useful for constructive discussion on what to do about it (if anything).

Abstraction without understanding = problems

Of course, this is not just an ASP.NET issue, but a general problem in software development, as Spolsky added:

The law of leaky abstractions means that whenever somebody comes up with a wizzy new code-generation tool that is supposed to make us all ever-so-efficient, you hear a lot of people saying “learn how to do it manually first, then use the wizzy tool to save time.” Code generation tools which pretend to abstract out something, like all abstractions, leak, and the only way to deal with the leaks competently is to learn about how the abstractions work and what they are abstracting. So the abstractions save us time working, but they don’t save us time learning.

And all this means that paradoxically, even as we have higher and higher level programming tools with better and better abstractions, becoming a proficient programmer is getting harder and harder.

– Joel Spolsky, The Law of Leaky Abstractions, November 11, 2002

This helps explain:

Why non-web developers or IE-only web developers like ASP.NET

It is an abstraction, so you don’t worry about it

Why web developers/designers despise ASP.NET

The abstraction leaks, horribly at times, and you want to worry about what is being abstracted

What is still interesting is that as a software developer (not necessarily a web specialist), Joel has understood some of the problems web developers and designers will have with .NET even though many Microsoft converts and ASP.NET fans do not!

Many software developers look down on web developers, thinking things like HTML and CSS are beneath them (its “just” the UI, or that JavaScript is not a real programming language, etc).

While those attitudes are generally changing, many software/mid-tier/non-web developers I come across still hold some of these views and don’t want to write anything in these languages, preferring the safety and familiarity of their IDE! (To be fair, some of these concerns are quite valid, too, and the principle for supporting the abstraction is actually often good; it is just that with web technology at the level it is at now, some of that abstract necessarily has to leak…)

Negative perception of IE and ASP.NET for Web Developers

It is interesting to note at various conferences like @media, @media AJAX, and on various blogs by noted web developers and designers, Microsoft gets a lot of stick for its IE web browser (as it is recognized as the poorest in quality of all the modern browsers, with the most bugs, biggest divergence from web standards and causing the most headaches for modern web development).

Indeed, since winning the “browser war” IE has stagnated and because of its dominance, web technologies have hardly changed in the last 10 years (we are still on HTML 4.01/XHTML 1.0, CSS 2.1 etc). As I posted earlier, IE has slowed down web development progress in recent years.

Combine that with the poor quality markup typically generated by ASP.NET, there has been a lot of criticism from the web development community aimed at Microsoft for either not caring about standards (or, as with ASP.NET 2 and IE 8) caring about it as an after thought or when threatened by rivals.

Here is quite a telling quote summarizing some of the implications:

I sympathize with your situation, but you desperately need to get your seniors and peers to understand what’s happening, and to assign your team way more resources. IE is a first-impression MSFT product for most web developers, and yet you all continue to underestimate the dramatic spillover effect this poor developer experience has had and will continue to have on your other products and services.

Let me drive this point home. I am a front-end programmer and a co-founder of a start-up. I can tell you categorically that my team:

- Won’t download and play with Silverlight
- Won’t build a Live widget
- Won’t consider any MSFT search or ad products in the future

And the reason is because of IE - because MSFT disregards its most important relationship with us. Until this relationship is repaired, nothing else stands a chance.

Please fight the good fight and drive this point home in your org. Good luck.

– Anonymous commenter to The First Year of IE7, November 30, 2007

It may be seem a bit harsh, as some developers and technical managers at Microsoft certainly seem to “get it”, but for whatever reason it has happened and left a sour taste in the mouths of many web designers and developers.

Leaky abstractions lead to productivity loss and unnecessary trade-offs

As I blogged earlier, the WYSIWYG paradigm that Visual Studio and other such IDEs encourage, is often inappropriate for web development as it encourages bad practice such as non-semantic markup.

The perceived productivity gains of such editors vanish when needing to deal with typical web related concerns (semantic markup, accessibility, progressive enhancement, SEO, etc). In fact, what often occurs is unnecessary trade-offs. E.g.

• A back-end developer uses an ASP.NET control that offers quick development with sophisticated capabilities just by dragging it onto a canvas and setting a few properties.
• A web developer then claims that control is poor from all the web development perspectives and lists a number of things to change.
• The back-end developer resists (especially if it comes from a “lowly” web developer!).
• Trade-offs are then made (or not).
• Reworking in full (if embraced at all) takes a long time.

Even though ASP.NET 2.0 is far better than 1.x, the abstractions still leak; it is too easy to create sites that either

• Rely on JavaScript
• Violate, or easily encourage ignoring, basic accessibility
• Encourage further leaky abstractions because of the ease at which various ASP.NET controls can be created (the idea is good, but the fundamental leakage means that it will carry through to various controls).

So what can people do?

As an overall thought: by its nature, abstracting away HTML is almost guaranteed to result in a leaky abstraction.

Instead, it may be better to embrace the reasons for the leak, and provide cleaner separation while empowering people to work with the markup, given it is so crucial from so many perspective.

So here are some examples of a few ways to deal with the challenge, all with their merits for different purposes. Your mileage with each option will vary according to need. In addition, it is not a complete list; there may be other ways to achieve what you need:

Strict discipline with the controls you choose to use

The controls idea is certainly nice. ASP.NET and others (e.g. Java Server Faces) have put a lot of resources into this approach, and is certainly appealing from various perspectives (packaging components, reusability, productising components, etc).

Despite the problems, trying to stick with the principle of controls based abstraction may still be worth trying to pursue.

If so, I think it would require being VERY careful about third party controls you use, or purchase.

The other problem I see with this is in the case of medium/large teams; this strict discipline is unlikely to hold for too long, and eventually a developer with less experience on web development (or no inclination to do the web bit) will insist on using some poor quality controls that offer perceived productivity gains (”they can fix the problems you highlight to them!”, or “it looks okay”, or “people with accessibility needs don’t use the web, so it doesn’t matter!”).

Your mileage on this option will vary, depending on your team and project, etc.

Inherit and override problematic controls

Don’t like the way asp:label outputs its HTML? Override its Render() method. Same with any other control.

Sounds nice and easy enough, but this option is likely to run into trouble when someone invariably will want to use or integrate a composite, possibly third party, control written elsewhere that cannot be overridden (easily). So ASP.NET adaptors may be a better alternative than this one.

ASP.NET Control Adaptors

Also known as CSS Adaptors (which reveals a fundamental misunderstanding; developers want semantic markup for all sorts of reasons, not just creating CSS-based layouts).

This approach tries to give developers the opportunity to clean up the HTML output by overriding the default implementation.

The kind of problems I see with this approach:

• Code examples I have seen that look awful — lots of string concatenations, obscure configuration settings, etc.
• It may not always be easy to deploy a quick change after initial development (need to recompile a .NET Assembly, for example).

Despite the potential problems, it may still be a useful option if there aren’t any other alternatives.

(Does this approach help prove the abstraction fails?)

Do nothing (wait for Microsoft to fix it)

It may seem odd to entertain an option of “do nothing”, but I often think this is an option that should be considered in many situations, if not just for completeness but because pondering on this may let you think out of the box, so to speak.

Why should this option be entertained for ASP.NET? Well, you could take the approach of:

• It’s not my problem; I just want to create cool apps
• Microsoft will always fix it; we can just wait
• If we get sued (accessibility compliance) or our bosses/managers complain, we can just blame Microsoft
• The abstraction might be leaky now, but that doesn’t mean it always will — look at the improvement of ASP.NET 2 vs ASP.NET 1.x
• I’m so fed up I have lost the will to live and do anything else other than just be assimilated
• Problems? What problems? (Already assimilated a long time ago )

And so on.

Needless to say, I would disagree with many of these perspectives. For example,

• I feel as a web developer, it is out responsibility to use the tools available to create sites that are as accessible to as wide an audience as possible.
• Claiming ignorance, or actively avoiding issues you know are there somehow does not feel right.
• Microsoft may certainly fix these issues over time. Reading blogs and interviews of key people in ASP.NET and Internet Explorer teams certainly gives some hope for the future, but it depends on whether you think you can wait or not (and whether you have enough information on when they will release specific fixes that are important to you).
• The controls paradigm might always be leaky regardless of what is done. I think it would still be important to know the output that is being created in order to fully understand the implications of what you are creating. If there are problems with the site, you are likely to have to sort it out, not Microsoft (or any other vendor/framework you go for).

For many web developers who know exactly what they need, having a tool do it for you, and to a lower quality, will be infuriating. Knowing it can be done better (not just for one’s own ego, but because it is good for the user/customer) can be compelling enough to want to do something about it.

Indeed, a principle behind a number of open source frameworks has been to try and make the web better, and keep it open because it is felt that open standards and related principles will lead to greater choice, diversity and vibrancy of the web. (Indeed, a lot of “Web 2.0″ and the social web has come about because of open source technologies and ideas.)

Use an open source alternative

Those loyal to Microsoft will not like to hear this, but this is a decent option in many scenarios.

As Simon Willison, co-creator of the popular Django web framework said a few years ago, one of the key benefits of the open source approach is you are in charge of your own destiny.

(I’m not going to debate the open vs closed source paradigms here; that is not the purpose of this blog. I will just say that I didn’t use to buy the arguments of open source in the past, but in the last 3 or 4 years I have come to respect it a LOT more — I have not regretted moving to Linux at home recently, one bit, for example, and looking at the types of projects flourishing is quite encouraging.)

Don’t use ASP.NET, but don’t throw the baby (.NET) out with the bath water (ASP.NET)

Okay, so for some scenarios, completely changing platforms may not be an option.

In addition, I have to admit, programming in .NET (C# 2, etc) is nice.

Although Microsoft may be losing credibility with web developers because of the poor quality Internet Explorer browser and because of the problems of ASP.NET from a web designer’s perspective, .NET more generally is actually quite good.

Unlike on the Internet Explorer side of things, the MSDN blogs seem to be more active regarding on-going developments on .NET. Scott Guthrie, for example, has an immensely useful and popular blog on all things about Visual Studio, ASP.NET etc. (Scott Guthrie is General Manager of development teams that build the .NET CLR and the core .NET Base Class Libraries, ASP.NET, Silverlight, IIS 7.0, Visual Studio Tools for ASP.NET, WPF and Silverlight.)

For some projects we have successfully used .NET but by-passed the ASP.NET controls, using XSLT instead.

XSLT — an open standard

Using XSLT to generate the output can be one way to overcome limitations of proprietary frameworks where you lack control of the output (ASP.NET) or where other frameworks may be open source but you have to learn their templating conventions.

XSLT can also make the markup generation simpler to unit test as part of a developer’s daily work, which is not always the case (or not necessarily as simple) in other templating systems, because you don’t have to be dependent on a running web server and all the set up that would come with it (thus making unit tests quick to run as well, increasing the likelihood of them being used and maintained).

Being platform agnostic, XSLT can be an effective part of a View in an MVC framework and be applied to different frameworks and platforms, reducing up-skilling costs in the long term.

If you consult or contract for a number of customers/clients, it would be nice if there was one common skill that is portable across all these frameworks (just like HTML/CSS/JavaScript typically is on the client side). XSLT could be such a thing, potentially, all the while giving you full control of the markup that you need to be create.

In my experience, and those of some of my colleagues at work, using XSLT to by-pass .NET controls has actually been more productive from an overall development perspective because we save time that would otherwise be needed battling the controls trying to force or work alternative markup options, building accessibility, etc. (And we have architected/developed sites for one of the largest online retailers in the world, so the strategies are not merely wishful thinking, but reasonably proven).

XSLT may not always be appropriate in all cases, and many are certainly put off by its oddities or perceived limitations. But, perhaps XSLT deserves another look?

(Indeed, a follow-up post does just that, starting by looking at XSLT in server side web frameworks.)

Do you find these problems with ASP.NET for your sites? How do you overcome them, or do you just stick with it?

Image credits

“Mona Unmasked” reposted with kind permission from gravityx9 from flickr. See the original image at http://www.flickr.com/photos/gx9/354387041/.