Summary

This post follows on from previous posts about the way markup is sometimes created.

In a previous post, I worried about some web frameworks that abstract away the creation of markup, thus preventing (or making it more difficult for) web developers to create what they need — the leaky abstraction.

This post explores one way to overcome this, using XSLT.

As this is a rather long post, here are the key points:

• Creating good quality markup is important for a variety of reasons (quick loading pages, cross browser compatibility, accessibility, etc).
• Some server side frameworks may either generate poor quality markup or use templating systems that are specific to that framework.
• Using XSLT to generate the output can be one way to overcome these limitations, while giving you full control of the markup that you need to be create.
• Markup generation via XSLT may be simpler to unit test as part of a developer’s daily work, which may not be the case (or not necessarily as simple) in other templating systems.
• Being platform agnostic, XSLT can be an effective part of a View in an MVC framework and be applied to different frameworks and platforms, reducing upskilling costs in the long term.
• XSLT may not always be appropriate in all cases, and many are certainly put off by its oddities or perceived limitations, but perhaps it deserves another look?

(Some forth-coming posts will provide some code-examples, in particular of unit-testing XSLT and how it might fit into MVC as well as how it can be written in a reusable manner.)

Poor quality markup from the server side.

A previous post on WYSIWYG editors described how the output quality from editors that hide content markup generation can result in markup that is of poor quality, hard to customize/correct/change, affect accessibility (which is a legal requirement in some countries), result in excess bloat, etc.

Some server-side techniques, tools, and frameworks can contribute to this poor markup. This can happen for number of reasons. For example,

• Popular IDEs may encourage the use of WYSIWYG editors to drop in and drag around HTML controls (or groups of them). As argued in that previous post, WYSIWYG as a paradigm for HTML may actually be inappropriate, (unless you treat WYSIWYG as WYSIWYG-when-css-is-not-enabled).
• Popular frameworks may abstract away the creation of HTML, to do it for you. ASP.NET and GWT are such examples. But, as argued in a previous post, such abstractions are leaky.
• Some popular scripting environments have also encouraged spaghetti code (mixing markup, presentation, and business logic, for example).

And so it seems there is a gap between web standards and server-side output generation that is web standards-friendly.

Web application frameworks help immensely

Various frameworks have been commonplace for a long time now. These help separate out the presentation, business/mid-tier and data access logic.

Most frameworks are not solely about markup generation (if that is their priority at all). Frameworks have been important to address a number of important issues from data access, scalability, performance, transaction management, object-relational mappings, code maintenance, and so on.

These are certainly of value and usually worth using as they have teams of people devoted to them. At their best, they empower the framework user (developers and designers, for example). Yet, on the other hand, some frameworks may take some things too far, thus disempowering framework users.

Some frameworks still produce poor quality markup

Those of you who have used ASP.NET 1.x will know what a mess its controls create in terms of markup output. (ASP.NET 2 is far better, but can still encourage poor markup or make full control trickier than it needs to be.)

While server side programming using the .NET framework, and C# in particular, are extremely powerful, when it comes to creating web sites, that power is used to typically generate poor quality markup (even though it may be maintainable!).

Some of this stems from the fact that although it tries to hide away the markup creation, ASP.NET is a leaky abstraction. (This isn’t just a problem with ASP.NET, but potentially anything that tries to make markup creation a hidden, dark art!)

Abstraction: a good idea but leaky

Abstraction and loose coupling are certainly important, especially between layers such as presentation, business/middle tier logic and data access. It is a common approach to writing software.

Understandably, developers often like the idea of abstracting away markup/output creation, too. In ASP.NET for example, you could create a simple site without almost writing any HTML. Instead, you instantiate/declare controls such as asp:Label, asp:Panel etc, and you trust .NET to create the markup for you(!).

Interesting attempts are even available so you don’t have to write JavaScript and you get the benefit of tool support in IDEs for .NET and Java. For example, Script# enables developers to write code in C# (or any .NET language) and then compile it into JavaScript. Google has also been doing such a thing for a long time with its Google Web Toolkit (GWT) for Java. Such ideas are certainly very interesting and appealing.

While appealing perhaps (and maybe more to back end developers), a web developer also loses the ability to control how the output is created.

Gaining control of how you create the output

Why does it matter?

Being able to control how you create the output ultimately gives you greatest flexibility, especially when you need to create a flexible, rich, yet lean web-based UI.

Using the “web standards” approach (i.e. following W3C recommendations) to achieve these things most efficiently has been well thrashed out for years.

And, just as developers take pride in the quality and readability of their server side code, web developers increasingly take pride in the sites and web apps they create from the code perspective, too.

People from anywhere can easily see your markup; less will see your server side code (though that is not an excuse to write poor server side code!).

Although this is by no means the only factor, when hiring we will often look at sites that people have mentioned in their CVs/resumes (or ask for examples to be sure to see what is theirs) and try to understand how they have coded bits.

Some frameworks give full control but may use specific templating languages

Templating languages that accompany many popular web development frameworks may be open, but use mechanisms specific to that framework.

But there are clearly advantages to this approach, too (else they wouldn’t be so common).

For example, templates based on HTML, combined with the language of that framework (e.g. PHP, or Python, or Ruby, or C#, etc.) means there is some familiarity for developers to get on with the task at hand.

Consultants and contractors moving between frameworks

If you work as a consultant or contractor for numerous clients, you may find yourself moving from project to project and then having to learn different templating languages, some with more limitations than others.

In some cases it may be that you end up limiting yourself to just advising on markup, or only providing XHTML/CSS/JavaScript template files, which other coders then have to reproduce.

I have talked to many who have been frustrated at this process, especially with .NET, it seems, as their nicely validating, clean, accessible markup becomes something completely unrecognisable, when ASP.NET gets its way! Unnecessary trade-offs then result.

At work, we have recently been hiring both permanent and contract web developers mostly for work with .NET platforms. We have found it hard to find people with exceptional skills at both the front end and .NET back-end. We tend to find either good front end skills but a different back end (PHP, Java, etc.) or good ASP.NET but little or no real front end knowledge.

In some instances, this is not a problem, but other times it has been preferable to have .NET and good client side skills.

XSLT as the templating mechanism can perhaps help achieve the best of both worlds:

XSLT — an open standard

XSLT could help achieve both abstraction and full control of the markup.

XSLT is a templating language for transforming XML documents into another format (another XML document or some other text-based format).

For web development this can be useful to output exactly what you want and thus gain full control of the markup output generation.

You can re-use XSLT, and even create a hierarchy of XSLTs, that follow a Method Template design pattern, similar to a class hierarchy in Object Oriented Programming, whereby the base XSLT is equivalent to an abstract base class, and other XSLTs that import the base one can override various templates. (A future post will demonstrate this.)

It does of course assume you can easily (or without much cost) get your data in XML format, so you can transform it.

XSLT in MVC

Web developers are familiar with separation of concerns (semantic HTML markup for good document structure, CSS for style/layout, JavaScript for behaviour) and progressive enhancement (or graceful degradation).

Back-end/mid-tier software developers are also familiar with separation of concerns (n-tier architectures, single responsibility per class, MVC, etc.)

The Model-View-Controller (MVC) pattern for developing user interfaces has been around for a while. It is a decent approach to separating concerns (the model to represent the data, the view to represent the presentation, and the controller to bind them together).

It can also be seen at a high level (a typical n-tier architecture), as well as being used within the presentation tier (the model may interact with the middle tier, the controller will be part of the presentation tier passing the data (model) to the view code to display it).

It is a well-proven pattern in graphical/desktop application development and being successfully applied by many web-based frameworks today.

MVC can also help improve testability. For example, unit testing can sometimes be easier by testing specific views, models and controllers in separation.

Microsoft has recently announced an MVC model for ASP.NET development. I understand one of the many drivers is the unit testability.

From an initial read, this sounds quite good. It encourages better separation. Interestingly it also seems not to support their post-back model which most of their controls are built from. It is therefore not clear to me at the moment if this means you write out more HTML (meaning you get more control of what you output) or if you still use packaged controls (which may risk getting HTML that is not appropriate, or hard to control).

But Microsoft is a bit behind on MVC for web development (though that can also mean it can learn from the earlier frameworks). Other popular frameworks have existed for a while that do this for the web, such as Ruby on Rails, Django, CakePHP, and many more etc. (I am quite impressed with the Python-based Django, for example, even though I have only had a cursory exposure to it — and I usually prefer curly brace type programming languages!)

While these all have their own merits, they all have different ways of constructing the markup.

XSLT, then, could be part of the View part of an MVC framework. (It need not be the only way to create the view, of course.) For example,

• A Controller can invoke a Model to get data the way it needs to
• The Controller can pass the Model to the View
• The View can internally serialize that data into XML and transform it with XSLT to the output.

In this way, the markup generation is still nicely abstracted away, but inside that abstraction itself, you have full control of what you need to output.

Advantages/Benefits of an XSLT-based approach

Here are a few more detailed reasons XSLT can be useful (though, of course, some of these reasons are by no means unique to an XSLT-based approach).

XSLT is an Open Standard

People often talk about the benefits of using open standards on the web client side of things (”web standards”, for example), so why not on the server side?

XSLT 1.0 also has reasonably universal support. (Universal 2.0 support would be even better, though we are not there yet, unfortunately.)

Better Separation of the View from the Model and Controller in MVC

A follow-up blog post will demonstrate this in PHP and, time-allowing, Mono/ASP.NET.

Unit Testable

Many framework templates have been tricky, if not impossible, to fully unit test in isolation from other layers and dependent systems.

ASP.NET’s new MVC model mentioned earlier has unit testing and test driven development as a driver for this approach, but from what I understand, while it makes controllers and models unit testable, the views are still not (please correct me if I am wrong in the comments below!).

From what I have seen of the other View templates in other frameworks, I believe the same constraint applies (but please let me know if I have missed something).

XSLT, however, requiring mostly an XML input (and possibly arguments passed in programmatically), can therefore make it easier to unit test the view, in isolation of a running web server, database, etc. It can therefore be easier to integrate into a developer’s coding process.

XSLT-based unit tests don’t necessarily need a whole web server set up to execute. Such unit tests, which are typically numerous by their nature, would be quick to run for the developer during development. This is important in the adoption of unit testing so it doesn’t make running them feel like watching paint dry after each modification!

Another follow-up blog post will attempt to demonstrate this.

Gain Full Control of the Markup Generation

As explained above, markup is important. XSLT gives you the ability to output the markup as you want, with little or no trade-off. (Some other frameworks also allow this, so this isn’t a unique advantage, by any means.)

Platform and Framework Agnostic

While the suggestion of using XSLT may introduce a problem of yet another technology, it may also enable skills-standardisation across platforms. This would be useful for say web developers that need to work on multiple projects, for different clients, using different technologies.

This of course makes the assumption that XSLT would be widely adopted in this way! If it were, migrating to desirable new frameworks where the View at least is somewhat XSLT-driven may not be as time-consuming (though of course time is still required on the middle-tier and database layers).

I have, for example, migrated an XSLT-driven site based on “classic” ASP, to PHP-driven XSLT, using the same pattern, but where the XSLT hardly changed at all, saving a fair chunk of time.

For a major client of ours at work, their extremely high-traffic .NET 1.1 web site (now moving to .NET 2.0) was architected to by-pass the .NET controls, to avoid all the problems described above, and we transformed XML with XSLT.

Doing this gave us a number of benefits all in one go:

1. We got the lean — and accessible — output needed
2. The “object-oriented” XSLT-based approach also allowed us to create reusable template hierarchies
3. The customer also wanted multiple sub-sites which was achieved with very little effort or duplication of code, while allowing each sub-site to be customised as needed — not just CSS or database-driven data differences, but template level changes such as markup and layout changes. (Had we attempted this with ASP.NET, we would have had to create Server Controls with lots of configuration and then in each sub-site we would have to wire them all up, which would lead to a lot of repetition.)
4. In one case, it wasn’t just separate sub-sites they wanted, but entirely separate web sites/applications (a public facing one, a kiosk-mode one, and a version of the site that could be used by customer support). All this ran off pretty much the same XSLTs with minor adjustments in inherited XSLTs for each web site.
5. The migration to .NET 2 involved no XSLT change (only C# changes to take advantage of things like generics, the improved XSLT processor, etc.)

And the sites comfortably take thousands of orders an hour on very few web servers (though of course performance is a bigger architectural issue; the point here is that XSLT is not a performance hindrance that some believe — see below for more on this).

Portable skill

Being both platform and framework agnostic makes this a portable skill. Of course, XSLT is not limited to just web-based scenarios, either, so it may be quite a good skill to have, generally.

Repeatable

Just as OO techniques and design patterns help with repeatability in different contexts, or just as adopting web standards/CSS-based layout helps to create a repeatable process that can be applied across any web site you wish to develop, using an XSLT-approach can apply to many different projects and in different frameworks.

Development Productivity; Help Inject Key Skills into Project Delivery

I have consulted at a few companies, and read/heard others describe similar scenarios, whereby skilled client side developers are limited to providing templates they throw over to the developers who are not often web developers.

This can result in either extra time explaining the required markup to the developers. Time may also be needed in working out alternatives (and unnecessary trade-offs) because of limitations in the framework of choice, or garbled markup coming out of the development process (ASP.NET, looking at you!).

This past year at work, we have been aggressively hiring for web development positions, looking for a mixture of strong web standards, and server side programming with .NET. We have found it incredibly hard to hire candidates who are exceptional in all these areas (many good web developers seem to avoid .NET for example, and many ASP.NET experts we find know little on web standards).

But we don’t always need everyone to have all those skills, either; the separation afforded by XSLT has allowed us to hire really strong people with parts of these and get them to work in parallel with others. Those well versed in web standards seem to find XSLT easier to adopt, in my experience, especially if full-blown, back-end programming is not their usual area.

Such developers are therefore not excluded from the development process.

Getting the key skills into the project in this way gives a number of benefits, including

• Greater productivity and adaptability: as project requirements change even during development, having more skilled, specialist staff than generalists in all these areas can pay off. (Generalists can often be extremely valuable too, not knocking that at all!)
• Parallel development. We have found it incredibly powerful when development phases are tight and mid-tier and client side development have to start somewhat in parallel. Some of our front-end guys who have not got skills in C# and ASP.NET are nonetheless able to fully contribute to the development process, and the project gains from their in-depth knowledge of the client side.

Encourage Front and Back-end Developers to Work More Closely

During day 1 of @media AJAX, interface designer Mike Stenhouse gave a talk from the perspective of a designer/front end developer having to know more about development, getting more involved in agile development, test-driven, even behaviour-driven development.

Designers understanding more software development processes, getting involved in it – not just throwing templates over the proverbial fence – would help communicate the importance of the “lowly” client side technologies.

At the same time, mid-tier developers get a chance to explain concerns such as architecture, OO, patterns, frameworks etc. This common understanding, even though they need not have to be experts on the other area, would allow them to be more productive.

Of course, Mike Stenhouse’s point isn’t unique to an XSLT-based approach (in fact, I think he was talking about Java or PHP). But one possible advantage an XSLT-based approach may have here is that it might be simpler to learn (controversial statement, maybe!) than some of the framework languages, such as Java, C#, PHP, etc. Where this is the case, XSLT can be a useful way to help bridge the gap.

Coding productivity

Using something like ASP.NET controls and simply dropping them onto a canvas seems really productive at first. I was certainly excited back in 2000/2001 as information about ASP.NET started to come out. I remember being really excited about the date control that ASP.NET had demoed years ago! Such a time saver – or so it seemed.

However, if you factor in the time you often need to sort out the mucky markup output, make things accessible, CSS-friendlier, and so on, the overall time, in my experience doesn’t seem that much better (in some cases worse, some cases better) than an XSLT-driven approach.

If you know the output you need to create, I find writing XSLT to be more liberating as you are in full control of your output destiny!

The switch I described earlier from ASP based XSLT-driven site to an PHP based XSLT-driven was also a switch from Windows/IIS to Linux/Apache (I was finally fed up of IIS’s limited configurability, in a hosted environment).

In just two weeks, a site with about 700 pages was converted, fully tested, and up and running again on a new hosting company. There was virtually no XSLT change, and the design pattern (a kind of Page Controller pattern which I might change slightly to MVC if time allows) was just reapplied to PHP. PHP 5’s object oriented capabilities meant further reduction in server side code. (Come to think of it, it wasn’t two weeks – it was a LOT less, as it was just spare time during some of the evenings.)

For work, most of our customers (one of them being amongst the largest in the world in their industry) use .NET. We often by-pass ASP.NET controls and use XSLT to generate the output instead. C#/.NET are still used on the server side, as .NET itself is quite powerful.

For the generation of the UI, XSLT has allowed us to even create multiple web sites simultaneously, as all the XSLTs can be reused. Where creating a separate sub-site in the past has take 6 months or more, our move to the XSLT-based approach has meant it has taken a fraction of that time (and included SEO, accessibility, web standards, etc.).

As another recent example: For my current project, a colleague (an excellent developer in general, though not well-versed in web-standards, yet) is creating a functionally rich and powerful page as part of the overall app we are developing. It has a complex grid of tabular data, with a folder/tree type browser on the side with lots of dynamic stuff possible on that page. Some controls created by a vendor for ASP.NET 2.0 are being used. While they are rich in functionality, the page produces just under 1MB of output!!!

The output from these controls are plagued with excess divs and unnecessary table-based layout (tables for tabular data of course is fine). Styling some of the specific features also becomes tricky (because the markup created is not very good and can’t be changed).

In addition, on a gut feel, the amount of C# used to prepare the data to bind it to the grid-based controls seems excessive. In the output, there is a lot of unnecessary (and bloated) JavaScript needed to get it to behave as needed, too.

This also (currently) runs like a dog on IE 6 because of all that JavaScript and HTML bloat! I have spent a total of 15 days just trying to get the resizer/splitter control that contains the table and tree to resize properly by battling its JavaScript (to be fair, during that time in the summer, I discovered the excellent jQuery, which helped reduce that effort considerably but I was still using jQuery to do unnecessary stuff)!

In my own opinion, it would have been much more efficient to have written the whole thing with XSLT and use high quality client-side components from people like jQuery, YUI, Dojo, etc. The trees and tabular data on that page are really well suited for XSLT processing. To be fair, the guy writing the page isn’t as well versed in XSLT/web development but an experienced GUI developer, so the control approach makes perfect sense for him and he has certainly made me appreciate that approach more than I used to. (I do still feel that the amount of code needed feels excessive and the experience so far has strengthened my view that while ASP.NET is powerful on the server side, I prefer to use XSLT to create the markup, where possible.)

Drawbacks (or perceived drawbacks) to the XSLT approach

XSLT is certainly not to everyone’s liking. The right tools may not be there in the favored IDE, or it may just look too weird for some people (it is more like functional programming than procedural or object oriented, so can sometimes take a while to get used to). Here are the disadvantages of XSLT I can think of, and where I can, some ways to overcome those problems.

Not everyone likes XSLT!

Many who are used to OO or procedural programming may find this functional programming language strange to get around. Angle bracket coding isn’t to everyone’s taste! And these personal preferences can be important for a team/project. Initially I didn’t find it too intuitive, but after a few days with it, it grew on me.

(And I certainly don’t hold the position that one must only use XSLT. Different people will prefer different technologies, and are of course free to use them.)

Yet another technology to learn

I kind of appreciate this concern, but also in some ways do not. For example, XQuery, or the more generic and powerful LINQ from Microsoft, get many developers excited, and they are willing to learn those if needed. People will invest time in learning CSS, JavaScript, etc.

So, perhaps it could be argued that just like any other technology, if XSLT is determined to be appropriate for the project at hand, then that can be learned too.

XSLT 1.0 is Verbose and Weakly Typed

Some simple things in XSLT 1.0 are a bit verbose to code (though XSLT 1.0 can often be written more cleanly than what many people write).

XSLT (and XPath) are indeed weak on typing — at least 1.0, though given the nature and purpose of XSLT this is not too limiting a factor as it would be if this were a full-blown language. (Some popular languages for the web are also weakly typed, too, e.g. PHP and JavaScript and the “limited” typing can also result in many capabilities, depending how they are used.)

XSLT 2.0 and XPath 2.0 are more powerful and overcome many of these limitations, adding many more powerful capabilities, allowing for leaner code and improved performance.

However, while XSLT 2.0 is an open standard, many have not supported it. (Microsoft, for example, has a decent implementation of XSLT 1.0 in .NET — and MSXML — but has hinted that XSLT 2.0 is not a priority for .NET at the moment — see also this discussion for more information.)

XSLT is a limited language

In a way, if compared to more general purpose languages such as Java or C#, XSLT (especially 1.0) is very limited in what you do.

That can be off-putting straight away.

However, (using extension objects etc aside), the simplicity of the language can also be considered a strength; it may help architect a better separation of data, presentation and logic. I personally have not felt too much pain from its limitation.

Poor Tool Support Compared to Other Languages?

Tool support for XSLT does not seem as good as ones for say C# or Java.

With C#, both Visual Studio in-built refactoring or JetBrain’s excellent Resharper makes refactoring a lot easier. Same with Eclipse and other such IDEs for Java and various other languages.

Need to rename a class that is used throughout your super-massive project? No problem, just right-click the definition and rename; the IDE will correctly update everything for you. Want to implement missing implementations of an interface? Right-click and get it to fill out the missing implementations. Want to move a block of code into a separate method? Highlight it, right click and select an option to extract a method. Want to find all the places where a particular method is used? Right click it and select find all instances. And so on.

In addition, many IDEs and plugins for code assistance goes beyond “intellisense”, advising on good coding practices, code formatting, etc. This is immensely powerful, especially when establishing team standards.

With XSLT, however, the level of sophistication is not as high, as far as I am aware. For example, finding instances of where a given XSLT template is used seems limited. The best editor I have come across for this purpose is oXygen (let me know if there are better ones!). It is has various refactoring and finding capabilities, but finds don’t support searches for XSLT template modes. (I find using moded templates is generally a cleaner way to code many XSLT.)

A developer is therefore reduced to global finds within their text editor of choice. To be honest, I don’t particularly find this a problem, but I know that some developers, especially when used to using tools like JetBrain’s excellent refactoring tools, will find this backward, and I can’t blame them!

Many IDE vendors, such as Microsoft, don’t have their priorities on XSLT. While somewhat understandable, it does contribute to the chicken-egg situation:

• IDE tools are not as good for XSLT;
• XSLT therefore not as widely adopted in that sphere of development;
• IDE tools remain limited for XSLT;
• XSLT development seen as backward from an IDE/tool support perspective so usually not looked at.

As to what IDEs/tools/editors are useful:

• A number of people at work like to use XML Spy for XSLT development, though I personally don’t like it’s text editor.
• I tend to use jEdit, which is a Java-based IDE, so I have used it on Linux, PC and Mac, quite nicely (it is hideous to look at initially, and I have spent a long time customising it! I will explain that in some future blog post!).
• Eclipse and the Aptana plugin looks really useful, and I am beginning to look at that more.
• Many people also swear by oXygen, so I am looking at that a bit more, too.
• But, for my current project, for the past year, at work I have been content with using Visual Studio 2005 (and now 2008) for XSLT. It has an XSLT debugger (and recently a profiler) but I have not needed to use the XSLT debugger yet, so can’t comment on it.

If you know of any other useful XSLT tools, please comment so!

Refactoring in a mixed language environment can be problematic

Suppose you have an XSLT that transforms some XML which comes from a Java or C# serialized object.

If you modify that object in a simple way (rename the class or a public member), in IDEs such as Visual Studio and Eclipse, you can very quickly update all usages of your class/method throughout your project and unit tests, and you are done.

But, imagine you are renaming class members that get serialized in such a way that the name of that class member is the name of the element/attribute in the XML representation. IDEs aren’t going to easily know that and you are likely to have to manually update the XSLTs.

Unit Tested XSLTs may help to some extent to find where changes are needed, but it may still be more work than just working in C# or Java.

Getting data as XML may not be practical

Using XSLT assumes you can easily (or without much cost) get your data in XML format, so you can transform it. This may not be possible in many cases.

In many instances, with in-built XML serialization capabilities, such costs are often minimized or minimal. (You are often doing well if this is your biggest problem in your overall system architecture!)

Even if serialization is not possible, XSLT performance can benefit from well structured XML. Is it easy to create that structure from your object model? Depends on the project at hand.

Perceived Performance Concerns

A common question I often get in discussions of XSLT is concerns of performance.

XSLT performance, amongst other things, is also affected by:

• The XML being transformed – its structure, size, etc.
• The efficiency of the XPath expressions (e.g. using ‘//’ or invoking dynamic scripting languages from within XSLT can be performance killers).
• XSLT itself (e.g. loading, interpreting and executing XSLT in general, whether or not the XSLT Processor is compiled/cached/reused or not, etc.)

As well as addressing the XML and XPath concerns, the XSLT performance concern can also be addressed in many cases.

Using cachable XSLT Processor implementations can dramatically boost performance. (Kind of like XSLT being compiled into the native code, like Java bytecode or .NET IL, etc. and then caching them in memory across sessions, for subsequent reuse). We use this approach on one of UK’s most profitable and biggest online retail web sites, for example.

Also, as a high level language, XSLT has potential to be optimized further at the implementation level resulting in better performance without changes needed to the XSLT itself.

(Extensibility through XSLT extension objects may offer some performance gains if you write extension objects in a compiled language such as Java or C#. A risk may be that you lose some portability — though how much that is a risk depends on your circumstances.)

Fragment Caching is Hard

In addition to the performance boost from using cachable XSLT Processors, the ultimate caching strategy is being able to cache the entire output of a page. Subsequent requests for the same page require very few server resources.

However, there are times where the entire output is not easy or practical to cache, but fragments of the output could be. An example is a page of personalized data combined with general, cachable data.

XSLT in a View is typically used to generate the whole output, so fragment caching could be hard. Ways to mitigate might include implementing caching strategies in Models (which might be worth doing anyway), pipelining XSLTs, where the output of one transformation is fed into the input for another XSLT allowing cached fragments to be saved temporarily in the process.

In this case, specific templating languages such as that from Django, ASP.NET etc may offer better abilities. XSLT 2.0 may also have some more features in this area (it has better support for multiple XML documents, for example). If XSLT were to ever become even more popular for this usage, then I imagine there would be more advances in this area, too.

It implies error-prone XML-based templates

Consider this concern from Django’s design philosophies behind its templating system:

XML should not be used for template languages

Using an XML engine to parse templates introduces a whole new world of human error in editing templates — and incurs an unacceptable level of overhead in template processing.

— Design philosophies, Django, last accessed January 4, 2008

While the above is not specifically geared towards XSLT, human error is certainly an issue with XML-based templates, or XSLT-authoring.

That error, if it comes at template creation time, is typically caught during the development/debug/test cycles, as malformed XML will error early.

Of course, if content comes from external sources (user generated, from third parties, etc) the well-formedness of that XML cannot be guaranteed. However, these are usually not likely to be within the template itself. It may or may not be possible to cleanse such data, depending on the project.

In addition, as addressed earlier, I do not believe the overhead of template processing is that bad. Cached XSLT processors, as well as other caching strategies can help, for example.

A colleague of mine found an ASP.NET data grid control taking some 2 minutes to bind and render a standard .NET dataset. He changed that into an XSLT transformation instead and the data transformed in around 20 seconds — clearly an exceptional case compared to most web needs (2 minutes or 20 seconds implies a LOT of data), but XSLT is not automatically slow.

In addition, depending on the application, the generation of the XML may come entirely through serialization of objects, which in languages such as Java and .NET (not sure about python, which Django is in) is reasonably fast.

In a current project I serialize a very large object (about 200K when saved as XML) and transform it into HTML. The HTML is a very complex form of over 300 form fields! (Fear not, we use a tab-like interface and various other techniques to group them!) Using the cached XSLT Processors on my developer box, the transformation takes just 5-10 milliseconds. The far bigger cost is getting the data from the database in the first place (about 50 to 120 milliseconds, but that is on a developer database on the same box as the rest of the stack). The serialization takes about 5 milliseconds. Even when the XSLT processor is not cached, the transformation times are very small (I think about 150-300 milliseconds depending on the data).

That same design philosophy’s very next point was that you should

Assume designer competence

The template system shouldn’t be designed so that templates necessarily are displayed nicely in WYSIWYG editors such as Dreamweaver. That is too severe of a limitation and wouldn’t allow the syntax to be as nice as it is. Django expects template authors are comfortable editing HTML directly.

— Design philosophies, Django, last accessed January 4, 2008

I would tend to agree with the bit about assuming designer competence, which is actually why I find XML for templates are not as bad as it may initially seem.

As I mentioned earlier, in my experience, when trying to hire front end developers, we have found they are quite capable at adopting XSLT.

It is not that Django’s is wrong in my opinion; I am sure there are many for whom XML-based templates are not appropriate. My point here is that there are also many for whom XSLT will be seen as appropriate, too.

(And this is not a criticism of Django, either. I have only looked at it very, very, very briefly and am already impressed with it!)

Coding Productivity is Perceived to be Poor; Hand writing all that code seems inefficient

A common concern I hear (usually from middle-tier developers who want to avoid writing HTML) is that it is inefficient to hand write all that code. I can certainly understand their concerns; it is something I shared when I used to do more middle-tier development than front-end.

However, I actually find the opposite can be true: After using it for many years, a few colleagues of mine and I find it is far more productive creating web sites this way, because we know what we need to output, and we don’t spend ages battling tools that generate HTML to make them do stuff they don’t easily want to. We also help minimize, or completely avoid the excessive bloat that many tools create.

In addition, past experiences suggest that many of our XSLT projects seem to have less code than would be needed in say a C#/ASP.NET template (like the tree/grid example mentioned earlier).

Unlikely to get vendor buy-in if they have vested interest in other ways

Vendor buy-in may be limited if there is an inherent interest in vendor lock-in.

This is a mixed one too. Take .NET for example. Microsoft is unlikely to do away with its control-based approach and cause more upheaval to its target developer audience by doing this. It also reduces the dependency on Microsoft!

I would think that if this XSLT idea has any large-scale merit, open source platforms/frameworks are more likely to adopt it, though that is not necessarily a given, either.

XSLT for web-based Views hasn’t taken off yet, so it must be a bad choice

I think this point can have merit. However, I think there may be a number of different reasons why XSLT has not taken off.

For example, it may not have been appropriate in the past because XSLT tools were not as mature. While the XSLT tools were maturing, other technologies came through.

In addition, not all data is always easy to represent as, or convert into, XML, and the tools built into languages these days such as serialization may not have been around sufficiently in the past.

But the chicken-egg scenario mentioned above about IDEs not supporting XSLT as much so XSLT not getting as much traction in this area, could also be a factor. And, as hinted above, some vendors may have interests in promoting their own ways of doing things as an open standard may make it easier for someone to move to another platform, if needed.

Of course, developers not liking the language may also be a factor. (It seems to be quite big in the Java world, from what I can tell, though I don’t have much hands-on experience with Java these days – if anyone has some more insights here, please let me know!)

We have used this XSLT approach very successfully on a number of projects, one of which is one of the most prominent online retail web sites in the UK, if not the world. We haven’t really had a chance to write about it. Maybe there are some (many?) others who have also not had a chance to write about similar approaches? The BBC, I understand from some colleagues/friends that have worked there in the past, use it but in a more off-line scenario to pre-generate certain pages/sub-sites. So it can’t be all bad!

In addition, valid (or invalid) reasons to reject it in the past do not always automatically apply in for now or the future, especially in a dynamically changing environment such as the Internet. Times and technology change, so maybe it is at least worth re-validating (not adopting blindly of course!).

In conclusion

Despite the concerns, or perceived concerns, I feel the potential advantages make it worth considering.

There are certainly XSLT-based frameworks out there, though not as popular. (Apache’s Cocoon project comes to mind, for example.)

I would think that just as advocates of web standards have pushed hard to get open standards adopted for client side web development, maybe there is a case for XSLT on the server side to aide in that process.

But it also depends on personalities, priorities, and specific requirements, too. Some people really don’t like XML-based things, which is fair enough as sometimes there are legitimate reasons for it.

Others may not be convinced by the need for controlling markup, thinking it is unimportant or that their favourite vendor will sort it all out for them!

I am not arguing that XSLT should be the only way of doing things. I think it can be useful in many situations, and I have certainly found it to be so. Following the ideas of using web standards on the client side, using XSLT on the server side (at least for the generation of output) appeals to me too. What I therefore hope to do in coming posts is demonstrate some possibilities.

Some examples

To explore this further, as this post is already quite long, in the next few days or weeks (whenever I get the time!) I will add posts to demonstrate

• Unit testable XSLT
• “Object-oriented” XSLT
• XSLT as the View part of MVC, or as part of a Page Controller pattern.

[You might, rightly, wonder why this is a WordPress blog, and not something I have written using say PHP with XSLT! Fair, but lack of spare time is the main problem: It has taken me two years just to get round to starting this blog! One day I will convert, time allowing!]

In the meanwhile, if you have got this far, thanks for bearing with me!

Do you have similar concerns, or agree/disagree?

Image credits

“XSL Transformers” reposted with kind permission from NaGraDim from flickr. See the original image at http://www.flickr.com/photos/nagradim/750087828/.

Translation

Update: November 7, 2012: This article has been translated to Serbo-Croatian language by Jovana Milutinovich from WebHostingGeeks.