When transforming XML via XSLT, make sure the output setting for indenting is turned off and honoured by your code.

Turning it off will

• Speed up the transform time
• Reduce the output size

How is XSLT output indent turned off?

Indenting output should be off by default. If for whatever reason it is not, it can be turned off simply by using this:

<xsl:output indent="no" />

I have seen a lot of .NET XSLT ignore this important setting. Sometimes it is because the output is written to a stream or something else which doesn’t take the output settings from the XSLT.

When transforming to an XmlWriter in .NET, be sure to correctly create the XML Writer using the overload that takes in output settings obtained from a loaded XSLT. E.g:

XslCompiledTransform xslt = GetXslt(XsltPath);

// The 2nd argument honours XSLT the output settings!
using (XmlWriter w = XmlWriter.Create(sb, xslt.OutputSettings))
{
    XsltArgumentList args = GetXsltArgs();

    XPathNavigator navigator = _someXmlData.CreateNavigator();

    if (navigator == null)
        throw new NullReferenceException("navigator");

    if (w == null)
        throw new NullReferenceException("w");

    xslt.Transform(navigator, args, w);
}

(xsl:output has many other useful attributes worth looking into.)

What kind of savings do you get?

The savings will differ for many reasons (the XML document size, the structure of the XML, the types of transformation being done etc etc), so it is hard to give a definitive savings. But here’s an illustrative example:

A few weeks back, a colleague at work was having trouble with an XSLT that was taking a long time to transform.

We started to have a look; I was expecting to find inefficient XPath, or an XML document structure that wasn’t conducive to decent transformation speed, or something like that.

However, I noticed the first thing was his output was set to be indented, and the XML input was HUGE.

So, the first thing we did was turn it off.

That alone reduced a 12 minute transform (on a 300MB document) to just 1 minute!

In another scenario, a 70K XML document was taking about 0.25 seconds to transform. Turning off indenting shaved a few more milliseconds (can’t remember exact amount now) — and saved about 2K in the output HTML that it generated.

Why can this make such a difference?

I think the specifics may vary depending on the XSLT parser you are using, but I believe it is basically this:

• Each newline/white space/tab(s) created for the indent requires an extra text node to contain these characters which requires extra memory (though at this point may not add that much time to the transformation).
• When the transform is then saved to a file, or written out to an output stream strings are often involved. String processing can be expensive in many programming languages, so each of these indented text nodes needs handling. For very large documents (as above) this can require a lot of unnecessary processing.

But doesn’t this make the output harder to read?

The consumer of a transformation result is likely to be another process such as another XSLT in a pipeline, another process, or even a web browser.

None of these typically care about the extra white space, which also would require more processing when loading.

If you need to view the XML, it may be worth keeping the indent off and manually opening it in a text editor that has the ability to “pretty print” it for you. (Warning: some editors and IDEs, e.g. Visual Studio, can do automatically pretty print an XML document for you when you open it, making you think the XML itself had the indented output!)

Other savings are still possible

Turning off the indent is just one of many things you can look into. Other things include the following (though your mileage may vary):

• Use attributes instead of elements (where possible; usually this is for simple values, such as numbers, dates, and very limited strings, and where the element is not expected to be indented)
• Look at the XML structure to see if can be improved to make XSLT processing easier
• Cache the XSLT Processor
• Cache the output

I’ll try to expand on some of those in future posts.

Some more detailed XSLT performance tips which also created cleaner code were covered in an earlier post.

A CDN is a network of servers around the world to serve content from your site from the nearest physical location. All the large sites (Yahoo, Google, Amazon, etc) use them.

After reading the above post, I was also curious to find out how if Google App Engine helps in the following:

• Compression
• Expires headers and versioning

A comment in the original post implies compression is on by default, which is what I’d expect.

Using far future expires headers can be a great performance boost for your site, and is easy to apply to assets such as images, CSS, and JavaScript files.

At the same time, if you change such a file you want to be sure your repeat visitors will not see the old one because it was cached far into the future.

So, with a simple URL Rewrite Rule we can make things like /css/version-1/site.css point to /css/site.css.

If you update site.css, you can change the version number. Browsers will not have a file from this new path in their cache so will download it and cache it into the future again.

Is it possible to do this with Google App Engine? It looks promising…

If I can find a spare moment, I may try this out on this site (which, admittedly doesn’t get that much traffic to make it worth bothering about!) — and then I’ll try it on a site that takes up most of my spare time, which gets a LOT of traffic, where it would actually be worth doing!

This will allow you to write web pages that refer to those scripts rather than copies on your own site, reducing your bandwidth, but also leveraging the infrastructure capabilities of Google, such as their content distributed network (which means users would be served those files from a location much closer to them), properly compressed, minified, cacheable files, etc.

In addition, if your visitors have been to other sites using the same technique, they would not need to download the same libraries all over again, and with increasingly rich web sites, these files can get quite large.

So, if you use jQuery, you can use this in your web pages:

<script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.2.6/jquery.min.js"></script>

Of course, this is not for everyone, and it may be worth bearing in mind some concerns raised by Peter Michaux, who also provides a useful way to handle the scenario where you think access to Google from the user’s location might be restricted (e.g. blocked from a paranoid office!):

<script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.2.6/jquery.min.js"></script>
<script type="text/javascript">
  if (!jQuery) {
    // load from your own server
    document.write('<script type="text/javascript" src="/js/jquery.min.js"><\/script>');
  }
</script>

Still, it could be a useful technique to further improve the download/load performance for a number of sites.

A number of months back, Yahoo was the first to announce something like this when they said they would host their YUI library for you so it is good to see others getting into this.

One of the suggestions, an old one, is to set the HTTP Expires header well into the future, especially for resources that your page uses such as images, CSS and JavaScript files, so that the browser almost definitely cache them. (They don’t get cached as often as you might expect!)

Setting Expires Header in Apache

With Apache, one easy way to do this is to generate a link that contains a build/version number or date in the path, e.g: /css/{build-number-or-date}/file.css

In Apache, in your configuration file (e.g. .htaccess if that is all you have access to) you can set the Expires header and rewrite your url to the real location.

For example, I have actually used the following for this web site (at time of writing):

<IfModule mod_expires.c>
ExpiresActive On
ExpiresByType text/html "access plus 1 seconds"
ExpiresByType image/gif "access plus 2 years"
ExpiresByType image/jpeg "access plus 2 years"
ExpiresByType image/png "access plus 2 years"
ExpiresByType text/css "access plus 2 years"
ExpiresByType text/javascript "access plus 2 years"
ExpiresByType application/x-javascript "access plus 2 years"
</IfModule>

In the rewrite rule above, things like css/1/file.css can get rewritten to css/file.css, where ’1′ is a build number, and css/file.css is the real location on disc.

This allows the developer to keep their files in a certain place and then either their build/deploy procedures (ideally) or in their code, manually, point to a build number that the rest of the world would see.

This way each time you deploy a change, the build number changes, the browser sees this new URL and requests it again only then.

Setting Expires Header in ASP.NET

With ASP.NET and IIS 6 because of the weak integration, things are not quite as easy. With IIS 7 this should be much more configurable, but for now, the basic thing to do is to consider using a file extensions that will trigger ASP.NET to handle the request (e.g. .aspx). Then, you can set the expires header via ASP.NET and then use RewritePath() method (on the HttpContext class) to rewrite to the real file (e.g. maybe rewrite css/1/file.aspx to css/1/file.css).

There are numerous ways to do URL Rewriting in ASP.NET but most ultimately end up encapsulating RewritePath in some way or form. Some people have written custom configuration handlers, others can just write some bespoke code. We have used this on high volume sites for friendlier URLs, but the same principle can apply for this performance tip. Scott Guthry from Microsoft has a great write-up on using URL rewriting in ASP.NET.

This might incur an initial performance hit (what is normally just a file served up by the web server is now processed by ASP.NET), but subsequent requests for pages that link to such resources will no longer need to communicate with the server.

Why use URL rewriting in the above examples?

Why not use a querystring and avoid the rewrite altogether? Cal Henderson from Flickr answers:

According the letter of the HTTP caching specification, user agents should never cache URLs with query strings. While Internet Explorer and Firefox ignore this, Opera and Safari don’t – to make sure all user agents can cache your resources, we need to keep query strings out of their URLs.

Cal Henderson, Serving JavaScript Fast, May 21, 2006

But won’t development be problematic?

One initial concern might be that with development and constant modification of CSS and JavaScript, for example, it will be harder to test. This can be addressed to. For example, on browsers such as Firefox and Internet Explorer, a ctrl refresh will reload the new file.

This still leaves the question of how to update the version/build number without manually having to update the template each time. To do that, your code that generates the output can simply insert the build or file last modified time into the URL path (looking up the file’s last modified time might incur an extra disk hit, which may not be as good for extremely high volume sites).

See Cal Henderson’s article above for more ideas on that.

Yahoo offers to host their YUI framework; even more performance

Yahoo has offered to freely host their YUI JavaScript framework for anyone to use. They configure it with all the performance suggestions they suggest, and version it in case you don’t get to update your code as and when they introduce new builds.

What is also interesting about this approach is that if you visit other sites that use YUI you download even less because those scripts are already cached by the browser because of the use of the Expires header.

Oftentimes the performance of the client side (in particular the perceived performance/download speed) is neglected or given lower priority.

People at Yahoo, Google and elsewhere have found that as much as 80% of the user’s time can be spent waiting for things other than the main HTML to download. Furthermore, browsers may not be caching as many files as you think.

(This is not discouraging performance tuning your overall architecture of course; its no good if it takes a long time to receive the first bytes of your page in the first place!)

Techniques in the old days

In the “old days” of the Internet, dial-up was the primary way to access the Internet. “Web standards” were a distant dream.

All sorts of tricks were used to speed up web sites: smaller images, omitting quotes in HTML attributes (yes, people went that far!), using fixed width tables in IE 5 (the best browser at the time, using <table style="table-layout:fixed">), balancing server response buffering vs creating the whole page first and then sending it all in one response, etc.

But some techniques considered back then are still relevant today, e.g:

• Putting images on different domains (though now other resources are worth considering too)
• Compressing the server output using gzip (a bit hit and miss on earlier versions of IIS though)
• Cache control using the Expires header
• Use CSS (though in those days I used it mostly for font definitions and browser sniffing was needed to load different CSS files!)

Techniques now; some of the same

Fast forward to now, and broadband is widespread today (but don’t forget, dial-up is still around!).

Web standards encourage CSS-based layouts, reducing overall HTML bloat (some pages my colleagues and I have converted are 75, even 80% smaller!). Table-based layout and the font tag are mostly put to rest by people using such techniques (though its easy to forget that even into 2007 most of the web is not this way, and even with modern techniques we sometimes see sites where “div-bloat” and “span-itis” replace table- and font-heavy HTML!).

Web pages are also a lot richer these days, especially with “Web 2.0″. This requires far more JavaScript than in the past, increased use of flash, larger CSS files and images for a richer (hopefully more usable!) experience.

Nate Koechley, a senior developer a Yahoo gave a great presentation at @media07, called High Performance Web Sites summarizing a number of recent findings and suggestions.

In recent months, others at Yahoo as well as Google and elsewhere have been providing a lot of analysis. Perhaps the highest level summary guidelines would be the following:

• Use GZip on all text-based output (which should be optimized anyway via web standards!)
• Try to reduce the number of HTTP requests (e.g. combine CSS and JavaScript files during development or during the build process, use CSS sprites, if possible etc)
• Help browsers ensure they can cache as many of your files as possible
• Let browsers download your page resources in parallel even though HTTP 1.1 limits you to 2 concurrent requests from a domain (through sub-domains, minimizing or eliminating loading CSS and JavaScript in a way that blocks the browser, etc)
• Disperse your content through things like content distribution and extra sub-domains (though be aware of extra DNS-lookups that might result)

Rather than dive into the details here (for now), the following may be useful starting points:

Further reading

• Exceptional Performance from the Yahoo Developer Network. Some gems from them include the following
  • Thirteen Simple Rules for Speeding Up Your Web Site
  • What the 80/20 Rule Tells Us about Reducing HTTP Requests
  • Browser Cache Usage – Exposed!
  • When the Cookie Crumbles
  • Maximizing Parallel Downloads in the Carpool Lane
• Optimizing Page Load Time from a Google engineer. (One of the many interesting tips was to ensure HTTP keep-alives were enabled. They usually are but for a major client of ours last year as we launched a new web site, all was fine until the actual launch day when things were so slow. For a couple of days the hardware engineers were looking into this and had brought in a Cisco engineer. I decided to have a quick look. Using the excellent LiveHttpHeaders extension for Firefox, I noticed that HTTP keep-alives were not set. There had been some hardware problems or something which turned this off. Amazing how much difference this setting can make!)
• Serving JavaScript fast from Vitamin. (Lots of good stuff here. See also the note about Apache’s mod_gzip and mod_deflate. In short, use mod_deflate if you can.)
• Efficient JavaScript from Dev Opera
• Reducing HTTP requests from fiftyfoureleven.com (also makes a note about the impact of http packet sizes worth reading)

Some useful tools

• YSlow is a plugin for Firebug that measures the 13 tips Yahoo mentions (see articles above)
• Firebug extension for Firefox is indispensable for all manner of web development. It has a tab graphically to show content being requested, how long it takes, etc. Very useful to visually see how well your page components download
• LiveHttpHeaders extension for Firefox allows you to see HTTP traffic in real time
• iehttpheaders is plugin for Internet Explorer, similar to LiveHttpHeaders
• JavaScript minification tools:
  • JSMin
  • shrinksafe

As an aside, JavaScript minification tools help eliminate white space, replace variable/function names etc in with smaller tokens. (Similar tools for CSS also exist.) A few thoughts/observations on these tools:

If you had only once choice, go with gzip. You can combine it too.

Some time ago I wondered if minification might actually reduce the repeating patterns that make compression effective. A quick test found it was better to avoid minification. I unfortunately never saved the test files.

I asked Nate Koechely about this at the above conference, and he also mentioned JSMin and shrinksafe. Those were the ones I had used so that night I tested again but of course with different JavaScript files. I think one of these tools had also been updated since. The test yielded better results by combining minification and gzip compression.

I will try some further tests in the future as it may be either the type of JavaScript file, the tool being used or a combination. A build/development process to make this maintainable and debuggable will also be important to consider.