RequestReduce

Released RequestReduce.SassLessCoffee: Now have RequestReduce compile your .less, .sass, .scss and .coffee files before minifying and bundling them by Matt Wrock

I’ve been wanting to get this post out for a few days since I launched this last Friday. I have to make this quick before I hop in the shower to go to work. I owe at least that to my coworkers.

Less, Sass and Coffee script compiling

I’ve had a few RequestReduce users asking me about adding Less, Sass and Coffee script compiling to RequestReduce. Technically, this has been doable for some time along side DotLess and SassLessCoffee.AspNet. However, I wanted to optimize the integration since by default, using the handlers provided by those two excellent projects bring in their own caching and, in the case of SassAndCoffee, its own bundling and an issue with setting “near” future caching headers (which is now fixed in the latest source code). This extra caching and bundling just adds unnecessary processing time and possibly disk space consumption.

RequestReduce.SassLessCoffee, available at http://RequestReduce.com and via Nuget now calls into the APIs exposed by both DotLess and SassAndCoffee.Core to have them simply compile the contents of your .less, .sass, .scss and .coffee files. RequestReduce then takes over with its own minification, bundling and caching logic.

To get this simply requires adding the RequestReduce.SassLessCoffee Dll and those of its dependencies to your project. No handler config entries are necessary in your web.config. This is because RequestReduce will map these extensions to its own handlers at run time.

Hopefully this is what the Less/Sass/Coffee users have been looking for in RequestReduce. Please let me know if you run into issues here.

New Look on the RequestReduce Homepage

I gave RequestReduce.com a facelift last weekend. A couple weeks ago Zach Holman gave me some good advice on my github readme. He commented that my headline and the first paragraph was too wordy. I had felt the same way for a while but this weekend finally did something about it. I shortened up my github headline and readme and took it a step further and totally revamped the homepage. It looked more like an essay than a pitch for RequestReduce. When I’m looking for OSS solutions, I typically don’t like essays as my introduction to the project. So now the first thing you see are basically bullet points highlighting what's important and some before and after images from www.webpagetest.org waterfall tests.

I’d really be interested in hearing your feedback on the new look. I already shrunk the huge Getting Started font based on some early comments. Does this quickly “bring the message home?” I did get a comment about the waterfall images appearing out of context and that the reader may not know what they are or why they are important. Their importance seem clear to me but then I do perf testing every day unlike a lot of other devs and web folk.

Adding CSS Inheritance Resolution to RequestReduce Sprite Generation Process by Matt Wrock

UPDATE: This functionality was released on 1/9/12 in v1.7.0.

Currently RequestReduce is unable to sprite alot of background images because of its limitation of treating each css class as an atomic unit. Inheritable dimensions and padding is lost which either makes an image unspritable by RequestReduce or the sprite is malformed because of improperly calculated dimensions.

Further, pre-sprited images in a document about to be reduced are often malformed and rendered in a distorted fashion because it is common practice to place the background url in one widely inherited class and have more specific classes define just the background-position of the individual image in the sprite. This makes many initial renderings using RequestReduce give a bad first impression especially if the user does not read the wikis on preparing the sprites and spriting requirements here and here. It can also make the exercise of converting docs to be compatible with RR spriting a time consuming and tedious endeavor.

I had initially thought this was a small edge case but it is proving to be an all too common problem and I think the adoption story could be dramatically improved by adding this feature. This is the one feature that can break the RequestReduce golden rule: do not break the page I am reducing. The addition of this feature can prevent this.

Unfortunately it is not a small effort. However it is not monumental either.

Released RequestReduce 1.5 Supporting Custom Minifiers and a Critical Performance Fix for v1.4 by Matt Wrock

I just released RequestReduce v1.5 on Nuget and on http://www.RequestReduce.com.

If you are currently on v1.4, you will definitely want to upgrade to this new version. There was a misconfiguration in v1.4 causing a Regex to recompile on every call. The impact will vary depending on the number of scripts and css links on your site but is could well be in the 100s of milliseconds.

Enough about that. The key feature that v1.5 adds is a small API allowing you to do the following:

  • Log RequestReduce exceptions thrown from its processing thread to your own error logging implementation including very simple support for Elmah logging.
  • Plug in a custom minifier to replace the RequestReduce defaulr Microsoft Ajax Minifier. You can also use this if you want to override the settings that RequestReduce uses.
  • Filter out certain CSS, Javascript or Image resources from being processed by RequestReduce including entire pages or areas of your site from being processed. The API allows you to evaluate any property in the HttpRequest to decide if a resource should be filtered.

Here are the details on how to use the API:

Logging RequestReduce Errors

Especially if you already have a error logging mechanism, it is advisable that you log any errors encountered by the RequestReduce reduction processing. This will aid in bug reporting and generalk troubleshooting. To do this, you simply need to provide an Action delegate that will take charge of logging the passed Exception. Here is an example of how to use this if you use Elmah for error logging:

RequestReduce.Api.Registry.CaptureError(x => ErrorLog.GetDefault(null).Log(new Error(x)));

 

Injecting your own CSS or Javascript Minifier

RequestReduce attempts to follow a decoupled architecture which allows developers to swap out certain parts with their own behavior. To override RequestReduce's use of the Micosoft Ajax minifier library, you simply create a class that derrives from IMinifier. There is not much to IMinifier:

public interface IMinifier{    string Minify<T>(string unMinifiedContent) where T : IResourceType;}

Here Is RequestReduce's implementation:

public class Minifier : IMinifier{    private readonly Microsoft.Ajax.Utilities.Minifier minifier = new Microsoft.Ajax.Utilities.Minifier();    private readonly CodeSettings settings = new CodeSettings {EvalTreatment = EvalTreatment.MakeAllSafe};

    public string Minify<T>(string unMinifiedContent) where T : IResourceType    {        if (typeof(T) == typeof(CssResource))            return minifier.MinifyStyleSheet(unMinifiedContent);        if (typeof(T) == typeof(JavaScriptResource))            return minifier.MinifyJavaScript(unMinifiedContent, settings);

        throw new ArgumentException("Cannot Minify Resources of unknown type", "unMinifiedContent");    }}

It's not difficult to imagine how you would change this implementation to use something like the YUI Compressor for.Net. Lets say you had a YuiMinifier class that you want RequestReduce to use instead of its own minifier. It might look something like this:

public class YuiMinifier : IMinifier{    public string Minify<T>(string unMinifiedContent) where T : IResourceType    {        if (typeof(T) == typeof(CssResource))            return CssCompressor.Compress(unMinifiedContent);        if (typeof(T) == typeof(JavaScriptResource))            return JavaScriptCompressor.Compress(unMinifiedContent);

        throw new ArgumentException("Cannot Minify Resources of unknown type", "unMinifiedContent");    }}

You would just need to add the following code to your startup code:

RequestReduce.Api.Registry.RegisterMinifier<YuiMinifier>();

That's it. Now your minification code will be used.

Filtering Resources from the Reduction Processing

If you would like to have certain CSS or javascript resources filtered from the reduction processing or if you would like to exclude certain images or even entire requests from all response transformations, then use the AddFilter method:

public static void AddFilter(IFilter filter)
 
RequestReduce provides four concrete types deriving from IFilter:
 
public class CssFilter : Filter<CssJsFilterContext>public class JavascriptFilter : Filter<CssJsFilterContext>public class SpriteFilter : Filter<SpriteFilterContext>public class PageFilter : Filter<PageFilterContext>

All of these derrive from:

public class Filter<T> : IFilter where T : class, IFilterContext

The IFilter implementations really don't do anything on their own other than pass a IFilterContext which exposes various properties of the request. Each of the IFilter implementations provide a single constructor which takes a Predicate where T is an IFilterContext that RequestReduce populates and provides to the predicate delegate.

There are three different implementations of IFilterContext available:

public class CssJsFilterContext : IFilterContext{    public HttpRequestBase HttpRequest { private set; get; }    public string FilteredUrl { private set; get; } //The Css or JS url to be processed    public string FilteredTag { private set; get; } //The entire HTML tag being processed}

public class PageFilterContext : IFilterContext{    public HttpRequestBase HttpRequest { private set; get; }}

public class SpriteFilterContext : IFilterContext{    //BackgroundImageClass has several properties that include all the CSS background attributes    public BackgroundImageClass BackgroundImage { private set; get; } }

Depending on the type of filter (Page, css, javascript or sprite) for each item that RequestReduce processes, it will provide the appropriate IFilterContext to your predicate. If you return true, RequestReduce will eliminate that resource from being processed. Here is an example of how to have RequestReduce ignore any image over 100px X 100px.

RequestReduce.Api.Registry.AddFilter(new SpriteFilter(x => x.Width > 100 && x.Height > 100));
 
Make sure to add these filters inside your App start or some other pre application, one time execution block.

Microsoft’s MSDN and Technet Forums and Search adopt RequestReduce for Web Performance Optimization by Matt Wrock

This week RequestReduce was launched on Microsoft’s MSDN and Technet web sites specifically with the Forums and Search applications. So if you ever land on a Forums page within these sites or conduct a search within the MSDN or Technet environment, RequestReduce is crunching and merging the CSS and javascript as well as spriting their background images resulting in about a 25-30% Reduction in the number of HTTP requests made.. RequestReduce already services the MSDN and Technet gallery sites which include Code Samples and the Visual Studio Gallery. All of these sites combined represent several million page views a day and prove that RequestReduce can perform under considerable load.

This migration process did surface some “edge case” bugs in the Sql Server cache flushing logic that would likely only surface in high traffic environments. The Sql Server cache is an add on assembly that is aimed to synchronize the RequestReduce generated Scripts, CSS and images across multiple servers and ensure that users do not view different or broken content when moving from one server to another but rather guarantees a consistent user experience throughout a session.

The discovery of these bugs provided the bulk of the code that went into this week’s 1.4 Release of RequestReduce. The release was largely a fix and polish release but here are some highlights:

Dashboard Page

Troubeshooting issues in the Forums/Search migration inspired me to create a dashboard page to assist in catching and isolating some tough to catch bugs. The page is currently extremely bare bones and rather cryptic. It displays what css and javascript has been processed, what is queued for processing and what is currently in the middle of being processed. It also provides links for flushing individual merged css/js files from the cache. The page can be accessed from:

http://<your site>/<RequestReduce virtual directory>/dashboard

Its greatest benefit is to assist in determining what RequestReduce is currently up to if anything and where RequestReduce is in terms of processing all queued content. Expect this page to get much more informative and maybe prettier in coming releases.

Performance Optimization

The javascript processing introduced in release version 1.3, added a lot of work to the Response Filter which has to iterate over every character of the response stream to the browser. This is the single piece of code that needs the most perf optimization since it is the only request blocking activity within RequestReduce. Its speed has a direct impact on the Time to First Byte download of the hosting RequestReduce enabled web page.

So I changed some List<T> implementations to arrays and removed some unnecessary lines of code. Granted these are micro optimizations but even 50 – 100ms of time filtering a page is far too long and this process has to be extremely optimized. Its also a bear to read, write and change which I plan to address in a future release. Sure I could eliminate several lines of code and overly complicated logic by using regular expressions, but that would actually degrade performance significantly.

Speaking of regular expressions. I do make extensive use of them in the background processing of content. Especially in CSS parsing for sprites and imports. So I eliminated a lot of my static Regex instances and replaced them with instance member Regex’s in a singleton class. .net will cache the most recently used 15 static regular expressions in an application and I do not want to infringe upon that cache especially when I have about 8.Using a singleton allows me to manage my own cache.

Support for CSS @Media

CSS media types is something that can be applied to an entire CSS stylesheet, instructing the browser to use its styles only for specific media. For example: print and screen are the most popular and obvious. RequestReduce had been oblivious to the Media attribute of the Link tag and @import directive. This caused some CSS to break on certain sites. Especially if the last CSS on a page was targeted for Print media and strips out a lot of styling. This was resulting in the web experience having the styles striped out as well. No longer. RequestReduce now wraps the contents of any <Link/> or @import inside @media braces to preserve the media constraints they specify.

Support for ScriptResource.axd scripts and all other Compressed content

RequestReduce already supported the minification and merging of WebResource.axd files. However, RequestReduce always assumed that the content it was requesting was uncompressed text since it did not specify compression in its Accept-Encoding request headers. Well it ends up that ScriptResource.axd files violate HTTP by sending GZIP’d content regardless of the value in this header. This is a show stopper for an upcoming large RequestReduce adoption in the near future that includes pages with over 30 to 40 CSS and script files including tons of ScriptResource.axd files. RequestReduce now looks to see if a Response is compressed with gzip or deflate and uncompresses accordingly. RequestReduce leaves the recompression up to IIS.

This really makes RequestReduce an attractive solution for sites mired in asp.net 2.0/asp.net ajax constructs that had good intentions of encapsulating scripts but horrible perf implications.

Released: RequestReduce 1.3 Now Runs on .net 3.5 and a Preview of Support for Third Party Javascripts in your Minification and Merge Process by Matt Wrock

Welcome .net 3.5 Users!

Earlier (MUCH earlier) this morning I released version 1.3 of RequestReduce. While not as exciting as the previous javascript crunching and merging release, I am very happy that a broader base of users will now have access to this resource. It can now be installed on any server running .net 3.5. Huge thanks to my co worker Matt Hawley (@matthawley) for this pull request.

As part of this effort, Matt ripped out the Sql synchronization code and moved it into a new assembly RequestReduce.SqlServer which is now available as a separate nuget package. This code uses EntityFramework Code First which is limited to .net4.0. So if you need this functionality (most do not), you must be running .net4 (I’m also open to pull requests using a different data access strategy – not at all married to EF). If you are interested in learning more about synchronizing RequestReduce across multiple servers using SqlServer, see this wiki.

What else is in this release?

  1. Fixed javascript inside IE conditional comments. RequestReduce will ignore these just as it does with CSS links inside conditional comments
  2. Added a configuration setting to turn off image spriting: imageSpritingDisabled
  3. Added some optimizations to the Response filter

Troubleshooting Guide

In addition to these features, I have added a troubleshooting wiki to the github site. I have gotten a couple issues reported from people who were unable to get RequestReduce running at first due to issues in their environment. This guide should allow most users to at least begin troubleshooting their situation. However, if for any reason you find yourself spending a lot of time investigating this, please do not hesitate to file an issue and I will get on it usually within 24 hours. If most cannot successfully use RequestReduce from a typical nuget install, I have failed.

What’s Next?

The two key features up next for RequestReduce are:

  1. Support for the CSS Media attribute. RequestReduce has no awareness of this attribute and ignores it. While most do not use this, it can have breaking consequences right now with RequestReduce.
  2. Automatic Content refreshing. This is a feature I’m particularly excited about because it will not only make changing content refresh more quickly, it will allow users to have many third party scripts included in the crunch and merge process. Right now RequestReduce ignores many third party scripts that have expiring headers. With this feature, RequestReduce will occasionally send a head request to the original script in the background to see if content has changed, if it has, RequestReduce will refetch and reduce the content.

This Release (1.3) is now available at http://www.RequestReduce.com and on Nuget. Don’t forget: if there are features in RequestReduce you would like to see added, please suggest them on the github issues page. I received two requests last week and both were added to this release.