Since my application was developed in Perl and LWP-Simple, the caching options was not to many. Using Varnish caches was little to extravagant for this solution, so my eyes focused on Apaches own mod_cache module.

Mod_cache can handle both memory- and disk caching.

mod_mem_cache

With simplicity in mind, I started to implement the memory solution, mod_mem_cache. The amount of different pages needed simultaneous in the cache was not bigger than it would be achieved by allocating roughly 10 MB for this purpose.

The setup was quite easy (read about it here), but it has one serious flaw – it does not support caching of compressed files. I’m not sure why, I think it has something to do with the order Apache handles these modules.

mod_disk_cache

Since I don’t consider disabling compression an option (see why), the next module to try out was the mod_disk_cache. This module caches all text files in a specified directory, valid for the time set in the file’s header expire time. Here is the setup I ended up using:

# your Apache httpd.conf
<IfModule mod_cache.c>
  <IfModule mod_disk_cache.c>
  CacheRoot /tmp/apache-cache
  CacheEnable disk /
  CacheDirLevels 2
  CacheDirLength 1
  CacheIgnoreCacheControl On
  CacheIgnoreNoLastMod On
  CacheIgnoreHeaders Accept Accept-Language Accept-Charset \
     Cache-Control User-Agent Cookie Host Referer
  CacheDisable /images
  CacheDisable /stylesheets
  CacheDisable /javascripts
  CacheDefaultExpire 300
  </IfModule>
</IfModule>

mod_disk_cache config explained

• The CacheRoot needs to be writeable for your Apache user.
• In order to ignore all requests from browsers to not load cached files, I set CacheIgnoreCacheControl On
• CacheIgnoreHeaders lists all header entities that you want to remove from the cached file to be able to serve the same cached version to many user agents (browser types). The cached file’s header information contain these parameters if you don’t list them here, and one difference in those headers results in a new cached version of the same file.
• CacheDisable specifies directories not intended for caching. All flat files and images are practically served the same way from where they are, this cache function are mainly for dynamically generated content. Like my Perl XML-parsing pages

For further information, please read Apache’s highly informative documentation on mod_caching.

I am now happily serving rather sluggish Perl-parsing-scripts blazingly fast from this simple caching mechanism!

Fishing at Lake Høvringen, Rondane, Norway

I have during the last weeks upgraded this service to also consist of a iPhone-ified version. And since the iPhone functions require some hefty CSS and Javascript-libraries, bandwidth-optimalization comes handy.

Mod_deflate

I proxy my Rails-application through Apache2 and that’s where this gzip-compression-trick called mod_deflate applies. You have to compile your Apache2-installation with --enable-deflate. When that is done, you could just insert this snippet inside your virtual domain space in httpd.conf.

<Location />
  SetOutputFilter DEFLATE
  SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$  \
		no-gzip dont-vary
  SetEnvIfNoCase Request_URI \.pdf$ no-gzip dont-vary
  SetEnvIfNoCase Request_URI \.(?:exe|t?gz|zip|bz2|sit|rar)$ \
    no-gzip dont-vary
</Location>

Result in bandwidth usage

Before applying mod_deflate the front page on MobilQuiz clocked in on 33 KB

After applying mod_deflate the front page only weights 17 KB

That’s a 48,5% reduction. Not bad, huh?

photo credit: CAVE CANEM

Here is my Apache rewrite rule that I use to

1. check if its an iPhone, redirect to optimized version
2. Identify the most common browsers by User Agent and redirect them to web version
3. redirect all others to mobile version

Rewrite rule

ServerName www.mydomain.no

# This enables rewriting in this directory
RewriteEngine On

# Catch iPhone-users first, easiest to discover
RewriteCond %{HTTP_USER_AGENT} .*Mobile.*Safari
RewriteRule ^[\./](.*)$ http://beta.mydomain.no/iphone  [L]

# Catch most familiar web browsers and redirect to web version,
# except Opera Mini and SymbianOS (which identifies itself as Safari)
RewriteCond %{HTTP_USER_AGENT} ((.*MSIE.*Windows\ NT.*)| \
  (Lynx.*)|(.*Safari.*)|(.*Opera.*)|(.*Firefox.*)|(.*Konqueror.*))
RewriteCond %{HTTP_USER_AGENT} !(.*Opera\ Mini.*)
RewriteCond %{HTTP_USER_AGENT} !(.*SymbianOS.*)
RewriteRule ^[\./](.*)$ http://beta.mydomain.no/web  [L]

# Browsers that match neither block, such as regular screen
# browsers, could be caught by a final rewrite rule placed here,
# or we could leave it out and have nothing happen to the
# requested URL. This is the default.

RewriteRule ^[\./](.*)$ http://beta.mydomain.no/mobile  [L]

Notice that it doesn’t sniff for “iPhone”, but “Mobile Safari”. This makes also iPod Touch compatible, which currently is the only device using the same browser.

Alternative rewrite rule

This is the old rewrite rule that I used. This should only be used as a starting grid for further development. The WAP-test is not optimal, seems like Nokia’s and Sony Ericsson’s latest browsers get identified as ordinary web browsers without WAP-support and as a result gets redirected to the web version.

This set of rules where:

1. check if its an iPhone, redirect to optimized version
2. check if it doesn’t support WAP, redirect to web version
3. redirect all others to mobile version

ServerName mydomain.no

# This enables rewriting in this directory
RewriteEngine On

# Catch iPhone-users first, easiest to discover
RewriteCond %{HTTP_USER_AGENT} .*Mobile.*Safari
RewriteRule ^[\./](.*)$ http://beta.mydomain.no/iphone  [L]

# Catch XHTML MP browsers first and return XHTML MP content to them
RewriteCond %{HTTP_ACCEPT} !text/vnd\.wap\.wml
RewriteRule ^[\./](.*)$ http://beta.mydomain.no/web  [L]

# Browsers that match neither block, such as regular screen
# browsers, could be caught by a final rewrite rule placed here,
# or we could leave it out and have nothing happen to the
# requested URL. This is the default.

RewriteRule ^[\./](.*)$ http://beta.mydomain.no/mobile  [L]

Please leave a comment if you see or have a smarter way of doing this.

I have been looking for a clean and simple CRM solution for my web site for some time. I use this web site more like a sketch block where I ramble down some of my findings and howto’s mainly for my self. To simplify this process, I wanted to convert my old, file based PHP-site to something a bit more dynamic and easily maintainable. I didn’t do a deep analysis of what’s available, but I quickly browsed through some alternatives (Drupal, Expression Engine, Movable Type and Wordpress). To make a long story short; I went for Wordpress because of earlier experience with this tool.

It was quite easy to cut’n paste all my content into its “pages” system. The problem arised when I needed to figure out how i could redirect all my .html and .php filenames to folder names that all new systems base their URL structure on.

photo credit: MR+G

Most of the content are from 2000-2003 and all of it are comprehensively crawled by Google and linked up by others. So breaking those links was not an option. Hacking Wordpress to make it possible to use .php and .html endings was not either to smart, both because it was too time consuming and also that I didn’t wanted to alter Wordpress and make it harder for me to future upgrades.

Solution: Rewrite-rules in .htaccess files

And that’s really why I wrote this article – it wasn’t too much information about this on the web. This is what I wanted to do:

Permantent redirect (301) /linux/mutt.php to /linux/mutt/
and remove all index.php so /linux/index.php becomes /linux/

My .htaccess file

RewriteEngine On
RewriteBase /

# Removes index.php-pages
RewriteCond %{THE_REQUEST} \/index\.php(.*)\ HTTP/ [NC]
RewriteRule ^([^.]*)index\.php(.*)$ http://webdirect.no/$1 [R=301,L]

# Externally redirect direct client requests for .php files to non-.php URLs
RewriteCond %{THE_REQUEST} ^GET\ /([^/]+/){1,10}[^.]+\.php(\?[^\ ]*)?\ HTTP/
RewriteRule ^(([^/]+/)*[^.]+)\.php$ http://webdirect.no/$1 [R=301,L] 

# Externally redirect direct client requests for .php files to non-.php URLs
RewriteCond %{THE_REQUEST} ^GET\ /([^/]+/){1,10}[^.]+\.html(\?[^\ ]*)?\ HTTP/
RewriteRule ^(([^/]+/)*[^.]+)\.html$ http://webdirect.no/$1 [R=301,L]

# standard wordpress rewrite
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]

Rewrite rules explained

# Removes index.php-pages
RewriteCond %{THE_REQUEST} \/index\.php(.*)\ HTTP/ [NC]
RewriteRule ^([^.]*)index\.php(.*)$ http://webdirect.no/$1 [R=301,L]

This one just removes all /index.php from requests and creates a permanent redirect (HTTP 301) so crawles can detect that this page has moved to a new URL

# Externally redirect direct client requests for .php files to non-.php URLs
RewriteCond %{THE_REQUEST} ^GET\ /([^/]+/){1,10}[^.]+\.php(\?[^\ ]*)?\ HTTP/
RewriteRule ^(([^/]+/)*[^.]+)\.php$ http://webdirect.no/$1 [R=301,L]

Removes .php extensions if the depth of the URL strucure are between 3 and 10 slashes. This to allow some root files to have .php ending, like /login.php for example.

So, what about .php files from Wordpress?

Yes, that’s right. Wordpress uses .php-extention on many internal files that’s crucial to make the dog fly, and since rewrite-rules in .htaccess files are inhereted, we have to turn off rewriting in some folders (/wp-admin).

This .htaccess file turns of all rewrite rules so Wordpress can function like it’s meant to in this folder.

.htaccess file inside /wp-admin

<IfModule mod_rewrite.c>
RewriteEngine Off
</IfModule>

As always, if you know a smarter way to do this, leave a comment, please!