vADC Docs

Adding meta-tags to a website with Stingray Traffic Manager

by sphillips on ‎02-21-2013 02:23 AM - edited on ‎07-07-2015 02:43 PM by PaulWallace (1,205 Views)

baldy_fat_geek2.pngMeta-tags and the meta-description are used by search engines and other tools to infer more information about a website, and their judicious and responsible use can have a positive effect on a page's ranking in search engine results.  Suffice to say, a page without any 'meta' information is likely to score lower than the same page with some appropriate information.

 

This article (originally published December 2006) describes how you can automatically infer and insert meta tags into a web page, on the fly.

 

Rewriting a response

 

First, decide what to use to generate a list of related keywords.

 

It would have been nice to have been able to slurp up all the text on the page and calculate the most commonly occurring unusual words. Surely that would have been the über-geek thing to do? Well, not really: unless I was careful I could end-up slowing down each response, and  there would be the danger that I produced a strange list of keywords that didn’t accurately represent what the page is trying to say (and could also be widely “Off-message”).

 

So I instead turned to three sources of on-message page summaries - the title tag, the contents of the big h1 tag and the elements of the page path.

 

The script

 

First I had to get the response body:

 

$body = http.getResponseBody();

 

This will be grepped for keywords, mangled to add the meta-tags and then returned by setting the response body:

 

http.setResponseBody( $body );

 

Next I had to make a list of keywords. As I mentioned before, my first plan was to look at the path: by converting slashes to commas I should be able to generate some correct keywords, something like this:

 

$path = http.getPath();
$path = string.regexsub( $path, "/+", "; ","g" );

 

After adding a few lines to first tidy-up the path: removing slashes at the beginning and end; and replacing underscores with spaces, it worked pretty well.

 

And, for solely aesthetic reasons I added

 

$path = string.uppercase($path);

 

Then, I took a look at the title tag. Something like this did the trick:

 

if( string.regexmatch( $body, "<title>\\s*(.*?)\\s*</title>", "i" ) ) {
$title_tag_text = $1;
}

 

(the “i” flag here makes the search case-insensitive, just in-case).

 

This, indeed, worked fine. With a little cleaning-up, I was able to generate a meta-description similarly: I just stuck them together after adding some punctuation (solely to make it nicer when read: search engines often return the meta-description in the search result).

 

After playing with this for a while I wasn’t completely satisfied with the results: the meta-keywords looked great; but the meta-description was a little lacking in the real english department.

 

So, instead I turned my attention to the h1 tag on each page: it should already be a mini-description of each page. I grepped it in a similar fashion to the title tag and the generated description looked vastly improved.

 

Lastly, I added some code to check if a page already has a meta-description or meta-keywords to prevent the automatic tags being inserted in this case. This allows us to gradually add meta-tags by hand to our pages - and it means we always have a backup should we forget to add metas to a new page in the future.

 

The finished script looked like this:

 

# Only process HTML responses  
$ct = http.getResponseHeader( "Content-Type" );  
if( ! string.startsWith( $ct, "text/html" ) ) break;  
  
  
$body = http.getResponseBody();  
$path = http.getPath();  
  
  
# remove the first and last slashes; convert remaining slashes  
$path = string.regexsub( $path, "^/?(.*?)/?$", "$1" );  
$path = string.replaceAll( $path, "_", " " );  
$path = string.replaceAll( $path, "/", ", " );  
  
if( string.regexmatch( $body, "<h1.*?>\\s*(.*?)\\s*</h1>", "i" ) ) {  
   $h1 = $1; $h1 = string.regexsub( $h1, "<.*?>", "", "g" );  
}  
  
if( string.regexmatch( $body, "<title>\\s*(.*?)\\s*</title>", "i" ) ) {  
   $title = $1; $title = string.regexsub( $title, "<.*?>", "", "g" );  
}  
  
if( $h1 ) {  
   $description = "Riverbed - " . $h1 . ": " . $title;  
   $keywords = "Riverbed, " . $path . ", " . $h1 . ", " . $title;  
} else {  
   $description = "Riverbed - " . $path . ": " . $title;  
   $keywords = "Riverbed, " . $path . ", " . $title;  
}  
  
# only rewrite the meta-keywords if we don't already have some  
if(! string.regexmatch( $body, "<meta\\s+name='keywords'", "i" ) ) {  
   $meta_keywords = "    <meta name='keywords' content='" . $keywords ."'/>\n";  
}  
  
# only rewrite the meta-description if we don't already have one  
if(! string.regexmatch( $body, "<meta\\s+name='description'", "i" ) ) {  
   $meta_description = "    <meta name='description' content='" . $description . "'/>";  
}  
  
# find the title and stick the new meta tags in afterwards  
if( $meta_keywords || $meta_description ) {  
   $body = string.regexsub( $body,  
      "(<title>.*</title>)", "$1\n" . $meta_keywords . $meta_description );  
   http.setResponseBody( $body );  
}   

 

It should be fairly easy to adapt it to another site assuming the pages are built consistently.

 

This article was originally written by Sam Phillips (his own pic above) in December 2006, modified and tested in February 2013

Contributors