Configuring Caching On The Isa Server

Configuring Caching on the ISA Server

Written by: Yousif Yalda

Copyright 2007-2008

How Caching Works

Copyright 2007-2008

Internet Content

Ram:Cached Content____________

Directory File

Hard Disk:Cached Content

________

Backup Directory File

Client PC

(Vapt-Sec.com)

ISA

Figure 1-0

Continued

The ISA Server is always being a proactive partner in your caching mechanism . We got this PC and it’s going to Vapt-Sec.com. So, it’s going to come up here to ISA Server

because that’s the default gateway. Now the ISA Server will look into it’s cache, and if it’s not there, it’s going to retrieve that from Internet. Now, when it comes back, the ISA Server initially store that content in it’s ram. Cache content is stored in it’s memory. If you are looking for an incredible way to speed up your internet access, definitely load your ISA Server with a bunch of memory. The more memory it has, the more cache

content it can store. When you have cache content being accessed from the ram, it’s access is lightening fast, so the client’s will be amazed at how fast the content is being processed. The ISA Server doesn’t keep all it’s content in memory. It also stores some of it on the hard drive. The ISA Server will move content that’s not being accessed so frequently, out of the ram and into the hard drive. Now, if it sees that content is being

accessed quite a bit, it will move it back into the ram and so can be accessed very fast. Now, we also have this directory file which is stored In memory and it is a listing of every piece of content that ISA Server has in it’s cache. The ISA Server does not want to have

to look into a directory of files every single time, it needs to figure out if something is cache. Instead, it has a very quick access directory file stored in the ram, listing every

single website that is cached in that directory file. Copyright 2007-2008

* Refer to Figure 1-0

Continued

Since it’s in the ram, what happens if the ISA Server reboots? Well the directory file is gone. However, thankfully, we do have a backup directory file sitting on the hard disk that can be copied to the ram when the ISA Server restarts. If you lose that directory file, the

ISA Server will have no idea what it has in cache and this will all eventually be emptied out. When you look in the cache content on the hard drive, the ISA Server does not maintain at all in a big directory web pages. It actually maintains one single file per partition of cache

content. This allows it to do it’s own indexing, it’s own defragmenting of that file, and it keeps that file very efficient, but all of those web pages will be stored in a single file. Now because the ISA Server is keeping track of what content seems to be popular, and it’s

keeping that in the ram, what that leaves open is the feature called Active Caching. What that allows the ISA Server is to proactively update cache content that it can consider

popular without it ever being accessed. Cache content will eventually time out. It has a time to live, it has to either be dumped or refreshed. Now, you don’t want to cache content to reach it’s time and be dumped, especially if it’s a lot of it during your peak hours. Just

before your peak hours, it can go out and update that cache content and make sure it’s time to live, is refreshed right before every one accessed it.

Copyright 2007-2008


Continued

It also has the ability to do content prefetching, which allows you, as the administrator to have control and say “Well, at 7:45 PM I want

you to go and download all the news websites”. Everyone accesses the news as soon as they get into the day. So, you have the ISA Server proactively go out and cache that information at specific times. Imagine this, this client instead of going to Vapt-Sec.com,

goes to Amazon. COM. They purchase their favorite book. They type in their credit card information into a web page, now will that be

cached and stored on the ISA Server, would that be some security holes? The answer is no, most likely. Well firstly, this is known as

HTTP Caching, not HTTPS Caching. Copyright 2007-2008


Continued

Any web page that has security enabled, will not be cached. Most websites that have credit card information able to be stored, will have security of some form, being turned on. Let’s say this security is not turned on. Let’s say it is an HTTP website and they are allowing you

to type in your credit card information. Can we dictate that it’s not cached then? Well you can, and there’s a few ways to do it. First off,

the web administrator out on the Internet, can type into the HTML code some commands that dictate to the ISA Server or any caching server or client because Internet Explorer can cache as well. It can dictate to that client “Do not cache this web page.” This is known as

Cache Control Code in the HTML header. Also, you as an ISA Administrator on the ISA Server can dictate that certain websites

can’t be cached. There are very few security concerns when you are thinking about Caching.

Copyright 2007-2008


ConfiguringFirstly, expand the ISA Management Console

Copyright 2007-2008Figure 1-1

Continued

We also want to expand the ISAServer Console Item. Underneath, we will see the Cache Configuration category. Let’s expand this. To configure the Cache Configuration; the major

settings, you are going to right click on it and go to the Properties button. Inside you will initially see all the storage space that you have set up on this server.

Copyright 2007-2008


ContinuedLet’s click on the HTTP tab in the window open.


Continued

There are many options to select from.

Copyright 2007-2008

+ Frequently (Expire Immediately)

+ Normally

+ Less Frequently (Reduced network traffic is more important)

+ Set Time To Live (TTL)


Continued

If you choose “Frequently”, you notice all fields are not able to modified. Overall, “Frequently” will expire your cache content as it is downloading unless the source specifies an expiration date. Here’s

the idea. As a website administrator, you could code what the expiration date of your content is. For example, maybe you have a website that you set up and you change it once a week. Well, you

could code in the HTML header that this will expire a week from; and the date you have in there as well. Not many website administrators do that, so if you choose “Frequently”, expect that your caching will

be kept at a minimum.

Copyright 2007-2008


Continued

If you choose “Normally”; checked by default, allows you to have HTTP Caching and a balanced setting. You will retrieve some from the internet, however some of it you will retrieve from the cache.

Copyright 2007-2008


Continued

Choosing “Set Time To Live (SSL)” means cache stored on your ISA Server will be stored 20% of the age since it was created. So, if it was created 100 days ago, it will store maximum of 20 days, or in

this case, no more than day. It will store the percentage since it was created, and so either 20 days if it was created 100 days ago, or the

items over rule. No less than 15 minutes will that ever be in the cache, and no more than 1 day that it will ever be in the cache.

Copyright 2007-2008


Continued

If you choose “Less Frequently”, the field goes up to no less than 30 minutes or no more than 2 days. Most administrators will choose to

select the “Normally” setting for a balanced configuration.

Copyright 2007-2008


Continued

If you choose “Less Frequently”, the field goes up to no less than 30 minutes or no more than 2 days. Most administrators will choose to

select the “Normally” setting for a balanced configuration.

Copyright 2007-2008

Continued

Let’s go on to the FTP Tab in the open console.

Copyright 2007-2008

FTP

Figure 1-3

Continued

Enabling FTP Caching does allow us to cache FTP objects. These are files you download from an FTP site. By default, the time lived for all FTP objects is going to be 1440 minutes. Divided by 60, that is 24 hours! So, FTP objects will be cached by default for 24 hours. In my suggestion, it would be highly recommended to crank that up to 48

hours or beyond. The reason why, is FTP objects, hardly ever change. These are .ZIP files and if the file name stays the same,

most likely the object will stay the same. You could even give this to the “weeks” if you wanted to, and it would still be safe unless you

found that people were downloading content or FTP files that were constantly changing.

Copyright 2007-2008


Continued

Let’s go on to the Active Caching Tab in the open console.

Copyright 2007-2008

Active Caching

Figure 1-4

Continued

The configuration of Active Caching is a little odd. First off, Active Caching updates files that are stored in the cache that are about to expire. So, if it’s a Frequently accessed file, you do not have to wait for it to expire, a client to access it, re-downloaded. ISA Server can recognize it and update it on it’s own and take care of it when the bandwidth requirements are fairly low. The reason I say it’s a little odd is because you check the box to enable it, and you have three

options; frequently, normally, and less frequently. Overall, Frequently is going to be the top performance for the client because items in the cache will most often be up to date. Less frequently will mean that it won’t update them as often. Some of the things may time out but the

very popular web pages that are stored will be updated.

Copyright 2007-2008


Continued

This is why it is a little odd. As a network administrator, what does frequently mean to me? What does less frequently mean to me? I

want time, I want algorithm, I want something to by. Unfortunately the for Active Caching is completely undocumented by Microsoft.

However, I’ve done a little research and what I found out is that it works like SETI. It’s a program that stands for the Search for extra

Terrestrial Intelligence. It’s a Screen-Saver that installs on your computer that when your process cycles are fairly low, it goes out

and downloads extra terrestrial files on the internet and tries to find them in the Galaxy. Well, this works in a similar fashion. When your processor cycles are low on the ISA Server, it assumes that network activity is low. So, that is when it will go out and actively cache the files. When it’s Frequently, it will update as many of them as it can while the processors cycles are low. Less frequently will go out and

wait a longer time before it actually starts updating.

Copyright 2007-2008


Continued

Let’s go on to the Advanced Tab in the open console.

Copyright 2007-2008

Advanced

Figure 1-5

Continued

Let’s go on the Advanced tab and you’ll see just how chewable the ISA Server really is! These are all the tweaking options of the cache that you can get in and find out what the caching will really be doing.

Firstly, “Do not cache objects larger than”; if you want to limit how large an item in your cache can really be, you can check this box and

specify size. Most of the time you won’t have to, so overall the default is appropriately set. You can limit the size to say, you have a lot of people downloading big pictures, so you might say 500 KB is

the limit of an object in the cache. You can go beyond the KB range, but I have yet to seen a web page that uses the limit in the GB range,

but it could happen.

Copyright 2007-2008


Continued

We also have “Cache objects that have an unspecified last modification time” and that is checked by the default. Now if you can

remember the first tab HTTP (Figure 1-2), one of things it uses to calculate how long the time to live of any object in the cache would be, is the last modification time. Some pages may not have a last modification time. If you uncheck this box, it will not cache those.

However, most of the time it’s safe to cache those and it’s especially safe to cache those because normally, those don’t change.

Copyright 2007-2008


Continued

“Cache objects even if they do not have an HTTP status code of 200”; now that’s a cryptic one! Overall, HTTP status code 200 is OK. If you access a website and everything downloads “OK”, you get a status code of 200. What it is saying, is that cache objects that even don’t come out as OK, this is a symptom known as negative caching. You

ever go into a website where the server is down? And your sitting there and it’s loading forever? And finally comes back and says “Page Not Found” or “Server is Unavailable” or a message familiar. Well, if you keep this checked, ISA Server will those web pages so that the next

time someone accesses that website, they won’t get that loading forever problem. Instead, it’ll be an instantaneous “Page not found,

server is down”. That’s good most of the time because usually when something’s down, it will be down for a while.

Copyright 2007-2008


Continued

“Cache dynamic content (objects with question marks in the URL”; dynamic content are just pages that are generated

dynamically, such as when you go to MSN.com, it let’s you customize it by typing in your zip code so you can get your local weather, news, and you can keep even type in your own stock

quotes that you want to get on that web page. Now that is considered a dynamic page because it’s generated just for you.

Now, I would say in less than 3% of the cases, caching dynamic content is useful because it’s hardly ever re-used. It’s always changing, always being modified depending on what

web browsers accessing it and the time of the day. So, it would be a rare occasion that you would actually choose Cache

dynamic content. Most of the time the default of leaving this checked is just fine.

Copyright 2007-2008


Continued

“Maximum size of URL cached in memory (bytes) ”; this is how much an individual URL can store in memory. For instance, if I

go to Yahoo.com, you remember before that the cache is stored on the memory and the hard drive. Well, by default, it will

only store about 12.8 Kilo Bytes in the memory. I would say nowadays it’s safe to increase that with something like 128 Kilo Bytes. The reason I say that is because web pages have grown

so much, even in the last couple of years dynamic content, large graphics, and high speed internet connection has really

cranked up the size of these web pages. So, this is the maximum size an individual URL can store in the memory. I

would say it’s safe to keep it around 128 Kilo Bytes as long as you have a decent amount of memory in the ISA Server. When

I say “decent”, I mean a Giga Byte or more for memory.

Copyright 2007-2008


Continued

“If Web site of expired object cannot be reached:” , do you A, not return the objects, or B, return an expired object? Here’s

what this means. Let’s say you went to Yahoo.com and it cached that information, and then Yahoo’s server went down. They are inaccessible, you could still get the Yahoo web page from the cache, but if the time to live expires, is it still valid?

Well, by default it will be. It will be for less than the percentage mentioned in the previous figure display. This get’s really

confusing so I’ll explain. Let’s say the time to live on the Yahoo web page 12 hours. Well, it went down in 12 hours later, the cache expired. Someone then, went to access it. With this

setting, it’s going to keep it 50% longer than the time to live. So, in essence, an additional 6 hours on top of the 12 it was

originally in there.

Copyright 2007-2008


Continued

However, it’s limited by the “But more than (minutes)” option. In this case, 60 minutes by default. Overall, we could have

returned it up to 6 hours, which is 50% the time to live, but we’re going to limit it down to 60 minutes because that’s going

to tie it down to no more than 60 minutes.

Copyright 2007-2008


Continued

Finally, we have the “Percentage of free memory to use for caching:” In this case, up to 50% of your free memory will be

used to store cache files. If you open Performance Monitor, and you are tracking your memory and noticing that it never really

get’s above a certain amount, then it would be safe to increase this. However, if you are noticing that in performance monitor, all your physical memory is being used up and it’s starting to

use the page file, I would almost say to decrease this because it’s hurting your server performance.

Copyright 2007-2008


Continued

These are the advanced options that you can apply to make ISA Server cache exactly how you want it to.

Copyright 2007-2008


Change your cache configuration based

on your storage space


Continued

Click on the “Drives” and if you want to change what your cache size or what it’s using to cache, you just

double-click the ISA Server.

Copyright 2007-2008


Figure 2-1

Continued

You remember this screen, this is going to list all the drives you have in your ISA Server, how much free space they have, and much how cache size they have. So, if you want to increase it, just type in a new number and click “Set” and you are done! It will ask you to restart the services, which will take place before the caching will update and use that extra storage space. As

long as people aren’t currently active, using the internet, it’s OK to do that.

Copyright 2007-2008


Scheduled Content Download


Continued

Right-click on the “Scheduled Content Download Jobs” menu function and hit “New” and finally hit “Job”. This allows you as the administrator to specify exactly what will be downloaded

from the internet and when. This will prompt you with a wizard. For the job name, I will type “Cache VaptSec” and hit “Next”. From there, it’s going to tell you, what is going to be the start

date to begin downloading. Now, if this is a one-time scheduled content job, you can type in that date and time now. Most likely,

this will be a reoccurring thing. Otherwise, you probably wouldn’t create a whole scheduled job just for a one-time

download.

Copyright 2007-2008


Continued

We’ll adjust the date to 7/7/2007 and the time 6:26:08 PM and hit “Next”. From there, it’s going to prompt you for whether you

want to download the content just once on that date, daily, every single day at that time, or do you want to download it every week on your chosen days. You will specify when and where you download this content, and hit “Next”. Now again,

you’ll have to do some planning for this because this is going to require some analysis to realize that “Hey, every body in my company seems to access Vapt-Sec at Six O’ clock every

evening.” So, this is going to take a little analysis to know what is being accessed when.

Copyright 2007-2008


Continued

This will ask you what URL you wish to download. In this case, we will go ahead and type in http://www.vapt-sec.com. First off, I almost always check off “Content only from URL domain (not sites to which it links)”. Overall, if you do not check this, it will

download vapt-sec.com and any web site vapt-sec.com links to. Now, maybe Vapt-Sec has links to Microsoft.com and

Cisco.com for extra research in evaluation statistics in security audits, can you imagine the impact on your memory and your hard drive space if you decided to download the entire Cisco and Microsoft web site? I couldn’t imagine it. That is literally

Gigs and Gigs of data that you would probably run out of space, so I would definitely make sure you would check that option off.

Copyright 2007-2008


Continued

The 2nd option given is if you want to cache dynamic content from vapt-sec.com. These override your default cache settings

that we just specified. So, if you want to say that Dynamic content for this web site is OK, then you can freely do that, and

hit “Next”.

Copyright 2007-2008


Continued

Download Content Configuration Settings Wizard


Continued

You have a few more options. We can override the time to live settings that we specified as well. First off, we have the check

box “Always override object’s TTL”. So if Vapt-sec is specifying time to live on their web pages, you can override that with your

own custom value down there. We can say all the Vapt-Sec content will have a time to live of about 600 minutes and we’ll override that. We can also override the time to live if it’s not

defined by Vapt-Sec, so that’s our option to override the time to live settings.

Copyright 2007-2008


Continued

We also have the Links Depth, again, something I would highly recommend that you use, especially for large web sites. This is

how deep it’s going to dig in the web site. If I were to click around in every link and every web page inside in Vapt-Sec.com, by default, every single one of them would be

cached. There is no maximum depth. If we check the “Cache up to maximum links depth of” option, we get to say the number of levels that we want to go into and when I say levels, think of the web page, when you first arrived there, you have the web page and an initial set of links. You click that set of links and

you go to another level; one level down. Then, you go to another level once you click links on that page and so fourth.

You get deeper and deeper into the web site.

Copyright 2007-2008


Continued

So this is how many layers do you want to carve out of that web site and pre-cache. I usually find that if you go more than 2 to 3,

you start getting a lot of content, unless you have a very specific reason for doing so. You can also limit the number of

cached objects. This right here, by default is the maximum number of cached objects that you could specify. You cannot

go any larger than that. Maybe Vapt-Sec has a lot of objects, a lot of graphics, and you only want to cache 10 of them, you can

go ahead and type that in right here. So, again totally customizable for what you want from that web site you are

downloading from.

Copyright 2007-2008


Continued

You hit “Next”. It will give you a summary screen and hit “Finish”. That will store that Scheduled Content Download Job and it’s going to run the first time when we get to July 7, 2007,

6:26 PM downloaded Weekly.

Copyright 2007-2008


Discussed

//Caching Overview\\

Copyright 2007-2008

//Configuring Cache Policy\\

//Configuring Cache Settings\\

//Configuring Scheduled Content Downloads\\

Technology

Configuring Caching On The Isa Server