Who is visiting your Web site? What browsers do they use? Where do they go in the site? What pages do they look at? Your Web server log files contain the answers to these questions and more. Once you start using your server log information, you’ll wonder how you ever got along without it.
WHO GOES THERE?
Think of your server logs as a virtual visitor sign-in sheet. They record where your visitors came from and where they go on your site.
Server log files tell you:
- Which pages get the most traffic – and the least.
- What sites are referring visitors to you.
- Which pages visitors look at.
- What browsers and operating systems are most popular with visitors.
- When search engine spiders and directory editors visit.
This data can often help you flag specific problems on your Web site. If you have a lot of visitors, but few sales, check your server logs to see how many visitors actually see your product offerings. Wondering just how effective that expensive ad campaign really is? Your server logs can tell you how much traffic – and sales – it’s really generating.
You can also use the data to uncover hidden problems on your site. Suppose you find that 20% of your visitors use the WebTV browser. Since sites display much differently in WebTV than in Explorer or Netscape, you’ll probably need to consider optimizing your site for the WebTV browser.
READING YOUR LOG FILES
The server stores visitor information in files with the .log extension – open them as text files if you want to look at the raw log data. Unfortunately, you can’t do much useful analysis with the raw data unless you’re a math whiz. Here is how the log file records a single request to a Web server:
209.240.221.71 – – [03/Jan/2001:15:20:06 -0800] “GET /Inauguration.htm HTTP/1.0” 200 8788 “http://www.democrats.com/” “Mozilla/3.0 WebTV/1.2 (compatible; MSIE 2.0)”
HERE’S WHAT IT ALL MEANS
- -SERVER LOG INFO- -COMPONENT- -NAME MEANING-
209.240.221.71 remotehost Name of the computer requesting the Web page.
– rfc931 The name of the remote user. This field is usually blank.
– authuser Login of the remote user. This is also usually blank.
[03/Jan/2001:15:20:06 date Date and time of the -0800] request.
“GET /Inauguration.htm request URL of the file requested. HTTP/1.0” This is noted exactly as the user requested it.
200 status Error or status code generated by the request.
8788 bytes Size (in bytes) of the document returned to the client.
www.democrats.com/ referrer The URL the visitor came from immediately before they requested the file.
“Mozilla/3.0 WebTV/1.2 agent Records the visitor’s (compatible; MSIE 2.0)” browser and operating system.
The server logs provide a lot of important information: the trick is to put that data into an understandable format. If you have a large ecommerce site, then you probably already have a custom system in place. Smaller commercial or personal sites often rely on the Web site statistics programs supplied by their Web hosts.
LOG FILE ANALYSIS PACKAGES
If your Web host doesn’t offer site statistics – or charges you for them – consider using some of the Web site analysis packages available for download. The CounterGuide.com Web site lists 20 packages with short reviews and product links. Packages vary widely in price and performance. Several like Analog, http Analyze, and RealTracker are free for non-commercial sites.
Analog
CounterGuide.com
http-Analyze
RealTracker
A basic package will organize the log information into several sections. At a minimum, you need to know the following:
Number of computers requesting content: This tells you how many visitors actually came to your site during a certain period – usually 24 hours. It’s a much more useful gauge of site traffic than the total number of hits.
Files requested for download: The number of hits to your site. Each hit represents an individual file sent from the server – that includes image files, CGI scripts, and HTML pages. The log data should be broken out by each individual file name so you can tell exactly how many page views you have.
HTML page requests: The number of page views, which is a count of the number of times each individual Web page was requested. Ideally, you want visitors to look at more than just your home page. Divide the page view count by the number of visitors and you can get a rough estimate of how many pages each visitor actually sees.
Browser and operating system: Shows you which browser and operating system your visitors used. Use this information to decide which (if any) browser or operating system-specific elements it’s safe to include on your site.
Referrer: A record of what page a visitor was at immediately before they arrived at your site. This information helps you determine which search engines send you the most traffic, the effectiveness of ad campaigns, and which links are the most active.
Note that the different software packages often use different terminology, so they may not use these exact terms. But you can easily evaluate them by paying close attention to the product specifications.
IDENTIFY WEB SITE PROBLEMS
Web site problems that completely break a Web page are pretty easy to spot, but others are more difficult to find and correct. Suppose your site gets a large number of unique visitors, but the total number of page views is small.
There could be a single problem – or a combination, such as poor site navigation, page content that isn’t compelling, or design problems that keep visitors from viewing the page properly.
Improve poor site navigation
Fix design problems
NetMechanic’s Web site tools can help you correct many of the problems you find during server log analysis. HTML Toolbox identifies HTML code errors and browser-specific tags that can turn away visitors while Browser Photo shows you actual snapshots of your Web page using 14 different browser and operating system combinations.
Detect HTML errors and more with NetMechanic’s HTML Toolbox
Improve your designs with NetMechanic’s Browser Photo
Use server log data to find problems on your site – then let NetMechanic help you fix them!
Larisa Thomason is Senior Web Analyst, Specializing in Accessibility at http://www.NetMechanic.com a leading developer of online maintenance, monitoring and promotion services that has “tuned up” over 32 million Web pages. She can be reached at larisa.thomason@netmechanic.com. Learn more about NetMechanic tools at www.netmechanic.com.
