HTTP-log comes from a HTTP-server

During the last 10 years, many people has chosen to try to follow user behaviour on their website, through external JavaScript-based logging tools. In the beginning there were many different solutions from newcommers and established companies within the log analysis market, but now it seems as if Google Analytics has won most of this market, propably because of the pricing model which apparently is free but not quite transparent.

Google Analytics and other external JavaScript-based logging tools for analysing web traffic gives great value to marketing people, and often shows more about user behaviour than a log file from your web server (or the log fil of your hosting providers web server) can tell. The added information can be stuff like tracking the mouse (eg. which links did the user almost click on etc.). This info gives a lot of value from a marketing perspective.

But you shouldn't miss the chance of using the real log file of the web server, because it also tells a lot that the external tools mentioned above doesn't tell.

The external JavaScript-based logging tools only logs information when a HTML-page with no JavaScript errors is being rendered in the browser. So errors and deep links directly to pictures, PDF's and other kinds of files aren't being logged.

The web servers own log file will tell you all about errors and hits to files. So if you have a lot of traffic to a specific PDF-file or an old webpage which doesn't exist any longer, this log file will give you this valuable knowledge.

At Semaphor we are using our own tool Web Log Consolidator to consolidate all log information from different servers, which is relevant for a specific customer. After assembling a log file for the customer, we run this file through AWFFull (a Webalizer fork) to generate a report for the customer. There are lots of other backend webserver log analysis tools you can use. 10 years ago the most well known and widely used commercial tool was WebTrends.

We'll be happy to help you getting a grip of this valuable data.

- Tobias