Access Logs Most Common Patterns | XpoLog Log Analysis & Viewer System Download

Access Log comprised of two main patterns – Common and Combined. The common pattern presents most common parameters like host name, timestame, user names etc. The Combined pattern includes the common pattern and adds user-agent and an http referer. 

Boosting your customers’ satisfaction, site performance, up time and security stability up starts from constantly understanding your access logs data patterns. In this article we present what common Access logs patterns look like, how to read and understand them and ways XpoLog Access log solution can visualize insights using Apps and Dashboards.

As a website owner and administrator, that considers website uptime, performance and customers’ success as top priorities, XpoLog Access Log Analysis automatically and in real time parses and extracts website’s insights and suggests action items to resolve issues. XpoLog uses advanced Access Log patterns recognition algorithm and uses key indicators such as HTTP Status code, Top URLs, Hits over Time, Top hits per user, Top hits per client IP, Number of users over time, top countries, users GEO IP, Average Bandwidth (MB) Over Time and more.

In a previous post, what is an access log or access log 101, Access logs were introduced whereas in this article we will see what an Access log looks like by analyzing its patterns.

Access Logs Common Patterns | Xpolog Log Analyzer Free Download

 

Two main Access Log patterns

An Access log is often referred to as Raw Data and vast majority of them look very similar and contain similar record patterns whether an e-commerce, security, financial, or media log.

There are two main patterns we see most often, the common and the combined.

(1) The common Access Log pattern

%h %l %u %t “%r” %s %b   where:

  • %h – Remote host name (or IP address if enableLookups for the connector is false)
  • %l – Remote logical username from identd (always returns ‘-‘)
  • %u – Remote user that was authenticated
  • %t – Date and time, in Common Log Format
  • %r – First line of the request
  • %s – HTTP status code of the response
  • %b – Bytes sent, excluding HTTP headers, or ‘-‘ if no bytes were sent

(2) The combined Access Log pattern (begins like the common pattern) is:

%h %l %u %t “%r” %s %b “%{Referer}i” “%{User-Agent}i”

Here is a real life example of a combined access log pattern:

[Client IP] 127.0.0.1 |  [Remote Logical Username] – |  [Remote User] – |  [Date] 12/Dec/2016:13:46:02 ET |  [Method] GET |  [URL] xpolog.com/download?moduleId=access-log-blog-article-post-2  |  [Protocol] HTTP/1.1 |  [Status] 200 |  [Bytes Sent] 4168 | [Referer] http://localhost:30303/logeye/slides.jsp |  [User-Agent] Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.85 Safari/537.36

 

Analyzing Big Raw Data using Access Logs common patterns

Now imagine millions of rows being generated daily into hundreds of files across multiple servers. Tasks of viewing, parsing (machine), searching (queries), having alerts, looking at dashboards, having insights and doing it all over again continuously.

XpoLog Access logs Analysis solution analyzes tera bytes of data in real time and presents live customizable dashboards of the most relevant insights and action items.

 Free XpoLog Access Log Analyzer 

Status code examples

The HTTP status codes aforementioned has a 200 response code. A quick w3c lookup will let us know that an HTTP status code of the kind 2xx means that the client’s request was successfully received.

Continuing down the w3c list, we see that an HTTP status code of type 3xx means that the user agent needs to take further action.  An HTTP status code of type 4xx means the client has erred (the most famous one being 404 – Page Not Found). I doubt there are any internet surfers out there who have not stumbled across this status code. An HTTP status code of type 5xx means the server has erred. For a complete list, have a look at these status code definitions I have gathered for you.

Access Logs Common Patterns | Xpolog Log Analyzer Free Download

 

What useful insights are hidden inside Access logs?

What kind of information can be extracted from all these access logs? Remote host name/IP address, remote logical username, timestamp (date and time), HTTP status code of response, bytes sent, etc. are some examples.

Access Logs are created continuously and record every visit and use of a resource by the servers thus the sheer amount of access logs being created makes it impossible for humans to  read through let along analyze them or extract insights from them.

There is an abundant number of applications out there that parse and search access logs and gather information about them. But the real challenge is to understand Access Logs raw data and take actions on the information gathered so website admin will increase their customers satisfaction, loyalty and prevent errors and exceptions.

In the Internet-world, access logs can be analyzed to not only show us how many visitors (unique first-time requests) to a specific web page, number of  requests for each page of the site, usage patterns like the time of day, day of the week, and seasonally the page got the highest or lowest number of visits… etc. but how you can take on that information and optimize your site to best handle future customers’ visits and website performance.

That is where XpoLog Automatic Real Time Live Access Log Analysis steps in.

Access Logs Common Patterns | Xpolog Log Analyzer Free Download

 

 

 

Recent posts

Get Updates

Subscribe to get news, tips and insights directly to your inbox.
Subscribe

Thousands of Users & Companies Worldwide Already Use XpoLog. Download Free Now & Get Free Support Sessions With Every XpoLog Download To Improve Your Log Files Analysis Solution. 

Xpolog Log Analysis Center Worldwide customers

XpoLog Customer Success Michel

 

 

 

 

 

Customers Success @ XpoLog

Log Analysis and Management Solutions

 

XpoLog Log Management & Analysis Center Testimonials