Log Monitoring: What should we do before we start?

Nowadays the most successful monitoring tools have among their options the possibility of monitoring logs. In addition we can find in the market many log management and analytics tools.

On the other hand log analysis is fundamental for security. In fact log analysis has allowed the development of tools for event management and security information SIEM.

Log analysis is even fundamental for the definition of rules on security procedures and audit requirements generated by international organizations like HIPPA (Health Portability Accountability).

Despite this entire panorama, in business meetings where the advantages of application, networks and servers monitoring are being discussed, it is not weird we have the feeling that log monitoring is considered the ugly sister.

The reasons for this unfair evaluation are the basic limitations of log files:

Log files record information and events only on one specific system, so they are useful to determine the cause of problems in that particular system but when we need to resolve a problem that may involve several systems and devices, that’s a whole different story.
There are no standards for location, use, format and size of log files that must be met for different systems and devices, which makes the analysis more complicated and limits extracting maximum benefits from log files.

In this article we will try to review the main aspects we have to have in mind when we decide to start a log monitoring project. Let’s start by clarifying what a log is and how it is used.

A log file is a text file or XML file used to register the automatically produced and time-stamped documentation of events, behaviors and conditions relevant to a particular system.

Generally every software, application, operating system and network device produces log files.

As we said before, there are no standards for logs. However in general we can say that a log file of any system, application or device must include:

Time-stamp: information about the time the event has happened; date, hour, minute and second.
Category: any log file includes some kind of event classification, indicating importance or impact over the system.
Description: Here we can find information about the specific event or condition.

The following are examples for logs and how they are used:

On a web server: an access log can be useful to identify number of visitors, the domains from which they are visiting, the number of requests for each page, usage patterns according day of the week or even the hour of the day.
Operating system: use syslog files to register events, errors, user access, warnings, etc. By reviewing its data, an administrator can check if all processes are loading successfully or the root cause of a specific problem.
In Microsoft Exchange: transactions logs are files used to convey information (email messages, new users, folders deleted, etc) to the database of Exchange. Everything is sent first to the transaction log and then to the database when the system allows it.
In network routers: log files register failing processes, connections and disconnections from wan services and devices, VPN connections status, etc. In firewall: log files register which network connections were allowed and dropped.

It is clear that logs are containers of large and varied information that can be valuable for those who intend to carry out activities like:

Application, network and server monitoring
Optimization and debugging errors
Forensic analysis and root cause analysis.
Vulnerability assessment.
Compliance with legal regulations or register events with audit purposes.
Defining capacity plans or architectural changes.

Now let’s review the main aspects we have to have in mind when we decide to start a project of log monitoring:

Define a goal for log file analysis

Since log files analysis can contribute with different activities a very crucial point is to define the goal and objectives we want to achieve with this project.

Perhaps our interest is focused on improving our troubleshooting capabilities and reducing the time it takes determine the root cause of the problems that may arise.

Security may be our main interest, or to cover the requirements imposed by some regulatory organizations.

When we start a log monitoring project, having a clear goal allows us to size correctly the project and make correct decisions during the evaluation of tools.

Then you have to define the objectives on these specific matters:

Visibility: Specify if you are interested in having a dashboard that allows you to access to the log information or you are more interested in custom made reports.
Accessibility: How important for our goal it is cross analysis? How vital it is the chance to do different queries on the data from logs?
Integration: Do we have already a monitoring tool or a log management tool? How we want to integrate those tools in our project?
Alerts: It is crucial alerts management when certain events or patterns are detected?
Scalability: How do we think the requirements of log monitoring will grow in short and medium term? In devices or systems? With different goals?

We can check Pandora FMS event console and its event correlation alerts in this post in this same blog.

Log files list

It is important to define a first list of log files that must be collected and analyzed including the following information about each log:

Type: Operating System, identification tools, network, Applications, endpoint security, etc.
Systems: Windows 2008 Server or Linux server, Active directory, LDAP, DHCP Server, Router Cisco 1941, Cisco ASA, VPN, MacAfee endpoint security, Apache server, JBoss Server, etc.
Log identification: name and location.
Size: log size in MB.
Register description: brief description on what events are registered in this particular log file.

Type	OS
System	Windows Server 2008 R2
Name and IP address	MDRServer / 191.168.4.56
Log	Setup / Systemroot\System32\Winevt\logs\setup.evtx
Size	1028 MB
Description	Registers events during configuration and performance statistics

Evaluate tools

The recommendation is to evaluate the characteristics of monitoring tools with our objectives and needs.

At this point we must consider we can find tools with different approaches to log monitoring:

Software as a service (SaaS) that does not require installations but the hiring of a remote service
Locally installed tools
Tools based on hardware appliances

We can check the Pandora FMS approach to log monitoring here following this link.

With the information collected here we will be able to establish a project plan that takes us into the interesting and important world of log monitoring.