Data Collection

Keys-by-name_0012_Log-Data-Collection

When XpoLog collects your data, it uses existing protocols, so there is no need for infrastructure changes or agents, you get immediate value.

  • File system – Any Log Format

XpoLog can collect any type of data from a local or remote file system, logs, files, applications, servers, or any type of log or machine data.
XpoLog can collect log files from Windows Event Logs, SysLog, Database tables, and all textual files of any format, XpoLog can also track and collect end-user transaction logs. The data can come from multiple logs from local and/or remote infrastructures, including servers, applications, application servers, Windows machines, UNIX and Linux machines, network and security devices, databases, and data centers, all in real-time through a standard web-browser. In short, there is very little cyberspace in today’s world XpoLog cannot gather data from, so you’re probably covered.
XpoLog has a log collection policy that determines how to collect the log information into its repository. It can collect data at any given frequency from just seconds and upwards, as well as keep the data available in the XpoLog repository for as long as needed. You can also define your own custom log collection policy if need be. The XpoLog data collection mechanism knows how to detect dynamic changes in the log data, and manages references to the sources to ensure that the data collection is complete.

  • SysLog

XpoLog can run as a SysLog server.
Once your network devices are sending data to XpoLog’s machine, XpoLog is automatically processing the data making it available for viewing and searching. By using the existing protocol, with no need for infrastructure changes or agents, you are getting immediate value. XpoLog can function as a SysLog server. You can configure XpoLog to monitor incoming SysLog messages and decode them. XpoLog uses UPD or TCP SysLog listeners to receive events from one or more source devices (which send SysLog messages) for easy searching, reporting, and alerting. Once XpoLog receives SysLog data, it automatically knows how to create dedicated logs per source device, and begin the parsing process.

  • Agent-less over SSH

XpoLog can access log files on remote servers over SSH agent-less, and collect all the data with no need for an agent, to ensure you are getting immediate value without complex setup.
XpoLog can access log files on remote servers over SSH agent-less, using only common protocols such as SSH to connect to UNIX machines. All you need to do is give XpoLog reading permission to the required sources. To make sure that the applications’ and systems’ normal work flow stays unaffected, XpoLog is completely non-intrusive and has a very low footprint (<2% from a single CPU core) thus having no impact on the sources. By using the existing protocol, there is no need for changing your infrastructure and no need for agents, which means XpoLog will give you immediate value.

  • Agents – Data Forward

For data transfer between unique environments with no direct access, you can install XpoLog as an agent in order to forward data to the log repository platform.
Although XpoLog is completely agentless, there are cases where there is no available connectivity to the sources using standard protocols. This could be a case where you wanted to transfer files from one environment to another, for example between Windows and Linux, where there is no protocol from Linux to Windows through the standard file system, so we must go through HTTP. In cases like these, XpoLog can be installed as an agent on the remote source and collect data using the HTTP/S protocol. If you need to collect data as is, from multiple sources, into a central repository, you will need an agent for the log synchronization too. You can install XpoLog as an agent or structure to deploy several instances at once.

  • Databases – JDBC

XpoLog can access databases on remote servers and collect all the data with no need for an agent, only common protocols such as JDBC drivers to connect to database tables.
You can add data from a database into XpoLog as a log, provided that the database is connected to a JDBC driver. The XpoLog server can then read log files into XpoLog from the database servers. Database table configuration in XpoLog is based on a common SQL query, and there is no need for agents. There is no limitation on the source files’ sizes; since XpoLog manages the data so efficiently that the source files can be 1 KB or 1 TB, without causing any delay to XpoLog’s functionality. Any JDBC supported database can be accessed.

  • Windows Events

XpoLog automatically recognizes multiple types of Windows Event logs, and connects and parses them automatically.
XpoLog automatically recognizes multiple types of Windows Event logs and sees which are system, which are security, and which are application. The XpoLog server can read and collect data from log files from Windows Network servers and supports Windows Event logs which are added from remote servers from a Windows environment.

  • Zip, GZIP sources

XpoLog knows how to collect and extract data from a specific file from a zip or gzip, simply by indicating the name of the file, without extracting the zip/gzip file.
On a Windows machine, when adding a local log, you can define zipped logs (single/multiple files) without extracting them. On a Unix machine, you can define Gzipped logs (single/multiple files) without extracting them. All you need to do is give XpoLog the name of the file you want from the zip or gzip, and XpoLog will find it and bring out the relevant data from this relevant file. This can save you from having to look through search results and analytics of irrelevant data until you find what you are looking for.

  • HDFS

You can integrate XpoLog to HDFS. XpoLog automatically presents an Hadoop HDFS Integration. When saving, XpoLog applies an automated pattern on the incoming log, and the Log Viewer opens displaying the parsed records of the new log.
XpoLog accesses logs from a directory situated in the Hadoop environment. When adding logs to XpoLog, XpoLog automatically presents an Add Hadoop account page if you do not already have one. When saving, XpoLog applies an automated pattern on the incoming log, and the Log Viewer opens displaying the parsed records of the new log. If need be you can alter the pattern to tune the log and parse it in more detail. You can add a logs directory that resides on a local or remote machine to XpoLog Center.

  • Logstash

XpoLog can receive data sent by logstash.
XpoLog can automatically be fully integrated to Logstash. If you’re already using logstash, it is very easy to integrate. XpoLog even has its own logstash output.

  • Fluentd

You can use Fluentd to send data to XpoLog.
You can send data from Fluentd to the XpoLog SysLog server.

Get 30 Days Free PRO Trial Today

When your Trial ends, your account will automatically switch to the Free 1GB/Day Light account.

Download Now

See online demo or schedual a 1-on-1 demo. Ask support.