• Home
  • Devops
  • Opensource
More

    Using Rsyslog to ship the logs over TCP on CentOS

    Rsyslog is the one open source tool for log processing. It offers high performance and security. Rsyslog consists following features,

    1. Multi-threading
    2. TCP, SSL, TLS, RELP
    3. MySQL, PostgreSQL, Oracle and more
    4. Filter any part of syslog message
    5. Fully configurable output format
    6. Suitable for enterprise-class relay chains
      In this blog post, we will learn about how to use Rsyslog to ship logs over TCP.

    Prerequisite:

    • Linux Hands on.

    Step 1 | Install Rsyslog

    Create rsyslog.repo:

    $ sudo vi /etc/yum.repos.d/rsyslog.repo

    Paste following content to /etc/yum.repos.d/rsyslog.repo

    [rsyslog-v5-stable]
    name=Adiscon Rsyslog v5-stable for CentOS-$releasever-$basearch
    baseurl=http://rpms.adiscon.com/v5-stable/epel-$releasever/$basearch
    enabled=0
    gpgcheck=0
    protect=1[rsyslog-v6-beta]
    name=Adiscon Rsyslog v6-beta for CentOS-$releasever-$basearch
    baseurl=http://rpms.adiscon.com/v6-beta/epel-$releasever/$basearch
    enabled=0
    gpgcheck=0
    protect=1[rsyslog-v6-stable]
    name=Adiscon Rsyslog v6-stable for CentOS-$releasever-$basearch
    baseurl=http://rpms.adiscon.com/v6-stable/epel-$releasever/$basearch
    enabled=0
    gpgcheck=0
    protect=1[rsyslog-v7-beta]
    name=Adiscon Rsyslog v7-beta for CentOS-$releasever-$basearch
    baseurl=http://rpms.adiscon.com/v7-beta/epel-$releasever/$basearch
    enabled=0
    gpgcheck=0
    protect=1[rsyslog-v7-devel]
    name=Adiscon Rsyslog v7-devel for CentOS-$releasever-$basearch
    baseurl=http://rpms.adiscon.com/v7-devel/epel-$releasever/$basearch
    enabled=0
    gpgcheck=0
    protect=1[rsyslog-v7-stable]
    name=Adiscon Rsyslog v7-stable for CentOS-$releasever-$basearch
    baseurl=http://rpms.adiscon.com/v7-stable/epel-$releasever/$basearch
    enabled=1
    gpgcheck=0
    protect=1[rsyslog-v8-devel]
    name=Adiscon Rsyslog v8-devel for CentOS-$releasever-$basearch
    baseurl=http://rpms.adiscon.com/v8-devel/epel-$releasever/$basearch
    enabled=0
    gpgcheck=0
    protect=1[rsyslog-v8-stable]
    name=Adiscon Rsyslog v8-stable for CentOS-$releasever-$basearch
    baseurl=http://rpms.adiscon.com/v8-stable/epel-$releasever/$basearch
    enabled=1
    gpgcheck=0
    protect=1

    Run yum update

    $ sudo yum -y update

    Install Rsyslog

    $ yum -y install rsyslog

    Step 2 | Rsyslog configuration to ship logs over TCP.

    Rsyslog consist legacy syntax too. But I am not using legacy syntax for configuration.

    Your default /etc/rsyslog.conf should consist ,

    # rsyslog configuration file
    # note that most of this config file uses old-style format,
    # because it is well-known AND quite suitable for simple cases
    # like we have with the default config. For more advanced
    # things, RainerScript configuration is suggested.# For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html
    # If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html#### MODULES ####module(load="imfile" PollingInterval="10")
    #module(load="imuxsock") # provides support for local system logging (e.g. via logger command)
    #module(load="imklog") # provides kernel logging support (previously done by rklogd)
    #module(load"immark") # provides --MARK-- message capability# Provides UDP syslog reception
    # for parameters see http://www.rsyslog.com/doc/imudp.html
    #module(load="imudp") # needs to be done just once
    #input(type="imudp" port="514")# Provides TCP syslog reception
    # for parameters see http://www.rsyslog.com/doc/imtcp.html
    #module(load="imtcp") # needs to be done just once
    #input(type="imtcp" port="514")
    #### GLOBAL DIRECTIVES ##### Use default timestamp format
    #$template FileFormat,"%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n"
    #$template TraditionalFileFormat,"%TIMESTAMP% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n
    #$ActionFileDefaultTemplate TraditionalFileFormat
    # File syncing capability is disabled by default. This feature is usually not required,
    # not useful and an extreme performance hit
    #$ActionFileEnableSync on# Include all config files in /etc/rsyslog.d/
    $IncludeConfig /etc/rsyslog.d/*.conf
    #### RULES ##### Log all kernel messages to the console.
    # Logging much else clutters up the screen.
    #kern.* /dev/console# Log anything (except mail) of level info or higher.
    # Don't log private authentication messages!
    *.info;mail.none;authpriv.none;cron.none /var/log/messages# The authpriv file has restricted access.
    authpriv.* /var/log/secure# Log all the mail messages in one place.
    mail.* /var/log/maillog
    # Log cron stuff
    cron.* /var/log/cron# Everybody gets emergency messages
    *.emerg :omusrmsg:*# Save news errors of level crit and higher in a special file.
    uucp,news.crit /var/log/spooler# Save boot messages also to boot.log
    local7.* /var/log/boot.log
    # ### begin forwarding rule ###
    # The statement between the begin ... end define a SINGLE forwarding
    # rule. They belong together, do NOT split them. If you create multiple
    # forwarding rules, duplicate the whole block!
    # Remote Logging (we use TCP for reliable delivery)
    #
    # An on-disk queue is created for this action. If the remote host is
    # down, messages are spooled to disk and sent when it is up again.
    #$WorkDirectory /var/lib/rsyslog # where to place spool files
    #$ActionQueueFileName fwdRule1 # unique name prefix for spool files
    #$ActionQueueMaxDiskSpace 2g # 1gb space limit (use as much as possible)
    #$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
    #$ActionQueueType LinkedList # run asynchronously
    #$ActionResumeRetryCount -1 # infinite retries if host is down# Use standard RFC5424 log format for local logs
    #$ActionFileDefaultTemplate RSYSLOG_SyslogProtocol23Format
    # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
    #*.* @@remote-host:514
    # ### end of the forwarding rule ###

    As I am using non-legacy syntax, you can configure Rsyslog at /etc/rsyslog.d
    This directory should consist all condiguration files.
    For Example we are having following logs ,

    Jan 13, 2016 7:51:30 PM org.apache.cxf.phase.PhaseInterceptorChain doDefaultLogging
    WARNING: Interceptor for {http://endpoints.example.com/}SearchServiceImplService#{http://endpoints.example.com/}search has thrown exception, unwinding now
    org.apache.cxf.binding.soap.SoapFault: Error writing to XMLStreamWriter.
    at org.apache.cxf.binding.soap.interceptor.SoapOutInterceptor$SoapOutEndingInterceptor.handleMessage(SoapOutInterceptor.java:286)
    at org.apache.cxf.binding.soap.interceptor.SoapOutInterceptor$SoapOutEndingInterceptor.handleMessage(SoapOutInterceptor.java:268)
    at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:243)
    Jan 13, 2016 7:51:32 PM org.apache.cxf.phase.PhaseInterceptorChain doDefaultLogging
    WARNING: Interceptor for {http://endpoints.example.com/}SearchServiceImplService#{http://endpoints.example.com/}search has thrown exception, unwinding now
    org.apache.cxf.binding.soap.SoapFault: Error writing to XMLStreamWriter.
    at org.apache.cxf.binding.soap.interceptor.SoapOutInterceptor$SoapOutEndingInterceptor.handleMessage(SoapOutInterceptor.java:286)
    at org.apache.cxf.binding.soap.interceptor.SoapOutInterceptor$SoapOutEndingInterceptor.handleMessage(SoapOutInterceptor.java:268)
    at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:243)

    It is multiline log which is located at /var/log/apache.log. To ship this log I will be writing configuration /etc/rsyslog.d/logging.conf,

    input(type="imfile"
    File="/var/log/apache.log"
    Tag="apache-log"
    startmsg.regex="^((Jan)|(Feb)|(Mar)|(Apr)|(May)|(Jun)|(Jul)|(Aug)|(Sep)|(Oct)|(Nov)|(Dec))[[:space:]]([0-9]|[0-9][0-9]),[[:space:]]([0-9][0-9][0-9][0-9])[[:space:]]([0-9]|[0-9][0-9]):([0-9]|[0-9][0-9]):([0-9]|[0-9][0-9])[[:space:]](AM|PM)"
    )
    template(name="ForwardFormat" type="list") {
    property(name="msg" spifno1stsp="on" )
    property(name="msg")
    }
    if $programname == 'apache-log' then {
    action(
    type="omfwd"
    Target="10.112.34.12"
    Port="5001"
    Protocol="tcp"
    Template="ForwardFormat"
    )
    stop
    }

    If you are dealing with elasticsearch, then you can use Elasticsearch plugin for shipping logs.
    Here I divided configuration into three parts,
    i. Input
    ii. Template
    iii. Action

    i. Input: It consist type as imfile , the location of a file to tail, tag for the log and regex especially for distinguishing multiline logs.

    ii. Template: Used to format the logs.

    iii. Action: Where to send logs. Type of action is omfwd and we are forwarding logs to 5001 TCP port of host 10.112.34.12.

    You need to make sure TCP port should be open and reachable for the host.
    This is the simplest way to use Rsyslog to ship logs over TCP.

    Thank you for reading the blog. We do support the open source writing about open source technologies.

    Recent Articles

    SED Command : Search And Replace in file

    This is 1m read blog. In this blog we will check how to replace source string with replacement string for file with...

    How to check top 10 CPU consuming processes on Linux ?

    In shared linux server environment it is important to track the usage of high CPU consuming processes. If...

    Explained: Keep Specific Metrics in Prometheus using Prometheus Job

    This is 1m read article written by Kuberneteslab. In this article we will discuss about the keep action for keeping certain metrics in...

    How to check the Disk Statistics on Linux using Terminal ?

    This is 1m read article written by Kuberneteslab. In this article we will discuss about the disk statistics and how to view...

    How to get resources from Kubernetes Cluster using Kubectl ?

    Get Resources from K8S Cluster using Kubectl cmd: In this article we will look into some commands...

    Related Stories

    Subscribe
    Notify of
    guest
    0 Comments
    Inline Feedbacks
    View all comments

    Stay on op - Ge the daily news in your inbox

    0
    Would love your thoughts, please comment.x
    ()
    x