Introduction to the MikroTik CRS 125

I have been using a MikroTik Cloud Router Switch (CRS) as my home router for the last couple of years. These are cost efficient networking devices which provide a great way of experiencing enterprise level functionality. A brief description of the device from the MikroTik website.

Cloud Router Switch is our new Smart Switch series. It combines the best features of a fully functional router and a Layer 3 switch, is powered by the familiar RouterOS. All the specific Switch configuration options are available in a special Switch menu, but if you want, ports can be removed from the switch configuration, and used for routing purposes.

Choose ports for Wire speed switching, or for routing purposes. 

Perfect SOHO gateway router, switch, all in one box: 

  • Ethernet, Fiber, or 4G (with optional USB modem) gateway connection to Internet 
  • RouterOS gateway/firewall/VPN router with passive cooling 
  • up to twenty-five gigabit switch ports (1xSFP and 24xRJ45)

img

Introduction to the MikroTik for Splunk App

Prior to experimenting with this Splunk App I had the CRS125 integrated with ArcSight as a training exercise. Recently however I decided to become more acquainted with Splunk, which is billed as a competitor to ArcSight in some areas.

Ideally I wanted to replicate the functionality and provide logging on the key functionality I use in the CRS125, notably:

  • DHCP
  • DNS & Cache
  • WiFi Hotspot
  • Firewall
  • VPN
  • Packet Sniffing

When migrating over to Splunk I realised how little I knew about replicating the intricacies of data parsing from the ArcSight FlexConnector framework over to Splunk (with the initially baffling props.conf and transforms.conf files).

At this point I searched the internet to see if anyone had done this work before, this is usually a wasted endeavour with ArcSight as the community is quite a lot smaller and often more limited with what they can share based on company policies.
There were two entries on SplunkBase (the Splunk App Store) however both of these looked quite out of date and/or quite simple in their nature:

A further search took me to the MikroTik Forums and a goldmine of an article written by Jotne titled Using Splunk to analyse MikroTik logs 2.5 (Graphing everything). You will have to be a MikroTik forum member to view the full article https://forum.mikrotik.com/viewtopic.php?t=137338 however I have summarised my experiences with the app.

Configuring Splunk

I’m not running my Splunk Server as root, so I selected port 1514 as a non root user cannot access the standard 514 port. Note: Changing the port here will need a modification to the [source::udp:514] line in Splunk props.conf file.

Add a UDP listener

Settings -> Data Inputs -> Add New (to the right of UDP) -> Port 1514 -> Next -> Select syslog for sourcetype -> Next -> Submit
img

Download the App

As of writing this post the download is available directly below:
https://forum.mikrotik.com/download/file.php?id=34532

Extract the SPL file

Simply unzip the zip file and you’re left with a single .SPL file which contains the application. The SPL file is itself a tar archive of the folder structure.

bas@bun:~$ tar --list -f MikroTik.spl
MikroTik/appserver/
MikroTik/static/
MikroTik/metadata/
MikroTik/default/
MikroTik/appserver/static/
MikroTik/appserver/static/dashboard.css
...

Install app from file

From the start page in Splunk Web click Apps -> Manage Apps, then select Install App from File and select the SPL file.

Restart Splunk

From the start page in Splunk Web click Settings -> Server Controls -> Restart Splunk

Configuring MikroTik

The majority of this configuration will be carried out through the terminal window on the MikroTik device.

Configure Syslog Client

This will provide a syslog target which we will later use when configuring loging destinations.

[admin@MikroTik] /system logging action> add name=logserver target=remote remote=192.168.88.11 remote-port=1514
[admin@MikroTik] /system logging action> print
Flags: * - default 
 0 * name="memory" target=memory memory-lines=1000 memory-stop-on-full=no 

 1 * name="disk" target=disk disk-file-name="log" disk-lines-per-file=1000 
     disk-file-count=2 disk-stop-on-full=no 

 2 * name="echo" target=echo remember=yes 

 3 * name="remote" target=remote remote=0.0.0.0 remote-port=514 
     src-address=0.0.0.0 bsd-syslog=no syslog-time-format=bsd-syslog 
     syslog-facility=daemon syslog-severity=auto 

 4   name="logserver" target=remote remote=192.168.88.11 remote-port=1514 
     src-address=0.0.0.0 bsd-syslog=no syslog-time-format=bsd-syslog 
     syslog-facility=daemon syslog-severity=auto

Configure Logging Policies

Generic Logging

The author of the App suggests that you send all DHCP logs including debug and all other logs that are not debug.
It is very important that the prefix is “MikroTik” and not “mikrotik”.
Splunk uses the MikroTik prefix to find out what type of syslog data that is coming to it.

[admin@MikroTik] /system logging> add action=logserver prefix=MikroTik topics=d
hcp
[admin@MikroTik] /system logging> add action=logserver prefix=MikroTik topics=!debug
[admin@MikroTik] /system logging> print
Flags: X - disabled, I - invalid, * - default
 #    TOPICS                         ACTION                         PREFIX
 0  * info                           memory
 1  * error                          memory
 2  * warning                        memory
 3  * critical                       echo
 4    dhcp                           memory
 5    dhcp                           logserver                      MikroTik
 6    !debug                         logserver                      MikroTik

Firewall and NAT Logging

To log the Firewall and Nat rules, you need to turn on logging and add Log Prefix (under action).

On the WebUI navigate to IP -> Firewall and double click your Firewall rule, scroll down to the Logging section:

img

Ensure the log prefix for all firewall rules starts with FW_

Repeat the same process for the NAT rules

img

Ensure the log prefix for all NAT rules starts with NAT_

Accounting Data

As each packet passes through the router, the packet source and destination addresses are matched against an IP pair list in the accounting table and the traffic for that pair is increased

[admin@MikroTik] /ip accounting> set enabled=yes threshold=2560
[admin@MikroTik] /ip accounting> print
                enabled: yes
  account-local-traffic: no
              threshold: 2560

Scripted Data

To get all the other data like Traffic accounting, uPnP, System health, System resources and DHCP pool information you need a script configured on the router.

This script must be named Data_to_Splunk_using_Syslog

A copy of the script is available here.

It is configured as shown below:
img

Schedule the script to run every 5 minutes:

[admin@MikroTik] /system scheduler> add interval=5m name="Data to Splunk" on-event=Data_to_Splunk_using_Syslog
[admin@MikroTik] /system scheduler> print
Flags: X - disabled
 #   NAME                                                                                    START-DATE  START-TIME                                                                                  INTERVAL             ON-EVENT                                                                                   RUN-COUNT
 0   Data to Splunk                                                                          jan/07/2019 13:04:45                                                                                    5m                   Data_to_Splunk_using_Syslog                                                                        0
 

Using the Splunk MikroTik App

What does it provide?

  • MikroTik System Changes
  • MikroTik DNS Live usage
  • MikroTik Volt/Temperature
  • MikroTik Resource
  • MikroTik Live attack
  • MikroTik DNS request
  • MikroTik Web Proxy
  • MikroTik Firewall Rules
  • MikroTik Traffic
  • MikroTik DHCP request
  • MikroTik DHCP pool information
  • MikroTik Remote Connection
  • MikroTik uPnP
  • MiroTik Hotspot login/logout information
  • MikroTik Wifi connection
  • MikroTik Wifi strength
  • MikroTik Uptime
  • MikroTik Device List

How does it do it?

Eventtypes
Defines event types using eventtypes.conf in the following regex format:

[dns_question]
search = "dns,packet * question:"

Event types are a categorization system to help you make sense of your data. Event types let you sift through huge amounts of data, find similar patterns, and create alerts and reports.

img

Field Extractions

These are configured in props.conf and use regex to extract field values from the raw message.

EXTRACT-mikrotik_dns_src = dns,packet.*from\s(?<src_ip>[^:]+):(?<src_port>\d+)
EXTRACT-mikrotik_dns_site = dns,packet.*question:\s(?<site1>[^:]+):(?<site_type>[^:]+):(?<site_direction>[^:]+)

Filtering out

Filtering out in Splunk involves sending events to the nullQueue. This requires configuration in both the props.conf and transforms.conf files.

props.conf has the following lines which identifies a source to run the transforms against.

[source::udp:514]
TRANSFORMS-dns=remove_dns_query,remove_dns_answer

transforms.conf has the following lines which use regex to identify what to send to the nullQueue.

[remove_dns_query]
REGEX = dns,packet.*ra:(0|1) QUERY
DEST_KEY = queue
FORMAT = nullQueue

[remove_dns_answer]
REGEX = dns,packet.*answer:
DEST_KEY = queue
FORMAT = nullQueue

Setting the sourcetype

This is done to modify the sourcetype from syslog to mikrotik if the “MikroTik:” tag is identified in the message.
props.conf has the following lines which force anything received on udp 514 to be evaluated by the force_mikrotik transform:

[source::udp:514]
TRANSFORMS-force_mikrotik = force_mikrotik

transforms.conf has the following entry which sets the correct sourcetype if the message matches the regex.

[force_mikrotik]
DEST_KEY =  MetaData:Sourcetype
REGEX =  \sMikroTik:\s
FORMAT =  sourcetype::mikrotik

Results

After allowing some time for Splunk to ingest the data, the samples below show the output from a few of the views.

img
img
img
img
img