Introduction

The ArcSight CounterACT Connector is an extensible framework built on top of the standard ArcSight SmartConnector. This allows incredibly flexible integrations between the ArcSight ESM and third party products / external scripts / generic commands which produce a usable output. In this post I am going to show a simple but useful example of running two commands through the CounterACT connector, on a centralised server, parsing the result of the commands into actionable events which can then be used as part of the workflow within the ESM.

The two commands I intend to use in this example are Nmap and whois both of which are useful to Analysts using the ESM, however by default, without the CounterACT connector would have to be installed locally on the Analysts’ PC. When using the CounterACT connector these commands can be installed on a central server reducing the number of firewall rules which need to be opened.

Installation

The installation of the CounterACT connector follows the standard SmartConnector process for the most part, in my examples below I am using a CentOS 7 server:

Choose the installation directory, ensuring ownership by the arcsight user. e.g chown arcsight:arcsight /opt/arcsight/connectors/counteract
Make sure the ArcSight Connector binary is marked as executable. e.g chmod +x /home/arcsight/ArcSight-7.4.0.7963.0-Connector-Linux64.bin

[arcsight@connectors connectors]$ ll
total 0
drwxrwxr-x. 2 arcsight arcsight  6 Dec 22 18:09 counteract
drwxrwxr-x. 3 arcsight arcsight 20 Sep 17 01:09 syslog_tcp
drwxrwxr-x. 3 arcsight arcsight 20 Sep 29 03:39 syslog_udp
drwxrwxr-x. 3 arcsight arcsight 20 Dec 10 15:44 threatintel
[arcsight@connectors connectors]$ /home/arcsight/ArcSight-7.4.0.7963.0-Connector-Linux64.bin 
Preparing to install...
Extracting the JRE from the installer archive...
Unpacking the JRE...
Extracting the installation resources from the installer archive...
Configuring the installer for this system's environment...

Launching installer...

Go through the motions of installing a standard connector:

  • enter the directory created earlier
  • don’t create links (unless you really want to)
  • press enter and let the installation complete
Pre-Installation Summary
------------------------

Please Review the Following Information Before Continuing:

Product Name:
    ArcSight SmartConnector

Install Folder:
    /opt/arcsight/connectors/counteract

Link Folder:
    DO NOT INSTALL



PRESS <ENTER> TO CONTINUE: 

===============================================================================
Installing...
-------------

 [==================|==================|==================|==================]
 [------------------|------------------|------------------|------------------]



===============================================================================
Installation Complete
---------------------

At this stage the core components of the connector should be installed and the installation will exit. The installer will instruct you to finish the configuration of the SmartAgent, please go to the folder (in my case /opt/arcsight/connectors/counteract/current/bin/ and execute the script ./runagentsetup.sh

Follow the instructions entered below:

[arcsight@connectors bin]$ ./runagentsetup.sh 

Assuming ARCSIGHT_HOME: /opt/arcsight/connectors/counteract/current
Assuming JAVA_HOME: /opt/arcsight/connectors/counteract/current/jre

ArcSight Agent Setup starting...

Connector Setup Wizard starting in mode [CONSOLE]
[Fri Dec 22 18:12:16 GMT 2017] [INFO ] Checking for a running instance of connector...
[Fri Dec 22 18:12:17 GMT 2017] [INFO ] Starting up connector...

Connector Setup
---------------

--------------------------------------------------------------------------------
What would you like to do?


0-	Add a Connector
1-	Set Global Parameters

Please select an option: [Add a Connector] [0..1/cancel] :


--------------------------------------------------------------------------------
Select the connector to configure


Type:
	0-	Amazon Web Services CloudTrail
	1-	Apache HTTP Server Access File
	2-	Apache HTTP Server Error File
	3-	Apache Tomcat File
	4-	ArcSight Asset Import File
	5-	ArcSight CEF Cisco FireSIGHT Syslog
	6-	ArcSight CEF Encrypted Syslog (UDP)
	7-	ArcSight Common Event Format File
	8-	ArcSight Common Event Format Hadoop
	9-	ArcSight Common Event Format Multiple File
	10-	ArcSight Common Event Format REST 
	11-	ArcSight FlexConnector CounterACT
	12-	ArcSight FlexConnector File
	13-	ArcSight FlexConnector ID-Based DB
	14-	ArcSight FlexConnector JSON Folder Follower
	15-	ArcSight FlexConnector Multiple DB
	16-	ArcSight FlexConnector Multiple Folder File
	17-	ArcSight FlexConnector Regex File
	18-	ArcSight FlexConnector Regex Folder File
	19-	ArcSight FlexConnector REST
(N)ext -	------------- Next page ------------- 
Please select an option [0..19]: 

Type 11 to select the ArcSight FlexConnector CounterACT option, note that this number changes from time to time

Please select an option [0..19]: 11


Please verify the following parameters

Type: ArcSight FlexConnector CounterACT


Are the values correct [yes/no/back/cancel]?yes


--------------------------------------------------------------------------------
Enter the parameter details

Configuration File:

At this stage we haven’t introduced anything about the configuration file, to allow us to finish installing the connector at this stage enter tmp here, the connector will attempt to find this file, but fail verification, this doesn’t matter at this stage and serves as a nice reminder that the full filename is tmp.counteract.properties

I finish off the configuration of the connector below:

Configuration File: tmp


Please verify the following parameters

Configuration File: tmp


Are the values correct [yes/no/back/cancel]?yes


   |                                        | 0%Verifying the parameters
   |########################################| 100%
Connector parameters did not pass the verification with error [0:Failed to read configuration from file [tmp] (Unable to load properties from file [tmp.counteract.properties])
]. Do you still want to continue?
[yes/no] yes
--------------------------------------------------------------------------------
Enter the type of destination


0-	ArcSight Manager (encrypted)
1-	ArcSight Logger SmartMessage (encrypted)
2-	ArcSight Logger SmartMessage Pool (encrypted)
3-	CEF File
4-	Event Broker (CEF Kafka)
5-	CEF Syslog
6-	CEF Encrypted Syslog (UDP)
7-	CSV File
8-	Raw Syslog

Please select an option: [ArcSight Manager (encrypted)] [0..8/back/cancel] :

Manager Hostname: esm
Manager Port[8443]: 
User: admin
Password: 
                             
AUP Master Destination:      
	0-	true
	1-	false
Please select an option [0..1][false]: 
Filter Out All Events:
	0-	true
	1-	false
Please select an option [0..1][false]: 
Enable Demo CA:
	0-	true
	1-	false
Please select an option [0..1][false]: 


Please verify the following parameters

Manager Hostname: esm
Manager Port: 8443
User: admin
Password: ********
AUP Master Destination: false
Filter Out All Events: false
Enable Demo CA: false


Are the values correct [yes/no/back/cancel]?

--------------------------------------------------------------------------------
Enter the connector details


Name[]: BASEC_CounterACT
Location[]: connectors.basec.co.uk
DeviceLocation[]: /opt/arcsight/connectors/counteract
Comment[]: 


Please verify the following parameters

Name: BASEC_CounterACT
Location: connectors.basec.co.uk
DeviceLocation: /opt/arcsight/connectors/counteract
Comment: 


Are the values correct [yes/no/back/cancel]?

Registering destination
   |########################################| 100%
--------------------------------------------------------------------------------
Following certificate will be imported into connector trust store:
Host/port: esm_8443
Details: CN=esm, OU=ESM, O=Arcsight, L=95014, ST=CA, C=US





0-	Import the certificate to connector from destination
1-	Do not import the certificate to connector from destination

Please select an option: [Import the certificate to connector from destination] [0..1/back/cancel] : 


   |                                        | 0%Importing certificate, registering destination and restarting the container
   |########################################| 100%
--------------------------------------------------------------------------------
Add connector Summary
Following are the added connector details:
Connector Name [BASEC_CounterACT], Connector Type [ngflexcounteract]


Continue [yes] ?

--------------------------------------------------------------------------------
Would you like to continue or exit?


0-	Continue
1-	Exit

Please select an option: [Continue] [0..1/back/cancel] :1

Making it work

We have already seen reference to the counteract.properties file. This is the file which controls what is executed, and what information is provided to the script/application/command. This file must be created under the flexagent directory of the connector, in my example this is /opt/arcsight/connectors/counteract/current/user/agent/flexagent.

My file, which executes Nmap and whois is below:

command.count=2
command[0].name=whois
command[0].displayname=whois
command[0].parameter.count=1
command[0].parameter[0].name=ipaddress
command[0].parameter[0].displayname=Ip Address
command[0].action=/usr/bin/whois ${ipaddress}
command[1].name=nmap
command[1].displayname=nmap
command[1].parameter.count=1
command[1].parameter[0].name=ipaddress
command[1].parameter[0].displayname=Ip Address
command[1].action=/bin/nmap ${ipaddress}

Important points to note:

  • You can include many commands by increasing the count and numbering them sequentially.
  • You can include many parameters within the command
  • Each commmand name must be unique
  • Each paramater name within each command must be unique and also numbered sequentially
  • The full path must be provided for the command which is listed after action
  • Parameters must be provided in the ${name} format

At this stage you can keep the tmp.counteract.properties filename, or name it something different. Note that if you name it something different you must re-run ./runagentsetup.sh to provide the updated configuration name.

You should also notice the connector will have appeared in ESM as a normal SmartConnector would.

img

Executing a Command

Whilst you have noticed the Connector has appeared in ESM as expected, there is an option available which is special. You are able to execute the commands directly by right clicking on the Connector, selecting Send Command -> CounterACT -> your command

img

After entering the IP Address when prompted (I entered 123.123.123.123) a new window will open in the central Viewer panel. Once the command has finished executing the result will be displayed in this central window, as shown below:

img

A much better and more professional method would be to utilise the Integration Command capability of the ArcSight ESM to execute the CounterACT connector command. This is relatively self explanatory, the screenshot below shows a simple configuration where the $selectedItem is provided as the parameter to the script when executed.

img

Parsing the Standard Output

Another possibility is using the output of this command, to re-inject another event into ArcSight, this is done by using an additionalregex processor or a “second level parser”. These parsers have only a subset of the standard parser functionality and often need to be used in a slightly different way, for example if you have more than one additionalregex processor they are all used sequentially as a map file would be. This means you may need multiple files to fully parse your output into an event, this may seem inefficient but the throughput on commands executed is drastically lower than a standard connector handles.

The following is a sample output from an Nmap command:

Standard Output:

Starting Nmap 6.40 ( http://nmap.org ) at 2018-03-05 20:12 GMT
Nmap scan report for esm (192.168.3.9)
Host is up (0.000012s latency).
Not shown: 997 closed ports
PORT     STATE SERVICE
22/tcp   open  ssh
8443/tcp open  https-alt
9000/tcp open  cslistener
MAC Address: 00:0C:29:A7:D8:F6 (VMware)

Nmap done: 1 IP address (1 host up) scanned in 0.04 seconds

Additional regex processors should be placed in the following sub folder /opt/arcsight/connectors/counteract/current/user/agent/fcp/additionalregexparsing/ngflexcounteract/ and have the name format regex.0.sdkrfilereader.properties where 0 is increased sequentially as needed.

My files are regex.0.sdkrfilereader.properties:

  • Performs its parsing on event.rawEvent (the full Standard Output)
  • (?s) defines a multiline regex
  • This parser is looking for something unique in an Nmap message e.g “Starting Nmap”
  • The regex capture takes the entire message and puts it into deviceCustomString1
  • The regex also sets the deviceProduct to “Nmap” if the deviceCustomString1 is not null.
source.field=event.rawEvent
regex=(?s)(.*?Starting Nmap.*)
token.count=1
token[0].name=NmapMessage
event.deviceCustomString1=NmapMessage
event.deviceProduct=__ifThenElse(NmapMessage,,,__stringConstant("Nmap"))

regex.1.sdkrfilereader.properties:

  • Captures whether or not the host is “up” or “down” and puts it in deviceCustomString3
source.field=event.rawEvent
regex=(?s).*?Host is ([^\\s]+).*
token.count=1
token[0].name=HostStatus
event.deviceCustomString3=HostStatus

regex.2.sdkrfilereader.properties

  • Captutes the targetted IP and Hostname
  • Note that __regexToken and other double underscore operators are availble
source.field=event.rawEvent
regex=(?s).*?Nmap scan report for ([^\\s]+) \\(([^\\)]+)\\).*
token.count=1
token[0].name=DestinationHost
token[1].name=DestinationAddress
event.destinationHostName=DestinationHost
event.destinationAddress=__regexTokenAsAddress(DestinationAddress,"(.*)")

The end result of the configuration files and additional regex parsers is below, this output will be viewable by creating an ActiveChannel viewing your CounterACT connector events:
img