L1 Application Monitoring Web Services - Indicators and Warnings

Introduction

The ArcSight Activate L1 - Application Monitoring Web Services - Indicators and Warnings package has been developed to detect anomalies and suspicious requests that are recorded in web server logs that may not be visible to other protective measures such as IDS/IPS and WAF due to the use of encrypted transport. In addition, the package provides a set of HTTP request method and response code filters supported by the HTTP Protocol to monitor possible DoS HTTP flooding attacks, as well as identifies some suspicious XSS and SQL injection attacks.

It also can provide indicators and warnings of potential security incidents at the application level, such as L1 Application Monitoring Web Applications or other packages.

Author:

Sean Davies HPE

Seema Khan HPE

Nellie Wang HPE

Main Use Cases

Implemented Use Cases:

The L1 Application Monitoring Web Services - Indicators and Warnings package has addressed the following use cases
  • Use Case 1 - HTTP Request/Response Suspicious Activity
  • Use Case 2 - Suspicious HTTP Request
  • Use Case 3 - Suspicious URL Pattern Detection
Future Use Cases:
  • Monitor excessive HTTP request or response activity from multiple sources (DDoS)

  • Monitor vulnerable HTTP methods flooding from multiple sources

Supported Log Sources

The L1 Application Monitoring Web Services - Indicators and Warnings package currently supports the following log source types.

Vendor Product Version(s) Comments
Microsoft Internet Information Services 6.0 / 7.0 / 7.5 / 8.0 / 8.5 Requires a parser override for Microsoft IIS SmartConnector releases prior to 7.2.2. Attached Below.
Apache Apache 1.x / 2.x Requires to follow ArcSight ApacheSyslogConfig.pdf to do Apache logging configuration and modify log User-Agent field
Apache Tomcat 2.x Requires to follow ArcSight ApacheTomcatFileConfig.pdf to configure web server logs

SmartConnector Configuration

  • Microsoft Internet Information Server
Follow the MicrosoftIISFileConfig.pdf to install your SmartConnector for the 'Microsoft IIS Logs' option.

a) Select Connector type “Microsoft IIS File” for single web application

b) Provide web server log folder of IIS

e.g. C:\inetpub\logs\LogFiles\W3SVC1

c) If ESM does not receive logs from connector for IIS, then modify the connector c:\<CONNECTOR_HOME>\current\user\agent\agent.properties file for correct logfile parameters as:

agents[0].logfile.name.prefix=u_ex
agents[0].logfilehome=C\:\\inetpub\\logs\\LogFiles\\W3SVC1
agents[0].logfilename=C\:\\inetpub\\logs\\LogFiles\\W3SVC1

d) For IIS 7.0, 7.5, and 8.0 there is a known issue with default IPv6 address showing up in logs instead of IPv4. This shows the Attacker/Target IP address fields as blank in the Active Channel.

IIS-7-7.5-8.0-IPv6-issue.png

To fix this, disable IPv6 on that network interface card and add registry key for DisabledComponents as in this article:

https://support.microsoft.com/en-us/help/929852/how-to-disable-ipv6-or-its-components-in-windows

Note: IIS Parser override is required if the version of SmartConnector is prior to v7.2.2. Please follow the below steps to apply the Microsoft Internet Information Server parser override ( iis_parser_overide_activate.zip).

- Create a new folder called iis under ARCSIGHT_HOME/user/agent/fcp

- Put the attached 'iis_file.sdkfilereader.properties file into the ARCSIGHT_HOME/user/agent/fcp/iis folder

- Restart SmartConnector
  • Apache HTTP

Follow the ApacheSyslogConfig .pdf to install your SmartConnector for 'Apache HTTP Server Syslog' option.

  • Apache Tomcat

Follow the ApacheTomcatFileConfig .pdf to install your SmartConnector for 'Apache Tomcat File' option. To process HTTP requests and responses related logs you need to select 'apache_tomcat_access_file' log type. Select 'apache_tomcat_access_file' log type when monitoring Apache Tomcat server administrative management.

ApacheTomcatlogFile.PNG

Package Download Instructions

The L1 Application Monitoring Web Services package can be downloaded from the ArcSight Marketplace, or internally from the GitHub web site (https://github.com/ArcSightActivate/Application).

Package Installation

Introduction

The installation of the Activate content is now being done through a wrapper. We have started doing this to maintain any required configuration that was previously done and to support future customizations within specific elements.

You will have to run an installer script as per the instruction provided.

The installer script will automatically manipulate files, allowing you to keep your customizations, while we push contents or updates up to standard resources. If the packages in the bundle are installed via the ArcSight Console, we will overwrite your configuration and you will have to uninstall and reinstall this particular package.

Prerequisites

Ensure the following is complete:
  • Activate Base Package v2.5.0 or later has been installed
  • Ensure that the ESM is setup to sort packages by their IDs:

    - Open ESM server.properties (<ARCSIGHT_HOME>/manager/config)

    - Add following line:

    export.archive.reference.sort.order=id

  • For ESM in Compact Mode: Restart the ESM Manager

    or For ESM in Distributed Mode: Restart the ESM Manager, Aggregator(s), and Correlator(s)

Package Install

Ensure the following is complete:
  • Activate Base Package (v2.5.1.0) has been installed

1. Download and extract L1-Application Monitoring Web Services - Indicators and Warnings Package zip file into your ArcSight console's home directory.

2. Copy both L1-Application_Monitoring_Web_Services_-_Indicators_and_Warnings_0.1.0.0.bat file as well as the L1-Application_Monitoring_Web_Services_-_Indicators_and_Warnings_0.1.0.0.arb file to the current directory on the path where the ArcSight Console is installed i.e. c:\arcsight\console\current

 

3. Execute the L1-Application_Monitoring_Web_Services_-_Indicators_and_Warnings_0.1.0.0 .bat file. Provide ESM, port, username, and password. The installation script will import and install contained package.

Imported and Installed package:

L1-Application_Monitoring_Web_Services_-_Indicators_and_Warnings

L1-App Monitoring Package Install using batch file
4. Verify the Package from Console L1-App-Mon-package-install.png

Package Uninstallation

Follow the following steps to uninstall the L1 Application Monitoring Web Services - Indicators and Warnings package:
  • Highlight the package (see the below screenshot), right click 'Uninstall Package'

Uninstall and Delete package
  • Highlight this uninstalled package, and select 'Delete Package'
  • This content should be removed from ESM

Package Configuration

N/A

Test Plan

The link below is a test plan for this package:

Extensibility

Ideas on how to extend the package for new log sources, new use cases.

Resources

The link below contains a table of all resources included in this package:
Topic attachments
I Attachment Action Size Date Who Comment
ApacheSyslogConfig.pdfpdf ApacheSyslogConfig.pdf manage 130.9 K 17 Jul 2017 - 19:43 SeemaKhan Apache Syslog Config
ApacheTomcatFileConfig.pdfpdf ApacheTomcatFileConfig.pdf manage 278.7 K 26 Apr 2017 - 23:05 NellieWang Appache Tomcat File Configuration file
ApacheTomcatlogFile.PNGPNG ApacheTomcatlogFile.PNG manage 271.7 K 24 Apr 2017 - 17:52 NellieWang  
IIS-7-7.5-8.0-IPv6-issue.pngpng IIS-7-7.5-8.0-IPv6-issue.png manage 7.2 K 10 May 2017 - 17:45 SeemaKhan IIS version 7.0,7.5,8.0 logs show IPv6 address by default
L1-App-Mon-batch-file-install.pngpng L1-App-Mon-batch-file-install.png manage 102.1 K 03 May 2017 - 23:39 SeemaKhan  
L1-App-Mon-package-Uninstall-and-Delete.pngpng L1-App-Mon-package-Uninstall-and-Delete.png manage 34.8 K 04 May 2017 - 00:20 SeemaKhan Uninstall and Delete package
L1-App-Mon-package-install.pngpng L1-App-Mon-package-install.png manage 27.1 K 03 May 2017 - 23:49 SeemaKhan L1-App Mon package
MicrosoftIISFileConfig.pdfpdf MicrosoftIISFileConfig.pdf manage 236.0 K 26 Apr 2017 - 23:00 NellieWang Microsoft IIS File Configuration Document
iis_parser_overide_activate.zipzip iis_parser_overide_activate.zip manage 0.9 K 19 Feb 2016 - 17:20 SeanDavies Microsoft Internet Information Server Parser Override for Activate L1 Web Server Monitoring
Topic revision: r30 - 16 May 2018, EstebanHerrera


 


Activate Wiki 2.1.0.0

This site is powered by FoswikiCopyright &© by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback