Configuring CEF via AMA
Using the new AMA agent to forward CEF events into Sentinel
Overview of Microsoft Agent Changes
Microsoft have been developing the new Azure Monitoring Agent (AMA) to replace the MMA/OMS agents used previously to collect events from a host or syslog from network devices. The MMA/OMS agents are set to be depreciated on the 31st August 2024. Although this gives Microsoft customers plenty of time to switch over, it is a good idea to start planning the migration from the legacy agents to the new AMA a soon as possible.
To understand the differences between the agents, Microsoft has published the table below, to indicate any differences/similarities between the new AMA and the Legacy MMA.
Source: https://learn.microsoft.com/en-us/azure/azure-monitor/agents/agents-overview
As the table shows, the new AMA covers the same features as the legacy agent however it is useful to run through the Microsoft migration guidance and tools below to ensure any monitoring disruptions are kept at a minimum.
Migration Guidance:
https://learn.microsoft.com/en-us/azure/azure-monitor/agents/azure-monitor-agent-migration
Migration Tools:
https://learn.microsoft.com/en-us/azure/azure-monitor/agents/azure-monitor-agent-migration-tools
Legacy Collection of CEF Syslog via Legacy OMS Agent
Previously collecting CEF formatted Syslog logs would consist of using a Linux host running the OMS Agent and using Rsyslog to forward events to the relevant ports 25224 (syslog) or 25226 (CEF) which the OMS agent was listening on. A diagram below describes the overall architecture of this solution for an on-premise solution:
Source: https://learn.microsoft.com/en-us/azure/sentinel/connect-common-event-format
The deployment of the OMS agent and collection of syslog or CEF syslog didn’t change between on-premise log forwarders and the azure based log forwarders. However, with the new AMA agent, this adds a little more complexity and in the next section I will go over one option to simplify the deployment.
Additional Step for On-Premise Log Forwarders
When deploying a log forwarder on-premise it is required to deploy the Azure ARC agent on the server prior to creating the Data Collection Rule, otherwise the server will not be able to be selected when configuring the resource section of the DCR.
The Azure ARC agent is used to build a connection between an on-premise server and Azure. There are a couple of ways to create the ARC deployment script which depends if you are deploying ARC to a large amount of servers or just one. To configure the ARC deployment script follow the steps below:
Open Azure portal by going to portal.azure.com
Go to Azure ARC
On the overview section select “Add your infrastructure for free”
Select Servers
Choose either add a single server or multiple (For this blog, we will add a single server)
Provide the subscription and resource group where this server will be located. It is recommended that you create a new resource group for your Azure ARC servers than use an existing one.
Select the Region, Server OS, and the Connectivity Method
Add any tags that you require
Download the script and run the script on the Linux server
When prompted go to the authorization link and sign-in with your credentials.
Once completed you should see the server in the Azure ARC Servers section, once the on-premise log forward shows in this section you can proceed to creating the Data Collection Rule to collect CEF logs above.
Collecting CEF Logs With AMA
To deploy the AMA agent on the log forwarder, a Data Collection Rule (DCR) will be used. There are several ways to configure the DCR, however, I will show the easiest way to configure this.
Open Azure portal by going to portal.azure.com
Go to your Sentinel Workspace
Click on Data Connectors and open the connector “Common Event Format (CEF) via AMA (Preview)”
Select “Create Data Collection Rule”
Provide a rule name, the Subscription and Resource this rule should be located
Select the log forwarder virtual machine under the Resources section
On the Collect section, enable the Facility the CEF events will be forwarded to from the syslog devices, do not collect logs from other facilities that are not forwarding syslog.
Finish by Reviewing and Creating the rule.
This will deploy the AMA agent for you on the Linux forwarder, next is to run the following command on the Linux server:
sudo wget -O Forwarder_AMA_installer.py https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/DataConnectors/Syslog/Forwarder_AMA_installer.py&&sudo python Forwarder_AMA_installer.py
To validate the log forwarder is configured successfully, you can run the following command to generate a test message on a specific facility:
logger -p local3.warn -P 514 -n 127.0.0.1 --rfc3164 -t CEF "0|Mock-test|MOCK|common=event-format-test|end|TRAFFIC|1|rt=$common=event-formatted-receive_time" Once this is ran, the log should appear in the table “CommonSecurityLog”.
Note: As a Data Collection Rule (DCR) will be used it is useful to forward CEF events to a separate facility to the one used for standard syslog. This will help avoid syslog events being collected by the wrong data collection rule. For example, you could setup forwarding your syslog and CEF to the following facilities:
AV Logs → Local 1 Facility
Proxy Logs → Local 2 Facility
Firewall CEF Events → Local 3 Facility