Table of Contents |
---|
Create an Azure Sentinel Instance
Getting Started
Once we have created an account in Azure, we can create a sentinel instance. See https://azure.microsoft.com/en-ca/services/azure-sentinel/
Login via: https://portal.azure.com/#home
You will first need to first search for Sentinel and then create a workspace. Then you can add a sentinel instance to your workspace.
Collect Data
Connectors
Kubernetes
Alcide kAudit (Preview)
Alcide kAudit connector automatically exports your Kubernetes cluster audit logs into Azure Sentinel in real time. The kAudit connector provides enhanced visibility and observability into your Kubernetes audit logs. Alcide kAudit gives you robust security and monitoring capabilities for forensics purposes.
For more information about connecting to Azure Sentinel, see Connect Alcide kAudit to Azure Sentinel.
Data ingestion method: Azure Functions and the REST API.
Data Collection
Azure Sentinel has a number of connectors that can be used to send data and logs to Azure.
Kubernetes Connector
We can use fluent-bit's build in azure connector to send data to Azure. See Fluentbit to Azure Analytics.
Common Event Format
Common Event Format (CEF) is an industry standard format on top of Syslog messages, used by many security vendors to allow event interoperability among different platforms. By connecting your CEF logs to Azure Sentinel, you can take advantage of search & correlation, alerting, and threat intelligence enrichment for each log.
...
Syslog Connector
https://docs.microsoft.com/en-us/azure/sentinel/connect-syslog
Install the Linux agent
Code Block |
---|
wget https://raw.githubusercontent.com/Microsoft/OMS-Agent-for-Linux/master/installer/scripts/onboard_agent.sh && sh onboard_agent.sh -w xxx -s xxx -d opinsights.azure.com |
...
Code Block | ||
---|---|---|
| ||
--2021-07-02 10:47:00-- https://raw.githubusercontent.com/Microsoft/OMS-Agent-for-Linux/master/installer/scripts/onboard_agent.sh Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ... Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 3184 (3.1K) [text/plain] Saving to: 'onboard_agent.sh' onboard_agent.sh 100%[===================>] 3.11K --.-KB/s in 0s 2021-07-02 10:47:00 (9.09 MB/s) - 'onboard_agent.sh' saved [3184/3184] --2021-07-02 10:47:00-- https://github.com/microsoft... Shell bundle exiting with code 0 |
Uninstall
Code Block |
---|
$ sudo /opt/microsoft/omsagent/bin/uninstall # Purge - might be the better way to uninstall $ wget -O onboard_agent.sh https://raw.githubusercontent.com/Microsoft/OMS-Agent-for-Linux/releasesmaster/installer/downloadscripts/OMSAgentonboard_v1.13.35-0/omsagent-1.13.35-0.universal.x64.sh Resolving github.com (github.com)... 140.82.113.4 Connecting to github.com (github.com)|140.82.113.4|:443... connected. HTTP request sent, awaiting response... 302 Found Location: https://github-releases.githubusercontent.com/43709699/32a20280-8351-11eb-9b26-60e519b0d475?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210702%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210702T144616Z&X-Amz-Expires=300&X-Amz-Signature=b0b831e92322e2228dc25293996461a88c04ee282172d91117006eb0a88609aa&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=43709699&response-content-disposition=attachment%3B%20filename%3Domsagent-1.13.35-0.universal.x64.sh&response-content-type=application%2Foctet-stream [following] --2021-07-02 10:47:00-- https://github-releases.githubusercontent.com/43709699/32a20280-8351-11eb-9b26-60e519b0d475?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210702%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210702T144616Z&X-Amz-Expires=300&X-Amz-Signature=b0b831e92322e2228dc25293996461a88c04ee282172d91117006eb0a88609aa&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=43709699&response-content-disposition=attachment%3B%20filename%3Domsagent-1.13.35-0.universal.x64.sh&response-content-type=application%2Foctet-stream Resolving github-releases.githubusercontent.com (github-releases.githubusercontent.com)... 185.199.110.154, 185.199.111.154, 185.199.109.154, ... Connecting to github-releases.githubusercontent.com (github-releases.githubusercontent.com)|185.199.110.154|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 141916898 (135M) [application/octet-stream] Saving to: 'omsagent-1.13.35-0.universal.x64.sh' omsagent-1.13.35-0. 100%[===================>] 135.34M 5.79MB/s in 23s 2021-07-02 10:47:24 (5.79 MB/s) - 'omsagent-1.13.35-0.universal.x64.sh' saved [141916898/141916898] Checking host architecture ... Extracting... ----- Upgrading package: omi (omi-1.6.4-0.ulinux.x64) ----- Selecting previously unselected package omi. (Reading database ... 146300 files and directories currently installed.) Preparing to unpack 110/omi-1.6.4-0.ulinux.x64.deb ... Creating omiusers group ... Creating omi group ... Creating omi service account ... Unpacking omi (1.6.4.0) ... Setting up omi (1.6.4.0) ... Generating a RSA private key .............+++++ ....................................................................................+++++ writing new private key to '/etc/opt/omi/ssl/omikey.pem' ----- 2021-07-02 10:47:33 : Crontab not configured to update omi keytab automatically. Skip unconfigure 2021-07-02 10:47:33 : Crontab configured to update omi keytab automatically Checking if cron is installed... Checking if cron/crond service is started... Set up a cron job to OMI logrotate every 15 minutes Configuring OMI service ... Created symlink /etc/systemd/system/multi-user.target.wants/omid.service → /lib/systemd/system/omid.service. Trying to start omi with systemctl omi is started. Processing triggers for systemd (245.4-4ubuntu3.2) ... Trying to stop omi with systemctl omi is stopped. Trying to start omi with systemctl omi is started. ----- Upgrading package: scx (scx-1.6.4-7.universal.x64) ----- Selecting previously unselected package scx. (Reading database ... 146363 files and directories currently installed.) Preparing to unpack .../scx-1.6.4-7.universal.x64.deb ... Unpacking scx (1.6.4.7) ... Setting up scx (1.6.4.7) ... Generating certificate with hostname="deepthought" Trying to stop omi with systemctl omi is stopped. Trying to start omi with systemctl omi is started. ----- Upgrading package: omsagent (omsagent-1.13.35-0.universal.x64) ----- Selecting previously unselected package omsagent. (Reading database ... 146408 files and directories currently installed.) Preparing to unpack .../omsagent-1.13.35-0.universal.x64.deb ... Creating omsagent group ... Creating omsagent service account ... Creating nxautomation group ... Creating nxautomation service account ... Unpacking omsagent (1.13.35.0) ... Setting up omsagent (1.13.35.0) ... -e info Reading onboarding params from: /etc/omsagent-onboard.conf Workspace bda5ca3c-70bc-41b5-9c6b-9daa6ed318ba already onboarded and agent is running. Symbolic links have not been created; re-onboarding to create them info Generating certificate ... -e info Agent GUID is d9f64fc4-0ecf-4f92-8f8b-18919fabb8c0 -e info Onboarding success Configure syslog... Configuring rsyslog for OMS logging Restarting service: rsyslog Configure heartbeat monitoring agent... Configure log rotate for workspace bda5ca3c-70bc-41b5-9c6b-9daa6ed318ba... INFO: Configuring OMS agent service bda5ca3c-70bc-41b5-9c6b-9daa6ed318ba ... -e error MetaConfig generation script not available at /opt/microsoft/omsconfig/Scripts/python3/OMS_MetaConfigHelper.py Trying to stop omi with systemctl omi is stopped. Trying to start omi with systemctl omi is started. Configure log rotate for workspace bda5ca3c-70bc-41b5-9c6b-9daa6ed318ba... Processing triggers for systemd (245.4-4ubuntu3.2) ... Applying Syslog conf hotfix... ----- Upgrading package: omsconfig (omsconfig-1.1.1-930.x64) ----- Selecting previously unselected package omsconfig. (Reading database ... 156339 files and directories currently installed.) Preparing to unpack .../omsconfig-1.1.1-930.x64.deb ... Using python3 Cleanning up existing dsc_hosts... chmod: cannot access '/opt/dsc': No such file or directory Cleaned up existing dsc_hosts... Unpacking omsconfig (1.1.1.930) ... Setting up omsconfig (1.1.1.930) ... Using python3 Running python3, python version is , python3 /opt/microsoft/omsconfig/Scripts/python3/nxDSCLog.py:53: SyntaxWarning: "is" with a literal. Did you mean "=="? if message is None or len(message) is 0: VERBOSE from InstallModule.py: Extracting module zip file from /opt/microsoft/omsconfig/module_packages/nx_1.0.zip to /opt/microsoft/omsconfig/modules VERBOSE from InstallModule.py: Installing resource MSFT_nxPackageResource VERBOSE from InstallModule.py: Updated permissions of file: /opt/omi/lib/libMSFT_nxPackageResource_root-oms.so to 0o644 VERBOSE from InstallModule.py: Updated permissions of file: /opt/dsc/lib/MSFT_nxPackageResource/libMSFT_nxPackageResource.so to 0o644 VERBOSE from InstallModule.py: Updated permissions of file: /etc/opt/omi/conf/omiregister/root-oms/MSFT_nxPackageResource.reg to 0o644 VERBOSE from InstallModule.py: Installing resource MSFT_nxServiceResource VERBOSE from InstallModule.py: Updated permissions of file: /opt/omi/lib/libMSFT_nxServiceResource_root-oms.so to 0o644 VERBOSE from InstallModule.py: Updated permissions of file: /opt/dsc/lib/MSFT_nxServiceResource/libMSFT_nxServiceResource.so to 0o644 VERBOSE from InstallModule.py: Updated permissions of file: /etc/opt/omi/conf/omiregister/root-oms/MSFT_nxServiceResource.reg to 0o644 VERBOSE from InstallModule.py: Installing resource MSFT_nxAvailableUpdatesResource VERBOSE from InstallModule.py: Updated permissions of file: /opt/omi/lib/libMSFT_nxAvailableUpdatesResource_root-oms.so to 0o644 VERBOSE from InstallModule.py: Updated permissions of file: /opt/dsc/lib/MSFT_nxAvailableUpdatesResource/libMSFT_nxAvailableUpdatesResource.so to 0o644 VERBOSE from InstallModule.py: Updated permissions of file: /etc/opt/omi/conf/omiregister/root-oms/MSFT_nxAvailableUpdatesResource.reg to 0o644 VERBOSE from InstallModule.py: Installing resource MSFT_nxUserResource VERBOSE from InstallModule.py: Updated permissions of file: /opt/omi/lib/libMSFT_nxUserResource_root-oms.so to 0o644 VERBOSE from InstallModule.py: Updated permissions of file: /opt/dsc/lib/MSFT_nxUserResource/libMSFT_nxUserResource.so to 0o644 VERBOSE from InstallModule.py: Updated permissions of file: /etc/opt/omi/conf/omiregister/root-oms/MSFT_nxUserResource.reg to 0o644 VERBOSE from InstallModule.py: Installing resource MSFT_nxGroupResource VERBOSE from InstallModule.py: Updated permissions of file: /opt/omi/lib/libMSFT_nxGroupResource_root-oms.so to 0o644 VERBOSE from InstallModule.py: Updated permissions of file: /opt/dsc/lib/MSFT_nxGroupResource/libMSFT_nxGroupResource.so to 0o644 VERBOSE from InstallModule.py: Updated permissions of file: /etc/opt/omi/conf/omiregister/root-oms/MSFT_nxGroupResource.reg to 0o644 The result code is 0 VERBOSE from InstallModule.py: Extracting module zip file from /opt/microsoft/omsconfig/module_packages/nxOMSPerfCounter_2.3.zip to /opt/microsoft/omsconfig/modules VERBOSE from InstallModule.py: Installing resource MSFT_nxOMSPerfCounterResource VERBOSE from InstallModule.py: Updated permissions of file: /opt/omi/lib/libMSFT_nxOMSPerfCounterResource_root-oms.so to 0o644 VERBOSE from InstallModule.py: Updated permissions of file: /opt/dsc/lib/MSFT_nxOMSPerfCounterResource/libMSFT_nxOMSPerfCounterResource.so to 0o644 VERBOSE from InstallModule.py: Updated permissions of file: /etc/opt/omi/conf/omiregister/root-oms/MSFT_nxOMSPerfCounterResource.reg to 0o644 The result code is 0 VERBOSE from InstallModule.py: Extracting module zip file from /opt/microsoft/omsconfig/module_packages/nxOMSSyslog_2.5.zip to /opt/microsoft/omsconfig/modules VERBOSE from InstallModule.py: Installing resource MSFT_nxOMSSyslogResource VERBOSE from InstallModule.py: Updated permissions of file: /opt/omi/lib/libMSFT_nxOMSSyslogResource_root-oms.so to 0o644 VERBOSE from InstallModule.py: Updated permissions of file: /opt/dsc/lib/MSFT_nxOMSSyslogResource/libMSFT_nxOMSSyslogResource.so to 0o644 VERBOSE from InstallModule.py: Updated permissions of file: /etc/opt/omi/conf/omiregister/root-oms/MSFT_nxOMSSyslogResource.reg to 0o644 The result code is 0 VERBOSE from InstallModule.py: Extracting module zip file from /opt/microsoft/omsconfig/module_packages/nxOMSSudoCustomLog_2.7.zip to /opt/microsoft/omsconfig/modules VERBOSE from InstallModule.py: Installing resource MSFT_nxOMSSudoCustomLogResource VERBOSE from InstallModule.py: Updated permissions of file: /opt/omi/lib/libMSFT_nxOMSSudoCustomLogResource_root-oms.so to 0o644 VERBOSE from InstallModule.py: Updated permissions of file: /opt/dsc/lib/MSFT_nxOMSSudoCustomLogResource/libMSFT_nxOMSSudoCustomLogResource.so to 0o644 VERBOSE from InstallModule.py: Updated permissions of file: /etc/opt/omi/conf/omiregister/root-oms/MSFT_nxOMSSudoCustomLogResource.reg to 0o644 The result code is 0 VERBOSE from InstallModule.py: Extracting module zip file from /opt/microsoft/omsconfig/module_packages/nxOMSKeyMgmt_1.0.zip to /opt/microsoft/omsconfig/modules VERBOSE from InstallModule.py: Installing resource MSFT_nxOMSKeyMgmtResource VERBOSE from InstallModule.py: Updated permissions of file: /opt/omi/lib/libMSFT_nxOMSKeyMgmtResource_root-oms.so to 0o644 VERBOSE from InstallModule.py: Updated permissions of file: /opt/dsc/lib/MSFT_nxOMSKeyMgmtResource/libMSFT_nxOMSKeyMgmtResource.so to 0o644 VERBOSE from InstallModule.py: Updated permissions of file: /etc/opt/omi/conf/omiregister/root-oms/MSFT_nxOMSKeyMgmtResource.reg to 0o644 The result code is 0 VERBOSE from InstallModule.py: Extracting module zip file from /opt/microsoft/omsconfig/module_packages/nxFileInventory_1.3.zip to /opt/microsoft/omsconfig/modules VERBOSE from InstallModule.py: Installing resource MSFT_nxFileInventoryResource VERBOSE from InstallModule.py: Updated permissions of file: /opt/omi/lib/libMSFT_nxFileInventoryResource_root-oms.so to 0o644 VERBOSE from InstallModule.py: Updated permissions of file: /opt/dsc/lib/MSFT_nxFileInventoryResource/libMSFT_nxFileInventoryResource.so to 0o644 VERBOSE from InstallModule.py: Updated permissions of file: /etc/opt/omi/conf/omiregister/root-oms/MSFT_nxFileInventoryResource.reg to 0o644 The result code is 0 -bash: /opt/microsoft/omsconfig/Scripts/InstallModule.py: /usr/bin/python: bad interpreter: No such file or directory gpg: keybox '/etc/opt/omi/conf/omsconfig/keymgmtring.gpg' created gpg: directory '/etc/opt/omi/conf/omsconfig/.gnupg' created gpg: /etc/opt/omi/conf/omsconfig/.gnupg/trustdb.gpg: trustdb created gpg: key C4EC49E544BC4178: public key "Microsoft (Release Signing) <msgpgkey@microsoft.com>" imported gpg: Total number processed: 1 gpg: imported: 1 gpg: keybox '/etc/opt/omi/conf/omsconfig/keyring.gpg' created gpg: key 20541A3DDE321294: public key "Microsoft (Release Signing) <dscgpgkey@microsoft.com>" imported gpg: Total number processed: 1 gpg: imported: 1 Trying to stop omi with systemctl omi is stopped. Trying to start omi with systemctl omi is started. Running python3 VERBOSE from OMS_MetaConfigHelper.py: OMS config path being read: /etc/opt/microsoft/omsagent/bda5ca3c-70bc-41b5-9c6b-9daa6ed318ba/conf/omsadmin.conf VERBOSE from OMS_MetaConfigHelper.py: Output from3: /opt/microsoft/omsconfig/Scripts/python3/SetDscLocalConfigurationManager.py -configurationmof /etc/opt/omi/conf/omsconfig/generated_meta_config.mof: Opened the dsc host lock file at the path '/opt/dsc/dsc_host_lock' [2021/07/02 10:48:09] [1294393] [INFO] [0] [/opt/microsoft/omsconfig/Scripts/python3/SetDscLocalConfigurationManager.py:0] dsc_host lock file is acquired by : SendMetaConfigurationApply Operation SendMetaConfigurationApply completed successfully. Operation was successful. Operation SendMetaConfigurationApply completed successfully. Operation was successful. VERBOSE from OMS_MetaConfigHelper.py: Successfully configured omsconfig. Applying DSC nxOMSSyslog hotfix... Trying to stop omi with systemctl omi is stopped. Trying to start omi with systemctl omi is started. ----- Updating bundled packages ----- Checking if Apache is installed ... Apache not found, will not install Checking if Docker is installed... Docker version greater or equal than 17.* found. Docker agent will be installed Extracting... Updating container agent ... ----- Updating package: docker-cimprov (docker-cimprov-1.0.0-37.universal.x86_64) ----- Selecting previously unselected package docker-cimprov. (Reading database ... 156440 files and directories currently installed.) Preparing to unpack docker-cimprov-1.0.0-37.universal.x86_64.deb ... Unpacking docker-cimprov (1.0.0.37) ... Setting up docker-cimprov (1.0.0.37) ... Trying to stop omi with systemctl omi is stopped. Trying to start omi with systemctl omi is started. instance of Container_HostInventory { [Key] InstanceID=55def11b-96b9-495b-9f1d-6cb84bd3b3f3 Computer=deepthought DockerVersion=19.03.13 OperatingSystem=Ubuntu Core 18 Volume=local Network=bridge host ipvlan macvlan null overlay NodeRole=Not Orchestrated OrchestratorType=None } Checking if MySQL is installed ... MySQL not found, will not install Extracting... Updating auoms ... ----- Updating package: auoms (auoms-2.3.4-31.universal.x64) ----- Selecting previously unselected package auoms. (Reading database ... 156465 files and directories currently installed.) Preparing to unpack auoms-2.3.4-31.universal.x64.deb ... Unpacking auoms (2.3.4.31) ... Setting up auoms (2.3.4.31) ... Processing triggers for systemd (245.4-4ubuntu3.2) ... OMS Troubleshooter is installed. You can run the Troubleshooter with the following command: $ sudo /opt/microsoft/omsagent/bin/troubleshooter Shell bundle exiting with code 0 |
Queries
Code Block |
---|
Syslog
| where Facility in ("authpriv","auth")
| extend c = extract( "Accepted\\s(publickey|password|keyboard-interactive/pam)\\sfor ([^\\s]+)",1,SyslogMessage)
| where isnotempty(c)
| count
Syslog
| where Facility in ("authpriv","auth")
| where SyslogMessage contains "authentication failure "
| count
|
References
agent.sh && sudo sh onboard_agent.sh --purge
|
Querying the Logs
From the Log screen, we can create queries to search our logs.
Query Examples
Code Block |
---|
Syslog
| where Facility in ("authpriv","auth")
| extend c = extract( "Accepted\\s(publickey|password|keyboard-interactive/pam)\\sfor ([^\\s]+)",1,SyslogMessage)
| where isnotempty(c)
Syslog
| where Facility in ("authpriv","auth")
| where SyslogMessage contains "authentication failure "
// Search in multiple tables
// Search both Syslog and Event tables for the term "login".
search in (Syslog, Event) "authentication failure"
| where TimeGenerated > ago(24h) // return records from the last 24 hours
// All Syslog with errors
// Last 100 Syslog with erros.
Syslog
| where SeverityLevel == "err" or SeverityLevel == "error"
| top 100 by TimeGenerated desc
//login errors
Syslog
| where SeverityLevel == "err" or SeverityLevel == "error"
| where Facility == "auth"
| where SyslogMessage contains "kex_exchange_identification"
//any auth errors
Syslog
| where SeverityLevel == "err" or SeverityLevel == "error"
| where Facility == "auth"
//failed login
Syslog
| where Facility == "auth"
| where SyslogMessage contains "Failed password"
| order by EventTime asc
//failed login with extraction of fields
Syslog
| where Facility == "auth"
| where SyslogMessage contains "Failed password"
| extend user = extract("Failed password for invalid user (.*) from (.*) port (.*) (.*)", 1, SyslogMessage, typeof(string))
| extend remoteIp = extract("Failed password for invalid user (.*) from (.*) port (.*) (.*)", 2, SyslogMessage, typeof(string))
| extend port = extract("Failed password for invalid user (.*) from (.*) port (.*) (.*)", 3, SyslogMessage, typeof(string))
| extend protocol = extract("Failed password for invalid user (.*) from (.*) port (.*) (.*)", 4, SyslogMessage, typeof(string))
| order by EventTime asc
//kubernetes logs gathered using fluent-bit
fluentbit_CL
| where kubernetes_labels_app_s contains "ubuntu"
| where log_s contains "Failed password"
|
Analytics
Navigate to the Analytics page by clicking Analytics in the left side menu.
Create a Rule
To create a new rule, simply click the Create pull down and select Scheduled Query Rule.
Define the general information about the rule: name, description. You also have the ability to categorize the rule into group of tactics.
Define the query, how often to run the query and the alert threshold.
Enable incident creation
Define Response
Review and Save
Incidents
Incidents created from a analytics rule, can be displayed on the Incidents screen.
View details of the triggered incident by clicking the incident and clicking View Full Details.
Clicking Events will display the log records that triggered the incident.
References
Reference | URL |
---|---|
What is Azure Sentinel and why you should care | Azure Tips and Tricks | https://www.youtube.com/watch?v=dRpOR2GpL1s |
Azure Sentinel | https://azure.microsoft.com/en-ca/services/azure-sentinel/ |
Azure Portal | https://portal.azure.com/#home |
Samples for queries for Azure Data Explorer and Azure Monitor | https://docs.microsoft.com/en-us/azure/data-explorer/kusto/query/samples?pivots=azuredataexplorer |
Tutorial: Create custom analytics rules to detect threats | https://docs.microsoft.com/en-us/azure/sentinel/tutorial-detect-threats-custom |
Azure Sentinel Lab Series | Setup Syslog Collector and install Azure Sentinel Agent | EP1 | https://www.youtube.com/watch?v=KJbIH7egdVI |
Reference | URL |
What is Azure Sentinel and why you should care | Azure Tips and Tricks | https://www.youtube.com/watch?v=dRpOR2GpL1s |
Azure Sentinel | https://azure.microsoft.com/en-ca/services/azure-sentinel/ |
Azure Portal | https://portal.azure.com/#home |