Your Credentials
To determine your credentials in Azure:
- locate your Log Analytics workspace.
- Select Agents management.
- To the right of Workspace ID, select the copy icon, and then paste the ID as the value of the Customer ID variable.
- To the right of Primary Key, select the copy icon, and then paste the ID as the value of the Shared Key variable.
Azure HTTP Data Collector API
https://docs.microsoft.com/en-us/azure/azure-monitor/logs/data-collector-api
Request Body
The body of the message must be in JSON.
It must include one or more records with the property name and value pairs in the following format. The property name can only contain letters, numbers, and underscore (_).
JSON [ { "property 1": "value1", "property 2": "value2", "property 3": "value3", "property 4": "value4" } ]
Sample Script/Program
Sample powershell script to push data to your workspace.
LogType: MyRecordType_CL
Querying Submitted Data
Kafka Connect with Azure Log Analytics Sink Connector
https://www.confluent.de/hub/chaitalisagesh/kafka-connect-log-analytics
Log Analytics Agent for Linux
https://docs.microsoft.com/en-us/azure/azure-monitor/agents/agent-linux
Pushes data to Azure Data Collector API.
Logstash
https://docs.microsoft.com/en-us/azure/sentinel/connect-logstash
Pushes data to Azure Data Collector API.
"The components for log parsing are different per logging tool. Fluentd uses standard built-in parsers (JSON, regex, csv etc.) and Logstash uses plugins for this. This makes Fluentd favorable over Logstash, because it does not need extra plugins installed, making the architecture more complex and more prone to errors"
Fluent-bit
https://docs.fluentbit.io/manual/pipeline/outputs/azure
Pushes data to Azure Data Collector API.
References
Reference | URL |
---|---|
Azure HTTP Data Collector API | https://docs.microsoft.com/en-us/azure/azure-monitor/logs/data-collector-api |
Azure Log Analytics Sink Connector | https://www.confluent.de/hub/chaitalisagesh/kafka-connect-log-analytics |
Log Analytics Agent for Linux | https://docs.microsoft.com/en-us/azure/azure-monitor/agents/agent-linux |
Logstash | https://docs.microsoft.com/en-us/azure/sentinel/connect-logstash |
Fluent-bit | https://docs.fluentbit.io/manual/pipeline/outputs/azure |
Kubernetes Logging: Comparing Fluentd vs. Logstash |