Skip to content

Added support for customizable resource type. #19

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
vishaalcharpe merged 1 commit into from
Jul 3, 2025
Merged

Added support for customizable resource type. #19

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,3 +18,11 @@
- Dont send event metadata by default. Add config include_metadata_keys for including custom metadata
## 2.0.1
- Fix incomplete json metadata error.
## 2.0.2
- Added domain support for GovCloud
## 2.0.3
- Added validation for domain support
## 2.0.4
- Added resource type in metadata.
## 2.0.5
- Added support for customizable resource type.
22 changes: 11 additions & 11 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,17 +29,17 @@ The allowed values for portal_domain are ["logicmonitor.com", "lmgov.us", "qa-lm

## Important options

| Option | Description| Default |
| --- | --- | --- |
| batch_size | Event batch size to send to LM Logs.| 100 |
| message_key | Key that will be used by the plugin as the system key | "message" |
| lm_property | Key that will be used by LM to match resource based on property | "system.hostname" |
| keep_timestamp | If false, LM Logs will use the ingestion timestamp as the event timestamp | true |
| timestamp_is_key | If true, LM Logs will use a specified key as the event timestamp | false |
| timestamp_key | If timestamp_is_key is set, LM Logs will use this key in the event as the timestamp | "logtimestamp" |
| include_metadata | If true, all metadata fields will be sent to LM Logs | false |
| include_metadata_keys | Array of json keys for which plugin looks for these keys and adds as event meatadata. A dot "." can be used to add nested subjson. If config `include_metadata` is set to true, all metadata will be sent regardless of this config. | [] |

| Option | Description | Default |
|-----------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------|
| batch_size | Event batch size to send to LM Logs. | 100 |
| message_key | Key that will be used by the plugin as the system key | "message" |
| lm_property | Key that will be used by LM to match resource based on property | "system.hostname" |
| keep_timestamp | If false, LM Logs will use the ingestion timestamp as the event timestamp | true |
| timestamp_is_key | If true, LM Logs will use a specified key as the event timestamp | false |
| timestamp_key | If timestamp_is_key is set, LM Logs will use this key in the event as the timestamp | "logtimestamp" |
| include_metadata | If true, all metadata fields will be sent to LM Logs | false |
| include_metadata_keys | Array of json keys for which plugin looks for these keys and adds as event meatadata. A dot "." can be used to add nested subjson. If config `include_metadata` is set to true, all metadata will be sent regardless of this config. | [] |
| resource_type | If a Resource Type is explicitly specified, that value will be statically applied to all ingested logs. If set to `predef.externalResourceType`, the Resource Type will be assigned dynamically based on the `predef.externalResourceType` property set for the specific resource the logs are mapped to in LM. If left blank, the Resource Type field will remain unset in the ingested logs. | "" |
See the [source code](lib/logstash/outputs/lmlogs.rb) for the full list of options

The syntax for `message_key` and `source_key` values are available in the [Logstash Event API Documentation](https://www.elastic.co/guide/en/logstash/current/event-api.html)
Expand Down
8 changes: 7 additions & 1 deletion lib/logstash/outputs/lmlogs.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,9 @@ class InvalidHTTPConfigError < StandardError; end
# json keys for which plugin looks for these keys and adds as event meatadata. A dot "." can be used to add nested subjson.
config :include_metadata_keys, :validate => :array, :required => false, :default => []

# Customised resource_type to map logs to existing LM resources
config :resource_type, :validate => :string, :required => false

@@MAX_PAYLOAD_SIZE = 8*1024*1024

# For developer debugging.
Expand Down Expand Up @@ -308,7 +311,10 @@ def processEvent(event)
if @include_metadata
lmlogs_event = event_json
lmlogs_event.delete("@timestamp") # remove redundant timestamp field
lmlogs_event["_resource.type"]="Logstash"
unless @resource_type.to_s.strip.empty?
lmlogs_event["_resource.type"] = @resource_type
end

if lmlogs_event.dig("event", "original") != nil
lmlogs_event["event"].delete("original") # remove redundant log field
end
Expand Down
2 changes: 1 addition & 1 deletion lib/logstash/outputs/version.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# frozen_string_literal: true

module LmLogsLogstashPlugin
VERSION = '2.0.4'
VERSION = '2.0.5'
end
3 changes: 1 addition & 2 deletions spec/outputs/metadata_spec.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,8 +76,7 @@ def check_same_hash(h1,h2)
"nested" => {"nested2" => {"nested3" => "value",
"nested3b" => "value"},
"nested_ignored" => "somevalue"
},
"_resource.type"=>"Logstash"
}
}
puts " actual : #{constructed_event} \n expected : #{expected_event}"
puts " hash diff : #{Hashdiff.diff(constructed_event,expected_event)}"
Expand Down