I am not able to adjust the parser to correctly receive messages in elasticsearch.... the message is arriving with several backslashes and I am not able to adjust it.
Could you kindly help me?
Config:
input
<source>
@type sqs_poll
tag sqs.message
aws_access_key "#{ENV['AWS_ACCESS_KEY_ID']}"
aws_secret_key "#{ENV['AWS_SECRET_ACCESS_KEY']}"
sqs_url https://xxxxxxx/SES-deliveries
</source>
<filter sqs.message>
@type parser
remove_prefix sqs
format json
key_name body
</filter>
ERROR:
```
2022-09-13 13:46:46 +0000 [error]: #0 SQS exception error="undefined method body' for \"{\\n \\\"Type\\\" : \\\"Notification\\\",\\n \\\"MessageId\\\" : \\\"e5e19ed8-8b49-56b2-9dfb-c542074ff55b\\\",\\n \\\"TopicArn\\\" : \\\"ARNxxxx:SES-deliveries\\\",\\n \\\"Message\\\" : \\\"{\\\\\\\"notificationType\\\\\\\":\\\\\\\"Delivery\\\\\\\",\\\\\\\"mail\\\\\\\":{\\\\\\\"timestamp\\\\\\\":\\\\\\\"2022-09-13T13:46:45.964Z\\\\\\\",\\\\\\\"source\\\\\\\":\\\\\\\"email@example.com\\\\\\\",\\\\\\\"sourceArn\\\\\\\":\\\\\\\myarrnxxxxxxx\\\\\\\",\\\\\\\"sourceIp\\\\\\\":\\\\\\\"18.169.148.178\\\\\\\",\\\\\\\"callerIdentity\\\\\\\":\\\\\\\"userxxxxxxx-222546\\\\\\\",\\\\\\\"sendingAccountId\\\\\\\":\\\\\\\"3545345345345\\\\\\\",\\\\\\\"messageId\\\\\\\":\\\\\\\"010b0183371a2d4c-266c2577-7178-4870-886d-76f370c0d68b-000000\\\\\\\",\\\\\\\"destination\\\\\\\":[\\\\\\\"user@example.com\\\\\\\"],\\\\\\\"headersTruncated\\\\\\\":false,\\\\\\\"headers\\\\\\\":[{\\\\\\\"name\\\\\\\":\\\\\\\"Received\\\\\\\",\\\\\\\"value\\\\\\\":\\\\\\\"from [127.0.0.1] (message header (UTC)\\\\\\\"},{\\\\\\\"name\\\\\\\":\\\\\\\"Content-Type\\\\\\\",\\\\\\\"value\\\\\\\":\\\\\\\"multipart/alternative; boundary=\\\\\\\\\\\\\\\"--_NmP-1592f316e080f725-Part_1\\\\\\\\\\\\\\\"\\\\\\\"},{\\\\\\\"name\\\\\\\":\\\\\\\"From\\\\\\\",\\\\\\\"value\\\\\\\":\\\\\\\"email@example.com\\\\\\\"},{\\\\\\\"name\\\\\\\":\\\\\\\"To\\\\\\\",\\\\\\\"value\\\\\\\":\\\\\\\"email@example.com\\\\\\\"},{\\\\\\\"name\\\\\\\":\\\\\\\"Reply-To\\\\\\\",\\\\\\\"value\\\\\\\":\\\\\\\"email@example.com\\\\\\\"},{\\\\\\\"name\\\\\\\":\\\\\\\"Subject\\\\\\\",\\\\\\\"value\\\\\\\":\\\\\\\"Strapi test mail to: email@example.com\\\\\\\\"},{\\\\\\\"name\\\\\\\":\\\\\\\"Message-ID\\\\\\\",\\\\\\\"value\\\\\\\":\\\\\\\"<5f8732f5-9545-004b-c687-9c3631684530@example.com\>\\\\\\\"},{\\\\\\\"name\\\\\\\":\\\\\\\"Date\\\\\\\",\\\\\\\"value\\\\\\\":\\\\\\\"Tue, 13 Sep 2022 13:46:45 +0000\\\\\\\"},{\\\\\\\"name\\\\\\\":\\\\\\\"MIME-Version\\\\\\\",\\\\\\\"value\\\\\\\":\\\\\\\"1.0\\\\\\\"}],\\\\\\\"commonHeaders\\\\\\\":{\\\\\\\"from\\\\\\\":[\\\\\\\"email@example.com\\\\\\\"],\\\\\\\"replyTo\\\\\\\":[\\\\\\\"email@example.com\\\\\\\"],\\\\\\\"date\\\\\\\":\\\\\\\"Tue, 13 Sep 2022 13:46:45 +0000\\\\\\\",\\\\\\\"to\\\\\\\":[\\\\\\\"email@example.com\\\\\\\"],\\\\\\\"messageId\\\\\\\":\\\\\\\"<5f8732f5-9545-004b-c687-9c3631684530@example.com>\\\\\\\",\\\\\\\"subject\\\\\\\":\\\\\\\"Strapi test mail to: email@example.co\\\\\\\"}},\\\\\\\"delivery\\\\\\\":{\\\\\\\"timestamp\\\\\\\":\\\\\\\"2022-09-13T13:46:46.540Z\\\\\\\",\\\\\\\"processingTimeMillis\\\\\\\":576,\\\\\\\"recipients\\\\\\\":[\\\\\\\"email@example.com\\\\\\\"],\\\\\\\"smtpResponse\\\\\\\":\\\\\\\"250 2.0.0 Ok: queued as 79FB77D3D7\\\\\\\",\\\\\\\"remoteMtaIp\\\\\\\":\\\\\\\"54.228.209.176\\\\\\\",\\\\\\\"reportingMTA\\\\\\\":\\\\\\\"d218-4.smtp-out.eu-west-2.amazonses.com\\\\\\\"}}\\\",\\n \\\"Timestamp\\\" : \\\"2022-09-13T13:46:46.598Z\\\",\\n \\\"SignatureVersion\\\" : \\\"1\\\",\\n \\\"Signature\\\" : \\\"ashjdiaushdiuhasiuh23423hiuhiudhsfsdfhiu3423423==\\\",\\n \\\"SigningCertURL\\\" : \\\"https://sns.eu-west-2.amazonaws.com/SimpleNotificationService-sfdfsdfsdfsdfdsfsdfsdf.pem\\\",\\n \\\"UnsubscribeURL\\\" : \\\"https://sns.eu-west-2.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=arn:aasdasdsadsadasdasd\\\"\\n}\":String\n\n 'body' => msg.body,\n ^^^^^" error_class="NoMethodError" 2022-09-13 13:46:46 +0000 [warn]: #0 suppressed same stacktrace 2022-09-13 13:46:46 +0000 [error]: #0 SQS exception error="undefined method body' for {"SenderId"=>"asdasdasdasdasd", "ApproximateFirstReceiveTimestamp"=>"13123123123123123", "ApproximateReceiveCount"=>"1", "SentTimestamp"=>"1663076806628"}:Aws::Xml::DefaultMap\n\n 'body' => msg.body,\n
I am not able to adjust the parser to correctly receive messages in elasticsearch.... the message is arriving with several backslashes and I am not able to adjust it.
Could you kindly help me?
Config:
input
2022-09-13 13:46:46 +0000 [error]: #0 SQS exception error="undefined method
body' for \"{\\n \\\"Type\\\" : \\\"Notification\\\",\\n \\\"MessageId\\\" : \\\"e5e19ed8-8b49-56b2-9dfb-c542074ff55b\\\",\\n \\\"TopicArn\\\" : \\\"ARNxxxx:SES-deliveries\\\",\\n \\\"Message\\\" : \\\"{\\\\\\\"notificationType\\\\\\\":\\\\\\\"Delivery\\\\\\\",\\\\\\\"mail\\\\\\\":{\\\\\\\"timestamp\\\\\\\":\\\\\\\"2022-09-13T13:46:45.964Z\\\\\\\",\\\\\\\"source\\\\\\\":\\\\\\\"email@example.com\\\\\\\",\\\\\\\"sourceArn\\\\\\\":\\\\\\\myarrnxxxxxxx\\\\\\\",\\\\\\\"sourceIp\\\\\\\":\\\\\\\"18.169.148.178\\\\\\\",\\\\\\\"callerIdentity\\\\\\\":\\\\\\\"userxxxxxxx-222546\\\\\\\",\\\\\\\"sendingAccountId\\\\\\\":\\\\\\\"3545345345345\\\\\\\",\\\\\\\"messageId\\\\\\\":\\\\\\\"010b0183371a2d4c-266c2577-7178-4870-886d-76f370c0d68b-000000\\\\\\\",\\\\\\\"destination\\\\\\\":[\\\\\\\"user@example.com\\\\\\\"],\\\\\\\"headersTruncated\\\\\\\":false,\\\\\\\"headers\\\\\\\":[{\\\\\\\"name\\\\\\\":\\\\\\\"Received\\\\\\\",\\\\\\\"value\\\\\\\":\\\\\\\"from [127.0.0.1] (message header (UTC)\\\\\\\"},{\\\\\\\"name\\\\\\\":\\\\\\\"Content-Type\\\\\\\",\\\\\\\"value\\\\\\\":\\\\\\\"multipart/alternative; boundary=\\\\\\\\\\\\\\\"--_NmP-1592f316e080f725-Part_1\\\\\\\\\\\\\\\"\\\\\\\"},{\\\\\\\"name\\\\\\\":\\\\\\\"From\\\\\\\",\\\\\\\"value\\\\\\\":\\\\\\\"email@example.com\\\\\\\"},{\\\\\\\"name\\\\\\\":\\\\\\\"To\\\\\\\",\\\\\\\"value\\\\\\\":\\\\\\\"email@example.com\\\\\\\"},{\\\\\\\"name\\\\\\\":\\\\\\\"Reply-To\\\\\\\",\\\\\\\"value\\\\\\\":\\\\\\\"email@example.com\\\\\\\"},{\\\\\\\"name\\\\\\\":\\\\\\\"Subject\\\\\\\",\\\\\\\"value\\\\\\\":\\\\\\\"Strapi test mail to: email@example.com\\\\\\\\"},{\\\\\\\"name\\\\\\\":\\\\\\\"Message-ID\\\\\\\",\\\\\\\"value\\\\\\\":\\\\\\\"<5f8732f5-9545-004b-c687-9c3631684530@example.com\>\\\\\\\"},{\\\\\\\"name\\\\\\\":\\\\\\\"Date\\\\\\\",\\\\\\\"value\\\\\\\":\\\\\\\"Tue, 13 Sep 2022 13:46:45 +0000\\\\\\\"},{\\\\\\\"name\\\\\\\":\\\\\\\"MIME-Version\\\\\\\",\\\\\\\"value\\\\\\\":\\\\\\\"1.0\\\\\\\"}],\\\\\\\"commonHeaders\\\\\\\":{\\\\\\\"from\\\\\\\":[\\\\\\\"email@example.com\\\\\\\"],\\\\\\\"replyTo\\\\\\\":[\\\\\\\"email@example.com\\\\\\\"],\\\\\\\"date\\\\\\\":\\\\\\\"Tue, 13 Sep 2022 13:46:45 +0000\\\\\\\",\\\\\\\"to\\\\\\\":[\\\\\\\"email@example.com\\\\\\\"],\\\\\\\"messageId\\\\\\\":\\\\\\\"<5f8732f5-9545-004b-c687-9c3631684530@example.com>\\\\\\\",\\\\\\\"subject\\\\\\\":\\\\\\\"Strapi test mail to: email@example.co\\\\\\\"}},\\\\\\\"delivery\\\\\\\":{\\\\\\\"timestamp\\\\\\\":\\\\\\\"2022-09-13T13:46:46.540Z\\\\\\\",\\\\\\\"processingTimeMillis\\\\\\\":576,\\\\\\\"recipients\\\\\\\":[\\\\\\\"email@example.com\\\\\\\"],\\\\\\\"smtpResponse\\\\\\\":\\\\\\\"250 2.0.0 Ok: queued as 79FB77D3D7\\\\\\\",\\\\\\\"remoteMtaIp\\\\\\\":\\\\\\\"54.228.209.176\\\\\\\",\\\\\\\"reportingMTA\\\\\\\":\\\\\\\"d218-4.smtp-out.eu-west-2.amazonses.com\\\\\\\"}}\\\",\\n \\\"Timestamp\\\" : \\\"2022-09-13T13:46:46.598Z\\\",\\n \\\"SignatureVersion\\\" : \\\"1\\\",\\n \\\"Signature\\\" : \\\"ashjdiaushdiuhasiuh23423hiuhiudhsfsdfhiu3423423==\\\",\\n \\\"SigningCertURL\\\" : \\\"https://sns.eu-west-2.amazonaws.com/SimpleNotificationService-sfdfsdfsdfsdfdsfsdfsdf.pem\\\",\\n \\\"UnsubscribeURL\\\" : \\\"https://sns.eu-west-2.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=arn:aasdasdsadsadasdasd\\\"\\n}\":String\n\n 'body' => msg.body,\n ^^^^^" error_class="NoMethodError" 2022-09-13 13:46:46 +0000 [warn]: #0 suppressed same stacktrace 2022-09-13 13:46:46 +0000 [error]: #0 SQS exception error="undefined methodbody' for {"SenderId"=>"asdasdasdasdasd", "ApproximateFirstReceiveTimestamp"=>"13123123123123123", "ApproximateReceiveCount"=>"1", "SentTimestamp"=>"1663076806628"}:Aws::Xml::DefaultMap\n\n 'body' => msg.body,\n