Implement transport standard RFC 6587

[vbohata]

There is RFC 6587 - transmission of Syslog Messages over TCP. We have some sources which follow this standard and we are unable to ingest them as syslog message splitting is not per new line but controlled by octet counting - https://tools.ietf.org/html/rfc6587#section-3.4

I think this plugin should support it.

[drolfe]

This is causing a massive issue for us

We are receiving juniper srx traffic logs over tcp and need to use logstash the filter the messages

any ETA on logstash support for this RFC framing ?

[drolfe]

I'm unfamiliar with filebeat configuration

What I'm thinking is to use filebeat as a shim layer for the RFC6587 framing ie

• RFC6587 syslog =>
• Filebeat(to split out the syslog messages from the framing) =>
• Logstash(to filter the messages) =>
• Elastic or further processing

Would something like the below work to get the syslog to be understood by logstash ?

filebeat.inputs:
- type: syslog
  format: rfc5424
  framing: rfc6587
  protocol.tcp:
    host: "0.0.0.0:514"

output.logstash:
  hosts: ["10.10.10.5:5044"]

I also have a case open with elastic for this, case number #01384962

[ThePsyjo]

The best solution for this would be an additional codec plugin which implements rfc6587. With this, it could also be used with the input-tcp plugin and probably as an output too.

[codebrain]

It has been quite a number of years since a logstash-codec-frame plugin was created, but would it help?

[ThePsyjo]

I managed to build a codec plugin. It's not perfect (yet), but i hope it will get better over time. So far, it is working quite well for me when receiving data from fortigates.

https://github.com/ThePsyjo/logstash-codec-rfc6587

input {
  tcp {
    type => "tcp"
    port => 7514
    mode => "server"
    codec => "rfc6587"
  }
}