Lumberjack (Beats) output

[xied75]

Sorry that I only just discovered this brilliant project, 👍

The more I read the usage doc, the more I felt that, (correct me if I'm wrong), this is kind of like a Logstash in Process? Even the conf json looks similar. (By all means, this is Great).

Just wondering if my understanding is correct or not. If not far off, what do you think about the FileBeat or Beats protocol in general? Should we also add an output in Beats?

[karolz-ms]

Dong, thank you for kind words

I think "Logstash in-proces" describes fairly well what EventFlow is about ☺️

Beats is a great idea IMO (and fluentD FluentBit too). Nobody asked us to add an output for Beats yet, but I think it would fit very well with the rest of EventFlow. Would you be willing to try to implement it? We are taking PRs 😉

[jeremysmith1]

Hmm, I am not sure you could make an output to one of the Beats products. Beats work as a lightweight data shipper, not as data sink. I do not believe any of the Beats have data pushed to them.

Logstash, on the other hand, could be used as a data sink. The Logstash Filter plugins would be particularly useful as this would allow you to do many types of transforms on the data.

You may already be able to achieve this using the EventFlow StdOuput with the Logstash Stdin Input plugin.

Another option could be to make a EventFlow output that pushes events to a Logstash supported messaging queue like rabbitmq or kafka. That is added complexity that may not be necessary.

[xied75]

I think what we mean is to "output" in Beats protocol, not output into Beats.

[jeremysmith1]

My misunderstanding. Do they use their own protocol? I saw some stuff about the Lumberjack protocol.

[xied75]

@jeremysmith1 You are correct! Lumberjack upon tcp. Would love to help if I could find some free time.