Home / server / Questions / 1084654
In Process
Aditya K
Asked: 2021-11-27 03:43:08 +0800 CST

Can logstash "pull" data?

  • 772

I have two servers. Server A is running Elasticsearch and Logstash. Server B is running filebeat and is also the server which contains all the logs I'm trying to analyse.

Server A is behind a firewall, it can reach out to the internet, but there's no possibility of allowing any inbound traffic.

Server B is in AWS and under my control for inbound and outbound traffic.

Is there any way I can get data from server B to server A's logstash? Filebeat in the usual way would have to be able to push data to logstash, but this isn't possible in my scenario.

elasticsearch filebeat logstash

1 Answers

  1. With Filebeat (or any other Beat) on B, your options are limited to:

    • Elasticsearch — not viable here.
    • Redis — Some possibilities.
    • Kafka — Similar possibilities.

    If A can call into B's network spaces, you can have B's Filebeat deposit logs into Redis (or Elasticache Redis), or Kafka (or AWS Managed Kafka Services), where the Logstash on A can connect to Redis or Kafka.

    For the security minded, Redis now has the ability to support both authentication and TLS over the connection. In versions 6.x and later, especially the AWS varieties, Redis has actual ACL support.

    • 0