I would like to know what is the easiest way to forward my docker container logs to an ELK server, so far the solutions I have tried after having searched the internet didn't work at all.
Basically I have a docker image that I run using docker-compose, this container does not log anything locally (it is composed of different services but none of them are logstash or whatever) but I see logging through docker logs -tf imageName
or docker-compose logs
. Since I am starting the containers with compose I cannot make use (or at least I don't know how) of the --logs-driver
option of docker.
Thus I was wondering if someone may enlighten me a bit regarding how to forward that logs to an ELK container for example.
Thanks in advance,
Regards
SOLUTION:
Thanks to madeddie I could achieve to solve my issue in the following way, mention that I used the basic ELK-stack-in-containers which madeddie suggested in his post.
First I update the docker-compose.yml
file of my container to add entries for the logging reference as madeddie told me,I included an entry per service, a snippet of my docker-compose looks like this
version: '2'
services:
mosquitto:
image: ansi/mosquitto
ports:
- "1883:1883" # Public access to MQTT
- "12202:12202" #mapping logs
logging:
driver: gelf
options:
gelf-address: udp://localhost:12202
redis:
image: redis
command: redis-server --appendonly yes
ports:
- "6379:6379" # No public access to Redis
- "12203:12203" #mapping logs
volumes:
- /home/dockeruser/redis-data:/data
logging:
driver: gelf
options:
gelf-address: udp://localhost:12203
Secondly, I had to use a different port number per sevice in order to be able to forward the logs.
Finally,I updated my elk container docker-compose.yml
file to map each of the upd port where I was sending my logs to the one that logstash listens to
logstash:
build: logstash/
command: logstash -f /etc/logstash/conf.d/logstash.conf
volumes:
- ./logstash/config:/etc/logstash/conf.d
ports:
- "5000:5000"
- "12202:12201/udp" #mapping mosquitto logs
- "12203:12201/udp" #mapping redis logs
This configuration and adding the entry of gelf {}
in logstash.conf
made it work, it is important as well to set up properly the IP address of the docker service.
REgards!