Ping a Specific Port

Question

Shailesh Sutar

Asked: 2017-01-03 07:57:24 +0800 CST2017-01-03 07:57:24 +0800 CST 2017-01-03 07:57:24 +0800 CST

logrotate not writing logs to correct file

772

I have a production system on which there are multiple nginx apps hosted. those are nginx applications servers. Now for each of these nginx app servers a logrotate config file is configured in /etc/logrotate.d/ folder. one of the example of logrotate config file is given below.

/var/www/example/current/log/example.log {
  daily
  missingok
  rotate 1
  delaycompress
  compress
  notifempty
  copytruncate
}

I checked following Logrotate not working and checked my config and didn't find any duplicate entries for application servers or nginx as well. below is the definition of my nginx logrotate file.

/var/log/nginx/*.log {
    daily
    missingok
    rotate 3
  compress
  delaycompress
    notifempty
    create 0640 www-data adm
    sharedscripts
    postrotate
    if [ -s /var/run/nginx.pid ]; then
      kill -USR1 `cat /var/run/nginx.pid`
    fi
    endscript
}

My logs are still getting written in *.log.1 and not in *.log Can anyone help with this? I am trying to understand what is wrong here. Below is the logrotate command output in debug mode.

logrotate --force -d /etc/logrotate.d/nginx
reading config file /etc/logrotate.d/nginx

Handling 1 logs

rotating pattern: /var/log/nginx/*.log  forced from command line (3 rotations)
empty log files are not rotated, old logs are removed
considering log /var/log/nginx/access.log
  log does not need rotating
considering log /var/log/nginx/elasticsearch.access.log
  log does not need rotating
considering log /var/log/nginx/error.log
  log does not need rotating
considering log /var/log/nginx/grafana.access.log
  log does not need rotating
considering log /var/log/nginx/example-admin.access.log
  log needs rotating
considering log /var/log/nginx/example-admin.error.log
  log does not need rotating
considering log /var/log/nginx/example-administrators.access.log
  log needs rotating
considering log /var/log/nginx/example-administrators.error.log
  log does not need rotating
considering log /var/log/nginx/example-alot.access.log
  log does not need rotating
considering log /var/log/nginx/example-alot.error.log
  log does not need rotating
considering log /var/log/nginx/example-api_clients.access.log
  log does not need rotating
considering log /var/log/nginx/example-api_clients.error.log
  log does not need rotating
considering log /var/log/nginx/example-assets.access.log
  log does not need rotating
considering log /var/log/nginx/example-backgrounds.access.log
  log does not need rotating
considering log /var/log/nginx/example-backgrounds.error.log
  log does not need rotating
considering log /var/log/nginx/example-blog.access.log
  log does not need rotating
considering log /var/log/nginx/example-blog.error.log
  log does not need rotating
considering log /var/log/nginx/example-blog-service.access.log
  log does not need rotating
considering log /var/log/nginx/example-blog-service.error.log
  log does not need rotating
considering log /var/log/nginx/example-boards.access.log
  log does not need rotating
considering log /var/log/nginx/example-boards.error.log
  log does not need rotating
considering log /var/log/nginx/example-bolt.access.log
  log does not need rotating
considering log /var/log/nginx/example-bolt.error.log
  log does not need rotating
considering log /var/log/nginx/example-bonobo.access.log
  log does not need rotating
considering log /var/log/nginx/example-bonobo.error.log
  log does not need rotating
considering log /var/log/nginx/example-boss.access.log
  log does not need rotating
considering log /var/log/nginx/example-boss.error.log
  log does not need rotating
considering log /var/log/nginx/example-callbacks.access.log
  log does not need rotating
considering log /var/log/nginx/example-callbacks.error.log
  log does not need rotating
considering log /var/log/nginx/example-cashier.access.log
  log does not need rotating
considering log /var/log/nginx/example-cashier.error.log
  log does not need rotating
considering log /var/log/nginx/example-dashboard-old.access.log
  log does not need rotating
considering log /var/log/nginx/example-dcl_shipments.access.log
  log does not need rotating
considering log /var/log/nginx/example-dcl_shipments.error.log
  log does not need rotating
considering log /var/log/nginx/example-device-registry.access.log
  log does not need rotating
considering log /var/log/nginx/example-device-registry.error.log
  log does not need rotating
considering log /var/log/nginx/example-docs.access.log
  log does not need rotating
considering log /var/log/nginx/example-exports.access.log
  log does not need rotating
considering log /var/log/nginx/example-exports.error.log
  log does not need rotating
considering log /var/log/nginx/example-fake-api.access.log
  log does not need rotating
considering log /var/log/nginx/example-fake-api.error.log
  log does not need rotating
considering log /var/log/nginx/example-fashion-police.access.log
  log does not need rotating
considering log /var/log/nginx/example-fashion-police.error.log
  log does not need rotating
considering log /var/log/nginx/example-flight-recorder.access.log
  log does not need rotating
considering log /var/log/nginx/example-flight-recorder.error.log
  log does not need rotating
considering log /var/log/nginx/example-gone.access.log
  log does not need rotating
considering log /var/log/nginx/example-gone.error.log
  log does not need rotating
considering log /var/log/nginx/example-go-portal.access.log
  log does not need rotating
considering log /var/log/nginx/example-go-portal.error.log
  log does not need rotating
considering log /var/log/nginx/example-help-service.access.log
  log does not need rotating
considering log /var/log/nginx/example-help-service.error.log
  log does not need rotating
considering log /var/log/nginx/example-kfc.access.log
  log does not need rotating
considering log /var/log/nginx/example-kfc.error.log
  log does not need rotating
considering log /var/log/nginx/example-kredits.access.log
  log does not need rotating
considering log /var/log/nginx/example-kredits.error.log
  log does not need rotating
considering log /var/log/nginx/example-login.access.log
  log does not need rotating
considering log /var/log/nginx/example-login.error.log
  log does not need rotating
considering log /var/log/nginx/example-manual-activations.access.log
  log does not need rotating
considering log /var/log/nginx/example-manual-activations.error.log
  log does not need rotating
considering log /var/log/nginx/example-maps.access.log
  log does not need rotating
considering log /var/log/nginx/example-mobile-api.access.log
  log does not need rotating
considering log /var/log/nginx/example-mobile-api.error.log
  log does not need rotating
considering log /var/log/nginx/example-mqttproxy.access.log
  log does not need rotating
considering log /var/log/nginx/example-mqttproxy.error.log
  log does not need rotating
considering log /var/log/nginx/example-network-coverage.access.log
  log does not need rotating
considering log /var/log/nginx/example-network-coverage.error.log
  log does not need rotating
considering log /var/log/nginx/example-package_tracker.access.log
  log does not need rotating
considering log /var/log/nginx/example-package_tracker.error.log
  log does not need rotating
considering log /var/log/nginx/example-people.access.log
  log needs rotating
considering log /var/log/nginx/example-people.error.log
  log does not need rotating
considering log /var/log/nginx/example-postman-pat.access.log
  log does not need rotating
considering log /var/log/nginx/example-postman-pat.error.log
  log does not need rotating
considering log /var/log/nginx/example-premium.access.log
  log does not need rotating
considering log /var/log/nginx/example-premium.error.log
  log does not need rotating
considering log /var/log/nginx/example-push-notifications.access.log
  log does not need rotating
considering log /var/log/nginx/example-push-notifications.error.log
  log does not need rotating
considering log /var/log/nginx/example-referrific.access.log
  log does not need rotating
considering log /var/log/nginx/example-referrific.error.log
  log does not need rotating
considering log /var/log/nginx/example-services.access.log
  log does not need rotating
considering log /var/log/nginx/example-services.error.log
  log does not need rotating
considering log /var/log/nginx/example-store.access.log
  log does not need rotating
considering log /var/log/nginx/example-store.error.log
  log does not need rotating
considering log /var/log/nginx/example-subs.access.log
  log does not need rotating
considering log /var/log/nginx/example-subs.error.log
  log does not need rotating
considering log /var/log/nginx/example-updates.access.log
  log does not need rotating
considering log /var/log/nginx/example-updates.error.log
  log does not need rotating
considering log /var/log/nginx/example-website.access.log
  log does not need rotating
considering log /var/log/nginx/example-wildcard-redirect.access.log
  log does not need rotating
considering log /var/log/nginx/example-zendesk-proxy.access.log
  log does not need rotating
considering log /var/log/nginx/example-zendesk-proxy.error.log
  log does not need rotating
considering log /var/log/nginx/kibana.access.log
  log does not need rotating
considering log /var/log/nginx/prometheus.access.log
  log does not need rotating
considering log /var/log/nginx/prometheus_alertmanager.access.log
  log does not need rotating
considering log /var/log/nginx/prometheus_statsd_bridge.access.log
  log does not need rotating
rotating log /var/log/nginx/example-admin.access.log, log->rotateCount is 3
dateext suffix '-20170104'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
compressing log with: /bin/gzip
renaming /var/log/nginx/example-admin.access.log.3.gz to /var/log/nginx/example-admin.access.log.4.gz (rotatecount 3, logstart 1, i 3), 
renaming /var/log/nginx/example-admin.access.log.2.gz to /var/log/nginx/example-admin.access.log.3.gz (rotatecount 3, logstart 1, i 2), 
renaming /var/log/nginx/example-admin.access.log.1.gz to /var/log/nginx/example-admin.access.log.2.gz (rotatecount 3, logstart 1, i 1), 
renaming /var/log/nginx/example-admin.access.log.0.gz to /var/log/nginx/example-admin.access.log.1.gz (rotatecount 3, logstart 1, i 0), 
rotating log /var/log/nginx/example-administrators.access.log, log->rotateCount is 3
dateext suffix '-20170104'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
compressing log with: /bin/gzip
renaming /var/log/nginx/example-administrators.access.log.3.gz to /var/log/nginx/example-administrators.access.log.4.gz (rotatecount 3, logstart 1, i 3), 
renaming /var/log/nginx/example-administrators.access.log.2.gz to /var/log/nginx/example-administrators.access.log.3.gz (rotatecount 3, logstart 1, i 2), 
renaming /var/log/nginx/example-administrators.access.log.1.gz to /var/log/nginx/example-administrators.access.log.2.gz (rotatecount 3, logstart 1, i 1), 
renaming /var/log/nginx/example-administrators.access.log.0.gz to /var/log/nginx/example-administrators.access.log.1.gz (rotatecount 3, logstart 1, i 0), 
rotating log /var/log/nginx/example-people.access.log, log->rotateCount is 3
dateext suffix '-20170104'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
compressing log with: /bin/gzip
renaming /var/log/nginx/example-people.access.log.3.gz to /var/log/nginx/example-people.access.log.4.gz (rotatecount 3, logstart 1, i 3), 
renaming /var/log/nginx/example-people.access.log.2.gz to /var/log/nginx/example-people.access.log.3.gz (rotatecount 3, logstart 1, i 2), 
renaming /var/log/nginx/example-people.access.log.1.gz to /var/log/nginx/example-people.access.log.2.gz (rotatecount 3, logstart 1, i 1), 
renaming /var/log/nginx/example-people.access.log.0.gz to /var/log/nginx/example-people.access.log.1.gz (rotatecount 3, logstart 1, i 0), 
renaming /var/log/nginx/example-admin.access.log to /var/log/nginx/example-admin.access.log.1
creating new /var/log/nginx/example-admin.access.log mode = 0640 uid = 33 gid = 4
renaming /var/log/nginx/example-administrators.access.log to /var/log/nginx/example-administrators.access.log.1
creating new /var/log/nginx/example-administrators.access.log mode = 0640 uid = 33 gid = 4
renaming /var/log/nginx/example-people.access.log to /var/log/nginx/example-people.access.log.1
creating new /var/log/nginx/example-people.access.log mode = 0640 uid = 33 gid = 4
running postrotate script
running script with arg /var/log/nginx/*.log : "
    if [ -s /var/run/nginx.pid ]; then
      kill -USR1 `cat /var/run/nginx.pid`
    fi
"
removing old log /var/log/nginx/example-admin.access.log.4.gz
error: error opening /var/log/nginx/example-admin.access.log.4.gz: No such file or directory

And when i do force logrotate, I end up having errors below.

sudo logrotate -f /etc/logrotate.conf
logrotate_script: 2: logrotate_script: /etc/init.d/example-admin: not found
error: error running non-shared postrotate script for /var/www/admin/current/log/example-admin.log of '/var/www/admin/current/log/example-admin.log '
logrotate_script: 2: logrotate_script: /etc/init.d/example-administrators: not found
error: error running non-shared postrotate script for /var/www/administrators/current/log/example-administrators.log of '/var/www/administrators/current/log/example-administrators.log '
logrotate_script: 2: logrotate_script: /etc/init.d/example-alot: not found
error: error running non-shared postrotate script for /var/www/alot/current/log/example-alot.log of '/var/www/alot/current/log/example-alot.log '
logrotate_script: 2: logrotate_script: /etc/init.d/example-bonobo: not found
error: error running non-shared postrotate script for /var/www/bonobo/current/log/example-bonobo.log of '/var/www/bonobo/current/log/example-bonobo.log '
logrotate_script: 2: logrotate_script: /etc/init.d/example-cashier: not found
error: error running non-shared postrotate script for /var/www/cashier/current/log/example-cashier.log of '/var/www/cashier/current/log/example-cashier.log '
logrotate_script: 2: logrotate_script: /etc/init.d/example-dcl-shipments: not found
error: error running non-shared postrotate script for /var/www/dcl_shipments/current/log/example-dcl_shipments.log of '/var/www/dcl_shipments/current/log/example-dcl_shipments.log '
logrotate_script: 2: logrotate_script: /etc/init.d/example-people: not found
error: error running non-shared postrotate script for /var/www/people/current/log/example-people.log of '/var/www/people/current/log/example-people.log '
logrotate_script: 2: logrotate_script: /etc/init.d/example-store: not found
error: error running non-shared postrotate script for /var/www/store/current/log/example-store.log of '/var/www/store/current/log/example-store.log '
logrotate_script: 2: logrotate_script: /etc/init.d/example-storefront: not found
error: error running non-shared postrotate script for /var/www/storefront/current/log/example-storefront.log of '/var/www/storefront/current/log/example-storefront.log '
logrotate_script: 2: logrotate_script: /etc/init.d/example-subs: not found
error: error running non-shared postrotate script for /var/www/subs/current/log/example-subs.log of '/var/www/subs/current/log/example-subs.log '

Below is the logrotate command ouput in debug mode.

sudo logrotate -dv /etc/logrotate.d/example-alot
reading config file /etc/logrotate.d/example-alot

Handling 1 logs

rotating pattern: /var/www/alot/current/log/example-alot.log  after 1 days (1 rotations)
empty log files are not rotated, old logs are removed
considering log /var/www/alot/current/log/example-alot.log
  log needs rotating
rotating log /var/www/alot/current/log/example-alot.log, log->rotateCount is 1
dateext suffix '-20170105'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
compressing log with: /bin/gzip
renaming /var/www/alot/current/log/example-alot.log.1.gz to /var/www/alot/current/log/example-alot.log.2.gz (rotatecount 1, logstart 1, i 1),
renaming /var/www/alot/current/log/example-alot.log.0.gz to /var/www/alot/current/log/example-alot.log.1.gz (rotatecount 1, logstart 1, i 0),
copying /var/www/alot/current/log/example-alot.log to /var/www/alot/current/log/example-alot.log.1
truncating /var/www/alot/current/log/example-alot.log
running postrotate script
running script with arg /var/www/alot/current/log/example-alot.log: "
      echo "Appending to" example-alot > /var/www/alot/current/log/example-alot.log
      /etc/init.d/example-alot reload >> /dev/null
"
removing old log /var/www/alot/current/log/example-alot.log.2.gz


sudo logrotate -dv /etc/logrotate.d/example-subs
reading config file /etc/logrotate.d/example-subs

Handling 1 logs

rotating pattern: /var/www/subs/current/log/example-subs.log  after 1 days (1 rotations)
empty log files are not rotated, old logs are removed
considering log /var/www/subs/current/log/example-subs.log
  log does not need rotating

2 Answers

Voted

turbophi · Answer 1 · 2017-01-03T09:54:48+08:00

turbophi

2017-01-03T09:54:48+08:002017-01-03T09:54:48+08:00

If nginx is writing to the old log file, this might happen because your postrotate action is not working.

Did you check that /var/run/nginx.pid is existing and contains the correct pid?

Moreover you could try

logrotate --force -d /etc/logrotate.d/nginx

and then check the output. It gives good indications.

0

ScienceGoneBad · Answer 2 · 2021-05-29T17:38:49+08:00

ScienceGoneBad

2021-05-29T17:38:49+08:002021-05-29T17:38:49+08:00

If you're using syslog or syslog-ng, you need to make sure something like this is part of the shared postrotate scripts:

sharedscripts
postrotate
/bin/kill -HUP `cat /var/run/syslogd.pid 2> /dev/null` 2> /dev/null || true
endscript

syslog writes files to inodes, not file names, so if you don't bounce syslog, it continues to write to the same inode (even though the file name changed) HUPing the syslog process resets the inode to the correct name. It's the same reason that syslog fails to write to files that are non-existent. You have to touch the file before starting syslog so that the inode of the file can be determined.

Hope that helps

0

logrotate not writing logs to correct file

Can you pass user/pass for HTTP Basic Authentication in URL parameters?

Ping a Specific Port

Check if port is open or closed on a Linux server?

How to automate SSH login with password?

How do I tell Git for Windows where to find my private RSA key?

What's the default superuser username/password for postgres after a new install?

What port does SFTP use?

Command line to list users in a Windows Active Directory group?

What is a Pem file and how does it differ from other OpenSSL Generated Key File Formats?

How to determine if a bash variable is empty?