user99201's questions

I am trying to setup an apache website for SSL with a self-signed cert. I setup the vhost and everything looks ok to me. But the site is not coming up and on the server itself port 443 is not listening.

Website: https://beta.jokefire.com

VHOST config:

<VirtualHost *:443>
     SSLEngine On
     SSLCertificateFile /etc/pki/tls/certs/beta.jokefire.com.crt
     SSLCertificateKeyFile /etc/pki/tls/private/beta.jokefire.com.key
     #SSLCACertificateFile /etc/pki/tls/certs/root-certificate.crt  #If using a self-signed certificate or a root certificate provided by ca-certificates, omit this line

     ServerAdmin [email protected]
     ServerName beta.jokefire.com
     DocumentRoot /var/www/jf-beta
     ErrorLog logs/jf_beta.jokefire.com_ssl_error_log
     CustomLog logs/jf_beta.jokefire.com_ssl_access_log combine
</VirtualHost>

I am listening on port 80 but not on port 443:

[root@web1:~] #lsof -i :80
COMMAND   PID   USER   FD   TYPE     DEVICE SIZE/OFF NODE NAME
httpd   14489   root    4u  IPv6 1575659387      0t0  TCP *:http (LISTEN)
httpd   14490 apache    4u  IPv6 1575659387      0t0  TCP *:http (LISTEN)
httpd   14497 apache    4u  IPv6 1575659387      0t0  TCP *:http (LISTEN)
httpd   14497 apache   27u  IPv6 1575659779      0t0  TCP web1.jokefire.com:http->37.151.164.13.megaline.telecom.kz:54674 (ESTABLISHED)
httpd   14498 apache    4u  IPv6 1575659387      0t0  TCP *:http (LISTEN)
httpd   14499 apache    4u  IPv6 1575659387      0t0  TCP *:http (LISTEN)
httpd   14499 apache   27u  IPv6 1575659460      0t0  TCP web1.jokefire.com:http->7.bl.bot.semrush.com:23728 (ESTABLISHED)
httpd   14501 apache    4u  IPv6 1575659387      0t0  TCP *:http (LISTEN)
httpd   14502 apache    4u  IPv6 1575659387      0t0  TCP *:http (LISTEN)
httpd   14525 apache    4u  IPv6 1575659387      0t0  TCP *:http (LISTEN)
httpd   14541 apache    4u  IPv6 1575659387      0t0  TCP *:http (LISTEN)
httpd   14542 apache    4u  IPv6 1575659387      0t0  TCP *:http (LISTEN)
[root@web1:~] #lsof -i :443
[root@web1:~] #

Here's my cert and key and directories:

[root@web1:~] #ls -lh /etc/pki/tls/certs/beta.jokefire.com.crt /etc/pki/tls/certs/beta.jokefire.com.crt
-rw-------. 1 root root 1.5K Oct 17 16:14 /etc/pki/tls/certs/beta.jokefire.com.crt
-rw-------. 1 root root 1.5K Oct 17 16:14 /etc/pki/tls/certs/beta.jokefire.com.crt

[root@web1:~] #ls -ld /etc/pki/tls/certs /etc/pki/tls/private/
drwx------. 2 root root 4096 Oct 17 16:20 /etc/pki/tls/certs
drwx------. 2 root root 4096 Oct 17 16:13 /etc/pki/tls/private/

And the SSL log files that I've defined in the apache vhost are there on the filesystem, but they are empty:

[root@web1:~] #ls -lh /var/log/httpd/jf_beta.jokefire.com_ssl_access_log /var/log/httpd/jf_beta.jokefire.com_ssl_error_log
-rw-r--r--. 1 root root 0 Oct 17 16:36 /var/log/httpd/jf_beta.jokefire.com_ssl_access_log
-rw-r--r--. 1 root root 0 Oct 17 16:36 /var/log/httpd/jf_beta.jokefire.com_ssl_error_log

What am I doing wrong?

I'm trying to install a package called mcgw onto a centos 7.7 machine. And I'm getting a dependency problem:

--> Running transaction check
---> Package mcgw.x86_64 0:1.0.010-0 will be installed
--> Processing Dependency: libcrypto.so()(64bit) for package: mcgw-1.0.010-0.x86_64
Loading mirror speeds from cached hostfile
--> Processing Dependency: libssl.so()(64bit) for package: mcgw-1.0.010-0.x86_64
--> Finished Dependency Resolution
Error: Package: mcgw-1.0.010-0.x86_64 (/mcgw-1.0.010-0.x86_64)
           Requires: libssl.so()(64bit)
Error: Package: mcgw-1.0.010-0.x86_64 (/mcgw-1.0.010-0.x86_64)
           Requires: libcrypto.so()(64bit)
 You could try using --skip-broken to work around the problem
 You could try running: rpm -Va --nofiles --nodigest

But I have openssl and openssl libraries installed.

openssl11-libs-1.1.1c-2.el7.x86_64
openssl-1.0.2k-19.el7.x86_64
openssl-libs-1.0.2k-19.el7.x86_64
python-backports-ssl_match_hostname-3.5.0.1-1.el7.noarch
openssl11-1.1.1c-2.el7.x86_64`

How do I get around this problem?

I'm trying to exclude *.i686 packages from installing when I try to install the x86_64 version of libcrypto.so.10.

If I put any of the following (one at a time) into my /etc/yum.conf under [main]:

multilib_policy=best
exactarch=1
exclude=*.i386 *.i686
exclude=*.i?86

And I try to install the package it says that it isn't there:

sudo yum install libcrypto.so.10
Loaded plugins: fastestmirror, rhnplugin, tsflags, versionlock
This system is receiving updates from RHN Classic or Red Hat Satellite.
Loading mirror speeds from cached hostfile
No package libcrypto.so.10 available.
Error: Nothing to do

However if I remove any of those settings it tries to install both i686 and the x86_64 version of libcrypto.so.10. I am using Centos version: CentOS Linux release 7.7.1908 (Core)

How can I exclude *.i686 packages in the /etc/yum.conf file?

I have an ansible inventory that I need to remove all cases that start with site2-.

For example I have a list that includes groups named:

[site2-static-web]
site2-ansible
site2-accounts
site2-admin
site2-analytics
site2-audit
site2-botruntime
site2-campaigns
site2-config
site2-coupons
site2-edge
site2-minio
site2-permissions
site2-registry
site2-surveys
site2-websockets
site2-amc-server
site2-static-web
site2-elasticsearch
site2-logstash
site2-kibana
site2-keycloak
site2-pgsql
site2-mysql
site2-rabbitmq
site2-redis
site2-zipkin

I tried sed -i.bak 's/site2-*///g' mmp_default but all that got me was the site2 removed with the ends left in.

site2-ansible
site2-accounts

Became:

ansible
accounts

How can I do this correctly?

I'm running an SSH tunnel command that looks like this:

ssh -v [email protected] -L 9999:10.238.93.43:8991

But when I test listening connections with the netstat command I don't see the tunnel port:

bastion001 ABB-LAB] 0 15:45:16 [~] $ netstat -tulpn | grep -i listen
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      -
tcp        0      0 0.0.0.0:111             0.0.0.0:*               LISTEN      -
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      -
tcp6       0      0 ::1:25                  :::*                    LISTEN      -
tcp6       0      0 :::111                  :::*                    LISTEN      -
tcp6       0      0 :::8080                 :::*                    LISTEN      1078/java
tcp6       0      0 :::80                   :::*                    LISTEN      -
tcp6       0      0 :::8081                 :::*                    LISTEN      1078/java
tcp6       0      0 :::22                   :::*                    LISTEN      -

What's going on here? Why can't I see the tunnel port?

I can get a list of groups on a linux server with the getent group command. However that only gives me local groups in the /etc/group file.

The server is connected to IPA that is used to create active directory groups. How can I get a list of groups IPA/Active Directory groups on the server.

I have an IP address configured for my Azure VM that I don't want to lose, because it's configured in DNS. However I would like to redeploy the server due to some issues.

Is there a way to re-deploy an Azure VM and keep the same IP as before?

I'm trying to ssh into a red hat 7 linux server and I am unable to. I am getting an error:

ssh [email protected]
OpenSSH_7.4p1, OpenSSL 1.0.2k-fips  26 Jan 2017
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 62: Applying options for *
debug1: Executing proxy command: exec /usr/bin/sss_ssh_knownhostsproxy -p 22 10.48.216.147
debug1: permanently_drop_suid: 1990975652
debug1: identity file /home/tdunphy/.ssh/id_rsa type 1
debug1: key_load_public: No such file or directory
debug1: identity file /home/tdunphy/.ssh/id_rsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/tdunphy/.ssh/id_dsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/tdunphy/.ssh/id_dsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/tdunphy/.ssh/id_ecdsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/tdunphy/.ssh/id_ecdsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/tdunphy/.ssh/id_ed25519 type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/tdunphy/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.4
ssh_exchange_identification: Connection closed by remote host

I've checked my permissions on my client side and they seem good:

[tdunphy@bastion-server01 ~]$ ls -ld .ssh
drwx------. 2 tdunphy tdunphy 58 Aug 12 17:34 .ssh
[tdunphy@bastion-server01 ~]$ ls -lh .ssh/*
-rw-------. 1 tdunphy tdunphy  408 Sep 11  2018 .ssh/authorized_keys
-rw-------. 1 tdunphy tdunphy 1.7K Aug 12 17:20 .ssh/id_rsa
-rw-------. 1 tdunphy tdunphy  408 Aug 12 17:34 .ssh/id_rsa.pub

This is the sshd_config from the server I'm trying to ssh from:

Protocol 2
SyslogFacility AUTHPRIV
PasswordAuthentication yes
ChallengeResponseAuthentication no
GSSAPICleanupCredentials yes
AcceptEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES
AcceptEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT
AcceptEnv LC_IDENTIFICATION LC_ALL LANGUAGE
AcceptEnv XMODIFIERS
X11Forwarding yes
Subsystem       sftp    /usr/libexec/openssh/sftp-server
KerberosAuthentication no
PubkeyAuthentication yes
UsePAM yes
GSSAPIAuthentication yes
ClientAliveInterval 900
ClientAliveCountMax 0
IgnoreRhosts yes
HostbasedAuthentication no
PermitRootLogin no
PermitEmptyPasswords no
Banner /etc/issue
PermitUserEnvironment no
Ciphers aes128-ctr,aes192-ctr,aes256-ctr

There is nothing in /etc/hosts.allow and /etc/hosts.deny.

I tried doing a restorecon -v /home/tdunphy/.ssh but that didn't do any good.

What can I do to log into the remote machine?

What's the difference between these two array notations?

• "${aws_user_roles[@]}"
• "${aws_user_roles[*]}"

I'm trying to setup a mail relay server on a local test VM. I'm using Ubuntu 18.04.

When I send email using my postfix config, the mail is never received and I see this entry in my mail log:

status=sent (delivered to command: procmail -a "$EXTENSION")

This is my postfix config:

tdunphy@tdunphy-dev:~$ postconf -n
alias_database = hash:/etc/aliases
alias_maps = hash:/etc/aliases
append_dot_mydomain = no
biff = no
compatibility_level = 2
inet_interfaces = all
inet_protocols = all
mailbox_command = procmail -a "$EXTENSION"
mailbox_size_limit = 0
mydestination = $myhostname, company.com, tdunphy-VirtualBox, localhost.localdomain, localhost
myhostname = noreply.company.com
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
myorigin = /etc/mailname
readme_directory = no
recipient_delimiter = +
relayhost = smtpout.us.companyworld.company.com
smtp_sasl_auth_enable = no
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu)
smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
smtpd_tls_cert_file = /etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file = /etc/ssl/private/ssl-cert-snakeoil.key
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtpd_use_tls = yes

I'm sending email on the command line using this command:

 mail -s "Test Subject" [email protected] < mail_test.txt

This is what I see in the logs:

Jul  9 14:31:22 tdunphy-VirtualBox postfix/local[7906]: 1C485E36C: to=<[email protected]>, relay=local, delay=35, delays=35/0.01/0/0.02, dsn=2.0.0, status=sent (delivered to command: procmail -a "$EXTENSION")
Jul  9 14:31:22 tdunphy-VirtualBox postfix/qmgr[7890]: 1C485E36C: removed
Jul  9 14:31:25 tdunphy-VirtualBox postfix/smtpd[7901]: disconnect from localhost[127.0.0.1] ehlo=1 mail=1 rcpt=1 data=1 quit=1 unknown=0/1 commands=5/6

But the email never arrives. How do I resolve that?

I'm on a red hat 7 machine, and I need to open all ports to a specific IP on the firewall.

I tried this command:

firewall-cmd --permanent --zone=public --add-rich-rule='   rule family="ipv4"   source address="64.39.96.0/20"   port protocol="tcp" port="*" accept'

But I'm getting an invalid port error for the *

Does anyone know and can tell me how to do this correctly?

I'm trying to mount an NFS volume on a centos 7.2 server:

When I try to mount the NFS share point, this is the response I get back:

[root@web1:~] #mount -t nfs nfs1.example.com:/var/nfs/home /home
mount.nfs: an incorrect mount option was specified

I checked and I have nfs-utils-1.3.0-0.21.el7.x86_64 installed on both machines. Both the nfs client and the nfs server OSes are Centos 7.2

To troubleshoot this, I reduced the listings in the /etc/exports file on the NFS server to just the following:

/var/nfs/home web1.example.com(rw,sync,no_root_squash,no_all_squash)

If I do a showmount from the server I'm trying to mount the nfs share on, this is what I see:

[root@web1:~] #showmount -e nfs1.example.com
Export list for nfs1.example.com:
/var/nfs/home web1.example.com

If I do a mount -v this is what I get:

[root@web1:~] #mount -v -t nfs nfs1.example.com:/var/nfs/home /home
mount.nfs: timeout set for Fri Jan 13 11:04:19 2017 mount.nfs: trying text-based options 'vers=4,addr=162.xxx.xxx..94,clientaddr=162.xxx.xxx.6'
mount.nfs: mount(2): Invalid argument mount.nfs: an incorrect mount option was specified

In dmesg I find:

[44428.405419] nfsd: last server has exited, flushing export cache

And I'm seeing this in dmesg:

[ 7.373186] FS-Cache: Netfs 'nfs' registered for caching
[ 7.422181] Key type dns_resolver registered
[ 7.456581] NFS: Registering the id_resolver key type
[ 7.462309] Key type id_resolver registered
[ 7.462386] Key type id_legacy registered
[ 7.514441] SELinux: initialized (dev 0:40, type nfs4), uses genfs_contexts
[ 8.474503] NFSD: starting 90-second grace period (net ffffffff819a29c0) –
[ 16.952180] perf samples too long (2623 > 2500), lowering kernel.perf_event_max_sample_rate to 50000
[ 24.429251] SELinux: initialized (dev tmpfs, type tmpfs), uses transition SIDs
[ 38.368207] perf samples too long (5162 > 5000), lowering kernel.perf_event_max_sample_rate to 25000
[ 38.427323]

Not specifying -t nfs in the command gives the same result:

[root@nfs1:~] #mount nfs1.example.com:/var/nfs/home /home
mount.nfs: an incorrect mount option was specified

These are the nfs file systems I have:

[root@nfs1:~] #grep nfs /proc/filesystems
nodev   nfsd
nodev   nfs
nodev   nfs4

Here are the modules loaded for nfs:

[root@nfs1:~] #lsmod | grep ^nfs
nfsv4                 474203  0
nfs                   241266  1 nfsv4
nfsd                  284378  13
nfs_acl                12837  1 nfsd

It's not a firewall issue because I get the same exact failure when I run the mount command from the NFS server itself. It gets the same error that it's clients do.

I should stress that this did at one time work fine. But now it's broken to the point where it can't be used.

Can someone please help me troubleshoot this? I'm really stuck at this point.

My old company has custom AWS login URLs that look like this:

https://company-us-aws-dev.signin.aws.amazon.com/console
https://company-us-aws-qa.signin.aws.amazon.com/console
https://company-us-aws-stage.signin.aws.amazon.com/console
https://company-us-aws-prod.signin.aws.amazon.com/console

And when you would click on any of the above links, you would see a login page that looks like this:

It's a login page that gives you a chance to switch to your aws root account. But I don't know what this page is called.

I now need to set this up at my new company. Is there any documentation you can point me to, or instruction you can give for how to do this?

I need to find out how to create these custom URLs in DNS and how to point those url's to these sign on screens. If I just create an A record in DNS and point it to an AWS IP, I just get a 404 screen when I go that URL in my browser.

Please help

I followed this tutorial from Digital Ocean on how to install an ELK stack on a CentOS 7 machine.

Digital Ocean ELK Setup CentOS

It seemed pretty good, and got me as far as having an initial Elastic Search node working correctly and have kibana 4 running behind NGINX. But in installing Logstash I ran into an issue where it doesn't seem to create any indexes in elasticsearch!! I'm sure it's a config issue somewhere. But where I don't know!

I notice that when I cat out the elasticsearch indexes using the _cat API after restarting logstash, logstash hasn't created any indexes.

curl http://localhost:9200/_cat/indices
yellow open .kibana  1 1 1 0 2.4kb 2.4kb
yellow open security 5 1 0 0  575b  575b

Here we have kibana's index and what I think is a standard ES index called 'security'. But it seems that logstash is not communicating with ES! And everything is running on the same machine.

These are the versions of ES and LS I have installed:

elasticsearch-1.5.2-1.noarch
logstash-1.5.1-1.noarch
logstash-forwarder-0.4.0-1.x86_64

And the way they have it setup in the tutorial I followed, you have 3 config files going into the logstash conf.d directory.

In /etc/logstash/conf.d/01-lumberjack-input.conf I have:

  lumberjack {
    port => 5000
    type => "logs"
    ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
    ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
  }
}

In /etc/logstash/conf.d/10-syslog.conf I have:

filter {
  if [type] == "syslog" {
    grok {
      match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
      add_field => [ "received_at", "%{@timestamp}" ]
      add_field => [ "received_from", "%{host}" ]
    }
    syslog_pri { }
    date {
      match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
    }
  }
}

I also have my own config from my previous logstash server that I put into a file called 20-logstash.conf that is listening on port 2541:

I have the following in /etc/logstash/conf.d/20-logstash.conf

input {


   lumberjack {
       # The port to listen on
       port => 2541

       # The paths to your ssl cert and key
        ssl_certificate => "/etc/pki/tls/certs/lumberjack.crt"
        ssl_key => "/etc/pki/tls/private/lumberjack.key"

         # Set this to whatever you want.
         type => "logstash"
         codec => "json"
       }
}


filter {
   if [type] == "postfix" {
      grok {
            match => [ "message", "%{SYSLOGBASE}", "timestamp", "MMM dd HH:mm:ss" ]
            add_tag => [ "postfix", "grokked" ]
      }
   }
}


filter {
   if [type] == "system" {
      grok {
            match => [ "message", "%{SYSLOGBASE}" ]
            add_tag => [ "system", "grokked" ]
      }
   }
}

filter {
   if [type] == "syslog" {
      grok {
            match => [ "message", "%{SYSLOGBASE}" ]
            add_tag => [ "syslog", "grokked" ]
      }
   }
}


filter {
   if [type] == "security" {
      grok {
            match => [ "message", "%{SYSLOGBASE}" ]
            add_tag => [ "security", "grokked" ]
      }
   }
}



output {

  stdout {
           #debug => true
           #debug_format => "json"
    }

  elasticsearch {
    host => "logs.mydomain.com"
  }
}

And in /etc/logstash/conf.d/30-lumberjack-output.conf I have ouput going to ES:

output {
  elasticsearch { host => localhost }
  stdout { codec => rubydebug }
}

And now, after restarting logstash again, I see that logstash is listening on the ports that I specify in the config:

[root@logs:/etc/logstash] #lsof -i :5000
COMMAND   PID     USER   FD   TYPE   DEVICE SIZE/OFF NODE NAME
java    23893 logstash   16u  IPv6 11665234      0t0  TCP *:commplex-main (LISTEN)
[root@logs:/etc/logstash] #lsof -i :2541
COMMAND   PID     USER   FD   TYPE   DEVICE SIZE/OFF NODE NAME
java    23893 logstash   18u  IPv6 11665237      0t0  TCP *:lonworks2 (LISTEN)

As of now logstash is running and not producing any log output:

#ps -ef | grep logstash | grep -v grep
logstash 23893     1 16 11:49 ?        00:01:45 /bin/java -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -Djava.awt.headless=true -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Djava.io.tmpdir=/var/lib/logstash -Xmx500m -Xss2048k -Djffi.boot.library.path=/opt/logstash/vendor/jruby/lib/jni -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -Djava.awt.headless=true -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Djava.io.tmpdir=/var/lib/logstash -Xbootclasspath/a:/opt/logstash/vendor/jruby/lib/jruby.jar -classpath : -Djruby.home=/opt/logstash/vendor/jruby -Djruby.lib=/opt/logstash/vendor/jruby/lib -Djruby.script=jruby -Djruby.shell=/bin/sh org.jruby.Main --1.9 /opt/logstash/lib/bootstrap/environment.rb logstash/runner.rb agent -f /etc/logstash/conf.d -l /var/log/logstash/logstash.log

ls -lh /var/log/logstash/logstash.log
-rw-r--r--. 1 logstash logstash 0 Jun 22 11:49 /var/log/logstash/logstash.log

But still there aren't any indexes created in elasticsearch:

#curl http://localhost:9200/_cat/indices
yellow open .kibana  1 1 1 0 2.4kb 2.4kb
yellow open security 5 1 0 0  575b  575b

And when I go to configure Kibana it says that it can't find any patterns to search on using "logstash-*".

Where can I go from here to get this to work? The configs themselves are unchanged from what I've shown you before.

I haven't tried pointing any logstash forwarders to it out.. but I tried writing stdin to the elasticsearch cluster with this command:

logstash -e 'input { stdin { } } output { elasticsearch { host => localhost } }'

And I got this error back:

`Got error to send bulk of actions: blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];[SERVICE_UNAVAILABLE/2/no master]; {:level=>:error}
Failed to flush outgoing items {:outgoing_count=>1, :exception=>org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];[SERVICE_UNAVAILABLE/2/no master];, :backtrace=>["org.elasticsearch.cluster.block.ClusterBlocks.globalBlockedException(org/elasticsearch/cluster/block/ClusterBlocks.java:151)", "org.elasticsearch.cluster.block.ClusterBlocks.globalBlockedRaiseException(org/elasticsearch/cluster/block/ClusterBlocks.java:141)", "org.elasticsearch.action.bulk.TransportBulkAction.executeBulk(org/elasticsearch/action/bulk/TransportBulkAction.java:210)", "org.elasticsearch.action.bulk.TransportBulkAction.access$000(org/elasticsearch/action/bulk/TransportBulkAction.java:73)", "org.elasticsearch.action.bulk.TransportBulkAction$1.onFailure(org/elasticsearch/action/bulk/TransportBulkAction.java:148)", "org.elasticsearch.action.support.TransportAction$ThreadedActionListener$2.run(org/elasticsearch/action/support/TransportAction.java:137)", "java.util.concurrent.ThreadPoolExecutor.runWorker(java/util/concurrent/ThreadPoolExecutor.java:1142)", "java.util.concurrent.ThreadPoolExecutor$Worker.run(java/util/concurrent/ThreadPoolExecutor.java:617)", "java.lang.Thread.run(java/lang/Thread.java:745)"], :level=>:warn}`

Any thoughts on what could be happening?

I wrote a small bash script to use with nagios to check if nrpe is running.

The check works locally when run as root, but fails from the monitoring host.

From the host I'm trying to monitor, I have this line in my nrpe.conf:

 command[check_nrpe]=/usr/lib64/nagios/plugins/check_nrpe.sh

And made sure the check script is owned by the nagios user:

[root@ops:~] #ls -l /usr/lib64/nagios/plugins/check_nrpe.sh
-rwxr-xr-x. 1 **nagios nagios** 203 Jun  9 20:29     **/usr/lib64/nagios/plugins/check_nrpe.sh**

And if I run the script as the root user I get the correct result:

 [root@ops:~] #/usr/lib64/nagios/plugins/check_nrpe.sh OK: NRPE is running with pid: 24538
24538

But when I run it from the nagios host the check produces the opposite result:

[root@monitor1:~] #/usr/local/nagios/libexec/check_nrpe -H ops.mydomain.com -c     check_nrpe
**CRITICAL: NRPE is **NOT** Running**

If I go back to the host I'm trying to monitor and become the nagios user I get the same incorrect result as I do on the nagios host.

[root@ops:~] #su - nagios
Last login: Tue Jun  9 20:43:42 UTC 2015 on pts/3

-bash-4.2$ /usr/lib64/nagios/plugins/check_nrpe.sh
**CRITICAL: NRPE is **NOT** Running**

If I give the nagios user sudo access to that script, I can get the correct result as the nagios user on the local host.

In /etc/sudoers I give the nagios user access to the command and disabled tty by putting:

    nagios ALL=(ALL)    NOPASSWD: /usr/lib64/nagios/plugins/check_nrpe.sh    !requiretty

And now if I become the nagios user on the local host and use sudo the check produces the correct result.

[root@ops:~] #su - nagios
Last login: Tue Jun  9 23:37:09 UTC 2015 on pts/0

-bash-4.2$ sudo /usr/lib64/nagios/plugins/check_nrpe.sh
**OK: NRPE is running with pid: 24538**
24538

If I then edit my nrpe conf file on the local host to use sudo before command. In nrpe.conf I put:

[root@ops:~] #grep check_nrpe /etc/nagios/nrpe.cfg
command[check_nrpe]=/bin/sudo /usr/lib64/nagios/plugins/check_nrpe.sh

And restarted the nrpe service:

[root@ops:~] #systemctl restart nrpe
[root@ops:~] #lsof -i :5666
COMMAND  PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
nrpe    6137 nrpe    4u  IPv4 493404      0t0  TCP *:5666 (LISTEN)
nrpe    6137 nrpe    5u  IPv6 493405      0t0  TCP *:5666 (LISTEN)

But when I go back to the nagios host and run the check again, I get an output error:

[root@monitor1:~] #/usr/local/nagios/libexec/check_nrpe -H ops.jokefire.com -c     check_nrpe
 **NRPE: Unable to read output**

This is the contents of my check nrpe script:

[root@ops:~] #cat /usr/lib64/nagios/plugins/check_nrpe.sh
#!/bin/bash

pid=$(lsof -i :5666 | awk '{print $2}' | grep -i -v pid)

if [[ $pid ]]
then
  echo "OK: NRPE is running with pid: $pid"
  exit 0
else
  echo "CRITICAL: NRPE is **NOT** Running"
  exit 2
fi

HELP!! How do I get this check to return the correct result from the nagios host?

Thanks

I don't remember changing anything at all on my logstash server. As of just yesterday it was working fine! And I used it to run some queries.

However.. today when I went to my logstash page, I see only thing message come up:

Connection Failed

Possibility #1: Your elasticsearch server is down or unreachable

This can be caused by a network outage, or a failure of the Elasticsearch process. If you have     recently run a query that required a terms facet to be executed it is possible the process has run out     of memory and stopped. Be sure to check your Elasticsearch logs for any sign of memory pressure.

Possibility #2: You are running Elasticsearch 1.4 or higher

Elasticsearch 1.4 ships with a security setting that prevents Kibana from connecting. You will need     to set http.cors.allow-origin in your elasticsearch.yml to the correct protocol, hostname, and port (if     not 80) that your access Kibana from. Note that if you are running Kibana in a sub-url, you should     exclude the sub-url path and only include the protocol, hostname and port. For example,     http://mycompany.com:8080, not http://mycompany.com:8080/kibana.

Click back, or the home button, when you have resolved the connection issue

And I can easily tell that elasticsearch 1.1.1 is running on that host:

[root@logs:~] #ps -ef | grep elasticsearch | grep -i -v -e grep -e screen

root     16666  9640  6 09:00 pts/1    00:05:49 /etc/alternatives/javahome/bin/java -Xms256m -Xmx1g     -Xss256k -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -    XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError     -Delasticsearch -Des.foreground=yes -Des.path.home=/usr/local/elasticsearch-1.1.1 -cp     :/usr/local/elasticsearch-1.1.1/lib/elasticsearch-1.1.1.jar:/usr/local/elasticsearch-    1.1.1/lib/*:/usr/local/elasticsearch-1.1.1/lib/sigar/* org.elasticsearch.bootstrap.Elasticsearch

And I can see, in fact that elastic search is listening on the right ports:

[root@logs:~] #netstat -tulpn | grep -i listen | grep java

tcp        0      0 :::2541                     :::*                        LISTEN      16722/java

tcp        0      0 :::9200                     :::*                        LISTEN      16666/java    

tcp        0      0 :::9300                     :::*                        LISTEN      16666/java

tcp        0      0 :::9301                     :::*                        LISTEN      16722/java

All of the above ports are for elasticsearch. I use port 2541 for logstash. That's fine too.

And logstash is doing it's thing as well. I usually leave logstash running in verbose tucked away in a screen session. And when I go to that screen session I can see all the log information flying by.

I've tried stopping and starting both elasticsearch as well as logstash. But I still see the same message.

Can someone please let me know what's going on? How do I correct this?

Thanks

I have an SSL site I'm trying to deliver with apache that is timing out when you load it in a web browser.

I am seeing this message in the browser:

The connection has timed out

The server at solr1.mydomain.com is taking too long to respond.

The site could be temporarily unavailable or too busy. Try again in a few moments.
If you are unable to load any pages, check your computer's network connection.
If your computer or network is protected by a firewall or proxy, make sure that Firefox is permitted to access the Web.

I checked the logs and I am seeing SSL errors in the logs that say:

[Sun Nov 02 17:43:13.859447 2014] [ssl:warn] [pid 23687] AH01909: RSA certificate configured for solr1.mydomain.com:443 does NOT include an ID which matches the server name

Which is strange. Because when creating the certs for the site, I made sure to specify the output of hostname -f on the server for the common name.

This is an amazon EC2 instance. I'm not sure if that could have something to do with this problem.

Here is my apache configuration:

LoadModule    jk_module  modules/mod_jk.so
JkWorkersFile /etc/httpd/conf/workers.properties
JkShmFile     /var/log/httpd/mod_jk.shm
JkLogFile     /var/log/httpd/mod_jk.log
JkLogLevel    info
JkLogStampFormat "[%a %b %d %H:%M:%S %Y] "

Virtual Host config:

 <VirtualHost *:80>
   ServerName solr1.mydomain.com
   ErrorLog logs/solr1_error_80_log
       LogFormat "{ \
           \"host\":\"solr1.mydomain.com\", \
      \"path\":\"/var/log/httpd/solr1_access_log\", \
      \"tags\":[\"mydomain Trac 80\",\"solr1.mydomain.com\"], \
      \"message\": \"%h %l %u %t \\\"%r\\\" %>s %b\", \
      \"timestamp\": \"%{%Y-%m-%dT%H:%M:%S%z}t\", \
      \"clientip\": \"%a\", \
      \"duration\": %D, \
      \"status\": %>s, \
      \"request\": \"%U%q\", \
      \"urlpath\": \"%U\", \
      \"urlquery\": \"%q\", \
      \"method\": \"%m\", \
      \"bytes\": %B, \
      \"vhost\": \"%v\" \
    }" solr1_access_json
   CustomLog  logs/solr1_80_access_log solr1_80_access_json
   RewriteEngine On
   RewriteCond %{HTTPS} off
   RewriteRule (.*) https ://%{HTTP_HOST}%{REQUEST_URI}
   RewriteRule  ^/$    /solr [L,R=301]

   </VirtualHost>

   <VirtualHost *:443>
   RewriteEngine On
   RewriteRule  ^/$    /solr [L,R=301]

   ServerName solr1.mydomain.com
   ErrorLog logs/solr1_error_443_log
   LogFormat "{ \
      \"host\":\"solr1.mydomain.com\", \
      \"path\":\"/var/log/httpd/solr1_443_access_log\", \
      \"tags\":[\"mydomain Trac 443\",\"solr1.mydomain.com\"], \
      \"message\": \"%h %l %u %t \\\"%r\\\" %>s %b\", \
      \"timestamp\": \"%{%Y-%m-%dT%H:%M:%S%z}t\", \
      \"clientip\": \"%a\", \
      \"duration\": %D, \
      \"status\": %>s, \
      \"request\": \"%U%q\", \
      \"urlpath\": \"%U\", \
      \"urlquery\": \"%q\", \
      \"method\": \"%m\", \
      \"bytes\": %B, \
      \"vhost\": \"%v\" \
    }" solr1_443_access_json
   CustomLog  logs/solr1_443_access_log solr1_443_access_json

   SSLEngine on
   SSLCertificateFile /etc/pki/tls/certs/solr1.crt
   SSLCertificateKeyFile /etc/pki/tls/private/solr1.key

   # Select the timestamp log format
   JkLogStampFormat "[%a %b %d %H:%M:%S %Y] "
   # Send everything for context /examples to worker named worker1 (ajp13)
   <Directory /usr/share/tomcat/webapps/solr>
      Options  +Indexes +FollowSymLinks
      AllowOverride All
      Order allow,deny
      allow from all
    </Directory>

   Alias /solr /usr/share/tomcat/webapps/solr
   JkMount  /test/* worker1
   JkMount  /solr/* worker1

   <Location "/solr/">
     AuthType Basic
     AuthName "JF Admin Page"
     AuthUserFile /etc/httpd/auth
     Require valid-user
   </Location>

 </VirtualHost>

I'm hoping I can get some advice that can help me get past this problem!