My developer hypothesis is packet is getting dropped at arp layer. We increased the default value of neigh.default.gc_thresh3 1024 --> 2048 and now everything looks good. But I want to understand is there is a way to figure it out arp packet loss.I try to search for systemtap/tools but didn't find anything. Any help is really appreciated.
net.ipv4.neigh.default.gc_thresh3=<n>
I am following this doc
https://www.terraform.io/docs/providers/aws/d/kms_secrets.html
As I want to store mysql password in encrypted format
This looks good
$ aws kms encrypt --key-id arn:aws:kms:us-west-2:>id>:key/7e791977-123456 --plaintext fileb:///tmp/dbpass --output text --query CiphertextBlob
Now when I am trying to use in my code based on above doc
data "aws_kms_secret" "rds" {
secret {
name = "db-password"
payload = "pay load here"
}
}
Snip of my terraform code
resource "aws_db_instance" "my-test-sql" {
instance_class = "${var.db_instance}"
engine = "mysql"
engine_version = "5.7"
multi_az = true
storage_type = "gp2"
allocated_storage = 20
name = "mytestrds"
username = "admin"
password = "${data.aws_kms_secret.rds.db-password}"
It's failing due to this error
Error: Unsupported attribute
on rds/main.tf line 16, in resource "aws_db_instance" "my-test-sql":
16: password = "${data.aws_kms_secret.rds.db-password}"
This object has no argument, nested block, or exported attribute
named "db-password".
$ terraform version
Terraform v0.12.13
Is anyone run into similar sort of issue/error with terraform version 0.12?
Update
I even tried what suggested in doc
password = "${data.aws_kms_secret.rds.plaintext["db-password"]}"
But it's failing due to different error
Error: Unsupported attribute
on rds/main.tf line 14, in resource "aws_db_instance" "my-test-sql":
14: password = "${data.aws_kms_secret.rds.plaintext["db-password"]}"
This object has no argument, nested block, or exported attribute named "plaintext".
I am new to docker swarm and want to understand, how docker stack deploy works
version: '3'
services:
web:
image: nginx
ports:
- "80:80"
redis:
image: "redis:alpine"
When I run this command, it works perfectly fine
$ docker stack deploy -c docker-compose.yml mystack
Creating network mystack_default
Creating service mystack_web
Creating service mystack_redis
To verify it
$ docker stack ps mystack
ID NAME IMAGE NODE
DESIRED STATE CURRENT STATE ERROR PORTS
rcci590y5c6o mystack_redis.1 redis:alpine
plakhera1.example.com Running Running 9 seconds ago
lty84a77b4my mystack_web.1 nginx:latest
plakhera2.mylabserver.com Running Running 2 seconds ago
Now my question is how to make sure that every time I deploy this application/service Redis will come up on plakhera1.example.com and nginx on plakhera2.example.com. I believe currently its picking nodes randomly, so how can I make sure it always pick only those nodes.
Update: The scenario is my web is the basic app but the Redis does a lot of heavy lifting, that why I always want Redis to be on the beefy node, is there is any way I can specify via docker stack deploy that web always boots up on node1 and Redis on node2
I am trying to write a script or one liner to find out docker image is used by which docker container
eg: Let say I have a docker image 9f38484d220f, now I need to find out which container is created using this image.
I know docker inspect <container id> give me both Image as well as a container id but is there is any other way to do that?
I am trying to figure out how logrotate work regarding disk space but not able to figure it out?
Scenario:
Let say I have 2GB of the log file and free disk space is also 2 GB, does logrotate first check do we have a sufficient disk space before rotating/compressing that file
In this scenario is it going to fail as it needs 2GB of space?
OR
logrotate going to compress this file let say ~400MB as it thinks it already has sufficient space?
Anyone knows why I am seeing multiple device for the same LUN?I checked with storage team they only exposed 360000970000196801239533036304532 for this server but why I am seeing these many partition(p1..p7)
lrwxrwxrwx 1 root root 8 Jan 20 16:30 360000970000196801239533036304532 -> ../dm-39
lrwxrwxrwx 1 root root 8 Jan 20 16:30 360000970000196801239533036304532p1 -> ../dm-64
lrwxrwxrwx 1 root root 8 Jan 20 16:30 360000970000196801239533036304532p2 -> ../dm-65
lrwxrwxrwx 1 root root 8 Jan 20 16:30 360000970000196801239533036304532p3 -> ../dm-66
lrwxrwxrwx 1 root root 8 Jan 20 16:30 360000970000196801239533036304532p4 -> ../dm-67
lrwxrwxrwx 1 root root 8 Jan 20 16:30 360000970000196801239533036304532p5 -> ../dm-69
lrwxrwxrwx 1 root root 8 Jan 20 16:30 360000970000196801239533036304532p6 -> ../dm-70
lrwxrwxrwx 1 root root 8 Jan 20 16:30 360000970000196801239533036304532p7 -> ../dm-71
AFAIK I understand devices in dev mapper are created early in the boot process and /dev/dm-n are for internal use only
Even in the output of fdisk I see those 2 devices
Disk /dev/mapper/360000970000196801239533036304532: 515.4 GB, 515396075520 bytes
Device Boot Start End Blocks Id System
/dev/mapper/360000970000196801239533036304532p1 1 62661 503316479+ ee GPT
Disk /dev/mapper/360000970000196801239533036304536: 515.4 GB, 515396075520 bytes
When I am trying to create Physical Volume on original LUN it fails due to this error
pvcreate /dev/mapper/360000970000196801239533036304532
Device /dev/mapper/360000970000196801239533036304532 not found (or ignored by filtering).
I already checked lvm.conf there is nothing defined in filtering
System info
cat /etc/redhat-release
CentOS release 6.8 (Final)
device-mapper-multipath-0.4.9-93.el6.x86_64
device-mapper-persistent-data-0.6.2-0.1.rc7.el6.x86_64
device-mapper-multipath-libs-0.4.9-93.el6.x86_64
device-mapper-libs-1.02.117-7.el6_8.1.x86_64
device-mapper-event-libs-1.02.117-7.el6_8.1.x86_64
device-mapper-event-1.02.117-7.el6_8.1.x86_64
lvm2-2.02.143-7.el6_8.1.x86_64
device-mapper-1.02.117-7.el6_8.1.x86_64
lvm2-libs-2.02.143-7.el6_8.1.x86_64
I am trying to copy(~7TB of data using rsync) between two server in same data center in the backend its using EMC VMAX3
After copying ~30-40GB of data multipath start failing
Dec 15 01:57:53 test.example.com multipathd: 360000970000196801239533037303434: Recovered to normal mode
Dec 15 01:57:53 test.example.com multipathd: 360000970000196801239533037303434: remaining active paths: 1
Dec 15 01:57:53 test.example.com kernel: sd 1:0:2:20: [sdeu] Result: hostbyte=DID_ERROR driverbyte=DRIVER_OK
[root@test log]# multipath -ll |grep -i fail
|- 1:0:0:15 sdq 65:0 failed ready running
- 3:0:0:15 sdai 66:32 failed ready running
We are using default multipath.conf
HBA driver version 8.07.00.26.06.8-k
HBA model QLogic Corp. ISP8324-based 16Gb Fibre Channel to PCI Express Adapter
OS: CentOS 64-bit/2.6.32-642.6.2.el6.x86_64
Hardware:Intel/HP ProLiant DL380 Gen9
Already verified this solution and checked with EMC everything looks good https://access.redhat.com/solutions/438403
Some more info
- There is no drop/error packet on the network side.
Already disabled THP
echo never > /sys/kernel/mm/redhat_transparent_hugepage/enabled
Vmcore collected during the time of issue
I am able to collect vmcore during the time of issue
KERNEL: /usr/lib/debug/lib/modules/2.6.32-642.6.2.el6.x86_64/vmlinux
DUMPFILE: vmcore [PARTIAL DUMP]
CPUS: 36
DATE: Fri Dec 16 00:11:26 2016
UPTIME: 01:48:57
LOAD AVERAGE: 0.41, 0.49, 0.60
TASKS: 1238
NODENAME: test.example.com
RELEASE: 2.6.32-642.6.2.el6.x86_64
VERSION: #1 SMP Wed Oct 26 06:52:09 UTC 2016
MACHINE: x86_64 (2297 Mhz)
MEMORY: 511.9 GB
PANIC: "BUG: unable to handle kernel NULL pointer dereference at 0000000000000018"
PID: 15840
COMMAND: "kjournald"
TASK: ffff884023446ab0 [THREAD_INFO: ffff88103def4000]
CPU: 2
STATE: TASK_RUNNING (PANIC)
After Enbaling Debug mode on the qlogic sid
qla2xxx [0000:0b:00.0]-3822:5: FCP command status: 0x2-0x0 (0x70000) nexus=5:1:0 portid=1f0160 oxid=0x800 cdb=2a200996238000038000 len=0x70000 rsp_info=0x0 resid=0x0 fw_resid=0x0 sp=ffff882189d42580 cp=ffff88276d249480.
qla2xxx [0000:84:00.0]-3822:7: FCP command status: 0x2-0x0 (0x70000) nexus=7:0:3 portid=450000 oxid=0x4de cdb=2a20098a5b0000010000 len=0x20000 rsp_info=0x0 resid=0x0 fw_resid=0x0 sp=ffff882189d421c0 cp=ffff8880237e0880.
I need to match this entry
2015/10/30 23:58:21 pid 22223 [email protected] 192.168.0.1 [p4/2012.2/LINUX26X86_64/536738] 'test-monitor show'
To match this I wrote this Regex
P4_DATE (?>\d\d){1,2}\/(?:0[1-9]|1[0-2])\/(?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9])
P4_TIME (?:2[0123]|[01]?[0-9])\:(?:[0-5][0-9])\:(?:(?:[0-5]?[0-9]|60)(?:[:.,][0-9]+)?)
P4_PID \b(?:[1-9][0-9]*)\b
P4_USER \b\w+\b
P4_HOSTNAME \b(?:[0-9A-Za-z][0-9A-Za-z-]{0,62})(?:\.(?:[0-9A-Za-z][0-9A-Za-z-]{0,62}))*(\.?|\b)
P4_IP (?<![0-9])(?:(?:[0-1]?[0-9]{1,2}|2[0-4][0-9]|25[0-5])[.](?:[0-1]?[0-9]{1,2}|2[0-4][0-9]|25[0-5])[.](?:[0-1]?[0-9]{1,2}|2[0-4][0-9]|25[0-5])[.](?:[0-1]?[0-9]{1,2}|2[0-4][0-9]|25[0-5]))(?![0-9])
and then overall
P4_MATCH %(P4_DATE:p4date} %{P4_TIME:p4time} pid %{P4_PID:p4pid} %{P4_USER:p4user}\@%{P4_HOSTNAME:p4client} %{P4_IP:p4remoteclient} [%{DATA:p4version}] \'%{DATA:p4action}\'" }
and then match via
match => [ "message", "%{P4_MATCH}" ]
But still getting "_beats_input_codec_plain_applied, _grokparsefailure"
I am not a regular expression expert but any help is really appreciated.
I have a really weird requirement (Security team doesn't even want people who don't have access to see the Jenkins UI), where I want to run Jenkins behind reverse proxy (Apache) and then secure it via Active Directory Authentication.
Here is my config
LDAPCacheEntries 0
LDAPCacheTTL 5
LDAPOpCacheEntries 0
LDAPOpCacheTTL 5
<VirtualHost 0.0.0.0:80>
ServerName localhost
RequestHeader unset WWW-Authenticate
ProxyPass / http://localhost:8080/ retry=1 acquire=3000 timeout=600 ttl=60 Keepalive=On
ProxyPassReverse / http://localhost:8080/
ProxyRequests Off
ProxyPreserveHost On
AllowEncodedSlashes On
<Proxy http://localhost:8080/>
Order deny,allow
Allow from all
</Proxy>
<Location />
AuthLDAPBindDN “testuser”
AuthLDAPBindPassword “testpasswd”
AuthLDAPURL "ldaps://testserver.example.com:636/ou=test,ou=users,dc=internal,dc=example,dc=com?sAMAccountName?sub?(objectClass=user)"
AuthType Basic
AuthName "AD/LDAP Credentials"
AuthBasicProvider ldap
AuthUserFile /dev/null
require ldap-group CN=Admin,OU=Security,OU=Groups,DC=internal,DC=example,DC=com
</Location>
LogLevel debug
ErrorLog /var/log/httpd/jenkins-error_log
CustomLog /var/log/httpd/jenkins-access_log common
Now everything looks OK, except that the page takes forever to load and sometimes I get a timeout (that's why I have the 600 sec in timeout value). So just want to confirm if there is anything wrong with my config?
I am new to AWS this might sound like a weird question.My question is when I try to create a new EBS volume and try to attach it with running EC2 instance,The connectivity between two(EC2 instance --> EBS volumes) is over Public IP or that connectivity is using Private IP.I understand that EBS provides just a block level storage but I want to understand how EC2 instance is going to access that storage.
I am facing this issue intermittently when I am trying to check-in files
svn ci -m "commit message" file
Warning: post commit FS processing had error:
sqlite[S5]: database is locked
Some more info
Server side OS --> Centos6.5 and svn version 1.8.5
Client Side OS --> Centos 5.11 and svn version 1.7.7
I already check some forums and these are the suggestion but it doesn't resolve my issue
1: Check server load(Server is fairly idle when I faced this issue..checked load average,IO and memory utilization)
2: There is a longer commit is in progress at that time
3: Both server and client are in same geographic location
As the error is intermittent and I checked svn repo file is there(i.e file checked in properly)so I am not sure what is the issue.Any help is really appreciated.
Sorry I am newbie to Elasticsearch and I am using elasticsearch-river-jdbc to connect to myql database,everything is working fine except for the fact that everytime it run as per schedule its inserting duplicate record.This is what I am using
curl -XPUT 'localhost:9200/_river/my_jdbc_river/_meta' -d '{
"type" : "jdbc",
"schedule" : "0 0-59 0-23 ? * *",
"jdbc" : {
"url" : "jdbc:mysql://localhost:3306/test",
"user" : "test",
"password" : "test",
"sql" : "select * from test"
}
}'
I go through some docs where it's mentioned that we can run sql query select based on _id but my question is when we created this river only then this unique id is created and that is created on the Elasticsearch side so as per my understanding mysql has no knowledge about this.Please let me know if I am missing something
So if I am writing sql satement like this
"sql" : "select id as _id,a1,a2 from test"
[2015-03-10 13:16:00,018][ERROR][river.jdbc.RiverPipeline ] com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown column 'id' in 'field list'
java.io.IOException: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Unknown column 'id' in 'field list'
First of all I apologize for this,I am pretty bad in regular-expression and try to wrote custom pattern(as I am unable to find something in existing grok pattern or may be I am missing something) for parsing svn logs which is in the format of
r24|prashant|2015-02-26 12:38:04 -0800 (Thu, 26 Feb 2015)|33|Log: ABC-123 / Initial version||A test/log_testing1 A test/log_testing2 A test/log_testing3 A test/log_testing4 A test/log_testing5 \n
So it's in the format of
$REVISION:$USER ID:$DATE:$CHECKED IN MESSAGE:$FILE CHECKED IN
So I wrote some custom pattern
SVN [r0-9]
SVN_TIMESTAMP %{YEAR}-%{MONTHNUM}-%{MONTHDAY} %{HOUR}:?%{MINUTE}(?::?%{SECOND})?%{ISO8601_TIMEZONE}? (%{DAY}, %{MONTHDAY} %{MONTH} %{YEAR})
and the my logstash-conf would look like this for filter section
filter {
grok {
match => { "message" => "%{SVN:revision}|%{USERNAME:username}|%{SVN_TIMESTAMP:svntimestamp}|%{GREEDYDATA:syslog_message}||%{GREEDYDATA:syslog_message" }
}
}
I am not sure it's correct but as usual it's not working.Any help is really appreciated
My requirement is to send subversion logs(i.e username,revision number...) to logstash for parsing(then store it in elastic search and finally displayed it via kibana).Since subversion use its own file-based database(FSFS) and not plain text file,I have two options
I also thought about post commit script that whenever any user checkin it will trigger it and then store the logs in text file,but its same as Point 1.
Any help/idea to do that is really appreciated
EDITWe write this small post-commit as I mentioned in point 3 so that whenever user check-in we can save the metadata in some file and then via syslog we can transfer this log to logstash server.One of the biggest drawback of this approach is I am dealing with TB of data and 15+ checkin per min,this file goes really big(we can use logrotate) but at the same time facing locking condition issue(as multiple user try to check-in and writing to the same file) which will eventually lead to race condition and make the situation more worst.Pasting post-commit hook below so that it might be useful for other people
#!/bin/sh
REPOS="$1"
REV="$2"
LOG="/tmp/svn.log"
var1=/usr/bin/svnlook info -r $REV $REPOS | tr '\n' '|'`
var2=/usr/bin/svnlook changed -r $REV $REPOS | tr '\n' ' '`
echo "r${REV}|${var1}|${var2}\n" | tee -a ${LOG} 2>&1
echo " " | tee -a ${LOG} 2>&1
First of all I know this is a complete mess but due to some developer legacy code(hard coded value),I am forced to install 32 bit rpm on 64 bit machine.
cat /etc/redhat-release
CentOS release 6.5 (Final)
Problem is when I am trying to install any package its pulling both 32 as well as 64bit RPM,I am aware of the fact that
multilib_policy=all(by default)
so it try to pull both package, and as this is a 64bit machine even setting multilib_policy=best will not help,so I try to exclude it in yum.conf using
exclude=*.x86_64
but still no luck.Any help is appreciated.
yum install gcc
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package gcc.x86_64 0:4.4.7-3.el6 will be updated
---> Package gcc.i686 0:4.4.7-4.el6 will be an update