I have an application distributed as docker image basing on CentOS 7.
Also my host machine running the docker contains is CentOS 7.
Recently RedHat released 5 important patches to fix kernel flaws.
I have yum update the host machine
Do I need update the docker image with yum update?
I guess kernel flaws won't affect docker containers, right?
I am trying to setup a CouchDB cluster with 3 nodes in my local machine. I am using the official docker image from here: https://hub.docker.com/r/klaemo/couchdb/
First, I create 3 data volumes which will be used by the containers.
docker create -v /opt/couchdb/data --name couchdb_d1 klaemo/couchdb:2.0.0
docker create -v /opt/couchdb/data --name couchdb_d2 klaemo/couchdb:2.0.0
docker create -v /opt/couchdb/data --name couchdb_d3 klaemo/couchdb:2.0.0
Then create a user-defined network, so that these 3 nodes can communicate with each other.
docker network create --driver bridge couch
Then start three containers, each of them runs a CouchDB nodes
docker run -d --name couchdbA \
--net=couch \
--hostname couchdbA \
-p 5984:5984 \
-e NODENAME=couchdbA.couch \
-e COUCHDB_USER=root \
-e COUCHDB_PASSWORD=654321 \
-v /opt/couchdb/data \
--volumes-from couchdb_d1 \
"klaemo/couchdb:2.0.0"
docker run -d --name couchdbB \
--net=couch \
--hostname couchdbB \
-p 15984:5984 \
-e NODENAME=couchdbB.couch \
-e COUCHDB_USER=root \
-e COUCHDB_PASSWORD=654321 \
-v /opt/couchdb/data \
--volumes-from couchdb_d2 \
"klaemo/couchdb:2.0.0"
docker run -d --name couchdbC \
--net=couch \
--hostname couchdbC \
-p 25984:5984 \
-e NODENAME=couchdbC.couch \
-e COUCHDB_USER=root \
-e COUCHDB_PASSWORD=654321 \
-v /opt/couchdb/data \
--volumes-from couchdb_d3 \
"klaemo/couchdb:2.0.0"
Next, I open the http://xxx.xxx.xxx.xxx:yyyyy/_utils/#setup on each of the container, and configure the cluster.
For couchdbA, I added the hosts
For couchdbB, I added the hosts
For couchdbC, I added the hosts
Everything looks fine so far, I see system databases are created in all three nodes.
Now I try to verify by installation by click the "Verify" button and it ends with an error.
The error message is unclear, and I can't find anything from log.
[debug] 2016-12-19T12:33:45.246054Z [email protected] <0.846.0> 0fbf5d2a41 Successful cookie auth as: "root"
[debug] 2016-12-19T12:33:45.246505Z [email protected] <0.2648.0> eabba5f813 no record of user root
[debug] 2016-12-19T12:33:45.246530Z [email protected] <0.2648.0> eabba5f813 timeout 6000
[debug] 2016-12-19T12:33:45.246547Z [email protected] <0.2648.0> eabba5f813 Successful cookie auth as: "root"
[debug] 2016-12-19T12:33:45.247127Z [email protected] <0.847.0> fdbd749b7a cache miss for root
[debug] 2016-12-19T12:33:45.247444Z [email protected] <0.847.0> fdbd749b7a no record of user root
[debug] 2016-12-19T12:33:45.247475Z [email protected] <0.847.0> fdbd749b7a timeout 6000
[debug] 2016-12-19T12:33:45.247495Z [email protected] <0.847.0> fdbd749b7a Successful cookie auth as: "root"
[notice] 2016-12-19T12:33:45.248915Z [email protected] <0.2648.0> eabba5f813 10.0.5.96:5984 10.0.5.233 root GET /_users 200 ok 3
[notice] 2016-12-19T12:33:45.250319Z [email protected] <0.846.0> 0fbf5d2a41 10.0.5.96:5984 10.0.5.233 root GET /_replicator 200 ok 5
[notice] 2016-12-19T12:33:45.250486Z [email protected] <0.847.0> fdbd749b7a 10.0.5.96:5984 10.0.5.233 root GET /ts 200 ok 4
[debug] 2016-12-19T12:33:45.259598Z [email protected] <0.847.0> c06a5322af cache miss for root
[debug] 2016-12-19T12:33:45.260079Z [email protected] <0.847.0> c06a5322af no record of user root
[debug] 2016-12-19T12:33:45.260115Z [email protected] <0.847.0> c06a5322af timeout 6000
[debug] 2016-12-19T12:33:45.260137Z [email protected] <0.847.0> c06a5322af Successful cookie auth as: "root"
[notice] 2016-12-19T12:33:45.263568Z [email protected] <0.847.0> c06a5322af 10.0.5.96:5984 10.0.5.233 root GET /_utils/dashboard.assets/img/couchdb-logo.png 200 ok 4
[debug] 2016-12-19T12:33:47.651581Z [email protected] <0.847.0> 26f439ef76 cache miss for root
[debug] 2016-12-19T12:33:47.652225Z [email protected] <0.846.0> ae16fb68d4 cache miss for root
[debug] 2016-12-19T12:33:47.652505Z [email protected] <0.847.0> 26f439ef76 no record of user root
[debug] 2016-12-19T12:33:47.652542Z [email protected] <0.847.0> 26f439ef76 timeout 6000
[debug] 2016-12-19T12:33:47.652562Z [email protected] <0.847.0> 26f439ef76 Successful cookie auth as: "root"
[debug] 2016-12-19T12:33:47.652626Z [email protected] <0.846.0> ae16fb68d4 no record of user root
[debug] 2016-12-19T12:33:47.652663Z [email protected] <0.846.0> ae16fb68d4 timeout 6000
[debug] 2016-12-19T12:33:47.653002Z [email protected] <0.846.0> ae16fb68d4 Successful cookie auth as: "root"
[notice] 2016-12-19T12:33:47.653907Z [email protected] <0.846.0> ae16fb68d4 10.0.5.96:5984 10.0.5.233 undefined GET /verifytestdb_replicate 404 ok 2
[notice] 2016-12-19T12:33:47.654781Z [email protected] <0.847.0> 26f439ef76 10.0.5.96:5984 10.0.5.233 undefined GET /verifytestdb 404 ok 3
[debug] 2016-12-19T12:33:47.660531Z [email protected] <0.847.0> 095c417920 cache miss for root
[debug] 2016-12-19T12:33:47.660915Z [email protected] <0.847.0> 095c417920 no record of user root
[debug] 2016-12-19T12:33:47.660950Z [email protected] <0.847.0> 095c417920 timeout 6000
[debug] 2016-12-19T12:33:47.660969Z [email protected] <0.847.0> 095c417920 Successful cookie auth as: "root"
[notice] 2016-12-19T12:33:47.711571Z [email protected] <0.847.0> 095c417920 10.0.5.96:5984 10.0.5.233 undefined PUT /verifytestdb 500 ok 51
[debug] 2016-12-19T12:33:47.746832Z [email protected] <0.846.0> 8ce6af042b cache miss for root
[debug] 2016-12-19T12:33:47.748104Z [email protected] <0.846.0> 8ce6af042b no record of user root
[debug] 2016-12-19T12:33:47.748281Z [email protected] <0.846.0> 8ce6af042b timeout 6000
[debug] 2016-12-19T12:33:47.748392Z [email protected] <0.846.0> 8ce6af042b Successful cookie auth as: "root"
[notice] 2016-12-19T12:33:47.750359Z [email protected] <0.846.0> 8ce6af042b 10.0.5.96:5984 10.0.5.233 root GET /_utils/dashboard.assets/ZeroClipboard.swf?noCache=1482150820246 200 ok 4
Am I doing anything wrong?
My kubernetes cluster is running in Google Container Engine.
Once before I changed the machine-type by using a different instance template.
I am upgrading the kubernetes cluster with following command
gcloud container clusters upgrade CLUSTER_NAME --master
gcloud container clusters upgrade CLUSTER_NAME --cluster-version=1.2.1
After the upgrading, it seems the NODE is rolled back to previous instance template. How to upgrade the cluster without losing my instance template?
How to check the used disk space of Google Persistent Disk without looking into each of them?
Suppose I have a k8s cluster whose services IP range is 10.198.240.0/20.
Now I want this IP range to be accessible from another cluster or data center.
Is it feasible to run OpenVPN within kubernetes POD so that this IP range is accessible via VPN?
I can't use Google's HTTP load balance because it does not support web socket.
So my NGINX is running in Google Container Engine, behind Google Network Load Balance.
When the request arrives NGINX in docker container, it gets the IP of the NODE instead of the client ip.
2016/04/07 10:53:47 [error] 94#0: *73 access forbidden by rule, client: 10.132.0.9
How to solve this problem?
I have the following IP range as ClusterIP from Kubernetes cluster.
$ gcloud beta container clusters describe CLUSTER_NAME | grep servicesIpv4Cidr
servicesIpv4Cidr: 10.86.240.0/20
I want to allow this IP range accessible from my DC via VPN.
After adding the "Remote peer IP address" to the next hop from my DC
How should I create the route on GCE?
According to the doc of gcloud compute routes create command
Exactly one of --next-hop-address, --next-hop-gateway, --next-hop-vpn-tunnel, or --next-hop-instance must be provided with this command.
If I set one of the NODE to --next-hop-instance, what if this NODE is down?
How can I implement a route without single-point-of-failure?
When the service is created with ClusterIP Type, the cluster ip is fixed or it could be changed?
I wonder that because I want to access the ClusterIP via VPN from my own DC
If it is fixed, then the IP address can be configured in client directly
When creating the Target Pool for Network Load Balance, there is a health check option.
Also there is a property named livenessProbe in the container spec.
A liveness probe checks if the container in which it is configured is still running. If the liveness probe fails, the kubelet kills the container, which will be subjected to its restart policy. Set a liveness check by configuring the template.spec.containers.livenessprobe stanza of a pod configuration.
Is the health check needless when the container is configured with livenessProbe ?
As I understand, if the container is down, POD will be restarted automatically. In this case, no need for health check.
What about the NODE is down? As I understand, kubernetes will start the POD in another NODE, which means the POD will be restarted again.
It seems to me, in any case, health check is needless when livenessProbe is configured.
I am running pureftp in docker container as non-root user file_app.
Here is the permission I assigned
chown -R file_app:file_app /etc/pure-ftpd
chown -R file_app:file_app /etc/ssl/private
chown -R file_app:file_app /var/log/pure-ftp
chown -R file_app:file_app /usr/share/lintian/overrides/pure-ftpd
chown -R file_app:file_app /usr/share/nginx # FTP HOME DIR
When I try to start /usr/sbin/pure-ftpd as user file_app, I get this error
Unable to start a standalone server: Permission denied
When I start as root, everything works
How to run pure-ftpd as non-root user?
My development environments are running on Google Container Engine, and the following PODs created by Replication Controller
NAME READY STATUS RESTARTS AGE NODE
couchdb-dev-ocbud 1/1 Running 3 13h cz5w
couchdb-stage-8f9bn 1/1 Running 1 13h uqu4
etcd-1-rmwzy 1/1 Running 0 3d q0cz
etcd-2-n4ckp 1/1 Running 8 3d uqu4
etcd-3-yzz2x 1/1 Running 0 3d yt9e
mongodb-dev-ig9xo 1/1 Running 3 16h cz5w
mysql-dev-rykih 1/1 Running 3 17h cz5w
mysql-stage-n240p 1/1 Running 3 16h cz5w
redis-dev-19dxg 0/1 Running 5 3d cz5w
redis-dev-s5v6k 1/1 Running 0 3d yt9e
redis-dev-wccyb 0/1 Running 8 3d uqu4
redis-stage-qnbb6 0/1 Running 8 3d uqu4
redis-stage-xb54r 0/1 Running 0 3d yt9e
redis-stage-xntc2 0/1 Running 5 3d cz5w
shadowsocks-b8009 1/1 Running 0 2d q0cz
shadowsocks-i1anu 1/1 Running 0 2d yt9e
ts-stage-4esg8 1/1 Running 8 3d uqu4
ts-stage-cer7a 1/1 Running 5 3d cz5w
ts-stage-dtpdh 1/1 Running 0 3d yt9e
ts-stage-mah7w 1/1 Running 0 3d q0cz
uls-dev-upibo 1/1 Running 5 1d cz5w
uls-stage-zht0j 1/1 Running 6 1d uqu4
zookeeper-1-4dklm 1/1 Running 0 3d q0cz
zookeeper-2-pw13k 1/1 Running 8 3d uqu4
zookeeper-3-u9a34 1/1 Running 0 3d yt9e
PODs on NODE uqu4 were restarted for 8 times without my interaction.
Here is the termination reason from kubectl describe pod <pod>, error code is 137
Last Termination State: Terminated
Reason: Error
Exit Code: 137
Started: Mon, 21 Mar 2016 08:33:24 +0000
Finished: Mon, 21 Mar 2016 21:04:57 +0000
Ready: True
Restart Count: 8
When I ssh to the uqu4 node, I receives a warning as below
WARNING: Could not setup log file in /root/.config/gcloud/logs, (OSError: [Errno 28] No space left on device: '/root/.config/gcloud/logs/2016.03.22')
The df -h looks ok
Filesystem Size Used Avail Use% Mounted on
rootfs 99G 14G 82G 14% /
udev 10M 0 10M 0% /dev
tmpfs 750M 340K 750M 1% /run
/dev/disk/by-uuid/6be8ff15-205a-4019-99e0-92d9c347301b 99G 14G 82G 14% /
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 1.5G 1.7M 1.5G 1% /run/shm
cgroup 3.7G 0 3.7G 0% /sys/fs/cgroup
tmpfs 3.7G 8.0K 3.7G 1% /var/lib/kubelet/pods/46f374dc-ecbe-11e5-bf3b-42010af00080/volumes/kubernetes.io~secret/default-token-binen
tmpfs 3.7G 8.0K 3.7G 1% /var/lib/kubelet/pods/4a17371c-ecbe-11e5-bf3b-42010af00080/volumes/kubernetes.io~secret/default-token-binen
/dev/sdb 976M 187M 722M 21% /var/lib/kubelet/plugins/kubernetes.io/gce-pd/mounts/etcd-2-data-disk
/dev/sdb 976M 187M 722M 21% /var/lib/kubelet/pods/4a13021d-ecbe-11e5-bf3b-42010af00080/volumes/kubernetes.io~gce-pd/etcd-data
tmpfs 3.7G 8.0K 3.7G 1% /var/lib/kubelet/pods/4a13021d-ecbe-11e5-bf3b-42010af00080/volumes/kubernetes.io~secret/default-token-binen
/dev/sdc 976M 9.5M 900M 2% /var/lib/kubelet/plugins/kubernetes.io/gce-pd/mounts/zookeeper-2-data-disk
/dev/sdc 976M 9.5M 900M 2% /var/lib/kubelet/pods/4a5933ee-ecbe-11e5-bf3b-42010af00080/volumes/kubernetes.io~gce-pd/zookeeper-2-data
tmpfs 3.7G 8.0K 3.7G 1% /var/lib/kubelet/pods/4a5933ee-ecbe-11e5-bf3b-42010af00080/volumes/kubernetes.io~secret/default-token-binen
tmpfs 3.7G 8.0K 3.7G 1% /var/lib/kubelet/pods/b93210e7-ecfb-11e5-a962-42010af00080/volumes/kubernetes.io~secret/default-token-binen
/dev/sdd 30G 48M 28G 1% /var/lib/kubelet/plugins/kubernetes.io/gce-pd/mounts/uls-stage-data-disk
/dev/sdd 30G 48M 28G 1% /var/lib/kubelet/pods/f2764484-ee6b-11e5-a962-42010af00080/volumes/kubernetes.io~gce-pd/uls-stage-data-disk
tmpfs 3.7G 8.0K 3.7G 1% /var/lib/kubelet/pods/f2764484-ee6b-11e5-a962-42010af00080/volumes/kubernetes.io~secret/default-token-binen
/dev/sde 50G 52M 47G 1% /var/lib/kubelet/plugins/kubernetes.io/gce-pd/mounts/couchdb-stage-data-disk
/dev/sde 50G 52M 47G 1% /var/lib/kubelet/pods/e721dfb1-ef5b-11e5-a962-42010af00080/volumes/kubernetes.io~gce-pd/couchdb-stage-data-disk
tmpfs 3.7G 8.0K 3.7G 1% /var/lib/kubelet/pods/e721dfb1-ef5b-11e5-a962-42010af00080/volumes/kubernetes.io~secret/default-token-binen
/dev/disk/by-uuid/6be8ff15-205a-4019-99e0-92d9c347301b 99G 14G 82G 14% /var/lib/docker/aufs
none 99G 14G 82G 14% /var/lib/docker/aufs/mnt/8d9c854d1688439657c6b55107f6898d6b9fbdb74b9610dd0b48a1b22c6102d1
none 99G 14G 82G 14% /var/lib/docker/aufs/mnt/9e09bc6c69af03192569ba25762861edd710bf45baf65c449a4caf5ad69500f3
none 99G 14G 82G 14% /var/lib/docker/aufs/mnt/f82c122422db51310ce965173ca2b043ffa7b55b84f5b28bf9c19004a3e44fa9
none 99G 14G 82G 14% /var/lib/docker/aufs/mnt/6a0ccec3cedbcdf481a2ce528f2dcc9d1626f263591bebdb96a77beea0c0443f
none 99G 14G 82G 14% /var/lib/docker/aufs/mnt/ae8059fb1c2abbbffc72813a0355a4dd3d2633c720ef61b16d19a46ed2d63358
none 99G 14G 82G 14% /var/lib/docker/aufs/mnt/9d5b9ad1148e1ee4e10f826fc500f0a5c549bdc9ed66519e5f3b222d99641dfd
none 99G 14G 82G 14% /var/lib/docker/aufs/mnt/668f95f658cb13457b193f31716df5e5b8da7f227bc3ae1e0367098ec20580b0
none 99G 14G 82G 14% /var/lib/docker/aufs/mnt/bdf7d3660b81879c75a0048f921fa47b0138c3a9ec5454e85a55e62ccf9d86fe
none 99G 14G 82G 14% /var/lib/docker/aufs/mnt/8cb75d5e0df5d34ceefe41ec55a88198568a0670b6bddade4d8bb7194ba49779
none 99G 14G 82G 14% /var/lib/docker/aufs/mnt/a9bb332d1aebc349d1440416a59f898f9ed12be1c744e11e8f3e502dd630df0e
none 99G 14G 82G 14% /var/lib/docker/aufs/mnt/36a2bd14af419e19fe89fe32e3f02f490f5553246e76d6c7636ae80e6bba8662
none 99G 14G 82G 14% /var/lib/docker/aufs/mnt/a8c983eb3b1701263d1501b025f080ae0d967ddee2fd4bd5071e6e9297b389b9
none 99G 14G 82G 14% /var/lib/docker/aufs/mnt/e0131ab5360fce8e3a83522b9bc7176d005b893b726bf616d0ee2f7e5ab4269e
none 99G 14G 82G 14% /var/lib/docker/aufs/mnt/2e1fd00cb2ec9ca11323b3ac66f413b6873ca2e949ceb3ba5eb368de8de18af5
none 99G 14G 82G 14% /var/lib/docker/aufs/mnt/78c89fcc2b2a81c8544883209fac002a3525fed8504ebf43722b592179264dea
none 99G 14G 82G 14% /var/lib/docker/aufs/mnt/4e56c31cbc3dfde7df17c1075595d80214dc81e55093ee9d9b63ef88b09502ad
Here is result from free
total used free shared buffers cached
Mem: 7679824 5625036 2054788 0 207872 1148568
-/+ buffers/cache: 4268596 3411228
Swap: 0 0 0
What is the reason causing the PODs restart?
I have a lot of persistent disks attached to PODs running in GCE(Google Container Engine)
How can I check/monitor the used space of them without entering each of the POD?
Also is it possible to set up some alert when disk is almost full?
I want to add a new node in existing kubernetes cluster, but with a different machine type. For the new node I will add label for it so that only some application will run on it.
I tried the following command
gcloud compute instance-groups managed resize CONTAINER_GROUP --zone ZONE --size 5 --machine-type n1-standard-8
And it returns an error
ERROR: (gcloud.compute.instance-groups.managed.resize) unrecognized arguments: --machine-type n1-standard-8
How can I add a new node into existing kubernetes cluster with a different machine type?
I have a Replication Controller which creates more than one PODs. Within each POD, there is a container running an application.
Now within the container, it can call kubernetes API as below.
KUBE_TOKEN=$(</var/run/secrets/kubernetes.io/serviceaccount/token)
curl --silent --insecure --header "Authorization: Bearer $KUBE_TOKEN" \
https://$KUBERNETES_SERVICE_HOST:$KUBERNETES_PORT_443_TCP_PORT/api/v1/namespaces/$POD_NAMESPACE/pods/$POD_NAME
POD_NAME is the name of the POD;
POD_NAMESPACE is the namespace of the POD
The application instances in contains make up a cluster. And they elect one of the application instances as master, the others are slaves which replicates the data from master in real-time. When the master instance fails, another slave instance will be promoted to master.
When the application in the container is promoted to master, I want the master application instance to call the kubernetes API so that all the traffic goes to this POD/container.
{
"kind": "Service",
"apiVersion": "v1",
"metadata": {
"name": "my-cluster"
},
"spec": {
"ports": [{
"port": 33379,
"targetPort": 33379
}],
"type" : "None",
"selector": {
"name": "<POD_NAME>"
}
}
}
I specified the POD_NAME as the selector, but the service can not find any of the POD.
How can I configure the service so that all the traffic goes to that POD?
Within the Kubernetes cluster which enables SkyDNS (Google Container Engine), how can I find the IP:PORT of the SkyDNS?
I want to add that DNS server to NODE's resolve.conf
I have a cluster with 3 nodes.
First I create 3 Persistent Disks in Google Cloud.
Now I start a DaemonSet, which would create a POD on each of the node.
The POD mounts a persistent disk to store state. The POD is not stateless. In another word, Each node contains its own state stored on its own persistent disk. An individual persistent disk can not be attached by more than one POD -- otherwise it is read-only.
How to make the POD automatically pick one free Persistent Disk not used by others and mount?
I am using persistent disks to store database files. Google says its Standard persistent disks (HDD) is redundant and reliable Will the disk crash or go into read-only mode?
Do I need a secondary disk for backup?
The idea is explained here
In simple words, Nginx Proxy plays the role of load balance and transmits the HTTP/HTTPS requests to servers.
If the number of request times within a certain period from an individual IP exceeds a threshold, it will trigger a captcha for the upcoming requests.
And the end-user must input the correct captcha code before he can continue to access the site.
Do you know any open source / free NGINX module for this usage? I searched on the Internet and here is one of them: https://github.com/snbuback/nginx seems it needs modification.
Any suggestion / experience is welcome, thank you
I have a web site generating files at runtime, this web site is deployed on 2 servers
Operation system, Windows Web Server 2008 R2.
I am looking for a manner to sync the changed and new files from one server to another, one-way sync.
DFS-R, anybody succeed in installing DFS-R on Windows Web Server 2008 R2? here is an article http://blogs.technet.com/b/filecab/archive/2008/05/14/installing-dfs-replication-on-windows-server-2008.aspx, which does not work for 2008 R2 any more.
robocopy, I tried this one, but it does not perform very well and sometimes there is everal minutes delay before the changed file is synced.
Please do you have any suggestion to me?
Thanks