Sagar's questions

I have the following ClusterRoles defined:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: deployer-system
rules:
- apiGroups:
  - '*'
  resources:
  - '*'
  verbs:
  - list
- nonResourceURLs:
  - '*'
  verbs:
  - list

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: deployer-nonsystem
rules:
- apiGroups:
  - '*'
  resources:
  - secrets
  - PersistentVolumes
  - Role
  - RoleBinding
  verbs:
  - list
- apiGroups:
  - '*'
  resources:
  - deployments
  - pods
  verbs:
  - get
  - list
  - watch
  - create
  - update
  - patch
- nonResourceURLs:
  - '*'
  verbs:
  - list

I use the following RoleBindings to assign a user the above ClusterRole:

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: theuser-nonsystem-role-binding
  namespace: nonsystem-namespace
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: deployer-nonsystem
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: User
  name: theuser

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: theuser-kube-system-role-binding
  namespace: system-namespace
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: deployer-system
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: User
  name: theuser

However, they are able to run a kubectl -n [NAMESPACE] get secrets -o yaml and see all the secrets. I would expect this call to be forbidden based on the ClusterRole specifications above.

Am I missing or misunderstanding something here? Why can the user "get" secrets?

UPDATE: Please note: my issues is NOT that they can list secrets. My issue here is that the user can "get" secrets (different verbs!)

We have a Jenkins server that uses Kerberos-SSO, with a fallback to Basic if SSO is not configured on the browser or using curl.

When I use curl with the --negotiate argument, however, it doesn't send basic credentials information when asked for it even though the server is clearly sending www-authenticate negotiation headers.

Command used:

curl --verbose --negotiate --basic --user "username":"password" myserver.mycompany.com

* About to connect() to myserver.mycompany.com port 443 (#0)
*   Trying 10.5.24.212...
* Connected to myserver.mycompany.com (10.5.24.212) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
*   subject: CN=*.mycompany.com,O=MyCompany International,L=New York,ST=New York,C=US
*   start date: Aug 31 17:14:45 2020 GMT
*   expire date: Aug 31 17:14:45 2022 GMT
*   common name: *.mycompany.com
*   issuer: CN=MyCompany Intermediate CA 3,DC=ad,DC=mycompany,DC=com
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: myserver.mycompany.com
> Accept: */*
> 
< HTTP/1.1 401 Unauthorized
< date: Thu, 01 Oct 2020 16:04:44 GMT
< server: Apache
< set-cookie: gssapi_session=;Max-Age=0;path=/;httponly;
< www-authenticate: Negotiate
< www-authenticate: Basic realm="MyCompany SSO"
< cache-control: no-cache
< set-cookie: gssapi_session=;Max-Age=0;path=/;httponly;
< content-length: 381
< content-type: text/html; charset=iso-8859-1
< 
* Ignoring the response-body
* Connection #0 to host myserver.mycompany.com left intact
* Issue another request to this URL: 'https://myserver.mycompany.com/'
* Found bundle for host myserver.mycompany.com: 0xe8ef50
* Re-using existing connection! (#0) with host myserver.mycompany.com
* Connected to myserver.mycompany.com (10.5.24.212) port 443 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: myserver.mycompany.com
> Accept: */*
> 
< HTTP/1.1 401 Unauthorized
< date: Thu, 01 Oct 2020 16:04:44 GMT
< server: Apache
< set-cookie: gssapi_session=;Max-Age=0;path=/;httponly;
* gss_init_sec_context() failed: : No Kerberos credentials available (default cache: KEYRING:persistent:0)
< www-authenticate: Negotiate
< www-authenticate: Basic realm="MyCompany SSO"
< cache-control: no-cache
< set-cookie: gssapi_session=;Max-Age=0;path=/;httponly;
< content-length: 381
< content-type: text/html; charset=iso-8859-1
< 
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>401 Unauthorized</title>
</head><body>
<h1>Unauthorized</h1>
<p>This server could not verify that you
are authorized to access the document
requested.  Either you supplied the wrong
credentials (e.g., bad password), or your
browser doesn't understand how to supply
the credentials required.</p>
</body></html>

Curl information:

$ curl --version
curl 7.29.0 (x86_64-redhat-linux-gnu) libcurl/7.29.0 NSS/3.44 zlib/1.2.7 libidn/1.28 libssh2/1.8.0
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smtp smtps telnet tftp 
Features: AsynchDNS GSS-Negotiate IDN IPv6 Largefile NTLM NTLM_WB SSL libz unix-sockets 

Ultimate goal: Create an automated installation disk for CentOS, using a kickstart config file stored on an internal website.

What I've done: Created a script that

1. downloads an ISO
2. unpacks it
3. updates the isolinux.cfg file with my own menu option
4. recreates the ISO using genisoimage

All of this works, and the ISO is recreated without any errors. However, when I try to use the ISO, to start an installation, the sistem always dies at:

"new value non-existent xfs filesystem is not valid as a default fs type"
Pane is dead

Unlike this question here, my error happens pre-installation, and I'm already using lang = en_US.UTF-8. I even tried en_US, but it did not make a difference.

Lines I added to the isolinux.cfg file:

label install
  menu label ^Kickstart Installation
  menu default
  kernel vmlinuz
  append initrd=initrd.img text ramdisk=100000 lang=en_US.UTF-8 keymap=us ipv6.disable=1 ip=dhcp install inst.ks=cdrom:/kickstart.cfg

I searched online for the error, and a lot of comments seem to be related to mismatching versions for init and the kernel, or corrupted files. I am not replacing any files, and the only change I make is to the configuration file isolinux.cfg. There is no mixing of files from different versions/DVDs/images.

Has anyone had/seen the above, and have any ideas on what I should try next? I've tried playing around with the options I am putting into the file, but it doesn't seem to make a difference.

Other Info

Original image: CentOS 7.2 - DVD ISO

State of the system:

Ctrl-Alt-F1:

Ctrl-Alt-F2:

EDIT

Script that I am writing, as requested:

#!/bin/bash
#############################################################
function prepare {
    rm -rf ${ISO_EXTRACTION_DIR}
    if [ -d ${ISO_MOUNT_DIR} ]
    then
        umount ${ISO_MOUNT_DIR} 2>&1 || true
        rm -rf ${ISO_MOUNT_DIR}
    fi
    mkdir -p ${ISO_MOUNT_DIR} ${ISO_EXTRACTION_DIR}
}

function extract_iso {
    echo "Extracting ISO"

    cd $(dirname ${ISO_MOUNT_DIR})
    cp -pRf ${ISO_MOUNT_DIR}/* ${ISO_EXTRACTION_DIR}

    echo "Extracted $(ls ${ISO_EXTRACTION_DIR} | wc -l) files and directories from ISO"
}

function update_config {
    echo "Updating configuration"
    cd ${ISO_EXTRACTION_DIR}

    if [ -d "isolinux" ]
    then
        isolinux_cfg_file="${ISO_EXTRACTION_DIR}/isolinux/isolinux.cfg"
    else
        isolinux_cfg_file="${ISO_EXTRACTION_DIR}/isolinux.cfg"
    fi

    sed -i 's/timeout 600/timeout 50/' ${isolinux_cfg_file}  # Shorten timeout
    sed -i '/menu default/d' ${isolinux_cfg_file}            # Delete existing menu default
    echo "label linux" >> ${isolinux_cfg_file}
    echo "  menu default" >> ${isolinux_cfg_file}
    echo "  kernel vmlinuz" >> ${isolinux_cfg_file}
    echo "  append initrd=initrd.img text ramdisk=100000 lang=en_US.UTF-8 keymap=us ipv6.disable=1 ip=dhcp install inst.ks=cdrom:/kickstart.cfg" >> ${isolinux_cfg_file}
}

function repackage_iso {
    echo "Repackaging ISO"

    cd ${ISO_EXTRACTION_DIR}
    volume_id="${LINUX_DISTRO}-${LINUX_VERSION}-bootable"

    if [ -d "isolinux" ]
    then
        boot_cat_file="isolinux/boot.cat"
        isolinux_bin_file="isolinux/isolinux.bin"
    else
        boot_cat_file="boot.cat"
        isolinux_bin_file="isolinux.bin"
    fi

    iso_file_name="$(echo $(basename ${IMAGE_FILENAME}) | cut -d '.' -f 1)-bootable.iso"

    genisoimage \
        -U -r -v -T -J -joliet-long \
        -V 'CentOS 7 x86_64' \
        -volset "${LINUX_DISTRO}-${LINUX_VERSION}" \
        -A "${LINUX_DISTRO}-${LINUX_VERSION}" \
        -b ${isolinux_bin_file} \
        -c ${boot_cat_file} \
        -no-emul-boot \
        -boot-load-size 4 \
        -boot-info-table \
        -eltorito-alt-boot \
        -e images/efiboot.img \
        -no-emul-boot \
        -o "${IMAGE_DIR}/${iso_file_name}" .
}

prepare
check_input
mount_iso
extract_iso
unmount_iso
update_config
repackage_iso

Scenario: Let's assume I have a script that spawns a few processes. There may be times when the script has to be aborted. Most times this is fine. However, sometimes one or two of the spawned processes remain running, and are orphaned, because the script has been killed.

Is there any way to find the processes based on their environment (specific environment strings and values set when the script ran/spawned them), and kill them?

OS: Windows 7

We have a client server setup, where the client sets up an SSH tunnel and uses port forwarding to send data to the server:

ssh -N -L 5000:localhost:5500 user@serveraddress

The normal number of SSH connections at the server is ~150, and while everything is normal, the server software processes incoming connections pretty fast (a few seconds at most).

However, recently we have noticed that the number of SSH connections rises to 900+. At this point, the server software sees connects to it and accepts these connections, but no data is coming in.

Has anyone seen such symptoms with SSH before? Any ideas on what the issue could be?

Server OS: Red Hat Linux 5.5
Firewall: Disabled
Key Exchange: Tested

EDIT: Adding parts of log data from /var/log/secure on the server side

There seems to be a lot of the following in the log file.

Apr 10 00:07:33 myserver sshd[15038]: fatal: Write failed: Connection timed out
Apr 10 00:12:01 myserver sshd[5259]: fatal: Read from socket failed: Connection reset by peer
Apr 10 00:44:48 myserver sshd[17026]: fatal: Write failed: No route to host
Apr 10 02:09:16 myserver sshd[10398]: fatal: Read from socket failed: Connection reset by peer
Apr 10 02:22:47 myserver sshd[24581]: fatal: Read from socket failed: Connection reset by peer
Apr 10 03:05:57 myserver sshd[12003]: fatal: Read from socket failed: Connection reset by peer
Apr 10 03:23:19 myserver sshd[22421]: fatal: Write failed: Connection timed out
Apr 10 08:13:43 myserver sshd[31993]: fatal: Read from socket failed: Connection reset by peer
Apr 10 08:36:39 myserver sshd[7759]: fatal: Read from socket failed: Connection reset by peer
Apr 10 09:02:32 myserver sshd[12470]: fatal: Write failed: Broken pipe
Apr 10 12:08:05 myserver sshd[728]: fatal: Write failed: Connection reset by peer
Apr 10 12:35:53 myserver sshd[6184]: fatal: Read from socket failed: Connection reset by peer
Apr 10 12:43:14 myserver sshd[2663]: fatal: Write failed: Connection timed out

NOTE: After about 10-15 minutes of the 900+ connections, the system will recover by itself - the number of connections will drop to a normal range, and the server will start getting data again. It sounds like a DOS/DDOS, but this is on an internal network.

ADDENDUM: Checked the connections status based on @kranteg's question. We just had another outage, and these are the results based on a script I wrote for all incoming SSH connections:

===                                                        
Tue Apr 15 12:22:07 EDT 2014 -> Total SSH connections: 996 
===                                                        
0 SYN_SENT                                             
1 SYN_RECV                                             
0 FIN_WAIT1                                            
0 FIN_WAIT2                                            
15 TIME_WAIT                                            
0 CLOSED                                               
760 CLOSE_WAIT                                           
143 ESTABLISHED                                          
77 LAST_ACK                                             
0 LISTEN                                               
0 CLOSING                                              
0 UNKNOWN                                              
===                                                        
===
Tue Apr 15 12:22:17 EDT 2014 -> Total SSH connections: 977
===
0 SYN_SENT
2 SYN_RECV
1 FIN_WAIT1
0 FIN_WAIT2
15 TIME_WAIT
0 CLOSED
756 CLOSE_WAIT
127 ESTABLISHED
76 LAST_ACK
0 LISTEN
0 CLOSING
0 UNKNOWN
===
===
Tue Apr 15 12:22:26 EDT 2014 -> Total SSH connections: 979
===
0 SYN_SENT
2 SYN_RECV
1 FIN_WAIT1
0 FIN_WAIT2
12 TIME_WAIT
0 CLOSED
739 CLOSE_WAIT
148 ESTABLISHED
77 LAST_ACK
0 LISTEN
0 CLOSING
0 UNKNOWN
===

It looks like there is a jump in the number of connections in CLOSE_WAIT. During "normal" operation, the number in CLOSE_WAIT is either 0 or very close to it.

I have an application that connects to a database and retrieves information from it. The way it is written is: for every request received, create a new connection to the DB server and retrieve data.

When testing, if the network cable is removed from the host, and reconnected, the application goes back to retrieving data normally. However, if the "connected" option is unchecked and rechecked to simulate a disconnection, the application cannot connect to the database again, even though all handlers etc are created for each connection.

Question: Is disconnecting the network cable from the host different than unchecking the "Connected" option in VM configuration?

I know this has been asked before, but I looked at the other questions and they didn't help, so here goes...

I installed cygwin (only OpenSSH and whatever else it automatically chose for it). I ran the ssh-host-config and entered 'yes' for everything (basically a manual ssh-host-config -y).

It installed the service.

However, everytime the service starts, I get the following in the /var/log/sshd.log:

    158 [unknown (0xED4)] ? 2672 handle_exceptions: Exception: STATUS_ACCESS_VIOLATION
Exception: STATUS_ACCESS_VIOLATION at eip=7C919AF2
eax=00000000 ebx=00000000 ecx=000007B4 edx=77FEE0C0 esi=77FEE0B0 edi=00000000
ebp=0022EE94 esp=0022EE20 program=
cs=001B ds=0023 es=0023 fs=003B gs=0000 ss=0023
Stack trace:
Frame     Function  Args
0022EE94  7C919AF2  (00FEE0B0, 77FE25EE, 77FEE0B0, 7C90DCAE)
0022EECC  7C901046  (00261E90, 0022F094, 00000000, 00000000)
0022EEE4  77FE25A8  (0022EFF4, 0022F094, 0022EF50, 0022EF50)
0022EFF8  77FE1D0F  (00010002, 0022F014, 00261E90, 0022F094)
0022F01C  77FE1CB9  (00010002, 00261E90, 0022F094, 00000000)
0022F04C  77FE1E37  (00010002, 0022F0C8, 0022F094, 0022F1F0)
0022F060  77DE54D9  (0022F0C8, 0022F094, 00002000, 7C809B59)
0022F1F0  6108E202  (61600088, 00000003, 00020902, 00060000)
0022F220  6106E7C7  (0A000000, 00000003, 0022F24C, 0000B808)
0022F250  6106E872  (610CFE20, 610048AA, 7C980620, 7C8021B9)
0022F2A0  61004FF5  (610CFEE0, FFFFFFFE, 000007E4, 610CFE04)
0022F2F0  610052ED  (10003390, 10003390, 003620B0, 10000000)
0022F310  61007F9D  (10000000, 10003390, 7C91BF81, 10000240)
0022F330  100018A8  (10000000, 00000001, 00000000, 100017F0)
0022F350  7C90118A  (100017F0, 10000000, 00000001, 00000000)
0022F458  7C91B5D2  (00000000, C0150008, 00000000, 00000000)
End of stack trace (more stack frames may be present)  
  93485 [unknown (0xED4)] ? 2672 handle_exceptions: Exception: STATUS_ACCESS_VIOLATION
  94005 [unknown (0xED4)] ? 2672 handle_exceptions: Error while dumping state (probably corrupted stack)
  94706 [unknown (0xED4)] ? 2672 handle_exceptions: Exception: STATUS_ACCESS_VIOLATION
  95283 [unknown (0xED4)] ? 2672 handle_exceptions: Error while dumping state (probably corrupted stack)
  95985 [unknown (0xED4)] ? 2672 handle_exceptions: Exception: STATUS_ACCESS_VIOLATION
  96528 [unknown (0xED4)] ? 2672 handle_exceptions: Error while dumping state (probably corrupted stack)
  97223 [unknown (0xED4)] ? 2672 handle_exceptions: Exception: STATUS_ACCESS_VIOLATION
  97809 [unknown (0xED4)] ? 2672 handle_exceptions: Error while dumping state (probably corrupted stack)
  98540 [unknown (0xED4)] ? 2672 handle_exceptions: Exception: STATUS_ACCESS_VIOLATION
  99102 [unknown (0xED4)] ? 2672 handle_exceptions: Error while dumping state (probably corrupted stack)
  99823 [unknown (0xED4)] ? 2672 handle_exceptions: Exception: STATUS_ACCESS_VIOLATION
 100374 [unknown (0xED4)] ? 2672 handle_exceptions: Error while dumping state (probably corrupted stack)
 101105 [unknown (0xED4)] ? 2672 handle_exceptions: Exception: STATUS_ACCESS_VIOLATION
 101661 [unknown (0xED4)] ? 2672 handle_exceptions: Error while dumping state (probably corrupted stack)

And this goes on for as long as the service is running. I cannot connect to the machine.

However, if I run /usr/sbin/sshd manually, it starts fine, and I can connect to the machine.

The machine:

• Windows XP Pro
• Firewall - disabled
• Account: admin

Stuff I've already tried:

• Removing everything cygwin and reinstalling (registry keys included)
• Restarting the computer (multiple times)
• Installing cygwin in safe mode as suggested here
• Giving SYSTEM "Full Control" to the Cygwin folders, as suggested here

Any help is greatly appreciated!

FTP is completely disabled on our network due to security. No inbound or outbound ftp is allowed.

I just created a new install of RHEL on one of our systems, and the only two repositories (official) mentioned are both are ftp. Does anyone know if there is an official http based repository out there, and if so, the address of it?

We have a couple of production systems that were recently converted into virtual machines. There is an application of ours that frequently accesses a MySQL database, and for each query it creates a connection, queries, and disconnects that connection.

It is not the appropriate way to query (I know), but we have constraints that we can't seem to get around. Anyway, the issue is this: while the machine was a physical host, the program ran fine. Once converted to a virtual machine, we noticed intermittent connection issues to the database. There were, at one point, 24000+ socket connections in TIME_WAIT (on the physical host, the most I saw was 17000 - not good, but not causing problems).

I would like these connections to be reused, so that we don't see that connection problem, and so:

Questions:

Is it ok to set the value of tcp_tw_reuse to 1? What are the obvious dangers? Is there any reason I should never do it?

Also, is there any other way to get the system (RHEL/CentOS) to prevent so many connections from going into TIME_WAIT, or getting them to be reused?

Lastly, what would changing tcp_tw_recycle do, and would that help me?

In advance, thanks!

I have a virtual machine with Mandriva 2007.0 (yes, old - unfortunately we do not have a choice here). Anyway, the problem:

Before reboot: active network interface = eth0. No other interfaces present, and network manager confirms this. Static IP address set to 172.31.2.22. No issues, everything working properly, routing et al.

-------Reboot---------

After reboot: active network interface = eth1, with a DHCP address. Network manager shows eth0 as disconnected, and not connectable. When I try to set eth1 up with the static IP address (same one), it says "In Use". I then tried ifconfig eth0 172.31.2.29 just to free it up from the eth0 interface so I could use it with eth1 (since this is connected).

Result:

ifconfig eth0 172.31.2.29
SIOCSIFADDR: No such device
eth0: unknown interface: No such device

Nothing else changed. Any ideas what could be happening, or at least how I can get my IP address back?

The system I'm trying to set up has an IP address 172.31.2.1. Its default gateway is 172.31.254.1. This works fine, and I can reach anything the gateway allows me to go to (172.31.0.0/16 + 149.244.178.0/24).

There is another gateway at 149.244.178.1 which will allow me to reach anything in 149.244.0.0, and I'm trying to add that to the table so I can access 149.244.64.250.

This is what my routing table currently looks like:

Kernel IP Routing table
Destination    Gateway      Genmask      Flags    Metric    Ref    Use    Iface
172.31.0.0     *            255.255.0.0  U        0         0        0    eth0
169.254.0.0    *            255.255.0.0  U        0         0        0    eth0
default        172.31.254.1 0.0.0.0      UG       0         0        0    eth0

route add 149.244.0.0/16 gw 149.244.178.1 gives the error:

route: netmask 0000ffff doesn't make sense with host route.

route add -net 149.244.0.0/16 dev eth0 changes the route table to:

Kernel IP Routing table
Destination    Gateway      Genmask      Flags    Metric    Ref    Use    Iface
172.31.0.0     *            255.255.0.0  U        0         0        0    eth0
169.254.0.0    *            255.255.0.0  U        0         0        0    eth0
149.244.0.0    *            255.255.0.0  U        0         0        0    eth0
default        172.31.254.1 0.0.0.0      UG       0         0        0    eth0

However, trying to add the gateway still gives me the same error. Sorry if it sounds convoluted, and any help is appreciated!

The situation:

I have these virtual machines that are used for building software, and I'm trying to keep all the builds on the same "disk", if you will. The virtual machines:

2 x Windows
4 x Linux (3 Mandriva and 1 CentOS)

What is the best way to share a drive between virtual machines? I am currently using NFS (sharing a drive from Linux -> Windows using Services for Unix), but I'm not sure that is the most efficient.

So here's the issue. I have two different computers behind the same network. Both have the same gateway, IP address prefixes, etc.

Computer 1: Windows XP IP 192.168.0.15

This computer can ping and tracert out of the network to another computer I'm trying to reach

Computer 2: Linux (debian) IP 192.168.0.54

This computer can ping that out-of-network computer, but cannot traceroute to it.

This makes me believe that it is nothing to do with firewalls, but with the OS configuration.

The output of the route command:

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.0.0     *               255.255.255.0   U     0      0        0 eth0
169.254.0.0     *               255.255.0.0     U     0      0        0 eth0
default         192.168.0.1     0.0.0.0         UG    0      0        0 eth0

Any ideas as to what the issue might be?