Home / server / Questions

Questions[openshift-4](server)

Martin Hope
MariusM
Asked: 2021-11-16 08:49:41 +0800 CST
  • 0

I've got an Openshift cluster which runs Openshift Container Storage 4.4.1. I'd like to update OCP to a higher version, but as far as I understand, I need to upgrade operators first. OCS shows in console that upgrade is available, but it won't upgrade although approval is set to "Automatic". My suspicions are related to lib-bucket-provisioner operator. As I can see, there is one operator running, but two CSV are present: lib-bucket-provisioner.v1.0.0 and lib-bucket-provisioner.v2.0.0. First one is in "Pending" state, second one is "Succeeded".

I have checked the logs of catalog-operator pod (runing in openshift-operator-lifecycle-manager namespace). Some stuff, which seems to be related to the present issue, is such messages:

E1115 16:25:16.270062       1 queueinformer_operator.go:290] sync "openshift-storage" failed: error calculating generation changes due to new bundle: objectbucket.io/v1alpha1/ObjectBucketClaim (objectbucketclaims) already provided by lib-bucket-provisioner.v1.0.0

I wonder how to solve this situation, so that OCS operator could be upgraded. Would removal of lib-bucket-provisioner.v1.0.0 CSV help, since that one is in "Pending" state? As far as I can see, this one is included in OCS install plan, so that should be the way how it was installed. On the other hand, lib-bucket-provisioner.v2.0.0 seems to be installed manually. This one is also available in Operator Hub. So perhaps that is the one which in fact causes the problems, although it's status is OK. Since I wasn't the one who originally deployed this storage, I'm not sure what were the reasons behind installing v2 operator.

I was also wondering if removal of these CSVs may have any unwanted effects on existing data (removal of PVs or PVCs, etc.). Please advise.

storage openshift-4
Martin Hope
davidburtton
Asked: 2021-11-03 06:55:46 +0800 CST
  • 0

I'm trying to create a new container in RH OpenShift, but I keep getting the following error:

error: build error: Failed to push image: error copying layers and metadata from "containers-storage:[[email protected]...: i/o timeout

I've recently configured my Internal Image Registry so I'm wondering if that could be the problem?

I'm completely lost for what is wrong here!!

Thanks in advance

devops openshift openshift-4
Martin Hope
Eduardo Lucio
Asked: 2021-06-18 07:35:12 +0800 CST
  • 1

Simple question, but so far very difficult to answer... =-[

I am trying to deploy OpenShift (OKD) 4.5 or 4.7 as directed here Guide: Installing an OKD 4.5 Cluster. Look at the "Starting the control plane nodes" section.

I'm trying to create the cluster using an UPI (User Provisioned Infrastructure)/Bare Metal (KVM).

PROBLEM:

  • Version 4.5

The master node cannot finish installation after reboot. It keeps showing the following error...

[ 1304.254380] ignition[485]: GET https://api-int.mbr.okd.local:22623/config/master: attempt #92
[ 1314.264629] ignition[485]: GET error: Get "https://api-int.mbr.okd.local:22623/config/master": net/http: timeout awaiting response headers

For version 4.5 we use "Fedora CoreOS 32.20200715.3.0".

  • Version 4.7

The master node cannot finish installation after reboot. It keeps showing the following error...

[  543.933709] ignition[505]: GET https://api-int.mbr.okd.local:22623/config/master: attempt #112
[  543.939340] ignition[505]: GET error: Get "https://api-int.mbr.okd.loca1:22623/config/master": EOF

For version 4.7 we use "Fedora CoreOS 34.20210518.3.0".

I've waited hours and hours and the master nodes are still in the same situation. What can I do to resolve this?

Thanks! =D

MORE INFORMATION:

See if this helps...

This output occurs in okd_master_3 (10.3.0.7)....

[ 1304.254380] ignition[485]: GET https://api-int.mbr.okd.local:22623/config/master: attempt #92
[ 1314.264629] ignition[485]: GET error: Get "https://api-int.mbr.okd.local:22623/config/master": net/http: timeout awaiting response headers

Connecting okd_master_2 (10.3.0.6) from okd_services (10.3.0.14)...

NOTE: The okd_master_2 (10.3.0.6) was able to boot (reached login screen).

[root@okd_services ~]# ssh [email protected]
The authenticity of host '10.3.0.6 (10.3.0.6)' can't be established.
ECDSA key fingerprint is SHA256:1xdq65g0ljnZYR6uXHaXW6EsxO3u6X268s4Z9Kfq0ng.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '10.3.0.6' (ECDSA) to the list of known hosts.
Fedora CoreOS 32.20200629.3.0
Tracker: https://github.com/coreos/fedora-coreos-tracker
Discuss: https://discussion.fedoraproject.org/c/server/coreos/

Pinging the okd_bootstrap (10.3.0.4) from okd_master_2 (10.3.0.6)...

[core@localhost ~]$ ping 10.3.0.4
PING 10.3.0.4 (10.3.0.4) 56(84) bytes of data.
64 bytes from 10.3.0.4: icmp_seq=1 ttl=64 time=0.973 ms
64 bytes from 10.3.0.4: icmp_seq=2 ttl=64 time=0.801 ms
64 bytes from 10.3.0.4: icmp_seq=3 ttl=64 time=0.373 ms
64 bytes from 10.3.0.4: icmp_seq=4 ttl=64 time=0.647 ms
^C
--- 10.3.0.4 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3032ms
rtt min/avg/max/mdev = 0.373/0.698/0.973/0.220 ms

Calling the problematic URL from okd_master_2 (10.3.0.6)...

[core@localhost ~]$ curl -kv https://api-int.mbr.okd.local:22623/config/master
*   Trying 10.3.0.14:22623...
* Connected to api-int.mbr.okd.local (10.3.0.14) port 22623 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=api-int.mbr.okd.local
*  start date: Jun 16 23:52:22 2021 GMT
*  expire date: Jun 14 23:52:23 2031 GMT
*  issuer: OU=openshift; CN=root-ca
*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x561ed249aa40)
> GET /config/master HTTP/2
> Host: api-int.mbr.okd.local:22623
> user-agent: curl/7.69.1
> accept: */*
> 
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* Connection state changed (MAX_CONCURRENT_STREAMS == 250)!
< HTTP/2 500 
< content-length: 0
< date: Thu, 17 Jun 2021 14:55:43 GMT
< 
* Connection #0 to host api-int.mbr.okd.local left intact

INFRASTRUCTURE:

Virtual machines...

NAME           ROLE                   OS              IP          MAC
okd_boostrap   bootstrap              Fedora CoreOS   10.3.0.4    52:54:00:07:80:62
okd_master_1   master                 Fedora CoreOS   10.3.0.5    52:54:00:7d:97:70
okd_master_2   master                 Fedora CoreOS   10.3.0.6    52:54:00:6e:52:85
okd_master_3   master                 Fedora CoreOS   10.3.0.7    52:54:00:a3:65:d9
okd_worker_1   worker                 Fedora CoreOS   10.3.0.8    52:54:00:e3:7c:fb
okd_worker_2   worker                 Fedora CoreOS   10.3.0.9    52:54:00:20:ec:4f
okd_services   DNS/LB/web/NFS         CentOS 8        10.3.0.14   52:54:00:3a:fd:a2
                                                         10.2.0.18   52:54:00:92:ce:78
okd_pfsense    firewall/router/DHCP   FreeBSD         10.3.0.2 52:54:00:d8:27:82
                                                         10.2.0.19   52:54:00:ac:82:7d

 . OKD_LAN: "10.3.0";
 . EXT_LAN: "10.2.0".

Some acronyms...
 _ DNS - Domain Name System;
 _ LB - Load Balancing;
 _ Web - Web Server;
 _ NFS - Network File Sharing.

Network layout...

           ...→.[N]WAN/EXT_LAN([R]dhcp).←... (10.2.0.0/24)
           ↓                               ↓
          [I]WAN/EXT_LAN                  [I]WAN/EXT_LAN
  [V]OKD_PFSENSE([R]dhcp)                 [V]OKD_SERVICES
          [I]OKD_LAN                      [I]OKD_LAN
           ↑                               ↑
           .........→.[N]OKD_LAN.←.......... (10.3.0.0/24)
                       ↑
      ...................................
      ↓                ↓                ↓
     [V]OKD_BOOSTRAP  [V]OKD_MASTER_1  [V]OKD_WORKER_1
                      [V]OKD_MASTER_2  [V]OKD_WORKER_2
                      [V]OKD_MASTER_3

 _ [N] - Network;
 _ [R] - Network Resource;
 _ [I] - Network Interface;
 _ [V] - Virtual Machine.

CONFIGURATION FILES:

BIND 9 (DNS):

. db.10.3.0

$TTL    604800
@   IN  SOA okd-services.okd.local. admin.okd.local. (
        6       ; Serial
        604800  ; Refresh
        86400   ; Retry
        2419200 ; Expire
        604800  ; Negative Cache TTL
)

; Name servers - "NS" records.
    IN  NS  okd-services.okd.local.

; Name servers - "PTR" records.
14 IN  PTR okd-services.okd.local.

; OpenShift container platform cluster - "PTR" records.
4 IN  PTR okd-boostrap.mbr.okd.local.
5 IN  PTR okd-master-1.mbr.okd.local.
6 IN  PTR okd-master-2.mbr.okd.local.
7 IN  PTR okd-master-3.mbr.okd.local.
8 IN  PTR okd-worker-1.mbr.okd.local.
9 IN  PTR okd-worker-2.mbr.okd.local.
14 IN  PTR api.mbr.okd.local.
14 IN  PTR api-int.mbr.okd.local.

. db.okd.local

$TTL    604800
@   IN  SOA okd-services.okd.local. admin.okd.local. (
        1       ; Serial
        604800  ; Refresh
        86400   ; Retry
        2419200 ; Expire
        604800  ; Negative Cache TTL
)

; Name servers - "NS" records.
    IN  NS  okd-services

; Name servers - "A" records.
okd-services.okd.local. IN A 10.3.0.14

; OpenShift container platform cluster - "A" records.
okd-boostrap.mbr.okd.local. IN  A   10.3.0.4
okd-master-1.mbr.okd.local. IN  A   10.3.0.5
okd-master-2.mbr.okd.local. IN  A   10.3.0.6
okd-master-3.mbr.okd.local. IN  A   10.3.0.7
okd-worker-1.mbr.okd.local. IN  A   10.3.0.8
okd-worker-2.mbr.okd.local. IN  A   10.3.0.9

; Openshift internal cluster IPs - "A" records.
api.mbr.okd.local.              IN  A   10.3.0.14
api-int.mbr.okd.local.          IN  A   10.3.0.14
*.apps.mbr.okd.local.           IN  A   10.3.0.14
etcd-0.mbr.okd.local.           IN  A   10.3.0.5
etcd-1.mbr.okd.local.           IN  A   10.3.0.6
etcd-2.mbr.okd.local.           IN  A   10.3.0.7
cons-okd.apps.mbr.okd.local.    IN  A   10.3.0.14
oauth-okd.apps.mbr.okd.local.   IN  A   10.3.0.14

; OpenShift internal cluster IPs - "SRV" records.
_etcd-server-ssl._tcp.mbr.okd.local.    86400   IN  SRV 0   10  2380    etcd-0.mbr
_etcd-server-ssl._tcp.mbr.okd.local.    86400   IN  SRV 0   10  2380    etcd-1.mbr
_etcd-server-ssl._tcp.mbr.okd.local.    86400   IN  SRV 0   10  2380    etcd-2.mbr

. named.conf.local

zone "okd.local" {
    type master;
    file "/etc/named/zones/db.okd.local"; // Zone file path.
};

zone "0.3.10.in-addr.arpa" {
    type master;
    file "/etc/named/zones/db.10.3.0"; // 10.3.0.0/24 subnet.
};

. named.conf

//
// named.conf
//
// Provided by Red Hat bind package to configure the ISC BIND named(8) DNS server
// as a caching only nameserver (as a localhost DNS resolver only).
//
// See /usr/share/doc/bind*/sample/ for example named configuration files.
//
// See the BIND Administrator's Reference Manual (ARM) for details about the configuration
// located in /usr/share/doc/bind-{version}/Bv9ARM.html .

options {
    listen-on port 53 { 127.0.0.1; 10.3.0.14; };
    directory "/var/named";
    dump-file "/var/named/data/cache_dump.db";
    statistics-file "/var/named/data/named_stats.txt";
    memstatistics-file "/var/named/data/named_mem_stats.txt";
    recursing-file "/var/named/data/named.recursing";
    secroots-file "/var/named/data/named.secroots";
    allow-query { localhost; 10.3.0.0/24; };

    // - If you are building an AUTHORITATIVE DNS server, do NOT enable recursion.
    // - If you are building a RECURSIVE (caching) DNS server, you need to enable
    // recursion.
    // - If your recursive DNS server has a public IP address, you MUST enable access
    // control to limit queries to your legitimate users. Failing to do so will cause
    // your server to become part of large scale DNS amplification attacks. Implementing
    // BCP38 within your network would greatly reduce such attack surface.
    recursion yes;

    forwarders {
        8.8.8.8;
        8.8.4.4;
    };

    dnssec-enable yes;
    dnssec-validation yes;

    // Path to ISC DLV key.
    bindkeys-file "/etc/named.root.key";

    managed-keys-directory "/var/named/dynamic";

    pid-file "/run/named/named.pid";
    session-keyfile "/run/named/session.key";
};

logging {
    channel default_debug {
        file "data/named.run";
        severity dynamic;
    };
};

zone "." IN {
    type hint;
    file "named.ca";
};

include "/etc/named.rfc1912.zones";
include "/etc/named.root.key";
include "/etc/named/named.conf.local";

HAProxy (load balancer):

. haproxy.cfg

#---------------------------------------------------------------------
# Global settings.
#---------------------------------------------------------------------
global
    maxconn 20000
    log /dev/log local0 info
    chroot /var/lib/haproxy
    pidfile /var/run/haproxy.pid
    user haproxy
    group haproxy
    daemon

    # Turn on stats unix socket.
    stats socket /var/lib/haproxy/stats

#---------------------------------------------------------------------
# Common defaults that all the "listen" and "backend" sections will use if not designated
# in their block.
#---------------------------------------------------------------------
defaults
    mode http
    log global
    option httplog
    option dontlognull
    option http-server-close
    option redispatch
    retries 3
    timeout http-request 10s
    timeout queue 1m
    timeout connect 10s
    timeout client 300s
    timeout server 300s
    timeout http-keep-alive 10s
    timeout check 10s
    maxconn 20000

listen stats
    bind :9000
    mode http
    option forwardfor except 127.0.0.0/8
    stats enable
    stats uri /

frontend okd_k8s_api_fe
    bind :6443
    default_backend okd_k8s_api_be
    mode tcp
    option tcplog

backend okd_k8s_api_be
    balance source
    mode tcp
    server okd-boostrap 10.3.0.4:6443 check
    server okd-master-1 10.3.0.5:6443 check
    server okd-master-2 10.3.0.6:6443 check
    server okd-master-3 10.3.0.7:6443 check

frontend okd_machine_config_server_fe
    bind :22623
    default_backend okd_machine_config_server_be
    mode tcp
    option tcplog

backend okd_machine_config_server_be
    balance source
    mode tcp
    server okd-boostrap 10.3.0.4:22623 check
    server okd-master-1 10.3.0.5:22623 check
    server okd-master-2 10.3.0.6:22623 check
    server okd-master-3 10.3.0.7:22623 check

frontend okd_http_ingress_traffic_fe
    bind :80
    default_backend okd_http_ingress_traffic_be
    mode tcp
    option tcplog

backend okd_http_ingress_traffic_be
    balance source
    mode tcp
    server okd-worker-1 10.3.0.8:80 check
    server okd-worker-2 10.3.0.9:80 check

frontend okd_https_ingress_traffic_fe
    bind *:443
    default_backend okd_https_ingress_traffic_be
    mode tcp
    option tcplog

backend okd_https_ingress_traffic_be
    balance source
    mode tcp
    server okd-worker-1 10.3.0.8:443 check
    server okd-worker-2 10.3.0.9:443 check

OpenShift (OKD) "*.yaml" files:

. htpasswd_provider.yaml

apiVersion: config.openshift.io/v1
kind: OAuth
metadata:
  name: cluster
spec:
  identityProviders:
  - name: htpasswd_provider
    mappingMethod: claim
    type: HTPasswd
    htpasswd:
      fileData:
        name: htpass-secret

. install-config.yaml

apiVersion: v1
baseDomain: okd.local
metadata:
  name: mbr

compute:
- hyperthreading: Enabled
  name: worker
  replicas: 0

controlPlane:
  hyperthreading: Enabled
  name: master
  replicas: 3

networking:
  clusterNetwork:
  - cidr: 10.128.0.0/14
    hostPrefix: 23
  networkType: OpenShiftSDN
  serviceNetwork:
  - 172.30.0.0/16

platform:
  none: {}

fips: false

pullSecret: '{"auths":{"fake":{"auth": "bar"}}}'
sshKey: 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAA<SKIPPED>QbAKPwwhdCkTpd8= root@okd_services.my_domain.com.br'

. registry_pv.yaml

apiVersion: v1
kind: PersistentVolume
metadata:
  name: registry-pv
spec:
  capacity:
    storage: 45Gi
  accessModes:
    - ReadWriteMany
  persistentVolumeReclaimPolicy: Retain
  nfs:
    path: /var/nfsshare/registry
    server: 10.3.0.14

UPDATE:

. netstat -natup output...

[root@okd_services ~]# netstat -natup
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      906/sshd            
tcp        0      0 127.0.0.1:953           0.0.0.0:*               LISTEN      929/named           
tcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTEN      4572/haproxy        
tcp        0      0 0.0.0.0:22623           0.0.0.0:*               LISTEN      4572/haproxy        
tcp        0      0 0.0.0.0:9000            0.0.0.0:*               LISTEN      4572/haproxy        
tcp        0      0 0.0.0.0:6443            0.0.0.0:*               LISTEN      4572/haproxy        
tcp        0      0 0.0.0.0:111             0.0.0.0:*               LISTEN      1/systemd           
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      4572/haproxy        
tcp        0      0 192.168.122.1:53        0.0.0.0:*               LISTEN      1742/dnsmasq        
tcp        0      0 10.3.0.14:53            0.0.0.0:*               LISTEN      929/named           
tcp        0      0 127.0.0.1:53            0.0.0.0:*               LISTEN      929/named           
tcp        0      0 10.2.0.18:22            10.2.0.3:44536          ESTABLISHED 1854/sshd: root [pr 
tcp        0      0 10.3.0.14:52252         10.3.0.4:6443           ESTABLISHED 4572/haproxy        
tcp        0      0 10.3.0.14:52134         10.3.0.4:6443           ESTABLISHED 4572/haproxy        
tcp        0      1 10.3.0.14:42222         10.3.0.8:443            SYN_SENT    4572/haproxy        
tcp        0      0 10.3.0.14:6443          10.3.0.6:51962          ESTABLISHED 4572/haproxy        
tcp        0      0 10.3.0.14:52130         10.3.0.4:6443           ESTABLISHED 4572/haproxy        
tcp        0      0 10.3.0.14:6443          10.3.0.6:51946          ESTABLISHED 4572/haproxy        
tcp        0      1 10.3.0.14:40530         10.3.0.9:443            SYN_SENT    4572/haproxy        
tcp        0    196 10.2.0.18:22            10.2.0.3:44538          ESTABLISHED 5000/sshd: root [pr 
tcp        0      0 10.2.0.18:45472         10.2.0.5:389            ESTABLISHED 878/sssd_be         
tcp        0      0 10.3.0.14:51970         10.3.0.4:6443           ESTABLISHED 4572/haproxy        
tcp        0      0 10.3.0.14:54056         10.3.0.4:6443           ESTABLISHED 4572/haproxy        
tcp        0      0 10.2.0.18:33328         147.75.69.225:80        TIME_WAIT   -                   
tcp        0      0 10.3.0.14:6443          10.3.0.5:39976          ESTABLISHED 4572/haproxy        
tcp        0      0 10.3.0.14:6443          10.3.0.5:52462          ESTABLISHED 4572/haproxy        
tcp        0      1 10.3.0.14:41396         10.3.0.7:22623          SYN_SENT    4572/haproxy        
tcp        0      1 10.3.0.14:41964         10.3.0.9:80             SYN_SENT    4572/haproxy        
tcp        0      1 10.3.0.14:60674         10.3.0.7:6443           SYN_SENT    4572/haproxy        
tcp        0      0 10.3.0.14:6443          10.3.0.5:40024          ESTABLISHED 4572/haproxy        
tcp        0      0 10.2.0.18:43394         109.205.222.4:80        TIME_WAIT   -                   
tcp6       0      0 :::22                   :::*                    LISTEN      906/sshd            
tcp6       0      0 ::1:953                 :::*                    LISTEN      929/named           
tcp6       0      0 :::111                  :::*                    LISTEN      1/systemd           
tcp6       0      0 :::8080                 :::*                    LISTEN      1131/httpd          
tcp6       0      0 :::53                   :::*                    LISTEN      929/named           
udp        0      0 192.168.122.1:53        0.0.0.0:*                           1742/dnsmasq        
udp        0      0 10.3.0.14:53            0.0.0.0:*                           929/named           
udp        0      0 127.0.0.1:53            0.0.0.0:*                           929/named           
udp        0      0 0.0.0.0:67              0.0.0.0:*                           1742/dnsmasq        
udp        0      0 10.3.0.14:68            10.3.0.2:67             ESTABLISHED 893/NetworkManager  
udp        0      0 10.2.0.18:68            10.2.0.2:67             ESTABLISHED 893/NetworkManager  
udp        0      0 0.0.0.0:111             0.0.0.0:*                           1/systemd           
udp        0      0 127.0.0.1:323           0.0.0.0:*                           857/chronyd         
udp6       0      0 :::53                   :::*                                929/named           
udp6       0      0 :::111                  :::*                                1/systemd           
udp6       0      0 ::1:323                 :::*                                857/chronyd

Thanks! =D

openshift openshift-origin openshift-4
Martin Hope
JSM
Asked: 2020-10-06 23:18:37 +0800 CST
  • 1

I recently deployed a version 4.5 OKD cluster and everything seems fine, but I can't get the Google identity provider to work for signing in. I followed the instructions here, but with no success. When I try to log in with my company google identity I get a very generic error message:

login error

I ended up with the following OAuth config:

spec:
  identityProviders:
  - google:
      clientID: <my-ID>.apps.googleusercontent.com
      clientSecret:
        name: google-secret
      hostedDomain: <company domain>
    mappingMethod: claim
    name: googleidp
    type: Google

I set the google project up as described too. Just created Oauth2.0 credentials, call back URL (which is correct since I get back to OKD after consent screen), no extra scopes in consent; just profile, email and openid, and OKD is not requesting any other scopes. The project is set to internal so only company users can log in.

After some investigations I managed to increase verbosity on the oauth-openshift pods and here is what is seen in the logs for a failed attempt:

       1 handler.go:156] Got auth data
I0929 15:30:10.036799       1 round_trippers.go:423] curl -k -v -XPOST  -H "Content-Type: application/x-www-form-urlencoded" -H "Accept: application/json" 'https://www.googleapis.com/oauth2/v3/token'
I0929 15:30:10.071829       1 round_trippers.go:443] POST https://www.googleapis.com/oauth2/v3/token 401 Unauthorized in 35 milliseconds
I0929 15:30:10.071871       1 round_trippers.go:449] Response Headers:
I0929 15:30:10.071879       1 round_trippers.go:452]     Server: scaffolding on HTTPServer2
I0929 15:30:10.071885       1 round_trippers.go:452]     Cache-Control: private
I0929 15:30:10.071891       1 round_trippers.go:452]     X-Content-Type-Options: nosniff
I0929 15:30:10.071897       1 round_trippers.go:452]     Vary: Origin
I0929 15:30:10.071902       1 round_trippers.go:452]     Vary: X-Origin
I0929 15:30:10.071909       1 round_trippers.go:452]     Vary: Referer
I0929 15:30:10.071915       1 round_trippers.go:452]     Date: Tue, 29 Sep 2020 15:30:10 GMT
I0929 15:30:10.071920       1 round_trippers.go:452]     Alt-Svc: h3-Q050=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-27=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-T050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
I0929 15:30:10.071926       1 round_trippers.go:452]     Content-Type: application/json; charset=utf-8
I0929 15:30:10.071934       1 round_trippers.go:452]     X-Xss-Protection: 0
I0929 15:30:10.071939       1 round_trippers.go:452]     X-Frame-Options: SAMEORIGIN
I0929 15:30:10.072004       1 handler.go:176] Error getting access token: Unauthorized
E0929 15:30:10.072031       1 errorpage.go:26] AuthenticationError: Unauthorized
I0929 15:30:10.072428       1 httplog.go:90] verb="GET" URI="/oauth2callback/googleidp?state=<token>&code=<token>&scope=email%20profile%20openid%20https://www.googleapis.com/auth/userinfo.email%20https://www.googleapis.com/auth/userinfo.profile&authuser=0&hd=<companydomain>&prompt=consent" latency=35.835162ms resp=200 UserAgent="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.83 Safari/537.36" srcIP="10.129.2.28:49228":

I can't find a reason for the Unauthorized error so any help is much appreciated.

oauth openshift openshift-4
Martin Hope
Baptiste Mille-Mathias
Asked: 2020-07-18 05:36:15 +0800 CST
  • 5

In openshift 4.x, you have an API for project which seems to be totally similar to namespace in the sense that when you create a project there a namespace created and the other way around. I know namespace is a standard object in kubernetes and project is specific to Openshift. So what does project brings ?

# list projects
oc get projects                                                      
NAME                                                    DISPLAY NAME   STATUS
default                                                                Active
kube-node-lease                                                        Active
kube-public                                                            Active
kube-system                                                            Active
local-storage                                                          Active
openshift                                                              Active
openshift-apiserver                                                    Active

# list namespaces
$ oc get ns
NAME                                                    STATUS   AGE
default                                                 Active   17d
kube-node-lease                                         Active   17d
kube-public                                             Active   17d
kube-system                                             Active   17d
local-storage                                           Active   16d
openshift                                               Active   17d
openshift-apiserver                                     Active   17d
openshift-apiserver-operator                            Active   17d

The list is the same except different columns

oc get project foo
Error from server (NotFound): namespaces "foo" not found
oc get ns foo
Error from server (NotFound): namespaces "foo" not found

15:30 $ oc new-project foo                          
Now using project "foo" on server "https://api.goo.tadadidou.bo:6443".
...       
$ oc get project foo
NAME   DISPLAY NAME   STATUS
foo                   Active
$ oc get ns foo
NAME   STATUS   AGE
foo    Active   70s

Even the yaml output is similar, except the value of the field Kind: which is either Project or Namespace.

kubernetes openshift openshift-4