Is there any command-line tool for the OVN Cluster Health check? Or each of its components? and a command or commands which gives stats about the cluster.
For example, without a functional test, I want to know if everything works properly in our OVN Cluster which has deployed with Kolla-Ansible.
All OVSDBs and OVN-NB-DBs and OVN-SB-DBs are healthy or not?, all Daemons are healthy or not?
I'm deploying openstack on 3 nodes (controller, compute and block storage).
Each of my nodes have 2 attachements eno1 on the management network and eno2 on a provider network (192.168.3.0/24). I'm trying to attach an instance to the provider network, to do so I defined the network in openstack (called provider and subnet is called also provider 192.168.3.0/24) it appears to be active. I tried with the admin credentials and a suitable keypair, but I get same error.
I have installed : keystone, nova, neutron, placement, glance, cinder and horizon with the Victoria's guides.
When I try to create an instance with
openstack server create --flavor m1.nano --image cirros --nic net-id=92dc8bf4-30be-4fdb-a1f4-bc91126dbc38 --security-group default --key-name demokey provider-instance
I get in /etc/neutron/server.log (on controller node) :
ERROR neutron.plugins.ml2.managers [req-3114ffac-2825-4db6-8586-0b3685fa6b1d b21b8901642c470b8f668965997c7922 0f23d567d2ce4599a1571d8fd5982f9a - default default] Failed to bind port 66f23e36-a360-444c-a3f7-2f7a904037dd on host compute1 for vnic_type normal using segments [{'id': 'e238d39a-ec1a-447a-a272-19cf4e4f76e1', 'network_type': 'flat', 'physical_network': 'provider', 'segmentation_id': None, 'network_id': 'd4c8fb07-7e56-4768-b725-93f3b97e1772'}]`
I feel like linuxbridge fails to bind ports on compute node.
ip a on controller returns
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 40:a8:f0:2f:4a:6c brd ff:ff:ff:ff:ff:ff
inet 10.0.0.2/24 brd 10.0.0.255 scope global eno1
valid_lft forever preferred_lft forever
inet6 fe80::42a8:f0ff:fe2f:4a6c/64 scope link
valid_lft forever preferred_lft forever
3: eno2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 40:a8:f0:2f:4a:6d brd ff:ff:ff:ff:ff:ff
inet 192.168.3.20/24 brd 192.168.3.255 scope global eno2
valid_lft forever preferred_lft forever
inet6 fe80::42a8:f0ff:fe2f:4a6d/64 scope link
valid_lft forever preferred_lft forever
4: eno3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 40:a8:f0:2f:4a:6e brd ff:ff:ff:ff:ff:ff
5: eno4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 40:a8:f0:2f:4a:6f brd ff:ff:ff:ff:ff:ff
8: tap80268500-17@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether ca:64:af:92:69:09 brd ff:ff:ff:ff:ff:ff link-netns qdhcp-45320836-f1a3-4e96-a3d8-59b95d633d1e
But on compute node :
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 94:18:82:78:cd:5c brd ff:ff:ff:ff:ff:ff
inet 10.0.0.3/24 brd 10.0.0.255 scope global eno1
valid_lft forever preferred_lft forever
inet6 fe80::9618:82ff:fe78:cd5c/64 scope link
valid_lft forever preferred_lft forever
3: eno2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether 94:18:82:78:cd:5d brd ff:ff:ff:ff:ff:ff
inet 192.168.3.30/24 brd 192.168.3.255 scope global eno2
valid_lft forever preferred_lft forever
inet6 fe80::9618:82ff:fe78:cd5d/64 scope link
valid_lft forever preferred_lft forever
4: eno3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 94:18:82:78:cd:5e brd ff:ff:ff:ff:ff:ff
5: eno4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether 94:18:82:78:cd:5f brd ff:ff:ff:ff:ff:ff
There is not tap for the VM.
Also on controller ip netns return a qdhcp namespace while nothing on compute.
Configs files are coherents and same as in the tutorials. I'm a bit loss. Do you have a clue where I need to dig ?
Thanks a lot !
I am currently configuring a VPN using the VPNaaS of neutron. We are setting an IPSec tunnel with an external partner. The communication is established, but he ask me what type of tunnel sharing we are using. Here are the options he gave me:
I do not know how Openstack will configure this options, and I do not know were to look at this.
Does anyone know where to look at?
Maybe someone has same problem
I have installed Openstack Victoria on two virtualmachines (1 controller node, 1 compute node) running ubuntu 20.04. Each node has two network interfaces, mgmt network and provider network. I have created private network and I have attached it to router. With this configuration I am able to access internet.
But, when I attach floating IP to my instance, it lost internet connectivity. I can access this instance from outside, but instance cannot access network gateway. I checked it with ip netns exec <qrouter-id> ping 8.8.8.8. It is working until I attach FIP.
I think that is a routing problem but I cannot find where? Do you guys have any ideas?
10.0.0.0/24 - mgmt network
10.0.2.0/24 - external (provider) network
Configuration of Linuxbridge:
root@compute1:/# grep -v "^#" /etc/neutron/plugins/ml2/linuxbridge_agent.ini | grep -v "^$"
[DEFAULT]
[agent]
extensions = qos
[linux_bridge]
physical_interface_mappings = provider:ens34
[network_log]
[securitygroup]
enable_security_group = true
firewall_driver = neutron.agent.linux.iptables_firewall.IptablesFirewallDriver
[vxlan]
enable_vxlan = true
local_ip = 10.0.0.131
l2_population = true
Provider Network:
root@controller1:/# openstack subnet show provider
| Field | Value |
| allocation_pools | 10.0.2.50-10.0.2.150 |
| cidr | 10.0.2.0/24|
| created_at | 2021-02-22T16:17:20Z |
| description | |
| dns_nameservers | 8.8.8.8|
| dns_publish_fixed_ip | None |
| enable_dhcp | True |
| gateway_ip | 10.0.2.1|
| host_routes | |
| id | 7d07101a-4696-4ff8-88bc-fa4ffde1622f |
| ip_version | 4 |
| ipv6_address_mode | None |
| ipv6_ra_mode | None |
| name | provider |
| network_id | d65d17fe-9829-44d5-bf07-1abb70f9d523 |
| prefix_length | None |
| project_id | 957f142f850240b5801023369eace69a |
| revision_number | 0 |
| segment_id | None |
| service_types | |
| subnetpool_id | None |
Router:
root@controller1:/# openstack router show router1
| Field | Value |
| admin_state_up | UP |
| availability_zone_hints | |
| availability_zones | nova |
| created_at | 2021-02-22T16:17:51Z |
| description | |
| distributed | False |
| external_gateway_info | {"network_id": "d65d17fe-9829-44d5-bf07-1abb70f9d523", "external_fixed_ips": [{"subnet_id": "7d07101a-4696-4ff8-88bc-fa4ffde1622f", "ip_address": "10.0.2.51"}], "enable_snat": true} |
| flavor_id | None |
| ha | False |
| id | fa11f06e-906c-4ae9-8176-20fb74e1cacd |
| interfaces_info | [{"port_id": "67d37c5f-1250-45e7-a003-78493921b4d6", "ip_address": "172.16.1.1", "subnet_id": "b0762924-6c7a-453f-a9b8-788e15e5f0c0"}] |
| name | router1 |
| project_id | 957f142f850240b5801023369eace69a |
| revision_number | 4 |
| routes | |
| status | ACTIVE |
Network namespaces:
root@controller1:/# ip netns
qrouter-fa11f06e-906c-4ae9-8176-20fb74e1cacd (id: 3)
qdhcp-d65d17fe-9829-44d5-bf07-1abb70f9d523 (id: 0)
qdhcp-f6a245eb-001d-47b1-8af5-38178585fe87 (id: 6)
qdhcp-0fb79928-ae24-4d85-8c58-b1acb9c8c9d2 (id: 2)
qdhcp-0ab1f94c-1e06-485c-b024-548a927a5e36 (id: 1)
root@controller1:/# ip netns exec qrouter-fa11f06e-906c-4ae9-8176-20fb74e1cacd ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=128 time=11.7 ms
--- 8.8.8.8 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 11.679/11.679/11.679/0.000 ms
root@controller1:/# ip netns exec qrouter-fa11f06e-906c-4ae9-8176-20fb74e1cacd ip route
default via 10.0.2.1 dev qg-61a6ea6f-7e proto static
10.0.2.0/24 dev qg-61a6ea6f-7e proto kernel scope link src 10.0.2.51
172.16.1.0/24 dev qr-67d37c5f-12 proto kernel scope link src 172.16.1.1
So everything is working fine... And now i am attaching FIP
root@controller1:/# openstack floating ip list
| ID | Floating IP Address | Fixed IP Address | Port | Floating Network | Project |
| 8a3333a9-345d-4b2a-9d63-420f09e4c020 | 10.0.2.106| 172.16.1.236| edef7b03-25a9-43b4-9953-831539056ac3 | d65d17fe-9829-44d5-bf07-1abb70f9d523 | 957f142f850240b5801023369eace69a |
It is pingable from my local PC and i can access instance via SSH as well, but I cannot access internet from provider network:
root@controller1:/# ip netns exec qrouter-fa11f06e-906c-4ae9-8176-20fb74e1cacd ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
^C
--- 8.8.8.8 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2040ms
This is tcpdump from compute node:
root@compute1:/# tcpdump -i ens34 icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens34, link-type EN10MB (Ethernet), capture size 262144 bytes
17:30:00.258697 IP 10.0.2.106 > 8.8.8.8: ICMP echo request, id 41872, seq 0, length 64
17:30:01.259844 IP 10.0.2.106 > 8.8.8.8: ICMP echo request, id 41872, seq 1, length 64
So packets are going through provider interface ens34. I think that is routing problem on compute node but I cannot find where it is.
EDIT
Interfaces in namespace
root@controller1:/# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: qr-67d37c5f-12@if6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default qlen 1000
link/ether fa:16:3e:cb:0e:3a brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 172.16.1.1/24 brd 172.16.1.255 scope global qr-67d37c5f-12
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fecb:e3a/64 scope link
valid_lft forever preferred_lft forever
3: qg-61a6ea6f-7e@if7: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether fa:16:3e:60:e9:e4 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 10.0.2.51/24 brd 10.0.2.255 scope global qg-61a6ea6f-7e
valid_lft forever preferred_lft forever
inet 10.0.2.106/32 brd 10.0.2.106 scope global qg-61a6ea6f-7e
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fe60:e9e4/64 scope link
valid_lft forever preferred_lft forever
Routing table in namespace
root@controller1:/# ip r
default via 10.0.2.1 dev qg-61a6ea6f-7e proto static
10.0.2.0/24 dev qg-61a6ea6f-7e proto kernel scope link src 10.0.2.51
172.16.1.0/24 dev qr-67d37c5f-12 proto kernel scope link src 172.16.1.1
iptables in namespace
root@controller1:/# iptables-save
# Generated by iptables-save v1.8.4 on Tue Feb 23 12:51:04 2021
*raw
:PREROUTING ACCEPT [105:8611]
:OUTPUT ACCEPT [65:6090]
:neutron-l3-agent-OUTPUT - [0:0]
:neutron-l3-agent-PREROUTING - [0:0]
-A PREROUTING -j neutron-l3-agent-PREROUTING
-A OUTPUT -j neutron-l3-agent-OUTPUT
COMMIT
# Completed on Tue Feb 23 12:51:04 2021
# Generated by iptables-save v1.8.4 on Tue Feb 23 12:51:04 2021
*nat
:PREROUTING ACCEPT [8:1322]
:INPUT ACCEPT [18:1372]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:neutron-l3-agent-OUTPUT - [0:0]
:neutron-l3-agent-POSTROUTING - [0:0]
:neutron-l3-agent-PREROUTING - [0:0]
:neutron-l3-agent-float-snat - [0:0]
:neutron-l3-agent-snat - [0:0]
:neutron-postrouting-bottom - [0:0]
-A PREROUTING -j neutron-l3-agent-PREROUTING
-A OUTPUT -j neutron-l3-agent-OUTPUT
-A POSTROUTING -j neutron-l3-agent-POSTROUTING
-A POSTROUTING -j neutron-postrouting-bottom
-A neutron-l3-agent-OUTPUT -d 10.0.2.106/32 -j DNAT --to-destination 172.16.1.236
-A neutron-l3-agent-POSTROUTING ! -o qg-61a6ea6f-7e -m conntrack ! --ctstate DNAT -j ACCEPT
-A neutron-l3-agent-PREROUTING -d 10.0.2.106/32 -j DNAT --to-destination 172.16.1.236
-A neutron-l3-agent-PREROUTING -d 169.254.169.254/32 -i qr-+ -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 9697
-A neutron-l3-agent-float-snat -s 172.16.1.236/32 -j SNAT --to-source 10.0.2.106 --random-fully
-A neutron-l3-agent-snat -j neutron-l3-agent-float-snat
-A neutron-l3-agent-snat -o qg-61a6ea6f-7e -j SNAT --to-source 10.0.2.51 --random-fully
-A neutron-l3-agent-snat -m mark ! --mark 0x2/0xffff -m conntrack --ctstate DNAT -j SNAT --to-source 10.0.2.51 --random-fully
-A neutron-postrouting-bottom -m comment --comment "Perform source NAT on outgoing traffic." -j neutron-l3-agent-snat
COMMIT
# Completed on Tue Feb 23 12:51:04 2021
# Generated by iptables-save v1.8.4 on Tue Feb 23 12:51:04 2021
*mangle
:PREROUTING ACCEPT [105:8611]
:INPUT ACCEPT [99:7701]
:FORWARD ACCEPT [2:102]
:OUTPUT ACCEPT [65:6090]
:POSTROUTING ACCEPT [67:6192]
:neutron-l3-agent-FORWARD - [0:0]
:neutron-l3-agent-INPUT - [0:0]
:neutron-l3-agent-OUTPUT - [0:0]
:neutron-l3-agent-POSTROUTING - [0:0]
:neutron-l3-agent-PREROUTING - [0:0]
:neutron-l3-agent-float-snat - [0:0]
:neutron-l3-agent-floatingip - [0:0]
:neutron-l3-agent-mark - [0:0]
:neutron-l3-agent-scope - [0:0]
-A PREROUTING -j neutron-l3-agent-PREROUTING
-A INPUT -j neutron-l3-agent-INPUT
-A FORWARD -j neutron-l3-agent-FORWARD
-A OUTPUT -j neutron-l3-agent-OUTPUT
-A POSTROUTING -j neutron-l3-agent-POSTROUTING
-A neutron-l3-agent-POSTROUTING -o qg-61a6ea6f-7e -m connmark --mark 0x0/0xffff0000 -j CONNMARK --save-mark --nfmask 0xffff0000 --ctmask 0xffff0000
-A neutron-l3-agent-PREROUTING -j neutron-l3-agent-mark
-A neutron-l3-agent-PREROUTING -j neutron-l3-agent-scope
-A neutron-l3-agent-PREROUTING -m connmark ! --mark 0x0/0xffff0000 -j CONNMARK --restore-mark --nfmask 0xffff0000 --ctmask 0xffff0000
-A neutron-l3-agent-PREROUTING -j neutron-l3-agent-floatingip
-A neutron-l3-agent-PREROUTING -d 169.254.169.254/32 -i qr-+ -p tcp -m tcp --dport 80 -j MARK --set-xmark 0x1/0xffff
-A neutron-l3-agent-float-snat -m connmark --mark 0x0/0xffff0000 -j CONNMARK --save-mark --nfmask 0xffff0000 --ctmask 0xffff0000
-A neutron-l3-agent-mark -i qg-61a6ea6f-7e -j MARK --set-xmark 0x2/0xffff
-A neutron-l3-agent-scope -i qr-67d37c5f-12 -j MARK --set-xmark 0x4000000/0xffff0000
-A neutron-l3-agent-scope -i qg-61a6ea6f-7e -j MARK --set-xmark 0x4000000/0xffff0000
COMMIT
# Completed on Tue Feb 23 12:51:04 2021
# Generated by iptables-save v1.8.4 on Tue Feb 23 12:51:04 2021
*filter
:INPUT ACCEPT [3:741]
:FORWARD ACCEPT [2:102]
:OUTPUT ACCEPT [65:6090]
:neutron-filter-top - [0:0]
:neutron-l3-agent-FORWARD - [0:0]
:neutron-l3-agent-INPUT - [0:0]
:neutron-l3-agent-OUTPUT - [0:0]
:neutron-l3-agent-local - [0:0]
:neutron-l3-agent-scope - [0:0]
-A INPUT -j neutron-l3-agent-INPUT
-A FORWARD -j neutron-filter-top
-A FORWARD -j neutron-l3-agent-FORWARD
-A OUTPUT -j neutron-filter-top
-A OUTPUT -j neutron-l3-agent-OUTPUT
-A neutron-filter-top -j neutron-l3-agent-local
-A neutron-l3-agent-FORWARD -j neutron-l3-agent-scope
-A neutron-l3-agent-INPUT -m mark --mark 0x1/0xffff -j ACCEPT
-A neutron-l3-agent-INPUT -p tcp -m tcp --dport 9697 -j DROP
-A neutron-l3-agent-scope -o qr-67d37c5f-12 -m mark ! --mark 0x4000000/0xffff0000 -j DROP
COMMIT
# Completed on Tue Feb 23 12:51:04 2021
tcpdump from namespace
root@controller1:/# tcpdump -e -i any host 8.8.8.8
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked v1), capture size 262144 bytes
13:56:31.517183 Out fa:16:3e:d9:60:22 (oui Unknown) ethertype IPv4 (0x0800), length 100: 10.0.2.131 > 8.8.8.8: ICMP echo request, id 58998, seq 14, length 64
13:56:31.524754 In 00:50:56:f5:43:f8 (oui Unknown) ethertype IPv4 (0x0800), length 100: 8.8.8.8 > 10.0.2.131: ICMP echo reply, id 58998, seq 14, length 64
13:56:31.524803 Out fa:16:3e:e3:79:16 (oui Unknown) ethertype IPv4 (0x0800), length 100: 8.8.8.8 > 172.16.1.236: ICM P echo reply, id 34561, seq 14, length 64
13:56:32.518197 In fa:16:3e:6e:82:9e (oui Unknown) ethertype IPv4 (0x0800), length 100: 172.16.1.236 > 8.8.8.8: ICM P echo request, id 34561, seq 15, length 64
13:56:32.518237 Out fa:16:3e:d9:60:22 (oui Unknown) ethertype IPv4 (0x0800), length 100: 10.0.2.131 > 8.8.8.8: ICMP echo request, id 58998, seq 15, length 64
13:56:33.519420 In fa:16:3e:6e:82:9e (oui Unknown) ethertype IPv4 (0x0800), length 100: 172.16.1.236 > 8.8.8.8: ICM P echo request, id 34561, seq 16, length 64
13:56:33.519463 Out fa:16:3e:d9:60:22 (oui Unknown) ethertype IPv4 (0x0800), length 100: 10.0.2.131 > 8.8.8.8: ICMP echo request, id 58998, seq 16, length 64
13:56:34.520250 In fa:16:3e:6e:82:9e (oui Unknown) ethertype IPv4 (0x0800), length 100: 172.16.1.236 > 8.8.8.8: ICM P echo request, id 34561, seq 17, length 64
13:56:34.520291 Out fa:16:3e:d9:60:22 (oui Unknown) ethertype IPv4 (0x0800), length 100: 10.0.2.131 > 8.8.8.8: ICMP echo request, id 58998, seq 17, length 64
13:56:35.521179 In fa:16:3e:6e:82:9e (oui Unknown) ethertype IPv4 (0x0800), length 100: 172.16.1.236 > 8.8.8.8: ICM P echo request, id 34561, seq 18, length 64
13:56:35.521216 Out fa:16:3e:d9:60:22 (oui Unknown) ethertype IPv4 (0x0800), length 100: 10.0.2.131 > 8.8.8.8: ICMP echo request, id 58998, seq 18, length 64
13:56:36.522122 In fa:16:3e:6e:82:9e (oui Unknown) ethertype IPv4 (0x0800), length 100: 172.16.1.236 > 8.8.8.8: ICM P echo request, id 34561, seq 19, length 64
13:56:36.522158 Out fa:16:3e:d9:60:22 (oui Unknown) ethertype IPv4 (0x0800), length 100: 10.0.2.131 > 8.8.8.8: ICMP echo request, id 58998, seq 19, length 64
13:56:37.522683 In fa:16:3e:6e:82:9e (oui Unknown) ethertype IPv4 (0x0800), length 100: 172.16.1.236 > 8.8.8.8: ICM P echo request, id 34561, seq 20, length 64
I am installing an openstack infrastructure but i have an issue with the selfservice networks and the ipv6
On my provider network i have the fddd:3138:5d15:18::/64 subnet which is my local server range.
I've managed to make the ipv6 works in a SLAAC-SLAAC configuration with a second /64 subnet (fddd:3138:5d15:180::/64) and a static route in my switches (by defining the provider subnet + gateway and selfservice subnet)
But my question is : Is it possible to use the "none - slaac" mode (or other) in my selfservice network with the "provider range" fddd:3138:5d15:18::/64 and let my external router advertise the RA for the openstack instances ? Or is it only possible on a "provider" network without a neutron router ?
Just like that i don't have to create a second subnet for each vlan we have.
I hope it's understandable.
Thanks.