vpn basic ipv6 ping tests fail

Description

https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/netvirt-csit-1node-0cmb-1ctl-2cmp-openstack-queens-upstream-stateful-neon/302/odl_1/odl1_karaf.log.gz, also job 305, 306, 308 (for arp learning suite)

The following two tests fail with the ping error below. The tests fail randomly every few days or so.

vpn basic ipv6.Delete And Recreate Extra
vpn basic ipv6.Add Multiple Extra Routes And Check Datapath Before L3VPN Creation

'PING 2001:db9:cafe:d::10 (2001:db9:cafe:d::10): 56 data bytes --- 2001:db9:cafe:d::10 ping statistics --- 3 packets transmitted, 0 packets received, 100% packet loss $' does not contain '64 bytes'

Environment

None

Attachments

01 Oct 2018, 03:32 PM

Linked issues

relates to

OPNFLWPLUG-1038

forwardingrules-manager: Flow add with id L3.149652893104203.21.e7914c35-eafe-4208-9f79-9813ffbc7f69.74.2001:db9:cafe:d:0:0:0:0 failed with error [RpcError [message=Group 150070 not present in the config inventory

Activity

Show:

karthikeyan Krishnan October 10, 2018 at 11:51 AM

This is fixed in fluorine and neon release as part of ECMP issue(ECMP flow groups are existed before pushing flows) fix and please refer the below gerrit link details.

Neon:

https://git.opendaylight.org/gerrit/#/c/76628/

Fluorine:

https://git.opendaylight.org/gerrit/#/c/76629/

Note: These fix got merged into neon branch on Oct-04-2018. After that, we are not seeing this extra route v6 ping issue in "vpn basic ipv6" suite. Hence closing this issue.

karthikeyan Krishnan October 10, 2018 at 6:50 AM

Initial Analysis:

Extra route V6 traffic is not working because of absence of one of the V6 extra route entry in FIB table

Controller Node-1 (ECMP group 150066 is missing here)
==================
group_id=150070,type=select,bucket=actions=set_field:0x42b->tun_id,set_field:fa:16:3e:35:29:9c->eth_dst,load:0x200->NXM_NX_REG6[],resubmit(,220)
group_id=150072,type=select,bucket=actions=set_field:0x42b->tun_id,set_field:fa:16:3e:35:29:9c->eth_dst,load:0x200->NXM_NX_REG6[],resubmit(,220)

Extra route (2001:db9:cafe:d::/64) flow entry is missing in FIB table

cookie=0x8000003, duration=30.210s, table=21, n_packets=0, n_bytes=0, priority=74,ipv6,metadata=0x30e0a/0xfffffe,ipv6_dst=2001:db9:abcd:d::/64 actions=group:150072

Compute Node -1 (ECMP group 150066 is proper and this compute VM extra route traffic is working fine)
================
group_id=150072,type=select,bucket=actions=group:150066
group_id=150070,type=select,bucket=actions=group:150066
group_id=150066,type=all,bucket=actions=set_field:fe:16:3e:35:29:9c->eth_src,set_field:fa:16:3e:35:29:9c->eth_dst,load:0xb200->NXM_NX_REG6[],resubmit(,220)

Both the extra route flows are appearing on FIB entry

cookie=0x8000003, duration=31.399s, table=21, n_packets=0, n_bytes=0, priority=74,ipv6,metadata=0x30e0a/0xfffffe,ipv6_dst=2001:db9:cafe:d::/64 actions=group:150070 cookie=0x8000003, duration=31.380s, table=21, n_packets=0, n_bytes=0, priority=74,ipv6,metadata=0x30e0a/0xfffffe,ipv6_dst=2001:db9:abcd:d::/64 actions=group:150072

Compute Node-2 (ECMP group 150066 is missing here)
==============
group_id=150070,type=select,bucket=actions=set_field:0x42b->tun_id,set_field:fa:16:3e:35:29:9c->eth_dst,load:0x500->NXM_NX_REG6[],resubmit(,220)
group_id=150072,type=select,bucket=actions=set_field:0x42b->tun_id,set_field:fa:16:3e:35:29:9c->eth_dst,load:0x500->NXM_NX_REG6[],resubmit(,220)

Extra route (2001:db9:cafe:d::/64) flow entry is missing in FIB table

cookie=0x8000003, duration=33.972s, table=21, n_packets=6, n_bytes=708, priority=74,ipv6,metadata=0x30e0a/0xfffffe,ipv6_dst=2001:db9:abcd:d::/64 actions=group:150072