IllegalArgumentException: vrfEntry is missing mandatory descendant origin

Oh, sorry did not realise the date. I just took the last Nitrogen build which happens to be on Sept.

Please ignore my comments. Closing it again

Janki,

Why are you re-opening bugs with 6th September logs.

The fix for this landed only during end of September.

Please check for logs after Oct 1, to make sure this issue continues to exist.

Reopening, seen in
https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-newton-nodl-v2-upstream-stateful-nitrogen/197/odl1_karaf.log.gz

2017-09-06 04:53:58,644 | INFO | nPool-1-worker-0 | VpnInterfaceManager | 321 - org.opendaylight.netvirt.vpnmanager-impl - 0.5.0.SNAPSHOT | updateVpnInterfaceOnTepAdd: Updated label mapper : label 100028 dpn 9011006519764 prefix 10.10.10.250/32 nexthoplist [10.29.14.90] vpn 098e7860-5da5-45e4-b1c0-34ae67c433bd vpnid 100010 rd 098e7860-5da5-45e4-b1c0-34ae67c433bd interface 9011006519764:br-physnet1-pa:trunk
2017-09-06 04:53:58,644 | INFO | eChangeHandler-0 | TunnelInterfaceStateListener | 321 - org.opendaylight.netvirt.vpnmanager-impl - 0.5.0.SNAPSHOT | handleTunnelEventForDPN: Remote DpnId 9011006519764 VpnId 100010 rd 098e7860-5da5-45e4-b1c0-34ae67c433bd VpnInterface VpnInterface{getDpnId=9011006519764, getName=9011006519764:br-physnet1-pa:trunk, getVpnInstanceName=098e7860-5da5-45e4-b1c0-34ae67c433bd, isScheduledForRemove=true, augmentations={interface org.opendaylight.yang.gen.v1.urn.opendaylight.netvirt.l3vpn.rev130911.Adjacencies=Adjacencies{getAdjacency=[Adjacency{getAdjacencyType=PrimaryAdjacency, getIpAddress=10.10.10.250/32, getLabel=100028, getMacAddress=F6:00:00:FF:01:01, getNextHopIpList=[10.29.14.90], getSubnetId=Uuid [_value=61bffd7f-170e-4156-b3e4-0885521f219e], isPhysNetworkFunc=true, augmentations={}}]}}} srcTepIp 10.29.15.141 destTepIp 10.29.14.90
2017-09-06 04:53:58,647 | ERROR | rd-dispatcher-25 | SimpleShardDataTreeCohort | 212 - org.opendaylight.controller.sal-distributed-datastore - 1.6.0.SNAPSHOT | Transaction member-1-datastore-config-fe-0-txn-151828-0 failed to prepare
java.lang.IllegalArgumentException: Node (urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)vrfEntry[{(urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)destPrefix=10.10.10.250/32}] is missing mandatory descendant /(urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)origin
at com.google.common.base.Preconditions.checkArgument(Preconditions.java:395)[27:com.google.guava:22.0.0]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.MandatoryLeafEnforcer$Strict.enforceOnData(MandatoryLeafEnforcer.java:43)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.MandatoryLeafEnforcer.enforceOnTreeNode(MandatoryLeafEnforcer.java:58)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.ListEntryModificationStrategy.applyWrite(ListEntryModificationStrategy.java:49)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:210)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:139)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:252)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyMerge(AbstractNodeContainerModificationStrategy.java:170)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:213)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:139)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:252)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.ListEntryModificationStrategy.applyTouch(ListEntryModificationStrategy.java:55)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyMerge(AbstractNodeContainerModificationStrategy.java:170)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.ListEntryModificationStrategy.applyMerge(ListEntryModificationStrategy.java:40)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:213)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:139)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:252)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyMerge(AbstractNodeContainerModificationStrategy.java:170)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:213)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:139)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:252)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyMerge(AbstractNodeContainerModificationStrategy.java:170)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:213)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.StructuralContainerModificationStrategy.apply(StructuralContainerModificationStrategy.java:71)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:139)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:252)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:200)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.RootModificationApplyOperation.apply(RootModificationApplyOperation.java:78)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.yangtools.yang.data.impl.schema.tree.AbstractDataTreeTip.prepare(AbstractDataTreeTip.java:53)[372:org.opendaylight.yangtools.yang-data-impl:1.2.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.startPreCommit(ShardDataTree.java:820)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.preCommit(SimpleShardDataTreeCohort.java:103)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.CohortEntry.preCommit(CohortEntry.java:101)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.doCommit(ShardCommitCoordinator.java:295)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator$2.onSuccess(ShardCommitCoordinator.java:242)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator$2.onSuccess(ShardCommitCoordinator.java:236)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.successfulCanCommit(SimpleShardDataTreeCohort.java:171)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$0(ShardDataTree.java:725)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPending(ShardDataTree.java:769)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPendingTransaction(ShardDataTree.java:716)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardDataTree.startCanCommit(ShardDataTree.java:799)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.canCommit(SimpleShardDataTreeCohort.java:90)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.CohortEntry.canCommit(CohortEntry.java:97)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleCanCommit(ShardCommitCoordinator.java:236)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.ShardCommitCoordinator.handleReadyLocalTransaction(ShardCommitCoordinator.java:200)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.Shard.handleReadyLocalTransaction(Shard.java:729)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:333)[212:org.opendaylight.controller.sal-distributed-datastore:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270)[197:org.opendaylight.controller.sal-akka-raft:1.6.0.SNAPSHOT]
at org.opendaylight.controller.cluster.common.actor.AbstractUntypedPersistentActor.onReceiveCommand(AbstractUntypedPersistentActor.java:44)[205:org.opendaylight.controller.sal-clustering-commons:1.6.0.SNAPSHOT]
at akka.persistence.UntypedPersistentActor.onReceive(PersistentActor.scala:170)[37:com.typesafe.akka.persistence:2.4.18]
at org.opendaylight.controller.cluster.common.actor.MeteringBehavior.apply(MeteringBehavior.java:104)[205:org.opendaylight.controller.sal-clustering-commons:1.6.0.SNAPSHOT]
at akka.actor.ActorCell$$anonfun$become$1.applyOrElse(ActorCell.scala:544)[34:com.typesafe.akka.actor:2.4.18]
at akka.actor.Actor$class.aroundReceive(Actor.scala:502)[34:com.typesafe.akka.actor:2.4.18]
at akka.persistence.UntypedPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:168)[37:com.typesafe.akka.persistence:2.4.18]
at akka.persistence.Eventsourced$$anon$1.stateReceive(Eventsourced.scala:727)[37:com.typesafe.akka.persistence:2.4.18]
at akka.persistence.Eventsourced$class.aroundReceive(Eventsourced.scala:183)[37:com.typesafe.akka.persistence:2.4.18]
at akka.persistence.UntypedPersistentActor.aroundReceive(PersistentActor.scala:168)[37:com.typesafe.akka.persistence:2.4.18]
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)[34:com.typesafe.akka.actor:2.4.18]
at akka.actor.ActorCell.invoke(ActorCell.scala:495)[34:com.typesafe.akka.actor:2.4.18]
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)[34:com.typesafe.akka.actor:2.4.18]
at akka.dispatch.Mailbox.run(Mailbox.scala:224)[34:com.typesafe.akka.actor:2.4.18]
at akka.dispatch.Mailbox.exec(Mailbox.scala:234)[34:com.typesafe.akka.actor:2.4.18]
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)[391:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)[391:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)[391:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)[391:org.scala-lang.scala-library:2.11.11.v20170413-090219-8a413ba7cc]
2017-09-06 04:53:58,648 | ERROR | rd-dispatcher-25 | Shard | 205 - org.opendaylight.controller.sal-clustering-commons - 1.6.0.SNAPSHOT | member-1-shard-default-config An exception occurred while preCommitting transaction member-1-datastore-config-fe-0-txn-151828-0

https://git.opendaylight.org/gerrit/63482

https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-ocata-gate-stateful-oxygen/41/odl1_exception.log.gz

2017-09-19 09:10:11,275 | INFO | nsole user karaf | core | 112 - org.apache.karaf.log.core - 4.0.9 | ROBOT MESSAGE: Starting test Verify Datapath After OVS Restart
java.lang.IllegalArgumentException: Node (urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)vrfEntry[{(urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)destPrefix=10.10.10.250/32}] is missing mandatory descendant /(urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)origin
java.lang.IllegalArgumentException: Node (urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)vrfEntry[{(urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)destPrefix=10.10.10.250/32}] is missing mandatory descendant /(urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)origin
java.lang.IllegalArgumentException: Node (urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)vrfEntry[{(urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)destPrefix=10.10.10.250/32}] is missing mandatory descendant /(urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)origin
java.lang.IllegalArgumentException: Node (urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)vrfEntry[{(urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)destPrefix=10.10.10.250/32}] is missing mandatory descendant /(urn:opendaylight:netvirt:fibmanager?revision=2015-03-30)origin

2017-09-19 09:10:14,584 | ERROR | nPool-2-worker-1 | DataStoreJobCoordinator | 243 - org.opendaylight.genius.mdsalutil-api - 0.4.0.SNAPSHOT | Exception when executing jobEntry: JobEntry{key='VPNINTERFACE-114880759651325:br-physnet1-pa:trunk', mainWorker=org.opendaylight.netvirt.vpnmanager.TunnelInterfaceStateListener$UpdateVpnInterfaceOnTunnelEvent@3b05404f, rollbackWorker=null, retryCount=0, futures=null}
java.lang.RuntimeException: TransactionCommitFailedException{message=canCommit encountered an unexpected failure, errorList=[RpcError [message=canCommit encountered an unexpected failure, severity=ERROR, errorType=APPLICATION,