WriteTransaction.mergeParentStructure{Merge,Put} does not work with augmentation InstanceIdentifier

Description

Discovered in netconf-csit-1node-userfeatures-all-master #260 - The test was done with the master branch (7.0.0-SNAPSHOT) but 6.0.0 also has the same issue.

The following NETCONF create-device RPC fails:

POST Request : url=http://10.30.170.237:8181/rests/operations/netconf-node-topology:create-device path_url=/rests/operations/netconf-node-topology:create-device headers={'User-Agent': 'python-requests/2.31.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': 'application/xml', 'Connection': 'keep-alive', 'Content-Type': 'application/xml', 'Cookie': 'JSESSIONID=node0bnhlhqxk9lc11mgprjh259gwa3.node0', 'Content-Length': '292', 'Authorization': 'Basic YWRtaW46YWRtaW4='} body=<input xmlns="urn:opendaylight:netconf-node-topology"> <node-id>netconf-test-device</node-id> <host>10.30.171.161</host> <port>17830</port> <username>admin</username> <password>topsecret</password> <tcp-only>false</tcp-only> <keepalive-delay>0</keepalive-delay> </input>

with this response:

POST Response : url=http://10.30.170.237:8181/rests/operations/netconf-node-topology:create-device status=500, reason=Internal Server Error headers={'Content-Type': 'application/yang-data+xml', 'Content-Length': '247'} body=<errors xmlns="urn:ietf:params:xml:ns:yang:ietf-restconf"><error><error-tag>operation-failed</error-tag><error-info>&lt;severity>error&lt;/severity></error-info><error-message>RPC failed</error-message><error-type>rpc</error-type></error></errors>

Karaf log shows the following error:

2023-07-21T07:09:17,931 | ERROR | CommitFutures-0 | NetconfTopologyRPCProvider | 293 - org.opendaylight.netconf.topology - 7.0.0.SNAPSHOT | add-netconf-node RPC: Unable to add netconf node. org.opendaylight.mdsal.common.api.TransactionCommitFailedException: canCommit encountered an unexpected failure at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:37) ~[bundleFile:?] at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:18) ~[bundleFile:?] at org.opendaylight.yangtools.util.concurrent.ExceptionMapper.apply(ExceptionMapper.java:99) ~[bundleFile:?] at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker.handleException(ConcurrentDOMDataBroker.java:164) ~[bundleFile:?] at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker$1.onFailure(ConcurrentDOMDataBroker.java:108) ~[bundleFile:?] at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1119) ~[bundleFile:?] at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31) ~[bundleFile:?] at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1286) ~[bundleFile:?] at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1055) ~[bundleFile:?] at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:807) ~[bundleFile:?] at com.google.common.util.concurrent.SettableFuture.setException(SettableFuture.java:55) ~[bundleFile:?] at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction.lambda$directCommit$4(AbstractProxyTransaction.java:517) ~[bundleFile:?] at org.opendaylight.controller.cluster.access.client.ConnectionEntry.complete(ConnectionEntry.java:47) ~[bundleFile:?] at org.opendaylight.controller.cluster.access.client.AbstractClientConnection.receiveResponse(AbstractClientConnection.java:494) ~[bundleFile:?] at org.opendaylight.controller.cluster.access.client.AbstractReceivingClientConnection.receiveResponse(AbstractReceivingClientConnection.java:68) ~[bundleFile:?] at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onResponse(ClientActorBehavior.java:184) ~[bundleFile:?] at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onRequestFailure(ClientActorBehavior.java:196) ~[bundleFile:?] at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.internalOnRequestFailure(ClientActorBehavior.java:248) ~[bundleFile:?] at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onReceiveCommand(ClientActorBehavior.java:155) ~[bundleFile:?] at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onReceiveCommand(ClientActorBehavior.java:48) ~[bundleFile:?] at org.opendaylight.controller.cluster.access.client.AbstractClientActor.onReceiveCommand(AbstractClientActor.java:74) ~[bundleFile:?] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) ~[bundleFile:?] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) ~[bundleFile:?] at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) ~[bundleFile:?] at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) ~[bundleFile:?] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) ~[bundleFile:?] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) ~[bundleFile:?] at akka.actor.Actor.aroundReceive(Actor.scala:537) ~[bundleFile:?] at akka.actor.Actor.aroundReceive$(Actor.scala:535) ~[bundleFile:?] at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) ~[bundleFile:?] at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) ~[bundleFile:?] at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) ~[bundleFile:?] at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) ~[bundleFile:?] at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) ~[bundleFile:?] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) ~[bundleFile:?] at akka.actor.ActorCell.invoke(ActorCell.scala:547) ~[bundleFile:?] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) ~[bundleFile:?] at akka.dispatch.Mailbox.run(Mailbox.scala:231) ~[bundleFile:?] at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ~[bundleFile:?] at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) ~[?:?] at java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) ~[?:?] at java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) ~[?:?] at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) ~[?:?] at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) ~[?:?] Caused by: java.lang.IllegalArgumentException: Metadata not available for modification ModifiedNode{identifier=(urn:TBD:params:xml:ns:yang:network-topology?revision=2013-10-21)node[{(urn:TBD:params:xml:ns:yang:network-topology?revision=2013-10-21)node-id=netconf-test-device}], operation=TOUCH, childModification={(urn:opendaylight:netconf-node-topology?revision=2022-12-25)sleep-factor=ModifiedNode{identifier=(urn:opendaylight:netconf-node-topology?revision=2022-12-25)sleep-factor, operation=MERGE}, (urn:opendaylight:netconf-node-topology?revision=2022-12-25)default-request-timeout-millis=ModifiedNode{identifier=(urn:opendaylight:netconf-node-topology?revision=2022-12-25)default-request-timeout-millis, operation=MERGE}, (urn:opendaylight:netconf-node-topology?revision=2022-12-25)port=ModifiedNode{identifier=(urn:opendaylight:netconf-node-topology?revision=2022-12-25)port, operation=MERGE}, (urn:opendaylight:netconf-node-topology?revision=2022-12-25)connection-timeout-millis=ModifiedNode{identifier=(urn:opendaylight:netconf-node-topology?revision=2022-12-25)connection-timeout-millis, operation=MERGE}, (urn:opendaylight:netconf-node-topology?revision=2022-12-25)tcp-only=ModifiedNode{identifier=(urn:opendaylight:netconf-node-topology?revision=2022-12-25)tcp-only, operation=MERGE}, (urn:opendaylight:netconf-node-topology?revision=2022-12-25)lock-datastore=ModifiedNode{identifier=(urn:opendaylight:netconf-node-topology?revision=2022-12-25)lock-datastore, operation=MERGE}, (urn:opendaylight:netconf-node-topology?revision=2022-12-25)concurrent-rpc-limit=ModifiedNode{identifier=(urn:opendaylight:netconf-node-topology?revision=2022-12-25)concurrent-rpc-limit, operation=MERGE}, (urn:opendaylight:netconf-node-topology?revision=2022-12-25)host=ModifiedNode{identifier=(urn:opendaylight:netconf-node-topology?revision=2022-12-25)host, operation=MERGE}, (urn:opendaylight:netconf-node-topology?revision=2022-12-25)max-connection-attempts=ModifiedNode{identifier=(urn:opendaylight:netconf-node-topology?revision=2022-12-25)max-connection-attempts, operation=MERGE}, (urn:opendaylight:netconf-node-topology?revision=2022-12-25)credentials=ModifiedNode{identifier=(urn:opendaylight:netconf-node-topology?revision=2022-12-25)credentials, operation=MERGE}, (urn:opendaylight:netconf-node-topology?revision=2022-12-25)schema-cache-directory=ModifiedNode{identifier=(urn:opendaylight:netconf-node-topology?revision=2022-12-25)schema-cache-directory, operation=MERGE}, (urn:opendaylight:netconf-node-topology?revision=2022-12-25)between-attempts-timeout-millis=ModifiedNode{identifier=(urn:opendaylight:netconf-node-topology?revision=2022-12-25)between-attempts-timeout-millis, operation=MERGE}, (urn:opendaylight:netconf-node-topology?revision=2022-12-25)actor-response-wait-time=ModifiedNode{identifier=(urn:opendaylight:netconf-node-topology?revision=2022-12-25)actor-response-wait-time, operation=MERGE}, (urn:opendaylight:netconf-node-topology?revision=2022-12-25)schemaless=ModifiedNode{identifier=(urn:opendaylight:netconf-node-topology?revision=2022-12-25)schemaless, operation=MERGE}, (urn:opendaylight:netconf-node-topology?revision=2022-12-25)reconnect-on-changed-schema=ModifiedNode{identifier=(urn:opendaylight:netconf-node-topology?revision=2022-12-25)reconnect-on-changed-schema, operation=MERGE}, (urn:opendaylight:netconf-node-topology?revision=2022-12-25)keepalive-delay=ModifiedNode{identifier=(urn:opendaylight:netconf-node-topology?revision=2022-12-25)keepalive-delay, operation=MERGE}}} at com.google.common.base.Preconditions.checkArgument(Preconditions.java:218) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:206) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:219) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.applyWrite(AbstractNodeContainerModificationStrategy.java:193) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:218) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.MapModificationStrategy.lambda$apply$0(MapModificationStrategy.java:66) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.AutomaticLifecycleMixin.apply(AutomaticLifecycleMixin.java:61) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.MapModificationStrategy.apply(MapModificationStrategy.java:66) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:219) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:329) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.applyMerge(AbstractNodeContainerModificationStrategy.java:248) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:221) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:219) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:329) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.applyMerge(AbstractNodeContainerModificationStrategy.java:248) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:221) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.MapModificationStrategy.lambda$apply$0(MapModificationStrategy.java:66) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.AutomaticLifecycleMixin.apply(AutomaticLifecycleMixin.java:61) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.MapModificationStrategy.apply(MapModificationStrategy.java:66) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:219) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:329) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.applyMerge(AbstractNodeContainerModificationStrategy.java:248) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:221) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.StructuralContainerModificationStrategy.lambda$apply$0(StructuralContainerModificationStrategy.java:37) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.AutomaticLifecycleMixin.apply(AutomaticLifecycleMixin.java:61) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.StructuralContainerModificationStrategy.apply(StructuralContainerModificationStrategy.java:37) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:219) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:329) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:207) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.AbstractDataTreeTip.prepare(AbstractDataTreeTip.java:53) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.ShardDataTree.startPreCommit(ShardDataTree.java:1032) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.preCommit(SimpleShardDataTreeCohort.java:112) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.successfulDirectCanCommit(FrontendReadWriteTransaction.java:459) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction$8.onSuccess(FrontendReadWriteTransaction.java:436) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction$8.onSuccess(FrontendReadWriteTransaction.java:433) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.successfulCanCommit(SimpleShardDataTreeCohort.java:171) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$4(ShardDataTree.java:840) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPending(ShardDataTree.java:884) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPendingTransaction(ShardDataTree.java:831) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.ShardDataTree.startCanCommit(ShardDataTree.java:986) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.canCommit(SimpleShardDataTreeCohort.java:99) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.directCommit(FrontendReadWriteTransaction.java:433) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.handleCommitLocalTransaction(FrontendReadWriteTransaction.java:524) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.doHandleRequest(FrontendReadWriteTransaction.java:193) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:138) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:122) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:638) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:440) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:368) ~[bundleFile:?] at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) ~[bundleFile:?] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) ~[bundleFile:?] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) ~[bundleFile:?] at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) ~[bundleFile:?] at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) ~[bundleFile:?] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) ~[bundleFile:?] at scala.PartialFunction$OrElse.apply(PartialFunction.scala:266) ~[bundleFile:?] at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) ~[bundleFile:?] at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) ~[bundleFile:?] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) ~[bundleFile:?] at scala.PartialFunction$OrElse.apply(PartialFunction.scala:266) ~[bundleFile:?] at org.opendaylight.controller.cluster.common.actor.MeteringBehavior.apply(MeteringBehavior.java:93) ~[bundleFile:?] at org.opendaylight.controller.cluster.common.actor.MeteringBehavior.apply(MeteringBehavior.java:29) ~[bundleFile:?] at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) ~[bundleFile:?] at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) ~[bundleFile:?] at scala.runtime.AbstractPartialFunction.applyOrElse(AbstractPartialFunction.scala:27) ~[bundleFile:?] ... 17 more

Environment

None

Attachments

2
  • 11 Aug 2023, 12:38 PM
  • 11 Aug 2023, 12:33 PM

Activity

Show:

Ruslan Kashapov August 29, 2023 at 3:18 PM
Edited

confirmed: the issue is cause by usage of augmentation node with direct instance identifier in merge operation .

mdsal instance identifier was updated to point augmentation target node bc augmentation node is no longer exists in normalized data structure.

once updated to use target node instead of augmentation the issue is no longer reproducible

https://git.opendaylight.org/gerrit/c/netconf/+/107555

Sangwook Ha August 17, 2023 at 10:30 PM

mergeParentStructureMerge & mergeParentStructurePut don't seem to work for Augmentation which NetconfNode is and this probably is a fallout of MDSAL-820.
It appears that we can sidestep this issue if mergeParentStructureMerge is done with topology.Node wrapping NetconfNode instead of NetconfNode although this wouldn't address the root cause.
The reason why mergeParentStructurePut works for OPERATIONAL in NetconfDeviceTopologyAdapter is because topology.Node is created when NetconfDeviceTopologyAdapter is created, so the parent of NetconfNode is already available when it's updated.

Ivan Hraško August 17, 2023 at 9:06 AM

the probable problem here is that config data for topology does not exists and ensure parents is probably not a solution/working anymore?

Ivan Hraško August 17, 2023 at 7:08 AM

It means that I was not able to reproduce original error as reported -  I have just ended up with validation errors before and after my patch .

Interesting is that netcof codebase definitely contains some logic which writes nodes into datastore - but OPERATIONAL one. Please check - IIRC netconf-topology.

Ivan Hraško August 16, 2023 at 3:11 PM

I have changed operation but then I got another exception: https://git.opendaylight.org/gerrit/c/netconf/+/107439

Done

Details

Assignee

Reporter

Labels

Components

Fix versions

Affects versions

Priority

Created July 21, 2023 at 8:11 AM
Updated February 6, 2025 at 2:12 PM
Resolved August 30, 2023 at 11:45 AM