WriteTransaction.mergeParentStructure{Merge,Put} does not work with augmentation InstanceIdentifier
Description
Environment
Attachments
- 11 Aug 2023, 12:38 PM
- 11 Aug 2023, 12:33 PM
relates to
Activity
Ruslan Kashapov August 29, 2023 at 3:18 PMEdited
confirmed: the issue is cause by usage of augmentation node with direct instance identifier in merge operation .
mdsal instance identifier was updated to point augmentation target node bc augmentation node is no longer exists in normalized data structure.
once updated to use target node instead of augmentation the issue is no longer reproducible
Sangwook Ha August 17, 2023 at 10:30 PM
mergeParentStructureMerge
& mergeParentStructurePut
don't seem to work for Augmentation which NetconfNode
is and this probably is a fallout of MDSAL-820.
It appears that we can sidestep this issue if mergeParentStructureMerge is done with topology.Node
wrapping NetconfNode
instead of NetconfNode
although this wouldn't address the root cause.
The reason why mergeParentStructurePut
works for OPERATIONAL
in NetconfDeviceTopologyAdapter
is because topology.Node
is created when NetconfDeviceTopologyAdapter is created, so the parent of NetconfNode
is already available when it's updated.
Ivan Hraško August 17, 2023 at 9:06 AM
@Ruslan Kashapov the probable problem here is that config data for topology does not exists and ensure parents is probably not a solution/working anymore?
Ivan Hraško August 17, 2023 at 7:08 AM
@Ruslan Kashapov It means that I was not able to reproduce original error as reported - I have just ended up with validation errors before and after my patch .
Interesting is that netcof codebase definitely contains some logic which writes nodes into datastore - but OPERATIONAL one. Please check - IIRC netconf-topology.
Ivan Hraško August 16, 2023 at 3:11 PM
I have changed operation but then I got another exception: https://git.opendaylight.org/gerrit/c/netconf/+/107439
Discovered in netconf-csit-1node-userfeatures-all-master #260 - The test was done with the master branch (7.0.0-SNAPSHOT) but 6.0.0 also has the same issue.
The following NETCONF
create-device
RPC fails:POST Request : url=http://10.30.170.237:8181/rests/operations/netconf-node-topology:create-device path_url=/rests/operations/netconf-node-topology:create-device headers={'User-Agent': 'python-requests/2.31.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': 'application/xml', 'Connection': 'keep-alive', 'Content-Type': 'application/xml', 'Cookie': 'JSESSIONID=node0bnhlhqxk9lc11mgprjh259gwa3.node0', 'Content-Length': '292', 'Authorization': 'Basic YWRtaW46YWRtaW4='} body=<input xmlns="urn:opendaylight:netconf-node-topology"> <node-id>netconf-test-device</node-id> <host>10.30.171.161</host> <port>17830</port> <username>admin</username> <password>topsecret</password> <tcp-only>false</tcp-only> <keepalive-delay>0</keepalive-delay> </input>
with this response:
POST Response : url=http://10.30.170.237:8181/rests/operations/netconf-node-topology:create-device status=500, reason=Internal Server Error headers={'Content-Type': 'application/yang-data+xml', 'Content-Length': '247'} body=<errors xmlns="urn:ietf:params:xml:ns:yang:ietf-restconf"><error><error-tag>operation-failed</error-tag><error-info><severity>error</severity></error-info><error-message>RPC failed</error-message><error-type>rpc</error-type></error></errors>
Karaf log shows the following error:
2023-07-21T07:09:17,931 | ERROR | CommitFutures-0 | NetconfTopologyRPCProvider | 293 - org.opendaylight.netconf.topology - 7.0.0.SNAPSHOT | add-netconf-node RPC: Unable to add netconf node. org.opendaylight.mdsal.common.api.TransactionCommitFailedException: canCommit encountered an unexpected failure at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:37) ~[bundleFile:?] at org.opendaylight.mdsal.dom.broker.TransactionCommitFailedExceptionMapper.newWithCause(TransactionCommitFailedExceptionMapper.java:18) ~[bundleFile:?] at org.opendaylight.yangtools.util.concurrent.ExceptionMapper.apply(ExceptionMapper.java:99) ~[bundleFile:?] at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker.handleException(ConcurrentDOMDataBroker.java:164) ~[bundleFile:?] at org.opendaylight.controller.cluster.databroker.ConcurrentDOMDataBroker$1.onFailure(ConcurrentDOMDataBroker.java:108) ~[bundleFile:?] at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1119) ~[bundleFile:?] at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31) ~[bundleFile:?] at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1286) ~[bundleFile:?] at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1055) ~[bundleFile:?] at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:807) ~[bundleFile:?] at com.google.common.util.concurrent.SettableFuture.setException(SettableFuture.java:55) ~[bundleFile:?] at org.opendaylight.controller.cluster.databroker.actors.dds.AbstractProxyTransaction.lambda$directCommit$4(AbstractProxyTransaction.java:517) ~[bundleFile:?] at org.opendaylight.controller.cluster.access.client.ConnectionEntry.complete(ConnectionEntry.java:47) ~[bundleFile:?] at org.opendaylight.controller.cluster.access.client.AbstractClientConnection.receiveResponse(AbstractClientConnection.java:494) ~[bundleFile:?] at org.opendaylight.controller.cluster.access.client.AbstractReceivingClientConnection.receiveResponse(AbstractReceivingClientConnection.java:68) ~[bundleFile:?] at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onResponse(ClientActorBehavior.java:184) ~[bundleFile:?] at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onRequestFailure(ClientActorBehavior.java:196) ~[bundleFile:?] at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.internalOnRequestFailure(ClientActorBehavior.java:248) ~[bundleFile:?] at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onReceiveCommand(ClientActorBehavior.java:155) ~[bundleFile:?] at org.opendaylight.controller.cluster.access.client.ClientActorBehavior.onReceiveCommand(ClientActorBehavior.java:48) ~[bundleFile:?] at org.opendaylight.controller.cluster.access.client.AbstractClientActor.onReceiveCommand(AbstractClientActor.java:74) ~[bundleFile:?] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) ~[bundleFile:?] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) ~[bundleFile:?] at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) ~[bundleFile:?] at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) ~[bundleFile:?] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) ~[bundleFile:?] at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:269) ~[bundleFile:?] at akka.actor.Actor.aroundReceive(Actor.scala:537) ~[bundleFile:?] at akka.actor.Actor.aroundReceive$(Actor.scala:535) ~[bundleFile:?] at akka.persistence.AbstractPersistentActor.akka$persistence$Eventsourced$$super$aroundReceive(PersistentActor.scala:295) ~[bundleFile:?] at akka.persistence.Eventsourced$$anon$4.stateReceive(Eventsourced.scala:917) ~[bundleFile:?] at akka.persistence.Eventsourced.aroundReceive(Eventsourced.scala:245) ~[bundleFile:?] at akka.persistence.Eventsourced.aroundReceive$(Eventsourced.scala:244) ~[bundleFile:?] at akka.persistence.AbstractPersistentActor.aroundReceive(PersistentActor.scala:295) ~[bundleFile:?] at akka.actor.ActorCell.receiveMessage(ActorCell.scala:579) ~[bundleFile:?] at akka.actor.ActorCell.invoke(ActorCell.scala:547) ~[bundleFile:?] at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) ~[bundleFile:?] at akka.dispatch.Mailbox.run(Mailbox.scala:231) ~[bundleFile:?] at akka.dispatch.Mailbox.exec(Mailbox.scala:243) ~[bundleFile:?] at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373) ~[?:?] at java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182) ~[?:?] at java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655) ~[?:?] at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622) ~[?:?] at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165) ~[?:?] Caused by: java.lang.IllegalArgumentException: Metadata not available for modification ModifiedNode{identifier=(urn:TBD:params:xml:ns:yang:network-topology?revision=2013-10-21)node[{(urn:TBD:params:xml:ns:yang:network-topology?revision=2013-10-21)node-id=netconf-test-device}], operation=TOUCH, childModification={(urn:opendaylight:netconf-node-topology?revision=2022-12-25)sleep-factor=ModifiedNode{identifier=(urn:opendaylight:netconf-node-topology?revision=2022-12-25)sleep-factor, operation=MERGE}, (urn:opendaylight:netconf-node-topology?revision=2022-12-25)default-request-timeout-millis=ModifiedNode{identifier=(urn:opendaylight:netconf-node-topology?revision=2022-12-25)default-request-timeout-millis, operation=MERGE}, (urn:opendaylight:netconf-node-topology?revision=2022-12-25)port=ModifiedNode{identifier=(urn:opendaylight:netconf-node-topology?revision=2022-12-25)port, operation=MERGE}, (urn:opendaylight:netconf-node-topology?revision=2022-12-25)connection-timeout-millis=ModifiedNode{identifier=(urn:opendaylight:netconf-node-topology?revision=2022-12-25)connection-timeout-millis, operation=MERGE}, (urn:opendaylight:netconf-node-topology?revision=2022-12-25)tcp-only=ModifiedNode{identifier=(urn:opendaylight:netconf-node-topology?revision=2022-12-25)tcp-only, operation=MERGE}, (urn:opendaylight:netconf-node-topology?revision=2022-12-25)lock-datastore=ModifiedNode{identifier=(urn:opendaylight:netconf-node-topology?revision=2022-12-25)lock-datastore, operation=MERGE}, (urn:opendaylight:netconf-node-topology?revision=2022-12-25)concurrent-rpc-limit=ModifiedNode{identifier=(urn:opendaylight:netconf-node-topology?revision=2022-12-25)concurrent-rpc-limit, operation=MERGE}, (urn:opendaylight:netconf-node-topology?revision=2022-12-25)host=ModifiedNode{identifier=(urn:opendaylight:netconf-node-topology?revision=2022-12-25)host, operation=MERGE}, (urn:opendaylight:netconf-node-topology?revision=2022-12-25)max-connection-attempts=ModifiedNode{identifier=(urn:opendaylight:netconf-node-topology?revision=2022-12-25)max-connection-attempts, operation=MERGE}, (urn:opendaylight:netconf-node-topology?revision=2022-12-25)credentials=ModifiedNode{identifier=(urn:opendaylight:netconf-node-topology?revision=2022-12-25)credentials, operation=MERGE}, (urn:opendaylight:netconf-node-topology?revision=2022-12-25)schema-cache-directory=ModifiedNode{identifier=(urn:opendaylight:netconf-node-topology?revision=2022-12-25)schema-cache-directory, operation=MERGE}, (urn:opendaylight:netconf-node-topology?revision=2022-12-25)between-attempts-timeout-millis=ModifiedNode{identifier=(urn:opendaylight:netconf-node-topology?revision=2022-12-25)between-attempts-timeout-millis, operation=MERGE}, (urn:opendaylight:netconf-node-topology?revision=2022-12-25)actor-response-wait-time=ModifiedNode{identifier=(urn:opendaylight:netconf-node-topology?revision=2022-12-25)actor-response-wait-time, operation=MERGE}, (urn:opendaylight:netconf-node-topology?revision=2022-12-25)schemaless=ModifiedNode{identifier=(urn:opendaylight:netconf-node-topology?revision=2022-12-25)schemaless, operation=MERGE}, (urn:opendaylight:netconf-node-topology?revision=2022-12-25)reconnect-on-changed-schema=ModifiedNode{identifier=(urn:opendaylight:netconf-node-topology?revision=2022-12-25)reconnect-on-changed-schema, operation=MERGE}, (urn:opendaylight:netconf-node-topology?revision=2022-12-25)keepalive-delay=ModifiedNode{identifier=(urn:opendaylight:netconf-node-topology?revision=2022-12-25)keepalive-delay, operation=MERGE}}} at com.google.common.base.Preconditions.checkArgument(Preconditions.java:218) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:206) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:219) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.applyWrite(AbstractNodeContainerModificationStrategy.java:193) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:218) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.MapModificationStrategy.lambda$apply$0(MapModificationStrategy.java:66) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.AutomaticLifecycleMixin.apply(AutomaticLifecycleMixin.java:61) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.MapModificationStrategy.apply(MapModificationStrategy.java:66) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:219) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:329) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.applyMerge(AbstractNodeContainerModificationStrategy.java:248) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:221) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:219) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:329) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.applyMerge(AbstractNodeContainerModificationStrategy.java:248) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:221) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.MapModificationStrategy.lambda$apply$0(MapModificationStrategy.java:66) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.AutomaticLifecycleMixin.apply(AutomaticLifecycleMixin.java:61) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.MapModificationStrategy.apply(MapModificationStrategy.java:66) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:219) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:329) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.applyMerge(AbstractNodeContainerModificationStrategy.java:248) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:221) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.StructuralContainerModificationStrategy.lambda$apply$0(StructuralContainerModificationStrategy.java:37) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.AutomaticLifecycleMixin.apply(AutomaticLifecycleMixin.java:61) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.StructuralContainerModificationStrategy.apply(StructuralContainerModificationStrategy.java:37) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.mutateChildren(AbstractNodeContainerModificationStrategy.java:219) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.AbstractNodeContainerModificationStrategy.applyTouch(AbstractNodeContainerModificationStrategy.java:329) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.SchemaAwareApplyOperation.apply(SchemaAwareApplyOperation.java:207) ~[bundleFile:?] at org.opendaylight.yangtools.yang.data.tree.impl.AbstractDataTreeTip.prepare(AbstractDataTreeTip.java:53) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.ShardDataTree.startPreCommit(ShardDataTree.java:1032) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.preCommit(SimpleShardDataTreeCohort.java:112) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.successfulDirectCanCommit(FrontendReadWriteTransaction.java:459) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction$8.onSuccess(FrontendReadWriteTransaction.java:436) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction$8.onSuccess(FrontendReadWriteTransaction.java:433) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.successfulCanCommit(SimpleShardDataTreeCohort.java:171) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.ShardDataTree.lambda$processNextPendingTransaction$4(ShardDataTree.java:840) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPending(ShardDataTree.java:884) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.ShardDataTree.processNextPendingTransaction(ShardDataTree.java:831) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.ShardDataTree.startCanCommit(ShardDataTree.java:986) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.SimpleShardDataTreeCohort.canCommit(SimpleShardDataTreeCohort.java:99) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.directCommit(FrontendReadWriteTransaction.java:433) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.handleCommitLocalTransaction(FrontendReadWriteTransaction.java:524) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.FrontendReadWriteTransaction.doHandleRequest(FrontendReadWriteTransaction.java:193) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.FrontendTransaction.handleRequest(FrontendTransaction.java:138) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.AbstractFrontendHistory.handleTransactionRequest(AbstractFrontendHistory.java:122) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.LeaderFrontendState$Enabled.handleTransactionRequest(LeaderFrontendState.java:137) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.Shard.handleRequest(Shard.java:638) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.Shard.handleRequestEnvelope(Shard.java:440) ~[bundleFile:?] at org.opendaylight.controller.cluster.datastore.Shard.handleNonRaftCommand(Shard.java:368) ~[bundleFile:?] at org.opendaylight.controller.cluster.raft.RaftActor.handleCommand(RaftActor.java:270) ~[bundleFile:?] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24) ~[bundleFile:?] at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20) ~[bundleFile:?] at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) ~[bundleFile:?] at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) ~[bundleFile:?] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) ~[bundleFile:?] at scala.PartialFunction$OrElse.apply(PartialFunction.scala:266) ~[bundleFile:?] at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) ~[bundleFile:?] at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) ~[bundleFile:?] at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20) ~[bundleFile:?] at scala.PartialFunction$OrElse.apply(PartialFunction.scala:266) ~[bundleFile:?] at org.opendaylight.controller.cluster.common.actor.MeteringBehavior.apply(MeteringBehavior.java:93) ~[bundleFile:?] at org.opendaylight.controller.cluster.common.actor.MeteringBehavior.apply(MeteringBehavior.java:29) ~[bundleFile:?] at scala.PartialFunction.applyOrElse(PartialFunction.scala:214) ~[bundleFile:?] at scala.PartialFunction.applyOrElse$(PartialFunction.scala:213) ~[bundleFile:?] at scala.runtime.AbstractPartialFunction.applyOrElse(AbstractPartialFunction.scala:27) ~[bundleFile:?] ... 17 more