| Table of Contents | ||
|---|---|---|
|
Proposal 1 - Direct OpenFlowJava (OFJ) models use
Changes on bulk-o-matic enables to test using OFJ model to test performance improvements. Using those models gives should give us possibility to avoid translators and spare time and memory usage. This changes are related to the first changes proposal from this document.
Change of the service
In SalFlowService in OpenFlowPlugin (OFP) were added new RPC
rpc add-flow-direct {
description "Adding flow to openflow device.";
input {
uses tr:transaction-metadata;
leaf flow-ref {
type types:flow-ref;
}
uses node-flow-direct;
}
output {
uses tr:transaction-aware;
}
}
This RPC uses the openflow protocol specification flow.
Model comparison
OFP standard flow
AddFlowInput[
_flowRef=FlowRef [
_value=KeyedInstanceIdentifier {
targetType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.flow.inventory.rev130819.tables.table.Flow,
path= [
org.opendaylight.yang.gen.v1.urn.opendaylight.inventory.rev130819.Nodes,
org.opendaylight.yang.gen.v1.urn.opendaylight.inventory.rev130819.nodes.Node [
key=NodeKey [
_id=Uri [
_value=openflow:3
]
]
],
org.opendaylight.yang.gen.v1.urn.opendaylight.flow.inventory.rev130819.FlowCapableNode,
org.opendaylight.yang.gen.v1.urn.opendaylight.flow.inventory.rev130819.tables.Table [
key=TableKey [
_id=1
]
],
org.opendaylight.yang.gen.v1.urn.opendaylight.flow.inventory.rev130819.tables.table.Flow [
key=FlowKey [
_id=Uri [
_value=600
]
]
]
]
}
],
_flowTable=FlowTableRef [
_value=KeyedInstanceIdentifier {
targetType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.flow.inventory.rev130819.tables.Table,
path= [
org.opendaylight.yang.gen.v1.urn.opendaylight.inventory.rev130819.Nodes,
org.opendaylight.yang.gen.v1.urn.opendaylight.inventory.rev130819.nodes.Node [
key=NodeKey [
_id=Uri [
_value=openflow:3
]
]
],
org.opendaylight.yang.gen.v1.urn.opendaylight.flow.inventory.rev130819.FlowCapableNode,
org.opendaylight.yang.gen.v1.urn.opendaylight.flow.inventory.rev130819.tables.Table [
key=TableKey [
_id=1
]
]
]
}
],
_match=Match [
_ethernetMatch=EthernetMatch [
_ethernetType=EthernetType [
_type=EtherType [
_value=2048
],
augmentation= []
],
augmentation= []
],
_layer3Match=Ipv4Match [
_ipv4Source=Ipv4Prefix [
_value=0.0.0.100/32
],
augmentation= []
],
augmentation= []
],
_node=NodeRef [
_value=KeyedInstanceIdentifier {
targetType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.inventory.rev130819.nodes.Node,
path= [
org.opendaylight.yang.gen.v1.urn.opendaylight.inventory.rev130819.Nodes,
org.opendaylight.yang.gen.v1.urn.opendaylight.inventory.rev130819.nodes.Node [
key=NodeKey [
_id=Uri [
_value=openflow:3
]
]
]
]
}
],
_tableId=1,
augmentation= []
]
OFJ specification flow
AddFlowDirectInput [
_bufferId=4294967295,
_command=OFPFCADD,
_cookie=0,
_cookieMask=0,
_flags=FlowModFlags [
_oFPFFSENDFLOWREM=false,
_oFPFFCHECKOVERLAP=false,
_oFPFFRESETCOUNTS=false,
_oFPFFNOPKTCOUNTS=false,
_oFPFFNOBYTCOUNTS=false
],
_flowRef=FlowRef [
_value=KeyedInstanceIdentifier {
targetType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.flow.inventory.rev130819.tables.table.Flow,
path= [
org.opendaylight.yang.gen.v1.urn.opendaylight.inventory.rev130819.Nodes,
org.opendaylight.yang.gen.v1.urn.opendaylight.inventory.rev130819.nodes.Node [
key=NodeKey [
_id=Uri [
_value=openflow:1
]
]
],
org.opendaylight.yang.gen.v1.urn.opendaylight.flow.inventory.rev130819.FlowCapableNode,
org.opendaylight.yang.gen.v1.urn.opendaylight.flow.inventory.rev130819.tables.Table [
key=TableKey [
_id=1
]
],
org.opendaylight.yang.gen.v1.urn.opendaylight.flow.inventory.rev130819.tables.table.Flow [
key=FlowKey [
_id=Uri [
_value=600
]
]
]
]
}
],
_flowTable=FlowTableRef [
_value=KeyedInstanceIdentifier {
targetType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.flow.inventory.rev130819.tables.Table,
path= [
org.opendaylight.yang.gen.v1.urn.opendaylight.inventory.rev130819.Nodes,
org.opendaylight.yang.gen.v1.urn.opendaylight.inventory.rev130819.nodes.Node [
key=NodeKey [
_id=Uri [
_value=openflow:1
]
]
],
org.opendaylight.yang.gen.v1.urn.opendaylight.flow.inventory.rev130819.FlowCapableNode,
org.opendaylight.yang.gen.v1.urn.opendaylight.flow.inventory.rev130819.tables.Table [
key=TableKey [
_id=1
]
]
]
}
],
_hardTimeout=0,
_idleTimeout=0,
_match=Match [
_matchEntry= [
MatchEntry [
_matchEntryValue=Ipv4SrcCase [
_ipv4Src=Ipv4Src [
_ipv4Address=Ipv4Address [
_value=0.0.0.100
],
augmentation= []
],
augmentation= []
],
_oxmClass=class org.opendaylight.yang.gen.v1.urn.opendaylight.openflow.oxm.rev150225.OpenflowBasicClass,
_oxmMatchField=class org.opendaylight.yang.gen.v1.urn.opendaylight.openflow.oxm.rev150225.Ipv4Src,
_hasMask=false,
augmentation= []
],
MatchEntry [
_matchEntryValue=EthTypeCase [
_ethType=EthType [
_ethType=EtherType [
_value=2048
],
augmentation= []
],
augmentation= []
],
_oxmClass=class org.opendaylight.yang.gen.v1.urn.opendaylight.openflow.oxm.rev150225.OpenflowBasicClass,
_oxmMatchField=class org.opendaylight.yang.gen.v1.urn.opendaylight.openflow.oxm.rev150225.EthType,
_hasMask=false,
augmentation= []
]
],
_type=class org.opendaylight.yang.gen.v1.urn.opendaylight.openflow.oxm.rev150225.OxmMatchType,
augmentation= []
],
_node=NodeRef [
_value=KeyedInstanceIdentifier {
targetType=interface org.opendaylight.yang.gen.v1.urn.opendaylight.inventory.rev130819.nodes.Node,
path= [
org.opendaylight.yang.gen.v1.urn.opendaylight.inventory.rev130819.Nodes,
org.opendaylight.yang.gen.v1.urn.opendaylight.inventory.rev130819.nodes.Node [
key=NodeKey [
_id=Uri [
_value=openflow:1
]
]
]
]
}
],
_outGroup=4294967295,
_outPort=PortNumber [
_value=4294967295
],
_priority=32768,
_tableId=TableId [
_value=1
],
_version=4,
augmentation= []
]
Proposal 2 - Convert OpenflowPlugin models directly to raw bytes
Adding new OFJ serializers for OFP models is another way to avoid translators and spare time and memory usage. This changes are related to the second changes proposal from this document. This proposal is easier to adapt for downstream projects, because it do not requires change of models, but it is harder to implement than first one, because making custom serializers and deserializers for OFP models will take a lot more time, than just switching models in Proposal 1.
Change of the service
In SalFlowService in OpenFlowPlugin (OFP) were added new RPC
rpc add-flow-raw {
description "Adding flow to openflow device.";
input {
uses tr:transaction-metadata;
leaf flow-ref {
type types:flow-ref;
}
uses node-flow;
}
output {
uses tr:transaction-aware;
}
}
This RPC is exactly same as add-flow rpc, but it serves as path delimiter (if this rpc is called, we will skip translations and send openflowplugin models directly to openflowjava, so it can use our custom serializers).
Test results
Results for adding 5000 flows in 20 runs for each flow path with more advanced flow what have flow flags, instructions with actions and match with ipv4 source and destination.
Path types:
- NORMAL: Uses current path, translating openflowplugin models to openflowjava models and then to raw bytes
- DIRECT: Uses openflowjava models directly, translating them to raw bytes
- RAW: Uses openflowplugin models and translates them directly to raw bytes
Results:
| NORMAL | DIRECT | RAW | |
|---|---|---|---|
Runs | run # dur
1 2258 (warmup)
2 872
3 1163
4 610
5 561
6 590
7 643
8 395
9 591
10 294
11 731
12 630
13 666
14 549
15 575
16 643
17 634
18 537
19 572
20 658
| run # dur
1 417 (warmup)
2 764
3 541
4 556
5 618
6 647
7 520
8 622
9 549
10 626
11 518
12 523
13 559
14 592
15 585
16 531
17 576
18 428
19 519
20 925
| run # dur
1 615 (warmup)
2 614
3 598
4 683
5 546
6 622
7 591
8 639
9 525
10 592
11 650
12 616
13 487
14 245
15 336
16 552
17 501
18 546
19 615
20 388
|
Total | 11914 | 11199 | 10346 |
Average | 627 | 589 | 544 |
All numbers above are in milliseconds, and numbers in each run is duration between start of bulk-o-matic add flow job and successful flow add response from device for last flow in bulk-o-matic add flow job. So, for example when we are adding 5000 flows, we send RPC to bulk-o-matic, that will prepare these 5000 flows, and then sends each of them as RPC to SalFlowService. Last flow in this case will be flow with id 5500, because in bulk-o-matic, starting flow id is always 500. So, in this case, it will be duration between starting of sending these 5000 flows to SalFlowService and successful add flow response from device for flow with id 5500. We are getting all these informations from logs.
How to setup and run test
Testing script can be downloaded here and run on this patch from Gerrit (important, make sure you are on latest patchset).
./flowpathtest.sh -h
Usage:
-h show this message
-s skip adding flows, only process logs
-t <flow_type> flow type used for performance tests, can be NORMAL/DIRECT/RAW (NORMAL)
-p <log_path> path to karaf log file (distribution/karaf/target/assembly/data/log/karaf.log)
-f <num_flows> total number of flows added per loop (1000)
-r <num_runs> how many runs should test do, higher is better for accuracy (10)
-c <address> address of controller (127.0.0.1:8181)
-w <wait_time> how many seconds should test wait between each run (30)
Example usage:
./flowpathtest.sh -f 5000 -r 50 -w 10 -c "127.0.0.1:8181" -t "NORMAL" -p "distribution/karaf/target/assembly/data/log/karaf.log"
Command above will try to add 5000 flows via Bulk-o-matic 50 times each with NORMAL path (OpenflowPlugin models -> translators -> device) with delay of 10 seconds between each run to prevent interfering of runs between each other. Proper delay is very important, because if we do not set big enough delay, logs will be messed up and testing script will not be able to get time of last succesfully added flow to compute total duration.
This test requires to have INFO logging in FlowWriterDirectOFRpc and DEBUG logging in SalFlowServiceImpl and rest of logging on minimum, so your log4j configuration should look like this:
log4j.logger.ROOT=ERROR
log4j.logger.org.opendaylight=ERROR
log4j.logger.org.opendaylight.openflowplugin.impl.services.SalFlowServiceImpl=DEBUG
log4j.logger.org.opendaylight.openflowplugin.applications.bulk.o.matic.FlowWriterDirectOFRpc=INFO