mirror of
https://github.com/opennetworkinglab/onos.git
synced 2025-10-16 09:51:38 +02:00
65aa829cd1
- Added steps to exercise 2 to use wireshark to capture MyTunnel packets - Various formatting fix/improvements - Fixed markdown link to use relative paths - Fixed bm-cli command not to use sudo (not needed) Change-Id: I514da99a6cfadd048294c610ba201503c0339e89 (cherry picked from commit 19ea89f8540eef0e41fdf7b4fd9a2a1950f2ade9)
381 lines
15 KiB
Markdown
381 lines
15 KiB
Markdown
# ONOS+P4 Tutorial: Exercise 1
|
|
|
|
The goal of this exercise is to introduce P4 and P4Runtime support in ONOS,
|
|
along with the tools to practically experiment with it. In this exercise we will
|
|
see how the ONOS "pipeconf" mechanism allow one to re-use existing ONOS apps to
|
|
provide basic forwarding capabilities in a pipeline-agnostic manner, i.e.
|
|
independently of the P4 program.
|
|
|
|
To run this exercise you will need multiple terminal windows (or tabs) to
|
|
operate with the CLI of Mininet, ONOS, and BMv2. We use the following convention
|
|
to distinguish between commands of different CLIs:
|
|
|
|
* Commands starting with `$` are intended to be executed in the Ubuntu terminal
|
|
prompt;
|
|
* `onos>` for commands in the ONOS CLI;
|
|
* `mininet>` for the Mininet CLI;
|
|
* `RuntimeCmd:` for the BMv2 CLI.
|
|
|
|
## Exercise steps
|
|
|
|
1. On terminal window 1, **start ONOS with a small subset of the apps**
|
|
by executing the following command:
|
|
|
|
```
|
|
$ cd $ONOS_ROOT
|
|
$ ONOS_APPS=proxyarp,hostprovider,lldpprovider ok clean
|
|
```
|
|
|
|
The `$ONOS_ROOT` environment variable points to the root ONOS directory. The
|
|
`ok` command is an alias to run ONOS locally in your dev machine. Please
|
|
note that if this the first time you run ONOS on this machine, or if you
|
|
haven't built ONOS before, it can take some time (5-10 minutes depending on
|
|
your Internet speed).
|
|
|
|
Once ONOS has started you should see log messages being print on the screen.
|
|
|
|
2. On terminal window 2, **activate the BMv2 driver and tutorial pipeconf** via
|
|
the ONOS CLI.
|
|
|
|
1. Use the following command to **access the ONOS CLI**:
|
|
|
|
```
|
|
$ onos localhost
|
|
```
|
|
|
|
You should now see the ONOS CLI command prompt. For a list of possible
|
|
commands that you can use here, type:
|
|
|
|
```
|
|
onos> help onos
|
|
```
|
|
|
|
2. Enter the following command to **activate the BMv2 driver**:
|
|
|
|
```
|
|
onos> app activate org.onosproject.drivers.bmv2
|
|
```
|
|
|
|
You should see the following message on the ONOS log:
|
|
|
|
```
|
|
Application org.onosproject.drivers.bmv2 has been activated
|
|
```
|
|
|
|
3. Enter the following command to **activate the pipeconf**:
|
|
|
|
```
|
|
onos> app activate org.onosproject.p4tutorial.pipeconf
|
|
```
|
|
|
|
You should see the following messages on the log:
|
|
|
|
```
|
|
New pipeconf registered: p4-tutorial-pipeconf
|
|
Application org.onosproject.p4tutorial.pipeconf has been activated
|
|
```
|
|
|
|
Please note the specific name used for this pipeconf `p4-tutorial-pipeconf`. We
|
|
will later use this name to tell ONOS to deploy that specific P4 program
|
|
to the switches.
|
|
|
|
4. To **verify that you have activated all the required apps**, run the
|
|
following command:
|
|
|
|
```
|
|
onos> apps -a -s
|
|
```
|
|
|
|
Make sure you see the following list of apps displayed:
|
|
|
|
```
|
|
org.onosproject.generaldeviceprovider ... General Device Provider
|
|
org.onosproject.drivers ... Default Drivers
|
|
org.onosproject.proxyarp ... Proxy ARP/NDP
|
|
org.onosproject.lldpprovider ... LLDP Link Provider
|
|
org.onosproject.protocols.grpc ... gRPC Protocol Subsystem
|
|
org.onosproject.protocols.p4runtime ... P4Runtime Protocol Subsystem
|
|
org.onosproject.p4runtime ... P4Runtime Provider
|
|
org.onosproject.drivers.p4runtime ... P4Runtime Drivers
|
|
org.onosproject.hostprovider ... Host Location Provider
|
|
org.onosproject.drivers.bmv2 ... BMv2 Drivers
|
|
org.onosproject.p4tutorial.pipeconf ... P4 Tutorial Pipeconf
|
|
```
|
|
|
|
5. (optional) **Change flow rule polling interval**. Run the following
|
|
command in the ONOS CLI:
|
|
|
|
```
|
|
onos> cfg set org.onosproject.net.flow.impl.FlowRuleManager fallbackFlowPollFrequency 5
|
|
```
|
|
|
|
This command tells ONOS to check the state of flow rules on switches
|
|
every 5 seconds (default is 30). This is used to obtain more often flow
|
|
rules stats such as byte/packet counters. It helps also resolving more
|
|
quickly issues where some flow rules are installed in the ONOS store but
|
|
not on the device (which can often happen when emulating a large number
|
|
of devices in the same VM).
|
|
|
|
3. On terminal window 3, **run Mininet to set up a topology of BMv2 devices**.
|
|
|
|
1. To **run Mininet**, use the following command:
|
|
|
|
```
|
|
$ sudo -E mn --custom $BMV2_MN_PY --switch onosbmv2,pipeconf=p4-tutorial-pipeconf --controller remote,ip=127.0.0.1
|
|
```
|
|
|
|
The `--custom` argument tells Mininet to use the `bmv2.py` custom script
|
|
to execute the BMv2 switch. The environment variable `$BMV2_MN_PY`
|
|
points to the exact location of the script (you can use the command
|
|
`echo $BMV2_MN_PY` to find out the location).
|
|
|
|
The `--switch` argument specifies the kind of switch instance we want to
|
|
run inside Mininet. In this case we are running a version of BMv2 that
|
|
also produces some configuration files used by ONOS to discover the
|
|
device (see steps below), hence the name `onosbmv2`. The `pipeconf`
|
|
sub-argument is used to tell ONOS which pipeconf to deploy on all
|
|
devices.
|
|
|
|
The `--controller` argument specifies the address of the controller,
|
|
ONOS in this case, which is running on the same machine where we are
|
|
executing Mininet.
|
|
|
|
2. A set of **files are generated in the `/tmp` folder as part of this
|
|
startup process**, to view them (on a separate terminal window):
|
|
|
|
```
|
|
$ ls /tmp/bmv2-*
|
|
```
|
|
|
|
3. You will **find ONOS netcfg JSON files in this folder** for each BMv2
|
|
switch, open this file up, for example:
|
|
|
|
```
|
|
$ cat /tmp/bmv2-s1-netcfg.json
|
|
```
|
|
|
|
It contains the configuration for (1) the gRPC server and port used by the
|
|
BMv2 switch process for the P4Runtime service, (2) the ID of pipeconf to
|
|
deploy on the device, (3) switch ports details, and other
|
|
driver-specific information.
|
|
|
|
**This file is pushed to ONOS automatically by Mininet when executing
|
|
the switch instance**. If everything went as expected, you should see
|
|
the ONOS log populating with messages like:
|
|
|
|
```
|
|
Connecting to device device:bmv2:s1 with driver bmv2
|
|
[...]
|
|
Setting pipeline config for device:bmv2:s1 to p4-tutorial-pipeconf...
|
|
[...]
|
|
Device device:bmv2:s1 connected
|
|
[...]
|
|
```
|
|
|
|
4. **Check the BMv2 switch instance log**:
|
|
|
|
```
|
|
$ less /tmp/bmv2-s1-log
|
|
```
|
|
|
|
By scrolling the BMv2 log, you should see all P4Runtime messages
|
|
received by the switch. These messages are sent by ONOS and are used to
|
|
install table entries and to read counters. You should also see many
|
|
`PACKET_IN` and `PACKET_OUT` messages corresponding to packet-in/out
|
|
processed by the switch and used for LLDP-based link discovery.
|
|
|
|
Table entry messages are generated by ONOS according to the flow rules
|
|
generated by each app and based on the P4Info associated with
|
|
the `p4-tutorial-pipeconf`.
|
|
|
|
If you prefer to watch the BMv2 log updating in real time, you can use
|
|
the following command to print on screen all new messages:
|
|
|
|
```
|
|
$ bm-log s1
|
|
```
|
|
|
|
This command will show the log of the BMv2 switch in Mininet with name
|
|
"s1".
|
|
|
|
If needed, you can run BMv2 with **debug logging** enabled by passing
|
|
the sub-argument `loglevel=debug` when starting Mininet. For example:
|
|
|
|
```
|
|
$ sudo -E mn [...] --switch onosbmv2,loglevel=debug,pipeconf=p4-tutorial-pipeconf [...]
|
|
```
|
|
|
|
Debug logging in BMv2 is useful to observe the life of a packet inside the
|
|
pipeline, e.g. showing the header fields extracted by the parser for a
|
|
specific packet, the tables used to process the packets, matching table
|
|
entries (if any), etc.
|
|
|
|
5. **Check the flow rules inserted by each app in ONOS**. In the
|
|
ONOS CLI type:
|
|
|
|
```
|
|
onos> flows -s
|
|
```
|
|
|
|
You should see 3 flow rules:
|
|
|
|
```
|
|
deviceId=device:bmv2:s1, flowRuleCount=3
|
|
ADDED, bytes=0, packets=0, table=0, priority=40000, selector=[ETH_TYPE:arp], treatment=[immediate=[OUTPUT:CONTROLLER], clearDeferred]
|
|
ADDED, bytes=0, packets=0, table=0, priority=40000, selector=[ETH_TYPE:bddp], treatment=[immediate=[OUTPUT:CONTROLLER], clearDeferred]
|
|
ADDED, bytes=0, packets=0, table=0, priority=40000, selector=[ETH_TYPE:lldp], treatment=[immediate=[OUTPUT:CONTROLLER], clearDeferred]
|
|
```
|
|
|
|
These flow rules are installed automatically for each device by the
|
|
Proxy ARP and LLDP Link Discovery apps. The first one is used to
|
|
intercept ARP requests (`selector=[ETH_TYPE:arp]`), which are sent to
|
|
the controller (`treatment=[immediate=[OUTPUT:CONTROLLER]`), who in turn
|
|
will reply with an ARP response or broadcast the requests to all hosts.
|
|
The other two flow rules are used to intercept LLDP and BBDP packets
|
|
(for the purpose of link discovery).
|
|
|
|
These flow rules appear to be installed on table "0". This is a logical
|
|
table number mapped by the pipeconf's interpreter to the P4 table named
|
|
`t_l2_fwd` in [mytunnel.p4 (line 191)](pipeconf/src/main/resources/mytunnel.p4#L191).
|
|
|
|
This mapping is defined in
|
|
[PipelineInterpreterImpl.java (line 103)](pipeconf/src/main/java/org/onosproject/p4tutorial/pipeconf/PipelineInterpreterImpl.java#L103)
|
|
|
|
6. **Compare ONOS flow rules to the table entries installed on the BMv2
|
|
switch**.
|
|
|
|
We can **use the BMv2 CLI to dump all table entries currently
|
|
installed on the switch**. On a separate terminal window type:
|
|
|
|
```
|
|
$ bm-cli s1
|
|
```
|
|
|
|
This command will start the CLI for the BMv2 switch in Mininet with name
|
|
"s1".
|
|
|
|
On the BMv2 CLI prompt, type the following command:
|
|
|
|
```
|
|
RuntimeCmd: table_dump c_ingress.t_l2_fwd
|
|
```
|
|
|
|
You should see exactly 3 entries, each one corresponding to a flow rule
|
|
in ONOS. For example, the flow rule matching on ARP packets should look
|
|
like this in the BMv2 CLI:
|
|
|
|
```
|
|
...
|
|
**********
|
|
Dumping entry 0x2000002
|
|
Match key:
|
|
* standard_metadata.ingress_port: TERNARY 0000 &&& 0000
|
|
* ethernet.dst_addr : TERNARY 000000000000 &&& 000000000000
|
|
* ethernet.src_addr : TERNARY 000000000000 &&& 000000000000
|
|
* ethernet.ether_type : TERNARY 0806 &&& ffff
|
|
Priority: 16737216
|
|
Action entry: c_ingress.send_to_cpu -
|
|
...
|
|
```
|
|
|
|
Note how the ONOS selector `[ETH_TYPE:arp]` has been translated to an
|
|
entry matching only the header field `ethernet.ether_type`, while the
|
|
bits of all other fields are set as "don't care" (mask is all zeros).
|
|
While the ONOS treatment `[OUTPUT:CONTROLLER]` has been translated to
|
|
the action `c_ingress.send_to_cpu`.
|
|
|
|
**Important:** The BMv2 CLI is a powerful tool to debug the state of a
|
|
BMv2 switch. Type `help` to show a list of possible commands. This CLI
|
|
provides also auto-completion when pressing the `tab` key.
|
|
|
|
4. It is finally time to **test connectivity between the hosts** of our Mininet
|
|
network.
|
|
|
|
1. On the Mininet prompt, **start a ping between host1 and host2**:
|
|
|
|
```
|
|
mininet> h1 ping h2
|
|
```
|
|
|
|
The **ping should NOT work**, and the reason is that we did not activate
|
|
yet any ONOS app providing connectivity between hosts.
|
|
|
|
2. While leaving the ping running on Mininet, **activate the Reactive
|
|
Forwarding app using the ONOS CLI**:
|
|
|
|
```
|
|
onos> app activate org.onosproject.fwd
|
|
```
|
|
|
|
Once activated, you should see the the ping working. Indeed, this
|
|
app installs the necessary flow rules to forward packets between
|
|
the two hosts.
|
|
|
|
3. Use steps 3.v and 3.vi to **check the new flow rules**.
|
|
|
|
You should see 3 new flow rules.
|
|
|
|
The Reactive Forwarding app works in the following way. It installs a
|
|
low priority flow rule to intercepts all IPv4 packets via a
|
|
`send_to_cpu` action (`[OUTPUT:CONTROLLER]` in ONOS). When a packet is
|
|
received by the control plane, the packet is processed by the app, which
|
|
in turn, by querying the Topology service and the Host Location service
|
|
is ONOS, computes the shortest path between the two hosts, and installs
|
|
higher priority flow rules on each hop to forward packets between the
|
|
two hosts (after having re-injected that packet in the network via a
|
|
packet-out).
|
|
|
|
5. Congratulations, you completed the first exercise of the ONOS+P4 tutorial!
|
|
|
|
## Bonus exercise
|
|
|
|
As a bonus exercise, you can re-run Mininet with more switches and hosts to see
|
|
how Exercise 1 works with a more complex topology.
|
|
|
|
1. Quit the current Mininet topology. In the Mininet CLI type:
|
|
|
|
```
|
|
mininet> exit
|
|
```
|
|
|
|
2. Start a new Mininet topology with the following command:
|
|
|
|
```
|
|
$ sudo -E mn --custom $BMV2_MN_PY --switch onosbmv2,pipeconf=p4-tutorial-pipeconf --topo tree,3 --controller remote,ip=127.0.0.1
|
|
```
|
|
|
|
By using the argument `--topo` we are telling Mininet to emulate a Tree
|
|
topology with depth 3, i.e. with 7 switches, 6 links, and 8 hosts.
|
|
|
|
**Important:** due to the limited resources of the VM, when executing many
|
|
switches in Mininet, it might happen that some flow rules are not installed
|
|
correctly on the switch (showing state `PENDING_ADD` when using ONOS command
|
|
`flows`). In this case, ONOS provides an automatic reconciliation mechanism
|
|
that tries to re-install the failed entries. To force ONOS to perform this
|
|
process more often, **make sure to apply step 2.v**.
|
|
|
|
3. If the Reactive Forwarding app is still running (see step 4.ii),
|
|
you can ping all hosts in the network using the following Mininet command:
|
|
|
|
```
|
|
mininet> pingall
|
|
```
|
|
|
|
If everything went well, ping should work for every host pair in the
|
|
network.
|
|
|
|
4. You can visualize the topology using the ONOS web UI.
|
|
|
|
Open a browser from within the tutorial VM (e.g. Firefox) to
|
|
<http://127.0.0.1:8181/onos/ui/>. When asked, use the username `onos`
|
|
and password `rocks`. You should see a nice tree topology.
|
|
|
|
While here, feel free to interact with and discover the ONOS UI. For more
|
|
information on how to use the ONOS web UI please refer to this guide:
|
|
<https://wiki.onosproject.org/x/OYMg>
|
|
|
|
To show or hide hosts, press the `H` key on your keyboard.
|
|
|
|
5. Once done, to kill ONOS, press `ctrl-c` in the ONOS log terminal window. To
|
|
quit Mininet, press `ctrl-d` in the Mininet CLI or type `exit`.
|