New P4 dev and tutorial VM with Ubuntu 18.04

Change-Id: Iacf2ca8f8467dc91e696ace5ed30412f7869110a
(cherry picked from commit 2b8bcbcd3d0e3e66ee2bc681f0f1dfbb0e2a218f)

tools/dev/bash_profile

@@ -343,5 +343,5 @@ alias atttopo='onos-netcfg $OCI $ONOS_ROOT/tools/test/topos/attmpls-cfg.json'
 alias uktopo='onos-netcfg $OCI $ONOS_ROOT/tools/test/topos/uk-cfg.json'
 
 # Mininet command that uses BMv2 instead of OVS
-alias mn-bmv2='sudo -E mn --custom $BMV2_MN_PY --switch onosbmv2 --controller remote,ip=$OC1'
+alias mn-bmv2='sudo -E mn --custom $BMV2_MN_PY --switch onosbmv2 --controller remote,ip=$OCI'
-alias mn-stratum='sudo -E mn --custom $BMV2_MN_PY --switch stratum --controller remote,ip=$OC1'
+alias mn-stratum='sudo -E mn --custom $BMV2_MN_PY --switch stratum --controller remote,ip=$OCI'

tools/dev/mininet/bmv2.py

@@ -43,6 +43,17 @@ STRATUM_BINARY = '/bazel-bin/stratum/hal/bin/bmv2/' + STRATUM_BMV2
 STRATUM_INIT_PIPELINE = '/stratum/hal/bin/bmv2/dummy.json'
 
 
+def getEnvOrDefault(env, default):
+    try:
+        return os.environ[env]
+    except KeyError:
+        return default
+
+
+ONOS_WEB_USER = getEnvOrDefault('ONOS_WEB_USER', 'onos')
+ONOS_WEB_PASS = getEnvOrDefault('ONOS_WEB_PASS', 'rocks')
+
+
 def getStratumRoot():
     if 'STRATUM_ROOT' not in os.environ:
         raise Exception("Env variable STRATUM_ROOT not set")
@@ -272,9 +283,7 @@ nodes {{
         url = 'http://%s:8181/onos/v1/network/configuration/' % controllerIP
         # Instantiate password manager for HTTP auth
         pm = urllib2.HTTPPasswordMgrWithDefaultRealm()
-        pm.add_password(None, url,
+        pm.add_password(None, url, ONOS_WEB_USER, ONOS_WEB_PASS)
-                        os.environ['ONOS_WEB_USER'],
-                        os.environ['ONOS_WEB_PASS'])
         urllib2.install_opener(urllib2.build_opener(
             urllib2.HTTPBasicAuthHandler(pm)))
         # Push config data to controller

tools/dev/p4vm/.gitignore

@@ -1,2 +1,4 @@
 .vagrant
 *.ova
+*.log
+tmp/

tools/dev/p4vm/README.md

@@ -10,18 +10,18 @@ This document contains also instructions on how to download a pre-built VM.
 
 ## Contents
 
-The VM is based on Ubuntu 16.04 (server) and contains the following software:
+The VM is based on Ubuntu 18.04 (server) and contains the following software:
 
 - ONOS
-- BMv2 (P4 software switch with P4Runtime support)
+- BMv2 simple_switch_grpc (P4 software switch with P4Runtime support)
 - p4c (P4 compiler)
 - Mininet (network emulator)
 
 ### Tutorial VM
 
 It is possible to generate a variant of the VM to be used during tutorials. This
-version of the VM comes with a desktop environment and various code
+version of the VM comes with a desktop environment, pre-built ONOS and code
-editors with P4 syntax highlighting (vim, Sublime Text, and Atom).
+editors.
 
 ## Recommended system requirements
 
@@ -43,7 +43,7 @@ pre-built VM.
 
 ## Download a pre-built VM
 
-Building the VM takes around 30-50 minutes, depending on your Internet
+Building the VM takes around 30 minutes, depending on your Internet
 connection speed. If you would rather not wait, you can use the following link
 to download an Open Virtual Appliance (OVA) package to be imported using
 VirtualBox or any other x86 virtualization system that supports this format.
@@ -51,7 +51,7 @@ VirtualBox or any other x86 virtualization system that supports this format.
 Pre-built OVA package (approx. 1.5 GB):
 <http://onlab.vicci.org/onos/onos-p4-dev.ova>
 
-The tutorial variant of the OVA package can be found here (approx 2.3 GB):
+The tutorial variant of the OVA package can be found here (approx 3 GB):
 <http://onlab.vicci.org/onos/onos-p4-tutorial.ova>
 
 ### Login credentials
@@ -66,8 +66,8 @@ Use these credentials to log in the guest Ubuntu system.
 To build the VM you will need the following software installed in your host
 machine:
 
-- [Vagrant](https://www.vagrantup.com/) (tested v2.2.2)
+- [Vagrant](https://www.vagrantup.com/) (tested v2.2.4)
-- [VirtualBox](https://www.virtualbox.org/wiki/Downloads) (tested with v5.2.22)
+- [VirtualBox](https://www.virtualbox.org/wiki/Downloads) (tested with v6.0.6)
 
 Optionally, to export the VM as an OVA package you will also need
 [sshpass](https://gist.github.com/arunoda/7790979).
@@ -78,7 +78,7 @@ The VM can be generated locally using Vagrant. In a terminal window type:
 
 ```bash
 cd $ONOS_ROOT/tools/dev/p4vm
-vagrant up
+vagrant up dev
 ```
 
 Once Vagrant has provisioned the VM, you can access to it using the `vagrant
@@ -98,39 +98,41 @@ To generate the OVA file, in a terminal window type the following commands:
 
 ```bash
 cd $ONOS_ROOT/tools/dev/p4vm
-./export-ova.sh
+./export-ova.sh dev
 ```
 
 This script will:
 
 1. provision the VM using Vagrant;
-2. remove the `vagrant` user;
+2. reduce VM disk size (by removing build artifacts);
-3. reduce VM disk size (by removing build artifacts);
+3. generate a file named `onos-p4-dev.ova`.
-4. generate a file named `onos-p4-dev.ova`.
 
-### Building the tutorial VM
+### Build the tutorial VM
 
-To build the tutorial VM, simply set the environment variable `P4_VM_TYPE` to
+To build the tutorial VM, you can use the following command:
-`tutorial` before building.
-
-For example:
 
 ```bash
-P4_VM_TYPE=tutorial vagrant up
+./export-ova.sh tutorial
 ```
 
-In alternative, to generate the OVA package:
+## Known issues
 
-```bash
+**Dev VM (Ubuntu 18.04 Server)**
-P4_VM_TYPE=tutorial ./export-ova.sh
-```
 
-### Support for Ubuntu 18.04
+* VirtualBox shared folders are not mounted on startup even if "auto-mount" flag
+  is set. To fix it:
 
-We provide **experimental** support for Ubuntu 18.04 for both the dev and the
+  ```bash
-tutorial VM. To specify the Ubuntu version to use, set the environment
+  sudo systemctl edit --full vboxadd-service
-variable `P4_VM_UBUNTU_VERSION` before building. For example:
+  ```
 
-```bash
+  and remove `systemd-timesync.service` from the `Conflicts=` line, then reboot.
-P4_VM_UBUNTU_VERSION=18.04 ./export-ova.sh
+  (<https://superuser.com/questions/1351003/after-upgrade-to-ubuntu-lts-18-virtualbox-shared-folders-marked-auto-mount-do>)
-```
+
+**Tutorial VM (Ubuntu 18.04 Server)**
+
+* GNOME desktop can be very slow when using VirtualBox. Please select "Unity"
+  as your graphical shell when logging in:
+  <https://askubuntu.com/questions/1035410/ubuntu-18-04-gnome-hangs-on-virtualbox-with-3d-acceleration-enabled>
+
+* When using Unity, the Launcher menu is empty (no apps are shown).

tools/dev/p4vm/Vagrantfile

@@ -1,27 +1,59 @@
-P4_VM_TYPE = ENV['P4_VM_TYPE'] || "dev"
+USE_STRATUM = ENV['USE_STRATUM'] || "false"
-P4_VM_UBUNTU_VERSION = ENV['P4_VM_UBUNTU_VERSION'] || "16.04"
+
+REQUIRED_PLUGINS = %w( vagrant-vbguest vagrant-reload vagrant-disksize )
 
 Vagrant.configure(2) do |config|
-  if P4_VM_TYPE == "tutorial"
+
-    config.vm.box = "lasp/ubuntu" + P4_VM_UBUNTU_VERSION + "-desktop"
+  # Install plugins if missing...
-  else
+  _retry = false
-    config.vm.box = "bento/ubuntu-" + P4_VM_UBUNTU_VERSION
+  REQUIRED_PLUGINS.each do |plugin|
-  end
+    unless Vagrant.has_plugin? plugin
-  config.vm.provider "virtualbox" do |vb|
+      system "vagrant plugin install #{plugin}"
-    vb.name = "ONOS+P4 " + P4_VM_TYPE + " " + P4_VM_UBUNTU_VERSION + " " + Time.now.strftime("(%Y-%m-%d)")
+      _retry = true
-    vb.gui = true
-    vb.cpus = P4_VM_TYPE == "tutorial" ? 4 : 2
-    vb.memory = 4096
-    if P4_VM_TYPE == "tutorial"
-        vb.customize ["storageattach", :id,
-                        "--storagectl", "IDE Controller",
-                        "--port", "0", "--device", "1",
-                        "--type", "dvddrive",
-                        "--medium", "emptydrive"]
-        vb.customize ['modifyvm', :id, '--clipboard', 'bidirectional']
     end
   end
-  config.vm.hostname = "onos-p4-" + P4_VM_TYPE
+
+  if (_retry)
+    exec "vagrant " + ARGV.join(' ')
+  end
+
+  # Common config.
+  config.vm.box = "ubuntu/bionic64"
+  config.vbguest.auto_update = true
+  config.disksize.size = '50GB'
+  config.vm.synced_folder ".", "/vagrant", disabled: false, type: "virtualbox"
   config.vm.network "private_network", :type => 'dhcp', :adapter => 2
-  config.vm.provision "shell", path: "root-bootstrap.sh", :args => P4_VM_TYPE
+
+  # Dev VM
+  config.vm.define "dev" do |d|
+    d.vm.hostname = "onos-p4-dev"
+    d.vm.provider "virtualbox" do |vb|
+      vb.name = "ONOS+P4 Dev " + Time.now.strftime("(%Y-%m-%d)")
+      vb.gui = false
+      vb.cpus = 2
+      vb.memory = 4096
+      vb.customize ["modifyvm", :id, "--vram", "32"]
+    end
+    d.vm.provision "shell", path: "root-bootstrap.sh", :args => ["dev", USE_STRATUM]
+    d.vm.provision "shell", inline: "su sdn '/vagrant/user-bootstrap.sh' dev %s" % ["dev", USE_STRATUM]
+  end
+
+  # Tutorial VM (with desktop)
+  config.vm.define "tutorial" do |d|
+    d.vm.hostname = "onos-p4-tutorial"
+    d.vm.provider "virtualbox" do |vb|
+      vb.name = "ONOS+P4 Tutorial " + Time.now.strftime("(%Y-%m-%d)")
+      vb.gui = true
+      vb.cpus = 4
+      vb.memory = 4096
+      vb.customize ['modifyvm', :id, '--clipboard', 'bidirectional']
+      vb.customize ["modifyvm", :id, "--accelerate3d", "on"]
+      vb.customize ["modifyvm", :id, "--vram", "128"]
+    end
+    d.vm.provision "shell", path: "root-bootstrap.sh", :args => ["tutorial"]
+    d.vm.provision "shell", path: "tutorial-bootstrap.sh"
+    d.vm.provision "shell", inline: "su sdn '/vagrant/user-bootstrap.sh' %s %s" % ["tutorial", USE_STRATUM]
+  end
+
+  config.vm.provision :reload
 end

tools/dev/p4vm/export-ova.sh

@@ -2,7 +2,11 @@
 
 set -xe
 
-VM_TYPE=${P4_VM_TYPE:-dev}
+VM_TYPE=${1:-dev}
+USE_STRATUM=${USE_STRATUM:-false}
+STRATUM_BMV2_TAR=${STRATUM_BMV2_TAR-unknown}
+
+DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
 
 function wait_vm_shutdown {
     set +x
@@ -24,29 +28,51 @@ function wait_for_tcp_port {
     set -x
 }
 
-# Remove references to the existing vagrant-built VM (if any).
+rm -rf ./tmp
-# We want to build a new one from scratch, not start an existing one.
+if [[ ${VM_TYPE} = "tutorial" ]]
-rm -rf .vagrant/
+then
-vagrant box update
+    bazel build //:onos
-vagrant up
+    bazel build //:onos-package-admin
+    rm -rf ~/.m2/repository/org/onosproject
+    cd ${ONOS_ROOT}
+    onos-publish -l
+    cd ${DIR}
+    mkdir -p ./tmp
+    cp ../../../bazel-bin/onos.tar.gz ./tmp/onos.tar.gz
+    cp ../../../bazel-bin/onos-admin.tar.gz ./tmp/onos-admin.tar.gz
+    cp ../mininet/bmv2.py ./tmp/bmv2.py
+    mv ~/.m2/repository/org/onosproject ./tmp/artifacts
+    if [[ ${USE_STRATUM} = true ]]
+    then
+        cp ${STRATUM_BMV2_TAR} ./tmp/stratum_bmv2.tar.gz
+    fi
+fi
 
-SSH_PORT=`vagrant port --guest 22`
+# Initial provisioning if necessary.
-VB_UUID=`cat .vagrant/machines/default/virtualbox/id`
+USE_STRATUM=${USE_STRATUM} vagrant up ${VM_TYPE}
 
-# Take snapshot before cleanup for local use
+rm -rf ./tmp
-# e.g. to avoid re-building P4 tools from scratch
+
-vboxmanage controlvm ${VB_UUID} acpipowerbutton
+SSH_PORT=`vagrant port --guest 22 ${VM_TYPE}`
-wait_vm_shutdown ${VB_UUID}
+VB_UUID=`cat .vagrant/machines/${VM_TYPE}/virtualbox/id`
-VBoxManage snapshot ${VB_UUID} take "pre-cleanup"
+
+if [[ ${VM_TYPE} = "dev" ]]
+then
+    # Take snapshot before cleanup for local use
+    # e.g. to avoid re-building P4 tools from scratch
+    vboxmanage controlvm ${VB_UUID} acpipowerbutton
+    wait_vm_shutdown ${VB_UUID}
+    VBoxManage snapshot ${VB_UUID} take "pre-cleanup"
+    vagrant up ${VM_TYPE}
+    # SSH port forwarding might change after vagrant up.
+    SSH_PORT=`vagrant port --guest 22 ${VM_TYPE}`
+    wait_for_tcp_port 127.0.0.1 ${SSH_PORT}
+fi
 
 # Cleanup
-vagrant up
-# SSH port forwarding might change after vagrant up.
-SSH_PORT=`vagrant port --guest 22`
-wait_for_tcp_port 127.0.0.1 ${SSH_PORT}
 sshpass -p 'rocks' \
     ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no \
-    -p ${SSH_PORT} sdn@127.0.0.1 "bash /vagrant/pre-ova-cleanup.sh"
+    -p ${SSH_PORT} sdn@127.0.0.1 "bash /vagrant/ova-cleanup.sh"
 sleep 5
 vboxmanage controlvm ${VB_UUID} acpipowerbutton
 wait_vm_shutdown ${VB_UUID}
@@ -57,7 +83,10 @@ vboxmanage sharedfolder remove ${VB_UUID} -name "vagrant"
 rm -f onos-p4-${VM_TYPE}.ova
 vboxmanage export ${VB_UUID} -o onos-p4-${VM_TYPE}.ova
 
-sleep 1
+if [[ ${VM_TYPE} = "dev" ]]
-vboxmanage snapshot ${VB_UUID} restore pre-cleanup
+then
-sleep 1
+    sleep 1
-vboxmanage snapshot ${VB_UUID} delete pre-cleanup
+    vboxmanage snapshot ${VB_UUID} restore pre-cleanup
+    sleep 1
+    vboxmanage snapshot ${VB_UUID} delete pre-cleanup
+fi

tools/dev/p4vm/install-p4-tools.sh

@@ -19,9 +19,9 @@
 set -e
 set -x
 
-BMV2_COMMIT="8c6f852867c4a80b7c51c23db3419e1137f5038d"
+BMV2_COMMIT="99d83115d498f0ebcc729dad4fa4c50577eccc81"
 PI_COMMIT="f9a5c6c74f7dcde382e27c63af2fe5dffc755364"
-P4C_COMMIT="74bcfa32a6c782bc9a3c4c29d5656519dee4dfdb"
+P4C_COMMIT="e2934ab32ace8a877bf2b34704950a4da69b6202"
 
 # p4c seems to break when using protobuf versions newer than 3.2.0
 PROTOBUF_VER=${PROTOBUF_VER:-3.2.0}
@@ -93,7 +93,9 @@ function do_requirements {
         wget \
         unzip
 
-    sudo -H pip install setuptools cffi ipaddr ipaddress pypcap scapy
+    sudo -H pip2.7 install setuptools cffi ipaddr ipaddress pypcap \
+        git+https://github.com/p4lang/scapy-vxlan \
+        git+https://github.com/p4lang/ptf.git
 }
 
 function do_requirements_1604 {
@@ -147,8 +149,8 @@ function do_protobuf {
     # Hack to get the -std=c++11 flag when building 3.6.1
     # https://github.com/protocolbuffers/protobuf/blob/v3.6.1/python/setup.py#L208
     export KOKORO_BUILD_NUMBER="hack"
-    sudo -E python setup.py build --cpp_implementation
+    sudo -E python2.7 setup.py build --cpp_implementation
-    sudo -E pip install .
+    sudo -E pip2.7 install .
     unset KOKORO_BUILD_NUMBER
 }
 
@@ -177,8 +179,8 @@ function do_grpc {
     sudo ldconfig
     unset LDFLAGS
 
-    sudo pip install -r requirements.txt
+    sudo pip2.7 install -r requirements.txt
-    sudo pip install .
+    sudo pip2.7 install .
 }
 
 function checkout_bmv2 {
@@ -284,17 +286,6 @@ function do_p4c {
     sudo ldconfig
 }
 
-function do_ptf {
-    cd ${BUILD_DIR}
-    if [[ ! -d ptf ]]; then
-        git clone https://github.com/p4lang/ptf.git
-    fi
-    cd ptf
-    git pull origin master
-
-    sudo python setup.py install
-}
-
 function check_commit {
     if [[ ! -e $2 ]]; then
         return 0 # true
@@ -424,6 +415,5 @@ check_and_do ${BMV2_COMMIT} bmv2 do_pi_bmv2_deps bmv2-deps
 check_and_do ${PI_COMMIT} PI do_PI PI
 check_and_do ${BMV2_COMMIT} bmv2 do_bmv2 bmv2
 check_and_do ${P4C_COMMIT} p4c do_p4c p4c
-check_and_do master ptf do_ptf ptf
 
 all_done

tools/dev/p4vm/kernel-update.sh

@@ -0,0 +1,16 @@
+#!/usr/bin/env bash
+
+set -xe
+
+# There is a known issue with some kernel versions that affects PTF tests:
+# https://github.com/jafingerhut/p4-guide/tree/master/linux-veth-bug
+
+# Run this script to update the kernel inside the VM.
+
+KERNEL_VER="4.15.0-46-generic"
+
+apt-get update
+apt-get -y --no-install-recommends install \
+    linux-image-${KERNEL_VER} linux-headers-${KERNEL_VER}
+
+apt-mark hold ${KERNEL_VER}

tools/dev/p4vm/ova-cleanup.sh

@@ -6,18 +6,18 @@ set -xe
 sudo userdel -r -f vagrant
 
 # Free space on disk
-sudo rm -rf ~/p4tools/protobuf*
+sudo rm -rf ~/p4tools
-sudo rm -rf ~/p4tools/grpc*
-sudo rm -rf ~/p4tools/bmv2
-sudo rm -rf ~/p4tools/PI
-sudo rm -rf ~/p4tools/p4c
-sudo rm -rf ~/p4tools/scapy-vxlan
-sudo rm -rf ~/p4tools/ptf
 sudo rm -rf ~/quagga
+sudo rm -rf ~/mininet
+sudo rm -rf ~/.mininet_history
+sudo rm -rf ~/.viminfo
+sudo rm -rf ~/.ssh
+sudo rm -rf ~/.cache/pip
 
 sudo apt-get clean
 sudo apt-get -y autoremove
-sudo sudo rm -rf /tmp/*
+
+sudo rm -rf /tmp/*
 
 # Zerofill virtual hd to save space when exporting
 time sudo dd if=/dev/zero of=/tmp/zero bs=1M || true

tools/dev/p4vm/root-bootstrap.sh

@@ -1,55 +1,47 @@
-#!/bin/bash
+#!/usr/bin/env bash
+
 set -xe
 
-ONOS_BRANCH_DEV="master"
-ONOS_BRANCH_TUTORIAL="onos-1.14"
-BAZEL_VER="0.22.0"
-
-# There is a known issue with some kernel versions that affects PTF tests:
-# https://github.com/jafingerhut/p4-guide/tree/master/linux-veth-bug
-KERNEL_VER="4.15.0-46-generic"
-
 VM_TYPE=${1:-dev}
 
-if [[ ${VM_TYPE} = "tutorial" ]]
+BAZEL_VER="0.24.1"
-then
+CORRETTO_URL="https://d3pxv6yz143wms.cloudfront.net/8.212.04.2/java-1.8.0-amazon-corretto-jdk_8.212.04-2_amd64.deb"
-    ONOS_BRANCH=${ONOS_BRANCH_TUTORIAL}
+
-else
+# Disable automatic updates
-    ONOS_BRANCH=${ONOS_BRANCH_DEV}
+systemctl stop apt-daily.timer
-fi
+systemctl disable apt-daily.timer
+systemctl disable apt-daily.service
+systemctl stop apt-daily-upgrade.timer
+systemctl disable apt-daily-upgrade.timer
+systemctl disable apt-daily-upgrade.service
+
+# Remove Ubuntu user
+sudo userdel -r -f ubuntu
 
 # Create user sdn
 useradd -m -d /home/sdn -s /bin/bash sdn
+usermod -aG sudo sdn
+usermod -aG vboxsf sdn
 echo "sdn:rocks" | chpasswd
 echo "sdn ALL=(ALL) NOPASSWD:ALL" > /etc/sudoers.d/99_sdn
 chmod 440 /etc/sudoers.d/99_sdn
-usermod -aG vboxsf sdn
 update-locale LC_ALL="en_US.UTF-8"
 
-if [[ ${VM_TYPE} = "tutorial" ]]
+# Update and upgrade.
-then
-    su sdn <<'EOF'
-cd /home/sdn
-bash /vagrant/tutorial-bootstrap.sh
-EOF
-fi
-
-# Java 8
-apt-get install software-properties-common -y
-add-apt-repository ppa:webupd8team/java -y
 apt-get update
-
 DEBIAN_FRONTEND=noninteractive apt-get -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" upgrade
 
-echo "oracle-java8-installer shared/accepted-oracle-license-v1-1 select true" | debconf-set-selections
+wget -O corretto.deb ${CORRETTO_URL}
+
 apt-get -y --no-install-recommends install \
+    java-common \
+    ./corretto.deb \
+    maven \
     avahi-daemon \
     bridge-utils \
     git \
     git-review \
     htop \
-    oracle-java8-installer \
-    oracle-java8-set-default \
     python2.7 \
     python2.7-dev \
     valgrind \
@@ -57,6 +49,9 @@ apt-get -y --no-install-recommends install \
     tcpdump \
     vlan \
     ntp \
+    wget \
+    curl \
+    net-tools \
     vim nano emacs \
     arping \
     gawk \
@@ -66,30 +61,32 @@ apt-get -y --no-install-recommends install \
     automake \
     autoconf \
     libtool \
-    linux-image-${KERNEL_VER} \
     isc-dhcp-server
 
-DEBIAN_FRONTEND=noninteractive apt-get -yq install wireshark
+rm -f corretto.deb
 
-# Install Bazel
+rm -f /usr/bin/python
-BAZEL_SH="bazel-${BAZEL_VER}-installer-linux-x86_64.sh"
+ln -s `which python2.7` /usr/bin/python
-wget https://github.com/bazelbuild/bazel/releases/download/${BAZEL_VER}/${BAZEL_SH}
-chmod +x ${BAZEL_SH}
-./${BAZEL_SH}
-rm -f ${BAZEL_SH}
 
 # Install pip and some python deps (others are defined in install-p4-tools.sh)
 curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
 python2.7 get-pip.py --force-reinstall
 rm -f get-pip.py
-pip install ipaddress
+pip2.7 install ipaddress
+
+if [[ ${VM_TYPE} = "dev" ]]
+then
+    # Install Bazel
+    BAZEL_SH="bazel-${BAZEL_VER}-installer-linux-x86_64.sh"
+    wget https://github.com/bazelbuild/bazel/releases/download/${BAZEL_VER}/${BAZEL_SH}
+    chmod +x ${BAZEL_SH}
+    ./${BAZEL_SH}
+    rm -f ${BAZEL_SH}
+fi
 
 tee -a /etc/ssh/sshd_config <<EOF
 
 UseDNS no
 EOF
 
-su sdn <<'EOF'
+sed -i 's/PasswordAuthentication no/PasswordAuthentication yes/g' /etc/ssh/sshd_config
-cd /home/sdn
-bash /vagrant/user-bootstrap.sh ${ONOS_BRANCH}
-EOF

tools/dev/p4vm/start_onos.sh

@@ -0,0 +1,48 @@
+#!/usr/bin/env bash
+
+ONOS_TAR=~/onos.tar.gz
+
+[ -f $ONOS_TAR ] || (echo "$ONOS_TAR not found" && exit 1)
+
+ONOS_DIR=/tmp/$(tar tf $ONOS_TAR | head -n 1 | cut -d/ -f1)
+
+# Kill any running instances
+ps -ef | grep apache.karaf.main.Main | grep -v grep | awk '{print $2}' | xargs kill -9 &>/dev/null
+
+# Do not tolerate any errors from this point onward
+set -e
+
+echo "Running clean installation..."
+# Blitz previously unrolled onos- directory
+rm -fr $ONOS_DIR
+# Unroll new image from the specified tar file
+[ -f $ONOS_TAR ] && tar zxf $ONOS_TAR -C /tmp
+
+echo "Configuring password-less CLI access..."
+# Run using the secure SSH client
+[ ! -f ~/.ssh/id_rsa.pub ] && (echo "Missing SSH public key (~/.ssh/id_rsa.pub), please generate one using ssh-keygen"; exit 1)
+$ONOS_DIR/bin/onos-user-key $(id -un) "$(cut -d\  -f2 ~/.ssh/id_rsa.pub)"
+$ONOS_DIR/bin/onos-user-password onos rocks
+
+# Create config/cluster.json (cluster metadata)
+IP=${ONOS_IP:-127.0.0.1}
+echo "Creating local cluster configs for IP $IP..."
+[ -d $ONOS_DIR/config ] || mkdir -p $ONOS_DIR/config
+cat > $ONOS_DIR/config/cluster.json <<-EOF
+{
+  "name": "default-$RANDOM",
+  "node": {
+    "id": "$IP",
+    "ip": "$IP",
+    "port": 9876
+  },
+  "clusterSecret": "$RANDOM"
+}
+EOF
+
+# Change into the ONOS home directory
+cd $ONOS_DIR
+export ONOS_HOME=$PWD
+
+# Start ONOS as a server, but include any specified options
+./bin/onos-service server "$@"

tools/dev/p4vm/tutorial-bootstrap.sh

@@ -1,105 +1,30 @@
 #!/usr/bin/env bash
 
-# Installs desktop utilities and code editors.
-# Largely inspired by the P4.org tutorial VM scripts:
-# https://github.com/p4lang/tutorials/
-
 set -xe
 
-# Remove unneeded software
+# Desktop and other tutorial tools.
-sudo apt-get remove -y --purge \
+# VirtualBox doesn't like Gnome, use Unity:
-    libreoffice* \
+# https://askubuntu.com/questions/1035410/ubuntu-18-04-gnome-hangs-on-virtualbox-with-3d-acceleration-enabled
-    account-plugin-aim \
+echo "gdm3 shared/default-x-display-manager select lightdm" | debconf-set-selections
-    account-plugin-facebook \
+echo "lightdm shared/default-x-display-manager select lightdm" | debconf-set-selections
-    account-plugin-flickr \
-    account-plugin-jabber \
-    account-plugin-salut \
-    account-plugin-yahoo \
-    aisleriot \
-    gnome-mahjongg \
-    gnome-mines \
-    gnome-sudoku \
-    landscape-client-ui-install \
-    unity-lens-music \
-    unity-lens-photos \
-    unity-lens-video \
-    unity-scope-audacious \
-    unity-scope-chromiumbookmarks \
-    unity-scope-clementine \
-    unity-scope-colourlovers \
-    unity-scope-devhelp \
-    unity-scope-firefoxbookmarks \
-    unity-scope-gmusicbrowser \
-    unity-scope-gourmet \
-    unity-scope-musicstores \
-    unity-scope-musique \
-    unity-scope-openclipart \
-    unity-scope-texdoc \
-    unity-scope-tomboy \
-    unity-scope-video-remote \
-    unity-scope-virtualbox \
-    unity-scope-zotero \
-    unity-webapps-common
 
-sudo add-apt-repository ppa:webupd8team/sublime-text-3 -y
+# Install ubuntu desktop from tasksel
-sudo add-apt-repository ppa:webupd8team/atom -y
+apt-get -y --no-install-recommends tasksel
-sudo apt-get update
+tasksel ubuntu-desktop
+# Remove gnome, install unity
+apt-get remove gdm3 ubuntu-desktop
+DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
+    ubuntu-unity-desktop lightdm \
+    gnome-panel \
+    gnome-settings-daemon \
+    metacity \
+    nautilus
 
-sudo DEBIAN_FRONTEND=noninteractive apt-get -y install wireshark
+# FIXME: app menu is empty in unity
-echo "wireshark-common wireshark-common/install-setuid boolean true" | sudo debconf-set-selections
-sudo DEBIAN_FRONTEND=noninteractive dpkg-reconfigure wireshark-common
 
-sudo apt-get -y --no-install-recommends install \
+snap install intellij-idea-community --classic
-    atom \
+# TODO: install plugins, P4 plugin and Python CE
-    sublime-text-installer \
-    vim
 
-# TODO: Disable screensaver and automatically log into the SDN user
+DEBIAN_FRONTEND=noninteractive apt-get -y install wireshark
-
+echo "wireshark-common wireshark-common/install-setuid boolean true" | debconf-set-selections
-# Sublime
+DEBIAN_FRONTEND=noninteractive dpkg-reconfigure wireshark-common
-cd /home/sdn
-mkdir -p ~/.config/sublime-text-3/Packages/
-cd .config/sublime-text-3/Packages/
-git clone https://github.com/c3m3gyanesh/p4-syntax-highlighter.git
-
-# Atom
-apm install language-p4
-
-# Adding Desktop icons
-DESKTOP=/home/sdn/Desktop
-mkdir -p ${DESKTOP}
-
-cat > ${DESKTOP}/Wireshark.desktop << EOF
-[Desktop Entry]
-Encoding=UTF-8
-Type=Application
-Name=Wireshark
-Name[en_US]=Wireshark
-Icon=wireshark
-Exec=/usr/bin/wireshark
-Comment[en_US]=
-EOF
-
-cat > ${DESKTOP}/Sublime\ Text.desktop << EOF
-[Desktop Entry]
-Encoding=UTF-8
-Type=Application
-Name=Sublime Text
-Name[en_US]=Sublime Text
-Icon=sublime-text
-Exec=/opt/sublime_text/sublime_text
-Comment[en_US]=
-EOF
-
-cat > ${DESKTOP}/Atom.desktop << EOF
-[Desktop Entry]
-Encoding=UTF-8
-Type=Application
-Name=Atom
-Name[en_US]=Atom
-Icon=atom
-Exec=/usr/bin/atom
-Comment[en_US]=
-EOF
-
-chmod +x ${DESKTOP}/*.desktop

tools/dev/p4vm/user-bootstrap.sh

@@ -1,25 +1,85 @@
 #!/bin/bash
 set -xe
 
-ONOS_BRANCH=${1:-master}
+VM_TYPE=${1:-unknown}
+USE_STRATUM=${2:-false}
+
+cd /home/sdn
 
 cp /etc/skel/.bashrc ~/
 cp /etc/skel/.profile ~/
 cp /etc/skel/.bash_logout ~/
 
-# ONOS
+#  With Ubuntu 18.04 sometimes .cache is owned by root...
-git clone https://github.com/opennetworkinglab/onos.git -b ${ONOS_BRANCH}
+mkdir -p ~/.cache
-tee -a ~/.bashrc <<EOF
+sudo chown -hR sdn:sdn ~/.cache
+
+echo 'export JAVA_HOME=$(readlink -f /usr/bin/java | sed "s:bin/java::")' >>  ~/.bash_aliases
+
+if [[ ${VM_TYPE} = "dev" ]]
+then
+    git clone https://github.com/opennetworkinglab/onos.git
+    tee -a ~/.bash_aliases <<'EOF'
 
 # ONOS
 export ONOS_ROOT=~/onos
 source ~/onos/tools/dev/bash_profile
 source ~/onos/tools/dev/p4vm/bm-commands.sh
-export BMV2_INSTALL=/usr/local
+
+export OCI=127.0.0.1
+# Uncomment if ONOS runs on the host system and we access the VM via ssh
+# export OCI=`echo $SSH_CLIENT | awk '{ print $1}'`
+
+export OC1=$OCI
+export ONOS_APPS=gui,drivers.bmv2,lldpprovider,hostprovider
 EOF
+else
+    # Tutorial. Install ONOS release.
+    cp /vagrant/tmp/onos.tar.gz ~/
+    echo 'export OCI=127.0.0.1' >> ~/.bash_aliases
+    echo 'export OC1=$OCI' >> ~/.bash_aliases
+    echo 'export ONOS_INSTANCES="$OC1"' >> ~/.bash_aliases
+    echo 'export ONOS_WEB_USER=onos' >> ~/.bash_aliases
+    echo 'export ONOS_WEB_PASS=rocks' >> ~/.bash_aliases
+    echo 'export ONOS_APPS=gui,drivers.bmv2,lldpprovider,hostprovider' >> ~/.bash_aliases
+    cp /vagrant/start_onos.sh ~/
+    chmod +x ~/start_onos.sh
+    # onos-admin commands
+    mkdir ~/onos-admin
+    tar xzf /vagrant/tmp/onos-admin.tar.gz -C onos-admin --strip-components 1
+    echo 'export PATH=$PATH:~/onos-admin' >> ~/.bash_aliases
+    # Maven artifacts
+    mkdir -p ~/.m2/repository/org/onosproject
+    cp -r /vagrant/tmp/artifacts/* ~/.m2/repository/org/onosproject/
+    # Export alias for bm-* commands
+    cp /vagrant/bm-commands.sh ~/
+    echo 'source ~/bm-commands.sh' >> ~/.bash_aliases
+    # BMv2 custom Mininet switch classes.
+    cp /vagrant/tmp/bmv2.py ~/
+    echo 'export BMV2_MN_PY=~/bmv2.py' >> ~/.bash_aliases
+    if [[ ${USE_STRATUM} = true ]]
+    then
+        # Install stratum_bmv2 binary.
+        mkdir stratum
+        tar xzf /vagrant/tmp/stratum_bmv2.tar.gz -C stratum --strip-components 1
+    fi
+fi
 
 # Build and install P4 tools
-DEBUG_FLAGS=true FAST_BUILD=true bash /vagrant/install-p4-tools.sh
+DEBUG_FLAGS=true FAST_BUILD=true USE_STRATUM=false bash /vagrant/install-p4-tools.sh
+echo 'export BMV2_INSTALL=/usr/local' >> ~/.bash_aliases
+if [[ ${USE_STRATUM} = true ]]
+then
+    # Rebuild and install PI/BMv2 with stratum config parameters. Building first
+    # without stratum parameters is useful to get P4Runtime Python binding
+    # installed as well as simple_switch_grpc.
+    rm -rf ~/p4tools/bmv2/.last_built_commit*
+    rm -rf ~/p4tools/PI/.last_built_commit*
+    # Build up until bmv2. No need to re-build p4c and others.
+    DEBUG_FLAGS=true FAST_BUILD=true USE_STRATUM=true bash /vagrant/install-p4-tools.sh bmv2
+    echo 'export STRATUM_ROOT=~/stratum' >> ~/.bash_aliases
+fi
+
 # We'll delete bmv2 sources later...
 cp ~/p4tools/bmv2/tools/veth_setup.sh ~/veth_setup.sh
 cp ~/p4tools/bmv2/tools/veth_teardown.sh ~/veth_teardown.sh
@@ -28,28 +88,25 @@ cp ~/p4tools/bmv2/tools/veth_teardown.sh ~/veth_teardown.sh
 git clone git://github.com/mininet/mininet
 sudo ~/mininet/util/install.sh -nv
 
-# Trellis - checkout routing repo
+if [[ ${VM_TYPE} = "dev" ]]
-git clone https://github.com/opennetworkinglab/routing.git
+then
-
+    # Trellis - checkout routing repo
-# Trellis - install Quagga
+    git clone https://github.com/opennetworkinglab/routing.git
-git clone -b onos-1.11 https://gerrit.opencord.org/quagga
+    # Trellis - install Quagga
-cd quagga
+    git clone -b onos-1.11 https://gerrit.opencord.org/quagga
-./bootstrap.sh
+    cd quagga
-./configure --enable-fpm --sbindir=/usr/lib/quagga enable_user=root enable_group=root
+    ./bootstrap.sh
-make
+    ./configure --enable-fpm --sbindir=/usr/lib/quagga enable_user=root enable_group=root
-sudo make install
+    make
-cd ..
+    sudo make install
-sudo ldconfig
+    cd ..
-
+    sudo ldconfig
-# Trellis - modify apparmor for the DHCP to run properly
+    # Trellis - modify apparmor for the DHCP to run properly
-sudo /etc/init.d/apparmor stop
+    sudo /etc/init.d/apparmor stop
-sudo ln -s /etc/apparmor.d/usr.sbin.dhcpd /etc/apparmor.d/disable/
+    sudo ln -s /etc/apparmor.d/usr.sbin.dhcpd /etc/apparmor.d/disable/
-sudo apparmor_parser -R /etc/apparmor.d/usr.sbin.dhcpd
+    sudo apparmor_parser -R /etc/apparmor.d/usr.sbin.dhcpd
-sudo sed -i '30i  /var/lib/dhcp{,3}/dhcpclient* lrw,' /etc/apparmor.d/sbin.dhclient
+    sudo sed -i '30i  /var/lib/dhcp{,3}/dhcpclient* lrw,' /etc/apparmor.d/sbin.dhclient
-sudo /etc/init.d/apparmor start
+    sudo /etc/init.d/apparmor start
-
+    # fabric-p4test
-# fabric-p4test
+    git clone https://github.com/opennetworkinglab/fabric-p4test.git
-git clone https://github.com/opennetworkinglab/fabric-p4test.git
+fi
-
-# Set Python path for bmv2 in fabric.p4
-echo 'export PYTHONPATH=$PYTHONPATH:$ONOS_ROOT/tools/dev/mininet' >> ~/.bashrc