1. Deploy opnfv ARNO WITH FUEL IN virtual environment
Deploy opnfv ARNO WITH FUEL IN virtual environment
Version 3

2. CONTENT CONFIG FILES DEPLOYMENT VERIFICATION CHALLENGES
CONTENT
CONFIG FILES
DEPLOYMENT
VERIFICATION
CHALLENGES
MANUAL iso BUILD
INVESTIGATION TOPICS
Version 3

3.
Config file templates
For Bare Metal and Virtual deployment, template files are available:
fuel/deploy/templates/virtual_environment/conf/<dea.yaml,dha.yaml>
fuel/deploy/templates/hardware_environment/conf/<ericsson_montreal_lab, linux_foundation_lab, opnfv_box>/<dea.yaml,dha.yaml>
VM definition files:
fuel/deploy/templates/virtual_environment/vms/<controller.xml, compute.xml, fuel.xml>
fuel/deploy/templates/hardware_environment/vms/fuel.xml
Network config for Virtual deployment:
fuel/deploy/templates/virtual_environment/networks/fuel<x>.xml
Version 3

4. Config files content - DEA
Config files content - DEA
There are 2 config files wired out from fuel: DEA, DHA
Deployment Environment Adapter: dea.yaml
mode: ha
net_segment_type: gre
wanted_release: Juno on Ubuntu
Node roles
Fuel networks, ips, interfaces, bridge configs
OS components settings
...
Version 3

5. Config files content - DHA
Config files content - DHA
There are 2 config files wired out from fuel: DEA, DHA
Deployment Hardware Adapter: dha.yaml
adapter: libvirt
nodes:
- id: 1
libvirtName: controller1
libvirtTemplate: templates/virtual_environment/vms/controller.xml
- id: 5
libvirtName: fuel-master
libvirtTemplate: templates/virtual_environment/vms/fuel.xml
isFuel: yes
username: root
password: r00tme
virtNetConfDir: templates/virtual_environment/networks
disks:
fuel: 50G
controller: 50G
compute: 50G
Version 3

6. CONTENT CONFIG FILES DEPLOYMENT VERIFICATION CHALLENGES
CONTENT
CONFIG FILES
DEPLOYMENT
VERIFICATION
CHALLENGES
MANUAL ISO BUILD
INVESTIGATION TOPICS
Version 3

7. Auto deployer fuel/ci/deploy.sh Typically:
Auto deployer
fuel/ci/deploy.sh
Typically:
sudo ./deploy.sh -iso opnfv.iso -dea dea.yaml -dha dha.yaml -s /mnt/images -p PLUGIN
-> deploy OPNFV Cloud env with Fuel
-iso -> OPNFV iso downloaded or built from source
-p -> plugins for FUEL, like ODL puppet files to install ODL on nodes
Not ODL itself
Default ODL rpm for ARNO SR1 is integrated in OPNFV iso
Can be downloaded or built from source if different version required
-s -> storage location for generated VMs, user defined
-nf -> no fuel (redeployment)
-nh -> no health check
Version 3

8.
PREPARE ARNO ISO
DOWNLOAD ARNO ISO FILE:
It contains a Centos FUEL image containing all the requirements for OPNFV deployments (e.g. OpenStack Juno, ODL Lithium, Fuel plugins
Version 3

9. USE CORRECT NTP SETTINGS IN DEA.YAML
USE CORRECT NTP SETTINGS IN DEA.YAML
Section „fuel”:
NTP1:
NTP2:
NTP3:
Section „ntp_list”:
value:
Unless using a correct value, deployment will fail:
FUEL GUI ERROR:
Deployment has failed. Method granular_deploy. Failed to execute hook 'puppet' Puppet run failed. Check puppet logs for details
---
priority: 200
fail_on_error: true
type: puppet
uids:
- '3'
parameters:
puppet_modules: "/etc/puppet/modules"
puppet_manifest: "/etc/puppet/modules/osnailyfacter/modular/ntp/ntp-check.pp"
timeout: 600
cwd: "/" .
Inspect Astute logs for the details
Version 3

10. OPTIONAL ODL Build step
OPTIONAL ODL Build step
One should either use the ODL integrated in ARNO SR1 (iso downloaded or built manually in previour step already contains it, no manual step required) or build rpm from source, and add it to the deploy script as argument.
BUILD: (readme:
git clone
pip install fuel-plugin-builder
gem install fpm
fpb --build fuel-plugin-opendaylight/
INCLUDE TO DEPLOYMENT (by using argument for deploy.sh on next slide):
-p <fuel-plugin-opendaylight-rmp-dir>
Version 3

11. Deployment Typical script parameters:
Deployment
Typical script parameters:
sudo ~/opnfv/fuel/fuel/ci/deploy.sh \
-iso ~/opnfv/fuel/fuel/build/release/opnfv-P0000.iso \
-dea ~/opnfv/fuel/fuel/deploy/templates/virtual_environment/conf/dea.yaml \
-dha ~/opnfv/fuel/fuel/deploy/templates/virtual_environment/conf/dha.yaml \
-s ~/opnfv/fuel/fuel/deploy/images \
-nh # Would fail at Live-Migraition test phase because of a known ODL bug anyway, let’s test it afterwards
Console output:
Fuel Plugins Directory not specified! No external plugins will be installed!
Using OPNFV ISO file: ~/opnfv/fuel/fuel/build/release/opnfv-P0000.iso
Using image directory: ~/opnfv/fuel/fuel/deploy/images
...
Cleanup, Create Networks, power on VMs
Install Fuel Master
Insert ISO ~/opnfv/fuel/fuel/deploy/fueltmp/deploy-opnfv-P0000.iso
Inject our own astute.yaml settings
Let the Fuel deployment continue
Installing Fuel Plugins
Version 3

12.
Manual cleanup
May be useful if deployment script was stopped unexpectedly, and system ends in undefined state
sudo ~/opnfv/fuel/fuel/ci/deploy.sh -co \
-dha ../deploy/templates/virtual_environment/conf/dha.yaml
Version 3

13. CONTENT CONFIG FILES DEPLOYMENT VERIFICATION CHALLENGES
CONTENT
CONFIG FILES
DEPLOYMENT
VERIFICATION
CHALLENGES
MANUAL ISO BUILD
INVESTIGATION TOPICS
Version 3

14. Deploy verification 1 2015-10-22 Version 3
Verify FUEL deployment on GUI:
From Jumphost (Fuel Host):
$ firefox & (admin/admin)
Verify Networks
Health Check
Note: Live-Migraition test will fail (Bug in ODL currently), you can skip this test in the list if you choose to not see the error message, simply uncheck it in the list
Check fuel version and plugins:
ssh (pwd: r00tme)
~]# fuel plugins
fuel --versionid | name | version | package_version
---| | |
1 | opendaylight | | 2.0.0
~]# fuel --version
6.1.0
Verify OpenStack GUI:
$ firefox & (admin/admin)
Verify OpenDaylight:
Is GUI accessible - Point your browser to the following URL: :8181/index.html (admin/admin)
Version 3

15. Deploy verification 2 2015-10-22 Version 3 SSH to Controller:
nova-manage service list
Binary Host Zone Status State Updated_At
nova-consoleauth node-2.opnfvericsson.ca internal enabled :-) :08:20
nova-scheduler node-2.opnfvericsson.ca internal enabled :-) :08:21
nova-conductor node-2.opnfvericsson.ca internal enabled :-) :08:21
nova-cert node-2.opnfvericsson.ca internal enabled :-) :07:43
nova-compute node-1.opnfvericsson.ca nova enabled :-) :07:29
nova-compute node-3.opnfvericsson.ca nova enabled :-) :07:36
nova-compute node-4.opnfvericsson.ca nova enabled :-) :08:11
neutron agent-list
| id | agent_type | host | alive | admin_state_up | binary |
| 11709a29-24f8-46a2-b196-8b5545a6e0a5 | Metadata agent | node-2.opnfvericsson.ca | :-) | True | neutron-metadata-agent |
| 2cb0a373-c08a-4ea2-a819-13d7bf892f2e | L3 agent | node-2.opnfvericsson.ca | :-) | True | neutron-l3-agent |
| 42d08a62-83b5-4f12-a24c-8d375cde63f4 | DHCP agent | node-2.opnfvericsson.ca | :-) | True | neutron-dhcp-agent |
Version 3

16. CONTENT CONFIG FILES DEPLOYMENT VERIFICATION CHALLENGES
CONTENT
CONFIG FILES
DEPLOYMENT
VERIFICATION
CHALLENGES
MANUAL ISO BUILD
INVESTIGATION TOPICS
Version 3

17.
Challenges 1
sudo ./deploy.sh ...
error: Failed to start domain controller1
error: unsupported configuration: guest and host CPU are not compatible: Host CPU does not provide required features: erms, smep, fsgsbase, rdrand, f16c, avx, osxsave, xsave, tsc-deadline, x2apic
I am using Intel(R) Xeon(R) CPU 2.40GHz – Westmere architecture
I had to change arcthitecture in VM definition XMLs:
cd ../deploy/templates/virtual_environment/vms
sed -i 's/SandyBridge/Westmere/g' compute.xml controller.xml fuel.xml
cd -
I had to comment out flags not supported by my CPU in VM definition XMLs:
for flag in erms smep fsgsbase rdrand f16c avx osxsave xsave tsc-deadline x2apic;do
sed -i "/'${flag}'/s/^/<\!-- /" compute.xml controller.xml fuel.xml &&
sed -i "/'${flag}'/s/$/ -->/" compute.xml controller.xml fuel.xml
done
cd –
SOLVED
Version 3

18.
Challenges 2
Deployment seems to be pretty error-prone, if you CTRL+C, and see error „
OSError: [Errno 38] Function not implemented: '~/opnfv/genesis/fuel/deploy/fueltmp/origiso/boot.catalog‚”, iso is probably still mounted, check it with:
cat /etc/mtab
fuseiso ~/opnfv/genesis/fuel/deploy/fueltmp/origiso fuse.fuseiso rw,nosuid,nodev 0 0
Unmount it with:
sudo fusermount -u ~/opnfv/genesis/fuel/deploy/fueltmp/origiso
SOLVED
Version 3

19. Challenges 3 BUILD fails with build.sh:
Challenges 3
BUILD fails with build.sh:
Installing Fuel Plugins
Fofakeroot /opt/fuel-createmirror-6.1/fuel-createmirror
* INFO: Started /opt/fuel-createmirror-6.1/deb-mirror /opt/fuel-createmirror-6.1/config/mos-ubuntu-updatesonly.cfg
* FATAL: Upstream mirror mirror.fuel-infra.org does not support rsync protocol, aborting
* INFO: Started /opt/fuel-createmirror-6.1/deb-mirror /opt/fuel-createmirror-6.1/config/ubuntu.cfg
* FATAL: Upstream mirror archive.ubuntu.com does not support rsync protocol, aborting
* FATAL: Creation of MOS mirror FAILED, check logs at /var/log/mirror-sync
* FATAL: Creation of Ubuntu mirror FAILED, check logs at /var/log/mirror-sync
make[2]: Leaving directory `~/opnfv/genesis/fuel/build/f_isoroot/f_repobuild'
make[2]: *** [.nailgun] Error 1
semop(1): encountered an error: Invalid argument
make[1]: *** [f_repobuild] Error 2
make: *** [f_isoroot] Error 2
make[1]: Leaving directory `~/opnfv/genesis/fuel/build/f_isoroot'
make: *** [all] Error 2
Build: make all failed, exiting ...
Exiting ...
und plugin /root/plugins/opendaylight noarch.rpm, installing ...
Eject ISO file ~/opnfv/genesis/fuel/deploy/fueltmp/deploy-opnfv-P0000.iso
Remove ISO directory ~/opnfv/genesis/fuel/deploy/fueltmp
Deleting directory ~/opnfv/genesis/fuel/deploy/fueltmp
Installation of Fuel Plugin /root/plugins/opendaylight noarch.rpm failed: Traceback (most recent call last):
...
    plugin_tar = tarfile.open(plugin_path, 'r')
  File "/usr/lib64/python2.6/tarfile.py", line 1657, in open
    raise ReadError("file could not be opened successfully")
tarfile.ReadError: file could not be opened successfully
DISAPPEARED AFTER CLEANUP AND MAKE CLEAN
Version 3

20. Challenges 4 TRY TO REDEPLOY WITH „-nf” FLAG (NO FUEL): 2015-10-22
Trying to SSH into Fuel VM sleeping 10 seconds
Wait until Fuel menu is up
Inject our own astute.yaml settings
Modifying Fuel astute
Let the Fuel deployment continue
Found FUEL menu as PID 1292, now killing it
Wait until installation complete
Waiting for one minute for Fuel to stabilize
Delete deprecated fuel client config /etc/fuel/client/config.yaml
Installing Fuel Plugins
Found plugin /root/plugins/opendaylight noarch.rpm, installing ...
Eject ISO file ~/opnfv/fuel/fuel/deploy/fueltmp/deploy-opnfv-P0000.iso
Remove ISO directory ~/opnfv/fuel/fuel/deploy/fueltmp
Deleting directory ~/opnfv/fuel/fuel/deploy/fueltmp
Installation of Fuel Plugin /root/plugins/opendaylight noarch.rpm failed:
[Errno 14] PYCURL ERROR 6 - "Couldn't resolve host 'mirror.fuel-infra.org'"
Trying other mirror.
[Errno 14] PYCURL ERROR 6 - "Couldn't resolve host 'mirror.fuel-infra.org'"
TRY TO REDEPLOY WITH „-nf” FLAG (NO FUEL):
Verification of fuel plugin install looks to be failed but in reality it is successfully installed, verify it:
ssh (pwd: r00tme)
~]# fuel plugins
Fuel will not de redeployed with this flag, so the deployment can be continued.
A fix for this is already under review.
Version 3

21. Challenges 5 HANGS... NEED TO REDEPLOY WITH FUEL (WITHOUT „-nf” FLAG)
Challenges 5
~/opnfv/fuel/fuel/ci$ sudo ./deploy.sh -iso /home/egbohal/opnfv/fuel/fuel/build/release/opnfv-P0000.iso -dea /home/egbohal/opnfv/fuel/fuel/deploy/templates/virtual_environment/conf/dea.yaml -dha /home/egbohal/opnfv/fuel/fuel/deploy/templates/virtual_environment/conf/dha.yaml -s /home/egbohal/opnfv/fuel/fuel/deploy/images -nf
Namespace(cleanup=False, cleanup_only=False, dea_file='/home/egbohal/opnfv/fuel/fuel/deploy/templates/virtual_environment/conf/dea.yaml', dha_file='/home/egbohal/opnfv/fuel/fuel/deploy/templates/virtual_environment/conf/dha.yaml', fuel_only=False, fuel_plugins_dir=None, iso_file='/home/egbohal/opnfv/fuel/fuel/build/release/opnfv-P0000.iso', no_fuel=True, no_health_check=False, pxe_bridge='pxebr', storage_dir='/home/egbohal/opnfv/fuel/fuel/deploy/images')
Fuel Plugins Directory not specified! No external plugins will be installed!
Power OFF Node controller1
Power OFF Node compute1
Power OFF Node compute2
Power OFF Node compute3
Set boot order ['network', 'hd'] on Node controller1
Set boot order ['network', 'hd'] on Node compute1
Set boot order ['network', 'hd'] on Node compute2
Set boot order ['network', 'hd'] on Node compute3
Power ON Node controller1
Power ON Node compute1
Power ON Node compute2
Power ON Node compute3
Check prerequisites
Check supported release: Juno on Ubuntu
Check previous installation
Deleting environment 1
HANGS... NEED TO REDEPLOY WITH FUEL (WITHOUT „-nf” FLAG)
Version 3

22.
Challenges 6
If deployment hangs for a while, you can ssh to fuel and verify which state the deployment currently is to know where to dig in:
~]# watch -n 1 fuel task
id | status | name | cluster | progress | uuid
---| | | | |
1 | ready | check_networks | | | e e5b9-4b05-9eb c0ad0
4 | running | deployment | | | 119ff982-48f1-40cd-91a5-63f5ce02ae7d
2 | running | deploy | | | 4a37d b2-b056-afc87dc8d022
3 | ready | provision | | | 5ec218aa-3e8d-4dae-aa46-de8fcaef5e90
Version 3

23. CONTENT CONFIG FILES DEPLOYMENT VERIFICATION CHALLENGES
CONTENT
CONFIG FILES
DEPLOYMENT
VERIFICATION
CHALLENGES
MANUAL iso BUILD
INVESTIGATION TOPICS
Version 3

24. prerequisites – ssh key
prerequisites – ssh key
UPLOAD SSH KEYS TO:
gerrit.opnfv.org
github.com
Version 3

25. prerequisites - FW rules
prerequisites - FW rules
ALLOW FIREWALL for HTTP, HTTPS, SSH, GIT(9418), RSYNC(873) protocols to:
gerrit.opnfv.org, git.opnfv.org
pypi.python.org
get.docker.com
p80.pool.sks-keyservers.net
github.com
mirror.fuel-infra.org
archive.ubuntu.com
Version 3

26. Prerequisites - pkg dependencies
Prerequisites - pkg dependencies
sudo apt-get install -y libvirt-bin qemu-kvm python-pip fuseiso mkisofs
sudo apt-get install -y python-dev libz-dev libxml2-dev libxslt-dev
sudo pip install pyyaml netaddr paramiko lxml scp pycrypto ecdsa
curl -sSL | sudo apt-key add -
curl -sSL | sh
sudo usermod -aG docker <user>
logout & login again
Version 3

27.
PREPARE ARNO ISO
BUILD YOUR OWN ISO:
git clone
cd fuel/fuel/build
make all
It will generate a Centos FUEL image containing all the requirements for OPNFV deployments (e.g. OpenStack Juno, ODL Lithium, Fuel plugins):
download OpenStack and other components
configure docker containers
Prepare bootable iso img
Patch iso with fuel plugins
End of console output:
Making ISO...
...
Product ISO is ~/opnfv/fuel/fuel/build/release/opnfv-P0000.iso
Version 3

28. CONTENT CONFIG FILES DEPLOYMENT VERIFICATION CHALLENGES
CONTENT
CONFIG FILES
DEPLOYMENT
VERIFICATION
CHALLENGES
MANUAL ISO BUILD
INVESTIGATION TOPICS
Version 3

29. Proposed investigation item: Patch for Ovs puppet plugin
Proposed investigation item: Patch for Ovs puppet plugin
From Guo Ruijing
Install OVS from specified URL in ODL plugin
Uses pre_build_hook and puppet
Uploaded Sep 10, :17 AM
Not investigated, To Be Checked!
Version 3

30. Proposed investigation item: Alternative deployment option
Proposed investigation item: Alternative deployment option
devel pipeline/sandbox
Jonas Bjurel
Devel pipeline homepage:
Wiki:
OPNFV master and stable branches are currently in a little flux, but it should soon stabilize.
Version 3

31.
descriptions
genesis/fuel/ci/README: with examples for BM/VIRTUAL/FUEL_ONLY DEPLOYMENT
genesis/fuel/deploy/README: same as above, more verbose, many template configs
genesis/fuel/build/README: build container and .iso
Genesis doc:
Genesis doc:
Docker doc:
Version 3

32.
Version 3