1. Octavia Hands-On Lab Adam Harwell Carlos Garza Franklin Naval
German Eichberger
Michael Johnson
Al Miller
Susanne Balle
Stephen Balukoff

2. Are you ready?
Grab a USB key or download from
A computer with
8+ GB of RAM
VMware Workstation/Fusion - trial versions of both included on the USB drive
Windows users need ssh client (putty) or work in the vmware window
Copy OctaviaWorkshop from USB drive
Unzip with tar –jxvf or 7zip on windows
Start VMWare - make sure VT-X and nested Virtualization is enabled
VM also at ...
johnsom can cover this topic

3. VMWare Fusion Setup (VT-X Setting) Mac
I had to double-click the image in Finder, it didn't want to load from within VMWare's file select, and when I did so, it said "This virtual machine appears to be in use." and I had to click "Take Ownership". It then asked me if I wanted to upgrade the VM, and I said "don't upgrade". --Adam

4. Agenda Introduction and Architecture Operations Troubleshooting
How to contribute
Q&A

5. Introduction and Architecture

6. Octavia High Level Architecture

7. Operations

8. Ground rules
Slide:
To login User: ubuntu Pwd: ubuntu
sudo su - stack
cd /opt/stack/devstack/
. openrc admin admin
script /dev/null ← This will fix the devstack’s screen problem
now you are good to go

9. Create a load balancer
The vm contains two web servers on port 80, so as user admin:
nova list - take note of the two ips
LB: neutron lbaas-loadbalancer-create --name lb1 private-subnet
Check if ACTIVE with neutron lbaas-loadbalancer-list - take note of VIP
Listener: neutron lbaas-listener-create --loadbalancer lb1 --protocol HTTP --protocol- port 80 --name listener1
Pool: neutron lbaas-pool-create --lb-algorithm ROUND_ROBIN --listener listener1 -- protocol HTTP --name pool1
For each member: neutron lbaas-member-create --subnet private-subnet --address <ip> --protocol-port 80 pool1
Test: curl <vip>

10. Get information about amphora - 1
As admin:
Show all amphora vms: nova list --name amphora
Amphora info commands:
curl -k --cert /etc/octavia/certs/client.pem on lb-mgmt-net>:9443/0.5/info
curl -k --cert /etc/octavia/certs/client.pem on lb-mgmt-net>:9443/0.5/details
curl -k --cert /etc/octavia/certs/client.pem on lb-mgmt- net>:9443/0.5/listeners
curl -k --cert /etc/octavia/certs/client.pem on lb-mgmt- net>:9443/0.5/listeners/ <listener-id>

11. Get information about amphora - 2
As admin:
Log in to a VM:
ssh -i /etc/octavia/.ssh/octavia_ssh_key on lb-mgmt-net> -- good for debugging amphora changes
Check on agent: sudo service amphora-agent status
Check on haproxy: sudo service haproxy-<listener-id> status

12. Force/Simulate a fail over -1
Set up health monitoring:
Edit /etc/octavia/octavia.conf
Uncomment controller_ip_port_list and set to controller_ip_port_list = :5555 (this is some voodoo due to some bug in devstack we still need to fix)
Restart o-cw:
./rejoin-stack.sh
find o-cw screen; ctrl-c; cursor up; return
find o-hm screen; ctrl-c; cursor up; return
ctrl-a d
Create another loadbalancer (see the other slide) – make sure to use different names, e.g. lb2, listener2, etc.

13. Force/Simulate a fail over - 2
Perform a failover:
ssh into the amphora: ssh -i /etc/octavia/.ssh/octavia_ssh_key on lb- mgmt-net>
Stop the amphora agent: sudo service amphora-agent stop
Observe the fail over:
in one window curl the vip: while :; do curl <vip>; done;
in the other one monitor nova (repeat until ip changes): nova list --name amphora

14. Troubleshooting

15. Increase Log Level Edit /etc/octavia/octavia.conf
Uncomment debug = False and replace False with true. Alternatively you can also uncomment verbose=False and replace with True if you only require INFO level logging. We recommend DEBUG in most cases.
Restart o-cw, o-api, o-*:
./rejoin-stack.sh
find o-* screen; ctrl-c; cursor up; return
repeat until all services you like to debug are restarted
ctrl-a d

16. Log files on the host Review /opt/stack/logs
ls /opt/stack/logs/o-*.log
o-api.log o-cw.log o-hk.log o-hm.log
o-api.log - logs the api server. Useful to see if requests made it to the system and if they actually got executed
o-cw.log - logs the controller worker. This does most of the work and this logs will usually aid in debugging problems with nova, neutron, etc.
o-hk.log - logs the housekeeping manager. Here we do the deletion of amphora marked for deletion, certificate rotation (the ones on the amphora), etc.
o-hm.log - logs the information about the healthmanager and heartbeats the amphorae send; this is good for failover debugging

17. Log files on the amphora
Ssh into the amphora
cd /var/log/upstart/
sudo sudo less amphora-agent.log
Useful to review if the amphora is sending heartbeats and if the API commands were successful; essential for contoller-worker <- -> amphora communication debugging

18. Octavia Database mysql use octavia show tables;
There is some interesting stuff but let’s just look at
Amphora table - information about all the amphora in the system
Load_balancer, Listener, Member, Pool, Health_monitor - what you expect
amphora_health - this has a timestamp of the last time we received a heartbeat from the amphora

19. How to contribute? For more information:
Our next presentation “LBaaS V2 - Liberty and beyond” – 2:40 Thursday
Weekly meetings Wednesdays, UTC
Freenode IRC: #openstack-lbaas

20. Q&A