1. Bending Ironic for Big Iron
Doug Szumski, Mark Goddard, Forest Godfrey

2. Overview OpenStack We need to support:
The foundation of Cray’s next generation system management software
We need to support:
Booting large numbers of diskless compute nodes
Cinder integration for Ironic
Flexible provisioning of diskful nodes
Bareon agent for Ironic

3. Cinder integration for Ironic
Based on upstream spec by Satoru Moriya
Configured by instance_info fields
No additional database tables or changes to any APIs
Supports:
Booting disklessly from Cinder via iSCSI (no FC)
In-band connection to the iSCSI target
Attachment of additional volumes at boot time (but not dynamically)
Extended support for Dracut based ramdisks through the generation of the PXE config file
We’ve shared our implementation here:
XC series compute blade (4 nodes)
up to 48 blades per cabinet
100s of cabinets

4. Diskless boot
1. Nova boot from CLI 2. Request IP for instance 3. Get storage port MAC address (or IP) and IQN from Ironic node driver info 4. Lookup IP address of the node on the storage network from the storage port MAC address
5. Prepare Cinder volumes and retrieve iSCSI target info 6. Patch Ironic with block device info 7. Call Ironic to begin deployment
8. Cache kernel and ramdisk, build the kernel cmdline using Jinja2
9. Configure TFTP server 10. Setup DHCP for PXE boot
11. Set boot device to PXE
12. Reboot target node 13. Target node broadcasts, DHCP server responds with an IP and the location of the bootloader 14. PXE boot the kernel and ramdisk
15. Mount iSCSI targets and pivot into the rootfs

5. Bareon (Fuel) Agent What is Bareon? Why does Cray use Bareon?
“flexible and data driven interface to perform actions which are related to operating system installation” - wiki.openstack.org/wiki/Bareon
In particular, Cray uses the Bareon agent with Ironic
Similar in concept to the Ironic Python Agent (IPA)
Why does Cray use Bareon?
Deploy baremetal nodes in a flexible, perhaps non-cloud like way
Deploying multiple images / multi-boot
Support complex partitioning schemes
Eg. Creation of shared partitions, LVM groups, consistent identification of block devices.
Rsync deploy – useful for upgrades / updates
Run arbitrary actions during or post deploy

6. Bareon agent
1. Nova boot from CLI 2. Request IP 3. Nova calls Ironic 4. Configure TFTP server
5. Cache images (deploy kernel & ramdisk, filesystem, cloud_default_deploy_config, deploy_config and driver_actions) and write provision script for Bareon agent
6. Update MAC and PXE config
7. Set boot device to PXE
8. Reboot target node 9. Target node gets IP
10. PXE boot the Bareon agent
11. Bareon agent calls back 12. SFTP across provision script and forward rsync server port by SSH 13. Trigger provisioning by SSH:
provision --data_driver ironic --deploy_driver rsync 14. Partition and clean local storage, mount partitions, rsync filesystem across, write fstab, configure bootloader and unmount partitions 15. Run driver actions over SSH, eg update BIOS, SFTP file across from Swift 16. Set boot device to local disk
17. Reboot node

7. Scaling Ironic Where are we at? Immediate focus point
Diskless boot tested on a 128 node system
Read only Cinder volume with multi-attach and overlay filesystem
Ironic multi-conductor
Immediate focus point
Deploying OpenStack with Kolla
Support scaling of OpenStack services
Where do we want to go?
100,000 (?) nodes by 2018 for Shasta

8. Thank you for listening