1. VMware + Chef
A VMworld Workshop

2. Agenda 1
What is Chef? 2
Benefits of Infrastructure as Code 3
Installing Chef DK 4
Building Blocks of Chef 5
Chef Tools and Testing 6
Chef VMware Integrations 7
Hands On with vCloud Air 8
Q&A
CONFIDENTIAL

3. Introduction Curtis Stewart
Consultant for the DevOps by VMware Professional Services group
4+ years working with DevOps practices and Chef
Automation Enthusiast
Open Source Contributor
CONFIDENTIAL

4. About Us: VMware DevOps Consulting Services
Developer friendly. Enterprise ready.
Assessment
Strategy development
People, process & technology transformation for
Continuous delivery
Configuration management
Cloud operations
Security & resilience optimization
Our team of dedicated DevOps experts provide:
Build and deliver applications sooner
Fuel innovation and accelerate time to market
Transform your enterprise to support high velocity, modern application development
Deploy an agile future-ready datacenter where any app can thrive
For those not familiar with our DevOps servics – here’s a quick overview
CONFIDENTIAL

5. Chef Chef is a configuration management tool
Infrastructure as code
Composed of various components
Chef Server
chef-client
Nodes
Cookbooks
Resources
Recipes
Workstation
We’ll get into more details of the components here in a bit…
CONFIDENTIAL

6. Benefits – Infrastructure as Code
Rebuild your entire system from a code repository, data backups, and compute resources
Programmatically provision and configure components
Limits the needs for full instance backups
Provides the ability to keep base images lightweight
Executable documentation
Instance backups - Cloning instances can be very costly in terms of storage
Lightweight base images - Golden images need to be rebuilt when changes occur
Executable documentation – writing docs is a pain and most are outdated as soon as you’re “done”
CONFIDENTIAL

7. Lab 1: Quick Tour of Chef In this lab you will: This will demonstrate:
Install the Chef Development Kit
Sign up for hosted Chef
Verify connection to Chef server
This will demonstrate:
Setting up your workstation for Chef development
Overview of core components of Chef
- Because of our limited time, we’re going to jump right in to setting things up. As some of these items are being downloaded, we’ll continue to move forward and I’ll do more explaining of the components once we’re setup
CONFIDENTIAL

8. Lab 1: Quick Tour of Chef - Script
Go to this URL for a script of our workshop
Also, please clone the repo if you already have Git installed, otherwise click the “Download Zip” button and unpackage to your local machine
This also includes some helper files for moving along in the workshop
CONFIDENTIAL

9. Lab 1: Quick Tour of Chef – ChefDK Installation
Chef Downloads Page
Chef Development Kit is easiest way to get started
Most common Chef tools all bundled together
Install latest version – v0.7.0 – for your platform
Once package is downloaded, run the installer
Binaries on USB sticks were added prior to v0.7.0 release.
If network allows, go ahead and download latest, otherwise we’ll be OK with v0.6.2
In USB stick, under Chef/chefdk dmg for macs, .msi for windows
Once you’ve started the installation, please proceed to signup for hosted Chef
CONFIDENTIAL

10. Lab 1: Quick Tour of Chef – Hosted Chef Signup
Signup for a free trial of hosted Chef
Click Create New Organization
Enter full name and short name for your organization
Click Create Organization
Fill out sign up form
Create new organization
Names can be whatever you want (your name, company name, department, etc.)
Short name must be unique
If you already have an account and can’t remember your password, try ‘Forgot Password’; or use another account.
CONFIDENTIAL

11. Lab 1: Quick Tour of Chef – Chef User Key
Navigate to the Administration panel and select Users from the left menu.
Select your user, then click "Reset Key" from that Actions menu.
Click "Reset Key" to download your key.
Add downloaded user pem to:
/vmworld-chef-repo/.chef/[username].pem
Click Reset Key and save or copy your downloaded key file to the .chef directory under your vmware-chef-repo
CONFIDENTIAL

12. Lab 1: Quick Tour of Chef – Knife Setup
Copy /vmworld-chef- repo/.chef/knife.rb.example to:
./vmworld-chef-repo/.chef/knife.rb
Replace the following values with your signup information:
INSERT_CHEF_USERNAME
INSERT_CHEF_ORG_SHORTNAME
Go ahead and update your knife.rb (by copying from knife.rb.example and replacing the values)
Verify by running knife client list, should just see you a single ORG-validator key
Won’t go into much detail, but this is a special key for some bootstrapping workflows
CONFIDENTIAL

13. Lab 1: Quick Tour of Chef – Cookbook Upload
Upload your cookbook to the Chef server using knife
This will also verify that out knife.rb file has been properly updated
Let’s upload our hello_vworld cookbook to our Chef server
CONFIDENTIAL

14. Lab 1: Quick Tour of Chef – ChefDK Verification
Open Terminal (Mac) or PowerShell Console (Windows)
Execute $ chef verify
Tests the embedded ChefDK applications
CONFIDENTIAL

15. Chef Components: Cookbook
A cookbook is a collection of recipes
Cookbooks are typically designed to be composable so you can pick and choose the pieces you need
Cookbook - Container for everything you need for this specific piece of software, application, etc.
CONFIDENTIAL

16. Chef Components: Resources and Recipes
A resource is one of the most basic components of the Chef ecosystem
Examples of a resource: package, file, service
Resources are defined using attributes and actions
Recipes are a compilation of resources
Resources within a recipe are executed sequentially
CONFIDENTIAL

17. Chef – Workstation / Server / Node / Chef-Client / Knife
Your Chef workstation is where you’ll install the Chef DK to develop and test cookbooks as well as interact with the Chef server APIs
The Chef Server is a centrally managed host for storing your configuration data and node objects
All of this data is indexed and completely searchable via RESTful APIs
Chef Server available in two flavors
Hosted Chef – Centrally administered server hosted by Chef, Inc.
Open Source – Installable version to run on local infrastructure
A node is any physical, virtual, or cloud machine that is managed by the chef-client
The chef-client is an agent that’s installed on all nodes that actually performs the configurations
Knife is a command line tool that provides an interface between local chef components and the Chef server
CONFIDENTIAL

18. Chef – Tools and Testing
Test-Kitchen
Testing framework
Supports multiple drivers including vSphere, vCloud Air and many more
Berkshelf
Dependency manager for cookbooks
Supports multiple sources including Chef Supermarket, repositories, local paths
ChefSpec
Unit testing framework
Fast feedback
Serverspec
Integration testing framework
Verifies end state of system is what we expected
***Check time***
Explain test kitchen, but skim the rest to help speed things up
Let audience know we’re limited on time so we can dive too deep here
Be sure to expand a bit on each tool – use cases, benefits, features, etc.
Test-Kitchen
Allows execution of cookbooks in isolation
No Chef Server required
CONFIDENTIAL

19. VMware + Chef Integration
There are many products of VMware that Chef integrates with through various plugins
vCloud Air
knife-vcair
kitchen-vcair
VMware vSphere
knife-vsphere
chef-provisioning-vsphere
vRealize Automation
knife-vrealize
kitchen-vra
vRO Chef Server Workflows
VMware Integrated OpenStack
knife-openstack
kitchen-openstack
VMware Fusion / Workstation
Vagrant Plugins
VMware ESXi
knife-esx
CONFIDENTIAL

20. VMware + Chef: vCloud Air
knife-vcair
Create, bootstrap, and manage vCloud Air compute instances
List networks and templates
Supports OnDemand and Subscription services
kitchen-vcair
Test Kitchen driver for developing and testing cookbooks against vCloud Air
Currently supports Subscription services
CONFIDENTIAL

21. Lab 2: Chef + vCloud Air In this lab you will: This will demonstrate:
Install the knife-vcair plugin
Configure knife with your vCloud Air endpoint
Create and bootstrap a vCloud Air VM using your cookbook
This will demonstrate:
Provisioning a VM in vCloud Air through Chef tools
Bootstrapping your cookbook on your new VM
CONFIDENTIAL

22. Lab 2: Chef + vCloud Air – knife-vcair
$ chef exec gem install knife-vcair
Let’s go back to our hello_vmworld cookbook and update our default recipe by adding a few resources
CONFIDENTIAL

23. Lab 2: Chef + vCloud Air – knife.rb
knife[:vcair_username] = knife[:vcair_password] = “VMware123!”
Update /vmworld-chef-repo/.chef/knife.rb to include vcair settings
CONFIDENTIAL

24. Lab 2: Chef + vCloud Air – Create VM
$ knife vcair server create \ --ssh-password vmworld2015 \ --image "CentOS64-64BIT" \ --node-name YOURNAME-chef-node \ --customization-script bootstrap/install-linux-vcair-example.sh \ --run-list 'recipe[hello_vmworld::default]' \ --ssh-gateway \ --fog-version \ --vcair-net chef-routed-network \ --no-host-key-verify
NOW LET’S RUN IT
Once your node converges: got to to see your message
NOTE: It’s important to use this specified password because of how we’ve setup our SSH gateway (to make it easier on everyone so we didn’t have to create any private keys)
This provisioning and bootstrap will take between minutes, so let’s continue on going over other Vmware Chef integrations as our web servers come up. If you have questions or issues, raise a hand and the proctors will come around to support.
CONFIDENTIAL

25. Lab 2: Chef + vCloud Air
## This is sample kitchen-vagrant output….need to update with kitchen-vcair output ##
Should see output similar to this
CONFIDENTIAL

26. VMware + Chef: vSphere knife-vsphere chef-provisioning-vsphere
Integrates with your existing vSphere vCenter installation
List, clone, delete, snapshot VMs via knife
List datastores, resource pools and clusters.
Execute commands on running VMs
When cloning a VM, can interact with a customization specs to customize vCPUs, vRAM, IPs, hostname, etc at VM creation.
chef-provisioning-vsphere
Use machine resources to declare the topology of your app – machines and recipes to run on them
Includes Test Kitchen driver to test your cookbooks on your existing vSphere vCenter installation
***Careful with time here – skip through support and spend a couple minutes on chef-provisioning***
Knife-vsphere
Uses knife & fog to interact with the vSphere API.
Customization specs:
Requires a VM template that can be customized like Ubuntu, Red Hat or Windows.
Hint: when using CentOS, set the OS type to Red Hat not CentOS to take advantage of this!
CONFIDENTIAL

27. VMware + Chef: Chef Provisioning Example
Just wanted to spend a few extra minutes on chef-provisioning – true infrastructure as code! (VERY COOL)
Harnesses the simplicity and power of Chef to go one step further: to describe and automate the whole cluster with Chef, soup to nuts, hardware to network to software
CONFIDENTIAL

28. VMware + Chef: vRealize Suite
kitchen-vra
A driver to allow Test Kitchen to consume vRealize Automation (vRA) resources to perform testing
knife-vrealize
Knife plugin that interacts with vRA and vRealize Orchestrator (vRO)
List catalogs, servers
Display detailed server information
Create servers from catalog blueprints
Execute vRO workflows
vRO Chef Server Workflows
vRO plugin installed to vRO directly
Interfaces directly with Chef Server REST API
vRO Chef Server Workflows
These workflows will allow you to create a self service application portal using vRA/vCAC to provision VM’s and Chef to install and manage your applications
CONFIDENTIAL

29. VMware + Chef: VIO knife-openstack kitchen-openstack
Create, bootstrap, and manage OpenStack compute instances
List networks, floating IPs, security groups, images, snapshots, volumes, flavors
kitchen-openstack
Test Kitchen driver for testing your cookbooks on OpenStack cloud resources instead of your workstation
Supports various configurations including block device mappings, network settings, security groups
Knife openstack – Chef plugin that allows you to create, bootstrap, and manage OpenStack compute instances
Kitchen-openstack – Test Kitchen plugin that helps you develop and test your cookbooks on your OpenStack cloud
Importance of Test Kitchen: you’re testing on the exact same infrastructure that you will use to deploy to production
CONFIDENTIAL

30. VMware + Chef: Fusion and Workstation
Vagrant is a free and open-sorce tool used to quickly spin up & down new development environments, locally, on your workstation or laptop.
Used for development to test automation or deployments of new software, or in Chef’s case: cookbooks.
The vagrant plugin for VMware Fusion/Workstation is a pay for add on for the free vagrant
Vagrant uses local hypervisor on your workstation (VirtualBox, etc.)
Vagrant VMware Fusion/Workstation plugin allows Vagrant to use VMware products, which are faster and more stable
VMware Vagrant plug-ins are a commercial offering from HashiCorp
Use Chef to manage development machines that run under VMware Fusion or VMware Workstation.
Chef works with an open-source tool called Vagrant to quickly create and provision new development environments that run in VMs.
For VMware Fusion, use the vagrant-vmware-fusion plugin.
For VMware Workstation, use the vagrant-vmware-workstation plugin.
CONFIDENTIAL

31. VMware + Chef: ESXi knife-esx
Interacts with your solo, or non managed ESXi host.
Uses knife & fog to interact with APIs presented on the ESXi host.
Can list, delete and clone VMs on a single ESXi host.
Installs (bootstraps) Chef & makes it part of a Chef server and, additionally, can be configured to add the new node to specific environments and/or roles on your Chef server.
Community maintained & under active
***Skim through this slide***
Use Chef to automatically provision and manage virtual machines that are hosted by VMware vSphere ESXi
CONFIDENTIAL

32. Questions? …comments…and thank you!
Also, checkout our vmwaredevops/vmworld-chef-repo for more examples of VMware+Chef integrations!
CONFIDENTIAL