For Wolverhampton Linux User Group By Adam Sweet What On Earth Is IPMI? For Wolverhampton Linux User Group By Adam Sweet 04/02/2018

What On Earth Is IPMI? What is IPMI? What Can I Do With It? Versions of IPMI Configuring IPMI Basic IPMI Commands Controlling Remote Machines Other Useful Commands Using SOL to Get a Remote Console IPMIView Screenshots Further Configuration Negatives? Links

What is IPMI? IPMI is: Intelligent Platform Management Interface A specification created by an Intel led committee to create common interface for out-of-band hardware management Independent of the Operating System Works over UDP port 623 using RMCP (Remote Management Control Protocol), or locally via kernel drivers Only available on server hardware which supports it and is similar to LOM (Lights Out Management) and ILO Has CLI tools which is similar to the Cisco IOS CLI and some (normally proprietary) GUI tools IPMI requires a hardware device called a BMC (Baseboard Management Controller), either built-in to the motherboard or as a daughter card

What Can I Do With It? The primary benefits of IPMI: View server chassis and motherboard sensor output remotely, such as power status, chassis intrusion detection, fan speeds and motherboard temperatures Ability to remotely power on, power off, reboot the server and flash the identification light whether the OS is running or not Use SNMP to send Platform Event Traps Ability to run a console on a serial port and redirect the console over a network interface, which with BIOS and bootloader console redirection, lets you view the BIOS, bootloader, the bootup and shutdown procedures and console output remotely even if the OS dies. This is called Serial Over Lan (SOL) The BMC works regardless of whether the OS is operational or whether the machine is powered on, so long as there is power available

What Can I Do With It? The point of IPMI for me is: I can reboot and power machines on or off remotely when the OS hangs, without paying for an IP addressable Power Distribution Unit (PDU) A PDU only lets you choose whether to supply power to a machine or not, if you are supplying power to a machine but it isn't powered on, there is no way to turn it on I can get a serial console remotely in situations where SSH or telnet aren't available, such as managing the BIOS and the bootloader, without paying for an IP addressable KVM I can get remote serial console access when the OS has hung or kernel panicked, when a machine won't boot or shutdown, or when it goes into interactive fsck or maintenance mode at boot- time, without paying for an IP addressable KVM IP PDUs and IP KVMs are very expensive for a small business

Versions of IPMI There are currently 3 IPMI revisions: IPMI v1.0 - Autonomous access, logging and control. IPMI messaging command sets, sensor data records and event messages. Access through system interfaces like memory mapped IO, I2C bus etc IPMI v1.5 - Ability to send IPMI messages to BMC over LAN, LAN alerting. SOL using non-standard proprietary methods IPMI v2.0 - Serial Over LAN standardised, enabling console redirection, access control, enhanced authentication, packet encryption using RCMP+ instead of RCMP, SMbus interface SOL allows you to manage the server as though it were local when the OS locks up and SSH or telnet access are not available. IPMI v1.5 still allows to you to remotely power the system on and off and view sensor output

Configuring IPMI Of course you need a BMC first... You need to configure your BMC with it's own unique IP address, this can be in the BIOS, using the manufacturers tools, or in the OS after installing the right tool On your server you need to install OpenIPMI (aka openipmi in Debian derivs) and ipmitool (aka OpenIPMI-tools in RH/Fedora) On your network management node you need to install ipmitool (aka OpenIPMI-tools) You need to tell your OS to load the drivers, either manually, using /etc/modules (Debian derivs), or chkconfig ipmi on; service ipmi start (in RH/Fedora)

Configuring IPMI Getting a SOL console: Tell your BIOS to redirect a 19.2Kb vt100 console over com1 Some BMCs can do different speeds, but 19.2Kb is a default Tell grub to run a similar console over ttyS1 (aka com1): kernel /vmlinuz-2.6.22-14-amd64-server root=/dev/sda2 ro console=tty0 console=ttyS1,19200n8r Disable rhgb, splash image and hidden menu in Grub Tell init to do the same: (Deb) S1:2345:respawn:/sbin/getty -L ttyS1 19200 vt100 (RH) S1:2345:respawn:/sbin/agetty -h ttyS1 19200 vt100 Fedora seems to the above step for me recently Reboot and you're good to go

Basic IPMI Commands ipmitool syntax is similar to Cisco IOS For each command sent without options, it will provide you with a list of subcommands Basic command, run locally, as root: ipmitool <command> Such as: ipmitool chassis power status Chassis Power is on Try it without any commands at first to get a list of options and then build up your subcommands as you go

Controlling Remote Machines Using IPMI to Control Remote Machines Example command: ipmitool -I lan -H 192.168.10.15 -U admin -a chassis power status -I lan means we're talking to a BMC lan interface -H is the host IP or resolvable hostname -U is the BMC username -a means we will supply the password at a prompt the last part is the command itself

Other Useful Commands chassis power off (an immediate hard power off) chassis power on chassis power reboot (combination of off and then on) chassis power reset (like pressing reset switch) chassis status (sensor readings etc) lan print 1 (prints the lan settings) lan set 1 <subcommand> (configures lan settings) BMCs have more than one channel, 1 is normally the LAN interface

Further Configuration Some Supermicro BMCs arp gratuitously which degrades network performance, so turn it off: ipmitool lan set 1 arp generate off Set the BMC’s netmask as well as it’s default and backup gateways: ipmitool lan set 1 netmask 255.255.255.0 ipmitool lan set 1 defgw ipaddr 192.168.1.1 ipmitool lan set 1 bakgw ipaddr 192.168.1.254 You can also set the gateway MAC addresses if you want to. Set it’s SNMP community name so that we can send SNMP traps: ipmitool lan set 1 snmp <community name> Supermicro also provide a Graceful Shutdown daemon for Linux, but I haven't played with that

Using SOL to Get a Remote Console All the magic is in the BIOS and the BMC, it should just work if you did the BIOS, bootloader and init configuration You just need an IPMI SOL application, like Supermicro's IPMIView, I haven't used or heard of any others. IPMIView is a Java Swing app for Windows and Linux IPMIView Screenshots (no window borders for single window screenshots under Compiz-Fusion apparently)...

Supermicro IPMIView Screenshots... Search for Devices

Supermicro IPMIView Screenshots... Login

Supermicro IPMIView Screenshots... Sensors

Supermicro IPMIView Screenshots... Device Control

Supermicro IPMIView Screenshots... LAN and SNMP

Supermicro IPMIView Screenshots... User Config

Supermicro IPMIView Screenshots... SOL Console

Supermicro IPMIView Screenshots... Virtual Media

Negatives? Not always 100% reliable, I've had BMCs stop responding to LAN traffic and a BMC cold restart didn't work, I had to remove the power from the server and then bring it back up again SOL connections seem to lose characters or get some in the wrong order if you type too fast Both of the above might be Supermicro BMC or IPMIView implementation issues rather than IPMI itself Not all IPMI 2.0 BMCs support RCMP+ encrypted sessions, no prior IPMI versions do Not all BMCs can do speeds other than 19.2Kb/s SOL isn't a perfect environment, but it's better than travelling to your datacentre in an emergency

Links Most Important first: http://wiki.adamsweet.org/doku.php?id=ipmi_on_linux Others: http://www.intel.com/design/servers/ipmi/ http://en.wikipedia.org/wiki/Intelligent_Platform_Management_Interface