====== IPMI ======

FIXME Some of these are old notes and may need verification or updating.

**SuperMicro**: http://www.supermicro.com/products/nfo/IPMI.cfm

**Manual**: ftp://ftp.supermicro.com/CDR-X8_1.22_for_Intel_X8_platform/MANUALS/SMT_IPMI_User%27s_Guide.pdf

**Usage**: http://netkiller.github.io/monitoring/ipmitool.html

**Howto**: http://www.thomas-krenn.com/en/wiki/Configuring_IPMI_under_Linux_using_ipmitool

**DRAC**: http://www.theprojectbot.com/ipmitool-cheatsheet-and-configuring-drac-from-ipmitool/

**Best Practices**: http://www.supermicro.com/products/nfo/files/IPMI/Best_Practices_BMC_Security.pdf

:!: **The default username and password are ADMIN/ADMIN (all caps), at least for Supermicro.**

===== Firewall =====

^Function      ^Port   ^Protocol ^
|HTTP          |80     |TCP      |
|HTTPS         |443    |TCP      |
|IPMI          |623    |UDP      |
|Remote console|5900   |TCP      |
|Virtual media |623    |TCP      |

===== Documentation =====

SuperMicro IPMI Solution Guide: ftp://ftp.supermicro.com/CDR-0010_2.10_IPMI_Server_Managment/Manuals/Installation_20.pdf

===== AOC-IPMI20-E BMC =====

  * Shares LAN1 port (nearest the Keyboard/Mouse ports)
  * Test with machine turned off to avoid host OS interference
  * Sonora Comm will use 'admin' username and change password to router password
  * UDP Port 623 (may also need TCP Port 623?)
  * BMC will not respond to ping even when working properly

==== Install BMC ====

Most new servers have IPMI settings in the BIOS.  Older and cheaper BMCs require:

  * Boot to IPMI support CD and flash BMC firmware
  * IPMI 2.0
  * RMCP+

===== Mainboard and BMC specific =====

  * Bootable floppy images also on CD (needs Windows host)
  * ''ipnmac.exe'' (from DOS) sets IP and MAC addresses
  * Uses separate IP and MAC addresses (Broadcom NIC - use label on IPMI socket)
  * Uses LAN1 MAC address (Intel NIC)

===== Configure BMC - Linux Server =====

:!: Log in via SSH if you are off-site.

We need the gateway IP and MAC addresses for the IPMI interface.  You may be able to get them like this:

<file>
route -n
arp -n
</file>

==== Supermicro Agent ====

:!: I have not had to use this with recent CentOS 5 or 6.  Skip to ''ipmitool'' section.

This will install and add commands to ''/etc/rc.local'' to run ''smagent'' which is needed for graceful shutdowns and reboots of the Linux OS:

<file>
wget ftp://ftp.supermicro.com/CDR-0010_2.10_IPMI_Server_Managment/IPMI_Solution/Linux/GPC_Agent/GPC_Agent-2.1-050406.tar.gz

tar -xzvf GPC_Agent-2.1-050406.tar.gz
cd GPC_Agent
./quickinstall      # choose option #2
</file>

Manual commands to do the same:

<file>
modprobe ipmi_devintf
modprobe ipmi_si type=kcs ports=0xca8 regspacings=4
/usr/supermicro/smagent &

lsmod

modprobe ipmi_msghandler
modprobe ipmi_si
modprobe ipmi_devintf
</file>

==== ipmitool ====

:?: Works with IPMI kernel modules or smagent?

=== Ubuntu 20.04 ===

<file>
apt install ipmitool
</file>

=== CentOS 6 ===

<file>
modprobe ipmi_devintf
</file>

=== Centos 5 (XenServer) ===

<file>
modprobe ipmi_devintf
modprobe ipmi_si
</file>

If ''ipmitool'' is not already installed:

<file>
yum install OpenIPMI-tools
</file>

=== Install on Centos 4 ===

<file>
wget http://dag.wieers.com/rpm/packages/ipmitool/ipmitool-1.8.8-1.el4.rf.i386.rpm
rpm -Uvh ipmitool-*
</file>

===== Configure IPMI =====

Display the current configuration:

<file>
ipmitool -v lan print 1
</file>

Do the configuration:

<file>
ipmitool -v lan set 1 ipsrc static
ipmitool -v lan set 1 ipaddr <youripmiipaddress>
ipmitool -v lan set 1 netmask <youripminetmask>
ipmitool -v lan set 1 defgw ipaddr <youripmigwip>
ipmitool -v lan set 1 defgw macaddr <youripmigwmac>
ipmitool -v lan set 1 arp respond on
ipmitool -v lan set 1 auth ADMIN MD5
ipmitool -v lan set 1 access on
</file>

Show the IPMI users:

<file>
ipmitool user list 1
</file>

Configure the IPMI user (the default Supermicro 'ADMIN' user is usually #2):

:!: You may have trouble setting a password with symbols.  Try setting a simple password, then use the GUI interface to set a more complicated one.

<file>
ipmitool -v user set password 2 <youripmipassword>
ipmitool -v user priv 2 4
ipmitool -v channel setaccess 1 2 link=on ipmi=on callin=on privilege=4
ipmitool -v user enable 2
</file>

:!: Privilege level '4' is 'ADMINISTRATOR'.

==== IPMICLI (Supermicro) ====

:?: Use ipmitool instead?

<file>
wget ftp://ftp.supermicro.com/CDR-0010_2.10_IPMI_Server_Managment/IPMI_Solution/Linux/Administrator/CLI/readme

wget ftp://ftp.supermicro.com/CDR-0010_2.10_IPMI_Server_Managment/IPMI_Solution/Linux/Administrator/CLI/ipmicli
</file>

===== Management Workstation =====

Install IPMIView (assumes Java already installed on system):

<file>
mkdir -p ~/bin/IPMIView20
cd ~/bin/IPMIView20

wget ftp://ftp.supermicro.com/utility/IPMIView/Release/Jar/IPMIView20-Class_v2.6.44_Build080515.zip
unzip IPMIView20-Class_v2.6.44_Build080515.zip

chmod +x IPMIView20.sh

vim IPMIView20.sh

cd ~/bin/IPMIView20/
java -jar IPMIView20.jar
</file>

===== Troubleshooting =====

  * You may need to refresh the list of hosts to see them as on-line
  * Graceful reboot takes 1 minute to start
  * Can be seen on redirected text console
  * Redirected text console dies when login prompt is presented

<note warning>Be sure that the network cable for IPMI is plugged into the dedicated IPMI port BEFORE you connect power to the server.  The BMC selects between ''shared'' and ''dedicated'' at power-on.</note>

==== no iKVM64 in java.library.path ====

http://www.hackerway.ch/2013/01/28/how-to-resolve-no-ikvm64-in-java-library-path-error-ubuntu-x64-12-04-lts/

==== Reboot the BMC ====

:!: Try this first!

:!: Wait for the BMC to reboot.

Reboot the BMC, then try changing the configuration again:

<file>
ipmitool mc reset cold
</file>

**Factory BMC Reset**: http://www.supermicro.com/support/faqs/faq.cfm?faq=21709

Reset the BMC to factory defaults (from Supermicro FAQ):

<file>
ipmitool raw 0x3c 0x40
</file>

or, using a Supermicro tool:

<file>
./ipmicfg-linux.x86_64 -fdl
</file>

===== BIOS Updates =====

Supermicro IPMI controllers usually have a feature to install mainboard BIOS updates, as well as IPMI firmware updates, via the IPMI web interface.  This is a licensed commercial feature and requires a license key to function.

:!: **Trial license keys** are available from Supermicro.  They can take a few days to get.

:!: This feature can be a real benefit if you are having trouble updating a BIOS or if you need to **update a BIOS remotely**.

  * Send an e-mail to sales@supermicro.com
  * Ask for
    * Trial license key for IPMI BIOS updates
    * Part number ''SFT-OOB-LIC''
  * Include
    * Exact mainboard model number
    * MAC addresses on your mainboard including BMC

===== CLI =====

You can SSH into the BMC which will put you into SMASH.

You can switch to a BusyBox shell:

<file>
shell sh
</file>