====== Citrix XenServer ======
See also **[[virtualization:xenserver:xenserver_updates|Xenserver Updates]]**
See also **[[http://www.sonoracomm.com/wiki/doku.php?do=search&id=xenserver|Other XenServer Pages in this Wiki]]**
http://www.citrix.com/xenserver
* XenServer is a bare-metal (type 1) hypervisor, available for free
* XCP is an Open Source version of Citrix XenServer
* Citrix XenCenter can manage both
* XenCenter requires Windows to run
* Use tab-completion for ''xe'' commands
===== Official Citrix Resources =====
**Quick Installation Guide**: http://www.citrix.com/site/resources/dynamic/salesdocs/XenServer_Quick_Installation_Guide.pdf
**Other Docs**: http://www.citrix.com/xenserver_additionalresources_free
**P2V**: https://www.citrix.com/go/products/xenserver/xenserver-xenconvert-free.html
===== Updates =====
See also **[[virtualization:xenserver:xenserver_updates|Xenserver Updates]]**
===== PV vs. HVM =====
This page has lots of info clearly explained:
http://www.shermann.name/2011/07/quick-tip-for-installing-ubuntu-as.html
===== Delete Storage Repository =====
xe sr-list
xe pbd-list sr-uuid=your-SR-uuid
xe pbd-unplug uuid=your-PBD-uuid
xe pbd-destroy uuid=your-PBD-uuid
xe sr-forget uuid=your-SR-uuid
===== Create New Storage Repository =====
==== Local Storage ====
http://www.schirmacher.de/display/INFO/How+to+add+additional+disks+to+XenServer+host
http://www.xendesktopmaster.com/how-to-add-an-additional-local-disk-to-your-xenserver-5-5-host/
http://www.peppercrew.nl/index.php/2012/03/adding-extra-local-storage-space-to-xenserver-6-0/
:!: It is advisable to partition the new disk with one large partition as opposed to using the bare drive for LVM because many tools will report an unpartitioned drive as 'empty'.
:!: Verify the new local disk is visible to the XenServer host OS and create a single large partition of **type 8e00 (Linux LVM)**. Use ''gdisk'' to create a [[computing:storage:disk_partitioning|GPT partition table]] for disks over 2TB.
gdisk /dev/sdx #where x is your new drive
cat /proc/partitions
ll /dev/disk/by-id
Survey the existing storage repositories:
xe sr-list
=== Option 1 - Create a new SR ===
xe sr-create content-type=user type=lvm device-config:device=/dev/disk/by-id/scsi-SATA_ST31000528AS_9VP1JTGC-part1 shared=false name-label="Local storage 2"
=== Option 2 - Extend an Existing SR ===
* One benefit of extending a volume to a second drive is that you will have a single large volume to work with. If you need this, it's worth it.
* One down-side is that your volume now depends on two drives and is therefore twice as likely to fail.
After partitioning the drive (x) as per above:
vgdisplay
vgextend VG_XenStorage-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx /dev/sdx1
==== ISO Storage on CIFS ====
CIFS -> Centos CIFS ISO library -> \\192.168.0.6\nas_vg_1.nas_vol_1.no_backup\Centos
\\server\share\folder
Set username and password.
==== ISO Storage on NFS ====
You can't set folders using NFS, only shares, so .iso files must be in the top-level folder with NFS:
No_Backup NFS ISO library -> 192.168.0.6:/mnt/nas_vg_1/nas_vol_1/no_backup
==== ISO Storage on Local Disk ====
http://kb.softescu.ro/server-administration/linux-server-administration/how-to-create-a-local-iso-storage-repository-on-citrix-xenserver/
:!: This will only create a small SR to hold smallish (rescue?) images.
:!: DO NOT FILL the Dom0 partition too full!
:!: Note that this will probably break and have to be fixed after a version upgrade!
mkdir -p /var/opt/xen/iso_library
xe sr-create name-label="ISO Repository" type=iso device-config:location=/var/opt/xen/iso_library/ device-config:legacy_mode=true content-type=iso
=== Create an ISO Image ===
To create an ISO image from a physical CD/DVD:
* Create the ISO storage repository as above
* Place the optical disc in the XenServer's optical drive
* Use ''dd'' to create the image and place it in the SR
dd if=/dev/dvd of=/var/opt/xen/iso_library/Windows_Server_2008_R2_OEM.iso
===== Introduce a Local Storage Repository =====
http://www.schirmacher.de/display/INFO/How+to+reattach+a+disk+to+XenServer
http://support.citrix.com/article/CTX121896
===== Install Guest (New VM) =====
==== Centos 5.5 Minimal Netinstall ====
Start with 5.4 32-bit template and use mostly defaults.
Install from URL:
http://mirrors.easynews.com/linux/centos/5.5/os/i386/
http://mirrors.easynews.com/linux/centos/5.5/os/x86_64/
Do NOT start the VM automatically.
Adjust the Description and Storage Name and Storage Description as needed.
Start the VM - be patient because some large files are transferred over the 'net.
==== Debian 5 (Lenny) Net-Install ====
URL:
http://mirrors.easynews.com/linux/debian/
then:
debian.mirrors.easynews.com
apt-get install openssh-server
==== SME Server 8 Netinstall ====
URL:
http://distro.ibiblio.org/pub/linux/distributions/smeserver/releases/testing/8.0beta6/smeos/i386/
===== AutoStart VMs =====
http://burm.net/2012/01/28/xenserver-tips-and-tricks-auto-start-your-vm/
* As of XenServer 6, you can't configure a VM to auto-start in the GUI
* vApp has been introduced
* vApp gives control over the order and delays of a group of VMs
In a simple setup, you can configure VMs to auto-start from the CLI:
xe pool-list
xe pool-param-set uuid=UUID other-config:auto_poweron=true
xe vm-list
xe vm-param-set uuid=UUID other-config:auto_poweron=true
===== vApp =====
==== Manage from the CLI ====
=== List VMs ===
xe vm-list
=== Create a vApp ===
xe appliance-create name-label= [name-description=]
xe appliance-list
=== Add VMs to vApp ===
xe vm-param-set uuid= appliance= order=0 start-delay=45
xe vm-param-set uuid= appliance= order=1 start-delay=45
=== Delete a vApp ===
appliance-destroy uuid=
==== vApp Startup ====
http://www.virtues.it/2012/01/howto-autostart-xs-vapp/
* vApp feature introduced in XenServer 6
* vApp feature keyed to HA
* Not currently configurable via the GUI
* vApps won't auto-start on a cold boot
xe appliance-list
Append these lines to /etc/rc.local:
sleep 20
/opt/xensource/bin/xe appliance-start uuid=[uuid-vapp]
===== Install XenServer Tools =====
See **[[virtualization:xenserver:xenserver_tools|XenServer Client Issues]]**
===== Change XenServer Hostname =====
xe host-list
xe host-set-hostname-live host-uuid= host-name=
reboot
===== Networking =====
http://blog.vhowto.info/2010/10/17/xenserver-5-6-network-configuration-part-3/
Design Guide: http://support.citrix.com/servlet/KbServlet/download/27046-102-666250/XS-design-network_advanced.pdf
Move a XenServer Pool to a Different IP Subnet: http://support.citrix.com/article/CTX123477
* Define a gateway only on the Mangement interface
* You can't use a VLAN for the Management interface
* Unless it's the 'default' or 'native' VLAN
* Or handled by your switch
* Best practice is no VLAN on Management interface
==== Dedicated Storage Network ====
:!: See the **Admin Guide** for more info.
:!: Before dedicating a network interface as a storage interface for use with iSCSI or NFS SRs, you must ensure that the dedicated interface uses a separate IP subnet which is not routable from the main management interface. If this is not enforced, then storage traffic may be directed via the main management interface after a host reboot, due to the order in which network interfaces are initialized.
http://www.fatmin.com/2011/11/how-to-create-a-dedicated-storage-nic-in-xenserver.html
http://www.fatmin.com/2012/07/xenserver-how-to-build-and-configure-a-dedicated-nfs-storage-bond.html
:!: If you have an IP address set on a NIC that is neither management nor storage, the PIF (physical interface) cannot send or receive traffic on the PIF network. If you have an IP address on a bridge (i.e. xenbr1) which includes the target PIF, the following commands will not work.
http://support.citrix.com/article/CTX128623
First we need to get the uuid of the PIF (physical interface) that we want to use:
xe pif-list host-name-label=
Next we reconfigure our PIF:
xe pif-reconfigure-ip mode=static IP= netmask= uuid=
xe pif-param-set disallow-unplug=true uuid=
xe pif-param-set other-config:management_purpose="Storage" uuid=
Enabling jumbo frames in XenServer simply requires changing the MTU for each pool-wide network from the default of 1500 to 9000 and rebooting each member of the pool.
The following steps need to be performed on only one of the XenServers in the pool to enable jumbo frames:
xe pif-list
xe network-list
xe network-param-set uuid=[network uuid] MTU=9000
===== To get the uuid of the VDI of a specific VM =====
xe vm-disk-list vm=
===== Boot Linux to CD-ROM =====
xe vm-list
xe vm-param-set HVM-boot-policy="BIOS order" uuid=UUID of VM
Next, on the "General" tab, select the "Edit" button.
On the "Startup Options" tab, ensure that "DVD-Drive" is moved to the top.
Before starting VM, make sure that the ISO that you want is in the VM's DVD-Drive located on the "Console" tab.
Proceed to use SUSE CD-ROM (or DVD) to upgrade, or Linux Rescue media, etc. Once all changes have been made to the VM, you will need to revert back the change to the VM's parameter with the following command:
xe vm-param-set HVM-boot-policy="" uuid=UUID of VM
Reboot VM
Final caveat... mouse is unavailable as it is "HVM mode" and no device has been loaded.
===== Convert HVM <-> PV =====
http://support.citrix.com/article/CTX121875
http://linuxnet.ch/groups/linuxnet/wiki/51a8b/XenServer_HowTo_Convert_HVM_to_PV_RHELSuSE.html
http://djlab.com/2010/10/convert-xenserver-hvm-to-pv-paravirtual/
===== Xenserver Backup =====
See also **[[computing:backup:xenserver_backup|XenServer Backup]]**
See also **[[virtualization:xenserver:xenserver_backup_cli|XenServer CLI Backup]]**
See also **[[virtualization:xenserver:xenserver_backup_system|Free XenServer Backup System]]**
===== AoE Storage =====
See [[computing:storage:aoe|ATA over Ethernet (AoE)]]
http://forums.citrix.com/thread.jspa?threadID=296776
http://support.citrix.com/article/CTX116020
===== High Availability =====
http://forums.citrix.com/message.jspa?messageID=1367138
* Bonded NICs
* Separate network paths for
* VMs
* Storage
* Management
* 6 NICs per server!
* SAN/[[NAS]] storage
===== Firewall =====
XenCenter - Port 443
http://66.165.176.77/servlet/KbServlet/download/2389-102-647335/Citrix_TCP_Ports_0167.pdf
iptables -nL -v --line-numbers
==== NTP ====
iptables -I RH-Firewall-1-INPUT 13 -p udp --dport 123 -j ACCEPT
service iptables save
==== Sample Firewall ====
This sample firewall allows NTP and limits access to ports 22, 80 and 443 by the sourde IP.
/etc/sysconfig/iptables
# Generated by iptables-save v1.3.5 on Mon Apr 9 00:15:34 2012
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [135:25337]
:RH-Firewall-1-INPUT - [0:0]
-A INPUT -j RH-Firewall-1-INPUT
-A FORWARD -j RH-Firewall-1-INPUT
-A RH-Firewall-1-INPUT -i lo -j ACCEPT
-A RH-Firewall-1-INPUT -p icmp -m icmp --icmp-type any -j ACCEPT
-A RH-Firewall-1-INPUT -p esp -j ACCEPT
-A RH-Firewall-1-INPUT -p ah -j ACCEPT
-A RH-Firewall-1-INPUT -d 224.0.0.251 -p udp -m udp --dport 5353 -j ACCEPT
-A RH-Firewall-1-INPUT -p udp -m udp --dport 631 -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m tcp --dport 631 -j ACCEPT
-A RH-Firewall-1-INPUT -i xenapi -p udp -m udp --dport 67 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A RH-Firewall-1-INPUT -p udp -m state --state NEW -m udp --dport 694 -j ACCEPT
-A RH-Firewall-1-INPUT -s 209.104.9.32/255.255.255.240 -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
-A RH-Firewall-1-INPUT -s 209.193.64.248/255.255.255.248 -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
-A RH-Firewall-1-INPUT -s 72.200.111.140 -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
-A RH-Firewall-1-INPUT -s 209.193.64.2 -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
-A RH-Firewall-1-INPUT -s 209.104.9.32/255.255.255.240 -p tcp -m state --state NEW -m tcp --dport 80 -j ACCEPT
-A RH-Firewall-1-INPUT -s 209.193.64.248/255.255.255.248 -p tcp -m state --state NEW -m tcp --dport 80 -j ACCEPT
-A RH-Firewall-1-INPUT -s 72.200.111.140 -p tcp -m state --state NEW -m tcp --dport 80 -j ACCEPT
-A RH-Firewall-1-INPUT -s 209.193.64.2 -p tcp -m state --state NEW -m tcp --dport 80 -j ACCEPT
-A RH-Firewall-1-INPUT -p udp -m udp --dport 123 -j ACCEPT
-A RH-Firewall-1-INPUT -s 209.104.9.32/255.255.255.240 -p tcp -m state --state NEW -m tcp --dport 443 -j ACCEPT
-A RH-Firewall-1-INPUT -s 209.193.64.248/255.255.255.248 -p tcp -m state --state NEW -m tcp --dport 443 -j ACCEPT
-A RH-Firewall-1-INPUT -s 72.200.111.140 -p tcp -m state --state NEW -m tcp --dport 443 -j ACCEPT
-A RH-Firewall-1-INPUT -s 209.193.64.2 -p tcp -m state --state NEW -m tcp --dport 443 -j ACCEPT
-A RH-Firewall-1-INPUT -j REJECT --reject-with icmp-host-prohibited
COMMIT
# Completed on Mon Apr 9 00:15:34 2012
===== Resize Virtual Disk =====
Assumes:
1. Running LVM in the guest (you can do this without it, but it's difficult).
2. You're using a filesystem such as Ext3 on the partition you wish to expand which supports online expansion.
3. You're able to perform a backup just in case something goes wrong.
Steps:
__5-11. Run pvresize /dev/xvdX____" (replacing "X" with the disk identified in step 4).
__
1. Shut down the virtual machine from XenCenter.
2. Resize the VDI from XenCenter to the appropriate size
3. Start the VM up.
4. Run "fdisk -l" to locate the virtual disk by looking at the size.
5. Run "fdisk /dev/xvdX" (replacing "X" with the disk identified in step 4).
6. Create a new partition (usually primary) using all available space. The default options will be sufficient. To do this, type "n", follow the instructions to create the partition, then type "p" to verify the changes and then "w" to write and then "q" to quit.
7. Reboot the VM again to allow udev to create the appropriate /dev/ node for the partition just created.
8. Create a new physical volume by running "pvcreate /dev/xvdXY" (replacing "X" with the disk identified in step 4).
9. Locate the Volume Group containing the partition by running "mount". Look for the partition you wish to resize in the list. You should also see a reference to the Volume Group (e.g. "VolGroup00") and the Logical Volume (e.g. "LogVol00") take note of these two items.
10. Extend your Volume Group with the newly added Physical Volume by running vgextend as shown:
"vgextend VolGroupXX /dev/xvdXY" (Replace "VolGroupXX" with the Volume Group as identified in step 9, and "/dev/xvdXY" as identified in steps 4 and 6).
11. Confirm the Volume Group has free storage by running "vgs". This should indicate the total size and the amount of storage free for allocation to logical volumes.
12. Expand the Logical Volume using all available free extents by running:
lvextend -l+100%FREE /dev/VolGroupXX/LogVolYY
Remember to change VolGroupXX and LogVolYY as appropriate.
13. Assuming ext3 or ext2 is being used (other filesystems will have their own tools for this purpose), resize the filesystem by running:
resize2fs /dev/VolGroupXX/LogVolYY
14. As a precaution, reboot the system again checking the filesystem for errors:
shutdown -r now -F
15. Observe the output of the "fsck" execution during startup and monitor for any errors that may indicate corruption.
===== Guest GUI =====
==== X and VNC on XenServer ====
When you install Redhat on XenServer it does not by default enable a virtual video device. You are only given the text console by default.
Citrix's Virtual Machine Installation Guide does document how to enable the typical X console using VNC.
Check to make sure that vnc-server and gdm are installed.
rpm -q vnc-server gdm
If they are not, install them.
yum install vnc-server gdm
Modify ''/etc/gdm/custom.conf'' so the default server is a VNC server:
[servers]
0=VNC
[server-VNC]
name=VNC Server
command=/usr/bin/Xvnc -SecurityTypes None -geometry 1024x768 -depth 16 -BlacklistTimeout 0
flexible=true
When GDM is running it should be listening on port 5900. Make sure that the iptables firewall allows access to this port from any machine running XenCenter or wanting to connect.
iptables -N vnc
iptables -I INPUT -p tcp -m state --state NEW -m tcp --dport 5900 -j vnc
iptables -A vnc -s 134.114.0.0/255.255.0.0 -p tcp -m state --state NEW -m tcp --dport 5900 -j ACCEPT
iptables -A vnc -p tcp -m tcp -m state --state NEW --dport 5900 -j REJECT
Now everything is set up, but by default Redhat on XenServer starts up into runlevel 3. This does not by default start GDM. So you can modify the default runlevel in /etc/inittab or as I would recommend, simply use telinit 5 to start it and telinit 3 to turn it off. This saves memory and CPU cycles.
===== NFS =====
http://www.gossamer-threads.com/lists/xen/api/266896
http://dsstos.blogspot.com/2008/01/nfs-and-oracle-mount-options-noac.html
:!: In an NFS VHD storage repository, VM images are stored as thin-provisioned VHD format files on a shared NFS target.
:!: XenServer requires NFS Version 3 over TCP for remote storage use.
===== Physical to Virtual (P2V) =====
http://www.citrix.com/go/products/xenserver/xenserver-xenconvert-free.html
===== IPMI =====
See also **[[computing:remote:ipmi|IPMI]]**
First load the kernel modules:
modprobe ipmi_msghandler
modprobe ipmi_si
modprobe ipmi_devintf
Display current configuration:
ipmitool -v lan print 1
Then manage (adjusting for your network):
ipmitool -v lan set 1 ipsrc static
ipmitool -v lan set 1 ipaddr 10.10.4.xxx
ipmitool -v lan set 1 netmask 255.255.255.0
ipmitool -v lan set 1 defgw ipaddr 10.10.4.1
ipmitool -v lan set 1 defgw macaddr D4:CA:6D:1E:4C:97
ipmitool -I open user set password 2 xxxxxxxxx
ipmitool -v lan set 1 access on
http://10.10.4.xxx
:!: **Username: ADMIN**
===== Change Pool Master =====
http://www.bl-nk.net/2014/03/change-xenserver-pool-master/