Virtual Architects Support Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== OmniOS ======

**Howto**: https://napp-it.org/downloads/omnios.html

See also **[[va:va_shared_storage|Shared Storage System Notes]]**.

See also **[[computing:storage:napp-it|napp-it]]**.

See also **[[computing:storage:zfs|ZFS]]**.

:!: For large-memory systems, choose a larger boot drive or disable dump because the swap and dump devices take up a lot of space by default.

**Reference HowTo**: http://virtuallyhyper.com/2013/04/installing-and-configuring-omnios/

===== Documentation =====

http://www.napp-it.org/downloads/omnios_en.html

http://omnios.omniti.com/wiki.php

==== Cheat Sheets ====

  * How to create network configuration profiles?
  * How to configure VLANS?
  * How to manipulate Zones?
  * How to use ZFS shadow migration?

Solaris 11 Administrator Cheat Sheet:

http://www.oracle.com/technetwork/server-storage/solaris11/documentation/solaris-11-cheat-sheet-1556378.pdf

Solaris 11 Package Manager Cheat Sheet:

http://www.oracle.com/technetwork/server-storage/solaris11/documentation/ips-one-liners-032011-337775.pdf

=== Memory ===

<file>
echo ::memstat | mdb -k
swap -lh
</file>

=== Install Pkg File ===

<file>
pkgadd -d package_name.pkg
</file>

===== Full Reboot =====

A normal 'reboot' command does a fast reboot.  This does a full reboot:

<file>
reboot -p
</file>

:!: The following commands change the default behavior of the ''reboot'' command to do a full reboot not a fast reboot.

<file>
svccfg -s "system/boot-config:default" setprop config/fastreboot_default=false
svccfg -s "system/boot-config:default" setprop config/fastreboot_onpanic=false
svcadm refresh svc:system/boot-config:default
</file>

===== Updating =====

Update the 'pkg' package manager first:

<file>
pkg install pkg:/package/pkg
</file>

You can then do a trial run image update:

<file>
pkg image-update -nv
</file>

Run the same command again without the 'n' flag to actually do the update. 

If you get stuck with updating, try:

<file>
pkg refresh --full
pkg update -f
reboot -p
</file>

===== Upgrading =====

**Update Napp-IT first**: http://www.napp-it.org/doc/downloads/setup_napp-it_os.pdf

**Then OmniOS**:

https://napp-it.org/downloads/omnios-update_en.html

https://omniosce.org/upgrade

===== Manage Boot Environments =====

http://docs.oracle.com/cd/E23824_01/html/E21801/administer.html

<file>
beadm list                                    # list BE's
beadm rename <be-name> <new-be-name>          # rename a BE - can't rename active BE
beadm destroy <be-name>                       # delete a BE
beadm activate <be-name>                      # make another BE active on reboot
</file>

===== Installing =====

http://omnios.omniti.com/wiki.php/Installation

<note warning>If you are using SATA interfaces on the server mainboard, configure the BIOS to AHCI mode before installing OmniOS.  If you forget, you will have to reinstall to fix this.</note>

Download the CD image and burn it to a disc or download a USB image and burn it to a flash drive:

<file>
dd if=/path/to/image.usb-dd of=/dev/sdx bs=1M
</file>

===== Adjust Boot (GRUB) Timeout =====

<file>
bootadm set-menu timeout=5
</file>

===== Permit Root Login =====

:!: The system installs a root user with a **blank password**.

Set the root password with the ''passwd'' command.

Enable root login via ssh in the Napp-IT web interface: **Services > SSH allow root**

or by editing the ''/etc/ssh/sshd_config'' file, changing the ''PermitRootLogin'' option to ''yes''.

===== Networking =====

http://docs.oracle.com/cd/E19963-01/html/821-1458/gkind.html

New tools supersede the older ''ipconfig'' and ''ndd'' tools:

  * ''dladm''
  * ''ipadm''
  * Changes persist after reboot

Useful networking commands (examples below):

<file>
dladm show-phys
dladm show-link
ipadm show-if [interface]
ipadm show-addr [addrobj]
ipadm create-addr -T address-type -a address/prefixlen addrobj
ipadm show-addrprop [-p property] [addrobj]
</file>

==== Basic Network Configuration ====

http://docs.oracle.com/cd/E23824_01/html/821-1458/gljtt.html

<file>
dladm show-phys                                        # list available interfaces
dladm show-link                                        # shows link status
dladm show-ether                                       # shows Ethernet info
dladm show-aggr                                        # shows aggregated interfaces
ipadm show-addr                                        # show IP addressing
netstat -nr                                            # show IP routes
</file>

=== Static ===

Disable nwam (Network AutoMagic):

<file>
svcs nwam
svcs network/physical:default

svcadm disable svc:/network/physical:nwam
svcadm enable svc:/network/physical:default
</file>

Delete an IP interface before configuring it, if it already exists:

<file>
ipadm show-if
ipadm delete-if igb0
</file>

<file>
ipadm create-if igb0
ipadm create-addr -T static -a 192.168.1.4/24 igb0/v4  # assign an IPv4 address
route -p add default 192.168.1.1                       # assign a default route if not already configured
</file>

=== DHCP ===

<file>
ipadm create-addr -T dhcp igb0/dhcp
</file>

==== Name Resolution ====

:!: Use your domain name and your internal DNS server.

<file>
echo 'domain domainname.tld' > /etc/resolv.conf
echo 'nameserver 10.0.0.5' >> /etc/resolv.conf
echo 'nameserver 8.8.8.8' >> /etc/resolv.conf
cp /etc/nsswitch.dns /etc/nsswitch.conf
</file>

==== SSH ====

<file>
vi /etc/ssh/sshd_config

PermitRootLogin yes
GSSAPIAuthenication no                    # Fix slow logins
AllowUsers root backupuser                # If you need another user to use SSH
</file>

<file>
svcadm restart svc:/network/ssh:default
</file>

==== Link Aggregation (Bonding) ====

See also **[[computing:unix:unix_link_aggr|UNIX Link Aggreagation (Bonding)]]**

  * Napp-IT (free) doesn't do bonding
  * Use the CLI

==== Jumbo Frames ====

<file>
vi /kernel/drv/ixgbe.conf

default_mtu             = 9000;
</file>

===== Storage =====

==== Install LSI SAS2IRCU ====

:!: Needed for LSI HBAs

Download the SAS2IRCU archive from Broadcom and upload it to the desired server.

https://www.broadcom.com/support/download-search?pg=&pf=&pn=&pa=&po=&dk=sas2ircu

<file>
chmod +x sas2ircu
./sas2ircu list
./sas2ircu 0 display
</file>

==== Install LSI MegaCLI ====

:!: Only works with RAID controllers, not HBAs.

Download the MegaCLI archive from Broadcom and upload it to the desired server.

https://www.broadcom.com/support/download-search?pg=&pf=&pn=&pa=&po=&dk=megacli

<file>
pkgadd -d MegaCli.pkg

chmod +x /opt/MegaRAID/CLI/MegaCli
/opt/MegaRAID/CLI/MegaCli
</file>

==== Mirror the Boot Drive ====

FIXME This section is a mix of old and new information.  It needs editing.

See also **[[computing:storage:napp-it|Napp-IT ZFS Storage Server]]**

<note tip>Read the help first in **Napp-IT -> Home -> Disks -> Mirror Bootdisk**</note>

List all disks and ZFS pools:

<file>
echo |format
zpool list
cfgadm -s "select=type(disk)"
zpool status rpool
</file>

  * ''p0'' is entire disk
  * ''s0'' is first slice

Create one large partition:

<note tip>If there is any preexisting configuration on the disk (many drives come partitioned!), such as partition info, reinitialize it with **Napp-IT -> Home -> Disks -> Initialize**</note>

FIXME Is the following necessary?

Set up SMI label with same partitioning as original boot disk:

<file>
pfexec prtvtoc /dev/rdsk/c5t2d0s2 | pfexec fmthard -s - /dev/rdsk/c5t3d0s2
</file>

Add the second drive to the rpool:

<file>
pfexec zpool attach -f rpool c5t2d0s0 c5t3d0s0
</file>

If your bootdisk ends with ''s0'' (ex c0t0d0s0), ''s0'' is added to newdisk like:
''zpool attach -f rpool c0t0d0s0 c0t1d0s0''.

:!: Do not continue until the resilver is finished.

Check resilver progress:

<file>
zpool status -v rpool
</file>

Make the mirror drive bootable:

<file>
bootadm  install-bootloader -Mfv -P rpool
</file>

:!: To boot to the second drive, select it in the BIOS as the first boot device.

<file>
reboot -p
</file>

==== Package Management ====

http://omnios.omniti.com/wiki.php/Packaging

http://pkg.omniti.com/omniti-ms/en/index.shtml and

http://pkg.omniti.com/omnios/release/en/index.shtml

http://pkg.thetube.ch:10001

http://scott.mathematik.uni-ulm.de/

=== Configure Publishers (Repositories) ===

<file>
pkg publisher                                                        # list configured publishers
pkg set-publisher -g http://pkg.omniti.com/omniti-ms/ ms.omniti.com  # add a publisher
pkg unset-publisher ms.omniti.com                                    # remove a publisher
</file> 

===== Managing Dump and Swap =====

Planning for Swap Space: http://docs.oracle.com/cd/E19963-01/html/821-1459/fsswap-31050.html

Adjusting the Sizes of Your ZFS Swap and Dump Devices: http://docs.oracle.com/cd/E19963-01/html/821-1448/ggrln.html

==== Dump ====

:!: You only need a dump device if you need (and usually buy) help from OS devs, example from OmniTi.
If so, you may need a larger rpool commensurate with your server's RAM size.

Estimate needed dump space (128GB RAM):

<file>
root@stor-01:/root# dumpadm -e
Estimated dump size: 61.5G
</file>

Query the current dump device size:

<file>
root@stor-01:/root# zfs get volsize rpool/dump
NAME        PROPERTY  VALUE    SOURCE
rpool/dump  volsize   17.7G    local
</file>

Change the dump volume size:

<file>
zfs get volsize rpool/dump

zfs set volsize=4G rpool/dump
</file>

Disable dump and delete the dump volume:

<file>
dumpadm -d none
zfs destroy rpool/dump
</file>

==== Swap ====

:!: You can probably skip this section if you are configuring a storage server as the swap space is not pre-allocated.

Note the change in blocks allocated to swap:

<file>
swap -lh
swap -d /dev/zvol/dsk/rpool/swap
zfs volsize=4G rpool/swap
swap -a /dev/zvol/dsk/rpool/swap
swap -lh
</file>

===== User Management =====

==== Add User ====

<file>
useradd -g staff -d /home/newuser -m -s /bin/bash -c "Full Name" newuser
</file>

Make 'newuser' an administrator (sudo privileges):

<file>
visudo
</file>

Check for errors:

<file>
pwck
</file>

==== User Roles (RBAC) ====

https://blogs.oracle.com/observatory/en_US/entry/understading_rbac

Allow 'newuser' to 'su' to 'root':

<file>
usermod -R root newuser
</file>

===== Services (Daemons) =====

List services:

<file>
svcs
</file>

After making configuration file changes:

<file>
svcadm refresh <servicename>
</file>

:!: If services show as 'maintenance' rather than 'online', there is probably a configuration error.

===== FTP =====

https://www.napp-it.org/extensions/proftpd.html

Install ProFTPd:

<file>
wget -O - www.napp-it.org/proftpd | perl
</file>

This starts the service and can also be used to restart the ProFTPd service after making configuration changes:

**Services -> FTP -> Enable proftpd**

<file>
svcs proftpd
svcadm enable proftpd
svcadm disable proftpd
</file>

FIXME So far, I have only used manual methods for controlling users, home directories and permissions!


===== NTP =====

Configure NTP:

<file>
vi /etc/inet/ntp.conf
</file>

Edit:

<file>
pool 0.us.pool.ntp.org iburst
pool 1.us.pool.ntp.org
pool 2.us.pool.ntp.org
pool 3.us.pool.ntp.org
</file>

Set the time and enable the NTP daemon:

<file>
ntpdate pool.ntp.org

svcadm enable ntp
</file>

NTP takes a while to get synchronized.  Check the status (on localhost) like this:

<file>
ntpq -p
</file>

===== IPMI =====

<file>
pkg install ipmi ipmitool
</file>

====== Legacy Info Below ======

===== Add Unrecognized New Hardware =====

:!: While this section was written for the new Intel x540 10GbE NIC, it seems this driver has been updated and these steps are no longer necessary.  These notes are left as an example only.

Determine the 'vendor' and 'device' id's that aren't being recognized by the 'ixgbe' driver, then add them to the list:

<file>
scanpci

grep ixgbe /etc/driver_aliases

update_drv -a -i "pci8086,1528" ixgbe

ifconfig -a plumb
ifconfig -a

modinfo|grep ixgbe

more /etc/path_to_inst | grep ixgbe
</file>

===== Add New Persistent NIC =====

:!: If you are using Napp-IT, use it for assigning IP addresses.

<file>
vi /etc/hostname.ixgbe0
</file>

<file>
10.1.10.5 netmask 255.255.255.0
</file>

<file>
vi /etc/hostname.ixgbe1
</file>

<file>
10.1.10.6 netmask 255.255.255.0
</file>

<file>
vi /etc/inet/hosts
</file>

<file>
10.1.10.5       10g-eth-1 
10.1.10.6       10g-eth-2
</file>

===== Configure Jumbo Frames =====

==== Intel X540 10GbE NIC ====

http://docs.oracle.com/cd/E19254-01/820-7895-11/cggfhffg.html

<file>
nano /kernel/drv/ixgbe.conf

default_mtu             = 9000;

reboot
</file>

==== General Jumbo Frames Info ====

http://docs.oracle.com/cd/E19862-01/820-1606-12/jumbo_fms.html

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2012445

https://blogs.oracle.com/taylor22/entry/nfs_streaming_over_10_gbe

http://docs.oracle.com/cd/E19254-01/820-7895-11/cggfhffg.html

http://hardforum.com/showthread.php?t=1601960

  * For optimal network speed, use high quality NICs such as Intel server NICs
  * Jumbo frames helps for large data transfers, but it is slower for small data
Virtual Architects Support Wiki

User Tools

Site Tools

Page Tools
