Transcript Unix Profession Webcast October 2007
Solaris ZFS Boot on SPARC
© 2008 Dusan Baljevic The information contained herein is subject to change without notice
Traditional File Systems and ZFS
April 28, 2020 Dusan Baljevic 2
ZFS Goals
• Pooled storage design makes for easier administration • No need for a Volume Manager • Straightforward commands and a GUI • Snapshots & clones • Quotas and reservations • Compression • Pool migration • ACLs for security • Dynamic striping • Dynamic resilvering • Dynamic resizing April 28, 2020 Dusan Baljevic 3
ZFS Overheads
• Compression • Checksum - It has been measured to consume very roughly 1 Ghz of CPU to checksum 500MB/sec of data • RAID-Z As of the Solaris 10 11/06 release, checksum, compression, and RAID-Z parity computation all occur in the context of the thread synching pool data. Under load, that thread may be a performance choke point. It is expected that in future releases all these computations will be done in concurrent threads. Compression is no longer single-threaded via CR 6460622, and is part of the Solaris 10 8/07 release April 28, 2020 Dusan Baljevic 4
Solaris – ZFS Boot Project
The ZFS Boot project has been divided into three areas: x86 boot, SPARC boot, and install.
• • • • The main phases of the project are: A ZFS plug-in for the GRUB boot loader (available since OpenSolaris snv_62) Development of a boot loader on SPARC Enhancements necessary for implementing a ZFS root file systems Enhancements to the Solaris install and LiveUpgrade features in order to set up and maintain root file systems in ZFS pools April 28, 2020 Dusan Baljevic 5
Solaris ZFS with AVS
• Sun StorageTek Availability Suite (AVS), Remote Mirror Copy and Point-in-Time Copy services, previously known as SNDR (Sun Network Data Replicator) and II (Instant Image), are similar to the Veritas VVR (volume replicator) and Flashsnap (point-in-time copy) products, is currently available in the Solaris Express release • SNDR differs from the ZFS send and recv features, which are time-fixed replication features. For example, you can take a point-in-time snapshot, replicate it, or replicate it based on a differential of a prior snapshot. The combination of AVS II and SNDR features, also allows you to perform time-fixed replication. The other modes of the AVS SNDR replication feature allows you to obtain CDP (continuous data replication). ZFS does not currently have this feature April 28, 2020 Dusan Baljevic 6
Solaris ZFS – Third-Party Backups
• Sun StorEdge Enterprise Backup Software (Legato Networker 7.3.2 and above) product can fully back up and restore ZFS files including ACLs • EMC Networker version 7.3.2 and above backs up and restores ZFS file systems, including ZFS ACLs • Veritas NetBackup version 6.5 and above can backup and restore ZFS file systems, including ZFS ACLs • IBM Tivoli Storage Manager client software (5.4.1.2) backs up and restores ZFS file systems with both the CLI and the GUI. ZFS ACLs are also preserved • Computer Associates' BrightStor ARCserve product backs up and restores ZFS file systems, but ZFS ACLs are not preserved April 28, 2020 Dusan Baljevic 7
Solaris ZFS and Data Protector
Back Agents (Disk Agents) ZFS support: Solaris 10 (including ACL support) 6.0 Released ZFS support: Solaris 10 (excluding ACL support) 5.5
Released Data Protector supports ZFS now.
April 28, 2020 Dusan Baljevic 8
ZFS Boot Lab Server
HOSTNAME sft3 FQDN sft3.mydom.dom
MODEL SUNW,Sun-Fire-T2000 UNAME -A SunOS 5.11 snv_97 sun4v ARCH 64-bit sparcv9 kernel RUN LEVEL 3 PHYSICAL MEMORY 8 GB CPU 32 x 1000MHz virtual CPUs PAGESIZE 8192 bytes VOLUME MANAGER COUNT SINGLE Volume Manager VOLUME MANAGER Zettabyte File System (ZFS) April 28, 2020 Dusan Baljevic 9
Solaris – ZFS Disks
ZFS applies an EFI label when creating a storage pool with whole disks. Disks can be labelled with a traditional Solaris VTOC label when creating a storage pool with a disk slice.
Slices should only be used under the following conditions: • The device name is non-standard • A single disk is shared between ZFS and another file system, such as UFS • A disk is used as a swap or a dump device • Disk is used for booting the operating system * April 28, 2020 Dusan Baljevic 10
Solaris – EFI Versus VTOC (SMI) Labels
EFI is a new disk label that was introduced in the Solaris 9 4/03 Operating System version. The acronym EFI stands for Extensible Firmware Interface and this new label format is REQUIRED for all devices over 1 TB in size, and cannot be converted back to VTOC.
• Supported by 64-bit Solaris 9 Operating System (04/03) and above • Provides support for disks greater than 1 TB in size • Provides 7 useable slices - 0 thru 6 - where slice 2 is just another slice • Solaris ZFS uses EFI labels by default • Partitions (or slices) cannot overlap with the primary or backup label, or with any other partitions. The size of the EFI label is usually 34 sectors, so partitions start at sector 34. This feature means no partition can start at sector zero (0) April 28, 2020 Dusan Baljevic 11
Solaris – EFI Versus VTOC (SMI) Labels (continued)
• No cylinder or head information is stored in the label. Sizes are reported in sectors and blocks • Information that was stored in the alternate cylinders area, the last two cylinders of the disk, is now stored in slice 8 • With the "format" utility to change partition sizes, the unassigned partition tag is assigned to partitions with sizes equal to zero. By default, the "format" utility assigns the "usr" partition tag to any partition with a size greater than zero. You can use the "partition change" menu to reassign partition tags after the partitions are changed. You cannot change a partition with a non-zero size to the "unassigned" partition tag • EFI labels CAN be written to disks smaller than 1TB, where a standard VTOC label would normally apply, but VTOC cannot be forced onto devices larger than 1TB. This can be done by running "format -e". When you attempt to "label" the disk, you will be asked which type of label is to be written with April 28, 2020 Dusan Baljevic 12
Solaris – EFI Restrictions
• The layered software products not intended for systems with EFI-labelled disks might be incapable of accessing a disk with an EFI disk label. (for example, current VxVM versions). A disk with an EFI disk label is not recognized on systems running previous Solaris releases. Adding an EFI labelled disk to a system that does not support it: Dec 3 09:12:17
Solaris – EFI Restrictions (continued)
• The EFI specification prohibits overlapping slices. The whole disk is represented by "c#t#d#p6": lrwxrwxrwx 1 root root 70 Nov 25 18:54 c8t1d0p6 -> ../../devices/.../ssd@w50020f23000004aa,0:wd • The following format options are not applicable on disks with EFI labels: o The "save" option is not supported because disks with EFI labels do not need an entry in the "format.dat" file o The "backup" option is not applicable because the disk driver finds the primary label and writes it back to the disk • In the context of SunCluster 3.0/3.1 (refer to Sun Solution 213130): # scgdevs Could not stat: /dev/rdsk/../../devices/pci@1c,600000/scsi@2,1/sd@8,0:h,raw path not loaded.
# prtvtoc /dev/did/rdsk/d4s2 prtvtoc: /dev/did/rdsk/d4s2: No such device or address April 28, 2020 Dusan Baljevic 14
EFI Versus VTOC Disk Label *
# format -e c0t8d0
selecting c0t8d0 [disk formatted] FORMAT MENU: disk - select a disk type partition - select (define) a partition table current format repair label analyze defect backup verify save inquiry scsi cache volname !
Solaris – ZFS Boot Disk Slices
Two internal disks were available on SunFire T-2000: c0t0d0 c0t1d0 The following slices were created on c0t0d0: Current partition table (original): Total disk cylinders available: 14087 + 2 (reserved cylinders) Part Tag Flag Cylinders Size Blocks 0 root wm 0 - 14086 68.35GB (14087/0/0) 143349312 1 unassigned wm 0 0 (0/0/0) 0 2 backup wm 0 - 14086 68.35GB (14087/0/0) 143349312 3 unassigned wm 0 0 (0/0/0) 0 4 unassigned wm 0 0 (0/0/0) 0 5 unassigned wm 0 0 (0/0/0) 0 6 unassigned wm 0 0 (0/0/0) 0 7 unassigned wm 0 0 (0/0/0) 0 April 28, 2020 Dusan Baljevic 16
Solaris – ZFS Recommendations
• Run ZFS on a system that runs a 64-bit kernel • 1 GB or more of physical memory is recommended Approximately 64 KB of memory is consumed per mounted ZFS file system. On systems with many ZFS file systems, it is recommended to have 1 GB of extra memory for every 10,000 mounted file systems including snapshots. Be prepared for longer boot times on these systems as well.
• The minimum amount of available pool space that is required for a bootable ZFS root file system is larger than for a bootable UFS root file system because swap and dump devices are not shared in a ZFS root environment. In addition, the swap and dump devices are sized at 1/2 the size of RAM, but no more than 2 GB and no less than 512 MB (starting at build 96) • The minimum amount of available pool space for a bootable ZFS root file system depends upon the amount of physical memory, the disk space available, and the number of BEs to be created. Approximately 1 GB of memory and at least 32 GB of disk space are recommended April 28, 2020 Dusan Baljevic 17
Solaris – ZFS Recommendations (continued)
• Because ZFS caches data in kernel addressable memory, the kernel sizes will possibly be larger than with other file systems. Configure additional disk-based swap to account for this difference for systems with limited RAM. Use the size of physical memory as an upper bound to the extra amount of swap space that might be required • If possible, do not use slices on the same disk for both swap space and ZFS file systems. Keep the swap areas separate from the ZFS file systems. The best policy is to have enough RAM so that your system does not normally use the swap devices April 28, 2020 Dusan Baljevic 18
Solaris – ZFS Root File System Recommendations
Keep the root pool (the pool with the dataset that is allocated for the root file system) separate from pools that are used for data: • Some limitations on root pools exist that you would not want to place on data pools. Mirrored pools and pools with one disk will be supported. RAID-Z or unreplicated pools with more than one disk will not be supported • Data pools can be architecture-neutral. It might make sense to move a data pool between SPARC and Intel. Root pools are pretty much tied to a particular architecture • As a good practice, it is recommended to separate the "personality" of a system from its data April 28, 2020 Dusan Baljevic 19
Solaris ZFS Boot Installation
• Solaris Express Community Edition snv_97 SPARC (Solaris 10 Update 6 is the first official release, Nov 2008) • Boot off the DVD Choose Filesystem Type ──────────────────────────────────── Select the filesystem to use for your Solaris installation [] UFS [X] ZFS April 28, 2020 Dusan Baljevic 20
Solaris ZFS Boot Installation (continued)
Select Software ─────────────────────────────────────────── Select the Solaris software to install on the system.
NOTE: After selecting a software group, you can add or remove software by customizing it. However, this requires understanding of software dependencies and how Solaris software is packaged.
[X] Entire Distribution plus OEM support ....... 8527.00 MB [ ] Entire Distribution ........................ 8495.00 MB [ ] Developer System Support ................... 8190.00 MB [ ] End User System Support .................... 6178.00 MB [ ] Core System Support ........................ 907.00 MB [ ] Reduced Networking Core System Support ..... 849.00 MB April 28, 2020 Dusan Baljevic 21
Solaris ZFS Boot Installation (continued)
Select Disks ─────────────────────────────────────────── On this screen you must select the disks for installing Solaris software.
Start by looking at the Suggested Minimum field; this value is the approximate space needed to install the software you've selected. For ZFS, multiple disks will be configured as mirrors, so the disk you choose, or the slice within the disk must exceed the Suggested Minimum value. NOTE: ** denotes current boot disk Disk Device =============================================== [X] c0t0d0 Available Space 69994 MB (F4 to edit) [ ] [ ] [ ] c0t1d0 c2t50001FE150062328d3 c2t50001FE15006232Cd3 69994 MB 24560 MB 24560 MB April 28, 2020 Maximum Root Size: 69994 MB Suggested Minimum: 8527 MB 22
Solaris ZFS Boot Installation (continued)
Specify the name of the pool to be created from the disk(s) you have chosen.
Also specify the name of the dataset to be created within the pool that is to be used as the root directory for the filesystem.
ZFS Pool Name: rpool ZFS Root Dataset Name: snv_97 ZFS Pool Size (in MB): 69995 Size of Swap Area (in MB): 2048 Size of Dump Area (in MB): 2048 (Pool size must be between 8527 MB and 69995 MB) April 28, 2020 [] Keep / and /var combined [X ] Put /var on a separate dataset 23
Solaris ZFS Boot Installation – First Reboot
Rebooting with command: boot Boot device: disk File and args: zfs-file-system Loading: /platform/SUNW,Sun-Fire-T200/boot_archive Loading: /platform/sun4v/boot_archive ramdisk-root ufs-file-system Loading: /platform/SUNW,Sun-Fire-T200/kernel/sparcv9/unix Loading: /platform/sun4v/kernel/sparcv9/unix SunOS Release 5.11 Version snv_97 64-bit Copyright 1983-2008 Sun Microsystems, Inc. All rights reserved.
Hostname: sft3 Configuring devices.
Loading smf(5) service descriptions: 202/202 Creating new rsa public/private host key pair Creating new dsa public/private host key pair Sep 6 00:23:46 sft3 sendmail[8326]: My unqualified host name (sft3) unknown; sleeping for retry * April 28, 2020 Dusan Baljevic 24
Solaris ZFS Boot Installation – Boot from OpenBoot Prompt (OBP)
Rebooting with command:
boot -L
Boot device: /pci@780/pci@0/pci@9/scsi@0/disk@0 File and args: -L zfs-file-system Loading: /platform/sun4v/bootlst 1 snv_97 Select environment to boot: [ 1 - 1 ] April 28, 2020 Dusan Baljevic 25
Solaris ZFS Boot Installation – First Login #
df -F zfs –h
Filesystem size used avail capacity Mounted rpool/ROOT/snv_97 67G 6.2G 49G 12% / rpool/ROOT/snv_97/var 67G 144M 49G 1% /var rpool/export 67G 21K 49G 1% /export rpool/export/home 67G 406K 49G 1% /export/home rpool 67G 63K 49G 1% /rpool
#
uname
-a SunOS sft3 5.11 snv_97 sun4v sparc SUNW,Sun-Fire T200 April 28, 2020 Dusan Baljevic 26
Solaris ZFS – Legacy vs ZFS (V)fstab
#
cat /etc/vfstab
fd - /dev/fd fd - no /proc - /proc proc - no /dev/zvol/dsk/rpool/swap - swap - no /devices - /devices devfs - no sharefs - /etc/dfs/sharetab sharefs - no ctfs - /system/contract ctfs - no objfs - /system/object objfs - no swap - /tmp tmpfs - yes April 28, 2020 Dusan Baljevic 27
Solaris ZFS Boot – Dump and Swap Separation
#
dumpadm
Dump content: kernel pages Dump device: /dev/zvol/dsk/rpool/dump (dedicated) Savecore directory: /var/crash/sft2k1 Savecore enabled: yes #
swap -l
swapfile dev swaplo blocks free /dev/zvol/dsk/rpool/swap2 253,2 16 8388592 8388592 April 28, 2020 Dusan Baljevic 28
Solaris ZFS Pools and Datasets
#
zfs list
NAME USED AVAIL REFER MOUNTPOINT rpool 18.3G 48.6G 63K /rpool rpool/ROOT 6.29G 48.6G 18K legacy rpool/ROOT/snv_97 6.29G 48.6G 6.15G / rpool/ROOT/snv_97/var 144M 48.6G 144M /var rpool/dump 8.00G 48.6G 8.00G rpool/export 484K 48.6G 21K /export rpool/export/home 446K 48.6G 20K /export/home rpool/swap 8G 48.6G 16K April 28, 2020 Dusan Baljevic 29
Solaris ZFS Boot Installation – Change Size of Root File System
#
zfs set quota=4000m rpool/ROOT/snv_97
cannot set property for 'rpool/ROOT/snv_97': size is less than current used or reserved space
#
zfs set quota=8000m rpool/ROOT/snv_97
#
df -k /
Filesystem rpool/ROOT/snv_97 kbytes used avail capacity Mounted 8192000 6656377 1388629 83% / #
zfs set quota=none rpool/ROOT/snv_97
#
df -k /
Filesystem kbytes used avail capacity Mounted rpool/ROOT/snv_97 70189056 6656377 46604048 13% / April 28, 2020 Dusan Baljevic 30
Solaris ZFS Boot Disk Mirroring (part 1)
# prtvtoc /dev/rdsk/c0t0d0s2 | fmthard -s - /dev/rdsk/c0t1d0s2 # zpool attach -f rpool c0t0d0s0 c0t1d0s0 (root pool cannot be raidz yet) # zpool status rpool
pool: rpool state: ONLINE status: One or more devices is currently being resilvered. The pool will continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
scrub: resilver in progress for 0h6m, 71.40% done, 0h2m to go config: NAME rpool mirror STATE ONLINE ONLINE READ WRITE CKSUM 0 0 0 0 0 0 c0t0d0s0 c0t1d0s0 ONLINE ONLINE errors: No known data errors 0 0 0 0 0 0 April 28, 2020 Dusan Baljevic 31
Solaris ZFS Boot Disk Mirroring (part 2)
SPARC # installboot -F zfs /usr/platform/`uname -i`/lib/fs/zfs/bootblk \ /dev/rdsk/c0t1d0s0 X86 # installgrub /boot/grub/stage1 /boot/grub/stage2 /dev/rdsk/c0t1d0s0 # eeprom "use nvramrc?=true“ # prtconf -pv | grep bootpath bootpath: '/pci@780/pci@0/pci@9/scsi@0/disk@0,0:a‘ # eeprom "nvramrc=devalias zfsroot \ /pci@780/pci@0/pci@9/scsi@0/disk@0,0:a \ devalias zfsmirror /pci@780/pci@0/pci@9/scsi@0/disk@1,0:a“
April 28, 2020 | |
\/
format(1) lists this as "sd"
Dusan Baljevic 32
Solaris ZFS Boot Disk Mirroring (part 3)
# eeprom boot-device="zfsroot zfsmirror disk net" # eeprom
ttya-rts-dtr-off=false ttya-ignore-cd=true local-mac-address?=true … boot-device=zfsroot zfsmirror disk net use-nvramrc?=true nvramrc=devalias zfsroot /pci@780/pci@0/pci@9/scsi@0/disk@0,0:a devalias zfsmirror /pci@780/pci@0/pci@9/scsi@0/disk@1,0:a security-mode=none … April 28, 2020 Dusan Baljevic 33
Solaris ZFS Boot Installation – Change Primary Swap Device and Its Size (part 1)
#
zfs set quota=4192m rpool/swap
cannot set property for ‘rpool/swap': 'quota' does not apply to datasets of this type. Quotas cannot be set on volumes, as the "volsize" property acts as an implicit quota
#
swap –l
swapfile dev swaplo blocks free /dev/zvol/dsk/rpool/swap 253,2 16 16777200 16777200 April 28, 2020 Dusan Baljevic 34
Solaris ZFS Boot Installation – Change Primary Swap Device and Its Size (part 2)
#
zfs create -V 4096m rpool/swap2
#
zfs list
NAME rpool rpool/ROOT rpool/ROOT/snv_97 rpool/ROOT/snv_97/var rpool/dump rpool/export rpool/export/home rpool/swap rpool/swap2 USED 26.5G
6.49G
6.49G
144M 8.00G
466K 446K 8G 4G AVAIL 40.4G
40.4G
40.4G
40.4G
40.4G
40.4G
40.4G
48.4G
44.4G
REFER 63K 18K 6.35G
144M 8.00G 21K 406K 2.05M
16K MOUNTPOINT /rpool legacy / /var /export /export/home April 28, 2020 Dusan Baljevic 35
Solaris ZFS Boot Installation – Change Primary Swap Device and Its Size (part 3)
#
swap -a /dev/zvol/dsk/rpool/swap2
#
swap -l
swapfile dev /dev/zvol/dsk/rpool/swap 253,2 /dev/zvol/dsk/rpool/swap2 253,3 swaplo blocks free 16 16777200 16777200 16 8388592 8388592 #
swap -d /dev/zvol/dsk/rpool/swap
#
swap -l
swapfile dev /dev/zvol/dsk/rpool/swap2 253,3 swaplo blocks free 16 8388592 8388592 April 28, 2020 Dusan Baljevic 36
Solaris ZFS Boot Installation – Change Primary Swap Device and Its Size (part 4)
#
zfs destroy rpool/swap
#
zfs list
NAME rpool rpool/ROOT rpool/ROOT/snv_97 rpool/ROOT/snv_97/var 144M rpool/dump 8.00G
rpool/export 6.49G
6.49G
466K rpool/export/home rpool/swap2 USED 18.5G
446K 4G AVAIL 48.4G
48.4G
48.4G
48.4G
48.4G
48.4G
48.4G
52.4G
REFER 63K 18K 6.35G
144M 8.00G
21K 406K 16K MOUNTPOINT /rpool legacy / /var /export /export/home April 28, 2020 Dusan Baljevic 37
Solaris ZFS Boot Installation – Change Primary Swap Device and Its Size (part 5)
# df -k /tmp
Filesystem kbytes used avail capacity Mounted swap 8479424 40 8479384 1% /tmp
# grep swap /etc/vfstab
/dev/zvol/dsk/rpool/swap swap swap /tmp tmpfs no yes April 28, 2020 Dusan Baljevic 38
Solaris ZFS – Root Pool Detach Mirror
#
zpool offline rpool c0t1d0s0 *
#
zpool detach rpool c0t1d0s0
#
zpool status
pool: rpool state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM rpool ONLINE 0 0 0 c0t1d0s0 ONLINE 0 0 0 errors: No known data errors • • Two potential problems: When booting the mirror disk, if the primary disk is online, it will be resilvered with the old data There is no easy way to access the mirror disk data without rebooting April 28, 2020 Dusan Baljevic 39
Solaris ZFS – Example EFI and VTOC (part 1)
#
format 0. c7t600508B400102E8E0001500013FB0000d0
Continue? Y
April 28, 2020 Dusan Baljevic 40
Solaris ZFS – Example EFI and VTOC (part 2) EFI Label
Part Tag Flag First Sector Size Last Sector 0 usr wm 34 35.99GB 75481054 1 unassigned wm 0 0 0 2 unassigned wm 0 0 0 3 unassigned wm 0 0 0 4 unassigned wm 0 0 0 5 unassigned wm 0 0 0 6 unassigned wm 0 0 0 7 unassigned wm 0 0 0 8 reserved wm 75481055 8.00MB 75497438 April 28, 2020 Dusan Baljevic 41
Solaris ZFS – Example EFI and VTOC (part 3) VTOC Label
Part Tag Flag Cylinders Size Blocks 0 root wm 0 1 swap wu 16 15 128.00MB (16/0/0) 262144 31 128.00MB (16/0/0) 262144 2 backup wu 0 - 4605 35.98GB (4606/0/0) 75464704 3 unassigned wm 0 0 (0/0/0) 0 4 unassigned wm 0 0 (0/0/0) 0 5 unassigned wm 0 0 (0/0/0) 0 6 usr wm 32 - 4605 35.73GB (4574/0/0) 74940416 7 unassigned wm 0 0 (0/0/0) 0 April 28, 2020 Dusan Baljevic 42
Solaris ZFS – Example EFI and VTOC (part 4)
#
zpool create hppool raidz1 \ /dev/dsk/c7t600508B400102E8E0001500014000000d0s0 \ /dev/dsk/c7t600508B400102E8E0001500013FB0000d0s0 \ /dev/dsk/c7t600508B400102E8E0001500013FB0000d0s6
invalid vdev specification use '-f' to override the following errors: raidz contains devices of different sizes #
zpool create -f hppool raidz1 /dev/dsk/c7t600508B400102E8E0001500014000000d0s0 \ /dev/dsk/c7t600508B400102E8E0001500013FB0000d0s0 \ /dev/dsk/c7t600508B400102E8E0001500013FB0000d0s6
* April 28, 2020 Dusan Baljevic 43
Solaris ZFS – Example EFI and VTOC (part 5)
#
zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT hppool 370M 158K 370M 0% ONLINE rpool 33.8G 5.35G 28.4G 15% ONLINE #
zpool status hppool
pool: hppool state: ONLINE scrub: none requested NAME STATE READ WRITE CKSUM hppool ONLINE 0 0 0 raidz1 ONLINE 0 0 0 c7t600508B400102E8E0001500014000000d0s0 ONLINE 0 0 0 c7t600508B400102E8E0001500013FB0000d0s0 ONLINE 0 0 0 c7t600508B400102E8E0001500013FB0000d0s6 ONLINE 0 0 0 errors: No known data errors April 28, 2020 Dusan Baljevic 44
Solaris ZFS – Snapshot and Rollback *
#
zfs snapshot rpool/export/home@Dusan
#
zfs list
NAME rpool rpool/ROOT rpool/ROOT/snv_97 rpool/ROOT/snv_97/var rpool/dump rpool/export rpool/export/home rpool/export/home@Dusan rpool/swap USED AVAIL 22.5G
44.4G
6.49G
44.4G
6.49G
144M 8.00G
44.4G
44.4G
44.4G
428K 408K 0 44.4G
44.4G
8G #
zfs list -o snapdir rpool/export/home@Dusan
#
rm /export/home/dusan/somefile
#
zfs rollback rpool/export/home@Dusan
REFER 63K 18K 6.35G
144M 8.00G
20K 408K 408K 52.4G
MOUNT /rpool legacy / /var /export /export/home 2.05M
April 28, 2020 Dusan Baljevic 45
Solaris ZFS – Delegated Administration
# zpool get delegation rpool NAME PROPERTY VALUE SOURCE rpool delegation on default # zfs allow dusan create,destroy,mount,snapshot rpool/export/home # zfs allow rpool/export/home ------------------------------------------------------------ Local+Descendent permissions on (rpool/export/home) user dusan create,destroy,mount,snapshot -------------------------------------------------------------
April 28, 2020 Dusan Baljevic 46
Solaris ZFS – Snapshot and Clone
#
zfs snapshot rpool/export/home@Dusan2
#
zfs clone rpool/export/home@Dusan2 rpool/export/clone
#
zfs list
NAME rpool rpool/ROOT rpool/ROOT/snv_97 rpool/ROOT/snv_97/var USED 22.5G
6.49G
6.49G
144M rpool/dump rpool/export rpool/export/clone 8.00G
466K 0 rpool/export/home rpool/export/home@Dusan 446K 17K rpool/export/home@Dusan2 22.5K
rpool/swap 8G AVAIL 44.4G
44.4G
44.4G
44.4G
44.4G
44.4G
44.4G
44.4G
52.4G
REFER 63K 18K 6.35G
144M 8.00G
21K 412K 406K 406K 412K 2.05M
#
rm /export/home/dusan/somefile2
#
ls /export/clone/dusan/somefile2
April 28, 2020 Dusan Baljevic MOUNTPOINT /rpool legacy / /var /export /export/clone /export/home 47
Solaris ZFS Pool Command History
#
zpool history
History for 'rpool': 2008-09-05.22:55:48 zpool create -f -o failmode=continue -R /a -m legacy -o cachefile=/tmp/root/etc/zfs/zpool.cache rpool c0t0d0s0 2008-09-05.22:55:48 zfs set canmount=noauto rpool 2008-09-05.22:55:49 zfs set mountpoint=/rpool rpool 2008-09-05.22:55:49 zfs create -o mountpoint=legacy rpool/ROOT 2008-09-05.22:55:50 zfs create -b 8192 -V 8192m rpool/swap 2008-09-05.22:55:51 zfs create -b 131072 -V 8192m rpool/dump 2008-09-05.22:59:19 zfs create -o canmount=noauto rpool/ROOT/snv_97 2008-09-05.22:59:19 zfs create -o canmount=noauto rpool/ROOT/snv_97/var 2008-09-05.22:59:20 zpool set bootfs=rpool/ROOT/snv_97 rpool 2008-09-05.22:59:20 zfs set mountpoint=/ rpool/ROOT/snv_97 2008-09-05.22:59:21 zfs set canmount=on rpool 2008-09-05.22:59:22 zfs create -o mountpoint=/export rpool/export 2008-09-05.22:59:23 zfs create rpool/export/home 2008-09-06.09:22:47 zfs set quota=8000m rpool/ROOT/snv_97 April 28, 2020 Dusan Baljevic 48
What to Do if ZFS Panics
ZFS is designed to survive arbitrary hardware failures through the use of redundancy (mirroring or RAID-Z). However, certain failures in non replicated configurations can cause ZFS to panic when trying to load the pool. This is a bug, and will be fixed in the near releases. If you cannot boot due to a corrupt pool, do the following: boot using
'-m milestone=none
' #
mount -o remount /
#
rm /etc/zfs/zpool.cache
#
reboot
This will remove all knowledge of pools from your system. You will have to re-create your pool and restore from backup.
April 28, 2020 Dusan Baljevic 49
Solaris ZFS – Web Management
•
Web ZFS management (as of Solaris Express Community Release build 28): #
/usr/sbin/smcwebserver enable
#
/usr/sbin/smcwebserver start
https://servername:6789/zfs
April 28, 2020 Dusan Baljevic 50
Solaris ZFS – Redundant Copies for Data*
#
zfs create -o copies=1 mypool3/single
#
zfs create -o copies=2 mypool3/double
#
zfs create -o copies=3 mypool3/triple
#
cp -rp /etc /mypool3/single
#
cp -rp /etc /mypool3/double
#
cp -rp /etc /mypool3/triple
#
zfs list -r mypool3
NAME USED AVAIL REFER MOUNTPOINT mypool3 48.2M 310M 33.5K /mypool3 mypool3/double 16.0M 310M 16.0M /mypool3/double mypool3/single 8.09M 310M 8.09M /mypool3/single mypool3/triple 23.8M 310M 23.8M /mypool3/triple April 28, 2020 Dusan Baljevic 51
Solaris ZFS – Redundant Copies for Data (continued) *
April 28, 2020 Dusan Baljevic 52
Solaris ZFS – Redundant Copies for Data (continued) *
April 28, 2020 Dusan Baljevic 53
Solaris ZFS and JumpStart (part 1)
• For a JumpStart installation, you cannot use an existing ZFS storage pool to create a bootable ZFS root pool. Example:
install_type initial_install cluster SUNWCall pool rpool 20G 4g 4g any bootenv installbe bename myBE
• You must create your pool with disk slices rather than whole disks. If in the profile you create a pool with whole disks, such as c0t0d0, the installation fails “Invalid disk name (c0t0d0)” April 28, 2020 Dusan Baljevic 54
Solaris ZFS and JumpStart (part2)
• Some keywords that are allowed in a UFS specific profile are not allowed in a ZFS specific profile. Examples of those not valid for ZFS root pool: client_arch, client_root, backup_media, archive_location, client_swap, layout_constraint, partitioning, system_type, and so on. More details are in Solaris Installation Guide • You cannot upgrade with JumpStart - must use Solaris Live Upgrade April 28, 2020 Dusan Baljevic 55
Solaris ZFS and JumpStart Examples (part 1)
• Mirrored ZFS Root Pool
install_type initial_install cluster SUNWCall pool myrootpool auto auto auto mirror c0t0d0s0 c0t1d0s0 bootenv installbe bename solaris10_6
• Customizing the Disk Size For a ZFS Root Pool
install_type initial_install cluster SUNWCall pool zfspool 80g 2g 2g mirror any any bootenv installbe bename solaris10_6
April 28, 2020 Dusan Baljevic 56
Solaris ZFS and JumpStart Examples (part 2)
• Specifying where to Install the O/S
install_type initial_install cluster SUNWCall root_device c0t0d0s0 pool hprootpool auto auto auto rootdisk.s0
bootenv installbe bename mybootname dataset /var
April 28, 2020 Dusan Baljevic 57
Solaris ZFS and Live Upgrade (part 1)
# lustatus
ERROR: No boot environments are configured on this system ERROR: cannot determine list of all boot environment names
# lucreate -c BE1 -n BE2
Current boot environment is named
Creating initial configuration for primary boot environment
The device is not a root device for any boot environment; cannot get BE ID.
PBE configuration successful: PBE name
Source boot environment is
Creating boot environment
Cloning file systems from boot environment
Creating snapshot for
Creating clone for
Setting canmount=noauto for in zone
Creating snapshot for
Creating clone for
Setting canmount=noauto for in zone
Population of boot environment
Creation of boot environment
April 28, 2020 Dusan Baljevic 58
Solaris ZFS and Live Upgrade (part 2)
# lustatus
Boot Environment Is Active Active Can Copy Name Complete Now On Reboot Delete Status --------------- -------- ------- -------- ------ ----- BE1 yes yes yes no BE2 yes no no yes April 28, 2020 Dusan Baljevic 59
Solaris ZFS and Live Upgrade (part 3)
#
lufslist BE1
boot environment name: BE1 This boot environment is currently active.
This boot environment will be active on next system boot.
Filesystem fstype device size Mounted on Mount Options ---------------------- ----- ---------- ----------- /dev/zvol/dsk/rpool/swap swap 1073741824 ------------ rpool/ROOT/s10s_u6wos_07b zfs 4671150080 / rpool/ROOT/s10s_u6wos_07b/var zfs 73081856 /var rpool zfs 6820660736 /rpool rpool/export zfs 38912 /export hppool zfs 124806 /hppool rpool/export/home zfs 18432 /export/home #
lufslist BE2
boot environment name: BE2 Filesystem fstype device size Mounted on Mount Options ---------------------- ----- ---------- ------------ ------------ /dev/zvol/dsk/rpool/swap swap 1073741824 rpool/ROOT/BE2 zfs 104448 / rpool/export zfs 38912 /export rpool/export/home zfs 18432 /export/home hppool zfs 124806 /hppool rpool zfs 6820660736 /rpool rpool/ROOT/BE2/var zfs ? /var April 28, 2020 Dusan Baljevic 60
Solaris ZFS and Live Upgrade (part 4)
#
lumount BE2 /BE2
#
df -F zfs -h
Filesystem size used avail capacity Mounted on rpool/ROOT/s10s_u6wos_07b 33G 4.3G 27G 14% / rpool/ROOT/s10s_u6wos_07b/var 33G 70M 27G 1% /var rpool/export 33G 20K 27G 1% /export rpool/export/home 33G 18K 27G 1% /export/home rpool 33G 94K 27G 1% /rpool hppool 214M 24K 214M 1% /hppool rpool/ROOT/BE2 33G 4.3G 27G 14% /BE2 rpool/ROOT/BE2/var 33G 70M 27G 1% /BE2/var April 28, 2020 Dusan Baljevic 61
Solaris ZFS and Encryption Current integration target is build 105 of OpenSolaris (2009.04). Even if Sun management and marketing wanted ZFS Crypto in S10u7 it is not technically possible.
To backport ZFS crypto into a Solaris 10 update would require also backporting a large number of bug fixes and projects that were integrated into the Crypto Framework that haven't been yet been backported to Solaris 10.
April 28, 2020 Dusan Baljevic 62
Appendix ZFS Pool Properties
#
zfs get all rpool
NAME PROPERTY VALUE SOURCE rpool type filesystem rpool creation Fri Sep 5 22:55 2008 rpool used 18.3G rpool available 48.6G rpool referenced 63K rpool compressratio 1.00x rpool mounted yes rpool quota none default rpool reservation none default rpool recordsize 128K default rpool mountpoint /rpool local rpool sharenfs off default rpool checksum on default April 28, 2020 Dusan Baljevic 63
Appendix ZFS Pool Properties (continued)
NAME PROPERTY VALUE SOURCE rpool compression off default rpool atime on default rpool devices on default rpool exec on default rpool setuid on default rpool readonly off default rpool zoned off default rpool snapdir hidden default rpool aclmode groupmask default rpool aclinherit restricted default rpool canmount on default rpool shareiscsi off default rpool xattr on default rpool copies 1 default April 28, 2020 Dusan Baljevic 64
Appendix ZFS Pool Properties (continued)
NAME PROPERTY VALUE SOURCE rpool copies 1 default rpool version 3 rpool utf8only off rpool normalization none rpool casesensitivity sensitive rpool vscan off default rpool nbmand off default rpool sharesmb off default rpool refquota none default rpool refreservation none default rpool primarycache all default rpool secondarycache all default April 28, 2020 Dusan Baljevic 65
Appendix Solaris10 Update 6
* Installation Enhancements o Solaris Installation for ZFS Root Pools * System Administration Enhancements o ZFS Command Improvements and Changes + ZFS installation and boot support + Rolling back a ZFS dataset without unmounting + Enhancements to the zfs send command + ZFS quotas and reservations for file system data only + new ZFS storage pool properties + ZFS command history enhancements + support for upgrading ZFS filesystems + ZFS delegated administration + Setting up separate ZFS logging devices + Creating intermediate ZFS datasets + ZFS hot-plugging enhancements + GZIP compression now available for ZFS + Storing multiple copies of ZFS user data April 28, 2020 Dusan Baljevic 66
Appendix Solaris10 Update 6 (continued)
o Solaris Installation Tool Support of ZFS File Systems + Solaris interactive text installer to install a UFS or a ZFS root file system.
+ Custom JumpStart features to set up a profile to create a ZFS storage pool and designate a bootable ZFS file system.
feature.
+ Migrate a UFS root file system to a ZFS root file system by using the Solaris Live Upgrade + Set up a mirrored ZFS root pool by selecting two disks during the installation.
+ Automatically create swap and dump devices on ZFS volumes in the ZFS root pool.
o SunVTS 7.0 Patch Set 3 o lockstat Provider for DTrace. DTrace lockstat probes that displayed the spin count (spins) now returns spin time in nanoseconds.
* System Resource Enhancements o New Solaris Zones Features + Update on Attach.
If the new host has the same or later versions of the zone-dependent packages and their associated patches, using zoneadm attach with the -u option, updates those packages within the zone to match the new host.[...] This option also enables automatic migration between machine classes, such as from sun4u to sun4v.
+ Ability to Set Default Router in Shared-IP Zone + ZFS Zone Path Permitted o x86: New GRUB findroot Command o x64: Support for 256 Processors April 28, 2020 Dusan Baljevic 67
Appendix Solaris10 Update 6 (continued)
* System Performance Enhancements o SPARC: Solaris SPARC Boot Architecture Redesigned o x86: Kernel Support for Intel SSSE3, SSE4.1, SSE4.2, and AMD SSE4A * Security Enhancements o Separation of Duty Enforcement Through the Solaris Management Console o SHA256/SHA512 crypt(3C) Plug-in o pam_list Module * Desktop Enhancements o SPARC: Adobe Reader 8.1.2
o Flash Player 9.0.124.0
* Networking Enhancements o Communication Protocol Parser Utilities o SIP End-to-end Traffic Measurements and Logging * Device Management Enhancements o Faulty Device Retirement Feature o MPxIO Support for Hitachi Adaptable Modular Storage Series Arrays April 28, 2020 Dusan Baljevic 68
Appendix Solaris10 Update 6 (continued)
* Driver Enhancements o x86: NVIDIA ck804/mcp55 SATA Controller Driver o x86: LSI MegaRAID SAS Controllers Driver o ixgbe Driver. The ixgbe is a 10 Gigabit PCI Express Ethernet driver that supports Intel 82598 10 Gigabit Ethernet controller.
o SPARC: Support for aac Driver * Additional Software Enhancements o Perl Database Interface and Perl PostgreSQL Driver o PostgreSQL 8.3
* Language Support Enhancements o IIIMF Hangul Language Engine. The Hangul LE (Language Engine) is a new Korean input method.
* Freeware Enhancements o C-URL - The C-URL Wrappers Library o Libidn - Internationalized Domain Library o LibGD - The Graphics Draw Library o TIDY HTML Library April 28, 2020 Dusan Baljevic 69
Appendix Sun Versus NetApp * (part 1)
http://www.groklaw.net/article.php?story=20081007160707649&query=NetApp-Sun • • After NetApp filed its lawsuit to halt adoption of Sun's open source ZFS technology, Sun responded by filing re-examination requests with the PTO citing the extensive amount of highly relevant prior art that was not disclosed or considered when NetApp originally filed its patents. The patent office clearly agreed with the relevance of this prior art, as demonstrated by its rejection of the claims across all of the re-examinations. Of these patents, three have been described by NetApp as "core" (US Patent Nos. 6,857,001; 6,892,211; and 5,819,292). Here's the current status of each of them: NetApp Patent No. 6,857,001 - The PTO rejected all 63 claims of the patent based on 10 prior art references provided by Sun. In addition, the trial court has agreed to remove that patent from the litigation for now pending the final re examination by the PTO. NetApp Patent No. 6,892,211 - The PTO rejected all 24 claims of the patent based on 12 prior art references provided by Sun. There is currently a request pending before the trial judge to stay this patent from the litigation as well. NetApp Patent No. 5,819,292 – The PTO has rejected all of the asserted claims of this patent relying on at least two separate prior art references out of the many provided by Sun. (The examiner felt that to consider the other references would be "redundant".) April 28, 2020 Dusan Baljevic 70
Appendix Sun Versus NetApp (part 2)
• • The Markman Order: In summary, the court agreed with Sun's interpretation on six of the disputed terms (two of which the court adopted with slight modification) and with NetApp on one. As to the remaining terms, the court either formulated its own interpretation or requested that the parties propose a further construction (i.e. definition). Most significantly, the Court found each of the asserted claims in NetApp's 7,200,715 patent relating to RAID technology to be "indefinite" meaning that someone with experience in this area of technology could not understand the limits of the claimed invention. With regard to NetApp's '715 patent, the court agreed with Sun's position that the claims of the patent are flatly inconsistent with and impossible under the teaching of the patent specification. In effect, unless NetApp appeals and this finding is reversed, the '715 patent is effectively invalidated in this case and against others in the future.
In addition, the Court's findings on the terms "server identification data", "domain name", "portion of a communication" "element of a communication" and "completing a write operation within a local processing node" further strengthen our position that the processors, network interface and systems management software used across NetApp's product line infringe Sun's patents. April 28, 2020 Dusan Baljevic 71
Thank You!
© 2008 Dusan Baljevic The information contained herein is subject to change without notice