Dataset: Difference between revisions

From Scalelogic Wiki
Jump to navigation Jump to search
migrate>Un-K
No edit summary
No edit summary
 
(26 intermediate revisions by 3 users not shown)
Line 1: Line 1:
Creating a '''DATASET '''requires to define : 
Creating a''' DATASET '''requires to define:


1. '''NAME '''of the dataset. Only "a-z, A-Z, 0-9, . : _ - are allowed. 
*'''NAME''' of the dataset. Only "a-z, A-Z, 0-9, . : _ - are allowed.
*'''DEDUPLICATION''' - controls whether deduplication is in effect for a dataset. By default the deduplication is disabled. It can be enabled by choosing the level of the deduplication:<br/><br/>- ON<br/>- VERIFY<br/>- SHA256<br/>- SHA256, VERIFY


2.&nbsp;'''DEDUPLICATION&nbsp; - '''<span style="font-weight: normal">c</span><span style="font-weight: normal">ontrols whether deduplication is in effect for a dataset. </span>By default the deduplication is disabled. It can be enabled by choosing the level of the deduplication&nbsp;:


&nbsp; &nbsp; - ON


&nbsp; &nbsp; - VERIFY
*'''COPIES''' - controls the number of copies of data stored for this dataset. These copies are in addition to any redundancy provided by the pool, for example, mirroring or RAID-Z. The copies are stored on different disks, if possible. The space used by multiple copies is charged to the associated file and dataset, changing the '''used''' property and counting against quotas and reservations. By default it is set to '''1'''. Other values are '''2''' or '''3'''.
*'''COMPRESSION- '''controls the compression algorithm used for this dataset. By default it is set to '''lz4'''.&nbsp;If the compression is not needed, it can be disabled('''NONE''').
*'''SYNC'''. - ZFS synchronisation. By default it is set to '''standard'''.<br/>'''sync=standard'''<br/>Synchronous file system transactions (fsync, O_DSYNC, O_SYNC, etc) are written out (to the intent log and then secondly all devices written are flushed to ensure the data is stable (not cached by device controllers).<br/>'''sync=always'''<br/>For the ultra-cautious, every file system transaction is written and flushed to stable storage by a system call return. This obviously has a big performance penalty.<br/>'''sync=disabled'''<br/>Synchronous requests are disabled. File system transactions only commit to stable storage on the next DMU transaction group commit which can be many seconds. This option gives the highest performance. However, it is very dangerous as ZFS is ignoring the synchronous transaction demands of applications such as databases or NFS.<br/>Setting sync=disabled on the currently active root or /var file system may result in out-of-spec behavior, application data loss and increased vulnerability to replay attacks.<br/>This option does *NOT* affect ZFS on-disk consistency.


&nbsp; &nbsp; - SHA256
*'''LOG BIAS''' - Provides a hint to ZFS about handling of synchronous requests in this dataset. If logbias is set to '''latency''' (the default), ZFS uses the pool's log devices (if configured) to handle the requests at low latency. If logbias is set to '''throughput''', ZFS does not use the configured pool log devices. Instead, ZFS optimizes synchronous operations for global pool throughput and efficient use of resources.
*'''PRIMARY CACHE''' - controls what is cached in the primary cache (ARC). If this property is set to '''all''', then both user data and metadata is cached. If this property is set to '''none''', then neither user data nor metadata is cached. If this property is set to '''metadata''', then only metadata is cached. The default value is '''all'''.
*'''SECONDARY CACHE''' - controls what is cached in the secondary cache (L2ARC). If this property is set to '''all''', then both user data and metadata is cached. If this property is set to '''none''', then neither user data nor metadata is cached. If this property is set to metadata, then only metadata is cached. The default value is '''all'''.
*'''ACCESS TIME '''- controls whether the access time for files is updated when they are read. Turning this property off avoids producing write traffic when reading files and can result in significant performance gains, though it might confuse mailers and other similar utilities. The default value is '''on'''.
*'''QUOTA''' - Limits the amount of space a dataset and its descendents can consume. This property enforces a hard limit on the amount of space used. This includes all space consumed by descendents, including file systems and snapshots.<br/>Quota can’t be smaller than Reservation.<br/><br/>'''ZX up18 or higher'''
*'''ZFS RESERVATION''' - is an allocation of disk space from the pool that is guaranteed to be available to a dataset. As such, you cannot reserve disk space for a dataset if that space is not currently available in the pool. The total amount of all outstanding, unconsumed reservations cannot exceed the amount of unused disk space in the pool.<br/>Include snapshots and clones<br/>If set than disk space is guaranteed for datasets and their descendents like snapshots and clones<br/>If not set disk space is guaranteed only for a dataset without its descendents (does not include disk space consumed by snapshots and clones) This kind of reservation is also called refreservation in ZFS<br/><br/>'''Additional info about quota and ZFS reservation:'''<br/>You can use the quota property to set a limit on the amount of disk space a file system can use. In addition, you can use the reservation property to guarantee that a specified amount of disk space is available to a file system.<br/><br/>'''ZX up25 or higher'''
*'''recordsize=size'''<br/>Specifies a suggested block size for files in the file system. &nbsp;This property is designed solely for use with database workloads that access files in fixed-size records. &nbsp;ZFS automatically tunes block sizes according to internal algorithms optimized for typical access patterns. For databases that create very large files but access them in small random chunks, these algorithms may be suboptimal. &nbsp;Specifying a recordsize greater than or equal to the record size of the database can result in significant performance gains. &nbsp;Use of this property for general purpose file systems is strongly discouraged, and may adversely affect performance. You can choose between&nbsp;: 4; 8; 16; 32; 64; 128; 256; 512[KiB] and 1MB. Changing the file system's recordsize affects only files created afterward; existing files are unaffected.


&nbsp; &nbsp; - SHA256, VERIFY
[[Category:Help topics]]
 
3.&nbsp;'''COPIES''' - <span style="font-weight: normal">c</span>ontrols the number of copies of data stored for this dataset. These copies are in addition to any redundancy provided by the pool, for example, mirroring or RAID-Z. The copies are stored on different disks, if possible. The space used by multiple copies is charged to the associated file and dataset, changing the '''used''' property and counting against quotas and reservations..&nbsp;By default it is set to '''1'''. Other values are '''2''' or '''3'''.
 
4. '''COMPRESSION - '''<span style="font-weight: normal">c</span><span style="font-weight: normal">ontrols the compression algorithm used for this dataset.</span><span style="font-weight: normal">By default it is set to </span>'''lz4'''<span style="font-weight: normal">.</span>'''&nbsp;'''If the compression is not needed, it can be disabled ('''NONE''').
 
5.&nbsp;'''SYNC. - '''<span style="font-weight: normal">ZFS synchronisation. By default it is set to </span>'''standard'''<span style="font-weight: normal">. </span><br/>
 
'''sync=standard'''
 
<span style="font-weight: normal">Synchronous file system transactions (fsync, O_DSYNC, O_SYNC, etc) are written out (to the intent log and then secondly all devices written are flushed to ensure the data is stable (not cached by device controllers).</span>
 
'''sync=always'''
 
For the ultra-cautious, every file system transaction is written and flushed to stable storage by a system call return. This obviously has a big performance penalty.
 
'''sync=disabled'''
 
<span style="font-weight: normal">Synchronous requests are disabled. File system transactions only commit to stable storage on the next DMU transaction group commit which can be many seconds. This option gives the highest performance. However, it is very dangerous as ZFS is ignoring the synchronous transaction demands of applications such as databases or NFS.</span>
 
<span style="font-weight: normal">Setting </span>'''sync=disabled'''<span style="font-weight: normal">on the currently active root or /var file system may result in out-of-spec behavior, application data loss and increased vulnerability to replay attacks.</span>
 
<span style="font-weight: normal">This option does *NOT* affect ZFS on-disk consistency.</span>
 
'''6. LOG BIAS - ''''''p'''rovides a hint to ZFS about handling of synchronous requests in this dataset. If logbias is set to'''latency''' (the default), ZFS uses the pool's log devices (if configured) to handle the requests at low latency. If logbias is set to '''throughput''', ZFS does not use the configured pool log devices. Instead, ZFS optimizes synchronous operations for global pool throughput and efficient use of resources.
 
'''7. PRIMARY CACHE - '''<span style="font-weight: normal">c</span>ontrols what is cached in the primary cache (ARC). If this property is set to '''all''', then both user data and metadata is cached. If this property is set to '''none''', then neither user data nor metadata is cached. If this property is set to '''metadata''', then only metadata is cached. The default value is '''all'''.
 
'''8. SECONDARY CACHE - '''<span style="font-weight: normal">c</span>ontrols what is cached in the secondary cache (L2ARC). If this property is set to '''all''', then both user data and metadata is cached. If this property is set to '''none''', then neither user data nor metadata is cached. If this property is set to '''metadata''', then only metadata is cached. The default value is '''all'''.
 
'''9. ACCESS TIME''' <font color="#000000">'''-'''</font><font color="#000000"><span style="font-weight: normal">c</span></font><font color="#000000"><span style="font-weight: normal">ontrols whether the access time for files is updated when they are read. Turning this property off avoids producing write traffic when reading files and can result in significant performance gains, though it might confuse mailers and other similar utilities. The default value is</span></font><font color="#000000">'''on'''</font><font color="#000000"><span style="font-weight: normal">.</span></font>
 
<font color="#000000">'''10. QUOTA'''</font><font color="#000000"><span style="font-weight: normal">-</span></font><font color="#000000"><span style="font-weight: normal">Limits the amount of space a dataset and its descendents can consume. This property enforces a hard limit on the amount of space used. This includes all space consumed by descendents, including file systems and snapshots.</span></font><br/><font color="#000000"><span style="font-weight: normal">Quota can’t be smaller than Reservation.</span></font>
 
'''1''''''1''''''.''' '''ZFS ''''''RESERVATION -'''is an allocation of disk space from the pool that is guaranteed to be available to a dataset. As such, you cannot reserve disk space for a dataset if that space is not currently available in the pool. The total amount of all outstanding, unconsumed reservations cannot exceed the amount of unused disk space in the pool.<br/><br/>Additional info about quota and '''ZFS reservation''':<br/>You can use the quota property to set a limit on the amount of disk space a file system can use. In addition, you can use the reservation property to guarantee that a specified amount of disk space is available to a file system. Un-Kh properties apply to the dataset on which they are set and all descendents of that dataset.

Latest revision as of 12:37, 22 September 2020

Creating a DATASET requires to define:

  • NAME of the dataset. Only "a-z, A-Z, 0-9, . : _ - are allowed.
  • DEDUPLICATION - controls whether deduplication is in effect for a dataset. By default the deduplication is disabled. It can be enabled by choosing the level of the deduplication:

    - ON
    - VERIFY
    - SHA256
    - SHA256, VERIFY


  • COPIES - controls the number of copies of data stored for this dataset. These copies are in addition to any redundancy provided by the pool, for example, mirroring or RAID-Z. The copies are stored on different disks, if possible. The space used by multiple copies is charged to the associated file and dataset, changing the used property and counting against quotas and reservations. By default it is set to 1. Other values are 2 or 3.
  • COMPRESSION- controls the compression algorithm used for this dataset. By default it is set to lz4. If the compression is not needed, it can be disabled(NONE).
  • SYNC. - ZFS synchronisation. By default it is set to standard.
    sync=standard
    Synchronous file system transactions (fsync, O_DSYNC, O_SYNC, etc) are written out (to the intent log and then secondly all devices written are flushed to ensure the data is stable (not cached by device controllers).
    sync=always
    For the ultra-cautious, every file system transaction is written and flushed to stable storage by a system call return. This obviously has a big performance penalty.
    sync=disabled
    Synchronous requests are disabled. File system transactions only commit to stable storage on the next DMU transaction group commit which can be many seconds. This option gives the highest performance. However, it is very dangerous as ZFS is ignoring the synchronous transaction demands of applications such as databases or NFS.
    Setting sync=disabled on the currently active root or /var file system may result in out-of-spec behavior, application data loss and increased vulnerability to replay attacks.
    This option does *NOT* affect ZFS on-disk consistency.
  • LOG BIAS - Provides a hint to ZFS about handling of synchronous requests in this dataset. If logbias is set to latency (the default), ZFS uses the pool's log devices (if configured) to handle the requests at low latency. If logbias is set to throughput, ZFS does not use the configured pool log devices. Instead, ZFS optimizes synchronous operations for global pool throughput and efficient use of resources.
  • PRIMARY CACHE - controls what is cached in the primary cache (ARC). If this property is set to all, then both user data and metadata is cached. If this property is set to none, then neither user data nor metadata is cached. If this property is set to metadata, then only metadata is cached. The default value is all.
  • SECONDARY CACHE - controls what is cached in the secondary cache (L2ARC). If this property is set to all, then both user data and metadata is cached. If this property is set to none, then neither user data nor metadata is cached. If this property is set to metadata, then only metadata is cached. The default value is all.
  • ACCESS TIME - controls whether the access time for files is updated when they are read. Turning this property off avoids producing write traffic when reading files and can result in significant performance gains, though it might confuse mailers and other similar utilities. The default value is on.
  • QUOTA - Limits the amount of space a dataset and its descendents can consume. This property enforces a hard limit on the amount of space used. This includes all space consumed by descendents, including file systems and snapshots.
    Quota can’t be smaller than Reservation.

    ZX up18 or higher
  • ZFS RESERVATION - is an allocation of disk space from the pool that is guaranteed to be available to a dataset. As such, you cannot reserve disk space for a dataset if that space is not currently available in the pool. The total amount of all outstanding, unconsumed reservations cannot exceed the amount of unused disk space in the pool.
    Include snapshots and clones
    If set than disk space is guaranteed for datasets and their descendents like snapshots and clones
    If not set disk space is guaranteed only for a dataset without its descendents (does not include disk space consumed by snapshots and clones) This kind of reservation is also called refreservation in ZFS

    Additional info about quota and ZFS reservation:
    You can use the quota property to set a limit on the amount of disk space a file system can use. In addition, you can use the reservation property to guarantee that a specified amount of disk space is available to a file system.

    ZX up25 or higher
  • recordsize=size
    Specifies a suggested block size for files in the file system.  This property is designed solely for use with database workloads that access files in fixed-size records.  ZFS automatically tunes block sizes according to internal algorithms optimized for typical access patterns. For databases that create very large files but access them in small random chunks, these algorithms may be suboptimal.  Specifying a recordsize greater than or equal to the record size of the database can result in significant performance gains.  Use of this property for general purpose file systems is strongly discouraged, and may adversely affect performance. You can choose between : 4; 8; 16; 32; 64; 128; 256; 512[KiB] and 1MB. Changing the file system's recordsize affects only files created afterward; existing files are unaffected.