Dataset: Difference between revisions

From Scalelogic Wiki
Jump to navigation Jump to search
migrate>Un-K
No edit summary
migrate>Un-K
No edit summary
Line 3: Line 3:
1. '''NAME '''of the dataset. Only "a-z, A-Z, 0-9, . : _ - are allowed. 
1. '''NAME '''of the dataset. Only "a-z, A-Z, 0-9, . : _ - are allowed. 


2. '''DEDUPLICATION. '''By default the deduplication is disabled. It can be enabled by choosing the level of the deduplication :
2.&nbsp;'''DEDUPLICATION&nbsp; - '''<span style="font-weight: normal">c</span><span style="font-weight: normal">ontrols whether deduplication is in effect for a dataset. </span>By default the deduplication is disabled. It can be enabled by choosing the level of the deduplication&nbsp;:


&nbsp; &nbsp; - ON
&nbsp; &nbsp; - ON
Line 13: Line 13:
&nbsp; &nbsp; - SHA256, VERIFY
&nbsp; &nbsp; - SHA256, VERIFY


3.&nbsp;'''COPIES - number of copies of the data.&nbsp;'''By default it is set to 1. Other values are 2 or 3.
3.&nbsp;'''COPIES''' - <span style="font-weight: normal">c</span>ontrols the number of copies of data stored for this dataset. These copies are in addition to any redundancy provided by the pool, for example, mirroring or RAID-Z. The copies are stored on different disks, if possible. The space used by multiple copies is charged to the associated file and dataset, changing the '''used''' property and counting against quotas and reservations..&nbsp;By default it is set to '''1'''. Other values are '''2''' or '''3'''.


4. '''COMPRESSION. By default it is set to lz4.&nbsp;'''If the compression is not needed, it can be disabled (NONE).
4. '''COMPRESSION - '''<span style="font-weight: normal">c</span><span style="font-weight: normal">ontrols the compression algorithm used for this dataset.</span><span style="font-weight: normal">By default it is set to </span>'''lz4'''<span style="font-weight: normal">.</span>'''&nbsp;'''If the compression is not needed, it can be disabled ('''NONE''').


5.&nbsp;'''SYNC.'''
5.&nbsp;'''SYNC. - '''<span style="font-weight: normal">ZFS synchronisation. By default it is set to </span>'''standard'''<span style="font-weight: normal">. </span><br/>


'''6. LOG BIAS'''
'''sync=standard'''


'''7. PRIMARY CACHE.'''
<span style="font-weight: normal">Synchronous file system transactions (fsync, O_DSYNC, O_SYNC, etc) are written out (to the intent log and then secondly all devices written are flushed to ensure the data is stable (not cached by device controllers).</span>


'''8. SECONDARY CACHE.'''
'''sync=always'''


'''9. ACCESS TIME.''' The dataset can be also restricted with the Quota and Reservation. The Unit is&nbsp;'''GiB.'''
For the ultra-cautious, every file system transaction is written and flushed to stable storage by a system call return. This obviously has a big performance penalty.


'''10.''' '''ZFS reservation '''is an allocation of disk space from the pool that is guaranteed to be available to a dataset. As such, you cannot reserve disk space for a dataset if that space is not currently available in the pool. The total amount of all outstanding, unconsumed reservations cannot exceed the amount of unused disk space in the pool.<br/><br/>Additional info about quota and '''ZFS reservation''':<br/>You can use the quota property to set a limit on the amount of disk space a file system can use. In addition, you can use the reservation property to guarantee that a specified amount of disk space is available to a file system. Un-Kh properties apply to the dataset on which they are set and all descendents of that dataset.
'''sync=disabled'''
 
<span style="font-weight: normal">Synchronous requests are disabled. File system transactions only commit to stable storage on the next DMU transaction group commit which can be many seconds. This option gives the highest performance. However, it is very dangerous as ZFS is ignoring the synchronous transaction demands of applications such as databases or NFS.</span>
 
<span style="font-weight: normal">Setting </span>'''sync=disabled'''<span style="font-weight: normal">on the currently active root or /var file system may result in out-of-spec behavior, application data loss and increased vulnerability to replay attacks.</span>
 
<span style="font-weight: normal">This option does *NOT* affect ZFS on-disk consistency.</span>
 
'''6. LOG BIAS - ''''''p'''rovides a hint to ZFS about handling of synchronous requests in this dataset. If logbias is set to'''latency''' (the default), ZFS uses the pool's log devices (if configured) to handle the requests at low latency. If logbias is set to '''throughput''', ZFS does not use the configured pool log devices. Instead, ZFS optimizes synchronous operations for global pool throughput and efficient use of resources.
 
'''7. PRIMARY CACHE - '''<span style="font-weight: normal">c</span>ontrols what is cached in the primary cache (ARC). If this property is set to '''all''', then both user data and metadata is cached. If this property is set to '''none''', then neither user data nor metadata is cached. If this property is set to '''metadata''', then only metadata is cached. The default value is '''all'''.
 
'''8. SECONDARY CACHE - '''<span style="font-weight: normal">c</span>ontrols what is cached in the secondary cache (L2ARC). If this property is set to '''all''', then both user data and metadata is cached. If this property is set to '''none''', then neither user data nor metadata is cached. If this property is set to '''metadata''', then only metadata is cached. The default value is '''all'''.
 
'''9. ACCESS TIME''' <font color="#000000">'''-'''</font><font color="#000000"><span style="font-weight: normal">c</span></font><font color="#000000"><span style="font-weight: normal">ontrols whether the access time for files is updated when they are read. Turning this property off avoids producing write traffic when reading files and can result in significant performance gains, though it might confuse mailers and other similar utilities. The default value is</span></font><font color="#000000">'''on'''</font><font color="#000000"><span style="font-weight: normal">.</span></font>
 
<font color="#000000">'''10. QUOTA'''</font><font color="#000000"><span style="font-weight: normal">-</span></font><font color="#000000"><span style="font-weight: normal">Limits the amount of space a dataset and its descendents can consume. This property enforces a hard limit on the amount of space used. This includes all space consumed by descendents, including file systems and snapshots.</span></font><br/><font color="#000000"><span style="font-weight: normal">Quota can’t be smaller than Reservation.</span></font>
 
'''1''''''1''''''.''' '''ZFS ''''''RESERVATION -'''is an allocation of disk space from the pool that is guaranteed to be available to a dataset. As such, you cannot reserve disk space for a dataset if that space is not currently available in the pool. The total amount of all outstanding, unconsumed reservations cannot exceed the amount of unused disk space in the pool.<br/><br/>Additional info about quota and '''ZFS reservation''':<br/>You can use the quota property to set a limit on the amount of disk space a file system can use. In addition, you can use the reservation property to guarantee that a specified amount of disk space is available to a file system. Un-Kh properties apply to the dataset on which they are set and all descendents of that dataset.

Revision as of 12:24, 6 October 2016

Creating a DATASET requires to define : 

1. NAME of the dataset. Only "a-z, A-Z, 0-9, . : _ - are allowed. 

2. DEDUPLICATION  - controls whether deduplication is in effect for a dataset. By default the deduplication is disabled. It can be enabled by choosing the level of the deduplication :

    - ON

    - VERIFY

    - SHA256

    - SHA256, VERIFY

3. COPIES - controls the number of copies of data stored for this dataset. These copies are in addition to any redundancy provided by the pool, for example, mirroring or RAID-Z. The copies are stored on different disks, if possible. The space used by multiple copies is charged to the associated file and dataset, changing the used property and counting against quotas and reservations.. By default it is set to 1. Other values are 2 or 3.

4. COMPRESSION - controls the compression algorithm used for this dataset.By default it is set to lz4. If the compression is not needed, it can be disabled (NONE).

5. SYNC. - ZFS synchronisation. By default it is set to standard.

sync=standard

Synchronous file system transactions (fsync, O_DSYNC, O_SYNC, etc) are written out (to the intent log and then secondly all devices written are flushed to ensure the data is stable (not cached by device controllers).

sync=always

For the ultra-cautious, every file system transaction is written and flushed to stable storage by a system call return. This obviously has a big performance penalty.

sync=disabled

Synchronous requests are disabled. File system transactions only commit to stable storage on the next DMU transaction group commit which can be many seconds. This option gives the highest performance. However, it is very dangerous as ZFS is ignoring the synchronous transaction demands of applications such as databases or NFS.

Setting sync=disabledon the currently active root or /var file system may result in out-of-spec behavior, application data loss and increased vulnerability to replay attacks.

This option does *NOT* affect ZFS on-disk consistency.

'6. LOG BIAS - 'provides a hint to ZFS about handling of synchronous requests in this dataset. If logbias is set tolatency (the default), ZFS uses the pool's log devices (if configured) to handle the requests at low latency. If logbias is set to throughput, ZFS does not use the configured pool log devices. Instead, ZFS optimizes synchronous operations for global pool throughput and efficient use of resources.

7. PRIMARY CACHE - controls what is cached in the primary cache (ARC). If this property is set to all, then both user data and metadata is cached. If this property is set to none, then neither user data nor metadata is cached. If this property is set to metadata, then only metadata is cached. The default value is all.

8. SECONDARY CACHE - controls what is cached in the secondary cache (L2ARC). If this property is set to all, then both user data and metadata is cached. If this property is set to none, then neither user data nor metadata is cached. If this property is set to metadata, then only metadata is cached. The default value is all.

9. ACCESS TIME -controls whether the access time for files is updated when they are read. Turning this property off avoids producing write traffic when reading files and can result in significant performance gains, though it might confuse mailers and other similar utilities. The default value ison.

10. QUOTA-Limits the amount of space a dataset and its descendents can consume. This property enforces a hard limit on the amount of space used. This includes all space consumed by descendents, including file systems and snapshots.
Quota can’t be smaller than Reservation.

1'1'. ZFS 'RESERVATION -'is an allocation of disk space from the pool that is guaranteed to be available to a dataset. As such, you cannot reserve disk space for a dataset if that space is not currently available in the pool. The total amount of all outstanding, unconsumed reservations cannot exceed the amount of unused disk space in the pool.

Additional info about quota and ZFS reservation:
You can use the quota property to set a limit on the amount of disk space a file system can use. In addition, you can use the reservation property to guarantee that a specified amount of disk space is available to a file system. Un-Kh properties apply to the dataset on which they are set and all descendents of that dataset.