About: The Concept of NSS Quotas

Summary

This article describes the basic concepts and differences between the types of quotas NSS supports.

A quota is a slice of your storage capacity. When a quota is set on a path you are controlling or monitoring the growth of data in that path. There are multiple reasons for using quotas: control growth in order to prevent resource exhaustion, keep folder owners or administrators informed of data growth, trigger clean-up to prevent valuable resources from being consumed by zero-value data, etc.

Different types of policies, using quotas, can be useful to achieve different types of goals, all within the same environment. For this purpose NSS provides different types of quotas and policy configurations. Depending on the circumstances and the desired behaviour, a certain kind of quota should be preferred and applied. This article aims to guide NSS operators in this choice.

Before considering individual quota types It is important to initially understand the two main policy definitions: soft quotas and hard quotas.

SOFT QUOTAS
Soft quotas are an insightful monitoring tool and/or a powerful method of encouraging user compliance. Organizations beginning to introduce User Data Management policies that approach their users often begin with this type of policy; allowing users to continue with their established behaviour, but making them aware if their behaviour is outside of that which is expected, through notifications. Soft quotas are also a very useful method for storage owners to passively monitor user behaviour, without involving users.

HARD QUOTAS
Locking quotas limit the volume of data that can be stored in the managed path. These policies are a powerful reminder to users that the resource is being pro-actively managed and that data must be stored responsibly.

In order to ensure data integrity and prevent excessive system overhead NSS does not interrupt ongoing file operations as file packets are stored, meaning that a save operation into a quota that is not locked is always allowed, even if this operation will cause a locking threshold to be exceeded. Quota paths become locked after a locking threshold has been exceeded.

Quota Types, Usage and Operations

Northern Storage Suite (NSS) offers eight types of quotas, each with different configuration options.

USER QUOTA
This quota type is specific to Direct Attached Storage (DAS) and Storage Area Network (SAN) environments, where NSS is installed directly on the Windows Server.

Usage: User quotas are used to control/monitor the size of a folder and/or to control the amount of data a specific user account can own within that folder (ownership derived from file SID).

Quota monitoring operations: Quota usage levels are updated through CIFS. NSS registers to receive notifications of file operations within quota paths. When a notification is received NSS rescans the target folder, quota usage levels are updated, notifications are sent (when applicable) and the target path is added/removed from the internal lock list (when applicable). 

Quota locking operations: If configured as a hard quota a kernel-mode filter driver is used to prevent file operations in paths that have exceeded locked thresholds; paths that are included in the internal lock-list.  

OBJECT QUOTA
This quota type is the primary quota type used when controlling/monitoring Network Attached Storage (NAS) devices. The interfaces that are used in the function of these quota policies depend on the type of NAS device being managed. The availability of bespoke APIs from the hardware manufacturers have an effect on the type of quota locking mechanism available for hard quota policies.

Usage: Object quotas are used to control/monitor the size of a folder and/or to control the amount of data a specific user account can own within that folder (ownership derived from file SID).

Quota monitoring operations: Quota usage levels are updated through CIFS or through the use of bespoke APIs from the hardware manufacturers (EMC VNX, NetApp Clustered Data ONTAP, HDS/HNAS).

When using CIFS, NSS registers to receive notifications of file operations within quota paths. When a notification is received NSS rescans the target folder, quota usage levels are updated, notifications are sent (when applicable) and the target path is added/removed from the internal lock list (when applicable).

When using CEPA (EMC VNX), FPolicy (NetApp Clustered Data ONTAP) or HDS (HNAS), NSS receives notification of an intended file operation and allows or denies (when the operation is an intended save into a locked quota) that operation. After allowing an operation NSS rescans the target folder, quota usage levels are updated, notifications are sent (when applicable) and the target path is added/removed from the internal lock list (when applicable).

Quota locking operations: If configured as a hard quota then bespoke APIs are used to deny save operations in locked quotas. When notification is received of an intended save into a path that is included in the internal lock list, that operation is denied.  

When no bespoke API exists for file operation denial (VNXe, Isilon and generic NAS devices) it is possible to use a legacy method of locking quotas; ACL inversion. This is an invasive and fail-closed method of quota locking that also carries a high-level of network overhead. It should not be used without due caution, nor should it be used without full knowledge of possible negative effects and knowledge of how to manage such effects.

INTERVAL QUOTA
Interval quotas do not monitor file system objects in real-time, instead path status is evaluated according to an interval; quota paths are rescanned according to the interval configured. 

Usage: Interval quotas can be used to periodically update knowledge specific path's state. Drive roots (where there is frequent writing and deleting activity) can be effectively monitored with this quota type, avoiding excessive use of CPU time that would be required if object or user quotas were used at this level.  The use cases for interval quotas are somewhat limited. Northern recommends the use of NSS' reporting capabilities to gain this type of periodically updated overview.

Quota monitoring operations: Quota usage levels are updated through CIFS. When the given interval is reached NSS scans the quota path to establish the current size of that path, quota usage levels are updated, notifications are sent (when applicable) and the target path is added/removed from the internal lock list (when applicable). 

Quota locking operations: When an interval quota becomes locked the earliest it can automatically be unlocked is at the end of the configured interval. As such Northern strongly recommends that interval quotas are not configured as hard quotas.

AUTODIR QUOTAS
An AutoDir quota is a policy creation tool. All paths one level below the path of the AutoDir will receive a quota with the definitions of the AutoDir quota; object or user quotas (dependant on target storage device) are spawned for each current and new child folder to the AutoDir path.

Usage: AutoDir quotas are ideally suited for the automatic creation of large numbers of quotas. By specifying \\server\E$\HomeShares in the AutoDir configuration, an operator can create quotas for each and every individual home share below this path. In addition, any new home share created at a later date will receive a quota while the AutoDir quota remains enabled. Note that each quota created by the AutoDir can be individually modified at any time after creation.

Quota monitoring operations: When managing DAS/SAN, the AutoDir quota will create User Quotas (see above). When managing NAS devices the AutoDir quota will create Object Quotas (see above).

Quota locking operations: When managing DAS/SAN, the AutoDir quota will create User Quotas (see above). When managing NAS devices the AutoDir quota will create Object Quotas (see above).

AUTOUSER QUOTAS
An AutoUser quota is a policy creation tool. The path of the AutoUser will be scanned and individual quotas will be created for each user found to own files in that path, also any new user who saves into that path will receive a quota with the settings of the AutoUser quota.

Usage: AutoUser quotas are designed to be used on large shared folders where multiple users are contributing files. For example, the object D:\BigProjects could have an Auto User Quota assigned to it. The Quota size, notification levels, and notification recipients, and other Quota settings would be defined; then every user who has data in that directory will receive a Quota based on the settings for the Auto User Quota. Each and every new user who save data to that directory will also receive a Quota based on those settings. In most cases, there should be only one Auto User Quota per directory.

Quota monitoring operations: When managing DAS/SAN, the AutoUser quota will create User Quotas (see above). When managing NAS devices the AutoUser quota will create Object Quotas (see above).

Quota locking operations: When managing DAS/SAN, the AutoUser quota will create User Quotas (see above). When managing NAS devices the AutoUser quota will create Object Quotas (see above).

AUTOGROUP QUOTAS
An AutoGroup Quota is very similar to the AutoUser quota. The key difference is that the operator can specify that quotas are created only for users who are members of a specific NT Security Group.

Usage: AutoGroup quotas are used to automatically create quotas in project folders, where multiple users from multiple lines of business are storing files, that are more tuned to the individual needs of the target user. For example, one could automatically create large quotas for members of the Marketing group, who typically work with large files, and smaller quotas for members of the HRM group, who do most of their work in Office applications.

Quota monitoring operations: When managing DAS/SAN, the AutoGroup quota will create User Quotas (see above). When managing NAS devices the AutoGroup quota will create Object Quotas (see above).

Quota locking operations: When managing DAS/SAN, the AutoGroup quota will create User Quotas (see above). When managing NAS devices the AutoGroup quota will create Object Quotas (see above).

NOTE: If a user is member of two Groups and AutoGroup quotas are defined for both groups, then two quotas are created and the smallest quota has priority.

FILEBLOCK POLICIES
A File Block quota is a policy that defines types of files that cannot be saved into a target path.

Usage: File Block policies are used to prevent users from saving unwanted file types into target paths, preventing users from saving PST, MP4 or JPG files in their home shares, for example. To minimize system overhead the files are not parsed prior to being saved, instead NSS will compare the extension of the file to the list of configured policies. If there is a concern about users renaming files then NSS' binary file type reporting can be used to manage this behaviour.

FileBlock operations: NSS' kernel-mode filter driver (DAS/SAN) or bespoke APIs from NAS hardware manufacturers (EMC VNX, NetApp Clustered Data ONTAP, NetApp 7-Mode, HDS/HNAS) are used to monitor file operations, looking for attempts to save prohibited file types.

FileBlock enforcement: When an attempt is made to save a file that has an extension matching a configured block policy, the operation is denied. If configured, notifications can be sent to the offending user (EMC VNX, NetApp 7-Mode, NetApp Clustered Data ONTAP and HDS/HNAS only).

FILE ALLOW POLICIES
The reverse of a File Block quota, a File Allow quota is a policy that defines the types of files that can be saved into a target path.

Usage: File Allow policies can be used to ensure a folder that is intended to be used for a very specific purpose is only used for this purpose. A folder within a group drive where all and only MP3 files are to be stored, or a directory that a fax service writes to can be locked down with the configuration of a File Allow policy on the extension .FAX.

File Allow operations: NSS' kernel-mode filter driver (DAS/SAN) or bespoke APIs from NAS hardware manufacturers (EMC VNX, NetApp Clustered Data ONTAP, NetApp 7-Mode, HDS/HNAS) are used to monitor file operations, looking for attempts to save file types that do not match the policy.

File Allow enforcement: When an attempt is made to save a file that does not match the policy, the operation is denied. If configured, notifications can be sent to the offending user (EMC VNX, NetApp 7-Mode, NetApp Clustered Data ONTAP and HDS/HNAS only).

Quotas and Storage Devices

The table below provides a reference to identify which quota types are available on which target storage device:

  Device Type
Quota Type DAS/SAN Generic NAS Netapp NAS EMC Celerra NAS HDS/HNAS
User Quota

Yes.

Filter driver

No No No No

Object Quota

No No

Yes

fpolicy server

Yes

cepp service

Yes
Interval Yes Yes Yes Yes Yes
AutoDir quota

Yes

Yes

Yes

Yes

Yes
AutoUser quota Yes No Yes
Yes Yes
AutoGroup quota Yes No Yes
Yes Yes
File Block Yes No Yes Yes Yes
File Allow Yes No Yes Yes Yes

ADDITIONAL RESOURCES

  • KB1778 About: Quota Types in NSS
  • KB2917 How to: Roll out Quotas for Home Directories
  • KB2875 How to: Registering QSComApi.dll on a remote machine
  • KB2892 How to: Using the Northern Quota Server COM API
  • KB2893 What is/are: Northern Quota Server COM API Enums
  • KB3038 How to: How to use QSUnlock
  • KB2823 How to: Export and import Quotas
  • KB Article: 1752

    Updated: 6/27/2016

    • Category
      • Concept

    North America HQ

    NORTHERN Parklife, Inc.
    301Edgewater Place, Suite 100
    Wakefield, MA 01880
    USA

    Voice: 781.968.5424
    Fax: 781.968.5301

    salesUS@northern.net

     

    Additional Contact Information

    EMEA & APAC HQ

    NORTHERN Parklife AB
    St. Göransgatan 66
    112 33 Stockholm
    Sweden

    Voice: +46 8 457 50 00

    salesHQ@northern.net

    Northern Parklife



    ©2018 northern parklife

    privacy statement 
    terms of use