Thomas Henson

  • Data Engineering Courses
    • Installing and Configuring Splunk
    • Implementing Neural Networks with TFLearn
    • Hortonworks Getting Started
    • Analyzing Machine Data with Splunk
    • Pig Latin Getting Started Course
    • HDFS Getting Started Course
    • Enterprise Skills in Hortonworks Data Platform
  • Pig Eval Series
  • About
  • Big Data Big Questions

Isilon Quick Tips: Setting Up SmartQuotas in OneFS

November 1, 2016 by Thomas Henson Leave a Comment

Learn how to setup SmartQuotas with this Isilon Quick tips Episode.

Transcript

Hello and welcome back to another episode of Isilon quick tips today I want to talk about SmartQuotas. Smartquotas gives you the ability to set quotas on how much data is certain directories users or groups can go through so you’ll find it under our file system and smartQuotas is a licensed feature so you do have to have it license i have it enabled on this cluster and very straightforward let’s go ahead and create a storage quota and talk about our options I go saying you have the ability to set it at the directory level and with directories you can actually do it as a directory subdirectory you can filter down just like you can we r CQ policies or your snapshots we also have the option that you do a user quota and those can be applied to all users or you can actually set it down to individual users if you want and then you also have the option to do group quarters and you can set those groups on your group users or walkthrough just setting up one on a directory quota like this is the most popular one and the one that’s most pliable a lot of use cases and so I said we can set those at the directory and sub directory level so we can come in here and set one on our hdfs directory go ahead and set that policy up so far usage account you can actually set it to include your snapshot data and then you can also set it to include that data protection overhead so this is really giving you a true value of how much storage is being taken up by this group of users are these directories and so if you’re thinking about doing it from accounting perspective where you’re charging back to your users you might want to use this kind of accounting we’re saying hey you know it’s it’s not just about the space that you’re taking up and it’s the data protection overhead how we’re protecting the data then also the snapshots of the data as well i’m going to select any of those but we are going to come down to the usage limits and so this is the area where you can actually set those limitations so you can set it up to be no usage limit and what that is that’s just an accounting feature so if you’re saying hey you know what we don’t really want to set limits on users but we do want to know from accounting perspective how much data is being using on this specific smart program in our case it would be all the data that’s being used on ifs for / hdfs directory so how much is are hadoo storage infrastructure taking up let’s talk about setting up a specific usage limit and so we have three different options with the usage limits the first one is the Advisory limit not call this the nice guy limit so if you set an advisory limit this gives you the ability to come in and let’s just say that we know we’re setting up for ten terabytes and you know our users are you know captain 10 terabytes and it’s just going to send notifications whenever they go over that limitation is not going to stop them from riding and if you’re not going to give him any kind of grace period that’s what i call this one a nice guy the next one is the kind of tough guy and that’s our soft limit and so softly that actually gives you a grace period so if you come in here our directory and we can set it just like we were in our advisory and set it to 10 terabytes and say you know what we’re actually going to give him a grace period of you know 14 days and so what that is is going to allow them 14 days through giving them notifications to clear up and get their data back under you know 10 terabytes or you know however your set it up for you to go back in and say okay you know 10 terabytes went over that you know i’m gonna give you five more terabytes or something like that but it’s us just the kind of tough guy was like okay you don’t have to stop right now but at some point in the future you’re going to and you get to set that grace period the next one is what i call the tough guy and that’s the hard limit and so the hard limits not gonna let you writing any day to past it and so you don’t come back in center by threshold and say hey you know anything over 10 but you’re just not allowed to write you have to pick up phone call us we have to you know come in here and actually take that quote and bump it up for you if you need to but it also gives you the option to show the different space available to so one of the things that i recommend if you doing this hard limit for users is to go in and let them be able to see what that hard threshold is versus the size of the cluster because you want them to be able to say hey you know he’s only ten terabytes available now there’s only you know two terabytes left for bumping up to that limit vs if they’re just looking at the size of the cluster they’re probably thinking oh you know we’re good we can keep store and then one day they’re you know they’re not able to save something to do a lot so just my recommendation and then last but not least you have the option for these notifications so you have disabled the quota notifications use you know the fall trolls or you can actually set up your own custom rules so you just go through and create your quote here now you have your quota set up for your hdfs directory well thanks for joining me in this isilon quick tips and i hope to see you again soon

 

Related

Filed Under: Isilon Tagged With: Isilon, OneFS, quick tips

Subscribe to Newsletter

Archives

  • February 2021 (2)
  • January 2021 (5)
  • May 2020 (1)
  • January 2020 (1)
  • November 2019 (1)
  • October 2019 (9)
  • July 2019 (7)
  • June 2019 (8)
  • May 2019 (4)
  • April 2019 (1)
  • February 2019 (1)
  • January 2019 (2)
  • September 2018 (1)
  • August 2018 (1)
  • July 2018 (3)
  • June 2018 (6)
  • May 2018 (5)
  • April 2018 (2)
  • March 2018 (1)
  • February 2018 (4)
  • January 2018 (6)
  • December 2017 (5)
  • November 2017 (5)
  • October 2017 (3)
  • September 2017 (6)
  • August 2017 (2)
  • July 2017 (6)
  • June 2017 (5)
  • May 2017 (6)
  • April 2017 (1)
  • March 2017 (2)
  • February 2017 (1)
  • January 2017 (1)
  • December 2016 (6)
  • November 2016 (6)
  • October 2016 (1)
  • September 2016 (1)
  • August 2016 (1)
  • July 2016 (1)
  • June 2016 (2)
  • March 2016 (1)
  • February 2016 (1)
  • January 2016 (1)
  • December 2015 (1)
  • November 2015 (1)
  • September 2015 (1)
  • August 2015 (1)
  • July 2015 (2)
  • June 2015 (1)
  • May 2015 (4)
  • April 2015 (2)
  • March 2015 (1)
  • February 2015 (5)
  • January 2015 (7)
  • December 2014 (3)
  • November 2014 (4)
  • October 2014 (1)
  • May 2014 (1)
  • March 2014 (3)
  • February 2014 (3)
  • January 2014 (1)
  • September 2013 (3)
  • October 2012 (1)
  • August 2012 (2)
  • May 2012 (1)
  • April 2012 (1)
  • February 2012 (2)
  • December 2011 (1)
  • September 2011 (2)

Tags

Agile AI Apache Pig Apache Pig Latin Apache Pig Tutorial ASP.NET AWS Big Data Big Data Big Questions Book Review Books Data Analytics Data Engineer Data Engineers Data Science Deep Learning DynamoDB Hadoop Hadoop Distributed File System Hadoop Pig HBase HDFS IoT Isilon Isilon Quick Tips Learn Hadoop Machine Learning Machine Learning Engineer Management Motivation MVC NoSQL OneFS Pig Latin Pluralsight Project Management Python Quick Tip quick tips Scrum Splunk Streaming Analytics Tensorflow Tutorial Unstructured Data

Follow me on Twitter

My Tweets

Recent Posts

  • Tips & Tricks for Studying Machine Learning Projects
  • Getting Started as Big Data Product Marketing Manager
  • What is a Chief Data Officer?
  • What is an Industrial IoT Engineer with Derek Morgan
  • Ultimate List of Tensorflow Resources for Machine Learning Engineers

Copyright © 2023 · eleven40 Pro Theme on Genesis Framework · WordPress · Log in