Thomas Henson

Isilon Quick Tips: Setting SMB Shares in OneFS

by Leave a Comment

SMB Shares in Isilon’s OneFS

One of the keys capabilities with Isilon’s OneFS is creating Server Message Block (SMB) shares for network storage. In this episode of Isilon Quick Tips learn how to create SMB shares in OneFS.

 setting smb shares in OneFS

Setting SMB Shares in OneFS

Transcript

Welcome back to another episode  of Isilon Quick Tip and today we ‘re actually going to map  a shared drive  using  SMB so think of your windows  environment being able to set up shares for home  directories to share data  between it  maybe share files between  some sort of organization and today we ‘re going to actually look at how to do that through the protocols

So from  here in the OneFS directory the first  thing to go  to   use will go to our   protocol and yes and   B shares as you can  see from here that we already have one created  this one comes with default this  is that  ifs directory and so the   IFS   directories you know is everything that   in   I salon is under that directory   so if  we were back and we look at our file system storage using the file system explorer  that ‘s pulling up you can see  that under  RIFs directory we have a home  directory  data directory in a couple  different other  ones so we can actually   drive   drill down and look here now   i ‘m   going to   switch over   look   at our file  explorer  and you can see that   i already   have the   IFS directory match here  this  message is that what we ‘re seeing  within the one  of   us   web doing until you  see that i have my data directory  in my  home directory  when my sub directories in this   and  you ‘ll notice here when we  map   the  directory that  i use the IP address of  the know that  i was   using but that ‘s  only because  i don ‘t have the net   set up   on my local machine   but in most your  instances you ‘ll have that  DNS name that you ‘ll use  here   that   actually   you ‘re smart   connect name   let ‘s go through and actually set up  another  FB share so say that   within our  directory that we have a file share that we want to set up for all of our movie date and say that you know we have different movie research that we ‘re doing and  we ‘ve set up specific share  around that  so it ‘s really simple to do   so back   into our   protocols   when you are  going to create  an   SMB share will call  this one movie and this  is just all of  our movie research data  will set it for  everyone to share  and then here   we ‘ll  just actually  say the past so anything   and dr s remember back it ‘s under data   directory   you see that I ‘ve got some  files under here or I ‘m  DB movie  information  in   a couple different other   ones the   director has already   been  created  so we don ‘t need to check this  box here and we ‘re going let  it apply  the default  apo but you can actually set  it so that it doesn ‘t change any existing  this comes in really handy  whenever you ‘re setting up a share on a  subdirectory file or you didn ‘t want to   reverse all the subdirectories under it   just wanted to   make that share available  we ‘re  going to keep the account things  all the same here and we ‘ll go ahead and create that share  and now if we want to   set up that share so that we can see it  and open our file explorer  going here  map network drive going to select  the  specific  drugs just as a   reminder if you  have that smart connect name you want to use that here  I don ‘t have smart connect   setup for dns so I ‘m going use my   future  and now you have that file share  and now   I can start   moving over my movie data  and open up my files and being able to share documents for all of our movie research and that ‘s  how simple it is   the   setup an   SMB   share in   I   salons   oneFS   be  sure to subscribe to my channel so you can get more  Isilon quick tips

New Video Series: Isilon Quick Tips

by Leave a Comment

How can I protect my data in HDFS?

What is Isilon and how does it work with HDFS?

In the coming post I will explain how Isilon makes Hadoop so much easier to manage. First I thought I’d cover the basics on Isilon in my Isilon Quick Tips series below.

Isilon Quick Tips

Hadoop Career

Over a year ago I switched teams to join Dell EMC working on the Data Lake team. One of the platforms I work with is the Isilon Scale-out NAS (Gartner #1 in Scale-out NAS). It’s a really mind blowing system that supports HDFS as a protocol but also NFS, SMB, REST, SWIFT, HTTP, FTP protocols as well. Think of being able to move data into HDFS by just moving a file in your Windows environment. Oh and by the way it scales up to 90 PB of data (talking about BIG DATA).

What makes Isilon so awesome isn’t just the hardware but the software that runs Isilon. OneFS is the software that gives Isilon it’s power to store data at astronomical heights. One file system or OneFS is key to giving developers the ability to access Hadoop data thru HDFS using other protocols. Think about not having to land your data on your machine before ingesting into to HDFS. All of this is possible because OneFS treats HDFS as a protocol not storage system. So data can sit on Isilon, but be read as HDFS.

A huge benefit to using Isilon for HDFS storage is the when replicating data for data protection. I’ll follow up with a blog post dedicated to data protection in Hadoop in the future. Just know Isilon provides that missing piece in Hadoop for replication and data protection. Want to replicate or copy over 20 PB of data? No problem just use SyncIQ in OneFS.

Share the Isilon Knowledge

Along the way on the Data Lake team I’ve acquired some knowledge about managing Isilon clusters and wanted to get it out to the community. All these demos can be done using the Isilon Simulator on your local machine. The demos are meant to be easily consumable and all should be around 5 minutes long with a few outliers that bump up to an hour.

Isilon Quick Tips Videos Links

Be sure to subscribe to my YouTube channel to ensure that you never miss an Isilon Quick Tip or other Hadoop related tutorials. As always leave a comment or drop me an email with any ideas you have about new topics or things I’ve missed in my posts.

Isilon Quick Tips: SyncIQ Deep Dive

by 1 Comment

OneFS offers many options to customize replication policies using SyncIQ. In this episode of Isilon Quick Tips we walk through those options in deep dive into SyncIQ.

 

Transcript

hello and welcome back to another episode of ISILON on quick tips today we’re going to talk about SyncIQ and we’re going to go a little bit deeper than we have in the past the SyncIQ so before it was all about just setting up a one-time replication now let’s talk about some of the options and how we can really customize are seeking jobs so what we’re going to do is I’m going to swing over to policies and we look at a policy of already got created i’m just going to edit that policy that policy is my home shares and so this is all my corporate home directories here in Huntsville and this is something that I’m replicating to my secondary cluster so the first thing i want to talk about I want to talk about some of the differences between copy and synchronization copy is just when you’re specifically moving data from one directory to another directory you’re not caring about if the date has been deleted or if the data is merged so when you’re synchronizing that’s going to be different because synchronizing is actually going to keep your primary cluster and your secondary cluster in sync so a file has been moved to another directory inside of that directory will be replicated that way in your secondary cluster and so will delete if you have a file has been deleted your primary once the scene job completes it will be deleted on your secondary cluster as well some of the areas you would use these is if you want to have a backup of your data but you don’t want to sink it or maybe there’s certain directors you want to pull out that you want to have all of those copy that’s typically when you’re going to see them but most cases you’re going to use the synchronization so in our run job or run job is our option of windows is dropping around so we want to sync you know this is our SyncIQ policy when they’re going to kick off you have a couple different options the one we did before was our manual so that just saying hey you know just going to manually push the button and every time i do that i’m going to sink my going to start that state policy then you also have the option to do it on schedule this is the most common one used so this is says hey you know two times a day three times a day you can set it up however you want that you’re going to set a schedule that you know the state is going to be replicated you know let’s say you do at six in the morning then six in the afternoon you know you doing a weekly monthly or yearly basis another common way to run these is whenever sources modified and so you can have a source of this modified so you know think about if you move a file if you delete a fall anything like that any changes to that directory it’s going to go ahead and modify the source you do have to set a time frame around that so let’s say that you modify a file how fast you want that see you can send delay to happen a few different seconds minutes hours or days so you can say hey you know every time some modified let’s wait a few minutes and then go ahead and never replicated over and then you can also have it set up where whenever a snapshot of the source directory is taken that’s going to run that the state policy job to setting the source directory is very simple right so what directory or want to move in this case i’m moving all my data that’s in ifs is my source directory the cool thing is where you can really customize this job is not only were sent in from different directories or also can include directories or exclude directories you can come in and say hey you know all the directories under data i’m only going to move over RI salon support or you know I’m only one will move over my easy gather / or i can exclude directories and say you know what move everything that’s in data but these two directories here I salon support a nice long gather those are all administrative things that you know not really trying to replicate over the change in a lot there’s not any data in there that’s not recoverable don’t replicate those so that just gives you a lot more control so you can set you said something at a high level in the tree and then only replicate the times that you want to in that tree and not have to worry about okay you have to set up you know 15 different policies because I’ve got different datasets now you can still come back and set one or two policies to replicate over the data and then we talk about some of these advanced settings so you can actually set a priority on this and you can take it from you know a normal default policy or you can have always have it but this specific policies always going to be high level and never just make sure that that priority has been lifted on this job here and also you can set a limit on how long you’re going to keep the reports from these jobs because you know depending on how often these jobs running you start to have a lot of reports and so you have that option there let’s cancel out of this and I’m going to close out i will show you one more thing so we’re talking about setting up those on modified depending on how often those files change that’s how much bandwidth is going to go over your network so if you have some performance concerns about how often or how much data is being problem across you can actually set of performance role in these jobs and so one of the cool things that I really like about it is you can set it on a schedule so you say hey you know I really want to replicate this anytime my name is modified but there are certain business our times maybe or certain times certain days of the week that I’m really want to throttle back and say hey you know what it’s not as a bigger problem during this time you know the rest of the week let’s go ahead and have it you know open throttle there and so you can set a schedule man you can even set some bandwidth rules around the limitations and so you can kinda throttle back saying hey you know always want to be modifying that data but let’s just set a performance rule about how much bandwidth can be taken up and so that’s just a deeper dive on CQ and so you can really see how you can customize and design those same policies to fit whatever kind of rules that you want to have for replicating your data between your eyes on clusters thanks for taking the time to watch and i hope you’ll join me for another episode of ISILON on quick tips

Isilon Quick Tips: Change Password in OneFS 8.0

by Leave a Comment

Transcript

How to change the password in OneFS 8.0 . Alright . oh so we’ll start here are management console want to navigate to the access tab and then the memberships
in roles once when our memberships and rolls we can select a provider to change the admin password I’m going to use the file system from here I can just view and edit the admin you can see I have many different options here so i can change the password I can change my home directory so what I’m going to do is I’m going to add my password and I’m going to select the bin . SH from my unix shell if you don’t select something here you’ll get an error and then I’ll just save my changes and I close this out will log out and we’ll try a new password and it worked and that’s how you change the password and one of this.

Isilon Quick Tips: Setting Up One Time Sync in Isilon’s OneFS

by 1 Comment

Setting Up One Time Sync in Isilon's OneFS

Setting Up One Time Sync in Isilon’s OneFS

Transcript

Ever wanted to set up a SyncIQ job and OneFS? Well follow along in this tutorial here we are in our production cluster you can see that we have the name hints and cluster for this Isilon cluster and in DR cluster we have it named hensondr cluster and so now you can see we have two different ones that were going to try to replicate between now let’s look at some of the file shares to show us the data that we’re actually going to try to move we’re gonna be focusing on move in our research data so if we go to ifs research data our production cluster you can see we have a lot of different files but on DR site we don’t have any of those files so how can we replicate that demo data in the research folder over to our Isilon cluster we could just do a onetime sink so let’s go under the data protection sink IQ will actually navigate to the policies tab and then from here we can just create a simples SyncIQ policy so our policy name here we’re just going to name it research one time in our description will just say hey you know this is a one-time think that were moving the research folder will keep our job type is manual but we could run it on a schedule but we’re just doing a one-time snap here and so now we need to just pick our source directory so remember we’re doing that research data directory we’re going to worry about including or excluding folders we’re just going to sink all of them and now here’s what we’ll put in our target host if you remember back to our dr cluster it’s 10 . 111 that 158 . – 15 and so we’ll just internet target host name remember that your dr cluster and so that’s where we want to sink this so we’re going from production to dr and our target directory we’re just going to dump it into the IFS directory now we can just scroll down to the bottom will keep everything the same and now we are ready to create our policy and so our policy has been created but we still need to kick that job off so we can kick that job off actually just by clicking on this view edit tab and start job so you see we got a green check saying hey our policy job has been scheduled starting now if you see under the summary tab we’re starting to move data it’s going to take some time because we’re moving 10 gigabytes of data so depending on the connection speed and the other jobs that are running but after a few minutes that job will disappear from the summaries tab and you can navigate over to the reports tab and see hey the job was finished took a minute and 31 seconds now let’s verify it so we’re looking on DR cluster we can see that we have demo data folder and we have data in the folder let’s compare it with our demo data on our production cluster so research demo data we can see the folders match up so we just created our first thing I q job remembering your dr cluster all your files are read-only so if you’re going to read or try to ride any folders you can’t do that so in a fail over situation you can switch it to read and write but default it’s going to be only in a read-only mode so if you try to add a folder or any folders you’ll get an error saying that you can’t do that

Isilon Quick Tips: Setting Up NFS Export in OneFS

by Leave a Comment

Another Isilon Quick Tip, where I walk through setting up NFS export in OneFS. Setting up NFS exports is one of the baseline skills needed when working with OneFS.

 

NFS or Network File System is a protocol that allows file based access in a distributed environments. If you are familiar with Windows based systems it’s similar to the SMB protocol but mostly used in Linux/Unix environments. Chances are if you have any Linux/UNIX machines in your environment, you will have a need for using NFS exports.

When Do I Need an NFS Export?

Let’s jump into a couple use cases when you would want to mount an NFS export.

  • Suppose you need extra capacity on your local machine
  • Offload archive data to a network based file system
  • Allow for file sharing abilities for a group of users
  • Manage file access across a in a distributed environment
  • Large data transfers or access to large files across network

Setting Up NFS Export in OneFS

  1. Open OneFS WebGUI
  2. Navigate to Protocols –> UNIX Sharing (NFS)
  3. Click Create Export
  4. Select directory to be shared
  5. Click Create Export
  6. Mount NFS export on Linux/UNIX machine (see commands below)

Transcript

In this episode of Isilon Quick Tips, we’re going to focus on accessing NFS Exports from Isilon’s OneFS.

If you’re accessing Isilon from a Linux machine, you’ll want to make use of the network file system—or NFS—protocol. To do this, we’ll be using mount commands. But first, let’s set up a directory that we want to share out through an NFS export. All this will be done from OneFS web interface and a Linux command line. So, follow along.

From our Protocol tab, we’ll go to the UNIX Sharing or NFS. Within our NFS Exports, we’ll have one defaulted, and that default will be for our IFS directory. Remember, anything in that IFS directory is everything that’s in Isilon. So, that’s one that’s set up by default, but let’s set up one that is specific just to maybe our data. So, I’m going to create an export. We can select our path and we can go down as deep as we want. So, I could go into our data and do something off the home shares or some of the archive data. But I just want to set a top-level directory for just our data path and share this one out. So, I’m going to select ifs/data, and then this is all of our data in Isilon. You don’t have to set a description. It’s just good once you start managing quite a few of these. You want to be like, okay, you can look at it and say, “Hey, okay, that’s actually what this export supports.” With our permissions, we can restrict it to read-only, but we don’t want to do that because we want to be able to make this a working directory. But I will click the “Enable mount access to subdirectories.” So, we’re not only accessing that data – we’re actually accessing everything inside of data and all the subdirectories involved as well. From here, I’ll just create my export, and we get a green check, which means we’re good to go. We now have two exports available. We have one from our IFS and one from our data. So, now we’ll need to jump back into a Linux box and access this from the command line.

So, from our Linux machine, I’m just going to show my directory path. So, I’m here in the root directory and I’ve got some files here. The first thing I want to do—and one of the ways that I always troubleshoot setting up the NFS mounts—is let’s see what mounts are available. So, we’re going to run a showmount command, and what we’re expecting to see is that IFS export, and also the IFS data that we just set up. So, the syntax is just showmount -e, and it’s going to be our Isilon cluster name. So, I’ve just got an IP address for mine. All right, and just like we expected, we see our IFS data, and then our IFS, and those are both accessible to us. Now all we have to do is create a directory to put this in. So, from our root directory, I’m just going to use an mkdir, and let’s set up a directory called our data-share. Just confirm that it’s there. And now we’ll just that mount command. So, mount [Isilon cluster name]:, which export we’re going to use. Remember, we’re going to use the IFS data, but you could use the IFS and mount to all the data that’s in Isilon. Now we need to put the full path of the directory that we want to put it in. So, we just created the data-share, and then now we should be able to run LS on our data-share. And now we see that we have our data in here. So, we have our Isilon support, we have project data, we have that home share data and that archive data – all mounted here.

So, this is a quick way just to set up an NFS export from a Linux machine to your Isilon cluster. Thanks for joining me for another Isilon Quick Tip.