How can I protect my data in HDFS?
What is Isilon and how does it work with HDFS?
In the coming post I will explain how Isilon makes Hadoop so much easier to manage. First I thought I’d cover the basics on Isilon in my Isilon Quick Tips series below.
Hadoop Career
Over a year ago I switched teams to join Dell EMC working on the Data Lake team. One of the platforms I work with is the Isilon Scale-out NAS (Gartner #1 in Scale-out NAS). It’s a really mind blowing system that supports HDFS as a protocol but also NFS, SMB, REST, SWIFT, HTTP, FTP protocols as well. Think of being able to move data into HDFS by just moving a file in your Windows environment. Oh and by the way it scales up to 90 PB of data (talking about BIG DATA).
What makes Isilon so awesome isn’t just the hardware but the software that runs Isilon. OneFS is the software that gives Isilon it’s power to store data at astronomical heights. One file system or OneFS is key to giving developers the ability to access Hadoop data thru HDFS using other protocols. Think about not having to land your data on your machine before ingesting into to HDFS. All of this is possible because OneFS treats HDFS as a protocol not storage system. So data can sit on Isilon, but be read as HDFS.
A huge benefit to using Isilon for HDFS storage is the when replicating data for data protection. I’ll follow up with a blog post dedicated to data protection in Hadoop in the future. Just know Isilon provides that missing piece in Hadoop for replication and data protection. Want to replicate or copy over 20 PB of data? No problem just use SyncIQ in OneFS.
Share the Isilon Knowledge
Along the way on the Data Lake team I’ve acquired some knowledge about managing Isilon clusters and wanted to get it out to the community. All these demos can be done using the Isilon Simulator on your local machine. The demos are meant to be easily consumable and all should be around 5 minutes long with a few outliers that bump up to an hour.
Isilon Quick Tips Videos Links
- Isilon Quick Tips: Demo using SnapShotIQ to retrieve delete files with Windows Shadow Copy
- Isilon Quick Tips: Quick walk through on setting up a one-time SyncIQ job in OneFS
- Isilon Quick Tips: Deep Dive into SyncIQ options for customizing your backup strategy
- Isilon Quick Tips: Setting SmartQuotas to manage capacity on your Isilon Cluster
- Isilon Quick Tips: Learn how to setup an NFS export in OneFS
- Isilon Quick Tips: Changing Password through the Web interface in OneFS 8.0
- Isilon Quick Tips: Setting Up SMB Shares in OneFS
- Isilon Quick Tips: Enabling FTP in OneFS
- Isilon Quick Tips: Compare Snapshots in OneFS
Be sure to subscribe to my YouTube channel to ensure that you never miss an Isilon Quick Tip or other Hadoop related tutorials. As always leave a comment or drop me an email with any ideas you have about new topics or things I’ve missed in my posts.