Thomas Henson

  • Data Engineering Courses
    • Installing and Configuring Splunk
    • Implementing Neural Networks with TFLearn
    • Hortonworks Getting Started
    • Analyzing Machine Data with Splunk
    • Pig Latin Getting Started Course
    • HDFS Getting Started Course
    • Enterprise Skills in Hortonworks Data Platform
  • Pig Eval Series
  • About
  • Big Data Big Questions

5 Things Every Data Team Should Know About Transfer Learning

January 12, 2021 by Thomas Henson Leave a Comment

Did you know there is a technique in Deep Learning (DL) that doesn’t require large data sets and extremely long training times? It’s called Transfer Learning and the fact if you have done any “Hello World” image detection examples or followed my Tensorflow TFLeanring Course you have already used. Data Teams with Data Engineers and Data Scientist should know Transfer Learning. Let us jump into understanding Transfer Learning.

What is Transfer Learning

Transfer Learning is a Machine Learning (ML) technique that focusing on storing knowledge gained from one problem and applying to another related problem. Data Scientist start by building one model then use that same model as the starting point for a new model. Typically the secondary model is a related problem but not always. For example, let us take a model that was built by our friend Dwight to detect images of bears. Now Dwight has used that model to try and figure out how to identify the best bear. Part of that model does image detection it identifies a bear.

Dwight can now share his model with his best friend Jim who wants to build a model to detect dogs. Since the model that Dwight has already been pre-trained Jim can reduce his time in training.

Transfer Learning Bears

Transfer Learning speeds up time to results (does not guarantee results😊)

The second thing you need to know about Transfer Learning speeds up time to results. Think of Transfer Learning as a framework in programing languages. When I was a Web Developer in .NET community, I could build features within my Web Application quicker using .NET functions already built in. For example, connecting to a SQL data could be done using a built-in function called ConnectionString. The complicated details of building that connection to SQL server was abstracted away from me.

Using Transfer learning Data Teams are not starting from scratch which allows models to be built and trained faster. Just as frameworks allow to abstract away complexity, Transfer Learning is similar in that developers can focus on solving higher level problems. In our Bear detector example our friend Dwight has already done the hard work for building an image detector. Now Jim can change a few lines of code and build a new model.

Transfer Learning for Data Reduction

When we think of Deep Learning large data sets are what comes to mind. Transfer Learning allows Data Scientist to use smaller data sets to train models. By utilizing models already built for one task the model can then be retooled to solve a different problem. In our previous example of an image detector for Bears. How much data would need to be applied to create a new model to identify dogs? How about the Jetson Nano thumbs up or down project?

One area being impacted by Transfer Learning is Healthcare. Pretrained models are huge in helping with Healthcare models. For example, let us say there is a specific lung image detection model that is trained 80% of the way this is called a pretrained model. Data Teams can use this model to apply to their problem to take it the remaining 20% of way to train. Imagine one model trained to detect scar tissue can be used to detect other complex lung issues like Pneumothorax, Cancer, COPD, and more.

Transfer Learning

Most Computer Vision Already Incorporates Transfer Learning

For many reasons we have already discussed Object detection incorporated Transfer Learning. Edge detections is already designed.  An Edge is the sharp contrast in a image. For example, the below is a photo of a Jim from the Office, notice where his brown tie meets his yellow shirt? This would be an edge. Tensorflow and other Deep Learning Frameworks come with functions ready to do object detection. Those function already incorporate models that can detect edges in images.

Transfer Learning

One example is in the Jetson Nano Getting Started Project where you can build a model to detect Thumbs Up or Thumbs Down. Out of the box we just use the pretrained model and add our data. For this model we are adding our own images of thumbs up and thumbs down. Using Transfer Learning allows for Jetson Nano users to quick build an image detection with minimal coding and data.

NVIDIA has a Transfer Learning Toolkit in it is 2nd Generation

We all know here at Big Data Big Questions we love the NVIDIA team. Well at NVIDIA’s GPU Cloud or NGC they have catalog of Deep Learning frameworks like we have just talked about. Whether you are looking to train a model for healthcare with their Clara Framework or Natural Language Processing (NLP) with BERT. Many of these models come pretrained to apply your data to solve your problem. Here is the NVIDIA official statement on the NVIDIA Transfer Learning Toolkit:

To enable faster and accurate AI training, NVIDIA just released highly accurate, purpose-built, pretrained models with the NVIDIA Transfer Learning Toolkit (TLT) 2.0. You can use these custom models as the starting point to train with a smaller dataset and reduce training time significantly. These purpose-built AI models can either be used as-is, if the classes of objects match your requirements and the accuracy on your dataset is adequate, or easily adapted to similar domains or use cases.

By using NVIDIA’s TLT 2.0 data teams can reduce development by up to 10X. Even cutting development times in half is a huge game changer for A.I. development.

Wrapping Up Transfer Learning

Transfer Learning is a powerful technique within Deep Learning for helping put models into production faster and with smaller data sets. The key application of Transfer Learning is building off previous training just like we do as humans. The first time I learned to program with Java was hard! Object-Oriented programing was new to me. However, over time I got better, then when I switched to C# for it was a lot easier to take in the concepts and learn. See I was building off my previous training in Java to learn C#.

Want More Data Engineering Tips?

Sign up for the Big Data Big Questions newsletter to be sure and never miss a post or YouTube Episode  of Big Data Big Question where I answer questions from the community about Data Engineering questions.

Related

Filed Under: Deep Learning Tagged With: Deep Learning, Transfer Learning

Subscribe to Newsletter

Archives

  • February 2021 (2)
  • January 2021 (5)
  • May 2020 (1)
  • January 2020 (1)
  • November 2019 (1)
  • October 2019 (9)
  • July 2019 (7)
  • June 2019 (8)
  • May 2019 (4)
  • April 2019 (1)
  • February 2019 (1)
  • January 2019 (2)
  • September 2018 (1)
  • August 2018 (1)
  • July 2018 (3)
  • June 2018 (6)
  • May 2018 (5)
  • April 2018 (2)
  • March 2018 (1)
  • February 2018 (4)
  • January 2018 (6)
  • December 2017 (5)
  • November 2017 (5)
  • October 2017 (3)
  • September 2017 (6)
  • August 2017 (2)
  • July 2017 (6)
  • June 2017 (5)
  • May 2017 (6)
  • April 2017 (1)
  • March 2017 (2)
  • February 2017 (1)
  • January 2017 (1)
  • December 2016 (6)
  • November 2016 (6)
  • October 2016 (1)
  • September 2016 (1)
  • August 2016 (1)
  • July 2016 (1)
  • June 2016 (2)
  • March 2016 (1)
  • February 2016 (1)
  • January 2016 (1)
  • December 2015 (1)
  • November 2015 (1)
  • September 2015 (1)
  • August 2015 (1)
  • July 2015 (2)
  • June 2015 (1)
  • May 2015 (4)
  • April 2015 (2)
  • March 2015 (1)
  • February 2015 (5)
  • January 2015 (7)
  • December 2014 (3)
  • November 2014 (4)
  • October 2014 (1)
  • May 2014 (1)
  • March 2014 (3)
  • February 2014 (3)
  • January 2014 (1)
  • September 2013 (3)
  • October 2012 (1)
  • August 2012 (2)
  • May 2012 (1)
  • April 2012 (1)
  • February 2012 (2)
  • December 2011 (1)
  • September 2011 (2)

Tags

Agile AI Apache Pig Apache Pig Latin Apache Pig Tutorial ASP.NET AWS Big Data Big Data Big Questions Book Review Books Data Analytics Data Engineer Data Engineers Data Science Deep Learning DynamoDB Hadoop Hadoop Distributed File System Hadoop Pig HBase HDFS IoT Isilon Isilon Quick Tips Learn Hadoop Machine Learning Machine Learning Engineer Management Motivation MVC NoSQL OneFS Pig Latin Pluralsight Project Management Python Quick Tip quick tips Scrum Splunk Streaming Analytics Tensorflow Tutorial Unstructured Data

Follow me on Twitter

My Tweets

Recent Posts

  • Tips & Tricks for Studying Machine Learning Projects
  • Getting Started as Big Data Product Marketing Manager
  • What is a Chief Data Officer?
  • What is an Industrial IoT Engineer with Derek Morgan
  • Ultimate List of Tensorflow Resources for Machine Learning Engineers

Copyright © 2023 · eleven40 Pro Theme on Genesis Framework · WordPress · Log in

 

Loading Comments...