Why shouldn’t everything be a string? Sometimes it that would make the development world easy, but only in the short-term. Let’s dive into the DynamoDB Data Types to help build stronger Data Engineers.

What is DyanmoDB

DynamoDB is a NoSQL database built by Amazon for both the AWS cloud and off premise. The non-relational database supports both Key-Value and Document models. In this walk through we will focus on the key-value model. If you are interested in learning more DynamoDB Commands checkout 11 DynamoDB CLI commands.

Example DyanmoDB Table

Look back at the example DynanmoDB Table. In that example all the fields were Strings but waht happens if we added a date the college was founded? That field would need to be cast as a date so we could properly query/filter the fields.

DyanmoDB vs. Relational SQL Databases

NoSQL database differ from traditional Relational SQL database in that they are don’t have a schema. Since DynamnoDB doesn’t have a schema portions of the database will differ from a Relational Database. For example DynamoDB doesn’t have rows and columns, but instead has items and attributes. For those with a SQL background reference the table below to see the differences between DynamoDB & SQL databases.

SQL vs. DynamoDB Naming

DynamoDB Data Types

DynamoDB inherits most of it’s data types from Java. If you are familiar with way Java implements data types then the DynamoDB Data types should be easy to understand. However if you aren’t familiar with Java Data types no worries, you can learn through the examples below.

String – Base level data type for storing data inside DynamoDB. UTF 8 standard text with maximum storage up to 400 KB.

Number – Another based level data type for storing numeric data types in DynamoDB. Number type can be used a default type or specific Double, Long, BigDecimal, BigInterger, Float, or Byte.

Double – Number type that is 64 bit and mostly used when decimal point is needed.

Long – Number type and 64-bit primarily used for larger non-decimal number.

Byte – Smallest number with 8 bit. Commonly used for smaller numbers where storage matters

BigDecimal – Larger decimal point that give greater control over rounding action of decimal. Used when developers need to pass in parameters for controlling rounding.

BigInterger – Another developer friendly number type where typical rules can be overwritten to store data types in DynamoDB.

Float – Precision 32 bit number. Commonly used for precise numbers for scientific calculations.

Date – Used for specific UTC coded date and time. Format YYYY-MM-DD-HH:MM:SS

Calendar – Programmatic function that allows developers to express dates from date type by Year, Month, Date, Hour, Minute, or Second.

Sign up for my newsletter to be sure and never miss a post or YouTube Episode of Big Data Big Question where I answer questions from the community about Data Engineering questions.

Maybe it’s my background as a Web Developer, but whenever learning a new development stack I want to build a CRUD(Create, Read, Update, Delete) application first thing. Step one in creating a CRUD application for a database is create a table. In this post let’s see how quickly we use DynamoDB Create table from both AWS console and CLI. 

DynamoDB is a NoSQL database built by Amazon for both the AWS cloud and off premise. The non-relational database supports both Key-Value and Document models. In this walk through we will focus on the key-value model. If you are interested in learning more DynamoDB Commands checkout 11 DynamoDB CLI commands.

Table Example

In this tutorial let’s create a table for global college team registry. We want to track name, shortname, mascot, colors, and location. For example the University of South California would contain the following:

1. Name – University of South California
2. ShortName – USC
3. Mascot – Trojan
4. Colors – Cardinal, Gold
5. Location – Los Angeles, CA

Before loading our data into the table we are going create a table and assign data types to the items.

DynamoDB Create Table From Console

Let’s walk through the steps to creating a table in DynamoDB to track College Teams from the AWS Console. Make sure to log in and navigate to the DynamoDB service.

1. Create Table – Our table name is ‘collegeteam‘. The table name must be unique per AWS region.
2. Partition Key (Primary Key) – Primary Key is a combination of the Partition Key (that we are entering here) and Sort Key (if sort key is used). Since we are using both Parition Key and Sort Key DyanmoDB will hash the keys across the AWS region. For our college teams table our Partition Key for sorting our teams is ‘name’ (string).
3. Add Sort Key -The sort key is additional key to enrich queries associated with the Primary Key. All queries will be associated with the Primary Key.  For the college teams table we have ‘name’ as our primary key, lets’ add the ‘location’ as our sort key.
4. Default Settings/Options – Ready to create if not let’s discuss the advanced settings below
   1. Secondary Indexes – Remember we added additional sort keys tied to our Primary Key? We can use Secondary Indexes when we want to setup indexes independent of the Primary Key. For the college team tables let’s set a secondary index with ‘mascot’ (string) as the Partition Key (Primary Key) and ‘colors’ (string) as the Sort Key. The mascot secondary index will have separate read/write capacity units from the primary index.
   2. Provision Capacity – Default setting is for 5 RCUs (Read Capacity Units) and 5 WCUs (Write Capacity Units). To change provisioned capacity you must disable auto scaling. We will leave the provision capacity with the default setting.
   3. Auto Scaling – Enables DynamoDB Developer to configure how AWS will scale RCUs and WCUs to create elastic tables. We will leave the Auto Scaling enabled and keep the default settings.
   4. Create – Now we are ready to create our table.
   5. Items – After creating table we can navigate to Items and create items. Let’s create our first item for University of South California. Ensure to match the correct data types for each item.
      1. Name (string) – University of South California
      2. ShortName (string) – USC
      3. Mascot  (string)- Trojan
      4. Colors (string) – Cardinal, Gold
      5. Location (string) – Los Angeles, CA
   6. Save – Hit the Save button and now we have just created a college team DyanmoDB table.

Creating DynamoDB Table From CLI

In the first example we used the AWS console to create our college team table, but now let’s use the same example to create a DyanmoDB table from AWS CLI. Before creating the table make sure we have the AWS CLI configured.

1. DyanmoDB Create Table – All of the different options for the college team table are passed into the ‘aws dynamodb create-table’ command.

2. Add Items – Now that our table is created we can use the ‘put-item’ command to populate our DynamoDB table.

Want More Data Engineering Tips?

Sign up for my newsletter to be sure and never miss a post or YouTube Episode of Big Data Big Question where I answer questions from the community about Data Engineering questions.