Part 10. AWS at a glance - DynamoDB

DynamoDB

DynamoDB is a NoSQL database that supports multiple AZs and is fully managed on AWS. You need to consider how to model data and which database to choose depending on the type of application.

Primary key strategies include using a partition key (HASH) and using a partition key (HASH) and a sort key together.
DynamoDB uses PartiQL as its query language.
TTL does not consume WCU. (This is not a part that incurs additional costs) - epoch Converter
Expired items are automatically deleted within 48 hours.
You can use the method of storing large data in S3 and storing the metadata in DynamoDB.
Fully controllable by IAM.
The maximum size of an item in table is 400KB.

Capacity with RCU & WCU

Provisioned provides free tier as default. You must pay for provisioned capacity that you plan ahead of time. On-Demand mode automatically scales up/down based on user actions, so separate provisioning is not required, but it is much more expensive. Recommended for use in unpredictable cases. Switching to another mode is possible every 24 hours.

Table scaling is vertical.

Burst Capacity

Because there is Burst Capacity, RCU (Read Capacity Units) and WCU (Write Capacity Units) can be processed for excess capacity even in Provisioned mode. If ProvisionedThroughputExceededException occurs, it means that Burst Capacity has been used up.

Write Capacity Units

WCU uses 1 unit per second for up to 1KB items.
If 5 items need to be used per second and the item size is 4.5KB, the number of WCUs needed is 25WCU. 4.5KB is calculated by rounding up to 5KB.

Write Type

Concurrent writing -Conditional writing
Atomic writing -Batch write

Eventually Consistent Read vs Strongly Consistent Read

Unlike the basic Consistent Read, this is used when you want to read the data immediately after it has been written. You can use it by setting ConsistentRead to True, but in this case, the RCU costs twice as much, so the waiting time also increases.

In Eventually Consistent Read, RCU uses 2 units per second for up to 4KB items.
In Strongly Consistent Read, RCU uses 1 units per second for up to 4KB items.
If 5 items need to be read per second and the item size is 4.5KB, the number of RCUs needed is 10RCU. 4.5KB requires 2 RCUs for 8KB.

Operations

Basic Operations

Scan - export table premises
DeleteItem
DeleteTable
BatchWriteItem - There is no update function in batches, and batch operations of up to 25 items in Put and Delete and up to 16MB of data in Write are possible.
BatchGetItem - You can get up to 100 items and up to 16MB of data.

Conditional

attribute_exists
attribute_not_exists
attribute_type
contains (for string)
begins_with (for string)
IN (:a, :b), and, between :low and :high
size (string length) You can use it like this: This is an AWS CLI example for deleting an item from the ProductCatalog table. With condition-expression, you can see how a condition has been set so that items with specific conditions can be deleted.

aws dynamodb delete-item \
--table-name ProductCatalog \
--key '{"Id":{"N":"456"}}' \
--condition-expression "(ProductCategory IN (:cat1, :cat2)) and (Price between :lo and :hi)" \
--expression-attribute-values file://values.json

Copying DynamoDBTable

You can run AWS EMR Cluster with AWS Data Pipeline, read tables from it, and write data to S3. Alternatively, you can use Backup, Scan + PutItem / BatchWriteItem.

CLI

You can use --projection-expression in CLI to retrieve some properties.
You can move to the next page using --starting-token.

Index

Local Secondary Index (LSI)

It is an index with one scalar attribute (String, Number, Binary) and can have up to 5 LSIs per table. LSI cannot be changed and Write Capacity is the same as the basic table. A method of indexing data using the same partition key but a different sort key.

Global Secondary Index (GSI)

It is an index with scalar properties (String, Number, Binary) and operates independently of the primary partition key and sort key. Separate read and write capacity must be set, and performance problems may occur if exceeded. GSI's performance limitations may affect the performance of the main table. Although it has its own capacity, because data is replicated asynchronously, if the write capacity is not large enough, GSI write operations may be delayed and this may affect the speed of main table write operations.

Optimistic Locking

It is used when the probability of collisions is considered low, i.e. when the rate of read operations is higher. Normally, we believe there is no possibility of conflict and only check for conflict at the time of modification. Check whether data has changed based on version number or timestamp. To implement Optimistic Locking in AWS DynamoDB, you can add a version number field and update the version number every time it is modified. In case of a crash, a ConditionCheckFailedException exception is raised.

DynamoDB Accelerator, DAX

As a highly available cache for DynamoDB, no special logic conversion is required and hot key issues can be resolved. There is a TTL of 5 minutes per cache. DAX can have up to 5 nodes in a cluster, and a minimum of 3 nodes is recommended (1 per AZ). Elastic Cache is more suitable for storing and retrieving results through aggregation or data processing rather than simple queries like DAX.

No Free Tier
security group port 8111 or 9111

###Streams You can send information about table operations to Kinesis or Lambda as a DynamoDB stream. Like Kinesis Data Stream, it is composed of shards and does not require separate provision because AWS does it automatically.

Using Lambda DynamoDB trigger

Transaction

When multiple tasks are in progress, transaction processing is required so that the previous task can be initialized if it fails in the middle. In this case, twice the operating cost is applied, so twice the CU is required compared to the existing method of calculating CU.

If 5 transactions need to be written per second and the item size is 5KB, the number of WCU required is 50 WCU. Transaction costs twice as much
If 5 transactions need to be read per second and the item size is 5KB, the number of RCUs needed is 20RCU. RCU can handle 4KB. (see RCU calculation above)