DVA-C02 Preparation Material 2 - Storage

###S3

• S3 Transfer Acceleration + Multipart Upload
• A way to quickly upload compressed files to S3
• S3 Transfer Acceleration leverages Amazon CloudFront’s globally distributed edge locations
• Using Multipart Upload is advantageous for uploading large files and parallel processing.
• Save logs using CloudWatch
• If the source bucket and destination bucket are the same, additional logs are created for the logs recorded in the bucket.
• IAM roles and resource-based policies delegate access to accounts within a single partition
• Use S3 Object Ownership to make the default bucket owner the owner of all objects in the bucket
• Use IAM policies within an Amazon Cognito identity prefix to restrict users from using their own folders in Amazon S3
• Data access control
• Query string authentication, access control list (ACL), bucket policy, identity and access management policy (IAM)
• IAM database authentication only works with MySQL and PostgreSQL
• Replication
• S3 lifecycle operations are not replicated with S3 replication.
• Same-region replication (SRR) and cross-region replication (CRR) are configured at the S3 bucket level, shared prefix level, or object level using S3 object tags.
• S3 event alarm > Lambda trigger > DynamoDB record insertion
• S3 Object Lambda transforms or processes data every time an object is requested.
• Removing PII from data containing customer personal information, for example
• Most cost-effective solution to provide high availability for centralized configuration storage
• Migrate existing .xml files to an S3 bucket and update application code to read and write configuration files to Amazon S3 using AWS SDKs
• Force all data retrieval requests to provide encryption in transit
• Define a resource-based policy on the S3 bucket to deny access when the request meets the condition "aws:SecureTransport": "false"
• Ideal for storing reports with size greater than 1 MB, creating pre-signed URLs with expiration dates, and adding S3 Lifecycle configuration rules to delete old reports
• RDS is expensive
• DynamoDB cannot store large reports
• Macie can automatically detect PII within S3 buckets and protect or hide it

###RDS

• RDS storage auto-scaling
• Helps to automatically adjust storage scale
• Use IAM database authentication - MySQL, PostgreSQL
• Optimized reading of SQL queries is possible by creating a read replica
• IOPS (Input/Output Operation Per Second), storage performance indicator
• Applies to EBS, RDS, EFS resources
• Apply OS updates by performing maintenance in standby mode, then promoting standby to primary mode, and finally performing maintenance on the old primary mode, which becomes the new standby mode.
• I/O operations are not interrupted during automatic backup
• Multi-AZ standby cannot handle read requests
• Synchronously replicated from availability zone to standby instance
• Plans for disaster recovery
• Cross-Region Read Replicas
• Multi-AZ deployment that creates backups in a single AWS Region
• Proxy allows reuse as the connection is made through the proxy rather than having to open the connection every time

Aurora

• How a Lambda function safely retrieves data from an Amazon Aurora database within the same VPC
• Assign different security groups (SG1, SG2) to the Lambda function and Aurora database.
• Add an inbound rule to SG1 to allow TCP traffic for port 3306.
• There are different answers, but some raised the issue that there is no outbound setting for SG2.

DynamoDB

• Fully managed, serverless, key-value NoSQL database
• For efficient queries, create a table with the partition key and sort key as the primary key, and set the partition key and sort key as the global secondary index (GSI).
• LSI cannot be created from an already existing table.
• Check Lambda in advance to avoid duplicate processing requests
• For access, you need to create an IAM role with the AmazonDynamoDBReadOnlyAccess policy and apply it to the EC2 profile.
• Calculate units
• WCU uses 1 unit per second for items up to 1KB
• In Eventually Consistent Read, RCU uses 2 units per second for up to 4KB items.
• Throughput of 10 strongly consistent reads per second of 6KB size > 10 units required
• In Strongly Consistent Read, RCU uses 1 unit per second for up to 4KB items.
• In the case of Transaction, twice the CU is required when calculating existing units.
• Connect from internal VPC using VPC Endpoint, without Internet Gateway
• S3 backups can be created using on-demand and point-in-time recovery methods, but S3 cannot be accessed.
• AWS proposes a method of backing up to other resources (Data Pipeline, EMR, Glue)
• Maximum item size is 400KB
• It is recommended to use S3 to store images.
• Employee photos are stored in S3 and metadata information is stored in DynamoDB.
• Conditional writing PutItem, UpdateItem, DeleteItem
• Transactional read and write API table entries as a single operation
• TransactWriteItems - Single or all-or-nothing, idempotent operation.
• GetItem, UpdateItem
• If you set ConsistentRead to true, DynamoDB will read data using strong consistency.
• Data is up to date but load increases
• Applicable when using GetItem, Query, and Scan
• BatchGetItem
• Solution if UnprocessedKeys are included when making a direct batch request from a low-level API
• Increased backoff, provisioned read capacity
• Good means to retrieve specific items
• Identify changes by activating DynamoDB Streams
• There is a way to send queued messages to Lambda using SQS to avoid limiting data flow issues.

ElastiCache

• No permanent data storage, possibility of data loss
• Improve compute-intensive performance
• Improved latency and throughput for read-heavy application workloads
• .ebextensions/
• Redis provides snapshot functions, replication, and supports transactions that Memcached cannot do.
• Use Memcached to store and manage session data
• Store application data using an RDS for MySQL DB instance
• Cluster mode ElastiCache Redis
• A reliable, fully managed caching layer for RDS
• Horizontal expansion possible

###EBS

• Not suitable for serverless environments
• Stage security processing of EBS volumes > KMS, in-flight encryption (TLS, IPsec, SSH, VPN) supported
• Data that is encrypted includes data within the volume, snapshots of the volume, data created from those snapshots, and data moved to other instances.
• When an instance is running using CLI, the DeleteOnTermination property can be changed.
• EBS volumes cannot be accessed simultaneously by hundreds of EC2 instances.
• General Purpose SSD (gp2) volumes scale linearly from a minimum of 100 IOPS (33.33 GiB or less) to a maximum of 16,000 IOPS (5,334 GiB or more) with baseline performance of 3 IOPS per 1 GiB of volume size.
• That means reaching maximum IOPS at 5.3 TiB
• Much more expensive than S3

###EFS

• Amazon EFS is a file storage service available on Amazon compute (EC2, containers, serverless) and on-premises servers.
• Supports tasks hosted on Fargate or Amazon EC2 instances
• Designed to be accessible from other resources
• EFS Standard–IA (Infrequent Access) storage class
• EFS Standard storage class is FS used for frequent access.
• Windows does not work

Serverless Application Repository

• Easily assemble and deploy serverless architecture

CloudFront

• Reduce static content loading time
• Set up HTTPS for both communication between Client and CloudFront and between CloudFront and Backend.
• Only the AWS root user can create CloudFront key pairs, up to 2
• Signed URL
• When streaming a file or video, make it valid only under certain conditions.
• The signer is the entity that manages the public/private key pair as a signer.
• Manageable with CloudFront API
• When creating a signer, the public key is stored in CloudFront and the private key is used to sign part of the URL.
• Signers use their private keys to sign URLs or cookies, and CloudFront uses their public keys to verify the signature.
• The root user can create a key pair and then deliver the private key to the subject (Signer) needed for signing.
• Signed URL takes precedence over Signed Cookie
• Signed Cookie is used to set multiple resource limits
• AWS WAF is a web application firewall that can monitor HTTP and HTTPS requests forwarded to CloudFront and control access to content.
• Using ACM certificates with CloudFront requires certificates in the US East (N. Virginia) Region (us-east-1)

CORS

• How to enable cross-origin access
• <CORSConfiguration> <CORSRule>
• Restrict access to unauthorized domains
• Disable CORS to prevent other unauthorized domains from accessing your API