EMRFS S3 Optimized Committer and Committer Protocol for Improving Spark Write Performance - Why and How?
What are EMRFS S3 Optimized Committer and EMRFS S3 Optimized Committer Protocol and how to use and identify if these are working for your Spark Jobs to improve write performance?
Copy-on-Write or Merge-on-Read? What, When, and How?
Copy-on-Write or Merge-on-Read? Optimizing Row-level updates in Apache Iceberg Table by understanding both the approaches and deciding when to use which approach and its impact on the Read and Write speed of the table.
How to identify these using Iceberg Metadata tables on AWS?
Write-Audit-Publish Pattern with Apache Iceberg on AWS using WAP id
Detailed implementation of Write-Audit-Publish (WAP) Data Quality Pattern in AWS using Apache Iceberg WAP ID i.e. for Apache Iceberg < 1.2.0.
Write-Audit-Publish Pattern with Apache Iceberg on AWS using Branches
Detailed implementation of Write-Audit-Publish (WAP) Data Quality Pattern in AWS using Apache Iceberg Braches i.e. for Apache Iceberg > 1.2.0. It also covers the gotchas while using this pattern and using Athena as a query Engine.
PyDeequ - Testing Data Quality at Scale
How to use PyDeequ to test your data quality on AWS
Building Discord Bot with AWS Serverless - Part 3
This blog post is part of the Let's Build Series, where we pick and build an idea.
In
Building Discord Bot with AWS Serverless - Part 2
This blog post is part of the Let's Build Series, where we pick and build an idea.
In
Building Discord Bot with AWS Serverless - Part 1
This blog post is part of the Let's Build Series, where we pick and build an idea.
In
How I cleared AWS Data Analytics Speciality Certification
On 28th August 2023, After preparing for like 2.5 months, I appeared on AWS Certified Data Analytics Speciality Certification
