Apache Spark and Web Scraping
Used Python, Beautiful soup, and Django to web scrape data from the web
Used Apache Spark in Palantir Foundry to clean, transform, and lo...
Used Python, Beautiful soup, and Django to web scrape data from the web
Used Apache Spark in Palantir Foundry to clean, transform, and load the scraped data into data objects
Designed data ontologies to represent real-world business components
Python
Django
Apache Spark
View more
Python
Django
Apache Spark
View more
A Data Warehouse Project on AWS
Created a data warehouse on AWS RDS using dimensional modeling and receiving operational data from several sources
Used Airbyte to perfor...
Created a data warehouse on AWS RDS using dimensional modeling and receiving operational data from several sources
Used Airbyte to perform data extraction and loading while using Data Build Tool(DBT) to apply SQL transformation for various marts
Used Terraform scripts for Infrastructure as Code(IaC) for all the AWS infrastructure
The final cleaned data was loaded into data marts that supported PowerBI dashboards
PostgreSQL
ETL
Data modeling
View more
PostgreSQL
ETL
Data modeling
Data warehouse
Apache Airflow
DBT
View more