Specialist in GCP services. Experience in Data Engineering since 2010. Applied Data/machine learning engineer using python for the last few years. Understanding data and applying the ML algorithm to solve use cases include data preparation. Also expertise in making pipeline for data related projects.
See more: https://subhadip.ca
● End client - Best Buy.
● Responsible for designing and enhancing or developing warehouse components related to the Supply Cha...
● End client - Best Buy.
● Responsible for designing and enhancing or developing warehouse components related to the Supply Chain Decision Support team and DDO merchandise team along with mentoring leading other team members. ● Designed and developed warehouse components in aws for sources. ● Some business segments are moving from Teradata to GCP Bigquery. Responsible for analyzing data and providing solutions in gcp with equivalent business functionality. ● Technology Used: GCP services. Database-Teradata. ETL-Informatica, Aleryx and Apache Nifi.
● Responsible for designing and enhancing or developing warehouse components related to the DDO merchandise team along with mentoring leading other team members. ● Build various services in GCP to support Data Development Operation. ● Lead an offshore team for the collaborative development works. ● Technology Used: Cloud stack- GCP, OnPrem - Informatica(Powercenter, Data Quality and observability, MDM), Apache NiFi, Teradata.
● Responsible for building the entire pipeline from source to reporting in GCP. 3 ● Migrated entire customer data from AWS MySQL RDS t...
● Responsible for building the entire pipeline from source to reporting in GCP. 3 ● Migrated entire customer data from AWS MySQL RDS to GCP Firestore using python script. ● Entire analytical solution has been built on GCP using various GCP services like Firestore(Firebase), Cloud Storage, Bigquery, Data Flow, Data Transfer Service(DTS), Cloud Run, Cloud Functions, Pub-Sub, Firestore, Cloud SQL, Cloud Build, Airflow(For orchestration), Apache NiFi etc. ● Built ETL flows using DBT: ○ Created multiple models - Table/View/Incremental materialization. Used multiple incremental strategy as well(merge/append etc) ○ Integrated DBT models into Apache Airflow(GCP composer) for orchestration. ○ Generated DBT Documentation and served them. ● Built reporting for the business users in the Google Data Studio from the mart tables created as part of the analytical flow.
● Built Machine learning model to detect frauds in mobility subscription. ● Requirement gathering, understanding data, engineering fea...
● Built Machine learning model to detect frauds in mobility subscription. ● Requirement gathering, understanding data, engineering features for the data science model, building the model and tuning the hyperparameters and automating the process flow is the key role as MLE/DE. ● Technology used - Rapidminer(During POC), Python [PySpark and SciKit-Learn] and MSSQL Server, Alteryx(To explore ETL functionality), Tableau(demonstrate ML output to management)