Data Engineer
PETRONAS ICT
2020-07-01-Present
Projects:
Data Migration to Azure
Working on design and development of ETL pipelines for migrating oil data sets on Azure from differe...
Projects:
Data Migration to Azure
Working on design and development of ETL pipelines for migrating oil data sets on Azure from different source systems.
Key Achievements:
Implemented and supported Talend jobs to process 100 GBs of data from the sources like Excel, flat files, Oracle, etc.
Developed Java routines in Talend Studio for transforming data and enhancement of the process.
Designing, creating, testing, and maintaining the complete data management & processing systems.
Developed SQL Scripts for transforming data as per the end product requirements
Providing accurate estimates for project development and implementation. Work with management to meet those estimates
Perform in-depth analysis of research information for the purpose of identifying opportunities, developing proposals and recommendations for use by management
Python
PostgreSQL
Azure
View more
Python
PostgreSQL
Azure
Amazon S3
Talend
View more
Sr. Software Engineer
Nitor Infotech Pvt. Ltd.
2018-07-01-2020-03-01
Projects: Big Data Integration
Designed end-to-end data solutions with Big Data integration to transform the Accounts Payable data in t...
Projects: Big Data Integration
Designed end-to-end data solutions with Big Data integration to transform the Accounts Payable data in the required format for Data Visualization. Designed ETL pipelines and automated the process with fault tolerance. Technology Stack: Talend Studio, Hadoop – HIVE, HDFS, SQOOP, Spark, Unix, SQL
BI & Data warehousing
Developed an ETL pipeline for data integration on the AWS platform. Instrumental in building strong and robust Data Lake-Data Mart architecture on AWS platform enabling seamless data flow from various sources provisioning unified customer view. Technology Stack: Oracle, PL-SQL, AWS Aurora DB, AWS Glue, Pyspark, Python.
Machine Learning Project
Designed the ML model solution for the Classification problem of Image processing. Developed the ML model in Python to classify the input vehicle image as Dent, Scratch, Tear or Crack, etc for one of the insurance domain clients. Technology Stack: Machine Learning, Python, Tensor-flow.
Python
SQL
PostgreSQL
View more
Python
SQL
PostgreSQL
Talend
Apache Spark
Apache Hadoop
View more
Sr. Software Engineer
PRGX India Pvt. Ltd.
2016-03-01-2018-07-01
Projects:
Hi-Per Migration (Big Data Integration)
Conceptualize and design database solutions and optimize database systems for perfor...
Projects:
Hi-Per Migration (Big Data Integration)
Conceptualize and design database solutions and optimize database systems for performance, reliability, fault-tolerance, and scalability. Support big data analytics and cognition for large agile development projects. Collaborate with account supervisors and project managers to designed end-to-end data solutions and ETL pipelines with Big Data integration to transform the Accounts Payable data in the required format for Data Visualization. Technology Stack: Talend Studio, MSSQL, Hadoop – HIVE, HDFS, SQOOP, Spark, Unix, MySQL
Python
SQL
MySQL
Talend
Apache Hadoop
View more