Meenansha Sachdeva

NA

Build distributed, reliable and scalable data pipelines using Sqoop and Spark to acquire data from multiple OLTP databases and ingest them into a Data Lake instead of traditional data warehouses. 1. Develop Oozie workflows to orchestrate and schedule the entire process. 2. Develop batch processes using Spring batch to extract, transform and load data into MongoDB from hive tables. 3. Developed Rest APIs using FHIR framework to expose the data stored in MongoDB to clients. 4. Develop Jenkins pipeline to automate the deployment process of docker images into Openshift to support scheduled and unscheduled releases. 5. Used Sonar to ensure good unit test code coverage and the code is written up to set quality standards. 6. Developed automated testing framework using Junit and Cucumber.

1. Develop batch processes using Scala and Spark to ingest structured and semi-structured data(XML) into Hive tables. 2. Develop transformation logics using Scala and Spark to prepare data for provisioning to downstream clients. 3. Build data validation tool to ensure data integrity. 4. Developed a data security layer to mask sensitive data in test environment. 5. Develop Oozie workflows to orchestrate and schedule the entire process. 6. Designed Hbase tables to store all the metadata related to the batch jobs. 7. Automate all the manual processes using shell scripting. 8. Developed a centralized logging framework wherein the logs would be written to Kafka topics and later stored to HDFS.