João Paulo Martins da Rocha
Mentor
Rising Codementor
US$15.00
For every 15 mins
ABOUT ME
Data Engineer
Data Engineer
I’m a self-motivated engineer passionate about data warehousing and data engineering, capable of easily adapting to new environments. I enjoy discussions about technology and creative ways to approach complex problems.
I have extensive experience developing data applications across different industry sectors such as finance, telco, retail and marketing. During my career, I have worked with different data architectures and made use of multiple relational databases, the Hadoop ecosystem and cloud infrastructure.
I hold myself to a high level of attention to detail and thoroughness, I believe in a pragmatic view of benefits for each project I participate in.
Portuguese, Spanish, English
Bucharest (+03:00)
Joined August 2020
EXPERTISE
12 years experience
I've worked with multiple database engines, both relational and non-relational. This includes large Data Warehouses built in Oracle Exa...
I've worked with multiple database engines, both relational and non-relational. This includes large Data Warehouses built in Oracle Exadata, or Teradata and also NoSQL database instances acting as the main repository for transaction data or even supporting stateful transformations within streaming pipelines.
10 years experience
Participated in multiple data warehouse projects that included data modelling, data integration, data analysis and reporting. Implemented...
Participated in multiple data warehouse projects that included data modelling, data integration, data analysis and reporting. Implemented star-schemas, snow-flakes, bespoke models required for specific business requirements.
10 years experience
Multiple projects that included requirements gathering, data analysis, reporting and data integration within different relational databas...
Multiple projects that included requirements gathering, data analysis, reporting and data integration within different relational database engines.
4 years experience
Designed and built data pipelines with batch and real-time integration, mostly coded in Python and Java. Deployed in Hadoop clusters on-p...
Designed and built data pipelines with batch and real-time integration, mostly coded in Python and Java. Deployed in Hadoop clusters on-premises and cloud infrastructure.
3 years experience
Development of components for data pipelines, including ETL and streaming applications.
Development of components for data pipelines, including ETL and streaming applications.
2 years experience
REVIEWS FROM CLIENTS
João's profile has been carefully vetted and approved as a Codementor. Connect with João now, and leave a review for them once you're done!
EMPLOYMENTS
Senior Analyst, Data Engineer
KNEIP
2018-12-01-2020-03-01
- Developed Kafka Streams applications supporting an event-driven architecture with micro services.
- Developed Spark Streaming applicati...
- Developed Kafka Streams applications supporting an event-driven architecture with micro services.
- Developed Spark Streaming applications to consume data from Kafka and load a Fund Data Management data model in HBase.
- Participated in creating CI/CD pipeline for Kafka Streams applications, migrating to containers and Kubernetes orchestration.
- Implemented NiFi processor groups to integrate data sourced from files via FTP.
- Participated in data modelling for Fund Data Management.
Java
HBase
Apache Spark
View more
Java
HBase
Apache Spark
Apache Kafka
Kubernetes
Apache Hadoop
CI/CD
Kafka streams
Apache NiFi
View more
Senior Developer
Sagacity Solutions
2017-02-01-2018-11-01
- Developed a bespoke Value Based Management analytics solution for telecommunications company Telstra. The solution, within data warehou...
- Developed a bespoke Value Based Management analytics solution for telecommunications company Telstra. The solution, within data warehouse supported by Teradata, included modules for tenure and cashflow forecasts and also investment data integration.
- Designed and developed configuration-driven product for Value Based Management using Apache Spark, standardization of core algorithms.
- Supported the implementation of Value Based Management product for telecommunications group Tele2 in three different countries, Estonia, Latvia and Lithuania.
- Oversaw Value Based Management product operating in a Software-as-a-Service model using AWS.
- Developed ETL to enable a Revenue Assurance process related to call-center operations for telecommunications company TalkTalk. Used data warehouse supported by Netezza.
Python
SQL
MySQL
View more
Python
SQL
MySQL
Teradata
Apache Spark
Apache Hadoop
Apache Hive
View more
Senior Database Developer
Harte Hanks
2016-01-01-2017-01-01
- Participated in the evolutionary maintenance of a Single Customer View solution, a bespoke multi-source marketing database created for ...
- Participated in the evolutionary maintenance of a Single Customer View solution, a bespoke multi-source marketing database created for VMWare and supported by the Oracle database.
- Designed and developed ETL processes for marketing data using PL/SQL in an Oracle database.
- Reviewed long running SQL queries to optimize performance.
SQL
Oracle
Shell
View more
SQL
Oracle
Shell
Oracle PL/SQL
View more
PROJECTS
KNEIP Digital PlatformView Project
KNEIP
2020
A complete digital platform for Fund Data Management, capable of handling the entire life cycle of Fund Data, integrating multiple source...
A complete digital platform for Fund Data Management, capable of handling the entire life cycle of Fund Data, integrating multiple sources and capable of supporting multiple targets for reporting and publishing in different media. I was a senior data engineer within a cross-functional team responsible for supporting real-time data integration from different sources into a data model capable of supporting multiple products. The platform implemented an event-driven architecture with micro services. I was heavily involved in the development of the data ingest pipeline making use of Apache NiFi, Kafka Streams, Apache Spark and HBase.
Java
HBase
Apache Spark
View more
Java
HBase
Apache Spark
Apache Kafka
Kubernetes
Spark streaming
Apache Hadoop
CI/CD
Kafka streams
Apache NiFi
View more
VBM ProductView Project
Sagacity Solutions
2018
VBM stands for ‘Value Based Management’ and it is a solution which allows businesses to improve their profitability by providing detailed...
VBM stands for ‘Value Based Management’ and it is a solution which allows businesses to improve their profitability by providing detailed customer level insight on which customer delivers the most value. It also looks to create an appropriate and sustainable approach to governance and a culture to focus on long term value creation. I was the lead developer to create a configuration-driven product containing VBM’s core modules, namely tenure and cashflow forecasts and also investment data integration. I also participated in different implementations of this product, delivering client-specific customization and supporting technical deployment in different environments, cloud (AWS) and on-premises Hadoop cluster. The product is written in Python and supported by Apache Spark.
Python
SQL
Apache Spark
View more
Python
SQL
Apache Spark
Apache Hadoop
View more