Luis Farias
I am a seasoned Data Engineer with 3 (three) years of hands-on experience. My expertise lies in Python, particularly in constructing and optimizing ETL flows and building robust data pipelines that drive insightful business decisions. I have experience working for companies located in the United States of America (USA). I am a Brazilian professional seeking a contract arrangement that aligns with my preference for a professional relationship resembling employment. I am eager to establish a collaborative partnership where I can contribute my skills and expertise as if I were a dedicated team member.
REVIEWS FROM CLIENTS
Leverage AWS Glue for ETL transformations, employing Glue bookmarks and the data catalog to construct data models encompassing facts, ...
Leverage AWS Glue for ETL transformations, employing Glue bookmarks and the data catalog to construct data models encompassing facts, dimensions, and SCD tables. Gather requirements from diverse systems to formulate data models, facilitating the integration of various business logic within the data infrastructure. Optimize cost efficiency by transitioning data pipelines to Kubernetes (Amazon EKS), replacing the previously utilized EC2 infrastructure, resulting in notable cost reductions. Employ Apache Airflow as the principal orchestrator in the data pipeline, ensuring seamless and efficient workflow management. Collaborate with other teams to comprehend and enhance the overall data architecture, contributing to the continual improvement of data pipelines within the team. Collaborate directly with a cross-functional team based in the USA, fostering effective communication and coordination to ensure the successful implementation and enhancement of data pipelines and models.
Worked on multiple projects in different roles such as AtScale Lead Developer, Data Engineer, Python Developer, Data Scientist, and Sn...
Worked on multiple projects in different roles such as AtScale Lead Developer, Data Engineer, Python Developer, Data Scientist, and Snowflake Developer. Tasks included data modeling, developing measures and dimensions in AtScale Cube, creating documentation, developing end-to-end pipelines, converting HL7 delimited message into JSON and FHIR format, creating ER diagrams, designing and implementing Machine Learning models in Amazon SageMaker, creating dashboards using PowerBI and AtScale, and designing and implementing data models for analytics using a staged approach and target star schema.
Analyzed data from the Wind Turbines, such as active energy and wind speed, to provide insights for the client about the wind farm...
Analyzed data from the Wind Turbines, such as active energy and wind speed, to provide insights for the client about the wind farm's performance. Analyzed an efficient way to save the client's raw data in the Amazon S3 using the proper compression and partitioning techniques. Implemented unit tests throughout the Data pipelines using Pytest. Used Pyenv in Linux to create virtual environments and use different Python versions for different projects. Implemented and improved data visualization using Plotly in the application's backend. Created a data dictionary describing the meaning, unit, and source of the raw data.