- Find a mentor
- Web Programming
- Mobile App Programming
- Programming Languages
- Data Science /Engineering
- Database /Operations
- Find a freelancer
- Find developer jobs
Hadoop Big Data Cloud Engineer
I have specialized in Big Data technologies, especially Hadoop technologies like Apache Spark, Flume, HBase, HDFS, Hive LLAP, Impala, etc. This career has lead me into developing applications that implement Machine Learning models, predictive algorithms, NLP algorithms, and ingest large datasets. I'm very well versed in concurrent and parallel programming and am really good with both Object Oriented as well as Functional programming approaches. I really love teaching people and sharing my knowledge. I promise that in the time that I spend mentoring you, I will pour into you as much of my knowledge as I can to give you the best chance possible in the industry.
Central Time (US & Canada) (-06:00)
- 4 years experience
I worked in a fast-paced, goal-oriented Ruby on Rails Agile web app development team at Healthcare IT company for 6-9 months, and then single-handedly maintained a large Ruby on Rails app for a large financial institution for about 2 years. I've also done a few hackathons that involved building a web app for a non-profit in less than 24 hours. For 2 of the 3 hackathons, I organized and lead the team. Here's the end result of some of those challenges: http://principalsconnect.com/ http://adoption.kvc.org/
- 8 years experience
I have 4 years of extensive experience writing Java applications that run in a Hadoop environment. I've developed data pipelines in Apache Crunch, created custom Flume Clients, and also Flume Sources that plugin to the running Flume Agents per the Flume API, and written MapReduce algorithms that process data in the petabytes scale. Some of these Java applications ran as services and used Jersey to expose a web API and Solr to index data sitting in NoSQL storage. Spring is a bit of a heavy-weight. As a result, the various Hadoop engineering teams I've been on have chosen Dagger, rather than Spring, as the dependency injection framework for our applications since it's faster and more lightweight.
- 4 years experience
I have four (4) years of experience using Scala in Hadoop and data engineering environments. I've written a LOT of Scala. Some of the Flume plugins I've written have been written in Scala. I've written a Flume Client that used Scalatra to listen for events using Webhooks and then distributed them to an array of Flume Clients using the Akka concurrency framework. I've extended and refactored a Scala Akka web scraper. I've leveraged ScalaTest and ScalaCheck in practically all of my Scala projects. In my latest projects, I've leveraged the Typelevel/Cats library to take advantage of the categorical types that library provides (which facilitate's a more "Haskell" style to the code architecture). I've also written a few Spark applications in Scala. Some of my smaller projects have involved implementing a PageRank and Collaborative Filtering algorithms, and the most ambitious project I've done on Spark was an NLP classifier. I have a decent amount of experience architecting, profiling, and tuning Spark applications. All of the applications I've written in Scala have been highly concurrent, some of them have leveraged Software Transactional Memory libraries like ScalaSTM to simplify multi-threaded interactions with shared memory.
- 8 years experience
Python is a versatile language, and my experience in it is also very versatile. I've used Python to do data exploration using NumPy and Pandas, I've taken Python code from data scientists and translated it into PySpark or into Scala Spark applications, I've written some systems automation scripts in Python, and written scripts for moving or ingesting data between databases and systems on the petabytes scale. Most of my experience is in Python 2.7, but I am also familiar with legacy Python 2.6 and also Python 3.
- 2 years experience
I used Haskell as my vehicle for learning Functional Programming. Since Haskell is a pure functional language, it would not let me fall back on my old procedural object-oriented habits. I succeeded in learning enough Haskell on my own to become productive in the language. This learning enabled me to pick-up Scala more easily than most other members of the engineering teams I've been a part of. If you are learning Functional Programming or just getting started with Haskell, I can certainly get you to a level where you will be productive in the language.