Yieldbot is hiring a

Data Pipeline Software Engineer

Maynard, United States

As a member of the Data Pipeline team at Yieldbot, you will be responsible for developing and maintaining our distributed system for near real time data ingestion, archiving, indexing, and aggregation. These systems process billions of transactions per month. Your responsibilities will include managing and scaling multiple Elasticsearch clusters with varying availability requirements, partnering with our Ad Serving team to build a scalable data ingestion pipeline into Elasticsearch using Kafka, and partnering with our Applications teams to expose the data efficiently via APIs for business critical reporting. Yieldbot is a dynamic, growing business, and you will be crucial to our continued success as we scale the company.

What You’ll Do

 

  • Manage and support our Elasticsearch clusters in a highly available, fault tolerant, operational environment
  • Work proactively to identify trends in performance and address issues of scale quickly
  • Work closely with our application teams to understand their needs and provide the appropriate data services/tooling to support those needs

 

What You Bring

 

  • Experience with Elasticsearch, Logstash, and Kibana
  • Experience with virtual machines or containers
  • Experience with run-time metrics measurement and monitoring systems
  • Experience with Amazon Web Services or other cloud computing platforms
  • Experience with Clojure is a plus

 

What Else We’d be Psyched About

  • You love eliminating single points of failure.
  • You always ask “What happens if this fails?”
  • You understand the importance of postmortems and why they should be blameless.
  • You are self motivated and work well in small teams.
  • You are always looking for ways to improve systems and processes.
  • You love sharing your latest hacks and contributing to open source projects

 

Other jobs at Yieldbot