As a member of the Data Pipeline team at Yieldbot, you will be responsible for developing and maintaining our distributed system for near real time data ingestion, archiving, indexing, and aggregation. These systems process billions of transactions per month. Your responsibilities will include managing and scaling multiple Elasticsearch clusters with varying availability requirements, partnering with our Ad Serving team to build a scalable data ingestion pipeline into Elasticsearch using Kafka, and partnering with our Applications teams to expose the data efficiently via APIs for business critical reporting. Yieldbot is a dynamic, growing business, and you will be crucial to our continued success as we scale the company.
What You’ll Do
Manage and support our Elasticsearch clusters in a highly available, fault tolerant, operational environment
Work proactively to identify trends in performance and address issues of scale quickly
Work closely with our application teams to understand their needs and provide the appropriate data services/tooling to support those needs
What You Bring
Experience with Elasticsearch, Logstash, and Kibana
Experience with virtual machines or containers
Experience with run-time metrics measurement and monitoring systems
Experience with Amazon Web Services or other cloud computing platforms
Experience with Clojure is a plus
What Else We’d be Psyched About
You love eliminating single points of failure.
You always ask “What happens if this fails?”
You understand the importance of postmortems and why they should be blameless.
You are self motivated and work well in small teams.
You are always looking for ways to improve systems and processes.
You love sharing your latest hacks and contributing to open source projects