Pachyderm: Data Versioning, Data Pipelines, and Data Lineage

Staff Engineer at Pachyderm

San Francisco Bay Area or Remote / Remote
Job Type
11+ years
Connect directly with founders of the best YC-funded startups.
Apply to role ›
Joe Doliner
Joe Doliner

About the role

About Pachyderm

At Pachyderm, we're building an open-source enterprise-grade data science platform that lets you deploy and manage multi-stage, language-agnostic data pipelines while maintaining complete reproducibility and provenance. Our system, developed with open source roots, shifts the paradigm of data science workflows by providing reproducibility, data provenance, and opportunity for true collaboration. Pachyderm utilizes modern technologies like Docker and Kubernetes to build an entirely new method of analyzing data. Offered both as an in-house solution as well as hosted-service, Pachyderm brings together version-control for data with the tools to build scalable end-to-end ML/AI pipelines while empowering users to use any language, framework, or tool they want. If you want to learn more about our grand vision, read what has become our "manifesto."

Pachyderm is a rapidly growing, early-stage company funded by the top VC’s — Benchmark, Decibel, M12, and YCombinator. Like many modern companies, Pachyderm embraces a “Remote-first” approach to growing our team. It gives us a huge advantage in hiring top talent and diverse talent across the country while giving our team members the flexibility to work from anywhere.

You can check out our product on GitHub because it’s open-source and try our cloud service for free.

The Role

Love Docker, Kubernetes, Golang, and distributed systems?

As a staff engineer, you will provide technical leadership to our Cloud Infrastructure group. You will have a significant impact on the evolution of Pachyderm Hub’s architecture. Our mission is to build the best in class scalable data science platform as a service. You will be a highly effective developer, mentor, and own the technical direction for the product. You will lead the team by example and bias towards solving problems through design for long term stickiness. You are a passionate and pragmatic technologist who works collaboratively and genuinely invests in leveling up engineers around you. You are excited by building an exceptional platform service and are able to effectively drive technical decisions.

While your primary focus will of course be building the core product, you’ll also have direct exposure to our users and customers. You will have a significant influence on our product roadmap.

Pachyderm is just a small team right now, so you'd be getting in right at the ground floor and have an enormous impact on the success and direction of the company and product.

We offer significant equity, full benefits, and all the usual startup perks.


Experience working in backend services, data infrastructure, or distributed systems 10+ years of engineering experience working with complex distributed systems, relational databases, or cloud/web services Ability to lead and guide excellent engineering teams Ability to assess new technologies and make pragmatic choices that guide us towards a long-term vision Curious and have a design-driven approach to problem-solving Passionate for driving continual improvement initiative on engineering standard methodologies like coding, testing, or monitoring While it is a bonus, experience with Golang is not a strict requirement. Programming languages are just part of your arsenal and we’ve found that great engineers have no problem learning new tools. Must have strong communication skills when talking about technical concepts

Why you should join Pachyderm