Pachyderm: Data Versioning, Data Pipelines, and Data Lineage

Cloud Back-End Engineer at Pachyderm

San Francisco Bay Area or Remote / Remote
Job Type
1+ years
Connect directly with founders of the best YC-funded startups.
Apply to role ›
Joe Doliner
Joe Doliner

About the role

About Pachyderm

At Pachyderm, we're building an open-source enterprise-grade data science platform that lets you deploy and manage multi-stage, language-agnostic data pipelines while maintaining complete reproducibility and provenance. Our system, developed with open source roots, shifts the paradigm of data science workflows by providing reproducibility, data provenance, and opportunity for true collaboration. Pachyderm utilizes modern technologies like Docker and Kubernetes to build an entirely new method of analyzing data. Offered both as an in-house solution as well as hosted-service, Pachyderm brings together version-control for data with the tools to build scalable end-to-end ML/AI pipelines while empowering users to use any language, framework, or tool they want. If you want to learn more about our grand vision, read what has become our "manifesto."

Pachyderm is a rapidly growing, early-stage company funded by the top VC’s — Benchmark, Decibel, M12, and YCombinator. Like many modern companies, Pachyderm embraces a “Remote-first” approach to growing our team. It gives us a huge advantage in hiring top talent and diverse talent across the country while giving our team members the flexibility to work from anywhere.

You can check out our product on GitHub because it’s open-source and try our cloud service for free.

The Role Love Docker, Golang, and distributed systems?

Pachyderm is hiring backend systems engineers to help us build out the cloud product -- a scalable data science platform as a service -- Pachyderm Hub. You’ll be solving complex distributed systems problems every day and building a best-in-class platform as a service.

Our customers trust us with your most critical data science projects, hence your primary focus will be the development of highly available and mission-critical systems. You will own the API’s, backend services, and distributed systems for Hub. Your main focus will be the development of new features, improving existing functionality, and improve the health of the services by participating in on-call. You’ll also have direct exposure to our users and you will have a significant influence on our feature/product roadmap.

Pachyderm is just a small team right now, so you'd be getting in right at the ground floor and have an enormous impact on the success and direction of the company and product.

We offer significant equity, full benefits, and all the usual startup perks.


Experience working in backend services, data infrastructure, or distributed systems 2+ years of engineering experience working with complex distributed systems, relational databases, or cloud/web services While it is a bonus, experience with Golang is not a strict requirement. Programming languages are just part of your arsenal and we’ve found that great engineers have no problem learning new tools. Must have strong communication skills when talking about technical concepts Things change quickly as our product develops and breaking down major features into smaller and more easily executable PRs is an imperative skill.

Why you should join Pachyderm