Apache Hudi Expert

Promoted provides ranking-as-a-service to marketplaces and e-commerce apps. We built our own streaming analytics system to join and aggregate user events to power recommendations that are real-time and reactive.

We've implemented a simple v1 data infrastructure that relies heavily on S3, Kafka, Flink, Athena and Glue. We are hiring an Apache Hudi expert to help us build the next version of our data infrastructure on top of Hudi. Hudi will help us support advanced features (e.g. updating output rows), reduce custom code and reduce operational costs (e.g. optimize our blue/green deployments).

This will start as a contract opportunity with an opportunity to convert to full time. Depending on the candidate, we'll pay between $10k and $30k per month. The initial project will last about 1-2 months.

The work will include:

• Review and advise on the current Hudi integration design and plan.
• Help implement the migration of data producers and consumers to the new infrastructure. There will be 2 other engineers working on this too.
• Productionization (monitoring, logging, etc).