Sr. Software Engineer, Data Engineering

Job Description

The Hydrolix data lake platform allows data scientists to drastically improve their model training and development velocity. Our platform integrates seamlessly with state-of-the-art model development technologies.

At Hydrolix, you’ll play a pivotal role in building infrastructure solutions targeted at the developer ecosystem for model development. You’ll find creative ways to unlock the power of the Hydrolix platform for a broad set of developers.

Key Responsibilities

• Develop robust, efficient and performant backend systems for querying and processing massive datasets.
• Design, build, and optimize integrations with distributed processing frameworks like Apache Spark.
• Collaborate with various teams including, Core Platform, Data Intake, API, Product and Sales to understand customer needs and develop innovative solutions.
• Contribute to research and engineering decisions, particularly in the areas of integration with Spark environments in AWS EMR, Microsoft Fabric and Snowflake.
• Enhance our data processing capabilities and develop advanced features.
• Work across various components of our platform, from data ingestion pipelines to performance optimization and third-party integrations.

Qualifications and Skills

• Highly experienced backend distributed systems developer, with a proven ability to deliver high quality production software on Linux/Unix platforms.
• Proficiency in one or more compiled programming languages Scala or Java.
• Experience with distributed systems and distributed processing frameworks, such as Spark, Hive, Trino.
• Familiarity with open table formats like Apache Iceberg, Apache Hudi and Delta Lake.
• Familiarity with open storage formats like Parquet and Avro.
• Curious mindset with a demonstrated ability to learn new technologies and programming languages.
• Exceptional analytical, communication, and problem-solving skills.

To apply, please send your resume, cover letter, and any relevant references to careers@hydrolix.io

We look forward to seeing how you can make an impact at Hydrolix.

‍