Responsibilities

• Develop, monitor, and operate the most used and most critical curated data pipeline - Sales Order Data (incl. Post-order information, e.g. shipment, return, payment). This pipeline is processing hundreds of millions of records to provide high-quality datasets for analytical and machine learning use-cases
• Consulting with analysts, data scientists, and product managers to build and continuously improve "Single Source of Truth" KPI for business steering such as the central Profit Contribution measurement (PC II)
• Leverage and improve a cloud-based tech stack that includes AWS, Databricks, Kubernetes, Spark, Airflow, Python, and Scala

Requirements

• 3+ years of relevant experience
• Expertise in Apache Spark along with Spark streaming & Spark SQL
• Good hands on experience with Databricks and delta-lake
• Fluency in Scala programming language
• Good understanding & hands-on experience with CI/CD
• Rich working experience with Github
• Fluency working with AWS landscape
• Ability to build Apache Airflow pipelines
• Upper-intermediate or higher English level, both spoken and written (B2+)

Nice to have

• Presto
• Superset
• Starburst
• Exasol

We Offer

• Competitive compensation depending on experience and skills
• The individual career path
• Social package
• English classes with certified English teachers
• Unlimited access to LinkedIn learning solutions
• Flexible work hours

About EPAM

• EPAM is a leading global provider of digital platform engineering and development services. We are committed to positively impacting our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to learn and grow continuously. You will join a dedicated, creative, and diverse community that will help you discover your fullest potential. EPAM is committed to providing our global team of 60,000+ EPAMers with inspiring careers. EPAMers lead with passion and honesty and think creatively. Our people are the source of our success, and we value collaboration, try always to understand our customers' business, and strive for the highest standards of excellence