Senior Big Data Software Engineer
Epam
Kyiv, Ukraine
6 днів тому

DESCRIPTION

Our customer exposes an API for other businesses to be able to access risks associated with a person, e.g. understand their credit score.

Behind the scene there is sophisticated decisioning system and large data volumes. Currently back-end of this API has number of legacy versions, serving hundreds of clients, with individual installations for most clients.

We need to create a data lake for one of the biggest data analytics company working with personal information both domestically and internationally.

  • In a nutshell, this includes replatforming of on-premise Enterprise Data Hub from Hadoop cluster into GCP. Day to day tasks include but not limited to creating spark application which manipulate with data from different sources including Oracle, Google Cloud Storage, BigQuery;
  • creating pipelines via GCP Dataflow; working with Jenkins and AirFlow.

    Requirements

  • 5+ years of experience as a Java Developer
  • Proficiency in data engineering
  • Expertise in Big Data : Hadoop, Spark, Kafka
  • Strong knowledge of Scala
  • Expertise in Microservices Architecture
  • Ability to work with high volumes of data
  • Experience in working with AWS
  • Experience in working with GCP : Dataproc (Apache Spark), Dataflow (Apache Beam), BigQuery, Cloud Storage
  • Good understanding of Design Patterns, Clean Code, Unit testing
  • Experience working in Agile environment
  • Data modelling skills would be a plus
  • Experience in Jenkins, AirFlow with Groovy and Python
  • Excellent written and verbal communication skills
  • Intermediate or higher English level, both verbal and written (B1+)
  • We offer

  • Competitive compensation depending on experience and skills
  • Individual career path
  • Social package - medical insurance, sports
  • Unlimited access to LinkedIn learning solutions
  • Sick leave and regular vacation
  • Partial coverage of costs for certification and IT conferences
  • English classes with certified English teachers
  • Flexible work hours
  • Повідомте про це
    checkmark

    Thank you for reporting this job!

    Your feedback will help us improve the quality of our services.

    Надіслати заяву
    Моя електронна адреса
    Клацнувши по кнопці "# кнопка", я даю згоду neuvoo на обробку моїх даних та надсилання сповіщень електронною поштою, як це детально описано в Політиці конфіденційності neuvoo. Я можу будь-коли відкликати свою згоду або скасувати підписку.
    Продовжити
    Заява