Senior Big Data Engineer
Epam
Kharkiv, Ukraine
2 днів тому

DESCRIPTION

As a Senior Big Data Engineer , you will live on the front lines of the products we create, and build features used by millions of people every day.

Our customer is America's largest online lending marketplace. It connects borrowers with multiple lenders so they can find the best deals on loans, credit cards, deposit accounts, insurance and more.

Our project is about developing a new ETL pipelines and support existing ones.

Project technologies and tools

  • Cloud : AWS(EMR, S3, Athena, Glue etc.)
  • Tools and Frameworks : Hortonworks DataPlatform and DataFlow, Apache Yarn, Maven
  • Databases : Apache HBase, Apache Hive, Microsoft SQL Server
  • Processing : Apache Spark, Apache Kafka, Apache NiFi
  • Languages : Scala, Bash, Groovy
  • CI / CD : Jenkins
  • Responsibilities

  • Writing new and optimizing performance / codebase of existing ETL Spark jobs
  • Planning and developing modules for integration of external data warehouses (from acquired external businesses)
  • Writing new and optimizing existing Jenkins pipelines
  • Data validation and data discrepancy reporting
  • Bug reporting
  • Writing new and refactoring existing NiFi data flows
  • Re-writing batch NiFi flows into Spark jobs to improve performance
  • Providing overall improvements to development process like relentless refactoring of growing codebase and applying new approaches in time-consuming development operations
  • Developing and maintaining project documentation
  • Investigation and requirements clarification
  • Requirements

  • 3+ years of experience in Java or Python
  • Experience with HIVE, Spark will be a plus
  • Experience with Apache Hadoop will be a plus
  • Strong communication skills
  • Good spoken and written English
  • We offer

  • Competitive compensation depending on experience and skills
  • Individual career path
  • Unlimited access to LinkedIn learning solutions
  • Social package - medical insurance, sports
  • Compensation for sick lists and regular vacations
  • English classes with native speakers (certified English teachers)
  • Flexible work hours
  • Повідомте про це
    checkmark

    Thank you for reporting this job!

    Your feedback will help us improve the quality of our services.

    Надіслати заяву
    Моя електронна адреса
    Клацнувши по кнопці "# кнопка", я даю згоду neuvoo на обробку моїх даних та надсилання сповіщень електронною поштою, як це детально описано в Політиці конфіденційності neuvoo. Я можу будь-коли відкликати свою згоду або скасувати підписку.
    Продовжити
    Заява