Big Data Engineer with Spark
Codilime
Odesa, UA
49 хвилин тому
source : Just Join IT

Docker (nice to have)

Spark (regular)

Get to know us better

CodiLime is a software and network engineering industry expert and the first-choice service partner for top global networking hardware providers, software providers and telecoms.

We create proofs-of-concept, help our clients build new products, nurture existing ones and provide services in production environments.

Our clients include both tech startups and big players in various industries and geographic locations (US, Japan, Israel, Europe).

While no longer a startup - we have 200+ people on board and have been operating since 2011 we’ve kept our people-oriented culture. Our values are simple :

  • Act to deliver
  • Disrupt to grow.
  • Team up to win.
  • The project and the team

    Our client is a Stealth Startup, SaaS platform for creating / managing MLOps pipelines used for validating given hypotheses and steering base application flows in a feedback loop.

    The product aims at enriching the base application's functionality for customers from different industries. Therefore, apart from the core functionality, the tool also assumes integrations with third-party tools and APIs.

    Our project is supposed to handle large amounts of information and act instantaneously on their base, so big data handling is our bread and butter.

    What else you should know :

  • The team consists of less than 15 people including an architect, project manager, ML developers, and multi-tech and multi-language engineers familiar with numerous APIs, data structuring and processing techniques, presenting output in multiple ways depending on the business need.
  • We use SCRUM / Agile methodology.
  • Our tech stack for the project includes : GCP infra, Kafka, Spark, Hadoop; relational and non-relational databases, integration code pieces written in Java, Python, GoLang, C++, Javascript, SQL.
  • The client is based in the US.
  • We work on multiple interesting projects at a time, so we may invite you to an interview for a different project, if we see that your competencies and profile are well suited for it.

    Your role

    As a part of the project team, you will be responsible for :

  • Preparing a proof-of-concept or new product or delivering changes in small increments
  • Delivering CI changes, automated tests, and deployment configuration
  • Changing the PoC to a full-size feature to be added to the overall product
  • Validating the solution with end-users (demo deployment)
  • Taking an active role in the grooming sessions to deliver the most optimal technical solutions
  • Do we have a match?

    As a Senior Software Engineer you must meet the following criteria :

  • Very good knowledge of Java AND at least one other programming / scripting language (Python, GoLang, C++, C#, Javascript, SQL)
  • Very good understanding of pipeline driven data processing architecture
  • Good knowledge of Spark data processing and analytics
  • Good knowledge of calculation and visualization tools like Jupyter Notebooks
  • Good knowledge of relational and non-relational databases (i.e. Postgres, Elastic, Redis, Hadoop, Mongo)
  • Good knowledge of English (B2 level)
  • We would appreciate the nice-to-haves :

  • Good knowledge of data processing structures and algorithms
  • Experience working with systems which process hundreds of gigabytes of data
  • Experience working with distributed systems
  • More reasons to join us

  • Flexible working hours and approach to work : fully remotely, in the office or hybrid
  • Professional growth supported by internal training sessions and a training budget
  • Solid onboarding with a hands-on approach to give you an easy start
  • A great atmosphere among professionals who are passionate about their work
  • The ability to change the project you work on
  • Повідомте про це
    checkmark

    Thank you for reporting this job!

    Your feedback will help us improve the quality of our services.

    Застосувати
    Моя електронна адреса
    Клацнувши по кнопці "# кнопка", я даю згоду neuvoo на обробку моїх даних та надсилання сповіщень електронною поштою, як це детально описано в Політиці конфіденційності neuvoo. Я можу будь-коли відкликати свою згоду або скасувати підписку.
    Продовжити
    Заява