Scala/Big Data Software Engineer

avenga

Khmelnytskyi

1 днів тому

Skills and Qualifications :

At least 2 years of a relevant experience for Middle role and 4 years for Senior; Good knowledge and Hands-on Scala Programming;

Proficiency in Spark Scala for technical development and implementation; Data pipeline using Spark Scala; Load disparate data sets by leveraging Kafka consumers;

Good Understanding of Big Data technologies like Hadoop, Spark, Scala, Hive, HBase, Pig, Cascading; Ability to utilize Hive, Spark, Cassandra, Mesos and Kafka;

Good knowledge on Hive and HiveQL; Good Hands on in Hadoop stack (MapReduce, Hive, Sqoop, Oozie); Experience with AWS components and services, particularly EMR, S3, and Lambda;

Good understanding of file formats including JSON, Parquet, Avro, and others; Experience with open source technologies such as Cassandra;

Experience with messaging and complex event-processing systems such as Kafka and Storm; Extensive hands on Data frame and Dataset operations of Spark;

Excellent in writing shell scripts; Experience with application architecture in a big data environment; Experience with AWS technologies or equivalent cloud stack As a hands-on engineer, influence all architecture decisions Build reusable code, with the ability to scale with very large data volumes;

English Upper-Intermediate.

Responsibilities :

Programs, tests, and documents systems in accordance with programming standards and validation procedures; For more senior candidate to serve as technical lead on a project;

Plans and coordinates the complex design, development, implementation, maintenance, and level 4 support of associated systems;

Provides software, system and / or programming solutions across projects and / or technologies to meet internal and external needs;

Engineer systems and / or software, and / or transfer data for internal and external clients; Plans and coordinates peer review of work products such as code, designs, and test plans produced by other team members;

Interact with corporate and project team members to negotiate timelines, responsibilities, and deliverables; Provides expert technical advice and solutions for internal and external clients;

Provides expert review of database development work with regard to programming standards and validation procedures; Provides long term evaluations of systems, company wide;

Meets personal and professional milestones as assigned, and interacts with the project team to organize timelines, responsibilities and deliverables;

Performs other duties as directed.

Nice to have :

Knowledge on Cassandra Architecture and CQL; Knowledge on Java Programming; Data Migration / ETL knowledge.

Повідомте про це

Thank you for reporting this job!

Your feedback will help us improve the quality of our services.