Big Data Engineer Duties & Responsibilities

The role and function of a Big Data Engineer includes the following duties and responsibilities:

Responsible for Hadoop development
Implementation including loading from disparate data sets, preprocessing using Hive and Pig.
Scope and deliver various Big Data solutions
Ability to design solutions independently based on high-level architecture.
Manage the technical communication between the survey vendor and internal systems
Maintain the production systems (Kafka, Hadoop, Cassandra, Elasticsearch)
Collaborate with other development and research teams
Building a cloud based platform that allows easy development of new applications

Note that this is not an exhaustive list of Big Data Engineer duties and responsibilities. Job functions for specific Big Data Engineer roles may vary, depending on the industry and type of employer.

Share on