Data Engineer

Data Engineer Job Description Template

Our company is looking for a Data Engineer to join our team.

Responsibilities:

  • Assemble large, complex data sets that meet functional / non-functional business requirements;
  • Ability to interact directly with end customers;
  • Monitor KPIs for all software deployed in the field and present solutions towards continuous improvements;
  • Understanding of data modeling, data integration, and data representation (metadata, OWL, ontologies);
  • Participate in system monitoring, including through an on-call rotation;
  • Passion to build data systems from the ground up;
  • Create data catalog, data flow diagrams and interprets data results to users;
  • Work in an agile team with emphasis on quality, testability, and automation;
  • Implementation experience with data warehouse architecture & design, ETL design/development, and Analytics;
  • Ability to work on any product on the Google cloud platform;
  • Design and develop data-intensive distributed systems in Python with a focus on data engineering;
  • Perform database queries and database changes needed for the business metrics;
  • Collaborate with multiple cross functional teams such as Product Management, Solution Architecture, Security, and Software Engineering;
  • Developing techniques to analyse and enhance both structured and unstructured data;
  • Build and design highly scalable data pipelines using new generation tools and technologies to ingest data from various systems.

Requirements:

  • Object-oriented/object function scripting languages: Python, Java, C++, Scala, etc;
  • Experience building and optimizing ‘big data’ data pipelines, architectures and data sets;
  • Experience in Containerization (docker/Heroku);
  • Should be well versed with atleast one database(Mysql/Oracle/MongoDB/Postgres);
  • Relational SQL and NoSQL databases, including Postgres and Cassandra;
  • Technology experience: Big data tools: Hadoop, Spark, Kafka, etc;
  • Fluent in programming languages like Nodejs/Java/Python;
  • 4-7 years Python or Java programming;
  • We are looking for an engineer with Bachelor’s/Master’s degree;
  • Strong analytic skills related to working with unstructured data sets;
  • A successful history of manipulating, processing and extracting value from large disconnected data sets;
  • 2-4 years of experience as a data engineer for a product company;
  • Comfortable working in Linux environment;
  • In depth understanding of big data technologies and NoSql databases (Kafka, HBase, Spark, Cassandra, MongoDb etc);
  • Build tools to generate business insights for customer acquisition, operational efficiency and other key business performance metrics.