Data Engineer Job Description Template
Our company is looking for a Data Engineer to join our team.
Responsibilities:
- Assemble large, complex data sets that meet functional / non-functional business requirements;
- Ability to interact directly with end customers;
- Monitor KPIs for all software deployed in the field and present solutions towards continuous improvements;
- Understanding of data modeling, data integration, and data representation (metadata, OWL, ontologies);
- Participate in system monitoring, including through an on-call rotation;
- Passion to build data systems from the ground up;
- Create data catalog, data flow diagrams and interprets data results to users;
- Work in an agile team with emphasis on quality, testability, and automation;
- Implementation experience with data warehouse architecture & design, ETL design/development, and Analytics;
- Ability to work on any product on the Google cloud platform;
- Design and develop data-intensive distributed systems in Python with a focus on data engineering;
- Perform database queries and database changes needed for the business metrics;
- Collaborate with multiple cross functional teams such as Product Management, Solution Architecture, Security, and Software Engineering;
- Developing techniques to analyse and enhance both structured and unstructured data;
- Build and design highly scalable data pipelines using new generation tools and technologies to ingest data from various systems.
Requirements:
- Object-oriented/object function scripting languages: Python, Java, C++, Scala, etc;
- Experience building and optimizing ‘big data’ data pipelines, architectures and data sets;
- Experience in Containerization (docker/Heroku);
- Should be well versed with atleast one database(Mysql/Oracle/MongoDB/Postgres);
- Relational SQL and NoSQL databases, including Postgres and Cassandra;
- Technology experience: Big data tools: Hadoop, Spark, Kafka, etc;
- Fluent in programming languages like Nodejs/Java/Python;
- 4-7 years Python or Java programming;
- We are looking for an engineer with Bachelor’s/Master’s degree;
- Strong analytic skills related to working with unstructured data sets;
- A successful history of manipulating, processing and extracting value from large disconnected data sets;
- 2-4 years of experience as a data engineer for a product company;
- Comfortable working in Linux environment;
- In depth understanding of big data technologies and NoSql databases (Kafka, HBase, Spark, Cassandra, MongoDb etc);
- Build tools to generate business insights for customer acquisition, operational efficiency and other key business performance metrics.