Data Engineer
ALTERNATIVE PATH PRIVATE LIMITED
Description :Performing Web Scraping using various scraping techniques and then utilizing Pythons Pandas library for data cleaning and manipulation.Then ingesting the data into a Database/Warehouse, and scheduling the scrapers using Airflow or other toolsRole Overview :The Web Scraping Team at Alternative Path is seeking a creative and detail-oriented developer to contribute to client projects.The team develops essential applications, datasets, and alerts for various teams within the client's organization, supporting their daily investment decisions.The mission is to maintain operational excellence by delivering high-quality proprietary datasets, timely notifications, and exceptional service.We are seeking someone who is self-motivated, self-sufficient, with a passion for tinkering and a love for automation.In your role, you will :- Collaborate with analysts to understand and anticipate requirements.- Design, implement, and maintain Web scrapers for a wide variety of alternative datasets.- Perform Data Cleaning, Exploration, Transformation etc. of scraped data.- Collaborate with cross-functional teams to understand data requirements and implement efficient data processing workflows.- Author QC checks to validate data availability and integrity.- Maintain alerting systems and investigate time-sensitive data incidents to ensure smooth day-to-day operations.- Design and implement products and tools to enhance the Web scraping Platform.Qualifications :Must have :- Bachelor's/masters degree in computer science or in any related field- 2- 4 years of software development experience- Strong Python and SQL/Database skills- Strong expertise in using the Pandas library (Python) is a must- Experience with web technologies (HTML/JS, APIs, etc.)- Proven work experience in working with large data sets for Data cleaning, Data transformation, Data manipulation, and Data replacements.- Excellent verbal and written communication skills- Aptitude for designing infrastructure, data products, and tools for Data ScientistsPreferred :- Familiarity with scraping and common scraping tools (Selenium, scrapy, Fiddler, Postman, xpath) - Experience containerizing workloads with Docker (Kubernetes a plus)- Experience with build automation (Jenkins, Gitlab CI/CD) - Experience with AWS technologies like S3, RDS, SNS, SQS, Lambda, etc (ref:hirist.tech)
$ 1300 per month
...HEROIC.com ) is seeking a senior-level Threat Intelligence Data Engineer - Automated Collection & Dark Web Intelligence to design, build, and operate fully... ...understanding of web protocols, DOM parsing, and adversarial scraping environments Experience with asynchronous,...WebFull timeRemote job- ...Role Description :The current opportunity is for the position of Data Engineer. The right candidate shall be part of our technology team,... ...datasets, developing automation solutions, data extraction, and web scraping to identify actionable investment ideas and drive superior...Web
- ...Location: Pune JobType: full-time The Python Developer (Web Scraping) will be responsible for building, maintaining, and scaling web... ...efficiently handling large volumes of structured and unstructured data. This role focuses on reliability, performance optimization, and...WebFull timeWeekday work
- ...technologies across various domains of IT including Data, Web, Infrastructure, AI, and many others. Our... ...and experience in Data & AI, ML engineering, and solution delivery # Technical... ...research collaborations, and mentorship paths are interconnected. Whether you're building...WebContract workFlexible hours
- ...Who We Are : - Wayfair is on a path to be the worlds largest online... ...development, and launching of new data models, data pipelines, and... ...edge technologies in software engineering, big data, streaming, and cloud... ...the Data Engineering field with web scale data sets.- Expertise with...WebHybrid work
- ...About the team We are experts in data working to make it cost-... ...handling and serve as an escalation path for data incidents to minimize their impact. The Data Engineering Solutions team will work closely... ...of data pipelines and full stack web applications that support...WebFull time
- ...Python Web Scraping Engineer Chennai/Pune About the Job NIQ Digital Shelf is one of Europes fastest-growing companies in the Retail Analytics... ...space. Every day we collect and process over 60 billion data points from web and mobile sources to power real-time market...WebFull timeFlexible hours
- ABOUT THE SENIOR DATA ENGINEER ROLE : WIN is looking to add Senior Data Engineer to our growing team! Ideal candidates are passionate about... ...through hands on analyses and automated tools including web scraping, ETL pipelines and SQL queries.- Conduct in-depth research and...WebHybrid workRemote job
- 1. About the Role :As a Data Engineer for the Data Science team, you will play a pivotal role in enriching and maintaining the organizations... ...of data pipelines.- Build pipelines for extracting data via web scraping to source sector-specific datasets on an ad hoc basis.Data Modeling...Web
- Description : Responsibilities : - Responsible for maintaining, improving, and scaling web scraping systems to handle large data volumes efficiently.- Enhance and maintain existing web scraping and data extraction tools.- Diagnose and resolve scraping failures, bans, and performance...Web
- ...Welcome to Decision Foundry - Data Analytics Division! We are proud... ...a well-established 19-year web analytics company, we remain dedicated... ...seeking a Lead Data Platform Engineer to join our Data & Analytics... ...environments, and specialized web/email scraping workflows. · Reliability...WebFull time
- ...firm with 11-50 employees, offering comprehensive software and web development solutions. By integrating innovative design and quality... ...at our website . Job Overview We are seeking a Senior Data Engineer to join our dynamic team at ALIQAN Technologies. This is a...WebRemote job
- Data Engineer ( Gcp) Start Date Starts Immediately... ...in New Delhi, offering clients innovative design, software, and web development solutions all in one place. Our comprehensive approach...WebRemote jobImmediate start
- ...the cloud running. We support all AWS data centers and all of the servers, storage... ...of software, hardware, and network engineers, supply chain specialists, security experts... ..., hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from...Full timeContract workFor contractorsFor subcontractorLocal areaShift workRotating shift
- ...well as rail transport.The IT Digital and Data Services Competence Center of ContiTech caters... ...among other on areas of Data & Analytics, Web and Mobile Software Development and AI.The... ...reporting and dashboarding.Job Title : IT engineer Data Lakehouse Tech Lead.Job Description :...Web
- ...of technical challenges and mentor other engineers. You will play an active role in... ...just starting hasnt followed a traditional path or includes alternative experiences dont let it stop you from applying. Why AWS Amazon Web Services (AWS) is the worlds most comprehensive...Full timeFlexible hours
- ...Position Summary We are seeking a skilled engineer to support our fab automation initiative... ...will focus on designing and implementing data engineering solutions to build and maintain... ...-stack development for database-driven web applications. Responsibilities • Analyze...Web
- ...Amazon is expanding the Data Center management team in India. This... ...the design intent operational alternatives and contingency plans related... ...Center systems. Manage the engineering aspects of the Data Centers related... ...hasnt followed a traditional path or includes alternative...Full timeContract workFor contractorsFor subcontractorLocal areaShift workRotating shiftWeekend work
- Data Engineer Support Position Description Job Description: Support Data Engineer. Exp: 4+ years Technical Primary Skills – Mandatory... ...Python SDK . . Good Experience on supporting Python Based Web applications. . Ability to troubleshoot software applications...WebFull time
- ...new payment flows business and data solutions cyber security and B... ...and energetic Software Engineers who embrace solving complex challenges... ...with payments technology or web applications before joining Visa... .... Hybrid employees can alternate time between both remote and office...WebFull timeHybrid workWork at officeLocal areaRemote job2 days week3 days week
- ...Description Responsibilities Implement Data engineering solutions on premise or on cloud for IIOT Ecosystem. (Sensor Data) Leverage open... ...tools like Talend Node -Red etc. Experience with REST APIs Web Services Experience with Event Hub IoT Hub Azure Functions etc...WebFull time
- ...that finds creativity and problem-solving using technology, data and design exciting. Want to be at the centre of designing and... ...standards Work closely with web and DevOps engineering teams to integrate visualisations into digital platforms...Web
- ...your career. About the Team The Core Data team is a globally distributed group (... ...data (MPD) Role Overview As a Data Engineer IC3 in the Core Data team youll be responsible... ...Experience with data ingestion from web scraping or public APIs (e.g. job boards websites...WebFull time
- ...As a BI Data engineer you will be responsible for building and maintaining reporting solutions. You will have a thorough understanding of SAP... ...accounting stakeholders. Ongoing development and maintenance of SAP Web Intelligence reporting platform. Responsible for creating and...WebFull timeLocal area
- ...What you will do: Performing Data engineering via development & collaboration between various databases, Web Apps, Power Apps, enterprise platforms, machine learning models. Strong knowledge of Microsoft .Net, SQL Server, Git, Dev-Ops. Good know-how in Power BI, Power...WebFull time
- Description : - An Amazing Career Opportunity for Senior Data Engineer Data Analytics.- Location : Chennai, India (Hybrid).Profile Summary : The... ...secure access solutions.Here we deliver cloud-based solutions, web and mobile application development.The Chennai R&D Center is...WebHybrid workWorldwide
- ...We are looking for an experienced Data Engineer to become a valuable member of our energetic team. The perfect candidate will possess extensive... ...people we love. Software Engineering Applications, including Web and Mobile, are at the core of what we do at Mindera. We...WebFull timeFlexible hours
- Data Support Engineer Start Date Starts Immediately... ...hours ago Job Be an early applicant About the job Alternative Path is looking for a skilled Data Support Engineer, to collaborate...Remote jobImmediate startShift workNight shift
- ...performance) EnglishBhashi Building the data backbone for a Bharat-first learning platform... ...founding team and were looking for a Data Engineer Intern who wants hands-on exposure to real... .... Help ingest data from: App & web product events Analytics tools (Mixpanel...WebInternship
- ...internal product tools and report to the team’s Engineering Director. You will work day-to-day with a... ...WE ARE LOOKING FOR We’re looking for a Data Engineer to solve complex software... ...capabilities which will be consumed by modern web applications build with the most recent...WebFull timeCasual work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Engineer. Be the first to apply!
