Avatar for Zylotech

Self-Learning Customer Data Platform (CDP)

Data Engineer- Python,Spark,ETL

₹5.5L – ₹9L • No equity
Apply now
We are looking for a Data Engineer to join our growing team who will be involved in developing Machine Learning algorithms based programs to automate the data management and transformation at scale and optimize to prepare the data pipe for AI / Analytical technology stack. The ideal candidate is an experienced data pipeline builder and data wrangler who enjoys optimizing data systems and building them from the ground up. The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining, implementing, and monitoring them. You will also be responsible for integrating them with the architecture used across the company.

Must Have:-
- Spark framework for data matching (experience with Spark and Python / Scala)
- Quick slicing and dicing of data using Python(pandas, numpy, pyspark etc) & SQL
- Experience in Data Transformation rules on structured & semi-structured data in Hadoop / NoSQL environment
- Strong experience in ElasticSearch is a must.
- Familiarity with supervised learning methods
- Hands on experience using Scrappy & Selenium for scraping
- Familiarity with record de-duplication, entity resolution, data unification, anomaly detection & correction, standardization, matching in large data.
- Familiarity with development in Cloud environments like AWS / Azure / Google

Good to have
Realtime-Caching database system like Redis/Aerospike/etc
Experience on Queueing frameworks like Kafka/RabbitMQ
Experience with data transformations using PySpark
Experience on Hadoop cluster scaling is a plus
Docker, Docker Swarm and Kubernetes for Deployments
Experience with cloud services like AWS, Azure and Google Cloud
Functional Skill Set:
Record matching like (partial, fuzzy, etc)
Anomaly detection and standardizing the record
Good understanding of security and data protection
Strong project management and organizational skills
Strong analytic skills related to working with unstructured datasets
Big plus if you have past product development team experience, especially in B2B/ Enterprise/ Marketing/ Retail Industry software

More jobs at Zylotech

View all jobs

Data Engineer

Apply now

Sr Director/ VP - Engineering

Apply now

Product Manager

Apply now

Machine Learning Developer/ Engineer

Apply now