Big Data Engineer

About Us
Valiance is a global AI & Data analytics firm helping clients build cutting-edge technology solutions for digital transformation. We work with some of the marquee brands across India, US and APAC to build transformative solutions for Credit Risk, Fraud, Predictive Maintenance, Quality Inspection, Data lake, IOT analytics etc. Our team comprises 150+ professionals across Machine Learning, Data Engineering & Cloud expertise.

We are looking to hire a Senior Data Engineer to help our customers create scalable data engineering pipelines and infrastructure for downstream analytics workloads. You should be good at understanding client data needs, the landscape of various heterogeneous data sources, identifying a set of services for data ingestion & transformation workloads, and timely execution of projects.

Roles & Responsibilities:

1. As a data engineer with Pyspark & SQL skills you will be required to highly scalable, robust, and resilient data engineering pipelines .
2. You will be working closely with business stakeholders & the data science team to understand their data requirements and underlying business logic.
3. Deploy and monitor pyspark jobs on cloud infrastructure.
4. Troubleshoot job failures and ensure system recovery at earliest.
5. Attending regular client calls, communicating work status and pro-actively highlighting any delays to the product release.

Technical Skills :

1. Hands on experience on pyspark for at least 3 years
2. Solid programming experience in Python & SQL is required.
3. Working experience of any one cloud platform; AWS, GCP or Azure
4. Intermediate plus proficiency in shell scripting
5. Experience deploying ML algorithms in production is preferred

Personal Skills :

1. Excellent communication skills, both written & oral.
2. Ability to learn new skills quickly, adjust to the changing needs of the project.
3. You are highly enthusiastic about your work
4. Ability to multi-task, manage high-pressure release scenarios occasionally.