Data Engineer

Roles & Responsibilities

• Design and implement reliable ETL data pipelines for Data Warehousing
• Automate manual processes, optimize data delivery, and create/enhance data infrastructure for scaling.
• Optimize data custom extraction, transformation, and loading pipelines from multiple data sources.
• Assist internal and some external partners with data-related technical issues and support their data infrastructure needs.

Requirement

• Experience modeling facts and dimensions from unstructured and structured sources (RDBMs, NoSQL DBs, and API Data Feeds)
• Proficiency in at least one programming or scripting language (JavaScript, Python, C#, etc.)
• Maintain code versioning using Git or other version control systems.
• Build, maintain, and optimize data pipelines while keeping an eye toward security and privacy using industry-standard practices
• Effective in working across team boundaries to establish overarching data architecture, and provide guidance to individual teams

Qualifications

• Bachelor's or Master’s degree in Computer Science, Analytics or another applicable field is preferred (relevant experience may substitute for degree)
• 4+ years of recent relevant data engineering/data warehouse and reporting experience required
• Preferred: Hands-on experience in: AWS Glue, Redshift, Quick Sight or Snowflake. Also, exposure to: AWS KMS, AWS Firehose, EMR, Athena and Lake Formation.
• Experience building and optimizing data pipelines for acquisition and management of data including data security, data capture, data cataloguing, & data classification
• Knowledge of message queuing, stream processing and Change Data Capture mechanisms
• API development desired - needed to be able extract data from various sources.