Senior Data Engineer

POSITION SUMMARY

We’re looking for a Senior Data Engineer to support and evolve our data platform, which is designed to support analytical tasks for our innovative monitoring application. You will be a key voice as the company evolves this exciting application.

The ideal candidate is someone who has strong data architecture expertise, diverse database experience, and understands how to efficiently and effectively coordinate data preparation tasks.

This person's main focus will be designing data transformation processes, developing/modifying data models, and automating common analysis workflows with Apache Airflow. This will include developing data ETL/ELT pipelines via SQL/Python, developing automated processes to consume new data and augment / increment existing data, identifying opportunities for efficiency, keeping accurate and complete project documentation, and performing quality testing and data assurance.

While you will primarily have internally focused interactions (engineering, quality, project/product managers), you will also interact with customers in support of progressing data acquisition / pipeline needs or troubleshooting issues.

Key Responsibilities

• Quickly bridge-the-gap between source data and models needed to analytically process the data.
• Ensure all efforts contemplate a multi-client approach and the automation potential.
• Master the ingestion to analysis process, manifesting in the data platform.
• Ensure integrity and security of data.
• Enhance, standardize, and join different data sets to achieve data science and analytics objectives.
• Research and build efficient and scalable data storage and retrieval systems that enable interactive reporting on high dimensional data.
• Educate peers and management on technical tools, processes and best practices.
• Effectively translate technology terms and functionality into a business vocabulary understandable by non-technical staff.
• Maintain accurate, complete, and current documentation.

Qualifications

What we need

• 4+ years experience in ETL development, specifically designing ETL/ELT automated data transformation processes.
• Proficiency in SQL (on multiple database platforms).
• Expertise developing with Python, specifically working with packages like Pandas, dask, SQLAlchemy, boto3, and NumPy.
• Proficiency in designing and implementing database structures, ideally with Postgres.
• Proficiency acquiring, organizing, cleansing and manipulating large amounts of data.
• Demonstrated partnering and communication skills; able to shift readily between technical and non-technical terminology depending on the audience.
• Experience and proficiency in requirements elicitation and documentation for data processing.
• Experience and proficiency in developing automated data validation test scenarios and scripts.
• Excitement for the code you write and willingness to work hard at making it maintainable for your future self as well as your colleagues on the data team.
• Enjoy working in an agile environment and have a strong understanding of agile practices.
• Clear communicator; fluent in remote: You’ve worked closely with distributed teams that emphasize online communication (Slack, Github, Zoom, Jira/Confluence, GoogleDocs).
• Ability to function effectively in a fast-paced environment and manage multiple projects simultaneously.

What we’d really love

• Experience with the following are desired: Apache Airflow and AWS services like RDS, S3, and ECR/ECS. Apache Spark experience is a plus.
• Experience with RESTful APIs.
• Experience supporting self-service reporting tools such as Tableau, Looker, Periscope, Power BI, Spotfire, etc.
• Familiarity with financial systems integrations (SAP, Concur, Oracle, etc.) a plus.
• Experience working with financial transactions, PII, or in a regulated industry is a plus.
• An entrepreneurial spirit and experience with young companies is ideal.