Data Engineer$80k – $100k
The ideal candidate is an experienced data pipeline builder and data wrangler who enjoys optimizing data systems and building them from the ground up.
The person will be responsible for maintaining optimal data pipeline architecture and deployment, including building the infrastructure required for optimal extraction, transformation and loading of data using PostgreSQL, and AWS EMR to prepare data for BI or Machine Learning frameworks.
Candidate will also be responsible for maintaining quality of data in PostgreSQL by performing cleaning, transformation and integrity checking in an online environment. The person should also have some experience in working with large data sets and maintaining system or tools including Spark, Hadoop, Minio and Alluxio that are needed to managing large datasets.
- Advanced working SQL knowledge with RBDMS, as well as extensive knowledge in PostgreSQL.
- Experience building and optimizing big data pipelines, architectures and data sets.
- Strong analytic skills to working with less structured datasets.
- A successful history of interpreting data from different sources.
- Experience with big-data tools including Hadoop and Spark.
- Experience with HPC and related tools, including the Slurm Workload Manage.
Zensors at a glance
Zensors focuses on Artificial Intelligence, Sensors, Internet of Things, and User Interface Design. Their company has offices in San Francisco and Pittsburgh. They have a small team that's between 11-50 employees.
You can view their website at http://www.zensors.com