Senior Data Engineer

Sotheby’s has been uniting collectors with world-class works of art since 1744. Today, Sotheby’s remains synonymous with innovation, with a global network of over 1500 employees located in 80 offices in 40 countries around the globe. In addition to worldwide auctions, we offer Sotheby’s Financial Services, the only full-service art financing company. Sotheby’s also presents private sale opportunities in 70+ categories, including S|2, the gallery arm of Global Fine Art Division, and three retail businesses: Sotheby’s Wine, Sotheby’s Diamonds, and Sotheby’s Home. For advisory we offer services for private collectors, museums, corporations, artists, estates and foundations.

THE ROLE

At Sotheby’s we’re transforming the way we drive business decisions through data with the adoption of a fully cloud-native environment. This hands-on role will support the efforts of maintaining and growing an enterprise data lake on which this team and others perform their work. You will interface with technical system owners, security, and IT teams to understand and support their efforts. You will be responsible for implementing new datasets and data services to support our various applications and business functions. You will partner with development teams to create ETL processes that automate the transformation and centralization of data from all corners of the enterprise.

RESPONSIBILITIES

• Implement new ETLs for data collection from internal systems into GCP
• Help evangelize high quality software engineering practices towards building data infrastructure and pipelines at scale
• Work within and across agile teams to design, develop, test, implement, and support technical solutions across a full-stack of cloud development tools and technologies
• Create monitoring and alerting solutions for data pipeline statuses
• Implement specific Google Cloud data security and governance controls
• Maintain access controls for the data lake and associated Google Cloud products
• Ensure quality of the solutions are robust, scalable and efficient to meet the needs of the business

IDEAL EXPERIENCE & COMPETENCIES

• Bachelor’s degree in a quantitative field or equivalent experience
• Strong programming skills preferably in Python
• 2+ years experience supporting production cloud environments
• Strong understanding of IAM and cloud-based access and security controls
• Familiarity with ETL pipeline orchestration frameworks, such as Luigi or Airflow
• Experience with data processing and storage frameworks like Apache Beam, BigQuery, BigTable, Redshift, Kinesis, etc.
• Experience with log management and monitoring tools, including tools within Amazon Web Services and Google Cloud Platform as well as open source and third-party monitoring tools.
• Experience in managing projects and infrastructure for cloud-based platforms (security, authentication, monitoring, data governance)
• Experience working with containers and container services is also a significant plus