Senior Data Engineer (Senior Search Data Engineer)

nn**This role will work 100% remotely.

This role is open to candidates living in WA state.

W2 Only.**nn nn nn nnLooking for a multi-talented data engineering professional who is passionate about solving real problems and is excited about using the latest technology to do it on our Enterprise Search team.

We move the world with our imagery, and we want you to identify trends in datasets and transform data into a format that can be easily analyzed by developing, maintaining, and testing infrastructures for data generation.

nn nn nn nnOur customers have access to hundreds of millions of images.

The Enterprise Search team is responsible for realizing the best search, discovery, and personalization experiences for our customers.

The goal of this team is to assist the customers to complete their intents, and to provide the most relevant and personalized set of results.nn nn nn nnYour job is to create and optimize data infrastructure to support the Search team.

We are looking for a creative and curious engineer specialized in building and maintaining scalable data pipelines and infrastructure.

You will have the opportunity to lay the foundation for data pipelines specific to machine learning.

nn nn nn nnThis role requires a significant set of technical skills, including a strong command of multiple open source programming languages, deep knowledge of SQL database design, and strong communication skills used to work across departments to understand what business leaders want to gain from our massive datasets.nn nn nn nnWhat you’ll do:nn Assemble large, complex data sets that meet business requirements Define, architect, develop, and deploy infrastructure for large scale ETL pipelines with data processing frameworks that lay the foundation for robust, production level data science models in our products Create and maintain optimal data architecture, ensure data quality, velocity and security, effective data storing and retrieving, and reliable data flow via dashboard, alert, and monitoring Collaborate with other technology teams including data platform, search engineers, data scientists, and machine learning engineers to handle a wide variety of sources of structured/unstructured data, and integrate solutions into our engineering stack Keep up to date on the emerging best practices in data engineering, continuously evaluating and providing guidance on the use of new technologies that lay the foundation for data engineering best practices nn nn nnQualifications:nn Bachelor’s degree or higher in a quantitative/technical field (ex: Computer Science, Statistics, Engineering, Natural Sciences, Information Management, Mathematics, etc.) Experience in building and optimizing ‘big data’ data pipelines/ETL, architectures and datasets, and comfortable working within a large-scale distributed environment with tools such as Spark, Python, Quoble, Airflow, Kafka, Hadoop, Hive, etc Extensive experience with one or more open source programming languages: Python, Java, or Scala Experience with AWS cloud services: EC2, EMR, RDS, Redshift Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of NoSQL databases
– provided by Dice