Data Engineer II

Job description

DESCRIPTION

Cutting edge big data technology with Spark, Map Reduce, EMR, Distributed Pipeline? Check. Deep involvement with business strategy decision? Check. Work in one of the world's largest and most complex data environments? Check.

Are you excited about the idea to work with huge invaluable big data resources and big data technologies? Are you up to the challenge of working with top economics/ML researchers to bring the front tier economic theories, top industry big data and the cutting edge big data technology together and redefine key business strategy making? Do you want to be part of the pioneer effort to combine the engineering, ML and economics to create disruptive impact on the industry? If your answer is yes, come join us!

As Data Engineer, you will collaborate with research scientists and software engineers across the company to develop, test and deploy a wide range of econometric and ML models. You will face our ever-growing information challenges and provide solution to our analytics, and research science teams with the right data pipelines. You’ll build and maintain database solutions that are used for analytics, machine learning and drive business decision making while dealing efficiently with the massive scale of data available through our Data Warehouse and various data source.

You should be an expert in the architecture of DW solutions for the Enterprise using multiple platforms (RDBMS, Redshift). You should excel in the design, creation, management, and business use of extremely large datasets. You should have excellent business and communication skills to be able to work with business owners to develop and define key business questions, and to build data sets that answer those questions. Above all you should be passionate about working with huge data sets and someone who loves to bring datasets together to answer business questions and drive change.

In this opportunity, you will be working in one of the world's largest and most complex data environments. You will work with our marketing, delivery experience, finance, and technology teams to create and provide the analytic technologies that give our customers timely, flexible and structured access to their data and enable the data-driven decision making in Digital Media and Advertisement business.

Responsibilities:
· Create a productionized data platform that serves as an input to machine learning / econometric models.
· Provide technical and thought leadership for Data Engineering and Business Intelligence
· Establish key relationships which span Amazon business units and Business Intelligence teams.
· Create a Data Governance strategy for mitigating disparate data sources where applicable.
· Implement standardized, automated operational and quality control processes to deliver accurate and timely data and reporting to meet or exceed SLAs.
· Create and execute vision to develop a series of analytic dashboards and decision-support tools, encompassing key forecasting metrics to be tracked on a daily/weekly/monthly basis.
· Assist business leaders, scientists (economists and ML scientists), and software developers in creating and implementing business requirement documents to drive projects, working backward from customer needs.

PREFERRED QUALIFICATIONS

· 5+ years expertise in OLAP and BI tools such as Oracle Business Intelligence, etc.
· Experience with Big Data Technologies (Hadoop, Hive, Hbase, Pig, Spark, etc.)
· Experiences with Amazon Web Services tools such as Redshift and EMR or other similar platforms
· Experience leveraging Python, R or Matlab to manipulate data and set up automated processes as per business requirements
· Excellent communication skills to be able to work with business owners to develop and define key business questions and to build data sets that answer those questions

Amazon is an Equal Opportunity-Affirmative Action Employer – Minority / Female / Disability / Veteran / Gender Identity / Sexual Orientation

Desired profile

BASIC QUALIFICATIONS

· Bachelors or Masters Degree in Computer Science, Information Systems or related field.
· 7+ years experience in Data Engineering or Business Intelligence roles, including at least 3 years of experience with large scale business intelligence and data warehouse environments preferably including Big Data technologies, e.g., Spark.
· 5+ years of expert level data modeling, ETL and SQL skills, knowledge of industry best practices, and a deep understanding how data is extracted, transformed, scrubbed and loaded in a large Data Warehouse environment.
· 5+ proven experience building and managing large data warehouse & reporting solutions.
· 5+ years experience working with and leading agile development methodologies such as Sprint and Scrum.