AVP Technology ( Data Engineer)

Job Purpose

• Connect and model complex distributed data sets to build repositories, such as data warehouses, 
  data lakes, using appropriate technologies.
• Managing data related contexts ranging across addressing small to large data sets, 
  structured/unstructured or streaming data, extraction, transformation, curation, modelling, 
  building data pipelines, identifying right tools, writing SQL/Java/Scala code, etc.

Key Result Areas

• Create and maintain optimal data pipeline architecture.
• Assemble large, complex data sets that meet functional I non-functional business requirements.
• Identify, design, and implement internal process improvements: automating manual processes, 
  optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and Azure ‘big data’ technologies.
• Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.
• Work with stakeholders including the Executive, Product, Data and Design teams to assist with data­ related technical issues and support their data infrastructure needs.
• Keep data secure
• Create data products for analytics and data scientist team members

Knowledge, Skills and Experience

• Knowledge in data architecture, defining data retention policies, monitoring performance and advising any necessary infrastructure changes
• Strong development skills in Java, Python, Scala and SQL
• Clear hands-on mastery in big database systems- Hadoop ecosystem, Cloud technologies (AWS, Azure, 
• Google), in-memory database systems (HANA, Hazel cast, etc) and other database systems -traditional 
• RDBMS (Terradata, SQL Server, Oracle), and NoSQL databases (Cassandra, MongoDB, DynamoDB)
• Comfortable in dashboard development (Tableau, Powerbi, Qlik, etc) and in developing data analyticsmmodels (R, Python, Spark)
• Bachelor’s degree in Computer Science or equivalent; Masters preferred
• Minimum of 8 years’ hands-on experience with a strong data background
• Extensive experience working with Big Data tools and building data solutions for advanced analytics
• Practical knowledge across data extraction and transformation tools -traditional ETL tools (Informatica, Azure Data Factory) as well as more recent big data tools