Senior Data Engineer

Data Model and Structure Definition:

• Design and develop conceptual, logical, and physical data models for both structured and unstructured data.

• Collaborate with stakeholders to understand data requirements and translate them into effective data models.

• Ensure data models are aligned with business goals and effectively support transactional, analytical and operational needs.

Data Technologies:

• Design and develop data technology strategy and associated roadmaps.

• Utilize cloud-native technologies to implement and manage data models.

• Work with data tools and platforms (e.g., Hadoop, Spark, Kafka) to handle large volumes of data.

• Optimize data storage, retrieval, and processing in a cloud environment.

Data Integration and ETL:

• Design and implement ETL processes to integrate data from various sources into the data lakes

• Ensure data quality, consistency, and integrity throughout the data lifecycle.

• Develop and maintain documentation for data models and ETL processes.

Performance Tuning and Optimization:

• Analyze and optimize data models to improve performance and scalability.

• Monitor and troubleshoot data-related issues to ensure optimal system performance.

• Implement best practices for data management and governance, including defining best practices around ownership, linage, security, and quality.

Collaboration and Communication:

• Work closely with engineers, data scientists, and business analysts to deliver data solutions, oftentimes working directly in code alongside them

• Communicate complex data concepts and solutions to both technical and non-technical stakeholders, including executives and senior leadership

• Participate in data architecture reviews and provide recommendations for improvements.

• Partner with senior leadership to ensure strategic plans are being met.

• Clear roadblocks by building consensus across the organization via a variety of influencing techniques.

• Proficiency in cloud services and tools (e.g., Amazon S3, Azure Data Lake, Google BigQuery).

• Expert understanding of data modeling, database design, and delta lake concepts.

• Expert knowledge of data integration and ETL tools (e.g., dbt, Apache NiFi, Talend, Informatica, Azure Data Factory, Prefect).

• Strong familiarity with big data technologies (e.g., Hadoop, Spark, Kafka, Delta Lake, Iceberg, Arrow, Data Fusion).

• Strong familiarity with data governance tooling (e.g. Alation, Collibra, BigID, Purview, Monte Carlo)

• Strong programming skills in languages such as Python, Java, Scala, or Rust.

• Excellent problem-solving and analytical skills.

• Strong communication and interpersonal skills.

• Ability to work collaboratively in a team environment.

• Project management and organizational skills.