Lead BI Engineer

Responsibilities: Lead the design and optimization of scalable BI solutions and distributed data pipelines using Apache Spark. Architect enterprise Lakehouse environments and collaborate with cross-functional teams to deliver actionable business insights.

Requirements: Requires over 7 years of experience in advanced data engineering with expert-level Spark and Python/SQL proficiency. Must have proven experience architecting cloud-native data platforms and managing mission-critical production systems.

Key skills: Apache Spark, Lakehouse Architecture, Python, SQL, Cloud-native Data Platforms, Kafka, CI/CD, Infrastructure-as-Code, Data Modeling, Data Governance, Performance Tuning, Distributed Compute, ML Pipelines, Delta Lake, Data Ingestion, Production Reliability

Keywords: Apache Spark, Lakehouse, Delta Lake, Python, SQL, Azure, AWS, GCP, Kafka, Event Hubs, CI/CD, Infrastructure-as-Code, ACID Transactions, Schema Evolution, Data Governance, MLOps, Feature Stores, Distributed Compute, Data Pipelines, Performance Tuning, Data Security, Compliance, Audit, Data Quality, Data Cataloging, Business Intelligence, Data Engineering, Cloud-native, Distributed Streaming, Root-cause Analysis