Architecture Job Captain (Xico)

Project description We are seeking a highly experienced Databricks Architect to lead the design, implementation, and optimization of enterprise‑grade data platforms and AI‑enabled workloads for a global food corporation.

This role requires a hands‑on technical leader who can translate business and data requirements into scalable Databricks‑based solutions while ensuring robust governance, performance, and operational resilience.

The idóneo candidate will combine deep Databricks expertise with strong architecture leadership and the ability to operate within a managed service delivery model.

Responsibilities

• Lead the architecture, design, and implementation of Databricks‑based data platforms, pipelines, and AI/ML workloads.
• Define scalable lakehouse patterns using core Databricks capabilities such as Apache Spark, Delta Lake, Databricks SQL, MLflow, and governance components.
• Design robust batch and streaming data pipelines for high‑volume, business‑critical workloads.
• Establish architecture standards for performance optimization, workload orchestration, reliability, observability, and cost control.
• Translate business and technical requirements into solution blueprints, reference architectures, and implementation roadmaps.
• Collaborate with data engineers, analysts, data scientists, DevOps teams, and business stakeholders to deliver end‑to‑end solutions.
• Ensure strong data governance, security, and compliance practices across the Databricks environment.
• Support AI and advanced analytics use cases by enabling reliable feature engineering, model lifecycle practices, and production‑ready data foundations.
• Act as a trusted advisor to client stakeholders, offering recommendations on architecture decisions, risks, trade‑offs, and delivery priorities.

Qualifications

• Deep hands‑on expertise with the Databricks platform, including architecture, workspace design, cluster strategy, jobs orchestration, and platform optimization.
• Strong command of Apache Spark and distributed data processing concepts, including performance tuning and optimization for complex data engineering workloads.
• Proven experience designing and delivering enterprise data solutions using Delta Lake and lakehouse architecture principles.
• Strong proficiency in Python and SQL; Scala is an advantage.
• Demonstrated capability in building and optimizing ETL/ELT pipelines, data models, and large‑scale ingestion frameworks.
• Experience supporting AI/ML workloads on Databricks, including MLflow, model lifecycle considerations, and production‑ready data preparation.
• Solid knowledge of cloud‑native architecture patterns on AWS, Azure, or GCP, including storage, networking, identity, and security integration.
• Experience with data governance, access control, lineage, and compliance frameworks in enterprise environments.
• Familiarity with CI/CD, infrastructure‑as‑code, monitoring, and operational best practices for data platforms.
• Ability to engage in deep technical problem‑solving and contribute immediately to complex delivery scenarios with minimal ramp‑up time.