Talent.com
Data Pipeline Engineer (Python/Airflow)
Data Pipeline Engineer (Python/Airflow)Pentangle Tech Services | P5 Group • Guadalajara, Mexico Metropolitan Area, Mexico
Data Pipeline Engineer (Python / Airflow)

Data Pipeline Engineer (Python / Airflow)

Pentangle Tech Services | P5 Group • Guadalajara, Mexico Metropolitan Area, Mexico
Hace 6 días
Descripción del trabajo
  • Design, build, and maintain Airflow DAGs using TaskFlow, dynamic DAGs,
  • deferrable operators, providers, and the Secrets backend; manage cross‑DAG dependencies and SLAs.
  • Develop Python ETL / ELT code to ingest from APIs, object storage, message buses, and databases; package code as reusable libraries.
  • Operate Airflow on managed or self‑hosted platforms (e.g., Azure, Kubernetes deployments); implement blue / green or canary DAG releases.
  • Implement data quality and testing with unit tests for operators / hooks, and DAG validation in CI.
  • Build event‑driven pipelines for near‑real‑time processing; manage schemas and compatibility.
  • Model and manage data stores across SQL and blob storage; design partitioning, clustering, and retention.
  • Observability & lineage : instrument metrics / logs, set SLAs / alerts, drive post‑incident reviews and reliability improvements.
  • Security & governance : apply least‑privilege IAM, secrets management, PII
  • handling, and data contracts; enforce RBAC in Airflow and warehouses.
  • CI / CD & IaC : build pipelines to lint / test / deploy DAGs and Python packages;
  • provision infra with Terraform / Helm; containerize with Docker.
  • Cost & performance : tune task parallelism, autoscaling, storage formats, and compute footprints to optimize cost / perf.
  • Collaboration : work closely with Android / backend teams to define interfaces and data contracts; document decisions and operational runbooks.
  • Skills and Qualifications

    • 8+ years in data engineering or backend engineering with strong Python expertise.
    • 2+ years Airflow 2.x expertise (operators, hooks, sensors, TaskFlow, scheduler
    • tuning).
    • Proven experience designing reliable ETL / ELT at scale (batch and streaming) with
    • robust testing and monitoring.
    • Strong SQL and data modeling skills; hands‑on with one or more data warehouses
    • (BigQuery, Redshift, Snowflake) and relational systems (PostgreSQL / MySQL).
    • Familiarity with security best practices (RBAC, OAuth2 / OIDC for service
    • integrations), API gateways, and secrets management (Vault / AWS Secrets Manager / GCP Secret Manager).
    • Comfortable operating in production : monitoring, troubleshooting, and performance tuning.
    • Excellent written and verbal communication; clear trade‑off communication and
    • autonomous execution with well‑documented decisions.
    Crear una alerta de empleo para esta búsqueda

    Data Engineer • Guadalajara, Mexico Metropolitan Area, Mexico

    Ofertas relacionadas
    Senior Pipeline Engineer

    Senior Pipeline Engineer

    Pentangle Tech Services | P5 Group • Guadalajara, Mexico Metropolitan Area, Mexico
    Senior Data Pipeline Engineer (Python / Airflow).Own the design and operation of reliable, secure, and cost‑efficient data pipelines built. You’ll deliver batch and streaming ingestion,.You’ll lead Py...Mostrar más
    Última actualización: hace 1 hora • Oferta promocionada • Nueva oferta
    Python and Kubernetes Software Engineer - Data, Workflows, AI / ML & Analytics

    Python and Kubernetes Software Engineer - Data, Workflows, AI / ML & Analytics

    Canonical • Zapopan, Jalisco, Mexico
    Teletrabajo
    Python and Kubernetes Software Engineer - Data, Workflows, AI / ML & Analytics.Python and Kubernetes Software Engineer - Data, Workflows, AI / ML & Analytics. Python and Kubernetes Software Engineer - D...Mostrar más
    Última actualización: hace más de 30 días • Oferta promocionada
    Python and Kubernetes Software Engineer - Data, AI / ML & Analytics

    Python and Kubernetes Software Engineer - Data, AI / ML & Analytics

    Canonical • Guadalajara, Jalisco, Mexico
    Teletrabajo
    Python and Kubernetes Software Engineer - Data, AI / ML & Analytics.Be among the first 25 applicants.Get AI-powered advice on this job and more exclusive features. Canonical is a leading provider of o...Mostrar más
    Última actualización: hace más de 30 días • Oferta promocionada
    Senior Data Pipeline Engineer (Python / Airflow)

    Senior Data Pipeline Engineer (Python / Airflow)

    Pentangle Tech Services | P5 Group • Guadalajara, Mexico Metropolitan Area, Mexico
    Senior Data Pipeline Engineer (Python / Airflow).Design, build, and maintain Airflow DAGs using TaskFlow, dynamic DAGs,.Secrets backend. manage cross‑DAG dependencies and SLAs.Develop Python ETL / ELT ...Mostrar más
    Última actualización: hace 6 días • Oferta promocionada
    Principal Data Engineer

    Principal Data Engineer

    TouchTunes • Guadalajara, Jalisco, Mexico
    Your mission in the Data Engineering team : .At TouchTunes, your work matters, we are seeking a highly skilled and experienced Principal Data Platform Engineer to lead the design, development, and op...Mostrar más
    Última actualización: hace más de 30 días • Oferta promocionada
    Senior Python Engineer near shore

    Senior Python Engineer near shore

    ConsultNet Technology Services and Solutions • Jalisco, Mexico
    Teletrabajo
    Join to apply for the Senior Python Engineer near shore role at.ConsultNet Technology Services and Solutions.Build and scale backend systems in Python, PostgreSQL, and AWS / GCP.Design and optimize d...Mostrar más
    Última actualización: hace 7 días • Oferta promocionada
    Pavimentos Continuos en El Arenal

    Pavimentos Continuos en El Arenal

    Cronoshare.com.mx • El Arenal (Jalisco), mx
    Cronoshare es una plataforma online para profesionales que quieren encontrar nuevos clientes.Buscamos Pavimentos Continuos en El Arenal y alrededores. Pertenecer a la red de profesionales de Cronosh...Mostrar más
    Última actualización: hace más de 30 días • Oferta promocionada
    Geólogos / as en Villa Corona

    Geólogos / as en Villa Corona

    Cronoshare.com.mx • Villa Corona (Jalisco), mx
    Cronoshare es una plataforma online para profesionales que quieren encontrar nuevos clientes.Buscamos Geólogos / as en Villa Corona y alrededores. Pertenecer a la red de profesionales de Cronoshare no...Mostrar más
    Última actualización: hace más de 30 días • Oferta promocionada
    Data Engineer ID43407

    Data Engineer ID43407

    AgileEngine • Guadalajara, JAL, mx
    Quick Apply
    Fortune 500 brands and trailblazing startups across 17+ industries.We rank among the leaders in areas like application development and AI / ML, and our people-first culture has earned us multiple Bes...Mostrar más
    Última actualización: hace más de 30 días
    Data Pipeline Engineer...

    Data Pipeline Engineer...

    Global Connect Technologies • Guadalajara, Mexico Metropolitan Area, MX
    Job Title : Data Pipeline Engineer (Python / Airflow) Location : Guadalajara, MX Job-Type : Full-time Client : Cognizant Job Description : We are seeking an experienced Senior Data Pipeline Engineer ...Mostrar más
    Última actualización: hace 8 horas • Oferta promocionada • Nueva oferta
    Dj en Buenavista

    Dj en Buenavista

    Cronoshare.com.mx • Buenavista (Jalisco), mx
    Cronoshare es una plataforma online para profesionales que quieren encontrar nuevos clientes.Buscamos Dj en Buenavista y alrededores. Pertenecer a la red de profesionales de Cronoshare no tiene cost...Mostrar más
    Última actualización: hace más de 30 días • Oferta promocionada
    Data Engineer (Senior) ID43916

    Data Engineer (Senior) ID43916

    AgileEngine • Zapopan, JAL, mx
    Quick Apply
    Fortune 500 brands and trailblazing startups across 17+ industries.We rank among the leaders in areas like application development and AI / ML, and our people-first culture has earned us multiple Bes...Mostrar más
    Última actualización: hace 13 días
    BI / Data Engineer (Lead) ID41786

    BI / Data Engineer (Lead) ID41786

    AgileEngine • Guadalajara, JAL, mx
    Quick Apply
    Fortune 500 brands and trailblazing startups across 17+ industries.We rank among the leaders in areas like application development and AI / ML, and our people-first culture has earned us multiple Bes...Mostrar más
    Última actualización: hace más de 30 días
    Data Pipeline Engineer

    Data Pipeline Engineer

    Global Connect Technologies • Guadalajara, Mexico Metropolitan Area, Mexico
    Job Title : Data Pipeline Engineer (Python / Airflow).We are seeking an experienced Senior Data Pipeline Engineer to design, build, and operate secure, reliable, and cost-efficient data pipelines supp...Mostrar más
    Última actualización: hace 11 horas • Oferta promocionada • Nueva oferta
    Senior Data Pipeline Engineer (Python / Airflow)...

    Senior Data Pipeline Engineer (Python / Airflow)...

    Pentangle Tech Services | P5 Group • Guadalajara, Mexico Metropolitan Area, MX
    Job Title : Senior Data Pipeline Engineer (Python / Airflow) Location : Guadalajara, Mexico Duration : Long Term Responsibilities - Design, build, and maintain Airflow DAGs using TaskFlow, dynamic D...Mostrar más
    Última actualización: hace 6 días • Oferta promocionada
    Enfermera / o en Poncitlán

    Enfermera / o en Poncitlán

    Cronoshare.com.mx • Poncitlán (Jalisco), mx
    Cronoshare es una plataforma online para profesionales que quieren encontrar nuevos clientes.Buscamos Enfermera / o en Poncitlán y alrededores. Pertenecer a la red de profesionales de Cronoshare no ti...Mostrar más
    Última actualización: hace 5 días • Oferta promocionada
    Data Engineer (Lead) ID41785

    Data Engineer (Lead) ID41785

    AgileEngine • Guadalajara, JAL, mx
    Quick Apply
    Fortune 500 brands and trailblazing startups across 17+ industries.We rank among the leaders in areas like application development and AI / ML, and our people-first culture has earned us multiple Bes...Mostrar más
    Última actualización: hace más de 30 días
    Pipeline - Data Engineer

    Pipeline - Data Engineer

    Global Connect Technologies • Guadalajara, Mexico
    Job Title : Data Pipeline Engineer (Python / Airflow) Location : Guadalajara, MX.We are seeking an experienced Senior Data Pipeline Engineer to design, build, and operate secure, reliable, and cost-eff...Mostrar más
    Última actualización: hace menos de 1 hora • Oferta promocionada • Nueva oferta