Senior Data Engineer

About the RoleWe are looking for an experienced Lead Data Engineer with deep expertise in Databricks, PySpark, and AWS. The ideal candidate will have a strong understanding of media measurement datasets (Nielsen) and a proven ability to lead multiple concurrent projects across onsite and offshore teams. This role requires a senior technical leader to define architecture, manage delivery, and mentor teams within the fast-paced New York media industry.Key ResponsibilitiesTechnical Leadership: Architect and deliver enterprise-scale data platforms using Databricks, PySpark, Delta Lake, and AWS. Lead the development of media analytics pipelines, including the ingestion and modeling of Nielsen data.Project & Delivery Management: Manage scope, timelines, and risks for multiple simultaneous projects. Collaborate with New York-based stakeholders and drive sprint delivery.Onsite-Offshore Coordination: Act as the primary technical lead onsite, coordinating with offshore teams to ensure alignment, quality, and continuity.Team Mentorship: Mentor junior and mid-level engineers through code reviews and architectural guidance.Required SkillsDatabricks: Delta Lake, Workflows, Unity Catalog, Notebooks, and SQL.Data Processing: Strong PySpark and distributed computing experience.AWS: S3, EMR, Glue, Lambda, Step Functions, Kinesis, and IAM.Domain Expertise: Proven experience handling Nielsen datasets (National/Local TV ratings, Digital Ad ratings, Panels).Leadership: Experience leading large data engineering programs and managing senior stakeholders.Good to HaveKnowledge of Snowflake, Airflow, or Dagster.Infrastructure as Code (Terraform / CloudFormation).Exposure to ad-tech ecosystems (Comscore, Kantar, IAS, Moat) or ML feature pipelines.Soft SkillsStrong decision-making and ownership mindset.Exceptional communication skills for interacting with senior business stakeholders.Strategic, problem-solving, and delivery-focused approach.

Apply Now