Data Engineer

Job Description: Develop and maintain data pipelines using PySpark and Databricks Work within a metadata-driven ingestion framework to onboard new datasets Implement data quality checks and validation rules within pipelines Support ingestion from file-based sources and ingestion tools (e.g., Fivetran) Handle schema changes, incremental loads, and file processing patterns Contribute to data governance practices including tagging, metadata, and lineage Troubleshoot and resolve pipeline failures and performance issues Collaborate with architects and stakeholders on data onboarding and requirements Follow and contribute to coding standards, reusable components, and best practices Requirements: Experience in lieu of a Bachelor’s Degree 3+ years of relevant experience with a degree 5+ years of relevant experience without a degree Hands-on experience with PySpark and Databricks Strong SQL skills Experience building ETL/ELT data pipelines Understanding of Delta Lake concepts (merge, schema evolution, partitions) Familiarity with cloud platforms (Azure preferred) Basic experience with Git and version control Exposure to data catalog or governance tools (e.g., DataHub) Experience with Fivetran or similar ingestion tools Understanding of data quality and validation concepts Experience working with metadata-driven frameworks Strong problem-solving and debugging skills Ability to work in a structured, framework-driven environment Focus on data quality, not just pipeline execution Willingness to learn and adapt in a fast-evolving data ecosystem. Benefits: Medical, Dental, Vision plans Adoption, Fertility and Surrogacy Reimbursement up to $10,000 Paid Time Off and Sick Leave Paid Parental & Family Caregiver Leave Emergency Backup Care Long-Term, Short-Term Disability, and Critical Illness plans Life Insurance 401k/403B with Employer Match Tuition Assistance – $5,250/year and discounted educational opportunities through Guild Education Student Debt Pay Down – $10,000 Reimbursement for certifications and free access to complete CEUs and professional development Pet Insurance Legal Resources Plan Colleagues have the opportunity to earn an annual discretionary bonus if established system and employee eligibility criteria is met.

Apply Now