Job Title:
Principal Data Engineer (Azure)

Company: Tiger Analytics

Location: jersey city, NJ

Created: 2024-05-04

Job Type: Full Time

Job Description:

Tiger Analytics is a global AI and analytics consulting firm. With data and technology at the core of our solutions, we are solving problems that eventually impact the lives of millions globally. Our culture is modeled around expertise and respect with a team-first mindset. Headquartered in Silicon Valley, you'll find our delivery centers across the globe and offices in multiple cities across India, the US, UK, Canada, and Singapore, including asubstantial remote global workforce.We're Great Place to Work-Certifiedâ„¢. Working at Tiger Analytics, you'll be at the heart of an AI revolution. You'll work with teams that push the boundaries of what is possible and build solutions that energize and inspire.RequirementsCurious about the role? What your typical day would look like?As a Principal Data Engineer (Azure), you would have hands on experience working on Azure as cloud, Databricks and some exposureexperience on Data Modelling. You will build and learn about a variety of analytics solutions & platforms, data lakes, modern data platforms, data fabric solutions, etc. using different Open Source, Big Data, and Cloud technologies on Microsoft Azure.Design and build scalable & metadata-driven data ingestion pipelines (For Batch and Streaming Datasets)Conceptualize and execute high-performance data processing for structured and unstructured data, and dataharmonizationSchedule, orchestrate, and validate pipelinesDesign exception handling and log monitoring for debuggingIdeate with your peers to make tech stack and tools-related decisionsInteract and collaborate with multiple teams (ConsultingData Science & App Dev) and various stakeholders to meet deadlines, to bring Analytical Solutions to lifeWhat do we expect?Experience in implementing Data Lake with technologies like Azure Data Factory (ADF), PySpark, Databricks, ADLS,Azure SQL DatabaseA comprehensive foundation with working knowledge of Azure Synapse Analytics, Event Hub & StreamingAnalytics, Cosmos DB, and PurviewA passion for writing high-quality code and the code should be modular, scalable, and free of bugs (debuggingskills in SQL, Python, or ScalaJava).Enthuse to collaborate with various stakeholders across the organization and take complete ownership ofdeliverables.Experience in using big data technologies like Hadoop, Spark, Airflow, NiFi, Kafka, Hive, Neo4J, Elastic SearchAdept understanding of different file formats like Delta Lake, Avro, Parquet, JSON, and CSVGood knowledge of building and designing REST APIs with real-time experience working on Data Lake orLakehouse projects.Experience in supporting BI and Data Science teams in consuming the data in a secure and governed mannerCertifications like Data Engineering on Microsoft Azure (DP-203) or Databricks Certified Developer (DE) arevaluable addition.Note: The designation will be commensurate with expertise and experience. Compensation packages are among the best in the industry.Job RequirementMandatory: Azure Data Factory (ADF), PySpark, Databricks, ADLS, Azure SQL DatabaseOptional: Azure Synapse Analytics, Event Hub & Streaming Analytics, Cosmos DB and PurviewStrong programming, unit testing & debugging skills in SQL, Python or ScalaJavaSome experience of using big data technologies like Hadoop, Spark, Airflow, NiFi, Kafka, Hive, Neo4J, Elastic SearchGood Understanding of different file formats like Delta Lake, Avro, Parquet, JSON and CSVExperience of working in Agile projects and following DevOps processes with technologies like Git, Jenkins & Azure DevOpsGood to have:Experience of working on Data Lake & Lakehouse projectsExperience of building REST services and implementing service-oriented architecturesExperience of supporting BI and Data Science teams in consuming the data in a secure and governed mannerCertifications like Data Engineering on Microsoft Azure (DP-203) or Databricks Certified Developer (DE)BenefitsThis position offers an excellent opportunity for significant career development in a fast-growing and challenging entrepreneurial environment with a high degree of individual responsibility.