AZURE Databricks Data Engineer (Utility Domain) Job at New York Technology Partners, Des Moines, IA

WWlWa25UUjFFcE1LNDhENnE2QkVkUXdQUnc9PQ==
  • New York Technology Partners
  • Des Moines, IA

Job Description

Job Title: AZURE Databricks Data Engineer (Utility Domain)

***Location: Des Moines - IA - ONSITE 5 DAYS

***This is a C2H role. Candidates should be willing to join the end client directly. Please ensure that this is communicated and agreed to before submitting.

Experience Level: (8+ years)

About Us: We are seeking a highly experienced Data Engineers to join our dynamic team and lead our data engineering initiatives using Azure ML and Databricks.

Job Description:

Responsibilities:

Design, develop, and maintain scalable data pipelines and ETL processes to support data science and analytics initiatives.

Utilize Azure ML and Databricks to build and optimize data architectures and workflows.

Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and ensure data availability and quality.

Implement data integration solutions to consolidate data from various sources into a unified data platform.

Analyze the generation patterns from various Solar Generation facilities and help filed teams to identify the anomalies in generation.

Ensure data security, privacy, and compliance with relevant regulations and best practices.

Monitor and troubleshoot data pipelines and workflows to ensure reliability and performance.

Stay up-to-date with the latest advancements in data engineering, big data technologies, and cloud platforms, and apply them to improve existing processes and systems.

Qualifications:

Bachelor’s degree in computer science, Data Engineering, or a related field.

10+ years of experience in data engineering, data integration, and ETL processes.

Extensive experience with Azure ML and Databricks, including hands-on experience in building and optimizing data pipelines.

Proficiency in programming languages such as Python, SQL, and Scala.

Strong understanding of data engineering principles and best practices.

Experience with big data technologies such as Hadoop, Spark, and Kafka.

Excellent problem-solving skills and the ability to work independently and as part of a team.

Strong communication and interpersonal skills, with the ability to convey complex technical information to a non-technical audience.

Experience in solar generation data analysis is a plus.

Job Tags

Similar Jobs

Vito Esposito Salon

Hair Stylist Job at Vito Esposito Salon

 ...Job Title: Hair Stylist Location: Beverly Hills, CA At Vito Esposito Salon, we believe in the art of beauty, craftsmanship, and creating unforgettable experiences. Our luxury salon, nestled in the heart of Beverly Hills, is a reflection of Italian elegance and... 

FlexsJobs

Full-Time Data Entry Clerk-Remote Job at FlexsJobs

 ...Position Overview: We are looking for a detail-oriented and highly organized Data Entry Clerk to join our remote team. This role involves managing and entering data accurately into systems, ensuring data integrity, and maintaining confidentiality. The ideal candidate... 

Curate

Customer Support Representative II Job at Curate

 ...remote position. Annual salary of $53,000. The Grammys. The FedEx Open. The BET Awards. Over 400,000 weddings. These are a few of...  ...to ensure theyre getting the most value from our software and services. You will do this by understanding the customers perspective,... 

SCA Health

CEO Surgical Hospital - Physicians Medical Center Job at SCA Health

 ...Today, SCA Health has grown to 11,000 teammates who care for 1 million patients each year and support physician specialists holistically in many aspects of patient care. Together, our teammates create value in specialty care by aligning physicians, health plans and... 

Hospitality Confidential

Executive Chef Job at Hospitality Confidential

 ...Hospitality Confidential is pleased to present an exciting Executive Chef opportunity with a brewery designed to create a welcoming, adventurous atmosphere. The focus is on cultivating a space where guests enjoy exceptional food and drinks and where employees are excited...