Senior Data Engineer

Om jobbet

Job Description

Our client is seeking a highly skilled Senior Data Engineer to support advanced data processing and engineering initiatives within an enterprise automotive environment. The role focuses on designing and optimizing robust data pipelines, handling large-scale test data, and contributing to distributed system architectures.

The ideal candidate will bring strong expertise in Python-based development, data engineering practices, and cloud infrastructure, along with experience working in complex, production-grade environments.

Key Responsibilities

Design, develop, and maintain scalable ETL/ELT pipelines for large-scale data processing

Manage ingestion and transformation of high-volume test bed data across multiple formats

Build and optimize distributed data processing systems with a focus on performance and reliability

Implement object-oriented solutions using advanced programming patterns and concurrency techniques

Collaborate with cross-functional teams to support data-driven applications and workflows

Deploy and manage applications within Windows Server environments

Develop and maintain CI/CD pipelines using tools such as GitHub Actions

Monitor system performance and implement logging solutions, including integrations with monitoring tools like Grafana

Handle file system operations, including network drive configurations and NAS interactions

Ensure efficient database integration and management across Oracle, SQL, and MongoDB systems

Required Experience

Senior-level experience in data engineering or related roles

Strong hands-on experience with production-grade ETL pipeline development

Proven expertise in designing distributed systems and handling large-scale data processing

Experience working in enterprise environments, preferably within the automotive or testing domain

Solid understanding of concurrency, multiprocessing, and performance optimization techniques

Familiarity with Windows-based deployment and system environments

Skills Required

Advanced proficiency in Python (3.9-3.11)

Strong knowledge of Object-Oriented Programming principles

Experience with ETL/ELT pipelines and large-scale data ingestion

Proficiency in SQL and database technologies including Oracle and MongoDB

Hands-on experience with Microsoft Azure

Familiarity with CI/CD practices and tools (e.g., GitHub Actions)

Knowledge of file processing across multiple data formats

Experience with PowerShell scripting and Windows service deployment

Preferred / Additional Skills

Experience with domain-specific data frameworks (e.g., Sympathy for Data)

Knowledge of automotive test bed systems and related workflows

Understanding of network file systems (SMB/CIFS, NAS)

Experience with logging and monitoring solutions at file system and infrastructure level

Education

Bachelor's or Master's degree in Computer Science, Data Engineering, or a related field

Language Requirement

English - Professional Proficiency

Application Method: Interested candidates can apply by sending their profile to hr@semiconservicenordic.com