Big data Jobs

Senior Data Engineer #2595 Position Summary: Our partner is a global manufacturer modernizing its ERP-driven data environment into a cloud-native platform for trusted analytics and forecasting. Experience and Education: BS in Computer Science, Information Systems, Software Engineer or equivalent technical field Experience building enterprise data platforms using Databricks and Azure Background in designing and owning data models sourced from ERP systems Experience working with SAP or other large-scale ERP environments History of owning data pipelines end-to-end in production environments Exposure to forecasting, analytics, or ML-enabled data use cases Experience working with globally distributed or offshore teams Skills and Strengths: Databricks Azure Data engineering Data modeling ERP data environments SAP SQL Python Lakehouse architecture ETL design Data orchestration Data quality frameworks Performance optimization Enterprise data governance Analytics enablement Forecasting data pipelines Machine learning support Systems tradeoff analysis Cross-team collaboration Technical documentation Primary Job Responsibilities: Own the Databricks-based data engineering layer used across the enterprise Design and maintain high-quality data models sourced from ERP systems Ensure data pipelines are reliable, scalable, and performant Standardize ETL patterns, orchestration, and data quality checks Improve lakehouse performance and long-term maintainability Partner closely with BI leadership to enable trusted analytics and forecasting Support ML and advanced analytics use cases through strong data foundations Establish and reinforce enterprise data governance practices Collaborate with offshore and onshore teams to ensure consistency Decide when work requires immediate fixes versus backlog prioritization Clearly explain engineering tradeoffs across cost, performance, and mai...

Our platform serves as the core foundation for data-driven decision-making across the organization, facilitating robust metrics and semantic layers, comprehensive data quality, and rigorous governance. We are seeking a versatile Data Engineer to build critical data platform components, sophisticated database and lakehouse models, and robust integration and ingestion solutions.

MLOps Engineer – Contract (Glasgow / Hybrid) We are looking for an experienced MLOps Engineer to join a large-scale data and machine learning programme within a regulated enterprise environment. Show less [{"type":"people_also_viewed","href":"https://uk..com/jobs/view/senior-data-engineer-at-scott-logic-4308196581","jobId":"4308196581","label":"Senior Data Engineer"},{"type":"people_also_viewed","href":"https://uk..com/jobs/view/senior-data-engineer-at-res-4342195689","jobId":"4342195689","label":"Senior Data Engineer"},{"type":"people_also_viewed","href":"https://uk..com/jobs/view/software-engineer-data-infrastructure-acquisition-...

We are seeking a highly skilled Data Engineer focused on building, optimizing, and maintaining enterprise-grade ETL/ELT pipelines using AWS Glue, Amazon S3, and Databricks. The ideal candidate must be located in Columbus, Ohio and have strong experience with cloud data platforms, understands large-scale data processing patterns, and can design resilient, secure, and high-performance data pipelines in a production environment What You’ll Do Build, optimize, and operate scalable ETL/ELT pipelines using AWS Glue, Databricks, Spark, and Amazon S3 Transform raw, structured, and semi-structured data into analytics-ready datasets that power BI, reporting, and advanced analytics Design and maintain modern lakehouse architectures, including bronze, silver, and gold data layers Develop Spark workloads using Python (PySpark) and SQL, leveraging Delta Lake features such as ACID transactions, schema evolution, and time travel Automate and orchestrate reliable data workflows using Glue Workflows, Step Functions, Databricks Jobs, or equivalent tools Optimize performance, reliability, and cost across clusters, jobs, and pipelines Implement monitoring, logging, alerting, and recovery processes to ensure pipeline health and data quality Apply strong data governance and security practices, including IAM, encryption, access controls, validation, lineage tracking, and compliance support (SOX, PCI, SOC2) Collaborate closely with Analytics, BI, Data Science, and Application teams to translate business needs into scalable data solutions Participate in code reviews and architecture discussions, contributing to platform standards and best practices What You Bring 3+ years of experience in data engineering, ETL development, or cloud data platforms Hands-on experience with AWS Glue, Databricks, and Apache Spark Strong proficiency in Python (PySpark) and SQL Experience building and managing data lakes on Amazon S3 Understanding of data modeling concepts, including star schema and dimensional modeling Experience with Delta Lake, lakehouse architecture, and/or Unity Catalog Familiarity with orchestration tools such as AWS Step Functions, Airflow, or equivalents Knowledge of streaming pipelines (Kafka, Kinesis, Auto Loader) is a plus Experience working with version control (Git), CI/CD pipelines, and infrastructure...

Your work will realize our data strategy by developing cutting-edge data solutions. Build, enhance and scale platform data capabilities Contribute to the delivery of our data strategy by defining the architecture, design and development of new data features for our software in Python and SQL Continuously integrate, build and deploy our solution from test system to production Develop and maintain system architecture in partnership with chosen data and analytics solution providers Utilize Azure Databricks and related Azure services to build scalable data solutions Work independently on solutions for specified requirements and be responsible for complying with time and quality specifications Ensure that our software is secure, stable, scalable ...

Your work will underpin data integrity including developing, maintaining, and publishing commonly used terminology across MO. Good knowledge of standard data formats (XML, JSON, csv, etc) Proven experience of delivering BI solutions for business requirements Experience of developing using an Agile development approach Proficient in turning raw, structured and unstructured data into meaningful insights and recommendations Efficient at handling large data sets in data platforms (such as Oracle, Snowflake...

Key Skills Kafka, Confluent Kafka, MongoDB, AWS, APIs, ODS, Streaming Architecture, Solution Architecture, Event-Driven Architecture, Cloud Architecture Show less [{"type":"people_also_viewed","href":"https://uk..com/jobs/view/senior-data-engineer-at-res-4342195689","jobId":"4342195689","label":"Senior Data Engineer"},{"type":"people_also_viewed","href":"https://uk..com/jobs/view/senior-data-engineer-at-scott-logic-4308196581","jobId":"4308196581","label":"Senior Data Engineer"},{"type":"people_also_viewed","href":"https://uk..com/jobs/view/senior-lead-software-engineer-data-engineer-java-python-at-jpmorganchase-4351634261","jobId":"4351634261","label":"Senior Lead Software Engineer- Data Engineer, Java/Python"},{"type":"people_also_viewed","href":"https://uk..com/jobs/view/sr-cloud-data-platform-engineer-at-barclays-4348626763","jobId":"4348626763","label":"Sr. Cloud Data Platform Engineer"},{"type":"similar_search","href":"https://www..com/jobs/senior-data-architect-jobs","label":"Empleos de Arquitecto de datos senior"},{"type":"similar_search","href":"https://www..com/jobs/enterprise-data-architect-jobs","label":"Empleos de Arquitecto de Datos Empresariales"},{"type":"similar_search","href":"https://www..com/jobs/lead-data-architect-jobs","label":"Empleos de Arquitecto de Datos Líder"},{"type":"similar_search","href":"https://www....

Job Summary We are currently seeking a Data Analyst Intern for Summer 2026 (May 18 – early Aug). The Health Analytics Division specializes in developing advanced Information Technology and Data Management solutions for our government and commercial clients.