DevOps Engineer

We are seeking a highly skilled and experienced Platform Engineer to build and maintain the
core infrastructure and tooling that empowers our data scientists, economists, and
developers working on our cutting-edge Azure Databricks Economic Data Platform. This
platform is critical for our Monetary Analysis, Forecasting, and Modelling activities. The
Platform Engineer will focus on creating a self-service, scalable, and reliable platform that
streamlines development workflows, simplifies infrastructure management, and enhances
overall productivity. This role requires a strong understanding of cloud computing
(specifically Azure), infrastructure-as-code (IaC), DevOps practices, containerisation,
orchestration, and a passion for building developer-friendly platforms.
Key Responsibilities:

• Platform Design & Development:
  o Design, develop, and maintain the core platform infrastructure on Azure, including
  networking, compute, storage, security, and identity management.
  o Implement infrastructure-as-code (IaC) using tools like Terraform, ARM templates,
  or Bicep to automate infrastructure provisioning and management.
  o Develop and maintain platform components, such as APIs, CLIs, and web interfaces,
  to provide self-service capabilities to users.
• Azure Databricks Integration & Optimisation:
  o Deeply integrate Azure Databricks into the platform, ensuring seamless access and
  efficient resource utilisation.
  o Implement automation for Databricks workspace setup, cluster configuration, and
  job deployments.
  o Optimise Databricks workloads for performance, scalability, and cost-effectiveness.
• Containerisation & Orchestration:
  o Implement and manage containerised applications and services using Docker and
  Kubernetes (or Azure Kubernetes Service – AKS).
  o Design and implement container orchestration strategies for deploying and scaling
  platform components.
• CI/CD Pipeline Automation:
  o Design and implement robust CI/CD pipelines for building, testing, and deploying
  platform components and user applications.
  o Automate build processes, unit tests, integration tests, and deployment processes.
  o Implement advanced deployment strategies (e.g., blue/green deployments, canary
  releases).
• Monitoring, Logging & Alerting:
  o Implement comprehensive monitoring, logging, and alerting systems to proactively
  identify and address performance issues, errors, and security threats.
  o Use tools like Azure Monitor, Prometheus, Grafana, or similar to collect and analyse
  metrics, logs, and traces.
  o Configure alerts and notifications to ensure timely responses to critical events.
• Security & Compliance:
  o Implement security best practices and controls within the platform infrastructure
  and CI/CD pipelines.
  o Ensure compliance with relevant security standards and regulations.
  o Implement security scanning and vulnerability management processes.
• Documentation & Support:
  o Develop and maintain comprehensive documentation for the platform, including API
  documentation, user guides, and troubleshooting guides.
  o Provide support to users of the platform.
• Collaboration & Communication:
  o Collaborate closely with data scientists, economists, developers, and other
  stakeholders to understand their needs and gather feedback on the platform.
  o Communicate technical concepts effectively to both technical and non-technical
  audiences.
  Essential Skills & Experience:
• 5+ years of experience in platform engineering, DevOps engineering, or a related role.
• Strong experience with Azure cloud platform and services – Data Factory, Databricks,
  WebApp Service, API Management & APIOps.
• Good experience with IaC tools like Terraform, ARM templates, or Bicep.
• Solid understanding of CI/CD principles and experience with CI/CD tools like Github Actions,
  Azure DevOps, Jenkins, or GitLab CI.
• Preferred 2+ years of experience with Github Actions
• Strong experience with containerisation technologies like Docker and orchestration tools like
  Kubernetes (or AKS).
• Experience with monitoring and logging tools.
• Scripting skills in PowerShell, Bash, YAML and Python.
• Good understanding of networking concepts and security best practices.
• Excellent problem-solving and troubleshooting skills.
• Strong communication and collaboration skills.
• Experience with Azure Databricks and its integration with platform tooling.
• Experience with configuration management tools like Ansible, Puppet, or Chef.
• Experience building and maintaining internal developer platforms.
• Experience working in a regulated industry (e.g., financial services).
• Azure certifications (e.g., Azure DevOps Engineer Expert, Azure Administrator Associate,
  Azure Solutions Architect Expert).

Thank you for your interest in this role. Please also share your CV at bharath@lsarecruit.co.uk and if suitable, we will get in touch with you to discuss further.

Get in touch with Rachel, our AI agent, today at +44 204 620 2747