DevOps Engineer with Data Engineering Experience

Scispot is a fully configurable workflow automation platform for fast-growing life science companies. Our customers use Scispot to design and automate their workflows at all stages of their R&D, from planning, lab execution to reporting collaboratively. Scispot is building the world's first scientific ETL for unstructured, semi-structured and structured data. Scispot staging lakehouse connects with all the popular R&D sources such as Benchling, Quartzy, Lab Instruments, and others & converts them into machine-readable data to support downstream pipelines. We're searching for a seasoned DevOps Engineer with experience in Data Engineering. The right candidate will have an opportunity to become a senior architect as we are moving really fast and scaling our Infrastructure. In this role, you'll be instrumental in ensuring the scalability and stability of our infrastructure as we continue to grow. This position involves a balance of maintaining robust AWS infrastructure, implementing DevOps practices, and handling ETL tasks. Design, manage, and optimize a scalable, secure AWS infrastructure focusing on EC2, ECS, S3, and Lambda services. Implement strategies and solutions for scaling and stabilizing infrastructure. Integrate with various lab instruments using protocols such as ASTM, HL7. Build, configure, and maintain ETL pipelines for various Lab Instruments and Scispot cloud. Maintain high-availability systems, targeting 99.99% uptime. Drive the adoption of Infrastructure as Code (IAC) practices. Work with pub-sub architecture and Rabbit MQ. Develop and maintain ETL processes. Contribute to AI infrastructure using RAG and embeddings. Collaborate with cross-functional teams to translate their needs into technical requirements. Design, manage, and optimize a scalable, secure AWS infrastructure focusing on EC2, ECS, S3, and Lambda services. Implement strategies and solutions for scaling and stabilizing infrastructure. Integrate with various lab instruments using protocols such as ASTM, HL7. Build, configure, and maintain ETL pipelines for various Lab Instruments and Scispot cloud. Maintain high-availability systems, targeting 99.99% uptime. Drive the adoption of Infrastructure as Code (IAC) practices. Work with pub-sub architecture and Rabbit MQ. Develop and maintain ETL processes. Contribute to AI infrastructure using RAG and embeddings. Collaborate with cross-functional teams to translate their needs into technical requirements. Proven experience as a DevOps Engineer focusing on infrastructure management and scalability. Experience with Bioinformatics pipelines Extensive experience with AWS infrastructure including services like EC2, ECS, API Gateway, S3, and Lambda. Strong familiarity with maintaining high-availability systems. Proficiency in Infrastructure as Code (IAC) practices. Experience in a Data Engineering or similar role involving ETL processes. Experience with Rabbit MQ and pub-sub architectures. Familiarity with ETL tools. Experience with Elastic Search Familiarity with Java and Spring Boot is a plus. Proven experience as a DevOps Engineer focusing on infrastructure management and scalability. Experience with Bioinformatics pipelines Extensive experience with AWS infrastructure including services like EC2, ECS, API Gateway, S3, and Lambda. Strong familiarity with maintaining high-availability systems. Proficiency in Infrastructure as Code (IAC) practices. Experience in a Data Engineering or similar role involving ETL processes. Experience with Rabbit MQ and pub-sub architectures. Familiarity with ETL tools. Experience with Elastic Search Familiarity with Java and Spring Boot is a plus. We use AWS as our primary hosting service. Our tech stack comprises a Next.js frontend web app. On the backend, we use a NoSQL database (AWS DynamoDB) which is fronted by a GraphQL database abstraction API layer. We also use AWS Lambda as the middleware for our platform. To manage our user authentication, authorization and signups, we use AWS Cognito in combination with AWS IAM.