Machine Learning Engineer at Capitol AI

Capitol makes it easy to create compelling multimodal documents from data. Capitol lets users create custom document formats that can be reused to automate document creation and iterate with a model to create the perfect artifact. We have a D2C product that user's can buy with a credit card and B2B product for companies that want to embed our rich creation and editing experience into their application.

Role Overview

We're seeking a Machine Learning Engineer to lead our LLM evaluation and new model adoption/integration process. This critical role will drive the continuous improvement of our AI capabilities, ensuring Capitol AI remains at the forefront of multimodal content creation.

Key Responsibilities

1. Lead the development and implementation of comprehensive evaluation methodologies for our LLM systems

2. Spearhead the process of identifying, evaluating, and integrating new language models into our platform

3. Design and conduct experiments to assess model performance in multimodal content generation scenarios

4. Collaborate with product and engineering teams to translate evaluation insights into concrete platform improvements

5. Develop benchmarks and metrics to quantify the quality and effectiveness of generated content across various modalities

6. Optimize model performance for both our consumer-facing tool and API-integrated enterprise solutions

7. Stay abreast of the latest developments in LLM technology and evaluation techniques

Required Qualifications

- MA or PhD in Computer Science, Machine Learning, or related field, with a focus on Neural Networks, NLP or multimodal AI systems

• 3+ years of experience in applied machine learning

• Extensive experience with Python and deep learning frameworks such as PyTorch or TensorFlow

• Proven track record in developing evaluation metrics and methodologies for complex AI systems

• Strong background in NLP, including experience with state-of-the-art language models

• Experience with LLM fine-tuning and prompt engineering Preferred Qualifications

• Familiarity with multimodal content generation and document processing

• Familiarity with cloud platforms (AWS, GCP, or Azure) and MLOps tools

• Experience with API design and integration for AI services What We Offer

• Opportunity to shape the future of AI-driven content creation

• Work with data from leading organizations

• Meaningful Equity participation

• Remote work arrangement Capitol has an in-house LLM orchestration layer with a generation pipeline that includes our own implementation of function calling, RAG, and chain of thought reasoning. We also have our own fine-tuning pipeline for function-specific small models. Our backend is python, our cloud is managed in terraform, our application CRUD is Clojure (LISP fans welcome) and our frontend is React.

We have a 3 part interview process.

• Initial introductory call where you learn about job and we learn about you Initial introductory call where you learn about job and we learn about you

• technical interview with in person or take home eval technical interview with in person or take home eval

• final alignment interview with another staff member

Then we make an offer, or not depending on performance on steps 1-3. final alignment interview with another staff member

Then we make an offer, or not depending on performance on steps 1-3.