About the Role

This role involves conducting comprehensive failure analysis on AI agent performance in finance-sector tasks. The data scientist will identify patterns, root causes, and systemic issues while analyzing performance across multiple dimensions. They'll create dashboards, recommend improvements, and communicate insights to stakeholders.

Key Responsibilities

• Statistical failure analysis across task components
• Root cause analysis of AI agent failures
• Dimension analysis across finance sub-domains and file types
• Dashboard creation and reporting of failure clusters
• Recommend improvements to task design and evaluation criteria

Required Skills & Qualifications

Must Have:

• Experience with AI/ML model evaluation or quality assurance
• Background in finance or willingness to learn finance concepts
• Experience with multi-dimensional failure analysis
• Familiarity with benchmark datasets and evaluation frameworks