Senior AI/ML Test Engineer

Job Description

Strategic ACI is seeking a Senior AI/ML Test Engineer to support advanced AI/ML Test & Evaluation (T&E) activities for mission-critical national security programs. The Senior AI/ML Test Engineer is responsible for integrating MT&E capabilities into the ST&E environment and ensuring end-to-end evaluation workflows operate effectively within operationally realistic data and compute environments.

This role designs, develops, and executes integration testing activities focused on latency, throughput, runtime performance, and operational scalability of AI/ML evaluation pipelines. The Senior AI/ML Test Engineer supports the implementation of automated evaluation processes including data capture, drift detection, annotation workflows, and evaluation orchestration aligned with ST&E mission requirements.

The position provides senior-level technical leadership during the establishment of the evaluation architecture, resolving complex integration challenges between evaluation tools, AI/ML models, infrastructure components, and operational environments. This role supports MT&E and ST&E initiatives across evolving AI-enabled mission systems and operational workflows.

Salary will be commensurate within the advertised range, based on applicable qualifications.

Responsibilities:
Integrate MT&E capabilities into the ST&E environment supporting operational AI/ML evaluation activities.

Design and execute integration testing for AI/ML evaluation workflows, including latency, throughput, runtime performance, and scalability assessments.

Develop and support automated evaluation processes including data capture, annotation workflows, drift detection, and evaluation orchestration.

Establish and refine scalable evaluation architectures supporting AI/ML model testing across operationally realistic compute and data environments.

Resolve complex integration issues involving AI/ML evaluation tools, datasets, infrastructure, and operational mission systems.

Analyze system performance, data variability, infrastructure constraints, and operational impacts affecting evaluation reliability and scalability.

Support development and optimization of automated T&E pipelines enabling repeatable and rapid model assessment workflows.

Collaborate with AI/ML engineers, software developers, data scientists, and mission stakeholders to support operational evaluation objectives.

Provide technical leadership and guidance to mid-level engineers supporting steady-state execution and maintenance of evaluation environments.

Deliver technical findings, integration recommendations, and customer briefings supporting mission-focused AI/ML deployment decisions.

**Qualifications:**Required:
Active TS/SCI clearance and the ability to pass a CI polygraph within 30 days.

Bachelor’s degree in IT or STEM related field, or additional relevant experience in lieu of a degree.

8+ years of experience supporting AI/ML testing, system integration, or operational evaluation activities.

Experience designing or supporting automated testing pipelines, evaluation workflows, or distributed compute environments.

Familiarity with AI/ML model evaluation, integration testing, and performance assessment methodologies.

Experience troubleshooting complex integration and infrastructure challenges within mission or enterprise environments.

Strong analytical, technical, and communication skills with experience supporting customer briefings and technical assessments.

Desired:
Experience supporting DoD, NGA, or Intelligence Community programs.

Familiarity with cloud-based architectures, containerized workflows, and DevSecOps environments.

Experience with automated testing frameworks, CI/CD pipelines, and AI/ML operational environments.

Familiarity with drift detection, data capture pipelines, annotation workflows, and model performance analytics.

Experience supporting operational AI/ML evaluation environments and mission-focused testing activities.

,

About Strategic ACI

Strategic ACI is an Equal Opportunity Employer.

It has been and will continue to be a fundamental policy of Strategic ACI not to discriminate on the basis of race, color, creed, religion, gender, gender identity, pregnancy, marital status, partnership status, domestic violence victim status, sexual orientation, age, national origin, alienage or citizenship status, veteran or military status, disability, medical condition, genetic information, caregiver status, unemployment status or any other characteristic prohibited by federal, state and/or local laws.

This policy applies to all aspects of employment, including hiring, promotion, demotion, compensation, training, working conditions, transfer, job assignments, benefits, layoff, and termination. Applicants can learn more about the company’s status as an equal opportunity employer by viewing the federal EEO is the Law poster.

Why choose a career with Strategic ACI?

Strategic ACI believes that our greatest asset is our employees. Our goal is not to meet our staff’s expectations, but to exceed them. Competitive salaries, work-life balance, industry leading benefits packages, and family first values are at the core of Strategic ACI’s culture.

Where benefits meet your needs:
Competitive salary

100% benefits paid (Includes: Health, Dental, and Vision premiums) for all full-time employees, and their families

401(k) with 5% match vested at day one

PTO – 3 weeks and 3 days per year, increasing at 5 years service

11 Company Paid Holidays (aligned with Federal Government)

Long term/Short term disability

1.5x salary Life Insurance

$100 per month Cellular Device and IT Resource Benefit

$6,000 Cash in Lieu of Benefits per year, if employee is insured elsewhere

Tuition reimbursement of up to $5,250 per year for college or professional certifications

Retirement Savings 401(k) Contribution - Annually calculated as percentage of salary