AI Architect

Storyful is an equal opportunity employer

Job Description :

Reporting to: Chief Product & Technology Officer (CPTO)

Dublin (Hybrid — 3 days/week in office)

Team: Product & Engineering (foundational hire for Data Science / AI function)

Mission: Build a next-generation Risk & Insights Intelligence Platform that disrupts media monitoring, social listening, and LLM monitoring—from early prototypes to commercially successful, market-leading products.

This role is for someone who can architect and build (hands-on) agentic LLM Systems in production, partner deeply with Data Scientists, and obsess over evaluation, quality, and cost—while thriving in the ambiguity of zero-to-one product creation.

Why this role exists:

We’re building an AI-native platform that detects, explains, and helps teams respond to reputational and narrative risk. You’ll shape the technical direction including network science and explainability early: agent ecosystems, information retrieval (e.g. RAG + Graph RAG), multi-document reasoning, classification, scoring, evaluation, and LLMOps—and turn them into reliable product experiences.

What you’ll do (Responsibilities)

1) Architect and ship agentic GenAI systems

• Design and implement agent ecosystems (multi-agent architectures) that deliver real product outcomes (not demos).
• Build specialized agents for workflows like adverse media / risk detection, entity investigation, source authenticity, classification, and summarization—and orchestrate them reliably.
• Own the translation from research/prototypes into production-grade features (latency, reliability, observability, cost).

2) Build RAG + Graph RAG for multi-doc intelligence

• Deliver RAG chatbots for investigation and exploration across large document sets.
• Implement multi-document summarization, including Graph RAG patterns (graph extraction, linking entities/claims, narrative threads).
• Implement semantic chunking / paragraph splitting, retrieval strategies, and citation/grounding patterns suitable for risk/comms teams.
• deep agents or deep research; graph traversal strategies (network science); agentic RAG

3) Multi-document classification + scoring (risk-focused)

• Build instruction-based and ML-assisted classification pipelines for multi-document inputs (themes, narratives, risk taxonomy). Explore generating data to fine tune small models.
• Create scoring methodologies (e.g., risk score, severity, momentum/growth, confidence, exposure) with a clear rationale and calibration approach.
• Bonus: experience building “risk detection” classifiers and adverse media style pipelines.

4) Context engineering + automatic prompt improvement

• Lead prompt engineering practices across the product: reusable prompt assets, versioning, guardrails, and domain adaptation.
• Implement prompt evolution techniques (e.g., automated prompt iteration / prompt improvement loops) where it makes commercial sense.
• Understand the impact of the words in a prompt into the distribution of probabilities the LLM outputs, managing context, through graphs and information retrieval

5) Evaluation: make quality measurable and repeatable

• Build robust evaluation methodologies for prompts, RAG, summarization, and classification.

• Apply multiple evaluation techniques, including:

  • offline metrics (precision/recall/F1 where appropriate)
  • retrieval metrics and ablations
  • LLM-as-a-judge style evaluations with rubrics, controls, and drift detection

• Define quality gates that allow the team to move fast without breaking trust.

• Understanding an LLM as a neural network, and not only something that can be prompted and observed from the outside. For example understanding how entropy can be a signal to detect hallucinations while they unfold through the layers of the model.

6) LLMOps + cost control

• Implement LLMOps: experiment tracking, model/prompt versioning, dataset management, observability, and release practices.

• Build monitoring for quality + safety + cost, and actively optimize infrastructure spend in cloud environments.

• Deploying and maintaining open source models

7) Lead by influence (and occasionally by direct leadership)

• Bring “Senior/Lead Engineer” judgement: clean architecture, pragmatic decisions, mentoring, unblock teams.
• Partner tightly with Product, Design, Data Science, and Engineering—while also being able to execute independently.

What success looks like (first 6–12 months)

• A production-grade agentic architecture powering key workflows (investigate summarize classify score recommend action).
• A measurable evaluation framework where quality improves release over release.
• A Graph RAG (or equivalent) capability that materially improves multi-doc summarization accuracy and defensibility.
• Clear cost/performance tradeoffs and observability that make the system operable at scale.
• A team around you that’s leveled up in GenAI engineering practices.

Required experience (Must-have)

• Proven background as a Senior / Lead Engineer (or equivalent staff-level scope), owning architecture and delivery.

• Demonstrated experience building agentic GenAI architecture for commercially successful product features (not only internal prototypes).

• Strong experience working with Data Scientists on ML algorithms, NLP, evaluation design, and productionization.

• Hands-on experience in AWS and GCP (Azure acceptable as additional).

• Production experience with:

  • RAG chatbots
  • multi-document summarization (ideally Graph RAG)
  • multi-document classification
  • scoring methodologies (risk scoring is a strong bonus)

• Deep expertise in prompt engineering and evaluation, including both classical metrics (e.g., precision/recall) and LLM-as-a-judge approaches.

• Strong LLMOps and GenAI product design experience: experimentation deployment monitoring iteration.

Nice-to-have (Strong bonuses)

• Experience in risk/compliance domains (e.g., adverse media, AML, entity investigation workflows).
• Knowledge graphs in production (e.g., Neo4j) and graph extraction pipelines.
• Experience running annotation programs / building labeled datasets for NLP tasks.

Skills & tools (examples)

We don’t require exact matches, but we do expect you to be fluent in this class of tooling and able to choose pragmatically.

GenAI frameworks & LLMs

• LangChain, LlamaIndex
• OpenAI / Gemini / Claude
• Vector RAG + Graph RAG patterns

LLMOps / experimentation / observability

• MLflow (experiments, tracking)
• Langfuse (prompt & trace observability)

Data & retrieval

• Neo4j (graph), ElasticSearch
• Vector stores (Pinecone-style capability), embeddings, semantic chunking

Cloud / infrastructure (examples)

• AWS: Lambda, SQS/SNS, Kinesis, Glue, Athena, Redshift, DynamoDB, RDS, API Gateway, CloudFront, SageMaker, Comprehend, Kendra, Lex
• GCP (plus Azure exposure helpful)

Languages

• Python (primary), TypeScript, Java (Ruby on Rails experience welcome)

Job Category:

Storyful - Product & Technology