#LLM evaluation

2 articles tagged

Chart: Shieldstral 1.0 3B leads 4 of 18 text-safety benchmark rows and 2 of 3 multimodal rows on Mistral's tables

AI Strategy · Aug 24, 2026 · 11 min read

Mistral shipped Shieldstral 1.0 on 4 August 2026; it wins 4 of 18 text-safety rows

Mistral released Shieldstral 1.0 as Apache 2.0 open weights on 4 August 2026. On Mistral's own benchmark tables, GPT-OSS-Safeguard-20B leads 9 of 18 text rows to Shieldstral's 4.

Manu Shukla

#Mistral #content moderation #ai guardrails

Abstract 3D hero image of an AI agent evaluation dashboard with tool-call trajectories on a dark background

AI Tools · Aug 1, 2026 · 12 min read

LLM tool-use reliability in 2026: how to evaluate models for AI agents

State-of-the-art function-calling agents still solve under 50% of real tasks and stumble on retries. A 2026 guide to evaluating LLM tool-use reliability with pass^k, tau-bench, BFCL, and your own eval harness.

Manu Shukla

#AI agents #LLM evaluation #tool use