Positioning
Galileo sells an eval-to-guardrail story: the evaluator that scores your offline test set in development can be redeployed inline as a production guardrail. The differentiator is the Luna family of distilled judges, small purpose-trained models that approximate a frontier LLM judge at a fraction of the latency and cost. Distilled judges sit in a research lineage that goes back to Prometheus, Auto-J, JudgeLM, and follow-ups 1. The first Luna was a 440M-parameter DeBERTa-large model for RAG hallucination detection 2. Luna-2 is a family of fine-tuned Llama 3B and 8B models 3 with preset metrics for safety, RAG, and agents, including context adherence, prompt injection, tool selection quality, and action completion 4.
The product covers traces, datasets, experiments, 20+ out-of-the-box evaluators, custom LLM-judge and code metrics, an Insights engine for clustering failures, and runtime guardrails. Hosted SaaS, VPC, and on-prem are all options. The platform is not open source.
Ownership changed in 2026. Cisco announced its intent to acquire Galileo on April 9, 2026 and completed the deal on May 22, 2026, with Galileo going into the Splunk Observability portfolio 5. Splunk now calls it Splunk Agent Observability 6. As of October 2026 galileo.ai, its docs, and its self-serve plans are still live, but the old Luna-2 product page redirects to splunk.com.
The guardrail half also changed. Galileo deprecated Protect, its runtime-protection API, in June 2026 and tells new users to set up Agent Control instead 7. Agent Control is a control plane Galileo open-sourced under Apache 2.0 in March 2026: you put a @control() decorator on LLM and tool calls, and policies managed on a server decide whether to deny, steer, warn, log, or allow 8. Luna plugs in through Agent Control's Galileo evaluator package. The Agent Control docs still describe a galileo.luna2 evaluator 9, but release 8.0.0 of agent-control-evaluator-galileo (June 2026) removed it. Its replacement, galileo.luna, calls a Luna scorer by scorer_id, and the package README says to pin below 8 if you still need the old evaluator 10.
Strengths
- Luna distilled judges. Purpose-trained small models that score quality dimensions (context adherence, prompt injection, toxicity, tool selection) at production latency. Useful when a frontier-model judge call per request is too slow or too expensive.
- Eval-to-guardrail loop. The same Luna-2 metric that grades an offline experiment can run inline as a guardrail, which reduces the gap between pre-prod and prod metrics. One catch: Luna-2 metrics are separate metrics from their LLM-judge namesakes (
prompt_injection versus prompt_injection_luna), so to score both sides with the same function you have to run the Luna variant offline too 11. The classical risk with this design, well documented in independent reviews of LLM-evaluator systems, is that the same biases that fool the judge offline also fool it inline 12.
- Judges you can tune. Luna-2 is meant to be fine-tuned on your labeled data; Galileo's docs put the useful size at roughly 4,000 samples, and Luna Studio runs the fine-tune as a self-service workflow 3.
- Enterprise deployment options. VPC and on-prem are available for teams with strict data-residency rules, and Luna-2 can be self-hosted on your own GPUs 3.
Limitations
- Luna-2 is Enterprise-only. The self-serve tiers get tracing and LLM-judge evals. Luna-2 metrics, real-time guardrails, and VPC or on-prem deployment sit in the sales-led Enterprise tier, so the headline feature cannot be trialed without a sales cycle 13 3.
- Ownership in transition. Galileo is being integrated into Splunk Observability. Expect renames, packaging changes, and roadmap decisions driven by the Splunk product line. Ask which entity your contract is with and what the standalone galileo.ai roadmap looks like.
- Guardrail API churn. Teams that built on Protect have a migration to Agent Control ahead of them 7, and Agent Control's own Galileo evaluator already changed its interface once, in June 2026 10.
- Closed judges. Luna-2 is only available through Galileo's Enterprise tier 3. You can fine-tune the models through Galileo but not audit them the way you can audit an open judge. The platform is closed; Agent Control is the one Apache 2.0 piece.
- Vendor-published benchmarks. Galileo's own table puts Luna-2 at 2.50, 0.94, and 3,200 ms 3. Those are vendor numbers on vendor-chosen tasks; treat them as positioning and measure agreement with your own labels.
Best fit
Enterprise teams that already need a production guardrail layer and want to avoid maintaining two parallel scoring pipelines (one for eval, one for guardrails). Workloads where a distilled judge with sub-second latency is preferable to a frontier-model LLM-judge call per request. Regulated industries that need VPC or on-prem deployment. Splunk Observability shops, where Galileo is being built into AI Agent Monitoring 5.
Getting started
# pip install "galileo[openai]" "agent-control-evaluator-galileo<8"
# (the luna2 module below was removed in agent-control-evaluator-galileo 8.0.0)
import os
from galileo import GalileoMetrics
from galileo.experiments import run_experiment
from agent_control_evaluator_galileo.luna2 import Luna2Evaluator, Luna2EvaluatorConfig
# Pre-production: score an offline experiment with a Luna-2 metric (Enterprise tier).
results = run_experiment(
"support-bot-toxicity-luna",
dataset=eval_rows, # list of {"input": ...} dicts
function=answer_question, # your app: takes the input, returns the answer
metrics=[GalileoMetrics.input_toxicity_luna],
project=os.environ["GALILEO_PROJECT"],
)
# Production: run the same Luna-2 check inline through Agent Control's Galileo evaluator.
guard = Luna2Evaluator(
Luna2EvaluatorConfig(
stage_type="local",
metric="input_toxicity",
operator="gt",
target_value=0.5,
galileo_project=os.environ["GALILEO_PROJECT"],
)
)
async def handle(user_message: str) -> str:
verdict = await guard.evaluate(user_message)
if verdict.matched:
return safe_fallback()
return answer_question(user_message)
The experiment call comes from Galileo's Luna-2 experiments guide 11 and the Luna metric names from its metrics reference 4; the evaluator config comes from the Agent Control docs 9, which still describe the pre-8.0 galileo.luna2 evaluator, so the install line pins that version. On current releases, use LunaEvaluator and LunaEvaluatorConfig from agent_control_evaluator_galileo.luna, which take a scorer_id, threshold, and operator instead of a metric name 10. This snippet was assembled from those docs and was not run here. In a real deployment you would usually attach controls to a Galileo Log stream from the console and decorate functions with @control(), so the policy can change without a redeploy 8.
Pricing notes
Galileo now publishes a rate card (fetched as of 2026-10-02): Free at 100 a month billed yearly with 50,000 traces, standard RBAC, and Slack support, scaling with trace volume; and Enterprise at custom pricing with unlimited traces, hosted, VPC, or on-prem deployment, SSO, and real-time guardrails 13. Luna-2 needs Enterprise and L4-class or better GPUs, either Galileo-hosted or your own, and L4s only cover Log stream and experiment metrics, not runtime protection 3. If you plan to pay for Luna-2, run the free tier's LLM-judge metrics first and confirm they track your labels before you buy the distilled version.
Alternatives
Consider Promptfoo or DeepEval for OSS CI evals if you do not need an inline guardrail. Consider Langfuse or Arize Phoenix for OSS observability where the guardrail is a separate component. Consider Braintrust or LangSmith for hosted eval platforms without a first-party distilled-judge model family.
Citations and last verified
Verified 2026-10-02 against the Galileo pricing page and docs (Luna-2 overview, Luna-2 experiments guide, metrics reference, Protect deprecation notice), the Agent Control announcement, docs, and Galileo evaluator package, Cisco's acquisition post, and Splunk's Galileo page.