Positioning
Phoenix is the open-source side of the Arize stack: OpenTelemetry-native tracing, eval primitives, datasets, and a notebook-friendly UI. Arize AX is the enterprise platform built on the same trace model and adds production observability, alerting, and team workflows. Phoenix's license is the Elastic License v2, which permits internal use but restricts hosting Phoenix as a commercial service 1.
The product surface covers tracing, prompt experimentation, datasets, LLM-judge and code evaluators, and an evals library you can call from a notebook or CI.
Two changes affect what you write as of 2026-10-01. The evals library is now phoenix-evals 2.x: you build a judge with create_classifier or pick one from phoenix.evals.metrics (faithfulness, retrieval relevance, tool selection, tool invocation, PII detection, and more), then run it over a dataframe with evaluate_dataframe. The older run_evals and OpenAIModel names are no longer exported from phoenix.evals 2. And Phoenix now ships an agent. PXI is a built-in assistant that investigates failing traces, drafts prompts and evaluators, and turns failures into datasets. It needs an API key for at least one LLM provider, requires Phoenix 20.0.0 or newer, is labeled beta by Phoenix, and admins can disable it 3. The 14 September 2026 release notes add plugins for Claude Code, Codex, and Cursor 4.
Strengths
- OTel-native. Traces are emitted as OpenTelemetry spans, so the same instrumentation can fan out to Phoenix, AX, or any other OTel backend. This is the rare eval platform that does not lock you into a proprietary trace schema.
- High distribution. The Phoenix product page claims 3M+ monthly downloads and the GitHub repo shows 11.7k stars (as of 2026-10-01). Named customers for the OSS product include GitHub, Microsoft, MongoDB, and ByteDance. On the enterprise side, Arize's customer pages name Booking.com, Reddit, Tripadvisor, and PagerDuty for AX.
- Strong notebook story. The tutorials and cookbook walk through eval programs that run locally with no service dependency, which makes Phoenix easy to try on a Friday afternoon 5.
Limitations
- ELv2 restricts commercial hosting. You cannot resell Phoenix as a managed service. Most teams will not hit this limit, but it is worth knowing if you build internal platforms for external customers.
- Two products to keep straight. Phoenix and Arize AX are different SKUs with overlapping but distinct UIs. Picking one and sticking with it avoids confusion.
- Self-interested benchmarks. Performance and quality claims published by Arize are vendor-published; treat them as positioning.
Best fit
Hybrid architectures where Phoenix runs as the OSS observability hub and CI pushes eval results back through the OTel trace surface, RAG evaluation with the Phoenix evals library, and notebook-driven prototype evals that may later graduate to Arize AX without changing instrumentation.
Getting started
import phoenix as px
from phoenix.client import Client
from phoenix.evals import LLM, evaluate_dataframe
from phoenix.evals.metrics import FaithfulnessEvaluator
from phoenix.otel import register
from phoenix.trace.dsl import SpanQuery
tracer_provider = register(project_name="rag-prod")
px.launch_app() # local UI at http://localhost:6006
# Your RAG pipeline runs as normal; spans are captured automatically.
answers = run_rag_pipeline(questions)
# Pull the captured spans into a dataframe with input and output columns.
query = SpanQuery().where("name == 'rag.answer'").select(
input="input.value", output="output.value"
)
spans_df = Client().spans.get_spans_dataframe(project_identifier="rag-prod", query=query)
spans_df["context"] = attach_retrieved_context(spans_df) # your code: join retrieved passages per row
# Score each answer against its retrieved context.
llm = LLM(provider="openai", model="gpt-4o-mini")
results = evaluate_dataframe(
dataframe=spans_df,
evaluators=[FaithfulnessEvaluator(llm=llm)],
)
print(results.head())
FaithfulnessEvaluator grounds the answer against a context column you supply, which is the right check for a RAG pipeline. The HallucinationEvaluator in the same module grounds against the conversation history instead, so it fits chat agents more than retrieval pipelines 2. This snippet was assembled from the current Phoenix docs and source; it was not run here.
The Phoenix tutorials folder includes notebooks for an evals quickstart, agent evaluation, trace-level evals, and session-level evals 1. The evaluation cookbook walks through building a benchmark dataset and validating a custom LLM judge against it 5.
Pricing notes
Phoenix is free under the Elastic License v2 for internal use. Self-hosting is a Docker container; storage is SQLite by default and PostgreSQL is the production recommendation 6. Arize AX now publishes tiers (fetched as of 2026-10-01): AX Free at 50 per month (50k spans, 10 GB, 30-day retention), and custom Enterprise 7. The Phoenix site links a hosted Phoenix Cloud sign-in, but the pricing page lists no Phoenix Cloud limits, so confirm retention terms before relying on it in production.
Alternatives
Consider Langfuse for a more permissively licensed self-hostable hub (MIT core). Consider LangSmith for tighter LangChain gravity. Consider Braintrust for a hosted, framework-agnostic console. Consider Comet Opik for an Apache 2.0 alternative.
Citations and last verified
Verified 2026-10-01 against the Phoenix GitHub repository and PyPI listing, the Phoenix docs (release notes, PXI, self-hosting configuration), the arize-phoenix-evals source, and Arize pricing.