Positioning
HoneyHive is a hosted observability and eval platform, now positioned as agent observability, aimed at enterprise teams who need a vendor-grade UI without sending raw production traces to a shared vendor cloud. The product is OpenTelemetry-native, which keeps the instrumentation layer portable. The v2 docs describe a federated architecture: the control plane (authentication, RBAC, workspace and project configuration) stays with HoneyHive, and the data plane (trace ingestion, event enrichment, evaluation jobs, storage) runs in one of three places. Those are multi-tenant SaaS in AWS US-West-2, a dedicated data plane in your chosen AWS region, or a self-hosted deployment in your own environment 1. The management API calls these data planes "virtual data planes." Commonwealth Bank of Australia, which HoneyHive describes as Australia's largest bank, with 17M retail consumers and 55K internal users, is the publicly named customer.
The product covers tracing, dataset management, online and offline evals, and the usual production observability surface. The strongest fit is large enterprises with strict data-residency rules where Langfuse self-host is too much operational burden and a fully hosted SaaS is a non-starter. Self-hosted HoneyHive is run by HoneyHive, not by you, and starts with a sales conversation 2.
Strengths
- OpenTelemetry-native. Traces are emitted as OTel spans, so the instrumentation layer is portable across HoneyHive and any other OTel backend. This avoids the lock-in of proprietary trace schemas.
- Data-plane split for residency. Raw traces, evals, and datasets stay on the data plane, which can be dedicated to you in an AWS region or self-hosted, while HoneyHive's control plane handles authentication and configuration and, per the docs, never touches application data. For regulated industries this is often the only acceptable shape 3. Moving from shared to dedicated infrastructure does not require SDK changes 1.
- Public enterprise reference. Commonwealth Bank of Australia as a named customer is a useful signal that the platform handles regulated production traffic at scale.
Limitations
- Smaller community. Compared to Langfuse, Phoenix, and LangSmith, the public surface area (blog posts, integration recipes, third-party tutorials) is thinner. Internal teams typically rely on vendor support for non-trivial integrations.
- No public paid pricing. The Developer tier is free and published; everything above it is sales-led with no rate card. Confirm pricing terms, and what counts as an event, before committing to procurement.
- Self-hosting docs cover AWS only. The self-hosted guide describes AWS EKS with Terraform and Helm, in a dedicated account or your own VPC. If you need another cloud or an air-gapped cluster, confirm it before shortlisting 2.
- Vendor-published benchmarks. Performance claims from HoneyHive are vendor-published.
Best fit
Enterprise deployments where data residency is a hard requirement and self-hosted Langfuse is too operationally heavy. Production observability for regulated workloads in finance, healthcare, and government. Hybrid architectures where HoneyHive is the trace hub and CI pushes eval results from Promptfoo or DeepEval.
Getting started
import os
from honeyhive import HoneyHiveTracer, evaluate
from openinference.instrumentation.openai import OpenAIInstrumentor # swap for your provider
# The tracer reads HH_INGESTION_API_KEY from the environment.
tracer = HoneyHiveTracer.init(source="dev", session_name="rag-dev")
OpenAIInstrumentor().instrument(tracer_provider=tracer.provider)
# Your app runs as normal; OTel spans stream to the configured data plane.
answer = run_rag(question, context)
# Evaluators are plain callables: (outputs, inputs, ground_truth) -> score.
def contains_expected(outputs, inputs, ground_truth):
return 1.0 if ground_truth["answer"].lower() in outputs["answer"].lower() else 0.0
# Run a dataset-level eval from a notebook or CI (evaluate() needs HH_API_KEY).
evaluate(
function=lambda datapoint: {
"answer": run_rag(datapoint["inputs"]["question"], datapoint["inputs"]["context"])
},
dataset_id=os.environ["HH_DATASET_ID"],
evaluators=[contains_expected],
name="rag-answer-match",
)
The HoneyHive UI then renders the trace, the eval scores, and a diff against any previous run on the same dataset. LLM and Python evaluators can also run server-side after ingestion, including as online evals on production traces. This snippet follows the v2 quickstarts and was not run here.
Credentials moved in the Python SDK. v1.0.1 (May 5, 2026) deprecated the project parameter and HH_PROJECT, and v1.6.0 (September 17, 2026) added HH_INGESTION_API_KEY for tracing 4. Ingestion keys are bound to a single project, cannot read data back, and never expire, so a lapsed key cannot silently stop telemetry 5. Older tutorials that pass project= to HoneyHiveTracer.init predate the change. The SDK (honeyhive on PyPI, v1.6.1 as of 2026-09-29) is MIT licensed 6.
Pricing notes
HoneyHive publishes a free Developer plan (fetched as of 2026-10-01): 10,000 events per month, 30-day retention, up to 5 users, one workspace, and 1,000 requests per minute, on multi-tenant SaaS in AWS US-West-2. Enterprise is custom-priced through sales and adds unlimited users and workspaces, SSO and custom RBAC, an uptime SLA, and a choice of multi-tenant SaaS, single-tenant SaaS, hybrid, or self-hosted deployment. Startups with under $5M raised can get discounts 7. There is no published rate card between those two. Practical eval programs in production tend to follow the patterns described in applied-LLM writeups 8; budget accordingly.
Alternatives
Consider Langfuse for an OSS self-hostable hub. Consider Arize Phoenix for an OTel-native OSS alternative. Consider Braintrust or LangSmith for hosted-first platforms where residency is less of a constraint. For pure CI regression, DeepEval or Promptfoo.
Citations and last verified
Verified 2026-10-01 against HoneyHive's homepage and pricing page, the v2 docs (platform architecture, self-hosted setup, quickstarts, product and client changelogs), and the PyPI listing.