Positioning
TruLens is an OSS evaluation framework whose central abstraction is the feedback function: a callable that takes a trace (or part of one) and returns a score. The library ships feedback functions for the standard RAG metrics (groundedness, context relevance, answer relevance), plus general-purpose LLM-judge feedback functions for tone, conciseness, and custom rubrics. TruEra built it; Snowflake announced an agreement to acquire TruEra in May 2024 1, and the project site now describes TruLens as shepherded by Snowflake and maintained in open source 2. Releases are frequent: six minor versions shipped between July 23 and September 3, 2026, and 2.14.0 is current as of October 2026 3.
Since 2.7 (February 2026) the class behind a feedback function is Metric, configured with an explicit selectors dict that maps each function argument to a span attribute. Feedback and its chained .on_*() calls still work but emit a deprecation warning 4 3.
TruLens records OpenTelemetry spans; tracing is on by default and spans land in the TruLens database or a Snowflake event table. Since 2.14, TruSession(otel_exporter="otlp") sends traces plus GenAI token and latency metrics to an OTLP gRPC collector instead. "Instead" is the catch: OTLP mode replaces the TruLens store, so the TruLens dashboard and any feedback function that reads spans from that store no longer see them, and the pull request that added it says plainly that it is not a fan-out 5. If TruLens scores the spans, Phoenix or Langfuse does not also get them for free.
Strengths
- Composable feedback functions. The library treats evaluators as first-class objects you compose, so the same groundedness function can be reused across notebooks, CI, and production. The RAG metrics overlap with Ragas and are derived from the same line of work 6.
- Judge-alignment tooling in the box. 2.9 and 2.10 (July 2026) added the calibration loop most teams build by hand:
Jury ensembles several judge models into one metric, AlignmentReport and CriteriaABTest score judges and judge variants against a labeled golden set, CrossModelAlignment runs several judge models on the same inputs and shows where they diverge, so you can pick the cheapest one that stays close to a reference judge, and GoldenSetGenerator samples production records for labeling. Most of these live in the separate trulens-benchmark package 3.
- Offline and online in one library.
BatchEvaluator (2.13) runs metrics over a pandas DataFrame with no instrumentation 7, conversation-level metrics (2.12) score a whole multi-turn transcript once, and configure_online_eval (2.11) samples production traffic for automatic scoring 3.
- OTel emission. Spans are OpenTelemetry-shaped, and 2.14 can export them over OTLP to Tempo, Jaeger, Datadog, Honeycomb, or a collector, within the either-or limit above 5.
- Snowflake stewardship. Under Snowflake, the project has a clear long-term owner. The license remains MIT.
Limitations
- Self-hosted dashboard. The dashboard is a Streamlit app you run yourself, or Snowsight inside Snowflake. 2.13 added Trends, a dashboard page that charts metric scores, latency, and cost over time, and
Run.compare(), which flags per-item regressions between two runs with a bootstrap confidence interval and a permutation p-value 7. That narrows the gap with hosted hubs. Many teams still pair TruLens with a separate trace hub for visualization.
- Documentation lags features. As of October 2026, the OTLP exporter from 2.14.0 is documented in the repository and release notes but not on the OpenTelemetry page of trulens.org, and the site's banner still advertises 2.12 2 5. With six minor releases in six weeks and a class rename, examples written before February 2026 use the deprecated API. Read the release notes when you upgrade.
- Vendor-published benchmarks. Performance claims from TruLens or Snowflake should be treated as vendor positioning.
Best fit
RAG evaluation where you want composable feedback functions in code rather than a hosted metric library, OpenTelemetry-native eval pipelines, teams that want judge calibration tooling in the same library as their metrics, and Snowflake-aligned ML stacks that want native RAG eval primitives.
Getting started
# pip install trulens trulens-providers-openai trulens-apps-langchain
import numpy as np
from trulens.apps.langchain import TruChain
from trulens.core import Metric, Selector, TruSession
from trulens.providers.openai import OpenAI
session = TruSession()
provider = OpenAI(model_engine="gpt-4o-mini")
f_groundedness = Metric(
implementation=provider.groundedness_measure_with_cot_reasons,
name="Groundedness",
selectors={
"source": Selector.select_context(collect_list=True),
"statement": Selector.select_record_output(),
},
)
f_context_relevance = Metric(
implementation=provider.context_relevance_with_cot_reasons,
name="Context Relevance",
selectors={
"question": Selector.select_record_input(),
"context": Selector.select_context(collect_list=False),
},
agg=np.mean,
)
# Wrap a LangChain RAG chain; TruApp and TruLlama cover custom and LlamaIndex apps.
tru_chain = TruChain(
rag_chain,
app_name="rag-prod",
app_version="v1",
feedbacks=[f_groundedness, f_context_relevance],
)
with tru_chain as recording:
rag_chain.invoke("Is the lentil bowl vegan?")
# Scores the recorded call now and blocks until results land: one row per
# record, one column per metric.
print(recording.retrieve_feedback_results())
Groundedness gets the full list of retrieved chunks (collect_list=True) and the answer; context relevance runs once per chunk and averages. Earlier versions of this example used the deprecated Feedback class and scored groundedness against the question instead of the retrieved context, which the explicit selectors make harder to get wrong. run_dashboard(session) from trulens.dashboard renders the same records with their span trees.
Pricing notes
TruLens is free under the MIT license. LLM-call costs (for feedback functions that use an LLM judge) are paid to the underlying provider: OpenAI, Anthropic (2.11), Google, Bedrock, LiteLLM, local Ollama models (2.14), or Snowflake Cortex. Cortex moves the bill rather than removing it. Cortex AI functions are billed in credits per million input and output tokens, and the warehouse running the query is charged on top 8.
For production scoring, configure_online_eval sets a global or per-app sample rate, a cap on evals per minute, and a daily dollar budget. Sampling is a deterministic hash of the record id, so retries and replays get the same decision. The budget only counts providers that report cost, and explicit compute calls bypass all three limits 3. Cache feedback function results by trace id to avoid re-scoring on iterations.
Alternatives
Consider Ragas for the reference implementations of the RAGAS metrics. Consider DeepEval for pytest-style ergonomics. Consider Arize Phoenix for a stronger UI with OTel traces. For hosted dashboards, Langfuse or Braintrust.
Citations and last verified
Verified 2026-10-02 against the TruLens site and Metric migration guide, the GitHub release notes and change history for 2.7.0 through 2.14.0, the OTLP export pull request, the Snowflake acquisition post, and the Snowflake Cortex cost documentation. The getting-started example was checked against the 2.14.0 API.