#jA BUdZddlZddlZddlZddlmZmZddlmZddl m Z ddl m Z ddl mZddlmZdd lmZmZejeZd ed efd ZGd dZGddZeaejezed<daeed<GddZ ddddddddddd Z! d3ded ed!eed"ed df d#Z"d ejezfd$Z#d4d%ed&edzfd'Z$d efd(Z%d)Z&d*ed efd+Z'Gd,d-Z(Gd.d/Z)da*ee(ed0<d e(e)zfd1Z+d e(fd2Z,dS)5a OpenTelemetry distributed tracing instrumentation for Hindsight API. This module provides tracing for: - LLM API calls with full prompts/completions following GenAI semantic conventions - Token usage and model information - Error tracking and finish reasons Tracing is conditional and disabled by default. When enabled, traces are exported to Langfuse (or any OTLP-compatible backend) via OTLP HTTP protocol. N)AnyOptional)trace)OTLPSpanExporter)Resource)TracerProvider)BatchSpanProcessor)Status StatusCodeobjreturnct|tr|St|dr|St|dr|St|dr&t j|St|dr&t j|St j|S)zASerialize an object for span recording, handling Pydantic models.model_dump_jsonjson model_dumpdict) isinstancestrhasattrrrdumpsrr)r s ]/home/rurouni/.hermes/hermes-agent/venv/lib/python3.11/site-packages/hindsight_api/tracing.py_serialize_for_spanrs#s s%&&%""$$$sFxxzzsL!!,z#..**+++sF&z#((**%%% :c??c*eZdZdZdefdZdefdZdS) NoOpTracerzWNo-op tracer that provides the same interface as OpenTelemetry Tracer but does nothing.namec 8ddlm}|d}|S)z6Return a no-op context manager that yields a NoOpSpan.r)contextmanagerc3*KtVdSNNoOpSpanrrnoop_span_contextz;NoOpTracer.start_as_current_span..noop_span_context8s**     r) contextlibr)selfrkwargsrr$s rstart_as_current_spanz NoOpTracer.start_as_current_span4s>------     ! """rc tS)zReturn a no-op span.r!)r&rr's r start_spanzNoOpTracer.start_span>s zzrN)__name__ __module__ __qualname____doc__rr(r*r#rrrr1sPaa######srrczeZdZdZdededdfdZdeddfdZd eddfd Z dd ed e dzddfd Z dde dzddfdZ dS)r"zSNo-op span that provides the same interface as OpenTelemetry Span but does nothing.keyvaluer NcdSzNo-op.Nr#)r&r0r1s r set_attributezNoOpSpan.set_attributeF rstatuscdSr3r#)r&r6s r set_statuszNoOpSpan.set_statusJr5r exceptioncdSr3r#)r&r9s rrecord_exceptionzNoOpSpan.record_exceptionNr5rr attributescdSr3r#)r&rr<s r add_eventzNoOpSpan.add_eventRr5rend_timecdSr3r#)r&r?s rendz NoOpSpan.endVr5rr )r+r,r-r.rrr4r8 Exceptionr;rr>intrAr#rrr"r"Cs]]  S T           )       c td{ d      C$J $      rr"_tracerF_tracing_enabledc>eZdZdZdZdZdZdZdZdZ dZ d Z d Z d Z d Zd S)GenAIAttributesz*GenAI semantic convention attribute names.zgen_ai.operation.namezgen_ai.provider.namezgen_ai.request.modelzgen_ai.response.modelzgen_ai.usage.input_tokenszgen_ai.usage.output_tokenszgen_ai.system_instructionszgen_ai.input.messageszgen_ai.output.messageszgen_ai.response.finish_reasonsz error.typeN)r+r,r-r.OPERATION_NAME PROVIDER_NAME REQUEST_MODELRESPONSE_MODELUSAGE_INPUT_TOKENSUSAGE_OUTPUT_TOKENSSYSTEM_INSTRUCTIONSINPUT_MESSAGESOUTPUT_MESSAGESFINISH_REASONS ERROR_TYPEr#rrrGrGasZ44-N*M+M,N567,N.O6NJJJrrGopenai anthropicgooglegroqollamalmstudiomock) rSrTgeminivertexairVrWrXz openai-codexz claude-coderY development service_nameendpointheadersdeployment_environmentcVtj|d|d}i}|r^|dD]H}d|vrB|dd\}}|||<I|dr|n|d} t | |} t |} | t| tj | tj ta d atd | d |d S) a Initialize OpenTelemetry tracing with OTLP exporter. Args: service_name: Name of the service for resource attributes endpoint: OTLP endpoint URL (e.g., https://cloud.langfuse.com/api/public/otel) headers: Optional headers in format "key1=value1,key2=value2" deployment_environment: Deployment environment (e.g., development, staging, production) z0.4.8)z service.namezservice.versionzdeployment.environment.name,=z /v1/traces)r^r_)resourceTzTracing initialized: endpoint=z , service=N)rcreatesplitstripendswithrradd_span_processorr rset_tracer_provider get_tracerr+rDrEloggerinfo) r]r^r_r`re headers_dictpairr0r1 otlp_endpoint otlp_exporterproviders rinitialize_tracingrtsQ$(&+A  HL:MM#&& : :Dd{{!ZZQ// U,1KKMM SYY[[)!) 1 1, ? ?\HHE\E\E\M$M x000H  2= A ABBB h'''x((G KKXXX,XXYYYYYrctS)z Get the global tracer instance. Returns a no-op tracer if tracing is disabled, so callers don't need to check for None. This improves code readability by allowing direct use without null checks. )rDr#rrrlrls  Nr operationbank_idctrtddlm}|Sd|}t|}|r>t |dr.|d||r|d||S)a Create a parent span for a Hindsight operation (retain, reflect, consolidation, etc.). This creates the span hierarchy: - hindsight.{operation} (parent) - chat {model} (child LLM calls) Args: operation: Operation name (retain, reflect, consolidation, mental_model_refresh) bank_id: Optional bank ID for context Returns: Span context manager Nr) nullcontext hindsight.r4zhindsight.operationzhindsight.bank_id)rErDr%ryr(rr4)rvrwry span_namespans rcreate_operation_spanr}s w******{}}(Y((I  ( ( 3 3D =o..= 0)<<<  =   2G < < < KrctS)zCheck if tracing is enabled.)rEr#rris_tracing_enabledrs ricontentct|tkr+|dtdt|tz dzS|S)z'Truncate content if too large for span.Nz [TRUNCATED: z chars omitted])lenMAX_CONTENT_LENGTH)rs r_truncate_contentrsH 7||(((***+.sWPbAb.s.s.sss Nrc`eZdZdZdejfdZ ddedededee eefd e ed e d e d e d e ede e de ee eefddfdZdee eefdefdZde ed e edefdZdee eefde efdZdS)LLMSpanRecorderzY Records OpenTelemetry spans for LLM calls following GenAI semantic conventions. tracerc||_dSr r)r&rs r__init__zLLMSpanRecorder.__init__s  rNrsmodelscopemessagesresponse_content input_tokens output_tokensduration finish_reasonerror tool_callsr c T t||} d} |rd|}n| d|}tj}|t |dzz }|j||d5}|tj | |tj | |tj ||tj ||tj||tj||d||d|| rX|d t!| d | D}|d d |||}||| }||}i}|r||tj<|r||tj<|r||tj<| r"t1j| g|tj<|d || rt9| D]s\}}|dd|ddt1j|did}|d||t| r|t=t>j tC| |tj"tG| j$|%| n,|t=t>j&|'|ddddS#1swxYwYdS#tP$r*}tR*d|dYd}~dSd}~wwxYw)a Record a completed LLM call as a span with GenAI semantic conventions. This creates a span AFTER the call completes, using timestamps to set the correct start/end times. This approach works better with the existing sync metrics recording pattern. Args: provider: Hindsight provider name model: Model name scope: Scope identifier (memory, reflect, consolidation, etc.) messages: Input messages (chat history) response_content: Response text from LLM input_tokens: Input token count output_tokens: Output token count duration: Call duration in seconds finish_reason: Reason the model stopped (stop, length, tool_calls, etc.) error: Exception if call failed tool_calls: List of tool calls made (for function calling) chatrz iʚ;F) start_time end_on_exitzhindsight.scopezhindsight.provider.internalzgen_ai.tool_calls.countc:g|]}|ddS)r)get).0tcs r z3LLMSpanRecorder.record_llm_call..Qs&!J!J!J"&&"4"4!J!J!Jrzgen_ai.tool_calls.namesrbz)gen_ai.client.inference.operation.details)r<rrid arguments)z tool.nameztool.idztool.argumentszgen_ai.tool_call.)r?NzFailed to record LLM span: T)exc_info)+PROVIDER_NAME_MAPPINGrlowertimetime_nsrCrr(r4rGrHrIrJrKrLrMrjoin_format_messages_format_output_extract_system_instructionsrOrPrNrrrQr> enumerater8r r ERRORrrRtyper+r;OKrArBrmr)r&rsrrrrrrrrrrgenai_provideroperation_namer{ end_time_ns start_time_nsr| tool_namesinput_messages_jsonoutput_messages_jsonsystem_instructions event_attrsirtool_event_attrses rrecord_llm_callzLLMSpanRecorder.record_llm_callsDZ K266x~~7G7GIYIYZZN$N 8000  .7777 ,..K'#h.F*G*GGM22(!3B /""?#A>RRR""?#@.QQQ""?#@%HHH""?#A5III""?#E|TTT""?#F VVV""#4e<<<""#@(KKKX&&'@#j//RRR!J!Jz!J!J!JJ&&'@#((:BVBVWWW'+&;&;H&E&E#'+':':;K]'['[$'+&G&G&Q&Q#! &VBUK >?'XCWK ?@&[GZK CD ^BF*m_B]B]K >??* ]!*:!6!6]]2)+););')vvdB'7'7.2j R9P9P.Q.Q,,( '>1'>'>K[\\\\;OOF:+;SZZ$H$HIII&&'A4;;CWXXX))%0000OOF:=$9$9:::+...EB /B /B /B /B /B /B /B /B /B /B /B /B /B /B /B /B /B /H K K K LL:q::TL J J J J J J J J J Ks=BO3MO& O3&O**O3-O*.O33 P'=P""P'cp g}|D]h}|dd}t|trt|}||dd|dit j|S#t$r(}t d|Yd}~dSd}~wwxYw) z Format messages into GenAI semantic convention format (JSON array). Returns JSON string representation of message array. rrroleuserrrz!Failed to format input messages: N[]) rrrrappendrrrBrmwarning)r&r formattedmsgrrs rrz LLMSpanRecorder._format_messagess  I  '')R00gs++9/88G  # 7 7#*:i(( (    NNBqBB C C C44444 sBB B5 B00B5rc |dSt|trt|}tjd|dgS#t $r(}t d|Yd}~dSd}~wwxYw)zrsk     RRRRRR000000222222======22222222  8 $ $SS*$        2&0Z\\ "111$:    """/ 5Z5Z5Z5Zc]5Z 5Z  5Z5Z5Z5ZpEL:-S3:BD ssEEEEEEEEP        -1)000?-@@or