$jPdZddlZddlZddlZddlZddlmZmZddlm Z ddl Z ddl m Z m Z mZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZm Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(m)Z)m*Z*m+Z+m,Z,m-Z-m.Z.m/Z/m0Z0m1Z1m2Z2ej3e4Z5GddeZ6Gd d e6Z7Gd d e6Z8Gd dZ9Gdde6Z:Gdde6Z;Gdde6Z<Gdde6Z=Gdde6Z>de?de@de?fdZAGdde6ZBGdd e6ZCGd!d"e6ZDGd#d$e6ZEde6fd%ZFdS)&z Cross-encoder abstraction for reranking. Provides an interface for reranking with different backends. Configuration via environment variables - see hindsight_api.config for all env var names. N)ABCabstractmethod)ThreadPoolExecutor)'DEFAULT_LITELLM_API_BASEDEFAULT_RERANKER_COHERE_MODEL$DEFAULT_RERANKER_FLASHRANK_CACHE_DIR(DEFAULT_RERANKER_FLASHRANK_CPU_MEM_ARENA DEFAULT_RERANKER_FLASHRANK_MODELDEFAULT_RERANKER_GOOGLE_MODEL+DEFAULT_RERANKER_LITELLM_MAX_TOKENS_PER_DOCDEFAULT_RERANKER_LITELLM_MODEL"DEFAULT_RERANKER_LITELLM_SDK_MODEL!DEFAULT_RERANKER_LOCAL_BATCH_SIZE DEFAULT_RERANKER_LOCAL_FORCE_CPU%DEFAULT_RERANKER_LOCAL_MAX_CONCURRENTDEFAULT_RERANKER_LOCAL_MODEL(DEFAULT_RERANKER_LOCAL_TRUST_REMOTE_CODEDEFAULT_RERANKER_PROVIDER%DEFAULT_RERANKER_SILICONFLOW_BASE_URL"DEFAULT_RERANKER_SILICONFLOW_MODELDEFAULT_RERANKER_TEI_BATCH_SIZE!DEFAULT_RERANKER_TEI_HTTP_TIMEOUT#DEFAULT_RERANKER_TEI_MAX_CONCURRENT"DEFAULT_RERANKER_ZEROENTROPY_MODELENV_RERANKER_COHERE_API_KEYENV_RERANKER_COHERE_MODEL ENV_RERANKER_FLASHRANK_CACHE_DIR$ENV_RERANKER_FLASHRANK_CPU_MEM_ARENAENV_RERANKER_FLASHRANK_MODELENV_RERANKER_GOOGLE_PROJECT_ID ENV_RERANKER_LITELLM_SDK_API_KEYENV_RERANKER_LOCAL_FORCE_CPU!ENV_RERANKER_LOCAL_MAX_CONCURRENTENV_RERANKER_LOCAL_MODEL$ENV_RERANKER_LOCAL_TRUST_REMOTE_CODEENV_RERANKER_PROVIDER ENV_RERANKER_SILICONFLOW_API_KEYENV_RERANKER_TEI_BATCH_SIZEENV_RERANKER_TEI_HTTP_TIMEOUTENV_RERANKER_TEI_MAX_CONCURRENTENV_RERANKER_TEI_URL ENV_RERANKER_ZEROENTROPY_API_KEYceZdZdZeedefdZeddZede e eefde e fdZ dS) CrossEncoderModelz Abstract base class for cross-encoder reranking. Cross-encoders take query-document pairs and return relevance scores. returncdS)zFReturn a human-readable name for this provider (e.g., 'local', 'tei').Nselfs j/home/rurouni/.hermes/hermes-agent/venv/lib/python3.11/site-packages/hindsight_api/engine/cross_encoder.py provider_namezCrossEncoderModel.provider_nameFs  Nc KdS)z Initialize the cross-encoder model asynchronously. This should be called during startup to load/connect to the model and avoid cold start latency on first predict() call. Nr2r3s r5 initializezCrossEncoderModel.initializeLs  r7pairsc KdS)z Score query-document pairs for relevance. Args: pairs: List of (query, document) tuples to score Returns: List of relevance scores (higher = more relevant) Nr2r4r:s r5predictzCrossEncoderModel.predictVs  r7r0N) __name__ __module__ __qualname____doc__propertyrstrr6r9listtuplefloatr=r2r7r5r/r/?s  s   ^X    ^   4c3h#8  T%[    ^    r7r/ceZdZUdZdZedzed<dZeed<dddddde fde dzded e d e d e d e d efdZ e de fdZddZdeee e fdeefdZdeee e fdeefdZdS)LocalSTCrossEncodera Local cross-encoder implementation using SentenceTransformers. Call initialize() during startup to load the model and avoid cold starts. Default model is cross-encoder/ms-marco-MiniLM-L-6-v2: - Fast inference (~80ms for 100 pairs on CPU) - Small model (80MB) - Trained for passage re-ranking Uses a dedicated thread pool to limit concurrent CPU-bound work. N _executor_max_concurrentF model_namemax_concurrent force_cputrust_remote_codefp16bucket_batching batch_sizec|pt|_||_||_||_||_||_d|_|t_ dS)a Initialize local SentenceTransformers cross-encoder. Args: model_name: Name of the CrossEncoder model to use. Default: cross-encoder/ms-marco-MiniLM-L-6-v2 max_concurrent: Maximum concurrent reranking calls (default: 2). Higher values may cause CPU thrashing under load. force_cpu: Force CPU mode for local inference. Default: False trust_remote_code: Allow loading models with custom code (security risk). Required for some models like jina-reranker-v2-base-multilingual. Default: False (disabled for security) fp16: Use FP16 (half precision) inference. Faster on MPS and CUDA, may be slower on CPU. Default: False (opt-in via env var). bucket_batching: Sort pairs by token length before batching to reduce padding waste. 36-54% speedup, quality-identical. Default: False (opt-in via env var). batch_size: Batch size for predict() calls. Optimal values vary by hardware and model (MPS: 32, CUDA: 128+). Default: 32. N) rrMrOrPrQrRrS_modelrIrL)r4rMrNrOrPrQrRrSs r5__init__zLocalSTCrossEncoder.__init__vsL>%D(D"!2 .$ .<+++r7r0cdS)Nlocalr2r3s r5r6z!LocalSTCrossEncoder.provider_nameswr7cBK|jdS ddlm}n#t$rtdwxYwtd|jddl}|jrd}tdnd} |j p2t|j do|j j }|rd}n4#t$r'}td |Yd}~nd}~wwxYw ddlmcmcm}dd lm}t|d s0t+|d |jtd n#t$rYnwxYwt/j5t/jd t4t/jd dt/jd dt7jd}|j} |t6j ||j|ddi|j |_|| n#|| wxYw dddn #1swxYwY|j!r>|dkr8|jj"#tdtHj%OtMtHj'dtH_%tdtHj'ddStddS)z9Load the cross-encoder model and initialize the executor.Nr) CrossEncoderzmsentence-transformers is required for LocalSTCrossEncoder. Install it with: pip install sentence-transformersz1Reranker: initializing local provider with model cpuzEReranker: forcing CPU mode (HINDSIGHT_API_RERANKER_LOCAL_FORCE_CPU=1)mpsz/Failed to detect GPU/MPS, falling back to CPU: )XLMRobertaEmbeddings"create_position_ids_from_input_idszFReranker: applied transformers 5.x compatibility patch for XLM-RoBERTaignore)categoryz%.*was not found in model state dict.*)messagez.*UNEXPECTED.* transformerslow_cpu_mem_usageF)device model_kwargsrPz Reranker: FP16 inference enabledreranker max_workersthread_name_prefixz5Reranker: local provider initialized (max_concurrent=)z>Reranker: local provider initialized (using existing executor))(rUsentence_transformersrZ ImportErrorloggerinforMtorchrOcuda is_availablehasattrbackendsr\ Exceptionwarning4transformers.models.xlm_roberta.modeling_xlm_robertamodels xlm_robertamodeling_xlm_robertar]setattrr^warningscatch_warningsfilterwarnings UserWarninglogging getLoggerlevelsetLevelERRORrPrQmodelhalfrIrJrrL) r4rZrordhas_gpue xlm_moduler]transformers_loggeroriginal_levels r5r9zLocalSTCrossEncoder.initializes ; " F  : : : : : : :   E    YYYZZZ   > VF KK_ ` ` ` ` F V*1133ENE22Xu~7I7V7V7X7X"!F V V VTQRTTUUUUUUUU V  U U U U U U U U U U U U a a a a a a:'KLL f8(K  deee    D   $ & & = =  #H{ C C C C  #H6] ^ ^ ^ ^  #H6F G G G G#*"3N"C"C 06N  ( ( 7 7 7 =*lO!"5u!=&*&<  $,,^<<<<#,,^<<<<<' = = = = = = = = = = = = = = =. 9 <5 K  " " $ $ $ KK: ; ; ;  ( 0,>/?#----  ) KKvPcPsvvv w w w w w KKX Y Y Y Y Ys_.>AC DC;;DAE E$#E$:BI= H3I3I  II!Ir:c ddl}|jrtdkrfdttD t tt fd}fd|D}|j||jd }t|d r| nt|}d gtz}t|D]\}}||||<|S|j|jd }t|d r| nt|S) a,Synchronous prediction wrapper for thread pool execution. Supports two optimizations (controlled via .env): - bucket_batching: sort pairs by token length to reduce padding waste (36-54% speedup) - batch_size: explicit batch size for predict() calls (MPS optimal: 32) rNc~g|]9}t|dt|dz:S)rrlen.0ir:s r5 z5LocalSTCrossEncoder._predict_sync..s>VVVqs58A;''#eAhqk*:*::VVVr7c|SNr2)rlengthss r5z3LocalSTCrossEncoder._predict_sync..s WQZr7)keyc g|] }| Sr2r2rs r5rz5LocalSTCrossEncoder._predict_sync..s===E!H===r7F)rSshow_progress_bartolist) numpyrRrrangesortedrUr=rSrrrrE enumerate) r4r:npsorted_indices sorted_pairs sorted_scoresscoresnew_posorig_idxrs ` @r5 _predict_syncz!LocalSTCrossEncoder._predict_syncss    CJJNNWVVVE#e**DUDUVVVG#E#e**$5$5;O;O;O;OPPPN====n===L K// lq/rrM6=mX6V6VoM00222\`an\o\oMUSZZ'F%.~%>%> : :!#0#9x  M$$UtZ_$``")&(";";Mv}}fMr7cK|jtdtj}|t j|j|d{VS)a/ Score query-document pairs for relevance. Uses a dedicated thread pool with limited workers to prevent CPU thrashing. Args: pairs: List of (query, document) tuples to score Returns: List of relevance scores (raw logits from the model) N2Reranker not initialized. Call initialize() first.)rU RuntimeErrorasyncioget_event_looprun_in_executorrIrJrr4r:loops r5r=zLocalSTCrossEncoder.predict!st ; STT T%''))  )            r7r>)r?r@rArBrJr__annotations__rLintrrDboolrVrCr6r9rErFrGrr=r2r7r5rIrIdsq  ,0I!D(///OS"&"' %;&=&=$J&=&= &=  &=  &=&=&=&=&=&=PsX`Z`Z`Z`ZDN4c3h#8NT%[NNNN: 4c3h#8 T%[      r7rIc eZdZUdZdZejdzed<eZ e ed<de eddfde d e d e d e d e d e f dZede fdZdejdejde de dejf dZddZdejdejde dee deee e ff dZdeee e fdee fdZdeee e fdee fdZdS)RemoteTEICrossEncodera Remote cross-encoder implementation using HuggingFace Text Embeddings Inference (TEI) HTTP API. TEI supports reranking via the /rerank endpoint. See: https://github.com/huggingface/text-embeddings-inference Note: The TEI server must be running a cross-encoder/reranker model. Requests are made in parallel with configurable batch size and max concurrency (backpressure). Uses a GLOBAL semaphore to limit concurrent requests across ALL recall operations. N_global_semaphore_global_max_concurrentg>@?base_urltimeoutrSrN max_retries retry_delayc,|d|_||_||_||_||_||_d|_d|_tj tj |kr,|t_ tj |t_ dSdS)a Initialize remote TEI cross-encoder client. Args: base_url: Base URL of the TEI server (e.g., "http://localhost:8080") timeout: Request timeout in seconds (default: 30.0) batch_size: Maximum batch size for rerank requests (default: 128) max_concurrent: Maximum concurrent requests for backpressure (default: 8). This is a GLOBAL limit across all parallel recall operations. max_retries: Maximum number of retries for failed requests (default: 3) retry_delay: Initial delay between retries in seconds, doubles each retry (default: 0.5) /N)rstriprrrSrNrr _async_client _model_idrrrr Semaphore)r4rrrSrNrrs r5rVzRemoteTEICrossEncoder.__init__Js*!,,  $,&&7;%) " 3 ;$;~MM;I ! 86=6G6W6W ! 3 3 3NMr7r0cdS)Nteir2r3s r5r6z#RemoteTEICrossEncoder.provider_namepur7client semaphoremethodurlcKd}|j}|4d{Vt|jdzD]t} |dkr|j|fi|d{V} n|j|fi|d{V} | | ccdddd{VS#t jt jt j f$rh} | }||jkrQt d|dzd|jdzd| d|d tj |d{V|d z}Yd} ~ d} ~ wt j$r{} | jjd kr_||jkrT| }t d |dzd|jdzd| d|d tj |d{V|d z}nYd} ~ nd} ~ wwxYw dddd{Vn#1d{VswxYwY|) zeMake an async HTTP request with automatic retries on transient errors and semaphore for backpressure.NrGETzTEI request failed (attempt rz): z. Retrying in zs...rizTEI server error (attempt )rrrgetpostraise_for_statushttpx ConnectError ReadTimeout WriteTimeoutrmrursleepHTTPStatusErrorresponse status_code) r4rrrrkwargs last_errordelayattemptrrs r5_async_request_with_retryz/RemoteTEICrossEncoder._async_request_with_retrytsi           !1A!566  )3C)B)B6)B)B#B#B#B#B#B#B)4S)C)CF)C)C#C#C#C#C#C#C--///#OO              *E,=u?QR###!"J!11177Q;77IY\]I]77bc77+0777&mE222222222 ,   z-444CS9S9S%& 7177tGWZ[G[77`a77+0777&mE222222222  #                            >sOF5AB 4F5 &F/AD F5F$A0FF5FF55 F?F?c TK|jdStd|jd|jd|jdt j|j|_tj d} | |j|d|jd d{V}| }| d d |_td |jddS#t j$r&}d|_t!d |jd|d}~wwxYw)z:Initialize the HTTP client and verify server connectivity.Nz'Reranker: initializing TEI provider at z (batch_size=z, max_concurrent=rjrrrz/infomodel_idunknownz+Reranker: TEI provider initialized (model: z#Failed to connect to TEI server at z: )rrmrnrrSrNr AsyncClientrrrrjsonrr HTTPErrorr)r4init_semaphorerrnrs r5r9z RemoteTEICrossEncoder.initializes~   ) F  Tdm T T? T T=A=P T T T   #.t|DDD!*1-- [!;;"NEdm;R;R;RH==??D!XXj)<.s%MMM6VG_fWo6MMMr7zTEI rerank request failed: )rrrrrr)r4rrrrrresultsrs r5_rerank_query_groupz)RemoteTEICrossEncoder._rerank_query_groups B!;;=)))""#( <        HmmooGMMWMMM M B B B@Q@@AA A BsA AA6A11A6r:cK|sgSi}t|D].\}\}}||vrg||<||||f/g}|D]\}}d|D}d|D} tdt | jD]>} || | jz} | | | jz} ||| | f?dgt |z} t jfd|D}tj |d{V}t||D]\\}}}}|D]\}}||}|| |<| S)zQAsync implementation of predict that runs requests in parallel with backpressure.cg|]\}}|Sr2r2ridx_s r5rz8RemoteTEICrossEncoder._predict_async..777vsAs777r7cg|]\}}|Sr2r2rrtexts r5rz8RemoteTEICrossEncoder._predict_async..777gaT777r7rrcRg|]#\}}}j||$Sr2)rr)rrrrr4rs r5rz8RemoteTEICrossEncoder._predict_async..sF   VeV[]^`eD $ $T%7E5 Q Q   r7N) rappenditemsrrrSrrrgatherzip)r4r: query_groupsrrr tasks_info indexed_textsindicesrr batch_indices batch_texts all_scorestasksrr result_scoresoriginal_idx_in_batchr global_idxrs` @r5_predict_asyncz$RemoteTEICrossEncoder._predict_asyncs I:< "+E"2"2 4 4 C%L((&( U#   & &T{ 3 3 3 3>@ $0$6$6$8$8 G G E=77777G77777E1c%jj$/:: G G 'A,?(? @ #ADO(;$;< !!5-"EFFFF GUSZZ' );      is    ......./2*g.F.F / / *OQ]0= / /,%u$%:; ). :&& /r7chK|jtd||d{VS)a Score query-document pairs using the remote TEI reranker. Requests are made in parallel with configurable backpressure. Args: pairs: List of (query, document) tuples to score Returns: List of relevance scores Nr)rrr r<s r5r=zRemoteTEICrossEncoder.predictsF   %STT T((/////////r7r>)r?r@rArBrrrrrrrrrDrGrVrCr6rrResponserr9rErFrr r=r2r7r5rr9s  37w(4/666"ECEEE 9A $X$X$X$X $X  $X  $X$X$X$X$XLsX+!+$+ +  + ++++Z[[[[2B!B$B B Cy B eCJ BBBB4($uS#X*?(DK((((T04c3h#80T%[000000r7rc teZdZdZ ddededededef d Zdd Zd e e eefd e efdZ d S)_CohereCompatibleRerankClientu Internal HTTP client for Cohere-compatible /rerank endpoints. Shared by all providers that speak the Cohere rerank wire format — {model, query, documents[, top_n]} request and {results: [{index, relevance_score}, ...]} response. This covers SiliconFlow, ZeroEntropy, Jina, Voyage, BGE self-hosted, and Cohere itself when reached via a custom base_url (e.g. Azure AI Foundry). Not a CrossEncoderModel — providers compose it and expose their own provider_name / initialization logging. N@Tapi_keyr rerank_urlr include_top_ncZ||_||_||_||_||_d|_dSr)rrrrrr)r4rrrrrs r5rVz&_CohereCompatibleRerankClient.__init__s6  $ *7;r7r0NcrK|jdStj|jd|jdd|_dS)NBearer application/json) Authorization Content-Typerheaders)rrrrrr3s r5r9z(_CohereCompatibleRerankClient.initialize-sS   ) F".L!94.Frr7cg|]\}}|Sr2r2rs r5rz9_CohereCompatibleRerankClient.predict..Grr7F)rr documentsreturn_documentstop_nrrrrelevance_score)rrr setdefaultrrrrrrrrrr)r4r:rrrrrrrrbodyrritem original_idxrs r5r=z%_CohereCompatibleRerankClient.predict8s   %STT T I9; "+E"2"2 C C C%  # #E2 . . 5 5sDk B B B BUSZZ' $0$6$6$8$8 : : E=77777E77777G"$) ''D ! + #E W !/44T_44PPPPPPPPH  % % ' ' ']]__F 9b11 : :#G} ./49 7<011 : r7)rTr>) r?r@rArBrDrGrrVr9rErFr=r2r7r5rrs  $" < < < < <  <  < < < <     #4c3h#8#T%[######r7rc eZdZdZeddfdedededzdefdZed efd Z dd Z d e e eefd e efd Z d e e eefd e efdZdS)CohereCrossEncoderz Cohere cross-encoder implementation using the Cohere Rerank API. Supports rerank-english-v3.0 and rerank-multilingual-v3.0 models. Nrrrrrc||_||_||_||_d|_|rt ||||dnd|_dS)aU Initialize Cohere cross-encoder client. Args: api_key: Cohere API key model: Cohere rerank model name (default: rerank-english-v3.0) base_url: Custom base URL for Cohere-compatible API (e.g., Azure-hosted endpoint) timeout: Request timeout in seconds (default: 60.0) NF)rrrrr)rrrr_clientr _http_clientr4rrrrs r5rVzCohereCrossEncoder.__init__esj         )##      r7r0cdS)Ncoherer2r3s r5r6z CohereCrossEncoder.provider_namexr7cK|j|jr|jjrdS|jr d|jnd}td|j||j;|jd{VtddS ddl}n#t$rtdwxYw| |j |j |_td dS) zInitialize the Cohere client.N at z2Reranker: initializing Cohere provider with model zGReranker: Cohere provider initialized (Cohere-compatible HTTP endpoint)rzNcohere is required for CohereCrossEncoder. Install it with: pip install cohere)rrz%Reranker: Cohere provider initialized) r+r,rrrmrnrr9r/rlClientrr)r4 base_url_msgr/s r5r9zCohereCrossEncoder.initializes# < #(9 #d>O>] # F15F-dm---B  ccUaccddd   (#..00 0 0 0 0 0 0 0 KKa b b b b b t  t t t!"rsss t"==t|=TTDL KK? @ @ @ @ @s BB8r:cK|j|jtd|sgS|j |j|d{VSt j}|d|j|d{VS)z Score query-document pairs using the Cohere Rerank API. Args: pairs: List of (query, document) tuples to score Returns: List of relevance scores Nr)r+r,rr=rrr_predict_sync_sdkrs r5r=zCohereCrossEncoder.predicts < D$5$=STT T I   (*22599999999 9%''))$0FNNNNNNNNNr7ci}t|D]3\}\}}||g||f4dgt|z}|D]c\}}d|D}d|D} |j|||jd} | jD]} | j } | j } | || | <d|S)z0Synchronous predict using the native Cohere SDK.rcg|]\}}|Sr2r2rs r5rz8CohereCrossEncoder._predict_sync_sdk..rr7cg|]\}}|Sr2r2rs r5rz8CohereCrossEncoder._predict_sync_sdk..rr7F)rr rr!) rr$rrrr+rerankrrrr#)r4r:rrrrrrrrrrr'rs r5r7z$CohereCrossEncoder._predict_sync_sdks9; "+E"2"2 C C C%  # #E2 . . 5 5sDk B B B BUSZZ' $0$6$6$8$8 : : E=77777E77777G|**j!& +H#* : :%| .49 7<011 : r7r>)r?r@rArBrrDrGrVrCr6r9rErFr=r7r2r7r5r)r)^s3# " " " " * "  " " " " HsXAAAA*O4c3h#8OT%[OOOO.tE#s(O'<er7r)c eZdZdZdZdZeddfdedededzd efd Z e d efd Z dd Z de eeefd e efdZdS)ZeroEntropyCrossEncoderz ZeroEntropy cross-encoder implementation using the ZeroEntropy Rerank API. Supports zerank-2 (flagship) and zerank-2-small models. See: https://docs.zeroentropy.dev/models zhttps://api.zeroentropy.devz/v1/models/rerankNrrrrrc||_|r|dn|j|_t |||j|j||_dSNr)rrrr)rrDEFAULT_BASE_URLrr RERANK_PATHr+r-s r5rVz ZeroEntropyCrossEncoder.__init__sa 08S,,,d>S 4-;)9;;     r7r0cdS)N zeroentropyr2r3s r5r6z%ZeroEntropyCrossEncoder.provider_name}r7cK|jjdStd|j|jd{VtddS)Nz7Reranker: initializing ZeroEntropy provider with model z*Reranker: ZeroEntropy provider initialized)r+rrmrnrr9r3s r5r9z"ZeroEntropyCrossEncoder.initializesr < % 1 F ZdjZZ[[[l%%''''''''' @AAAAAr7r:cFK|j|d{VSrr+r=r<s r5r=zZeroEntropyCrossEncoder.predict.\))%000000000r7r>)r?r@rArBr@rArrDrGrVrCr6r9rErFr=r2r7r5r=r=s5%K 8#     *       sXBBBB14c3h#81T%[111111r7r=c eZdZdZdZeedfdedededefdZ e d efd Z dd Z d e eeefd e efdZd S)SiliconFlowCrossEncoderz SiliconFlow cross-encoder implementation. SiliconFlow (https://siliconflow.cn) exposes a Cohere-compatible /rerank endpoint. Shares the HTTP client with ZeroEntropy/Cohere-custom-endpoint via _CohereCompatibleRerankClient. rrrrrrc||_|d|_t|||j|j||_dSr?)rrrrrAr+r-s r5rVz SiliconFlowCrossEncoder.__init__ sT  ,, 4-;)9;;     r7r0cdS)N siliconflowr2r3s r5r6z%SiliconFlowCrossEncoder.provider_namerDr7NcK|jjdStd|jd|j|jd{VtddS)Nz/Reranker: initializing SiliconFlow provider at with model z*Reranker: SiliconFlow provider initialized)r+rrmrnrrr9r3s r5r9z"SiliconFlowCrossEncoder.initializes~ < % 1 F mdmmmaeakmmnnnl%%''''''''' @AAAAAr7r:cFK|j|d{VSrrGr<s r5r=zSiliconFlowCrossEncoder.predict$rHr7r>)r?r@rArBrArrrDrGrVrCr6r9rErFr=r2r7r5rJrJsK 8=            sXBBBB14c3h#81T%[111111r7rJcteZdZdZdZedefdZd dZde e eefde e fdZ dS) RRFPassthroughCrossEncodera Passthrough cross-encoder that preserves RRF scores without neural reranking. This is useful for: - Testing retrieval quality without reranking overhead - Deployments where reranking latency is unacceptable - Debugging to isolate retrieval vs reranking issues cdS)z)Initialize RRF passthrough cross-encoder.Nr2r3s r5rVz#RRFPassthroughCrossEncoder.__init__2s r7r0cdS)Nrrfr2r3s r5r6z(RRFPassthroughCrossEncoder.provider_name6rr7Nc>KtddS)zNo initialization needed.zJReranker: RRF passthrough provider initialized (neural reranking disabled)N)rmrnr3s r5r9z%RRFPassthroughCrossEncoder.initialize:s `aaaaar7r:c,Kdgt|zS)z Return neutral scores - actual ranking uses RRF scores from retrieval. Args: pairs: List of (query, document) tuples (ignored) Returns: List of 0.5 scores (neutral, lets RRF scores dominate) rrr<s r5r=z"RRFPassthroughCrossEncoder.predict>sus5zz!!r7r>) r?r@rArBrVrCrDr6r9rErFrGr=r2r7r5rRrR(s   sXbbbb "4c3h#8 "T%[ " " " " " "r7rRc eZdZUdZdZedzed<dZeed< dde dzd e dzd ed ed e f d Z e de fdZ ddZdeee e fdeefdZdeee e fdeefdZdS)FlashRankCrossEncodera FlashRank cross-encoder implementation. FlashRank is an ultra-lite reranking library that runs on CPU without requiring PyTorch or Transformers. It's ideal for serverless deployments with minimal cold-start overhead. Available models: - ms-marco-TinyBERT-L-2-v2: Fastest, ~4MB - ms-marco-MiniLM-L-12-v2: Best quality, ~34MB (default) - rank-T5-flan: Best zero-shot, ~110MB - ms-marco-MultiBERT-L-12: Multi-lingual, ~150MB NrJrKrLFrM cache_dir max_lengthrN cpu_mem_arenac|pt|_|pt|_||_||_d|_|t_dS)a Initialize FlashRank cross-encoder. Args: model_name: FlashRank model name. Default: ms-marco-MiniLM-L-12-v2 cache_dir: Directory to cache downloaded models. Default: system cache max_length: Maximum sequence length for reranking. Default: 512 max_concurrent: Maximum concurrent reranking calls. Default: 4 cpu_mem_arena: Enable ONNX Runtime CPU memory arena. Default: False. When True, ONNX pre-allocates a memory arena that never shrinks, causing RSS to grow monotonically. False trades slightly slower per-call allocation for bounded RSS. N) r rMr r[r\r]_rankerrYrL)r4rMr[r\rNr]s r5rVzFlashRankCrossEncoder.__init___sC*%H(H"J&J$* 0>---r7r0cdS)N flashrankr2r3s r5r6z#FlashRankCrossEncoder.provider_name{s{r7cK|jdS ddlm}n#t$rtdwxYwtd|jd|jd|js ddl}| }d|_ nd}|j|j d }|j r |j |d <|di||_|t|jd rddl}d}t|jd d}|r6dd lm}||dD]}t%|}|r;||||j_tdt*jOt/t*jdt*_tdt*jddStddS)zLoad the FlashRank model.Nr)RankerzWflashrank is required for FlashRankCrossEncoder. Install it with: pip install flashrankz5Reranker: initializing FlashRank provider with model z (cpu_mem_arena=rjF)rMr\r[session model_dir)Pathz*.onnx) sess_optionszBReranker: replaced FlashRank ONNX session with cpu_mem_arena=Falserargz9Reranker: FlashRank provider initialized (max_concurrent=zBReranker: FlashRank provider initialized (using existing executor)r2)r_rarcrlrmrnrMr] onnxruntimeSessionOptionsenable_cpu_mem_arenar\r[rrgetattrpathlibrfglobrDInferenceSessionrdrYrJrrL) r4rcortsession_options ranker_kwargs model_filererf candidates r5r9z FlashRankCrossEncoder.initializesU < # F y ( ( ( ( ( ( ( y y ywxx x y   5DO 5 5#1 5 5 5   ! # % % % %!0022O38O 0 0"O.2_DO\\ > 8)-M+ &v.. ..  &74<+K+K & % % % %J k4@@I ((((((!%i!5!5h!?!?I!$YJ b'*';';JUd';'e'e $ `aaa ! * 2.@1A#./// ! + KKtLaLqttt      KK\ ] ] ] ] ]s.r:cddlm}|sgSi}t|D].\}\}}||vrg||<||||f/dgt |z}|D]s\}}dt|D} d|D} ||| } |j| } | D]} | d}| d}| |}|||< t|S) z1Synchronous predict - processes each query group.r) RerankRequestrc$g|] \}\}}||dS))idrr2)rrrrs r5rz7FlashRankCrossEncoder._predict_sync..s(]]]LAy4q$//]]]r7cg|]\}}|Sr2r2rs r5rz7FlashRankCrossEncoder._predict_sync..s>>>fc1c>>>r7)rpassagesrwr)rarurrrrr_r;)r4r:rurrrrrrryglobal_indicesrequestrr local_idxrr s r5rz#FlashRankCrossEncoder._predict_syncsO++++++ I:< "+E"2"2 4 4 C%L((&( U#   & &T{ 3 3 3 3USZZ' $0$6$6$8$8 / / E=]]ImD\D\]]]H>> >>>N$m%(CCCGl))'22G" / /"4L w+I6 ). :&&  / r7cK|jtdtj}|t j|j|d{VS)z Score query-document pairs using FlashRank. Args: pairs: List of (query, document) tuples to score Returns: List of relevance scores (higher = more relevant) Nr)r_rrrrrYrJrrs r5r=zFlashRankCrossEncoder.predictsa < STT T%''))*?*I4K]_deeeeeeeeer7)NNrZrKFr>)r?r@rArBrJrrrLrrDrrVrCr6r9rErFrGrr=r2r7r5rYrYLsY  ,0I!D(///OS"& $# ??$J?:? ?  ?  ????8sX=^=^=^=^~ 4c3h#8 T%[    Df4c3h#8fT%[ffffffr7rYr max_tokensr0cddlm}|}||}t||kr|S||d|S)zFTruncate text to at most max_tokens using the shared tiktoken encoder.r)_get_tiktoken_encodingN) memory_enginerencoderdecode)rr~renctokenss r5_truncate_to_tokensrsd555555 " "C ZZ  F 6{{j   ::f[j[) * **r7c eZdZdZededefdededzdedede dzf d Z e d efd Z dd Z d eeeefd eefdZdS)LiteLLMCrossEncodera+ LiteLLM cross-encoder implementation using LiteLLM proxy's /rerank endpoint. LiteLLM provides a unified interface for multiple reranking providers via the Cohere-compatible /rerank endpoint. See: https://docs.litellm.ai/docs/rerank Supported providers via LiteLLM: - Cohere (rerank-english-v3.0, etc.) - prefix with cohere/ - Together AI - prefix with together_ai/ - Azure AI - prefix with azure_ai/ - Jina AI - prefix with jina_ai/ - AWS Bedrock - prefix with bedrock/ - Voyage AI - prefix with voyage/ Nrapi_baserrrmax_tokens_per_docc|d|_||_||_||_||_d|_dS)a Initialize LiteLLM cross-encoder client. Args: api_base: Base URL of the LiteLLM proxy (default: http://localhost:4000) api_key: API key for the LiteLLM proxy (optional, depends on proxy config) model: Reranking model name (default: cohere/rerank-english-v3.0) Use provider prefix (e.g., cohere/, together_ai/, voyage/) timeout: Request timeout in seconds (default: 60.0) max_tokens_per_doc: If set, truncate each document to this many tokens before sending to the reranker (uses tiktoken cl100k_base encoding). Useful for models with small context windows (e.g. 1024 tokens). rN)rrrrrrr)r4rrrrrs r5rVzLiteLLMCrossEncoder.__init__sB*!,,    "47;r7r0cdS)Nlitellmr2r3s r5r6z!LiteLLMCrossEncoder.provider_name*syr7cK|jdStd|jd|jddi}|jr d|j|d<t j|j||_td dS) z!Initialize the async HTTP client.Nz+Reranker: initializing LiteLLM provider at rOrrrrrz&Reranker: LiteLLM provider initialized) rrmrnrrrrrr)r4rs r5r9zLiteLLMCrossEncoder.initialize.s   ) F i$-ii]a]giijjj!#56 < @'?'?'?GO $".t|WUUU <=====r7r:c Kjtd|sgSi}t|D].\}\}}||vrg||<||||f/dgt |z}|D]\}}d|D}jfd|D}d|D} jjdj ||t |d d{V} | | } | d gD]?} | d } | d | d d}||| | <@|S)z Score query-document pairs using the LiteLLM proxy's /rerank endpoint. Args: pairs: List of (query, document) tuples to score Returns: List of relevance scores Nrrcg|]\}}|Sr2r2rs r5rz/LiteLLMCrossEncoder.predict..Vrr7c:g|]}t|jSr2rrrtr4s r5rz/LiteLLMCrossEncoder.predict..X'XXXQ,Q0GHHXXXr7cg|]\}}|Sr2r2rs r5rz/LiteLLMCrossEncoder.predict..Yrr7r)rrr r"rrrr#r) rrrrrrrrrrrrr)r4r:rrrrrrrrrrr&r'rs` r5r=zLiteLLMCrossEncoder.predict<s   %STT T I:< "+E"2"2 4 4 C%L((&( U#   & &T{ 3 3 3 3USZZ' $0$6$6$8$8 : : E=77777E&2XXXXRWXXX77777G"/44=)))!Z"!& ZZ 5H  % % ' ' ']]__F 9b11 : :#G} !2DHHWc4J4JKK49 7<011 : r7r>)r?r@rArBrrr rDrGrrVrCr6r9rErFr=r2r7r5rrs$1"3)T <<<t< <  <  $J <<<<8sX > > > >34c3h#83T%[333333r7rc eZdZdZeddefdedededzdededzf d Z e d efd Z dd Z d e eeefd e efdZdS)LiteLLMSDKCrossEncodera LiteLLM SDK cross-encoder for direct API integration. Supports reranking via LiteLLM SDK without requiring a proxy server. Supported providers: Cohere, DeepInfra, Together AI, HuggingFace, Jina AI, Voyage AI, AWS Bedrock. Example model names: - cohere/rerank-english-v3.0 - deepinfra/Qwen3-reranker-8B - together_ai/Salesforce/Llama-Rank-V1 - huggingface/BAAI/bge-reranker-v2-m3 Nrrrrrrch||_||_||_||_||_d|_d|_dS)ap Initialize LiteLLM SDK cross-encoder client. Args: api_key: API key for the reranking provider model: Model name with provider prefix (e.g., "deepinfra/Qwen3-reranker-8B") api_base: Custom base URL for API (optional) timeout: Request timeout in seconds (default: 60.0) max_tokens_per_doc: If set, truncate each document to this many tokens before sending to the reranker (uses tiktoken cl100k_base encoding). Useful for models with small context windows (e.g. 1024 tokens). FN)rrrrr _initialized_litellm)r4rrrrrs r5rVzLiteLLMSDKCrossEncoder.__init__s;(     "4! r7r0cdS)N litellm-sdkr2r3s r5r6z$LiteLLMSDKCrossEncoder.provider_namerDr7c K|jrdS ddl}||_n#t$rtdwxYw|jr d|jnd}t d|j|d|_t ddS) z"Initialize the LiteLLM SDK client.NrzTlitellm is required for LiteLLMSDKCrossEncoder. Install it with: pip install litellmr2r3z7Reranker: initializing LiteLLM SDK provider with model Tz*Reranker: LiteLLM SDK provider initialized)rrrrlrrmrnr)r4r api_base_msgs r5r9z!LiteLLMSDKCrossEncoder.initializes    F v NNN#DMM v v vtuu u v26F-dm---B  hdjhZfhhiii  @AAAAAs 3r:c lKjstd|sgSi}t|D].\}\}}||vrg||<||||f/dgt |z}|D]-\}}d|D}jfd|D}d|D} j||jd} j r j | d<j j di| d{V} t| d rO| j rH| j D]?} | d } | d | d d}||| | <@t| t r"t| D]\}}||| |<t"d t'| /|S)z Score query-document pairs using the LiteLLM SDK. Args: pairs: List of (query, document) tuples to score Returns: List of relevance scores rrcg|]\}}|Sr2r2rs r5rz2LiteLLMSDKCrossEncoder.predict..rr7Nc:g|]}t|jSr2rrs r5rz2LiteLLMSDKCrossEncoder.predict..rr7cg|]\}}|Sr2r2rs r5rz2LiteLLMSDKCrossEncoder.predict..rr7)rrr rrrrr#rz0Unexpected response format from LiteLLM rerank: r2)rrrrrrrrrrrarerankrrrr isinstancerErmrutype)r4r:rrrrrrrr rerank_kwargsrrr'rrs` r5r=zLiteLLMSDKCrossEncoder.predictsV  USTT T I:< "+E"2"2 4 4 C%L((&( U#   & &T{ 3 3 3 3USZZ' $0$6$6$8$8 d d E=77777E&2XXXXRWXXX77777G"< M } :,0M j)2T]2CC]CCCCCCCCH x++ d0@ d&.>>F#)'?L"JJ'8&**Wc:R:RSSE8=Jw|455 > Hd++ d )( 3 333HAu-2Jwqz**3bRVW_R`R`bbccccr7r>)r?r@rArBrr rDrGrrVrCr6r9rErFr=r2r7r5rrrs   8#)T *     $J 8sXBBBB$<4c3h#8<T%[<<<<<<r7rceZdZdZdZd dedzfdZedefdZddZ dd Z d e e eefde e fd Zd e e eefde e fd ZdS)JinaMLXCrossEncoderu Jina Reranker v3 MLX implementation for Apple Silicon. Uses jinaai/jina-reranker-v3-mlx — a 0.6B parameter multilingual listwise reranker optimized for Apple Silicon via the MLX framework. No transformers/PyTorch dependency. The model is downloaded automatically from HuggingFace Hub on first use. Requires: mlx>=0.31.0, mlx-lm>=0.31.1, safetensors>=0.6.2 zjinaai/jina-reranker-v3-mlxN model_pathc"||_d|_dS)z Args: model_path: Local path to the downloaded model directory. If None, the model is downloaded from HuggingFace Hub. N)r _reranker)r4rs r5rVzJinaMLXCrossEncoder.__init__s %r7r0cdS)Njina-mlxr2r3s r5r6z!JinaMLXCrossEncoder.provider_nameszr7c"K|jdSddl}|j} ddl}ddl}n:#t $r-}t |}d|vrd|vrt d|d}~wwxYwtj}| d|j d{VdS)Nrmlxmlx_lmz|mlx and mlx-lm are required for JinaMLXCrossEncoder. Install with: pip install mlx>=0.31.0 mlx-lm>=0.31.1 safetensors>=0.6.2) rrb AutoTokenizermlx.corerrlrDrrr _load_model)r4rbrrrexcmsgrs r5r9zJinaMLXCrossEncoder.initialize s > % F   &  OOO MMMM    c((CCHC$7$7Z  %''""4)9:::::::::::s! A(AAcddl}ddl}ddlm}ddlm}|j}|4td|j d||j }td ||||j |d  |_ | |_td dS) zADownload (if needed) and load the MLX reranker. Runs in a thread.rN)snapshot_downloadr) MLXRerankerzReranker: downloading z from HuggingFace Hub...)repo_idz,Reranker: loading jina-reranker-v3-mlx from zprojector.safetensors)rprojector_pathz'Reranker: jina-mlx provider initialized)os threadinghuggingface_hubrjina_mlx_rerankerrrrmrn HF_REPO_IDpathjoinrLock _mlx_lock)r4rrrrrs r5rzJinaMLXCrossEncoder._load_model,s 555555222222_   KKZZZZ [ [ [**4?CCCJ O:OOPPP$!7<< 4KLL   #)) =>>>>>r7r:c|sgSi}t|D]3\}\}}||g||f4dgt|z}|j5|D]V\}}d|D}d|D} |j||} | D]} | d} | d|| | <W dddn #1swxYwY|S)z/Score pairs grouped by query. Runs in a thread.rcg|]\}}|Sr2r2)rrdocs r5rz5JinaMLXCrossEncoder._predict_sync..Rs7773777r7cg|]\}}|Sr2r2rs r5rz5JinaMLXCrossEncoder._predict_sync..Ss:::633:::r7rr#N)rr$rrrrrr;) r4r:rrrrr indexed_docsdocsrrrr's r5rz!JinaMLXCrossEncoder._predict_syncEs I9; !*5!1!1 B B C%  # #E2 . . 5 5sCj A A A AUSZZ' ^ R R'3'9'9';'; R R#|77,777::\:::.//t<<%RRF#)'?L8>?P8QJw|455R  R R R R R R R R R R R R R R R Rs$A,CC!$C!cK|jtdtj}|d|j|d{VS)Nr)rrrrrrrs r5r=zJinaMLXCrossEncoder.predict[sX > !STT T%''))$0BEJJJJJJJJJr7rr>)r?r@rArBrrDrVrCr6r9rrErFrGrr=r2r7r5rrs/J3:sX;;;;B????24c3h#8T%[,K4c3h#8KT%[KKKKKKr7rc eZdZdZdZdZdgZedddfded ed edzd ed e f d Z e defdZ de eeffdZddZdeeeefdee fdZdeeeefdee fdZdS)GoogleCrossEncodera Google Discovery Engine cross-encoder using the Ranking REST API. Uses httpx + google-auth for lightweight REST calls (no gRPC/protobuf). Supports ADC (Application Default Credentials) or service account key file. Available models: - semantic-ranker-default-004: Best quality, 1024 tokens/record (recommended) - semantic-ranker-fast-004: Lower latency, 1024 tokens/record Max 200 records per API request. Location is always "global". z)https://discoveryengine.googleapis.com/v1z.https://www.googleapis.com/auth/cloud-platformNglobalr project_idrservice_account_keylocationrcv||_||_||_||_||_d|_d|_d|_dS)a Initialize Google Discovery Engine cross-encoder. Args: project_id: Google Cloud project ID model: Ranking model name (default: semantic-ranker-default-004) service_account_key: Path to service account JSON key file. If None, uses Application Default Credentials (ADC). location: API location (default: "global") timeout: Request timeout in seconds (default: 60.0) N)rrrrr _credentialsr+ _rank_url)r4rrrrrs r5rVzGoogleCrossEncoder.__init__usB&% #6     ,0 %)r7r0cdS)Ngoogler2r3s r5r6z GoogleCrossEncoder.provider_namer0r7cddl}|jjs;|j|jjjdd|jjiS)z3Get Authorization header with a fresh access token.rNrr) google.auth.transport.requestsrvalidrefreshauth transportrequestsRequesttoken)r4rs r5_get_auth_headersz$GoogleCrossEncoder._get_auth_headerssb---- & P   % %fk&;&D&L&L&N&N O O O!D4+<+B!D!DEEr7c K|jdS|jsdnd}td|jd|jd|d|jrQ dd lm}n#t$rtd wxYw|j |j|j |_ nK ddl }n#t$rtd wxYw|j|j \|_ }d |jd |jd}|jd|d|_t'j|j|_tddS)z'Initialize credentials and HTTP client.NADCservice_accountzAReranker: initializing Google Discovery Engine provider (project=z, model=z, auth=rjr)rzXgoogle-auth is required for GoogleCrossEncoder. Install it with: pip install google-auth)scopesz projects/z /locations/z&/rankingConfigs/default_ranking_configrz:rankrz6Reranker: Google Discovery Engine provider initialized)r+rrmrnrr google.oauth2rrl Credentialsfrom_service_account_fileSCOPESr google.authrdefaultrAPI_BASErrr4r)r4 auth_methodrrrranking_configs r5r9zGoogleCrossEncoder.initializes < # F#'#;ReeAR   S S S15 S SDO S S S     # K 9999999   !n !0 ; U U({!V!!D    """""   !n $*;#6#6dk#6#J#J D qvT_vvvvv MAANAAA|DL999  LMMMMMs AA.B##B=r:c F|sgSi}t|D].\}\}}||vrg||<||||f/dgt|z}|D]3\}}d|D}d|D} t dt||jD]} || | |jz} | | | |jz} dt| D} |j|j| |j || t| d}| | }| dgD](}t|d }|d || |<)5|S) z!Synchronous predict via REST API.rcg|]\}}|Sr2r2rs r5rz4GoogleCrossEncoder._predict_sync..rr7cg|]\}}|Sr2r2rs r5rz4GoogleCrossEncoder._predict_sync..rr7rc8g|]\}}t||dS))rwcontent)rD)rrrs r5rz4GoogleCrossEncoder._predict_sync..s*aaawq$#a&&T::aaar7)rrrecordstopN)rrrrwr)rrrrrMAX_RECORDS_PER_REQUESTr+rrrrrrrr)r4r:rrrrrrrr batch_startrrrrrrecordr|s r5rz GoogleCrossEncoder._predict_syncs I:< "+E"2"2 4 4 C%L((&( U#   & &T{ 3 3 3 3USZZ' $0$6$6$8$8 K K E=77777E77777G %QE D4PQQ K K #K+@\2\$\] ' kDD`6`(` a aa)T_J`J`aaa<,,N 2244!%!&#* #G -  ))+++!$jjB77KKF #F4L 1 1I;A'?J}Y788K' K.r7cK|jtd|sgStj}|d|j|d{VS)z Score query-document pairs using Google Discovery Engine Ranking API. Args: pairs: List of (query, document) tuples to score Returns: List of relevance scores (0-1, higher = more relevant) Nr)r+rrrrrrs r5r=zGoogleCrossEncoder.predictsg < STT T I%''))$0BEJJJJJJJJJr7r>)r?r@rArBrrrr rDrGrVrCr6dictrr9rErFrr=r2r7r5rrcse  ":H> ?F 3*.  ****!4Z *  *  ****8sXF4S>FFFF"N"N"N"NH*4c3h#8*T%[****XK4c3h#8KT%[KKKKKKr7rc ddlm}|}|j}|dkrJ|j}|st t dtdt||j |j |j S|dkr9t|j |j|j|j|j|j|jS|d krD|j}|st t*dtd t-||j|j S|d kr8|j}|st d tdt-||jd S|dkrt6jt<t>}t6jt@tB}t6jtDtGtHdv}tK|||S|dkr'tM|j'|j(|j)|j*S|dkrJ|j+}|st tXdtdt[||j.|j/|j*S|dkr>|j0}|st tbdtdte||j3S|dkrD|j4}|st tjdtdtm||j7|j8 S|dkrD|j9}|st ttdtdtw||j<|j= S|d!krt}S|d"krtSt d#|d$)%z Create a CrossEncoderModel instance based on configuration. Reads configuration via get_config() to ensure consistency across the codebase. Returns: Configured CrossEncoderModel instance r) get_configrz is required when z is 'tei')rrrSrNrX)rMrNrOrPrQrRrSr/z is 'cohere')rrr openrouterz{HINDSIGHT_API_RERANKER_OPENROUTER_API_KEY, HINDSIGHT_API_OPENROUTER_API_KEY, or HINDSIGHT_API_LLM_API_KEY is required when z is 'openrouter'z#https://openrouter.ai/api/v1/rerankra)true1yes)rMr[r]r)rrrrrz is 'litellm-sdk')rrrrrCz is 'zeroentropy')rrrMz is 'siliconflow'rz= (or HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID) is required when z is 'google')rrrrUrzUnknown reranker provider: z. Supported: 'local', 'tei', 'cohere', 'zeroentropy', 'siliconflow', 'google', 'flashrank', 'litellm', 'litellm-sdk', 'rrf', 'jina-mlx')@configrreranker_providerlowerreranker_tei_url ValueErrorr,r'rreranker_tei_http_timeoutreranker_tei_batch_sizereranker_tei_max_concurrentrIreranker_local_modelreranker_local_max_concurrentreranker_local_force_cpu reranker_local_trust_remote_codereranker_local_fp16reranker_local_bucket_batchingreranker_local_batch_sizereranker_cohere_api_keyrr)reranker_cohere_modelreranker_cohere_base_urlreranker_openrouter_api_keyreranker_openrouter_modelrenvironrr r rr rrDr rYrreranker_litellm_api_basereranker_litellm_api_keyreranker_litellm_model#reranker_litellm_max_tokens_per_docreranker_litellm_sdk_api_keyr"rreranker_litellm_sdk_modelreranker_litellm_sdk_api_basereranker_zeroentropy_api_keyr-r=reranker_zeroentropy_modelreranker_siliconflow_api_keyr(rJreranker_siliconflow_modelreranker_siliconflow_base_urlreranker_google_project_idr!rreranker_google_model#reranker_google_service_account_keyrRr) rrproviderrrrr[r]rs r5create_cross_encoder_from_envr&s$##### Z\\F'--//H5% j 4hhH]hhhii i$45!=     W  "2!?5$E+"A7     X  0 t ;rrOdrrrss s!.4    \ ! !4 iAViii "2:    [  ;=]^^JNN#CEijj   0#6^2_2_  %'')* %Zghhhh Y  "53/%I     ] " "5 3ooG\ooo &39%I     ] " "5 3ooG\ooo '3     ] " "5 3ooG\ooo '39    X  6  1HH$9HHH "!. & J    U  )+++ Z  "$$$ |( | | |   r7)GrBrrrr{abcrrconcurrent.futuresrrrrrr r r r r rrrrrrrrrrrrrrrrrrr r!r"r#r$r%r&r'r(r)r*r+r,r-rr?rmr/rIrrr)r=rJrRrYrDrrrrrrr&r2r7r5r)s ########111111 ((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((T  8 $ $" " " " " " " " JR R R R R +R R R jT0T0T0T0T0-T0T0T0nKKKKKKKK\sssss*sssl'1'1'1'1'1/'1'1'1T'1'1'1'1'1/'1'1'1T!"!"!"!"!"!2!"!"!"Hcfcfcfcfcf-cfcfcfL+c+s+s++++rrrrr+rrrj|||||.|||~oKoKoKoKoK+oKoKoKd[K[K[K[K[K*[K[K[K|y '8y y y y y y r7