Cj"%SrSSKJr SSKrSSKrSSKrSSKrSSKrSSKrSSK r SSK r SSK r SSK r SSK r SSKrSSKrSSKrSSKrSSKrSSKrSSKrSSKrSSKrSSKrSSKrSSKrSSKJrJr SSKJrJr SSK J!r! SSK J"r" SSK J#r# SS K$J%r%J&r&J'r' SSK(r(SSK)r)SSK*r*SSK+r+SS K)J,r, SS K-J.r. SS K/J0r0J1r1 SS K2J3r3 SSK4J5r5 SSK+J6r6 SSK7J8r8 SSK9r9SSK:r9SSK;r9SSKJ?r?J@r@JArA SSKBJCrC SSKDJErEJFrFJGrGJHrHJIrIJJrJJKrKJLrLJMrMJNrNJOrOJPrP SqQSqRSqSSqTSrUSrVSrWSrXSqYSrZSSK[J\r\J]r]J^r^J_r_J`r`JaraJbrbJcrc SSKdJere SSKdJfrfJgrgJhrhJiriJjrjJkrkJlrlJmrmJnrnJoroJprpJqrqJrrrJsrsJtrtJuruJvrvJwrwJxrxJyryJzrzJ{r{J|r|J}r}J~r~JrJrJrJrJrJrJrJr \gVs0sHoGRU_M snrS\S'\!GR"S5GRS 5GRS!S"S#9r\GR"\5rSSS5 \GR("\5rSSKr SS$KJrJrJrJrJrJrJrJrJrJrJrJrJrJr SS%KJrJr \(GaSS&KJrJr SS'KJr SS(KJr SS)KJr SS*KJr SS+KJr SS,KJrJr SS-KJr SS.KJrJrJrJrJr SS/KJr SS0KJr SS1KJr SS2KJrJr SS3KJr SS4KJr SS5KJr SS6KJr SS7KJr SS8KJr SS9KJr SS:KJr SS;KJr SSKJrJr SS?KJrJr SS@KJr SSAKJr SSBKJr SSCKJr SSDKJr SSEKJr SSFKJr SSGKJr SSHKJrJr SSIKJrJr SSJKGJGr SSKGKGJGr SSLGKGJGr SSMGKGJGr SSNGKGJGr SSOGK GJ Gr SSPGK GJ Gr SSQGK GJGr SSRGKGJGr SSSGKGJGr SSTGKGJGr SSUGKGJGr SSVGKGJGrGJGr SSWGKGJGr SSXK[GJGrGJGrGJGr SSYGKGJ Gr SSZGK!GJ"Gr" SS[GK#GJ$Gr$ SS\GK%GJ&Gr& SS]GK'GJ(Gr( SS^GK)GJ*Gr* SS_GK+GJ,Gr, S`SaGK-GJ.Gr.GJ/Gr/ S`SbGK0GJ1Gr1GJ2Gr2GJ3Gr3GJ4Gr4GJ5Gr5GJ6Gr6 S`ScGK7GJ8Gr8GJ9Gr9GJ:Gr:GJ;Gr;GJGr>GJ?Gr?GJ@Gr@GJAGrAGJBGrBGJCGrCGJDGrDGJEGrEGJFGrF \(aSSdK9GJGGrG SGrHSGrISGqJSGrKSGrLSGrMSGrNSGrOSGrPSGrQSGrRSGrSSGrTSGrUSGrVSGrWSGrXSGrYSGrZSGr[SGr\SGr]SGr^SGr_/Gq`Se\Sf'SGqa0GrbSg\Sh'0GrcSg\Si'SGrdSGreGS<GS=SkjjGrfSlGrgGS>SmjGrhGS?SnjGriGS?SojGrjGS@SpjGrkGS@SqjGrlGSASrjGrmGSBSsjGrnGSCStjGroGSDSujGrpGSESvjGrqGSESwjGrrGSFSxjGrsGSGSyjGrtGSHSzjGruGSIS{jGrvGSJS|jGrwGSKS}jGrxGSLS~jGryGSMSjGrzGSNSjGr{SGr|GSOGSPSjjGr}G\~"\gGR\gGR\gGRGR\gGRGR15GrGSQSjGrGSRGSSSjjGr\"\GS9GSTSj5GrGSTSjGrGSUSjGrGSVGSWSjjGrS/SS4GSXSjjGrGSYSjGrGSZGS[SjjGrGS\SjGrGS]GS^SjjGrGS_SjGrGS`SjGrGSRGS`SjjGrGSRGS`SjjGrGS`SjGrGSRGS`SjjGrGSRGS`SjjGrGSRGS`SjjGrGSRGS`SjjGrGSaSjGrGSbSjGrGSbSjGrGSRGS`SjjGrGSRGS`SjjGrGSRGS`SjjGrGSRGS`SjjGrGSRGS`SjjGrGSRGS`SjjGrGSRGS`SjjGrGSRGS`SjjGrGSRGScSjjGrGSRGS`SjjGrGSdSjGrSGrSGrSGrGSeSjGrGSfSjGrGSKSjGrGSgSjGrGShGSiSjjGrGSjSjGrGSkGSlSjjGrGSmGSnSjjGrSGrSGrGSoSjGrGSpSjGrGSqSjGr"SS5GrGSrGSsSjjGrGStSjGrGSuSjGrGSvGSwSjjGrGSRGSxSjjGrGSySjGrGSzSjGrGS{SjGrGS|SjGrGSRGS}SjjGrGSRGS~SjjGrGSSjGrGSSjGrGSSjGrGSSjGrGSSjGrGSSjGrGSSjGrGSSjGrGSSjGrGSSjGrSGrGSSjGrGSSjGrGSSjGrGSSjGrGSSjGrSGqSg\S'SGqS\S'GSSjGrGSSjGrGSSjGrGSSjGrGSSjGrGSSjGrGSSjGrGSSjGrSSGKGJGr "SSG\5GrGSSjGrGSSjGr\"\GS9GSRGSSjj5GrGSSjGrGSSjGrGSGSSjjGrGSGSSjjGr\"\GS9GSGSSjj5GrGSGSSjjGrG\GRG\GlG\GRG\GlGSSjGrGSSjGrSGrGSGSSjjGrGSSjGrSGrGSGSSjjGrSGrSGrSGrGSSjGrGSSjGrGSRGSSjjGrGSGSSjjGr0SS04GSSjjGr"SS5GrGSRGSSjjGrGSRGSSjjGrGSSjGrGSGrGSGrGSGrGSGrGSGrGSOGSGSjjGrS\HSjS4GSGSjjGrSGSGKGJGr "GSGS G\5GrG\"5GrGSRGSGS jjGrGSRGSGS jjGrGSGSGS jjGrGS Gr GSGr GSRGSjGr "GSGS5Gr "GSGS5Gr "GSGS\5GrGSGSjGrGSGSjGrGSGSjGrGSGSjGrGSGSjGrGSGSjGrGSGSjGrSGSK[GJGrGJGrGJGrGJGrGJGrGJGr GSGSjGrGSGSjGrGSGS jGrGSGS!jGrGSGS"jGr GSGS#jGr!GSGS$jGr"GSGS%jGr#GSRGSGS&jjGr$\"S`S9GSGS'j5Gr%GSGS(jGr&"GS)GS*5Gr'GSGSGS+jjGr(GSGS,jGr)GSGSGS-jjGr*GSGS.jGr+GSGS/jGr,GSGS0jGr-GSqGS1jGr.GSGS2jGr/GSGS3jGr0GSqGS4jGr1GSGS5jGr2GSGS6jGr3GSGS7jGr4GSGS8jGr5GSGS9jGr6GSGS:jGr7GSGS;jGr8gs snf!,(df  G N=f!\\\4aH \!GR&"SS 5r\GR"\5rSSS5 G M!,(df  G N=ff=f(zGUtility helpers for LiteLLM core request handling and provider support.) annotationsN) dataclassfield) lru_cachewraps) resources) iscoroutine)StringIO)abspathdirnamejoin)Proxy)get_environment_proxies)_parsing _pydantic)ResponseFormat) BaseModel)Encoding) Tokenizer)is_internal_call)_get_default_encoding_get_modified_max_tokens_get_token_counter_new)uuid) $DEFAULT_CHAT_COMPLETION_PARAM_VALUESDEFAULT_EMBEDDING_PARAM_VALUESDEFAULT_MAX_LRU_CACHE_SIZEDEFAULT_TRIM_RATIOFUNCTION_DEFINITION_TOKEN_COUNTINITIAL_RETRY_DELAYJITTERMAX_RETRY_DELAYMAX_TOKEN_TRIMMING_ATTEMPTS MINIMUM_PROMPT_CACHE_TOKEN_COUNTOPENAI_EMBEDDING_PARAMSTOOL_CHOICE_OBJECT_TOKEN_COUNTc,[cSSKJn Uq[$)z Get cached CustomLogger class. Lazy imports on first call to avoid loading custom_logger at import time. Subsequent calls use cached class for better performance. r CustomLogger) _CustomLogger"litellm.integrations.custom_loggerr)r(s _/home/rurouni/.local/share/pipx/venvs/strix-agent/lib/python3.13/site-packages/litellm/utils.py_get_cached_custom_loggerr-TsC$ c,[cSSKJn Uq[$)z Get cached CustomGuardrail class. Lazy imports on first call to avoid loading custom_guardrail at import time. Subsequent calls use cached class for better performance. rCustomGuardrail)_CustomGuardrail%litellm.integrations.custom_guardrailr1r0s r,_get_cached_custom_guardrailr4bsI* r.c,[cSSKJn Uq[$)z Get cached CachingHandlerResponse class. Lazy imports on first call to avoid loading caching_handler at import time. Subsequent calls use cached class for better performance. rCachingHandlerResponse)_CachingHandlerResponselitellm.caching.caching_handlerr7r6s r,$_get_cached_caching_handler_responser:ps&J"8 ""r.c,[cSSKJn Uq[$)z Get cached LLMCachingHandler class. Lazy imports on first call to avoid loading caching_handler at import time. Subsequent calls use cached class for better performance. rLLMCachingHandler)_LLMCachingHandlerr9r=r<s r,_get_cached_llm_caching_handlerr?~s!E. r.cd[c$SSKnURRRq[$)z Get cached audio_utils.utils module. Lazy imports on first call to avoid loading audio_utils.utils at import time. Subsequent calls use cached module for better performance. Nr)_audio_utils_module,litellm.litellm_core_utils.audio_utils.utilslitellm_core_utils audio_utilsutils)litellms r,_get_cached_audio_utilsrGs+";%88DDJJ r.)AllMessageValuesAllPromptValuesChatCompletionAssistantToolCall"ChatCompletionNamedToolChoiceParamChatCompletionToolParam$ChatCompletionToolParamFunctionChunkOpenAITextCompletionUserMessageOpenAIWebSearchOptions) FileTypes)!OPENAI_RESPONSE_HEADERS CallTypesChatCompletionDeltaToolCallChatCompletionMessageToolCallChoices CostPerTokenCredentialItemCustomHuggingfaceTokenizerDelta EmbeddingEmbeddingResponseFunction ImageResponse LlmProvidersLlmProvidersSetLLMResponseTypesMessage ModelInfo ModelInfoBase ModelResponseModelResponseStream ProviderFieldProviderSpecificModelInfoRawRequestTypedDictSandboxProvidersSearchProvidersSelectTokenizerResponseStreamingChoices TextChoicesTextCompletionResponseTranscriptionResponseUsageall_litellm_paramsdict_CALL_TYPE_ENUM_MAPz%litellm.litellm_core_utils.tokenizerszanthropic_tokenizer.jsonrutf-8)encoding) TYPE_CHECKINGAnyCallableDictIterableListLiteralMappingOptionalTupleTypeUnioncastget_args)BaseLLMModelInfotype_to_response_format_param)r7r=r()BaseFilesConfig)BaseRealtimeHTTPConfig)AllowedModelRegion)exception_type)_is_non_openai_azure_modelget_llm_provider)get_supported_openai_params)LiteLLMResponseObjectHandler#_handle_invalid_parallel_tool_calls convert_to_model_response_objectconvert_to_streaming_response#convert_to_streaming_response_async) get_api_base)ResponseMetadata)_parse_content_for_reasoning)LiteLLMLoggingObject(redact_message_input_output_from_logging)CustomStreamWrapper)$BaseGoogleGenAIGenerateContentConfig) BaseOCRConfig)BaseSandboxConfig)BaseSearchConfigBaseTextToSpeechConfigBedrockModelInfo)CohereModelInfo)MistralOCRConfig)get_coroutine_checkerget_litellm_logging_classget_set_callbacks) get_litellm_metadata_from_kwargsmap_finish_reasonprocess_response_headers)delete_nested_valueis_nested_path)*_get_base_model_from_litellm_call_metadataget_litellm_params)_ensure_extra_body_is_safe)get_formatted_prompt)get_response_headers)update_response_metadata)Rules)executor)BaseAnthropicMessagesConfig)BaseAudioTranscriptionConfig)AsyncHTTPHandler HTTPHandler)!get_num_retries_from_retry_policyreset_retry_policy)BaseBatchesConfig)BaseContainerConfig)BaseEmbeddingConfig)BaseImageEditConfig)BaseImageGenerationConfig)BaseImageVariationConfig)BasePassthroughConfig)BaseRealtimeConfig)BaseRerankConfig)BaseVectorStoreConfig)BaseVectorStoreFilesConfig)BaseVideoConfig)ANTHROPIC_API_ONLY_HEADERSAnthropicThinkingParam)RerankResponse) ChatCompletionDeltaToolCallChunkChatCompletionToolCallChunk#ChatCompletionToolCallFunctionChunkLiteLLM_Params) get_secret) BaseConfig)BaseTextCompletionConfig)BaseEvalsAPIConfig)BaseResponsesAPIConfig)BaseSkillsAPIConfig)_is_debugging_onverbose_logger)AzureBlobCacheCacheQdrantSemanticCache RedisCacheRedisSemanticCacheS3Cache)APIConnectionErrorAPIErrorAuthenticationErrorBadRequestErrorBudgetExceededErrorContentPolicyViolationErrorContextWindowExceededError NotFoundError OpenAIErrorPermissionDeniedErrorRateLimitErrorServiceUnavailableErrorTimeoutUnprocessableEntityErrorUnsupportedParamsError) MockExceptionOptional[List[str]] callback_listOptional[Dict[str, str]]additional_details local_cacheFcUS:Xa[R"U5 O9US:Xa[R"U5 OUS:Xa[R"U5 [R SLaUSLa [ U5 ggg![a gf=f)NDEBUGINFOERRORTF)rdebuginfoerrorrF set_verboseprint Exception)print_statement logger_only log_levels r, print_verbosers      1 &    0 ' !   1   $ &;%+? / ",@ &    sA9A>> B  B c[RHwnUS[R;a"[RRUS5 US[R;dMU[RRUS5 My g)z* Add custom_llm provider to provider list providerN)rFcustom_provider_map provider_listappend_custom_providers) custom_llms r,custom_llm_setuprsl11 j !)>)> >  ! ! ( (J)? @ j !)B)B B  % % , ,Z -C D 2r.cSSKJn SSKJn U[R;a+[ R "SUS[R35 gU"[X 5SSS9nU(GavUS:Xa[U5S La[RRU5 [RRU5 U[R;a[RRU5 U[R;a [RRU5 ggUS :Xa[U5S La[RR!U5 [RR#U5 U[R$;a[R$RU5 U[R&;a [R&RU5 ggggg) z4 Add a custom logger callback to the specific event r)+_custom_logger_compatible_callbacks_literal)$_init_custom_logger_compatible_classz Callback z@ is not a valid custom logger compatible callback. Known list - Ninternal_usage_cache llm_routersuccessFfailure)rFr*litellm.litellm_core_utils.litellm_loggingr )_known_custom_logger_compatible_callbacksrrr0_custom_logger_class_exists_in_success_callbackslogging_callback_manageradd_litellm_success_callback"add_litellm_async_success_callbacksuccess_callbackremove_async_success_callback0_custom_logger_class_exists_in_failure_callbacksadd_litellm_failure_callback"add_litellm_async_failure_callbackfailure_callback_async_failure_callback)callback logging_eventrr callback_classs r,-_add_custom_logger_callback_to_specific_eventr s DwHHHz!abicTcTbU V  9 8C!N  Y &@P  , , I I   , , O O 7333((//7::://66; Y &@P  , , I I   , , O O 7333((//7::://66; '+r.cl^[U4Sj[R[R-55$)az Returns True if an instance of the custom logger exists in litellm.success_callback or litellm._async_success_callback e.g if `LangfusePromptManagement` is passed in, it will return True if an instance of `LangfusePromptManagement` exists in litellm.success_callback or litellm._async_success_callback Prevents double adding a custom logger callback to the litellm callbacks c3N># UHn[U[T55v M g7fN isinstancetype.0cbrs r, C_custom_logger_class_exists_in_success_callbacks..J'LB 2tN+,,L"%)anyrFrrrs`r,rr@0 **W-L-LL r.cl^[U4Sj[R[R-55$)az Returns True if an instance of the custom logger exists in litellm.failure_callback or litellm._async_failure_callback e.g if `LangfusePromptManagement` is passed in, it will return True if an instance of `LangfusePromptManagement` exists in litellm.failure_callback or litellm._async_failure_callback Prevents double adding a custom logger callback to the litellm callbacks c3N># UHn[U[T55v M g7fr#r$r's r,r*C_custom_logger_class_exists_in_failure_callbacks..Zr,r-)r.rFrrr/s`r,rrPr0r.cURS5=(d 0nURS5=(d 0nURS5=(d /nU$)z, Get the request guardrails from the kwargs metadatarequester_metadata guardrailsget)kwargsr5r6applied_guardrailss r,get_request_guardrailsr<`sHzz*%+H!&:;Ar+// =C r.c`[U5n/n[5n[RHnUcM[ XC5(dMUR cM)UR SLaURUR 5 MUUR U;dMgURUR 5 M U$)zP - Add 'default_on' guardrails to the list - Add request guardrails to the list T)r<r4rF callbacksr%guardrail_name default_onr)r:request_guardrailsr;r1rs r,get_applied_guardrailsrBjs 0724O%%  Jx$I$I&&2&&$.&--h.E.EF,,0BB&--h.E.EF & r.c[[R[S5nUR S5nU(aL[ R (a6URU5nUR5HupEX@;dM XPU'M ggg)zA Updates kwargs with the credentials if credential_name in kwarg CredentialAccessorlitellm_credential_nameN) getattrsysmodules__name__r9rFcredential_listget_credential_valuesitems)r:rDcredential_namecredential_accessorkeyvalues r,load_credentials_from_listrQ~so !X!68LMjj!:;O7220FFW-335JC #s 63r.cr[RR5nU(aURU5 U$r#)rFr>copyextend)dynamic_callbacksreturned_callbackss r,get_dynamic_callbacksrWs1!**//1!!"34 r.cUS;a%US;aUSL=(a SUR5;$gUSL=(a SUR5;$)zB Check if the target model is a Gemini or Vertex AI Gemini model. )gemini vertex_aivertex_ai_beta)rZr[NrYTlowermodelcustom_llm_providers r,_is_gemini_modelrasQGG "A A$BU[[])B B   :U[[]!::r.c:X;aURUS5S$U$)z/ Remove thought signature from a tool call ID. rrsplit) tool_call_id separators r,!_remove_thought_signature_from_idrgs( !!)Q/22 r.cURS5nURS5nUS:Xa[U[5(a/nUHn[US5(aUR 5nO9[U[ 5(aUR 5nOURU5 M^[URS5[5(aXS;a[USU5US'URU5 M X@S'U$)zL Process assistant message to remove thought signatures from tool call IDs. role tool_calls assistant model_dumpid) r9r%listhasattrrlrrrSrstrrg)msg_copythought_signature_separatorrirjnew_tool_callstctc_dicts r,%_process_assistant_message_tool_callsrvs << Dl+J {z*d;;Br<((--/B%%'')%%b)'++d+S11.$-?$E 'B%GDM  ! !' *%&"0 Or.cURS5S:Xa>[URS5[5(aXS;a[USU5US'U$)zE Process tool message to remove thought signature from tool_call_id. ritoolre)r9r%rprg)rqrrs r,_process_tool_message_idrysW||Fv%*X\\.5QSV*W*W &>*B B'H(*E(H^ $ Or.c /nUHn[US5(aUR5nO9[U[5(aUR 5nOUR U5 M^[ XA5n[XA5nUR U5 M U$)z? Remove thought signatures from tool call IDs in all messages. rl)rorlr%rrrSrrvry)messagesrrprocessed_messagesmsgmsg_dicts r,(_remove_thought_signatures_from_messagesrs  3 % %~~'H T " "xxzH  % %c * 9   ,HR!!(+'* r.c^0[RSLa[R"S5 [ 5 [ U5nSU;aUSOSn[ [R[S5nU"5nURSS5n [U S9n [U 5S:GaU GHqm0[T0[5(aV[RR R#T0SSS9m0T0b([%U04S j[R&55(aMoT0[R(;a[R(R+T05 T0[R,;a[R.R1T05 T0[R2;a[R.R5T05 T0[R&;a[R.R7T05 T0[R8;dGMR[R.R;T05 GMt [=S [R&35 [[R(5S:d:[[R,5S:d[[R25S:a[[>5S:Xau[A[C[R([R,-[R2-55q[ [R[S 5n U "5"[>US 9 [[R(5S:a/n [E[R(5HNun m0URGT05(dM[RHR+T05 U R+U 5 MP [KU 5H"n [R(RU 5 M$ [[R,5S:Ga/n [E[R,5Hun m0URGT05(a2[R.R7T05 U R+U 5 MNT0S :XdT0S:Xa2[R.R7T05 U R+U 5 MT0[RL;dM[T0[5(dM[OT0S5 M [KU 5H"n [R,RU 5 M$ [[R25S:a/n [E[R25Hun m0URGT05(a2[R.R;T05 U R+U 5 MNT0[RL;dMd[T0[5(dM{[OT0S5 M [KU 5H"n [R2RU 5 M$ SnSnSnSnURQSS5b[US[@5(a/n [EUS5Hhun m0URGT05(dT0S :XdT0S:XdM*Ub'[U[@5(aUR+T05 OT0/nU R+U 5 Mj [KU 5Hn USRU 5 M URS5nURQSS5b)[US[@5(aURS5n[R(ahSSK*J+n U"U5n[RZ(a6URSS5 URSS5 URSS5 [SSSU3SS9 SU;aUSq.[U5S:aUSOURQSS5nUnU[^R`Rb:Xd=U[^RdRb:XdU[^RfRb:XGaSn[U5S:aUSnOURQSS5(aUSn[ [R[S5nURh"5(a[U[@5(a[U5S:a[US[j5(ayS US;ap[m5nUHBnURQS S!5nUcM[U[5(dM1URoU5 MD URqURs5US"9 [U[@5(a[U5S:aSS#K:J;n SS$KS5S?9 U.U4$![Xa UnGNf=f![Xa GNf=f![Xa-n![R"S([U!535 Sn!A!GNVSn!A!ff=f![Xa/n![R~"S5[U!535 S/nSn!A!GNSn!A!ff=f![Xan![R"S@5 U!eSn!A!ff=f)ANTze`litellm.set_verbose` is deprecated. Please set `os.environ['LITELLM_LOG'] = 'DEBUG'` for debug logs.rmrr>)rUrr c3N># UHn[U[T55v M g7fr#r$)r(r)rs r,r*!function_setup..$s&/"AB#2tH~66"Ar-z8Initialized litellm callbacks, Async Success Callbacks: r)r function_iddynamodb openmeterr rrs3r)safe_deep_copyr{inputpromptzlitellm.llm_callzKeyword Args: r)categorymessagelevel logger_fnr_rrcontentrr_)r)THOUGHT_SIGNATURE_SEPARATORr`r^zCRemoving thought signatures from tool call IDs for non-Gemini modelz6Error removing thought signatures from tool call IDs: queryfile)file_objr5 file_checksumspeechzdefault-message-value)GoogleGenAIAdapter)get_last_user_messagecontentsconfig)r_rrz0Error extracting messages from Google contents: Fr: call_typerlitellm_call_idlitellm_trace_id)r_r{streamrrrr start_timedynamic_success_callbacksdynamic_failure_callbacksdynamic_async_success_callbacksdynamic_async_failure_callbacksr:r;api_baselitellm_metadatastream_options)r_useroptional_paramslitellm_paramsrzKlitellm.utils.py::function_setup() - [Non-Blocking] Error in function_setup)arFrrwarningrrBrFrGrHrIpoprWlenr%rprClitellm_loggingr r.rinput_callbackrrrrrrrrrrrrnset enumerateis_async_callable_async_input_callbackreversedrr r9add_breadcrumb'litellm.litellm_core_utils.core_helpersrrturn_off_message_logginguser_logger_fnrR completionrP acompletionanthropic_messageshas_pre_call_rulesrrr writepre_call_rulesgetvalue1litellm.litellm_core_utils.get_llm_provider_logicr3litellm.litellm_core_utils.prompt_templates.factoryrrarrtuple embedding aembeddingimage_generationaimage_generation moderation amoderationatext_completiontext_completionrerankarerankatranscription transcriptionrGget_audio_file_content_hashaspeechr aresponses responsesgenerate_contentagenerate_contentgenerate_content_streamagenerate_content_stream,litellm.google_genai.adapters.transformationr8litellm.litellm_core_utils.prompt_templates.common_utilsr(translate_generate_content_to_completion_is_streaming_requestrSupdate_environment_variables exception)1original_function rules_objrargsr:r;rget_coroutine_checker_fncoroutine_checkerrU all_callbacksrremoved_async_itemsindexrrrrrdetails_to_logr_rr{rbuffermrrrr`_r| args_liste _file_objrDrrrcontents_param model_paramadapter transformedtransformed_messagesrr logging_objrrs1 @r,function_setuprs d" s U 4F;6:V^VD\ $+ KK !#:$  56 JJ{D ) .@QR }  !)h,,&99IInn t o H '3/")"A"A/,,!7#9#99**11(;7#;#;;44QQRZ[7#;#;;44QQRZ[7#B#BB44WWX`a7#B#BB44WWX`a)** J7KjKjJkl  && '! +7++,q07++,q0  !**../../M!( H(=?R S   m U w%% & *"$ #,W-C-C#Dx$66x@@1188B'..u5$E ""56&&**517 w'' (1 ,"$ #,W-E-E#Fx$66x@@44WW (..u5+x;/F44WW (..u5 Q QQ"8S11A(IV#$G(""56((,,U37 w'' (1 ,"$ #,W-E-E#Fx$66x@@44WW (..u5 Q QQ"8S11A(IV$G""56((,,U37  "  (  "  ( ::($ / ; % &A A #% #,V4F-G#Hx%77AA:-4'6Bz7HH8>>xH;C*7'..u5$I""56)*..u57(. 3E(F % ::($ / ; % &A A )/ 3E(F % > (R!/!7//"":t4""7D1""8T2 +((89  & #K0Nt9q=Qfjj$.G% --33 3I11777I88>>>H4y1}7J--!*-CKK17;E((**x..MA%x{D11!,!!AeeIr2G*z'3/G/G W-" (( //+)(D))c(ma.?- +1**5J*K'/5!;K&+4G<8A2Aq,E3FGG&,,a .V$&A.* &/1CF:. Y](,T I+=IaL#(#3D ,,22 2I00666"%d)a-tAwVZZ5NH 3399 9I77==="%d)a-tAwVH5EH --33 3I11777"%d)a-tAwVG_H 3399 9I55;;;"%d)a-tAwVH5EH ))// /9 @Q@Q@W@W3Wzz'*H 1177 7I33999.1$i!m47I13K'CCYCWMV#6Cz"?3&5}%Ez"$H **00 0IAQAQAWAW4Wzz'84H --33 3I//555 t9q=QZZ(C::j*AB  3399 9I77===I==CCCI>>DDD 3-0IMavzz*?U),TQd1gFJJwPR% !01"#45#ZZ(:;#)r!&?&?,K,K1 $+5b)9  )/ );N: &  'J % &- - 288J1K1P1P1RN- . "%%j11-34F-G-L-L-Nz*00)!::&6= 1 F""[ (!' (N )! !.!"**PQTUVQWPXYp 3$$Fs1vhO3  3p   Y  s G5E1*D<*C5B*#C$|62F8.A+} } >}*}?NB(~01~3D6 }} }}}} ~$"~  ~  $   7227c# USLaM[SUR35 SSKJn UR URXUS9S9 UR UUUS9 gg7f)NFz>Async Wrapper: Completed Call, calling async_success_handler: r)GLOBAL_LOGGING_WORKERresultrend_time)async_coroutine)rasync_success_handler)litellm.litellm_core_utils.logging_workerrensure_initialized_and_enqueue-handle_sync_success_callbacks_for_async_calls)rrrris_completion_with_fallbacksrs r,_client_async_logging_helperrs} %-L[MnMnLo p T<<'==x> =  AA! B ' .sAAcNURSS5nUc[RnURSS5(ag[[R [ S5n[[R [ S5nU"UURS5S9nU"5US'UbUnX 4$)za Get the number of retries from the kwargs and the retry policy. Used for the wrapper functions. num_retriesN retry_policyrrrr)r9rFrrFrGrHrI)r:rrrrretry_policy_num_retriess r,_get_wrapper_num_retriesrs**]D1K))  zz.$'',3 KK !#F- )%S[[%:r%async_pre_call_deployment_hook)r:rtyped_call_typemodified_kwargsr)rrs r,rrs #I.kkmO,.L%% h - -#BBF!"( &  s?B A4;BB"B#B,B4 BBBBc# [U5n[5n[RHEn[ XT5(dMUR U[[U5U5IShvN nUcMCUs $ U$![a SnNsf=fN#7f)zX Allow modifying / reviewing the response just after it's received from the deployment. N) rRrr-rFr>r%'async_post_call_success_deployment_hookrr`) request_dataresponserrr)rrs r,rrs #I.-.L%% h - -#KKd#3X>F! & O  s>B A4+B$B"B#B,B4 BBBBcUcgURnU[RR:XdU[RR:XGa[ U5nUSLag[ U[5(Ga[UR5S:GaURSRRnUGbURXqS9 UbURS5OSnUbUO[Rn U SLaUbSU;aUSbSn [ US[ 5(aUSRS5bUSn O3["R$R'US5(a [)USS9n U b.[R*R,R/U SSUS 9 UbSU;a[ US[ 5(aS US;axUSS S :XakS US;aa[ USS [ 5(aES US;a;USS SLa/[R*R,R/USS US 9 gggggggggggggg![0a Nf=f![2an U eSn A ff=f)NTrrenable_json_schema_validationresponse_format json_schemarschema)rrr& json_objectresponse_schemaenforce_validation)rIrRrrPrr r%rdrchoicesrrpost_call_rulesr9rFrrrr _completionsis_basemodel_typerrCjson_validation_rulevalidate_schema TypeErrorr) original_responser_rrrr is_coroutinemodel_response_per_request_validation_enable_json_schema_validationjson_response_formatrs r,post_call_processingr,sUk  $ )22IY11777 5 5 ; ;;./@A 4'##4mDD 1 9 9:Q>8I8Q8Q9!''''*5%55N5X $3#>!0 3 34S T%)4$;#F!8%,%J%J; >E'!)(7(C,=,P,;O.P$P$34E$Fv$N#0%1$5#23D#E%F$.$34E$F(9%&%) %"%" %9#23D#E%F$34E$F$8%"$(%( !( : : O O _ _+:;L+M(9,&.< !`!" %(%F %"%F%1%Q%Y%I!TRn[U5(a[RbUUR S5=(d 0R SS5nUb([R[ U5::a [ S5eT"U0UD6n[UUS9(a[SU;aSUSSLaK/n[U5HupgURU5 M [R"XQR SS5S9$U$U$[TX5 [RR5nSnUR S S5n S U;a![[R "55US '[ U5S :aUS OUR S S5n U c[#TRTU/UQ70UD6upU cS 5e[%U5 XS '['5n U "TUUS9n Xl[R*(aH[R,[R*:a&[/[R,[R*S9e[RbUUR S5=(d 0R SS5nUb([R[ U5::a [ S5e[1SUR SS5S[R2SUR S05R SS535 UR SS5c[R2cUR SS5SLGaUR S05R SS5SLaUR SS5SLaUR SS5SLaUR SS5SLaUR SS5SLaUR SS5SLaUR SS5SLaxUR SS5SLac[4R6"S5 U R9U =(d STU UUUUS 9n U R:b"[4R6"S!5 U R:$UR S"S5bU b[R<SLaU[>R@RB:Xd<U[>RDRB:XdU[>RFRB:XaU nUR S#S5bS$UR S#53nSn[ U5S%:aUS%nOUR SS5(aUSnUR S"5n[I5"U UUUSSS&9nUUS"'T"U0UD6n[RR5n[UUS9(aSU;aSUSSLaK/n[U5HupgURU5 M [R"XQR SS5S9$[K[LRN[S(5nU"UU U UUUS)9 U$SU;a USSLaU$SU;a USSLaU$SU;a USSLaU$SU;a USSLaU$S*U;a US*SLaU$[PRR"U5(aU$[UUU =(d SUTTS+9 U RWUUUS,9 [4RX"S-5 [ZR\"5n[K[LRN[S.5nUR^"UR`U RbUUU5 [K[LRN[S(5nU"UU U UUUS)9 U$![ a"n[1S'[U535 SnAGNSnAff=f![ GanTRnU[>RDRB:XGaUR S/S5=(d [Rd=(d SnUR S0S5(ab[K[LRN[S15n[K[LRN[S25nU"UUR S05S39nU"5US0'S[l2UR S405nS5UR S5=(d 0;nU(aU(d[gU[hRj5(d>[gU[hRl5(d[gU[hRn5(a UUS/'[Rp"U0UD6sSnA$GO[gU[RrRt5(aAU(a:U U;a4U(d-[ U5S :a UU US 'OUU US 'T"U0UD6sSnA$GO}U[>RvRB:XGa^UR S/S5=(d [Rd=(d SnUR S0S5(ab[K[LRN[S15n[K[LRN[S25nU"UUR S05S39nU"5US0'S[l2S5UR S5=(d 0;nU(aU(d}[gU[hRj5(d>[gU[hRl5(d[gU[hRn5(a UUS/'[Rx"U0UD6sSnA$[zR|"5n[RR5nU (aU RUUUU5 UeSnAff=f)6Nr5previous_modelszMax retries per request hit!rcomplete_responseTr{r{litellm_logging_objrrr_3logging_obj should not be None after function_setuprrequest_kwargsr current_cost max_budgetzSYNC kwargs[caching]: cachingF; litellm.cache: z#; kwargs.get('cache')['no-cache']: cachezno-cacherrraimg_generationrr _arealtimezINSIDE CHECKING SYNC CACHErr_rrrrr:rz Cache hit! max_tokens hf_model_name huggingface/rr_ base_modelr{user_max_tokens buffer_num buffer_perc&Error while checking max token limit: rrrr_r:rrrr&r_rrr)rrr:z0Wrapper: Completed Call, calling success_handlerrrrrrrcontext_window_fallback_dict model_group)@rI_is_async_requestrFnum_retries_per_requestr9rrrrrstream_chunk_builderprint_args_passed_to_litellmdatetimenowrpruuid4rrQr?_llm_caching_handlerr8 _current_costrrr;rr_sync_get_cache cached_result modify_paramsrRrrPrrrrFrGrHasyncior r,sync_set_cacher contextvars copy_contextsubmitrunsuccess_handlerrr%openairrrcompletion_with_retries exceptionsrrresponses_with_retries traceback format_excfailure_handler)rr:rr/rchunksidxchunkrrr_r=rScaching_handler_responserCr{rDmodified_max_tokensrrrctxrrrrrJ_is_litellm_router_calltraceback_exceptionrrs r,wrapperclient..wrappers &.. V $ $..:#)::j#9#?R"D"D%t##.66#o:NN'(FGG'77F$# (6123t;F&/&7  e,'8"77J)E"MM %%6E&&**, 6$>> I$@$@$F$FFU!&Jzz/48D'3FJJ4O3P%Q #H4y1}#'7J55#)*#5&,jj&>O*B*D##-!)(7#'$( +',?F<('77F((,,.H$# (6123t;F&/&7  e,'8"77J)E 07 H-/I0,-%$/#%#-!) "M&(VM-Bd-J 'F<,@D,H "f,8I1Jd1R !V+7G0HD0P f$ ):d)B $$V,,  !"(mt &"3#   ! / / 0     R S**,Cs{{84jAH OO++  (/ H%'A( $ %'%!  Mo!U!$J3q6("STTUp^ )22II00666JJ}d3Rw7J7JRd::nd338? H-/R95*1 H-/C*&#D"#%+ZZ%?#K +,>*#06zz2B0,+8JJz*0b+' (?#1foo66%a88%a)B)BCC0;}-&>>OOOD q'"4"4"O"OPP4!==34y1}">u"EQ*Fu*Mw,d=f==i11777JJ}d3Rw7J7JRd::nd338? H-/R95*1 H-/C*&#D"#%+ZZ%?#K +,>*#+8JJz*0b+' (?#1foo66%a88%a)B)BCC0;}-&==tNvNN"+"6"6"8 ((,,.H++*JG}^ sJ:]+?B]+B\<B ]+-]+]+]+$]+4]+]+]+1C ]+< ](]#]+#]((]++ m6Fm  mA&m 7m=E9m 6m^^^^^# [TX5 [RR5mSm[[R [ S5nURSS5m[5nU"TUTS9nTR nSU;a![[R"55US'[U5S:aUSOURSS5nURS5SLmURSS5 [R"5nTc[TR T T/UQ70UD6umnTcS 5e[!X5IShvN nUbUnURS 5n U bTR"U :waU TlTUS'[%U5 UTl[(R*(aH[(R,[(R*:a&[/[(R,[(R*S 9e[15(aB[3S URS S5S[(R4SURSS535 UR7U=(d STTTUUUS9IShvN n U bBU R8bU R:c U R8$U R<SLa U R:$URSS5bUb[(R>SLaU[@RBRD:Xd<U[@RFRD:XdU[@RHRD:XaUn URSS5bSURS53n Sn [U5S:aUSn OURSS5(aUSn URS5n [K5"UU U U SSS9nXS'T"U0UD6IShvN m[RR5m[OUUS9(ahSU;aUUSSLaM/n[QT5HunnURSU5 M [(RT"UURSS5S9$U"TTUUTTS9 T$U[@RVRD:XaT$[YTUUTT S 9 [ZRU5nUb[]UTUS!9IShvN mUR_TTUUS"9IShvN U(dG[TS#S5(aS2UUUUU4S$jjnUTl0O![bRd"[gTTTTTS%95 TRiTTTS&9 [kT[l5(a#U b U R:bURoU TTTS'9$U"TTUUTTS9 T$GN(GN![La"n[3S[U535 SnAGNSnAff=fGNNN![LGa n[pRr"5n[RR5mT(adU(d]TRuUUTT5 O![LanUeSnAff=fTRwUUTT5IShvN O![LanUeSnAff=fTR n[yXS(9unnU[@RBRD:XGaEURS)05nS*URS+5=(d 0;nU(aU(dS[(l=UUS,'TUS-'[kU[|R~5(aS.US/'O$[kU[|R5(aS0US/'[(R"U0UD6IShvN sSnA$![La GOTf=f[kU[(RR5(aJU(aCUU;a=U(d6[U5S:a UUUS'OUUUS'T"U0UD6IShvN sSnA$OU[@RRD:XaS*URS+5=(d 0;nU(aU(dS[(l=UUS,'TUS-'[kU[|R~5(aS.US/'O$[kU[|R5(aS0US/'[(R"U0UD6IShvN sSnA$![La Of=f[US,U5 [XS(9n[US1U5 UeSnAff=f7f)3Nrr2r4rrr_ fallbacks_is_litellm_internal_callr3rr6zASYNC kwargs[caching]: r9Fr:z; kwargs.get('cache'): r;rr>Tr?r@rArr{rBrGrr0r1rHrI)rrr)rrr:r_defer_async_loggingc H>[R"[TTTTTS95 g)Nrrrrr)rX create_taskr)rrrrrsr,_enqueue_deferred_logging@client..wrapper_async.._enqueue_deferred_loggings(++8,7'-+5)1=Y r.rur)_caching_handler_responseembedding_responserr)r:rrJrKr5rrexponential_backoff_retryretry_strategyconstant_retryrreturnNone)HrOrPrQrFrGrHrIr9r?rprrRrrrrrrrQrSrFr8rTrrrr;_async_get_cacherVfinal_embedding_cached_response embedding_all_elements_cache_hitrWrRrrPrrrrrrrrN arealtimer,rsrasync_set_cacherwrXrvrrr%r[2_combine_cached_embedding_response_with_api_resultrcrdreasync_failure_handlerrrr_rracompletion_with_retriesrarraresponses_with_retriessetattrr )!rr:_update_response_metadatar=rSrr_rrr _hook_streamryrCr{rDrjrrfrgrh_call_type_enumrwrmrrJrlrrrrrrrrs! @@@@@r, wrapper_asyncclient..wrapper_asyncsd $%6E&&**, $+ KK !#=% !7=jj !47 <=4E/!!5  &.. F *(+DJJL(9F$ %*-d)a-tAwVZZQU=V'-zz+'>d'J$ .5$4$8$8$:!h "&4%.. :'HL'PV'# V ' ED E'%C6$UUO*("::h/L'K,>,>,,N%1 ",7F( ) &v ./CK ,!!((7+=+==-%,%:%:#*#5#5  !!-fjjE.J-KK\]d]j]j\klCDJDNDNOVX\D]C^_+;;+2&7 +)'!< &)4-;;G1QQ5BBB.OOSWW4TTT <.:%))!6!6!$>> I$@$@$F$FFU!&Jzz/48D'3FJJ4O3P%Q #H4y1}#'7J55#)*#5&,jj&>O*B*D##-!)(7#'$( +',?<( -d=f==F((,,.H$# (6123t;F&/&7 U e,'8"77J)E.%$/#%#-!) "Mi11777 "( &"3#  255i@O*F!'#- '66"3 7  -;(>FF  =VK9''4(3#)'1%-9U   E E%! F 6#455-9-MM,^^.G'-)% _ &'%!  MKV6j!U!$J3q6("STTU>P FW "+"6"6"8 ((,,.H#<//. H!G%;;. H!G*22I":&"V KI11777/5zz2B0,+8JJz*0b+'  (?  +1<}-6G23%v448SF#34'6??;;7GF#34%,%E%Et%Vv%VVVV$q'"4"4"O"OPP4!==34y1}">u"EQ*Fu*Mw!2D!CF!CCCCi22888*7JJz*0b+'  (?  +1<}-6G23%v448SF#34'6??;;7GF#34%,%D%Dd%Uf%UUUU$ =+ +&FG Ay' *GoW sDc9V+U/DV+"U2#-V+cV++c,BV+0BU54 V+?V$B V+ c V+cV+8c93V+,V'-V+V)BV+!c" V+.c/V+2V+5 V!?VV+V!!V+'V+)V++ c6Ac8X  c XXXc"Y9X<:Y?c Y Y  YA7c A7] ]] cc ]c]A+c_c c cA cA7bbbcc b)&c(b))(cccr)rFrGrHrIrr)rrrnrrr'rs` @r,clientrs CKK)7 3EI YYv  BBH $CKK$9;RS(*<<=NOLr.cUcgURSS5SLdURSS5SLdURSS5SLdURSS5SLdURSS5SLdnURSS5SLdYURS S5SLdDURS S5SLd/URS S5SLdURS S5SLdUSLagg) aR Returns True if the call type is an internal async request. eg. litellm.acompletion, litellm.aimage_generation, litellm.acreate_batch, litellm._arealtime Args: kwargs (dict): The kwargs passed to the litellm function is_pass_through (bool): Whether the call is a pass-through call. By default all pass through calls are async. FrTrr<rrrrr= acreate_batchacreate_fine_tuning_jobr8)r:is_pass_throughs r,rLrLBs~ =%(D0 ::lE *d 2 ::' /4 7 ::mU +t 3 ::(% 0D 8 ::& .$ 6 ::i '4 / ::lE *d 2 ::ou - 5 ::/ 74 ? d " r.c2SU;a USSLagU[;$)a Returns True if the call type is a streaming request. Returns True if: - if "stream=True" in kwargs (litellm chat completion, litellm text completion, litellm messages) - if call_type is generate_content_stream or agenerate_content_stream (litellm google genai) rT)_STREAMING_CALL_TYPESrs r,rrls'6fX.$6 - --r.cFUb[USUSUSS9nU$[US9$)N identifierrevision auth_token)rrrr_)create_pretrained_tokenizer_select_tokenizer_helper)r_custom_tokenizer _tokenizers r,_select_tokenizerr{s?#0' 5%j1' 5  #% 00r.maxsizec[RSLa [U5$[U5nUbU$[U5$![a#n[ R "SU35 SnAN3SnAff=f)NTzError selecting tokenizer: )rFdisable_hf_tokenizer_download_return_openai_tokenizer_return_huggingface_tokenizerrrr)r_rrs r,rrsu,,4'..@.u5  M  $E ** @:1#>??@s< A)A$$A)cS[5S.$)Nopenai_tokenizerr& tokenizer)rrs r,rrs&5J5L MMr.cU[R;a!SU;a[R"S5nSUS.$U[R;a%SU;a[R "[ 5nSUS.$SUR5;dSUR5;a[R"S5nSUS.$S UR5;a[R"S 5nSUS.$g) Nz command-rz#Xenova/c4ai-command-r-v01-tokenizerhuggingface_tokenizerrzclaude-3zllama-2 replicatez#hf-internal-testing/llama-tokenizerzllama-3zXenova/llama-3-tokenizer)rF cohere_modelsrfrom_pretrainedanthropic_modelsfrom_strclaude_json_strr])r_cohere_tokenizerclaude_tokenizerrs r,rrs %%%+*>$44 1 0>NOO '** *z/F$--o>/>NOO ekkm #{ekkm'C--.ST /iHH ekkm #--.HI /iHHr.rcU=(d [US9n[US[5(aUSRUSS9nOUSRU5n[ US5(a UR $U$)a Encodes the given text using the specified model. Args: model (str): The name of the model to use for tokenization. custom_tokenizer (Optional[dict]): A custom tokenizer created with the `create_pretrained_tokenizer` or `create_tokenizer` method. Must be a dictionary with a string value for `type` and Tokenizer for `tokenizer`. Default is None. text (str): The text to be encoded. Returns: enc: The encoded text. rr)disallowed_specialids)rr%rencoderor)r_textrtokenizer_jsonencs r,rrsr&G):)GN.-x88[)00"0M[)006sEww Jr.TcU=(d [US9nUSS:Xa*U(a[USU5nUSRXS9nU$USRU5nU$)a Decodes token ids using the selected tokenizer. Args: skip_special_tokens: For HuggingFace tokenizers, keep the historical LiteLLM round-trip behavior by omitting special tokens by default. Set to False to inspect decoded BOS/EOS tokens. rr&rr)skip_special_tokens)r$_strip_huggingface_special_token_idsdecode)r_tokensrrrdecs r,rrsy&G):)GNf!88 9{+VF[)00 1   % , ,V 4C Jr.c UR5nUR5VVs1sHup4[USS5(dMUiM nnnU(dU$UVs/sH ofU;dM UPM sn$![a Us$f=fs snnfs snf)NspecialF)get_added_tokens_decoderrrLrF)rradded_tokens_decodertoken_id added_tokenspecial_token_idstokens r,rrs(AAC &:%?%?%A%A !H ; 5 1 %A  % Hve6G)GEv HH   Is(A-A?A? B$B- A<;A<c[R"XUS9nSUS.$![a8n[R"SUS35 [R"XS9nSnANBSnAff=f)a Creates a tokenizer from an existing file on a HuggingFace repository to be used with `token_counter`. Args: identifier (str): The identifier of a Model on the Hugging Face Hub, that contains a tokenizer.json file revision (str, defaults to main): A branch or commit id auth_token (str, optional, defaults to None): An optional auth token used to access private repositories on the Hugging Face Hub Returns: dict: A dictionary with the tokenizer and its type. )rrz%Error creating pretrained tokenizer: z-. Defaulting to version without 'auth_token'.)rNrr)rrrrr)rrrrrs r,rrskM-- j ,) DD M3A36c d --jL Ms A.AAc8[R"U5nSUS.$)z Creates a tokenizer from a valid JSON string for use with `token_counter`. Args: json (str): A valid JSON string representing a previously serialized tokenizer Returns: dict: A dictionary with the tokenizer and its type. rr)rr)jsonrs r,create_tokenizerr s""4(I+) DDr.rc Z[RSLag[5"UUUUUUUUU5 $)z\ The same as `litellm.litellm_core_utils.token_counter`. Kept for backwards compatibility. Tr)rFdisable_token_counterr) r_rrr{count_response_tokenstools tool_choiceuse_default_image_token_countdefault_token_counts r, token_counterr# s@.$$, ! #   %  r.c/SQnX;agg)zM Helper function to know if a provider implementation supports httpx timeout )r_azurebedrockTFr)r`supported_providerss r,supports_httpx_timeoutrJ s91 r.c[UUSS9$)ah Check if the given model supports system messages and return a boolean value. Parameters: model (str): The model name to be checked. custom_llm_provider (str): The provider to be checked. Returns: bool: True if the model supports system messages, False otherwise. Raises: Exception: If the given model is not found in model_prices_and_context_window.json. supports_system_messagesr_r`rO_supports_factoryr^s r,rrV s / & r.c[UUSS9$)a^ Check if the given model supports web search and return a boolean value. Parameters: model (str): The model name to be checked. custom_llm_provider (str): The provider to be checked. Returns: bool: True if the model supports web search, False otherwise. Raises: Exception: If the given model is not found in model_prices_and_context_window.json. supports_web_searchrrr^s r,rrk s / ! r.c[UUSS9$)a` Check if the given model supports URL context and return a boolean value. Parameters: model (str): The model name to be checked. custom_llm_provider (str): The provider to be checked. Returns: bool: True if the model supports URL context, False otherwise. Raises: Exception: If the given model is not found in model_prices_and_context_window.json. supports_url_contextrrr^s r,rr s / " r.c [R"XS9up n[XS9nURSS5nUcSnU$![a2n[ R "SUSUS[U535 SnAgSnAff=f) aj Check if the given model supports native streaming and return a boolean value. Parameters: model (str): The model name to be checked. custom_llm_provider (str): The provider to be checked. Returns: bool: True if the model supports native streaming, False otherwise. Raises: Exception: If the given model is not found in model_prices_and_context_window.json. r^supports_native_streamingTNzYModel not found or error in checking supports_native_streaming support. You passed model=, custom_llm_provider= . Error: F)rFr_get_model_info_helperr9rrrrp)r_r`r model_inforrs r,rr s+2+C+C, (Aq, %/NN3NPT$U! $ ,(, %(( ghmgnoEFYEZZcdghidjck l  s9< A8(A33A8c [[R[S5nU"XS9up n[RR[RR[RR[RR[RR/nX;ag[UUS S 9$![a2n[ R "SUSUS[U535 SnAgSnAff=f) aK Check if the given model + provider supports 'response_schema' as a param. Parameters: model (str): The model name to be checked. custom_llm_provider (str): The provider to be checked. Returns: bool: True if the model supports response_schema, False otherwise. Does not raise error. Defaults to 'False'. Outputs logging.error. rr^zOModel not found or error in checking response schema support. You passed model=rrNFTsupports_response_schemar)rFrGrHrIrrrrprFr^ PREDIBASE FIREWORKS_AI LM_STUDIONEBIUS DATABRICKSr)r_r`rrr*PROVIDERS_GLOBALLY_SUPPORT_RESPONSE_SCHEMAs r,rr s "3;;x#8:LM+;, (Aq &&))&&##'' 2.H / & # ]^c]ddz|O{PPYZ]^_Z`Ya b  s+B== C9(C44C9c[UUSS9$)zS Check if the given model supports parallel tool calls and return a boolean value. "supports_parallel_function_callingrrr^s r,rr s / 0 r.c[UUSS9$)m Check if the given model supports function calling and return a boolean value. Parameters: model (str): The model name to be checked. custom_llm_provider (Optional[str]): The provider to be checked. Returns: bool: True if the model supports function calling, False otherwise. Raises: Exception: If the given model is not found or there's an error in retrieval. supports_function_callingrrr^s r,rr s / ' r.c[XSS9$)zM Check if the given model supports `tool_choice` and return a boolean value. supports_tool_choicerrr^s r,rr  BX r.cH[XS9nUbURUS5SLagg)z^ Check if the given model supports a provider specific model info and return a boolean value. r^NFT)get_provider_infor9)r_r`rO provider_infos r,_supports_provider_info_factoryr s3&M ]%6%6sE%Bd%J r.c [R"XS9up n[XS9nURUS5SLagURU5c][ U5nUb>[R RU5=(d 0nURUS5SLag[ XU5nUbU$g![aKn[R"SUSUSUS[U535 [ XU5nUbUsSnA$SnAgSnAff=f) rr^FTN%Model not found or error in checking z support. You passed model=rr) rFrrr9_get_model_cost_key model_costrrrrrp) r_r`rOrrbare_model_key bare_entrysupported_by_providerrs r,rr sG(+2+C+C, (Aq,  >>#u % - ^^C ( 17N)$//33NCIr >>#u-5$CC% !%0,,  3C58STYSZZprEqFFOPSTUPVOW X !@ !  ! ,( ( s)6B)AB)B)) C>3;C9.C>9C>c [R"XS9up n[XS9nURU5nUSLagUcK[ U5nUb=[R RU5=(d 0nURU5SLagg![ a5n[R"SUSUSUS[U535 SnAgSnAff=f) u>Return True only when the model map explicitly sets *key* to ``False``. This is the opt-out mirror of :func:`_supports_factory`. Where ``_supports_factory`` requires an explicit ``True`` to return ``True``, this function requires an explicit ``False``. A missing key (``None``) is treated as *not* disabled so that unknown or newly-added models are allowed through without any model-map entry. Uses the same ``get_llm_provider`` → ``_get_model_info_helper`` chain as ``_supports_factory`` so caching, fallback, and normalisation improvements apply here automatically. r^FTNrz" disabled state. You passed model=rr) rFrrr9rrrrrrp) r_r`rOrrvalrrrs r,_is_explicitly_disabled_factoryrY s+2+C+C, (Aq, nnS! %< ;07N)$//33NCIr >>#&%/ 3C59 %w&<=P` sJyy~H C H ! !hm&F q#((8D>23 H ! !hm&F(:b>SD1 ??= ) )   d3}#5#78 92$4N     h ' ""&&y1  +=!>!J;   r.c^0n[U[5(aUnO)[U[5(a[R"US9n[ R R[ RR1nUR5GHup4URSS5n[U5mXR;d[U4SjU55(a#[RRU05nUnO[[[US95nUSnURS5cUR)SS5 [RRU5n U c[RRU5n U c0n S H#n X;dM X;dMUR)U S5 M% [+Xd5n [RR-U05R/U 5 [15 [$R2"SUSU35 URS5S:Xa9U[R4;a"[R4R7U5 GMGMURS5S:Xa9U[R8;a"[R8R7U5 GMGMURS5S:Xa9U[R:;a"[R:R7U5 GM\GM_URS5S:Xa9U[R<;a"[R<R7U5 GMGMURS5S:XaQUR?SS5n U S[R@;a%[R@R7U S5 GMGMURS5S:Xa9U[RB;a"[RBR7U5 GM^GMaURS5S:Xa9U[RD;a"[RDR7U5 GMGMURS5S:Xa9U[RF;a"[RFR7U5 GMGMURS5S:Xa9U[RH;a"[RHR7U5 GMHGMKURS5S:Xa9U[RJ;a"[RJR7U5 GMGMURS5S:Xa9U[RL;a"[RLR7U5 GMGMURS5S:Xa9U[RN;a"[RNR7U5 GM2GM5URS5S:Xa9U[RP;a"[RPR7U5 GMGMURS5S :Xa9U[RR;a"[RRR7U5 GMGMURS5S!:XdGMU[RT;dGM[RTR7U5 GM" U$![a 0nUn[!TUS9nUb>["H1n URU 5bMURU 5cM+XXi'M3 GNURS 5c,URS 5c[$R&"S US 35 GN>f=f)"a Register new / Override existing models (and their pricing) to specific providers. Provide EITHER a model cost dictionary or a url to a hosted json blob Example usage: model_cost_dict = { "gpt-4": { "max_tokens": 8192, "input_cost_per_token": 0.00003, "output_cost_per_token": 0.00006, "litellm_provider": "openai", "mode": "chat" }, } urlr3rc3L># UHnTRUS35v M g7f)r1N)r5)r(p_key_strs r,r*!register_model.. s)= 2PQH  1#Q ( (2Ps!$rrO)rOrNr+r.zregister_model: model=z not in built-in cost map and no prefix/region variant matched; cache cost fields will default to 0. To track cache cost, add cache_creation_input_token_cost and cache_read_input_token_cost to model_info)input_cost_per_tokenoutput_cost_per_tokenzadded/updated model=z in litellm.model_cost: r_text-completion-openaicohere anthropic openrouterr1rr2vercel_ai_gatewayzvertex_ai-text-modelszvertex_ai-code-text-modelszvertex_ai-chat-modelszvertex_ai-code-chat-modelsai21 nlp_cloud aleph_alpharnovita)+r%rrrprFget_model_cost_mapr^GITHUB_COPILOTrPCHATGPTrLr9r.rrget_model_inforr>_CACHE_PRICING_FIELDSrrrr( setdefaultr"$_invalidate_model_cost_lowercase_mapropen_ai_chat_completion_modelsaddopen_ai_text_completion_modelsrrrdopenrouter_modelsvercel_ai_gateway_modelsvertex_text_modelsvertex_code_text_modelsvertex_chat_modelsvertex_code_chat_models ai21_modelsnlp_cloud_modelsaleph_alpha_modelsbedrock_models novita_models)rloaded_model_cost_skip_get_model_info_providersrOrPrexisting_modelmodel_cost_key builtin_entryr _raw_entry _cost_fieldupdated_dictionary split_stringrDs @r,register_modelro s *d##& J $ $#66:F ##))""&" (--/ 99/4s8  5= 2P= : : %//33C"22J>JZ [  99' (H 4'@@@66::3?A YY) *.F F'@@@66::3?A YY) *h 6'///%%))#.0 YY) *k 9'222((,,S13 YY) *l :99S!,LBw'@'@@))--l2.>?A YY) *.A A':::0044S9; YY) *.E E'444**..s35 YY) *.J J'999//33C8: YY) *.E E'444**..s35 YY) *.J J'999//33C8: YY) *f 4'---##'',. YY) *k 9'222((,,S13 YY) *m ;'444**..s35 YY) *i 7'000&&**3/1 YY) *h 6'///%%))#.e0f M !#!$ A 8! !,!6!IIe,4 - 1 1% 8 D4A4HN1 "7II?@H "?@H"**06HH# s$/Z//4] '] ; ] >]  ] c@Ub[U[5(aX;aggNTF)r%rnr%additional_drop_paramss r,_should_drop_paramrt s"* -t 4 4 ' r.c0nUR5H(upEXA;dM XQU:wdM[XBS9SLdM$XSU'M* U$)NrrF)rLrt) passed_paramsdefault_paramsrsnon_default_paramsr%r&s r,_get_non_default_paramsry" sR##% A&&"QV%&q !& r.c 8^^^SSKJn [5n U RS5 U RS5mU RS5mU RS5n U R 5H upXU 'M SSSSSS.nU R 5V V s0sHupX;dM XU :wdMX_M sn n m0nUUU4SjnSnTb[ R U[T5S 9nTS :XdTS :XaTnOTS :XaT[R"5R5nU"US 9 [R"5RTUUTbTOSS9nO/Ub,URUS9nU"US 9 URTUUTbTOSS9n[UU TU URSS5S9nU$s sn n f)NrOPENAI_TRANSCRIPTION_PARAMSr|r` drop_paramsr:)languagerr temperaturetimestamp_granularitiesc>[TR55S:ah[TR55nUHGnTSLd[RSLaX ;aTR US5 M4X ;dM;[ SSTS3S9e T$g)NrTz1Setting user/encoding format is not supported by =. To drop it from the call, set `litellm.drop_params = True`. status_coderrkeysrnrFr}rr)supported_paramsrr%r`r}rxs r,_check_valid_arg;get_optional_params_transcription.._check_valid_argY s !&&( )A -*//12D4'7+>+>$+F/&**1d3.0$'"STgShif!g & % .r.r_rr_rgroqrFrxrr_r}rrsrrvr` openai_paramsrs)litellm.constantsr|localsrrLProviderConfigManager'get_provider_audio_transcription_configr^rF GroqSTTConfigget_supported_openai_params_sttmap_openai_params_sttrmap_openai_params/add_provider_specific_params_to_optional_paramsr9)r_r`r~rrrrr}r:r|rvspecial_paramsr%r&rwrrprovider_configrrxs ` ` @r,!get_optional_params_transcriptionr2 s>HM34'++,AB##M2K"&&x0N$$&a'#' N"''))DA  $%):$: ) O &?CO&/WW!"56X  h&*=*H,  &"002RRT*:;!//1GG1+'2'> E H   $*FFUFS*:;);;1+'2'> E < F'#/1%zz*BDI O ws FF)Fc>UcgSSSSSS.nURUS5$)z3Map OpenAI size parameter to Vertex AI aspectRatio.z1:1z16:9z9:16)256x256512x512 1024x1024 1792x1024 1024x1792r8)sizesize_to_aspect_ratios r,*_map_openai_size_to_vertex_ai_aspect_ratior s= |   # # e r.c H^^^ ^^[5mTRSS5mTRS5mTRSS5n TRSS5m TRSS5n TRS5n U R5H`upURS5(aTS:waTS :waM)US :XaTS :waM7URS 5(aTS :waTS :waM[UTU'Mb SSSSSSSSSS. n[ TUU S9m0nUU UUU4SjnU b?U R T=(d SS9nU"US9 U R TUT=(d ST bT OSS9nOTS:XdTS:XdT[R;aTnO|TS:XaC[RRTS9nUR TS9nU"US9 UR T0S9nO3TS :Xa-SS/nU"US9 Ub[U5US'Ub[U5US'[UR55nU b#U R T=(d SS9n[U5n[UTT=(d SUU S9nUR5VVs0sH8upUcM [!U["[45(a[%U5S:dM6X_M: nnnU$s snnf)Nr_r`rr}rsr:aws_r sagemakerr@vertex_rZr[) nqualityrrstyler imageConfigrweb_search_options)rvrwrsc @>[TR55S:a[TR55nUH_n[RSLdTSLa+X ;a&TR US5 TR US5 MFX ;dMM[ SSUSTSTS3S9e T$g) NrTrz Setting `z` is not supported by , rrr)rrr%r`r}r_rxrvs r,r7get_optional_params_image_gen.._check_valid_arg s !&&( )A -*//12D''4/;$3F/&**1d3!%%a..0$'"+A3.DEXDYY[\a[bc`!a& % .r.rrrFrr_rrxrrr sampleCount aspectRatiorr)rrrLr5ryrrrFopenai_compatible_providersBedrockImageGenerationget_config_classrrrnrrr%rrr)r_rrrrrrrr`rsrr}r:rr%r&rwrrr config_classrrxrvs` ` ` @@r,get_optional_params_image_genr s+ HM   gt ,E'++,AB#''(94@O##M48K*../GN"&&x0N$$& <<   9 ,1D 1S  / !&9[&H  LL # ##{2#'77  a' " N1#%5 ')O&& "*FF+2G  *:;);;1++2'2'> E <  x' ' ) '"E"E E,  )55FFUFS 'CC%CP*:;&88129   +=  *:; =-0VOM *  -W.OM * $N$7$7$9:M"*FF+2G -. E'#/52#5 O$))++DA  ",Qt "="=Q! +  s J'(JJc ^^^^[[R[S5n [ 5n U R SS5mU R S5n U R SS5mU R SS5nU R SS5=(d /nU R SS5 S:UUUU4Sjjn [ RU U TUTS9mSn 0nTb?T[RR5;a[RT[T5S 9n U bjU RTS 9nU "US 9 U RT0TTbTOS S 9nU(a5UH/nU[;aMUU ;dMU UcMUU;dM'U UUU'M1 GOTS:XaeTb^ST;aXSTR!5;aDSU=(d /;a5["R$SLdTSLaTR SS5 O ['SSS9eTnGO.TS:Xa:U "TTSS9nU "US 9 ["R("5RT0TTbTOS S 9nGOTS:Xa<U "T=(d SSSS9nU "US 9 ["R*"5RT0S9nGOTS:Xa8U "T=(d SSSS9nU "US 9 ["R,RT0US9nGOnTS:XdTS:Xa7U "TSSS9nU "US 9 ["R."5RT0US9unnGO+TS:XaN["R0"5R5nU "US 9 ["R0"5RT0S9nGOTS :XaS!T;a["R2"5nOS"T;a["R4"5nOS#T;a["R6"5nO|S$T;dS%T;a["R8"5nOZS&T;dS'T;a["R:"5nO8S(TR=5;a["R>"5nO/nU "US 9 0UEnU$UR5nU "US 9 URT0S9nGOTS):Xa3U "TS)SS9nU "US 9 ["R@"5RT0S9nGOTS*:Xa:U "TS*SS9nU "US 9 ["RB"5RT0TTbTOS S 9nGO`TS+:XaU "TS+SS9nU "US 9 ["RDRGT5(a+["RD"5RT0TTbTOS S 9nOy["RHRKT5(a+["RH"5RT0TTbTOS S 9nO*["RL"5RT0TTbTOS S 9n0UEUEnU$TS,:Xa:U "TS,SS9nU "US 9 ["RN"5RT0TTbTOS S 9nGOUcg0nTR5HnX ;dM TUX'M U(a/[RSLdTbTSLag[STSUSTS3S9eg)NTr does not support parameters: , for model=zf. To drop these, set `litellm.drop_params=True` or for proxy: `litellm_settings: drop_params: true` r)rrFr}r)runsupported_paramsr%r`r}r_rxs r,r8get_optional_params_embeddings.._check_valid_argF s  # #((*A((:1(="%+ ""d*'K4,?, #233QRdQeeqrwqxycd r.)rvrr`rsr_rrrFrr_ztext-embedding-3 dimensionsTrzSetting dimensions is not supported for OpenAI `text-embedding-3` and later models. To drop it from the call, set `litellm.drop_params = True`.rtriton embeddings)r_r` request_type databricksrr nvidia_nim)rxrr:rZrY lm_studiorzamazon.titan-embed-text-v1zamazon.titan-embed-image-v1zamazon.titan-embed-text-v2:0zcohere.embed-multilingual-v3zcohere.embed-v4 twelvelabsmarengonovamistraljina_aivoyagesapinfinity fireworks_ai)rxrr_ sambanovaovhcloudollamarzSetting z is not supported by rrr extra_body)rOptional[list])2rFrGrHrIrrPreProcessNonDefaultParams(embedding_pre_process_non_default_paramsr^ _member_map_valuesrget_provider_embedding_configrrr%rrFr}rTritonEmbeddingConfigDatabricksEmbeddingConfignvidiaNimEmbeddingConfigVertexAITextEmbeddingConfigLmStudioEmbeddingConfigAmazonTitanG1Config&AmazonTitanMultimodalEmbeddingG1ConfigAmazonTitanV2ConfigBedrockCohereEmbeddingConfig TwelveLabsMarengoEmbeddingConfigr]AmazonNovaEmbeddingConfigMistralEmbeddingConfigJinaAIEmbeddingConfigVoyageContextualEmbeddingConfigis_contextualized_embeddingsVoyageMultimodalEmbeddingConfigis_multimodal_embeddingsVoyageEmbeddingConfigGenAIHubEmbeddingConfigInfinityEmbeddingConfigFireworksAIEmbeddingConfigSambaNovaEmbeddingConfigOVHCloudEmbeddingConfigrrnrrrr9)r_rencoding_formatrr`r}rsrr:rrvrrrrrparamobject final_paramsrr%rxs` `` @r,get_optional_params_embeddingsr* s0 #* H<# HM'++,A4H"&&x0N##M48K*../GN)--.EtLRPR3T:& #KK') 3#9 L 6:OO' <#<#<#C#C#E E/MM!"56N  "+:+V+V,W,  *:;);;1'2'> E <  )33]*%e,8_4-:5-AOE**  (  "%/ 2 7 7 99%:%@bA ""d*kT.A"&&|T:, #n-  (6 3%  *:;!779KK1'2'> E L   ,6+2 ,%  *:;!;;=OO12P   ,6+2 ,%  *:;!::LL12fM   +/Bh/N6 +%  *:;  / / 1 C C12f D     +  + + - I I K  *:;!99;MM12N   ) '5 0!557F *e 3CCEF +u 4002F +u 48IU8R99;F U "i5&8==?F u{{} $668F!  .> ?%f:L !==?*:; 22123   )6 )%  *:;!88:LL12M   )6 )%  *:;!779KK1'2'> E L   (6 (%  *:;  2 2 O OPU V V779KK'9$&/:/F E L  4 4 M Me T T779KK'9$&/:/F E L &;;=OO#5 "+6+BK PO 5/4V4   %6 %%  *:;!99;MM1'2'> E N   *6 *%  *:;!99;MM1'2'> E N 5/4V4   .6 .%  *:;!<<>PP12UQ   +6 +%  *:;!::<NN1'2'> E O   *6 *%  *:;!99;MM1'2'> E N   ( - -,>,B,B<,POL ) !&&( )A -##t+{d/B.3356A&**1d3- #&'9&::OPcOdebc . x' 7 * w'J'J J !&&( )A -##t+{d/B.3356A&**1d3- #&'9&::OPcOdebc 1O,B'#/9>>@A%zz*BDI L|#L,F(G1(Lt, r.c[U[5(a7SU;a USSLaUS UR5Hup[U5 M U$[U[5(aUHn[U5 M U$)a clean out 'additionalProperties = False'. Causes vertexai/gemini OpenAI API Schema errors - https://github.com/langchain-ai/langchainjs/issues/5240 Relevant Issues: https://github.com/BerriAI/litellm/issues/6136, https://github.com/BerriAI/litellm/issues/6088 additionalPropertiesF)r%rrrL_remove_additional_propertiesrnrrOrPitems r,rrs{ &$ !V +7M0NRW0W-.!,,.JC )% 0) M FD ! !D )$ / Mr.c[U[5(a/SU;aUS UR5Hup[U5 M U$[U[5(aUHn[U5 M U$)zq Relevant Issues: https://github.com/BerriAI/litellm/issues/6136, https://github.com/BerriAI/litellm/issues/6088 strict)r%rrrL_remove_strict_from_schemarnrs r,rrsl&$ v x !,,.JC &u -) M FD ! !D &t , Mr.c2US::aU$[U[5(aMURSS5 URSS5 UR5Hup#[ X1S- 5 M U$[U[ 5(aUHn[ XAS- 5 M U$)a Remove JSON schema reference fields like '$id' and '$schema' that can cause issues with some providers. These fields are used for schema validation but can cause problems when the schema references are not accessible to the provider's validation system. Args: schema: The schema object to clean (dict, list, or other) max_depth: Maximum recursion depth to prevent infinite loops (default: 10) Relevant Issues: Mistral API grammar validation fails when schema contains $id and $schema references rz$idNz$schemar)r%rrrrL_remove_json_schema_refsrn)r max_depthrOrPrs r,rrsA~ &$ 5$ 9d#!,,.JC $UM :) M FD ! !D $Tq= 9 Mr.c/nUc0$UR5HnX1;dM URU5 M UHnURUS5 M U$)z3 Remove unsupported params from non_default_params N)rrr)rxsupported_openai_params remove_keysrrOs r,_remove_unsupported_paramsrs] K& #((*  /   u %+sD) r.cpUR5VVs0sHupU[;dMX_M snn$s snnf)a Filter out LiteLLM internal parameters from kwargs dict. Returns a new dict containing only non-LiteLLM parameters that should be passed to external provider APIs. Args: kwargs: Dictionary that may contain LiteLLM internal parameters Returns: Dictionary with LiteLLM internal parameters filtered out Example: >>> kwargs = {"query": "test", "shared_session": session_obj, "metadata": {}} >>> filtered = filter_out_litellm_params(kwargs) >>> # filtered = {"query": "test"} )rLrq)r:rOrPs r,filter_out_litellm_paramsrs9(&,\\^ %3zssBT7T  ^  s22c~\rSrSr\SSj5r\SSSjj5rSrg) ric UR5HwupgUS:XaM URS5(aUS:waURS5(dMAUS:XaUS:waMOURS5(aUS:waUS:waMsXpU'My UR5VVs0sH]upgUS :wdM US :wdMUS :wdMUS :wdM%US :wdM-US:wdM5Xe;dM<Xd;dMCXtU:wdMM[XcS9SLdM[Xg_M_ nnnU$s snnf)Naws_bedrock_project_idrrrr@rrZr[r_r` api_versionr}rrsrrF)rLr5rt) rvrr`rsdefault_param_values#additional_endpoint_specific_paramsr%r&rxs r,#base_pre_process_non_default_params>PreProcessNonDefaultParams.base_pre_process_non_default_paramssk#((*DA,,||F###y0+66{CCo%*=*L Y''';6'+;; ! %+.&++- -W  ..  &  & 00 11 @ - a00 ' AD-  ("!) sB D/D7D?DDDDD%D/ D=Dc l[RUUUU[Vs0sHowS_M snS/S9nU$s snf)Nrrvrr`rsr r )rrr%) rvrr`rsr_remove_sensitive_keysadd_provider_specific_paramsr%rxs r,rCPreProcessNonDefaultParams.embedding_pre_process_non_default_paramsFsR ' J J+-$7'=7N%O7N!g7N%O5J%!%z!:*n<&'=>%G"&&'=tD.# UHoTR5;v M g7fr#r\)r(phraserOs r,r*2remove_sensitive_keys_from_dict..sI3H$3Hs")rr.rr)dsensitive_key_phrasesrrOs @r,r!r!s]FKvvx I3HI I I   s # c  Hr.cj0n[RnX$S;aUS:Xa#[R"5RXS9nOUS:Xa#[R"5RXS9nOWUS:XdUS:Xa#[R "5RXS9nO(US:Xa"[R "5RXS9nSU;d S U;dS U;Ga[US :XGaTUS :wGaMUS:wGaFUS:wGa?US :wGa8US:wGa1US:wGa*US:wGa#US:wGaUS:wGaUS:wGaUS:wGaUS:wGaUS:waUS:waUS:waUS:waUS:waUS:waUS:waUS:waUS:waUS:waUS:waU[R;aUS :XaZS US!'S"[lS U;a(URS 5US#'URS$S%5 U$SU;aURS5US#'U$[R(a'URS URSS%55US#'U$[S&S'US(3S)9eU$)*z-For .completion(), preprocess optional params providersrrrrZr[watsonx functions function_callrrrHanyscale together_airrcerebrasxai ai21_chat volcenginedeepseek codestralrrJ cohere_chatrI ollama_chatrKrLnebiuswandbrformatTfunctions_unsupported_modelrNrz%Function calling is not supported by r*r) rF!common_cloud_provider_auth_paramsAzureOpenAIConfigmap_special_auth_paramsAmazonBedrockGlobalConfigVertexAIConfigIBMWatsonXAIConfigradd_function_to_promptrr)rvrxr`rcommon_auth_dicts r,pre_process_optional_paramsrIsO@@{;;  ' )%779QQ#0RO!I -113KK'4L  ; ."&66%446NN#0OO!I -%88:RR#0SO )) 0 0 ( ( 8 +#'??#w.#{2#z1#}4#v-#|3#z1#u,#{2#|3#z1#{2#y0#{2#}4#x/#y0#}4#|3#'::#x/#w.#7+N+NN"h.,2).00*..w7$$AB'**%t& !!$66*..{;$$AB ..ASAWAW/33KFB =>  - #CDWCXXYZ r.c' "'^^^^^9^:[5R5m:T:RS5n(T:RSS5 Sn)TbCT[V*s/sHn*U*RPM sn*;a[ R T[T5U&S9n)[T:U(TU!TU)S9m9[T:T9TS9n+SLUUUUU9U:4Sjjn,[[R[S5n-U-"TTU&S9n.U.cU-"TS S 9n.U.=(d /n.U=(d /nU.RU5 U,"U.=(d /S 9 TS :XaA[R"5R!TT9U+Tb[#T[$5(aTOS S9n+GOgTS:Xa[R&"5R!TT9U+Tb[#T[$5(aTOS S9n+[R&"5R!TT9U+Tb[#T[$5(aTOS S9n+GOTS:XdTS:XaA[R("5R!T9U+TTb[#T[$5(aTOS S9n+GOTS:Xa,[R*"5R!T9U+TTbTOS S9n+GObTS:XaA[R,"5R!T9U+TTb[#T[$5(aTOS S9n+GOTS:XaA[R."5R!T9U+TTb[#T[$5(aTOS S9n+GOTS:XaA[R0"5R!T9U+TTb[#T[$5(aTOS S9n+GOTS:XaA[R2"5R!T9U+TTb[#T[$5(aTOS S9n+GOFTS:XaA[R4"5R!T9U+TTb[#T[$5(aTOS S9n+GOTS:XaT[R6;ddT[R8;dPT[R:;d<T[R<;d(T[R>;dT[R@;aA[RB"5R!T9U+TTb[#T[$5(aTOS S9n+GO@TS:XaA[RD"5R!T9U+TTb[#T[$5(aTOS S9n+G OTS:Xd TS:XaGST;aA[RB"5R!T9U+TTb[#T[$5(aTOS S9n+G O[RFRITTS 9(aA[RF"5R!TT9U+Tb[#T[$5(aTOS S9n+G OBTS:XGacT[RJ;aST;aA[RL"5R!TT9U+Tb[#T[$5(aTOS S9n+G O[RN"5R!TT9U+Tb[#T[$5(aTOS S9n+G OT[RP;aA[RR"5R!T9U+TTb[#T[$5(aTOS S9n+G OJU)b.U)R!T9U+TTb[#T[$5(aTOS S9n+G O[RT"5R!T9U+TTb[#T[$5(aTOS S9n+G OTS:XaA[RV"5R!T9U+TTb[#T[$5(aTOS S9n+G OTS:XGa[[R[S5n/U/RX"T5n0U/RZ"T5n1U0S :XdU0S!:XaA[R\"5R!TT9U+Tb[#T[$5(aTOS S9n+G OU0S :XaA[R^"5R!TT9U+Tb[#T[$5(aTOS S9n+G OS U1;aU0S":XaU1[R`Rc5;aA[R`"5R!T9U+TTb[#T[$5(aTOS S9n+G OB[Rd"5R!T9U+TTb[#T[$5(aTOS S9n+G OU)bCU)R!T9U+TTb[#T[$5(aTOS S9n+U0S#:XaU/Rf"T:U+S$9n+G OTS%:XaA[Rh"5R!TT9U+Tb[#T[$5(aTOS S9n+G OrTS&:XaA[Rj"5R!T9U+TTb[#T[$5(aTOS S9n+G O+TS':XaA[Rl"5R!TT9U+Tb[#T[$5(aTOS S9n+GOTS(:XaA[Rn"5R!T9U+TTb[#T[$5(aTOS S9n+GOTS):XaA[Rp"5R!T9U+TTb[#T[$5(aTOS S9n+GOVTS*:XaA[Rr"5R!T9U+TTb[#T[$5(aTOS S9n+GOTS+:Xa1U)b.U)R!T9U+TTb[#T[$5(aTOS S9n+GOTS,:XdTS:XaA[RN"5R!T9U+TTb[#T[$5(aTOS S9n+GOTS-:XaA[RL"5R!T9U+TTb[#T[$5(aTOS S9n+GODTS.:XaA[Rt"5R!T9U+TTb[#T[$5(aTOS S9n+GOTS/:XaA[Rv"5R!T9U+TTb[#T[$5(aTOS S9n+GOTS0:XaA[Rx"5R!TT9U+Tb[#T[$5(aTOS S9n+GOoTS1:XaA[Rz"5R!T9U+TTb[#T[$5(aTOS S9n+GO(TS2:Xa&[R|"5R!TT9U+S39n+GOTS4:XdTS5:XaA[R~"5R!T9U+TTb[#T[$5(aTOS S9n+GOTS6:XaA[R"5R!T9U+TTb[#T[$5(aTOS S9n+GOhTS7:XaA[R"5R!T9U+TTb[#T[$5(aTOS S9n+GO!TS8:XaA[R"5R!T9U+TTb[#T[$5(aTOS S9n+GOTS9:XaA[R"5R!T9U+TTb[#T[$5(aTOS S9n+GOTS::XaA[R"5R!T9U+TTb[#T[$5(aTOS S9n+GOLTS;:XaA[R"5R!T9U+TTb[#T[$5(aTOS S9n+GOTS<:XaA[R"5R!T9U+TTb[#T[$5(aTOS S9n+GOTS=:XaA[R"5R!T9U+TTb[#T[$5(aTOS S9n+GOwTS>:Xa[R"5R!T9U+TTb[#T[$5(aTOS S9n+T:R5H:n2[R"5RU25(dM.[S?U235e GOTS@:XaA[R"5R!T9U+TTb[#T[$5(aTOS S9n+GOTS :XaA[R"5R!T9U+TTb[#T[$5(aTOS S9n+GOTTSA:XaA[R"5R!T9U+TTb[#T[$5(aTOS S9n+GO TSB:XGaU&=(d Tn3[R"5RU3SC9(aA[R"5R!T9U+U3Tb[#T[$5(aTOS S9n+GO[RRU3SC9(aA[R"5R!T9U+U3Tb[#T[$5(aTOS S9n+GO0[R"SDRU[R[SE555 U=(d9 [R=(d" [SE5=(d [Rn[R"5R!T9U+U3UTb[#T[$5(aTOS SF9n+OoU)b-U)R!T9U+TTb[#T[$5(aTOS S9n+O?[R"5R!T9U+TTb[#T[$5(aTOS S9n+[U+T:T[[R"55U!SG9n+[SHU+35 [U+T9USI9n+U!(au[[R[SJ5n4[[R[SK5n5U!V6s/sHn6U4"U65(dMU6PM n7n6U7H n8U5"U+U85n+M U+$s sn*fs sn6f)MNr:rC)r_rrC)rvrr`rsr_r)rvrxr`c d>[R"STST35 [R"ST35 [R"ST35 0nTR5H;nX ;dM US:Xd US:XdUS:XaMUS:XaTS :XaM,US :XaM4TUX'M= U(a[R S LdTb/TS La*UR5HnTR US 5 M g [S TS[UR55STS[UR55S3S9eg )z Check if the params passed to completion() are supported by the provider Args: supported_params: List[str] - supported params from the litellm config z LiteLLM completion() model= z ; provider = z( LiteLLM: Params passed to completion() z4 LiteLLM: Non-Default params passed to completion() rrrrr max_retriesTNrrrz. To drop these, set `litellm.drop_params=True` or for proxy: `litellm_settings: drop_params: true` . If you want to use these params dynamically send allowed_openai_params=z in your request.r) rrrrrFr}rrrn) rrr%r`r}r_rrxrvs r,r-get_optional_params.._check_valid_arg^s ,UG=AT@U V  7  G  CDVCW X  #((*A(;!'7"71=8Q&-?q,A&)+ ""d*'K4,?+002A&**1d33- #233QRVWiWnWnWpRqQrr~@EFF|}ABTBYBYB[}\|]]no r.r)r_r`rCr_r^rrJF)r_rxrr}anthropic_textr;rIrrmaritalkr predibase huggingfacer4rZrYr[r:rrrconverse converse_likeinvokeclaude_platform)rvr cloudflarerr<rNpetals deepinfra perplexityrtext-completion-codestraltext-completion-inceptionrrr5r6)r_rxrr7rMrr8 hosted_vllmvllmrbedrock_mantler9rKr0zLiteLLM now defaults to Watsonx's `/text/chat` endpoint. Please use the `watsonx_text` provider instead, to call the `/text/generation` endpoint. Param: watsonx_textr=rrzoAzure optional params - api_version: api_version={}, litellm.api_version={}, os.environ['AZURE_API_VERSION']={}AZURE_API_VERSION)rxrr_r r}rz Final returned optional params: )rrxrrr)rr)`rrSrr^rPrget_provider_chat_configr$rIrFrGrHrIrTrFAnthropicConfigrr%rAnthropicTextConfigCohereChatConfig TritonConfigMaritalkConfigReplicateConfigPredibaseConfigHuggingFaceChatConfigTogetherAIConfigr_r`r]r^vertex_language_modelsvertex_vision_modelsVertexGeminiConfigGoogleAIStudioGeminiConfigVertexAIAnthropicConfigis_supported_modelvertex_mistral_modelsCodestralTextCompletionConfig MistralConfigvertex_ai_ai21_modelsVertexAIAi21ConfigVertexAILlama3ConfigSagemakerConfigget_bedrock_routeget_base_modelAmazonConverseConfigAmazonBedrockOpenAIConfigAmazonAnthropicConfig get_legacy_anthropic_model_namesAmazonAnthropicClaudeConfigmap_claude_platform_auth_paramsCloudflareChatConfig OllamaConfigOllamaChatConfigNLPCloudConfig PetalsConfigDeepInfraConfigInceptionTextCompletionConfigDatabricksConfigNvidiaNimConfigCerebrasConfig XAIChatConfigAI21ChatConfigFireworksAIConfigVolcEngineConfigHostedVLLMChatConfig VLLMConfigGroqChatConfigBedrockMantleChatConfigDeepSeekChatConfigOpenrouterConfigIBMWatsonXChatConfigrrFis_watsonx_text_paramr OpenAIConfig NebiusConfigAzureOpenAIO1Configis_o_series_modelAzureOpenAIGPT5Configis_model_gpt_5_modelrrr?r rAZURE_DEFAULT_API_VERSIONrBOpenAILikeChatConfigrrnrr_apply_openai_param_overrides);r_r1r2rtop_prrrstopr?max_completion_tokens modalities predictionaudiopresence_penaltyfrequency_penalty logit_biasrr`rseedrrrLlogprobs top_logprobs extra_headersr parallel_tool_callsr}rreasoning_effort verbosityrsr{thinkingrsafety_identifierrCr:rrrrrrrr bedrock_routebedrock_base_modelr_azure_detection_modelrrrC nested_pathspathrxrvs;` ` ` ` @@r,get_optional_paramsrsFXHMMOM"&&x0NlD),0O&+>'3C'38|C,0HH!"56!I  8#%/5' 2#-/O ))V#* H<#3)<6X (-217R12)/Rk)!113EE1+*z+t/L/LF    0 0!557II1+*z+t/L/LJ  "557II1+*z+t/L/LJ    -1D1P!224FF1+*z+t/L/LG    (!..0BB1+'2'> E C   *!002DD1+*z+t/L/LE    +!113EE1+*z+t/L/LF    +!113EE1+*z+t/L/LF    -!779KK1+*z+t/L/LL    -!224FF1+*z+t/L/LG    + +++ G33 3 G.. . G33 3 G22 2 G00 0!446HH1+*z+t/L/LI    (!<<>PP1+*z+t/L/LQ    0 0{*x5/@!446HH1+*z+t/L/LI    ( ( ; ;)< < "99;MM1+*z+t/L/LN    + G11 1e#99;MM#+=(7 +6:kSW;X;X(!&N  #*"7"7"9"K"K'9$3'2z+t7T7T$"#L #g33 3%88:LL#5 /#.:k43P3P M O (-??#5 /#.:k43P3P @ O&::<NN#5 /#.:k43P3P O O  +!113EE1+*z+t/L/LF    )"3;;x#8:LM(::5A -<.J$1$4JM*o11,?FF02 -"!,/""  &1!3 8 8 :( :6AD :%($ (:$)0 H%'C* &-G/-OL ) ##%A%-*:*F%%2%5" & -(s > E EchU(a*UH$nX0;aM X1;aMURU5X'M& U$)a; If user passes in allowed_openai_params, apply them to optional_params These params will get passed as is to the LLM API since the user opted in to passing them in the request Only params the caller actually sent are forwarded. Previously this function unconditionally wrote `None` for any allowed param missing from the request, which then reached the provider SDK as a top-level kwarg it did not recognize (e.g. openai SDK raising `AsyncCompletions.create() got an unexpected keyword argument 'enable_thinking'`). See https://github.com/BerriAI/litellm/issues/25697 )r)rrxrrs r,rrs>*E'.%7%;%;E%BO " + r.cUR5VVs0sH2upUS:wdM US:wdMU[;dM!U[U:wdM0X_M4 nnnU$s snnf)Nr_r`)rLr)rvr%r&rxs r,get_non_default_paramsrs|"'') )DA L ** 99 9!<< )   s AAA AAc^UbU$UbU$Ub[US- S-5nUS:XaSnU$UbU$g)a Returns the max parallel requests to send to a deployment. Used in semaphore for async requests on router. Parameters: - max_parallel_requests - Optional[int] - max_parallel_requests allowed for that deployment - rpm - Optional[int] - requests per minute allowed for that deployment - tpm - Optional[int] - tokens per minute allowed for that deployment - default_max_parallel_requests - Optional[int] - default_max_parallel_requests allowed for any deployment Returns: - int or None (if all params are None) Order: max_parallel_requests > rpm > tpm / 6 (azure formula) > default max_parallel_requests Azure RPM formula: 6 rpm per 1000 TPM https://learn.microsoft.com/en-us/azure/ai-services/openai/quotas-limits Nrr)r)max_parallel_requestsrpmtpmdefault_max_parallel_requestscalculated_rpms r,calculate_max_parallel_requestsrsS:($$  S4Z!^, Q N & 2,, r.cURS05RS5nUc!URS05RS5nU$)z Returns the routing order for a deployment. Checks litellm_params first (static config), then model_info (dynamic/team models added via API where order lives in model_info, not litellm_params). rorderrr8) deploymentrs r,_get_deployment_orderrsE NN+R 0 4 4W =E }|R044W= Lr.cHUb.UVs/sHn[U5U:XdMUPM nnU(aU$U$UVVs/sHn[U5nUcMUPM nnnU(a [U5OSnUb%UVs/sHn[U5U:XdMUPM nnU$U$s snfs snnfs snfr#)rmin) healthy_deployments target_orderr,filteredro _valid_orders min_orderfiltered_deploymentss r,_get_order_filtered_deploymentsrs* *!.CA.F,.VA*   O"" . -J' 3A  -  6Cs=1I2 1 $Z0I= 1  $# 3   s!BBB B1BBcU(dU$[U5nUVs/sH4nURS5=(d 0RS5U;dM2UPM6 sn$s snf)a Filter out deployments whose `model_info.id` appears in `excluded_deployment_ids`. Used by weighted-routing failover so a single logical request can re-pick across the remaining deployments in the same model group after one of them has failed. If the filter would leave no deployments, an empty list is returned so the caller raises its usual no-deployments error and the weighted-failover helper falls through to the cross-group fallback path. Returning the original unfiltered list here would re-include the just-failed deployment. rrm)rr9)rexcluded_deployment_ids excluded_setr,s r,"_get_excluded_filtered_deploymentsrs` #""./L% $A EE,  %2 * *4 0 D $  s 1AAcvUS:XamUR=(d4 [R=(d [S5=(d [S5nUb[U[5(aU$UR$US:XaUR nUbU$UR$US:XaUR nUbU$UR$)z5 Return the region for a model, for a given provider rZVERTEXAI_LOCATIONVERTEX_LOCATIONrr0)vertex_locationrFrr%rpaws_region_namewatsonx_region_name region_name)r`rvertex_ai_locationrrs r,_get_model_regionr6s k)  * * -&& --. -+,   )j9KS.Q.Q% %  % %%  )(88  &" " '  % %%  ),@@  *& &  % %%r.c[R"URUS9up n[X S9nUc0[R "SR UR55 gUS:XaG[R"5R5n[R"5R5nOUS:XaG[R"5R5n[R"5R5nOUS:XaG[R"5R5n[R"5R5nOQUS:XaG[R"5R5n[R"5R5nO/n/nUHnXtR5;dM g UHnXtR5;dM g g) zq Infer if a model is in the EU or US region Returns: - str (region) - "eu" or "us" - None (if region not found) )r_r)r`rNz'Cannot infer model region for model: {}rrZrr0euus)rFrr_rrrr?rBget_eu_regionsget_us_regionsrErDrFr])rr_r`r model_region eu_regions us_regionsregions r,_infer_model_regionrQs(/'?'?"">($E1%/L 5 < <^=Q=Q R g%..0??A ..0??A  +++-<<> ++-<<>  )668GGI 668GGI  )//1@@B //1@@B    '') ) '') ) r.cLURS:Xag[US9nUbUS:Xagg)0 Return true/false if a deployment is in the EU rTrFrrrrs r, _is_region_eur4!!T)'nELLD$8 r.cLURS:Xag[US9nUbUS:Xagg)z0 Return true/false if a deployment is in the US rTrFrrs r, _is_region_usrrr.c&URU:Xagg)rTF)r)rallowed_model_regions r,is_region_allowedrs !!%99 r.c SUR;a[UR[5(a[UR[5(aURR SS5n[ R"5RUURURUR=(d [ RSU=(d SS9nURSS5nU$g) zE Pass the litellm params for an azure model, and get back the region rzazure/r chat)r_api_keyrr rmodez x-ms-regionN) r_r%rrprreplacerFAzureChatCompletion get_headersr rr9)rr_modelrrs r,get_model_regionrs >''' ~--s 3 3 ~.. 4 4%%--h; 446BB"**#,,&22Wg6W6W C !) ]D A r.cfURS5n[U5SSnU$![a gf=f)Nr{dr)r9rpr)r: _messagess r,get_first_chars_messagesrs=JJz*  N4C(  s # 00cHSRSU55n[U5$)Nrc3R# UHoR5(aMUv M g7fr#)isspace)r(chars r,r*$_count_characters..sHTTDDTs' ')r r)r filtered_texts r,_count_charactersr s!GGHTHHM } r.cn[US5(a[US5(a[USS5nU(a[US5(aURn/nUHn[US5(aCURH1n[US5(dMUR UR 5 M3 MW[US5(dMjUR UR 5 M SR U5$[US5(aP[US5(a?[USS5nSU;dS U;a&[USS5n[U[5(aU$S$URn/n UHn [U [5(aIU RRb0U R [U RR55 M_Ma[U [5(dMxU RRcMU R [U RR55 M SR U 5$) Nr&routputrrrdeltaz text.deltazoutput_text.delta)rorFr rrrr r%rprrUrrlr) response_objresponses_api_response output_listresponse_output_parts output_item content_part event_typer_choicesresponse_partschoices r,get_response_stringrs|V$$z)J)J")z4!H !g.Dh&O&O077K02 !* ; 22(3(;(; "<881889J9JK)<[&11)001A1AB +7701 1|V$$w)G)G\626 : %)< )JL'26E&uc225 : :>J=Q=QH!#N fg & &~~%%1%%c&..*@*@&AB2  0 1 1||##/%%c&,,*>*>&?@  77> ""r.cDU=(d [RnUS:XdUS:Xa-U=(d" [R=(d [S5nU$US:XdUS:Xa-U=(d" [R=(d [S5nU$US:Xa-U=(d" [R =(d [S5nU$US :Xa-U=(d" [R =(d [S 5nU$US :Xa-U=(d" [R=(d [S 5nU$US :XdUS:Xa-U=(d" [R=(d [S5nU$US:Xa-U=(d" [R=(d [S5nU$US:Xa-U=(d" [R=(d [S5nU$US:Xa-U=(d" [R=(d [S5nU$US:Xa?U=(d4 [R=(d [S5=(d [S5nU$US:Xa-U=(d" [R=(d [S5nU$US:Xa+U=(d" [R=(d [S5nU$)Nr_rHOPENAI_API_KEYrJrNANTHROPIC_API_KEYrM AI211_API_KEYrOALEPH_ALPHA_API_KEYbasetenBASETEN_API_KEYrIr;COHERE_API_KEYrQHUGGINGFACE_API_KEYrNNLP_CLOUD_API_KEYrREPLICATE_API_KEYr4TOGETHERAI_API_KEYTOGETHER_AI_TOKENr=NEBIUS_API_KEYr> WANDB_API_KEY)rFr openai_keyr anthropic_keyai21_keyaleph_alpha_key baseten_key cohere_keyhuggingface_key nlp_cloud_key replicate_keytogetherai_api_key nebius_key wandb_key) llm_providerdynamic_api_keyrs r, get_api_keyr7sW0Gx<3K#KOW//O:>N3OV NS  $ 8H(HUW22UjAT6UP NM  LW--LO1LJ NG  &  Sw.. S*=R2S D N=  "QW00QJ?P4Q: N7  !\]%BOW//O:>N3O4 N1  &  Sw.. S*=R2S . N'  $UW22UjAT6U$ N!  $UW22UjAT6U N  &  /)) /./ /-.  N  !OW//O:>N3O N  MW..M*_2M Nr.cSSKnSSKJn [US5(aUR"UR5$UR"5$)NrrPUTC)rProrQr:utcnow)dtrPs r,get_utc_datetimer=0s5!r5||BFF##  r.cSnU[R;aZS[RU;a[RUS$S[RU;a[RUS$[[R[ S5nU"US9up nUS:XaU"US9nU$U[R;a\S[RU;a[RUS$S[RU;a[RUS$g[ 5e![ a [ S US 35ef=f) a- Get the maximum number of output tokens allowed for a given model. Parameters: model (str): The name of the model. Returns: int: The maximum number of tokens allowed for the given model. Raises: Exception: If the model is not mapped yet. Example: >>> get_max_tokens("gpt-4") 8192 cSUS3n[RRU5nUR5 UR 5nURS5nUbU$g![ a gf=fNzhttps://huggingface.co/z/raw/main/config.jsonmax_position_embeddingsrFmodule_level_clientr9raise_for_statusrr model_name config_urlr config_jsonrAs r,_get_max_position_embeddings4get_max_tokens.._get_max_position_embeddingsLsv.zl:OP  2266zBH  % % '#--/K&1oo6O&P #&2..  AA A+*A+max_output_tokensr?rrrQrFNzModel zr isn't mapped yet. Add it here - https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json)rFrrFrGrHrIr)r_rIrr`rr?s r,get_max_tokensrN:sQ$& G&& &"g&8&8&??))%01DEE!3!3E!::))%0>>"3;;x#8:LM+;%+H(Aq - /5GJ  G&& &"g&8&8&??))%01DEE!3!3E!::))%0>>;+   UGN O   s1AD&,D&38D&,AD&-,D&D& D&&Ec2[R"SSU5$)Nz-\d+$rresubrMs r,_strip_stable_vertex_versionrSxs 66(B ++r.c2SSKJn UR"U5$)z Get the base model from the given model name. Handle model names like - "us.meta.llama3-2-11b-instruct-v1:0" -> "meta.llama3-2-11b-instruct-v1" AND "meta.llama3-2-11b-instruct-v1:0" -> "meta.llama3-2-11b-instruct-v1" rr)!litellm.llms.bedrock.common_utilsrry)rFrs r,_get_base_bedrock_modelrV|sC  * *: 66r.c2[R"SSU5$)z Strips the organization, custom suffix, and ID from an OpenAI fine-tuned model name. input: ft:gpt-3.5-turbo:my-org:custom_suffix:id output: ft:gpt-3.5-turbo Args: model_name (str): The full model name Returns: str: The stripped model name z (:[^:]+){3}$rrPrMs r,!_strip_openai_finetune_model_namerXs 66/2z 22r.cU(aUS;a [US9nU$U(aUS:XdUS:Xa [US9nU$U(aUS:Xa [US9nU$SU;a [US9nU$U$)N)rbedrock_converserMrZrYrzft:)rVrSrX)r_r`stripped_bedrock_model strip_versionstrip_finetunes r,r6r6sy26UU!8E!J%% {*.AX.M4F  "5"E4F  %:eL r._model_cost_lowercase_map_model_cost_mutation_generationc[$r#)r_rr.r,"get_model_cost_mutation_generationras **r.cjSq[S- q[R5 [R5 g)zInvalidate the case-insensitive lookup map for model_cost. Call this whenever litellm.model_cost is modified to ensure the map is rebuilt. Also clears related LRU caches that depend on model_cost data. Nr)r^r__cached_get_model_info cache_clear_cached_get_model_info_helperrr.r,rWrWs/!%#q(#&&(!--/r.cv[RVs0sHoR5U_M snq[$s snf)zRebuild the case-insensitive lookup map from the current model_cost. Returns: The rebuilt map (guaranteed to be not None). )rFrr]r^)r%s r,!_rebuild_model_cost_lowercase_maprgs58?7I7I J7I!A7I J $$!Ks6ct[5q[RU5nUbU[R;aU$g)z Handle stale _model_cost_lowercase_map entry (key was popped). Rebuilds the map and retries the lookup. Returns: The matched key if found after rebuild, None otherwise. N)rgr^r9rFr)potential_key_lower matched_keys r,_handle_stale_map_entry_rebuildrks;!B C+//0CDK;'2D2D#D r.cv[RH%nUR5U:XdM[5qUs $ g)z Handle new key added to model_cost without invalidating _model_cost_lowercase_map. Scans model_cost for case-insensitive match and rebuilds the map if found. Returns: The matched key if found, None otherwise. N)rFrr]rgr^)rirOs r,_handle_new_key_with_scanrms6!! 99;- -(I(K %J" r.cU[R;aU$[c [5qUR 5n[R U5nUbU[R;aU$Ub[ U5nUbU$g)a Get the actual key from model_cost, with case-insensitive fallback. WARNING: Only O(1) lookup operations are acceptable. O(n) lookups will cause severe CPU overhead. This function is called frequently during router operations. ALLOWED HELPER FUNCTIONS (conditionally called, O(n) operations are acceptable): - _rebuild_model_cost_lowercase_map: Rebuilds the lookup map (only when map is None) - _handle_stale_map_entry_rebuild: Rebuilds map when stale entry detected (rare case) If you need to add a new helper function with O(n) operations that is conditionally called and confirmed not to cause performance issues, add it to the allowed_helpers list in: tests/code_coverage_tests/check_get_model_cost_key_performance.py N)rFrr^rgr]r9rk) potential_keyrirjs r,rrs$***!($E$G!'--/+//0CDK;'2D2D#D56IJ  "  r.c([RU$r#)rFrrOs r,_get_model_info_from_model_costrr"s   c ""r.cfU(aURS5bUSU:waUS:XaUSRS5(agUS:XaUSRS5(agURS5(aUSRS5(agUS:XagUS:Xa USS;agUS :Xagg g) a Check if the model info provider matches the custom provider. A missing ``litellm_provider`` key and a ``litellm_provider`` set to ``None`` both mean "no specific provider constraint" and are treated as a wildcard match. ``register_model`` may persist ``None`` here via ``get_model_info`` when a deployment is registered without a provider, so normalising the two cases keeps custom pricing applied consistently. r3rZTrr litellm_proxyazure_ai)rr_githubF)r9r5rr`s r,_check_provider_matchrx&s)*6 ) *.A A + -* 3 *[ !3" N 2z 8 *^ $8% + +I 6 6: < *Y <  ? 2 J .:>P3QV 4  H , r.) TypedDictcH\rSrSr%S\S'S\S'S\S'S\S'S\S'Srg ) 'PotentialModelNamesAndCustomLLMProvideriXrp split_modelcombined_model_namestripped_model_namecombined_stripped_model_namer`rN)rIrrr__annotations__rrr.r,r{r{Xs "%%r.r{c Uc:[[R[S5nU"US9up1 nUn[ XS9nUnOU(aKUR US-5(a2URSS5SnUn[ X1S9nSRX5nO.UnSRX5n[ XS9nSRUU5n[UUUU[[U5S9$![a UnNf=f)Nrrr^r1rz{}/{})r|r}r~rr`) rFrGrHrIrr6r5rdr?r{rrp)r_r`rr|rr}r~rs r,_get_potential_model_namesr`s)" &s{{8'<>PQ 5EE5R 2Ka$/ (;$ !1!1c!""kk#q)!, #/ (/~~ ( $ %nn-@H/ (/~~  ( $ 3//%A &9:  ; K s+C C('C(cSUS3n[RRU5nUR5 UR 5nURS5nUbU$g![ a gf=fr@rBrEs r,rIrIsx*:,6KLJ..22:>!!#mmo #.//2K"L " .* * rKc[UUUS9$)zS _get_model_info_helper wrapped with lru_cache Speed Optimization to hit high RPS r_r`r)rrs r,reres "/ r.cSnU(a'U[;a[RU[U5S9nSnU(aUR US9nU$)Nrr)r_rget_provider_model_infor^r)r_r`rrs r,rrsX 37O2oE/GG,/B"CH 7;J$66U6C r.cB[SUR555$)zL Check if the potential model name is in the model cost (case-insensitive). c3N# UHn[[U55SLv M g7fr#)rrp)r(potential_model_names r,r*9_is_potential_model_name_in_model_cost..s($B  C 456dB$Bs#%)r.r)potential_model_namess r,&_is_potential_model_name_in_model_costrs' $9$@$@$B r.c p0[RE[REnX;aX@nUbUS:XaSnUb\US:XaVSU-[R;aSU-nO9US-[R;aUS-nOUS-[R ;aUS-n[ XS9n[R"SU35 USnUS nUS nUS n US nUn Sn U(a'U[;a[RU[U5S 9n U b,[U SS5n [U 5(aU "UUUS9n U bU $US:XaB[#US9n[%Sw0SU_SU_SS_SS_SS_SS_SS_SS_SS_SS_SS_S S_S!S_S"S_S#S_S$S_6$SnSn['U5nUb*Un[)[+[,U5S%9n[/UU S&9(dSnUc8['U5nUb*Un[)[+[,U5S%9n[/UU S&9(dSnUc8['U5nUb*Un[)[+[,U5S%9n[/UU S&9(dSnUc8['U5nUb*Un[)[+[,U5S%9n[/UU S&9(dSnUc8['U 5nUb*Un[)[+[,U5S%9n[/UU S&9(dSnUbUc [1S'5eUR3S5nUc'[R"S(R5X55 SnUR3S5nUc'[R"S)R5X55 Sn[%Sw0SU_SUR3SS5_SUR3SS5_SUR3SS5_SU_S*UR3S*S5_S+UR3S+S5_S,UR3S,S5_S-UR3S-S5_S.UR3S.S5_S/UR3S/S5_S0UR3S0S5_S1UR3S1S5_S2UR3S2S5_S3UR3S3S5_S4UR3S4S5_S5UR3S5S5_S6UR3S6S5_S7UR3S7S5_S8UR3S8S5_S9UR3S9S5_S:UR3S:S5_S;UR3S;S5_SUR3S>S5_S?UR3S?S5_S@UR3S@S5_SAUR3SAS5_SBUR3SBS5_SCUR3SCS5_SDUR3SDS5_SEUR3SE5_SFUR3SF5_SU_SGUR3SGS5_SHUR3SHS5_SIUR3SIS5_SJUR3SJS5_SKUR3SKS5_SLUR3SLS5_SMUR3SMS5_SNUR3SNS5_SOUR3SOS5_SPUR3SPS5_SQUR3SQS5_SRUR3SRS5_SSUR3SSS5_STUR3STS5_SUUR3SUS5_SVUR3SVS5_SWUR3SWS5_SXUR3SXS5_SYUR3SYS5_SZUR3SZS5_S[UR3S[S5_S\UR3S\S5_SUR3SU5_SUR3S5_SUR3SS5_SUR3SS5_S]UR3S]S5_SUR3SS5_S UR3S S5_S!UR3S!S5_S"UR3S"S5_S^UR3S^S5_S_UR3S_S5_S$UR3S$S5_S`UR3S`S5_SaUR3SaS5_SbUR3SbS5_ScUR3ScS5_SdUR3SdS5_SeUR3SeS5_SfUR3SfS5_SgUR3SgS5_ShUR3ShS5_SiUR3SiS5_SjUR3SjS5_SkUR3SkS5_S#UR3S#S5_SlUR3SlS5_SmUR3SmS5_SnUR3SnS5_SoUR3SoS5_SpUR3SpS5_SqUR3SqS5_SrUR3SrS5_SsUR3SsS5_StUR3StS5_6$![a$n[R "SUUU5 SnAGNSnAff=f![a8n[R"SuU35 [SvR5X55eSnAff=f)xzq Helper for 'get_model_info'. Separated out to avoid infinite loop caused by returning 'supported_openai_param's Nr[rZzmeta/z@latestr^z6checking potential_model_names in litellm.model_cost: r}r~rr|r`rrT)r_rrzcCould not get dynamic model info for model=%s, provider=%s; falling back to the static cost map: %srQrMrOr?max_input_tokensrLrFrrGr3rrrrrrsupports_assistant_prefillrrr rqrwz|This model isn't mapped yet. Add it here - https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.jsonz`model={}, custom_llm_provider={} has no input_cost_per_token in model_cost_map. Defaulting to 0.zamodel={}, custom_llm_provider={} has no output_cost_per_token in model_cost_map. Defaulting to 0.input_cost_per_token_flexinput_cost_per_token_priorityr+r-r.r/6cache_read_input_token_cost_above_200k_tokens_priority-cache_read_input_token_cost_above_272k_tokens6cache_read_input_token_cost_above_272k_tokens_priority-cache_read_input_token_cost_above_512k_tokens cache_read_input_token_cost_flex$cache_read_input_token_cost_priorityr,input_cost_per_character&input_cost_per_token_above_128k_tokens&input_cost_per_token_above_200k_tokens/input_cost_per_token_above_200k_tokens_priority&input_cost_per_token_above_272k_tokens/input_cost_per_token_above_272k_tokens_priority&input_cost_per_token_above_512k_tokensinput_cost_per_queryinput_cost_per_secondinput_cost_per_audio_tokeninput_cost_per_image_tokeninput_cost_per_imageinput_cost_per_audio_per_secondinput_cost_per_video_per_secondinput_cost_per_token_batchesoutput_cost_per_token_batchesoutput_cost_per_token_flexoutput_cost_per_token_priority(regional_processing_uplift_multiplier_eu(regional_processing_uplift_multiplier_usoutput_cost_per_audio_tokenoutput_cost_per_characteroutput_cost_per_reasoning_token'output_cost_per_token_above_128k_tokens+output_cost_per_character_above_128k_tokens'output_cost_per_token_above_200k_tokens0output_cost_per_token_above_200k_tokens_priority'output_cost_per_token_above_272k_tokens0output_cost_per_token_above_272k_tokens_priority'output_cost_per_token_above_512k_tokensoutput_cost_per_secondoutput_cost_per_second_1080p output_cost_per_video_per_secondoutput_cost_per_imageoutput_cost_per_image_tokenoutput_vector_sizecitation_cost_per_tokentiered_pricingrr rrrrrrrsupports_none_reasoning_effort!supports_minimal_reasoning_effortsupports_low_reasoning_effortsupports_xhigh_reasoning_effortsupports_max_reasoning_effort$bedrock_output_config_effort_ceilingsearch_context_cost_per_queryrrocr_cost_per_pageocr_cost_per_creditannotation_cost_per_pageprovider_specific_entryuses_embed_contentsupports_image_sizezError getting model info: zThis model isn't mapped yet. model={}, custom_llm_provider={}. Add it here - https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json.r)rF azure_llmsazure_embedding_modelsvertex_llama3_modelsrqrtrrrr_rrr^rFcallablerrrIrcrrrrrprxrr9r?)r_r`rrrrr}r~rr|model_cost_custom_llm_providerrprovider_get_model_infoprovider_model_inforr? _model_inforO _matched_key_input_cost_per_token_output_cost_per_tokens r,rrsG M**Mg.L.LM  %E  */BFV/V"-   */Bk/Q'">">>%"g&C&CC )"g&C&CC ) :!  DEZD[ \ 44IJ34IJ'< *( $,M: 34IJ)<&6: #6/#I3KKl3F&GLO  &&-o?OQU&V #/00*A#!) '+' +6227 - /5GJ %"&#'  &'  '( "/*.*.+/&*,0)-'+ $(! & 59K!%C//BCL'"=$sC.Q ,*(F#'K"259 +&C"Ad3PSn"UK0#.,J'+ "23OP +&C"Ad3PSn"UK0#.,J'+ "23FG +&C"Ad3PSn"UK0#.,J'+ "2;? +&C"Ad3PSn"UK0#.,J'+ "ck S6A__&6 !%,$$v}} )*%6Aoo'7 "&-$$w~~ *+& XX&??<>X"-1CT!JX#.//2Et"L X &; X +6///+ X/:oo3T/X1<5t1XCN//GCX$-8OO14-%X*?JooCT?+X0HSLdH1X6?JooCT?7X<HSLdH=XB?JooCT?CXH2=62IXN6A__:D6OXT;F//?;UXZ*5.*[X`8C:}XB:E>:CXH-8OO14-IXN+6///+OXT1<5t1UXZ9D=t9[X`=HOOA4=aXf9D=t9gXlBMFBmXr9D=t9sXxBMFByX~9D=t9XD(37OQU'VEXF.9__2D.GXL2=62MXR'2oo6Mt&TSXT-8OO14-UXZ$/??3G#N[X\)4-t)]Xb +/?FcXd"-&(;"eXj!__V,kXl*5.*mXr*5.*sXx!,0A4 HyXz+6///+{X@&1__5KT%RAXB,7??0$,CXH)4-t)IXN&1__5KT%ROXP'2oo6Mt&TQXR$/??3G#NSXT0;4d0UXZ+6///+[X`3>//73aXf%0OO4I4$PgXh&1__5KT%RiXj$/??3G#NkXl0;4d0mXr3>//73sXx/:oo3T/yX~1<5t1XD/:oo3T/EXJ6A__:D6KXP'2oo6Mt&TQXR/:oo3T/SXX OOE40YXZ OOE40[X\#.//2Et"L]X^%0OO4I4$P_X`*5.*aXf)4-t)gXl$/??3G#NmXn%0OO4I4$PoXX s!"**B+ d  9!=> n u u    sJD"g3% g2Ag3;ag3 g0 g+%g3+g00g33 h5=3h00h5c[R"XS9n[UUUUS9n[XS9nU(a"UR 5HupxUcM XU'M [ S0UDSU0D6$)Nr^r_r`rrrr)rFrrrrLrb) r_r`rrrrrrOrPs r,_build_model_inforjs} &AA)/ K&M'--/JC #(C 0  T{ T>> get_model_info("gpt-4") { "max_tokens": 8192, "input_cost_per_token": 0.00003, "output_cost_per_token": 0.00006, "litellm_provider": "openai", "mode": "chat", "supported_openai_params": ["temperature", "max_tokens", "top_p", "frequency_penalty", "presence_penalty"] } )rrcrs r,rTrTs%\ XOO !%h GGr.c[RS[RS[RS[RS[ RS[ RSSS0nURUS5$) zConverts standard python types to json schema types Parameters ---------- python_type_name : str __name__ of type Returns ------- str a standard JSON schema type, "string" if not recognized. stringintegernumberbooleanarrayrNoneTypenull)rprIrrrrnrrr9)python_type_namepython_to_json_schema_typess r,json_schema_typers] h i  y w xF# ' * *+;X FFr.c SSKnSSKJn SSKJn UR nUR"U5nU"U5nSRUSVs/sHoR5PM sn5n 0n /n UR"U5Rn U R5GHxup[US5(a [URR 5nOSnSnSnUSHnUR U :XdM[US 5(aUUR"nS U;aUR%S 5SnO#S U;a['[)U"U555nS n[U5nSRUR*Vs/sHoR5PM sn5nM UUUS.n[-UR5VVs/sH!unn[/U[&5(dMUU4PM# snn5X'UR0UR2:XdGMgU R5U 5 GM{ UU SU S.S.nU (aU USS'U$![ anUeSnAff=fs snf![ a GN f=fs snfs snnf)a<Using type hints and numpy-styled docstring, produce a dictionary usable for OpenAI function calling Parameters ---------- input_function : function A function with a numpy-style docstring Returns ------- dictionnary A dictionnary to add to the list passed to `functions` parameter of `litellm.completion` rN) literal_eval)NumpyDocString Summary annotation Parametersr&optional,{r)r& descriptionenumrr& propertiesnamerrrrequired)inspectastrnumpydoc.docscraperrrIgetdocr strip signaturerrLrorrrr&rdrprndescrrr%defaultemptyr)input_functionrrrrr docstringnumpydocsrrrequired_params param_info param_namer param_typeparam_description param_enum param_data param_dictr%r&rs r,function_to_dictr sR$5  " "D~.Ii(H))0CD0C1WWY0CDEKJO"">2==J'--/ 5, ' ')%*:*:*C*CDJJ  #<0J*,:v..!+J!Z/%/%5%5c%:1%=  *!),T,z2J-K)LJ)1J"2*!=J$(II*//.R/Qwwy/.R$S!!1&, "& * 0 0 2 I 21jC6HVaV 2 I"  ==EKK '  " ": .O0V"$ F+:|Z( MI  E: )! !/S JsAH1 I0I 0I 2I I 1 I;H==I IIcb[R"U5nURUS9n[U5$)Nr)r URL copy_withrp) original_urlnew_pathrA modified_urls r, modify_urlr`s+ ))L !C==h=/L | r.cSnSnU(aUnU(aUn[U5Vs/sH nSUS./PM n n[R"5n [R"UU UUUS9 [R"5n X- n U SSSS.$s snf![a*n [R"5n X- n U SSU S.sSn A $Sn A ff=f) NzHey, how's it goingrrrir)r_r{r`r force_timeoutr )total_response_time calls_madestatusrfailed)rangetimerFbatch_completionr)r_r`rr num_callsr test_prompt test_callsrr{rr response_timers r,load_test_modelr$fs(KJ   DI*DUVDUq&[9:DUHVJ    3'  99; - #0   W$  99; - #0    s#B 7B B=B82B=8B=cUS:Xa#[R"5R5$US:Xa#[R"5R5$US:Xa#[R"5R5$/$)z,Return the fields required for each providerrrru)rFrget_required_paramsrAzureAIStudioConfigr`s r,get_provider_fieldsr)skl*'')==??  (##%99;;  ***,@@BB r.c L[5R5VVs0sHupXcSO[US9_M nnn0n0n[R"SS5nU(aUR S5O/nUR5HdupUc0[ R"5X0'[ R"5X@'M8[ R"US9X0'[ R"US9X@'Mf UH1n[ R"5X1'[ R"5XA'M3 X44$s snnf)Nr@NO_PROXYr)proxy) rrLrosgetenvrdr  HTTPTransportAsyncHTTPTransport)rOrAproxiessync_proxy_mountsasync_proxy_mountsno_proxy no_proxy_urlsr,s r,!create_proxy_transport_and_mountsr6s017799HC [Ten49  yyT*H+3HNN3'Mmmo  =%*%8%8%:  "&+&>&>&@  #%*%8%8u%E  "&+&>&>U&K  # &!&!4!4!6"'":":"<  001sD cSn/nUcUUS.$[[R[S5nU"US9upx nU(G aUS:Xa+S[ R ;aSnGOURS5 GOUS :XaUS [ R ;a,S [ R ;aS [ R ;aSnGOJUR/S Q5 GO5US:Xa?S[ R ;dS[ R ;aSnGOURS5 G OUS:Xa+S[ R ;aSnG OURS5 G OUS:Xa+S[ R ;aSnG OURS5 G OUS:Xa+S[ R ;aSnG OpURS5 G O]US:Xa+S[ R ;aSnG O?URS5 G O,US:Xa+S[ R ;aSnG OURS5 G OUS:XaAS[ R ;aS[ R ;aSnG OURSS/5 G OUS:Xa+S[ R ;aSnG OURS5 G OUS :Xa+S![ R ;aSnG OeURS!5 G ORUS":Xa+S#[ R ;aSnG O4URS#5 G O!US$:Xa+S%[ R ;aSnG OURS%5 G OUS&:Xa+S'[ R ;aSnG OURS'5 G OUS(:Xa+S)[ R ;aSnG OURS)5 G OUS*:XdUS+:XaS,[ R ;aS-[ R ;ddS.[ R ;dPS/[ R ;d<S0[ R ;d(S1[ R ;dS2[ R ;aSnG OURS,5 URS-5 G OUS3;a+S4[ R ;aSnG OURS45 G OUS5:Xa+S6[ R ;aSnG OURS65 G OlUS7:Xa+S8[ R ;aSnG ONURS85 G O;US9:Xa+S:[ R ;aSnG OURS:5 G O US;:XaPS<[ R ;dS=[ R ;aSnG OURS<5 URS=5 G OUS>:Xa+S?[ R ;aSnG OURS?5 G OUS@:Xa+SA[ R ;aSnG OeURSA5 G ORUSB:Xa+SC[ R ;aSnG O4URSC5 G O!US&:Xa+S'[ R ;aSnG OURS'5 GOUSD:Xa+SE[ R ;aSnGOURSE5 GOUSF:Xa+S![ R ;aSnGOURS!5 GOUSG:Xa+SH[ R ;aSnGOpURSH5 GO]USI:XdUSJ:Xa+SK[ R ;aSnGO9URSK5 GO&USL:XdUSM:Xa+SN[ R ;aSnGOURSN5 GOUSO:Xa+SP[ R ;aSnGOURSP5 GOUSQ:Xa+SR[ R ;aSnGOURSR5 GOUSS:Xa+ST[ R ;aSnGOoURST5 GO\USU:Xa+SV[ R ;aSnGO>URSV5 GO+USW:Xa+SX[ R ;aSnGO URSX5 GOUSY:Xa+SZ[ R ;aSnGOURSZ5 GOUS[:XagS\[ R ;d<S][ R ;d(S^[ R ;dS_[ R ;aSnGOoURS\5 GO\US`:XadSa[ R ;a,Sb[ R ;dSc[ R ;aSnGOURSa5 URSc5 GOUSd:Xa+Se[ R ;aSnGOURSe5 GOUSf:Xa+Sg[ R ;aSnGOURSg5 GOUSh:Xa+Si[ R ;aSnGOrURSi5 GO_USj:Xa+Sk[ R ;aSnGOAURSk5 GO.USl:Xa+Sm[ R ;aSnGOURSm5 GOUSn:Xa)So[ R ;aSnGOURSo5 GOU[R;dRU[R;d>U[R;d*U[R;dURSp5(a+S[ R ;aSnGONURS5 GO;U[R;a?S[ R ;dS[ R ;aSnGOURS5 GOU[R ;a+S[ R ;aSnGOURS5 GOU[R";a+S[ R ;aSnGO}URS5 GOjU[R$;a+S[ R ;aSnGO>URS5 GO+U[R&;a+S[ R ;aSnGOURS5 GOU[R(;a+S[ R ;aSnGOURS5 GOU[R*;d+U[R,;dU[R.S;aAS[ R ;aS[ R ;aSnGOBURSS/5 GO-U[R0;a+S[ R ;aSnGOURS5 GOU[R2;a+S![ R ;aSnGOURS!5 GOU[R4;a+S#[ R ;aSnGOURS#5 GOpU[R6;a+S%[ R ;aSnGODURS%5 GO1U[R8;a*S'[ R ;aSnGOURS'5 OU[R:;a)S)[ R ;aSnOURS)5 OU[R<;a)Se[ R ;aSnOURSe5 OyU[R>;a)Sg[ R ;aSnONURSg5 Or( dashscopeDASHSCOPE_API_KEY modelscopeMODELSCOPE_API_KEYmoonshotMOONSHOT_API_KEYz gpt-imagec\UVs/sHo!UR5;dMUPM sn$s snf)zDFilter out keys that contain the exclude_pattern (case insensitive).r\)rexclude_patternrOs r,filter_missing_keys1validate_environment..filter_missing_keysSs%#Jtciik'ItJJJs))rrr r)rrrorprr)"rFrGrHrIrr-environrrTrFrXrZopen_ai_embedding_modelsopenai_image_generation_modelsr5rrreplicate_modelsr[r\datarobot_modelsr_r]models_by_providerhuggingface_modelsratogether_ai_modelsrcbaseten_modelsrbre nebius_models wandb_modelsr) r_rrr r8r9rrr`rps r,validate_environmentr}s $  L }#6(  #"3;;x#8:LM'7e'D$1 ( *2::-&*###$45 G + BJJ.'2::5#rzz1&*###L!K /#rzz1)RZZ7&*###$78 H ,2::-&*###$45 K /"bjj0&*###$78 L 0#rzz1&*###$89 $7 7*bjj8&*###$?@ K /$ 2&*###$9: K /!RZZ/4G2::4U&*###%79L$MN M 1$ 2&*###$9: F *+&*###N3 M 1#rzz1&*###$89 M 1$ 2&*###$9: I - BJJ.&*###$56 K /"bjj0&*###$78 I -1D 1S#rzz1+rzz9"**, BJJ.0BJJ>;::7::'+###$78##$;< $= = BJJ.&*###$56 J .!RZZ/&*###$67 K /"bjj0&*###$78 $4 4'2::5&*###$<= H , BJJ.4D 4R&*###$45##$45 F *+&*###N3 L 0#rzz1&*###$89 J .!RZZ/&*###$67 I - BJJ.&*###$56 E ) *&*###M2 K /+&*###N3 L 0#rzz1&*###$89 ; ."&AA"bjj0&*###$78 ; ."&AA"bjj0&*###$78 J .!RZZ/&*###$67 I - BJJ.&*###$56 F *+&*###N3 L 0%3&*###$:; H ,2::-&*###$45 J .!RZZ/&*###$67 N 2&"**4&"**4(BJJ6'2::5&*###$:; L 0#rzz1'2::5(BJJ6&*###$89##$9: H ,2::-&*###$45 H ,2::-&*###$45 G +"**,&*###O4 K /"bjj0&*###$78 L 0#rzz1&*###$89 J .!RZZ/&*###$67 W;; ;>>>888>>> ,,2::-&*###$45 g.. .#rzz1)RZZ7&*###$78 g++ +2::-&*###$45 g.. ."bjj0&*###$78 g// /#rzz1&*###$89 g66 6*bjj8&*###$?@ g.. .$ 2&*###$9: W// /22222;??!RZZ/4G2::4U&*###%79L$MN g00 0$ 2&*###$9: g)) )+&*###N3 g00 0#rzz1&*###$89 g00 0$ 2&*###$9: g,, , BJJ.&*###$56 g.. ."bjj0&*###$78 g++ +2::-&*###$45 g++ +2::-&*###$45 g** *"**,&*###O4K*<C *<D *<G  <A"#6 UUQ #"#s+} }}c.[R"U0UD6$r#)rFr)rr:s r,acreaterfs    / //r.cSRSU55nSnSU;a)SSKnSSKJnJnJn U"5nUR U5nU$[[5RU55nU$![a [S5 N_f=f)N c3*# UH oSv M g7f)rNr)r(rs r,r**prompt_token_calculator..ls?h7I&hsrclaudez:Anthropic import failed please run `pip install anthropic`) AI_PROMPT HUMAN_PROMPT Anthropic) r rJrrrr count_tokensrrr) r_r{r num_tokensrJrrr anthropic_objs r,prompt_token_calculatorrjs 88?h? ?DJ5 T  A@! "//5  .077=>  T R S TsA..BBcU[R;dU[R;a [RR U5 gSSS./n[R "XS9 g![a [SUSS9ef=f)Nrz Hello Worldrr_r{r)rr_r5) rFrXrZr_modelsretrieverrrrs r, valid_modelr|ss H W;; ;>>> MM " "5 )!'MBCH   U > HbBGGHsAA% A%%A;ctSSS./n[R"XUSS9 g![a g[a gf=f)a; Checks if a given API key is valid for a specific model by making a litellm.completion call with max_tokens=10 Args: model (str): The name of the model to check the API key against. api_key (str): The API key to be checked. Returns: bool: True if the API key is valid for the model, False otherwise. rzHey, how's it going?rr)r_r{rr?TF)rFrrr)r_rr{s r,check_valid_keyrsN ,BCDHG   s 7 77c<US:XagUS:XagUS:XagUS:agg)a Retries on 408, 409, 429 and 500 errors. Any client error in the 400-499 range that isn't explicitly handled (such as 400 Bad Request, 401 Unauthorized, 403 Forbidden, 404 Not Found, etc.) would not trigger a retry. Reimplementation of openai's should retry logic, since that one can't be imported. https://github.com/openai/openai-python/blob/af67cfab4210d8e497c05390ce14f39105c77519/src/openai/_base_client.py#L639 iTiirFr)rs r, _should_retryrs9cccc r.c\SSKnUbURS5n[U5nU$SnU$![ac URR U5nUcSnU$URR U5n[U[R"5- 5nU$f=f![a Sngf=f)z Reimplementation of openai's calculate retry after, since that one can't be imported. https://github.com/openai/openai-python/blob/af67cfab4210d8e497c05390ce14f39105c77519/src/openai/_base_client.py#L631 rNz retry-afterr2)emailr9rrrE parsedate_tz mktime_tzr)response_headersr retry_header retry_afterretry_date_tuple retry_dates r,&_get_retry_after_from_exception_headerrs  '+// >L @!,/ K @#(;;#;#;L#I #+"$K "'!6!67G!HJ"%j499;&>"?K @  sBB ,BB*BB> ", # [OPTIONAL] "post_message": " <> [/INST] " # [OPTIONAL] }, "user": { "pre_message": "[INST] ", # [OPTIONAL] "post_message": " [/INST]" # [OPTIONAL] }, "assistant": { "pre_message": " " # [OPTIONAL] "post_message": " " # [OPTIONAL] } } final_prompt_value="Now answer as best you can:" # [OPTIONAL] ) ``` rrrr )rr)rolesinitial_prompt_valuefinal_prompt_value) rFrGrHrIrrrFknown_tokenizer_configcustom_prompt_dict) r_rrrtokenizer_configcomplete_modelpotential_modelsrrs r,register_prompt_templatersTN&' "3;;x#8:LM u-a0&!A-#1G * *1 -"  % %%"A(<&8-G & &q )"  % %%!    s;B(( B54B5cN\rSrSrS S SjjrSrSrS SjrSrSr S r g) TextCompletionStreamWrapperi?Nc4XlX lX0lX@lgr#)completion_streamr_rr`)selfrr_rr`s r,__init__$TextCompletionStreamWrapper.__init__@s"3 ,#6 r.cU$r#rrs r,__iter__$TextCompletionStreamWrapper.__iter__L r.cU$r#rrs r, __aiter__%TextCompletionStreamWrapper.__aiter__Orr.c b[5nURSS5US'SUS'URSS5US'URSS5US'[5n[U[5(a[ eUSSSnUS US 'URS 5US 'USSS US 'USSS US 'U/US'UR (a4UR RSS5SLaURSS5US'U$![ an[ SUS[U535eSnAff=f)Nrmrrcreatedr_rrrrrreasoning_contentr finish_reason include_usageFTusagez=Error occurred converting to text completion object - chunk: z ; Error: )rnr9rmr%rUrrrp)rrhr text_choicesrrs r,!convert_to_text_completion_object=TextCompletionStreamWrapper.convert_to_text_completion_objectRsd -/H"YYtT2HTN!2HX "'))It":JKL%n< %3K=(<=M+QQO!2 2/X0 !  5 < [TU]X5 SUlgr#)superr _env_hash)r ttl_secondsmax_size __class__s r,rAvailableModelsCache.__init__s /(,r.c[RR5VVs0sHupURS5(dMX_M! nnn[ [ [ UR5555$s snnf)z/Create a hash of relevant environment variables)OPENAI ANTHROPICAZUREAWS)r-rrrLr5rphash frozenset)rr%r&env_varss r, _get_env_hash"AvailableModelsCache._get_env_hashsg ((* *||CD AD*  4 (.."23455  s A6A6cfUR5nURcXlgXR:g$)z+Check if environment variables have changedT)r2r%)r current_hashs r,_check_env_changed'AvailableModelsCache._check_env_changeds/))+ >> !)N~~--r.cSnUbUR5nUbUSU3n[R"UR55R 5$)Nr:)model_dump_jsonhashlibsha256r hexdigest)rr`r valid_strs r,_get_cache_key#AvailableModelsCache._get_cache_keysU   %&668I  *./q >)+DE  ==( ( r.crURX5nURU[R"U55 g)zSet cached model infoN)r? set_cacherSr )rr`ravailable_modelsrEs r,set_cached_model_info*AvailableModelsCache.set_cached_model_infos-''(;L  y$--0@"ABr.)r%)i,r)r&rr'rrrprr)r`rrOptional[LiteLLM_Params]rrpr)r`rrrOrr)r`rprrOrJr) rIrrrrr2r6r?rFrKr __classcell__r(s@r,r"r"s--6. >* >1 > >.237*1  &C C1C$ CCr.r"c>/n[RR5n[RHhnU(aX0:waMUR SS5nUnUR 5S3nUR 5S3nXb;dXr;dMWURU5 Mj U$)Nrr_API_KEY)r-rrrrFrrupperr)r`valid_providers environ_keysrenv_provider_1env_provider_2expected_provider_key_1expected_provider_key_2s r,#_infer_valid_provider_from_env_varsr["s"$O::??$L)) 8#B "))#r2!&4%9%9%;$0J^,,PT,  **+>PVW !=aSAB s#A"AA"" B,B B BcSSKJn UcU"SS9nUbX2lUbXBlU=(d [R n/n/nU(aU/nO [ U5nUHn[RS[U5S9n U(aX:waM.US:XaURS5 MGU b(U(a!UbUR[U UU55 Mr[R"[RR!U/55n URU 5 M U$!["a%n [$R&"S U 35 /sSn A $Sn A ff=f) a Returns a list of valid LLMs based on the set environment variables Args: check_provider_endpoint: If True, will check the provider's endpoint for valid models. custom_llm_provider: If provided, will only check the provider's endpoint for valid models. api_key: If provided, will use the API key to get valid models. api_base: If provided, will use the API base to get valid models. Returns: A list of valid LLMs rrNrrrrz Azure-LLMr])litellm.types.routerrrrrFcheck_provider_endpointr[rrr^rrTr`rSr rwr9rrr) rcr`rrrrrU valid_modelsrrmodels_for_providerrs r,get_valid_modelsrfUsT&< 8  !+"5N  %, "  &. # $ Fw'F'F &("$  23OABUVO'H3KK%h/LO #x'F7"##K0++(##7' &'+mm..228R@'###$787(: !=aSAB sDD ED?9E?Ec [5(dgSU;aUSSLaURS:XagSU;aUSSLaURS:XagSU;aUSSLaURS:XagSR[[U55nSRS UR 555n[ S 5 [ S 5 U(a'U(a [ S URS USUS35 OaU(a[ S URS US35 O=U(a[ S URS US35 O[ S URS35 [ S 5 g![a gf=f)NrTrrrr<img_generationrc3F# UHupUS[U53v M g7f)=N)repr)r(rOrPs r,r*/print_args_passed_to_litellm..s!W*##aU }5s!rzRequest to litellm:z litellm.(z)z())rrIr maprkrLrr)rrr:args_str kwargs_strs r,rOrOs   .  V #}%-!**l:  F "|$,!**k9   '()T1!**.>> 99St_-YYW WW     0  F "#4#=#=">azJazR  "#4#=#=">a |8T  ,->-G-G,H R Sd    s#E%E%E%/C5E%% E21E2czSURS5-S-URS5-nU$![a gf=f)Nztime-z %H-%M-%S-%frrm)strftimer9r)rr response_ids r,get_logging_idrtsO j))-8 83 >AQAQRVAW W  s *- ::cbUcgURS05nUbURSS5nUbU$URS5=(d 0n[[R[S5nU"US9nUbU$URS05n[[R[S5nU"US9$g)NrrCr5r)r5r)r9rFrGrHrI)model_call_detailsr _base_modelr5rbase_model_from_metadatars r,_get_base_model_from_metadatarys!'++,[TU]5 gr#)r$r)rr(s r,r$CustomModelResponseIterator.__init__Ds r.rr~)rIrrrrrrPrQs@r,rrCs r.rc[RSLagURS5nUb+[U[5(aURS5S:XagSU;agUSn[U[ 5(dgUHun[U[5(dMURS5nURS5S:XdMBUcMG[U[5(dM^URS5S:XdMu g g)a Returns true, if message is marked as needing to be cached. Used for anthropic/gemini context caching. Follows the anthropic format {"cache_control": {"type": "ephemeral"}} Can be disabled globally by setting litellm.disable_anthropic_gemini_context_caching_transform = True TF cache_controlr& ephemeralrr)rF2disable_anthropic_gemini_context_caching_transformr9r%rrrn)rmessage_level_cache_controlr content_itemrs r,is_cached_messagerHsAATI#*++o">#/ 2D 9 9 ' + +F 3{ Bi G gt $ $ ,-- $((9   V $ .)=$//!!&)[8  r.cURS5(dgURS5Sn[R"USS9n[R"U5R S5U:H$![ a gf=f)Nzdata:FrrT)validateru)r5rdbase64 b64decode b64encoderr)r decoded_bytess r,is_base64_encodedrysz||    GGCLO((T:  .55g>!CC sA*AA** A76A7c:SU;aURS5S$U$)z+ s: b64str OR data:image/png;base64,b64str rrrc)rs r,get_base64_strrs" axwws|A Hr.c>UHnURS5cM g g)za Returns true, if messages has tool call blocks. Used for anthropic/bedrock message validation. rjTFr8)r{rs r,has_tool_call_blocksrs%  ;;| $ 0 r.cUHRnURS5S:XdMURS5nUcM0[US5(a[U5S:dMR g g)a Returns true if ANY assistant message has thinking_blocks. This is used to prevent dropping the thinking param when some messages in the conversation already contain thinking blocks. Dropping thinking when thinking blocks exist causes Anthropic error: "When thinking is disabled, an assistant message cannot contain thinking" Related issue: https://github.com/BerriAI/litellm/issues/18926 rirkthinking_blocks__len__rTFr9ror)r{rrs r,)any_assistant_message_has_thinking_blocksrsV ;;v + -%kk*;QRi j r.c Ub>[U[5(a)[R(d[ U5S:aUSSnU$)z Validates and fixes OpenAI optional parameters. Args: stop: Stop sequences (string or list of strings) **kwargs: Additional optional parameters Returns: Validated stop parameter (truncated to 4 elements if needed) N)r%rnrFdisable_stop_sequence_limitr)rr:s r,validate_openai_optional_paramsr s=  tT " "33 t9q=8D Kr.c[R"S5RS5n[R"UR 55$![ a 0s$f=f)NrFz+model_prices_and_context_window_backup.json)rfilesjoinpathrloads read_textr)model_cost_paths r,_get_bundled_model_cost_mapr sQ#//)4== 9 zz/33566  sAA AAcXRSU34nUH)n[RRU5nUcM'Us $ [ 5nUHnURU5nUcMUs $ 0$)Nr1)rPrFrr9r)r_rcandidate_keysrrbundled_model_costs r,)_get_model_cost_entry_for_provider_configr s/q89N# ''++I6  ! $ 56# '++I6  ! $ Ir.c\rSrSr%SrS\S'\S-Sj5r\S.S/Sjj5r\S0Sj5r \S0Sj5r \S0S j5r \S0S j5r \S1S j5r \S1S j5r\S.S2S jj5r\S3Sj5r\S4Sj5r\S5Sj5r\\"\S9S5Sj55r\S6Sj5r\S.S7Sjj5r\S.S8Sjj5r\S9Sj5r\S:Sj5r\S;Sj5r\SSj5r\S?Sj5r\S@Sj5r \SASj5r!\S.SBSjj5r"\SCS j5r#\SDS!j5r$\SES"j5r%\SFS#j5r&\SGS$j5r'\SHS%j5r(\SIS&j5r)\SJS'j5r*\SKS(j5r+\SLS)j5r,\SMS*j5r-\SNS+j5r.S,r/g)Ori Nz3Optional[dict[LlmProviders, tuple[Callable, bool]]]_PROVIDER_CONFIG_MAPcv0[RSS4_[RSS4_[RSS4_[RSS4_[R SS4_[R SS4_[RS S4_[RS S4_[RS S4_[RS S4_[RS S4_[RSS4_[RSS4_[RSS4_[RSS4_[R SS4_[R"SS4_0[R$SS4_[R&SS4_[R(SS4_[R*SS4_[R,SS4_[R.SS4_[R0SS4_[R2SS4_[R4SS4_[R6SS4_[R8SS4_[R:SS4_[R<S S4_[R>S!S4_[R@S"S4_[RBS#S4_[RDS$S4_E0[RFS%S4_[RHS&S4_[RJS'S4_[RLS(S4_[RNS)S4_[RPS*S4_[RRS+S4_[RTS,S4_[RVS-S4_[RXS.S4_[RZS/S4_[R\S0S4_[R^S1S4_[R`S2S4_[RbS3S4_[RdS4S4_[RfS5S4_E0[RhS6S4_[RjS7S4_[RlS8S4_[RnS9S4_[RpS:S4_[RrS;S4_[RtS<S4_[RvS=S4_[RxS>S4_[RzS?S4_[R|S@S4_[R~SAS4_[RSBS4_[RSCS4_[RSDS4_[RSES4_[RSFS4_E0[RSGS4_[RSHS4_[RSIS4_[RSJS4_[RSKS4_[RSLS4_[RSMS4_[RSNS4_[RSOS4_[RSPS4_[RSQS4_[RSRS4_[RSSS4_[RSTS4_[RSUS4_[RSVS4_[RSWS4_E[RSXS4[RSYS4[RSZS4[RS[S4[RS\S4[RS]S4[RS^S4[RS_S4[RS`S4[RSaS4[RSbS4[RScS4[RSdS4[RSeS40E$)fzBuild the provider-to-config mapping dictionary. Returns a dict mapping provider to (factory_function, needs_model_parameter). This avoids expensive inspect.signature() calls at runtime. c,[R"5$r#)rFOpenAIGPTConfigrr.r,BProviderConfigManager._build_provider_config_map.. s '*A*A*Cr.Fc,[R"5$r#)rFrbrr.r,rr W-D-D-Fr.c,[RU5$r#)r_get_azure_ai_configrs r,rr s3HHOr.Tc,[RU5$r#)r_get_vertex_ai_configrs r,rr s3II%Pr.c,[RU5$r#)r_get_bedrock_configrs r,rr s3GGNr.c,[RU5$r#r_get_cohere_configrs r,rr 3FFuMr.c,[RU5$r#r rs r,rr r r.c,[R"5$r#)rFrrr.r,rr! G,F,F,Hr.c,[R"5$r#)rFrrr.r,rr! (>(>(@r.c,[R"5$r#)rFrrr.r,rr! 779r.c,[R"5$r#)rF A2AConfigrr.r,rr!s w'8'8':r.c,[R"5$r#)rFBytezChatConfigrr.r,rr! )@)@)Br.c,[R"5$r#)rFrrr.r,rr ! g.F.F.Hr.c,[R"5$r#)rFrrr.r,rr ! w'<'<'>r.c,[R"5$r#)rF ZAIChatConfigrr.r,rr !rr.c,[R"5$r#)rFLambdaAIChatConfigrr.r,rr ! W-G-G-Ir.c,[R"5$r#)rFInceptionChatConfigrr.r,rr ! W-H-H-Jr.c,[R"5$r#)rFLlamaAPIConfigrr.r,rr!s )?)?)Ar.c,[R"5$r#)rFOpenAITextCompletionConfigrr.r,rr!s ::! W-C-C-Er.c,[R"5$r#)rFAzureOpenAITextConfigrr.r,rr?!s g.K.K.Mr.c,[R"5$r#)rFrrr.r,rr@!rnr.c,[R"5$r#)rFOobaboogaConfigrr.r,rrA!rr.c,[R"5$r#)rFrrr.r,rrB!rbr.c,[R"5$r#)rFrrr.r,rrC!rr.c,[R"5$r#)rFPerplexityChatConfigrr.r,rrD!r2r.c,[R"5$r#rFrsrr.r,rrE!rLr.c,[R"5$r#ryrr.r,rrF!s W-B-B-Dr.c,[R"5$r#)rFrrr.r,rrG!s g.E.E.Gr.c,[R"5$r#)rFrrr.r,rrH!r-r.c,[R"5$r#)rF BasetenConfigrr.r,rrI!rLr.c,[R"5$r#)rFrrr.r,rrJ!rr.c,[R"5$r#)rFrrrr.r,rrL! ==?r.c,[R"5$r#)rFrrr.r,rrP!rr.c,[R"5$r#)rFSambanovaConfigrr.r,rrS!rr.c,[R"5$r#)rFrfrr.r,rrT!r-r.c,[R"5$r#)rFrrr.r,rrU!s (:(:(*>*@r.c,[R"5$r#)rFrhrr.r,rrW!rr.c,[R"5$r#)rFrerr.r,rrX!rr.c,[R"5$r#)rFrrr.r,rrY!rr.c,[R"5$r#)rFGenAIHubOrchestrationConfigrr.r,rr[! ;;=r.c,[R"5$r#)rFFeatherlessAIConfigrr.r,rr^!r/r.c,[R"5$r#)rF NovitaConfigrr.r,rr_!rr.c,[R"5$r#)rFrrr.r,rr`!rr.c,[R"5$r#)rF WandbConfigrr.r,rra!s )<)<)>r.c,[R"5$r#)rFDashScopeChatConfigrr.r,rrb!r$r.c,[R"5$r#)rFModelScopeChatConfigrr.r,rrc!r2r.c,[R"5$r#)rFMoonshotChatConfigrr.r,rrd!rr.c,[R"5$r#)rFDockerModelRunnerChatConfigrr.r,rrf!rr.c,[R"5$r#)rF V0ChatConfigrr.r,rri!s g&:&:&FKY L   >FMY N  " "%I5$QOY P  " "%JE$RQY R   !A5 ISY T  / /<2UY \  " "%F$N]Y ^  ! !$De#L_Y `  ' '*OQV)WaY b  ' '*NPU)VcY d  # #&Le%TeY f  ' '*OQV)WgY h  ' '*OQV)WiY j  " "%F$NkY l  % %(KU'SmY n  # #&Le%ToY p  #I5"QqY r  % %(Le'TsY t  #F"NuY v  #F"NwY x   "De!LyY z  $ $'NPU&V{Y |  ' '*OQV)W}Y ~  #BE"JY @  ! !$De#LAY B  #BE"JCY D   "H%!PEY F  & &)OQV(WGY H  $ $'Mu&UIY J  ' '9*KY R  $ $'Mu&USY T  " "%JE$RUY V  " "%I5$QWY X  " "%JE$RYY Z  " "%F$N[Y \  $ $'NPU&V]Y ^  $ $'I5&Q_Y `  # #&H%%PaY b  * *7-cY j  ! !$De#LkY l  " "%F$NmY n   "NPU!VoY p    @%HqY r  " "%Eu$MsY t  # #&Mu%UuY v  " "%Eu$MwY x  " "%F$NyY z  $ $'I5&Q{Y |  " "%F$N}Y ~  # #&Le%TY @  #BE"JAY B  " "%De$LCY D  # #&G%OEY F  ! !$De#LGY H  #BE"JIY J  # #&H%%PKY L  2 2?5MY T  2 2?5UY \  " "%F$N]Y ^  ! !$De#L_Y `     FyY z  " "%JE$R{Y |  # #&Le%T}Y ~  ! !$H%#P  , ,=/ OOF  # #&Le%T  ! !$H%#P  $ $'Mu&U  " "E%  ! !D$kY Y r.c(U=(d Un[R"5RUS9(a[R"5$[RR US9(a[R"5$[R "5$)a\Get Azure config based on model type. When *base_model* is provided (e.g. ``"azure/gpt-5.2"``), it is used for model-type detection instead of *model* (the deployment name). This allows non-standard deployment names like ``"azure/foo"`` to be routed through the correct config when the user specifies the true underlying model via ``base_model``. r)rFrrrrrB)r_rCdetection_models r,_get_azure_config'ProviderConfigManager._get_azure_config!sl%-  & & ( : : : Q..0 0  ( ( = =O = T002 2((**r.c0SSKJn URU5$)z(Get Azure AI config based on model type.rAzureFoundryModelInfo)"litellm.llms.azure_ai.common_utilsr get_azure_ai_config_for_model)r_r s r,r*ProviderConfigManager._get_azure_ai_config!s M$BB5IIr.cSU;a[R"5$SU;a[R"5$SU;a SSKJn U"5$U[R ;a0SU;a[R "5$[R"5$U[R;a[R"5$[R"5$)z)Get Vertex AI config based on model type.rYrzgpt-ossr)VertexAIGPTOSSTransformationr:) rFrmroFlitellm.llms.vertex_ai.vertex_ai_partner_models.gpt_oss.transformationr rqrrrsrtrurv)r_r s r,r+ProviderConfigManager._get_vertex_ai_config!s u --/ /  224 4 %  01 1 g33 3e#<<>>((* * g33 3--/ ///1 1r.cSSKJn U"US9$)z"Get Bedrock config based on model.r)get_bedrock_chat_configr)rUr! )r_r! s r,r)ProviderConfigManager._get_bedrock_config!s N&U33r.c[[R[S5nUR"U5nUS:Xa[ R "5$[ R"5$)z!Get Cohere config based on route.rv2)rFrGrHrIget_cohere_routerFCohereV2ChatConfigrd)r_rroutes r,r (ProviderConfigManager._get_cohere_config!sO"#++h"79JK007 D=--/ /''))r.cSSKJn U"5$)zGet LangGraph config.rLangGraphConfig)*litellm.llms.langgraph.chat.transformationr+ r* s r,r+ProviderConfigManager._get_langgraph_config!s O  r.cSSKJn U"5$)zGet LangFlow config.rLangFlowConfig))litellm.llms.langflow.chat.transformationr0 r/ s r,r*ProviderConfigManager._get_langflow_config!s Mr.cU[R:Xai[RR US9(a[R$[R R US9(a[R "5$U[R:Xa[RXS9$[Rc[R5[l [RRU5nUbUupEU(aU"U5$U"5$SSK Jn SSKJn UR#UR$5(aDURUR$5nUc['SUR$S35eU"U5"5$g) a< Returns the provider config for a given provider. Uses O(1) dictionary lookup for fast provider resolution. Python classes take priority over JSON (they have custom overrides). For Azure, *base_model* (when set) drives model-type detection so that non-standard deployment names still route to the correct config. r)r_rCNr)create_config_classJSONProviderRegistryz Provider z not found)r^r+rFopenaiOSeriesConfigis_model_o_series_modelOpenAIGPT5Configrr-rr rr r9'litellm.llms.openai_like.dynamic_configr4 $litellm.llms.openai_like.json_loaderr6 existsrPr) r_rrC config_entryconfig_factory needs_modelr4 r6 rs r,ra.ProviderConfigManager.get_provider_chat_config!sG |** ***BBBO222''<<5<I//11 |)) )(::;  ! 5 5 =%@@B " 6 -AAEEhO  #*6 'N%e,,%'' PM  & &x~~ 6 6266x~~FO& 9X^^,U:Xa SSK J!n U"5$[RRDU:Xa[RF"5$[RRHU:Xa[RJ"5$[RRLU:Xa[RN"5$[RRPU:Xa[RR"5$[RRTU:Xa SSK+J,n U"5$[RRZU:Xa SSK.J/n U"5$[RR`U:Xa[Rb"5$[RRdU:Xa[Rf"5$[RRhU:XaSS K5J6n U RoU5$[RRpU:Xa[Rr"5$g) Nr)OCIEmbedConfig)CohereEmbeddingConfig)r)VolcEngineEmbeddingConfig)DashScopeEmbeddingConfig)OpenrouterEmbeddingConfig)VercelAIGatewayEmbeddingConfig)SagemakerEmbeddingConfig):rFr^VOYAGErrrrrrrrIBMWatsonXEmbeddingConfigrrINFINITYrrrr %litellm.llms.oci.embed.transformationrB rr(litellm.llms.cohere.embed.transformationrC JINA_AI-litellm.llms.jina_ai.embedding.transformationrr0litellm.llms.volcengine.embedding.transformationrD r +litellm.llms.dashscope.embed.transformationrE r rrSnowflakeEmbeddingConfigrCometAPIEmbeddingConfigrRGithubCopilotEmbeddingConfigr0litellm.llms.openrouter.embedding.transformationrF r7litellm.llms.vercel_ai_gateway.embedding.transformationrG rGigaChatEmbeddingConfigrHostedVLLMEmbeddingConfigr/litellm.llms.sagemaker.embedding.transformationrH get_model_configrPerplexityEmbeddingConfig) r_rrB rC rrD rE rF rG rH s r,r3ProviderConfigManager.get_provider_embedding_config"s~  ' '8 377TT::< <  ' '8 377PPQVWW::< <  ! ! ( (H 4002 2  ! ! ( (H 4002 2  ! ! ) )X 5446 6  ! ! 7 78 C224 4  ! ! * *h 6224 4  ! ! + +x 7335 5  ! ! % % 1 L!# #  ' '8 3##//8; V(* *  ! ! ) )X 5 )* *  ! ! , , 8 -. .  ! ! + +x 7 ,- -  ! ! * *h 6224 4  ! ! + +x 7335 5  ! ! * *h 6224 4  ! ! 0 0H <779 9  ! ! , , 8 -. .  ! ! 3 3x ? 23 3  ! ! * *h 6224 4  ! ! - - 9446 6  ! ! + +x 7 ,<>@ @  ! ! * *h 6;;= =  ! ! , , 8 67 7  ! ! ( (H 45 @@BBDDFF  ! ! - - 9 67 7  ! ! ) )X 5 67 7  ! ! * *h 6 45 5  ! ! * *h 6 45 5  ! ! ) )X 5 34 4  ! ! - - 9 67 7  ! ! ( (H 4 23 3r.cSSKJn SSKJn [ U[ 5(a UR O [U5nSn[ U[ 5(aUnO [ U5n[RXQ5nUbU$URU5(a7URU5(a!URU5nUb U"U5"5$g![a Ntf=f)Nr)create_responses_config_classr5 )r: r r; r6 r%r^rPrprr _get_python_responses_api_configr< supports_responses_apir9)rr_r r6 provider_str provider_enumrrs r,!get_provider_responses_api_config7ProviderConfigManager.get_provider_responses_api_config#s   N)<@@HNNc(m  15 h - -$M  ,X 6 'GG    M & &   "99,GG266|DO*4_EGG%  s C CCc Ucg[RRU:Xa[R"5$[RRU:XaU=(a SUR 5;nU=(a3 SUR 5;=(d [ U5=(a U(+nU(a[R"5$[R"5$[RRU:Xa[R"5$[RRU:Xa*SSK J n Ub U"US9(a[R"5$g[RRU:Xa[R "5$[RR"U:Xa[R$"5$[RR&U:Xa[R("5$[RR*U:Xa[R,"5$[RR.U:Xa[R0"5$[RR2U:Xa1U(a)SUR 5;a[R4"5$g[RR6U:Xa[R8"5$[RR:U:Xa[R<"5$[RR>U:XaXSSK J!nJ"n U(aU"U[RF5(dg[RH"U"U[RF5S:HS 9$g) zFCheck for Python-class-based responses API configs (custom overrides).Nr o_seriesr)%github_copilot_supports_responses_apir)mantle_base_segmentmantle_supports_responsesz openai/v1)use_openai_path)%rFr^r+OpenAIResponsesAPIConfigr-r]r$AzureOpenAIOSeriesResponsesAPIConfigAzureOpenAIResponsesAPIConfigrXAIResponsesAPIConfigrR4litellm.llms.github_copilot.responses.transformationr GithubCopilotResponsesAPIConfigrSChatGPTResponsesAPIConfigr LiteLLMProxyResponsesAPIConfigrVolcEngineResponsesAPIConfigMANUSManusResponsesAPIConfigrPerplexityResponsesConfigrDatabricksResponsesAPIConfigrOpenRouterResponsesAPIConfigrHostedVLLMResponsesAPIConfigr(litellm.llms.bedrock_mantle.common_utilsr r rBedrockMantleResponsesAPIConfig)rr_ is_gpt_model is_o_seriesr r r s r,r 6ProviderConfigManager._get_python_responses_api_config<#s      & &( 2335 5  ! ! ' '8 3!;Uekkm%;Lekkm+D&u-Bl2B  CCEE<<>>  ! ! % % 1002 2  ! ! 0 0H < } EE R>>@@  ! ! ) )X 5446 6  ! ! / /8 ;99; ;  ! ! , , 8779 9  ! ! ' '8 3224 4  ! ! , , 8446 6  ! ! , , 8%++-/;;==  ! ! , , 8779 9  ! ! - - 9779 9  ! ! 0 0H <   9%ASAS T T:: 3E7;M;M N! r.cj[RRU:Xa[R"5$g)z Get provider-specific Skills API configuration Args: provider: The LLM provider Returns: Provider-specific Skills API config or None N)rFr^r,AnthropicSkillsConfig)rs r,get_provider_skills_api_config4ProviderConfigManager.get_provider_skills_api_config#s*    ) )X 5002 2r.cZ[RRU:Xa SSKJn U"5$g)z Get provider-specific Evals API configuration Args: provider: The LLM provider Returns: Provider-specific Evals API config or None r)OpenAIEvalsConfigN)rFr^r+(litellm.llms.openai.evals.transformationr )rr s r,get_provider_evals_api_config3ProviderConfigManager.get_provider_evals_api_config#s'    & &( 2 R$& &r.c"[RU:Xa[R"5$[RU:Xa[R "5$[R U:Xa[R"5$[R"5$r#) r^rrFFireworksAITextCompletionConfigrTogetherAITextCompletionConfigrrr(rs r,#get_provider_text_completion_config9ProviderConfigManager.get_provider_text_completion_config#sj  $ $ 0::< <  % % 199; ;  3 3x ?88: :1133r.c~[RU:Xa[R"5$[RU:Xa[R "5$[R U:Xa[R"5$[RU:Xa SSK J n U"5$[RU:Xa[R"5$[RU:Xa[R"5$[RU:Xa[R "5$[R"U:Xa[R$"5$[R&U:Xd[R(U:Xa SSKJn U"5$[R.U:Xd[R0U:Xa SSKJn U"5$[R6U:Xa[R8"5$[R:U:Xa[R<"5$[R>U:Xa SSK J!n U"5$[RDU:Xa SSK#J$n U"US9$g)Nr)VertexAIModelInfo)OllamaModelInfo) VLLMModelInforr r)%r^rrFrr+rrGeminiModelInfor#litellm.llms.vertex_ai.common_utilsr r rTOPAZTopazModelInfor,AnthropicModelInfor XAIModelInforr litellm.llms.ollama.common_utilsr rrlitellm.llms.vllm.common_utilsr LEMONADELemonadeChatConfigrr,rrUrrr r )r_rr r r rr s r,r-ProviderConfigManager.get_provider_model_info#s  $ $ 0,,. .  H ,**, ,  H ,**, ,  # #x / M$& &  ' '8 3113 3   8 +))+ +  # #x /--/ /    )'') )  H , 0H0HH0T H"$ $   ( *l.F.F(.R !? "  " "h .--/ /  " "h .))+ +  ! !X - J#% %  " "h . P(u5 5r.c4[RU:Xa SSKJn U"5$[RU:Xd[R U:Xa SSKJn U"5$[RU:Xa SSK J n U"5$[RU:Xa SSK J n U"5$g)Nr)BedrockPassthroughConfig)VLLMPassthroughConfig)AzurePassthroughConfig)WatsonxPassthroughConfig)r^r/litellm.llms.bedrock.passthrough.transformationr rr,litellm.llms.vllm.passthrough.transformationr r--litellm.llms.azure.passthrough.transformationr r/litellm.llms.watsonx.passthrough.transformationr )r_rr r r r s r,get_provider_passthrough_config5ProviderConfigManager.get_provider_passthrough_config#s   8 + ,- -   ( *l.F.F(.R )* *   8 + *+ +  ! !X - ,- -r.c[RU:Xa[R"5$[RU:Xa[R "5$gr#)r^r+rFOpenAIImageVariationConfigr TopazImageVariationConfigrs r,#get_provider_image_variation_config9ProviderConfigManager.get_provider_image_variation_config $sA   ( *557 7   8 +446 6r.cN[RU:Xa SSKJn U"5$[RU:Xa SSKJn U"5$[RU:Xa SSKJ n U"5$[RU:Xa SSK J n U"5$[RU:Xa SSKJn U"5$g)Nr)GoogleAIStudioFilesHandler)VertexAIFilesConfig)BedrockFilesConfig)ManusFilesConfig)AnthropicFilesConfig)r^r(litellm.llms.gemini.files.transformationr r+litellm.llms.vertex_ai.files.transformationr r)litellm.llms.bedrock.files.transformationr r 'litellm.llms.manus.files.transformationr r,+litellm.llms.anthropic.files.transformationr )r_rr r r r r s r,get_provider_files_config/ProviderConfigManager.get_provider_files_config$s   ( * ./ /  # #x / W&( (  ! !X - T%' '   8 + P#% %  # #x / X') )r.cF[RU:Xa SSKJn U"5$g)Nr)BedrockBatchesConfig)r^r+litellm.llms.bedrock.batches.transformationr )r_rr s r,get_provider_batches_config1ProviderConfigManager.get_provider_batches_config1$s!   8 + X') )r.cXSSKJn [RU:XaUR 5$g)Nr)BedrockVectorStore)Clitellm.integrations.vector_store_integrations.bedrock_vector_storer r^rget_initialized_custom_logger)rr s r, get_provider_vector_store_config6ProviderConfigManager.get_provider_vector_store_config<$s)    8 +%CCE Er.c[RRU:Xa SSKJn U"5$[RR U:Xa SSKJn U"5$[RRU:Xa+US:XdUc SSK J n U"5$US:Xa SSK J n U"5$g[RRU:Xa SS KJn U"5$[RR U:Xa SS KJn U"5$[RR&U:Xa SS KJn U"5$[RR,U:Xa SS KJn U "5$[RR2U:Xa SS KJn U "5$[RR8U:Xa SSKJn U "5$[RR>U:Xa SSK J!n U "5$g)zD v2 vector store config, use this for new vector store integrations r)OpenAIVectorStoreConfig)AzureOpenAIVectorStoreConfigrag_apiN)VertexVectorStoreConfig search_api) VertexSearchAPIVectorStoreConfig)BedrockVectorStoreConfig)PGVectorStoreConfig)AzureAIVectorStoreConfig)MilvusVectorStoreConfig)GeminiVectorStoreConfig)RAGFlowVectorStoreConfig)S3VectorsVectorStoreConfig)"rFr^r+0litellm.llms.openai.vector_stores.transformationr r-/litellm.llms.azure.vector_stores.transformationr r;litellm.llms.vertex_ai.vector_stores.rag_api.transformationr >litellm.llms.vertex_ai.vector_stores.search_api.transformationr r1litellm.llms.bedrock.vector_stores.transformationr PG_VECTOR3litellm.llms.pg_vector.vector_stores.transformationr r2litellm.llms.azure_ai.vector_stores.transformationr MILVUS0litellm.llms.milvus.vector_stores.transformationr r0litellm.llms.gemini.vector_stores.transformationr r1litellm.llms.ragflow.vector_stores.transformationr S3_VECTORS4litellm.llms.s3_vectors.vector_stores.transformationr ) rapi_typer r r r r r r r r r r s r,!get_provider_vector_stores_config7ProviderConfigManager.get_provider_vector_stores_configH$s    & &( 2 +, ,  ! ! ' '8 3 01 1  ! ! + +x 79$(8/00\)899 *`U ! ! ) )X 5 ,- -  ! ! + +x 7 '( (  ! ! * *h 6 ,- -  ! ! ( (H 4 +, ,  ! ! ( (H 4 +, ,  ! ! ) )X 5 ,- -  ! ! , , 8 ./ /r.cZ[RRU:Xa SSKJn U"5$g)Nr)OpenAIVectorStoreFilesConfig)rFr^r+5litellm.llms.openai.vector_store_files.transformationr* )rr* s r,&get_provider_vector_store_files_configU:XaSS K J!n U "U5$[RDU:XaSS K#J$n U "U5$[RJU:XaSSK&J'n U"U5$[RPU:XaSSK)J*n U"U5$[RVU:XaSSK,J-n U"U5$[R\U:XaSSK/J0n U"U5$[RbU:XaSSK2J3n U"U5$g)Nr)"get_openai_image_generation_config)!get_azure_image_generation_config)$get_azure_ai_image_generation_config)&get_xinference_image_generation_config)#get_recraft_image_generation_config) get_aiml_image_generation_config)$get_cometapi_image_generation_config)"get_gemini_image_generation_config)!LiteLLMProxyImageGenerationConfig)"get_fal_ai_image_generation_config)%get_stability_image_generation_config)$get_runwayml_image_generation_config)-get_black_forest_labs_image_generation_config)%get_vertex_ai_image_generation_config)&get_openrouter_image_generation_config)%get_dashscope_image_generation_config)&get_modelscope_image_generation_config)4r^r+$litellm.llms.openai.image_generationr/ r-#litellm.llms.azure.image_generationr0 r&litellm.llms.azure_ai.image_generationr1 XINFERENCE(litellm.llms.xinference.image_generationr2 RECRAFT%litellm.llms.recraft.image_generationr3 AIML"litellm.llms.aiml.image_generationr4 r&litellm.llms.cometapi.image_generationr5 r$litellm.llms.gemini.image_generationr6 r :litellm.llms.litellm_proxy.image_generation.transformationr7 FAL_AI$litellm.llms.fal_ai.image_generationr8 STABILITY'litellm.llms.stability.image_generationr9 RUNWAYML&litellm.llms.runwayml.image_generationr: BLACK_FOREST_LABS/litellm.llms.black_forest_labs.image_generationr; r'litellm.llms.vertex_ai.image_generationr< r(litellm.llms.openrouter.image_generationr= r 'litellm.llms.dashscope.image_generationr> r (litellm.llms.modelscope.image_generationr? )r_rr/ r0 r1 r2 r3 r4 r5 r6 r7 r8 r9 r: r; r< r= r> r? s r,$get_provider_image_generation_config:ProviderConfigManager.get_provider_image_generation_config$s&   ( * 6e< <   8 + 5U; ;  " "h . 8> >  $ $ 0 :%@ @  ! !X - 7u= =   ( * 4E: :  " "h . 8> >  H , 6e< <  ' '8 3 56 6  H , 6e< <  # #x / 9? ?  " "h . 8> >  + +x 7 AG G  # #x / 9? ?  $ $ 0 :%@ @  # #x / 9? ?  $ $ 0 :%@ @r.cN[RU:Xa SSKJn U"5$[RU:Xa SSKJn U"5$[RU:Xa SSKJ n U"5$[RU:Xa SSK J n U"5$[RU:Xa SSKJn U"5$g)Nr)OpenAIVideoConfig)AzureVideoConfig)GeminiVideoConfig)VertexAIVideoConfig)RunwayMLVideoConfig)r^r+)litellm.llms.openai.videos.transformationr[ r-(litellm.llms.azure.videos.transformationr\ r)litellm.llms.gemini.videos.transformationr] r,litellm.llms.vertex_ai.videos.transformationr^ rP +litellm.llms.runwayml.videos.transformationr_ )r_rr[ r\ r] r^ r_ s r,get_provider_video_config/ProviderConfigManager.get_provider_video_config%s   ( * S$& &   8 + Q#% %  H , S$& &  # #x / X&( (  " "h . W&( (r.c[RU:Xa SSKJn U"5$U[R[R 4;a SSKJn U"5$g)Nr)OpenAIContainerConfig)AzureContainerConfig)r^r+-litellm.llms.openai.containers.transformationrh r-r,litellm.llms.azure.containers.transformationri )rrh ri s r,get_provider_container_config3ProviderConfigManager.get_provider_container_config)%sN   ( * )* *  **L,C,CD D () )r.cF[RU:Xa SSKJn U"5$g)Nr)GeminiRealtimeConfig)r^r+litellm.llms.gemini.realtime.transformationro )r_rro s r,get_provider_realtime_config2ProviderConfigManager.get_provider_realtime_config;%s!   ( * X') )r.c[RU:Xa SSKJn U"5$[RU:Xa SSKJn U"5$g)z} Return the HTTP transformation config for realtime HTTP endpoints (POST /realtime/client_secrets and POST /realtime/calls). r)OpenAIRealtimeHTTPConfig)AzureRealtimeHTTPConfigN)r^r+0litellm.llms.openai.realtime.http_transformationrt r-/litellm.llms.azure.realtime.http_transformationru )r_rrt ru s r,!get_provider_realtime_http_config7ProviderConfigManager.get_provider_realtime_http_configF%sC   ( * ,- -    ) +, ,r.c[RU:Xa SSKJn U"US9$[RU:Xa SSKJn U"5$[RU:Xa SSKJ n U"5$[RU:Xa SSK J n U"5$[RU:XaSSKJn U"U5$[R U:XaSSKJn U"U5$[R&U:Xa SS KJn U"5$[R,U:XaSS KJn U "U5$[R2U:XaSS KJn U "U5$[R8U:XaSS KJn U "U5$[R>U:XaSS K J!n U "U5$g)Nr)get_openai_image_edit_configr)AzureImageEditConfig)RecraftImageEditConfig)BlackForestLabsImageEditConfig)get_azure_ai_image_edit_config)get_gemini_image_edit_config)LiteLLMProxyImageEditConfig)get_vertex_ai_image_edit_config)get_stability_image_edit_config)'get_bedrock_image_edit_config_for_model) get_openrouter_image_edit_config)"r^r+litellm.llms.openai.image_editr{ r-,litellm.llms.azure.image_edit.transformationr| rE .litellm.llms.recraft.image_edit.transformationr} rR 8litellm.llms.black_forest_labs.image_edit.transformationr~ r litellm.llms.azure_ai.image_editr rlitellm.llms.gemini.image_editr r 4litellm.llms.litellm_proxy.image_edit.transformationr r!litellm.llms.vertex_ai.image_editr rN !litellm.llms.stability.image_editr rLlitellm.llms.bedrock.image_edit.amazon_nova_canvas_image_edit_transformationr r"litellm.llms.openrouter.image_editr ) r_rr{ r| r} r~ r r r r r r r s r,get_provider_image_edit_config4ProviderConfigManager.get_provider_image_edit_config^%sS   ( * S/e< <   8 + () )  ! !X - *+ +  + +x 7 23 3  " "h . W1%8 8  H , S/6 6  ' '8 3 /0 0  # #x / 359 9  # #x / 359 9  ! !X - ;5A A  $ $ 0 4E: :r.cSSKJn U[RR:Xa SSKJn U"US9$U[RR:Xa SSKJ n U"US9$U[RR:Xa#SSK J nJ n US:XaU"5$US:XaU"5$g [[R ["S 5n[RR$U0nUR'US 5n U cg U "5$) z- Get OCR configuration for a given provider. r)VertexAIOCRConfig)get_azure_ai_ocr_configr)get_vertex_ai_ocr_config)ReductoParseLegacyConfigReductoParseV3Configzparse-v3z parse-legacyNr))litellm.llms.vertex_ai.ocr.transformationr rFr^r&litellm.llms.azure_ai.ocr.common_utilsr r'litellm.llms.vertex_ai.ocr.common_utilsr REDUCTO'litellm.llms.reducto.ocr.transformationr r rFrGrHrIrr9) r_rr r r r r rPROVIDER_TO_CONFIG_MAPrs r,get_provider_ocr_config-ProviderConfigManager.get_provider_ocr_config%s P w++44 4 V*7 7 w++55 5 X+%8 8 w++33 3   "+--&/11"3;;x#8:LM  ( (*:" .11(DA  ~r.cNSSKJn SSKJn SSKJn SSKJn SSKJ n SSK J n SSK J n SS KJn SS KJn SS KJn SS KJn SS KJn SSKJn SSKJn SSKJn SSKJn SSK J!n 0[DRFU _[DRHU_[DRJU _[DRLU_[DRNU_[DRPU_[DRRU_[DRTU_[DRVU_[DRXU _[DRZU _[DR\U_[DR^U _[DR`U_[DRbU_[DRdU_[DRfU_nURiUS5nUcgU"5$)z0 Get Search configuration for a given provider. r)APISerpentSearchConfig)BraveSearchConfig)DataForSEOSearchConfig)DuckDuckGoSearchConfig)ExaAISearchConfig)FastCRWSearchConfig)FirecrawlSearchConfig)GooglePSESearchConfig)LinkupSearchConfig)ParallelAISearchConfig)PerplexitySearchConfig)SearchAPIConfig)SearXNGSearchConfig)SerperSearchConfig)TavilySearchConfig)TinyfishSearchConfig)YouComSearchConfigN)5-litellm.llms.apiserpent.search.transformationr (litellm.llms.brave.search.transformationr -litellm.llms.dataforseo.search.transformationr -litellm.llms.duckduckgo.search.transformationr )litellm.llms.exa_ai.search.transformationr *litellm.llms.fastcrw.search.transformationr ,litellm.llms.firecrawl.search.transformationr -litellm.llms.google_pse.search.transformationr )litellm.llms.linkup.search.transformationr .litellm.llms.parallel_ai.search.transformationr -litellm.llms.perplexity.search.transformationr ,litellm.llms.searchapi.search.transformationr *litellm.llms.searxng.search.transformationr )litellm.llms.serper.search.transformationr )litellm.llms.tavily.search.transformationr +litellm.llms.tinyfish.search.transformationr *litellm.llms.you_com.search.transformationr rjrTAVILY PARALLEL_AIEXA_AIBRAVE GOOGLE_PSE DATAFORSEO FIRECRAWLFASTCRWSEARXNGLINKUP DUCKDUCKGO SEARCHAPISERPERYOU_COM APISERPENTTINYFISHr9)rr r r r r r r r r r r r r r r r r r rs r,get_provider_search_config0ProviderConfigManager.get_provider_search_config%s  OXXORVWP  YPRPPTQ"  & &(>"  " "$6"   ' ')?"   " "$5 "  ! !#4 "  & &(= "   & &(>"   % %'<"   # #%8"   # #%8"   " "$6"   & &(>"   % %"   " "$6"   # #%7"  & &(>!" "  $ $&:#" &.11(DA  ~r.cFSSKJn U[R:XaU"5$g)zB Get sandbox (code execution) configuration for a given provider. r)E2BSandboxConfigN)'litellm.llms.e2b.sandbox.transformationr riE2B)rr s r,get_provider_sandbox_config1ProviderConfigManager.get_provider_sandbox_config&s# M '++ +#% %r.cDSSKJn [RRU:Xa%UR S5(a SSKJn U"5$g [RRU:Xa SSK J n U"5$[RRU:Xa SSK J n U"5$[RRU:Xa SSKJn U"5$[RR"U:Xa SSKJn U"5$[RR(U:Xa SS KJn U"5$g ) z8 Get text-to-speech configuration for a given provider. rrzspeech/)AzureAVATextToSpeechConfig)ElevenLabsTextToSpeechConfig)RunwayMLTextToSpeechConfig)VertexAITextToSpeechConfig)MinimaxTextToSpeechConfig)AWSPollyTextToSpeechConfigN)3litellm.llms.base_llm.text_to_speech.transformationrrFr^r-r50litellm.llms.azure.text_to_speech.transformationr r 5litellm.llms.elevenlabs.text_to_speech.transformationr rP 3litellm.llms.runwayml.text_to_speech.transformationr r4litellm.llms.vertex_ai.text_to_speech.transformationr r2litellm.llms.minimax.text_to_speech.transformationr AWS_POLLY4litellm.llms.aws_polly.text_to_speech.transformationr ) r_rrr r r r r r s r,"get_provider_text_to_speech_config8ProviderConfigManager.get_provider_text_to_speech_config &s     % % 1 **233 +H= ! ! , , 8 01 1  ! ! * *h 6 ./ /  ! ! + +x 7 ./ /  ! ! ) )X 5 -. .  ! ! + +x 7 ./ /r.c[RRU:Xa SSKJn U"5$[RR U:Xa*SSKJn SSKJ n URU5(agU"5$g)Nr)GoogleGenAIConfig)VertexAIGoogleGenAIConfig)VertexAIPartnerModels) rFr^r/litellm.llms.gemini.google_genai.transformationr r2litellm.llms.vertex_ai.google_genai.transformationr 4litellm.llms.vertex_ai.vertex_ai_partner_models.mainr is_vertex_partner_model)r_rr r r s r,1get_provider_google_genai_generate_content_configGProviderConfigManager.get_provider_google_genai_generate_content_configB&sj    & &( 2 %& &  ! ! + +x 7  %</RSI 56 5 ==3 4K -- |#  @ @ L r.cU(dgSU;=(a SU;nURS5=(d U=(a URS5(+$)NFmax_chunks_per_docmax_tokens_per_docz /v1/rerankz /v2/rerank)endswith)rro uses_v1_paramss r,r` r` &sV *.DD$::   \ * >x00>>r.cUcUcgUbURU5(dUS-U-n[UUUSS9nU[:$![a#n[R "SU35 SnAgSnAff=f)z Returns true if the prompt is valid for prompt caching. OpenAI + Anthropic providers have a minimum token count of 1024 for prompt caching. NFr1T)r{rr_rz)Error in is_prompt_caching_valid_prompt: )r5rr$rrr)r_r{rr` token_countrs r,is_prompt_caching_valid_promptr &s     *53C3C 4 4 (#-5E#*.  >>> HLMs?5? A, A''A,cSn[R"X5nU(dg/nUH;n[[U5upVpxUS-US--U-US- -n UR U 5 M= U(a [ U5$S$)z Extracts the total duration (in seconds) from SRT or VTT content. Args: srt_or_vtt_content (str): The content of an SRT or VTT file as a string. Returns: Optional[float]: The total duration in seconds, or None if no timestamps are found. z"(\d{2}):(\d{2}):(\d{2})[.,](\d{3})Nirg@@)rQfindallrnrrr) srt_or_vtt_contenttimestamp_pattern timestamps durationsmatchhoursminutesseconds milliseconds total_secondss r, extract_duration_from_srt_or_vttr &s>-BJ I03C- w|3g= v@UU ' '3y>0D0r.cj[R"U5nUR0S9nURR S5nUR S5nUR S5Vs/sH of(dM UPM nnUR S5Vs/sH of(dM UPM nn/n [[U55H$n XzSUS[U5U - :XdMUSU U-n O Xx-n SSRU 5-n URU S9n [U RURS95$s snfs snf)z Adds an ending path to an API base URL while preventing duplicate path segments. Args: api_base: Base URL string ending_path: Path to append to the base URL Returns: Modified URL string with proper path handling )paramsr1Nr) r rrrrstriplstriprdrrr rpr ) r ending_pathrbase_url base_pathend_pathr base_segments end_segmentsfinal_segmentsr modified_pathrs r,_add_path_to_api_baser$ &s899X&L%%R%0H!!((-I!!#&H!* 4: 41Q 4M:'~~c282!aA2L8N 3}% &   -Es=/AA/E F F*2A.=N ' '5#((>22M%%=%9L |%%\-@-@%A BB%;8s% D+3D+ D0D0cUR5VVs0sH"upU[R;dMUcM X_M$ snn$s snnfr#)rLrFOPENAI_CHAT_COMPLETION_PARAMS)r r%r&s r,get_standard_openai_paramsr' &sKLLN "DA 55 5 :; "  sAAAc[RnU[-nUR5VVs0sHup4X2;dM X4_M nnnU$s snnfr#)rFr& rqrL)r:rrwr%r&rxs r,!get_non_default_completion_paramsr) &sR99M"%77N'1+B  s AAcSU;aUS$SU;aUS$URS5n[U[5(aSU;aUS$SU;aUS$g)zRead AI-SDK-style reasoning summary from optional_params or nested extra_body. Uses key membership (not ``or`` chains) so falsy values like ``""`` are not skipped. reasoningSummaryreasoning_summaryrN)r9r%rr)rrs r,peek_reasoning_summary_aliasesr- 'st _,122o-233 $$\2J*d##  +01 1 * ,12 2 r.cn[U5nURSS5nURSS5nUcUnURS5n[U[5(aY[U5nURSS5nURSS5nUcUnUcUnU(aXAS'X4$URSS5 X4$)zTCopy optional_params; remove reasoningSummary aliases from top-level and extra_body.r+ Nr, r)rrrr9r%)roprs_val snake_rs_valeb eb_rs_valeb_snake_rs_vals r,4strip_reasoning_summary_aliases_from_optional_paramsr5 's o B VV& -F66-t4L ~  B"d "XFF-t4 &&!4d; >F~( !|  : FF< & :r.cSSKJn U[-nUR5VVs0sHup4X2;dM X4_M nnnU$s snnf)Nrr{)rr|rqrL)r:r|rwr%r&rxs r,$get_non_default_transcription_paramsr7 1'sB=03EEN+1<<>U>41Q=T$!$>U Vs ==cJUcgUR5VVs0sH0upUS:wdM [U[5(dM$[U5U_M2 nnn[U5S:a.0nSnUR5HupUS:aX$U'US- nM UnUR 5$s snnf)a< Add metadata to openai optional parameters, excluding hidden params. OpenAI 'metadata' only supports string values. Args: params (dict): Dictionary of API parameters metadata (dict, optional): Metadata to include in the request Returns: dict: Updated parameters dictionary with visible metadata only N hidden_paramsrr)rLr%rprrS)r5r%r&visible_metadatafiltered_metadatargs r,add_openai_metadatar= 9'sNN$($DA   $.q#$6 A $( r!$**,DARx'(!$ 1HC--  ""!(s BBBcU(dgURS5n[U[5(a[U5nU(aU$[U5nU(aU$g)Nr6)r9r%rrr= )r5r6cleaned_metadatas r,get_requester_metadatar@ ^'sR !&:;$d++./AB # #*84 r.c hSSKJn SSKJn U"SSSS./SS S UR"5S S S 9n[ [ UR 5nS nU"S0UDUSS.D6 URRS5nU(a[[U5$[US9$![an[U5nSnANUSnAff=f)z Return the json str of the request This is currently in BETA, and tested for `/chat/completions` -> `litellm.completion` calls. rr9)Loggingz gpt-3.5-turborhirFr1234T)r_r{rrrrrlog_raw_request_responserzmy-fake-api-key)r2rNraw_request_typed_dict)rr) rPrrB rQrFrFrPrrprvr9rrh) endpointr:rPrB r2llm_api_endpointreceived_exceptionrrF s r,return_raw_requestrJ o's "B!!d34<<>!% w7$  3% 1CCGG ')?@@"$   $ V$s B B1 B,,B1c/nUHun[U[5(aURSS9nO%[U[5(aUR 5n[U[5(dMdUR U5 Mw U$)zb Fixes https://github.com/BerriAI/litellm/issues/9321 Where user passes in a pydantic base model Tr)r%rrlrrrSrrs r,rr'sk I dI & &???5D d # #99;D dD ! !   T "  r.c[SSSS9$)Nr) prompt_tokenscompletion_tokensr)rprr.r,get_empty_usagerO 's  r.c0U(dU(dU(aggrqr)rmock_tool_calls mock_timeouts r,should_run_mock_completionrS 's < r.clSSKJn U"5nX;a X nU"U5$[S[<SU<35e)zSLazy import handler for utils module with cached registry for improved performance.r)_get_lazy_import_registryzmodule z has no attribute )litellm._lazy_importsrU AttributeErrorrI)rrU registry handler_funcs r, __getattr__rZ 'sC@(*H ~ D!! 78,.@I JJr.)Fr)rrrz!Literal['DEBUG', 'INFO', 'ERROR'])rrprzLiteral['success', 'failure']rr)rz'CustomLogger'rr)r:Dict[str, Any]rr)r:rr)rUz4Optional[List[Union[str, Callable, 'CustomLogger']]]rr|)r_rr`rrr)rerprfrprrp)rqrrrrrprrr)r{r|rrrprr|)rrp)rrrr)r:r[ rrrz$Tuple[Optional[int], Dict[str, Any]])r:r[ rrrz*Optional[Union[float, int, httpx.Timeout]]rN)r:r[ rrp)rrrrrxrzOptional[CallTypes]r Optional[Any])rrr)r:rrrrr)r:r[ rzUnion[CallTypes, str]rrr#)r_rprz$Optional[CustomHuggingfaceTokenizer])r_rprrk)r_rprz!Optional[SelectTokenizerResponse])rrN)rr)r List[int]rrrr)rrrr] rr] )mainN)rrprr)rrp) rNNNFNNFN)rz.Optional[Union[dict, SelectTokenizerResponse]]rOptional[Union[str, List[str]]]r{Optional[List]rOptional[bool]r'Optional[List[ChatCompletionToolParam]]rz,Optional[ChatCompletionNamedToolChoiceParam]rra r Optional[int]rr)r`rprr)r_rpr`rrr)r_rpr`rrOrprzOptional[Literal[True]])r_rpr`rrOrprr)r_rpr`rrr)r#rzr$rrrrr)rOrprrprzOptional[Dict[str, Any]])rzUnion[str, dict])rvrrrwrrrsrrrr)NNNNNN)r_rpr`rpr~rrrrrrrc rz*Optional[List[Literal['word', 'segment']]]r}ra )rrrrp) NNNNNNNNNNNN)r_rrrc rrrrrrrrrrrrr`rrsrrr r}ra )NNNrNNN)r_rprrrrrrc r}ra rsrrr)r)rxrrrrrrr)r:rrrrr)FFN)rvrrrrrr`rprsrr_rprrrrrr rrr)r,rrrrr)rvrrrxrrr`rprrr)&NNNNNFNNNNNNNNNNNrNNNNNNNNNNNNNNNNNNNN)r_rprrr{ Optional[List[AllMessageValues]]rz Optional[AnthropicThinkingParam]rz Optional[OpenAIWebSearchOptions]rrrCr) rrrrvrrr`rprrrsrrrr)rrrrxrrrrn)rvrrrrr) rrc rrc rrc rrc rrc )rzUnion[Dict, Any]rrc )r List[Dict]rrc rr|)rre rzOptional[Iterable[str]]rr|)r`rprrrr)rrrzOptional[AllowedModelRegion])rrrr)rrrrprr)rrrrrr)r:rrrrp)rrprr)rz)Union[ModelResponse, ModelResponseStream]rrp)r5rpr6r)r_rprrc rM)rFrprrp)r_rpr`rrrp)rrr~)rzDict[str, str])rirprr)rorprr)rOrprrr)rrrr`rrr)r_rpr`rrr{)rFrprrc )r_rpr`rrrrrc)r_rpr`rrz#Optional[ProviderSpecificModelInfo])rr{rr)NNN) r_rpr`rrrrrrrc) r_rpr`rrrrrrrbr)r_rpr`rrrrrb)rrp)rrr)rrrrr) r_rpr`rprrprrpr rrr)r`rprzList[ProviderField])NNNN) r_rrrrrr rrrr)r_rprrp)rr)rOptional[httpx.Headers]r) rrrLrrrf rrrzUnion[float, int]) r_rprrrrrprrprrr)rrc )rz0Union[str, 'MockException', ModelResponseStream]rrc )r_rr r)r_rrrrr)r`rrr)rrr`rprrOrr)NNNNN) rcra r`rrrOrrrrrr)rrHrr)rrprr)rrprrp)r{List[AllMessageValues]rr)r`rprzList[ChatCompletionToolParam])rzUnion[BaseModel, dict]rrr)r{r|)rr` rzOptional[List[dict]])r"Optional['AnthropicThinkingParam']rrh )rrH)r{rg )rOptional[Union[dict, str]]rri )rr_ rr_ )rr[ )r_rprr^rr[ )r)rrrr z(Literal['litellm_logging', 'prometheus']rr)rrro r) r_rpr{rd rrb r`rrr)r rprr)rrpr rprrp)r rrrrr)rrrrr\ )rrrrzTuple[dict, Optional[Any]])r5zOptional[Mapping[str, Any]]rr)r5rr)rG rRr:rrrrh)rz List[Any]rre )rrp)rr\ rQ r\ rR r\ rr)rrprrx(9__doc__ __future__rrrXrbinasciirZrSrPr;rio itertoolsrloggingr-rrQstruct subprocessrGtextwrap threadingrrc dataclassesrr functoolsrr importlibrr r os.pathr r r dotenvr r_tiktokenr httpx._utilsr openai.librr*openai.types.chat.completion_create_paramsrpydanticrr tokenizersrrFlitellm.litellm_core_utils/litellm.litellm_core_utils.json_validation_rulelitellm._internal_contextrrV rrr litellm._uuidrrrrrrrr r!r"r#r$r%r&r8r>r2r*r-r4r:r?rArGrrHrIrJrKrLrMrNrOlitellm.types.utilsrPrQrRrSrTrUrVrWrXrYrZr[r\r]r^r_r`rarbrcrdrerfrgrhrirjrkrlrmrnrorprqrPrsrrrrfr json_data ImportErrorrW r% open_textdumpsrimportlib.metadatatypingrwrxryrzr{r|r}r~rrrrrr litellm.llms.base_llm.base_utilsrrr9r7r=r+r)*litellm.llms.base_llm.files.transformationr2litellm.llms.base_llm.realtime.http_transformationrlitellm.proxy._typesr2litellm.litellm_core_utils.exception_mapping_utilsrrrr6litellm.litellm_core_utils.get_supported_openai_paramsrFlitellm.litellm_core_utils.llm_response_utils.convert_dict_to_responserrrrr:litellm.litellm_core_utils.llm_response_utils.get_api_baser?litellm.litellm_core_utils.llm_response_utils.response_metadatarrr*litellm.litellm_core_utils.redact_messagesrr,litellm.litellm_core_utils.streaming_handlerr1litellm.llms.base_llm.google_genai.transformationr(litellm.llms.base_llm.ocr.transformationr,litellm.llms.base_llm.sandbox.transformationr+litellm.llms.base_llm.search.transformationrr rrUr litellm.llms.cohere.common_utilsr'litellm.llms.mistral.ocr.transformationr)litellm.litellm_core_utils.cached_importsrrrrrrr0litellm.litellm_core_utils.dot_notation_indexingrr-litellm.litellm_core_utils.get_litellm_paramsrr,litellm.litellm_core_utils.llm_request_utilsrBlitellm.litellm_core_utils.llm_response_utils.get_formatted_promptr9litellm.litellm_core_utils.llm_response_utils.get_headersrr litellm.litellm_core_utils.rulesr/litellm.litellm_core_utils.thread_pool_executorr7litellm.llms.base_llm.anthropic_messages.transformationr8litellm.llms.base_llm.audio_transcription.transformationr&litellm.llms.custom_httpx.http_handlerrr*litellm.router_utils.get_retry_from_policyrr,litellm.llms.base_llm.batches.transformationr/litellm.llms.base_llm.containers.transformationr.litellm.llms.base_llm.embedding.transformationr/litellm.llms.base_llm.image_edit.transformationr5litellm.llms.base_llm.image_generation.transformationr5litellm.llms.base_llm.image_variations.transformationr0litellm.llms.base_llm.passthrough.transformationr-litellm.llms.base_llm.realtime.transformationr+litellm.llms.base_llm.rerank.transformationr1litellm.llms.base_llm.vector_store.transformationr7litellm.llms.base_llm.vector_store_files.transformationr+litellm.llms.base_llm.videos.transformationrlitellm.types.llms.anthropicrrlitellm.types.rerankrrrrrbrlitellm.secret_managers.mainr)litellm.llms.base_llm.chat.transformationr/litellm.llms.base_llm.completion.transformationr*litellm.llms.base_llm.evals.transformationr.litellm.llms.base_llm.responses.transformationr+litellm.llms.base_llm.skills.transformationr_loggingrrcaching.cachingrrrrrrrarrrrrrrrrrrrrrrrsentry_sdk_instancecapture_exceptionrposthog slack_appalerts_channelheliconeLogger athinaLoggerpromptLayerLoggerlangsmithLogger logfireLoggerweightsBiasesLogger customLoggerlangFuseLoggeropenMeterLogger lagoLogger dataDogLoggerprometheusLogger dynamoLoggers3LoggergreenscaleLogger lunaryLogger aispendLoggersupabaseClientrrrrlast_fetched_atlast_fetched_at_keysrrr rrr<rBrQrWrargrvryrrrrr r rrr,rrLr0rrrrrrrrrrrrrrrrrrrrrrrrrrr r rrrrrrrrr(r!r4rUr>rortryrrrrrrrrrrr$r!rIrrrrrrrrrrrrrrrr rr7r=rNrSrVrXr6r^r_rarWrgrkrmrrrrxtyping_extensionsryr{rrIrerrrrrcrTrd cache_inforr rr$r)r6r}rrrrrrrrrrrrrrrrrrrlitellm.caching.in_memory_cacher r"r^r[r`rfrOrtryr{rrrrrrrrrrrrrrrrrrrrrrrrrrrr r` r r r$ r' r) r- r5 r7 r= r@ rJ rrO rS rZ )cts0r,r skM#        (&** 0*E !76         #      *"""""""""H5>>IbXXr\I>T> ! ?@ , - cG $ %()IIaL  %**Y'$ @J8RXQGNLC@H  X7HU OSSRRSSWVPPLLWWLK43 4333@@TTIIQQKK66$%            %' "'/1,1(* %*23:  1 ( E=="?= =@ "    "    ( $K  ;  14   F 14 @]]@  % #'  D '0)6   '0 / < 2"%2E.r$rjl b"  @"))**))//**00  .  .$ .  . JN 1  1"F 1 -. +/ +N.2 '+ $  % 8II"+II&CGEE2?E6 E GK,0#,159@D49)-$D$ *$ $ * $ 3 $>$$2$'$ $N ***D6:* *%2* *\6:   %2   6: %2 .   %2 9<   6r& &%2&9<& &R6: %2 6: %2 06: %2 .$6:   %2   6:) )%2))Z6:   %2   &       FQh  )- GU   &# %)!%JN"&Z ZZZ  Z # Z  ZHZ Zz( !%)"&)--1;?"&E EEE# E  E  E E E'E+E9E EV%) $"&2615n n n# n  n n0n/nb .*B7J "0J"J"f#().,0CCCC0 C  C  C#'C*C CL  ``-1`HK` `N       151515;?'+ $SF  F B/CF J/KF L/MF N9OF P%QF RSF \.2 9999 9 + 9  9x/3LP2  ((( ( ($1 (  (V DH#3@ D8<#4 6&&.<&&6,^  ":= "*74 +#\/d!; |, 7 3 (7;3: ()(+ 0%&&&R#,^('i+ +%2+,+\. -.# & /" %2(( B   *."! P P &P P  P  P j *."! U U&UU U  U> -.*." & /*."! PH PH&PHPH PH  PHf4??2==G4Xv " ( ( ( (   (  (  ( V  1: !"!% fV fV fVfV fV  fVR 0$ H.<15-J15 """." "  ": "  @& @& @&@& @&  @&FR%R%l>B-:< C < "D">4 9 TY6#06LP66| *#( j! j!j!! j!j!Z:9>C=>CD$% *.&<04%- 2/3)-/3!" O+O&O-O O  O  OOd1 h8##BB( .b&  $ .$ D* (I& 606'6"C(V!+!!J-1 )$2 1 $JJ^,>O:D  5>  6:)-  . 3'   <18#CL$2"#)"#"#J"* Z" "   K Kcw? % % ^Y/!   /1K  IIaL     !sN y#)2y:y(/y:( y72y:7y::{ z7+{ 7 { { {