]jשtdZddlZddlmZmZmZmZmZmZm Z ddl m Z m Z m Z mZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZm Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(m)Z)m*Z*ddl+m,Z,ddl-m.Z.ddl/m0Z1dd l/m2Z3dd l/m4Z5dd l/m6Z7dd l/m8Z9dd l/m4Z:dd l/m;Z<ddl/m=Z>ddl/m?Z@ddl/mAZBddlCmDZDGddZEdS)zs Main Firecrawl v2 API client. This module provides the main client class that orchestrates all v2 functionality. N)OptionalListDictAnyCallableUnionLiteral) ClientConfig ScrapeOptionsDocument SearchRequest SearchData SourceOptionCategoryOption CrawlRequest CrawlResponseCrawlJobCrawlParamsRequest PDFParserCrawlParamsData WebhookConfigAgentWebhookConfigCrawlErrorsResponseActiveCrawlsResponse MapOptionsMapData FormatOption WaitActionScreenshotAction ClickAction WriteAction PressAction ScrollAction ScrapeActionExecuteJavascriptAction PDFActionLocationPaginationConfig AgentOptions) HttpClient)FirecrawlError)scrape)crawl)batch)search)map)usage)extract)agent)browserWatcherc<eZdZdZededefdZ dd eed ed ee d e d e f dZ ddddddddddddddddddddddedee ddee eefdee edee edeed ee dee deedeee ee eeeffdee eddeddeedeedeed eed!eed"eed#ee d$eed%eedef,d&Zdddddddddd' d(ed)ee ed*ee ed+ee d,eedeed-eed ee d.eed%eedefd/Zddddddd0dd0d0d0ddddd0d0d1dddd2ded3eed4ee ed5ee ed6ee d7eed8d9eed:ed+ee d;edee d?ee d@eeeefd.eedAedBedCe d ee dDee d%eedef.dEZddddddd0dd0d0d0ddddd0d0ddFded3eed4ee ed5ee ed6ee d7eed8d9eed:ed+ee d;edee d?ee d@eeeefd.eedAedBed%eedef(dGZ dddHdIedJeedDee defdKZddHdLedDee defdMZdNede fdOZ!de"fdPZ#de"fdQZ$dddddddddRdedSeedTeed:eed+ee d7eed8d ee d%eedee%de&fdUZ'dNedefdVZ(ded3ede)fdWZ* ddddddddddddX dYee ed3eedZee ee+fd[eedd|Z?dddd}d~ee dee deefdZ@ddddedededd ee fdZAdefdZBdddeedfdZCdd1dddIededdCe d ee deDf dZEddddddddddddddddddddddddddd1dddYe edee ddee eefdee edee edeed ee dee deedeee ee eeeffdee eddeddeedeedeed eed!eed"eed#ee d$eed@eeeefddeed-eed?ee dBeed%eedeeedCe dee f:dZFdS)FirecrawlClientz| Main Firecrawl v2 API client. This client provides a clean, modular interface to all Firecrawl functionality. urlreturnc.d|vS)Nzapi.firecrawl.dev)lower)r:s [/home/rurouni/.hermes/hermes-agent/venv/lib/python3.11/site-packages/firecrawl/v2/client.py_is_cloud_servicez!FirecrawlClient._is_cloud_service@s"ciikk11Nhttps://api.firecrawl.dev?api_keyapi_urltimeout max_retriesbackoff_factorc|tjd}||r|stdt ||||||_t ||||||_dS)a Initialize the Firecrawl client. Args: api_key: Firecrawl API key (or set FIRECRAWL_API_KEY env var) api_url: Base URL for the Firecrawl API timeout: Request timeout in seconds max_retries: Maximum number of retries for failed requests backoff_factor: Exponential backoff factor for retries (e.g. 0.5 means wait 0.5s, then 1s, then 2s between retries) NFIRECRAWL_API_KEYzlAPI key is required for the cloud API. Set FIRECRAWL_API_KEY environment variable or pass api_key parameter.)rDrErFrGrH)rFrGrH)osgetenvr? ValueErrorr configr+ http_client)selfrDrErFrGrHs r>__init__zFirecrawlClient.__init__Ds$ ?i 344G  ! !' * * 7 -  ##)     &  #)    r@)formatsheaders include_tags exclude_tagsonly_main_contentrFwait_formobileparsersactionslocationskip_tls_verificationremove_base64_images fast_modeuse_mock block_adsproxymax_agestore_in_cache integrationrRrrSrTrUrVrWrXrYrZ) rr r!r"r#r$r%r&r'r[r(r\r]r^r_r`rarbrcrdcntd|||||||| | | | | ||||||||fDrltdidtdid|d|d|d|d|d|d |d | d | d | d | d| d|d|d|d|d|d|d|d|Dnd}t j|j||S)ab Scrape a single URL and return the document. Args: url: URL to scrape formats: List of formats to scrape headers: Dictionary of headers to use include_tags: List of tags to include exclude_tags: List of tags to exclude only_main_content: Whether to only scrape the main content timeout: Timeout in milliseconds wait_for: Wait for a specific element to be present mobile: Whether to use mobile mode parsers: List of parsers to use actions: List of actions to perform location: Location to scrape skip_tls_verification: Whether to skip TLS verification remove_base64_images: Whether to remove base64 images fast_mode: Whether to use fast mode use_mock: Whether to use mock mode block_ads: Whether to block ads proxy: Proxy to use max_age: Maximum age of the cache store_in_cache: Whether to store the result in the cache Returns: Document c3K|]}|duV dSN.0vs r> z)FirecrawlClient.scrape..s>\\1$\\\\\\r@ci|] \}}||| Srgrhrjkrks r> z*FirecrawlClient.scrape..s.(((1*+!*'r@rRrSrTrUrVrFrWrXrYrZr[r\r]r^r_r`rarbrcrdNrh)anyr dictitems scrape_moduler-rO)rPr:rRrSrTrUrVrFrWrXrYrZr[r\r]r^r_r`rarbrcrdoptionss r>r-zFirecrawlClient.scrapeosT\\'<Whjqs{~DFMOVX`bwyMOXZbdmotv}MOZ)[\\\\\/f-  ((!!!!!*\!*\ ! #4"3 !   !"!v! ! !"!'<&;!&:%9!$)!"! $)!!"e#!$ %!& .~'!((K)!*egg+(((   .bf/ 0#D$4c7CCCr@) sources categorieslimittbsr[ignore_invalid_urlsrFscrape_optionsrdqueryrvrwrxryrzr{c ht||||||||| |  } tj|j| S)a Search for documents. Args: query: Search query string limit: Maximum number of results to return (default: 5) tbs: Time-based search filter location: Location string for search timeout: Request timeout in milliseconds (default: 300000) page_options: Options for scraping individual pages Returns: SearchData containing the search results ) r|rvrwrxryr[rzrFr{rd)r search_moduler0rO) rPr|rvrwrxryr[rzrFr{rdrequests r>r0zFirecrawlClient.searchsN8 ! 3)#    #D$4g>>>r@F)prompt exclude_paths include_pathsmax_discovery_depthsitemapignore_sitemapignore_query_parametersrxcrawl_entire_domainallow_external_linksallow_subdomainsdelaymax_concurrencywebhookr{regex_on_full_urlzero_data_retention poll_intervalrFrequest_timeoutrdrrrrr)onlyincludeskiprrrrrrrrrrrrc|}|||rdnd}id|d|d|d|d|d |d | d | d | d | d| d|d|d|d|d|d|}|||d<tdi|}tj|j||||S)aR Start a crawl job and wait for it to complete. Args: url: Target URL to start crawling from prompt: Optional prompt to guide the crawl exclude_paths: Patterns of URLs to exclude include_paths: Patterns of URLs to include max_discovery_depth: Maximum depth for finding new URLs sitemap: Sitemap usage mode ("only" | "include" | "skip") ignore_sitemap: Deprecated alias for sitemap ("skip" when true, "include" when false) ignore_query_parameters: Ignore URL parameters limit: Maximum pages to crawl crawl_entire_domain: Follow parent directory links allow_external_links: Follow external domain links allow_subdomains: Follow subdomains delay: Delay in seconds between scrapes max_concurrency: Maximum number of concurrent scrapes webhook: Webhook configuration for notifications scrape_options: Page scraping configuration regex_on_full_url: Apply includePaths/excludePaths regex to the full URL (including query parameters) instead of just the pathname zero_data_retention: Whether to delete data after 24 hours poll_interval: Seconds between status checks timeout: Maximum seconds to wait for the entire crawl job to complete (None for no timeout) request_timeout: Timeout (in seconds) for each individual HTTP request, including pagination requests when fetching results. If there are multiple pages, each page request gets this timeout Returns: CrawlJob when job completes Raises: ValueError: If request is invalid Exception: If the crawl fails to start or complete TimeoutError: If timeout is reached Nrrr:rrrrrrxrrrrrrr{rrrdr)rrFrrh)r crawl_moduler.rO)rPr:rrrrrrrrxrrrrrrr{rrrrFrrdresolved_sitemaprequest_kwargsrs r>r.zFirecrawlClient.crawlsLx#  #(B)7FvvY  3 f  ]  ]  "#6  &'>   U  "#6  #$8   0  U    w  n  !2 "#6! " ;# &  '(8N9 %0000!   '+     r@)rrrrrrrrxrrrrrrr{rrrdc|}|||rdnd}id|d|d|d|d|d |d | d | d | d | d| d|d|d|d|d|d|}|||d<tdi|}tj|j|S)a Start an asynchronous crawl job. Args: url: Target URL to start crawling from prompt: Optional prompt to guide the crawl exclude_paths: Patterns of URLs to exclude include_paths: Patterns of URLs to include max_discovery_depth: Maximum depth for finding new URLs sitemap: Sitemap usage mode ("only" | "include" | "skip") ignore_sitemap: Deprecated alias for sitemap ("skip" when true, "include" when false) ignore_query_parameters: Ignore URL parameters limit: Maximum pages to crawl crawl_entire_domain: Follow parent directory links allow_external_links: Follow external domain links allow_subdomains: Follow subdomains delay: Delay in seconds between scrapes max_concurrency: Maximum number of concurrent scrapes webhook: Webhook configuration for notifications scrape_options: Page scraping configuration regex_on_full_url: Apply includePaths/excludePaths regex to the full URL (including query parameters) instead of just the pathname zero_data_retention: Whether to delete data after 24 hours Returns: CrawlResponse with job information Raises: ValueError: If request is invalid Exception: If the crawl operation fails to start Nrrr:rrrrrrxrrrrrrr{rrrdrrh)rr start_crawlrO)rPr:rrrrrrrrxrrrrrrr{rrrdrrrs r>rzFirecrawlClient.start_crawlGs6j#  #(B)7FvvY  3 f  ]  ]  "#6  &'>   U  "#6  #$8   0  U    w  n  !2 "#6! " ;# &  '(8N9 %0000'(8'BBBr@rjob_idpagination_configc<tj|j|||S)av Get the status of a crawl job. Args: job_id: ID of the crawl job pagination_config: Optional configuration for pagination behavior request_timeout: Timeout (in seconds) for each individual HTTP request. When auto-pagination is enabled (default) and there are multiple pages of results, this timeout applies to each page request separately, not to the entire operation Returns: CrawlJob with current status and data Raises: Exception: If the status check fails )rr)rget_crawl_statusrO)rPrrrs r>rz FirecrawlClient.get_crawl_statuss-.,   /+     r@next_urlc:tj|j||S)a7 Fetch a single page of crawl results using a next URL. Args: next_url: Opaque next URL from a prior crawl status response request_timeout: Timeout (in seconds) for the HTTP request Returns: CrawlJob with the page data and next URL (if any) r)rget_crawl_status_pagerOrPrrs r>rz%FirecrawlClient.get_crawl_status_pages* 1   +    r@crawl_idc6tj|j|S)a Retrieve error details and robots.txt blocks for a given crawl job. Args: crawl_id: The ID of the crawl job Returns: CrawlErrorsResponse containing per-URL errors and robots-blocked URLs )rget_crawl_errorsrOrPrs r>rz FirecrawlClient.get_crawl_errorss,T-=xHHHr@c4tj|jS)z Get a list of currently active crawl jobs. Returns: ActiveCrawlsResponse containing a list of active crawl jobs. )rget_active_crawlsrOrPs r>rz!FirecrawlClient.get_active_crawlss-d.>???r@c*|S)z List currently active crawl jobs for the authenticated team. Returns: ActiveCrawlsResponse containing the list of active crawl jobs )rrs r> active_crawlszFirecrawlClient.active_crawlss%%'''r@r0include_subdomainsrrxrrFrdr[r0rc td|||||||| fDrt||||||nd||| nd} tj|j|| S)a/Map a URL and return discovered links. Args: url: Root URL to explore search: Optional substring filter for discovered links include_subdomains: Whether to include subdomains ignore_query_parameters: Whether to ignore query parameters when mapping limit: Maximum number of links to return sitemap: Sitemap usage mode ("only" | "include" | "skip") timeout: Request timeout in milliseconds Returns: MapData containing the discovered links c3K|]}|duV dSrgrhris r>rlz&FirecrawlClient.map..s>OO1$OOOOOOr@Nrr)rqr map_moduler1rO) rPr:r0rrrxrrFrdr[rus r>r1zFirecrawlClient.mapsHOO1CE\^celnuxCEM)NOOOOO Y*1$;&2GG #     UY ~d.W===r@c6tj|j|S)z Cancel a crawl job. Args: crawl_id: The ID of the crawl job to cancel Returns: bool: True if the crawl was cancelled, False otherwise )r cancel_crawlrOrs r>rzFirecrawlClient.cancel_crawls()98DDDr@cXt||}tj|j|S)zDerive crawl parameters from natural-language prompt. Args: url: Root URL prompt: Instruction describing how to crawl Returns: CrawlParamsData with normalized crawl configuration )r:r)rrcrawl_params_previewrO)rPr:rrs r>rz$FirecrawlClient.crawl_params_preview s,%V<<<01A7KKKr@ rschema system_promptrenable_web_search show_sourcesr{rzrdr4urlsrrrrr r4c Ltj|j||||||||| | |  S)aStart an extract job (non-blocking). .. deprecated:: The extract endpoint is in maintenance mode and its use is discouraged. Review https://docs.firecrawl.dev/developer-guides/usage-guides/choosing-the-data-extractor to find a replacement. Args: urls: URLs to extract from (optional) prompt: Natural-language instruction for extraction schema: Target JSON schema for the output system_prompt: Optional system instruction allow_external_links: Allow hyperlinks in output enable_web_search: Whether to augment with web search show_sources: Include per-field/source mapping when available scrape_options: Scrape options applied prior to extraction ignore_invalid_urls: Skip invalid URLs instead of failing integration: Integration tag/name agent: Agent configuration Returns: Response payload with job id/status (poll with get_extract_status) r)extract_module start_extractrO) rPrrrrrrrr{rzrdr4s r>rzFirecrawlClient.start_extract-sFJ+   '!5/%) 3#    r@ rrrrrrr{rzrrFrdr4c Ptj|j||||||||| | | | | S)aExtract structured data and wait until completion. .. deprecated:: The extract endpoint is in maintenance mode and its use is discouraged. Review https://docs.firecrawl.dev/developer-guides/usage-guides/choosing-the-data-extractor to find a replacement. Args: urls: URLs to extract from (optional) prompt: Natural-language instruction for extraction schema: Target JSON schema for the output system_prompt: Optional system instruction allow_external_links: Allow hyperlinks in output enable_web_search: Whether to augment with web search show_sources: Include per-field/source mapping when available scrape_options: Scrape options applied prior to extraction ignore_invalid_urls: Skip invalid URLs instead of failing poll_interval: Seconds between status checks timeout: Maximum seconds to wait (None for no timeout) integration: Integration tag/name agent: Agent configuration Returns: Final extract response when completed r)rr3rO)rPrrrrrrrr{rzrrFrdr4s r>r3zFirecrawlClient.extractasLR%   '!5/%) 3'#    r@)rRrSrTrUrVrFrWrXrYrZr[r\r]r^r_r`rarbrcr append_to_idrzrrrdidempotency_keyrrcvtd|||||||| | | | | |||||||fDritdidtdid|d|d|d|d|d|d |d | d | d | d | d| d|d|d|d|d|d|d|Dnd}t j|j||||||||| S)aStart a batch scrape job over multiple URLs (non-blocking). Args: urls: List of URLs to scrape formats: Output formats to collect per URL headers: HTTP headers include_tags: HTML tags to include exclude_tags: HTML tags to exclude only_main_content: Restrict scraping to main content timeout: Per-request timeout in milliseconds wait_for: Wait condition in milliseconds mobile: Emulate mobile viewport parsers: Parser list (e.g., ["pdf"]) actions: Browser actions to perform location: Location settings skip_tls_verification: Skip TLS verification remove_base64_images: Remove base64 images from output fast_mode: Prefer faster scraping modes use_mock: Use a mock data source (internal/testing) block_ads: Block ads during scraping proxy: Proxy setting max_age: Cache max age store_in_cache: Whether to store results in cache webhook: Webhook configuration append_to_id: Append to an existing batch job ignore_invalid_urls: Skip invalid URLs without failing max_concurrency: Max concurrent scrapes zero_data_retention: Delete data after 24 hours integration: Integration tag/name idempotency_key: Header used to deduplicate starts Returns: Response payload with job id (poll with get_batch_scrape_status) c3K|]}|duV dSrgrhris r>rlz5FirecrawlClient.start_batch_scrape..>OO1$OOOOOOr@ci|] \}}||| Srgrhrns r>rpz6FirecrawlClient.start_batch_scrape...(((1()!('r@rRrSrTrUrVrFrWrXrYrZr[r\r]r^r_r`rarbrcN)rurrrzrrrdrrh)rqr rrrs batch_modulestart_batch_scraperO)rPrrRrSrTrUrVrFrWrXrYrZr[r\r]r^r_r`rarbrcrrrzrrrdrrus r>rz"FirecrawlClient.start_batch_scrapesnOO'<Whjqs{~DFMOVX`bwyMOXZbdmotv}M)NOOOOO-Y-  ((!!!!!*\!*\ ! #4"3 !   !"!v! ! !"!'<&;!&:%9!$)!"! $)!!"e#!$ %!& .~'!(egg)(((   ,UY- 0.   % 3+ 3#+    r@c:tj|j||S)aGet current status and any scraped data for a batch job. Args: job_id: Batch job ID pagination_config: Optional configuration for pagination behavior Returns: Status payload including counts and partial data )r)rget_batch_scrape_statusrO)rPrrs r>rz'FirecrawlClient.get_batch_scrape_statuss*3   /    r@c:tj|j||S)aBFetch a single page of batch scrape results using a next URL. Args: next_url: Opaque next URL from a prior batch scrape status response request_timeout: Timeout (in seconds) for the HTTP request Returns: BatchScrapeJob with the page data and next URL (if any) r)rget_batch_scrape_status_pagerOrs r>rz,FirecrawlClient.get_batch_scrape_status_pages*8   +    r@c6tj|j|S)zCancel a running batch scrape job. Args: job_id: Batch job ID Returns: True if the job was cancelled )rcancel_batch_scraperOrPrs r>rz#FirecrawlClient.cancel_batch_scrape*s/0@&IIIr@c6tj|j|S)zRetrieve error details for a batch scrape job. Args: job_id: Batch job ID Returns: Errors and robots-blocked URLs for the job ) batch_methodsget_batch_scrape_errorsrOrs r>rz'FirecrawlClient.get_batch_scrape_errors5s4T5EvNNNr@c6tj|j|S)aGet the current status (and data if completed) of an extract job. .. deprecated:: The extract endpoint is in maintenance mode and its use is discouraged. Review https://docs.firecrawl.dev/developer-guides/usage-guides/choosing-the-data-extractor to find a replacement. Args: job_id: Extract job ID Returns: Extract response payload with status and optional data )rget_extract_statusrOrs r>rz"FirecrawlClient.get_extract_status@s01A6JJJr@)rrd max_creditsstrict_constrain_to_urlsmodelrrrr)z spark-1-proz spark-1-minic Ftj|j|||||||| S)aTStart an agent job (non-blocking). Args: urls: URLs to process (optional) prompt: Natural-language instruction for the agent schema: Target JSON schema for the output (dict or Pydantic BaseModel) integration: Integration tag/name max_credits: Maximum credits to use (optional) model: Model to use for the agent ("spark-1-pro" or "spark-1-mini") webhook: Webhook URL or configuration for notifications Returns: Response payload with job id/status (poll with get_agent_status) )rrrdrrrr) agent_module start_agentrO) rPrrrrdrrrrs r>rzFirecrawlClient.start_agentPs<2'   ##%=    r@)rrdrrFrrrrc Jtj|j||||||||| |  S)aRun an agent and wait until completion. Args: urls: URLs to process (optional) prompt: Natural-language instruction for the agent schema: Target JSON schema for the output (dict or Pydantic BaseModel) integration: Integration tag/name poll_interval: Seconds between status checks timeout: Maximum seconds to wait (None for no timeout) max_credits: Maximum credits to use (optional) model: Model to use for the agent ("spark-1-pro" or "spark-1-mini") webhook: Webhook URL or configuration for notifications Returns: Final agent response when completed ) rrrdrrFrrrr)rr4rO) rPrrrrdrrFrrrrs r>r4zFirecrawlClient.agentusB:!   #'#%=    r@c6tj|j|S)zGet the current status (and data if completed) of an agent job. Args: job_id: Agent job ID Returns: Agent response payload with status and optional data )rget_agent_statusrOrs r>rz FirecrawlClient.get_agent_statuss,T-=vFFFr@c6tj|j|S)zCancel a running agent job. Args: job_id: Agent job ID Returns: True if the agent was cancelled )r cancel_agentrOrs r>rzFirecrawlClient.cancel_agents()96BBBr@c4tj|jS)zCGet current concurrency and maximum allowed for this team/key (v2).) usage_methodsget_concurrencyrOrs r>rzFirecrawlClient.get_concurrency,T-=>>>r@c4tj|jS)z-Get remaining credits for this team/key (v2).)rget_credit_usagerOrs r>rz FirecrawlClient.get_credit_usage-d.>???r@c4tj|jS)z$Get recent token usage metrics (v2).)rget_token_usagerOrs r>rzFirecrawlClient.get_token_usagerr@ by_api_keyc6tj|j|S)z!Get historical credit usage (v2).)rget_credit_usage_historicalrOrPrs r>rz+FirecrawlClient.get_credit_usage_historicals89I:VVVr@c6tj|j|S)z Get historical token usage (v2).)rget_token_usage_historicalrOrs r>rz*FirecrawlClient.get_token_usage_historicals78H*UUUr@c4tj|jS)z*Get metrics about the team's scrape queue.)rget_queue_statusrOrs r>rz FirecrawlClient.get_queue_statusrr@ttl activity_ttlstream_web_viewrrrc<tj|j|||S)aJCreate a new browser session. Args: ttl: Total time-to-live in seconds (30-3600, default 300) activity_ttl: Inactivity TTL in seconds (10-3600) stream_web_view: Whether to enable webview streaming Returns: BrowserCreateResponse with session id and CDP URL r)browser_moduler5rO)rPrrrs r>r5zFirecrawlClient.browsers-"%  %+     r@bashlanguagerF session_idcoder)pythonnoderc>tj|j||||S)aaExecute code in a browser session. Args: session_id: Browser session ID code: Code to execute language: Programming language ("python", "node", or "bash") timeout: Execution timeout in seconds (1-300, default 30) Returns: BrowserExecuteResponse with execution result r)rbrowser_executerO)rPrrrrFs r>rzFirecrawlClient.browser_executes0&-         r@c6tj|j|S)zDelete a browser session. Args: session_id: Browser session ID Returns: BrowserDeleteResponse )rdelete_browserrO)rPrs r>rzFirecrawlClient.delete_browsers,T-=zJJJr@statusr)active destroyedc8tj|j|S)zList browser sessions. Args: status: Filter by session status ("active" or "destroyed") Returns: BrowserListResponse with list of sessions r)r list_browsersrO)rPrs r>rzFirecrawlClient.list_browsers s'+      r@r.kindrrFr )r.r/c*t|||||S)aACreate a watcher for crawl or batch jobs. Args: job_id: Job ID to watch kind: Job kind ("crawl" or "batch") poll_interval: Seconds between status checks timeout: Maximum seconds to watch (None for no timeout) Returns: Watcher instance r r6)rPrr rrFs r>watcherzFirecrawlClient.watchers&tV$mU\]]]]r@)rRrSrTrUrVrFrWrXrYrZr[r\r]r^r_r`rarbrcrrrzrrrdrr wait_timeoutr cztd|||||||| | | | | |||||||fDritdidtdid|d|d|d|d|d|d |d | d | d | d | d| d|d|d|d|d|d|d|Dnd}t j|j||||||||||| S)zE Start a batch scrape job and wait until completion. c3K|]}|duV dSrgrhris r>rlz/FirecrawlClient.batch_scrape..nrr@ci|] \}}||| Srgrhrns r>rpz0FirecrawlClient.batch_scrape..Yrr@rRrSrTrUrVrFrWrXrYrZr[r\r]r^r_r`rarbrcN) rurrrzrrrdrrrFrh)rqr rrrsr batch_scraperO)rPrrRrSrTrUrVrFrWrXrYrZr[r\r]r^r_r`rarbrcrrrzrrrdrrr rus r>rzFirecrawlClient.batch_scrape4s%tOO'<Whjqs{~DFMOVX`bwyMOXZbdmotv}M)NOOOOO-Y-  ((!!!!!*\!*\ ! #4"3 !   !"!v! ! !"!'<&;!&:%9!$)!"! $)!!"e#!$ %!& .~'!(egg)(((   ,UY- 0(   % 3+ 3#+'     r@)NrANrBrCrg)F)G__name__ __module__ __qualname____doc__ staticmethodstrboolr?rfloatintrQrrrrr r-rrr rr0r rrr.rrr)rrrrrrrr(rr1rrrrr*rr3rrrrrrrrr4rrrrrrrrr5rrrr7r rrhr@r>r9r99s 2s2t222\2 "&2#' # ) ) #) ) % )  )  ) ) ) ) ^37,0,0,0,0!%"&!%KOAE)-04/3$("&$(#!%)-%)/KDKDKD KD$~./ KD $sCx.) KD tCy) KDtCy)KD$D>KD#KD3-KDKD%S 4c9n0E+F FGHKD$u&{ |}~KD:&KD (~KD 'tn!KD"D>#KD$3-%KD&D>'KD(})KD*#+KD,!-KD.c]/KD0 1KDKDKDKDb1559#!"&.2!%26%))?)?)?)?$|,- )? T.12 )? } )?c])?3-)?&d^)?#)?!/)?c])? )?)?)?)?^!%-1-1-1@D)-(-#$)%*!&#)-7;26"'$)!%+/%)1^ ^ ^ ^  ^  S * ^  S * ^ &c]^ '";<=^ !^ "&^ }^ "^ #^ ^ }^ "#!^ "%] 234#^ $!/%^ & '^ (")^ *+^ ,#-^ ."%/^ 0c]1^ 2 3^ ^ ^ ^ H!%-1-1-1@D)-(-#$)%*!&#)-7;26"'$)%)+QCQCQC QC QC  S * QC  S * QC&c]QC'";<=QC!QC"&QC}QC"QC#QCQC}QC "#!QC"%] 234#QC$!/%QC& 'QC(")QC*c]+QC, -QCQCQCQCl9= ,0     $$45 "%       D,0     "%       , I I1D I I I I@#7@@@@(3((((!%-126#@D!%%)'+&>&>&> &> &> %TN &> "*$ &>}&>'";<=&>#&>c]&>8$&> &>&>&>&>P ES ET E E E E L LS L_ L L L L%)2 !%+/'+/3,0'+48.2%)(,2 2 2 tCy!2  2 c3h( 2  } 2 'tn2 $D>2 tn2 !12 &d^2 c]2  %2 2 2 2 l%)8 !%+/'+/3,0'+48.2!%%)(,8 8 8 tCy!8  8 c3h( 8  } 8 'tn8 $D>8 tn8 !18 &d^8 8 #8 c]8  %8 8 8 8 |37,0,0,0,0!%"&!%KOAE)-04/3$("&$(#!%)-7;&*.2)-.2%))-;d d d 3id $~./ d $sCx.) d tCy) d tCy)d $D>d #d 3-d d %S 4c9n0E+F FGHd $u&{ |}~d :&d  (~d 'tn!d "D>#d $3-%d &D>'d (})d *#+d ,!-d .%] 234/d 0sm1d 2&d^3d 4"#5d 6&d^7d 8c]9d :"#;d d d d R9=   $$45    0,0     "%     * J# J$ J J J J Oc O O O OKKKKK$%)# !%%)%)37BF<@# # # tCy!#  #  # c] # c]# #+4.#  =>?# %%7 789# # # # N%)) !%%)!%%)37BF<@) ) ) tCy!)  )  ) c] ) ) #) c]) #+4.)  =>?) %%7 789) ) ) ) V Gs G G G G C3 C4 C C C C???@@@???WWdWWWWVVTVVVV@@@"&**.    c] sm  "$     :7=!%      23  #     6 K K K K K<@   !678    ,+2!% ^^^^&' ^  ^ # ^ ^^^^237,0,0,0,0!%"&!%KOAE)-04/3$("&$(#!%)-7;&*.2)-.2%))-&*?I I I 3iI $~./ I $sCx.) I tCy) I tCy)I $D>I #I 3-I I %S 4c9n0E+F FGHI $u&{ |}~I :&I  (~I 'tn!I "D>#I $3-%I &D>'I (})I *#+I ,!-I .%] 234/I 0sm1I 2&d^3I 4"#5I 6&d^7I 8c]9I :"#;I <=I >sm?I I I I I I r@r9)FrrKtypingrrrrrrr typesr r r rrrrrrrrrrrrrrrrrrr r!r"r#r$r%r&r'r(r)r*utils.http_clientr+utils.error_handlerr,methodsr-rtr.rr/rr0r~r1rrr2rr3rr4rr5rr r7r9rhr@r>r s  FFFFFFFFFFFFFFFFFF!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!D*)))))//////,,,,,,************,,,,,,&&&&&&++++++++++++......******......D D D D D D D D D D r@