_jr7SSKJrJrJrJr SSKJrJrJrJ r J r J r J r J r JrJrJr SSKJr SSKJr SSKJr SSKJr SSKrS \S \4S jrS \\\S \\ 4S jrS\\\4S \\\44SjrS\S \S \4SjrS SS.S\S\S\\S\\ S \4 Sjjjr!SS.S\S\S\\ S \4Sjjr"S SS.S\S\S\\ S\\S\\ S \\ 4 Sjjjr#S\S\S \$4Sjr%S\S \ S \ 4Sjr&S\S\S \ 4Sjr'S\S \4Sjr(g)!)OptionalDictAnyList) CrawlRequestCrawlJob CrawlResponseDocumentCrawlParamsRequestCrawlParamsData WebhookConfigCrawlErrorsResponseActiveCrawlsResponse ActiveCrawlPaginationConfig)handle_response_error)prepare_scrape_options)AsyncHttpClient)normalize_document_inputNrequestreturncUR(aURR5(d [S5eURbURS::a [S5eSUR0nUR(aURUS'UR b [ UR 5nU(aX!S'URbK[UR[5(aURUS'OURRSS 9US'URSSS 9nURSS5 URSS5 URS S5 S S SSSSSSSSSSSSSS.nUR5HupVXS;dM URU5X'M! URU5 [USS5b&[[US55R5US'U$)NURL cannot be emptyrzLimit must be positiveurlprompt scrapeOptionswebhookT) exclude_none)r exclude_unsetscrape_options includePaths excludePathsmaxDiscoveryDepthsitemapignoreQueryParametersdeduplicateSimilarURLscrawlEntireDomainallowExternalLinksallowSubdomainsignoreRobotsTxtrobotsUserAgentdelaymaxConcurrencyregexOnFullURLzeroDataRetention) include_paths exclude_pathsmax_discovery_depthr%ignore_query_parametersdeduplicate_similar_urlscrawl_entire_domainallow_external_linksallow_subdomainsignore_robots_txtrobots_user_agentr-max_concurrencyregex_on_full_urlzero_data_retention integration)rstrip ValueErrorlimitrr!rr isinstancestr model_dumppopitemsupdategetattr)rdataopts request_datafield_mappingssnakecamels S/home/rurouni/.local/lib/python3.13/site-packages/firecrawl/v2/methods/aio/crawl.py_prepare_crawl_requestrPs ;;gkk//11.//}} W]]a%7122 7;; D~~ X)%g&<&<= $( !" goos + +%ooDO%oo88d8KDO%%4t%LLUD!Xt$%t,''2#:$<2 4-..+-2N"',,.   &**51DK/ KK w t,8!''="ABHHJ] K data_listc/nU=(d /H?n[U[5(dM[U5nUR[ S0UD65 MA U$)N)rBdictrappendr )rR documentsdoc_data normalizeds rO_parse_crawl_documentsrZFsL "IOO h % %1(;J   X3 3 4$ rQbodyc dURS5(d[URSS55eURS5URSS5URSS5URSS5URS 5URS 5[URS /55S .$) NsuccesserrorUnknown error occurredstatus completedrtotal creditsUsed expiresAtnextrIr`rarb credits_used expires_atrerI)get ExceptionrZ)r[s rO_parse_crawl_status_responserkOs 88I  *BCDD((8$XXk1-'1%2hh{+ &txx';< rQclientcb# [U5nURSU5IShvN nURS:a [US5 UR 5nUR S5(a([ UR S5UR S5S9$[UR S S 55eN7f) a" Start a crawl job for a website. Args: client: Async HTTP client instance request: CrawlRequest containing URL and options Returns: CrawlResponse with job information Raises: ValueError: If request is invalid Exception: If the crawl operation fails to start z /v2/crawlNz start crawlr]idr)rorr^r_)rPpost status_coderjsonrir rj)rlrpayloadresponser[s rO start_crawlru^s%W-G[[g66Hs"h 6 ==?D xx DHHUODD DHHW&>? @@ 7s!B/B-B B/request_timeoutjob_idpagination_configrwc # URSU3US9IShvN nURS:a [US5 UR5n[ U5nUSnU(a UR OSnU(a"US(a[ UUSUUUS 9IShvN n[US US US US USU(dUSUS9$SUS9$NN37f)a Get the status of a crawl job. Args: client: Async HTTP client instance job_id: ID of the crawl job pagination_config: Optional configuration for pagination limits request_timeout: Timeout (in seconds) for each individual HTTP request. When auto-pagination is enabled (default) and there are multiple pages of results, this timeout applies to each page request separately, not to the entire operation Returns: CrawlJob with job information Raises: Exception: If the status check fails /v2/crawl/timeoutNrnzget crawl statusrITrervr`rarbrgrhrf)rirqrrrrk auto_paginate_fetch_all_pages_asyncr ) rlrxryrwrtr[rsrWr~s rOget_crawl_statusrws0ZZ*VH 5ZOOHs"h(:; ==?D*40GI8I%33dM0  FO  +    x +&g^,<($1WV_   8<  'P s"C CA9C C2C C next_urlc # URXS9IShvN nURS:a [US5 UR5n[ U5n[ USUSUSUSUS US US S 9$N_7f) a{ Fetch a single page of crawl results using the provided next URL. Args: client: Async HTTP client instance next_url: Opaque next URL from a prior crawl status response request_timeout: Timeout (in seconds) for the HTTP request Returns: CrawlJob with the page data and next URL (if any) Raises: Exception: If the request fails or returns an error response r|Nrnzget crawl status pager`rarbrgrhrerIrf)rirqrrrrkr )rlrrwrtr[rss rOget_crawl_status_pagers(ZZZBBHs"h(?@ ==?D*40G x +&g^,<( V_ V_  CsA7A5A A7initial_documentsc # UR5nUnSnU(a UROSnU(a UROSn U(a UROSn [R "5n U(aUbXx:aU$U b[R "5U - U :aU$UR XdS9IShvN n U RS:a4SSKn U RS5nURSSU R0S9 U$U R5n[U5nUS H(nU b[U5U :a OURU5 M* U b[U5U :aU$US nUS - nU(aMU$N![a U$f=f7f) a Fetch all pages of crawl results asynchronously. Args: client: Async HTTP client instance next_url: URL for the next page initial_documents: Documents from the first page pagination_config: Optional configuration for pagination limits request_timeout: Optional timeout (in seconds) for the underlying HTTP request Returns: List of all documents from all pages rNr|rn firecrawlzFailed to fetch next pagerq)extrarIre)copy max_pages max_results max_wait_timetime monotonicrirqlogging getLoggerwarningrrrkrjlenrV)rlrrryrwrW current_url page_countrrr start_timertrlogger page_data page_payloaddocuments rOrrs*"&&(IKJ0A!++dI3D#//$K7H%33dM!J   !z'> H E  %DNN,cN d . +MMO  7 BL %V,H'c)n .K   X & -  ##i.K*G  #6* a K +N =J  " % s>B/F1E12AF E3AF/F3 F=FFFc# URSU35IShvN nURS:a [US5 UR5nUR S5S:H$ND7f)z Cancel a crawl job. Args: client: Async HTTP client instance job_id: ID of the crawl job Returns: True if cancellation was successful Raises: Exception: If the cancellation operation fails r{Nrnz cancel crawlr` cancelled)deleterqrrrri)rlrxrtr[s rO cancel_crawlrs[]]Zx#899Hs"h7 ==?D 88H  ,, :sA!AAA!c# UR(aURR5(d [S5eUR(aURR5(d [S5eURURS.nUR SU5IShvN nUR S:a [ US5 UR5nURS5(d[URS S 55eURS 05n0nS S SSSSSSSSSSSSS.nUR5HupX;dM XXXi'M SU;a USn XS'SU;aUSUS'[S0UD6$N7f)aA Preview crawl parameters before starting a crawl job. Args: client: Async HTTP client instance request: CrawlParamsRequest containing URL and prompt Returns: CrawlParamsData containing crawl configuration Raises: ValueError: If request is invalid Exception: If the parameter preview fails rzPrompt cannot be empty)rrz/v2/crawl/params-previewNrnzcrawl params previewr]r^r_rIr1r2r3r%r4r5r6r7r8r9r:r;r!r=)r"r#r$r%r&r'r(r)r*r+r,r.rr0rrrT) rr?r@rrprqrrrrirjrFr ) rlrrsrtr[ params_data convertedmappingrNrMwks rOcrawl_params_previewr0ss ;;gkk//11.// >>!5!5!7!7122kkW^^? ==?D 88I  *BCDD((62&K "I''2!:"<24-..+)2G     *1I (K  #!)D#I )  'Y ''AFsB%E>'E<(BE> 3E>crawl_idcD# URSUS35IShvN nURS:a [US5 UR5nURSU5nURS/5URSURS /55S .n[ S 0UD6$N7f) z Get errors from a crawl job. Args: client: Async HTTP client instance crawl_id: ID of the crawl job Returns: CrawlErrorsResponse with errors and robots blocked Raises: Exception: If the error check operation fails r{z/errorsNrnzcheck crawl errorsrIerrors robotsBlockedrobots_blocked)rrrT)rirqrrrr)rlrrtr[rsrYs rOget_crawl_errorsrgsZZ*XJg >??Hs"h(<= ==?Dhhvt$G++h+!++ow{{CSUW7XYJ  , ,,@sB BBB c z# URS5IShvN nURS:a [US5 UR5nURS5(d[ URSS55eURS/5n/nUH|n[ U[ 5(dMURURS 5URS URS 55URS 5URS 5S.5 M~ [SUVs/sHn[S0UD6PM snS9$GNs snf7f)z Get active crawl jobs. Args: client: Async HTTP client instance Returns: ActiveCrawlsResponse with active crawl jobs Raises: Exception: If the active crawl jobs operation fails z/v2/crawl/activeNrnzget active crawlsr]r^r_crawlsroteamIdteam_idroptions)rorrrT)r]rrT) rirqrrrrjrBrUrVrr)rlrtr[ crawls_inrYcncs rOget_active_crawlsrsZZ 233Hs"h(;< ==?D 88I  *BCDD2&IJ  a     eeDk55155+;<uuU|55+   R\5]R\Bk6GB6GR\5] ^^!4 6^s)D;D3B D;)A0D;D6 .D;6D;)N))typingrrrrtypesrr r r r r rrrrrutils.error_handlerrutils.validationrutils.http_client_asyncrutils.normalizerrrUrPrZrCrkrufloatrrrboolrrrrrTrQrOrs,,    9667 -L-T-`htCy&9d8n tCH~ $sCx. AoA AA8593 (, 3 3 3 013 e_ 3  3t(, ! !!e_ !  !P59 G (, G GGH~G 01 G e_ G (^GT----*4(4(AS4(Xg4(n-?-c-FY-4_O_8L_rQ