Xj<dZddlZddlZddlZddlZddlZddlZddlmZm Z m Z ddl m Z ddl mZmZddlmZddlmZmZejeZd Zd Zd Zd Zd eZdZdZdZdZ ej!dZ"ej!dZ#de$de$fdZ%e dGddZ&e GddZ'e GddZ(de fdZ)d e$de$fd!Z*d e$d"e&de fd#Z+d e$d"e&de(dzfd$Z,dd%d e$d"e&d&e$d'e$d(e-d)e-dzd*e-dzd+ee'd,e.e$dzddfd-Z/d>d e$d"e&dzddfd.Z0d/e ddfd0Z1d/e d1e(ddfd2Z2ded3d4ed5e-d6e-de3fd7Z4d4ede3fd8Z5d9d:d4ed;e3de3fd<Z6d1e(d e$d"e&de$dzfd=Z7dS)?z?Best-effort local cache for SandboxPool hosts (host/pool mode).N)asdict dataclassfield)Path)AnyList) constants) WeakFileLockloggingg@iizhf-sandbox-pool-cache/vi@Bil(\iz[A-Za-z0-9._-]{1,128}z[0-9a-fA-F]{16,128}partsreturncdtg|R}tj|S)zAShort, stable digest of `parts` (which are joined unambiguously).)join_DIGEST_PREFIXhashlibsha256encode hexdigest)rmaterials OC:\Projects\Historical\venv\Lib\site-packages\huggingface_hub/_sandbox_cache.py_digestrDsD{{N3U3344H >(//++ , , 6 6 8 88T)frozenc eZdZUdZeed<eed<edzed<ededededzddfdZedefd Z dS) CacheContextalThe security context a cache entry belongs to. A cached entry is not just data: the fast path rebuilds a host transport from it and sends it the HF bearer plus the derived host token. So it may only ever be read back by the same *endpoint*, *credential* and *namespace* that wrote it -- anything else is a plain cache miss, which is what the docs promise and what keeps a `connect(namespace=...)` from being served hosts that belong to another namespace. `principal` is an opaque, stable id for the credential; the token itself is never stored nor used as a key (see `for_credential`). endpoint principalN namespacetokenrcJ||td|dd|S)aEBuild the context of a given HF token, identified by its fingerprint. The fingerprint is a local computation, which is what keeps a cache hit at zero round-trips -- resolving the credential to a user id would cost a `whoami` and defeat the purpose of the cache. It is derived from the token but does not contain it: only the digest is ever written to disk. A rotated or swapped credential therefore reads as a different principal and gets a cache miss (the cold path, which always works), rather than someone else's hosts. r N )rr r!)r)clsrr"r!s rfor_credentialzCacheContext.for_credential\s/sH U0K0KCRC0P\effffrcXtd|j|j|jpdddS)zHDigest of the whole context; the name of the cache subdirectory it owns.contextN)rrr r!)selfs rkeyzCacheContext.keyjs0y$-AUSUVVWZXZWZ[[r) __name__ __module__ __qualname____doc__str__annotations__ classmethodr&propertyr,rrrrJs  MMMNNNTz g gS gS4Z gTb g g g[ g\S\\\X\\\rrc^eZdZUdZeed<eed<eed<eed<eed<eed<dZeed <d S) CachedHostaCA single host Job of a pool, as last seen by some process. `base_url` + `nonce` are everything needed to rebuild the in-job server transport (`_SandboxServer`) without an `inspect_job` round-trip: the per-sandbox auth token is re-derived from the user's HF token and `nonce` (see `_derive_sandbox_token`). job_idownerbase_urlnoncecapacitylive updated_atN) r-r.r/r0r1r2intr?floatr5rrr7r7psd KKK JJJMMM JJJMMM IIIJrr7ceZdZUdZeed<eed<eed<eed<edzed<edzed<edzed <d Zeed <d Zeed <e e Z e e ed<eZeed<dZeed<dS) PoolCachezJCached view of one pool: its config (to boot new hosts) + its known hosts.pool_idimageflavorsandboxes_per_hostN max_hosts idle_timeoutr!r)rr()default_factoryhostsversionr>r?)r-r.r/r0r1r2r@rr(rlistrKrr7_CACHE_VERSIONrLr?rAr5rrrCrCsTT LLL JJJ KKKTz*TzHcGS#eD999E4 999!GS!!!JrrCc@ttjdz dz S)Nsandboxpools)rr HF_HOMEr5rr _pools_dirrSs  ! " "Y . 88rrDcjtfddDsdvrtdS)zAReturn `pool_id` if it is safe to use as a file name, else raise.c3 K|]}|vV dSNr5).0crDs r z!_check_pool_id..s' 5 5A1< 5 5 5 5 5 5r)/\r).z..zInvalid pool id: )any ValueError)rDs`r_check_pool_idr_sO 5 5 5 5!4 5 5 555:K9O9O8W88999 Nrr(cTt|jz t|dz S)z;Path of the cache file for `pool_id` in `context` (no I/O)..json)rSr,r_rDr(s rpool_cache_pathrcs) <<'+ %>'+B+B(I(I(I IIrcF t||}|dd5}tj|}dddn #1swxYwY|dt krdSd|dgD}td i|d|i}nD#t$rYdSt$r+}t d|d |Yd}~dSd}~wwxYwt||| }|#t d |d |d dS|S)aPReturn the cached view of `pool_id` for `context`, or `None`. Missing, corrupt, incompatible, mistyped and foreign-context files are all the same thing here: a cache miss, logged at debug level. The caller's cold path is always correct, so this never raises -- a bad cache costs latency, never an error mid-`create()`. rutf-8encodingNrLc&g|]}tdi|S)r5)r7rWhs r z#read_pool_cache..s"@@@Qa@@@rrKz+Ignoring unreadable sandbox pool cache for : rbz Ignoring sandbox pool cache for r\r5) rcopenjsonloadgetrNpoprCFileNotFoundError Exceptionloggerdebug_cache_rejection) rDr(pathfdatarKcacheereasons rread_pool_cacher~s w00 YYsWY - - 9Q<z#save_pool_cache..sPPPaahPPPrN) rDrErFrGrHrIr!rr(rKr?z'Could not write sandbox pool cache for rm)setrc_ensure_private_dirparentr r1 _LOCK_TIMEOUTr~rKr8rrrCr!rr,rMvaluestime _atomic_writertrurv)rDr(rErFrGrHrIrKrdeadrxexistingmergedhostr8r{r|s rsave_pool_cachers&  !CEEDQw00DK((( #d))g-} E E E ' '&w88HPPh,NHNNBPPPF + +&*t{## ) ) 64((((#5#)!+ ) 6==??++9;;   E $ & & &) ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' '* QQQ OwOOAOOPPPPPPPPPQs=A D/B9D" D/"D&&D/)D&*D// E$9 EE$c |&t||ddSt|d}tD]/}|r||z d0dS#t $r+}td|d|Yd}~dSd}~wwxYw)aHRemove the cache file(s) for `pool_id` (best-effort, never raises). Without a `context`, the pool's entry is removed from *every* context: deleting a pool is the one operation for which over-deleting is the safe direction (the cache is disposable), and the caller may not know which credential wrote the entry. NT missing_okraz(Could not delete sandbox pool cache for rm) rcunlinkr_rSiterdiris_dirrtrurv)rDr(filename directoryr|s rdelete_pool_cachers P   GW - - 4 4 4 E E E F$W--444#--// ? ?I!! ?X%--->>> ? ? PPP NNN1NNOOOOOOOOOPs&BA"B C B>>Crxc|ddtttjdz }|g|jRD].}||kr ||jvrdSt j|t/dS)zCreate the cache directory tree, readable by this user only. `mkdir`'s mode is masked by the umask and the tree may predate this code, so the modes are also set explicitly, from the leaf up to (and including) the sandbox directory. T)parentsexist_okmoderPN)mkdir _DIR_MODErr rRroschmod)rxrootrs rrrs  JJtdJ;;;  ! " "Y .D*T\**''   Y->!>!> EE I&&&&''rr{ctj|j|jdzd\}} t j|dd5}t jt||dd d d n #1swxYwYt j ||d S#t$r%t| d wxYw) ajWrite the cache via a temp file + `os.replace` so readers never see a partial file. `mkstemp` gives the temp file an unpredictable name and mode `0600` regardless of the umask (which `os.replace` then carries over to the final file), so no other user can read -- or pre-create and win a race on -- the host URLs and nonces on their way to disk. r\z.tmp)dirprefixsuffixwrfrgr )indentNTr) tempfilemkstemprnamerfdopenrodumprreplace BaseExceptionrr)rxr{fdtmprys rrrs4;ty3vVVVGB Yr3 1 1 1 2Q IfUmmQq 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3  S D))) s/B%A1% B1A55B8A59B/C)minimummaximumvaluerrcxt|to%t|t o||cxko|kncS)zJWhether `value` is a plausible count. `bool` is an `int`, but not a count.) isinstancer@bool)rrrs r _is_countr#sG eS ! ! a*UD*A*A&A agQVFaFaFaFaZaFaFaFaFaarct|ttfo*t|t od|cxko tkncSNr)rr@rAr_MAX_TIMESTAMP)rs r _is_timestampr(sK ec5\ * * k:eT3J3J/J kqTYOkOkOkOk]kOkOkOkOkkrF) allow_emptyrct|to,|st|dkot|tkSr)rr1len_MAX_STR)rrs r_is_textr,s7 eS ! ! `{'Dc%jj1n `#e**X`J``rcP|j|jkrdS|j|jkrd|jd|jS|j|jkrd|jd|jS|j|kr d|jd|St |jrt |jsdSt|j ds d |j S|j t|j ds d |j S|j %t|j t s d |j St|js d|jSt|jt"sdS|jD]W}t$t|jt*r|jnds d|jdcSt$t|jt*r|jndsd|jd|jcSt |js d|jdcSt|jt*rt2|js d|jdcSt|jrt|js d|jdcSt|js d|jdcSYd S)aoWhy this cache file must not be used, or `None` if it may be. Two jobs. First, the entry has to belong to the caller's security context -- same endpoint, same credential, same namespace -- which is what makes a cross-namespace or cross-principal hit a miss instead of a confidentiality bug. Second, a schema check. `dataclass` does not enforce types, so a well-shaped file with `"capacity": "50"` parses happily and only blows up much later, on `host.capacity - host.live`, as an uncaught `TypeError` in the middle of `create()`. Validating here turns that into the documented cache miss. z@it was written for a different endpoint, credential or namespacezit names endpoint z, not zit names namespace zit names pool z its image/flavor are not stringsr )rz0its sandboxes_per_host is not a positive count: Nz'its max_hosts is not a positive count: )rz/its idle_timeout is not a duration in seconds: z#its updated_at is not a timestamp: zits hosts are not a listr)zhost z is not a plausible job idz names a non-namespace owner: z has no URLz has no usable noncez& has a non-numeric capacity/live countz has no usable timestamp)r(r,rr!rDrrErFrrGrHrI _MAX_SECONDSrr?rrKrM _JOB_ID_RE fullmatchr8r1r9r:r; _NONCE_REr<r=)r{rDr(rs rrwrw0s } ##QQ ~)))PENPPGOSSS }B BBwBBB EK 2(>(>211 U-q 9 9 9_^%BZ^^^ "9U_a+P+P+P"LLLL %i8JT`.a.a.a%WASWWW ) * *JIU5EIII ek4 ( (*)) A A##:dk33O3O$WDKKUWXX ED4;DDD D D D##*TZ2M2M$UDJJSUVV UT4;TTdjTT T T T && 434;333 3 3 3$*c** =)2E2Edj2Q2Q =<4;<<< < < <'' Oy/C/C ON4;NNN N N NT_-- A@4;@@@ @ @ @ A 4rrV)8r0rrorrerr dataclassesrrrpathlibrtypingrrr)r utilsr r get_loggerr-rurNrHOST_TRUST_TTLrr _MAX_COUNTrrrcompilerrr1rrr7rCrSr_rcr~r@rrrrrrrrrrwr5rrrsFE  0000000000((((((((  H % %   <>;; #  RZ0 1 1  BJ- . . 9C9C9999  $"\"\"\"\"\"\"\"\J  "  &9D9999CCJSJ<JDJJJJ S<I