Xj@<XdZddlZddlZddlZddlZddlZddlZddlmZm Z m Z m Z ddl m Z ddlmZmZddlmZddlmZddlmZmZdd lmZmZmZdd lmZdd lmZm Z m!Z!m"Z"m#Z#m$Z$m%Z%dd l&m'Z'm(Z(m)Z)m*Z*dd l+m,Z,ddl-m.Z.ddl/m0Z0ddl1m2Z2ddl3m4Z4m5Z5ddl6m7Z7m8Z8m9Z9m:Z:m;Z;mZ>m?Z?m@Z@mAZAmBZBmCZCmDZDmEZEmFZFmGZGmHZHddlImJZJmKZKddlLmMZMmNZNmOZOddlPmQZQmRZRmSZSe.jTeUZVdeWdeWdzdeXeWeWdzffdZYdeZeWdzdedeWdzdeZe$dzfdZ[eGddZ\e ded eWd!eZeWdzd"eZeWdzd#eWdzd$eWdzd%eWdzd&eZeWdzd'eWdzd(eZeWdzd)eWdzd*eWdzd+e]dzd,eWdzd-eZeWdzd.eZeWdzdeWdzd/eWdzd0eZeWdzd1e^de e\ddff*d2Z_d3d4d(eZeWdzd)eWdzd1e^de`eWeWdzffd5Zad3d4d6eZeWd7e`eWeWdzfd1e^de`eWeWdzffd8Zbd9eZeWd:eZeWdeXeZeWeZeWffd;Zcdddd<d$eWdzd*eWdzdeWdzd&e`eWeWdzfd(e`eWeWdzfd=eZeWd/eWdzd0eZeWdzd>eZeWdzdeZeWfd?Zdd@ZedAd3dBdCe`eWefdDe eWd1e^ddfdEZfdFeWdGedDe eWdeWfdHZgdIZheeWeJdJKfZieeWdzeKdLKfZjeeWdzeKdMe>e NfZkeeZeWdzeKdOdPdQKfZleeWdzeKdRdSKfZmeeWdzeKdTKfZnee]dzeKdUddVWfZoee^eKdXdYdZKfZpee^eKd[d\KfZqeeWdzeKd]KfZreeWdzeKd^d_KfZseeZe]dzeKd`daKfZteeZe]dzeKdbdcKfZuee^eKdddeKfZveeWdzeKdfdgKfZweeZeWdzeKdhdiKfZxeeZeWdzeKdjdkKfZyeeWdzeKdldmdnKfZzee^dzeKdoKfZ{ee^dzeKdpKfZ|eeWeJdqKfZ}eeWeJdrKfZ~eeZeWdzeJdsKfZeeZeWeJdtKfZeeWeJduKfZeeZeWdzeJdvKfZeeWeJdwKfZeeZeWdzeKdxdydzKfZeHd{KZded|e!ddfd}Zded|e!d~e^ddfdZedddigd dd#eided&e8d(e=d,emd-eld.ed'e7d)ee#dfd-eeZeWdzeKdOdPdKfd,eeWdzeKdRdKfdee]eKddKfderdeAddfdZeddgd dZeddg d!deeZeWeJdKfderdeAddfdZedddg d"ded~epderdeAddf dÄZedgdĢ d#dedeeZe]dzeJdƬKfdeeZe]dzeKddɬKfdee^eKdd̬KfderdeAddfd̈́Zeddg d!dederdeAddfdЄZedgdҢ d$deeZeWeJdӬKfd*eeWdzeKdԬKfderdeAddf dՄZedgdע d#ded,emd-eldee^eKddجKfderdeAddfdلZedgdڢ ddede@d1e?derdeAddf d܄ZeHdݬKZeedެߦedddigd d%d e~d!ed#ejd$ekd&e8d(e=d,emd-eld.ed'e7d)eedfd-eeZeWdzeKdOdPdKfd,eeWdzeKdRdKfderdeAddfdZeddg d!deeZeWeJdKfderdeAddfdZeddg d!dederdeAddfdZeddg d!dederdeAddfdZeddg d!dederdeAddfdZeddg d!dede}derdeAddf dZeddg d!dederdeAddfdZedgd d#ded,emd-eldee^eKdːdKfderdeAddfdZeHdKZeedެߦedddid d g d(de}d e~d!ede{de|d#ejd$ekd&e8d(e=d,emd-eld.ed'e7d)e|j pdg|pg| t | gng |jd<|VddddS#1swxYwYdS)awMerge the CLI flags with the `[tool.hf-jobs]` table the script carries, if any. An explicit CLI flag always wins over the script; `env`, `labels`, `secrets` and `volumes` are merged entry by entry (by key, by name and by mount path respectively), so a script value and a CLI value only ever conflict when they target the same entry. `network_aliases` is the exception: the aliases a Job claims form a set, so `--network-alias` replaces the script's list rather than adding to it. A URL script is downloaded to a temporary file that lives until the `with` block exits: submit the Job inside the block. key cli_valueNrAcb||St|x}||SdS)z2Keep the CLI value, and fall back to the script's.N)getattradd)rr script_valuerqheaders rIpickz$_resolve_uv_job_config..picksA$  ' 4 44 A$$$##4rKrgrhrirjr@rkrpc$i|] \}}|v || Sr{r{).0rvalueenv_maps rI z*_resolve_uv_job_config..s*\\\ZS%T[I[I[c5I[I[I[rKc3 K|] }d|V dS)zenv.Nr{rrs rI z)_resolve_uv_job_config..s(>>C<#<<>>>>>>rKrc3 K|] }d|V dS)zsecrets.Nr{rs rIrz)_resolve_uv_job_config..s+FF+c++FFFFFFrKrrc$i|] \}}|v || Sr{r{)rrr labels_maps rIrz*_resolve_uv_job_config..s*eee UsZdOdOdeOdOdOdrKc3 K|] }d|V dS)zlabels.Nr{rs rIrz)_resolve_uv_job_config..s(DDs?S??DDDDDDrKc3 K|] }d|V dS)zvolumes.Nr{rspecs rIrz)_resolve_uv_job_config..s+MM,d,,MMMMMMrKrMr@)rdrfrgrhrirjr@rkrlrmrnrorLrprq rhrjr@rlrmrorkrpextra config_parts)!r>rr=rzrvrprr3rlitemsupdate_parse_secrets_map_resolve_script_secretsrm_parse_labels_maprnr_merge_volume_specsrLrcrdrarrf_name_hash_partsrhrjr@rorkrgrri)!rMrdrfr|rgrhrirlr}rmr~rjrrrr`r@rkrprrZr script_env secrets_mapscript_secrets script_labelsroscript_volume_specsconfigrrqrrs! @@@@rI_resolve_uv_job_configrsBD   U62."2"2 #  c cDj S4Z       We$$h''h''$y'**Di00 _m<<  56#9 5 OO- . . .$4OX..\\\\6:3C3C3E3E\\\ >>:>>>>>>+Z+7+(,PPP 0V]^^^FF~FFFFFF77;7 &u4888>B g6=gfkF]ffk-B-Bceg eeeem6I6I6K6Keee DDmDDDDDD4 44 ,? "fn,],]) )MM9LMMMMMM=#)r'%#pDD)D\WZfo)p)p)p+1r#!   $  & &%B"-!="N$. "N!'!4"("6$*$: 8(8 +',".6-A3x==//r %%%FM& !( kUUUUUUUUUUUUUUUUUUsKK<<LLFrct||fd|pgD}|r1|s/tdd|d|ddit|S)a7Parse `--secrets`/`--secrets-file`, failing if a bare `--secrets NAME` is not set locally. Same behavior as for the secrets requested by a script (see `_resolve_script_secrets`): a missing secret is an error rather than silently dropped, except in `--dry-run` where it is displayed as ``. cLg|] }d|v|xv!S=strip)rrrrs rI z&_parse_secrets_map..,s@rrr#U2B2BPUP[P[P]P]H]fqGqGqtGqGqGqrKz9The following secret(s) are not set in your environment: , $. Export them locally (e.g. `export r)=...`) or use `--secrets-file` otherwise.)r3rjoinryfromkeys)rmr~rmissingrrs @@rIrr#s 66KrrrrrBrrrG w  h RYHZHZ h h29!* h h h    5k 4T]]733 44rKnames cli_secretsc |siSt}i}g}|D],}||vr||vr ||||<||-|r1|s/tdd|d|dd|r:t jdt dd|d|rd nd d i|t|S) aResolve the secrets requested by a script from the caller's environment. A script only lists secret *names*: values always come from whoever runs it (`HF_TOKEN` also resolves from `hf auth login`). A requested secret that is not set locally is an error rather than an empty value: forgetting to export a secret before launching is easy, and debugging the Job that results from it is not. In `--dry-run` a missing secret is not an error but a `None` value, displayed as `` in the summary: nothing is submitted, so the point is to see the full configuration of a script - including one whose secrets are not provisioned locally (yet). zTThe script requires the following secret(s), which are not set in your environment: rrrrzThe script's [z] table requests z+: the value(s) from your local environment zwould bezwill bez sent to the Job.) r0r\rrr;warningr<ryr)rrrenvironresolvedrrs rIrr5sV  #%%G&(HG!! ;    7??$T]HTNN NN4  w  hcgclclmtcucu h h29!* h h h      TZ T T$))H:M:M T T*1@JJy T T T    2h 1$--00 11rK cli_specs script_specscFd|Dfd|D}||z|fS)zMerge `-v` specs with the ones from the script. Returns `(all specs, specs from the script)`. A CLI mount overrides the script's mount at the same path. c<h|]}t||dSrOr$r rs rI z&_merge_volume_specs..as*MMM4|Dd333A6MMMrKcFg|]}t||dv|Srr )rrcli_mount_pathss rIrz'_merge_volume_specs..bs6 d d dT\$D-I-I-I!-LTc-c-cD-c-c-crKr{)rrkeptrs @rIrr\sB NM9MMMO d d d d\ d d dD ) T !!rK)rkrprrorc |ptjj|pd|pd|pdg|pgdt|Dt|||pgS)aThe resolved launch values hashed into a Job's default name, on top of its image/script. Same inputs for `hf jobs run` and `hf jobs uv run`, so that both name Jobs after what they actually submit (`extra` carries what is specific to UV runs). Env *values* are part of the hash since they change what the Job does; secrets only contribute their names, never their values. rc3*K|]\}}|d|VdS)rNr{)rrrs rIrz#_name_hash_parts..~s4 A AzsES  5   A A A A A ArK)r CPU_BASICrsortedrrs rIrrfs& -+'- 2R  +2  B AVCIIKK-@-@ A A A       R  rKz (from script)r{rqrvaluesrqcfd|D}td|Ddg}|D]G\}|vrtndz }|fdt|dDz }Hd|}|rt j|ddSt j|dS) adEcho the launch configuration of a Job before submitting it. Printed on every run, so that what is sent to the Jobs API is always visible - in particular the values injected by a script's `[tool.hf-jobs]` table, which are marked as such. Goes to stderr for a real run, and to stdout for `--dry-run`, where it is the output of the command. cNg|]!\}}|ddgidfv|t||f"S)NrF)_format_job_value)rrrrqs rIrz&_print_job_summary..sO    C r2r51 1 1 UK889 1 1 1rKc3:K|]\}}t|VdSNlen)rr_s rIrz%_print_job_summary..s,,,VS!C,,,,,,rKzJob configuration:rc\g|](\}}d|dkrndd|)S) rr)ljust)rindexlinerwidths rIrz&_print_job_summary..sU   HSt C!##22599 C CT C C   rK z (dry run) Job not submitted.N) rmax _FROM_SCRIPT enumeraterFrr;textlog) rrqrrowslinesrsummaryrrs ` @@rI_print_job_summaryrs%     ,,..   D ,,t,,, , ,E ! "E  U  !3!3;      W`afalalmqararWsWs    iiG G;;;<<<<< rKrrcxdkrLi}|D]3\}}|pd}|dt|dkr |dddzn|||<4nnxdkr d |D}nHxd kr d |D}n"d kr d |D}n t|Sdfd|DS)zVFormat config entries, redacting secrets and marking values inherited from the script.rlrr<N;u…rmc,i|]\}}||d|dndS)rNz***z r{rritems rIrz%_format_job_value..s8tttXbX\^btSS0@uukSStttrKrnc$i|] \}}||d|Srr{rs rIrz%_format_job_value..s,NNN*$t--t--NNNrKrLci|]}||Sr{r{rs rIrz%_format_job_value..s444dtT444rKrc3HK|]\}}|d|vrtndzVdS).rN)r)rrrrqrs rIrz$_format_job_value..sHwwYcY]_cT__d__ -K-K\\QSTwwwwwwrK)rrrvr)rrrqentriesrrrs` ` rIrrs6 UUUUG#kkmm Z Z dzr#' Y Ys4yy2~~$ss)e*;*;SW Y Y  ZYYYYttfkfqfqfsfstttGG XXXXNN NNNGG YY44e444GG u::  99wwwwwgngtgtgvgvwww w wwrKg?zThe Docker image to use.)helpz.Use a custom Docker image with `uv` installed.zcFlavor for the hardware. Run 'hf jobs hardware' to list available flavors. Defaults to `cpu-basic`.)r click_typez-lz--labelz3Set labels. E.g. --label KEY=VALUE or --label LABELz--namezName the Job. Stored as the `name` label. Names do not have to be unique. Defaults to the image or script name plus a short hash of the resolved launch configuration.zPMax duration: int with s (seconds, default), m (minutes), h (hours) or d (days).z --attemptsz9Maximum attempts, including the first run. Defaults to 1.)minrz-dz--detachz3Run the Job in the background and print the Job ID.z --dry-runz@Print the resolved Job configuration without submitting the Job.zVThe namespace where the job will be running. Defaults to the current user's namespace.z--resource-group-idzThe ID of the resource group to create the Job in. Used to control access to resources within an organization and for cost attribution/spending-limit features.z--exposezExpose a container port through the jobs proxy. Repeat the flag for multiple ports (e.g. `--expose 8000 --expose 8001`). Each exposed port is reachable on the public jobs domain; access requires an HF token with read access to the job's namespace.z--expose-publiczExpose a container port through the jobs proxy without authentication. Repeat the flag for multiple ports. No need to also pass the port with `--expose`.z--sshzMake the job's container reachable over SSH. Connect with `hf jobs ssh `. Requires an SSH public key registered on https://huggingface.co/settings/keys.z--network-groupa Join a network group. Jobs in the same namespace and resource group sharing a group are placed together and reach each other on every port. Inside each member, `$HF_NETWORK_GROUP_HOSTNAME` resolves to every member. Lowercase alphanumerics and dashes, 46 characters max.z--network-aliaszClaim an alias in the network group. Members reach the jobs claiming it at `${HF_NETWORK_GROUP_PREFIX}`. Repeat the flag for several aliases. Lowercase alphanumerics and dashes, 34 characters max, unique within the job. Requires `--network-group`.z--withz%Run with the given packages installedz-pz--pythonz5The Python interpreter to use for the run environmentz!Suspend (pause) the scheduled Jobz8Allow multiple instances of this Job to run concurrentlyz]One of @annually, @yearly, @monthly, @weekly, @daily, @hourly, or a CRON schedule expression.z$UV script to run (local file or URL)zArguments for the scriptzThe command to run.zJob ID (or 'namespace/job_id')zJob IDs (or 'namespace/job_id')z2Scheduled Job ID (or 'namespace/scheduled_job_id')z-vz--volumeaMount one or more volumes. Format: hf://[TYPE/]SOURCE:/MOUNT_PATH[:ro|:rw] or LOCAL_DIR:/MOUNT_PATH[:ro|:rw]. TYPE is one of: models, datasets, spaces, buckets. TYPE defaults to models if omitted. models, datasets and spaces are always mounted read-only. buckets are read+write by default. A local directory source is first synced to a bucket and mounted read-only by default. E.g. -v hf://datasets/org/ds:/data or -v hf://buckets/org/b:/mnt:ro or -v ./inputs:/inputszRun and manage Jobs on the Hub.jobc||j|jjdD]}t j|||j|jj}|jjtj krD|jj rd|jj nd}td|jd|jjd|t jd|jd d S) zcStream Job logs until the Job ends, then fail the command if the Job did not complete successfully.Tr?r@followr?r@z: rzJob z finished with stage 'rDz completedN) fetch_job_logsidownerrr;r wait_for_jobstatusstager COMPLETEDmessager)rMrrfinalrs rI_stream_logs_and_check_statusrs!!39>RV!WW    CFcin  E EE |X///161EM-u|+---2\eh\\el>P\\SZ\\]]]H (EH ( ( ()))))rKdetachcn|jjd|j}t|jjt r@dd|jjD}tj d|t|jj trtj d|dt|j tr'|j j}tj d|d|d |r5tj d |d |d tj d |ddSt||dS)zZPrint hints about how to reach a started Job, then stream its logs unless `detach` is set.rCrc3 K|] }d|V dS)rNr{)rurls rIrz._hint_or_follow_started_job..s(FFcFFFFFFrKz Exposed ports are reachable at: zUse `hf jobs ssh z&` to open an SSH session into the job.zJoined network group 'zK'. Jobs of this namespace and resource group started with `--network-group z` reach each other at `$HF_NETWORK_GROUP_HOSTNAME` (every member) or `${HF_NETWORK_GROUP_PREFIX}` (members claiming an alias).Use `hf jobs logs -f z&` to stream logs, or `hf jobs inspect z` to check status.zUse `hf jobs wait z` to block until it finishes.N)rrr isinstancer expose_urlsrxrr;rYssh_urlrvnetworkrgroupr)rMrrjob_refurlsrs rI_hint_or_follow_started_jobrse**#&**G#*($//=yyFFsz/EFFFFF ;T;;<<<#*$c**V TWTTTUUU#+z**  !  RU R R % R R R    sssX_sssttt LgLLLMMM!#s+++++rKrunignore_unknown_optionsT)zFhf jobs run --name hello-world python:3.12 python -c 'print("Hello!")'z1hf jobs run --detach python:3.12 python script.pyz3hf jobs run -e FOO=foo python:3.12 python script.pyz;hf jobs run --secrets HF_TOKEN python:3.12 python script.pyz^hf jobs run -v hf://org/my-model:/data -v hf://buckets/org/b:/mnt python:3.12 python script.py)context_settingsexamplescommandexpose expose_publicssh network_aliasresource_group_idtokenformat json_outputquietct|||t||}t||| }t||pi}|dt ||t | | | t| gnd||||pg|| t|}| rdnt|||}tid |d tj |d | ptjjd | d | d|d|d|pgd|dd d|pgDdd d|pgDd|d|dd |pgd|d|| | rdS|||||||| | | |||||||}t#jd|j|jpid|jt/||sI|jpid}t#jd|d |jjd!|jd"t7||| #dS)$z Run a Job.rrrN) rhrjrr@rlrmrorkrprrrrgrrhrjrrlrmrLrnrrRc34K|]}t|VdSrrvrports rIrzjobs_run..(BBTs4yyBBBBBBrKrc34K|]}t|VdSrr&r's rIrzjobs_run..(%P%PDc$ii%P%P%P%P%P%PrKrrkrprr@)rgrrlrmrnrLrhrjrrrrrkrprr@ Job startedrrr Job auto-named '('. Pass `--name` or run `hf jobs labels rC --name NAME` to rename it.r)r5r3rr setdefaultrrrvr2rarshlexrrrrrun_jobr;r]rrngetr _has_explicit_namerYrrr) rgrrlrmrrr`r}r~rhrjrrrrrrrkrrr@rr r!r"rrrrMrLr auto_names rIjobs_runr8sfNfk5111C**G$WlGLLLK"5t444:J$  ))1)=s8}}oo4###\r+ -      & 5 ! ! !Cddd#>v3Zc#d#d#dG U uz'**  f; 5 ;  w    7   {  v|  j  chhBBV\rBBBBB  SXX%P%PMzhf jobs logs -f zhf jobs logs --tail 20 z#hf jobs logs -f --tail 100 )rrz-fz--followzqFollow log output (stream until the job completes). Without this flag, only currently available logs are printed.tailz-nz--tailzvNumber of lines to show from the end of the logs. When combined with --follow, starts streaming from the last N lines.c.t||\}}t|}|||||}|D]}tj||r|d|n|}|rtjd|ddStjd|ddS) aFetch the logs of a Job. By default, prints currently available logs and exits (non-blocking). Use --follow/-f to stream logs in real-time until the job completes. Use --tail/-n to limit the number of lines returned (server-side when supported). Note: following exits when the log stream ends, regardless of whether the Job succeeded or failed. Run `hf jobs inspect ` to check the final status. r$)r?r@rr:rC#Stream ended. Run `hf jobs inspect 6` to check the final status (e.g. COMPLETED or ERROR).z2If the job is still running, use `hf jobs logs -f z(` to stream new logs until it completes.N)rJr2rr;rrY) r?rr:r@rrMr9rrs rI jobs_logsr>.sN5VYGGFI 5 ! ! !C   VyVZ  [ [D  )2>%%V%%%G   vwvvvwwwww  r r r r     rKncTd}d}t|D]}t||dS)Nzz)end)rangeprint)r?LINE_UP LINE_CLEARis rI _clear_linerGdsAGJ 1XX'' g:&&&&&''rKmetrics table_headersc ~||ddt|ddz dtd|dz|dz d dt|dd t|dd t|d d t|ddg|drt|dtrΉgfdt t |ddz Dz}t |t|dD]s\}|d|}|ddtd|dz|dz d dt|dd t|dd gz t|Sdgt |t z zz gS)N cpu_usage_pct%cpu_millicoresg@@r$dmemory_used_bytesmemory_total_byteszB / Brx_bpszbps / tx_bpsbpsgpusc6g|]}dgtzS)rr)rrrows rIrz*_format_job_stats_rows..us%.h.h.h1tc#hh.h.h.hrK utilizationN/A)roundrr ryrBrzipr)r?rHrIrgpu_idgpurXs @rI_format_job_stats_rowsr_ks  ? #&&& g&'&0!44 w233g>R6SSUV W WZZZ  34 5 5ii<PdHe;f;fiii ) * *VV,wx?P2Q2QVVV Cv :gfot<< '*e.h.h.h.hcRYZ`RaNbNbefNfHgHg.h.h.h&htVGFO%<%<==  KC&/&)C }%(((s#677#>R:SSUVWWZZZ$7 899ii|CPdLe?f?fiii CC  E7c-((3s883 44C 5LrKmetrics_streamc#NK|D]}d|t|||fVd|gfVdS)NFTr_)r?r`rIrHs rI_get_jobs_stats_rowsrcsU"TTV3FG]SSSSSSS  rKstats)z hf jobs statszhf jobs stats zhf jobs stats -f job_idsziFollow stats output (live view until the Jobs complete). Without this flag, a single snapshot is printed.c.|rkrMtAdt|zfdD}t7t1||} dddn #1swxYwYdfd|D} tj d| d dS)!zFetch the resource usage statistics and metrics of Jobs. By default, prints a snapshot of the current stats and exits (non-blocking). Use --follow/-f to display live stats until the Jobs complete. zD`--follow` displays a live terminal view and is not supported with 'z3' output. Run without `--follow` to get a snapshot.Nr$rcJg|] }|jr |jjnddv|j!S)UNKNOWN)RUNNINGUPDATING)rrrrrs rIrzjobs_stats..sA   $'J=   IBYYY FYYYrK)r@rzHNo running jobs. Use `hf jobs ps -a` to list finished (and failed) jobs.) zJOB IDzCPU %zNUM CPUzMEM %z MEM USAGEzNET I/Oz GPU UTIL %z GPU MEM %z GPU MEM USAGEr?rAchtt|dS)Nr)nextiterfetch_job_metrics)r?rMr@s rIfetch_latest_metricsz(jobs_stats..fetch_latest_metricss/S22&I2VVWWY]^^ ^rKzNo stats available for Job z. Is it running?cHg|]}t|d|D]}|S)r?rb)rrrXrIs rIrzjobs_stats..s:qqqD5KDQYN\`bo5p5pqqcCqqqqrK)headersrRc32K|]}d|dVdS)rCr?Nr{)rrr@s rIrzjobs_stats..s5RRD9 ? ?tH~ ? ?RRRRRRrKzUse `hf jobs stats -f z` to follow live stats.id_keyc&g|]}d|vsd|vrdndS)rCUSAGEz-- / --z--r{)rrs rIrzjobs_stats..s/pppTZ#--7f3D3DII4ppprKr$c*g|]}|D]}|Sr{r{rr?rXrows_per_job_ids rIrzjobs_stats..s,[[[f?SYCZ[[Cc[[[[rKcHg|]}||ddS)Tr)r?r`rI)ro)rr?rMr@rIs rIrzjobs_stats..sR    !"%"7"7vQZcg"7"h"h!.     rK) kwargs_listrQc*g|]}|D]}|Sr{r{rys rIrzjobs_stats..s,cccf?[aKbccCcccccrKc3&K|] }d|V dS)rCNr{)rr?r@s rIrzjobs_stats..s1FFF9//v//FFFFFFrKr<r=)!r;moder9humanrrrJr\r2whoami list_jobsrtablerYrvryrmultiprocessingpool ThreadPoolmapr\rr _tabulatertimerCiflatmap_unorderedrcpopSTATS_UPDATE_MIN_INTERVALrG)rerr@r parsed_idsr?rprlatest_metricsrrHrjob_refsrXlast_update_time total_rowsr|donenowrMrzrIs ` @@@rI jobs_statsrsx4 #(l000 9SVS[Sa 9 9 9     & &F  , ,F$*8C pp^klmlnln^oppp pC'*eOF # #9;;[[[[[[[  i M:::;;;      "    #5T;O]h"i"i"i ' ' D&$ /##FD1111*.')++C%%)BBBAJ/000ccccccc i MBBBCCC#&  ')'''''''''''''''>xxFFFFgFFFFFHH s8 s s sttttts%FFF#D!OOOzlist | ls | ps)z hf jobs lsz hf jobs ls -az&hf jobs ls --status running,schedulingzhf jobs ls --name training-v2z+hf jobs ls --label env=prod --label team=mlz%hf jobs ls --all --label hf-sandbox=1rNallz-az--allzWShow all Jobs (default shows running and scheduling). Cannot be combined with --status.rz--statuszfOnly show Jobs with the given status. Comma-separated or repeated, e.g. `--status running,scheduling`.)rrz{Only show Jobs with the given `key=value` label. Repeat to require several labels, e.g. `--label env=prod --label team=ml`.zFOnly show Jobs with the given name (shortcut for `--label name=NAME`).limitz--limitzCMaximum number of Jobs to display. Set to 0 to show all (no limit).ct|}|r|rtdg}|pgD]4} |d| dD5|r|} n'|rd} n"tjjtjjg} i} |pgD]6} d| vrtd| d| d\} } | | | <7|d | vrtd || d <||| | pd }d }|d krEttj ||dz}t||kr d}|d|}nt|}g}|D]d}t|}|dpi}|dpg}|dd|d<|dpid pd|d <|dpd|d<|rd|nd|d<|dr$|dddddnd|d<|dpidd |d<t#|d!|d"<||ft'j|gd#d$|rt'jd%|d&|st|s| rXd'gd(|Dd)| D}t'jd*|dS|st'jd+dSdSdS),zList Jobs. Use `--status` to filter by status (see [`JobStage`] for possible values) and `--label` to filter by `key=value` labels. A Job must match every filter to be listed. r$0`-a`/`--all` cannot be combined with `--status`.c3fK|],}||V-dSrrrparts rIrzjobs_ps..As7VVTVDJJLLVVVVVVrK,NrInvalid label filter '"': must be in the form 'key=value'r6Cannot filter by both `--name` and `--label name=...`.)r@rrnFrr$T durationsrrrr?rnrZ docker_image image/spacerR created_atTcreatedrrrh running_secsruntime)r?rrrrrrrrruzOutput truncated to z4 Jobs. Use `--limit 0` to show all (or `--limit N`).rc3 K|] }d|V dSzstatus=Nr{rss rIrzjobs_ps..~(77Q=Q==777777rKc3,K|]\}}d|d|VdSzlabel=rNr{rkvs rIrzjobs_ps..~::g:gPTPQST;KA;K;K;K;K:g:g:g:g:g:grKzNo jobs matched filters: zHNo running jobs. Use `-a`/`--all` to include finished (and failed) jobs.)r2rrUrFrrir SCHEDULINGrrx itertoolsislicerr:r5rreplacer"r\r;rrYrr)rrrrrr@rrM raw_statusesrserver_statusesrnrr jobs_iter truncatedjobs job_itemsrjob_itemrcmd filters_msgs rIjobs_psrs6t 5 ! ! !C KvKIJJJ!L2WWVVU[[5E5EVVVVVVVN& N#+183F3LM F  d??\D\\\]] ]ZZ__ Us   V  STT Tv /RXR`\` aaII qyyI$Y ::;; t99u  I">"G%/2=chhsmmmOW||\hOiOith|4SbS9AA#sKKKot&ll844:??SS-immN.K.KLL""""I\\\ e ccccddd a  a6 a))h77,777h:g:gX^XdXdXfXf:g:g:ghK H>>> ? ? ? ? ? a H_ ` ` ` ` `aa  a arKhardwarezhf jobs hardwarec t}|}g}|D]}d}|jr)|jjd|jjd|jjd}|jr d|jdnd}|jrd|jdzd nd}||j|j |j |j |j |||d tj|tjd d S) z(List available hardware options for Jobsrzx  ()$z.4ffreerz.2f)rz pretty namecpuramstorage acceleratorzcost/minz cost/hourzLUse `hf jobs run --flavor ...` to request a specific hardware flavor.N)r2list_jobs_hardwarerquantitymodelvram unit_cost_usdr\r pretty_namerrephemeral_storager;rrY)rM hardware_listrhwaccelerator_infocost_min cost_hours rI jobs_hardwarers+ ,,C**,,M E   > l"$."9kkR^=QkkUWUcUhkkk 131AM-r'----v797GS3(2-3333V  !~vv//$&   IeH [\\\\\rKinspectzhf jobs inspect z*Job IDs to inspect (or 'namespace/job_id')cg}|D]*}t|\}||+|}t|fd|D}tjd|DddS)z0Display detailed information on one or more Jobsr$c>g|]}|S)r) inspect_job)rr?rMr@s rIrz jobs_inspect..s) V V VFCOO6YO ? ? V V VrKcTg|]%}tt||j&S)rn) _surface_namer:rnrks rIrz jobs_inspect..s0YYYS}/44SZHHHYYYrKrrtNrJr\r2r;r)rer@rrr?rrMs ` @rI jobs_inspectrsJ""8KK &!!!!G 5 ! ! !C V V V V Vg V V VDIYYTXYYYbfggggggrKrerunzhf jobs rerun zhf jobs rerun --detach ct||\}}t|}|||}tjd|j|jpid|jt|||dS)z*Run a new Job with an existing Job's spec.r$rr,rr-r1N) rJr2 rerun_jobr;r]rrnr5r r)r?rr@rrMrs rI jobs_rerunrs5VYGGFI 5 ! ! !C --v- ; ;CJ}sz/?R.D.DV.L.LRURYZZZZS888888rK)zhf jobs expose 8000z*hf jobs expose 8000 --public 9000zhf jobs expose --clearportszlPorts to expose through the jobs proxy. Access requires an HF token with read access to the job's namespace.publicz--publiczExpose a port without authentication. Repeat the flag for multiple ports. No need to also pass the port as a positional argument.clearz--clearzClose all exposed ports.c(|r|s|rtd|s|s|stdt||\}}t|}|||||}t jd|j|jpg|jpgdS)z'Replace exposed ports on a running Job.z6`--clear` cannot be combined with ports or `--public`.zKPass at least one port or `--public` port, or `--clear` to close all ports.r$)r?rrr@zExposed ports updated)rr public_portsN) rrJr2update_job_exposer;r]rrr)r?rrrr@rrMrs rI jobs_exposers8 Q%Q6QOPPP ffvfdeee4VYGGFI 5 ! ! !C  ve6]f  g gCJ&369IrX[XiXomopppppprKcancelzhf jobs cancel ct||\}}t|}|||tjd|dS)z Cancel a Jobr$rz Job cancelledrN)rJr2 cancel_jobr;r])r?r@rrMs rI jobs_cancelrsY5VYGGFI 5 ! ! !CNN&IN666J6******rKwait)zhf jobs wait z"hf jobs wait z"hf jobs ls -q | xargs hf jobs waitz,Job IDs to wait for (or 'namespace/job_id').zTMax time to wait: int with s (seconds, default), m (minutes), h (hours) or d (days).c g}t}|D]?}t||\}}||||@t |dkrCt dddt|tDz| }|t|nd} t|} tj dt |d } | || | } n7#t$r*| d t d |d dwxYw| t | dtjd| Dd| D} | rOt t | dt | ddd| DzdS)aWait for one or more Jobs to reach a terminal state. Blocks until every Job has finished, then exits with code 0 if all Jobs completed successfully, or a non-zero exit code if any Job was canceled, errored or deleted. All Jobs must belong to the same namespace. r$z0All Job IDs must be in the same namespace, got: rc34K|]}t|VdSrr&)rnss rIrzjobs_wait..!s(FFBBFFFFFFrK)rNr$z Waiting for z Job(s) to finish...)rjr@z Timed out.zTimed out after z waiting for Job(s) to finish.z Job(s) finished.cfg|].}|jt|jj|jjd/S))rrr)rrvrrrrks rIrzjobs_wait../s7mmmadcfs3:+;'<'<I[\\mmmrKcFg|]}|jjtjk|Sr{)rrrrrks rIrzjobs_wait..0s* L L LcSZ%59K%K%Kc%K%K%KrKz of z' Job(s) did not complete successfully: c3DK|]}|jd|jjdVdS)rrN)rrrrks rIrzjobs_wait..4s:KKS3688SZ%5888KKKKKKrK)rzrJr\rrrrrrvrr#r2r;rr TimeoutErrorrr)rerjr@rr namespacesr? parsed_idparsed_namespace timeout_secsrMrrfaileds rI jobs_waitrs>>JJ))&B69&U&U# #)$$$'(((( : >iiFF&*E*E*EFFFFF G     I.5.A>'***tL 5 ! ! !C ZLs:LLL M MF] LIVV ]]] L!!!Q'QQQRRX\\] KK3t99///000Immhlmmmnnn L LT L L LF  6{{ R RD R R RiiKKFKKKKK L     s D4Ern)z*hf jobs labels --name training-v2z8hf jobs labels --label env=prod --label team=mlzhf jobs labels --clearzRemove all labels from the job.c|s||std|s||rtdt||\}}t|}|)|s'|||jpi}i|d|i}nt ||pi}||||} tjd | j | d dS) zfUpdate labels on a Job. Passing --label replaces all existing labels; passing --name alone keeps them.NePlease set a name with --name or at least one label with --label. To remove all labels, pass --clear.lCannot set a name or labels and clear them at the same time. Please use --name/--label or --clear, not both.r$rrr)r?rnr@Labels updatedrr) rrJr2rrnrupdate_job_labelsr;r]rr5) r?rrrr@rrMcurrent_labelsrnrs rI jobs_labelsr8s"  T\%\ s     !u! z   5VYGGFI 5 ! ! !C )LLSYWY1N1FD11"5t444:  vf  R RCJCFF1C1CDDDDDDrK)zhf jobs ssh zhf jobs ssh --dry-runz)hf jobs ssh -i ~/.ssh/id_ed25519 identity_filecJt||\}}t|}|||}|jjt d|jjtvr%t d|jd|jjd|jjtj krtjd|jd |jjd }| |j|tj g }|jjtj kr:| d t d|jd |jjd| dt|jj}t|jd|j|j||dS)a SSH into a running Job. If the Job is not yet running, waits until it reaches the RUNNING state before connecting. Requires the Job to be started with SSH enabled (`hf jobs run --ssh ...`) and your SSH public key to be registered at https://huggingface.co/settings/keys. r$rNz[SSH is not enabled on this job. Start a job with SSH support using `hf jobs run --ssh ...`.zCannot SSH into job 'z%': job has already finished (stage: 'z').zWaiting for job 'z' to be running (stage: 'z')...)r?r@stagesz Job finished.z1': job finished before reaching RUNNING (stage: 'zJob is running.@)r(r r)rJr2rrrrrrrrrir;rrrr1usernamehostnamer() r?r rr@rrMrrrs rIjobs_sshr]s*5VYGGFI 5 ! ! !C //9/ = =C z!tuuu z...qsvqq\_\f\lqqqrrr z8+++hhhQTQ[Qahhhiicf 8K[J\]] : x/ / / KK ( ( (vvvadakaqvvv   %&&&sz)**G  00g.00 \# rKzFRun UV scripts (Python with inline dependencies) on HF infrastructure.uvr)z,hf jobs uv run --name my-script my_script.pyz$hf jobs uv run --detach my_script.pyz1hf jobs uv run --flavor a10g-small ml_training.pyz+hf jobs uv run --with transformers train.pyzNhf jobs uv run -v hf://org/my-model:/data -v hf://buckets/org/b:/mnt script.pyz"hf jobs uv run --dry-run script.pywith_ct|||t|}td+id|d|d|d|d|d|d|d |d | d |d | d | d| d|d|d|d|d|d|d|5}tid|dt j|jdd|pgd|jptd|j ptj j d|j d |jd| d |jd |jd|jd|jddd|pgDddd|pgDd|d|jdd|j||jd |j|!|r d"d"d"d"S||j|j||j |j|j|j|j|j|j |j| ||||j|jpd"||j#}d"d"d"n #1swxYwYt5jd$|j|jpid|j%t?|||jsI|jpid}t5j d&|d'|j!j"d(|jd)tG||| *d"S),8Run a UV script (local file or URL) on HF infrastructurer$rMrdrfr|rgrhrirlr}rmr~rjrrrr`r@rkrprargswithrRrLrnrc34K|]}t|VdSrr&r's rIrzjobs_uv_run..("F"F3t99"F"F"F"F"F"FrKrc34K|]}t|VdSrr&r's rIrzjobs_uv_run..()T)T#d)))T)T)T)T)T)TrKrrr@rN)rdrfr|rirgrlrmrnrLrhrjrrrrrkrprr@r,r-r.r/rCr0r1r{)$r5r2rrr3rrfrgrrhrrrrirjrlrmrornrkrpr@rq run_uv_jobrdrLr;r]rr5r r6rYrrr) rdrfrgrhrlrmrrr`r}r~rjrrrrrrrkrrr@rrrir r!r"rMrrr7s rI jobs_uv_runrsVfk5111 5 ! ! !C     C v  K U   e   v  v  C   "\   T e  v! ")# $$m% && ' () E *  &  6#566 "-- 9)9   &-F;+@+F   &-  6> H vz 6> 6. &- #(("F"F2"F"F"FFF  )T)T @SQS)T)T)T!T!T s   !5! ""388F,B#C#C# $&7#-'   **/    2   aE E E E E E E E bnn=*=, N=N=N' ."2:d/&'  cE E E E E E E E E E E E E E E LJ}sz/?R.D.DV.L.LRURYZZZZ dE6+= > > Z%2**622   Ty T T"y~ T T03 T T T    S888888s(DH A+H  HHceZdZdZdZdZdS)ScheduledJobStatusFilterzPossible values for `hf jobs scheduled ls --status`. Scheduled Jobs are not "running": they are either active (i.e. they will trigger new runs) or suspended. active suspendedN)rrrsrtruACTIVE SUSPENDEDr{rKrIrr s$ FIIIrKrz,Create and manage scheduled Jobs on the Hub. scheduledzRhf jobs scheduled run "0 0 * * *" --name daily-script python:3.12 python script.pyschedulesuspend concurrencyct|||t|| }t|| |}t||pi}|dt ||t | | |t|gnd|||| pgt|}|rdnt| ||}tid |d |d |d |d tj |d| ptjjd| d|d|d|d| pgd|dd d|pgDdd d|pgDd|d|||rdS||||||||||| | |||||}t#jd|j|jjpidt/||sN|jjpid}t#jd|d |jjd!|jd"t#jd#|jjd!|jd$dS)%zSchedule a Job.rrrN)rhrjrr@rlrmrorr$rr$r%r&rgrrhrjrrlrmrLrnrrRc34K|]}t|VdSrr&r's rIrz scheduled_run.._r)rKrc34K|]}t|VdSrr&r's rIrz scheduled_run..`r+rKrr@)rgrr$r%r&rlrmrnrLrhrjrrrrr@Scheduled Job createdrScheduled Job auto-named '2'. Pass `--name` or run `hf jobs scheduled labels rCr0Use `hf jobs scheduled inspect ` to view its details.)r5r3rrr2rrrvr2rarr3rrrrcreate_scheduled_jobr;r]rjob_specrnr5r6rYrr)r$rgrr%r&rlrmrrr`r}r~rhrjrrrrrr@rr r!r"rrrrMrL scheduled_jobr7s rI scheduled_runr2sr@fk5111C**G$WlGLLLK"5t444:J$  ))1)=s8}}oo4###\r " 5 ! ! !Cddd#>v3Zc#d#d#dG  w  ;  U  uz'**  f; 5 ;   w    7  {  v|  j  chhBBV\rBBBBB  SXX%P%PM}||V?dSr)rlowerrs rIrzscheduled_ps..sH^^TQUQ[Q[Q]Q]^DJJLL..00^^^^^^rKrc>g|]}|ttv|Sr{)tuplerrs rIrz scheduled_ps..s+\\\a1EBZ<[<[3[3[3[3[3[rKzInvalid status filter(s) z: expected one of cg|] }|j Sr{)rrs rIrz scheduled_ps..s;;;Q;;;rKrTrrrr$Nrr)r@rnFr0rlast_jobrrnrZrrrRatrrlast_runnext_job_run_atnext_runr%)rrr$rrr;r=r%rrrc3 K|] }d|V dSrr{rs rIrzscheduled_ps..rrKc3,K|]\}}d|d|VdSrr{rs rIrzscheduled_ps..rrKz#No scheduled jobs matched filters: zENo active scheduled jobs. Use `-a`/`--all` to include suspended ones.rr-z(` to view details about a scheduled job.zUse `hf jobs scheduled trigger z)` to trigger a scheduled job immediately.)r2rrUrFrr!r"list_scheduled_jobsr%r\r:r5rrr;rrrrY)rrrrr@rrMrrunknown_statuses show_activeshow_suspendedrn raw_labelrscheduled_jobs filtered_jobsr1r%rsjrr0 status_dictr9rr first_item_ids rI scheduled_psrJsp 5 ! ! !C KvKIJJJ!L2__^^U[[=M=M^^^^^^^\\<\\\  ?(8 ? ?;;":;;; ? ? ?   .5E 1;|K  F[b i  aIaaabb b__S!,, Us   V  STT Tv,,ySW,XXNM',, '05  >   {  ]++++#%E!"%%88J''-2hhx((.B ??:..ll9%%+ X..4"99&AAJUV &ll>::Ce]+.9#((3---EYDLnQYQ]Q]^bQcQcn8D>#2#.66sC@@@inZEP__UfEgEg rK) *3B3 / 7 7S A A Amr Z((9--6Y TI ggg ^  ^6 ^))h77,777h:g:gX^XdXdXfXf:g:g:ghK HH;HH I I I I ^ H\ ] ] ] ma  j=jjjkkk k=kkklllllmmrKzhf jobs scheduled inspect scheduled_job_idsz>Scheduled Job IDs to inspect (or 'namespace/scheduled_job_id')cg}|D]*}t|\}||+|}t|fd|D}tjd|DddS)z:Display detailed information on one or more scheduled Jobsr$c>g|]}|S)scheduled_job_idr@)inspect_scheduled_job)rrOrMr@s rIrz%scheduled_inspect..$s=  !!3Cy!YYrKc^g|]*}tt||jj+Sr)rr:r0rn)rrGs rIrz%scheduled_inspect..(s4iiiTV}/33BKrOct||\}}t|}|||tjd|dS)zDelete a scheduled Job.r$rNzScheduled Job deletedrN)rJr2delete_scheduled_jobr;r]rOr@rrMs rIscheduled_deleterW+b#??OQZ"["[i 5 ! ! !C.>)TTTJ&+;<<<<<ct||\}}t|}|||tjd|tjd|ddS)z Suspend (pause) a scheduled Job.r$rNzScheduled Job suspendedrzUse `hf jobs scheduled resume z` to resume it.N)rJr2suspend_scheduled_jobr;r]rYrVs rIscheduled_suspendr[8s#??OQZ"["[i 5 ! ! !C/?9UUUJ(-=>>>>H O.> O O OPPPPPrKresumezhf jobs scheduled resume ct||\}}t|}|||tjd|dS)z!Resume (unpause) a scheduled Job.r$rNzScheduled Job resumedrN)rJr2resume_scheduled_jobr;r]rVs rIscheduled_resumer_FrXrK reschedulez-hf jobs scheduled reschedule "0 9 * * 1"ct||\}}t|}||||}tjd|j|jtjd|jj d|jddS) z!Change when a scheduled Job runs.r$)rOr$r@zScheduled Job rescheduled)rr$r-rCz` to see its next run.N) rJr2update_scheduled_job_scheduler;r]rr$rYrr)rOr$r@rrMrs rIscheduled_reschedulercSs#??OQZ"["[i 5 ! ! !C + +=MX`lu + v vCJ*sv MMMMH ^sy~ ^ ^ ^ ^ ^_____rKtriggerzhf jobs scheduled trigger ct||\}}t|}|||}tjd|j|jtjd|jj d|jddS) zJTrigger a scheduled Job to run immediately (does not change the schedule).r$rNzScheduled Job triggered)rr r rCz` to stream logs.N) rJr2trigger_scheduled_jobr;r]rr rYrr)rOr@rrMrs rIscheduled_triggerrgbs#??OQZ"["[i 5 ! ! !C # #5EQZ # [ [CJ(SVAAAAH OSY^ O Ocf O O OPPPPPrK)z1hf jobs scheduled labels --name daily-scriptz>hf jobs scheduled labels --label env=prod --label team=mlz%hf jobs scheduled labels --clearz)Remove all labels from the scheduled job.c|s||std|s||rtdt||\}}t|}|.|s,|||jjpi}i|d|i}nt ||pi}||||} tj d | j | d dS) zpUpdate labels on a scheduled Job. Passing --label replaces all existing labels; passing --name alone keeps them.Nrrr$rNrr)rOrnr@rr) rrJr2rPr0rnrupdate_scheduled_job_labelsr;r]rr5) rOrrrr@rrMrrnr1s rIscheduled_labelsrjps0"  T\%\ s     !u! z   #??OQZ"["[i 5 ! ! !C   % %7GS\ % ] ] f m sqs 2N1FD11"5t444:33)&I4MJM$46::f;M;MNNNNNNrKz)Schedule UV scripts on HF infrastructure.zBhf jobs scheduled uv run "0 0 * * *" --name daily-script script.pyz.rrKrc34K|]}t|VdSrr&r's rIrz#scheduled_uv_run..rrKrr)rdrfr$r%r&r|rirgrlrmrnrLrhrjrrrrr@r*rr+r,rCr0r-r.r{)%r5r2rrkrprr<rr3rrfrgrrhrrrrirjrlrmrornr@rqcreate_scheduled_uv_jobrdrLr;r]rr0r5r6rYrr)r$rdrfr%r&rgrhrlrmrrr`r}r~rjrrrrrr@rrrir r!r"rMrrr7s rIscheduled_uv_runrosLfk5111 5 ! ! !C     C v  K U   e   v  v  C   "\   T e  v! ")# &d' () *+ K ,    6#9 B(BBB   H 7 { &    6#566   "--  9)9 &-F;+@+F &- 6> H vz 6> 6. &-  #(("F"F2"F"F"FFF! " )T)T @SQS)T)T)T!T!T# $&7#-'   **/    2   mK K K K K K K K n))=*#=, N=N=N'/&'*  oK K K K K K K K K K K K K K K XJ&369L9RPR8W8WX^8_8_```` dE6+= > > \(.B33F;;   ^ ^ ^), ^ ^:=& ^ ^ ^   H ^sy~ ^ ^ ^ ^ ^_____s(D(HAHH H rcD|pid}||Sd|i|S)aPromote the `name` label to a top-level `name` field for display. The `name` is kept inside `labels` too (the dataclasses and API payloads are unchanged); this only makes it a first-class column/field in command outputs. No-op when there is no name. rr5)rrnrs rIrrs5 Lb  f % %D | D !D !!rKcH|dupd|vptd|pgDS)zlWhether the Job was named on purpose: via `--name`, a `name=` label, or the script's `[tool.hf-jobs]` table.Nz labels.namec3VK|]$}|ddddkV%dS)rr$rrN)rF)rrs rIrz%_has_explicit_name../s:GG4tzz#q!!!$.GGGGGGrK)any)rrrqs rIr6r6*sC D H K ' H GG5;BGGG G GrKc|s|dSi}|pgD](}d|vr|ddn|df\}}|||<)|d|vrtd||d<|S)aParse label key-value pairs from CLI arguments. Args: labels: List of label strings in KEY=VALUE format. If KEY only, then VALUE is set to empty string. Returns: Dictionary mapping label keys to values, or None if no labels or name provided. Nrr$rrz4--name and --label name=... cannot both be provided.)rFr)rnrr label_varrrs rIrr3s dlt!#J\r  03y0@0@Y__S!,,,yRTo U 3  Z  QRR R! 6 rKrrrcdtg||RD}ttjjt |dz}t |t |z|kr|t|}||xxdzcc<t |t |z|kr.|t |z t |z ||z||<t |t |z|kdt |zj|}g}| |j|| |jd|D|D]9}dt||D}| |j|:d |S)z Inspired by: - stackoverflow.com/a/8356620/593036 - stackoverflow.com/questions/9535954/printing-lists-as-tabular-data c@g|]}td|DS)c3NK|] }tt|V!dSrrrv)rxs rIrz'_tabulate...Ps.//ac#a&&kk//////rK)r)rcols rIrz_tabulate..Ps/OOOC#//3/////OOOrK rQz{{:{}}} cg|]}d|zS)-r{)rws rIrz_tabulate..Zs$A$A$AS1W$A$A$ArKcg|]Q\}}tt||krt|d|dz dznt|RS)Nz...rz)rr{ col_widths rIrz_tabulate..\sf   9033q66{{Y/F/FCFF?Y]? #e + +CPQFF   rKr) r\rshutilget_terminal_sizecolumnsrsumrr r\r) rrr col_widthsterminal_widthcol_to_minimize row_formatrrXrow_format_argss rIrrIsPO3;N;Ng;N;N;NOOOJ133;S\\B=NOON g,,Z (> 9 9$**3z??;;?###)### w<<#j// )^ ; ;*83z??*JSQX\\*Y\fgv\w*wJ ' g,,Z (> 9 9 4*s7||+3Z@J E LL""G,--- LL""$A$Aj$A$A$ABCCC::  #C 4 4     &Z&89999 99U  rKrqueuefunc.kwargscF|di|D]}||dS)Nr{)put)rrrr]s rI_write_generator_to_queuergs<$.... &rKrr|c#Ktfd|D} dVn?#t$r2td|DrrYnYnwxYwZn#t $rYnwxYw d|DdS#t j$rYdSwxYw# d|Dw#t j$rYwwxYwxYw)a; Takes a function that returns an iterable of items, and run it in parallel using threads to return the flattened iterable of items as they arrive. This is inspired by those three `map()` variants, and is the mix of all three: * `imap()`: like `map()` but returns an iterable instead of a list of results * `imap_unordered()`: like `imap()` but the output is sorted by time of arrival * `flatmap()`: like `map()` but given a function which returns a list, `flatmap()` returns the flattened list that is the concatenation of all the output lists cLg|] }t|f!Sr{) apply_asyncr)rrrrrs rIrz&iflatmap_unordered..|s2rrr\bT%%&?%vAVWWrrrrKT皙?rjc3>K|]}|VdSr)readyr async_results rIrz%iflatmap_unordered..s.NN |))++NNNNNNrKc:g|]}|dS)rrrqrs rIrz&iflatmap_unordered..s) N N N \  d  + + N N NrKN)r r5r remptyKeyboardInterruptrr)rrr| async_resultsrs`` @rIrrlsfggErrrrrrfqrrrM  iii------   NN NNNNNSXS^S^S`S`E           N N N N N N N N+    DD   N N N N N N N+    D sBAB7A<7B;A<<BB6 B B6 BB6 B B32B36C8 CCCCCC)NNNNNNNNNNFFNNFNNNNNNFF)FNNN)NFNN)FNNNrNNN)rAN)NN)FNN)NNFNN)NNN)NNNNNNNNNNNNFFNNFNNNNNNNNFF)NNNNNNNNNNNNFNNNNNNFF)FNNNNN)NNNNNNNNNNNNNNFNNNNNNNNFF)r{)rurrmultiprocessing.poolr3rrcollections.abcrrrr contextlibr dataclassesrr enumr pathlibr rr r typingrrr urllib.parserhuggingface_hubrrrrrrrhuggingface_hub._jobs_apirrrrhuggingface_hub.errorsrhuggingface_hub.utilsr$huggingface_hub.utils._cache_managerrhuggingface_hub.utils._hf_urisr!huggingface_hub.utils._parsingr"r# _cli_utilsr%r&r'r(r)r*r+r,r-r.r/r0r1r2r3r4r5r6 _frameworkr7r8_outputr9r:r;_uv_script_headerr<r=r> get_loggerrrloggerrvr7rJrxrarcintboolrryrrrrrrrrImageArgImageOpt FlavorOpt LabelsOptNameOpt TimeoutOpt AttemptsOpt DetachOpt DryRunOpt NamespaceOptResourceGroupIdOpt ExposeOptExposePublicOpt SshEnabledOptNetworkGroupOptNetworkAliasOptWithOpt PythonOpt SuspendOptConcurrencyOpt ScheduleArg ScriptArg ScriptArgsArg CommandArgJobIdArg JobIdsArgScheduledJobIdArg JobVolumesOptjobs_clirrrr8r>rGr_rcrrrrrrrrrruv_app add_grouprr scheduled_appr2rJrRrWr[r_rcrgrjscheduled_uv_approrr6rrrrrrrr{rKrIrsb&GF EEEEEEEEEEEE%%%%%%((((((((**********!!!!!!`````````````````` ,+++++))))))======777777JJJJJJJJ()(((((((::::::::::IIIIIIIIII  H % %..t.cSVY]S]oH^....8) #Y )',)9 RTejtCH~z#^bos6x3xsxCxSxxxx(   H '  $J F =  $J F r:k**   I F  B   $J Fv $J F _ $J F  d  c     ?AmLMAmN OAmP QAmAmAm  AmHy,L+MNN#xx  S Q    xx x xxxONx0x+J*KLL# = =' = =  = = = =ML =y,L+MNN# Q Q' Q Q  Q Q Q QON Qx+J*KLL# = =' = =  = = = =ML =|/^._``# ` `' ` ` `  `  ` ` `a` `y,L+MNN# Q Q' Q Q  Q Q Q QON Q bg" OO'O O O T66)2]^^^^ _ O  O  O OOOOB!=&QRRR (t444 .5LF"&"& #' %),0" 7s`s`s` s`s` s`  s`  s` s` s`s` s` s` s`s`!s`s` !s`"#s`$ %s`&#'s`(*)s`*+s`, -s`. /s`0 1s`2 3s`45s`6 7s`8 9s`s`s`s`r "S#X "4S>D3H "TRUWZRZ^ " " " "S4ZS D0@zZ]hlGKd3i$.ttTWY\T\~`dOd,DcCi)DI#6 GCLLU1XXc8A;>N5OY]bf    ) 3 # $d  a[ rK