Xj>\$& dZddlmZmZddlmZddlZddlmZddl m Z m Z m Z m Z mZddlmZd d lmZmZmZmZd d lmZd d lmZmZd d lmZmZd dlmZm Z ej!e"Z#edZ$e$%dgd ddee&edfdee'eddfdeedzeddfdee'eddfd ed!df d"Z(d#e&d!e'fd$Z)e$%d%gd& dd#ee&dzed'fd(ee'ed)d*d+fd,ee'ed-d.fd/ee'ed0d1d2fd3ed ed!dfd4Z*d5e&dzd3e&dzd(e'd,e'd/e'd e&dzd!dfd6Z+d7ee e zd8e&d!ee e zfd9Z,d#e&d(e'd,e'd/e'd e&dzd!df d:Z-e$%d;dfd ed!dfd?Z.e$%d@gdA ddee&ed>fdBee'edCdDdEfdFee'edGdHfd ed!df dIZ/e$%dJgdK dd#ee&edLfd/ee'ed0d1dMfdBee'edCdDdEfdNee'edOdPfdQee0e&dzedRfdSee0e&dzedTfd ed!dfdUZ1e$%dVgdW ddXee&edYfdZee&ed[fd ed!dfd\Z2e$%d]gd^ ddee&ed>fdee'edd_fd`ee'edadbfd ed!df dcZ3e$%ddgde ddfee&dzedgfdhee&dzedifd@ee'edjfdkee'edldmfdnee'edodpfdqee&dzedrfdsee&dzedtfdNee'edOdufdQee0e&dzedvfdSee0e&dzedwfdxee&dzedyfdzee'ed{d|fd}ee'ed~dfdee'edddfd ed!df dZ4e$%dgdeddS)z7Contains commands to interact with buckets via the CLI.)IterableIterator) AnnotatedN)logging) BUCKET_PREFIX BucketFile BucketFolder FilterMatcher_parse_bucket_uri) REPO_REGIONS) SearchOptTokenOpt get_hf_api typer_factory)make_cp) format_sizeprint_file_listing)ArgumentOption) OutputFormatoutz"Commands to interact with buckets.)helpcreate)zhf buckets create my-bucketz hf buckets create user/my-bucketz-hf buckets create hf://buckets/user/my-bucketz*hf buckets create user/my-bucket --privatez+hf buckets create user/my-bucket --exist-okz,hf buckets create user/my-bucket --region us)nameexamplesF bucket_idzTBucket ID: bucket_name, namespace/bucket_name, or hf://buckets/namespace/bucket_nameprivatez --privatezCreate a private bucket.regionz--regionzdCloud region in which to create the bucket. Can be one of 'us' or 'eu'. Requires Team plan or above.exist_okz --exist-okz3Do not raise an error if the bucket already exists.tokenreturnctt|}|tr=t|}|jr t jd|dtd|j}|||r|nd||}tj d|j |j dS) zCreate a new bucket.r"z-Cannot specify a prefix for bucket creation: . Use namespace/bucket_name or namespace/bucket_name.N)rr r!zBucket created)uriurl)r startswithrr path_in_repoclick BadParameterid create_bucketrresultr(to_urir))rrr r!r"apiparsed bucket_urls LC:\Projects\Historical\venv\Lib\site-packages\huggingface_hub/cli/buckets.pyrr1sT 5 ! ! !CM**"9--   $W WW1>WWW I """, #J JZ^%:%:%<%<*.QQQQQQargumentc||tr|ttd}n|}d|vS)zFCheck if argument is a bucket ID (namespace/name) vs just a namespace.N/)r*rlen)r7paths r5 _is_bucket_idr<osA=))M**,,- $;r6z list | ls) zhf buckets listzhf buckets list huggingfacez$hf buckets list --search "my-prefix"zhf buckets list user/my-bucketz!hf buckets list user/my-bucket -Rz!hf buckets list user/my-bucket -hz%hf buckets list user/my-bucket --treez(hf buckets list user/my-bucket --tree -hz+hf buckets list hf://buckets/user/my-bucketz%hf buckets list user/my-bucket/sub -RzyNamespace (user or org) to list buckets, or bucket ID (namespace/bucket_name(/prefix) or hf://buckets/...) to list files.human_readablez--human-readablez-hz$Show sizes in human readable format.as_treez--treez3List files in tree format (only for listing files). recursivez --recursivez-Rz0List files recursively (only for listing files).searchc|duot|}|r,|tjdt|||||dSt ||||||dS)zList buckets or files in a bucket. When called with no argument or a namespace, lists buckets. When called with a bucket ID (namespace/bucket_name), lists files in the bucket. Nz'Cannot use --search when listing files.)r7r=r>r?r") namespacer@r=r>r?r")r<r,r- _list_files _list_buckets)r7r=r>r?r@r" is_file_modes r5list_cmdrFxsr4'CM(,C,CL  $%NOO O)        )       r6rBc|rtjd|rtjd|K|tr1|t td}|d}t |}fd|||D}tj |dd i dS) zList buckets in a namespace.z'Cannot use --tree when listing buckets.z,Cannot use --recursive when listing buckets.Nr9r%cg|]:}|j|jrt|jn|j|j|jd;S))r.rsize total_files created_at)r.rrrIrJrK).0bucketr=s r5 z!_list_buckets..se     )~@N_K ^<<r?r"r2itemss ` r5rDrDsL !JKKKQ !OPPP!5!5m!D!Dc-00223 $$S)) 5 ! ! !C    &&6&JJ   EIe 1222222r6rTprefixc#K|d}|D]0}|r(|j|ks|j|dr|V1dS)aDrop entries that are not `prefix` itself or below it. The server matches `prefix` lexically, so listing "logs" also returns "logs.json" or "logs_backup/...". Filesystem semantics require path components => keep only what's actually under the prefix. r9N)rQr;r*)rTrUitems r5_iter_under_prefixrXsk ]]3  F f,, 0D0D\\\0R0R,JJJr6c R|r.tjtjkrt jdt |}t|}tt| |j |j pd||j }t||||dS)zList files in a bucket.z%Cannot use --tree with --format json.r%NrUr?)r=r>r?)rmoderjsonr,r-rr listrXlist_bucket_treer.r+r)r7r=r>r?r"r2r3rTs r5rCrCsJ38|000 !HIII 5 ! ! !C x ( (F    *2d# !         Eu^WXabbbbbbr6infozhf buckets info user/my-bucketz+hf buckets info hf://buckets/user/my-bucketzFBucket ID: namespace/bucket_name or hf://buckets/namespace/bucket_namect|}t|}||j}t j|ddS)zGet info about a bucket.r%r.)id_keyN)rr bucket_infor.rdict)rr"r2r3rMs r5r_r_sO" 5 ! ! !C y ) )F __VY ' 'FHVD!!!!!!r6delete)z hf buckets delete user/my-bucketz-hf buckets delete hf://buckets/user/my-bucketz&hf buckets delete user/my-bucket --yesz-hf buckets delete user/my-bucket --missing-okyesz--yesz-yzSkip confirmation prompt. missing_okz --missing-okz3Do not raise an error if the bucket does not exist.c|tr>t|}|jr t jd|dtd|j}n$d|vr t jd|dtdtjd|d| t| }| || tj d | dS)zDelete a bucket. This deletes the entire bucket and all its contents. Use `hf buckets rm` to remove individual files. z-Cannot specify a prefix for bucket deletion: r&r'r9Invalid bucket ID: -. Must be in format namespace/bucket_name or z(Are you sure you want to delete bucket ''?rer%)rfzBucket deleted)rN) r*rr r+r,r-r.rconfirmr delete_bucketr0)rrerfr"r3r2s r5rdrd)s-JM**  "9--   $W WW1>WWW I I    a) a a;H a a a   KH9HHHcRRRR 5 ! ! !CiJ777J9555555r6z remove | rm)z)hf buckets remove user/my-bucket/file.txtz2hf buckets rm hf://buckets/user/my-bucket/file.txtz.hf buckets rm user/my-bucket/logs/ --recursivez:hf buckets rm user/my-bucket --recursive --include "*.tmp"z8hf buckets rm user/my-bucket/data/ --recursive --dry-runzBucket path: namespace/bucket_name/path or hf://buckets/namespace/bucket_name/path. With --recursive, namespace/bucket_name is also accepted to target all files.z0Remove files recursively under the given prefix.dry_runz --dry-runz8Preview what would be deleted without actually deleting.includezQInclude only files matching pattern (can specify multiple). Requires --recursive.excludezLExclude files matching pattern (can specify multiple). Requires --recursive.cht|}|j|j}|dkr|stjddd|s|r|stjdt |} |r%t jd} g} t| |pdd |D]R} t| tr;| | | d t| d S| d t| d |s|r t!|| fd| D} n| } d| D}t#d| D}t%|d }|st jddSt|d|}|sP|sNt jdd|Dt jd|ddd|rMt jdfd|Dt jd|ddS| |t jd|dd t||!dS|}|stjd"|r7t jd#t0d$|t jd%dSt jd&|d'd|| |gt jd(|)dS)*zbRemove files from a bucket. To delete an entire bucket, use `hf buckets delete` instead. z?No file path specified. To remove files, provide a path (e.g. 'zd/FILE') or use --recursive to remove all files. To delete the entire bucket, use `hf buckets delete z`.z,--include and --exclude require --recursive.r%zListing files from remoteNTrZzListing files from remote (z files))include_patternsexclude_patternscHg|]}|j|S)matchesr;)rLfmatchers r5rNzremove..s,MMM1W__QV5L5LMQMMMr6cg|] }|j Srv)r;rLrxs r5rNzremove..s444af444r6c3$K|] }|jV dSN)rIr{s r5 zremove..s$77A777777r6)r=zNo files to remove.z file(s) totaling  c3 K|] }d|V dS)z Nrv)rLr;s r5r~zremove..s(BBt{D{{BBBBBBr6zRemove z from 'rjFrkc36K|]}dtd|VdS)delete: r9N)r)rLr;rs r5r~zremove..s:ccPTK-KKKTKKccccccr6z (dry run) z would be removed.)rdzRemoved ')r files_deletedrIzFile path cannot be empty.rr9z"(dry run) 1 file would be removed.zRemove 'z' from 'z File removed)r;r)r r.r+r,r-rrstatusrXr^ isinstancerappendupdater:doner sumrtextjoinrlbatch_bucket_filesr0r)r7r?rernrorpr"r3rUr2r all_filesrW matched_files file_paths total_sizesize_str count_label file_pathrrys @@r5removercs"zx ( (F I  F ||I|  R  R RDM R R R   Q7QIQ !OPPP 5 ! ! !C?H788&( &  ~ !      U UD$ ++ U  &&& SC NNSSSTTT I#i..IIIJJJ  &g &#WwWWWGMMMM MMMMM%M44m444 7777777 z$???  H* + + + FZFFHFF  P7 P HTYYBBzBBBBB C C C KC+CCiCCC O O O O   HTYYccccXbccccc d d d HA+AAA B B B F y<<< 7{ 7 79 7 7 7j//         C$%ABB B   HF FyFF9FF G G G H9 : : : F ?y??)???SIIII y)=== > YGGGGGGr6move)z/hf buckets move user/old-bucket user/new-bucketz/hf buckets move user/my-bucket my-org/my-bucketzIhf buckets move hf://buckets/user/old-bucket hf://buckets/user/new-bucketfrom_idzMSource bucket ID: namespace/bucket_name or hf://buckets/namespace/bucket_nameto_idzRDestination bucket ID: namespace/bucket_name or hf://buckets/namespace/bucket_namect|}|jr tjd|dtdt|}|jr tjd|dtdt |}||j|jtj d|j|jdS)z2Move (rename) a bucket to a new name or namespace.z)Cannot specify a prefix for bucket move: r&r'r%)rrz Bucket movedN) r r+r,r-rr move_bucketr.rr0)rrr" parsed_from parsed_tor2s r5rrs2$G,,K   S S S-: S S S   "%((I   S S S-: S S S   5 ! ! !COOKN),O???J~{~Y\JJJJJJr6settings)z,hf buckets settings user/my-bucket --privatez+hf buckets settings user/my-bucket --publicz9hf buckets settings hf://buckets/user/my-bucket --privatezMake the bucket private.publicz--publiczMake the bucket public.c||krtjd|tr>t |}|jr tjd|dtd|j}n$d|vr tjd|dtdt|}||| tj d || tj d |d dS)z$Update bucket settings (visibility).z-Specify exactly one of --private or --public.z-Cannot specify a prefix for bucket settings: r&r'r9rhrir%)rzBucket settings updated)rrzRun `hf buckets info z` to view bucket details.N) r,r-r*rr r+r.rupdate_bucket_settingsrr0hint)rrrr"r3r2s r5rrsG@& !PQQQM**  "9--   $W WW1>WWW I I    a) a a;H a a a   5 ! ! !Cy':::J(IwOOOOH IY I I IJJJJJr6sync)z2hf buckets sync ./data hf://buckets/user/my-bucketz2hf buckets sync hf://buckets/user/my-bucket ./dataz;hf buckets sync ./data hf://buckets/user/my-bucket --deletez^hf buckets sync hf://buckets/user/my-bucket ./data --include "*.safetensors" --exclude "*.tmp"zIhf buckets sync ./data hf://buckets/user/my-bucket --plan sync-plan.jsonlz'hf buckets sync --apply sync-plan.jsonlzrsm>=........ ######"!!!!! ::::::::((((((((&&&&&&&&  H % %m!EFFF     0   90R0R g    0R  +    0R t w     0R*  F    +0R8 90R: ;0R0R0R  0RfCD     0    E> >  d W    >    7    > $  F    %> 2   C    3> B C> D E> F G> > > > B 3Tz 3 $J 3 3 3  3 : 3  3 3 3 3F hzL'@A 3 S[\fiu\uSv    cccc c : c  cccc6  (5 " " Y     "  "  " " " "   .  -.6.6 Y    .6    ,     .6  F    .6, -.6. /.6.6.6.6b    6    [BHBH a    BH   C    BH$    ,     %BH4  K    5BHB S Dd    CBHN S D_    OBHZ [BH\ ]BHBHBH  BHJ  *"K"K  `    "K  e     "K "K "K"K"K"KJ  *  +,K,K Y    ,K  +    ,K   *     ,K* +,K, -,K,K,K,Kh       &         yrPrP  d ^     rP  d c     rP  B     rP&  N    'rP4  O    5rPB  d P     CrPN  d 6     OrPZ  H    [rPh S DI    irPt S DI    urP@ d ;    ArPL  T    MrPZ  V    [rPh   8    irPx yrPz {rPrPrP  rPv      ')r6