j3$ddlZddlZddlmZddlmZmZmZmZm Z m Z m Z m Z ddl mZddlmZdede ee ed ffd efd Zded efd Zd e eefd e e eeeefffdZ dd e eefde eeefde e ee ed ffde eegefded e e eeff dZGddeZdZded ejfdZded efdZded efdZGddeZdS)N)Path)AnyCallablecastDictListOptionalTupleUnion)Image) VisionDatasetfilename extensions.returnc|t|tr|nt |S)aChecks if a file is an allowed extension. Args: filename (string): path to a file extensions (tuple of strings): extensions to consider (lowercase) Returns: bool: True if the filename ends with one of given extensions )lowerendswith isinstancestrtuple)rrs LC:\Projects\Historical\venv\Lib\site-packages\torchvision/datasets/folder.pyhas_file_allowed_extensionr s? >>   $ $:j#3N3N%eZZTYZdTeTe f ffc,t|tS)zChecks if a file is an allowed image extension. Args: filename (string): path to a file Returns: bool: True if the filename ends with a known image extension )rIMG_EXTENSIONS)rs r is_image_filers &h ? ??r directoryctdtj|D}|std|ddt |D}||fS)zWFinds the class folders in a dataset. See :class:`DatasetFolder` for details. c3LK|]}||jV dSN)is_dirname).0entrys r zfind_classes..)s1UUEellnnUUZUUUUUUrz"Couldn't find any class folder in .ci|]\}}|| Sr))r$icls_names r z find_classes..-sFFFKAxHaFFFr)sortedosscandirFileNotFoundError enumerate)rclasses class_to_idxs r find_classesr4$ss UURZ -B-BUUUUUG S QY Q Q QRRRFF9W3E3EFFFL L  rFr3 is_valid_file allow_emptyctj|}|t|\}}n|st dduo|du}duo|du}|s|rt ddt dt ffd }ttt gt f|}g}t} t| D]} || } tj || } tj | sJttj| dD]u\} }}t|D]_}tj | |}||r2|| f}||| | vr| | `vt| | z }|rk|sid d t|d }2|d t#t rnd z }t%||S) zGenerates a list of samples of a form (path_to_sample, class). See :class:`DatasetFolder` for details. Note: The class_to_idx parameter is here optional and will use the logic of the ``find_classes`` function by default. NzE'class_to_index' must have at least one entry to collect any samples.zMBoth extensions and is_valid_file cannot be None or not None at the same timexrc$t|Sr!)r)r8rs rr5z#make_dataset..is_valid_fileMs-a<< >MI|002233 < < "<0 W\\)\:: w}}Z((  %bgjd&K&K&KLL < Y>Y0t _c_h_his_t_tvv vC$$$ rceZdZdZ ddeeefdeegefde e edfde ed e ed e eege fd e d dffd Z e ddeeefdeeefde e edfd e eege fd e d ee eeff dZdeeefd e eeeeefffdZded e eeffdZd efdZxZS) DatasetFolderaA generic data loader. This default directory structure can be customized by overriding the :meth:`find_classes` method. Args: root (str or ``pathlib.Path``): Root directory path. loader (callable): A function to load a sample given its path. extensions (tuple[string]): A list of allowed extensions. both extensions and is_valid_file should not be passed. transform (callable, optional): A function/transform that takes in a sample and returns a transformed version. E.g, ``transforms.RandomCrop`` for images. target_transform (callable, optional): A function/transform that takes in the target and transforms it. is_valid_file (callable, optional): A function that takes path of a file and check if the file is a valid file (used to check of corrupt files) both extensions and is_valid_file should not be passed. allow_empty(bool, optional): If True, empty folders are considered to be valid classes. An error is raised on empty folders if False (default). Attributes: classes (list): List of the class names sorted alphabetically. class_to_idx (dict): Dict with items (class_name, class_index). samples (list): List of (sample path, class_index) tuples targets (list): The class_index value for each image in the dataset NFrNloaderr. transformtarget_transformr5r6rc0t|||||j\}} ||j| |||} ||_||_||_| |_| |_ d| D|_ dS)N)rXrY)r3rr5r6cg|] }|d S)r r))r$ss r z*DatasetFolder.__init__..s...!...r) super__init__r4rNrTrWrr2r3samplestargets) selfrNrWrrXrYr5r6r2r3r` __class__s rr_zDatasetFolder.__init__s EUVVV $ 1 1$) < <## I%!'# $   $ ( ..g... rrr3cL|tdt|||||S)aGenerates a list of samples of a form (path_to_sample, class). This can be overridden to e.g. read files from a compressed zip file instead of from the disk. Args: directory (str): root dataset directory, corresponding to ``self.root``. class_to_idx (Dict[str, int]): Dictionary mapping class name to class index. extensions (optional): A list of allowed extensions. Either extensions or is_valid_file should be passed. Defaults to None. is_valid_file (optional): A function that takes path of a file and checks if the file is a valid file (used to check of corrupt files) both extensions and is_valid_file should not be passed. Defaults to None. allow_empty(bool, optional): If True, empty folders are considered to be valid classes. An error is raised on empty folders if False (default). Raises: ValueError: In case ``class_to_idx`` is empty. ValueError: In case ``extensions`` and ``is_valid_file`` are None or both are not None. FileNotFoundError: In case no valid file was found for any class. Returns: List[Tuple[str, int]]: samples of a form (path_to_sample, class) Nz*The class_to_idx parameter cannot be None.)rr5r6)r=rT)rr3rr5r6s rrTzDatasetFolder.make_datasets?@  IJJ J | -ep    rc t|S)utFind the class folders in a dataset structured as follows:: directory/ ├── class_x │ ├── xxx.ext │ ├── xxy.ext │ └── ... │ └── xxz.ext └── class_y ├── 123.ext ├── nsdf3.ext └── ... └── asd932_.ext This method can be overridden to only consider a subset of classes, or to adapt to a different dataset directory structure. Args: directory(str): Root directory path, corresponding to ``self.root`` Raises: FileNotFoundError: If ``dir`` has no class folders. Returns: (Tuple[List[str], Dict[str, int]]): List of all classes and dictionary mapping each class to an index. )r4)rbrs rr4zDatasetFolder.find_classess6I&&&rindexc|j|\}}||}|j||}|j||}||fS)z Args: index (int): Index Returns: tuple: (sample, target) where target is class_index of the target class. )r`rWrXrY)rbrfr;targetsamples r __getitem__zDatasetFolder.__getitem__sd|E* fT"" > %^^F++F  ,**622Fv~rc*t|jSr!)lenr`)rbs r__len__zDatasetFolder.__len__s4<   r)NNNNF)NNF)__name__ __module__ __qualname____doc__r rrrrr r r>r_ staticmethodrintrrTr4rjrm __classcell__rcs@rrVrVms@15(,/39=!//CI/#$/U38_- / H% / #8, / # 56// //////8159=! & & d#& 38n& U38_-&  # 56 &  & eCHo  & & & \& P'eCI&6'5cDQTVYQYNAZ;['''':sCx"!!!!!!!!!rrV) z.jpgz.jpegz.pngz.ppmz.bmpz.pgmz.tifz.tiffz.webpr;ct|d5}tj|}|dcdddS#1swxYwYdS)NrbRGB)openr convert)r;fimgs r pil_loaderr}s dD  "Qjmm{{5!!""""""""""""""""""s)AA A ctddl} ||S#t$rt|cYSwxYw)Nr)accimager OSErrorr})r;rs raccimage_loaderr sQOOO ~~d###    $ s 77cfddlm}|dkrt|St|S)Nr)get_image_backendr) torchvisionrrr})r;rs rdefault_loaderrsD------j((t$$$$rceZdZdZddeddfdeeefdee dee de ege fdee ege fd e f fd Z xZ S) ImageFolderatA generic data loader where the images are arranged in this way by default: :: root/dog/xxx.png root/dog/xxy.png root/dog/[...]/xxz.png root/cat/123.png root/cat/nsdf3.png root/cat/[...]/asd932_.png This class inherits from :class:`~torchvision.datasets.DatasetFolder` so the same methods can be overridden to customize the dataset. Args: root (str or ``pathlib.Path``): Root directory path. transform (callable, optional): A function/transform that takes in a PIL image and returns a transformed version. E.g, ``transforms.RandomCrop`` target_transform (callable, optional): A function/transform that takes in the target and transforms it. loader (callable, optional): A function to load an image given its path. is_valid_file (callable, optional): A function that takes path of an Image file and check if the file is a valid file (used to check of corrupt files) allow_empty(bool, optional): If True, empty folders are considered to be valid classes. An error is raised on empty folders if False (default). Attributes: classes (list): List of the class names sorted alphabetically. class_to_idx (dict): Dict with items (class_name, class_index). imgs (list): List of (image path, class_index) tuples NFrNrXrYrWr5r6c t|||tnd|||||j|_dS)N)rXrYr5r6)r^r_rr`imgs)rbrNrXrYrWr5r6rcs rr_zImageFolder.__init__?sS   +3NN-'#    L r)rnrorprqrr rrr rrr>r_rtrus@rrrsD)-/3'59=!!!CI!H%!#8, ! #$ !  # 56 !!!!!!!!!!!rr)NNNF)r.os.pathpathlibrtypingrrrrrr r r PILr visionrrr>rrrsr4rTrVrr}rrrr)rrrs JJJJJJJJJJJJJJJJJJJJ!!!!!! g g%U3PS8_@T:U gZ^ g g g g @C @D @ @ @ @ !E#t), !tCy$sCx.7P1Q ! ! ! !.28<59 99S$Y94S>*9sE#s(O3459HcUD[12 9  9  %S/ 9999xQ!Q!Q!Q!Q!MQ!Q!Q!h]"S"U["""" # #          2!2!2!2!2!-2!2!2!2!2!r