j" d dl Z d dlZd dlZd dlmZ d dlmZ d dlmZm Z m
Z
m Z m Z m
Z
mZ d dlZddlmZ ddlmZmZmZ dd d
d Zd Z G d
de Zd deeef de e de
e eef e e f fdZdeeef dededdfdZd deeef de e ddfdZ d deeef de e deddfdZ d deeef de e de e e deddf
dZ dS ) N)contextmanager)Path)AnyDictIteratorListOptionalTupleUnion ) ImageFolder)check_integrityextract_archiveverify_str_arg)zILSVRC2012_img_train.tar 1d675b47d978889d74fa0da5fadfb00e)zILSVRC2012_img_val.tar 29b22e2961454d5413ddabcf34fc5622)z ILSVRC2012_devkit_t12.tar.gz fa75699e90414af021442c21a62c3abf)trainvaldevkitzmeta.binc v e Zd ZdZd deeef dededdf fd
Zd
d Z e
defd
Z defd Z xZ
S )ImageNeta] `ImageNet `_ 2012 Classification Dataset.
.. note::
Before using this class, it is required to download ImageNet 2012 dataset from
`here `_ and
place the files ``ILSVRC2012_devkit_t12.tar.gz`` and ``ILSVRC2012_img_train.tar``
or ``ILSVRC2012_img_val.tar`` based on ``split`` in the root directory.
Args:
root (str or ``pathlib.Path``): Root directory of the ImageNet Dataset.
split (string, optional): The dataset split, supports ``train``, or ``val``.
transform (callable, optional): A function/transform that takes in a PIL image
and returns a transformed version. E.g, ``transforms.RandomCrop``
target_transform (callable, optional): A function/transform that takes in the
target and transforms it.
loader (callable, optional): A function to load an image given its path.
Attributes:
classes (list): List of the class name tuples.
class_to_idx (dict): Dict with items (class_name, class_index).
wnids (list): List of the WordNet IDs.
wnid_to_idx (dict): Dict with items (wordnet_id, class_index).
imgs (list): List of (image path, class_index) tuples
targets (list): The class_index value for each image in the dataset
r rootsplitkwargsreturnNc t j | x}| _ t |dd | _ | t | j d t j | j
fi | || _ | j | _ | j
| _ fd| j D | _ d t | j D | _
d S )Nr )r r r c g | ]
}| S r ).0wnidwnid_to_classess NC:\Projects\Historical\venv\Lib\site-packages\torchvision/datasets/imagenet.py
z%ImageNet.__init__..= s EEE$-EEE c $ i | ]
\ }}|D ]}||S r r )r idxclssclss r#
z%ImageNet.__init__..> s- _ _ _)#tZ^ _ _SVS# _ _ _ _r% )ospath
expanduserr r r parse_archivesload_meta_filesuper__init__ split_folderclasseswnids class_to_idx wnid_to_idx enumerate)selfr r r r" __class__s @r# r1 zImageNet.__init__1 s 7--d333ty#E74DEE
(33A6*55f555 \
,EEEE$*EEE _ _y 7N7N _ _ _r% c l t t j | j t
st
| j t j | j sB| j dk rt | j d S | j dk rt | j d S d S d S )Nr r ) r r+ r, joinr META_FILEparse_devkit_archiveisdirr2 r parse_train_archiveparse_val_archiver8 s r# r. zImageNet.parse_archives@ s rw||DIy A ABB , + + +w}}T .// -zW$$#DI.....u$$!$),,,,, - - %$r% c V t j | j | j S N)r+ r, r; r r rA s r# r2 zImageNet.split_folderJ s w||DItz222r% c & dj di | j S )NzSplit: {split}r )format__dict__rA s r#
extra_reprzImageNet.extra_reprN s & &77 777r% )r )r N)__name__
__module__ __qualname____doc__r strr r r1 r. propertyr2 rG
__classcell__)r9 s @r# r r s 4
`
`U39 -
`c
`s
`W[
`
`
`
`
`
` - - - - 3c 3 3 3 X38C 8 8 8 8 8 8 8 8r% r r filer c |t }t j | | }t | rt j |d S d}t | || )NT) weights_onlyzThe meta file {} is not present in the root directory or is corrupted. This file is automatically created by the ImageNet dataset.) r< r+ r, r; r torchload RuntimeErrorrE )r rO msgs r# r/ r/ R so |
7<<d # #Dt 3z$T2222
J 3::dD11222r% md5c t t j | | | s%d}t | || d S )Nz{The archive {} is not present in the root directory or is corrupted. You need to download it externally and place it in {}.)r r+ r, r; rT rE )r rO rV rU s r# _verify_archiverX a sW 27<<d33S 9 9 3
E 3::dD11222 3 3r% c
H
ddl m
dt dt t t
t f t t t t df f f f
fd }dt dt t
fd}t dt t fd }t d }||d }|d
}t | || | 5 }t t j
| | | t j
|d } || \ } || }
fd |
D } t j | | ft j
| t" ddd dS # 1 swxY w Y dS )
aI Parse the devkit archive of the ImageNet2012 classification dataset and save
the meta information in a binary file.
Args:
root (str or ``pathlib.Path``): Root directory containing the devkit archive
file (str, optional): Name of devkit archive. Defaults to
'ILSVRC2012_devkit_t12.tar.gz'
r N devkit_rootr .c t j | dd } |d d t t d }fdt
| D t t d d \ }}}d |D }d
t || D }d t || D }||fS ) Ndatazmeta.matT)
squeeze_mesynsets c 2 g | ]\ }}|d k | S )r r )r r' num_childrenmetas r# r$ z@parse_devkit_archive..parse_meta_mat..y s- aaa /c<|_`O`O`S O`O`O`r% c R g | ]$}t | d %S )z, )tupler )r r( s r# r$ z@parse_devkit_archive..parse_meta_mat..{ s, ???t5D))**???r% c i | ]\ }}|| S r r )r r' r! s r# r* z@parse_devkit_archive..parse_meta_mat..| s CCCYS$sDCCCr% c i | ]\ }}|| S r r )r r! r( s r# r* z@parse_devkit_archive..parse_meta_mat..} s LLL*$4LLLr% )r+ r, r; loadmatlistzipr7 )
rZ metafile
nums_childrenidcsr4 r3 idx_to_wnidr" rb sios
@r# parse_meta_matz,parse_devkit_archive..parse_meta_matu s 7<< VZ@@{{8{55i@ S$Z((+
aaaa9]3K3Kaaa #CJ / / 3 eW??w???CC#dE2B2BCCC LLE78K8KLLLO++r% c t j | dd }t | 5 }| }d d d n # 1 swxY w Y d |D S )Nr\ z&ILSVRC2012_validation_ground_truth.txtc , g | ]}t | S r )int)r val_idxs r# r$ zKparse_devkit_archive..parse_val_groundtruth_txt.. s 555G 555r% )r+ r, r; open readlines)rZ rO txtfhval_idcss r# parse_val_groundtruth_txtz7parse_devkit_archive..parse_val_groundtruth_txt s w||K1YZZ
$ZZ )5 ((H ) ) ) ) ) ) ) ) ) ) ) ) ) ) )55H5555s AAAc 3 K t j } | V t j | d S # t j | w xY wrC )tempfilemkdtempshutilrmtree)tmp_dirs r# get_tmp_dirz)parse_devkit_archive..get_tmp_dir sN "$$ #MMM M' " " " " "FM' " " " "s 1 Ar r ILSVRC2012_devkit_t12c g | ]
}| S r r )r r' rn s r# r$ z(parse_devkit_archive.. s :::#[%:::r% )scipy.ioiorL r
r rs r r r ARCHIVE_METArX r r+ r, r; rR saver< )r rO rp ry r archive_metarV r rZ r" rx val_wnidsrn ro s @@r# r= r= j s ,C ,E$sCx.$sERUWZRZOG[B\2\,] , , , , , ,6s 6tCy 6 6 6 6 ## # # # ^# )L |A
q/CD$$$$ P' T400':::gll7,CDD '5~k'B'B$ _,,[99:::::::
OY/ dI1N1NOOOP P P P P P P P P P P P P P P P P Ps B0FF Fr folderc t d }||d }|d }t | || t j | | t t j | | fdt j D }|D ]7}t |t j | d d 8dS )a Parse the train images archive of the ImageNet2012 classification dataset and
prepare it for usage with the ImageNet dataset.
Args:
root (str or ``pathlib.Path``): Root directory containing the train images archive
file (str, optional): Name of train images archive. Defaults to
'ILSVRC2012_img_train.tar'
folder (str, optional): Optional name for train images folder. Defaults to
'train'
r Nr r c P g | ]"}t j | #S r r+ r, r; )r archive
train_roots r# r$ z'parse_train_archive.. s) XXXg Z11XXXr% T)remove_finished)r rX r+ r, r; r listdirsplitext)r rO r r rV archivesr r s @r# r? r? s (L |A
q/CD$$$$ dF++JBGLLt,,j999XXXXJAWAWXXXH U U!1!1'!:!:1!=tTTTTTU Ur% r r4 c t d }||d }|d }|t | d }t | || t j | | t
t j | | t fdt j D }t | D ]4}t j
t j | 5t || D ]V\ }}t j
|t j |t j | WdS )az Parse the validation images archive of the ImageNet2012 classification dataset
and prepare it for usage with the ImageNet dataset.
Args:
root (str or ``pathlib.Path``): Root directory containing the validation images archive
file (str, optional): Name of validation images archive. Defaults to
'ILSVRC2012_img_val.tar'
wnids (list, optional): List of WordNet IDs of the validation images. If None
is given, the IDs are loaded from the meta file in the root directory
folder (str, optional): Optional name for validation images folder. Defaults to
'val'
r Nr r c 3 X K | ]$}t j | V %d S rC r )r imageval_roots r# z$parse_val_archive.. s3 TTeBGLL511TTTTTTr% )r r/ rX r+ r, r; r sortedr setmkdirrj r} movebasename)
r rO r4 r r rV imagesr! img_filer s
@r# r@ r@ sJ &L |A
q/C } t$$Q'D$$$$w||D&))HBGLLt,,h777
TTTTrz(?S?STTT
T
TFE
/ /
h--.... eV,, X Xh Hbgll8T27;K;KH;U;U V VWWWWX Xr% rC )Nr )NNr ) r+ r} r{
contextlibr pathlibr typingr r r r r r
r rR r r
utilsr r r r r< r rL r/ rX r= r? r@ r r% r# r s
% % % % % % D D D D D D D D D D D D D D D D D D C C C C C C C C C C N IR
98 98 98 98 98{ 98 98 98x 3 3sDy) 3# 3%PTUXZ]U]P^`deh`iPiJj 3 3 3 3 3%T * 3# 3C 3D 3 3 3 33P 3PuS$Y / 3Px} 3PPT 3P 3P 3P 3PlU UeCI . Uhsm UTW Ufj U U U U6 jo!X !X
T
!X"*3-!X?GS ?R!Xcf!X !X !X !X !X !X !Xr%