BjM T d Z ddlmZ dZdgZddlmZ ddlZddlm Z m
Z
m Z m Z m
Z
mZmZmZmZmZmZ ddlmZmZmZmZmZmZ dd lm Z m Z dd
l m Z m Z m!Z!m"Z" dd l#m$Z$ erdd l%m&Z& dd
lm'Z' ddl(m)Z)m*Z*m+Z+ dZ,e
e e-e-f e-e-gdf Z. G d dee Z/ G d de! Z0dS )zCUse the HTMLParser library to parse HTML files that aren't too bad. ) annotationsMITHTMLParserTreeBuilder)
HTMLParserN) AnyCallablecastDictIterableListOptional
TYPE_CHECKINGTupleTypeUnion)
AttributeDictCDataComment DeclarationDoctypeProcessingInstruction)EntitySubstitution
UnicodeDammit)DetectsXMLParsedAsHTMLHTMLHTMLTreeBuilderSTRICTParserRejectedMarkup)
BeautifulSoup)NavigableString) _Encoding
_Encodings
_RawMarkupz html.parserc e Zd ZU dZded< dZded< edd.dZd ed < ded< ded< d/dZd0dZ d1d2d Z d1d3d Z
d4d Z e j
d! Z e j
d" Zed5d% Zd6d&Zd6d'Zd4d(Zd7d*Zd4d+Zd4d,Zd-S )8BeautifulSoupHTMLParserreplacestrREPLACEignoreIGNOREon_duplicate_attributesoupr argsr r- &Union[str, _DuplicateAttributeHandler]kwargsc || _ || _ |j j | _ t j | g|R i | g | _ | d S N)r. r- builderattribute_dict_classr __init__ already_closed_empty_element_initialize_xml_detector)selfr. r- r/ r1 s HC:\Projects\Historical\venv\Lib\site-packages\bs4/builder/_htmlparser.pyr6 z BeautifulSoupHTMLParser.__init__U sc &<#$(L$E!D242226222 -/) %%''''' z List[str]r7 messagereturnNonec t | r3 r )r9 r< s r: errorz BeautifulSoupHTMLParser.errorp s #7+++r; tagattrs List[Tuple[str, Optional[str]]]c d | ||d | |d dS )zHandle an incoming empty-element tag.
html.parser only calls this method when the markup looks like
.
F)handle_empty_elementcheck_already_closedN)handle_starttag
handle_endtag)r9 rA rB s r: handle_startendtagz*BeautifulSoupHTMLParser.handle_startendtag s@
S%e DDD
3U;;;;;r; TrE boolc 4 | }|D ]Y\ }}|d}||v rG| j }|| j k r |d| j fv r|||< 1t t
| } |||| T|||< Z| j j j r| \ }} ndx}} | j
|dd||| }
|
:|
j r3|r1| |d | j
| | j | | dS dS )zHandle an opening tag, e.g. ''
:param handle_empty_element: True if this tag is known to be
an empty-element tag (i.e. there is not expected to be any
closing tag).
N )
sourceline sourceposFrF )r5 r- r+ r) r _DuplicateAttributeHandlerr. r4 store_line_numbersgetposrH is_empty_elementrI r7 append_root_tag_name_root_tag_encountered) r9 rA rB rE attr_dictkeyvalueon_duperN rO tagObjs r: rH z'BeautifulSoupHTMLParser.handle_starttag sv $(#<#<#>#> ' 'JC }i 5dk)) t| 444%*IcNN "#=w G GGGIsE2222!& # 9 / *$(KKMM !J %) )J** tY: +
&"9 >R
s ? ? ?
- 4 4S 9 9 9 & & &s + + + + + ' &r; rG c |r%|| j v r | j | dS | j | dS )zHandle a closing tag, e.g. ''
:param tag: A tag name.
:param check_already_closed: True if this tag is expected to
be the closing portion of an empty-element tag,
e.g. ''.
N)r7 remover. rI )r9 rA rG s r: rI z%BeautifulSoupHTMLParser.handle_endtag sT )C4+L$L$L
- 4 4S 9 9 9 9 9 I # #C ( ( ( ( (r; datac : | j | dS )z4Handle some textual data that shows up between tags.N)r. handle_datar9 r^ s r: r` z#BeautifulSoupHTMLParser.handle_data s d#####r; z
^([0-9]+)(.*)z^([0-9a-f]+)(.*)nameTuple[str, bool, str]c d}d}d}d}| j }| d s| d r|dd }d}| j }d} t || }ni# t $ r\ | | }|Bt | d | }| d }Y nw xY w|d}|}nt j | \ }}|||fS )
a Convert a numeric character reference into an actual character.
:param name: The number of the character reference, as
obtained by html.parser
:return: A 3-tuple (dereferenced, replacement_added,
extra_data). `dereferenced` is the dereferenced character
reference, or the empty string if there was no
reference. `replacement_added` is True if the reference
could only be dereferenced by replacing content with U+FFFD
REPLACEMENT CHARACTER. `extra_data` is a portion of data
following the character reference, which was deemed to be
normal data and not part of the reference at all.
rM F
xX N r ) &_DECIMAL_REFERENCE_WITH_FOLLOWING_DATA
startswith"_HEX_REFERENCE_WITH_FOLLOWING_DATAint
ValueErrorsearchgroupsr numeric_character_reference) clsrb dereferencedreplacement_added
extra_databasereg real_namematchs r: (_dereference_numeric_character_referencez@BeautifulSoupHTMLParser._dereference_numeric_character_reference s !&
8 ??3 94??3#7#7 98DD8C"& /D$II / / / JJt$$E q 14 8 8 "\\^^A .
/" L JJ.;.WXa.b.b +L+ .
::s A! !A#CCc | | \ }}}|r d| j _ || | || | dS dS )zHandle a numeric character reference by converting it to the
corresponding Unicode character and treating it as textual
data.
:param name: Character number, possibly in hexadecimal.
TN)rz r. contains_replacement_charactersr` )r9 rb rs rt ru s r: handle_charrefz&BeautifulSoupHTMLParser.handle_charref" st 7;6c6cdh6i6i3 ' =8 )rA r( rB rC r= r> )T)rA r( rB rC rE rK r= r> )rA r( rG rK r= r> )r^ r( r= r> )rb r( r= rc )rb r( r= r> )r r( r= r> )__name__
__module__ __qualname__r) __annotations__r+ r6 r@ rJ rH rI r` recompilerj rl classmethodrz r} r r r r r r; r: r& r& > s G F $ JQ ( ( ( ( ( (. CBBB++++, , , , < < < <0 &* <, <, <, <, <,|) ) ) ) )$$ $ $ $ .8RZ-H-H*)34F)G)G&4; 4; 4; [4;l
)
)
)
) &# # # ## # # #
1 1 1 1 1 1r; r& c e Zd ZU dZdZded< dZded< eZded< ee e
gZ d ed
< d ed < dZ ded
< d#d$ fd
Z
d%d&d Zefd'd"Z xZS )(r zA Beautiful soup `bs4.builder.TreeBuilder` that uses the
:py:class:`html.parser.HTMLParser` parser, found in the Python
standard library.
FrK is_xmlT picklabler( NAMEz
Iterable[str]featuresz$Tuple[Iterable[Any], Dict[str, Any]] parser_argsTRACKS_LINE_NUMBERSNOptional[Iterable[Any]]
parser_kwargsOptional[Dict[str, Any]]r1 r c t }dD ] }||v r| | }|||< ! t t | j di | |pg }|pi }| | d|d< ||f| _ dS )a Constructor.
:param parser_args: Positional arguments to pass into
the BeautifulSoupHTMLParser constructor, once it's
invoked.
:param parser_kwargs: Keyword arguments to pass into
the BeautifulSoupHTMLParser constructor, once it's
invoked.
:param kwargs: Keyword arguments for the superclass constructor.
r, Fconvert_charrefsNr )dictpopsuperr r6 updater )r9 r r r1 extra_parser_kwargsargrY __class__s r: r6 z HTMLParserTreeBuilder.__init__ s $ #ff. 1 1Cf}}
3+0#C(3#T**3==f===!'R %+
0111,1
() ' 7r; markupr$ user_specified_encodingOptional[_Encoding]document_declared_encodingexclude_encodingsOptional[_Encodings]r= DIterable[Tuple[str, Optional[_Encoding], Optional[_Encoding], bool]]c # 8 K t |t r
|dddfV dS g }|r| | g }|r| | t |||d| }|j t d |j |j |j |j fV dS )a2 Run any preliminary steps necessary to make incoming markup
acceptable to the parser.
:param markup: Some markup -- probably a bytestring.
:param user_specified_encoding: The user asked to try this encoding.
:param document_declared_encoding: The markup itself claims to be
in this encoding.
:param exclude_encodings: The user asked _not_ to try any of
these encodings.
:yield: A series of 4-tuples: (markup, encoding, declared encoding,
has undergone character replacement)
Each 4-tuple represents a strategy for parsing the document.
This TreeBuilder uses Unicode, Dammit to convert the markup
into Unicode, so the ``markup`` element of the tuple will
always be a string.
NFT)known_definite_encodingsuser_encodingsis_htmlr zPCould not convert input to Unicode, and html.parser will not accept bytestrings.)
isinstancer( rT r unicode_markupr original_encodingdeclared_html_encodingr| )r9 r r r r r r dammits r: prepare_markupz$HTMLParserTreeBuilder.prepare_markup s 2 fc " " 4u- - - - F 57 " E
% + +,C D D D*, % >
! !"< = = = %=) /
( 'b
%(-6
r;
_parser_class type[BeautifulSoupHTMLParser]r> c " | j \ }}t |t sJ | j J || j g|R i |} | | | n!# t $ r}t | d}~ww xY wg |_ dS )z
:param markup: The markup to feed into the parser.
:param _parser_class: An HTMLParser subclass to use. This is only intended for use in unit tests.
N) r r r( r. feedcloseAssertionErrorr r7 )r9 r r r/ r1 parseres r: r zHTMLParserTreeBuilder.feed s
' f &#&&&&&
y$$$ ty:4:::6:: * KK LLNNNN * * * 'q)) ) *
/1+++s )A' '
B1B B)NN)r r r r r1 r )NNN)
r r$ r r r r r r r= r )r r$ r r r= r> )r r r __doc__r r r
HTMLPARSERr r r r r r6 r r& r
__classcell__)r s @r: r r q s FID #T6 2H22225555 !%$$$$ 0426 8 8 8 8 8 8 8B 8<:>26 F F F F FP Ul 1 1 1 1 1 1 1 1 1r; )1r
__future__r __license____all__ html.parserr r typingr r r r
r r r
r r r r bs4.elementr r r r r r
bs4.dammitr r bs4.builderr r r r bs4.exceptionsr bs4r r! bs4._typingr" r# r$ r r( rP r& r r r; r: r s I I " " " " " " # " " " " " 9 8 8 8 8 8 8 8 0 / / / / / !!!!!!++++++
%tCH~sC&@$&F G p1 p1 p1 p1 p1j*@ p1 p1 p1f T1 T1 T1 T1 T1O T1 T1 T1 T1 T1r;