Current Dir: /snap/core22/2437/usr/lib/python3/dist-packages/chardet/__pycache__/

[FILE] big5freq.cpython-310.pyc [ edit | delete | rename ]
[FILE] big5prober.cpython-310.pyc [ edit | delete | rename ]
[FILE] chardistribution.cpython-310.pyc [ edit | delete | rename ]
[FILE] charsetgroupprober.cpython-310.pyc [ edit | delete | rename ]
[FILE] charsetprober.cpython-310.pyc [ edit | delete | rename ]
[FILE] codingstatemachine.cpython-310.pyc [ edit | delete | rename ]
[FILE] compat.cpython-310.pyc [ edit | delete | rename ]
[FILE] cp949prober.cpython-310.pyc [ edit | delete | rename ]
[FILE] enums.cpython-310.pyc [ edit | delete | rename ]
[FILE] escprober.cpython-310.pyc [ edit | delete | rename ]
[FILE] escsm.cpython-310.pyc [ edit | delete | rename ]
[FILE] eucjpprober.cpython-310.pyc [ edit | delete | rename ]
[FILE] euckrfreq.cpython-310.pyc [ edit | delete | rename ]
[FILE] euckrprober.cpython-310.pyc [ edit | delete | rename ]
[FILE] euctwfreq.cpython-310.pyc [ edit | delete | rename ]
[FILE] euctwprober.cpython-310.pyc [ edit | delete | rename ]
[FILE] gb2312freq.cpython-310.pyc [ edit | delete | rename ]
[FILE] gb2312prober.cpython-310.pyc [ edit | delete | rename ]
[FILE] hebrewprober.cpython-310.pyc [ edit | delete | rename ]
[FILE] jisfreq.cpython-310.pyc [ edit | delete | rename ]
[FILE] jpcntx.cpython-310.pyc [ edit | delete | rename ]
[FILE] langbulgarianmodel.cpython-310.pyc [ edit | delete | rename ]
[FILE] langgreekmodel.cpython-310.pyc [ edit | delete | rename ]
[FILE] langhebrewmodel.cpython-310.pyc [ edit | delete | rename ]
[FILE] langhungarianmodel.cpython-310.pyc [ edit | delete | rename ]
[FILE] langrussianmodel.cpython-310.pyc [ edit | delete | rename ]
[FILE] langthaimodel.cpython-310.pyc [ edit | delete | rename ]
[FILE] langturkishmodel.cpython-310.pyc [ edit | delete | rename ]
[FILE] latin1prober.cpython-310.pyc [ edit | delete | rename ]
[FILE] mbcharsetprober.cpython-310.pyc [ edit | delete | rename ]
[FILE] mbcsgroupprober.cpython-310.pyc [ edit | delete | rename ]
[FILE] mbcssm.cpython-310.pyc [ edit | delete | rename ]
[FILE] sbcharsetprober.cpython-310.pyc [ edit | delete | rename ]
[FILE] sbcsgroupprober.cpython-310.pyc [ edit | delete | rename ]
[FILE] sjisprober.cpython-310.pyc [ edit | delete | rename ]
[FILE] universaldetector.cpython-310.pyc [ edit | delete | rename ]
[FILE] utf8prober.cpython-310.pyc [ edit | delete | rename ]
[FILE] version.cpython-310.pyc [ edit | delete | rename ]
[FILE] __init__.cpython-310.pyc [ edit | delete | rename ]

Viewing: /snap/core22/2437/usr/lib/python3/dist-packages/chardet/__pycache__/charsetprober.cpython-310.pyc

o

-��_��@s0dd
lZdd
l
Z
ddlmZ
Gdd�de�Zd
S)�N�
)
�ProbingStatec@sneZ
dZd
Zddd�
Zdd�Zedd��
Zd	d
�Zedd��
Z	d
d�Z
edd��
Zedd��
Z
edd��
ZdS)�
CharSetProbergffffff�?NcCsd|_|
|_
t�t�
|_dS�
N)�_state�lang_filter�logging�	getLogger�__name__�logger)�selfr�r
�7/usr/lib/python3/dist-packages/chardet/charsetprober.py�__init__'s


zCharSetProber.__init__c

Cstj
|_dSr)r�	DETECTINGr�
rr
r
r�reset,s
zCharSetProber.resetc


C�dSrr
rr
r
r�charset_name/szCharSetProber.charset_namec
Crrr
)r�bufr
r
r�feed3�
zCharSetProber.feedc


Cs|jSr)
rrr
r
r�state6szCharSetProber.statec


Csd
S)Ngr
rr
r
r�get_confidence:rzCharSetProber.get_confidencec

Cst�
d
d|�}|S)Ns([-])+�
 )�re�sub)
rr
r
r�filter_high_byte_only=s
z#CharSetProber.filter_high_byte_onlyc
Cs\t�}
t
�d
|�}|D] }|
�|dd��

|dd�}|��s&|dkr&d}|
�|�

q|
S)u9
        We define three types of bytes:
        alphabet: english alphabets [a-zA-Z]
        international: international characters [€-ÿ]
        marker: everything else [^a-zA-Z€-ÿ]

        The input buffer can be thought to contain a series of words delimited
        by markers. This function works to filter all words that contain at
        least one international character. All contiguous sequences of markers
        are replaced by a single space ascii character.

        This filter applies to all scripts which do not use English characters.
        s%[a-zA-Z]*[�-�]+[a-zA-Z]*[^a-zA-Z�-�]?N����
�r)�	bytearrayr�findall�extend�isalpha)r�filtered�words�word�	last_charr
r
r�filter_international_wordsBs
�



z(CharSetProber.filter_international_wordsc
Cs�t�}
d
}d}t
t|�
�
D]7}|||d�}|dkrd
}n|dkr$d}|dkrD|��sD||kr@|s@|
�|||��

|
�d�

|d}q
|sP|
�||d	��

|
S)
a�

        Returns a copy of ``buf`` that retains only the sequences of English
        alphabet and high byte characters that are not between <> characters.
        Also retains English alphabet and high byte characters immediately
        before occurrences of >.

        This filter can be applied to all scripts which contain both English
        characters and extended ASCII characters, but is currently only used by
        ``Latin1Prober``.
        Fr
r�
>�
<TrrN)r �range�lenr#r")rr$�in_tag�prev�curr�buf_charr
r
r�filter_with_english_lettersgs$






�z)CharSetProber.filter_with_english_lettersr)r
�
__module__�__qualname__�SHORTCUT_THRESHOLDrr�propertyrrrr�staticmethodrr(r1r
r
r
rr#s 








$
r)rr�enumsr�objectrr
r
r
r�<module>
s


Upload File: 



    CMD: