D7net
Home
Console
Upload
information
Create File
Create Folder
About
Tools
:
/
opt
/
alt
/
python37
/
lib
/
python3.7
/
site-packages
/
charset_normalizer
/
__pycache__
/
Filename :
models.cpython-37.pyc
back
Copy
B �B�hj0 � @ s� d dl mZ d dlmZ d dlmZ d dlmZ d dlm Z d dl m Z m Z m Z mZ ddlmZmZ dd lmZmZmZ G d d � d �ZG d d � d �Zeeef Ze e ZG dd� d�Z dS )� )� annotations)�aliases)�sha256)�dumps)�sub)�Any�Iterator�List�Tuple� )� RE_POSSIBLE_ENCODING_INDICATION�TOO_BIG_SEQUENCE)� iana_name�is_multi_byte_encoding� unicode_rangec @ s� e Zd ZdAdddddddd�d d �Zd dd �d d�Zd dd �dd�Zedd�dd��Zdd�dd�Zdd�dd�Z d dd �dd�Z edd�dd ��Z ed d�d d ��Z edd�d d!��Z edd�d"d#��Zed d�d$d%��Zedd�d&d'��Zedd�d(d)��Zedd�d*d+��Zedd�d,d-��Zedd�d.d/��Zedd�d0d1��Zed2d�d3d4��Zedd�d5d6��Zed d�d7d8��Zed d�d9d:��ZdBddd<�d=d>�Zedd�d?d@��ZdS )C� CharsetMatchN�bytes�str�float�bool�CoherenceMatchesz str | None)�payload�guessed_encoding�mean_mess_ratio�has_sig_or_bom� languages�decoded_payload�preemptive_declarationc C sL || _ || _|| _|| _|| _d | _g | _d| _d | _d | _ || _ || _ d S )Ng ) �_payload� _encoding�_mean_mess_ratio� _languages�_has_sig_or_bom�_unicode_ranges�_leavesZ_mean_coherence_ratio�_output_payload�_output_encoding�_string�_preemptive_declaration)�selfr r r r r r r � r* �J/opt/alt/python37/lib/python3.7/site-packages/charset_normalizer/models.py�__init__ s zCharsetMatch.__init__�object)�other�returnc C s> t |t�s&t |t�r"t|�| jkS dS | j|jko<| j|jkS )NF)� isinstancer r r �encoding� fingerprint)r) r. r* r* r+ �__eq__* s zCharsetMatch.__eq__c C s� t |t�st�t| j|j �}t| j|j �}|dk rJ|dkrJ| j|jkS |dk r�|dkr�t| j�tkrt| j|jk S | j |j kS | j|jk S )zQ Implemented to make sorted available upon CharsetMatches items. g{�G�z�?g{�G�z�?) r0 r � ValueError�abs�chaos� coherence�lenr r �multi_byte_usage)r) r. Zchaos_differenceZcoherence_differencer* r* r+ �__lt__1 s zCharsetMatch.__lt__)r/ c C s dt t| ��t | j� S )Ng �?)r8 r �raw)r) r* r* r+ r9 G s z CharsetMatch.multi_byte_usagec C s" | j d kr t| j| jd�| _ | j S )N�strict)r' r r r )r) r* r* r+ �__str__K s zCharsetMatch.__str__c C s d| j � d| j� d�S )Nz<CharsetMatch 'z' bytes(z)>)r1 r2 )r) r* r* r+ �__repr__Q s zCharsetMatch.__repr__�Nonec C s8 t |t�r|| kr"td�|j���d |_| j�|� d S )Nz;Unable to add instance <{}> as a submatch of a CharsetMatch)r0 r r4 �format� __class__r' r$ �append)r) r. r* r* r+ � add_submatchT s zCharsetMatch.add_submatchc C s | j S )N)r )r) r* r* r+ r1 _ s zCharsetMatch.encodingz list[str]c C sH g }x>t �� D ]2\}}| j|kr,|�|� q| j|kr|�|� qW |S )z� Encoding name are known by many name, using this could help when searching for IBM855 when it's listed as CP855. )r �itemsr1 rB )r) Z also_known_as�u�pr* r* r+ �encoding_aliasesc s z CharsetMatch.encoding_aliasesc C s | j S )N)r" )r) r* r* r+ �bomp s zCharsetMatch.bomc C s | j S )N)r" )r) r* r* r+ �byte_order_markt s z CharsetMatch.byte_order_markc C s dd� | j D �S )z� Return the complete list of possible languages found in decoded sequence. Usually not really useful. Returned list may be empty even if 'language' property return something != 'Unknown'. c S s g | ] }|d �qS )r r* )�.0�er* r* r+ � <listcomp>~ s z*CharsetMatch.languages.<locals>.<listcomp>)r! )r) r* r* r+ r x s zCharsetMatch.languagesc C sp | j sbd| jkrdS ddlm}m} t| j�r8|| j�n|| j�}t|�dksVd|krZdS |d S | j d d S )z� Most probable language found in decoded sequence. If none were detected or inferred, the property will return "Unknown". �asciiZEnglishr )�encoding_languages�mb_encoding_languagesz Latin Based�Unknown)r! �could_be_from_charsetZcharset_normalizer.cdrN rO r r1 r8 )r) rN rO r r* r* r+ �language� s zCharsetMatch.languagec C s | j S )N)r )r) r* r* r+ r6 � s zCharsetMatch.chaosc C s | j s dS | j d d S )Ng r r )r! )r) r* r* r+ r7 � s zCharsetMatch.coherencec C s t | jd dd�S )N�d � )�ndigits)�roundr6 )r) r* r* r+ � percent_chaos� s zCharsetMatch.percent_chaosc C s t | jd dd�S )NrS rT )rU )rV r7 )r) r* r* r+ �percent_coherence� s z CharsetMatch.percent_coherencec C s | j S )z+ Original untouched bytes. )r )r) r* r* r+ r; � s zCharsetMatch.rawzlist[CharsetMatch]c C s | j S )N)r$ )r) r* r* r+ �submatch� s zCharsetMatch.submatchc C s t | j�dkS )Nr )r8 r$ )r) r* r* r+ � has_submatch� s zCharsetMatch.has_submatchc C s@ | j d k r| j S dd� t| �D �}ttdd� |D ���| _ | j S )Nc S s g | ] }t |��qS r* )r )rJ �charr* r* r+ rL � s z*CharsetMatch.alphabets.<locals>.<listcomp>c S s h | ] }|r|�qS r* r* )rJ �rr* r* r+ � <setcomp>� s z)CharsetMatch.alphabets.<locals>.<setcomp>)r# r �sorted�list)r) Zdetected_rangesr* r* r+ � alphabets� s zCharsetMatch.alphabetsc C s | j gdd� | jD � S )z� The complete list of encoding that output the exact SAME str result and therefore could be the originating encoding. This list does include the encoding available in property 'encoding'. c S s g | ] }|j �qS r* )r1 )rJ �mr* r* r+ rL � s z6CharsetMatch.could_be_from_charset.<locals>.<listcomp>)r r$ )r) r* r* r+ rQ � s z"CharsetMatch.could_be_from_charset�utf_8)r1 r/ c s~ � j dks� j |krx|� _ t� �}� jdk rj� j�� dkrjtt� fdd�|dd� dd�}||dd� }|�|d�� _� jS ) z� Method to get re-encoded bytes payload using given target encoding. Default to UTF-8. Any errors will be simply ignored by the encoder NOT replaced. N)zutf-8�utf8rb c s< | j | �� d | �� d � �| �� d t� j��dd��S )Nr r �_�-)�string�span�replace�groupsr r& )ra )r) r* r+ �<lambda>� s z%CharsetMatch.output.<locals>.<lambda>i r )�countrh )r& r r( �lowerr r �encoder% )r) r1 �decoded_stringZpatched_headerr* )r) r+ �output� s zCharsetMatch.outputc C s t | �� ��� S )zw Retrieve the unique SHA256 computed using the transformed (re-encoded) payload. Not the original one. )r ro � hexdigest)r) r* r* r+ r2 � s zCharsetMatch.fingerprint)NN)rb ) �__name__� __module__� __qualname__r, r3 r: �propertyr9 r= r> rC r1 rG rH rI r rR r6 r7 rW rX r; rY rZ r` rQ ro r2 r* r* r* r+ r s2 r c @ s� e Zd ZdZd dd�dd�Zdd�d d �Zd d d �dd�Zdd�dd�Zdd�dd�Zd dd �dd�Z dd�dd�Z dd�d d �Z dS ) �CharsetMatchesz� Container with every CharsetMatch items ordered by default from most probable to the less one. Act like a list(iterable) but does not implements all related methods. Nzlist[CharsetMatch] | None)�resultsc C s |r t |�ng | _d S )N)r^ �_results)r) rv r* r* r+ r, � s zCharsetMatches.__init__zIterator[CharsetMatch])r/ c c s | j E d H d S )N)rw )r) r* r* r+ �__iter__� s zCharsetMatches.__iter__z int | strr )�itemr/ c C sN t |t�r| j| S t |t�rFt|d�}x | jD ]}||jkr0|S q0W t�dS )z� Retrieve a single item either by its position or encoding name (alias may be used here). Raise KeyError upon invalid index or encoding not present in results. FN)r0 �intrw r r rQ �KeyError)r) ry �resultr* r* r+ � __getitem__ s zCharsetMatches.__getitem__rz c C s t | j�S )N)r8 rw )r) r* r* r+ �__len__ s zCharsetMatches.__len__r c C s t | j�dkS )Nr )r8 rw )r) r* r* r+ �__bool__ s zCharsetMatches.__bool__r? c C s~ t |t�s td�t|j����t|j�tk rbx4| j D ]*}|j |j kr4|j |j kr4|� |� dS q4W | j � |� t| j �| _ dS )z~ Insert a single match. Will be inserted accordingly to preserve sort. Can be inserted as a submatch. z-Cannot append instance '{}' to CharsetMatchesN)r0 r r4 r@ r rA r8 r; r rw r2 r6 rC rB r^ )r) ry �matchr* r* r+ rB s zCharsetMatches.appendzCharsetMatch | Nonec C s | j s dS | j d S )zQ Simply return the first match. Strict equivalent to matches[0]. Nr )rw )r) r* r* r+ �best) s zCharsetMatches.bestc C s | � � S )zP Redundant method, call the method best(). Kept for BC reasons. )r� )r) r* r* r+ �first1 s zCharsetMatches.first)N) rq rr rs �__doc__r, rx r} r~ r rB r� r� r* r* r* r+ ru � s ru c @ sN e Zd Zdddddddddddd� dd�Zed d �d d ��Zdd �d d�ZdS )�CliDetectionResultr z str | Nonez list[str]r r ) �pathr1 rG �alternative_encodingsrR r` r r6 r7 � unicode_path� is_preferredc C sF || _ | | _|| _|| _|| _|| _|| _|| _|| _| | _ | | _ d S )N) r� r� r1 rG r� rR r` r r6 r7 r� ) r) r� r1 rG r� rR r` r r6 r7 r� r� r* r* r+ r, = s zCliDetectionResult.__init__zdict[str, Any])r/ c C s2 | j | j| j| j| j| j| j| j| j| j | j d� S )N) r� r1 rG r� rR r` r r6 r7 r� r� ) r� r1 rG r� rR r` r r6 r7 r� r� )r) r* r* r+ �__dict__W s zCliDetectionResult.__dict__c C s t | jddd�S )NT� )� ensure_ascii�indent)r r� )r) r* r* r+ �to_jsong s zCliDetectionResult.to_jsonN)rq rr rs r, rt r� r� r* r* r* r+ r� < s "r� N) � __future__r Zencodings.aliasesr �hashlibr �jsonr �rer �typingr r r r Zconstantr r �utilsr r r r ru r r ZCoherenceMatchr r� r* r* r* r+ �<module> s iC