Ug{,ddlmZddlmZddlmZddlZddlm Z dddd Z dd Z Gd d eZ d Z GddeZdZdZdddZddZGddeZdZdS))Counter)suppress) NamedTupleN is_scalar_nanFreturn_inverse return_countscj|jtkrt|||St|||S)aHelper function to find unique values with support for python objects. Uses pure python method for object dtype, and numpy method for all other dtypes. Parameters ---------- values : ndarray Values to check for unknowns. return_inverse : bool, default=False If True, also return the indices of the unique values. return_counts : bool, default=False If True, also return the number of times each unique item appears in values. Returns ------- unique : ndarray The sorted unique values. unique_inverse : ndarray The indices to reconstruct the original array from the unique array. Only provided if `return_inverse` is True. unique_counts : ndarray The number of times each of the unique values comes up in the original array. Only provided if `return_counts` is True. r )dtypeobject_unique_python _unique_np)valuesr r s T/var/www/surfInsights/venv3-11/lib/python3.11/site-packages/sklearn/utils/_encode.py_uniquer sO>|v >     ~]   ctj|||}d\}}|r|^}}|r|^}}|s|r|d}|jrzt|dretj|tj}|d|dz}|r ||||k<|r,tj||d||<|d|dz}|f}|r||fz }|r||fz }t|dkr|dn|S)zHelper function to find unique values for numpy arrays that correctly accounts for nans. See `_unique` documentation for details.r )NNrNr)npuniquesizer searchsortednansumlen)rr r uniquesinversecountsnan_idxrets rrr3sDi~]G!OGV#"&$#'!*|+ gbk22+/'2622-GaK-(  1)0GGg% &  + fVGHH%566F7OMgkM*F *C z yXX]]3q66+rc.eZdZUdZeed<eed<dZdS) MissingValuesz'Data class for missing data informationrnonecg}|jr|d|jr|tj|S)z3Convert tuple to a list where None is always first.N)r%appendrr)selfoutputs rto_listzMissingValues.to_listbsF 9 MM$    8 " MM"& ! ! ! rN)__name__ __module__ __qualname____doc__bool__annotations__r*rrr$r$\s=11 III JJJrr$cd|D}|s|tddfSd|vr7t|dkrtdd}n#tdd}ntdd}||z }||fS)a.Extract missing values from `values`. Parameters ---------- values: set Set of values to extract missing from. Returns ------- output: set Set with missing values extracted. missing_values: MissingValues Object with missing value information. c4h|]}|t||SNr).0values r z#_extract_missing..|s,U]mE6J6J]]]]rF)rr%NrT)r$r)rmissing_values_setoutput_missing_valuesr)s r_extract_missingr:ls ! <}U;;;;; !!! ! " "a ' '$1e$$G$G$G ! !%2d$F$F$F ! ! -$U C C C( (F ( ((rc(eZdZdZfdZdZxZS)_nandictz!Dictionary with support for nans.ct||D]\}}t|r ||_dSdSr4)super__init__itemsr nan_value)r(mappingkeyr6 __class__s rr?z_nandict.__init__sc !!!!--//  JCS!! !&   rclt|drt|r|jSt|)NrA)hasattrrrAKeyErrorr(rCs r __missing__z_nandict.__missing__7 4 % % "-*<*< "> !smmr)r+r,r-r.r?rI __classcell__rDs@rr<r<sM++rr<ctdt|Dtjfd|DS)z,Map values based on its position in uniques.ci|]\}}|| Sr1r1)r5ivals r z#_map_to_integer..s>>>Cc1>>>rc g|] }| Sr1r1)r5vtables r z#_map_to_integer..s...!U1X...r)r< enumeraterarray)rrrTs @r_map_to_integerrXsL >>9W+=+=>>> ? ?E 8....v... / //rc" t|}t|\}}t|}||t j||j}nP#t$rCtdtd|DD}td|wxYw|f}|r|t||fz }|r|t||fz }t|dkr|dn|S)Nr c3$K|] }|jV dSr4)r-)r5ts r z!_unique_python..s$LL!q~LLLLLLrc34K|]}t|VdSr4)type)r5rSs rr]z!_unique_python..s(2K2Kq4772K2K2K2K2K2KrzPEncoders require their input argument must be uniformly strings or numbers. Got rr) setr:sortedextendr*rrWr TypeErrorrX _get_countsr)rr r uniques_setmissing_valuesrtypesr"s rrrs<  &kk &6{&C&C# ^%%~--//000(7&,777    LLs2K2KF2K2K2K/K/KLLLLL /', / /    *C3 0022/ FG,,..XX]]3q66+s A2A55A CT) check_unknownc2|jjdvrB t||S#t$r$}t dt |d}~wwxYw|r1t ||}|rt dt |tj||S)aHelper function to encode values into [0, n_uniques - 1]. Uses pure python method for object dtype, and numpy method for all other dtypes. The numpy method has the limitation that the `uniques` need to be sorted. Importantly, this is not checked but assumed to already be the case. The calling method needs to ensure this for all non-object values. Parameters ---------- values : ndarray Values to encode. uniques : ndarray The unique values in `values`. If the dtype is not object, then `uniques` needs to be sorted. check_unknown : bool, default=True If True, check for values in `values` that are not in `unique` and raise an error. This is ignored for object dtype, and treated as True in this case. This parameter is useful for _BaseEncoder._transform() to avoid calling _check_unknown() twice. Returns ------- encoded : ndarray Encoded values OUSz%y contains previously unseen labels: N) r kindrXrG ValueErrorstr_check_unknownrr)rrrhediffs r_encoderqs:|E!! O"6733 3 O O OMSVVMMNN N O  V!&'22D V !TT!T!TUUUw///s AA  Ac6 d}|jjdvrt|}t|\}}t|t\ |z }|jo j }|jo j } fd |rO|s|s|r!t j fd|D}n(t jt|t}t|}|r| d|r| t jnt j |} t j| |d}|rE|jrt j||}n(t jt|t}t j|rSt j|} | r+|jr|rt j|} d|| <|| }t|}|r||fS|S) a Helper function to check for unknowns in values to be encoded. Uses pure python method for object dtype, and numpy method for all other dtypes. Parameters ---------- values : array Values to check for unknowns. known_values : array Known values. Must be unique. return_mask : bool, default=False If True, return a mask of the same shape as `values` indicating the valid values. Returns ------- diff : list The unique values present in `values` and not in `know_values`. valid_mask : boolean array Additionally returned if ``return_mask=True``. NrjcN|vp jo|dupjot|Sr4)r%rr)r6missing_in_uniquesres ris_validz _check_unknown..is_validsD$)%*"TM)&))!%((  rc&g|] }|Sr1r1)r5r6rus rrUz"_check_unknown..s!&K&K&K5xx&K&K&KrrZT assume_uniquer)r rkr`r:rr%rrWonesrr/listr'r setdiff1drisinisnanany)r known_values return_mask valid_mask values_setmissing_in_valuesrp nan_in_diff none_in_diff unique_values diff_is_nanis_nanrurtres @@@rrnrnsJ2J |E!![[ (8(D(D% %,'' *:;*G*G' 'K''+J4F4J0J (-M6H6M2M         > >{ >l >X&K&K&K&KF&K&K&KLL WS[[=== Dzz   KK     KK    &)) |My >WV\:: WS[[===  8L ! ! % % ' ' *(4..K   *9++Xf--F)*Jv&[L)Dzz Z Krc.eZdZdZfdZdZdZxZS) _NaNCounterz$Counter with support for nan values.cpt||dSr4)r>r?_generate_items)r(r@rDs rr?z_NaNCounter.__init__Ds/ --e4455555rc#K|D]=}t|s|Vt|dsd|_|xjdz c_>dS)z>Generate items without nans. Stores the nan counts separately. nan_countrrN)rrFr)r(r@items rrz_NaNCounter._generate_itemsGsf  D &&  4-- #!" NNa NNN   rclt|drt|r|jSt|)Nr)rFrrrGrHs rrIz_NaNCounter.__missing__QrJr)r+r,r-r.r?rrIrKrLs@rrrAs\..66666   rrc|jjdvrt|}tjt |tj}t|D]<\}}tt5||||<dddn #1swxYwY=|St|d\}}tj ||d}tj |drtj |drd|d<tj |||} tj|tj}|| ||<|S)zGet the count of each of the `uniques` in `values`. The counts will use the order passed in by `uniques`. For non-object dtypes, `uniques` is assumed to be sorted and `np.nan` is at the end. OUrZNT)r rwr)r rkrrzerosrint64rVrrGrr|r}r zeros_like) rrcounterr)rOrrr uniques_in_valuesunique_valid_indicess rrdrdWsy |D  f%%#g,,bh777 )) * *GAt(## * *#DMq  * * * * * * * * * * * * * * * &vTBBBM6dKKK x b!""%rx '<'<% $"?='BS:TUU ]7"( 3 3 3F &'; rs######',5&&&&&R&,&,&,&,R     J    #)#)#)L     t    000 ,,,4/3'0'0'0'0'0TRRRRj',r