UgRddlmZddlmZmZddlZddlmZm Z m Z m Z m Z ddl mZddlmZmZddlmZdd lmZdd lmZmZmZmZdd lmZdd lmZmZm Z m!Z!dd l"m#Z#ddl$m%Z%ddl&m'Z'm(Z(ddl)m*Z*m+Z+m,Z,m-Z-ddl.m/Z/m0Z0dZ1dZ2Gdde e eZ3Gdde3Z4GddeZ5dZ6dZ7Gdde3Z8dS))MutableMapping)IntegralRealN) BaseEstimatorClassifierMixinMetaEstimatorMixin _fit_contextclone)NotFittedError) check_scoringget_scorer_names) _BaseScorer)_safe_indexing) HasMethodsInterval RealNotInt StrOptions)_get_response_values_binary)MetadataRouter MethodMapping_raise_for_paramsprocess_routing) available_if)type_of_target)Paralleldelayed)_check_method_params _num_samplescheck_is_fitted indexable)StratifiedShuffleSplitcheck_cvcfd}|S)zCheck if we can delegate a method to the underlying estimator. First, we check the fitted estimator if available, otherwise we check the unfitted estimator. c~t|drt|jnt|jdS)N estimator_T)hasattrgetattrr' estimator)selfattrs p/var/www/surfInsights/venv3-11/lib/python3.11/site-packages/sklearn/model_selection/_classification_threshold.pycheckz_estimator_has..check/sA 4 & & * DOT * * * * DND ) ) )t)r,r.s` r-_estimator_hasr1(s# Lr/c,|tjddg}nRtj||kd}tj||kd}tj||g}||||ktS)z;Threshold `y_score` and return the associated class labels.Nrr")nparray flatnonzeroastypeint)y_score thresholdclasses pos_labelmap_thresholded_score_to_label pos_label_idx neg_label_idxs r-!_threshold_scores_to_class_labelsr?9s)+1a&)9)9&&w)';<autorCrDr*response_method_parameter_constraintsrErGc"||_||_dSNrF)r+r*rGs r-__init__z BaseThresholdClassifier.__init__is".r/c4|jdkrddg}n|j}|S)zDefine the response method.rErCrDrI)r+rGs r-_get_response_methodz,BaseThresholdClassifier._get_response_methodms+  6 ) ).0CDOO"2Or/F)prefer_skip_nested_validationc Vt||dt||\}}t|d}|dkrtd||j||fi|t |jdr|jj|_t |jdr|jj|_|S)Fit the classifier. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) Training data. y : array-like of shape (n_samples,) Target values. **params : dict Parameters to pass to the `fit` method of the underlying classifier. Returns ------- self : object Returns an instance of self. Ny) input_namebinaryz=Only binary classification is supported. Unknown label type: n_features_in_feature_names_in_) rr!r ValueError_fitr(r'rUrV)r+XrRparamsy_types r-rBzBaseThresholdClassifier.fitus0 &$---A1c222 X  XPVXX   !Q!!&!!! 4?$4 5 5 A"&/"@D  4?$7 8 8 G%)_%FD " r/c|jjS)zClasses labels.)r'classes_r+s r-r]z BaseThresholdClassifier.classes_s''r/cVt|d|j|S)aPredict class probabilities for `X` using the fitted estimator. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) Training vectors, where `n_samples` is the number of samples and `n_features` is the number of features. Returns ------- probabilities : ndarray of shape (n_samples, n_classes) The class probabilities of the input samples. r')r r'rCr+rYs r-rCz%BaseThresholdClassifier.predict_probas* l+++,,Q///r/predict_log_probacVt|d|j|S)aPredict logarithm class probabilities for `X` using the fitted estimator. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) Training vectors, where `n_samples` is the number of samples and `n_features` is the number of features. Returns ------- log_probabilities : ndarray of shape (n_samples, n_classes) The logarithm class probabilities of the input samples. r')r r'rar`s r-raz)BaseThresholdClassifier.predict_log_proba* l+++00333r/cVt|d|j|S)aDecision function for samples in `X` using the fitted estimator. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) Training vectors, where `n_samples` is the number of samples and `n_features` is the number of features. Returns ------- decisions : ndarray of shape (n_samples,) The decision function computed the fitted estimator. r')r r'rDr`s r-rDz)BaseThresholdClassifier.decision_functionrcr/cdddddS)NTz*Threshold at probability 0.5 does not holdzDue to the cross-validation and sample ordering, removing a sample is not strictly equal to putting is weight to zero. Specific unit tests are added for TunedThresholdClassifierCV specifically.)check_classifiers_traincheck_sample_weights_invariance) binary_only _xfail_checksr0r^s r- _more_tagsz"BaseThresholdClassifier._more_tagss'+WT   r/N)__name__ __module__ __qualname____doc___required_parametersrrrHdict__annotations__rLrNr rBpropertyr]rr1rCrarDrjr0r/r-rArAEs4(= J/ 0 0 J23 4 4 'J'U'U'UVVW $$D6</////\&+%% %N((X(\..112200320"\..!4556644764"\..!4556644764"      r/rAceZdZUdZiejedhegeeddgdZe e d<ddddfd Z d Z d Z d ZxZS) FixedThresholdClassifieraBinary classifier that manually sets the decision threshold. This classifier allows to change the default decision threshold used for converting posterior probability estimates (i.e. output of `predict_proba`) or decision scores (i.e. output of `decision_function`) into a class label. Here, the threshold is not optimized and is set to a constant value. Read more in the :ref:`User Guide `. .. versionadded:: 1.5 Parameters ---------- estimator : estimator instance The binary classifier, fitted or not, for which we want to optimize the decision threshold used during `predict`. threshold : {"auto"} or float, default="auto" The decision threshold to use when converting posterior probability estimates (i.e. output of `predict_proba`) or decision scores (i.e. output of `decision_function`) into a class label. When `"auto"`, the threshold is set to 0.5 if `predict_proba` is used as `response_method`, otherwise it is set to 0 (i.e. the default threshold for `decision_function`). pos_label : int, float, bool or str, default=None The label of the positive class. Used to process the output of the `response_method` method. When `pos_label=None`, if `y_true` is in `{-1, 1}` or `{0, 1}`, `pos_label` is set to 1, otherwise an error will be raised. response_method : {"auto", "decision_function", "predict_proba"}, default="auto" Methods by the classifier `estimator` corresponding to the decision function for which we want to find a threshold. It can be: * if `"auto"`, it will try to invoke `"predict_proba"` or `"decision_function"` in that order. * otherwise, one of `"predict_proba"` or `"decision_function"`. If the method is not implemented by the classifier, it will raise an error. Attributes ---------- estimator_ : estimator instance The fitted classifier used when predicting. classes_ : ndarray of shape (n_classes,) The class labels. n_features_in_ : int Number of features seen during :term:`fit`. Only defined if the underlying estimator exposes such an attribute when fit. feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Only defined if the underlying estimator exposes such an attribute when fit. See Also -------- sklearn.model_selection.TunedThresholdClassifierCV : Classifier that post-tunes the decision threshold based on some metrics and using cross-validation. sklearn.calibration.CalibratedClassifierCV : Estimator that calibrates probabilities. Examples -------- >>> from sklearn.datasets import make_classification >>> from sklearn.linear_model import LogisticRegression >>> from sklearn.metrics import confusion_matrix >>> from sklearn.model_selection import FixedThresholdClassifier, train_test_split >>> X, y = make_classification( ... n_samples=1_000, weights=[0.9, 0.1], class_sep=0.8, random_state=42 ... ) >>> X_train, X_test, y_train, y_test = train_test_split( ... X, y, stratify=y, random_state=42 ... ) >>> classifier = LogisticRegression(random_state=0).fit(X_train, y_train) >>> print(confusion_matrix(y_test, classifier.predict(X_test))) [[217 7] [ 19 7]] >>> classifier_other_threshold = FixedThresholdClassifier( ... classifier, threshold=0.1, response_method="predict_proba" ... ).fit(X_train, y_train) >>> print(confusion_matrix(y_test, classifier_other_threshold.predict(X_test))) [[184 40] [ 6 20]] rEbooleanN)r9r;rH)r9r;rGcjt||||_||_dSNrF)superrLr;r9)r+r*r9r;rG __class__s r-rLz!FixedThresholdClassifier.__init__Gs4 9oNNN""r/c t|dfi|}t|jj||fi|jj|_|S)rQrB)rr r*rBr')r+rYrRrZ routed_paramss r-rXzFixedThresholdClassifier._fitSsN((e>>v>> 3%//3AqXXMV # #(<(O(OUX  !% 0 '   r/ct|jj|jt dd}|S)KGet metadata routing of this object. Please check :ref:`User Guide ` on how the routing mechanism works. Returns ------- routing : MetadataRouter A :class:`~sklearn.utils.metadata_routing.MetadataRouter` encapsulating routing information. ownerrBcalleecallerr*method_mapping)rryrkaddr*rr+routers r-get_metadata_routingz-FixedThresholdClassifier.get_metadata_routingsU dn&=>>>BBn(??..eE.JJC   r/)rkrlrmrnrArHrrstrrprqrLrXrr __classcell__rys@r-rtrtsUUn$ ! 8$ j&**D1CD1$$$D # # # # # # #0   >r/rtc>eZdZdZfdZedZdZxZS) _CurveScoreraScorer taking a continuous response and output a score for each threshold. Parameters ---------- score_func : callable The score function to use. It will be called as `score_func(y_true, y_pred, **kwargs)`. sign : int Either 1 or -1 to returns the score with `sign * score_func(estimator, X, y)`. Thus, `sign` defined if higher scores are better or worse. kwargs : dict Additional parameters to pass to the score function. thresholds : int or array-like Related to the number of decision thresholds for which we want to compute the score. If an integer, it will be used to generate `thresholds` thresholds uniformly distributed between the minimum and maximum predicted scores. If an array-like, it will be used as the thresholds. response_method : str The method to call on the estimator to get the response values. c`t||||||_dS)N) score_funcsignkwargsrG)rxrL _thresholds)r+rrr thresholdsrGrys r-rLz_CurveScorer.__init__s@ !+    &r/cv||j|j|||j}||_|S)z0Create a continuous scorer from a normal scorer.)rrrGrr) _score_func_sign_kwargs_get_metadata_request_metadata_request)clsscorerrGrinstances r- from_scorerz_CurveScorer.from_scorersK3)+!>    &,%A%A%C%C"r/c  |j| ij| tjt r@t jt j t j j}nt j j} fd|D}t j ||fS)aEvaluate predicted target values for X relative to y_true. Parameters ---------- method_caller : callable Returns predictions given an estimator, method name, and other arguments, potentially caching results. estimator : object Trained estimator to use for scoring. X : {array-like, sparse matrix} of shape (n_samples, n_features) Test data that will be fed to estimator.predict. y_true : array-like of shape (n_samples,) Gold standard target values for X. **kwargs : dict Other parameters passed to the scorer. Refer to :func:`set_score_request` for more details. Returns ------- scores : ndarray of shape (thresholds,) The scores associated to each threshold. potential_thresholds : ndarray of shape (thresholds,) The potential thresholds used to compute the scores. r;c hg|].}jjt|jfiz/Sr0)rrr?r]).0thr*r;scoring_kwargsr+r8y_trues r- z'_CurveScorer._score..so     Jd1R!3Y !      r/) _get_pos_label_response_methodr isinstancerrr3linspaceminmaxasarrayr4) r+ method_callerr*rYrrpotential_thresholdsscore_thresholdsr;rr8s ` ` ` @@@r-_scorez_CurveScorer._scores<'')) - t,a9   4DL3F3 d& 1 1 @#%;w$2B$$ $&:d.>#?#?           +    x())+???r/) rkrlrmrnrL classmethodrrrrs@r-rrss2&&&&&  [ 5@5@5@5@5@5@5@r/rc|tt||t||} }t||t||} } t|||} t|||} |j|| fi| n|||} } } ||| | fi| S)aFit a classifier and compute the scores for different decision thresholds. Parameters ---------- classifier : estimator instance The classifier to fit and use for scoring. If `classifier` is already fitted, it will be used as is. X : {array-like, sparse matrix} of shape (n_samples, n_features) The entire dataset. y : array-like of shape (n_samples,) The entire target vector. fit_params : dict Parameters to pass to the `fit` method of the underlying classifier. train_idx : ndarray of shape (n_train_samples,) or None The indices of the training set. If `None`, `classifier` is expected to be already fitted. val_idx : ndarray of shape (n_val_samples,) The indices of the validation set used to score `classifier`. If `train_idx`, the entire set will be used. curve_scorer : scorer instance The scorer taking `classifier` and the validation set as input and outputting decision thresholds and scores as a curve. Note that this is different from the usual scorer that output a single score value: * when `score_method` is one of the four constraint metrics, the curve scorer will output a curve of two scores parametrized by the decision threshold, e.g. TPR/TNR or precision/recall curves for each threshold; * otherwise, the curve scorer will output a single score value for each threshold. score_params : dict Parameters to pass to the `score` method of the underlying scorer. Returns ------- scores : ndarray of shape (thresholds,) or tuple of such arrays The scores computed for each decision threshold. When TPR/TNR or precision/ recall are computed, `scores` is a tuple of two arrays. potential_thresholds : ndarray of shape (thresholds,) The decision thresholds used to compute the scores. They are returned in ascending order. Nindices)rrrB) classifierrYrR fit_params train_idxval_idx curve_scorer score_paramsX_trainX_valy_trainy_valfit_params_trainscore_params_vals r-_fit_and_score_over_thresholdsrsz'955~a7Q7Q'955~a7Q7Q/:yQQQ/<QQQ w<<+;<<<<)*A|&u < E5 E E4D E EEr/cdtjfdt||DdS)alCompute the mean interpolated score across folds by defining common thresholds. Parameters ---------- target_thresholds : ndarray of shape (thresholds,) The thresholds to use to compute the mean score. cv_thresholds : ndarray of shape (n_folds, thresholds_fold) The thresholds used to compute the scores for each fold. cv_scores : ndarray of shape (n_folds, thresholds_fold) The scores computed for each threshold for each fold. Returns ------- mean_score : ndarray of shape (thresholds,) The mean score across all folds for each target threshold. cBg|]\}}tj||Sr0)r3interp)rsplit_thresholds split_scoretarget_thresholdss r-rz,_mean_interpolated_score..cs>   - + I')9; G G   r/r)axis)r3meanzip)r cv_thresholds cv_scoress` r-_mean_interpolated_scorerOsS& 7    14]I1N1N       r/c eZdZUdZiejeeee e ge e ddddgdedhe e d d d gd ge dgd gd gdZeed<dddddddddfd ZdZdZdZdZxZS)TunedThresholdClassifierCVa>Classifier that post-tunes the decision threshold using cross-validation. This estimator post-tunes the decision threshold (cut-off point) that is used for converting posterior probability estimates (i.e. output of `predict_proba`) or decision scores (i.e. output of `decision_function`) into a class label. The tuning is done by optimizing a binary metric, potentially constrained by a another metric. Read more in the :ref:`User Guide `. .. versionadded:: 1.5 Parameters ---------- estimator : estimator instance The classifier, fitted or not, for which we want to optimize the decision threshold used during `predict`. scoring : str or callable, default="balanced_accuracy" The objective metric to be optimized. Can be one of: * a string associated to a scoring function for binary classification (see model evaluation documentation); * a scorer callable object created with :func:`~sklearn.metrics.make_scorer`; response_method : {"auto", "decision_function", "predict_proba"}, default="auto" Methods by the classifier `estimator` corresponding to the decision function for which we want to find a threshold. It can be: * if `"auto"`, it will try to invoke, for each classifier, `"predict_proba"` or `"decision_function"` in that order. * otherwise, one of `"predict_proba"` or `"decision_function"`. If the method is not implemented by the classifier, it will raise an error. thresholds : int or array-like, default=100 The number of decision threshold to use when discretizing the output of the classifier `method`. Pass an array-like to manually specify the thresholds to use. cv : int, float, cross-validation generator, iterable or "prefit", default=None Determines the cross-validation splitting strategy to train classifier. Possible inputs for cv are: * `None`, to use the default 5-fold stratified K-fold cross validation; * An integer number, to specify the number of folds in a stratified k-fold; * A float number, to specify a single shuffle split. The floating number should be in (0, 1) and represent the size of the validation set; * An object to be used as a cross-validation generator; * An iterable yielding train, test splits; * `"prefit"`, to bypass the cross-validation. Refer :ref:`User Guide ` for the various cross-validation strategies that can be used here. .. warning:: Using `cv="prefit"` and passing the same dataset for fitting `estimator` and tuning the cut-off point is subject to undesired overfitting. You can refer to :ref:`TunedThresholdClassifierCV_no_cv` for an example. This option should only be used when the set used to fit `estimator` is different from the one used to tune the cut-off point (by calling :meth:`TunedThresholdClassifierCV.fit`). refit : bool, default=True Whether or not to refit the classifier on the entire training set once the decision threshold has been found. Note that forcing `refit=False` on cross-validation having more than a single split will raise an error. Similarly, `refit=True` in conjunction with `cv="prefit"` will raise an error. n_jobs : int, default=None The number of jobs to run in parallel. When `cv` represents a cross-validation strategy, the fitting and scoring on each data split is done in parallel. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. random_state : int, RandomState instance or None, default=None Controls the randomness of cross-validation when `cv` is a float. See :term:`Glossary `. store_cv_results : bool, default=False Whether to store all scores and thresholds computed during the cross-validation process. Attributes ---------- estimator_ : estimator instance The fitted classifier used when predicting. best_threshold_ : float The new decision threshold. best_score_ : float or None The optimal score of the objective metric, evaluated at `best_threshold_`. cv_results_ : dict or None A dictionary containing the scores and thresholds computed during the cross-validation process. Only exist if `store_cv_results=True`. The keys are `"thresholds"` and `"scores"`. classes_ : ndarray of shape (n_classes,) The class labels. n_features_in_ : int Number of features seen during :term:`fit`. Only defined if the underlying estimator exposes such an attribute when fit. feature_names_in_ : ndarray of shape (`n_features_in_`,) Names of features seen during :term:`fit`. Only defined if the underlying estimator exposes such an attribute when fit. See Also -------- sklearn.model_selection.FixedThresholdClassifier : Classifier that uses a constant threshold. sklearn.calibration.CalibratedClassifierCV : Estimator that calibrates probabilities. Examples -------- >>> from sklearn.datasets import make_classification >>> from sklearn.ensemble import RandomForestClassifier >>> from sklearn.metrics import classification_report >>> from sklearn.model_selection import TunedThresholdClassifierCV, train_test_split >>> X, y = make_classification( ... n_samples=1_000, weights=[0.9, 0.1], class_sep=0.8, random_state=42 ... ) >>> X_train, X_test, y_train, y_test = train_test_split( ... X, y, stratify=y, random_state=42 ... ) >>> classifier = RandomForestClassifier(random_state=0).fit(X_train, y_train) >>> print(classification_report(y_test, classifier.predict(X_test))) precision recall f1-score support 0 0.94 0.99 0.96 224 1 0.80 0.46 0.59 26 accuracy 0.93 250 macro avg 0.87 0.72 0.77 250 weighted avg 0.93 0.93 0.92 250 >>> classifier_tuned = TunedThresholdClassifierCV( ... classifier, scoring="balanced_accuracy" ... ).fit(X_train, y_train) >>> print( ... f"Cut-off point found at {classifier_tuned.best_threshold_:.3f}" ... ) Cut-off point found at 0.342 >>> print(classification_report(y_test, classifier_tuned.predict(X_test))) precision recall f1-score support 0 0.96 0.95 0.96 224 1 0.61 0.65 0.63 26 accuracy 0.92 250 macro avg 0.78 0.80 0.79 250 weighted avg 0.92 0.92 0.92 250 r"Nleft)closedz array-like cv_objectprefitrg?neitherru random_state)scoringrcvrefitn_jobsrstore_cv_resultsrHbalanced_accuracyrEdTF)rrGrrrrrrct||||_||_||_||_||_||_| |_dSrw) rxrLrrrrrrr) r+r*rrGrrrrrrrys r-rLz#TunedThresholdClassifierCV.__init__!s[ 9oNNN $  ( 0r/c tjtr2djcxkrdkr nntdjjnjdkrYjdurt d tjdn"#t$r}td|d }~wwxYwjnGtjd jd ur' dkrt d td fi| _dkr3j_jd t!t#fg}nt%j_t%jjfijj}jrjj}}}net-jfijj\} } t/| }t/| }t1jj| }jj||fi|t3t5jfd|D\} } t9d| Drt dt;d| D} t=d| D}tjt@rtCj"| |j}ntCj#j}tI|| | }|%}||_&||_'j(r ||d_)S)aFit the classifier and post-tune the decision threshold. Parameters ---------- X : {array-like, sparse matrix} of shape (n_samples, n_features) Training data. y : array-like of shape (n_samples,) Target values. **params : dict Parameters to pass to the `fit` method of the underlying classifier and to the `scoring` scorer. Returns ------- self : object Returns an instance of self. rr")n_splits test_sizerrTz'When cv='prefit', refit cannot be True.r]z-When cv='prefit', `estimator` must be fitted.N)rRrFz1When cv has several folds, refit cannot be False.rBr)rc 3K|]Z\}}ttdkrtnjj||jjjV[dS)r)rrrrrN)rrr r*rB _curve_scorerrscore) rrrrYrrr{r+rRs r- z2TunedThresholdClassifierCV._fit..}s * *'Iw8677)+xE*%%%Z,6:'#!%!3!.!5!;    * * * * * *r/c3XK|]%}tj|d|dV&dS)rN)r3isclose)rrs r-rz2TunedThresholdClassifierCV._fit..s6AARrz"Q%B((AAAAAAr/zrThe provided estimator makes constant predictions. Therefore, it is impossible to optimize the decision threshold.c3>K|]}|VdSrK)rrrs r-rz2TunedThresholdClassifierCV._fit..@  '7  " "      r/c3>K|]}|VdSrK)rrs r-rz2TunedThresholdClassifierCV._fit..rr/)num)rscores)*rrrr#rrrWr r*r r$ get_n_splitsr_get_curve_scorerrr'rangerr splitsplitterrBnextrrrrranyrrrrr3rrrargmax best_score_best_threshold_r cv_results_)r+rYrRrZexcsplitsrrrrrrr min_threshold max_thresholddecision_thresholdsobjective_scoresbest_idxrrr{s``` @@@r-rXzTunedThresholdClassifierCV._fit7s4( dgt $ $ VTWq'dgD>v>> !3355 >>"nDOJU<??3345FF#DN33DOt~..JRXaCCm&<&BCCFz 56=;R;V"2 $HBHQ$R$R]5K5Q$R$RSS 1(I66(I66#7}.2I$$$  DO  E E4D E E E#& )XT[ ) ) ) * * * * * * * * *+1 * * *  $  = AA=AAA A A A    ;H       ;H      dox 0 0 >"$+}$/###  #%*T_"="= 3    $**,,+H528<   1*  D   s7B B,B''B,ct|d|j}t|j|||\}}t ||j|j|S)r}r'r) r rrrr'rNr?rr])r+rYr;r8rs r-rz"TunedThresholdClassifierCV.predicts| l+++&5577 0 O  % % ' '    1 T)4=)   r/ct|jj|jt dd|jt dd|t dd}|S) rrrBrrr)rrr)rr)rryrkrr*rrrrs r-rz/TunedThresholdClassifierCV.get_metadata_routings !8 9 9 9 S.,22%2NNS,22'%2PPS--//,22'%2PP  r/ct|j|j}t|||j}|S)z8Get the curve scorer based on the objective metric used.)r)r r*rrrrNr)r+rrs r-rz,TunedThresholdClassifierCV._get_curve_scorersJ EEE#// T..00$/  r/)rkrlrmrnrArHrsetrcallablerrrrrprqrLrXrrrrrs@r-rrksp``D$ ! 8$ Jss++--.. / /    x!T&AAA<P  Jz " " HZc) < < < T"'(&K!$$$D.$ 1111111,uuun   4:r/r)9collections.abcrnumbersrrnumpyr3baserrr r r exceptionsr metricsr rmetrics._scorerrutilsrutils._param_validationrrrrutils._responserutils.metadata_routingrrrrutils.metaestimatorsrutils.multiclassrutils.parallelrrutils.validationrrr r!_splitr#r$r1r?rArtrrrrr0r/r-rs******""""""""('''''*)))))""""""RRRRRRRRRRRR999999 0/////------........ 54444444" W W Wa a a a a o/A=a a a Hqqqqq6qqqhf@f@f@f@f@;f@f@f@RFFFFFFR8@@@@@!8@@@@@r/