From 53c83815d279a1c7a75438afeafe039b96b793a1 Mon Sep 17 00:00:00 2001 From: fzh <1155233259@link.cuhk.edu.hk> Date: Sat, 11 Apr 2026 02:03:28 +0800 Subject: [PATCH 1/2] docs: update two notebooks --- doc/source/autoapi/rehline/index.rst | 213 +++- doc/source/example.rst | 4 + doc/source/examples/MonotonicSVM.ipynb | 295 ++++++ doc/source/examples/Smooth_SVM_new.ipynb | 1187 ++++++++++++++++++++++ doc/source/tutorials/constraint.rst | 7 + doc/source/tutorials/loss.rst | 1 + 6 files changed, 1692 insertions(+), 15 deletions(-) create mode 100644 doc/source/examples/MonotonicSVM.ipynb create mode 100644 doc/source/examples/Smooth_SVM_new.ipynb diff --git a/doc/source/autoapi/rehline/index.rst b/doc/source/autoapi/rehline/index.rst index 2df4f41..96d1691 100644 --- a/doc/source/autoapi/rehline/index.rst +++ b/doc/source/autoapi/rehline/index.rst @@ -19,6 +19,8 @@ Overview - ReHLine Minimization. * - :py:obj:`plqERM_Ridge ` - Empirical Risk Minimization (ERM) with a piecewise linear-quadratic (PLQ) objective with a ridge penalty. + * - :py:obj:`plqERM_ElasticNet ` + - Empirical Risk Minimization (ERM) with a piecewise linear-quadratic (PLQ) objective with a elastic net penalty. * - :py:obj:`plq_Ridge_Classifier ` - Empirical Risk Minimization (ERM) Classifier with a Piecewise Linear-Quadratic (PLQ) loss * - :py:obj:`plq_Ridge_Regressor ` @@ -32,7 +34,7 @@ Overview :widths: auto :class: summarytable - * - :py:obj:`ReHLine_solver `\ (X, U, V, Tau, S, T, A, b, Lambda, Gamma, xi, max_iter, tol, shrink, verbose, trace_freq) + * - :py:obj:`ReHLine_solver `\ (X, U, V, Tau, S, T, A, b, rho, Lambda, Gamma, xi, mu, max_iter, tol, shrink, verbose, trace_freq) - \- * - :py:obj:`plqERM_Ridge_path_sol `\ (X, y, \*None, loss, constraint, eps, n_Cs, Cs, max_iter, tol, verbose, shrink, warm_start, return_time) - Compute the PLQ Empirical Risk Minimization (ERM) path over a range of regularization parameters. @@ -497,7 +499,161 @@ Classes -.. py:class:: plq_Ridge_Classifier(loss, constraint=[], C=1.0, U=np.empty((0, 0)), V=np.empty((0, 0)), Tau=np.empty((0, 0)), S=np.empty((0, 0)), T=np.empty((0, 0)), A=np.empty((0, 0)), b=np.empty((0, )), max_iter=1000, tol=0.0001, shrink=1, warm_start=0, verbose=0, trace_freq=100, fit_intercept=True, intercept_scaling=1.0, class_weight=None) +.. py:class:: plqERM_ElasticNet(loss, constraint=[], C=1.0, l1_ratio=0.5, U=np.empty(shape=(0, 0)), V=np.empty(shape=(0, 0)), Tau=np.empty(shape=(0, 0)), S=np.empty(shape=(0, 0)), T=np.empty(shape=(0, 0)), A=np.empty(shape=(0, 0)), b=np.empty(shape=0), max_iter=1000, tol=0.0001, shrink=1, warm_start=0, verbose=0, trace_freq=100) + + Bases: :py:obj:`rehline._base._BaseReHLine`, :py:obj:`sklearn.base.BaseEstimator` + + Empirical Risk Minimization (ERM) with a piecewise linear-quadratic (PLQ) objective with a elastic net penalty. + + .. math:: + + \min_{\mathbf{\beta} \in \mathbb{R}^d} C \sum_{i=1}^n \text{PLQ}(y_i, \mathbf{x}_i^T \mathbf{\beta}) + \text{l1_ratio} \| \mathbf{\beta} \|_1 + \frac{1}{2} (1 - \text{l1_ratio}) \| \mathbf{\beta} \|_2^2, \ \text{ s.t. } \ + \mathbf{A} \mathbf{\beta} + \mathbf{b} \geq \mathbf{0}, + + The function supports various loss functions, including: + - 'hinge', 'svm' or 'SVM' + - 'check' or 'quantile' or 'quantile regression' or 'QR' + - 'sSVM' or 'smooth SVM' or 'smooth hinge' + - 'TV' + - 'huber' or 'Huber' + - 'SVR' or 'svr' + + The following constraint types are supported: + * 'nonnegative' or '>=0': A non-negativity constraint. + * 'fair' or 'fairness': A fairness constraint. + * 'custom': A custom constraint, where the user must provide the constraint matrix 'A' and vector 'b'. + + Parameters + ---------- + loss : dict + A dictionary specifying the loss function parameters. + + constraint : list of dict + A list of dictionaries, where each dictionary represents a constraint. + Each dictionary must contain a 'name' key, which specifies the type of constraint. + + C : float, default=1.0 + Regularization parameter. The strength of the regularization is + inversely proportional to C. Must be strictly positive. + `C` will be absorbed by the ReHLine parameters when `self.make_ReLHLoss` is conducted. + + l1_ratio : float, default=0.5 + The ElasticNet mixing parameter, with 0 <= l1_ratio < 1. For l1_ratio = 0 the penalty + is an L2 penalty. For 0 < l1_ratio < 1, the penalty is a combination of L1 and L2. + + verbose : int, default=0 + Enable verbose output. Note that this setting takes advantage of a + per-process runtime setting in liblinear that, if enabled, may not work + properly in a multithreaded context. + + max_iter : int, default=1000 + The maximum number of iterations to be run. + + _U, _V: array of shape (L, n_samples), default=np.empty(shape=(0, 0)) + The parameters pertaining to the ReLU part in the loss function. + + _Tau, _S, _T: array of shape (H, n_samples), default=np.empty(shape=(0, 0)) + The parameters pertaining to the ReHU part in the loss function. + + _A: array of shape (K, n_features), default=np.empty(shape=(0, 0)) + The coefficient matrix in the linear constraint. + + _b: array of shape (K, ), default=np.empty(shape=0) + The intercept vector in the linear constraint. + + Attributes + ---------- + coef\_ : array-like + The optimized model coefficients. + + n_iter\_ : int + The number of iterations performed by the ReHLine solver. + + opt_result\_ : object + The optimization result object. + + dual_obj\_ : array-like + The dual objective function values. + + primal_obj\_ : array-like + The primal objective function values. + + Methods + ------- + fit(X, y, sample_weight=None) + Fit the model based on the given training data. + + decision_function(X) + The decision function evaluated on the given dataset. + + Notes + ----- + The `plqERM_ElasticNet` class is a subclass of `_BaseReHLine` and `BaseEstimator`, which suggests that it is part of a larger framework for implementing ReHLine algorithms. + + + + Overview + ======== + + + .. list-table:: Methods + :header-rows: 0 + :widths: auto + :class: summarytable + + * - :py:obj:`fit `\ (X, y, sample_weight) + - Fit the model based on the given training data. + * - :py:obj:`decision_function `\ (X) + - The decision function evaluated on the given dataset + + + Members + ======= + + .. py:method:: fit(X, y, sample_weight=None) + + Fit the model based on the given training data. + + Parameters + ---------- + + X: {array-like} of shape (n_samples, n_features) + Training vector, where `n_samples` is the number of samples and + `n_features` is the number of features. + + y : array-like of shape (n_samples,) + The target variable. + + sample_weight : array-like of shape (n_samples,), default=None + Array of weights that are assigned to individual + samples. If not provided, then each sample is given unit weight. + + Returns + ------- + self : object + An instance of the estimator. + + + + + .. py:method:: decision_function(X) + + The decision function evaluated on the given dataset + + Parameters + ---------- + X : array-like of shape (n_samples, n_features) + The data matrix. + + Returns + ------- + ndarray of shape (n_samples, ) + Returns the decision function of the samples. + + + + +.. py:class:: plq_Ridge_Classifier(loss, constraint=[], C=1.0, U=np.empty((0, 0)), V=np.empty((0, 0)), Tau=np.empty((0, 0)), S=np.empty((0, 0)), T=np.empty((0, 0)), A=np.empty((0, 0)), b=np.empty((0, )), max_iter=1000, tol=0.0001, shrink=1, warm_start=0, verbose=0, trace_freq=100, fit_intercept=True, intercept_scaling=1.0, class_weight=None, multi_class=[], n_jobs=None) Bases: :py:obj:`rehline._class.plqERM_Ridge`, :py:obj:`sklearn.base.ClassifierMixin` @@ -511,6 +667,7 @@ Classes - Supports optional intercept fitting (via an augmented constant feature). - Provides standard methods ``fit``, ``predict``, and ``decision_function``. - Integrates with scikit-learn ecosystem (e.g., GridSearchCV, Pipeline). + - Supports multiclass classification via OvR or OvO method. Parameters ---------- @@ -571,17 +728,33 @@ Classes - 'balanced' uses n_samples / (n_classes * n_j). - dict maps label -> weight in the ORIGINAL label space. + multi_class : str or list, default=[] + Method for multiclass classification. Options: + - 'ovo': One-vs-One, trains K*(K-1)/2 binary classifiers. + - 'ovr': One-vs-Rest, trains K binary classifiers. + - [ ] or ignored when only 2 classes are present. + + n_jobs : int or None, default=None + Number of parallel jobs for multiclass fitting. + None means 1 (serial). -1 means use all available CPUs. + Passed directly to joblib.Parallel. + + Attributes ---------- - coef_ : ndarray of shape (n_features,) - Coefficients excluding the intercept. + ``coef_ ``: ndarray of shape (n_features,) for binary, (n_estimators, n_features) for multiclass + Coefficients of all fitted classifiers, excluding the intercept. - intercept_ : float - Intercept term. 0.0 if ``fit_intercept=False``. + ``intercept_ ``: float for binary, ndarray of shape (n_estimators,) for multiclass + Intercept term(s). 0.0 if ``fit_intercept=False``. - classes_ : ndarray of shape (2,) + classes_ : ndarray of shape (n_classes,) Unique class labels in the original label space. + estimators_ : list, only present for multiclass + For OvR: list of (coef, intercept) tuples, length K. + For OvO: list of (coef, intercept, cls_i, cls_j) tuples, length K*(K-1)/2. + _label_encoder : LabelEncoder Encodes original labels into {0,1} for internal training. @@ -631,20 +804,30 @@ Classes Compute the decision function for samples in X. + For binary classification, returns a 1D array of scores. + For OvR multiclass, returns a 2D array of shape (n_samples, K). + For OvO multiclass, returns a 2D array of shape (n_samples, K*(K-1)/2). + + Using coef_.T works uniformly for both binary (n_features,) and + multiclass (n_estimators, n_features) shapes. + Parameters ---------- - X : array-like of shape (n_samples, n_features) + X : array-like of shape (n_samples, n_features) Input samples. Returns ------- - ndarray of shape (n_samples,) + ndarray of shape (n_samples,) or (n_samples, n_estimators) Continuous scores for each sample. .. py:method:: predict(X) Predict class labels for samples in X. + For binary classification, thresholds the decision score at 0. + For OvR, takes the argmax across K classifiers. + For OvO, uses majority voting across K*(K-1)/2 classifiers. Parameters ---------- @@ -722,11 +905,11 @@ Classes Attributes ---------- - coef_ : ndarray of shape (n_features,) + ``coef_`` : ndarray of shape (n_features,) Learned linear coefficients (excluding the intercept term). - intercept_ : float + ``intercept_`` : float Intercept term extracted from the last coefficient when ``fit_intercept=True``, otherwise 0.0. - n_features_in_ : int + ``n_features_in_`` : int Number of input features seen during :meth:`fit` (before intercept augmentation). Notes @@ -748,7 +931,7 @@ Classes * - :py:obj:`fit `\ (X, y, sample_weight) - If ``fit_intercept=True``, a constant column (value = ``intercept_scaling``) is appended * - :py:obj:`decision_function `\ (X) - - Compute f(X) = X @ coef_ + intercept_. + - Compute f(X) = X @ ``coef_`` + ``intercept_``. * - :py:obj:`predict `\ (X) - Predict targets as the linear decision function. @@ -782,7 +965,7 @@ Classes .. py:method:: decision_function(X) - Compute f(X) = X @ coef_ + intercept_. + Compute f(X) = X @ ``coef_`` + ``intercept_``. Parameters ---------- @@ -1059,7 +1242,7 @@ Classes Functions --------- -.. py:function:: ReHLine_solver(X, U, V, Tau=np.empty(shape=(0, 0)), S=np.empty(shape=(0, 0)), T=np.empty(shape=(0, 0)), A=np.empty(shape=(0, 0)), b=np.empty(shape=0), Lambda=np.empty(shape=(0, 0)), Gamma=np.empty(shape=(0, 0)), xi=np.empty(shape=(0, 0)), max_iter=1000, tol=0.0001, shrink=1, verbose=1, trace_freq=100) +.. py:function:: ReHLine_solver(X, U, V, Tau=np.empty(shape=(0, 0)), S=np.empty(shape=(0, 0)), T=np.empty(shape=(0, 0)), A=np.empty(shape=(0, 0)), b=np.empty(shape=0), rho=0.0, Lambda=np.empty(shape=(0, 0)), Gamma=np.empty(shape=(0, 0)), xi=np.empty(shape=(0, 0)), mu=np.empty(shape=(0, 0)), max_iter=1000, tol=0.0001, shrink=1, verbose=1, trace_freq=100) .. py:function:: plqERM_Ridge_path_sol(X, y, *, loss, constraint=[], eps=0.001, n_Cs=100, Cs=None, max_iter=5000, tol=0.0001, verbose=0, shrink=1, warm_start=False, return_time=True) diff --git a/doc/source/example.rst b/doc/source/example.rst index 4d79644..682ed50 100644 --- a/doc/source/example.rst +++ b/doc/source/example.rst @@ -20,6 +20,8 @@ Example Gallery examples/Sklearn_Mixin.ipynb examples/Multiclass_Classification.ipynb examples/NMF.ipynb + examples/Smooth_SVM_new.ipynb + examples/MonotonicSVM.ipynb List of Examples ---------------- @@ -39,3 +41,5 @@ List of Examples examples/Sklearn_Mixin.ipynb examples/Multiclass_Classification.ipynb examples/NMF.ipynb + examples/Smooth_SVM_new.ipynb + examples/MonotonicSVM.ipynb diff --git a/doc/source/examples/MonotonicSVM.ipynb b/doc/source/examples/MonotonicSVM.ipynb new file mode 100644 index 0000000..ee005f1 --- /dev/null +++ b/doc/source/examples/MonotonicSVM.ipynb @@ -0,0 +1,295 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "S7Tq3zYqOu6u" + }, + "source": [ + "# Monotonic SVM\n", + "The Monotonic SVM solves the following optimization problem:\n", + "\n", + "$$\n", + " \\min_{\\mathbf{\\beta} \\in \\mathbb{R}^d} \\frac{C}{n} \\sum_{i=1}^n (1 - y_i \\mathbf{\\beta}^\\intercal \\mathbf{x}_i)_+ + \\frac{1}{2} \\|\\mathbf{\\beta}\\|_2^2,\n", + "$$\n", + "\n", + "$$\n", + " \\text{subject to} \\quad \\beta_j \\le \\beta_{j+1} \\quad \\forall j \\in \\{1, \\dots, d-1\\} \\quad (\\text{Increasing})\n", + "$$\n", + "$$\n", + " \\text{or} \\quad \\beta_j \\ge \\beta_{j+1} \\quad \\forall j \\in \\{1, \\dots, d-1\\} \\quad (\\text{Decreasing})\n", + "$$\n", + "\n", + "where:\n", + "\n", + "* $\\mathbf{x}_i \\in \\mathbb{R}^d$ is a feature vector\n", + "* $y_i \\in \\{-1, 1\\}$ is a binary label\n", + "* $\\beta_j$ represents the $j$-th component of the coefficient vector $\\mathbf{\\beta}$\n", + "\n", + "The monotonicity constraints ensure that the learned coefficients $\\beta$ follow a strictly non-decreasing or non-increasing order, useful when incorporating prior domain knowledge.\n", + "\n", + "> **Note.** Since the hinge loss is a plq function and the monotonicity constraints are purely linear (e.g., $\\beta_j - \\beta_{j+1} \\le 0$), we can optimize it using `rehline.plq_Ridge_Classifier`.\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "g0aAqkkRKc6z" + }, + "outputs": [], + "source": [ + "## install rehline\n", + "%pip install rehline -q" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "id": "4ibK-1gsR0ZB" + }, + "outputs": [], + "source": [ + "## simulate data\n", + "from sklearn.datasets import make_classification\n", + "from sklearn.preprocessing import StandardScaler\n", + "import numpy as np\n", + "\n", + "scaler = StandardScaler()\n", + "\n", + "n, d = 10000, 5\n", + "X, y = make_classification(n_samples=n, n_features=d, n_redundant=0, random_state=42)\n", + "y = 2*y - 1\n", + "X = scaler.fit_transform(X)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Oak-k1Ps9hDS" + }, + "source": [ + "## SVM as baseline" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "Uk31Pe_cg702" + }, + "outputs": [], + "source": [ + "## we first run a SVM\n", + "from rehline import plq_Ridge_Classifier\n", + "\n", + "clf = plq_Ridge_Classifier(loss={'name': 'svm'}, C=0.001, max_iter=10000)\n", + "clf.fit(X=X, y=y)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "OU3yWxlR9d29" + }, + "source": [ + "## Monotonic constraint" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "APrF6OYJl1_t" + }, + "outputs": [], + "source": [ + "## solve SVM with Monotonicity Constraint via `plq_Ridge_Classifier`\n", + "from rehline import plq_Ridge_Classifier\n", + "mclf = plq_Ridge_Classifier(\n", + " loss={'name': 'svm'},\n", + " constraint = [{'name': 'monotonic', 'decreasing': True}],\n", + " C=0.001,\n", + " max_iter=10000\n", + ")\n", + "mclf.fit(X=X, y=y)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "GbDGb79w9aOh" + }, + "source": [ + "## Results" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "fDmbka9Wx-jd", + "outputId": "65947a94-eb63-4a00-d968-68772c68c954" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + " Model Performance\n", + " Standard SVM 0.8870\n", + "Monotonic SVM 0.7332\n" + ] + } + ], + "source": [ + "import pandas as pd\n", + "\n", + "## score\n", + "score = clf.decision_function(X)\n", + "mscore = mclf.decision_function(X)\n", + "\n", + "svm_perf = clf.score(X, y)\n", + "msvm_perf = mclf.score(X, y)\n", + "\n", + "## Create a pandas DataFrame to store the results\n", + "results = pd.DataFrame({\n", + " 'Model': ['Standard SVM', 'Monotonic SVM'],\n", + " 'Performance': [svm_perf, msvm_perf]\n", + "})\n", + "\n", + "## Print the results as a table\n", + "print(results.to_string(index=False))" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 528 + }, + "id": "CdiaiZCTQiqm", + "outputId": "9b4ddc16-c48b-4f94-8fdc-0844381a0db5" + }, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "
" + ], + "image/png": "iVBORw0KGgoAAAANSUhEUgAABW0AAAHqCAYAAAB/bWzAAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAXvFJREFUeJzt3XlYlPX+//HXALIKGIK4gbviCkdcjrlhoYgd08q0tCOS2Uobpemvcmkz9WgezbRTKWaapqWeo7kk5ZKV5oKZaQcL01JA3EUDhfv3h1/mOA4gKDg3zvNxXXPVfO7P3PO+uVHevrjnc1sMwzAEAAAAAAAAADAFF0cXAAAAAAAAAAD4H0JbAAAAAAAAADARQlsAAAAAAAAAMBFCWwAAAAAAAAAwEUJbAAAAAAAAADARQlsAAAAAAAAAMBFCWwAAAAAAAAAwEUJbAAAAAAAAADARQlsAAAAAAAAAMBFCWwCowM6ePauHHnpI1atXl8Vi0TPPPCNJysjIUL9+/VS1alVZLBZNnTpV69evl8Vi0fr160v1HmPHjpXFYin74m+QXr16adiwYY4uo1BJSUmyWCw6cOBAub3HTz/9JDc3N/3444/l9h4AADirAwcOyGKxKCkpydGlOJVJkyapfv36cnV1VUREhCTp4sWLGjFihEJCQuTi4qK+fftKkiwWi8aOHVuq/V9r32wWEydOVFhYmPLz8x1dip0b8WfmwoULCgkJ0TvvvFNu7wHcCIS2AExr9+7d6tevn+rUqSNPT0/VqlVL3bt31/Tp0yVJO3bskMVi0UsvvVTkPlJTU2WxWJSYmCjpfwGki4uLDh06ZDf/9OnT8vLyksViUUJCQonqzMvL05w5cxQVFaWAgAB5eHiobt26io+P17Zt267hyEvujTfeUFJSkh577DHNmzdPf//73yVJzz77rNasWaNRo0Zp3rx56tmzZ7nWcb0OHz6ssWPHKiUlpUz3u3nzZq1du1YvvPBCme63ImnWrJnuuOMOjR492tGlAAAqoIJfMFosFn399dd22w3DUEhIiCwWi/72t785oEJ7P/30k8aOHVuuvxQtL0ePHtXTTz+tsLAweXl5qVq1amrXrp1eeOEFnT17VhcuXFBgYKA6depU5D4Kzknr1q0l/S+AtFgs+uijjwp9TceOHWWxWNSiRYsS17p+/Xrdfffdql69utzd3VWtWjX17t1bn332WekOupTWrl2rESNGqGPHjpozZ47eeOMNSdLs2bM1adIk9evXT3PnztWzzz5brnWUhTfeeEPLli0r032ePn1aEyZM0AsvvCAXF+eMfCpVqqTExES9/vrr+vPPPx1dDnDtDAAwoc2bNxvu7u5Gw4YNjVdffdV47733jNGjRxs9evQwGjRoYJ0XFhZm1K9fv8j9jB071pBkbN++3TAMwxgzZowhyfD09DQmTJhgN3/OnDmGp6enIcl44oknrlrnuXPnjJ49exqSjC5duhiTJk0yPvjgA+Pll182mjRpYlgsFuPQoUPX8BUomfbt2xsdO3a0Gw8ODjYGDRpkM5aXl2ecP3/eyMvLK9V7XLhwwTh//vx11Xk133//vSHJmDNnTpnut0+fPkaPHj3KdJ9lac6cOYYkIy0trVzf5/PPPzckGfv37y/X9wEA3HwKflZ5enoajz32mN32r776ypBkeHh4GHfccYcDKrS3ePFiQ5Lx1Vdflft75efnG+fPnzcuXrx43fs6duyYERoaalSpUsVITEw0/vWvfxnjx4837r//fsPX19faLzz66KOGxWIxDhw4UOh+1q9fb0gyJk+ebBjG/86Rp6enERsbazc/LS3Nur158+YlqnX06NGGJKNRo0bG6NGjjQ8++MCYOHGiERUVZUgy5s+ff21fhBJ44YUXDBcXFyMnJ8dmfMCAAUatWrXs5p8/f964cOFCqd7jWvvm0vLx8THi4uLKdJ9vvfWW4efnV+79+7Uq+H4r677/SidOnDDc3d2NDz74oFzfByhPbjc+JgaAq3v99dfl7++v77//XlWqVLHZlpmZaf3/QYMG6eWXX9Z3332nv/71r3b7+fjjjxUWFma90qBAr1699PHHH2vEiBE24wsWLNAdd9yhTz/9tER1Dh8+XKtXr9Zbb71lXZqgwJgxY/TWW2+VaD/XKjMzU82aNSt0/Mqvm4uLizw9PUv9Hm5ubnJzq3g/LjIzM7Vy5UrNmjXL0aU4XHR0tG655RbNnTtXr7zyiqPLAQBUQL169dLixYs1bdo0m75gwYIFioyMVFZWlgOrcxyLxXJN/VVhPvjgAx08eFCbN2/WrbfearPt9OnTcnd3l3Sp/501a5Y+/vhjjRw50m4/CxYskIuLi+677z6b8V69eunf//63srKyFBgYaDM/ODhYjRo10okTJ65a55IlS/TKK6+oX79+WrBggSpVqmTdNnz4cK1Zs0YXLlwo1bGXRmZmpry8vKxfj8vHr+x/JV3T+bnWvtkM5syZozvvvLPC1l9WqlSpoh49eigpKUkPPvigo8sBrolzXisPwPR++eUXNW/evNDGq1q1atb/HzRokKRLzeaVtm/frp9//tk653IDBw5USkqK9u3bZx1LT0/Xl19+qYEDB5aoxt9//13vvvuuunfvbhfYSpKrq6uef/551a5d2zq2c+dOxcbGys/PT5UrV9btt9+u7777zu61J0+e1DPPPKOQkBB5eHioYcOGmjBhgnVdqoKPuaWlpWnlypXWj7wVfITRMAzNmDHDOn75a65cm2vLli3q1auXbrnlFvn4+KhVq1b65z//ad1e1Jq2H330kSIjI+Xl5aWAgADdd999dktOREVFqUWLFvrpp5/UrVs3eXt7q1atWpo4caJ1zvr169W2bVtJUnx8vM2xSJeWuLjnnntUvXp1eXp6qnbt2rrvvvt06tSpYs6OtHLlSl28eFHR0dE24xcuXNC4cePUqFEjeXp6qmrVqurUqZO++OILSZcaXYvFop07d9rt84033pCrq6v++OMPm+P74Ycf1LVrV3l7e6thw4ZasmSJJGnDhg1q3769vLy81KRJE61bt67Ymgu88847at68uTw8PFSzZk098cQTOnnypN28GTNmqH79+vLy8lK7du20adMmRUVFKSoqymZepUqVFBUVpeXLl5fo/QEAuNL999+vY8eOWX9eSlJubq6WLFlSZO+UnZ2t5557ztrPNGnSRP/4xz9kGIbNvIJlqZYtW6YWLVrIw8NDzZs31+rVq+32ebVeKikpSffee68kqVu3bta+4vL+pyQ/Z0vSw0hFr8+5b98+9e/fX0FBQdY+4MUXXyzy6ytd6n9dXV0LvRDBz8/PGsJ17NhRdevWLbT/vXDhgpYsWaJu3bqpZs2aNtv69OkjDw8PLV682GZ8wYIF6t+/v1xdXYutr8DLL7+sgIAAzZ492yawLRATE2OzVEZmZqaGDh2q4OBgeXp6Kjw8XHPnzrV7XX5+vqZOnarmzZvL09NTwcHBeuSRR2yCZIvFojlz5ig7O9uu//3qq6+0Z88eu3Ne2Jq2f/zxh4YOHaqaNWvKw8ND9erV02OPPabc3FxJxffNPXv2lL+/v7y9vdW1a1dt3rzZZk5B77x//34NGTJEVapUkb+/v+Lj43Xu3DmbY8nOztbcuXOtNQ8ZMkSSdObMGT3zzDOqW7euPDw8VK1aNXXv3l07duwo9tykpaXphx9+sOt/JWnhwoWKjIyUr6+v/Pz81LJlS2vP/+uvv8pisRR6wck333wji8Wijz/+2Ob4/vvf/+qBBx6Qv7+/goKC9PLLL8swDB06dEh9+vSRn5+fqlevrsmTJxdbc4Evv/xSnTt3lo+Pj6pUqaI+ffpo7969dvPWr1+vNm3ayNPTUw0aNNC7775b5L9Xunfvrq+//lrHjx8vUQ2A2RDaAjClOnXqaPv27Ve9eVK9evV066236pNPPlFeXp7NtoJGtrB/SHTp0kW1a9e2aXYXLVqkypUr64477ihRjatWrdLFixet68hezZ49e9S5c2ft2rVLI0aM0Msvv6y0tDRFRUVpy5Yt1nnnzp1T165d9dFHH2nw4MGaNm2aOnbsqFGjRlnX5m3atKnmzZunwMBARUREaN68eZo3b57atm2refPmSbrUpBSMF+WLL75Qly5d9NNPP+npp5/W5MmT1a1bN61YsaLYY3n99dc1ePBgNWrUSFOmTNEzzzyj5ORkdenSxe4fPSdOnFDPnj0VHh6uyZMnKywsTC+88IJWrVplPZaCqz8ffvhha81dunRRbm6uYmJi9N133+nJJ5/UjBkz9PDDD+vXX38tNMS83DfffKOqVauqTp06NuNjx47VuHHj1K1bN7399tt68cUXFRoaam2C+/XrJy8vL82fP99un/Pnz1dUVJRq1aplc3x/+9vf1L59e02cOFEeHh667777tGjRIt13333q1auX3nzzTWVnZ6tfv346c+ZMsXWPHTtWTzzxhGrWrKnJkyfrnnvu0bvvvqsePXrYXLUyc+ZMJSQkqHbt2po4caI6d+6svn376vfffy90v5GRkfrxxx91+vTpYt8fAIDC1K1bVx06dLAGN9KlXujUqVN2V3RKl9ZVvfPOO/XWW2+pZ8+emjJlipo0aaLhw4db+5nLff3113r88cd13333aeLEifrzzz91zz336NixY9Y5JemlunTpoqeeekqS9P/+3/+z9hVNmzaVVPKfs9LVe5ii/PDDD2rfvr2+/PJLDRs2TP/85z/Vt29f/ec//yn2dXXq1FFeXl6xvZt0KewbOHCgdu/erT179thsW716tY4fP17oRQve3t7q06ePzTnctWuX9uzZU+KLFlJTU7Vv3z717dtXvr6+V51//vx5RUVFad68eRo0aJAmTZokf39/DRkyxOYiAUl65JFHNHz4cHXs2FH//Oc/FR8fr/nz5ysmJsZ6bubNm6fOnTvLw8PDrv8NCwtT7dq17c75lQ4fPqx27dpp4cKFGjBggKZNm6a///3v2rBhg02oeqUvv/xSXbp00enTpzVmzBi98cYbOnnypG677TZt3brVbn7//v115swZjR8/Xv3791dSUpLGjRtn3T5v3jx5eHioc+fO1pofeeQRSdKjjz6qmTNn6p577tE777yj559/Xl5eXoWGmJf75ptvJMnuU4ZffPGF7r//ft1yyy2aMGGC3nzzTUVFRVkD5/r166tjx45F9r++vr7q06ePzfiAAQOUn5+vN998U+3bt9drr72mqVOnqnv37qpVq5YmTJighg0b6vnnn9fGjRuLrXvdunWKiYlRZmamxo4dq8TERH3zzTfq2LGjzdrUO3fuVM+ePXXs2DGNGzdOQ4cO1SuvvFLkusCRkZEyDMP6dQEqHIcuzgAARVi7dq3h6upquLq6Gh06dDBGjBhhrFmzxsjNzbWbO2PGDEOSsWbNGutYXl6eUatWLaNDhw42cwvWtD169Kjx/PPPGw0bNrRua9u2rREfH28YhlGiNW2fffZZQ5Kxc+fOEh1T3759DXd3d+OXX36xjh0+fNjw9fU1unTpYh179dVXDR8fH+O///2vzetHjhxpuLq6GgcPHrSO1alTp9D14wqrv2A9s4L13S5evGjUq1fPqFOnjnHixAmbufn5+db/L/iaFThw4IDh6upqvP766zav2b17t+Hm5mYz3rVrV0OS8eGHH1rHcnJyjOrVqxv33HOPdayoNW137txpSDIWL15sd4xX06lTJyMyMtJuPDw8/Kpr7t1///1GzZo1bdYx27Fjh12NBce3YMEC69i+ffsMSYaLi4vx3XffWcfXrFlj9/or17TNzMw03N3djR49eti899tvv21IMmbPnm0YxqWvYdWqVY22bdvarNGWlJRkSDK6du1qd0wLFiwwJBlbtmwp9tgBALhcwc+q77//3nj77bcNX19f49y5c4ZhGMa9995rdOvWzTAM+55k2bJlhiTjtddes9lfv379DIvFYrPOuiTD3d3dZmzXrl2GJGP69OnWsZL2UkWtaVvSn7OGUfIeprD1Obt06WL4+voav/32m837X95fFSY9Pd0ICgoyJBlhYWHGo48+aixYsMA4efKk3dw9e/YYkoxRo0bZjN93332Gp6encerUKetYQQ+4ePFiY8WKFYbFYrH2k8OHD7feH6Jr165XXdN2+fLlhiTjrbfeKnZegalTpxqSjI8++sg6lpuba3To0MGoXLmycfr0acMwDGPTpk2FroW7evVqu/G4uDjDx8fH7r2Kql+SMWbMGOvzwYMHGy4uLsb3339vN7fgHF3ZN+fn5xuNGjUyYmJibM7juXPnjHr16hndu3e3jhX0zg8++KDNvu+66y6jatWqNmNFrWnr7+9fovtrXOmll14yJBlnzpyxGX/66acNPz+/Ytdefvfddw1Jxt69e61jubm5RmBgoE2NBcf38MMPW8cuXrxo1K5d27BYLMabb75pHT9x4oTh5eVl8/rC/sxEREQY1apVM44dO2Yd27Vrl+Hi4mIMHjzYOta7d2/D29vb+OOPP6xjqamphpubm82/VwocPnzYkFTovUyAioArbQGYUvfu3fXtt9/qzjvv1K5duzRx4kTFxMSoVq1a+ve//20zd8CAAapUqZLNVbMbNmzQH3/8UehVBgUGDhyo/fv36/vvv7f+t6RXGUiyXrFYkqsM8vLytHbtWvXt21f169e3jteoUUMDBw7U119/bd3f4sWL1blzZ91yyy3KysqyPqKjo5WXl3fV31SX1M6dO5WWlqZnnnnGbhmKwj5eVOCzzz5Tfn6++vfvb1Nf9erV1ahRI3311Vc28ytXrqwHHnjA+tzd3V3t2rXTr7/+etUa/f39JUlr1qwp9sqHwhw7dky33HKL3XiVKlW0Z88epaamFvnawYMH6/DhwzbHMn/+fHl5eemee+6xmVu5cmWbK4yaNGmiKlWqqGnTpmrfvr11vOD/izvudevWKTc3V88884zN3X6HDRsmPz8/rVy5UpK0bds2HTt2TMOGDbNZV3DQoEGFHrMk67izrjkIALh+/fv31/nz57VixQqdOXNGK1asKLJ3+vzzz+Xq6mq96rXAc889J8Mw7K5WjY6OVoMGDazPW7VqJT8/P+vPzdL0UkUp6c/ZAtfSwxw9elQbN27Ugw8+qNDQUJttxfVXkhQcHKxdu3bp0Ucf1YkTJzRr1iwNHDhQ1apV06uvvmqzrESzZs30l7/8RQsXLrSOZWdn69///rf+9re/yc/Pr9D36NGjhwICArRw4UIZhqGFCxfq/vvvL7auy5Wm/5UufR9Ur17d5j0qVaqkp556SmfPntWGDRskXep//f391b17d5v+MjIyUpUrV7brL69Vfn6+li1bpt69e6tNmzZ224s6RykpKUpNTdXAgQN17Ngxa33Z2dm6/fbbtXHjRusyZgUeffRRm+edO3fWsWPHSvSppypVqmjLli06fPhwKY7uUv/r5uamypUr2+0vOzvbZnmTK/Xv31+enp42V9uuWbNGWVlZNn8OCjz00EPW/3d1dVWbNm1kGIaGDh1q875NmjQp9s/MkSNHlJKSoiFDhiggIMA63qpVK3Xv3l2ff/65pEt/B6xbt059+/a1WfqjYcOGio2NLXTf9L+o6AhtAZhW27Zt9dlnn+nEiRPaunWrRo0apTNnzqhfv3766aefrPOqVq2qmJgYLV26VH/++aekS0sjuLm5qX///kXu/y9/+YvCwsK0YMECzZ8/X9WrV9dtt91W4voKmuGrfdxdutTAnzt3Tk2aNLHb1rRpU+Xn51vXg01NTdXq1asVFBRk8yhYm+ryG7Fdj19++UWS1KJFi1K9LjU1VYZhqFGjRnY17t27166+2rVr2zXAt9xyS4ludFGvXj0lJibq/fffV2BgoGJiYjRjxoyrrmdbwLhizTxJeuWVV3Ty5Ek1btxYLVu21PDhw/XDDz/YzOnevbtq1KhhbVrz8/P18ccfq0+fPnb/SCns+Pz9/RUSEmI3JqnY4/7tt98kye77xN3dXfXr17duL/hvw4YNbea5ubmpbt26he674GtxtX8wAgBQlIJ+ZMGCBfrss8+Ul5enfv36FTr3t99+U82aNe1+bhZ8ZL3gZ1mBKwNOybZfKE0vVZSS/pwtcC09TEE4Vdr+qkCNGjU0c+ZMHTlyRD///LOmTZumoKAgjR49Wh988IHN3EGDBiktLc360e9ly5bp3LlzxV60UKlSJd17771asGCBNm7cqEOHDpXqooXS9L/Spa95o0aNbEJyyf77IDU1VadOnVK1atXs+suzZ8+WWf979OhRnT59+pr6X0mKi4uzq+/9999XTk6OXX965fd0QYBYkh544sSJ+vHHHxUSEqJ27dpp7NixJbrgoSiPP/64GjdurNjYWNWuXVsPPvig3ZrRVapUUe/evW0uhJk/f75q1apV6L+Rrjw+f39/eXp62tzkrmD8Wvpf6dL3SUE4npmZqfPnz9v1v5J9T1yA/hcVXcW7HTgAp+Pu7q62bduqbdu2aty4seLj47V48WKNGTPGOueBBx7QihUrtGLFCt1555369NNP1aNHDwUFBRW774EDB2rmzJny9fXVgAED7BrK4oSFhUmSdu/erYiIiGs6tsLk5+ere/fuGjFiRKHbGzduXGbvdS3y8/NlsVi0atWqQm9YceVv9ou6qUVhgWphJk+erCFDhmj58uVau3atnnrqKY0fP17fffedzU3erlS1atVCG8QuXbrol19+se7v/fff11tvvaVZs2ZZrxhwdXXVwIED9d577+mdd97R5s2bdfjw4UKvMijq+K73uMtawdfiykYaAIDSGDhwoIYNG6b09HTFxsYWetPYa2G2n5uSY2uyWCxq3LixGjdurDvuuEONGjXS/Pnzba5uvP/++zVixAgtWLBAt956qxYsWKBbbrlFvXr1KnbfAwcO1KxZszR27FiFh4erWbNmJa7r8v63LOXn56tatWqFrqkq6ao9fXkruIp20qRJRfb9ZdkD9+/fX507d9bSpUu1du1aTZo0SRMmTNBnn31W5FWl0qX+9+LFizpz5ozNL0yqVaumlJQUrVmzRqtWrdKqVas0Z84cDR482OamcIMHD9bixYv1zTffqGXLlvr3v/+txx9/vNB/IxV2fGb7c0z/i4qOK20BVCgFH2M6cuSIzfidd94pX19fLViwQKtWrdKJEyeKvcqgwMCBA3XkyBH997//LdVVBpIUGxsrV1dXffTRR1edGxQUJG9vb/3888922/bt2ycXFxfrlZkNGjTQ2bNnFR0dXeijsCtRrkXBRxCvdrO3wl5nGIbq1atXaH2F3fH4aq722++WLVvqpZde0saNG7Vp0yb98ccfmjVrVrGvCQsLU1paWqHbAgICFB8fr48//liHDh1Sq1at7O4qPHjwYJ0+fVr/+c9/NH/+fAUFBSkmJqZUx1VaBTdNu/L7JDc3V2lpadbtBf/dv3+/zbyLFy/a3KzhcmlpaXJxcXF46A8AqNjuuusuubi46Lvvviu2d6pTp44OHz5sd0Xmvn37rNtLozS9VFF9RUl/zl6PgqUbSttfXW2ft9xyi13/W7NmTXXr1k2LFy9WRkaGvvjiC/Xr10/u7u7F7q9Tp04KDQ3V+vXrS93/Nm7cWE2aNNHy5ct19uzZq86vU6eOUlNT7ZYOuPL7oEGDBjp27Jg6duxYaH8ZHh5eqjqLEhQUJD8/v2vqf6VLVxoX1aNXqlSp1PUU1wPXqFFDjz/+uJYtW6a0tDRVrVpVr7/+erH7KwjVC+uB3d3d1bt3b73zzjv65Zdf9Mgjj+jDDz+06Sd79uypoKAgzZ8/X0uXLtW5c+dKfNPla1XUn0vp0vdJYGCgfHx8VK1aNXl6etr1v5J9T1yg4OtQ1E3pALMjtAVgSl999VWhv5EtWNPoyo/PeHl56a677tLnn3+umTNnysfHx+4Op4Vp0KCBpk6dqvHjx6tdu3alqjEkJETDhg3T2rVrNX36dLvt+fn5mjx5sn7//Xe5urqqR48eWr58uU2olpGRoQULFqhTp07Wj5v1799f3377rdasWWO3z5MnT+rixYulqrMorVu3Vr169TR16lSdPHnSZltxvw2/++675erqqnHjxtnNMwzD5i7PJeXj4yNJdnWcPn3a7nhbtmwpFxcX5eTkFLvPDh066MSJE3YfJbuyvsqVK6thw4Z2+2vVqpVatWql999/X59++qnuu+8+m/Vjy0N0dLTc3d01bdo0m6/tBx98oFOnTumOO+6QdOmXF1WrVtV7771n8/WZP39+kR8/2759u5o3b25dpgEAgGtRuXJlzZw5U2PHjlXv3r2LnNerVy/l5eXp7bffthl/6623ZLFYir1asDCl6aWK6itK+nP2egQFBalLly6aPXu2Dh48aLPtalcbbtmyRdnZ2XbjW7du1bFjxwr9+PigQYOUmZmpRx55RBcuXCjRRQsWi0XTpk3TmDFjrimQGzdunI4dO6aHHnqo0L507dq1WrFihaRL3wfp6elatGiRdfvFixc1ffp0Va5cWV27dpV0qf/Ny8vTq6++are/ixcv2p3La+Xi4qK+ffvqP//5j7Zt22a3vahzFBkZqQYNGugf//hHoWH10aNHr6keHx8fu2PLy8uzW2qhWrVqqlmzZon6X0l2x3Zl/+vi4qJWrVpJks0+3dzcdP/99+uTTz5RUlKSWrZsaZ1XXmrUqKGIiAjNnTvX5mvx448/au3atdYrx11dXRUdHa1ly5bZrPW7f/9+uzWyC2zfvl0Wi8X6dQEqGpZHAGBKTz75pM6dO6e77rpLYWFhys3N1TfffKNFixapbt26io+Pt3vNAw88oA8//FBr1qzRoEGDrA371Tz99NPXXOfkyZP1yy+/6KmnntJnn32mv/3tb7rlllt08OBBLV68WPv27bPepOq1117TF198oU6dOunxxx+Xm5ub3n33XeXk5GjixInWfQ4fPtx6E4khQ4YoMjJS2dnZ2r17t5YsWaIDBw6UyUd8XFxcNHPmTPXu3VsRERGKj49XjRo1tG/fPu3Zs6fQ0Fi6FHS/9tprGjVqlA4cOKC+ffvK19dXaWlpWrp0qR5++GE9//zzpaqlQYMGqlKlimbNmiVfX1/5+Pioffv22rVrlxISEnTvvfeqcePGunjxoubNmydXV1e7G4Jd6Y477pCbm5vWrVunhx9+2DrerFkzRUVFKTIyUgEBAdq2bZuWLFmihIQEu30MHjzYeiyFLY1Q1oKCgjRq1CiNGzdOPXv21J133qmff/5Z77zzjtq2bWutwd3dXWPHjtWTTz6p2267Tf3799eBAweUlJSkBg0a2F21ceHCBW3YsEGPP/54uR8DAODmFxcXd9U5vXv3Vrdu3fTiiy/qwIEDCg8P19q1a7V8+XI988wzNjcdK6mS9lIRERFydXXVhAkTdOrUKXl4eOi2225TtWrVSvRz9npNmzZNnTp1UuvWrfXwww+rXr16OnDggFauXKmUlJQiXzdv3jzNnz9fd911lyIjI+Xu7q69e/dq9uzZ8vT01P/7f//P7jX33HOPHn/8cS1fvlwhISHq0qVLiWrs06dPiS5wKMyAAQO0e/duvf7669q5c6fuv/9+1alTR8eOHdPq1auVnJxsXRf14Ycf1rvvvqshQ4Zo+/btqlu3rpYsWaLNmzdr6tSp1o/wd+3aVY888ojGjx+vlJQU9ejRQ5UqVVJqaqoWL16sf/7zn0Wun1xab7zxhtauXauuXbvq4YcfVtOmTXXkyBEtXrxYX3/9daFLfri4uOj9999XbGysmjdvrvj4eNWqVUt//PGHvvrqK/n5+ek///lPqWuJjIzUunXrNGXKFNWsWVP16tVTkyZNVLt2bfXr10/h4eGqXLmy1q1bp++//16TJ08udn/169dXixYttG7dOj344IPW8YceekjHjx/Xbbfdptq1a+u3337T9OnTFRERYXcV6uDBgzVt2jR99dVXmjBhQqmP6VpMmjRJsbGx6tChg4YOHarz589r+vTp8vf3t/k03NixY7V27Vp17NhRjz32mPUXQy1atCj0z9YXX3yhjh07qmrVqjfkOIAyZwCACa1atcp48MEHjbCwMKNy5cqGu7u70bBhQ+PJJ580MjIyCn3NxYsXjRo1ahiSjM8//7zQOWPGjDEkGUePHi32/SUZTzzxRIlqvXjxovH+++8bnTt3Nvz9/Y1KlSoZderUMeLj442dO3fazN2xY4cRExNjVK5c2fD29ja6detmfPPNN3b7PHPmjDFq1CijYcOGhru7uxEYGGjceuutxj/+8Q8jNzfXOq9OnTrGHXfcUaL6v/rqK0OS8dVXX9mMf/3110b37t0NX19fw8fHx2jVqpUxffp06/aCr9mVPv30U6NTp06Gj4+P4ePjY4SFhRlPPPGE8fPPP1vndO3a1WjevLnda+Pi4ow6derYjC1fvtxo1qyZ4ebmZkgy5syZY/z666/Ggw8+aDRo0MDw9PQ0AgICjG7duhnr1q2z22dh7rzzTuP222+3GXvttdeMdu3aGVWqVDG8vLyMsLAw4/XXX7f5uhY4cuSI4erqajRu3LjQ/Rd1fCU9L3PmzDEkGWlpaTbz3n77bSMsLMyoVKmSERwcbDz22GPGiRMn7PY3bdo0o06dOoaHh4fRrl07Y/PmzUZkZKTRs2dPm3mrVq0yJBmpqamFHgcAAEUp+Fn1/fffFzuvsJ99Z86cMZ599lmjZs2aRqVKlYxGjRoZkyZNMvLz823mFdV31alTx4iLi7MZK2kv9d577xn169c3XF1d7fqfkvycLWkPk5aWZu1bLvfjjz8ad911l1GlShXD09PTaNKkifHyyy/b7e9yP/zwgzF8+HCjdevWRkBAgOHm5mbUqFHDuPfee40dO3YU+bp7773XkGSMGDGi0O0FPeDixYuLff+ijrkoycnJRp8+fYxq1aoZbm5uRlBQkNG7d29j+fLlNvMyMjKM+Ph4IzAw0HB3dzdatmxp9/Uq8K9//cuIjIw0vLy8DF9fX6Nly5bGiBEjjMOHD1vnxMXFGT4+PiWuX5IxZswYm7HffvvNGDx4sBEUFGR4eHgY9evXN5544gkjJyfHMIyi++adO3cad999t1G1alXDw8PDqFOnjtG/f38jOTnZOqeof28U1vft27fP6NKli+Hl5WVIMuLi4oycnBxj+PDhRnh4uLU/Dw8PN955551Cv2ZXmjJlilG5cmXj3Llz1rElS5YYPXr0MKpVq2a4u7sboaGhxiOPPGIcOXKk0H00b97ccHFxMX7//Xe7bUUdX0nPS1F/ZtatW2d07NjR8PLyMvz8/IzevXsbP/30k93+kpOTjb/85S+Gu7u70aBBA+P99983nnvuOcPT09Nm3smTJw13d3fj/fffL/QYgYrAYhgOXNkdAIBytGnTJkVFRWnfvn1q1KhRqV+flZWlGjVqaPTo0Xr55ZfLocKylZ+fr6CgIN1999167733rON9+/aVxWLR0qVLHVgdAAAAytupU6dUv359TZw4UUOHDr2mffzlL39RQECAkpOTy7i68tG3b1/t2bNHqamp1rGpU6dq4sSJ+uWXX+Tl5eXA6oBrx5q2AICbVufOndWjRw+bj0yWRlJSkvLy8sr9BgzX4s8//7Rbd+3DDz/U8ePHFRUVZR3bu3evVqxYUegacQAAALi5+Pv7a8SIEZo0aZLdDeBKYtu2bUpJSdHgwYPLobrrd/78eZvnqamp+vzzz2363wsXLmjKlCl66aWXCGxRoXGlLQAAV/jyyy/1008/6eWXX1a3bt302WefObokO+vXr9ezzz6re++9V1WrVtWOHTv0wQcfqGnTptq+fftV7xwNAAAAFPjxxx+1fft2TZ48WVlZWfr111/l6enp6LLs1KhRQ0OGDFH9+vX122+/aebMmcrJydHOnTuv6ZN1gJlxIzIAAK7wyiuv6JtvvlHHjh01ffp0R5dTqLp16yokJETTpk3T8ePHFRAQoMGDB+vNN98ksAUAAECpLFmyRK+88oqaNGmijz/+2JSBrST17NlTH3/8sdLT0+Xh4aEOHTrojTfeILDFTYkrbQEAAAAAAADARFjTFgAAAAAAAABMhNAWAAAAAAAAAEyENW3LUH5+vg4fPixfX19ZLBZHlwMAAIByYhiGzpw5o5o1a8rF5fqvg6CPBAAAcA4l7SMJbcvQ4cOHFRIS4ugyAAAAcIMcOnRItWvXvu790EcCAAA4l6v1kYS2ZcjX11fSpS+6n5+fg6sBAABAeTl9+rRCQkKs/d/1oo8EAABwDiXtIwlty1DBR9n8/PxotgEAAJxAWS1lQB8JAADgXK7WR3IjMgAAAAAAAAAwEUJbAAAAAAAAADARQlsAAAAAAAAAMBFCWwAAAAAAAAAwEUJbAAAAAAAAADARQlsAAAAAAAAAMBFCWwAAAAAAAAAwEUJbAAAAAAAAADARQlsAAAAAAAAAMBFCWwAAAAAAAAAwEUJbAAAAAAAAADARQlsAAAAAAAAAMBFCWwAAAAAAAAAwEUJbAAAAAAAAADARQlsAAAAAAAAAMBE3RxcAOLuDBw8qKyvL0WXgCoGBgQoNDXV0GQAAAAAAwAkR2gIOdPDgQTVt2lTnzp1zdCm4gre3t/bu3UtwCwAAAAAAbjhCW8CBsrKydO7cOc19e7rCGjVydDn4P/tSUxWX8KSysrIIbQEAAAAAwA1HaHuFFStW6LnnnlN+fr5eeOEFPfTQQ44uCU4grFEjtW7V0tFlAAAAAAAAwAQIbS9z8eJFJSYm6quvvpK/v78iIyN11113qWrVqo4uDQAAAAAAAICTcHF0AWaydetWNW/eXLVq1VLlypUVGxurtWvXOrosAAAAAAAAAE7kpgptN27cqN69e6tmzZqyWCxatmyZ3ZwZM2aobt268vT0VPv27bV161brtsOHD6tWrVrW57Vq1dIff/xxI0oHAAAAAAAAAEk3WWibnZ2t8PBwzZgxo9DtixYtUmJiosaMGaMdO3YoPDxcMTExyszMvMGVAgAAAAAAAEDhbqo1bWNjYxUbG1vk9ilTpmjYsGGKj4+XJM2aNUsrV67U7NmzNXLkSNWsWdPmyto//vhD7dq1K3J/OTk5ysnJsT4/ffp0GRwFAAAAbnb0kQAAACjOTXWlbXFyc3O1fft2RUdHW8dcXFwUHR2tb7/9VpLUrl07/fjjj/rjjz909uxZrVq1SjExMUXuc/z48fL397c+QkJCyv04AAAAUPHRRwIAAKA4ThPaZmVlKS8vT8HBwTbjwcHBSk9PlyS5ublp8uTJ6tatmyIiIvTcc8+patWqRe5z1KhROnXqlPVx6NChcj0GAAAA3BzoIwEAAFCcm2p5hLJw55136s477yzRXA8PD3l4eJRzRQAAALjZ0EcCAACgOE5zpW1gYKBcXV2VkZFhM56RkaHq1as7qCoAAAAAAAAAsOU0oa27u7siIyOVnJxsHcvPz1dycrI6dOjgwMoAAAAAAAAA4H9uquURzp49q/3791ufp6WlKSUlRQEBAQoNDVViYqLi4uLUpk0btWvXTlOnTlV2drbi4+MdWDUAAAAAAAAA/M9NFdpu27ZN3bp1sz5PTEyUJMXFxSkpKUkDBgzQ0aNHNXr0aKWnpysiIkKrV6+2uzkZAAAAAAAAADjKTRXaRkVFyTCMYuckJCQoISHhBlUEAAAAAAAAAKXjNGvaAgAAAAAAAEBFQGgLAAAAAAAAACZyUy2PAAAAADiTgwcPKisry9Fl4AqBgYEKDQ0t1/fg3JvTjTj3AADnQGgLAAAAVEAHDx5UWFiYzp8/7+hScAUvLy/t27ev3MI7zr15lfe5BwA4D0JbAAAAoALKysrS+fPndV+/v6tatWBHl4P/k5mZoYVL5ikrK6vcgjvOvTndiHMPAHAehLYAAABABVatWrBq1wxxdBlwAM49AAA3L25EBgAAAAAAAAAmQmgLAAAAAAAAACZCaAsAAAAAAAAAJkJoCwAAAAAAAAAmQmgLAAAAAAAAACZCaAsAAAAAAAAAJkJoCwAAAAAAAAAmQmgLAAAAAAAAACZCaAsAAAAAAAAAJkJoCwAAAAAAAAAmQmgLAAAAAAAAACZCaAsAAAAAAAAAJkJoCwAAAAAAAAAmQmgLAAAAAAAAACZCaAsAAAAAAAAAJkJoCwAAAAAAAAAmQmgLAAAAAAAAACZCaAsAAAAAAAAAJkJoCwAAAAAAAAAmQmgLAAAAAAAAACZCaAsAAAAAAAAAJkJoCwAAAAAAAAAmQmgLAAAAAAAAACZCaAsAAAAAAAAAJkJoCwAAAAAAAAAmQmgLAAAAAAAAACZCaAsAAAAAAAAAJkJoCwAAAAAAAAAmQmgLAAAAAAAAACZCaAsAAAAAAAAAJkJoCwAAAAAAAAAmQmgLAAAAAAAAACZCaAsAAAAAAAAAJkJoCwAAAAAAAAAmQmgLAAAAAAAAACZCaAsAAAAAAAAAJkJoCwAAAAAAAAAmQmgLAAAAAAAAACZCaAsAAAAAAAAAJkJoCwAAAAAAAAAmQmgLAAAAAAAAACZCaAsAAAAAAAAAJkJoCwAAAAAAAAAmQmgLAAAAAAAAACZCaAsAAAAAAAAAJkJoCwAAAAAAAAAmQmgLAAAAAAAAACZCaAsAAAAAAAAAJkJoCwAAAAAAAAAmQmgLAAAAAAAAACbi5ugCzOauu+7S+vXrdfvtt2vJkiWOLgfATezgwYPKyspydBm4QmBgoEJDQx1dBgAAAADAiRHaXuHpp5/Wgw8+qLlz5zq6FAA3sYMHD6pp06Y6d+6co0vBFby9vbV3716CWwAAAACAwxDaXiEqKkrr1693dBkAbnJZWVk6d+6c5r49XWGNGjm6HPyffampikt4UllZWYS2AAAAAACHqVCh7caNGzVp0iRt375dR44c0dKlS9W3b1+bOTNmzNCkSZOUnp6u8PBwTZ8+Xe3atXNMwQBwFWGNGql1q5aOLgMAAAAAAJhIhboRWXZ2tsLDwzVjxoxCty9atEiJiYkaM2aMduzYofDwcMXExCgzM9M6JyIiQi1atLB7HD58+EYdBgAAAAAAAAAUqUJdaRsbG6vY2Ngit0+ZMkXDhg1TfHy8JGnWrFlauXKlZs+erZEjR0qSUlJSyqyenJwc5eTkWJ+fPn26zPYNAACAmxd9JAAAAIpToa60LU5ubq62b9+u6Oho65iLi4uio6P17bfflst7jh8/Xv7+/tZHSEhIubwPAAAAbi70kQAAACjOTRPaZmVlKS8vT8HBwTbjwcHBSk9PL/F+oqOjde+99+rzzz9X7dq1iw18R40apVOnTlkfhw4duub6AQAA4DzoIwEAAFCcCrU8wo2wbt26Es/18PCQh4dHOVYDAACAmxF9JAAAAIpz01xpGxgYKFdXV2VkZNiMZ2RkqHr16g6qCgAAAAAAAABK56YJbd3d3RUZGank5GTrWH5+vpKTk9WhQwcHVgYAAAAAAAAAJVehlkc4e/as9u/fb32elpamlJQUBQQEKDQ0VImJiYqLi1ObNm3Url07TZ06VdnZ2YqPj3dg1QAAAAAAAABQchUqtN22bZu6detmfZ6YmChJiouLU1JSkgYMGKCjR49q9OjRSk9PV0REhFavXm13czIAAAAAAAAAMKsKFdpGRUXJMIxi5yQkJCghIeEGVQQAAAAAAAAAZeumWdMWAAAAAAAAAG4GhLYAAAAAAAAAYCKEtgAAAAAAAABgIoS2AAAAAAAAAGAihLYAAAAAAAAAYCKEtgAAAAAAAABgIoS2AAAAAAAAAGAibo4uAAAAAAAAlNzBgweVlZXl6DJwhcDAQIWGhjq6DAA3CUJbAAAAAAAqiIMHDyosLEznz593dCm4gpeXl/bt20dwC6BMENoCAAAAAFBBZGVl6fz587qv399VrVqwo8vB/8nMzNDCJfOUlZVFaAugTBDaAgAAAABQwVSrFqzaNUMcXQYAoJxwIzIAAAAAAAAAMBFCWwAAAAAAAAAwEUJbAAAAAAAAADARQlsAAAAAAAAAMBFCWwAAAAAAAAAwEUJbAAAAAAAAADARQlsAAAAAAAAAMBFCWwAAAAAAAAAwEUJbAAAAAAAAADARQlsAAAAAAAAAMBFCWwAAAAAAAAAwEUJbAAAAAAAAADARQlsAAAAAAAAAMBFCWwAAAAAAAAAwEUJbAAAAAAAAADARQlsAAAAAAAAAMBFCWwAAAAAAAAAwEUJbAAAAAAAAADARQlsAAAAAAAAAMBFCWwAAAAAAAAAwEUJbAAAAAAAAADARQlsAAAAAAAAAMBFCWwAAAAAAAAAwEUJbAAAAAAAAADARQlsAAAAAAAAAMBFCWwAAAAAAAAAwEUJbAAAAAAAAADARQlsAAAAAAAAAMBFCWwAAAAAAAAAwEUJbAAAAAAAAADARQlsAAAAAAAAAMBFCWwAAAAAAAAAwEUJbAAAAAAAAADARQlsAAAAAAAAAMBFCWwAAAAAAAAAwEUJbAAAAAAAAADARQlsAAAAAAAAAMBFCWwAAAAAAAAAwEUJbAAAAAAAAADARQlsAAAAAAAAAMBFCWwAAAAAAAAAwEUJbAAAAAAAAADARQlsAAAAAAAAAMBFCWwAAAAAAAAAwEUJbAAAAAAAAADARQtvLHDp0SFFRUWrWrJlatWqlxYsXO7okAAAAAAAAAE7GzdEFmImbm5umTp2qiIgIpaenKzIyUr169ZKPj4+jSwMAAAAAAADgJAhtL1OjRg3VqFFDklS9enUFBgbq+PHjhLYAAAAAAAAAbpgKtTzCxo0b1bt3b9WsWVMWi0XLli2zmzNjxgzVrVtXnp6eat++vbZu3XpN77V9+3bl5eUpJCTkOqsGAAAAAAAAgJKrUKFtdna2wsPDNWPGjEK3L1q0SImJiRozZox27Nih8PBwxcTEKDMz0zonIiJCLVq0sHscPnzYOuf48eMaPHiw/vWvf5X7MQEAAAAAAADA5SrU8gixsbGKjY0tcvuUKVM0bNgwxcfHS5JmzZqllStXavbs2Ro5cqQkKSUlpdj3yMnJUd++fTVy5EjdeuutV52bk5NjfX769OkSHgkAAACcGX0kAAAAilOhrrQtTm5urrZv367o6GjrmIuLi6Kjo/Xtt9+WaB+GYWjIkCG67bbb9Pe///2q88ePHy9/f3/rg6UUAAAAUBL0kQAAACjOTRPaZmVlKS8vT8HBwTbjwcHBSk9PL9E+Nm/erEWLFmnZsmWKiIhQRESEdu/eXeT8UaNG6dSpU9bHoUOHrusYAAAA4BzoIwEAAFCcCrU8Qnnr1KmT8vPzSzzfw8NDHh4e5VgRAAAAbkb0kQAAACjOTXOlbWBgoFxdXZWRkWEznpGRoerVqzuoKgAAAAAAAAAonZsmtHV3d1dkZKSSk5OtY/n5+UpOTlaHDh0cWBkAAAAAAAAAlFyFWh7h7Nmz2r9/v/V5WlqaUlJSFBAQoNDQUCUmJiouLk5t2rRRu3btNHXqVGVnZys+Pt6BVQMAAAAAAABAyVWo0Hbbtm3q1q2b9XliYqIkKS4uTklJSRowYICOHj2q0aNHKz09XREREVq9erXdzckAAAAAAAAAwKwqVGgbFRUlwzCKnZOQkKCEhIQbVBEAAAAAAAAAlK2bZk1bAAAAAAAAALgZENoCAAAAAAAAgIkQ2gIAAAAAAACAiRDaAgAAAAAAAICJENoCAAAAAAAAgIkQ2gIAAAAAAACAiRDaAgAAAAAAAICJENoCAAAAAAAAgIkQ2gIAAAAAAACAiRDaAgAAAAAAAICJENoCAAAAAAAAgIkQ2gIAAAAAAACAiRDaAgAAAAAAAICJENoCAAAAAAAAgIkQ2gIAAAAAAACAiRDaAgAAAAAAAICJuJVmcv369Us079dff72mYgAAAAAAAADA2ZUqtD1w4IDq1KmjgQMHqlq1auVVEwAAAAAAAAA4rVKFtosWLdLs2bM1ZcoUxcbG6sEHH1SvXr3k4sIqCwAAAAAAAABQFkqVtt57771atWqV9u/fr8jISD377LMKCQnRyJEjlZqaWl41AgAAAAAAAIDTuKZLZGvVqqUXX3xRqampWrBggbZs2aKwsDCdOHGirOsDAAAAAAAAAKdSquURLvfnn39qyZIlmj17trZs2aJ7771X3t7eZVkbAAAAAAAAADidUoe2W7Zs0QcffKBPPvlE9evX14MPPqhPP/1Ut9xyS3nUBwAAAAAAAABOpVShbfPmzZWZmamBAwdqw4YNCg8PL6+6AAAAAAAAAMAplSq03bt3r3x8fPThhx9q3rx5Rc47fvz4dRcGAAAAAAAAAM6oVKHtnDlzyqsOAAAAAAAAAIBKGdo+8MADcnV1La9aAAAAAAAAAMDpuZRmcu3atTVy5EilpqaWVz0AAAAAAAAA4NRKFdo+/vjjWrJkicLCwtS5c2clJSXp3Llz5VUbAAAAAAAAADidUoW2L7/8svbv36/k5GTVr19fCQkJqlGjhoYNG6YtW7aUV40AAAAAAAAA4DRKtaZtgaioKEVFRWnGjBlauHChkpKS1KFDBzVt2lRDhw5VYmJiWdcJAABwUzh48KCysrIcXQauEBgYqNDQUEeXAQAAAEi6xtC2QOXKlfXQQw/poYce0sqVKzV48GANHz6c0BYAAKAQBw8eVNOmTVleyoS8vb21d+9eglsAAACYwnWFtufOndMnn3yiOXPm6Ouvv1aDBg00fPjwsqoNAADgppKVlaVz585p7tvTFdaokaPLwf/Zl5qquIQnlZWVRWgLAAAAU7im0Pabb77R7NmztXjxYl28eFH9+vXTq6++qi5dupR1fQAAADedsEaN1LpVS0eXAQAAAMCkShXaTpw4UXPmzNF///tftWnTRpMmTdL9998vX1/f8qoPAAAAAAAAAJxKqULbSZMm6e9//7sWL16sFi1alFdNAAAAAAAAAOC0XEoz+dNPP9Vtt91mE9h++OGHqlevnqpVq6aHH35YOTk5ZV4kAAAAAAAAADiLUoW248eP1549e6zPd+/eraFDhyo6OlojR47Uf/7zH40fP77MiwQAAAAAAAAAZ1Gq0DYlJUW333679fnChQvVvn17vffee0pMTNS0adP0ySeflHmRAAAAAAAAAOAsShXanjhxQsHBwdbnGzZsUGxsrPV527ZtdejQobKrDgAAAAAAAACcTKlC2+DgYKWlpUmScnNztWPHDv31r3+1bj9z5owqVapUthUCAAAAAAAAgBMpVWjbq1cvjRw5Ups2bdKoUaPk7e2tzp07W7f/8MMPatCgQZkXCQAAAAAAAADOwq00k1999VXdfffd6tq1qypXrqy5c+fK3d3dun327Nnq0aNHmRcJAAAAAAAAAM6iVKFtYGCgNm7cqFOnTqly5cpydXW12b548WJVrly5TAsEAAAAAAAAAGdSqtC2gL+/f6HjAQEB11UMAAAAAAAAADi7Uq1pCwAAAAAAAAAoX4S2AAAAAAAAAGAihLYAAAAAAAAAYCKEtgAAAAAAAABgIoS2AAAAAAAAAGAihLYAAAAAAAAAYCKEtgAAAAAAAABgIoS2AAAAAAAAAGAihLYAAAAAAAAAYCKEtgAAAAAAAABgIoS2AAAAAAAAAGAihLYAAAAAAAAAYCKEtgAAAAAAAABgIoS2AAAAAAAAAGAihLYAAAAAAAAAYCKEtgAAAAAAAABgIoS2AAAAAAAAAGAihLYAAAAAAAAAYCKEtgAAAAAAAABgIoS2AAAAAAAAAGAihLYAAAAAAAAAYCKEtgAAAAAAAABgIoS2AAAAAAAAAGAihLYAAAAAAAAAYCKEtpc5efKk2rRpo4iICLVo0ULvvfeeo0sCAAAAAAAA4GTcHF2Amfj6+mrjxo3y9vZWdna2WrRoobvvvltVq1Z1dGkAAAAAAAAAnARX2l7G1dVV3t7ekqScnBwZhiHDMBxcFQAAAAAAAABnUqFC240bN6p3796qWbOmLBaLli1bZjdnxowZqlu3rjw9PdW+fXtt3bq1VO9x8uRJhYeHq3bt2ho+fLgCAwPLqHoAAAAAAAAAuLoKFdpmZ2crPDxcM2bMKHT7okWLlJiYqDFjxmjHjh0KDw9XTEyMMjMzrXMK1qu98nH48GFJUpUqVbRr1y6lpaVpwYIFysjIuCHHBgAAAAAAAABSBVvTNjY2VrGxsUVunzJlioYNG6b4+HhJ0qxZs7Ry5UrNnj1bI0eOlCSlpKSU6L2Cg4MVHh6uTZs2qV+/foXOycnJUU5OjvX56dOnS3gkAAAAcGb0kQAAAChOhbrStji5ubnavn27oqOjrWMuLi6Kjo7Wt99+W6J9ZGRk6MyZM5KkU6dOaePGjWrSpEmR88ePHy9/f3/rIyQk5PoOAgAAAE6BPhIAAADFuWlC26ysLOXl5Sk4ONhmPDg4WOnp6SXax2+//abOnTsrPDxcnTt31pNPPqmWLVsWOX/UqFE6deqU9XHo0KHrOgYAAAA4B/pIAAAAFKdCLY9Q3tq1a1fi5RMkycPDQx4eHuVXEAAAAG5K9JEAAAAozk1zpW1gYKBcXV3tbhyWkZGh6tWrO6gqAAAAAAAAACidmya0dXd3V2RkpJKTk61j+fn5Sk5OVocOHRxYGQAAAAAAAACUXIVaHuHs2bPav3+/9XlaWppSUlIUEBCg0NBQJSYmKi4uTm3atFG7du00depUZWdnKz4+3oFVAwAAAAAAAEDJVajQdtu2berWrZv1eWJioiQpLi5OSUlJGjBggI4eParRo0crPT1dERERWr16td3NyQAAAAAAAADArCpUaBsVFSXDMIqdk5CQoISEhBtUEQAAAAAAAACUrZtmTVsAAAAAAAAAuBkQ2gIAAAAAAACAiRDaAgAAAAAAAICJENpeYcaMGapbt648PT3Vvn17bd261dElAQAAAAAAAHAihLaXWbRokRITEzVmzBjt2LFD4eHhiomJUWZmpqNLAwAAAAAAAOAkCG0vM2XKFA0bNkzx8fFq1qyZZs2aJW9vb82ePdvRpQEAAAAAAABwEoS2/yc3N1fbt29XdHS0dczFxUXR0dH69ttvHVgZAAAAAAAAAGfi5ugCzCIrK0t5eXkKDg62GQ8ODta+ffsKfU1OTo5ycnKsz0+fPi1JSklJUeXKlUv83keOHNHJkydLXzTKVZUqVVSjRo1yfY+9e/dKklZ/+aX2paaW63uh5A4cOiTpf+enPHDuzelGnPsC/N1vTuX9dz9/9s3pWv7snz179rresyz6yIJ69/38kzKPZlxXPSg7x48fk3Rj+gjOvbnciHN/+f45/+Zyo86/RB9pVjciQ+Dcm1Npz31J+0iLYRjGtRZ1Mzl8+LBq1aqlb775Rh06dLCOjxgxQhs2bNCWLVvsXjN27FiNGzfuRpYJAAAAEzl16pT8/PxK/Tr6SAAAAOd2tT6S0Pb/5ObmytvbW0uWLFHfvn2t43FxcTp58qSWL19u95rCrpAICQnRhg0bSnWFxAMPPKBxL4xQ3ZCQ6z4OlI0Dhw5pzISJ+uijj9S0adNyfS9+U2ZO/JbUed2oq+z5u998btTf/fzZN6druUKia9eu1xzalkUfKfH9ZFb0Ec7rRpx7ifNvVjeyj+xxey8FBFQt1/dCyR0/fkxrkz8v1z6Sc29O13LuS9pHsjzC/3F3d1dkZKSSk5OtoW1+fr6Sk5OVkJBQ6Gs8PDzk4eFhNx4REVHq5r3nbbepdauWpa4b5WPHD7s1ZsJENW3aVK1bt3Z0OQBuUvzdby783Y/SKFjO4FqVZR8JAHA+YU2aqXZNfvlvFr8fPqS1yZ/fkD6Sc28u13LuS9pHEtpeJjExUXFxcWrTpo3atWunqVOnKjs7W/Hx8Y4uDQAAAAAAAICTILS9zIABA3T06FGNHj1a6enpioiI0OrVq+1uTgYAAAAAAAAA5YXQ9goJCQlFLocAAAAAAAAAAOXNxdEFAAAAAAAAAAD+h9AWAAAAAAAAAEyE0BYAAAAAAAAATITQ9gonT55UmzZtFBERoRYtWui9995zdEkAAAAAAAAAnAg3IruCr6+vNm7cKG9vb2VnZ6tFixa6++67VbVqVUeXBgAAAAAAAMAJcKXtFVxdXeXt7S1JysnJkWEYMgzDwVUBAAAAAAAAcBYVKrTduHGjevfurZo1a8pisWjZsmWFzpsxY4bq1q0rT09PtW/fXlu3bi3V+5w8eVLh4eGqXbu2hg8frsDAwDKoHgAAAAAAAACurkKFttnZ2QoPD9eMGTOKnLNo0SIlJiZqzJgx2rFjh8LDwxUTE6PMzEzrnIL1aq98HD58WJJUpUoV7dq1S2lpaVqwYIEyMjLK/dgAAAAAAAAAQKpga9rGxsYqNja22DlTpkzRsGHDFB8fL0maNWuWVq5cqdmzZ2vkyJGSpJSUlBK9X3BwsMLDw7Vp0yb169fPbntOTo5ycnKsz0+fPl3CIwEAAIAzo48EAABAcSrUlbZXk5ubq+3btys6Oto65uLioujoaH377bcl2kdGRobOnDkjSTp16pQ2btyoJk2aFDp3/Pjx8vf3tz5CQkKu/yAAAABw06OPBAAAQHFuqtA2KytLeXl5Cg4OthkPDg5Wenp6ifbx22+/qXPnzgoPD1fnzp315JNPqmXLloXOHTVqlE6dOmV9HDp06LqPAQAAADc/+kgAAAAUx+HLI4wcOVITJkwods7evXsVFhZ2Q+pp165diZdP8PDwkIeHR/kWBAAAgJsOfSQAAACK4/DQ9rnnntOQIUOKnVO/fv0S7SswMFCurq52Nw7LyMhQ9erVr7VEAAAAAAAAALhhHB7aBgUFKSgoqEz25e7ursjISCUnJ6tv376SpPz8fCUnJyshIaFM3gMAAAAAAAAAypPDQ9vSOHv2rPbv3299npaWppSUFAUEBCg0NFSSlJiYqLi4OLVp00bt2rXT1KlTlZ2drfj4eEeVDQAAAAAAAAAlVqFC223btqlbt27W54mJiZKkuLg4JSUlSZIGDBigo0ePavTo0UpPT1dERIRWr15td3MyAAAAAAAAADCjChXaRkVFyTCMq85LSEhgOQQAAAAAAAAAFZKLowsAAAAAAAAAAPwPoS0AAAAAAAAAmAihLQAAAAAAAACYCKEtAAAAAAAAAJgIoS0AAAAAAAAAmAihLQAAAAAAAACYCKEtAAAAAAAAAJgIoS0AAAAAAAAAmAihLQAAAAAAAACYCKEtAAAAAAAAAJgIoS0AAAAAAAAAmAihLQAAAAAAAACYCKEtAAAAAAAAAJgIoS0AAAAAAAAAmAihLQAAAAAAAACYCKEtAAAAAAAAAJgIoS0AAAAAAAAAmAihLQAAAAAAAACYCKEtAAAAAAAAAJgIoS0AAAAAAAAAmAihLQAAAAAAAACYCKEtAAAAAAAAAJgIoS0AAAAAAAAAmAihLQAAAAAAAACYCKEtAAAAAAAAAJgIoS0AAAAAAAAAmAihLQAAAAAAAACYCKEtAAAAAAAAAJgIoS0AAAAAAAAAmAihLQAAAAAAAACYCKEtAAAAAAAAAJgIoS0AAAAAAAAAmAihLQAAAAAAAACYCKEtAAAAAAAAAJgIoS0AAAAAAAAAmAihLQAAAAAAAACYCKEtAAAAAAAAAJgIoS0AAAAAAAAAmAihLQAAAAAAAACYCKEtAAAAAAAAAJgIoS0AAAAAAAAAmAihLQAAAAAAAACYCKEtAAAAAAAAAJgIoS0AAAAAAAAAmAihLQAAAAAAAACYCKEtAAAAAAAAAJgIoS0AAAAAAAAAmAihLQAAAAAAAACYiJujC8Al+1JTHV0CLsP5AAAAAAAAgKMQ2jpYYGCgvL29FZfwpKNLwRW8vb0VGBjo6DIAAAAAAADgZAhtHSw0NFR79+5VVlaWo0vBFQIDAxUaGuroMgAAAAAAAOBkCG1NIDQ0lHAQAAAAAAAAgCRuRAYAAAAAAAAApkJoCwAAAAAAAAAmQmgLAAAAAAAAACZCaAsAAAAAAAAAJkJoCwAAAAAAAAAmQmgLAAAAAAAAACZCaAsAAAAAAAAAJkJoCwAAAAAAAAAmQmgLAAAAAAAAACZCaAsAAAAAAAAAJkJoCwAAAAAAAAAmQmgLAAAAAAAAACZCaAsAAAAAAAAAJkJoCwAAAAAAAAAmQmgLAAAAAAAAACZCaAsAAAAAAAAAJkJoCwAAAAAAAAAmQmh7hUOHDikqKkrNmjVTq1attHjxYkeXBAAAAAAAAMCJuDm6ALNxc3PT1KlTFRERofT0dEVGRqpXr17y8fFxdGkAAAAAAAAAnACh7RVq1KihGjVqSJKqV6+uwMBAHT9+nNAWAAAAAAAAwA1RoZZH2Lhxo3r37q2aNWvKYrFo2bJlhc6bMWOG6tatK09PT7Vv315bt269pvfbvn278vLyFBISch1VAwAAAAAAAEDJVajQNjs7W+Hh4ZoxY0aRcxYtWqTExESNGTNGO3bsUHh4uGJiYpSZmWmdExERoRYtWtg9Dh8+bJ1z/PhxDR48WP/617/K9ZgAAAAAAAAA4HIVanmE2NhYxcbGFjtnypQpGjZsmOLj4yVJs2bN0sqVKzV79myNHDlSkpSSklLsPnJyctS3b1+NHDlSt956a7HzcnJyrM9Pnz5dwiMBAACAM6OPBAAAQHEq1JW2V5Obm6vt27crOjraOubi4qLo6Gh9++23JdqHYRgaMmSIbrvtNv39738vdu748ePl7+9vfbCMAgAAAEqCPhIAAADFualC26ysLOXl5Sk4ONhmPDg4WOnp6SXax+bNm7Vo0SItW7ZMERERioiI0O7duwudO2rUKJ06dcr6OHTo0HUfAwAAAG5+9JEAAAAojsOXRxg5cqQmTJhQ7Jy9e/cqLCzshtTTqVMn5efnl2iuh4eHPDw8yrkiAAAA3GzoIwEAAFAch4e2zz33nIYMGVLsnPr165doX4GBgXJ1dVVGRobNeEZGhqpXr36tJQIAAAAAAADADePw0DYoKEhBQUFlsi93d3dFRkYqOTlZffv2lSTl5+crOTlZCQkJZfIeAAAAAAAAAFCeHB7alsbZs2e1f/9+6/O0tDSlpKQoICBAoaGhkqTExETFxcWpTZs2ateunaZOnars7GzFx8c7qmwAAAAAAAAAKLEKFdpu27ZN3bp1sz5PTEyUJMXFxSkpKUmSNGDAAB09elSjR49Wenq6IiIitHr1arubkwEAAAAAAFREmZkZV5+EG4bzgfJQoULbqKgoGYZx1XkJCQkshwAAAAAAAG4qgYGB8vLy0sIl8xxdCq7g5eWlwMDAcn8fAmJzKc/zUaFCWwAAAAAAAGcVGhqqffv2KSsry9Gl4AqBgYHWpTvLa/8E9uZUXoE9oS0AAAAAAEAFERoaWq7hIMyJwN68yiuwJ7QFAAAAAAAATI7A3rm4OLoAAAAAAAAAAMD/ENoCAAAAAAAAgIkQ2gIAAAAAAACAiRDaAgAAAAAAAICJENoCAAAAAAAAgIkQ2gIAAAAAAACAiRDaAgAAAAAAAICJENoCAAAAAAAAgIkQ2gIAAAAAAACAiRDaAgAAAAAAAICJuDm6AAAAnNW+1FRHl4DLcD4AAAAAmAWhLQAAN1hgYKC8vb0Vl/Cko0vBFby9vRUYGOjoMgAAAAA4OUJbAABusNDQUO3du1dZWVmOLgVXCAwMVGhoqKPLAAAAAODkCG0BAHCA0NBQwkEAAAAAQKG4ERkAAAAAAAAAmAihLQAAAAAAAACYCKEtAAAAAAAAAJgIoS0AAAAAAAAAmAihLQAAAAAAAACYCKEtAAAAAAAAAJgIoS0AAAAAAAAAmAihLQAAAAAAAACYCKEtAAAAAAAAAJgIoS0AAAAAAAAAmAihLQAAAAAAAACYCKEtAAAAAAAAAJgIoS0AAAAAAAAAmAihLQAAAAAAAACYCKEtAAAAAAAAAJiIm6MLuJkYhiFJOn36tIMrAQAAQHkq6PcK+r/rRR8JAADgHEraRxLalqEzZ85IkkJCQhxcCQAAAG6EM2fOyN/fv0z2I9FHAgAAOIur9ZEWo6wuD4Dy8/N1+PBh+fr6ymKxOLqcG+706dMKCQnRoUOH5Ofn5+hycINx/p0X5965cf6dl7Ofe8MwdObMGdWsWVMuLte/4hh9pHN/Pzkzzr1z4/w7N86/83L2c1/SPpIrbcuQi4uLateu7egyHM7Pz88p/9DhEs6/8+LcOzfOv/Ny5nNfFlfYFqCPvMSZv5+cHefeuXH+nRvn33k587kvSR/JjcgAAAAAAAAAwEQIbQEAAAAAAADARAhtUWY8PDw0ZswYeXh4OLoUOADn33lx7p0b5995ce5Rlvh+cl6ce+fG+XdunH/nxbkvGW5EBgAAAAAAAAAmwpW2AAAAAAAAAGAihLYAAAAAAAAAYCKEtgAAAAAAAABgIoS2KDMzZsxQ3bp15enpqfbt22vr1q2OLgk3wMaNG9W7d2/VrFlTFotFy5Ytc3RJuEHGjx+vtm3bytfXV9WqVVPfvn31888/O7os3AAzZ85Uq1at5OfnJz8/P3Xo0EGrVq1ydFlwkDfffFMWi0XPPPOMo0tBBUUP6bzoI50XfaTzoo/E5egji0doizKxaNEiJSYmasyYMdqxY4fCw8MVExOjzMxMR5eGcpadna3w8HDNmDHD0aXgBtuwYYOeeOIJfffdd/riiy904cIF9ejRQ9nZ2Y4uDeWsdu3aevPNN7V9+3Zt27ZNt912m/r06aM9e/Y4ujTcYN9//73effddtWrVytGloIKih3Ru9JHOiz7SedFHogB95NVZDMMwHF0EKr727durbdu2evvttyVJ+fn5CgkJ0ZNPPqmRI0c6uDrcKBaLRUuXLlXfvn0dXQoc4OjRo6pWrZo2bNigLl26OLoc3GABAQGaNGmShg4d6uhScIOcPXtWrVu31jvvvKPXXntNERERmjp1qqPLQgVDD4kC9JHOjT7SudFHOh/6yJLhSltct9zcXG3fvl3R0dHWMRcXF0VHR+vbb791YGUAbqRTp05JutR0wXnk5eVp4cKFys7OVocOHRxdDm6gJ554QnfccYfNz3+gNOghARSgj3RO9JHOiz6yZNwcXQAqvqysLOXl5Sk4ONhmPDg4WPv27XNQVQBupPz8fD3zzDPq2LGjWrRo4ehycAPs3r1bHTp00J9//qnKlStr6dKlatasmaPLwg2ycOFC7dixQ99//72jS0EFRg8JQKKPdEb0kc6NPrLkCG0BANftiSee0I8//qivv/7a0aXgBmnSpIlSUlJ06tQpLVmyRHFxcdqwYQMNtxM4dOiQnn76aX3xxRfy9PR0dDkAgAqOPtL50Ec6L/rI0iG0xXULDAyUq6urMjIybMYzMjJUvXp1B1UF4EZJSEjQihUrtHHjRtWuXdvR5eAGcXd3V8OGDSVJkZGR+v777/XPf/5T7777roMrQ3nbvn27MjMz1bp1a+tYXl6eNm7cqLfffls5OTlydXV1YIWoKOghAdBHOif6SOdFH1k6rGmL6+bu7q7IyEglJydbx/Lz85WcnMy6NMBNzDAMJSQkaOnSpfryyy9Vr149R5cEB8rPz1dOTo6jy8ANcPvtt2v37t1KSUmxPtq0aaNBgwYpJSWFRhslRg8JOC/6SFyOPtJ50EeWDlfaokwkJiYqLi5Obdq0Ubt27TR16lRlZ2crPj7e0aWhnJ09e1b79++3Pk9LS1NKSooCAgIUGhrqwMpQ3p544gktWLBAy5cvl6+vr9LT0yVJ/v7+8vLycnB1KE+jRo1SbGysQkNDdebMGS1YsEDr16/XmjVrHF0abgBfX1+7NQd9fHxUtWpV1iJEqdFDOjf6SOdFH+m86COdG31k6RDaokwMGDBAR48e1ejRo5Wenq6IiAitXr3a7sYSuPls27ZN3bp1sz5PTEyUJMXFxSkpKclBVeFGmDlzpiQpKirKZnzOnDkaMmTIjS8IN0xmZqYGDx6sI0eOyN/fX61atdKaNWvUvXt3R5cGoIKhh3Ru9JHOiz7SedFHAiVnMQzDcHQRAAAAAAAAAIBLWNMWAAAAAAAAAEyE0BYAAAAAAAAATITQFgAAAAAAAABMhNAWAAAAAAAAAEyE0BYAAAAAAAAATITQFgAAAAAAAABMhNAWAAAAAAAAAEyE0BYAAAAAAAAATITQFgBQIVksFi1btszRZQAAAKACoYcEUFEQ2gKAiQwZMkQWi8XusX///jLZf1JSkqpUqVIm+7pWQ4YMUd++fR1aAwAAwM2EHhIAbj5uji4AAGCrZ8+emjNnjs1YUFCQg6op2oULF1SpUiVHlwEAAADRQwLAzYYrbQHAZDw8PFS9enWbh6urqyRp+fLlat26tTw9PVW/fn2NGzdOFy9etL52ypQpatmypXx8fBQSEqLHH39cZ8+elSStX79e8fHxOnXqlPXqi7Fjx0oq/GNiVapUUVJSkiTpwIEDslgsWrRokbp27SpPT0/Nnz9fkvT++++radOm8vT0VFhYmN55551SHW9UVJSeeuopjRgxQgEBAapevbq1rgKpqanq0qWLPD091axZM33xxRd2+zl06JD69++vKlWqKCAgQH369NGBAwckSfv27ZO3t7cWLFhgnf/JJ5/Iy8tLP/30U6nqBQAAMCN6SHpIADcXQlsAqCA2bdqkwYMH6+mnn9ZPP/2kd999V0lJSXr99detc1xcXDRt2jTt2bNHc+fO1ZdffqkRI0ZIkm699VZNnTpVfn5+OnLkiI4cOaLnn3++VDWMHDlSTz/9tPbu3auYmBjNnz9fo0eP1uuvv669e/fqjTfe0Msvv6y5c+eWar9z586Vj4+PtmzZookTJ+qVV16xNtX5+fm6++675e7uri1btmjWrFl64YUXbF5/4cIFxcTEyNfXV5s2bdLmzZtVuXJl9ezZU7m5uQoLC9M//vEPPf744zp48KB+//13Pfroo5owYYKaNWtWqloBAAAqEnpIekgAFZQBADCNuLg4w9XV1fDx8bE++vXrZxiGYdx+++3GG2+8YTN/3rx5Ro0aNYrc3+LFi42qVatan8+ZM8fw9/e3myfJWLp0qc2Yv7+/MWfOHMMwDCMtLc2QZEydOtVmToMGDYwFCxbYjL366qtGhw4dij3GPn36WJ937drV6NSpk82ctm3bGi+88IJhGIaxZs0aw83Nzfjjjz+s21etWmVT87x584wmTZoY+fn51jk5OTmGl5eXsWbNGuvYHXfcYXTu3Nm4/fbbjR49etjMBwAAqKjoIS+hhwRwM2FNWwAwmW7dumnmzJnW5z4+PpKkXbt2afPmzTZXReTl5enPP//UuXPn5O3trXXr1mn8+PHat2+fTp8+rYsXL9psv15t2rSx/n92drZ++eUXDR06VMOGDbOOX7x4Uf7+/qXab6tWrWye16hRQ5mZmZKkvXv3KiQkRDVr1rRu79Chg838Xbt2af/+/fL19bUZ//PPP/XLL79Yn8+ePVuNGzeWi4uL9uzZI4vFUqo6AQAAzIoekh4SwM2F0BYATMbHx0cNGza0Gz979qzGjRunu+++226bp6enDhw4oL/97W967LHH9PrrrysgIEBff/21hg4dqtzc3GIbbovFIsMwbMYuXLhQaG2X1yNJ7733ntq3b28zr2D9tJK68mYUFotF+fn5JX792bNnFRkZaV0j7XKX34Bj165dys7OlouLi44cOaIaNWqUqk4AAACzooekhwRwcyG0BYAKonXr1vr5558LbcYlafv27crPz9fkyZPl4nJpyfJPPvnEZo67u7vy8vLsXhsUFKQjR45Yn6empurcuXPF1hMcHKyaNWvq119/1aBBg0p7OCXWtGlTHTp0yKZB/u6772zmtG7dWosWLVK1atXk5+dX6H6OHz+uIUOG6MUXX9SRI0c0aNAg7dixQ15eXuVWOwAAgKPRQ9JDAqiYuBEZAFQQo0eP1ocffqhx48Zpz5492rt3rxYuXKiXXnpJktSwYUNduHBB06dP16+//qp58+Zp1qxZNvuoW7euzp49q+TkZGVlZVmb6ttuu01vv/22du7cqW3btunRRx+1u3KhMOPGjdP48eM1bdo0/fe//9Xu3bs1Z84cTZkypcyOOzo6Wo0bN1ZcXJx27dqlTZs26cUXX7SZM2jQIAUGBqpPnz7atGmT0tLStH79ej311FP6/fffJUmPPvqoQkJC9NJLL2nKlCnKy8sr9U00AAAAKhp6SHpIABUToS0AVBAxMTFasWKF1q5dq7Zt2+qvf/2r3nrrLdWpU0eSFB4erilTpmjChAlq0aKF5s+fr/Hjx9vs49Zbb9Wjjz6qAQMGKCgoSBMnTpQkTZ48WSEhIercubMGDhyo559/vkTrlz300EN6//33NWfOHLVs2VJdu3ZVUlKS6tWrV2bH7eLioqVLl+r8+fNq166dHnroIZs12STJ29tbGzduVGhoqO6++241bdpUQ4cO1Z9//ik/Pz99+OGH+vzzzzVv3jy5ubnJx8dHH330kd577z2tWrWqzGoFAAAwG3pIekgAFZPFuHIBGgAAAAAAAACAw3ClLQAAAAAAAACYCKEtAAAAAAAAAJgIoS0AAAAAAAAAmAihLQAAAAAAAACYCKEtAAAAAAAAAJgIoS0AAAAAAAAAmAihLQAAAAAAAACYCKEtAAAAAAAAAJgIoS0AAAAAAAAAmAihLQAAAAAAAACYCKEtAAAAAAAAAJgIoS0AAAAAAAAAmMj/B+MKNOehGacZAAAAAElFTkSuQmCC\n" + }, + "metadata": {} + }, + { + "output_type": "stream", + "name": "stdout", + "text": [ + "\n", + "Coefficients (monotonic decreasing):\n", + "[ 0.6899 0.68989 0.05962 -0.0126 -0.01276]\n", + "Monotonic descreasing satisfied: True\n" + ] + } + ], + "source": [ + "## Visualize the feature coefficients\n", + "import seaborn as sns\n", + "import matplotlib.pyplot as plt\n", + "\n", + "df_coef = pd.DataFrame({\n", + " 'Feature Index': range(len(clf.coef_.flatten())),\n", + " 'SVM': clf.coef_.flatten(),\n", + " 'Monotonic SVM': mclf.coef_.flatten()\n", + "})\n", + "\n", + "fig, axes = plt.subplots(1, 2, figsize=(14, 5), sharey=True)\n", + "sns.barplot(data=df_coef, x=\"Feature Index\", y=\"SVM\", ax=axes[0], color='#FFE4E1', edgecolor='black')\n", + "axes[0].axhline(0, color='black', linewidth=1)\n", + "axes[0].set_yscale('symlog', linthresh=0.005)\n", + "axes[0].set_title(\"SVM Coefficients (symlog)\")\n", + "\n", + "sns.barplot(data=df_coef, x=\"Feature Index\", y=\"Monotonic SVM\", color='#8A8293', ax=axes[1], edgecolor='black')\n", + "axes[1].axhline(0, color='black', linewidth=1)\n", + "axes[1].set_yscale('symlog', linthresh=0.005)\n", + "axes[1].set_title(\"Monotonic SVM Coefficients (symlog)\")\n", + "plt.tight_layout()\n", + "plt.show()\n", + "\n", + "## Print the results of monotonic constraint\n", + "np.set_printoptions(precision=5, suppress=True)\n", + "print(\"\\nCoefficients (monotonic decreasing):\")\n", + "print(mclf.coef_)\n", + "print(\"Monotonic descreasing satisfied:\",\n", + " np.all(mclf.coef_[:-1] >= mclf.coef_[1:]))\n" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 927 + }, + "id": "GmOPIjrhyAZ3", + "outputId": "85bb891a-5a4e-4df0-983a-ec904b48fd29" + }, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "
" + ], + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjsAAAHHCAYAAABZbpmkAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAMzBJREFUeJzt3Xl8FHWe//F35yDhSmIgSTdHAnhBEAg38VqOQMSsiuAJYkDEHSbJqBlFUZRDB1xwhBEDOLMKOhJh8BYQCVFgVgJCALlZ4+CGge4ExE4gSBKT/v3hj14zQAOh09WpvJ6PRz1mur6frvqU7fGm6ltVFpfL5RIAAIBJBRjdAAAAQF0i7AAAAFMj7AAAAFMj7AAAAFMj7AAAAFMj7AAAAFMj7AAAAFMj7AAAAFMj7AAAAFMj7AAAAFMj7ADwe7t27dJdd92luLg4hYaGqnXr1ho8eLDmzZunbdu2yWKxaPLkyef9/rfffiuLxaLMzExJ0tSpU2WxWBQQEKBDhw6dVV9aWqrGjRvLYrEoPT29zo4LgG8QdgD4tY0bN6pXr1765ptvNH78eL322mt6+OGHFRAQoD/96U/q0aOHOnbsqHffffe828jOzpYkPfDAAzXWh4SEnPN7H3zwgXcPAoChgoxuAAA8+cMf/qDw8HBt2bJFERERNcaKi4slSaNGjdJzzz2nTZs2qV+/fmdt491331XHjh3Vo0ePGutvvfVWvfvuu5o4cWKN9dnZ2UpJSdH777/v3YMBYAjO7ADwa9999506d+58VtCRpOjoaEm/hB3p/87g/Fp+fr4OHDjgrvm1kSNHaseOHdq/f797ncPh0BdffKGRI0d66QgAGI2wA8CvxcXFKT8/X7t37z5vTfv27XX99dfrb3/7m6qqqmqMnQlA5wovN998s9q0aVMjJC1btkzNmjVTSkqKl44AgNEIOwD82hNPPKFTp04pISFB119/vZ566imtWbNGlZWVNepGjRqloqIi5ebmutdVV1dr2bJlSkxMVIcOHc7atsVi0X333Vdj3s6SJUs0fPhwhYSE1N1BAfApwg4AvzZ48GDl5eXp9ttv1zfffKNZs2YpOTlZrVu31ieffOKuu/feexUcHFzjLM369et1+PDhc17COmPkyJEqKCjQli1b3P/LJSzAXAg7APxe79699cEHH+jHH3/U119/rUmTJunEiRO66667tHfvXklSixYtlJycrA8//FCnT5+W9MslrKCgIN1zzz3n3Xb37t3VsWNHZWdna8mSJbJarRo4cKBPjguAbxB2ANQbjRo1Uu/evTVjxgwtWLBAlZWVWr58uXv8gQceUGlpqVasWKGKigq9//77GjJkiKKiojxud+TIkVq2bJmys7N17733KiCAfzUCZsI/0QDqpV69ekmS7Ha7e93tt9+u5s2bKzs7W5999pl+/PFHj5ewzhg5cqTsdrv+53/+h0tYgAnxnB0Afu3LL79U//79ZbFYaqxftWqVJOnaa691r2vcuLHuvPNOLVu2TKdOnVLTpk11xx13XHAfV155pebOnauffvpJffr08e4BADAcYQeAX8vIyNCpU6d05513qmPHjqqoqNDGjRu1bNkytWvXTmPHjq1R/8ADD+jtt9/W559/rlGjRqlp06YXtZ9HH320LtoH4AcIOwD82ssvv6zly5dr1apV+vOf/6yKigrFxsbqt7/9rSZPnnzWwwYHDhwom80mu91+UZewAJifxeVyuYxuAgAAoK4wQRkAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaz9mRVF1drSNHjqh58+ZnPaUVAAD4J5fLpRMnTqhVq1Ye32lH2JF05MgRtW3b1ug2AABALRw6dEht2rQ57zhhR1Lz5s0l/fIXKywszOBuAADAxSgtLVXbtm3d/x0/H8KO5L50FRYWRtgBAKCeudAUFCYoAwAAUyPsAAAAUyPsAAAAU2PODgAA9URVVZUqKyuNbsNngoODFRgYeNnbIewAAODnXC6XHA6HnE6n0a34XEREhKxW62U9B4+wAwCAnzsTdKKjo9WkSZMG8QBcl8ulU6dOqbi4WJJks9lqvS3CDgAAfqyqqsoddFq0aGF0Oz7VuHFjSVJxcbGio6NrfUmLCcoAAPixM3N0mjRpYnAnxjhz3JczV4mwAwBAPdAQLl2dizeOm7ADAABMjbADAABMjbADAABMjbADAABMjbADAABqePvtt9WiRQuVl5fXWD9s2DCNHj3aoK5qz9CwM3XqVFkslhpLx44d3eOnT59WWlqaWrRooWbNmmnEiBEqKiqqsY3CwkKlpKSoSZMmio6O1pNPPqmff/7Z14cCAIBp3H333aqqqtInn3ziXldcXKyVK1fqoYceMrCz2jH8oYKdO3fW2rVr3Z+Dgv6vpccff1wrV67U8uXLFR4ervT0dA0fPlxfffWVpF8etJSSkiKr1aqNGzfKbrfrwQcfVHBwsGbMmOHzYwHwi549eshut3ussdlsyt+2zUcdAbgUjRs31siRI7Vo0SLdfffdkqR33nlHsbGx6t+/v7HN1YLhYScoKEhWq/Ws9SUlJXrjjTeUnZ2tgQMHSpIWLVqkTp06adOmTerXr5/WrFmjvXv3au3atYqJiVFCQoJeeOEFPfXUU5o6daoaNWrk68MBIMlut6tg+1aPNVEdrlarCzz+nUAEGGf8+PHq3bu3Dh8+rNatW2vx4sUaM2ZMvXzej+Fh59tvv1WrVq0UGhqqxMREzZw5U7GxscrPz1dlZaWSkpLctR07dlRsbKzy8vLUr18/5eXlqUuXLoqJiXHXJCcna8KECdqzZ4+6d+9+zn2Wl5fXuA5ZWlpadwcI4JyqqqsuGIiu6t7LR90A+Ffdu3dXt27d9Pbbb2vIkCHas2ePVq5caXRbtWLonJ2+fftq8eLFWr16tRYsWKCDBw/qpptu0okTJ+RwONSoUSNFRETU+E5MTIwcDoekX16M9uugc2b8zNj5zJw5U+Hh4e6lbdu23j0wAABM4OGHH9bixYu1aNEiJSUl1dv/XhoadoYOHaq7775bXbt2VXJyslatWiWn06m//e1vdbrfSZMmqaSkxL0cOnSoTvcHAEB9NHLkSP3zn//UX/7yl3o5MfkMv7r1PCIiQtdcc40KCgpktVpVUVEhp9NZo6aoqMg9x8dqtZ51d9aZz+eaB3RGSEiIwsLCaiwAAKCm8PBwjRgxQs2aNdOwYcOMbqfW/CrsnDx5Ut99951sNpt69uyp4OBg5ebmuscPHDigwsJCJSYmSpISExO1a9cuFRcXu2tycnIUFham+Ph4n/cPAP6ma5euioqK8rh07dLV6Dbhxw4fPqxRo0YpJCTE6FZqzdAJyk888YRuu+02xcXF6ciRI5oyZYoCAwN1//33Kzw8XOPGjVNmZqYiIyMVFhamjIwMJSYmql+/fpKkIUOGKD4+XqNHj9asWbPkcDg0efJkpaWl1esfBQC8xe6wa9ozszzWTJkx0UfdoD758ccftW7dOq1bt07z5883up3LYmjY+ec//6n7779fP/zwg6KionTjjTdq06ZNioqKkiTNmTNHAQEBGjFihMrLy5WcnFzjL3hgYKBWrFihCRMmKDExUU2bNlVqaqqmT59u1CEBAGAK3bt3148//qj//M//1LXXXmt0O5fF0LCzdOlSj+OhoaHKyspSVlbWeWvi4uK0atUqb7cGAECD9v333xvdgtf41ZwdAAAAbyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUzP8recAAKB2CgsLdezYMZ/sq2XLloqNjfXJvryNsAMAQD1UWFioTp066dSpUz7ZX5MmTbRv377LCjwffPCBFi5cqPz8fB0/flzbt29XQkKC95o8D8IOAAD10LFjx3Tq1Cm98dqruvbqq+t0Xwe+/Vbj0n+nY8eOXVbYKSsr04033qh77rlH48eP92KHnhF2AACox669+mp179rF6DYuyujRoyX5/unMTFAGAACmRtgBAACmRtgBAABet2TJEjVr1sy9/P3vfzesF+bsAAAAr7v99tvVt29f9+fWrVsb1gthBwAAeF3z5s3VvHlzo9uQRNgBAAA+cvz4cRUWFurIkSOSpAMHDkiSrFarrFZrne2XsAPAb5WUlKiVzeaxxmazKX/bNh91BPifA99+W2/28cknn2js2LHuz/fdd58kacqUKZo6dapX9nEuhB0AfququkoF27d6rLmqey8fdQP4l5YtW6pJkyYal/47n+yvSZMmatmy5WVtY8yYMRozZox3GroEhB0AAOqh2NhY7du3j3djXQTCDgAA9VRsbGy9DSC+xHN2AACAqRF2AACAqRF2AACAqRF2AACAqRF2AACAqRF2AACAqRF2AACAqfGcHQAA6qnCwkIeKngRCDsAANRDhYWF6tixo3766Sef7K9x48bav3//JQWeDRs2aPbs2crPz5fdbteHH36oYcOG1V2T50HYAQCgHjp27Jh++uknpY4aL2tMqzrdl6PoiN5a8hcdO3bsksJOWVmZunXrpoceekjDhw+vww49I+wAQD3VtUtX2R12jzVOp9M3zcAw1phWim0TZ3Qb5zR06FANHTrU6DYIOwBQX9kddk17ZpbHmownxvuoG8B/EXYAoIFzOp2KioryWGOz2rRz104fdQR4F2EHABq46mrXBc8QTZkx0UfdAN7Hc3YAAICpEXYAAICpcRkLAADUiZMnT6qgoMD9+eDBg9qxY4ciIyN9+oBCwg4At549eshu93wrs81mU/62bT7qCMCFOIqO+O0+tm7dqgEDBrg/Z2ZmSpJSU1O1ePFib7R2UQg7ANzsdrsKtm/1WHNV914+6gaAJy1btlTjxo311pK/+GR/jRs3VsuWLS/pO/3795fL5aqjji4eYQcAgHooNjZW+/fv591YF4GwAwBAPRUbG1tvA4gvEXYAXJKSkhK1stk81jhLnL5pBgAuAmEHwCWpqq664LyeyHYdfNQNAFwYz9kBAKAe8IeJvkbwxnETdgAA8GPBwcGSpFOnThnciTHOHPeZvw61wWUsAAD8WGBgoCIiIlRcXCxJatKkiSwWi8Fd1T2Xy6VTp06puLhYERERCgwMrPW2CDsAAPg5q9UqSe7A05BERES4j7+2CDsAAPg5i8Uim82m6OhoVVZWGt2OzwQHB1/WGZ0zCDsAANQTgYGBXvmPf0PDBGUAAGBqhB0AAGBqhB0AAGBqhB0AAGBqhB0AAGBqhB0AAGBqhB0AAGBqhB0AAGBqPFQQAPxQ1y5dZXfYPdY4nU7fNAPUc4QdAPBDdodd056Z5bEm44nxPurml2AVFRXlscZmtWnnrp0+6gi4eH4Tdl566SVNmjRJjz76qObOnStJOn36tH7/+99r6dKlKi8vV3JysubPn6+YmBj39woLCzVhwgR9+eWXatasmVJTUzVz5kwFBfnNoQFAvVdd7bpg+JoyY6KPugEujV/M2dmyZYtef/11de3atcb6xx9/XJ9++qmWL1+u9evX68iRIxo+fLh7vKqqSikpKaqoqNDGjRv11ltvafHixXr++ed9fQgAAMBPGR52Tp48qVGjRukvf/mLrrjiCvf6kpISvfHGG3rllVc0cOBA9ezZU4sWLdLGjRu1adMmSdKaNWu0d+9evfPOO0pISNDQoUP1wgsvKCsrSxUVFUYdEgAA8COGh520tDSlpKQoKSmpxvr8/HxVVlbWWN+xY0fFxsYqLy9PkpSXl6cuXbrUuKyVnJys0tJS7dmz57z7LC8vV2lpaY0FAACYk6ETW5YuXapt27Zpy5YtZ405HA41atRIERERNdbHxMTI4XC4a34ddM6Mnxk7n5kzZ2ratGmX2T0Af1BSUqJWNpvHGpvNpvxt23zUEQB/Y1jYOXTokB599FHl5OQoNDTUp/ueNGmSMjMz3Z9LS0vVtm1bn/YAwDuqqqtUsH2rx5qruvfyUTcA/JFhl7Hy8/NVXFysHj16KCgoSEFBQVq/fr1effVVBQUFKSYmRhUVFWc9R6KoqEhWq1WSZLVaVVRUdNb4mbHzCQkJUVhYWI0FAACYk2FhZ9CgQdq1a5d27NjhXnr16qVRo0a5/39wcLByc3Pd3zlw4IAKCwuVmJgoSUpMTNSuXbtUXFzsrsnJyVFYWJji4+N9fkwAAMD/GHYZq3nz5rruuutqrGvatKlatGjhXj9u3DhlZmYqMjJSYWFhysjIUGJiovr16ydJGjJkiOLj4zV69GjNmjVLDodDkydPVlpamkJCQnx+TAAAwP/49ZP35syZo4CAAI0YMaLGQwXPCAwM1IoVKzRhwgQlJiaqadOmSk1N1fTp0w3sGgAA+BO/Cjvr1q2r8Tk0NFRZWVnKyso673fi4uK0atWqOu4MAADUV4Y/ZwcAAKAuEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpBRndAAA0NF27dJXdYfdY43Q6fdMM0AAQdgDAx+wOu6Y9M8tjTcYT433UDWB+XMYCAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmxhOUAcCLeBUE4H8IOwDgRbwKAvA/XMYCAACmRtgBAACmRtgBAACmxpwdAIBXOJ1ORUVFeayxWW3auWunjzoCfkHYAQB4RXW164KTs6fMmOijboD/w2UsAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgatyNBTQQPXv0kN1+gXc2lTh90wwA+BBhB2gg7Ha7CrZv9VgT2a6Dj7oBAN/hMhYAADA1wg4AADA1wg4AADA1wg4AADA1wg4AADA1wg4AADA1wg4AADA1wg4AADA1wg4AADA1Q8POggUL1LVrV4WFhSksLEyJiYn67LPP3OOnT59WWlqaWrRooWbNmmnEiBEqKiqqsY3CwkKlpKSoSZMmio6O1pNPPqmff/7Z14cCAAD8lKGvi2jTpo1eeuklXX311XK5XHrrrbd0xx13aPv27ercubMef/xxrVy5UsuXL1d4eLjS09M1fPhwffXVV5KkqqoqpaSkyGq1auPGjbLb7XrwwQcVHBysGTNmGHloAIBzcDqdioqK8lhjs9q0c9dOH3WEhsDQsHPbbbfV+PyHP/xBCxYs0KZNm9SmTRu98cYbys7O1sCBAyVJixYtUqdOnbRp0yb169dPa9as0d69e7V27VrFxMQoISFBL7zwgp566ilNnTpVjRo1MuKwAADnUV3t0rRnZnmsmTJjoo+6QUPhN3N2qqqqtHTpUpWVlSkxMVH5+fmqrKxUUlKSu6Zjx46KjY1VXl6eJCkvL09dunRRTEyMuyY5OVmlpaXas2ePz48BAAD4H8Pfer5r1y4lJibq9OnTatasmT788EPFx8drx44datSokSIiImrUx8TEyOFwSJIcDkeNoHNm/MzY+ZSXl6u8vNz9ubS01EtHAwAA/I3hZ3auvfZa7dixQ5s3b9aECROUmpqqvXv31uk+Z86cqfDwcPfStm3bOt0fAAAwjuFhp1GjRrrqqqvUs2dPzZw5U926ddOf/vQnWa1WVVRUyOl01qgvKiqS1WqVJFmt1rPuzjrz+UzNuUyaNEklJSXu5dChQ949KAAA4DcMDzv/qrq6WuXl5erZs6eCg4OVm5vrHjtw4IAKCwuVmJgoSUpMTNSuXbtUXFzsrsnJyVFYWJji4+PPu4+QkBD37e5nFgAAYE6GztmZNGmShg4dqtjYWJ04cULZ2dlat26dPv/8c4WHh2vcuHHKzMxUZGSkwsLClJGRocTERPXr10+SNGTIEMXHx2v06NGaNWuWHA6HJk+erLS0NIWEhBh5aAAAwE8YGnaKi4v14IMPym63Kzw8XF27dtXnn3+uwYMHS5LmzJmjgIAAjRgxQuXl5UpOTtb8+fPd3w8MDNSKFSs0YcIEJSYmqmnTpkpNTdX06dONOiQAAOBnDA07b7zxhsfx0NBQZWVlKSsr67w1cXFxWrVqlbdbAwAAJlGrOTsdOnTQDz/8cNZ6p9OpDh06XHZTAAAA3lKrsPP999+rqqrqrPXl5eU6fPjwZTcFAADgLZd0GeuTTz5x//8zk4jPqKqqUm5urtq1a+e15gAAAC7XJYWdYcOGSZIsFotSU1NrjAUHB6tdu3b64x//6LXmAAAALtclhZ3q6mpJUvv27bVlyxa1bNmyTpoCAG8qKSlRK5vNY43NZlP+tm0+6giAL9XqbqyDBw96uw8AqDNV1VUq2L7VY81V3Xv5qBsAvlbrW89zc3OVm5ur4uJi9xmfM958883LbgwAAMAbahV2pk2bpunTp6tXr16y2WyyWCze7gsAAMArahV2Fi5cqMWLF2v06NHe7gcAAMCravWcnYqKCl1//fXe7gUAAMDrahV2Hn74YWVnZ3u7FwAAAK+r1WWs06dP689//rPWrl2rrl27Kjg4uMb4K6+84pXmAAAALletws7OnTuVkJAgSdq9e3eNMSYrAwAAf1KrsPPll196uw8AAIA6Uas5OwAAAPVFrc7sDBgwwOPlqi+++KLWDQEAAHhTrcLOmfk6Z1RWVmrHjh3avXv3WS8IBQAAMFKtws6cOXPOuX7q1Kk6efLkZTUEAADgTV6ds/PAAw/wXiwAAOBXvBp28vLyFBoa6s1NAgAAXJZaXcYaPnx4jc8ul0t2u11bt27Vc88955XGAAAAvKFWYSc8PLzG54CAAF177bWaPn26hgwZ4pXGAAAAvKFWYWfRokXe7gMAAKBO1CrsnJGfn699+/ZJkjp37qzu3bt7pSkAAABvqVXYKS4u1n333ad169YpIiJCkuR0OjVgwAAtXbpUUVFR3uwRAACg1mp1N1ZGRoZOnDihPXv26Pjx4zp+/Lh2796t0tJS/e53v/N2jwDgF7p26aqoqCiPi9PpNLpNAP+iVmd2Vq9erbVr16pTp07udfHx8crKymKCMgDTsjvsmvbMLI81GU+M91E3AC5Wrc7sVFdXKzg4+Kz1wcHBqq6uvuymAAAAvKVWYWfgwIF69NFHdeTIEfe6w4cP6/HHH9egQYO81hwAAMDlqlXYee2111RaWqp27drpyiuv1JVXXqn27durtLRU8+bN83aPAAAAtVarOTtt27bVtm3btHbtWu3fv1+S1KlTJyUlJXm1OQAAgMt1SWd2vvjiC8XHx6u0tFQWi0WDBw9WRkaGMjIy1Lt3b3Xu3Fl///vf66pXAACAS3ZJYWfu3LkaP368wsLCzhoLDw/Xf/zHf+iVV17xWnMAAACX65LCzjfffKNbbrnlvONDhgxRfn7+ZTcFAADgLZcUdoqKis55y/kZQUFBOnr06GU3BQAA4C2XFHZat26t3bt3n3d8586dstlsl90UAACAt1xS2Ln11lv13HPP6fTp02eN/fTTT5oyZYr+/d//3WvNAQAAXK5LuvV88uTJ+uCDD3TNNdcoPT1d1157rSRp//79ysrKUlVVlZ599tk6aRQAAKA2LinsxMTEaOPGjZowYYImTZokl8slSbJYLEpOTlZWVpZiYmLqpFEAAIDauOSHCsbFxWnVqlX68ccfVVBQIJfLpauvvlpXXHFFXfQHAABwWWr1BGVJuuKKK9S7d29v9gIAAOB1tXo3FgAAQH1B2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKZW64cKAgBQF5xOp6KiojzW2Kw27dy100cdob4j7AAA/Ep1tUvTnpnlsWbKjIk+6gZmwGUsAABgaoQdAABgaoQdAABgaszZAUygZ48estvtHmucJU7fNAMAfoawA5iA3W5XwfatHmsi23XwUTcA4F+4jAUAAEyNsAMAAEyNsAMAAEzN0LAzc+ZM9e7dW82bN1d0dLSGDRumAwcO1Kg5ffq00tLS1KJFCzVr1kwjRoxQUVFRjZrCwkKlpKSoSZMmio6O1pNPPqmff/7Zl4cCAAD8lKFhZ/369UpLS9OmTZuUk5OjyspKDRkyRGVlZe6axx9/XJ9++qmWL1+u9evX68iRIxo+fLh7vKqqSikpKaqoqNDGjRv11ltvafHixXr++eeNOCQAAOBnDL0ba/Xq1TU+L168WNHR0crPz9fNN9+skpISvfHGG8rOztbAgQMlSYsWLVKnTp20adMm9evXT2vWrNHevXu1du1axcTEKCEhQS+88IKeeuopTZ06VY0aNTLi0AAAgJ/wqzk7JSUlkqTIyEhJUn5+viorK5WUlOSu6dixo2JjY5WXlydJysvLU5cuXRQTE+OuSU5OVmlpqfbs2XPO/ZSXl6u0tLTGAgAAzMlvwk51dbUee+wx3XDDDbruuuskSQ6HQ40aNVJERESN2piYGDkcDnfNr4POmfEzY+cyc+ZMhYeHu5e2bdt6+WgAAIC/8Juwk5aWpt27d2vp0qV1vq9JkyappKTEvRw6dKjO9wkAAIzhF09QTk9P14oVK7Rhwwa1adPGvd5qtaqiokJOp7PG2Z2ioiJZrVZ3zddff11je2fu1jpT869CQkIUEhLi5aMAAAD+yNAzOy6XS+np6frwww/1xRdfqH379jXGe/bsqeDgYOXm5rrXHThwQIWFhUpMTJQkJSYmateuXSouLnbX5OTkKCwsTPHx8b45EAAA4LcMPbOTlpam7Oxsffzxx2revLl7jk14eLgaN26s8PBwjRs3TpmZmYqMjFRYWJgyMjKUmJiofv36SZKGDBmi+Ph4jR49WrNmzZLD4dDkyZOVlpbG2RsAAGBs2FmwYIEkqX///jXWL1q0SGPGjJEkzZkzRwEBARoxYoTKy8uVnJys+fPnu2sDAwO1YsUKTZgwQYmJiWratKlSU1M1ffp0Xx0GAADwY4aGHZfLdcGa0NBQZWVlKSsr67w1cXFxWrVqlTdbAwAAJuE3d2MBAADUBcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNb94gjIAGK2kpEStbDaPNWUnT/qoGwDeRNgBAElV1VUq2L7VY01kXAcfdQPAm7iMBQAATI2wAwAATI3LWAAgyVXt0uzZsz3XXMRT3wH4H8IOAPx/A24a7HF8xrz5HscB+CcuYwEAAFMj7AAAAFMj7AAAAFMj7AAAAFMj7AAAAFMj7AAAAFMj7AAAAFPjOTsAcAkmT/u9x/GgQP4MCfgbwg4AXIKPFv/V4/jge+70UScALhZhB/BjPXv0kN1uv2Cds8RZ980AQD1F2AH8mN1uV8H2rResi2zXwQfdAED9xMVlAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgakFGNwAAwKVyOp2KioryWGOz2rRz104fdQR/RtgBANQ71dUuTXtmlseaRyc+QiCCJMIOgAbAVe3S7NmzjW4DPnYxgWjKjIk+6gZGIuwAaBAG3DTY4/iMefN91AkAX2OCMgAAMDXO7AAG6dmjh+x2u8caZ4nTN80AgIkRdgCD2O12FWzf6rEmsl0HH3UDAObFZSwAAGBqhB0AAGBqhB0AAGBqhB0AAGBqhB0AAGBqhB0AAGBqhB0AAGBqhoadDRs26LbbblOrVq1ksVj00Ucf1Rh3uVx6/vnnZbPZ1LhxYyUlJenbb7+tUXP8+HGNGjVKYWFhioiI0Lhx43Ty5EkfHgUAAPBnhoadsrIydevWTVlZWeccnzVrll599VUtXLhQmzdvVtOmTZWcnKzTp0+7a0aNGqU9e/YoJydHK1as0IYNG/TII4/46hAAAICfM/QJykOHDtXQoUPPOeZyuTR37lxNnjxZd9xxhyTp7bffVkxMjD766CPdd9992rdvn1avXq0tW7aoV69ekqR58+bp1ltv1csvv6xWrVr57FgAAIB/8ts5OwcPHpTD4VBSUpJ7XXh4uPr27au8vDxJUl5eniIiItxBR5KSkpIUEBCgzZs3n3fb5eXlKi0trbEAAABz8tuw43A4JEkxMTE11sfExLjHHA6HoqOja4wHBQUpMjLSXXMuM2fOVHh4uHtp27atl7sHAAD+okG+CHTSpEnKzMx0fy4tLSXwAPCaydN+73E8KNBv/5wJmJLfhh2r1SpJKioqks1mc68vKipSQkKCu6a4uLjG937++WcdP37c/f1zCQkJUUhIiPebBgBJHy3+q8fxwffc6aNOAEh+fBmrffv2slqtys3Nda8rLS3V5s2blZiYKElKTEyU0+lUfn6+u+aLL75QdXW1+vbt6/OeAQCA/zH0zM7JkydVUFDg/nzw4EHt2LFDkZGRio2N1WOPPaYXX3xRV199tdq3b6/nnntOrVq10rBhwyRJnTp10i233KLx48dr4cKFqqysVHp6uu677z7uxIKhevboIbvd7rHGWeL0TTMm56p2afbs2Ua3AcCPGRp2tm7dqgEDBrg/n5lHk5qaqsWLF2vixIkqKyvTI488IqfTqRtvvFGrV69WaGio+ztLlixRenq6Bg0apICAAI0YMUKvvvqqz48F+DW73a6C7Vs91kS26+CjbsxvwE2DPY7PmDffR50A8EeGhp3+/fvL5XKdd9xisWj69OmaPn36eWsiIyOVnZ1dF+0BAAAT8Ns5OwAAAN5A2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKbmt+/GAvwVT0cGgPqFsANcIp6ODAD1C5exAACAqRF2AACAqRF2AACAqRF2AACAqTFBGQDQYDmdTkVFRXmssVlt2rlrp486Ql0g7AAAGqzqapemPTPLY82UGRN91A3qCmEHAAwwedrvPY4HBTLLAPAWwg4AGOCjxX/1OD74njt91AlgfvzRAQAAmBphBwAAmBphBwAAmBpzdoBf4SWf8CdMYga8g7AD/Aov+YQ/YRIz4B2EHQB+y1Xt0uzZs41uA0A9R9gB4NcG3DTY4/iMefN91AmA+ooLvgAAwNQIOwAAwNQIOwAAwNQIOwAAwNQIOwAAwNS4GwsNBg8M9C/cVg7AVwg7aDB4YKD/4bZyAL7AZSwAAGBqhB0AAGBqhB0AAGBqhB0AAGBqTFAGgHps8rTfexwPCuTPtABhBwDqsY8W/9Xj+OB77vRRJ+bldDoVFRXlscZmtWnnrp0+6giXirADAIAH1dUuTXtmlseaKTMm+qgb1AbnNwEAgKlxZgcAGrigwADm/sDUCDsAYHIXCjISc39gboQdADA5ggwaOs5LAgAAU+PMDgx1MW8it9lsyt+27bK3wxvNAaBhIuzAUBfzJvKruvfyynZ4ozkANEyEHfi9kpIStbLZPNZw1sa/uKpdmj17ttFtAD7Dgwf9G2EHfq+quoqzNvXQgJsGexyfMW++jzoB6h4PHvRvTFAGAACmRtgBAACmxmUsAMBF4SnLqK8IO6gz3A4OmAsPJ0R9RdhBneF28PrnYu6iclW7fNQNAHgHYQdADRdzFxW3leN8uNR1ftyebhzCDoBLxm3lOB8udZ0ft6cbp+FGbAAA0CBwZgcA4FNc6oKvEXYAAD51MZe6CETwJsIOaoXbygHUpYY694dJzHXDNGEnKytLs2fPlsPhULdu3TRv3jz16dPH6LbqpYsNMsf+UeCxhtvKAeDSMIm5bpgi7CxbtkyZmZlauHCh+vbtq7lz5yo5OVkHDhxQdHS00e35zMWEFJvNpvxt2zzW8Hwcc+JN5AAaKlOEnVdeeUXjx4/X2LFjJUkLFy7UypUr9eabb+rpp582uDvfuZiQclX3Xj7qBv6IW8YBNET1PuxUVFQoPz9fkyZNcq8LCAhQUlKS8vLyDOzs4l3MGZmysjI1bdrUYw1zZBouztqgIbqYScy+rPEVb83r6dqlq+yOC1wNMMn8oHofdo4dO6aqqirFxMTUWB8TE6P9+/ef8zvl5eUqLy93fy4pKZEklZaWer2/m2+6SQ6Hw2NNSWmJDu7c4bEm7rquOpD/9QVrSk+c8FjjdDpl/Ze/Vufq50LbcblcpqyprqrWCy++eMEaf9qO9Mux9el5wwVqslR26tQFt0MNNfWlZsn81z3W3JE60qc1P53+yWNNYIBFz07NvGDNhbZjkfRzxc8ea/bv36cWLVp4rCkpKdF/vvCqx5qnnvvdBbdTVnZKTZs28VhjjbEqb5P3T0Cc+e+2y3WB19i46rnDhw+7JLk2btxYY/2TTz7p6tOnzzm/M2XKFJckFhYWFhYWFhMshw4d8pgV6v2ZnZYtWyowMFBFRUU11hcVFclqtZ7zO5MmTVJm5v+l6+rqah0/flwtWrSQxWKp037rm9LSUrVt21aHDh1SWFiY0e00SPwG/oHfwXj8Bv7Bn34Hl8ulEydOqFWrVh7r6n3YadSokXr27Knc3FwNGzZM0i/hJTc3V+np6ef8TkhIiEJCQmqsi4iIqONO67ewsDDD/6Zu6PgN/AO/g/H4DfyDv/wO4eHhF6yp92FHkjIzM5WamqpevXqpT58+mjt3rsrKytx3ZwEAgIbLFGHn3nvv1dGjR/X888/L4XAoISFBq1evPmvSMgAAaHhMEXYkKT09/byXrVB7ISEhmjJlylmX/eA7/Ab+gd/BePwG/qE+/g4Wl+tC92sBAADUX7w2FgAAmBphBwAAmBphBwAAmBphBwAAmBphB5esvLxcCQkJslgs2rFjh9HtNCjff/+9xo0bp/bt26tx48a68sorNWXKFFVUVBjdmqllZWWpXbt2Cg0NVd++ffX1157fUwfvmjlzpnr37q3mzZsrOjpaw4YN04EDB4xuq0F76aWXZLFY9NhjjxndykUh7OCSTZw48YKP5kbd2L9/v6qrq/X6669rz549mjNnjhYuXKhnnnnG6NZMa9myZcrMzNSUKVO0bds2devWTcnJySouLja6tQZj/fr1SktL06ZNm5STk6PKykoNGTJEZWVlRrfWIG3ZskWvv/66unbtanQrF41bz3FJPvvsM2VmZur9999X586dtX37diUkJBjdVoM2e/ZsLViwQP/4xz+MbsWU+vbtq969e+u1116T9MvraNq2bauMjAw9/fTTBnfXMB09elTR0dFav369br75ZqPbaVBOnjypHj16aP78+XrxxReVkJCguXPnGt3WBXFmBxetqKhI48eP11//+lc1adLE6Hbw/5WUlCgyMtLoNkypoqJC+fn5SkpKcq8LCAhQUlKS8vLyDOysYSspKZEk/r43QFpamlJSUmr8M1EfmOYJyqhbLpdLY8aM0W9+8xv16tVL33//vdEtQVJBQYHmzZunl19+2ehWTOnYsWOqqqo669UzMTEx2r9/v0FdNWzV1dV67LHHdMMNN+i6664zup0GZenSpdq2bZu2bNlidCuXjDM7DdzTTz8ti8Xicdm/f7/mzZunEydOaNKkSUa3bEoX+zv82uHDh3XLLbfo7rvv1vjx4w3qHPCttLQ07d69W0uXLjW6lQbl0KFDevTRR7VkyRKFhoYa3c4lY85OA3f06FH98MMPHms6dOige+65R59++qksFot7fVVVlQIDAzVq1Ci99dZbdd2qqV3s79CoUSNJ0pEjR9S/f3/169dPixcvVkAAf26pCxUVFWrSpInee+89DRs2zL0+NTVVTqdTH3/8sXHNNUDp6en6+OOPtWHDBrVv397odhqUjz76SHfeeacCAwPd66qqqmSxWBQQEKDy8vIaY/6GsIOLUlhYqNLSUvfnI0eOKDk5We+995769u2rNm3aGNhdw3L48GENGDBAPXv21DvvvOPX/4Ixg759+6pPnz6aN2+epF8uo8TGxio9PZ0Jyj7icrmUkZGhDz/8UOvWrdPVV19tdEsNzokTJ/S///u/NdaNHTtWHTt21FNPPeX3lxSZs4OLEhsbW+Nzs2bNJElXXnklQceHDh8+rP79+ysuLk4vv/yyjh496h6zWq0GdmZemZmZSk1NVa9evdSnTx/NnTtXZWVlGjt2rNGtNRhpaWnKzs7Wxx9/rObNm8vhcEiSwsPD1bhxY4O7axiaN29+VqBp2rSpWrRo4fdBRyLsAPVKTk6OCgoKVFBQcFbI5CRt3bj33nt19OhRPf/883I4HEpISNDq1avPmrSMurNgwQJJUv/+/WusX7RokcaMGeP7hlDvcBkLAACYGrMaAQCAqRF2AACAqRF2AACAqRF2AACAqRF2AACAqRF2AACAqRF2AACAqRF2AACAqRF2AACAqRF2AEC/vOEcgDkRdgD4tffee09dunRR48aN1aJFCyUlJamsrEyS9Oabb6pz584KCQmRzWZTenq6+3uFhYW644471KxZM4WFhemee+5RUVGRe3zq1KlKSEjQf/3Xf6l9+/YKDQ2VJDmdTj388MOKiopSWFiYBg4cqG+++ca3Bw3Aqwg7APyW3W7X/fffr4ceekj79u3TunXrNHz4cLlcLi1YsEBpaWl65JFHtGvXLn3yySe66qqrJEnV1dW64447dPz4ca1fv145OTn6xz/+oXvvvbfG9gsKCvT+++/rgw8+0I4dOyRJd999t4qLi/XZZ58pPz9fPXr00KBBg3T8+HFfHz4AL+FFoAD81rZt29SzZ099//33iouLqzHWunVrjR07Vi+++OJZ38vJydHQoUN18OBBtW3bVpK0d+9ede7cWV9//bV69+6tqVOnasaMGTp8+LCioqIkSf/93/+tlJQUFRcXKyQkxL29q666ShMnTtQjjzxSh0cLoK4EGd0AAJxPt27dNGjQIHXp0kXJyckaMmSI7rrrLlVWVurIkSMaNGjQOb+3b98+tW3b1h10JCk+Pl4RERHat2+fevfuLUmKi4tzBx1J+uabb3Ty5Em1aNGixvZ++uknfffdd3VwhAB8gbADwG8FBgYqJydHGzdu1Jo1azRv3jw9++yzys3N9cr2mzZtWuPzyZMnZbPZtG7durNqIyIivLJPAL5H2AHg1ywWi2644QbdcMMNev755xUXF6ecnBy1a9dOubm5GjBgwFnf6dSpkw4dOqRDhw7VuIzldDoVHx9/3n316NFDDodDQUFBateuXV0dEgAfI+wA8FubN29Wbm6uhgwZoujoaG3evFlHjx5Vp06dNHXqVP3mN79RdHS0hg4dqhMnTuirr75SRkaGkpKS1KVLF40aNUpz587Vzz//rN/+9rf6t3/7N/Xq1eu8+0tKSlJiYqKGDRumWbNm6ZprrtGRI0e0cuVK3XnnnR6/C8B/EXYA+K2wsDBt2LBBc+fOVWlpqeLi4vTHP/5RQ4cOlSSdPn1ac+bM0RNPPKGWLVvqrrvukvTL2aCPP/5YGRkZuvnmmxUQEKBbbrlF8+bN87g/i8WiVatW6dlnn9XYsWN19OhRWa1W3XzzzYqJianz4wVQN7gbCwAAmBrP2QEAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKb2/wBPfFiQRY1aIwAAAABJRU5ErkJggg==\n" + }, + "metadata": {} + }, + { + "output_type": "display_data", + "data": { + "text/plain": [ + "
" + ], + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjsAAAHHCAYAAABZbpmkAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAP/lJREFUeJzt3Xl4FfXd///XSUgCAZIQyEIKCYvKKousEUpZ0rCJIAFuFgGBgtJAi7FoY2UVpbVWKIiALQZBIha/IouKLLKoLCoVlC1AbjCyhIAhBIKEcDK/P/xx7p5yEiScdfJ8XNe5Lmbec2ben6DwYuYzMxbDMAwBAACYlJ+nGwAAAHAlwg4AADA1wg4AADA1wg4AADA1wg4AADA1wg4AADA1wg4AADA1wg4AADA1wg4AADA1wg4ASDp58qQsFouWLl3q6VYAOBlhBygnli5dKovFIovFos8+++yWumEYql27tiwWix566CEPdHirQ4cOafr06Tp58qSnW7lj58+f1+9//3s1bNhQlSpVUmRkpNq2batnnnlGV65cUVFRkWrUqKGOHTuWuI+bvycPPPCAJGnbtm2238O33nrL4Xc6dOggi8Wipk2bumRcgC8i7ADlTMWKFZWenn7L+u3bt+vUqVMKCgryQFeOHTp0SDNmzHBL2ImLi9OPP/6o4cOH3/W+cnNz1bp1ay1btky9e/fWvHnzlJKSonvuuUcLFy7UhQsXFBAQoIEDB2rnzp367rvvHO5nx44dOnXqlB599FG79SX9Hp48eVI7d+5UxYoV73oMgJlU8HQDANyrV69eWrVqlebNm6cKFf7vj4D09HS1atVKFy5c8GB3nmOxWJwWEpYsWaKsrCx9/vnnevDBB+1q+fn5CgwMlCQNGzZMixYt0ttvv60//vGPt+wnPT1dfn5+Gjx4sN36Xr16ae3atbpw4YJq1Khht31UVJTuvfdeXbx40SljAcyAMztAOTNkyBD98MMP2rRpk23d9evX9e6772ro0KEOv1NQUKCnnnpKtWvXVlBQkBo0aKCXX35ZhmHYbWexWDRhwgS9//77atq0qYKCgtSkSRNt2LDhln1+/fXX6tmzp0JCQlSlShV169ZNu3fvttWXLl2qgQMHSpK6dOliu3yzbds22zavvfaamjRpoqCgIMXExCg5OVl5eXl2x+ncubOaNm2qQ4cOqUuXLgoODtYvfvELvfTSS3bblTRn58iRIxo0aJAiIiJUqVIlNWjQQH/6059K/PlKUmZmpvz9/dW+fftbaiEhIbZQ1aFDB9WpU8fhWZqioiK9++676tKli2JiYuxqffv2VVBQkFatWmW3Pj09XYMGDZK/v3+p/QHlDWEHKGfq1Kmj+Ph4vf3227Z1H330kS5dunTLGQTpp3kjDz/8sObMmaMePXrolVdeUYMGDTR58mSlpKTcsv1nn32m3/72txo8eLBeeuklXbt2TUlJSfrhhx9s2xw8eFC//OUvtX//fj399NOaMmWKTpw4oc6dO2vPnj2SpE6dOul3v/udJOnZZ5/V8uXLtXz5cjVq1EiSNH36dCUnJysmJkZ/+9vflJSUpMWLFysxMVFFRUV2PV28eFE9evRQ8+bN9be//U0NGzbUM888o48++qjUn9U333yjdu3a6ZNPPtHYsWP197//Xf369dO6detK/V5cXJysVquWL19e6nYWi0VDhw7Vt99+q4MHD9rVNmzYoNzcXA0bNuyW7wUHB6tv3752v4f79+/XwYMHSwysQLlmACgX0tLSDEnGl19+abz66qtG1apVjatXrxqGYRgDBw40unTpYhiGYcTFxRm9e/e2fe/99983JBmzZs2y29+AAQMMi8ViHD9+3LZOkhEYGGi3bv/+/YYkY/78+bZ1/fr1MwIDA43MzEzbujNnzhhVq1Y1OnXqZFu3atUqQ5KxdetWu2Pn5OQYgYGBRmJiomG1Wm3rX331VUOS8cYbb9jW/epXvzIkGcuWLbOtKywsNKKjo42kpCTbuhMnThiSjLS0NNu6Tp06GVWrVjW+++47u+MXFxcbpcnOzjYiIiIMSUbDhg2NJ554wkhPTzfy8vJu2fbgwYOGJCM1NdVu/eDBg42KFSsaly5dsq3bunWrIclYtWqVsX79esNisRhZWVmGYRjG5MmTjXr16tnG3KRJk1J7BMoTzuwA5dCgQYP0448/av369bp8+bLWr19f4hmBDz/8UP7+/razLDc99dRTMgzjlrMjCQkJql+/vm25WbNmCgkJ0f/+7/9KkqxWqzZu3Kh+/fqpXr16tu1q1qypoUOH6rPPPlN+fn6p/W/evFnXr1/XpEmT5Of3f3+MjR07ViEhIfrggw/stq9SpYrdJN/AwEC1bdvW1pMj58+f144dOzR69GjFxsba1SwWS6n9RUVFaf/+/XriiSd08eJFLVq0SEOHDlVkZKSef/55u8t/jRs3VsuWLbVy5UrbuoKCAq1du1YPPfSQQkJCHB4jMTFR4eHhWrlypQzD0MqVKzVkyJBS+wLKK8IOUA5FREQoISFB6enpeu+992S1WjVgwACH23733XeKiYlR1apV7dbfvJz033cS/XcwkKRq1arZJsyeP39eV69eVYMGDW7ZrlGjRiouLtb3339fav83j/nf+wgMDFS9evVu6alWrVq3BJT/7MmRm0GorLdw16xZUwsXLtTZs2eVkZGhefPmKSIiQlOnTtWSJUvsth02bJhOnDihnTt3SpLef/99Xb161eElrJtu3s2Vnp6uHTt26Pvvv+cSFlACwg5QTg0dOlQfffSRFi1apJ49eyosLMwp+y1pcqzxX5OZ3cmTPVksFt13332aOHGiduzYIT8/P61YscJumyFDhsjPz882UTk9PV3VqlVTr169St330KFDtW/fPk2fPl3NmzdX48aNXTYOwJcRdoBy6pFHHpGfn592795d6hmBuLg4nTlzRpcvX7Zbf+TIEVv9TkRERCg4OFgZGRm31I4cOSI/Pz/Vrl1bUsmXi24e87/3cf36dZ04ceKOe3Lk5iW2AwcO3PW+/nOf1apV09mzZ+3Wx8TEqEuXLlq1apXOnTunTZs2acCAAbZb1EvSsWNHxcbGatu2bZzVAUpB2AHKqSpVqmjhwoWaPn26+vTpU+J2vXr1ktVq1auvvmq3fs6cObJYLOrZs+cdHdff31+JiYlas2aN3cMCz507p/T0dHXs2NE2T6Vy5cqSdMvt5AkJCQoMDNS8efPszs4sWbJEly5dUu/eve+oJ0ciIiLUqVMnvfHGG8rKyrKr3e6M0J49e1RQUHDL+i+++EI//PCDw0t4w4YNU05Ojh5//HEVFRWVegnrJovFonnz5mnatGlOeRgiYFY8VBAox0aOHHnbbfr06aMuXbroT3/6k06ePKnmzZtr48aNWrNmjSZNmmQ3GfnnmjVrljZt2qSOHTvqt7/9rSpUqKDFixersLDQ7vk3LVq0kL+/v/7yl7/o0qVLCgoKUteuXRUZGanU1FTNmDFDPXr00MMPP6yMjAy99tpratOmzS1PHC6refPmqWPHjnrggQc0btw41a1bVydPntQHH3ygffv2lfi95cuXa8WKFXrkkUfUqlUrBQYG6vDhw3rjjTdUsWJFPfvss7d8JykpSb/97W+1Zs0a1a5dW506dfpZPfbt21d9+/Yt6xCBcoGwA6BUfn5+Wrt2raZOnap33nlHaWlpqlOnjv7617/qqaeeKtM+mzRpok8//VSpqamaPXu2iouL1a5dO7311ltq166dbbvo6GgtWrRIs2fP1pgxY2S1WrV161ZFRkZq+vTpioiI0Kuvvqonn3xS4eHhGjdunF588UUFBAQ4ZezNmzfX7t27NWXKFC1cuFDXrl1TXFycBg0aVOr3Hn/8cQUHB2vLli1as2aN8vPzFRERocTERKWmpqply5a3fCckJER9+vTRqlWrNGTIkNve8QXg57MYnpw1CAAA4GLM2QEAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKbGc3YkFRcX68yZM6patSrPtgAAwEcYhqHLly8rJiZGfn4ln78h7Eg6c+aM7V08AADAt3z//feqVatWiXXCjqSqVatK+umHdfOdPAAAwLvl5+erdu3atr/HS0LY0f+9WTkkJISwAwCAj7ndFBQmKAMAAFMj7AAAAFMj7AAAAFNjzg4AAD7CarWqqKjI0224TUBAgPz9/e96P4QdAAC8nGEYys7OVl5enqdbcbuwsDBFR0ff1XPwCDsAAHi5m0EnMjJSwcHB5eIBuIZh6OrVq8rJyZEk1axZs8z7IuwAAODFrFarLehUr17d0+24VaVKlSRJOTk5ioyMLPMlLSYoAwDgxW7O0QkODvZwJ55xc9x3M1eJsAMAgA8oD5euHHHGuAk7AADA1Ag7AADA1Ag7AADA1Ag7AADA1Ag7AADAzrJly1S9enUVFhbare/Xr5+GDx/uoa7KjrADAADsDBw4UFarVWvXrrWty8nJ0QcffKDRo0d7sLOy4aGCAOBD+jz0sM7nnHdYi4iM0Lr1ax3WgDtRqVIlDR06VGlpaRo4cKAk6a233lJsbKw6d+7s2ebKgLADAD7kfM55jRgyzmFt2duvu7kbmNnYsWPVpk0bnT59Wr/4xS+0dOlSPfbYYz75vB/CDgAAuEXLli3VvHlzLVu2TImJiTp48KA++OADT7dVJoQdAADg0G9+8xvNnTtXp0+fVkJCgmrXru3plsqECcoAAMChoUOH6tSpU/rHP/7hkxOTbyLsAAAAh0JDQ5WUlKQqVaqoX79+nm6nzAg7AACgRKdPn9awYcMUFBTk6VbKjDk7AADgFhcvXtS2bdu0bds2vfbaa55u564QdgAAwC1atmypixcv6i9/+YsaNGjg6XbuCmEHAADc4uTJk55uwWmYswMAAEyNsAMAAEyNsAMAAEyNsAMAAEyNsAMAAEyNsAMAAEyNW88BAPBRWVlZunDhgluOVaNGDcXGxrrlWM5G2AEAwAdlZWWpUaNGunr1qluOFxwcrMOHD99V4Hnvvfe0aNEi7d27V7m5ufr666/VokUL5zVZAsIOAAA+6MKFC7p69aqWvDpPDe6916XHyjh2TGMm/E4XLly4q7BTUFCgjh07atCgQRo7dqwTOywdYQcAAB/W4N571bLZ/Z5u42cZPny4JPc/nZkJygAAwNQIOwAAwNQIOwAAwOlWrFihKlWq2D6ffvqpx3rxaNhZuHChmjVrppCQEIWEhCg+Pl4fffSRrX7t2jUlJyerevXqqlKlipKSknTu3Dm7fWRlZal3794KDg5WZGSkJk+erBs3brh7KAAA4D88/PDD2rdvn+3TunVrj/Xi0QnKtWrV0p///Gfde++9MgxDb775pvr27auvv/5aTZo00ZNPPqkPPvhAq1atUmhoqCZMmKD+/fvr888/lyRZrVb17t1b0dHR2rlzp86ePasRI0YoICBAL774oieHBgBAuVa1alVVrVrV021I8nDY6dOnj93yCy+8oIULF2r37t2qVauWlixZovT0dHXt2lWSlJaWpkaNGmn37t1q3769Nm7cqEOHDmnz5s2KiopSixYt9Pzzz+uZZ57R9OnTFRgY6IlhAYBHHD12VO3bxjusRURGaN36tW7uCLCXm5urrKwsnTlzRpKUkZEhSYqOjlZ0dLTLjus1t55brVatWrVKBQUFio+P1969e1VUVKSEhATbNg0bNlRsbKx27dql9u3ba9euXbr//vsVFRVl26Z79+4aP368Dh48qJYtWzo8VmFhoQoLC23L+fn5rhsYANyBPg89rPM550usHzt+rMSa9YZVI4aMc1hb9vbrd90bvFPGsZL/m/C2Y6xdu1ajRo2yLQ8ePFiSNG3aNE2fPt0px3DE42Hn22+/VXx8vK5du6YqVapo9erVaty4sfbt26fAwECFhYXZbR8VFaXs7GxJUnZ2tl3QuVm/WSvJ7NmzNWPGDOcOBACc4HzO+RIDiySlTn3Sjd3Am9WoUUPBwcEaM+F3bjlecHCwatSocVf7eOyxx/TYY485p6E74PGw06BBA+3bt0+XLl3Su+++q5EjR2r79u0uPWZqaqpSUlJsy/n5+apdu7ZLjwkAgDPFxsbq8OHDvBvrZ/B42AkMDNQ999wjSWrVqpW+/PJL/f3vf9f//M//6Pr168rLy7M7u3Pu3Dnbdb3o6Gh98cUXdvu7ebdWadf+goKCFBQU5OSRAADgXrGxsT4bQNzJ42HnvxUXF6uwsFCtWrVSQECAtmzZoqSkJEk/TWTKyspSfPxPE/Di4+P1wgsvKCcnR5GRkZKkTZs2KSQkRI0bN/bYGAB4l4EDBijvYq7DWli1cK169103dwTAnTwadlJTU9WzZ0/Fxsbq8uXLSk9P17Zt2/Txxx8rNDRUY8aMUUpKisLDwxUSEqKJEycqPj5e7du3lyQlJiaqcePGGj58uF566SVlZ2frueeeU3JyMmduANjkXczVmhXLHdb6Dhvu5m4AuJtHw05OTo5GjBihs2fPKjQ0VM2aNdPHH3+sX//615KkOXPmyM/PT0lJSSosLFT37t312muv2b7v7++v9evXa/z48YqPj1flypU1cuRIzZw501NDAoDbKu2Oq9LutgJQNh4NO0uWLCm1XrFiRS1YsEALFiwocZu4uDh9+OGHzm4NgAeUl8tNpd1xxd1WgPN53ZwdAOUXl5sAuAJhB4BTlXZ2RjLXGRoAvoGwA8CpSjs7I3GGBoD7EXYAAPBRWVlZPFTwZyDsAPAJGRlH9etuXR3WuDSG8igrK0sNGzbUjz/+6JbjVapUSUeOHLmjwLNjxw799a9/1d69e3X27FmtXr1a/fr1c12TJSDsAPARBpOXgf9w4cIF/fjjjxo5bKyio2Jceqzsc2f05op/6MKFC3cUdgoKCtS8eXONHj1a/fv3d2GHpSPsAADgw6KjYhRbK87TbTjUs2dP9ezZ09NtEHYA+L7SLnFJUmZmphu7AeBtCDsATKDkS1yS1LB1Wzf2AsDb+Hm6AQAAAFci7AAAAFPjMhYAuAAv+wS8B2EHAFyAl30C0pUrV3T8+HHb8okTJ7Rv3z6Fh4e79QGFhB0AAHxY9rkzXnuMr776Sl26dLEtp6SkSJJGjhyppUuXOqO1n4WwAwCAD6pRo4YqVaqkN1f8wy3Hq1SpkmrUqHFH3+ncubMMw3BRRz8fYQcAAB8UGxurI0eO8G6sn4GwAwCAj4qNjfXZAOJO3HoOAABMjbADAABMjbADAIAP8IaJvp7gjHETdgAA8GIBAQGSpKtXr3q4E8+4Oe6bP4eyYIIyAABezN/fX2FhYcrJyZEkBQcHy2KxeLgr1zMMQ1evXlVOTo7CwsLk7+9f5n0RdgAA8HLR0dGSZAs85UlYWJht/GVF2AGAMijt3VcS77+Cc1ksFtWsWVORkZEqKirydDtuExAQcFdndG4i7ABAGZT27iuJ91/BNfz9/Z3yl395wwRlAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgajxUEEC5lpFxVL/u1tVhLaxauFa9+66bOwLgbIQdAOWcoTUrljus9B023M29AHAFLmMBAABTI+wAAABTI+wAAABTI+wAAABTI+wAAABTI+wAAABTI+wAAABTI+wAAABT82jYmT17ttq0aaOqVasqMjJS/fr1U0ZGht02nTt3lsVisfs88cQTdttkZWWpd+/eCg4OVmRkpCZPnqwbN264cygAAMBLefQJytu3b1dycrLatGmjGzdu6Nlnn1ViYqIOHTqkypUr27YbO3asZs6caVsODg62/dpqtap3796Kjo7Wzp07dfbsWY0YMUIBAQF68cUX3ToeALdX2usZMjMz3dwNgPLAo2Fnw4YNdstLly5VZGSk9u7dq06dOtnWBwcHKzo62uE+Nm7cqEOHDmnz5s2KiopSixYt9Pzzz+uZZ57R9OnTFRgY6NIxALhTJb+eoWHrtm7uBUB54FXvxrp06ZIkKTw83G79ihUr9NZbbyk6Olp9+vTRlClTbGd3du3apfvvv19RUVG27bt3767x48fr4MGDatmy5S3HKSwsVGFhoW05Pz/fFcMBTGvggAHKu5jrsMbZGQDexmvCTnFxsSZNmqQOHTqoadOmtvVDhw5VXFycYmJi9M033+iZZ55RRkaG3nvvPUlSdna2XdCRZFvOzs52eKzZs2drxowZLhoJYH55F3M5OwPAZ3hN2ElOTtaBAwf02Wef2a0fN26c7df333+/atasqW7duikzM1P169cv07FSU1OVkpJiW87Pz1ft2rXL1jgAAPBqXnHr+YQJE7R+/Xpt3bpVtWrVKnXbdu3aSZKOHz8uSYqOjta5c+fstrm5XNI8n6CgIIWEhNh9AACAOXk07BiGoQkTJmj16tX65JNPVLdu3dt+Z9++fZKkmjVrSpLi4+P17bffKicnx7bNpk2bFBISosaNG7ukbwAA4Ds8ehkrOTlZ6enpWrNmjapWrWqbYxMaGqpKlSopMzNT6enp6tWrl6pXr65vvvlGTz75pDp16qRmzZpJkhITE9W4cWMNHz5cL730krKzs/Xcc88pOTlZQUFBnhweAADwAh49s7Nw4UJdunRJnTt3Vs2aNW2fd955R5IUGBiozZs3KzExUQ0bNtRTTz2lpKQkrVu3zrYPf39/rV+/Xv7+/oqPj9ejjz6qESNG2D2XBwAAlF8ePbNjGEap9dq1a2v79u233U9cXJw+/PBDZ7UFAABMxCsmKAMAALgKYQcAAJgaYQcAAJgaYQcAAJia1zxBGQDgOkePHVX7tvEOaxGREVq3fq2bOwLch7ADAOWA9YZVI4aMc1hb9vbrbu4GcC8uYwEAAFMj7AAAAFMj7AAAAFMj7AAAAFMj7AAAAFPjbiwADg0cMEB5F3Md1jIzM93cDVyptNvSJW5Nh+8j7ABwKO9irtasWO6w1rB1Wzd3A1cq7bZ0iVvT4fu4jAUAAEyNsAMAAEyNy1gAUIKMjKP6dbeuDmtZ3590bzMAyoywAwAlMkqct3TfA23c3AuAsuIyFgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDUeKgjA9AoLC7Xkn0tKrAEwN8IOANMzDEPNm7YqsQbA3LiMBQAATI2wAwAATI2wAwAATI2wAwAATI0JygCAMuvz0MM6n3PeYS0iMkLr1q91c0fArQg7AIAyO59zXiOGjHNYW/b2627uBnCMy1gAAMDUCDsAAMDUCDsAAMDUCDsAAMDUmKAMAGVw9cerWrxkbol1Q1b3NQOgVIQdAOVacbFR4ktCf7z6Y4k1P4uf5s96scT9PjLqUaf0B+DuEXYAlHOlvCS0lJrEC0QBX8GcHQAAYGqEHQAAYGqEHQAAYGoeDTuzZ89WmzZtVLVqVUVGRqpfv37KyMiw2+batWtKTk5W9erVVaVKFSUlJencuXN222RlZal3794KDg5WZGSkJk+erBs3brhzKAAAwEt5NOxs375dycnJ2r17tzZt2qSioiIlJiaqoKDAts2TTz6pdevWadWqVdq+fbvOnDmj/v372+pWq1W9e/fW9evXtXPnTr355ptaunSppk6d6okhAQAAL+PRu7E2bNhgt7x06VJFRkZq79696tSpky5duqQlS5YoPT1dXbt2lSSlpaWpUaNG2r17t9q3b6+NGzfq0KFD2rx5s6KiotSiRQs9//zzeuaZZzR9+nQFBgZ6YmgAAMBLeNWcnUuXLkmSwsPDJUl79+5VUVGREhISbNs0bNhQsbGx2rVrlyRp165duv/++xUVFWXbpnv37srPz9fBgwcdHqewsFD5+fl2HwAAYE5eE3aKi4s1adIkdejQQU2bNpUkZWdnKzAwUGFhYXbbRkVFKTs727bNfwadm/WbNUdmz56t0NBQ26d27dpOHg0AAPAWXhN2kpOTdeDAAa1cudLlx0pNTdWlS5dsn++//97lxwQAAJ7hFU9QnjBhgtavX68dO3aoVq1atvXR0dG6fv268vLy7M7unDt3TtHR0bZtvvjiC7v93bxb6+Y2/y0oKEhBQUFOHgXgWwYOGKC8i7kl1jMzM93YDQC4jkfDjmEYmjhxolavXq1t27apbt26dvVWrVopICBAW7ZsUVJSkiQpIyNDWVlZio+PlyTFx8frhRdeUE5OjiIjIyVJmzZtUkhIiBo3buzeAQE+JO9irtasWF5ivWHrtm7sBgBcx6NhJzk5Wenp6VqzZo2qVq1qm2MTGhqqSpUqKTQ0VGPGjFFKSorCw8MVEhKiiRMnKj4+Xu3bt5ckJSYmqnHjxho+fLheeuklZWdn67nnnlNycjJnbwDACY4eO6r2beMd1o4dP+bmboA759Gws3DhQklS586d7danpaXpsccekyTNmTNHfn5+SkpKUmFhobp3767XXnvNtq2/v7/Wr1+v8ePHKz4+XpUrV9bIkSM1c+ZMdw0DAEzNesOqEUPGOaylTn3Szd0Ad87jl7Fup2LFilqwYIEWLFhQ4jZxcXH68MMPndkaAAAwCa+5GwsAAMAVCDsAAMDUCDsAAMDUCDsAAMDUCDsAAMDUCDsAAMDUCDsAAMDUyhR26tWrpx9++OGW9Xl5eapXr95dNwUAAOAsZQo7J0+elNVqvWV9YWGhTp8+fddNAQAAOMsdPUF57dq1tl9//PHHCg0NtS1brVZt2bJFderUcVpzAADfVdo7tSIiI7Ru/VqHNcDZ7ijs9OvXT5JksVg0cuRIu1pAQIDq1Kmjv/3tb05rDgDgu0p7p9ayt193czcoz+4o7BQXF0uS6tatqy+//FI1atRwSVMAAADOUqYXgZ44ccLZfQAAALhEmd96vmXLFm3ZskU5OTm2Mz43vfHGG3fdGAAAgDOUKezMmDFDM2fOVOvWrVWzZk1ZLBZn9wUAAOAUZQo7ixYt0tKlSzV8+HBn9wMAAOBUZQo7169f14MPPujsXgCgXDBk1eIlc0usAXCuMoWd3/zmN0pPT9eUKVOc3Q8AmF5AhQqaP+tFh7VHRj3q5m4A8ytT2Ll27Zpef/11bd68Wc2aNVNAQIBd/ZVXXnFKcwDwcxQWFmrJP5eUWC8uNtzYDQBvU6aw880336hFixaSpAMHDtjVmKwMwN0Mw1Dzpq1K28JtvQDwPmUKO1u3bnV2HwAAAC5RpheBAgAA+Ioyndnp0qVLqZerPvnkkzI3BAAA4ExlCjs35+vcVFRUpH379unAgQO3vCAUAADAk8oUdubMmeNw/fTp03XlypW7aggAAMCZnDpn59FHH+W9WAAAwKs4Nezs2rVLFStWdOYuAQAA7kqZLmP179/fbtkwDJ09e1ZfffUVT1UGAABepUxhJzQ01G7Zz89PDRo00MyZM5WYmOiUxgAAAJyhTGEnLS3N2X0AAAC4RJnCzk179+7V4cOHJUlNmjRRy5YtndIUAPg6q7XkN5sDcK8yhZ2cnBwNHjxY27ZtU1hYmCQpLy9PXbp00cqVKxUREeHMHgHA51So4F/im837j+bN5oA7lelurIkTJ+ry5cs6ePCgcnNzlZubqwMHDig/P1+/+93vnN0jAABAmZXpzM6GDRu0efNmNWrUyLaucePGWrBgAROUAQCAVynTmZ3i4mIFBATcsj4gIEDFxcV33RQAAICzlCnsdO3aVb///e915swZ27rTp0/rySefVLdu3ZzWHAAAwN0qU9h59dVXlZ+frzp16qh+/fqqX7++6tatq/z8fM2fP9/ZPQIAAJRZmebs1K5dW//+97+1efNmHTlyRJLUqFEjJSQkOLU5AACAu3VHZ3Y++eQTNW7cWPn5+bJYLPr1r3+tiRMnauLEiWrTpo2aNGmiTz/91FW9AgAA3LE7OrMzd+5cjR07ViEhIbfUQkND9fjjj+uVV17RL3/5S6c1CKDsBg4YoLyLuQ5rmZmZbu4GADzjjsLO/v379Ze//KXEemJiol5++eW7bgqAc+RdzNWaFcsd1hq2buvmbgDAM+7oMta5c+cc3nJ+U4UKFXT+/Pm7bgoAAMBZ7ijs/OIXv9CBAwdKrH/zzTeqWbPmXTcFAADgLHcUdnr16qUpU6bo2rVrt9R+/PFHTZs2TQ899NDP3t+OHTvUp08fxcTEyGKx6P3337erP/bYY7JYLHafHj162G2Tm5urYcOGKSQkRGFhYRozZoyuXLlyJ8MCAAAmdkdzdp577jm99957uu+++zRhwgQ1aNBAknTkyBEtWLBAVqtVf/rTn372/goKCtS8eXONHj1a/fv3d7hNjx49lJaWZlsOCgqyqw8bNkxnz57Vpk2bVFRUpFGjRmncuHFKT0+/k6EBAACTuqOwExUVpZ07d2r8+PFKTU2VYRiSJIvFou7du2vBggWKior62fvr2bOnevbsWeo2QUFBio6Odlg7fPiwNmzYoC+//FKtW7eWJM2fP1+9evXSyy+/rJiYmJ/dCwAAMKc7fqhgXFycPvzwQ128eFHHjx+XYRi69957Va1aNVf0p23btikyMlLVqlVT165dNWvWLFWvXl2StGvXLoWFhdmCjiQlJCTIz89Pe/bs0SOPPOJwn4WFhSosLLQt5+fnu6R3AADgeWV6grIkVatWTW3atHFmL7fo0aOH+vfvr7p16yozM1PPPvusevbsqV27dsnf31/Z2dmKjIy0+06FChUUHh6u7OzsEvc7e/ZszZgxw6W9A3CuwsJCLfnnEoe14mLDzd0A8CVlDjvuMHjwYNuv77//fjVr1kz169fXtm3b7uqFo6mpqUpJSbEt5+fnq3bt2nfVKwDXMgxDzZu2Kqnq1l4A+BavDjv/rV69eqpRo4aOHz+ubt26KTo6Wjk5OXbb3LhxQ7m5uSXO85F+mgf03xOdAV/FU5IBoHQ+FXZOnTqlH374wfYsn/j4eOXl5Wnv3r1q1eqnf/F98sknKi4uVrt27TzZKuA2PCUZAErn0bBz5coVHT9+3LZ84sQJ7du3T+Hh4QoPD9eMGTOUlJSk6OhoZWZm6umnn9Y999yj7t27S/rpTes9evTQ2LFjtWjRIhUVFWnChAkaPHgwd2IBAABJd/hQQWf76quv1LJlS7Vs2VKSlJKSopYtW2rq1Kny9/fXN998o4cfflj33XefxowZo1atWunTTz+1uwS1YsUKNWzYUN26dVOvXr3UsWNHvf76654aEgAA8DIePbPTuXNn27N6HPn4449vu4/w8HAeIAjANKxWqxYvmVti3ZDVfc0AJuFTc3YAwOwqVPDX/Fkvllh/ZNSjbuzGdY4eO6r2beNLrEdERmjd+rVu7AhmRtgBALid9YZVI4aMK7G+7G2mI8B5PDpnBwAAwNUIOwAAwNQIOwAAwNQIOwAAwNQIOwAAwNQIOwAAwNQIOwAAwNQIOwAAwNQIOwAAwNQIOwAAwNQIOwAAwNQIOwAAwNQIOwAAwNQIOwAAwNQqeLoBALc3cMAA5V3MdVjLzMx0czcA4FsIO4APyLuYqzUrljusNWzd1s3dAIBv4TIWAAAwNc7sAPAahYWFWvLPJQ5rxcWGm7sBYBaEHQBewzAMNW/aqqSqW3sBYB5cxgIAAKZG2AEAAKbGZSwAgNc5euyo2reNd1iLiIzQuvVr3dwRfBlhBwDgdaw3rBoxZJzD2rK3X3dzN/B1XMYCAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmxlvPAcAkDFm1eMncEmtAeUXYAQCTCKhQQfNnveiw9sioR93cDeA9uIwFAABMjbADAABMjbADAABMzaNhZ8eOHerTp49iYmJksVj0/vvv29UNw9DUqVNVs2ZNVapUSQkJCTp27JjdNrm5uRo2bJhCQkIUFhamMWPG6MqVK24cBQAA8GYenaBcUFCg5s2ba/To0erfv/8t9Zdeeknz5s3Tm2++qbp162rKlCnq3r27Dh06pIoVK0qShg0bprNnz2rTpk0qKirSqFGjNG7cOKWnp7t7OADgclZryXdcAXDMo2GnZ8+e6tmzp8OaYRiaO3eunnvuOfXt21eStGzZMkVFRen999/X4MGDdfjwYW3YsEFffvmlWrduLUmaP3++evXqpZdfflkxMTFuGwsAuEOFCv4l3nHVfzR3XAGOeO2cnRMnTig7O1sJCQm2daGhoWrXrp127dolSdq1a5fCwsJsQUeSEhIS5Ofnpz179pS478LCQuXn59t9AACAOXlt2MnOzpYkRUVF2a2Pioqy1bKzsxUZGWlXr1ChgsLDw23bODJ79myFhobaPrVr13Zy9wAAwFuUy4cKpqamKiUlxbacn59P4IFHDRwwQHkXc0usZ2ZmurEbADAXrw070dHRkqRz586pZs2atvXnzp1TixYtbNvk5OTYfe/GjRvKzc21fd+RoKAgBQUFOb9poIzyLuZqzYrlJdYbtm7rxm4AwFy89jJW3bp1FR0drS1bttjW5efna8+ePYqPj5ckxcfHKy8vT3v37rVt88knn6i4uFjt2rVze88AAMD7ePTMzpUrV3T8+HHb8okTJ7Rv3z6Fh4crNjZWkyZN0qxZs3Tvvffabj2PiYlRv379JEmNGjVSjx49NHbsWC1atEhFRUWaMGGCBg8ezJ1Y8DqlXariMhUAuI5Hw85XX32lLl262JZvzqMZOXKkli5dqqeffloFBQUaN26c8vLy1LFjR23YsMH2jB1JWrFihSZMmKBu3brJz89PSUlJmjdvntvHAtxOaZequEwFAK7j0bDTuXNnGYZRYt1isWjmzJmaOXNmiduEh4fzAEEAAFAir52gDACAI0ePHVX7tvEOaxGREVq3fq2bO4K3I+wAAHyK9YZVI4aMc1hb9vbrbu4GvsBr78YCAABwBsIOAAAwNcIOAAAwNcIOAAAwNSYoAwBMo7Q7tSTu1iqvCDsAANMo7U4tibu1yisuYwEAAFMj7AAAAFMj7AAAAFMj7AAAAFMj7AAAAFMj7AAAAFPj1nMAblVYWKgl/1zisFZcbLi5GwDlAWEHgFsZhqHmTVuVVHVrLwDKBy5jAQAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAU+M5O4ATDRwwQHkXcx3WMjMz3dwNAEAi7ABOlXcxV2tWLHdYa9i6rZu7AQBIhB0AKBesVqsWL5nrsGbI6t5mADcj7ABwqtLefSXx/itPqVDBX/Nnveiw9sioR93cDeBehB0ATlX6u68k3n8FwN0IOwDuGG8uh686euyo2reNd1iLiIzQuvVr3dwR3IGwA+CO8eZy+CrrDatGDBnnsLbs7dfd3A3chefsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAU+NFoMAdGjhggPIu5jqsZWZmurkbAMDteHXYmT59umbMmGG3rkGDBjpy5Igk6dq1a3rqqae0cuVKFRYWqnv37nrttdcUFRXliXZRTuRdzNWaFcsd1hq2buvmbgAAt+P1l7GaNGmis2fP2j6fffaZrfbkk09q3bp1WrVqlbZv364zZ86of//+HuwWAAB4G68+syNJFSpUUHR09C3rL126pCVLlig9PV1du3aVJKWlpalRo0bavXu32rdv7+5WAcCUDFm1eMncEmuAt/P6sHPs2DHFxMSoYsWKio+P1+zZsxUbG6u9e/eqqKhICQkJtm0bNmyo2NhY7dq1i7AD3EZhYaGW/HNJqXVAkgIqVND8WS86rD0y6lE3dwPcOa8OO+3atdPSpUvVoEEDnT17VjNmzNAvf/lLHThwQNnZ2QoMDFRYWJjdd6KiopSdnV3qfgsLC+3+IM/Pz3dF+4BXMwxDzZu2KrUOAGbg1WGnZ8+etl83a9ZM7dq1U1xcnP71r3+pUqVKZd7v7Nmzb5n4DAAAzMnrJyj/p7CwMN133306fvy4oqOjdf36deXl5dltc+7cOYdzfP5TamqqLl26ZPt8//33LuwaAAB4klef2flvV65cUWZmpoYPH65WrVopICBAW7ZsUVJSkiQpIyNDWVlZio+PL3U/QUFBCgoKckfL8EGlPUdH4lk6AOBrvDrs/OEPf1CfPn0UFxenM2fOaNq0afL399eQIUMUGhqqMWPGKCUlReHh4QoJCdHEiRMVHx/P5GTcldKeoyPxLB0A8DVeHXZOnTqlIUOG6IcfflBERIQ6duyo3bt3KyIiQpI0Z84c+fn5KSkpye6hggDuXnGxUeLdWsXFTF4G4Du8OuysXLmy1HrFihW1YMECLViwwE0dAeVJaXdrEXYA+A6fmqAMAABwpwg7AADA1Ag7AADA1Ag7AADA1Lx6gjIAwPWs1pJf9AmYAWEHAMq5ChX8S3zRpyT1H83LPuHbuIwFAABMjbADAABMjctYMK3S3nEVVi1cq959180dAQA8gbAD0yrtHVd9hw13czcAvN3RY0fVvq3jF0lHREZo3fq1bu4IzkLYAQBAkvWGVSOGjHNYW/b2627uBs5E2EG5lJFxVL/u1tVhLTMz083dAABcibCDcsoo8RJXw9Zt3dwLAG/HJS7fRtgBAOA2uMTl2wg7AACXMFTyk5kNWd3bDMo1wg4AwCUCKlQo8cnMj4ziqcxwHx4qCAAATI2wAwAATI3LWACAMuON6fAFhB0AQJmV9sZ03pYOb8FlLAAAYGqc2YHPKu1FnxJPQgYA/ISwA59V2os+JZ6ELEmFhYVa8s8lDmvFxYabuwEAzyDsACZmGIaaN21VUtWtvQCApzBnBwAAmBphBwAAmBphBwAAmBpzdgAAuAtHjx1V+7bxJda/y/pOcbFxDmsRkRFat36tq1rD/4+wAwDAXbDesGrEkHEl1lOnPllifdnbr7uqLfwHwg7g47i9HABKR9gBfBy3lwNA6Qg78GqlPSW5PD0hmbM3MJvbvUDUkNV9zcD0CDvwaqU9Jbk8PSGZszcwm9JeICpJj4ziJaJwHm49BwAApkbYAQAApkbYAQAApkbYAQAApsYEZcALlHa3lcQdV0B51Oehh3U+57zDGk9evjOEHbhFabeQh1UL16p333VzR96l9LutJO64Asqf8znnefKykxB24Bal3ULed9hwN3fjGTwrB/j5SnsOD8/gwZ0i7ABuwrNygJ+vtOfw8Awe3CnCDuBEnL0BvJsh7zpjVNob048dP+bmbsyLsAOPy8g4ql936+qw5muvhODsDeDdAipU8KozRqW9MT116pMlfq+0kCQxgfm/mSbsLFiwQH/961+VnZ2t5s2ba/78+Wrbtvy8TsC3GbwSAoBTlHbmxlPHdMUZo9JCklT6BObyeJeXKcLOO++8o5SUFC1atEjt2rXT3Llz1b17d2VkZCgyMtLT7ZULpd1tJfneGRoA3qu0yculnbmRpP6jy3b2prRAc7tjeuKM0e0ujz0/5a8Oa8/N/EOZzxh5c4gyRdh55ZVXNHbsWI0aNUqStGjRIn3wwQd644039Mc//tHD3ZUPpd1tJXGGBoDzlDZ5uaxh5nZKCzSuOubdKOvlsbs5Y+TNt8r7fNi5fv269u7dq9TUVNs6Pz8/JSQkaNeuXR7szDeVl+fhlDaRuLCwsMzfZRIyAHgfnw87Fy5ckNVqVVRUlN36qKgoHTlyxOF3CgsL7f5Cu3TpkiQpPz/f6f2NGD5cl/IuOqyFhlXTsuUlnw3xhAvnc/T2Px0n8CG/GVfiz+jGjRvKv3y5xP0WFxeXWC9r7ccff9Srry4o8ZgFBVdLrBfdsOqeeo0c1qzWko95s17Sd4uNYhVcvVpCzShT7W6+623HdNV+y8sxXbXf8nLM29Vv3Lih117/m8OacRfHNAxDP1770W21n3oqeSzFxo0y7zfjaIZat2rjsPa//5tZ4ndvWG+45O/Ym/s0jNv8Q9PwcadPnzYkGTt37rRbP3nyZKNt27YOvzNt2jRDP90aw4cPHz58+PDx8c/3339falbw+TM7NWrUkL+/v86dO2e3/ty5c4qOjnb4ndTUVKWkpNiWi4uLlZubq+rVq8tisTitt/z8fNWuXVvff/+9QkJCnLZfb1bexlzexisx5vIw5vI2Xokx++qYDcPQ5cuXFRMTU+p2Ph92AgMD1apVK23ZskX9+vWT9FN42bJliyZMmODwO0FBQQoKCrJbFxYW5rIeQ0JCfPY/pLIqb2Mub+OVGHN5UN7GKzFmXxQaGnrbbXw+7EhSSkqKRo4cqdatW6tt27aaO3euCgoKbHdnAQCA8ssUYed//ud/dP78eU2dOlXZ2dlq0aKFNmzYcMukZQAAUP6YIuxI0oQJE0q8bOUpQUFBmjZt2i2XzMysvI25vI1XYszlQXkbr8SYzc5iGLe7XwsAAMB3+Xm6AQAAAFci7AAAAFMj7AAAAFMj7AAAAFMj7LhZYWGhWrRoIYvFon379nm6HZd6+OGHFRsbq4oVK6pmzZoaPny4zpw54+m2XObkyZMaM2aM6tatq0qVKql+/fqaNm2arl+/7unWXOaFF17Qgw8+qODgYJc+mNOTFixYoDp16qhixYpq166dvvjiC0+35DI7duxQnz59FBMTI4vFovfff9/TLbnc7Nmz1aZNG1WtWlWRkZHq16+fMjIyPN2WyyxcuFDNmjWzPUgwPj5eH330kafbcjnCjps9/fTTt32stVl06dJF//rXv5SRkaH/9//+nzIzMzVgwABPt+UyR44cUXFxsRYvXqyDBw9qzpw5WrRokZ599llPt+Yy169f18CBAzV+/HhPt+IS77zzjlJSUjRt2jT9+9//VvPmzdW9e3fl5OR4ujWXKCgoUPPmzbVgQckv2DWb7du3Kzk5Wbt379amTZtUVFSkxMREFRQUeLo1l6hVq5b+/Oc/a+/evfrqq6/UtWtX9e3bVwcPHvR0a67lnNdx4uf48MMPjYYNGxoHDx40JBlff/21p1tyqzVr1hgWi8W4fv26p1txm5deesmoW7eup9twubS0NCM0NNTTbThd27ZtjeTkZNuy1Wo1YmJijNmzZ3uwK/eQZKxevdrTbbhdTk6OIcnYvn27p1txm2rVqhn//Oc/Pd2GS3Fmx03OnTunsWPHavny5QoODvZ0O26Xm5urFStW6MEHH1RAQICn23GbS5cuKTw83NNtoAyuX7+uvXv3KiEhwbbOz89PCQkJ2rVrlwc7gytdunRJksrF/7dWq1UrV65UQUGB4uPjPd2OSxF23MAwDD322GN64okn1Lp1a0+341bPPPOMKleurOrVqysrK0tr1qzxdEtuc/z4cc2fP1+PP/64p1tBGVy4cEFWq/WW185ERUUpOzvbQ13BlYqLizVp0iR16NBBTZs29XQ7LvPtt9+qSpUqCgoK0hNPPKHVq1ercePGnm7LpQg7d+GPf/yjLBZLqZ8jR45o/vz5unz5slJTUz3d8l37uWO+afLkyfr666+1ceNG+fv7a8SIETJ87KHddzpmSTp9+rR69OihgQMHauzYsR7qvGzKMl7ADJKTk3XgwAGtXLnS0624VIMGDbRv3z7t2bNH48eP18iRI3Xo0CFPt+VSvC7iLpw/f14//PBDqdvUq1dPgwYN0rp162SxWGzrrVar/P39NWzYML355puubtVpfu6YAwMDb1l/6tQp1a5dWzt37vSpU6Z3OuYzZ86oc+fOat++vZYuXSo/P9/6N0VZfo+XLl2qSZMmKS8vz8Xduc/169cVHBysd999V/369bOtHzlypPLy8kx/ltJisWj16tV2YzezCRMmaM2aNdqxY4fq1q3r6XbcKiEhQfXr19fixYs93YrLmOZFoJ4QERGhiIiI2243b948zZo1y7Z85swZde/eXe+8847atWvnyhad7ueO2ZHi4mJJP91+70vuZMynT59Wly5d1KpVK6Wlpflc0JHu7vfYTAIDA9WqVStt2bLF9hd+cXGxtmzZ4nUvHUbZGYahiRMnavXq1dq2bVu5CzrST/9d+9qfy3eKsOMGsbGxdstVqlSRJNWvX1+1atXyREsut2fPHn355Zfq2LGjqlWrpszMTE2ZMkX169f3qbM6d+L06dPq3Lmz4uLi9PLLL+v8+fO2WnR0tAc7c52srCzl5uYqKytLVqvV9uyoe+65x/bfuS9LSUnRyJEj1bp1a7Vt21Zz585VQUGBRo0a5enWXOLKlSs6fvy4bfnEiRPat2+fwsPDb/lzzCySk5OVnp6uNWvWqGrVqrb5WKGhoapUqZKHu3O+1NRU9ezZU7Gxsbp8+bLS09O1bds2ffzxx55uzbU8ei9YOXXixAnT33r+zTffGF26dDHCw8ONoKAgo06dOsYTTzxhnDp1ytOtuUxaWpohyeHHrEaOHOlwvFu3bvV0a04zf/58IzY21ggMDDTatm1r7N6929MtuczWrVsd/n6OHDnS0625TEn/z6alpXm6NZcYPXq0ERcXZwQGBhoRERFGt27djI0bN3q6LZdjzg4AADA135tQAAAAcAcIOwAAwNQIOwAAwNQIOwAAwNQIOwAAwNQIOwAAwNQIOwAAwNQIOwAAwNQIOwAAwNQIOwBwG0VFRZ5uAcBdIOwA8LjOnTtr4sSJmjRpkqpVq6aoqCj94x//sL10s2rVqrrnnnv00UcfSZIuXryoYcOGKSIiQpUqVdK9996rtLQ02/5OnTqlIUOGKDw8XJUrV1br1q21Z88eW33hwoWqX7++AgMD1aBBAy1fvtyuH4vFooULF+rhhx9W5cqV9cILL0iS1qxZowceeEAVK1ZUvXr1NGPGDN24ccMNPyEAd4OwA8ArvPnmm6pRo4a++OILTZw4UePHj9fAgQP14IMP6t///rcSExM1fPhwXb16VVOmTNGhQ4f00Ucf6fDhw1q4cKFq1Kgh6ac3d//qV7/S6dOntXbtWu3fv19PP/20iouLJUmrV6/W73//ez311FM6cOCAHn/8cY0aNUpbt26162f69Ol65JFH9O2332r06NH69NNPNWLECP3+97/XoUOHtHjxYi1dutQWhAB4L14ECsDjOnfuLKvVqk8//VSSZLVaFRoaqv79+2vZsmWSpOzsbNWsWVO7du3Siy++qBo1auiNN964ZV+vv/66/vCHP+jkyZMKDw+/pd6hQwc1adJEr7/+um3doEGDVFBQoA8++EDST2d2Jk2apDlz5ti2SUhIULdu3ZSammpb99Zbb+npp5/WmTNnnPODAOASnNkB4BWaNWtm+7W/v7+qV6+u+++/37YuKipKkpSTk6Px48dr5cqVatGihZ5++mnt3LnTtt2+ffvUsmVLh0FHkg4fPqwOHTrYrevQoYMOHz5st65169Z2y/v379fMmTNVpUoV22fs2LE6e/asrl69WrZBA3CLCp5uAAAkKSAgwG7ZYrHYrbNYLJKk4uJi9ezZU999950+/PBDbdq0Sd26dVNycrJefvllVapUySn9VK5c2W75ypUrmjFjhvr373/LthUrVnTKMQG4Bmd2APikiIgIjRw5Um+99Zbmzp1ruyzVrFkz7du3T7m5uQ6/16hRI33++ed26z7//HM1bty41OM98MADysjI0D333HPLx8+PP0oBb8aZHQA+Z+rUqWrVqpWaNGmiwsJCrV+/Xo0aNZIkDRkyRC+++KL69eun2bNnq2bNmvr6668VExOj+Ph4TZ48WYMGDVLLli2VkJCgdevW6b333tPmzZtve8yHHnpIsbGxGjBggPz8/LR//34dOHBAs2bNcsewAZQR/xwB4HMCAwOVmpqqZs2aqVOnTvL399fKlStttY0bNyoyMlK9evXS/fffrz//+c/y9/eXJPXr109///vf9fLLL6tJkyZavHix0tLS1Llz51KP2b17d61fv14bN25UmzZt1L59e82ZM0dxcXGuHi6Au8TdWAAAwNQ4swMAAEyNsAMAAEyNsAMAAEyNsAMAAEyNsAMAAEyNsAMAAEyNsAMAAEyNsAMAAEyNsAMAAEyNsAMAAEyNsAMAAEyNsAMAAEzt/wP8DVxdkqR68wAAAABJRU5ErkJggg==\n" + }, + "metadata": {} + } + ], + "source": [ + "import warnings\n", + "warnings.filterwarnings(\"ignore\", \"is_categorical_dtype\")\n", + "warnings.filterwarnings(\"ignore\", \"use_inf_as_na\")\n", + "\n", + "df = pd.DataFrame({'score': score, 'mscore': mscore, 'y': y})\n", + "\n", + "sns.histplot(df, x=\"score\", hue=\"y\").set_title(\"SVM\")\n", + "plt.show()\n", + "sns.histplot(df, x=\"mscore\", hue=\"y\").set_title(\"Monotonic SVM\")\n", + "plt.show()" + ] + } + ], + "metadata": { + "colab": { + "provenance": [] + }, + "kernelspec": { + "display_name": "Python 3", + "name": "python3" + }, + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} \ No newline at end of file diff --git a/doc/source/examples/Smooth_SVM_new.ipynb b/doc/source/examples/Smooth_SVM_new.ipynb new file mode 100644 index 0000000..c5ba24c --- /dev/null +++ b/doc/source/examples/Smooth_SVM_new.ipynb @@ -0,0 +1,1187 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "W9_lmFre4Rse" + }, + "source": [ + "# Smooth SVM\n", + "Smooth SVMs solve the following optimization problem:\n", + "$$\n", + " \\min_{\\mathbf{\\beta} \\in \\mathbb{R}^d} \\\n", + " \\frac{C}{n} \\sum_{i=1}^n V(y_i \\mathbf{\\beta}^\\intercal \\mathbf{x}_i) + \\frac{1}{2} \\| \\mathbf{\\beta} \\|_2^2\n", + "$$\n", + "where $\\mathbf{x}_i \\in \\mathbb{R}^d$ is a feature vector, $y_i \\in \\{-1, 1\\}$ is a binary label, and $V(·)$ is the smoothed hinge loss (shown here with a default smoothing parameter τ=1):\n", + "$$\n", + " V(z) = \\begin{cases}\n", + " 0 & \\text{if } z \\ge 1 \\\\\n", + " \\frac{(1-z)^2}{2} & \\text{if } 0 < z < 1 \\\\\n", + " \\frac{1}{2}- z & \\text{if } z \\le 0\n", + " \\end{cases}\n", + "$$\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "lO5s9H4tbAP4" + }, + "source": [ + "> **Note.** Since the smooth hinge loss is a plq function, we can optimize it using `rehline.plq_Ridge_Classifier`. \\\n", + "Moreover, this wrapper adapts the `plqERM_Ridge` into a classifier, compatible with the scikit-learn API." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "sRNXaXpO4Pt2" + }, + "outputs": [], + "source": [ + "## install rehline\n", + "%pip install rehline -q" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "id": "lRz5G7RAaFtj" + }, + "outputs": [], + "source": [ + "## set up plotting style\n", + "import seaborn as sns\n", + "import matplotlib.pyplot as plt\n", + "custom_palette = ['#FFE4E1', '#3D325C']\n", + "sns.set_palette(custom_palette)" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": { + "id": "ZawpNRuQR9Uu" + }, + "outputs": [], + "source": [ + "## simulate data\n", + "from sklearn.datasets import make_classification\n", + "from sklearn.preprocessing import StandardScaler\n", + "import numpy as np\n", + "\n", + "scaler = StandardScaler()\n", + "\n", + "n, d = 10000, 5\n", + "X, y = make_classification(n_samples=n, n_features=d, random_state=42)\n", + "y = 2*y - 1\n", + "X = scaler.fit_transform(X)" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 80 + }, + "id": "lskTdleDZwpa", + "outputId": "3408050f-4e9d-49a3-f702-8158500bf421" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "plq_Ridge_Classifier(loss={'name': 'sSVM'})" + ], + "text/html": [ + "
plq_Ridge_Classifier(loss={'name': 'sSVM'})
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
" + ] + }, + "metadata": {}, + "execution_count": 4 + } + ], + "source": [ + "## solve Smooth SVM via `plq_Ridge_Classifier`\n", + "from rehline import plq_Ridge_Classifier\n", + "\n", + "clf = plq_Ridge_Classifier(loss={'name': 'sSVM'}, C=1.0)\n", + "clf.fit(X=X, y=y)" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 472 + }, + "id": "g5DCBXT5Z5uR", + "outputId": "78025cf1-710b-4965-ac8c-e3ba6792716a" + }, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "
" + ], + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjsAAAHHCAYAAABZbpmkAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAANvZJREFUeJzt3X10FNX9x/HPbswjkATIE2ASUNAQngWEFFQCkRSjBUEtQikgomKgCr8ijVKIaEXQCiVGsK2CWlM8tCoUKBiCiJKIGgvl+ZQWfqEJSRYRwmMSNvP7w7K/xmQTSDa7m+H9OmfPcefenfnOHJSPM3futRiGYQgAAMCkrJ4uAAAAoCkRdgAAgKkRdgAAgKkRdgAAgKkRdgAAgKkRdgAAgKkRdgAAgKkRdgAAgKkRdgAAgKkRdgBcMywWi6ZPn+7pMgC4GWEHQK327Nmj++67T7GxsQoICFCHDh105513KiMjw9Ol1Sk3N1fp6ek6deqUy/dts9n0xBNPKC4uToGBgYqIiNCtt96qOXPm6OzZs6qsrFRYWJgGDx7sdB+GYSg6Olq33HKLJGnbtm2yWCyyWCz6wx/+UOtvBg0aJIvFou7du7v8nIBrAWEHQA25ubnq16+fdu/eralTp+rVV1/Vww8/LKvVqt/85jeeLq9Oubm5evbZZ10edk6ePKl+/frp7bffVkpKipYtW6ZZs2apc+fOWr58uU6cOCFfX1/df//9ys3N1f/+7//Wup/t27fr3//+t37yk59U2x4QEKCsrKwa/Y8eParc3FwFBAS49HyAa8l1ni4AgPf51a9+pZCQEH355ZcKDQ2t1lZaWuqZojzsjTfeUEFBgXbs2KEf/OAH1drKysrk5+cnSRo/frxWrFihP/7xj/rFL35RYz9ZWVmyWq0aO3Zste133XWX1q1bpxMnTigsLKxa/8jISHXp0kXffvttE5wZYH7c2QFQwz//+U9169atRtCRpIiIiGrfL4+DWbNmjeLj4xUYGKiEhATt2bNHkvT666+rc+fOCggI0JAhQ3T06NEa+1yzZo369u2rwMBAhYWF6Sc/+YkKCwtr9Nu6datuu+02tWjRQqGhoRo5cqQOHDjgaE9PT9fs2bMlSZ06dXI8Hvr+MT/88EN1795d/v7+6tatmzZt2nRF18THx0cDBw6s0RYcHOy48zJo0CB17Nix1rs0lZWV+tOf/qTExES1b9++WtvIkSPl7++vNWvWVNuelZWlBx54QD4+PvXWCKB2hB0ANcTGxio/P1979+69ov6ffvqp/ud//kcTJ05Uenq6Dhw4oLvvvluZmZlatmyZHn/8cc2ePVt5eXl66KGHqv121apVjr/MFy5cqKlTp+r999/X4MGDqz2K2rJli5KTk1VaWqr09HTNmjVLubm5GjRokCPMjB49Wg8++KAkacmSJXrnnXf0zjvvKDw83LGfzz77TI8//rjGjh2rxYsX6+LFixozZoy++eabeq+J3W7XO++8U2c/i8WicePGac+ePdq3b1+1tk2bNunkyZMaP358jd8FBQVp5MiR+uMf/+jYtnv3bu3bt0/jxo2r85gA6mEAwPd89NFHho+Pj+Hj42MkJCQYTz31lLF582ajoqKiRl9Jhr+/v3HkyBHHttdff92QZERFRRllZWWO7WlpaYYkR9+KigojIiLC6N69u3HhwgVHv/Xr1xuSjHnz5jm29e7d24iIiDC++eYbx7bdu3cbVqvV+OlPf+rY9tJLL1U7xvdr9fPzMw4fPlxtH5KMjIyMOq9JcXGxER4ebkgy4uLijMcee8zIysoyTp06VaPvvn37DElGWlpate1jx441AgICjNOnTzu2ffzxx4YkY82aNcb69esNi8ViFBQUGIZhGLNnzzZuuOEGwzAM44477jC6detWZ40AasedHQA13HnnncrLy9OPfvQj7d69W4sXL1ZycrI6dOigdevW1eg/bNgwdezY0fF9wIABkqQxY8aoVatWNbb/61//kiR99dVXKi0t1eOPP15tAG5KSori4uK0YcMGSdLx48e1a9cuTZo0SW3atHH069mzp+68805t3Ljxis8tKSlJN954Y7V9BAcHO2pyJjIyUrt379Zjjz2mb7/9VitWrNC4ceMUERGh5557ToZhOPrGx8erT58+Wr16tWPbuXPntG7dOt19990KDg6u9RjDhw9XmzZttHr1ahmGodWrVzvuVAFoOMIOgFr1799f77//vr799lt98cUXSktL05kzZ3Tfffdp//791frGxMRU+x4SEiJJio6OrnX75YG2l99Yuvnmm2scPy4uztFeV7+uXbvqxIkTOnfu3BWd1/drlaTWrVtf0eDfdu3aafny5Tp+/LgOHTqkZcuWKTw8XPPmzdMbb7xRre/48eN15MgR5ebmSvpunND58+drfYR12eW3ubKysrR9+3YdO3aMR1iACxB2ANTJz89P/fv31wsvvKDly5ersrKyxiBaZ4NnnW3/77sg7uaKmiwWi2666SbNmDFD27dvl9Vq1bvvvlutz4MPPiir1eoYqJyVlaXWrVvrrrvuqnPf48aN065du5Senq5evXopPj7+iusCUDvCDoAr1q9fP0nfPVZyhdjYWEnSoUOHarQdOnTI0V5Xv4MHDyosLEwtWrSQ9F0QcacbbrhBrVu3rnFN2rdvr8TERK1Zs0YlJSXKzs7Wfffd53hF3ZnBgwcrJiZG27Zt464O4CKEHQA1fPzxx7Xe6bg8Nqa2x0kN0a9fP0VERGjFihUqLy93bP/rX/+qAwcOKCUlRdJ3j4969+6tt956q9obWnv37tVHH31U7W7J5dDj6kkFd+7cWeujsi+++ELffPNNrddk/PjxKi0t1aOPPqrKyso6H2FdZrFYtGzZMs2fP18TJkxwSe3AtY5JBQHUMGPGDJ0/f1733nuv4uLiVFFRodzcXL333nvq2LGjJk+e7JLj+Pr6atGiRZo8ebLuuOMOPfjggyopKdFvfvMbdezYUTNnznT0femllzRixAglJCRoypQpunDhgjIyMhQSEqL09HRHv759+0qSnnnmGY0dO1a+vr665557HCGood555x29++67uvfee9W3b1/5+fnpwIEDevPNNxUQEKCnn366xm/GjBmjxx9/XGvXrlV0dLRuv/32KzrWyJEjNXLkyEbVC+D/EXYA1PDyyy9rzZo12rhxo37729+qoqJCMTExevzxxzV37txaJxtsqEmTJikoKEgvvvii5syZoxYtWujee+/VokWLqh0nKSlJmzZt0vz58zVv3jz5+vrqjjvu0KJFi9SpUydHv/79++u5557TihUrtGnTJlVVVenIkSONDjuPPvqogoKClJOTo7Vr16qsrEzh4eEaPny40tLS1KdPnxq/CQ4O1j333KM1a9bowQcfdPsjNgDfsRieHCkIAADQxBizAwAATI2wAwAATI2wAwAATI2wAwAATI2wAwAATI2wAwAATI15diRVVVWpqKhIrVq1Yh4MAACaCcMwdObMGbVv315Wq/P7N4QdSUVFRTVWZwYAAM3DsWPHdP311zttJ+xIatWqlaTvLlZwcLCHqwEAAFeirKxM0dHRjr/HnSHs6P9XSQ4ODibsAADQzNQ3BIUBygAAwNQIOwAAwNQIOwAAwNQYswMAQDNht9tVWVnp6TLcxtfXVz4+Po3eD2EHAAAvZxiGiouLderUKU+X4nahoaGKiopq1Dx4hB0AALzc5aATERGhoKCga2ICXMMwdP78eZWWlkqS2rVr1+B9EXYAAPBidrvdEXTatm3r6XLcKjAwUJJUWlqqiIiIBj/SYoAyAABe7PIYnaCgIA9X4hmXz7sxY5UIOwAANAPXwqOr2rjivAk7AADA1Ag7AADA1Ag7AADA1Ag7AADA1Ag7AACgmrfffltt27ZVeXl5te2jRo3ShAkTPFRVwxF2AABANffff7/sdrvWrVvn2FZaWqoNGzbooYce8mBlDcOkggDcamhiomw2m9P28PBwbf34YzdWBOD7AgMDNW7cOK1cuVL333+/JOkPf/iDYmJiNGTIEM8W1wCEHQBuZbPZtHPLZqftA5KS3VgNAGemTp2q/v37q7CwUB06dNCqVas0adKkZjnfD2EHAADU0KdPH/Xq1Utvv/22hg8frn379mnDhg2eLqtBCDsAvEphYaF6dO/utJ3HXID7PPzww1q6dKkKCwuVlJSk6OhoT5fUIIQdAF7FMKp4zAV4iXHjxunnP/+5fve73+ntt9/2dDkNxttYAACgViEhIRozZoxatmypUaNGebqcBiPsAAAApwoLCzV+/Hj5+/t7upQG4zEWAACo4dtvv9W2bdu0bds2vfbaa54up1EIOwBgQoMH3aaSkmKn7ZGRUfpsx6durAjNTZ8+ffTtt99q0aJFuvnmmz1dTqMQdgDAhEpKijUz9Wmn7UsyX3BjNWiOjh496ukSXIawAwDNUH13bgoLi9xYDeDdCDsA0AzVd+dm5pxpbqwG8G68jQUAAEyNsAMAAEyNsAMAAEyNsAMAAEyNAcoAADRTBQUFOnHihFuOFRYWppiYGLccy9UIOwBcamhiomw2m9P240W8Eg24QkFBgbp27arz58+75XhBQUE6cOBAowLP+++/rxUrVig/P18nT57U3/72N/Xu3dt1RTpB2AHgUjabrc5VyzvExbuxGsC8Tpw4ofPnz+uNV5fp5i5dmvRYh/7xD02Z/jOdOHGiUWHn3LlzGjx4sB544AFNnTrVhRXWjbADAEAzdnOXLurTs4eny7giEyZMkOT+2ZkZoAwAAEyNsAMAAEyNsAMAAFzu3XffVcuWLR2fTz/91GO1MGYHAAC43I9+9CMNGDDA8b1Dhw4eq4WwAwAAXK5Vq1Zq1aqVp8uQRNgBAABucvLkSRUUFKjoP/NtHTp0SJIUFRWlqKioJjsuYQcAgGbs0D/+0WyOsW7dOk2ePNnxfezYsZKk+fPnKz093SXHqA1hBwCAZigsLExBQUGaMv1nbjleUFCQwsLCGrWPSZMmadKkSa4p6CoQdgAAaIZiYmJ04MAB1sa6AoQdAACaqZiYmGYbQNyJeXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpMc8OgKsyNDFRNpvNafvx/6x5A6DpFRQUMKngFSDsALgqNptNO7dsdtreIS7ejdUA166CggLFxcXpwoULbjleYGCgDh48eFWBZ/v27XrppZeUn5+v48eP64MPPtCoUaOarkgnCDsA4IUGD7pNJSXFTtsLC7mDdq07ceKELly4oInjpyoqsn2THqu4pEhvvfs7nThx4qrCzrlz59SrVy899NBDGj16dBNWWDfCDgB4oZKSYs1Mfdpp+8w509xYDbxZVGR7xVwf6+kyajVixAiNGDHC02UwQBkAAJgbYQcAAJgaYQcAAJgaYQcAAJia14SdF198URaLRU8++aRj28WLF5Wamqq2bduqZcuWGjNmjEpKSqr9rqCgQCkpKQoKClJERIRmz56tS5cuubl6AADgrbzibawvv/xSr7/+unr27Flt+8yZM7VhwwatWbNGISEhmj59ukaPHq0dO3ZIkux2u1JSUhQVFaXc3FwdP35cP/3pT+Xr66sXXnjBE6cCAAD+4+zZszp8+LDj+5EjR7Rr1y61adPGrRMUejzsnD17VuPHj9fvfvc7Pf/8847tp0+f1htvvKGsrCwNHTpUkrRy5Up17dpVn3/+uQYOHKiPPvpI+/fv15YtWxQZGanevXvrueee05w5c5Seni4/Pz9PnRYA1Il5dOAqxSVN/2elocf46quvlJiY6Pg+a9YsSdLEiRO1atUqV5R2RTwedlJTU5WSkqKkpKRqYSc/P1+VlZVKSkpybIuLi1NMTIzy8vI0cOBA5eXlqUePHoqMjHT0SU5O1rRp07Rv3z716dOn1mOWl5ervLzc8b2srKwJzgwAnGMeHTRWWFiYAgMD9da7v3PL8QIDAxUWFnZVvxkyZIgMw2iiiq6cR8PO6tWr9fXXX+vLL7+s0VZcXCw/Pz+FhoZW2x4ZGani4mJHn/8OOpfbL7c5s3DhQj377LONrB4AAM+JiYnRwYMHWRvrCngs7Bw7dkxPPPGEsrOzFRAQ4NZjp6WlOW6lSd/d2YmOjnZrDYA3qm+RT4mFPgFvEhMT02wDiDt5LOzk5+ertLRUt9xyi2Ob3W7X9u3b9eqrr2rz5s2qqKjQqVOnqt3dKSkpUVRUlCQpKipKX3zxRbX9Xn5b63Kf2vj7+8vf39+FZwOYQ32LfEos9Amg+fHYq+fDhg3Tnj17tGvXLsenX79+Gj9+vOOffX19lZOT4/jNoUOHVFBQoISEBElSQkKC9uzZo9LSUkef7OxsBQcHKz6e/yADAAAP3tlp1aqVunfvXm1bixYt1LZtW8f2KVOmaNasWWrTpo2Cg4M1Y8YMJSQkaODAgZKk4cOHKz4+XhMmTNDixYtVXFysuXPnKjU1lTs3AABT8YaBvp7givP2+NtYdVmyZImsVqvGjBmj8vJyJScn67XXXnO0+/j4aP369Zo2bZoSEhLUokULTZw4UQsWLPBg1QAAuI6vr68k6fz58woMDPRwNe53/vx5Sf9/HRrCq8LOtm3bqn0PCAhQZmamMjMznf4mNjZWGzdubOLKAADwDB8fH4WGhjqGbAQFBclisXi4qqZnGIbOnz+v0tJShYaGysfHp8H78qqwAwAAarr80s1/j1G9VoSGhtb50tGVIOwAAODlLBaL2rVrp4iICFVWVnq6HLfx9fVt1B2dywg7AAA0Ez4+Pi75y/9a4zWrngMAADQF7uwAwDWosLBIXTp3qbNPZGSUPtvxqZsqApoOYQcArkF2u73OhUglaUnmC26qBmhaPMYCAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmxttYAJqVwsJC9ejevc4+4eHh2vrxx26qCIC3I+wAaFYMo0o7t2yus8+ApGQ3VQOgOeAxFgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDUmFQQAFxs86DaVlBTX2aewsMhN1QAg7ACAi5WUFGtm6tN19pk5Z5qbqgHAYywAAGBqhB0AAGBqhB0AAGBqhB0AAGBqhB0AAGBqhB0AAGBqvHoOAKhVYWGRunTu4rQ9MjJKn+341I0VAQ1D2AEA1Mput9c5X9CSzBfcWA3QcDzGAgAApsadHQCmU1hYqB7duzttDw8P19aPP3ZjRQA8ibADwHQMo0o7t2x22j4gKdmN1QDwNB5jAQAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAU7vO0wUAQHMzeNBtKikpdtpeWFjkxmoA1IewAwBXqaSkWDNTn3baPnPONDdWA6A+PMYCAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACm5tGws3z5cvXs2VPBwcEKDg5WQkKC/vrXvzraL168qNTUVLVt21YtW7bUmDFjVFJSUm0fBQUFSklJUVBQkCIiIjR79mxdunTJ3acCAAC8lEfDzvXXX68XX3xR+fn5+uqrrzR06FCNHDlS+/btkyTNnDlTf/nLX7RmzRp98sknKioq0ujRox2/t9vtSklJUUVFhXJzc/XWW29p1apVmjdvnqdOCQAAeJnrPHnwe+65p9r3X/3qV1q+fLk+//xzXX/99XrjjTeUlZWloUOHSpJWrlyprl276vPPP9fAgQP10Ucfaf/+/dqyZYsiIyPVu3dvPffcc5ozZ47S09Pl5+fnidMCAABexGvG7Njtdq1evVrnzp1TQkKC8vPzVVlZqaSkJEefuLg4xcTEKC8vT5KUl5enHj16KDIy0tEnOTlZZWVljrtDAADg2ubROzuStGfPHiUkJOjixYtq2bKlPvjgA8XHx2vXrl3y8/NTaGhotf6RkZEqLi6WJBUXF1cLOpfbL7c5U15ervLycsf3srIyF50NAADwNh6/s3PzzTdr165d2rlzp6ZNm6aJEydq//79TXrMhQsXKiQkxPGJjo5u0uMBAADP8XjY8fPzU+fOndW3b18tXLhQvXr10m9+8xtFRUWpoqJCp06dqta/pKREUVFRkqSoqKgab2dd/n65T23S0tJ0+vRpx+fYsWOuPSkAAOA1PB52vq+qqkrl5eXq27evfH19lZOT42g7dOiQCgoKlJCQIElKSEjQnj17VFpa6uiTnZ2t4OBgxcfHOz2Gv7+/43X3yx8AAGBOHh2zk5aWphEjRigmJkZnzpxRVlaWtm3bps2bNyskJERTpkzRrFmz1KZNGwUHB2vGjBlKSEjQwIEDJUnDhw9XfHy8JkyYoMWLF6u4uFhz585Vamqq/P39PXlqAADAS3g07JSWluqnP/2pjh8/rpCQEPXs2VObN2/WnXfeKUlasmSJrFarxowZo/LyciUnJ+u1115z/N7Hx0fr16/XtGnTlJCQoBYtWmjixIlasGCBp04JAAB4GY+GnTfeeKPO9oCAAGVmZiozM9Npn9jYWG3cuNHVpQEAAJPwujE7AAAAruTxeXYAAM1TYWGRunTu4rQ9MjJKn+341I0VAbUj7AAAGsRut2tm6tNO25dkvuDGagDneIwFAABMjTs7APA9gwfdppIS50vOFBYWubEaAI1F2AGA7ykpKa7z8czMOdPcWA2AxuIxFgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDWWiwCuIUMTE2Wz2Zy2Hy9izScA5kPYAa4hNptNO7dsdtreIS7ejdUAgHs06DHWDTfcoG+++abG9lOnTumGG25odFEAAACu0qCwc/ToUdnt9hrby8vLVVhY2OiiAAAAXOWqHmOtW7fO8c+bN29WSEiI47vdbldOTo46duzosuIAXB3G5ABATVcVdkaNGiVJslgsmjhxYrU2X19fdezYUb/+9a9dVhyAq8OYHACo6arCTlVVlSSpU6dO+vLLLxUWFtYkRQEAALhKg97GOnLkiKvrAAAAaBINfvU8JydHOTk5Ki0tddzxuezNN99sdGEAAACu0KCw8+yzz2rBggXq16+f2rVrJ4vF4uq6AAAAXKJBYWfFihVatWqVJkyY4Op6AAAAXKpB8+xUVFToBz/4gatrAQAAcLkGhZ2HH35YWVlZrq4FAADA5Rr0GOvixYv67W9/qy1btqhnz57y9fWt1v7KK6+4pDgAAIDGalDY+fvf/67evXtLkvbu3VutjcHKAADAmzQo7Hz88ceurgMAAKBJNGjMDgAAQHPRoDs7iYmJdT6u2rp1a4MLAgAAcKUGhZ3L43Uuq6ys1K5du7R3794aC4QCAAB4UoPCzpIlS2rdnp6errNnzzaqIAAAAFdy6Zidn/zkJ6yLBQAAvIpLw05eXp4CAgJcuUsAAIBGadBjrNGjR1f7bhiGjh8/rq+++kq//OUvXVIYAACAKzQo7ISEhFT7brVadfPNN2vBggUaPny4SwoDAABwhQaFnZUrV7q6DgAAgCbRoLBzWX5+vg4cOCBJ6tatm/r06eOSogAAAFylQWGntLRUY8eO1bZt2xQaGipJOnXqlBITE7V69WqFh4e7skYAAIAGa9DbWDNmzNCZM2e0b98+nTx5UidPntTevXtVVlamn/3sZ66uEQAAoMEadGdn06ZN2rJli7p27erYFh8fr8zMTAYoAwAAr9KgOztVVVXy9fWtsd3X11dVVVWNLgoAAMBVGhR2hg4dqieeeEJFRUWObYWFhZo5c6aGDRvmsuIAAAAaq0Fh59VXX1VZWZk6duyoG2+8UTfeeKM6deqksrIyZWRkuLpGAACABmvQmJ3o6Gh9/fXX2rJliw4ePChJ6tq1q5KSklxaHAAAQGNd1Z2drVu3Kj4+XmVlZbJYLLrzzjs1Y8YMzZgxQ/3791e3bt306aefNlWtAAAAV+2qws7SpUs1depUBQcH12gLCQnRo48+qldeecVlxQEAADTWVT3G2r17txYtWuS0ffjw4Xr55ZcbXRQAoPkrLCxSl85dnLZHRkbpsx08DUDTu6qwU1JSUusr546dXXedbDZbo4sCADR/drtdM1Ofdtq+JPMFN1aDa9lVPcbq0KGD9u7d67T973//u9q1a9foogAAAFzlqsLOXXfdpV/+8pe6ePFijbYLFy5o/vz5uvvuu11WHAAAQGNd1WOsuXPn6v3339dNN92k6dOn6+abb5YkHTx4UJmZmbLb7XrmmWeapFAAAICGuKqwExkZqdzcXE2bNk1paWkyDEOSZLFYlJycrMzMTEVGRjZJoQDgKoWFherRvbvTdput1I3VAGhqVz2pYGxsrDZu3Khvv/1Whw8flmEY6tKli1q3bt0U9QGAyxlGlXZu2ey0vf1NXZ22AWh+GjSDsiS1bt1a/fv3d2UtAAAALtegtbEAAACaC8IOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNY+GnYULF6p///5q1aqVIiIiNGrUKB06dKhan4sXLyo1NVVt27ZVy5YtNWbMGJWUlFTrU1BQoJSUFAUFBSkiIkKzZ8/WpUuX3HkqAADAS3k07HzyySdKTU3V559/ruzsbFVWVmr48OE6d+6co8/MmTP1l7/8RWvWrNEnn3yioqIijR492tFut9uVkpKiiooK5ebm6q233tKqVas0b948T5wSAADwMg2eVNAVNm3aVO37qlWrFBERofz8fN1+++06ffq03njjDWVlZWno0KGSpJUrV6pr1676/PPPNXDgQH300Ufav3+/tmzZosjISPXu3VvPPfec5syZo/T0dPn5+Xni1AAAgJfwqjE7p0+fliS1adNGkpSfn6/KykolJSU5+sTFxSkmJkZ5eXmSpLy8PPXo0aPamlzJyckqKyvTvn37aj1OeXm5ysrKqn0AAIA5eU3Yqaqq0pNPPqlBgwap+38W6CsuLpafn59CQ0Or9Y2MjFRxcbGjz/cXH738/XKf71u4cKFCQkIcn+joaBefDQAA8BZeE3ZSU1O1d+9erV69usmPlZaWptOnTzs+x44da/JjAgAAz/DomJ3Lpk+frvXr12v79u26/vrrHdujoqJUUVGhU6dOVbu7U1JSoqioKEefL774otr+Lr+tdbnP9/n7+8vf39/FZwEAALyRR+/sGIah6dOn64MPPtDWrVvVqVOnau19+/aVr6+vcnJyHNsOHTqkgoICJSQkSJISEhK0Z88elZaWOvpkZ2crODhY8fHx7jkRAADgtTx6Zyc1NVVZWVlau3atWrVq5RhjExISosDAQIWEhGjKlCmaNWuW2rRpo+DgYM2YMUMJCQkaOHCgJGn48OGKj4/XhAkTtHjxYhUXF2vu3LlKTU3l7g0AAPBs2Fm+fLkkaciQIdW2r1y5UpMmTZIkLVmyRFarVWPGjFF5ebmSk5P12muvOfr6+Pho/fr1mjZtmhISEtSiRQtNnDhRCxYscNdpAAAAL+bRsGMYRr19AgIClJmZqczMTKd9YmNjtXHjRleWBgAATMJr3sYCAABoCoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgatd5ugAAcLdLl+zKyMhw3m63u7EaAE2NsAPgGmQoof9tdbQ7D0JwncLCInXp3MVpe2RklD7b8akbK4JZEXYAAB5ht9s1M/Vpp+1LMl9wYzUwM8bsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAU2MGZaAZGZqYKJvN5rT9eFGRG6sBgOaBsAM0IzabTTu3bHba3iEu3o3VAEDzwGMsAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgasyzA3gRJg30DkZVlRa9ku603Wq1uK8YAI1G2AG8CJMGeger1ap3Ml5z2n7X+AfcWA2AxuIxFgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMLXrPF0AALjapUt2ZWRkOO9gNG7/RlWVFr2S7rTdarU07gAAXIqwA8CEDCX0v62O9jqC0BWwWq16J+M1p+13jX+gUfsH4Fo8xgIAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKbm0bCzfft23XPPPWrfvr0sFos+/PDDau2GYWjevHlq166dAgMDlZSUpH/84x/V+pw8eVLjx49XcHCwQkNDNWXKFJ09e9aNZwEAALyZR8POuXPn1KtXL2VmZtbavnjxYi1btkwrVqzQzp071aJFCyUnJ+vixYuOPuPHj9e+ffuUnZ2t9evXa/v27XrkkUfcdQoAAMDLeXSenREjRmjEiBG1thmGoaVLl2ru3LkaOXKkJOntt99WZGSkPvzwQ40dO1YHDhzQpk2b9OWXX6pfv36SpIyMDN111116+eWX1b59e7edCwAA8E5eO6ngkSNHVFxcrKSkJMe2kJAQDRgwQHl5eRo7dqzy8vIUGhrqCDqSlJSUJKvVqp07d+ree++tdd/l5eUqLy93fC8rK2u6EwH+Y2hiomw2W519jhcVuakawPsVFhapS+cudfaJjIzSZzs+dVNFaK68NuwUFxdLkiIjI6ttj4yMdLQVFxcrIiKiWvt1112nNm3aOPrUZuHChXr22WddXDFQN5vNpp1bNtfZp0NcvJuqab7qXQpCavRyEPAOdrtdM1OfrrPPkswX3FQNmjOvDTtNKS0tTbNmzXJ8LysrU3R0tAcrAnDl6lsKQmrschAAzMVrXz2PioqSJJWUlFTbXlJS4miLiopSaWlptfZLly7p5MmTjj618ff3V3BwcLUPAAAwJ6+9s9OpUydFRUUpJydHvXv3lvTdHZidO3dq2rRpkqSEhASdOnVK+fn56tu3ryRp69atqqqq0oABAzxVOgDATeob18OYHkgeDjtnz57V4cOHHd+PHDmiXbt2qU2bNoqJidGTTz6p559/Xl26dFGnTp30y1/+Uu3bt9eoUaMkSV27dtUPf/hDTZ06VStWrFBlZaWmT5+usWPH8iYWAFwD6hvXw5geSB4OO1999ZUSExMd3y+Po5k4caJWrVqlp556SufOndMjjzyiU6dOafDgwdq0aZMCAgIcv3n33Xc1ffp0DRs2TFarVWPGjNGyZcvcfi4AAMA7eTTsDBkyRIbh/LUJi8WiBQsWaMGCBU77tGnTRllZWU1RHgAAMAGvHaAMAADgCl47QBlobuqbNJAJAwHAMwg7gIvUN2kgEwYCgGfwGAsAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgay0UAgIsZVVVa9Ep6nX2sVot7igFA2AEAV7NarXon47U6+9w1/gE3VQOAx1gAAMDUuLMDXKGhiYmy2WxO248XFbmxGgDAlSLs4JpRX1gJDw/X1o8/dtpus9m0c8tmp+0d4uIbVR8AoGkQdnDNqC+sDEhKdmM1AAB3YcwOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNd7GAv6jsLBQPbp3d9rOPDoA0DwRdoD/MIwq5tGB29S3fhZrZwGuQ9gBAA+ob/0s1s4CXIewA8CrXLpkV0ZGhvMOhvtqAWAOhB0AXsZQQv/b6mivIwgB31NYWKQunbs4bY+MjNJnOz51Y0XwBMIOAMC07Ha7ZqY+7bR9SeYLbqwGnsKr5wAAwNQIOwAAwNQIOwAAwNQIOwAAwNQIOwAAwNR4GwuAWzGPDgB3I+wAcDPm0QHgXjzGAgAApkbYAQAApkbYAQAApkbYAQAApkbYAQAApsbbWACAaxarol8bCDsAgGsWq6JfGwg7MIWhiYmy2Wx19jleVOSmagAA3oSwg2ahvjBzvKhIBfv31LmPDnHxri4L8Bir1aJFr6TX2Q7gO4QdNAs2m007t2x22k6QgdkYVVV1hhkfH6veyXjNaftd4x9ogqqA5omwAwBeyGqtO8ykEGaAK0bYAQATqu/OEI+5cC0h7ACACdV3Z4jHXLiWEHYAuNSlS3ZlZNSxcrnhvloAQCLsALhK9YcZQwn9b6tjD3X8FgCaAGEHblHfq+Ph4eHa+vHHbqwIDUeYAdC8EHbgFvW9Oj4gKdmN1QAAriWEHXiFwsJC9eje3Wk7sx8DABqKsAOvYBhVTBoIAGgShB0AAJxgVXRzIOwAAOAEq6Kbg9XTBQAAADQl7uwAwDWovuUkJJaUgHkQdgA41DthoMQMyCZR33ISEktKXIn6xvRIUqnNpojwcKftjPtpeoQdAP+lvgkDJSYNvHawmGj96hvTI0kz50xj3I+HEXbgEvXNkMw8OUDzw2KiMAvCDlyivhmSmScHMJ/67vz4+Fi4MwSvYJqwk5mZqZdeeknFxcXq1auXMjIydOutt3q6LLdg3SkAnlDfnZ+U8Q9wZwhewRRh57333tOsWbO0YsUKDRgwQEuXLlVycrIOHTqkiIgIT5fX5Jp63an6wpTEYyoAgPcyRdh55ZVXNHXqVE2ePFmStGLFCm3YsEFvvvmmfvGLX3i4usZr6vEwV7L/gv176twHj6m8Q31vU12qvFT321a8aQU3YgD0d5iluek1+7BTUVGh/Px8paWlObZZrVYlJSUpLy/Pg5W5TlOPh2G8zZWpN0hcsruxGmfqe5sqo952wF1cMQDaam3+44KYpbnpNfuwc+LECdntdkVGRlbbHhkZqYMHD9b6m/LycpWXlzu+nz59WpJUVlbm8vruTknRiRMnnLaHhYVp/YYNde7Dbrer7MwZp+2GYdTZ/u9//1vxXbs6bT9WUKCXXn7ZaXtl5aU693+5T2P2Ud85NLb9imqsqKyz3aiqUs9ufets94breO78+SZrd8cxqNE72r2hhiq7XQtfnldnjVarRSsWOf93ZsxDE+rch8UiXbh4oc72un5vtdbdfiXHMAyjzvaqqqom+fvpagy/M1k2W6nTdtuJEwoPC3PaHh4eoY+ynf9PdUNdvi6GUc9taaOZKywsNCQZubm51bbPnj3buPXWW2v9zfz58w19d8OeDx8+fPjw4dPMP8eOHaszKzT7OzthYWHy8fFRSUlJte0lJSWKioqq9TdpaWmaNWuW43tVVZVOnjyptm3bymJp2C3PsrIyRUdH69ixYwoODm7QPq51XMPG4xo2Dtev8biGjcc1vHKGYejMmTNq3759nf2afdjx8/NT3759lZOTo1GjRkn6Lrzk5ORo+vTptf7G399f/v7+1baFhoa6pJ7g4GD+cDYS17DxuIaNw/VrPK5h43ENr0xISEi9fZp92JGkWbNmaeLEierXr59uvfVWLV26VOfOnXO8nQUAAK5dpgg7P/7xj2Wz2TRv3jwVFxerd+/e2rRpU41BywAA4NpjirAjSdOnT3f62Mod/P39NX/+/BqPx3DluIaNxzVsHK5f43ENG49r6HoWw6jvfS0AAIDmy+rpAgAAAJoSYQcAAJgaYQcAAJgaYQcAAJgaYaeJbNiwQQMGDFBgYKBat27tmPAQV6e8vFy9e/eWxWLRrl27PF1Os3H06FFNmTJFnTp1UmBgoG688UbNnz9fFRUVni7Nq2VmZqpjx44KCAjQgAED9MUXX3i6pGZj4cKF6t+/v1q1aqWIiAiNGjVKhw4d8nRZzdaLL74oi8WiJ5980tOlmAJhpwn8+c9/1oQJEzR58mTt3r1bO3bs0Lhx4zxdVrP01FNP1TsNOGo6ePCgqqqq9Prrr2vfvn1asmSJVqxYoaefdr6y8rXuvffe06xZszR//nx9/fXX6tWrl5KTk1Va6nzxQ/y/Tz75RKmpqfr888+VnZ2tyspKDR8+XOfOnfN0ac3Ol19+qddff109e/b0dCnm4ZrlOHFZZWWl0aFDB+P3v/+9p0tp9jZu3GjExcUZ+/btMyQZf/vb3zxdUrO2ePFio1OnTp4uw2vdeuutRmpqquO73W432rdvbyxcuNCDVTVfpaWlhiTjk08+8XQpzcqZM2eMLl26GNnZ2cYdd9xhPPHEE54uyRS4s+NiX3/9tQoLC2W1WtWnTx+1a9dOI0aM0N69ez1dWrNSUlKiqVOn6p133lFQUJCnyzGF06dPq02bNp4uwytVVFQoPz9fSUlJjm1Wq1VJSUnKy8vzYGXN1+nTpyWJP3NXKTU1VSkpKdX+LKLxCDsu9q9//UuSlJ6errlz52r9+vVq3bq1hgwZopMnT3q4uubBMAxNmjRJjz32mPr16+fpckzh8OHDysjI0KOPPurpUrzSiRMnZLfbaywxExkZqeLiYg9V1XxVVVXpySef1KBBg9S9e3dPl9NsrF69Wl9//bUWLlzo6VJMh7BzhX7xi1/IYrHU+bk8TkKSnnnmGY0ZM0Z9+/bVypUrZbFYtGbNGg+fhWdd6TXMyMjQmTNnlJaW5umSvc6VXsP/VlhYqB/+8Ie6//77NXXqVA9VjmtJamqq9u7dq9WrV3u6lGbj2LFjeuKJJ/Tuu+8qICDA0+WYDstFXCGbzaZvvvmmzj433HCDduzYoaFDh+rTTz/V4MGDHW0DBgxQUlKSfvWrXzV1qV7rSq/hAw88oL/85S+yWCyO7Xa7XT4+Pho/frzeeuutpi7Va13pNfTz85MkFRUVaciQIRo4cKBWrVolq5X/v6lNRUWFgoKC9Kc//anam5MTJ07UqVOntHbtWs8V18xMnz5da9eu1fbt29WpUydPl9NsfPjhh7r33nvl4+Pj2Ga322WxWGS1WlVeXl6tDVfHNAuBNrXw8HCFh4fX269v377y9/fXoUOHHGGnsrJSR48eVWxsbFOX6dWu9BouW7ZMzz//vON7UVGRkpOT9d5772nAgAFNWaLXu9JrKH13RycxMdFxd5Gg45yfn5/69u2rnJwcR9ipqqpSTk6ORxcYbk4Mw9CMGTP0wQcfaNu2bQSdqzRs2DDt2bOn2rbJkycrLi5Oc+bMIeg0EmHHxYKDg/XYY49p/vz5io6OVmxsrF566SVJ0v333+/h6pqHmJiYat9btmwpSbrxxht1/fXXe6KkZqewsFBDhgxRbGysXn75ZdlsNkdbVFSUByvzXrNmzdLEiRPVr18/3XrrrVq6dKnOnTunyZMne7q0ZiE1NVVZWVlau3atWrVq5RjrFBISosDAQA9X5/1atWpVY3xTixYt1LZtW8Y9uQBhpwm89NJLuu666zRhwgRduHBBAwYM0NatW9W6dWtPl4ZrRHZ2tg4fPqzDhw/XCIg8ua7dj3/8Y9lsNs2bN0/FxcXq3bu3Nm3aVGPQMmq3fPlySdKQIUOqbV+5cqUmTZrk/oKA/8KYHQAAYGo8xAcAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAfbfyOQBzIuwA8Gp/+tOf1KNHDwUGBqpt27ZKSkrSuXPnJElvvvmmunXrJn9/f7Vr167aCuUFBQUaOXKkWrZsqeDgYD3wwAMqKSlxtKenp6t37976/e9/r06dOikgIECSdOrUKT388MMKDw9XcHCwhg4dqt27d7v3pAG4FGEHgNc6fvy4HnzwQT300EM6cOCAtm3bptGjR8swDC1fvlypqal65JFHtGfPHq1bt06dO3eWJFVVVWnkyJE6efKkPvnkE2VnZ+tf//qXfvzjH1fb/+HDh/XnP/9Z77//vnbt2iVJuv/++1VaWqq//vWvys/P1y233KJhw4bp5MmT7j59AC7CQqAAvNbXX3+tvn376ujRo4qNja3W1qFDB02ePFnPP/98jd9lZ2drxIgROnLkiKKjoyVJ+/fvV7du3fTFF1+of//+Sk9P1wsvvKDCwkKFh4dLkj777DOlpKSotLRU/v7+jv117txZTz31lB555JEmPFsATeU6TxcAAM706tVLw4YNU48ePZScnKzhw4frvvvuU2VlpYqKijRs2LBaf3fgwAFFR0c7go4kxcfHKzQ0VAcOHFD//v0lSbGxsY6gI0m7d+/W2bNn1bZt22r7u3Dhgv75z382wRkCcAfCDgCv5ePjo+zsbOXm5uqjjz5SRkaGnnnmGeXk5Lhk/y1atKj2/ezZs2rXrp22bdtWo29oaKhLjgnA/Qg7ALyaxWLRoEGDNGjQIM2bN0+xsbHKzs5Wx44dlZOTo8TExBq/6dq1q44dO6Zjx45Ve4x16tQpxcfHOz3WLbfcouLiYl133XXq2LFjU50SADcj7ADwWjt37lROTo6GDx+uiIgI7dy5UzabTV27dlV6eroee+wxRUREaMSIETpz5ox27NihGTNmKCkpST169ND48eO1dOlSXbp0SY8//rjuuOMO9evXz+nxkpKSlJCQoFGjRmnx4sW66aabVFRUpA0bNujee++t87cAvBdhB4DXCg4O1vbt27V06VKVlZUpNjZWv/71rzVixAhJ0sWLF7VkyRL9/Oc/V1hYmO677z5J390NWrt2rWbMmKHbb79dVqtVP/zhD5WRkVHn8SwWizZu3KhnnnlGkydPls1mU1RUlG6//XZFRkY2+fkCaBq8jQUAAEyNeXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICp/R9m5X3KpffJoQAAAABJRU5ErkJggg==\n" + }, + "metadata": {} + } + ], + "source": [ + "import seaborn as sns\n", + "import pandas as pd\n", + "import warnings\n", + "warnings.filterwarnings(\"ignore\", \"is_categorical_dtype\")\n", + "warnings.filterwarnings(\"ignore\", \"use_inf_as_na\")\n", + "\n", + "score = clf.decision_function(X)\n", + "df = pd.DataFrame({'score': score, 'y': y})\n", + "sns.histplot(df, x=\"score\", hue=\"y\").set_title('Smooth SVM')\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Mfu2LWh7MuGj" + }, + "source": [ + "## With Pipeline\n", + "`plq_Ridge_Classifier` can be integrated into a scikit-learn Pipeline to streamline preprocessing including scaling." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "id": "MCEXWUchaaP_" + }, + "outputs": [], + "source": [ + "## simulate data\n", + "from sklearn.datasets import make_classification\n", + "from sklearn.preprocessing import StandardScaler\n", + "from sklearn.pipeline import Pipeline\n", + "import numpy as np\n", + "\n", + "n, d = 10000, 5\n", + "X, y = make_classification(n_samples=n, n_features=d, random_state=42)\n", + "y = 2*y - 1" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 155 + }, + "id": "PwMysLRWaiv_", + "outputId": "428843f0-59b4-4a72-a2db-664b4b269b47" + }, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "Pipeline(steps=[('scaler', StandardScaler()),\n", + " ('clf', plq_Ridge_Classifier(loss={'name': 'sSVM'}))])" + ], + "text/html": [ + "
Pipeline(steps=[('scaler', StandardScaler()),\n",
+              "                ('clf', plq_Ridge_Classifier(loss={'name': 'sSVM'}))])
In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.
" + ] + }, + "metadata": {}, + "execution_count": 7 + } + ], + "source": [ + "## solve SVM via `plq_Ridge_Classifier`\n", + "from rehline import plq_Ridge_Classifier\n", + "\n", + "pipe = Pipeline([\n", + " ('scaler', StandardScaler()),\n", + " ('clf', plq_Ridge_Classifier(loss={'name': 'sSVM'}, C=1.0))\n", + "])\n", + "pipe.fit(X=X, y=y)" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 472 + }, + "id": "ZMofqZ_zakNc", + "outputId": "f051e47f-aeea-41f4-f283-4c3e4ab4cf93" + }, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "
" + ], + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjsAAAHHCAYAAABZbpmkAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAANvZJREFUeJzt3X10FNX9x/HPbswjkATIE2ASUNAQngWEFFQCkRSjBUEtQikgomKgCr8ijVKIaEXQCiVGsK2CWlM8tCoUKBiCiJKIGgvl+ZQWfqEJSRYRwmMSNvP7w7K/xmQTSDa7m+H9OmfPcefenfnOHJSPM3futRiGYQgAAMCkrJ4uAAAAoCkRdgAAgKkRdgAAgKkRdgAAgKkRdgAAgKkRdgAAgKkRdgAAgKkRdgAAgKkRdgAAgKkRdgBcMywWi6ZPn+7pMgC4GWEHQK327Nmj++67T7GxsQoICFCHDh105513KiMjw9Ol1Sk3N1fp6ek6deqUy/dts9n0xBNPKC4uToGBgYqIiNCtt96qOXPm6OzZs6qsrFRYWJgGDx7sdB+GYSg6Olq33HKLJGnbtm2yWCyyWCz6wx/+UOtvBg0aJIvFou7du7v8nIBrAWEHQA25ubnq16+fdu/eralTp+rVV1/Vww8/LKvVqt/85jeeLq9Oubm5evbZZ10edk6ePKl+/frp7bffVkpKipYtW6ZZs2apc+fOWr58uU6cOCFfX1/df//9ys3N1f/+7//Wup/t27fr3//+t37yk59U2x4QEKCsrKwa/Y8eParc3FwFBAS49HyAa8l1ni4AgPf51a9+pZCQEH355ZcKDQ2t1lZaWuqZojzsjTfeUEFBgXbs2KEf/OAH1drKysrk5+cnSRo/frxWrFihP/7xj/rFL35RYz9ZWVmyWq0aO3Zste133XWX1q1bpxMnTigsLKxa/8jISHXp0kXffvttE5wZYH7c2QFQwz//+U9169atRtCRpIiIiGrfL4+DWbNmjeLj4xUYGKiEhATt2bNHkvT666+rc+fOCggI0JAhQ3T06NEa+1yzZo369u2rwMBAhYWF6Sc/+YkKCwtr9Nu6datuu+02tWjRQqGhoRo5cqQOHDjgaE9PT9fs2bMlSZ06dXI8Hvr+MT/88EN1795d/v7+6tatmzZt2nRF18THx0cDBw6s0RYcHOy48zJo0CB17Nix1rs0lZWV+tOf/qTExES1b9++WtvIkSPl7++vNWvWVNuelZWlBx54QD4+PvXWCKB2hB0ANcTGxio/P1979+69ov6ffvqp/ud//kcTJ05Uenq6Dhw4oLvvvluZmZlatmyZHn/8cc2ePVt5eXl66KGHqv121apVjr/MFy5cqKlTp+r999/X4MGDqz2K2rJli5KTk1VaWqr09HTNmjVLubm5GjRokCPMjB49Wg8++KAkacmSJXrnnXf0zjvvKDw83LGfzz77TI8//rjGjh2rxYsX6+LFixozZoy++eabeq+J3W7XO++8U2c/i8WicePGac+ePdq3b1+1tk2bNunkyZMaP358jd8FBQVp5MiR+uMf/+jYtnv3bu3bt0/jxo2r85gA6mEAwPd89NFHho+Pj+Hj42MkJCQYTz31lLF582ajoqKiRl9Jhr+/v3HkyBHHttdff92QZERFRRllZWWO7WlpaYYkR9+KigojIiLC6N69u3HhwgVHv/Xr1xuSjHnz5jm29e7d24iIiDC++eYbx7bdu3cbVqvV+OlPf+rY9tJLL1U7xvdr9fPzMw4fPlxtH5KMjIyMOq9JcXGxER4ebkgy4uLijMcee8zIysoyTp06VaPvvn37DElGWlpate1jx441AgICjNOnTzu2ffzxx4YkY82aNcb69esNi8ViFBQUGIZhGLNnzzZuuOEGwzAM44477jC6detWZ40AasedHQA13HnnncrLy9OPfvQj7d69W4sXL1ZycrI6dOigdevW1eg/bNgwdezY0fF9wIABkqQxY8aoVatWNbb/61//kiR99dVXKi0t1eOPP15tAG5KSori4uK0YcMGSdLx48e1a9cuTZo0SW3atHH069mzp+68805t3Ljxis8tKSlJN954Y7V9BAcHO2pyJjIyUrt379Zjjz2mb7/9VitWrNC4ceMUERGh5557ToZhOPrGx8erT58+Wr16tWPbuXPntG7dOt19990KDg6u9RjDhw9XmzZttHr1ahmGodWrVzvuVAFoOMIOgFr1799f77//vr799lt98cUXSktL05kzZ3Tfffdp//791frGxMRU+x4SEiJJio6OrnX75YG2l99Yuvnmm2scPy4uztFeV7+uXbvqxIkTOnfu3BWd1/drlaTWrVtf0eDfdu3aafny5Tp+/LgOHTqkZcuWKTw8XPPmzdMbb7xRre/48eN15MgR5ebmSvpunND58+drfYR12eW3ubKysrR9+3YdO3aMR1iACxB2ANTJz89P/fv31wsvvKDly5ersrKyxiBaZ4NnnW3/77sg7uaKmiwWi2666SbNmDFD27dvl9Vq1bvvvlutz4MPPiir1eoYqJyVlaXWrVvrrrvuqnPf48aN065du5Senq5evXopPj7+iusCUDvCDoAr1q9fP0nfPVZyhdjYWEnSoUOHarQdOnTI0V5Xv4MHDyosLEwtWrSQ9F0QcacbbrhBrVu3rnFN2rdvr8TERK1Zs0YlJSXKzs7Wfffd53hF3ZnBgwcrJiZG27Zt464O4CKEHQA1fPzxx7Xe6bg8Nqa2x0kN0a9fP0VERGjFihUqLy93bP/rX/+qAwcOKCUlRdJ3j4969+6tt956q9obWnv37tVHH31U7W7J5dDj6kkFd+7cWeujsi+++ELffPNNrddk/PjxKi0t1aOPPqrKyso6H2FdZrFYtGzZMs2fP18TJkxwSe3AtY5JBQHUMGPGDJ0/f1733nuv4uLiVFFRodzcXL333nvq2LGjJk+e7JLj+Pr6atGiRZo8ebLuuOMOPfjggyopKdFvfvMbdezYUTNnznT0femllzRixAglJCRoypQpunDhgjIyMhQSEqL09HRHv759+0qSnnnmGY0dO1a+vr665557HCGood555x29++67uvfee9W3b1/5+fnpwIEDevPNNxUQEKCnn366xm/GjBmjxx9/XGvXrlV0dLRuv/32KzrWyJEjNXLkyEbVC+D/EXYA1PDyyy9rzZo12rhxo37729+qoqJCMTExevzxxzV37txaJxtsqEmTJikoKEgvvvii5syZoxYtWujee+/VokWLqh0nKSlJmzZt0vz58zVv3jz5+vrqjjvu0KJFi9SpUydHv/79++u5557TihUrtGnTJlVVVenIkSONDjuPPvqogoKClJOTo7Vr16qsrEzh4eEaPny40tLS1KdPnxq/CQ4O1j333KM1a9bowQcfdPsjNgDfsRieHCkIAADQxBizAwAATI2wAwAATI2wAwAATI2wAwAATI2wAwAATI2wAwAATI15diRVVVWpqKhIrVq1Yh4MAACaCcMwdObMGbVv315Wq/P7N4QdSUVFRTVWZwYAAM3DsWPHdP311zttJ+xIatWqlaTvLlZwcLCHqwEAAFeirKxM0dHRjr/HnSHs6P9XSQ4ODibsAADQzNQ3BIUBygAAwNQIOwAAwNQIOwAAwNQYswMAQDNht9tVWVnp6TLcxtfXVz4+Po3eD2EHAAAvZxiGiouLderUKU+X4nahoaGKiopq1Dx4hB0AALzc5aATERGhoKCga2ICXMMwdP78eZWWlkqS2rVr1+B9EXYAAPBidrvdEXTatm3r6XLcKjAwUJJUWlqqiIiIBj/SYoAyAABe7PIYnaCgIA9X4hmXz7sxY5UIOwAANAPXwqOr2rjivAk7AADA1Ag7AADA1Ag7AADA1Ag7AADA1Ag7AACgmrfffltt27ZVeXl5te2jRo3ShAkTPFRVwxF2AABANffff7/sdrvWrVvn2FZaWqoNGzbooYce8mBlDcOkggDcamhiomw2m9P28PBwbf34YzdWBOD7AgMDNW7cOK1cuVL333+/JOkPf/iDYmJiNGTIEM8W1wCEHQBuZbPZtHPLZqftA5KS3VgNAGemTp2q/v37q7CwUB06dNCqVas0adKkZjnfD2EHAADU0KdPH/Xq1Utvv/22hg8frn379mnDhg2eLqtBCDsAvEphYaF6dO/utJ3HXID7PPzww1q6dKkKCwuVlJSk6OhoT5fUIIQdAF7FMKp4zAV4iXHjxunnP/+5fve73+ntt9/2dDkNxttYAACgViEhIRozZoxatmypUaNGebqcBiPsAAAApwoLCzV+/Hj5+/t7upQG4zEWAACo4dtvv9W2bdu0bds2vfbaa54up1EIOwBgQoMH3aaSkmKn7ZGRUfpsx6durAjNTZ8+ffTtt99q0aJFuvnmmz1dTqMQdgDAhEpKijUz9Wmn7UsyX3BjNWiOjh496ukSXIawAwDNUH13bgoLi9xYDeDdCDsA0AzVd+dm5pxpbqwG8G68jQUAAEyNsAMAAEyNsAMAAEyNsAMAAEyNAcoAADRTBQUFOnHihFuOFRYWppiYGLccy9UIOwBcamhiomw2m9P240W8Eg24QkFBgbp27arz58+75XhBQUE6cOBAowLP+++/rxUrVig/P18nT57U3/72N/Xu3dt1RTpB2AHgUjabrc5VyzvExbuxGsC8Tpw4ofPnz+uNV5fp5i5dmvRYh/7xD02Z/jOdOHGiUWHn3LlzGjx4sB544AFNnTrVhRXWjbADAEAzdnOXLurTs4eny7giEyZMkOT+2ZkZoAwAAEyNsAMAAEyNsAMAAFzu3XffVcuWLR2fTz/91GO1MGYHAAC43I9+9CMNGDDA8b1Dhw4eq4WwAwAAXK5Vq1Zq1aqVp8uQRNgBAABucvLkSRUUFKjoP/NtHTp0SJIUFRWlqKioJjsuYQcAgGbs0D/+0WyOsW7dOk2ePNnxfezYsZKk+fPnKz093SXHqA1hBwCAZigsLExBQUGaMv1nbjleUFCQwsLCGrWPSZMmadKkSa4p6CoQdgAAaIZiYmJ04MAB1sa6AoQdAACaqZiYmGYbQNyJeXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpMc8OgKsyNDFRNpvNafvx/6x5A6DpFRQUMKngFSDsALgqNptNO7dsdtreIS7ejdUA166CggLFxcXpwoULbjleYGCgDh48eFWBZ/v27XrppZeUn5+v48eP64MPPtCoUaOarkgnCDsA4IUGD7pNJSXFTtsLC7mDdq07ceKELly4oInjpyoqsn2THqu4pEhvvfs7nThx4qrCzrlz59SrVy899NBDGj16dBNWWDfCDgB4oZKSYs1Mfdpp+8w509xYDbxZVGR7xVwf6+kyajVixAiNGDHC02UwQBkAAJgbYQcAAJgaYQcAAJgaYQcAAJia14SdF198URaLRU8++aRj28WLF5Wamqq2bduqZcuWGjNmjEpKSqr9rqCgQCkpKQoKClJERIRmz56tS5cuubl6AADgrbzibawvv/xSr7/+unr27Flt+8yZM7VhwwatWbNGISEhmj59ukaPHq0dO3ZIkux2u1JSUhQVFaXc3FwdP35cP/3pT+Xr66sXXnjBE6cCAAD+4+zZszp8+LDj+5EjR7Rr1y61adPGrRMUejzsnD17VuPHj9fvfvc7Pf/8847tp0+f1htvvKGsrCwNHTpUkrRy5Up17dpVn3/+uQYOHKiPPvpI+/fv15YtWxQZGanevXvrueee05w5c5Seni4/Pz9PnRYA1Il5dOAqxSVN/2elocf46quvlJiY6Pg+a9YsSdLEiRO1atUqV5R2RTwedlJTU5WSkqKkpKRqYSc/P1+VlZVKSkpybIuLi1NMTIzy8vI0cOBA5eXlqUePHoqMjHT0SU5O1rRp07Rv3z716dOn1mOWl5ervLzc8b2srKwJzgwAnGMeHTRWWFiYAgMD9da7v3PL8QIDAxUWFnZVvxkyZIgMw2iiiq6cR8PO6tWr9fXXX+vLL7+s0VZcXCw/Pz+FhoZW2x4ZGani4mJHn/8OOpfbL7c5s3DhQj377LONrB4AAM+JiYnRwYMHWRvrCngs7Bw7dkxPPPGEsrOzFRAQ4NZjp6WlOW6lSd/d2YmOjnZrDYA3qm+RT4mFPgFvEhMT02wDiDt5LOzk5+ertLRUt9xyi2Ob3W7X9u3b9eqrr2rz5s2qqKjQqVOnqt3dKSkpUVRUlCQpKipKX3zxRbX9Xn5b63Kf2vj7+8vf39+FZwOYQ32LfEos9Amg+fHYq+fDhg3Tnj17tGvXLsenX79+Gj9+vOOffX19lZOT4/jNoUOHVFBQoISEBElSQkKC9uzZo9LSUkef7OxsBQcHKz6e/yADAAAP3tlp1aqVunfvXm1bixYt1LZtW8f2KVOmaNasWWrTpo2Cg4M1Y8YMJSQkaODAgZKk4cOHKz4+XhMmTNDixYtVXFysuXPnKjU1lTs3AABT8YaBvp7givP2+NtYdVmyZImsVqvGjBmj8vJyJScn67XXXnO0+/j4aP369Zo2bZoSEhLUokULTZw4UQsWLPBg1QAAuI6vr68k6fz58woMDPRwNe53/vx5Sf9/HRrCq8LOtm3bqn0PCAhQZmamMjMznf4mNjZWGzdubOLKAADwDB8fH4WGhjqGbAQFBclisXi4qqZnGIbOnz+v0tJShYaGysfHp8H78qqwAwAAarr80s1/j1G9VoSGhtb50tGVIOwAAODlLBaL2rVrp4iICFVWVnq6HLfx9fVt1B2dywg7AAA0Ez4+Pi75y/9a4zWrngMAADQF7uwAwDWosLBIXTp3qbNPZGSUPtvxqZsqApoOYQcArkF2u73OhUglaUnmC26qBmhaPMYCAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmxttYAJqVwsJC9ejevc4+4eHh2vrxx26qCIC3I+wAaFYMo0o7t2yus8+ApGQ3VQOgOeAxFgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDUmFQQAFxs86DaVlBTX2aewsMhN1QAg7ACAi5WUFGtm6tN19pk5Z5qbqgHAYywAAGBqhB0AAGBqhB0AAGBqhB0AAGBqhB0AAGBqhB0AAGBqvHoOAKhVYWGRunTu4rQ9MjJKn+341I0VAQ1D2AEA1Mput9c5X9CSzBfcWA3QcDzGAgAApsadHQCmU1hYqB7duzttDw8P19aPP3ZjRQA8ibADwHQMo0o7t2x22j4gKdmN1QDwNB5jAQAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAU7vO0wUAQHMzeNBtKikpdtpeWFjkxmoA1IewAwBXqaSkWDNTn3baPnPONDdWA6A+PMYCAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACmRtgBAACm5tGws3z5cvXs2VPBwcEKDg5WQkKC/vrXvzraL168qNTUVLVt21YtW7bUmDFjVFJSUm0fBQUFSklJUVBQkCIiIjR79mxdunTJ3acCAAC8lEfDzvXXX68XX3xR+fn5+uqrrzR06FCNHDlS+/btkyTNnDlTf/nLX7RmzRp98sknKioq0ujRox2/t9vtSklJUUVFhXJzc/XWW29p1apVmjdvnqdOCQAAeJnrPHnwe+65p9r3X/3qV1q+fLk+//xzXX/99XrjjTeUlZWloUOHSpJWrlyprl276vPPP9fAgQP10Ucfaf/+/dqyZYsiIyPVu3dvPffcc5ozZ47S09Pl5+fnidMCAABexGvG7Njtdq1evVrnzp1TQkKC8vPzVVlZqaSkJEefuLg4xcTEKC8vT5KUl5enHj16KDIy0tEnOTlZZWVljrtDAADg2ubROzuStGfPHiUkJOjixYtq2bKlPvjgA8XHx2vXrl3y8/NTaGhotf6RkZEqLi6WJBUXF1cLOpfbL7c5U15ervLycsf3srIyF50NAADwNh6/s3PzzTdr165d2rlzp6ZNm6aJEydq//79TXrMhQsXKiQkxPGJjo5u0uMBAADP8XjY8fPzU+fOndW3b18tXLhQvXr10m9+8xtFRUWpoqJCp06dqta/pKREUVFRkqSoqKgab2dd/n65T23S0tJ0+vRpx+fYsWOuPSkAAOA1PB52vq+qqkrl5eXq27evfH19lZOT42g7dOiQCgoKlJCQIElKSEjQnj17VFpa6uiTnZ2t4OBgxcfHOz2Gv7+/43X3yx8AAGBOHh2zk5aWphEjRigmJkZnzpxRVlaWtm3bps2bNyskJERTpkzRrFmz1KZNGwUHB2vGjBlKSEjQwIEDJUnDhw9XfHy8JkyYoMWLF6u4uFhz585Vamqq/P39PXlqAADAS3g07JSWluqnP/2pjh8/rpCQEPXs2VObN2/WnXfeKUlasmSJrFarxowZo/LyciUnJ+u1115z/N7Hx0fr16/XtGnTlJCQoBYtWmjixIlasGCBp04JAAB4GY+GnTfeeKPO9oCAAGVmZiozM9Npn9jYWG3cuNHVpQEAAJPwujE7AAAAruTxeXYAAM1TYWGRunTu4rQ9MjJKn+341I0VAbUj7AAAGsRut2tm6tNO25dkvuDGagDneIwFAABMjTs7APA9gwfdppIS50vOFBYWubEaAI1F2AGA7ykpKa7z8czMOdPcWA2AxuIxFgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDWWiwCuIUMTE2Wz2Zy2Hy9izScA5kPYAa4hNptNO7dsdtreIS7ejdUAgHs06DHWDTfcoG+++abG9lOnTumGG25odFEAAACu0qCwc/ToUdnt9hrby8vLVVhY2OiiAAAAXOWqHmOtW7fO8c+bN29WSEiI47vdbldOTo46duzosuIAXB3G5ABATVcVdkaNGiVJslgsmjhxYrU2X19fdezYUb/+9a9dVhyAq8OYHACo6arCTlVVlSSpU6dO+vLLLxUWFtYkRQEAALhKg97GOnLkiKvrAAAAaBINfvU8JydHOTk5Ki0tddzxuezNN99sdGEAAACu0KCw8+yzz2rBggXq16+f2rVrJ4vF4uq6AAAAXKJBYWfFihVatWqVJkyY4Op6AAAAXKpB8+xUVFToBz/4gatrAQAAcLkGhZ2HH35YWVlZrq4FAADA5Rr0GOvixYv67W9/qy1btqhnz57y9fWt1v7KK6+4pDgAAIDGalDY+fvf/67evXtLkvbu3VutjcHKAADAmzQo7Hz88ceurgMAAKBJNGjMDgAAQHPRoDs7iYmJdT6u2rp1a4MLAgAAcKUGhZ3L43Uuq6ys1K5du7R3794aC4QCAAB4UoPCzpIlS2rdnp6errNnzzaqIAAAAFdy6Zidn/zkJ6yLBQAAvIpLw05eXp4CAgJcuUsAAIBGadBjrNGjR1f7bhiGjh8/rq+++kq//OUvXVIYAACAKzQo7ISEhFT7brVadfPNN2vBggUaPny4SwoDAABwhQaFnZUrV7q6DgAAgCbRoLBzWX5+vg4cOCBJ6tatm/r06eOSogAAAFylQWGntLRUY8eO1bZt2xQaGipJOnXqlBITE7V69WqFh4e7skYAAIAGa9DbWDNmzNCZM2e0b98+nTx5UidPntTevXtVVlamn/3sZ66uEQAAoMEadGdn06ZN2rJli7p27erYFh8fr8zMTAYoAwAAr9KgOztVVVXy9fWtsd3X11dVVVWNLgoAAMBVGhR2hg4dqieeeEJFRUWObYWFhZo5c6aGDRvmsuIAAAAaq0Fh59VXX1VZWZk6duyoG2+8UTfeeKM6deqksrIyZWRkuLpGAACABmvQmJ3o6Gh9/fXX2rJliw4ePChJ6tq1q5KSklxaHAAAQGNd1Z2drVu3Kj4+XmVlZbJYLLrzzjs1Y8YMzZgxQ/3791e3bt306aefNlWtAAAAV+2qws7SpUs1depUBQcH12gLCQnRo48+qldeecVlxQEAADTWVT3G2r17txYtWuS0ffjw4Xr55ZcbXRQAoPkrLCxSl85dnLZHRkbpsx08DUDTu6qwU1JSUusr546dXXedbDZbo4sCADR/drtdM1Ofdtq+JPMFN1aDa9lVPcbq0KGD9u7d67T973//u9q1a9foogAAAFzlqsLOXXfdpV/+8pe6ePFijbYLFy5o/vz5uvvuu11WHAAAQGNd1WOsuXPn6v3339dNN92k6dOn6+abb5YkHTx4UJmZmbLb7XrmmWeapFAAAICGuKqwExkZqdzcXE2bNk1paWkyDEOSZLFYlJycrMzMTEVGRjZJoQDgKoWFherRvbvTdput1I3VAGhqVz2pYGxsrDZu3Khvv/1Whw8flmEY6tKli1q3bt0U9QGAyxlGlXZu2ey0vf1NXZ22AWh+GjSDsiS1bt1a/fv3d2UtAAAALtegtbEAAACaC8IOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNY+GnYULF6p///5q1aqVIiIiNGrUKB06dKhan4sXLyo1NVVt27ZVy5YtNWbMGJWUlFTrU1BQoJSUFAUFBSkiIkKzZ8/WpUuX3HkqAADAS3k07HzyySdKTU3V559/ruzsbFVWVmr48OE6d+6co8/MmTP1l7/8RWvWrNEnn3yioqIijR492tFut9uVkpKiiooK5ebm6q233tKqVas0b948T5wSAADwMg2eVNAVNm3aVO37qlWrFBERofz8fN1+++06ffq03njjDWVlZWno0KGSpJUrV6pr1676/PPPNXDgQH300Ufav3+/tmzZosjISPXu3VvPPfec5syZo/T0dPn5+Xni1AAAgJfwqjE7p0+fliS1adNGkpSfn6/KykolJSU5+sTFxSkmJkZ5eXmSpLy8PPXo0aPamlzJyckqKyvTvn37aj1OeXm5ysrKqn0AAIA5eU3Yqaqq0pNPPqlBgwap+38W6CsuLpafn59CQ0Or9Y2MjFRxcbGjz/cXH738/XKf71u4cKFCQkIcn+joaBefDQAA8BZeE3ZSU1O1d+9erV69usmPlZaWptOnTzs+x44da/JjAgAAz/DomJ3Lpk+frvXr12v79u26/vrrHdujoqJUUVGhU6dOVbu7U1JSoqioKEefL774otr+Lr+tdbnP9/n7+8vf39/FZwEAALyRR+/sGIah6dOn64MPPtDWrVvVqVOnau19+/aVr6+vcnJyHNsOHTqkgoICJSQkSJISEhK0Z88elZaWOvpkZ2crODhY8fHx7jkRAADgtTx6Zyc1NVVZWVlau3atWrVq5RhjExISosDAQIWEhGjKlCmaNWuW2rRpo+DgYM2YMUMJCQkaOHCgJGn48OGKj4/XhAkTtHjxYhUXF2vu3LlKTU3l7g0AAPBs2Fm+fLkkaciQIdW2r1y5UpMmTZIkLVmyRFarVWPGjFF5ebmSk5P12muvOfr6+Pho/fr1mjZtmhISEtSiRQtNnDhRCxYscNdpAAAAL+bRsGMYRr19AgIClJmZqczMTKd9YmNjtXHjRleWBgAATMJr3sYCAABoCoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgatd5ugAAcLdLl+zKyMhw3m63u7EaAE2NsAPgGmQoof9tdbQ7D0JwncLCInXp3MVpe2RklD7b8akbK4JZEXYAAB5ht9s1M/Vpp+1LMl9wYzUwM8bsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAUyPsAAAAU2MGZaAZGZqYKJvN5rT9eFGRG6sBgOaBsAM0IzabTTu3bHba3iEu3o3VAEDzwGMsAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgasyzA3gRJg30DkZVlRa9ku603Wq1uK8YAI1G2AG8CJMGeger1ap3Ml5z2n7X+AfcWA2AxuIxFgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMLXrPF0AALjapUt2ZWRkOO9gNG7/RlWVFr2S7rTdarU07gAAXIqwA8CEDCX0v62O9jqC0BWwWq16J+M1p+13jX+gUfsH4Fo8xgIAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKbm0bCzfft23XPPPWrfvr0sFos+/PDDau2GYWjevHlq166dAgMDlZSUpH/84x/V+pw8eVLjx49XcHCwQkNDNWXKFJ09e9aNZwEAALyZR8POuXPn1KtXL2VmZtbavnjxYi1btkwrVqzQzp071aJFCyUnJ+vixYuOPuPHj9e+ffuUnZ2t9evXa/v27XrkkUfcdQoAAMDLeXSenREjRmjEiBG1thmGoaVLl2ru3LkaOXKkJOntt99WZGSkPvzwQ40dO1YHDhzQpk2b9OWXX6pfv36SpIyMDN111116+eWX1b59e7edCwAA8E5eO6ngkSNHVFxcrKSkJMe2kJAQDRgwQHl5eRo7dqzy8vIUGhrqCDqSlJSUJKvVqp07d+ree++tdd/l5eUqLy93fC8rK2u6EwH+Y2hiomw2W519jhcVuakawPsVFhapS+cudfaJjIzSZzs+dVNFaK68NuwUFxdLkiIjI6ttj4yMdLQVFxcrIiKiWvt1112nNm3aOPrUZuHChXr22WddXDFQN5vNpp1bNtfZp0NcvJuqab7qXQpCavRyEPAOdrtdM1OfrrPPkswX3FQNmjOvDTtNKS0tTbNmzXJ8LysrU3R0tAcrAnDl6lsKQmrschAAzMVrXz2PioqSJJWUlFTbXlJS4miLiopSaWlptfZLly7p5MmTjj618ff3V3BwcLUPAAAwJ6+9s9OpUydFRUUpJydHvXv3lvTdHZidO3dq2rRpkqSEhASdOnVK+fn56tu3ryRp69atqqqq0oABAzxVOgDATeob18OYHkgeDjtnz57V4cOHHd+PHDmiXbt2qU2bNoqJidGTTz6p559/Xl26dFGnTp30y1/+Uu3bt9eoUaMkSV27dtUPf/hDTZ06VStWrFBlZaWmT5+usWPH8iYWAFwD6hvXw5geSB4OO1999ZUSExMd3y+Po5k4caJWrVqlp556SufOndMjjzyiU6dOafDgwdq0aZMCAgIcv3n33Xc1ffp0DRs2TFarVWPGjNGyZcvcfi4AAMA7eTTsDBkyRIbh/LUJi8WiBQsWaMGCBU77tGnTRllZWU1RHgAAMAGvHaAMAADgCl47QBlobuqbNJAJAwHAMwg7gIvUN2kgEwYCgGfwGAsAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgay0UAgIsZVVVa9Ep6nX2sVot7igFA2AEAV7NarXon47U6+9w1/gE3VQOAx1gAAMDUuLMDXKGhiYmy2WxO248XFbmxGgDAlSLs4JpRX1gJDw/X1o8/dtpus9m0c8tmp+0d4uIbVR8AoGkQdnDNqC+sDEhKdmM1AAB3YcwOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNd7GAv6jsLBQPbp3d9rOPDoA0DwRdoD/MIwq5tGB29S3fhZrZwGuQ9gBAA+ob/0s1s4CXIewA8CrXLpkV0ZGhvMOhvtqAWAOhB0AXsZQQv/b6mivIwgB31NYWKQunbs4bY+MjNJnOz51Y0XwBMIOAMC07Ha7ZqY+7bR9SeYLbqwGnsKr5wAAwNQIOwAAwNQIOwAAwNQIOwAAwNQIOwAAwNR4GwuAWzGPDgB3I+wAcDPm0QHgXjzGAgAApkbYAQAApkbYAQAApkbYAQAApkbYAQAApsbbWACAaxarol8bCDsAgGsWq6JfGwg7MIWhiYmy2Wx19jleVOSmagAA3oSwg2ahvjBzvKhIBfv31LmPDnHxri4L8Bir1aJFr6TX2Q7gO4QdNAs2m007t2x22k6QgdkYVVV1hhkfH6veyXjNaftd4x9ogqqA5omwAwBeyGqtO8ykEGaAK0bYAQATqu/OEI+5cC0h7ACACdV3Z4jHXLiWEHYAuNSlS3ZlZNSxcrnhvloAQCLsALhK9YcZQwn9b6tjD3X8FgCaAGEHblHfq+Ph4eHa+vHHbqwIDUeYAdC8EHbgFvW9Oj4gKdmN1QAAriWEHXiFwsJC9eje3Wk7sx8DABqKsAOvYBhVTBoIAGgShB0AAJxgVXRzIOwAAOAEq6Kbg9XTBQAAADQl7uwAwDWovuUkJJaUgHkQdgA41DthoMQMyCZR33ISEktKXIn6xvRIUqnNpojwcKftjPtpeoQdAP+lvgkDJSYNvHawmGj96hvTI0kz50xj3I+HEXbgEvXNkMw8OUDzw2KiMAvCDlyivhmSmScHMJ/67vz4+Fi4MwSvYJqwk5mZqZdeeknFxcXq1auXMjIydOutt3q6LLdg3SkAnlDfnZ+U8Q9wZwhewRRh57333tOsWbO0YsUKDRgwQEuXLlVycrIOHTqkiIgIT5fX5Jp63an6wpTEYyoAgPcyRdh55ZVXNHXqVE2ePFmStGLFCm3YsEFvvvmmfvGLX3i4usZr6vEwV7L/gv176twHj6m8Q31vU12qvFT321a8aQU3YgD0d5iluek1+7BTUVGh/Px8paWlObZZrVYlJSUpLy/Pg5W5TlOPh2G8zZWpN0hcsruxGmfqe5sqo952wF1cMQDaam3+44KYpbnpNfuwc+LECdntdkVGRlbbHhkZqYMHD9b6m/LycpWXlzu+nz59WpJUVlbm8vruTknRiRMnnLaHhYVp/YYNde7Dbrer7MwZp+2GYdTZ/u9//1vxXbs6bT9WUKCXXn7ZaXtl5aU693+5T2P2Ud85NLb9imqsqKyz3aiqUs9ufets94breO78+SZrd8cxqNE72r2hhiq7XQtfnldnjVarRSsWOf93ZsxDE+rch8UiXbh4oc72un5vtdbdfiXHMAyjzvaqqqom+fvpagy/M1k2W6nTdtuJEwoPC3PaHh4eoY+ynf9PdUNdvi6GUc9taaOZKywsNCQZubm51bbPnj3buPXWW2v9zfz58w19d8OeDx8+fPjw4dPMP8eOHaszKzT7OzthYWHy8fFRSUlJte0lJSWKioqq9TdpaWmaNWuW43tVVZVOnjyptm3bymJp2C3PsrIyRUdH69ixYwoODm7QPq51XMPG4xo2Dtev8biGjcc1vHKGYejMmTNq3759nf2afdjx8/NT3759lZOTo1GjRkn6Lrzk5ORo+vTptf7G399f/v7+1baFhoa6pJ7g4GD+cDYS17DxuIaNw/VrPK5h43ENr0xISEi9fZp92JGkWbNmaeLEierXr59uvfVWLV26VOfOnXO8nQUAAK5dpgg7P/7xj2Wz2TRv3jwVFxerd+/e2rRpU41BywAA4NpjirAjSdOnT3f62Mod/P39NX/+/BqPx3DluIaNxzVsHK5f43ENG49r6HoWw6jvfS0AAIDmy+rpAgAAAJoSYQcAAJgaYQcAAJgaYQcAAJgaYaeJbNiwQQMGDFBgYKBat27tmPAQV6e8vFy9e/eWxWLRrl27PF1Os3H06FFNmTJFnTp1UmBgoG688UbNnz9fFRUVni7Nq2VmZqpjx44KCAjQgAED9MUXX3i6pGZj4cKF6t+/v1q1aqWIiAiNGjVKhw4d8nRZzdaLL74oi8WiJ5980tOlmAJhpwn8+c9/1oQJEzR58mTt3r1bO3bs0Lhx4zxdVrP01FNP1TsNOGo6ePCgqqqq9Prrr2vfvn1asmSJVqxYoaefdr6y8rXuvffe06xZszR//nx9/fXX6tWrl5KTk1Va6nzxQ/y/Tz75RKmpqfr888+VnZ2tyspKDR8+XOfOnfN0ac3Ol19+qddff109e/b0dCnm4ZrlOHFZZWWl0aFDB+P3v/+9p0tp9jZu3GjExcUZ+/btMyQZf/vb3zxdUrO2ePFio1OnTp4uw2vdeuutRmpqquO73W432rdvbyxcuNCDVTVfpaWlhiTjk08+8XQpzcqZM2eMLl26GNnZ2cYdd9xhPPHEE54uyRS4s+NiX3/9tQoLC2W1WtWnTx+1a9dOI0aM0N69ez1dWrNSUlKiqVOn6p133lFQUJCnyzGF06dPq02bNp4uwytVVFQoPz9fSUlJjm1Wq1VJSUnKy8vzYGXN1+nTpyWJP3NXKTU1VSkpKdX+LKLxCDsu9q9//UuSlJ6errlz52r9+vVq3bq1hgwZopMnT3q4uubBMAxNmjRJjz32mPr16+fpckzh8OHDysjI0KOPPurpUrzSiRMnZLfbaywxExkZqeLiYg9V1XxVVVXpySef1KBBg9S9e3dPl9NsrF69Wl9//bUWLlzo6VJMh7BzhX7xi1/IYrHU+bk8TkKSnnnmGY0ZM0Z9+/bVypUrZbFYtGbNGg+fhWdd6TXMyMjQmTNnlJaW5umSvc6VXsP/VlhYqB/+8Ie6//77NXXqVA9VjmtJamqq9u7dq9WrV3u6lGbj2LFjeuKJJ/Tuu+8qICDA0+WYDstFXCGbzaZvvvmmzj433HCDduzYoaFDh+rTTz/V4MGDHW0DBgxQUlKSfvWrXzV1qV7rSq/hAw88oL/85S+yWCyO7Xa7XT4+Pho/frzeeuutpi7Va13pNfTz85MkFRUVaciQIRo4cKBWrVolq5X/v6lNRUWFgoKC9Kc//anam5MTJ07UqVOntHbtWs8V18xMnz5da9eu1fbt29WpUydPl9NsfPjhh7r33nvl4+Pj2Ga322WxWGS1WlVeXl6tDVfHNAuBNrXw8HCFh4fX269v377y9/fXoUOHHGGnsrJSR48eVWxsbFOX6dWu9BouW7ZMzz//vON7UVGRkpOT9d5772nAgAFNWaLXu9JrKH13RycxMdFxd5Gg45yfn5/69u2rnJwcR9ipqqpSTk6ORxcYbk4Mw9CMGTP0wQcfaNu2bQSdqzRs2DDt2bOn2rbJkycrLi5Oc+bMIeg0EmHHxYKDg/XYY49p/vz5io6OVmxsrF566SVJ0v333+/h6pqHmJiYat9btmwpSbrxxht1/fXXe6KkZqewsFBDhgxRbGysXn75ZdlsNkdbVFSUByvzXrNmzdLEiRPVr18/3XrrrVq6dKnOnTunyZMne7q0ZiE1NVVZWVlau3atWrVq5RjrFBISosDAQA9X5/1atWpVY3xTixYt1LZtW8Y9uQBhpwm89NJLuu666zRhwgRduHBBAwYM0NatW9W6dWtPl4ZrRHZ2tg4fPqzDhw/XCIg8ua7dj3/8Y9lsNs2bN0/FxcXq3bu3Nm3aVGPQMmq3fPlySdKQIUOqbV+5cqUmTZrk/oKA/8KYHQAAYGo8xAcAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAAKZG2AEAfbfyOQBzIuwA8Gp/+tOf1KNHDwUGBqpt27ZKSkrSuXPnJElvvvmmunXrJn9/f7Vr167aCuUFBQUaOXKkWrZsqeDgYD3wwAMqKSlxtKenp6t37976/e9/r06dOikgIECSdOrUKT388MMKDw9XcHCwhg4dqt27d7v3pAG4FGEHgNc6fvy4HnzwQT300EM6cOCAtm3bptGjR8swDC1fvlypqal65JFHtGfPHq1bt06dO3eWJFVVVWnkyJE6efKkPvnkE2VnZ+tf//qXfvzjH1fb/+HDh/XnP/9Z77//vnbt2iVJuv/++1VaWqq//vWvys/P1y233KJhw4bp5MmT7j59AC7CQqAAvNbXX3+tvn376ujRo4qNja3W1qFDB02ePFnPP/98jd9lZ2drxIgROnLkiKKjoyVJ+/fvV7du3fTFF1+of//+Sk9P1wsvvKDCwkKFh4dLkj777DOlpKSotLRU/v7+jv117txZTz31lB555JEmPFsATeU6TxcAAM706tVLw4YNU48ePZScnKzhw4frvvvuU2VlpYqKijRs2LBaf3fgwAFFR0c7go4kxcfHKzQ0VAcOHFD//v0lSbGxsY6gI0m7d+/W2bNn1bZt22r7u3Dhgv75z382wRkCcAfCDgCv5ePjo+zsbOXm5uqjjz5SRkaGnnnmGeXk5Lhk/y1atKj2/ezZs2rXrp22bdtWo29oaKhLjgnA/Qg7ALyaxWLRoEGDNGjQIM2bN0+xsbHKzs5Wx44dlZOTo8TExBq/6dq1q44dO6Zjx45Ve4x16tQpxcfHOz3WLbfcouLiYl133XXq2LFjU50SADcj7ADwWjt37lROTo6GDx+uiIgI7dy5UzabTV27dlV6eroee+wxRUREaMSIETpz5ox27NihGTNmKCkpST169ND48eO1dOlSXbp0SY8//rjuuOMO9evXz+nxkpKSlJCQoFGjRmnx4sW66aabVFRUpA0bNujee++t87cAvBdhB4DXCg4O1vbt27V06VKVlZUpNjZWv/71rzVixAhJ0sWLF7VkyRL9/Oc/V1hYmO677z5J390NWrt2rWbMmKHbb79dVqtVP/zhD5WRkVHn8SwWizZu3KhnnnlGkydPls1mU1RUlG6//XZFRkY2+fkCaBq8jQUAAEyNeXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICpEXYAAICp/R9m5X3KpffJoQAAAABJRU5ErkJggg==\n" + }, + "metadata": {} + } + ], + "source": [ + "import seaborn as sns\n", + "import pandas as pd\n", + "import warnings\n", + "import matplotlib.pyplot as plt\n", + "warnings.filterwarnings(\"ignore\", \"is_categorical_dtype\")\n", + "warnings.filterwarnings(\"ignore\", \"use_inf_as_na\")\n", + "\n", + "score = pipe.decision_function(X)\n", + "df = pd.DataFrame({'score': score, 'y': y})\n", + "sns.histplot(df, x=\"score\", hue=\"y\").set_title('Smooth SVM')\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "k_h8PGzS7AC2" + }, + "source": [ + "## Hyperparameter Tuning with GridSearchCV\n", + "\n", + "Due to its compatibility with the scikit-learn API, `GridSearchCV` can be applied to determine the optimal hyperparameters for the ReHLine model." + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "dZ-8KTfP7B2M", + "outputId": "56ed5d61-a189-4935-d16b-3fc6ec2d720d" + }, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Best Parameters: {'clf__C': 0.1}\n", + "Best CV Accuracy: 0.8920\n" + ] + } + ], + "source": [ + "from sklearn.model_selection import GridSearchCV\n", + "\n", + "# Define the parameter grid to search\n", + "param_grid = {'clf__C': [0.1, 1.0, 10.0]}\n", + "\n", + "# Create the GridSearchCV object\n", + "grid_search = GridSearchCV(pipe, param_grid, cv=5)\n", + "grid_search.fit(X, y)\n", + "\n", + "# Print the best parameters and score\n", + "print(f\"Best Parameters: {grid_search.best_params_}\")\n", + "print(f\"Best CV Accuracy: {grid_search.best_score_:.4f}\")\n" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 472 + }, + "id": "UgMTfMwk9jdf", + "outputId": "7df98ae9-b1ce-47b3-c3c4-4e25bb42c2f6" + }, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "
" + ], + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjsAAAHHCAYAAABZbpmkAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAO5lJREFUeJzt3XtcVVX+//H3AeWmAl64aYCWFuINw0uklReClCkt0yxz1MqmQqf0O2ZWo3S1tNSJIW2a0rIc+zrTxbRU1LwkpoZpaupMk34xkIuZ4hUQ9u+PhvObEx5AOJzL9vV8PM7jMWettff+7JON7/Zeey+LYRiGAAAATMrL1QUAAAA0JMIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOgAZlsVg0YcIEV5fhFIMHD9b48eNdXUaDeeKJJ9S7d29XlwFcMsIO4Eb27NmjO++8U9HR0fLz81ObNm108803Kz093dWlVSsrK0tpaWk6ceKEw/ddVFSkRx99VDExMfL391doaKh69eqlqVOn6vTp0yorK1OrVq3Ut29fu/swDEORkZG69tprJUkbNmyQxWKRxWLRe++9d9Ft+vTpI4vFos6dO9eqzi1btmjNmjWaOnVqlb6CggL94Q9/UExMjAICAtSkSRPFx8fr+eefd9hvlpWVpb59+yogIEDh4eH6/e9/r9OnT9dq2/nz52v48OGKioqSxWLR2LFjLzruscce0+7du7V8+XKH1Aw4C2EHcBNZWVnq0aOHdu/erfHjx+vPf/6zHnjgAXl5eelPf/qTq8urVlZWlp555hmHh53jx4+rR48eevfdd5WSkqLXXntNkydPVvv27TV//nwdO3ZMjRs31vDhw5WVlaX/+7//u+h+Nm3apB9//FH33nuvTbufn5+WLFlSZfzhw4eVlZUlPz+/Wtc6e/ZsDRw4UO3bt7dp37Fjhzp37qyMjAzdcMMNmjNnjl599VV1795dL730kkaMGFHrY9iza9cuDRw4UGfPntWcOXP0wAMP6C9/+YuGDx9eq+1ffvllrV+/Xp06dVKjRo3sjgsPD9eQIUP0yiuv1LtmwJns/6kG4FQvvPCCgoKCtGPHDgUHB9v0FRYWuqYoF3vrrbeUk5OjLVu26Prrr7fpKy4ulo+PjyRp1KhRWrBggf72t7/piSeeqLKfJUuWyMvLSyNHjrRpHzx4sJYvX65jx46pVatWNuPDwsLUoUMH/fzzzzXWWVhYqJUrV2rBggU27SdOnNDtt98ub29vffPNN4qJibHpf+GFF/Tmm2/WuP+aPPnkk2revLk2bNigwMBASVLbtm01fvx4rVmzRklJSdVuv3HjRutVnaZNm1Y7dsSIERo+fLh++OEHXXnllfWuHXAGruwAbuLf//63OnXqVCXoSFJoaKjN98p5MMuWLVNsbKz8/f2VkJCgPXv2SJLeeOMNtW/fXn5+furXr58OHz5cZZ/Lli1TfHy8/P391apVK917773Kzc2tMm79+vW64YYb1KRJEwUHB2vIkCHav3+/tT8tLU1TpkyRJLVr1856e+jXx/z444/VuXNn+fr6qlOnTlq1alWtfhNvb29dd911VfoCAwOtV1769Omjtm3bXvQqTVlZmf7+97+rf//+at26tU3fkCFD5Ovrq2XLltm0L1myRCNGjJC3t3eNNUrSypUrdeHCBSUmJtq0v/HGG8rNzdWcOXOqBB1JCgsL09NPP12rY9hTXFyszMxM3XvvvdagI0m//e1v1bRpU/3v//5vjfuIjo6WxWKp1fEqz/GTTz6pW8GACxB2ADcRHR2t7Oxs7d27t1bjN2/erP/5n//RmDFjlJaWpv379+s3v/mNMjIy9Nprr+mRRx7RlClTtHXrVt1333022y5atMj6l/nMmTM1fvx4ffjhh+rbt6/Nrai1a9cqOTlZhYWFSktL0+TJk5WVlaU+ffpYw8wdd9yhu+++W5I0d+5cLV68WIsXL1ZISIh1P19++aUeeeQRjRw5UrNmzdL58+c1bNgw/fTTTzX+JuXl5Vq8eHG14ywWi+655x7t2bNH+/bts+lbtWqVjh8/rlGjRlXZLiAgQEOGDNHf/vY3a9vu3bu1b98+3XPPPdUe879lZWWpZcuWio6Otmlfvny5/P39deedd9ZqP6dPn9axY8dq/Jw8edK6zZ49e3ThwgX16NHDZl8+Pj6Ki4vTN998U+vzqI2goCBdddVV2rJli0P3CzQoA4BbWLNmjeHt7W14e3sbCQkJxuOPP26sXr3aKC0trTJWkuHr62scOnTI2vbGG28Ykozw8HCjuLjY2j5t2jRDknVsaWmpERoaanTu3Nk4d+6cddyKFSsMScb06dOtbXFxcUZoaKjx008/Wdt2795teHl5Gb/97W+tbbNnz7Y5xq9r9fHxMb7//nubfUgy0tPTq/1N8vPzjZCQEEOSERMTYzz00EPGkiVLjBMnTlQZu2/fPkOSMW3aNJv2kSNHGn5+fsbJkyetbV988YUhyVi2bJmxYsUKw2KxGDk5OYZhGMaUKVOMK6+80jAMw7jpppuMTp06VVujYRhG3759jfj4+CrtzZs3N7p161bj9pXGjBljSKrxc9NNN1m3WbZsmSHJ2LRpU5X9DR8+3AgPD6/18Q3DMJo0aWKMGTOm2jFJSUlGx44dL2m/gCtxZQdwEzfffLO2bt2q2267Tbt379asWbOUnJysNm3aXPTpl4EDB6pt27bW75WPBA8bNkzNmjWr0v7DDz9Ikr7++msVFhbqkUcesZmAm5KSopiYGK1cuVKSdPToUe3atUtjx45VixYtrOO6du2qm2++WZ999lmtzy0xMVFXXXWVzT4CAwOtNdkTFham3bt366GHHtLPP/+sBQsW6J577lFoaKiee+45GYZhHRsbG6vu3btr6dKl1rYzZ85o+fLl+s1vfmNzi+e/JSUlqUWLFlq6dKkMw9DSpUutV6pq66efflLz5s2rtBcXF9v8s6jJ448/rszMzBo/r776qnWbc+fOSZJ8fX2r7M/Pz8/a70jNmzfXsWPHHL5foKEwQRlwIz179tSHH36o0tJS7d69Wx999JHmzp2rO++8U7t27VJsbKx1bFRUlM22QUFBkqTIyMiLtldOtK18Yumaa66pcvyYmBh9+eWXNY7r2LGjVq9erTNnzqhJkyY1nteva5V++QuzNpN/IyIiNH/+fL3++uv617/+pdWrV+vll1/W9OnTFRERoQceeMA6dtSoUfrDH/6grKwsXX/99fr444919uzZi97CqlT5NNeSJUvUq1cvHTly5JJuYVX67+BVKTAwUKdOnar1PmJjY23+GdeGv7+/JKmkpKRK3/nz5639jmQYRq3n+ADugCs7gBvy8fFRz5499eKLL2r+/PkqKyurMonW3uRZe+0X+8vYWRxRk8Vi0dVXX62JEydq06ZN8vLy0vvvv28z5u6775aXl5d1ovKSJUvUvHlzDR48uNp933PPPdq1a5fS0tLUrVu3Sw4cLVu2vGhwi4mJ0T//+U+VlpbWaj8nT55Ufn5+jZ/jx49bt4mIiJD0y5W4Xzt69GiVSdmO8PPPP9s8vQa4O8IO4OYqJ55e7C+zuqicRHvw4MEqfQcPHrT2VzfuwIEDatWqlfWqjrP/K//KK69U8+bNq/wmrVu3Vv/+/bVs2TIVFBQoMzNTd955p/URdXv69u2rqKgobdiwoU5XdWJiYnTo0KEq7bfeeqvOnTunf/zjH7Xaz6OPPqqIiIgaP3fccYd1m86dO6tRo0b6+uuvbfZVWlqqXbt2KS4u7pLPpyaHDh1Sx44dHb5foKEQdgA38cUXX1z0Skfl3JiL3U6qix49eig0NFQLFiywufXx+eefa//+/UpJSZH0yxWDuLg4vfPOOzZPaO3du1dr1qyxuVpSGXoc/VLBbdu26cyZM1Xat2/frp9++umiv8moUaNUWFio3/3udyorK6v2FlYli8Wi1157TTNmzNDo0aMvuc6EhAT9/PPPVeYgPfTQQ4qIiND//M//6J///GeV7QoLC/X8889bv9dlzk5QUJASExP13nvv2dwyW7x4sU6fPm3zYsGzZ8/qwIED9Zpvc/LkSf373/+u8t4jwJ0xZwdwExMnTtTZs2d1++23KyYmRqWlpcrKytIHH3ygtm3baty4cQ45TuPGjfXyyy9r3Lhxuummm3T33XeroKBAf/rTn9S2bVtNmjTJOnb27NkaNGiQEhISdP/99+vcuXNKT09XUFCQ0tLSrOPi4+MlSU899ZRGjhypxo0b69Zbb63VfJ7qLF68WO+//75uv/12xcfHy8fHR/v379fbb78tPz8/Pfnkk1W2GTZsmB555BF98sknioyM1I033lirYw0ZMkRDhgypU50pKSlq1KiR1q5dqwcffNDa3rx5c3300UcaPHiw4uLidO+991p/q507d+pvf/ubEhISrOPrMmdH+uXlhNdff71uuukmPfjgg/rxxx/16quvKikpSbfccot13Pbt29W/f3/NmDHD5p/fp59+qt27d0v65b1E3377rTWE3Xbbberatat17Nq1a2UYRp1/K8AlXPcgGID/9vnnnxv33XefERMTYzRt2tTw8fEx2rdvb0ycONEoKCiwGSvJSE1NtWk7dOiQIcmYPXu2Tft/P2b93z744AOje/fuhq+vr9GiRQtj1KhRxo8//lilrrVr1xp9+vQx/P39jcDAQOPWW281vvvuuyrjnnvuOaNNmzaGl5eXzWPoF6vVMAwjOjq6xkecv/32W2PKlCnGtddea7Ro0cJo1KiRERERYQwfPtzYuXOn3e2GDx9uSDIef/zxi/bb+01+rbaPnhuGYdx2223GwIEDL9qXl5dnTJo0ybj66qsNPz8/IyAgwIiPjzdeeOEFm0fi62Pz5s3G9ddfb/j5+RkhISFGamqqzSsIDOP/n/eMGTNs2qt75H3hwoU2Y++66y6jb9++DqkZcBaLYbhw1iIAmMTmzZvVr18/HThwQB06dHB1OQ0iPz9f7dq109KlS7myA49C2AEABxk0aJCuuOIKh6x35Y6eeOIJrV+/Xtu3b3d1KcAlIewAAABT42ksAABgaoQdAABgaoQdAABgaoQdAABgarxUUFJFRYXy8vLUrFkzFrcDAMBDGIahU6dOqXXr1vLysn/9hrAjKS8vr8pK0QAAwDMcOXJEV1xxhd1+wo6kZs2aSfrlxwoMDHRxNQAAoDaKi4sVGRlp/XvcHsKO/v+KzYGBgYQdAAA8TE1TUJigDAAATI2wAwAATI2wAwAATI05OwAAeIjy8nKVlZW5ugynady4sby9veu9H8IOAABuzjAM5efn68SJE64uxemCg4MVHh5er/fgEXYAAHBzlUEnNDRUAQEBl8ULcA3D0NmzZ1VYWChJioiIqPO+CDsAALix8vJya9Bp2bKlq8txKn9/f0lSYWGhQkND63xLiwnKAAC4sco5OgEBAS6uxDUqz7s+c5UIOwAAeIDL4dbVxTjivAk7AADA1Ag7AADA1Ag7AADA1Ag7AADA1Ag7AADAxrvvvquWLVuqpKTEpn3o0KEaPXq0i6qqO8IOAACwMXz4cJWXl2v58uXWtsLCQq1cuVL33XefCyurG14qCMCpBvTvr6KiIrv9lS8PsyckJETrv/iiIUoD8B/+/v665557tHDhQg0fPlyS9N577ykqKkr9+vVzbXF1QNgB4FRFRUXatna13f42MbHV9vdOTG6IsgD8yvjx49WzZ0/l5uaqTZs2WrRokcaOHeuR7/sh7AAAgCq6d++ubt266d1331VSUpL27dunlStXurqsOiHsAACAi3rggQc0b9485ebmKjExUZGRka4uqU6YoAwAAC7qnnvu0Y8//qg333zTIycmVyLsAACAiwoKCtKwYcPUtGlTDR061NXl1Bm3sQDAhPr2uUEFBfl2+8PCwvXlls1OrAieKjc3V6NGjZKvr6+rS6kzwg4AmFBBQb4mpT5pt39uxotOrAae6Oeff9aGDRu0YcMGvf76664up14IOwAAoIru3bvr559/1ssvv6xrrrnG1eXUC2EHADxQTbepcnPznFgNzOjw4cOuLsFhCDsA4IFquk01aerDTqwGcG88jQUAAEyNsAMAAEyN21gAHKqmhT6P5jGXBIBzEXYAOFRtFvoEAGfiNhYAADA1ruwAAOChcnJydOzYMaccq1WrVoqKinLKsRyNsAMAgAfKyclRx44ddfbsWaccLyAgQPv3769X4Pnwww+1YMECZWdn6/jx4/rmm28UFxfnuCLtIOwAAOCBjh07prNnz+qtP7+mazp0aNBjHfzXv3T/hN/r2LFj9Qo7Z86cUd++fTVixAiNHz/egRVWj7ADAIAHu6ZDB3Xv2sXVZdTK6NGjJTn/7cxMUAYAAKZG2AEAAKZG2AEAAA73/vvvq2nTptbP5s2bXVYLc3YAAIDD3Xbbberdu7f1e5s2bVxWC2EHAAA4XLNmzdSsWTNXlyGJsAMAAJzk+PHjysnJUd5/1sg7ePCgJCk8PFzh4eENdlzCDgAAHuzgv/7lMcdYvny5xo0bZ/0+cuRISdKMGTOUlpbmkGNcDGEHAAAP1KpVKwUEBOj+Cb93yvECAgLUqlWreu1j7NixGjt2rGMKugSEHQAAPFBUVJT279/P2li1QNgBAMBDRUVFeWwAcSbeswMAAEyNsAMAAEyNsAMAAEyNOTsALsmA/v1VVFRkt//of96fAQDugrAD4JIUFRVp29rVdvvbxMQ6sRoAqBm3sQAAgKkRdgAAgKlxGwsAAA+Vk5PDSwVrgbADAG6ob58bVFCQb7c/N5eJ4Je7nJwcxcTE6Ny5c045nr+/vw4cOHBJgWfTpk2aPXu2srOzdfToUX300UcaOnRowxVpB2EHANxQQUG+JqU+abd/0tSHnVgN3NGxY8d07tw5jRk1XuFhrRv0WPkFeXrn/Td17NixSwo7Z86cUbdu3XTffffpjjvuaMAKq0fYAQDAg4WHtVbUFdGuLuOiBg0apEGDBrm6DCYoAwAAcyPsAAAAU3ObsPPSSy/JYrHoscces7adP39eqampatmypZo2baphw4apoKDAZrucnBylpKQoICBAoaGhmjJlii5cuODk6gEAgLtyi7CzY8cOvfHGG+ratatN+6RJk/Tpp59q2bJl2rhxo/Ly8mwmOJWXlyslJUWlpaXKysrSO++8o0WLFmn69OnOPgUAAOCmXB52Tp8+rVGjRunNN99U8+bNre0nT57UW2+9pTlz5mjAgAGKj4/XwoULlZWVpa+++kqStGbNGn333Xd67733FBcXp0GDBum5555TRkaGSktLXXVKAADAjbg87KSmpiolJUWJiYk27dnZ2SorK7Npj4mJUVRUlLZu3SpJ2rp1q7p06aKwsDDrmOTkZBUXF2vfvn12j1lSUqLi4mKbDwA4U98+N6hD+w52P7xHB2Zw+vRp7dq1S7t27ZIkHTp0SLt27VJOTo5T63Dpo+dLly7Vzp07tWPHjip9+fn58vHxUXBwsE17WFiY8vPzrWP+O+hU9lf22TNz5kw988wz9aweAOqO9+jAUfILGj4Y1/UYX3/9tfr372/9PnnyZEnSmDFjtGjRIkeUVisuCztHjhzRo48+qszMTPn5+Tn12NOmTbP+4JJUXFysyMhIp9YAuKMB/furqKio2jFH87jiALiDVq1ayd/fX++8/6ZTjufv769WrVpd0jb9+vWTYRgNVFHtuSzsZGdnq7CwUNdee621rby8XJs2bdKf//xnrV69WqWlpTpx4oTN1Z2CggKFh4dLksLDw7V9+3ab/VY+rVU55mJ8fX3l6+vrwLMBzKGoqEjb1q6udkybmFgnVQOgOlFRUTpw4ABrY9WCy8LOwIEDtWfPHpu2cePGKSYmRlOnTlVkZKQaN26sdevWadiwYZKkgwcPKicnRwkJCZKkhIQEvfDCCyosLFRoaKgkKTMzU4GBgYqN5f+QAQDmFhUV5bEBxJlcFnaaNWumzp0727Q1adJELVu2tLbff//9mjx5slq0aKHAwEBNnDhRCQkJuu666yRJSUlJio2N1ejRozVr1izl5+fr6aefVmpqKlduAACAJDdfG2vu3Lny8vLSsGHDVFJSouTkZL3++uvWfm9vb61YsUIPP/ywEhIS1KRJE40ZM0bPPvusC6sGAADuxK3CzoYNG2y++/n5KSMjQxkZGXa3iY6O1meffdbAlQEA4FruMNHXFRxx3i5/zw4AALCvcePGkqSzZ8+6uBLXqDzvyt+hLtzqyg4AwDlyc/PUoX2HaseEhYXryy2bnVQR7PH29lZwcLAKCwslSQEBAbJYLC6uquEZhqGzZ8+qsLBQwcHB8vb2rvO+CDsAcBkqLy+v9qWGkjQ340UnVYOaVL5OpTLwXE6Cg4OrfZ1MbRB2AABwcxaLRREREQoNDVVZWZmry3Gaxo0b1+uKTiXCDgAAHsLb29shf/lfbpigDAAATI2wAwAATI2wAwAATI2wAwAATI2wAwAATI2wAwAATI2wAwAATI2wAwAATI2wAwAATI03KAPwKLm5uerSuXO1Y0JCQrT+iy+cVBEAd0fYAeBRDKNC29aurnZM78RkJ1UDwBNwGwsAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaT2MBgIP17XODCgryqx2Tm5vnpGoAEHYAwMEKCvI1KfXJasdMmvqwk6oBwG0sAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgajx6DsB0cnNz1aVzZ7v9ISEhWv/FF06sCIArEXYAmI5hVGjb2tV2+3snJjuxGs+Vm5unDu072O0PCwvXl1s2O7EioG4IOwCAiyovL6/25YhzM150YjVA3TFnBwAAmBphBwAAmBphBwAAmBphBwAAmBphBwAAmBphBwAAmBphBwAAmBphBwAAmBphBwAAmBphBwAAmBphBwAAmBphBwAAmBphBwAAmBphBwAAmBphBwAAmBphBwAAmBphBwAAmBphBwAAmBphBwAAmFojVxcAAJ6mb58bVFCQb7c/NzfPidUAqAlhBwAuUUFBvialPmm3f9LUh51YDYCacBsLAACYGmEHAACYGmEHAACYGmEHAACYGmEHAACYGmEHAACYGmEHAACYGmEHAACYGmEHAACYGmEHAACYGmEHAACYGmEHAACYmkvDzvz589W1a1cFBgYqMDBQCQkJ+vzzz63958+fV2pqqlq2bKmmTZtq2LBhKigosNlHTk6OUlJSFBAQoNDQUE2ZMkUXLlxw9qkAAAA35dKwc8UVV+ill15Sdna2vv76aw0YMEBDhgzRvn37JEmTJk3Sp59+qmXLlmnjxo3Ky8vTHXfcYd2+vLxcKSkpKi0tVVZWlt555x0tWrRI06dPd9UpAQAAN9PIlQe/9dZbbb6/8MILmj9/vr766itdccUVeuutt7RkyRINGDBAkrRw4UJ17NhRX331la677jqtWbNG3333ndauXauwsDDFxcXpueee09SpU5WWliYfHx9XnBYAAHAjbjNnp7y8XEuXLtWZM2eUkJCg7OxslZWVKTEx0TomJiZGUVFR2rp1qyRp69at6tKli8LCwqxjkpOTVVxcbL06dDElJSUqLi62+QAAAHNyedjZs2ePmjZtKl9fXz300EP66KOPFBsbq/z8fPn4+Cg4ONhmfFhYmPLz8yVJ+fn5NkGnsr+yz56ZM2cqKCjI+omMjHTsSQEAALfh8rBzzTXXaNeuXdq2bZsefvhhjRkzRt99912DHnPatGk6efKk9XPkyJEGPR4AAHAdl87ZkSQfHx+1b99ekhQfH68dO3boT3/6k+666y6VlpbqxIkTNld3CgoKFB4eLkkKDw/X9u3bbfZX+bRW5ZiL8fX1la+vr4PPBAAAuCOXX9n5tYqKCpWUlCg+Pl6NGzfWunXrrH0HDx5UTk6OEhISJEkJCQnas2ePCgsLrWMyMzMVGBio2NhYp9cOAADcj0uv7EybNk2DBg1SVFSUTp06pSVLlmjDhg1avXq1goKCdP/992vy5Mlq0aKFAgMDNXHiRCUkJOi6666TJCUlJSk2NlajR4/WrFmzlJ+fr6efflqpqalcuQEAAJJcHHYKCwv129/+VkePHlVQUJC6du2q1atX6+abb5YkzZ07V15eXho2bJhKSkqUnJys119/3bq9t7e3VqxYoYcfflgJCQlq0qSJxowZo2effdZVpwQAANyMS8POW2+9VW2/n5+fMjIylJGRYXdMdHS0PvvsM0eXBgAATMLlE5QBAJ4pNzdPHdp3sNsfFhauL7dsdmJFwMURdgAAdVJeXq5JqU/a7Z+b8aITqwHsc7unsQAAAByJsAMAAEyNsAMAAEyNOTsA8Ct9+9ygggL76+vl5uY5sRoA9UXYAYBfKSjIr3bi7aSpDzuxGgD1xW0sAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgaoQdAABgajx6DlxGBvTvr6KiIrv9R/N4fwwA8yHsAJeRoqIibVu72m5/m5hYJ1YDAM7BbSwAAGBqhB0AAGBqhB0AAGBqhB0AAGBqdQo7V155pX766acq7SdOnNCVV15Z76IAAAAcpU5h5/DhwyovL6/SXlJSotzc3HoXBQAA4CiX9Oj58uXLrf979erVCgoKsn4vLy/XunXr1LZtW4cVBwAAUF+XFHaGDh0qSbJYLBozZoxNX+PGjdW2bVu9+uqrDisOAACgvi4p7FRUVEiS2rVrpx07dqhVq1YNUhSAuuENyQBQVZ3eoHzo0CFH1wHAAXhDMgBUVeflItatW6d169apsLDQesWn0ttvv13vwgAAAByhTmHnmWee0bPPPqsePXooIiJCFovF0XUBAAA4RJ3CzoIFC7Ro0SKNHj3a0fUAAAA4VJ3es1NaWqrrr7/e0bUAAAA4XJ3CzgMPPKAlS5Y4uhYAAACHq9NtrPPnz+svf/mL1q5dq65du6px48Y2/XPmzHFIcQAAAPVVp7Dz7bffKi4uTpK0d+9emz4mKwMAAHdSp7DzxRdfOLoOAACABlGnOTsAAACeok5Xdvr371/t7ar169fXuSAAAABHqlPYqZyvU6msrEy7du3S3r17qywQCgAA4Ep1Cjtz5869aHtaWppOnz5dr4IAAAAcyaFzdu69917WxQIAAG7FoWFn69at8vPzc+QuAQAA6qVOt7HuuOMOm++GYejo0aP6+uuv9cc//tEhhQEAADhCncJOUFCQzXcvLy9dc801evbZZ5WUlOSQwgAAAByhTmFn4cKFjq4DAACgQdQp7FTKzs7W/v37JUmdOnVS9+7dHVIUAACAo9Qp7BQWFmrkyJHasGGDgoODJUknTpxQ//79tXTpUoWEhDiyRgAAgDqr09NYEydO1KlTp7Rv3z4dP35cx48f1969e1VcXKzf//73jq4RAACgzup0ZWfVqlVau3atOnbsaG2LjY1VRkYGE5QBAIBbqdOVnYqKCjVu3LhKe+PGjVVRUVHvogAAABylTmFnwIABevTRR5WXl2dty83N1aRJkzRw4ECHFQcAAFBfdQo7f/7zn1VcXKy2bdvqqquu0lVXXaV27dqpuLhY6enpjq4RAACgzuo0ZycyMlI7d+7U2rVrdeDAAUlSx44dlZiY6NDiAAAA6uuSruysX79esbGxKi4ulsVi0c0336yJEydq4sSJ6tmzpzp16qTNmzc3VK0AAACX7JLCzrx58zR+/HgFBgZW6QsKCtLvfvc7zZkzx2HFAQAA1NclhZ3du3frlltusduflJSk7OzsehcFAADgKJcUdgoKCi76yHmlRo0aqaioqN5FAQAAOMolTVBu06aN9u7dq/bt21+0/9tvv1VERIRDCgMAeLbc3Dx1aN/Bbn9YWLi+3MI8TzS8Swo7gwcP1h//+Efdcsst8vPzs+k7d+6cZsyYod/85jcOLRAA4JnKy8s1KfVJu/1zM150YjW4nF1S2Hn66af14Ycf6uqrr9aECRN0zTXXSJIOHDigjIwMlZeX66mnnmqQQgHAUXJzc9Wlc2e7/UVFhU6sBkBDu6SwExYWpqysLD388MOaNm2aDMOQJFksFiUnJysjI0NhYWENUigAOIphVGjb2tV2+1tf3dFuHwDPc8kvFYyOjtZnn32mn3/+Wd9//70Mw1CHDh3UvHnzhqgPAACgXur0BmVJat68uXr27OnIWgAAAByuTmtjAQAAeArCDgAAMDXCDgAAMDXCDgAAMDXCDgAAMDWXhp2ZM2eqZ8+eatasmUJDQzV06FAdPHjQZsz58+eVmpqqli1bqmnTpho2bJgKCgpsxuTk5CglJUUBAQEKDQ3VlClTdOHCBWeeCgAAcFMuDTsbN25UamqqvvrqK2VmZqqsrExJSUk6c+aMdcykSZP06aefatmyZdq4caPy8vJ0xx13WPvLy8uVkpKi0tJSZWVl6Z133tGiRYs0ffp0V5wSAABwM3V+z44jrFq1yub7okWLFBoaquzsbN144406efKk3nrrLS1ZskQDBgyQJC1cuFAdO3bUV199peuuu05r1qzRd999p7Vr1yosLExxcXF67rnnNHXqVKWlpcnHx8cVpwYAANyEW83ZOXnypCSpRYsWkqTs7GyVlZUpMTHROiYmJkZRUVHaunWrJGnr1q3q0qWLzTIVycnJKi4u1r59+y56nJKSEhUXF9t8AACAOblN2KmoqNBjjz2mPn36qPN/FujLz8+Xj4+PgoODbcaGhYUpPz/fOubX63FVfq8c82szZ85UUFCQ9RMZGengswEAAO7CbcJOamqq9u7dq6VLlzb4saZNm6aTJ09aP0eOHGnwYwIAANdw6ZydShMmTNCKFSu0adMmXXHFFdb28PBwlZaW6sSJEzZXdwoKChQeHm4ds337dpv9VT6tVTnm13x9feXr6+vgswAAAO7IpVd2DMPQhAkT9NFHH2n9+vVq166dTX98fLwaN26sdevWWdsOHjyonJwcJSQkSJISEhK0Z88eFRYWWsdkZmYqMDBQsbGxzjkRAADgtlx6ZSc1NVVLlizRJ598ombNmlnn2AQFBcnf319BQUG6//77NXnyZLVo0UKBgYGaOHGiEhISdN1110mSkpKSFBsbq9GjR2vWrFnKz8/X008/rdTUVK7eAAAA14ad+fPnS5L69etn075w4UKNHTtWkjR37lx5eXlp2LBhKikpUXJysl5//XXrWG9vb61YsUIPP/ywEhIS1KRJE40ZM0bPPvuss04DAAC4MZeGHcMwahzj5+enjIwMZWRk2B0THR2tzz77zJGlAQAAk3Cbp7EAAAAaAmEHAACYGmEHAACYGmEHAACYGmEHAACYGmEHAACYGmEHAACYGmEHAACYGmEHAACYGmEHAACYGmEHAACYGmEHAACYGmEHAACYGmEHAACYGmEHAACYWiNXFwAAuDzl5uapQ/sOdvvDwsL15ZbNTqwIZkXYAQC4RHl5uSalPmm3f27Gi06sBmbGbSwAAGBqXNkBcNm5cKFc6enp9vvLy51YDYCGRtgBcBkylNDzhmr67QchAJ6H21gAAMDUCDsAAMDUCDsAAMDUCDsAAMDUCDsAAMDUCDsAAMDUCDsAAMDUCDsAAMDUCDsAAMDUCDsAAMDUCDsAAMDUWBsL8CAD+vdXUVGR3f6jeXlOrAYAPANhB/AgRUVF2rZ2td3+NjGxTqwGADwDt7EAAICpEXYAAICpEXYAAICpEXYAAICpMUEZcCM8bQUAjkfYAdwIT1u5B6OiQi/PSbPb7+VlcV4xAOqNsAMAv+Ll5aXF6a/b7R88aoQTqwFQX8zZAQAApkbYAQAApkbYAQAApkbYAQAApkbYAQAApkbYAQAApkbYAQAApkbYAQAApkbYAQAApkbYAQAApkbYAQAApkbYAQAApkbYAQAApkbYAQAApkbYAQAApkbYAQAApkbYAQAApkbYAQAAptbI1QUAgKNduFCu9PR0+wMM59UCwPUIOwBMyFBCzxuq6a8mCNVm7xUVenlOmt1+Ly9LvfYPwLEIOwBwiby8vLQ4/XW7/YNHjXBiNQBqwpwdAABgaoQdAABgaoQdAABgaoQdAABgai4NO5s2bdKtt96q1q1by2Kx6OOPP7bpNwxD06dPV0REhPz9/ZWYmKh//etfNmOOHz+uUaNGKTAwUMHBwbr//vt1+vRpJ54FAABwZy4NO2fOnFG3bt2UkZFx0f5Zs2bptdde04IFC7Rt2zY1adJEycnJOn/+vHXMqFGjtG/fPmVmZmrFihXatGmTHnzwQWedAgAAcHMuffR80KBBGjRo0EX7DMPQvHnz9PTTT2vIkCGSpHfffVdhYWH6+OOPNXLkSO3fv1+rVq3Sjh071KNHD0lSenq6Bg8erFdeeUWtW7d22rkANRnQv7+KioqqHXM0L89J1QDA5cNt37Nz6NAh5efnKzEx0doWFBSk3r17a+vWrRo5cqS2bt2q4OBga9CRpMTERHl5eWnbtm26/fbbXVE6cFFFRUXatnZ1tWPaxMQ6qRrA/eXm5qlD+w7VjgkLC9eXWzY7qSJ4KrcNO/n5+ZKksLAwm/awsDBrX35+vkJDQ236GzVqpBYtWljHXExJSYlKSkqs34uLix1VNgDAQcrLyzUp9clqx8zNeNFJ1cCTuW3YaUgzZ87UM8884+oyANRBjeteSax9BcCG24ad8PBwSVJBQYEiIiKs7QUFBYqLi7OOKSwstNnuwoULOn78uHX7i5k2bZomT55s/V5cXKzIyEgHVg+g4dS07pVU37WvAJiL275np127dgoPD9e6deusbcXFxdq2bZsSEhIkSQkJCTpx4oSys7OtY9avX6+Kigr17t3b7r59fX0VGBho8wEAAObk0is7p0+f1vfff2/9fujQIe3atUstWrRQVFSUHnvsMT3//PPq0KGD2rVrpz/+8Y9q3bq1hg4dKknq2LGjbrnlFo0fP14LFixQWVmZJkyYoJEjR/IkFgBcBmqaxMwEZkguDjtff/21+vfvb/1eeWtpzJgxWrRokR5//HGdOXNGDz74oE6cOKG+fftq1apV8vPzs27z/vvva8KECRo4cKC8vLw0bNgwvfbaa04/FwCA89U0iZkJzJBcHHb69esnw7A/k9BisejZZ5/Vs88+a3dMixYttGTJkoYoDwAAmIDbztkBAABwBMIOAAAwNcIOAAAwNbd9zw7gaWpa+4p1rwDANQg7gIPUtPYV614BgGtwGwsAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaYQcAAJgaLxUEAAczKir08py0asd4eVmcUwwAwg4AOJqXl5cWp79e7ZjBo0Y4qRoA3MYCAACmRtgBAACmxm0soJZY1RwAPBNhB5eNmsJKSEiI1n/xhd1+VjUHAM9E2MFlo6aw0jsx2YnVAACchTk7AADA1Ag7AADA1Ag7AADA1Ag7AADA1Ag7AADA1HgaC/iP3Nxcdenc2W4/79EBAM9E2AH+wzAqeI8OnKamxUJZKBRwHMIOALdy4UK50tPT7Q8wnFdLQ6ppsVAWCgUch7ADwM0YSuh5QzX91QQhALgIwg4AwLRyc/PUoX0Hu/1hYeH6cstmJ1YEVyDsAABMq7y8XJNSn7TbPzfjRSdWA1fh0XMAAGBqhB0AAGBqhB0AAGBqhB0AAGBqhB0AAGBqhB0AAGBqhB0AAGBqvGcHgFNdLstBAHAfhB0ATsZyEACci9tYAADA1Ag7AADA1Ag7AADA1JizAwC4bLEq+uWBsAMAuGyxKvrlgbADUxjQv7+KioqqHXM0L89J1QAA3AlhBx6hpjBzNC9POd/tqXYfbWJiHV0W4DJeXha9PCet2n4AvyDswCMUFRVp29rVdvsJMrjceHt7aXH663b7B48a4cRqAPdG2AEAN2RUVFR75aa+23PlB5cTwg4AuCEvr+qv3KTUcOWmpu258oPLCe/ZAQAApsaVHQAOxUKfANwNYQfAJak5zLDQJwD3QtiBU9T06HhISIjWf/GFEytC3RFmAHgWwg6coqZHx6/o2FldOne2288LAQEAdUXYgVswjAreowMAaBCEHQAA7GChUHMg7AAAYAcLhZoD79kBAACmxpUdALgM1WY5CpaUgFkQdgDgMlTTchISS0rAPAg7AKxqfGGgxBuQgf9S0wRmSSosKlJoSIjdfiY5NzzCDoD/UtMLAyVeGnj5YOX0mtU0gVmSJk19mEnOLkbYgUPU9IZkXgoIeB5WTodZEHbgEDW9IZmXAgLmU9OVH29vC1eG4BZME3YyMjI0e/Zs5efnq1u3bkpPT1evXr1cXZZTsO4UAFeo6cpPyqgRXBmCWzBF2Pnggw80efJkLViwQL1799a8efOUnJysgwcPKjQ01NXlNbiarqr0Tkyu1/5rClMSt6kAAO7LFGFnzpw5Gj9+vMaNGydJWrBggVauXKm3335bTzzxhIurq7+Gng9Tm/3nfLen2n1wm8o91PQ01YWyC9U/bcWTVnAiJkD/giUpGp7Hh53S0lJlZ2dr2rRp1jYvLy8lJiZq69atLqzMcRp6PgzzbWqnxiBxodyJ1dhT09NU6TX2A87iiAnQXl6ePy+IJSkanseHnWPHjqm8vFxhYWE27WFhYTpw4MBFtykpKVFJSYn1+8mTJyVJxcXFDq/vNykpOnbsmN3+Vq1aacXKldXuo7y8XMWnTtntNwyj2v4ff/xRsR072u0/kpOj2a+8Yre/rOxCtfuvHFOffdR0DvXtr1WNpWXV9hsVFeraKb7afnf4Hc+cPdtg/c44BjW6R7871FBRXq6Zr0yvtkYvL4sWvGz/35lh942udh8Wi3Tu/Llq+6vb3sur+v7aHMMwjGr7KyoqGuTvp0uRdHOyiooK7fYXHTumkFat7PaHhIRqTab9/6iuq8rfxTBquCxteLjc3FxDkpGVlWXTPmXKFKNXr14X3WbGjBmGfrlgz4cPHz58+PDx8M+RI0eqzQoef2WnVatW8vb2VkFBgU17QUGBwsPDL7rNtGnTNHnyZOv3iooKHT9+XC1btpTFcmmXPIuLixUZGakjR44oMDDw0k8A/IYOwu9Yf/yGjsHvWH/8hrVjGIZOnTql1q1bVzvO48OOj4+P4uPjtW7dOg0dOlTSL+Fl3bp1mjBhwkW38fX1la+vr01bcHBwveoIDAzkD2Q98Rs6Br9j/fEbOga/Y/3xG9YsKCioxjEeH3YkafLkyRozZox69OihXr16ad68eTpz5oz16SwAAHD5MkXYueuuu1RUVKTp06crPz9fcXFxWrVqVZVJywAA4PJjirAjSRMmTLB726oh+fr6asaMGVVui6H2+A0dg9+x/vgNHYPfsf74DR3LYhg1Pa8FAADgubxcXQAAAEBDIuwAAABTI+wAAABTI+wAAABTI+w42MqVK9W7d2/5+/urefPm1hcd4tKVlJQoLi5OFotFu3btcnU5HuPw4cO6//771a5dO/n7++uqq67SjBkzVFpa6urS3F5GRobatm0rPz8/9e7dW9u3b3d1SR5l5syZ6tmzp5o1a6bQ0FANHTpUBw8edHVZHu2ll16SxWLRY4895upSPBphx4H+8Y9/aPTo0Ro3bpx2796tLVu26J577nF1WR7r8ccfr/EV4KjqwIEDqqio0BtvvKF9+/Zp7ty5WrBggZ580v6qypA++OADTZ48WTNmzNDOnTvVrVs3JScnq7DQ/uKHsLVx40alpqbqq6++UmZmpsrKypSUlKQzZ864ujSPtGPHDr3xxhvq2rWrq0vxfI5ZjhNlZWVGmzZtjL/+9a+uLsUUPvvsMyMmJsbYt2+fIcn45ptvXF2SR5s1a5bRrl07V5fh1nr16mWkpqZav5eXlxutW7c2Zs6c6cKqPFthYaEhydi4caOrS/E4p06dMjp06GBkZmYaN910k/Hoo4+6uiSPxpUdB9m5c6dyc3Pl5eWl7t27KyIiQoMGDdLevXtdXZrHKSgo0Pjx47V48WIFBAS4uhxTOHnypFq0aOHqMtxWaWmpsrOzlZiYaG3z8vJSYmKitm7d6sLKPNvJkycliT97dZCamqqUlBSbP5OoO8KOg/zwww+SpLS0ND399NNasWKFmjdvrn79+un48eMurs5zGIahsWPH6qGHHlKPHj1cXY4pfP/990pPT9fvfvc7V5fito4dO6by8vIqS8yEhYUpPz/fRVV5toqKCj322GPq06ePOnfu7OpyPMrSpUu1c+dOzZw509WlmAZhpwZPPPGELBZLtZ/KORKS9NRTT2nYsGGKj4/XwoULZbFYtGzZMhefhevV9ndMT0/XqVOnNG3aNFeX7HZq+xv+t9zcXN1yyy0aPny4xo8f76LKcTlKTU3V3r17tXTpUleX4lGOHDmiRx99VO+//778/PxcXY5psFxEDYqKivTTTz9VO+bKK6/Uli1bNGDAAG3evFl9+/a19vXu3VuJiYl64YUXGrpUt1bb33HEiBH69NNPZbFYrO3l5eXy9vbWqFGj9M477zR0qW6rtr+hj4+PJCkvL0/9+vXTddddp0WLFsnLi/+2sae0tFQBAQH6+9//bvME5ZgxY3TixAl98sknrivOA02YMEGffPKJNm3apHbt2rm6HI/y8ccf6/bbb5e3t7e1rby8XBaLRV5eXiopKbHpQ+2YZiHQhhISEqKQkJAax8XHx8vX11cHDx60hp2ysjIdPnxY0dHRDV2m26vt7/jaa6/p+eeft37Py8tTcnKyPvjgA/Xu3bshS3R7tf0NpV+u6PTv3996hZGgUz0fHx/Fx8dr3bp11rBTUVGhdevWuWSBYU9lGIYmTpyojz76SBs2bCDo1MHAgQO1Z88em7Zx48YpJiZGU6dOJejUEWHHQQIDA/XQQw9pxowZioyMVHR0tGbPni1JGj58uIur8xxRUVE235s2bSpJuuqqq3TFFVe4oiSPk5ubq379+ik6OlqvvPKKioqKrH3h4eEurMy9TZ48WWPGjFGPHj3Uq1cvzZs3T2fOnNG4ceNcXZrHSE1N1ZIlS/TJJ5+oWbNm1vlOQUFB8vf3d3F1nqFZs2ZV5jg1adJELVu2ZO5TPRB2HGj27Nlq1KiRRo8erXPnzql3795av369mjdv7urScBnJzMzU999/r++//75KQOSutX133XWXioqKNH36dOXn5ysuLk6rVq2qMmkZ9s2fP1+S1K9fP5v2hQsXauzYsc4vCPgP5uwAAABT40Y+AAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAAAwNcIOAOiXlc8BmBNhB4Bb+/vf/64uXbrI399fLVu2VGJios6cOSNJevvtt9WpUyf5+voqIiLCZoXynJwcDRkyRE2bNlVgYKBGjBihgoICa39aWpri4uL017/+Ve3atZOfn58k6cSJE3rggQcUEhKiwMBADRgwQLt373buSQNwKMIOALd19OhR3X333brvvvu0f/9+bdiwQXfccYcMw9D8+fOVmpqqBx98UHv27NHy5cvVvn17SVJFRYWGDBmi48ePa+PGjcrMzNQPP/ygu+66y2b/33//vf7xj3/oww8/1K5duyRJw4cPV2FhoT7//HNlZ2fr2muv1cCBA3X8+HFnnz4AB2EhUABua+fOnYqPj9fhw4cVHR1t09emTRuNGzdOzz//fJXtMjMzNWjQIB06dEiRkZGSpO+++06dOnXS9u3b1bNnT6WlpenFF19Ubm6uQkJCJElffvmlUlJSVFhYKF9fX+v+2rdvr8cff1wPPvhgA54tgIbSyNUFAIA93bp108CBA9WlSxclJycrKSlJd955p8rKypSXl6eBAwdedLv9+/crMjLSGnQkKTY2VsHBwdq/f7969uwpSYqOjrYGHUnavXu3Tp8+rZYtW9rs79y5c/r3v//dAGcIwBkIOwDclre3tzIzM5WVlaU1a9YoPT1dTz31lNatW+eQ/Tdp0sTm++nTpxUREaENGzZUGRscHOyQYwJwPsIOALdmsVjUp08f9enTR9OnT1d0dLQyMzPVtm1brVu3Tv3796+yTceOHXXkyBEdOXLE5jbWiRMnFBsba/dY1157rfLz89WoUSO1bdu2oU4JgJMRdgC4rW3btmndunVKSkpSaGiotm3bpqKiInXs2FFpaWl66KGHFBoaqkGDBunUqVPasmWLJk6cqMTERHXp0kWjRo3SvHnzdOHCBT3yyCO66aab1KNHD7vHS0xMVEJCgoYOHapZs2bp6quvVl5enlauXKnbb7+92m0BuC/CDgC3FRgYqE2bNmnevHkqLi5WdHS0Xn31VQ0aNEiSdP78ec2dO1d/+MMf1KpVK915552Sfrka9Mknn2jixIm68cYb5eXlpVtuuUXp6enVHs9iseizzz7TU089pXHjxqmoqEjh4eG68cYbFRYW1uDnC6Bh8DQWAAAwNd6zAwAATI2wAwAATI2wAwAATI2wAwAATI2wAwAATI2wAwAATI2wAwAATI2wAwAATI2wAwAATI2wAwAATI2wAwAATI2wAwAATO3/AX3mCfMKVdoTAAAAAElFTkSuQmCC\n" + }, + "metadata": {} + } + ], + "source": [ + "import seaborn as sns\n", + "import pandas as pd\n", + "import warnings\n", + "import matplotlib.pyplot as plt\n", + "warnings.filterwarnings(\"ignore\", \"is_categorical_dtype\")\n", + "warnings.filterwarnings(\"ignore\", \"use_inf_as_na\")\n", + "\n", + "score = grid_search.decision_function(X)\n", + "df = pd.DataFrame({'score': score, 'y': y})\n", + "sns.histplot(data=df, x=\"score\", hue=\"y\").set_title(\"Smooth SVM (C=0.1)\")\n", + "plt.show()" + ] + } + ], + "metadata": { + "colab": { + "provenance": [] + }, + "kernelspec": { + "display_name": "Python 3", + "name": "python3" + }, + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} \ No newline at end of file diff --git a/doc/source/tutorials/constraint.rst b/doc/source/tutorials/constraint.rst index e81c25a..6c483ca 100644 --- a/doc/source/tutorials/constraint.rst +++ b/doc/source/tutorials/constraint.rst @@ -73,6 +73,13 @@ Increasing: :math:`\beta_i \le \beta_{i+1}`. Decreasing: :math:`\beta_i \ge \bet # Monotonically decreasing constraint = [{'name': 'monotonic', 'decreasing': True}] +**Related Example** + +.. nblinkgallery:: + :name: monotonic-gallery + + ../examples/MonotonicSVM.ipynb + Custom Constraints ^^^^^^^^^^^^^^^^^^ Define arbitrary linear constraints of the form :math:`A\beta + b \ge 0`. diff --git a/doc/source/tutorials/loss.rst b/doc/source/tutorials/loss.rst index cb336c0..1af7abe 100644 --- a/doc/source/tutorials/loss.rst +++ b/doc/source/tutorials/loss.rst @@ -53,6 +53,7 @@ A smoothed version of the Hinge loss (using ReHU) that is differentiable everywh :name: ssvm-gallery ../examples/SVM.ipynb + ../examples/Smooth_SVM_new.ipynb Squared SVM ^^^^^^^^^^^ From 4039741a2c67c4123fe5706260c571238c988c2e Mon Sep 17 00:00:00 2001 From: fzh <1155233259@link.cuhk.edu.hk> Date: Sat, 11 Apr 2026 13:23:16 +0800 Subject: [PATCH 2/2] docs: two notebooks with index removed --- doc/source/autoapi/rehline/index.rst | 1419 -------------------------- 1 file changed, 1419 deletions(-) delete mode 100644 doc/source/autoapi/rehline/index.rst diff --git a/doc/source/autoapi/rehline/index.rst b/doc/source/autoapi/rehline/index.rst deleted file mode 100644 index 96d1691..0000000 --- a/doc/source/autoapi/rehline/index.rst +++ /dev/null @@ -1,1419 +0,0 @@ - -rehline -======= - -.. py:module:: rehline - - -Overview --------- - -.. list-table:: Classes - :header-rows: 0 - :widths: auto - :class: summarytable - - * - :py:obj:`CQR_Ridge ` - - Composite Quantile Regressor (CQR) with a ridge penalty. - * - :py:obj:`ReHLine ` - - ReHLine Minimization. - * - :py:obj:`plqERM_Ridge ` - - Empirical Risk Minimization (ERM) with a piecewise linear-quadratic (PLQ) objective with a ridge penalty. - * - :py:obj:`plqERM_ElasticNet ` - - Empirical Risk Minimization (ERM) with a piecewise linear-quadratic (PLQ) objective with a elastic net penalty. - * - :py:obj:`plq_Ridge_Classifier ` - - Empirical Risk Minimization (ERM) Classifier with a Piecewise Linear-Quadratic (PLQ) loss - * - :py:obj:`plq_Ridge_Regressor ` - - Empirical Risk Minimization (ERM) regressor with a Piecewise Linear-Quadratic (PLQ) loss - * - :py:obj:`plqMF_Ridge ` - - Matrix Factorization (MF) with a piecewise linear-quadratic objective and ridge penalty. - - -.. list-table:: Function - :header-rows: 0 - :widths: auto - :class: summarytable - - * - :py:obj:`ReHLine_solver `\ (X, U, V, Tau, S, T, A, b, rho, Lambda, Gamma, xi, mu, max_iter, tol, shrink, verbose, trace_freq) - - \- - * - :py:obj:`plqERM_Ridge_path_sol `\ (X, y, \*None, loss, constraint, eps, n_Cs, Cs, max_iter, tol, verbose, shrink, warm_start, return_time) - - Compute the PLQ Empirical Risk Minimization (ERM) path over a range of regularization parameters. - * - :py:obj:`make_mf_dataset `\ (n_users, n_items, n_factors, n_interactions, density, noise_std, seed, rating_min, rating_max, return_params) - - Generate synthetic rating data using matrix factorization model. - - - -Classes -------- - -.. py:class:: CQR_Ridge(quantiles, C=1.0, max_iter=1000, tol=0.0001, shrink=1, warm_start=0, verbose=0, trace_freq=100) - - Bases: :py:obj:`rehline._base._BaseReHLine`, :py:obj:`sklearn.base.BaseEstimator` - - Composite Quantile Regressor (CQR) with a ridge penalty. - - It allows for the fitting of a linear regression model that minimizes a composite quantile loss function. - - .. math:: - - \min_{\mathbf{\beta} \in \mathbb{R}^d, \mathbf{\beta_0} \in \mathbb{R}^K} \sum_{k=1}^K \sum_{i=1}^n \text{PLQ}(y_i, \mathbf{x}_i^T \mathbf{\beta} + \mathbf{\beta_0k}) + \frac{1}{2} \| \mathbf{\beta} \|_2^2. - - - Parameters - ---------- - quantiles : list of float (n_quantiles,) - The quantiles to be estimated. - - C : float, default=1.0 - Regularization parameter. The strength of the regularization is - inversely proportional to C. Must be strictly positive. - `C` will be absorbed by the ReHLine parameters when `self.make_ReLHLoss` is conducted. - - verbose : int, default=0 - Enable verbose output. Note that this setting takes advantage of a - per-process runtime setting in liblinear that, if enabled, may not work - properly in a multithreaded context. - - max_iter : int, default=1000 - The maximum number of iterations to be run. - - tol : float, default=1e-4 - The tolerance for the stopping criterion. - - shrink : float, default=1 - The shrinkage of dual variables for the ReHLine algorithm. - - warm_start : bool, default=False - Whether to use the given dual params as an initial guess for the - optimization algorithm. - - trace_freq : int, default=100 - The frequency at which to print the optimization trace. - - Attributes - ---------- - coef\_ : array-like - The optimized model coefficients. - - intercept\_ : array-like - The optimized model intercepts. - - quantiles\_: array-like - The quantiles to be estimated. - - n_iter\_ : int - The number of iterations performed by the ReHLine solver. - - opt_result\_ : object - The optimization result object. - - dual_obj\_ : array-like - The dual objective function values. - - primal_obj\_ : array-like - The primal objective function values. - - Methods - ------- - fit(X, y, sample_weight=None) - Fit the model based on the given training data. - - predict(X) - The prediction for the given dataset. - - - Overview - ======== - - - .. list-table:: Methods - :header-rows: 0 - :widths: auto - :class: summarytable - - * - :py:obj:`fit `\ (X, y, sample_weight) - - Fit the model based on the given training data. - * - :py:obj:`predict `\ (X) - - The decision function evaluated on the given dataset - - - Members - ======= - - .. py:method:: fit(X, y, sample_weight=None) - - Fit the model based on the given training data. - - Parameters - ---------- - - X: {array-like} of shape (n_samples, n_features) - Training vector, where `n_samples` is the number of samples and - `n_features` is the number of features. - - y : array-like of shape (n_samples,) - The target variable. - - sample_weight : array-like of shape (n_samples,), default=None - Array of weights that are assigned to individual - samples. If not provided, then each sample is given unit weight. - - Returns - ------- - self : object - An instance of the estimator. - - - - - .. py:method:: predict(X) - - The decision function evaluated on the given dataset - - Parameters - ---------- - X : array-like of shape (n_samples, n_features) - The data matrix. - - Returns - ------- - ndarray of shape (n_samples, n_quantiles) - Returns the decision function of the samples. - - - - -.. py:class:: ReHLine(C=1.0, U=np.empty(shape=(0, 0)), V=np.empty(shape=(0, 0)), Tau=np.empty(shape=(0, 0)), S=np.empty(shape=(0, 0)), T=np.empty(shape=(0, 0)), A=np.empty(shape=(0, 0)), b=np.empty(shape=0), max_iter=1000, tol=0.0001, shrink=1, warm_start=0, verbose=0, trace_freq=100) - - Bases: :py:obj:`rehline._base._BaseReHLine`, :py:obj:`sklearn.base.BaseEstimator` - - ReHLine Minimization. - - .. math:: - - \min_{\mathbf{\beta} \in \mathbb{R}^d} \sum_{i=1}^n \sum_{l=1}^L \text{ReLU}( u_{li} \mathbf{x}_i^\intercal \mathbf{\beta} + v_{li}) + \sum_{i=1}^n \sum_{h=1}^H {\text{ReHU}}_{\tau_{hi}}( s_{hi} \mathbf{x}_i^\intercal \mathbf{\beta} + t_{hi}) + \frac{1}{2} \| \mathbf{\beta} \|_2^2, \\ \text{ s.t. } - \mathbf{A} \mathbf{\beta} + \mathbf{b} \geq \mathbf{0}, - - where :math:`\mathbf{U} = (u_{li}),\mathbf{V} = (v_{li}) \in \mathbb{R}^{L \times n}` - and :math:`\mathbf{S} = (s_{hi}),\mathbf{T} = (t_{hi}),\mathbf{\tau} = (\tau_{hi}) \in \mathbb{R}^{H \times n}` - are the ReLU-ReHU loss parameters, and :math:`(\mathbf{A},\mathbf{b})` are the constraint parameters. - - Parameters - ---------- - C : float, default=1.0 - Regularization parameter. The strength of the regularization is - inversely proportional to C. Must be strictly positive. - - _U, _V: array of shape (L, n_samples), default=np.empty(shape=(0, 0)) - The parameters pertaining to the ReLU part in the loss function. - - _Tau, _S, _T: array of shape (H, n_samples), default=np.empty(shape=(0, 0)) - The parameters pertaining to the ReHU part in the loss function. - - _A: array of shape (K, n_features), default=np.empty(shape=(0, 0)) - The coefficient matrix in the linear constraint. - - _b: array of shape (K, ), default=np.empty(shape=0) - The intercept vector in the linear constraint. - - verbose : int, default=0 - Enable verbose output. - - max_iter : int, default=1000 - The maximum number of iterations to be run. - - tol : float, default=1e-4 - The tolerance for the stopping criterion. - - shrink : float, default=1 - The shrinkage of dual variables for the ReHLine algorithm. - - warm_start : bool, default=False - Whether to use the given dual params as an initial guess for the - optimization algorithm. - - trace_freq : int, default=100 - The frequency at which to print the optimization trace. - - Attributes - ---------- - coef\_ : array-like - The optimized model coefficients. - - n_iter\_ : int - The number of iterations performed by the ReHLine solver. - - opt_result\_ : object - The optimization result object. - - dual_obj\_ : array-like - The dual objective function values. - - primal_obj\_ : array-like - The primal objective function values. - - _Lambda: array-like - The optimized dual variables for ReLU parts. - - _Gamma: array-like - The optimized dual variables for ReHU parts. - - _xi: array-like - The optimized dual variables for linear constraints. - - Examples - -------- - - >>> ## test SVM on simulated dataset - >>> import numpy as np - >>> from rehline import ReHLine - - >>> # simulate classification dataset - >>> n, d, C = 1000, 3, 0.5 - >>> np.random.seed(1024) - >>> X = np.random.randn(1000, 3) - >>> beta0 = np.random.randn(3) - >>> y = np.sign(X.dot(beta0) + np.random.randn(n)) - - >>> # Usage of ReHLine - >>> n, d = X.shape - >>> U = -(C*y).reshape(1,-1) - >>> L = U.shape[0] - >>> V = (C*np.array(np.ones(n))).reshape(1,-1) - >>> clf = ReHLine(C=C) - >>> clf._U, clf._V = U, V - >>> clf.fit(X=X) - >>> print('sol privided by rehline: %s' %clf.coef_) - >>> sol privided by rehline: [ 0.7410154 -0.00615574 2.66990408] - >>> print(clf.decision_function([[.1,.2,.3]])) - >>> [0.87384162] - - References - ---------- - .. [1] `Dai, B., Qiu, Y,. (2023). ReHLine: Regularized Composite ReLU-ReHU Loss Minimization with Linear Computation and Linear Convergence `_ - - - Overview - ======== - - - .. list-table:: Methods - :header-rows: 0 - :widths: auto - :class: summarytable - - * - :py:obj:`fit `\ (X, sample_weight) - - Fit the model based on the given training data. - * - :py:obj:`decision_function `\ (X) - - The decision function evaluated on the given dataset - - - Members - ======= - - .. py:method:: fit(X, sample_weight=None) - - Fit the model based on the given training data. - - Parameters - ---------- - - X: {array-like} of shape (n_samples, n_features) - Training vector, where `n_samples` is the number of samples and - `n_features` is the number of features. - - sample_weight : array-like of shape (n_samples,), default=None - Array of weights that are assigned to individual - samples. If not provided, then each sample is given unit weight. - - Returns - ------- - self : object - An instance of the estimator. - - - .. py:method:: decision_function(X) - - The decision function evaluated on the given dataset - - Parameters - ---------- - X : array-like of shape (n_samples, n_features) - The data matrix. - - Returns - ------- - ndarray of shape (n_samples, ) - Returns the decision function of the samples. - - - - -.. py:class:: plqERM_Ridge(loss, constraint=[], C=1.0, U=np.empty(shape=(0, 0)), V=np.empty(shape=(0, 0)), Tau=np.empty(shape=(0, 0)), S=np.empty(shape=(0, 0)), T=np.empty(shape=(0, 0)), A=np.empty(shape=(0, 0)), b=np.empty(shape=0), max_iter=1000, tol=0.0001, shrink=1, warm_start=0, verbose=0, trace_freq=100) - - Bases: :py:obj:`rehline._base._BaseReHLine`, :py:obj:`sklearn.base.BaseEstimator` - - Empirical Risk Minimization (ERM) with a piecewise linear-quadratic (PLQ) objective with a ridge penalty. - - .. math:: - - \min_{\mathbf{\beta} \in \mathbb{R}^d} \sum_{i=1}^n \text{PLQ}(y_i, \mathbf{x}_i^T \mathbf{\beta}) + \frac{1}{2} \| \mathbf{\beta} \|_2^2, \ \text{ s.t. } \ - \mathbf{A} \mathbf{\beta} + \mathbf{b} \geq \mathbf{0}, - - The function supports various loss functions, including: - - 'hinge', 'svm' or 'SVM' - - 'check' or 'quantile' or 'quantile regression' or 'QR' - - 'sSVM' or 'smooth SVM' or 'smooth hinge' - - 'TV' - - 'huber' or 'Huber' - - 'SVR' or 'svr' - - The following constraint types are supported: - * 'nonnegative' or '>=0': A non-negativity constraint. - * 'fair' or 'fairness': A fairness constraint. - * 'custom': A custom constraint, where the user must provide the constraint matrix 'A' and vector 'b'. - - Parameters - ---------- - loss : dict - A dictionary specifying the loss function parameters. - - constraint : list of dict - A list of dictionaries, where each dictionary represents a constraint. - Each dictionary must contain a 'name' key, which specifies the type of constraint. - - C : float, default=1.0 - Regularization parameter. The strength of the regularization is - inversely proportional to C. Must be strictly positive. - `C` will be absorbed by the ReHLine parameters when `self.make_ReLHLoss` is conducted. - - verbose : int, default=0 - Enable verbose output. Note that this setting takes advantage of a - per-process runtime setting in liblinear that, if enabled, may not work - properly in a multithreaded context. - - max_iter : int, default=1000 - The maximum number of iterations to be run. - - _U, _V: array of shape (L, n_samples), default=np.empty(shape=(0, 0)) - The parameters pertaining to the ReLU part in the loss function. - - _Tau, _S, _T: array of shape (H, n_samples), default=np.empty(shape=(0, 0)) - The parameters pertaining to the ReHU part in the loss function. - - _A: array of shape (K, n_features), default=np.empty(shape=(0, 0)) - The coefficient matrix in the linear constraint. - - _b: array of shape (K, ), default=np.empty(shape=0) - The intercept vector in the linear constraint. - - Attributes - ---------- - coef\_ : array-like - The optimized model coefficients. - - n_iter\_ : int - The number of iterations performed by the ReHLine solver. - - opt_result\_ : object - The optimization result object. - - dual_obj\_ : array-like - The dual objective function values. - - primal_obj\_ : array-like - The primal objective function values. - - Methods - ------- - fit(X, y, sample_weight=None) - Fit the model based on the given training data. - - decision_function(X) - The decision function evaluated on the given dataset. - - Notes - ----- - The `plqERM_Ridge` class is a subclass of `_BaseReHLine` and `BaseEstimator`, which suggests that it is part of a larger framework for implementing ReHLine algorithms. - - - - Overview - ======== - - - .. list-table:: Methods - :header-rows: 0 - :widths: auto - :class: summarytable - - * - :py:obj:`fit `\ (X, y, sample_weight) - - Fit the model based on the given training data. - * - :py:obj:`decision_function `\ (X) - - The decision function evaluated on the given dataset - - - Members - ======= - - .. py:method:: fit(X, y, sample_weight=None) - - Fit the model based on the given training data. - - Parameters - ---------- - - X: {array-like} of shape (n_samples, n_features) - Training vector, where `n_samples` is the number of samples and - `n_features` is the number of features. - - y : array-like of shape (n_samples,) - The target variable. - - sample_weight : array-like of shape (n_samples,), default=None - Array of weights that are assigned to individual - samples. If not provided, then each sample is given unit weight. - - Returns - ------- - self : object - An instance of the estimator. - - - - - .. py:method:: decision_function(X) - - The decision function evaluated on the given dataset - - Parameters - ---------- - X : array-like of shape (n_samples, n_features) - The data matrix. - - Returns - ------- - ndarray of shape (n_samples, ) - Returns the decision function of the samples. - - - - -.. py:class:: plqERM_ElasticNet(loss, constraint=[], C=1.0, l1_ratio=0.5, U=np.empty(shape=(0, 0)), V=np.empty(shape=(0, 0)), Tau=np.empty(shape=(0, 0)), S=np.empty(shape=(0, 0)), T=np.empty(shape=(0, 0)), A=np.empty(shape=(0, 0)), b=np.empty(shape=0), max_iter=1000, tol=0.0001, shrink=1, warm_start=0, verbose=0, trace_freq=100) - - Bases: :py:obj:`rehline._base._BaseReHLine`, :py:obj:`sklearn.base.BaseEstimator` - - Empirical Risk Minimization (ERM) with a piecewise linear-quadratic (PLQ) objective with a elastic net penalty. - - .. math:: - - \min_{\mathbf{\beta} \in \mathbb{R}^d} C \sum_{i=1}^n \text{PLQ}(y_i, \mathbf{x}_i^T \mathbf{\beta}) + \text{l1_ratio} \| \mathbf{\beta} \|_1 + \frac{1}{2} (1 - \text{l1_ratio}) \| \mathbf{\beta} \|_2^2, \ \text{ s.t. } \ - \mathbf{A} \mathbf{\beta} + \mathbf{b} \geq \mathbf{0}, - - The function supports various loss functions, including: - - 'hinge', 'svm' or 'SVM' - - 'check' or 'quantile' or 'quantile regression' or 'QR' - - 'sSVM' or 'smooth SVM' or 'smooth hinge' - - 'TV' - - 'huber' or 'Huber' - - 'SVR' or 'svr' - - The following constraint types are supported: - * 'nonnegative' or '>=0': A non-negativity constraint. - * 'fair' or 'fairness': A fairness constraint. - * 'custom': A custom constraint, where the user must provide the constraint matrix 'A' and vector 'b'. - - Parameters - ---------- - loss : dict - A dictionary specifying the loss function parameters. - - constraint : list of dict - A list of dictionaries, where each dictionary represents a constraint. - Each dictionary must contain a 'name' key, which specifies the type of constraint. - - C : float, default=1.0 - Regularization parameter. The strength of the regularization is - inversely proportional to C. Must be strictly positive. - `C` will be absorbed by the ReHLine parameters when `self.make_ReLHLoss` is conducted. - - l1_ratio : float, default=0.5 - The ElasticNet mixing parameter, with 0 <= l1_ratio < 1. For l1_ratio = 0 the penalty - is an L2 penalty. For 0 < l1_ratio < 1, the penalty is a combination of L1 and L2. - - verbose : int, default=0 - Enable verbose output. Note that this setting takes advantage of a - per-process runtime setting in liblinear that, if enabled, may not work - properly in a multithreaded context. - - max_iter : int, default=1000 - The maximum number of iterations to be run. - - _U, _V: array of shape (L, n_samples), default=np.empty(shape=(0, 0)) - The parameters pertaining to the ReLU part in the loss function. - - _Tau, _S, _T: array of shape (H, n_samples), default=np.empty(shape=(0, 0)) - The parameters pertaining to the ReHU part in the loss function. - - _A: array of shape (K, n_features), default=np.empty(shape=(0, 0)) - The coefficient matrix in the linear constraint. - - _b: array of shape (K, ), default=np.empty(shape=0) - The intercept vector in the linear constraint. - - Attributes - ---------- - coef\_ : array-like - The optimized model coefficients. - - n_iter\_ : int - The number of iterations performed by the ReHLine solver. - - opt_result\_ : object - The optimization result object. - - dual_obj\_ : array-like - The dual objective function values. - - primal_obj\_ : array-like - The primal objective function values. - - Methods - ------- - fit(X, y, sample_weight=None) - Fit the model based on the given training data. - - decision_function(X) - The decision function evaluated on the given dataset. - - Notes - ----- - The `plqERM_ElasticNet` class is a subclass of `_BaseReHLine` and `BaseEstimator`, which suggests that it is part of a larger framework for implementing ReHLine algorithms. - - - - Overview - ======== - - - .. list-table:: Methods - :header-rows: 0 - :widths: auto - :class: summarytable - - * - :py:obj:`fit `\ (X, y, sample_weight) - - Fit the model based on the given training data. - * - :py:obj:`decision_function `\ (X) - - The decision function evaluated on the given dataset - - - Members - ======= - - .. py:method:: fit(X, y, sample_weight=None) - - Fit the model based on the given training data. - - Parameters - ---------- - - X: {array-like} of shape (n_samples, n_features) - Training vector, where `n_samples` is the number of samples and - `n_features` is the number of features. - - y : array-like of shape (n_samples,) - The target variable. - - sample_weight : array-like of shape (n_samples,), default=None - Array of weights that are assigned to individual - samples. If not provided, then each sample is given unit weight. - - Returns - ------- - self : object - An instance of the estimator. - - - - - .. py:method:: decision_function(X) - - The decision function evaluated on the given dataset - - Parameters - ---------- - X : array-like of shape (n_samples, n_features) - The data matrix. - - Returns - ------- - ndarray of shape (n_samples, ) - Returns the decision function of the samples. - - - - -.. py:class:: plq_Ridge_Classifier(loss, constraint=[], C=1.0, U=np.empty((0, 0)), V=np.empty((0, 0)), Tau=np.empty((0, 0)), S=np.empty((0, 0)), T=np.empty((0, 0)), A=np.empty((0, 0)), b=np.empty((0, )), max_iter=1000, tol=0.0001, shrink=1, warm_start=0, verbose=0, trace_freq=100, fit_intercept=True, intercept_scaling=1.0, class_weight=None, multi_class=[], n_jobs=None) - - Bases: :py:obj:`rehline._class.plqERM_Ridge`, :py:obj:`sklearn.base.ClassifierMixin` - - Empirical Risk Minimization (ERM) Classifier with a Piecewise Linear-Quadratic (PLQ) loss - and ridge penalty, compatible with the scikit-learn API. - - This wrapper makes ``plqERM_Ridge`` behave as a classifier: - - Accepts arbitrary binary labels in the original label space. - - Computes class weights on original labels (if ``class_weight`` is set). - - Encodes labels with ``LabelEncoder`` into {0,1}, then maps to {-1,+1} for training. - - Supports optional intercept fitting (via an augmented constant feature). - - Provides standard methods ``fit``, ``predict``, and ``decision_function``. - - Integrates with scikit-learn ecosystem (e.g., GridSearchCV, Pipeline). - - Supports multiclass classification via OvR or OvO method. - - Parameters - ---------- - loss : dict - Dictionary specifying the loss function parameters. Examples include: - - {'name': 'svm'} - - {'name': 'sSVM'} - - {'name': 'huber'} - and other PLQ losses supported by ``plqERM_Ridge``. - - constraint : list of dict, default=[] - Optional constraints. Each dictionary must include a ``'name'`` key. - Examples: {'name': 'nonnegative'}, {'name': 'fair'}, {'name': 'custom'}. - - C : float, default=1.0 - Inverse regularization strength. - - _U, _V, _Tau, _S, _T : ndarray, default empty - Parameters for the PLQ representation of the loss function. - Typically built internally by helper functions. - - _A : ndarray of shape (K, n_features), default empty - Linear-constraint coefficient matrix. - - _b : ndarray of shape (K,), default empty - Linear-constraint intercept vector. - - max_iter : int, default=1000 - Maximum number of iterations for the ReHLine solver. - - tol : float, default=1e-4 - Convergence tolerance. - - shrink : int, default=1 - Shrinkage parameter for the solver. - - warm_start : int, default=0 - Whether to reuse the previous solution for initialization. - - verbose : int, default=0 - Verbosity level for the solver. - - trace_freq : int, default=100 - Frequency (in iterations) at which solver progress is traced - when ``verbose > 0``. - - fit_intercept : bool, default=True - Whether to fit an intercept term. If True, a constant feature column is added - to ``X`` during training. The last learned coefficient is extracted as - ``intercept_``. - - intercept_scaling : float, default=1.0 - Value used for the constant feature column when ``fit_intercept=True``. - Matches the convention used in scikit-learn's ``LinearSVC``. - - class_weight : dict, 'balanced', or None, default=None - Class weights applied like in LinearSVC: - - 'balanced' uses n_samples / (n_classes * n_j). - - dict maps label -> weight in the ORIGINAL label space. - - multi_class : str or list, default=[] - Method for multiclass classification. Options: - - 'ovo': One-vs-One, trains K*(K-1)/2 binary classifiers. - - 'ovr': One-vs-Rest, trains K binary classifiers. - - [ ] or ignored when only 2 classes are present. - - n_jobs : int or None, default=None - Number of parallel jobs for multiclass fitting. - None means 1 (serial). -1 means use all available CPUs. - Passed directly to joblib.Parallel. - - - Attributes - ---------- - ``coef_ ``: ndarray of shape (n_features,) for binary, (n_estimators, n_features) for multiclass - Coefficients of all fitted classifiers, excluding the intercept. - - ``intercept_ ``: float for binary, ndarray of shape (n_estimators,) for multiclass - Intercept term(s). 0.0 if ``fit_intercept=False``. - - classes_ : ndarray of shape (n_classes,) - Unique class labels in the original label space. - - estimators_ : list, only present for multiclass - For OvR: list of (coef, intercept) tuples, length K. - For OvO: list of (coef, intercept, cls_i, cls_j) tuples, length K*(K-1)/2. - - _label_encoder : LabelEncoder - Encodes original labels into {0,1} for internal training. - - - Overview - ======== - - - .. list-table:: Methods - :header-rows: 0 - :widths: auto - :class: summarytable - - * - :py:obj:`fit `\ (X, y, sample_weight) - - Fit the classifier to training data. - * - :py:obj:`decision_function `\ (X) - - Compute the decision function for samples in X. - * - :py:obj:`predict `\ (X) - - Predict class labels for samples in X. - - - Members - ======= - - .. py:method:: fit(X, y, sample_weight=None) - - Fit the classifier to training data. - - Parameters - ---------- - X : array-like of shape (n_samples, n_features) - Training features. - - y : array-like of shape (n_samples,) - Target labels. - - sample_weight : array-like of shape (n_samples,), default=None - Per-sample weights. If None, uniform weights are used. - - Returns - ------- - self : object - Fitted estimator. - - - .. py:method:: decision_function(X) - - Compute the decision function for samples in X. - - For binary classification, returns a 1D array of scores. - For OvR multiclass, returns a 2D array of shape (n_samples, K). - For OvO multiclass, returns a 2D array of shape (n_samples, K*(K-1)/2). - - Using coef_.T works uniformly for both binary (n_features,) and - multiclass (n_estimators, n_features) shapes. - - Parameters - ---------- - X : array-like of shape (n_samples, n_features) - Input samples. - - Returns - ------- - ndarray of shape (n_samples,) or (n_samples, n_estimators) - Continuous scores for each sample. - - - .. py:method:: predict(X) - - Predict class labels for samples in X. - For binary classification, thresholds the decision score at 0. - For OvR, takes the argmax across K classifiers. - For OvO, uses majority voting across K*(K-1)/2 classifiers. - - Parameters - ---------- - X : array-like of shape (n_samples, n_features) - Input samples. - - Returns - ------- - y_pred : ndarray of shape (n_samples,) - Predicted class labels in the original label space. - - - - -.. py:class:: plq_Ridge_Regressor(loss={'name': 'QR', 'qt': 0.5}, constraint=[], C=1.0, U=np.empty((0, 0)), V=np.empty((0, 0)), Tau=np.empty((0, 0)), S=np.empty((0, 0)), T=np.empty((0, 0)), A=np.empty((0, 0)), b=np.empty((0, )), max_iter=1000, tol=0.0001, shrink=1, warm_start=0, verbose=0, trace_freq=100, fit_intercept=True, intercept_scaling=1.0) - - Bases: :py:obj:`rehline._class.plqERM_Ridge`, :py:obj:`sklearn.base.RegressorMixin` - - Empirical Risk Minimization (ERM) regressor with a Piecewise Linear-Quadratic (PLQ) loss - and a ridge penalty, implemented as a scikit-learn compatible estimator. - - This wrapper adds standard sklearn conveniences while delegating loss/constraint construction - to :class:`plqERM_Ridge` (via `_make_loss_rehline_param` / `_make_constraint_rehline_param`). - - Notes - ----- - - **Intercept handling**: if ``fit_intercept=True``, a constant column (value = ``intercept_scaling``) - is appended to the right of the design matrix before calling the base solver. The last learned - coefficient is then split out as ``intercept_``. - → The column indices of the original features reamin; therefore, ``sen_idx`` in the constraint ``fair`` follow the original index. - - **Constraint handling**: constraints are passed through unchanged; the base class will call - ``_make_constraint_rehline_param(constraint, X, y)`` on the matrix given to `fit`. - With your updated implementation, ``fair`` must be specified as - ``{'name': 'fair', 'sen_idx': list[int], 'tol_sen': list[float]}``. - - Parameters - ---------- - loss : dict, default={'name': 'QR', 'qt': 0.5} - PLQ loss configuration (e.g., median Quantile Regression). Examples: - ``{'name': 'QR', 'qt': 0.5}``, ``{'name': 'huber', 'tau': 1.0}``, - ``{'name': 'SVR', 'epsilon': 0.1}``. - Required keys depend on the chosen loss and are consumed by the underlying solver. - constraint : list of dict, default=[] - Constraint specifications. Supported by your updated `_make_constraint_rehline_param`: - - ``{'name': 'nonnegative'}`` or ``{'name': '>=0'}`` - - ``{'name': 'fair', 'sen_idx': list[int], 'tol_sen': list[float]}`` - - ``{'name': 'custom', 'A': ndarray[K, d], 'b': ndarray[K]}`` - - Note: when ``fit_intercept=True``, a constant column is appended **as the last column**; - since you index sensitive columns by ``sen_idx`` on the *original* features, indices stay valid. - C : float, default=1.0 - Regularization parameter (absorbed by ReHLine parameters inside the solver). - _U, _V, _Tau, _S, _T : ndarray, default empty - Advanced PLQ parameters for the underlying ReHLine formulation (usually left as defaults). - _A, _b : ndarray, default empty - Optional linear constraint matrices (used only if ``constraint`` contains ``{'name': 'custom'}``). - (Your `_make_constraint_rehline_param` is responsible for validating their shapes.) - max_iter : int, default=1000 - Maximum iterations for the ReHLine solver. - tol : float, default=1e-4 - Convergence tolerance for the ReHLine solver. - shrink : int, default=1 - Shrink parameter passed to the solver (see solver docs). - warm_start : int, default=0 - Warm start flag passed to the solver (see solver docs). - verbose : int, default=0 - Verbosity for the solver (0: silent). - trace_freq : int, default=100 - Iteration frequency to trace solver internals (if ``verbose`` is enabled). - fit_intercept : bool, default=True - If ``True``, append a constant column (value = ``intercept_scaling``) to the design matrix - before calling the solver. The learned last coefficient is then split as ``intercept_``. - intercept_scaling : float, default=1.0 - Scaling applied to the appended constant column when ``fit_intercept=True``. - - Attributes - ---------- - ``coef_`` : ndarray of shape (n_features,) - Learned linear coefficients (excluding the intercept term). - ``intercept_`` : float - Intercept term extracted from the last coefficient when ``fit_intercept=True``, otherwise 0.0. - ``n_features_in_`` : int - Number of input features seen during :meth:`fit` (before intercept augmentation). - - Notes - ----- - This estimator **does not support sparse input**. If you need sparse support, convert inputs to dense - or wrap this estimator in a scikit-learn :class:`~sklearn.pipeline.Pipeline` with a transformer that - densifies data (at the cost of memory). - - - Overview - ======== - - - .. list-table:: Methods - :header-rows: 0 - :widths: auto - :class: summarytable - - * - :py:obj:`fit `\ (X, y, sample_weight) - - If ``fit_intercept=True``, a constant column (value = ``intercept_scaling``) is appended - * - :py:obj:`decision_function `\ (X) - - Compute f(X) = X @ ``coef_`` + ``intercept_``. - * - :py:obj:`predict `\ (X) - - Predict targets as the linear decision function. - - - Members - ======= - - .. py:method:: fit(X, y, sample_weight=None) - - If ``fit_intercept=True``, a constant column (value = ``intercept_scaling``) is appended - to the **right** of ``X`` before calling the base solver. The base class - (:class:`plqERM_Ridge`) will construct the loss and constraints via its internal helpers - on the matrix passed here. After solving, the last coefficient is split as - ``intercept_`` and removed from ``coef_``. - - Parameters - ---------- - X : ndarray of shape (n_samples, n_features) - Training design matrix (dense). Sparse inputs are not supported. - y : ndarray of shape (n_samples,) - Target values. - sample_weight : ndarray of shape (n_samples,), default=None - Optional per-sample weights; forwarded to the underlying solver. - - Returns - ------- - self : object - Fitted estimator. - - - - .. py:method:: decision_function(X) - - Compute f(X) = X @ ``coef_`` + ``intercept_``. - - Parameters - ---------- - X : ndarray of shape (n_samples, n_features) - Input data (dense). Must have the same number of features as seen in :meth:`fit`. - - Returns - ------- - scores : ndarray of shape (n_samples,) - Predicted real-valued scores.、 - - - .. py:method:: predict(X) - - Predict targets as the linear decision function. - - Parameters - ---------- - X : ndarray of shape (n_samples, n_features) - Input data (dense). - - Returns - ------- - y_pred : ndarray of shape (n_samples,) - Predicted target values (real-valued). - - - - -.. py:class:: plqMF_Ridge(n_users, n_items, loss, biased=True, constraint_user=[], constraint_item=[], rank=10, C=1.0, rho=0.5, init_mean=0.0, init_sd=0.1, random_state=None, max_iter=10000, tol=0.0001, shrink=1, trace_freq=100, max_iter_CD=10, tol_CD=0.0001, verbose=0) - - Bases: :py:obj:`rehline._base._BaseReHLine`, :py:obj:`sklearn.base.BaseEstimator` - - Matrix Factorization (MF) with a piecewise linear-quadratic objective and ridge penalty. - - .. math:: - \min_{\substack{ - \mathbf{P} \in \mathbb{R}^{n \times k}\ - \pmb{\alpha} \in \mathbb{R}^n \\ - \mathbf{Q} \in \mathbb{R}^{m \times k}\ - \pmb{\beta} \in \mathbb{R}^m - }} - \left[ - \sum_{(u,i)\in \Omega} C \cdot \text{PLQ}(r_{ui}, \ \mathbf{p}_u^T \mathbf{q}_i + \alpha_u + \beta_i) - \right] - + - \left[ - \frac{\rho}{n}\sum_{u=1}^n(\|\mathbf{p}_u\|_2^2 + \alpha_u^2) - + \frac{1-\rho}{m}\sum_{i=1}^m(\|\mathbf{q}_i\|_2^2 + \beta_i^2) - \right] - - .. math:: - \ \text{ s.t. } \ - \mathbf{A}_{\text{user}} \begin{pmatrix} \alpha_u \\ \mathbf{p}_u \end{pmatrix} + \mathbf{b}_{\text{user}} \geq \mathbf{0},\ u = 1,\dots,n - \quad \text{and} \quad - \mathbf{A}_{\text{item}} \begin{pmatrix} \beta_i \\ \mathbf{q}_i \end{pmatrix} + \mathbf{b}_{\text{item}} \geq \mathbf{0},\ i = 1,\dots,m - - The function supports various loss functions, including: - - 'hinge', 'svm' or 'SVM' - - 'MAE' or 'mae' or 'mean absolute error' - - 'squared hinge' or 'squared svm' or 'squared SVM' - - 'MSE' or 'mse' or 'mean squared error' - - The following constraint types are supported: - * 'nonnegative' or '>=0': A non-negativity constraint. - * 'fair' or 'fairness': A fairness constraint. - * 'custom': A custom constraint, where the user must provide the constraint matrix 'A' and vector 'b'. - - Parameters - ---------- - n_users : int - Number of unique users in the dataset (or number of rows in target sparse matrix). - - n_items : int - Number of unique items in the dataset (or number of columns in target sparse matrix). - - loss : dict - A dictionary specifying the loss function parameters. - - constraint_user : list of dict - A list of dictionaries, where each dictionary represents a constraint to user side parameters. - Each dictionary must contain a 'name' key, which specifies the type of constraint. - - constraint_item : list of dict - A list of dictionaries, where each dictionary represents a constraint to item side parameters. - Each dictionary must contain a 'name' key, which specifies the type of constraint. - - biased : bool, default=True - Whether to include user and item bias terms in the model. - - rank : int, default=10 - Dimensionality of the latent factor vectors (number of factors). - - C : float, default=1.0 - Regularization parameter. The strength of the regularization is - inversely proportional to `C`. Must be strictly positive. - `C` will be absorbed by the ReHLine parameters when `_cast_sample_weight()` is conducted. - - rho : float, default=0.5 - Regularization strength ratio between user and item factors. Must be within the range of (0,1). - - init_mean : float, default=0.0 - Mean of the Gaussian distribution for initializing latent factors. - - init_sd : float, default=0.1 - Standard deviation of the Gaussian distribution for initializing latent factors. - - random_state : int or RandomState, default=None - Random seed for reproducible initialization of latent factors. - - max_iter : int, default=10000 - The maximum number of iterations to be run for the ReHLine solver. - - tol : float, default=1e-4 - The tolerance for the stopping criterion for the ReHLine solver. - - shrink : float, default=1 - The shrinkage of dual variables for the ReHLine solver. - - trace_freq : int, default=100 - The frequency at which to print the optimization trace for the ReHLine solver. - - max_iter_CD : int, default=10 - Maximum number of iterations for coordinate descent steps. - - tol_CD : float, default=1e-4 - The tolerance for the stopping criterion for coordinate descent steps. - - verbose : int, default=0 - Verbosity level. - 0: No output - 1: CD iteration progress information - 2: ReHLine solver optimization information - 3: All information of CD and ReHLine - - Attributes - ---------- - n_users : int - Number of unique users in the dataset (or number of rows in target sparse matrix). - - n_items : int - Number of unique items in the dataset (or number of columns in target sparse matrix). - - n_ratings : int - Number of ratings in the training data. Available after fitting. - - P : ndarray of shape (n_users, rank) - User latent factor matrix. Learned during fitting. - - Q : ndarray of shape (n_items, rank) - Item latent factor matrix. Learned during fitting. - - bu : ndarray of shape (n_users,) or None - User bias terms. Learned during fitting. Only available if `biased=True`. - - bi : ndarray of shape (n_items,) or None - Item bias terms. Learned during fitting. Only available if `biased=True`. - - Iu : list of ndarray - List where each element contains indices of items rated by the corresponding user. - Available after fitting. - - Ui : list of ndarray - List where each element contains indices of users who rated the corresponding item. - Available after fitting. - - history : ndarray of shape (max_iter_CD + 1, 2) - Optimization history containing loss and objective values at each coordinate descent iteration. - First column: cumulative loss term values. Second column: objective function values (with penalty). - - sample_weight : ndarray of shape (n_ratings,) - Sample weights used during fitting. Available after fitting. - - Methods - ------- - fit(X, y, sample_weight=None) - Fit the model based on the given training data. - - decision_function(X) - The decision function evaluated on the given dataset. - - obj(X, y) - Compute the values of loss term and objective function. - - Notes - ----- - The `plqMF_Ridge` class is a subclass of `_BaseReHLine` and `BaseEstimator`, which suggests that it is part of a larger framework for implementing ReHLine algorithms. - - - - Overview - ======== - - - .. list-table:: Methods - :header-rows: 0 - :widths: auto - :class: summarytable - - * - :py:obj:`fit `\ (X, y, sample_weight) - - Fit the model based on the given training data. - * - :py:obj:`decision_function `\ (X) - - The decision function evaluated on the given dataset - * - :py:obj:`obj `\ (X, y) - - Compute the values of loss term and objective function. - - - Members - ======= - - .. py:method:: fit(X, y, sample_weight=None) - - Fit the model based on the given training data. - - Parameters - ---------- - X : array-like of shape (n_ratings, 2) - Input data where first column contains user ID and - second column contains item ID. - - y : array-like of shape (n_ratings,) - Target rating values. - - sample_weight : array-like of shape (n_samples,), default=None - Array of weights that are assigned to individual samples. - If not provided, then each sample is given unit weight. - - Returns - ------- - self : object - An instance of the estimator. - - - - .. py:method:: decision_function(X) - - The decision function evaluated on the given dataset - - Parameters - ---------- - X : array-like of shape (n_samples, 2) - Training data where first column contains user ID and - second column contains item ID. - - Returns - ------- - prediction : ndarray of shape (n_samples,) - Predicted ratings for the input pairs. - - - .. py:method:: obj(X, y) - - Compute the values of loss term and objective function. - - Parameters - ---------- - X : array-like of shape (n_ratings, 2) - User-item rating pairs. - - y : array-like of shape (n_ratings,) - Actual rating values. - - Returns - ------- - loss_term : float - The data fitting term (sum of loss values). - - objective_value : float - The total objective value including regularization. - - - - - -Functions ---------- -.. py:function:: ReHLine_solver(X, U, V, Tau=np.empty(shape=(0, 0)), S=np.empty(shape=(0, 0)), T=np.empty(shape=(0, 0)), A=np.empty(shape=(0, 0)), b=np.empty(shape=0), rho=0.0, Lambda=np.empty(shape=(0, 0)), Gamma=np.empty(shape=(0, 0)), xi=np.empty(shape=(0, 0)), mu=np.empty(shape=(0, 0)), max_iter=1000, tol=0.0001, shrink=1, verbose=1, trace_freq=100) - -.. py:function:: plqERM_Ridge_path_sol(X, y, *, loss, constraint=[], eps=0.001, n_Cs=100, Cs=None, max_iter=5000, tol=0.0001, verbose=0, shrink=1, warm_start=False, return_time=True) - - Compute the PLQ Empirical Risk Minimization (ERM) path over a range of regularization parameters. - This function evaluates the model's performance for different values of the regularization parameter - and provides structured benchmarking output. - - Parameters - ---------- - X : ndarray of shape (n_samples, n_features) - Training input samples. - - y : ndarray of shape (n_samples,) - Target values corresponding to each input sample. - - loss : dict - Dictionary describing the PLQ loss function parameters. Used to construct the loss object internally. - - constraint : list of dict, optional (default=[]) - List of constraints applied to the optimization problem. Each constraint should be represented - as a dictionary compatible with the solver. - - - eps : float, default=1e-3 - Defines the length of the regularization path when `Cs` is not provided. - The values of `C` will range from `10^log10(eps)` to `10^-log10(eps)`. - - n_Cs : int, default=100 - Number of regularization values to evaluate if `Cs` is not provided. - - Cs : array-like of shape (n_Cs,), optional - Explicit values of regularization strength `C` to use. If `None`, the values are generated - logarithmically between 1e-2 and 1e3. - - max_iter : int, default=5000 - Maximum number of iterations allowed for the optimization solver at each `C`. - - tol : float, default=1e-4 - Tolerance for solver convergence. - - verbose : int, default=0 - Controls verbosity level of output. Set to higher values (e.g., 1 or 2) for detailed progress logs. - When verbose = 1, only print path results table; - when verbose = 2, print path results table and path solution plot. - - shrink : float, default=1 - Shrinkage factor for the solver, potentially influencing convergence behavior. - - warm_start : bool, default=False - If True, reuse the previous solution to warm-start the next solver step, speeding up convergence. - - return_time : bool, default=True - If True, return timing information for each value of `C`. - - Returns - ------- - Cs : ndarray of shape (n_Cs,) - Array of regularization parameters used in the path. - - times : list of float - Time in seconds taken to fit the model at each `C`. Returned only if `return_time=True`. - - n_iters : list of int - Number of iterations used by the solver at each regularization value. - - obj_values : list of float - Final objective values (including loss and regularization terms) at each `C`. - - L2_norms : list of float - L2 norm of the coefficients (excluding bias) at each `C`. - - coefs : ndarray of shape (n_features, n_Cs) - Learned model coefficients at each regularization strength. - - Example - ------- - - >>> # generate data - >>> np.random.seed(42) - >>> n, d, C = 1000, 5, 0.5 - >>> X = np.random.randn(n, d) - >>> beta0 = np.random.randn(d) - >>> y = np.sign(X.dot(beta0) + np.random.randn(n)) - >>> # define loss function - >>> loss = {'name': 'svm'} - >>> Cs = np.logspace(-1,3,15) - >>> constraint = [{'name': 'nonnegative'}] - - - >>> # calculate - >>> Cs, times, n_iters, losses, norms, coefs = plqERM_Ridge_path_sol( - ... X, y, loss=loss, Cs=Cs, max_iter=100000,tol=1e-4,verbose=2, - ... warm_start=False, constraint=constraint, return_time=True - ... ) - - - -.. py:function:: make_mf_dataset(n_users, n_items, n_factors=20, n_interactions=None, density=0.01, noise_std=0.1, seed=None, rating_min=1.0, rating_max=5.0, return_params=True) - - Generate synthetic rating data using matrix factorization model. - - Creates synthetic user-item rating data based on the matrix factorization - approach commonly used in recommender systems. The ratings are generated - as: rating = mu + user_bias + item_bias + user_factor * item_factor + noise - - Parameters - ---------- - n_users : int - Number of users in the synthetic dataset - - n_items : int - Number of items in the synthetic dataset - - n_factors : int, default=20 - Number of latent factors for user and item embeddings - - n_interactions : int, optional - Exact number of user-item interactions. If None, calculated as density * total_pairs - - density : float, default=0.01 - Density of the rating matrix (ignored if n_interactions is specified) - - noise_std : float, default=0.1 - Standard deviation of Gaussian noise added to ratings - - seed : int, optional - Random seed for reproducible results - - rating_min : float, default=1.0 - Minimum possible rating value - - rating_max : float, default=5.0 - Maximum possible rating value - - return_params : bool, default=True - If True, returns the underlying model parameters (P, Q, bu, bi, mu) - - Returns - ------- - dict - Dictionary containing: - - - **X** : ndarray of shape (n_interactions, 2) - User-item pairs where X[:, 0] are user indices and X[:, 1] are item indices - - **y** : ndarray of shape (n_interactions,) - Synthetic ratings for each user-item pair - - **params** : dict, optional - Only returned if return_params=True. Contains: - - * **P** : ndarray of shape (n_users, n_factors) - User factor matrix - * **Q** : ndarray of shape (n_items, n_factors) - Item factor matrix - * **bu** : ndarray of shape (n_users,) - User biases - * **bi** : ndarray of shape (n_items,) - Item biases - * **mu** : float - Global mean rating - - Notes - ----- - The rating generation follows the standard matrix factorization model: - - r_ui = μ + b_u + b_i + p_u · q_i^T + ε - - where ε ~ N(0, noise_std²) - - The generated ratings are clipped to stay within [rating_min, rating_max] range. - - - - -