mirror of https://github.com/kubeflow/examples.git
873 lines
58 KiB
Plaintext
873 lines
58 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"# 🪙 American Express - Default Prediction Competition Original Notebook\n",
|
|
"\n",
|
|
"\n",
|
|
"---\n",
|
|
"\n",
|
|
"In this [Kaggle competition](https://www.kaggle.com/competitions/g-research-crypto-forecasting/overview), you'll use your machine learning expertise to predict credit default. This competition is hosted by American Express. \n",
|
|
"\n",
|
|
"> American Express is a globally integrated payments company. The largest payment card issuer in the world, they provide customers with access to products, insights, and experiences that enrich lives and build business success.\n",
|
|
"\n",
|
|
"The dataset provided is an industrial scale data set of about 5.5 million rows. It has been pre-processed and converted to a lightweight version by raddar for ease of training and better result. This dataset is available in a [parquet format][1].\n",
|
|
"\n",
|
|
"[1]: https://www.kaggle.com/datasets/raddar/amex-data-integer-dtypes-parquet-format"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"## Install necessary packages\n",
|
|
"\n",
|
|
"We can install the necessary package by either running pip install --user <package_name> or include everything in a requirements.txt file and run pip install --user -r requirements.txt. We have put the dependencies in a requirements.txt file so we will use the former method.\n",
|
|
"\n",
|
|
"NOTE: After installing python packages, restart notebook kernel before proceeding."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 1,
|
|
"metadata": {
|
|
"tags": [
|
|
"skip"
|
|
]
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"!pip install -r requirements.txt --user --quiet"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"## Imports\n",
|
|
"\n",
|
|
"In this section we import the packages we need for this example. Make it a habit to gather your imports in a single place. It will make your life easier if you are going to transform this notebook into a Kubeflow pipeline using Kale."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"metadata": {
|
|
"tags": [
|
|
"imports"
|
|
]
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"import numpy as np\n",
|
|
"import pandas as pd\n",
|
|
"import os, subprocess\n",
|
|
"import random, zipfile, joblib\n",
|
|
"import scipy.stats\n",
|
|
"import warnings\n",
|
|
"import gc, wget\n",
|
|
"\n",
|
|
"from sklearn.model_selection import StratifiedKFold\n",
|
|
"from lightgbm import LGBMClassifier, log_evaluation\n",
|
|
"\n",
|
|
"warnings.filterwarnings(\"ignore\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"## Project hyper-parameters\n",
|
|
"\n",
|
|
"In this cell, we define the different hyper-parameters. Defining them in one place makes it easier to experiment with their values and also facilitates the execution of HP Tuning experiments using Kale and Katib."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 3,
|
|
"metadata": {
|
|
"tags": [
|
|
"pipeline-parameters"
|
|
]
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Hyper-parameters\n",
|
|
"N_EST = 30\n",
|
|
"LR = 0.1"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"Set random seed for reproducibility"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 4,
|
|
"metadata": {
|
|
"tags": [
|
|
"skip"
|
|
]
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"def fix_all_seeds(seed):\n",
|
|
" np.random.seed(seed)\n",
|
|
" random.seed(seed)\n",
|
|
" os.environ['PYTHONHASHSEED'] = str(seed)\n",
|
|
"\n",
|
|
"fix_all_seeds(2022)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"## Download data\n",
|
|
"\n",
|
|
"In this section, we download the data from kaggle using the Kaggle API credentials"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 5,
|
|
"metadata": {
|
|
"tags": [
|
|
"block:download_data"
|
|
]
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"CompletedProcess(args=['kaggle', 'datasets', 'download', '-d', 'raddar/amex-data-integer-dtypes-parquet-format'], returncode=0)"
|
|
]
|
|
},
|
|
"execution_count": 5,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"# setup kaggle environment for data download\n",
|
|
"dataset = \"amex-data-integer-dtypes-parquet-format\"\n",
|
|
"\n",
|
|
"# setup kaggle environment for data download\n",
|
|
"with open('/secret/kaggle-secret/password', 'r') as file:\n",
|
|
" kaggle_key = file.read().rstrip()\n",
|
|
"with open('/secret/kaggle-secret/username', 'r') as file:\n",
|
|
" kaggle_user = file.read().rstrip()\n",
|
|
"\n",
|
|
"os.environ['KAGGLE_USERNAME'], os.environ['KAGGLE_KEY'] = kaggle_user, kaggle_key\n",
|
|
"\n",
|
|
"# download kaggle's Amex-credit-prediction data\n",
|
|
"subprocess.run([\"kaggle\",\"datasets\", \"download\", \"-d\", f'raddar/{dataset}'])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"metadata": {
|
|
"tags": [
|
|
"block:"
|
|
]
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"CompletedProcess(args=['rm', 'data/train_labels.zip'], returncode=0)"
|
|
]
|
|
},
|
|
"execution_count": 6,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"# path to download to\n",
|
|
"data_path = 'data'\n",
|
|
"\n",
|
|
"# extract Amex-credit-prediction.zip to data_path\n",
|
|
"with zipfile.ZipFile(f\"{dataset}.zip\",\"r\") as zip_ref:\n",
|
|
" zip_ref.extractall(data_path)\n",
|
|
" \n",
|
|
"# download kaggle's Amex-credit-prediction train_labels.zip\n",
|
|
"download_link = \"https://github.com/kubeflow/examples/blob/master/american-express-default-kaggle-competition/data/train_labels.zip?raw=true\"\n",
|
|
"wget.download(download_link, f'{data_path}/train_labels.zip')\n",
|
|
"\n",
|
|
"# extract Amex-credit-prediction.zip to data_path\n",
|
|
"with zipfile.ZipFile(f'{data_path}/train_labels.zip','r') as zip_ref:\n",
|
|
" zip_ref.extractall(data_path)\n",
|
|
" \n",
|
|
"# delete zipfiles\n",
|
|
"subprocess.run(['rm', f'{dataset}.zip'])\n",
|
|
"subprocess.run(['rm', f'{data_path}/train_labels.zip'])"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"## Load the dataset\n",
|
|
"\n",
|
|
"First, let us load and analyze the data.\n",
|
|
"\n",
|
|
"The data is in csv format, thus, we use the handy read_csv pandas method."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 7,
|
|
"metadata": {
|
|
"tags": [
|
|
"block:load_data",
|
|
"prev:download_data"
|
|
]
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"TRAIN_CSV = (f'{data_path}/train.parquet')\n",
|
|
"TEST_CSV = f'{data_path}/test.parquet'\n",
|
|
"TARGET_CSV = f'{data_path}/train_labels.csv'"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 8,
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"target shape: (458913,)\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"df_train = pd.read_parquet(TRAIN_CSV)\n",
|
|
"df_test = pd.read_parquet(TEST_CSV)\n",
|
|
"target = pd.read_csv(TARGET_CSV).target.values\n",
|
|
"print(f\"target shape: {target.shape}\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 9,
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"(5531451, 190)"
|
|
]
|
|
},
|
|
"execution_count": 9,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"df_train.shape"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 10,
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"customer_ID 0\n",
|
|
"S_2 0\n",
|
|
"P_2 45985\n",
|
|
"D_39 0\n",
|
|
"B_1 0\n",
|
|
" ... \n",
|
|
"D_141 101548\n",
|
|
"D_142 4587043\n",
|
|
"D_143 0\n",
|
|
"D_144 40727\n",
|
|
"D_145 0\n",
|
|
"Length: 190, dtype: int64"
|
|
]
|
|
},
|
|
"execution_count": 10,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"df_train.isna().sum()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"### Define Helper Functions"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 11,
|
|
"metadata": {
|
|
"tags": [
|
|
"functions"
|
|
]
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# @yunchonggan's fast metric implementation\n",
|
|
"# From https://www.kaggle.com/competitions/amex-default-prediction/discussion/328020\n",
|
|
"def amex_metric(y_true: np.array, y_pred: np.array) -> float:\n",
|
|
"\n",
|
|
" # count of positives and negatives\n",
|
|
" n_pos = y_true.sum()\n",
|
|
" n_neg = y_true.shape[0] - n_pos\n",
|
|
"\n",
|
|
" # sorting by descring prediction values\n",
|
|
" indices = np.argsort(y_pred)[::-1]\n",
|
|
" preds, target = y_pred[indices], y_true[indices]\n",
|
|
"\n",
|
|
" # filter the top 4% by cumulative row weights\n",
|
|
" weight = 20.0 - target * 19.0\n",
|
|
" cum_norm_weight = (weight / weight.sum()).cumsum()\n",
|
|
" four_pct_filter = cum_norm_weight <= 0.04\n",
|
|
"\n",
|
|
" # default rate captured at 4%\n",
|
|
" d = target[four_pct_filter].sum() / n_pos\n",
|
|
"\n",
|
|
" # weighted gini coefficient\n",
|
|
" lorentz = (target / n_pos).cumsum()\n",
|
|
" gini = ((lorentz - cum_norm_weight) * weight).sum()\n",
|
|
"\n",
|
|
" # max weighted gini coefficient\n",
|
|
" gini_max = 10 * n_neg * (1 - 19 / (n_pos + 20 * n_neg))\n",
|
|
"\n",
|
|
" # normalized weighted gini coefficient\n",
|
|
" g = gini / gini_max\n",
|
|
"\n",
|
|
" return 0.5 * (g + d)\n",
|
|
"\n",
|
|
"def lgb_amex_metric(y_true, y_pred):\n",
|
|
" \"\"\"The competition metric with lightgbm's calling convention\"\"\"\n",
|
|
" return ('amex_metric_score',\n",
|
|
" amex_metric(y_true, y_pred),\n",
|
|
" True)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"## Feature Engineering"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 12,
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"features_avg = ['B_1', 'B_2', 'B_3', 'B_4', 'B_5', 'B_6', 'B_8', 'B_9', 'B_10', 'B_11', 'B_12', 'B_13', 'B_14', 'B_15', \n",
|
|
" 'B_16', 'B_17', 'B_18', 'B_19', 'B_20', 'B_21', 'B_22', 'B_23', 'B_24', 'B_25', 'B_28', 'B_29', 'B_30', \n",
|
|
" 'B_32', 'B_33', 'B_37', 'B_38', 'B_39', 'B_40', 'B_41', 'B_42', 'D_39', 'D_41', 'D_42', 'D_43', 'D_44', \n",
|
|
" 'D_45', 'D_46', 'D_47', 'D_48', 'D_50', 'D_51', 'D_53', 'D_54', 'D_55', 'D_58', 'D_59', 'D_60', 'D_61', \n",
|
|
" 'D_62', 'D_65', 'D_66', 'D_69', 'D_70', 'D_71', 'D_72', 'D_73', 'D_74', 'D_75', 'D_76', 'D_77', 'D_78', \n",
|
|
" 'D_80', 'D_82', 'D_84', 'D_86', 'D_91', 'D_92', 'D_94', 'D_96', 'D_103', 'D_104', 'D_108', 'D_112', 'D_113', \n",
|
|
" 'D_114', 'D_115', 'D_117', 'D_118', 'D_119', 'D_120', 'D_121', 'D_122', 'D_123', 'D_124', 'D_125', 'D_126', \n",
|
|
" 'D_128', 'D_129', 'D_131', 'D_132', 'D_133', 'D_134', 'D_135', 'D_136', 'D_140', 'D_141', 'D_142', 'D_144', \n",
|
|
" 'D_145', 'P_2', 'P_3', 'P_4', 'R_1', 'R_2', 'R_3', 'R_7', 'R_8', 'R_9', 'R_10', 'R_11', 'R_14', 'R_15', 'R_16', \n",
|
|
" 'R_17', 'R_20', 'R_21', 'R_22', 'R_24', 'R_26', 'R_27', 'S_3', 'S_5', 'S_6', 'S_7', 'S_9', 'S_11', 'S_12', 'S_13', \n",
|
|
" 'S_15', 'S_16', 'S_18', 'S_22', 'S_23', 'S_25', 'S_26']\n",
|
|
"features_min = ['B_2', 'B_4', 'B_5', 'B_9', 'B_13', 'B_14', 'B_15', 'B_16', 'B_17', 'B_19', 'B_20', 'B_28', 'B_29', 'B_33', 'B_36', \n",
|
|
" 'B_42', 'D_39', 'D_41', 'D_42', 'D_45', 'D_46', 'D_48', 'D_50', 'D_51', 'D_53', 'D_55', 'D_56', 'D_58', 'D_59', \n",
|
|
" 'D_60', 'D_62', 'D_70', 'D_71', 'D_74', 'D_75', 'D_78', 'D_83', 'D_102', 'D_112', 'D_113', 'D_115', 'D_118', 'D_119', \n",
|
|
" 'D_121', 'D_122', 'D_128', 'D_132', 'D_140', 'D_141', 'D_144', 'D_145', 'P_2', 'P_3', 'R_1', 'R_27', 'S_3', 'S_5', \n",
|
|
" 'S_7', 'S_9', 'S_11', 'S_12', 'S_23', 'S_25']\n",
|
|
"features_max = ['B_1', 'B_2', 'B_3', 'B_4', 'B_5', 'B_6', 'B_7', 'B_8', 'B_9', 'B_10', 'B_12', 'B_13', 'B_14', 'B_15', 'B_16', 'B_17', \n",
|
|
" 'B_18', 'B_19', 'B_21', 'B_23', 'B_24', 'B_25', 'B_29', 'B_30', 'B_33', 'B_37', 'B_38', 'B_39', 'B_40', 'B_42', 'D_39', \n",
|
|
" 'D_41', 'D_42', 'D_43', 'D_44', 'D_45', 'D_46', 'D_47', 'D_48', 'D_49', 'D_50', 'D_52', 'D_55', 'D_56', 'D_58', 'D_59', \n",
|
|
" 'D_60', 'D_61', 'D_63', 'D_64', 'D_65', 'D_70', 'D_71', 'D_72', 'D_73', 'D_74', 'D_76', 'D_77', 'D_78', 'D_80', 'D_82', \n",
|
|
" 'D_84', 'D_91', 'D_102', 'D_105', 'D_107', 'D_110', 'D_111', 'D_112', 'D_115', 'D_116', 'D_117', 'D_118', 'D_119', \n",
|
|
" 'D_121', 'D_122', 'D_123', 'D_124', 'D_125', 'D_126', 'D_128', 'D_131', 'D_132', 'D_133', 'D_134', 'D_135', 'D_136', \n",
|
|
" 'D_138', 'D_140', 'D_141', 'D_142', 'D_144', 'D_145', 'P_2', 'P_3', 'P_4', 'R_1', 'R_3', 'R_5', 'R_6', 'R_7', 'R_8', \n",
|
|
" 'R_10', 'R_11', 'R_14', 'R_17', 'R_20', 'R_26', 'R_27', 'S_3', 'S_5', 'S_7', 'S_8', 'S_11', 'S_12', 'S_13', 'S_15', 'S_16', \n",
|
|
" 'S_22', 'S_23', 'S_24', 'S_25', 'S_26', 'S_27']\n",
|
|
"features_last = ['B_1', 'B_2', 'B_3', 'B_4', 'B_5', 'B_6', 'B_7', 'B_8', 'B_9', 'B_10', 'B_11', 'B_12', 'B_13', 'B_14', 'B_15', 'B_16', \n",
|
|
" 'B_17', 'B_18', 'B_19', 'B_20', 'B_21', 'B_22', 'B_23', 'B_24', 'B_25', 'B_26', 'B_28', 'B_29', 'B_30', 'B_32', 'B_33', \n",
|
|
" 'B_36', 'B_37', 'B_38', 'B_39', 'B_40', 'B_41', 'B_42', 'D_39', 'D_41', 'D_42', 'D_43', 'D_44', 'D_45', 'D_46', 'D_47', \n",
|
|
" 'D_48', 'D_49', 'D_50', 'D_51', 'D_52', 'D_53', 'D_54', 'D_55', 'D_56', 'D_58', 'D_59', 'D_60', 'D_61', 'D_62', 'D_63', \n",
|
|
" 'D_64', 'D_65', 'D_69', 'D_70', 'D_71', 'D_72', 'D_73', 'D_75', 'D_76', 'D_77', 'D_78', 'D_79', 'D_80', 'D_81', 'D_82', \n",
|
|
" 'D_83', 'D_86', 'D_91', 'D_96', 'D_105', 'D_106', 'D_112', 'D_114', 'D_119', 'D_120', 'D_121', 'D_122', 'D_124', 'D_125', \n",
|
|
" 'D_126', 'D_127', 'D_130', 'D_131', 'D_132', 'D_133', 'D_134', 'D_138', 'D_140', 'D_141', 'D_142', 'D_145', 'P_2', 'P_3', \n",
|
|
" 'P_4', 'R_1', 'R_2', 'R_3', 'R_4', 'R_5', 'R_6', 'R_7', 'R_8', 'R_9', 'R_10', 'R_11', 'R_12', 'R_13', 'R_14', 'R_15', \n",
|
|
" 'R_19', 'R_20', 'R_26', 'R_27', 'S_3', 'S_5', 'S_6', 'S_7', 'S_8', 'S_9', 'S_11', 'S_12', 'S_13', 'S_16', 'S_19', 'S_20', \n",
|
|
" 'S_22', 'S_23', 'S_24', 'S_25', 'S_26', 'S_27']"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 13,
|
|
"metadata": {
|
|
"tags": [
|
|
"block:feature_engineering",
|
|
"prev:load_data"
|
|
]
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# feature engineering gotten from https://www.kaggle.com/code/ambrosm/amex-lightgbm-quickstart\n",
|
|
"def get_features(df, \n",
|
|
" features_avg, \n",
|
|
" features_min, \n",
|
|
" features_max, \n",
|
|
" features_last\n",
|
|
" ):\n",
|
|
" '''\n",
|
|
" This function takes a dataframe with all features and returns the aggregated feature grouped by the customer id.\n",
|
|
" \n",
|
|
" df - dataframe\n",
|
|
" '''\n",
|
|
" cid = pd.Categorical(df.pop('customer_ID'), ordered=True) # get customer id\n",
|
|
" last = (cid != np.roll(cid, -1)) # mask for last statement of every customer\n",
|
|
" \n",
|
|
" df_avg = (df\n",
|
|
" .groupby(cid)\n",
|
|
" .mean()[features_avg]\n",
|
|
" .rename(columns={f: f\"{f}_avg\" for f in features_avg})\n",
|
|
" ) \n",
|
|
" \n",
|
|
" df_min = (df\n",
|
|
" .groupby(cid)\n",
|
|
" .min()[features_min]\n",
|
|
" .rename(columns={f: f\"{f}_min\" for f in features_min})\n",
|
|
" )\n",
|
|
" gc.collect()\n",
|
|
" print('Computed min')\n",
|
|
" \n",
|
|
" df_max = (df\n",
|
|
" .groupby(cid)\n",
|
|
" .max()[features_max]\n",
|
|
" .rename(columns={f: f\"{f}_max\" for f in features_max})\n",
|
|
" )\n",
|
|
" gc.collect()\n",
|
|
" print('Computed max')\n",
|
|
" \n",
|
|
" df = (df.loc[last, features_last]\n",
|
|
" .rename(columns={f: f\"{f}_last\" for f in features_last})\n",
|
|
" .set_index(np.asarray(cid[last]))\n",
|
|
" )\n",
|
|
" gc.collect()\n",
|
|
" print('Computed last')\n",
|
|
" \n",
|
|
" df_ = pd.concat([df, df_min, df_max, df_avg], axis=1, )\n",
|
|
" \n",
|
|
" del df, df_avg, df_min, df_max, cid, last\n",
|
|
" \n",
|
|
" return df_"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 14,
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Computed min\n",
|
|
"Computed max\n",
|
|
"Computed last\n",
|
|
"Computed min\n",
|
|
"Computed max\n",
|
|
"Computed last\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"# apply feature engineering function\n",
|
|
"train = get_features(df_train, features_avg, features_min, features_max, features_last)\n",
|
|
"test = get_features(df_test, features_avg, features_min, features_max, features_last)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 15,
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"B_1_last False\n",
|
|
"B_2_last True\n",
|
|
"B_3_last True\n",
|
|
"B_4_last False\n",
|
|
"B_5_last False\n",
|
|
" ... \n",
|
|
"S_18_avg False\n",
|
|
"S_22_avg True\n",
|
|
"S_23_avg True\n",
|
|
"S_25_avg True\n",
|
|
"S_26_avg False\n",
|
|
"Length: 469, dtype: bool"
|
|
]
|
|
},
|
|
"execution_count": 15,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"# check null values\n",
|
|
"train.isna().any()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"## Modelling: StratifiedKFold\n",
|
|
"\n",
|
|
"We cross-validate with a six-fold StratifiedKFold to handle the imbalanced nature of the target.\n",
|
|
"\n",
|
|
"Lightgbm handles null values efficiently."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 16,
|
|
"metadata": {
|
|
"tags": [
|
|
"block:modelling",
|
|
"prev:feature_engineering"
|
|
]
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"469 features\n",
|
|
"[20]\tvalid_0's binary_logloss: 0.267976\tvalid_0's amex_metric_score: 0.750976\n",
|
|
"Score = 0.7604229987279087\n",
|
|
"Fold 0\n",
|
|
"[20]\tvalid_0's binary_logloss: 0.267339\tvalid_0's amex_metric_score: 0.753257\n",
|
|
"Score = 0.7624180573803372\n",
|
|
"Fold 1\n",
|
|
"OOF Score: 0.76142\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"# Cross-validation\n",
|
|
"\n",
|
|
"features = [f for f in train.columns if f != 'customer_ID' and f != 'target']\n",
|
|
"\n",
|
|
"print(f\"{len(features)} features\")\n",
|
|
"\n",
|
|
"score_list = [] # lgbm score per fold\n",
|
|
"y_pred_list = [] # fold predictions list\n",
|
|
"\n",
|
|
"# init StratifiedKFold\n",
|
|
"kf = StratifiedKFold(n_splits=4)\n",
|
|
"\n",
|
|
"for fold, (idx_tr, idx_va) in enumerate(kf.split(train, target)):\n",
|
|
" \n",
|
|
" X_tr, X_va, y_tr, y_va, model = None, None, None, None, None\n",
|
|
"\n",
|
|
" X_tr = train.iloc[idx_tr][features]\n",
|
|
" X_va = train.iloc[idx_va][features]\n",
|
|
" y_tr = target[idx_tr]\n",
|
|
" y_va = target[idx_va]\n",
|
|
" \n",
|
|
" # init model\n",
|
|
" model = LGBMClassifier(n_estimators=N_EST,\n",
|
|
" learning_rate=LR, \n",
|
|
" random_state=2022)\n",
|
|
" # fit model\n",
|
|
" model.fit(X_tr, y_tr,\n",
|
|
" eval_set = [(X_va, y_va)], \n",
|
|
" eval_metric=[lgb_amex_metric],\n",
|
|
" early_stopping_rounds=30,\n",
|
|
" callbacks=[log_evaluation(20)])\n",
|
|
" \n",
|
|
" X_tr, y_tr = None, None\n",
|
|
" \n",
|
|
" # fold validation set predictions\n",
|
|
" y_va_pred = model.predict_proba(X_va, raw_score=True)\n",
|
|
" \n",
|
|
" # model score\n",
|
|
" score = amex_metric(y_va, y_va_pred)\n",
|
|
"\n",
|
|
" print(f\"Score = {score}\")\n",
|
|
" score_list.append(score)\n",
|
|
" \n",
|
|
" # test set predictions\n",
|
|
" y_pred_list.append(model.predict_proba(test[features], raw_score=True))\n",
|
|
" \n",
|
|
" print(f\"Fold {fold}\") \n",
|
|
"\n",
|
|
"# save model\n",
|
|
"joblib.dump(model, 'lgb.jl')\n",
|
|
"print(f\"OOF Score: {np.mean(score_list):.5f}\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 19,
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"image/png": "iVBORw0KGgoAAAANSUhEUgAABLMAAAI/CAYAAACMIJv7AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAABda0lEQVR4nO3dfbTdZXnn//eHI5LQkNBK8EcBDShjKk9RN6BQFFELNtSCpWqnWrEPGaZQfIAOsc5U7LTToA6UisoEKVQKlYqiCAp2HGkpVeQEEkIAq0AspCqVh0gkoITr98f+hh4P52Gfx713eL/Wysre9/f63vf1Ze1luq5e9/1NVSFJkiRJkiT1g+26nYAkSZIkSZLUKYtZkiRJkiRJ6hsWsyRJkiRJktQ3LGZJkiRJkiSpb1jMkiRJkiRJUt+wmCVJkiRJkqS+8axuJ9Dvdtlll1q0aFG305AkSZIkSdpmrFq16gdVtXCkaxazpmjRokUMDg52Ow1JkiRJkqRtRpLvjHbNbYaSJEmSJEnqGxazJEmSJEmS1DcsZkmSJEmSJKlveGbWFK3dsJFFy6/udhqSJEmSJOkZav2Kpd1OYVb1TGdWki1JVie5Lcmnk+w4StyeSb6a5PYk65K8c5x5L0py/CTyOSLJoRO9T5IkSZIkSTOnZ4pZwOaqWlJV+wE/Bk4cJe4J4NSqejHwcuCkJC+egXyOACxmSZIkSZIk9ZBeKmYNdT3wwpEuVNV3q+rm5vMjwB3A7p1MmuSPk9zUdH+tTJJm/JSm0+vWJJ9Ksoh2Me3dTbfY4dPxUJIkSZIkSZqanjszK8mzgNcD13QQuwh4CXBjh9OfW1V/0tx7MXAM8AVgObBXVT2eZOeqejjJecCmqvrwCOsuA5YBDMxf2OHSkiRJkiRJmqpe6syam2Q1MAj8K3DBWMFJ5gGfAd5VVT/scI1XJ7kxyVrgSGDfZvxW4JIkb6W9jXFMVbWyqlpV1RrYcUGHS0uSJEmSJGmqeqkza3NVLekkMMn2tAtZl1TVZzu8Zw7wMaBVVfcmOQOY01xeCrwS+BXgfUn2n2DukiRJkiRJmgW91JnVkeacqwuAO6rqrAncurVw9YOmq+v4Zr7tgD2r6qvA6cACYB7wCLDTtCUuSZIkSZKkKeu7YhZwGPA24MjmcPbVSX55vJuq6mHgfOA24FrgpubSAPA3zdbDW4C/bGK/ABznAfCSJEmSJEm9I1XV7Rz6WqvVqsHBwW6nIUmSJEmStM1IsqqqWiNd68fOLEmSJEmSJD1D9dIB8D8lyXOAr4xw6TVV9cAI8R+lvQVxqHOq6sKZyE+SJEmSJEmzr2eLWU3BaskE4k+auWwkSZIkSZLUC9xmKEmSJEmSpL5hMUuSJEmSJEl9w2KWJEmSJEmS+obFLEmSJEmSJPUNi1mSJEmSJEnqGz37NsN+sXbDRhYtv7rbaUiSJEnST1m/Ymm3U5CkGTFrnVlJtiRZnWRdkjVJTk0y6vpJDm7iVzfxxw259s4ktzVzvWucdS9Kcvwk8j0iyaETvU+SJEmSJEkzZzY7szZX1RKAJLsClwLzgfePEn8b0KqqJ5LsBqxJ8gVgMfB7wMHAj4FrklxVVd+e5nyPADYB/zzN80qSJEmSJGmSunJmVlXdDywDTk6SUWIeraonmq9zgGo+/wJw45Dr/wC8sZN1k/xxkpuarq6VW9dOckqS25PcmuRTSRYBJwLvbjrDDp/0w0qSJEmSJGnadO0A+Kq6GxgAdh0tJskhSdYBa4ETm+LVbcDhSZ6TZEfgl4E9O1z23Ko6qKr2A+YCxzTjy4GXVNUBzTrrgfOAs6tqSVVdPyyvZUkGkwxueXRjx88sSZIkSZKkqenptxlW1Y1VtS9wEPDeJHOq6g7gTODLwDXAamBLh1O+OsmNSdYCRwL7NuO3ApckeSvwxKh3/0deK6uqVVWtgR0XTOyhJEmSJEmSNGldK2Yl2Zt2Eer+8WKbAtYmYL/m+wVV9bKqeiXwEPAvHaw3B/gYcHxV7Q+cT3v7IsBS4KPAS4GbkviWR0mSJEmSpB7UlWJWkoW0t/GdW1U1SsxeW4tKSZ5P++D39c33XZu/n0f7vKxLO1h2a+HqB0nmAcc3c2wH7FlVXwVOBxYA84BHgJ0m83ySJEmSJEmaGbPZgTQ3yWpge9pb+S4Gzhoj/heB5Ul+AjwJ/H5V/aC59pkkzwF+ApxUVQ+Pt3hVPZzkfNpnbn0PuKm5NAD8TZIFQIC/bGK/AFye5FeBPxh+bpYkSZIkSZJmX0ZpjFKHWq1WDQ4OdjsNSZIkSZKkbUaSVVXVGulaTx8AL0mSJEmSJA3V9YPOkxxF++2EQ91TVcdNcJ6PAocNGz6nqi6cSn6SJEmSJEnqHV0vZlXVtcC10zDPSdOQjiRJkiRJknqY2wwlSZIkSZLUNyxmSZIkSZIkqW9YzJIkSZIkSVLfsJglSZIkSZKkvmExS5IkSZIkSX2j628z7HdrN2xk0fKru52GJElSz1q/Ymm3U5AkSduQnujMSrIlyeoka5LcnOTQDu6Zn+S+JOeOE7c+yS6TyOmEJD8/0fskSZIkSZI0c3qimAVsrqolVXUg8F7gzzu4538C/ziDOZ0AWMySJEmSJEnqIb1SzBpqPvDQWAFJXgY8F/jyRCZO8rkkq5KsS7KsGRtIclGS25KsTfLuJMcDLeCSpmNs7iSfRZIkSZIkSdOoV87MmptkNTAH2A04crTAJNsB/xt4K/DaCa7z21X1YFOcuinJZ4BFwO5VtV8z/85V9XCSk4HTqmpwwk8jSZIkSZKkGdErnVlbtxkuBo4GPpkko8T+PvDFqrpvEuuckmQN8HVgT2Af4G5g7yQfSXI08MPxJkmyLMlgksEtj26cRBqSJEmSJEmajF7pzHpKVX2tObB9IXD/CCGvAA5P8vvAPODZSTZV1fKx5k1yBO1OrldU1aNJrgPmVNVDSQ4EjgJOBN4E/PY4Oa4EVgLssNs+NYHHkyRJkiRJ0hT0XDEryWJgAHhgpOtV9ZtDYk8AWuMVshoLgIeaQtZi4OXNHLsAP66qzyT5JvA3TfwjwE6TfhBJkiRJkiRNu14pZm09MwsgwNurass0r3ENcGKSO4Bv0t5qCLA7cGFzFhe036YIcBFwXpLNtLu5Nk9zPpIkSZIkSZqgVLlLbiparVYNDnpGvCRJkiRJ0nRJsqqqWiNd65UD4CVJkiRJkqRx9co2w6dJsj9w8bDhx6vqkFHibwR2GDb8tqpaOxP5SZIkSZIkafb1bDGrKUItmUD8iEUuSZIkSZIkbTvcZihJkiRJkqS+YTFLkiRJkiRJfcNiliRJkiRJkvqGxSxJkiRJkiT1DYtZkiRJkiRJ6hs9+zbDfrF2w0YWLb+622lIkrRNW79iabdTkCRJUo+wM0uSJEmSJEl9oyeKWUm2JFmdZE2Sm5McOk78NUkeTnJVB3Nfl6Q1iZyOTfLiid4nSZIkSZKkmdMTxSxgc1UtqaoDgfcCfz5O/IeAt81wTscCFrMkSZIkSZJ6SK8Us4aaDzw0VkBVfQV4ZKITJ/l4ksEk65J8YMj4iiS3J7k1yYebzrA3AB9qOsZeMNG1JEmSJEmSNP165QD4uUlWA3OA3YAjZ2id91XVg0kGgK8kOQDYABwHLK6qSrJzVT2c5Ergqqq6fPgkSZYBywAG5i+coVQlSZIkSZI0XK90Zm3dZrgYOBr4ZJLMwDpvSnIzcAuwL+1thBuBx4ALkrwReHS8SapqZVW1qqo1sOOCGUhTkiRJkiRJI+mVYtZTquprwC7AtLY8JdkLOA14TVUdAFwNzKmqJ4CDgcuBY4BrpnNdSZIkSZIkTZ9e2Wb4lCSLgQHggWmeej7wI2BjkucCrweuSzIP2LGqvpjkBuDuJv4RYKdpzkGSJEmSJElT0CvFrK1nZgEEeHtVbRktOMn1wGJgXpL7gN+pqmvHWqCq1iS5BbgTuBe4obm0E/D5JHOatd/TjH8KOD/JKcDxVXXXSPPuv/sCBlcs7eQZJUmSJEmSNEU9UcyqqoEJxh8+gdgjhnw+YZSwg0e47wbaZ2pJkiRJkiSpR/TcmVmSJEmSJEnSaHqiM2skSfYHLh42/HhVHTJK/BXAXsOGTx9v+6EkSZIkSZL6R88Ws6pqLbBkAvHHzVw2kiRJkiRJ6gVuM5QkSZIkSVLfsJglSZIkSZKkvmExS5IkSZIkSX3DYpYkSZIkSZL6Rs8eAN8v1m7YyKLlV3c7DUmSZtX6FUu7nYIkSZKeoezMkiRJkiRJUt/omWJWkvclWZfk1iSrkxwyStwFSdY0cZcnmTfbuUqSJEmSJKk7eqKYleQVwDHAS6vqAOC1wL2jhL+7qg5s4v4VOHmW0pQkSZIkSVKX9UQxC9gN+EFVPQ5QVT+oqn8bKbCqfgiQJMBcoEabNMmvJLkxyS1J/m+S5ybZLsn6JDsPiftWc+0FSb6eZG2SP02yaTofUpIkSZIkSVPTK8WsLwN7JvmXJB9L8qqxgpNcCHwPWAx8ZIzQfwJeXlUvAT4F/LeqehL4PHBcM9chwHeq6vvAOcA5VbU/cN8Y6y9LMphkcMujGzt/SkmSJEmSJE1JTxSzqmoT8DJgGfDvwGVJThgj/h3AzwN3AG8eY+o9gGuTrAX+ENi3Gb9syH1vab4DvAL4dPP50jHWX1lVrapqDey4YIzlJUmSJEmSNJ16opgFUFVbquq6qno/7XOwfm28eNrdVmPFfQQ4t+m0+i/AnGb8a8ALkywEjgU+O8X0JUmSJEmSNAt6opiV5EVJ9hkytAT4zghxSfLCrZ+BNwB3jjH1AmBD8/ntWwerqoArgLOAO6rqgebS1/mP4thbJv4kkiRJkiRJmknP6nYCjXnAR5pD2Z8Avk17y+FwAf46yfzm8xrgv44x7xnAp5M8BPw/YK8h1y4DbgJOGDL2LuBvkrwPuAYY90Cs/XdfwOCKpeOFSZIkSZIkaRr0RDGrqlYBh3YQ9yRw2ATm/Tztw95HujZIuyA21AbaB8ZXkrcAL+p0LUmSJEmSJM28nihm9ZCXAec2WxgfBn67u+lIkiRJkiRpqJ4tZiW5gp/eFghwelVdO0Ls+4BfHzb86ar6s4msWVXXAwdOKFFJkiRJkiTNmp4tZlXVcROI/TNgQoUrSZIkSZIk9Z+eeJuhJEmSJEmS1AmLWZIkSZIkSeobFrMkSZIkSZLUNyxmSZIkSZIkqW/07AHw/WLtho0sWn51t9OQJPWY9SuWdjsFSZIkaZtkZ5YkSZIkSZL6Rk8Us5JsSbI6yZokNyc5dIzY5zcxq5OsS3LiOHOvT7LLJHI6IcnPT/Q+SZIkSZIkzZxe2Wa4uaqWACQ5Cvhz4FWjxH4XeEVVPZ5kHnBbkiur6t+mOacTgNuA6Z5XkiRJkiRJk9Qrxayh5gMPjXaxqn485OsOTKC7LMnngD2BOcA5VbUyyQBwAdACCvgr4N7m+yVJNtMunm2e4HNIkiRJkiRpmvVKMWtuktW0i0y7AUeOFZxkT+Bq4IXAH06gK+u3q+rBJHOBm5J8BlgE7F5V+zVz71xVDyc5GTitqgZHWH8ZsAxgYP7CDpeWJEmSJEnSVPXEmVk02wyrajFwNPDJJBktuKruraoDaBez3p7kuR2uc0qSNcDXaXdo7QPcDeyd5CNJjgZ+ON4kVbWyqlpV1RrYcUGHS0uSJEmSJGmqeqWY9ZSq+hqwCzBuy1PTkXUbcPh4sUmOAF5Le8vggcAtwJyqegg4ELgOOBH4xCRTlyRJkiRJ0gzruWJWksXAAPDAKNf3aLYJkuRngV8EvtnB1AuAh6rq0WaNlzdz7AJsV1WfAf478NIm/hFgp6k8iyRJkiRJkqZXr52ZBRDg7VW1ZZTYXwD+d5JqYj9cVWs7WOMa4MQkd9Aufn29Gd8duDDJ1sLee5u/LwLOG+8A+P13X8DgiqUdLC9JkiRJkqSpSlV1O4e+1mq1anDwaWfES5IkSZIkaZKSrKqq1kjXem6boSRJkiRJkjSaXtlm+DRJ9gcuHjb8eFUdMkr8jcAOw4bf1uEWREmSJEmSJPWBni1mNUWoJROIH7HIJUmSJEmSpG2H2wwlSZIkSZLUNyxmSZIkSZIkqW9YzJIkSZIkSVLfsJglSZIkSZKkvtGzB8D3i7UbNrJo+dXdTkOSnjHWr1ja7RQkSZIkdZGdWZIkSZIkSeobs1LMSrIlyeok65KsSXJqknHXTvK8JJuSnNZ83zPJV5Pc3sz1zpnPXpIkSZIkSb1itrYZbq6qJQBJdgUuBeYD7x/nvrOALw35/gRwalXdnGQnYFWSv6+q22cgZ0mSJEmSJPWYWd9mWFX3A8uAk5NktLgkxwL3AOuG3Pvdqrq5+fwIcAew+xhz/F6Sm5pusM8k2THJgiTf2doZluRnktybZPskByW5teki+1CS26bloSVJkiRJkjQtunJmVlXdDQwAu450Pck84HTgA6PNkWQR8BLgxjGW+mxVHVRVB9IufP1OVW0EVgOvamKOAa6tqp8AFwL/peki2zLG2suSDCYZ3PLoxjGWlyRJkiRJ0nTq1QPgzwDOrqpNI11sil2fAd5VVT8cY579klyfZC3wm8C+zfhlwJubz28BLkuyM7BTVX2tGb90tEmramVVtaqqNbDjgk6fSZIkSZIkSVM0W2dm/ZQke9PufLp/lJBDgOOTfBDYGXgyyWNVdW6S7WkXsi6pqs+Os9RFwLFVtSbJCcARzfiVwP9K8nPAy4D/B+w0+SeSJEmSJEnSbJj1YlaShcB5wLlVVSPFVNXhQ+LPADY1hawAFwB3VNVZHSy3E/DdpgD2m8CGZv5NSW4CzgGuqqotwMNJHklySFXdSLtjS5IkSZIkST1ktopZc5OsBran/UbCi2m/qXCiDgPeBqxt5gP4o6r64ijx/4P2mVr/3vw9tPvqMuDT/Ee3FsDvAOcneRL4B8ADsSRJkiRJknpIRmmOekZKMm/rOV1JlgO7VdU7x7qn1WrV4ODgrOQnSZIkSZL0TJBkVVW1RrrWlTOzetjSJO+l/d/lO8AJ3U1HkiRJkiRJQ3W1mJXkKODMYcP3VNVxE5zno7S3IA51TlVdOJF5quoy2tsPJUmSJEmS1IO6WsyqqmuBa6dhnpOmIR1JkiRJkiT1uO26nYAkSZIkSZLUKYtZkiRJkiRJ6hsWsyRJkiRJktQ3LGZJkiRJkiSpb1jMkiRJkiRJUt/o6tsMtwVrN2xk0fKru52GJHXN+hVLu52CJEmSpGeQnunMSvK+JOuS3JpkdZJDRom7IMmaJu7yJPPGmPOMJKdNIpclSX55ovdJkiRJkiRpZvVEMSvJK4BjgJdW1QHAa4F7Rwl/d1Ud2MT9K3DyDKS0BLCYJUmSJEmS1GN6opgF7Ab8oKoeB6iqH1TVv40UWFU/BEgSYC5QnSyQ5PeS3NR0dX0myY7N+K8nua0Z/8ckzwb+BHhz0yH25ml4PkmSJEmSJE2DXilmfRnYM8m/JPlYkleNFZzkQuB7wGLgIx2u8dmqOqiqDgTuAH6nGf9j4Khm/A1V9eNm7LKqWlJVl42w/rIkg0kGtzy6scPlJUmSJEmSNFU9Ucyqqk3Ay4BlwL8DlyU5YYz4dwA/T7so1Wnn1H5Jrk+yFvhNYN9m/AbgoiS/Bwx0mO/KqmpVVWtgxwUdLi9JkiRJkqSp6oliFkBVbamq66rq/bTPwfq18eKBT40XN8RFwMlVtT/wAWBOM8+JwH8H9gRWJXnO5J5AkiRJkiRJM60nillJXpRknyFDS4DvjBCXJC/c+hl4A3Bnh8vsBHw3yfa0O7O2zvmCqrqxqv6YdlfYnsAjTbwkSZIkSZJ6yLO6nUBjHvCRJDsDTwDfpr3lcLgAf51kfvN5DfBfO1zjfwA30i5Y3ch/FKs+1BTSAnylmfNfgeVJVgN/PtK5WZIkSZIkSZp9qeroZYAaRavVqsHBwW6nIUmSJEmStM1IsqqqWiNd64lthpIkSZIkSVInemWb4dMkuQLYa9jw6VV17Qix7wN+fdjwp6vqz2YqP0mSJEmSJM2+ni1mVdVxE4j9M8DClSRJkiRJ0jbObYaSJEmSJEnqGxazJEmSJEmS1DcsZkmSJEmSJKlvWMySJEmSJElS37CYJUmSJEmSpL7Rs28z7BdrN2xk0fKru52GJE3Z+hVLu52CJEmSJI1r1jqzkmxJsjrJuiRrkpyaZNz1kzwvyaYkpw0Z+6sk9ye5rYP7L0py/CTyPSLJoRO9T5IkSZIkSTNnNrcZbq6qJVW1L/A64PXA+zu47yzgS8PGLgKOnt70nuYIwGKWJEmSJElSD+nKmVlVdT+wDDg5SUaLS3IscA+wbtj9/wg8ONF1k/xxkpuS3JZk5da1k5yS5PYktyb5VJJFwInAu5tussMnupYkSZIkSZKmX9cOgK+qu4EBYNeRrieZB5wOfGAalz23qg6qqv2AucAxzfhy4CVVdQBwYlWtB84Dzm66ya4fltuyJINJBrc8unEa05MkSZIkSdJYevlthmfQLiZtmsY5X53kxiRrgSOBfZvxW4FLkrwVeGK8SapqZVW1qqo1sOOCaUxPkiRJkiRJY+na2wyT7A1sAe4fJeQQ4PgkHwR2Bp5M8lhVnTvJ9eYAHwNaVXVvkjOAOc3lpcArgV8B3pdk/8msIUmSJEmSpJnVlWJWkoW0t/GdW1U1UkxVHT4k/gxg02QLWY2thasfNFsYjwcub96ouGdVfTXJPwFvAeYBjwDzp7CeJEmSJEmSptlsbjOc2xymvg74v8CXmeR5WEn+Fvga8KIk9yX5nfHuqaqHgfOB24BrgZuaSwPA3zRbD28B/rKJ/QJwnAfAS5IkSZIk9Y6M0hilDrVarRocHOx2GpIkSZIkSduMJKuqqjXStV4+AF6SJEmSJEn6KV07AH6rJEcBZw4bvqeqjpvgPB8FDhs2fE5VXTiV/CRJkiRJktQ7ul7MqqpraZ9hNdV5TpqGdCRJkiRJktTD3GYoSZIkSZKkvmExS5IkSZIkSX3DYpYkSZIkSZL6hsUsSZIkSZIk9Q2LWZIkSZIkSeobXX+bYb9bu2Eji5Zf3e00JGnK1q9Y2u0UJEmSJGlcs9aZlWRLktVJ1iVZk+TUJOOun+R5STYlOa35PifJN5o51iX5wDj3X5ekNYl8j03y4oneJ0mSJEmSpJkzm9sMN1fVkqraF3gd8Hrg/R3cdxbwpSHfHweOrKoDgSXA0UlePt3JAscCFrMkSZIkSZJ6SFfOzKqq+4FlwMlJMlpckmOBe4B1Q+6tqtrUfN2++VOdrJvk40kGh3d0JVmR5PYktyb5cJJDgTcAH2q6yV4w0WeUJEmSJEnS9OvamVlVdXeSAWBX4PvDryeZB5xOu4vrtGHXBoBVwAuBj1bVjR0u+76qerC5/ytJDgA2AMcBi6uqkuxcVQ8nuRK4qqouHyG3ZbSLcQzMX9jh0pIkSZIkSZqqXn6b4RnA2UO6sJ5SVVuqagmwB3Bwkv06nPNNSW4GbgH2pb2NcCPwGHBBkjcCj443SVWtrKpWVbUGdlzQ4dKSJEmSJEmaqq51ZiXZG9gC3D9KyCHA8Uk+COwMPJnksao6d2tA00H1VeBo4LZx1tuLdofXQVX1UJKLgDlV9USSg4HXAMcDJwNHTunhJEmSJEmSNCO6UsxKshA4Dzi3qkY876qqDh8SfwawqarObe79SVPImkt7G+KZHSw7H/gRsDHJc2kfQH9ds51xx6r6YpIbgLub+EeAnSb3hJIkSZIkSZoJs1nMmptkNe0D258ALqb9psKJ2g346+bcq+2Av6uqq8a7qarWJLkFuBO4F7ihubQT8Pkkc4AA72nGPwWcn+QU4PiqumsSuUqSJEmSJGkaZZTGKHWo1WrV4OBgt9OQJEmSJEnaZiRZVVWtka718gHwkiRJkiRJ0k/p2gHwWyU5iqefeXVPVR03wXmuAPYaNnx6VV07lfwkSZIkSZLUO7pezGqKTVMuOE20+CVJkiRJkqT+4zZDSZIkSZIk9Q2LWZIkSZIkSeobFrMkSZIkSZLUNyxmSZIkSZIkqW9YzJIkSZIkSVLf6PrbDPvd2g0bWbT86m6nIUlPWb9iabdTkCRJkqQZM2udWUm2JFmdZF2SNUlOTTLu+kmel2RTktOa73OSfKOZY12SD8x89pIkSZIkSeoFs9mZtbmqlgAk2RW4FJgPvH+c+84CvjTk++PAkVW1Kcn2wD8l+VJVfX0GcpYkSZIkSVIP6cqZWVV1P7AMODlJRotLcixwD7BuyL1VVZuar9s3f2qMOf44yU1JbkuyMm2Lk3xjSMyiJGubz7+c5M4kq5L8ZZKrpvKskiRJkiRJmj5dOwC+qu4GBoBdR7qeZB5wOvC0bYRJBpKsBu4H/r6qbhxjqXOr6qCq2g+YCxxTVXcCz06yVxPzZuCyJHOA/wO8vqpeBiwcJbdlSQaTDG55dGMnjytJkiRJkqRp0MtvMzwDOHtIF9ZTqmpLs2VxD+DgJPuNMc+rk9zYdF4dCezbjP8d7SIWzd+XAYuBu6vqnmb8b0easKpWVlWrqloDOy6Y4GNJkiRJkiRpsrr2NsMkewNbaHdXjeQQ4PgkHwR2Bp5M8lhVnbs1oKoeTvJV4GjgthHWmAN8DGhV1b1JzgDmNJcvAz6d5LPtqepbSZZMy8NJkiRJkiRpRnSlMyvJQuA82lsARzzvqqoOr6pFVbUI+Avgf1XVuUkWJtm5mWcu8DrgzlGW2lq4+kGzbfH4IfPfRbuY9j9oF7YAvgnsnWRR831r55YkSZIkSZJ6wGx2Zs1tzrnaHngCuJj2mwonajfgr5MM0C7G/V1VjXhIe9O5dT7trq3vATcNC7kM+BCwVxO/OcnvA9ck+dEI8ZIkSZIkSeqijNIY9YyVZF5VbWresvhR4FtVdfZo8a1WqwYHB2cvQUmSJEmSpG1cklVV1RrpWi8fAN8tv9d0kK0DFtB+u6EkSZIkSZJ6QNcOgN8qyVHAmcOG76mq4yY4zxU02wWHOL2qrp3IPE0X1qidWJIkSZIkSeqerhezmmLThApOo8wzoeKXJEmSJEmS+o/bDCVJkiRJktQ3LGZJkiRJkiSpb1jMkiRJkiRJUt+wmCVJkiRJkqS+YTFLkiRJkiRJfaPrbzPsd2s3bGTR8qu7nYYkPWX9iqXdTkGSJEmSZsysdWYl2ZJkdZJ1SdYkOTXJuOsneV6STUlOGzL2ziS3NXO9a5z7L0py/CTyPSLJoRO9T5IkSZIkSTNnNjuzNlfVEoAkuwKXAvOB949z31nAl7Z+SbIf8HvAwcCPgWuSXFVV357mfI8ANgH/PM3zSpIkSZIkaZK6cmZWVd0PLANOTpLR4pIcC9wDrBsy/AvAjVX1aFU9AfwD8MZO1k3yx0luarq6Vm5dO8kpSW5PcmuSTyVZBJwIvLvpJjt8Ms8pSZIkSZKk6dW1A+Cr6m5gANh1pOtJ5gGnAx8Yduk24PAkz0myI/DLwJ4dLntuVR1UVfsBc4FjmvHlwEuq6gDgxKpaD5wHnF1VS6rq+gk8miRJkiRJkmZIL7/N8AzaxaRNQwer6g7gTODLwDXAamBLh3O+OsmNSdYCRwL7NuO3ApckeSvwxHiTJFmWZDDJ4JZHN3a4tCRJkiRJkqaqa8WsJHvTLkLdP0rIIcAHk6wH3gX8UZKTAarqgqp6WVW9EngI+JcO1psDfAw4vqr2B84H5jSXlwIfBV4K3JRkzLPEqmplVbWqqjWw44LxlpYkSZIkSdI0mc0D4J+SZCHtbXznVlWNFFNVhw+JPwPYVFXnNt93rar7kzyP9nlZL+9g2a2Fqx80WxiPBy5v3qi4Z1V9Nck/AW8B5gGP0D6gXpIkSZIkST1iNotZc5OsBranvZXvYtpvKpyMzyR5DvAT4KSqeni8G6rq4STn0z5z63vATc2lAeBvkiwAAvxlE/sF2sWuXwX+wHOzJEmSJEmSui+jNEapQ61WqwYHB7udhiRJkiRJ0jYjyaqqao10rZcPgJckSZIkSZJ+SlfOzBoqyVG030441D1VddwE5/kocNiw4XOq6sKp5CdJkiRJkqTe0fViVlVdC1w7DfOcNA3pSJIkSZIkqYe5zVCSJEmSJEl9w2KWJEmSJEmS+obFLEmSJEmSJPUNi1mSJEmSJEnqGxazJEmSJEmS1De6/jbDfrd2w0YWLb+622lI6mHrVyztdgqSJEmStM2wM0uSJEmSJEl9oyeKWUm2JFmdZE2Sm5McOkbskiRfS7Iuya1J3jzO3NclaU0ip2OTvHii90mSJEmSJGnm9EQxC9hcVUuq6kDgvcCfjxH7KPBbVbUvcDTwF0l2noGcjgUsZkmSJEmSJPWQXilmDTUfeGi0i1X1L1X1rebzvwH3Aws7mTjJx5MMNl1dHxgyviLJ7U2n14ebzrA3AB9qOsZeMKUnkiRJkiRJ0rTolQPg5yZZDcwBdgOO7OSmJAcDzwbu6nCd91XVg0kGgK8kOQDYABwHLK6qSrJzVT2c5Ergqqq6fIR1lwHLAAbmd1RHkyRJkiRJ0jTolc6srdsMF9PeOvjJJBnrhiS7ARcD76iqJztc501JbgZuAfalvY1wI/AYcEGSN9LexjimqlpZVa2qag3suKDDpSVJkiRJkjRVvVLMekpVfQ3YhTG2DiaZD1xNu9Pq653Mm2Qv4DTgNVV1QHP/nKp6AjgYuBw4Brhmak8gSZIkSZKkmdIr2wyfkmQxMAA8MMr1ZwNXAJ8caQvgGOYDPwI2Jnku8HrguiTzgB2r6otJbgDubuIfAXaa5GNIkiRJkiRpBvRKMWvrmVkAAd5eVVtGiX0T8ErgOUlOaMZOqKrVo8QDUFVrktwC3AncC9zQXNoJ+HySOc3a72nGPwWcn+QU4PiqGvFcrv13X8DgiqXjPJ4kSZIkSZKmQ6qq2zn0tVarVYODg91OQ5IkSZIkaZuRZFVVtUa61nNnZkmSJEmSJEmj6ZVthk+TZH/abysc6vGqOmSU+CuAvYYNn15V185EfpIkSZIkSZp9PVvMqqq1wJIJxB83c9lIkiRJkiSpF7jNUJIkSZIkSX3DYpYkSZIkSZL6hsUsSZIkSZIk9Q2LWZIkSZIkSeobPXsAfL9Yu2Eji5Zf3e00JPWw9SuWdjsFSZIkSdpm2JklSZIkSZKkvjFrxawkW5KsTrIuyZokpyYZd/0kz0uyKclpQ8Z2TnJ5kjuT3JHkFWPcf1GS4yeR7xFJDp3ofZIkSZIkSZo5s7nNcHNVLQFIsitwKTAfeP84950FfGnY2DnANVV1fJJnAztOc64ARwCbgH+egbklSZIkSZI0CV3ZZlhV9wPLgJOTZLS4JMcC9wDrhowtAF4JXNDM9eOqeriTdZP8cZKbktyWZOXWtZOckuT2JLcm+VSSRcCJwLubbrLDJ/WgkiRJkiRJmlZdOzOrqu4GBoBdR7qeZB5wOvCBYZf2Av4duDDJLUk+keRnOlz23Ko6qKr2A+YCxzTjy4GXVNUBwIlVtR44Dzi7qpZU1fXDcluWZDDJ4JZHN3a4tCRJkiRJkqaqlw+AP4N2MWnTsPFnAS8FPl5VLwF+RLsY1YlXJ7kxyVrgSGDfZvxW4JIkbwWeGG+SqlpZVa2qag3suKDDpSVJkiRJkjRVs3lm1k9JsjewBbh/lJBDgOOTfBDYGXgyyWPA5cB9VXVjE3c5HRSzkswBPga0qureJGcAc5rLS2lvXfwV4H1J9p/UQ0mSJEmSJGlGdaWYlWQh7W1851ZVjRRTVYcPiT8D2FRV5zbf703yoqr6JvAa4PYOlt1auPpBs4XxeODy5o2Ke1bVV5P8E/AWYB7wCO0D6iVJkiRJktQjZrOYNTfJamB72lv5Lqb9psLJ+APa2wKfDdwNvGO8G6rq4STnA7cB3wNuai4NAH/THCwf4C+b2C/QLnb9KvAHw8/N2mr/3RcwuGLpJB9DkiRJkiRJE5FRGqPUoVarVYODg91OQ5IkSZIkaZuRZFVVtUa61ssHwEuSJEmSJEk/pWsHwG+V5CjgzGHD91TVcROc56PAYcOGz6mqC6eSnyRJkiRJknpH14tZVXUtcO00zHPSNKQjSZIkSZKkHuY2Q0mSJEmSJPUNi1mSJEmSJEnqGxazJEmSJEmS1DcsZkmSJEmSJKlvdP0A+H63dsNGFi2/uttpSJoF61cs7XYKkiRJkvSMZ2eWJEmSJEmS+kZPFLOSbEmyOsltSb6QZOdx4q9J8nCSqzqY+7okrUnkdGySF0/0PkmSJEmSJM2cnihmAZuraklV7Qc8CJw0TvyHgLfNcE7HAhazJEmSJEmSekivFLOG+hqw+1gBVfUV4JGJTpzk40kGk6xL8oEh4yuS3J7k1iQfTnIo8AbgQ03H2AsmupYkSZIkSZKmX08dAJ9kAHgNcMEMLfG+qnqwWecrSQ4ANgDHAYurqpLsXFUPJ7kSuKqqLh8hz2XAMoCB+QtnKFVJkiRJkiQN1yudWXOTrAa+BzwX+PsZWudNSW4GbgH2pb2NcCPwGHBBkjcCj443SVWtrKpWVbUGdlwwQ6lKkiRJkiRpuF4pZm2uqiXA84Ew/plZE5ZkL+A04DVVdQBwNTCnqp4ADgYuB44BrpnutSVJkiRJkjQ9eqWYBUBVPQqcApyaZLq3QM4HfgRsTPJc4PUASeYBC6rqi8C7gQOb+EeAnaY5B0mSJEmSJE3BmAWjJO8Z63pVnTW96UBV3ZLkVuA3gItHyet6YDEwL8l9wO9U1bXjzLsmyS3AncC9wA3NpZ2AzyeZQ7srbOszfwo4P8kpwPFVdddI8+6/+wIGVyyd0DNKkiRJkiRpcsbrfpqVzqSqmjfs+6+ME3/4BOY+YsjnE0YJO3iE+26gfaaWJEmSJEmSesSYxayq+sBsJSJJkiRJkiSNp6NzqZLsAXwEOKwZuh54Z1XdN1OJJdmfp28zfLyqDhkl/gpgr2HDp4+3/VCSJEmSJEn9o9ND1i8ELgV+vfn+1mbsdTORFEBVrQWWTCD+uJnKRZIkSZIkSb2h07cZLqyqC6vqiebPRcDCGcxLkiRJkiRJeppOi1kPJHlrkoHmz1uBB2YyMUmSJEmSJGm4TotZvw28Cfge8F3geOCEGcpJkiRJkiRJGlGnZ2b9CfD2qnoIIMnPAR+mXeSSJEmSJEmSZkWnnVkHbC1kAVTVg8BLZiYlSZIkSZIkaWSddmZtl+Rnh3VmdXrvNm3tho0sWn51t9OQnpHWr1ja7RQkSZIkSbOs086s/w18Lcn/TPI/gX8GPjiRhZJsSbI6yZokNyc5dJz4a5I8nOSqYeMnJ/l2kkqyyzhznJDk3Ink2dy3KMl/nuh9kiRJkiRJmlkdFbOq6pPAG4HvN3/eWFUXT3CtzVW1pKoOBN4L/Pk48R8C3jbC+A3Aa4HvTHD9iVgEWMySJEmSJEnqMR1vFayq24Hbp2nd+cBDYwVU1VeSHDHC+C0ASSa0YJJfAf478GzgAeA3q+r7SV4FnLN1euCVwArgF5KsBv66qs6e0GKSJEmSJEmaEbN57tXcpjg0B9gNOHIW1wb4J+DlVVVJfhf4b8CpwGnASVV1Q5J5wGPAcuC0qjpmpImSLAOWAQzMXzgryUuSJEmSJGl2i1mbq2oJQJJXAJ9Msl9V1SytvwdwWZLdaHdn3dOM3wCcleQS4LNVdd94XV9VtRJYCbDDbvvMVv6SJEmSJEnPeJ0eAD+tquprwC7AbLY1fQQ4t6r2B/4L7Q4xqmoF8LvAXOCGJItnMSdJkiRJkiRNwGx2Zj2lKRgN0D67arYsADY0n98+JJcXVNVaYG2Sg4DFwL3ATrOYmyRJkiRJkjowm51Zc5Osbs7Nugx4e1VtGS04yfXAp4HXJLkvyVHN+ClJ7qO9bfDWJJ/ocP0zgE8nWQX8YMj4u5LcluRW4CfAl4BbgS1J1iR598QeU5IkSZIkSTMls3dk1bap1WrV4OBgt9OQJEmSJEnaZiRZVVWtka515cwsSZIkSZIkaTK6cmbWVkn2By4eNvx4VR0ywXneAbxz2PANVXXSVPKTJEmSJElSb+lqMas5eH3JNMxzIXDhlBOSJEmSJElST3OboSRJkiRJkvqGxSxJkiRJkiT1DYtZkiRJkiRJ6hsWsyRJkiRJktQ3LGZJkiRJkiSpb3T1bYbbgrUbNrJo+dXdTkN6Rlq/Ymm3U5AkSZIkzbJZ6cxKsiXJ6iTrkqxJcmqScddO8rwkm5Kc1nzfM8lXk9zezPXOacrvn6djHkmSJEmSJM2s2erM2lxVSwCS7ApcCswH3j/OfWcBXxry/Qng1Kq6OclOwKokf19Vt08luao6dCr3S5IkSZIkaXbM+plZVXU/sAw4OUlGi0tyLHAPsG7Ivd+tqpubz48AdwC7jzHHdUnOTjKY5I4kByX5bJJvJfnTIXGbmr+PaO65PMmdSS4ZK0dJkiRJkiTNrq4cAF9VdwMDwK4jXU8yDzgd+MBocyRZBLwEuHGc5X5cVS3gPODzwEnAfsAJSZ4zQvxLgHcBLwb2Bg4bYe1lTYFscMujG8dZXpIkSZIkSdOlV99meAZwdlVtGuliU+z6DPCuqvrhOHNd2fy9FljXdHc9DtwN7DlC/Deq6r6qehJYDSwaHlBVK6uqVVWtgR0XdPI8kiRJkiRJmgZdeZthkr2BLcD9o4QcAhyf5IPAzsCTSR6rqnOTbE+7kHVJVX22g+Ueb/5+csjnrd9Hev6hMVtGiZEkSZIkSVIXzHqhJslC2lv+zq2qGimmqg4fEn8GsKkpZAW4ALijqs6ajXwlSZIkSZLUO2armDU3yWpge9pvJLyY9psKJ+ow4G3A2mY+gD+qqi9OR5KSJEmSJEnqbRmlOUodarVaNTg42O00JEmSJEmSthlJVjUv9HuaXj0AXpIkSZIkSXqarh5unuQo4Mxhw/dU1XETnOejtLcgDnVOVV04lfwkSZIkSZLUW7pazKqqa4Frp2Gek6YhHUmSJEmSJPU4txlKkiRJkiSpb1jMkiRJkiRJUt+wmCVJkiRJkqS+YTFLkiRJkiRJfcNiliRJkiRJkvpGV99muC1Yu2Eji5Zf3e00pJ6wfsXSbqcgSZIkSdrG9URnVpItSVYnWZPk5iSHdhi/OsmV48Rel6Q1iZyOTfLiid4nSZIkSZKkmdMrnVmbq2oJQJKjgD8HXtVJ/Aw6FrgKuH2G15EkSZIkSVKHeqIza5j5wEMzMXGSjycZTLIuyQeGjK9IcnuSW5N8uOkMewPwoab76wUzkY8kSZIkSZImplc6s+YmWQ3MAXYDjhwnfk6SQeAJYEVVfa7Ddd5XVQ8mGQC+kuQAYANwHLC4qirJzlX1cLN98aqqunz4JEmWAcsABuYv7HBpSZIkSZIkTVWvdGZtrqolVbUYOBr4ZJKMEf/8qmoB/xn4iwl0Tr0pyc3ALcC+wIuBjcBjwAVJ3gg8Ot4kVbWyqlpV1RrYcUGHS0uSJEmSJGmqeqWY9ZSq+hqwCzBqy1NVbWj+vhu4DnjJePMm2Qs4DXhNVR0AXA3MqaongIOBy4FjgGum+AiSJEmSJEmaIT1XzEqyGBgAHhjl+s8m2aH5vAtwGJ0d0j4f+BGwMclzgdc3c8wDFlTVF4F3Awc28Y8AO03hUSRJkiRJkjTNeu3MLIAAb6+qLaPE/gLwf5I8SbsYt6Kqxi1mVdWaJLcAdwL3Ajc0l3YCPp9kTrP2e5rxTwHnJzkFOL6q7prEc0mSJEmSJGkapaq6nUNfa7VaNTg42O00JEmSJEmSthlJVjXnpT9Nz20zlCRJkiRJkkbTK9sMnybJ/sDFw4Yfr6pDRom/Athr2PDpVXXtTOQnSZIkSZKk2dezxayqWgssmUD8cTOXjSRJkiRJknqB2wwlSZIkSZLUNyxmSZIkSZIkqW9YzJIkSZIkSVLfsJglSZIkSZKkvmExS5IkSZIkSX2jZ99m2C/WbtjIouVXdzsNqSesX7G02ylIkiRJkrZxM9aZlWRLktVJ1iVZk+TUJKOul+TgJn51E3/ckGvvTHJbM9e7ZipnSZIkSZIk9baZ7MzaXFVLAJLsClwKzAfeP0r8bUCrqp5IshuwJskXgMXA7wEHAz8GrklyVVV9ewZzlyRJkiRJUg+alTOzqup+YBlwcpKMEvNoVT3RfJ0DVPP5F4Abh1z/B+CNo62V5LokZycZTHJHkoOSfDbJt5L86ZC4zyVZ1XR7LWvGnt/E7ZJkuyTXJ/mlqf8XkCRJkiRJ0nSYtTOzquruJAPArsD3R4pJcgjwV8Dzgbc1XVq3AX+W5DnAZuCXgcFxlvtxVbWSvBP4PPAy4EHgriRnV9UDwG9X1YNJ5gI3JflMVX0nyZnAx4FvALdX1ZdHyHMZ7eIcA/MXTvQ/hSRJkiRJkiapp95mWFU3VtW+wEHAe5PMqao7gDOBLwPXAKuBLeNMdWXz91pgXVV9t6oeB+4G9myunZJkDfD1ZmyfJodP0N4OeSJw2ih5rqyqVlW1BnZcMLmHlSRJkiRJ0oTNWjEryd60i1D3jxfbFLA2Afs13y+oqpdV1SuBh4B/GWeKx5u/nxzyeev3ZyU5Angt8IqqOhC4hfbWRpLsCOzRxM8b98EkSZIkSZI0a2almJVkIXAecG5V1SgxeyV5VvP5+bQPfl/ffN+1+ft5tM/LunSKKS0AHqqqR5MsBl4+5NqZwCXAHwPnT3EdSZIkSZIkTaOZPDNrbpLVwPbAE8DFwFljxP8isDzJT2h3UP1+Vf2gufaZ5sysnwAnVdXDU8ztGuDEJHcA36S91ZAkr6K9xfGwqtqS5NeSvKOqLpziepIkSZIkSZoGGaVRSh1qtVo1ODjeefSSJEmSJEnqVJJVVdUa6VpPHQAvSZIkSZIkjWUmtxmOKMlRtM+lGuqeqjpugvN8FDhs2PA5bgmUJEmSJEnads16MauqrgWunYZ5TpqGdCRJkiRJktRH3GYoSZIkSZKkvmExS5IkSZIkSX3DYpYkSZIkSZL6hsUsSZIkSZIk9Q2LWZIkSZIkSeobs/42w23N2g0bWbT86m6nIfWE9SuWdjsFSZIkSdI2ric6s5JsSbI6yW1JvpBk5zFilyT5WpJ1SW5N8uZx5r4uSWsSOR2b5MUTvU+SJEmSJEkzpyeKWcDmqlpSVfsBDwInjRH7KPBbVbUvcDTwF2MVv6bgWMBiliRJkiRJUg/plWLWUF8Ddh/tYlX9S1V9q/n8b8D9wMJOJk7y8SSDTVfXB4aMr0hye9Pp9eEkhwJvAD7UdIy9YEpPJEmSJEmSpGnRU2dmJRkAXgNc0GH8wcCzgbs6XOJ9VfVgs85XkhwAbACOAxZXVSXZuaoeTnIlcFVVXT7xJ5EkSZIkSdJM6JXOrLlJVgPfA54L/P14NyTZDbgYeEdVPdnhOm9KcjNwC7Av7W2EG4HHgAuSvJH2Nsbx1l7WdHgNbnl0Y4dLS5IkSZIkaap6pZi1uaqWAM8HwthnZpFkPnA17U6rr3eyQJK9gNOA11TVAc39c6rqCeBg4HLgGOCa8eaqqpVV1aqq1sCOCzpZXpIkSZIkSdOgV4pZAFTVo8ApwKlJRtwCmeTZwBXAJye4BXA+8CNgY5LnAq9v5psHLKiqLwLvBg5s4h8BdprUg0iSJEmSJGlG9FQxC6CqbgFuBX5jlJA3Aa8ETmgOZ1+dZEkH866hvb3wTuBS4Ibm0k7AVUluBf4JeE8z/ingD5Pc4gHwkiRJkiRJvSFV1e0c+lqr1arBwcFupyFJkiRJkrTNSLKqqlojXeu5zixJkiRJkiRpNCOeS9ULkuxP+22FQz1eVYeMEn8FsNew4dOr6tqZyE+SJEmSJEmzr2eLWVW1FlgygfjjZi4bSZIkSZIk9QK3GUqSJEmSJKlvWMySJEmSJElS37CYJUmSJEmSpL5hMUuSJEmSJEl9w2KWJEmSJEmS+kbPvs2wX6zdsJFFy6/udhrStFq/Ymm3U5AkSZIkaUR2ZkmSJEmSJKlvzFgxK8mWJKuTrEuyJsmpScZdL8nzkmxKclrzfc8kX01yezPXO2cqZ0mSJEmSJPW2mdxmuLmqlgAk2RW4FJgPvH+c+84CvjTk+xPAqVV1c5KdgFVJ/r6qbp+BnCVJkiRJktTDZmWbYVXdDywDTk6S0eKSHAvcA6wbcu93q+rm5vMjwB3A7mPMcV2Ss5MMJrkjyUFJPpvkW0n+dEjc55Ksarq9ljVjz2/idkmyXZLrk/zSFB9fkiRJkiRJ02TWDoCvqruTDAC7At8ffj3JPOB04HXAaSPNkWQR8BLgxnGW+3FVtZotiZ8HXgY8CNyV5OyqegD47ap6MMlc4KYkn6mq7yQ5E/g48A3g9qr68gh5LKNdnGNg/sIOnl6SJEmSJEnToZcOgD8DOLuqNo10sSl2fQZ4V1X9cJy5rmz+Xgusa7q7HgfuBvZsrp2SZA3w9WZsH4Cq+gTt7ZAnMkpRrapWVlWrqloDOy7o9PkkSZIkSZI0RbPWmZVkb2ALcP8oIYcAxyf5ILAz8GSSx6rq3CTb0y5kXVJVn+1gucebv58c8nnr92clOQJ4LfCKqno0yXXAnCbPHYE9mvh5wCMdPaAkSZIkSZJm3KwUs5IsBM4Dzq2qGimmqg4fEn8GsKkpZAW4ALijqs6appQWAA81hazFwMuHXDsTuAT4DnA+cMw0rSlJkiRJkqQpmsli1twkq4Htab+R8GLabyqcqMOAtwFrm/kA/qiqvjiF3K4BTkxyB/BN2lsNSfIq4CDgsKrakuTXkryjqi4cbaL9d1/A4IqlU0hFkiRJkiRJncoojVLqUKvVqsHBwW6nIUmSJEmStM1IsqqqWiNd66UD4CVJkiRJkqQxzdoB8FslOYr2uVRD3VNVx01wno/S3oI41DljbQmUJEmSJElSf5v1YlZVXQtcOw3znDQN6UiSJEmSJKmPuM1QkiRJkiRJfcNiliRJkiRJkvqGxSxJkiRJkiT1DYtZkiRJkiRJ6huzfgD8tmbtho0sWn51t9OQRrR+xdJupyBJkiRJ0rSyM0uSJEmSJEl9oyeKWUm2JFmdZE2Sm5McOkbsq5vYrX8eS3LsGPHXJWlNIqdjk7x4ovdJkiRJkiRp5vTKNsPNVbUEIMlRwJ8DrxopsKq+CmyN/Tng28CXZyCnY4GrgNtnYG5JkiRJkiRNQk90Zg0zH3iow9jjgS9V1aOdBCf5eJLBJOuSfGDI+Ioktye5NcmHm86wNwAfarq/XjDhp5AkSZIkSdK065XOrLlJVgNzgN2AIzu87y3AWRNY531V9WCSAeArSQ4ANgDHAYurqpLsXFUPJ7kSuKqqLh8+SZJlwDKAgfkLJ7C8JEmSJEmSpqJXOrM2V9WSqloMHA18MknGuiHJbsD+wLUTWOdNSW4GbgH2BV4MbAQeAy5I8kZg3C6vqlpZVa2qag3suGACy0uSJEmSJGkqeqWY9ZSq+hqwCzBey9ObgCuq6iedzJtkL+A04DVVdQBwNTCnqp4ADgYuB44Brpls7pIkSZIkSZpZPVfMSrIYGAAeGCf0N4C/ncDU84EfARuTPBd4fbPePGBBVX0ReDdwYBP/CLDTBOaXJEmSJEnSDOu1M7MAAry9qraMFpxkEbAn8A+dLlBVa5LcAtwJ3Avc0FzaCfh8kjnN2u9pxj8FnJ/kFOD4qrprpHn3330BgyuWdpqGJEmSJEmSpqAnillVNTDB+PXA7h3GHjHk8wmjhB08wn030D5TS5IkSZIkST2i57YZSpIkSZIkSaPpic6skSTZH7h42PDjVXXIKPFXAHsNGz69qibytkNJkiRJkiT1sJ4tZlXVWmDJBOKPm7lsJEmSJEmS1AvcZihJkiRJkqS+YTFLkiRJkiRJfcNiliRJkiRJkvqGxSxJkiRJkiT1jZ49AL5frN2wkUXLr+52GtrGrV+xtNspSJIkSZLUE+zMkiRJkiRJUt+YtWJWki1JVidZl2RNklOTjLt+kucl2ZTktGHjA0luSXLVOPdfl6Q1iXyPTfLiid4nSZIkSZKkmTObnVmbq2pJVe0LvA54PfD+Du47C/jSCOPvBO6YxvyGOxawmCVJkiRJktRDurLNsKruB5YBJyfJaHFJjgXuAdYNG98DWAp8YiLrJvl4ksGmO+wDQ8ZXJLk9ya1JPpzkUOANwIeabrIXTGQdSZIkSZIkzYyuHQBfVXcnGQB2Bb4//HqSecDptLu4Tht2+S+A/wbsNMFl31dVDzbrfiXJAcAG4DhgcVVVkp2r6uEkVwJXVdXlI+S2jHYxjoH5CyeYgiRJkiRJkiarlw+APwM4u6o2DR1Mcgxwf1WtmsScb0pyM3ALsC/tbYQbgceAC5K8EXh0vEmqamVVtaqqNbDjgkmkIUmSJEmSpMnoWmdWkr2BLcD9o4QcAhyf5IPAzsCTSR4DdgfekOSXgTnA/CR/U1VvHWe9vWh3eB1UVQ8luQiYU1VPJDkYeA1wPHAycOSUH1CSJEmSJEnTrivFrCQLgfOAc6uqRoqpqsOHxJ8BbKqqc5uh9zbjRwCnjVfIaswHfgRsTPJc2gfQX9dsZ9yxqr6Y5Abg7ib+ESa+jVGSJEmSJEkzaDaLWXOTrAa2B54ALqb9psJZUVVrktwC3AncC9zQXNoJ+HySOUCA9zTjnwLOT3IKcHxV3TXSvPvvvoDBFUtnNnlJkiRJkiQBkFEao9ShVqtVg4OD3U5DkiRJkiRpm5FkVVW1RrrWywfAS5IkSZIkST+lawfAb5XkKODMYcP3VNVxE5znCmCvYcOnV9W1U8lPkiRJkiRJvaPrxaym2DTlgtNEi1+SJEmSJEnqP24zlCRJkiRJUt+wmCVJkiRJkqS+YTFLkiRJkiRJfcNiliRJkiRJkvpG1w+A73drN2xk0fKru52GtnHrVyztdgqSJEmSJPUEO7MkSZIkSZLUN2asmJVkS5LVSdYlWZPk1CTjrpfkeUk2JTltyNj6JGub+QZnKmdJkiRJkiT1tpncZri5qpYAJNkVuBSYD7x/nPvOAr40wvirq+oH05qhJEmSJEmS+sqsbDOsqvuBZcDJSTJaXJJjgXuAdZNdK8l1Sc5OMpjkjiQHJflskm8l+dMhcZ9LsqrpHFvWjD2/idslyXZJrk/yS5PNRZIkSZIkSdNr1g6Ar6q7kwwAuwLfH349yTzgdOB1wGnDbwe+nKSA/1NVK8dZ7sdV1UryTuDzwMuAB4G7kpxdVQ8Av11VDyaZC9yU5DNV9Z0kZwIfB74B3F5VXx4h12W0i3MMzF/Y8X8DSZIkSZIkTU0vvc3wDODsqto0QvPWL1bVhma74t8nubOq/nGMua5s/l4LrKuq7wIkuRvYE3gAOCXJcU3cnsA+wANV9Ykkvw6cCCwZafKmmLYSYIfd9qmJPaYkSZIkSZIma9aKWUn2BrYA948ScghwfJIPAjsDTyZ5rKrOraoN0N6umOQK4GBgrGLW483fTw75vPX7s5IcAbwWeEVVPZrkOmBOk+eOwB5N/DzgkQk8piRJkiRJkmbQrBSzkiwEzgPOraoRO5mq6vAh8WcAm6rq3CQ/A2xXVY80n38J+JMpprQAeKgpZC0GXj7k2pnAJcB3gPOBY6a4liRJkiRJkqbJTBaz5iZZDWwPPAFcTPtNhRP1XOCKZuvhs4BLq+qaKeZ2DXBikjuAbwJfB0jyKuAg4LCq2pLk15K8o6ounOJ6kiRJkiRJmgYZpVFKHWq1WjU4ONjtNCRJkiRJkrYZSVZVVWuka9vNdjKSJEmSJEnSZM362wyTHEX7XKqh7qmq40aKH2OejwKHDRs+xy2BkiRJkiRJ265ZL2ZV1bXAtdMwz0nTkI4kSZIkSZL6iNsMJUmSJEmS1DcsZkmSJEmSJKlvWMySJEmSJElS37CYJUmSJEmSpL5hMUuSJEmSJEl9Y9bfZritWbthI4uWX93tNLSNW79iabdTkCRJkiSpJ8x6Z1aS9yVZl+TWJKuTHDJK3CVJvpnktiR/lWT7ZvxXh9w7mOQXx1hrUZLbJpnnH03mPkmSJEmSJM2cWS1mJXkFcAzw0qo6AHgtcO8o4ZcAi4H9gbnA7zbjXwEOrKolwG8Dn5ihdC1mSZIkSZIk9ZjZ3ma4G/CDqnocoKp+MFpgVX1x6+ck3wD2aMY3DQn7GaA6WTjJIuDi5h6Ak6vqn5PsBlwGzKf93+O/AkuBuUlWA+uq6jc7WUOSJEmSJEkza7a3GX4Z2DPJvyT5WJJXjXdDs73wbcA1Q8aOS3IncDXt7qxO3A+8rqpeCrwZ+Mtm/D8D1zadXgcCq6tqObC5qpaMVMhKsqzZ4ji45dGNHS4vSZIkSZKkqZrVYlbTVfUyYBnw78BlSU4Y57aPAf9YVdcPmeeKqloMHAv8zw6X3x44P8la4NPAi5vxm4B3JDkD2L+qHungOVZWVauqWgM7LuhweUmSJEmSJE3VrB8AX1Vbquq6qno/cDLwa6PFJnk/sBB4zyhz/SOwd5JdOlj63cD3aXdftYBnD5njlcAG4KIkvzWBx5EkSZIkSdIsmu0D4F+UZJ8hQ0uA74wS+7vAUcBvVNWTQ8ZfmCTN55cCOwAPdLD8AuC7zVxvAwaaOZ4PfL+qzqd9mPxLm/ifbH2DoiRJkiRJknrDbB8APw/4SJKdgSeAb9PecjiS82gXur7W1K4+W1V/QruT67eS/ATYDLy5qjo5BP5jwGeazqtrgB8140cAf9jMtwnY2pm1Erg1yc0eAC9JkiRJktQb0lkdSKNptVo1ODjY7TQkSZIkSZK2GUlWVVVrpGuzfmaWJEmSJEmSNFmzvc3waZJcAew1bPj0qrp2AnPsD1w8bPjxqjpkqvlJkiRJkiSpd3S9mFVVx03DHGtpHyYvSZIkSZKkbZjbDCVJkiRJktQ3LGZJkiRJkiSpb1jMkiRJkiRJUt+wmCVJkiRJkqS+YTFLkiRJkiRJfaPrbzPsd2s3bGTR8qu7nYb6xPoVS7udgiRJkiRJfa0rnVlJtiRZnWRNkpuTHDpG7PObmNVJ1iU5cTZzlSRJkiRJUu/oVmfW5qpaApDkKODPgVeNEvtd4BVV9XiSecBtSa6sqn+bnVQlSZIkSZLUK3rhzKz5wEOjXayqH1fV483XHRgn5ySbknyo6eL6v0kOTnJdkruTvKGJWZTk+qbj66nOsCTHJflK2nZL8i9J/r9pek5JkiRJkiRNUbc6s+YmWQ3MAXYDjhwrOMmewNXAC4E/HKcr62eA/1dVf5jkCuBPgdcBLwb+GrgSuB94XVU9lmQf4G+BVlVdkeTXgJOAo4H3V9X3RshnGbAMYGD+ws6fWpIkSZIkSVPSC9sMXwF8Msl+VVUjBVfVvcABSX4e+FySy6vq+6PM/WPgmubzWuDxqvpJkrXAomZ8e+DcJEuALcB/GnL/HwC3AV+vqr8dJZ+VwEqAHXbbZ8ScJUmSJEmSNP26vs2wqr4G7AKM2+LUdGTdBhw+RthPhhTFngQeb+59kv8o3r0b+D5wINACnj3k/j2a+56bpOv/fSRJkiRJkvQful6sSbIYGAAeGOX6HknmNp9/FvhF4JtTXHYB8N2mwPW2Zn2SPAv4K+A3gDuA90xxHUmSJEmSJE2jbp+ZBRDg7VW1ZZTYXwD+d5JqYj9cVWunuP7HgM8k+S3aWxJ/1Iz/EXB9Vf1TkjXATUmurqo7prieJEmSJEmSpkFGOaZKHWq1WjU4ONjtNCRJkiRJkrYZSVZVVWuka13fZihJkiRJkiR1qlvbDJ8myf7AxcOGH6+qQ0aJvxHYYdjw26ZhC6IkSZIkSZJ6VM8Us5oi1JIJxI9Y5JIkSZIkSdK2y22GkiRJkiRJ6hsWsyRJkiRJktQ3LGZJkiRJkiSpb1jMkiRJkiRJUt+wmCVJkiRJkqS+0TNvM+xXazdsZNHyq7udhvrE+hVLu52CJEmSJEl9rSc6s5JsSbI6yW1JvpBk5zFin5/k5iZ+XZITZzFVSZIkSZIkdVFPFLOAzVW1pKr2Ax4EThoj9rvAK6pqCXAIsDzJz89CjpIkSZIkSeqyXilmDfU1YPfRLlbVj6vq8ebrDozzDEk+nmSw6eL6QDN2dJJPD4k5IslVzeffSfIvSb6R5Pwk5075iSRJkiRJkjQteqqYlWQAeA1w5Thxeya5FbgXOLOq/m2M8PdVVQs4AHhVkgOA/wsckuRnmpg3A59qOrz+B/By4DBg8SjrL2sKZINbHt04gSeUJEmSJEnSVPRKMWtuktXA94DnAn8/VnBV3VtVBwAvBN6e5LljhL8pyc3ALcC+wIur6gngGuBXkjwLWAp8HjgY+IeqerCqfgJ8eqQJq2plVbWqqjWw44IJPagkSZIkSZImr1eKWZubM7CeD4Sxz8x6StORdRtw+EjXk+wFnAa8pil+XQ3MaS5/CngTcCQwWFWPTOUBJEmSJEmSNPN6pZgFQFU9CpwCnNp0TD1Nkj2SzG0+/yzwi8A3R5lyPvAjYGPTvfX6Idf+AXgp8Hu0C1sAN9Heivizzfq/NsVHkiRJkiRJ0jTqqWIWQFXdAtwK/MYoIb8A3JhkDe2C1Ierau0oc62hvb3wTuBS4IYh17YAV9EucF3VjG0A/hfwjSZ2PeChWJIkSZIkST0iVdXtHHpKknlVtanpzLoC+KuqumK0+FarVYODg7OXoCRJkiRJ0jYuyarmhX5P03OdWT3gjOYw+tuAe4DPdTUbSZIkSZIkPWXEc6l6QZL9gYuHDT9eVYeMEn8jsMOw4beNtgVxNFV12kTiJUmSJEmSNHt6tpjVFKGWTCB+xCKXJEmSJEmSth1uM5QkSZIkSVLfsJglSZIkSZKkvmExS5IkSZIkSX3DYpYkSZIkSZL6hsUsSZIkSZIk9Y2efZthv1i7YSOLll/d7TQ0y9avWNrtFCRJkiRJekaalc6sJFuSrE6yLsmaJKcmGXftJM9LsinJaUPGdk5yeZI7k9yR5BUzm70kSZIkSZJ6xWx1Zm2uqiUASXYFLgXmA+8f576zgC8NGzsHuKaqjk/ybGDHac5VkiRJkiRJPWrWz8yqqvuBZcDJSTJaXJJjgXuAdUPGFgCvBC5o5vpxVT08xhy/l+SmphvsM0l2TLIgyXe2doYl+Zkk9ybZPslBSW5tusg+lOS26XhmSZIkSZIkTY+uHABfVXcDA8CuI11PMg84HfjAsEt7Af8OXJjkliSfSPIzYyz12ao6qKoOBO4AfqeqNgKrgVc1MccA11bVT4ALgf/SdJFtmdTDSZIkSZIkacb06tsMzwDOrqpNw8afBbwU+HhVvQT4EbB8jHn2S3J9krXAbwL7NuOXAW9uPr8FuCzJzsBOVfW1ZvzS0SZNsizJYJLBLY9unMBjSZIkSZIkaSq68jbDJHvT7ny6f5SQQ4Djk3wQ2Bl4MsljwOXAfVV1YxN3OWMXsy4Cjq2qNUlOAI5oxq8E/leSnwNeBvw/YKdO86+qlcBKgB1226c6vU+SJEmSJElTM+vFrCQLgfOAc6tqxEJQVR0+JP4MYFNVndt8vzfJi6rqm8BrgNvHWG4n4LtJtqfdmbWhmX9TkptoHyZ/VVVtAR5O8kiSQ5pi2Vum+qySJEmSJEmaXrNVzJqbZDWwPfAEcDHtNxVOxh8AlzRvMrwbeMcYsf8DuJH2OVs38tPdV5cBn+Y/urUAfgc4P8mTwD8A7iGUJEmSJEnqIRmlOeoZKcm8red0JVkO7FZV7xzrnlarVYODg7OSnyRJkiRJ0jNBklVV1RrpWlfOzOphS5O8l/Z/l+8AJ3Q3HUmSJEmSJA3V1WJWkqOAM4cN31NVx01wno8Chw0bPqeqLpzIPFV1Ge3th5IkSZIkSepBXS1mVdW1wLXTMM9J05COJEmSJEmSetx23U5AkiRJkiRJ6pTFLEmSJEmSJPUNi1mSJEmSJEnqGxazJEmSJEmS1DcsZkmSJEmSJKlvdPVthtuCtRs2smj51d1OQ7Ns/Yql3U5BkiRJkqRnJDuzJEmSJEmS1Dd6ppiVZEuS1UluS/LpJDuOEjcnyTeSrEmyLskHxpn3uiStSeRzbJIXT/Q+SZIkSZIkzZyeKWYBm6tqSVXtB/wYOHGUuMeBI6vqQGAJcHSSl89APscCFrMkSZIkSZJ6SC8Vs4a6HnjhSBeqbVPzdfvmT3UyaZKPJxkc3tGVZEWS25PcmuTDSQ4F3gB8qOkWe8HUHkeSJEmSJEnToecOgE/yLOD1wDVjxAwAq2gXvD5aVTd2OP37qurB5v6vJDkA2AAcByyuqkqyc1U9nORK4KqqunyE9ZcBywAG5i+cyONJkiRJkiRpCnqpM2tuktXAIPCvwAWjBVbVlqpaAuwBHJxkvw7XeFOSm4FbgH1pbyPcCDwGXJDkjcCj401SVSurqlVVrYEdF3S4tCRJkiRJkqaqlzqzNjcFqo41HVRfBY4GbhsrNslewGnAQVX1UJKLgDlV9USSg4HXAMcDJwNHTiJ/SZIkSZIkzbBe6szqSJKFSXZuPs8FXgfc2cGt84EfARuTPJf2VkaSzAMWVNUXgXcDBzbxjwA7TW/2kiRJkiRJmope6szq1G7AXzfnXm0H/F1VXTXeTVW1JskttAtf9wI3NJd2Aj6fZA4Q4D3N+KeA85OcAhxfVXeNNO/+uy9gcMXSKT2QJEmSJEnSWH7yk59w33338dhjj3U7lWk1Z84c9thjD7bffvuO70lVRy8C1CharVYNDg52Ow1JkiRJkrQNu+eee9hpp514znOeQ5JupzMtqooHHniARx55hL322uunriVZVVWtke7ru22GkiRJkiRJzzSPPfbYNlXIAkjCc57znAl3m/XsNsMkzwG+MsKl11TVAyPEXwHsNWz49Kq6dibykyRJkiRJmk3bUiFrq8k8U88Ws5qC1ZIJxB83c9lIkiRJkiQ9sx166KH88z//c7fT6N1iliRJkiRJkka2aPnV0zrf+g5ebtcLhSzwzCxJkiRJkiR1YN68eQBcd911vOpVr+JXf/VX2XvvvVm+fDmXXHIJBx98MPvvvz933XUXACeccAInnngirVaL//Sf/hNXXXXVtORhZ5YkSZIkSZImZM2aNdxxxx383M/9HHvvvTe/+7u/yze+8Q3OOeccPvKRj/AXf/EXAKxfv55vfOMb3HXXXbz61a/m29/+NnPmzJnS2nZmSZIkSZIkaUIOOuggdtttN3bYYQde8IIX8Eu/9EsA7L///qxfv/6puDe96U1st9127LPPPuy9997ceeedU17bzqwpWrth47TvU9Xs6WRPsCRJkiRJ+mk77LDDU5+32267p75vt912PPHEE09dG/62wul4I6OdWZIkSZIkSZoRn/70p3nyySe56667uPvuu3nRi1405TlnrDMryRZgLbA98ATwSeDsqnpylPhFwB3AN5uhr1fVic21lwEXAXOBLwLvrKqaqdwlSZIkSZI0dc973vM4+OCD+eEPf8h555035fOyYGa3GW6uqiUASXYFLgXmA+8f4567tt4zzMeB3wNupF3MOhr40nQmK0mSJEmS1C+6cWzOpk2bADjiiCM44ogjnhq/7rrrnvo8/NprX/tazjvvvGnNY1a2GVbV/cAy4ORMcHNkkt2A+VX19aYb65PAsWPEX5fk7CSDSe5IclCSzyb5VpI/HRL3uSSrkqxLsqwZe34Tt0uS7ZJcn+SXJvPMkiRJkiRJmn6zdgB8Vd2dZADYFfj+KGF7JbkF+CHw36vqemB34L4hMfc1Y2P5cVW1krwT+DzwMuBB4K4kZ1fVA8BvV9WDSeYCNyX5TFV9J8mZtDvBvgHcXlVfHj55U/xaBjAwf2Fn/wEkSZIkSZKeQS666KIZmbeX3mb4XeB5VfVAc0bW55LsO8m5rmz+Xgusq6rvAiS5G9gTeAA4JclxTdyewD7AA1X1iSS/DpwILBlp8qpaCawE2GG3fTy7S5IkSZIkaZbMWjEryd7AFuD+ka5X1ePA483nVUnuAv4TsAHYY0joHs3YWB5v/n5yyOet35+V5AjgtcArqurRJNcBc5o8dxyy3jzgkfGfTpIkSZIkaWZVFRM8vannTeb9frNyZlaShcB5wLmjvYUwycJmG+LWwtc+wN1NV9UPk7y8OW/rt2hvHZyKBcBDTSFrMfDyIdfOBC4B/hg4f4rrSJIkSZIkTdmcOXN44IEHJlX86VVVxQMPPDDhNxzOZGfW3CSrge2BJ4CLgbPGiH8l8CdJfkK7g+rEqnqwufb7wEXAXNpvMZzqmwyvAU5McgfwTeDrAEleBRwEHFZVW5L8WpJ3VNWFo020/+4LGOzCGwQkSZIkSdIzxx577MF9993Hv//7v3c7lWk1Z84c9thjj/EDh8i2VNHrhlarVYODg91OQ5IkSZIkaZuRZFVVtUa6NivbDCVJkiRJkqTpMOtvM0xyFO1zqYa6p6qOGyl+jHk+Chw2bPicsbYESpIkSZIkqb/NejGrqq4Frp2GeU6ahnQkSZIkSZLURzwza4qSPEL7EHmpE7sAP+h2Euob/l40Ef5eNBH+XtQpfyuaCH8vmgh/LxrP86tq4UgXZr0zaxv0zdEOJJOGSzLo70Wd8veiifD3oonw96JO+VvRRPh70UT4e9FUeAC8JEmSJEmS+obFLEmSJEmSJPUNi1lTt7LbCaiv+HvRRPh70UT4e9FE+HtRp/ytaCL8vWgi/L1o0jwAXpIkSZIkSX3DzixJkiRJkiT1DYtZU5Dk6CTfTPLtJMu7nY96S5K/SnJ/ktuGjP1ckr9P8q3m75/tZo7qDUn2TPLVJLcnWZfknc24vxc9TZI5Sb6RZE3ze/lAM75Xkhubf5MuS/Lsbueq3pFkIMktSa5qvvt70YiSrE+yNsnqJIPNmP8eaURJdk5yeZI7k9yR5BX+XjSSJC9q/ndl658fJnmXvxdNlsWsSUoyAHwUeD3wYuA3kry4u1mpx1wEHD1sbDnwlaraB/hK8116Aji1ql4MvBw4qfnfE38vGsnjwJFVdSCwBDg6ycuBM4Gzq+qFwEPA73QvRfWgdwJ3DPnu70VjeXVVLamqVvPdf480mnOAa6pqMXAg7f+d8feip6mqbzb/u7IEeBnwKHAF/l40SRazJu9g4NtVdXdV/Rj4FPCrXc5JPaSq/hF4cNjwrwJ/3Xz+a+DY2cxJvamqvltVNzefH6H9fwjujr8XjaDaNjVft2/+FHAkcHkz7u9FT0myB7AU+ETzPfh70cT475GeJskC4JXABQBV9eOqehh/Lxrfa4C7quo7+HvRJFnMmrzdgXuHfL+vGZPG8tyq+m7z+XvAc7uZjHpPkkXAS4Ab8feiUTRbxlYD9wN/D9wFPFxVTzQh/pukof4C+G/Ak8335+DvRaMr4MtJViVZ1oz575FGshfw78CFzTbmTyT5Gfy9aHxvAf62+ezvRZNiMUvqkmq/StTXieopSeYBnwHeVVU/HHrN34uGqqotTZv+HrQ7hRd3NyP1qiTHAPdX1apu56K+8YtV9VLaR2mclOSVQy/675GGeBbwUuDjVfUS4EcM2yLm70XDNWc0vgH49PBr/l40ERazJm8DsOeQ73s0Y9JYvp9kN4Dm7/u7nI96RJLtaReyLqmqzzbD/l40pmY7x1eBVwA7J3lWc8l/k7TVYcAbkqynfSTCkbTPuPH3ohFV1Ybm7/tpn2dzMP57pJHdB9xXVTc23y+nXdzy96KxvB64uaq+33z396JJsZg1eTcB+zRvA3o27VbJK7uck3rflcDbm89vBz7fxVzUI5rzay4A7qiqs4Zc8veip0myMMnOzee5wOton7P2VeD4JszfiwCoqvdW1R5VtYj2/63y/6rqN/H3ohEk+ZkkO239DPwScBv+e6QRVNX3gHuTvKgZeg1wO/5eNLbf4D+2GIK/F01S2p18mowkv0z7HIoB4K+q6s+6m5F6SZK/BY4AdgG+D7wf+Bzwd8DzgO8Ab6qq4YfE6xkmyS8C1wNr+Y8zbf6I9rlZ/l70U5IcQPuA1AHa/0+pv6uqP0myN+3Om58DbgHeWlWPdy9T9ZokRwCnVdUx/l40kuZ3cUXz9VnApVX1Z0meg/8eaQRJltB+ucSzgbuBd9D824S/Fw3TFMn/Fdi7qjY2Y/7viybFYpYkSZIkSZL6htsMJUmSJEmS1DcsZkmSJEmSJKlvWMySJEmSJElS37CYJUmSJEmSpL5hMUuSJEmSJEl9w2KWJEmSJEmS+obFLEmSJEmSJPUNi1mSJEmSJEnqG/8/Bkfq0YINPYMAAAAASUVORK5CYII=\n",
|
|
"text/plain": [
|
|
"<Figure size 1440x720 with 1 Axes>"
|
|
]
|
|
},
|
|
"metadata": {
|
|
"needs_background": "light"
|
|
},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"# feature importance for top 30 features\n",
|
|
"fea_imp = pd.DataFrame({'imp':model.feature_importances_, 'col': features})\n",
|
|
"fea_imp = fea_imp.sort_values(['imp', 'col'], ascending=[True, False]).iloc[-30:]\n",
|
|
"_ = fea_imp.plot(kind='barh', x='col', y='imp', figsize=(20, 10))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"## Evaluation"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 22,
|
|
"metadata": {
|
|
"tags": [
|
|
"block:evaluation_result",
|
|
"prev:modelling"
|
|
]
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"model = joblib.load('lgb.jl')"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 25,
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"binary_logloss = model.booster_.best_score.get('valid_0').get('binary_logloss')"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 26,
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"amex_metric_score = model.booster_.best_score.get('valid_0').get('amex_metric_score')"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 27,
|
|
"metadata": {
|
|
"tags": [
|
|
"pipeline-metrics"
|
|
]
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"0.24605493989573005\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"print(binary_logloss)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 28,
|
|
"metadata": {
|
|
"tags": [
|
|
"pipeline-metrics"
|
|
]
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"0.7625788091718922\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"print(amex_metric_score)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"source": [
|
|
"## Submission"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"tags": [
|
|
"skip"
|
|
]
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"sub = pd.DataFrame({'customer_ID': test.index,\n",
|
|
" 'prediction': np.mean(y_pred_list, axis=0)})\n",
|
|
"sub.to_csv('submission.csv', index=False)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"tags": [
|
|
"skip"
|
|
]
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"sub"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"kubeflow_notebook": {
|
|
"autosnapshot": true,
|
|
"experiment": {
|
|
"id": "2efb8e27-3b2e-439b-a53c-b1f9d7b94cfc",
|
|
"name": "g-research-crypto-forecasting"
|
|
},
|
|
"experiment_name": "g-research-crypto-forecasting",
|
|
"katib_metadata": {
|
|
"algorithm": {
|
|
"algorithmName": "grid"
|
|
},
|
|
"maxFailedTrialCount": 3,
|
|
"maxTrialCount": 12,
|
|
"objective": {
|
|
"objectiveMetricName": "",
|
|
"type": "minimize"
|
|
},
|
|
"parallelTrialCount": 3,
|
|
"parameters": []
|
|
},
|
|
"katib_run": false,
|
|
"pipeline_description": "forecasting short term returns in 14 popular cryptocurrencies.",
|
|
"pipeline_name": "g-research-crypto-forecasting-pipeline",
|
|
"snapshot_volumes": true,
|
|
"steps_defaults": [
|
|
"label:access-ml-pipeline:true",
|
|
"label:kaggle-secret:true",
|
|
"label:access-rok:true"
|
|
],
|
|
"volume_access_mode": "rwm",
|
|
"volumes": [
|
|
{
|
|
"annotations": [],
|
|
"mount_point": "/home/jovyan",
|
|
"name": "test-workspace-qtvmt",
|
|
"size": 32,
|
|
"size_type": "Gi",
|
|
"snapshot": false,
|
|
"type": "clone"
|
|
}
|
|
]
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.6.9"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 4
|
|
}
|