{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "86eaceaa",
   "metadata": {},
   "source": [
    "# 自动化特征工程\n",
    "\n",
    "在`menu_feature_enginer.ipynb`中，我们手动进行了特征工程。\n",
    "\n",
    "这个notebook中，我们通过`featuretools`实现自动化这个过程。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2750562b",
   "metadata": {},
   "source": [
    "关于featuretools说明：\n",
    "\n",
    "- woodwork 为字段赋予逻辑类型和语义标签：\n",
    "  - 比如int64这些，可以分为邮政编码、手机号、等等\n",
    "- featuretools 利用woodwork生成的table再去 自动特征\n",
    "\n",
    "[featuretools](https://featuretools.alteryx.com/en/stable/)\n",
    "\n",
    "[woodwork](https://woodwork.alteryx.com/en/stable/) \n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c03cdd2",
   "metadata": {},
   "source": [
    "文中目前并没有用到woodwork赋值语义，目前都是让他自动推断的。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "046a0c70",
   "metadata": {},
   "source": [
    "## 导入"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "832d1182",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "c:\\Users\\63517\\miniconda3\\envs\\data-analysis\\lib\\site-packages\\woodwork\\__init__.py:2: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.\n",
      "  import pkg_resources\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ft 1.31.0\n",
      "np 1.26.4\n",
      "pd 2.3.1\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "import featuretools as ft\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "\n",
    "import woodwork as ww\n",
    "import gc\n",
    "gc.enable()\n",
    "\n",
    "print('ft', ft.__version__)\n",
    "print('np', np.__version__)\n",
    "print('pd', pd.__version__)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "744ddffc",
   "metadata": {},
   "outputs": [],
   "source": [
    "application_train = pd.read_csv('data/application_train.csv')\n",
    "application_test = pd.read_csv('data/application_test.csv')\n",
    "bureau = pd.read_csv('data/bureau.csv')\n",
    "bureau_balance = pd.read_csv('data/bureau_balance.csv')\n",
    "credit_card_balance = pd.read_csv('data/credit_card_balance.csv')\n",
    "installments_payments = pd.read_csv('data/installments_payments.csv')\n",
    "previous_application = pd.read_csv('data/previous_application.csv')\n",
    "pos_cash_balance = pd.read_csv('data/POS_CASH_balance.csv')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "285c4220",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(307511, 123) (48744, 123)\n"
     ]
    }
   ],
   "source": [
    "application_train['set'] = 'train'\n",
    "application_test['set'] = 'test'\n",
    "application_test['TARGET'] = np.nan\n",
    "print(application_train.shape, application_test.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "43c71f95",
   "metadata": {},
   "outputs": [],
   "source": [
    "app = pd.concat([application_train, application_test], ignore_index=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f0a16e35",
   "metadata": {},
   "source": [
    "我们让测试集和训练集一起进行特征工程，这也是符合的。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "56cb4541",
   "metadata": {},
   "source": [
    "有一些字段不应该参与特征工程，我们保留下来，最后合并"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "ff47b843",
   "metadata": {},
   "outputs": [],
   "source": [
    "app_target = app[['SK_ID_CURR', 'TARGET']]\n",
    "app_set = app[['SK_ID_CURR', 'set']]\n",
    "app_for_es = app.drop(columns=['set', 'TARGET']) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "cde4c548",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(356255, 121)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "app_for_es.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e5e8d9f2",
   "metadata": {},
   "source": [
    "## 创建一个entityset\n",
    "- entity表示一个表；他必须有主键唯一的列。比如`sk_id_curr`\n",
    "- entity set 表示一组表及其关系\n",
    "\n",
    "`add_dataframe` 会将 df转为 woodwork的table"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "153a2978",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([100002, 100003, 100004, ..., 456223, 456224, 456250], dtype=int64)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "app_for_es['SK_ID_CURR'].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "32964789",
   "metadata": {},
   "outputs": [],
   "source": [
    "es = ft.EntitySet(id='clients')\n",
    "\n",
    "# 有主键唯一列\n",
    "es = es.add_dataframe(dataframe_name=\"app\", dataframe=app_for_es, index=\"SK_ID_CURR\")\n",
    "es = es.add_dataframe(dataframe_name=\"bureau\", dataframe=bureau, index=\"SK_ID_BUREAU\")\n",
    "es = es.add_dataframe(dataframe_name=\"previous\", dataframe=previous_application, index=\"SK_ID_PREV\")\n",
    "\n",
    "# 没有主键唯一的列，需要make_index, 创建一列主键\n",
    "es = es.add_dataframe(dataframe_name=\"bureau_balance\", dataframe=bureau_balance, make_index = True, index = 'bureaubalance_index')\n",
    "es = es.add_dataframe(dataframe_name=\"credit\", dataframe=credit_card_balance, make_index=True, index='credit_index')\n",
    "es = es.add_dataframe(dataframe_name=\"installments\", dataframe=installments_payments, make_index=True, index='installments_index')\n",
    "es = es.add_dataframe(dataframe_name=\"cash\", dataframe=pos_cash_balance,make_index=True, index=\"cash_index\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "8a301cae",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Entityset: clients\n",
       "  DataFrames:\n",
       "    app [Rows: 356255, Columns: 121]\n",
       "    bureau [Rows: 1716428, Columns: 17]\n",
       "    previous [Rows: 1670214, Columns: 37]\n",
       "    bureau_balance [Rows: 27299925, Columns: 4]\n",
       "    credit [Rows: 3840312, Columns: 24]\n",
       "    installments [Rows: 13605401, Columns: 9]\n",
       "    cash [Rows: 10001358, Columns: 9]\n",
       "  Relationships:\n",
       "    No relationships"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "es"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c59cefc8",
   "metadata": {},
   "source": [
    "## 为entityset添加replationship\n",
    "需要为这几个表建立关系：一对多等， 类似父子"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "174ca453",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 父亲dfname, 父亲列名； 字dfname, 子列名\n",
    "es = es.add_relationship(\"app\", \"SK_ID_CURR\", \"bureau\", \"SK_ID_CURR\")\n",
    "es = es.add_relationship(\"bureau\", \"SK_ID_BUREAU\", \"bureau_balance\", \"SK_ID_BUREAU\")\n",
    "\n",
    "es = es.add_relationship(\"app\", \"SK_ID_CURR\", \"previous\", \"SK_ID_CURR\")\n",
    "es = es.add_relationship(\"previous\", \"SK_ID_PREV\", \"cash\", \"SK_ID_PREV\")\n",
    "es = es.add_relationship(\"previous\", \"SK_ID_PREV\", \"installments\", \"SK_ID_PREV\")\n",
    "es = es.add_relationship(\"previous\", \"SK_ID_PREV\", \"credit\", \"SK_ID_PREV\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "c1a06102",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Entityset: clients\n",
       "  DataFrames:\n",
       "    app [Rows: 356255, Columns: 121]\n",
       "    bureau [Rows: 1716428, Columns: 17]\n",
       "    previous [Rows: 1670214, Columns: 37]\n",
       "    bureau_balance [Rows: 27299925, Columns: 4]\n",
       "    credit [Rows: 3840312, Columns: 24]\n",
       "    installments [Rows: 13605401, Columns: 9]\n",
       "    cash [Rows: 10001358, Columns: 9]\n",
       "  Relationships:\n",
       "    bureau.SK_ID_CURR -> app.SK_ID_CURR\n",
       "    bureau_balance.SK_ID_BUREAU -> bureau.SK_ID_BUREAU\n",
       "    previous.SK_ID_CURR -> app.SK_ID_CURR\n",
       "    cash.SK_ID_PREV -> previous.SK_ID_PREV\n",
       "    installments.SK_ID_PREV -> previous.SK_ID_PREV\n",
       "    credit.SK_ID_PREV -> previous.SK_ID_PREV"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "es"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b7608093",
   "metadata": {},
   "source": [
    "我们现在就准备好了entityset\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0d59f505",
   "metadata": {},
   "source": [
    "## feature primitive\n",
    "用于entityset的操作，代表了之前手动特征的操作。分为两类：\n",
    "- `agg`: 输出一个值\n",
    "- `transform`：输出向量"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "dbb7e5dc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>name</th>\n",
       "      <th>type</th>\n",
       "      <th>description</th>\n",
       "      <th>valid_inputs</th>\n",
       "      <th>return_type</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>max_min_delta</td>\n",
       "      <td>aggregation</td>\n",
       "      <td>Determines the difference between the max and ...</td>\n",
       "      <td>&lt;ColumnSchema (Semantic Tags = ['numeric'])&gt;</td>\n",
       "      <td>&lt;ColumnSchema (Semantic Tags = ['numeric'])&gt;</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>max_consecutive_negatives</td>\n",
       "      <td>aggregation</td>\n",
       "      <td>Determines the maximum number of consecutive n...</td>\n",
       "      <td>&lt;ColumnSchema (Logical Type = Double)&gt;, &lt;Colum...</td>\n",
       "      <td>&lt;ColumnSchema (Logical Type = Integer) (Semant...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>any</td>\n",
       "      <td>aggregation</td>\n",
       "      <td>Determines if any value is 'True' in a list.</td>\n",
       "      <td>&lt;ColumnSchema (Logical Type = BooleanNullable)...</td>\n",
       "      <td>&lt;ColumnSchema (Logical Type = Boolean)&gt;</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>count_inside_nth_std</td>\n",
       "      <td>aggregation</td>\n",
       "      <td>Determines the count of observations that lie ...</td>\n",
       "      <td>&lt;ColumnSchema (Semantic Tags = ['numeric'])&gt;</td>\n",
       "      <td>&lt;ColumnSchema (Logical Type = Integer) (Semant...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>trend</td>\n",
       "      <td>aggregation</td>\n",
       "      <td>Calculates the trend of a column over time.</td>\n",
       "      <td>&lt;ColumnSchema (Semantic Tags = ['numeric'])&gt;, ...</td>\n",
       "      <td>&lt;ColumnSchema (Semantic Tags = ['numeric'])&gt;</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                        name         type  \\\n",
       "0              max_min_delta  aggregation   \n",
       "1  max_consecutive_negatives  aggregation   \n",
       "2                        any  aggregation   \n",
       "3       count_inside_nth_std  aggregation   \n",
       "4                      trend  aggregation   \n",
       "\n",
       "                                         description  \\\n",
       "0  Determines the difference between the max and ...   \n",
       "1  Determines the maximum number of consecutive n...   \n",
       "2       Determines if any value is 'True' in a list.   \n",
       "3  Determines the count of observations that lie ...   \n",
       "4        Calculates the trend of a column over time.   \n",
       "\n",
       "                                        valid_inputs  \\\n",
       "0       <ColumnSchema (Semantic Tags = ['numeric'])>   \n",
       "1  <ColumnSchema (Logical Type = Double)>, <Colum...   \n",
       "2  <ColumnSchema (Logical Type = BooleanNullable)...   \n",
       "3       <ColumnSchema (Semantic Tags = ['numeric'])>   \n",
       "4  <ColumnSchema (Semantic Tags = ['numeric'])>, ...   \n",
       "\n",
       "                                         return_type  \n",
       "0       <ColumnSchema (Semantic Tags = ['numeric'])>  \n",
       "1  <ColumnSchema (Logical Type = Integer) (Semant...  \n",
       "2            <ColumnSchema (Logical Type = Boolean)>  \n",
       "3  <ColumnSchema (Logical Type = Integer) (Semant...  \n",
       "4       <ColumnSchema (Semantic Tags = ['numeric'])>  "
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 内置原语\n",
    "ft.list_primitives().head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f416af3",
   "metadata": {},
   "source": [
    "```{warning} \n",
    "我们必须清晰，哪些原语作用哪些列！ 一些常见的。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "951bac5d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>name</th>\n",
       "      <th>type</th>\n",
       "      <th>description</th>\n",
       "      <th>valid_inputs</th>\n",
       "      <th>return_type</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>37</th>\n",
       "      <td>mean</td>\n",
       "      <td>aggregation</td>\n",
       "      <td>Computes the average for a list of values.</td>\n",
       "      <td>&lt;ColumnSchema (Semantic Tags = ['numeric'])&gt;</td>\n",
       "      <td>&lt;ColumnSchema (Semantic Tags = ['numeric'])&gt;</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    name         type                                 description  \\\n",
       "37  mean  aggregation  Computes the average for a list of values.   \n",
       "\n",
       "                                    valid_inputs  \\\n",
       "37  <ColumnSchema (Semantic Tags = ['numeric'])>   \n",
       "\n",
       "                                     return_type  \n",
       "37  <ColumnSchema (Semantic Tags = ['numeric'])>  "
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ft.list_primitives()[ft.list_primitives()['name'] == 'mean']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "999db041",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>name</th>\n",
       "      <th>type</th>\n",
       "      <th>description</th>\n",
       "      <th>valid_inputs</th>\n",
       "      <th>return_type</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>62</th>\n",
       "      <td>percent_true</td>\n",
       "      <td>aggregation</td>\n",
       "      <td>Determines the percent of `True` values.</td>\n",
       "      <td>&lt;ColumnSchema (Logical Type = BooleanNullable)...</td>\n",
       "      <td>&lt;ColumnSchema (Logical Type = Double) (Semanti...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            name         type                               description  \\\n",
       "62  percent_true  aggregation  Determines the percent of `True` values.   \n",
       "\n",
       "                                         valid_inputs  \\\n",
       "62  <ColumnSchema (Logical Type = BooleanNullable)...   \n",
       "\n",
       "                                          return_type  \n",
       "62  <ColumnSchema (Logical Type = Double) (Semanti...  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ft.list_primitives()[ft.list_primitives()['name'] == 'percent_true']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "fa0b77d4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>name</th>\n",
       "      <th>type</th>\n",
       "      <th>description</th>\n",
       "      <th>valid_inputs</th>\n",
       "      <th>return_type</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>58</th>\n",
       "      <td>mode</td>\n",
       "      <td>aggregation</td>\n",
       "      <td>Determines the most commonly repeated value.</td>\n",
       "      <td>&lt;ColumnSchema (Semantic Tags = ['category'])&gt;</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    name         type                                   description  \\\n",
       "58  mode  aggregation  Determines the most commonly repeated value.   \n",
       "\n",
       "                                     valid_inputs return_type  \n",
       "58  <ColumnSchema (Semantic Tags = ['category'])>        None  "
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ft.list_primitives()[ft.list_primitives()['name'] == 'mode']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "33f20c2c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>name</th>\n",
       "      <th>type</th>\n",
       "      <th>description</th>\n",
       "      <th>valid_inputs</th>\n",
       "      <th>return_type</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>count</td>\n",
       "      <td>aggregation</td>\n",
       "      <td>Determines the total number of values, excludi...</td>\n",
       "      <td>&lt;ColumnSchema (Semantic Tags = ['index'])&gt;</td>\n",
       "      <td>&lt;ColumnSchema (Logical Type = IntegerNullable)...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     name         type                                        description  \\\n",
       "31  count  aggregation  Determines the total number of values, excludi...   \n",
       "\n",
       "                                  valid_inputs  \\\n",
       "31  <ColumnSchema (Semantic Tags = ['index'])>   \n",
       "\n",
       "                                          return_type  \n",
       "31  <ColumnSchema (Logical Type = IntegerNullable)...  "
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ft.list_primitives()[ft.list_primitives()['name'] == 'count']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "5d09c839",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>name</th>\n",
       "      <th>type</th>\n",
       "      <th>description</th>\n",
       "      <th>valid_inputs</th>\n",
       "      <th>return_type</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>181</th>\n",
       "      <td>month</td>\n",
       "      <td>transform</td>\n",
       "      <td>Determines the month value of a datetime.</td>\n",
       "      <td>&lt;ColumnSchema (Logical Type = Datetime)&gt;</td>\n",
       "      <td>&lt;ColumnSchema (Logical Type = Ordinal: [1, 2, ...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      name       type                                description  \\\n",
       "181  month  transform  Determines the month value of a datetime.   \n",
       "\n",
       "                                 valid_inputs  \\\n",
       "181  <ColumnSchema (Logical Type = Datetime)>   \n",
       "\n",
       "                                           return_type  \n",
       "181  <ColumnSchema (Logical Type = Ordinal: [1, 2, ...  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ft.list_primitives()[ft.list_primitives()['name'] == 'month']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "2c42dc72",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>name</th>\n",
       "      <th>type</th>\n",
       "      <th>description</th>\n",
       "      <th>valid_inputs</th>\n",
       "      <th>return_type</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>std</td>\n",
       "      <td>aggregation</td>\n",
       "      <td>Computes the dispersion relative to the mean v...</td>\n",
       "      <td>&lt;ColumnSchema (Semantic Tags = ['numeric'])&gt;</td>\n",
       "      <td>&lt;ColumnSchema (Semantic Tags = ['numeric'])&gt;</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  name         type                                        description  \\\n",
       "2  std  aggregation  Computes the dispersion relative to the mean v...   \n",
       "\n",
       "                                   valid_inputs  \\\n",
       "2  <ColumnSchema (Semantic Tags = ['numeric'])>   \n",
       "\n",
       "                                    return_type  \n",
       "2  <ColumnSchema (Semantic Tags = ['numeric'])>  "
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ft.list_primitives()[ft.list_primitives()['name'] == 'std']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4b896447",
   "metadata": {},
   "source": [
    "## deep feature synthesis\n",
    "就是在多层表关系中，不断agg。 比如`MAX(previous(MEAN(installments.payment))))` 这对人来说很难理解的很多时候。\n",
    "\n",
    "这和`menu_feature_enginer.ipynb` 手动对banlance进行多层聚合到app上，是一样的。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "593fd255",
   "metadata": {},
   "source": [
    "我们不应该过度追求多的原语操作， 这回造成维度灾难，"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b43bef8f",
   "metadata": {},
   "source": [
    "默认的，我们不需要指定分开 categorical和numeric列，ft会自动识别类型，进行聚合，转换\n",
    "\n",
    "因此，这个过程也不涉及编码的操作。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f230f890",
   "metadata": {},
   "source": [
    "类别列：`mode` 众数， `num_unique` 一共有多少特征"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "cb0ed4bd",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: total: 28min 23s\n",
      "Wall time: 29min 6s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "default_agg_primitives = [\"count\", \"mean\", \"max\", 'sum', 'std', \"mode\", \"num_unique\"]\n",
    "default_trans_primitives =  [\"month\", \"weekday\"]\n",
    "\n",
    "# 返回特征矩阵； 特征\n",
    "feature_matrix, features = ft.dfs(\n",
    "    entityset = es,\n",
    "    target_dataframe_name = 'app', # 最后要关联到这个表，以这个为主\n",
    "    agg_primitives= default_agg_primitives,\n",
    "    trans_primitives=default_trans_primitives,\n",
    "    max_depth=2,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9de9db35",
   "metadata": {},
   "source": [
    "这一步是最大的耗时操作，经历了30min"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5888d570",
   "metadata": {},
   "source": [
    "- feature_matrix就是df， \n",
    "- features元素是feature对象，记录了生成过程。\n",
    "  - 通过 `.get_name()`获取特征名字"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d5f3b544",
   "metadata": {},
   "source": [
    "## 特征选择\n",
    "featuretools 也提供了一些维度进行特征选择：\n",
    "- 移除大量空的列：ft.selection.remove_highly_null_features\n",
    "- 移除只有单一值得特征：ft.selection.remove_single_value_features\n",
    "- 移除高度相关得特征：ft.selection.remove_highly_correlated_features\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "69f33a6e",
   "metadata": {},
   "outputs": [],
   "source": [
    "from featuretools import selection"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "7a166cac",
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_matrix, features = selection.remove_highly_null_features(feature_matrix, features)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "ddddf3ad",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(356255, 1216)\n"
     ]
    }
   ],
   "source": [
    "print(feature_matrix.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "2342d158",
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_matrix, features  = selection.remove_single_value_features(feature_matrix, features)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "aa68a10a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(356255, 1169)\n"
     ]
    }
   ],
   "source": [
    "print(feature_matrix.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "d2104892",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: total: 36min 12s\n",
      "Wall time: 35min 41s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "feature_matrix, features = selection.remove_highly_correlated_features(feature_matrix, features)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aceb2cfa",
   "metadata": {},
   "source": [
    "```{warning} 性能警告\n",
    "计算相关系数矩阵是一个极其**耗时**的操作。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4e649b2d",
   "metadata": {},
   "source": [
    "对于相关性分析的进一步认识：\n",
    "\n",
    "Pearson 相关系数仅在衡量**连续变量**之间的**线性关系**时有意义。对于非线性关系或离散特征，其结果可能产生误导。比如：我们不应该对onehot分类变量进行相关分析，它们之间存在严格的线性关系（例如：$type_a+ type_b = 1$），"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "85cd5363",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(356255, 589)\n"
     ]
    }
   ],
   "source": [
    "print(feature_matrix.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "16518814",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>NAME_CONTRACT_TYPE</th>\n",
       "      <th>CODE_GENDER</th>\n",
       "      <th>FLAG_OWN_CAR</th>\n",
       "      <th>FLAG_OWN_REALTY</th>\n",
       "      <th>CNT_CHILDREN</th>\n",
       "      <th>AMT_INCOME_TOTAL</th>\n",
       "      <th>AMT_CREDIT</th>\n",
       "      <th>AMT_ANNUITY</th>\n",
       "      <th>NAME_TYPE_SUITE</th>\n",
       "      <th>NAME_INCOME_TYPE</th>\n",
       "      <th>...</th>\n",
       "      <th>STD(credit.previous.NFLAG_INSURED_ON_APPROVAL)</th>\n",
       "      <th>STD(credit.previous.SELLERPLACE_AREA)</th>\n",
       "      <th>SUM(credit.previous.AMT_ANNUITY)</th>\n",
       "      <th>SUM(credit.previous.AMT_APPLICATION)</th>\n",
       "      <th>SUM(credit.previous.AMT_DOWN_PAYMENT)</th>\n",
       "      <th>SUM(credit.previous.DAYS_FIRST_DRAWING)</th>\n",
       "      <th>SUM(credit.previous.DAYS_LAST_DUE)</th>\n",
       "      <th>SUM(credit.previous.DAYS_TERMINATION)</th>\n",
       "      <th>SUM(credit.previous.RATE_DOWN_PAYMENT)</th>\n",
       "      <th>SUM(credit.previous.SELLERPLACE_AREA)</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SK_ID_CURR</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>100002</th>\n",
       "      <td>Cash loans</td>\n",
       "      <td>M</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>0</td>\n",
       "      <td>202500.0</td>\n",
       "      <td>406597.5</td>\n",
       "      <td>24700.5</td>\n",
       "      <td>Unaccompanied</td>\n",
       "      <td>Working</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100003</th>\n",
       "      <td>Cash loans</td>\n",
       "      <td>F</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>0</td>\n",
       "      <td>270000.0</td>\n",
       "      <td>1293502.5</td>\n",
       "      <td>35698.5</td>\n",
       "      <td>Family</td>\n",
       "      <td>State servant</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100004</th>\n",
       "      <td>Revolving loans</td>\n",
       "      <td>M</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>0</td>\n",
       "      <td>67500.0</td>\n",
       "      <td>135000.0</td>\n",
       "      <td>6750.0</td>\n",
       "      <td>Unaccompanied</td>\n",
       "      <td>Working</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100006</th>\n",
       "      <td>Cash loans</td>\n",
       "      <td>F</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>0</td>\n",
       "      <td>135000.0</td>\n",
       "      <td>312682.5</td>\n",
       "      <td>29686.5</td>\n",
       "      <td>Unaccompanied</td>\n",
       "      <td>Working</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>81000.0</td>\n",
       "      <td>1620000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2191458.0</td>\n",
       "      <td>2191458.0</td>\n",
       "      <td>2191458.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-6.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100007</th>\n",
       "      <td>Cash loans</td>\n",
       "      <td>M</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>0</td>\n",
       "      <td>121500.0</td>\n",
       "      <td>513000.0</td>\n",
       "      <td>21865.5</td>\n",
       "      <td>Unaccompanied</td>\n",
       "      <td>Working</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 589 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "           NAME_CONTRACT_TYPE CODE_GENDER  FLAG_OWN_CAR  FLAG_OWN_REALTY  \\\n",
       "SK_ID_CURR                                                                 \n",
       "100002             Cash loans           M         False             True   \n",
       "100003             Cash loans           F         False            False   \n",
       "100004        Revolving loans           M          True             True   \n",
       "100006             Cash loans           F         False             True   \n",
       "100007             Cash loans           M         False             True   \n",
       "\n",
       "            CNT_CHILDREN  AMT_INCOME_TOTAL  AMT_CREDIT  AMT_ANNUITY  \\\n",
       "SK_ID_CURR                                                            \n",
       "100002                 0          202500.0    406597.5      24700.5   \n",
       "100003                 0          270000.0   1293502.5      35698.5   \n",
       "100004                 0           67500.0    135000.0       6750.0   \n",
       "100006                 0          135000.0    312682.5      29686.5   \n",
       "100007                 0          121500.0    513000.0      21865.5   \n",
       "\n",
       "           NAME_TYPE_SUITE NAME_INCOME_TYPE  ...  \\\n",
       "SK_ID_CURR                                   ...   \n",
       "100002       Unaccompanied          Working  ...   \n",
       "100003              Family    State servant  ...   \n",
       "100004       Unaccompanied          Working  ...   \n",
       "100006       Unaccompanied          Working  ...   \n",
       "100007       Unaccompanied          Working  ...   \n",
       "\n",
       "           STD(credit.previous.NFLAG_INSURED_ON_APPROVAL)  \\\n",
       "SK_ID_CURR                                                  \n",
       "100002                                                NaN   \n",
       "100003                                                NaN   \n",
       "100004                                                NaN   \n",
       "100006                                                0.0   \n",
       "100007                                                NaN   \n",
       "\n",
       "           STD(credit.previous.SELLERPLACE_AREA)  \\\n",
       "SK_ID_CURR                                         \n",
       "100002                                       NaN   \n",
       "100003                                       NaN   \n",
       "100004                                       NaN   \n",
       "100006                                       0.0   \n",
       "100007                                       NaN   \n",
       "\n",
       "           SUM(credit.previous.AMT_ANNUITY)  \\\n",
       "SK_ID_CURR                                    \n",
       "100002                                  0.0   \n",
       "100003                                  0.0   \n",
       "100004                                  0.0   \n",
       "100006                              81000.0   \n",
       "100007                                  0.0   \n",
       "\n",
       "            SUM(credit.previous.AMT_APPLICATION)  \\\n",
       "SK_ID_CURR                                         \n",
       "100002                                       0.0   \n",
       "100003                                       0.0   \n",
       "100004                                       0.0   \n",
       "100006                                 1620000.0   \n",
       "100007                                       0.0   \n",
       "\n",
       "            SUM(credit.previous.AMT_DOWN_PAYMENT)  \\\n",
       "SK_ID_CURR                                          \n",
       "100002                                        0.0   \n",
       "100003                                        0.0   \n",
       "100004                                        0.0   \n",
       "100006                                        0.0   \n",
       "100007                                        0.0   \n",
       "\n",
       "            SUM(credit.previous.DAYS_FIRST_DRAWING)  \\\n",
       "SK_ID_CURR                                            \n",
       "100002                                          0.0   \n",
       "100003                                          0.0   \n",
       "100004                                          0.0   \n",
       "100006                                    2191458.0   \n",
       "100007                                          0.0   \n",
       "\n",
       "            SUM(credit.previous.DAYS_LAST_DUE)  \\\n",
       "SK_ID_CURR                                       \n",
       "100002                                     0.0   \n",
       "100003                                     0.0   \n",
       "100004                                     0.0   \n",
       "100006                               2191458.0   \n",
       "100007                                     0.0   \n",
       "\n",
       "            SUM(credit.previous.DAYS_TERMINATION)  \\\n",
       "SK_ID_CURR                                          \n",
       "100002                                        0.0   \n",
       "100003                                        0.0   \n",
       "100004                                        0.0   \n",
       "100006                                  2191458.0   \n",
       "100007                                        0.0   \n",
       "\n",
       "            SUM(credit.previous.RATE_DOWN_PAYMENT)  \\\n",
       "SK_ID_CURR                                           \n",
       "100002                                         0.0   \n",
       "100003                                         0.0   \n",
       "100004                                         0.0   \n",
       "100006                                         0.0   \n",
       "100007                                         0.0   \n",
       "\n",
       "            SUM(credit.previous.SELLERPLACE_AREA)  \n",
       "SK_ID_CURR                                         \n",
       "100002                                        0.0  \n",
       "100003                                        0.0  \n",
       "100004                                        0.0  \n",
       "100006                                       -6.0  \n",
       "100007                                        0.0  \n",
       "\n",
       "[5 rows x 589 columns]"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "feature_matrix.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "3961173a",
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_matrix.to_parquet(\"ft_feature_matrix.parquet\")\n",
    "ft.save_features(features, \"ft_feature_definitions.json\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dbf7be2f",
   "metadata": {},
   "source": [
    "上面的计算太不容易了，我们将特征矩阵保存下来"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "772343c4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(356255, 1233)"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "feature_matrix.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "418073bc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>FLAG_OWN_CAR</th>\n",
       "      <th>FLAG_OWN_REALTY</th>\n",
       "      <th>CNT_CHILDREN</th>\n",
       "      <th>AMT_INCOME_TOTAL</th>\n",
       "      <th>AMT_CREDIT</th>\n",
       "      <th>AMT_ANNUITY</th>\n",
       "      <th>REGION_POPULATION_RELATIVE</th>\n",
       "      <th>DAYS_BIRTH</th>\n",
       "      <th>DAYS_EMPLOYED</th>\n",
       "      <th>DAYS_REGISTRATION</th>\n",
       "      <th>...</th>\n",
       "      <th>MODE(credit.previous.PRODUCT_COMBINATION)_POS mobile without interest</th>\n",
       "      <th>MODE(credit.previous.PRODUCT_COMBINATION)_POS other with interest</th>\n",
       "      <th>MODE(credit.previous.PRODUCT_COMBINATION)_POS others without interest</th>\n",
       "      <th>MODE(credit.previous.WEEKDAY_APPR_PROCESS_START)_FRIDAY</th>\n",
       "      <th>MODE(credit.previous.WEEKDAY_APPR_PROCESS_START)_MONDAY</th>\n",
       "      <th>MODE(credit.previous.WEEKDAY_APPR_PROCESS_START)_SATURDAY</th>\n",
       "      <th>MODE(credit.previous.WEEKDAY_APPR_PROCESS_START)_SUNDAY</th>\n",
       "      <th>MODE(credit.previous.WEEKDAY_APPR_PROCESS_START)_THURSDAY</th>\n",
       "      <th>MODE(credit.previous.WEEKDAY_APPR_PROCESS_START)_TUESDAY</th>\n",
       "      <th>MODE(credit.previous.WEEKDAY_APPR_PROCESS_START)_WEDNESDAY</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SK_ID_CURR</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>100002</th>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>0</td>\n",
       "      <td>202500.0</td>\n",
       "      <td>406597.5</td>\n",
       "      <td>24700.5</td>\n",
       "      <td>0.018801</td>\n",
       "      <td>-9461</td>\n",
       "      <td>-637</td>\n",
       "      <td>-3648.0</td>\n",
       "      <td>...</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100003</th>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>0</td>\n",
       "      <td>270000.0</td>\n",
       "      <td>1293502.5</td>\n",
       "      <td>35698.5</td>\n",
       "      <td>0.003541</td>\n",
       "      <td>-16765</td>\n",
       "      <td>-1188</td>\n",
       "      <td>-1186.0</td>\n",
       "      <td>...</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100004</th>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>0</td>\n",
       "      <td>67500.0</td>\n",
       "      <td>135000.0</td>\n",
       "      <td>6750.0</td>\n",
       "      <td>0.010032</td>\n",
       "      <td>-19046</td>\n",
       "      <td>-225</td>\n",
       "      <td>-4260.0</td>\n",
       "      <td>...</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100006</th>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>0</td>\n",
       "      <td>135000.0</td>\n",
       "      <td>312682.5</td>\n",
       "      <td>29686.5</td>\n",
       "      <td>0.008019</td>\n",
       "      <td>-19005</td>\n",
       "      <td>-3039</td>\n",
       "      <td>-9833.0</td>\n",
       "      <td>...</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100007</th>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>0</td>\n",
       "      <td>121500.0</td>\n",
       "      <td>513000.0</td>\n",
       "      <td>21865.5</td>\n",
       "      <td>0.028663</td>\n",
       "      <td>-19932</td>\n",
       "      <td>-3038</td>\n",
       "      <td>-4311.0</td>\n",
       "      <td>...</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 1233 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "            FLAG_OWN_CAR  FLAG_OWN_REALTY  CNT_CHILDREN  AMT_INCOME_TOTAL  \\\n",
       "SK_ID_CURR                                                                  \n",
       "100002             False             True             0          202500.0   \n",
       "100003             False            False             0          270000.0   \n",
       "100004              True             True             0           67500.0   \n",
       "100006             False             True             0          135000.0   \n",
       "100007             False             True             0          121500.0   \n",
       "\n",
       "            AMT_CREDIT  AMT_ANNUITY  REGION_POPULATION_RELATIVE  DAYS_BIRTH  \\\n",
       "SK_ID_CURR                                                                    \n",
       "100002        406597.5      24700.5                    0.018801       -9461   \n",
       "100003       1293502.5      35698.5                    0.003541      -16765   \n",
       "100004        135000.0       6750.0                    0.010032      -19046   \n",
       "100006        312682.5      29686.5                    0.008019      -19005   \n",
       "100007        513000.0      21865.5                    0.028663      -19932   \n",
       "\n",
       "            DAYS_EMPLOYED  DAYS_REGISTRATION  ...  \\\n",
       "SK_ID_CURR                                    ...   \n",
       "100002               -637            -3648.0  ...   \n",
       "100003              -1188            -1186.0  ...   \n",
       "100004               -225            -4260.0  ...   \n",
       "100006              -3039            -9833.0  ...   \n",
       "100007              -3038            -4311.0  ...   \n",
       "\n",
       "            MODE(credit.previous.PRODUCT_COMBINATION)_POS mobile without interest  \\\n",
       "SK_ID_CURR                                                                          \n",
       "100002                                                  False                       \n",
       "100003                                                  False                       \n",
       "100004                                                  False                       \n",
       "100006                                                  False                       \n",
       "100007                                                  False                       \n",
       "\n",
       "            MODE(credit.previous.PRODUCT_COMBINATION)_POS other with interest  \\\n",
       "SK_ID_CURR                                                                      \n",
       "100002                                                  False                   \n",
       "100003                                                  False                   \n",
       "100004                                                  False                   \n",
       "100006                                                  False                   \n",
       "100007                                                  False                   \n",
       "\n",
       "            MODE(credit.previous.PRODUCT_COMBINATION)_POS others without interest  \\\n",
       "SK_ID_CURR                                                                          \n",
       "100002                                                  False                       \n",
       "100003                                                  False                       \n",
       "100004                                                  False                       \n",
       "100006                                                  False                       \n",
       "100007                                                  False                       \n",
       "\n",
       "            MODE(credit.previous.WEEKDAY_APPR_PROCESS_START)_FRIDAY  \\\n",
       "SK_ID_CURR                                                            \n",
       "100002                                                  False         \n",
       "100003                                                  False         \n",
       "100004                                                  False         \n",
       "100006                                                  False         \n",
       "100007                                                  False         \n",
       "\n",
       "            MODE(credit.previous.WEEKDAY_APPR_PROCESS_START)_MONDAY  \\\n",
       "SK_ID_CURR                                                            \n",
       "100002                                                  False         \n",
       "100003                                                  False         \n",
       "100004                                                  False         \n",
       "100006                                                  False         \n",
       "100007                                                  False         \n",
       "\n",
       "            MODE(credit.previous.WEEKDAY_APPR_PROCESS_START)_SATURDAY  \\\n",
       "SK_ID_CURR                                                              \n",
       "100002                                                  False           \n",
       "100003                                                  False           \n",
       "100004                                                  False           \n",
       "100006                                                  False           \n",
       "100007                                                  False           \n",
       "\n",
       "            MODE(credit.previous.WEEKDAY_APPR_PROCESS_START)_SUNDAY  \\\n",
       "SK_ID_CURR                                                            \n",
       "100002                                                  False         \n",
       "100003                                                  False         \n",
       "100004                                                  False         \n",
       "100006                                                  False         \n",
       "100007                                                  False         \n",
       "\n",
       "            MODE(credit.previous.WEEKDAY_APPR_PROCESS_START)_THURSDAY  \\\n",
       "SK_ID_CURR                                                              \n",
       "100002                                                  False           \n",
       "100003                                                  False           \n",
       "100004                                                  False           \n",
       "100006                                                   True           \n",
       "100007                                                  False           \n",
       "\n",
       "            MODE(credit.previous.WEEKDAY_APPR_PROCESS_START)_TUESDAY  \\\n",
       "SK_ID_CURR                                                             \n",
       "100002                                                  False          \n",
       "100003                                                  False          \n",
       "100004                                                  False          \n",
       "100006                                                  False          \n",
       "100007                                                  False          \n",
       "\n",
       "            MODE(credit.previous.WEEKDAY_APPR_PROCESS_START)_WEDNESDAY  \n",
       "SK_ID_CURR                                                              \n",
       "100002                                                  False           \n",
       "100003                                                  False           \n",
       "100004                                                  False           \n",
       "100006                                                  False           \n",
       "100007                                                  False           \n",
       "\n",
       "[5 rows x 1233 columns]"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "feature_matrix.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ba9fd9dc",
   "metadata": {},
   "source": [
    "```{note} 特征选择是否一定必要？(高缺失、高单一、高相关)\n",
    "- 对于**树模型**（如 LightGBM, XGBoost,），严格的特征选择并非必要。树模型天然对共线特征、缺失值和冗余特征具有鲁棒性。\n",
    "- 对于**线性模型**（如 Logistic Regression, Ridge），特征选择至关重要，尤其是共线高相关特征\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "69e5c6ee",
   "metadata": {},
   "source": [
    "## model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4e7fd336",
   "metadata": {},
   "source": [
    "### 导入数据"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2c4c3d1f",
   "metadata": {},
   "source": [
    "特征选择后， 我们可以对train, test分开了。 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "0de86fad",
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_matrix = pd.read_parquet(\"ft_feature_matrix.parquet\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d35bff97",
   "metadata": {},
   "source": [
    "前面的过程，其实都不涉及编码。所以我们在模型之前进行编码。也省去了一些存储"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "030fa8c1",
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_matrix = pd.get_dummies(feature_matrix)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "c5dea0b0",
   "metadata": {},
   "outputs": [],
   "source": [
    "final_fm = feature_matrix.reset_index()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "id": "448cf4f0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(307511, 1235) (48744, 1234)\n"
     ]
    }
   ],
   "source": [
    "final_fm = pd.merge(final_fm, app_target, on='SK_ID_CURR', how='left')\n",
    "final_fm = pd.merge(final_fm, app_set, on='SK_ID_CURR', how='left')\n",
    "\n",
    "train = final_fm[final_fm['set'] == 'train']\n",
    "test = final_fm[final_fm['set'] == 'test']\n",
    "\n",
    "train, test = train.align(test, join = 'inner', axis = 1)\n",
    "train = train.drop(columns=['set'])\n",
    "test = test.drop(columns = ['TARGET', 'set'])\n",
    "print(train.shape, test.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "ad5160ec",
   "metadata": {},
   "outputs": [],
   "source": [
    "train.to_feather('checkpoints/05_train_merged_ft.feather')\n",
    "test.to_feather('checkpoints/05_test_merged_ft.feather')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "2410a2d5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "del final_fm,feature_matrix,features,es\n",
    "gc.collect()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "b40111a2",
   "metadata": {},
   "outputs": [],
   "source": [
    "train = pd.read_feather('checkpoints/05_train_merged_ft.feather')\n",
    "test = pd.read_feather('checkpoints/05_test_merged_ft.feather')\n",
    "\n",
    "train_labels = train['TARGET']\n",
    "train_ids = train['SK_ID_CURR']\n",
    "test_ids = test['SK_ID_CURR']\n",
    "train_features = train.drop(columns=['TARGET', 'SK_ID_CURR'])\n",
    "test_features = test.drop(columns=['SK_ID_CURR'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "87a586b7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(307511, 1235) (48744, 1234)\n"
     ]
    }
   ],
   "source": [
    "print(train.shape, test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cc64ccd8",
   "metadata": {},
   "source": [
    "### hgbm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "id": "9c4bd585",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: total: 9min 6s\n",
      "Wall time: 1min 10s\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-1 {\n",
       "  /* Definition of color scheme common for light and dark mode */\n",
       "  --sklearn-color-text: #000;\n",
       "  --sklearn-color-text-muted: #666;\n",
       "  --sklearn-color-line: gray;\n",
       "  /* Definition of color scheme for unfitted estimators */\n",
       "  --sklearn-color-unfitted-level-0: #fff5e6;\n",
       "  --sklearn-color-unfitted-level-1: #f6e4d2;\n",
       "  --sklearn-color-unfitted-level-2: #ffe0b3;\n",
       "  --sklearn-color-unfitted-level-3: chocolate;\n",
       "  /* Definition of color scheme for fitted estimators */\n",
       "  --sklearn-color-fitted-level-0: #f0f8ff;\n",
       "  --sklearn-color-fitted-level-1: #d4ebff;\n",
       "  --sklearn-color-fitted-level-2: #b3dbfd;\n",
       "  --sklearn-color-fitted-level-3: cornflowerblue;\n",
       "\n",
       "  /* Specific color for light theme */\n",
       "  --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n",
       "  --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-icon: #696969;\n",
       "\n",
       "  @media (prefers-color-scheme: dark) {\n",
       "    /* Redefinition of color scheme for dark theme */\n",
       "    --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n",
       "    --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-icon: #878787;\n",
       "  }\n",
       "}\n",
       "\n",
       "#sk-container-id-1 {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 pre {\n",
       "  padding: 0;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 input.sk-hidden--visually {\n",
       "  border: 0;\n",
       "  clip: rect(1px 1px 1px 1px);\n",
       "  clip: rect(1px, 1px, 1px, 1px);\n",
       "  height: 1px;\n",
       "  margin: -1px;\n",
       "  overflow: hidden;\n",
       "  padding: 0;\n",
       "  position: absolute;\n",
       "  width: 1px;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-dashed-wrapped {\n",
       "  border: 1px dashed var(--sklearn-color-line);\n",
       "  margin: 0 0.4em 0.5em 0.4em;\n",
       "  box-sizing: border-box;\n",
       "  padding-bottom: 0.4em;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-container {\n",
       "  /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n",
       "     but bootstrap.min.css set `[hidden] { display: none !important; }`\n",
       "     so we also need the `!important` here to be able to override the\n",
       "     default hidden behavior on the sphinx rendered scikit-learn.org.\n",
       "     See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n",
       "  display: inline-block !important;\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-text-repr-fallback {\n",
       "  display: none;\n",
       "}\n",
       "\n",
       "div.sk-parallel-item,\n",
       "div.sk-serial,\n",
       "div.sk-item {\n",
       "  /* draw centered vertical line to link estimators */\n",
       "  background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n",
       "  background-size: 2px 100%;\n",
       "  background-repeat: no-repeat;\n",
       "  background-position: center center;\n",
       "}\n",
       "\n",
       "/* Parallel-specific style estimator block */\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item::after {\n",
       "  content: \"\";\n",
       "  width: 100%;\n",
       "  border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n",
       "  flex-grow: 1;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel {\n",
       "  display: flex;\n",
       "  align-items: stretch;\n",
       "  justify-content: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item:first-child::after {\n",
       "  align-self: flex-end;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item:last-child::after {\n",
       "  align-self: flex-start;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-parallel-item:only-child::after {\n",
       "  width: 0;\n",
       "}\n",
       "\n",
       "/* Serial-specific style estimator block */\n",
       "\n",
       "#sk-container-id-1 div.sk-serial {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "  align-items: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  padding-right: 1em;\n",
       "  padding-left: 1em;\n",
       "}\n",
       "\n",
       "\n",
       "/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n",
       "clickable and can be expanded/collapsed.\n",
       "- Pipeline and ColumnTransformer use this feature and define the default style\n",
       "- Estimators will overwrite some part of the style using the `sk-estimator` class\n",
       "*/\n",
       "\n",
       "/* Pipeline and ColumnTransformer style (default) */\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable {\n",
       "  /* Default theme specific background. It is overwritten whether we have a\n",
       "  specific estimator or a Pipeline/ColumnTransformer */\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "/* Toggleable label */\n",
       "#sk-container-id-1 label.sk-toggleable__label {\n",
       "  cursor: pointer;\n",
       "  display: flex;\n",
       "  width: 100%;\n",
       "  margin-bottom: 0;\n",
       "  padding: 0.5em;\n",
       "  box-sizing: border-box;\n",
       "  text-align: center;\n",
       "  align-items: start;\n",
       "  justify-content: space-between;\n",
       "  gap: 0.5em;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 label.sk-toggleable__label .caption {\n",
       "  font-size: 0.6rem;\n",
       "  font-weight: lighter;\n",
       "  color: var(--sklearn-color-text-muted);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 label.sk-toggleable__label-arrow:before {\n",
       "  /* Arrow on the left of the label */\n",
       "  content: \"▸\";\n",
       "  float: left;\n",
       "  margin-right: 0.25em;\n",
       "  color: var(--sklearn-color-icon);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "/* Toggleable content - dropdown */\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content {\n",
       "  max-height: 0;\n",
       "  max-width: 0;\n",
       "  overflow: hidden;\n",
       "  text-align: left;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content pre {\n",
       "  margin: 0.2em;\n",
       "  border-radius: 0.25em;\n",
       "  color: var(--sklearn-color-text);\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-toggleable__content.fitted pre {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n",
       "  /* Expand drop-down */\n",
       "  max-height: 200px;\n",
       "  max-width: 100%;\n",
       "  overflow: auto;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n",
       "  content: \"▾\";\n",
       "}\n",
       "\n",
       "/* Pipeline/ColumnTransformer-specific style */\n",
       "\n",
       "#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator-specific style */\n",
       "\n",
       "/* Colorize estimator box */\n",
       "#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-label label.sk-toggleable__label,\n",
       "#sk-container-id-1 div.sk-label label {\n",
       "  /* The background is the default theme color */\n",
       "  color: var(--sklearn-color-text-on-default-background);\n",
       "}\n",
       "\n",
       "/* On hover, darken the color of the background */\n",
       "#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "/* Label box, darken color on hover, fitted */\n",
       "#sk-container-id-1 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator label */\n",
       "\n",
       "#sk-container-id-1 div.sk-label label {\n",
       "  font-family: monospace;\n",
       "  font-weight: bold;\n",
       "  display: inline-block;\n",
       "  line-height: 1.2em;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-label-container {\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "/* Estimator-specific */\n",
       "#sk-container-id-1 div.sk-estimator {\n",
       "  font-family: monospace;\n",
       "  border: 1px dotted var(--sklearn-color-border-box);\n",
       "  border-radius: 0.25em;\n",
       "  box-sizing: border-box;\n",
       "  margin-bottom: 0.5em;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-estimator.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "/* on hover */\n",
       "#sk-container-id-1 div.sk-estimator:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-1 div.sk-estimator.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Specification for estimator info (e.g. \"i\" and \"?\") */\n",
       "\n",
       "/* Common style for \"i\" and \"?\" */\n",
       "\n",
       ".sk-estimator-doc-link,\n",
       "a:link.sk-estimator-doc-link,\n",
       "a:visited.sk-estimator-doc-link {\n",
       "  float: right;\n",
       "  font-size: smaller;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1em;\n",
       "  height: 1em;\n",
       "  width: 1em;\n",
       "  text-decoration: none !important;\n",
       "  margin-left: 0.5em;\n",
       "  text-align: center;\n",
       "  /* unfitted */\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted,\n",
       "a:link.sk-estimator-doc-link.fitted,\n",
       "a:visited.sk-estimator-doc-link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "/* Span, style for the box shown on hovering the info icon */\n",
       ".sk-estimator-doc-link span {\n",
       "  display: none;\n",
       "  z-index: 9999;\n",
       "  position: relative;\n",
       "  font-weight: normal;\n",
       "  right: .2ex;\n",
       "  padding: .5ex;\n",
       "  margin: .5ex;\n",
       "  width: min-content;\n",
       "  min-width: 20ex;\n",
       "  max-width: 50ex;\n",
       "  color: var(--sklearn-color-text);\n",
       "  box-shadow: 2pt 2pt 4pt #999;\n",
       "  /* unfitted */\n",
       "  background: var(--sklearn-color-unfitted-level-0);\n",
       "  border: .5pt solid var(--sklearn-color-unfitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted span {\n",
       "  /* fitted */\n",
       "  background: var(--sklearn-color-fitted-level-0);\n",
       "  border: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link:hover span {\n",
       "  display: block;\n",
       "}\n",
       "\n",
       "/* \"?\"-specific style due to the `<a>` HTML tag */\n",
       "\n",
       "#sk-container-id-1 a.estimator_doc_link {\n",
       "  float: right;\n",
       "  font-size: 1rem;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1rem;\n",
       "  height: 1rem;\n",
       "  width: 1rem;\n",
       "  text-decoration: none;\n",
       "  /* unfitted */\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 a.estimator_doc_link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "#sk-container-id-1 a.estimator_doc_link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "#sk-container-id-1 a.estimator_doc_link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>HistGradientBoostingClassifier(max_depth=5)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" checked><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow\"><div><div>HistGradientBoostingClassifier</div></div><div><a class=\"sk-estimator-doc-link fitted\" rel=\"noreferrer\" target=\"_blank\" href=\"https://scikit-learn.org/1.6/modules/generated/sklearn.ensemble.HistGradientBoostingClassifier.html\">?<span>Documentation for HistGradientBoostingClassifier</span></a><span class=\"sk-estimator-doc-link fitted\">i<span>Fitted</span></span></div></label><div class=\"sk-toggleable__content fitted\"><pre>HistGradientBoostingClassifier(max_depth=5)</pre></div> </div></div></div></div>"
      ],
      "text/plain": [
       "HistGradientBoostingClassifier(max_depth=5)"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "from sklearn.ensemble import HistGradientBoostingClassifier\n",
    "\n",
    "hist_gradient_boost_model= HistGradientBoostingClassifier(\n",
    "    max_iter = 100, # 树个数\n",
    "    learning_rate = 0.1,\n",
    "    max_depth = 5,\n",
    ")\n",
    "hist_gradient_boost_model.fit(train_features, train_labels)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "id": "4f8a5b3d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, 'hist gb Roc curve, auc=0.812')"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjIAAAFGCAYAAABwl3UTAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8ekN5oAAAACXBIWXMAAA9hAAAPYQGoP6dpAABMnElEQVR4nO3deVxVZf4H8A+LAiqgoCyKAS6puKAompoiamruW4ZmLrk2pWPZONZvxrLJMStzMmuq0XCpyMk0xIXSFBNXVAg1U1PcEGRfBYTL+f1xhsM9cHcu93Dh83697svnXJ5zzvde7/K9z/Oc57ERBEEAERERkRWyVToAIiIiIlMxkSEiIiKrxUSGiIiIrBYTGSIiIrJaTGSIiIjIajGRISIiIqvFRIaIiIisFhMZIiIislpMZIiIiMhq2Zuy05w5c7Bt2zYAQEhICGJiYswWUExMDEJDQ6VtTjxsOBsbG6kcHh6OOXPmKBcMEZGVu3nzJs6dO4fk5GQIggAfHx/06dMH7dq1s2gcDx8+xIULF3DlyhVkZ2ejvLwcrq6u6NixI/r06YPmzZtbNB5DlJeX4+rVq7h06RLu37+PgoICODs7w9PTE8HBwWZ9Dk1KZOqbnJwc/Otf/5K2ly1bVidfGNZkyJAhOHbsmNa/29rawsXFBW5ubujWrRsGDhyImTNnonXr1haMkoioutjYWPz1r3/FyZMnNf59wIABWLduHZ588slajePWrVtYvXo1du7ciaKiIo117O3tMWbMGPz9739H7969DT52Xl4ezp8/j7i4OOl2+/Zt6e+mNFLk5eUhMjISkZGROHLkCLKzs7XW7dixI5YsWYIXX3wR9vY1TEUEE8yePVsAIAAQQkJCTDmEVkePHpWObWJ4RktKSpKdMykpySLnNTf1xxAeHq5oLCEhIbJ4DLnZ2dkJy5cvF4qKihSNnYgarnfffVews7Mz6PPq3XffrbU4vvvuO6Fp06ZGfX6+9957eo+bmJgodO7cWbCxsdF5PGO/27OzswVHR0ejP/f79OlT4+9ctshQrbO3t4evr6/sPpVKhezsbOTm5sruW79+PS5duoSoqCg0atTI0qESUQO2detWrFy5Utpu3LgxwsLCEBwcjPLycsTFxeG///0vHj16BJVKhZUrV8LLywuzZ882axxHjx7F9OnTUVZWJt3XqVMnTJw4Ee3atYOtrS1u376NqKgo/PrrrwDEz88VK1bAzc0N8+bN03rszMxM/P7772aNFwDKyspQXFwsu69Dhw4IDQ1Fx44d0bJlS+Tm5iIuLg579uyRWpjOnTuHYcOG4cSJE/Dy8jLt5KZkP7XZIqMEtsiYn3qLjK+vr9Z6N2/eFF5//XWhUaNGsvj/+c9/Wi5YImrw7ty5Izg4OEifQW3bthV+++23avUuX74stG3bVqrn4OAg3Llzx2xxlJeXC507d5Z9Hr711luCSqXSWP+zzz4TbG1tpbrNmzcXcnNztR6/aq+Ht7e3MG7cOOHtt98WWrZsafJ3e3p6ugBAcHFxEZYtWyZcvHhRa93k5GRh8ODBsjjCwsKMOp86JjICE5naYGgiUyEyMlIWf4sWLYRHjx7VfqBERIIgzJs3T9ZNc/bsWa11z5w5I+t+mj9/vtniOHPmjOyzcOrUqXr3efXVV2X7fPPNN1rr/vbbb8Ibb7wh7NmzR7h3757sb76+viZ/t+fk5AhvvPGGkJWVZVD9goICISAgQBb3zZs3jTpnBbN3LT18+BBHjx7FrVu3UFBQAA8PDzz55JPo2LGjuU8lKS0tRUJCAhITE5GVlYWSkhI0adIEnp6eaN++PQIDA+Hk5FRr56+JCxcuICEhAWlpaXB3d4e/vz9CQkLM3q1y6dIl/Prrr0hOTkbTpk3RoUMHDB06tM5034wfPx7Dhw/H4cOHAQDZ2dk4c+aMwYPpSktLERsbixs3biA9PR3Ozs7w9vbGoEGD4OHhUaPYSktLcfr0aenYKpUKbm5u6Ny5M/r06YMmTZrU6Pj65OXlITY2FsnJycjIyIC9vT08PT3RrVs3BAYGws7OrlbPb4rCwkIcO3YMd+/eRWZmJtzd3fH888/X+nOlSWlpKU6cOIGbN28iLS0NTZo0QZs2bRASEoKWLVtaPB59CgoKkJiYiKtXryIjIwNlZWVo0aIFfH19MXDgQLi4uCgdok4ZGRmIjY1FamoqsrKy4ODggNatW6NHjx4ICAiQXV1ZV+Tk5GD79u3SdkV3kjZ9+/ZFWFgYvv76awDAtm3b8MEHH8DV1bXGsZw/f162raubSL3Ohx9+KG0nJCRg+vTpGut26dIFa9asqVmQGri6uhp13KZNm2LVqlUICwuT7jtw4ABeeukl409uSvajqUWmpKREWLFiheDi4qJxQM+QIUOEK1eu6D22MYN9i4uLhbfeekvw8PDQOZioUaNGQmhoqHD48GGtj8OQmyEtC4b66aefqjUfVtw8PT2FdevWCeXl5YIgyFtajh49qvWY6vUqWmRiY2OFPn36aDyPu7u78MEHH2htsqwJY1tkBEEQ1q5dK4vv888/17tPVlaW8Oc//1nr687W1lYIDQ0V4uLijH4Mt2/fFl544QXB2dlZ62vCwcFBePrpp4Uff/zR6OPrc/LkSWHEiBGCvb291vM3b95cmDt3rsb3lqktjervi9mzZxtUr+JzICcnR5g3b57QpEmTarEmJSUJ77zzjrTdpEkTIT8/36jnpHv37tL+06ZN01n3wYMHwosvvqj1tWFnZyeMGzdOuHbtmlEx1IaUlBThww8/FJ588slq3axVY540aZLOZnt1hv5fqjP1dbN//36hf//+OgeRenp6CkuWLBHu3r1r0DEt5euvvzb4c7bCkSNHDG4FMYb6ewSAcPXqVb37FBcXy/ZZsGCBSeeuSYuMKdLS0mRxv/rqqyYdxyyJTGZmphAcHKw3EXB3d9f7BjQ0kcnOzhZ69+5tVCKyfPlyrY/DkolM1S9sbbcJEyYIpaWlBr/B1OuFh4cLO3bs0PmhWHEbO3as2btxTElkvvjiC1lc+sbJJCQkCJ6engY9l7a2tgaN6K/w5ZdfyvrL9d3M+aYvKysTXnzxRaNem2+++Wa141g6kblx44bg7++vNcakpCQhKSlJ9kW3Y8cOg5+Xixcvyo4XGRmptW50dLTWBKbqzcHBQdi7d6/BcdSG5cuXG/X/7ejoKOzcuVPvcS2RyBQWFgoTJ040Kn6lu76reu6556TYnJycDPo8fPTokeDk5CTtN3PmTLPE8sknn8ieqwsXLujdJzMzU7bPG2+8YdK5LZ3IPHz4UBb3okWLTDpOjbuWVCoVwsLCEBcXJ13PPmjQILRo0QIpKSn47rvvpFHVmZmZmDVrFuLi4mrcHL5o0SJZE5y/vz/GjRuHxx9/HM2aNUNhYSHu37+P+Ph4xMTE4OHDh9WOUdH1VFZWJrt+3tfXV+N17T4+PjWKGRCbIF9//XVp28bGBsOHD8dTTz0FNzc3pKSkICoqCmfPnkVkZCRWr15t0nkuXbqETz75BKWlpXBzc0NYWBi6d++OsrIynDt3Drt27UJhYSEAYN++fVi4cCHCw8Nr/PhqIiMjQ7atq9vr2rVrCA0Nlc1T0KlTJ0ydOhV+fn7Izc3FkSNHEB0djfLycpSXl2PFihVo1KgRli1bpjOODz/8EMuXL5fd17VrV4wePRp+fn5wcHBAWloaLly4gMOHDyMnJ8fox6qNIAiYOnUqfvjhB+k+W1tbDB48GKGhoWjdujXKysqQnJyMM2fOICYmBqWlpWY7v6lKSkowbdo0JCUlwc7ODqNHj8bgwYPh5uaGtLQ0HDhwADY2NlL3SGxsLADg66+/xsyZMw06x1dffSWV3dzc8PTTT2ust3fvXkydOlX2vPTr1w+jRo2Cj48PiouLERcXh++++w5FRUUoKSnBlClTcOjQIYSEhNTgWTAPf39/PPnkk+jWrRvc3NygUqlw584d/PTTTzh37hwAoLi4GDNnzkT79u2NmjvE3IqLizF06FCcOXNGuq9Ro0YYPnw4Bg0aBE9PTxQXF+P27ds4efIkTp48ifLycsXi1ebixYtSuU+fPgZ1uTdq1Ai9e/eWXsvqx6iJ/v37y7aPHTuGXr166dyn6nwvgwcPNksstS0pKUm27enpadqBTMl+1LP8il9XHTp0EC5dulStrkqlEl566SVZ1vXDDz9oPbYhLTI3b96U1Vm2bJlQVlam9ZgPHz4Utm/frrWrwlKDfVNTU4XmzZtL53F2dhYOHTqkse6WLVsEOzs72Wh0wPAWmYr9hgwZIqSlpVWre+PGDaFbt26yfczZPWJKi8ywYcNk8ezatUtjPZVKJQwYMEBWd9WqVRpfA7/88ovg7u4u+/Wt6XVaITY2VjaIz9XVVecv3+LiYuHrr782+ZdEVe+++67scXXt2lU4f/681vqZmZnC2rVrhY0bN1b7myVbZNT/rxMSEnQe/9///rdU397eXnjw4IHemMrLy4XHHntM2m/x4sUa6929e1do0aKFVM/d3V04ePCgxrpJSUmy90C7du2EwsJCvbHUhhUrVggzZswQzpw5o7Pe3r17ZZ8hffv21Vm/tltkFi9eLKs7aNAg4fr161rrJycnCytWrBD++9//GhSLJahUKtn8J3PmzDF4X/Xn19HR0Wzd9P369ZOO6+3tLSQnJ2utm5OTI3Tp0kWq36NHD5PjsHSLzD/+8Q/Z6+fAgQMmHafGiQwg9tPruvzs0aNHQocOHaT6zz77rNa6hiQyW7Zskf7u4eGhM4kxhKUSmRUrVsjOExUVpbP++++/X+2LwtBEBoDQvn17nZfh3blzR/ah36tXL1MfWjXGJjI//PCDLHZ7e3uNCZggCML3338vq7tkyRKdxz5+/LgsIZw4caLWuoGBgbIPJlPG1pgqJSVF1p3VqVMnISMjw+TjWTqRcXR0NKg/PzMzU2jcuLG0n6YkrKpjx47JzhUbG6uxXlhYmFTHyclJiI+P13nclJQUwc3NTdpn06ZNemOpDcZMAvnTTz/Jnotz585prVubicyFCxdk9Z588knh4cOHBj8OTT766COhffv2tXL76KOPNJ7z3r171X4UGervf/+7bN+qVwGZ6vLly7KE1cfHR/j666+FgoICqU5RUZHwww8/CJ06dZLqtWzZUrh8+bLJ57VkIlNQUCC0bt1aOp+bm5tQXFxs0rHMksi8//77evdRz7zatWuntZ4hicyaNWsM/kViCEskMmVlZYKXl5d0jhEjRujdp7S0VGjXrp3JiYwhv3rWrVsn28eQ/lhDGJrIJCUlCf/3f/9XbSzP3Llzte7z1FNPSfVatWol5OXl6Y3nhRdekPaxs7PTONjw8OHDshgsPZfN3/72N9n5T5w4UaPjWTqR+ctf/mJwbBMmTJD269evn976CxculOr7+/trrHPv3j3ZwOjVq1cbFMv69eulfXr06GHwY1DSkCFDpJjXrFmjtV5tJjIzZ86UJbGmXjqr7s0336z2OWaum6ZxZIIgXo6sXm/Dhg0Gx6v+2gFg0AUthvr111+rtZrb2dkJbdq0EXx8fKp9Zg4bNky4ceNGjc5pyUTm5ZdflsW/du1ak49lltWvZ82apbdOv379pHJSUlKN+vXVL+G8cuWKbHbYuioxMRGpqanStiELOtrb2+O5554z6Xxubm6YNGmS3npz5syRjVfat2+fSefTJTk5GR06dJDd2rVrhxYtWsDf3x9r1qyRvR46d+6M9evXazxWUVERjh49Km3PmDEDzs7OemN48cUXpbJKpcKPP/5Yrc6uXbukcpMmTUy7DLAG1M8/cOBADBgwwKLnr6nnn3/e4Lrqr+szZ87gxo0bWus+evRI9tzMmDFDY71du3ZJM6Ha2dkZ/P+nHndiYmK1sVp10bBhw6Ry1ct1LUGlUsnGcU2ZMgX+/v4Wj8McCgoKZNuOjo4G71t1Wo+qx6qJHj164Ndff8Unn3wiXXKvUqmQnJyMe/fuSZ+Zjo6O+Oc//4kff/zR4otZmmr37t3YtGmTtN21a1e88sorJh+vxomMn5+fQfN0qC8GKAhCjZIP9YFP+fn5mDhxIq5fv27y8Syh6oeNoYOxTB20FRoaatBCXB4eHggMDJS2KwYTmlNZWRlu3LghuyUlJVUbJGtjY4OwsDCcOHECLVq00HisCxcuyKbtHjVqlEEx9OnTB61atZK2z549W61OxaA9ABg6dKhF5+vIzMzE1atXpW1DktC6xNnZGd26dTO4/rhx42TPb8V8HJocPHgQWVlZ0ra2wcEnTpyQysHBwXB3dzcollatWsHPz0/ajouLM2g/JalP5Z6cnGzx8yckJMi+tM31en3rrbcgiD0FZr+99dZbGs9ZdVr9xo0bGxyvg4ODbFvbwo6muH37Np555hm8/PLLyMvL01qvuLgYb7zxBjp27Ij9+/eb7fy1JT4+Xtb44eTkhIiIiGrPpTFqnMgYOsq4adOmsu2KK2ZMMWjQIHTp0kXajomJQadOnTBo0CC8++67OH36dJ24kkOd+uhsZ2dntGnTxqD91B+nMbp27WpS3WvXrpl0PnNYu3YtIiIi4ObmprVO1YS1e/fuBh+/R48eWo8DQLb+SJ8+fQw+rjlcuXIFgiAodv6a8vf3N2qiM0dHR0yZMkXa1pXIqP8tKCgInTt31liv4upIwPj3jfrn2L1794za15wePnyIiIgIzJ49G7169UKrVq3g6OgIGxsb2W3BggXSPkq0SF+5ckW2bW2vV3VVW2AePXpk8L4lJSWybXNNvHr27Fn07NkTu3fvhiAIcHd3x5o1a3Dx4kUUFhaiqKgIV69exUcffYS2bdsCEL9jxo4di08//dQsMdSGP/74A08//bT0/W9ra4sdO3YY9TmuSY0TGWOa4dSpf2gby9bWFhEREbJZOQVBQGxsLF5//XX0798fbm5uGDt2LHbs2GHWLNlU6h822lobNGnevLlJ51NvfTCmrq5l103l6+tb7dfRgwcPcOrUKcyfP1+61HHlypVYtGiRztdG1fiMmbVX1+PMzc2VtfSYvHiZiTIzM2Xblj5/TRnSvVeVesvKtWvXNLYG5ufny7o7dV2qrf4choeHV/vy13VTv3zYnJfTG+Orr76Cn58fZsyYge3btyMhIQEZGRnVviyrqtqiYAnW/npV16xZM9m2Mc9n1e+WqscyRUZGBsaPHy+9Dh9//HH8+uuveOONN9CtWzc0adIEjo6OePzxx7F06VIkJibKuqGXLl1aJ1sV7969i+HDh+PBgwfSfZ999pnsB42pzDJGRgmBgYGIj4/HzJkzNV7zX1BQgP3792PWrFno0KED/vvf/yoQZSX1D6OaNF0ayphfBupjjszZx6uLh4cHnnjiCfznP//BgQMHpP/DL774Am+//bbW/dRb8uzt7Y1aYkG9VbDq48zPz5dtm+MDyRhKn7+mDOnGrGrIkCGylklNrTLff/+99GVhZ2enddp1wHwtE5rmnKpt77//Pp5//nmkp6dX+5ubmxt8fHzQvn176aaewNfkR6Gp1F+v9vb2NeoWUFrVLmRjfsxVrWtKQl/Ve++9J33Z29jY4Ntvv9XZgt+8eXN899130me+SqXCm2++WeM4zOnBgwcYPny4bL62Dz74QNayWBNWm8gA4gR1O3bsQHJyMsLDwzFr1iyNA87u37+PZ599Fp988okCUYrU3yzGJAtVv+AMZUwrlHpyoMQX6PDhw/HOO+9I2//4xz9kv5DVqcdXVlZmVBeirsdZ9QPIUgldXTm/EmxtbWWJybfffguVSiWro57cDB06VOcvf/WE3N3dXfbFb8zNmBZTc0hMTJRNkunp6Yl//vOfuHDhAoqLi5GZmYm7d+/ijz/+kG7vvvuuRWOsSv31WlZWprfVqC7z9vaW9SzcuXPH4H3V6zo6OsLb27vG8ezcuVMqDxw4UO9keIA4BnXy5MnS9qFDh2o0fMOcMjMzMXz4cNmwhTfffLPapKM1YfZFI5XQqlUrzJkzR7oS6NatW4iKisKWLVtk/eavvfYaJk+ebJYXm7HUu8HS09NRXFxsULecqf31mn7ZaaN+lYalP8QrvPrqqwgPD8fvv/8OlUqFV155BSdPnqxWr2p86enpsoHkuqg/J1WP4+rqCnt7e6l7Sf0KM0uoOjA1NTUVnTp1qtExTV2cz5JdsTNnzsQHH3wAQHzMR44cwVNPPSVtq1+hpm8GYHd3d2lQ5KJFi2plYbza8Mknn0gJnJeXF86fP6/3NW1o95cprwFD/v81vV59fX2NPldVGzduxMaNG2t8HE2WLl2KpUuXVrvf1tYWHTt2lGbm/eOPPww+pvrVdo8//jhsbWvWNpCbmytLjoyZtbl3795S4l9WVobr16+jZ8+eNYqnpnJzczFixAhcunRJuu+1117TOvDaVFbdIqONn58flixZgvj4eKxYsUK6v7i4WHbJoCWpD2ZSqVQGT2cdHx9v0vnUXzjG1H388cdNOl9N2dvby17cp06dwp49e6rV69Chg2zbmGnBExMTpbKm1djVB4jWxtVbulRdFdgc56+60rShXSbqfdi1LTAwUHa1k3oLTEREhPQF7+TkJPvFqYl64qc+cLuuO3LkiFRetmyZQYl51andtVF/DZjz/z8gIEC2ba73S1ZWVrUrHM11U7/yrSr1CwHOnz9vUEtvaWmp7GrUmg5YBapfBFP1IhldqtZVemxoQUEBRo0ahQsXLkj3/elPf8L7779v9nPVy0Smgo2NDdauXSv79VB1tD1QfU2f2lgLpG/fvrIvqsjISIP2MzXxiomJkQ1e1SYtLU32Ba/k1QfPPPOMLMGouBRTXVBQkGw8RnR0tEHHPn/+vKxFRn1eowqDBg2SykeOHNF5yaO5ubm5yRIpcyTcrq6usu379+/r3afqh7MlqM8ps3v3bukDWD2pmTBhgt5uT/V1kg4fPmzU1SdKUv9/UZ8KQRf1lipd1C8WMOT/HwBOnz6tt05gYKCse0mpH4jmor5uV1FRkexSfm2OHz8uGxg8evToGsdRtaXYmJbhlJQU2bauqz9rW1FREcaOHSt7Lc2dO1c2d4w51etEBhCbDdu3by9ta/pyr/oBWRuXM3p4eGDo0KHS9ueff663efjKlSuIiooy6XxZWVkaWzSqCg8Pl41LGDt2rEnnMwdbW1u89tpr0nZiYmK1x+Dk5CR7Hr/55huDxpN89tlnUtnOzg4jR46sVueZZ56Ryg8fPrT4mCr188fGxuLUqVM1Ol7jxo1l86MY8gW1a9cui4/Pee6556QkPz8/H1FRUbh69aosoTJkYckpU6ZITft5eXn44osvaidgM1NP1g25YubYsWMGt7iqt7AmJCToPb4gCNi2bZve49rZ2clayHbt2mVwK5EuSswjAwBjxoyR/aDdvHmz3li3bNkilRs1amSWRMbJyUnWIvfzzz8b/MP6p59+ksoODg547LHHahyPKR49eoRJkybh2LFj0n1hYWHYvHmzyd3depkyHbD6tNeGTmNs6LTXhixRcPv2bYNjLSoqkq0n9O6772qsp76uxfbt2w0+vjH27Nkje2yTJ0/WurhXfn6+0Lt372rTbJtzraWqC+z17Nmzpg9RYsqikYIgLsLo7e0t7RsYGCiUl5fL6lRda2nZsmU6j3ny5EnZQpC61lrq1auXbNp1S6619ODBA9nidTVda0kQBOHZZ5+VjtexY0fh0aNHWutmZWUJbdu2lT23hi5RUNPpzAcPHiwda/z48bI1bFq2bCmUlpYadJypU6dK+zk7OwuJiYlGxaFrzaOqn03aprw3lvpaOfPnz9dZNy8vT+jcubMsDl3vrytXrsjqbt26VefxP/jgg2qfI9o+qxMSEmT1Bg0aZNSaUXXN3LlzZUsBnD17Vmvds2fPyj5TXnjhBZ3HDg8Plz1X4eHhWuvOmTNHVvfTTz/VG/vevXtl+wwfPlzvPprUdImC0tJSYdKkSbJYJk6caPD711RWmcjMnj1bCAoKErZt26ZznZ2ioiLZeiAAtC6oNWLECKlOt27dzLJuiCYjR46UxRMSEiKcPHlS+ntZWZlw4MABaTVT9VVNjUlkKhZJDA0NFdLT06vVvXnzptC9e3fZPtHR0WZ7nKYmMoJQfQXo3bt3y/6uafXrt99+W2NSGBsbK7Rq1Uqqp2/169OnT8vW63F1ddW5ZtWjR4+EiIgIrasxG6vqF4m+1a+zsrKEd999V+vCi1WT5z/96U8an6ebN29KSVzFivaWTGS++OIL6ViNGjWSJVQvvfSSwce5deuWbBHI5s2bC19//bXO1YDLy8uFU6dOCfPnzxdGjhyptV5tJTLqa87Y2dkJ3377rcZ6N27cEIKDg2Xvb0PeX+oLobZs2VJjcqdSqYQNGzYItra2sv9/XZ/VVWOvSGZ0rX59//59YeXKlXVq9esKt27dki1m+thjjwm//fZbtXqXL1+WvT4bN26s98e1MYnMhQsXZP8HjRs3FjZv3qy1/p49ewRnZ2fZ8U1dRbomiUx5eXm179tRo0YJJSUlJsViDBtBMH4Sgjlz5kjNjyEhIYiJidG7z61bt2SXRiclJcmavSvExMQgNDRU2tYUnvr5nZyc0L9/fwQHB6NNmzZwcXFBQUEBfvvtN+zZs0fWb/jCCy/ImgPVRUREyNZwsbGxQdu2bdG8eXOpOax169Y4cOCA3seqS0pKCgYNGlRtbZmmTZvCzc1NuqIJEJsHz5w5Ixt5/ssvv8jGcqhTb7Z79dVX8emnn6K4uBhubm6YPn06unfvjrKyMpw7dw7fffedbGDZnDlzEB4eXqPHpm7IkCFS06Kvry9u3bpl8L55eXl47LHHpC6+nj174sKFC7LHd+3aNTzxxBOyeRy6dOmCqVOnwtfXF7m5uTh69CgOHjwo6zrbsGEDli1bpvP8H330UbU6Xbt2xZgxY+Dn5wcHBwekp6cjISEBhw4dQmZmpsHvA30EQcDUqVOxe/du6T5bW1uEhIQgNDQU3t7eKC8vR3JyMuLi4nDkyBGUlJTgzTff1Nh0rlKpEBwcLBs03r17d0ybNg1t2rRBbm4uTp06hcjISJSUlKB79+7o1KmTtLbR7NmzsXXrVo2xmvI5oE1OTg68vLw0XsZ76tQpPPHEEwYf66effsKECRNk3Sjt2rXDyJEj0blzZ7i6uqKwsBDp6em4ePEiTp48KX1O6HocVT+btD3nxrpx4wYCAgJkY3qGDRuGkSNHolWrVsjOzsbJkycRFRWFkpISNGvWDH/605/w3nvvAdD//tq9e7ds0jEHBwfMmDEDffv2hZ2dHZKSkrBnzx5pgPTq1atl85Bo+6wGxK6w0NBQWbdl48aNMXz4cAwaNAgeHh4oKSnBnTt3cPr0aRw/fhwqlQrh4eEGrTdnaVu2bMH8+fOlbQcHB4SFhSE4OBiCICAuLg7ffvut7P9q8+bNmDdvns7jbt26FXPnzpW29T3+1157rdp6c127dsW4cePQrl072NjY4O7duzh48GC1ye/CwsIQERGhM57nnntO4xQXt2/floZfODo6apy/ZsqUKVi3bl21+48fP15tSZ22bdsaNW9av379dM7yrZUp2U9daJFRr2PIbcKECXqXCK+aTVa9GduyoM3t27ertShUvXl5eQmxsbFCcXGx7P6EhAStx1WvFx4eLuzYsaPaCqmabmPGjNHZ5WCKmrTICIIg/PWvf5XFuGfPnmp1EhISBE9PT4P+/21sbIT33nvP4PNv3rxZ9utM382cK8WWlZXJVns25KardeDixYtCy5Yt9R6jQ4cOws2bN01a/docj79qkzQgdo+a4tSpU7IuSkNvQ4cO1XrMiIgIva9JU23dulXWyqLt1qxZM2Hv3r2yX/iGvL8WL15s0Htk9erVRq+aXlhYKIwbN86o51lXi4TS3nnnHYP+L2xtbXWuPK7OmBYZQRBbN5YuXWr063f69Ol6v+cEQf75bOxN22dC1e9uU26mfo5YZSITHx8vvPrqq0JAQEC1ZtCqt8DAQGHHjh0GP7aoqCghLCxM6Nixo9CsWTPZ8c2VyAiC2JQbEREhjB07VvDx8REaN24seHl5CQMGDBA+/PBDITMzUxAEQUhJSTH4Q0XTGyU2NlbjWBsAgpubm/DBBx/obHY3VU0TmZSUFNl4kZ49e1YbKyMIYtfK0qVLqzWtqn/YhIaG6uzv1ubGjRvCc889Jzg5OWl9fTk6Ogrjx48Xjhw5YvTx9YmJiREGDx6s80PV3d1dWLRokXDt2jWdx7p69arw1FNPaX0M8+fPl8ZTKZXIVB37BEBYtWqVyccrKCgQ1qxZU23cT9Wbk5OTMGLECOGLL77QOaZs0aJF0j49evTQ+Hqsiejo6GrjXypudnZ2wqhRo4SrV68KgiAYnciUl5cLGzZskI0FVL8FBAQI+/btEwTB8M/qqnbv3i0EBQXpfK7btGkjLF++XLh//76pT5NFHDt2TOjfv7/Wx9G/f3/h2LFjBh/P2ESmwpEjR4Snn35ab2I1aNAgoxLr+pbImNS1VJdkZWUhMTERN2/eREZGBh49egRnZ2e0bt0avXv3tpplzbU5fPiwNEGYg4MD8vPzjZqWv8KlS5eQkJCA5ORkNG3aFB06dMDQoUONavaryx49eoTjx49Lr4OmTZvC29sbISEhRq3HpElRURFiY2Nx69YtZGRkwM7ODu7u7ujUqRN69+5ttoXitMnIyMDx48dx//59ZGdnw8nJCV5eXujatSt69Ohh1CRcSUlJ+OWXX5CamgpHR0c89thjGDJkiGITIVrKlStXEB8fj/T0dOTn56Np06bw9PRE586d0bVrV4Om2O/cubO0QvmuXbvMskZMVYIg4Pz58zh//jwyMzPh4uKC1q1bY8CAAWZZz6ikpATHjh3D1atXUVBQAG9vb3Tt2hXBwcFmiF507949nDp1CqmpqcjNzUXTpk3Rpk0bdO/e3eRFcJVy48YNxMXFSSuMt2nTBsHBwbIrYS0hNzcXZ8+exY0bN5CTkwNBEODq6gpfX1/069dPNuFqQ2T1iUx9p95X2qdPnzq5GBhRfZeamirNCN6tWzckJibW3qWkRGSUej+PjDVLT0+XDcDVNPcJEdU+9QHAq1atYhJDVIcwkbGw7Oxs2eJZ2hQUFCAsLEyaVtvGxkY26p2ILKcikQkICKiVLiUiMh0TGQt78OABAgICpEtsqy7umJGRgW3btiEoKEi2BstLL71k8X5ZIhJ99tlnEAQBly9frvHCgERkXhwjY2G///57tQFvLVq0gIuLC/Lz85GdnV1t7px+/frh559/NmoBMSIiooaAPy0szN7evlr/enZ2Nm7fvo2srCxZEmNra4vZs2fjyJEjTGKIiIg0YIuMAu7evYvIyEgcP34cv/32G5KTk5Gfny9d1uvv74/Q0FBMnz4dAQEBSodrUeXl5bh//z6cnZ05oJKoDhIEAfn5+WjdujW72ahOYCJjpVQqFa5cuYJz587h/PnzOHfuHH799VcUFRUB0D21fE3FxMTgyy+/RGxsLFJSUuDk5AQ/Pz+MHz8eixYtki5TNcW9e/fQtm1bM0ZLRLXh7t278PHxUToMItgrHQCZZtq0abL1eCyhrKwMixcvrrZeVXFxMbKzsxEfH4+NGzciPDwcEyZMMOkczs7OAMQPSRcXlxrHTETmlZeXh7Zt20rvVSKlMZGxUuoLIQKAm5sb3N3dcf369Vo758KFC6V5bVxdXTFv3jwEBQWhsLAQe/fuxf79+5GdnY1p06YhOjpatsCeoSq6k1xcXJjIENVh7PqluoKJjJXq27cvunTpgt69e6N3797w9/evtsKqOR08eFBKYry9vXHs2DF07NhR+vvChQvx8ccfY+nSpXj06BHmzZuH33//vd4sgUBERHUTExkr9cYbb1j0fKtWrZLKmzZtkiUxFZYsWYJDhw4hKioKSUlJCA8Px6JFiywZJhERNTAcck563bx5E+fOnQMA+Pv7Y9KkSVrrvvLKK1I5IiKi1mMjIqKGjYkM6RUdHS2VR40apbNvfNCgQWjWrBkA4Pjx4ygoKKj1+IiIqOFiIkN6Xbx4USoHBwfrrGtvb49evXoBEOeEuXLlSq3GRkREDRsTGdLr6tWrUtnf319vffU66vsSERGZGwf7kl45OTlSuWXLlnrru7u7a9yXiCxDEIDCQqCoCCgoAG7dEv+9fh1wcADi44FWrYDSUqCkpPL2yitAz55KR09kHCYypJf6OBdHR0e99Z2cnKRyfn6+zrolJSUoKSmRtvPy8kyIkKhhUKmAlBTg0iUxOSksBH77DWjWDLh7F7hzBzh/3vTjT5rERIasDxMZMoq5J8Fau3YtVq9ebdZjElmj8nLg9m0xMTl/HsjJAXJzgQcPgP37LROD2m8KIqvBRIb0qrgKCYC0lpMu6nX0TWP++uuv49VXX5W2K6Y/J6pvVCqx1SQpCTh+XOzqSUoCdu0CWrcG7t83z3latQK8vICLF4ExY4CmTQEPDyArC+jTR0xWOnUSEycPD8DRUexucnAQ4yCyNkxkSK/mzZtL5YyMDL31MzMzNe6riYODAxwcHEwNjahOUamA338Hrl4Fzp0TW1i++Ub8m52d+HdN9CUx9vbirbgYaNkSmD5dHP8SEAC0aycmK+3bAz4+QKNG5n1MRHUdExnSq1OnTjh69CgAICkpCUOGDNFZPykpSbYvUX2UmQkkJgIHDogtK6mpYvKirXtGWxJTwdUVCA4GXFyAESPEhMXPT0xOWrYUEyEiqo6JDOnVvXt3qRwXF6dzPaeysjLEx8cDAGxtbREQEFDr8RHVtkePgF9/FbuEfv5ZLCcnG3+coUOBwEAxQfHyArp3Bzp0YCsKUU0wkSG9Ro0aJZWjo6MhCILWQb/qs/kOHjwYTZs2tUiMROZSUiIOuE1MFAfdnjwpXq5cXq5/X39/oFcvoFs3scvH11ccj+LtXftxEzVUTGRIr3bt2iE4OBhxcXFISkrCnj17MHnyZI11N2zYIJXDwsIsFSKRSQRBHMdy4QLw1Vdi4pKaKrbA6OLiAvToISYtgYFA165iy4oB0ywRkZkxkWngYmJiEBoaCgDw9fXFrVu3NNZbvXo1Ro8eDQB4+eWX0aNHD3To0EFWZ9OmTYiKigIgzu6rqwuKSAkqldgtdOwYEB0tXtmTkqJ/vy5dgN69xat+hg0TExczz0RARCZiImOlkpKSsGXLFtl9iYmJUjk+Ph5/+9vfZH8PCgrS2pKiz9NPP425c+ciPDwcKSkp6NOnD+bPn4+goCAUFhZi79692LdvHwCgcePG2LJlCxo3bmzSuYjMKTkZOHUK2L1bTF6ys7XXtbERW2mee05saQkMFJMXNzfLxUtExmEiY6Vu376NNWvWaP17YmKiLLEBgNmzZ5ucyADAF198ARsbG3z55ZfIzc3F+vXrq9Vp0aIFwsPDpVYeIku7dw/Yuxe4fBmIiRHHu+gSGgr07y9eMTRkCKBnxgAiqmOYyJDB7O3tsWXLFjz//PPYsmULTpw4gZSUFDg6OsLPzw/jx4/H4sWL4c2RjWRhd+4A338vtrrExmqv5+gIhISIE8X17i0mL7xiiMi62QiCICgdBFGFvLw8uLq6Ijc3Fy4uLkqHQ3XY9etAZKSYvJw6pb1e375iS8vgwcDw4eIMtmQ6vkeprmGLDBFZjRs3xHEuO3YAZ85orvP448Azz4gtLkOHihPNEVH9xUSGiOq0GzeAzZvFrqPr1zXX6dJFnLZ/6lSxTEQNBxMZIqpzbtwAtm0Tu40uX9Zcp0sXYNYsYNIkcdI5ImqYmMgQUZ2Qmwvs3AmEhwOnT2uuExwMTJwIPPWUeFk053IhIiYyRKQYQQCOHgV27RJn1s3Pr14nMBAICwOefx5o08byMRJR3cZEhogsrqwM+PprYP16cXbdqgICxG6jZ58VF1gkItKGiQwRWczDh8C//w188IG4ppE6Bwdgxgxg4UKgXz92GxGRYZjIEFGtUqnES6YjIoA9e8RkRl2/fsDLLwMTJgDOzsrESETWi4kMEdWKkhLg00/FFpiql03b2ADjxgHLl4sT1RERmYqJDBGZVWEhsGED8PHHQFqa/G9NmgCTJwN/+QvQo4cy8RFR/cJEhojMIicH+PJL4P33q49/GToUePFFcY0jJydFwiOieoqJDBHVSFISsHEj8NlnQHFx5f22tsD48cCbbwI9eyoWHhHVc0xkiMgkN24Aq1cD334LlJbK/zZyJPCvfwGdOysSGhE1IExkiMgo9+8Dq1aJM/CWl1feb28PLFgALFokTmJHRGQJTGSIyCD374tJSnS0OKFdhSZNgMWLgWXLgLZtFQuPiBooJjJEpFNenngV0tq14iXVFZo1A5YuBVasAFxdlYuPiBo2JjJEpFFZmTiId80aICur8v5mzYARI8Q5Yjw9lYuPiAhgIkNEGhw5Avz5z8ClS5X32dpWtsB4eysXGxGROiYyRCQ5cQJYtw6IipLfP3Uq8NZbQNeuioRFRKQVExkiwsOHwGuvicsJqOvSRexCGjJEkbCIiPSyVToAIlJWTAzQu7c8ifH0BDZtAhITmcQQUd3GFhmiBiovT1y0cfPmyvscHcXBvUuWAI0aKRcbEZGhmMgQNUCnTgHTpwO3b1fe17evOMldQIBycRERGYtdS0QNiCCIizoOHlyZxDRpIt536hSTGCKyPmyRIWogLl0CXnkFOHy48r7gYOCbb4AOHZSLi4ioJtgiQ1TPCYK4gGNQkDyJWbYMOHmSSQwRWTe2yBDVY3l5wKxZQGRk5X0+PuKMvZMmKRcXEZG5MJEhqqdSUoCJE4GzZyvvW7QI+PBDcVwMEVF9wESGqB5KSAAmTADu3BG3XVzEK5ImT1Y0LCIis+MYGaJ6pGI8TL9+lUmMt7e4dhKTGCKqj9giQ1RPFBcDL7wARERU3hcYCOzfD7Rpo1xcRES1iS0yRPXAtWvihHbqScyf/gScOcMkhojqN7bIEFm5XbuA2bPFhR8BoHFjYNs2ICxM2biIiCyBLTJEVuydd4BnnqlMYtq1E2foZRJDRA0FExkiKyQIwNq1wN//Xnnf2LHA+fPixHdERA0Fu5aIrExpKfD888DOnZX3rVsH/OUvgI2NcnERESmBiQyRFXn4UJyR96efKu/7xz+AFSuUi4mISElMZIisRFkZMH16ZRLTqBGwYwfw7LPKxkVEpCQmMkRWoLgYWLgQ2LtX3HZ0FMtPPaVsXERESmMiQ1TH5eYCI0ZUrpnUuLG4CCSTGCIiXrVEVKdlZACjRsmTmK++EhMbIiJiiwxRnfXgATBsGHD5srjt6gpERQGDBikbFxFRXcIWGaI66N49YPjwyiTG2xuIiWESQ0RUFVtkiOqYBw+AkBDg5k1x29MT+OUXoEMHZeMiIqqL2CJDVIfcuQMMGVKZxLRpI7bEMIkhItKMLTJEdcRvvwEjR4rdSgDQti1w9CjQvr2ycRER1WVskSGqA65dA0JDK5OY9u2ZxBARGYKJDJHC0tPFy6nT0sTtnj2B48eZxBARGYJdS0QKys4Gpk0Dbt8Wt7t1E5cgaNVK2biIiKwFExkihdy5I052d+WKuO3uDuzfzySGiMgY7FoiUkBBAfD005VJTMuWwL59wGOPKRsXEZG1YYsMkYWVlwOTJolXKQFAu3bAgQNAp07KxkVEZI3YIkNkQYIALF8OHD4sbru6igtAMokhIjINExkiC1q/HvjXv8SyjQ3w/ffiAF8iIjINExkiCzl8GPjrXyu3P/tMXBSSiIhMx0SGyAIePABmzBDHxwDA4sXAwoXKxkREVB8wkSGqZWVlwPTp4sR3gHjJ9caNysZERFRfMJEhqmWvvy4uNwAAXl5AeDjQqJGyMRER1RdMZIhq0aFDwAcfiGV7eyAiQkxmiIjIPJjIENWS+/eBWbMqt//5T2DIEMXCISKql5jIENUClQp49lkgNVXcHjZMnD+GiIjMi4kMUS346CMgNlYse3kBO3YAtny3ERGZHT9aiczs0iXgjTcqt//7X8DbW7l4iIjqMyYyRGaUmQmMHQuUlIjbixcDgwYpGxMRUX3GRIbIjF5/Hbh9WywHBlZesURERLWDiQyRmXz3HfCf/4hlZ2cgKgpo2lTZmIiI6jsmMkRm8OCB2I1U4a23gLZtFQuHiKjBYCJDZAbLlwNZWWJ56lTglVeUjYeIqKFgIkNUQz/8AHz9tVhu2hTYtAmwsVE0JCKiBoOJDFENFBQAL75Yub1xI+DpqVw8REQNDRMZohp4443K2XvHjAHmzlU2HiKihoaJDJGJfvwR+PhjsezoCHz4IbuUiIgsjYkMkQlKS4ElSyq3164FHn9cuXiIiBoqJjJEJnj/feD6dbE8aBDw5z8rGw8RUUPFRIbISDdvAqtXi2VbW2DDBnYpEREphYkMkZFeeQV49EgsL1kC9O6tbDxERA0ZExkiI8TEAHv3iuVWrYBVqxQNh4iowWMiQ2SgsjL5MgRr1gBubsrFQ0RETGSIDLZxI3D1qlh+4gnghReUjYeIiJjIEBkkJQX4298qtz/8ELCzUy4eIiISMZEhMsDGjUBRkVieOxfo31/ZeIiISMREhkiP3Fzgs8/Esr098PbbysZDRESVmMjUA5GRkZg6dSp8fX3h6OgIDw8P9O/fH++99x5yc3PNdp45c+bAxsbG4FtMTIzZzq2kTZuAnByxPHMm4OOjaDhERKTGXukAyHT5+fmYMWMG9u3bJ7s/PT0d6enpOH36ND7++GPs3LkTAwYMUChK65aTI87iC4iT3q1cqWg4RERUBRMZK1VWVoYpU6bg0KFDAABPT08sWLAAAQEByMrKQkREBE6cOIF79+5hzJgxiI2NRdeuXc12/s8//xweHh4663Tr1s1s51PKxx+LXUuA2BrTqZOy8RARkRwTGSv1n//8R0piAgICcOTIEXh6ekp/f+mll/Daa69h/fr1yMnJwaJFixAbG2u2848YMQJ+fn5mO15dVFAgXp0EiEsR/N//KRsPERFVxzEyVkilUuFttRGnO3bskCUxFdatW4eePXsCAE6cOIEff/zRUiHWCx99VDk25tln2RpDRFQXMZGxQseOHUNqaioAICQkBEFBQRrr2dnZYenSpdJ2RESEReKrDwoLxUG+gDg2hksREBHVTUxkrFB0dLRUHj16tM666n8/ePBgrcVU33z4IfC/XBHjxgGdOysbDxERacZExgpdvHhRKgcHB+us6+npibZt2wIA0tLSkJ6ebpYYFi5cKF3u7erqiscffxzPP/88IiMjIQiCWc6hlPx8YMMGsWxrC6xdq2w8RESkHRMZK3S1YsEfAP7+/nrrq9dR37cmDh06hDt37qCkpAR5eXm4fv06vvrqK0ycOBFBQUFmO48StmwBsrPF8rRpQECAsvEQEZF2vGrJCuVUjEAF0LJlS7313d3dNe5riqZNm2LYsGHo27cv/Pz80LhxYzx48ADHjx/Hnj17UFpaioSEBPTv3x8nTpxAly5danQ+S1OpgPXrK7c5bwwRUd3GRMYKFRQUSGVHR0e99Z2cnKRyfn6+yed9+eWXsWnTJjRr1kzj327evImpU6ciPj4e2dnZeOaZZ5CYmAhbW+0NfyUlJSgpKZG28/LyTI7PHH76Cbh3TyyPHg0EBioaDhER6cGuJTJYnz59NCYxFdq1a4cff/xRmijv8uXL+P7773Uec+3atXB1dZVuFeN5lPLvf1eW589XLg4iIjIMExkrpJ5MFBcX661fVLFsMwBnZ+daialCq1at8Oc//1narrp8QlWvv/46cnNzpdvdu3drNT5dEhOBinDbtAHGj1csFCIiMhC7lqxQ8+bNkf2/0agZGRk6W0kAIDMzU7ZvbQsNDZXKV65c0VnXwcEBDg4OtR2SQd55B6i44OqllwA7O2XjISIi/dgiY4U6qU0xm5SUpLe+ep1OFpieVn0Ack0HF1vK1avA7t1iuVUrQK1RiYiI6jAmMlaoe/fuUjkuLk5n3QcPHkjdNR4eHmjVqlWtxgZANleNJVqAzGHDBvGKJQBYuhRo0kTZeIiIyDBMZKzQqFGjpLK+2XoPHDgglfXNAmwuR48elcqWaAGqqevXxbljAKBpU2DJEmXjISIiwzGRsUIhISHw8vICAMTExODChQsa66lUKmzcuFHaDgsLq/XY0tLS8NFHH0nbY8eOrfVz1tTHHwNlZWJ5+XLA1VXZeIiIyHBMZKyQnZ0dVqmtYjhr1iykpaVVq7dy5UokJCQAAAYOHIiRI0dqPF5MTAxsbGxgY2MDPz8/jXW2bduG6OhoncsPJCUlYdSoUVLXUpcuXTB16lQDH5UyioqA7dvFsqMj8MorysZDRETG4VVLVmrBggXYs2cPDh06hMuXLyMwMBALFixAQEAAsrKyEBERgdjYWACAq6srPv/88xqdLz4+Hh999BFat26NESNGoEePHvD09ESjRo2QlpYmzez76NEjAECLFi3w3Xffwa6OX/qzdSuQmyuWp00DrGRIDxER/Q8TGStlb2+P77//HjNmzMC+ffuQmpqKf/zjH9Xq+fj4YOfOnejatatZznv//n1s3bpVZ53g4GBs374dna1gyeht2yrLS5cqFwcREZmGiYwVc3Z2RlRUFCIjI7F9+3bExcUhLS0Nzs7OaN++PSZPnoxFixbB1QyDPv7yl7+gT58+OH36NOLj45GamorMzEwUFhbCxcUFPj4+6NevH5555hkMHz4cNjY2ZniEtev334EzZ8Ry9+5A797KxkNERMazEXQNeiCysLy8PLi6uiI3NxcuLi61eq633gJWrxbL778PvPZarZ6OqF6w5HuUyBAc7EsNUnk5EB4ulu3sxPExRERkfZjIUIP088/AnTti+amngMceUzYeIiIyDRMZapA2b64sz5mjWBhERFRDTGSowcnOBvbuFcstWwKTJysbDxERmY6JDDU4mzcDxcViefp0oFEjZeMhIiLTMZGhBufbbyvLXFeJiMi6MZGhBuX6daBiaapevYCOHZWNh4iIaoaJDDUoERGV5enTlYuDiIjMg4kMNSi7d1eWn31WuTiIiMg8mMhQg3HpEvDrr2I5OJhzxxAR1QdMZKjB2LGjsjxzpnJxEBGR+TCRoQahvByoWLS7USMgLEzRcIiIyEyYyFCDcP48kJYmlkePBjw8lI2HiIjMg4kMNQh79lSWn35auTiIiMi8mMhQg/Ddd+K/trbAxImKhkJERGbERIbqvT/+EG8A8OSTgKensvEQEZH5MJGhei8ysrI8erRycRARkfkxkaF67/vvK8sTJigXBxERmR8TGarXcnOBuDix3KED0LmzsvEQEZF5MZGheu3wYaCsTCyzW4mIqP5hIkP12v79leVRo5SLg4iIagcTGaq3BAH4+Wex7OAAhIYqGw8REZkfExmqt/74A7hzRyw/+STg6KhsPEREZH5MZKjeOnSosjxsmHJxEBFR7WEiQ/XWsWOV5REjlIuDiIhqDxMZqpfKy4GjR8WyszPQo4ey8RARUe1gIkP10sWLQHq6WA4JARo1UjYeIiKqHUxkqF46cKCyPHKkcnEQEVHtYiJD9dIvv1SWOT6GiKj+YiJD9Y4gAOfOiWUPD6BjR2XjISKi2sNEhuqdO3eAjAyx3LMnYGOjaDhERFSLmMhQvVOxSCQA9O2rXBxERFT7mMhQvXPiRGWZiQwRUf3GRIbqnTNnKssDByoXBxER1T4mMlSvlJRUDvTt0AFwc1M2HiIiql1MZKheuXQJKC0Vy088oWwsRERU+5jIUL1y4UJluXdv5eIgIiLLYCJD9Yr6QN8+fZSLg4iILIOJDNUrCQniv3Z2TGSIiBoCJjJUbxQXA7/9JpY7dQIcHZWNh4iIah8TGao34uMrB/oGBysbCxERWQYTGao3zp6tLPfrp1wcRERkOUxkqN44f76yzBYZIqKGgYkM1RsViUyjRkC3bsrGQkRElsFEhuqFwkLgyhWx3L07B/oSETUUTGSoXkhMBARBLPfqpWwsRERkOUxkqF64fr2y3LWrcnEQEZFlMZGhekE9kenQQbk4iIjIspjIUL2QmFhZ7tJFuTiIiMiymMhQvXDunPiviwvQrp2ysRARkeUwkSGrl5sL3L8vlgMDAVu+qomIGgx+5JPV+/33ynLnzsrFQURElsdEhqze1auVZSYyREQNCxMZsnrqVyx16qRcHEREZHlMZMjq3bpVWfb3VywMIiJSABMZsno3b1aW/fwUC4OIiBTARIasXkUi4+UFNGmibCxERGRZTGTIqhUUAKmpYpkz+hIRNTxMZMiqcXwMEVHDxkSGrJr6+BgmMkREDQ8TGbJqf/xRWe7YUbk4iIhIGUxkyKolJVWW2SJDRNTwMJEhq6Y+RoaXXhMRNTxMZMiqVSwWaWcnXn5NREQNCxMZsmoVl163aiUmM0RE1LAwkSGrVVpamcj4+CgbCxERKYOJDFmte/eA8nKx7OurbCxERKQMJjJktZKTK8tskSEiapiYyJDVqhjoCwCtWysXBxERKYeJDFmtBw8qy7xiiYioYWIiQ1YrJaWyzESGiKhhYiJDVostMkRExESGrNbdu5XlNm2Ui4OIiJTDRIasVkUi06QJ4OambCxERKQMJjJktSq6lry8ABsbZWMhIiJlMJEhq1RSAmRni2VPT2VjISIi5TCRIauUllZZ9vZWLg4iIlIWExmyShkZleWWLZWLg4iIlMVEhqxSRbcSwIG+REQNGRMZskp5eZVlV1fl4iAiImUxkSGrlJtbWXZxUS4OIiJSFhMZskr5+ZVlJjJERA0XE5l6IDIyElOnToWvry8cHR3h4eGB/v3747333kOuetOFGcXExGDWrFlo164dnJyc4ObmhqCgILz11ltIUV8EqZYUFFSWmzat9dMREVEdZa90AGS6/Px8zJgxA/v27ZPdn56ejvT0dJw+fRoff/wxdu7ciQEDBpjlnGVlZVi8eDG2bNkiu7+4uBjZ2dmIj4/Hxo0bER4ejgkTJpjlnJoUFlaWnZ1r7TRERFTHMZGxUmVlZZgyZQoOHToEAPD09MSCBQsQEBCArKwsRERE4MSJE7h37x7GjBmD2NhYdO3atcbnXbhwIcLDwwEArq6umDdvHoKCglBYWIi9e/di//79yM7OxrRp0xAdHY3Q0NAan1MT9RaZZs1q5RRERGQNBLJKn376qQBAACAEBAQIqamp1eosX75cqjNw4MAan/PAgQPS8by9vYVr165Vq7Nx40apjr+/v1BSUmLUOXJzcwUAQm5urs568+YJAiDeEhONOgUR1YCh71EiS+EYGSukUqnw9ttvS9s7duyAp4Z5+tetW4eePXsCAE6cOIEff/yxRuddtWqVVN60aRM6duxYrc6SJUswbtw4AEBSUpLUemNu6l1LHCNDRNRwMZGxQseOHUNqaioAICQkBEFBQRrr2dnZYenSpdJ2RESEyee8efMmzp07BwDw9/fHpEmTtNZ95ZVXzHJOXZjIEBERwETGKkVHR0vl0aNH66yr/veDBw+a5ZyjRo2CjY7lpgcNGoRm/xu4cvz4cRSoD2gxk+LiyrKTk9kPT0REVoKJjBW6ePGiVA4ODtZZ19PTE23btgUApKWlIT09vdbPaW9vj169egEAysvLceXKFZPOqcujR5Xlxo3NfngiIrISTGSs0NWrV6Wyv7+/3vrqddT3revn1EU9kWnUyOyHJyIiK8FExgrl5ORI5ZYGLP3s7u6ucd+6fk5dSkvFf+3sxBsRETVMnEfGCqmPOXF0dNRb30ltEEm++tz+deCcJSUlKCkpkbbz1FeD1KGiRYbdSkREDRtbZMhougb6Gmvt2rVwdXWVbhXjefSpSGTYrURE1LAxkbFCzdSmsi1Wv3xHi6KiIqnsbOJ8/urnVD9eTc/5+uuvIzc3V7rdvXvXoHgOHgQuXwZiYw2qTkRE9RS7lqxQ8+bNkZ2dDQDIyMiQJRmaZGZmyvY19ZwVMjIy9NY39JwODg5wcHAwOh4/P6N3ISKieogtMlaoU6dOUjkpKUlvffU66vvW9XMSERHpw0TGCnXv3l0qx8XF6az74MEDqbvGw8MDrVq1qvVzlpWVIT4+HgBga2uLgIAAk85JRESkDxMZKzRq1CiprG+23gMHDkhlfbMAG3rO6OhoCIKgta76bL6DBw9GU64hQEREtYSJjBUKCQmBl5cXACAmJgYXLlzQWE+lUmHjxo3SdlhYmMnnbNeunTSjb1JSEvbs2aO17oYNG8xyTiIiIn2YyFghOzs72UrUs2bNQlpaWrV6K1euREJCAgBg4MCBGDlypMbjxcTEwMbGBjY2NvDTMYp29erVUvnll1/GH3/8Ua3Opk2bEBUVBUCc3Xfu3LmGPCQiIiKT8KolK7VgwQLs2bMHhw4dwuXLlxEYGIgFCxYgICAAWVlZiIiIQOz/rk12dXXF559/XuNzPv3005g7dy7Cw8ORkpKCPn36YP78+QgKCkJhYSH27t2Lffv2AQAaN26MLVu2oDFnrCMiolpkI+ga7EB1Wn5+PmbMmCElD5r4+Phg586dGDBggNY6MTExCA0NBQD4+vri1q1bWuuWlZVh0aJF+PLLL7XWadGiBcLDwzFhwgT9D6KK3NxcNG/eHHfv3oWLi4vR+xNR7crLy0Pbtm2Rk5MDV1dXpcMhYouMNXN2dkZUVBQiIyOxfft2xMXFIS0tDc7Ozmjfvj0mT56MRYsWmfXDxt7eHlu2bMHzzz+PLVu24MSJE0hJSYGjoyP8/Pwwfvx4LF68GN7e3iYdv2I5A0Nn+CUiZeTn5zORoTqBLTJUp5SXl+P+/ftwdnbWuxRCxS9Dtt4Q1Ywx7yVBEJCfn4/WrVvD1pbDLEl5bJGhOsXW1hY+Pj5G7ePi4sJEhsgMDH0vsSWG6hKm00RERGS1mMgQERGR1WIiQ1bLwcEBb775pkmLThJRJb6XyJpxsC8RERFZLbbIEBERkdViIkNERERWi4kMERERWS0mMkRERGS1mMiQVYmMjMTUqVPh6+sLR0dHeHh4oH///njvvfeQm5urdHhEdZpKpcKlS5ewdetWLFmyBP3790eTJk1gY2MDGxsbzJkzR+kQiYzGmX3JKmhbIDM9PR3p6ek4ffo0Pv74Y70LZBI1ZNOmTcPu3buVDoPIrNgiQ3VeWVkZpkyZIiUxnp6e+Nvf/oZvvvkGmzZtwsCBAwEA9+7dw5gxY3D58mUlwyWqs1QqlWzbzc0NHTt2VCgaIvNgiwzVef/5z39w6NAhAEBAQACOHDkCT09P6e8vvfQSXnvtNaxfvx45OTlYtGgRYmNjlQqXqM7q27cvunTpgt69e6N3797w9/fH1q1bMXfuXKVDIzIZJ8SjOk2lUsHHxwepqakAgPPnzyMoKEhjvT59+iAhIQEAEB0djZEjR1oyVCKrpJ7IzJ49G1u3blU2ICIjsWuJ6rRjx45JSUxISIjGJAYA7OzssHTpUmk7IiLCIvEREZGymMhQnRYdHS2VR48erbOu+t8PHjxYazEREVHdwUSG6rSLFy9K5eDgYJ11PT090bZtWwBAWloa0tPTazU2IiJSHhMZqtOuXr0qlf39/fXWV6+jvi8REdVPTGSoTsvJyZHKLVu21Fvf3d1d475ERFQ/MZGhOq2goEAqOzo66q3v5OQklfPz82slJiIiqjuYyBAREZHVYiJDdVqzZs2kcnFxsd76RUVFUtnZ2blWYiIiorqDiQzVac2bN5fKGRkZeutnZmZq3JeIiOonJjJUp3Xq1EkqJyUl6a2vXkd9XyIiqp+YyFCd1r17d6kcFxens+6DBw9w9+5dAICHhwdatWpVq7EREZHymMhQnTZq1CiprG+23gMHDkhlfbMAExFR/cBEhuq0kJAQeHl5AQBiYmJw4cIFjfVUKhU2btwobYeFhVkkPiIiUhYTGarT7OzssGrVKml71qxZSEtLq1Zv5cqV0srXAwcO5MrXREQNhI0gCILSQRDpUlZWhtGjR+PQoUMAAC8vLyxYsAABAQHIyspCREQEYmNjAQCurq44ceIEunbtqmTIRHVSUlIStmzZIrsvMTERUVFRAIAePXpg3Lhxsr8HBQVh8uTJFouRyFhMZMgq5OfnY8aMGdi3b5/WOj4+Pti5cycGDBhgwciIrEdMTAxCQ0ON2mf27NnYunVr7QREZAbsWiKr4OzsjKioKPzwww+YPHky2rZtCwcHB7Rs2RL9+vXDunXrcOnSJSYxREQNDFtkiIiIyGqxRYaIiIisFhMZIiIislpMZIiIiMhqMZEhIiIiq8VEhoiIiKwWExkiIiKyWkxkiIiIyGoxkSEiIiKrxUSGiIiIrBYTGSIiIrJaTGSIiIjIajGRISIiIqvFRIaIiIisFhMZIiIislpMZIiIiMhq/T/spmT7MrGf/gAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 300x300 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.metrics import  roc_curve, roc_auc_score\n",
    "\n",
    "train_prob = hist_gradient_boost_model.predict_proba(train_features)\n",
    "\n",
    "fpr, tpr, thresholds = roc_curve(train_labels, train_prob[:, 1])\n",
    "auc = roc_auc_score(train_labels, train_prob[:, 1])\n",
    "\n",
    "plt.figure(figsize=(3,3))\n",
    "plt.plot(fpr, tpr, color='blue', lw=2)\n",
    "plt.title(f'hist gb Roc curve, auc={auc:.3f}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "id": "bd53bd44",
   "metadata": {},
   "outputs": [],
   "source": [
    "import time\n",
    "import os\n",
    "\n",
    "def submit(ids, pred, name, feature_count=None):\n",
    "    \"\"\"\n",
    "    ids: 测试集的 SK_ID_CURR\n",
    "    pred: 模型预测概率\n",
    "    name: 你的实验备注 (如 'lgb_v1', 'baseline')\n",
    "    feature_count: 可选，记录模型使用了多少个特征\n",
    "    \"\"\"\n",
    "    # 1. 创建提交 DataFrame\n",
    "    submit_df = pd.DataFrame({\n",
    "        'SK_ID_CURR': ids,\n",
    "        'TARGET': pred\n",
    "    })\n",
    "\n",
    "    # 2. 生成时间戳 (格式: 0213_1530)\n",
    "    timestamp = time.strftime(\"%m%d_%H%M\")\n",
    "    \n",
    "    # 3. 构造文件名\n",
    "    # 格式: 0213_1530_lgb_v1_f542.csv\n",
    "    f_str = f\"_f{feature_count}\" if feature_count else \"\"\n",
    "    filename = f\"{timestamp}_{name}{f_str}.csv\"\n",
    "    \n",
    "    # 4. 确保保存目录存在 (可选)\n",
    "    if not os.path.exists('submissions'):\n",
    "        os.makedirs('submissions')\n",
    "    \n",
    "    save_path = os.path.join('submissions', filename)\n",
    "    \n",
    "    # 5. 保存并打印提示\n",
    "    submit_df.to_csv(save_path, index=False)\n",
    "    \n",
    "    return submit_df\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "id": "9159d1f5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>SK_ID_CURR</th>\n",
       "      <th>TARGET</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>307511</th>\n",
       "      <td>100001</td>\n",
       "      <td>0.047845</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>307512</th>\n",
       "      <td>100005</td>\n",
       "      <td>0.114776</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>307513</th>\n",
       "      <td>100013</td>\n",
       "      <td>0.023606</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>307514</th>\n",
       "      <td>100028</td>\n",
       "      <td>0.038234</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>307515</th>\n",
       "      <td>100038</td>\n",
       "      <td>0.167165</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        SK_ID_CURR    TARGET\n",
       "307511      100001  0.047845\n",
       "307512      100005  0.114776\n",
       "307513      100013  0.023606\n",
       "307514      100028  0.038234\n",
       "307515      100038  0.167165"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hist_gradient_boost_model_pred = hist_gradient_boost_model.predict_proba(test_features)\n",
    "submit_df = submit(test['SK_ID_CURR'], hist_gradient_boost_model_pred[:, 1], \n",
    "    name='hgbm_baseline',\n",
    "    feature_count=train_features.shape[1]\n",
    "    )\n",
    "submit_df.head()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2dcb7871",
   "metadata": {},
   "source": [
    "得分76，和手动工程相差不多，特征数也差不多"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "de90f986",
   "metadata": {},
   "source": [
    "### lgbm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "id": "7593a7b1",
   "metadata": {},
   "outputs": [],
   "source": [
    "import re\n",
    "# 1. 定义清理函数\n",
    "def clean_names(df):\n",
    "    # 替换所有非字母、数字的字符为下划线\n",
    "    # 这里的正则 [^A-Za-z0-9_] 会匹配空格、斜杠、括号等所有特殊字符\n",
    "    df.columns = [re.sub(r'[^A-Za-z0-9_]+', '_', col) for col in df.columns]\n",
    "    # 顺便处理一下可能出现的重复下划线，比如 __\n",
    "    df.columns = [re.sub(r'_+', '_', col).strip('_') for col in df.columns]\n",
    "    return df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "id": "78573141",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[LightGBM] [Info] Number of positive: 24825, number of negative: 282686\n",
      "[LightGBM] [Info] Auto-choosing col-wise multi-threading, the overhead of testing was 0.679221 seconds.\n",
      "You can set `force_col_wise=true` to remove the overhead.\n",
      "[LightGBM] [Info] Total Bins 90425\n",
      "[LightGBM] [Info] Number of data points in the train set: 307511, number of used features: 1088\n",
      "[LightGBM] [Info] [binary:BoostFromScore]: pavg=0.080729 -> initscore=-2.432486\n",
      "[LightGBM] [Info] Start training from score -2.432486\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n",
      "[LightGBM] [Warning] No further splits with positive gain, best gain: -inf\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<style>#sk-container-id-2 {\n",
       "  /* Definition of color scheme common for light and dark mode */\n",
       "  --sklearn-color-text: #000;\n",
       "  --sklearn-color-text-muted: #666;\n",
       "  --sklearn-color-line: gray;\n",
       "  /* Definition of color scheme for unfitted estimators */\n",
       "  --sklearn-color-unfitted-level-0: #fff5e6;\n",
       "  --sklearn-color-unfitted-level-1: #f6e4d2;\n",
       "  --sklearn-color-unfitted-level-2: #ffe0b3;\n",
       "  --sklearn-color-unfitted-level-3: chocolate;\n",
       "  /* Definition of color scheme for fitted estimators */\n",
       "  --sklearn-color-fitted-level-0: #f0f8ff;\n",
       "  --sklearn-color-fitted-level-1: #d4ebff;\n",
       "  --sklearn-color-fitted-level-2: #b3dbfd;\n",
       "  --sklearn-color-fitted-level-3: cornflowerblue;\n",
       "\n",
       "  /* Specific color for light theme */\n",
       "  --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, white)));\n",
       "  --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, black)));\n",
       "  --sklearn-color-icon: #696969;\n",
       "\n",
       "  @media (prefers-color-scheme: dark) {\n",
       "    /* Redefinition of color scheme for dark theme */\n",
       "    --sklearn-color-text-on-default-background: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-background: var(--sg-background-color, var(--theme-background, var(--jp-layout-color0, #111)));\n",
       "    --sklearn-color-border-box: var(--sg-text-color, var(--theme-code-foreground, var(--jp-content-font-color1, white)));\n",
       "    --sklearn-color-icon: #878787;\n",
       "  }\n",
       "}\n",
       "\n",
       "#sk-container-id-2 {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 pre {\n",
       "  padding: 0;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 input.sk-hidden--visually {\n",
       "  border: 0;\n",
       "  clip: rect(1px 1px 1px 1px);\n",
       "  clip: rect(1px, 1px, 1px, 1px);\n",
       "  height: 1px;\n",
       "  margin: -1px;\n",
       "  overflow: hidden;\n",
       "  padding: 0;\n",
       "  position: absolute;\n",
       "  width: 1px;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-dashed-wrapped {\n",
       "  border: 1px dashed var(--sklearn-color-line);\n",
       "  margin: 0 0.4em 0.5em 0.4em;\n",
       "  box-sizing: border-box;\n",
       "  padding-bottom: 0.4em;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-container {\n",
       "  /* jupyter's `normalize.less` sets `[hidden] { display: none; }`\n",
       "     but bootstrap.min.css set `[hidden] { display: none !important; }`\n",
       "     so we also need the `!important` here to be able to override the\n",
       "     default hidden behavior on the sphinx rendered scikit-learn.org.\n",
       "     See: https://github.com/scikit-learn/scikit-learn/issues/21755 */\n",
       "  display: inline-block !important;\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-text-repr-fallback {\n",
       "  display: none;\n",
       "}\n",
       "\n",
       "div.sk-parallel-item,\n",
       "div.sk-serial,\n",
       "div.sk-item {\n",
       "  /* draw centered vertical line to link estimators */\n",
       "  background-image: linear-gradient(var(--sklearn-color-text-on-default-background), var(--sklearn-color-text-on-default-background));\n",
       "  background-size: 2px 100%;\n",
       "  background-repeat: no-repeat;\n",
       "  background-position: center center;\n",
       "}\n",
       "\n",
       "/* Parallel-specific style estimator block */\n",
       "\n",
       "#sk-container-id-2 div.sk-parallel-item::after {\n",
       "  content: \"\";\n",
       "  width: 100%;\n",
       "  border-bottom: 2px solid var(--sklearn-color-text-on-default-background);\n",
       "  flex-grow: 1;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-parallel {\n",
       "  display: flex;\n",
       "  align-items: stretch;\n",
       "  justify-content: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  position: relative;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-parallel-item {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-parallel-item:first-child::after {\n",
       "  align-self: flex-end;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-parallel-item:last-child::after {\n",
       "  align-self: flex-start;\n",
       "  width: 50%;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-parallel-item:only-child::after {\n",
       "  width: 0;\n",
       "}\n",
       "\n",
       "/* Serial-specific style estimator block */\n",
       "\n",
       "#sk-container-id-2 div.sk-serial {\n",
       "  display: flex;\n",
       "  flex-direction: column;\n",
       "  align-items: center;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  padding-right: 1em;\n",
       "  padding-left: 1em;\n",
       "}\n",
       "\n",
       "\n",
       "/* Toggleable style: style used for estimator/Pipeline/ColumnTransformer box that is\n",
       "clickable and can be expanded/collapsed.\n",
       "- Pipeline and ColumnTransformer use this feature and define the default style\n",
       "- Estimators will overwrite some part of the style using the `sk-estimator` class\n",
       "*/\n",
       "\n",
       "/* Pipeline and ColumnTransformer style (default) */\n",
       "\n",
       "#sk-container-id-2 div.sk-toggleable {\n",
       "  /* Default theme specific background. It is overwritten whether we have a\n",
       "  specific estimator or a Pipeline/ColumnTransformer */\n",
       "  background-color: var(--sklearn-color-background);\n",
       "}\n",
       "\n",
       "/* Toggleable label */\n",
       "#sk-container-id-2 label.sk-toggleable__label {\n",
       "  cursor: pointer;\n",
       "  display: flex;\n",
       "  width: 100%;\n",
       "  margin-bottom: 0;\n",
       "  padding: 0.5em;\n",
       "  box-sizing: border-box;\n",
       "  text-align: center;\n",
       "  align-items: start;\n",
       "  justify-content: space-between;\n",
       "  gap: 0.5em;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 label.sk-toggleable__label .caption {\n",
       "  font-size: 0.6rem;\n",
       "  font-weight: lighter;\n",
       "  color: var(--sklearn-color-text-muted);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 label.sk-toggleable__label-arrow:before {\n",
       "  /* Arrow on the left of the label */\n",
       "  content: \"▸\";\n",
       "  float: left;\n",
       "  margin-right: 0.25em;\n",
       "  color: var(--sklearn-color-icon);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 label.sk-toggleable__label-arrow:hover:before {\n",
       "  color: var(--sklearn-color-text);\n",
       "}\n",
       "\n",
       "/* Toggleable content - dropdown */\n",
       "\n",
       "#sk-container-id-2 div.sk-toggleable__content {\n",
       "  max-height: 0;\n",
       "  max-width: 0;\n",
       "  overflow: hidden;\n",
       "  text-align: left;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-toggleable__content.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-toggleable__content pre {\n",
       "  margin: 0.2em;\n",
       "  border-radius: 0.25em;\n",
       "  color: var(--sklearn-color-text);\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-toggleable__content.fitted pre {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 input.sk-toggleable__control:checked~div.sk-toggleable__content {\n",
       "  /* Expand drop-down */\n",
       "  max-height: 200px;\n",
       "  max-width: 100%;\n",
       "  overflow: auto;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {\n",
       "  content: \"▾\";\n",
       "}\n",
       "\n",
       "/* Pipeline/ColumnTransformer-specific style */\n",
       "\n",
       "#sk-container-id-2 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-label.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator-specific style */\n",
       "\n",
       "/* Colorize estimator box */\n",
       "#sk-container-id-2 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-estimator.fitted input.sk-toggleable__control:checked~label.sk-toggleable__label {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-label label.sk-toggleable__label,\n",
       "#sk-container-id-2 div.sk-label label {\n",
       "  /* The background is the default theme color */\n",
       "  color: var(--sklearn-color-text-on-default-background);\n",
       "}\n",
       "\n",
       "/* On hover, darken the color of the background */\n",
       "#sk-container-id-2 div.sk-label:hover label.sk-toggleable__label {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "/* Label box, darken color on hover, fitted */\n",
       "#sk-container-id-2 div.sk-label.fitted:hover label.sk-toggleable__label.fitted {\n",
       "  color: var(--sklearn-color-text);\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Estimator label */\n",
       "\n",
       "#sk-container-id-2 div.sk-label label {\n",
       "  font-family: monospace;\n",
       "  font-weight: bold;\n",
       "  display: inline-block;\n",
       "  line-height: 1.2em;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-label-container {\n",
       "  text-align: center;\n",
       "}\n",
       "\n",
       "/* Estimator-specific */\n",
       "#sk-container-id-2 div.sk-estimator {\n",
       "  font-family: monospace;\n",
       "  border: 1px dotted var(--sklearn-color-border-box);\n",
       "  border-radius: 0.25em;\n",
       "  box-sizing: border-box;\n",
       "  margin-bottom: 0.5em;\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-0);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-estimator.fitted {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-0);\n",
       "}\n",
       "\n",
       "/* on hover */\n",
       "#sk-container-id-2 div.sk-estimator:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-2);\n",
       "}\n",
       "\n",
       "#sk-container-id-2 div.sk-estimator.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-2);\n",
       "}\n",
       "\n",
       "/* Specification for estimator info (e.g. \"i\" and \"?\") */\n",
       "\n",
       "/* Common style for \"i\" and \"?\" */\n",
       "\n",
       ".sk-estimator-doc-link,\n",
       "a:link.sk-estimator-doc-link,\n",
       "a:visited.sk-estimator-doc-link {\n",
       "  float: right;\n",
       "  font-size: smaller;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1em;\n",
       "  height: 1em;\n",
       "  width: 1em;\n",
       "  text-decoration: none !important;\n",
       "  margin-left: 0.5em;\n",
       "  text-align: center;\n",
       "  /* unfitted */\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted,\n",
       "a:link.sk-estimator-doc-link.fitted,\n",
       "a:visited.sk-estimator-doc-link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "div.sk-estimator:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link:hover,\n",
       ".sk-estimator-doc-link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "div.sk-estimator.fitted:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover,\n",
       "div.sk-label-container:hover .sk-estimator-doc-link.fitted:hover,\n",
       ".sk-estimator-doc-link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "/* Span, style for the box shown on hovering the info icon */\n",
       ".sk-estimator-doc-link span {\n",
       "  display: none;\n",
       "  z-index: 9999;\n",
       "  position: relative;\n",
       "  font-weight: normal;\n",
       "  right: .2ex;\n",
       "  padding: .5ex;\n",
       "  margin: .5ex;\n",
       "  width: min-content;\n",
       "  min-width: 20ex;\n",
       "  max-width: 50ex;\n",
       "  color: var(--sklearn-color-text);\n",
       "  box-shadow: 2pt 2pt 4pt #999;\n",
       "  /* unfitted */\n",
       "  background: var(--sklearn-color-unfitted-level-0);\n",
       "  border: .5pt solid var(--sklearn-color-unfitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link.fitted span {\n",
       "  /* fitted */\n",
       "  background: var(--sklearn-color-fitted-level-0);\n",
       "  border: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "\n",
       ".sk-estimator-doc-link:hover span {\n",
       "  display: block;\n",
       "}\n",
       "\n",
       "/* \"?\"-specific style due to the `<a>` HTML tag */\n",
       "\n",
       "#sk-container-id-2 a.estimator_doc_link {\n",
       "  float: right;\n",
       "  font-size: 1rem;\n",
       "  line-height: 1em;\n",
       "  font-family: monospace;\n",
       "  background-color: var(--sklearn-color-background);\n",
       "  border-radius: 1rem;\n",
       "  height: 1rem;\n",
       "  width: 1rem;\n",
       "  text-decoration: none;\n",
       "  /* unfitted */\n",
       "  color: var(--sklearn-color-unfitted-level-1);\n",
       "  border: var(--sklearn-color-unfitted-level-1) 1pt solid;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 a.estimator_doc_link.fitted {\n",
       "  /* fitted */\n",
       "  border: var(--sklearn-color-fitted-level-1) 1pt solid;\n",
       "  color: var(--sklearn-color-fitted-level-1);\n",
       "}\n",
       "\n",
       "/* On hover */\n",
       "#sk-container-id-2 a.estimator_doc_link:hover {\n",
       "  /* unfitted */\n",
       "  background-color: var(--sklearn-color-unfitted-level-3);\n",
       "  color: var(--sklearn-color-background);\n",
       "  text-decoration: none;\n",
       "}\n",
       "\n",
       "#sk-container-id-2 a.estimator_doc_link.fitted:hover {\n",
       "  /* fitted */\n",
       "  background-color: var(--sklearn-color-fitted-level-3);\n",
       "}\n",
       "</style><div id=\"sk-container-id-2\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>LGBMClassifier(max_depth=3, n_jobs=-1, random_state=42)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator fitted sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-2\" type=\"checkbox\" checked><label for=\"sk-estimator-id-2\" class=\"sk-toggleable__label fitted sk-toggleable__label-arrow\"><div><div>LGBMClassifier</div></div><div><span class=\"sk-estimator-doc-link fitted\">i<span>Fitted</span></span></div></label><div class=\"sk-toggleable__content fitted\"><pre>LGBMClassifier(max_depth=3, n_jobs=-1, random_state=42)</pre></div> </div></div></div></div>"
      ],
      "text/plain": [
       "LGBMClassifier(max_depth=3, n_jobs=-1, random_state=42)"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_features = clean_names(train_features)\n",
    "test_features = clean_names(test_features)\n",
    "\n",
    "from lightgbm import LGBMClassifier\n",
    "lgbm_model = LGBMClassifier(\n",
    "    n_estimators=100,      # 对应 max_iter，树的个数\n",
    "    learning_rate=0.1,     # 学习率\n",
    "    max_depth=3,           # 树的最大深度\n",
    "    random_state=42,       # 保证结果可复现\n",
    "    n_jobs=-1              # 使用所有 CPU 核心加速\n",
    ")\n",
    "lgbm_model.fit(train_features, train_labels)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "id": "2fcd0bdb",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAvIAAAIpCAYAAADAYtygAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8ekN5oAAAACXBIWXMAAA9hAAAPYQGoP6dpAADTGElEQVR4nOzdd1yV9f//8ccBFCeQkisHkigg44A4cqLiKlMx0MxyZWbulYlZ2vq4t/Wp7JNaOUCcqaVpahbixoVamseZpqjgCtf5/cGP6+uJLZaiz/vtdm6feF/vdV0HP7zO+7yu92WyWq1WREREREQkT7F70BMQEREREZGcUyAvIiIiIpIHKZAXEREREcmDFMiLiIiIiORBCuRFRERERPIgBfIiIiIiInmQAnkRERERkTxIgbyIiIiISB6kQF5EREREJA9SIC8iIiIikgcpkBcRkTzHYrFgMpkyff0bNmzYgMlkYtSoUf/KePfL7NmzMZlMjBkz5kFP5R8zatQoTCYTGzZseNBTEfnHODzoCYiIiNyrp59+mpdffvlBT0NE5IFQIC8iInlWpUqV8txquIjI/aLUGhEReeTt2bOHF198kdKlS5M/f34qVKhA3759SUhISFP3yy+/pHXr1ri5uVGgQAGKFStGs2bNWL9+vU29UaNG0bBhQwDee+89m7Qei8UCQHBwcIZpPl26dLGpC/+X8jJ79mxWrlxJvXr1KFq0KG5ubkadGzduMGnSJAIDAylcuDBFixalXr16LF++PHcXCXBzc8PNzY3ExETeeOMNSpcuTeHChalfvz47d+4E4MyZM3Tu3JkSJUpQqFAhmjVrxuHDh9P0ZTKZCA4O5sSJE7Rv357ixYtTuHBhgoODiYmJSXf8hIQEBg4cSMWKFXF0dKREiRK0b9+e+Pj4NHVTr9/vv//O5MmTqVq1Ko6OjnTp0oXg4GDee+89ABo2bGi8L3dfx/Xr19OtWzeqVKlCkSJFKFKkCEFBQXz++efpzi31fM6dO0e3bt0oUaIEBQsWpFatWhmm71y+fJn3338fPz8/ChcujLOzMwEBAbzzzjvcvHnTpu7Ro0fp3r075cuXx9HRkdKlS9OlSxeOHTuWbt8ioBV5ERF5xC1fvpx27dphb29Pq1atKFeuHPHx8cyYMYPVq1ezZcsWnnjiCaN+79698ff3JyQkhCeffJJTp06xdOlSQkJCWLx4Ma1btwZSgnSLxcKcOXNo0KABwcHBRh8uLi65mvPChQtZs2YNLVu2pFevXly+fBmA5ORkmjdvzoYNGwgICODVV1/l5s2brFy5ktatWzN9+nT69OmTq7Fv3LhBkyZN+Ouvv2jfvj1nz54lKiqKkJAQYmJiaN68OaVKleLll1/m8OHDfPvtt7Rs2ZL9+/djb29v09fFixepU6cOpUuXpkePHpw6dYrIyEgaNmzI6tWrba5ZQkICtWrV4vDhwwQHB/Piiy9isViIjo5m5cqV/PDDDzzzzDNp5tu3b19iY2N57rnnaNmyJSVLljT63bhxI507dzYC+Lvfl7Fjx3L48GFq1apFaGgoly5d4vvvv+f111/n0KFDTJw4Mc1Yly5dok6dOjg5OdGxY0f+/PNPIiMjadasGTt27MDHx8eoe/78eRo0aEB8fDxms5mePXty584dDh48yNixYxk8eLAxny1bttCsWTOuXr3K888/T6VKlbBYLMydO5fvvvuOzZs34+7ufm9vqDzarCIiInnM0aNHrYD16aefto4cOTLNa/PmzVar1Wo9f/681cnJyVq2bFnrsWPHbPqYN2+eFbD26dPHpvz3339PM97p06etZcqUsXp4eNiUr1+/3gpYR44cme48GzRoYM3oT23nzp2tgPXo0aNG2axZs6yA1WQyWX/44Yc0bYYPH24FrKNGjbLeuXPHKE9KSrIGBQVZ8+fPbz116lS6490tdZzRo0fblFeoUMEKWMPDw603b940yseMGWMFrC4uLtaBAwfajP3GG29YAevixYtt+gKsgPWVV16xqb9hwwaryWSyVqpUyXr79m2jvFu3blbAGhERYdPP999/bwWsHh4eNvVTr196763VarWOHDnSCljXr1+f7jVI732+efOmtUmTJlZ7e/s0faaeT69evWzm8cUXX1gB6+uvv25TPzw83ApYhw8fnmacM2fOGNf3xo0bVjc3N2vRokWtcXFxNvU2bdpktbe3t7Zs2TLdcxBRIC8iInlOaiCf0Wvy5MlWq9VqnTRpkhWwfv311+n2ExgYaHV1dc3WmH379rUCVovFYpT9U4F8aGhomvq3b9+2PvHEE9ZKlSrZBMapli9fbgWs06dPz/Jcsgrk7z5Hq9VqPX78uBWwFilSxHrlyhWbYz/99FO61wCw2tvbW48fP55m/Oeee84KWDdt2mS1Wq3W5ORka8GCBa3Fixe3Xr16NU39Zs2a2dS3Wv/v+k2dOjXdc8wqkM/IokWLrIB19uzZac6ncOHC1suXL9uU37x50+rg4GANDAw0ys6cOWM1mUzWp59+2nrjxo1Mx1u8eLEVsH7wwQfpHm/btq3Vzs7OmpiYmKPzkMeDUmtERCTPatasGd9//32Gx2NjY43/TS+P+6+//uL8+fOcP38eV1dXAH7//XdGjx7Njz/+yKlTp0hOTrZpc/r0aSpUqHAfzyKtGjVqpCk7dOgQFy9epEyZMkb+993OnTsHwMGDB3M1touLS5rzK126NAAeHh4ULlw43WOnTp1K01eFChUoV65cmvJ69eqxcuVK4uLiqFu3LgcPHuT69esEBwdTqFChNPWDg4NZvXq1Uf9u6V2r7Lh8+TITJkxg6dKlHDlyhKtXr9ocP336dJo2Hh4eFClSxKbMwcGBkiVLcunSJaNs+/btWK1WGjZsSL58+TKdR+rv6MGDB9O9cfvMmTPcuXOHX3/9laCgoGyenTwuFMiLiMgj68KFCwB8/PHHmda7evUqrq6uHD58mBo1apCUlETDhg15/vnncXJyws7Ojg0bNrBx48Y0gf0/oWTJkmnKUs9l//797N+/P8O2fw9Ic8rZ2TlNmYNDSrjg5OSU4bG/37wJUKJEiXTHSD2/xMREAJKSkmzK/65UqVI29dPrKydu3LhBcHAwO3fuJCAggFdeeYXixYvj4OBg3PeQ3vuc3rWBlGtw+/Zt4+fUoP6pp57Kci6p7+vcuXMzrZfb91UeTQrkRUTkkZUaeO7du9fmRsSMTJ48mYsXL/LNN9/QsWNHm2M9e/Zk48aNORrfzi5lc7hbt24ZAW+q9ILSVOntdJN6Li+88ALR0dE5mseD8ueff6ZbfvbsWeD/AuPUc0stz6h+eh8k7uXhX8uWLWPnzp10796dmTNn2hxbsGABc+bMyXGfd0u9iTW9byn+LvWcUm8aFskJbT8pIiKPrJo1awKwefPmbNU/cuQIAK1atbIpv3PnDr/88kua+qm7tNy9Gnu31N1w/h7Q3blzh927d2drTqm8vLxwcnJi+/bt6a5+P4yOHTvGiRMn0pRv2rQJALPZDICnpycFChRg27ZtXLt2LU391A9QqfWzI7P3JqP3+e655UZQUBB2dnasX78+y/cqp7+jIndTIC8iIo+srl27UrRoUd5+++1001GuXbtm5CgDRm74zz//bFNv7Nix7Nu3L037YsWKAXDy5Ml0x0/NaZ49e7ZN+aRJkzh69Gj2T4SU9I033niDY8eOMWTIkHQDxH379mW4Cv4g3L59m7fffhur1WqUbdy4kVWrVlGpUiVq164NQP78+enQoQPnz59n9OjRNn2sXbuW7777jkqVKlGnTp1sj53Ze5PR+7xx48Y0K/T3omTJkrzwwgscOXIk3fsZ/vzzT27dugVA69atKV++PJMmTeKnn35KU/fmzZtp5imSSqk1IiLyyHryySeZP38+4eHh+Pv707x5czw9Pfnrr784duwYGzdupHbt2sYNsz179mTWrFm0bdvWeIhRbGwsO3fu5LnnnmPlypU2/Xt6elKmTBkWLFhAoUKFKFu2LCaTiTfeeANnZ2e6du3KuHHjGDVqFHFxcTz99NNs376dffv20aBBgxyn6rz33nvs3LmTadOmsXLlSho0aGDsdb937152797N5s2bM8xN/7f5+fmxYcMGatWqRaNGjTh9+jQLFiwgX758zJw500g9gpQPSxs3buTDDz8kJiaGmjVrGvvIFypUiFmzZtnUz0rqg6DefvttDh48iLOzM87Ozrzxxhs8//zzuLm5MW7cOPbt24ePjw+HDh1ixYoVtGnThkWLFuX63D/55BP27dvHRx99xKpVq2jUqBFWq5Vff/2VNWvWcPbsWVxcXHB0dCQ6OpoWLVrQoEEDGjdubKSBHT9+nE2bNlG8ePFc38Qsj6gHvGuOiIhIjqVuP9msWbNs1T948KD11VdftVaoUMGaP39+6xNPPGH19fW19uvXz7p161abuuvXr7fWqVPHWrRoUauLi4v12Wefte7YsSPD7QxjY2OtDRo0sBYtWtTY/vLuLSV37txpbdy4sbVQoUJWJycna+vWra2//fZbpttPzpo1K8NzuXXrlvWzzz6z1qlTx+rk5GR1dHS0li9f3tq8eXPrf//73zTbQ6Yns+0nK1SokG4bwNqgQYM05anvRefOndOtf+zYMWt4eLj1iSeesBYsWNBav359688//5zuGOfOnbP269fPWqFCBWu+fPmsrq6u1rCwMOvevXvT1E3v+v3d7Nmzrb6+vlZHR0crYHNuv//+u/WFF16wPvnkk9ZChQpZq1evbl2wYEGGW4pmdP5Wa8bXLTEx0frOO+9YPT09rY6OjlZnZ2er2Wy2vvvuu2m2pTx58qS1f//+Vg8PD6ujo6PVycnJ6uXlZe3evbt13bp1GZ6jPN5MVutd33eJiIiI3Acmk4kGDRqwYcOGBz0VkUeWcuRFRERERPIgBfIiIiIiInmQAnkRERERkTxIu9aIiIjIfadb8ET+eVqRFxERERHJg7QiLyJ53p07dzh9+jRFixa9p8e1i4iIPCysViuXL1+mTJkyWT47QYG8iOR5p0+fply5cg96GiIiIvfNiRMnKFu2bKZ1FMiLSJ5XtGhRIOX/9JycnB7wbERERO5dUlIS5cqVM/62ZUaBvIjkeanpNE5OTgrkRUTkkZCdVFEF8iLyyKg/Yj72jgUf9DREROQxsmN8pwc2tnatERERERHJgxTIi4iIiIjkQQrkRURERETyIAXyIiIiIiJ5kAJ5EREREZE8SIG8yAPi4OCA2Ww2XnPnziUxMZFKlSpx6tQpABISEnB3dycyMtKolz9/fvz8/DCbzYwfPz7dvn/55Rdq1qyJv78/Xl5efPbZZ8axqVOnUrlyZSpXrkyzZs2wWCwAWCwWgoKCbPrp0qULK1asAMDNzQ0/Pz/8/Pxo0KABx44dM+p99tlneHl54ePjQ0BAANHR0QAEBwfj6elpzH3gwIEZXo+XXnqJKlWq4OPjQ0RERM4vqIiIyGNG20+KPCAuLi7ExcWlKR8xYgSDBw9mwYIFDB8+nH79+tG+fXvat28PpATUMTExFClSJMO+u3fvzvLly/Hw8ODGjRtGsL548WIWLFjA9u3bcXJyYuHChYSGhrJz585szTl13BEjRvDhhx8yc+ZMli1bxpdffsmmTZtwdXXl0qVLfPfdd0ab6OhofHx8suy7U6dOzJs3j1u3btGkSRN+/PFHGjVqlG7d5ORkkpOTjZ+TkpKyNX8REZFHiVbkRR4ynTt35vjx40ycOJHY2Fj69OmT4z7OnTtHsWLFAMifPz+VK1cGYOLEiYwZM8Z4aFJ4eDjOzs6sXbs2R/3XrVuXkydPAjB+/HjGjRuHq6srkPIBpUOHDjmec/PmzYGUbyp8fX2NbyXSM3r0aJydnY1XuXLlcjyeiIhIXqdAXuQBuXTpkk1qzaZNm4CUJ7lNmDCBIUOGMGnSJBwccv7FWe/evfHw8CA8PJw5c+Zw+/ZtAOLj4wkICLCpGxAQwIEDB3LU/6pVq2jVqpXRp9lszrBuWFiYcY6TJ0/Osu+kpCRWrlxJcHBwhnUiIiJITEw0XidOnMjR/EVERB4FSq0ReUAySq0BWL16NaVKlWL//v00btw4x32/9957vPTSS3z//fdMnjyZtWvX8vXXX2faJqNHQd9dXrt2bc6fP4+TkxMfffRRlm0h+6k1AFarlS5dutCrV69MV9kdHR1xdHTMVp8iIiKPKq3Iizxkjh49SmRkJFu2bGHKlCmcP3/+nvqpUqUK/fv354cffmDJkiUAeHl5pcmH37VrF15eXhQvXpyLFy/aHLtw4YKRMgMpOfLHjx/Hz8+PkSNHGn3u2rXrnub4d0OHDqVYsWIMHjz4vvQnIiLyKFMgL/KQGTBgAB988AHly5dnwIABDB8+PMd9rFq1CqvVCsDevXupUKECAIMGDSIiIsK4OXThwoVcvHiRxo0bU6RIEVxcXIiJiQHg5MmT7N27l6pVq9r07eDgwOTJk5kzZw4XL17kzTffZNiwYSQkJAApKUNz5szJ8Zw//fRT4uLi+O9//5vjtiIiIo8jpdaIPCCpOfKpunXrRpUqVbh8+TLh4eEA9OrVi6CgIHbs2EG1atWy3fecOXMYOHAgBQsWpGDBgsyaNQtIyVc/fvw41apVw2QyUaFCBZYsWYKdnZ3RrlevXiQlJeHg4MBnn32W7u44Tz31FB06dODzzz/nrbfe4vTp09SpUwcHBwccHR15++23jbphYWEUKFAAAD8/P7766qt059ynTx8qVqxI9erVAejfvz9du3bN9jmLiIg8bkzW1GU7EZE8KikpCWdnZ/z7foq9Y8EHPR0REXmM7Bjf6b72l/o3LTEx0dhlLiNKrRERERERyYOUWiOSh3300UcsXLjQpmzQoEF06nR/Vwfup969e/PLL7/YlE2YMIGQkJAHNCMREZG8Sak1IpLn5eRrSBERkYeZUmtERERERB5xCuRFRERERPIgBfIiIiIiInmQbnYVkUdG/RHztf2kiMgj7H5v9ZjXaUVeRERERCQPUiAvIiIiIpIHKZAXEREREcmDFMiLiIiIiORBCuRFRERERPIgBfIiIiIiInmQAnmRB8TBwQGz2Wy85s6dS2JiIpUqVeLUqVMAJCQk4O7uTmRkpFEvf/78+Pn5YTabGT9+fLp9//LLL9SsWRN/f3+8vLz47LPPjGNTp06lcuXKVK5cmWbNmmGxWACwWCwEBQXZ9NOlSxdWrFgBgJubG35+fvj5+dGgQQOOHTtm1Pvss8/w8vLCx8eHgIAAoqOjAQgODsbT09OY+8CBAzO8Hi+//DL+/v74+vryxhtvcOfOnZxfVBERkceI9pEXeUBcXFyIi4tLUz5ixAgGDx7MggULGD58OP369aN9+/a0b98eSAmoY2JiKFKkSIZ9d+/eneXLl+Ph4cGNGzeMYH3x4sUsWLCA7du34+TkxMKFCwkNDWXnzp3ZmnPquCNGjODDDz9k5syZLFu2jC+//JJNmzbh6urKpUuX+O6774w20dHR+Pj4ZNn3J598gpOTEwDt27dn2bJlhIaGpls3OTmZ5ORk4+ekpKRszV9ERORRohV5kYdM586dOX78OBMnTiQ2NpY+ffrkuI9z585RrFgxAPLnz0/lypUBmDhxImPGjDEC5vDwcJydnVm7dm2O+q9bty4nT54EYPz48YwbNw5XV1cg5QNKhw4dcjzn1DndunWL69evYzKZMqw7evRonJ2djVe5cuVyPJ6IiEhep0Be5AG5dOmSTWrNpk2bADCZTEyYMIEhQ4YwadIkHBxy/sVZ79698fDwIDw8nDlz5nD79m0A4uPjCQgIsKkbEBDAgQMHctT/qlWraNWqldGn2WzOsG5YWJhxjpMnT86037CwMEqWLEmRIkWM/tMTERFBYmKi8Tpx4kSO5i8iIvIoUGqNyAOSUWoNwOrVqylVqhT79++ncePGOe77vffe46WXXuL7779n8uTJrF27lq+//jrTNhmtgN9dXrt2bc6fP4+TkxMfffRRlm0h+6k1qXVv3LhB586dWbduHU2aNEm3nqOjI46OjtnqU0RE5FGlFXmRh8zRo0eJjIxky5YtTJkyhfPnz99TP1WqVKF///788MMPLFmyBAAvL680+fC7du3Cy8uL4sWLc/HiRZtjFy5cMFJmICVH/vjx4/j5+TFy5Eijz127dt3THNOTP39+2rRpw9KlS+9bnyIiIo8iBfIiD5kBAwbwwQcfUL58eQYMGMDw4cNz3MeqVauwWq0A7N27lwoVKgAwaNAgIiIijJtDFy5cyMWLF2ncuDFFihTBxcWFmJgYAE6ePMnevXupWrWqTd8ODg5MnjyZOXPmcPHiRd58802GDRtGQkICkJIyNGfOnBzN99atW8YNubdv32blypV4enrm+LxFREQeJ0qtEXlAUnPkU3Xr1o0qVapw+fJlwsPDAejVqxdBQUHs2LGDatWqZbvvOXPmMHDgQAoWLEjBggWZNWsWkJKDfvz4capVq4bJZKJChQosWbIEOzs7o12vXr1ISkrCwcGBzz77LN3dcZ566ik6dOjA559/zltvvcXp06epU6cODg4OODo68vbbbxt1w8LCKFCgAAB+fn589dVXafq7ffs2HTp04MqVK1itVurXr0/Pnj2zfb4iIiKPI5M1ddlORCSPSkpKwtnZGf++n2LvWPBBT0dERP4hO8Z3etBT+Mel/k1LTEw0dnTLiFJrRERERETyIKXWiORhH330EQsXLrQpGzRoEJ06PbwrFr179+aXX36xKZswYQIhISEPaEYiIiJ5k1JrRCTPy8nXkCIiIg8zpdaIiIiIiDziFMiLiIiIiORBCuRFRERERPIg3ewqIo+M+iPma/tJEZGH3OOwheS/RSvyIiIiIiJ5kAJ5EREREZE8SIG8iIiIiEgepEBeRERERCQPUiAvIiIiIpIHKZAXEREREcmDFMiLPCAODg6YzWbjNXfuXBITE6lUqRKnTp0CICEhAXd3dyIjI416+fPnx8/PD7PZzPjx49Pt+5dffqFmzZr4+/vj5eXFZ599ZhybOnUqlStXpnLlyjRr1gyLxQKAxWIhKCjIpp8uXbqwYsUKANzc3PDz88PPz48GDRpw7Ngxo95nn32Gl5cXPj4+BAQEEB0dDUBwcDCenp7G3AcOHJjh9fjwww8pX748rq6uOb+YIiIijyHtIy/ygLi4uBAXF5emfMSIEQwePJgFCxYwfPhw+vXrR/v27Wnfvj2QElDHxMRQpEiRDPvu3r07y5cvx8PDgxs3bhjB+uLFi1mwYAHbt2/HycmJhQsXEhoays6dO7M159RxR4wYwYcffsjMmTNZtmwZX375JZs2bcLV1ZVLly7x3XffGW2io6Px8fHJsu9mzZrx6quv4uvrm2Xd5ORkkpOTjZ+TkpKyNX8REZFHiVbkRR4ynTt35vjx40ycOJHY2Fj69OmT4z7OnTtHsWLFAMifPz+VK1cGYOLEiYwZMwYnJycAwsPDcXZ2Zu3atTnqv27dupw8eRKA8ePHM27cOGMl3cXFhQ4dOuR4ztWrV6d06dLZqjt69GicnZ2NV7ly5XI8noiISF6nQF7kAbl06ZJNas2mTZsAMJlMTJgwgSFDhjBp0iQcHHL+xVnv3r3x8PAgPDycOXPmcPv2bQDi4+MJCAiwqRsQEMCBAwdy1P+qVato1aqV0afZbM6wblhYmHGOkydPztmJZCAiIoLExETjdeLEifvSr4iISF6i1BqRBySj1BqA1atXU6pUKfbv30/jxo1z3Pd7773HSy+9xPfff8/kyZNZu3YtX3/9daZtTCZTluW1a9fm/PnzODk58dFHH2XZFrKfWpMTjo6OODo63tc+RURE8hqtyIs8ZI4ePUpkZCRbtmxhypQpnD9//p76qVKlCv379+eHH35gyZIlAHh5eaXJh9+1axdeXl4UL16cixcv2hy7cOGCzc2nMTExHD9+HD8/P0aOHGn0uWvXrnuao4iIiNw7BfIiD5kBAwbwwQcfUL58eQYMGMDw4cNz3MeqVauwWq0A7N27lwoVKgAwaNAgIiIijJtDFy5cyMWLF2ncuDFFihTBxcWFmJgYAE6ePMnevXupWrWqTd8ODg5MnjyZOXPmcPHiRd58802GDRtGQkICkJIyNGfOnHs+fxEREckepdaIPCCpOfKpunXrRpUqVbh8+TLh4eEA9OrVi6CgIHbs2EG1atWy3fecOXMYOHAgBQsWpGDBgsyaNQtIyVc/fvw41apVw2QyUaFCBZYsWYKdnZ3RrlevXiQlJeHg4MBnn32W7u44Tz31FB06dODzzz/nrbfe4vTp09SpUwcHBwccHR15++23jbphYWEUKFAAAD8/P7766qt05zxq1Ci++OILLl68SNmyZRk6dCj9+vXL9jmLiIg8bkzW1GU7EZE8KikpCWdnZ/z7foq9Y8EHPR0REcnEjvGdHvQUHmqpf9MSExONXeYyotQaEREREZE8SKk1InnYRx99xMKFC23KBg0aRKdOD+9qR+/evfnll19syiZMmEBISMgDmpGIiEjepNQaEcnzcvI1pIiIyMNMqTUiIiIiIo84BfIiIiIiInmQAnkRERERkTxIN7uKyCOj/oj52n5S5D7TVoEiDy+tyIuIiIiI5EEK5EVERERE8iAF8iIiIiIieZACeRERERGRPEiBvIiIiIhIHqRAXkREREQkD1IgL5JHOTg4EBAQgLe3N9WqVWPmzJk2x61WK+XLl2fo0KEALFiwgFatWhnHz58/j4eHBxcuXGDZsmUEBATg7++Pj48Py5Yty3Dc4OBgPD09MZvNVK1alXnz5hnHXF1dAbBYLBQqVAiz2YyXlxedOnUiOTkZs9mM2WymVKlSlC1bFrPZTL169bBYLAQFBdmMExYWxoYNG3J7mURERB5Z2kdeJI9ycXFh165dABw7dozQ0FCsVis9evQAYPPmzZQuXZrFixczbtw4XnzxRWbOnMl3331HixYtGD58OG+++SZFixalb9++7NixgyeffJIrV65w7ty5TMeOjo7Gx8eHM2fOYDabeemll9LU8fb2Zvv27dy6dYuQkBCio6OJi4sDYNSoUbi6utKnTx8gJfDPieTkZJKTk42fk5KSctReRETkUaAVeZFHQIUKFZg4cSKffPKJURYZGUnPnj2pWLEiW7ZsAWDGjBkMHTqU2NhYdu/eTffu3bl8+TK3b9+maNGiABQpUoSKFStma9wrV64Y7TLi4OBArVq1OHny5D2eXVqjR4/G2dnZeJUrV+6+9S0iIpJXKJAXeUQEBgZy6NAhICWt5ttvvyU0NJTw8HAiIyMB8PLyokWLFjRv3pzp06djZ2dHsWLFCAkJwc3NjVdeeYXFixdnOVZYWBi+vr74+Pjw/vvvZ1r3+vXrxMTE0KxZs0zrxcfHG6k3ZrOZtWvXZlg3IiKCxMRE43XixIks5ywiIvKoUSAv8oiwWq3Gf2/atAlvb29cXFxo27YtS5YsMY737NkTd3d3atSoYdSfM2cO3333HT4+Prz55pu88847mY4VHR3N3r17OXDgAMOGDePy5ctp6qQG5iVLljTy4TPj7e1NXFyc8QoJCcmwrqOjI05OTjYvERGRx40CeZFHRFxcHJ6engBERUURGxuLm5sbQUFBnDlzhpiYGADs7Oyws0v7Tz8gIIC33nqLBQsWsGTJkmyNWbFiRUqXLs2BAwfSHEsNzH///Xe2bdvG8uXLc3F2IiIi8ncK5EUeASdOnGDIkCH06dOHO3fusHTpUg4dOoTFYsFisTB16lSioqLSbXvlyhV++ukn4+e9e/dSoUKFbI17/vx5fv/9d8qXL59hHVdXV0aPHs3YsWNzdlIiIiKSKQXyInnUpUuXMJvNeHt706ZNG3r27Mmrr77Kxo0b8fDwoHjx4kbd1q1bs3jxYu7cuZOmH6vVyujRo40tJb/55humTJmS6dhhYWGYzWbq16/PRx99RKlSpTKt/8ILL3D58mXjplsRERHJPZP17sRaEZE8KCkpCWdnZ/z7foq9Y8EHPR2RR8qO8Z0e9BREHiupf9MSExOzvAdMK/IiIiIiInmQHgglIunq3bs3v/zyi03ZhAkTMt1NRkRERP49Sq0RkTwvJ19DioiIPMyUWiMiIiIi8ohTIC8iIiIikgcpkBcRERERyYN0s6uIPDLqj5iv7SflvtCWiyKSF2hFXkREREQkD1IgLyIiIiKSBymQFxERERHJgxTIi4iIiIjkQQrkRURERETyIAXyIiIiIiJ5kAJ5kTzmiy++wNHRkUuXLgEwe/ZsTCYTW7duNeqMHTsWk8mExWKhWbNmmM1mypcvT4kSJTCbzZjNZi5fvpxu//v27aNx48Z4eHjg5eVFr169uHnzJrNnzzbaV6lShcmTJxttRo0aRdmyZY2+69WrZ8wttY2HhwetWrViz549Rrvg4GD27duX4zmKiIiI9pEXyXOioqIICgpiyZIldO3aFQAfHx+ioqKoUaMGAMuXL8fDwwOA1atXAylB9b59+5gwYUKGfV+7do3WrVvz6aef0qRJE6xWK/PmzSM5ORmATp06MWHCBM6dO4enpydhYWGUK1cOgGHDhtGnT580faa2AViyZAkhISHs3buXkiVLGnVyMkeA5ORkY04ASUlJmdYXERF5FGlFXiQPOX/+PL///jtjx44lKirKKG/cuDHr16/HarVy5MgRSpQogZOTU477nzdvHvXr16dJkyYAmEwmOnbsSJEiRWzqPfnkk3h4eHD69Okc9R8aGkqLFi2YP39+jud2t9GjR+Ps7Gy8Uj9MiIiIPE4UyIvkIYsWLaJt27bUqVOHgwcPkpCQAICDgwOBgYFs2bKFqKgowsPD76n/+Ph4zGZzlvUsFgtXr17F39/fKBszZoyREtOxY8cM2wYGBnLo0KF7ml+qiIgIEhMTjdeJEydy1Z+IiEhepEBeJA+JjIykXbt2mEwmWrduzZIlS4xj7dq1Iyoqim+//Zbnn3/+nvq3Wq2YTKYMj3/11Vf4+vri4eFB3759KVCggHFs2LBhxMXFERcXx9y5czMdI7ccHR1xcnKyeYmIiDxuFMiL5BFnz54lJiaGsLAw3NzciIyMJDIy0jjesGFDVqxYQalSpShatOg9jeHt7c2uXbsyPN6pUyf27t1LTEwMQ4cO5cyZMzkeIy4uDk9Pz3uan4iIiPwfBfIieUR0dDRvvPEGFosFi8XC6dOn+fXXXzl37hyQkl4zZswYhg8ffs9jdOzYkY0bN7Ju3TogZfX8f//7H1euXLGpV716dV5++WWmT5+eo/6XLVvG999/T4cOHe55jiIiIpJCgbxIHhEVFUWbNm2Mn00mEy1btrRZfW/bti1BQUH3PEahQoVYunQpH374IZUrV6Zq1ars2rULR0fHNHWHDRvG7NmzuXbtGmCbI282m7lx4waQko5jNpupXLky//vf//jhhx8oUaLEPc9RREREUpis9yNhVUTkAUpKSsLZ2Rn/vp9i71jwQU9HHgE7xnd60FMQkcdU6t+0xMTELO8B04q8iIiIiEgepAdCiTyGEhISaNy4sU1ZoUKFiImJeUAzuj9++rCDdrAREZHHhgJ5kcdQ8eLFiYuLe9DTEBERkVxQao2IiIiISB6kQF5EREREJA9SIC8iIiIikgcpR15EHhn1R8zX9pOPGW0TKSKPM63Ii4iIiIjkQQrkRURERETyIAXyIiIiIiJ5kAJ5EREREZE8SIG8iIiIiEgepEBeRERERCQPUiAv8g9xcHAgICAAb29vqlWrxsyZM22OW61Wypcvz9ChQwFYsGABrVq1Mo6fP38eDw8PLly4wLJlywgICMDf3x8fHx+WLVuW4bjBwcF4enpiNpsxm80MHDgQgC5duvDEE09w48YNo26LFi1wc3MDwGKxUKhQIcxmM97e3vTs2ZM7d+5gsVgICgpKM05cXBy1a9emSpUqeHl5sWDBAm7duoW/vz/bt2836o0YMYL33nuPDRs24OLiYszLbDZz6NAhY9yAgACqVKlC7dq1Wbx4cc4vuIiIyGNG+8iL/ENcXFzYtWsXAMeOHSM0NBSr1UqPHj0A2Lx5M6VLl2bx4sWMGzeOF198kZkzZ/Ldd9/RokULhg8fzptvvknRokXp27cvO3bs4Mknn+TKlSucO3cu07Gjo6Px8fFJU162bFnWrFlDy5YtuXDhQpp+vL292b59O7du3aJRo0YsXbqUwMDANP1cvXqV0NBQvvrqK+rVq8fp06dp2LAhHh4eTJ8+nb59+xITE4PFYmHJkiXs2LGD2NhYQkJCiI6OtunLYrEY4wLs3buXVq1aUaRIEZo2bZru+SUnJ5OcnGz8nJSUlOn1EBEReRRpRV7kX1ChQgUmTpzIJ598YpRFRkbSs2dPKlasyJYtWwCYMWMGQ4cOJTY2lt27d9O9e3cuX77M7du3KVq0KABFihShYsWK9zSPdu3aERUVBcCSJUto06ZNuvUcHByoXbs2hw8fTvf4vHnzCA4Opl69egCUKVOGiIgIJk2aRP369Xn66af56quvGDhwIGPGjKFAgQLZnqOvry/vvvuuzbX6u9GjR+Ps7Gy8ypUrl+3+RUREHhUK5EX+JYGBgRw6dAhISav59ttvCQ0NJTw8nMjISAC8vLxo0aIFzZs3Z/r06djZ2VGsWDFCQkJwc3PjlVdeyVbaSVhYmJG+MnnyZKM8ICCAAwcOkJycTHR0NGFhYem2v3btGuvWrcPX1zfd4/Hx8QQEBNiUpfYNMGHCBN5++21u3brF888/b9RZu3atTWrN9evXs7xW6YmIiCAxMdF4nThxIsO6IiIijyql1oj8S6xWq/HfmzZtwtvbGxcXF9q2bUv16tWZOHEiJpOJnj17snbtWmrUqGHUnzNnDrt27WLNmjW8+eab7Nq1iw8++CDDsTJKrQFo2rQp8+bN46+//qJs2bI2x+Lj4zGbzdjZ2dG6dWtatGiBxWLJ8bmWKlWKRo0apfmgkF5qTXruvlbpcXR0xNHRMcfzEhEReZQokBf5l8TFxeHp6QlAVFQUsbGxxo2mZ8+eJSYmhjp16mBnZ4edXdovywICAggICKBRo0Z07do100A+M+Hh4TRq1IiRI0emOXZ3rnpmvLy8iImJsSnbtWsXXl5exs8ZnUd23H2tREREJH1KrRH5F5w4cYIhQ4bQp08f7ty5w9KlS40dWywWC1OnTjVy1//uypUr/PTTT8bPe/fupUKFCvc8F7PZzLvvvkuHDh3uuY+OHTuyfv16Nm3aBMDp06f5z3/+w4ABA+65z1T79+/ngw8+oFevXrnuS0RE5FGmFXmRf8ilS5cwm83cuHGDggUL8sYbb/Dqq6+yfv16PDw8KF68uFG3devWfPDBBzb57KmsViujR4+mR48eFChQgGLFivHZZ59lOnZYWJhxg6mfnx9fffWVzfHUgPvKlSvZOpc9e/bYpOEsXLiQxYsX06dPHy5cuICdnR0jR46kevXqmfaTmiOf6pNPPqFMmTJGzv3169cpXrw4EydOpEmTJtmam4iIyOPKZM0qGVVE5CGXlJSEs7Mz/n0/xd6x4IOejvyLdozv9KCnICJyX6X+TUtMTMTJySnTukqtERERERHJg5RaI5JH9e7dm19++cWmbMKECYSEhDygGT14P33YIcvVCxERkUeFAnmRPOrjjz9+0FMQERGRB0ipNSIiIiIieZACeRERERGRPEiBvIiIiIhIHqQceRF5ZNQfMV/bTz5mtP2kiDzOtCIvIiIiIpIHKZAXEREREcmDFMiLiIiIiORBCuRFRERERPIgBfIiIiIiInmQAnl5KJlMJnr16mX8/Mcff2Bvb8+oUaOMMqvVSvny5Rk6dKhRlpiYSKVKlTh16hQACQkJuLu7c/bs2XTHGTVqFDNmzPhnTiKXLBYLhQoVIiAggCpVqlC7dm0WL15sU+fKlSsUKlSITz75BIAxY8bQr18/4/jBgwfx9fXlxo0bzJw5Ez8/P/z8/PD39yc2NjbDsS9dukTnzp1xd3enatWqPP/88xw7dsyYk9lsxsvLi06dOnHz5k0ANmzYgIuLC2az2XgdOnQoy/NIfQ/efvttzGYz3t7exhhms5n169ffz8sqIiLyyND2k/JQKlasGLGxsdy+fRt7e3uio6OpWrWqTZ3NmzdTunRpFi9ezLhx4wBwdnZmxIgRDB48mAULFjB8+HD69etHyZIl7/scU+f2T/L29mb79u0A7N27l1atWlGkSBGaNm0KwPLlywkICCAyMpJevXoxePBgAgIC2LdvHz4+PvTv35+JEyfy559/MmXKFLZt20ahQoVISEggOTk5w3G7du1KQEAAc+bMAWD79u2cO3cOV1dXY063bt0iJCSEqKgoOnbsCEBISAjR0dE2fVkslizPA+Cjjz7io48+wmKxEBYWZtRPT3Jyss38k5KScnJZRUREHglakZeHkslkol69emzcuBGAJUuW0LZtW5s6kZGR9OzZk4oVK7JlyxajvHPnzhw/fpyJEycSGxtLnz59Mh1r27ZtBAcHU7lyZZYuXQrA7NmzGTJkiFEnKCgIi8WCxWLB39+f1157jYCAAJKTk+nfvz81atTAbDYb7Y8cOUK9evUIDAykVq1aHDhwINN+s8PX15d3333XWH1PvQYffvghZ8+e5fTp0+TLl48pU6bQr18/li5dagTLf/75J4ULF6ZAgQIAFC9enDJlyqQ7zm+//caePXsYMWKEzTyDgoJs6jk4OFCrVi1OnjyZrflndh45NXr0aJydnY1XuXLl7rkvERGRvEqBvDy02rVrR1RUFKdPnyZ//vy4uroax6xWK99++y2hoaGEh4cTGRlpHDOZTEyYMIEhQ4YwadIkHBwy/+IpPj6eNWvWsG7dOvr168f169czrb9//3769u3Lnj17mDVrFm5ubmzdupWffvqJiIgIkpOTKV26NGvXrmXnzp1MmjSJ4cOH5+5i/H+BgYEcOnQISFmF3r17Nw0aNKBt27bGSnhISAiurq707NmTSZMmAeDv70/RokVxd3enR48erF27NsMxDhw4gJ+fH3Z2mf/fw/Xr14mJiaFZs2ZG2dq1a21SazK6lnefx72IiIggMTHReJ04ceKe+xIREcmrFMjLQ6t27dps3bqVBQsWEBYWZnNs06ZNeHt74+LiQtu2bVmyZAlWq9U4vnr1akqVKsX+/fuzHCc0NJT8+fNTrlw5PD09swwwK1eujJ+fHwA//PADn332GWazmfr163P16lVOnTpFcnIyXbt2xcfHh549exIfH38PVyCtu89x6dKltGzZEjs7O+NDT6qePXtSu3ZtKlSoAIC9vT1r165l7ty5lClTho4dO/K///0vwzFMJlOGc4iPj8dsNlOyZEnKli2L2Ww2joWEhBAXF2e8ChZM/ymrd5/HvXB0dMTJycnmJSIi8rhRIC8PLZPJRP369RkzZgyhoaE2x6KiooiNjcXNzY2goCDOnDlDTEwMAEePHiUyMpItW7YwZcoUzp8/n+U4f//ZwcGBO3fuGGV352MXKlTI+G+r1crnn39uBK7Hjx/H3d2dKVOmULFiRfbu3cuaNWuM9pn1mx1xcXF4enoa1yA6Oho3NzfatGnDtm3bjDQXOzu7NCvqJpOJOnXqGDeXLlmyJN0xvL292bNnT4bBtre3N3Fxcfz+++9s27aN5cuX5+gc/n4eIiIicm8UyMtDrXfv3owdO5bixYsbZXfu3GHp0qXGjigWi4WpU6caK9IDBgzggw8+oHz58gwYMCDLtJalS5dy48YNTp48yaFDh6hcuTIVKlRg9+7dQMoKdEar9CEhIXz66afcvn0bSAlQISXtpUyZMphMJr7++mujfnb7Tc/+/fv54IMP6NWrF5cuXWLXrl2cOnXKuAaDBw9m4cKF6bY9ffq0MTeAffv2Gav1f+fh4YGPjw//+c9/jLLNmzezdetWm3qurq6MHj2asWPHZvsc/n4eIiIicu+0a4081Dw8PPDw8LAp27hxIx4eHjbBfevWrfnggw949tlnuXz5MuHh4QD06tWLoKAgduzYQbVq1dIdIyAggKZNm3L69GmmTp1KwYIFqVu3Lq6urvj5+REYGIiXl1e6bV9//XV+//13zGYzVquVypUrs3jxYnr27MkLL7zA3LlzCQkJMepnt99U8fHxBAQEcP36dYoXL87EiRNp0qQJs2bNomnTpja75oSGhtK3b18GDhyYpp+bN28yYMAAzp49S758+XBzc8swtQZg1qxZ9OvXD3d3dwoVKsTTTz/N9OnTbb5NAHjhhRd4//33jZuNU3PkU33yySeUKVMmw/MQERGRe2ey5jZZVUTkAUtKSsLZ2Rn/vp9i75h+Xr48mnaM7/SgpyAicl+l/k1LTEzM8h4wpdaIiIiIiORBSq2Rx8Lq1at56623bMrq16/PtGnTHtCM/k9CQgKNGze2KStUqJBx8+4/KTQ0lKNHj9qUzZs3D29v7398bBEREckdpdaISJ6Xk68hRUREHmZKrRERERERecQpkBcRERERyYMUyIuIiIiI5EEK5EVERERE8iDtWiMij4z6I+ZrH/k8SHvBi4jcG63Ii4iIiIjkQQrkRURERETyIAXyIiIiIiJ5kAJ5EREREZE8SIG8iIiIiEgepEA+h0wmE7169TJ+/uOPP7C3t2fUqFEAjBo1irJly2I2mzGbzdSrV8+mfd26dWnXrp1NWXBwsE29FStW0KVLl3/sHFJZLBaCgoIA+PTTT4mMjARg9uzZ/Pnnn5m2vXTpEp07d8bd3Z2qVavy/PPPc+zYMSwWCyaTiS+++MKoGxYWxoYNG+jatStms5lKlSrh4uJiXKNDhw6lO4aDgwMBAQF4e3tTrVo1Zs6caRybPXs2JUqUwGw2U6VKFSZPnmzTdu3atZhMJvbv3w/A0qVL6dChg3H89ddf5/nnnzd+fuutt5g2bZrNNdmwYQMmk4m1a9ca9YKCgrBYLAAkJibSrVs33N3dqVatGnXq1OH7778HUn4vWrRogdlsxtPTk9deey3T6wm2vxsff/yxcX0cHByM/547dy5dunThiSee4MaNG0bbFi1a4Obmlmn/mzZtombNmnh6euLl5cW7774LQJcuXVixYoVR7+5rkNl1vvt33cvLi/nz5xvHunTpgru7uzHv9u3bG22cnJy4ePEiAFeuXMHNzY0TJ04YdYsVK2a0TW0nIiIiaWn7yRwqVqwYsbGx3L59G3t7e6Kjo6latapNnWHDhtGnT580bU+dOkVCQgLHjx/n6tWrFC5c2DhmsVjYtGlTmsD/XqTOLSd69uxp/Pfs2bMJCgqiRIkSGdbv2rUrAQEBzJkzB4Dt27dz7tw5XF1dKVWqFJMmTaJbt27Y2f3fZ8VZs2YBKQHyjBkziI6OznROLi4u7Nq1C4Bjx44RGhqK1WqlR48eAHTq1IkJEyZw7tw5PD09CQsLo1y5cgBERUVRu3ZtoqKieO+993jmmWcYOHCg0fe+ffu4c+eO8XNsbCwTJkxIM4eyZcsyevRoQkJC0r0Gfn5+HDlyBJPJxJEjR/jxxx8BGDlyJOHh4XTr1s0YLzN//93o3bs3vXv3BsDV1ZW4uDij7g8//EDZsmVZs2YNLVu25MKFC5w7dy7T/k+fPs3LL7/M0qVLCQgI4NatW8b7kZXMrnPq7/qhQ4cICgoiLCyMfPnyATBt2jRatmyZpj9nZ2dmzJjBO++8Y5SVK1fOOMcuXboQFhaWbttUycnJJCcnGz8nJSVl61xEREQeJVqRzyGTyUS9evXYuHEjAEuWLKFt27bZahsVFUW7du1o1qyZzQoowODBg/nPf/6TrX5Onz5Ny5Yt8ff3JzAwkCNHjjB79mw6dOjAc889R4cOHTh79ixt2rQhKCiIunXrcvDgQQB+/fVXgoKCqFmzJjNmzDD6HDVqFDNmzGDJkiVs376dsLAwatasme74v/32G3v27GHEiBFGWVBQkLGK+9RTTxEYGMjChQuzdT7ZUaFCBSZOnMgnn3yS5tiTTz6Jh4cHp0+fBuDWrVt8//33fPnll8YcSpYsCcCZM2e4evUqBQsWpHLlyvz222/cvn2b/fv3Yzab0/Rds2ZNkpOT2bJli035b7/9xu7du3n33XcxmUwAPP3008bK+5kzZ3jyySeN+j4+PpmeX2a/G+lp164dUVFRQMrvYJs2bTKt/8knn9CtWzcCAgKAlG87svMtwd3+fp3vVqVKFQoXLsyFCxey7KdHjx58+eWXXL16NUfj32306NE4Ozsbr9QPFiIiIo8TBfL3IDWIOn36NPnz58fV1dXm+JgxY4w0gY4dOxrlqcFaeHi4kcaSKiQkhAsXLhgr0Jnp168frVu3Zvfu3WzevJnSpUsDsHXrVubPn09UVBQDBgzgnXfeYfv27UyePJkBAwYAMGDAAIYPH86WLVtsVstThYaGEhQURHR0dJrgNdWBAwfw8/NLt32qYcOGMXbs2CzPJScCAwPTTcOxWCxcvXoVf39/ANatW0etWrWoUqUKLi4u7NmzB4BatWoRGxvL1q1bCQoKolatWmzevJk9e/bg5eVlrCT/XUREBKNHj7Ypy+oa9OzZk1deeYUmTZowfvx4EhISMj23zH430hMQEMCBAwdITk4mOjqasLCwTOvHx8en+0ElJ/5+ne+2bds2KlasaHxggpTf09R/B2+++aZR/sQTTxAWFmaTKpVTERERJCYmGq8TJ07cc18iIiJ5lQL5e1C7dm22bt3KggUL0g2ghg0bRlxcHHFxccydOxfASJmoWrUqjRo1Ytu2bVy+fDlNu78HjOn5+eefefXVVwFwdHSkUKFCADRr1gwnJycAfvzxR1599VXMZjOvvfYaZ86cAWDHjh2EhoYC8PLLL9/T+VutVmMVOiM+Pj6UK1eOVatW3dMYGY17t6+++gpfX188PDzo27cvBQoUACAyMtLINb975fqZZ54hNjaW2NhYatasSc2aNdm8eTOxsbE888wzGY773HPPcezYMSPfPnUud1+Dfv364evrS/PmzQF49tln+e2333jllVfYtGkTtWrVskkFuVt2fjfS07RpU+bNm8dff/1F2bJlM62b2XuWXvndZRldZ0j50Ort7c0zzzzDyJEjbfqYNm2a8e9g/PjxNscGDx7MjBkzbPL8c8LR0REnJyebl4iIyONGgfw9MJlM1K9fnzFjxhhBcVaioqI4fvw4bm5uVKpUiUuXLrF8+XKbOm3atOHAgQP8+uuvWY6fntSAPtWOHTuMQCo1/zirADw7vL292bNnT5rA+u/SW8nOjbi4ODw9PY2fO3XqxN69e4mJiWHo0KGcOXOGmzdv8u233zJo0CDc3NyYMGGCEcinrshv2bKFmjVr4ufnx549e7IM5CHlZtgxY8YYP//9GkybNo1vv/2W8+fPG3WefPJJOnXqxPLly3F0dGTv3r3p9p2d3430hIeHM3jw4CzTalLnm9G3PcWLFzduPgW4cOGCzbdM6V3nVMOGDSM+Pp5Fixbx6quv8tdff2U5F4BSpUoREhLC119/na36IiIiktZ9D+S3bdvGhx9+yEsvvUTz5s1p3Lhxmjrnz5/n9OnT2cqnfVj17t2bsWPHUrx48WzVj4yMZOPGjVgsFiwWC4sWLTICzFQmk4k333wzzQ4sf1evXj3+97//ASk3/V27di1NnQYNGvDZZ58BcOfOHSOIDAwMZNmyZQDMmzcv3f6LFi2a6Yqwh4cHPj4+Njn9mzdvZuvWrTb1atWqRb58+TJM0cmJEydOMGTIkHRvIq5evTovv/wy06dPZ82aNdStW5fjx49jsVg4efIkrq6u7Nq1i4CAAPbt28fJkycpU6YMDg4O5MuXj/Xr11OrVq1Mxw8PD2f79u2cOnXK5hp8+OGHRjB//fp1o/769euNn8+ePUtCQgJPPfVUun1n53cjPWazmXfffddmN56M9OrViy+//NL4QHf79m2mT58OQP369VmwYAG3bt0C4Jtvvkn3puu7r/PftW7dmoCAgBwF5kOHDmXKlCnZri8iIiK27lsgf/jwYerXr0+tWrUYOXIkkZGRrFmzhg0bNqSpO2bMGMqVK4e3tze3b9++X1P4V3l4eNC1a9d0j92dI5+6vWJCQoJNbnHDhg3ZsmULiYmJNm1feumlLHecmTp1KkuXLsXPz4/atWvbrJCmmj59Oj/88AN+fn74+PiwcuVKAKZMmcKHH35IzZo1yZ8/f7r9d+nShS5dumR4syuk7EBz8OBB3N3d8fHxYcyYMZQqVSpNvYiICE6ePJnp+WTk0qVLmM1mvL29adOmDT179jRSiv5u2LBhzJ49m9mzZ6dZoQ4NDSUyMpJ8+fLh6elJxYoVjWNBQUHY2dkZ9xlkxN7enkGDBtlc61mzZnHkyBHc3d2pUaMGvXv3NrYh3bZtG4GBgfj5+RESEsKYMWPSHePo0aPZ/t1Iz4ABAzLdXSjVU089xZw5c+jRoweenp74+fkZH6RbtWpFYGAggYGB+Pv7c/HiReOeir9Lvc7pfXgcOXIkU6ZMMT7Y3J0jn96uP+7u7tSuXTvLuYuIiEj6TNas8iOyYefOnTRq1IjLly+nSbcwmUxpgnWLxYK7uzsmk4mVK1caecUiIvciKSkJZ2dn/Pt+ir1jwQc9HcmhHeM7PegpiIg8NFL/piUmJmZ5D1iuV+SvX79OmzZtSEpKwsHBgbfffptDhw5lmhrg5uZGtWrVgJQ9sUVEREREJGdy/UComTNncvLkSezs7Fi4cCGtWrUCUra7y0zdunXZsWMH27Zty+0UHlmzZs1i6tSpNmUvvvgiw4YN+9fmEBoaytGjR23K5s2bh7e3930bI3Wv9rtt2LABFxeX+zbGw2D16tW89dZbNmX169dn2rRp922MvXv38sorr9iUVapUKcuHb4mIiEjek+vUmsaNG7Nhwwaee+45m502li1bRmhoaLqpNQCff/45PXv2pFSpUuk+YEZEJLty8jWkiIjIw+xfTa1J3Vv72WefzVG7YsWKASk3NIqIiIiISM7kOpBP3X86Oztn3O3mzZu5HVpERERE5LGV60De2dkZyPnK+rFjxwCyvQ+7iIiIiIj8n1zf7Orm5kZCQgJbt26lW7du2W73/fffA1C1atXcTkFEBID6I+Zr+8k8SNtPiojcm1yvyDdu3Bir1UpUVBQJCQnZavPjjz/y008/YTKZ0n1QjIiIiIiIZC7Xgfxrr72Gg4MDiYmJdOjQId0nPt5t8+bNxiPlCxUqlKNVfBERERERSZHrQN7d3Z0hQ4ZgtVpZt24d3t7ejB8/nr179xp19u3bx/z58wkPD6devXqcO3cOk8nE+++/b+xeIyIiIiIi2ZfrHHmAjz76iBMnTjB37lxOnDhhPLDIZDIB4O/vb9RN3ba+R48eDBw48H4MLyIiIiLy2Mn1ijykBOxff/01//3vfylVqhRWqzXDV4kSJfj000/573//ez+GFhERERF5LN2XFflUr7/+Ol27dmXNmjX89NNPWCwWLl26RJEiRShbtizBwcG0aNGCggW1q4SIiIiISG7kekX++PHjHD9+nLNnzwKQP39+WrZsybhx44iKimLNmjUsXryYadOm0bZt2/sWxJtMJnr16mX8/Mcff2Bvb8+oUaOMMqvVSvny5Rk6dKhRlpiYSKVKlTh16hQACQkJuLu7G/O/3ywWC0FBQf9I39m1fPlyJk+e/I+OMXv2bEqUKIHZbMbDw4NWrVqxZ88emzpr167FZDIZTwNu0aIFixcvNo5/8cUXdO3aldu3b9OrVy98fX3x8fGhdu3aXLlyJd1xM7u+6b3/ADNnzsTPzw8/Pz/8/f2JjY2la9eumM1mKlWqhIuLC2azGbPZzKFDh9Lt28HBgYCAALy9valWrRozZ840jo0aNYoZM2bY1Hd1dQVgw4YNNv1nNobFYqFOnToUKFDApr/0rk9iYqLRX6lSpShbtixms5l69eql6ffq1auUKFGC69ev25R7eXlx8OBBunTpgru7u9Ff+/btjfNK7dfLy4v58+cbbX/55Rdq1qyJv78/Xl5efPbZZ2muxZ07d3j77bepVKkSlSpVol27dsZOVxaLBZPJxBdffGH0GRYWxoYNG9K9NiIiInKf9pE3mUy88cYbaYKXf1KxYsWIjY3l9u3b2NvbEx0dnWZP+s2bN1O6dGkWL17MuHHjgJQHWI0YMYLBgwezYMEChg8fTr9+/ShZsuS/NvfcSj3n7GrVqtU/OJv/06lTJyZMmADAkiVLCAkJYe/evca1jYqKonbt2kRFRfHee+8xffp0WrZsSYsWLbhx4wbjxo3j559/JjIykqtXr7Jnzx5MJhO//vor+fLly/F80nv/T548yZQpU9i2bRuFChUiISGB5ORkZs2aBaQE2jNmzCA6OjrTvl1cXNi1axeQ8nCz0NBQrFYrPXr0yHJeISEhWfYP4OTkxKRJk1i+fLlNeXrXp0CBAsTFxQEpwbOrqyt9+vRJt9/ChQvToEEDVq5cSVhYGAB79+4lf/78eHp6AjBt2jRatmyZpu2wYcPo06cPhw4dIigoiLCwMPLly0f37t1Zvnw5Hh4e3LhxA4vFkqbtlClTiI+PJz4+nvz58zN+/Hi6du1qnF+pUqWYNGkS3bp1w84u8zWG5ORkkpOTjZ+TkpIyrS8iIvIoyvWKfP78+QGoUaNGrieTEyaTiXr16rFx40YgJXBs27atTZ3IyEh69uxJxYoV2bJli1HeuXNnjh8/zsSJE4mNjc0w4IGUAKFjx474+flhNpuJiYkhKSmJRo0aERgYSEBAAD///DMAp0+fpk6dOvj7++Pn52esSN+8eZPOnTvj5eVF+/btjRt+05MagJnNZtq0acPVq1eBlA9M77//PrVr12bDhg3MmjWLGjVq4Ofnx7vvvgvA0KFDjYAUoGvXrqxYsYLZs2czZMgQAH7//XeCg4Px8/OjVatWXLhwAYDg4GD27dsHpOwyFBwcDMD69evx9fXF398/R98shIaG0qJFC2PV9tatW3z//fd8+eWXLFy4EIBKlSoRFhbGuHHjGDlyJH379qVEiRKcOXMGV1dX42bpypUr4+jomO2xU6X3/v/5558ULlyYAgUKAClPFi5TpkyO+75bhQoVmDhxIp988kmu+vm7YsWKUbNmzTQfYu7H9Wnfvr3xPgAsXLiQdu3aZbt9lSpVKFy4sPH7c+7cOWMHqvz581O5cuU0bSZOnMiUKVOM/88YPHgw+/fv57fffgPgqaeeIjAw0GZeGRk9ejTOzs7Gq1y5ctmeu4iIyKMi14F8qVKlgP8L6P9N7dq1IyoqitOnT5M/f34jfQFS0iq+/fZbQkNDCQ8PJzIy0jhmMpmYMGECQ4YMYdKkSTg4ZPzFxPvvv4+bmxt79uxhx44d+Pj4ULBgQZYtW8bOnTtZtmyZsfvO/PnzCQ4OZvfu3ezcuZNKlSoBcODAASIiIoiPj+fs2bNG4J+ehIQEGjVqRFxcHOXLl+fjjz82jhUvXpyYmBhKly7NqlWr2Lx5M3FxcezatYvNmzcTHh5uBEE3b95kw4YNNG3a1Kb/fv360atXL/bs2UOdOnVsUpHSM2nSJCZNmsTu3btZt25dpnX/LjAw0EgbWbduHbVq1aJKlSq4uLgYH3LefvttFixYwM8//2ykSoWFhREVFUW1atUYNmwYBw4cyNG4kPH77+/vT9GiRXF3d6dHjx6sXbs2x31nda5ZWbt2rU1qzd9TXLJyP67Pc889x8aNG43nPvw9kO/Xr58xvzfffDNN+23btlGxYkXj25bevXvj4eFBeHg4c+bM4fbt2zb1k5KSuH79OhUqVDDK7Ozs8PPzs5n/sGHDGDt2bJbzj4iIIDEx0XidOHEiZxdARETkEZDrQL5atWoAHDx4MNeTyanatWuzdetWFixYYKQIpNq0aRPe3t64uLjQtm1blixZYrMSvnr1akqVKmXka2fkxx9/5I033gDA3t4eJycnrFYrQ4cOxdfXl1atWhEfHw9A9erVmTdvHiNHjuTAgQMUKlQISFm99PT0xGQyERAQkG7aQSpHR0dCQ0MBePHFF4mJiTGOhYeHAylB8ebNm6lWrRqBgYEcOHCAI0eOUL16dQ4fPsylS5dYu3Yt9erVS/MBa9u2bUY/r7zyCps2bcr0/OvUqcOwYcOYNm1alg/7+ru7r3dkZKQRKKZ+AAMoWLAg4eHhdOrUyUgXKl++PL/++isjR47k8uXL1KxZM8v36e8yev/t7e1Zu3Ytc+fOpUyZMnTs2JH//e9/Oeo7q3NNXSm/291lISEhxMXFGa+c3jdyP65PwYIFadiwIatWrWLv3r0ULFgQDw8P4/i0adOM+Y0fP94oHzNmDN7e3jzzzDOMHDnSKH/vvffYvHkzdevWZfLkyXTp0sVmPKvVmu51+TsfHx/KlSvHqlWrMq3n6OiIk5OTzUtERORxk+tA/pVXXsFqtfL1119z48aN+zGnbDOZTNSvX58xY8YYwW+qqKgoYmNjcXNzIygoiDNnzhhB8dGjR4mMjGTLli1MmTKF8+fP52jcuXPncvXqVXbt2sWuXbu4c+cOAPXr1+fnn3+mTJkytGvXzsj9vTvtwd7ePs1qZVbnmCr1g0FqLnZqoHX48GFefvllICUffvny5RmmStzd393BlYODg3Eed+ceDxs2jC+//JIrV65QvXp14ybh7IiLi8PT05ObN2/y7bffMmjQINzc3JgwYYIRyEPKyuzfc6ILFixIq1at+Pjjj3nllVf47rvvsj0uZP7+m0wm49uIGTNmsGTJkhz1ndm5Qso3JxcvXjSOXbhwwebbovsht9cH/u8DVVRUlHFDa1aGDRtGfHw8ixYt4tVXX+Wvv/4yjlWpUoX+/fvzww8/pLmmzs7OFChQwOZD7J07d9i9ezdeXl42dSMiIhg9enSOz0dERORxk+tAvk2bNrRt25ajR4/y0ksv5XjVNrd69+7N2LFjKV68uFF2584dli5dyqFDh7BYLFgsFqZOnWoEjwMGDOCDDz6gfPnyDBgwgOHDh2fYf0hIiLHn/e3bt0lKSiIpKYmSJUvi4OBAdHS0EcwcO3aMkiVL8vrrr/Pyyy+n2bUlO5KTk1m2bBnwfzeH/l2jRo2IjIw0gsWTJ08au3+Eh4czb968dNNqAIKCgli0aBEA8+bNM3Y1qVChgnGz5N07yRw5cgR/f3+GDx+Ol5cXR48ezdZ5LFu2jO+//54OHTqwZs0a6taty/Hjx7FYLJw8eRJXV1fjZtG/27lzJ2fOnAFSUoQOHTpkk5KRlcze/9OnTxvnCSn3A+Sk7/ScOHGCIUOGGPda1KtXj6VLlxr/Fr755pt0d4+5V7m9PqlatGjBpk2bmD9/fo7y4wFat25NQEAAX3/9NQCrVq0yvpXYu3dvuvMZOHAgAwcOND7wT5w4kapVq9p8EwBQq1Yt8uXLZ3Nfi4iIiKSV611rjh8/zujRo7l+/TqLFy9m69atdOvWjXr16lG2bNlspQ2UL1/+nsf38PBIEwhs3LgRDw8Pm+C+devWfPDBBzz77LNcvnzZSC/p1asXQUFB7Nixw0gTuts777xDz5498fX1xd7env/+97+89NJLPPfcc9SoUYO6desa42zYsIHx48eTL18+XFxcmD9/vs2KZXak5sGPGjWKChUq8NFHH6Wp4+Pjw1tvvUVwcDB37tyhaNGiLFiwgOLFi1OzZk0OHDhAgwYN0r1vYdq0aXTt2pX333+fChUqMGfOHAAGDRpE+/bt+fzzz3nmmWeM+pMnT2b9+vXY29tTvXp1m2N/99VXX7F27VquXbuGp6cnP/zwAyVKlCAqKoo2bdrY1A0NDSUyMpKAgIA0/fz55590796dmzdvcvv2bZo3b84LL7yQ4bh79uyhbNmyxs+p+drpvf+DBg1iwIABnD17lnz58uHm5nZPqTWXLl3CbDZz48YNChYsyBtvvMGrr74KpOThd+vWjWeeeQaTyUTFihX5/PPPjbapOfKpPvnkk3Q/sCUlJeHt7U1SUhL29vZMmDABi8WS4+uTkQIFChASEsLBgwepWLGizbF+/foxYsQIIOUG7PTuJRg5ciRdunShe/fuzJkzh4EDB1KwYEEKFixoc9N1qoEDB3L+/Hm8vLwwmUyYzeZ060HKqnx6H0RFRETk/5ismW2hkg12dnYZpmtkawImE7du3crNFB4prq6uOU71EXncJSUl4ezsjH/fT7F31APn8pod4zs96CmIiDw0Uv+mJSYmZnkP2H15suvfPwvk8rOBiIiIiIhkIdeBfOfOne/HPB641atX89Zbb9mU1a9fn2nTpv0j44WGhqbJN583b95Dvxr/b1+nVAkJCTRu3NimrFChQja7+vwTatasaXPzL/zf01nvl7179/LKK6/YlFWqVClbD416kH2LiIjIg5Xr1BoRkQctJ19DioiIPMxy8jct17vWiIiIiIjIv0+BvIiIiIhIHqRAXkREREQkD7ovu9aIiDwM6o+Yr+0nH1LaYlJE5P7LdSDv7u6eq/Ymk4kjR47kdhoiIiIiIo+VXAfyFosFk8mU5d7xqQ+J+nu9nDw8SkREREREUuQ6kC9fvnyWwfjt27e5cOEC165dA1KC9zJlyuDgoMweEREREZF7cV9W5LMrLi6OiRMnMnfuXDw8PFi8ePF9fbCOiIiIiMjj4l/dtcZsNvP1118zadIkNmzYQNu2bbNMyRERERERkbQeyPaTAwYMoFatWmzcuJE5c+Y8iCmIiIiIiORpD2wf+fDwcKxWqwJ5kXR88cUXODo6cunSJQBmz56NyWRi69atRp2xY8diMpmwWCw0a9YMs9lM+fLlKVGiBGazGbPZzOXLlzMcY8SIEZQuXZo7d+4YZaNGjcLJyYmLFy8CcOXKFdzc3ID/u7H9iy++MOqHhYWxYcMGANzc3Lhy5YpxbPbs2QwZMgSALl26sGLFCrp27YrZbKZSpUq4uLgY82zYsCGTJk0y2q5bt46GDRve28UTERF5TDywQL5cuXIAxMfHP6gpiDy0oqKiCAoKYsmSJUaZj48PUVFRxs/Lly/Hw8MDgNWrVxMXF8f7779Pp06diIuLIy4ujqJFi2Y4xqJFi3Bzc2Pjxo025c7OzsyYMSPdNqVKlWLSpEk2wX9OzJo1i7i4OL744gtCQkKMeS5atIiPP/6Ys2fPcuvWLQYPHpzhHACSk5NJSkqyeYmIiDxuHlggn7ripz/AIrbOnz/P77//ztixY20C98aNG7N+/XqsVitHjhyhRIkSODk53dMYu3btokSJEvTp08dmDIAePXrw5ZdfcvXq1TTtnnrqKQIDA1m4cOE9jZuRYsWKERERwbBhw5gxYwaNGzematWqGdYfPXo0zs7Oxit1YUBERORx8sAC+dRAoGTJkg9qCiIPpUWLFtG2bVvq1KnDwYMHSUhIAMDBwYHAwEC2bNlCVFQU4eHh9zxGVFQU7du3p1WrVqxcuZLbt28bx5544gnCwsKYOXNmum2HDRvG2LFj73nsjLz66qscOnSIjz/+mFGjRmVaNyIigsTERON14sSJ+z4fERGRh92/HshfuXKFvn37snbtWkwmEw0aNPi3pyDyUIuMjKRdu3aYTCZat25tk17Trl07oqKi+Pbbb3n++efveYxFixbxwgsvULRoUQIDA1m/fr3N8dTUlhs3bqRp6+PjQ7ly5Vi1apVNeXrPk8jJA99MJhPdu3fn+eefzzQlCMDR0REnJyebl4iIyOMm1/vId+vWLVv1bty4walTp9i6dSt//fVXyuAODsbNcCICZ8+eJSYmhrCwMCAlF3z//v107NgRgIYNG9K7d298fHyyDHYzsn37do4fP07NmjUBuHr1Kk8++SQhISFGnVKlShESEsLXX3+dbh8RERG8+eabNt+oFS9enIsXL1KkSBEALly4gKura47mZmdnh53dA/uiUEREJE/JdSCfuptGdqXuG1+gQAG++OILfH19czsFkUdGdHQ0b7zxBpMnTwZS/r24ublx7tw5IOXD75gxYyhfvvw9jxEZGcnYsWPp378/AJcvX8bT05Nbt27Z1Bs6dCiNGzdOt49atWqRL18+tmzZQp8+fQCoX78+X3/9NcOHD+evv/4iOjqaCRMm3PM8RUREJHP3ZenLarVm++Xu7k7v3r3Zs2cPL7300v0YXuSRERUVRZs2bYyfTSYTLVu2tFl9b9u2LUFBQfc8RnR0NK1btzZ+Llq0KGazmbVr19rUc3d3p3bt2hn2ExERwcmTJ42f3333XXbv3o3ZbKZGjRq0bds20/YiIiKSOyZrLh+teuzYsWzVc3R0xMXFhQIFCuRmOBGRNJKSknB2dsa/76fYOxZ80NORdOwY3+lBT0FEJE9I/ZuWmJiY5T1guU6tqVChQm67EBERERGRHMp1IC8iD6eEhIQ0Oe6FChUiJibmAc1IRERE7qdcp9a8//77ALz44otUrlw52+2OHDnC3LlzgZTcWhGRe5WTryFFREQeZjn5m5brQN7Ozg6TycSSJUto1apVttutXr2aFi1aYDKZbB5GIyKSUwrkRUTkUZGTv2nasFlEREREJA96YIH8nTt3Uiagh7+IiIiIiOTYA7vZ9fTp0wD3/HRKEZG/qz9ivraf/AdpC0kRkYfLA1kOP3nyJJ9++ikAHh4eD2IKIiIiIiJ5Wo5W5KdOncrUqVPTPdajRw8GDBiQaXur1crVq1dJSEgAUp5a+eyzz+ZkCiIiIiIiQg4D+UuXLmGxWDCZTNy92Y3VauXPP//M8eBeXl4MGjQox+1ERERERB53OQrkXVxc0jzJ9dixY5hMJlxdXSlUqFCm7e3s7ChSpAgVK1YkJCSEbt26ZdlGRERERETSylEg379/f/r3729TlrrrzMyZM3O0j7yIiIiIiNy7XO9aU758eUwmk1bWRURERET+RbnetcZisXD06FFCQkLux3xE8qQTJ07Qpk0bnn76aby9venYsSMXL15k3bp1BAQE4OnpidlsZt26dUab4OBgPD098fX1xcvLi6FDh3L9+nUg5d9VoUKFMJvNxmv9+vUZjr9//36aNm3K008/TWBgIK1bt+a3334DoEuXLri7uxv9tG/fHoBRo0bh5OTExYsXAbhy5Qpubm424wcEBFClShVq167N4sWLjfFGjRpF2bJljT7r1asHwOzZsylRogRms5kqVaowefLkTK9bly5deOKJJ7hx44ZR1qJFC2MeIiIikrEHto+8yKPCarUSGhpKv379WLp0KQBr1qwhPj6erl278v333+Pt7U18fDzNmzdn8+bNPPXUUwBER0fj4+PDtWvX6NmzJ6+++irz5s0DwNvbm+3bt2c5/rVr12jVqhUff/wxzZs3B2D9+vX8/vvvxvau06ZNo2XLlmnaOjs7M2PGDN555500x+4ef+/evbRq1YoiRYrQtGlTAIYNG0afPn3StOvUqRMTJkzg3LlzeHp6EhYWRrly5TKcf9myZVmzZg0tW7bkwoULnDt3LstzTk5OJjk52fg5KSkpyzYiIiKPGj1WVSSX1q1bR9GiRenU6f8eltO0aVNWrVrFq6++ire3N5ASGHfr1o1PPvkkTR+FChXi448/ZuXKlcb2rNk1d+5c6tevbwTxAA0bNqRZs2ZZtu3RowdffvklV69ezbSer68v7777brpzz8iTTz6Jh4eH8fC3jLRr146oqCgAlixZQps2bbLse/To0Tg7OxuvzD4oiIiIPKru+4r8H3/8QWxsLCdPniQpKYnbt29n2ebdd9+939MQ+dfEx8djNpvTLe/atatNWUBAAHPmzEm3n6JFi+Lu7s7hw4cpWbJkmn4XLVrE008/nabdgQMH0h3/bv369WPEiBEANGnShPHjxwPwxBNPEBYWxsyZM+nevXumfQQGBjJu3Djj5zFjxvDFF18AULVqVebOnWtT32KxcPXqVfz9/TPtNyAggOXLl5OcnEx0dDSTJ082+s1IRESEzda1SUlJCuZFROSxc98C+bi4OIYOHWqTA5xdCuQlL7NarZhMpvvWV6rsptb8ffzatWuTkJBAx44djX9bGaXWAAwePJi6devafKOQ1dwg49Sar776itWrV3Pw4EE+/vhjChQokOU5NG3alHnz5vHXX39RtmzZLOs7Ojri6OiYZT0REZFH2X1JrVm1ahW1a9dm3bp1WK3WTF9Amp9F8jJvb2927dqVptzLy4udO3falO3atQsvL690+7ly5QpHjx418tpzMn5cXJzxc0xMDBEREdnOGy9VqhQhISF8/fXXmdaLi4vD09Mzy/46derE3r17iYmJYejQoZw5cybLNuHh4QwePDhbaTUiIiKSIteBfEJCAi+99BJ//fUXBQsW5J133uH7778HwGQy8eGHH7JixQqmT59OixYtjPIuXbqwfv16fvzxx9xOQeSBCgkJITExkW+++cYo+/bbb2nevDn/+9//iI+PB1JSYP73v//xxhtvpOnj+vXr9OnTh1atWlGsWLEcjd+xY0c2bNjAmjVrbPrLiaFDhzJlypQMj+/fv58PPviAXr16ZbvP6tWr8/LLLzN9+vQs65rNZt599106dOiQ7f5FREQed7lOrfn0009JSkrCZDKxfPlyGjVqZHPcx8eHZ599FoDevXsTExNDu3btmDNnDt7e3gwZMiS3UxB5oEwmE0uXLqVPnz6MGjUKR0dHAgMDmT59Ol9++aXxQdfR0ZH//e9/NqkjYWFh5MuXj1u3btGqVStGjRplHPt7jvyIESMICwtLM36hQoVYtmwZAwcOpFevXpQoUYJSpUrxwQcfGHXuzpF3dXVl7dq1Nn24u7tTu3ZtfvnlF5vxAwICuH79OsWLF2fixIk0adLEOH53jjzA1q1b08xt2LBh1KxZk7fffjvLZ00MGDAASPlmQkRERLJmsuYyv6VBgwb8/PPPNG/enJUrVxrldnZ2mEwmlixZkuaJr3v27KF69epYrVa2bt2a5Y16IiKZSUpKwtnZGf++n2LvWPBBT+eRtWN85vdRiIhI7qX+TUtMTMTJySnTurlOrTl48CBAhg+EunXrVpoyPz8/2rdvz61bt5g5c2ZupyAiIiIi8tjJdWrNpUuXANLsNJGaLnDt2rV02wUHB/PNN98oR14kmxISEmjcuLFNWaFChYiJiXlAM8q+jz76iIULF9qUDRo0KMudckRERCRjuQ7k8+fPn+6qe9GiRbl48WKGD4NJzZfN6mExIpKiePHiNrvT5CVvv/02b7/99j8+zk8fdsjya0gREZFHRa5Ta0qXLg3AhQsXbMrd3d0B0t2WD+Dw4cNA+qk3IiIiIiKSuVwH8j4+PsD/5cqnqlGjBlarlZUrV3Lu3DmbY8nJycZuFxUqVMjtFEREREREHju5DuTr1auH1Wpl06ZNNuWp+0FfvXqVJk2a8N133/Hrr7+yatUq6tevz/HjxzGZTBk+bVJERERERDKW6+0nf/vtN6pUqYKdnR2//vqrkVIDEBoayrJly9J9fL3VauXJJ59kz549lCxZMjdTEJHHnLafvD+0vaSIyIOXk+0nc32zq4eHB3PmzOHatWskJyfbHPvmm29o164d3333XZp25cuXZ/HixQriRURERETuQa4DeYBXXnkl3fLChQuzcuVKYmJiWLNmDWfOnKFw4cJUr16dtm3bkj9//vsxvIiIiIjIY+e+BPJZqV27NrVr1/43hhIREREReSzk+mZXERERERH59ymQFxERERHJg+5ras2dO3dYtGgRq1evJj4+ngsXLnDz5k2OHDliU2/fvn3GHblVq1a9n1MQEREREXks3LcV+V9++YVKlSrx4osvMmvWLLZs2cKvv/6KxWJJU3fJkiXUq1ePunXr8tdff92vKchDwGQy0atXL+PnP/74A3t7e0aNGgXAqFGjKFu2LGazGbPZTL169Wza161bl3bt2tmUBQcH29RbsWIFXbp0+cfOISMNGzb8x8c4ceIEbdq04emnn8bb25uOHTty8eJFNmzYgMlkYu3atUbdoKAgLBYLzZo1w2w2U758eUqUKGFc28uXL6fp32KxUKhQIcxmM97e3vTs2ZM7d+4AcOTIEUwmE6tWrQKgZ8+eTJo0yWi7bt064xrc6/s8e/Zs7O3t+fXXX422rq6u3L5926hbqlQpo+3ffz9ERETk/9yXQH7NmjU0atSIY8eOYbVasbe3x9nZOcP6r7/+OiaTiaSkJCNokEdDsWLFiI2N5fbt2wBER0en+dZl2LBhxMXFERcXZ/MgsVOnTpGQkEBsbCxXr161aWOxWNI8dCw3UueXE+vXr79v46fHarUSGhpK27ZtOXLkCPHx8XTu3JmLFy8CULZsWUaPHp2m3erVq4mLi+P999+nU6dOxrUtWrRouuN4e3sTFxfHnj17iI+PZ+nSpQBERkZSu3ZtIiMjAfjPf/7Dxx9/zNmzZ7l16xaDBw9mxowZQO7e5zJlyjB27Fibuvb29kbdnj17Gm0zes+Tk5NJSkqyeYmIiDxuch3IX7p0iQ4dOnDz5k2KFi3KzJkzuXTpErNmzcqwTYkSJahTpw6Qssonjw6TyUS9evXYuHEjkPLtS9u2bbPVNioqinbt2tGsWTNWrFhhc2zw4MH85z//yVY/Xbp0oWfPntSpUwdvb282b94MpKwS9+zZk5CQEAYOHMhvv/1GkyZNqFatGk2bNuWPP/4gPj6eBg0aGH1t3LiRsLAwIGXlGFIC7gEDBuDj44PZbDZWyWfPns2QIUOMtqkr5leuXKF58+b4+vri6+vL6tWr0533unXrKFq0KJ06/d9DeZo2bWo8ZK1mzZokJyezZcuWbF2HrDg4OFC7dm0OHz4MpFz/mTNn8uOPP5KcnEyxYsWIiIhg2LBhzJgxg8aNGxvBem7e5xdeeIGff/6ZEydO3PPcR48ejbOzs/EqV67cPfclIiKSV+U6kP/444+5ePEi+fLlY/Xq1bz66qsUKlQoy3a1atXCarWyc+fO3E5BHjLt2rUjKiqK06dPkz9/fiMATjVmzBgjjaJjx45GeWogHx4ebqwKpwoJCeHChQvs2rUrW3P4448/+Pnnn/nqq6947bXXjPK9e/eyatUqpk2bRq9evfjiiy/YsWMHr732Gu+99x7e3t5cuHCBM2fOALBw4ULCw8Nt+l60aBGHDx9mz549LF26lO7du2eaIrZ69WqKFy/O3r172bNnD88880y69eLj4zGbzZmeV0RERLqr8vfi2rVrrFu3Dl9fXw4dOkT+/Pnx9vamXr16xoeNV199lUOHDvHxxx8baTOp7vV9dnBwoF+/fkycOPGe5x4REUFiYqLxys2HAhERkbwq14H8qlWrMJlMhIWFUbNmzWy3q1y5MgC///57bqcgD5natWuzdetWFixYYKxm3+3ulIu5c+cCcPz4ca5evUrVqlVp1KgR27ZtS5PjPWzYsGwHse3bt8dkMhEUFMSNGze4dOkSAK1btyZ//vxcvnyZX375hdatW2M2m3n//fc5deoUAG3btmXx4sXcuXOH7777jueee86m759//pmXXnoJOzs73NzcqFy5MocOHcpwLr6+vmzatImhQ4cSGxub4eOWrVYrJpMp0/N67rnnOHbsGPv378/WdUhP6geGunXr0rJlS1q0aEFkZKRxb0JqgA4pK+/du3fn+eefT5Oqcy/vc6pXX32V5cuXc/78+Xs6B0dHR5ycnGxeIiIij5tc71qTetNao0aNctQuNYc+MTExt1OQh4zJZKJ+/fqMGTOGAwcOMH/+/CzbREVFcfz4cdzc3ICUlK3ly5fbrOS2adOGd9991+ZGyczmkN7Pqd8WWa1WSpUqRVxcXJq24eHh9OvXDx8fHwICAihSpEimY6UG4A4ODsaNo5CSxw0pH1p37drFypUr6d+/P506daJPnz5p+vH29jby1TPz1ltvMWbMmCzrZcTb25vt27fblEVFRXHp0iVmzJjBnTt3uHz5Mn/99RcFChTAzs4OO7u0n/nv5X1OVaBAAXr06MHUqVPv+TxEREQed7lekU+9yeyJJ57IUbvUIMfB4V95uKz8y3r37s3YsWMpXrx4tupHRkayceNGLBYLFouFRYsWGavCqUwmE2+++SaTJ0/Osr+FCxcaqVv58+dPc/O1k5MTTz75pHGz9c2bNzlw4AAAPj4+/Pnnn3z88cdp0mogZWedBQsWcOfOHY4dO8bhw4epXLkyFSpUYPfu3UDKqnfqKv3p06cpXLgwnTp1on///ul+eICU9KHExES++eYbo+zbb79N861VeHg427dvN75ByK34+Hjy5cvHyZMnsVgsHD9+nBYtWvDdd99l2Tan7/PdevXqxTfffGP8f4GIiIjkTK4D+WLFigHk+Cvy1Bvs/p5XK48GDw8Punbtmu6xu3OnzWYzhw4dIiEhAX9/f6NOw4YN2bJlS5pvbF566SXs7e2zHN/NzY26devy8ssvM3PmzHTrzJ07l0mTJuHv74/ZbLa5ibRt27Z8++23adJqUo+5u7vj5+dH69atmTlzJgUKFKBu3bq4urri5+fHuHHj8PLyAlLy8qtXr47ZbGbGjBkMGjQo3fmYTCaWLl1KVFQUlSpVomrVqkRFRRn/xlLZ29szaNAgI48/tyIjI2nTpo1NWWhoaJr7FNKTk/f5xo0bNsednJx46aWXuHLlyj3PXURE5HFmslqt1tx0UK9ePWJiYujYsSNfffWVUb5s2TJCQ0MxmUzpbvVXrVo14uLieP7557OVTiCSXV26dCEsLIyWLVs+6KnIvyT1AXP+fT/F3rHgg55OnrVjfKesK4mIyD8q9W9aYmJilveA5XpFvmnTplitVhYtWsTx48ez1SYyMtLYfaRZs2a5nYKIiIiIyGMn1wnqPXr0YOzYsVy/fp02bdqwatUqSpUqlWH9yMhIunfvDqSk5XTu3Dm3U5DH1KxZs9LcLPniiy8ye/bsBzOhHEjdE/5uGzZswMXF5b70n5CQQOPGjW3KChUqRExMzH3pX0RERB68XKfWAEyfPp3+/ftjMpkoWrQoHTt2xN7enhkzZhh5v/v27WPx4sXs3LnT2OXjm2++oUOHDvfjPETkMZaTryFFREQeZjn5m3ZfAnmAESNGMHr06Cz3wk49/uGHHxIREXE/hhaRx5wCeREReVT8qznyqT788ENWrlxJQEAAVqs1w5evry+rVq1SEC8iIiIikgvZXpFP3ZGmUaNGlC1bNtO6+/bt46effsJisXDp0iWKFClC2bJlCQ4OJjAwMPezFhG5i1bkRUTkUfGPpNbY2dlhMplYsmQJrVq1Msq7desGQL9+/TCbzfc+axGRe6TtJ+8PbT8pIvLg/aupNbNnz2bOnDnZ3npSRERERERyL9uBvINDyk6Vepy6iIiIiMiDl+1APvUx8QcPHvzHJiMiIiIiItmT7QdCBQQEsHr1aqZPn07lypUJDAzE0dHROP7nn3/ec3pN+fLl76mdiIiIiMjjKtuBfNeuXVm9ejUJCQm89NJLNsesViuvv/76PU3AZDJx69ate2orIiIiIvK4ynZqTbt27ejVq1eafeFTZbZ3fFYvERERERHJmWyvyAPMmDGD7t27s3LlSk6cOEFycjJz5szBZDIRHBysFBmRHLJYLPTs2ZPff/8dgF69etG5c2eqVq3K6dOnAZg/fz49e/bk4sWL2NnZ8d133zFz5kwWL16Mm5sbdevW5ZtvvgFS/o2eP3+eUaNGpTue1Wrlww8/5KuvvsLR0ZEiRYowZswYgoODAVi7di1NmjRh3759VK1aFYANGzbQpk0b3Nzc+Ouvv2jZsiUTJkzI9LysVisVKlTgxRdfZNy4cUb5yZMn6d+/P7t27eKJJ57giSee4D//+Q81atRg1KhRfPHFF7i6ugJQtGhRNm3adM/XVkRE5FGXo0AewGw22+wXP2fOHAD69+9vs7+8iGTOarUSGhpKREQE7dq1IykpiWbNmlG+fHmcnJywWCy4ubkRGxuLu7s78fHx+Pj4EBsbyzPPPGP088svv3D06FEqVqyY5ZjTpk1j69at7N69m0KFCvHHH3+wdetW43hUVBS1a9cmKiqK9957zygPCQkhOjqaa9euYTabCQ0NpU6dOhmOs3nzZkqXLs3ixYuNQN5qtdKmTRt69erFokWLAIiLi+PgwYPUqFEDgGHDhtGnT58szyM5OdlmB62kpKQs24iIiDxqcr2PvIjcm7Vr1+Li4kK7du0AcHJyYvTo0UycOJFnnnmG2NhYAHbt2sXrr7/O5s2bAdIE8v3797dZ9c7M+PHjmTFjBoUKFQKgdOnStG7dGoBbt27x/fff8+WXX7Jw4cJ02xcqVIjAwEBOnjyZ6TiRkZH07NmTihUrsmXLFuN8ixYtajxEDlIWBl588cVszf1uo0ePxtnZ2XiVK1cux32IiIjkdbkO5NevX8+PP/6Y6eqciKQVHx9PQECATVlAQAAHDhygVq1axMbGkpycjMlkokGDBmzevBmr1cquXbuoVq2a0aZTp0788MMPnDlzJtPxkpKSuHbtGhUqVEj3+Lp166hVqxZVqlTBxcWFPXv2pKlz4cIFDh48SL169TIcx2q18u233xIaGkp4eDiRkZEAHDhwIMunP48ZM8b41q9jx44Z1ouIiCAxMdF4nThxItN+RUREHkW5DuQbNGhAgwYNKF68+P2Yj8hjzWQyARgr8jt37sRsNuPp6cmBAwc4ePAgbm5uFCxY0GiTP39+evfuzaRJk7Ldf3oiIyONbwfatWtHVFSUcWzt2rX4+/vz1FNP8eyzz1KmTJkM+9m0aRPe3t64uLjQtm1blixZYtzYfvf4YWFheHt789prrxllw4YNIy4ujri4OObOnZvhGI6Ojjg5Odm8REREHjdKrRF5QLy8vNi5c6dN2c6dO/Hy8sLHx4fDhw+zceNGatasiclkwsnJie+++84mrSZVjx49WLhwIRcvXsxwPCcnJwoUKMCxY8fSHLt58ybffvstgwYNws3NjQkTJtgE8iEhIezevZv4+HhmzpxJXFxchuNERUURGxuLm5sbQUFBnDlzhpiYGLy9vW3aRUdH88knn2Q6ZxEREcmYAnmRByQkJISLFy8aAXNSUhIREREMGjQIOzs7/P39+fzzz6lVqxYANWrUYMaMGekG8oULF6Zr1658/vnnmY755ptv0q9fP65duwbAqVOnWLhwIWvWrKFu3bocP34ci8XCyZMncXV1ZdeuXTbtK1asyFtvvcX48ePT7f/OnTssXbqUQ4cOYbFYsFgsTJ06laioKEJCQkhMTDRukAe4fv169i+YiIiI2FAgL/KA2NnZsWTJEr744gsqV65MtWrVCA8P54UXXgBS0msuX76Mu7s7ADVr1uTo0aNGYP93ffv25dKlS5mO2a9fPwIDA/Hz88PX15fw8HBKlChBVFQUbdq0sakbGhpq5LffrVevXmzcuDHdG143btyIh4eHTapd69atWbx4MVarlaVLl7J06VIqVqzIM888w7Rp0xg4cKBR9+4cebPZzI0bNzI9HxERkceZyaonMolIHpeUlISzszP+fT/F3rFg1g0kXTvGd3rQUxAReeyl/k1LTEzM8h4wrciLiIiIiORBOX4glIg8/D766KM0e8EPGjSITp3u34rrrFmzmDp1qk3Ziy++yLBhw+7bGDn104cdtIONiIg8NpRaIyJ5Xk6+hhQREXmYKbVGREREROQRp0BeRERERCQPUiAvIiIiIpIH6WZXEXlk1B8xX9tPZkFbTIqIPDq0Ii8iIiIikgcpkBcRERERyYMUyIuIiIiI5EEK5EVERERE8iAF8iIiIiIieZACeeH999/Hx8cHX19fgoKCOHr0KK6urjZ1Ro0axYwZMwAIDg7G29vb5riXlxfBwcH/2BwtFgtBQUH/WP/ZsXz5ciZPnvyvjFW3bl3atWtnU5bZdd+yZQtmsxmz2UyRIkXw9PTEbDYzcODADMewWq2UL1+eoUOH2pS7ubnx8ssvGz/PmDGDUaNGASm/B05OTly8eBGAK1eu4ObmBsCGDRsICwtLM+d9+/YB4OrqyokTJ4x5FitWDHd3d8xmM6GhoTz99NOcOHHCaPvyyy8zZ86cbFwtERGRx5O2n3zMxcTEsGHDBuLi4nBwcODkyZMULlw4y3b29vbs27cPHx8f9u7dS758+f6F2d5ft2/fxt7ePtv1W7Vq9Q/O5v+cOnWKhIQEjh8/ztWrV23ej4yue82aNYmLiwNSgucZM2bg4+OT6TibN2+mdOnSLF68mHHjxtkc++WXXzh69CgVK1ZM087Z2ZkZM2bwzjvv5PjcypUrZ8yzS5cuhIWF0bJlSwC++uorhgwZQmRkJDExMVgsFjp10laJIiIiGdGK/GPuzJkzPPHEEzg4pHymK1u2LE888USW7dq1a0dUVBQAUVFRaVaP/y4pKYmOHTvi5+eH2WwmJiaGpKQkGjVqRGBgIAEBAfz8888AnD59mjp16uDv74+fnx979uwB4ObNm3Tu3BkvLy/at2+P1WrNcDxXV1f69OmD2WymTZs2XL16FUhZbX7//fepXbs2GzZsYNasWdSoUQM/Pz/effddAIYOHcqsWbOMvrp27cqKFSuYPXs2Q4YMAeD3338nODgYPz8/WrVqxYULFwDbFeh9+/YZ31KsX78eX19f/P39s/xmIfV6NmvWjBUrVuTqumcmMjKSnj17UrFiRbZs2WJzrH///mmC+1Q9evTgyy+/NK7p/dKpUyf++OMPfvzxRwYMGMCMGTMwmUzp1k1OTiYpKcnmJSIi8rhRIP+Ya9KkCb/++iteXl7079+fbdu2ZatdixYt+P777wFYvXo1zZs3z7T++++/j5ubG3v27GHHjh34+PhQsGBBli1bxs6dO1m2bJmRBjJ//nyCg4PZvXs3O3fupFKlSgAcOHCAiIgI4uPjOXv2rBH4pychIYFGjRoRFxdH+fLl+fjjj41jxYsXJyYmhtKlS7Nq1So2b95MXFwcu3btYvPmzYSHh7Nw4UIg5cPDhg0baNq0qU3//fr1o1evXuzZs4c6deoYqScZmTRpEpMmTWL37t2sW7cu07qpAXp4eDiRkZE2x3J63TNitVr59ttvCQ0NTXecTp068cMPP3DmzJk0bZ944gnCwsKYOXPmPY2dmRkzZhAWFkaNGjUwm80Z1hs9ejTOzs7Gq1y5cvd9LiIiIg87BfKPuaJFi7Jr1y6mTZtGgQIFaNKkCWvWrEm37t2rowUKFMDDw4Ovv/6aypUrU6BAgUzH+fHHH3njjTeAlPQQJycnrFYrQ4cOxdfXl1atWhEfHw9A9erVmTdvHiNHjuTAgQMUKlQIgCpVquDp6YnJZCIgIACLxZLheI6OjoSGhgLw4osvEhMTYxwLDw8HYN26dWzevJlq1aoRGBjIgQMHOHLkCNWrV+fw4cNcunSJtWvXUq9ePfLnz2/T/7Zt24x+XnnlFTZt2pTp+depU4dhw4Yxbdo0rl27lmG91HSaqlWr0qhRI7Zt28bly5eN4zm97hnZtGkT3t7euLi40LZtW5YsWWLzDUf+/Pnp3bs3kyZNSrf94MGDmTFjBjdu3DDKMlo9z6g8PX5+fvj4+NCjR49M60VERJCYmGi87s6tFxEReVwoR15wcHCgSZMmNGnSBFdXV5YvX06+fPm4c+cOdnYpn/UuXLiAl5eXTbvw8HC6detmk4aSE3PnzuXq1avs2rULe3t7I2CvX78+P//8MytWrKBdu3aMHTsWPz8/HB0djbb29vbcvn0722PdHUymjmO1WunRo4eRUnO3Vq1asXz5cjZs2JBu+srd/VmtVuNnBwcH7ty5A6Skf6QaNmwYLVq0YOXKlVSvXp0tW7bw1FNPpek3KiqK48ePGzeQXrp0ieXLl9OxY0ejTm6ve+o4sbGxxjhnz54lJiaGOnXqGHV69OiBj48P3bp1S9O+VKlShISE8PXXXxtlxYsXN26CTXXhwoU0N05nxc7Ozvi9y4ijo6PN74OIiMjjSCvyj7lDhw5x5MgRICUg3b9/PxUqVOCZZ55hwYIFQEowuWbNGmrVqmXTtkWLFrz11lvZSu8ICQnhv//9L5Byk2lqXnPJkiVxcHAgOjqav/76C4Bjx45RsmRJXn/9dV5++WUjRz4nkpOTWbZsGZAStNauXTtNnUaNGhEZGWkEnydPniQhIQFICZbnzZuXbloNQFBQEIsWLQJg3rx51KtXD4AKFSoYN3MuXrzYqH/kyBH8/f0ZPnw4Xl5eHD16NN15R0ZGsnHjRiwWCxaLhUWLFhk58alyct3Tc+fOHZYuXcqhQ4eMcaZOnZpmnMKFC9O1a1c+//zzdPsZOnQoU6ZMMX728PDAYrEYv09xcXHcunWLkiVL3tM8RUREJHMK5B9zV65c4eWXX6Zq1ar4+Phw584d+vTpw9SpU/n6668xm80EBwczYsQIKlSoYNPW0dGRt956K1sro++88w4WiwVfX1+qVavG/v37eemll9i4cSM1atRg8+bNFC9eHEjZxtBsNhMQEMDatWvp3r17js8rNQ/ebDZz9OhRevXqlaaOj48Pb731FsHBwfj6+tKuXTvjBs6aNWty4MAB6tatmyatBmDatGlMnz4dPz8/fvrpJ0aOHAnAoEGDGDduHHXr1rVJO5k8eTJVq1bFz8+P8uXL88wzz6Tp8+jRoyQkJODv72+UNWzYkC1btpCYmGiU5eS6p2fjxo14eHgY1xugdevWLF682Pg2IVXfvn25dOlSuv24u7vbfEBydHRk1qxZdOjQAbPZTO/evW1W7EVEROT+Mlkz2/pDJI9ydXXl/PnzD3oa8i9JSkrC2dkZ/76fYu9Y8EFP56G2Y7y29BQReZil/k1LTEzEyckp07pakRcRERERyYN0s6vcV6tXr+att96yKatfvz7Tpk37R8YLDQ1Nk28+b968h341/t+6TrNmzWLq1Kk2ZS+++CLDhg27r+OIiIjIv0+pNSKS5+Xka0gREZGHmVJrREREREQecQrkRURERETyIAXyIiIiIiJ5kAJ5EREREZE8SLvWiMgjo/6I+dpHPgvaR15E5NGhFXkRERERkTxIgbyIiIiISB6kQF5EREREJA9SIC8iIiIikgcpkBcRERERyYMUyD+ETCYTvXr1Mn7+448/sLe3Z9SoUQCMGjWKsmXLYjabMZvN1KtXz6Z93bp1adeunU1ZcHCwTb0VK1bQpUuXf+wcMtKwYcN/fIysrh+A1WqlfPnyDB061ChLTEykUqVKnDp1CoCEhATc3d05e/bsPzJPi8VCUFBQtuvPnDkTPz8//Pz88Pf3JzY2Fkh5b/ft2wfAnj17ePrpp9m9e3e6fdz9u+Ph4UGHDh04duyYcdzNzc0Yo2nTppw5c8am3MfHB19fX8aMGcPt27cznOvs2bMpUaKEMU6rVq3Ys2ePcTw4OBhPT0/jd3jgwIEAdOnSBXd3d6M8Ojo629dHRETkcaNA/iFUrFgxYmNjjUApOjqaqlWr2tQZNmwYcXFxxMXFsWnTJqP81KlTJCQkEBsby9WrV23aWCwWm7q5lVkgl5H169fft/Ezkp3rt3nzZkqXLs3ixYuNMmdnZ0aMGMHgwYMBGD58OP369aNkyZL/+JyzcvLkSaZMmUJsbCx79uzhxx9/pHz58jZ1fvvtN9q2bcvcuXPx9/fPsK/U353ffvuNOnXq0LBhQ5KTk43jMTEx7Nmzh6CgIP7zn//YlO/bt48NGzawfv163nnnnUzn3KlTJ2Ocrl27EhISYvOhKDo62vgdnjx5slE+bdo0ozwsLCzdvpOTk0lKSrJ5iYiIPG4UyD+ETCYT9erVY+PGjQAsWbKEtm3bZqttVFQU7dq1o1mzZqxYscLm2ODBg20Cs8x06dKFnj17UqdOHby9vdm8eTOQsqLbs2dPQkJCGDhwIL/99htNmjShWrVqNG3alD/++IP4+HgaNGhg9LVx40YjIHN1dQVSVsQHDBiAj48PZrOZtWvXAikruUOGDDHaBgUFYbFYuHLlCs2bN8fX1xdfX19Wr16d4dyzc/0iIyPp2bMnFStWZMuWLUZ5586dOX78OBMnTiQ2NpY+ffpkOE5SUhIdO3bEz88Ps9lMTEwMSUlJNGrUiMDAQAICAvj5558BOH36NHXq1MHf3x8/Pz9jdfrmzZt07twZLy8v2rdvj9VqTXesP//8k8KFC1OgQAEAihcvTpkyZYzjx48f57nnnuPzzz+nVq1aGc757/r06UO5cuX47rvv0hyrX78+hw8fTlNevHhxPv30U/773/9mON+/Cw0NpUWLFsyfPz/bc8vM6NGjcXZ2Nl7lypW7L/2KiIjkJQrkH1Lt2rUjKiqK06dPkz9/fiMATjVmzBgj/aBjx45GeWogHx4eTmRkpE2bkJAQLly4wK5du7I1hz/++IOff/6Zr776itdee80o37t3L6tWrWLatGn06tWLL774gh07dvDaa6/x3nvv4e3tzYULF4y0jIULFxIeHm7T96JFizh8+DB79uxh6dKldO/enf/X3n2HRXG1fwP/rjQBARXsBVB6XRAbVhRNsaKAvWs0FuxGE39ijEaMJeiDxkfNY0mMAVGxFzSiiMEKiKJYVyVqjKA0BQTm/YN3J6ywy4JEXPP9XNde1+7MKfcMk3jP2TNnc3JylMZy9OhRmJqaIjExEVeuXEHbtm1Vxq7q/AmCgP3798PHx6fEeZJIJFixYgVmzZqFVatWQVtb+W+mLVq0CBYWFrhy5QouXboEJycn6OvrY+/evbh8+TL27t0rThnZsWMHOnfujISEBFy+fBlWVlYAgOvXr2PevHlISkrCn3/+KSb+b3J1dYWRkRGaNWuGzz77TLzxkRsyZAgWLVqELl26qDwvpXF3d0dycnKJ7fv374ezs3OpdSwtLQEU3WBUtB9fX1/xGi4+Ih8QECBuV9b+vHnzkJ6eLr4ePnyodhxEREQfCiby7ylPT0+cP38ev/76a6nTC4pPrdm+fTuAolHZ7OxsODo6okuXLrhw4QIyMzNL1Fu6dKlaMQwYMAASiQQeHh7Iy8vDixcvAAB9+vSBrq4uMjMzERMTgz59+kAqlWLRokXi/PJ+/fph9+7dKCwsxOHDh9GjRw+Fts+cOYPBgwejWrVqsLCwgI2NTanJpJyzszOio6MxZ84cxMbGwtjYWGXsqs5fdHQ0HBwcULNmTfTr1w979uxRGFk+evQo6tevj2vXrqns47fffsPnn38OANDS0oKxsTEEQcCcOXPg7OyM3r17IykpCQDQsmVL/PLLLwgMDMT169dhYGAAALC1tYWdnR0kEgnc3Nwgk8lK7UtLSwvHjx/H9u3b0bBhQwwZMgQ//vijuL9r167YsmUL8vPzVcZcmjdH1T09PSGVSpGZmYl58+apXa+8/RSfWiO/4QEUp9bUrVu31Lb09PRgbGys8CIiIvq3YSL/npJIJOjYsSOCgoLg4+OjVp2wsDA8ePAAFhYWsLKywosXL7Bv3z6FMn379sX169dx8+ZNtWIo7bM8CRUEAfXr1xeTrsTEROzfvx8A4Ofnh/DwcJw5cwZubm6oUaOGyr4EQYBEIoG2tjYKCwvF7fK52zY2NoiLi4OTkxOmTp2KkJCQMmNXdv7CwsIQGxsLCwsLeHh44MmTJzh79iwA4N69ewgNDcW5c+cQHByMZ8+elXWaFGzfvh3Z2dmIi4tDXFyceCwdO3bEmTNn0LBhQ/j7+4t/Fz09PbGulpaWyucOJBIJ2rVrh4ULFyIkJAR79uwR9y1cuBD169dX+OZEXfHx8bCzsxM/nz17FvHx8di2bRtq1qxZah2ZTIZq1aopTbTV6YeIiIjeDhP599ikSZOwbNkymJqaqlU+NDQUp06dgkwmg0wmw65duxAWFqZQRiKRYPbs2QpTGZTZuXMnBEHA5cuXoaurCxMTE4X9xsbGqFOnDg4dOgSgaL739evXAQBOTk54+vQp1q5dW2JaDVC0ss6vv/6KwsJC3L9/H7dv34aNjQ3Mzc3FFVeSkpLEUfpHjx7B0NAQw4cPx9SpUxEfH19m/KWdv8LCQkRERCA5OVk8T6tXrxbP07Rp0/DNN9+gadOmmDZtGr788kul7Xt7e+OHH34AUPTgr/yhy3r16kFbWxvh4eHidKH79++jXr16GD9+PIYOHaqwgos6Hj16pHDMV69ehbm5uUKZjRs34o8//ijzIdTifvjhB6SkpODjjz9Wu05aWho+//xzTJo0qcTNnjJ79+7FkSNHMGjQILX7ISIiItWUTwCmKmdtbQ1ra+tS9wUFBWHTpk3i59DQUKSmpiqsVuLl5YXhw4cjPT1doe7gwYOxYMGCMvu3sLBA+/bt8fz5c4VpHMVt374dEyZMwLx585Cfn4/Zs2fD3t4eQNH0mhUrVpRat1+/fjhz5gxcXFygra2NjRs3onr16mjfvj3MzMzg4uICd3d3sa3ExETMmjULWlpa0NfXVxpPcaWdv1OnTsHa2lohue/Tpw+++eYbfPrpp8jMzBRvPCZOnAgPDw9cunQJLVq0KNH+//3f/2HChAlwdnaGlpYWfvjhBwwePBg9evRAq1at0L59e7GfqKgoLF++HDo6OqhZsyZ27Nih8pmAN71+/RrTpk3Dn3/+CR0dHVhYWJQ4Bzo6Oti1axc6duyIJk2a4LPPPiu1Lfm1k52djZYtW+K3335T+GZAGU9PTxQWFkIikWDo0KEKDyWXZtu2bTh+/DhevnwJOzs7REZGKozg+/r6ig/vuri4YNu2bWXGQERERH+TCOWd6Er/CiNHjoSvry969uxZ1aEQlSkjIwMmJiZwnbIeWnr6VR3Oe+3S8uFVHQIREakg/zctPT29zGfAOLWGiIiIiEgDcWrNv9zmzZuxevVqhW0DBw7Eli1bqiagcmjdurXCDxkBRVNYlD2gWVFHjx7FF198obCtY8eOWLNmTaX2I+fj44N79+4pbPvll1/g4OCgdhvK/q5z586tlBjl3vW5ISIior9xag0RabzyfA1JRET0PuPUGiIiIiKiDxwTeSIiIiIiDcREnoiIiIhIAzGRJyIiIiLSQFy1hog+GB3n7+A68mXgOvJERB8OjsgTEREREWkgJvJERERERBqIiTwRERERkQZiIk9EREREpIGYyBMRERERaSAm8u+QRCLBxIkTxc+PHz+GlpYWFi5cKG4TBAFNmzbFnDlzxG3p6emwsrLCH3/8AQBITU1Fs2bN8Oeff5baz8KFCxESEvLPHEQlmT9/Pho0aIDCwkJx28KFC6GlpYUnT56I2z7//HNIJBIUFBRAKpVCKpWifv36aNy4MaRSKTp06KC0j+joaLRu3Rp2dnawt7fHggULxH7k9e3t7bFjxw6xzsiRI9GsWTOxrwEDBpSoY21tjUGDBuH+/ftiPQsLC6Snp5crxqioKEgkEoSFhYnbQkNDIZFIEBUVJW67c+cOJBIJDh06JG77/fff0bJlS/H8RUVFwdPTE4IglNpXYWEhvvrqK1hZWcHKygr+/v5ITU1FdnY26tati1evXimUt7e3x40bN9Q6H8rOoaurK2xtbTF+/HikpqYCAGQyGTw8PBT6GjlyJA4cOCCeRxcXF7G/5cuXl3o8RERExET+napduzZiY2NRUFAAAAgPD4ejo6NCmd9//x0NGjTA7t27xW0mJiaYP38+Zs6cCQD48ssvERAQgHr16lV6jPLY/mm7du2ChYUFTp06pbDdwcEB4eHhYiznz59H7dq1oaWlhfj4eMTHx2PChAmYO3cu4uPjER0dXWr7jx49wtChQ7F+/XrcuHEDiYmJaNKkibhfXj8iIgKfffYZXr9+Le5bs2aN2FdoaGiJOrdu3UK7du3g5eWF3NxccX95YwQAZ2dn7Ny5U/y8c+dOODs7K5QJDQ2Fp6enQixt27aFo6MjNmzYgPz8fEydOhUhISGQSCSl9hMcHIykpCQkJSXh9u3baNmyJUaNGgVDQ0N06tQJBw8eFMsmJiZCV1cXdnZ2ap0PZecwISEB165dQ61atdC7d2+l5+BNZ8+eFfubPXt2qWVyc3ORkZGh8CIiIvq3YSL/DkkkEnTo0EFMXvfs2YN+/foplAkNDcWECRNgaWmJc+fOidtHjBiBBw8eYOXKlYiNjcXkyZNV9nXhwgV07twZNjY2iIiIAABs2bIFs2bNEst4eHhAJpNBJpPB1dUV48aNg5ubG3JzczF16lS0atUKUqlUrH/nzh106NAB7u7uaNOmDa5fv66yXWXi4uJQt25dTJ48WWE0GgD69+8v3sRERUWhQ4cOSpNTVdatW4fRo0fDzc0NAKCtrY1x48aVKGdrawtDQ0OkpaWVq/3JkyejSZMmOHz4cLljK87GxgYpKSnIzs5GVlYWHj16BBsbG4UyYWFh2LhxI3777TeFG4dly5bhu+++w8KFC9G2bVu4u7sr7WflypUIDg6Grq4uAGDmzJm4du0abt26hQEDBpS4mfD391f7GFSdQ21tbSxZsgSPHz9GfHy82m2WZenSpTAxMRFfxW/SiIiI/i2YyL9j/v7+CAsLw6NHj6CrqwszMzNxnyAI2L9/P3x8fODn56cw+imRSLBixQrMmjULq1atgra26t/ySkpKwrFjx3DixAkEBASUmDrxpmvXrmHKlCm4cuUKNm/eDAsLC5w/fx6nT5/GvHnzkJubiwYNGuD48eO4fPkyVq1ahS+//LJC5yAsLAwDBgxA7969cfDgQYVvAczMzKCrq4tHjx4hLCysXAllcUlJSZBKpWWWu3DhAiwtLRW+3QgICBCndigbEQYAd3d3JCcnVyi+4nr27In9+/dj//796Nmzp8K+5ORk6OrqwsHBAR06dMDRo0fFffXq1cPYsWMREhKCJUuWKG0/IyMDr169grm5ubitWrVqcHFxwfXr19GjRw+cOnUKL1++BFAykS/rfJR2DovT0tKCq6ur2ufK09NT7K/4fwPFzZs3D+np6eLr4cOHarVNRET0IeEvu75jnp6emDJlCn799Vf4+voiJydH3BcdHQ0HBwfUrFkT/fr1Q8uWLbFy5UpxRPro0aOoX78+rl27hq5du6rsx8fHB7q6umjSpAns7OzKTKJsbGzg4uICAIiMjMS1a9ewdetWAEB2djb++OMP1KpVC5MmTcKVK1dQrVo1hdHh8ti1axeio6NhZGQEd3d3nDx5Et7e3uL+/v37IzQ0FBcuXMD69esr1IcgCCpH8oOCgrBu3TrcvHlTnJ8tt2bNmhIJtbI+KoO/vz/mzp0LQRDw3Xff4fLly+K+0NBQMamW3wQWn6YSGRmJGjVq4OHDhzA1NVUap6pzoa+vDy8vLxw6dAi2trbQ19eHtbW1uF/Z+VB1DkuLAYDSOIpvP3v2LGrUqKGyPT09Pejp6aksQ0RE9KHjiPw7JpFI0LFjRwQFBcHHx0dhX1hYGGJjY2FhYQEPDw88efIEZ8+eBQDcu3cPoaGhOHfuHIKDg/Hs2bMy+3nzs7a2tsLDpcUTcQMDA/G9IAjYsGGDOE/5wYMHaNasGYKDg2FpaYnExEQcO3ZMrK+q3TddvHgRDx48QOvWrWFhYYGYmJgSo64+Pj5YunQpOnXqVKFpNUDRXPu4uDil++fOnYukpCTs2rULY8aMUbihUld8fLw4j/xtWFtb49GjR3jy5AmsrKwU9oWFhSE4OBgWFhYICAjAwYMHxVh37NgBMzMzbNmyBVOmTFHavomJCapXr64w3amwsBAJCQmwt7cH8PdNgvzbEnWoew4LCwuRmJgIOzs7mJqa4vnz5wr709LSFL6ZIiIiIvUwka8CkyZNwrJlyxRGUAsLCxEREYHk5GRx3vrq1avFOeTTpk3DN998g6ZNm2LatGllTmuJiIhAXl4eUlJSkJycDBsbG5ibmyMhIQFA0dQTZaP03t7eWL9+vTjlRT63OSMjAw0bNoREIsFPP/0klle3XaBohHnZsmXiMd69exeHDh1Cfn6+WMbMzAzLli1TWOGnvCZOnIj//e9/YuwFBQX4z3/+U6Jcnz594ObmpnA86vjhhx+QkpKCjz/+uMIxFrd48WJ88803CtuSkpKgo6ODlJQUyGQyPHjwAJ988gkOHz6MrKwsBAYGYuXKlfD29kadOnXwyy+/KG1/+vTpmD59OvLy8gAUzZl3dHQUR94/+eQTREdHY8eOHeWezqTqHObn52P+/Plo2LAhXF1dUaNGDdSsWVO8QU1JSUFiYmKJh76JiIiobJxaUwWsra0Vpi4AwKlTp2Btba2Q3Pfp0wfffPMNPv30U2RmZsLPzw9AUZLq4eGBS5cuoUWLFqX24ebmhu7du+PRo0dYvXo19PX10b59e5iZmcHFxQXu7u7iaOybxo8fj7t370IqlUIQBNjY2GD37t2YMGEC+vfvj+3btytMhVG3XaBopZ6TJ0+Kn42MjCCVSnH8+HGFcqNGjVLahjoaNWqErVu34rPPPkNGRga0tLSUJqiBgYEYOXIkxo4dC6BoTvj8+fMBFN1UyGMLCgrCpk2bkJ2djZYtW+K3336rtOkdpU2VCg0NRd++fRW2+fj4iN/MDBkyBE2bNgUArFq1Ct26dUOfPn1gaGhYoq3p06fj2bNnsLe3h0QigVQqxebNm8X91atXh7e3N27cuAFLS0uFusrOR3GlncMvv/wSOTk56NKlC/bu3SuW3bp1KyZOnIiMjAxoa2vjv//9r8JUGk9PT1SrVjTG8PHHHyMoKEjluSMiIvq3kgiVNdGXiKiKZGRkwMTEBK5T1kNLT7+qw3mvXVo+vKpDICIiFeT/pqWnp8PY2FhlWU6tISIiIiLSQJxao8GOHj2KL774QmFbx44dsWbNmiqK6G+pqaklposYGBiIc6MrS2JiIoYNG6awzcrKSvxRqffBu4xxyZIlCmvCA8CMGTMwfDhHYYmIiD40nFpDRBqvPF9DEhERvc84tYaIiIiI6APHRJ6IiIiISAMxkSciIiIi0kB82JWIPhgd5+/g8pNl4PKTREQfDo7IExERERFpICbyREREREQaiIk8EREREZEGYiJPRERERKSBmMgTEREREWkgJvJERERERBqIibwGkEgkmDhxovj58ePH0NLSwsKFCwEACxcuROPGjSGVSiGVStGhQweF+u3bt4e/v7/Cts6dOyuUO3DgAEaOHKk0hpEjR+LAgQNvfzD/oKFDh6JVq1YK20aOHIlatWohLy9P3PbJJ5/AwsICDx8+FM9Z7dq10axZM0ilUgwYMEBpH3v27IGrqyscHR3h5OSEdevWif00a9YMrq6ucHFxwYkTJ8Q6nTt3hp2dndjX9OnTS9SxtbXF+PHjkZqaKtYzMzMrd4yPHz/GJ598AqlUCjs7O4wbNw4AsGXLFsyaNUuhrIWFBbKysiCTySCRSPDdd9+J+86dOweJRIItW7YoPRfy43J2doa9vT3mzJmDV69eAQBkMhkMDAzE2KVSKU6ePAkA0NbWFre1bNkS8fHxOHfunLitRo0a4vmSnysiIiIqievIa4DatWsjNjYWBQUF0NLSQnh4OBwdHRXKzJ07F5MnTy5R948//kBqaioePHiA7OxsGBoaivtkMhmio6NLJP6VTR73PyknJwenT5+GsbEx7t27B0tLS3Ff48aNcezYMfTs2RNpaWn466+/AABNmjRBfHw8gKKk2tfXFz179lTaR1xcHL744gscPXoUlpaWePXqFUJDQ8X9a9asQc+ePXH8+HFMmDABt27dEveFh4fDycmpRJvyOvn5+Zg/fz569+6NmJgYcX95YwwMDISfnx9Gjx4NALh69aqKs/Y3a2tr7Nu3D3PmzAEAhIWFwdnZucx68uN6+fIlJkyYgDFjxuCXX34BADg4OODixYsl6tSsWVM8pl27dmHRokXYvXu3uK1z584ICQkp9XzJ5ebmIjc3V/yckZGh1nESERF9SDgirwEkEgk6dOiAU6dOASgaFe7Xr59adcPCwuDv74+PPvqoxIj6zJkz8e2336odx4EDB9CuXTs4ODjg999/B1D0bUBISIhYxszMDAAQFRWF7t27w9/fH15eXsjKysLw4cPRsmVLeHh4iMlqbGwsPD094ebmhi5duuDx48cq21Xm8OHD8PLywqBBgxAWFqawz9/fX9y2Z88e9O3bV+1jLm7lypX46quvxJsEfX39Ur/FaN++PVJSUsrVtra2NpYsWYLHjx+LCW1FPHnyBHXq1BE/q0qGizM2NoaZmRnu3r0LQRAQFRWFLl26qN2vgYEB1q5di4MHDyp8q1CWjIwMmJiYqF1ebunSpTAxMRFfTZo0KXcbREREmo6JvIaQJ6OPHj2Crq5uicQ2KChInJowZMgQcbs8kffz81MYPQYAb29vpKWlIS4uTq0YHj9+jDNnzmDbtm3ilA1Vzp07h+DgYJw+fRqLFy+Gj48PLly4gIiICHGqkIODA6KjoxEXF4exY8cqTO8oj7CwMAwYMAB+fn4lEnk3Nzdcv34dubm5CA8Ph6+vb4X6SEpKglQqLbPcwYMH0bt3b4Vtvr6+4t/n+++/L7WelpYWXF1dkZycXKH4AGDChAkYNmwYunXrhuXLl5crqZafu9jYWLRo0QLa2uX7ws7IyAjNmjXD7du3Afx9vuSvO3fuAABevHgBqVQKGxsbzJw5E1988UW5+gGAefPmIT09XXw9fPiw3G0QERFpOk6t0RCenp6YMmUKfv31V/j6+iInJ0dhf2lTa+TTaRwdHWFra4sxY8YgMzMTRkZGCvWWLl2K4cPL/tn2AQMGQCKRwMPDA3l5eXjx4oXK8u3atUPDhg0BAJGRkTh06BC+/vprAEBqairy8vLw/PlzDB06FHfv3kV+fj7Mzc3VOR0KXr16hZiYGGzbtg06OjooKCjArVu3YG1tLZbp3r07fvnlF+Tk5KBx48bl7gMABEGARCJRuj8gIAAzZszAw4cPERsbq7BP2dSa0vp4G59++ilu3bqFw4cPIzw8HBs2bMDVq1eVxl18e+/evfHxxx/jzz//hL+/P44cOVLu/ovHr87UmvDwcEyePBnHjx8vVz96enrQ09Mrd3xEREQfEo7IawiJRIKOHTsiKCgIPj4+atUJCwvDgwcPYGFhASsrK7x48QL79u1TKNO3b19cv34dN2/eVCuGNz9ra2ujsLBQ3FZ83rKBgYH4XhAEHDhwAPHx8YiPj0dKSgp0dXWxYMEC9OjRA1evXsWWLVvE+qrafdPBgweRlpYGa2trWFhY4P79+yVG5f38/DBz5swKT6sBihJTVd9erFmzBjdv3kRQUBBGjRpV7vYLCwuRmJgIOzu7CscIAHXq1MHw4cOxb98+6OnpITExEaampnj+/LlCuZycHIW/kZGREerWrYtDhw7By8ur3P1mZWXh3r17CjdQZenZsyfOnj1b7r6IiIiIibxGmTRpEpYtWwZTU1O1yoeGhuLUqVOQyWSQyWTYtWtXiQRXIpFg9uzZSqd7FLdz504IgoDLly9DV1cXJiYmMDc3F0dXIyMjkZWVVWpdb29vrF27VvyckJAAoGiOtHyE/Oeffxb3q9uu/Di3b98uHue5c+dKHKdUKsWCBQswaNCgMo9TGfkzBTKZDEBRIrx+/foS5QICAvD69WtERkaq3bb8YdeGDRvC1dW1wjGePHlSXDnmzz//RGpqKho1aoSWLVvi1KlTePbsGQBg3759cHFxKXFz9tVXX2Hp0qXlfjj51atXmDx5Mnr37o3atWurXe/s2bNo1qxZufoiIiKiIkzkNYi1tbXSkd7ic+SlUimSk5ORmpqqkBR6eXnh3LlzSE9PV6g7ePBgtRI3CwsLtG/fHkOHDsXGjRsBAP369cP9+/fRokULHDlyROlNxoIFC/D06VM4OzvDwcEBmzZtAgDMmjUL06ZNQ/v27RVGh9VtNzs7W3ywVs7GxgYFBQW4ceOGQtlp06ahbt26ZR6nMu7u7vj222/Ru3dvODg4lFjqUk4ikSAwMBCrVq0StxWfI198GlNAQABcXFzg4OCAtLQ07N27t8LxAcCFCxfg7u4OFxcXeHt7IygoCA0aNEC9evWwbNkydOvWDVKpFP/5z39KvQnx8PBQ+0Fq+XE5OzvD3d0d9erVw4YNG8R9b86RDw8PB/D3HHlXV1fMnj1boQ4RERGpTyK87aRcIqIqJl/9xnXKemjp6Vd1OO+1S8vLfh6GiIiqjvzftPT0dBgbG6ssyxF5IiIiIiINxFVrSMHmzZuxevVqhW0DBw7E3LlzqyiivyUmJmLYsGEK26ysrMQpG5Xl6NGjJZZE7NixI9asWVOp/byNdxnjpEmTFH6kCgBWrFgBb2/vSu+LiIiI1MepNUSk8crzNSQREdH7jFNriIiIiIg+cEzkiYiIiIg0EBN5IiIiIiINxIddieiD0XH+jn/98pNcXpKI6N+DI/JERERERBqIiTwRERERkQZiIk9EREREpIGYyBMRERERaSAm8kREREREGoiJPBERERGRBmIiX4xEIsH8+fPFz7NmzcKWLVsAAJ07d8bVq1fFfVFRUfD19QUALFy4EFpaWnjy5Im4//PPP4dEIlHaV/H6csX7UBXLyJEjceDAAQBATk4Oxo8fj+bNm6N58+YYM2YMXr58CQDYsmULZs2apdCHhYUFsrKyIJPJYGBgAKlUKr5Onjyp1nmqLAsWLEB0dPQ/2ocgCPjmm29gbW0NJycntGnTBlFRUQCA1NRU+Pv7w8rKClZWVvjyyy9RWFgIoOjc1a1bF1KpFNbW1ujduzeuXLkittu5c2fY2dmJ52769OlKY5CXdXV1RcuWLREfHy/us7CwgIuLi9jO8uXLxTrya0FZmdzcXEybNg3NmzeHk5MTunTpgoSEBLFc/fr10bhxY0ilUnTo0KHU2KKiolCzZk1IpVLY2dkpXC/Fz4H8lZmZKe4fOnQoWrVqpdBeTEwMWrduDVdXV9jb2+O///2vuG/16tWwsbGBjY0NPvroI8hkMnGfquudiIiISsd15IupUaMGtm/fjjlz5sDY2LhcdR0cHBAeHo7JkyejoKAA58+fR+3atf/xWObMmQMdHR3cvn0bADBlyhTMnDkTP/zwg1oxX7x4scIxFldQUAAtLa1y1Vm0aFGl9K3KmjVrcP78eSQkJMDAwACPHz/G+fPnAQCjRo1Chw4dEBYWhry8PAwYMADBwcGYMWMGAGD48OFYsWIFAGDPnj3w9vZGYmIi6tWrBwAIDw+Hk5OTWnHIy27atAmzZ89GZGSkuO/s2bOoUaOGyvqllZk7dy5evnyJ5ORkaGtr49atW7h//754o7Bw4UKYmZlh8uTJKtv29vZGeHg4Xr58CalUCh8fH7Rr167EOSguJycHp0+fhrGxMe7duwdLS0sAwNixY7Fv3z5YW1sjLy9PTNZ3796NX3/9FRcvXoSxsTF27twJHx8fXL58GRKJpNz/7eXm5iI3N1f8nJGRUWYdIiKiDw1H5IvR09PDkCFD1EqC39S/f3/s3r0bQNEoZ4cOHVSOyFdGLFlZWdi+fTuCgoIgkUggkUgQFBSEX3/9VWHk9G1FRUWhS5cu6NWrF+zt7cWRU5lMBldXV4wbNw5ubm7Izc3F1KlT0apVK0ilUkRERAAAWrVqhbt374rt2draIi0tTeGbhWPHjkEqlcLJyQkzZsyAIAgAADMzM7FeSEgIFi5cCAAIDg6Gra0tXFxc8PnnnyuNffny5QgJCYGBgQEAoEGDBujTpw9u3bqFa9euiSPQurq6+P7777Fy5cpS2/Hx8cEnn3yCHTt2VOAM/q19+/ZISUl5qzYAIDs7G9u2bcOqVaugrV10P25tbQ1vb+8Kt2lgYAB3d3e14jt8+DC8vLwwaNAghIWFidv/+usv8QZWV1cXNjY2AICVK1ciKChITNL9/PxgYmKC48ePAyj/f3tLly6FiYmJ+GrSpEm5jpWIiOhDwET+DVOnTsWGDRuQk5NTrnpmZmbQ1dXFo0ePEBYWBn9//388ljt37sDc3FxhpLZGjRqwsLDAnTt3ymw/KSlJYdqEqjrnzp3DunXrkJCQgCNHjoij2teuXcOUKVNw5coVbN68GRYWFjh//jxOnz6NefPmITc3F35+fggPDwcAXLx4Ec2aNVP4tuLVq1cYN24cIiIicOXKFdy8eRN79uxRGfuiRYtw+fJlXLlyBUFBQaWWycjIwMuXL2Fubl7qsbu6uircbFlYWODVq1dKR3fd3d2RnJwsfvb19RXP3ffff68yXrmDBw+id+/eCts8PT3FdkJDQ0ut92aZO3fuoGnTpmWO5JdHWloabty4oTANZ9u2bWK/Xl5e4vawsDAMGDAAfn5+Con8pEmTYG1tDT8/P2zduhUFBQUAis63m5ubQn9ubm64fv26+Lk8/+3NmzcP6enp4uvhw4cVPm4iIiJNxak1b6hTpw569uyJ//3vfwrbSxtdf3Nb//79ERoaigsXLmD9+vUq+1E2Wl98u7JY1FVWH+WZWtOuXTtx1LNfv344e/Ys+vbtCxsbG7i4uAAAIiMjce3aNWzduhVA0ajxH3/8AV9fXwwcOBBz5szBzp074efnp9B2cnIybG1tYWFhAQAYPHgwoqOj0a9fP6XxtGrVCkOHDoWfnx/69u1b7nOgjPybAHX2lWdqja+vL3Jzc5GVlaUwRx6o2NSahISEt/rGp7jjx4/D1dUVN2/exPTp09GwYUNxX2lTa169eoWYmBhs27YNOjo6KCgowK1bt2BtbY2vv/4agwcPxpEjR/D999/j+PHj+Omnn9SKozzXu56eHvT09Mp3oERERB8YjsiXYtasWVi9ejXy8/PFbaampnj+/Ln4OS0tTWHaB1A0/WLp0qXo1KlTmUnWm+0pa7O0WOSaN2+O+/fvIysrS9yWlZWFP//8E7a2tqX2kZOTI04zKY83j0f+uXhbgiBgw4YNiI+PR3x8PB48eIBmzZrB0tIShYWFkMlk2Lt3r8rEW96OvP3i/RafE33w4EFMnjwZMTEx6NixY6ntGBsbo3r16rh//36Jffb29khISBAfbgUgPgCsbI52fHw87OzsVMauTHh4OO7evYthw4ZhypQpFWqjOCsrK9y/fx/Z2dlv3Za3tzcSEhKQlJSEjRs3lrjReNPBgweRlpYGa2trWFhY4P79+wqj8ra2tpg6dSoiIyPFb1bs7e1x+fJlhXbi4uJgb2+vsE3V9U5ERESKmMiXokmTJmjXrh127dolbuvYsSN+/vlnAEBhYSF27NhRYiUQMzMzLFu2DBMnTiyzD2tra8hkMnE6S3x8PPLz88UHKVXFIlejRg0MGjQIc+fOhSAIEAQBc+fOxeeffw59fX20bNkSp06dwrNnzwAA+/btg4uLS4VGcmNiYpCSkoK8vDxERESgbdu2Jcp4e3tj/fr14nSK4gmhr68vvvzyS1haWpZ4CNjW1hY3b97E/fv3UVhYiF9//VU8tyYmJrh//z5ev34tzqcvLCzEw4cP0bVrV6xatQoymUzs802zZ89GQECAuJLPH3/8gZ07d8LGxgb29vbinPjXr19j+vTp4oOub9q7dy+OHDmCQYMGleOsKZJIJFi8eDHOnj2rMEWnIgwNDTFs2DDMmjVLPPbr16/j6NGjFW7T0tISX3zxhbgqjjKhoaHYvn07ZDIZZDIZzp07Jybyhw4dEr+5SExMFKc1zZgxA/PmzROnLe3cuRPPnz9H165dFdpWdb0TERGRIk6tUeKLL74Qp4gAwPjx4zF16lS4urpCEAR0794dAwcOLFFv1KhRarWvp6eHzZs3Y9CgQcjLy4OhoaHSKQhvxlLc8uXLERAQACsrK6Snp6NXr174v//7PwBAvXr1sGzZMnTr1g2CIKBOnToKywHK58jLzZ8/v8SSmHLt2rVDQEAArl+/jv79+6NVq1YKywcCRefo7t27kEqlEAQBNjY24gPAfn5+mDt3Ln788ccSbevr62PDhg3o06cP8vPz0b17d3HUfvHixejSpQssLS3RrFkzAEUr5AwZMgSZmZkQBAFff/210hVzAgICkJ6eDhcXF+jr68PIyAhLly4FULS84oQJE2BlZQVBEODn54dp06aJdbdt24bjx4/j5cuXsLOzQ2RkJOrWrSvu9/X1RfXq1QEALi4u2LZtW6kxFGdgYIBZs2YhODhYfLDT09MT1aoV3VN//PHHCAoKQn5+vsLUkdLKBAUFYfbs2bC2toahoSHq1q2L4ODgMmNQZeLEibCxsREfeJWfA7nt27cjKipK4VhtbGxQUFCAGzduYOvWrZg+fTr09fWhr6+PzZs3Ayg6Vw8ePECLFi0gkUhgbm6OPXv2iMdUnKrrnYiIiP4mEVRNCiaNcvXqVfTp0wehoaHw8PCotHajoqIQEhIiPrBK/6zc3FzY2NggKSkJhoaGVR2ORsjIyICJiQlcp6yHlp5+VYdTpS4tH17VIRAR0VuQ/5uWnp5e5pLMHJH/gDg5Oam1Wg29v5KSktC/f39MnDiRSTwRERGpxET+H5aamlpiHrCBgQHOnj1bRREppyrWzp07V01QalqyZAl27typsG3GjBkYPvzdjU5OmjQJMTExCttWrFhRrrXdHRwcFJZkrCyJiYkYNmyYwjYrKyt+y0JERKTBOLWGiDReeb6GJCIiep+V5980rlpDRERERKSBmMgTEREREWkgJvJERERERBqID7sS0Qej4/wdXH6Sy08SEf1rcESeiIiIiEgDMZEnIiIiItJATOSJiIiIiDQQE3kiIiIiIg3ERJ6IiIiISAMxkSciIiIi0kBM5KnKSCQSTJw4Ufz8+PFjaGlpYeHChQCAhQsXonHjxpBKpZBKpejQoYNC/fbt28Pf319hW+fOnRXKHThwACNHjlQaw8iRI3HgwIG3P5h/QFRUFGrWrAk3NzfY2Niga9euOHXqlEKZO3fuQCKR4NChQwCACRMmYNWqVeL+EydOwMvLCwCwaNEiODk5wdnZGR4eHrh3757Svh8+fIi+ffuiefPmcHBwwJAhQ/D8+XMxJqlUCjs7O8yaNUuss2XLFtStW1f8e0mlUmRmZpZ5HPK/wahRoyCVSmFlZSX2IZVKkZycXCnnk4iI6EPDdeSpytSuXRuxsbEoKCiAlpYWwsPD4ejoqFBm7ty5mDx5com6f/zxB1JTU/HgwQNkZ2fD0NBQ3CeTyRAdHV0i8a9s8rj/Sd7e3ggPDwcAnD59Gn5+fjhx4gScnJwAAKGhofD09ERoaCg+/fRTfPvtt2jZsiWGDBkCU1NTzJw5E9u3b8fZs2cRFRWF+Ph4aGtrIyUlReGcFScIAnx8fBAQEICIiAgAwLFjx/D8+XOFmF6+fAmpVAofHx+0a9cOADB8+HCsWLGi3McBAJs3bwZQdAMTEhIili9Nbm4ucnNzxc8ZGRlqnU8iIqIPCUfkqcpIJBJ06NBBHJ3ds2cP+vXrp1bdsLAw+Pv746OPPioxoj5z5kx8++23asdx4MABtGvXDg4ODvj9998BFH0bEBISIpYxMzMDUJRkdu/eHf7+/vDy8kJWVhaGDx+Oli1bwsPDAzExMQCA2NhYeHp6ws3NDV26dMHjx49VtquOjh07YsKECdi0aZPCedi4cSN+++035Obmonbt2pg3bx7mzp2LkJAQdO3aFY6Ojnjy5Alq1aoFbe2ie/fGjRujVq1apfZz4sQJGBkZYfjwv39YqHv37mjWrJlCOQMDA7i7uyMlJUXtY1B2HOW1dOlSmJiYiK8mTZpUuC0iIiJNxUSeqpS/vz/CwsLw6NEj6Orqlkhsg4KCxCkWQ4YMEbfLE3k/Pz+EhoYq1PH29kZaWhri4uLUiuHx48c4c+YMtm3bhnHjxpVZ/ty5cwgODsbp06exePFi+Pj44MKFC4iIiBCnCjk4OCA6OhpxcXEYO3YsvvvuO7ViKYu7u7s41SQ5ORm6urpwcHBAhw4dcPToUQDAmDFjkJycjLVr14rTlLp164abN2/C3t4eU6dOxYULF5T2kZSUBKlUWmYsaWlpuHHjhsI3H9u2bRP/XvIpPWUdR0XMmzcP6enp4uvhw4cVbouIiEhTMZGnKuXp6Ynz58/j119/ha+vb4n9c+fORXx8POLj47F9+3YAEKfTODo6okuXLrhw4QIyMzNL1Fu6dKlaMQwYMAASiQQeHh7Iy8vDixcvVJZv164dGjZsCACIjIxEYGAgpFIpevbsidTUVOTl5eH58+fw8fGBk5MTFi1ahKSkJLViKYsgCOL70NBQ8RkB+Q0RUPRNx9ixY9GrVy8YGRkBAIyMjBAXF4c1a9agevXq6NatG44dO6a0D4lEojSG48ePw9XVFY0aNcKnn34qngugaGqN/O918uRJtY6jIvT09GBsbKzwIiIi+rdhIk9VSiKRoGPHjggKCoKPj49adcLCwvDgwQNYWFjAysoKL168wL59+xTK9O3bF9evX8fNmzfViuHNz9ra2igsLBS3FZ+PbWBgIL4XBAEHDhwQk9eUlBTo6upiwYIF6NGjB65evYotW7aI9VW1q474+HjY2dkBKDoPwcHBsLCwQEBAAA4ePIicnBwAQLVq1VCtmuJ/3tra2ujWrRuWLVuGr776qsQ5k3NwcFD5bYa3tzcSEhKQlJSEjRs3Ij4+vlzH8OZxEBERUcUwkacqN2nSJCxbtgympqZqlQ8NDcWpU6cgk8kgk8mwa9cucTRaTiKRYPbs2fj+++/LbG/nzp0QBAGXL1+Grq4uTExMYG5uLiaokZGRyMrKKrWut7c31q5dK35OSEgAUPTwZePGjQEAP//8s7hf3XZLExMTg//+978YO3YskpKSoKOjg5SUFMhkMjx48ACffPIJDh8+XGrd5ORk3LlzB0DRzce1a9dgbm6u9JjS09MV4t6/fz/u3r2rUM7S0hJffPEFli9frvYxvHkcREREVHFM5KnKWVtbY9SoUaXuKz5HXr4UYWpqKlxdXcUyXl5eOHfuHNLT0xXqDh48WK1VZSwsLNC+fXsMHToUGzduBAD069cP9+/fR4sWLXDkyBGlNxkLFizA06dP4ezsDAcHB/EBzlmzZmHatGlo3769wgi+uu3KHT9+XFy2MTAwEGFhYXB0dERoaCj69u2rUNbHx6fE8wJyWVlZGDp0KBwdHeHk5ITCwsJSVwMCim6CIiIiEBYWBisrKzg6OiIsLAy1a9cuUXbixIk4deqU+MBr8TnyUqkUMplM5XEQERFRxUmEt52sSkRUxTIyMmBiYgLXKeuhpadf1eFUqUvLh5ddiIiI3lvyf9PS09PLfAaMI/JERERERBqIPwhF/wqbN2/G6tWrFbYNHDgQc+fOraKI/paYmIhhw4YpbLOyslL5g0iVpXXr1iUeuJX/EisRERG93zi1hog0Xnm+hiQiInqfcWoNEREREdEHjok8EREREZEGYiJPRERERKSB+LArEX0wOs7f8a9YfpJLTBIREcAReSIiIiIijcREnoiIiIhIAzGRJyIiIiLSQEzkiYiIiIg0EBN5IiIiIiINxESeiIiIiEgDMZEntWlra0MqlcLR0RG9evXCixcvAAAymQwGBgaQSqXi6+TJkwCAlJQU9O/fH82aNUOLFi3g7e2N8+fPAwAWLlyIkJAQAEBhYSG++uorWFlZwcrKCv7+/khNTRXbl0gk2LRpkxiLr68voqKilMZqYWEBFxcXuLi4oFOnTrh//77C/vbt28Pf3x8AsHbtWjFu+TFKpVJs374dI0eOxIEDBwAAnTt3RocOHcQ2Dhw4gJEjR4qf9+7dCzc3Nzg4OMDV1RWff/45Xr58qTTGhw8fom/fvmjevDkcHBwwZMgQPH/+HFFRUfD19S31mLKyshT+FsVjlZcZOnSoWCckJAQLFy5UeYxvioiIwKBBg8TP48ePR69evcTPX3zxBdasWaP0uKKjo9G6dWvY2dnB3t4eCxYsAPD33/urr76CVCqFg4ODwnWzdOlSBAQEiO3cuHEDzs7OyMvLU9oXERHRvxkTeVJbzZo1ER8fj2vXrqFmzZpYu3atuM/BwQHx8fHiy8vLC4IgoG/fvujRowfu3r2LS5cuYcWKFbh7926JtoODg5GUlISkpCTcvn0bLVu2xKhRo8T99evXx6pVq1BYWKh2vGfPnsWVK1fQoUMHLF68WNz+xx9/IDU1FbGxscjOzsakSZPEuOXHGB8fjyFDhpRoUyaTITo6usT2uLg4zJo1Czt37kRSUhIuX74MZ2dnPH/+vNTYBEGAj48P+vXrhzt37iApKQkjRoxQWv5NxeN8M9aYmBjcu3dPoXx5jrFt27aIjY0VP1+9ehXPnj0TP8fGxqJt27alxvXo0SMMHToU69evx40bN5CYmIgmTZoolFmyZAni4+Nx6NAhhetm1qxZ+O2333D16lUAwNSpU7Fy5Uro6uqW6Cc3NxcZGRkKLyIion8bJvJUIe3atUNKSorKMsePH4eRkRFGjx4tbpNKpRg4cGCJsitXrkRwcLCYtM2cORPXrl3DrVu3AACNGjWCu7s7du7cWe5Y27dvrxBrWFgY/P398dFHH4mj7eqaOXMmvv322xLbV6xYgfnz58PKygoAoKWlhYkTJ6JRo0altnPixAkYGRlh+PC/f9ine/fuaNasWbniKc3UqVPx3XffVbh+vXr1AABPnjxBdnY29PX1YWNjg1u3bqGgoADXrl2DVCotte66deswevRouLm5ASj65mDcuHFq9aujo4Pg4GAEBAQgIiICNWrUQPfu3Ustu3TpUpiYmIivN28WiIiI/g2YyFO5FRQUIDIyEj179hS3JSUlKUz1uHPnDq5fv6404SsuIyMDr169grm5ubitWrVqcHFxwfXr18Vtc+fOxbJly8od76FDh9C7d2/xszyR9/PzQ2hoaLna8vb2RlpaGuLi4hS2q3uscvLzVVEvXrxQON/FvyUYPnw4IiMj8eTJkwq336ZNG8TGxuL8+fPw8PBAmzZt8Pvvv+PKlSuwt7eHjo5OqfXe9ri8vb1hZmaGCRMmYNWqVUrLzZs3D+np6eLr4cOHFe6TiIhIU2lXdQCkOeTJY0pKChwdHfHRRx+J+xwcHHDx4kWF8oIgQCKRiJ99fX2RlJSEdu3aYePGjUrLKePk5IQmTZrg0KFDasXr6emJZ8+ewdjYGEuWLAEAPHjwANnZ2XB0dIStrS3GjBmDzMxMGBkZqdUmUHRDsXTpUoXR9OLHkJSUhMGDByM9PR0bN26Et7d3iTbUPWZl5NNjSqOrq4tJkyZh1apVaNq0aYXal0+vMTExQevWrWFubo6NGzciOztb6bQa4O2PCwAmTJiA/Px8hRu7N+np6UFPT++t+iEiItJ0HJEntcmTxwcPHiA/Px/r1q1TWV4+/1kuPDwc69atKzEP3MTEBNWrV4dMJhO3FRYWIiEhAfb29gpl582bh6VLl6oV79mzZ/HgwQO4uLggMDAQQNFo/IMHD2BhYQErKyu8ePEC+/btU6s9ub59++L69eu4efNmqccqf9+pUyfk5OSU2oaDg0OJUf3K9Nlnn2Hnzp1qz7l/k3xE/ty5c2jdujVcXFxw5coVlfPjgco5rmrVqqFaNf6viYiIqCz815LKzcDAAKtXr8bKlSuRn5+vtJy3tzfS09OxdetWcdurV69KLTt9+nRMnz5dXKFk5cqVcHR0hLW1tUK5Nm3aQEdHB+fOnVMrVm1tbXz//ffYunUrnj9/jtDQUJw6dQoymQwymQy7du1CWFiYWm3JSSQSzJ49G99//724bebMmViyZAnu3LlT5rECf5+bn3/+Wdy2f//+Uh8ErghDQ0OMGjUKGzZsqFB9Nzc3XL16FSkpKWjYsCG0tbWho6ODkydPok2bNkrrTZw4Ef/73//Em5qCggL85z//qVAMREREpBoTeaoQDw8PODs7Y9euXQBKzpEPDw+HRCJBREQEIiIiYGlpibZt22LNmjWYPn16ifamT58OW1tb2Nvbw8rKCufOncPmzZtL7XvevHllPmhbXKNGjTBo0CBs2LABqampcHV1Ffd5eXnh3LlzSE9PL9fxDx48GFpaWuJnd3d3LF26FP3794eDgwM8PT1Rp04deHp6llpffm7CwsJgZWUFR0dHhIWFoXbt2gCK5vU3btxYfL15vG/OkS9tOcgpU6aIS4SWl46ODuzs7GBpaSlu8/DwQLVq1dCgQQOl9Ro1aoStW7fis88+g52dHVxcXJCWllahGIiIiEg1iSAIQlUHQUT0NjIyMmBiYgLXKeuhpadf1eH84y4tH152ISIi0kjyf9PS09NhbGyssixH5ImIiIiINBBXrSGN5uPjU+LHj3755Rc4ODhUUUQltW7dGrm5uQrboqKiULNmzaoJqJhJkyYhJiZGYduKFStKXWmnuMTERAwbNkxhm5WVFcLDwys9RiIiIiodp9YQkcYrz9eQRERE77Py/JvGEXki0njy8YiMjIwqjoSIiOjtyP8tU2esnYk8EWm81NRUAECTJk2qOBIiIqLKkZmZCRMTE5VlmMgTkcaTL9v54MGDMv+nR1SWjIwMNGnSBA8fPuRULaoUvKaoPARBQGZmJho2bFhmWSbyRKTx5L8Ea2Jiwn8kqdIYGxvzeqJKxWuK1KXuoBSXnyQiIiIi0kBM5ImIiIiINBATeSLSeHp6eggMDISenl5Vh0IfAF5PVNl4TdE/hevIExERERFpII7IExERERFpICbyREREREQaiIk8EREREZEGYiJPRERERKSBmMgTkUbau3cvfH19YW5ujurVq6Nu3bpo27YtvvvuO6Snp1d1eFTFMjMzsWvXLkyePBmenp6oU6cOdHR0YGxsDDs7OwwfPhxHjhxBedZ7ePz4MQIDA+Hu7o7atWvDwMAAzZo1w4gRI3Dq1Kl/8GjofTdixAhIJBLxtXDhQrXq8ZqityYQEWmQjIwMoWfPngIApa/GjRsLMTExVR0qVZGVK1cK1atXV3mNyF8dOnQQ7t+/X2abu3fvFmrWrKmyrXHjxgn5+fnv4AjpfXL48OES10JgYGCZ9XhNUWXQ/qdvFIiIKkt+fj769++PyMhIAEC9evUwbtw4ODg4IC0tDTt27EBMTAxSUlLQo0cPnDlzBo6OjlUcNb1rN2/eRE5ODgCgcePG6Nq1Kzw8PFCnTh28evUK586dw88//4ysrCxER0ejc+fOiI2NRd26dUtt7/jx4xgwYABev34NAOjRowd69+4NQ0NDXL58GZs2bUJGRgY2btwIANiwYcO7OVCqchkZGfjss88AAIaGhsjOzlarHq8pqjRVfSdBRKSudevWiSNVDg4OwpMnT0qUmTlzplimXbt2VRAlVbUJEyYI3bt3F44dOyYUFBSUWkYmkwm2trbitTJq1KhSy7169Upo2rSpWO4///lPiTLJyclC/fr1xTLHjh2r1OOh99dnn30mfgs4Y8YMtUbkeU1RZWIiT0QaIT8/X+EftkuXLiktJ5VKxXJHjhx5x5FSVUtNTVWrXHx8vHidGBgYCNnZ2SXKhISEiGV69eqltK1du3aJ5dq0aVPh2Elz/Pbbb4JEIhEACPv37xcCAwPVSuR5TVFl4sOuRKQRTp06hSdPngAAOnXqBHd391LLaWlpISAgQPy8Y8eOdxIfvT9q166tVjlXV1fY2dkBAF6+fInbt2+XKBMaGiq+nzFjhtK2+vbtCwsLCwBAbGwsZDKZ+gGTxnn58iXGjh0LQRAwYMAA9OzZU+26vKaoMjGRJyKNcOTIEfH9p59+qrJs8f2HDx/+x2IizWdkZCS+f/XqlcK+zMxMxMTEiOU6dOigtJ1q1arh448/Fj/zuvuwzZs3D3fv3kWtWrWwevVqtevxmqLKxkSeiDRCYmKi+L5ly5Yqy9arVw9NmjQBADx9+hR//fXXPxobaabc3FzcvHlT/Gxubq6wPykpCYWFhQAANzc3aGlpqWyv+HV59erVSoyU3idnz55FSEgIAGDlypWoV6+e2nV5TVFlYyJPRBohOTlZfG9paVlm+eJlitclktuxY4f4mwPu7u6oX7++wn5ec/SmnJwcjB49GoWFhejatStGjRpVrvq8pqiyMZEnIo3w4sUL8b2ZmVmZ5U1NTUutSwQAf/31F+bMmSN+nj9/fokyvOboTQsWLEBycjL09fXx3//+t9z1eU1RZWMiT0QaISsrS3xfvXr1Msvr6+uL7zMzM/+RmEgz5eXloX///uKUq759+8LHx6dEOV5zVNzFixexatUqAMDXX3+N5s2bl7sNXlNU2ZjIExHRv0ZhYSFGjx6N6OhoAEDz5s3xv//9r8x6Eonknw6N3mN5eXkYNWoUCgoK4ObmpnK1GXXxmqLKwESeiDRCjRo1xPfyX+1UpfgKJMVXJqF/L0EQMGHCBGzfvh0A0LRpUxw/fhy1atUqtXzxa+7NFW1Kw2vuw7V48WJcvXoVWlpa2LRpU5kPqSrDa4oqGxN5ItIINWvWFN8/e/aszPKpqaml1qV/J0EQMHHiRPEn7xs3bozffvtNXKe7NLzmCAASEhIQFBQEoGjdd2W/YaEOXlNU2bSrOgAiInXY2tri3r17AIB79+6pTMDkZYrXpX8vQRAwadIkrF+/HgDQqFEjnDx5ssw5zsWvm+LXkzK85j5MW7ZswevXr1GtWjXo6Ohg8eLFpZY7ffq0wnt5OVtbW/j5+Ynv5XhNUWVgIk9EGsHZ2Vn8UagLFy7Ay8tLadk///wTDx8+BADUrVsXderUeScx0vtHnsT/8MMPAICGDRvi5MmTsLKyKrOug4MDqlWrhsLCQsTFxaGgoEDllIoLFy6I752cnN4+eHovCIIAoOj5im+//VatOidPnsTJkycBAH369BETeV5TVNk4tYaINEJ5fuHw0KFD4vuyfgWWPlxvJvENGjTAyZMnYW1trVZ9IyMjtGvXDkDRiiFnzpxRWrawsBBHjx4VP3/yySdvETl9qHhNUWVjIk9EGqFTp07iD/ZERUXh8uXLpZYrKCjAmjVrxM8DBw58J/HR+2fy5MliEl+/fn2cPHkSNjY25Wqj+PWzcuVKpeUiIiLEaRBt2rQpc+oXaY7g4GAIglDmKzAwUKwTGBgobo+IiFBoj9cUVSYm8kSkEbS0tLBgwQLx8/Dhw/H06dMS5ebOnYv4+HgAQLt27fDRRx+9qxDpPTJlyhSsW7cOwN9JfEXmGI8ePRpNmzYFAOzfvx9r164tUebWrVuYNGmS+Pmbb76pYNT0b8BriioT58gTkcYYN24c9uzZg8jISFy7dg2urq4YN24cHBwckJaWhh07dohfVZuYmFTolxdJ882fPx8hISEAitbqnjp1Km7cuIEbN26orOfu7i4mWHLVq1fHjz/+iE8//RSvX7/G5MmTceTIEfTu3RuGhoa4fPkyNm3ahPT0dABF16i3t/c/c2D0QeA1RZVKICLSIBkZGULPnj0FAEpfjRs3FmJiYqo6VKoinTp1Unl9KHtt3rxZaZu7d+8WatasqbL+uHHjhPz8/Hd3oPReCQwMFK+FwMDAMsvzmqLKwBF5ItIoRkZG2L9/P/bu3Ytt27bhwoULePr0KYyMjNC8eXP069cP48ePh4mJSVWHSh8QHx8ftGnTBj/88AP2798PmUyGnJwcNGjQAO3bt8eYMWPQqVOnqg6TNAivKaoMEkH4/+sqERERERGRxuDDrkREREREGoiJPBERERGRBmIiT0RERESkgZjIExERERFpICbyREREREQaiIk8EREREZEGYiJPRERERKSBmMgTEREREWkgJvJERERERBqIiTwRERERkQZiIk9EREREpIGYyBMR0QdLJpNBIpFAIpFg5MiRVR0OEVGl0q7qAIiIiEgzvXjxAsHBwQAAqVSKvn37Vmk8RP82TOSJiIioQl68eIGvv/4aADBixAgm8kTvGBN5IiL6YFlYWEAQhKoOg4joH8E58kREREREGoiJPBERERGRBmIiT0REH6yyVq3p3LmzuB8ABEHAL7/8gu7du6Nhw4bQ19eHra0tAgIC8McffyjUzcvLw5YtW9C5c2c0aNAA1atXh5WVFWbMmIHU1FSVcVlYWEAikcDCwgIAkJOTg9WrV6Nt27aoU6cO9PX1YWVlhUmTJuH27dtqH+/p06cxZswY2NjYwMjICAYGBmjWrBmGDBmCAwcOlFn/zbhyc3Oxdu1a8Ri1tLRgYWEhnldLS0ux7tatW8VzWfwVFRWl0EdOTg727t2LgIAAeHp6ok6dOtDR0YGRkRGsra0xbNgwREZGlhnrwoULS/Rx6dIljBo1Cs2aNUP16tVhamoKLy8vbN68GYWFhWqdw8LCQuzcuRNDhgyBlZUVjI2Noauri/r168PLywuBgYG4fv26yjbS09OxcuVKeHt7o2HDhtDT00Pt2rXRokULzJs3r8S1RFRhAhER0Qfq3r17AgABgDBixIgS+zt16iTuz87OFnr06CF+fvNVp04dITExURAEQfjzzz8FT09PpWWbNGki3Lt3T2lc5ubmAgDB3NxcePjwoeDq6qq0rerVqws//fSTyuN8+fKlMGDAAKVtyF9du3YV0tLS1Irr3r17gpOTU4k25PvK6kv+OnnypEIflpaWatXr06ePkJmZqTTWwMBAhT6CgoIELS0tpe316tVLyMvLU3keExMTBQcHB7XiUyYsLEyoXbu2yrrVq1cXtmzZojIWInXwYVciIiIAo0ePxsGDB+Hu7o5BgwahadOmePr0KbZu3YqLFy/ir7/+Qv/+/ZGQkIAePXrg4sWL8PLyQt++fdGgQQM8fPgQGzZsQHJyMh4+fIhRo0bh5MmTKvt8/fo1/Pz8kJCQADc3NwwZMgRNmjTBkydPsHPnTpw5cwY5OTkYMWIEatasiZ49e5Zoo7CwEL169cKJEycAAAYGBhg+fDjatm0LLS0txMXFYfPmzUhLS8OJEyfQuXNnxMbGQl9fX2lcubm56NevH65evYo2bdrA19cXTZo0wYsXL5CYmIi6detiz549ePr0KcaPHw8A8PLyQkBAQIm2nJycFD6/fPkSNWvWRJcuXeDm5gZzc3MYGBggIyMDV65cQWhoKB4/foy9e/di9OjRCAsLK/Nvt2nTJmzfvh116tTByJEj4eLigmrVqiE2NhabNm3Cq1evsH//fgQFBeH//u//Sm1D/vfMysoCADRs2BD+/v5wdXWFoaEhnj17hri4OBw6dEjpiPrGjRsxfvx4CIIAbW1t9OzZE126dEH9+vWRnZ2NmJgYbN++Ha9evcLIkSOhq6uLQYMGlXl8REpV9Z0EERHRP6U8I/IAhLlz5wqFhYUKZV6/fi106dJFLOPh4SEAENavX1+ivYyMDIUR3QsXLpQal3zkW/6aMmWKUFBQUKLct99+K5apX7++kJGRUaLMihUrxDJNmzYVbt26VaLM48ePFUbXp02bplZcQUFBpZaTK+v8lubQoUMqR8azs7MFHx8fsd3o6OhSyxUfkQcgdOrUSXj+/HmJcjExMYK2trYAQDA1NRVycnJKlMnMzBSaNGkitjVmzBjh5cuXpfZbWFgo7N69u8T2hIQEQU9PT/xGJj4+vtT6N27cEBo3biwAEIyMjITU1FSl54KoLEzkiYjog1WeRL5z585K24mOjlZIGkePHq207E8//SSW++abb0otUzxh9vDwKDWJl+vVq5dYdu3atQr78vLyhAYNGggABIlEIpw9e1ZpOzdv3hQTTX19feHZs2cq4+rTp4/StuQqksirIz09XTA0NBQACGPHji21TPFEvlatWsJff/2ltL3BgwervDH47rvvxP2ffPJJiZs5dchvPrS0tITLly+rLBsZGan2zRKRKnzYlYiICMC0adOU7mvdujV0dHTEz1OnTlVatkOHDuL7a9euldnvrFmzUK2a8n+O58yZI74PDw9X2Pf777/j8ePHAIBOnTqhbdu2StuxtraGn58fAODVq1c4fPiwyrhKmybzrhgbG8PZ2RkAEBsbW2b54cOHw8zMTOn+rl27iu9L+5ts27ZNfB8UFCQ+/KyuFy9eYO/evQCAbt26wc3NTWV5+UOwAHD06NFy9UVUHOfIExERASqTYB0dHZiamuLJkycwMDAQk8zS1K9fX3z//PnzMvv19vZWud/T0xOGhobIzs7GhQsXIAiCmGgWT3I/+uijMvv6+OOP8fPPP4t1hw4dWmo5LS0teHp6ltleRT1//hzbt2/HkSNHcPXqVaSmpiI7O7vUH+9KSUkps72yYm3UqJFC32/GIk/uLS0t4eLios4hKIiJiRFXxTEyMkJERESZdWrUqAEASEpKKnd/RHJM5ImIiACVI7oAoKenBwAwNTVVOWIrLwcULbWoSq1atWBqaqqyTLVq1WBpaYmrV68iKysLmZmZMDY2BgBxNB4AbGxsVLYDALa2tuL7R48eKS1namqK6tWrl9leRezduxdjxowpc4lOuYyMjDLLqPu3A0r+TVJSUsQbCAcHB7ViepNMJhPf79y5Ezt37lS7blpaWoX6JAKYyBMREQGAyuktFSmnDkNDw3KXK57IZ2Zmlqst+Sjwm3XfpGpFm7fx+++/w9fXF/n5+QAAFxcXeHt7w8rKCrVq1YKenp54kzR//nxcu3ZNrfXf3+ZvUvxGofj5KY8XL15UuP/Xr19XuC4RE3kiIqIqkp2dXe5yRkZGpb5Xpy350opv1n1XFixYICbxa9euxcSJE5WWXbJkyTuJSX5TBCien/IofgMQHBys8hkKosrEh12JiIiqyPPnz8ucWlFYWIh79+4BKEoYiyfgDRo0EN/fvHmzzP6Kl5E/bPmuvH79WvwF1hYtWqhM4gHF6Sr/pEaNGonfAlR0vnrjxo3F91evXq2UuIjUwUSeiIioCkVGRqrc//vvv4uj7a1atVKYn9+mTRvxvTqrnxw5cqTUuhVVfEpLaQ+qFvfs2TNxNN7Kykpl2fPnz+PZs2dvHZ86ateuDUdHRwDAvXv3cOXKlXK30alTJ/HvcuDAAeTl5VVqjETKMJEnIiKqQqtWrVKZBK9YsUJ87+vrq7Cvbdu24sj6qVOnVC7VePv2bfEhTAMDA3z66advEzYAxSklZU3tKT6H//bt2yrLBgYGvl1g5TR8+HDx/dy5c8u8KXmTmZkZevToAQB48uQJVq5cWanxESnDRJ6IiKgKnT9/HjNmzCj1oc7vvvtOXMqwQYMGGDZsmMJ+HR0dzJo1C0DRiPjAgQNx586dEu08ffoU/fv3F1dsGT9+PGrXrv3WsdeuXRsmJiYAgPj4eJUJsLGxsbiyzqVLl0qsiQ8ABQUFmD59usI3B+/ChAkTxOkxhw8fxrhx4/Dq1atSywqCIK4ZX9zixYvF1XHmz5+P1atXqzwf6enpCA4OxvHjxyvhCOjfig+7EhERVZGGDRuiadOmCA4OxunTpzFkyBA0btwYT58+RVhYGKKjowEUTWHZuHFjqauqTJ06FQcPHsSJEydw//59uLi4YOTIkWjTpg20tLQQHx+PH3/8UZyL7+zsXKkPknbt2hW7d+/GnTt34O/vj379+qFmzZriVJNWrVqJNw3Tpk0T58b7+/tjwIAB6NSpE2rVqoXbt29j+/btuH79OpycnKCnp4dLly5VWpyqGBkZITw8HF27dkV2djZ+/PFHHD58GAMGDICrqysMDQ2RmpqKhIQEHDhwAA8fPiyRpLu6umLTpk0YMWIECgsLMW3aNKxbtw4+Pj6wt7eHoaEhMjMzcefOHZw/fx6nTp1CXl4efvrpp3dyjPSBqqJflCUiIvrH3bt3TwAgABBGjBhRYn+nTp3E/WUxNzcXAAjm5uZllpW32alTpzLbSklJEVxdXcU6b76qV68u/PTTTyr7e/nypeDv76+0DfmrS5cuQmpqaqUco1xCQoJgYGCgtM+TJ0+KZQsLC4XRo0erjNHZ2Vm4e/dumX+bwMDAUvsozcmTJ8WygYGBSsvFxcUJ1tbWZZ5HiUSitI1jx44JjRs3LrMNAIKenp5w+PBhlbETqcKpNURERFWoUaNGiI2NRXBwMNq0aQNTU1Po6emhWbNmmDBhAhITE5X+Aqucvr4+QkNDERUVhZEjR6J58+YwNDRE9erVYW5ujoEDB2Lfvn04ceJEpUypKc7FxQVxcXEYP368OPKs7AezJBIJfvzxR+zatQsfffQRTE1NoaOjg/r166NTp04ICQnB+fPnYWlpWakxqksqlSIpKQk//fQT+vfvj6ZNm0JfX1+M0cvLC4sWLVK5QlC3bt1w584dbNmyBX5+frC0tESNGjWgra2NWrVqwc3NDSNHjsS2bdvw5MkTfPzxx+/wCOlDIxGEcj7RQURERG/FwsIC9+/fh7m5+TtbZpGIPjwckSciIiIi0kBM5ImIiIiINBATeSIiIiIiDcREnoiIiIhIAzGRJyIiIiLSQFy1hoiIiIhIA3FEnoiIiIhIAzGRJyIiIiLSQEzkiYiIiIg0EBN5IiIiIiINxESeiIiIiEgDMZEnIiIiItJATOSJiIiIiDQQE3kiIiIiIg30/wA5NNj5chyi2wAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 800x600 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "features_importance = pd.DataFrame(\n",
    "    {\n",
    "        'importance': lgbm_model.feature_importances_,\n",
    "        'feature': lgbm_model.feature_name_\n",
    "    }\n",
    ")\n",
    "features_importance_plot = features_importance.sort_values(by='importance', ascending=False).head(20)\n",
    "\n",
    "plt.figure(figsize=(8, 6), dpi=100) \n",
    "sns.barplot(data=features_importance_plot, x='importance', y='feature')\n",
    "\n",
    "plt.yticks(fontsize=7) # 进一步微调\n",
    "plt.title('Feature Importance', fontsize=14)\n",
    "plt.tight_layout()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fddbd75e",
   "metadata": {},
   "source": [
    "可以看到，和之前手动特征工程结果几乎一样。这是符合的。\n",
    "\n",
    "思想过程几乎一样，一个自动一个手动而已"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "data-analysis",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.25"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}