{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c5f3d4a6-19ff-4311-a071-6f86210485b1",
   "metadata": {},
   "source": [
    "# _MethylBERT_ fine-tuning\n",
    "\n",
    "Once the pure cell-type data is preprocessed following the [previous tutorial](https://github.com/hanyangii/methylbert/blob/main/tutorials/02_Preprocessing_training_data.ipynb), you can fine-tune the pre-trained MethylBERT model. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "03dab99c-174f-44dc-beba-c763c5204b00",
   "metadata": {},
   "outputs": [
    
   ],
   "source": [
    "from methylbert.utils import set_seed\n",
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\") # Ignore warnings for a clear notebook\n",
    "\n",
    "set_seed(42)\n",
    "seq_len=150\n",
    "n_mers=3\n",
    "batch_size=5\n",
    "num_workers=20\n",
    "output_path = \"tmp/fine_tune/\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f40327b2-acd1-4e13-a0ce-f6d695e447d2",
   "metadata": {},
   "source": [
    "### Load training and evaluation data\n",
    "\n",
    "The preprocessed data must be loaded into the `MethylBertFinetuneDataset` and the [pytorch DataLoader](https://pytorch.org/tutorials/beginner/basics/data_tutorial.html) object. \n",
    "\n",
    "You can change the number of workers and the batch size depending on the CPU/GPU resources you have.\n",
    "Please find more detailed explanation [here](https://pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "990b1d40-9f2f-4fca-9340-2d465ff735b9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Building Vocab\n",
      "Total number of sequences :  3052\n",
      "# of reads in each label:  [1561. 1491.]\n",
      "Total number of sequences :  763\n",
      "# of reads in each label:  [383. 380.]\n"
     ]
    }
   ],
   "source": [
    "from torch.utils.data import DataLoader\n",
    "from methylbert.data.vocab import MethylVocab\n",
    "from methylbert.data.dataset import MethylBertFinetuneDataset\n",
    "\n",
    "# Creat a look-up table\n",
    "tokenizer = MethylVocab(n_mers)\n",
    "\n",
    "# Load the data files int a data set object\n",
    "train_dataset = MethylBertFinetuneDataset(\"tmp/train_seq.csv\", \n",
    "                                          tokenizer, \n",
    "                                          seq_len=seq_len)\n",
    "test_dataset = MethylBertFinetuneDataset(\"tmp/test_seq.csv\", \n",
    "                                         tokenizer,seq_len=seq_len) \n",
    "\n",
    "# Load the data into a data loader\n",
    "train_data_loader = DataLoader(train_dataset, batch_size= batch_size, \n",
    "                               num_workers= num_workers, pin_memory=False,  \n",
    "                               shuffle=True)\n",
    "test_data_loader = DataLoader(test_dataset, batch_size= batch_size, \n",
    "                              num_workers= num_workers, pin_memory=True,  \n",
    "                              shuffle=False)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8704759b-3457-4a71-8343-7e4b57f62286",
   "metadata": {},
   "source": [
    "### Create _MethylBERT_ model\n",
    "\n",
    "`MethylBERT FinetuneTrainer` gets the `DataLoader` objects and hyperparameters for the training as input. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "50725fec-1640-4555-90ba-fe65b0989dbb",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The model is loaded on CPU\n"
     ]
    }
   ],
   "source": [
    "from methylbert.trainer import MethylBertFinetuneTrainer\n",
    "import os\n",
    "\n",
    "trainer = MethylBertFinetuneTrainer(len(tokenizer), \n",
    "                      save_path=output_path, \n",
    "                      train_dataloader=train_data_loader, \n",
    "                      test_dataloader=test_data_loader,\n",
    "                      lr=1e-4, with_cuda=False, \n",
    "                      log_freq=1,\n",
    "                      #eval_freq=10, #activate this only when you want to evaluate the model with test_data_loader\n",
    "                      warmup_step=3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3490be8a-1b04-43e4-b214-89b64d8fd0dc",
   "metadata": {},
   "source": [
    "#### Load pre-training model from the uploaded models (huggingface.co/hanyangii/)\n",
    "\n",
    "Pre-trained models with 2, 4, 6, 8, and 12 encoder blocks are available for MethylBERT. \n",
    "You can give an argument \"hanyangii/methylbert_hg19_{number of encoder blocks}l\" to the `load` function. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "ec5e444c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Restore the pretrained model hanyangii/methylbert_hg19_4l\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "5ed74f3e19bd494e99be7dafd39cecb3",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "config.json:   0%|          | 0.00/1.11k [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "5e0b633737fd4d408985e8c9a56060a1",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "pytorch_model.bin:   0%|          | 0.00/120M [00:00<?, ?B/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Some weights of MethylBertEmbeddedDMR were not initialized from the model checkpoint at hanyangii/methylbert_hg19_4l and are newly initialized: ['dmr_encoder.0.weight', 'read_classifier.0.bias', 'read_classifier.0.weight', 'read_classifier.3.bias', 'read_classifier.3.weight', 'read_classifier.4.bias', 'read_classifier.4.weight']\n",
      "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total Parameters: 46929874\n"
     ]
    }
   ],
   "source": [
    "trainer.load(\"hanyangii/methylbert_hg19_4l\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "82de1a2d-3b09-4454-8b80-35b407474d35",
   "metadata": {},
   "source": [
    "#### Load pre-training model from a local directory\n",
    "\n",
    "If you have the pre-trained model saved in a directory containing `config.json` and `pytorch_model.bin` files"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "f33bb4e8-4134-436f-b60c-b27547e4096a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[0m\u001b[01;36mconfig.json\u001b[0m@  \u001b[01;36mpytorch_model.bin\u001b[0m@\n"
     ]
    }
   ],
   "source": [
    "ls ../test/data/pretrained_model/"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "082653be-527e-4777-97fb-713fe5329ed7",
   "metadata": {},
   "source": [
    "You can directly load the model using `load` function to `MethylBertFinetuneTrainer`. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "4e164645-3ecf-45dd-8736-25548d7debef",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Restore the pretrained model ../test/data/pretrained_model/\n",
      "Total Parameters: 32754130\n"
     ]
    }
   ],
   "source": [
    "trainer.load(\"../test/data/pretrained_model/\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "00fd81c0-2cbe-472c-a64f-7092b5ac1df2",
   "metadata": {},
   "source": [
    "#### Train the _MethylBERT_ model without pre-training\n",
    "If you do not have a pre-trained _MethylBERT_ model, you can create a new _MethylBERT_ model from a configuration file. \n",
    "For example, the baseline BERT model config file can be downlaoded from from https://huggingface.co/bert-base-uncased/blob/main/config.json."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "b8aa6175-25cb-40cc-b539-0ae82ca5997b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"architectures\": [\n",
      "    \"BertForMaskedLM\"\n",
      "  ],\n",
      "  \"attention_probs_dropout_prob\": 0.1,\n",
      "  \"gradient_checkpointing\": false,\n",
      "  \"hidden_act\": \"gelu\",\n",
      "  \"hidden_dropout_prob\": 0.1,\n",
      "  \"hidden_size\": 768,\n",
      "  \"initializer_range\": 0.02,\n",
      "  \"intermediate_size\": 3072,\n",
      "  \"layer_norm_eps\": 1e-12,\n",
      "  \"max_position_embeddings\": 512,\n",
      "  \"model_type\": \"bert\",\n",
      "  \"num_attention_heads\": 12,\n",
      "  \"num_hidden_layers\": 12,\n",
      "  \"pad_token_id\": 0,\n",
      "  \"position_embedding_type\": \"absolute\",\n",
      "  \"transformers_version\": \"4.6.0.dev0\",\n",
      "  \"type_vocab_size\": 2,\n",
      "  \"use_cache\": true,\n",
      "  \"vocab_size\": 30522\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "cat ../test/data/config.json"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0eda78e0-bb1f-4f48-9882-353e0d45b746",
   "metadata": {},
   "source": [
    "Then `create_model` function of the trainer creates a new _MethylBERT_ from the config file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "23a590ce-b632-4497-9398-678ade2d2350",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total Parameters: 32754130\n"
     ]
    }
   ],
   "source": [
    "trainer.create_model(config_file=\"../test/data/pretrained_model/config.json\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "681cf823-5802-4fd7-b4c0-3dec19d0fe7a",
   "metadata": {},
   "source": [
    "`train` function starts the training of _MethylBERT_ model during the given number of steps. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "aaf5dd61-2705-4e16-b06d-fe63957f8371",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Train Step 0 iter - loss : 0.802449 / lr : 0.000033\n",
      "Running time for iter = 0.6947791576385498\n",
      "Step 0 loss (0.695615) is lower than the current min loss (inf). Save the model at tmp/fine_tune/\n",
      "Step:0 Model Saved on: tmp/fine_tune/\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Train Step 1 iter - loss : 0.823174 / lr : 0.000067\n",
      "Running time for iter = 0.621410608291626\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Train Step 2 iter - loss : 1.258267 / lr : 0.000100\n",
      "Running time for iter = 0.5103580951690674\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Train Step 3 iter - loss : 0.490022 / lr : 0.000100\n",
      "Running time for iter = 0.5122716426849365\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Train Step 4 iter - loss : 1.356685 / lr : 0.000100\n",
      "Running time for iter = 0.5860369205474854\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Train Step 5 iter - loss : 0.959048 / lr : 0.000100\n",
      "Running time for iter = 0.6873147487640381\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Train Step 6 iter - loss : 0.939311 / lr : 0.000100\n",
      "Running time for iter = 0.4517858028411865\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Train Step 7 iter - loss : 0.776077 / lr : 0.000100\n",
      "Running time for iter = 0.6021583080291748\n",
      "Step 7 loss (0.691268) is lower than the current min loss (0.695615). Save the model at tmp/fine_tune/\n",
      "Step:7 Model Saved on: tmp/fine_tune/\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Train Step 8 iter - loss : 0.657497 / lr : 0.000100\n",
      "Running time for iter = 0.4712505340576172\n",
      "Step 8 loss (0.685933) is lower than the current min loss (0.691268). Save the model at tmp/fine_tune/\n",
      "Step:8 Model Saved on: tmp/fine_tune/\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Train Step 9 iter - loss : 0.690212 / lr : 0.000100\n",
      "Running time for iter = 0.4791133403778076\n"
     ]
    }
   ],
   "source": [
    "trainer.train(steps=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e0906eea-c0ac-435c-944d-13cd4381c3da",
   "metadata": {},
   "source": [
    "It creates six files in the given `save_path`:\n",
    "1. `config.json` and `pytorch_model.bin`: model configuration and the trained `MethylBERT` model\n",
    "2. `dmr_encoder.pickle` : The trained DMR encoder in the _MethylBERT_ model\n",
    "3. `read_classification_model.pickle` : The trained fully connected neural network for read classification\n",
    "4. `train.csv` and `eval.csv` : tracked training and evaluation loss and accuracy values during the training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "982c6c2d-a6ea-4925-9f13-1fe965a057fc",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "config.json         eval.csv           read_classification_model.pickle\n",
      "dmr_encoder.pickle  pytorch_model.bin  train.csv\n"
     ]
    }
   ],
   "source": [
    "ls tmp/fine_tune/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "df842203-c972-4c81-b708-75113d9e6e57",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>step</th>\n",
       "      <th>loss</th>\n",
       "      <th>ctype_acc</th>\n",
       "      <th>lr</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0.802449</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0.000033</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0.823174</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000067</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>1.258267</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0.000100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>0.490022</td>\n",
       "      <td>0.8</td>\n",
       "      <td>0.000100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>1.356685</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0.000100</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   step      loss  ctype_acc        lr\n",
       "0     0  0.802449        0.2  0.000033\n",
       "1     1  0.823174        0.0  0.000067\n",
       "2     2  1.258267        0.2  0.000100\n",
       "3     3  0.490022        0.8  0.000100\n",
       "4     4  1.356685        0.2  0.000100"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "df_train  = pd.read_csv(\"tmp/fine_tune/train.csv\", sep=\"\\t\")\n",
    "df_train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "e4078b38-6e6b-498c-9a8d-7681de8e309d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>step</th>\n",
       "      <th>loss</th>\n",
       "      <th>ctype_acc</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0.695615</td>\n",
       "      <td>0.522936</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>0.876368</td>\n",
       "      <td>0.501966</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>0.932430</td>\n",
       "      <td>0.501966</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>0.942029</td>\n",
       "      <td>0.501966</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>0.849511</td>\n",
       "      <td>0.501966</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   step      loss  ctype_acc\n",
       "0     0  0.695615   0.522936\n",
       "1     1  0.876368   0.501966\n",
       "2     2  0.932430   0.501966\n",
       "3     3  0.942029   0.501966\n",
       "4     4  0.849511   0.501966"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_eval  = pd.read_csv(\"tmp/fine_tune/eval.csv\", sep=\"\\t\")\n",
    "df_eval.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "f096d980-c69e-480c-a7cf-16e6086d75d8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEKCAYAAAAfGVI8AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAA/gUlEQVR4nO3dd3zU9f3A8dfnLvMyLptwCXsnYU+liqhFFNQq1Elddf20drjbqq2/Lmv9tW6tWrWtFrXgBCe4ByggIAESNoQkF5KQ5LLHfX5/fC8QRiDj7r433s/HIw9yd9/73jtHcu/vZ70/SmuNEEKI8GUxOwAhhBDmkkQghBBhThKBEEKEOUkEQggR5iQRCCFEmJNEIIQQYc5niUAp9axSqkwpteE4x01WSrUppeb7KhYhhBCd82WL4Hlg9rEOUEpZgT8D7/kwDiGEEMfgs0Sgtf4UqDzOYTcBi4EyX8UhhBDi2CLMemGlVBZwHnAqMPk4x14LXAsQFxc3ceTIkb4PUAghQsjq1avLtdbpR3vMtEQAPAjcobVuU0od80Ct9VPAUwCTJk3Sq1at8n10QggRQpRSuzp7zMxEMAl4yZME0oCzlFKtWuvXTYxJCCHCjmmJQGs9qP17pdTzwBJJAkII4X8+SwRKqYXAKUCaUqoI+A0QCaC1ftJXryuEEKJ7fJYItNYXd+PYK3wVhxAieLS0tFBUVERjY6PZoQStmJgYsrOziYyM7PJzzBwjEEKIQxQVFZGQkMDAgQM53iQScSStNRUVFRQVFTFo0KDjP8FDSkwIIQJGY2MjqampkgR6SClFampqt1tUkgiEEAFFkkDv9OT9k0QgBPD6t3spr20yOwwhTCGJQIS94qoGfv7yWh5cVmh2KMJkVVVVPP744z167llnnUVVVVWXj//tb3/LAw880KPX8jZJBCLsFThdALy5tpjGljaToxFmOlYiaGs79u/G22+/TVJSkg+i8j1JBCLsFZYaiaCmsZXlm6T+YTi788472bZtG+PGjeO2227j448/ZubMmVxyySWMHj0agB/84AdMnDiR3NxcnnrqqQPPHThwIOXl5ezcuZNRo0ZxzTXXkJuby6xZs2hoaDjm665du5Zp06YxZswYzjvvPPbv3w/Aww8/TE5ODmPGjOGiiy4C4JNPPmHcuHGMGzeO8ePH43K5ev1zy/RREfYKnC7SE6KxKsWi1XuYM6av2SEJ4N638tlYXOPVc+Y4EvnN2bmdPn7fffexYcMG1q5dC8DHH3/M119/zYYNGw5Mx3z22WdJSUmhoaGByZMnM2/ePFJTUw85z5YtW1i4cCFPP/00F1xwAYsXL2bBggWdvu5ll13GI488wowZM7jnnnu49957efDBB7nvvvvYsWMH0dHRB7qdHnjgAR577DGmT59ObW0tMTExvXtTkBaBEBQ6XYzMTOD8CVl8uqWcshpZzCQOmjJlyiFz8h9++GHGjh3LtGnT2LNnD1u2bDniOYMGDWLcuHEATJw4kZ07d3Z6/urqaqqqqpgxYwYAl19+OZ9++ikAY8aM4dJLL+WFF14gIsK4bp8+fTo333wzDz/8MFVVVQfu7w1pEYiw1ubWbC2rZcHUAcybmM3jH2/j9bV7ufbkIWaHFvaOdeXuT3FxcQe+//jjj1m2bBlfffUVNpuNU0455ahz9qOjow98b7Vaj9s11JmlS5fy6aef8uabb/K73/2O/Px87rzzTubMmcPbb7/NtGnTWLZsGb0tzS8tAhHW9lTW09jiZnhmAkPS4xnXL4nFq/eitTY7NGGChISEY/a5V1dXk5ycjM1mY/PmzaxYsaLXr2m320lOTuazzz4D4N///jczZszA7XazZ88eZs6cyf33309VVRW1tbVs27aN0aNHc8cddzBp0iQ2b97c6xikRSDCWvuMoRF9EgCYPzGbu17fwIa9NYzOtpsZmjBBamoq06dPJy8vjzPPPJM5c+Yc8vjs2bN58sknGTNmDCNGjGDatGleed1//vOfXH/99dTX1zN48GCee+452traWLBgAdXV1Wit+cUvfkFSUhJ33303H330EVarlZycHM4888xev74Ktisf2ZhGeNMjy7fwfx8Ukn/vGcRFR1Bd38LkPy7jkin9+e05gdE1EU42bdrEqFGjzA4j6B3tfVRKrdZaTzra8dI1JMJagdNFv5RY4qKNxrHdFsn3c/rwxtq9NLe6TY5OCP+QRCDC2hZn7YFuoXbzJ2Szv76FDzfLmgIRHiQRiLDV3Opm275ahh+WCE4alkZ6QjSL1xSZFJkQ/iWJQIStnRV1tLr1EYkgwmrhvPFZfLS5jAopRCfCgCQCEbYKPKUlDk8EAPMmZNPq1ryxttjfYQnhd5IIRNgqdLqwWhSD0+OOeGxEZgKjs+wsWi3dQyL0SSIQYavQ6WJgqo2YSOtRH58/MZuNJTVer3cjwkN7Ebqu3m8mSQQibBU6axmReWS3ULtzxjqItCoZNBYhTxKBCEuNLW3srKhjWEbniSA5LopTR2bwxtq9tLTJmoJw8cILLzBlyhTGjRvHddddR1tbG0888QS33377gWOef/55brrpJqDzstRd8de//pW8vDzy8vJ48MEHAairq2POnDmMHTuWvLw8Xn75ZcAokd1ekvrWW2/1zg/rISUmRFjaWlaL1hyzRQAwf2I/3st38knBPk7P6eOn6AQA79wJpd9595yZo+HM+zp9eNOmTbz88st88cUXREZGcsMNN/Diiy8yf/58TjjhBO6//34AXn75ZX79618DXStLfTSrV6/mueeeY+XKlWitmTp1KjNmzGD79u04HA6WLl0KGPWNKisree2119i8eTNKqW7thNYV0iIQYelYM4Y6OmVEOqlxUdI9FCaWL1/O6tWrmTx5MuPGjWP58uVs376d9PR0Bg8ezIoVK6ioqKCgoIDp06cDXStLfTSff/455513HnFxccTHx3P++efz2WefMXr0aJYtW8Ydd9zBZ599ht1uJzExkZiYGK6++mpeffVVbDabV39uaRGIsFRY5iLKamFg6rH/oCKtFs4dl8ULK3axv66Z5LgoP0UojnXl7itaay6//HL+9Kc/HfHYhRdeyCuvvMLIkSM577zzUEp1uSx1Z691NMOHD2f16tW8/fbb/PKXv2TWrFncc889fP311yxfvpyXXnqJRx99lA8//LBXP2tH0iIIU8FWbNDbCktdDMmIJ8J6/D+BeROzaG5z89Z6WVMQ6k477TQWLVpEWZlRXqSyspJdu3YBcP755/P666+zcOFCLrzwQqB3ZalPPvlkXn/9derr66mrq+O1117jpJNOori4GJvNxoIFC7j11ltZs2YNtbW1VFdXc9ZZZ/Hggw8e2EHNW6RFEKbuWLyeyrpmnrl8stmhmKLQWcukgcldOjbXYWdU30QWry7ishMG+jYwYaqcnBx+//vfM2vWLNxuN5GRkTz22GMMGDCA5ORkcnJy2LhxI1OmTAF6V5Z6woQJXHHFFQfOdfXVVzN+/Hjee+89brvtNiwWC5GRkTzxxBO4XC7OPfdcGhsb0Vrzt7/9zas/t5ShDkOuxhYm/n4ZFgX5987GalFmh+RXrsYWRv/2fW47YwQ3zhzapec889l2fr90Ex/84mSGHWdcQfSclKH2DilDLY5r2SYnza1uGlvcbN9Xa3Y4flfoNH7mw6uOHssPxmcRYVEskkFjEYIkEYShJetKiIk0/us3FFebHI3/bWnflew4U0c7SouP5pQR6by2Zi+tsqZAhBhJBGGmur6FT7fs4+Ip/YmOsJC/N/zKJxQ4XdiirGQlxXbrefMnZlPmauLzrYFVHiDUBFt3daDpyfsniSDMvLexlJY2zQ/GZTGybyL5YVhHp9DpYlhGPJZujo3MHJlBki1SCtH5UExMDBUVFZIMekhrTUVFBTExMd16nswaCjNvrSumf4qNMdl2ch2JLFlXjNYapcJnwLigtJaZI9K7/bzoCCvnjHXw0jd7qG5owR4b6YPowlt2djZFRUXs27fP7FCCVkxMDNnZ2d16jiSCMFJR28SX2yq47uTBKKXIc9j5z8rdFO1voF+Kd1cqBqrKumbKa5u6NT7Q0fyJ2fzrq10sWV/MpVMHeDk6ERkZyaBBg8wOI+xI11AYeTe/lDa3Zu4YBwC5jkQANuwNnwHjQmfXSkt0ZnSWnWEZ8SyW7iERQiQRhJEl60oYnB7HqL7Gh+CIzASsFhVW4wSFPZgx1JFSivkTs1mzuyosp96K0CSJIEyU1TSyYkcFZ49xHBgPiIm0MiwjnvwwmkJaUOoiMSaCjIToHp/jvPFZWBRSiE6EDJ8lAqXUs0qpMqXUhk4ev1Qptd7z9aVSaqyvYhHw9nclaA1nj+17yP05jkQ2hFmLYERmQq8GxzMSYzh5eDqvrtlLm1tmt4jg58sWwfPA7GM8vgOYobUeA/wO6N6ODqJblqwvYWRmAkMP24glz2Fnn6uJspquVUwMZlprCkpdPR4f6GjehGxKqhv5aluFFyITwlw+SwRa60+BymM8/qXWer/n5gqge/OdRJcVVzWwatd+5o7pe8Rj7QPG4TBOUOZqoqaxtcfjAx19P6cPCTER0j0kQkKgjBH8GHinsweVUtcqpVYppVbJ/OLuW7q+BODAbKGOcg4kgtAfJ+jqZjRdERNp5eyxDt7ZUIKrsaXX5xPCTKYnAqXUTIxEcEdnx2itn9JaT9JaT0pP7/5CoHC3ZH0xo7PsDEyLO+KxhJhIBqbawqJF0Nupo4ebPzGbxhY373xX6pXzCWEWUxOBUmoM8AxwrtZaOlt9YHdFPeuKqo/aLdQu12EPi+JzBaUu0uKjSfHSLmPj+yUxOC1OSk6IoGdaIlBK9QdeBX6ktS40K45Qt+Q7Y1etOcdKBFmJ7KlsoLo+tLs4jBlD8V47n1KKeROz+XpnJbsq6rx2XiH8zZfTRxcCXwEjlFJFSqkfK6WuV0pd7znkHiAVeFwptVYpJbvN+MBb60oY3z+J7OTOS0jkOuwA5JeEbqvA7dZsKav1WrdQu/PGZ6EULF6z16vnFcKffFZrSGt98XEevxq42levL2Dbvlo2ldRwz9ycYx7XPnNoY3ENJw5J80dofre3qoH65rZubUbTFY6kWKYPSePVNUX8/LRh3a5oKkQgMH2wWPjOknUlKHXsbiEwNl3JTIwJ6ZpD7TOGfLHN5PyJ2RTtb+DrnZ3OlhYioEkiCFFaa95aX8zkgSn0STx+bfJcR2jvTVBwYMaQ98YI2p2Rm0l8dIQMGougJYkgRBU4XWwtq+Xs47QG2uVm2dm2r5aG5jYfR2aOQqeLrKRYEmK8v4dAbJSVOaP78vZ3JdQ1tXr9/EL4miSCELVkXQkWBWeO7mIicCTi1rCpNDRbBYXOWp+0BtrNm5hNfXMb726QNQUi+EgiCEFaa5asL+bEIWmkxXetymZelmfmUAh2D7W2udlWVstwL5SW6Mzkgcn0T7FJyQkRlCQRhKD84hp2VtQfcxHZ4Rz2GJJskeSH4IDxzop6mtvcDM/wXSJQSjFvQjZfbqugaH+9z15HCF+QRBCC3lpfTIRFMTsvs8vPUUqF7IBxbzej6arzJ2QB8JqsKRBBRhJBiNFas2RdCScNSyPJ1r1SCnkOOwWlLlra3D6KzhwFpS6UgqEZvhsjAOiXYmPa4BQWrylCa9mnQAQPSQQh5ts9VeytajhqpdHjyXEk0tzmZosztLZg3FLmYmBqHDGRVp+/1rwJ2eysqGf1rv3HP1iIACGJIMQsWVdClNXC93P7dPu5BweMQ2ucwNiMxretgXZnje6LLcoqawpEUJFEEELcbs3S74qZMSKdxB7Mlx+UGoctyhpS4wSNLW3srKj3eo2hzsRFRzA7L5Ol60tobAnNNRki9EgiCCHf7KzEWdPE2WO73y0EYLEoRvVNDKkWwfZ9dbS5td8SARglJ1xNrbyXL2sKRHCQRBBClqwvISbSwmkjM3p8jjxHIhuLa3CHyKbs/pox1NG0QalkJcVK95AIGpIIQkRrm5t3NpRw2sg+xEX3vKhsrsNOXXMbO0Okvn6h00WkVTEw9cjd2XzFYlHMm5DFF1vLKa1u9NvrCtFTkghCxModlZTXNndrEdnR5GaF1mb2hU4Xg9LiiIrw76/6+ROycWt49VtpFYjAJ4kgRLy1rpi4KCsze9EtBDAsI4FIqwqZrSsLnC6/jg+0G5gWx+SBySxeLWsKROCTRBACWtrcvJtfyvdz+vR6rnxUhIURmQlsDIEWQV1TK3sqG7y+GU1XzZuQzbZ9dazdU2XK6wvRVZIIQsDnW8upqm/p0SKyo8ntaye/uCbor2S3lBkL43xZbO5YzhrTl+gIixSiEwFPEkEIWLKuhISYCE4a7p1tJnOzEqmsa6YkyAc6D8wYMqlFkBgTyey8TN5cWyxrCkRAk0QQ5Bpb2ng/v5QzcjOJjvBOCYUDm9kHefdQYamL6AgL/VJspsUwb0I2NY2tLN9UZloMQhyPJIIg92nhPlxNrT1eRHY0o/omoBRBv4dxgdPFsD7xWE3cUH760DQyE2NYtHqPaTEIcTySCILckvUlJNsiOXFIqtfOaYuKYEh6fPC3CEyaMdSR1aI4b0IWn24pp8wV3F1tInRJIghiDc1tLNvkZHZeXyKt3v2vNPYmCN4WQXV9C86aJtPGBzqaNyGbNrfmjW+LzQ5FiKOSRBDEPiooo765rcsb1HdHriORkupGKuuavX5ufygsMwaKzZox1NHQjHjG9UtikawpEAFKEkEQe2tdMWnx0Uwd7L1uoXZ5juAuSV1Q6kkEAdAiAKMQXYHTFfTdbSI0SSIIUrVNrXy4uYw5ozN9Mhia4zBKTWzYG5wfXIVOF/HRETjsMWaHAsDZYxxERVikEJ0ISJIIgtTyTU6aWt3M9eJsoY6SbFFkJ8cGdYtgeJ94lDJvxlBHdlsk38/pwxtr99LcGlpbgYrgJ4kgSL21roTMxBgm9k/22WsE62b2WmsKnS6/lp7uivkTstlf38KHm2VNgQgskgiCUHVDC58UljF3TF8sPpwjn+uws6O8jtqmVp+9hi+U1zazv74lYMYH2p00LI30hGgpOSECjiSCIPR+fiktbdpn3ULt8jwlqTeVBFeroL20RKAlggirhfPGZ/HR5jIqapvMDkeIAyQRBKEl60volxLL2Gy7T1+nvdREsK0wDrQZQx3Nm5BNq1vzxlpZUyAChySCIFNZ18wXW8uZM9rh84HQjIRo0uKjg26coNDpIiUuirT4KLNDOcKIzARGZ9ll9pAIKJIIgsy7G0ppdete70TWFUopch2JwdcicAbWjKHDzZuQxcaSmqDrchOhSxJBkFmyvpjBaXHkeub5+1peViJby2ppag2OMspaa7Y4awOitERnzhmXRaRVsVhaBSJASCIIIvtcTazYXsHcMX39drWb67DT6tYUltb65fV6q7i6kdqmVoYFcCJIiYvi1JEZvL52Ly1tsqZAmE8SQRB5Z0MJbo3PZwt11N7yCJY9jAs9A8WBtobgcPMn9qO8tplPC/eZHYoQvksESqlnlVJlSqkNnTyulFIPK6W2KqXWK6Um+CqWUPHWumKG94n362yY/ik2EmIigmaFcUH71NGMwE4Ep4xIJzUuSgaNRUDwZYvgeWD2MR4/Exjm+boWeMKHsQS9kuoGvtm5n7O9tC9xVymlyOmbGDQ1hwpLXWQmxmC3RZodyjFFWi2cOy6L5ZvK2B+kFV5F6PBZItBafwpUHuOQc4F/acMKIEkp5fupMEFq6foSwL/dQu3ysuxsLq2hNQj6swvLXAFReror5k3MornNzVvrZU2BMJeZYwRZQMf9+4o894mjWLK+hFxHIoPS4vz+2rmORBpb3Gwvr/P7a3dHm9uYMTQ8I97sULok12FnZGaCzB4SpjMzERxt2stRd+1QSl2rlFqllFq1b1/4Da7tqaxn7Z4q5vq5W6hdbpDsTbC7sp6mVnfQtAjA2KdgXVE1WzxjG0KYwcxEUAT063A7GzhqG1lr/ZTWepLWelJ6erpfggskS9q7hfywiOxohqTHER1hIT/AxwnaS0sE8hqCw507LgurRbFICtEJE5mZCN4ELvPMHpoGVGutS0yMJ2AtWV/MuH5J9EuxmfL6EVYLI/smBvwU0vZic8P6BEfXEEB6QjQzR6Tz2pq9QTEGI0KTL6ePLgS+AkYopYqUUj9WSl2vlLrec8jbwHZgK/A0cIOvYglmO8rryC+uMa010C7PszdBIO+5W+h00T/Fhi0qwuxQumXehGzKXE18vrXc7FBEmPLZX4zW+uLjPK6BG331+qFiyTqjt2yOyYkg12HnxZW72VPZQP9Uc1omx1PoqTEUbE4dlUGSLZLFa/ZyyogMs8MRYUhWFge4t9YXM2VgCn3tsabG0b7COFAHjJtb3WzfVxeQpaePJzrCyjljHbyXX0p1Q4vZ4YgwJIkggBU6XRQ6a5k71vzlFSMyE7BaVMCWpN5RXkerWwd8aYnOzJ+YTXOr+8B6ESH8SRJBAFuyrhiLgjPzzE8EMZFWhmXEB+yAcUGA7krWVaOz7AzLiGfR6j3HP1gILwuuUbUworVmyfoSpg1OJT0h2uxwAGOc4NMtgbmOY4vThdWiGJzu/wV33qCUYt7EbO57ZzNX/3MVA1Jt9E8xvvql2MhOjiUm0mp2mCJESSIIUPnFNWwvr+PqkwabHcoBuY5EFq8poqymkYzEGLPDOURBqYuBqTaiI4L3w/LiKf3ZXFLDxpIaPt+6j8aWQ6eTZibGHEgM/VNs9E+N9SSLONLiowJ2Ix4R+CQRBKgl60uIsChm52WaHcoBBweMawIuERQ6XeT4abMeX7HHRvLgReMBo0W4r7aJPZX17K6sZ3dFA7sr69lTWc8XW8tZXNN4yHNjI62HJomUWPp7WhXZyTZpTYhj6lIiUEr9DHgOcAHPAOOBO7XW7/swtrBldAsVM31oGilxgbPvbk6HmUMzRwbONMeG5jZ2Vdbzg/GhU6pKKUVGQgwZCTFMHJByxOONLW0U7W9gT2U9uyrq2F15aKJoaDl0R7k+idGHJQobA1KN2+nx0dKaCHNdbRFcpbV+SCl1BpAOXImRGCQR+MC6omqK9jfws9OGmR3KIRJiIhmYajt6SWp3GzRWe76qoKHq4PeN1YfePuSxGhg4Hc74IyT2rJbS1rJatA6u0hK9FRNpZWhGPEOPUmBPa015bfOBxLC7w9dX2yp4dc3ew85lOWQ8on+KjdFZdiYNPDIBidDU1UTQfrlwFvCc1nqdkksIn3lrXTFRVguzcv3cLdTSeNwP8T9Zt9K8cz88H+U51nNc03GmlSorxCZBjB1iPP8m9QNrNGx8HbYsg9PuhslXg6V73RgHS0uETyI4FqUU6QnRpCdEM3FA8hGPN7a0sbeqgd0VhyaJPZX1fLmtgvpmozXxx/NGc8nU/v4OX5igq4lgtVLqfWAQ8EulVAIghVF8wO3WLF1fwsnD07HH+mBzFa2heA2s/Q+UrD/0w76t6djPjYxjtIqjqCWa1tYsIhKzISPX8wGfZHy4H/5h3347Kh46u3Y45Q5Yegu8czusWwhnPwR9x3b5Ryp0uoiyWhgYoCueA01MpJUh6fEMSe+8NXH7onXc9fp3pMRFBdQ4lfCNriaCHwPjgO1a63qlVApG95DwstW791Na08gvzxrp3RO7nLD+ZSMB7NsEETGQPRkyRhkf2kf9AE86eDvGDhFRrCncx2XPfs1/Zk7lxCFp3oktZTAseBU2LIZ3fwlPnQJT/wdm/gqij18yosDpYkhGPBFWWRbTW+2ticcuncClz6zkpy99y7+umsK0walmhyZ8qKuJ4ARgrda6Tim1AJgAPOS7sMLXknXFREdYOG1Un96frLUZtrwH374IW94H3QbZU2Dug5B3vvHh3k0HZg7trfFeIgCjtTB6Pgw9DZbdCyseg41vwFn3w8g5x3xqYamLKYOkP9ubbFERPHv5ZH7496+45p+rePm6E4J+VpboXFcvoZ4A6pVSY4HbgV3Av3wWVZhqc2uWflfKqSMziI/uxcze0u/gnTvhryPh5QVQ/C2ceBPc+A1c/QFMurJHSQAgNT6azMQY39Ucik2Gsx+Eq96HmER46RJ46VKoPnq9/prGFoqrG4NqM5pgkRwXxb+umkJ8TASXP/c1eyrrzQ5J+EhXE0Grp1roucBDWuuHAPnL87KV2ysor23i7J7sS1xfCSv/Dk+eBE9+D1b9AwaeBJf8F36RD9+/F9KHeyXOvKxE39cc6j8VrvsUTv8tbF0Oj02FFU8Ys5M62OKsBWB4hvw6+oIjKZZ/XTWF5lY3P/rHSsprjzOOJIJSVxOBSyn1S+BHwFKllBXwwUhmeHtrfQm2KCszu1qKuK0VCt+Dl38EDww3BluVgjP/ArcUwAX/hOGzwOrddYM5Djvb9tXS0Nx2/IN7wxoJ3/sF3PAV9J8G794JT880Wjge7TOGgrXYXDAY1ieBZ6+YTGlNI1c+9w21Ta1mhyS8rKuJ4EKgCWM9QSnGJvN/8VlUYailzc27G0o4fVQfYqOOM31yXwF8cA/8LRf+cwHs+hKmXAvXf2FcRU+9Fmy+6zPPcyTi1rCp1E+VSFMGwaWLYP5z4CqFp081ur6aXBSUurBFWclKMrdMd6ibOCCZxy+dwMaSGq7/92qaW2XSYCjp0qWi1rpUKfUiMFkpNRf4WmstYwRe9OW2CvbXt3S+E1ljtTGr5tsXYe8qY17+8DNg3KUwbBZE+G8Fcm6WZzP7vdVM6H/kPHWfUMoY4B5yKnz4O1j5JGx8g8TIaxjWZyoWiyxr8bVTR/bhz/PGcOt/13HLf9fx0IXj5H0PEV0tMXEBRgvgY4zFZY8opW7TWi/yYWxh5a11xSTERDBjRPrBO91tsOMTY8rnpregtREycmDWH2DMBRBvTpkHhz2GJFukOXsTxCbBnP+DMRfBkp9zs/NezkiYDlV/NxaoCZ+aPzGb8tom7ntnM6lxUfzm7BwpTxECutp5/Gtgsta6DEAplQ4sAyQReEFTaxvv5ZcyKyfTqJ5Zsc1YWLV2IdQUGTN8xi8wrv4d4ztfmOUnSinyHHZzN6npN5mKS97j7/ffyu11rxmDyTN/BVOv9/qYiDjUdScPptzVxDOf7yA9IZobZw41OyTRS139i7G0JwGPCmRTG6/5rLCctsZark74Ap69E3Z/CcpidIPM+h2MOAsiA6vaZ64jkee+2ElLm5tIkxZyFZY38VTb2Zw29zqmbvojvP9rWP+SsTI5a6IpMYUDpRS/OmsUFXXN/OW9AlLjorhoipSiCGZdTQTvKqXeAxZ6bl8IvO2bkMKI1rDrCxLee4RVMR9hW9kEqUPhtHtg7MU9LsLmD7lZdprb3Gxx1pq20Kh9xtDAoaNg/CvGArR37oCnTzMGz0+9y1iLILzOYlHcP38MlXXN/Oo1oxSF32tjCa/p0qWc1vo24ClgDDAWeEprfYcvAwtpVXvgk/vh4XHw/Bxyqz5hQ/L3jUVUP1kFJ90S0EkADq4wNnPrygKnC3tsJBkJ0UZ3We4P4CdfG4Xrvn4KHptiJAetTYsxlEVaLTx+6QRGZydx08Jv+XpHpdkhiR7qcptea71Ya32z1voXWuvXfBlUSPvsr/DgaPjoD5DUn3WT/szkpsdomfOQsYgqSAbeBqXGYYuystHEcYLCUhcj+iQcOlgZY4c5D8DVy8GWBq9cBgsvgqrdpsUZyuKiI3juislkJcfy439+w2Z/TSkWXnXMRKCUcimlao7y5VJKyf94d218E5bfCznnwM/Ww+Vv8VT1FOLiE5kaZLVyLBZFTt9E35WaOA6tNYVOF8P6dFKULnsiXPuxMcNqx6fGYPKXjxiL8IRXpXhKUcRFRXDZP6QURTA6ZiLQWidorROP8pWgtZbO1+4o/Q5eu86o+HneU5A8gLqmVpZvdnJmXt+grJyZ60hkY3ENbrf/u16cNU3UNLYee0WxNQJO/AncuBIGnQzv32VUNi1a5bc4w0V2so1/XjWFxpY2Ln/2ayqkFEVQCb5Pn2BUVw4LLzFKOl/4woEZQMs2OWlscXe+iCzA5WbZqWtuY2dFnd9fu8AzUDy8K5vRJPWHi1+CC/4N9RXwzOmw9FZjkZ7wmhGZCfzjisnsrWrgque/oU5KUQQNSQS+1tps9FPXlcFFL0LCwZkVS9aX0CcxmslBuiXgwQFj//cSFpZ2IxGAMfaSc47ROph6HXzzDDw6BfJfk8FkL5o8MIXHLpnAhuIarn9BSlEEC0kEvqQ1vHMb7PoCzn0MsiYceKimsYVPCvYxZ7QjaJfpD8tIIMpqMWWcoMDpIj0hmpS4bpbWiEmEM/8M1yw3Vmb/9wqjXtP+XT6JMxydntOHP50/ms+2lHPrf9eZ0nUoukcSgS998wysfh6+d7Ox6UoHH+Q7aW5zM3dscHYLAURFWBieGU/+0Taz97EtThfDOxso7oqsiXDNR3DGH2HnF8Zg8sf3QVOt94IMYxdM6sfts0fw5rpifrd0I1paXQFNEoGvbP/EWNw0/Ew49e4jHn5rfTFZSbGM75fk/9i8KLevnfziar/+obvdmkJnbde7hTpjjYATbjS6i4bPgo//BA+PNxJ4W4t3gg1j/zNjCFdNH8RzX+zkiU+2mR2OOAZJBL5QuQP+ezmkDYPznwLLoW/z/rpmPt9SztyxfYO+YFdeViL761soqW7022sW7W+goaWNEb1NBO2S+sEF/4IfLzNWdi+9BR6fJovRekkpxV1zRnHuOAf3v1vAK6v2mB2S6IQkAm9rcsHCi40PkIsXHrXEwXv5pbS6NWePCezVw12R4zBKUm/Y679xggMzhry9GU2/yXDl28YMI0uEMcj/zOlG15HoEYtF8Zf5YzlpWBq/fPU7lm10mh2SOApJBN7kdsOr10J5obE7WMrgox62ZH0JA1NtB2bdBLNRfROwKPxaibS9xtCwjF6MEXRGKRhxprHJzzmPQk0xPH8W/OdCKNvk/dcLA1ERFp5cMJE8RyI3/mcNq3ZKKYpAI4nAmz76AxS8DbPvg8GnHPWQ8tomvtxWztwxjqDvFgKwRUUwOD3erzOHCp0uspJiSYjx4W6p1giY8CO4aTWc9hvY9RU8cSK8cSNU7/Xd64aouOgInr1iMllJsVz1/DcUeKb/isAgicBbNiyGzx6ACZfDlGsAaGxpo9Dp4v38Up7+dDu/fu07rnr+G9yanm1QH6ByHX7YzL6DgtJezhjqjigbnHQz/GwtTLsB1r8Cj0yAZb+Fhir/xBAiUuOj+edVU4iNsnLZsysp2i+lKAKF7ODRS67GFpybVzDorRtw2sfxaOPlbHtqBbsq6imtOXQANckWyYAUG9eePDikNlvPc9h5Y20xFbVNpMZH+/S1WtrcbN9Xd+hObv5gS4Ez/mCUt/7oD/D5g8bU4JNuNRJ/hG9/7lDRL8UoRfHDJ7/isme/ZtH1J3Z/LYjwOkkEXVBV38zOinp2VdSxs7yeXZV17PLcVrVlvBF9FyXEc67zOlTdfgam2pg+NI0BqTYGpNoYmBrHgFQbSbbQ/IVvH+vIL67h5OG+/YDeVVFHc5vbezOGuit5gDET7ISfwLLfGJvhrPy7sffB6B8eMUNMHGlkZiL/uHwyP/rHSq58/hsWXjMVW5R8FJnJp+++Umo28BBgBZ7RWt932ON24AWgvyeWB7TWz/kypqPRWrOvtondFfUHP/A9/+6qqKe64dA55Q57DP1TbZwxIpmb9vwvaXUN7Dr3VT4ZMYX46PD7hc71zBzyRyIoKDUWfPV6DUFv9R0DP3oNtn0EH9wDr10LXz0Cp98LQ08zN7YgMGVQCo9cPJ7rX1jN/7ywhmcun2TaTnfCh4lAKWUFHgO+DxQB3yil3tRab+xw2I3ARq312Z59kAuUUi9qrZu9HY/brSmtaWRnRfvV/KEf+PXNbQeOtSijmuKAVBvnjHV4ruzjGJhqo1+KjZhIqzE99I0boWY9XPAvhuSc6O2Qg4bdFkl2cqxfNqkpdLqwKBjqixlDPTFkJgz6BPJfheX/Cy+cb0wUOP1ecIwzO7qANis3kz+eN5o7X/2O2xet5/9+ODZoy60EO19evk4BtmqttwMopV4CzgU6JgINJChj+kw8UAn4pGTh62v3cvMr6w7cjrJa6JcSy4DUOKYNTjnQfTMgNY6spFiiIo5zdbLiCVj7Isy4E3LO9UXIQaW9JLWvFTpdDEiNM5JxoLBYjBIio86GVc8au889NQPy5sNpd0PyQLMjDFgXTel/yN7Hv54zKiRm0wUbXyaCLKDjUsIiYOphxzwKvAkUAwnAhVrrI8oVKqWuBa4F6N+/Z5tkTx6Ywh/Oyzvwgd/XHou1p1cfW5cbfcOjzoYZsmMnGAPG7+U7cTW2+HRaZ0Fvawz5UkQ0TPsfGHcJfPEQfPW4sTp58tVw8m0Ql2p2hAHphlOGsM/VxDOf7yA9IZrrZgwxO6Sw48tOuaN9yh6+Xv8MYC3gAMYBjyqljlhlpbV+Sms9SWs9KT29Z33Q/VJsXDp1ANOHppGdbOt5EijfCouuhIwc+MGTMjjokZtl/LdtKvHd/PDGljZ2lteZN1DcVTF2OO0e+OkaGHcxfP13Y3/qTx+AZpkyeTilFPfMzeHssQ7+9M5mFq0uMjuksOPLT7EioF+H29kYV/4dXQm8qg1bgR3ASB/G1DuN1cb+t5YIuOg/EB2gV6YmyDswYOy7cYJt+2pxax+UlvCVRAec8wjcsAIGngQf/s4oarf6edky8zAWi+L/fjiW7w1N447F6/lws5Si8CdfJoJvgGFKqUFKqSjgIoxuoI52A6cBKKX6ACOA7T6MqefcbbDox7B/h7HTVfIAsyMKKBmJMaTFR7PBhyWptzgDZMZQd6WPgIv/A1e+a+yW9tbPjFXKm5dKUbsOoiIsPPmjieQ6ErnhxTWs3iWlKPzFZ4lAa90K/AR4D9gEvKK1zldKXa+Uut5z2O+AE5VS3wHLgTu01uW+iqlXlv0Wtn4AZz0AA6ebHU1Aysvy7Wb2BU4XkVbFwNQ4n72GTw04AX78vrFdqXbDS5fAs7Nh90qzIwsY8Z5SFH3tsVz1/Cq2OKUUhT/4tINba/221nq41nqI1voPnvue1Fo/6fm+WGs9S2s9Wmudp7V+wZfx9Ni6l+DLh2HyNTDpSrOjCVi5jkS2lNXS2NJ2/IN7oLDUxeC0+OPP6ApkShmTDG5YAXMfNFqYz86Cly6FfYVmRxcQ0uKj+ddVU4iKsHDx0yulSJ0fBPFflJ8UrYI3f2r08c7+k9nRBLRch502tz5QHdTbCpyu4BkfOB5rhHFR8dNvYeZdxkZGj08zftdK1oV9l1G/FBsLr5lKfLSVi59ewQsrdskuZz4kieBYaoqNK7WETGPjEqsPq12GgLwOK4y9ra6plaL9DYwI1KmjPRUVBzNuM4raTbkG1v4H/n4yPDbF2DqzfKvZEZpmaEYCb9z4PaYPTeOu1zfwy1e/o6nVN63NcCeJoDMtDUYSaK41NiqxpZgdUcDrlxJLQkyETzap2VJmDBQPC7aB4q6KS4Mz/wy3FMDcv0FchpEIHp0If58BXz4SluWv7bZI/nH5ZG6cOYSXvtnDhX9fQakfd8MLF5IIjkZro4levMYoMNYnx+yIgoJSymclqQs99esDfg1Bb8WlwqSr4Mql8It8mPUHY1zh/bvgb7nw3FnwzT+grsLsSP3GalHcdsZInrh0AoVOF2c/+rmMG3iZJIKj+eIh+O4Vo6LkyDlmRxNUch12NpXU0Np2xALxXilwuoiJtNAvxebV8wY0exac+BO49mO4aQ3M/BXUlcPSm+GBYfDCPGMiQ1N4zKw5c3RfXr9xOnFRMm7gbZIIDlf4njFVNPd8o9a86JZcRyJNrW62l9d59byFThfDMhJ6viI82KUOgRm3w40r4frP4cSbjFlGr10Hfxlq7K+88U1oCe1uk+F9EnjjJ9/jezJu4FXhVzP5WPYVGIvGMkfDuY8ZTXLRLXlZB1cYe3PhV0Gpi+8NS/Pa+YKWUsbvZ+ZoOP23sOdr2LAI8l8z6hpFJ8LIuTB6Hgw6xZidFGLssZE8c/lk/vZBIY9+tJXNpS6eXDCRTHuM2aEFLWkRtKuvNMpHRMbAxQuNLQpFtw1OiyM6wuLVFcZV9c2UuZpCf3ygu5SC/lPhrL/AzZuN/RFGnWOsWH5hHvzfcFh6i7Hfstu7XXVms1oUt54xgicXyLiBN0giAKPuy6IroWoPXPgi2LPNjihoRVgtjOrr3RXGhe2lJUJlDYEvWCNgyKnwg8fgti3G7/Ggk+HbF+G52fDgaHj/7pBbozA7T8YNvEESARgzMrZ/bEzb6394pWzRXe0zh7z1B1ngDJMZQ94SEQ2j5sIPn4fbtsL5T0OfXFjxuLFG4dHJIbVG4fBxgzsXy7hBd0kiWPMvWPkETLsBJvzI7GhCQq7DjquxlT2VDV45X2Gpi4ToCPpKH3D3RcfDmAvg0lfg1i1GWYuEzA5rFE6GLx6G6uAu/dw+bvCTmUN5eZWsN+iu8E4Eu1fAkpth8Ez4/u/MjiZk5GW1b2bvne6hQqeLYX3iZeeq3rKlGGUtrlgCN2+EM/5olFT/4G5jjcKzZ8I3zxhTVIPQ4eMGcx/5nG9k3KBLwjcRVO2BlxdAUj/44XMhObvCLMP7GNM8vbGHsdZG7aIRMj7gXYkOOOFGuOZDo97RqXdBQ6UxuPzAcPjnOfDBPbDuZSjdAK1e30bcZ9rHDeKjrVz81Ar+LeMGxxWen37NdfDSxdDaBFcshdhksyMKKTGRVoZlxHtlhfG+2ib217cE3x4EwSRlsLGV5km3QtlG+G4RbPnA2Je7zZMALBGQNtzYma9PDvTJM763ZwfkNOv2cYOfv/Qtd7++gQ1F1fzvD3KJjgigva4DSPglAq3h9RuMq5xLXjE2DRFel+uw80nhvl6fp7DUmDEkA8V+oJQxqNwnF07/DbS1QMVWcOYbCcKZf3DdQrtou5EYMnIOPjdjlLFdp8nssUador8tK+SRD7dS4JT1Bp0Jv0Tw2QOw8XX4/v/C8FlmRxOych2JLF5TRFlNIxmJPf/Da58xJFNHTWCNND7UM0Yden9jNZRtMhJDe5L4bhGs+sfBY+z9PS2H3INJInWo3yv4WiyKW2aNINeRyC2vrGPuI5/zxIIJTB4oRSQ7Cq9EsHkpfPh7GHMhnPhTs6MJae0rjDcUV3NqLxLBFqeLlLgo0uKjvRWa6K0YO/SfZny109qYeVS2EZwbwLnR+H7rMnB79me2RkHaCE/LIQcyPC2IhEyfdy/NzuvLkPR4rv33ai5+agW/OSeXBVP7ywQEj/BJBM58ePVayJoIZz8ckP2aoWRUX+MKPn9vDaeO7NPj8xQ4XQwPtT0IQpFSxsSLpH4w/IyD97c2Q3nhoQlix6ew/qWDx8QmH0wK7eMP6SONqa9eNKxPAq/fOF3GDY4ifBJBQxUkDTBWXEZKH6GvJcREMigtrlcDxlprCktdzJ8oK72DVkQUZOYZX1xw8P76yoPdS2WeLqa1Lxr7f7RLHgjJgyDSZvzNRsQa/0bGHvz+kH89XxExHf499Ln26NigGzdobGljw95q1uzeT57DzolDvV9zK3wSwcDpRtVGS/jOmPW3HEci6/ZU9fj5e6saqGtuk/GBUGRLMf4mB04/eJ/bDdW7PWMPG40EUbUH6vYZG0W1Nnb4t77HL22xRnNLZAw3JkVRVmah7m9R1KUmExefcFgS8SSSiBhj1hQAHaahHjIltbv3c5T7NVpraptaKatpZJ+riTJXE/vrmnBrTSSwN+c0GHpdj3/2zoRPIgBJAn6W57CzdH0J1fUt2G3dHyQslNIS4cVi8bQCBh5/HxCtjenfrQ1G6e3WBiNJHPi+432HJxHjvpjWBpJcLtZuL2XPvgZGupvoY2tEuUoPnqOl3niOu0PJikO6lVWv7tdo3G5o09Dm1rRpoyWcDqSjGG1RWKMUVosFq0VhcUzo6rvZLeGVCIRf5ToOrjDuSXO2vdhcyG5PKXpOKc8VewzE9vw0icDYhhZ+/tK3fFSwjwsmZfO/5+YRE+n9cQOtNbsq6lmzez/f7q5ize79bC510eY2WgSD0uIY3z+J8f2TmdA/iRF9Eoiw+ufiVRKB8JmDiaCmZ4mg1EVmYgz2WP9OORThpX29wYPLCnn4w60UOGv5uxfGDWqbWlm/p+rAB/+3e6qorDMW6MVHRzC2n50bThnC+P5JjOuXTEpclDd+nB6RRCB8JjU+mr72mB7XHCpwumR8QPiFxaK4edYIchx2bnllbbfXG7jdmu3ldXy7ez9rdlfx7e79FDpdeC72GZoRz2kjM5gwIJnx/ZMCbrc9SQTCp3IdiWzowcyhNrdmS1ktJw5J9UFUQhzd7LxMhqRPP7je4OwcFkwbcMR6g+qGFtZ1uNpfu6eK6oYWABJjIhjXP5nZeZmM75/MuH5JAd+qlUQgfCrXYefDzWXUN7dii+r6r9uuijqaW91SY0j4Xft6g1+8vJa738jnu73VXDl90CEf/Fv31aK1MVQxok8CZ43OPNC3PzgtHksAXe13hSQC4VO5jkTcGjaVuJg4oOvF/Q7sSiaJQJjAHhvJM5dNOjBu8MoqY7+GZFsk4/snc85YBxMGJDMm205CTGBf7XeFJALhU7meUhMbi6u7mQiMqaPDZFWxMEn7uMH0oWnsrWpgfP9kBqbaQrIshSQC4VMOewzJtshurzAucLron2LrVneSEL4wdXDoj1PJCivhU0opch32bm9SU1jqkm4hIfxEEoHwudysRApLa2ludXfp+OZWNzvK6xiRKd1CQviDJALhc7kOO81tbraUubp0/I7yOlrdWloEQviJJALhcx1XGHfFgc1oJBEI4ReSCITPDUqNIy7KysYuJoLCUhdWi2JwepyPIxNCgCQC4QcWi2JU30Q27O3agHGB08WgtDjZMEQIP5FEIPwiL8vOxpIa3O5OarJ3UOh0SelpIfxIEoHwixxHIvXNbeyoqDvmcQ3NbeyurJeFZEL4kU8TgVJqtlKqQCm1VSl1ZyfHnKKUWquUyldKfeLLeIR58hzGCuPjDRhvLTNquEiLQAj/8VkiUEpZgceAM4Ec4GKlVM5hxyQBjwPnaK1zgR/6Kh5hrmF94omyWsg/zjjBgRlDUn5aCL/xZYtgCrBVa71da90MvASce9gxlwCvaq13A2ity3wYjzBRpNXC8Mz447YICp0uoiIsDEix+SkyIYQvE0EWsKfD7SLPfR0NB5KVUh8rpVYrpS472omUUtcqpVYppVbt27fPR+EKX8tz2MkvrkZ3tok3UFDqYmh6vN+26BNC+DYRHK1E3+GfABHARGAOcAZwt1Jq+BFP0voprfUkrfWk9PR070cq/CLXkcj++haKqxs7PWaL08VwGSgWwq98mQiKgH4dbmcDxUc55l2tdZ3Wuhz4FBjrw5iEidpLUnc2TlDTaCQJGR8Qwr98mQi+AYYppQYppaKAi4A3DzvmDeAkpVSEUsoGTAU2+TAmYaJRmYlYFJ1uXbnFM1AsM4aE8C+fFXvXWrcqpX4CvAdYgWe11vlKqes9jz+ptd6klHoXWA+4gWe01ht8FZMwV2yUlcHp8WzspCR1QansSiaEGXy664fW+m3g7cPue/Kw238B/uLLOETgyHMksnJH5VEfK3S6iIuykpUU6+eohAhvMjVD+FWuw05JdSMVtU1HPFbodDG0T0LQbfwtRLCTRCD8Kjer85LURo0hmTEkhL9JIhB+ldvXmDl0+NaV5bVNlNc2y/iAECaQRCD8ym6LJDs59ogWQWH7jCGZOiqE30kiEH6X57AfsUlNYansSiaEWSQRCL/LdSSyo7wOV2PLgfsKy2qxx0aSkRBtYmRChCdJBMLv8jwrjDeVHNzMvrDU2IxGKZkxJIS/SSIQfte+mX371pVaawqcLoZnyowhIcwgiUD4XUZiDGnx0QcGjEtrGnE1tkppCSFMIolAmCIvK5F8zxTSAs9A8TBJBEKYQhKBMEWuI5EtZbU0trSxxSk1hoQwkyQCYYo8h502t6bQ6aLA6SI9IZqUuCizwxIiLEkiEKbI9Wxmv2Fvjae0hLQGhDCLJAJhin4psSTERPDd3moKnS7pFhLCRJIIhCmUUuQ6Evlgo5PGFrdsTymEiSQRCNPkOuyUe8pRy/aUQphHEoEwTZ6nJDXAsAxpEQhhFkkEwjTtA8ZZSbEkxESaHI0Q4UsSgTDN4LQ4YiItUnpaCJP5dM9iIY4lwmrh7rk5DE6TbiEhzCSJQJjq0qkDzA5BiLAnXUNCCBHmJBEIIUSYk0QghBBhThKBEEKEOUkEQggR5iQRCCFEmJNEIIQQYU4SgRBChDmltTY7hm5RSu0DdvXw6WlAuRfDCXbyfhxK3o+D5L04VCi8HwO01ulHeyDoEkFvKKVWaa0nmR1HoJD341Dyfhwk78WhQv39kK4hIYQIc5IIhBAizIVbInjK7AACjLwfh5L34yB5Lw4V0u9HWI0RCCGEOFK4tQiEEEIcRhKBEEKEubBJBEqp2UqpAqXUVqXUnWbHYyalVD+l1EdKqU1KqXyl1M/MjslsSimrUupbpdQSs2Mxm1IqSSm1SCm12fM7coLZMZlFKfULz9/IBqXUQqVUjNkx+UJYJAKllBV4DDgTyAEuVkrlmBuVqVqBW7TWo4BpwI1h/n4A/AzYZHYQAeIh4F2t9UhgLGH6viilsoCfApO01nmAFbjI3Kh8IywSATAF2Kq13q61bgZeAs41OSbTaK1LtNZrPN+7MP7Qs8yNyjxKqWxgDvCM2bGYTSmVCJwM/ANAa92sta4yNShzRQCxSqkIwAYUmxyPT4RLIsgC9nS4XUQYf/B1pJQaCIwHVpocipkeBG4H3CbHEQgGA/uA5zxdZc8opeLMDsoMWuu9wAPAbqAEqNZav29uVL4RLolAHeW+sJ83q5SKBxYDP9da15gdjxmUUnOBMq31arNjCRARwATgCa31eKAOCMsxNaVUMkbPwSDAAcQppRaYG5VvhEsiKAL6dbidTYg28bpKKRWJkQRe1Fq/anY8JpoOnKOU2onRZXiqUuoFc0MyVRFQpLVubyEuwkgM4eh0YIfWep/WugV4FTjR5Jh8IlwSwTfAMKXUIKVUFMaAz5smx2QapZTC6APepLX+q9nxmElr/UutdbbWeiDG78WHWuuQvOrrCq11KbBHKTXCc9dpwEYTQzLTbmCaUsrm+Zs5jRAdOI8wOwB/0Fq3KqV+AryHMfL/rNY63+SwzDQd+BHwnVJqree+X2mt3zYvJBFAbgJe9Fw0bQeuNDkeU2itVyqlFgFrMGbafUuIlpqQEhNCCBHmwqVrSAghRCckEQghRJiTRCCEEGFOEoEQQoQ5SQRCCBHmJBEI0QNKqZ8rpWxmxyGEN8j0USF6wLMSeZLWutzsWITorbBYUCZEb3iKrr2CUZrECvwXo/bMR0qpcq31TKXULOBeIBrYBlypta71JIyXgZme012itd7q759BiGORriEhjm82UKy1HuupS/8gRq2qmZ4kkAbcBZyutZ4ArAJu7vD8Gq31FOBRz3OFCCiSCIQ4vu+A05VSf1ZKnaS1rj7s8WkYGx594SnZcTkwoMPjCzv8G7a7fYnAJV1DQhyH1rpQKTUROAv4k1Lq8Jr0CvhAa31xZ6fo5HshAoK0CIQ4DqWUA6jXWr+AsVHJBMAFJHgOWQFMV0oN9RxvU0oN73CKCzv8+5V/ohai66RFIMTxjQb+opRyAy3A/2B08byjlCrxjBNcASxUSkV7nnMXUOj5PloptRLjwquzVoMQppHpo0L4kEwzFcFAuoaEECLMSYtACCHCnLQIhBAizEkiEEKIMCeJQAghwpwkAiGECHOSCIQQIsz9P1HtR7gM5BzoAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "sns.lineplot(data=df_train, x=\"step\", y=\"loss\", label=\"train loss\")\n",
    "sns.lineplot(data=df_eval, x=\"step\", y=\"loss\", label=\"eval loss\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "d5dfc084-6172-4747-8de0-f1470c5930b3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZsAAAEGCAYAAACzYDhlAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAmhUlEQVR4nO3de3RV9Z338fc3h1wI4ZIbFEELU+lUvGOWYqsuL1MB6xJvFOzMiNa1GFt9xk6nzzM4T12rfdr1jH1s57HMeFl2dIodnyZRq6StLVhaax1FDBUvoAgqI1HrPgkQEiIhl+/zx9lojCE5wNnZ5/J5rXXWOWfv32/v7z7i+WT/zr6YuyMiIhKlorgLEBGR/KewERGRyClsREQkcgobERGJnMJGREQiNybuArJVTU2Nz5gxI+4yRERyyoYNG1rdvXbwdIXNQcyYMYPm5ua4yxARySlm9l9DTdcwmoiIRE5hIyIikVPYiIhI5BQ2IiISOYWNiIhELtKwMbP5ZrbFzLaZ2fIh5puZrQjnv2hmc0bqa2ZVZva4mW0NnyvD6dVm9jsz6zSzfx20ntPM7KVwWSvMzKLcbhER+ajIwsbMEsAdwAJgNnCVmc0e1GwBMCt8LAPuSqPvcmCtu88C1obvAfYBtwDfGKKcu8LlH1jX/AxsooiIpCnK82xOB7a5+xsAZlYPLAQ2D2izELjfU/c5WGdmk8xsKjBjmL4LgXPD/iuBJ4B/cPe9wFNmduzAIsLlTXD3Z8L39wOXAr/K8PbKIP09/bT8sIW+PX1xlyIih+CTt3ySouLM7otEGTbTgB0D3rcAZ6TRZtoIfae4+7sA7v6umU1Oo46WIdbxMWa2jNQeEMccc8wIi5WRtK5q5Y3//kbqjQYuRXLGMf94DBRndplRhs1QXy+D79R2sDbp9M1kHamJ7vcA9wDU1dXprnJHKKgPKJ5SzGff/iyWUNqIFLIoDxBoAY4e8H468E6abYbr+144NHZgiCxIo47pI9QhGdbb0cvOX+5k8qLJChoRiTRsngNmmdlMMysBlgBNg9o0AVeHR6XNBdrDIbLh+jYBS8PXS4FVwxURLq/DzOaGR6FdPVIfOXJtTW307+tn8pKRRjlFpBBENozm7r1mdiOwGkgA97n7JjO7Ppx/N/AYcBGwDegCrh2ub7joW4FGM7sOeAtYdGCdZrYdmACUmNmlwIXuvhn4CvBjYCypAwN0cEDEgvqA0qNLmXDmhLhLEZEsYKkDwWSwuro611WfD0/Prh6envI00/52Gsd+/9iRO4hI3jCzDe5eN3i6riAgGdf6SCve4xpCE5EPKGwk44L6gLI/K2P8aePjLkVEsoTCRjJqf7CfXb/dxeQlk9FVgUTkAIWNZFTy4ST0oSE0EfkIhY1kVNAQUH5cOeNOGBd3KSKSRRQ2kjHdb3fT/mS7htBE5GMUNpIxwYMBOExerCE0EfkohY1kTLIhScUpFZT/eXncpYhIllHYSEa8v/199qzbQ+3i2rhLEZEspLCRjEg2JgENoYnI0BQ2khFBfcD4M8YzdubYuEsRkSyksJEj1vVaF53Pd2qvRkQOSmEjRyxoCMBg8hcVNiIyNIWNHLGgPmDiWRMpnVYadykikqUUNnJEOl/upGtzly5PIyLDUtjIEQnqAyiC2it1yLOIHJzCRg6bu5NsSFJ5fiUlk0viLkdEspjCRg5b5x87eX/b+xpCE5ERKWzksAX1ATbGqLmsJu5SRCTLKWzksHi/EzQGVM6rpLiqOO5yRCTLKWzksOxZt4fut7o1hCYiaYk0bMxsvpltMbNtZrZ8iPlmZivC+S+a2ZyR+ppZlZk9bmZbw+fKAfNuDttvMbN5A6YvDpe/ycz+T5TbXCiChgArNWou0RCaiIwssrAxswRwB7AAmA1cZWazBzVbAMwKH8uAu9LouxxY6+6zgLXhe8L5S4DjgfnAnWaWMLNq4DbgAnc/HphiZhdEs9WFwfucZGOS6i9UM2bCmLjLEZEcEOWezenANnd/w933A/XAwkFtFgL3e8o6YJKZTR2h70JgZfh6JXDpgOn17t7t7m8C28Ll/Bnwmrsnw3a/Aa7I8LYWlN1P7mb/n/brWmgikrYow2YasGPA+5ZwWjpthus7xd3fBQifD3zjHazPNuAzZjbDzMaQCqejhyrYzJaZWbOZNSeTyaGaCKkhtKJxRVR/oTruUkQkR0QZNkPdhN7TbJNO37TW5+67gK8ADcAfgO1A71ALcPd73L3O3etqa3VG/FD6e/pJPpSk5pIaEuMScZcjIjkiygH3Fj66BzEdeCfNNiXD9H3PzKa6+7vhkFsw0vrc/efAzyG19wL0HeY2Fbzdv91Nb1uvhtBE5JBEuWfzHDDLzGaaWQmpH++bBrVpAq4Oj0qbC7SHQ2PD9W0CloavlwKrBkxfYmalZjaT1EEH6wHMbHL4XAl8Ffi3zG9uYQjqAxITE1TNr4q7FBHJIZHt2bh7r5ndCKwGEsB97r7JzK4P598NPAZcROp3lS7g2uH6hou+FWg0s+uAt4BFYZ9NZtYIbCY1THaDux/Yg/mhmZ0cvv5f7v5aVNudz/q7+0k+kqTm0hqKSnWKloikz9xH+imkMNXV1Xlzc3PcZWSV1qZWXl74Mif+6kSq5+vgABH5ODPb4O51g6frz1NJW1AfMKZ6DJUXVI7cWERkAIWNpKWvq4/WplZqr6ilqFj/bETk0OhbQ9LS9ss2+vf261poInJYFDaSlqAhoOQTJUw6Z1LcpYhIDlLYyIh69/Sy85c7qV1UiyWGOndWRGR4ChsZUWtTK/37+nUip4gcNoWNjCjZkKT06FImnDkh7lJEJEcpbGRYPbt62Ll6J5MXT8aKNIQmIodHYSPDan2kFe9xahfrwqQicvgUNjKsoD6g7FNljD9tfNyliEgOU9jIQe0P9rNr7a7UEJppCE1EDp/CRg4q+XAS+tGJnCJyxBQ2clBBfUD57HLGnTAu7lJEJMcpbGRI3W930/6Hdg2hiUhGKGxkSMGDATg6kVNEMkJhI0MK6gMqTqmg/M/L4y5FRPKAwkY+5v0336fj2Q4dGCAiGaOwkY9JNiYBdCKniGSMwkY+JmgIGH/GeMbOGBt3KSKSJxQ28hFdW7rofL5TQ2giklGRho2ZzTezLWa2zcyWDzHfzGxFOP9FM5szUl8zqzKzx81sa/hcOWDezWH7LWY2b8D0q8zspXAdvzazmii3O5cFDQEYTF6ksBGRzIksbMwsAdwBLABmA1eZ2exBzRYAs8LHMuCuNPouB9a6+yxgbfiecP4S4HhgPnCnmSXMbAzwQ+A8dz8JeBG4MZKNznHuTlAfMPHsiZROK427HBHJI1Hu2ZwObHP3N9x9P1APLBzUZiFwv6esAyaZ2dQR+i4EVoavVwKXDphe7+7d7v4msC1cjoWPcZY6O3EC8E7mNzf37X15L12vdOncGhHJuCjDZhqwY8D7lnBaOm2G6zvF3d8FCJ8PfDMO2cfde4CvAC+RCpnZwL2Ht0n5LWgIoAhqr9RRaCKSWVGGzVDXOPE026TTN631mVkxqbA5FTiK1DDazUMuwGyZmTWbWXMymRxhdfnlwBBa5QWVlEwuibscEckzUYZNC3D0gPfT+fjw1cHaDNf3vXCojfA5GGFZpwC4++vu7kAj8NmhCnb3e9y9zt3ramsL66/7zj92su/1fRpCE5FIRBk2zwGzzGymmZWQ+vG+aVCbJuDq8Ki0uUB7ODQ2XN8mYGn4eimwasD0JWZWamYzSR10sB54G5htZgfS4/PAK5ne2FwX1AdYsVFzmQ7UE5HMGxPVgt2918xuBFYDCeA+d99kZteH8+8GHgMuIvVjfhdw7XB9w0XfCjSa2XXAW8CisM8mM2sENgO9wA3u3ge8Y2bfBp40sx7gv4BrotruXOT9TtAQUHlhJcVVxXGXIyJ5yFIjSzJYXV2dNzc3x13GqGh/up3nP/c8n/nJZ/jEX30i7nJEJIeZ2QZ3rxs8XVcQEIL6gKKyImou0RCaiERDYVPgvM9JPpik6qIqxkyIbFRVRAqcwqbA7X5yN/v/tF/XQhORSClsClxQH1A0rojqL1THXYqI5DGFTQHr7+kn+XCSmktqSJQn4i5HRPKYwqaA7Vq7i962Xg2hiUjkFDYFLNmQJDExQdW8qrhLEZE8p7ApUP3d/SR/lqT2slqKSvXPQESipW+ZArXz1zvp29NH7eLCugaciMRDYVOggoaAMdVjqLygcuTGIiJHSGFTgPq6+mhtaqX2ylqKivVPQESip2+aAtT2yzb69/brdgIiMmoUNgUoqA8o+UQJk86ZFHcpIlIgFDYFpndPLzsf20ntolosMdTNTUVEMk9hU2Bam1rp39evEzlFZFQpbApMUB9QenQpE+ZOiLsUESkgCpsC0rOzh11rdjF58WSsSENoIjJ6FDYFpPWRVrzHNYQmIqNOYVNAgoaAsk+VUTGnIu5SRKTAKGwKxP5gP7vW7mLyksmYaQhNREaXwqZAJB9KQj86kVNEYhFp2JjZfDPbYmbbzGz5EPPNzFaE8180szkj9TWzKjN73My2hs+VA+bdHLbfYmbzwmnjzWzjgEermd0e5XZno6AhoHx2OeNOGBd3KSJSgCILGzNLAHcAC4DZwFVmNntQswXArPCxDLgrjb7LgbXuPgtYG74nnL8EOB6YD9xpZgl373D3Uw48gP8CfhbNVmen7re7af9Du4bQRCQ2Ue7ZnA5sc/c33H0/UA8sHNRmIXC/p6wDJpnZ1BH6LgRWhq9XApcOmF7v7t3u/iawLVzOB8xsFjAZ+EMGtzPrBQ8G4BpCE5H4RBk204AdA963hNPSaTNc3ynu/i5A+HzgGzSd9V0FNLi7D1WwmS0zs2Yza04mk8NsWm4J6gMqTq2g/NPlcZciIgUqyrAZarxm8Jf8wdqk0/dw1rcE+OnBFuDu97h7nbvX1dbmx03F3n/zfTqe7dBejYjEKsqwaQGOHvB+OvBOmm2G6/teONRG+Byksz4zOxkY4+4bDmdjclWyMbWHpjtyikicogyb54BZZjbTzEpI7VU0DWrTBFwdHpU2F2gPh8aG69sELA1fLwVWDZi+xMxKzWwmqYMO1g9Y11UMs1eTr4L6gAlzJzB2xti4SxGRAjYmqgW7e6+Z3QisBhLAfe6+ycyuD+ffDTwGXETqx/wu4Nrh+oaLvhVoNLPrgLeARWGfTWbWCGwGeoEb3L1vQElfDNdVMLq2dNG5sZNP/d9PxV2KiBQ4O8hv5QWvrq7Om5ub4y7jiGz/X9vZ/q3tnLnjTEqnlcZdjogUADPb4O51g6frCgJ5yt0J6gMmnj1RQSMisVPY5Km9L++l65UuXeFZRLKCwiZPBfUBJKD2Ch2FJiLxGzFszKzIzF4ejWIkM9ydoCGg8vxKSiaXxF2OiMjIYePu/cALZnbMKNQjGdCxoYN9r+/TiZwikjXSPfR5KrDJzNYDew9MdPdLIqlKjkiyIYkVGzWX18RdiogIkH7YfDvSKiRjvD81hFY1r4riyuK4yxERAdIMG3f/fdSFSGbseWYP3Tu6mfm/Z8ZdiojIB4YNGzPrYOgLYBrg7j4hkqrksAUNAUVlRdRcoiE0Eckew4aNu48frULkyHmfk3wwSdUXqhgzIbIrEYmIHDKdZ5NHdj+5m/1/2q+j0EQk6yhs8khQH1A0rojqL1THXYqIyEcobPJEf08/yYeS1FxSQ6I8EXc5IiIfobDJE7vW7qJ3Z6+uhSYiWUlhkyeC+oDExARV86riLkVE5GMUNnmgv7uf1kdaqb2slqJS/ScVkeyjb6Y8sPPXO+nb06chNBHJWgqbPBDUB4ypHsOk8yfFXYqIyJAUNjmub28frU2t1F5ZS1Gx/nOKSHbSt1OOa/tlG/1d/RpCE5GsprDJcUFDQMknSph09qS4SxEROahIw8bM5pvZFjPbZmbLh5hvZrYinP+imc0Zqa+ZVZnZ42a2NXyuHDDv5rD9FjObN2B6iZndY2avmdmrZnZFlNs9Wnr39NL2yzZqv1iLJSzuckREDiqysDGzBHAHsACYDVxlZrMHNVsAzAofy4C70ui7HFjr7rOAteF7wvlLgOOB+cCd4XIA/icQuPunw+XlxS0TWle14t2ua6GJSNaLcs/mdGCbu7/h7vuBemDhoDYLgfs9ZR0wycymjtB3IbAyfL0SuHTA9Hp373b3N4Ft4XIAvgz8E6Ruc+3urRne1lgEDQGlx5QyYa7u9CAi2S3KsJkG7BjwviWclk6b4fpOcfd3AcLnA3/WD9nHzCaF779jZn80swfNbMpQBZvZMjNrNrPmZDKZxibGp2dnD7tW72LyFydjRRpCE5HsFmXYDPUNOPhGbAdrk07fdNc3BpgO/Ke7zwGeAb4/1ALc/R53r3P3utra2hFWF6/WR1rxXtdRaCKSE6IMmxbg6AHvpwPvpNlmuL7vhUNthM/BCMtqA7qAR8LpDwJzyHFBfcDYY8dSMaci7lJEREYUZdg8B8wys5lmVkLqx/umQW2agKvDo9LmAu3h0NhwfZuApeHrpcCqAdOXmFmpmc0kddDBend34OfAuWG7C4DNGd7WUbX/vf3s+u0uahfXYqYhNBHJfpHdO9jde83sRmA1kADuc/dNZnZ9OP9u4DHgIlI/5ncB1w7XN1z0rUCjmV0HvAUsCvtsMrNGUkHSC9zg7n1hn38AfmJmtwPJA+vJVcmHk9CPhtBEJGdY6g9/Gayurs6bm5vjLmNIz5/zPD1tPZy+6fSRG4uIjCIz2+DudYOn6woCOWZfyz7an2rXXo2I5BSFTY5JPpgERydyikhOUdjkmKA+oOLUCso/XR53KSIiaVPY5JD333yfjvUdGkITkZyjsMkhQUPqlKLaL2b3CaciIoMpbHJIsiHJhLkTGDtjbNyliIgcEoVNjtj76l46N3ZqCE1EcpLCJkckG5JgULtIQ2giknsUNjnA3QnqAyaeM5HSo0rjLkdE5JApbHLA3pf20vVql86tEZGcpbDJAUFDAAmovUJDaCKSmxQ2We7AEFrlBZWUTC6JuxwRkcOisMlyHc0d7Htjn4bQRCSnKWyyXNAQYMVGzWU1cZciInLYFDZZzPudZEOSqnlVFFcWx12OiMhhU9hksT3P7KG7pVsncopIzlPYZLGgPqCorIjqS6rjLkVE5IgobLKU9znBgwFVX6hizPjI7t4tIjIqFDZZavfvd9PzXo+G0EQkLyhsslRQH1A0rojqizSEJiK5L9KwMbP5ZrbFzLaZ2fIh5puZrQjnv2hmc0bqa2ZVZva4mW0NnysHzLs5bL/FzOYNmP5EOG1j+Mjq3YX+nn6SDyepWVhDojwRdzkiIkcssrAxswRwB7AAmA1cZWazBzVbAMwKH8uAu9LouxxY6+6zgLXhe8L5S4DjgfnAneFyDvhLdz8lfASZ3t5M2vWbXfTu7NWJnCKSN6Lcszkd2Obub7j7fqAeWDiozULgfk9ZB0wys6kj9F0IrAxfrwQuHTC93t273f1NYFu4nJwT1AckJiaomlcVdykiIhkRZdhMA3YMeN8STkunzXB9p7j7uwDh84E//0da37+HQ2i3mJkNVbCZLTOzZjNrTiaTI21fJPr29dH6aCu1l9dSVKqf1EQkP0T5bTbUF7qn2Sadvoeyvr909xOBs8PHXw+1AHe/x93r3L2utjaeKyzv/PVO+vb0aQhNRPJKlGHTAhw94P104J002wzX971wqI3w+cDvLwft4+5vh88dwP8ji4fXkg1JimuKmXT+pLhLERHJmCjD5jlglpnNNLMSUj/eNw1q0wRcHR6VNhdoD4fGhuvbBCwNXy8FVg2YvsTMSs1sJqmDDtab2RgzqwEws2LgYuDlKDb4SPXt7aO1qZWaK2ooKtYQmojkj8hOTXf3XjO7EVgNJID73H2TmV0fzr8beAy4iNSP+V3AtcP1DRd9K9BoZtcBbwGLwj6bzKwR2Az0Aje4e5+ZjQNWh0GTAH4D/Ciq7T4Sbb9oo7+rXydyikjeMfeRfgopTHV1dd7c3Dyq63z58pfZs24PZ+44E0sMeQyDiEhWM7MN7l43eLrGarJE755e2h5ro3ZRrYJGRPKOwiZLtK5qxbtdQ2gikpcUNlkiqA8oPaaUCXMnxF2KiEjGKWyyQM/OHnat2cXkxZM5yPmmIiI5TWGTBZI/S+K9GkITkfylsMkCQX3A2GPHUnFqRdyliIhEQmETs/3v7Wf373YzeYmG0EQkfylsYpZ8KAn9ULs4nmuxiYiMBoVNzIKGgPLjy6k4QUNoIpK/FDYx2teyj/Y/tOsKzyKS9xQ2MUo2pu6Zo7ARkXynsIlR0BBQMaeC8k+Xx12KiEikFDYxef+N9+lY36G9GhEpCAqbmASNqXu+1X5RR6GJSP5T2MQkqA+YMHcCY2eMjbsUEZHIKWxisPfVvex9Ya8uTyMiBUNhE4NkQxIMahdpCE1ECoPCZpS5O0F9wMRzJlJ6VGnc5YiIjAqFzSjb+9Jeul7t0hCaiBQUhc0oC+oDSEDtFRpCE5HCobAZRQeG0CovqKSktiTuckRERk2kYWNm881si5ltM7PlQ8w3M1sRzn/RzOaM1NfMqszscTPbGj5XDph3c9h+i5nNG2J9TWb2chTbmo6O5g72vblPQ2giUnAiCxszSwB3AAuA2cBVZjZ7ULMFwKzwsQy4K42+y4G17j4LWBu+J5y/BDgemA/cGS7nQD2XA52Z39L0BfUBVmzUXFoTZxkiIqMuyj2b04Ft7v6Gu+8H6oGFg9osBO73lHXAJDObOkLfhcDK8PVK4NIB0+vdvdvd3wS2hcvBzCqArwPfjWA70+L9TrIxSdX8Koori+MqQ0QkFlGGzTRgx4D3LeG0dNoM13eKu78LED4fGJMars93gB8AXcMVbGbLzKzZzJqTyeRwTQ9Z+9PtdLd061poIlKQogyboe5x7Gm2SadvWuszs1OAY939kRH64+73uHudu9fV1mb2aLGgPqCorIjqS6ozulwRkVwQZdi0AEcPeD8deCfNNsP1fS8caiN8DkZY1pnAaWa2HXgK+LSZPXFYW3SY+nv7ST6YpPriasaMHzOaqxYRyQpRhs1zwCwzm2lmJaR+vG8a1KYJuDo8Km0u0B4OjQ3XtwlYGr5eCqwaMH2JmZWa2UxSBx2sd/e73P0od58BnAW85u7nRrHBB9P++3Z6gh5qF+vcGhEpTJH9me3uvWZ2I7AaSAD3ufsmM7s+nH838BhwEakf87uAa4frGy76VqDRzK4D3gIWhX02mVkjsBnoBW5w976otu9QBA0BiYoE1RdpCE1ECpO5j/RTSGGqq6vz5ubmI15O//5+np76NFXzq5j9wOAjv0VE8ouZbXD3usHTdQWBiO36zS56d/bqRE4RKWj6tTpiQUNAYmKCqgur4i5FJKf19PTQ0tLCvn374i5FgLKyMqZPn05xcXrnDSpsItS3r4/WR1qpvbKWolLtRIociZaWFsaPH8+MGTMwG+pMBxkt7k5bWxstLS3MnDkzrT76BozQzl/vpK+jT0NoIhmwb98+qqurFTRZwMyorq4+pL1MhU2EgvqA4ppiJp0/Ke5SRPKCgiZ7HOp/C4VNRPr29tH287bUENoYfcwiUtj0LRiRtl+00d/VrxM5RfJIRUVF5Ou4++67uf/++yNfz0CPPvoomzdvjnQdOkAgIkF9QMnUEiadPSnuUkQky/T19ZFIJIacd/3114/6Oh999FEuvvhiZs+O7lxAhU0Eett7aftVG0ddfxSW0BizSKZt/dpWOjdm9vZUFadUMOv2WWm3v+2222hsbKS7u5vLLruMb3/72wBceuml7Nixg3379nHTTTexbNmy1PIrKvj617/O6tWr+cEPfsD8+fO56aab+MUvfsHYsWNZtWoVU6ZM4Vvf+hYVFRV84xvf4Nxzz+WMM87gd7/7Hbt37+bee+/l7LPPpquri2uuuYZXX32V4447ju3bt3PHHXdQV/fRcylnzJjBl7/8ZdasWcONN95IR0cH99xzD/v37+fYY4/lJz/5CRs3bqSpqYnf//73fPe73+Xhhx8G4IYbbiCZTFJeXs6PfvQjPvOZzxzR56thtAi0rmrFu123ExDJU2vWrGHr1q2sX7+ejRs3smHDBp588kkA7rvvPjZs2EBzczMrVqygra0NgL1793LCCSfw7LPPctZZZ7F3717mzp3LCy+8wDnnnMOPfvSjIdfV29vL+vXruf322z8ItDvvvJPKykpefPFFbrnlFjZs2HDQWsvKynjqqadYsmQJl19+Oc899xwvvPACxx13HPfeey+f/exnueSSS7jtttvYuHEjn/rUp1i2bBn/8i//woYNG/j+97/PV7/61SP+zLRnE4GgPqD0k6VMmDsh7lJE8tKh7IFEYc2aNaxZs4ZTTz0VgM7OTrZu3co555zDihUreOSR1B1NduzYwdatW6muriaRSHDFFVd8sIySkhIuvvhiAE477TQef/zxIdd1+eWXf9Bm+/btADz11FPcdNNNAJxwwgmcdNJJB6118eLFH7x++eWX+eY3v8nu3bvp7Oxk3rx5H2vf2dnJ008/zaJFiz6Y1t3dPeJnMhKFTYb1tPWw6/FdTP+76TpMUyRPuTs333wzf/M3f/OR6U888QS/+c1veOaZZygvL+fcc8/94FyUsrKyj/xmUlxc/MF3RCKRoLe3d8h1lZaWfqzNoVzTcty4cR+8vuaaa3j00Uc5+eST+fGPf8wTTzzxsfb9/f1MmjSJjRs3pr2OdGgYLcOSP0viva4TOUXy2Lx587jvvvvo7Ez9bvT2228TBAHt7e1UVlZSXl7Oq6++yrp16yJZ/1lnnUVjYyMAmzdv5qWXXkqrX0dHB1OnTqWnp4cHHnjgg+njx4+no6MDgAkTJjBz5kwefPBBIBVsL7zwwhHXrLDJsKAhYOyssVScGv0hkiISjwsvvJAvfelLnHnmmZx44olceeWVdHR0MH/+fHp7eznppJO45ZZbmDt3biTr/+pXv0oymeSkk07ie9/7HieddBITJ04csd93vvMdzjjjDD7/+c9/5Af/JUuWcNttt3Hqqafy+uuv88ADD3Dvvfdy8sknc/zxx7Nq1aphlpoe3WLgIA7nFgPuzut//zqlx5Ry9NeOHrmDiKTtlVde4bjjjou7jKzQ19dHT08PZWVlvP7661xwwQW89tprlJSUjGodQ/03OdgtBvSbTQaZGcf+87FxlyEiea6rq4vzzjuPnp4e3J277rpr1IPmUClsRERyzPjx48nEzR1Hk36zEZGcoWH/7HGo/y0UNiKSE8rKymhra1PgZIED97MpKytLu4+G0UQkJ0yfPp2WlhaSyWTcpQgf3qkzXQobEckJxcXFad8VUrJPpMNoZjbfzLaY2TYzWz7EfDOzFeH8F81szkh9zazKzB43s63hc+WAeTeH7beY2bwB039tZi+Y2SYzu9vMhr70qYiIRCKysAm/0O8AFgCzgavMbPD1qxcAs8LHMuCuNPouB9a6+yxgbfiecP4S4HhgPnDngFD5orufDJwA1AIfXvRHREQiF+WezenANnd/w933A/XAwkFtFgL3e8o6YJKZTR2h70JgZfh6JXDpgOn17t7t7m8C28Ll4O57wjZjgBJAvzCKiIyiKH+zmQbsGPC+BTgjjTbTRug7xd3fBXD3d83swEXIpgHrBvWZduCNma0mFT6/Ah4aqmAzW0ZqDwug08y2DLN9w6kBWg+zbz7S5/EhfRYfpc/jQ/nyWXxyqIlRhs1QlzwevEdxsDbp9D2k9bn7PDMrAx4Azgc+dj1vd78HuGeE9YzIzJqHulxDodLn8SF9Fh+lz+ND+f5ZRDmM1gIMvEDYdOCdNNsM1/e9cKiN8DlId33uvg9o4uPDeSIiEqEow+Y5YJaZzTSzElI/3jcNatMEXB0elTYXaA+HyIbr2wQsDV8vBVYNmL7EzErNbCapgw7Wm1nFgHAaA1wEvBrFBouIyNAiG0Zz914zuxFYDSSA+9x9k5ldH86/G3iM1Jf/NqALuHa4vuGibwUazew64C3CI8vCZTcCm4Fe4AZ37zOzcUCTmZWGy/otcHdU2x064qG4PKPP40P6LD5Kn8eH8vqz0C0GREQkcro2moiIRE5hIyIikVPYZNBIl+cpJGZ2tJn9zsxeCS8TdFPcNcXNzBJm9ryZ/SLuWuJmZpPM7CEzezX8N3Jm3DXFycz+Lvz/5GUz+2l4mkZeUdhkSJqX5ykkvcDfu/txwFzghgL/PABuAl6Ju4gs8UPg1+7+GeBkCvhzMbNpwN8Cde5+AqkDmZbEW1XmKWwyJ53L8xQMd3/X3f8Yvu4g9WUybfhe+cvMpgNfAP4t7lriZmYTgHOAewHcfb+77461qPiNAcaGp2eU8/FzEnOewiZzDnbpnYJnZjOAU4FnYy4lTrcD/wPoj7mObPBnQBL493BY8d/CUxQKkru/DXyf1Kkc75I633BNvFVlnsImcw7nEjt5z8wqgIeBrw24IGpBMbOLgcDdN8RdS5YYA8wB7nL3U4G9hFdvL0ThbVIWAjOBo4BxZvZX8VaVeQqbzEnn8jwFxcyKSQXNA+7+s7jridHngEvMbDup4dXzzew/4i0pVi1Ai7sf2NN9iFT4FKq/AN5096S79wA/Az4bc00Zp7DJnHQuz1MwzMxIjcm/4u7/HHc9cXL3m919urvPIPXv4rfunnd/uabL3f8E7DCzPw8nXUDqyh+F6i1grpmVh//fXEAeHjCh20JnyAiX2ClEnwP+GnjJzDaG0/7R3R+LryTJIv8NeCD8w+wNwktVFSJ3f9bMHgL+SOoozufJw0vX6HI1IiISOQ2jiYhI5BQ2IiISOYWNiIhETmEjIiKRU9iIiEjkFDYiWczMvmZm5XHXIXKkdOizSBYLrzpQ5+6tcdciciR0UqdIlggvRtlI6lJHCeBBUtfK+p2Ztbr7eWZ2IfBtoBR4HbjW3TvDUGoAzgsX9yV33zba2yByMBpGE8ke84F33P3k8L4mt5O6vt55YdDUAN8E/sLd5wDNwNcH9N/j7qcD/xr2FckaChuR7PES8Bdm9j0zO9vd2wfNn0vqxnz/GV4CaCnwyQHzfzrguaDvfCnZR8NoIlnC3V8zs9OAi4B/MrPB9zQx4HF3v+pgizjIa5HYac9GJEuY2VFAl7v/B6mbac0BOoDxYZN1wOfM7NiwfbmZfXrAIhYPeH5mdKoWSY/2bESyx4nAbWbWD/QAXyE1HPYrM3s3/N3mGuCnZlYa9vkm8Fr4utTMniX1R+TB9n5EYqFDn0XygA6RlmynYTQREYmc9mxERCRy2rMREZHIKWxERCRyChsREYmcwkZERCKnsBERkcj9f/d0RhyNzqYAAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.lineplot(data=df_train, x=\"step\", y=\"lr\", label=\"learning rate\", color=\"m\")\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "methylbert",
   "language": "python",
   "name": "methylbert"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}