{ "cells": [ { "cell_type": "markdown", "id": "19bcabe9", "metadata": { "id": "19bcabe9" }, "source": [ "\n", "Contents:\n", "- [Neural Networks with PyTorch](#Neural-Networks-with-PyTorch)\n", " - [Classification with Fashion MNIST dataset](#Classification-with-Fashion-MNIST-dataset)\n", " - [Getting the dataset](#Getting-the-dataset)\n", " - [Setting up the PyTorch Lightning model](#Setting-up-the-PyTorch-Lightning-model)\n", " - [Training the model using the PyTorch Lightning Trainer class](#Training-the-model-using-the-PyTorch-Lightning-Trainer-class)\n", " - [Evaluating the model using TensorBoard](#Evaluating-the-model-using-TensorBoard)\n", " - [Visualizing model performance using csv logs](#Visualizing-model-performance-using-csv-logs)\n", " - [Evaluating the trained model on the test dataset](#Evaluating-the-trained-model-on-the-test-dataset)\n", " - [Making predictions](#Making-predictions)\n", " - [Saving and reloading the trained model](#Saving-and-reloading-the-trained-model)\n", " - [Early stopping and saving the model during training](#Early-stopping-and-saving-the-model-during-training)\n", " - [Regression with neural networks: predicting the fuel efficiency of a car](#Regression-with-neural-networks:-predicting-the-fuel-efficiency-of-a-car)\n", " - [Preprocessing the dataset](#Preprocessing-the-dataset)\n", " - [Training and evaluating the network](#Training-and-evaluating-the-network)\n", " - [Convolutional neural networks](#Convolutional-neural-networks)\n", " - [Hyperparameter tuning](#Hyperparameter-tuning)" ] }, { "cell_type": "markdown", "id": "4606fc54-5719-4685-afc8-ff484b20b784", "metadata": { "id": "4606fc54-5719-4685-afc8-ff484b20b784" }, "source": [ "# Neural Networks with PyTorch" ] }, { "cell_type": "markdown", "id": "af65bed0-3d4c-403a-91db-1ccef9f9b490", "metadata": { "id": "af65bed0-3d4c-403a-91db-1ccef9f9b490" }, "source": [ "There are different python libraries that support Deep Learning, some of the more popular ones are Keras, Tensorflow and PyTorch. PyTorch was developed by the Facebook Research lab and released in 2016, and it has been widely adopted by the industry and used in developing deep learning solutions, such as Tesla Autopilot, Uber's Pyro, and Hugging Face's Transformers. Hence, we will also use PyTorch in this course. It is a free and open source software that allows us to build, train, evaluate and execute neural networks. In recent years, the PyTorch community developed several different libraries and APIs on top of Py-Torch. Notable examples include fastai, Catalyst and PyTorch Lightning. We will use PyTorch Lightning (Lightning for short), which is a widely used as it makes training deep neural networks simpler by removing much of the boilerplate code.\n", "\n", "Note that this notebook is adapted from chapter 12, 13 and 14 of the book [Machine Learning with PyTorch and Scikit-Learn, S. Raschka, Y. Liu, V. Mirjalili](https://github.com/rasbt/machine-learning-book).\n", "\n", "Let's start with installing `pytorch`, and additionaly `torchvision` which consists of popular datasets, model architectures, and common image transformations for computer vision.\n", "To install PyTorch, it's recommended to check the exact instructions based on the operating system on this [link](https://pytorch.org/get-started/locally/). To install localy you should run the following:\n", "\n", "`pip install torch torchvision torchinfo pytorch-lightning tensorboard`\n", "\n", "It is recommended that you run this notebook on [google colab](https://colab.research.google.com/) and change the runtime type to GPU to have the notebook executed in a shorter time period. In this case, no need to install pytorch, but only t additional libraries `pytorch-lightning`, `torchinfo` and `ray[tune]`, which can be done by uncommenting code in this notebook .\n", "`pip install torch torchvision torchinfo pytorch-lightning tensorboard ray[tune]\n", "If you have any trouble installing these libraries locally, try creating a new environment." ] }, { "cell_type": "markdown", "id": "9c11cf2a-5a51-4395-8f8b-9ec47bf5aa73", "metadata": { "id": "9c11cf2a-5a51-4395-8f8b-9ec47bf5aa73" }, "source": [ "Once installed, the core functionality of PyTorch is provided in a module named `torch`. This is the name we use when importing the library. The `torch` module contains all the primary functions and classes we need to work with PyTorch, such as tensors, neural network layers, and optimization algorithms. Now let's import the library and check the version:" ] }, { "cell_type": "code", "execution_count": 1, "id": "e3ba96ec-1e5c-4521-8bd7-d4f9e1bb321d", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "e3ba96ec-1e5c-4521-8bd7-d4f9e1bb321d", "outputId": "d2d39040-57f8-4762-e860-0b1a59672477" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "2.6.0+cu124\n" ] } ], "source": [ "import torch\n", "print(torch.__version__)" ] }, { "cell_type": "markdown", "id": "4be3487a-a45b-4a74-bd03-1d5cdad1ca51", "metadata": { "id": "4be3487a-a45b-4a74-bd03-1d5cdad1ca51" }, "source": [ "Uncomment the following when running on colab (it may take a few minutes to execute)" ] }, { "cell_type": "code", "execution_count": 2, "id": "9dafdded-8b06-403a-9256-5968ec6d05c5", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "9dafdded-8b06-403a-9256-5968ec6d05c5", "outputId": "be4aa24f-3819-4d1b-fbae-7b2ca229a107" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m823.0/823.0 kB\u001b[0m \u001b[31m12.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m363.4/363.4 MB\u001b[0m \u001b[31m4.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m13.8/13.8 MB\u001b[0m \u001b[31m67.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m24.6/24.6 MB\u001b[0m \u001b[31m64.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m883.7/883.7 kB\u001b[0m \u001b[31m43.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m664.8/664.8 MB\u001b[0m \u001b[31m1.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m211.5/211.5 MB\u001b[0m \u001b[31m5.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m56.3/56.3 MB\u001b[0m \u001b[31m15.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m127.9/127.9 MB\u001b[0m \u001b[31m7.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m207.5/207.5 MB\u001b[0m \u001b[31m5.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m21.1/21.1 MB\u001b[0m \u001b[31m92.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m961.5/961.5 kB\u001b[0m \u001b[31m52.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25h" ] } ], "source": [ "!pip install -q pytorch-lightning torchinfo tensorboard" ] }, { "cell_type": "code", "execution_count": 3, "id": "081293f0-5d05-4704-9503-d9bd859fb1f7", "metadata": { "id": "081293f0-5d05-4704-9503-d9bd859fb1f7" }, "outputs": [], "source": [ "import torch.nn as nn\n", "import pytorch_lightning as pl\n", "from torchvision import datasets, transforms\n", "from torch.utils.data import Subset, DataLoader, TensorDataset\n", "import torch.optim as optim\n", "from torchinfo import summary\n", "from torchmetrics.classification import Accuracy\n", "from torchmetrics.regression import MeanAbsoluteError\n", "from pytorch_lightning.callbacks import ModelCheckpoint, EarlyStopping\n", "from pytorch_lightning.loggers import CSVLogger, TensorBoardLogger" ] }, { "cell_type": "code", "execution_count": 4, "id": "e3c07801-7dda-4bb3-a927-46abcc3ca511", "metadata": { "id": "e3c07801-7dda-4bb3-a927-46abcc3ca511" }, "outputs": [], "source": [ "import matplotlib.pyplot as plt\n", "import pandas as pd\n", "import numpy as np\n", "import os\n", "import json\n", "import sklearn\n", "from sklearn.model_selection import train_test_split\n", "from sklearn.preprocessing import StandardScaler, OneHotEncoder\n", "from sklearn.compose import ColumnTransformer\n" ] }, { "cell_type": "markdown", "id": "2634252e-1f6a-4e49-9c94-a900bf76f7f9", "metadata": { "id": "2634252e-1f6a-4e49-9c94-a900bf76f7f9" }, "source": [ "## Classification with Fashion MNIST dataset" ] }, { "cell_type": "markdown", "id": "bb886bba-19be-4328-9c90-5dfd88771038", "metadata": { "id": "bb886bba-19be-4328-9c90-5dfd88771038" }, "source": [ "To get started with neural networks, we will classify different types of fashion products.\n", "We will use Fashion MNIST dataset which contains 70,000 gray scale images in 10 categories. The images show individual articles of clothing at low resolution (28 by 28 pixels), as seen here:\n", "\n", "\n", " \n", " \n", "
\n", " \"Fashion\n", "
\n", " Fashion-MNIST samples (by Zalando, MIT License).
 \n", "
\n", "\n", "Fashion MNIST is a dataset often used as a starting point for training of neural networks. Compared to MNIST dataset of handwritten digits, it is a slightly more challenging problem.\n", "\n", "Here, 60,000 images are used to train the network and 10,000 images to evaluate how accurately the network learned to classify images.\n", "\n", "Let's import and load the Fashion MNIST data directly from *torchvision.datasets*. ." ] }, { "cell_type": "markdown", "id": "968ee0ea-79f7-44e0-91c1-68941f67c0d4", "metadata": { "id": "968ee0ea-79f7-44e0-91c1-68941f67c0d4" }, "source": [ "### Getting the dataset" ] }, { "cell_type": "code", "execution_count": 5, "id": "0c469a7e-11e8-4678-aff7-8d26c2b4b450", "metadata": { "id": "0c469a7e-11e8-4678-aff7-8d26c2b4b450" }, "outputs": [], "source": [ "transform = transforms.Compose(\n", " [transforms.ToTensor()])" ] }, { "cell_type": "code", "execution_count": 6, "id": "152cf5c3-bd15-45e6-855f-4eaf746ece17", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "152cf5c3-bd15-45e6-855f-4eaf746ece17", "outputId": "5883d182-f0c5-4066-9264-064e9bd4a6db" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "100%|██████████| 26.4M/26.4M [00:01<00:00, 18.3MB/s]\n", "100%|██████████| 29.5k/29.5k [00:00<00:00, 298kB/s]\n", "100%|██████████| 4.42M/4.42M [00:00<00:00, 5.51MB/s]\n", "100%|██████████| 5.15k/5.15k [00:00<00:00, 16.5MB/s]\n" ] } ], "source": [ "training_set = datasets.FashionMNIST('./data', train=True, transform=transform, download=True)\n", "test_set = datasets.FashionMNIST('./data', train=False, transform=transform, download=True)" ] }, { "cell_type": "markdown", "id": "ca5d5f94-9076-4b0d-9fb8-43ab92e906c7", "metadata": { "id": "ca5d5f94-9076-4b0d-9fb8-43ab92e906c7" }, "source": [ "Here, we are creating the train and test sets. They will be downloaded in the \"*data/*\" folder. The *train* argument specifies whether to download train or validation set. The features in this project are the pixels of the image. We defined a custom transformation using *torchvision\n", "transforms.Compose*. In this simple case, our transformation consisted only of one method, *ToTensor()*. The *ToTensor* method converts the pixel features into a floating type and also normalizes the pixels from the [0, 255] to [0, 1] range.\n", "\n", "A tensor is a multi-dimensional array — a generalization of scalars, vectors, and matrices to any number of dimensions. Tensors are the core data structure used in PyTorch (and other deep learning frameworks like TensorFlow) to represent and manipulate data efficiently." ] }, { "cell_type": "markdown", "id": "8e593803-0309-464d-a016-d333e34c03ee", "metadata": { "id": "8e593803-0309-464d-a016-d333e34c03ee" }, "source": [ "The images are 1x28x28 tensors. The *labels* are integers, ranging from 0 to 9. These correspond to the *class* of clothing the image represents:\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
LabelClass
0T-shirt/top
1Trouser
2Pullover
3Dress
4Coat
5Sandal
6Shirt
7Sneaker
8Bag
9Ankle boot
\n", "\n", "Each image is mapped to a single label." ] }, { "cell_type": "code", "execution_count": 7, "id": "b0dc632c-eb1c-4136-8931-1c933d508b36", "metadata": { "id": "b0dc632c-eb1c-4136-8931-1c933d508b36" }, "outputs": [], "source": [ "class_names = ('T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat',\n", " 'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle Boot')" ] }, { "cell_type": "markdown", "id": "23e6563b-9d2b-40bb-9147-88b76290df16", "metadata": { "id": "23e6563b-9d2b-40bb-9147-88b76290df16" }, "source": [ "Let's check the size of the training and the test set:" ] }, { "cell_type": "code", "execution_count": 8, "id": "da18ca35-88a1-4254-855c-8d2485d02b4b", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "da18ca35-88a1-4254-855c-8d2485d02b4b", "outputId": "6f3cf8e0-cbea-4997-cb7a-a1f11631a035" }, "outputs": [ { "data": { "text/plain": [ "(60000, 10000)" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "len(training_set), len(test_set)" ] }, { "cell_type": "markdown", "id": "4855d8bf-6e56-4967-80ad-3cfe3f598e99", "metadata": { "id": "4855d8bf-6e56-4967-80ad-3cfe3f598e99" }, "source": [ "Let's check the shape of the data, by looking at the first datapoint:" ] }, { "cell_type": "code", "execution_count": 9, "id": "4df9d169-1461-47c2-a539-61ce1d228acb", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "4df9d169-1461-47c2-a539-61ce1d228acb", "outputId": "b1a7dc15-1ad0-4ec8-f573-34d80aac17fa" }, "outputs": [ { "data": { "text/plain": [ "(tensor([[[0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000],\n", " [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000],\n", " [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000],\n", " [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0039, 0.0000, 0.0000, 0.0510,\n", " 0.2863, 0.0000, 0.0000, 0.0039, 0.0157, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0039, 0.0039, 0.0000],\n", " [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0118, 0.0000, 0.1412, 0.5333,\n", " 0.4980, 0.2431, 0.2118, 0.0000, 0.0000, 0.0000, 0.0039, 0.0118,\n", " 0.0157, 0.0000, 0.0000, 0.0118],\n", " [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0235, 0.0000, 0.4000, 0.8000,\n", " 0.6902, 0.5255, 0.5647, 0.4824, 0.0902, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0471, 0.0392, 0.0000],\n", " [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.6078, 0.9255,\n", " 0.8118, 0.6980, 0.4196, 0.6118, 0.6314, 0.4275, 0.2510, 0.0902,\n", " 0.3020, 0.5098, 0.2824, 0.0588],\n", " [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0039, 0.0000, 0.2706, 0.8118, 0.8745,\n", " 0.8549, 0.8471, 0.8471, 0.6392, 0.4980, 0.4745, 0.4784, 0.5725,\n", " 0.5529, 0.3451, 0.6745, 0.2588],\n", " [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0039, 0.0039, 0.0039, 0.0000, 0.7843, 0.9098, 0.9098,\n", " 0.9137, 0.8980, 0.8745, 0.8745, 0.8431, 0.8353, 0.6431, 0.4980,\n", " 0.4824, 0.7686, 0.8980, 0.0000],\n", " [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.7176, 0.8824, 0.8471,\n", " 0.8745, 0.8941, 0.9216, 0.8902, 0.8784, 0.8706, 0.8784, 0.8667,\n", " 0.8745, 0.9608, 0.6784, 0.0000],\n", " [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.7569, 0.8941, 0.8549,\n", " 0.8353, 0.7765, 0.7059, 0.8314, 0.8235, 0.8275, 0.8353, 0.8745,\n", " 0.8627, 0.9529, 0.7922, 0.0000],\n", " [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0039, 0.0118, 0.0000, 0.0471, 0.8588, 0.8627, 0.8314,\n", " 0.8549, 0.7529, 0.6627, 0.8902, 0.8157, 0.8549, 0.8784, 0.8314,\n", " 0.8863, 0.7725, 0.8196, 0.2039],\n", " [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0235, 0.0000, 0.3882, 0.9569, 0.8706, 0.8627,\n", " 0.8549, 0.7961, 0.7765, 0.8667, 0.8431, 0.8353, 0.8706, 0.8627,\n", " 0.9608, 0.4667, 0.6549, 0.2196],\n", " [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0157, 0.0000, 0.0000, 0.2157, 0.9255, 0.8941, 0.9020,\n", " 0.8941, 0.9412, 0.9098, 0.8353, 0.8549, 0.8745, 0.9176, 0.8510,\n", " 0.8510, 0.8196, 0.3608, 0.0000],\n", " [0.0000, 0.0000, 0.0039, 0.0157, 0.0235, 0.0275, 0.0078, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.9294, 0.8863, 0.8510, 0.8745,\n", " 0.8706, 0.8588, 0.8706, 0.8667, 0.8471, 0.8745, 0.8980, 0.8431,\n", " 0.8549, 1.0000, 0.3020, 0.0000],\n", " [0.0000, 0.0118, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.2431, 0.5686, 0.8000, 0.8941, 0.8118, 0.8353, 0.8667,\n", " 0.8549, 0.8157, 0.8275, 0.8549, 0.8784, 0.8745, 0.8588, 0.8431,\n", " 0.8784, 0.9569, 0.6235, 0.0000],\n", " [0.0000, 0.0000, 0.0000, 0.0000, 0.0706, 0.1725, 0.3216, 0.4196,\n", " 0.7412, 0.8941, 0.8627, 0.8706, 0.8510, 0.8863, 0.7843, 0.8039,\n", " 0.8275, 0.9020, 0.8784, 0.9176, 0.6902, 0.7373, 0.9804, 0.9725,\n", " 0.9137, 0.9333, 0.8431, 0.0000],\n", " [0.0000, 0.2235, 0.7333, 0.8157, 0.8784, 0.8667, 0.8784, 0.8157,\n", " 0.8000, 0.8392, 0.8157, 0.8196, 0.7843, 0.6235, 0.9608, 0.7569,\n", " 0.8078, 0.8745, 1.0000, 1.0000, 0.8667, 0.9176, 0.8667, 0.8275,\n", " 0.8627, 0.9098, 0.9647, 0.0000],\n", " [0.0118, 0.7922, 0.8941, 0.8784, 0.8667, 0.8275, 0.8275, 0.8392,\n", " 0.8039, 0.8039, 0.8039, 0.8627, 0.9412, 0.3137, 0.5882, 1.0000,\n", " 0.8980, 0.8667, 0.7373, 0.6039, 0.7490, 0.8235, 0.8000, 0.8196,\n", " 0.8706, 0.8941, 0.8824, 0.0000],\n", " [0.3843, 0.9137, 0.7765, 0.8235, 0.8706, 0.8980, 0.8980, 0.9176,\n", " 0.9765, 0.8627, 0.7608, 0.8431, 0.8510, 0.9451, 0.2549, 0.2863,\n", " 0.4157, 0.4588, 0.6588, 0.8588, 0.8667, 0.8431, 0.8510, 0.8745,\n", " 0.8745, 0.8784, 0.8980, 0.1137],\n", " [0.2941, 0.8000, 0.8314, 0.8000, 0.7569, 0.8039, 0.8275, 0.8824,\n", " 0.8471, 0.7255, 0.7725, 0.8078, 0.7765, 0.8353, 0.9412, 0.7647,\n", " 0.8902, 0.9608, 0.9373, 0.8745, 0.8549, 0.8314, 0.8196, 0.8706,\n", " 0.8627, 0.8667, 0.9020, 0.2627],\n", " [0.1882, 0.7961, 0.7176, 0.7608, 0.8353, 0.7725, 0.7255, 0.7451,\n", " 0.7608, 0.7529, 0.7922, 0.8392, 0.8588, 0.8667, 0.8627, 0.9255,\n", " 0.8824, 0.8471, 0.7804, 0.8078, 0.7294, 0.7098, 0.6941, 0.6745,\n", " 0.7098, 0.8039, 0.8078, 0.4510],\n", " [0.0000, 0.4784, 0.8588, 0.7569, 0.7020, 0.6706, 0.7176, 0.7686,\n", " 0.8000, 0.8235, 0.8353, 0.8118, 0.8275, 0.8235, 0.7843, 0.7686,\n", " 0.7608, 0.7490, 0.7647, 0.7490, 0.7765, 0.7529, 0.6902, 0.6118,\n", " 0.6549, 0.6941, 0.8235, 0.3608],\n", " [0.0000, 0.0000, 0.2902, 0.7412, 0.8314, 0.7490, 0.6863, 0.6745,\n", " 0.6863, 0.7098, 0.7255, 0.7373, 0.7412, 0.7373, 0.7569, 0.7765,\n", " 0.8000, 0.8196, 0.8235, 0.8235, 0.8275, 0.7373, 0.7373, 0.7608,\n", " 0.7529, 0.8471, 0.6667, 0.0000],\n", " [0.0078, 0.0000, 0.0000, 0.0000, 0.2588, 0.7843, 0.8706, 0.9294,\n", " 0.9373, 0.9490, 0.9647, 0.9529, 0.9569, 0.8667, 0.8627, 0.7569,\n", " 0.7490, 0.7020, 0.7137, 0.7137, 0.7098, 0.6902, 0.6510, 0.6588,\n", " 0.3882, 0.2275, 0.0000, 0.0000],\n", " [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.1569,\n", " 0.2392, 0.1725, 0.2824, 0.1608, 0.1373, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000],\n", " [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000],\n", " [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000,\n", " 0.0000, 0.0000, 0.0000, 0.0000]]]),\n", " 9)" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "training_set[0]" ] }, { "cell_type": "code", "execution_count": 10, "id": "04d8a4d7-08e7-4be6-942d-d013be49bc5c", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "04d8a4d7-08e7-4be6-942d-d013be49bc5c", "outputId": "97c38e1b-ce3d-400f-caf1-c38721ce2b0c" }, "outputs": [ { "data": { "text/plain": [ "(torch.Size([1, 28, 28]), 9)" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "training_set[0][0].shape, training_set[0][1]" ] }, { "cell_type": "markdown", "id": "6ea11856", "metadata": { "id": "6ea11856" }, "source": [ "Note that the sample in this dataset comes in a tuple of (tensor, label), and we can see that the tensor is 1x28x28. When working with images in deep learning (especially in PyTorch), the shape of an image tensor is typically represented as: Number of color channels × Height (number of pixels) × Width (number of pixels). For gray scale images there is only 1 channel, as it is in this case, and for color RGB images, these channels refer to red, green and blue intensities." ] }, { "cell_type": "markdown", "id": "afd91710-01be-42bb-9ab1-5fb66aaf6e03", "metadata": { "id": "afd91710-01be-42bb-9ab1-5fb66aaf6e03" }, "source": [ "Let's visualize a few images:" ] }, { "cell_type": "code", "execution_count": 11, "id": "8dcef8e7-aca4-4538-a161-56323eec1cbd", "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 613 }, "id": "8dcef8e7-aca4-4538-a161-56323eec1cbd", "outputId": "555862dd-e5b5-4cbb-c154-dd0a014ea780", "scrolled": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA9EAAAKVCAYAAADIjmi9AAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAxXlJREFUeJzs3Xd4VFX+P/B3CMmkJ5QUQklCEwHB3dB7iWSp0hQ7iApqENF1VfarYmdFV1EXQVwFVBBFQZS6iIhKLzZUEDAgLSEB0iH1/P7gx8hwPwfuTSYkM/f9ep48mg8nd86dnHPvPZmZ9/VRSikQERERERER0SXVqOoOEBEREREREXkKLqKJiIiIiIiITOIimoiIiIiIiMgkLqKJiIiIiIiITOIimoiIiIiIiMgkLqKJiIiIiIiITOIimoiIiIiIiMgkLqKJiIiIiIiITOIimoiIiIiIiMgkLqKr2JgxYxASEnLJdr169UKvXr0qv0NEVejAgQPw8fHBSy+9dMm2Tz75JHx8fC5Dr4guD45/IiL74jnAs3ARXQ5vvPEGfHx80LFjx6ruSrmNGTMGPj4+zq+aNWuiYcOGuOGGG/DLL79U6mMXFBTgySefxFdffVWpj0Pud/6YudhXdfvdmhlzp06dQs2aNfHRRx8BAJ5//nl8+umnl6eD5BE4/onca+7cuS5zJyAgALGxsUhOTsZrr72G3Nzcqu4ikRPPAXS+mlXdAU80f/58xMfHY+vWrdi3bx+aNm1a1V0qF4fDgf/+978AgJKSEuzfvx+zZs3CqlWr8MsvvyA2NrZSHregoABPPfUUAPDVdQ/z3nvvuXz/7rvvYs2aNYb6lVdeWel9eeyxx/Doo4+aamtmzK1evRo+Pj7o168fgLMnkJEjR2Lo0KHu6C55AY5/osrx9NNPIyEhAcXFxUhLS8NXX32FSZMm4eWXX8Znn32GNm3aVHUXiXgOIBdcRFuUmpqKjRs3YvHixRg/fjzmz5+PKVOmVHW3yqVmzZq45ZZbXGqdOnXCoEGDsHz5ctx1111V1DOqri4cL5s3b8aaNWsM9cuhZs2aqFnz4oewsrIyFBUVmdreihUr0LVrV0RERLihd+SNOP6JKkf//v3Rrl075/eTJ0/Gl19+iUGDBmHIkCH49ddfERgYKP5sfn4+goODL1dXycZ4DqDz8e3cFs2fPx+1atXCwIEDMXLkSMyfP9/Q5vzPNMyePRtNmjSBw+FA+/btsW3btks+xvfff4/IyEj06tULeXl52naFhYWYMmUKmjZtCofDgYYNG+Lhhx9GYWFhufcvJiYGAAwT8/fff8d1112H2rVrIygoCJ06dcLy5csNP3/8+HHccccdiI6ORkBAANq2bYt58+Y5//3AgQOIjIwEADz11FPOt748+eST5e4zeY7t27cjOTkZdevWRWBgIBISEjB27Fix7aXmjvR5IB8fH0yYMAHz589Hq1at4HA4MGvWrEuOubKyMqxatQoDBw50bic/Px/z5s1zth8zZoyz/XfffYf+/fsjLCwMISEh6Nu3LzZv3uzSl3NvU/z6668xfvx41KlTB2FhYbjttttw6tSp8j6F5ME4/jn+ybw+ffrg8ccfx8GDB/H+++8D+DNHZv/+/RgwYABCQ0Nx8803Azg7jqdPn45WrVohICAA0dHRGD9+vGG8mZmHCxcuRGJiIkJDQxEWFoarrroKr7766uXZcfJaPAd41zmAr0RbNH/+fAwfPhz+/v648cYbMXPmTGzbtg3t27c3tF2wYAFyc3Mxfvx4+Pj4YNq0aRg+fDh+//13+Pn5idvftm0bkpOT0a5dOyxdulT7l9eysjIMGTIE3377LcaNG4crr7wSP/30E1555RX89ttvpj/HkJmZCQAoLS3F77//jkceeQR16tTBoEGDnG3S09PRpUsXFBQUYOLEiahTpw7mzZuHIUOG4OOPP8awYcMAAKdPn0avXr2wb98+TJgwAQkJCVi0aBHGjBmDrKws3H///YiMjMTMmTNxzz33YNiwYRg+fDgA8K1aNnD8+HH069cPkZGRePTRRxEREYEDBw5g8eLFhrblmTvnfPnll/joo48wYcIE1K1bF23btr3kmNu2bRsyMjIwYMAAAGffsnXnnXeiQ4cOGDduHACgSZMmAICff/4Z3bt3R1hYGB5++GH4+fnhzTffRK9evbB+/XpDVsKECRMQERGBJ598Env27MHMmTNx8OBBfPXVVwwFsRGOf45/su7WW2/FP//5T/zvf/9zvjuupKQEycnJ6NatG1566SUEBQUBAMaPH4+5c+fi9ttvx8SJE5Gamor//Oc/+O6777Bhwwb4+fmZmodr1qzBjTfeiL59++KFF14AAPz666/YsGED7r///sv/JJBX4DnAC88Bikzbvn27AqDWrFmjlFKqrKxMNWjQQN1///0u7VJTUxUAVadOHXXy5ElnfenSpQqA+vzzz5210aNHq+DgYKWUUt9++60KCwtTAwcOVGfOnHHZZs+ePVXPnj2d37/33nuqRo0a6ptvvnFpN2vWLAVAbdiw4aL7Mnr0aAXA8FW/fn21Y8cOl7aTJk1SAFweKzc3VyUkJKj4+HhVWlqqlFJq+vTpCoB6//33ne2KiopU586dVUhIiMrJyVFKKZWRkaEAqClTply0j1T9paSkKLOHkSVLligAatu2bdo2VubOlClTDI8NQNWoUUP9/PPPLvVLjbnHH39cxcXFudSCg4PV6NGjDW2HDh2q/P391f79+521o0ePqtDQUNWjRw9nbc6cOQqASkxMVEVFRc76tGnTFAC1dOlS7fNAnoHj/yyOfyqvc+PkYvMiPDxc/eUvf1FK/Xnt8uijj7q0+eabbxQANX/+fJf6qlWrXOpm5uH999+vwsLCVElJSXl3i2yC54Cz7HoO4Nu5LZg/fz6io6PRu3dvAGff7jBq1CgsXLgQpaWlhvajRo1CrVq1nN93794dwNm3Rl9o3bp1SE5ORt++fbF48WI4HI6L9mXRokW48sor0aJFC2RmZjq/+vTp49zepQQEBGDNmjVYs2YNVq9ejTfffBMhISEYMGAAfvvtN2e7FStWoEOHDujWrZuzFhISgnHjxuHAgQPONO8VK1YgJiYGN954o7Odn58fJk6ciLy8PKxfv/6SfSLvde5zNsuWLUNxcfFF21qZOxfq2bMnWrZsaalvK1ascL6N6WJKS0vxv//9D0OHDkXjxo2d9Xr16uGmm27Ct99+i5ycHJefGTdunMtfju+55x7UrFkTK1assNRH8mwc/2dx/JNVISEhhpTue+65x+X7RYsWITw8HNdcc43LNVFiYiJCQkKc10Rm5mFERATy8/OxZs0a9+8M2RbPAWd50zmAi2iTSktLsXDhQvTu3RupqanYt28f9u3bh44dOyI9PR1r1641/EyjRo1cvj83IS78LMCZM2cwcOBA/OUvf8FHH30Ef3//S/Zn7969+PnnnxEZGeny1bx5cwBn3zZyKb6+vkhKSkJSUhL69euHcePG4YsvvkB2djYmT57sbHfw4EFcccUVhp8/lz548OBB53+bNWuGGjVqXLQdebe8vDykpaU5vzIyMgCcPbCPGDECTz31FOrWrYtrr70Wc+bMET/Db3buSBISEiz1Ny0tDTt37jR1AsnIyEBBQYF2PpSVleHQoUMu9WbNmrl8HxISgnr16uHAgQOW+kmegeOf45/cKy8vD6Ghoc7va9asiQYNGri02bt3L7KzsxEVFWW4LsrLy3NeE5mZh/feey+aN2+O/v37o0GDBhg7dixWrVp1eXaWPB7PAfY5B3ARbdKXX36JY8eOYeHChWjWrJnz6/rrrwcAMWDM19dX3JZSyuV7h8OBgQMHYsuWLaYP1GVlZbjqqqucryRf+HXvvfda3MOzGjRogCuuuAJff/11uX6e6KWXXkK9evWcX+fyAnx8fPDxxx9j06ZNmDBhAo4cOYKxY8ciMTHREKBndu5IdDkCOitXrkRAQIDzHSZEFcHxT+Q+hw8fRnZ2tsutRB0Oh+GP9WVlZYiKitJeEz399NMAzM3DqKgofP/99/jss88wZMgQrFu3Dv3798fo0aMv346Tx+I5wD4YLGbS/PnzERUVhRkzZhj+bfHixViyZAlmzZplefACZyfW/Pnzce211+K6667DypUrL3n/5CZNmuCHH35A37593f7B/JKSEpcJHRcXhz179hja7d692/nv5/77448/oqyszOUEd2E7jw8SoIu67bbbXN76f+Gc6NSpEzp16oTnnnsOCxYswM0334yFCxfizjvvrLQ+XWzMLV++HL179zb0U/qZyMhIBAUFaedDjRo10LBhQ5f63r17XU5OeXl5OHbsmDPAg7wLxz/HP7nPufvvJicnX7RdkyZN8MUXX6Br166mrsMuNQ/9/f0xePBgDB48GGVlZbj33nvx5ptv4vHHH3dZ0BNdiOcA+5wD+Eq0CadPn8bixYsxaNAgjBw50vA1YcIE5Obm4rPPPiv3Y/j7+2Px4sVo3749Bg8ejK1bt160/fXXX48jR47grbfeEvubn59frn789ttv2LNnD9q2beusDRgwAFu3bsWmTZuctfz8fMyePRvx8fHOz14MGDAAaWlp+PDDD53tSkpK8PrrryMkJAQ9e/YEAGeSZlZWVrn6SNVb48aNnR8TSEpKQteuXQGcfRvShX9FvfrqqwGgQrdlM0M35oqLi7FmzRrxbUzBwcGG9r6+vujXrx+WLl3q8lak9PR0LFiwAN26dUNYWJjLz8yePdvl808zZ85ESUkJ+vfvX7GdomqJ45/jn9zjyy+/xDPPPIOEhATnbax0rr/+epSWluKZZ54x/FtJSYlzLJuZhydOnHD59xo1ajiTjCt7rpLn4znAPucAvhJtwmeffYbc3FwMGTJE/PdOnTohMjIS8+fPx6hRo8r9OIGBgVi2bBn69OmD/v37Y/369WjdurXY9tZbb8VHH32Eu+++G+vWrUPXrl1RWlqK3bt346OPPsLq1avRrl27iz5eSUmJ896LZWVlOHDgAGbNmoWysjJMmTLF2e7RRx/FBx98gP79+2PixImoXbs25s2bh9TUVHzyySfOV53HjRuHN998E2PGjMGOHTsQHx+Pjz/+GBs2bMD06dOdn2kKDAxEy5Yt8eGHH6J58+aoXbs2Wrdurd1X8g7z5s3DG2+8gWHDhqFJkybIzc3FW2+9hbCwsEr/i6RuzGVkZCAnJ0c8gSQmJuKLL77Ayy+/jNjYWCQkJKBjx4549tlnsWbNGnTr1g333nsvatasiTfffBOFhYWYNm2aYTtFRUXo27cvrr/+euzZswdvvPEGunXrpj2ekHfi+Of4J72VK1di9+7dKCkpQXp6Or788kusWbMGcXFx+OyzzxAQEHDRn+/ZsyfGjx+PqVOn4vvvv0e/fv3g5+eHvXv3YtGiRXj11VcxcuRIU/PwzjvvxMmTJ9GnTx80aNAABw8exOuvv46rr77amfFCZBXPAV54Dqi6YHDPMXjwYBUQEKDy8/O1bcaMGaP8/PxUZmamM6L+xRdfNLTDBRHz59/i6pzMzEzVsmVLFRMTo/bu3auUMt7iSqmzt4964YUXVKtWrZTD4VC1atVSiYmJ6qmnnlLZ2dkX3SfpFldhYWGqb9++6osvvjC0379/vxo5cqSKiIhQAQEBqkOHDmrZsmWGdunp6er2229XdevWVf7+/uqqq65Sc+bMMbTbuHGjSkxMVP7+/rzdlQezcnuHnTt3qhtvvFE1atRIORwOFRUVpQYNGqS2b9/ubGNl7uhu75CSkiI+vjTmHnroIdWyZUux/e7du1WPHj1UYGCgAuByq4edO3eq5ORkFRISooKCglTv3r3Vxo0bXX7+3O0d1q9fr8aNG6dq1aqlQkJC1M0336xOnDhxqaeLPADHP8c/Vcy5cXLuy9/fX8XExKhrrrlGvfrqq85bY54jXTOdb/bs2SoxMVEFBgaq0NBQddVVV6mHH35YHT16VCllbh5+/PHHql+/fioqKkr5+/urRo0aqfHjx6tjx45VzpNAHovnAHufA3yUMvEpdSIiL9SyZUsMGjRI/OtpRc2dOxe33347tm3bdsl3hRBVBY5/IiL74jmgYvh2biKypaKiIowaNcqZsE9kJxz/RET2xXNAxXERTUS25O/v7/LZfyI74fgnIrIvngMqjuncRERERERERCbxM9FEREREREREJvGVaCIiIiIiIiKTuIgmIiIiIiIiMqnSgsVmzJiBF198EWlpaWjbti1ef/11dOjQ4ZI/V1ZWhqNHjyI0NBQ+Pj6V1T0iLaUUcnNzERsbixo1yvd3pvKOf4BzgKqWO8Y/wHMAeS6eA8jOOP7JziyN/8q4+fTChQuVv7+/euedd9TPP/+s7rrrLhUREaHS09Mv+bOHDh1SAPjFryr/OnTo0GUf/5wD/KouX+Ud/xWdAxz//KouXzwH8MvOXxz//LLzl5nxXynBYh07dkT79u3xn//8B8DZvyo1bNgQ9913Hx599NGL/mx2djYiIiLc3SUiy7KyshAeHm755yoy/gH7zIFmzZqJ9Zdeekmsf/rpp2L9xx9/NNSKiorEtsXFxWK9ZcuWYn3QoEGGWmpqqtj2tddeE+vZ2dlivbor7/gHeA64UN26dQ21m2++WWz7wQcfiPXjx4+7tU9mXHXVVYZa8+bNxbZLly4V6yUlJW7t0+XEc4B1jRo1EuvdunUz1AYOHCi2PXnypFj/8MMPxfoPP/xgqOnG6ZAhQ8R6z549xfrp06dN92Pu3Lli3VNx/FNMTIyhlpaWVgU9ufzMjH+3v527qKgIO3bswOTJk521GjVqICkpCZs2bTK0LywsRGFhofP73Nxcd3eJqFzK8zYiq+MfqD5zQNrfSvgbm5Ovr69YDw4OFuv+/v6mt6PbdllZmVj38/MT60FBQYZaQECA2Nbb3nZW3v3hOcBIekuYw+Ew3baqSPNIN1e8bfwD9jsHuINu/ErHb+n4CsgLVwCoWVO+ZJV+T7pzgO74HRISItal/dGdi7wNxz9Vp/PR5WZm/Lv92cnMzERpaSmio6Nd6tHR0eJfL6ZOnYrw8HDnV8OGDd3dJaLLxur4BzgHyLvwHEB2xnMA2RnHP9lJlf+JYfLkycjOznZ+HTp0qKq7RHRZcQ6QnXH8k91xDpCdcfyTp3L727nr1q0LX19fpKenu9TT09PF99Y7HA7t29qIPI3V8Q9U3hzQvRVF9xZtK2/dvvrqq8X6DTfcINZHjBhhqJWWloptdW/nfu6558R6nTp1xLo7/Pbbb4Za27Ztxbbnv33tfBeOhXNWr15tqOk+D75r1y5dF6sdO58DdG8JlT6Heeutt4ptR40aJdYzMzPFuvT5f10mQGhoqFjXPf8NGjQw1HSffdbN50WLFol1b1WdzgHu0L9/f7H+wAMPiHXdW7Glt0CfOXNGbBsfHy/WFy5cKNYvfNUTAA4cOCC21X1G/9ixY2JdyrQYOXKk2Pb+++8X62vXrhXrEydOFOuezNvGv1W633WtWrUMtRMnToht77rrLrGuG9NWxMbGivV169aJ9cDAQEPt4MGDYtu//e1vYj0/P99k7zyP21+J9vf3R2JiostAKisrw9q1a9G5c2d3PxxRtcLxT3bHOUB2xvFPdsbxT3ZSKfeJfvDBBzF69Gi0a9cOHTp0wPTp05Gfn4/bb7+9Mh6OqFrh+Ce74xwgO+P4Jzvj+Ce7qJRF9KhRo5CRkYEnnngCaWlpuPrqq7Fq1SrxLTdE3objn+yOc4DsjOOf7Izjn+yiUhbRADBhwgRMmDChsjZPVK1x/JPdcQ6QnXH8k51x/JMdVNoimoiqltV7PIeFhRlq7777rti2TZs2Yl13T0Hpvo+6UJmTJ0+KdV1wkXTf2vDwcLGtLuBCd/9oK8/htm3bxLruvqRdunQx1JYtWya2/eabb8S6LpyKqkZeXp5Yl8KJdEF0//d//yfWW7RoIdalV3d0IT2nTp0S67p+r1mzxlBbsWKF2FYXqkaeo0mTJobaTTfdJLb98ccfxbru3s/SuUF33NWlM1u5f7Bu27q6NEcBOYisuLhYbKu7D3L9+vXFuhQk+dBDD4ltyTPo7k8eGRlpqEnBjQDw008/iXXd+P/kk08MtVtuucVS/3TXY1lZWYaadK0IeHeAmE6V3+KKiIiIiIiIyFNwEU1ERERERERkEhfRRERERERERCZxEU1ERERERERkEhfRRERERERERCYxndtD+Pj4iHUr6cGhoaFivVu3bmJ95cqVpret658uCVBKvHQXXV8kVhOsvdnixYsNtbi4OLHt8ePHxbou+bRmTeOhRjcGdL8/aRu69pmZmWJb3XjU0aWNW3H69GmxLqVh6sZjjx49xLousXn37t0me0eXg7+/v6EmpZ4CwH/+8x+xPnHiRLFeWFhoqOnSuXWPuWPHDrE+Z84cQy0hIUFsm5GRIdbJc/z973831Kz+XnXHTOkuBbpzgK6empoq1qVkbd1dEXTnKN2ckejuFKE7Rx08eFCst27d2lAbOHCg2Hb58uUme0dV6cSJE2JdOm7q2tauXVusx8TEiPX77rvPUGvbtq3YVndnFd2dG6Qxreu3HfGVaCIiIiIiIiKTuIgmIiIiIiIiMomLaCIiIiIiIiKTuIgmIiIiIiIiMomLaCIiIiIiIiKTmM7tIXSJl1JKZNOmTcW2d955p1jXpQfn5+cbalKiMABs3bpVrFtJ4dalMuv2XdfeymNKac1KKW2CpzdITEwU61ISty7lWpdCqku/lpJS69evL7YNCgoS67pxUFxcbLp/ulRV3Vjy8/Mz1HTjKzc3V6wfPnxYrFsZp7p+6+b0Qw89ZHrbVPny8vIMtbp164ptdUm+Dz74oFhv0KCBoRYZGSm21aUb69JWpT5aScknzzJ37lxD7YEHHhDb6lK709PTxbp0dxDp2H0xRUVFYl03lyQ5OTliXXcdZIWuf+Hh4WL90KFDhhpTuD3b77//LtY7depkqOmuAaQ7LgDWjrEHDhwQ6927dxfrR44cEeuBgYGGmu4azY74SjQRERERERGRSVxEExEREREREZnERTQRERERERGRSVxEExEREREREZnERTQRERERERGRSUzn9hC61GMptbdPnz5i26SkJLGuSw92OByGmi6V75prrhHr//3vf8W6lOCplBLb6pKJdUJCQgw1Xdp2QUGBpW17g969e4t16fct1QD986kbp1La5COPPCK2PXr0qFjXjdPY2FhD7dixY2JbXcK3LlVV2n9pfAHAX//6V7F+3333iXUp+VyXfKx7vkeOHCnWmc5dvVhJYreSNAzI4ygtLU1sqzt+65LypWOv7jitq5PnkO6ysWnTJrHtkCFDxPqWLVvEunRs041HXVq87jgtzQHdnUR0j6k79kpp3rr0ex3dYz766KOWtkPV3y+//CLWdddGEunOOIB+/Ldp08b0tnUp9Lrkb2le6BLu7YivRBMRERERERGZxEU0ERERERERkUlcRBMRERERERGZxEU0ERERERERkUkMFvMQukABSfv27cV6fHy8WNcFHkghTKtXrxbb/uUvfxHr06ZNE+vbt2831H766Sex7a+//irWO3ToINal/d+4caPYVgpNUUp5dXCCLoxKCj+yEmgHAAEBAWI9OzvbUHvrrbfEtv369RPruuCuOXPmGGrjx48X2+7atUus165dW6xL+y+F4gHAK6+8ItbvvfdesS4FduieP10AXosWLcR68+bNDbXffvtNbEuVTzqWWg1S1M3FiIiIcvfrUqSwGV2/dcFM5Nlee+01sX7//feL9T/++EOsZ2RkGGq6ACXd8S43N1esS3TzRfeYuvHr5+dnuh/h4eFifeXKlWLdm68z7OrIkSNivbi42FDTBZ1KYw7QB6bu3LnTUNONUV3/dPNFOgdI13N2xVeiiYiIiIiIiEziIpqIiIiIiIjIJC6iiYiIiIiIiEziIpqIiIiIiIjIJC6iiYiIiIiIiExinGY1IyXhAfpE1GuuucZQa9eundhWl9YXHBws1qWEX6kGANu2bRPr+/btE+shISGGWufOncW2w4cPF+tS2qGuL3feeafYtrCw0FArKSnBN998I7b3Bm3bthXrhw4dMtR06ZEOh8PSY4aFhZluu2rVKrGuS1Vt2bKlofbQQw+JbZcsWSLWBw8eLNalxFYpCRMAEhMTxbqUeg7I806XzFxWVibWdUm40lxiOnfVkY53ujl05swZsa5LT5XGhpWk1YuR5r/umKBLlifPIR3vdMevbt26ifXnnnvO9OPpUrh1jxkYGCjWT58+bajp0rZ1delaANCPdyttP//8c9PbIM929OhRsS5dr+qOx7rzve7c8MsvvxhquoRv3RjVJW5L5ymr5xFvxleiiYiIiIiIiEziIpqIiIiIiIjIJC6iiYiIiIiIiEziIpqIiIiIiIjIJC6iiYiIiIiIiExiOvdlUJlJds8884yhVq9ePUvbCAoKEutSQmZRUZHYVpfUqUsKl9IHdanHuoRvXYJnSkqKoda4cWOx7ciRI8W6N2jdurVYz8jIEOvS82k15VeXnnrixAmxLtH1W5eeKo13XUKsrt+6pHepvS5FXkeX1lm/fn1DzWo6t5RKCwDdu3c31ObNm6frIlUyKRFYNxZ1dV2qqtTeHdsA5GOCbhu6YwV5Dt05VXLs2DGxvn//frGekJBgqOnShnV3ErGSWqwbp3l5eWI9MjJSrFuZAwcPHhTrZB+ZmZliPT4+3lDbvXu32FY3L3THaV3ivER3Da/btnRNortesiO+Ek1ERERERERkEhfRRERERERERCZxEU1ERERERERkEhfRRERERERERCZxEU1ERERERERkEtO5LwOlVKVt+9SpU4aaLp1bl+TrcDjEupT4FxISIrbVpQnq0pqllE0pURgAunTpItZ1CZlRUVGG2qpVq8S23uyRRx4R67rfiZRaqkuL1m1DNw6khFNdcnudOnXEeu3atcW6n5+foRYdHS221aVK6vrt7+9vqEVERIhtR40aJdZr1aol1qX5GB4ebrqtrn+A/rmlqiEdqwoKCsS2upRrK8naunmrY+UcpUvJJwL04zQ0NNRQ06Vt665JcnJyxLp0HNQd03XpxDpWEsuPHz9uadvkfdLS0ky31c0V6ZrmYu0lumO6btu6ayNpHSCtO+yKr0QTERERERERmcRFNBEREREREZFJXEQTERERERERmcRFNBEREREREZFJloPFvv76a7z44ovYsWMHjh07hiVLlmDo0KHOf1dKYcqUKXjrrbeQlZWFrl27YubMmWjWrJk7+03/X1BQkKGmCx/Q1XUBN9nZ2YbaiRMnxLbx8fFiXRduIIXh6Pon7SOgD8+RwkoaNmwotrXKk8b/xo0bxXpMTIxYb9q0qaEWFhYmtg0ODhbre/fuFevS72rz5s1iW13YjK4ubVsXziSFZADyeNRtWzdOc3Nzxfpvv/0m1qVxbTVU6ujRo2L9008/FesV5UnjvzqxEgijGwO68S+1t/J4FyPNF12wmBTo6I3sNgd0Y0k3Hg8fPizW27RpY3rbujFmJSxJd30QEBAg1nXhjVJAWd26dcW2R44cEes60vyyEmRWFew2/t3FSiCj1TBiqb2V66WL1aVrI13Anx1ZPtPm5+ejbdu2mDFjhvjv06ZNw2uvvYZZs2Zhy5YtCA4ORnJysjYpkciTcPyTnXH8k91xDpCdcfwT/cnyK9H9+/dH//79xX9TSmH69Ol47LHHcO211wIA3n33XURHR+PTTz/FDTfcULHeElUxjn+yM45/sjvOAbIzjn+iP7n1M9GpqalIS0tDUlKSsxYeHo6OHTti06ZN4s8UFhYiJyfH5YvIE5Vn/AOcA+QdOP7J7jgHyM44/slu3LqIPneT8ejoaJd6dHS09gbkU6dORXh4uPPLXZ9dJbrcyjP+Ac4B8g4c/2R3nANkZxz/ZDdVns49efJkZGdnO78OHTpU1V0iuqw4B8jOOP7J7jgHyM44/slTWf5M9MWcS/1NT09HvXr1nPX09HRcffXV4s84HA44HA53dqPasZJErUvICwkJEeuxsbGGmi4FUFfXPf9FRUWGmi7JOyIiQqzr0rylZGJ/f3+xrS71ODw8XKz/+OOPhpru+WvXrp2hVlpaiu+++05sfzHlGf9A5c2BmTNnWqrXqlXLUNMlat5zzz1ivWfPnmL95MmThtquXbvEtllZWWJdSmAF9GnG7mBl7uqCU6yM05tvvtlC76qX6jb+q4I0hwB5jOoS4XXJrO5K3Jboklyl9GDdONcl9uvSkL0xaIhzADhw4IBYl8av7pyvm0e6bUuJ1nXq1BHbnjp1yvQ2APm6STcXq3uydmXj+NfTHWOtsHK3G935RcfK+Sg/P9/Str2ZW8/KCQkJiImJwdq1a521nJwcbNmyBZ07d3bnQxFVOxz/ZGcc/2R3nANkZxz/ZDeWX4nOy8vDvn37nN+npqbi+++/R+3atdGoUSNMmjQJzz77LJo1a4aEhAQ8/vjjiI2NdbmPHJGn4vgnO+P4J7vjHCA74/gn+pPlRfT27dvRu3dv5/cPPvggAGD06NGYO3cuHn74YeTn52PcuHHIyspCt27dsGrVKu1buog8Ccc/2RnHP9kd5wDZGcc/0Z8sL6J79eqlfV8+cPZ99U8//TSefvrpCnWMqDri+Cc74/gnu+McIDvj+Cf6U5WncxMRERERERF5Cremc5NM+qudLlFYl849atQosX4uDfF8GRkZYtvAwECxrksNlNJWdffvk5K8AX3yd3FxsaEmJcEC+n7r0jdnzJhhqOmSIXWPaUdSaunWrVvFtrqk9z59+oh1aQ7okll1Kb+6OWMl9VKXQKmrS9u2kmYP6NOJN27cKNbJc1m5M8LFXs2xwsp2dOPcSvK3bh5mZ2eLdW9M4Sa906dPi3Urx2ldW93Yk46xum3o0rnr1q0r1kNDQ8W6RHcHCSJ33F1Bd/y2cocSXT905xFpTRIVFWX68bwdX4kmIiIiIiIiMomLaCIiIiIiIiKTuIgmIiIiIiIiMomLaCIiIiIiIiKTuIgmIiIiIiIiMonRxJeBlACtS/LV2bVrl1iXUl91CZFWE8GlBD5d0uqJEyfEuq4vUpqmLpVZl6Z5+PBhsX7TTTcZai+++KLYdvPmzWLdm+kSHqXflW6c6pIcc3JyxLo09nTjzmpqsbQ/7ko+tsJKQiYAZGVlVXjbugTaqth/0j/vVsdGdSHtjy6dnryTlVRtACgpKRHr0l1DdOcX3TlfR2qv27bubh/Hjx8X65GRkYZaXl6ehd4R6a+7rLS1cncF3TzUbUN3pxppO/Hx8WJbO+Ir0UREREREREQmcRFNREREREREZBIX0UREREREREQmcRFNREREREREZJLHB4vpPiSvC3KRPoCv205xcbHY1l1BG1asWLFCrOfn5xtqp0+fFtv6+/uLdV0YjhQEontepaAwQP8cWmmre751fWnTpo2hlp2dbbof3k73+7byu9q/f79Y1wWLuSNcT9dvdwSLWQn90PVbF6Kno3uuJLrjli6cjaqGlQAx3XFN97u2ojK3bXUs6tpbPY9S1bD6+wsNDRXrtWrVMtQKCgrEtrVr1zbZu7MyMzMNtaCgILFteHi4WLdyPtKdL+Li4kxvA3DPtSF5BivXGFbWKe54PMBa8DCDxf7EV6KJiIiIiIiITOIimoiIiIiIiMgkLqKJiIiIiIiITOIimoiIiIiIiMgkLqKJiIiIiIiITPKodG4pPU6XCFqdUg979OhhqI0YMUJs27VrV7GuS7E8ceKEoaZL4ZYSkgH9cyg9pi7Bz+FwiHVdareUnqzbRx3dfubl5Rlqw4cPF9t+/vnnlh7Tm0mJkLqxoUuA1yWcSuNDN0d141SXNimNJV1bXV2Xhiltu7CwUGyrS4PVPWZ1OkaRe1g53llJmwespV9bSQm/GCvJ97q67jh95syZ8neMLhurKerSXT0AYNeuXYbaoUOHxLa6Y6luzERHRxtqunPRgQMHLG1bSvM+duyY2DY2Nlask300b95crEvHQd3c0l0D6UjnAKvXQFauU+rWrWuhd96Nr0QTERERERERmcRFNBEREREREZFJXEQTERERERERmcRFNBEREREREZFJXEQTERERERERmeRR6dy6dFIrateuLdalVMVmzZqZbgvoE6CltD5dwq8uJViXXF2nTh1D7ejRo2JbXfqkLj01KirKUNMlXurSNDdu3CjWQ0JCDDUpxRzQJxhmZ2eL9eLiYkOtU6dOYlv6ky5dV6L7nejmqJV0Yt0csNIXq+nEVlK7df3WPSdWkr91rLSlqmMl+dRK2nx5HrOyWH08q/OZPFv37t3F+u+//26oHTx4UGyru1bJyckR62FhYYaalKoNWL+zRL169cS6JCYmRqxL11IAcPz4cUNNN1+spqRT1bjyyivF+uHDhw016VoVAPz8/Cw9pnS9467jtLRWkdLwAaBLly5iXbcO8AY8uxERERERERGZxEU0ERERERERkUlcRBMRERERERGZxEU0ERERERERkUlcRBMRERERERGZ5FHp3FLC8jPPPCO2jYyMFOsRERFiXUoV1iX8ZmVlifWSkhKxnpuba6jpkiB1iXq6REkp9e76668X227fvl2sh4aGinUplS8+Pl5sq3PVVVeZfsxDhw6JbXXJ5IGBgWJdSv6Oi4vTdZHcqH79+mL91KlThppufllN7b7c6cS6fuiSNnX9s5ogTtVfVfxOpflidU5YSQrX7aOuXrOmR11m2Jp0bNOlQjds2FCst2zZUqxL6dy667G6deuK9X379on14OBgQy0hIUFsq7t+kxK+rcrLyxPrN910k1ifPn26ocYUbs/Wt29fsS4dS61e01TmHT2sXI/t379fbHvPPfeIdaZzExEREREREREX0URERERERERmcRFNREREREREZBIX0UREREREREQmVdvEjxo1ahg+XP/aa68Z2tWrV0/8eSko7GJ1XXiVxN/f39K2daFgkvDwcLGuC8b617/+ZfrxdB/6P3r0qFg/c+aMobZ27VqxrRQaAgDNmjUT63Xq1DHUdGFrfn5+Yt1KwFNGRobYlv5kNYhCogvXk1idR7qwDaluNZhD114KedGNRymI72KPqduOlW1Q9aIbR9KYtjoWdcc7idXxYjXMT6Lrt+6clpOTY3rbdHlYCbVKTk4W67/88otYDwgIMNR0Y0AXYHrkyBGx3qJFC0NNty+HDx8W623atBHr6enphpp0/QLIAZqAPnCzadOmhpouPI08gxSADMjXpbowL6vXL+4Ib9Qd66V5K60NAKBz584V7oen4SvRRERERERERCZxEU1ERERERERkEhfRRERERERERCZxEU1ERERERERkEhfRRERERERERCZV23TuG2+80ZDeKyVU79+/X/z5kJAQS/XatWub7psuVVeXQnro0CFDTZeIHRQUJNalhEgAmDdvnqE2dOhQse3nn38u1nVJmNJzlZiYKLbt3bu3WNcl/klJ3A6HQ2yrS3HWkZJwdb+zhg0bGmplZWXaFFC6OF1CtZRCqUvy1iVW6tJWpcRK3TZ0CfBWUi91ba0k/ANARESEpfZU/Vm5k4AugVXHamJrZdGl5+v6oTuuk2fTpVn/+OOPYl06JuvO7VbHjO54L9GdR3R1KYlYum4A9GnjVlLImc7t2XTX01Jyu+762OoxXRr/7jovSNvWrVNiYmLEum4+664XPQlfiSYiIiIiIiIyiYtoIiIiIiIiIpO4iCYiIiIiIiIyiYtoIiIiIiIiIpMsLaKnTp2K9u3bIzQ0FFFRURg6dCj27Nnj0ubMmTNISUlBnTp1EBISghEjRmhDsYg8DecA2RnHP9kZxz/ZHecA0Z8spXOvX78eKSkpaN++PUpKSvDPf/4T/fr1wy+//ILg4GAAwAMPPIDly5dj0aJFCA8Px4QJEzB8+HBs2LDBUscyMjIMSadSynVoaKj487rUN2kbgJxErUuODAsLE+snT54U6wcPHjT1eABw+vRpsS4lRAJywvGSJUvEtj/99JNY16UJSonlunTjrKwssV5cXCzWpX7r0jF1ibe69lKKre532bx5c7FvunTuyzkHPJHud2KFO1KIdamXVhORpce02j9dCnlgYGCF+lEVOP4vTkpzB+Qxo0sUri6/a0A/diW6Y71uLnoiO45/3fXBsWPHxHpAQIBYz8vLM9R088Udx0zdNnTnKCuJ4Lo7MURHR4t13fVEZGSk6cesLuw4ByS1atUS63Xr1hXr0h8RdHNFdw6wcu2hu4uC1Wsj6dr5f//7n9j2uuuuE+u6u/ps3LhRrHsSS4voVatWuXw/d+5cREVFYceOHejRoweys7Px9ttvY8GCBejTpw8AYM6cObjyyiuxefNmdOrUyX09J6oCnANkZxz/ZGcc/2R3nANEf6rQn4izs7MB/PmK5Y4dO1BcXIykpCRnmxYtWqBRo0bYtGlTRR6KqFriHCA74/gnO+P4J7vjHCA7s/RK9PnKysowadIkdO3aFa1btwYApKWlwd/fHxERES5to6OjkZaWJm6nsLDQ5a3XupvSE1U3nANkZxz/ZGfuGv8A5wB5Jp4DyO7K/Up0SkoKdu3ahYULF1aoA1OnTkV4eLjzq2HDhhXaHtHlwjlAdsbxT3bmrvEPcA6QZ+I5gOyuXIvoCRMmYNmyZVi3bh0aNGjgrMfExKCoqMgQMpWeno6YmBhxW5MnT0Z2drbzSxf8RVSdcA6QnXH8k525c/wDnAPkeXgOILL4dm6lFO677z4sWbIEX331FRISElz+PTExEX5+fli7di1GjBgBANizZw/++OMPdO7cWdymw+EQUxGPHTtmSC+VEugOHz4sbvdcSuCFdMl5Urp0Zmam2DYjI0Os65Impf3TJU7r0vp0KeRS0p6u31deeaVYz8/PF+vSgezUqVNiW12ypa4vUpKrLk1Tl/qqS+qUDtTnPrdzoauvvtpQKywsxPr168X2l3MOeCJ3JPG6I524MtO5ddu2ms4dFBRkqS/VAcf/xenuAiDRjRddenB1SbnW9Vt3nPbEca5TGeMfqN5zoFGjRmJdN05110HS3NBd7+iShXXblujSk3XHY922pXpqaqrYtlmzZmJdd3un8PBwQ026Kwqgv/vL5cZzwFnStSOgv8aQxrTVO33ozgHSPNKdi3Tz1sr1yxVXXCG21c0h3drDduncKSkpWLBgAZYuXYrQ0FDn5xvCw8MRGBiI8PBw3HHHHXjwwQdRu3ZthIWF4b777kPnzp2ZyEdegXOA7Izjn+yM45/sjnOA6E+WFtEzZ84EAPTq1culPmfOHIwZMwYA8Morr6BGjRoYMWIECgsLkZycjDfeeMMtnSWqapwDZGcc/2RnHP9kd5wDRH+y/HbuSwkICMCMGTMwY8aMcneKqLriHCA74/gnO+P4J7vjHCD6U/X4kBURERERERGRByj3faIr208//WSoLV682FAbO3as+PNHjx4V67///rtYP3PmjKEWEhIittWFgumCrqQP+F8YmnbO+ffKO58uaEP6q2BBQYHY9tixY6a3oXtMXXCA9PwB+uewqKjIUJPC3S5W1wXZSEEIF4ZfnCMFfkh9swN3BHrp6Ma7Fbr+WQkLs9oPK8+JLvRDN3fd8ZxQ9aILc5HGkS7gyGr4XWWSxrRuPOuOx02bNhXr33//fbn7RZeP7jilO97prj+kgDndtZTuHGwlFEl37aGbd7prr/r16xtq27dvF9v26NFDrOuuvaTrKV0gWnUJFqOzBg8eLNathOnqxrOurhvT0jnDanix7t7cUr91Keu6uXXVVVeJdW/AV6KJiIiIiIiITOIimoiIiIiIiMgkLqKJiIiIiIiITOIimoiIiIiIiMgkLqKJiIiIiIiITKq26dySqVOnGmq6hM+HHnpIrMfHx4t1KVFPlwqdn58v1nUpllJiqy7lWrcNXWKrlEqpS+XT1XWJslJ7q8mxuvZSKrYuebB27dpiXZdgKCUH/vjjj2Lb999/X6zbkfS7sprYrUtVlZJZrdL9vqU5YzX5uDKTyd2Rzl2Z/SP3iY2NNd1Wl26s+11bGf9Wx4uuL9Jj6uaQbs7p0mrJM9StW1es664bMjIyxHrr1q0NNatJwbrHlMZeaGiopW3o7jDSpk0bQ2358uViW901o+4xpSRu3bUhVS9NmjQR67pxJ12X6o67uiR2XSq2lBS+bNkyse3p06fFuu4aLTc3V6xLgoODxXqrVq1Mb8PT8JVoIiIiIiIiIpO4iCYiIiIiIiIyiYtoIiIiIiIiIpO4iCYiIiIiIiIyiYtoIiIiIiIiIpOqbQygj4+PIQVUSgpduXKl+PO6eu/evcW6lPwdFxcntg0PDxfruqQ9KT1Vl8CoS/LVOX78uKGmS2Y9cuSIWC8sLBTreXl5hpqVROGL9aW4uNhQKygoENvqntc1a9aI9V9//dVQ27hxo66LdBlIv0PdWNel/+rGgVS3kjZ8sceU6Ma07jF1rM4lqv50Cb/SnQ5048jqHRqkeWR1bEnHY912dHNId3eFgwcPWuoLVS+6dG7d8e7EiRNiXbpu0l0HHTt2TKzrUq5PnTplqOnuomL1OC2Rro10/QD0c0bqY7169cS2e/bsMdk7uhx06de9evUyvQ3duAgMDLTUF914lOjuoqC7s4pEd+2mO//99NNPprftafhKNBEREREREZFJXEQTERERERERmcRFNBEREREREZFJXEQTERERERERmcRFNBEREREREZFJ1TadWymlTS+tiHXr1on1Tp06md5GixYtxLouxTIrK8tQa9Cggdj2wIEDYl2Xnrp//36xTmSVO+bb0aNHxXrz5s0NNV1KpC6xUleXko+tbkO371IKpS5RVsdqCrOVbVD1snXrVrEujf+IiAix7enTpy09ppTarZtb7hhHuvRgXWLrb7/9VuHHpKqjS13X3U2jVq1aprcdEBAg1nVJwbpjb2RkpKGWkZEhtg0ODja9DUC+rmvSpInYVnd+sXK3iNDQULEtVS9vvfWWWJ89e7ZYl47TmZmZYlvdONKx0l73mLq7DklrD90YDQsLE+uvvvqqyd55Hr4STURERERERGQSF9FEREREREREJnERTURERERERGQSF9FEREREREREJlXbYLHqbPfu3RXexq5du9zQE6LqRReWJIW56EJidAF9unAWqS6FjZWHFJakCwQ7dOiQWA8KChLrunAaiZVgGqo6urCld99911Dr3bu32FY3/nWBSNJ41AWL6ejGlzT+U1NTxba60E7dc0KeoVmzZmJdNw50YWES3bjTHTPPnDkj1jdu3Gio3XTTTWJb3Xln7dq1Yl3qo67fuvNffn6+WJeeQ908Is9w1VVXifWffvrJ9DYKCwstPWZUVJTpttHR0WI9MDBQrEvzRRcslpycLNYPHjxosneeh69EExEREREREZnERTQRERERERGRSVxEExEREREREZnERTQRERERERGRSVxEExEREREREZnko5RSVd2J8+Xk5CA8PLyqu0GE7OxshIWFXfbHrao54OPjY6hZPTy8+OKLYt3hcBhqWVlZYlurydpSUmpeXp7YVrc/0r4DcsqxLhG7qKhIrNeqVUusb9261VBbtmyZ2LYq2G38u4NuHLnjNFu7dm2xHhMTY6hZ/b2lpaWZrusSknUq8zmpbJwD+jRrXQK8lTsJ6O5QoEvzbdCggVg/cOCAWKeK4fivXN26dRPrLVu2FOt9+vQR6w888IChduzYMbGt7hpNl/C9cOFCQ23lypViW29jZvzzlWgiIiIiIiIik7iIJiIiIiIiIjKJi2giIiIiIiIik7iIJiIiIiIiIjJJToyoQp4QNkL2UFVj0ZMfVxc6JIXK6NqWlpZaekwpyKawsFBsWxXBYrr9LC4uFuvVhSePw6pSmX3XbVuaL7rQJx3dmHbH/vD36TmPK7HaFyvtrY47XXuqHBz/lUt3nNZdSxQUFIh1K/NCdz1y+vRpsV7dr1Mqk5lxWO3SuQ8fPoyGDRtWdTeIcOjQIW0aaGXiHKDqgOOf7I5zgOyM45/szMz4r3aL6LKyMhw9ehShoaHIzc1Fw4YNcejQoSqJ2b8ccnJyvH4fAc/aT6UUcnNzERsbq71dR2XiHPA+nrSPHP+XlyeNjYrwpP2sLnNAKYVGjRp5xHNWEZ40NsrLk/axuox/ngO8hyfto5XxX+3ezl2jRg3nyv/cWyzDwsKq/ZNeUXbYR8Bz9rMq71HIOeC9PGUfOf4vPzvsI+A5+1kd5kBOTg4Az3nOKsoO++kp+1gdxj/Ac4C38ZR9NDv+GSxGREREREREZBIX0UREREREREQmVetFtMPhwJQpU+BwOKq6K5XGDvsI2Gc/3c0Ozxv3kXTs8LzZYR8B++ynO9nlObPDftphHyuDHZ437qPnqnbBYkRERERERETVVbV+JZqIiIiIiIioOuEimoiIiIiIiMgkLqKJiIiIiIiITOIimoiIiIiIiMikar2InjFjBuLj4xEQEICOHTti69atVd2lcvv6668xePBgxMbGwsfHB59++qnLvyul8MQTT6BevXoIDAxEUlIS9u7dWzWdLaepU6eiffv2CA0NRVRUFIYOHYo9e/a4tDlz5gxSUlJQp04dhISEYMSIEUhPT6+iHldv3jT+Ae+fAxz/7udNc8Dbxz/AOeBuHP8c/3bmTeMf4Bw4x5vmQLVdRH/44Yd48MEHMWXKFOzcuRNt27ZFcnIyjh8/XtVdK5f8/Hy0bdsWM2bMEP992rRpeO211zBr1ixs2bIFwcHBSE5OxpkzZy5zT8tv/fr1SElJwebNm7FmzRoUFxejX79+yM/Pd7Z54IEH8Pnnn2PRokVYv349jh49iuHDh1dhr6snbxv/gPfPAY5/9/K2OeDt4x/gHHAnjn+OfzvztvEPcA6c41VzQFVTHTp0UCkpKc7vS0tLVWxsrJo6dWoV9so9AKglS5Y4vy8rK1MxMTHqxRdfdNaysrKUw+FQH3zwQRX00D2OHz+uAKj169crpc7uk5+fn1q0aJGzza+//qoAqE2bNlVVN6slbx7/StljDnD8V4w3zwE7jH+lOAcqguOf49/OvHn8K8U54C1zoFq+El1UVIQdO3YgKSnJWatRowaSkpKwadOmKuxZ5UhNTUVaWprL/oaHh6Njx44evb/Z2dkAgNq1awMAduzYgeLiYpf9bNGiBRo1auTR++ludhv/gHfOAY7/8rPbHPDG8Q9wDpQXxz/Hv53ZbfwDnAOeOgeq5SI6MzMTpaWliI6OdqlHR0cjLS2tinpVec7tkzftb1lZGSZNmoSuXbuidevWAM7up7+/PyIiIlzaevJ+Vga7jX/A++YAx3/F2G0OeNv4BzgHKoLjH87vPXV/Of7Lz27jH+Ac8NT9rFnVHSDvlJKSgl27duHbb7+t6q4QXXYc/2R3nANkZxz/ZHd2mAPV8pXounXrwtfX15DWlp6ejpiYmCrqVeU5t0/esr8TJkzAsmXLsG7dOjRo0MBZj4mJQVFREbKyslzae+p+Vha7jX/Au+YAx3/F2W0OeNP4BzgHKorjH87vPXF/Of4rxm7jH+Ac8NT9rJaLaH9/fyQmJmLt2rXOWllZGdauXYvOnTtXYc8qR0JCAmJiYlz2NycnB1u2bPGo/VVKYcKECViyZAm+/PJLJCQkuPx7YmIi/Pz8XPZzz549+OOPPzxqPyub3cY/4B1zgOPffew2B7xh/AOcA+7C8c/xb2d2G/8A54DHzoEqjTW7iIULFyqHw6Hmzp2rfvnlFzVu3DgVERGh0tLSqrpr5ZKbm6u+++479d133ykA6uWXX1bfffedOnjwoFJKqX/9618qIiJCLV26VP3444/q2muvVQkJCer06dNV3HPz7rnnHhUeHq6++uordezYMedXQUGBs83dd9+tGjVqpL788ku1fft21blzZ9W5c+cq7HX15G3jXynvnwMc/+7lbXPA28e/UpwD7sTxz/FvZ942/pXiHDjHm+ZAtV1EK6XU66+/rho1aqT8/f1Vhw4d1ObNm6u6S+W2bt06BcDwNXr0aKXU2Xj7xx9/XEVHRyuHw6H69u2r9uzZU7WdtkjaPwBqzpw5zjanT59W9957r6pVq5YKCgpSw4YNU8eOHau6Tldj3jT+lfL+OcDx737eNAe8ffwrxTngbhz/HP925k3jXynOgXO8aQ74KKWUe17TJiIiIiIiIvJu1fIz0URERERERETVERfRRERERERERCZxEU1ERERERERkEhfRRERERERERCZxEU1ERERERERkEhfRRERERERERCZxEU1ERERERERkEhfRRERERERERCZxEU1ERERERERkEhfRRERERERERCZxEU1ERERERERkEhfRRERERERERCZxEU1ERERERERkEhfRHsLHxwdPPvmk8/u5c+fCx8cHBw4cqLI+EXmiAwcOwMfHBy+99FJVd4XINJ4DiNyrInNozJgxiI+Pd3ufiC4Xjv+K4yK6kpwbnOe+AgIC0Lx5c0yYMAHp6elV3T2iSvXTTz9h5MiRiIuLQ0BAAOrXr49rrrkGr7/+elV3jeiy4DmAyIjnBrIzjn/vUrOqO+Dtnn76aSQkJODMmTP49ttvMXPmTKxYsQK7du1CUFBQVXePyO02btyI3r17o1GjRrjrrrsQExODQ4cOYfPmzXj11Vdx3333VXUXiS4bngOIzuK5geyM49/7cBFdyfr374927doBAO68807UqVMHL7/8MpYuXYobb7yxintXefLz8xEcHFzV3aAq8NxzzyE8PBzbtm1DRESEy78dP368ajp1mRUUFHCBRAB4DiA6h+cGsjOOf+/Dt3NfZn369AEApKamolevXujVq5ehTUU+a/DGG2+gVatWcDgciI2NRUpKCrKyspz/PmHCBISEhKCgoMDwszfeeCNiYmJQWlrqrK1cuRLdu3dHcHAwQkNDMXDgQPz888+G/oaEhGD//v0YMGAAQkNDcfPNN5er/+T59u/fj1atWhlOEgAQFRXl/H8fHx9MmDABn376KVq3bg2Hw4FWrVph1apVhp87cuQIxo4di+joaGe7d955x6VNUVERnnjiCSQmJiI8PBzBwcHo3r071q1bd8k+K6Uwbtw4+Pv7Y/Hixc76+++/j8TERAQGBqJ27dq44YYbcOjQIZef7dWrF1q3bo0dO3agR48eCAoKwj//+c9LPibZE88BZFdmzw1z5sxBnz59EBUVBYfDgZYtW2LmzJmGn4mPj8egQYPw7bffokOHDggICEDjxo3x7rvvGtr+/PPP6NOnDwIDA9GgQQM8++yzKCsrM7RbunQpBg4ciNjYWDgcDjRp0gTPPPOMy5wgKg+Of+/DRfRltn//fgBAnTp13L7tJ598EikpKYiNjcW///1vjBgxAm+++Sb69euH4uJiAMCoUaOQn5+P5cuXu/xsQUEBPv/8c4wcORK+vr4AgPfeew8DBw5ESEgIXnjhBTz++OP45Zdf0K1bN0MQQUlJCZKTkxEVFYWXXnoJI0aMcPv+kWeIi4vDjh07sGvXrku2/fbbb3HvvffihhtuwLRp03DmzBmMGDECJ06ccLZJT09Hp06d8MUXX2DChAl49dVX0bRpU9xxxx2YPn26s11OTg7++9//olevXnjhhRfw5JNPIiMjA8nJyfj++++1fSgtLcWYMWPw7rvvYsmSJRg+fDiAs381vu2229CsWTO8/PLLmDRpEtauXYsePXq4LEoA4MSJE+jfvz+uvvpqTJ8+Hb1797b0nJF98BxAdmX23DBz5kzExcXhn//8J/7973+jYcOGuPfeezFjxgxD23379mHkyJG45ppr8O9//xu1atXCmDFjXP7Qk5aWht69e+P777/Ho48+ikmTJuHdd9/Fq6++atje3LlzERISggcffBCvvvoqEhMT8cQTT+DRRx+t+BNAtsbx74UUVYo5c+YoAOqLL75QGRkZ6tChQ2rhwoWqTp06KjAwUB0+fFj17NlT9ezZ0/Czo0ePVnFxcS41AGrKlCmG7aempiqllDp+/Ljy9/dX/fr1U6Wlpc52//nPfxQA9c477yillCorK1P169dXI0aMcNn+Rx99pACor7/+WimlVG5uroqIiFB33XWXS7u0tDQVHh7uUh89erQCoB599FGrTxN5of/973/K19dX+fr6qs6dO6uHH35YrV69WhUVFbm0A6D8/f3Vvn37nLUffvhBAVCvv/66s3bHHXeoevXqqczMTJefv+GGG1R4eLgqKChQSilVUlKiCgsLXdqcOnVKRUdHq7FjxzprqampCoB68cUXVXFxsRo1apQKDAxUq1evdrY5cOCA8vX1Vc8995zL9n766SdVs2ZNl3rPnj0VADVr1iyrTxV5MZ4DiFyZPTecO6afLzk5WTVu3NilFhcX5zJmlTo7DxwOh/r73//urE2aNEkBUFu2bHFpFx4e7jKHdI89fvx4FRQUpM6cOeOsSXOU6GI4/r0PX4muZElJSYiMjETDhg1xww03ICQkBEuWLEH9+vXd+jhffPEFioqKMGnSJNSo8eev9a677kJYWJjzVQcfHx9cd911WLFiBfLy8pztPvzwQ9SvXx/dunUDAKxZswZZWVm48cYbkZmZ6fzy9fVFx44dxbfI3nPPPW7dJ/JM11xzDTZt2oQhQ4bghx9+wLRp05CcnIz69evjs88+c2mblJSEJk2aOL9v06YNwsLC8PvvvwM4+zbrTz75BIMHD4ZSymUsJicnIzs7Gzt37gQA+Pr6wt/fHwBQVlaGkydPoqSkBO3atXO2OV9RURGuu+46LFu2DCtWrEC/fv2c/7Z48WKUlZXh+uuvd3nMmJgYNGvWzDD+HQ4Hbr/9dvc8geRVeA4gOsvsuSEwMND5/9nZ2cjMzETPnj3x+++/Izs722WbLVu2RPfu3Z3fR0ZG4oorrnCeQwBgxYoV6NSpEzp06ODSTvrIwfmPnZubi8zMTHTv3h0FBQXYvXt3xZ4AsjWOf+/DYLFKNmPGDDRv3hw1a9ZEdHQ0rrjiCpcLHHc5ePAgAOCKK65wqfv7+6Nx48bOfwfOvp1v+vTp+Oyzz3DTTTchLy8PK1aswPjx4+Hj4wMA2Lt3L4A/P793obCwMJfva9asiQYNGrhtf8iztW/fHosXL0ZRURF++OEHLFmyBK+88gpGjhyJ77//Hi1btgQANGrUyPCztWrVwqlTpwAAGRkZyMrKwuzZszF79mzxsc4P5Jg3bx7+/e9/Y/fu3c63rwJAQkKC4eemTp2KvLw8rFy50vC51L1790IphWbNmomP6efn5/J9/fr1nQt4ovPxHED0JzPnhg0bNmDKlCnYtGmT4bP72dnZCA8Pd35/qXMIcHZudOzY0dDuwrkCnP3s6GOPPYYvv/wSOTk5hscmqgiOf+/CRXQl69ChgzOZ9UI+Pj5QShnqlf0B/k6dOiE+Ph4fffQRbrrpJnz++ec4ffo0Ro0a5WxzLnDgvffeQ0xMjGEbNWu6Dh2Hw1EpF4bk2fz9/dG+fXu0b98ezZs3x+23345FixZhypQpAOD87OWFzs2Lc+PwlltuwejRo8W2bdq0AXA2BGzMmDEYOnQo/vGPfyAqKgq+vr6YOnWq83Oo50tOTsaqVaswbdo09OrVCwEBAc5/Kysrg4+PD1auXCn2MSQkxOX78/96S3Q+ngOIjHTnhltuuQV9+/ZFixYt8PLLL6Nhw4bw9/fHihUr8MorrxjCkC51DrEiKysLPXv2RFhYGJ5++mk0adIEAQEB2LlzJx555BExiImoPDj+vQMX0VWoVq1aLm+5OOf8VwzMiouLAwDs2bMHjRs3dtaLioqQmpqKpKQkl/bXX389Xn31VeTk5ODDDz9EfHw8OnXq5Pz3c2+xjYqKMvwsUXmcW0gcO3bM9M9ERkYiNDQUpaWllxyHH3/8MRo3bozFixc7X00D4FywX6hTp064++67MWjQIFx33XVYsmSJc2HQpEkTKKWQkJCA5s2bm+4vkRU8BxC5nhs+//xzFBYW4rPPPnN5lc3MXRZ04uLinO+sON+ePXtcvv/qq69w4sQJLF68GD169HDWU1NTy/3YRJfC8e+5+GfjKtSkSRPs3r0bGRkZztoPP/yADRs2WN5WUlIS/P398dprr7n8Bertt99GdnY2Bg4c6NJ+1KhRKCwsxLx587Bq1Spcf/31Lv+enJyMsLAwPP/88y5viz3n/D4TnW/dunXiX0FXrFgBQH4LkY6vry9GjBiBTz75REy0PH8cnvuL7PmPvWXLFmzatEm7/aSkJCxcuBCrVq3Crbfe6vxL6/Dhw+Hr64unnnrKsC9KKZf0cKLy4jmA7MTMuUE6jmdnZ2POnDnlftwBAwZg8+bN2Lp1q7OWkZGB+fPnu7STHruoqAhvvPFGuR+b6ByOf+/DV6Kr0NixY/Hyyy8jOTkZd9xxB44fP45Zs2ahVatWhs8iXEpkZCQmT56Mp556Cn/7298wZMgQ7NmzB2+88Qbat2+PW265xaX9X//6VzRt2hT/93//h8LCQpe38QFnP+82c+ZM3HrrrfjrX/+KG264AZGRkfjjjz+wfPlydO3aFf/5z38q/ByQ97nvvvtQUFCAYcOGoUWLFigqKsLGjRudr3ZZDeD617/+hXXr1qFjx46466670LJlS5w8eRI7d+7EF198gZMnTwIABg0ahMWLF2PYsGEYOHAgUlNTMWvWLLRs2dIlQOlCQ4cOxZw5c3DbbbchLCwMb775Jpo0aYJnn30WkydPxoEDBzB06FCEhoYiNTUVS5Yswbhx4/DQQw9V6Hki4jmA7MTMuSE9PR3+/v4YPHgwxo8fj7y8PLz11luIioqy9C6m8z388MN477338Le//Q33338/goODMXv2bMTFxeHHH390tuvSpQtq1aqF0aNHY+LEifDx8cF7771XrrfGEl2I498LXcYkcFs5d/uRbdu2XbTd+++/rxo3bqz8/f3V1VdfrVavXl2u25uc85///Ee1aNFC+fn5qejoaHXPPfeoU6dOiY/9f//3fwqAatq0qbZ/69atU8nJySo8PFwFBASoJk2aqDFjxqjt27c724wePVoFBwdfdD/JPlauXKnGjh2rWrRooUJCQpS/v79q2rSpuu+++1R6erqzHQCVkpJi+Pm4uDg1evRol1p6erpKSUlRDRs2VH5+fiomJkb17dtXzZ4929mmrKxMPf/88youLk45HA71l7/8RS1btswwn86/xdX53njjDQVAPfTQQ87aJ598orp166aCg4NVcHCwatGihUpJSVF79uxxtunZs6dq1apVeZ8u8lI8BxC5Mntu+Oyzz1SbNm1UQECAio+PVy+88IJ65513DOM9Li5ODRw40PA40q3jfvzxR9WzZ08VEBCg6tevr5555hn19ttvG7a5YcMG1alTJxUYGKhiY2OdtyECoNatW+dsx1v8kFUc/97HRyn+iYGIiIiIiIjIDH4mmoiIiIiIiMgkLqKJiIiIiIiITOIimoiIiIiIiMgkLqKJiIiIiIiITOIimoiIiIiIiMgkLqKJiIiIiIiITKq0RfSMGTMQHx+PgIAAdOzYEVu3bq2shyKqdjj+ye44B8jOOP7Jzjj+yQ4q5T7RH374IW677TbMmjULHTt2xPTp07Fo0SLs2bMHUVFRF/3ZsrIyHD16FKGhofDx8XF314guSSmF3NxcxMbGokYN639nqsj4BzgHqGpVdPwDPAeQZ+M5gOyM45/szNL4V5WgQ4cOKiUlxfl9aWmpio2NVVOnTr3kzx46dEgB4Be/qvzr0KFDl338cw7wq7p8lXf8V3QOcPzzq7p88RzALzt/cfzzy85fZsZ/TbhZUVERduzYgcmTJztrNWrUQFJSEjZt2mRoX1hYiMLCQuf3yv0vjFe5unXrGmo9e/YU2952221iPTs7W6zv2bPHUCsuLhbbhoeHi/WOHTuK9W3bthlqTz31lNj2zJkzYt2ThYaGWv4Zq+MfsMccIM9TnvEP8BxQUV27dhXrqampYv3o0aMVfsxGjRqJ9b/+9a+G2qefflrhx/MUPAeQnXH8k52ZGf9uX0RnZmaitLQU0dHRLvXo6Gjs3r3b0H7q1KnahVll0L01pDInrfR2AD8/P7FtcHCwWNctjAMCAkw9nq7txR5Tal+Zb62pit/NxZRnX62Of+DyzwEiM8o716v7OUDHyv5W5jGpZk35tFzet9abodu27jxlFzwHkJ1x/JOdmRn/VZ7OPXnyZGRnZzu/Dh06VNVdIrqsOAfIzjj+ye44B8jOOP7JU7n9lei6devC19cX6enpLvX09HTExMQY2jscDjgcDnd3g6hKWB3/AOcAeReeA8jOeA4gO+P4Jztx+yLa398fiYmJWLt2LYYOHQrgbNLe2rVrMWHCBHc/nJY73hosfZYZAO6//36xnpSUJNalg0N+fr7ptgDQoUMHsT5ixAixLtG9Jfzw4cOmH3PDhg1i25MnT4r1r7/+Wqy//vrrhtqpU6fEtp6kuox/oqriqXNAOmeUlZVZ2kaDBg3E+tixYw21v//972LbsLAwS49ZmUpLSw219957T2z7yCOPiPVXX321wv3Qvd3c6u/ncvDU8U/kDhz/ZCduX0QDwIMPPojRo0ejXbt26NChA6ZPn478/HzcfvvtlfFwRNUKxz/ZHecA2RnHP9kZxz/ZRaUsokeNGoWMjAw88cQTSEtLw9VXX41Vq1YZggaIvBHHP9kd5wDZGcc/2RnHP9lFpSyiAWDChAl86wbZFsc/2R3nANkZxz/ZGcc/2UGVp3MTEREREREReQofVc3uap6Tk4Pw8PAKb8dqsFiTJk0Mtc8//1xse2Hq4DlnzpwR61KglxTYAsDlhvPn0wV3hYSEVHjb/v7+Yj0yMtJQ093DVLcNXb2goMBQmzVrlth2yZIlYr2yZWdnV0nIj7vmAFFFeOv4d0dI1c6dO8V6s2bNxHpAQIChJh0DAX3opLQNQA5kzMrKEtvWq1dPrAcFBYl1qY+BgYFiW+lcBOjPXV988YWhdvPNN4ttdSo7cMxb54Cn0l3XSeNANwasXvJWl/vHd+nSRaxv3LhRrF9xxRWG2m+//Sa21fWb498zVJcx6g664MpXXnlFrOvOxVJIs24NpGNm/POVaCIiIiIiIiKTuIgmIiIiIiIiMomLaCIiIiIiIiKTuIgmIiIiIiIiMomLaCIiIiIiIiKTKu0+0VXNagLd1KlTDbW0tDSxrS5t1M/Pz3RfSkpKxLa6lD1d8qmUNqdLCZfS6gAgODhYrEup4rp+6x5Tl54qpXanpKSIbdesWSPW8/LyxDoRke5YaiW5edOmTWL9qquuEuu6c4Z07NWdo3R3NNDddSEmJsZQi42NFdvqEsGLiorEupTEffr0abGtrq47L950002Gmu5cNHToULGu+11Kv/vqnkpL7uWu33dljptevXoZarpjiy75//nnnxfr0hzo16+f2NZqajHpWb0zkDu2IdV123BH/3THdGnNAACtW7cW65988omh1rx5c7FtaGioWNedGy7X8Z6vRBMRERERERGZxEU0ERERERERkUlcRBMRERERERGZxEU0ERERERERkUlcRBMRERERERGZ5LXp3Dr16tUT61LCaXZ2tthWl56qS64OCgoy1HQppLo0a10KqZTYqktxDQgIEOu6vkjb0e2j7jF1CdpSmreuH4MHDxbrH3zwgVgnIrKazjls2DBDrWPHjmLbw4cPi3Vd8qmUZqo7pltJYAWA3Nxc0/3QnV907aXjupTYDej3R3fO+OOPPww1XXpw//79xfrKlSvFOpO4qx93JKbr2uuuP6y47bbbxPrmzZsNte7du4ttJ06cKNaPHj0q1tu0aWOo7d27V2y7c+dOsT5p0iSx/v3334t1qly6Mao7xlrZhq+vr+lt6I71NWvKyz7d3RWk7ehSuHv06CHWFy9eLNal7ezevVtsq7t7j46uj+7GV6KJiIiIiIiITOIimoiIiIiIiMgkLqKJiIiIiIiITOIimoiIiIiIiMgkLqKJiIiIiIiITLJdOnetWrXEupTOrUt81KVz69KlpXRSh8MhttUlnOqS/awk/umS/XTbkPpiJcUVACIjI8V6ZmamoaZ7Xq+55hqxznRuItId16wm9koJotJxCgBCQ0PFelZWlliXkkJ1KalWk1ml9FR3pVNL29E9r1ZTaaXEct0dMVasWCHWdXfbSEtLM9R0z7cuPZw8W4sWLcS6bhz06tVLrLdr185Q011Hzp07V6x//fXXYl1K3E5MTBTbtm/fXqwXFRWJ9aZNmxpq+/btE9tS5XPHMdnKOU3X1mpqtbQOaNiwodh2+fLlYl13lx7pnPbggw+KbY8cOSLWdeeXy3WHBr4STURERERERGQSF9FEREREREREJnERTURERERERGQSF9FEREREREREJtkuWKxNmzZiXfqAuxQ2BshBLhernzlzxlA7evSo2Hb//v1i/cCBA2I9Pz/f1OPp2gL6oAEp6Ev3/A0aNEis6/oSERFhqIWEhIhtdYFtRERWA8SWLl0q1qVQMF0gSlxcnOltAHI4i9VAK9355XLTBbbo6rrfj3TO1Z2jTp8+LdZ1YVALFy403Q+6PNwR9BMUFCTWu3TpYqhJ4XIAkJOTI9bffvttsf7AAw8Yarrrt1deeUWsR0VFiXXpOdmzZ4/YVhc4pgtela69GCxWdaTjty5I2Kro6GhDTRd+V6dOHbEuBejptq0L5zt16pRY183F8PBwQ23Hjh1i2+qqepyViYiIiIiIiDwAF9FEREREREREJnERTURERERERGQSF9FEREREREREJnERTURERERERGSS7dK5pdROAPjmm28MtZtvvlls27p1a7H+/PPPi/Xdu3eb7J2eLpUyMDDQVA3Qp1wHBASIdSkp9YMPPhDbTp48Waxv27ZNrEuJfwUFBWLbxo0bi3UiIqs6d+5suq10hwIA8PHxEetWEqCtplzr6PpSWaz228pz5efnJ7bVnaN0ibLSed4d6dBUflIauy6dWPe70t3BQ0qi1l2n6RLdx48fL9b/9re/GWqrV68W2+ocP37cdFtdkvfJkyfFev369cX62LFjDbUNGzaIbXft2mWyd1ReVsZ/kyZNxPr06dPFunS3m9zcXLFtq1atxPqRI0dMt//qq68sbUN3Hi0sLDTUdMnflenC341SynRyOl+JJiIiIiIiIjKJi2giIiIiIiIik7iIJiIiIiIiIjKJi2giIiIiIiIik7iIJiIiIiIiIjLJdunc06ZNE+tSEtu6devEtt99951YDwsLE+tSOrcusTQnJ0esnzhxQqxnZWUZasXFxWJbq+mp4eHhhpou2W///v1iXZdwnpeXZ6jp9lFK8CPPYiVBWDdOpXRLQJ67um3okh9LSkpM9k6vRg35b5JmUx7LQ5dmLO0P04nPOn36tFiXEkStpG0D+nEuHZN1vzvd8Vs3dqXfq67fujGqq1vZto5uP6Xjui7FVbpTBKA/vzz00EMme0eXi5XjtI5u7krjt0+fPmLb999/X6zffffdlvpSWerUqSPWddeX27dvF+vS/HI4HKYes6ysDKdOnbpYN8ki3XFdorueHjNmjFjXXTtXloyMDLGuu4vCTz/9JNY/+ugjQ+3o0aNiWyvXf7r2uvNzRa7/+Eo0ERERERERkUlcRBMRERERERGZxEU0ERERERERkUlcRBMRERERERGZxEU0ERERERERkUm2S+devXq1WO/bt6+hNmLECLFtv379xPq8efPE+j333GOoRUREiG2bNm0q1kNCQsS6lG6pS7HTJZ8WFRWJdSn1TpdsmZubK9YfeeQR04+pS4McPny4WO/SpYtYP3nypFinquOOZGhdsqKVbbsjhVuazwDw2GOPifX69etX+DF1rCR+2k3btm3Fet26dcW6dGcEXdqo7pipa3/mzBlDzWqau5W6bk5Y3bYVuv3RjVFpPteqVUtsq3u+3TGf6fJwxzlAd53x9ddfm6pdTGBgoFiX5q7VfbFy7qpXr57YVnddo3tOVq5caajFxsaKbePi4ly+Ly0tZTp3NaRL4ZaOvbp1gDuuGXR3LtJdq+vGUs+ePQ21F154QWxr9a4QVtpXJJ2er0QTERERERERmcRFNBEREREREZFJXEQTERERERERmWR5Ef31119j8ODBiI2NhY+PDz799FOXf1dK4YknnkC9evUQGBiIpKQk7N271139JapSHP9kZxz/ZHecA2RnHP9Ef7IcLJafn4+2bdti7Nix4ofIp02bhtdeew3z5s1DQkICHn/8cSQnJ+OXX37Rhq5cTv/617/EuvRh+6NHj4ptf/31V7E+ePBgsf7EE0+Y7J3+Q/+FhYViXfrwvC70QhfCogsg8PPzM9R0AWe6D+Fv3bpVrKelpRlqurAC3QG4KgLEPH38VydWg8LcESJ04403ivW//OUvYv26664z1E6fPi22zczMFOsffPCBpb5YoQsLfPjhhw21Z599tsKP50njv2ZN+fSmO95J4y44OFhsqwst0Y1p6Viqa6sL6LLSXhcUZnXbVsJZdNu2ct7RtdX1o0GDBiZ75z6eNAfsQhpLVueAjtTeasiRFZGRkWI9Ly9PrOvmrvSc6K7fLpx3um0CHP9Vyco1k9UAMd35Ujomv/vuu2Jb6XoJ0M85KUhZF/Cnu+7SadmypaE2Y8YMse3hw4cNjzVu3DhTj2N5Ed2/f3/0799f/DelFKZPn47HHnsM1157LYCzT3Z0dDQ+/fRT3HDDDVYfjqha4fgnO+P4J7vjHCA74/gn+pNbPxOdmpqKtLQ0JCUlOWvh4eHo2LEjNm3a5M6HIqp2OP7Jzjj+ye44B8jOOP7Jbtx6n+hzb9GNjo52qUdHR4tv3wXOvk35/LcqS/fqJPIE5Rn/AOcAeQeOf7I7zgGyM45/spsqT+eeOnUqwsPDnV8NGzas6i4RXVacA2RnHP9kd5wDZGcc/+Sp3LqIjomJAQCkp6e71NPT053/dqHJkycjOzvb+XXo0CF3donosinP+Ac4B8g7cPyT3XEOkJ1x/JPduPXt3AkJCYiJicHatWtx9dVXAzj7towtW7bgnnvuEX/G4XDA4XC4sxsXtXjxYrHet29fQ61du3Zi25UrV4r1zz77TKxHRUUZan/88YfY1kpSNgAx7VCXsqejS0QtKCgw1IqKisS2YWFhYj0uLk6sT5o0yXTbXr16ifXvvvtOrH///fdivbKVZ/wDl38OVCYr6ZG6FG4dKckRkBMhu3TpIrbt16+fWN+/f79YvzC1EdC/1Sw+Pl6sDxgwQKy7gy6opWPHjpX2mDrVbfz/9a9/Feu6Y6k0HnWporrjoC5BVErF1W1DRzdfdCnEVtrqzjtW2lrZBiA/t7pk1tzcXLGuSyyWxv+WLVss9K58qtscsAsradm6trq5a2VcW73jhER3R4DRo0eL9WXLlon1BQsWGGq6+XLhtV5508c5/iuX1WsmK6ycR3RjTnfHnPDwcLGenZ1tqPXp00dsK12LAfo1naRWrVpi/aabbnL53spzYXkRnZeXh3379jm/T01Nxffff4/atWujUaNGmDRpEp599lk0a9bMGW8fGxuLoUOHWn0oomqH45/sjOOf7I5zgOyM45/oT5YX0du3b0fv3r2d3z/44IMAzv6VbO7cuXj44YeRn5+PcePGISsrC926dcOqVat4fzjyChz/ZGcc/2R3nANkZxz/RH+yvIju1avXRd9S4OPjg6effhpPP/10hTpGVB1x/JOdcfyT3XEOkJ1x/BP9qcrTuYmIiIiIiIg8BRfRRERERERERCa5NZ3bE7Rs2VKsS8mMupvDb968Wax37dpVrLdu3dpQ070dxmrCqZQip9u2LjlSV5f6okut0z1XUkIkICdo//7772Jb3e0OfvvtN7FOZ+mShXW/Q39/f0PNXQnCkoiICLH+3HPPifVRo0aJdSlF/tixY2LbrVu3inVdYrOUFrx7926xbYMGDcT6M888I9YlUpI/oN/3l19+Way3aNHCUEtMTBTb7tixw2TvPIvuuGZlXhQXF1daX3R3RdCl1OoSc6W7MejmuG7frdDNcV2/pQRWQE4h1p3/rD5X0t0fbrzxRrEtlY87kqg9gTTvrF6nWUm7zszMFOu6u5Ho7iLz5ptvGmpNmjQR227cuNHleyvpxOR+7phbVq/33TFvdQnaoaGhYr127dqGmi75W9e/48ePi3Xp3P3VV1+JbXXXi2bwlWgiIiIiIiIik7iIJiIiIiIiIjKJi2giIiIiIiIik7iIJiIiIiIiIjKJi2giIiIiIiIik2yXzt24cWOxLiWc6tJ2dUnUUkowICeL5ubmim116am6dFIpJdJKEuTFSOmpurTayMhIsa57TqS0Pt3zrUtxjomJEeu6lG9vZTWFUcdqErekb9++Yn3EiBGG2k033SS2PXHihFj/5ZdfxLo0N8LCwsS2derUEetSOj8gj19dGqruuKDbz3/84x+m+/HTTz+JdV06cUBAgKGmO+Z4K6v7Kx17dXNCdxy0mgheWduoTLp9l86hgLU0b12St27fCwsLxbo0/sm9vC2F2wp3XWNdffXVhtoPP/wgtl24cKFYHzRokFhPTk421KQ7cADGO6DY+XdbHVTm81+Zyett27YV6z/++KNYj42NNdRuuOEGsa3umu6pp54S69L6Zc2aNWLbiqgeZ2UiIiIiIiIiD8BFNBEREREREZFJXEQTERERERERmcRFNBEREREREZFJtgsW0wWUnDlzxlDThUfoAmuCgoLEuvRBfikQ7GJ1K2Ezun3UbUMXNCBtRxdMoet3ZmamWJfUrl1brOsCa6RQAsB+wWK6EAp3hJ9MnDhRrN99991iPTo6WqwfPnzYUNOFZen6rdu2RDemdc+Vbs5I28nIyBDb6oIvdDZu3GioDRs2zNI2HnvsMbF+7733Gmp//PGH2PaWW25x+b6srMwr5tA///lPsa4LxpIC6nTBbbpjle54ZzXkr7qQjuu6sDXdnNM9h35+foaa7twaGBgo1nVBfEOHDjXUdL8DhijRxbgjvPWRRx4R69JxZObMmWLbW2+9VazrgjhXrFhhqMXFxYlt3REqSlVHOrbpjmu662ndmJa2ozuW6oIec3JyxLo7zov/93//J9alebto0aIKP96F+Eo0ERERERERkUlcRBMRERERERGZxEU0ERERERERkUlcRBMRERERERGZxEU0ERERERERkUlM5/7/pJQ4XdroyZMnxbouQVTajq4fVpNCpfa6beiS8HRptVKqqi7ZT7c/aWlpYt1KGrou+Ts0NFSse7O//vWvhto111wjtr3iiivEekBAgFiX0s5DQkLEtllZWWL9yJEjYj08PNx0P3R13bguKCgw1KTkX0A/B3RjTxrvuuOCLilYGusA0KFDB0Pt6NGjYlvd70FKPQeAvXv3Gmq6uwfcddddLt+fOXMGU6ZMEdt6ksaNG4t1XYKodLzTJUsfPHhQrOvOAVbSU6s73RzSJfzqxq40t3TPie4coDsfHThwwPS2iS5GOjfEx8eLbZ988kmxrhu/0p0eRo4cKbaVjumAtbuX6K716Czp2Gb1bjc60nWD7lqiMlm9c4lk27ZtYn3dunViPTk52fS2dazeGUg6R1u5W5BZfCWaiIiIiIiIyCQuoomIiIiIiIhM4iKaiIiIiIiIyCQuoomIiIiIiIhM4iKaiIiIiIiIyCTbpXPrSAlvuhS79PR0sa5LZrVCl/in64uUzKhLE7SaMiilUuqS8HR0ia0SXf+spnZ7i3HjxhkSCYcPH25oZyURGND/TqREayn5+mLb1iXxSuM3Pz9fbKtL/talkErb1iV86/qtS2GWxpju+dY9pi4pPCcnx1ArKSkR2546dUqs69pLffTWNPv69euLdV0auS6hU2qvmyu647HumGTlDg26bbvjHKCjO8ZKdd22dannUjI/ICcF65Lsw8LCxLpu/Dds2FCs01m6caobB9WdtD+6Y70u5Vd3rmvRooWh9uKLL4ptdQnauvH497//3VCzmiJ/9dVXi3Xp7gSbNm2ytO3qyMqdDnRjQNdeqnvqnNCxmgj+ySefGGo//fST2Pb222+3tG3pXGLlPAfoz/Pfffedpb6UF1+JJiIiIiIiIjKJi2giIiIiIiIik7iIJiIiIiIiIjKJi2giIiIiIiIik7iIJiIiIiIiIjLJduncVpIPdcl+uqRcXQqv9Ji6BDrdY+pSSKV0OyvJgxcjPabVhG9dkrGUwKxLN9ax2t7TLFy40PC8btu2zdCuS5cu4s+3bt1arMfFxYl1Kb25Vq1aYltdUqIuyVIaN5GRkWJbXd1KIrIugVXXbytpxnl5eWJdlzauS3iW5peu37rUYl17qS+69OTly5dfsl/VWffu3S21141R6bnU/e50v4/atWuLdSmJWnc81o1zK8dvq8d6d9A9V7rUY2k/dQnyunmr+z14+50bKspK4rDu3K5TFWPPyp1EdONRl/IvJWh/+eWXYttOnTqJ9euuu06su4Pu+Zb2X7fvnkTaX6sp3O4gpbYDwNixY8W6lOiekZFh6TGt3NFBd32sO2Y+88wzYj0qKspQGzFihK6LllhJCrd6R4z9+/eb3vaF48fKuOEr0UREREREREQmcRFNREREREREZBIX0UREREREREQmcRFNREREREREZJLtgsUqk+6D/NIH4nVBCFaDuyRWwxSsBJHpwmN0/dYFi+3bt89Qu/rqq8W2use0GnjiaXx8fAz7uGvXLkO7LVu2WNquw+EQ6wkJCYZa06ZNxbbx8fFiPTY2VqxLc8PqHNAFS2RmZhpquvCvEydOiHUp6E5X17U9ffq0WLcS5qILCrM61qXnRBd8VhVBQO4khXZdjC5gTRp3uuc9IiLC9DYAuY9Wx7muvVS3epzWsRLQZTWETWqvC2bT9cPTQvA8UXU6PlgJkLISngYATz75pFg/evSooda2bVux7ahRoyw9pjvo9rNu3bqGmm6OVlc1a9Y0/M6lMaA7ZuqOD7oQrbvuustQS0tLu1Q3XUjXUQBw7bXXGmpXXHGFpW1bCSTWHXcbNmwo1q+//nqxPmDAAJO901/v666NpPORbh91Ibe69t9++61Yl0jPn9njHl+JJiIiIiIiIjKJi2giIiIiIiIik7iIJiIiIiIiIjKJi2giIiIiIiIik7iIJiIiIiIiIjLJduncubm5Yj04ONhQs5pkqkumkxIRdclvuqQ5HWk7ugRLXV2XfCptW5eEazVp+Y8//jDU2rVrJ7bVpelaSY71RNnZ2YaaNE7r1asn/rzVROeTJ08aal999ZXYVpdEbyUp2cq4A/RjSeqLbtu69OuaNeVDobTtkJAQsW1kZKRYDwsLE+t+fn6Gmu750/UvKChIrEvHOd22Dx486PJ9aWkpfv31V7FtdbR+/XpL7a0ce3XJt7qxqEuDlY5hVse/bgxI29H1W3dM0LWXtm31uKt7rqT90e2j7nmtTsnRnsRKyrUuiT46Olqs685HunOJFe74fT/11FNiXTfG2rRpY6gNGzaswv0A9ONdouufbhtSOrenqaz0/b/+9a9iXRrTVq9Hjh8/Ltal64PBgweLbT///HOxrmNlXixYsECsr1q1Sqzv37/f9LZ1KdzuoDve6O46snHjxkrry/n4SjQRERERERGRSVxEExEREREREZnERTQRERERERGRSVxEExEREREREZlkaRE9depUtG/fHqGhoYiKisLQoUOxZ88elzZnzpxBSkoK6tSpg5CQEIwYMQLp6elu7TRRVeEcIDvj+Cc74/gnu+McIPqTpXTu9evXIyUlBe3bt0dJSQn++c9/ol+/fvjll1+cqcEPPPAAli9fjkWLFiE8PBwTJkzA8OHDsWHDhkrZAR1dCq+VpL2cnBxLjyml7QLWEot1/dPtj5SqajWVWZfuKG1blx5uNVH2wIEDhpru+dMlx+raV6aqngNSEqEundAqKV3e6u9El1ztcDhMb1tHlwoszV2rqZ5WEod1Cf9Hjx4V67r5KM0N3XNiNZlVal9QUCC21fVbUtXjXzJw4EBL7aW7JejqusR13QWhbtvSGNX97nTHWN0YkMaXlfTli/VF6rduG7qxe+bMGbEuzTmr6dy641BlqY7jvzyspPm2bNlSrDds2FCs666bpDsJ6I5J7lC/fn2x3qVLF7Guu+NE9+7d3danC0m/B3fcoQUAGjVqVK4+XcrlnANdunQxHBOk/fr444/Fn9cde2JjY033QbpLCiDfzQTQJ1RL12nTp08X21pN55YsXbpUrLdu3VqsDx06tMKPWZl0dwlwRyK4dL40e4y0tIi+MAJ97ty5iIqKwo4dO9CjRw9kZ2fj7bffxoIFC9CnTx8AwJw5c3DllVdi8+bN6NSpk5WHI6p2OAfIzjj+yc44/snuOAeI/lShz0Sf+wtN7dq1AQA7duxAcXExkpKSnG1atGiBRo0aYdOmTeI2CgsLkZOT4/JF5Ck4B8jOOP7Jztwx/gHOAfJcPAeQnZV7EV1WVoZJkyaha9euzrcHpKWlwd/f3/Cye3R0NNLS0sTtTJ06FeHh4c4v3VuEiKobzgGyM45/sjN3jX+Ac4A8E88BZHflXkSnpKRg165dWLhwYYU6MHnyZGRnZzu/Dh06VKHtEV0unANkZxz/ZGfuGv8A5wB5Jp4DyO4sfSb6nAkTJmDZsmX4+uuv0aBBA2c9JiYGRUVFyMrKcvkrVHp6OmJiYsRtORwOMXSIqDrjHCA74/gnO3Pn+Ac4B8jz8BxAZHERrZTCfffdhyVLluCrr75CQkKCy78nJibCz88Pa9euxYgRIwAAe/bswR9//IHOnTu7r9cm+2qlLqWCHjlyxNJj6hJ+3ZHAqEtbtZLMqntMXcKptD+650+3jdDQULH+22+/GWpW02qtppC7gyfNAauklEOryYenTp1yV3eoGqqO4/9vf/ubpfa6uyUUFhYaarrj1z333CPW33//fbEu3V1Bl/KuO97pkr+lY6+Vc9HF6lJfdBe7unTj8PBwsb5+/XpDLS4uTmyblZUl1q2Ijo4W61ZuvVMV4//C85yVZG2z27zYtjdu3Fjhx6sKs2fPFuvNmzcX61ZT/t3BHXdX0V17tWjRolx9upTLOQfi4+MNx84333zT0O6ZZ54Rfz4vL0+s69K5pfa684XuLefn/0HhfFaO09OmTRPr//3vf8X6Cy+8YKj17t1bbLtmzRqxfuLECbFeXdSrV0+su+Pz8xce96wcYy0tolNSUrBgwQIsXboUoaGhzs83hIeHIzAwEOHh4bjjjjvw4IMPonbt2ggLC8N9992Hzp07M5GPvALnANkZxz/ZGcc/2R3nANGfLC2iZ86cCQDo1auXS33OnDkYM2YMAOCVV15BjRo1MGLECBQWFiI5ORlvvPGGWzpLVNU4B8jOOP7Jzjj+ye44B4j+ZPnt3JcSEBCAGTNmYMaMGeXuFFF1xTlAdsbxT3bG8U92xzlA9KcK3SeaiIiIiIiIyE7Klc7tyXR/RatRw/j3BKvBYtI2dI/p5+dnaRu6AAIrAWVWwkQA94Re6EJlfv75Z0NNt++6elUEixFR9SKFdgH64K7g4GCxbuVYumTJErH++uuvi/WbbrrJUNOFltWpU0esHz16VKxbSbXV7aPuHCCFmdWtW1dsqws42rJli1h/9dVXDbWePXuKbXX9tvI7GzJkiFh/6623TG+jKrgjSKwi29SdZ1esWCHW69evL9anTp1qqH3wwQem+3ExTzzxhKGmCxyUxh0A7Nq1yy19udx0gay1atW6zD1xvwULFhhqd911l6HWqlUr8ed1z4HuWCXdx1p3vrjwPtjnZGZminVd8KLkH//4h6V6RkaGoaYLhZ0yZYrpfgDy9bfVYGR30D3f7gidrMj+8JVoIiIiIiIiIpO4iCYiIiIiIiIyiYtoIiIiIiIiIpO4iCYiIiIiIiIyiYtoIiIiIiIiIpOYzv3/SQl0f/zxh6VtFxYWinUpOU+XHFtSUmLpMa0kaFtNuZbquiRYXfKgLtlQSj7X9UOXnKdLpSQi+9Ad03Xp1+5I89R59NFHLdWt0B1jpf20eicGK+ncOTk5ui5WGt3+6M4BUjLt4MGDxbbVOZ27W7duhn208js5deqUWM/Pzxfr0jXMmTNnxLa6epMmTcT63//+d0Nt7dq1Ytvjx4+L9X79+on1iRMnGmrr168X27pjLlYmq2nsuus63e/H0x04cMBQ69Spk9j20KFDYl13d5zo6GhDTXfs0c0h3TWy9HvVbfvkyZNiXbfGkKSnp4t1qyn0lXF3gHOk50qXKq67049uPyW6c2hF5gpfiSYiIiIiIiIyiYtoIiIiIiIiIpO4iCYiIiIiIiIyiYtoIiIiIiIiIpO4iCYiIiIiIiIyyWvjja0kTutYTSHVpfJJ9eLiYrFt7dq1xbqUwg3Iad5W9vFi7aXUR91zokvhjo2NFetSGp6/v7/YVpfAqmtPRPZx5513ivURI0aI9aCgILEuHe90x92qYDUl2ROlpqaK9cjISLGuS1qXUlg3bNhQ7n5VlUaNGhnOc/Hx8YZ2uucnLCxMrOuuP6RUYN3dMXTJx/PnzxfrP/74o6HWt29fsW2XLl3Eeps2bcS69LuV0sABOd0c0F+/WUlErgoFBQVi/X//+99l7snlMXXqVEPtpptuEts2aNBArOuuefPy8gw13Z10dONIN1+kRHBdSrgucd3X11esh4SEGGo333yz2FZH95i6/XEHK2sVXbK2LslfotvHiuAr0UREREREREQmcRFNREREREREZBIX0UREREREREQmcRFNREREREREZBIX0UREREREREQmeW06ty7FTpeoJ6VcW01y++STT8S6lJCpS5TTJVFL/dPRbcNqYrmUyqfrR3Z2tljfvn27WJfotq2rV0bSHhF5Fl1Cc1xcnFjXpTSHh4cbah988EG5+1VeuuOalbpSytJjWmmvS2vV1XXnF+kxV69eLbbVJbCHhoaK9eXLlxtqL7zwgti2OluwYEGlbLdOnTpiXUoz1t0xxGrysTQfdSncut/rihUrxLr0POnSw3Wqewq3ji6d/4EHHjDUnnnmmcruTqXbtWuXoaYbc3/729/E+tNPPy3W27dvb6jpEu6rk2+++cZQW7duXRX0xBoryd+6Y8XRo0dNb8PqedEMrkKIiIiIiIiITOIimoiIiIiIiMgkLqKJiIiIiIiITOIimoiIiIiIiMgkrw0WCwwMFOu6AAIpnCUiIsLSY06dOtVSe3Kl+9C/LlDH6u+HiOzjjz/+EOsOh0OsS2FGuvAkneDgYLGen59vehtWg7uqO13IpxQY+f3334tti4uLxXpISIhYnzFjhrnO2dSJEycs1an6OnDggFjnHABWrVplqS5p3ry5WE9MTBTrbdq0Eev169c31GrVqmW6HwBw5MgRsX733Xeb3oaVIOHKZiXMb9q0aWJ9z549prehC5auCL4STURERERERGQSF9FEREREREREJnERTURERERERGQSF9FEREREREREJnERTURERERERGSS16Zznzx5Uqz/9ttvYv3w4cOG2pYtWyw9pi71TqJLoraz+fPni/XGjRuL9Z07d1Zmd4jIg+mOx//4xz/EunTOOHbsmKXHtJI2ahdWznXHjx8X66dPnxbrurRVT00yJ3KXxx9/vKq74BV0awZd/YMPPqjM7lRYdVp7WOnLF198UeHHKy0trfA2LsRXoomIiIiIiIhM4iKaiIiIiIiIyCQuoomIiIiIiIhM4iKaiIiIiIiIyKRqFyxW2R96P3PmjFiXQmiKi4stbbs6fWDfE+l+NwUFBWLd6u/Hqqr6fXIcUXXg6eNftx1dGJV0PLEaRMK5a2TlOdEd03Nyciy1LykpMf2YF+Ppc4CoIjj+yc7MjEMfVc1G6+HDh9GwYcOq7gYRDh06hAYNGlz2x+UcoOqA45/sjnOA7Izjn+zMzPivdovosrIyHD16FKGhocjNzUXDhg1x6NAhhIWFVXXXKkVOTo7X7yPgWfuplEJubi5iY2NRo8bl/8QD54D38aR95Pi/vDxpbFSEJ+1ndZkDSik0atTII56zivCksVFenrSP1WX88xzgPTxpH62M/2r3du4aNWo4V/7n3mIdFhZW7Z/0irLDPgKes5/h4eFV9ticA97LU/aR4//ys8M+Ap6zn9VhDpx7G7unPGcVZYf99JR9rA7jH+A5wNt4yj6aHf8MFiMiIiIiIiIyiYtoIiIiIiIiIpOq9SLa4XBgypQpcDgcVd2VSmOHfQTss5/uZofnjftIOnZ43uywj4B99tOd7PKc2WE/7bCPlcEOzxv30XNVu2AxIiIiIiIiouqqWr8STURERERERFSdcBFNREREREREZBIX0UREREREREQmcRFNREREREREZFK1XkTPmDED8fHxCAgIQMeOHbF169aq7lK5ff311xg8eDBiY2Ph4+ODTz/91OXflVJ44oknUK9ePQQGBiIpKQl79+6tms6W09SpU9G+fXuEhoYiKioKQ4cOxZ49e1zanDlzBikpKahTpw5CQkIwYsQIpKenV1GPqzdvGv+A988Bjn/386Y54O3jH+AccDeOf45/O/Om8Q9wDpzjTXOg2i6iP/zwQzz44IOYMmUKdu7cibZt2yI5ORnHjx+v6q6VS35+Ptq2bYsZM2aI/z5t2jS89tprmDVrFrZs2YLg4GAkJyfjzJkzl7mn5bd+/XqkpKRg8+bNWLNmDYqLi9GvXz/k5+c72zzwwAP4/PPPsWjRIqxfvx5Hjx7F8OHDq7DX1ZO3jX/A++cAx797edsc8PbxD3AOuBPHP8e/nXnb+Ac4B87xqjmgqqkOHTqolJQU5/elpaUqNjZWTZ06tQp75R4A1JIlS5zfl5WVqZiYGPXiiy86a1lZWcrhcKgPPvigCnroHsePH1cA1Pr165VSZ/fJz89PLVq0yNnm119/VQDUpk2bqqqb1ZI3j3+l7DEHOP4rxpvngB3Gv1KcAxXB8c/xb2fePP6V4hzwljlQLV+JLioqwo4dO5CUlOSs1ahRA0lJSdi0aVMV9qxypKamIi0tzWV/w8PD0bFjR4/e3+zsbABA7dq1AQA7duxAcXGxy362aNECjRo18uj9dDe7jX/AO+cAx3/52W0OeOP4BzgHyovjn+Pfzuw2/gHOAU+dA9VyEZ2ZmYnS0lJER0e71KOjo5GWllZFvao85/bJm/a3rKwMkyZNQteuXdG6dWsAZ/fT398fERERLm09eT8rg93GP+B9c4Djv2LsNge8bfwDnAMVwfEP5/eeur8c/+Vnt/EPcA546n7WrOoOkHdKSUnBrl278O2331Z1V4guO45/sjvOAbIzjn+yOzvMgWr5SnTdunXh6+trSGtLT09HTExMFfWq8pzbJ2/Z3wkTJmDZsmVYt24dGjRo4KzHxMSgqKgIWVlZLu09dT8ri93GP+Bdc4Djv+LsNge8afwDnAMVxfEP5/eeuL8c/xVjt/EPcA546n5Wy0W0v78/EhMTsXbtWmetrKwMa9euRefOnauwZ5UjISEBMTExLvubk5ODLVu2eNT+KqUwYcIELFmyBF9++SUSEhJc/j0xMRF+fn4u+7lnzx788ccfHrWflc1u4x/wjjnA8e8+dpsD3jD+Ac4Bd+H45/i3M7uNf4BzwGPnQJXGml3EwoULlcPhUHPnzlW//PKLGjdunIqIiFBpaWlV3bVyyc3NVd9995367rvvFAD18ssvq++++04dPHhQKaXUv/71LxUREaGWLl2qfvzxR3XttdeqhIQEdfr06SruuXn33HOPCg8PV1999ZU6duyY86ugoMDZ5u6771aNGjVSX375pdq+fbvq3Lmz6ty5cxX2unrytvGvlPfPAY5/9/K2OeDt418pzgF34vjn+Lczbxv/SnEOnONNc6DaLqKVUur1119XjRo1Uv7+/qpDhw5q8+bNVd2lclu3bp0CYPgaPXq0UupsvP3jjz+uoqOjlcPhUH379lV79uyp2k5bJO0fADVnzhxnm9OnT6t7771X1apVSwUFBalhw4apY8eOVV2nqzFvGv9Kef8c4Ph3P2+aA94+/pXiHHA3jn+OfzvzpvGvFOfAOd40B3yUUso9r2kTERERERERebdq+ZloIiIiIiIiouqIi2giIiIiIiIik7iIJiIiIiIiIjKJi2giIiIiIiIik7iIJiIiIiIiIjKJi2giIiIiIiIik7iIJiIiIiIiIjKJi2giIiIiIiIik7iIJiIiIiIiIjKJi2giIiIiIiIik7iIJiIiIiIiIjKJi2giIiIiIiIik7iIJiIiIiIiIjKJi+hq5MCBA/Dx8cFLL710ybZPPvkkfHx8LkOviCrPmDFjEBIScsl2vXr1Qq9evSq/Q0Q2MXfuXPj4+ODAgQOWf3bMmDGIj493e5+ILifOAbIjK2sNujguoi3w8fEx9fXVV19VdVddFBQU4Mknn7xov06dOoWaNWvio48+AgA8//zz+PTTTy9PB8mjvPHGG/Dx8UHHjh2ruivlNmbMGJc5W7NmTTRs2BA33HADfvnll0p9bDPzkbzTTz/9hJEjRyIuLg4BAQGoX78+rrnmGrz++utV3TWiy4JzgOyA49wealZ1BzzJe++95/L9u+++izVr1hjqV155ZaX35bHHHsOjjz5qqm1BQQGeeuopANC+mrd69Wr4+PigX79+AM4uokeOHImhQ4e6o7vkRebPn4/4+Hhs3boV+/btQ9OmTau6S+XicDjw3//+FwBQUlKC/fv3Y9asWVi1ahV++eUXxMbGVsrjmpmP5H02btyI3r17o1GjRrjrrrsQExODQ4cOYfPmzXj11Vdx3333VXUXiSoV5wDZAce5fXARbcEtt9zi8v3mzZuxZs0aQ/1yqFmzJmrWvPivr6ysDEVFRaa2t2LFCnTt2hURERFu6B15q9TUVGzcuBGLFy/G+PHjMX/+fEyZMqWqu1UuNWvWNMzdTp06YdCgQVi+fDnuuuuuKuoZeaPnnnsO4eHh2LZtm+E4e/z48arpFNFlxDlAdsBxfvbFgqCgoKruRqXj27kvo+3btyM5ORl169ZFYGAgEhISMHbsWLHt7Nmz0aRJEzgcDrRv3x7btm1z+XfpM9E+Pj6YMGEC5s+fj1atWsHhcGDWrFmIjIwEADz11FPOt68++eSTzp8rKyvDqlWrMHDgQOd28vPzMW/ePGf7MWPGONt/99136N+/P8LCwhASEoK+ffti8+bNLn0591mjr7/+GuPHj0edOnUQFhaG2267DadOnSrvU0hVbP78+ahVqxYGDhyIkSNHYv78+YY253/e5lLjWPL9998jMjISvXr1Ql5enrZdYWEhpkyZgqZNm8LhcKBhw4Z4+OGHUVhYWO79i4mJAQDDH6h+//13XHfddahduzaCgoLQqVMnLF++3PDzx48fxx133IHo6GgEBASgbdu2mDdvnvPfDxw4cMn5SN5p//79aNWqlfiHyqioKOf/z5kzB3369EFUVBQcDgdatmyJmTNnGn4mPj4egwYNwrfffosOHTogICAAjRs3xrvvvmto+/PPP6NPnz4IDAxEgwYN8Oyzz6KsrMzQbunSpRg4cCBiY2PhcDjQpEkTPPPMMygtLa3YzhOBc4Dswew4P3fN/umnn6J169ZwOBxo1aoVVq1aZfi5I0eOYOzYsYiOjna2e+edd1zaFBUV4YknnkBiYiLCw8MRHByM7t27Y926dZfss1IK48aNg7+/PxYvXuysv//++0hMTERgYCBq166NG264AYcOHXL52V69eqF169bYsWMHevTogaCgIPzzn/+85GN6A74SfZkcP34c/fr1Q2RkJB599FFERETgwIEDLoP1nAULFiA3Nxfjx4+Hj48Ppk2bhuHDh+P333+Hn5/fRR/nyy+/xEcffYQJEyagbt26aNu2LWbOnIl77rkHw4YNw/DhwwEAbdq0cf7Mtm3bkJGRgQEDBgA4+7b1O++8Ex06dMC4ceMAAE2aNAFw9kTUvXt3hIWF4eGHH4afnx/efPNN9OrVC+vXrzd8TnbChAmIiIjAk08+iT179mDmzJk4ePAgvvrqKwajeaD58+dj+PDh8Pf3x4033oiZM2di27ZtaN++vaFtecbxtm3bkJycjHbt2mHp0qUIDAwU25WVlWHIkCH49ttvMW7cOFx55ZX46aef8Morr+C3334z/Xn+zMxMAEBpaSl+//13PPLII6hTpw4GDRrkbJOeno4uXbqgoKAAEydORJ06dTBv3jwMGTIEH3/8MYYNGwYAOH36NHr16oV9+/ZhwoQJSEhIwKJFizBmzBhkZWXh/vvvR2Rk5CXnI3mnuLg4bNq0Cbt27ULr1q217WbOnIlWrVphyJAhqFmzJj7//HPce++9KCsrQ0pKikvbffv2YeTIkbjjjjswevRovPPOOxgzZgwSExPRqlUrAEBaWhp69+6NkpISPProowgODsbs2bPFuTV37lyEhITgwQcfREhICL788ks88cQTyMnJwYsvvujeJ4Rsh3OA7MDsOAeAb7/9FosXL8a9996L0NBQvPbaaxgxYgT++OMP1KlTB8DZa5BOnTo5F92RkZFYuXIl7rjjDuTk5GDSpEkAgJycHPz3v//FjTfeiLvuugu5ubl4++23kZycjK1bt+Lqq68W+1BaWoqxY8fiww8/xJIlS5wvqD333HN4/PHHcf311+POO+9ERkYGXn/9dfTo0QPfffedyx8JTpw4gf79++OGG27ALbfcgujo6Ao/jx5BUbmlpKQos0/hkiVLFAC1bds2bZvU1FQFQNWpU0edPHnSWV+6dKkCoD7//HNnbcqUKYbHBqBq1Kihfv75Z5d6RkaGAqCmTJkiPu7jjz+u4uLiXGrBwcFq9OjRhrZDhw5V/v7+av/+/c7a0aNHVWhoqOrRo4ezNmfOHAVAJSYmqqKiImd92rRpCoBaunSp9nmg6mn79u0KgFqzZo1SSqmysjLVoEEDdf/997u0szKOR48erYKDg5VSSn377bcqLCxMDRw4UJ05c8Zlmz179lQ9e/Z0fv/ee++pGjVqqG+++cal3axZsxQAtWHDhovuy+jRoxUAw1f9+vXVjh07XNpOmjRJAXB5rNzcXJWQkKDi4+NVaWmpUkqp6dOnKwDq/fffd7YrKipSnTt3ViEhISonJ0cpden5SN7pf//7n/L19VW+vr6qc+fO6uGHH1arV692OT4qpVRBQYHhZ5OTk1Xjxo1danFxcQqA+vrrr52148ePK4fDof7+9787a+fG75YtW1zahYeHKwAqNTX1oo89fvx4FRQU5DInR48ebThnEF0K5wDZgdlxDkD5+/urffv2OWs//PCDAqBef/11Z+2OO+5Q9erVU5mZmS4/f8MNN6jw8HDnmC0pKVGFhYUubU6dOqWio6PV2LFjnbVz12gvvviiKi4uVqNGjVKBgYFq9erVzjYHDhxQvr6+6rnnnnPZ3k8//aRq1qzpUu/Zs6cCoGbNmmX1qfJ4fDv3ZXLuLzbLli1DcXHxRduOGjUKtWrVcn7fvXt3AGffUnopPXv2RMuWLS31bcWKFc6/PF1MaWkp/ve//2Ho0KFo3Lixs16vXj3cdNNN+Pbbb5GTk+PyM+PGjXN51fGee+5BzZo1sWLFCkt9pKo3f/58REdHo3fv3gDOvhVp1KhRWLhwofhWNyvjeN26dUhOTkbfvn2xePFiOByOi/Zl0aJFuPLKK9GiRQtkZmY6v/r06ePc3qUEBARgzZo1WLNmDVavXo0333wTISEhGDBgAH777TdnuxUrVqBDhw7o1q2bsxYSEoJx48bhwIEDzjTvFStWICYmBjfeeKOznZ+fHyZOnIi8vDysX7/+kn0i73XNNddg06ZNGDJkCH744QdMmzYNycnJqF+/Pj777DNnu/NfHcvOzkZmZiZ69uyJ33//HdnZ2S7bbNmypXNeAUBkZCSuuOIKlzm2YsUKdOrUCR06dHBpd/PNNxv6eP5j5+bmIjMzE927d0dBQQF2795dsSeAbI9zgOzA7DgHgKSkJOc7PYGz70oLCwtzjl+lFD755BMMHjwYSimX653k5GRkZ2dj586dAABfX1/4+/sDOPtuvZMnT6KkpATt2rVztjlfUVERrrvuOixbtgwrVqxwBgsDwOLFi1FWVobrr7/e5TFjYmLQrFkzwzWWw+HA7bff7p4n0INwEe1meXl5SEtLc35lZGQAOLu4HTFiBJ566inUrVsX1157LebMmSN+frNRo0Yu359biJj5LHFCQoKl/qalpWHnzp2mFtEZGRkoKCjAFVdcYfi3K6+8EmVlZYbPSjRr1szl+5CQENSrV69c92WkqlNaWoqFCxeid+/eSE1Nxb59+7Bv3z507NgR6enpWLt2reFnzI7jM2fOYODAgfjLX/6Cjz76yHkSuJi9e/fi559/RmRkpMtX8+bNAZgL7/D19UVSUhKSkpLQr18/jBs3Dl988QWys7MxefJkZ7uDBw9qx/y5fz/332bNmqFGjRoXbUf21b59eyxevBinTp3C1q1bMXnyZOTm5mLkyJHOP8Zs2LABSUlJCA4ORkREBCIjI52fL7twAXHhHAPOzrPz59i5cXkhaUz//PPPGDZsGMLDwxEWFobIyEhn+N6Fj01UHpwDZAdmxjlw6fGbkZGBrKwszJ4923C9c27Rev71zrx589CmTRsEBASgTp06iIyMxPLly8WxO3XqVHz66af4+OOPDXcK2bt3L5RSaNasmeFxf/31V8M1Vv369U1du3kbfibazV566SXn7WuAs5+NOBe09PHHH2Pz5s34/PPPsXr1aowdOxb//ve/sXnzZoSEhDh/xtfXV9y2UuqSj6/7DKnOypUrERAQ4Hx1kUjy5Zdf4tixY1i4cCEWLlxo+Pf58+e7/BUTMD+OHQ4HBgwYgKVLl2LVqlUun0fWKSsrw1VXXYWXX35Z/PeGDRtechuSBg0a4IorrsDXX39drp8nMsPf3x/t27dH+/bt0bx5c9x+++1YtGgRbrnlFvTt2xctWrTAyy+/jIYNG8Lf3x8rVqzAK6+8YghCqsi54kJZWVno2bMnwsLC8PTTT6NJkyYICAjAzp078cgjj4ghTETlxTlAdqAb5+fuanKp8XtuzN1yyy0YPXq02PZcpsr777+PMWPGYOjQofjHP/6BqKgo+Pr6YurUqdi/f7/h55KTk7Fq1SpMmzYNvXr1QkBAgPPfysrK4OPjg5UrV4p9PH/NAlhfe3gLLqLd7LbbbnN52+eFA6tTp07o1KkTnnvuOSxYsAA333wzFi5ciDvvvLPS+nSxAK/ly5ejd+/ehn5KPxMZGYmgoCDs2bPH8G+7d+9GjRo1DIuXvXv3uizQ8/LycOzYMWeIGXmG+fPnIyoqCjNmzDD82+LFi7FkyRLMmjWrXAdSHx8fzJ8/H9deey2uu+46rFy58pL3T27SpAl++OEH9O3b1+0BdSUlJS6p4HFxcdoxf+7fz/33xx9/RFlZmcur0Re2Y6Aena9du3YAgGPHjuHzzz9HYWEhPvvsM5dXKMx8PEEnLi4Oe/fuNdQvHNNfffUVTpw4gcWLF6NHjx7Oempqarkfm8gMzgGyg/PHuVmRkZEIDQ1FaWkpkpKSLtr2448/RuPGjbF48WKX6wzdbUg7deqEu+++G4MGDcJ1112HJUuWOO9M0qRJEyilkJCQ4HyHHxnx7dxu1rhxY+dbRJOSktC1a1cAZ9/CeuFfRs8l5VXkljxmnLtXW1ZWlku9uLgYa9asEd/KHRwcbGjv6+uLfv36YenSpS5vx05PT8eCBQvQrVs3hIWFufzM7NmzXT4DPnPmTJSUlKB///4V2ym6bE6fPo3Fixdj0KBBGDlypOFrwoQJyM3NNXzWx4pzt1Vo3749Bg8ejK1bt160/fXXX48jR47grbfeEvubn59frn789ttv2LNnD9q2beusDRgwAFu3bsWmTZuctfz8fMyePRvx8fHODIIBAwYgLS0NH374obNdSUkJXn/9dYSEhKBnz54A9PORvNu6devEV8fO5UNcccUVzr/4n98uOzsbc+bMKffjDhgwAJs3b3aZUxkZGYbb00mPXVRUhDfeeKPcj010Ps4BsgMz49wsX19fjBgxAp988gl27dpl+PdzHxk91xZwHb9btmxxuXa5UFJSEhYuXIhVq1bh1ltvdb7yPXz4cPj6+uKpp54y7ItSCidOnDC9D96Mr0RfJvPmzcMbb7yBYcOGoUmTJsjNzcVbb72FsLCwSn9VNjAwEC1btsSHH36I5s2bo3bt2mjdujUyMjKQk5MjLqITExPxxRdf4OWXX0ZsbCwSEhLQsWNHPPvss1izZg26deuGe++9FzVr1sSbb76JwsJCTJs2zbCdoqIi9O3bF9dffz327NmDN954A926dcOQIUMqdZ/JfT777DPk5uZqf2edOnVCZGQk5s+fj1GjRpX7cQIDA7Fs2TL06dMH/fv3x/r167W3h7j11lvx0Ucf4e6778a6devQtWtXlJaWYvfu3fjoo4+wevVq5199dUpKSvD+++8DOPvWpQMHDmDWrFkoKytz+cvto48+ig8++AD9+/fHxIkTUbt2bcybNw+pqan45JNPnK86jxs3Dm+++SbGjBmDHTt2ID4+Hh9//DE2bNiA6dOnIzQ01Lmf0ny81K0wyLPdd999KCgowLBhw9CiRQsUFRVh48aN+PDDDxEfH4/bb78d6enp8Pf3x+DBgzF+/Hjk5eXhrbfeQlRUlKVXL8738MMP47333sPf/vY33H///c7b+5x758Q5Xbp0Qa1atTB69GhMnDgRPj4+eO+998r1tlgiCecA2YGZcW7Fv/71L6xbtw4dO3bEXXfdhZYtW+LkyZPYuXMnvvjiC5w8eRIAMGjQICxevBjDhg3DwIEDkZqailmzZqFly5Yu76670NChQzFnzhzcdtttCAsLw5tvvokmTZrg2WefxeTJk3HgwAEMHToUoaGhSE1NxZIlSzBu3Dg89NBDFXqevMLljAL3NlZucbVz50514403qkaNGimHw6GioqLUoEGD1Pbt251tzo+dvxAuuCWO7hZXKSkp4uNv3LhRJSYmKn9/f+e2HnroIdWyZUux/e7du1WPHj1UYGCgAuByu6udO3eq5ORkFRISooKCglTv3r3Vxo0bXX7+3C2u1q9fr8aNG6dq1aqlQkJC1M0336xOnDhxqaeLqpHBgwergIAAlZ+fr20zZswY5efnpzIzMy2N4/NvcXVOZmamatmypYqJiVF79+5VShlvcaXU2dtHvfDCC6pVq1bK4XCoWrVqqcTERPXUU0+p7Ozsi+6TdIursLAw1bdvX/XFF18Y2u/fv1+NHDlSRUREqICAANWhQwe1bNkyQ7v09HR1++23q7p16yp/f3911VVXqTlz5hjaSfORvNvKlSvV2LFjVYsWLVRISIjy9/dXTZs2Vffdd59KT093tvvss89UmzZtVEBAgIqPj1cvvPCCeueddwy34omLi1MDBw40PI40V3788UfVs2dPFRAQoOrXr6+eeeYZ9fbbbxu2uWHDBtWpUycVGBioYmNjnbdmAaDWrVvnbMfb+1B5cA6QHZgd57pr9ri4OMMtZtPT01VKSopq2LCh8vPzUzExMapv375q9uzZzjZlZWXq+eefV3FxccrhcKi//OUvatmyZYaxqrtGe+ONNxQA9dBDDzlrn3zyierWrZsKDg5WwcHBqkWLFiolJUXt2bPH2aZnz56qVatW5X26PJqPUvwTm121bNkSgwYNEl9Brqi5c+fi9ttvx7Zt2y75iiAREREREZGn4Nu5baqoqAijRo3C9ddfX9VdISIiIiIi8hhcRNuUv7+/NrGPiIiIiIiIZEznJiIiIiIiIjKJn4kmIiIiIiIiMomvRBMRERERERGZxEU0ERERERERkUmVFiw2Y8YMvPjii0hLS0Pbtm3x+uuvo0OHDpf8ubKyMhw9ehShoaHw8fGprO4RaSmlkJubi9jYWNSoUb6/M5V3/AOcA1S13DH+AZ4DyHPxHEB2xvFPdmZp/FfGzacXLlyo/P391TvvvKN+/vlnddddd6mIiAiXm4zrHDp0SAHgF7+q/OvQoUOXffxzDvCrunyVd/xXdA5w/POrunzxHMAvO39x/PPLzl9mxn+lBIt17NgR7du3x3/+8x8AZ/+q1LBhQ9x333149NFHL/qz2dnZiIiIcHeX3CooKEisP/bYY2K9Y8eOhtqCBQvEtm+//Xb5O3YZDB06VKzfdtttYn3NmjVifebMme7qUqXJyspCeHi45Z+ryPgHPGMOVHdNmzYV60lJSWL91KlTYr2wsNBQ27Jli9j22LFjJnvnGco7/gHvPweQPfAcQHbG8U92Zmb8u/3t3EVFRdixYwcmT57srNWoUQNJSUnYtGmToX1hYaHLhWpubq67u+R2ureXBAQEiPXg4GBDzd/f3619ulz8/PzEurSPAOBwOCqzO5WqPG8jsjr+Ac+cA9Wdr6+vWNeNR918LCsrM9Qq8hZnd9ONUXf8bbS8b6OzwzmA7IHnALIzjn+yMzPj3+1Xg5mZmSgtLUV0dLRLPTo6GmlpaYb2U6dORXh4uPOrYcOG7u4S0WVjdfwDnAPkXXgOIDvjOYDsjOOf7KTKX1KZPHkysrOznV+HDh2q6i4RXVacA2RnHP9kd5wDZGcc/+Sp3P527rp168LX1xfp6eku9fT0dMTExBjaOxyOavuW31mzZon1Hj16iHXdW0gvfC4A4JlnnhHb3n///WJdd1D57bffDLWcnByxbe3atcV6ly5dxLr0FtewsDCx7dGjR8X6PffcI9YHDRpkqI0bN05s+/vvv4v16sjq+Aeq9xyoCu54i7LuM/ft27cX67qPKVj5vfz3v/8V623bthXrgYGBhto333wjtv373/8u1k+fPi3WpWNRaWmp2NbdvOkcQGQVzwFkZxz/ZCdufyXa398fiYmJWLt2rbNWVlaGtWvXonPnzu5+OKJqheOf7I5zgOyM45/sjOOf7KRS7hP94IMPYvTo0WjXrh06dOiA6dOnIz8/H7fffntlPBxRtcLxT3bHOUB2xvFPdsbxT3ZRKYvoUaNGISMjA0888QTS0tJw9dVXY9WqVYagASJvxPFPdsc5QHbG8U92xvFPdlEpi2gAmDBhAiZMmFBZmyeq1jj+ye44B8jOOP7Jzjj+yQ4qbRHtaXr37m2oJSQkiG2///57sR4aGirWpfvK/vjjj2LbyMhIsd64cWOxHhQUZKjt2LFDbHvVVVeJ9eLiYrGemZlpqH333Xdi26ioKLGempoq1iMiIgy1f//732LbYcOGiXXyTu4IFtP9xTsrK0us6+4TXVRUZKhJYxcAbr75ZrEuBYgB8rxr3bq12LakpESsT5w4UaxL+6MLISMiIiIia6r8FldEREREREREnoKLaCIiIiIiIiKTuIgmIiIiIiIiMomLaCIiIiIiIiKTuIgmIiIiIiIiMonp3P9fv379DLUDBw6IbR0Oh1jXJejWrGl8mqXk64ttQ5dY7Ovra6i1bNlSbHvmzBmxnp+fL9Zzc3MNtfr164ttCwoKxLqUTA4AR44cMdTCwsLEtl27dhXrGzZsEOvk2XRjpuz/tXfnsVmV6f/Hr4J0gS7sXYDSwoAYFxAs0AFGVBTJuLCoGeMYjeMYtWiUiZM4bomOwUi+o6OiRsdAnFFUEpHRmSFBZAkIyDoKCMNS9i6W7qV0Pb8/Zigp57r4ndM+5VnO+5XwBx9vT895et/nPDeFz9PaquZaE3V2drY61pqn2joS0Zu4rfVSXl6u5lazvtb8ba3zP/3pT2pusV4rAAAAdB4/iQYAAAAAwCM20QAAAAAAeMQmGgAAAAAAj9hEAwAAAADgEZtoAAAAAAA8op37f7KyslxZdXW1OtZq525qalJzrfnXOkZDQ4OaW43APXr0cGVWw29LS4uaW63YPXv2dGVWu7HW5C0i4jiOmmsNzNbYKVOmqDnt3NHNmqdWO7fl+uuvd2XJycnq2NraWjXXGr4t2pq70Ne01rrW2v/DDz/4OnZGRoaaFxcXuzK/recAAADQ8ZNoAAAAAAA8YhMNAAAAAIBHbKIBAAAAAPCITTQAAAAAAB6xiQYAAAAAwKPAtXNbDbVaQ3VVVZU61soTExM9n4fWzHuh3KI1BTc2NnoeK2K/Jtq5WGOtY9fX16u5xmoJHjlypOdjIHpYbezW/LXk5eW5Mq2dWkSksrJSza05pp2jNaf79+/v+RgieqP98uXL1bE33nijmm/fvl3Nteu32tABAADgDz+JBgAAAADAIzbRAAAAAAB4xCYaAAAAAACP2EQDAAAAAOBR4IrFcnNz1VwrzEpKSlLHWsViFRUVaq4VdPXr108d29zcrOYJCQlqrpUFWQVnVrFQU1OTmmvnbR3DKgWz8tOnT6u5ZtCgQZ7HInpYc8kq4rJMnTrV8zGsNfr111+r+bBhwzwfe8CAAWq+Y8cONb/66qtdmVXQ9/nnn6v5kSNH1FzT0tLieSxiU05OjpoPHjxYzdevX9+FZwMAQPTiJ9EAAAAAAHjEJhoAAAAAAI/YRAMAAAAA4BGbaAAAAAAAPGITDQAAAACAR4Fr587IyFDzhoYGV2Y1S1utwlZTbvfu3V1ZbW2tr2P36tVLzbU2b+u8/bRwi+gN2taxtddPRKS4uFjNe/bs6cpSUlLUsadOnVJzqw35p59+UnNEFm1diNgN9Zbhw4e7Mmu95Ofnq7k1x7T1WF1drY5ds2aNmlvNx0uWLHFlzzzzjDrWEqqGc8SWO++8U81feuklNV+xYoWaa232u3fv7viJXST33HOPK9u/f7869rvvvuvq0wEAxCB+Eg0AAAAAgEdsogEAAAAA8IhNNAAAAAAAHrGJBgAAAADAIzbRAAAAAAB4FLh27v79+6t5UVGRK0tLS1PHTpkyRc0/+ugjNT958qQry8zMVMcmJCSoeX19vZprjdtWM29LS4uaNzY2qnmPHj08fT0RkdLSUjWfOHGimmst3z/++KM6NjU1Vc0vvfRSNaedOzpY89EyefJkNdda2q0G4b59+6p5nz591FxrJx44cKA61mqi/9nPfqbm1nxHMHTrpv8ZtvUJCIMGDVLzP//5z67MaoQ/dOiQml955ZVq/t5777mySZMmqWP9Sk5OdmUPPPCAOtZ6biclJam59ukX2nMYF4/2SQKh+hSBxx9/3JVt375dHev3vYr2fuL7779Xx544ccI6xYvu6aefdmXWc/Hvf/97V58OEJP4STQAAAAAAB6xiQYAAAAAwCM20QAAAAAAeMQmGgAAAAAAj9hEAwAAAADgUeDaubUmXxG9KfS6665Tx1pNoddcc42ar1u3zpVdddVV6tjKyko1txpbtYZXq0E7Pj5ezbt3767miYmJrsxqNz569Kianz59Ws0nTJjg6euJiBw7dkzNx4wZo+br169Xc0QWv82s9957r5pra8Ca0+Xl5Wputd83Nze7Mq21/kLHsCxdutSV/d///Z869ne/+52aW69hVzbhIjS079GFWA3y2qcUHD58WB1rfXKBdj8W0Zvof/3rX6tjV69erea33HKLms+aNcuVWW3b2jNURGTx4sVqbrUQI3y0e7J2f72QadOmqfknn3ziyqy5PnPmTDUfPXq0mmvvYR599FF1rNV+v2XLFjXfunWrK9u7d686NicnR81vuOEGNR86dKgrs9YX7dzBYT13rLWlzemDBw/6OnYsv/fgJ9EAAAAAAHjEJhoAAAAAAI/YRAMAAAAA4BGbaAAAAAAAPPJdLLZu3TpZsGCBbNu2TYqKimTZsmXtihocx5EXXnhB3n//famsrJRJkybJO++8IyNGjAjleXfYX/7yFzVfuXKlK7OKXB5//HE1f+CBB9R81KhRruzMmTPq2MbGRjW3SsG0wjGr+Mj6R//W19QKNVJSUtSxeXl5an7XXXep+ZNPPunKBg8erI59+OGH1byhoUHNu1K0z/9w0UplWlpafB3jxhtvVPOysjJXZs2lnj17qrm1Zqy1oenXr5/nsSIif/3rX12ZVbSzfPlyNb/99tvVvKuKPII2/7XSOhH79fXzuvud/7t27VLziooKV3b55ZerYzds2KDm27dvV3NtTr/55pvq2OPHj6v5zp071Vwr0bOusaioSM0t2rPukkv0tztWEaclaGvAYq0NqwRVu7dp741E7PcN1n19xowZrswqabW+31aBqXbeVtFdVVWVmg8ZMkTNtfdNpaWl6ljrfvHZZ5+peWZmpisbOXKkOtYP5v+FhaNca9iwYa7s+eefV8dapZPXXnutmmulc6+99po6NpIKxAoKClyZ9Syynote+P5JdF1dnYwePVoWLlyo/vdXX31V3njjDXn33Xdl8+bN0qtXL5k+fbq5aQSiCfMfQcb8R9CxBhBkzH/gHN8/iZ4xY4b6p34i//1TiNdff12effbZtp+QfPjhh5Keni5ffPGF/OpXv+rc2QJhxvxHkDH/EXSsAQQZ8x84J6T/JrqwsFCKi4vbfd5YWlqaTJgwQTZu3Kj+Pw0NDVJdXd3uFxCNOjL/RVgDiA3MfwQdawBBxvxH0IR0E11cXCwiIunp6e3y9PT0tv92vvnz50taWlrbL+vfjgCRriPzX4Q1gNjA/EfQsQYQZMx/BE3Y27mffvppqaqqavtllTsAsYo1gCBj/iPoWAMIMuY/opXvfxN9IRkZGSIiUlJS0q4ZsKSkRMaMGaP+PwkJCZKQkBDK0+iQI0eOuLLZs2f7OsYPP/yg5lOmTHFlVpOp32Y/bbzVmmkd22r+1v5KjdVArLUvi4iUl5er+XPPPafm0awj818kctZAV/LT2njVVVepeW5urpofOnTIlSUmJqpjrXKTo0ePqvnw4cNdmbV2/bYta/ecSZMmqWM//vhjX8cOh2iZ/35aha2m4Ujy1FNPubKvv/5aHWu1udfU1Ki59ma2pKREHTt37lw1X7t2rZp3Je1+47eFuyOiZQ1Y7wW03Brr93538803uzLtUzpERN566y01P3jwoJpfeumlns/j/J+SnmU9o7RPdKitrVXHWveW+vp6z+Pr6urUsUuXLlVz6x6l/TTX+sSZ81vPW1tb5eTJk+rYC4mW+W+x5romFJ/QYH0qiPXpCrfddpuaa03sliuuuELNtRZuEX3OTJ48WR27fv16z+fh17hx49T87bffVnPtOq1PObmo7dwXkpubKxkZGbJq1aq2rLq6WjZv3iz5+fmh/FJAxGH+I8iY/wg61gCCjPmPoPH9k+ja2lo5cOBA2+8LCwtl586d0rdvX8nOzpYnnnhC/vjHP8qIESMkNzdXnnvuOcnKymr3OXJAtGL+I8iY/wg61gCCjPkPnON7E71161a57rrr2n4/b948ERG57777ZPHixfL73/9e6urq5KGHHpLKykqZPHmyrFixwvzrlUA0Yf4jyJj/CDrWAIKM+Q+c43sTPXXq1Av+nf+4uDh58cUX5cUXX+zUiQGRiPmPIGP+I+hYAwgy5j9wTtjbuQEAAAAAiBYhbeeOBlb7ntaSaDUtWi2fVju31uRo/Ume9TWtFr/m5mZXZrU1Wse2mrW1czx9+rQ69vx2x46wzsPitx0U4eOn5fimm25Sc2vdNTY2ujKrhfuSS/RbXkpKipprjaFFRUXq2AEDBqi5dd7Z2dmu7KWXXlLHWhYvXqzm999/v6/jRKrz79d+mk+te72fuXi2bfZ89957r5rPmDFDza+//nrPX9OvzZs3u7LPPvtMHWudn3Uv1Z4Z1jPgjjvuUHM/7dzWMyAtLU3Nk5OT1TwpKcmVZWVlqWMrKipcWUtLi/k8jwV+1ob1vsFqxN63b5+aP//8867sgQceUMda31ftkxhERD766CM1D4XevXu7sunTp6tjrQbqYcOGqbn23tBqILeeL1bbuNYqrj0rRdzv35qbmzvUzt0Vzp+r2twNRVN2R8b7oT3vX375ZXWsdR+0PkXk8OHDrsz6ZBzrkxhuvfVWNa+srHRls2bNUsdOnDhRzU+dOqXm2nsj674ydOhQNbcawXNyclyZ1UzeGfwkGgAAAAAAj9hEAwAAAADgEZtoAAAAAAA8YhMNAAAAAIBHbKIBAAAAAPAocO3cVvue1k7qp8VVRKSurs7zWKsl0fpAeq2FW0Rv8bOu0W9brXYu1jVaDcR+WOfRlY2JCC2rVVJbX1rztYjI448/ruY7d+5Uc63NMT4+Xh1rrbvq6mo115SVlam51cBqtdtqX9Nq1T5y5IiaT506Vc1vueUWV/bVV1+pY6OJ9lqG6v7w+uuvu7K8vDx1rNVwqjX5ioi8/fbbruzRRx/1fG5+Pfzww2p+9913q/m1116r5lrDaWpqqjrWmrtaK62IyMqVK12Z9SkPVnu+n0+tsJ5R+/fvd2WNjY1R1c5tPduttWE9a8ePH+/K+vTpo46dO3eumn/zzTdqrt2TtPklIvK3v/1Nza0GeI31SQzWeymL1k786aefqmOt3GoFLigocGU33nijOtaa61brvPYM1JroI935czgU93vrmay1wvfv318da93XrPUycuRIV3b8+HF17L///W81t9rftet55JFH1LHafBbR78cW615q3b+HDBmi5tocbWhoUMeWlpaqea9evdT8n//8pyuzmuzPP4bjOOanUJyPn0QDAAAAAOARm2gAAAAAADxiEw0AAAAAgEdsogEAAAAA8ChwxWJ+WCVJVjFFKEpOrH8kb43Xypms87PKFLTSJxGRpKQkV2b9o////Oc/au6H33IURB5rLmmeffZZNbdKKKxCjKNHj7qyUaNGqWOtNVpbW6vmflhlPda608ZbZRZWUZq1HmfMmOHKrHKmJUuWqHkk6IpSGcvu3btd2T333KOOte53Bw8eVPOZM2e6sldeeUUdq81nv6znhVVYY5WcafNuw4YN6tgdO3aouVXQVVhY6Mq+++47z+dxIdozsF+/furYn376yZX5uY91pbi4ONdzUVsDfteFVTqkFX1p60JEZM2aNWpuFWOtXr3alU2ePFkdq5UCieglcBa/Bat+xvt9va2yzMzMTFdmFaJppVciImlpaWquPS+t59zJkyfb/d5vqe7FpD1Pree93/IvrRiyZ8+e6lirXNLaN2jzaM+ePerYKVOmqHl5ebmaa6Vb2n1NxH4fcOLECTXXWPsU6/liFW5qRcXWsa3vg/YcEdGLW7XiRBF3eVxrayvFYgAAAAAAhBqbaAAAAAAAPGITDQAAAACAR2yiAQAAAADwiE00AAAAAAAe0c4dQllZWWquNaUmJib6Orbf1m6N1bhoNRZrx/bb8D148GA1P378uCuzWjMRefw0Tlvuv/9+Na+oqFBzay5pLZnff/+9OnbEiBFqbrV1Hj582JVZLamNjY1qbtEaXq0GSqtVddWqVWpeUFDg61wiUY8ePVz3BO31qaqqUv9/vw2677//viu7++671bFWM/GLL76o5ps2bXJl06dP93weInZr/cSJE13ZsGHD1LHWc8daL1u2bHFl1ly0jq2tIRGRvLw8V2Y13lrtrlaTudaQO2jQIHXsv/71L1cWKe3cjuN0SSO99vwV0RvWrbZoq6F+165daq7N3+3bt6tjrXu99WkEmlB9D/28/taa/u1vf6vmK1ascGXWM+rUqVNqbrUIa/dF6zU5v507Uj4R5a677nI188+fP981btGiRer/bzVUa83NIvprXFZWpo617oMDBgxQc+04ffv2VcdaLddWO712733rrbfUsdb7bGuPoR3bev9n3act2mtlvX7WXsdqG9eOo336QGfxk2gAAAAAADxiEw0AAAAAgEdsogEAAAAA8IhNNAAAAAAAHrGJBgAAAADAI9q5L8BvQ2F+fr6aa61y5zcOnqW1iorYrZRJSUmex/pt59ZaH63z085DRGTgwIFqrrWD+m3+RmhprY3WGvDTwi0icuutt7oyq4G1srJSza05lpqa6sqsBm2r9dJaj0OHDnVl1hqwztt6rfzM60OHDqn5gw8+6PkY0SYnJ8f1Wt98882ucVbbqHU/sdps6+rqXFlCQoI6dubMmWputUtrze3vvfeeOtZqbLVakrXny969e9Wx1vVoLcEieoO21exssRpb161b58pGjx6tjrVa6K3Gbe2ZZrVJa+vT7/0t2owfP17N/bTrZmRkqPmZM2fUXGtpt75/w4cP93weFmu9ZGZmqrn2HBHRn1PWpyhY1zN79mw1P3bsmCuzPp1Cuz+J2K3F2j3HOu/m5mY1D7eVK1e67uPauWr3KRGRyy+/vNPnYLVwW+9HcnNz1Vz7Pln3Y+v7ZH1N7T2J9T4lLS3N19fU3qdY88Vac9ZzUXttrXnu956szf/09HR17NixY9v9vqmpSV2bGn4SDQAAAACAR2yiAQAAAADwiE00AAAAAAAesYkGAAAAAMAjNtEAAAAAAHgU5/itoO5i1dXVZnvcxWa1vlov2cKFC9U8MTHRlVlNeFZTttVul5KS4srq6+vVsVa7Xa9evTyPt5ptrWN8/vnnar5kyRJXZjX7has5sqqqymzs7EqRtAZCZceOHa7MaoLdv3+/mg8ZMkTNtfWVk5Pj+dxE7OZereXbane12rattlptvPV937lzp5pfffXVaq7xez8L9/wfNGiQq5lVa1u1vneXXnqpeXyN1jbcp08fdaw1R61mVu3+nZ2drY7V1oqI3SCvfRrDiBEj1LFHjhxRc62FXkR/lljPAOv8rGea1p5ufbKE1eRvte1rx87KylLHas3kra2tUlpaGvY1MGHCBNdzUWt6Li4uVo9jfWJA//79PedWI7D1CQhWrr2fsBr0rYbejz76SM219w7W+wY/73es42jNvyL2M8DKtWeg9p5OxF5HVq61QVuflvLBBx+0+31ra6uUlZWFff6npKS4nlvW/TtSWN8PbX5Z7dxW47r1DNfeA1msuWh9TS2PsG1jp52/5hzHkdraWk/zn59EAwAAAADgEZtoAAAAAAA8YhMNAAAAAIBHbKIBAAAAAPBIb3IKIK3gwiqasAqwrNIGrVjI+of5VnGARSsm8Ft6YZV7aIUCfsaK2OU+Guv1RvhY32/re3XNNdeo+ejRo11ZWVmZOjYvL0/NKyoq1LywsNCVHThwQB1rlbaMHTtWzbVCqA0bNqhjJ06cqOZWeYh2bOt1raqqUnM/oq0MpL6+3jX/Bg0a5BpnlX9ZJW3l5eVqrh2ntLRUHWsVXVkFcFqJmFUgduWVV6q5Nf+1eXTixAl1rFWuZRXTaMVi1ny2cuseor221rPVmv9aGZx1HOvZqhWlRcpa2bt3r+u8tblnfV/79eun5lYRWVFRkSuz5oZVuGOVlmnzwyr5subMM888o+aHDx/2fGxrjln8vCe76qqr1Nyav1puzT2rsMqye/duV2bdtz788MN2v4+U92LavU2bd9Zc7N69u5pb74W1Ejm/x7Bo31frnumnjFTEXi8aaz77mefWWOs8rFy7fmt9+n29ta9plV+e/yxqbW2V2tpab1/H11kBAAAAABBgbKIBAAAAAPCITTQAAAAAAB6xiQYAAAAAwCM20QAAAAAAeEQ79//4aaazmgBPnTql5gMGDHBlWuugiN0ebLVf+2nls1oGrWvXxlvNkVaj3vDhwz2end0IaZ1fpDSoRiq/TYna6++3pfPVV19V84aGBldmff+sZkqrWTQnJ8fT1xMR2bdvn5rv2bNHzdPT013Z0KFD1bG7du1Sc6uhXnttrXVuNTPHspqaGtfa116z6upq9f+35rk1N7TjWE3e1n36sssu83wu1qc57N+/X82t+7fWnmxde2VlpZrv3btXzfv27evKtAZnEZFRo0apufV6a+dttRhbz9akpCQ1P3r0qOfz01pYI+XZor0en376aaeP62cuWS3XycnJam7NPe19kzU3/DZR9+7d25VZ7xus+Ws9d7T3Nj179vR8HiL2663d7617i9VObN3/tNb548ePq2Ot+1wk0q7Xeg2ArsZPogEAAAAA8IhNNAAAAAAAHrGJBgAAAADAIzbRAAAAAAB45GsTPX/+fMnLy5OUlBQZOHCgzJw501XUc+bMGSkoKJB+/fpJcnKyzJkzR0pKSkJ60kC4sAYQZMx/BBnzH0HHGgDO8dXOvXbtWikoKJC8vDxpbm6WP/zhD3LTTTfJnj172hocn3zySfnHP/4hS5culbS0NJk7d67Mnj1bNmzY0CUXECp+2rmHDBmi5larotb0mZCQoI6Nj4/3fAxrvHVsq33SOrbWfGq1ijc3N6u51TastW/6bSC32iq7UrjXgDZPrSbTxsZGNffbuK156qmn1HzChAlqvnbtWlf285//XB1rzSWruVdrPrVek8zMTDW3mpI1Dz74oJpPnDhRzceMGaPm2lqyWu5/+uknbyfXxS7m/NfuB1q7tNW6bM1/rXFaRG8PrqurU8f6+UQDEf0ea83RPn36qLn2KQ8i+v3eehZZ5209d7T5aB3Dasi17jfaWrQa+Ovr69XcanfWXu8dO3b4Oj9NuO//oWI9O7X5bq2B0tLSkJ4TokOsrAEgFHxtolesWNHu94sXL5aBAwfKtm3b5Be/+IVUVVXJBx98IB9//LFcf/31IiKyaNEiueyyy2TTpk3mm0wgWrAGEGTMfwQZ8x9BxxoAzunUv4k++5Ohs3+yv23bNmlqapJp06a1jRk1apRkZ2fLxo0b1WM0NDRIdXV1u19AtGANIMiY/wiyUMx/EdYAohfPAARZhzfRra2t8sQTT8ikSZPkiiuuEBGR4uJiiY+Pd33gfHp6uhQXF6vHmT9/vqSlpbX9sv6qNBBpWAMIMuY/gixU81+ENYDoxDMAQdfhTXRBQYHs2rVLPvnkk06dwNNPPy1VVVVtv44dO9ap4wEXC2sAQcb8R5CFav6LsAYQnXgGIOh8/Zvos+bOnStfffWVrFu3rl0RSEZGhjQ2NkplZWW7P4UqKSmRjIwM9VgJCQlmERYQqVgDCDLmP4IslPNfhDWA6MMzAPC5iXYcRx577DFZtmyZrFmzRnJzc9v993HjxkmPHj1k1apVMmfOHBER2bdvnxw9elTy8/NDd9ZhZrXBau2uIiIVFRWuzGpgtRplrdZeLddatUXsdm7ra57/13EuNNY6dmJiopqnpaW5srKyMnWsn+b0rhbuNaC1zlrfE4vVIKz9FarHHntMHTtv3jw1//bbb9Vce3haY8eOHavmycnJau7n+v02k992222u7Msvv1THzpgxw9extXOx5rrVTG7RjmO18PsR7vm/bNkyV2a1948YMULNrTdyWkP7sGHD1LFWY7F1v9Ma57WmcRG7tbuwsFDNy8vLXVltba3n8xAR869Zas+uULT7i+j3Iasl3/rkBuuNt9Y+7XcNacI9/4FwYw0A5/jaRBcUFMjHH38sy5cvl5SUlLYHb1pamiQlJUlaWpr85je/kXnz5knfvn0lNTVVHnvsMcnPz6eRDzGBNYAgY/4jyJj/CDrWAHCOr030O++8IyIiU6dObZcvWrRI7r//fhERee2116Rbt24yZ84caWhokOnTp8vbb78dkpMFwo01gCBj/iPImP8IOtYAcI7vv879/5OYmCgLFy6UhQsXdvikgEjFGkCQMf8RZMx/BB1rADinU58TDQAAAABAkHSonTvozn6o/PmsUhmtFEUr1hIROXXqlJpbxWLanwpaRTtWYY1VQqOdY01NjTrWKquycq3cxyoWw4Xdcccdar5o0SI1t74nWiGd9afOVknP2c+KPN+2bdtc2ZVXXqmOPXDggJpb47XrsYqIrFKpWbNmqblVIqax1qgf1ut98uRJX8fR7gFa2VIssIqu9u3b5yvHxaXNx6KiojCcCQAA/vGTaAAAAAAAPGITDQAAAACAR2yiAQAAAADwiE00AAAAAAAesYkGAAAAAMAj2rn/Jy4uzvPY3NxcNW9sbPR87F69eqljDx06pOYJCQkez04kNTVVzSsqKtTcOu+UlBRXpjU4i4g0NDSoufW6Jicnq7mfYwRRZmamK1uwYIE6trm5Wc2thnWrcVtjNXxb8zQ/P9+Vbdq0SR07bNgwNbfOe+DAga7Mml+ff/65mn/xxRdq7oe1jiza98dq566srPR1bNYMAABA1+En0QAAAAAAeMQmGgAAAAAAj9hEAwAAAADgEZtoAAAAAAA8YhMNAAAAAIBHtHN3QEtLi5pbDdVao7XV5NvU1KTm8fHxaq61fPft21cdW1hY6OvYmm7d9D93sV6THj16eD62368ZRLfddpsrs77fxcXFat6zZ0811xq3ExMTPY8VseeB1hZ9zTXXqGNPnDih5lu2bFHzcePGubKcnBx17Jw5c9TcorWNW+u8rq6u08e2WN9LAAAAXHzsTgAAAAAA8IhNNAAAAAAAHrGJBgAAAADAIzbRAAAAAAB4xCYaAAAAAACPaOfuAKtZ209DdWlpqTq2tbVVza1GYO3Y1nmUl5erudXWXFtb68qspmzrvC1nzpzxPNbvsWPZhx9+6MruvPNOdexll12m5ikpKWruOI4r89vGbn2vTp8+7cqshu/hw4er+YABA9S8d+/eruy6665Tx/rV3NzseazVrO9n/CWX6Ldkv83f2mvr51oAAABg4yfRAAAAAAB4xCYaAAAAAACP2EQDAAAAAOARm2gAAAAAADyiWKwDRo4cqeZawZGIXiBkje3Tp4+ax8fHq3n//v1dWWpqqjp2xIgRaj5w4EA1v/rqq13Zt99+q461yqri4uLU3Cpnw4XV19e7smnTpqljBw8erOb33Xefmt9yyy2ubOzYsepYaz52pcTERDX/5S9/6crWrFnTxWfjtn//fl/jtXvAwYMH1bG7d+/2dWyr+A0AAACdx0+iAQAAAADwiE00AAAAAAAesYkGAAAAAMAjNtEAAAAAAHjEJhoAAAAAAI9o5/6f1tZWz2O3bt2q5lpTtohIaWmpKztz5ow6tqysTM2bm5vVfNCgQa4sMzNTHbt9+3Y1t5qWc3JyXJnjOOrY06dPq/mYMWPUvLi4WM01fr43OOf48eNq/vLLL/vKNVZD/bBhw9Rca50vLy9Xx1oN1QcOHPB4duGxYMECNd+yZYuaa/cF6zU5deqUr3Ox7hcAAADoPH4SDQAAAACAR2yiAQAAAADwiE00AAAAAAAesYkGAAAAAMCjiCsWs4qrIunrNjQ0qLlVFqaNb2xsVMc2NTWpuVUUpB3HOg/r2HFxcWqunbf1OlmvSX19vZr7eb2jYU7Ewtf1o6WlRc2tearNPWtsJBXJheJ7YV2n9hrG2rVH09cFzscaQJAx/xFkXuZhxG2ia2pqwvJ1/bx5feutt7rwTHC+cG0sampqJC0tLSxfN9JZDdpWHq1CMffWr18fgjO5+Jj/CDrWAIKM+Y8g8zL/45wI+yOf1tZWOXnypKSkpEhNTY0MGTJEjh07JqmpqeE+tS5RXV0d89coEl3X6TiO1NTUSFZWlnTrdvH/xQNrIPZE0zUy/y+uaJobnRFN1xkpa8BxHMnOzo6K16wzomludFQ0XWOkzH+eAbEjmq7Rz/yPuJ9Ed+vWTQYPHiwi5/6acWpqasS/6J0VhGsUiZ7rDMefvp7FGohd0XKNzP+LLwjXKBI91xkJa6C6ulpEouc166wgXGe0XGMkzH8RngGxJlqu0ev8p1gMAAAAAACP2EQDAAAAAOBRRG+iExIS5IUXXpCEhIRwn0qXCcI1igTnOkMtCK8b1whLEF63IFyjSHCuM5SC8poF4TqDcI1dIQivG9cYvSKuWAwAAAAAgEgV0T+JBgAAAAAgkrCJBgAAAADAIzbRAAAAAAB4xCYaAAAAAACPInoTvXDhQsnJyZHExESZMGGCfPfdd+E+pQ5bt26d3HrrrZKVlSVxcXHyxRdftPvvjuPI888/L5mZmZKUlCTTpk2T/fv3h+dkO2j+/PmSl5cnKSkpMnDgQJk5c6bs27ev3ZgzZ85IQUGB9OvXT5KTk2XOnDlSUlISpjOObLE0/0Vifw0w/0MvltZArM9/EdZAqDH/mf9BFkvzX4Q1cFYsrYGI3UR/+umnMm/ePHnhhRdk+/btMnr0aJk+fbqUlpaG+9Q6pK6uTkaPHi0LFy5U//urr74qb7zxhrz77ruyefNm6dWrl0yfPl3OnDlzkc+049auXSsFBQWyadMmWblypTQ1NclNN90kdXV1bWOefPJJ+fLLL2Xp0qWydu1aOXnypMyePTuMZx2ZYm3+i8T+GmD+h1asrYFYn/8irIFQYv4z/4Ms1ua/CGvgrJhaA06EGj9+vFNQUND2+5aWFicrK8uZP39+GM8qNETEWbZsWdvvW1tbnYyMDGfBggVtWWVlpZOQkOAsWbIkDGcYGqWlpY6IOGvXrnUc57/X1KNHD2fp0qVtY3788UdHRJyNGzeG6zQjUizPf8cJxhpg/ndOLK+BIMx/x2ENdAbzn/kfZLE8/x2HNRArayAifxLd2Ngo27Ztk2nTprVl3bp1k2nTpsnGjRvDeGZdo7CwUIqLi9tdb1pamkyYMCGqr7eqqkpERPr27SsiItu2bZOmpqZ21zlq1CjJzs6O6usMtaDNf5HYXAPM/44L2hqIxfkvwhroKOY/8z/Igjb/RVgD0boGInITXVZWJi0tLZKent4uT09Pl+Li4jCdVdc5e02xdL2tra3yxBNPyKRJk+SKK64Qkf9eZ3x8vPTu3bvd2Gi+zq4QtPkvEntrgPnfOUFbA7E2/0VYA53B/Je230fr9TL/Oy5o81+ENRCt13lJuE8AsamgoEB27dol69evD/epABcd8x9BxxpAkDH/EXRBWAMR+ZPo/v37S/fu3V1tbSUlJZKRkRGms+o6Z68pVq537ty58tVXX8nq1atl8ODBbXlGRoY0NjZKZWVlu/HRep1dJWjzXyS21gDzv/OCtgZiaf6LsAY6i/kvbb+Pxutl/ndO0Oa/CGsgWq8zIjfR8fHxMm7cOFm1alVb1traKqtWrZL8/PwwnlnXyM3NlYyMjHbXW11dLZs3b46q63UcR+bOnSvLli2Tb775RnJzc9v993HjxkmPHj3aXee+ffvk6NGjUXWdXS1o818kNtYA8z90grYGYmH+i7AGQoX5z/wPsqDNfxHWQNSugbDWml3AJ5984iQkJDiLFy929uzZ4zz00ENO7969neLi4nCfWofU1NQ4O3bscHbs2OGIiPOnP/3J2bFjh3PkyBHHcRznlVdecXr37u0sX77c+f77753bb7/dyc3Nderr68N85t498sgjTlpamrNmzRqnqKio7dfp06fbxjz88MNOdna288033zhbt2518vPznfz8/DCedWSKtfnvOLG/Bpj/oRVrayDW57/jsAZCifnP/A+yWJv/jsMaOCuW1kDEbqIdx3HefPNNJzs724mPj3fGjx/vbNq0Kdyn1GGrV692RMT167777nMc57/19s8995yTnp7uJCQkODfccIOzb9++8J60T9r1iYizaNGitjH19fXOo48+6vTp08fp2bOnM2vWLKeoqCh8Jx3BYmn+O07srwHmf+jF0hqI9fnvOKyBUGP+M/+DLJbmv+OwBs6KpTUQ5ziOE5qfaQMAAAAAENsi8t9EAwAAAAAQidhEAwAAAADgEZtoAAAAAAA8YhMNAAAAAIBHbKIBAAAAAPCITTQAAAAAAB6xiQYAAAAAwCM20QAAAAAAeMQmGgAAAAAAj9hEAwAAAADgEZtoAAAAAAA8YhMNAAAAAIBH/w9OYacftrZBWQAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "fig = plt.figure(figsize=(12, 8))\n", "for i in range(15):\n", " image, label = training_set[i]\n", " plt.subplot(3, 5, i+1)\n", " plt.imshow(image[0,:,:], cmap='gray')\n", " plt.title(class_names[label])" ] }, { "cell_type": "markdown", "id": "19e7ce9d-5da0-4b7a-a9ab-6b97e46cf013", "metadata": { "id": "19e7ce9d-5da0-4b7a-a9ab-6b97e46cf013" }, "source": [ "Next, we will split the training dataset into a train and validation dataset. The model will not use the validation data for training, only for evaluating the loss and any model metrics on this data at the end of each epoch during training to monitor the network performance for signs of overfitting. We will use the first 48,000 data points for the training the model and the remaining data points, from index 48,000 to the end of the dataset, are used for validating the model (valid_set)." ] }, { "cell_type": "code", "execution_count": 12, "id": "41944f1e-ce00-4327-a3b3-299b681a92ec", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "41944f1e-ce00-4327-a3b3-299b681a92ec", "outputId": "c7a9e9c9-29fc-4347-c648-771623d709bf" }, "outputs": [ { "data": { "text/plain": [ "(48000, 12000)" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "torch.manual_seed(0)\n", "\n", "train_set = Subset(training_set, torch.arange(48000))\n", "valid_set = Subset(training_set, torch.arange(48000, len(training_set)))\n", "len(train_set), len(valid_set)" ] }, { "cell_type": "markdown", "id": "0a038901-6fe2-4da3-a8c0-f07e4995ddeb", "metadata": { "id": "0a038901-6fe2-4da3-a8c0-f07e4995ddeb" }, "source": [ "Next, we will pass the dataset as an argument to `DataLoader`. This wraps an iterable over our dataset, and supports automatic batching, sampling, shuffling and multiprocess data loading.\n", "We cannot train the model on 60,000 images at once. Hence, we need to create smaller training batches because our CPU or GPU usually can't handle such a large batch. So, we split our data into sets of 64 items per batch. With the code below, each element in the dataloader itereable will return a batch of 64 datapoints. When training a neural network using stochastic gradient descent optimization, it is important to feed training data as a randomly shuffled batch, hence we will set the *shuffle* parameter to *True* for the training dataset, so it tells PyTorch to randomly reorder the dataset at the beginning of each epoch. We do not shuffle the test set, to get deterministic results and to allow for comparisons. If the validation set is shuffled, slight changes in performance could be due to the shuffling rather than changes in the model or training process. Additionally, since shuffling of training data is used to prevent the model from overfitting, this concern is not applicable to the testset." ] }, { "cell_type": "code", "execution_count": 13, "id": "a72fb5a3-0193-492e-8682-d65027056e5b", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "a72fb5a3-0193-492e-8682-d65027056e5b", "outputId": "1e906fd9-3c32-4123-b313-e249504e35d2" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/usr/local/lib/python3.11/dist-packages/torch/utils/data/dataloader.py:624: UserWarning: This DataLoader will create 4 worker processes in total. Our suggested max number of worker in current system is 2, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.\n", " warnings.warn(\n" ] } ], "source": [ "batch_size = 64\n", "torch.manual_seed(0)\n", "train_loader= DataLoader(train_set, batch_size = batch_size, shuffle=True, num_workers=4, persistent_workers=True)\n", "val_loader = DataLoader(valid_set, batch_size=batch_size, num_workers=4, persistent_workers=True)\n", "test_loader = DataLoader(test_set, batch_size=batch_size, num_workers=4, persistent_workers=True)" ] }, { "cell_type": "markdown", "id": "730f11ee-ca7a-4008-a729-0128cff75365", "metadata": { "id": "730f11ee-ca7a-4008-a729-0128cff75365" }, "source": [ "The num_workers parameter specifies how many subprocesses (CPU threads) to use for loading the data in parallel. If num_workers=0 then data loading is done in the main (training) process.\n", "This is simple and safe, but slower, especially with large datasets or complex transformations." ] }, { "cell_type": "markdown", "id": "c994dd42-b201-4d42-aa99-d81eb599ed34", "metadata": { "id": "c994dd42-b201-4d42-aa99-d81eb599ed34" }, "source": [ "### Setting up the PyTorch Lightning model" ] }, { "cell_type": "markdown", "id": "ecc9571d-9581-495a-b64f-b3d51dfde891", "metadata": { "id": "ecc9571d-9581-495a-b64f-b3d51dfde891" }, "source": [ "We will start by creating a new class that inherits properties and behaviors from LightningModule, giving access to Lightning's built-in training loop, logging, checkpointing, etc." ] }, { "cell_type": "code", "execution_count": 14, "id": "824f783a-3eb1-45b8-a3da-e09ba3aed465", "metadata": { "id": "824f783a-3eb1-45b8-a3da-e09ba3aed465" }, "outputs": [], "source": [ "class MNISTModel(pl.LightningModule):\n", " def __init__(self):\n", " super().__init__()\n", " self.model = nn.Sequential(\n", " nn.Flatten(),\n", " nn.Linear(28*28, 128),\n", " nn.ReLU(),\n", " nn.Linear(128, 10)\n", " )\n", " self.loss_fn = nn.CrossEntropyLoss()\n", " self.train_acc = Accuracy(task=\"multiclass\", num_classes=10)\n", " self.val_acc = Accuracy(task=\"multiclass\", num_classes=10)\n", " self.test_acc = Accuracy(task=\"multiclass\", num_classes=10)\n", "\n", "\n", " def forward(self, x):\n", " return self.model(x)\n", "\n", " def training_step(self, batch, batch_idx):\n", " x, y = batch\n", " logits = self(x)\n", " loss = self.loss_fn(logits, y)\n", " preds = torch.argmax(logits, dim=1)\n", " acc = self.train_acc(preds, y)\n", " self.log(\"train_loss\", loss, prog_bar=True, on_step=False, on_epoch=True)\n", " self.log(\"train_acc\", acc, prog_bar=False, on_step=False, on_epoch=True)\n", " return loss\n", "\n", "\n", " def validation_step(self, batch, batch_idx):\n", " x, y = batch\n", " logits = self(x)\n", " loss = self.loss_fn(logits, y)\n", " preds = torch.argmax(logits, dim=1)\n", " acc = self.val_acc(preds, y)\n", " self.log(\"val_loss\", loss, on_step=False, on_epoch=True, prog_bar=False)\n", " self.log(\"val_acc\", acc, on_step=False, on_epoch=True, prog_bar=False)\n", " return loss\n", "\n", " def test_step(self, batch, batch_idx):\n", " x, y = batch\n", " logits = self(x)\n", " loss = self.loss_fn(logits, y)\n", " preds = torch.argmax(logits, dim=1)\n", " acc = self.test_acc(preds, y)\n", " self.log(\"test_acc\", acc, on_step=False, on_epoch=True)\n", " return loss\n", "\n", " def configure_optimizers(self):\n", " return optim.Adam(self.parameters(), lr=0.001)" ] }, { "cell_type": "markdown", "id": "6b9cc29d-1266-4168-8c42-2d2bf14363a0", "metadata": { "id": "6b9cc29d-1266-4168-8c42-2d2bf14363a0" }, "source": [ "Let's look at the code step by step." ] }, { "cell_type": "markdown", "id": "dd6d3106-fa45-4cbf-8bb8-ded52ca69b32", "metadata": { "id": "dd6d3106-fa45-4cbf-8bb8-ded52ca69b32" }, "source": [ "With `init` method we initialize the model, Now, let's see the network we defined.\n", "\n", "*nn.Sequential*: a sequence of modules or layers in a specific order, where the output of one module becomes the input for the next. These layers in sequence are:\n", "- *nn.Flatten*: this layer transforms a multi-dimensional input (like an image) into a one-dimensional tensor, which is necessary for feeding data into a fully connected linear layer.\n", "- *nn.Linear(28 * 28, 128)*: a fully connected linear layer that takes an input of size $28*28$ (the flattened image) and outputs a tensor of size 128, with a *nn.ReLU()* which is a non-linear activation function (ReLU - Rectified Linear Unit) that introduces non-linearity to the model, allowing it to learn more complex patterns.\n", "- *nn.Linear(128, 10)*: The second linear layer that reduces the size from 128 to 10, which could correspond to the number of classes in our classification task (10 different types of clothing in the Fashion MNIST dataset). " ] }, { "cell_type": "markdown", "id": "7c0e8029-a331-4e7d-9d88-f8b3722dc884", "metadata": { "id": "7c0e8029-a331-4e7d-9d88-f8b3722dc884" }, "source": [ "To better understand this architecture, let's visualize it:" ] }, { "cell_type": "code", "execution_count": 31, "id": "86d662d7-c5ed-45ba-b8fd-ecd921b33ba7", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "86d662d7-c5ed-45ba-b8fd-ecd921b33ba7", "outputId": "7a41b9b1-3a13-42c5-cd9d-a196a26f69e7" }, "outputs": [ { "data": { "text/plain": [ "==========================================================================================\n", "Layer (type:depth-idx) Output Shape Param #\n", "==========================================================================================\n", "MNISTModel [1, 10] --\n", "├─Sequential: 1-1 [1, 10] --\n", "│ └─Flatten: 2-1 [1, 784] --\n", "│ └─Linear: 2-2 [1, 128] 100,480\n", "│ └─ReLU: 2-3 [1, 128] --\n", "│ └─Linear: 2-4 [1, 10] 1,290\n", "==========================================================================================\n", "Total params: 101,770\n", "Trainable params: 101,770\n", "Non-trainable params: 0\n", "Total mult-adds (Units.MEGABYTES): 0.10\n", "==========================================================================================\n", "Input size (MB): 0.00\n", "Forward/backward pass size (MB): 0.00\n", "Params size (MB): 0.41\n", "Estimated Total Size (MB): 0.41\n", "==========================================================================================" ] }, "execution_count": 31, "metadata": {}, "output_type": "execute_result" } ], "source": [ "model = MNISTModel()\n", "\n", "summary(model, input_size=(1, 28, 28))" ] }, { "cell_type": "markdown", "id": "96587b0e", "metadata": { "id": "96587b0e" }, "source": [ "Note that fully connected, linear, layers often have a lot of parameters. The number of parameters in the previous case can be explained as follows:\n", "\n", "- Flatten Layer: This layer converts the 2D input (28x28 image) into a 1D array of 784 values. Since it doesn't have any parameters itself, it doesn't add any parameters.\n", "\n", "- First Linear Layer (Hidden Layer): This layer has 128 neurons. Each neuron in this layer is connected to every neuron in the previous layer (784 neurons in the flattened layer). As such, there are 784 * 128 weights (connection weights) between the input and hidden layer; additionally, there's one bias term per neuron in the hidden layer, thus there are 128 bias terms. In total, this layer has 784 * 128 + 128 = 100,480 parameters.\n", "\n", "- ReLU Activation Function: This layer doesn't add any parameters. It simply applies the rectified linear unit function element-wise to the output of the previous layer.\n", "\n", "- Second Linear Layer (Output Layer): This layer has 10 neurons. Each neuron in this layer is connected to every neuron in the previous layer (128 neurons in the hidden layer). Therefore, there are 128 * 10 weights (connection weights) between the hidden and output layer; additionally, there's one bias term per neuron in the output layer, thus there are 10 bias terms. In total, this layer has 128 * 10 + 10 = 1,290 parameters.\n", "\n", "\n", "This gives the model quite a lot of flexibility to fit the training data, but it also means that the model runs the risk of overfitting, especially when we do not have a lot of training data." ] }, { "cell_type": "markdown", "id": "38848242-c9cd-4203-b9f6-1984d73eed17", "metadata": { "id": "38848242-c9cd-4203-b9f6-1984d73eed17" }, "source": [ "We can also get a slightly less detailed view of the network architecture with a simple print:" ] }, { "cell_type": "code", "execution_count": 16, "id": "ac7314a1-a5c5-40ee-9069-de213899072d", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "ac7314a1-a5c5-40ee-9069-de213899072d", "outputId": "467f07aa-c943-43b9-e6cf-289393ad5a45" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "MNISTModel(\n", " (model): Sequential(\n", " (0): Flatten(start_dim=1, end_dim=-1)\n", " (1): Linear(in_features=784, out_features=128, bias=True)\n", " (2): ReLU()\n", " (3): Linear(in_features=128, out_features=10, bias=True)\n", " )\n", " (loss_fn): CrossEntropyLoss()\n", " (train_acc): MulticlassAccuracy()\n", " (val_acc): MulticlassAccuracy()\n", " (test_acc): MulticlassAccuracy()\n", ")\n" ] } ], "source": [ "print(model)" ] }, { "cell_type": "markdown", "id": "fb02b51a-6de6-4acc-a168-aa09e0843993", "metadata": { "id": "fb02b51a-6de6-4acc-a168-aa09e0843993" }, "source": [ "Next we defined Loss function which measures how accurate the model is during training. We want to minimize this function to \"steer\" the model in the right direction.\n", "\n", "We will use *CrossEntropyLoss* which is widely used in multi-class classification problems. Other common loss functions are *MSELoss* (Mean Square Error) for regression tasks, and *NLLLoss* (Negative Log Likelihood) also for classification. Note that CrossEntropyLoss expects logits, not probabilities, as it internally applies softmax, hence we did not use as the last layer Softmax function to convert logits to probabilities.\n", "\n", "To track accuracy during training, validation and test we use torchmetrics.Accuracy, and task=\"multiclass\" specifies it's a classification task with 10 distinct classes. Note that torchmetrics.Accuracy supports raw logits for classification tasks, so we could pass logits, as well as the class predictions." ] }, { "cell_type": "markdown", "id": "551e1c4d-0a6b-4a1d-b389-eaf0bbe2e6a9", "metadata": { "id": "551e1c4d-0a6b-4a1d-b389-eaf0bbe2e6a9" }, "source": [ "The `forward` method implements a simple forward pass that returns the logits (outputs of the last fully connected layer of our network) when we call our model on the input data.\n", "The logits, computed via the forward method by calling self(x), are used for the training, validation, and test steps, which we will describe next." ] }, { "cell_type": "markdown", "id": "37feac4f-8baf-4211-8be8-a544270796d0", "metadata": { "id": "37feac4f-8baf-4211-8be8-a544270796d0" }, "source": [ "The `training_step` method is executed on each individual batch during training, with the following steps:\n", "- `logits = self(x)`: a forward pass\n", "- `loss_fn(logits, y)`: calculates loss against true labels\n", "- `preds = torch.argmax(logits, dim=1)`: does prediction by choosing the class with the highest value of logit\n", "- `train_acc(preds, y)` : computes accuracy by comparing predicted and true values\n", "- `self.log(...)`: logs metrics for visualization in the progress bar or TensorBoard, as we will see later. We log the values after each epoch, and not after each batch to get less noisy plots." ] }, { "cell_type": "markdown", "id": "6db443f1-441d-4a0c-86e1-3e1a07235f74", "metadata": { "id": "6db443f1-441d-4a0c-86e1-3e1a07235f74" }, "source": [ "The `validation_step` method is similar to `training_step` but used during validation and it logs validation loss and accuracy per batch.\n", "\n", "Finally, via the `configure_optimizers method`, we specify the optimizer used for training. This is the method how the model updates the weights based on the data and the loss function. For example, Adam optimization is a stochastic gradient descent method with adaptive learning rate often used in practice.\n", "The `training_step`, `validation_step`, `test_step` and `configure_optimizers` methods are methods that are specifically recognized by Lightning.\n" ] }, { "cell_type": "markdown", "id": "f02c73e9-77ce-4f50-9e20-c54b9f8b898e", "metadata": { "id": "f02c73e9-77ce-4f50-9e20-c54b9f8b898e" }, "source": [ "### Training the model using the PyTorch Lightning Trainer class" ] }, { "cell_type": "markdown", "id": "21420512-e0ff-48f6-96ce-2cadf8674cc4", "metadata": { "id": "21420512-e0ff-48f6-96ce-2cadf8674cc4" }, "source": [ "Now we can reap the rewards from setting up the model with the specifically named methods, as well as the Lightning data module. Lightning implements a Trainer class that makes the training model\n", "convenient by taking care of all the intermediate steps and \"auto\" option automatically chooses the best available hardware accelerator (if there are GPUs, it will use them).\n", "\n", "Next, we will redundantly define two logging options:\n", "- CSVlogger to save in a csv file\n", "- TensorBoardLogger that uses tensorboard, visualization toolkit\n", "\n", "With TensorBoard we do not have to define many options, we can simply visualize the dashboard. However, if running locally, you might have difficulty visualizing this dashboard, and furthermore, csv logs can be loaded with Pandas for basic analysis or custom plotting." ] }, { "cell_type": "code", "execution_count": 32, "id": "1c8d417c-78c9-47b2-9c19-b5f5b6519434", "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 468, "referenced_widgets": [ "444777773e9d4c42830a97148913445e", "a4a871dee0b744d89067df5bd70ce0ec", "e20581a183bc41e598f34f50dd4e55ef", "b017226b497844db9dd1d95aedc9fcd5", "e4f75da22fe54a90a2be9edf6ca4ca43", "819d5a370adb4b73b45b01b1bf6824c6", "d3bcb89756d0454c878db1875cfb1d27", "b2fe95fa2e6c4413aa3dd17e041e47be", "a31fad6e7d3141058788c0529fb3b45b", "0eaa31608dab4a64aba2a09b8200200b", "2ba9e3c6007a4dc880ec4757de48a69b", "f3a533b63cd34af5a847963af2693875", "3f10e06684d0488db866f8875daa132c", "9815b5c8652847ce84ac589783ca1e4c", "1f9583df3f81471e83bf33f55ac9a990", "d3db93df4d4a4d5a98d095779236d800", "3e1321c4048740979b952685101ee3a2", "5844fad7feca4caba721c106061613f7", "335f062e448b4754bacc27528430781b", "b8b3ef82e97b4e0490573dda90227e72", "2ab47c175ed14c77892fc4509e5b7b55", "acfbdcfb31394e3496ab75c7e6c0a8c5", "5c29713940094ff3a0e73054dcd80de8", "d9633cd7d7b149dab214496d1fd2f876", "73b3c34a07b84d268ba8d20eef65cc1a", "3b71b27cf5bd4d5e8a0962c646144238", "04544f5d24fb456aa9b9997469e7968d", "f926b90e1a764b9eb8c3cea3aa21cf18", "b20ec8f2faa94e6ea6017162884e4b3c", "9286c93d11f34008ab81a95bf16f6b4b", "09964e13315444668b8ef036621fe3ac", "55ce4f9585e54cc68211559bc392df1b", "8fd75a6f4cea4ef7bc7e641777e3c554", "9629e738b04c40a485ba201e54100f04", "32d0b577656e4db1bba3c7ec1dd4b14e", "b6ba50fde7b746aeb4a5fd50968a8f9b", "7da0496a061e4c7084fee85fb457349a", "f5ea4d3b8139420e9e714845a34d407d", "45d673640dba4b1b8de3c44d3b9c6dff", "872bf623b12e47e392be655cc0c1f4f1", "161f4e5c97734ad89bef49fa8f9d6bc1", "c6506c6aa9ca47d8ba23dd22cf09924b", "7aed5f3dbb734b36812f247cb2925276", "17c74e62e20e42e1b27a6ab5df0d8e26", "80363fc9e7174b9eaa1f502e25a44a20", "62d50bd4188c470484da2808a0e0987f", "4abcbee49e9b4f6fa20945f319e8fc8a", "e397cfc976404e0186ff91633bfb02d8", "14c62c68634946079e92b0443e309ca2", "b72261e955984b7ab00aa09ec91eb859", "13525fee791b46aabbf912aff988a017", "44651b4e3e2a40ff84ae20c275a27b94", "c5737fc6068d44649723b46603559da8", "b7f995f81b2742fca81c658a5d4f6e0f", "97ed2480134b43b69caad381232ae1f6", "67f992e5fa0b44cb9ca0a1077b0fd0b3", "07304a89464a49ad85a801a6588badf8", "983dcc2c7328405ba8473c18055d666f", "6c8d317455db4b9c97742ba569d38765", "dc225614361640dbb26c053388c51182", "120f3943f62e4f7ba46802e1c81819e5", "02327c540e4240ad93f69159a586a695", "6465c4379d0a4a6f8b66ae32a86164f1", "c73584d6817141b79c48dcc76f92748b", "89c8e4bb19c842468d0b63381c13e1ea", "ad70d64891f04b54923bb598f3c4be82", "693e44a6d4704b22b4a3e0e7aacc2a89", "b20e3823631b40969109c4434edcf4cb", "bf1dee758cae4172b5bdda4d4f6aa5d4", "5bcf39a25f814726a6d355141d719e55", "01de57f75ac14b35b664691111c87324", "b81e3780dc124754b6d2ea32907f9b85", "a06b075a94f246e2a76e8f2c7f1c6a97", "4ee347f24ff24cbaa172d96a5e87cc5c", "99af57b1e9a646058f5f0299254e98c6", "f6243fce8c9b49758d7abe02d709b4e0", "6521b00b305b4b2d8023320fcbccdaa3", "cd84ee816aeb4bcfb3ca35dcfd5d46a7", "a612e8e330314d26a0f26c8034e7cabb", "253755895c0c49a58f39f5267ec5744a", "5a2bd91a27ba409b9980cac971db5ba0", "bb303da048854cda97dde88e2b3e78ce", "4d9514a4fec549b9b1bcb1600c70ce80", "12af8881c6354f71abbe13a61ec30cbf", "4712494388bb4f5d9f4de160c9fa5244", "ba1be115b69643b4aa876309eb31f09f", "af22275aa51b41d4a8a0c093d16872f4", "54bf090e91494f1481d935cfd96e6c7d", "c38569a43bb2473ca4e711d156a12da7", "d5155b0d36604131933312f10e047b8a", "7771e0e6d0624963bf22ea034e5071b2", "9bd73d53ce484777962f80a368c6c76d", "225855b374144cabaec2a9f18a69b64a", "7b7e77ea3aa24f41b61d8515e8dbaed6", "39e66ed2644742a0b0d3c1ef0307f463", "68443ed500474fdfb361e58cbb955b2c", "a192193ae05c4045b53471ef3c48af25", "3f21584337fe40e18100e9f4b2bbb45d", "321461735c1d4acb863914733f4f29a1", "4be7367744834d5cabcc2f02710f686e", "94cf9f53d8224cd781fb345b8c4df6c2", "8089561b2d8e49c1bc6b51ca33824556", "9254d3bbd2494eb7aceaf1f58c425b70", "d89e2268f03345f0b315b3c96b1c1a8f", "53b5a097f3644889a8d64acc9685d3b0", "12e51cc17e204feab9a269642ee418b7", "b625021b1fb8465dbcff5396934e0acd", "c2e67f94f8b04c919834d47215cf1c25", "9b1432a4677940b7a4698c73606af8ce", "109402bb49874e108ee3f3b09ba37b33", "1636d928c41742448cfe0cde7401d67d", "4e3e6efcdb0247f6942ec0ed84745d7f", "cf36208b80a94aa48e418c9c8401e322", "2a83d0800a01460594dcc543a845c183", "2f281a3813604a688bac66766044e211", "2f82090308c2469ea2bfc13d354a11be", "7c26402ac7784a5c8e3e3f181d829538", "633e575d2a904eadb3836d9221911746", "b3f9bdff639d4773b3e387205d40bc42", "a90892d849cd4e6aa6d632a2ce7be940", "9d3e2878bcfc472b9954e2cfe66e6bd0", "1e4fe4d1038b4f7f91499cc793c28321", "918b50556def45fba40a7f6d673f6a09", "f7a583b7d9c24fc290de4f6de44f8321", "9f40085152dc4afb8b73bbda14159334", "45d6a2c7dd4f47dbb99562dcd6955250", "7a1d37c4e45547fabed832ae9518f14d", "cdf6e175b8fd41e6a76e2edccd52c046", "cc936ab4d60f43c9b2828dc46dbb6b29", "63680227a1344f589f74852ee161b58c", "3d5134fb97ca43de90f67e7a833aebd9", "80f19b2199cb4dbda6fe96b7b4228f9d" ] }, "id": "1c8d417c-78c9-47b2-9c19-b5f5b6519434", "outputId": "56f2d325-66b7-4dc3-c4d1-bf2c7cc74a6b" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "INFO:pytorch_lightning.utilities.rank_zero:You are using the plain ModelCheckpoint callback. Consider using LitModelCheckpoint which with seamless uploading to Model registry.\n", "INFO:pytorch_lightning.utilities.rank_zero:GPU available: True (cuda), used: True\n", "INFO:pytorch_lightning.utilities.rank_zero:TPU available: False, using: 0 TPU cores\n", "INFO:pytorch_lightning.utilities.rank_zero:HPU available: False, using: 0 HPUs\n", "INFO:pytorch_lightning.accelerators.cuda:LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]\n", "INFO:pytorch_lightning.callbacks.model_summary:\n", " | Name | Type | Params | Mode \n", "---------------------------------------------------------\n", "0 | model | Sequential | 101 K | train\n", "1 | loss_fn | CrossEntropyLoss | 0 | train\n", "2 | train_acc | MulticlassAccuracy | 0 | train\n", "3 | val_acc | MulticlassAccuracy | 0 | train\n", "4 | test_acc | MulticlassAccuracy | 0 | train\n", "---------------------------------------------------------\n", "101 K Trainable params\n", "0 Non-trainable params\n", "101 K Total params\n", "0.407 Total estimated model params size (MB)\n", "9 Modules in train mode\n", "0 Modules in eval mode\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "444777773e9d4c42830a97148913445e", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Sanity Checking: | | 0/? [00:00 {\n", " const url = new URL(await google.colab.kernel.proxyPort(6006, {'cache': true}));\n", " url.searchParams.set('tensorboardColab', 'true');\n", " const iframe = document.createElement('iframe');\n", " iframe.src = url;\n", " iframe.setAttribute('width', '100%');\n", " iframe.setAttribute('height', '800');\n", " iframe.setAttribute('frameborder', 0);\n", " document.body.appendChild(iframe);\n", " })();\n", " " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "%load_ext tensorboard\n", "%tensorboard --logdir mnist-model-tb/lightning_logs/" ] }, { "cell_type": "markdown", "id": "LxCAY3AT0BZD", "metadata": { "id": "LxCAY3AT0BZD" }, "source": [ "The plots contain lot of information, let's go through some of the more important info. If we run the training code multiple\n", "times, Lightning will track them as separate subfolders: version_0, version_1, version_2, and so forth.\n", "Since we logged only per epoch, in the settings on the right tab, we should set smooting to 0.\n", "The first plot shows the relationship of step (batch) vs epoch. Since we have 750 steps per epoch, hence we have that epoch 1 (y-axis value of 1) corresponds to 749 steps. And in all the subsequent graphs, we see that the accuracy and loss have 10 datapoints, where steps are increments of 749." ] }, { "cell_type": "markdown", "id": "4nkz9V-kD2Nt", "metadata": { "id": "4nkz9V-kD2Nt" }, "source": [ "### Visualizing model performance using csv logs" ] }, { "cell_type": "markdown", "id": "P7aJIaZrLSzQ", "metadata": { "id": "P7aJIaZrLSzQ" }, "source": [ "Next let's visualize the model perfromance using the csv of metrics we logged. We will create a function for plotting which we will use throughout the notebook." ] }, { "cell_type": "code", "execution_count": 33, "id": "2Xivgd0XEYQD", "metadata": { "id": "2Xivgd0XEYQD" }, "outputs": [], "source": [ "def plot_performance(metrics_path):\n", " if os.path.exists(metrics_path+'/lightning_logs'):\n", " versions = [d for d in os.listdir(metrics_path+'/lightning_logs') if d.startswith(\"version_\")]\n", " latest_version = max(versions, key=lambda x: int(x.split(\"_\")[1]))\n", " else:\n", " latest_version='version_0'\n", " metrics_df = pd.read_csv(metrics_path+'/lightning_logs/'+latest_version+'/metrics.csv')\n", "\n", " # Since the logger saves training and validation metric at different times,\n", " # when it saves training, validation metrics are left as nan, and the other way around as well.\n", " # So below we just look for non nan values\n", " metrics_df = metrics_df.groupby(['epoch', 'step'], sort=False, as_index=False).last().reset_index(drop=True)\n", " # epochs are recorderd as starting from 0\n", " metrics_df['epoch']=metrics_df['epoch']+1\n", " fig, ax = plt.subplots(1, 2, figsize=(12, 5))\n", "\n", " ax[0].plot(metrics_df['epoch'], metrics_df['train_loss'], label='Training Loss')\n", " ax[0].plot(metrics_df['epoch'], metrics_df['val_loss'], label='Validation Loss')\n", " ax[0].set_xlabel(\"Epoch\")\n", " ax[0].set_ylabel(\"Loss\")\n", " ax[0].set_title(\"Training and Validation Loss\")\n", " ax[0].legend()\n", "\n", " ax[1].plot(metrics_df['epoch'], metrics_df['train_acc'], label='Training Accuracy')\n", " ax[1].plot(metrics_df['epoch'], metrics_df['val_acc'], label='Validation Accuracy')\n", " ax[1].set_xlabel(\"Epoch\")\n", " ax[1].set_ylabel(\"Accuracy\")\n", " ax[1].set_title(\"Training and Validation Accuracy\")\n", " ax[1].legend()\n", "\n", " plt.tight_layout()\n", "\n", " return metrics_df\n" ] }, { "cell_type": "code", "execution_count": 34, "id": "p8HQyMARE90C", "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 380 }, "id": "p8HQyMARE90C", "outputId": "d5534841-c947-4c0a-a654-f78326c39ca7" }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAABKUAAAHqCAYAAADVi/1VAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAA4fdJREFUeJzs3Xd4U3X/xvF3uncLdDHK6mDvJauAogxFwMmQJeDjwIUTRQQc6IMiigo+KuBCcSL+kK1sBGTLaplllS7a0pau5Pz+CERC2ZSm435dVy7NmZ8TEji58x0mwzAMREREREREREREipCTowsQEREREREREZGyR6GUiIiIiIiIiIgUOYVSIiIiIiIiIiJS5BRKiYiIiIiIiIhIkVMoJSIiIiIiIiIiRU6hlIiIiIiIiIiIFDmFUiIiIiIiIiIiUuQUSomIiIiIiIiISJFTKCUiIiIiIiIiIkVOoZRICTV48GCqV69+TfuOHTsWk8lUuAUVMwcPHsRkMjFz5swiP7fJZGLs2LG25zNnzsRkMnHw4MHL7lu9enUGDx5cqPVcz3tFRETE0XTPc2m65/mX7nlESh6FUiKFzGQyXdFj2bJlji61zHviiScwmUzs3bv3otu8/PLLmEwmtm3bVoSVXb1jx44xduxYtmzZ4uhSbM7eJL/zzjuOLkVERG4A3fOUHLrnKTq7du3CZDLh4eFBamqqo8sRKfZcHF2ASGnz1Vdf2T3/8ssvWbx4cYHlderUua7zfPrpp1gslmvad/To0bz44ovXdf7SoH///kyZMoVZs2YxZsyYC27z7bff0qBBAxo2bHjN5xkwYAB9+vTB3d39mo9xOceOHWPcuHFUr16dxo0b2627nveKiIjIxeiep+TQPU/R+frrrwkNDeXkyZP8+OOPDBs2zKH1iBR3CqVECtkDDzxg9/yvv/5i8eLFBZafLysrCy8vrys+j6ur6zXVB+Di4oKLiz7+rVq1IiIigm+//faCN2hr167lwIEDvPXWW9d1HmdnZ5ydna/rGNfjet4rIiIiF6N7npJD9zxFwzAMZs2aRb9+/Thw4ADffPNNsQ2lMjMz8fb2dnQZIuq+J+IIHTt2pH79+mzcuJHo6Gi8vLx46aWXAPj111+5/fbbqVSpEu7u7oSHh/Paa69hNpvtjnF+n/lzu0r973//Izw8HHd3d1q0aMGGDRvs9r3Q+Aomk4kRI0YwZ84c6tevj7u7O/Xq1WPBggUF6l+2bBnNmzfHw8OD8PBwPvnkkyses2HlypXce++9VK1aFXd3d8LCwnj66ac5ffp0gevz8fHh6NGj9OrVCx8fH4KCgnj22WcLvBapqakMHjwYf39/AgICGDRo0BU3l+7fvz+7d+9m06ZNBdbNmjULk8lE3759yc3NZcyYMTRr1gx/f3+8vb1p3749f/7552XPcaHxFQzD4PXXX6dKlSp4eXnRqVMnduzYUWDflJQUnn32WRo0aICPjw9+fn5069aNrVu32rZZtmwZLVq0AGDIkCG27hJnx5a40PgKmZmZPPPMM4SFheHu7k6tWrV45513MAzDbrureV9cq4SEBIYOHUpISAgeHh40atSIL774osB23333Hc2aNcPX1xc/Pz8aNGjA+++/b1ufl5fHuHHjiIyMxMPDgwoVKtCuXTsWL15caLWKiMjV0T2P7nnK0j3P6tWrOXjwIH369KFPnz6sWLGCI0eOFNjOYrHw/vvv06BBAzw8PAgKCqJr1678/fffdtt9/fXXtGzZEi8vL8qVK0d0dDSLFi2yq/ncMb3OOn+8rrN/LsuXL+fRRx8lODiYKlWqAHDo0CEeffRRatWqhaenJxUqVODee++94LhgqampPP3001SvXh13d3eqVKnCwIEDSUpKIiMjA29vb5588skC+x05cgRnZ2cmTJhwha+klCX62UDEQZKTk+nWrRt9+vThgQceICQkBLD+o+Hj48PIkSPx8fHhjz/+YMyYMaSnpzNx4sTLHnfWrFmcOnWK//znP5hMJv773/9y1113sX///sv+erRq1Sp+/vlnHn30UXx9ffnggw+4++67iYuLo0KFCgBs3ryZrl27UrFiRcaNG4fZbGb8+PEEBQVd0XX/8MMPZGVl8cgjj1ChQgXWr1/PlClTOHLkCD/88IPdtmazmS5dutCqVSveeecdlixZwrvvvkt4eDiPPPIIYL3R6dmzJ6tWreLhhx+mTp06/PLLLwwaNOiK6unfvz/jxo1j1qxZNG3a1O7c33//Pe3bt6dq1aokJSXx2Wef0bdvX4YPH86pU6f4/PPP6dKlC+vXry/QfPxyxowZw+uvv0737t3p3r07mzZt4rbbbiM3N9duu/379zNnzhzuvfdeatSowYkTJ/jkk0/o0KEDO3fupFKlStSpU4fx48czZswYHnroIdq3bw9AmzZtLnhuwzC48847+fPPPxk6dCiNGzdm4cKFPPfccxw9epT33nvPbvsreV9cq9OnT9OxY0f27t3LiBEjqFGjBj/88AODBw8mNTXVdmOzePFi+vbtyy233MLbb78NWMdsWL16tW2bsWPHMmHCBIYNG0bLli1JT0/n77//ZtOmTdx6663XVaeIiFw73fPonqes3PN88803hIeH06JFC+rXr4+Xlxfffvstzz33nN12Q4cOZebMmXTr1o1hw4aRn5/PypUr+euvv2jevDkA48aNY+zYsbRp04bx48fj5ubGunXr+OOPP7jtttuu+PU/16OPPkpQUBBjxowhMzMTgA0bNrBmzRr69OlDlSpVOHjwIFOnTqVjx47s3LnT1qoxIyOD9u3bs2vXLh588EGaNm1KUlISc+fO5ciRIzRu3JjevXsze/ZsJk2aZNdi7ttvv8UwDPr3739NdUspZ4jIDfXYY48Z53/UOnToYADGtGnTCmyflZVVYNl//vMfw8vLy8jOzrYtGzRokFGtWjXb8wMHDhiAUaFCBSMlJcW2/NdffzUA47fffrMte/XVVwvUBBhubm7G3r17bcu2bt1qAMaUKVNsy3r06GF4eXkZR48etS2LjY01XFxcChzzQi50fRMmTDBMJpNx6NAhu+sDjPHjx9tt26RJE6NZs2a253PmzDEA47///a9tWX5+vtG+fXsDMGbMmHHZmlq0aGFUqVLFMJvNtmULFiwwAOOTTz6xHTMnJ8duv5MnTxohISHGgw8+aLccMF599VXb8xkzZhiAceDAAcMwDCMhIcFwc3Mzbr/9dsNisdi2e+mllwzAGDRokG1Zdna2XV2GYf2zdnd3t3ttNmzYcNHrPf+9cvY1e/311+22u+eeewyTyWT3HrjS98WFnH1PTpw48aLbTJ482QCMr7/+2rYsNzfXaN26teHj42Okp6cbhmEYTz75pOHn52fk5+df9FiNGjUybr/99kvWJCIiN47ueS5/fbrnsSpt9zyGYb1/qVChgvHyyy/blvXr189o1KiR3XZ//PGHARhPPPFEgWOcfY1iY2MNJycno3fv3gVek3Nfx/Nf/7OqVatm99qe/XNp165dgXupC71P165dawDGl19+aVs2ZswYAzB+/vnni9a9cOFCAzDmz59vt75hw4ZGhw4dCuwnYhiGoe57Ig7i7u7OkCFDCiz39PS0/f+pU6dISkqiffv2ZGVlsXv37sse9/7776dcuXK252d/Qdq/f/9l9+3cuTPh4eG25w0bNsTPz8+2r9lsZsmSJfTq1YtKlSrZtouIiKBbt26XPT7YX19mZiZJSUm0adMGwzDYvHlzge0ffvhhu+ft27e3u5bff/8dFxcX26+IYB3P4PHHH7+iesA6JsaRI0dYsWKFbdmsWbNwc3Pj3nvvtR3Tzc0NsDa5TklJIT8/n+bNm1+wGfylLFmyhNzcXB5//HG75v9PPfVUgW3d3d1xcrL+VW02m0lOTsbHx4datWpd9XnP+v3333F2duaJJ56wW/7MM89gGAbz58+3W36598X1+P333wkNDaVv3762Za6urjzxxBNkZGSwfPlyAAICAsjMzLxkV7yAgAB27NhBbGzsddclIiKFR/c8uucpC/c88+fPJzk52e6epm/fvmzdutWuu+JPP/2EyWTi1VdfLXCMs6/RnDlzsFgsjBkzxvaanL/NtRg+fHiBMb/OfZ/m5eWRnJxMREQEAQEBdq/7Tz/9RKNGjejdu/dF6+7cuTOVKlXim2++sa37559/2LZt22XHmpOyS6GUiINUrlzZ9g/+uXbs2EHv3r3x9/fHz8+PoKAg21/iaWlplz1u1apV7Z6fvVk7efLkVe97dv+z+yYkJHD69GkiIiIKbHehZRcSFxfH4MGDKV++vG3MhA4dOgAFr+9sH/uL1QPWfvAVK1bEx8fHbrtatWpdUT0Affr0wdnZmVmzZgGQnZ3NL7/8Qrdu3exudr/44gsaNmxoG68oKCiIefPmXdGfy7kOHToEQGRkpN3yoKAgu/OB9WbwvffeIzIyEnd3dwIDAwkKCmLbtm1Xfd5zz1+pUiV8fX3tlp+dHelsfWdd7n1xPQ4dOkRkZGSBG67za3n00UeJioqiW7duVKlShQcffLDAGA/jx48nNTWVqKgoGjRowHPPPVfsp7UWESkLdM+je56ycM/z9ddfU6NGDdzd3dm7dy979+4lPDwcLy8vu5Bm3759VKpUifLly1/0WPv27cPJyYm6dete9rxXo0aNGgWWnT59mjFjxtjG3Dr7uqemptq97vv27aN+/fqXPL6TkxP9+/dnzpw5ZGVlAdYujR4eHrbQU+R8CqVEHOTcXyXOSk1NpUOHDmzdupXx48fz22+/sXjxYtsYOlcyxe3FZjwxzhvMsbD3vRJms5lbb72VefPm8cILLzBnzhwWL15sG5zy/OsrqtlbgoODufXWW/npp5/Iy8vjt99+49SpU3b93r/++msGDx5MeHg4n3/+OQsWLGDx4sXcfPPNN3Tq4TfffJORI0cSHR3N119/zcKFC1m8eDH16tUrsimPb/T74koEBwezZcsW5s6daxsbolu3bnbjaERHR7Nv3z6mT59O/fr1+eyzz2jatCmfffZZkdUpIiIF6Z5H9zxXoiTf86Snp/Pbb79x4MABIiMjbY+6deuSlZXFrFmzivS+6fwB8s+60Gfx8ccf54033uC+++7j+++/Z9GiRSxevJgKFSpc0+s+cOBAMjIymDNnjm02wjvuuAN/f/+rPpaUDRroXKQYWbZsGcnJyfz8889ER0fblh84cMCBVf0rODgYDw8P9u7dW2DdhZadb/v27cTExPDFF18wcOBA2/LrmR2tWrVqLF26lIyMDLtfDvfs2XNVx+nfvz8LFixg/vz5zJo1Cz8/P3r06GFb/+OPP1KzZk1+/vlnu2bTF2p6fSU1A8TGxlKzZk3b8sTExAK/xP3444906tSJzz//3G55amoqgYGBtudX05S7WrVqLFmyhFOnTtn9cni2q8TZ+opCtWrV2LZtGxaLxa611IVqcXNzo0ePHvTo0QOLxcKjjz7KJ598wiuvvGL71bp8+fIMGTKEIUOGkJGRQXR0NGPHji220zGLiJRVuue5errnsSqO9zw///wz2dnZTJ061a5WsP75jB49mtWrV9OuXTvCw8NZuHAhKSkpF20tFR4ejsViYefOnZccWL5cuXIFZl/Mzc3l+PHjV1z7jz/+yKBBg3j33Xdty7KzswscNzw8nH/++eeyx6tfvz5NmjThm2++oUqVKsTFxTFlypQrrkfKHrWUEilGzv46c+4vKbm5uXz88ceOKsmOs7MznTt3Zs6cORw7dsy2fO/evQX65F9sf7C/PsMweP/996+5pu7du5Ofn8/UqVNty8xm81X/49erVy+8vLz4+OOPmT9/PnfddRceHh6XrH3dunWsXbv2qmvu3Lkzrq6uTJkyxe54kydPLrCts7NzgV/WfvjhB44ePWq3zNvbG+CKpoXu3r07ZrOZDz/80G75e++9h8lkuuKxMgpD9+7diY+PZ/bs2bZl+fn5TJkyBR8fH1s3h+TkZLv9nJycaNiwIQA5OTkX3MbHx4eIiAjbehERKT50z3P1dM9jVRzveb7++mtq1qzJww8/zD333GP3ePbZZ/Hx8bF14bv77rsxDINx48YVOM7Z6+/VqxdOTk6MHz++QGulc1+j8PBwu/HBAP73v/9dtKXUhVzodZ8yZUqBY9x9991s3bqVX3755aJ1nzVgwAAWLVrE5MmTqVChQpHeW0rJo5ZSIsVImzZtKFeuHIMGDeKJJ57AZDLx1VdfFWlz38sZO3YsixYtom3btjzyyCO2f+jr16/Pli1bLrlv7dq1CQ8P59lnn+Xo0aP4+fnx008/XdfYRD169KBt27a8+OKLHDx4kLp16/Lzzz9f9dgDPj4+9OrVyzbGwvlT1t5xxx38/PPP9O7dm9tvv50DBw4wbdo06tatS0ZGxlWdKygoiGeffZYJEyZwxx130L17dzZv3sz8+fML/Lp2xx13MH78eIYMGUKbNm3Yvn0733zzjd2vjWC9KQkICGDatGn4+vri7e1Nq1atLjh2QI8ePejUqRMvv/wyBw8epFGjRixatIhff/2Vp556ym6Az8KwdOlSsrOzCyzv1asXDz30EJ988gmDBw9m48aNVK9enR9//JHVq1czefJk26+aw4YNIyUlhZtvvpkqVapw6NAhpkyZQuPGjW3jQtStW5eOHTvSrFkzypcvz99//82PP/7IiBEjCvV6RETk+ume5+rpnsequN3zHDt2jD///LPAYOpnubu706VLF3744Qc++OADOnXqxIABA/jggw+IjY2la9euWCwWVq5cSadOnRgxYgQRERG8/PLLvPbaa7Rv35677roLd3d3NmzYQKVKlZgwYQJgvT96+OGHufvuu7n11lvZunUrCxcuLPDaXsodd9zBV199hb+/P3Xr1mXt2rUsWbKEChUq2G333HPP8eOPP3Lvvffy4IMP0qxZM1JSUpg7dy7Tpk2jUaNGtm379evH888/zy+//MIjjzyCq6vrNbyyUmYUwQx/ImXaxaZHrlev3gW3X716tXHTTTcZnp6eRqVKlYznn3/eNr3qn3/+advuYtMjT5w4scAxOW+62ItNj/zYY48V2Pf8KWUNwzCWLl1qNGnSxHBzczPCw8ONzz77zHjmmWcMDw+Pi7wK/9q5c6fRuXNnw8fHxwgMDDSGDx9um2733Kl9Bw0aZHh7exfY/0K1JycnGwMGDDD8/PwMf39/Y8CAAcbmzZuveHrks+bNm2cARsWKFS84/e6bb75pVKtWzXB3dzeaNGli/N///V+BPwfDuPz0yIZhGGaz2Rg3bpxRsWJFw9PT0+jYsaPxzz//FHi9s7OzjWeeeca2Xdu2bY21a9caHTp0KDC17q+//mrUrVvXNlX12Wu/UI2nTp0ynn76aaNSpUqGq6urERkZaUycONFumuGz13Kl74vznX1PXuzx1VdfGYZhGCdOnDCGDBliBAYGGm5ubkaDBg0K/Ln9+OOPxm233WYEBwcbbm5uRtWqVY3//Oc/xvHjx23bvP7660bLli2NgIAAw9PT06hdu7bxxhtvGLm5uZesU0RECofueezpnseqtN/zvPvuuwZgLF269KLbzJw50wCMX3/91TAMw8jPzzcmTpxo1K5d23BzczOCgoKMbt26GRs3brTbb/r06UaTJk0Md3d3o1y5ckaHDh2MxYsX29abzWbjhRdeMAIDAw0vLy+jS5cuxt69ewvUfPbPZcOGDQVqO3nypO0+zMfHx+jSpYuxe/fuC153cnKyMWLECKNy5cqGm5ubUaVKFWPQoEFGUlJSgeN2797dAIw1a9Zc9HURMQzDMBlGMfo5QkRKrF69erFjxw5iY2MdXYqIiIjIDaN7HpHL6927N9u3b7+iMdikbNOYUiJy1U6fPm33PDY2lt9//52OHTs6piARERGRG0D3PCJX7/jx48ybN48BAwY4uhQpAdRSSkSuWsWKFRk8eDA1a9bk0KFDTJ06lZycHDZv3kxkZKSjyxMREREpFLrnEblyBw4cYPXq1Xz22Wds2LCBffv2ERoa6uiypJjTQOcictW6du3Kt99+S3x8PO7u7rRu3Zo333xTN2ciIiJSquieR+TKLV++nCFDhlC1alW++OILBVJyRdRSSkREREREREREipzGlBIRERERERERkSKnUEpERERERERERIqcxpS6AIvFwrFjx/D19cVkMjm6HBEREXEQwzA4deoUlSpVwslJv+Vdju6hREREBK78Hkqh1AUcO3aMsLAwR5chIiIixcThw4epUqWKo8so9nQPJSIiIue63D2UQqkL8PX1Bawvnp+fn4OrEREREUdJT08nLCzMdm8gl6Z7KBEREYErv4dSKHUBZ5ub+/n56YZKRERE1BXtCukeSkRERM51uXsoDY4gIiIiIiIiIiJFTqGUiIiIiIiIiIgUOYVSIiIiIiIiIiJS5DSmlIiIlBhms5m8vDxHlyGljJub2yWnKpbCpc+xlFaurq44Ozs7ugwRkRJFoZSIiBR7hmEQHx9Pamqqo0uRUsjJyYkaNWrg5ubm6FJKNX2OpSwICAggNDRUkyOIiFwhhVIiIlLsnf0iGxwcjJeXl272pdBYLBaOHTvG8ePHqVq1qt5bN5A+x1KaGYZBVlYWCQkJAFSsWNHBFYmIlAwKpUREpFgzm822L7IVKlRwdDlSCgUFBXHs2DHy8/NxdXV1dDmlkj7HUhZ4enoCkJCQQHBwsLryiYhcAQ2gICIixdrZsWe8vLwcXImUVme77ZnNZgdXUnrpcyxlxdn3uMZNExG5MgqlRESkRFBXH7lR9N4qOnqtpbTTe1xE5OoolBIRERERERERkSKnUEpERKSEqF69OpMnT77i7ZctW4bJZNJsZyLFjD7LIiIiVgqlRERECpnJZLrkY+zYsdd03A0bNvDQQw9d8fZt2rTh+PHj+Pv7X9P5rpS+MEtpVdY+y+eqXbs27u7uxMfHF9k5RUSk7NHseyIiIoXs+PHjtv+fPXs2Y8aMYc+ePbZlPj4+tv83DAOz2YyLy+X/SQ4KCrqqOtzc3AgNDb2qfUTkX2X1s7xq1SpOnz7NPffcwxdffMELL7xQZOe+kLy8PM2MKSJSSqmllIiISCELDQ21Pfz9/TGZTLbnu3fvxtfXl/nz59OsWTPc3d1ZtWoV+/bto2fPnoSEhODj40OLFi1YsmSJ3XHP7/JjMpn47LPP6N27N15eXkRGRjJ37lzb+vNbMM2cOZOAgAAWLlxInTp18PHxoWvXrnZfvPPz83niiScICAigQoUKvPDCCwwaNIhevXpd8+tx8uRJBg4cSLly5fDy8qJbt27Exsba1h86dIgePXpQrlw5vL29qVevHr///rtt3/79+xMUFISnpyeRkZHMmDHjmmsRuRpl9bP8+eef069fPwYMGMD06dMLrD9y5Ah9+/alfPnyeHt707x5c9atW2db/9tvv9GiRQs8PDwIDAykd+/edtc6Z84cu+MFBAQwc+ZMAA4ePIjJZGL27Nl06NABDw8PvvnmG5KTk+nbty+VK1fGy8uLBg0a8O2339odx2Kx8N///peIiAjc3d2pWrUqb7zxBgA333wzI0aMsNs+MTERNzc3li5detnXREREbgyFUkXsWOppvlsfx+GULEeXIiJSIhmGQVZuvkMehmEU2nW8+OKLvPXWW+zatYuGDRuSkZFB9+7dWbp0KZs3b6Zr16706NGDuLi4Sx5n3Lhx3HfffWzbto3u3bvTv39/UlJSLrp9VlYW77zzDl999RUrVqwgLi6OZ5991rb+7bff5ptvvmHGjBmsXr2a9PT0Al8gr9bgwYP5+++/mTt3LmvXrsUwDLp3726bMv2xxx4jJyeHFStWsH37dt5++21bC5RXXnmFnTt3Mn/+fHbt2sXUqVMJDAy8rnqkeNBn2V5x+SyfOnWKH374gQceeIBbb72VtLQ0Vq5caVufkZFBhw4dOHr0KHPnzmXr1q08//zzWCwWAObNm0fv3r3p3r07mzdvZunSpbRs2fKy5z3fiy++yJNPPsmuXbvo0qUL2dnZNGvWjHnz5vHPP//w0EMPMWDAANavX2/bZ9SoUbz11lu2vzdmzZpFSEgIAMOGDWPWrFnk5OTYtv/666+pXLkyN99881XXJyJS0h1PO82PG4+wN+GUQ+tQ970i9sJP21gZm8To2+swrH1NR5cjIlLinM4zU3fMQoece+f4Lni5Fc4/nePHj+fWW2+1PS9fvjyNGjWyPX/ttdf45ZdfmDt3boFf9881ePBg+vbtC8Cbb77JBx98wPr16+natesFt8/Ly2PatGmEh4cDMGLECMaPH29bP2XKFEaNGmVr2fDhhx/aWi1di9jYWObOncvq1atp06YNAN988w1hYWHMmTOHe++9l7i4OO6++24aNGgAQM2a//77GBcXR5MmTWjevDlgbWEipYM+y/aKy2f5u+++IzIyknr16gHQp08fPv/8c9q3bw/ArFmzSExMZMOGDZQvXx6AiIgI2/5vvPEGffr0Ydy4cbZl574eV+qpp57irrvuslt2buj2+OOPs3DhQr7//ntatmzJqVOneP/99/nwww8ZNGgQAOHh4bRr1w6Au+66ixEjRvDrr79y3333AdYWZ4MHD8ZkMl11fSIiJU3a6Tz+2p/Mmr1JrNqbxL7ETACevCWSp2/1dVhdailVxDpEWccQWB6T6OBKRETEkc6GLGdlZGTw7LPPUqdOHQICAvDx8WHXrl2XbV3RsGFD2/97e3vj5+dHQkLCRbf38vKyfYkFqFixom37tLQ0Tpw4YdeqwdnZmWbNml3VtZ1r165duLi40KpVK9uyChUqUKtWLXbt2gXAE088weuvv07btm159dVX2bZtm23bRx55hO+++47GjRvz/PPPs2bNmmuuReRGKG2f5enTp/PAAw/Ynj/wwAP88MMPnDpl/SV9y5YtNGnSxBZInW/Lli3ccsstlz3P5Zz/uprNZl577TUaNGhA+fLl8fHxYeHChbbXddeuXeTk5Fz03B4eHnbdETdt2sQ///zD4MGDr7tWEZHiKCffzNp9ybyzcA+9PlpNk/GL+M9XG/li7SH2JWbiZIJGYQFU9PdwaJ1qKVXEOkQF8fq8Xaw/kEJ2nhkPV2dHlyQiUqJ4ujqzc3wXh527sHh7e9s9f/bZZ1m8eDHvvPMOEREReHp6cs8995Cbm3vJ45w/+K/JZLJ1o7nS7QuzK9O1GDZsGF26dGHevHksWrSICRMm8O677/L444/TrVs3Dh06xO+//87ixYu55ZZbeOyxx3jnnXccWrNcP32W7RWHz/LOnTv566+/WL9+vd3g5mazme+++47hw4fj6el5yWNcbv2F6jzblfdc57+uEydO5P3332fy5Mk0aNAAb29vnnrqKdvrernzgvXvmsaNG3PkyBFmzJjBzTffTLVq1S67n4hISWCxGOw8ns6afUms2pvM+gPJZOfZ/ztSM8ibdhGBtI0I5KaaFfD3dPwkEgqlilhEsA8V/T04npbNX/uT6Vgr2NEliYiUKCaTqdC63RQnq1evZvDgwbauNhkZGRw8eLBIa/D39yckJIQNGzYQHR0NWL+Mbtq0icaNG1/TMevUqUN+fj7r1q2zdd9LTk5mz5491K1b17ZdWFgYDz/8MA8//DCjRo3i008/5fHHHwesM5UNGjSIQYMG0b59e5577jmFUqWAPss3zrV+lj///HOio6P56KOP7JbPmDGDzz//nOHDh9OwYUM+++wzUlJSLthaqmHDhixdupQhQ4Zc8BxBQUF2A7LHxsaSlXX5sVZXr15Nz549ba24LBYLMTExtr9HIiMj8fT0ZOnSpQwbNuyCx2jQoAHNmzfn008/ZdasWXz44YeXPa+ISHF2OCWLVWe6463dl0xKpv0PIEG+7rQNr0DbM0FUpYDLB/hFrfTdCRRzJpOJ6MggZv99mBUxSQqlREQEsH6h+vnnn+nRowcmk4lXXnnlkq0kbpTHH3+cCRMmEBERQe3atZkyZQonT568ojFXtm/fjq/vv2MSmEwmGjVqRM+ePRk+fDiffPIJvr6+vPjii1SuXJmePXsC1rFjunXrRlRUFCdPnuTPP/+kTp06AIwZM4ZmzZpRr149cnJy+L//+z/bOpHiqKR+lvPy8vjqq68YP3489evXt1s3bNgwJk2axI4dO+jbty9vvvkmvXr1YsKECVSsWJHNmzdTqVIlWrduzauvvsott9xCeHg4ffr0IT8/n99//93W8urmm2/mww8/pHXr1pjNZl544YUCrb4uJDIykh9//JE1a9ZQrlw5Jk2axIkTJ2yhlIeHBy+88ALPP/88bm5utG3blsTERHbs2MHQoUPtrmXEiBF4e3vbzQooIlISpGTmsmZfEqv3JrN6bxJx502g5u3mzE01rSFUu8hAIoN9iv24eQqlHKBDrTOhVKzGlRIREatJkybx4IMP0qZNGwIDA3nhhRdIT08v8jpeeOEF4uPjGThwIM7Ozjz00EN06dIFZ+fLd3c62yLjLGdnZ/Lz85kxYwZPPvkkd9xxB7m5uURHR/P777/bvoiazWYee+wxjhw5gp+fH127duW9994DwM3NjVGjRnHw4EE8PT1p37493333XeFfuEghKamf5blz55KcnHzBoKZOnTrUqVOHzz//nEmTJrFo0SKeeeYZunfvTn5+PnXr1rW1rurYsSM//PADr732Gm+99RZ+fn52fze8++67DBkyhPbt21OpUiXef/99Nm7ceNnrGT16NPv376dLly54eXnx0EMP0atXL9LS0mzbvPLKK7i4uDBmzBiOHTtGxYoVefjhh+2O07dvX5566in69u2Lh4djx1EREbmc07lmNhxMYfWZ1lA7jtn/e+LiZKJJ1QBrCBURSKOwAFydS9bQ4SbD0QNJFEPp6en4+/uTlpaGn59foR8/LSuPJq8twmLA6hdvpnIxbEInIlJcZGdnc+DAAWrUqKEvEA5gsVioU6cO9913H6+99pqjy7khLvUeu9H3BKXNxV4vfY4dryx8lq/EwYMHCQ8PZ8OGDTRt2rTQj6/3uohcj3yzhe1H02wh1KZDqeSa7Vvb1g71tYVQLWqUx8e9eLY1utJ7qOJZfSnn7+VK47AANsWlsiImkb4tqzq6JBEREQAOHTrEokWL6NChAzk5OXz44YccOHCAfv36Obo0EbkK+izby8vLIzk5mdGjR3PTTTfdkEBKRORqGYbBvsRM6+DksUms3Z/Mqex8u20q+XvYuuO1Dq9AsG/pCrwVSjlIh6hghVIiIlLsODk5MXPmTJ599lkMw6B+/fosWbJE4ziJlDD6LNtbvXo1nTp1Iioqih9//NHR5YhIGZaQns3qfUmsirWOCxWfnm233s/DhTbhgbSNtLaGql7Bq9iPC3U9FEo5SHRUIO8tiWHV3iTyzRZcSli/TxERKZ3CwsJYvXq1o8sQkeukz7K9jh07olFLRMQRMnLyWbc/mVV7k1i9N4mYExl2691cnGhRvRxtwq0hVP3K/jg7ld4Q6nwKpRykYZUAArxcSc3KY8vhVJpXLzilroiIiIiIiIiUHLn5FrYcTrWFUFsOp2K2/BuKm0xQv5K/bVyo5tXL4eF6+QllSiuFUg7i7GSibUQg87YdZ0VMokIpERERERERkRLGMAx2x59i9ZkQat2BFLJyzXbbVK/gRduIQNpGBNK6ZgXKebs5qNriR6GUA3WICmLetuMsj01i5G21HF2OiIiIiIiIiFzG0dTTrI61zpC3Zl8SSRm5dusreLvRJiKQdhEVaBMeSFh5LwdVWvwplHKg6MggALYdSSUlM5fySktFREREREREipXsPDOrYpNYFpPA6r3JHEjKtFvv6epMq5rlaRtubQ1VO9QXpzI0LtT1UCjlQKH+HtQK8WXPiVOs2pvEnY0qObokERERERERkTIvIyefP3cnsGBHPMt2J5B5Tpc8ZycTjar40+5Ml7wmVcvh5qLJy66FQikH61AriD0nTrEiJlGhlIiIiIiIiIiDnMzMZcmuEyzcEc+K2CRy8y22dRX9Pbi1bgjtI4NoVbM8fh6uDqy09FCU52Bnu/CtiEnUNLUiImKnY8eOPPXUU7bn1atXZ/LkyZfcx2QyMWfOnOs+d2EdR0T0WRYRKc5OpGfz1dqD9P/sL5q/sYTnftzGkl0J5OZbqF7Bi4c7hPPrY21Z8+LNjO9Zn1vrhiiQKkRqKeVg1ukfnUg4lcPu+FPUqejn6JJEROQ69ejRg7y8PBYsWFBg3cqVK4mOjmbr1q00bNjwqo67YcMGvL29C6tMAMaOHcucOXPYsmWL3fLjx49Trly5Qj3X+WbOnMlTTz1FamrqDT2PyLXSZ/nqnD59msqVK+Pk5MTRo0dxd3cvkvOKiFytuOQsFuw4zoJ/4tkUl2q3rk5FP7rWC6Vr/VCiQnwwmTQ21I2kUMrBPFydualmBZbtSWRFTKJCKRGRUmDo0KHcfffdHDlyhCpVqtitmzFjBs2bN7/qL7EAQUFBhVXiZYWGhhbZuUSKK32Wr85PP/1EvXr1MAyDOXPmcP/99xfZuc9nGAZmsxkXF33dERHr3wmxCRks+CeeBf/Es/N4ut36plUD6Fo/lC71QqlWoXB/NJBLU/e9YqBD1JkufLGJDq5EREQKwx133EFQUBAzZ860W56RkcEPP/zA0KFDSU5Opm/fvlSuXBkvLy8aNGjAt99+e8njnt/lJzY2lujoaDw8PKhbty6LFy8usM8LL7xAVFQUXl5e1KxZk1deeYW8vDzA2lJp3LhxbN26FZPJhMlkstV8fpef7du3c/PNN+Pp6UmFChV46KGHyMjIsK0fPHgwvXr14p133qFixYpUqFCBxx57zHauaxEXF0fPnj3x8fHBz8+P++67jxMnTtjWb926lU6dOuHr64ufnx/NmjXj77//BuDQoUP06NGDcuXK4e3tTb169fj999+vuRYpm/RZvrrP8ueff84DDzzAAw88wOeff15g/Y4dO7jjjjvw8/PD19eX9u3bs2/fPtv66dOnU69ePdzd3alYsSIjRowA4ODBg5hMJrtWYKmpqZhMJpYtWwbAsmXLMJlMzJ8/n2bNmuHu7s6qVavYt28fPXv2JCQkBB8fH1q0aMGSJUvs6srJyeGFF14gLCwMd3d3IiIi+PzzzzEMg4iICN555x277bds2YLJZGLv3r2XfU1ExHEMw2Dr4VTeXrCbW95dzm3vrWDS4hh2Hk/H2clEm/AKjO9Zj79G3cLPj7bloehwBVIOoJ8OioHoM6HUhgMnycrNx8tNfywiIhdlGJCX5Zhzu3rBFTThdnFxYeDAgcycOZOXX37Z1uz7hx9+wGw207dvXzIyMmjWrBkvvPACfn5+zJs3jwEDBhAeHk7Lli0vew6LxcJdd91FSEgI69atIy0tzW7MmrN8fX2ZOXMmlSpVYvv27QwfPhxfX1+ef/557r//fv755x8WLFhg+5Lm7+9f4BiZmZl06dKF1q1bs2HDBhISEhg2bBgjRoyw+7L+559/UrFiRf7880/27t3L/fffT+PGjRk+fPhlr+dC13c2kFq+fDn5+fk89thj3H///bYvof3796dJkyZMnToVZ2dntmzZgqurdYyHxx57jNzcXFasWIG3tzc7d+7Ex8fnqusozj766CMmTpxIfHw8jRo1YsqUKRd97+Tl5TFhwgS++OILjh49Sq1atXj77bfp2rXrNR/zuumzDJSez/K+fftYu3YtP//8M4Zh8PTTT3Po0CGqVasGwNGjR4mOjqZjx4788ccf+Pn5sXr1avLz8wGYOnUqI0eO5K233qJbt26kpaWxevXqy75+53vxxRd55513qFmzJuXKlePw4cN0796dN954A3d3d7788kt69OjBnj17qFq1KgADBw5k7dq1fPDBBzRq1IgDBw6QlJSEyWTiwQcfZMaMGTz77LO2c8yYMYPo6GgiIiKuuj4RubHMFoO/D6Yw/594Fu2I51hatm2dm7MT7SMD6VI/lM51Qijv7ebASuUspR/FQM1AbyoHeHI09TR/7U/m5tohji5JRKT4ysuCNx00W+lLx8Dtyn5Be/DBB5k4cSLLly+nY8eOgPWLzN13342/vz/+/v52X3Ief/xxFi5cyPfff39FX2SXLFnC7t27WbhwIZUqWV+PN998k27dutltN3r0aNv/V69enWeffZbvvvuO559/Hk9PT3x8fHBxcblkF59Zs2aRnZ3Nl19+aRsH58MPP6RHjx68/fbbhIRY/90qV64cH374Ic7OztSuXZvbb7+dpUuXXlMotXTpUrZv386BAwcICwsD4Msvv6RevXps2LCBFi1aEBcXx3PPPUft2rUBiIyMtO0fFxfH3XffTYMGDQCoWbPmVddQnM2ePZuRI0cybdo0WrVqxeTJk+nSpQt79uwhODi4wPajR4/m66+/5tNPP6V27dosXLiQ3r17s2bNGpo0aXJNx7xu+iwDpeezPH36dLp162Ybv6pLly7MmDGDsWPHAtbA09/fn++++84WHkdFRdn2f/3113nmmWd48sknbctatGhx2dfvfOPHj+fWW2+1PS9fvjyNGjWyPX/ttdf45ZdfmDt3LiNGjCAmJobvv/+exYsX07lzZ8D+74vBgwczZswY1q9fT8uWLcnLy2PWrFkFWk+JiOPk5ltYsy+JhTviWbTjBMmZubZ1Xm7OdKodTNd6oXSsFYSvBigvdhzefe+jjz6ievXqeHh40KpVK9avX3/RbWfOnGlrknz24eHhYbfN4MGDC2xz/q+AxY3JZKJDrbOz8CU5uBoRESkMtWvXpk2bNkyfPh2AvXv3snLlSoYOHQqA2Wzmtddeo0GDBpQvXx4fHx8WLlxIXFzcFR1/165dhIWF2b7EArRu3brAdrNnz6Zt27aEhobi4+PD6NGjr/gc556rUaNGdgMzt23bFovFwp49e2zL6tWrh7Ozs+15xYoVSUhIuKpznXvOsLAwWyAFULduXQICAti1axcAI0eOZNiwYXTu3Jm33nrLrhvQE088weuvv07btm159dVX2bZt2zXVUVxNmjSJ4cOHM2TIEOrWrcu0adPw8vKyvd/O99VXX/HSSy/RvXt3atasySOPPEL37t159913r/mYZYU+y5f/LJvNZr744gseeOAB27IHHniAmTNnYrFYp1PfsmUL7du3twVS50pISODYsWPccsstV3U9F9K8eXO75xkZGTz77LPUqVOHgIAAfHx82LVrl+2127JlC87OznTo0OGCx6tUqRK333677c//t99+Iycnh3vvvfe6axWRa5eVm8+Cf47z1HebafbaYgbP2MC36w+TnJmLv6crdzetwqcDm7PplVv5qF9TejSqpECqmHJoS6lr+UXOz8/P7h/NC42E37VrV2bMmGF7XhJm/oiODGLWujhWxGhcKRGRS3L1srZycNS5r8LQoUN5/PHH+eijj5gxYwbh4eG2Lz4TJ07k/fffZ/LkyTRo0ABvb2+eeuopcnNzL3PUK7d27Vr69+/PuHHj6NKli62VwrlBRGE6/8umyWSyfSG9EcaOHUu/fv2YN28e8+fP59VXX+W7776jd+/eDBs2jC5dujBv3jwWLVrEhAkTePfdd3n88cdvWD1FJTc3l40bNzJq1CjbMicnJzp37szatWsvuE9OTk6BH/I8PT1ZtWrVNR/zuumzfMWK+2d54cKFHD16tMDA5mazmaVLl3Lrrbfi6el50f0vtQ6s70Wwjg9z1sXGuDp/VsNnn32WxYsX88477xAREYGnpyf33HOP7c/ncucGGDZsGAMGDOC9995jxowZ3H///Xh5Xd17SESuX9rpPP7YfYIF/8SzPCaR7Lx//14K8nWnS70QutarSKua5XF1dnj7G7lCDg2lzv1FDmDatGnMmzeP6dOn8+KLL15wH5PJdNlZRNzd3UvcrEFtIirg7GRif1Imh1OyCCuvf+hERC7IZLribjeOdt999/Hkk08ya9YsvvzySx555BHbjymrV6+mZ8+etpYFFouFmJgY6tate0XHrlOnDocPH+b48eNUrFgRgL/++stumzVr1lCtWjVefvll27JDhw7ZbePm5obZbL7suWbOnElmZqbtC9/q1atxcnKiVq1aV1Tv1Tp7fYcPH7a1ltq5cyepqal2r1FUVBRRUVE8/fTT9O3blxkzZtC7d28AwsLCePjhh3n44YcZNWoUn376aakIpZKSkjCbzbauVmeFhISwe/fuC+7TpUsXJk2aRHR0NOHh4SxdupSff/7Z9md/LccEa9iVk5Nje56enn7RbQvQZxkoHZ/lzz//nD59+tjVB/DGG2/w+eefc+utt9KwYUO++OIL8vLyCoRevr6+VK9enaVLl9KpU6cCxz87W+Hx48dt3U3PHfT8UlavXs3gwYNtfy9kZGRw8OBB2/oGDRpgsVhYvny5rfve+bp37463tzdTp05lwYIFrFix4orOLSLXLykjh8U7rUHUmn1J5Jn/DaerlPOkW/1QutYPpUlYOZycLj9WoBQ/DosPz/4id+5f/lfyi1xGRgbVqlUjLCyMnj17smPHjgLbLFu2jODgYGrVqsUjjzxCcnLyDbmGwuTn4UrTqgEALFdrKRGRUsHHx4f777+fUaNGcfz4cQYPHmxbFxkZyeLFi1mzZg27du3iP//5j93McpfTuXNnoqKiGDRoEFu3bmXlypUFvhBGRkYSFxfHd999x759+/jggw/45Zdf7LapXr06Bw4cYMuWLSQlJdkFDGf1798fDw8PBg0axD///MOff/7J448/zoABAwqEGFfLbDazZcsWu8euXbvo3LkzDRo0oH///mzatIn169czcOBAOnToQPPmzTl9+jQjRoxg2bJlHDp0iNWrV7Nhwwbq1KkDwFNPPcXChQs5cOAAmzZt4s8//7StK4vef/99IiMjqV27Nm5ubowYMYIhQ4bYWqBcqwkTJtjGVfL397frblma6LN8cYmJifz2228MGjSI+vXr2z0GDhzInDlzSElJYcSIEaSnp9OnTx/+/vtvYmNj+eqrr2w9IMaOHcu7777LBx98QGxsLJs2bWLKlCmAtTXTTTfdxFtvvcWuXbtYvny53RhblxIZGcnPP//Mli1b2Lp1K/369bNr9VW9enUGDRrEgw8+yJw5czhw4ADLli3j+++/t23j7OzM4MGDGTVqFJGRkRfsXikihedo6mmmrzrAfZ+speUbSxj183aWxySSZzaIDPbhiZsjmPdEO1Y+34mXb69Ls2rlFUiVYA4LpS71i1x8fPwF96lVqxbTp0/n119/5euvv8ZisdCmTRuOHDli26Zr1658+eWXLF26lLfffpvly5fTrVu3S/5ylJOTQ3p6ut3DETpEnR1XSqGUiEhpMXToUE6ePEmXLl3sxowZPXo0TZs2pUuXLnTs2JHQ0FB69ep1xcd1cnLil19+4fTp07Rs2ZJhw4bxxhtv2G1z55138vTTTzNixAgaN27MmjVreOWVV+y2ufvuu+natSudOnUiKCjoglPZe3l5sXDhQlJSUmjRogX33HMPt9xyCx9++OHVvRgXkJGRQZMmTewePXr0wGQy8euvv1KuXDmio6Pp3LkzNWvWZPbs2YD1S2JycjIDBw4kKiqK++67j27dujFu3DjAGnY99thj1KlTh65duxIVFcXHH3983fUWB4GBgTg7OxcIPk6cOHHRluJBQUHMmTOHzMxMDh06xO7du/Hx8bEN6HwtxwQYNWoUaWlptsfhw4ev8+qKL32WL+zsoOkXGg/qlltuwdPTk6+//poKFSrwxx9/kJGRQYcOHWjWrBmffvqprdXUoEGDmDx5Mh9//DH16tXjjjvuIDY21nas6dOnk5+fT7NmzXjqqad4/fXXr6i+SZMmUa5cOdq0aUOPHj3o0qULTZs2tdtm6tSp3HPPPTz66KPUrl2b4cOHk5mZabfN0KFDyc3NtfXwEJHCtS8xg4/+3MudH66i7Vt/MP7/drL+QAoWAxpW8ee5LrVYMrIDi0d2YORttahXyf+CQ/lIyWMyzu2cXYSOHTtG5cqVWbNmjd2vDc8//zzLly9n3bp1lz1GXl4ederUoW/fvrz22msX3Gb//v2Eh4ezZMmSiw6eOHbsWNtN7LnS0tLw8/O7wiu6ftuOpHLnh6vxcXdh85hb1Q9WRATIzs7mwIED1KhRo8CYOCKF4VLvsfT0dPz9/Yv8nuByWrVqRcuWLW0tSSwWC1WrVmXEiBEXHQLhXGfvoe677z7efPPNQjkmXPz10udYSrqVK1dyyy23cPjw4Uu2KtN7XeTKGIbBzuPpLPwnnvn/xBObkGFbZzJBi+rl6VovlNvqhVClnIa2KYmu9B7KYWNKXesvcudydXWlSZMm7N2796Lb1KxZk8DAQPbu3XvRUGrUqFGMHDnS9jw9Pd0hzc/rV/KnvLcbKZm5bDp0klY1KxR5DSIiIlL8jRw5kkGDBtG8eXNatmzJ5MmTyczMtLXiGDhwIJUrV2bChAkArFu3jqNHj9K4cWOOHj3K2LFjsVgsPP/881d8TJGyKCcnh8TERMaOHcu999573V2WRcoyi8Vg8+GTLPgnngU74jmcctq2ztXZRJvwQLrWD6VznRCCfIv/ZGVSOBwWSrm5udGsWTOWLl1qa+JssVhYunQpI0aMuKJjmM1mtm/fTvfu3S+6zZEjR0hOTrYNHHkh7u7uxWKGPicnE+0jA/l1yzFWxCYqlBIREZELuv/++0lMTGTMmDHEx8fTuHFjFixYYPvCHBcXZzdeVHZ2NqNHj2b//v34+PjQvXt3vvrqKwICAq74mCJl0bfffsvQoUNp3LgxX375paPLESlx8swW1u1PYcGO4yzacYKEU/+Od+fh6kSHqCC61g/l5toh+Hu6XuJIUlo5rPsewOzZsxk0aBCffPKJ7Re577//nt27dxMSElLgV77x48dz0003ERERQWpqKhMnTmTOnDls3LiRunXrkpGRwbhx47j77rsJDQ1l3759PP/885w6dYrt27dfcfDkyKb6P208wjM/bKVBZX9+e7xdkZ5bRKQ4UlcIudFKYve94krd96Ss03tdyrrsPDM7j6ez9XAqm+NSWR6TSNrpPNt6X3cXbqkTTNf6oURHBeHl5rB2MnKDFfvue3D1v/KdPHmS4cOHEx8fT7ly5WjWrBlr1qyxTbnr7OzMtm3b+OKLL0hNTaVSpUrcdtttvPbaa8WiJdSVaB8VCMD2o2kkZeQQ6FMy6hYREREREZGyw2wx2J+YwZbDqWw9ksrWw2nsOp5OvsW+3UsFbzduqxdCl3qhtAkPxM1FYyfLvxweS44YMeKi3fWWLVtm9/y9997jvffeu+ixPD09WbhwYWGWV+SCfT2oU9GPXcfTWRWbRK8mlR1dkoiIiIiIiJRhhmEQn57N1sOpbDmcxtbDqWw/mkZGTn6BbQN93GhUJYBGYQG0rFGeFtXL4+ykmfLkwhweSklBHaKC2HU8nRUxiQqlRETOsFgsji5BSikHjmRQ5uhzLKWd3uNSWqSdzmP7kTS2Hkm1toQ6nGo3HtRZnq7ONKjiT+OwgDNBlD+VAzwxmRRCyZVRKFUMRUcFMm35PlbEJmGxGDgpVRaRMszNzQ0nJyeOHTtGUFAQbm5uutGRQmMYBomJiZhMJlxdNcDqjaLPsZR2hmGQm5tLYmIiTk5OuLm5ObokkSuWk29m1/FTbD0TPm05ksr+xMwC2zk7magV4kujsAAah/nTKCyAiCAfXJzVHU+unUKpYqh5tfJ4uTmTlJHDzuPp1K/s7+iSREQcxsnJiRo1anD8+HGOHTvm6HKkFDKZTFSpUgVnZ2dHl1Jq6XMsZYWXlxdVq1a1GxdXpDixWAz2J2VaA6gj1hBq5/F08swFWw1XLe9Fo7AAGp1pCVWvkj+ebvq3UgqXQqliyM3FiTbhFViyK4EVsYkKpUSkzHNzc6Nq1ark5+djNpsdXY6UMq6urgqkioA+x1LaOTs74+LiolaAUqycSM+2db/beiSVbYfTOHWBcaDKe7vRqIq19VOjM13xynurxZ/ceAqliqnoqCBrKBWTyKMdIxxdjoiIw53tXqUuViIllz7HIiI3Tnp2Hv8cSWPLmRZQWw+nEZ+eXWA7D1cnGlT2tw1G3jgsgCrlNA6UOIZCqWIqOjIIgL8PniQjJx8fd/1RiYiIiIiICOTmW9gdn/7vbHhHUtmXmMH5c3c4maBWqN+/raCqBBAVonGgpPhQ0lFMVQ/0pmp5L+JSsli7L5lb64Y4uiQREREREREpYhaLwcHkzDNjQKWx5XAqO4+lk2suONtjWHlPGlWxtn5qFBZAvUp+eLnpa78UX3p3FmMdooL46q9DrIhJVCglIiIiIiJSBiSkZ7P1SJrdYOTp2QXHgSrn5Wpr/dQ4LICGVfyp4OPugIpFrp1CqWIs+mwoFZvo6FJERERERETkBkg4lc2y3Yksj0lkc9xJjqUVHAfK3eXMOFC2gcj9qVreS+NASYmnUKoYax1eARcnE4eSsziYlEn1QG9HlyQiIiIiIiLXwWIx2HEsnaW7T/DH7gS2HUmzW+9kgqgQX9tA5I3C/IkK8cVV40BJKaRQqhjzcXehefVy/LU/hRWxiQqlRERERERESqCs3HxWxSbxx+4E/tidQMKpHLv1jar406l2MK1rVqB+ZX+8NdGVlBF6pxdz0VFB1lAqJpGBras7uhwRERERERG5AodTsmwh1Nr9yeTm/zswubebM+0jg7i5TjAdawUR7OvhwEpFHEehVDEXHRnEfxfsYc0+619ibi5qsikiIiIiIlLc5JstbD6cytJdCfyx+wQxJzLs1oeV9+SW2iHcUieYljXK4+7i7KBKRYoPhVLFXN2KfgT6uJGUkcvfh1JoEx7o6JJEREREREQESMvKY3lsIn/sOsGymERSs/Js65ydTDSvVo6bawdzS51gwoN8NDC5yHkUShVzTk4moiOD+HnzUVbEJCmUEhERERERcRDDMNiXmMHSXQks3Z3AxkMnMVsM23p/T1c61Qri5johdIgMwt/L1YHVihR/CqVKgOios6FUIi92q+3ockRERERERMqMnHwz6w+knOmWl0BcSpbd+qgQH24+0y2vSVgALpolT+SKKZQqAdpFWltH7TyeTsKpbA2CJyIiIiIicgMlnMpm2e5E/tidwMrYRDJzzbZ1bs5OtA6vwC11gulUK5iw8l4OrFSkZFMoVQIE+rjToLI/24+msTImibubVXF0SSIiIiIiIqWGYRjsOJZuG6R865E0u/VBvu7cUjuYm2sH0zYiEG93fZUWKQz6JJUQ0VGBbD+axorYRIVSIiIiIiIi1ykrN5/Ve5P5Y/cJ/tidwIn0HLv1Dav4Wwcprx1CvUp+ODlpkHKRwqZQqoSIjgzioz/3sTI2CYvF0F+IIiIiIiIiV+lwShZ/7klg6a4E1u5PJjffYlvn5eZM+8hAbqkdQsdaQQT7adgUkRtNoVQJ0bRaOXzcXUjJzOWfY2k0rBLg6JJERERERESKNbPFYHPcSZbuTuCPXQnsOXHKbn1YeU9uqR3CzbWDaVWzPO4uzg6qVKRsUihVQrg6O9EmvAKLdp5gRUyiQikREREREZELSMvKY3lsIn/sOsGymERSs/Js65ydTDSrVu5Mt7xgIoJ9MJnUC0XEURRKlSDRUUFnQqkkRtwc6ehyREREREREHM4wDPYlZvLH7hMs3ZXA34dOYrYYtvX+nq50rBXEzbWD6RAVRICXmwOrFZFzKZQqQTpEBQGwMe4k6dl5+Hm4OrgiERERERGRopebb2H9gRSWnhmk/FBylt36yGAfbq5jHaS8adUAXJydHFSpiFyKQqkSJKy8FzUDvdmflMmavcl0rR/q6JJERERERESKRFZuPsv3JLJwRzxLdydwKjvfts7N2YmbwitwS+1gbq4dTFh5LwdWKiJXSqFUCRMdFcT+pExWxCYqlBIRERERkVItNSuXJbsSWLgjnhUxieScM1teoI+7NYSqE0y7iEC83fX1VqSk0ae2hImOCmTmmoMs35OIYRgalE9EREREREqV+LRsFu2MZ+GOeP7an2I3PlTV8l50qRdCl3qhNKlaDmcnfR8SKckUSpUwN9WsgJuzE0dTT7M/KZPwIB9HlyQiIiIiInJd9idmsHDHCRbuiGfL4VS7dbVDfelSL5Qu9UKpU9FXP8yLlCIKpUoYLzcXWtQox+q9yayISVQoJSIiIiIiJY5hGOw4ls7CHdYWUTEnMuzWN6tWztYiqloFbwdVKSI3mkKpEig6MsgWSg1pW8PR5YiIiIiIiFyW2WKw8dBJWxB15ORp2zoXJxOtwyvQpV4ot9UNIdjPw4GVikhRUShVAkVHBTFh/m7W7k8mO8+Mh6uzo0sSEREREREpICffzJp9ySzaEc/inSdIysi1rfNwdaJDVBBd64dyc60Q/L1cHVipiDiCQqkSqHaoL8G+7iScyuHvgydpFxno6JJEREREREQAyMzJZ9meRBbuiOfP3Qmcysm3rfPzcKFznRC61A8lOjIITzf9wC5SlimUKoFMJhPRUUH8uPEIK2ITFUqJiIiIiIhDpWTmsmTXCRbtiGdFbBK5+RbbumBfd26rF0LXehVpVbM8rs5ODqxURIoThVIllC2Uiknkpe51HF2OiIiIiIiUMcdST7NoRzwLd5xg/cEUzBbDtq56BS/rjHn1Q2lcJQAnJ82YJyIFKZQqodpHBGIywe74U8SnZRPqr4EARURERETkxtqXmMGCf+JZtCOerUfS7NbVreh3JogKoVaILyaTgigRuTSFUiVUOW83GlYJYOvhVFbEJnJf8zBHlyQiIiIiIqWMYRj8czSdhTviWbAjnr0JGbZ1JhM0r1bOGkTVCyWsvJcDKxWRkkihVAnWITLQGkrFKJQSEREREZHCYbYYbDiYwsId8SzacYKjqadt61ydTbQJD6RLvVA61w0m2Fc9NkTk2imUKsGio4L44I+9rIxNwmwxcFY/bRERERERuQY5+WZW701i4T8nWLzrBCmZubZ1nq7OdKodRJd6oXSqHYyfh6sDKxWR0kShVAnWOCwAXw8X0k7nse1IKk2qlnN0SSIiIiIiUkJk5OTz5+4EFu6IZ9meRDJy8m3rArxc6VwnhC71QmkfGYiHq7MDKxWR0kqhVAnm4uxEu4hA5v8Tz4qYJIVSIiIiIiJySckZOSzZdYKFO06wam8SufkW27pQPw9uqxdC13qhtKxRHhdnJwdWKiJlgUKpEi46KsgaSsUm8mTnSEeXIyIiIiIixYzFYvDL5qN8//dhNhxMwWL8u65moDe31Qula/1QGlb2x0lDgohIEVIoVcJFRwUBsDnuJGlZefh7qX+3iIiIiIhY/X0whXG/7WT70TTbsvqV/ehS1xpERQT7YDIpiBIRx1AoVcJVDvAkItiHvQkZrN6XRPcGFR1dkoiIiIiIONjR1NO8NX83v209BoCvuwsPdwznzkaVCCvv5eDqRESsFEqVAtGRQexNyGBFTKJCKRERERGRMux0rplpy/fxyYp9ZOdZMJmgT4swnrmtFoE+7o4uT0SKmmFAfg7kZkJuBuRl/fv/uVlQviYE13ZYeQqlSoHoqECmrz7A8phEDMNQ81sRERERkTLGMAzmbj3GW/N3czwtG4CWNcoz5o661K/s7+DqROSyDAPMuWcCo3MfFwiSrnj5meeG5eLnjX4Obh5ddNd5HoVSpcBNNSvg7uLE8bRs9iZkEBni6+iSRERERESkiGw7ksq433ay8dBJwDrEx8u316Fb/VD9YC1yI+TnXjoUys08s+7M/182SMqEvEyw5N/Yul08wM3b+nA981/f0Bt7zsuV5NCzS6HwcHWmZY3yrIxNYnlMokIpEREREZEyICE9m/8u3MOPG48A4OXmzKMdwxnWviYers4Ork6kFMjPgSMbYP9yOLAcEvdYAyRL3o09r7P7mfDIB9y8CgZJ5z/slp+7jw+4nrO/U/H7e8HhodRHH33ExIkTiY+Pp1GjRkyZMoWWLVtecNuZM2cyZMgQu2Xu7u5kZ2fbnhuGwauvvsqnn35Kamoqbdu2ZerUqURGRt7Q63C0DlFBrIxNYkVsEsPa13R0OSIiIiIicoNk55mZvvoAH/2xl8xcMwB3NanM811rE+rv4eDqREowiwXit1kDqP3L4NBayD998e2d3c8EQD5nwiGvywRDV7Dc1RucHR7VFBmHXuns2bMZOXIk06ZNo1WrVkyePJkuXbqwZ88egoODL7iPn58fe/bssT0/vznqf//7Xz744AO++OILatSowSuvvEKXLl3YuXMnHh6l9y/oDlFBvD5vF+v2J5OdZ9YvIyIiIiIipYxhGCzccYI3ft/J4RTrF+XGYQG82qMuTaqWc3B1IiWQYUDyPjiwzNoa6uBKOH3SfhvvIKjRAWp2gCotwMP/3zDJ2dUhZZcmDg2lJk2axPDhw22tn6ZNm8a8efOYPn06L7744gX3MZlMhIZeuM+jYRhMnjyZ0aNH07NnTwC+/PJLQkJCmDNnDn369LkxF1IMRAT7UNHfg+Np2aw7kEKHqCBHlyQiIiIiIoVk1/F0xv+2k7X7kwEI8XPnxW616dmoMk5OGjdK5IqlH4cDK6wtoQ4sh/Sj9uvdfKF623+DqOC6oLHZbhiHhVK5ubls3LiRUaNG2ZY5OTnRuXNn1q5de9H9MjIyqFatGhaLhaZNm/Lmm29Sr149AA4cOEB8fDydO3e2be/v70+rVq1Yu3btRUOpnJwccnJybM/T09Ov9/KKnMlkIjoyiNl/H2ZFTKJCKRERERGRUiA5I4dJi2P4dn0cFgPcXZx4KLomD3cIx9u97HTxEblmp1Ph4KozXfKWQ9Ie+/XObhDW6t8QqlLTMtV9ztEc9konJSVhNpsJCQmxWx4SEsLu3bsvuE+tWrWYPn06DRs2JC0tjXfeeYc2bdqwY8cOqlSpQnx8vO0Y5x/z7LoLmTBhAuPGjbvOK3K86ChrKLU8JpFXHF2MiIiIiIhcszyzhS/XHmLykhhOZVtn5Lq9QUVe7FabsPJeDq5OpBjLOw2H11kDqP3L4PgWMCznbGCCio2sAVTNjhB2k3U8J3GIEhX/tW7dmtatW9uet2nThjp16vDJJ5/w2muvXfNxR40axciRI23P09PTCQsLu65aHaFdRCBOJtibkMGx1NNUCvB0dEkiIiIiInKV/tyTwGv/t5P9iZkA1Kvkx5g76tKqZgUHVyZSDJnzrcHT2e54cevAnGO/TYVIawhVowNUbwde5R1RqVyAw0KpwMBAnJ2dOXHihN3yEydOXHTMqPO5urrSpEkT9u7dC2Db78SJE1SsWNHumI0bN77ocdzd3XF3d7/KKyh+/L1caRwWwKa4VFbEJNKnZVVHlyQiIiIiIldob0IGr8/bybI9iQAE+rjx7G21uLd5GM4aN0rEyjAgcc+/3fEOroKcNPttfCv+2x2vRgfwr+yYWuWyHBZKubm50axZM5YuXUqvXr0AsFgsLF26lBEjRlzRMcxmM9u3b6d79+4A1KhRg9DQUJYuXWoLodLT01m3bh2PPPLIjbiMYic6KsgaSsUqlBIRERERKQnSsvKYvDSGr9YeIt9i4OpsYkjbGoy4OQI/D83uJULq4X9DqAMrIOO84Xk8/KF6e2t3vBodIDBSg5OXEA7tvjdy5EgGDRpE8+bNadmyJZMnTyYzM9M2G9/AgQOpXLkyEyZMAGD8+PHcdNNNREREkJqaysSJEzl06BDDhg0DrIN9P/XUU7z++utERkZSo0YNXnnlFSpVqmQLvkq7DlFBTF4Sy8rYJPLNFlycnRxdkoiIiIiIXEC+2cJ3Gw7z7qI9nMzKA6BznRBevr0ONQK9HVydiANlpVjDp7NBVMo++/UuHlD1pjOtoTpax4hycnZIqXJ9HBpK3X///SQmJjJmzBji4+Np3LgxCxYssA1UHhcXh5PTv6HKyZMnGT58OPHx8ZQrV45mzZqxZs0a6tata9vm+eefJzMzk4ceeojU1FTatWvHggUL8PDwKPLrc4SGVQII8HIlNSuPrUdSaVZNfWVFRERERIqbNXuTGP9/O9kdfwqAyGAfxvSoS/tIzaItZVBuJhxaCweWWUOo+O2A8e96kzNUbvpvl7wqLcG1bHzHL+1MhmEYl9+sbElPT8ff35+0tDT8/PwcXc5Ve2zWJuZtO84Tt0Qy8tYoR5cjIiJSYpX0e4KiptdL5PIOJWfyxrxdLNppHVvX39OVkbdG0b9VVfVykLLDnAdHN57pjrccDq8HS579NkF1rK2ganaAam2sXfSkxLjSe4ISNfueXJkOkUHM23ac5TGJCqVERERERIqBjJx8PvxjL9NXHSDXbMHZycQDraryVOcoynm7Obo8kRvLYoGEHf+GUIfWQG6G/Tb+VaFmNNToCDWiwTfEEZVKEVMoVQpFR1mb/G47ksrJzFz9IyciIlIKffTRR0ycOJH4+HgaNWrElClTaNmy5UW3nzx5MlOnTiUuLo7AwEDuueceJkyYYBviwGw2M3bsWL7++mvi4+OpVKkSgwcPZvTo0Zg0WKzINbNYDH7cdISJC/eQeMo6TX37yEBeuaMuUSG+Dq5O5AYxDDh50H5w8qwk+208y1vDp7OtocrV0ODkZZBCqVIo1N+DWiG+7DlxilV7k+jRqJKjSxIREZFCNHv2bEaOHMm0adNo1aoVkydPpkuXLuzZs4fg4OAC28+aNYsXX3yR6dOn06ZNG2JiYhg8eDAmk4lJkyYB8PbbbzN16lS++OIL6tWrx99//82QIUPw9/fniSeeKOpLFCkV/j6YwrjfdrL9qHW6+uoVvHjljrrcXDtYYa+ULoYBqYfgwEo4uMr6SD9iv42rt7UbXs0O1rGhQuqDk7qslnUKpUqp6KhA9pw4xYqYRIVSIiIipcykSZMYPny4bcbiadOmMW/ePKZPn86LL75YYPs1a9bQtm1b+vXrB0D16tXp27cv69ats9umZ8+e3H777bZtvv32W9avX18EVyRSuhxNPc1b83fz29ZjAPi6u/DELZEMalMdNxd9CZdS4uQhOHhOCJV22H69kytUaW5tCVWjA1RuBi7qxSP2FEqVUh2igvl05QFWxCZiGIZ+iRERESklcnNz2bhxI6NGjbItc3JyonPnzqxdu/aC+7Rp04avv/6a9evX07JlS/bv38/vv//OgAED7Lb53//+R0xMDFFRUWzdupVVq1bZWlJdSE5ODjk5Obbn6enphXCFIiXX6Vwz05bv45MV+8jOs2AyQZ8WYTxzWy0CfdwdXZ7I9Tl56N8A6uAqSIuzX+/kag2eqrezPsJagZuXY2qVEkOhVCnVvHo5PFydOJGew54Tp6gdqhlwRERESoOkpCTMZjMhIfYDwIaEhLB79+4L7tOvXz+SkpJo164dhmGQn5/Pww8/zEsvvWTb5sUXXyQ9PZ3atWvj7OyM2WzmjTfeoH///hetZcKECYwbN65wLkykBDMMg7lbj/HW/N0cT8sGoGWN8oy5oy71K2vGMCmhUuPOCaFWWp+fy8nlnBCqPYS1BDdvx9QqJZZCqVLKw9WZm2pWYNmeRFbEJCqUEhERKcOWLVvGm2++yccff0yrVq3Yu3cvTz75JK+99hqvvPIKAN9//z3ffPMNs2bNol69emzZsoWnnnqKSpUqMWjQoAsed9SoUYwcOdL2PD09nbCwsCK5JpHiYtuRVMb9tpONh04CUDnAk5dvr0O3+qHqrXAjnDoBh/+CuL8geR+UqwZBtSCoNgTVAe8Kjq6w5Eo9/G8AdbEQqlJTqNH+nJZQCqHk+iiUKsWiI4POhFJJPBQd7uhyREREpBAEBgbi7OzMiRMn7JafOHGC0NDQC+7zyiuvMGDAAIYNGwZAgwYNyMzM5KGHHuLll1/GycmJ5557jhdffJE+ffrYtjl06BATJky4aCjl7u6Ou7u6JEnZlJCezX8X7uHHjdbBnD1dnXm0YzjDo2vi4ers4OpKCcOApFiIW2sNoeLWwskDl97HK9AaUAXXPhNU1ToTVgVqZrfzpR05Z2DyldaBys9lcobKTa2toM6GUO4+jqlVSi2FUqVYh1pB8H+w/kAKWbn5eLnpj1tERKSkc3Nzo1mzZixdupRevXoBYLFYWLp0KSNGjLjgPllZWTidN8ORs7P1S7NhGJfcxmKxFPIViJRs2Xlmpq8+wEd/7CUz1wzAXU0q83zX2oT6ezi4uhIuPweObz0nhPoLTqect5HJOmtb1ZuswdPJQ5C4BxJ3W0OVrCQ4tMr6OJdn+QuHVT7BZSesSjtyTkuoVXDyoP16Wwh1dkyomxRCyQ2nlKIUqxnoTeUAT46mnmbd/hQ61S44RbSIiIiUPCNHjmTQoEE0b96cli1bMnnyZDIzM22z8Q0cOJDKlSszYcIEAHr06MGkSZNo0qSJrfveK6+8Qo8ePWzhVI8ePXjjjTeoWrUq9erVY/PmzUyaNIkHH3zQYdcpUpwYhsHCHSd44/edHE45DUDjsABe7VGXJlXLObi6Eup0Khxebw2hDq+DoxshP9t+GxcPqNzcGkJVbQ1hLcDjIuN05WZCUow1pErY9W9YdfKgNdyKW2N9nMsj4MJhlW9oyQ+r0o6eF0Kd18rM5AyVmvw7JlTVVuDu65hapcxSKFWKmUwmoqOC+HZ9HMtjEhVKiYiIlBL3338/iYmJjBkzhvj4eBo3bsyCBQtsg5/HxcXZtXoaPXo0JpOJ0aNHc/ToUYKCgmwh1FlTpkzhlVde4dFHHyUhIYFKlSrxn//8hzFjxhT59YkUN7uOpzP+t52s3Z8MQIifOy92q03PRpVxcirhwUVRMQxIO/xvC6i4vyBhJ2DYb+dVwRo+nQ2hQhuCi9uVncPN2xqyVGpivzw3C5JjLxBWHYDsVOsYVYf/st/H3f9MQFULguv8O26VX+XiG1alH7MPoVL22683OdmHUGGtwENjD4tjmYyzbbbFJj09HX9/f9LS0vDzK9kf0gX/xPPw1xupGejNH892dHQ5IiIiJUppuicoCnq9pLRJzshh0uIYvl0fh8UANxcn/hNdk4c7hOPtrt/3L8litoZOZ8eCivsL0o8W3K58uH0IVSG86EKfvOx/w6rE3f8GVin7wTBfeB83338DqnNbV/mHFX1YlX4MDq6GgysuHkJVbHxOS6ibFEJJkbnSewL9TVrKtYmogLOTif1JmRxOySKsvJejSxIRERERKdbyzBa+XHuIyUtiOJWdD8DtDSryYrfaup++mNwsa/e7syHUkQ2Qk26/jZMLVGx0phteK2tI4uPA3hyuHhDawPo4V34OJO+1BlXntq5K2Qe5p+Do39bHudx8IDDqAmFVVThvvL5rln78vJZQ++zXm5ysr69dCHWRro4ixYRCqVLOz8OVplUD2HDwJCtiE+nfqpqjSxIRERERKbZWxSYx9rcd7E3IAKBuRT9e7VGXVjUrOLiyYiYj0drl7WxXvONbwJJvv42br3UMqLMtoSo3s3axK+5c3CGknvVxrvxcaxB0fliVvBdyM+DYJuvjXK5eFw6rAqpfPqw6FW8fQiXvtV9vcrJ2b6zeDmpEK4SSEkmhVBkQHRlkDaViFEqJiIiIiFzI4ZQsXp+3k4U7TgBQ3tuN57rU4r7mYTiX9XGjDAOS950ZkPxMCHV+QALgW/FMAHUmhAqpB07ORV/vjeLiZh1fKriO/XJzHqQcgMRzxqtK2G3tGpiXZQ3sjm8571ieEBj5b0gVXAfKVbcGXQdXnQmhYs8rwAQVG1pbQZ1tCeUZcMMuV6QoKJQqAzrUCuLdxTGs3ptMntmCq3MhNR8VERERESnhTueambp8H58s30dOvgVnJxMDW1fjqc5R+Hu6Oro8x8jPhfht9uNBZSUV3C647jmz4rWCgKrFdxDwG8nZFYKirI9zmfOtM/8l7jqnddVu6wyB+aetr3H8tksc+NwQqp31dVYIJaWMQqkyoH4lf8p7u5GSmcvmuFRa1ijv6JJERERERBzKMAx+3x7PG/N2ciwtG4A24RUYe2c9okJ8HVxdEctOhyPr/+2Kd+Rva2hyLmd3a/e7qjdZH2EtwbOcY+otKZxdIDDC+qjT49/lFvOZsGqPfeuq5P1Qvsa/IVS11nqNpdRTKFUGODmZaBcRyNytx1gRk6hQSkRERETKtD3xpxg7dwdr9ycDUDnAk9G316Fr/VBMZaGlT9rRf1tAxf0FCTvAsNhv41kOwm76tyVUpcbWsZbk+jk5W2cZrBAOtbs7uhoRh1IoVUZ0iApi7tZjLI9J5NkutRxdjoiIiIhIkUvLyuO9JTF89dchzBYDdxcnHu4QzsMdwvF0KyVjH1nMcPokZCVDZpL1v1ln/psYYw2h0uIK7leu+r9jQVVtDRUiC2/WOBGRi1AoVUa0jwoE4J9jaSRn5FDBR79yiIiIiEjZYLYYfP/3YSYu3ENKZi4A3eqH8lL3OoSV93JwdZdgGNaBsm3h0jmP85edfX76JGBc+rhnZ22zhVA3gW9okVySiMi5FEqVEcG+HtSp6Meu4+ms2ptEz8aVHV2SiIiIiMgNt/FQCmPn7mT70TQAIoN9eLVHPdpFBhZ9MRYzZKXYt17KTDqzLOm8gOnMsvzsazuXhz94BYJXBevDuwL4h1kHJK/SHNzL2LhZIlIsKZQqQ6KjAtl1PJ3lMYkKpURERESkVEtIz+at+bv5efNRAHw9XHi6cxQDWlcrnNmoDQNyM88Jk1IKdpfLTLZ/fjqVy7ZiuhBnt38DJu8zIVOB5+cs8ypvnRFORKSYUyhVhnSICuKT5ftZEZOExWLg5FQGBnEUERERkTIlN9/CjNUH+GBpLJm5ZkwmuK9ZGM91rUXg1QxhEbcOTvxzge5ySf8GUOacayvSs9x5QVJ58A48L1g6J3By84GyMAC7iJQ5CqXKkObVyuPl5kxSRg674tOpV8nf0SWJiIiIiBSaP/ck8NpvO9mflAlA47AAxt1Zj0ZhAVd+EHMeLHoF1k29su2d3c8JlCqcFy6VL7jMsxw462uYiAgolCpT3FycaF2zAkt3J7AiJkmhlIiIiIiUCgeTMnnt/3aydHcCAIE+7rzYrTZ3Nal8db0DMhLhxyFwcKX1eeRt4FvxvGCpgn0A5eqlVkwiItdIoVQZ06FWEEt3J7A8JoFHOoY7uhwRERERkWuWmZPPR3/u5bOVB8g1W3BxMvFguxo8fnMEvh5XOabS0U0wewCkH7F2l+v9CdS548YULiIigEKpMic6MgiAjYdOkpmTj7e73gIiIiIiUrIYhsHcrceY8Ptu4tOts9O1jwzk1R71iAj2ufoDbpkFvz1lHSOqQgT0mQVBtQq3aBERKUCJRBlTPdCbquW9iEvJYu2+ZDrXDXF0SSIiIiIiV2zHsTTGzt3BhoMnAaha3otX7qhL5zrBmK62G505Dxa+DOs/sT6P6gZ3fQIeGuZCRKQoKJQqg6KjAvn6rzhWxCYqlBIRERGREuFkZi7vLt7DrHVxWAzwdHXmsU7hDGtfEw9X56s/YEYC/DAYDq22Pu84CqKfByenQq1bREQuTqFUGdQhKpiv/4pjeUyio0sREREREbkks8Vg1rpDvLMohrTTeQDc0bAiL3WvQ6UAz2s76NGNZ8aPOgruftbxo2p3L8SqRUTkSiiUKoNah1fAxcnEoeQsDiVnUq2Ct6NLEhEREREpYN3+ZMb+tpNdx9MBqB3qy9g763FTzQrXftDNX8P/jbSOHxUYZR0/KjCykCoWEZGroVCqDPJxd6FZtXKsO5DCiphEBrRWKCUiIiIixcfxtNO8+ftuftt6DAB/T1eeuS2Kfi2r4uJ8jd3r8nNh4SjY8Jn1ea3bofc08PArpKpFRORqKZQqozrUCmLdgRSWxyQyoHV1R5cjIiIiIkJ2npnPVx3gwz/2cjrPjMkE/VpW5ZnbalHe2+3aD3zqBPwwCOLWAibo9BK0f1bjR4mIOJhCqTIqOjKI/y7Yw9p9yeTmW3Bz0T/IIiIiIuIYhmGwZFcCr/3fTuJSsgBoUb0cr/aoR/3K1zkT3pG/YfYDcOq4dfyouz6FWl0LoWoREbleCqXKqLoV/Qj0cSMpI5eNh07SOvw6+uWLiIiIiFyjfYkZjP9tp20SnhA/d17qXoc7G1XCZDJd38E3fQnzngFzLgTWOjN+VEQhVC0iIoVBoVQZ5eRkon1kEL9sPsqK2ESFUiIiIiJSpE5l5zHlj71MX3WAfIuBm7MTQ9vXYESnCLzdr/NrSn4uLHgB/p5ufV77Duv4Ue6+11+4iIgUGoVSZViHKGsotXxPIi90re3ockRERESkDLBYDH7ZfJS3Fuwm8VQOALfUDmb0HXWpEVgIE/CciofvB8LhdYAJbh4N7UZq/CgRkWJIoVQZ1i4yEICdx9NJPJVDkK+7gysSERERkdJs25FUxs7dwaa4VABqBHoz5o66dKodXDgnOLweZg+AjHhw94e7P4Oo2wrn2CIiUugUSpVhgT7u1K/sxz9H01kZm8hdTas4uiQRERERKYWSMnJ4Z+EeZv99GMMAbzdnHr8lkiFtq+Pu4lw4J/l7Bvz+HFjyIKgO9PkGKoQXzrFFROSGUChVxnWICuKfo+ksj1EoJSIiIiKFK89s4eu/DjFpcQynsvMBuKtJZV7oVpsQP4/COUl+jjWM2vSF9XndntDzY3D3KZzji4jIDaNQqoyLjgzioz/3sTI2CYvFwMnpOmc4EREREREB1uxNYuxvO4g5kQFA/cp+jO1Rj+bVyxfeSdKPw/cD4MgGwAS3jIF2T8P1ztonIiJFQqFUGde0Wjl83F1Iycxlx7F0GlTxd3RJIiIiIlKCHTmZxRvzdjH/n3gAynu78VyXWtzXPAznwvwBNO4v64DmGSfAwx/ung6RnQvv+CIicsMplCrjXJ2daB1egcU7T7AiNlGhlIiIiIhck+w8M9OW72Pqsn3k5FtwdjIx4KZqPN05Cn8v18I7kWHA39Nh/gvW8aOC60Gfr6F8zcI7h4iIFAmFUkKHqCAW7zzB8j2JPNYpwtHliIiIiEgJs+1IKo98vYmjqacBuKlmecbeWY/aoX6Fe6K8bPj9Wdj8lfV5vd7Q8yNw8y7c84iISJFQKCV0iAoCYFPcSU5l5+HrUYi/ZImIiIhIqbYp7iSDPl/PqZx8Kvl78PLtdeneIBRTYY/rlHbUOn7U0Y1gcoJbXoW2T2r8KBGREkyhlBBW3osagd4cSMpkzb5kutQLdXRJIiIiIlICbDyUwqDpG8jIyadl9fJMH9ICH/cb8BXj0Brr+FGZieARAPdMh4hbCv88IiJSpJwcXYAUD2dbS62ISXRwJSIiIiJSEqw/kMLAz9eTkZNP65oVmPngDQikDAPWfwpf9LAGUiH14aFlCqREREoJh4dSH330EdWrV8fDw4NWrVqxfv36K9rvu+++w2Qy0atXL7vlgwcPxmQy2T26du16AyovXaKjAgFYHpOIYRgOrkZEREREirO1+5IZNH09mblm2kUEMn1wC7zcCjmQysuGX0dYx5Cy5EP9u2HoIihfo3DPIyIiDuPQUGr27NmMHDmSV199lU2bNtGoUSO6dOlCQkLCJfc7ePAgzz77LO3bt7/g+q5du3L8+HHb49tvv70R5ZcqN9WsgJuzE0dOnuZAUqajyxERERGRYmr13iSGzFzP6Twz0VFBfDaoOZ5uzoV7krQjMKMrbPnaOn7Uba/D3Z9rQHMRkVLGoaHUpEmTGD58OEOGDKFu3bpMmzYNLy8vpk+fftF9zGYz/fv3Z9y4cdSseeFpX93d3QkNDbU9ypUrd6MuodTwcnOheXXr66QufCIiIiJyIctjEnlw5gay8yzcXDuY/w1ohodrIQdSB1fBJx3g2GbwLA8P/AxtHteA5iIipZDDQqnc3Fw2btxI586d/y3GyYnOnTuzdu3ai+43fvx4goODGTp06EW3WbZsGcHBwdSqVYtHHnmE5OTkS9aSk5NDenq63aMsOjuu1HKFUiIiIiJynj93JzD8y7/JybfQuU4IUx9oWriBlGHAuk/gy56QlQShDazjR4V3KrxziIhIseKwUCopKQmz2UxISIjd8pCQEOLj4y+4z6pVq/j888/59NNPL3rcrl278uWXX7J06VLefvttli9fTrdu3TCbzRfdZ8KECfj7+9seYWFh13ZRJVz0mVDqr/0p5ORf/PUSERERkbJlyc4T/OerjeTmW+hSL4SP+zfF3aUQA6m80zDnEZj/vHX8qAb3wYOLoFy1wjuHiIgUOzdgvtYb49SpUwwYMIBPP/2UwMDAi27Xp08f2/83aNCAhg0bEh4ezrJly7jllgvP0jFq1ChGjhxpe56enl4mg6naob4E+7qTcCqHvw+epG3ExV9nERERESkbFu6IZ8SsTeSZDW5vUJHJfRrj6lyIv22nHobZ/eH4VjA5w22vwU2PqrueiEgZ4LBQKjAwEGdnZ06cOGG3/MSJE4SGhhbYft++fRw8eJAePXrYllksFgBcXFzYs2cP4eHhBfarWbMmgYGB7N2796KhlLu7O+7u7tdzOaWCyWQiOiqIHzceYUVMokIpERERkTLu9+3HeeLbzeRbDHo0qsR79zXCpTADqQMr4YdBkJUMXhXgnhlQs0PhHV9ERIo1h3Xfc3Nzo1mzZixdutS2zGKxsHTpUlq3bl1g+9q1a7N9+3a2bNlie9x555106tSJLVu2XLRl05EjR0hOTqZixYo37FpKk2iNKyUiIiIiwG9bj/H4mUCqd5PKhRtIGQb8NfXM+FHJULGRdfwoBVIiImWKQ7vvjRw5kkGDBtG8eXNatmzJ5MmTyczMZMiQIQAMHDiQypUrM2HCBDw8PKhfv77d/gEBAQC25RkZGYwbN467776b0NBQ9u3bx/PPP09ERARdunQp0msrqdpHBGIywe74U5xIzybEz8PRJYmIiIhIEft1y1Genr0FiwF3N63Cf+9piLNTIXWny82C/3sKts22Pm/YB3pMBlfPwjm+iIiUGA4Npe6//34SExMZM2YM8fHxNG7cmAULFtgGP4+Li8PJ6cp/jXF2dmbbtm188cUXpKamUqlSJW677TZee+01dc+7QuW83WhY2Z+tR9JYEZPIvc3L3thaIiIiImXZTxuP8NyPW7EYcH/zMCbc1QCnwgqkTh6C2Q9A/Dbr+FFd3oRW/9H4USIiZZTJMAzD0UUUN+np6fj7+5OWloafn5+jyylykxbt4YM/9nJHw4p82K+po8sRERFxmLJ+T3C19HqVfN9vOMwLP2/DMKBfq6q83rN+4QVS+5fBD0PgdAp4BcK9M6FG+8I5toiIFCtXek/gsDGlpPg6O67Uqr1JmC3KLEVERETKglnr4nj+J2sgNbB1Nd7oVUiBlGHAmg/hq97WQKpiY+v4UQqkRETKPIVSUkDjsAB8PVxIzcpj+9E0R5cjIiIiIjfYV2sP8tIv2wEY0rY64+6sh6kwutTlZsFPw2DRy2BYoFE/eHABBGiICBERUSglF+Di7ES7iEAAVmgWPhEREZFSbcbqA7zy6w4AhrevwZg76hZOIHXyIHx+G/zzIzi5QLeJ0OtjDWguIiI2CqXkgs524VuuUEpERESk1Pps5X7G/bYTgIc7hPNS9zqFE0jt+wP+1xFObAfvIBg4F1o9pAHNRUTEjkNn35Pi62woteVwKmmn8/D3dHVwRSIiIiJSmKYt38db83cD8PjNEYy8Ner6AynDgDUfwJKx1u56lZrC/V+Df+XrL1hEREodtZSSC6oc4El4kDdmi8GavUmOLkdERERECtFHf+61BVJPdY7kmdtqXX8glZsJPz4Ii8dYA6nGD8CQ+QqkRETkohRKyUV1iAoG1IVPRETkelWvXp3x48cTFxfn6FJEeH9JLBMX7gHgmVujeKpz1LUdyGKG5H2wex6sfBc+6ww7fraOH3X7u9DzQ3D1KMTKRUSktFH3Pbmo6KhApq8+wIqYRAzDKJzxBURERMqgp556ipkzZzJ+/Hg6derE0KFD6d27N+7u7o4uTcoQwzB4b3EMH/yxF4AXutbmkY7hl9/RnA8nD0DibusjYTck7oGkGDDn2G/rHQz3fQnVWt+AKxARkdLGZBiG4egiipv09HT8/f1JS0vDz8/P0eU4zOlcM43GLyI338KSkdFEBPs6uiQREZEiVdj3BJs2bWLmzJl8++23mM1m+vXrx4MPPkjTpk0LoVrH0z1U8WUYBhMX7uHjZfsAeLl7HYZH17TfyJwHKfvPhE97IGGX9b/JsWDOvfCBXTwgMAqCakNwHWjcD3xDb/DViIhIcXel9wRqKSUX5enmTKsa5VkZm8TymCSFUiIiItepadOmNG3alHfffZePP/6YF154galTp9KgQQOeeOIJhgwZopbJUugMw+Ct+bv5ZMV+AMZ2j2Rw7RzY8ct54dNesORd+CCuXueET7Wt/w2qBQHVwMm5CK9GRERKE4VSckkdooLOhFKJDG1Xw9HliIiIlGh5eXn88ssvzJgxg8WLF3PTTTcxdOhQjhw5wksvvcSSJUuYNWuWo8uU0iI/ByMplp8W/oFX7GY+dj1CG78kApbFwR/5F97H1dsaNp0fPvlXBScNRysiIoVLoZRcUnRUEMzbxbr9yWTnmfFw1S9hIiIiV2vTpk3MmDGDb7/9FicnJwYOHMh7771H7dq1bdv07t2bFi1aOLBKKbHysq1d7BL3nBnzydryyUjZj8kwcw/8e9efeea/br4XDp/8qih8EhGRIqNQSi4pMtiHUD8P4tOzWX8gxRpSiYiIyFVp0aIFt956K1OnTqVXr164uroW2KZGjRr06dPHAdVJiZF32jq4uC18OjPw+MkDYFgKbG4C0g0vYo3KlKvWgJp1m/8bRPlVBnUVFRERB1MoJZdkMpnoEBXE7L8PszwmUaGUiIjINdi/fz/VqlW75Dbe3t7MmDGjiCqSYi0389/w6ex4T4m74eRB4CJzFHn4Q1AdCK6NJbAW0/e487/d7iSaAvjv3Y24t3lYUV6BiIjIFVEoVdTMeXDiH6jUxNGVXLHoM6HUiphER5ciIiJSIiUkJBAfH0+rVq3slq9btw5nZ2eaN29+1cf86KOPmDhxIvHx8TRq1IgpU6bQsmXLi24/efJkpk6dSlxcHIGBgdxzzz1MmDABDw8P2zZHjx7lhRdeYP78+WRlZREREcGMGTOuqT65AjkZkLSnYPiUGsdFwyfPcrbwydblLqgO+ASDyYTZYvDCT9v4cfcRnEww6b5G9G5SpUgvS0RE5EoplCpqK96BFf+FNk9Ap5fAxd3RFV1Wu4hAnEwQm5DBsdTTVArwdHRJIiIiJcpjjz3G888/XyCUOnr0KG+//Tbr1q27quPNnj2bkSNHMm3aNFq1asXkyZPp0qULe/bsITg4uMD2s2bN4sUXX2T69Om0adOGmJgYBg8ejMlkYtKkSQCcPHmStm3b0qlTJ+bPn09QUBCxsbGUK1fu2i9cLuz0Sfj9edj+AxcNn7wCC473FFQHvAMv2u3ObDF47oet/Lz5KM5OJt67vzF3Nqp0465DRETkOimUKkqGAelHrX3+V0+G2MXQexpUbOjoyi7J38uVxmEBbIpLZWVsIve3qOrokkREREqUnTt30rRp0wLLmzRpws6dO6/6eJMmTWL48OEMGTIEgGnTpjFv3jymT5/Oiy++WGD7NWvW0LZtW/r16wdA9erV6du3r10Y9vbbbxMWFmbXhbBGDc28W+gOrIRfHob0I9bn3sHWwCm4zr/jPQXVtoZPVyHfbGHk91uZu/UYzk4mPujThNsbVrwBFyAiIlJ4NLVGUTKZoOeHcP831l+/EnbApzfDiolgvsi0vMXE2bGklqsLn4iIyFVzd3fnxIkTBZYfP34cF5er+40wNzeXjRs30rlzZ9syJycnOnfuzNq1ay+4T5s2bdi4cSPr168HrGNc/f7773Tv3t22zdy5c2nevDn33nsvwcHBNGnShE8//fSqapNLyM+FxWPgix7WQKp8TRi6BJ6LhcH/B90nQothUL3dVQdSeWYLT363hblbj+HiZOKjfgqkRESkZFAo5Qh17oBH/4Lad4AlD/54HabfBkmxjq7sos6GUqtik8g3F5zdRURERC7utttuY9SoUaSlpdmWpaam8tJLL3Hrrbde1bGSkpIwm82EhITYLQ8JCSE+Pv6C+/Tr14/x48fTrl07XF1dCQ8Pp2PHjrz00ku2bfbv38/UqVOJjIxk4cKFPPLIIzzxxBN88cUXF60lJyeH9PR0u4dcQOIe+OwWWP0+YECTAfCflRDW4roPnZtv4fFZm5m3/TiuziamPtCMrvUVSImISMmgUMpRfILg/q+h9//A3R+OboRp7eCvqWApfqFPoyoB+Hu6kp6dz9YjaZffQURERGzeeecdDh8+TLVq1ejUqROdOnWiRo0axMfH8+67797w8y9btow333yTjz/+mE2bNvHzzz8zb948XnvtNds2FouFpk2b8uabb9KkSRMeeughhg8fzrRp0y563AkTJuDv7297hIVphjc7hgHrP4VPOkD8Nusg5fd/bW057+5z3YfPzbfw2KxNLNgRj5uzE58MaMatdUMuv6OIiEgxoVDKkUwmaHQ/PLoWanaC/GxY8CJ8eeeZWVeKD2cnE+0irU3JNQufiIjI1alcuTLbtm3jv//9L3Xr1qVZs2a8//77bN++/aqDnMDAQJydnQt0Bzxx4gShoaEX3OeVV15hwIABDBs2jAYNGtC7d2/efPNNJkyYgOXMj2EVK1akbt26dvvVqVOHuLiL35Ocbf119nH48OGrupZSLSMBZt0Pvz8L+aet93qPrIU6PQrl8Dn5Zh75eiOLd57AzcWJ/w1sxs21FUiJiEjJooHOiwP/yjDgF/j7c1j0ChxcCR+3ga5vWpt3X2SGlaLWITKIeduOszwmkadvjXJ0OSIiIiWKt7c3Dz300HUfx83NjWbNmrF06VJ69eoFWFs5LV26lBEjRlxwn6ysLJyc7H+LdHZ2BsAwrLO/tW3blj179thtExMTQ7Vq1S5ai7u7O+7uxX8m4SK3ZwH8+hhkJYGzO3QeC60eBqfC+T04O8/Mf77ayPKYRNxdnPhsUHPaRwYVyrFFRESKkkKp4sJksg5uWbMTzHkUDv8Fcx+H3fOgxwfg6/hfvtpHWVtKbTuSSmpWLgFebg6uSEREpGTZuXMncXFx5Obm2i2/8847r+o4I0eOZNCgQTRv3pyWLVsyefJkMjMzbbPxDRw4kMqVKzNhwgQAevTowaRJk2jSpAmtWrVi7969vPLKK/To0cMWTj399NO0adOGN998k/vuu4/169fzv//9j//973+FcOVlRG4WLBpt/aERILge3P0phNQrtFNk55kZ/uXfrIxNwtPVmc8HNadNxNUNjC4iIlJcKJQqbiqEw5DfYe2H1gHQYxbAx63g9klQ/y6HllbR35NaIb7sOXGKVXuTuKNhJYfWIyIiUlLs37+f3r17s337dkwmk611kulMa2iz2XxVx7v//vtJTExkzJgxxMfH07hxYxYsWGAb/DwuLs6uZdTo0aMxmUyMHj2ao0ePEhQURI8ePXjjjTds27Ro0YJffvmFUaNGMX78eGrUqMHkyZPp37//9V5+2XBsC/w8HJJirM9vegxuGQOuHoV2iqzcfIZ98Tdr9iXj5ebM9MEtuKlmhUI7voiISFEzGWfviq7C4cOHMZlMVKlSBYD169cza9Ys6tatWyjN0h0tPT0df39/0tLS8PPzc1whJ3bCL/+xDowJUP9u6P4OeJV3WElvzNvJpysPcG+zKky8t5HD6hARESkKhXVPcLZF0meffUaNGjVYv349ycnJPPPMM7zzzju0b9++EKt2nGJzD1WULGZY8wH88YZ1VmWfUOg9FcJvLtTTZObk8+DMDaw7kIK3mzMzH2xJi+qOuycUERG5lCu9J7imju39+vXjzz//BCA+Pp5bb72V9evX8/LLLzN+/Phrq1gKCqkLw5ZC9PNgcoZ/foKPW0PMIoeVFB1lHa9gRWwi15BnioiIlElr165l/PjxBAYG4uTkhJOTE+3atWPChAk88cQTji5PrlXqYfjiTlgy1hpI1elhncCmkAOpjJx8Bs9Yz7oDKfi6u/Dl0FYKpEREpFS4plDqn3/+oWXLlgB8//331K9fnzVr1vDNN98wc+bMwqxPXNzg5pdh6GIIjIKMeJh1L8x9AnJOFXk5LaqXx8PViRPpOcScyCjy84uIiJREZrMZX19fwDp73rFjxwCoVq1agcHFpYTY/iNMbQuHVoGrN9z5Idz3VaG3aE/PzmPg5+vYcPAkvh4ufDWsFc2qlSvUc4iIiDjKNYVSeXl5tplWlixZYhucs3bt2hw/frzwqpN/VWkG/1kBNz1qfb7pC5jaBg6uKtIyPFydbWMXrIhJLNJzi4iIlFT169dn69atALRq1Yr//ve/rF69mvHjx1OzZk0HVydXJTsNfhoOPw2FnDSo3BweXglNC3/G5LTTeQz4fD2b4lLx93Rl1rCbaBwWUKjnEBERcaRrCqXq1avHtGnTWLlyJYsXL6Zr164AHDt2jAoVNNjiDePqCV0nwKD/g4CqkBoHM++ABS9B3ukiKyP6zJTDyxVKiYiIXJHRo0djsVgAGD9+PAcOHKB9+/b8/vvvfPDBBw6uTq7YoTUwtR1s/x5MTtDhBXhwgXWimkKWmpXLA5+tY+vhVMp5uTJreCsaVPEv9POIiIg40jXNvvf222/Tu3dvJk6cyKBBg2jUyDrg9dy5c23d+uQGqtEeHlkDC1+CTV/CXx/B3iXQexpUbnrDT392XKn1B1M4nWvG0835hp9TRESkJOvSpYvt/yMiIti9ezcpKSmUK1fONgOfFGPmPFg2AVa9B4YFAqrBXZ9C1VY35HQnM3Pp/9k6dh5Pp7y3G98Ma0WdimVk4HgRESlTrimU6tixI0lJSaSnp1Ou3L992h966CG8vLwKrTi5BHdfuHMK1L4D5j4OSXvgs84Q/SxEPwfOrjfs1OFB3lQO8ORo6mn+OpBMp1rBN+xcIiIiJV1eXh6enp5s2bKF+vXr25aXL6+BqkuEpL3w83A4tsn6vFE/6PY2eNyYkCg5I4f+n61jd/wpAn3c+GbYTdQK9b0h5xIREXG0a+q+d/r0aXJycmyB1KFDh5g8eTJ79uwhOFgBRZGK6gKP/gX17gLDDMvfhk9vhhM7b9gpTSaTrbXU8j3qwiciInIprq6uVK1aFbPZ7OhS5GoYBmycCZ+0twZSHgFw70zoPfWGBVKJp3Lo++lf7I4/RZCvO989pEBKRERKt2sKpXr27MmXX34JQGpqKq1ateLdd9+lV69eTJ06tVALlCvgVR7unQH3TAfPchC/Df7XAVa/D5YbcwPcISoQgBWxCqVEREQu5+WXX+all14iJSXF0aXIlchMhu/6w29PQl4W1Ii2Dp1Qr/cNO2VCejZ9P/2LmBMZhPhZA6mIYAVSIiJSul1TKLVp0ybat28PwI8//khISAiHDh3iyy+/1GCdjlT/bmurqcguYM6FxWNg5u2Qsr/QT9UmIhBnJxP7EzM5cjKr0I8vIiJSmnz44YesWLGCSpUqUatWLZo2bWr3kGIkdglMbQ175oGTK9z6Ggz4Ffwr37BTxqdl0+d/f7E3IYOK/h7Mfqg14UE+N+x8IiIixcU1jSmVlZWFr6/1l5tFixZx11134eTkxE033cShQ4cKtUC5Sr6h0G82bP4KFoyCuLXWWWJuew2aP1hoUxX7ebjStGoAGw6eZEVMEv1aVS2U44qIiJRGvXr1cnQJcjl5p2Hxq7D+E+vzwFpw92dQseENPe3xtNP0/d9fHEzOonKAJ98Ov4mqFTRGq4iIlA3XFEpFREQwZ84cevfuzcKFC3n66acBSEhIwM9PM4M4nMkETQdCjQ4w51E4tArmjYTd86Dnh+BXqVBOEx0ZxIaDJ1kek6BQSkRE5BJeffVVR5cglxK/HX4aDom7rM9bPgS3jgdXzxt62iMns+j36TriUrKoUs4aSIWVVyAlIiJlxzV13xszZgzPPvss1atXp2XLlrRu3Rqwtppq0qRJoRYo16FcNRj0G3SZAC4esG8pfHwTbJ1tHbzzOp0d7HzN3mTyzJbrPp6IiIhIkbJYYM0U6yQxibvAOxj6/wjdJ97wQOpwShZ9/vcXcSlZVC3vxez/tFYgJSIiZc41tZS65557aNeuHcePH6dRo0a25bfccgu9e9+4ASDlGjg5QetHIaIz/PIf6+wxvzwEu3+DOyaDd+A1H7pBZX/Ke7uRkpnLlsOptKiuqa1FREQuxMnJCdMlutBrZj4HSD8GvzwMB5Zbn0d1gzungE9QkZz+1bk7OHLyNDUCvZk1vBUV/W9sCCYiIlIcXVMoBRAaGkpoaChHjhwBoEqVKrRs2bLQCpNCFhQFQxfDqvdg+Vuw6zeI+wt6vA+1b7+mQzo5mWgXEcjcrcdYtidBoZSIiMhF/PLLL3bP8/Ly2Lx5M1988QXjxo1zUFVl2I451pn1slPBxRO6vgnNhhTa2JuXk5Wbz6rYJACmPtBUgZSIiJRZ1xRKWSwWXn/9dd59910yMjIA8PX15ZlnnuHll1/GyemaegXKjebsAh2eg6jbrL8MJuyE7/pBo37Q7S3w8L/qQ95cO5i5W4/x6coDNK9enk61gm9A4SIiIiVbz549Cyy75557qFevHrNnz2bo0KEOqKoMyjkF81+ALd9Yn1dsbB3MPDCySMtYszeZXLOFKuU8qRXiW6TnFhERKU6uKT16+eWX+fDDD3nrrbfYvHnz/7d353FV1un/x1/nHHYEZBcQBTdwQ0yUXFJMUlsctc3KSbPFFmtqnOXbptZUMjUzjr82rUbbS6embLeFRDPXVNzBXdxAkB0UhHN+fxxEKSwV5D4H3s/H4350zn3f5z7Xfc6MXl7n87k+bNiwgZkzZ/LCCy8wbdq0xo5RGltYL5icBgMfBEyw8T14eQDsSTvvS10dF8bwbqFUVlm5+611fLctp7GjFRERabYuvfRSUlNTjQ6jZTiwBuYOqilImWDQVPso8iYuSAEsyTwKQFJM8K9O6xQREWnuLqgo9eabb/Kf//yHe++9l7i4OOLi4rjvvvt47bXXeOONNxo5RLkoXNztq8rcvhj8o6H4ILw1Gr74M1SWnfNlXC1mXhp/CVf3DKOy2so976xj8ZYjFzFwERGR5uH48eM8//zzREREGB1K81ZdBUtSYP5IKNgHfpFw2xeQPANc3Jo8HJvNRlpmLoBGmIuISIt3QdP38vPziY2N/cX+2NhY8vPzGxyUNKF2l8I9y+G7GbD2P7D2NfsqfWNfgchz6xHmajHz/26Kx2I28enGw0x5bwP/7yYb18SFX+TgRUREnIO/v3+dETE2m42SkhK8vLx45513DIysmcvfAx9NhoNr7c973gBX/RM8WxsW0q6jpRwqPI6bi5n+HQMNi0NERMQRXFBRqlevXrz44os8//zzdfa/+OKLxMXFNUpg0oTcW8HV/4KYq+CT++0J3PwR9ul9SY/YR1X9BheLmX+Pi8fFbOKjDYf4w/sbqKq2Maa3fv0VERH597//XacoZTabCQ4OJjExEX9/fwMja6ZsNvs0va/+DypLwd0Xrp4FcTcYHVnt1L3E6AC83C54zSEREZFm4YL+Jnzuuee4+uqr+e677+jfvz8AK1eu5MCBA3z55ZeNGqA0oU7D4L6V9gRu0wL7Sn07v4Wxc6FNz998ucVs4h839MLFYuK/Px3kj/9Np8pq4/o+bZsgeBEREcd12223GR1Cy1Geb19Zb/un9uftB9pzmdbtjI2rhqbuiYiInHZBPaWGDBnCjh07GDt2LIWFhRQWFnLttdeydetW3n777caOUZqSZ2u49hUY9w54BUHOFnh1KCz7p70nw2+wmE38/do4bklsh80Gf/lwIwvWZF38uEVERBzY66+/zgcffPCL/R988AFvvvmmARE1U7uXwJwB9oKU2QWGzYCJnzlMQarkxEnW7rO3uhgaq6KUiIjIBRWlAMLDw3nmmWf43//+x//+9z+efvppCgoKmDdv3nld56WXXiIqKgoPDw8SExNZs2bNOb1uwYIFmEwmxowZU2e/zWZj+vTphIWF4enpSXJyMjt37jyvmAToOgruWwWx14D1JHz/lH1KX95vf5Zms4lnxvRgYv/22Gzw8EebeXvV/iYIWkRExDGlpKQQFBT0i/0hISHMnDnTgIiamZMn4OvH4O0xUHIEAjvDnd/BZVPBbDE6ulo/7jrGyWob7QO9iA7yNjocERERw11wUaoxLFy4kKlTpzJjxgzWr19Pr169GDFiBEePHv3V1+3bt48///nPXHbZZb849txzz/H8888zd+5cVq9ejbe3NyNGjODEiRMX6zaar1bB9hFTY18Bdz849BPMvQxWzQWr9VdfajKZeOJ33bljUDQA0xZt4Y0f9zZF1CIiIg4nKyuL6OjoX+xv3749WVkaUdwgOdvgtcth5Yv25wm3w91LIby3sXHVY+kOe46rqXsiIiJ2hhalZs2axV133cWkSZPo1q0bc+fOxcvLi/nz55/1NdXV1YwfP54nn3ySDh061Dlms9mYPXs2jz/+OKNHjyYuLo633nqLw4cPs2jRoot8N82UyQS9boL7VkCHoVB1HBb/H7w9Ggp/PYk2mUw8fnVX7h5i/56e+Gwb//lhT1NELSIi4lBCQkLYtGnTL/Zv3LiRwECtwHZBrFZYNQdeTYKjW+1tB25eANf8G9wcbxSSzWZjSYa9n1RSTLDB0YiIiDgGw4pSlZWVrFu3juTk5NPBmM0kJyezcuXKs77ub3/7GyEhIdxxxx2/OLZ3716ys7PrXNPPz4/ExMRfvaacA7+2cOvH9mWUXb1g7zJ4eQBseMe+ws1ZmEwmHh4ZywOXdwLg6S+283LarqaKWkRExCHcfPPN/OEPf2DJkiVUV1dTXV3N999/z4MPPshNN91kdHjOpyQb3r0eFj8M1RXQ6Qq4dwXEXGl0ZGeVmVNCdvEJ3F3MXNpBhUgRERE4z9X3rr322l89XlhYeM7XysvLo7q6mtDQ0Dr7Q0NDycjIqPc1y5cvZ968eaSnp9d7PDs7u/YaP7/mqWP1qaiooKKiovZ5cXHxudxCy2MyQb+7oOPlsOheOLAaPpkC2z+HK58F//ZneZmJPw2PwcVs5t/f7eC5xZlUVdv4w7DOTXwDIiIixnjqqafYt28fw4YNw8XFnn5ZrVYmTJignlLna/vn8OkDcDwfXDxg+NPQ9057nuLATo2SGtAxEA9Xx+lzJSIiYqTzKkr5+fn95vEJEyY0KKCzKSkp4dZbb+W1116rt1FoQ6SkpPDkk0826jWbtcCOMOkrWPE8LJkJO76CHYshejD0/r29Sbqr5y9e9mByZ1wsJv7xdSazvt1BVbWVP17RBZODJ5EiIiIN5ebmxsKFC3n66adJT0/H09OTnj170r59/T/oSD0qy2DxI7C+ZrXCNj3h2v9ASKyxcZ2jJZk1/aS06p6IiEit8ypKvf766432xkFBQVgsFnJycursz8nJoU2bNr84f/fu3ezbt49Ro0bV7rPWNNt2cXEhMzOz9nU5OTmEhYXVuWZ8fPxZY3nkkUeYOnVq7fPi4mIiIyMv6L5aDLMFBv0ROg+Hrx+FPWmwd6l9c/eFHtdC/O+hbUKdXy6nDO2Eq8XEzC8zeP77XZy02vjriBgVpkREpEXo3LkznTtrpPB5O7Yb3r0B8ncDJhjwAFz+OLi4Gx3ZOSk+cZJ1+wsASOqiopSIiMgphvWUcnNzo0+fPqSmptbus1qtpKam0r9//1+cHxsby+bNm0lPT6/dfve73zF06FDS09OJjIwkOjqaNm3a1LlmcXExq1evrveap7i7u+Pr61tnk3MU2h0mfAIPboKkR6B1O6gohnVvwLxkeCkRls+2936oMXlwR6Zf0w2AOWm7eeaL7dh+pS+ViIiIs7vuuut49tlnf7H/ueee44YbbjAgIifjE2b/Qcw3AiZ+CsOfcpqCFMDynXlUW210CPamXaCX0eGIiIg4jPMaKdXYpk6dysSJE0lISKBfv37Mnj2bsrIyJk2aBMCECROIiIggJSUFDw8PevToUef1rVu3Bqiz/6GHHuLpp5+mc+fOREdHM23aNMLDwxkzZkxT3VbL5N8ekh6GwX+F/cthw7uw7RPIy4TvZkDq36BTMvQeD12u5PZB0bhaTEz7ZCv/Wb6XKquNGaO6acSUiIg0S8uWLeOJJ574xf4rr7ySf/3rX00fkLNx84Kb3gfvQPD0Nzqa87Yko2bqXoxGSYmIiJzJ0KLUuHHjyM3NZfr06WRnZxMfH8/ixYtrG5VnZWVhNp/fYK6//vWvlJWVMXnyZAoLCxk0aBCLFy/Gw8PjYtyC/JzZbO8tFT0YrvoHbP0Y0t+1N0Xf+bV98wyAuBu5NX48Ltf25NGPN/PGin2crLby1OgemM0qTImISPNSWlqKm5vbL/a7urpqgZVzFdTJ6AguiM1mI22Hvcl5UkywwdGIiIg4FpNN86Z+obi4GD8/P4qKijSVr7Hk7bQXpzYugJIjp/e36cmGwKu5Y30U+TZfxiVEknJtTxWmRETEITRWTtCvXz+uueYapk+fXmf/E088wWeffca6desaGqpDUA71S1sOFXHNC8vxdLWQPuMK3F208p6IiDR/55oTGDpSSlqQoM6Q/AQMfRz2LIEN70Dml5C9md7Zm/nJw4Wvqy7hv+uH8NfqSp69oQ8WFaZERKSZmDZtGtdeey27d+/m8ssvByA1NZX33nuPDz/80ODo5GJKq1l1b2CnIBWkREREfkZFKWlaFhfofIV9K8+HzR9C+juYj2zkSssarrSsIWfbayx58UqSxj2ES6hzLPMsIiLya0aNGsWiRYuYOXMmH374IZ6envTq1Yvvv/+egIAAo8OTiygtU1P3REREzkbT9+qhoecGyN4C6e9Suf593CoLandbI/pi7j0eelwLHn4GBigiIi3RxcoJiouLef/995k3bx7r1q2jurq60a5tJOVQdRWWV3LJU99itcHy/xtKW3+tvCciIi3DueYE59dFXORiadMDRqbg9tcdpA94kVTrJVTZzJgPrYXPH4J/xsD/7oI9aWC1Gh2tiIjIBVm2bBkTJ04kPDycf/3rX1x++eWsWrXK6LDkIlm2Mw+rDbqEtlJBSkREpB6avieOxcWN+OG3sqTdCAa/k8rVtmXc5vUjESf3w+b/2je/dhB/M8TfAv5RRkcsIiLyq7Kzs3njjTeYN28excXF3HjjjVRUVLBo0SK6detmdHhyEZ3qJ5UUE2JwJCIiIo5JI6XEIQ2NDeHvE5J5y/Q7BpbM5MnQF6i6ZBK4+0FRFix9Fv5fL3jjGkh/HyrLjA5ZRETkF0aNGkVMTAybNm1i9uzZHD58mBdeeMHosKQJWK02lqqflIiIyK9SUUoc1uAuwbx+W188XV14fX8gtx29meN/2AbXzYMOQwET7PsBFt1jn973yf2QtQrUJk1ERBzEV199xR133MGTTz7J1VdfjcWi1ddaii2HizhWVom3m4WE9mpmLyIiUh8VpcShDegUxBuT+uLlZmH5rjwmvbuZsi5jYMIieGgzDH0c/KOhsgQ2vA3zR8CLCfDDv6D4sNHhi4hIC7d8+XJKSkro06cPiYmJvPjii+Tl5RkdljSBJRn2UVKDOgfh5qKUW0REpD76G1IcXmKHQN6+ox+t3F1YtSef215fQ2lFFbSOhCF/gT9sgNu+hPjx4OoNx3ZB6t/g393hnetgy0dw8oTRtyEiIi3QpZdeymuvvcaRI0e4++67WbBgAeHh4VitVr799ltKSkqMDlEukiU1/aSGqp+UiIjIWZlsNs11+jktZ+yYNmQVMGH+GkpOVHFJu9a8cXs/fD1c655UUQrbFsGGdyFrxen9Hq2h5w3QezyExYPJ1ISRi4iIs7oYOUFmZibz5s3j7bffprCwkCuuuIJPP/20Ua5tNOVQdvlllfR5+ltsNlj5yOWE+XkaHZKIiEiTOtecQCOlxGn0bufPe3deip+nK+uzCrn1P6spKj9Z9yT3VtD793D7V/DAerjsz+AbAScKYe1r8GoSzBkIK1+C0lwjbkNERFq4mJgYnnvuOQ4ePMj7779vdDhyESzbkYvNBrFtfFSQEhER+RUqSolT6dnWj/fuSsTfy5WNB4sYP28VBWWV9Z8c2BGGTbP3nvr9R9DjOrC4w9Gt8PWjMCsWFoyHjC+h+mT91xAREblILBYLY8aMaTajpOS02ql7sZq6JyIi8mtUlBKn0z3cj/cnX0qgtxtbDhVz82urOFZacfYXmC3QaRhcPx/+nAlX/wvCLwFrFWR8Dgtuhlld4evH4Oj2prsRERERaXaqrTaW7bCPxk7qEmxwNCIiIo5NRSlxSrFtfFkw+VKCfdzJyC7h5tdWkVvyK4WpUzz9oe+dMHkJ3LcK+t8P3sFQlgsrX4SXL4VXh8LGhaB2ayIiInKeNh4spKD8JD4eLlzS3t/ocERERByailLitDqH+rBg8qWE+rqzI6eUm15dydHi81hlL6QrjHgGpm6Hm96H2GvA7AKH18PHk+G/E+B4wcW7AREREWl20jLsU/cGdw7G1aJUW0RE5Nfob0pxah2DW7Fwcn/C/TzYnVvGuFdXcaTo+PldxOIKsVfBTe/CnzJh6GNgdoXtn8KcQbB/xW9fQ0RERARIq5m6NyRGU/dERER+i4pS4vSigrxZeHd/2vp7sjevjHGvrOJgQfmFXcw7CIb8Fe78FgI6QvFBeONqWDITqqsaN3ARERFpVnJLKth0sAhQPykREZFzoaKUNAuRAV4svLs/7QK8yMovZ9wrqziQf4GFKYDw3nD3MogfDzYrLH0W3rgKCvY3XtAiIiLSrCytGSXVI8KXEF8Pg6MRERFxfCpKSbMR0dqT/97dn+ggbw4VHmfcKyvZl1d24Rd0bwVjXobr5oG7LxxYDXMvgy0fNV7QIiIi0mykZdr7SSV1CTE4EhEREeegopQ0K238PFg4+VI6BntzuOgE415dye7c0oZdtOf1cM8P0LYfVBTBh5PgkylQ2YCCl4iIiDQrVdVWltWMlBoaq6l7IiIi50JFKWl2Qnw9WDC5PzGhPuQUV3DTq6vYmVPSsIv6R8Gkr2DwXwATbHgHXhkMh9MbIWIRERFxdukHCik+UUVrL1fiI/2NDkdERMQpqCglzVKwjzvv3ZVI1zBfckvshamM7OKGXdTiApc/Drd9Dr4RcGwX/CcZVrwIVmvjBC4iIiJOaUnN1L3LOgdjMZsMjkZERMQ5qCglzVZgK3fevyuRHhG+HCur5OZXV7H1cFHDLxw1CO5ZDrHXgPUkfPMYvHs9lB5t+LVFRETEKS3JqJm6F6OpeyIiIudKRSlp1lp7ufHunZfSK7I1BeUnueW11Ww+2AiFKa8AGPcOXPNvcPGE3akwZwDs/K7h1xYRERGnklN8gm1HijGZYHAXFaVERETOlYpS0uz5ebry9h39uKRda4qOn+SW/6xiQ1ZBwy9sMkHC7TA5DUK6Q1kuvHsdLH4Eqioafn0RERFxCksz7aOk4iL8CGrlbnA0IiIizkNFKWkRfD1ceeuORPpFBVByoopb563hp335jXPxkFi463vod7f9+aqX4T/DIHdH41xfREREHNqpflJJMSEGRyIiIuJcVJSSFqOVuwtv3N6X/h0CKa2oYsL8Nazec6xxLu7qAVc9BzcvBK9AyN4Mrw6B9W+BzdY47yEiIiIO52S1leU78wAYGquilIiIyPlQUUpaFC83F+bf1pfLOgdRXlnNxNfXsGJXXuO9QcxIuOdH6JAEJ8vh0wfgg9vgeCNMFxQRERGHs25/ASUVVQR4uxEX4Wd0OCIiIk5FRSlpcTzdLLw2IYGkmGBOnLQy6Y21LNuR23hv4BsGv/8Ykp8EswtsWwRzL4P9KxvvPURERMQhnJq6N6RLMGazyeBoREREnIuKUtIiebhaeOXWPiR3DaGiysqdb/7EkoyjjfcGZjMMegju+Ab8o6HoALxxFaT9HaqrGu99RESkxXrppZeIiorCw8ODxMRE1qxZ86vnz549m5iYGDw9PYmMjOSPf/wjJ06cqPfcv//975hMJh566KGLEHnzcqrJeVKMVt0TERE5XypKSYvl7mLh5fF9GNm9DZXVVia//RPfbM1u3DeJ6AP3/AC9bgabFdJS4M1roPBA476PiIi0KAsXLmTq1KnMmDGD9evX06tXL0aMGMHRo/X/wPLee+/x8MMPM2PGDLZv3868efNYuHAhjz766C/OXbt2La+88gpxcXEX+zac3uHC42Rkl2AyweDOKkqJiIicLxWlpEVzczHzwi29uToujJPVNu57dz1fbT7SuG/i7gNj58K1/wE3H8haCXMHwtZFjfs+IiLSYsyaNYu77rqLSZMm0a1bN+bOnYuXlxfz58+v9/wVK1YwcOBAbrnlFqKiohg+fDg333zzL0ZXlZaWMn78eF577TX8/f2b4lacWlrNKKneka3x93YzOBoRERHno6KUtHiuFjP/b1w8o+PDqbLauP/9DSzacKjx3yjuBvuoqYgEOFEEH0yET+6HyrLGfy8REWm2KisrWbduHcnJybX7zGYzycnJrFxZf//CAQMGsG7dutoi1J49e/jyyy+56qqr6pw3ZcoUrr766jrX/jUVFRUUFxfX2VqStJp+UkkxWnVPRETkQqgoJQK4WMzMujGe6y5pS7XVxkML05n81k8cLChv3DcKiIbbF8NlfwJMsOFteGUIHNnYuO8jIiLNVl5eHtXV1YSGhtbZHxoaSnZ2/dPQb7nlFv72t78xaNAgXF1d6dixI0lJSXWm7y1YsID169eTkpJyzrGkpKTg5+dXu0VGRl7YTTmhyiorP9as4DtURSkREZELoqKUSA2L2cQ/ro/jvqSOuJhNfLMth+RZS3lpyS4qqqob8Y1cYdh0mPgp+ITBsZ3wn2RY+TLYbI33PiIiIjXS0tKYOXMmL7/8MuvXr+ejjz7iiy++4KmnngLgwIEDPPjgg7z77rt4eHic83UfeeQRioqKarcDB1pOz8Sf9uVTVllNUCt3uof7Gh2OiIiIU1JRSuQMZrOJv46M5csHLyMxOoATJ6384+tMrpz9Az/szG3cN4seDPeugJiroboSvn4E3r0BShtxFUAREWl2goKCsFgs5OTk1Nmfk5NDmzZt6n3NtGnTuPXWW7nzzjvp2bMnY8eOZebMmaSkpGC1Wlm3bh1Hjx7lkksuwcXFBRcXF5YuXcrzzz+Pi4sL1dX1/zjj7u6Or69vna2lWFIzdW9Il2DMZpPB0YiIiDgnFaVE6tEl1IcFky/l/90UT7CPO3vyyrh13hqmvLueI0XHG++NvALgpnfh6n+Biwfs+hbmDIRd3zXee4iISLPi5uZGnz59SE1Nrd1ntVpJTU2lf//+9b6mvLwcs7lu2mexWACw2WwMGzaMzZs3k56eXrslJCQwfvx40tPTa8+V05bUNDkfGqtV90RERC6Ui9EBiDgqk8nE6PgIhsaGMPvbnby5ch9fbD7CksyjPDisM5MGRuPm0gh1XZMJ+t4J7QbA/+6Ao9vgneug//32aX4u7g1/DxERaVamTp3KxIkTSUhIoF+/fsyePZuysjImTZoEwIQJE4iIiKjtDzVq1ChmzZpF7969SUxMZNeuXUybNo1Ro0ZhsVjw8fGhR48edd7D29ubwMDAX+wXOJBfzq6jpVjMJi7rpKKUiIjIhVJRSuQ3+Hq4Mn1UN67v05bpn2zhp/0FpHyVwQfrDvLU6B707xjYOG8U2g3u+h6+mQZrX4OVL8K+H+C6+RDUqXHeQ0REmoVx48aRm5vL9OnTyc7OJj4+nsWLF9c2P8/KyqozMurxxx/HZDLx+OOPc+jQIYKDgxk1ahTPPPOMUbfg1NJ22EdJXdKuNX5ergZHIyIi4rxMNps6K/9ccXExfn5+FBUVtajeCPLbrFYbH204RMqX2zlWVgnA6PhwHruqKyG+594Y9jdlfAmfTIHj+eDqDVc9B/Hj7aOqRESkySgnOD8t5fO64421pGYc5S8jYpgyVD8ciYiI/Ny55gTqKSVyHsxmE9f3acv3f0piQv/2mE3wSfphLv/XUuYt30tVtbVx3ij2Krj3R3sz9JNl9gLVh5PgeGHjXF9EREQuyImT1azYfQyAoTEhBkcjIiLi3FSUErkAfl6u/G10Dz69fxC9IltTWlHFU59v45oXlrN2X37jvIlvONy6CIbNALMLbP0Y5l4GWasb5/oiIiJy3tbszef4yWpCfd3pGuZjdDgiIiJOTUUpkQboEeHHx/cOIOXanrT2ciUju4Qb5q7kT//dSG5JRcPfwGyBy6bC7d+AfxQUZcHrV0Las2Ctf3luERERuXiWZB4FIKlLCCZNqxcREWkQFaVEGshsNnFzv3Ys+VMSN/eLxGSC/60/yOX/SuOtlfuotjZC27a2feDuHyBuHNiqIW0mvHENFB5o+LVFRETknC3NtDc5HxqrVfdEREQaSkUpkUbi7+1GyrVxfHTvAHpE+FJyoorpn2zldy8uZ31WQcPfwMMXrn0Vxr4Kbq0gawXMHQjbPmn4tUVEROQ37csrY09eGS5mEwM7BRkdjoiIiNNTUUqkkfVu588nUwbx1Jge+Hq4sPVwMde+vIKH/7eJ/JoV+xqk1zi45weI6AMniuC/E+DTP0BlecOvLSIiImeVVjN1LyHKHx8PV4OjERERcX4qSolcBBaziVsvbc/3f07i+j5tAViw9gCX/yuN91ZnYW3olL6ADnD71zDoj4AJ1r8Jrw6B7M0ND15ERETqlbajZuqeVt0TERFpFIYXpV566SWioqLw8PAgMTGRNWvWnPXcjz76iISEBFq3bo23tzfx8fG8/fbbdc657bbbMJlMdbaRI0de7NsQqVdQK3f+eUMvPrynP7FtfCgsP8mjH29m7Ms/sulgYcMubnGF5CdgwiJo1QbydsBrl8OqOWBrhD5WIiIiUuvEyWpW7j4GQJKKUiIiIo3C0KLUwoULmTp1KjNmzGD9+vX06tWLESNGcPTo0XrPDwgI4LHHHmPlypVs2rSJSZMmMWnSJL7++us6540cOZIjR47Ubu+//35T3I7IWSVEBfD5A4OYMaobPu4ubDxYxOiXfuSxjzdTWN7AKX0dkuDeFdDlSqiuhMUPw3s3Qmluo8QuIiIisHLPMSqqrIT7edAltJXR4YiIiDQLhhalZs2axV133cWkSZPo1q0bc+fOxcvLi/nz59d7flJSEmPHjqVr16507NiRBx98kLi4OJYvX17nPHd3d9q0aVO7+fv7N8XtiPwqF4uZSQOjSf3TEMb2jsBmg3dXZ3H5v5by358ONGxKn3cg3Pw+XPVPsLjDzm/sTdB3f994NyAiItKCpWXYfzRNig3BZDIZHI2IiEjzYFhRqrKyknXr1pGcnHw6GLOZ5ORkVq5c+Zuvt9lspKamkpmZyeDBg+scS0tLIyQkhJiYGO69916OHTv2q9eqqKiguLi4ziZysYT4evDvcfEsmHwpXUJbkV9WyV8/3MQNr6xk6+GiC7+wyQT97oLJSyC4K5TmwNtj4ZvHoaoRGqyLiIi0UDabjSWZ9hHISV2CDY5GRESk+TCsKJWXl0d1dTWhoaF19oeGhpKdnX3W1xUVFdGqVSvc3Ny4+uqreeGFF7jiiitqj48cOZK33nqL1NRUnn32WZYuXcqVV15JdXX1Wa+ZkpKCn59f7RYZGdnwGxT5DZd2COSLP1zGo1fF4uVmYd3+Aka9sJwnPt1K8YmTF37h0O72wlTCHfbnK16wj5r6+B5IfQp+mg87v4WcbfbV+0RERORX7c0rIyu/HFeLiYGdgowOR0REpNlwMTqA8+Xj40N6ejqlpaWkpqYydepUOnToQFJSEgA33XRT7bk9e/YkLi6Ojh07kpaWxrBhw+q95iOPPMLUqVNrnxcXF6swJU3C1WJm8uCOjOoVztNfbOeLTUd4Y8U+Pt90hMeujmVMfMSFTRFw9YRrZkHHy+HT++1N0PN21H+uuy/4RoBfRM1/I8943Nb+X1ePht2oiIiIEzs1SqpfdADe7k6XPouIiDgsw/5WDQoKwmKxkJOTU2d/Tk4Obdq0OevrzGYznTp1AiA+Pp7t27eTkpJSW5T6uQ4dOhAUFMSuXbvOWpRyd3fH3d39wm5EpBGE+Xny0i2XcHPfPKZ/uoU9uWX8ceFG3l9zgKdG9yCmjc+FXbjrNRCZCHuWQNFB+1Z8CIoOQdEBOFEIFcWQWwy5289+Ha8ge6HKL/J0AcuvLfi2tT9u1QYsStJFRKR5Ssu095MaqlX3REREGpVh/4p0c3OjT58+pKamMmbMGACsViupqancf//953wdq9VKRUXFWY8fPHiQY8eOERYW1tCQRS66QZ2D+OrBy/jPD3t54fudrNmbz1XP/8DtA6N4MLkLrS7k19lWwRB3Y/3HKstOF6hOFauKa4pXRYfs+06WQ3mefTuysf7rmCzgE1Z3hNWpUVanHnsF2vteiYiIOJHyyipW78kHIElFKRERkUZl6NCGqVOnMnHiRBISEujXrx+zZ8+mrKyMSZMmATBhwgQiIiJISUkB7L2fEhIS6NixIxUVFXz55Ze8/fbbzJkzB4DS0lKefPJJrrvuOtq0acPu3bv561//SqdOnRgxYoRh9ylyPtxdLEwZ2okxvSP422db+XprDq/9sJdPNx7m8au7cU1cWOOt+uPmDcFd7Ft9bDY4XnDGCKufj7Y6CCWHwVplL2YVHzz7e7l4gG943RFWZz72jQAP38a5LxERkUayYtcxKquttPX3pGOwt9HhiIiINCuGFqXGjRtHbm4u06dPJzs7m/j4eBYvXlzb/DwrKwuz+XQv9rKyMu677z4OHjyIp6cnsbGxvPPOO4wbNw4Ai8XCpk2bePPNNyksLCQ8PJzhw4fz1FNPaXqeOJ2I1p68cmsCSzKP8sSnW9l/rJwH3t/AgrVZPPm7HnQKaXXxgzCZwCvAvoXF1X+OtRpKj9YUqg6cHmF15uPSHKg6Afl77NvZuPv9bLRVRE3R6ozClYv+v+ywju2GXd/B7u/tz7uPhdhrwL0J/rcqInKRLDlj6l6j/SgkIiIiAJhsNpvN6CAcTXFxMX5+fhQVFeHrq5EbYrwTJ6t5ddkeXlqyi4oqK64WE3de1oEHLu+El5sT9HKqqoDiw3X7WZ052qr44LmvBOgdbO9t1e5S6JAE7QeA+wX23JKGqSyHfcth17f2FR0L9v7yHFcve2Eqbpz9+1LvMXEyygnOT3P7vGw2G4OeXcKhwuPMvy2By2NDf/tFIiIics45gYpS9WhuCZU0H1nHynnys62kZth/tQ3382D6qG6M6N7G+X+9rSg5o6fVoXqmCx60j7b6ObMLRCTYCx4dkqBtAlhcmzr6lsFmg2O77AWoXd/ZC1LVZ/T0M7tAu/7QKdn+XW36L+TvPn3cOxh6XG/vcRbeWz3GxCkoJzg/ze3z2plTwhX/Xoabi5mN04fj6WYxOiQRERGnoKJUAzS3hEqan++25fDEZ1s5WHAcgMFdgnnyd92JDmrGvS5sNijPtxet8nbCvh9gTxoU7Kt7nlsr++ipU0WqkG4qfjREZRns/eH0aKjC/XWP+7aFzsnQ6QroMKTuqDWbDQ6th00LYcuHUH7s9LGgLvbiVM8bwD+qSW5F5EIoJzg/ze3zenXZbmZ+mcHgLsG8dXs/o8MRERFxGipKNUBzS6ikeTpeWc2ctF3MXbqHymorbhYzdw/pwH1JnVrWL7n5e2HvUtiz1P7fMwsfAN4hED34dJGqdaQRUToPmw3ydtSMhvoW9q+A6srTx82u9qJf5yvsI6KCY8+t6Fd90t5ratNCyPii7qi3dv3tBapuY+z9y0QciHKC89PcPq9bXlvFit3HmDGqG5MGRhsdjoiIiNNQUaoBmltCJc3b3rwyZny6lWU7cgFo6+/JE6O6k9ytBfa9sFohZ4t9BNXepfaCysnyuucEdLSP6OmQBFGXqQgCUFEKe5fVjIb6Doqy6h73a3d6NFT04IY3Lj9RDBmfw8YF9vel5q8hsyt0GWHvP9V5OLh6NOx9RBqBcoLz05w+r9KKKnr/7RtOVttY8uek5j0aWUREpJGpKNUAzSmhkpbBZrOxeEs2f/t8G0eK7CNQhsWG8MTvuhMZ4GVwdAaqqoCDa+1Fqj1L4dA6sFWfcYIJwuMhuqZI1e5ScPU0JtamZLNBbsYZo6FWgvXk6eMWN2g/sGY01BUQ1PniTYEsPgybP7T3n8rZfHq/h5995FTcOPtIqjNWYhVpSsoJzk9z+ry+3prN3W+vIyrQi7S/DDU6HBEREaeiolQDNKeESlqW8soqnk/dxX9+2EOV1Ya7i5n7kjpx95AOeLi2oCl9Z3OiCPb9eHokVW5G3eMW95pV/WqKVGHxYG4mn9uJYvs97/rOPhqq+GDd463bny5CRV8GbgaMCMjZai9Obf7A3tz+FL9Ie++puHEQEtv0cUmLppzg/DSnz+uRjzbx/poD3DYgiid+193ocERERJyKilIN0JwSKmmZdh0tYfonW1mx295fqV2AF/cmdWRs7wgVp85UfKSmH1WafSs5Uve4h599ulr0EOgwFAI7Ok/TdJsNjm47vVJe1kqwVp0+bnGHqEGnC1GOdG9WK+xfbu8/te1TqCg+fSysl7041eM68GljXIzSYignOD/N5fOy2Wz0T/me7OITvDGpL0kxIUaHJCIi4lRUlGqA5pJQSctms9n4fNMRnv5iGznFFQAE+7gzaWAU4xPb4+fpanCEDsZms6/qd6pAtW85VBTVPce3bU3D9CH2QpWPg/XtOlFkj33Xd7Arte5oI4CADvYCVKdke0HKzQmmdp48DjsW20dQ7fzmdGHNZLZ/F3HjIPaahve5EjkL5QTnp7l8XhnZxYyc/QMermbSpw/XDzoiIiLnSUWpBmguCZUIQFlFFe+vyWLe8r21/aa83SzcktiO2wdFE+bXAnooXYjqKjiSDnuW2PtRHVhddxU6gJBu9sJI9BCIGgjuPk0bo81mb+x+ajTUgdV1R0O5eNibuZ9aKS+wY9PG19jKjsHWj+wFqoNrTu939bIXpuLG2b8Pi4thIUrzo5zg/DSXz2tO2m6eXZzB5bEhzL+tr9HhiIiIOB0VpRqguSRUImc6WW3ls42HeWXpHjJzSgBwMZsYHR/B3UM60CW0iQsqzqay3D4F7lQ/qiObqF01DsDsAhEJNSOpkqBtAlguwmi044X2QtnO7+yFqNLsuscDO9lHQ3VOtjcrb66N2/P3wKYP7FP88nef3u8dDD2uh7gbIby340xJFKelnOD8NJfP68ZXVrJmbz5/G92dCf2jjA5HRETE6ago1QDNJaESqY/NZiMtM5e5S3ezem9+7f5hsSHcPaQjfaP8Mekf8r+t7BjsW3Z6Zb+CvXWPu3rbR0+dGkkV2v3CCiQ2G2RvOmM01Jq6Kwi6etn7XnVKtm8B0Q25K+djs8Gh9fbi1JYPofzY6WNBXezFqZ43gH+UYSGKc1NOcH6aw+dVfOIkvf/2LdVWG8v+MpR2gU4w1VlERMTBqCjVAM0hoRI5FxuyCnh12R4Wb83m1J8Evdu15u7BHRneLRSzWcWpc1awz16c2pMGe5dBeV7d497BNQ3Tk+w9qVq3O/u1jhfA7u/to6F2p0JpTt3jQV1Oj4ZqNwBcPRr5ZpxU9Un757ZpIWR8AVUnTh9r199eoOo2BrwCDAtRnI9ygvPTHD6vLzcf4b5319Mx2JvUPyUZHY6IiIhTUlGqAZpDQiVyPvbmlfHaD3v4cN1BKqusAHQI8mby4A6M0Yp9589qhaNbTzdN378CTpbXPSegw+mpfu0HQeF+e3PyXd/CwbVgs54+19XbXsg6NRrKv33T3YuzOlEMGZ/DxgX2IuGpqZYWN+g83N5/qssIcHE3NExxfMoJzk9z+Lz+8sFGPlh3kDsGRTPtmm5GhyMiIuKUVJRqgOaQUIlciNySCt5YsZe3V+6n+IS9YbZW7GsEVZX2QtOpflQHf6o7Ba8+wV2h0zB7k/J2/VU8aYjiw7D5Q3uD9JzNp/d7+NlHTsWNs3/GZrNhIYrjUk5wfpz987LZbPSbmUpuSQXv3JHIoM5BRockIiLilFSUagBnT6hEGqq0oooFWrHv4jlRDPt/PN2PKnc7uLWyj5o6NRqqdaTRUTZPOVvtxanNH0DxodP7/dpB3A32AlVwjHHxicNRTnB+nP3z2nKoiGteWI6Xm4UN06/A3UUjhUVERC6EilIN4OwJlUhj0Yp9TeR4gX2Knoub0ZG0HFYr7F9u7z+17VOoKD59LKyXvTjV4zrwaWNcjOIQlBOcH2f/vF78fif//GYHyV1D+c/EBKPDERERcVoqSjWAsydUIo1NK/ZJs3byOOxYbB9BtfMbsNqnrmIy20evxd0EsVeDeytDwxRjKCc4P87+eV0/ZwU/7S/gmbE9GJ+o/n0iIiIX6lxzApcmjElEnJTJZGJobAhDY0PqrNiXmnGU1IyjWrFPnJurJ3Qfa9/KjsHWj+wFqoNr7Kv57f4ezK4Q2Mk+te/UFhRj36fVD0WahcLyStZnFQCQFBNicDQiIiItg4pSInJeerfzZ87v+9RZsW9DViH3vLNOK/aJ8/MOhH532bf8PbDpA/sUv/zd9t5fudvrnm8yg3+UvUAV3AWCY08/dtf0VhFnsmxnHlYbdAltRURr9U4UERFpCpq+Vw9nH3ou0pS0Yp80ezYbFGZB3g7IzYS8TPt/czPhROHZX+cbAUFd6o6sCo4Bb63m5UyUE5wfZ/68pv43nY/WH+LuwR145KquRocjIiLi1NRTqgGcOaESMYpW7JMWx2aD0qN1i1R5mZC7A0qzz/46r8DTBargmJrCVSz4hoN6szkc5QTnx1k/L6vVRt9nvuNYWSXv3ZXIgI4qHouIiDSEilIN4KwJlYgj0Ip9IsDxwpqRVRk1xaqax4VZZ3+Nmw8EdbYXqIK7nC5c+UeBWdNhjaKc4Pw46+e16WAhv3vxR1q5u7B+2hW4uZiNDklERMSpqdG5iBjC1WLm2kvaMrZ3RJ0V+/63/iD/W39QK/ZJy+DZGiL72bczVZbDsZ0/G1mVae9fVVkCh9fbtzNZ3Os2WT81siqwI7i4N9ktiTRnSzJyARjUKUgFKRERkSakopSIXBRasU+kHm5eENbLvp2pqtJemDo1/S83w/44bydUnYCjW+3bmUwW+yiqn4+sCuoC7q2a7JYaxGaD6kqoLIOTx+Fkec3j8prH5WfsO/6zx2Wnj58sBw8/GPeO0XckTmpJ5lEAkmKCDY5ERESkZVFRSkQuOq3YJ/IbXNwgJNa+nclaXbfJ+pl9qyqK7KsC5u+GzC/qvs4vsv4m614B5x9b9cnThaIzi0C1xaPj51hI+vm5NY9t1Rf+uZ2pVWjjXEdanPyySjYeLAQgKSbE2GBERERaGPWUqoez9kMQcRZasU+kgWw2KMn+WZP1mhFWZblnf513sL1AFdjR/vwXhaafPy4Da1XT3JPZ1T6SzLVmq/PYu+ax5+nHPz/u7gtdhjd6WMoJzo8zfl6LNhzioYXpdA3z5asHLzM6HBERkWZBPaVExGEF+7jzlxGx3JvUqc6Kfc8tzuSl73dpxT6R32IygW+YfeuQVPdYef4ZTdZ3nC5cFR2wF6zKcmH/8gt4T8s5FIe8wNW75viZj89SSDrzWhYVo8UYmronIiJiHBWlRMQwrdxduPOyDkwcEFVnxb7XftjLGyv2MTo+gsmDtWKfyHnxCoB2l9q3M1WU2otVeTsgf699Rb+zFZLqKzRZXO3FMJFmpNpqY9kO++jCoZq6JyIi0uRUlBIRw51txb4P1x3kw3VasU+kUbi3gohL7JuIALDxYCEF5Sfx8XDhknatjQ5HRESkxVFRSkQchlbsExGRppSWYZ+6N7hzMC4Ws8HRiIiItDwqSomIQ/qtFfvG9Y3kd/Hh6jslIiIXLK1m6p76SYmIiBhDPwmJiEOLDvJm5tie/Ph/lzNlaEd8PVzYk1dGylcZDPj794x7ZSXvr8miqPyk0aGKiIgTyS2pYNPBIgCGqCglIiJiCI2UEhGncOaKfZ+mH2ZR+iHW7M1ndc02/ZMtJMWEMCY+gmFdQ/BwtRgdsoiIOLClNaOkekT4EuLjYXA0IiIiLZOKUiLiVFq5u3BLYjtuSWzHocLjfLbxMIs2HCIju4Rvt+Xw7bYcWrm7MLJHG0bHhzOgYxAW9Z8SEZGfScu095PSqnsiIiLGUVFKRJxWRGtP7hnSkXuGdCQzu4RF6Yf4NP0whwqP167cF+zjzqi4cEbHhxPX1k+r94mICFXVVpbV9pNSUUpERMQoKkqJSLMQ08aH/xsZy1+Gx7Auq4BFGw7xxeYj5JZUMP/Hvcz/cS8dgrz5XXw4o+MjiA7yNjpkERExSPqBQopPVNHay5X4yNZGhyMiItJiqSglIs2K2Wyib1QAfaMCmDGqOz/szGVR+mG+3ZbNnrwyZn+3k9nf7aRXWz9Gx0dwTa8w9RIREWlhltRM3RvcOVhTvEVERAykopSINFtuLmaGdQ1lWNdQSiuq+HZbNos2HGb5rjw2Hixi48Einv5iGwM7BTE6PoIR3UPx8XA1OmwREbnIlmTYp+4NjdWqeyIiIkZSUUpEWoRW7i6M7d2Wsb3bkldawRebjrAo/RAbsgr5YWceP+zM47GPzSR3DWV0fDhJMSG4uZiNDltERBpZTvEJth0pxmSyj5QSERER46goJSItTlArdyYOiGLigCj2Hyvj0/TDLEo/xO7cMr7YfIQvNh/Bz9OVq3q2YXR8BP2iAjBreoeISLOwNNM+SiqubWsCW7kbHI2IiEjLpqKUiLRo7QO9eWBYZ+6/vBNbDxezaMMhPt14mKMlFby/5gDvrzlAuJ8Ho+LDGd0rgq5hPlrBT0TEiZ3qJ5XURaOkREREjKailIgIYDKZ6BHhR48IPx65qiur9xxjUfohvtqSzeGiE7yydA+vLN1Dl9BWjI6P4He9wokM8DI6bBEROQ8nq60s35kHwNDYEIOjERERETVMERH5GYvZxIBOQTx3fS/WPpbM3N9fwsjubXCzmNmRU8o/vs7ksueWcP2cFby9aj/5ZZVGhywiLdBLL71EVFQUHh4eJCYmsmbNml89f/bs2cTExODp6UlkZCR//OMfOXHiRO3xlJQU+vbti4+PDyEhIYwZM4bMzMyLfRtNat3+Akoqqgj0diMuws/ocERERFo8jZQSEfkVHq4WRvYIY2SPMIqOn+TrLdksSj/Eyj3H+Gl/AT/tL+DJT7cyuEswo+PDuaJbKF5u+qNVRC6uhQsXMnXqVObOnUtiYiKzZ89mxIgRZGZmEhLyyxFA7733Hg8//DDz589nwIAB7Nixg9tuuw2TycSsWbMAWLp0KVOmTKFv375UVVXx6KOPMnz4cLZt24a3t3dT3+JFcWrq3uAuweoVKCIi4gBMNpvNZnQQjqa4uBg/Pz+Kiorw9fU1OhwRcUDZRSf4fJO9QfqWQ8W1+73cLAzvFsro3hEM6hSEq0UDUkWcmaPmBImJifTt25cXX3wRAKvVSmRkJA888AAPP/zwL86///772b59O6mpqbX7/vSnP7F69WqWL19e73vk5uYSEhLC0qVLGTx48DnF5aif1ykjZy8jI7uE/3dTPKPjI4wOR0REpNk615xA/1oSEbkAbfw8uPOyDnz+wGV8N3Uwf7i8E+0CvCivrGZR+mEmvb6WS2emMv2TLazbX4Dq/yLSWCorK1m3bh3Jycm1+8xmM8nJyaxcubLe1wwYMIB169bVTvHbs2cPX375JVddddVZ36eoqAiAgICARozeOIcLj5ORXYLZBIM7q8m5iIiII9AcExGRBuoU4sPU4TH88YoubDhQyCcbDvH5piMcK6vkrZX7eWvlfiIDPBndK4IxvcPpFOJjdMgi4sTy8vKorq4mNDS0zv7Q0FAyMjLqfc0tt9xCXl4egwYNwmazUVVVxT333MOjjz5a7/lWq5WHHnqIgQMH0qNHj7PGUlFRQUVFRe3z4uLis55rtLTMXADiI1vj7+1mcDQiIiICDjBS6nyadH700UckJCTQunVrvL29iY+P5+23365zjs1mY/r06YSFheHp6UlycjI7d+682LchIoLJZOKSdv48OboHqx8dxhuT+jK2dwRebhYO5B/nxSW7SJ61jKuf/4FXl+0mu+jEb19URKQRpKWlMXPmTF5++WXWr1/PRx99xBdffMFTTz1V7/lTpkxhy5YtLFiw4Fevm5KSgp+fX+0WGRl5McJvFGk1/aSGxmjVPREREUdh6Eip823SGRAQwGOPPUZsbCxubm58/vnnTJo0iZCQEEaMGAHAc889x/PPP8+bb75JdHQ006ZNY8SIEWzbtg0PD4+mvkURaaFcLGaSYkJIigmhvLKK77Yf5ZMNh1i6I5eth4vZeriYlK8yuDQ6kNHx4Yzo3ka/3IvIOQkKCsJisZCTk1Nnf05ODm3atKn3NdOmTePWW2/lzjvvBKBnz56UlZUxefJkHnvsMczm079T3n///Xz++ecsW7aMtm3b/mosjzzyCFOnTq19Xlxc7JCFqcoqKz/uygNgaKyKUiIiIo7C0JFSs2bN4q677mLSpEl069aNuXPn4uXlxfz58+s9PykpibFjx9K1a1c6duzIgw8+SFxcXG2DTpvNxuzZs3n88ccZPXo0cXFxvPXWWxw+fJhFixY14Z2JiJzm5ebC73qFM++2vqx5LJmnx/Sgb5Q/Nhus3HOMhz/aTJ+nv+W6OSt48fudbDlUpB5UInJWbm5u9OnTp07TcqvVSmpqKv3796/3NeXl5XUKTwAWiwWg9s8bm83G/fffz8cff8z3339PdHT0b8bi7u6Or69vnc0R/bQvn7LKaoJaudMtzDFjFBERaYkMGyl1qknnI488Urvvt5p0nslms/H999+TmZnJs88+C8DevXvJzs6u0/jTz8+PxMREVq5cyU033VTvtZypH4KIOLcAbzd+f2l7fn9pew7kl/PZpsN8mn6YjOwS1u0vYN3+Av75zQ5CfNxJiglmaEwIAzsH4evhanToIuJApk6dysSJE0lISKBfv37Mnj2bsrIyJk2aBMCECROIiIggJSUFgFGjRjFr1ix69+5NYmIiu3btYtq0aYwaNaq2ODVlyhTee+89PvnkE3x8fMjOzgbsuZSnp6cxN9pIltRM3UuKCcZsNhkcjYiIiJxiWFHqQpp0gn0lmIiICCoqKrBYLLz88stcccUVALXJU33XPHWsPikpKTz55JMXeisiIhckMsCL+5I6cV9SJw4XHictM5clmUf5cVceR0sq+O9PB/nvTwdxMZtIiPJnaEwIQ2ND6BzSCpNJ/6gSacnGjRtHbm4u06dPJzs7m/j4eBYvXlybA2VlZdUZGfX4449jMpl4/PHHOXToEMHBwYwaNYpnnnmm9pw5c+YA9pHpZ3r99de57bbbLvo9XUxLapqcq5+UiIiIY3G61fd8fHxIT0+ntLSU1NRUpk6dSocOHX6RQJ0PZ+mHICLNV3hrT25JbMctie2oqKpm7d4ClmQeZUnmUfbklrFqTz6r9uST8lUGEa09a0dRDegUiJeb0/1RLiKN4P777+f++++v91haWlqd5y4uLsyYMYMZM2ac9XrNddrwgfxydh0txWI2MahzkNHhiIiIyBkM+5fMhTTpBPsUv06dOgEQHx/P9u3bSUlJISkpqfZ1OTk5hIWF1blmfHz8Wa/p7u6Ou7t7A+5GRKTxuLtYGNQ5iEGdg5h2TTf2HyurHUW1cvcxDhUe593VWby7Ogs3i5nEDgG1o6iig7yNDl9ExKGk7bCPkurTzh8/T02FFhERcSSGFaXObNI5ZswY4HSTzrP96lcfq9Va2w8qOjqaNm3akJqaWluEKi4uZvXq1dx7772NfQsiIk2ifaA3Ewd4M3FAFMcrq1m15xhLMo/yfcZRDhYc54edefywM4+/fb6NqEAvkmoKVInRAXi4WowOX0TEUGkZNf2kYoMNjkRERER+ztA5H+fbpDMlJYWEhAQ6duxIRUUFX375JW+//XZtDwSTycRDDz3E008/TefOnYmOjmbatGmEh4fXFr5ERJyZp5uFobH2otOTv7OxO7eMtJppfmv25rPvWDlvrNjHGyv24eFqZmDHIJJiQ0jqEkxkgJfR4YuINKkTJ6tZsfsYAEld1E9KRETE0RhalDrfJp1lZWXcd999HDx4EE9PT2JjY3nnnXcYN25c7Tl//etfKSsrY/LkyRQWFjJo0CAWL16Mh4dHk9+fiMjFZDKZ6BTSik4hrbjzsg6UVlTx4648e5EqI5fs4hOkZhwltWaUQOeQVgyNDSEpJpiE9gG4uZh/4x1ERJzbmr35HD9ZTaivO13DfIwOR0RERH7GZGuuXS0boLi4GD8/P4qKivD19TU6HBGR82az2cjILmFJ5lHSMnJZl1VAtfX0H/et3F0Y1CmIobHBJMWEEOqrwr1IfZQTnB9H+7ye/Gwrr/+4j5v6RvL36+KMDkdERKTFONecQEs2iYg0QyaTia5hvnQN8+W+pE4UlZ/kh125LMnIZemOo+SVVrJ4azaLt2YD0C3Ml6Gx9hX94iNb42LRKCoRcX5LM+1NzpNi1E9KRETEEakoJSLSAvh5uXJNXDjXxIVjtdrYcriIJRn2Ff02Hixk25Fith0p5qUlu/HzdGVwl2CGxgQzuEswQa20OqmIOJ99eWXsySvDxWxiYKcgo8MRERGReqgoJSLSwpjNJuLatiaubWseTO7MsdIKlu08NYoql6LjJ/ls42E+23gYkwni2rZmaIx9FFXPCD/MZpPRtyAi8pvSMu399BKi/PHxcDU4GhEREamPilIiIi1cYCt3xvZuy9jebamqtrLxYGHtKKqth4vZeKCQjQcKmf3dTgK93RhSU6Aa3DkYPy/9Q09EHFPaDvvUvaExWnVPRETEUakoJSIitVwsZvq0D6BP+wD+PCKGnOITLM20F6h+2JnHsbJKPlp/iI/WH8Jsgj7t/UmKCWFoTAhdw3wwmTSKSkSMd+JkNSt3HwNgaKyKUiIiIo5KRSkRETmrUF8PbuwbyY19I6mssrJufwFpmUdZknmUHTmlrN1XwNp9Bfzj60xCfd0ZGhNCUkwIAzsFarqMiBhm5Z5jVFRZCffzoHNIK6PDERERkbNQUUpERM6Jm4uZ/h0D6d8xkEeu6srBgnLSMnNJyzzKj7uOkVNcwYK1B1iw9gBmE3QL96VfVCD9ov1JiApQw3QRaTJpGfZ+UkmxIRrBKSIi4sBUlBIRkQvS1t+L31/ant9f2p4TJ6tZszefJZlHScvMZW9eGVsOFbPlUDHzf9wLQMdgb/pFB9A3KoB+0QG09fcy+A5EpDmy2WwsyVQ/KREREWegopSIiDSYh6uFwV2CGdwlmBmj4EjRcdbszWftvnzW7i0gM6eE3bll7M4t4/01BwAI9/Ogb7S9QNUvKoBOIa00okFEGmxvXhlZ+eW4WcwM6BhodDgiIiLyK1SUEhGRRhfm58no+AhGx0cAUFBWyU/7C1i7L5/Ve/PZeqiIw0Un+CT9MJ+kHwYgwNuNhPb+taOpuof74mIxG3kbIuKETo2S6hcdgLe7Ul0RERFHpr+pRUTkovP3duOKbqFc0S0UgPLKKjZkFbJ6bz5r9+az4UAB+WWVfLMth2+25QDg7Wbhkvb+tdP94iNb4+FqMfI2RMQJpGXW9JOKCTY4EhEREfktKkqJiEiT83JzYWCnIAZ2CgKgssrK5kNFNdP97NP+ik9U8cPOPH7YmQeAm8VMXFs/+5S/qAD6RPnjqxX+ROQM5ZVVrN6TD0CS+kmJiIg4PBWlRETEcG4uZvq096dPe3/uGdIRq9VGZk4Ja/fls2avfTtaUsFP+wv4aX8Bc9iNyQRd2/jae1LVTPkL9tEKfyIt2Ypdx6isthIZ4EnHYG+jwxEREZHfoKKUiIg4HLPZRNcwX7qG+TKhfxQ2m42s/PLaAtXaffnsO1bOtiPFbDtSzBsr9gEQHeRNv6iA2tFUkQGeap4u0oIsqZm6NzQmRP/fFxERcQIqSomIiMMzmUy0D/SmfaA3NyREAnC0+ARraqb7rdlXQEZ2MXvzytibV8bCn+wr/LXxrVnhL8qfftGBdA5phdmsf6iKNEc2m420mibn6iclIiLiHFSUEhERpxTi68E1ceFcExcOQNHxk6zbn8+avQWs2XuMzYeKyC4+wWcbD/PZRvsKf36ervSNOr3CX48IP1y1wp9Is7DraCmHCo/j5mKmf4cgo8MRERGRc6CilIiINAt+nq5cHhvK5bH2Ff6OV1aTfqCwdrrf+qwCio6f5LvtR/luu32Kj6erhUvat7av8BcVQO92/ni6aYU/EWd0aupe/w6B+v+xiIiIk1BRSkREmiVPNwv9OwbSv2MgACerrWw9XFwz3c9eqCosP8mPu47x465jALiYTfRs60e/KHvz9IT2Afh5aYU/EWegqXsiIiLOR0UpERFpEVwtZuIjWxMf2Zq7BnfAarWxK7e0TvP0I0Un2JBVyIasQl5ZtgeTCWJCfUiMDmBQ52D6dwyklbv+6hRxNKUVVazdlw/Ym5yLiIiIc1BmLSIiLZLZbKJLqA9dQn34/aXtsdlsHCw4XlugWrMvnz25ZWRkl5CRXcKbK/fjajFxSTt/BncJZkiXYLqF+apxuogD+HFXHierbUQHeRMV5G10OCIiInKOVJQSERHBvsJfZIAXkQFeXNenLQC5JRX8tC+fFbuPsWxnLvuPlbN6bz6r9+bzj68zCWrlxqBOQQzuEsxlnYMJ9nE3+C5EWqa0mn5SQ7po6p6IiIgzUVFKRETkLIJ93LmyZxhX9gwDYP+xMpbtyGXpjjxW7s4jr7SSRemHWZRuX92ve7gvg7sEM7hzMH3a++PmopX9RC42m83Gkgx7P6mhsZq6JyIi4kxUlBIRETlH7QO9ubW/N7f2j6Kyysr6rAKW7chl2c5cthwqZuth+zYnbTfeNY3WTxWpNKVI5OLIzCkhu/gEHq5mEqMDjA5HREREzoOKUiIiIhfAzcXMpR0CubRDIH8dGUtuSQXLd+WybEceP+zMJa+0ku+2H+W77fZpRe0CvBjcJYjBnYMZ0ClIDdNFGsmpUVIDOgbh4WoxOBoRERE5H8qIRUREGkGwjztje7dlbO+2WK02th0pZtnOXJbtyGXd/gKy8st5Z1UW76zKwsVs4pL2/gxRw3SRBltS009qaIz6SYmIiDgbFaVEREQamdlsokeEHz0i/LgvqROlFVWs2n2MpTVT/fYfK2fN3nzW1DRMD/R247LOapgucr6KT5xk3f4CAJJi1E9KRETE2agoJSIicpG1cnchuVsoyd1CgV82TD9WVrdherewmobpXYJIaB+ghukiZ7F8Zx7VVhsdg72JDPAyOhwRERE5TypKiYiINLHfapi+7Yh9m7t0N15uFvp3sDdMH9JFDdNFzrQk49TUPY2SEhERcUYqSomIiBjo5w3T80orWL4zr6ZIlUdeaQWpGUdJzVDDdJEz2Ww20nbYm5xr6p6IiIhzUiYrIiLiQIJauTOmdwRjekecV8P0wZ2D6R6uhunScmw9XExuSQVebhb6RvsbHY6IiIhcABWlREREHNTPG6aXVVSxcvex2iLVvnoapg/qbB9FdVmXIEJ8PIy+BZGLJq1m1b2BnYJwd7EYHI2IiIhcCBWlREREnIT3zxqmZx0rZ+nOXJZm5tY2TP8k/TCf1DRM7xrmy+AuQQzpHEyfKH/9w12albTMU1P3gg2ORERERC6UilIiIiJOql2gF7cGtufWS9vX2zB9+xH79srSPXi5Wbi0Q6B9ql+XYKICvTCZNNVPnFNheSXrswoA9ZMSERFxZipKiYiINAPn0jD9+4yjfF/TMD3U152+UQG1W0wbHyzqRyVO4oedeVhtEBPqQ0RrT6PDERERkQukopSIiEgz9POG6duzi1m2w16k+ml/PjnFFXy+6QifbzoCgI+HC33a+9M3KoB+0QH0jPDDw1XT/cQxLanpJ6WpeyIiIs5NRSkREZFmzmw20T3cj+7hftyb1JHjldWkHyhk7b581u7LZ/3+AkpOVJGWmVvbp8fNYqZXpB8JUQH0iwrgkvb++Hm6GnwnImC12lha209KU/dEREScmYpSIiIiLYynm4X+HQPp3zEQgKpqK9uPlNQWqdbuyyevtJK1+wpYu6+AOezGZLJPleoXfXrKXxs/re4nTW/L4SKOlVXSyt2FhCh/o8MRERGRBlBRSkREpIVzsZjp2daPnm39uH1QNDabjX3Hylm793SRat+xcjKyS8jILuGtlfsBiAzwpG/7APrWFKo6BnurebpcdEsy7KOkBnUKwtViNjgaERERaQgVpURERKQOk8lEdJA30UHe3Ng3EoCjJSf4aV8Ba/bm89P+fLYdLuZA/nEO5B/iow2HAAjwdiOhvT/9ogNIiAqge7ivigbS6E71kxoaq35SIiIizk5FKREREflNIT4eXNUzjKt6hgFQcuIk67MK+WlfPmv25pN+oJD8skq+2ZbDN9tyAPB0tdC7Xeva5unxka3xdlfqIRcuv6ySjQcLAfWTEhERaQ6UGYqIiMh58/FwZUiXYIZ0sY9WqayysvlQEWv35fPTvnzW7iug6PhJVuw+xordxwCwmE30CPelb5R9JFXfKH8CW7kbeRviZJbtyMVmg65hvoT6qqeZiIiIs1NRSkRERBrMzcVMn/b+9GnvD0M6YrXa2Hm0tLYn1U/7CjhUeJyNB4vYeLCI/yzfC0CHYG/6RZ1unh4Z4Km+VHJWtVP3YjR1T0REpDlQUUpEREQandlsIqaNDzFtfPj9pe0BOFR4vE7z9B05pezJLWNPbhkL1h4AINTXvbZA1TcqgJg2PljMKlIJVFttLNthb3I+NFZT90RERJoDFaVERESkSUS09iSidwRjekcAUFheyU/7CmqLVJsPFZFTXMHnm47w+aYjAPh4uNCnvX9tkSqurR8erhYjb0MMsvFgIQXlJ/H1cKF3ZGujwxEREZFGoKKUiIiIGKK1lxvJ3UJJ7hYKwPHKatIP1DRP35fP+v0FlJyoIi0zl7RM+wgZN4uZXpF+JEQF0C8qgEva++Pn6WrkbUgTScuwT927rEswLlrVUUREpFlQUUpEREQcgqebhf4dA+nfMRCAqmorGdklrNmbz0/781mzt4C80grW7itg7b4C5rAbkwliQn3oFx3AY1d3xd1Fo6iaq7RTU/e06p6IiEizYfjPTC+99BJRUVF4eHiQmJjImjVrznrua6+9xmWXXYa/vz/+/v4kJyf/4vzbbrsNk8lUZxs5cuTFvg0RERFpZC4WMz0i/Lh9UDQvj+/D2seGkfbnJJ67Po4b+rQlOsgbmw0yskv4bluOClLNWG5JBZsOFgHUrvgoIiIizs/QkVILFy5k6tSpzJ07l8TERGbPns2IESPIzMwkJOSXv4KlpaVx8803M2DAADw8PHj22WcZPnw4W7duJSIiova8kSNH8vrrr9c+d3fXctMiIiLOzmQyERXkTVSQNzcmRAJwtOQE6/YVUF5ZbXB0cjH5ebryzh2JbDtSRLCP8joREZHmwmSz2WxGvXliYiJ9+/blxRdfBMBqtRIZGckDDzzAww8//Juvr66uxt/fnxdffJEJEyYA9pFShYWFLFq06ILjKi4uxs/Pj6KiInx9fS/4OiIiIuLclBOcH31eIiIiAueeExg2fa+yspJ169aRnJx8OhizmeTkZFauXHlO1ygvL+fkyZMEBATU2Z+WlkZISAgxMTHce++9HDt2rFFjFxERERERERGRhjFs+l5eXh7V1dWEhobW2R8aGkpGRsY5XeP//u//CA8Pr1PYGjlyJNdeey3R0dHs3r2bRx99lCuvvJKVK1disdTfa6KiooKKiora58XFxRdwRyIiIiIiIiIicq6cdvW9v//97yxYsIC0tDQ8PDxq99900021j3v27ElcXBwdO3YkLS2NYcOG1XutlJQUnnzyyYses4iIiIiIiIiI2Bk2fS8oKAiLxUJOTk6d/Tk5ObRp0+ZXX/vPf/6Tv//973zzzTfExcX96rkdOnQgKCiIXbt2nfWcRx55hKKiotrtwIED534jIiIiIiIiIiJy3gwrSrm5udGnTx9SU1Nr91mtVlJTU+nfv/9ZX/fcc8/x1FNPsXjxYhISEn7zfQ4ePMixY8cICws76znu7u74+vrW2URERERERERE5OIxrCgFMHXqVF577TXefPNNtm/fzr333ktZWRmTJk0CYMKECTzyyCO15z/77LNMmzaN+fPnExUVRXZ2NtnZ2ZSWlgJQWlrKX/7yF1atWsW+fftITU1l9OjRdOrUiREjRhhyjyIiIiIiIiIi8kuG9pQaN24cubm5TJ8+nezsbOLj41m8eHFt8/OsrCzM5tN1szlz5lBZWcn1119f5zozZszgiSeewGKxsGnTJt58800KCwsJDw9n+PDhPPXUU7i7uzfpvYmIiIiIiIiIyNmZbDabzeggHE1xcTF+fn4UFRVpKp+IiEgL5sg5wUsvvcQ//vEPsrOz6dWrFy+88AL9+vU76/mzZ89mzpw5ZGVlERQUxPXXX09KSkqdBWPO95o/58ifl4iIiDSdc80JDJ2+JyIiIiLnb+HChUydOpUZM2awfv16evXqxYgRIzh69Gi957/33ns8/PDDzJgxg+3btzNv3jwWLlzIo48+esHXFBEREWkoFaVEREREnMysWbO46667mDRpEt26dWPu3Ll4eXkxf/78es9fsWIFAwcO5JZbbiEqKorhw4dz8803s2bNmgu+poiIiEhDqSglIiIi4kQqKytZt24dycnJtfvMZjPJycmsXLmy3tcMGDCAdevW1Rah9uzZw5dffslVV111wdcEqKiooLi4uM4mIiIicq4MbXQuIiIiIucnLy+P6urq2oVhTgkNDSUjI6Pe19xyyy3k5eUxaNAgbDYbVVVV3HPPPbXT9y7kmgApKSk8+eSTDbwjERERaak0UkpERESkmUtLS2PmzJm8/PLLrF+/no8++ogvvviCp556qkHXfeSRRygqKqrdDhw40EgRi4iISEugkVIiIiIiTiQoKAiLxUJOTk6d/Tk5ObRp06be10ybNo1bb72VO++8E4CePXtSVlbG5MmTeeyxxy7omgDu7u64u7s38I5ERESkpdJIKREREREn4ubmRp8+fUhNTa3dZ7VaSU1NpX///vW+pry8HLO5btpnsVgAsNlsF3RNERERkYbSSKl62Gw2ADXrFBERaeFO5QKncgNHMXXqVCZOnEhCQgL9+vVj9uzZlJWVMWnSJAAmTJhAREQEKSkpAIwaNYpZs2bRu3dvEhMT2bVrF9OmTWPUqFG1xanfuua5UA4lIiIicO45lIpS9SgpKQEgMjLS4EhERETEEZSUlODn52d0GLXGjRtHbm4u06dPJzs7m/j4eBYvXlzbqDwrK6vOyKjHH38ck8nE448/zqFDhwgODmbUqFE888wz53zNc6EcSkRERM70WzmUyeZoP/05AKvVyuHDh/Hx8cFkMhkdjsMrLi4mMjKSAwcO4Ovra3Q48jP6fhyfviPHpu/H8V3M78hms1FSUkJ4ePgvpr/JLymHOnf6s8Xx6TtybPp+HJ++I8d2sb+fc82hNFKqHmazmbZt2xodhtPx9fXVHzYOTN+P49N35Nj0/Ti+i/UdOdIIKUenHOr86c8Wx6fvyLHp+3F8+o4c28X8fs4lh9JPfiIiIiIiIiIi0uRUlBIRERERERERkSanopQ0mLu7OzNmzMDd3d3oUKQe+n4cn74jx6bvx/HpOxJnpP/dOj59R45N34/j03fk2Bzl+1GjcxERERERERERaXIaKSUiIiIiIiIiIk1ORSkREREREREREWlyKkqJiIiIiIiIiEiTU1FKLkhKSgp9+/bFx8eHkJAQxowZQ2ZmptFhya/4+9//jslk4qGHHjI6FKlx6NAhfv/73xMYGIinpyc9e/bkp59+MjosqVFdXc20adOIjo7G09OTjh078tRTT6FWjMZZtmwZo0aNIjw8HJPJxKJFi+oct9lsTJ8+nbCwMDw9PUlOTmbnzp3GBCtyFsqhnIvyJ8ekHMqxKYdyLI6eP6koJRdk6dKlTJkyhVWrVvHtt99y8uRJhg8fTllZmdGhST3Wrl3LK6+8QlxcnNGhSI2CggIGDhyIq6srX331Fdu2beNf//oX/v7+RocmNZ599lnmzJnDiy++yPbt23n22Wd57rnneOGFF4wOrcUqKyujV69evPTSS/Uef+6553j++eeZO3cuq1evxtvbmxEjRnDixIkmjlTk7JRDOQ/lT45JOZTjUw7lWBw9f9Lqe9IocnNzCQkJYenSpQwePNjocOQMpaWlXHLJJbz88ss8/fTTxMfHM3v2bKPDavEefvhhfvzxR3744QejQ5GzuOaaawgNDWXevHm1+6677jo8PT155513DIxMAEwmEx9//DFjxowB7L/yhYeH86c//Yk///nPABQVFREaGsobb7zBTTfdZGC0ImenHMoxKX9yXMqhHJ9yKMfliPmTRkpJoygqKgIgICDA4Ejk56ZMmcLVV19NcnKy0aHIGT799FMSEhK44YYbCAkJoXfv3rz22mtGhyVnGDBgAKmpqezYsQOAjRs3snz5cq688kqDI5P67N27l+zs7Dp/1vn5+ZGYmMjKlSsNjEzk1ymHckzKnxyXcijHpxzKeThC/uTSJO8izZrVauWhhx5i4MCB9OjRw+hw5AwLFixg/fr1rF271uhQ5Gf27NnDnDlzmDp1Ko8++ihr167lD3/4A25ubkycONHo8AT7L7HFxcXExsZisViorq7mmWeeYfz48UaHJvXIzs4GIDQ0tM7+0NDQ2mMijkY5lGNS/uTYlEM5PuVQzsMR8icVpaTBpkyZwpYtW1i+fLnRocgZDhw4wIMPPsi3336Lh4eH0eHIz1itVhISEpg5cyYAvXv3ZsuWLcydO1cJlYP473//y7vvvst7771H9+7dSU9P56GHHiI8PFzfkYg0CuVQjkf5k+NTDuX4lEPJ+dD0PWmQ+++/n88//5wlS5bQtm1bo8ORM6xbt46jR49yySWX4OLigouLC0uXLuX555/HxcWF6upqo0Ns0cLCwujWrVudfV27diUrK8ugiOTn/vKXv/Dwww9z00030bNnT2699Vb++Mc/kpKSYnRoUo82bdoAkJOTU2d/Tk5O7TERR6IcyjEpf3J8yqEcn3Io5+EI+ZOKUnJBbDYb999/Px9//DHff/890dHRRockPzNs2DA2b95Menp67ZaQkMD48eNJT0/HYrEYHWKLNnDgwF8sAb5jxw7at29vUETyc+Xl5ZjNdf+atFgsWK1WgyKSXxMdHU2bNm1ITU2t3VdcXMzq1avp37+/gZGJ1KUcyrEpf3J8yqEcn3Io5+EI+ZOm78kFmTJlCu+99x6ffPIJPj4+tfNN/fz88PT0NDg6AfDx8flFfwpvb28CAwPVt8IB/PGPf2TAgAHMnDmTG2+8kTVr1vDqq6/y6quvGh2a1Bg1ahTPPPMM7dq1o3v37mzYsIFZs2Zx++23Gx1ai1VaWsquXbtqn+/du5f09HQCAgJo164dDz30EE8//TSdO3cmOjqaadOmER4eXrvCjIgjUA7l2JQ/OT7lUI5POZRjcfj8ySZyAYB6t9dff93o0ORXDBkyxPbggw8aHYbU+Oyzz2w9evSwubu722JjY22vvvqq0SHJGYqLi20PPvigrV27djYPDw9bhw4dbI899pitoqLC6NBarCVLltT7d8/EiRNtNpvNZrVabdOmTbOFhoba3N3dbcOGDbNlZmYaG7TIzyiHcj7KnxyPcijHphzKsTh6/mSy2Wy2pil/iYiIiIiIiIiI2KmnlIiIiIiIiIiINDkVpUREREREREREpMmpKCUiIiIiIiIiIk1ORSkREREREREREWlyKkqJiIiIiIiIiEiTU1FKRERERERERESanIpSIiIiIiIiIiLS5FSUEhERERERERGRJqeilIhIIzOZTCxatMjoMERERESchvInkZZJRSkRaVZuu+02TCbTL7aRI0caHZqIiIiIQ1L+JCJGcTE6ABGRxjZy5Ehef/31Ovvc3d0NikZERETE8Sl/EhEjaKSUiDQ77u7utGnTps7m7+8P2IeGz5kzhyuvvBJPT086dOjAhx9+WOf1mzdv5vLLL8fT05PAwEAmT55MaWlpnXPmz59P9+7dcXd3JywsjPvvv7/O8by8PMaOHYuXlxedO3fm008/vbg3LSIiItIAyp9ExAgqSolIizNt2jSuu+46Nm7cyPjx47npppvYvn07AGVlZYwYMQJ/f3/Wrl3LBx98wHfffVcnaZozZw5Tpkxh8uTJbN68mU8//ZROnTrVeY8nn3ySG2+8kU2bNnHVVVcxfvx48vPzm/Q+RURERBqL8icRuShsIiLNyMSJE20Wi8Xm7e1dZ3vmmWdsNpvNBtjuueeeOq9JTEy03XvvvTabzWZ79dVXbf7+/rbS0tLa41988YXNbDbbsrOzbTabzRYeHm577LHHzhoDYHv88cdrn5eWltoA21dffdVo9ykiIiLSWJQ/iYhR1FNKRJqdoUOHMmfOnDr7AgICah/379+/zrH+/fuTnp4OwPbt2+nVqxfe3t61xwcOHIjVaiUzMxOTycThw4cZNmzYr8YQFxdX+9jb2xtfX1+OHj16obckIiIiclEpfxIRI6goJSLNjre39y+GgzcWT0/PczrP1dW1znOTyYTVar0YIYmIiIg0mPInETGCekqJSIuzatWqXzzv2rUrAF27dmXjxo2UlZXVHv/xxx8xm83ExMTg4+NDVFQUqampTRqziIiIiJGUP4nIxaCRUiLS7FRUVJCdnV1nn4uLC0FBQQB88MEHJCQkMGjQIN59913WrFnDvHnzABg/fjwzZsxg4sSJPPHEE+Tm5vLAAw9w6623EhoaCsATTzzBPffcQ0hICFdeeSUlJSX8+OOPPPDAA017oyIiIiKNRPmTiBhBRSkRaXYWL15MWFhYnX0xMTFkZGQA9pVdFixYwH333UdYWBjvv/8+3bp1A8DLy4uvv/6aBx98kL59++Ll5cV1113HrFmzaq81ceJETpw4wb///W/+/Oc/ExQUxPXXX990NygiIiLSyJQ/iYgRTDabzWZ0ECIiTcVkMvHxxx8zZswYo0MRERERcQrKn0TkYlFPKRERERERERERaXIqSomIiIiIiIiISJPT9D0REREREREREWlyGiklIiIiIiIiIiJNTkUpERERERERERFpcipKiYiIiIiIiIhIk1NRSkREREREREREmpyKUiIiIiIiIiIi0uRUlBIRERERERERkSanopSIiIiIiIiIiDQ5FaVERERERERERKTJqSglIiIiIiIiIiJN7v8D+lofs4s/tegAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "metrics_df=plot_performance('mnist-model-csv')" ] }, { "cell_type": "code", "execution_count": 35, "id": "GIff0oHPRHWB", "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 363 }, "id": "GIff0oHPRHWB", "outputId": "a66af43e-7f24-48f9-f4cf-001df6ac58d4" }, "outputs": [ { "data": { "application/vnd.google.colaboratory.intrinsic+json": { "summary": "{\n \"name\": \"metrics_df\",\n \"rows\": 10,\n \"fields\": [\n {\n \"column\": \"epoch\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 3,\n \"min\": 1,\n \"max\": 10,\n \"num_unique_values\": 10,\n \"samples\": [\n 9,\n 2,\n 6\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"step\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 2270,\n \"min\": 749,\n \"max\": 7499,\n \"num_unique_values\": 10,\n \"samples\": [\n 6749,\n 1499,\n 4499\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"train_acc\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 0.03068903606365735,\n \"min\": 0.8038958311080933,\n \"max\": 0.9067083597183228,\n \"num_unique_values\": 10,\n \"samples\": [\n 0.9028333425521852,\n 0.8547083139419556,\n 0.8907291889190674\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"train_loss\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 0.09512786206132345,\n \"min\": 0.2538183033466339,\n \"max\": 0.5726363658905029,\n \"num_unique_values\": 10,\n \"samples\": [\n 0.264552503824234,\n 0.4098412692546844,\n 0.3011041581630707\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"val_acc\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 0.011377296218544103,\n \"min\": 0.8504166603088379,\n \"max\": 0.8851666450500488,\n \"num_unique_values\": 10,\n \"samples\": [\n 0.8845000267028809,\n 0.8607500195503235,\n 0.8794166445732117\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"val_loss\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 0.03643069222455714,\n \"min\": 0.3213385641574859,\n \"max\": 0.4301650524139404,\n \"num_unique_values\": 10,\n \"samples\": [\n 0.3213385641574859,\n 0.3992041051387787,\n 0.3320866227149963\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}", "type": "dataframe", "variable_name": "metrics_df" }, "text/html": [ "\n", "
\n", "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
epochsteptrain_acctrain_lossval_accval_loss
017490.8038960.5726360.8504170.430165
1214990.8547080.4098410.8607500.399204
2322490.8690830.3654630.8631670.382029
3429990.8776670.3380870.8748330.349268
4537490.8860830.3171650.8760000.344083
5644990.8907290.3011040.8794170.332087
6752490.8950630.2880010.8770000.344981
7859990.8981460.2771400.8806670.328662
8967490.9028330.2645530.8845000.321339
91074990.9067080.2538180.8851670.323239
\n", "
\n", "
\n", "\n", "
\n", " \n", "\n", " \n", "\n", " \n", "
\n", "\n", "\n", "
\n", " \n", "\n", "\n", "\n", " \n", "
\n", "\n", "
\n", " \n", " \n", " \n", "
\n", "\n", "
\n", "
\n" ], "text/plain": [ " epoch step train_acc train_loss val_acc val_loss\n", "0 1 749 0.803896 0.572636 0.850417 0.430165\n", "1 2 1499 0.854708 0.409841 0.860750 0.399204\n", "2 3 2249 0.869083 0.365463 0.863167 0.382029\n", "3 4 2999 0.877667 0.338087 0.874833 0.349268\n", "4 5 3749 0.886083 0.317165 0.876000 0.344083\n", "5 6 4499 0.890729 0.301104 0.879417 0.332087\n", "6 7 5249 0.895063 0.288001 0.877000 0.344981\n", "7 8 5999 0.898146 0.277140 0.880667 0.328662\n", "8 9 6749 0.902833 0.264553 0.884500 0.321339\n", "9 10 7499 0.906708 0.253818 0.885167 0.323239" ] }, "execution_count": 35, "metadata": {}, "output_type": "execute_result" } ], "source": [ "metrics_df" ] }, { "cell_type": "markdown", "id": "O14kpEUZJPcd", "metadata": { "id": "O14kpEUZJPcd" }, "source": [ "Now, we should look at training and validation accuracy/loss plots to see how well our model is learning and whether it's overfitting, underfitting, or behaving as expected.\n", "Loss is the value of the cost function which should be decreasing over time, and accuracy reflects the model performance, hence it should be increasing over time.\n", "Validation loss and validation accuracy refer to the parameters evaluated on the validation dataset.\n", "Note that training and validation loss are calculated at different times. Training Loss is ften reported as an average over an entire epoch. Since the model's weights are updated after each batch within the epoch, then the average training loss reflects the model's performance across many different weight configurations during the learning process for that epoch. But, validation loss is calculated after the training epoch is complete, using the model weights as they exist at the end of that epoch. So we might see better validation than training loss in the beginning as the model might make significant improvements during the first epoch. The reported average training loss includes the poorer performance from the start of the epoch, while the validation loss reflects the better performance achieved by the end of the epoch using the updated weights.\n", "Ideally, we would like to see loss decreasing, both training and validation, and accuracy increasing as we keep training the network, with validation and training accuracy close. We can see that both training loss and the validation loss decrease during training, while the training accuracy and the validation accuracy increase.\n", "\n", "\n", "Good! Why did we choose 10 epochs, could we have benefited from longer training?\n", "\n", "To help with this question we track the network performance during training on a validation dataset. We can detect overfitting by comparing the performance on the test and the validation dataset - typically indicated by the model's performance improving on the training data but stagnating or worsening on the validation data. Here in our example, we did not see that happen. We could have set a much higher number of epochs and used **Early Stopping** technique where training is halted once the model's performance on the validation set stops improving. We will do this next.\n", "\n", "What if we had both accuracies remaining low, with losses that do not change much during training? This would be a case of underfitting.\n", "And if we had training loss decreasing, while validation kept increasing, this would be a case of overfitting.\n", "\n", "In our case, we also have that validation loss is not smooth, this is still acceptable. We could try a lower learning rate, say lr=1e-4, or again usie early stopping.\n" ] }, { "cell_type": "markdown", "id": "037dd026-614d-4eef-b5ef-77db2504a16f", "metadata": { "id": "037dd026-614d-4eef-b5ef-77db2504a16f" }, "source": [ "If we are not satisfied with the performance of our model, we should go back and tune the hyperparameters. The first one to check is the learning rate. If that doesn't help, we can try another optimizer (and always retune the learning rate after changing any hyperparameter). If the performance is still not great, then we can try tuning model hyperparameters such as the number of layers, the number of neurons per layer, and the types of activation functions to use for each hidden layer.\n", "For both the number of layers, and the number of neurons in the hidden layer, we can try increasing the number until the network starts overfitting. In general, it seems better to increase the number of layers than the number of neurons per layer. We can also try tuning other hyperparameters, such as the *batch size* .\n", "\n", "In the sections below we will have a part dedicated to model tuning.\n", "\n", "Once we are satisfied with the model's validation accuracy, we should evaluate it on the test set to estimate the generalization error before we deploy the model to production." ] }, { "cell_type": "markdown", "id": "4cf7757c-7579-4aea-b4cd-11a98997c5f4", "metadata": { "id": "4cf7757c-7579-4aea-b4cd-11a98997c5f4" }, "source": [ "### Evaluating the trained model on the test dataset" ] }, { "cell_type": "code", "execution_count": null, "id": "6d56a3e6-85d3-4ab5-b3a0-7fba62ebe114", "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 201, "referenced_widgets": [ "374298183b3d4096a8739529cfbf6933", "14300c7dd14146d2afa35d5a83f03024", "e4019cc439d647a8b447367827b44e4d", "b7d69f00abf44aa5908b7543b0780b1c", "08b4788623974884bd4512be6222538f", "080b50127a6345e6865f98ff06d66e9f", "fa3c6d7a0eac4c3e98c275fc2dbab212", "a85a29c916944b2d8ef8579f3a0521e4", "88be2abb00014a5dbb136f180392bbc1", "52e3286dfe334a77bfe1ac68e5ef6775", "450f3a364264429288012140c6458ba3" ] }, "id": "6d56a3e6-85d3-4ab5-b3a0-7fba62ebe114", "outputId": "bbab5f38-0f70-454e-8d10-3ea8b6623292" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/usr/local/lib/python3.11/dist-packages/torch/utils/data/dataloader.py:624: UserWarning: This DataLoader will create 4 worker processes in total. Our suggested max number of worker in current system is 2, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.\n", " warnings.warn(\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "374298183b3d4096a8739529cfbf6933", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Testing: | | 0/? [00:00┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n", "┃ Test metric ┃ DataLoader 0 ┃\n", "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n", "│ test_acc │ 0.883899986743927 │\n", "└───────────────────────────┴───────────────────────────┘\n", "\n" ], "text/plain": [ "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n", "┃\u001b[1m \u001b[0m\u001b[1m Test metric \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m DataLoader 0 \u001b[0m\u001b[1m \u001b[0m┃\n", "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n", "│\u001b[36m \u001b[0m\u001b[36m test_acc \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m 0.883899986743927 \u001b[0m\u001b[35m \u001b[0m│\n", "└───────────────────────────┴───────────────────────────┘\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [ "[{'test_acc': 0.883899986743927}]" ] }, "execution_count": 21, "metadata": {}, "output_type": "execute_result" } ], "source": [ "trainer.test(model, dataloaders=test_loader)" ] }, { "cell_type": "markdown", "id": "2107534c-78a2-41e6-b521-e368776335a7", "metadata": { "id": "2107534c-78a2-41e6-b521-e368776335a7" }, "source": [ "The resulting test set performance, after training for 10 epochs in total, is approximately 88 percent." ] }, { "cell_type": "markdown", "id": "93a54b61-2ca6-47de-b6aa-e008059428e4", "metadata": { "id": "93a54b61-2ca6-47de-b6aa-e008059428e4" }, "source": [ "### Making predictions" ] }, { "cell_type": "markdown", "id": "87917c63-292f-4cea-8782-749a384f22fd", "metadata": { "id": "87917c63-292f-4cea-8782-749a384f22fd" }, "source": [ "With the model trained, we can use it to make predictions. It's important to note that in PyTorch, models behave differently during training and inference, especially for certain layers like dropout and batch normalization. Though we are not using either here, it's important to know that there is a difference and that we should always tell the model if we are doing inference and not training, using model.eval()." ] }, { "cell_type": "code", "execution_count": null, "id": "be87b31a-a063-4fa2-b2fd-5d907af6c0c7", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "be87b31a-a063-4fa2-b2fd-5d907af6c0c7", "outputId": "64f76c1e-b28b-4d40-9db8-27dc31787c61" }, "outputs": [ { "data": { "text/plain": [ "tensor([[-10.8473, -13.4199, -13.0705, -17.4330, -13.2875, -0.3448, -8.9074,\n", " -1.5693, -9.2230, 3.9222]], grad_fn=)" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x, y = test_set[0][0], test_set[0][1]\n", "model.eval()\n", "prediction = model(x)\n", "prediction" ] }, { "cell_type": "markdown", "id": "98b17aeb-36f9-47b4-a244-4640720396ee", "metadata": { "id": "98b17aeb-36f9-47b4-a244-4640720396ee" }, "source": [ "A prediction is an array of 10 numbers. They represent the logit of probability that the image corresponds to each of the 10 different articles of clothing. We can see which label has the highest probability:" ] }, { "cell_type": "code", "execution_count": null, "id": "e1ed30a5-f2df-4d4e-b826-1c3fd240d015", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "e1ed30a5-f2df-4d4e-b826-1c3fd240d015", "outputId": "3a1c67eb-f7c2-4830-8ef6-e7261a341a6d" }, "outputs": [ { "data": { "text/plain": [ "tensor([9])" ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "torch.argmax(prediction, dim=1)" ] }, { "cell_type": "markdown", "id": "87cc566c-1f18-49cb-8f1b-adafd77418e3", "metadata": { "id": "87cc566c-1f18-49cb-8f1b-adafd77418e3" }, "source": [ "So, the model is most confident that this image is an ankle boot, or *class_names[9]*. Examining the test label shows that this classification is correct:" ] }, { "cell_type": "code", "execution_count": null, "id": "7346f83d-da76-4fb6-98d1-a7a82c06aded", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "7346f83d-da76-4fb6-98d1-a7a82c06aded", "outputId": "98364efc-e7b6-4ec4-a461-58728477f5b9" }, "outputs": [ { "data": { "text/plain": [ "9" ] }, "execution_count": 24, "metadata": {}, "output_type": "execute_result" } ], "source": [ "y" ] }, { "cell_type": "markdown", "id": "74fec67b-9e9f-45db-94c8-ec89b28171fa", "metadata": { "id": "74fec67b-9e9f-45db-94c8-ec89b28171fa" }, "source": [ "### Saving and reloading the trained model" ] }, { "cell_type": "markdown", "id": "3Z9jhaYvcI-E", "metadata": { "id": "3Z9jhaYvcI-E" }, "source": [ "Lightning allows us to load a trained model from saved checkpoint. By default, the only checkpoint saved is at the end of the training.\n", "Checkpoints are snapshots of the model's state saved to disk during training. They are a zipped .ckpt file that contain Python dictionary typically with:\n", "- Model weights (state_dict)\n", "- Optimizer state (for resuming training)\n", "- Training epoch or step\n", "Checkpoints allows us to resume training, evaluate, or deploy our model later — without retraining from scratch. Let's check what is inside our checkpoint file:" ] }, { "cell_type": "code", "execution_count": null, "id": "adg0XdTUevkX", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "adg0XdTUevkX", "outputId": "49476728-60a6-4cd8-fc8f-cdda0888cf67" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "dict_keys(['epoch', 'global_step', 'pytorch-lightning_version', 'state_dict', 'loops', 'callbacks', 'optimizer_states', 'lr_schedulers'])\n" ] } ], "source": [ "ckpt_path='mnist-model-tb/lightning_logs/version_0/checkpoints/epoch=9-step=7500.ckpt'\n", "ckpt = torch.load(ckpt_path)\n", "print(ckpt.keys())" ] }, { "cell_type": "code", "execution_count": null, "id": "5mDsw-zZcHkW", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "5mDsw-zZcHkW", "outputId": "7ec452c2-6626-47e3-c0bb-54e9790f2766" }, "outputs": [ { "data": { "text/plain": [ "tensor([9])" ] }, "execution_count": 26, "metadata": {}, "output_type": "execute_result" } ], "source": [ "loaded_model = MNISTModel.load_from_checkpoint(ckpt_path)\n", "model.eval()\n", "x, y = test_set[0][0], test_set[0][1]\n", "prediction = model(x)\n", "torch.argmax(prediction, dim=1)\n" ] }, { "cell_type": "markdown", "id": "a378b310-c693-4e05-b2fa-418cc1aaa5cd", "metadata": { "id": "a378b310-c693-4e05-b2fa-418cc1aaa5cd" }, "source": [ "Lightning also allows us to load a trained model and train it for additional epochs conveniently." ] }, { "cell_type": "code", "execution_count": null, "id": "B3KOCdFmWvZa", "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 468, "referenced_widgets": [ "e70d53a09d1c4a0aa46f9868575833e4", "7b315d75b3f740a791453887294f30f5", "5759a46f103e4133997e113e44233ef5", "b235cf55e08540f0a165268bf4bc97a9", "a268ffca8fb54bea9341ef154ac742f5", "8332169a6f5b4ae5be980cef4f259504", "cad65de7b5ca4351bc61087625de4232", "847dbf7ff7ab4931bbb0b271f9a74d3f", "c470c831a38a4f379836fdbfde58089d", "92f18e3b662040418bdf96d7d7c9ae46", "21ac563ed5474ab79e1c33756dca7e50", "41e5f578f018446094246f6f7ea62959", "08bf3aae048240f286810a23a57ab6ce", "de153318bf764d0680b7756a090f30c8", "eee6719b6a3942ff8eed17671b90d10c", "5fe7490849734721b89511036558047a", "50df2093ef3a4575988f8f255c58002a", "8b38ad55167246b3976dcee637b8b982", "50827632c1804f0e8c22023c63e655b6", "96ce55d2e0c44d588e8040c01567cee7", "d70c4c53afad4c84ad592dad70697f60", "357a9216989c492d97b09b931478a616", "3012d6e77a544de5a13c9355a993d935", "d7e73012bf44455db890cefa226f4ed6", "578833b8285246c9bd4d021d1430c13e", "af9891a0ce674f21a4d5c0f3162beef6", "d9748c38ccda41ac94a034dc6d21826b", "7c2b1d5c43d9475b859fda52dc67bea0", "a6102a3f12d643929e967b23ecd6c846", "020ddcb8c2974493b3ef5dbf064f5aa7", "8feb44c4e6024da8904c7de5be4a719f", "c96af87632a44a5083114c846315d299", "33546ec140a441cbb25c5300b986347f", "4a7ddad69238464caeac96bf27616795", "6977852255e840f0958d1ac1f9985747", "f7361a84df9341ae81113eefc565fd6f", "b1750c0a227340c3b9223f8b7a22837c", "9ad2b3dd5dbe4f02afab448c4e3590e5", "2ad2e85b1f9a4f1e9e7582301dcef5bd", "c03f373f314b4907806a22a0405ea2e0", "0548fb9ec6b04228bb98a8227f0af719", "7f2b768fd52b498a8fcb9e3516651fb0", "31ea9d1a96c2488792e65dba04c2d747", "fee86f6183a9499ea4c932c59f5f5cb1", "f8e8c806d6784ecda642a2ddfd9d7dd5", "353c41034cdd4320ab78a0794520260c", "b01ef2caca6a4d2799b4fcef11f5ac56", "43c8c9ebf96a46679497d85c218b3ed2", "5cade6652ee04d139f9f801d0467cc7f", "e6c06a7218714c2ba1b79f3611b1bc44", "8521faf1a6934b1b9d3fd7635bf50c6f", "0f6c7b386ca848389ed069e8df65ac51", "c603a7c416784cb78123c13a7bb7ca8e", "1127e918c45f4ebe8f94224e1bc4c107", "dd8a1ed1a0cc47a1844fbce14f5b4619", "9553a95e54c847668587d3945bfed92a", "403ef89515f1429f9adf3603536c22f1", "3044a6ff3d324f0a802157ea2e4aa3ab", "5eddda5ad6c14519a8ecbfaf97299f16", "aed2d75c28324b5db72ce18d4d767465", "15a8c8397d34449e9c5b52d170c089b1", "a555413eee4a404781069cd27cc2353d", "6bbb4911ba9548f1a276bd6b6b05c566", "e1b2999baf654e52a7fd0a3961c90d6d", "d73eb94946924b09bfd4e541385db470", "0a1d99965825404aa13980a984fe6fe1", "905c6725e6a84022b0a98415ff7ec5a4", "9c6c2de9d24745349f05e083bdde03e7", "9ca7a649268748ef94db3c3b76edff30", "edc5c94f45954a3dba073c565a3c0e78", "af067ae63c0449cc8db04ea8429bea51", "dca9606689ec470094fffd439140f2e8", "99411929fad34ec5a8eb8f5b5e0adcca", "ea9b9d0b964e409999b904b734c61d90", "9dc43c1439a9452ca98bff07604bbb8d", "a616f5cdf95944a486ef0dade17c22d2", "18d5a8adae3c4770a3b1608c2381ec2c" ] }, "id": "B3KOCdFmWvZa", "outputId": "74c43480-5f42-4c68-a7cb-9590c22b96e1" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "INFO:pytorch_lightning.utilities.rank_zero:You are using the plain ModelCheckpoint callback. Consider using LitModelCheckpoint which with seamless uploading to Model registry.\n", "INFO:pytorch_lightning.utilities.rank_zero:GPU available: False, used: False\n", "INFO:pytorch_lightning.utilities.rank_zero:TPU available: False, using: 0 TPU cores\n", "INFO:pytorch_lightning.utilities.rank_zero:HPU available: False, using: 0 HPUs\n", "/usr/local/lib/python3.11/dist-packages/pytorch_lightning/callbacks/model_checkpoint.py:654: Checkpoint directory mnist-model-tb/lightning_logs/version_0/checkpoints exists and is not empty.\n", "INFO:pytorch_lightning.utilities.rank_zero:Restoring states from the checkpoint path at mnist-model-tb/lightning_logs/version_0/checkpoints/epoch=9-step=7500.ckpt\n", "INFO:pytorch_lightning.callbacks.model_summary:\n", " | Name | Type | Params | Mode\n", "--------------------------------------------------------\n", "0 | model | Sequential | 101 K | eval\n", "1 | loss_fn | CrossEntropyLoss | 0 | eval\n", "2 | train_acc | MulticlassAccuracy | 0 | eval\n", "3 | val_acc | MulticlassAccuracy | 0 | eval\n", "4 | test_acc | MulticlassAccuracy | 0 | eval\n", "--------------------------------------------------------\n", "101 K Trainable params\n", "0 Non-trainable params\n", "101 K Total params\n", "0.407 Total estimated model params size (MB)\n", "0 Modules in train mode\n", "9 Modules in eval mode\n", "INFO:pytorch_lightning.utilities.rank_zero:Restored all states from the checkpoint at mnist-model-tb/lightning_logs/version_0/checkpoints/epoch=9-step=7500.ckpt\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "e70d53a09d1c4a0aa46f9868575833e4", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Sanity Checking: | | 0/? [00:00" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plot_performance('mnist-model-csv')" ] }, { "cell_type": "markdown", "id": "EP-V677faBc0", "metadata": { "id": "EP-V677faBc0" }, "source": [ "Indeed, we can see that training for five more epochs did not improve the validation accuracy. We could have avoided unnecesary training using Early stopping, as we will see next." ] }, { "cell_type": "markdown", "id": "d23659d2-8eb9-49c5-9251-956145ac0fad", "metadata": { "id": "d23659d2-8eb9-49c5-9251-956145ac0fad" }, "source": [ "### Early stopping and saving the model during training" ] }, { "cell_type": "markdown", "id": "VMcVYvlOggSH", "metadata": { "id": "VMcVYvlOggSH" }, "source": [ "Early stopping is a regularization technique used during training that automatically stops training when the model stops improving on a validation set. It prevents overfitting and saves time by stopping training before the model starts to degrade on unseen data. It works by defining which metric should we track on the validation set (parameter `monitor`), and once it stops improving (we need to define is increasing or decreasing of metric an improvement and this is defined by the parameter `mode`) for a predefined number of epochs (defined with parameter `patience`)" ] }, { "cell_type": "code", "execution_count": null, "id": "bW14s035gt5v", "metadata": { "id": "bW14s035gt5v" }, "outputs": [], "source": [ "early_stop_cb = EarlyStopping(\n", " monitor=\"val_loss\",\n", " patience=5,\n", " mode=\"min\",\n", " verbose=True\n", ")" ] }, { "cell_type": "markdown", "id": "ewoxzjg5nO8e", "metadata": { "id": "ewoxzjg5nO8e" }, "source": [ "Let's also define that we should save only the top model during traing based also on validation accuracy, in the file best-mnist.ckpt." ] }, { "cell_type": "code", "execution_count": null, "id": "wNddGgcUnP32", "metadata": { "id": "wNddGgcUnP32" }, "outputs": [], "source": [ "checkpoint_cb = ModelCheckpoint(\n", " monitor=\"val_acc\",\n", " filename=\"best-mnist\",\n", " save_top_k=1,\n", " mode=\"max\"\n", ")" ] }, { "cell_type": "markdown", "id": "RZ4IBzx2n9ry", "metadata": { "id": "RZ4IBzx2n9ry" }, "source": [ "Let's now train the model from scratch for 30 epochs, with early stopping." ] }, { "cell_type": "code", "execution_count": null, "id": "HXRMTONWnuP5", "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 468, "referenced_widgets": [ "c657bb4bf8a840de907a975f59dbe56f", "71a418eeb77e4054aadc84674b299831", "13f7be10f92f4c109a94b36ff1e2c451", "99a0b785644345789d7d742b671bf2c0", "ed280f20224e44a68d922ac93e4b8535", "0fc154788e684df8a130caf7b17b7525", "96d84418502647bfb4be5657fb3748da", "5177ff4d431345078fed6720d3495903", "9b5707edf0b94b5b8c12681ccda04b80", "91d18522722343d093658dfeb083aea4", "64b137dc1d4844b9bfdc4e6b5458d8a2", "10d41feda0be433b950916933eadbd74", "a68dcf701662428787d82111a3d37c1a", "79cd96c7295d48a98101d8473fde6044", "f740a916dfb440428b78251df30599f0", "5c402ddbe66741e4a02f6527ae73fa42", "a58ab633de7943ffbf232eafdc657092", "a0e3dd6b200a4cba8ca4a3eeb83fa78c", "82fb641562594413a8f3f8f3e2c23422", "cce75e14ba91469a82f7fe2372efb03b", "d7f046cc048a4ca9896cdfd0c81f7037", "bd5593c92ca84e54ae48aa1b8843db59", "0f50c49a81fb4c7ea8a30ba5c33a242c", "72f4c55eb4ab4746a40b88ad6a6be877", "dc9e789d02804f3d876cad49d752daaf", "081df58e447c4b7c8cc34600b5d3c923", "1b78304fe9ae4ab6bc8a92e782229685", "5e4c87a8903a4e56a1072021016f58f8", "dbd91e35f4a94c7fa7685348a529b66a", "7caefec1f51e4c2b980ea5f64b472a49", "3dc22a5addd74a9ebb99bcd0a8942cb4", "5718a429b8384387807d735641c3ea1f", "afdce55760c4431dbb0dd4bd9827aad9", "690eb4b23fde4d86b52bcb262bb5ad48", "b8114465819c42c9bf42dfacd49ee50c", "8f3c6eb0609242ae880c2001f678c266", "b33dc5241faf426cb9195a9d786b419f", "bcaef4dc343d42f9b754393bf31a19f4", "adff2e30300e41e79bd2760576763477", "647fa4607d21458e83718a79e2a50a39", "5b5216781d6d4008a8217e6adde5fbe3", "b48dff1b7cc841d9969506f26061d2c4", "603e4d9032be4d139d033dd2f5db5572", "f1daa2804ac74805a950e481557edab0", "14c999697262418fb144c1e3ffed5ad4", "8c16363b28884d07b4d21c5bf903a575", "7800fbc10e1c48888b3dea7ce30c7316", "fbcbdc0774e141dcadb7897303965fb7", "255c48c8d5dd4850b3af2863c2578028", "22a5b300c7c344f0b4a80c07f641b1a6", "a1774b87f7e8497f9d354d56d44a276d", "26d47c45fc7747b8b4b5514c572b05b6", "6cbe35d95599497fadc93b79b380a0ad", "69d34e3c4935407b9d4e668236bd02b4", "4d7bde4f23e64616b66e2ca00862c201", "0c411d3a002543c4a741207ac56f4824", "93153ab1a75b41bba6bbd57f30f8750b", "e095049b602048c99d8d1dd8552fcfe9", "dcbae4193030459081a305c51d52e4e7", "9854b37a6206460793b035b1b43ecb5a", "cb90f97c63ec4e2582478369aa628137", "18b7ef59edf14b10ac2135a23331812a", "064e3ee7eab74e2397fc3da137d444bb", "bbb7526753ca4f7a9298bf6e3377198b", "6ab4a6bc2faf4706ad47e0c3d21b3c5a", "05344f36aa5f4141b3a25c5e6c30f518", "1e2908b2752c4290942a00c41f14acb9", "d340af6a5d4249b2a6449b489888f483", "2587d30d2a694e02aa5ccd0be498204c", "55af8a65276e410cadffa20cb08bbf81", "502d84ffc424432eba0d29e315a636de", "07cf8f8336874708a0b92c2311efc94e", "535e920fb8c942f59b30cfc20483397c", "a2ef959ead414ce1ad692ce4c5a1ebf5", "f5fd0185784d4b89b871e734a2efdfd2", "3810b69aa3184ccb91d8502c015dce6c", "7f6b7ce508254497ac4170d76254615a", "b56d30ff3d884bc893cb823335d547a5", "7a36260b66144a648aede88a6f3fde32", "e56b3c60aa0c48458005049281673649", "56787584d42f4982ba55fbf03a517d9e", "aa4cef8a3b824553b4a85c68a4cfc6e9", "3bc766accb9b46f4a64256b0ee748dfa", "fb93df8350ac4f9db1cbc09ea2d5de5c", "f4241da035db4178bf2adf24e8b4e3f2", "9703098f1e634fa2815e64c2f5f3790a", "6e1e72f4008346f8995d5950b56752d9", "69ec8af183064d149451015c5715dbb7", "25e6a6604f8945248381e22498912592", "1430e9b8e2934d2e91e45b356aaa6fb0", "ad9f4e32300f4406a9f5189044cae2ee", "3d8ee79c5d184c049ad943dc24f0db45", "8c7bc6b2ea2342a2b20a8d6419c1ae9d", "9c5e867f28f0484d839c280d08f58b41", "4317be8e910d4488bf9d2e446fda5442", "4dde3401e64d4a3b8eac548cdbbb5626", "3686607130864dba852689774f6f5ccd", "cbc951497bea49bb8617bb265148db3a", "578c2fba93a542b1a401a38852177dbd" ] }, "id": "HXRMTONWnuP5", "outputId": "4f09559a-4ca5-48d0-d3c0-988840f8c35b" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "INFO:pytorch_lightning.utilities.rank_zero:GPU available: False, used: False\n", "INFO:pytorch_lightning.utilities.rank_zero:TPU available: False, using: 0 TPU cores\n", "INFO:pytorch_lightning.utilities.rank_zero:HPU available: False, using: 0 HPUs\n", "INFO:pytorch_lightning.callbacks.model_summary:\n", " | Name | Type | Params | Mode\n", "--------------------------------------------------------\n", "0 | model | Sequential | 101 K | eval\n", "1 | loss_fn | CrossEntropyLoss | 0 | eval\n", "2 | train_acc | MulticlassAccuracy | 0 | eval\n", "3 | val_acc | MulticlassAccuracy | 0 | eval\n", "4 | test_acc | MulticlassAccuracy | 0 | eval\n", "--------------------------------------------------------\n", "101 K Trainable params\n", "0 Non-trainable params\n", "101 K Total params\n", "0.407 Total estimated model params size (MB)\n", "0 Modules in train mode\n", "9 Modules in eval mode\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "c657bb4bf8a840de907a975f59dbe56f", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Sanity Checking: | | 0/? [00:00= min_delta = 0.0. New best score: 0.318\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "14c999697262418fb144c1e3ffed5ad4", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Validation: | | 0/? [00:00" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plot_performance('mnist-model-es')" ] }, { "cell_type": "markdown", "id": "97069db8-f530-4cd1-b462-d8274c6e5584", "metadata": { "id": "97069db8-f530-4cd1-b462-d8274c6e5584" }, "source": [ "If we are running the notebook in colab, then we should download the trained model to our computer. Uncomment the following lines of code to do so. Alternatively we can also [connect personal Google Drive to Google Colab](https://saturncloud.io/blog/how-to-download-multiple-files-or-an-entire-folder-from-google-colab/)." ] }, { "cell_type": "code", "execution_count": null, "id": "1555ad71-dae1-4daf-bada-3c78755b2073", "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 17 }, "id": "1555ad71-dae1-4daf-bada-3c78755b2073", "outputId": "0f0f9873-42fa-45a9-8267-ad864e6059f7" }, "outputs": [ { "data": { "application/javascript": [ "\n", " async function download(id, filename, size) {\n", " if (!google.colab.kernel.accessAllowed) {\n", " return;\n", " }\n", " const div = document.createElement('div');\n", " const label = document.createElement('label');\n", " label.textContent = `Downloading \"${filename}\": `;\n", " div.appendChild(label);\n", " const progress = document.createElement('progress');\n", " progress.max = size;\n", " div.appendChild(progress);\n", " document.body.appendChild(div);\n", "\n", " const buffers = [];\n", " let downloaded = 0;\n", "\n", " const channel = await google.colab.kernel.comms.open(id);\n", " // Send a message to notify the kernel that we're ready.\n", " channel.send({})\n", "\n", " for await (const message of channel.messages) {\n", " // Send a message to notify the kernel that we're ready.\n", " channel.send({})\n", " if (message.buffers) {\n", " for (const buffer of message.buffers) {\n", " buffers.push(buffer);\n", " downloaded += buffer.byteLength;\n", " progress.value = downloaded;\n", " }\n", " }\n", " }\n", " const blob = new Blob(buffers, {type: 'application/binary'});\n", " const a = document.createElement('a');\n", " a.href = window.URL.createObjectURL(blob);\n", " a.download = filename;\n", " div.appendChild(a);\n", " a.click();\n", " div.remove();\n", " }\n", " " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/javascript": [ "download(\"download_1db38470-a974-4221-a288-58fbe5f5beff\", \"best-mnist.ckpt\", 1228832)" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from google.colab import files\n", "files.download( best_path )" ] }, { "cell_type": "markdown", "id": "40bed9b6-229d-4846-9981-a5b3dcd8d324", "metadata": { "id": "40bed9b6-229d-4846-9981-a5b3dcd8d324" }, "source": [ "## Regression with neural networks: predicting the fuel efficiency of a car" ] }, { "cell_type": "markdown", "id": "13f3f182-89a3-4ae5-8182-9840e6f389bb", "metadata": { "id": "13f3f182-89a3-4ae5-8182-9840e6f389bb" }, "source": [ "In this section, we will see how to use neural networks for regression." ] }, { "cell_type": "markdown", "id": "c98b8fd7-89fd-4d11-b04c-912299e929a7", "metadata": { "id": "c98b8fd7-89fd-4d11-b04c-912299e929a7" }, "source": [ "### Preprocessing the dataset" ] }, { "cell_type": "markdown", "id": "5764f106-2583-4f9b-8065-f3fde69ac3db", "metadata": { "id": "5764f106-2583-4f9b-8065-f3fde69ac3db" }, "source": [ "We will use Auto MPG dataset, which is a common machine learning benchmark dataset for predicting the fuel efficiency of a car in MPG. The full dataset and its description are available from [UCI's machine learning repository](https://archive.ics.uci.edu/ml/datasets/auto+mpg).\n", "We are going to treat five features from the Auto MPG dataset (number of cylinders, displacement,\n", "horsepower, weight, and acceleration) as numeric continuous features. The model year\n", "can be regarded as an ordered categorical (ordinal) feature. Lastly, the manufacturing origin can be\n", "regarded as an unordered categorical (nominal) feature with three possible discrete values, 1, 2, and\n", "3, which correspond to the US, Europe, and Japan, respectively.\n", "Let's first load the data and apply the necessary preprocessing steps, including dropping the incomplete\n", "rows, partitioning the dataset into training and test datasets, as well as standardizing the continuous\n", "features." ] }, { "cell_type": "code", "execution_count": null, "id": "58d9354a-c05d-4da5-bd60-f530c128d7d8", "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 206 }, "id": "58d9354a-c05d-4da5-bd60-f530c128d7d8", "outputId": "a29acbda-e723-4018-c9ab-7f8fca378c3e" }, "outputs": [ { "data": { "application/vnd.google.colaboratory.intrinsic+json": { "summary": "{\n \"name\": \"df\",\n \"rows\": 398,\n \"fields\": [\n {\n \"column\": \"MPG\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 7.815984312565782,\n \"min\": 9.0,\n \"max\": 46.6,\n \"num_unique_values\": 129,\n \"samples\": [\n 17.7,\n 30.5,\n 30.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"Cylinders\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 1,\n \"min\": 3,\n \"max\": 8,\n \"num_unique_values\": 5,\n \"samples\": [\n 4,\n 5,\n 6\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"Displacement\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 104.26983817119581,\n \"min\": 68.0,\n \"max\": 455.0,\n \"num_unique_values\": 82,\n \"samples\": [\n 122.0,\n 307.0,\n 360.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"Horsepower\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 38.49115993282855,\n \"min\": 46.0,\n \"max\": 230.0,\n \"num_unique_values\": 93,\n \"samples\": [\n 92.0,\n 100.0,\n 52.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"Weight\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 846.8417741973271,\n \"min\": 1613.0,\n \"max\": 5140.0,\n \"num_unique_values\": 351,\n \"samples\": [\n 3730.0,\n 1995.0,\n 2215.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"Acceleration\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 2.7576889298126757,\n \"min\": 8.0,\n \"max\": 24.8,\n \"num_unique_values\": 95,\n \"samples\": [\n 14.7,\n 18.0,\n 14.3\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"Model Year\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 3,\n \"min\": 70,\n \"max\": 82,\n \"num_unique_values\": 13,\n \"samples\": [\n 81,\n 79,\n 70\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"Origin\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 0,\n \"min\": 1,\n \"max\": 3,\n \"num_unique_values\": 3,\n \"samples\": [\n 1,\n 3,\n 2\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}", "type": "dataframe", "variable_name": "df" }, "text/html": [ "\n", "
\n", "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
MPGCylindersDisplacementHorsepowerWeightAccelerationModel YearOrigin
018.08307.0130.03504.012.0701
115.08350.0165.03693.011.5701
218.08318.0150.03436.011.0701
316.08304.0150.03433.012.0701
417.08302.0140.03449.010.5701
\n", "
\n", "
\n", "\n", "
\n", " \n", "\n", " \n", "\n", " \n", "
\n", "\n", "\n", "
\n", " \n", "\n", "\n", "\n", " \n", "
\n", "\n", "
\n", "
\n" ], "text/plain": [ " MPG Cylinders Displacement Horsepower Weight Acceleration \\\n", "0 18.0 8 307.0 130.0 3504.0 12.0 \n", "1 15.0 8 350.0 165.0 3693.0 11.5 \n", "2 18.0 8 318.0 150.0 3436.0 11.0 \n", "3 16.0 8 304.0 150.0 3433.0 12.0 \n", "4 17.0 8 302.0 140.0 3449.0 10.5 \n", "\n", " Model Year Origin \n", "0 70 1 \n", "1 70 1 \n", "2 70 1 \n", "3 70 1 \n", "4 70 1 " ] }, "execution_count": 36, "metadata": {}, "output_type": "execute_result" } ], "source": [ "url = 'http://archive.ics.uci.edu/ml/machine-learning-databases/auto-mpg/auto-mpg.data'\n", "column_names = ['MPG', 'Cylinders', 'Displacement', 'Horsepower', 'Weight',\n", " 'Acceleration', 'Model Year', 'Origin']\n", "\n", "df = pd.read_csv(url, names=column_names,\n", " na_values = \"?\", comment='\\t',\n", " sep=\" \", skipinitialspace=True)\n", "\n", "df.head()" ] }, { "cell_type": "code", "execution_count": null, "id": "10c4d551-6985-4285-bdec-72cc2a5a8c7d", "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 335 }, "id": "10c4d551-6985-4285-bdec-72cc2a5a8c7d", "outputId": "16e0cc98-fd5c-4edf-b4ee-ac4a3af9cd68" }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
0
MPG0
Cylinders0
Displacement0
Horsepower6
Weight0
Acceleration0
Model Year0
Origin0
\n", "

" ], "text/plain": [ "MPG 0\n", "Cylinders 0\n", "Displacement 0\n", "Horsepower 6\n", "Weight 0\n", "Acceleration 0\n", "Model Year 0\n", "Origin 0\n", "dtype: int64" ] }, "execution_count": 37, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df.isna().sum()" ] }, { "cell_type": "markdown", "id": "dc249f99-5531-49de-9ff0-e43b4f7a6e89", "metadata": { "id": "dc249f99-5531-49de-9ff0-e43b4f7a6e89" }, "source": [ "We see that the feature *Horsepower* has missing values. While we could apply imputation techniques, here we will simply drop them to focus on the other steps." ] }, { "cell_type": "code", "execution_count": null, "id": "c354a888-ad89-4e42-a1e7-c3796d16add2", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "c354a888-ad89-4e42-a1e7-c3796d16add2", "outputId": "dde0b2c8-d798-4bb2-e813-4e56d0f19ab4" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "Index: 392 entries, 0 to 397\n", "Data columns (total 8 columns):\n", " # Column Non-Null Count Dtype \n", "--- ------ -------------- ----- \n", " 0 MPG 392 non-null float64\n", " 1 Cylinders 392 non-null int64 \n", " 2 Displacement 392 non-null float64\n", " 3 Horsepower 392 non-null float64\n", " 4 Weight 392 non-null float64\n", " 5 Acceleration 392 non-null float64\n", " 6 Model Year 392 non-null int64 \n", " 7 Origin 392 non-null int64 \n", "dtypes: float64(5), int64(3)\n", "memory usage: 27.6 KB\n" ] } ], "source": [ "df = df.dropna()\n", "df.info()" ] }, { "cell_type": "code", "execution_count": null, "id": "1a509910-090a-48cb-812f-527ddfc2d87d", "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 300 }, "id": "1a509910-090a-48cb-812f-527ddfc2d87d", "outputId": "0d1c88f5-dc7c-4b71-d6de-618cd5a8cad1" }, "outputs": [ { "data": { "application/vnd.google.colaboratory.intrinsic+json": { "summary": "{\n \"name\": \"df\",\n \"rows\": 8,\n \"fields\": [\n {\n \"column\": \"MPG\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 131.3073847234828,\n \"min\": 7.805007486571799,\n \"max\": 392.0,\n \"num_unique_values\": 8,\n \"samples\": [\n 23.445918367346938,\n 22.75,\n 392.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"Cylinders\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 136.88494671848898,\n \"min\": 1.7057832474527843,\n \"max\": 392.0,\n \"num_unique_values\": 6,\n \"samples\": [\n 392.0,\n 5.471938775510204,\n 8.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"Displacement\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 142.90038032216435,\n \"min\": 68.0,\n \"max\": 455.0,\n \"num_unique_values\": 8,\n \"samples\": [\n 194.41198979591837,\n 151.0,\n 392.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"Horsepower\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 118.66115898346133,\n \"min\": 38.49115993282855,\n \"max\": 392.0,\n \"num_unique_values\": 8,\n \"samples\": [\n 104.46938775510205,\n 93.5,\n 392.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"Weight\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 1537.3621950208644,\n \"min\": 392.0,\n \"max\": 5140.0,\n \"num_unique_values\": 8,\n \"samples\": [\n 2977.5841836734694,\n 2803.5,\n 392.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"Acceleration\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 133.82983176169847,\n \"min\": 2.7588641191880816,\n \"max\": 392.0,\n \"num_unique_values\": 8,\n \"samples\": [\n 15.541326530612244,\n 15.5,\n 392.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"Model Year\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 118.17309651349842,\n \"min\": 3.6837365435778318,\n \"max\": 392.0,\n \"num_unique_values\": 8,\n \"samples\": [\n 75.9795918367347,\n 76.0,\n 392.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"Origin\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 138.07048932607407,\n \"min\": 0.805518183418305,\n \"max\": 392.0,\n \"num_unique_values\": 6,\n \"samples\": [\n 392.0,\n 1.5765306122448979,\n 3.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}", "type": "dataframe" }, "text/html": [ "\n", "
\n", "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
MPGCylindersDisplacementHorsepowerWeightAccelerationModel YearOrigin
count392.000000392.000000392.000000392.000000392.000000392.000000392.000000392.000000
mean23.4459185.471939194.411990104.4693882977.58418415.54132775.9795921.576531
std7.8050071.705783104.64400438.491160849.4025602.7588643.6837370.805518
min9.0000003.00000068.00000046.0000001613.0000008.00000070.0000001.000000
25%17.0000004.000000105.00000075.0000002225.25000013.77500073.0000001.000000
50%22.7500004.000000151.00000093.5000002803.50000015.50000076.0000001.000000
75%29.0000008.000000275.750000126.0000003614.75000017.02500079.0000002.000000
max46.6000008.000000455.000000230.0000005140.00000024.80000082.0000003.000000
\n", "
\n", "
\n", "\n", "
\n", " \n", "\n", " \n", "\n", " \n", "
\n", "\n", "\n", "
\n", " \n", "\n", "\n", "\n", " \n", "
\n", "\n", "
\n", "
\n" ], "text/plain": [ " MPG Cylinders Displacement Horsepower Weight \\\n", "count 392.000000 392.000000 392.000000 392.000000 392.000000 \n", "mean 23.445918 5.471939 194.411990 104.469388 2977.584184 \n", "std 7.805007 1.705783 104.644004 38.491160 849.402560 \n", "min 9.000000 3.000000 68.000000 46.000000 1613.000000 \n", "25% 17.000000 4.000000 105.000000 75.000000 2225.250000 \n", "50% 22.750000 4.000000 151.000000 93.500000 2803.500000 \n", "75% 29.000000 8.000000 275.750000 126.000000 3614.750000 \n", "max 46.600000 8.000000 455.000000 230.000000 5140.000000 \n", "\n", " Acceleration Model Year Origin \n", "count 392.000000 392.000000 392.000000 \n", "mean 15.541327 75.979592 1.576531 \n", "std 2.758864 3.683737 0.805518 \n", "min 8.000000 70.000000 1.000000 \n", "25% 13.775000 73.000000 1.000000 \n", "50% 15.500000 76.000000 1.000000 \n", "75% 17.025000 79.000000 2.000000 \n", "max 24.800000 82.000000 3.000000 " ] }, "execution_count": 39, "metadata": {}, "output_type": "execute_result" } ], "source": [ "df.describe()" ] }, { "cell_type": "markdown", "id": "28c3a243-e858-400f-8c33-dc71f9aeab37", "metadata": { "id": "28c3a243-e858-400f-8c33-dc71f9aeab37" }, "source": [ "Now, let's split the data into train and test. To simplify the code we will not use the validation set to track the network performance on on it, but in general this also should be done for regression tasks, just as in classification." ] }, { "cell_type": "code", "execution_count": null, "id": "97899312-bb0d-4ace-abba-2ec8176a4e91", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "97899312-bb0d-4ace-abba-2ec8176a4e91", "outputId": "ba21f1fd-a3b3-4981-b520-48d56c5a61a7" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(313, 7)\n", "(79, 7)\n" ] } ], "source": [ "X_train, X_test, y_train, y_test = train_test_split(df.drop(columns='MPG'), df['MPG'], random_state=0, test_size=0.2)\n", "\n", "\n", "print(X_train.shape)\n", "print(X_test.shape)" ] }, { "cell_type": "markdown", "id": "9ff3e1d4-75e8-4627-a728-341d51111b35", "metadata": { "id": "9ff3e1d4-75e8-4627-a728-341d51111b35" }, "source": [ "Let's check the model year distribution:" ] }, { "cell_type": "code", "execution_count": null, "id": "043acd79-12e9-49f9-8ada-f55fc3514c61", "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 447 }, "id": "043acd79-12e9-49f9-8ada-f55fc3514c61", "outputId": "2a7086ca-519a-4d41-c066-00471990646d" }, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 41, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjIAAAGdCAYAAAAIbpn/AAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAI+NJREFUeJzt3X1QlPX+//HXIrcqi4kJGhBWlpVZRyrdsjslzZyyYE431smMOdVEplKT+e1+OoXVCasJ9UyjmKdM42R2N+kpNLtDS9Ss5oQ3WVjcaCmsN7GQfH5/NO2vFRRYF6796PMxszPttRcX7z6D+PTaa3ddxhgjAAAAC0U4PQAAAECwCBkAAGAtQgYAAFiLkAEAANYiZAAAgLUIGQAAYC1CBgAAWIuQAQAA1op0eoCO1tTUpMrKSsXHx8vlcjk9DgAAaANjjHbv3q2+ffsqIuLg512O+JCprKxUamqq02MAAIAgbNu2TSkpKQd9/IgPmfj4eEm/L4Tb7XZ4GgAA0BZer1epqan+v8cP5ogPmT+eTnK73YQMAACWae2yEC72BQAA1iJkAACAtQgZAABgLUIGAABYi5ABAADWImQAAIC1CBkAAGAtQgYAAFiLkAEAANYiZAAAgLUIGQAAYC1CBgAAWIuQAQAA1iJkAACAtSKdHgAADkf6fe86PUK7fT99jNMjAEcMzsgAAABrETIAAMBahAwAALAWIQMAAKxFyAAAAGsRMgAAwFqEDAAAsBYhAwAArEXIAAAAaxEyAADAWoQMAACwFiEDAACsRcgAAABrETIAAMBahAwAALCWoyHzyCOPyOVyBdwGDBjgf7y+vl65ublKTExU9+7dlZ2drZqaGgcnBgAA4cTxMzKnn366qqqq/LdPPvnE/9iUKVP09ttvq7i4WCtXrlRlZaWysrIcnBYAAISTSMcHiIxUcnJys+11dXWaM2eOFixYoOHDh0uSioqKdOqpp2rVqlUaOnRoZ48KAADCjONnZDZt2qS+ffvqhBNO0A033KCKigpJUllZmRobG5WZmenfd8CAAUpLS1NpaelBj+fz+eT1egNuAADgyORoyAwZMkTz5s3T0qVLNWvWLG3dulUXXHCBdu/ererqakVHR6tHjx4BX5OUlKTq6uqDHjM/P18JCQn+W2pqagf/XwAAAKc4+tTS6NGj/f89aNAgDRkyRMcff7xee+01xcXFBXXMadOmKS8vz3/f6/USMwAAHKEcf2rpz3r06KGTTz5ZmzdvVnJyshoaGlRbWxuwT01NTYvX1PwhJiZGbrc74AYAAI5MYRUye/bs0ZYtW9SnTx9lZGQoKipKJSUl/sfLy8tVUVEhj8fj4JQAACBcOPrU0j333KMrrrhCxx9/vCorK/Xwww+rS5cuuv7665WQkKCcnBzl5eWpZ8+ecrvdmjhxojweD69YAgAAkhwOmR9//FHXX3+9fvnlFx177LEaNmyYVq1apWOPPVaSNGPGDEVERCg7O1s+n0+jRo3SzJkznRwZAACEEZcxxjg9REfyer1KSEhQXV0d18sAR6D0+951eoR2+376GKdHAMJeW//+dvwN8QAAwO8I8/YLq4t9AQAA2oOQAQAA1iJkAACAtQgZAABgLUIGAABYi5ABAADWImQAAIC1CBkAAGAtQgYAAFiLkAEAANYiZAAAgLUIGQAAYC1CBgAAWIuQAQAA1iJkAACAtQgZAABgLUIGAABYi5ABAADWImQAAIC1CBkAAGAtQgYAAFiLkAEAANYiZAAAgLUIGQAAYC1CBgAAWIuQAQAA1iJkAACAtQgZAABgLUIGAABYi5ABAADWImQAAIC1CBkAAGAtQgYAAFiLkAEAANYiZAAAgLUIGQAAYC1CBgAAWIuQAQAA1iJkAACAtSKdHsBm6fe96/QIQfl++hinRwAAICQ4IwMAAKxFyAAAAGsRMgAAwFqEDAAAsBYhAwAArEXIAAAAaxEyAADAWoQMAACwFiEDAACsRcgAAABrETIAAMBahAwAALAWIQMAAKxFyAAAAGuFTchMnz5dLpdLkydP9m+rr69Xbm6uEhMT1b17d2VnZ6umpsa5IQEAQFgJi5D54osv9K9//UuDBg0K2D5lyhS9/fbbKi4u1sqVK1VZWamsrCyHpgQAAOHG8ZDZs2ePbrjhBr344os65phj/Nvr6uo0Z84cFRQUaPjw4crIyFBRUZE+++wzrVq1ysGJAQBAuHA8ZHJzczVmzBhlZmYGbC8rK1NjY2PA9gEDBigtLU2lpaWdPSYAAAhDkU5+84ULF2rt2rX64osvmj1WXV2t6Oho9ejRI2B7UlKSqqurD3pMn88nn8/nv+/1ekM2LwAACC+OnZHZtm2bJk2apFdeeUWxsbEhO25+fr4SEhL8t9TU1JAdGwAAhBfHQqasrEzbt2/X4MGDFRkZqcjISK1cuVLPP/+8IiMjlZSUpIaGBtXW1gZ8XU1NjZKTkw963GnTpqmurs5/27ZtWwf/nwAAAKc49tTSiBEj9NVXXwVsmzBhggYMGKCpU6cqNTVVUVFRKikpUXZ2tiSpvLxcFRUV8ng8Bz1uTEyMYmJiOnR2AAAQHhwLmfj4eA0cODBgW7du3ZSYmOjfnpOTo7y8PPXs2VNut1sTJ06Ux+PR0KFDnRgZAACEGUcv9m3NjBkzFBERoezsbPl8Po0aNUozZ850eiwAABAmwipkPvzww4D7sbGxKiwsVGFhoTMDAQCAsOb4+8gAAAAEi5ABAADWImQAAIC1CBkAAGAtQgYAAFiLkAEAANYiZAAAgLUIGQAAYC1CBgAAWIuQAQAA1iJkAACAtQgZAABgLUIGAABYi5ABAADWImQAAIC1CBkAAGAtQgYAAFiLkAEAANYiZAAAgLUIGQAAYC1CBgAAWIuQAQAA1iJkAACAtQgZAABgLUIGAABYi5ABAADWImQAAIC1CBkAAGAtQgYAAFiLkAEAANYiZAAAgLUIGQAAYC1CBgAAWIuQAQAA1iJkAACAtQgZAABgLUIGAABYi5ABAADWImQAAIC1CBkAAGCtSKcHANoi/b53nR6h3b6fPsbpEQDgiMcZGQAAYK2gQua7774L9RwAAADtFlTInHTSSbrkkkv08ssvq76+PtQzAQAAtElQIbN27VoNGjRIeXl5Sk5O1m233abPP/881LMBAAAcUlAhc9ZZZ+m5555TZWWl5s6dq6qqKg0bNkwDBw5UQUGBduzYEeo5AQAAmjmsi30jIyOVlZWl4uJiPfnkk9q8ebPuuecepaam6qabblJVVVWo5gQAAGjmsEJmzZo1uuOOO9SnTx8VFBTonnvu0ZYtW/T++++rsrJSY8eODdWcAAAAzQT1PjIFBQUqKipSeXm5Lr/8cs2fP1+XX365IiJ+76J+/fpp3rx5Sk9PD+WsAAAAAYIKmVmzZumWW27RzTffrD59+rS4T+/evTVnzpzDGg4AAOBQggqZTZs2tbpPdHS0xo8fH8zhAQAA2iSoa2SKiopUXFzcbHtxcbFeeumlwx4KAACgLYIKmfz8fPXq1avZ9t69e+uJJ5447KEAAADaIqiQqaioUL9+/ZptP/7441VRUXHYQwEAALRFUCHTu3dvbdiwodn2L7/8UomJiYc9FAAAQFsEFTLXX3+97rrrLq1YsUL79+/X/v37tXz5ck2aNEnXXXddqGcEAABoUVAh89hjj2nIkCEaMWKE4uLiFBcXp5EjR2r48OHtukZm1qxZGjRokNxut9xutzwej9577z3/4/X19crNzVViYqK6d++u7Oxs1dTUBDMyAAA4AgUVMtHR0Vq0aJG+/fZbvfLKK1q8eLG2bNmiuXPnKjo6us3HSUlJ0fTp01VWVqY1a9Zo+PDhGjt2rL755htJ0pQpU/T222+ruLhYK1euVGVlpbKysoIZGQAAHIGCeh+ZP5x88sk6+eSTg/76K664IuD+448/rlmzZmnVqlVKSUnRnDlztGDBAg0fPlzS7y/7PvXUU7Vq1SoNHTr0cEYHAABHgKBCZv/+/Zo3b55KSkq0fft2NTU1BTy+fPnyoI5ZXFysvXv3yuPxqKysTI2NjcrMzPTvM2DAAKWlpam0tPSgIePz+eTz+fz3vV5vu2cBAAB2CCpkJk2apHnz5mnMmDEaOHCgXC5X0AN89dVX8ng8qq+vV/fu3fXGG2/otNNO0/r16xUdHa0ePXoE7J+UlKTq6uqDHi8/P1+PPvpo0PMAAAB7BBUyCxcu1GuvvabLL7/8sAc45ZRTtH79etXV1ek///mPxo8fr5UrVwZ9vGnTpikvL89/3+v1KjU19bDnBAAA4SeokImOjtZJJ50UkgH+fKyMjAx98cUXeu6553TttdeqoaFBtbW1AWdlampqlJycfNDjxcTEKCYmJiSzAQCA8BbUq5buvvtuPffcczLGhHoeNTU1yefzKSMjQ1FRUSopKfE/Vl5eroqKCnk8npB/XwAAYJ+gzsh88sknWrFihd577z2dfvrpioqKCnh88eLFbTrOtGnTNHr0aKWlpWn37t1asGCBPvzwQy1btkwJCQnKyclRXl6eevbsKbfbrYkTJ8rj8fCKJQAAICnIkOnRo4euvvrqw/7m27dv10033aSqqiolJCRo0KBBWrZsmS699FJJ0owZMxQREaHs7Gz5fD6NGjVKM2fOPOzvCwAAjgxBhUxRUVFIvvmcOXMO+XhsbKwKCwtVWFgYku8HAACOLEG/Id5vv/2mDz/8UFu2bNG4ceMUHx+vyspKud1ude/ePZQzAsARJf2+d50eod2+nz7G6RGAFgUVMj/88IMuu+wyVVRUyOfz6dJLL1V8fLyefPJJ+Xw+zZ49O9RzAgAANBPUq5YmTZqks88+W7t27VJcXJx/+9VXXx3wKiMAAICOFNQZmY8//lifffZZsw+ITE9P108//RSSwQAAAFoT1BmZpqYm7d+/v9n2H3/8UfHx8Yc9FAAAQFsEFTIjR47Us88+67/vcrm0Z88ePfzwwyH52AIAAIC2COqppWeeeUajRo3Saaedpvr6eo0bN06bNm1Sr1699Oqrr4Z6RgAAgBYFFTIpKSn68ssvtXDhQm3YsEF79uxRTk6ObrjhhoCLfwEAADpS0O8jExkZqRtvvDGUswAAALRLUCEzf/78Qz5+0003BTUMAABAewQVMpMmTQq439jYqH379ik6Olpdu3YlZAAAQKcI6lVLu3btCrjt2bNH5eXlGjZsGBf7AgCAThNUyLSkf//+mj59erOzNQAAAB0lZCEj/X4BcGVlZSgPCQAAcFBBXSPz1ltvBdw3xqiqqkovvPCCzj///JAMBgAA0JqgQuaqq64KuO9yuXTsscdq+PDheuaZZ0IxFwAAQKuCCpmmpqZQzwEAANBuIb1GBgAAoDMFdUYmLy+vzfsWFBQE8y0AAABaFVTIrFu3TuvWrVNjY6NOOeUUSdLGjRvVpUsXDR482L+fy+UKzZQAAAAtCCpkrrjiCsXHx+ull17SMcccI+n3N8mbMGGCLrjgAt19990hHRIAAKAlQV0j88wzzyg/P98fMZJ0zDHH6B//+AevWgIAAJ0mqJDxer3asWNHs+07duzQ7t27D3soAACAtggqZK6++mpNmDBBixcv1o8//qgff/xRr7/+unJycpSVlRXqGQEAAFoU1DUys2fP1j333KNx48apsbHx9wNFRionJ0dPP/10SAcEAAA4mKBCpmvXrpo5c6aefvppbdmyRZJ04oknqlu3biEdDgAA4FCCCpk/VFVVqaqqShdeeKHi4uJkjOEl14DF0u971+kREKZs/Nn4fvoYp0dAJwjqGplffvlFI0aM0Mknn6zLL79cVVVVkqScnBxeeg0AADpNUCEzZcoURUVFqaKiQl27dvVvv/baa7V06dKQDQcAAHAoQT219N///lfLli1TSkpKwPb+/fvrhx9+CMlgAAAArQnqjMzevXsDzsT8YefOnYqJiTnsoQAAANoiqJC54IILNH/+fP99l8ulpqYmPfXUU7rkkktCNhwAAMChBPXU0lNPPaURI0ZozZo1amho0L333qtvvvlGO3fu1KeffhrqGQEAAFoU1BmZgQMHauPGjRo2bJjGjh2rvXv3KisrS+vWrdOJJ54Y6hkBAABa1O4zMo2Njbrssss0e/Zs3X///R0xEwAAQJu0+4xMVFSUNmzY0BGzAAAAtEtQTy3deOONmjNnTqhnAQAAaJegLvb97bffNHfuXH3wwQfKyMho9hlLBQUFIRkOAADgUNoVMt99953S09P19ddfa/DgwZKkjRs3BuzDZy0BAIDO0q6Q6d+/v6qqqrRixQpJv38kwfPPP6+kpKQOGQ4AAOBQ2nWNjDEm4P57772nvXv3hnQgAACAtgrqYt8/HBg2AAAAnaldIeNyuZpdA8M1MQAAwCntukbGGKObb77Z/8GQ9fX1uv3225u9amnx4sWhmxAAAOAg2hUy48ePD7h/4403hnQYAACA9mhXyBQVFXXUHAAAAO12WBf7AgAAOImQAQAA1iJkAACAtQgZAABgLUIGAABYi5ABAADWImQAAIC1CBkAAGAtQgYAAFjL0ZDJz8/XOeeco/j4ePXu3VtXXXWVysvLA/apr69Xbm6uEhMT1b17d2VnZ6umpsahiQEAQDhxNGRWrlyp3NxcrVq1Su+//74aGxs1cuRI7d2717/PlClT9Pbbb6u4uFgrV65UZWWlsrKyHJwaAACEi3Z91lKoLV26NOD+vHnz1Lt3b5WVlenCCy9UXV2d5syZowULFmj48OGSfv+8p1NPPVWrVq3S0KFDnRgbAACEibC6Rqaurk6S1LNnT0lSWVmZGhsblZmZ6d9nwIABSktLU2lpaYvH8Pl88nq9ATcAAHBkcvSMzJ81NTVp8uTJOv/88zVw4EBJUnV1taKjo9WjR4+AfZOSklRdXd3icfLz8/Xoo4929LhWS7/vXadHAAAgJMLmjExubq6+/vprLVy48LCOM23aNNXV1flv27ZtC9GEAAAg3ITFGZk777xT77zzjj766COlpKT4tycnJ6uhoUG1tbUBZ2VqamqUnJzc4rFiYmIUExPT0SMDAIAw4OgZGWOM7rzzTr3xxhtavny5+vXrF/B4RkaGoqKiVFJS4t9WXl6uiooKeTyezh4XAACEGUfPyOTm5mrBggV68803FR8f77/uJSEhQXFxcUpISFBOTo7y8vLUs2dPud1uTZw4UR6Ph1csAQAAZ0Nm1qxZkqSLL744YHtRUZFuvvlmSdKMGTMUERGh7Oxs+Xw+jRo1SjNnzuzkSQEAQDhyNGSMMa3uExsbq8LCQhUWFnbCRAAAwCZh86olAACA9iJkAACAtQgZAABgLUIGAABYi5ABAADWImQAAIC1CBkAAGCtsPisJeBIxKeMA87iz+DRgTMyAADAWoQMAACwFiEDAACsRcgAAABrETIAAMBahAwAALAWIQMAAKxFyAAAAGsRMgAAwFqEDAAAsBYhAwAArEXIAAAAaxEyAADAWoQMAACwFiEDAACsRcgAAABrETIAAMBahAwAALAWIQMAAKxFyAAAAGsRMgAAwFqEDAAAsBYhAwAArEXIAAAAaxEyAADAWoQMAACwFiEDAACsRcgAAABrETIAAMBahAwAALAWIQMAAKxFyAAAAGsRMgAAwFqEDAAAsBYhAwAArEXIAAAAaxEyAADAWoQMAACwFiEDAACsRcgAAABrETIAAMBahAwAALAWIQMAAKxFyAAAAGsRMgAAwFqEDAAAsBYhAwAArEXIAAAAazkaMh999JGuuOIK9e3bVy6XS0uWLAl43Bijhx56SH369FFcXJwyMzO1adMmZ4YFAABhx9GQ2bt3r84880wVFha2+PhTTz2l559/XrNnz9bq1avVrVs3jRo1SvX19Z08KQAACEeRTn7z0aNHa/To0S0+ZozRs88+qwceeEBjx46VJM2fP19JSUlasmSJrrvuus4cFQAAhKGwvUZm69atqq6uVmZmpn9bQkKChgwZotLS0oN+nc/nk9frDbgBAIAjU9iGTHV1tSQpKSkpYHtSUpL/sZbk5+crISHBf0tNTe3QOQEAgHPCNmSCNW3aNNXV1flv27Ztc3okAADQQcI2ZJKTkyVJNTU1Adtramr8j7UkJiZGbrc74AYAAI5MYRsy/fr1U3JyskpKSvzbvF6vVq9eLY/H4+BkAAAgXDj6qqU9e/Zo8+bN/vtbt27V+vXr1bNnT6WlpWny5Mn6xz/+of79+6tfv3568MEH1bdvX1111VXODQ0AAMKGoyGzZs0aXXLJJf77eXl5kqTx48dr3rx5uvfee7V3717deuutqq2t1bBhw7R06VLFxsY6NTIAAAgjLmOMcXqIjuT1epWQkKC6urqQXy+Tft+7IT0eAAC2+X76mA45blv//g7ba2QAAABaQ8gAAABrETIAAMBahAwAALAWIQMAAKxFyAAAAGsRMgAAwFqEDAAAsBYhAwAArEXIAAAAaxEyAADAWoQMAACwFiEDAACsRcgAAABrETIAAMBahAwAALAWIQMAAKxFyAAAAGsRMgAAwFqEDAAAsBYhAwAArEXIAAAAaxEyAADAWoQMAACwFiEDAACsRcgAAABrETIAAMBahAwAALAWIQMAAKxFyAAAAGsRMgAAwFqEDAAAsBYhAwAArEXIAAAAaxEyAADAWoQMAACwFiEDAACsRcgAAABrETIAAMBahAwAALAWIQMAAKxFyAAAAGsRMgAAwFqEDAAAsBYhAwAArEXIAAAAaxEyAADAWoQMAACwFiEDAACsRcgAAABrETIAAMBahAwAALAWIQMAAKxFyAAAAGsRMgAAwFpWhExhYaHS09MVGxurIUOG6PPPP3d6JAAAEAbCPmQWLVqkvLw8Pfzww1q7dq3OPPNMjRo1Stu3b3d6NAAA4LCwD5mCggL9/e9/14QJE3Taaadp9uzZ6tq1q+bOnev0aAAAwGGRTg9wKA0NDSorK9O0adP82yIiIpSZmanS0tIWv8bn88nn8/nv19XVSZK8Xm/I52vy7Qv5MQEAsElH/P365+MaYw65X1iHzM8//6z9+/crKSkpYHtSUpK+/fbbFr8mPz9fjz76aLPtqampHTIjAABHs4RnO/b4u3fvVkJCwkEfD+uQCca0adOUl5fnv9/U1KSdO3cqMTFRLpcrZN/H6/UqNTVV27Ztk9vtDtlxj1SsV9uxVm3HWrUda9V2rFXbdeRaGWO0e/du9e3b95D7hXXI9OrVS126dFFNTU3A9pqaGiUnJ7f4NTExMYqJiQnY1qNHj44aUW63mx/0dmC92o61ajvWqu1Yq7Zjrdquo9bqUGdi/hDWF/tGR0crIyNDJSUl/m1NTU0qKSmRx+NxcDIAABAOwvqMjCTl5eVp/PjxOvvss3Xuuefq2Wef1d69ezVhwgSnRwMAAA4L+5C59tprtWPHDj300EOqrq7WWWedpaVLlza7ALizxcTE6OGHH272NBZaxnq1HWvVdqxV27FWbcdatV04rJXLtPa6JgAAgDAV1tfIAAAAHAohAwAArEXIAAAAaxEyAADAWoRMK9LT0+VyuZrdcnNzJUn19fXKzc1VYmKiunfvruzs7GZv4He0ONRa7dy5UxMnTtQpp5yiuLg4paWl6a677vJ/FtbRprWfqz8YYzR69Gi5XC4tWbLEmWEd1pa1Ki0t1fDhw9WtWze53W5deOGF+vXXXx2c2hmtrVV1dbX+9re/KTk5Wd26ddPgwYP1+uuvOzy1M/bv368HH3xQ/fr1U1xcnE488UQ99thjAZ/rY4zRQw89pD59+iguLk6ZmZnatGmTg1M7p7X1amxs1NSpU3XGGWeoW7du6tu3r2666SZVVlZ2/HAGh7R9+3ZTVVXlv73//vtGklmxYoUxxpjbb7/dpKammpKSErNmzRozdOhQc9555zk7tEMOtVZfffWVycrKMm+99ZbZvHmzKSkpMf379zfZ2dlOj+2I1n6u/lBQUGBGjx5tJJk33njDkVmd1tpaffbZZ8btdpv8/Hzz9ddfm2+//dYsWrTI1NfXOzu4A1pbq0svvdScc845ZvXq1WbLli3mscceMxEREWbt2rXODu6Axx9/3CQmJpp33nnHbN261RQXF5vu3bub5557zr/P9OnTTUJCglmyZIn58ssvzZVXXmn69etnfv31Vwcnd0Zr61VbW2syMzPNokWLzLfffmtKS0vNueeeazIyMjp8NkKmnSZNmmROPPFE09TUZGpra01UVJQpLi72P/6///3PSDKlpaUOThke/rxWLXnttddMdHS0aWxs7OTJwk9La7Vu3Tpz3HHHmaqqqqM6ZA504FoNGTLEPPDAAw5PFZ4OXKtu3bqZ+fPnB+zTs2dP8+KLLzoxnqPGjBljbrnlloBtWVlZ5oYbbjDGGNPU1GSSk5PN008/7X+8trbWxMTEmFdffbVTZw0Hra1XSz7//HMjyfzwww8dOhtPLbVDQ0ODXn75Zd1yyy1yuVwqKytTY2OjMjMz/fsMGDBAaWlpKi0tdXBS5x24Vi2pq6uT2+1WZGTYvy9jh2pprfbt26dx48apsLDwoJ8rdjQ6cK22b9+u1atXq3fv3jrvvPOUlJSkiy66SJ988onTozqupZ+r8847T4sWLdLOnTvV1NSkhQsXqr6+XhdffLGzwzrgvPPOU0lJiTZu3ChJ+vLLL/XJJ59o9OjRkqStW7equro64Pd7QkKChgwZclT+fm9tvVpSV1cnl8vVoZ93KImnltpj0aJFpkuXLuann34yxhjzyiuvmOjo6Gb7nXPOOebee+/t7PHCyoFrdaAdO3aYtLQ083//93+dPFn4aWmtbr31VpOTk+O/L87IGGOar1VpaamRZHr27Gnmzp1r1q5dayZPnmyio6PNxo0bHZ7WWS39XO3atcuMHDnSSDKRkZHG7XabZcuWOTilc/bv32+mTp1qXC6XiYyMNC6XyzzxxBP+xz/99FMjyVRWVgZ83V//+ldzzTXXdPa4jmttvQ7066+/msGDB5tx48Z1+GxH9z+F22nOnDkaPXp0qx8pjkOvldfr1ZgxY3TaaafpkUce6fzhwsyBa/XWW29p+fLlWrduncOThZ8D16qpqUmSdNttt/k/f+0vf/mLSkpKNHfuXOXn5zs2q9Na+jP44IMPqra2Vh988IF69eqlJUuW6JprrtHHH3+sM844w8FpO99rr72mV155RQsWLNDpp5+u9evXa/Lkyerbt6/Gjx/v9Hhhpz3r1djYqGuuuUbGGM2aNavjh+vwVDpCfP/99yYiIsIsWbLEv62kpMRIMrt27QrYNy0tzRQUFHTyhOGjpbX6g9frNR6Px4wYMeKovGDuQC2t1aRJk4zL5TJdunTx3ySZiIgIc9FFFzk3rMNaWqvvvvvOSDL//ve/A/a95pprOuVfguGqpbXavHmzkWS+/vrrgH1HjBhhbrvtts4e0XEpKSnmhRdeCNj22GOPmVNOOcUYY8yWLVuMJLNu3bqAfS688EJz1113ddaYYaO19fpDQ0ODueqqq8ygQYPMzz//3CmzcY1MGxUVFal3794aM2aMf1tGRoaioqJUUlLi31ZeXq6Kigp5PB4nxgwLLa2V9PuZmJEjRyo6OlpvvfWWYmNjHZowfLS0Vvfdd582bNig9evX+2+SNGPGDBUVFTk0qfNaWqv09HT17dtX5eXlAftu3LhRxx9/fGePGDZaWqt9+/ZJkiIiAn/td+nSxX9m62iyb9++Q65Fv379lJycHPD73ev1avXq1Ufl7/fW1kv6/2diNm3apA8++ECJiYmdM1yn5JLl9u/fb9LS0szUqVObPXb77bebtLQ0s3z5crNmzRrj8XiMx+NxYMrwcLC1qqurM0OGDDFnnHGG2bx5c8BLRH/77TeHpnXWoX6uDqSj/BqZQ63VjBkzjNvtNsXFxWbTpk3mgQceMLGxsWbz5s0OTOq8g61VQ0ODOemkk8wFF1xgVq9ebTZv3mz++c9/GpfLZd59912HpnXO+PHjzXHHHed/OfHixYtNr169Aq5vnD59uunRo4d58803zYYNG8zYsWOP2pdft7ZeDQ0N5sorrzQpKSlm/fr1Ab/jfT5fh85GyLTBsmXLjCRTXl7e7LFff/3V3HHHHeaYY44xXbt2NVdffbWpqqpyYMrwcLC1WrFihZHU4m3r1q3ODOuwQ/1cHehoD5nW1io/P9+kpKSYrl27Go/HYz7++ONOnjB8HGqtNm7caLKyskzv3r1N165dzaBBg5q9HPto4fV6zaRJk0xaWpqJjY01J5xwgrn//vsD/tJtamoyDz74oElKSjIxMTFmxIgRbfrzeiRqbb22bt160N/xB74/Vqi5jPnT2xgCAABYhGtkAACAtQgZAABgLUIGAABYi5ABAADWImQAAIC1CBkAAGAtQgYAAFiLkAEAANYiZAAAgLUIGQAAYC1CBgAAWIuQAQAA1vp/rBHozlJa8j4AAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "X_train['Model Year'].plot(kind='hist')" ] }, { "cell_type": "markdown", "id": "59bf9d9b-1f39-4b8c-ad7e-c2048981adbe", "metadata": { "id": "59bf9d9b-1f39-4b8c-ad7e-c2048981adbe" }, "source": [ "Next, let's group the rather fine-grained model year (ModelYear) information into buckets to simplify\n", "the learning task for the model. We will use the following buckets:\n", "- 0 if year < 73\n", "- 1 if 73 ≤ year < 76\n", "- 2 if 76 ≤ year < 79\n", "- 3 if year ≥ 79" ] }, { "cell_type": "code", "execution_count": null, "id": "f4b1c76b-e3a1-4000-97f1-7a2aa5b0cb25", "metadata": { "id": "f4b1c76b-e3a1-4000-97f1-7a2aa5b0cb25" }, "outputs": [], "source": [ "bins = [50, 72, 75, 78, 99]\n", "labels = [0,1,2,3]\n", "X_train['Model Year bin'] = pd.cut(X_train['Model Year'], bins=bins, labels=labels)\n", "X_test['Model Year bin'] = pd.cut(X_test['Model Year'], bins=bins, labels=labels)" ] }, { "cell_type": "code", "execution_count": null, "id": "709af809-6fd3-422a-8d23-3cef447b4f32", "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 206 }, "id": "709af809-6fd3-422a-8d23-3cef447b4f32", "outputId": "bbfcb267-6588-4ec9-9ac0-c242f7151b60" }, "outputs": [ { "data": { "application/vnd.google.colaboratory.intrinsic+json": { "summary": "{\n \"name\": \"X_train\",\n \"rows\": 313,\n \"fields\": [\n {\n \"column\": \"Cylinders\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 1,\n \"min\": 3,\n \"max\": 8,\n \"num_unique_values\": 5,\n \"samples\": [\n 6,\n 5,\n 8\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"Displacement\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 103.20115286770567,\n \"min\": 68.0,\n \"max\": 455.0,\n \"num_unique_values\": 75,\n \"samples\": [\n 113.0,\n 183.0,\n 318.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"Horsepower\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 37.915348155970854,\n \"min\": 46.0,\n \"max\": 230.0,\n \"num_unique_values\": 88,\n \"samples\": [\n 93.0,\n 70.0,\n 82.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"Weight\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 841.1349467037953,\n \"min\": 1613.0,\n \"max\": 5140.0,\n \"num_unique_values\": 286,\n \"samples\": [\n 2020.0,\n 2725.0,\n 2000.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"Acceleration\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 2.785475580851138,\n \"min\": 8.0,\n \"max\": 24.8,\n \"num_unique_values\": 89,\n \"samples\": [\n 16.7,\n 16.6,\n 19.4\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"Model Year\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 3,\n \"min\": 70,\n \"max\": 82,\n \"num_unique_values\": 13,\n \"samples\": [\n 71,\n 74,\n 77\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"Origin\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 0,\n \"min\": 1,\n \"max\": 3,\n \"num_unique_values\": 3,\n \"samples\": [\n 3,\n 1,\n 2\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"Model Year bin\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 4,\n \"samples\": [\n 3,\n 1,\n 2\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}", "type": "dataframe", "variable_name": "X_train" }, "text/html": [ "\n", "
\n", "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
CylindersDisplacementHorsepowerWeightAccelerationModel YearOriginModel Year bin
220485.070.01945.016.87732
2566225.0100.03430.017.27812
3014105.070.02200.013.27913
1936200.081.03012.017.67612
574113.095.02278.015.57230
\n", "
\n", "
\n", "\n", "
\n", " \n", "\n", " \n", "\n", " \n", "
\n", "\n", "\n", "
\n", " \n", "\n", "\n", "\n", " \n", "
\n", "\n", "
\n", "
\n" ], "text/plain": [ " Cylinders Displacement Horsepower Weight Acceleration Model Year \\\n", "220 4 85.0 70.0 1945.0 16.8 77 \n", "256 6 225.0 100.0 3430.0 17.2 78 \n", "301 4 105.0 70.0 2200.0 13.2 79 \n", "193 6 200.0 81.0 3012.0 17.6 76 \n", "57 4 113.0 95.0 2278.0 15.5 72 \n", "\n", " Origin Model Year bin \n", "220 3 2 \n", "256 1 2 \n", "301 1 3 \n", "193 1 2 \n", "57 3 0 " ] }, "execution_count": 43, "metadata": {}, "output_type": "execute_result" } ], "source": [ "X_train.head()" ] }, { "cell_type": "code", "execution_count": null, "id": "616fdefb-7d26-457f-8079-e2a70ff36252", "metadata": { "id": "616fdefb-7d26-457f-8079-e2a70ff36252" }, "outputs": [], "source": [ "X_train=X_train.drop(columns='Model Year')\n", "X_test=X_test.drop(columns='Model Year')" ] }, { "cell_type": "markdown", "id": "9f1ade43-b3bb-4e33-abdc-bc78cd6df538", "metadata": { "id": "9f1ade43-b3bb-4e33-abdc-bc78cd6df538" }, "source": [ "After checking that the mapping was done correctly, we can drop the original column." ] }, { "cell_type": "code", "execution_count": null, "id": "3e238782-b197-4553-a54d-f45f2455296e", "metadata": { "id": "3e238782-b197-4553-a54d-f45f2455296e" }, "outputs": [], "source": [ "num_cols = ['Cylinders', 'Displacement', 'Horsepower', 'Weight', 'Acceleration']\n", "\n", "ohe_cols=['Origin']" ] }, { "cell_type": "code", "execution_count": null, "id": "81db7575-db1e-4507-b456-9bff7886f475", "metadata": { "id": "81db7575-db1e-4507-b456-9bff7886f475" }, "outputs": [], "source": [ "ct = ColumnTransformer([\n", " ('ohe', OneHotEncoder(sparse_output=False), ohe_cols),\n", " ('scaler', StandardScaler(), num_cols)\n", " ], remainder='passthrough')\n", "\n", "ct.fit(X_train)\n", "X_train = ct.transform(X_train)\n", "X_test = ct.transform(X_test)" ] }, { "cell_type": "code", "execution_count": null, "id": "9a2e988d-1939-4def-8ebd-810278e70b0f", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "9a2e988d-1939-4def-8ebd-810278e70b0f", "outputId": "adcbc6ff-9145-47cc-939c-19ece3e962df" }, "outputs": [ { "data": { "text/plain": [ "((313, 9), (313,), (79, 9), (79,))" ] }, "execution_count": 47, "metadata": {}, "output_type": "execute_result" } ], "source": [ "X_train.shape, y_train.shape, X_test.shape, y_test.shape" ] }, { "cell_type": "markdown", "id": "62af9265-68e0-44a0-9ba5-0365e210666c", "metadata": { "id": "62af9265-68e0-44a0-9ba5-0365e210666c" }, "source": [ "Next we will convert the dataset first to tensor type, then to to Tensor Dataset that we can then use for *DataLoader* as previously seen." ] }, { "cell_type": "code", "execution_count": null, "id": "d587ef33-06da-43a3-a56c-db0e27314107", "metadata": { "id": "d587ef33-06da-43a3-a56c-db0e27314107" }, "outputs": [], "source": [ "train_ds = TensorDataset(torch.tensor(X_train, dtype=torch.float32), torch.tensor(y_train.values, dtype=torch.float32))\n", "test_ds = TensorDataset(torch.tensor(X_test, dtype=torch.float32), torch.tensor(y_test.values, dtype=torch.float32))" ] }, { "cell_type": "markdown", "id": "deb77197-22de-4c1c-afd3-906023893a34", "metadata": { "id": "deb77197-22de-4c1c-afd3-906023893a34" }, "source": [ "We have a much smaller dataset than in the previous example, and we will use a smaller batch size." ] }, { "cell_type": "code", "execution_count": null, "id": "ad59f0a9-6c86-46fb-8949-754357b0bd6f", "metadata": { "id": "ad59f0a9-6c86-46fb-8949-754357b0bd6f" }, "outputs": [], "source": [ "batch_size = 8\n", "torch.manual_seed(1)\n", "train_dl = DataLoader(train_ds, batch_size, shuffle=True)\n", "test_dl = DataLoader(test_ds, batch_size, shuffle=True)" ] }, { "cell_type": "markdown", "id": "179a8691-67f3-426e-aad9-3e7c12419e91", "metadata": { "id": "179a8691-67f3-426e-aad9-3e7c12419e91" }, "source": [ "### Training and evaluating the network" ] }, { "cell_type": "code", "execution_count": null, "id": "6sTNebFd11Og", "metadata": { "id": "6sTNebFd11Og" }, "outputs": [], "source": [ "class regressionModel(pl.LightningModule):\n", " def __init__(self):\n", " super().__init__()\n", " self.model = nn.Sequential(\n", " nn.Linear(9, 8),\n", " nn.ReLU(),\n", " nn.Linear(8, 4),\n", " nn.ReLU(),\n", " nn.Linear(4, 1),\n", " )\n", " self.loss_fn = nn.MSELoss()\n", " self.train_mae = MeanAbsoluteError()\n", " self.test_mae = MeanAbsoluteError()\n", "\n", " def forward(self, x):\n", " return self.model(x)\n", "\n", " def training_step(self, batch, batch_idx):\n", " x, y = batch\n", " preds = self(x)\n", " loss = self.loss_fn(preds.squeeze(), y)\n", " mae_val = self.train_mae(preds.squeeze(dim=-1), y)\n", " self.log(\"train_mae\", mae_val, on_step=False, on_epoch=True)\n", " self.log(\"train_loss\", loss, prog_bar=True, on_step=False, on_epoch=True)\n", " return loss\n", "\n", " def test_step(self, batch, batch_idx):\n", " x, y = batch\n", " preds = self(x)\n", " # prediction will be in shape [batch_size, 1], but our y is in the shape [batch_size]\n", " # with squeeze() we reshape predictions to [batch_size]\n", " loss = self.loss_fn(preds.squeeze(), y)\n", " mae_val = self.test_mae(preds.squeeze(dim=-1), y)\n", " self.log(\"test_mae\", mae_val, on_step=False, on_epoch=True)\n", " self.log(\"test_loss\", loss, on_step=False, on_epoch=True)\n", " return loss\n", "\n", " def configure_optimizers(self):\n", " return optim.SGD(self.parameters(), lr=0.001)\n", "\n", "model= regressionModel()" ] }, { "cell_type": "code", "execution_count": null, "id": "74389961-8319-44f4-8325-ef4a1413bb9b", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "74389961-8319-44f4-8325-ef4a1413bb9b", "outputId": "af36bb3a-7d5c-486b-ca32-c591e6d61ee2" }, "outputs": [ { "data": { "text/plain": [ "=================================================================\n", "Layer (type:depth-idx) Param #\n", "=================================================================\n", "regressionModel --\n", "├─Sequential: 1-1 --\n", "│ └─Linear: 2-1 80\n", "│ └─ReLU: 2-2 --\n", "│ └─Linear: 2-3 36\n", "│ └─ReLU: 2-4 --\n", "│ └─Linear: 2-5 5\n", "├─MSELoss: 1-2 --\n", "├─MeanAbsoluteError: 1-3 --\n", "├─MeanAbsoluteError: 1-4 --\n", "=================================================================\n", "Total params: 121\n", "Trainable params: 121\n", "Non-trainable params: 0\n", "=================================================================" ] }, "execution_count": 51, "metadata": {}, "output_type": "execute_result" } ], "source": [ "\n", "summary(model)" ] }, { "cell_type": "code", "execution_count": null, "id": "0fc5925b-624b-433f-954e-9221fcce7591", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "0fc5925b-624b-433f-954e-9221fcce7591", "outputId": "16f49693-7919-41eb-aa1c-5643ed2d4afc" }, "outputs": [ { "data": { "text/plain": [ "regressionModel(\n", " (model): Sequential(\n", " (0): Linear(in_features=9, out_features=8, bias=True)\n", " (1): ReLU()\n", " (2): Linear(in_features=8, out_features=4, bias=True)\n", " (3): ReLU()\n", " (4): Linear(in_features=4, out_features=1, bias=True)\n", " )\n", " (loss_fn): MSELoss()\n", " (train_mae): MeanAbsoluteError()\n", " (test_mae): MeanAbsoluteError()\n", ")" ] }, "execution_count": 52, "metadata": {}, "output_type": "execute_result" } ], "source": [ "model" ] }, { "cell_type": "markdown", "id": "669e886d-1d00-46b7-91ee-c279bf4fb0d5", "metadata": { "id": "669e886d-1d00-46b7-91ee-c279bf4fb0d5" }, "source": [ "The input layer has 9 neurons, which is as expected, as our training dataset has 9 features. Looking at the model for classification we built previously, the main difference is the fact that the output layer has a single neuron (since we only want to predict a single value) and uses linear activation function." ] }, { "cell_type": "markdown", "id": "e5394c48-d531-4579-b24f-01e1b3999539", "metadata": { "id": "e5394c48-d531-4579-b24f-01e1b3999539" }, "source": [ "Since this is a regression task, we set the loss function to the mean squared error.\n", "\n", "We also changed the optimizer, but this was just to show another option that we can use, not that it is more suitable for the regression task. And now we are also logging MAE, for illustration. Now, we will also specify the folder where we want to save the logs." ] }, { "cell_type": "code", "execution_count": null, "id": "Oxu0bCOV3kd5", "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 451, "referenced_widgets": [ "b3cc361c0c1b4895a1343693bab9c73f", "d2c171df7a954896a1ecabdaebe9f46e", "fb63ccc027ab46a5b3f1923292820406", "31a6eae7f3ec40849d57e9f6d11bc7af", "822be6819d49429b8510a4b4a17e305b", "b40393ad178c41a690c6af61f8eebfec", "b64e2600c41b491ea916e4fbe9400a19", "6e2a4427d0874568820647b8a66da50b", "2748e2a9625b4e72a051d704a80e7ee4", "84973c4191734c179409fd12c58e357a", "4d999b0472014da3a9b46c228d1fce12" ] }, "id": "Oxu0bCOV3kd5", "outputId": "53d12909-2d68-4964-c0df-3ecbfcbc1538" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "INFO:pytorch_lightning.utilities.rank_zero:You are using the plain ModelCheckpoint callback. Consider using LitModelCheckpoint which with seamless uploading to Model registry.\n", "INFO:pytorch_lightning.utilities.rank_zero:GPU available: False, used: False\n", "INFO:pytorch_lightning.utilities.rank_zero:TPU available: False, using: 0 TPU cores\n", "INFO:pytorch_lightning.utilities.rank_zero:HPU available: False, using: 0 HPUs\n", "INFO:pytorch_lightning.callbacks.model_summary:\n", " | Name | Type | Params | Mode \n", "--------------------------------------------------------\n", "0 | model | Sequential | 121 | train\n", "1 | loss_fn | MSELoss | 0 | train\n", "2 | train_mae | MeanAbsoluteError | 0 | train\n", "3 | test_mae | MeanAbsoluteError | 0 | train\n", "--------------------------------------------------------\n", "121 Trainable params\n", "0 Non-trainable params\n", "121 Total params\n", "0.000 Total estimated model params size (MB)\n", "9 Modules in train mode\n", "0 Modules in eval mode\n", "/usr/local/lib/python3.11/dist-packages/pytorch_lightning/loops/fit_loop.py:310: The number of training batches (40) is smaller than the logging interval Trainer(log_every_n_steps=50). Set a lower value for log_every_n_steps if you want to see logs for the training epoch.\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "b3cc361c0c1b4895a1343693bab9c73f", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Training: | | 0/? [00:00" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "metrics_df = pd.read_csv('regression_logs/lightning_logs/version_0/metrics.csv')\n", "epochs = metrics_df['epoch'].unique()\n", "train_loss = metrics_df['train_loss'].values\n", "train_mae = metrics_df['train_mae'].values\n", "fig, ax = plt.subplots(1, 2, figsize=(12, 5))\n", "\n", "ax[0].plot(epochs, train_loss, label='Training Loss')\n", "ax[0].set_xlabel(\"Epoch\")\n", "ax[0].set_ylabel(\"Loss\")\n", "\n", "ax[1].plot(epochs, train_mae, label='Training MAE')\n", "ax[1].set_xlabel(\"Epoch\")\n", "ax[1].set_ylabel(\"MAE\")\n", "\n", "plt.tight_layout()\n" ] }, { "cell_type": "markdown", "id": "e0007b04-9455-4f8a-be5e-2c2a6b22a11e", "metadata": { "id": "e0007b04-9455-4f8a-be5e-2c2a6b22a11e" }, "source": [ "Now, let's evaluate the network performance on the test set:" ] }, { "cell_type": "code", "execution_count": null, "id": "NVbnvCUq5s_m", "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 200, "referenced_widgets": [ "5781187ea9a740d98dda6f3fbacc8737", "257ab33f9aa54f2591244ee2f6a9c7d7", "51ffaf781dae41c4bb6c65a11210d5ba", "917deda0fee94af18bd7107fb350aa6c", "3d99763d11d44de582b43729f8ce0058", "5697065ca3a8452ea04f7eb3ea425a5c", "66d983c505884401b827b457b2f76adc", "0e5f5f15321c4b0aab696bf95f431bd8", "8531be16189a4f7ca5a697cd22a515e4", "5003c39b9f134d3b806469e0ded2e69f", "119e0bef11194cf3940b6f095c7f72d2" ] }, "id": "NVbnvCUq5s_m", "outputId": "2df67401-77e2-4289-a61a-58a7cfacc200" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/usr/local/lib/python3.11/dist-packages/pytorch_lightning/trainer/connectors/data_connector.py:476: Your `test_dataloader`'s sampler has shuffling enabled, it is strongly recommended that you turn shuffling off for val/test dataloaders.\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "5781187ea9a740d98dda6f3fbacc8737", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Testing: | | 0/? [00:00┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n", "┃ Test metric ┃ DataLoader 0 ┃\n", "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n", "│ test_loss │ 6.802803993225098 │\n", "│ test_mae │ 1.8828424215316772 │\n", "└───────────────────────────┴───────────────────────────┘\n", "\n" ], "text/plain": [ "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n", "┃\u001b[1m \u001b[0m\u001b[1m Test metric \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m DataLoader 0 \u001b[0m\u001b[1m \u001b[0m┃\n", "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n", "│\u001b[36m \u001b[0m\u001b[36m test_loss \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m 6.802803993225098 \u001b[0m\u001b[35m \u001b[0m│\n", "│\u001b[36m \u001b[0m\u001b[36m test_mae \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m 1.8828424215316772 \u001b[0m\u001b[35m \u001b[0m│\n", "└───────────────────────────┴───────────────────────────┘\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [ "[{'test_mae': 1.8828424215316772, 'test_loss': 6.802803993225098}]" ] }, "execution_count": 55, "metadata": {}, "output_type": "execute_result" } ], "source": [ "trainer.test(model, dataloaders=test_dl)" ] }, { "cell_type": "markdown", "id": "1a065716-7bbe-48da-91ce-7572ef22d124", "metadata": { "id": "1a065716-7bbe-48da-91ce-7572ef22d124" }, "source": [ "The MSE on the test set is 6.8 and MAE is 1.88" ] }, { "cell_type": "markdown", "id": "0771e292-d5f0-4cc4-b1fe-e527a293afe8", "metadata": { "id": "0771e292-d5f0-4cc4-b1fe-e527a293afe8" }, "source": [ "## Convolutional neural networks" ] }, { "cell_type": "markdown", "id": "9c5b5a72-4328-420c-8a15-f9b75883f976", "metadata": { "id": "9c5b5a72-4328-420c-8a15-f9b75883f976" }, "source": [ "Convolutional Neural Networks (CNNs) are different from other types of neural networks mainly because they are specifically designed for processing images and similar types of data. They use special layers called convolutional layers to detect patterns such as edges in images. These layers look at small areas of the image at a time, which helps the CNN understand the overall picture better. CNNs also have pooling layers that simplify the information by combining similar features into one, reducing the amount of data the network needs to process. This makes CNNs more efficient and better suited for image-related tasks than regular neural networks. To illustrate the training of CNNs, we are again going to use the MINST digit dataset." ] }, { "cell_type": "code", "execution_count": null, "id": "871d0ec6-ab59-4c04-9cfb-30bc6251aada", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "871d0ec6-ab59-4c04-9cfb-30bc6251aada", "outputId": "1531d459-64f4-4526-f553-4c6d8695a724" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "100%|██████████| 9.91M/9.91M [00:00<00:00, 134MB/s]\n", "100%|██████████| 28.9k/28.9k [00:00<00:00, 28.0MB/s]\n", "100%|██████████| 1.65M/1.65M [00:00<00:00, 86.5MB/s]\n", "100%|██████████| 4.54k/4.54k [00:00<00:00, 5.79MB/s]\n" ] } ], "source": [ "transform = transforms.Compose([transforms.ToTensor()])\n", "\n", "mnist_dataset = datasets.MNIST('./data',\n", " train=True,\n", " transform=transform,\n", " download=True)\n", "\n", "mnist_valid_dataset = Subset(mnist_dataset, torch.arange(10000))\n", "mnist_train_dataset = Subset(mnist_dataset, torch.arange(10000, len(mnist_dataset)))\n", "mnist_test_dataset = datasets.MNIST('./data',\n", " train=False,\n", " transform=transform,\n", " download=True)" ] }, { "cell_type": "markdown", "id": "76341b28-0340-4a80-b8cb-6472f080342a", "metadata": { "id": "76341b28-0340-4a80-b8cb-6472f080342a" }, "source": [ "After loading the dataset just like we did for the fashion dataset, we will pass the training, and validation dataset through the loader, so batches can be created during training and evaluation." ] }, { "cell_type": "code", "execution_count": null, "id": "817f80f2-23c3-447d-a345-98295f66005b", "metadata": { "id": "817f80f2-23c3-447d-a345-98295f66005b" }, "outputs": [], "source": [ "batch_size = 64\n", "torch.manual_seed(1)\n", "train_dl = DataLoader(mnist_train_dataset, batch_size, shuffle=True)\n", "valid_dl = DataLoader(mnist_valid_dataset, batch_size, shuffle=False)\n", "test_dl = DataLoader(mnist_test_dataset, batch_size=batch_size)" ] }, { "cell_type": "markdown", "id": "4d1f1f70-a692-4631-9a9e-5f9ea74da337", "metadata": { "id": "4d1f1f70-a692-4631-9a9e-5f9ea74da337" }, "source": [ "Now, let's again define our network architecture sequentially." ] }, { "cell_type": "code", "execution_count": null, "id": "l_iUqJemRY_k", "metadata": { "id": "l_iUqJemRY_k" }, "outputs": [], "source": [ "class CNNModel(pl.LightningModule):\n", " def __init__(self):\n", " super().__init__()\n", " self.feature_extractor = nn.Sequential(\n", " #Input image size: 28x28 with 1 channel (grayscale).\n", " nn.Conv2d(in_channels=1, out_channels=32, kernel_size=5, padding=2),\n", " # after the previous layer, the output shape becomes (batch_size, 32, 28, 28)\n", " nn.ReLU(),\n", " nn.MaxPool2d(kernel_size=2),\n", " # after max pooling, the spatial dimensions are halved and the output shape becomes (batch_size, 32, 14, 14)\n", "\n", " nn.Conv2d(in_channels=32, out_channels=64, kernel_size=5, padding=2),\n", " # output shape becomes (batch_size, 64, 14, 14)\n", " nn.ReLU(),\n", " nn.MaxPool2d(kernel_size=2)\n", " # spatial dimensions are halved again, and the output shape becomes (batch_size, 64, 7, 7)\n", " )\n", " self.classifier = nn.Sequential(\n", " nn.Flatten(),\n", " nn.Linear(64 * 7 * 7, 1024),\n", " nn.ReLU(),\n", " nn.Dropout(p=0.5),\n", " nn.Linear(1024, 10)\n", " )\n", "\n", " self.loss_fn = nn.CrossEntropyLoss()\n", " self.train_acc = Accuracy(task=\"multiclass\", num_classes=10)\n", " self.val_acc = Accuracy(task=\"multiclass\", num_classes=10)\n", " self.test_acc = Accuracy(task=\"multiclass\", num_classes=10)\n", "\n", " def forward(self, x):\n", " x = self.feature_extractor(x)\n", " x = self.classifier(x)\n", " return x\n", "\n", " def training_step(self, batch, batch_idx):\n", " x, y = batch\n", " logits = self(x)\n", " loss = self.loss_fn(logits, y)\n", " preds = torch.argmax(logits, dim=1)\n", " acc = self.train_acc(preds, y)\n", " self.log(\"train_loss\", loss, prog_bar=True, on_step=False, on_epoch=True)\n", " self.log(\"train_acc\", acc, on_step=False, on_epoch=True)\n", " return loss\n", "\n", "\n", " def validation_step(self, batch, batch_idx):\n", " x, y = batch\n", " logits = self(x)\n", " loss = self.loss_fn(logits, y)\n", " preds = torch.argmax(logits, dim=1)\n", " acc = self.val_acc(preds, y)\n", " self.log(\"val_loss\", loss, on_step=False, on_epoch=True)\n", " self.log(\"val_acc\", acc, on_step=False, on_epoch=True)\n", " return loss\n", "\n", " def test_step(self, batch, batch_idx):\n", " x, y = batch\n", " logits = self(x)\n", " loss = self.loss_fn(logits, y)\n", " preds = torch.argmax(logits, dim=1)\n", " acc = self.test_acc(preds, y)\n", " self.log(\"test_acc\", acc, on_step=False, on_epoch=True)\n", " return loss\n", "\n", " def configure_optimizers(self):\n", " return optim.Adam(self.parameters(), lr=0.001)\n", "\n" ] }, { "cell_type": "code", "execution_count": null, "id": "b715832b-a16d-4df7-92d8-54e289d474de", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "b715832b-a16d-4df7-92d8-54e289d474de", "outputId": "4b10bf61-3116-495f-f956-17fd78817821" }, "outputs": [ { "data": { "text/plain": [ "torch.Size([4, 10])" ] }, "execution_count": 59, "metadata": {}, "output_type": "execute_result" } ], "source": [ " # batch of 4 grayscale images\n", "x = torch.ones((4, 1, 28, 28))\n", "model = CNNModel()\n", "model(x).shape # torch.Size([4, 10]) for 10 classes" ] }, { "cell_type": "code", "execution_count": null, "id": "RmNlG-9eUDsc", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "RmNlG-9eUDsc", "outputId": "0477a425-1295-4fe3-83b9-9db2cd0d8d66" }, "outputs": [ { "data": { "text/plain": [ "CNNModel(\n", " (feature_extractor): Sequential(\n", " (0): Conv2d(1, 32, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2))\n", " (1): ReLU()\n", " (2): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n", " (3): Conv2d(32, 64, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2))\n", " (4): ReLU()\n", " (5): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n", " )\n", " (classifier): Sequential(\n", " (0): Flatten(start_dim=1, end_dim=-1)\n", " (1): Linear(in_features=3136, out_features=1024, bias=True)\n", " (2): ReLU()\n", " (3): Dropout(p=0.5, inplace=False)\n", " (4): Linear(in_features=1024, out_features=10, bias=True)\n", " )\n", " (loss_fn): CrossEntropyLoss()\n", " (train_acc): MulticlassAccuracy()\n", " (val_acc): MulticlassAccuracy()\n", " (test_acc): MulticlassAccuracy()\n", ")" ] }, "execution_count": 60, "metadata": {}, "output_type": "execute_result" } ], "source": [ "model" ] }, { "cell_type": "markdown", "id": "1aaf3e9e-50f1-4c02-9060-2b09f80f7a51", "metadata": { "id": "1aaf3e9e-50f1-4c02-9060-2b09f80f7a51" }, "source": [ "\n", "The first block, inside feature extractor:\n", "- Our first layer is Conv2d layers. This is the convolution layer that will deal with our input images, which are seen as 2-dimensional matrices. For grayscale images, like we have here, there is only one color channel, so the number of input channels is 1. The number *32* in the first convolutional layer is the number of filters in the layer. This number can be adjusted to be higher or lower, depending on the size of the dataset. With `out_channels=32` it produces an output with 32 feature maps. Each of the 32 output channels will contain the result of convolving one of the filters with the input. Kernel size is the size of the filter matrix for our convolution. So a kernel size of 5 means we will have a 5x5 filter matrix. The activation function we will be using for our first layer is the ReLU, or Rectified Linear Activation. The parameter *padding* refers to the adding zeros to an image to obtain a specific dimensions of the output. Here, padding=2 means that two rows of zeros will be added to the top and bottom, and two columns of zeros will be added to the left and right sides of the input tensor.\n", "- Next, we have a max pooling layer which uses a kernel size of 2, and this parameter specifies the size of the window over which to take the maximum. In this case, a 2x2 window is used. For each 2x2 area in the input feature map, the maximum value is selected and forms the output feature map. This effectively reduces the height and width of the feature map by a factor of 2.\n", "- Next we have another convolution later with a ReLu function, and this time it has 32 input channels, since the previous convolutional layer produces 32 output channels.\n", "- Then we have another pooling layer." ] }, { "cell_type": "markdown", "id": "d8cd7aa5-f1e6-4a0e-9921-e6790081d861", "metadata": { "id": "d8cd7aa5-f1e6-4a0e-9921-e6790081d861" }, "source": [ "The next block that we want to add is a classifier that consists of two fully connected layers with a dropout layer in between. The input to this block must have a shape[batch_size × input_units]. Thus, we need to flatten the output of the previous layers to meet this requirement for the first fully connected layer. The last fully connected layer has 10 output units for the 10 class labels in the MNIST dataset." ] }, { "cell_type": "markdown", "id": "3e705b80-52db-4d90-b10b-f55a49537b5e", "metadata": { "id": "3e705b80-52db-4d90-b10b-f55a49537b5e" }, "source": [ "The last fully connected layer, named 'fc2', has 10 output units for the 10 class labels in the MNIST dataset which we pass through the sofmax activation to obtain the class-membership probabilities of each input example." ] }, { "cell_type": "markdown", "id": "9576626c-5dd7-4eb4-ab27-f255f94d4653", "metadata": { "id": "9576626c-5dd7-4eb4-ab27-f255f94d4653" }, "source": [ "Note that we have a dropout layer between the two fully connected layers. Dropout layer is one of most commonly used regularization techniques for neural networks. The intuitive explanation for dropout is that because individual nodes in the network cannot rely on the output of the others, each node must output features that are useful on their own. The \"dropout rate\" is the fraction of the elements that are being zeroed-out; it is usually set between 0.2 and 0.5. This values is also another hyperparameter that can be tuned. The zeroed elements are chosen independently for each forward call and are sampled from a Bernoulli distribution. Note that this zeroing out only happens during training not inference. The settings for training model.train() and evaluation model.eval() will automatically set the mode for the dropout layer." ] }, { "cell_type": "code", "execution_count": null, "id": "k1xM5GFIXKO2", "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 433, "referenced_widgets": [ "309453e78b0b409f839117ba156d7e3a", "1090c5c6bc654c6f9cdc0f72e33b85b3", "decba3296d474b8282f988401495487f", "f895a99a7f574d8f9c97b60dcbadf497", "b8b9dab360ae4dbf990ebd3053f45794", "c936e1097b1245e6a4a325a472133874", "40852b870a4649a186fe0c71cad0a234", "61c5c53594ad47e2adafa99dcd0e35ae", "e4151182f97341d5a4e89803082573f1", "5ae6d72b5c77491fb79d2432bedc2851", "3ae8014f7d6b4bbe92b00ebeb0675c09", "ba8ea911e487489e8b02ebbdb070f328", "8d95d25d373946ecba61a0ccbe0669c2", "4d2b5f4881924960b3cd760d13419998", "b109c57180354eb19f3cd789b0850693", "9f2d589d84074072b23bed0d11d5e88d", "052bf7dce8c64f74afde401d6db3bcbd", "2c16d456a401438d80dafbba28deb9da", "ccffac78d24b4b9e8566d7dfca09cbc0", "6eb08bee2fb44dce91845beea5046dda", "afa4e9f3fea84e4eae0c77cabf930835", "342e8b4b20774be9a2fa68eaa0921bbe", "d9e313817f2c4f48a868abe60a834cb7", "848a4beda86441c98ca874b076d3ed6e", "477b20e171b744518f2691d343188714", "2443930de8284148a7ed0ff64cab9b75", "3c162e4b29c84f5ca2a3243f35d59641", "a22df4d448a341ccab59fe1150b4f2e8", "17a0a9e061ab4525b90ef33ac3d4b72f", "a4438910cc994c67b39f2888afdecc4c", "71f46192712f4bacb9ecb4aeb49b2bed", "8804fff7f53540299db7371d1a65363b", "9bd6317622fb488db10c34d028559fd9", "7c4db9a587b44dc1899704ae84f5af71", "3e2b4e7404af40b7afb610cca201e71c", "49e3239d4f104f92af2d00d01f0a76e7", "79a2d947cfaf4fcb9743cd77692301d1", "c4f3642f668d428ab641ad442478fc9b", "c44a38fdfc5246668f1e2ef9a1281747", "4c7877887f8542f0910796a8bd913a5f", "94cfd4cd1bff47aab343b287f74a2d55", "a3f8d90bf76b48259c4f7583af4c578e", "429607e5cad94841902812a8f67e0937", "71ce69a2659542e5a94c6f5d4096f76b", "31b28cd4fae74a2b97bad887e3eb80a0", "728aae7b6ea6444a8f6b0d90ef226227", "4b5ebb81686c48f9a73c499ca760f363", "4839d58911154c8f837977abe16d46d2", "57d415032ed943849994248d7b11805a", "4dbb2d40666045a9939db57b5c35904b", "b7d52b33551a4b529e6f5d6f60163377", "b4abb63a4f8b49419c1c7bb0ebe2fcb2", "63d8c4fbd2c949369ef7c0a8dc521262", "ab3c4c6b11ef4a9ea4e0ae95fbc3fcd7", "7595b08f182748c1bf291bb7e87a4026", "2c7efd8bd13b46348cb47cf6ac17b2e1", "6278c96f595540908667778e749dea0e", "7e812e59e93046e587395a74553d64c9", "2ff41ac3750b46d8a8fecbbaeaa5a686", "e1dc62bffeaf433f919405f27a5de450", "2f5794667f664d28b826ca1db413b843", "a5f611dbbe774f18a7abd07a06e7d1aa", "9500a787c0c14ecbb0efc701cca7ccb7", "654774ba937040d78ea466ae64689edd", "026899f8dfd540eba8b3808137d633ae", "e0f92f8764984229856f8f40cd3c3ebc", "026f844cf3cf4753b2995b1ccfa9d701", "3e1ee0c4b88e43c4b159a228492f19bf", "11b4e8f5d295479dba05dd4892d48909", "2655dda6711644e7a2d2211d6ea15fee", "d581b3cb4f5c4932be8862be509d97f9", "613e7eb36f6f457d8c95a6754d74a8c7", "22c761c949834cbcaa19ccbfde70fbb3", "e90c40de39c84c57bba7213db55188df", "2d7ca0e3646a458a89fac34134580e0c", "8746ad0c90964b3f93af266a36938393", "030f1eec8eb94fa3856885a0a0391a7b", "c5ce3129674d402ba2a5f1a2944ac78d", "89a6ad22340345f090543af2d990df2d", "486351e0ff0f42e9836dd953f39a9d63", "07b85b31f0ee4c568618b9e9fbb4c929", "40e69c44b12841d18b93d292a7a17b4f", "251b15cd84f7403a9343bee611369f6d", "aa7b28492f7545f18d20315fa4e74035", "c2482b58f84e4ce3aa69990ae5b88783", "7f43dfbaeb66467690691f944a5ef1e1", "543aeec44aa5464e976988c64da40a6e", "ffb0411f39e64b8fb6506e650eca2a6d", "36949c9ea04348cab86fec20876876f9", "6c696e3e8973465a94241c198ba451d6", "94562514188d4d00b185a9f1074f92bd", "81d35d9ef9d840bfae39787980ae162d", "4d89064ba55b4a3b9bf9d53b15c00502", "e3b6051d3aaa44e48a862a6d24a4dc6e", "8755ded3a6e44494b9bbf91079e839c3", "5d6adabf50e047d3897a5265248953b4", "4a6d32c082cc482c8f4c4991ef7cc5e2", "4b04920b8f784bb489fd352eecf74faa", "26699a4bafba4da486e03e3f673c73c5", "ebe15f8117e142ac9004ab18241ffeea", "812135e52bd74893b94e553f5bd30ece", "053fdebd6262410aa47e339b85e4cc7d", "724cd131a4d640bd8b0862cfa68a449b", "28af3c2c45dd4554949ee2099aab336c", "51c98590ad9a43a9980dc3596b3fd9d6", "c3bde68c577d4655b959472f0d6d607b", "4fff93b748fe4c0cb5276b6bae2a1bda", "ceaea24c3f6e4507b304441b6771f4d3", "2007c0564fbb4758899075f09181ae3b", "747d0e4538984d99a5509dc40bf88199", "729b1ab41a204bfaa94912320251ddaa", "8e4608ea6244450caa4a5989b1b90ab0", "526e3d1b822347e4969898926dccf66a", "79efec810775459da051e56249e197dc", "43f412f3c8224f7b8a7aa666ffeb5da3", "68a00d93cec5412487ff7151e697d2fa", "b69801b7120b4186b9e27cc8904173bd", "64e81873af0449b99c9417fcaf68175e", "6a285b553a1849c7a5414882fd1e011e", "803d0c75e1e44892a9e2cbb63211ca5f", "20032203372344d59e59f030eac46f52", "9ae8702030a7443db132d95c835a80d7", "c148e7eb982d428fa4d2a8c29564e73b", "a95ecb0d965d484cbf04e3a36399f51b", "ed28332348ee4148bccdee516777b94c", "63c614b65ba34979bdceb203598eabac", "306c26e12ef2438ead540532792ed165", "a63ee1b22f964f9991bb7698ab8bd135", "e84e6b84e77a44bcb8e215ab862871ee", "61fe68e00f8d40baa2dccd882b725f21", "17cf42bb46984789a70568f2306e7001", "5aba43fc321d49399f868fecdb0da9a1" ] }, "id": "k1xM5GFIXKO2", "outputId": "39df4f58-8f9e-4452-fda8-8fba43c6b677" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "INFO:pytorch_lightning.utilities.rank_zero:You are using the plain ModelCheckpoint callback. Consider using LitModelCheckpoint which with seamless uploading to Model registry.\n", "INFO:pytorch_lightning.utilities.rank_zero:GPU available: False, used: False\n", "INFO:pytorch_lightning.utilities.rank_zero:TPU available: False, using: 0 TPU cores\n", "INFO:pytorch_lightning.utilities.rank_zero:HPU available: False, using: 0 HPUs\n", "INFO:pytorch_lightning.callbacks.model_summary:\n", " | Name | Type | Params | Mode \n", "-----------------------------------------------------------------\n", "0 | feature_extractor | Sequential | 52.1 K | train\n", "1 | classifier | Sequential | 3.2 M | train\n", "2 | loss_fn | CrossEntropyLoss | 0 | train\n", "3 | train_acc | MulticlassAccuracy | 0 | train\n", "4 | val_acc | MulticlassAccuracy | 0 | train\n", "5 | test_acc | MulticlassAccuracy | 0 | train\n", "-----------------------------------------------------------------\n", "3.3 M Trainable params\n", "0 Non-trainable params\n", "3.3 M Total params\n", "13.099 Total estimated model params size (MB)\n", "17 Modules in train mode\n", "0 Modules in eval mode\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "309453e78b0b409f839117ba156d7e3a", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Sanity Checking: | | 0/? [00:00" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plot_performance('cnn-logs')" ] }, { "cell_type": "code", "execution_count": null, "id": "JF613wCDYaJV", "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 146, "referenced_widgets": [ "bc3767f26b4e4f9ebcacd7b3eec922f3", "4406a71e0da14d659937b464de933045", "962b2e4e1a924b9fa9eef589de15aafc", "5a254bebeb2d41f3befb6ef0efc3ed7a", "fd6c43dad70e4faf818237acaa7dc2c8", "e48b05e26b2a4befa93a908f3e7b7947", "d9b8f3663bc340639c61a44f2f038eb5", "9ff8986a1fe144579c8567a4e6399f83", "a23a5c904b5541fda21c99315a891782", "099d93675b344a448790fd638e8cb5c5", "d887a20ea9fd4e0b87c97c5d4bd6484e" ] }, "id": "JF613wCDYaJV", "outputId": "ded702b9-dc06-4cbb-d6ed-a7ad7f79d5cb" }, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "bc3767f26b4e4f9ebcacd7b3eec922f3", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Testing: | | 0/? [00:00┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n", "┃ Test metric ┃ DataLoader 0 ┃\n", "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n", "│ test_acc │ 0.9933000206947327 │\n", "└───────────────────────────┴───────────────────────────┘\n", "\n" ], "text/plain": [ "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n", "┃\u001b[1m \u001b[0m\u001b[1m Test metric \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m DataLoader 0 \u001b[0m\u001b[1m \u001b[0m┃\n", "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n", "│\u001b[36m \u001b[0m\u001b[36m test_acc \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m 0.9933000206947327 \u001b[0m\u001b[35m \u001b[0m│\n", "└───────────────────────────┴───────────────────────────┘\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [ "[{'test_acc': 0.9933000206947327}]" ] }, "execution_count": 63, "metadata": {}, "output_type": "execute_result" } ], "source": [ "trainer.test(model, dataloaders=test_dl)" ] }, { "cell_type": "markdown", "id": "e256458a-a472-48a5-a0ab-5e847ae775c5", "metadata": { "id": "e256458a-a472-48a5-a0ab-5e847ae775c5" }, "source": [ "The CNN model achieves an accuracy of 99.3 percent." ] }, { "cell_type": "markdown", "id": "dff02c8e-4f48-47ad-8691-8a5b7516f0b5", "metadata": { "id": "dff02c8e-4f48-47ad-8691-8a5b7516f0b5" }, "source": [ "## Hyperparameter tuning" ] }, { "cell_type": "markdown", "id": "b0a27f1e-0fcd-4071-9cf9-c729e2cf21c5", "metadata": { "id": "b0a27f1e-0fcd-4071-9cf9-c729e2cf21c5" }, "source": [ "As we saw previously, hyperparameter tuning can make the difference between an average model and a highly accurate one. Often simple things like choosing a different learning rate or changing a network layer size can have a dramatic impact on the model performance.\n", "Fortunately, there are tools that help with finding the best combination of parameters, and Ray Tune is one of them. We can install this library with:\n", "\n", "`pip install \"ray[tune]\"`" ] }, { "cell_type": "markdown", "id": "de96787c", "metadata": { "id": "de96787c" }, "source": [ "Note that Ray is only supported for python <=3.11. If you have python 3.12, run this notebook only on Colab, or make a new environment with python 3.11." ] }, { "cell_type": "code", "execution_count": null, "id": "2JvE-szbhB3n", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "2JvE-szbhB3n", "outputId": "eb469dc3-daf9-4896-b94c-e922170a1823" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m101.7/101.7 kB\u001b[0m \u001b[31m7.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m68.1/68.1 MB\u001b[0m \u001b[31m8.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25h" ] } ], "source": [ "!pip install -q \"ray[tune]\"" ] }, { "cell_type": "code", "execution_count": null, "id": "sLJTO7e76_0i", "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 76 }, "id": "sLJTO7e76_0i", "outputId": "af1ca6df-e97c-483e-8afd-67d600882e8d" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2025-04-10 16:34:09,402\tINFO worker.py:1852 -- Started a local Ray instance.\n" ] }, { "data": { "text/html": [ "
\n", "
\n", "
\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Python version:3.11.11
Ray version:2.44.1
\n", "\n", "
\n", "
\n" ], "text/plain": [ "RayContext(dashboard_url='', python_version='3.11.11', ray_version='2.44.1', ray_commit='daca7b2b1a950dc7f731e34e74c76ae383794ffe')" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import ray\n", "from ray import tune\n", "from ray import cluster_resources\n", "from ray.tune import Tuner, TuneConfig, RunConfig, CheckpointConfig\n", "from ray.tune.schedulers import ASHAScheduler\n", "from ray.tune.integration.pytorch_lightning import TuneReportCallback, TuneReportCheckpointCallback\n", "ray.init()" ] }, { "cell_type": "markdown", "id": "efd49c0d-8b87-4746-8d98-58d146b6cf10", "metadata": { "id": "efd49c0d-8b87-4746-8d98-58d146b6cf10" }, "source": [ "To illustrate hyperparameter tuning process, we will again use MNIST fashion dataset and tune the number of neurons in the hidden layer as well as the value of the learning rate in our optimizer and the batch size.\n", "We will again define the structure of our neural network, but this time we define the number of neurons in the hidden layer as a parameter `hidden_size`, and the learning rate as a parameter `lr`." ] }, { "cell_type": "code", "execution_count": null, "id": "KSWVN0kia4c7", "metadata": { "id": "KSWVN0kia4c7" }, "outputs": [], "source": [ "class MNISTModel(pl.LightningModule):\n", " def __init__(self, hidden_size=16, lr=1e-3):\n", " super().__init__()\n", " self.save_hyperparameters()\n", " self.model = nn.Sequential(\n", " nn.Flatten(),\n", " nn.Linear(28*28, self.hparams.hidden_size),\n", " nn.ReLU(),\n", " nn.Linear(self.hparams.hidden_size, 10)\n", " )\n", " self.loss_fn = nn.CrossEntropyLoss()\n", " self.train_acc = Accuracy(task=\"multiclass\", num_classes=10)\n", " self.val_acc = Accuracy(task=\"multiclass\", num_classes=10)\n", " self.test_acc = Accuracy(task=\"multiclass\", num_classes=10)\n", "\n", "\n", " def forward(self, x):\n", " return self.model(x)\n", "\n", " def training_step(self, batch, batch_idx):\n", " x, y = batch\n", " logits = self(x)\n", " loss = self.loss_fn(logits, y)\n", " preds = torch.argmax(logits, dim=1)\n", " acc = self.train_acc(preds, y)\n", " self.log(\"train_loss\", loss, prog_bar=True, on_step=False, on_epoch=True)\n", " self.log(\"train_acc\", acc, on_step=False, on_epoch=True)\n", " return loss\n", "\n", "\n", " def validation_step(self, batch, batch_idx):\n", " x, y = batch\n", " logits = self(x)\n", " loss = self.loss_fn(logits, y)\n", " preds = torch.argmax(logits, dim=1)\n", " acc = self.val_acc(preds, y)\n", " self.log(\"val_loss\", loss, on_step=False, on_epoch=True)\n", " self.log(\"val_acc\", acc, on_step=False, on_epoch=True)\n", " return loss\n", "\n", " def test_step(self, batch, batch_idx):\n", " x, y = batch\n", " logits = self(x)\n", " loss = self.loss_fn(logits, y)\n", " preds = torch.argmax(logits, dim=1)\n", " acc = self.test_acc(preds, y)\n", " self.log(\"test_acc\", acc, on_step=False, on_epoch=True)\n", " return loss\n", "\n", " def configure_optimizers(self):\n", " return optim.Adam(self.parameters(), lr=self.hparams.lr)" ] }, { "cell_type": "markdown", "id": "hraR8Ggzcs4i", "metadata": { "id": "hraR8Ggzcs4i" }, "source": [ "Now, we will create data module to include the code for getting the dataset, but we will set batch size as a parameter that we will later vary." ] }, { "cell_type": "code", "execution_count": null, "id": "38Iduqm8ctRm", "metadata": { "id": "38Iduqm8ctRm" }, "outputs": [], "source": [ "def create_dataloaders(batch_size):\n", " transform = transforms.Compose(\n", " [transforms.ToTensor()])\n", " training_set = datasets.FashionMNIST('./data', train=True, transform=transform, download=True)\n", "\n", "\n", " train_set = Subset(training_set, torch.arange(48000))\n", " valid_set = Subset(training_set, torch.arange(48000, len(training_set)))\n", "\n", " train_loader= DataLoader(train_set, batch_size = batch_size, shuffle=True, num_workers=4, persistent_workers=True)\n", " val_loader = DataLoader(valid_set, batch_size=batch_size, num_workers=4, persistent_workers=True)\n", "\n", " return train_loader, val_loader" ] }, { "cell_type": "markdown", "id": "_DDr5_YhbYEx", "metadata": { "id": "_DDr5_YhbYEx" }, "source": [ "Next, we need to define search space for these three parameters with a dictionary:" ] }, { "cell_type": "code", "execution_count": null, "id": "b09c97c4-7599-4751-a3da-1fa4a5e2dac2", "metadata": { "id": "b09c97c4-7599-4751-a3da-1fa4a5e2dac2" }, "outputs": [], "source": [ "search_space = {\n", " \"hidden_size\": tune.choice([2 ** i for i in range(4, 9)]),\n", " \"lr\": tune.loguniform(1e-4, 1e-1),\n", " \"batch_size\": tune.choice([8, 16, 64])\n", "}" ] }, { "cell_type": "markdown", "id": "vOFbl6l_jf2L", "metadata": { "id": "vOFbl6l_jf2L" }, "source": [ "The *tune.choice()* accepts a list of values that are uniformly sampled from defined ranges. In this example, the *n1* parameter should be a power of 2 between 4 and 8, so either 16, 32, 64, 128, or 256. The *lr* (learning rate) should be uniformly sampled between 0.0001 and 0.1. Lastly, the batch size is a choice between 8, 16 and 64. At each trial, Ray Tune will now randomly sample a combination of parameters from these search spaces, and the number of times it samples is defined by a parameter *num_samples* which we will set later. By default, tune automatically runs N concurrent trials, where N is the number of CPUs (cores) on the machine. If we can afford longer times, we can increase the number of samples." ] }, { "cell_type": "markdown", "id": "T0pYcbDwjgf-", "metadata": { "id": "T0pYcbDwjgf-" }, "source": [ "Now, we will define ray tune training function." ] }, { "cell_type": "code", "execution_count": null, "id": "8SIyVpQAjqWE", "metadata": { "id": "8SIyVpQAjqWE" }, "outputs": [], "source": [ "def train_model(config):\n", " train_loader, val_loader = create_dataloaders(config[\"batch_size\"])\n", "\n", " model = MNISTModel(hidden_size=config[\"hidden_size\"], lr=config[\"lr\"])\n", " trainer = pl.Trainer(\n", " max_epochs=10,\n", " logger=True,\n", " enable_progress_bar=False,\n", " accelerator=\"auto\",\n", " callbacks=[\n", " TuneReportCheckpointCallback({\"val_acc\": \"val_acc\"}, filename=\"tune-checkpoint.ckpt\", on=\"validation_epoch_end\")]\n", " )\n", " trainer.fit(model, train_loader, val_loader)" ] }, { "cell_type": "markdown", "id": "wjqUd_G1kSUe", "metadata": { "id": "wjqUd_G1kSUe" }, "source": [ "With parameter `on=\"validation_epoch_end\"`, we specify that at the end of each validation epoch, the logged value of val_acc is reported back to ray tune for progress tracking.\n", "\n", "Additionally, we will use the ASHAScheduler which will terminate bad performing trials early, based on the metric we specify. Ray Tune will monitor this metric to decide which trials (individual runs of a model with a specific set of hyperparameters) to stop early. Here we will use validation accuracy as our metric of interest. `mode` max tells the scheduler that the goal is to maximize the metric of interest. `max_t` defines the maximum number of epochs that any trial is allowed to run. `grace_period` controls how long to wait before the scheduler starts stopping trials.\n", "`reduction_factor` determines how aggressively the scheduler prunes less promising trials. A reduction_factor of 2 means that at each pruning step, the scheduler will keep only the top 50% of trials (those with the best performance on the metric being optimized) and stop the rest. This effectively reduces the number of active trials by half at each pruning step." ] }, { "cell_type": "code", "execution_count": null, "id": "Gj1LLLtIlYW4", "metadata": { "id": "Gj1LLLtIlYW4" }, "outputs": [], "source": [ "scheduler = ASHAScheduler(\n", " metric=\"val_acc\",\n", " mode=\"max\",\n", " max_t=3,\n", " grace_period=2,\n", " reduction_factor=2 )" ] }, { "cell_type": "markdown", "id": "Cxr-bniokxr7", "metadata": { "id": "Cxr-bniokxr7" }, "source": [ "Check do we have GPUs available" ] }, { "cell_type": "code", "execution_count": null, "id": "BGeSSaDPob_4", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "BGeSSaDPob_4", "outputId": "ae1bab5e-9756-413c-fba5-c68c0fc899ee" }, "outputs": [ { "data": { "text/plain": [ "{'CPU': 2.0,\n", " 'node:__internal_head__': 1.0,\n", " 'node:172.28.0.12': 1.0,\n", " 'object_store_memory': 3987909427.0,\n", " 'memory': 9305121997.0}" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "\n", "available_resources = cluster_resources()\n", "available_resources" ] }, { "cell_type": "code", "execution_count": null, "id": "eSHD5rqHkfD6", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "eSHD5rqHkfD6", "outputId": "3b64dd04-8e0d-4cf1-a550-faa26c64ebea" }, "outputs": [ { "data": { "text/plain": [ "{'cpu': 1, 'gpu': 0.0}" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "use_gpus = (available_resources.get(\"GPU\", 0)/2)\n", "resources_per_trial = {\"cpu\": 1, \"gpu\": use_gpus }\n", "trainable_with_resources = tune.with_resources(train_model, resources_per_trial)\n", "resources_per_trial" ] }, { "cell_type": "code", "execution_count": null, "id": "J0EhI9NwkSj-", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "J0EhI9NwkSj-", "outputId": "1c3dc18d-beff-4231-c7a5-1535323d632a" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "+--------------------------------------------------------------------+\n", "| Configuration for experiment train_model_2025-04-10_16-59-56 |\n", "+--------------------------------------------------------------------+\n", "| Search algorithm BasicVariantGenerator |\n", "| Scheduler FIFOScheduler |\n", "| Number of trials 10 |\n", "+--------------------------------------------------------------------+\n", "\n", "View detailed results here: /root/ray_results/train_model_2025-04-10_16-59-56\n", "To visualize your results with TensorBoard, run: `tensorboard --logdir /tmp/ray/session_2025-04-10_16-34-02_911899_547/artifacts/2025-04-10_16-59-56/train_model_2025-04-10_16-59-56/driver_artifacts`\n", "\n", "Trial status: 10 PENDING\n", "Current time: 2025-04-10 16:59:57. Total running time: 0s\n", "Logical resource usage: 0/2 CPUs, 0/0 GPUs\n", "+---------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size |\n", "+---------------------------------------------------------------------------------+\n", "| train_model_3b16c_00000 PENDING 256 0.00078507 16 |\n", "| train_model_3b16c_00001 PENDING 16 0.00371585 64 |\n", "| train_model_3b16c_00002 PENDING 256 0.0238983 64 |\n", "| train_model_3b16c_00003 PENDING 64 0.00639545 8 |\n", "| train_model_3b16c_00004 PENDING 32 0.0721965 16 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------+\n", "\n", "Trial train_model_3b16c_00000 started with configuration:\n", "+--------------------------------------------------+\n", "| Trial train_model_3b16c_00000 config |\n", "+--------------------------------------------------+\n", "| batch_size 16 |\n", "| hidden_size 256 |\n", "| lr 0.00079 |\n", "+--------------------------------------------------+\n", "\n", "Trial train_model_3b16c_00001 started with configuration:\n", "+--------------------------------------------------+\n", "| Trial train_model_3b16c_00001 config |\n", "+--------------------------------------------------+\n", "| batch_size 64 |\n", "| hidden_size 16 |\n", "| lr 0.00372 |\n", "+--------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ " 0%| | 0.00/26.4M [00:00= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=8317)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00001_1_batch_size=64,hidden_size=16,lr=0.0037_2025-04-10_16-59-56/checkpoint_000002)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 2 RUNNING | 8 PENDING\n", "Current time: 2025-04-10 17:01:57. Total running time: 2min 0s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00001 with val_acc=0.8462499976158142 and params={'hidden_size': 16, 'lr': 0.0037158515476409053, 'batch_size': 64}\n", "+-------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+-------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00000 RUNNING 256 0.00078507 16 |\n", "| train_model_3b16c_00001 RUNNING 16 0.00371585 64 3 85.1126 0.84625 |\n", "| train_model_3b16c_00002 PENDING 256 0.0238983 64 |\n", "| train_model_3b16c_00003 PENDING 64 0.00639545 8 |\n", "| train_model_3b16c_00004 PENDING 32 0.0721965 16 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+-------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=8317)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00001_1_batch_size=64,hidden_size=16,lr=0.0037_2025-04-10_16-59-56/checkpoint_000003)\n", "\u001b[36m(train_model pid=8318)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00000_0_batch_size=16,hidden_size=256,lr=0.0008_2025-04-10_16-59-56/checkpoint_000000)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 2 RUNNING | 8 PENDING\n", "Current time: 2025-04-10 17:02:27. Total running time: 2min 30s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00001 with val_acc=0.8504166603088379 and params={'hidden_size': 16, 'lr': 0.0037158515476409053, 'batch_size': 64}\n", "+-------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+-------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00000 RUNNING 256 0.00078507 16 1 126.852 0.846833 |\n", "| train_model_3b16c_00001 RUNNING 16 0.00371585 64 4 110.256 0.850417 |\n", "| train_model_3b16c_00002 PENDING 256 0.0238983 64 |\n", "| train_model_3b16c_00003 PENDING 64 0.00639545 8 |\n", "| train_model_3b16c_00004 PENDING 32 0.0721965 16 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+-------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=8317)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00001_1_batch_size=64,hidden_size=16,lr=0.0037_2025-04-10_16-59-56/checkpoint_000004)\n", "\u001b[36m(train_model pid=8317)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00001_1_batch_size=64,hidden_size=16,lr=0.0037_2025-04-10_16-59-56/checkpoint_000005)\n", "2025-04-10 17:02:55,723\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 2 RUNNING | 8 PENDING\n", "Current time: 2025-04-10 17:02:57. Total running time: 3min 0s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00001 with val_acc=0.8539999723434448 and params={'hidden_size': 16, 'lr': 0.0037158515476409053, 'batch_size': 64}\n", "+-------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+-------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00000 RUNNING 256 0.00078507 16 1 126.852 0.846833 |\n", "| train_model_3b16c_00001 RUNNING 16 0.00371585 64 6 157.316 0.854 |\n", "| train_model_3b16c_00002 PENDING 256 0.0238983 64 |\n", "| train_model_3b16c_00003 PENDING 64 0.00639545 8 |\n", "| train_model_3b16c_00004 PENDING 32 0.0721965 16 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+-------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025-04-10 17:03:18,368\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=8317)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00001_1_batch_size=64,hidden_size=16,lr=0.0037_2025-04-10_16-59-56/checkpoint_000006)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 2 RUNNING | 8 PENDING\n", "Current time: 2025-04-10 17:03:27. Total running time: 3min 30s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.846833348274231 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+-------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+-------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00000 RUNNING 256 0.00078507 16 1 126.852 0.846833 |\n", "| train_model_3b16c_00001 RUNNING 16 0.00371585 64 7 179.949 0.841833 |\n", "| train_model_3b16c_00002 PENDING 256 0.0238983 64 |\n", "| train_model_3b16c_00003 PENDING 64 0.00639545 8 |\n", "| train_model_3b16c_00004 PENDING 32 0.0721965 16 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+-------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025-04-10 17:03:43,155\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=8317)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00001_1_batch_size=64,hidden_size=16,lr=0.0037_2025-04-10_16-59-56/checkpoint_000007)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 2 RUNNING | 8 PENDING\n", "Current time: 2025-04-10 17:03:57. Total running time: 4min 0s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00001 with val_acc=0.8508333563804626 and params={'hidden_size': 16, 'lr': 0.0037158515476409053, 'batch_size': 64}\n", "+-------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+-------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00000 RUNNING 256 0.00078507 16 1 126.852 0.846833 |\n", "| train_model_3b16c_00001 RUNNING 16 0.00371585 64 8 204.723 0.850833 |\n", "| train_model_3b16c_00002 PENDING 256 0.0238983 64 |\n", "| train_model_3b16c_00003 PENDING 64 0.00639545 8 |\n", "| train_model_3b16c_00004 PENDING 32 0.0721965 16 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+-------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025-04-10 17:04:06,885\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=8317)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00001_1_batch_size=64,hidden_size=16,lr=0.0037_2025-04-10_16-59-56/checkpoint_000008)\n", "\u001b[36m(train_model pid=8318)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00000_0_batch_size=16,hidden_size=256,lr=0.0008_2025-04-10_16-59-56/checkpoint_000001)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 2 RUNNING | 8 PENDING\n", "Current time: 2025-04-10 17:04:27. Total running time: 4min 30s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8665833473205566 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+-------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+-------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00000 RUNNING 256 0.00078507 16 2 236.093 0.866583 |\n", "| train_model_3b16c_00001 RUNNING 16 0.00371585 64 9 228.428 0.857833 |\n", "| train_model_3b16c_00002 PENDING 256 0.0238983 64 |\n", "| train_model_3b16c_00003 PENDING 64 0.00639545 8 |\n", "| train_model_3b16c_00004 PENDING 32 0.0721965 16 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+-------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=8317)\u001b[0m `Trainer.fit` stopped: `max_epochs=10` reached.\n", "\u001b[36m(train_model pid=8317)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00001_1_batch_size=64,hidden_size=16,lr=0.0037_2025-04-10_16-59-56/checkpoint_000009)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "Trial train_model_3b16c_00001 completed after 10 iterations at 2025-04-10 17:04:31. Total running time: 4min 34s\n", "+------------------------------------------------------------+\n", "| Trial train_model_3b16c_00001 result |\n", "+------------------------------------------------------------+\n", "| checkpoint_dir_name checkpoint_000009 |\n", "| time_this_iter_s 24.28758 |\n", "| time_total_s 252.71603 |\n", "| training_iteration 10 |\n", "| val_acc 0.85117 |\n", "+------------------------------------------------------------+\n", "\n", "Trial train_model_3b16c_00002 started with configuration:\n", "+-------------------------------------------------+\n", "| Trial train_model_3b16c_00002 config |\n", "+-------------------------------------------------+\n", "| batch_size 64 |\n", "| hidden_size 256 |\n", "| lr 0.0239 |\n", "+-------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=9613)\u001b[0m \r", " 0%| | 0.00/26.4M [00:00= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=9613)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00002_2_batch_size=64,hidden_size=256,lr=0.0239_2025-04-10_16-59-56/checkpoint_000000)\n", "\u001b[36m(train_model pid=8318)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00000_0_batch_size=16,hidden_size=256,lr=0.0008_2025-04-10_16-59-56/checkpoint_000002)\n", "2025-04-10 17:05:45,317\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 2 RUNNING | 1 TERMINATED | 7 PENDING\n", "Current time: 2025-04-10 17:05:57. Total running time: 6min 0s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8817499876022339 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00000 RUNNING 256 0.00078507 16 3 327.211 0.88175 |\n", "| train_model_3b16c_00002 RUNNING 256 0.0238983 64 1 42.7755 0.83125 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00003 PENDING 64 0.00639545 8 |\n", "| train_model_3b16c_00004 PENDING 32 0.0721965 16 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=9613)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00002_2_batch_size=64,hidden_size=256,lr=0.0239_2025-04-10_16-59-56/checkpoint_000001)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 2 RUNNING | 1 TERMINATED | 7 PENDING\n", "Current time: 2025-04-10 17:06:27. Total running time: 6min 30s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8817499876022339 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00000 RUNNING 256 0.00078507 16 3 327.211 0.88175 |\n", "| train_model_3b16c_00002 RUNNING 256 0.0238983 64 2 73.4361 0.8025 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00003 PENDING 64 0.00639545 8 |\n", "| train_model_3b16c_00004 PENDING 32 0.0721965 16 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=9613)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00002_2_batch_size=64,hidden_size=256,lr=0.0239_2025-04-10_16-59-56/checkpoint_000002)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 2 RUNNING | 1 TERMINATED | 7 PENDING\n", "Current time: 2025-04-10 17:06:57. Total running time: 7min 0s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8817499876022339 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00000 RUNNING 256 0.00078507 16 3 327.211 0.88175 |\n", "| train_model_3b16c_00002 RUNNING 256 0.0238983 64 3 102.94 0.8405 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00003 PENDING 64 0.00639545 8 |\n", "| train_model_3b16c_00004 PENDING 32 0.0721965 16 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=9613)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00002_2_batch_size=64,hidden_size=256,lr=0.0239_2025-04-10_16-59-56/checkpoint_000003)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 2 RUNNING | 1 TERMINATED | 7 PENDING\n", "Current time: 2025-04-10 17:07:28. Total running time: 7min 31s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8817499876022339 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00000 RUNNING 256 0.00078507 16 3 327.211 0.88175 |\n", "| train_model_3b16c_00002 RUNNING 256 0.0238983 64 4 132.583 0.843083 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00003 PENDING 64 0.00639545 8 |\n", "| train_model_3b16c_00004 PENDING 32 0.0721965 16 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=8318)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00000_0_batch_size=16,hidden_size=256,lr=0.0008_2025-04-10_16-59-56/checkpoint_000003)\n", "\u001b[36m(train_model pid=9613)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00002_2_batch_size=64,hidden_size=256,lr=0.0239_2025-04-10_16-59-56/checkpoint_000004)\n", "2025-04-10 17:07:32,532\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 2 RUNNING | 1 TERMINATED | 7 PENDING\n", "Current time: 2025-04-10 17:07:58. Total running time: 8min 1s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8799999952316284 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00000 RUNNING 256 0.00078507 16 4 431.83 0.88 |\n", "| train_model_3b16c_00002 RUNNING 256 0.0238983 64 5 162.23 0.839333 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00003 PENDING 64 0.00639545 8 |\n", "| train_model_3b16c_00004 PENDING 32 0.0721965 16 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=9613)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00002_2_batch_size=64,hidden_size=256,lr=0.0239_2025-04-10_16-59-56/checkpoint_000005)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 2 RUNNING | 1 TERMINATED | 7 PENDING\n", "Current time: 2025-04-10 17:08:28. Total running time: 8min 31s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8799999952316284 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00000 RUNNING 256 0.00078507 16 4 431.83 0.88 |\n", "| train_model_3b16c_00002 RUNNING 256 0.0238983 64 6 191.301 0.838333 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00003 PENDING 64 0.00639545 8 |\n", "| train_model_3b16c_00004 PENDING 32 0.0721965 16 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=9613)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00002_2_batch_size=64,hidden_size=256,lr=0.0239_2025-04-10_16-59-56/checkpoint_000006)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 2 RUNNING | 1 TERMINATED | 7 PENDING\n", "Current time: 2025-04-10 17:08:58. Total running time: 9min 1s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8799999952316284 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00000 RUNNING 256 0.00078507 16 4 431.83 0.88 |\n", "| train_model_3b16c_00002 RUNNING 256 0.0238983 64 7 220.352 0.814083 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00003 PENDING 64 0.00639545 8 |\n", "| train_model_3b16c_00004 PENDING 32 0.0721965 16 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=9613)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00002_2_batch_size=64,hidden_size=256,lr=0.0239_2025-04-10_16-59-56/checkpoint_000007)\n", "2025-04-10 17:09:14,154\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=8318)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00000_0_batch_size=16,hidden_size=256,lr=0.0008_2025-04-10_16-59-56/checkpoint_000004)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 2 RUNNING | 1 TERMINATED | 7 PENDING\n", "Current time: 2025-04-10 17:09:28. Total running time: 9min 31s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8841666579246521 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00000 RUNNING 256 0.00078507 16 5 536.006 0.884167 |\n", "| train_model_3b16c_00002 RUNNING 256 0.0238983 64 8 248.942 0.815 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00003 PENDING 64 0.00639545 8 |\n", "| train_model_3b16c_00004 PENDING 32 0.0721965 16 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=9613)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00002_2_batch_size=64,hidden_size=256,lr=0.0239_2025-04-10_16-59-56/checkpoint_000008)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 2 RUNNING | 1 TERMINATED | 7 PENDING\n", "Current time: 2025-04-10 17:09:58. Total running time: 10min 1s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8841666579246521 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00000 RUNNING 256 0.00078507 16 5 536.006 0.884167 |\n", "| train_model_3b16c_00002 RUNNING 256 0.0238983 64 9 282.164 0.84875 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00003 PENDING 64 0.00639545 8 |\n", "| train_model_3b16c_00004 PENDING 32 0.0721965 16 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=9613)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00002_2_batch_size=64,hidden_size=256,lr=0.0239_2025-04-10_16-59-56/checkpoint_000009)\n", "\u001b[36m(train_model pid=9613)\u001b[0m `Trainer.fit` stopped: `max_epochs=10` reached.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "Trial train_model_3b16c_00002 completed after 10 iterations at 2025-04-10 17:10:01. Total running time: 10min 4s\n", "+------------------------------------------------------------+\n", "| Trial train_model_3b16c_00002 result |\n", "+------------------------------------------------------------+\n", "| checkpoint_dir_name checkpoint_000009 |\n", "| time_this_iter_s 28.80246 |\n", "| time_total_s 310.96663 |\n", "| training_iteration 10 |\n", "| val_acc 0.82633 |\n", "+------------------------------------------------------------+\n", "\n", "Trial train_model_3b16c_00003 started with configuration:\n", "+-------------------------------------------------+\n", "| Trial train_model_3b16c_00003 config |\n", "+-------------------------------------------------+\n", "| batch_size 8 |\n", "| hidden_size 64 |\n", "| lr 0.0064 |\n", "+-------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=11070)\u001b[0m \r", " 0%| | 0.00/26.4M [00:00= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=8318)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00000_0_batch_size=16,hidden_size=256,lr=0.0008_2025-04-10_16-59-56/checkpoint_000006)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 2 RUNNING | 2 TERMINATED | 6 PENDING\n", "Current time: 2025-04-10 17:11:58. Total running time: 12min 1s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8864166736602783 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00000 RUNNING 256 0.00078507 16 7 689.834 0.886417 |\n", "| train_model_3b16c_00003 RUNNING 64 0.00639545 8 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00004 PENDING 32 0.0721965 16 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "Trial status: 2 RUNNING | 2 TERMINATED | 6 PENDING\n", "Current time: 2025-04-10 17:12:28. Total running time: 12min 31s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8864166736602783 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00000 RUNNING 256 0.00078507 16 7 689.834 0.886417 |\n", "| train_model_3b16c_00003 RUNNING 64 0.00639545 8 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00004 PENDING 32 0.0721965 16 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=11070)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00003_3_batch_size=8,hidden_size=64,lr=0.0064_2025-04-10_16-59-56/checkpoint_000000)\n", "2025-04-10 17:12:58,663\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 2 RUNNING | 2 TERMINATED | 6 PENDING\n", "Current time: 2025-04-10 17:12:58. Total running time: 13min 1s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8864166736602783 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00000 RUNNING 256 0.00078507 16 7 689.834 0.886417 |\n", "| train_model_3b16c_00003 RUNNING 64 0.00639545 8 1 151.898 0.806667 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00004 PENDING 32 0.0721965 16 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "Trial status: 2 RUNNING | 2 TERMINATED | 6 PENDING\n", "Current time: 2025-04-10 17:13:28. Total running time: 13min 31s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8871666789054871 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00000 RUNNING 256 0.00078507 16 8 760.467 0.887167 |\n", "| train_model_3b16c_00003 RUNNING 64 0.00639545 8 1 151.898 0.806667 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00004 PENDING 32 0.0721965 16 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "Trial status: 2 RUNNING | 2 TERMINATED | 6 PENDING\n", "Current time: 2025-04-10 17:13:58. Total running time: 14min 1s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8871666789054871 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00000 RUNNING 256 0.00078507 16 8 760.467 0.887167 |\n", "| train_model_3b16c_00003 RUNNING 64 0.00639545 8 1 151.898 0.806667 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00004 PENDING 32 0.0721965 16 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025-04-10 17:14:09,980\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=8318)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00000_0_batch_size=16,hidden_size=256,lr=0.0008_2025-04-10_16-59-56/checkpoint_000008)\u001b[32m [repeated 2x across cluster]\u001b[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 2 RUNNING | 2 TERMINATED | 6 PENDING\n", "Current time: 2025-04-10 17:14:28. Total running time: 14min 31s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8878333568572998 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00000 RUNNING 256 0.00078507 16 9 831.775 0.887833 |\n", "| train_model_3b16c_00003 RUNNING 64 0.00639545 8 1 151.898 0.806667 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00004 PENDING 32 0.0721965 16 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "Trial status: 2 RUNNING | 2 TERMINATED | 6 PENDING\n", "Current time: 2025-04-10 17:14:58. Total running time: 15min 1s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8878333568572998 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00000 RUNNING 256 0.00078507 16 9 831.775 0.887833 |\n", "| train_model_3b16c_00003 RUNNING 64 0.00639545 8 1 151.898 0.806667 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00004 PENDING 32 0.0721965 16 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025-04-10 17:15:11,265\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=11070)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00003_3_batch_size=8,hidden_size=64,lr=0.0064_2025-04-10_16-59-56/checkpoint_000001)\n", "2025-04-10 17:15:21,507\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=8318)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00000_0_batch_size=16,hidden_size=256,lr=0.0008_2025-04-10_16-59-56/checkpoint_000009)\n", "\u001b[36m(train_model pid=8318)\u001b[0m `Trainer.fit` stopped: `max_epochs=10` reached.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "Trial train_model_3b16c_00000 completed after 10 iterations at 2025-04-10 17:15:21. Total running time: 15min 24s\n", "+------------------------------------------------------------+\n", "| Trial train_model_3b16c_00000 result |\n", "+------------------------------------------------------------+\n", "| checkpoint_dir_name checkpoint_000009 |\n", "| time_this_iter_s 71.51173 |\n", "| time_total_s 903.28658 |\n", "| training_iteration 10 |\n", "| val_acc 0.8835 |\n", "+------------------------------------------------------------+\n", "\n", "Trial status: 3 TERMINATED | 1 RUNNING | 6 PENDING\n", "Current time: 2025-04-10 17:15:28. Total running time: 15min 31s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00003 RUNNING 64 0.00639545 8 2 288.209 0.837333 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00004 PENDING 32 0.0721965 16 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "\n", "Trial train_model_3b16c_00004 started with configuration:\n", "+-------------------------------------------------+\n", "| Trial train_model_3b16c_00004 config |\n", "+-------------------------------------------------+\n", "| batch_size 16 |\n", "| hidden_size 32 |\n", "| lr 0.0722 |\n", "+-------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=12479)\u001b[0m \r", " 0%| | 0.00/26.4M [00:00= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=12479)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00004_4_batch_size=16,hidden_size=32,lr=0.0722_2025-04-10_16-59-57/checkpoint_000000)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 3 TERMINATED | 2 RUNNING | 5 PENDING\n", "Current time: 2025-04-10 17:16:59. Total running time: 17min 2s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00003 RUNNING 64 0.00639545 8 2 288.209 0.837333 |\n", "| train_model_3b16c_00004 RUNNING 32 0.0721965 16 1 77.6564 0.156333 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=11070)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00003_3_batch_size=8,hidden_size=64,lr=0.0064_2025-04-10_16-59-56/checkpoint_000002)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 3 TERMINATED | 2 RUNNING | 5 PENDING\n", "Current time: 2025-04-10 17:17:29. Total running time: 17min 32s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00003 RUNNING 64 0.00639545 8 3 419.61 0.825667 |\n", "| train_model_3b16c_00004 RUNNING 32 0.0721965 16 1 77.6564 0.156333 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "Trial status: 3 TERMINATED | 2 RUNNING | 5 PENDING\n", "Current time: 2025-04-10 17:17:59. Total running time: 18min 2s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00003 RUNNING 64 0.00639545 8 3 419.61 0.825667 |\n", "| train_model_3b16c_00004 RUNNING 32 0.0721965 16 1 77.6564 0.156333 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=12479)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00004_4_batch_size=16,hidden_size=32,lr=0.0722_2025-04-10_16-59-57/checkpoint_000001)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 3 TERMINATED | 2 RUNNING | 5 PENDING\n", "Current time: 2025-04-10 17:18:29. Total running time: 18min 32s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00003 RUNNING 64 0.00639545 8 3 419.61 0.825667 |\n", "| train_model_3b16c_00004 RUNNING 32 0.0721965 16 2 140.319 0.16275 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "Trial status: 3 TERMINATED | 2 RUNNING | 5 PENDING\n", "Current time: 2025-04-10 17:18:59. Total running time: 19min 2s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00003 RUNNING 64 0.00639545 8 3 419.61 0.825667 |\n", "| train_model_3b16c_00004 RUNNING 32 0.0721965 16 2 140.319 0.16275 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025-04-10 17:19:04,103\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=12479)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00004_4_batch_size=16,hidden_size=32,lr=0.0722_2025-04-10_16-59-57/checkpoint_000002)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 3 TERMINATED | 2 RUNNING | 5 PENDING\n", "Current time: 2025-04-10 17:19:29. Total running time: 19min 32s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00003 RUNNING 64 0.00639545 8 3 419.61 0.825667 |\n", "| train_model_3b16c_00004 RUNNING 32 0.0721965 16 3 204.98 0.157167 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=11070)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00003_3_batch_size=8,hidden_size=64,lr=0.0064_2025-04-10_16-59-56/checkpoint_000003)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 3 TERMINATED | 2 RUNNING | 5 PENDING\n", "Current time: 2025-04-10 17:19:59. Total running time: 20min 2s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00003 RUNNING 64 0.00639545 8 4 556.826 0.81475 |\n", "| train_model_3b16c_00004 RUNNING 32 0.0721965 16 3 204.98 0.157167 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=12479)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00004_4_batch_size=16,hidden_size=32,lr=0.0722_2025-04-10_16-59-57/checkpoint_000003)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 3 TERMINATED | 2 RUNNING | 5 PENDING\n", "Current time: 2025-04-10 17:20:29. Total running time: 20min 32s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00003 RUNNING 64 0.00639545 8 4 556.826 0.81475 |\n", "| train_model_3b16c_00004 RUNNING 32 0.0721965 16 4 269.621 0.169083 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "Trial status: 3 TERMINATED | 2 RUNNING | 5 PENDING\n", "Current time: 2025-04-10 17:20:59. Total running time: 21min 2s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00003 RUNNING 64 0.00639545 8 4 556.826 0.81475 |\n", "| train_model_3b16c_00004 RUNNING 32 0.0721965 16 4 269.621 0.169083 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025-04-10 17:21:10,453\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=12479)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00004_4_batch_size=16,hidden_size=32,lr=0.0722_2025-04-10_16-59-57/checkpoint_000004)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 3 TERMINATED | 2 RUNNING | 5 PENDING\n", "Current time: 2025-04-10 17:21:29. Total running time: 21min 32s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00003 RUNNING 64 0.00639545 8 4 556.826 0.81475 |\n", "| train_model_3b16c_00004 RUNNING 32 0.0721965 16 5 331.296 0.160917 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025-04-10 17:21:54,882\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=11070)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00003_3_batch_size=8,hidden_size=64,lr=0.0064_2025-04-10_16-59-56/checkpoint_000004)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 3 TERMINATED | 2 RUNNING | 5 PENDING\n", "Current time: 2025-04-10 17:21:59. Total running time: 22min 2s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00003 RUNNING 64 0.00639545 8 5 691.779 0.839 |\n", "| train_model_3b16c_00004 RUNNING 32 0.0721965 16 5 331.296 0.160917 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=12479)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00004_4_batch_size=16,hidden_size=32,lr=0.0722_2025-04-10_16-59-57/checkpoint_000005)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 3 TERMINATED | 2 RUNNING | 5 PENDING\n", "Current time: 2025-04-10 17:22:29. Total running time: 22min 32s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00003 RUNNING 64 0.00639545 8 5 691.779 0.839 |\n", "| train_model_3b16c_00004 RUNNING 32 0.0721965 16 6 393.251 0.17525 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "Trial status: 3 TERMINATED | 2 RUNNING | 5 PENDING\n", "Current time: 2025-04-10 17:22:59. Total running time: 23min 2s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00003 RUNNING 64 0.00639545 8 5 691.779 0.839 |\n", "| train_model_3b16c_00004 RUNNING 32 0.0721965 16 6 393.251 0.17525 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=12479)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00004_4_batch_size=16,hidden_size=32,lr=0.0722_2025-04-10_16-59-57/checkpoint_000006)\n", "2025-04-10 17:23:13,903\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 3 TERMINATED | 2 RUNNING | 5 PENDING\n", "Current time: 2025-04-10 17:23:29. Total running time: 23min 32s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00003 RUNNING 64 0.00639545 8 5 691.779 0.839 |\n", "| train_model_3b16c_00004 RUNNING 32 0.0721965 16 7 454.707 0.156417 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "Trial status: 3 TERMINATED | 2 RUNNING | 5 PENDING\n", "Current time: 2025-04-10 17:23:59. Total running time: 24min 2s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00003 RUNNING 64 0.00639545 8 5 691.779 0.839 |\n", "| train_model_3b16c_00004 RUNNING 32 0.0721965 16 7 454.707 0.156417 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=11070)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00003_3_batch_size=8,hidden_size=64,lr=0.0064_2025-04-10_16-59-56/checkpoint_000005)\n", "2025-04-10 17:24:15,310\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 3 TERMINATED | 2 RUNNING | 5 PENDING\n", "Current time: 2025-04-10 17:24:29. Total running time: 24min 32s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00003 RUNNING 64 0.00639545 8 6 827.74 0.8515 |\n", "| train_model_3b16c_00004 RUNNING 32 0.0721965 16 8 516.104 0.1645 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "Trial status: 3 TERMINATED | 2 RUNNING | 5 PENDING\n", "Current time: 2025-04-10 17:24:59. Total running time: 25min 2s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00003 RUNNING 64 0.00639545 8 6 827.74 0.8515 |\n", "| train_model_3b16c_00004 RUNNING 32 0.0721965 16 8 516.104 0.1645 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025-04-10 17:25:16,998\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=12479)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00004_4_batch_size=16,hidden_size=32,lr=0.0722_2025-04-10_16-59-57/checkpoint_000008)\u001b[32m [repeated 2x across cluster]\u001b[0m\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 3 TERMINATED | 2 RUNNING | 5 PENDING\n", "Current time: 2025-04-10 17:25:29. Total running time: 25min 32s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00003 RUNNING 64 0.00639545 8 6 827.74 0.8515 |\n", "| train_model_3b16c_00004 RUNNING 32 0.0721965 16 9 577.783 0.161167 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "Trial status: 3 TERMINATED | 2 RUNNING | 5 PENDING\n", "Current time: 2025-04-10 17:25:59. Total running time: 26min 2s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00003 RUNNING 64 0.00639545 8 6 827.74 0.8515 |\n", "| train_model_3b16c_00004 RUNNING 32 0.0721965 16 9 577.783 0.161167 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025-04-10 17:26:18,281\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=12479)\u001b[0m `Trainer.fit` stopped: `max_epochs=10` reached.\n", "\u001b[36m(train_model pid=12479)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00004_4_batch_size=16,hidden_size=32,lr=0.0722_2025-04-10_16-59-57/checkpoint_000009)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "Trial train_model_3b16c_00004 completed after 10 iterations at 2025-04-10 17:26:18. Total running time: 26min 21s\n", "+------------------------------------------------------------+\n", "| Trial train_model_3b16c_00004 result |\n", "+------------------------------------------------------------+\n", "| checkpoint_dir_name checkpoint_000009 |\n", "| time_this_iter_s 61.26871 |\n", "| time_total_s 639.05217 |\n", "| training_iteration 10 |\n", "| val_acc 0.15942 |\n", "+------------------------------------------------------------+\n", "\n", "Trial status: 4 TERMINATED | 1 RUNNING | 5 PENDING\n", "Current time: 2025-04-10 17:26:29. Total running time: 26min 33s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00003 RUNNING 64 0.00639545 8 6 827.74 0.8515 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 PENDING 64 0.00226128 16 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025-04-10 17:26:32,041\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=11070)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00003_3_batch_size=8,hidden_size=64,lr=0.0064_2025-04-10_16-59-56/checkpoint_000006)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "Trial train_model_3b16c_00005 started with configuration:\n", "+--------------------------------------------------+\n", "| Trial train_model_3b16c_00005 config |\n", "+--------------------------------------------------+\n", "| batch_size 16 |\n", "| hidden_size 64 |\n", "| lr 0.00226 |\n", "+--------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=15268)\u001b[0m \r", " 0%| | 0.00/26.4M [00:00= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=15268)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00005_5_batch_size=16,hidden_size=64,lr=0.0023_2025-04-10_16-59-57/checkpoint_000000)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 4 TERMINATED | 2 RUNNING | 4 PENDING\n", "Current time: 2025-04-10 17:28:30. Total running time: 28min 33s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00003 RUNNING 64 0.00639545 8 7 968.901 0.851417 |\n", "| train_model_3b16c_00005 RUNNING 64 0.00226128 16 1 77.9188 0.844083 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025-04-10 17:28:37,773\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=11070)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00003_3_batch_size=8,hidden_size=64,lr=0.0064_2025-04-10_16-59-56/checkpoint_000007)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 4 TERMINATED | 2 RUNNING | 4 PENDING\n", "Current time: 2025-04-10 17:29:00. Total running time: 29min 3s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00003 RUNNING 64 0.00639545 8 8 1094.62 0.848167 |\n", "| train_model_3b16c_00005 RUNNING 64 0.00226128 16 1 77.9188 0.844083 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=15268)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00005_5_batch_size=16,hidden_size=64,lr=0.0023_2025-04-10_16-59-57/checkpoint_000001)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 4 TERMINATED | 2 RUNNING | 4 PENDING\n", "Current time: 2025-04-10 17:29:30. Total running time: 29min 33s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00003 RUNNING 64 0.00639545 8 8 1094.62 0.848167 |\n", "| train_model_3b16c_00005 RUNNING 64 0.00226128 16 2 139.784 0.867667 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "Trial status: 4 TERMINATED | 2 RUNNING | 4 PENDING\n", "Current time: 2025-04-10 17:30:00. Total running time: 30min 3s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00003 RUNNING 64 0.00639545 8 8 1094.62 0.848167 |\n", "| train_model_3b16c_00005 RUNNING 64 0.00226128 16 2 139.784 0.867667 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025-04-10 17:30:06,250\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=15268)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00005_5_batch_size=16,hidden_size=64,lr=0.0023_2025-04-10_16-59-57/checkpoint_000002)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 4 TERMINATED | 2 RUNNING | 4 PENDING\n", "Current time: 2025-04-10 17:30:30. Total running time: 30min 33s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00003 RUNNING 64 0.00639545 8 8 1094.62 0.848167 |\n", "| train_model_3b16c_00005 RUNNING 64 0.00226128 16 3 201.087 0.864583 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=11070)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00003_3_batch_size=8,hidden_size=64,lr=0.0064_2025-04-10_16-59-56/checkpoint_000008)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 4 TERMINATED | 2 RUNNING | 4 PENDING\n", "Current time: 2025-04-10 17:31:00. Total running time: 31min 3s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00003 RUNNING 64 0.00639545 8 9 1231.91 0.845 |\n", "| train_model_3b16c_00005 RUNNING 64 0.00226128 16 3 201.087 0.864583 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=15268)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00005_5_batch_size=16,hidden_size=64,lr=0.0023_2025-04-10_16-59-57/checkpoint_000003)\n", "2025-04-10 17:31:09,139\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 4 TERMINATED | 2 RUNNING | 4 PENDING\n", "Current time: 2025-04-10 17:31:30. Total running time: 31min 33s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00003 RUNNING 64 0.00639545 8 9 1231.91 0.845 |\n", "| train_model_3b16c_00005 RUNNING 64 0.00226128 16 4 263.921 0.866167 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "Trial status: 4 TERMINATED | 2 RUNNING | 4 PENDING\n", "Current time: 2025-04-10 17:32:00. Total running time: 32min 3s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00003 RUNNING 64 0.00639545 8 9 1231.91 0.845 |\n", "| train_model_3b16c_00005 RUNNING 64 0.00226128 16 4 263.921 0.866167 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025-04-10 17:32:13,681\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=15268)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00005_5_batch_size=16,hidden_size=64,lr=0.0023_2025-04-10_16-59-57/checkpoint_000004)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 4 TERMINATED | 2 RUNNING | 4 PENDING\n", "Current time: 2025-04-10 17:32:30. Total running time: 32min 33s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00003 RUNNING 64 0.00639545 8 9 1231.91 0.845 |\n", "| train_model_3b16c_00005 RUNNING 64 0.00226128 16 5 328.446 0.865667 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "Trial status: 4 TERMINATED | 2 RUNNING | 4 PENDING\n", "Current time: 2025-04-10 17:33:00. Total running time: 33min 3s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00003 RUNNING 64 0.00639545 8 9 1231.91 0.845 |\n", "| train_model_3b16c_00005 RUNNING 64 0.00226128 16 5 328.446 0.865667 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025-04-10 17:33:18,129\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=15268)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00005_5_batch_size=16,hidden_size=64,lr=0.0023_2025-04-10_16-59-57/checkpoint_000005)\n", "2025-04-10 17:33:18,838\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=11070)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00003_3_batch_size=8,hidden_size=64,lr=0.0064_2025-04-10_16-59-56/checkpoint_000009)\n", "\u001b[36m(train_model pid=11070)\u001b[0m `Trainer.fit` stopped: `max_epochs=10` reached.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "Trial train_model_3b16c_00003 completed after 10 iterations at 2025-04-10 17:33:19. Total running time: 33min 22s\n", "+------------------------------------------------------------+\n", "| Trial train_model_3b16c_00003 result |\n", "+------------------------------------------------------------+\n", "| checkpoint_dir_name checkpoint_000009 |\n", "| time_this_iter_s 143.73959 |\n", "| time_total_s 1375.6535 |\n", "| training_iteration 10 |\n", "| val_acc 0.85217 |\n", "+------------------------------------------------------------+\n", "\n", "Trial status: 5 TERMINATED | 1 RUNNING | 4 PENDING\n", "Current time: 2025-04-10 17:33:30. Total running time: 33min 33s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00005 RUNNING 64 0.00226128 16 6 392.883 0.86825 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00006 PENDING 32 0.0179783 16 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "\n", "Trial train_model_3b16c_00006 started with configuration:\n", "+--------------------------------------------------+\n", "| Trial train_model_3b16c_00006 config |\n", "+--------------------------------------------------+\n", "| batch_size 16 |\n", "| hidden_size 32 |\n", "| lr 0.01798 |\n", "+--------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=17089)\u001b[0m \r", " 0%| | 0.00/26.4M [00:00= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=15268)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00005_5_batch_size=16,hidden_size=64,lr=0.0023_2025-04-10_16-59-57/checkpoint_000006)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 5 TERMINATED | 2 RUNNING | 3 PENDING\n", "Current time: 2025-04-10 17:34:30. Total running time: 34min 33s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00005 RUNNING 64 0.00226128 16 7 456.521 0.873333 |\n", "| train_model_3b16c_00006 RUNNING 32 0.0179783 16 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "Trial status: 5 TERMINATED | 2 RUNNING | 3 PENDING\n", "Current time: 2025-04-10 17:35:00. Total running time: 35min 3s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00005 RUNNING 64 0.00226128 16 7 456.521 0.873333 |\n", "| train_model_3b16c_00006 RUNNING 32 0.0179783 16 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025-04-10 17:35:02,474\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=17089)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00006_6_batch_size=16,hidden_size=32,lr=0.0180_2025-04-10_16-59-57/checkpoint_000000)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 5 TERMINATED | 2 RUNNING | 3 PENDING\n", "Current time: 2025-04-10 17:35:30. Total running time: 35min 34s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00005 RUNNING 64 0.00226128 16 7 456.521 0.873333 |\n", "| train_model_3b16c_00006 RUNNING 32 0.0179783 16 1 84.4981 0.786 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=15268)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00005_5_batch_size=16,hidden_size=64,lr=0.0023_2025-04-10_16-59-57/checkpoint_000007)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 5 TERMINATED | 2 RUNNING | 3 PENDING\n", "Current time: 2025-04-10 17:36:01. Total running time: 36min 4s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00005 RUNNING 64 0.00226128 16 8 526.025 0.869917 |\n", "| train_model_3b16c_00006 RUNNING 32 0.0179783 16 1 84.4981 0.786 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=17089)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00006_6_batch_size=16,hidden_size=32,lr=0.0180_2025-04-10_16-59-57/checkpoint_000001)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 5 TERMINATED | 2 RUNNING | 3 PENDING\n", "Current time: 2025-04-10 17:36:31. Total running time: 36min 34s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00005 RUNNING 64 0.00226128 16 8 526.025 0.869917 |\n", "| train_model_3b16c_00006 RUNNING 32 0.0179783 16 2 151.39 0.800333 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=15268)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00005_5_batch_size=16,hidden_size=64,lr=0.0023_2025-04-10_16-59-57/checkpoint_000008)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 5 TERMINATED | 2 RUNNING | 3 PENDING\n", "Current time: 2025-04-10 17:37:01. Total running time: 37min 4s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00005 RUNNING 64 0.00226128 16 9 602.888 0.874833 |\n", "| train_model_3b16c_00006 RUNNING 32 0.0179783 16 2 151.39 0.800333 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=17089)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00006_6_batch_size=16,hidden_size=32,lr=0.0180_2025-04-10_16-59-57/checkpoint_000002)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 5 TERMINATED | 2 RUNNING | 3 PENDING\n", "Current time: 2025-04-10 17:37:31. Total running time: 37min 34s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00005 RUNNING 64 0.00226128 16 9 602.888 0.874833 |\n", "| train_model_3b16c_00006 RUNNING 32 0.0179783 16 3 226.744 0.766833 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025-04-10 17:37:55,764\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=15268)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00005_5_batch_size=16,hidden_size=64,lr=0.0023_2025-04-10_16-59-57/checkpoint_000009)\n", "\u001b[36m(train_model pid=15268)\u001b[0m `Trainer.fit` stopped: `max_epochs=10` reached.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "Trial train_model_3b16c_00005 completed after 10 iterations at 2025-04-10 17:37:56. Total running time: 37min 59s\n", "+------------------------------------------------------------+\n", "| Trial train_model_3b16c_00005 result |\n", "+------------------------------------------------------------+\n", "| checkpoint_dir_name checkpoint_000009 |\n", "| time_this_iter_s 67.5672 |\n", "| time_total_s 670.45531 |\n", "| training_iteration 10 |\n", "| val_acc 0.87633 |\n", "+------------------------------------------------------------+\n", "\n", "Trial status: 6 TERMINATED | 1 RUNNING | 3 PENDING\n", "Current time: 2025-04-10 17:38:01. Total running time: 38min 4s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00006 RUNNING 32 0.0179783 16 3 226.744 0.766833 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00007 PENDING 32 0.000213498 64 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "\n", "Trial train_model_3b16c_00007 started with configuration:\n", "+--------------------------------------------------+\n", "| Trial train_model_3b16c_00007 config |\n", "+--------------------------------------------------+\n", "| batch_size 64 |\n", "| hidden_size 32 |\n", "| lr 0.00021 |\n", "+--------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=18328)\u001b[0m \r", " 0%| | 0.00/26.4M [00:00= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=17089)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00006_6_batch_size=16,hidden_size=32,lr=0.0180_2025-04-10_16-59-57/checkpoint_000003)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "Trial status: 6 TERMINATED | 2 RUNNING | 2 PENDING\n", "Current time: 2025-04-10 17:38:31. Total running time: 38min 34s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00006 RUNNING 32 0.0179783 16 4 288.313 0.8165 |\n", "| train_model_3b16c_00007 RUNNING 32 0.000213498 64 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=18328)\u001b[0m \n", "\u001b[36m(train_model pid=18328)\u001b[0m | Name | Type | Params | Mode \n", "\u001b[36m(train_model pid=18328)\u001b[0m ---------------------------------------------------------\n", "\u001b[36m(train_model pid=18328)\u001b[0m 0 | model | Sequential | 25.4 K | train\n", "\u001b[36m(train_model pid=18328)\u001b[0m 1 | loss_fn | CrossEntropyLoss | 0 | train\n", "\u001b[36m(train_model pid=18328)\u001b[0m 2 | train_acc | MulticlassAccuracy | 0 | train\n", "\u001b[36m(train_model pid=18328)\u001b[0m 3 | val_acc | MulticlassAccuracy | 0 | train\n", "\u001b[36m(train_model pid=18328)\u001b[0m 4 | test_acc | MulticlassAccuracy | 0 | train\n", "\u001b[36m(train_model pid=18328)\u001b[0m ---------------------------------------------------------\n", "\u001b[36m(train_model pid=18328)\u001b[0m 25.4 K Trainable params\n", "\u001b[36m(train_model pid=18328)\u001b[0m 0 Non-trainable params\n", "\u001b[36m(train_model pid=18328)\u001b[0m 25.4 K Total params\n", "\u001b[36m(train_model pid=18328)\u001b[0m 0.102 Total estimated model params size (MB)\n", "\u001b[36m(train_model pid=18328)\u001b[0m 9 Modules in train mode\n", "\u001b[36m(train_model pid=18328)\u001b[0m 0 Modules in eval mode\n", "\u001b[36m(train_model pid=18328)\u001b[0m /usr/local/lib/python3.11/dist-packages/torch/utils/data/dataloader.py:624: UserWarning: This DataLoader will create 4 worker processes in total. Our suggested max number of worker in current system is 2, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.\n", "\u001b[36m(train_model pid=18328)\u001b[0m warnings.warn(\n", "2025-04-10 17:38:57,770\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=18328)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00007_7_batch_size=64,hidden_size=32,lr=0.0002_2025-04-10_16-59-57/checkpoint_000000)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 6 TERMINATED | 2 RUNNING | 2 PENDING\n", "Current time: 2025-04-10 17:39:01. Total running time: 39min 4s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00006 RUNNING 32 0.0179783 16 4 288.313 0.8165 |\n", "| train_model_3b16c_00007 RUNNING 32 0.000213498 64 1 39.3679 0.774167 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025-04-10 17:39:22,051\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=18328)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00007_7_batch_size=64,hidden_size=32,lr=0.0002_2025-04-10_16-59-57/checkpoint_000001)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 6 TERMINATED | 2 RUNNING | 2 PENDING\n", "Current time: 2025-04-10 17:39:31. Total running time: 39min 34s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00006 RUNNING 32 0.0179783 16 4 288.313 0.8165 |\n", "| train_model_3b16c_00007 RUNNING 32 0.000213498 64 2 63.6393 0.817667 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=18328)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00007_7_batch_size=64,hidden_size=32,lr=0.0002_2025-04-10_16-59-57/checkpoint_000002)\n", "2025-04-10 17:39:44,920\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "2025-04-10 17:39:55,267\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=17089)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00006_6_batch_size=16,hidden_size=32,lr=0.0180_2025-04-10_16-59-57/checkpoint_000004)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 6 TERMINATED | 2 RUNNING | 2 PENDING\n", "Current time: 2025-04-10 17:40:01. Total running time: 40min 4s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00006 RUNNING 32 0.0179783 16 5 377.22 0.800333 |\n", "| train_model_3b16c_00007 RUNNING 32 0.000213498 64 3 86.4804 0.8325 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025-04-10 17:40:08,929\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=18328)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00007_7_batch_size=64,hidden_size=32,lr=0.0002_2025-04-10_16-59-57/checkpoint_000003)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 6 TERMINATED | 2 RUNNING | 2 PENDING\n", "Current time: 2025-04-10 17:40:31. Total running time: 40min 34s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00006 RUNNING 32 0.0179783 16 5 377.22 0.800333 |\n", "| train_model_3b16c_00007 RUNNING 32 0.000213498 64 4 110.482 0.840417 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025-04-10 17:40:32,505\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=18328)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00007_7_batch_size=64,hidden_size=32,lr=0.0002_2025-04-10_16-59-57/checkpoint_000004)\n", "2025-04-10 17:40:55,225\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=18328)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00007_7_batch_size=64,hidden_size=32,lr=0.0002_2025-04-10_16-59-57/checkpoint_000005)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 6 TERMINATED | 2 RUNNING | 2 PENDING\n", "Current time: 2025-04-10 17:41:01. Total running time: 41min 4s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00006 RUNNING 32 0.0179783 16 5 377.22 0.800333 |\n", "| train_model_3b16c_00007 RUNNING 32 0.000213498 64 6 156.74 0.844083 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025-04-10 17:41:19,834\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=18328)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00007_7_batch_size=64,hidden_size=32,lr=0.0002_2025-04-10_16-59-57/checkpoint_000006)\n", "2025-04-10 17:41:30,195\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=17089)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00006_6_batch_size=16,hidden_size=32,lr=0.0180_2025-04-10_16-59-57/checkpoint_000005)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 6 TERMINATED | 2 RUNNING | 2 PENDING\n", "Current time: 2025-04-10 17:41:31. Total running time: 41min 34s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00006 RUNNING 32 0.0179783 16 6 472.126 0.78 |\n", "| train_model_3b16c_00007 RUNNING 32 0.000213498 64 7 181.34 0.849417 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025-04-10 17:41:41,860\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=18328)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00007_7_batch_size=64,hidden_size=32,lr=0.0002_2025-04-10_16-59-57/checkpoint_000007)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 6 TERMINATED | 2 RUNNING | 2 PENDING\n", "Current time: 2025-04-10 17:42:01. Total running time: 42min 4s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00006 RUNNING 32 0.0179783 16 6 472.126 0.78 |\n", "| train_model_3b16c_00007 RUNNING 32 0.000213498 64 8 203.354 0.84825 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025-04-10 17:42:06,607\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=18328)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00007_7_batch_size=64,hidden_size=32,lr=0.0002_2025-04-10_16-59-57/checkpoint_000008)\n", "2025-04-10 17:42:29,080\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=18328)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00007_7_batch_size=64,hidden_size=32,lr=0.0002_2025-04-10_16-59-57/checkpoint_000009)\n", "\u001b[36m(train_model pid=18328)\u001b[0m `Trainer.fit` stopped: `max_epochs=10` reached.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "Trial train_model_3b16c_00007 completed after 10 iterations at 2025-04-10 17:42:29. Total running time: 42min 32s\n", "+------------------------------------------------------------+\n", "| Trial train_model_3b16c_00007 result |\n", "+------------------------------------------------------------+\n", "| checkpoint_dir_name checkpoint_000009 |\n", "| time_this_iter_s 22.46022 |\n", "| time_total_s 250.54858 |\n", "| training_iteration 10 |\n", "| val_acc 0.85558 |\n", "+------------------------------------------------------------+\n", "\n", "Trial status: 7 TERMINATED | 1 RUNNING | 2 PENDING\n", "Current time: 2025-04-10 17:42:31. Total running time: 42min 35s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00006 RUNNING 32 0.0179783 16 6 472.126 0.78 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00007 TERMINATED 32 0.000213498 64 10 250.549 0.855583 |\n", "| train_model_3b16c_00008 PENDING 256 0.00135129 8 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "\n", "Trial train_model_3b16c_00008 started with configuration:\n", "+--------------------------------------------------+\n", "| Trial train_model_3b16c_00008 config |\n", "+--------------------------------------------------+\n", "| batch_size 8 |\n", "| hidden_size 256 |\n", "| lr 0.00135 |\n", "+--------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=19552)\u001b[0m \r", " 0%| | 0.00/26.4M [00:00= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=17089)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00006_6_batch_size=16,hidden_size=32,lr=0.0180_2025-04-10_16-59-57/checkpoint_000006)\n", " 0%| | 0.00/29.5k [00:00= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 7 TERMINATED | 2 RUNNING | 1 PENDING\n", "Current time: 2025-04-10 17:45:02. Total running time: 45min 5s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00006 RUNNING 32 0.0179783 16 9 675.872 0.807 |\n", "| train_model_3b16c_00008 RUNNING 256 0.00135129 8 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00007 TERMINATED 32 0.000213498 64 10 250.549 0.855583 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "Trial status: 7 TERMINATED | 2 RUNNING | 1 PENDING\n", "Current time: 2025-04-10 17:45:32. Total running time: 45min 35s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00006 RUNNING 32 0.0179783 16 9 675.872 0.807 |\n", "| train_model_3b16c_00008 RUNNING 256 0.00135129 8 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00007 TERMINATED 32 0.000213498 64 10 250.549 0.855583 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025-04-10 17:45:37,497\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=19552)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00008_8_batch_size=8,hidden_size=256,lr=0.0014_2025-04-10_16-59-57/checkpoint_000000)\n", "\u001b[36m(train_model pid=17089)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00006_6_batch_size=16,hidden_size=32,lr=0.0180_2025-04-10_16-59-57/checkpoint_000009)\n", "\u001b[36m(train_model pid=17089)\u001b[0m `Trainer.fit` stopped: `max_epochs=10` reached.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "Trial train_model_3b16c_00006 completed after 10 iterations at 2025-04-10 17:45:57. Total running time: 46min 0s\n", "+------------------------------------------------------------+\n", "| Trial train_model_3b16c_00006 result |\n", "+------------------------------------------------------------+\n", "| checkpoint_dir_name checkpoint_000009 |\n", "| time_this_iter_s 62.75633 |\n", "| time_total_s 738.62819 |\n", "| training_iteration 10 |\n", "| val_acc 0.804 |\n", "+------------------------------------------------------------+\n", "\n", "Trial status: 8 TERMINATED | 1 RUNNING | 1 PENDING\n", "Current time: 2025-04-10 17:46:02. Total running time: 46min 5s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00008 RUNNING 256 0.00135129 8 1 168.701 0.830917 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00006 TERMINATED 32 0.0179783 16 10 738.628 0.804 |\n", "| train_model_3b16c_00007 TERMINATED 32 0.000213498 64 10 250.549 0.855583 |\n", "| train_model_3b16c_00009 PENDING 128 0.000180116 64 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "\n", "Trial train_model_3b16c_00009 started with configuration:\n", "+--------------------------------------------------+\n", "| Trial train_model_3b16c_00009 config |\n", "+--------------------------------------------------+\n", "| batch_size 64 |\n", "| hidden_size 128 |\n", "| lr 0.00018 |\n", "+--------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=20507)\u001b[0m \r", " 0%| | 0.00/26.4M [00:00= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=20507)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00009_9_batch_size=64,hidden_size=128,lr=0.0002_2025-04-10_16-59-57/checkpoint_000001)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 8 TERMINATED | 2 RUNNING\n", "Current time: 2025-04-10 17:47:32. Total running time: 47min 35s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00008 RUNNING 256 0.00135129 8 1 168.701 0.830917 |\n", "| train_model_3b16c_00009 RUNNING 128 0.000180116 64 2 61.434 0.832417 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00006 TERMINATED 32 0.0179783 16 10 738.628 0.804 |\n", "| train_model_3b16c_00007 TERMINATED 32 0.000213498 64 10 250.549 0.855583 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025-04-10 17:47:40,801\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=20507)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00009_9_batch_size=64,hidden_size=128,lr=0.0002_2025-04-10_16-59-57/checkpoint_000002)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 8 TERMINATED | 2 RUNNING\n", "Current time: 2025-04-10 17:48:02. Total running time: 48min 5s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00008 RUNNING 256 0.00135129 8 1 168.701 0.830917 |\n", "| train_model_3b16c_00009 RUNNING 128 0.000180116 64 3 86.1632 0.833 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00006 TERMINATED 32 0.0179783 16 10 738.628 0.804 |\n", "| train_model_3b16c_00007 TERMINATED 32 0.000213498 64 10 250.549 0.855583 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025-04-10 17:48:03,990\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=20507)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00009_9_batch_size=64,hidden_size=128,lr=0.0002_2025-04-10_16-59-57/checkpoint_000003)\n", "2025-04-10 17:48:28,982\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=20507)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00009_9_batch_size=64,hidden_size=128,lr=0.0002_2025-04-10_16-59-57/checkpoint_000004)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 8 TERMINATED | 2 RUNNING\n", "Current time: 2025-04-10 17:48:32. Total running time: 48min 35s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00008 RUNNING 256 0.00135129 8 1 168.701 0.830917 |\n", "| train_model_3b16c_00009 RUNNING 128 0.000180116 64 5 134.313 0.850167 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00006 TERMINATED 32 0.0179783 16 10 738.628 0.804 |\n", "| train_model_3b16c_00007 TERMINATED 32 0.000213498 64 10 250.549 0.855583 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=19552)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00008_8_batch_size=8,hidden_size=256,lr=0.0014_2025-04-10_16-59-57/checkpoint_000001)\n", "2025-04-10 17:48:52,251\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=20507)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00009_9_batch_size=64,hidden_size=128,lr=0.0002_2025-04-10_16-59-57/checkpoint_000005)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 8 TERMINATED | 2 RUNNING\n", "Current time: 2025-04-10 17:49:02. Total running time: 49min 5s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00008 RUNNING 256 0.00135129 8 2 359.876 0.864 |\n", "| train_model_3b16c_00009 RUNNING 128 0.000180116 64 6 157.561 0.853667 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00006 TERMINATED 32 0.0179783 16 10 738.628 0.804 |\n", "| train_model_3b16c_00007 TERMINATED 32 0.000213498 64 10 250.549 0.855583 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025-04-10 17:49:16,814\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=20507)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00009_9_batch_size=64,hidden_size=128,lr=0.0002_2025-04-10_16-59-57/checkpoint_000006)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 8 TERMINATED | 2 RUNNING\n", "Current time: 2025-04-10 17:49:32. Total running time: 49min 35s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00008 RUNNING 256 0.00135129 8 2 359.876 0.864 |\n", "| train_model_3b16c_00009 RUNNING 128 0.000180116 64 7 182.106 0.856667 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00006 TERMINATED 32 0.0179783 16 10 738.628 0.804 |\n", "| train_model_3b16c_00007 TERMINATED 32 0.000213498 64 10 250.549 0.855583 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025-04-10 17:49:40,637\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=20507)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00009_9_batch_size=64,hidden_size=128,lr=0.0002_2025-04-10_16-59-57/checkpoint_000007)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 8 TERMINATED | 2 RUNNING\n", "Current time: 2025-04-10 17:50:02. Total running time: 50min 5s\n", "Logical resource usage: 2.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00008 RUNNING 256 0.00135129 8 2 359.876 0.864 |\n", "| train_model_3b16c_00009 RUNNING 128 0.000180116 64 8 205.91 0.864 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00006 TERMINATED 32 0.0179783 16 10 738.628 0.804 |\n", "| train_model_3b16c_00007 TERMINATED 32 0.000213498 64 10 250.549 0.855583 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025-04-10 17:50:04,454\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=20507)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00009_9_batch_size=64,hidden_size=128,lr=0.0002_2025-04-10_16-59-57/checkpoint_000008)\n", "\u001b[36m(train_model pid=20507)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00009_9_batch_size=64,hidden_size=128,lr=0.0002_2025-04-10_16-59-57/checkpoint_000009)\n", "\u001b[36m(train_model pid=20507)\u001b[0m `Trainer.fit` stopped: `max_epochs=10` reached.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "Trial train_model_3b16c_00009 completed after 10 iterations at 2025-04-10 17:50:30. Total running time: 50min 33s\n", "+------------------------------------------------------------+\n", "| Trial train_model_3b16c_00009 result |\n", "+------------------------------------------------------------+\n", "| checkpoint_dir_name checkpoint_000009 |\n", "| time_this_iter_s 25.27942 |\n", "| time_total_s 254.97694 |\n", "| training_iteration 10 |\n", "| val_acc 0.87042 |\n", "+------------------------------------------------------------+\n", "\n", "Trial status: 9 TERMINATED | 1 RUNNING\n", "Current time: 2025-04-10 17:50:32. Total running time: 50min 35s\n", "Logical resource usage: 1.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00008 RUNNING 256 0.00135129 8 2 359.876 0.864 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00006 TERMINATED 32 0.0179783 16 10 738.628 0.804 |\n", "| train_model_3b16c_00007 TERMINATED 32 0.000213498 64 10 250.549 0.855583 |\n", "| train_model_3b16c_00009 TERMINATED 128 0.000180116 64 10 254.977 0.870417 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "Trial status: 9 TERMINATED | 1 RUNNING\n", "Current time: 2025-04-10 17:51:02. Total running time: 51min 5s\n", "Logical resource usage: 1.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00008 RUNNING 256 0.00135129 8 2 359.876 0.864 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00006 TERMINATED 32 0.0179783 16 10 738.628 0.804 |\n", "| train_model_3b16c_00007 TERMINATED 32 0.000213498 64 10 250.549 0.855583 |\n", "| train_model_3b16c_00009 TERMINATED 128 0.000180116 64 10 254.977 0.870417 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025-04-10 17:51:21,069\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=19552)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00008_8_batch_size=8,hidden_size=256,lr=0.0014_2025-04-10_16-59-57/checkpoint_000002)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 9 TERMINATED | 1 RUNNING\n", "Current time: 2025-04-10 17:51:32. Total running time: 51min 35s\n", "Logical resource usage: 1.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00008 RUNNING 256 0.00135129 8 3 512.238 0.872583 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00006 TERMINATED 32 0.0179783 16 10 738.628 0.804 |\n", "| train_model_3b16c_00007 TERMINATED 32 0.000213498 64 10 250.549 0.855583 |\n", "| train_model_3b16c_00009 TERMINATED 128 0.000180116 64 10 254.977 0.870417 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "Trial status: 9 TERMINATED | 1 RUNNING\n", "Current time: 2025-04-10 17:52:02. Total running time: 52min 5s\n", "Logical resource usage: 1.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00008 RUNNING 256 0.00135129 8 3 512.238 0.872583 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00006 TERMINATED 32 0.0179783 16 10 738.628 0.804 |\n", "| train_model_3b16c_00007 TERMINATED 32 0.000213498 64 10 250.549 0.855583 |\n", "| train_model_3b16c_00009 TERMINATED 128 0.000180116 64 10 254.977 0.870417 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "Trial status: 9 TERMINATED | 1 RUNNING\n", "Current time: 2025-04-10 17:52:32. Total running time: 52min 36s\n", "Logical resource usage: 1.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00008 RUNNING 256 0.00135129 8 3 512.238 0.872583 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00006 TERMINATED 32 0.0179783 16 10 738.628 0.804 |\n", "| train_model_3b16c_00007 TERMINATED 32 0.000213498 64 10 250.549 0.855583 |\n", "| train_model_3b16c_00009 TERMINATED 128 0.000180116 64 10 254.977 0.870417 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025-04-10 17:52:45,969\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=19552)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00008_8_batch_size=8,hidden_size=256,lr=0.0014_2025-04-10_16-59-57/checkpoint_000003)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 9 TERMINATED | 1 RUNNING\n", "Current time: 2025-04-10 17:53:03. Total running time: 53min 6s\n", "Logical resource usage: 1.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00008 RUNNING 256 0.00135129 8 4 597.131 0.867 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00006 TERMINATED 32 0.0179783 16 10 738.628 0.804 |\n", "| train_model_3b16c_00007 TERMINATED 32 0.000213498 64 10 250.549 0.855583 |\n", "| train_model_3b16c_00009 TERMINATED 128 0.000180116 64 10 254.977 0.870417 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "Trial status: 9 TERMINATED | 1 RUNNING\n", "Current time: 2025-04-10 17:53:33. Total running time: 53min 36s\n", "Logical resource usage: 1.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00008 RUNNING 256 0.00135129 8 4 597.131 0.867 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00006 TERMINATED 32 0.0179783 16 10 738.628 0.804 |\n", "| train_model_3b16c_00007 TERMINATED 32 0.000213498 64 10 250.549 0.855583 |\n", "| train_model_3b16c_00009 TERMINATED 128 0.000180116 64 10 254.977 0.870417 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "Trial status: 9 TERMINATED | 1 RUNNING\n", "Current time: 2025-04-10 17:54:03. Total running time: 54min 6s\n", "Logical resource usage: 1.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00008 RUNNING 256 0.00135129 8 4 597.131 0.867 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00006 TERMINATED 32 0.0179783 16 10 738.628 0.804 |\n", "| train_model_3b16c_00007 TERMINATED 32 0.000213498 64 10 250.549 0.855583 |\n", "| train_model_3b16c_00009 TERMINATED 128 0.000180116 64 10 254.977 0.870417 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=19552)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00008_8_batch_size=8,hidden_size=256,lr=0.0014_2025-04-10_16-59-57/checkpoint_000004)\n", "2025-04-10 17:54:10,239\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 9 TERMINATED | 1 RUNNING\n", "Current time: 2025-04-10 17:54:33. Total running time: 54min 36s\n", "Logical resource usage: 1.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00008 RUNNING 256 0.00135129 8 5 681.389 0.871667 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00006 TERMINATED 32 0.0179783 16 10 738.628 0.804 |\n", "| train_model_3b16c_00007 TERMINATED 32 0.000213498 64 10 250.549 0.855583 |\n", "| train_model_3b16c_00009 TERMINATED 128 0.000180116 64 10 254.977 0.870417 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "Trial status: 9 TERMINATED | 1 RUNNING\n", "Current time: 2025-04-10 17:55:03. Total running time: 55min 6s\n", "Logical resource usage: 1.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00008 RUNNING 256 0.00135129 8 5 681.389 0.871667 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00006 TERMINATED 32 0.0179783 16 10 738.628 0.804 |\n", "| train_model_3b16c_00007 TERMINATED 32 0.000213498 64 10 250.549 0.855583 |\n", "| train_model_3b16c_00009 TERMINATED 128 0.000180116 64 10 254.977 0.870417 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "Trial status: 9 TERMINATED | 1 RUNNING\n", "Current time: 2025-04-10 17:55:33. Total running time: 55min 36s\n", "Logical resource usage: 1.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00008 RUNNING 256 0.00135129 8 5 681.389 0.871667 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00006 TERMINATED 32 0.0179783 16 10 738.628 0.804 |\n", "| train_model_3b16c_00007 TERMINATED 32 0.000213498 64 10 250.549 0.855583 |\n", "| train_model_3b16c_00009 TERMINATED 128 0.000180116 64 10 254.977 0.870417 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025-04-10 17:55:34,533\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=19552)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00008_8_batch_size=8,hidden_size=256,lr=0.0014_2025-04-10_16-59-57/checkpoint_000005)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 9 TERMINATED | 1 RUNNING\n", "Current time: 2025-04-10 17:56:03. Total running time: 56min 6s\n", "Logical resource usage: 1.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00008 RUNNING 256 0.00135129 8 6 765.681 0.87175 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00006 TERMINATED 32 0.0179783 16 10 738.628 0.804 |\n", "| train_model_3b16c_00007 TERMINATED 32 0.000213498 64 10 250.549 0.855583 |\n", "| train_model_3b16c_00009 TERMINATED 128 0.000180116 64 10 254.977 0.870417 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "Trial status: 9 TERMINATED | 1 RUNNING\n", "Current time: 2025-04-10 17:56:33. Total running time: 56min 36s\n", "Logical resource usage: 1.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00008 RUNNING 256 0.00135129 8 6 765.681 0.87175 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00006 TERMINATED 32 0.0179783 16 10 738.628 0.804 |\n", "| train_model_3b16c_00007 TERMINATED 32 0.000213498 64 10 250.549 0.855583 |\n", "| train_model_3b16c_00009 TERMINATED 128 0.000180116 64 10 254.977 0.870417 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=19552)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00008_8_batch_size=8,hidden_size=256,lr=0.0014_2025-04-10_16-59-57/checkpoint_000006)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 9 TERMINATED | 1 RUNNING\n", "Current time: 2025-04-10 17:57:03. Total running time: 57min 6s\n", "Logical resource usage: 1.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00008 RUNNING 256 0.00135129 8 7 851.713 0.883167 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00006 TERMINATED 32 0.0179783 16 10 738.628 0.804 |\n", "| train_model_3b16c_00007 TERMINATED 32 0.000213498 64 10 250.549 0.855583 |\n", "| train_model_3b16c_00009 TERMINATED 128 0.000180116 64 10 254.977 0.870417 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "Trial status: 9 TERMINATED | 1 RUNNING\n", "Current time: 2025-04-10 17:57:33. Total running time: 57min 36s\n", "Logical resource usage: 1.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00008 RUNNING 256 0.00135129 8 7 851.713 0.883167 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00006 TERMINATED 32 0.0179783 16 10 738.628 0.804 |\n", "| train_model_3b16c_00007 TERMINATED 32 0.000213498 64 10 250.549 0.855583 |\n", "| train_model_3b16c_00009 TERMINATED 128 0.000180116 64 10 254.977 0.870417 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "Trial status: 9 TERMINATED | 1 RUNNING\n", "Current time: 2025-04-10 17:58:03. Total running time: 58min 6s\n", "Logical resource usage: 1.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00008 RUNNING 256 0.00135129 8 7 851.713 0.883167 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00006 TERMINATED 32 0.0179783 16 10 738.628 0.804 |\n", "| train_model_3b16c_00007 TERMINATED 32 0.000213498 64 10 250.549 0.855583 |\n", "| train_model_3b16c_00009 TERMINATED 128 0.000180116 64 10 254.977 0.870417 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025-04-10 17:58:31,941\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=19552)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00008_8_batch_size=8,hidden_size=256,lr=0.0014_2025-04-10_16-59-57/checkpoint_000007)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 9 TERMINATED | 1 RUNNING\n", "Current time: 2025-04-10 17:58:33. Total running time: 58min 36s\n", "Logical resource usage: 1.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00008 RUNNING 256 0.00135129 8 8 943.065 0.880333 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00006 TERMINATED 32 0.0179783 16 10 738.628 0.804 |\n", "| train_model_3b16c_00007 TERMINATED 32 0.000213498 64 10 250.549 0.855583 |\n", "| train_model_3b16c_00009 TERMINATED 128 0.000180116 64 10 254.977 0.870417 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "Trial status: 9 TERMINATED | 1 RUNNING\n", "Current time: 2025-04-10 17:59:03. Total running time: 59min 6s\n", "Logical resource usage: 1.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00008 RUNNING 256 0.00135129 8 8 943.065 0.880333 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00006 TERMINATED 32 0.0179783 16 10 738.628 0.804 |\n", "| train_model_3b16c_00007 TERMINATED 32 0.000213498 64 10 250.549 0.855583 |\n", "| train_model_3b16c_00009 TERMINATED 128 0.000180116 64 10 254.977 0.870417 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "Trial status: 9 TERMINATED | 1 RUNNING\n", "Current time: 2025-04-10 17:59:33. Total running time: 59min 36s\n", "Logical resource usage: 1.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00008 RUNNING 256 0.00135129 8 8 943.065 0.880333 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00006 TERMINATED 32 0.0179783 16 10 738.628 0.804 |\n", "| train_model_3b16c_00007 TERMINATED 32 0.000213498 64 10 250.549 0.855583 |\n", "| train_model_3b16c_00009 TERMINATED 128 0.000180116 64 10 254.977 0.870417 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\u001b[36m(train_model pid=19552)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00008_8_batch_size=8,hidden_size=256,lr=0.0014_2025-04-10_16-59-57/checkpoint_000008)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trial status: 9 TERMINATED | 1 RUNNING\n", "Current time: 2025-04-10 18:00:03. Total running time: 1hr 0min 6s\n", "Logical resource usage: 1.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00008 with val_acc=0.8836666941642761 and params={'hidden_size': 256, 'lr': 0.0013512864268980524, 'batch_size': 8}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00008 RUNNING 256 0.00135129 8 9 1033.12 0.883667 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00006 TERMINATED 32 0.0179783 16 10 738.628 0.804 |\n", "| train_model_3b16c_00007 TERMINATED 32 0.000213498 64 10 250.549 0.855583 |\n", "| train_model_3b16c_00009 TERMINATED 128 0.000180116 64 10 254.977 0.870417 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "Trial status: 9 TERMINATED | 1 RUNNING\n", "Current time: 2025-04-10 18:00:33. Total running time: 1hr 0min 36s\n", "Logical resource usage: 1.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00008 with val_acc=0.8836666941642761 and params={'hidden_size': 256, 'lr': 0.0013512864268980524, 'batch_size': 8}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00008 RUNNING 256 0.00135129 8 9 1033.12 0.883667 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00006 TERMINATED 32 0.0179783 16 10 738.628 0.804 |\n", "| train_model_3b16c_00007 TERMINATED 32 0.000213498 64 10 250.549 0.855583 |\n", "| train_model_3b16c_00009 TERMINATED 128 0.000180116 64 10 254.977 0.870417 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "Trial status: 9 TERMINATED | 1 RUNNING\n", "Current time: 2025-04-10 18:01:03. Total running time: 1hr 1min 6s\n", "Logical resource usage: 1.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00008 with val_acc=0.8836666941642761 and params={'hidden_size': 256, 'lr': 0.0013512864268980524, 'batch_size': 8}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00008 RUNNING 256 0.00135129 8 9 1033.12 0.883667 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00006 TERMINATED 32 0.0179783 16 10 738.628 0.804 |\n", "| train_model_3b16c_00007 TERMINATED 32 0.000213498 64 10 250.549 0.855583 |\n", "| train_model_3b16c_00009 TERMINATED 128 0.000180116 64 10 254.977 0.870417 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "Trial status: 9 TERMINATED | 1 RUNNING\n", "Current time: 2025-04-10 18:01:33. Total running time: 1hr 1min 37s\n", "Logical resource usage: 1.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00008 with val_acc=0.8836666941642761 and params={'hidden_size': 256, 'lr': 0.0013512864268980524, 'batch_size': 8}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00008 RUNNING 256 0.00135129 8 9 1033.12 0.883667 |\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00006 TERMINATED 32 0.0179783 16 10 738.628 0.804 |\n", "| train_model_3b16c_00007 TERMINATED 32 0.000213498 64 10 250.549 0.855583 |\n", "| train_model_3b16c_00009 TERMINATED 128 0.000180116 64 10 254.977 0.870417 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2025-04-10 18:01:35,929\tWARNING experiment_state.py:206 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds and may become a bottleneck. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.\n", "You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.\n", "You can suppress this warning by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0). Set it to 0 to completely suppress this warning.\n", "\u001b[36m(train_model pid=19552)\u001b[0m Checkpoint successfully created at: Checkpoint(filesystem=local, path=/root/ray_results/train_model_2025-04-10_16-59-56/train_model_3b16c_00008_8_batch_size=8,hidden_size=256,lr=0.0014_2025-04-10_16-59-57/checkpoint_000009)\n", "\u001b[36m(train_model pid=19552)\u001b[0m `Trainer.fit` stopped: `max_epochs=10` reached.\n", "2025-04-10 18:01:36,125\tINFO tune.py:1009 -- Wrote the latest version of all result files and experiment state to '/root/ray_results/train_model_2025-04-10_16-59-56' in 0.0097s.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "Trial train_model_3b16c_00008 completed after 10 iterations at 2025-04-10 18:01:36. Total running time: 1hr 1min 39s\n", "+------------------------------------------------------------+\n", "| Trial train_model_3b16c_00008 result |\n", "+------------------------------------------------------------+\n", "| checkpoint_dir_name checkpoint_000009 |\n", "| time_this_iter_s 93.92165 |\n", "| time_total_s 1127.03847 |\n", "| training_iteration 10 |\n", "| val_acc 0.87842 |\n", "+------------------------------------------------------------+\n", "\n", "Trial status: 10 TERMINATED\n", "Current time: 2025-04-10 18:01:36. Total running time: 1hr 1min 39s\n", "Logical resource usage: 1.0/2 CPUs, 0/0 GPUs\n", "Current best trial: 3b16c_00000 with val_acc=0.8834999799728394 and params={'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| Trial name status hidden_size lr batch_size iter total time (s) val_acc |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "| train_model_3b16c_00000 TERMINATED 256 0.00078507 16 10 903.287 0.8835 |\n", "| train_model_3b16c_00001 TERMINATED 16 0.00371585 64 10 252.716 0.851167 |\n", "| train_model_3b16c_00002 TERMINATED 256 0.0238983 64 10 310.967 0.826333 |\n", "| train_model_3b16c_00003 TERMINATED 64 0.00639545 8 10 1375.65 0.852167 |\n", "| train_model_3b16c_00004 TERMINATED 32 0.0721965 16 10 639.052 0.159417 |\n", "| train_model_3b16c_00005 TERMINATED 64 0.00226128 16 10 670.455 0.876333 |\n", "| train_model_3b16c_00006 TERMINATED 32 0.0179783 16 10 738.628 0.804 |\n", "| train_model_3b16c_00007 TERMINATED 32 0.000213498 64 10 250.549 0.855583 |\n", "| train_model_3b16c_00008 TERMINATED 256 0.00135129 8 10 1127.04 0.878417 |\n", "| train_model_3b16c_00009 TERMINATED 128 0.000180116 64 10 254.977 0.870417 |\n", "+---------------------------------------------------------------------------------------------------------------------------+\n", "\n" ] } ], "source": [ "tuner = Tuner(\n", " trainable_with_resources,\n", " param_space=search_space,\n", " tune_config=TuneConfig(\n", " metric=\"val_acc\",\n", " mode=\"max\",\n", " num_samples=10,\n", " ),\n", " run_config = RunConfig(\n", " checkpoint_config=CheckpointConfig(\n", " num_to_keep=1,\n", " checkpoint_score_attribute=\"val_acc\",\n", " checkpoint_score_order=\"max\"\n", " ))\n", ")\n", "\n", "results = tuner.fit( )" ] }, { "cell_type": "markdown", "id": "4b3def2c-ce0e-49ad-93aa-a41b6658e765", "metadata": { "id": "4b3def2c-ce0e-49ad-93aa-a41b6658e765" }, "source": [ "After the tuning is done, we will read out the best parameters, and the validation loss and accuracy from that trial." ] }, { "cell_type": "code", "execution_count": null, "id": "txZiI30z2jD_", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "txZiI30z2jD_", "outputId": "8ad6258e-d479-4564-c330-d8fa7da9fd79" }, "outputs": [ { "data": { "text/plain": [ "{'hidden_size': 256, 'lr': 0.0007850701267804025, 'batch_size': 16}" ] }, "execution_count": 28, "metadata": {}, "output_type": "execute_result" } ], "source": [ "best_result = results.get_best_result(metric=\"val_acc\", mode=\"max\")\n", "best_result.config" ] }, { "cell_type": "code", "execution_count": null, "id": "35aDqyK97ZXQ", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "35aDqyK97ZXQ", "outputId": "adeedf8b-faba-43ba-d2ea-2066a89de0b0" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Best trial final validation accuracy: 0.8834999799728394\n" ] } ], "source": [ "print(f\"Best trial final validation accuracy: {best_result. metrics['val_acc']}\")" ] }, { "cell_type": "markdown", "id": "GuVXPTFDTZ5b", "metadata": { "id": "GuVXPTFDTZ5b" }, "source": [ "Now, if we want to use the model with the best parameters, we will load its parameters from the checkpoint file, as we already defined we only keep the model from the epoch with highest validation accuracy." ] }, { "cell_type": "code", "execution_count": null, "id": "dr3b3LVzRx5A", "metadata": { "id": "dr3b3LVzRx5A" }, "outputs": [], "source": [ "best_checkpoint_path=best_result.path\n", "checkpoint_folder = [f for f in os.listdir(best_checkpoint_path) if \"checkpoint\" in f][0]\n", "best_model = MNISTModel.load_from_checkpoint(best_checkpoint_path+'/'+checkpoint_folder+'/'+'tune-checkpoint.ckpt')" ] }, { "cell_type": "markdown", "id": "245a5a0b-6622-46ec-a91f-227d1409d755", "metadata": { "id": "245a5a0b-6622-46ec-a91f-227d1409d755" }, "source": [ "Now, let's evaluate the performance of the best model on the test set. Note that we tuned the batch size for model training. However, batch size for the test set does not impact its performance, since the weights are fixed after training, it only impacts the speed of inference, as larger batch size leads to faster evaluation, but increases memory usage." ] }, { "cell_type": "code", "execution_count": null, "id": "SpeA-kZU2_s0", "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 236, "referenced_widgets": [ "17b32bb6e2034b82a7d71d06478fec95", "da75d186211847b6b2056b5dbea3ae4c", "995443f0e8eb49bcaa920b08b566fae4", "1b1110ce0c184b108e1f194b9f3144b3", "ad3f8b3f273a4af8aa34a1b6fae96e5e", "d8cd077e4d8644b49adc7b1c6cfe1915", "104e7496489049a0a7343a8f423904de", "94253e895c68406c884a698bcb082234", "91b82484785a4a668f8371fd6e273e1b", "3885c5e58e6344eca57a4aa3de725b83", "5d735c747ef240a5bf936747d4af1fcb" ] }, "id": "SpeA-kZU2_s0", "outputId": "be25f1e3-23f1-4944-a730-913b53a65665" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "INFO:pytorch_lightning.utilities.rank_zero:You are using the plain ModelCheckpoint callback. Consider using LitModelCheckpoint which with seamless uploading to Model registry.\n", "INFO:pytorch_lightning.utilities.rank_zero:GPU available: False, used: False\n", "INFO:pytorch_lightning.utilities.rank_zero:TPU available: False, using: 0 TPU cores\n", "INFO:pytorch_lightning.utilities.rank_zero:HPU available: False, using: 0 HPUs\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "17b32bb6e2034b82a7d71d06478fec95", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Testing: | | 0/? [00:00┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n", "┃ Test metric ┃ DataLoader 0 ┃\n", "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n", "│ test_acc │ 0.8788999915122986 │\n", "└───────────────────────────┴───────────────────────────┘\n", "\n" ], "text/plain": [ "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n", "┃\u001b[1m \u001b[0m\u001b[1m Test metric \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m DataLoader 0 \u001b[0m\u001b[1m \u001b[0m┃\n", "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n", "│\u001b[36m \u001b[0m\u001b[36m test_acc \u001b[0m\u001b[36m \u001b[0m│\u001b[35m \u001b[0m\u001b[35m 0.8788999915122986 \u001b[0m\u001b[35m \u001b[0m│\n", "└───────────────────────────┴───────────────────────────┘\n" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [ "[{'test_acc': 0.8788999915122986}]" ] }, "execution_count": 31, "metadata": {}, "output_type": "execute_result" } ], "source": [ "transform = transforms.Compose(\n", " [transforms.ToTensor()])\n", "test_set = datasets.FashionMNIST('./data', train=False, transform=transform, download=True)\n", "test_loader = DataLoader(test_set, batch_size=128, num_workers=4, persistent_workers=True)\n", "trainer = pl.Trainer()\n", "trainer.test(best_model, dataloaders=test_loader)" ] } ], "metadata": { "accelerator": "GPU", "colab": { "gpuType": "T4", "provenance": [], "toc_visible": true }, "kernelspec": { "display_name": "ml2025", "language": "python", "name": "ml2025" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.8" }, "widgets": { "application/vnd.jupyter.widget-state+json": { "01de57f75ac14b35b664691111c87324": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%" } }, "020ddcb8c2974493b3ef5dbf064f5aa7": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "02327c540e4240ad93f69159a586a695": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "026899f8dfd540eba8b3808137d633ae": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "026f844cf3cf4753b2995b1ccfa9d701": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_3e1ee0c4b88e43c4b159a228492f19bf", "IPY_MODEL_11b4e8f5d295479dba05dd4892d48909", "IPY_MODEL_2655dda6711644e7a2d2211d6ea15fee" ], "layout": "IPY_MODEL_d581b3cb4f5c4932be8862be509d97f9" } }, "030f1eec8eb94fa3856885a0a0391a7b": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "04544f5d24fb456aa9b9997469e7968d": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%" } }, "052bf7dce8c64f74afde401d6db3bcbd": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "05344f36aa5f4141b3a25c5e6c30f518": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "053fdebd6262410aa47e339b85e4cc7d": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_4fff93b748fe4c0cb5276b6bae2a1bda", "max": 157, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_ceaea24c3f6e4507b304441b6771f4d3", "value": 157 } }, "0548fb9ec6b04228bb98a8227f0af719": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "064e3ee7eab74e2397fc3da137d444bb": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "07304a89464a49ad85a801a6588badf8": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_120f3943f62e4f7ba46802e1c81819e5", "placeholder": "​", "style": "IPY_MODEL_02327c540e4240ad93f69159a586a695", "value": "Validation DataLoader 0: 100%" } }, "07b85b31f0ee4c568618b9e9fbb4c929": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_543aeec44aa5464e976988c64da40a6e", "placeholder": "​", "style": "IPY_MODEL_ffb0411f39e64b8fb6506e650eca2a6d", "value": " 157/157 [00:10<00:00, 15.14it/s]" } }, "07cf8f8336874708a0b92c2311efc94e": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "080b50127a6345e6865f98ff06d66e9f": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "081df58e447c4b7c8cc34600b5d3c923": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_5718a429b8384387807d735641c3ea1f", "placeholder": "​", "style": "IPY_MODEL_afdce55760c4431dbb0dd4bd9827aad9", "value": " 188/188 [00:03<00:00, 58.72it/s]" } }, "08b4788623974884bd4512be6222538f": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": "100%" } }, "08bf3aae048240f286810a23a57ab6ce": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_50df2093ef3a4575988f8f255c58002a", "placeholder": "​", "style": "IPY_MODEL_8b38ad55167246b3976dcee637b8b982", "value": "Epoch 14: 100%" } }, "09964e13315444668b8ef036621fe3ac": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "099d93675b344a448790fd638e8cb5c5": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "0a1d99965825404aa13980a984fe6fe1": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "0c411d3a002543c4a741207ac56f4824": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_93153ab1a75b41bba6bbd57f30f8750b", "IPY_MODEL_e095049b602048c99d8d1dd8552fcfe9", "IPY_MODEL_dcbae4193030459081a305c51d52e4e7" ], "layout": "IPY_MODEL_9854b37a6206460793b035b1b43ecb5a" } }, "0e5f5f15321c4b0aab696bf95f431bd8": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "0eaa31608dab4a64aba2a09b8200200b": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "0f50c49a81fb4c7ea8a30ba5c33a242c": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_72f4c55eb4ab4746a40b88ad6a6be877", "IPY_MODEL_dc9e789d02804f3d876cad49d752daaf", "IPY_MODEL_081df58e447c4b7c8cc34600b5d3c923" ], "layout": "IPY_MODEL_1b78304fe9ae4ab6bc8a92e782229685" } }, "0f6c7b386ca848389ed069e8df65ac51": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "0fc154788e684df8a130caf7b17b7525": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "104e7496489049a0a7343a8f423904de": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "1090c5c6bc654c6f9cdc0f72e33b85b3": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_c936e1097b1245e6a4a325a472133874", "placeholder": "​", "style": "IPY_MODEL_40852b870a4649a186fe0c71cad0a234", "value": "Sanity Checking DataLoader 0: 100%" } }, "109402bb49874e108ee3f3b09ba37b33": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "10d41feda0be433b950916933eadbd74": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_a68dcf701662428787d82111a3d37c1a", "IPY_MODEL_79cd96c7295d48a98101d8473fde6044", "IPY_MODEL_f740a916dfb440428b78251df30599f0" ], "layout": "IPY_MODEL_5c402ddbe66741e4a02f6527ae73fa42" } }, "1127e918c45f4ebe8f94224e1bc4c107": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "119e0bef11194cf3940b6f095c7f72d2": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "11b4e8f5d295479dba05dd4892d48909": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_e90c40de39c84c57bba7213db55188df", "max": 157, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_2d7ca0e3646a458a89fac34134580e0c", "value": 157 } }, "120f3943f62e4f7ba46802e1c81819e5": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "12af8881c6354f71abbe13a61ec30cbf": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "12e51cc17e204feab9a269642ee418b7": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "13525fee791b46aabbf912aff988a017": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "13f7be10f92f4c109a94b36ff1e2c451": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_5177ff4d431345078fed6720d3495903", "max": 2, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_9b5707edf0b94b5b8c12681ccda04b80", "value": 2 } }, "14300c7dd14146d2afa35d5a83f03024": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_080b50127a6345e6865f98ff06d66e9f", "placeholder": "​", "style": "IPY_MODEL_fa3c6d7a0eac4c3e98c275fc2dbab212", "value": "Testing DataLoader 0: 100%" } }, "1430e9b8e2934d2e91e45b356aaa6fb0": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_9c5e867f28f0484d839c280d08f58b41", "placeholder": "​", "style": "IPY_MODEL_4317be8e910d4488bf9d2e446fda5442", "value": "Validation DataLoader 0: 100%" } }, "14c62c68634946079e92b0443e309ca2": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%" } }, "14c999697262418fb144c1e3ffed5ad4": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_8c16363b28884d07b4d21c5bf903a575", "IPY_MODEL_7800fbc10e1c48888b3dea7ce30c7316", "IPY_MODEL_fbcbdc0774e141dcadb7897303965fb7" ], "layout": "IPY_MODEL_255c48c8d5dd4850b3af2863c2578028" } }, "15a8c8397d34449e9c5b52d170c089b1": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "161f4e5c97734ad89bef49fa8f9d6bc1": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "1636d928c41742448cfe0cde7401d67d": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_4e3e6efcdb0247f6942ec0ed84745d7f", "IPY_MODEL_cf36208b80a94aa48e418c9c8401e322", "IPY_MODEL_2a83d0800a01460594dcc543a845c183" ], "layout": "IPY_MODEL_2f281a3813604a688bac66766044e211" } }, "17a0a9e061ab4525b90ef33ac3d4b72f": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "17b32bb6e2034b82a7d71d06478fec95": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_da75d186211847b6b2056b5dbea3ae4c", "IPY_MODEL_995443f0e8eb49bcaa920b08b566fae4", "IPY_MODEL_1b1110ce0c184b108e1f194b9f3144b3" ], "layout": "IPY_MODEL_ad3f8b3f273a4af8aa34a1b6fae96e5e" } }, "17c74e62e20e42e1b27a6ab5df0d8e26": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "17cf42bb46984789a70568f2306e7001": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "18b7ef59edf14b10ac2135a23331812a": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "18d5a8adae3c4770a3b1608c2381ec2c": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "1b1110ce0c184b108e1f194b9f3144b3": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_3885c5e58e6344eca57a4aa3de725b83", "placeholder": "​", "style": "IPY_MODEL_5d735c747ef240a5bf936747d4af1fcb", "value": " 79/79 [00:01<00:00, 42.92it/s]" } }, "1b78304fe9ae4ab6bc8a92e782229685": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%" } }, "1e2908b2752c4290942a00c41f14acb9": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_d340af6a5d4249b2a6449b489888f483", "IPY_MODEL_2587d30d2a694e02aa5ccd0be498204c", "IPY_MODEL_55af8a65276e410cadffa20cb08bbf81" ], "layout": "IPY_MODEL_502d84ffc424432eba0d29e315a636de" } }, "1e4fe4d1038b4f7f91499cc793c28321": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_918b50556def45fba40a7f6d673f6a09", "IPY_MODEL_f7a583b7d9c24fc290de4f6de44f8321", "IPY_MODEL_9f40085152dc4afb8b73bbda14159334" ], "layout": "IPY_MODEL_45d6a2c7dd4f47dbb99562dcd6955250" } }, "1f9583df3f81471e83bf33f55ac9a990": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_2ab47c175ed14c77892fc4509e5b7b55", "placeholder": "​", "style": "IPY_MODEL_acfbdcfb31394e3496ab75c7e6c0a8c5", "value": " 750/750 [00:10<00:00, 71.49it/s, v_num=2, train_loss=0.254]" } }, "20032203372344d59e59f030eac46f52": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "2007c0564fbb4758899075f09181ae3b": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "21ac563ed5474ab79e1c33756dca7e50": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "225855b374144cabaec2a9f18a69b64a": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%" } }, "22a5b300c7c344f0b4a80c07f641b1a6": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "22c761c949834cbcaa19ccbfde70fbb3": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "2443930de8284148a7ed0ff64cab9b75": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_8804fff7f53540299db7371d1a65363b", "placeholder": "​", "style": "IPY_MODEL_9bd6317622fb488db10c34d028559fd9", "value": " 157/157 [00:11<00:00, 14.23it/s]" } }, "251b15cd84f7403a9343bee611369f6d": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "253755895c0c49a58f39f5267ec5744a": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_4712494388bb4f5d9f4de160c9fa5244", "max": 188, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_ba1be115b69643b4aa876309eb31f09f", "value": 188 } }, "255c48c8d5dd4850b3af2863c2578028": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%" } }, "257ab33f9aa54f2591244ee2f6a9c7d7": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_5697065ca3a8452ea04f7eb3ea425a5c", "placeholder": "​", "style": "IPY_MODEL_66d983c505884401b827b457b2f76adc", "value": "Testing DataLoader 0: 100%" } }, "2587d30d2a694e02aa5ccd0be498204c": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_a2ef959ead414ce1ad692ce4c5a1ebf5", "max": 188, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_f5fd0185784d4b89b871e734a2efdfd2", "value": 188 } }, "25e6a6604f8945248381e22498912592": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_1430e9b8e2934d2e91e45b356aaa6fb0", "IPY_MODEL_ad9f4e32300f4406a9f5189044cae2ee", "IPY_MODEL_3d8ee79c5d184c049ad943dc24f0db45" ], "layout": "IPY_MODEL_8c7bc6b2ea2342a2b20a8d6419c1ae9d" } }, "2655dda6711644e7a2d2211d6ea15fee": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_8746ad0c90964b3f93af266a36938393", "placeholder": "​", "style": "IPY_MODEL_030f1eec8eb94fa3856885a0a0391a7b", "value": " 157/157 [00:10<00:00, 15.49it/s]" } }, "26699a4bafba4da486e03e3f673c73c5": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "26d47c45fc7747b8b4b5514c572b05b6": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "2748e2a9625b4e72a051d704a80e7ee4": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "28af3c2c45dd4554949ee2099aab336c": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%" } }, "2a83d0800a01460594dcc543a845c183": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_a90892d849cd4e6aa6d632a2ce7be940", "placeholder": "​", "style": "IPY_MODEL_9d3e2878bcfc472b9954e2cfe66e6bd0", "value": " 188/188 [00:02<00:00, 78.12it/s]" } }, "2ab47c175ed14c77892fc4509e5b7b55": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "2ad2e85b1f9a4f1e9e7582301dcef5bd": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "2ba9e3c6007a4dc880ec4757de48a69b": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "2c16d456a401438d80dafbba28deb9da": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "2c7efd8bd13b46348cb47cf6ac17b2e1": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_6278c96f595540908667778e749dea0e", "IPY_MODEL_7e812e59e93046e587395a74553d64c9", "IPY_MODEL_2ff41ac3750b46d8a8fecbbaeaa5a686" ], "layout": "IPY_MODEL_e1dc62bffeaf433f919405f27a5de450" } }, "2d7ca0e3646a458a89fac34134580e0c": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "2f281a3813604a688bac66766044e211": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%" } }, "2f5794667f664d28b826ca1db413b843": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "2f82090308c2469ea2bfc13d354a11be": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "2ff41ac3750b46d8a8fecbbaeaa5a686": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_026899f8dfd540eba8b3808137d633ae", "placeholder": "​", "style": "IPY_MODEL_e0f92f8764984229856f8f40cd3c3ebc", "value": " 157/157 [00:10<00:00, 15.39it/s]" } }, "3012d6e77a544de5a13c9355a993d935": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_d7e73012bf44455db890cefa226f4ed6", "IPY_MODEL_578833b8285246c9bd4d021d1430c13e", "IPY_MODEL_af9891a0ce674f21a4d5c0f3162beef6" ], "layout": "IPY_MODEL_d9748c38ccda41ac94a034dc6d21826b" } }, "3044a6ff3d324f0a802157ea2e4aa3ab": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_6bbb4911ba9548f1a276bd6b6b05c566", "max": 188, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_e1b2999baf654e52a7fd0a3961c90d6d", "value": 188 } }, "306c26e12ef2438ead540532792ed165": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "309453e78b0b409f839117ba156d7e3a": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_1090c5c6bc654c6f9cdc0f72e33b85b3", "IPY_MODEL_decba3296d474b8282f988401495487f", "IPY_MODEL_f895a99a7f574d8f9c97b60dcbadf497" ], "layout": "IPY_MODEL_b8b9dab360ae4dbf990ebd3053f45794" } }, "31a6eae7f3ec40849d57e9f6d11bc7af": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_84973c4191734c179409fd12c58e357a", "placeholder": "​", "style": "IPY_MODEL_4d999b0472014da3a9b46c228d1fce12", "value": " 40/40 [00:00<00:00, 101.55it/s, v_num=0, train_loss=9.240]" } }, "31b28cd4fae74a2b97bad887e3eb80a0": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_728aae7b6ea6444a8f6b0d90ef226227", "IPY_MODEL_4b5ebb81686c48f9a73c499ca760f363", "IPY_MODEL_4839d58911154c8f837977abe16d46d2" ], "layout": "IPY_MODEL_57d415032ed943849994248d7b11805a" } }, "31ea9d1a96c2488792e65dba04c2d747": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "321461735c1d4acb863914733f4f29a1": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "32d0b577656e4db1bba3c7ec1dd4b14e": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_45d673640dba4b1b8de3c44d3b9c6dff", "placeholder": "​", "style": "IPY_MODEL_872bf623b12e47e392be655cc0c1f4f1", "value": "Validation DataLoader 0: 100%" } }, "33546ec140a441cbb25c5300b986347f": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "335f062e448b4754bacc27528430781b": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "342e8b4b20774be9a2fa68eaa0921bbe": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "353c41034cdd4320ab78a0794520260c": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_e6c06a7218714c2ba1b79f3611b1bc44", "placeholder": "​", "style": "IPY_MODEL_8521faf1a6934b1b9d3fd7635bf50c6f", "value": "Validation DataLoader 0: 100%" } }, "357a9216989c492d97b09b931478a616": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "3686607130864dba852689774f6f5ccd": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "36949c9ea04348cab86fec20876876f9": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_6c696e3e8973465a94241c198ba451d6", "IPY_MODEL_94562514188d4d00b185a9f1074f92bd", "IPY_MODEL_81d35d9ef9d840bfae39787980ae162d" ], "layout": "IPY_MODEL_4d89064ba55b4a3b9bf9d53b15c00502" } }, "374298183b3d4096a8739529cfbf6933": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_14300c7dd14146d2afa35d5a83f03024", "IPY_MODEL_e4019cc439d647a8b447367827b44e4d", "IPY_MODEL_b7d69f00abf44aa5908b7543b0780b1c" ], "layout": "IPY_MODEL_08b4788623974884bd4512be6222538f" } }, "3810b69aa3184ccb91d8502c015dce6c": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "3885c5e58e6344eca57a4aa3de725b83": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "39e66ed2644742a0b0d3c1ef0307f463": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "3ae8014f7d6b4bbe92b00ebeb0675c09": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "3b71b27cf5bd4d5e8a0962c646144238": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_55ce4f9585e54cc68211559bc392df1b", "placeholder": "​", "style": "IPY_MODEL_8fd75a6f4cea4ef7bc7e641777e3c554", "value": " 188/188 [00:01<00:00, 101.89it/s]" } }, "3bc766accb9b46f4a64256b0ee748dfa": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "3c162e4b29c84f5ca2a3243f35d59641": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%" } }, "3d5134fb97ca43de90f67e7a833aebd9": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "3d8ee79c5d184c049ad943dc24f0db45": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_cbc951497bea49bb8617bb265148db3a", "placeholder": "​", "style": "IPY_MODEL_578c2fba93a542b1a401a38852177dbd", "value": " 188/188 [00:03<00:00, 50.12it/s]" } }, "3d99763d11d44de582b43729f8ce0058": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": "100%" } }, "3dc22a5addd74a9ebb99bcd0a8942cb4": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "3e1321c4048740979b952685101ee3a2": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "3e1ee0c4b88e43c4b159a228492f19bf": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_613e7eb36f6f457d8c95a6754d74a8c7", "placeholder": "​", "style": "IPY_MODEL_22c761c949834cbcaa19ccbfde70fbb3", "value": "Validation DataLoader 0: 100%" } }, "3e2b4e7404af40b7afb610cca201e71c": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_c44a38fdfc5246668f1e2ef9a1281747", "placeholder": "​", "style": "IPY_MODEL_4c7877887f8542f0910796a8bd913a5f", "value": "Validation DataLoader 0: 100%" } }, "3f10e06684d0488db866f8875daa132c": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_3e1321c4048740979b952685101ee3a2", "placeholder": "​", "style": "IPY_MODEL_5844fad7feca4caba721c106061613f7", "value": "Epoch 9: 100%" } }, "3f21584337fe40e18100e9f4b2bbb45d": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "403ef89515f1429f9adf3603536c22f1": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_15a8c8397d34449e9c5b52d170c089b1", "placeholder": "​", "style": "IPY_MODEL_a555413eee4a404781069cd27cc2353d", "value": "Validation DataLoader 0: 100%" } }, "40852b870a4649a186fe0c71cad0a234": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "40e69c44b12841d18b93d292a7a17b4f": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%" } }, "41e5f578f018446094246f6f7ea62959": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_08bf3aae048240f286810a23a57ab6ce", "IPY_MODEL_de153318bf764d0680b7756a090f30c8", "IPY_MODEL_eee6719b6a3942ff8eed17671b90d10c" ], "layout": "IPY_MODEL_5fe7490849734721b89511036558047a" } }, "429607e5cad94841902812a8f67e0937": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "4317be8e910d4488bf9d2e446fda5442": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "43c8c9ebf96a46679497d85c218b3ed2": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_1127e918c45f4ebe8f94224e1bc4c107", "placeholder": "​", "style": "IPY_MODEL_dd8a1ed1a0cc47a1844fbce14f5b4619", "value": " 188/188 [00:04<00:00, 42.51it/s]" } }, "43f412f3c8224f7b8a7aa666ffeb5da3": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%" } }, "4406a71e0da14d659937b464de933045": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_e48b05e26b2a4befa93a908f3e7b7947", "placeholder": "​", "style": "IPY_MODEL_d9b8f3663bc340639c61a44f2f038eb5", "value": "Testing DataLoader 0: 100%" } }, "444777773e9d4c42830a97148913445e": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_a4a871dee0b744d89067df5bd70ce0ec", "IPY_MODEL_e20581a183bc41e598f34f50dd4e55ef", "IPY_MODEL_b017226b497844db9dd1d95aedc9fcd5" ], "layout": "IPY_MODEL_e4f75da22fe54a90a2be9edf6ca4ca43" } }, "44651b4e3e2a40ff84ae20c275a27b94": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "450f3a364264429288012140c6458ba3": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "45d673640dba4b1b8de3c44d3b9c6dff": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "45d6a2c7dd4f47dbb99562dcd6955250": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%" } }, "4712494388bb4f5d9f4de160c9fa5244": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "477b20e171b744518f2691d343188714": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_a4438910cc994c67b39f2888afdecc4c", "max": 157, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_71f46192712f4bacb9ecb4aeb49b2bed", "value": 157 } }, "4839d58911154c8f837977abe16d46d2": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_ab3c4c6b11ef4a9ea4e0ae95fbc3fcd7", "placeholder": "​", "style": "IPY_MODEL_7595b08f182748c1bf291bb7e87a4026", "value": " 157/157 [00:10<00:00, 14.39it/s]" } }, "486351e0ff0f42e9836dd953f39a9d63": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_c2482b58f84e4ce3aa69990ae5b88783", "max": 157, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_7f43dfbaeb66467690691f944a5ef1e1", "value": 157 } }, "49e3239d4f104f92af2d00d01f0a76e7": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_94cfd4cd1bff47aab343b287f74a2d55", "max": 157, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_a3f8d90bf76b48259c4f7583af4c578e", "value": 157 } }, "4a6d32c082cc482c8f4c4991ef7cc5e2": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "4a7ddad69238464caeac96bf27616795": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_6977852255e840f0958d1ac1f9985747", "IPY_MODEL_f7361a84df9341ae81113eefc565fd6f", "IPY_MODEL_b1750c0a227340c3b9223f8b7a22837c" ], "layout": "IPY_MODEL_9ad2b3dd5dbe4f02afab448c4e3590e5" } }, "4abcbee49e9b4f6fa20945f319e8fc8a": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_44651b4e3e2a40ff84ae20c275a27b94", "max": 188, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_c5737fc6068d44649723b46603559da8", "value": 188 } }, "4b04920b8f784bb489fd352eecf74faa": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "4b5ebb81686c48f9a73c499ca760f363": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_b4abb63a4f8b49419c1c7bb0ebe2fcb2", "max": 157, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_63d8c4fbd2c949369ef7c0a8dc521262", "value": 157 } }, "4be7367744834d5cabcc2f02710f686e": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_94cf9f53d8224cd781fb345b8c4df6c2", "IPY_MODEL_8089561b2d8e49c1bc6b51ca33824556", "IPY_MODEL_9254d3bbd2494eb7aceaf1f58c425b70" ], "layout": "IPY_MODEL_d89e2268f03345f0b315b3c96b1c1a8f" } }, "4c7877887f8542f0910796a8bd913a5f": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "4d2b5f4881924960b3cd760d13419998": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_ccffac78d24b4b9e8566d7dfca09cbc0", "max": 782, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_6eb08bee2fb44dce91845beea5046dda", "value": 782 } }, "4d7bde4f23e64616b66e2ca00862c201": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "4d89064ba55b4a3b9bf9d53b15c00502": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%" } }, "4d9514a4fec549b9b1bcb1600c70ce80": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "4d999b0472014da3a9b46c228d1fce12": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "4dbb2d40666045a9939db57b5c35904b": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "4dde3401e64d4a3b8eac548cdbbb5626": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "4e3e6efcdb0247f6942ec0ed84745d7f": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_2f82090308c2469ea2bfc13d354a11be", "placeholder": "​", "style": "IPY_MODEL_7c26402ac7784a5c8e3e3f181d829538", "value": "Validation DataLoader 0: 100%" } }, "4ee347f24ff24cbaa172d96a5e87cc5c": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "4fff93b748fe4c0cb5276b6bae2a1bda": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "5003c39b9f134d3b806469e0ded2e69f": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "502d84ffc424432eba0d29e315a636de": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%" } }, "50827632c1804f0e8c22023c63e655b6": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "50df2093ef3a4575988f8f255c58002a": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "5177ff4d431345078fed6720d3495903": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "51c98590ad9a43a9980dc3596b3fd9d6": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "51ffaf781dae41c4bb6c65a11210d5ba": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_0e5f5f15321c4b0aab696bf95f431bd8", "max": 10, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_8531be16189a4f7ca5a697cd22a515e4", "value": 10 } }, "526e3d1b822347e4969898926dccf66a": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_64e81873af0449b99c9417fcaf68175e", "max": 157, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_6a285b553a1849c7a5414882fd1e011e", "value": 157 } }, "52e3286dfe334a77bfe1ac68e5ef6775": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "535e920fb8c942f59b30cfc20483397c": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "53b5a097f3644889a8d64acc9685d3b0": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "543aeec44aa5464e976988c64da40a6e": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "54bf090e91494f1481d935cfd96e6c7d": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "55af8a65276e410cadffa20cb08bbf81": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_3810b69aa3184ccb91d8502c015dce6c", "placeholder": "​", "style": "IPY_MODEL_7f6b7ce508254497ac4170d76254615a", "value": " 188/188 [00:04<00:00, 44.46it/s]" } }, "55ce4f9585e54cc68211559bc392df1b": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "56787584d42f4982ba55fbf03a517d9e": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_6e1e72f4008346f8995d5950b56752d9", "placeholder": "​", "style": "IPY_MODEL_69ec8af183064d149451015c5715dbb7", "value": " 188/188 [00:03<00:00, 58.57it/s]" } }, "5697065ca3a8452ea04f7eb3ea425a5c": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "5718a429b8384387807d735641c3ea1f": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "5759a46f103e4133997e113e44233ef5": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_847dbf7ff7ab4931bbb0b271f9a74d3f", "max": 2, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_c470c831a38a4f379836fdbfde58089d", "value": 2 } }, "5781187ea9a740d98dda6f3fbacc8737": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_257ab33f9aa54f2591244ee2f6a9c7d7", "IPY_MODEL_51ffaf781dae41c4bb6c65a11210d5ba", "IPY_MODEL_917deda0fee94af18bd7107fb350aa6c" ], "layout": "IPY_MODEL_3d99763d11d44de582b43729f8ce0058" } }, "578833b8285246c9bd4d021d1430c13e": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_020ddcb8c2974493b3ef5dbf064f5aa7", "max": 188, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_8feb44c4e6024da8904c7de5be4a719f", "value": 188 } }, "578c2fba93a542b1a401a38852177dbd": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "57d415032ed943849994248d7b11805a": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%" } }, "5844fad7feca4caba721c106061613f7": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "5a254bebeb2d41f3befb6ef0efc3ed7a": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_099d93675b344a448790fd638e8cb5c5", "placeholder": "​", "style": "IPY_MODEL_d887a20ea9fd4e0b87c97c5d4bd6484e", "value": " 157/157 [00:10<00:00, 14.91it/s]" } }, "5a2bd91a27ba409b9980cac971db5ba0": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_af22275aa51b41d4a8a0c093d16872f4", "placeholder": "​", "style": "IPY_MODEL_54bf090e91494f1481d935cfd96e6c7d", "value": " 188/188 [00:02<00:00, 68.34it/s]" } }, "5aba43fc321d49399f868fecdb0da9a1": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "5ae6d72b5c77491fb79d2432bedc2851": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "5b5216781d6d4008a8217e6adde5fbe3": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "5bcf39a25f814726a6d355141d719e55": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_f6243fce8c9b49758d7abe02d709b4e0", "placeholder": "​", "style": "IPY_MODEL_6521b00b305b4b2d8023320fcbccdaa3", "value": " 188/188 [00:01<00:00, 103.57it/s]" } }, "5c29713940094ff3a0e73054dcd80de8": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_d9633cd7d7b149dab214496d1fd2f876", "IPY_MODEL_73b3c34a07b84d268ba8d20eef65cc1a", "IPY_MODEL_3b71b27cf5bd4d5e8a0962c646144238" ], "layout": "IPY_MODEL_04544f5d24fb456aa9b9997469e7968d" } }, "5c402ddbe66741e4a02f6527ae73fa42": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": "100%" } }, "5cade6652ee04d139f9f801d0467cc7f": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%" } }, "5d6adabf50e047d3897a5265248953b4": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "5d735c747ef240a5bf936747d4af1fcb": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "5e4c87a8903a4e56a1072021016f58f8": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "5eddda5ad6c14519a8ecbfaf97299f16": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_d73eb94946924b09bfd4e541385db470", "placeholder": "​", "style": "IPY_MODEL_0a1d99965825404aa13980a984fe6fe1", "value": " 188/188 [00:03<00:00, 57.91it/s]" } }, "5fe7490849734721b89511036558047a": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": "100%" } }, "603e4d9032be4d139d033dd2f5db5572": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "613e7eb36f6f457d8c95a6754d74a8c7": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "61c5c53594ad47e2adafa99dcd0e35ae": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "61fe68e00f8d40baa2dccd882b725f21": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "6278c96f595540908667778e749dea0e": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_2f5794667f664d28b826ca1db413b843", "placeholder": "​", "style": "IPY_MODEL_a5f611dbbe774f18a7abd07a06e7d1aa", "value": "Validation DataLoader 0: 100%" } }, "62d50bd4188c470484da2808a0e0987f": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_b72261e955984b7ab00aa09ec91eb859", "placeholder": "​", "style": "IPY_MODEL_13525fee791b46aabbf912aff988a017", "value": "Validation DataLoader 0: 100%" } }, "633e575d2a904eadb3836d9221911746": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "63680227a1344f589f74852ee161b58c": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "63c614b65ba34979bdceb203598eabac": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%" } }, "63d8c4fbd2c949369ef7c0a8dc521262": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "6465c4379d0a4a6f8b66ae32a86164f1": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "647fa4607d21458e83718a79e2a50a39": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "64b137dc1d4844b9bfdc4e6b5458d8a2": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "64e81873af0449b99c9417fcaf68175e": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "6521b00b305b4b2d8023320fcbccdaa3": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "654774ba937040d78ea466ae64689edd": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "66d983c505884401b827b457b2f76adc": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "67f992e5fa0b44cb9ca0a1077b0fd0b3": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_07304a89464a49ad85a801a6588badf8", "IPY_MODEL_983dcc2c7328405ba8473c18055d666f", "IPY_MODEL_6c8d317455db4b9c97742ba569d38765" ], "layout": "IPY_MODEL_dc225614361640dbb26c053388c51182" } }, "68443ed500474fdfb361e58cbb955b2c": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "68a00d93cec5412487ff7151e697d2fa": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "690eb4b23fde4d86b52bcb262bb5ad48": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_b8114465819c42c9bf42dfacd49ee50c", "IPY_MODEL_8f3c6eb0609242ae880c2001f678c266", "IPY_MODEL_b33dc5241faf426cb9195a9d786b419f" ], "layout": "IPY_MODEL_bcaef4dc343d42f9b754393bf31a19f4" } }, "693e44a6d4704b22b4a3e0e7aacc2a89": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_b20e3823631b40969109c4434edcf4cb", "IPY_MODEL_bf1dee758cae4172b5bdda4d4f6aa5d4", "IPY_MODEL_5bcf39a25f814726a6d355141d719e55" ], "layout": "IPY_MODEL_01de57f75ac14b35b664691111c87324" } }, "6977852255e840f0958d1ac1f9985747": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_2ad2e85b1f9a4f1e9e7582301dcef5bd", "placeholder": "​", "style": "IPY_MODEL_c03f373f314b4907806a22a0405ea2e0", "value": "Validation DataLoader 0: 100%" } }, "69d34e3c4935407b9d4e668236bd02b4": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "69ec8af183064d149451015c5715dbb7": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "6a285b553a1849c7a5414882fd1e011e": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "6ab4a6bc2faf4706ad47e0c3d21b3c5a": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "6bbb4911ba9548f1a276bd6b6b05c566": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "6c696e3e8973465a94241c198ba451d6": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_e3b6051d3aaa44e48a862a6d24a4dc6e", "placeholder": "​", "style": "IPY_MODEL_8755ded3a6e44494b9bbf91079e839c3", "value": "Validation DataLoader 0: 100%" } }, "6c8d317455db4b9c97742ba569d38765": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_89c8e4bb19c842468d0b63381c13e1ea", "placeholder": "​", "style": "IPY_MODEL_ad70d64891f04b54923bb598f3c4be82", "value": " 188/188 [00:01<00:00, 94.72it/s]" } }, "6cbe35d95599497fadc93b79b380a0ad": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "6e1e72f4008346f8995d5950b56752d9": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "6e2a4427d0874568820647b8a66da50b": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "6eb08bee2fb44dce91845beea5046dda": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "71a418eeb77e4054aadc84674b299831": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_0fc154788e684df8a130caf7b17b7525", "placeholder": "​", "style": "IPY_MODEL_96d84418502647bfb4be5657fb3748da", "value": "Sanity Checking DataLoader 0: 100%" } }, "71ce69a2659542e5a94c6f5d4096f76b": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "71f46192712f4bacb9ecb4aeb49b2bed": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "724cd131a4d640bd8b0862cfa68a449b": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_2007c0564fbb4758899075f09181ae3b", "placeholder": "​", "style": "IPY_MODEL_747d0e4538984d99a5509dc40bf88199", "value": " 157/157 [00:10<00:00, 14.65it/s]" } }, "728aae7b6ea6444a8f6b0d90ef226227": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_4dbb2d40666045a9939db57b5c35904b", "placeholder": "​", "style": "IPY_MODEL_b7d52b33551a4b529e6f5d6f60163377", "value": "Validation DataLoader 0: 100%" } }, "729b1ab41a204bfaa94912320251ddaa": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_8e4608ea6244450caa4a5989b1b90ab0", "IPY_MODEL_526e3d1b822347e4969898926dccf66a", "IPY_MODEL_79efec810775459da051e56249e197dc" ], "layout": "IPY_MODEL_43f412f3c8224f7b8a7aa666ffeb5da3" } }, "72f4c55eb4ab4746a40b88ad6a6be877": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_5e4c87a8903a4e56a1072021016f58f8", "placeholder": "​", "style": "IPY_MODEL_dbd91e35f4a94c7fa7685348a529b66a", "value": "Validation DataLoader 0: 100%" } }, "73b3c34a07b84d268ba8d20eef65cc1a": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_9286c93d11f34008ab81a95bf16f6b4b", "max": 188, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_09964e13315444668b8ef036621fe3ac", "value": 188 } }, "747d0e4538984d99a5509dc40bf88199": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "7595b08f182748c1bf291bb7e87a4026": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "7771e0e6d0624963bf22ea034e5071b2": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_68443ed500474fdfb361e58cbb955b2c", "max": 188, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_a192193ae05c4045b53471ef3c48af25", "value": 188 } }, "7800fbc10e1c48888b3dea7ce30c7316": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_26d47c45fc7747b8b4b5514c572b05b6", "max": 188, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_6cbe35d95599497fadc93b79b380a0ad", "value": 188 } }, "79a2d947cfaf4fcb9743cd77692301d1": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_429607e5cad94841902812a8f67e0937", "placeholder": "​", "style": "IPY_MODEL_71ce69a2659542e5a94c6f5d4096f76b", "value": " 157/157 [00:10<00:00, 14.36it/s]" } }, "79cd96c7295d48a98101d8473fde6044": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_82fb641562594413a8f3f8f3e2c23422", "max": 750, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_cce75e14ba91469a82f7fe2372efb03b", "value": 750 } }, "79efec810775459da051e56249e197dc": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_803d0c75e1e44892a9e2cbb63211ca5f", "placeholder": "​", "style": "IPY_MODEL_20032203372344d59e59f030eac46f52", "value": " 157/157 [00:10<00:00, 14.41it/s]" } }, "7a1d37c4e45547fabed832ae9518f14d": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "7a36260b66144a648aede88a6f3fde32": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_3bc766accb9b46f4a64256b0ee748dfa", "placeholder": "​", "style": "IPY_MODEL_fb93df8350ac4f9db1cbc09ea2d5de5c", "value": "Validation DataLoader 0: 100%" } }, "7aed5f3dbb734b36812f247cb2925276": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "7b315d75b3f740a791453887294f30f5": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_8332169a6f5b4ae5be980cef4f259504", "placeholder": "​", "style": "IPY_MODEL_cad65de7b5ca4351bc61087625de4232", "value": "Sanity Checking DataLoader 0: 100%" } }, "7b7e77ea3aa24f41b61d8515e8dbaed6": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "7c26402ac7784a5c8e3e3f181d829538": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "7c2b1d5c43d9475b859fda52dc67bea0": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "7c4db9a587b44dc1899704ae84f5af71": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_3e2b4e7404af40b7afb610cca201e71c", "IPY_MODEL_49e3239d4f104f92af2d00d01f0a76e7", "IPY_MODEL_79a2d947cfaf4fcb9743cd77692301d1" ], "layout": "IPY_MODEL_c4f3642f668d428ab641ad442478fc9b" } }, "7caefec1f51e4c2b980ea5f64b472a49": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "7da0496a061e4c7084fee85fb457349a": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_7aed5f3dbb734b36812f247cb2925276", "placeholder": "​", "style": "IPY_MODEL_17c74e62e20e42e1b27a6ab5df0d8e26", "value": " 188/188 [00:01<00:00, 101.25it/s]" } }, "7e812e59e93046e587395a74553d64c9": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_9500a787c0c14ecbb0efc701cca7ccb7", "max": 157, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_654774ba937040d78ea466ae64689edd", "value": 157 } }, "7f2b768fd52b498a8fcb9e3516651fb0": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "7f43dfbaeb66467690691f944a5ef1e1": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "7f6b7ce508254497ac4170d76254615a": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "80363fc9e7174b9eaa1f502e25a44a20": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_62d50bd4188c470484da2808a0e0987f", "IPY_MODEL_4abcbee49e9b4f6fa20945f319e8fc8a", "IPY_MODEL_e397cfc976404e0186ff91633bfb02d8" ], "layout": "IPY_MODEL_14c62c68634946079e92b0443e309ca2" } }, "803d0c75e1e44892a9e2cbb63211ca5f": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "8089561b2d8e49c1bc6b51ca33824556": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_b625021b1fb8465dbcff5396934e0acd", "max": 188, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_c2e67f94f8b04c919834d47215cf1c25", "value": 188 } }, "80f19b2199cb4dbda6fe96b7b4228f9d": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "812135e52bd74893b94e553f5bd30ece": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_51c98590ad9a43a9980dc3596b3fd9d6", "placeholder": "​", "style": "IPY_MODEL_c3bde68c577d4655b959472f0d6d607b", "value": "Validation DataLoader 0: 100%" } }, "819d5a370adb4b73b45b01b1bf6824c6": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "81d35d9ef9d840bfae39787980ae162d": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_4b04920b8f784bb489fd352eecf74faa", "placeholder": "​", "style": "IPY_MODEL_26699a4bafba4da486e03e3f673c73c5", "value": " 157/157 [00:10<00:00, 14.63it/s]" } }, "822be6819d49429b8510a4b4a17e305b": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": "100%" } }, "82fb641562594413a8f3f8f3e2c23422": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "8332169a6f5b4ae5be980cef4f259504": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "847dbf7ff7ab4931bbb0b271f9a74d3f": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "848a4beda86441c98ca874b076d3ed6e": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_a22df4d448a341ccab59fe1150b4f2e8", "placeholder": "​", "style": "IPY_MODEL_17a0a9e061ab4525b90ef33ac3d4b72f", "value": "Validation DataLoader 0: 100%" } }, "84973c4191734c179409fd12c58e357a": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "8521faf1a6934b1b9d3fd7635bf50c6f": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "8531be16189a4f7ca5a697cd22a515e4": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "872bf623b12e47e392be655cc0c1f4f1": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "8746ad0c90964b3f93af266a36938393": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "8755ded3a6e44494b9bbf91079e839c3": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "8804fff7f53540299db7371d1a65363b": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "88be2abb00014a5dbb136f180392bbc1": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "89a6ad22340345f090543af2d990df2d": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_251b15cd84f7403a9343bee611369f6d", "placeholder": "​", "style": "IPY_MODEL_aa7b28492f7545f18d20315fa4e74035", "value": "Validation DataLoader 0: 100%" } }, "89c8e4bb19c842468d0b63381c13e1ea": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "8b38ad55167246b3976dcee637b8b982": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "8c16363b28884d07b4d21c5bf903a575": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_22a5b300c7c344f0b4a80c07f641b1a6", "placeholder": "​", "style": "IPY_MODEL_a1774b87f7e8497f9d354d56d44a276d", "value": "Validation DataLoader 0: 100%" } }, "8c7bc6b2ea2342a2b20a8d6419c1ae9d": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%" } }, "8d95d25d373946ecba61a0ccbe0669c2": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_052bf7dce8c64f74afde401d6db3bcbd", "placeholder": "​", "style": "IPY_MODEL_2c16d456a401438d80dafbba28deb9da", "value": "Epoch 9: 100%" } }, "8e4608ea6244450caa4a5989b1b90ab0": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_68a00d93cec5412487ff7151e697d2fa", "placeholder": "​", "style": "IPY_MODEL_b69801b7120b4186b9e27cc8904173bd", "value": "Validation DataLoader 0: 100%" } }, "8f3c6eb0609242ae880c2001f678c266": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_5b5216781d6d4008a8217e6adde5fbe3", "max": 188, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_b48dff1b7cc841d9969506f26061d2c4", "value": 188 } }, "8fd75a6f4cea4ef7bc7e641777e3c554": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "8feb44c4e6024da8904c7de5be4a719f": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "905c6725e6a84022b0a98415ff7ec5a4": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_9c6c2de9d24745349f05e083bdde03e7", "IPY_MODEL_9ca7a649268748ef94db3c3b76edff30", "IPY_MODEL_edc5c94f45954a3dba073c565a3c0e78" ], "layout": "IPY_MODEL_af067ae63c0449cc8db04ea8429bea51" } }, "917deda0fee94af18bd7107fb350aa6c": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_5003c39b9f134d3b806469e0ded2e69f", "placeholder": "​", "style": "IPY_MODEL_119e0bef11194cf3940b6f095c7f72d2", "value": " 10/10 [00:00<00:00, 135.45it/s]" } }, "918b50556def45fba40a7f6d673f6a09": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_7a1d37c4e45547fabed832ae9518f14d", "placeholder": "​", "style": "IPY_MODEL_cdf6e175b8fd41e6a76e2edccd52c046", "value": "Validation DataLoader 0: 100%" } }, "91b82484785a4a668f8371fd6e273e1b": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "91d18522722343d093658dfeb083aea4": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "9254d3bbd2494eb7aceaf1f58c425b70": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_9b1432a4677940b7a4698c73606af8ce", "placeholder": "​", "style": "IPY_MODEL_109402bb49874e108ee3f3b09ba37b33", "value": " 188/188 [00:02<00:00, 88.97it/s]" } }, "9286c93d11f34008ab81a95bf16f6b4b": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "92f18e3b662040418bdf96d7d7c9ae46": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "93153ab1a75b41bba6bbd57f30f8750b": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_cb90f97c63ec4e2582478369aa628137", "placeholder": "​", "style": "IPY_MODEL_18b7ef59edf14b10ac2135a23331812a", "value": "Validation DataLoader 0: 100%" } }, "94253e895c68406c884a698bcb082234": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "94562514188d4d00b185a9f1074f92bd": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_5d6adabf50e047d3897a5265248953b4", "max": 157, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_4a6d32c082cc482c8f4c4991ef7cc5e2", "value": 157 } }, "94cf9f53d8224cd781fb345b8c4df6c2": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_53b5a097f3644889a8d64acc9685d3b0", "placeholder": "​", "style": "IPY_MODEL_12e51cc17e204feab9a269642ee418b7", "value": "Validation DataLoader 0: 100%" } }, "94cfd4cd1bff47aab343b287f74a2d55": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "9500a787c0c14ecbb0efc701cca7ccb7": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "9553a95e54c847668587d3945bfed92a": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_403ef89515f1429f9adf3603536c22f1", "IPY_MODEL_3044a6ff3d324f0a802157ea2e4aa3ab", "IPY_MODEL_5eddda5ad6c14519a8ecbfaf97299f16" ], "layout": "IPY_MODEL_aed2d75c28324b5db72ce18d4d767465" } }, "9629e738b04c40a485ba201e54100f04": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_32d0b577656e4db1bba3c7ec1dd4b14e", "IPY_MODEL_b6ba50fde7b746aeb4a5fd50968a8f9b", "IPY_MODEL_7da0496a061e4c7084fee85fb457349a" ], "layout": "IPY_MODEL_f5ea4d3b8139420e9e714845a34d407d" } }, "962b2e4e1a924b9fa9eef589de15aafc": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_9ff8986a1fe144579c8567a4e6399f83", "max": 157, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_a23a5c904b5541fda21c99315a891782", "value": 157 } }, "96ce55d2e0c44d588e8040c01567cee7": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "96d84418502647bfb4be5657fb3748da": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "9703098f1e634fa2815e64c2f5f3790a": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "97ed2480134b43b69caad381232ae1f6": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "9815b5c8652847ce84ac589783ca1e4c": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_335f062e448b4754bacc27528430781b", "max": 750, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_b8b3ef82e97b4e0490573dda90227e72", "value": 750 } }, "983dcc2c7328405ba8473c18055d666f": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_6465c4379d0a4a6f8b66ae32a86164f1", "max": 188, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_c73584d6817141b79c48dcc76f92748b", "value": 188 } }, "9854b37a6206460793b035b1b43ecb5a": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%" } }, "99411929fad34ec5a8eb8f5b5e0adcca": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "995443f0e8eb49bcaa920b08b566fae4": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_94253e895c68406c884a698bcb082234", "max": 79, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_91b82484785a4a668f8371fd6e273e1b", "value": 79 } }, "99a0b785644345789d7d742b671bf2c0": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_91d18522722343d093658dfeb083aea4", "placeholder": "​", "style": "IPY_MODEL_64b137dc1d4844b9bfdc4e6b5458d8a2", "value": " 2/2 [00:00<00:00, 18.56it/s]" } }, "99af57b1e9a646058f5f0299254e98c6": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "9ad2b3dd5dbe4f02afab448c4e3590e5": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%" } }, "9ae8702030a7443db132d95c835a80d7": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_c148e7eb982d428fa4d2a8c29564e73b", "IPY_MODEL_a95ecb0d965d484cbf04e3a36399f51b", "IPY_MODEL_ed28332348ee4148bccdee516777b94c" ], "layout": "IPY_MODEL_63c614b65ba34979bdceb203598eabac" } }, "9b1432a4677940b7a4698c73606af8ce": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "9b5707edf0b94b5b8c12681ccda04b80": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "9bd6317622fb488db10c34d028559fd9": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "9bd73d53ce484777962f80a368c6c76d": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_3f21584337fe40e18100e9f4b2bbb45d", "placeholder": "​", "style": "IPY_MODEL_321461735c1d4acb863914733f4f29a1", "value": " 188/188 [00:01<00:00, 104.79it/s]" } }, "9c5e867f28f0484d839c280d08f58b41": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "9c6c2de9d24745349f05e083bdde03e7": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_dca9606689ec470094fffd439140f2e8", "placeholder": "​", "style": "IPY_MODEL_99411929fad34ec5a8eb8f5b5e0adcca", "value": "Validation DataLoader 0: 100%" } }, "9ca7a649268748ef94db3c3b76edff30": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_ea9b9d0b964e409999b904b734c61d90", "max": 188, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_9dc43c1439a9452ca98bff07604bbb8d", "value": 188 } }, "9d3e2878bcfc472b9954e2cfe66e6bd0": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "9dc43c1439a9452ca98bff07604bbb8d": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "9f2d589d84074072b23bed0d11d5e88d": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": "100%" } }, "9f40085152dc4afb8b73bbda14159334": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_3d5134fb97ca43de90f67e7a833aebd9", "placeholder": "​", "style": "IPY_MODEL_80f19b2199cb4dbda6fe96b7b4228f9d", "value": " 188/188 [00:01<00:00, 106.22it/s]" } }, "9ff8986a1fe144579c8567a4e6399f83": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "a06b075a94f246e2a76e8f2c7f1c6a97": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "a0e3dd6b200a4cba8ca4a3eeb83fa78c": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "a1774b87f7e8497f9d354d56d44a276d": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "a192193ae05c4045b53471ef3c48af25": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "a22df4d448a341ccab59fe1150b4f2e8": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "a23a5c904b5541fda21c99315a891782": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "a268ffca8fb54bea9341ef154ac742f5": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%" } }, "a2ef959ead414ce1ad692ce4c5a1ebf5": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "a31fad6e7d3141058788c0529fb3b45b": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "a3f8d90bf76b48259c4f7583af4c578e": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "a4438910cc994c67b39f2888afdecc4c": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "a4a871dee0b744d89067df5bd70ce0ec": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_819d5a370adb4b73b45b01b1bf6824c6", "placeholder": "​", "style": "IPY_MODEL_d3bcb89756d0454c878db1875cfb1d27", "value": "Sanity Checking DataLoader 0: 100%" } }, "a555413eee4a404781069cd27cc2353d": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "a58ab633de7943ffbf232eafdc657092": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "a5f611dbbe774f18a7abd07a06e7d1aa": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "a6102a3f12d643929e967b23ecd6c846": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "a612e8e330314d26a0f26c8034e7cabb": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_4d9514a4fec549b9b1bcb1600c70ce80", "placeholder": "​", "style": "IPY_MODEL_12af8881c6354f71abbe13a61ec30cbf", "value": "Validation DataLoader 0: 100%" } }, "a616f5cdf95944a486ef0dade17c22d2": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "a63ee1b22f964f9991bb7698ab8bd135": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "a68dcf701662428787d82111a3d37c1a": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_a58ab633de7943ffbf232eafdc657092", "placeholder": "​", "style": "IPY_MODEL_a0e3dd6b200a4cba8ca4a3eeb83fa78c", "value": "Epoch 6: 100%" } }, "a85a29c916944b2d8ef8579f3a0521e4": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "a90892d849cd4e6aa6d632a2ce7be940": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "a95ecb0d965d484cbf04e3a36399f51b": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_e84e6b84e77a44bcb8e215ab862871ee", "max": 157, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_61fe68e00f8d40baa2dccd882b725f21", "value": 157 } }, "aa4cef8a3b824553b4a85c68a4cfc6e9": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%" } }, "aa7b28492f7545f18d20315fa4e74035": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "ab3c4c6b11ef4a9ea4e0ae95fbc3fcd7": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "acfbdcfb31394e3496ab75c7e6c0a8c5": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "ad3f8b3f273a4af8aa34a1b6fae96e5e": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": "100%" } }, "ad70d64891f04b54923bb598f3c4be82": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "ad9f4e32300f4406a9f5189044cae2ee": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_4dde3401e64d4a3b8eac548cdbbb5626", "max": 188, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_3686607130864dba852689774f6f5ccd", "value": 188 } }, "adff2e30300e41e79bd2760576763477": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "aed2d75c28324b5db72ce18d4d767465": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%" } }, "af067ae63c0449cc8db04ea8429bea51": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%" } }, "af22275aa51b41d4a8a0c093d16872f4": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "af9891a0ce674f21a4d5c0f3162beef6": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_c96af87632a44a5083114c846315d299", "placeholder": "​", "style": "IPY_MODEL_33546ec140a441cbb25c5300b986347f", "value": " 188/188 [00:03<00:00, 53.12it/s]" } }, "afa4e9f3fea84e4eae0c77cabf930835": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "afdce55760c4431dbb0dd4bd9827aad9": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "b017226b497844db9dd1d95aedc9fcd5": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_0eaa31608dab4a64aba2a09b8200200b", "placeholder": "​", "style": "IPY_MODEL_2ba9e3c6007a4dc880ec4757de48a69b", "value": " 2/2 [00:00<00:00, 40.24it/s]" } }, "b01ef2caca6a4d2799b4fcef11f5ac56": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_0f6c7b386ca848389ed069e8df65ac51", "max": 188, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_c603a7c416784cb78123c13a7bb7ca8e", "value": 188 } }, "b109c57180354eb19f3cd789b0850693": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_afa4e9f3fea84e4eae0c77cabf930835", "placeholder": "​", "style": "IPY_MODEL_342e8b4b20774be9a2fa68eaa0921bbe", "value": " 782/782 [03:03<00:00,  4.26it/s, v_num=0, train_loss=0.00933]" } }, "b1750c0a227340c3b9223f8b7a22837c": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_31ea9d1a96c2488792e65dba04c2d747", "placeholder": "​", "style": "IPY_MODEL_fee86f6183a9499ea4c932c59f5f5cb1", "value": " 188/188 [00:03<00:00, 57.70it/s]" } }, "b20e3823631b40969109c4434edcf4cb": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_b81e3780dc124754b6d2ea32907f9b85", "placeholder": "​", "style": "IPY_MODEL_a06b075a94f246e2a76e8f2c7f1c6a97", "value": "Validation DataLoader 0: 100%" } }, "b20ec8f2faa94e6ea6017162884e4b3c": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "b235cf55e08540f0a165268bf4bc97a9": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_92f18e3b662040418bdf96d7d7c9ae46", "placeholder": "​", "style": "IPY_MODEL_21ac563ed5474ab79e1c33756dca7e50", "value": " 2/2 [00:00<00:00, 34.25it/s]" } }, "b2fe95fa2e6c4413aa3dd17e041e47be": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "b33dc5241faf426cb9195a9d786b419f": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_603e4d9032be4d139d033dd2f5db5572", "placeholder": "​", "style": "IPY_MODEL_f1daa2804ac74805a950e481557edab0", "value": " 188/188 [00:03<00:00, 51.24it/s]" } }, "b3cc361c0c1b4895a1343693bab9c73f": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_d2c171df7a954896a1ecabdaebe9f46e", "IPY_MODEL_fb63ccc027ab46a5b3f1923292820406", "IPY_MODEL_31a6eae7f3ec40849d57e9f6d11bc7af" ], "layout": "IPY_MODEL_822be6819d49429b8510a4b4a17e305b" } }, "b3f9bdff639d4773b3e387205d40bc42": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "b40393ad178c41a690c6af61f8eebfec": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "b48dff1b7cc841d9969506f26061d2c4": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "b4abb63a4f8b49419c1c7bb0ebe2fcb2": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "b56d30ff3d884bc893cb823335d547a5": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_7a36260b66144a648aede88a6f3fde32", "IPY_MODEL_e56b3c60aa0c48458005049281673649", "IPY_MODEL_56787584d42f4982ba55fbf03a517d9e" ], "layout": "IPY_MODEL_aa4cef8a3b824553b4a85c68a4cfc6e9" } }, "b625021b1fb8465dbcff5396934e0acd": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "b64e2600c41b491ea916e4fbe9400a19": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "b69801b7120b4186b9e27cc8904173bd": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "b6ba50fde7b746aeb4a5fd50968a8f9b": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_161f4e5c97734ad89bef49fa8f9d6bc1", "max": 188, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_c6506c6aa9ca47d8ba23dd22cf09924b", "value": 188 } }, "b72261e955984b7ab00aa09ec91eb859": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "b7d52b33551a4b529e6f5d6f60163377": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "b7d69f00abf44aa5908b7543b0780b1c": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_52e3286dfe334a77bfe1ac68e5ef6775", "placeholder": "​", "style": "IPY_MODEL_450f3a364264429288012140c6458ba3", "value": " 157/157 [00:02<00:00, 59.50it/s]" } }, "b7f995f81b2742fca81c658a5d4f6e0f": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "b8114465819c42c9bf42dfacd49ee50c": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_adff2e30300e41e79bd2760576763477", "placeholder": "​", "style": "IPY_MODEL_647fa4607d21458e83718a79e2a50a39", "value": "Validation DataLoader 0: 100%" } }, "b81e3780dc124754b6d2ea32907f9b85": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "b8b3ef82e97b4e0490573dda90227e72": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "b8b9dab360ae4dbf990ebd3053f45794": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%" } }, "ba1be115b69643b4aa876309eb31f09f": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "ba8ea911e487489e8b02ebbdb070f328": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_8d95d25d373946ecba61a0ccbe0669c2", "IPY_MODEL_4d2b5f4881924960b3cd760d13419998", "IPY_MODEL_b109c57180354eb19f3cd789b0850693" ], "layout": "IPY_MODEL_9f2d589d84074072b23bed0d11d5e88d" } }, "bb303da048854cda97dde88e2b3e78ce": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%" } }, "bbb7526753ca4f7a9298bf6e3377198b": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "bc3767f26b4e4f9ebcacd7b3eec922f3": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_4406a71e0da14d659937b464de933045", "IPY_MODEL_962b2e4e1a924b9fa9eef589de15aafc", "IPY_MODEL_5a254bebeb2d41f3befb6ef0efc3ed7a" ], "layout": "IPY_MODEL_fd6c43dad70e4faf818237acaa7dc2c8" } }, "bcaef4dc343d42f9b754393bf31a19f4": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%" } }, "bd5593c92ca84e54ae48aa1b8843db59": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "bf1dee758cae4172b5bdda4d4f6aa5d4": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_4ee347f24ff24cbaa172d96a5e87cc5c", "max": 188, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_99af57b1e9a646058f5f0299254e98c6", "value": 188 } }, "c03f373f314b4907806a22a0405ea2e0": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "c148e7eb982d428fa4d2a8c29564e73b": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_306c26e12ef2438ead540532792ed165", "placeholder": "​", "style": "IPY_MODEL_a63ee1b22f964f9991bb7698ab8bd135", "value": "Validation DataLoader 0: 100%" } }, "c2482b58f84e4ce3aa69990ae5b88783": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "c2e67f94f8b04c919834d47215cf1c25": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "c38569a43bb2473ca4e711d156a12da7": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_d5155b0d36604131933312f10e047b8a", "IPY_MODEL_7771e0e6d0624963bf22ea034e5071b2", "IPY_MODEL_9bd73d53ce484777962f80a368c6c76d" ], "layout": "IPY_MODEL_225855b374144cabaec2a9f18a69b64a" } }, "c3bde68c577d4655b959472f0d6d607b": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "c44a38fdfc5246668f1e2ef9a1281747": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "c470c831a38a4f379836fdbfde58089d": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "c4f3642f668d428ab641ad442478fc9b": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%" } }, "c5737fc6068d44649723b46603559da8": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "c5ce3129674d402ba2a5f1a2944ac78d": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_89a6ad22340345f090543af2d990df2d", "IPY_MODEL_486351e0ff0f42e9836dd953f39a9d63", "IPY_MODEL_07b85b31f0ee4c568618b9e9fbb4c929" ], "layout": "IPY_MODEL_40e69c44b12841d18b93d292a7a17b4f" } }, "c603a7c416784cb78123c13a7bb7ca8e": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "c6506c6aa9ca47d8ba23dd22cf09924b": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "c657bb4bf8a840de907a975f59dbe56f": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_71a418eeb77e4054aadc84674b299831", "IPY_MODEL_13f7be10f92f4c109a94b36ff1e2c451", "IPY_MODEL_99a0b785644345789d7d742b671bf2c0" ], "layout": "IPY_MODEL_ed280f20224e44a68d922ac93e4b8535" } }, "c73584d6817141b79c48dcc76f92748b": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "c936e1097b1245e6a4a325a472133874": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "c96af87632a44a5083114c846315d299": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "cad65de7b5ca4351bc61087625de4232": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "cb90f97c63ec4e2582478369aa628137": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "cbc951497bea49bb8617bb265148db3a": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "cc936ab4d60f43c9b2828dc46dbb6b29": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "cce75e14ba91469a82f7fe2372efb03b": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "ccffac78d24b4b9e8566d7dfca09cbc0": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "cd84ee816aeb4bcfb3ca35dcfd5d46a7": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_a612e8e330314d26a0f26c8034e7cabb", "IPY_MODEL_253755895c0c49a58f39f5267ec5744a", "IPY_MODEL_5a2bd91a27ba409b9980cac971db5ba0" ], "layout": "IPY_MODEL_bb303da048854cda97dde88e2b3e78ce" } }, "cdf6e175b8fd41e6a76e2edccd52c046": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "ceaea24c3f6e4507b304441b6771f4d3": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "cf36208b80a94aa48e418c9c8401e322": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_633e575d2a904eadb3836d9221911746", "max": 188, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_b3f9bdff639d4773b3e387205d40bc42", "value": 188 } }, "d2c171df7a954896a1ecabdaebe9f46e": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_b40393ad178c41a690c6af61f8eebfec", "placeholder": "​", "style": "IPY_MODEL_b64e2600c41b491ea916e4fbe9400a19", "value": "Epoch 19: 100%" } }, "d340af6a5d4249b2a6449b489888f483": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_07cf8f8336874708a0b92c2311efc94e", "placeholder": "​", "style": "IPY_MODEL_535e920fb8c942f59b30cfc20483397c", "value": "Validation DataLoader 0: 100%" } }, "d3bcb89756d0454c878db1875cfb1d27": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "d3db93df4d4a4d5a98d095779236d800": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": "100%" } }, "d5155b0d36604131933312f10e047b8a": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_7b7e77ea3aa24f41b61d8515e8dbaed6", "placeholder": "​", "style": "IPY_MODEL_39e66ed2644742a0b0d3c1ef0307f463", "value": "Validation DataLoader 0: 100%" } }, "d581b3cb4f5c4932be8862be509d97f9": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%" } }, "d70c4c53afad4c84ad592dad70697f60": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "d73eb94946924b09bfd4e541385db470": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "d7e73012bf44455db890cefa226f4ed6": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_7c2b1d5c43d9475b859fda52dc67bea0", "placeholder": "​", "style": "IPY_MODEL_a6102a3f12d643929e967b23ecd6c846", "value": "Validation DataLoader 0: 100%" } }, "d7f046cc048a4ca9896cdfd0c81f7037": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "d887a20ea9fd4e0b87c97c5d4bd6484e": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "d89e2268f03345f0b315b3c96b1c1a8f": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%" } }, "d8cd077e4d8644b49adc7b1c6cfe1915": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "d9633cd7d7b149dab214496d1fd2f876": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_f926b90e1a764b9eb8c3cea3aa21cf18", "placeholder": "​", "style": "IPY_MODEL_b20ec8f2faa94e6ea6017162884e4b3c", "value": "Validation DataLoader 0: 100%" } }, "d9748c38ccda41ac94a034dc6d21826b": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%" } }, "d9b8f3663bc340639c61a44f2f038eb5": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "d9e313817f2c4f48a868abe60a834cb7": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_848a4beda86441c98ca874b076d3ed6e", "IPY_MODEL_477b20e171b744518f2691d343188714", "IPY_MODEL_2443930de8284148a7ed0ff64cab9b75" ], "layout": "IPY_MODEL_3c162e4b29c84f5ca2a3243f35d59641" } }, "da75d186211847b6b2056b5dbea3ae4c": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_d8cd077e4d8644b49adc7b1c6cfe1915", "placeholder": "​", "style": "IPY_MODEL_104e7496489049a0a7343a8f423904de", "value": "Testing DataLoader 0: 100%" } }, "dbd91e35f4a94c7fa7685348a529b66a": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "dc225614361640dbb26c053388c51182": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%" } }, "dc9e789d02804f3d876cad49d752daaf": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_7caefec1f51e4c2b980ea5f64b472a49", "max": 188, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_3dc22a5addd74a9ebb99bcd0a8942cb4", "value": 188 } }, "dca9606689ec470094fffd439140f2e8": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "dcbae4193030459081a305c51d52e4e7": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_6ab4a6bc2faf4706ad47e0c3d21b3c5a", "placeholder": "​", "style": "IPY_MODEL_05344f36aa5f4141b3a25c5e6c30f518", "value": " 188/188 [00:03<00:00, 57.94it/s]" } }, "dd8a1ed1a0cc47a1844fbce14f5b4619": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "de153318bf764d0680b7756a090f30c8": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_50827632c1804f0e8c22023c63e655b6", "max": 750, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_96ce55d2e0c44d588e8040c01567cee7", "value": 750 } }, "decba3296d474b8282f988401495487f": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_61c5c53594ad47e2adafa99dcd0e35ae", "max": 2, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_e4151182f97341d5a4e89803082573f1", "value": 2 } }, "e095049b602048c99d8d1dd8552fcfe9": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_064e3ee7eab74e2397fc3da137d444bb", "max": 188, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_bbb7526753ca4f7a9298bf6e3377198b", "value": 188 } }, "e0f92f8764984229856f8f40cd3c3ebc": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "e1b2999baf654e52a7fd0a3961c90d6d": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "e1dc62bffeaf433f919405f27a5de450": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%" } }, "e20581a183bc41e598f34f50dd4e55ef": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_b2fe95fa2e6c4413aa3dd17e041e47be", "max": 2, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_a31fad6e7d3141058788c0529fb3b45b", "value": 2 } }, "e397cfc976404e0186ff91633bfb02d8": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_b7f995f81b2742fca81c658a5d4f6e0f", "placeholder": "​", "style": "IPY_MODEL_97ed2480134b43b69caad381232ae1f6", "value": " 188/188 [00:01<00:00, 105.40it/s]" } }, "e3b6051d3aaa44e48a862a6d24a4dc6e": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "e4019cc439d647a8b447367827b44e4d": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_a85a29c916944b2d8ef8579f3a0521e4", "max": 157, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_88be2abb00014a5dbb136f180392bbc1", "value": 157 } }, "e4151182f97341d5a4e89803082573f1": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "e48b05e26b2a4befa93a908f3e7b7947": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "e4f75da22fe54a90a2be9edf6ca4ca43": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%" } }, "e56b3c60aa0c48458005049281673649": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_f4241da035db4178bf2adf24e8b4e3f2", "max": 188, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_9703098f1e634fa2815e64c2f5f3790a", "value": 188 } }, "e6c06a7218714c2ba1b79f3611b1bc44": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "e70d53a09d1c4a0aa46f9868575833e4": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_7b315d75b3f740a791453887294f30f5", "IPY_MODEL_5759a46f103e4133997e113e44233ef5", "IPY_MODEL_b235cf55e08540f0a165268bf4bc97a9" ], "layout": "IPY_MODEL_a268ffca8fb54bea9341ef154ac742f5" } }, "e84e6b84e77a44bcb8e215ab862871ee": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "e90c40de39c84c57bba7213db55188df": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "ea9b9d0b964e409999b904b734c61d90": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "ebe15f8117e142ac9004ab18241ffeea": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_812135e52bd74893b94e553f5bd30ece", "IPY_MODEL_053fdebd6262410aa47e339b85e4cc7d", "IPY_MODEL_724cd131a4d640bd8b0862cfa68a449b" ], "layout": "IPY_MODEL_28af3c2c45dd4554949ee2099aab336c" } }, "ed280f20224e44a68d922ac93e4b8535": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%" } }, "ed28332348ee4148bccdee516777b94c": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_17cf42bb46984789a70568f2306e7001", "placeholder": "​", "style": "IPY_MODEL_5aba43fc321d49399f868fecdb0da9a1", "value": " 157/157 [00:10<00:00, 14.48it/s]" } }, "edc5c94f45954a3dba073c565a3c0e78": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_a616f5cdf95944a486ef0dade17c22d2", "placeholder": "​", "style": "IPY_MODEL_18d5a8adae3c4770a3b1608c2381ec2c", "value": " 188/188 [00:03<00:00, 50.54it/s]" } }, "eee6719b6a3942ff8eed17671b90d10c": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_d70c4c53afad4c84ad592dad70697f60", "placeholder": "​", "style": "IPY_MODEL_357a9216989c492d97b09b931478a616", "value": " 750/750 [00:21<00:00, 35.61it/s, v_num=0, train_loss=0.222]" } }, "f1daa2804ac74805a950e481557edab0": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "f3a533b63cd34af5a847963af2693875": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_3f10e06684d0488db866f8875daa132c", "IPY_MODEL_9815b5c8652847ce84ac589783ca1e4c", "IPY_MODEL_1f9583df3f81471e83bf33f55ac9a990" ], "layout": "IPY_MODEL_d3db93df4d4a4d5a98d095779236d800" } }, "f4241da035db4178bf2adf24e8b4e3f2": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": "2", "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "f5ea4d3b8139420e9e714845a34d407d": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": "hidden", "width": "100%" } }, "f5fd0185784d4b89b871e734a2efdfd2": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "f6243fce8c9b49758d7abe02d709b4e0": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "f7361a84df9341ae81113eefc565fd6f": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_0548fb9ec6b04228bb98a8227f0af719", "max": 188, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_7f2b768fd52b498a8fcb9e3516651fb0", "value": 188 } }, "f740a916dfb440428b78251df30599f0": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_d7f046cc048a4ca9896cdfd0c81f7037", "placeholder": "​", "style": "IPY_MODEL_bd5593c92ca84e54ae48aa1b8843db59", "value": " 750/750 [00:20<00:00, 36.31it/s, v_num=0, train_loss=0.187]" } }, "f7a583b7d9c24fc290de4f6de44f8321": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_cc936ab4d60f43c9b2828dc46dbb6b29", "max": 188, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_63680227a1344f589f74852ee161b58c", "value": 188 } }, "f895a99a7f574d8f9c97b60dcbadf497": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_5ae6d72b5c77491fb79d2432bedc2851", "placeholder": "​", "style": "IPY_MODEL_3ae8014f7d6b4bbe92b00ebeb0675c09", "value": " 2/2 [00:00<00:00,  9.02it/s]" } }, "f8e8c806d6784ecda642a2ddfd9d7dd5": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_353c41034cdd4320ab78a0794520260c", "IPY_MODEL_b01ef2caca6a4d2799b4fcef11f5ac56", "IPY_MODEL_43c8c9ebf96a46679497d85c218b3ed2" ], "layout": "IPY_MODEL_5cade6652ee04d139f9f801d0467cc7f" } }, "f926b90e1a764b9eb8c3cea3aa21cf18": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "fa3c6d7a0eac4c3e98c275fc2dbab212": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "fb63ccc027ab46a5b3f1923292820406": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_6e2a4427d0874568820647b8a66da50b", "max": 40, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_2748e2a9625b4e72a051d704a80e7ee4", "value": 40 } }, "fb93df8350ac4f9db1cbc09ea2d5de5c": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "fbcbdc0774e141dcadb7897303965fb7": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_69d34e3c4935407b9d4e668236bd02b4", "placeholder": "​", "style": "IPY_MODEL_4d7bde4f23e64616b66e2ca00862c201", "value": " 188/188 [00:03<00:00, 53.52it/s]" } }, "fd6c43dad70e4faf818237acaa7dc2c8": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": "inline-flex", "flex": null, "flex_flow": "row wrap", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": "100%" } }, "fee86f6183a9499ea4c932c59f5f5cb1": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "ffb0411f39e64b8fb6506e650eca2a6d": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } } } } }, "nbformat": 4, "nbformat_minor": 5 }