Imagenet

jaximagenetexamplesflax

Flax Imagenet Example

Open In Colab

Demonstration notebook for https://github.com/google/flax/tree/main/examples/imagenet

The Flax Notebook Workflow:

  1. Run the entire notebook end-to-end and check out the outputs.
    • This will open Python files in the right-hand editor!
    • You'll be able to interactively explore metrics in TensorBoard.
  2. Change config and train for different hyperparameters. Check out the updated TensorBoard plots.
  3. Update the code in train.py. Thanks to %autoreload, any changes you make in the file will automatically appear in the notebook. Some ideas to get you started:
    • Change the model.
    • Log some per-batch metrics during training.
    • Add new hyperparameters to configs/default.py and use them in train.py.
  4. At any time, feel free to paste code from train.py into the notebook and modify it directly there!

Setup

[28]
GPU 0: Tesla T4 (UUID: GPU-d1652fa8-88b9-3d02-7a65-e7ebabeb0372)
[29]
  Preparing metadata (setup.py) ... done
[30]
[31]
mount_gdrive
<IPython.core.display.Javascript object>
<IPython.core.display.Javascript object>
<IPython.core.display.Javascript object>
<IPython.core.display.Javascript object>
[ ]

Imports / Helpers

[33]
[StreamExecutorGpuDevice(id=0, process_index=0, slice_index=0)]
[34]
[35]
[36]
The autoreload extension is already loaded. To reload it, use:
  %reload_ext autoreload

Dataset

[37]
INFO:absl:No config specified, defaulting to config: imagenette/full-size-v2
INFO:absl:Load dataset info from /root/tensorflow_datasets/imagenette/full-size-v2/1.0.0
INFO:absl:Reusing dataset imagenette (/root/tensorflow_datasets/imagenette/full-size-v2/1.0.0)
INFO:absl:Constructing tf.data.Dataset imagenette for split train, from /root/tensorflow_datasets/imagenette/full-size-v2/1.0.0
tfds.core.DatasetInfo(
,    name='imagenette',
,    full_name='imagenette/full-size-v2/1.0.0',
,    description="""
,    Imagenette is a subset of 10 easily classified classes from the Imagenet
,    dataset. It was originally prepared by Jeremy Howard of FastAI. The objective
,    behind putting together a small version of the Imagenet dataset was mainly
,    because running new ideas/algorithms/experiments on the whole Imagenet take a
,    lot of time.
,    
,    This version of the dataset allows researchers/practitioners to quickly try out
,    ideas and share with others. The dataset comes in three variants:
,    
,    *   Full size
,    *   320 px
,    *   160 px
,    
,    Note: The v2 config correspond to the new 70/30 train/valid split (released in
,    Dec 6 2019).
,    """,
,    config_description="""
,    full-size variant.
,    """,
,    homepage='https://github.com/fastai/imagenette',
,    data_path='/root/tensorflow_datasets/imagenette/full-size-v2/1.0.0',
,    file_format=tfrecord,
,    download_size=1.45 GiB,
,    dataset_size=1.46 GiB,
,    features=FeaturesDict({
,        'image': Image(shape=(None, None, 3), dtype=uint8),
,        'label': ClassLabel(shape=(), dtype=int64, num_classes=10),
,    }),
,    supervised_keys=('image', 'label'),
,    disable_shuffling=False,
,    splits={
,        'train': <SplitInfo num_examples=9469, num_shards=16>,
,        'validation': <SplitInfo num_examples=3925, num_shards=4>,
,    },
,    citation="""@misc{imagenette,
,      author    = "Jeremy Howard",
,      title     = "imagenette",
,      url       = "https://github.com/fastai/imagenette/"
,    }""",
,)
[38]
[39]
INFO:absl:Constructing tf.data.Dataset imagenette for split train[0:9469], from /root/tensorflow_datasets/imagenette/full-size-v2/1.0.0
WARNING:absl:options.experimental_threading is deprecated. Use options.threading instead.
INFO:absl:Constructing tf.data.Dataset imagenette for split validation[0:3925], from /root/tensorflow_datasets/imagenette/full-size-v2/1.0.0
WARNING:absl:options.experimental_threading is deprecated. Use options.threading instead.
[40]
{'image': (TensorShape([128, 224, 224, 3]), tf.float32),
, 'label': (TensorShape([128]), tf.int64)}

Training from scratch

[ ]
[42]
batch_size: 32
,cache: false
,dataset: imagenette
,half_precision: true
,learning_rate: 0.1
,log_every_steps: 100
,model: ResNet18
,momentum: 0.9
,num_epochs: 5.0
,num_train_steps: -1
,prefetch: 1
,shuffle_buffer_size: 128
,steps_per_eval: -1
,warmup_epochs: 0.5
[43]
INFO:absl:Constructing tf.data.Dataset imagenette for split train[0:9469], from /root/tensorflow_datasets/imagenette/full-size-v2/1.0.0
WARNING:absl:options.experimental_threading is deprecated. Use options.threading instead.
INFO:absl:Constructing tf.data.Dataset imagenette for split validation[0:3925], from /root/tensorflow_datasets/imagenette/full-size-v2/1.0.0
WARNING:absl:options.experimental_threading is deprecated. Use options.threading instead.
[44]
INFO:absl:No config specified, defaulting to config: imagenette/full-size-v2
INFO:absl:Load dataset info from /root/tensorflow_datasets/imagenette/full-size-v2/1.0.0
INFO:absl:Constructing tf.data.Dataset imagenette for split train[0:9469], from /root/tensorflow_datasets/imagenette/full-size-v2/1.0.0
WARNING:absl:options.experimental_threading is deprecated. Use options.threading instead.


ResNet18_lr=0.03
INFO:absl:Constructing tf.data.Dataset imagenette for split validation[0:3925], from /root/tensorflow_datasets/imagenette/full-size-v2/1.0.0
WARNING:absl:options.experimental_threading is deprecated. Use options.threading instead.
INFO:absl:Restoring legacy Flax checkpoint from models/ResNet18_lr=0.03/checkpoint_1475
INFO:absl:Initial compilation, this might take some minutes...
INFO:absl:No config specified, defaulting to config: imagenette/full-size-v2
INFO:absl:Load dataset info from /root/tensorflow_datasets/imagenette/full-size-v2/1.0.0
INFO:absl:Constructing tf.data.Dataset imagenette for split train[0:9469], from /root/tensorflow_datasets/imagenette/full-size-v2/1.0.0
WARNING:absl:options.experimental_threading is deprecated. Use options.threading instead.


ResNet18_lr=0.1
INFO:absl:Constructing tf.data.Dataset imagenette for split validation[0:3925], from /root/tensorflow_datasets/imagenette/full-size-v2/1.0.0
WARNING:absl:options.experimental_threading is deprecated. Use options.threading instead.
INFO:absl:Restoring legacy Flax checkpoint from models/ResNet18_lr=0.1/checkpoint_1475
INFO:absl:Initial compilation, this might take some minutes...
INFO:absl:No config specified, defaulting to config: imagenette/full-size-v2
INFO:absl:Load dataset info from /root/tensorflow_datasets/imagenette/full-size-v2/1.0.0
INFO:absl:Constructing tf.data.Dataset imagenette for split train[0:9469], from /root/tensorflow_datasets/imagenette/full-size-v2/1.0.0
WARNING:absl:options.experimental_threading is deprecated. Use options.threading instead.


ResNet18_lr=0.3
INFO:absl:Constructing tf.data.Dataset imagenette for split validation[0:3925], from /root/tensorflow_datasets/imagenette/full-size-v2/1.0.0
WARNING:absl:options.experimental_threading is deprecated. Use options.threading instead.
INFO:absl:Restoring legacy Flax checkpoint from models/ResNet18_lr=0.3/checkpoint_1475
INFO:absl:Initial compilation, this might take some minutes...
[45]
upload_data

Load pre-trained model

[46]
-rw-r--r-- 1 root root 196M Apr  3 13:38 checkpoint_250200
[47]
[48]
INFO:absl:Restoring legacy Flax checkpoint from ./checkpoint_250200

Inference

[49]
{'image': TensorShape([32, 224, 224, 3]), 'label': TensorShape([32])}
[50]
[51]
[16, 24]
[52]
Output
[53]
INFO:absl:Constructing tf.data.Dataset imagenette for split validation[0:3925], from /root/tensorflow_datasets/imagenette/full-size-v2/1.0.0
WARNING:absl:options.experimental_threading is deprecated. Use options.threading instead.
[54]
Step 1/7...
Step 2/7...
Step 3/7...
Step 4/7...
Step 5/7...
Step 6/7...
Step 7/7...
Array(0.9118304, dtype=float32)