• Tutorials >
  • Ray Tune을 이용한 하이퍼파라미터 튜닝
Shortcuts

Ray Tune을 이용한 하이퍼파라미터 튜닝

번역: 심형준 하이퍼파라미터 튜닝은 보통의 모델과 매우 정확한 모델간의 차이를 만들어 낼 수 있습니다. 종종 다른 학습률(Learnig rate)을 선택하거나 layer size를 변경하는 것과 같은 간단한 작업만으로도 모델 성능에 큰 영향을 미치기도 합니다. 다행히, 최적의 매개변수 조합을 찾는데 도움이 되는 도구가 있습니다. Ray Tune 은 분산 하이퍼파라미터 튜닝을 위한 업계 표준 도구입니다. Ray Tune은 최신 하이퍼파라미터 검색 알고리즘을 포함하고 TensorBoard 및 기타 분석 라이브러리와 통합되며 기본적으로 Ray’ 의 분산 기계 학습 엔진 을 통해 교육을 지원합니다. 이 튜토리얼은 Ray Tune을 파이토치 학습 workflow에 통합하는 방법을 알려줍니다. CIFAR10 이미지 분류기를 훈련하기 위해 파이토치 문서에서 이 튜토리얼을 확장할 것입니다. 아래와 같이 약간의 수정만 추가하면 됩니다. 1. 함수에서 데이터 로딩 및 학습 부분을 감싸두고, 2. 일부 네트워크 파라미터를 구성 가능하게 하고, 3. 체크포인트를 추가하고 (선택 사항), 4. 모델 튜닝을 위한 검색 공간을 정의합니다. | 이 튜토리얼을 실행하기 위해 아래의 패키지가 설치되어 있는지 확인하십시오. - ray[tune]: 배포된 하이퍼파라미터 튜닝 라이브러리 - torchvision: 데이터 트랜스포머의 경우 설정 / Imports ————— import들로 시작합니다.

from functools import partial
import numpy as np
import os
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
from torch.utils.data import random_split
import torchvision
import torchvision.transforms as transforms
from ray import tune
from ray.tune import CLIReporter
from ray.tune.schedulers import ASHAScheduler

대부분의 import들은 파이토치 모델을 빌드하는데 필요합니다. 마지막 세 개의 import들만 Ray Tune을 사용하기 위한 것입니다.

Data loaders

data loader를 자체 함수로 감싸두고 전역 데이터 디렉토리로 전달합니다. 이런 식으로 서로 다른 실험들 간에 데이터 디렉토리를 공유할 수 있습니다.

def load_data(data_dir="./data"):
    transform = transforms.Compose([
        transforms.ToTensor(),
        transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))
    ])

    trainset = torchvision.datasets.CIFAR10(
        root=data_dir, train=True, download=True, transform=transform)

    testset = torchvision.datasets.CIFAR10(
        root=data_dir, train=False, download=True, transform=transform)

    return trainset, testset

구성 가능한 신경망

구성 가능한 파라미터만 튜닝이 가능합니다. 이 예시를 통해 fully connected layer 크기를 지정할 수 있습니다.

class Net(nn.Module):
    def __init__(self, l1=120, l2=84):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(3, 6, 5)
        self.pool = nn.MaxPool2d(2, 2)
        self.conv2 = nn.Conv2d(6, 16, 5)
        self.fc1 = nn.Linear(16 * 5 * 5, l1)
        self.fc2 = nn.Linear(l1, l2)
        self.fc3 = nn.Linear(l2, 10)

    def forward(self, x):
        x = self.pool(F.relu(self.conv1(x)))
        x = self.pool(F.relu(self.conv2(x)))
        x = x.view(-1, 16 * 5 * 5)
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        return x

학습 함수

흥미롭게 하기 위해 파이토치 문서에서 예제에 일부를 변경하여 소개합니다.

훈련 스크립트를 train_cifar(config, checkpoint_dir=None, data_dir=None) 함수로 감싸둡니다. 짐작할 수 있듯이, config 매개변수는 훈련할 하이퍼파라미터를 받습니다. checkpoint_dir 매개변수는 체크포인트를 복원하는 데 사용됩니다. data_dir 은 데이터를 읽고 저장하는 디렉토리를 지정하므로, 여러 실행들이 동일한 데이터 소스를 공유할 수 있습니다.

net = Net(config["l1"], config["l2"])

if checkpoint_dir:
    model_state, optimizer_state = torch.load(
        os.path.join(checkpoint_dir, "checkpoint"))
    net.load_state_dict(model_state)
    optimizer.load_state_dict(optimizer_state)

또한, 옵티마이저의 학습률(learning rate)을 구성할 수 있습니다.

optimizer = optim.SGD(net.parameters(), lr=config["lr"], momentum=0.9)

또한 학습 데이터를 학습 및 검증 세트로 나눕니다. 따라서 데이터의 80%는 모델 학습에 사용하고, 나머지 20%에 대해 유효성 검사 및 손실을 계산합니다. 학습 및 테스트 세트를 반복하는 배치 크기도 구성할 수 있습니다.

DataParallel을 이용한 GPU(다중)지원 추가

이미지 분류는 GPU를 사용할 때 이점이 많습니다. 운좋게도 Ray Tune에서 파이토치의 추상화를 계속 사용할 수 있습니다. 따라서 여러 GPU에서 데이터 병렬 훈련을 지원하기 위해 모델을 nn.DataParallel 으로 감쌀 수 있습니다.

device = "cpu"
if torch.cuda.is_available():
    device = "cuda:0"
    if torch.cuda.device_count() > 1:
        net = nn.DataParallel(net)
net.to(device)

device 변수를 사용하여 사용 가능한 GPU가 없을 때도 학습이 가능한지 확인합니다. 파이토치는 다음과 같이 데이터를 GPU메모리에 명시적으로 보내도록 요구합니다.

for i, data in enumerate(trainloader, 0):
    inputs, labels = data
    inputs, labels = inputs.to(device), labels.to(device)

이 코드는 이제 CPU들, 단일 GPU 및 다중 GPU에 대한 학습을 지원합니다. 특히 Ray는 부분GPU 도 지원하므로 모델이 GPU 메모리에 적합한 상황에서는 테스트 간에 GPU를 공유할 수 있습니다. 이는 나중에 다룰 것입니다.

Ray Tune과 소통하기

가장 흥미로운 부분은 Ray Tune과의 소통입니다.

with tune.checkpoint_dir(epoch) as checkpoint_dir:
    path = os.path.join(checkpoint_dir, "checkpoint")
    torch.save((net.state_dict(), optimizer.state_dict()), path)

tune.report(loss=(val_loss / val_steps), accuracy=correct / total)

여기서 먼저 체크포인트를 저장한 다음 일부 메트릭을 Ray Tune에 다시 보냅니다. 특히, validation loss와 accuracy를 Ray Tune으로 다시 보냅니다. 그 후 Ray Tune은 이러한 메트릭을 사용하여 최상의 결과를 유도하는 하이퍼파라미터 구성을 결정할 수 있습니다. 이러한 메트릭들은 또한 리소스 낭비를 방지하기 위해 성능이 좋지 않은 실험을 조기에 중지하는 데 사용할 수 있습니다.

체크포인트 저장은 선택사항이지만 Population Based Training 과 같은 고급 스케줄러를 사용하려면 필요합니다. 또한 체크포인트를 저장하면 나중에 학습된 모델을 로드하고 평가 세트(test set)에서 검증할 수 있습니다.

Full training function

전체 코드 예제는 다음과 같습니다.

def train_cifar(config, checkpoint_dir=None, data_dir=None):
    net = Net(config["l1"], config["l2"])

    device = "cpu"
    if torch.cuda.is_available():
        device = "cuda:0"
        if torch.cuda.device_count() > 1:
            net = nn.DataParallel(net)
    net.to(device)

    criterion = nn.CrossEntropyLoss()
    optimizer = optim.SGD(net.parameters(), lr=config["lr"], momentum=0.9)

    if checkpoint_dir:
        model_state, optimizer_state = torch.load(
            os.path.join(checkpoint_dir, "checkpoint"))
        net.load_state_dict(model_state)
        optimizer.load_state_dict(optimizer_state)

    trainset, testset = load_data(data_dir)

    test_abs = int(len(trainset) * 0.8)
    train_subset, val_subset = random_split(
        trainset, [test_abs, len(trainset) - test_abs])

    trainloader = torch.utils.data.DataLoader(
        train_subset,
        batch_size=int(config["batch_size"]),
        shuffle=True,
        num_workers=8)
    valloader = torch.utils.data.DataLoader(
        val_subset,
        batch_size=int(config["batch_size"]),
        shuffle=True,
        num_workers=8)

    for epoch in range(10):  # loop over the dataset multiple times
        running_loss = 0.0
        epoch_steps = 0
        for i, data in enumerate(trainloader, 0):
            # get the inputs; data is a list of [inputs, labels]
            inputs, labels = data
            inputs, labels = inputs.to(device), labels.to(device)

            # zero the parameter gradients
            optimizer.zero_grad()

            # forward + backward + optimize
            outputs = net(inputs)
            loss = criterion(outputs, labels)
            loss.backward()
            optimizer.step()

            # print statistics
            running_loss += loss.item()
            epoch_steps += 1
            if i % 2000 == 1999:  # print every 2000 mini-batches
                print("[%d, %5d] loss: %.3f" % (epoch + 1, i + 1,
                                                running_loss / epoch_steps))
                running_loss = 0.0

        # Validation loss
        val_loss = 0.0
        val_steps = 0
        total = 0
        correct = 0
        for i, data in enumerate(valloader, 0):
            with torch.no_grad():
                inputs, labels = data
                inputs, labels = inputs.to(device), labels.to(device)

                outputs = net(inputs)
                _, predicted = torch.max(outputs.data, 1)
                total += labels.size(0)
                correct += (predicted == labels).sum().item()

                loss = criterion(outputs, labels)
                val_loss += loss.cpu().numpy()
                val_steps += 1

        with tune.checkpoint_dir(epoch) as checkpoint_dir:
            path = os.path.join(checkpoint_dir, "checkpoint")
            torch.save((net.state_dict(), optimizer.state_dict()), path)

        tune.report(loss=(val_loss / val_steps), accuracy=correct / total)
    print("Finished Training")

보다시피, 대부분의 코드는 원본 예제에서 직접 적용되었습니다.

Test set 정확도(accuracy)

일반적으로 머신러닝 모델의 성능은 모델 학습에 사용되지 않은 데이터를 사용해 테스트합니다. Test set 또한 함수로 감싸둘 수 있습니다.

def test_accuracy(net, device="cpu"):
    trainset, testset = load_data()

    testloader = torch.utils.data.DataLoader(
        testset, batch_size=4, shuffle=False, num_workers=2)

    correct = 0
    total = 0
    with torch.no_grad():
        for data in testloader:
            images, labels = data
            images, labels = images.to(device), labels.to(device)
            outputs = net(images)
            _, predicted = torch.max(outputs.data, 1)
            total += labels.size(0)
            correct += (predicted == labels).sum().item()

    return correct / total

이 함수는 또한 device 파라미터를 요구하므로, test set 평가를 GPU에서 수행할 수 있습니다.

검색 공간 구성

마지막으로 Ray Tune의 검색 공간을 정의해야 합니다. 예시는 아래와 같습니다.

config = {
    "l1": tune.sample_from(lambda _: 2**np.random.randint(2, 9)),
    "l2": tune.sample_from(lambda _: 2**np.random.randint(2, 9)),
    "lr": tune.loguniform(1e-4, 1e-1),
    "batch_size": tune.choice([2, 4, 8, 16])
}

tune.sample_from() 함수를 사용하면 고유한 샘플 방법을 정의하여 하이퍼파라미터를 얻을 수 있습니다. 이 예제에서 l1l2 파라미터는 4와 256 사이의 2의 거듭제곱이어야 하므로 4, 8, 16, 32, 64, 128, 256입니다. lr (학습률)은 0.0001과 0.1 사이에서 균일하게 샘플링 되어아 합니다. 마지막으로, 배치 크기는 2, 4, 8, 16중에서 선택할 수 있습니다.

각 실험에서, Ray Tune은 이제 이러한 검색 공간에서 매개변수 조합을 무작위로 샘플링합니다. 그런 다음 여러 모델을 병렬로 훈련하고 이 중에서 가장 성능이 좋은 모델을 찾습니다. 또한 성능이 좋지 않은 실험을 조기에 종료하는 ASHAScheduler 를 사용합니다.

상수 data_dir 파라미터를 설정하기 위해 functools.partialtrain_cifar 함수를 감싸둡니다. 또한 각 실험에 사용할 수 있는 자원들(resources)을 Ray Tune에 알릴 수 있습니다.

gpus_per_trial = 2
# ...
result = tune.run(
    partial(train_cifar, data_dir=data_dir),
    resources_per_trial={"cpu": 8, "gpu": gpus_per_trial},
    config=config,
    num_samples=num_samples,
    scheduler=scheduler,
    progress_reporter=reporter,
    checkpoint_at_end=True)

파이토치 DataLoader 인스턴스의 num_workers 을 늘리기 위해 CPU 수를 지정하고 사용할 수 있습니다. 각 실험에서 선택한 수의 GPU들은 파이토치에 표시됩니다. 실험들은 요청되지 않은 GPU에 액세스할 수 없으므로 같은 자원들을 사용하는 중복된 실험에 대해 신경쓰지 않아도 됩니다.

부분 GPUs를 지정할 수도 있으므로, gpus_per_trial=0.5 와 같은 것 또한 가능합니다. 이후 각 실험은 GPU를 공유합니다. 사용자는 모델이 여전히 GPU메모리에 적합한지만 확인하면 됩니다.

모델을 훈련시킨 후, 가장 성능이 좋은 모델을 찾고 체크포인트 파일에서 학습된 모델을 로드합니다. 이후 test set 정확도(accuracy)를 얻고 모든 것들을 출력하여 확인할 수 있습니다.

전체 주요 기능은 다음과 같습니다.

def main(num_samples=10, max_num_epochs=10, gpus_per_trial=2):
    data_dir = os.path.abspath("./data")
    load_data(data_dir)
    config = {
        "l1": tune.sample_from(lambda _: 2 ** np.random.randint(2, 9)),
        "l2": tune.sample_from(lambda _: 2 ** np.random.randint(2, 9)),
        "lr": tune.loguniform(1e-4, 1e-1),
        "batch_size": tune.choice([2, 4, 8, 16])
    }
    scheduler = ASHAScheduler(
        metric="loss",
        mode="min",
        max_t=max_num_epochs,
        grace_period=1,
        reduction_factor=2)
    reporter = CLIReporter(
        # parameter_columns=["l1", "l2", "lr", "batch_size"],
        metric_columns=["loss", "accuracy", "training_iteration"])
    result = tune.run(
        partial(train_cifar, data_dir=data_dir),
        resources_per_trial={"cpu": 2, "gpu": gpus_per_trial},
        config=config,
        num_samples=num_samples,
        scheduler=scheduler,
        progress_reporter=reporter)

    best_trial = result.get_best_trial("loss", "min", "last")
    print("Best trial config: {}".format(best_trial.config))
    print("Best trial final validation loss: {}".format(
        best_trial.last_result["loss"]))
    print("Best trial final validation accuracy: {}".format(
        best_trial.last_result["accuracy"]))

    best_trained_model = Net(best_trial.config["l1"], best_trial.config["l2"])
    device = "cpu"
    if torch.cuda.is_available():
        device = "cuda:0"
        if gpus_per_trial > 1:
            best_trained_model = nn.DataParallel(best_trained_model)
    best_trained_model.to(device)

    best_checkpoint_dir = best_trial.checkpoint.value
    model_state, optimizer_state = torch.load(os.path.join(
        best_checkpoint_dir, "checkpoint"))
    best_trained_model.load_state_dict(model_state)

    test_acc = test_accuracy(best_trained_model, device)
    print("Best trial test set accuracy: {}".format(test_acc))


if __name__ == "__main__":
    # You can change the number of GPUs per trial here:
    main(num_samples=10, max_num_epochs=10, gpus_per_trial=0)

Out:

Files already downloaded and verified
Files already downloaded and verified
== Status ==
Current time: 2022-09-12 15:20:12 (running for 00:00:00.26)
Memory usage on this node: 4.7/62.7 GiB
Using AsyncHyperBand: num_stopped=0
Bracket: Iter 8.000: None | Iter 4.000: None | Iter 2.000: None | Iter 1.000: None
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (9 PENDING, 1 RUNNING)
+-------------------------+----------+-----------------+--------------+------+------+-------------+
| Trial name              | status   | loc             |   batch_size |   l1 |   l2 |          lr |
|-------------------------+----------+-----------------+--------------+------+------+-------------|
| train_cifar_f53e3_00000 | RUNNING  | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  |
| train_cifar_f53e3_00001 | PENDING  |                 |           16 |   64 |    8 | 0.000158014 |
| train_cifar_f53e3_00002 | PENDING  |                 |            4 |    8 |   32 | 0.011652    |
| train_cifar_f53e3_00003 | PENDING  |                 |           16 |  256 |   64 | 0.065364    |
| train_cifar_f53e3_00004 | PENDING  |                 |            4 |    4 |  128 | 0.000644664 |
| train_cifar_f53e3_00005 | PENDING  |                 |            8 |   64 |    8 | 0.00661774  |
| train_cifar_f53e3_00006 | PENDING  |                 |            2 |   32 |   32 | 0.0275734   |
| train_cifar_f53e3_00007 | PENDING  |                 |            4 |  128 |    8 | 0.0687161   |
| train_cifar_f53e3_00008 | PENDING  |                 |            2 |   32 |  256 | 0.0188278   |
| train_cifar_f53e3_00009 | PENDING  |                 |           16 |  256 |   64 | 0.00179698  |
+-------------------------+----------+-----------------+--------------+------+------+-------------+


(func pid=1340) Files already downloaded and verified
(func pid=1340) Files already downloaded and verified
(func pid=1376) Files already downloaded and verified
(func pid=1386) Files already downloaded and verified
(func pid=1378) Files already downloaded and verified
(func pid=1382) Files already downloaded and verified
(func pid=1373) Files already downloaded and verified
(func pid=1370) Files already downloaded and verified
(func pid=1380) Files already downloaded and verified
(func pid=1384) Files already downloaded and verified
(func pid=1372) Files already downloaded and verified
(func pid=1378) Files already downloaded and verified
(func pid=1382) Files already downloaded and verified
(func pid=1384) Files already downloaded and verified
(func pid=1370) Files already downloaded and verified
(func pid=1373) Files already downloaded and verified
(func pid=1376) Files already downloaded and verified
(func pid=1386) Files already downloaded and verified
(func pid=1380) Files already downloaded and verified
(func pid=1372) Files already downloaded and verified
== Status ==
Current time: 2022-09-12 15:20:22 (running for 00:00:09.80)
Memory usage on this node: 10.0/62.7 GiB
Using AsyncHyperBand: num_stopped=0
Bracket: Iter 8.000: None | Iter 4.000: None | Iter 2.000: None | Iter 1.000: None
Resources requested: 20.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (10 RUNNING)
+-------------------------+----------+-----------------+--------------+------+------+-------------+
| Trial name              | status   | loc             |   batch_size |   l1 |   l2 |          lr |
|-------------------------+----------+-----------------+--------------+------+------+-------------|
| train_cifar_f53e3_00000 | RUNNING  | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  |
| train_cifar_f53e3_00001 | RUNNING  | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 |
| train_cifar_f53e3_00002 | RUNNING  | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    |
| train_cifar_f53e3_00003 | RUNNING  | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    |
| train_cifar_f53e3_00004 | RUNNING  | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 |
| train_cifar_f53e3_00005 | RUNNING  | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  |
| train_cifar_f53e3_00006 | RUNNING  | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   |
| train_cifar_f53e3_00007 | RUNNING  | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   |
| train_cifar_f53e3_00008 | RUNNING  | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   |
| train_cifar_f53e3_00009 | RUNNING  | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  |
+-------------------------+----------+-----------------+--------------+------+------+-------------+


== Status ==
Current time: 2022-09-12 15:20:27 (running for 00:00:14.82)
Memory usage on this node: 10.3/62.7 GiB
Using AsyncHyperBand: num_stopped=0
Bracket: Iter 8.000: None | Iter 4.000: None | Iter 2.000: None | Iter 1.000: None
Resources requested: 20.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (10 RUNNING)
+-------------------------+----------+-----------------+--------------+------+------+-------------+
| Trial name              | status   | loc             |   batch_size |   l1 |   l2 |          lr |
|-------------------------+----------+-----------------+--------------+------+------+-------------|
| train_cifar_f53e3_00000 | RUNNING  | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  |
| train_cifar_f53e3_00001 | RUNNING  | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 |
| train_cifar_f53e3_00002 | RUNNING  | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    |
| train_cifar_f53e3_00003 | RUNNING  | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    |
| train_cifar_f53e3_00004 | RUNNING  | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 |
| train_cifar_f53e3_00005 | RUNNING  | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  |
| train_cifar_f53e3_00006 | RUNNING  | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   |
| train_cifar_f53e3_00007 | RUNNING  | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   |
| train_cifar_f53e3_00008 | RUNNING  | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   |
| train_cifar_f53e3_00009 | RUNNING  | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  |
+-------------------------+----------+-----------------+--------------+------+------+-------------+


(func pid=1384) [1,  2000] loss: 2.302
(func pid=1380) [1,  2000] loss: 2.337
(func pid=1376) [1,  2000] loss: 2.210
(func pid=1372) [1,  2000] loss: 2.148
(func pid=1378) [1,  2000] loss: 1.984
(func pid=1382) [1,  2000] loss: 2.342
== Status ==
Current time: 2022-09-12 15:20:32 (running for 00:00:19.83)
Memory usage on this node: 10.3/62.7 GiB
Using AsyncHyperBand: num_stopped=0
Bracket: Iter 8.000: None | Iter 4.000: None | Iter 2.000: None | Iter 1.000: None
Resources requested: 20.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (10 RUNNING)
+-------------------------+----------+-----------------+--------------+------+------+-------------+
| Trial name              | status   | loc             |   batch_size |   l1 |   l2 |          lr |
|-------------------------+----------+-----------------+--------------+------+------+-------------|
| train_cifar_f53e3_00000 | RUNNING  | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  |
| train_cifar_f53e3_00001 | RUNNING  | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 |
| train_cifar_f53e3_00002 | RUNNING  | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    |
| train_cifar_f53e3_00003 | RUNNING  | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    |
| train_cifar_f53e3_00004 | RUNNING  | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 |
| train_cifar_f53e3_00005 | RUNNING  | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  |
| train_cifar_f53e3_00006 | RUNNING  | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   |
| train_cifar_f53e3_00007 | RUNNING  | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   |
| train_cifar_f53e3_00008 | RUNNING  | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   |
| train_cifar_f53e3_00009 | RUNNING  | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  |
+-------------------------+----------+-----------------+--------------+------+------+-------------+


(func pid=1340) [1,  2000] loss: 2.301
(func pid=1370) [1,  2000] loss: 2.328
(func pid=1373) [1,  2000] loss: 2.127
(func pid=1386) [1,  2000] loss: 2.005
== Status ==
Current time: 2022-09-12 15:20:37 (running for 00:00:24.85)
Memory usage on this node: 10.3/62.7 GiB
Using AsyncHyperBand: num_stopped=0
Bracket: Iter 8.000: None | Iter 4.000: None | Iter 2.000: None | Iter 1.000: None
Resources requested: 20.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (10 RUNNING)
+-------------------------+----------+-----------------+--------------+------+------+-------------+
| Trial name              | status   | loc             |   batch_size |   l1 |   l2 |          lr |
|-------------------------+----------+-----------------+--------------+------+------+-------------|
| train_cifar_f53e3_00000 | RUNNING  | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  |
| train_cifar_f53e3_00001 | RUNNING  | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 |
| train_cifar_f53e3_00002 | RUNNING  | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    |
| train_cifar_f53e3_00003 | RUNNING  | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    |
| train_cifar_f53e3_00004 | RUNNING  | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 |
| train_cifar_f53e3_00005 | RUNNING  | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  |
| train_cifar_f53e3_00006 | RUNNING  | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   |
| train_cifar_f53e3_00007 | RUNNING  | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   |
| train_cifar_f53e3_00008 | RUNNING  | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   |
| train_cifar_f53e3_00009 | RUNNING  | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  |
+-------------------------+----------+-----------------+--------------+------+------+-------------+


(func pid=1384) [1,  4000] loss: 1.133
(func pid=1380) [1,  4000] loss: 1.166
Result for train_cifar_f53e3_00000:
  accuracy: 0.1249
  date: 2022-09-12_15-20-39
  done: false
  experiment_id: ac009e0800c846ee9349150915540216
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 1
  loss: 2.291645025253296
  node_ip: 172.17.0.2
  pid: 1340
  should_checkpoint: true
  time_since_restore: 22.208389282226562
  time_this_iter_s: 22.208389282226562
  time_total_s: 22.208389282226562
  timestamp: 1662963639
  timesteps_since_restore: 0
  training_iteration: 1
  trial_id: f53e3_00000
  warmup_time: 0.00513148307800293

(func pid=1372) [1,  4000] loss: 1.062
(func pid=1376) [1,  4000] loss: 0.972
(func pid=1378) [1,  4000] loss: 0.832
Result for train_cifar_f53e3_00001:
  accuracy: 0.1103
  date: 2022-09-12_15-20-42
  done: true
  experiment_id: 5fd237127def415a82f88659bb5f8de2
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 1
  loss: 2.3098061038970945
  node_ip: 172.17.0.2
  pid: 1370
  should_checkpoint: true
  time_since_restore: 22.186343669891357
  time_this_iter_s: 22.186343669891357
  time_total_s: 22.186343669891357
  timestamp: 1662963642
  timesteps_since_restore: 0
  training_iteration: 1
  trial_id: f53e3_00001
  warmup_time: 0.004497528076171875

(func pid=1382) [1,  4000] loss: 1.171
Result for train_cifar_f53e3_00003:
  accuracy: 0.2106
  date: 2022-09-12_15-20-43
  done: false
  experiment_id: 821f3e234403400b9a55e6316ac6fcbe
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 1
  loss: 2.1076839881896974
  node_ip: 172.17.0.2
  pid: 1373
  should_checkpoint: true
  time_since_restore: 23.03403401374817
  time_this_iter_s: 23.03403401374817
  time_total_s: 23.03403401374817
  timestamp: 1662963643
  timesteps_since_restore: 0
  training_iteration: 1
  trial_id: f53e3_00003
  warmup_time: 0.004432201385498047

== Status ==
Current time: 2022-09-12 15:20:43 (running for 00:00:30.56)
Memory usage on this node: 9.6/62.7 GiB
Using AsyncHyperBand: num_stopped=1
Bracket: Iter 8.000: None | Iter 4.000: None | Iter 2.000: None | Iter 1.000: -2.291645025253296
Resources requested: 18.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (9 RUNNING, 1 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00000 | RUNNING    | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 2.29165 |     0.1249 |                    1 |
| train_cifar_f53e3_00002 | RUNNING    | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    |         |            |                      |
| train_cifar_f53e3_00003 | RUNNING    | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.10768 |     0.2106 |                    1 |
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 |         |            |                      |
| train_cifar_f53e3_00005 | RUNNING    | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  |         |            |                      |
| train_cifar_f53e3_00006 | RUNNING    | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   |         |            |                      |
| train_cifar_f53e3_00007 | RUNNING    | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   |         |            |                      |
| train_cifar_f53e3_00008 | RUNNING    | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   |         |            |                      |
| train_cifar_f53e3_00009 | RUNNING    | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  |         |            |                      |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


Result for train_cifar_f53e3_00009:
  accuracy: 0.4085
  date: 2022-09-12_15-20-43
  done: false
  experiment_id: c5c4e7bd12f54712bd1a85923db0ab39
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 1
  loss: 1.603364821243286
  node_ip: 172.17.0.2
  pid: 1386
  should_checkpoint: true
  time_since_restore: 23.682846069335938
  time_this_iter_s: 23.682846069335938
  time_total_s: 23.682846069335938
  timestamp: 1662963643
  timesteps_since_restore: 0
  training_iteration: 1
  trial_id: f53e3_00009
  warmup_time: 0.0035982131958007812

(func pid=1384) [1,  6000] loss: 0.748
(func pid=1380) [1,  6000] loss: 0.779
(func pid=1376) [1,  6000] loss: 0.588
(func pid=1372) [1,  6000] loss: 0.719
== Status ==
Current time: 2022-09-12 15:20:48 (running for 00:00:36.19)
Memory usage on this node: 9.7/62.7 GiB
Using AsyncHyperBand: num_stopped=1
Bracket: Iter 8.000: None | Iter 4.000: None | Iter 2.000: None | Iter 1.000: -2.1996645067214966
Resources requested: 18.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (9 RUNNING, 1 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00000 | RUNNING    | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 2.29165 |     0.1249 |                    1 |
| train_cifar_f53e3_00002 | RUNNING    | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    |         |            |                      |
| train_cifar_f53e3_00003 | RUNNING    | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.10768 |     0.2106 |                    1 |
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 |         |            |                      |
| train_cifar_f53e3_00005 | RUNNING    | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  |         |            |                      |
| train_cifar_f53e3_00006 | RUNNING    | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   |         |            |                      |
| train_cifar_f53e3_00007 | RUNNING    | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   |         |            |                      |
| train_cifar_f53e3_00008 | RUNNING    | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   |         |            |                      |
| train_cifar_f53e3_00009 | RUNNING    | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.60336 |     0.4085 |                    1 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


Result for train_cifar_f53e3_00005:
  accuracy: 0.4073
  date: 2022-09-12_15-20-50
  done: false
  experiment_id: 2b5a4bcb8ace4f5a96fd386a3ddf2adc
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 1
  loss: 1.627538460111618
  node_ip: 172.17.0.2
  pid: 1378
  should_checkpoint: true
  time_since_restore: 30.039711713790894
  time_this_iter_s: 30.039711713790894
  time_total_s: 30.039711713790894
  timestamp: 1662963650
  timesteps_since_restore: 0
  training_iteration: 1
  trial_id: f53e3_00005
  warmup_time: 0.004018545150756836

(func pid=1382) [1,  6000] loss: 0.780
(func pid=1340) [2,  2000] loss: 2.245
(func pid=1384) [1,  8000] loss: 0.558
(func pid=1380) [1,  8000] loss: 0.583
== Status ==
Current time: 2022-09-12 15:20:55 (running for 00:00:42.54)
Memory usage on this node: 9.7/62.7 GiB
Using AsyncHyperBand: num_stopped=1
Bracket: Iter 8.000: None | Iter 4.000: None | Iter 2.000: None | Iter 1.000: -2.1076839881896974
Resources requested: 18.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (9 RUNNING, 1 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00000 | RUNNING    | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 2.29165 |     0.1249 |                    1 |
| train_cifar_f53e3_00002 | RUNNING    | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    |         |            |                      |
| train_cifar_f53e3_00003 | RUNNING    | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.10768 |     0.2106 |                    1 |
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 |         |            |                      |
| train_cifar_f53e3_00005 | RUNNING    | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.62754 |     0.4073 |                    1 |
| train_cifar_f53e3_00006 | RUNNING    | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   |         |            |                      |
| train_cifar_f53e3_00007 | RUNNING    | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   |         |            |                      |
| train_cifar_f53e3_00008 | RUNNING    | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   |         |            |                      |
| train_cifar_f53e3_00009 | RUNNING    | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.60336 |     0.4085 |                    1 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1376) [1,  8000] loss: 0.424
(func pid=1372) [1,  8000] loss: 0.561
(func pid=1386) [2,  2000] loss: 1.527
(func pid=1373) [2,  2000] loss: 2.157
Result for train_cifar_f53e3_00000:
  accuracy: 0.2535
  date: 2022-09-12_15-20-59
  done: false
  experiment_id: ac009e0800c846ee9349150915540216
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 2
  loss: 2.0461291580200194
  node_ip: 172.17.0.2
  pid: 1340
  should_checkpoint: true
  time_since_restore: 41.75753426551819
  time_this_iter_s: 19.549144983291626
  time_total_s: 41.75753426551819
  timestamp: 1662963659
  timesteps_since_restore: 0
  training_iteration: 2
  trial_id: f53e3_00000
  warmup_time: 0.00513148307800293

(func pid=1378) [2,  2000] loss: 1.561
(func pid=1384) [1, 10000] loss: 0.445
(func pid=1380) [1, 10000] loss: 0.467
(func pid=1382) [1,  8000] loss: 0.587
Result for train_cifar_f53e3_00009:
  accuracy: 0.4694
  date: 2022-09-12_15-21-03
  done: false
  experiment_id: c5c4e7bd12f54712bd1a85923db0ab39
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 2
  loss: 1.4509816469192505
  node_ip: 172.17.0.2
  pid: 1386
  should_checkpoint: true
  time_since_restore: 43.81000900268555
  time_this_iter_s: 20.12716293334961
  time_total_s: 43.81000900268555
  timestamp: 1662963663
  timesteps_since_restore: 0
  training_iteration: 2
  trial_id: f53e3_00009
  warmup_time: 0.0035982131958007812

== Status ==
Current time: 2022-09-12 15:21:03 (running for 00:00:51.29)
Memory usage on this node: 9.6/62.7 GiB
Using AsyncHyperBand: num_stopped=1
Bracket: Iter 8.000: None | Iter 4.000: None | Iter 2.000: -1.748555402469635 | Iter 1.000: -2.1076839881896974
Resources requested: 18.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (9 RUNNING, 1 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00000 | RUNNING    | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 2.04613 |     0.2535 |                    2 |
| train_cifar_f53e3_00002 | RUNNING    | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    |         |            |                      |
| train_cifar_f53e3_00003 | RUNNING    | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.10768 |     0.2106 |                    1 |
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 |         |            |                      |
| train_cifar_f53e3_00005 | RUNNING    | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.62754 |     0.4073 |                    1 |
| train_cifar_f53e3_00006 | RUNNING    | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   |         |            |                      |
| train_cifar_f53e3_00007 | RUNNING    | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   |         |            |                      |
| train_cifar_f53e3_00008 | RUNNING    | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   |         |            |                      |
| train_cifar_f53e3_00009 | RUNNING    | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.45098 |     0.4694 |                    2 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1376) [1, 10000] loss: 0.329
(func pid=1372) [1, 10000] loss: 0.445
Result for train_cifar_f53e3_00003:
  accuracy: 0.2304
  date: 2022-09-12_15-21-04
  done: true
  experiment_id: 821f3e234403400b9a55e6316ac6fcbe
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 2
  loss: 2.0674913246154785
  node_ip: 172.17.0.2
  pid: 1373
  should_checkpoint: true
  time_since_restore: 44.78643202781677
  time_this_iter_s: 21.752398014068604
  time_total_s: 44.78643202781677
  timestamp: 1662963664
  timesteps_since_restore: 0
  training_iteration: 2
  trial_id: f53e3_00003
  warmup_time: 0.004432201385498047

(func pid=1378) [2,  4000] loss: 0.770
(func pid=1384) [1, 12000] loss: 0.370
== Status ==
Current time: 2022-09-12 15:21:09 (running for 00:00:57.33)
Memory usage on this node: 9.1/62.7 GiB
Using AsyncHyperBand: num_stopped=2
Bracket: Iter 8.000: None | Iter 4.000: None | Iter 2.000: -2.0461291580200194 | Iter 1.000: -2.1076839881896974
Resources requested: 16.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (8 RUNNING, 2 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00000 | RUNNING    | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 2.04613 |     0.2535 |                    2 |
| train_cifar_f53e3_00002 | RUNNING    | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    |         |            |                      |
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 |         |            |                      |
| train_cifar_f53e3_00005 | RUNNING    | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.62754 |     0.4073 |                    1 |
| train_cifar_f53e3_00006 | RUNNING    | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   |         |            |                      |
| train_cifar_f53e3_00007 | RUNNING    | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   |         |            |                      |
| train_cifar_f53e3_00008 | RUNNING    | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   |         |            |                      |
| train_cifar_f53e3_00009 | RUNNING    | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.45098 |     0.4694 |                    2 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1380) [1, 12000] loss: 0.389
Result for train_cifar_f53e3_00004:
  accuracy: 0.3968
  date: 2022-09-12_15-21-10
  done: false
  experiment_id: 64a5d19e7cba4006b59267ed3c905929
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 1
  loss: 1.583160871720314
  node_ip: 172.17.0.2
  pid: 1376
  should_checkpoint: true
  time_since_restore: 50.34315085411072
  time_this_iter_s: 50.34315085411072
  time_total_s: 50.34315085411072
  timestamp: 1662963670
  timesteps_since_restore: 0
  training_iteration: 1
  trial_id: f53e3_00004
  warmup_time: 0.004603147506713867

Result for train_cifar_f53e3_00002:
  accuracy: 0.1762
  date: 2022-09-12_15-21-10
  done: true
  experiment_id: 56ee7bcfd1674b41bb6a6b6844b1f2de
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 1
  loss: 2.1789206350803374
  node_ip: 172.17.0.2
  pid: 1372
  should_checkpoint: true
  time_since_restore: 50.463458776474
  time_this_iter_s: 50.463458776474
  time_total_s: 50.463458776474
  timestamp: 1662963670
  timesteps_since_restore: 0
  training_iteration: 1
  trial_id: f53e3_00002
  warmup_time: 0.004069089889526367

(func pid=1340) [3,  2000] loss: 1.948
(func pid=1382) [1, 10000] loss: 0.468
== Status ==
Current time: 2022-09-12 15:21:15 (running for 00:01:03.14)
Memory usage on this node: 8.5/62.7 GiB
Using AsyncHyperBand: num_stopped=3
Bracket: Iter 8.000: None | Iter 4.000: None | Iter 2.000: -2.0461291580200194 | Iter 1.000: -2.1076839881896974
Resources requested: 14.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (7 RUNNING, 3 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00000 | RUNNING    | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 2.04613 |     0.2535 |                    2 |
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.58316 |     0.3968 |                    1 |
| train_cifar_f53e3_00005 | RUNNING    | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.62754 |     0.4073 |                    1 |
| train_cifar_f53e3_00006 | RUNNING    | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   |         |            |                      |
| train_cifar_f53e3_00007 | RUNNING    | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   |         |            |                      |
| train_cifar_f53e3_00008 | RUNNING    | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   |         |            |                      |
| train_cifar_f53e3_00009 | RUNNING    | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.45098 |     0.4694 |                    2 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


Result for train_cifar_f53e3_00005:
  accuracy: 0.4525
  date: 2022-09-12_15-21-16
  done: false
  experiment_id: 2b5a4bcb8ace4f5a96fd386a3ddf2adc
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 2
  loss: 1.5118700037240982
  node_ip: 172.17.0.2
  pid: 1378
  should_checkpoint: true
  time_since_restore: 56.34231638908386
  time_this_iter_s: 26.30260467529297
  time_total_s: 56.34231638908386
  timestamp: 1662963676
  timesteps_since_restore: 0
  training_iteration: 2
  trial_id: f53e3_00005
  warmup_time: 0.004018545150756836

(func pid=1384) [1, 14000] loss: 0.315
(func pid=1386) [3,  2000] loss: 1.354
(func pid=1380) [1, 14000] loss: 0.333
(func pid=1376) [2,  2000] loss: 1.591
Result for train_cifar_f53e3_00000:
  accuracy: 0.3397
  date: 2022-09-12_15-21-18
  done: false
  experiment_id: ac009e0800c846ee9349150915540216
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 3
  loss: 1.8014725217819214
  node_ip: 172.17.0.2
  pid: 1340
  should_checkpoint: true
  time_since_restore: 61.03483581542969
  time_this_iter_s: 19.2773015499115
  time_total_s: 61.03483581542969
  timestamp: 1662963678
  timesteps_since_restore: 0
  training_iteration: 3
  trial_id: f53e3_00000
  warmup_time: 0.00513148307800293

Result for train_cifar_f53e3_00007:
  accuracy: 0.1007
  date: 2022-09-12_15-21-18
  done: true
  experiment_id: 69606cadda3e4736836fa74cc2a806bf
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 1
  loss: 2.3831287833690644
  node_ip: 172.17.0.2
  pid: 1382
  should_checkpoint: true
  time_since_restore: 58.64971470832825
  time_this_iter_s: 58.64971470832825
  time_total_s: 58.64971470832825
  timestamp: 1662963678
  timesteps_since_restore: 0
  training_iteration: 1
  trial_id: f53e3_00007
  warmup_time: 0.0035381317138671875

Result for train_cifar_f53e3_00009:
  accuracy: 0.5385
  date: 2022-09-12_15-21-23
  done: false
  experiment_id: c5c4e7bd12f54712bd1a85923db0ab39
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 3
  loss: 1.2876202399253844
  node_ip: 172.17.0.2
  pid: 1386
  should_checkpoint: true
  time_since_restore: 63.477320194244385
  time_this_iter_s: 19.667311191558838
  time_total_s: 63.477320194244385
  timestamp: 1662963683
  timesteps_since_restore: 0
  training_iteration: 3
  trial_id: f53e3_00009
  warmup_time: 0.0035982131958007812

== Status ==
Current time: 2022-09-12 15:21:23 (running for 00:01:10.98)
Memory usage on this node: 7.8/62.7 GiB
Using AsyncHyperBand: num_stopped=4
Bracket: Iter 8.000: None | Iter 4.000: None | Iter 2.000: -1.7789995808720587 | Iter 1.000: -2.1433023116350176
Resources requested: 12.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (6 RUNNING, 4 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00000 | RUNNING    | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.80147 |     0.3397 |                    3 |
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.58316 |     0.3968 |                    1 |
| train_cifar_f53e3_00005 | RUNNING    | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.51187 |     0.4525 |                    2 |
| train_cifar_f53e3_00006 | RUNNING    | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   |         |            |                      |
| train_cifar_f53e3_00008 | RUNNING    | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   |         |            |                      |
| train_cifar_f53e3_00009 | RUNNING    | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.28762 |     0.5385 |                    3 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1384) [1, 16000] loss: 0.274
(func pid=1380) [1, 16000] loss: 0.292
(func pid=1378) [3,  2000] loss: 1.488
(func pid=1376) [2,  4000] loss: 0.788
== Status ==
Current time: 2022-09-12 15:21:28 (running for 00:01:15.99)
Memory usage on this node: 7.9/62.7 GiB
Using AsyncHyperBand: num_stopped=4
Bracket: Iter 8.000: None | Iter 4.000: None | Iter 2.000: -1.7789995808720587 | Iter 1.000: -2.1433023116350176
Resources requested: 12.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (6 RUNNING, 4 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00000 | RUNNING    | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.80147 |     0.3397 |                    3 |
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.58316 |     0.3968 |                    1 |
| train_cifar_f53e3_00005 | RUNNING    | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.51187 |     0.4525 |                    2 |
| train_cifar_f53e3_00006 | RUNNING    | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   |         |            |                      |
| train_cifar_f53e3_00008 | RUNNING    | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   |         |            |                      |
| train_cifar_f53e3_00009 | RUNNING    | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.28762 |     0.5385 |                    3 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1340) [4,  2000] loss: 1.727
(func pid=1384) [1, 18000] loss: 0.245
(func pid=1380) [1, 18000] loss: 0.260
(func pid=1376) [2,  6000] loss: 0.518
== Status ==
Current time: 2022-09-12 15:21:33 (running for 00:01:21.01)
Memory usage on this node: 7.9/62.7 GiB
Using AsyncHyperBand: num_stopped=4
Bracket: Iter 8.000: None | Iter 4.000: None | Iter 2.000: -1.7789995808720587 | Iter 1.000: -2.1433023116350176
Resources requested: 12.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (6 RUNNING, 4 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00000 | RUNNING    | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.80147 |     0.3397 |                    3 |
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.58316 |     0.3968 |                    1 |
| train_cifar_f53e3_00005 | RUNNING    | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.51187 |     0.4525 |                    2 |
| train_cifar_f53e3_00006 | RUNNING    | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   |         |            |                      |
| train_cifar_f53e3_00008 | RUNNING    | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   |         |            |                      |
| train_cifar_f53e3_00009 | RUNNING    | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.28762 |     0.5385 |                    3 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1378) [3,  4000] loss: 0.733
Result for train_cifar_f53e3_00000:
  accuracy: 0.3937
  date: 2022-09-12_15-21-35
  done: false
  experiment_id: ac009e0800c846ee9349150915540216
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 4
  loss: 1.6428803331375121
  node_ip: 172.17.0.2
  pid: 1340
  should_checkpoint: true
  time_since_restore: 78.5964891910553
  time_this_iter_s: 17.56165337562561
  time_total_s: 78.5964891910553
  timestamp: 1662963695
  timesteps_since_restore: 0
  training_iteration: 4
  trial_id: f53e3_00000
  warmup_time: 0.00513148307800293

(func pid=1386) [4,  2000] loss: 1.234
(func pid=1384) [1, 20000] loss: 0.221
(func pid=1380) [1, 20000] loss: 0.233
(func pid=1376) [2,  8000] loss: 0.384
== Status ==
Current time: 2022-09-12 15:21:40 (running for 00:01:28.31)
Memory usage on this node: 7.9/62.7 GiB
Using AsyncHyperBand: num_stopped=4
Bracket: Iter 8.000: None | Iter 4.000: -1.6428803331375121 | Iter 2.000: -1.7789995808720587 | Iter 1.000: -2.1433023116350176
Resources requested: 12.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (6 RUNNING, 4 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00000 | RUNNING    | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.64288 |     0.3937 |                    4 |
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.58316 |     0.3968 |                    1 |
| train_cifar_f53e3_00005 | RUNNING    | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.51187 |     0.4525 |                    2 |
| train_cifar_f53e3_00006 | RUNNING    | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   |         |            |                      |
| train_cifar_f53e3_00008 | RUNNING    | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   |         |            |                      |
| train_cifar_f53e3_00009 | RUNNING    | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.28762 |     0.5385 |                    3 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


Result for train_cifar_f53e3_00005:
  accuracy: 0.4719
  date: 2022-09-12_15-21-41
  done: false
  experiment_id: 2b5a4bcb8ace4f5a96fd386a3ddf2adc
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 3
  loss: 1.5201182079076767
  node_ip: 172.17.0.2
  pid: 1378
  should_checkpoint: true
  time_since_restore: 81.34858274459839
  time_this_iter_s: 25.006266355514526
  time_total_s: 81.34858274459839
  timestamp: 1662963701
  timesteps_since_restore: 0
  training_iteration: 3
  trial_id: f53e3_00005
  warmup_time: 0.004018545150756836

Result for train_cifar_f53e3_00009:
  accuracy: 0.5647
  date: 2022-09-12_15-21-42
  done: false
  experiment_id: c5c4e7bd12f54712bd1a85923db0ab39
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 4
  loss: 1.2156572710037232
  node_ip: 172.17.0.2
  pid: 1386
  should_checkpoint: true
  time_since_restore: 82.55355477333069
  time_this_iter_s: 19.076234579086304
  time_total_s: 82.55355477333069
  timestamp: 1662963702
  timesteps_since_restore: 0
  training_iteration: 4
  trial_id: f53e3_00009
  warmup_time: 0.0035982131958007812

(func pid=1376) [2, 10000] loss: 0.304
== Status ==
Current time: 2022-09-12 15:21:47 (running for 00:01:35.05)
Memory usage on this node: 7.9/62.7 GiB
Using AsyncHyperBand: num_stopped=4
Bracket: Iter 8.000: None | Iter 4.000: -1.4292688020706177 | Iter 2.000: -1.7789995808720587 | Iter 1.000: -2.1433023116350176
Resources requested: 12.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (6 RUNNING, 4 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00000 | RUNNING    | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.64288 |     0.3937 |                    4 |
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.58316 |     0.3968 |                    1 |
| train_cifar_f53e3_00005 | RUNNING    | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.52012 |     0.4719 |                    3 |
| train_cifar_f53e3_00006 | RUNNING    | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   |         |            |                      |
| train_cifar_f53e3_00008 | RUNNING    | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   |         |            |                      |
| train_cifar_f53e3_00009 | RUNNING    | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.21566 |     0.5647 |                    4 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


Result for train_cifar_f53e3_00008:
  accuracy: 0.1597
  date: 2022-09-12_15-21-48
  done: true
  experiment_id: 1ec5d65804124fe1863344b157169d0c
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 1
  loss: 2.2149300938725474
  node_ip: 172.17.0.2
  pid: 1384
  should_checkpoint: true
  time_since_restore: 88.18377780914307
  time_this_iter_s: 88.18377780914307
  time_total_s: 88.18377780914307
  timestamp: 1662963708
  timesteps_since_restore: 0
  training_iteration: 1
  trial_id: f53e3_00008
  warmup_time: 0.0041506290435791016

(func pid=1340) [5,  2000] loss: 1.618
Result for train_cifar_f53e3_00006:
  accuracy: 0.0995
  date: 2022-09-12_15-21-49
  done: true
  experiment_id: d3a0ed7fb8cf4a499a1a31fac14dbf1c
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 1
  loss: 2.369646112084389
  node_ip: 172.17.0.2
  pid: 1380
  should_checkpoint: true
  time_since_restore: 89.44423007965088
  time_this_iter_s: 89.44423007965088
  time_total_s: 89.44423007965088
  timestamp: 1662963709
  timesteps_since_restore: 0
  training_iteration: 1
  trial_id: f53e3_00006
  warmup_time: 0.00433039665222168

(func pid=1378) [4,  2000] loss: 1.429
Result for train_cifar_f53e3_00004:
  accuracy: 0.4314
  date: 2022-09-12_15-21-52
  done: false
  experiment_id: 64a5d19e7cba4006b59267ed3c905929
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 2
  loss: 1.49821612098217
  node_ip: 172.17.0.2
  pid: 1376
  should_checkpoint: true
  time_since_restore: 92.80570340156555
  time_this_iter_s: 42.462552547454834
  time_total_s: 92.80570340156555
  timestamp: 1662963712
  timesteps_since_restore: 0
  training_iteration: 2
  trial_id: f53e3_00004
  warmup_time: 0.004603147506713867

== Status ==
Current time: 2022-09-12 15:21:52 (running for 00:01:40.29)
Memory usage on this node: 6.6/62.7 GiB
Using AsyncHyperBand: num_stopped=6
Bracket: Iter 8.000: None | Iter 4.000: -1.4292688020706177 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 8.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (4 RUNNING, 6 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00000 | RUNNING    | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.64288 |     0.3937 |                    4 |
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.49822 |     0.4314 |                    2 |
| train_cifar_f53e3_00005 | RUNNING    | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.52012 |     0.4719 |                    3 |
| train_cifar_f53e3_00009 | RUNNING    | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.21566 |     0.5647 |                    4 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


Result for train_cifar_f53e3_00000:
  accuracy: 0.4197
  date: 2022-09-12_15-21-54
  done: false
  experiment_id: ac009e0800c846ee9349150915540216
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 5
  loss: 1.5830435286521911
  node_ip: 172.17.0.2
  pid: 1340
  should_checkpoint: true
  time_since_restore: 96.96949553489685
  time_this_iter_s: 18.373006343841553
  time_total_s: 96.96949553489685
  timestamp: 1662963714
  timesteps_since_restore: 0
  training_iteration: 5
  trial_id: f53e3_00000
  warmup_time: 0.00513148307800293

(func pid=1386) [5,  2000] loss: 1.120
(func pid=1378) [4,  4000] loss: 0.717
== Status ==
Current time: 2022-09-12 15:21:59 (running for 00:01:46.70)
Memory usage on this node: 6.7/62.7 GiB
Using AsyncHyperBand: num_stopped=6
Bracket: Iter 8.000: None | Iter 4.000: -1.4292688020706177 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 8.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (4 RUNNING, 6 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00000 | RUNNING    | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.58304 |     0.4197 |                    5 |
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.49822 |     0.4314 |                    2 |
| train_cifar_f53e3_00005 | RUNNING    | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.52012 |     0.4719 |                    3 |
| train_cifar_f53e3_00009 | RUNNING    | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.21566 |     0.5647 |                    4 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1376) [3,  2000] loss: 1.512
Result for train_cifar_f53e3_00009:
  accuracy: 0.5834
  date: 2022-09-12_15-22-00
  done: false
  experiment_id: c5c4e7bd12f54712bd1a85923db0ab39
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 5
  loss: 1.180810843372345
  node_ip: 172.17.0.2
  pid: 1386
  should_checkpoint: true
  time_since_restore: 100.84891867637634
  time_this_iter_s: 18.295363903045654
  time_total_s: 100.84891867637634
  timestamp: 1662963720
  timesteps_since_restore: 0
  training_iteration: 5
  trial_id: f53e3_00009
  warmup_time: 0.0035982131958007812

== Status ==
Current time: 2022-09-12 15:22:05 (running for 00:01:53.33)
Memory usage on this node: 6.7/62.7 GiB
Using AsyncHyperBand: num_stopped=6
Bracket: Iter 8.000: None | Iter 4.000: -1.4292688020706177 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 8.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (4 RUNNING, 6 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00000 | RUNNING    | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.58304 |     0.4197 |                    5 |
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.49822 |     0.4314 |                    2 |
| train_cifar_f53e3_00005 | RUNNING    | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.52012 |     0.4719 |                    3 |
| train_cifar_f53e3_00009 | RUNNING    | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.18081 |     0.5834 |                    5 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1340) [6,  2000] loss: 1.558
Result for train_cifar_f53e3_00005:
  accuracy: 0.4898
  date: 2022-09-12_15-22-06
  done: false
  experiment_id: 2b5a4bcb8ace4f5a96fd386a3ddf2adc
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 4
  loss: 1.425996267414093
  node_ip: 172.17.0.2
  pid: 1378
  should_checkpoint: true
  time_since_restore: 106.16049981117249
  time_this_iter_s: 24.811917066574097
  time_total_s: 106.16049981117249
  timestamp: 1662963726
  timesteps_since_restore: 0
  training_iteration: 4
  trial_id: f53e3_00005
  warmup_time: 0.004018545150756836

(func pid=1376) [3,  4000] loss: 0.740
== Status ==
Current time: 2022-09-12 15:22:11 (running for 00:01:58.67)
Memory usage on this node: 6.7/62.7 GiB
Using AsyncHyperBand: num_stopped=6
Bracket: Iter 8.000: None | Iter 4.000: -1.425996267414093 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 8.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (4 RUNNING, 6 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00000 | RUNNING    | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.58304 |     0.4197 |                    5 |
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.49822 |     0.4314 |                    2 |
| train_cifar_f53e3_00005 | RUNNING    | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.426   |     0.4898 |                    4 |
| train_cifar_f53e3_00009 | RUNNING    | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.18081 |     0.5834 |                    5 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


Result for train_cifar_f53e3_00000:
  accuracy: 0.4319
  date: 2022-09-12_15-22-11
  done: false
  experiment_id: ac009e0800c846ee9349150915540216
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 6
  loss: 1.556455490398407
  node_ip: 172.17.0.2
  pid: 1340
  should_checkpoint: true
  time_since_restore: 114.24931812286377
  time_this_iter_s: 17.27982258796692
  time_total_s: 114.24931812286377
  timestamp: 1662963731
  timesteps_since_restore: 0
  training_iteration: 6
  trial_id: f53e3_00000
  warmup_time: 0.00513148307800293

(func pid=1386) [6,  2000] loss: 1.043
(func pid=1376) [3,  6000] loss: 0.498
(func pid=1378) [5,  2000] loss: 1.396
== Status ==
Current time: 2022-09-12 15:22:16 (running for 00:02:03.97)
Memory usage on this node: 6.7/62.7 GiB
Using AsyncHyperBand: num_stopped=6
Bracket: Iter 8.000: None | Iter 4.000: -1.425996267414093 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 8.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (4 RUNNING, 6 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00000 | RUNNING    | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.55646 |     0.4319 |                    6 |
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.49822 |     0.4314 |                    2 |
| train_cifar_f53e3_00005 | RUNNING    | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.426   |     0.4898 |                    4 |
| train_cifar_f53e3_00009 | RUNNING    | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.18081 |     0.5834 |                    5 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


Result for train_cifar_f53e3_00009:
  accuracy: 0.5778
  date: 2022-09-12_15-22-19
  done: false
  experiment_id: c5c4e7bd12f54712bd1a85923db0ab39
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 6
  loss: 1.200250594806671
  node_ip: 172.17.0.2
  pid: 1386
  should_checkpoint: true
  time_since_restore: 119.08098316192627
  time_this_iter_s: 18.232064485549927
  time_total_s: 119.08098316192627
  timestamp: 1662963739
  timesteps_since_restore: 0
  training_iteration: 6
  trial_id: f53e3_00009
  warmup_time: 0.0035982131958007812

(func pid=1376) [3,  8000] loss: 0.369
(func pid=1340) [7,  2000] loss: 1.501
(func pid=1378) [5,  4000] loss: 0.711
== Status ==
Current time: 2022-09-12 15:22:24 (running for 00:02:11.58)
Memory usage on this node: 6.7/62.7 GiB
Using AsyncHyperBand: num_stopped=6
Bracket: Iter 8.000: None | Iter 4.000: -1.425996267414093 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 8.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (4 RUNNING, 6 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00000 | RUNNING    | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.55646 |     0.4319 |                    6 |
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.49822 |     0.4314 |                    2 |
| train_cifar_f53e3_00005 | RUNNING    | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.426   |     0.4898 |                    4 |
| train_cifar_f53e3_00009 | RUNNING    | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.20025 |     0.5778 |                    6 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1376) [3, 10000] loss: 0.292
== Status ==
Current time: 2022-09-12 15:22:29 (running for 00:02:16.58)
Memory usage on this node: 6.7/62.7 GiB
Using AsyncHyperBand: num_stopped=6
Bracket: Iter 8.000: None | Iter 4.000: -1.425996267414093 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 8.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (4 RUNNING, 6 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00000 | RUNNING    | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.55646 |     0.4319 |                    6 |
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.49822 |     0.4314 |                    2 |
| train_cifar_f53e3_00005 | RUNNING    | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.426   |     0.4898 |                    4 |
| train_cifar_f53e3_00009 | RUNNING    | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.20025 |     0.5778 |                    6 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


Result for train_cifar_f53e3_00000:
  accuracy: 0.4613
  date: 2022-09-12_15-22-29
  done: false
  experiment_id: ac009e0800c846ee9349150915540216
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 7
  loss: 1.4805973236083985
  node_ip: 172.17.0.2
  pid: 1340
  should_checkpoint: true
  time_since_restore: 132.12202548980713
  time_this_iter_s: 17.87270736694336
  time_total_s: 132.12202548980713
  timestamp: 1662963749
  timesteps_since_restore: 0
  training_iteration: 7
  trial_id: f53e3_00000
  warmup_time: 0.00513148307800293

Result for train_cifar_f53e3_00005:
  accuracy: 0.4925
  date: 2022-09-12_15-22-31
  done: false
  experiment_id: 2b5a4bcb8ace4f5a96fd386a3ddf2adc
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 5
  loss: 1.4614243346452713
  node_ip: 172.17.0.2
  pid: 1378
  should_checkpoint: true
  time_since_restore: 131.32750821113586
  time_this_iter_s: 25.16700839996338
  time_total_s: 131.32750821113586
  timestamp: 1662963751
  timesteps_since_restore: 0
  training_iteration: 5
  trial_id: f53e3_00005
  warmup_time: 0.004018545150756836

(func pid=1386) [7,  2000] loss: 0.966
Result for train_cifar_f53e3_00004:
  accuracy: 0.4512
  date: 2022-09-12_15-22-34
  done: false
  experiment_id: 64a5d19e7cba4006b59267ed3c905929
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 3
  loss: 1.4612255866408348
  node_ip: 172.17.0.2
  pid: 1376
  should_checkpoint: true
  time_since_restore: 134.2412371635437
  time_this_iter_s: 41.43553376197815
  time_total_s: 134.2412371635437
  timestamp: 1662963754
  timesteps_since_restore: 0
  training_iteration: 3
  trial_id: f53e3_00004
  warmup_time: 0.004603147506713867

== Status ==
Current time: 2022-09-12 15:22:34 (running for 00:02:21.73)
Memory usage on this node: 6.6/62.7 GiB
Using AsyncHyperBand: num_stopped=6
Bracket: Iter 8.000: None | Iter 4.000: -1.425996267414093 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 8.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (4 RUNNING, 6 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00000 | RUNNING    | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.4806  |     0.4613 |                    7 |
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.46123 |     0.4512 |                    3 |
| train_cifar_f53e3_00005 | RUNNING    | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.46142 |     0.4925 |                    5 |
| train_cifar_f53e3_00009 | RUNNING    | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.20025 |     0.5778 |                    6 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


Result for train_cifar_f53e3_00009:
  accuracy: 0.6217
  date: 2022-09-12_15-22-37
  done: false
  experiment_id: c5c4e7bd12f54712bd1a85923db0ab39
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 7
  loss: 1.082267993402481
  node_ip: 172.17.0.2
  pid: 1386
  should_checkpoint: true
  time_since_restore: 137.6894724369049
  time_this_iter_s: 18.608489274978638
  time_total_s: 137.6894724369049
  timestamp: 1662963757
  timesteps_since_restore: 0
  training_iteration: 7
  trial_id: f53e3_00009
  warmup_time: 0.0035982131958007812

(func pid=1378) [6,  2000] loss: 1.379
(func pid=1376) [4,  2000] loss: 1.435
(func pid=1340) [8,  2000] loss: 1.451
== Status ==
Current time: 2022-09-12 15:22:42 (running for 00:02:30.17)
Memory usage on this node: 6.7/62.7 GiB
Using AsyncHyperBand: num_stopped=6
Bracket: Iter 8.000: None | Iter 4.000: -1.425996267414093 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 8.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (4 RUNNING, 6 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00000 | RUNNING    | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.4806  |     0.4613 |                    7 |
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.46123 |     0.4512 |                    3 |
| train_cifar_f53e3_00005 | RUNNING    | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.46142 |     0.4925 |                    5 |
| train_cifar_f53e3_00009 | RUNNING    | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.08227 |     0.6217 |                    7 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


Result for train_cifar_f53e3_00000:
  accuracy: 0.4743
  date: 2022-09-12_15-22-47
  done: false
  experiment_id: ac009e0800c846ee9349150915540216
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 8
  loss: 1.4436590198516845
  node_ip: 172.17.0.2
  pid: 1340
  should_checkpoint: true
  time_since_restore: 149.8423056602478
  time_this_iter_s: 17.720280170440674
  time_total_s: 149.8423056602478
  timestamp: 1662963767
  timesteps_since_restore: 0
  training_iteration: 8
  trial_id: f53e3_00000
  warmup_time: 0.00513148307800293

(func pid=1376) [4,  4000] loss: 0.711
(func pid=1378) [6,  4000] loss: 0.702
(func pid=1386) [8,  2000] loss: 0.907
== Status ==
Current time: 2022-09-12 15:22:52 (running for 00:02:39.57)
Memory usage on this node: 6.7/62.7 GiB
Using AsyncHyperBand: num_stopped=6
Bracket: Iter 8.000: -1.4436590198516845 | Iter 4.000: -1.425996267414093 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 8.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (4 RUNNING, 6 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00000 | RUNNING    | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.44366 |     0.4743 |                    8 |
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.46123 |     0.4512 |                    3 |
| train_cifar_f53e3_00005 | RUNNING    | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.46142 |     0.4925 |                    5 |
| train_cifar_f53e3_00009 | RUNNING    | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.08227 |     0.6217 |                    7 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


Result for train_cifar_f53e3_00009:
  accuracy: 0.6208
  date: 2022-09-12_15-22-55
  done: false
  experiment_id: c5c4e7bd12f54712bd1a85923db0ab39
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 8
  loss: 1.1144612930774689
  node_ip: 172.17.0.2
  pid: 1386
  should_checkpoint: true
  time_since_restore: 155.81853008270264
  time_this_iter_s: 18.12905764579773
  time_total_s: 155.81853008270264
  timestamp: 1662963775
  timesteps_since_restore: 0
  training_iteration: 8
  trial_id: f53e3_00009
  warmup_time: 0.0035982131958007812

(func pid=1376) [4,  6000] loss: 0.469
Result for train_cifar_f53e3_00005:
  accuracy: 0.4746
  date: 2022-09-12_15-22-56
  done: false
  experiment_id: 2b5a4bcb8ace4f5a96fd386a3ddf2adc
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 6
  loss: 1.4867421466350554
  node_ip: 172.17.0.2
  pid: 1378
  should_checkpoint: true
  time_since_restore: 156.89899611473083
  time_this_iter_s: 25.57148790359497
  time_total_s: 156.89899611473083
  timestamp: 1662963776
  timesteps_since_restore: 0
  training_iteration: 6
  trial_id: f53e3_00005
  warmup_time: 0.004018545150756836

(func pid=1340) [9,  2000] loss: 1.406
== Status ==
Current time: 2022-09-12 15:23:01 (running for 00:02:49.42)
Memory usage on this node: 6.7/62.7 GiB
Using AsyncHyperBand: num_stopped=6
Bracket: Iter 8.000: -1.2790601564645767 | Iter 4.000: -1.425996267414093 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 8.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (4 RUNNING, 6 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00000 | RUNNING    | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.44366 |     0.4743 |                    8 |
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.46123 |     0.4512 |                    3 |
| train_cifar_f53e3_00005 | RUNNING    | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48674 |     0.4746 |                    6 |
| train_cifar_f53e3_00009 | RUNNING    | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.11446 |     0.6208 |                    8 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1376) [4,  8000] loss: 0.344
Result for train_cifar_f53e3_00000:
  accuracy: 0.5
  date: 2022-09-12_15-23-04
  done: false
  experiment_id: ac009e0800c846ee9349150915540216
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 9
  loss: 1.3949406002044678
  node_ip: 172.17.0.2
  pid: 1340
  should_checkpoint: true
  time_since_restore: 167.34802556037903
  time_this_iter_s: 17.505719900131226
  time_total_s: 167.34802556037903
  timestamp: 1662963784
  timesteps_since_restore: 0
  training_iteration: 9
  trial_id: f53e3_00000
  warmup_time: 0.00513148307800293

(func pid=1378) [7,  2000] loss: 1.370
(func pid=1386) [9,  2000] loss: 0.845
== Status ==
Current time: 2022-09-12 15:23:09 (running for 00:02:57.06)
Memory usage on this node: 6.7/62.7 GiB
Using AsyncHyperBand: num_stopped=6
Bracket: Iter 8.000: -1.2790601564645767 | Iter 4.000: -1.425996267414093 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 8.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (4 RUNNING, 6 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00000 | RUNNING    | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39494 |     0.5    |                    9 |
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.46123 |     0.4512 |                    3 |
| train_cifar_f53e3_00005 | RUNNING    | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48674 |     0.4746 |                    6 |
| train_cifar_f53e3_00009 | RUNNING    | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.11446 |     0.6208 |                    8 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1376) [4, 10000] loss: 0.274
Result for train_cifar_f53e3_00009:
  accuracy: 0.6209
  date: 2022-09-12_15-23-14
  done: false
  experiment_id: c5c4e7bd12f54712bd1a85923db0ab39
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 9
  loss: 1.1149455190181732
  node_ip: 172.17.0.2
  pid: 1386
  should_checkpoint: true
  time_since_restore: 174.27626705169678
  time_this_iter_s: 18.45773696899414
  time_total_s: 174.27626705169678
  timestamp: 1662963794
  timesteps_since_restore: 0
  training_iteration: 9
  trial_id: f53e3_00009
  warmup_time: 0.0035982131958007812

(func pid=1378) [7,  4000] loss: 0.705
Result for train_cifar_f53e3_00004:
  accuracy: 0.5046
  date: 2022-09-12_15-23-16
  done: false
  experiment_id: 64a5d19e7cba4006b59267ed3c905929
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 4
  loss: 1.3342860118269921
  node_ip: 172.17.0.2
  pid: 1376
  should_checkpoint: true
  time_since_restore: 176.07758498191833
  time_this_iter_s: 41.836347818374634
  time_total_s: 176.07758498191833
  timestamp: 1662963796
  timesteps_since_restore: 0
  training_iteration: 4
  trial_id: f53e3_00004
  warmup_time: 0.004603147506713867

== Status ==
Current time: 2022-09-12 15:23:16 (running for 00:03:03.56)
Memory usage on this node: 6.6/62.7 GiB
Using AsyncHyperBand: num_stopped=6
Bracket: Iter 8.000: -1.2790601564645767 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 8.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (4 RUNNING, 6 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00000 | RUNNING    | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39494 |     0.5    |                    9 |
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.33429 |     0.5046 |                    4 |
| train_cifar_f53e3_00005 | RUNNING    | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48674 |     0.4746 |                    6 |
| train_cifar_f53e3_00009 | RUNNING    | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.11495 |     0.6209 |                    9 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1340) [10,  2000] loss: 1.368
== Status ==
Current time: 2022-09-12 15:23:21 (running for 00:03:08.57)
Memory usage on this node: 6.7/62.7 GiB
Using AsyncHyperBand: num_stopped=6
Bracket: Iter 8.000: -1.2790601564645767 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 8.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (4 RUNNING, 6 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00000 | RUNNING    | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39494 |     0.5    |                    9 |
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.33429 |     0.5046 |                    4 |
| train_cifar_f53e3_00005 | RUNNING    | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48674 |     0.4746 |                    6 |
| train_cifar_f53e3_00009 | RUNNING    | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.11495 |     0.6209 |                    9 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


Result for train_cifar_f53e3_00000:
  accuracy: 0.5003
  date: 2022-09-12_15-23-22
  done: true
  experiment_id: ac009e0800c846ee9349150915540216
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 10
  loss: 1.3902738251686095
  node_ip: 172.17.0.2
  pid: 1340
  should_checkpoint: true
  time_since_restore: 184.83101630210876
  time_this_iter_s: 17.482990741729736
  time_total_s: 184.83101630210876
  timestamp: 1662963802
  timesteps_since_restore: 0
  training_iteration: 10
  trial_id: f53e3_00000
  warmup_time: 0.00513148307800293

Result for train_cifar_f53e3_00005:
  accuracy: 0.4654
  date: 2022-09-12_15-23-23
  done: false
  experiment_id: 2b5a4bcb8ace4f5a96fd386a3ddf2adc
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 7
  loss: 1.629510760140419
  node_ip: 172.17.0.2
  pid: 1378
  should_checkpoint: true
  time_since_restore: 182.97178840637207
  time_this_iter_s: 26.072792291641235
  time_total_s: 182.97178840637207
  timestamp: 1662963803
  timesteps_since_restore: 0
  training_iteration: 7
  trial_id: f53e3_00005
  warmup_time: 0.004018545150756836

(func pid=1376) [5,  2000] loss: 1.338
(func pid=1386) [10,  2000] loss: 0.790
== Status ==
Current time: 2022-09-12 15:23:28 (running for 00:03:15.50)
Memory usage on this node: 6.1/62.7 GiB
Using AsyncHyperBand: num_stopped=7
Bracket: Iter 8.000: -1.2790601564645767 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 6.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (3 RUNNING, 7 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.33429 |     0.5046 |                    4 |
| train_cifar_f53e3_00005 | RUNNING    | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.62951 |     0.4654 |                    7 |
| train_cifar_f53e3_00009 | RUNNING    | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.11495 |     0.6209 |                    9 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1376) [5,  4000] loss: 0.671
(func pid=1378) [8,  2000] loss: 1.351
Result for train_cifar_f53e3_00009:
  accuracy: 0.6232
  date: 2022-09-12_15-23-32
  done: true
  experiment_id: c5c4e7bd12f54712bd1a85923db0ab39
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 10
  loss: 1.144857809972763
  node_ip: 172.17.0.2
  pid: 1386
  should_checkpoint: true
  time_since_restore: 192.63090991973877
  time_this_iter_s: 18.354642868041992
  time_total_s: 192.63090991973877
  timestamp: 1662963812
  timesteps_since_restore: 0
  training_iteration: 10
  trial_id: f53e3_00009
  warmup_time: 0.0035982131958007812

(func pid=1376) [5,  6000] loss: 0.444
== Status ==
Current time: 2022-09-12 15:23:37 (running for 00:03:25.12)
Memory usage on this node: 5.5/62.7 GiB
Using AsyncHyperBand: num_stopped=8
Bracket: Iter 8.000: -1.2790601564645767 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 4.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (2 RUNNING, 8 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.33429 |     0.5046 |                    4 |
| train_cifar_f53e3_00005 | RUNNING    | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.62951 |     0.4654 |                    7 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1378) [8,  4000] loss: 0.692
== Status ==
Current time: 2022-09-12 15:23:42 (running for 00:03:30.14)
Memory usage on this node: 5.5/62.7 GiB
Using AsyncHyperBand: num_stopped=8
Bracket: Iter 8.000: -1.2790601564645767 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 4.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (2 RUNNING, 8 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.33429 |     0.5046 |                    4 |
| train_cifar_f53e3_00005 | RUNNING    | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.62951 |     0.4654 |                    7 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1376) [5,  8000] loss: 0.329
== Status ==
Current time: 2022-09-12 15:23:47 (running for 00:03:35.15)
Memory usage on this node: 5.5/62.7 GiB
Using AsyncHyperBand: num_stopped=8
Bracket: Iter 8.000: -1.2790601564645767 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 4.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (2 RUNNING, 8 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.33429 |     0.5046 |                    4 |
| train_cifar_f53e3_00005 | RUNNING    | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.62951 |     0.4654 |                    7 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


Result for train_cifar_f53e3_00005:
  accuracy: 0.5097
  date: 2022-09-12_15-23-49
  done: true
  experiment_id: 2b5a4bcb8ace4f5a96fd386a3ddf2adc
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 8
  loss: 1.4889359748840332
  node_ip: 172.17.0.2
  pid: 1378
  should_checkpoint: true
  time_since_restore: 209.05268478393555
  time_this_iter_s: 26.080896377563477
  time_total_s: 209.05268478393555
  timestamp: 1662963829
  timesteps_since_restore: 0
  training_iteration: 8
  trial_id: f53e3_00005
  warmup_time: 0.004018545150756836

(func pid=1376) [5, 10000] loss: 0.264
== Status ==
Current time: 2022-09-12 15:23:54 (running for 00:03:41.56)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.4436590198516845 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.33429 |     0.5046 |                    4 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


Result for train_cifar_f53e3_00004:
  accuracy: 0.5208
  date: 2022-09-12_15-23-57
  done: false
  experiment_id: 64a5d19e7cba4006b59267ed3c905929
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 5
  loss: 1.2990890190422535
  node_ip: 172.17.0.2
  pid: 1376
  should_checkpoint: true
  time_since_restore: 217.4419882297516
  time_this_iter_s: 41.36440324783325
  time_total_s: 217.4419882297516
  timestamp: 1662963837
  timesteps_since_restore: 0
  training_iteration: 5
  trial_id: f53e3_00004
  warmup_time: 0.004603147506713867

== Status ==
Current time: 2022-09-12 15:24:02 (running for 00:03:49.93)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.4436590198516845 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.29909 |     0.5208 |                    5 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


== Status ==
Current time: 2022-09-12 15:24:07 (running for 00:03:54.95)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.4436590198516845 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.29909 |     0.5208 |                    5 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1376) [6,  2000] loss: 1.306
== Status ==
Current time: 2022-09-12 15:24:12 (running for 00:03:59.97)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.4436590198516845 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.29909 |     0.5208 |                    5 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


== Status ==
Current time: 2022-09-12 15:24:17 (running for 00:04:04.98)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.4436590198516845 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.29909 |     0.5208 |                    5 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1376) [6,  4000] loss: 0.649
== Status ==
Current time: 2022-09-12 15:24:22 (running for 00:04:09.99)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.4436590198516845 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.29909 |     0.5208 |                    5 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1376) [6,  6000] loss: 0.432
== Status ==
Current time: 2022-09-12 15:24:27 (running for 00:04:15.01)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.4436590198516845 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.29909 |     0.5208 |                    5 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


== Status ==
Current time: 2022-09-12 15:24:32 (running for 00:04:20.02)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.4436590198516845 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.29909 |     0.5208 |                    5 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1376) [6,  8000] loss: 0.325
== Status ==
Current time: 2022-09-12 15:24:37 (running for 00:04:25.04)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.4436590198516845 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.29909 |     0.5208 |                    5 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1376) [6, 10000] loss: 0.260
== Status ==
Current time: 2022-09-12 15:24:42 (running for 00:04:30.05)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.4436590198516845 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.29909 |     0.5208 |                    5 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


== Status ==
Current time: 2022-09-12 15:24:47 (running for 00:04:35.06)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.4436590198516845 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.29909 |     0.5208 |                    5 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


Result for train_cifar_f53e3_00004:
  accuracy: 0.5042
  date: 2022-09-12_15-24-48
  done: false
  experiment_id: 64a5d19e7cba4006b59267ed3c905929
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 6
  loss: 1.358498250722885
  node_ip: 172.17.0.2
  pid: 1376
  should_checkpoint: true
  time_since_restore: 268.12124729156494
  time_this_iter_s: 50.679259061813354
  time_total_s: 268.12124729156494
  timestamp: 1662963888
  timesteps_since_restore: 0
  training_iteration: 6
  trial_id: f53e3_00004
  warmup_time: 0.004603147506713867

== Status ==
Current time: 2022-09-12 15:24:53 (running for 00:04:40.61)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.4436590198516845 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.3585  |     0.5042 |                    6 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


== Status ==
Current time: 2022-09-12 15:24:58 (running for 00:04:45.63)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.4436590198516845 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.3585  |     0.5042 |                    6 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1376) [7,  2000] loss: 1.255
== Status ==
Current time: 2022-09-12 15:25:03 (running for 00:04:50.64)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.4436590198516845 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.3585  |     0.5042 |                    6 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


== Status ==
Current time: 2022-09-12 15:25:08 (running for 00:04:55.66)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.4436590198516845 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.3585  |     0.5042 |                    6 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1376) [7,  4000] loss: 0.637
== Status ==
Current time: 2022-09-12 15:25:13 (running for 00:05:00.67)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.4436590198516845 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.3585  |     0.5042 |                    6 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


== Status ==
Current time: 2022-09-12 15:25:18 (running for 00:05:05.69)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.4436590198516845 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.3585  |     0.5042 |                    6 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1376) [7,  6000] loss: 0.431
== Status ==
Current time: 2022-09-12 15:25:23 (running for 00:05:10.70)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.4436590198516845 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.3585  |     0.5042 |                    6 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


== Status ==
Current time: 2022-09-12 15:25:28 (running for 00:05:15.72)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.4436590198516845 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.3585  |     0.5042 |                    6 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1376) [7,  8000] loss: 0.319
== Status ==
Current time: 2022-09-12 15:25:33 (running for 00:05:20.74)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.4436590198516845 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.3585  |     0.5042 |                    6 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


== Status ==
Current time: 2022-09-12 15:25:38 (running for 00:05:25.75)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.4436590198516845 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.3585  |     0.5042 |                    6 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1376) [7, 10000] loss: 0.254
== Status ==
Current time: 2022-09-12 15:25:43 (running for 00:05:30.77)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.4436590198516845 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.3585  |     0.5042 |                    6 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


Result for train_cifar_f53e3_00004:
  accuracy: 0.5394
  date: 2022-09-12_15-25-46
  done: false
  experiment_id: 64a5d19e7cba4006b59267ed3c905929
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 7
  loss: 1.2707357055932282
  node_ip: 172.17.0.2
  pid: 1376
  should_checkpoint: true
  time_since_restore: 326.4747221469879
  time_this_iter_s: 58.353474855422974
  time_total_s: 326.4747221469879
  timestamp: 1662963946
  timesteps_since_restore: 0
  training_iteration: 7
  trial_id: f53e3_00004
  warmup_time: 0.004603147506713867

== Status ==
Current time: 2022-09-12 15:25:51 (running for 00:05:38.97)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.4436590198516845 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.27074 |     0.5394 |                    7 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1376) [8,  2000] loss: 1.237
== Status ==
Current time: 2022-09-12 15:25:56 (running for 00:05:43.98)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.4436590198516845 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.27074 |     0.5394 |                    7 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


== Status ==
Current time: 2022-09-12 15:26:01 (running for 00:05:49.00)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.4436590198516845 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.27074 |     0.5394 |                    7 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1376) [8,  4000] loss: 0.623
== Status ==
Current time: 2022-09-12 15:26:06 (running for 00:05:54.01)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.4436590198516845 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.27074 |     0.5394 |                    7 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


== Status ==
Current time: 2022-09-12 15:26:11 (running for 00:05:59.03)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.4436590198516845 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.27074 |     0.5394 |                    7 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


== Status ==
Current time: 2022-09-12 15:26:16 (running for 00:06:04.05)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.4436590198516845 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.27074 |     0.5394 |                    7 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1376) [8,  6000] loss: 0.422
== Status ==
Current time: 2022-09-12 15:26:21 (running for 00:06:09.06)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.4436590198516845 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.27074 |     0.5394 |                    7 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1376) [8,  8000] loss: 0.320
== Status ==
Current time: 2022-09-12 15:26:26 (running for 00:06:14.08)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.4436590198516845 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.27074 |     0.5394 |                    7 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


== Status ==
Current time: 2022-09-12 15:26:31 (running for 00:06:19.09)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.4436590198516845 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.27074 |     0.5394 |                    7 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1376) [8, 10000] loss: 0.254
== Status ==
Current time: 2022-09-12 15:26:36 (running for 00:06:24.11)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.4436590198516845 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.27074 |     0.5394 |                    7 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


Result for train_cifar_f53e3_00004:
  accuracy: 0.5278
  date: 2022-09-12_15-26-40
  done: false
  experiment_id: 64a5d19e7cba4006b59267ed3c905929
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 8
  loss: 1.3013075383067132
  node_ip: 172.17.0.2
  pid: 1376
  should_checkpoint: true
  time_since_restore: 380.1486439704895
  time_this_iter_s: 53.67392182350159
  time_total_s: 380.1486439704895
  timestamp: 1662964000
  timesteps_since_restore: 0
  training_iteration: 8
  trial_id: f53e3_00004
  warmup_time: 0.004603147506713867

== Status ==
Current time: 2022-09-12 15:26:45 (running for 00:06:32.65)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.3724832790791988 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.30131 |     0.5278 |                    8 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


== Status ==
Current time: 2022-09-12 15:26:50 (running for 00:06:37.66)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.3724832790791988 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.30131 |     0.5278 |                    8 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1376) [9,  2000] loss: 1.229
== Status ==
Current time: 2022-09-12 15:26:55 (running for 00:06:42.67)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.3724832790791988 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.30131 |     0.5278 |                    8 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1376) [9,  4000] loss: 0.625
== Status ==
Current time: 2022-09-12 15:27:00 (running for 00:06:47.68)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.3724832790791988 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.30131 |     0.5278 |                    8 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


== Status ==
Current time: 2022-09-12 15:27:05 (running for 00:06:52.69)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.3724832790791988 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.30131 |     0.5278 |                    8 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1376) [9,  6000] loss: 0.419
== Status ==
Current time: 2022-09-12 15:27:10 (running for 00:06:57.70)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.3724832790791988 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.30131 |     0.5278 |                    8 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1376) [9,  8000] loss: 0.309
== Status ==
Current time: 2022-09-12 15:27:15 (running for 00:07:02.71)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.3724832790791988 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.30131 |     0.5278 |                    8 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1376) [9, 10000] loss: 0.252
== Status ==
Current time: 2022-09-12 15:27:20 (running for 00:07:07.72)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.3724832790791988 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.30131 |     0.5278 |                    8 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


Result for train_cifar_f53e3_00004:
  accuracy: 0.5409
  date: 2022-09-12_15-27-24
  done: false
  experiment_id: 64a5d19e7cba4006b59267ed3c905929
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 9
  loss: 1.2470624084591866
  node_ip: 172.17.0.2
  pid: 1376
  should_checkpoint: true
  time_since_restore: 424.46292757987976
  time_this_iter_s: 44.31428360939026
  time_total_s: 424.46292757987976
  timestamp: 1662964044
  timesteps_since_restore: 0
  training_iteration: 9
  trial_id: f53e3_00004
  warmup_time: 0.004603147506713867

== Status ==
Current time: 2022-09-12 15:27:29 (running for 00:07:16.97)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.3724832790791988 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.24706 |     0.5409 |                    9 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1376) [10,  2000] loss: 1.231
== Status ==
Current time: 2022-09-12 15:27:34 (running for 00:07:21.98)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.3724832790791988 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.24706 |     0.5409 |                    9 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


== Status ==
Current time: 2022-09-12 15:27:39 (running for 00:07:27.00)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.3724832790791988 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.24706 |     0.5409 |                    9 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1376) [10,  4000] loss: 0.613
== Status ==
Current time: 2022-09-12 15:27:44 (running for 00:07:32.01)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.3724832790791988 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.24706 |     0.5409 |                    9 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


== Status ==
Current time: 2022-09-12 15:27:49 (running for 00:07:37.02)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.3724832790791988 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.24706 |     0.5409 |                    9 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1376) [10,  6000] loss: 0.418
== Status ==
Current time: 2022-09-12 15:27:54 (running for 00:07:42.03)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.3724832790791988 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.24706 |     0.5409 |                    9 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


== Status ==
Current time: 2022-09-12 15:27:59 (running for 00:07:47.05)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.3724832790791988 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.24706 |     0.5409 |                    9 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1376) [10,  8000] loss: 0.318
== Status ==
Current time: 2022-09-12 15:28:04 (running for 00:07:52.07)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.3724832790791988 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.24706 |     0.5409 |                    9 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


== Status ==
Current time: 2022-09-12 15:28:09 (running for 00:07:57.09)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.3724832790791988 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.24706 |     0.5409 |                    9 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


(func pid=1376) [10, 10000] loss: 0.242
== Status ==
Current time: 2022-09-12 15:28:14 (running for 00:08:02.10)
Memory usage on this node: 4.9/62.7 GiB
Using AsyncHyperBand: num_stopped=9
Bracket: Iter 8.000: -1.3724832790791988 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 2.0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (1 RUNNING, 9 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00004 | RUNNING    | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.24706 |     0.5409 |                    9 |
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


Result for train_cifar_f53e3_00004:
  accuracy: 0.5413
  date: 2022-09-12_15-28-15
  done: true
  experiment_id: 64a5d19e7cba4006b59267ed3c905929
  hostname: 6b6e7c4b64b7
  iterations_since_restore: 10
  loss: 1.2698858614534139
  node_ip: 172.17.0.2
  pid: 1376
  should_checkpoint: true
  time_since_restore: 475.2083933353424
  time_this_iter_s: 50.74546575546265
  time_total_s: 475.2083933353424
  timestamp: 1662964095
  timesteps_since_restore: 0
  training_iteration: 10
  trial_id: f53e3_00004
  warmup_time: 0.004603147506713867

== Status ==
Current time: 2022-09-12 15:28:15 (running for 00:08:02.70)
Memory usage on this node: 4.5/62.7 GiB
Using AsyncHyperBand: num_stopped=10
Bracket: Iter 8.000: -1.3724832790791988 | Iter 4.000: -1.3801411396205425 | Iter 2.000: -1.5118700037240982 | Iter 1.000: -2.196925364476442
Resources requested: 0/32 CPUs, 0/2 GPUs, 0.0/44.76 GiB heap, 0.0/9.31 GiB objects (0.0/1.0 accelerator_type:P100)
Result logdir: /root/ray_results/train_cifar_2022-09-12_15-20-12
Number of trials: 10/10 (10 TERMINATED)
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+
| Trial name              | status     | loc             |   batch_size |   l1 |   l2 |          lr |    loss |   accuracy |   training_iteration |
|-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------|
| train_cifar_f53e3_00000 | TERMINATED | 172.17.0.2:1340 |           16 |  128 |   64 | 0.00027664  | 1.39027 |     0.5003 |                   10 |
| train_cifar_f53e3_00001 | TERMINATED | 172.17.0.2:1370 |           16 |   64 |    8 | 0.000158014 | 2.30981 |     0.1103 |                    1 |
| train_cifar_f53e3_00002 | TERMINATED | 172.17.0.2:1372 |            4 |    8 |   32 | 0.011652    | 2.17892 |     0.1762 |                    1 |
| train_cifar_f53e3_00003 | TERMINATED | 172.17.0.2:1373 |           16 |  256 |   64 | 0.065364    | 2.06749 |     0.2304 |                    2 |
| train_cifar_f53e3_00004 | TERMINATED | 172.17.0.2:1376 |            4 |    4 |  128 | 0.000644664 | 1.26989 |     0.5413 |                   10 |
| train_cifar_f53e3_00005 | TERMINATED | 172.17.0.2:1378 |            8 |   64 |    8 | 0.00661774  | 1.48894 |     0.5097 |                    8 |
| train_cifar_f53e3_00006 | TERMINATED | 172.17.0.2:1380 |            2 |   32 |   32 | 0.0275734   | 2.36965 |     0.0995 |                    1 |
| train_cifar_f53e3_00007 | TERMINATED | 172.17.0.2:1382 |            4 |  128 |    8 | 0.0687161   | 2.38313 |     0.1007 |                    1 |
| train_cifar_f53e3_00008 | TERMINATED | 172.17.0.2:1384 |            2 |   32 |  256 | 0.0188278   | 2.21493 |     0.1597 |                    1 |
| train_cifar_f53e3_00009 | TERMINATED | 172.17.0.2:1386 |           16 |  256 |   64 | 0.00179698  | 1.14486 |     0.6232 |                   10 |
+-------------------------+------------+-----------------+--------------+------+------+-------------+---------+------------+----------------------+


Best trial config: {'l1': 256, 'l2': 64, 'lr': 0.0017969799553603804, 'batch_size': 16}
Best trial final validation loss: 1.144857809972763
Best trial final validation accuracy: 0.6232
Files already downloaded and verified
Files already downloaded and verified
Best trial test set accuracy: 0.6204

코드를 실행하면 결과는 다음과 같습니다.

Number of trials: 10 (10 TERMINATED)
+-----+------+------+-------------+--------------+---------+------------+--------------------+
| ... |   l1 |   l2 |          lr |   batch_size |    loss |   accuracy | training_iteration |
|-----+------+------+-------------+--------------+---------+------------+--------------------|
| ... |   64 |    4 | 0.00011629  |            2 | 1.87273 |     0.244  |                  2 |
| ... |   32 |   64 | 0.000339763 |            8 | 1.23603 |     0.567  |                  8 |
| ... |    8 |   16 | 0.00276249  |           16 | 1.1815  |     0.5836 |                 10 |
| ... |    4 |   64 | 0.000648721 |            4 | 1.31131 |     0.5224 |                  8 |
| ... |   32 |   16 | 0.000340753 |            8 | 1.26454 |     0.5444 |                  8 |
| ... |    8 |    4 | 0.000699775 |            8 | 1.99594 |     0.1983 |                  2 |
| ... |  256 |    8 | 0.0839654   |           16 | 2.3119  |     0.0993 |                  1 |
| ... |   16 |  128 | 0.0758154   |           16 | 2.33575 |     0.1327 |                  1 |
| ... |   16 |    8 | 0.0763312   |           16 | 2.31129 |     0.1042 |                  4 |
| ... |  128 |   16 | 0.000124903 |            4 | 2.26917 |     0.1945 |                  1 |
+-----+------+------+-------------+--------------+---------+------------+--------------------+


Best trial config: {'l1': 8, 'l2': 16, 'lr': 0.00276249, 'batch_size': 16, 'data_dir': '...'}
Best trial final validation loss: 1.181501
Best trial final validation accuracy: 0.5836
Best trial test set accuracy: 0.5806

대부분의 실험은 자원 낭비를 막기 위해 일찍 중단되었습니다. 가장 좋은 결과를 얻은 실험은 58%의 정확도를 달성했으며, 이는 테스트 세트에서 확인할 수 있습니다.

이것이 전부입니다! 이제 파이토치 모델의 매개변수를 조정할 수 있습니다.

Total running time of the script: ( 8 minutes 22.054 seconds)

Gallery generated by Sphinx-Gallery


이 튜토리얼이 어떠셨나요?

© Copyright 2022, PyTorch & 파이토치 한국 사용자 모임(PyTorch Korea User Group).

Built with Sphinx using a theme provided by Read the Docs.

PyTorchKorea @ GitHub

파이토치 한국 사용자 모임을 GitHub에서 만나보세요.

GitHub로 이동

한국어 튜토리얼

한국어로 번역 중인 PyTorch 튜토리얼입니다.

튜토리얼로 이동

커뮤니티

다른 사용자들과 의견을 나누고, 도와주세요!

커뮤니티로 이동