I created different vector versions of the embedding- from one to five vectors, and a 75 vector version. After doing far too many x/y/z plots, I determined that the best version for the intended use was the Five Vector version that was trained until step 500 at a batch size of 4 using the "d-adaptation" learning algo, and used the cosine annealing learning rate scheduler at a cycle of 1000 steps with 50 warmup steps. AdamW was enabled and left at default settings. The extension I used to train the embedding is this extension for A1111 WebUI.
You can checkout all the other versions if near the download button you click the arrow, and then click "training data". Five vectors at 500 steps "won" because the other version were either too baked (too many steps) or the differences were negligible. I tried to train until 5000 steps, but sometimes google colab would error out around the 4000-5000 step mark, which lead to a few of embeddings not having a 4500 or 5000 step version. The .optim files are included if you are interesting in resuming training on one of the embeddings.
There were 5000 training images that were used, though since the "chosen" version was trained until step 500 at batch 4, that means only 2000 of the images contributed the training of that embedding. If you are interested in the training images, I have them uploaded as a separate "version" called "Actual Training Images".
I created different vector versions of the embedding- from one to five vectors, and a 75 vector version. After doing far too many x/y/z plots, I determined that the best version for the intended use was the Five Vector version that was trained until step 500 at a batch size of 4 using the "d-adaptation" learning algo, and used the cosine annealing learning rate scheduler at a cycle of 1000 steps with 50 warmup steps. AdamW was enabled and left at default settings. The extension I used to train the embedding is this extension for A1111 WebUI.
You can checkout all the other versions if near the download button you click the arrow, and then click "training data". Five vectors at 500 steps "won" because the other version were either too baked (too many steps) or the differences were negligible. I tried to train until 5000 steps, but sometimes google colab would error out around the 4000-5000 step mark, which lead to a few of embeddings not having a 4500 or 5000 step version. The .optim files are included if you are interesting in resuming training on one of the embeddings.
There were 5000 training images that were used, though since the "chosen" version was trained until step 500 at batch 4, that means only 2000 of the images contributed the training of that embedding. If you are interested in the training images, I have them uploaded as a separate "version" called "Actual Training Images".