Join Our Discord (630+ Members)

Arcane Diffusion

Arcane Diffusion is a fine-tuned AI model created by Nitrosocke that is trained on images from the TV show Arcane. It can be used to generate creative and unique text based on prompts given to it, with the tokens "arcane style" for an extra effect. This model can be used for text generation, image captioning, and other creative tasks.

Follow AI Models on Google News

An easy & free way to support AI Models is to follow our google news feed! More followers will help us reach a wider audience!

Google News: AI Models

Subscribe or Contribute

Arcane Diffusion

This is the fine-tuned Stable Diffusion model trained on images from the TV Show Arcane. Use the tokens arcane style in your prompts for the effect.

If you enjoy my work, please consider supporting me Become A Patreon

🧨 Diffusers

This model can be used just like any other Stable Diffusion model. For more information, please have a look at the Stable Diffusion .

You can also export the model to ONNX , MPS and/or FLAX/JAX .

#!pip install diffusers transformers scipy torch
from diffusers import StableDiffusionPipeline
import torch

model_id = "nitrosocke/Arcane-Diffusion"
pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16)
pipe = pipe.to("cuda")

prompt = "arcane style, a magical princess with golden hair"
image = pipe(prompt).images[0]

image.save("./magical_princess.png")

Gradio & Colab

We also support a Gradio Web UI and Colab with Diffusers to run fine-tuned Stable Diffusion models: Open In Spaces Open In Colab

img

Sample images from v3:

output Samples v3 output Samples v3

Sample images from the model:

output Samples

Sample images used for training:

Training Samples

Version 3 (arcane-diffusion-v3): This version uses the new train-text-encoder setting and improves the quality and edibility of the model immensely. Trained on 95 images from the show in 8000 steps.

Version 2 (arcane-diffusion-v2): This uses the diffusers based dreambooth training and prior-preservation loss is way more effective. The diffusers where then converted with a script to a ckpt file in order to work with automatics repo. Training was done with 5k steps for a direct comparison to v1 and results show that it needs more steps for a more prominent result. Version 3 will be tested with 11k steps.

Version 1 (arcane-diffusion-5k): This model was trained using Unfrozen Model Textual Inversion utilizing the Training with prior-preservation loss methods. There is still a slight shift towards the style, while not using the arcane token.

Related Posts

John Mayer AI Voice

John Mayer AI Voice

Discover the unique sound of AI John Mayer’s collection of songs!

Beyonce AI Voice

Beyonce AI Voice

Experience the future of music with AI Beyonce’s collection of songs!

Polo G AI Voice

Polo G AI Voice

Discover AI Polo G’s innovative collection of songs produced using VITS Retrieval voice conversion methods.