Thursday, December 29, 2022

Robots and discriminatory 👾 smut come to WikiMedia Commons

Summary[edit]

Description

An X/Y plot of algorithmically-generated AI artworks depicting a woman in various different settings, created using a custom-trained anime-focused Stable Diffusion-based model known as "Anything V3.0" (with hash 1a7df6b8) created by Furqanil Taqwa. This plot serves to demonstrate the usage of Hypernetworks, a technique created by Kurumuz in 2021 which allows Stable Diffusion-based image generation models to imitate the art style of specific artists, even if the artist is not recognised by the original diffusion model, by applying a small neural network at various points within the larger network.

Hypernetworks are small pre-trained neural networks that steer results towards a particular direction, for example applying visual styles and motifs, when used in conjunction with a larger neural network. The Hypernetwork processes the image by finding key areas of importance such as hair and eyes, and patches them in secondary latent space. They are significantly smaller in filesize compared to DreamBooth models, another method for fine-training AI diffusion models, making Hypernetworks a viable alternative to DreamBooth models in some, but not all, use-cases. Hypernetwork training also requires only 6GB of VRAM, compared to the ~20GB VRAM required for DreamBooth training (although this VRAM requirement can be lowered using DeepSpeed). A downside to Hypernetworks is that they are comparatively less flexible and accurate, and can sometimes lead to unpredictable results. For this reason, Hypernetworks are suited towards applying visual style or cleaning up blemishes in human anatomy, while DreamBooth models are more adept at depicting specific user-defined subjects.

Procedure/Methodology

These images were generated using an NVIDIA RTX 4090; since Ada Lovelace chipsets (using compute capability 8.9, which requires CUDA 11.8) are not fully supported by the pyTorch dependency libraries currently used by Stable Diffusion, I've used a custom build of xformers, along with pyTorch cu116 and cuDNN v8.6, as a temporary workaround. Front-end used for the entire generation process is Stable Diffusion web UI created by AUTOMATIC1111.

Hypernetworks trained on the artstyles of the following artists were used:

  • As109, a Chinese artist. Trained on 440 samples using 75,000 steps on 0.0000005 LR.
  • Asanagi (朝凪), a Japanese artist and the sole member of the Fatalpulse dōjin circle. Trained using 118,500 steps.
  • homunculus (ホムンクルス), a Japanese artist and mangaka. Trained using 90,000 steps on 5e-7 LR with no normalisation, and layer structure 1.0, 2.0, 1.0.
  • j.k., a Canadian artist.
  • Ohisashiburi (お久しぶり), a Japanese artist. Trained with 1e-5 LR up to 7,000 steps, and 5e-6 LR up to 180,000 steps, with layer structure 1.0, 1.5, 1.5, 1.0, mish activation function, normal weight initialisation, layer norm set to false, dropout usage set to true.
  • Takayaki (たかやKi), a Japanese artist and member of the Jenoa Cake (じぇのばけーき) dōjin circle. Trained on 90 samples using 100,000 steps on 0.0000005 LR.

A batch of 768x1024 images were generated with txt2img using the following prompts:

young woman, fully clothed, volumetric lighting, mountain forest background

Negative prompt: nude, lowres, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality, normal quality, jpeg artifacts, signature, watermark, username, blurry, artist name

Settings: Steps: 100, Sampler: DPM2, CFG scale: 7, Size: 768x1024, Highres. fix, Denoising strength: 0.7, Clip skip: 2

During the generation of this batch, an X/Y plot was generated using the "X/Y plot" txt2img script, along with the following settings:

  • X-axis: Hypernetwork: None, 151abd09, a00f10d3, e0a6b144, 97fa462a, a177a153, 9936f48b
  • Y-axis: Prompt S/R: mountain forest background, sitting in front of a computer. corporate office background, romantic date. french cafe background
This script repeats the same prompt and seed value for each hypernetwork, and also searches for the first value (in this case "mountain forest background") within the prompt, replacing the string with the subsequent comma-separated values.
Date
SourceOwn work
AuthorBenlisquare
Permission
(Reusing this file)
Output images

As the creator of the output images, I release this image under the licence displayed within the template below.

Stable Diffusion AI model

The Stable Diffusion AI model is released under the CreativeML OpenRAIL-M License, which "does not impose any restrictions on reuse, distribution, commercialization, adaptation" as long as the model is not being intentionally used to cause harm to individuals, for instance, to deliberately mislead or deceive, and the authors of the AI models claim no rights over any image outputs generated, as stipulated by the license.

Anything V3.0 model

Anything V3.0, created by Furqanil Taqwa, is released under the CreativeML OpenRAIL-M License.

Addendum on datasets used to teach AI neural networks
Artworks generated by Stable Diffusion are algorithmically created based on the AI diffusion model's neural network as a result of learning from various datasets; the algorithm does not use preexisting images from the dataset to create the new image. Ergo, generated artworks cannot be considered derivative works of components from within the original dataset, nor can any resemblance to any particular artist's drawing style fall foul of de minimis. While an artist can claim copyright over individual works, they cannot claim copyright over mere resemblance over an artistic drawing or painting style. In simpler terms, Vincent van Gogh can claim copyright to The Starry Night, however he cannot claim copyright to a picture of a T-34 tank painted with similar brushstroke styles as Gogh's The Starry Night created by someone else.

The uploader otherwise blocked also for such (algorithmic) "hate" and "harassment":

This account is currently blocked. (Show block details) The latest block log entry is provided below for reference:



Deletion discussion with WikiMedians' pro and con AI "art"arguments

Other WikiMedia AI generation deletion requests, including this one: 

Description

Algorithmically-generated AI artworks consisting of young women wearing modern-day clothing, in imitation thick brushstroke painted style, created using the Stable Diffusion V1-4 AI diffusion model.

Procedure/Methodology

All artworks created using a single NVIDIA RTX 3090. Front-end used for the entire generation process is Stable Diffusion web UI created by AUTOMATIC1111.

A batch of nine 512x512 images were generated with txt2img using the following prompts:

Prompt: busty young girl, art style of Michael Garmash

Negative prompt: (((deformed))), bad anatomy, disfigured, mutation, mutated, (extra_limb), (ugly), ((((mutated hands and fingers))))

Steps: 50, Sampler: LMS, CFG scale: 7.5

Then, two passes of the SD upscale script using "Real-ESRGAN 4x plus anime 6B" were run within img2img. The first pass used a tile overlap of 64, denoising strength of 0.3, 50 sampling steps with Euler a, and a CFG scale of 7. The second pass used a tile overlap of 128, denoising strength of 0.1, 10 sampling steps with Euler a, and a CFG scale of 7.


Ver. 1.0.1

No comments:

Post a Comment

Do consider considering first...

AIs analyze loaded questions and rhetorical devices

DECEMBER 5, 2023 | PART OF HARVARD, UPENN & MIT PRESIDENTS ON COMBATING ANTISEMITISM University Presidents Testify on College Campus Ant...