Hitchhiker's Guide To The Latent Space: Community Notebook Document

Last Updated: Wed May 18

Added [Batch] Neural Neighbor Style Transfer

TEXT TO IMAGE

• Total Text To Image Beginners: Start Here

\/\/\/\/\/\/\/\/\/

• https://colab.research.google.com/drive/1wkF67ThUz37T2_oPIuSwuO4e_-0vjaLs?usp=sharing

/\/\/\/\/\/\/\/\/\

• Diffusion Animation

• By no means ready Zooming nshepperd's JAX CLIP Guided Diffusion v2.5

• https://colab.research.google.com/drive/1HaZsgAeGBri531QwJO2uaWl9pcaj2_Gc?usp=sharing#scrollTo=aK3TdkjKK5_c

• This notebook isn’t finished, even as far as beta notebooks go, please don’t complain if you have issues

• Disco Diffusion v4.1 [w/ Video Inits, Recovery & DDIM Sharpen]

• https://colab.research.google.com/drive/1sHfRn5Y0YKYKi1k-ifUSBFRNJ8_1sa39?usp=sharing

• Disco Diffusion v5.2 

• https://colab.research.google.com/github/alembics/disco-diffusion/blob/main/Disco_Diffusion.ipynb

• Disco Diffusion v5 Turbo [w/ 3D animation]

• https://colab.research.google.com/github/zippy731/disco-diffusion-turbo/blob/main/Disco_Diffusion_v5_Turbo_%5Bw_3D_animation%5D.ipynb#scrollTo=LHLiO56OfwgD

• Loud Disco Diffusion v5 [w/ Audio prompts + 3D animation]

• https://colab.research.google.com/github/apolinario/multimodalai/blob/main/Loud_Disco_Diffusion_v5_[w_3D_animation_and_audio_input].ipynb

• Ian Henderson’s [zooming] CLIP Guided Diffusion HQ (512x512 edit)

• https://colab.research.google.com/drive/1F2M1T2ZQtanFpjBUyId1VaxmqPb4eY5N

• Infinite Diffusion 2.5D

• https://colab.research.google.com/drive/1VJrfInU5RbciXXD_8jzY-FntFqiyj6au#scrollTo=ce8f43b8

• Diffusion Still images

• 2nd NeuralismAI JAX v2.6 Edit

• https://colab.research.google.com/drive/1Q0fz3yaqRwMuBw043ZYa1d5twv6NCaFG

• Edit of jax notebook with these new features

• - Save Settings to txt file

• - Batching settings (import csv with params, prompts from text file, image batching, prompt with keyword batching)

• - Prompt Queuing system

• - Video progress output

• - Intermediate saves

• - Diffusion model selection

• - huemin's simple symmetry and multiple prompts and weights

• - prof's symmetry models

• Nshepperd's JAX CLIP Guided Diffusion v2.7

• https://colab.research.google.com/drive/1Z5kK1WXTkYoMAVN6FqkQg0Fa4bE5BnxG?usp=sharing

• (Huemin's Implementation) Nshepperd's JAX CLIP Guided Diffusion v2.7

• https://colab.research.google.com/drive/1hcDA5skuzk2IoC0wkQam4i-5k7c7iR1T?usp=sharing

• nshepperd's JAX CLIP Guided Diffusion v2.6? WIP

• https://colab.research.google.com/drive/1fW_tPEX7iD3xZK3VBDQ_Y2WnfdSzpacM?usp=sharing#scrollTo=zxGgJmRzq3Cs

•  QoL tweaks nshepperd's JAX CLIP Guided Diffusion v2.6? WIP (With ViT-L/14 and Symmetry)

• https://colab.research.google.com/drive/12ecr18kOuzE5I6wLm5BwQXBznLDXut4z#scrollTo=DiQ-zsUVZAo4

• Disco Diffusion v4.1 [w/ Video Inits, Recovery & DDIM Sharpen]

• https://colab.research.google.com/drive/1sHfRn5Y0YKYKi1k-ifUSBFRNJ8_1sa39?usp=sharing

• QoL tweaks for nshepperd's JAX CLIP Guided Diffusion v2.5

• https://colab.research.google.com/drive/1yShW_3NR1Q2iJvXrq_vVESI0yM9hK3CZ

• nshepperd's JAX CLIP Guided Diffusion v2.4

• https://colab.research.google.com/drive/10YWuTxtBI7PS0xBJCLAUjhR5cB0UUXe-

• QoL tweaks for nshepperd's JAX CLIP Guided Diffusion v2.4

• https://colab.research.google.com/drive/13DxqmDiHHqvZZnIzNhQl-elDlpIYvVGn?usp=sharing

• nshepperd's JAX CLIP Guided Diffusion v2.3

• https://colab.research.google.com/drive/12Bod44YVIXYRh39WRqp0kNz8OUBNFk9Z?usp=sharing#scrollTo=OoIL7ayzq7kC

• Prof's tweaks for nshepperd's JAX CLIP Guided Diffusion v2.3

• https://colab.research.google.com/drive/11iJo0h_ELYkuV-aAHU7RBIXJGaKvzBDU

• Not Working Anymore Quick CLIP Guided Diffusion HQ 256x256 and 512x512

• https://colab.research.google.com/drive/1FuOobQOmDJuG7rGsMWfQa883A9r4HxEO?usp=sharing

• K80 Safe (Legacy) Quick CLIP Guided Diffusion HQ 256x256 and 512x512

• https://colab.research.google.com/drive/1jUjUc9CQdmtXdVNtTOEngZd9EaabkaV0?usp=sharing

• Note that only the 256x model will work with a k80

• Text to Image (CC12M Diffusion with Classifier Free Guidance)

• https://colab.research.google.com/drive/1TBo4saFn1BCSfgXsmREFrUl3zSQFg6CC

• Velocity Diffusion aka v-diffusion (Yfcc_1 and Yfcc_2 Diffusion Models)

• https://colab.research.google.com/drive/1M4QKDt5ExIo3GLml0YKcJM9gfaX9ptfg

• CLIP Guided Diffusion HQ 512x512 Uncond_Fast Dango233

• https://colab.research.google.com/drive/14xBm1aSxQLbq26-jmDJi8I1HJ4ti5ybt

• Dall-E

• RuDalle Video

• https://colab.research.google.com/github/sberbank-ai/ru-dalle/blob/master/jupyters/videodalle.ipynb#scrollTo=-ozOC5bw26ge

• I haven't tested this

• Zack's QoL Tweaks For minDALL-E

• https://colab.research.google.com/drive/1x0lSUwmUUw4apmurcyFuBlxvp2FGdVaO?usp=sharing

• K80 Safe ruDalle

• https://colab.research.google.com/drive/1wGE-046et27oHvNlBNPH07qrEQNE04PQ?usp=sharing

• Philipuss's ruDALLE Notebook

• https://colab.research.google.com/drive/1-oTNPYrJxQGJItA_uaSrc59H9lqT_1EV

• ruDALLE arbitrary resolution

• https://colab.research.google.com/drive/1DbqOIUIVBPOrJ4MeaV4YkAlb7ilWQjKZ

• ruDALLE arbitrary resolution v2.0

• https://colab.research.google.com/drive/1JznXpirarS-zAZqXGOWRWDgFxLdL_xAU

• ruDALLE-Outpainting

• https://colab.research.google.com/gist/eyaler/0cee9a71f5dd3fdfa9c0c03656ebdd4c/rudalle-outpainting.ipynb

• ruDALLE tuning

• https://colab.research.google.com/drive/11bGsgfXn10SBG_zzpZ88Px3o8i1r9gYA

• Looking Glass 1.3

• https://colab.research.google.com/drive/15vFLeepkSTr1qd4xs31g9kMEiwkWP0sh

• Creator of Looking Glass requests that you do not make NFTs with it.

• Looking Glass v1.1 Google Drive

• https://colab.research.google.com/drive/1utvr4f743zchN72FZfrAX0RKqONwMWK6?usp=sharing

• Glide

• GLID-3 [GLIDE + Latent Diffusion + CLIP]

• https://colab.research.google.com/drive/1x4p2PokZ3XznBn35Q5BBD6K6Zs-tot5t?usp=sharing

• LAIONIDE v3

• https://colab.research.google.com/gist/afiaka87/8655b15c94bf0e80f586ce54cfe39ab5/laionide-v3.ipynb

• https://replicate.com/afiaka87/laionide-v3

• GLIDE Text2Im

• https://colab.research.google.com/drive/1pHh7xqSbPK4Kis36HlnD1mdxNQ-55Dzg

• GLIDE Clip-Conditioned Text2Im

• https://colab.research.google.com/drive/1DAlt3Eowa2vlps75P3HLQOBfzyjXwE_y

• PyGlide (Non Colab Version)

• https://replicate.com/afiaka87/pyglide

• VQGAN+CLIP

• Hypertron v2

• https://colab.research.google.com/drive/1N4UNSbtNMd31N_gAT9rAm8ZzPh62Y5ud?usp=sharing#scrollTo=CppIQlPhhwhs

• Masked Prompts VQGANCLIP_zquantize

• https://colab.research.google.com/drive/1B9hPy1-6qhnRL3JNusFmfyWoYvjiJ1jq?usp=sharing

• Mse regularized Modified_VQGANCLIP_zquantize

• https://colab.research.google.com/drive/1gFn9u3oPOgsNzJWEFmdK-N9h_y65b8fj

• Experimental VQGAN+CLIP Branch with custom flavors and pixel art

• https://colab.research.google.com/drive/1jx3klUxlGbYUwvtqzC9SYl4XZKHL3R81

• K80 Safe Turbo 3D-Zooming VQGAN+CLIP animations (with WAV2CLIP audio prompts)
  https://colab.research.google.com/github/pollinations/hive/blob/main/notebooks/2%20Text-To-Video/1%20CLIP-Guided%20VQGAN%203D%20Turbo%20Zoom.ipynb

•  K80 Safe Zooming VQGAN+CLIP animations (With Visual Editor)

• https://colab.research.google.com/github/chigozienri/VQGAN-CLIP-animations/blob/main/VQGAN-CLIP-animations.ipynb#scrollTo=o8u5ohZEe_i8

• Also try using this tool in tandem. The author says it supersedes “Music-driven VQGAN Animations”

• https://audio-keyframe-generator.glitch.me/

• Multi Perceptor VQGAN + CLIP [Public]

• https://colab.research.google.com/drive/1peZ98vBihDD9A1v7JdH5VvHDUuW5tcRK good for vqgan style transfer apparently

• K80 Safe Pollinations' VQGAN+CLIP with video and text prompts

• https://colab.research.google.com/github/pollinations/hive/blob/main/notebooks/2%20Text-To-Video/7%20CLIP-Guided%20VQGAN%20-%20Video.ipynb

•  K80 Safe Superseded Music-driven VQGAN Animations

• https://colab.research.google.com/github/chigozienri/VQGAN-CLIP-animations/blob/main/Music-driven-VQGAN-animations.ipynb

Other

• MindsEye - a multi-notebook UI for running multiple notebooks on top of Google Colab (Diffusion and VQGAN+CLIP)

• https://colab.research.google.com/drive/1cg0LZ5OfN9LAIB37Xq49as0fSJxcKtC5

• Guide: https://multimodal.art/mindseye

• Create Realistic AI-Generated Images With VQGAN + CLIP

• Pollinations - Many different ai art tools in an easy-to-use web interface

• https://pollinations.ai

• Dream Shadowing (CLIP + DIP) v2

• https://colab.research.google.com/drive/1eg2xDQKv1-4IWe9cAPrlrB2pq0QdIq_I

• K80 SafeQuadVisions

• https://colab.research.google.com/drive/1qgMT4-_kDIgZnNGMmrxmwzT3N6Ittw6B?usp=sharing#scrollTo=MEGzL9rYK_-I

• K80 SafeQuantumVisions

• https://colab.research.google.com/drive/1UoY_XWvVtz4Vt6MDyNsDPBMt4hxnOkGJ#scrollTo=MEGzL9rYK_-I&uniqifier=22

• K80 SafeCLIP Pointillist

• https://colab.research.google.com/drive/1XTHi7pv3nT5eSj4qNRAcW8jDOMsLofNJ?usp=sharing

• Audio&Text-to-Video - StyleGAN-3 Dance

• https://colab.research.google.com/github/pollinations/hive/blob/main/notebooks/3%20Audio-To-Video/2%20StyleGAN-3%20Dance.ipynb

• Neural Cellular Automata Pytorch Zack Edit

• https://colab.research.google.com/drive/1P7GFZl5JOxyh3JWFgCkeT0jq56gtzsxY?usp=sharing

• Structured Dreaming - Styledreams (StyleGan2 Fine Tuning)

• https://colab.research.google.com/github/ekgren/StructuredDreaming/blob/main/colabs/Structured_Dreaming_Styledreams.ipynb

• Pixray PixelDraw

• https://colab.research.google.com/github/dribnet/clipit/blob/master/demos/PixelDrawer.ipynb

• CLIPDraw Ai Telephone (v2)

• https://colab.research.google.com/drive/1mwcwJk5Nk1ogwQzfkxRIX5pLo-GbS4N9?usp=sharing

• Aphantasia

• https://colab.research.google.com/github/eps696/aphantasia/blob/master/Aphantasia.ipynb

• text2voxels beta

• https://colab.research.google.com/drive/1y0cR5goZ2go6SlYqVIZy7e7g0cOE3prE

• Latent Composition Interactive Composition

• https://colab.research.google.com/drive/1j7Bz9vdVnxzOgokawA39hCJZLTmVDq6_

• Deep Daze

• https://colab.research.google.com/drive/1_YOHdORb0Fg1Q7vWZ_KlrtFe9Ur3pmVj

• The Big Sleep Customized NMKD Public

• https://colab.research.google.com/drive/1Q2DIeMqYm_Sc5mlurnnurMMVqlgXpZNO?usp=sharing

• Looped Gif Creator (Looped Gif Text2im)

• https://colab.research.google.com/github/PHoepner/big-sleep/blob/main/Looped_Gif_Creator.ipynb#scrollTo=TxZFUZJB6Esl

• FuseDream

• https://colab.research.google.com/drive/17qkzkoQQtzDRFaSCJQzIaNj88xjO9Rm9?usp=sharing

• LipGan - Generate talking animations from image and audio

• https://colab.research.google.com/github/tugstugi/dl-colab-notebooks/blob/master/notebooks/LipGAN.ipynb

• Dream Fields - Text-Guided 3D Object generation

• https://colab.research.google.com/github/pollinations/hive/blob/main/notebooks/2%20Text-To-Video/2%20Dream%20Fields.ipynb

GANS

• GFPGAN_inference - Face Restoration

• https://colab.research.google.com/drive/1sVsoBd9AjckIXThgtZhGrHRfFI6UUYOo

• JoJoGan

• https://huggingface.co/spaces/akhaliq/JoJoGAN

• AnimeGANv2

• https://huggingface.co/spaces/akhaliq/AnimeGANv2

• ArcaneGAN

• https://huggingface.co/spaces/akhaliq/ArcaneGAN

• ChromaGAN: Adversarial Picture Colorization

• https://colab.research.google.com/github/pvitoria/ChromaGAN/blob/master/DemoChromaGAN.ipynb

• StyleGAN - Text-Guided Editing of Images

• https://colab.research.google.com/github/orpatashnik/StyleCLIP/blob/main/notebooks/optimization_playground.ipynb

• StyleGAN-NADA: CLIP-Guided Domain Adaptation of Image Generators

• https://colab.research.google.com/github/rinongal/stylegan-nada/blob/main/stylegan_na

• Lucid Sonic Dreams XL (Audio reactive - Lucid Sonic Dreams with Stylegan-XL)

• https://colab.research.google.com/github/pollinations/hive/blob/main/notebooks/3%20Audio-To-Video/2%20Lucid%20Sonic%20Dreams%20XL.ipynbda.ipynb

STYLE TRANSFER

• Zack’s Pick Fast Neural Style Transfer

• https://huggingface.co/spaces/aravinds1811/neural-style-transfer

• NEW! [Batch] Neural Neighbor Style Transfer

• https://colab.research.google.com/github/MSFTserver/AI-Colab-Notebooks/blob/main/NNST_Batch.ipynb

• Image-to-painting

• https://colab.research.google.com/drive/1XwZ4VI12CX2v9561-WD5EJwoSTJPFBbr?usp=sharing

• Style-transfer-pytorch

• https://colab.research.google.com/drive/1Tmuwmncao5E3D-5tTIVQjRy2YQ8JdpoB?usp=sharing#scrollTo=EhKX2A6TA-wL

• Style-transfer-2d (Lucid)

• https://colab.research.google.com/github/tensorflow/lucid/blob/master/notebooks/differentiable-parameterizations/style_transfer_2d.ipynb

• Batch image VQGAN+CLIP- public.ipynb

• https://colab.research.google.com/drive/1sfZIuv8d40dXlPaNBpXgHMwLZ3VkLgnG?usp=sharing

UTILITIES

• Pytti Utils

• Pyttipanna. A UI for Pytti.

• https://pyttipanna.xyz/

• Quaternions - visualize Pytti’s rotate_3D parameter

• https://quaternions.online/

• https://eater.net/quaternions

• Pytti Handbook

• https://docs.google.com/document/d/1EvkiHa12ButetruSBr82MJeomHfVRkvczB9-FgqtJ48/edit#heading=h.h22zbxjv567m

• Inpainting and BG Removal

• Background Removal: Robust High-Resolution Video Matting with Temporal Guidance.

• https://colab.research.google.com/drive/10z-pNKRnVNsp0Lq9tH1J_XPZ7CBC_uHm?usp=sharing

•  Flow-edge Guided Video Completion

• https://colab.research.google.com/drive/1pb6FjWdwq_q445rG2NP0dubw7LKNUkqc?usp=sharing

• BackgroundMattingV2 - Remove Video BG

• https://colab.research.google.com/drive/1Y9zWfULc8-DDTSsCH-pX6Utw8skiJG5s?usp=sharing

• BackgroundMattingV2 - Remove Image BG

• https://colab.research.google.com/drive/1cTxFq1YuoJ5QPqaTcnskwlHDolnjBkB9?usp=sharing

• LaMa-inpainting

• https://colab.research.google.com/github/saic-mdal/lama/blob/master/colab/LaMa_inpainting.ipynb

• Mae inpainting 

• https://colab.research.google.com/github/facebookresearch/mae/blob/main/demo/mae_visualize.ipynb#scrollTo=12251916-7bf0-4ee5-ba7d-3dea4441d0a4

• Upscaling 

• Popular Zack’s Pick SuperRes Diffusion v1.1

• https://colab.research.google.com/drive/19euI_7GAgbvMoZsuPj9SZseDeuFnwBj8#scrollTo=_hrsrSgFpB1V

• ESRGAN

• https://colab.research.google.com/github/styler00dollar/Colab-ESRGAN/blob/master/Colab-ESRGAN.ipynb

• SwinIR

• https://colab.research.google.com/gist/JingyunLiang/a5e3e54bc9ef8d7bf594f6fee8208533/swinir-demo-on-real-world-image-sr.ipynb

• Waifu2x

• http://waifu2x.udp.jp/

• Interpolation and Slowmo

• AnimationKit_Rife-RealESRGAN_Upscaling-Interpolation

• https://colab.research.google.com/github/sadnow/AnimationKit-AI/blob/main/AnimationKit_Rife_RealESRGAN_Upscaling_Interpolation.ipynb

• Colab-Super-SloMo

• https://colab.research.google.com/github/MSFTserver/AI-Colab-Notebooks/blob/main/Super_SloMo.ipynb

• Adding more frames via interpolation then slowing down videos is commonly used to make shorter animations longer

• RIFE_Colab

• https://colab.research.google.com/github/HeylonNHP/RIFE-Colab/blob/main/RIFE_Colab.ipynb#scrollTo=cC0Ehp_BweZi

• Flowframes

• https://nmkd.itch.io/flowframes

• FILM_Colab

• https://colab.research.google.com/drive/1tbbbnQge0yb0LmnWNchEKNhjtBNC6jX-

• Image Caption Generation

• Personality-clip

• https://colab.research.google.com/drive/171GirNbCVc-ScyBynI3Uy2fgYcmW3BB9

• Antarctic-Captions

• https://colab.research.google.com/drive/1FwGEVKXvmpeMvAYqGr4z7Nt3llaZz-F8https://colab.research.google.com/drive/1FwGEVKXvmpeMvAYqGr4z7Nt3llaZz-F8

• Remove Signatures, Artifacts etc.

• Cleanup Pictures

• https://cleanup.pictures/

• Deep Watermark Removal

• https://colab.research.google.com/github/styler00dollar/Colab-deep-watermark/blob/main/Colab-deep-watermark.ipynb

• Freely Usable Media Files

• Wikimedia Commons

• https://commons.wikimedia.org/wiki/Main_Page

• Pexels

• https://www.pexels.com/

• RawPixel

• https://www.rawpixel.com/public-domain

• Unsplash

• https://unsplash.com/

• Gif Maker

• EzGif

• https://ezgif.com/maker

• Misc

•  Dughogan VQGAN Prompt Generator

• https://colab.research.google.com/drive/14P3TXG_Z6dXQEn6wADB7VBzFF35na1-Y#scrollTo=wBqowTj41W2g

• Barebones

• https://colab.research.google.com/drive/1Ml8hy_jXDeUR8zjNhEokbZg79QbaD-cz?usp=sharing#scrollTo=GBRlz7DRueqx

• Multiple Prompts

• MadLib VQGAN/Diffusion Prompt Generator

• https://colab.research.google.com/drive/1RM6SySSTlPtWy5B3OUOHB9CyCUNKHlhm

• Artist Studies - resources for gathering artist names to use in prompts

• https://remidurant.com/artists/#

• Visualize how CLIP "sees"

• https://twitter.com/remi_durant/status/1460607677801897990?s=20

• ~200 CLIP+VQGAN keywords on 4 subjects

• https://imgur.com/a/SALxbQm

• VQGAN+CLIP 125 Keyword Modifier Comparison

• https://creator.nightcafe.studio/vqgan-clip-keyword-modifier-comparison#results

• CLIP FRONT - Search engine to see typical images CLIP matches to a given text prompt

• https://rom1504.github.io/clip-retrieval/?back=https%3A%2F%2Fsplunk.vra.ro&index=laion_400m_128G&useMclip=false

• In Browser Image Compressor

• https://donaldcwl.github.io/browser-image-compression/example/basic.html

MUSIC GENERATION

• Voice Synthesis

• Mellotron Singing multispeaker voice synthesis

• https://colab.research.google.com/github/yhgon/mellotron/blob/master/inference_colab.ipynb

• Jukebox

• One Click Jukebox with Autosave

• https://colab.research.google.com/drive/1gWP6fSqd_vs65tI5uOGcJssGqTq2YWzO?usp=sharing

• Jukebox Songeater - multiple artist support

• https://colab.research.google.com/github/songeater/jukebox/blob/master/jb_songeater_github_v5.ipynb

•  K80 Safe Jukebox 1b_Finetuned

• https://colab.research.google.com/drive/1IZl9V2PAvaGBvPShNhAITi7OclufJhxD?usp=sharing

• a notebook for free or pro users to access finetuned models provided by the community (keep in mind that if too many people want to access the same model at the same time, Google will "block" it for a while 

• K80 Safe Jukebox SMarioMan notebook + speed upsampling - optimized for 1b_lyrics model

• https://colab.research.google.com/drive/1FZpXEK_LWyoIYoef408SXbe51S9GrJwY?usp=sharing