santa hat
deerdeer nosedeer glow
Sign In

WHY ARE THERE PICKLE FILES?

Fist thing that I learnt about coming here was it was important to stay away from Pickles. People were saying no to pickles and only download SafeTensor.

So that is what i do but it seems I am missing out on certain VAE and embedding because they are pickles and so i wont download them. If the pickles are shown as verified then why cant they be converted to SafeTensor.

3 Answers

Verified status on Civitai for .pt files means that they have been tested in specific environment and most likely not contain malicious code.

If you so afraid of pickles, then you can convert them yourself. Here is the colab for converting VAEs and embeddings into safetensors: https://github.com/DiffusionDalmation/pt_to_safetensors_converter_notebook

I'm using this notebook for converting my embedding models prior publishing, didn't tryed VAEs convertion though.

Pickles are not inherently bad. They're a known security vulnerability and can potentially execute arbitrary code. That doesn't mean you should avoid them at all cost. As kinkau mentioned, they're already scanned for suspicious imports by the server so very likely benign. Plus most are published by well known creators.

There are far worse things to be found on the web. You'd be shocked what went around in the 90s.

As for the "why", people have been saying embeddings cannot be created in safetensors format; it's probably still not built in to training so you could count the number of people who would bother with extra steps on one hand.

You can just take vae from me.

And the answer above is correct - a lot of authors don't want to do more when they can do less.

Your answer