Foreword
I am writing this article to clarify a few things for myself. Some people break new ground, I always try to explore new territory. This means that there are always more questions than answers once a problem has been successfully solved.
Pickle Tensor and Safetensors
When I started building my first Embeddings and Hypernetworks for CIVITAI, I learned that .pt file potentially dangerous. In Pickle Tensor files there can be executable Python code per definition. So the logical step is a conversion of an insecure Pickle Tensor file to a secure Safetensors file.
When thinking about a conversion from one file format to the other the first thing I do is an Internet search on the topic. I found some interesting resources, but none of them were working for Embeddings as well as for Hypernetworks. I tried also to modify the approaches with less success.
So I started to learn how Pickle Tensor files and Safetensor files are internally organized. That was the key to writing the first converter for an Embedding. The result can be found in [1,2]. On model is a Pickle Tensor file, the other one is a Safetensors file.
Outlook
Based on my first converter from Pickle Tensor Embedding to Safetensors Embedding I will try to do the same approach to write a converter for Hypernetworks. After that I will try to that also for LoRa files as well. Last but not least I will check if my approach is also working for base models. I always assume that a .pt file is available as a zip archive.
Finally
Have a nice day. Have fun. Be inspired!
Resources
[1] https://civitai.com/models/756860/toadstoolembedding?modelVersionId=846305
[2] https://civitai.com/models/763411/toadstoolsafetensorsembedding?modelVersionId=853869