While it gives me a pickling warning with .ckpt format, once converted to safetensors with the convertor, it works well!