civitai-sync is a computer-based tool to download your Civitai onsite generations, and to keep downloading in future when you create more.
It downloads each image in your generation feed, and saves the generation data so that prompts can be searched as text. Everything is saved in date-ordered folders. Images are named to be in chronological order.
Check the latest discussion on the article about the tool: https://civitai.com/articles/5676
You can also comment below.
Click "Show More" for installation instructions.
For info on the retention of generations: https://civitai.com/articles/5604
civitai-sync is a Command Line Interface (CLI). It runs in your computer terminal, called “Command prompt” in Windows. Don’t worry, you’ll be fine. Follow the steps below.
Download civitai-sync
Download the tool from this tool page. It's a zip file.
On your computer, Unzip (or “Extract”) it into a folder, wherever you want to keep it
Install Node.js, download it from: https://nodejs.org (if you already have Node, it needs version 18 or above)
If you have a nice new shiny clickable Node.js icon on your desktop or menu, don’t click it. You don’t need to. That’s for something else. Do this:
Install it
If you are updating your civitai-sync, skip to the section "Migrate from an earlier version".
If you’re on Windows, in Explorer, double-click on the file called "install_win". This will open a command prompt, install the software dependencies and then close. Alternatively, install manually with the steps below.
On Linux, Mac, open your Terminal app.
On Windows, your “Command Prompt”.Inside the terminal, change directory to the program folder, wherever you saved it
(the path and name must match your program folder):
cd Downloads/civitai-sync
On Windows, use backslashes, not forward-slashes:
cd C:\Downloads\civitai-syncRun this line to install the software dependencies (npm is the Node Package Manager):
npm install
Run it
If you’re on Windows, you can double-click on the file "Civitai-Sync" to start the program.
To add it to your Desktop, right-click on "Civitai-Sync" > "Send to" > "Desktop (create shortcut)".
If the clickable icon doesn’t work, then manually install as above, and follow the steps below.
On all platforms, in the terminal, if you didn't already,
"cd"change directory to the program folder, as above.Type “
npm run cli”
Migrate from an earlier version
If you have already downloaded your generations with an earlier version of the tool:
Download the latest program file and unzip it. Now you have two versions downloaded - the old version and the new one.
All that is different in the new version is: the contents of the "src" folder, the README, and package.json.
Either: Copy those new files over to your existing program folder, and you don't need to do the "Install" step again.
Or: If you want, and you have a "config" and "generations" folder within your existing version, you can move them into the new version folder and then "Install".
The API data for generations has changed and will need to be downloaded again.
To keep the new API data separate from the old data, rename your existing data folder from, say, "generations/data" to "generations/data.old" before you download with the new version. The data folder will be recreated automatically when you download.
Already downloaded images will be renamed to be in chronological order.
Create API key
On the Civitai website, go to "API keys" in your Account Settings, near the bottom of the page: https://civitai.com/user/account
"Add API key" > Name it, e.g. 'civitai-sync' > Save
You will be shown the key only once > Copy it
In civitai-sync, choose "Set API key", and paste the key.
(To paste, right-click > "Paste", or press "Ctrl+V" or "Ctrl+Shift+V")
You can set a password to encrypt your key, e.g. if you are on a shared computer. You'd need the password each time you run the program (you can remove or change it later). Or press "Enter" for no password.
Download generations
When your key is saved, you can choose "Download generations".
If you have multiple thousands to download, it will take a while, so be prepared to leave it running. It is rate-limited to 1 generation per second, to not burden the servers.
Downloads are saved inside the program folder, in "generations".
There are "data" and "media" folders, both containing folders for each generation date, inside of which are the data and media files.
You can still generate on the website while you download. It is best to start the download when you don't have pending generations in the queue.
Once you've downloaded everything in your generation feed, whenever you generate more, you can run "Download generations" again, and only your latest generations will be downloaded.
Download location
You can set the download location to be anywhere on the file system, e.g. an external hard drive.
It is recommended to change it to a folder outside of the program - because if you later update or re-install the program, you may accidentally copy over the whole folder, including your downloads.
Select "Download options" > "Data download location"
And "Download options" > "Media download location"
Generation data
The data files are from Civitai's API and are text files in JSON format. You can search for prompts from your file explorer.
The images are JPEGs and are already embedded with the generation data, in the EXIF data 'UserComment' tag.
View the prompt and parameters stored within image files:
Chrome browser extension:
https://chromewebstore.google.com/detail/exif-metadata-viewer/Stable Diffusion Prompt Reader:
https://github.com/receyuki/stable-diffusion-prompt-readerWindows: In Explorer, click on image > properties > select tab with details > select "comments"
Images are named "{GENERATION_ID}_{SEED}.jpeg".
Download only data, not images
If you only want the data files that contain the prompts and parameters:
Select "Download options" > "Change download type to: only data".
Select the option again to change it back to "data and media".
Download problems?
If your first download was stopped early or interrupted – to resume downloading your oldest generations, use “Download options" > "Download oldest".
If you need to fill any gaps in your download record, e.g. because of deleted files, or you download only data and then want images, use "Download options" > "Download missing".
Multiple accounts
To download more than one account, run the program with a unique path for each account:
npm run cli config/bob
This will create a configuration file inside the program folder within the "config" folder, called "bob.json".
When you run "npm run cli" without specifying an account, a config file is saved at "config/default.json".
You can give a full path to anywhere on the file system, and the config file will be loaded from there.
What is this? Will it farm my buzz?
Good question, you've got your wits about you. Take a look at the code, and see what other people are saying.
Problems? Thoughts? Get in touch
Make a comment, start a chat.
@monkeypuzzle