Introducing: Ktiseos Nyx Trainer!

Introduction

You thought this was going to be some long shpeel with AI jargon huh?

No, this is a reality check and a "DEV LOG" update!

The Details

This is yes a "FULL STACK" of sorts SD-scripts based Next JS front end, python back end (FastAPI) "WEBUI" that will eventually do more than just your average lora.

We're running on React 19, Next JS 15 (whatever one isn't currently getting attack vectors left right and centre). We're heavily inspired by Bmaltais's Gradio edition of SD-Scripts, and while this does feel like a Wheel reinvention, for years I've been with people who have said "That's really difficult to navigate."

That's due to how navigation works with Gradio works, and how exposing everything at once in the early days of SDXL was really difficult for myself and others to navigate. That's not to say that system is BAD at all - it's a "BATTLE TESTED CHICKEN SHIT READY" (as the LLM's would call it) system.

The History

I came from colab, using Holo's and Linaqruf's colabs is where I started, and Civitai's lora training was my next stage after that. In fact that was sort of what got me into development, was tinkering with colab notebooks.

Patching things, yanking things from here and there.
Yet there was a scary part somewhere in 2024: Realizing I can hook up to LLM's and start learning way more faster. (And stupid).

Coming from just doing the occasional Jupyter and Colab notebooks, I started noticing niche pockets of things that didn't work for me personally. That was never my intention to steam roll, or replace current technology. Being that usually people assume that's my intention that don't understand development - I have felt like I've been reinventing the wheels in a bad way.

The Fork

I started majorly developing Dataset tools, and then realized there was a dream still: to put in motion that I needed to use SD-scripts in a way that suited me. (It still didn't after i was done so hold fire). I held off for a good six months with several notebooks sitting in a folder waiting for me to figure out how to do more advanced coding (er uh more stupid in the terminal, get it it right dusk).

By the time I had a chance to have a break in the development cycle of Dataset tools, I broke through a few barriers. (Translation: I did the finger walking, I wrote down what i needed and I told gemini and claude to build it.) We forked Derrian Distro's backend, and Jelosus and Android's colab editions of it. The builds in the early days were meant to be simple enoguh that most people can use it.

It exploded. IT made me cry, the widgets were beautiful but half the time it broke. To be honest? The Jupyter edition went archived because of a race condition that may not have even been able to be solved. We re opened it because there's a UTF encoding issue with the emojis, that never was an issue before.

The race condition? It was either AI introduced or it was literally something i read up on: Ipywidgets and Ipython have a problematic relationship with remote uploads. It's not as easy as just settings a CORS middleware, or fast API in terms of what I was going for. It could've been better with FastAPI, but in terms of my understanding of the notebook - it would not have helped.

The Ideation

Since around October, i'd struggled to want to train loras on my system because things were breaking left and and right. I was given some preview credits for Claude Code in the browser (as a Claude Pro subscriber at the time) - and without really understanding how to use the non terminal equivalent I decided to just ask questions. I asked what services were, and how code worked with it. Knowing that I was looking into Electron for Dataset tools, and I was peeking into NextJS for another app I was playing with - I had a clue where I needed to go.

We'd already moved the notebooks to a new repository for better tracking, and meaning that i didn't keep accidentally pull-request spamming poor Jelosus2's repository. (Which will still happen on the fork copy, because i'm not updating notebooks on the main repo anymore.)

After this happened, went through got Ideas, looked at how I'd already implemented things with plural chat and just went to work. Went a LITTLE NUTS on the pretty and feature creep at first, because who doesn't wanna make their own open source contributions to the NextJS development world? (And put development off for like ever because pretties.)

Once this sort of "WAIT WE NEED TO MAKE THIS WORK" happened, it was all hands on deck. It was one set of hands on deck sadly. Still is.

It's the familiarity of dark/light mode, with the ease of use of a jupyter notebook but styled in such a way that's not top down and hard to read. It was accessibility in mind. It still is.

The How and When

The how will you ever get to use it and when, or is this the how DO I use it and when will it be finished?

Those are hard questions, even the meme side of it. Right now it's in "ALPHA" but as in not entirely as useable as i'd like. Instead of me finishing code i've been exhausted enough just to write this article for you and introduce how far it's come in.

RIGHT NOW, the training configs are a little broken. Being that this is in Alpha stages, there's nothing that's 100% as solid as i'd like. The tagging setup is a little wonky, and there's some really amazing ideas I had that won't go away but things aren't as accessible as i'd like.

I don't have a demo handy, but I am curious if i could install a demo on vercel with just a dev outlook that doesn't work so people could SEE how it should work. I'll look at that before Alpha stages are closed off before we move into beta maybe.

TRAINING is broken a little i'm working on fixing it today, pydantic and Zustand and Zod which i understand THE LOGIC BEHIND but syntax good freaking luck... Are all kinda acting like maniac toddlers and not speaking properly to each other.

WHEN WILL IT BE USEABLE? - HOPEFULLY I can fix training today. THERE IS A manual workaround in CLI if you have access to either created or pre-existing TOML files. It's just that most of the time I would rather have a graphical way of seeing it.

The other issue with training is: the assumption from LLM's that everything is "ADVANCED" - there's no way to do WandB logging yet, and there's defintatley no real way to hook it through tensorboard because "PEOPLE DONT USE THAT ITS TOO HARD BASKET" according to Claude. (It assumes that i don't understand how to train a lora, because i'm programmer stupid)

The hilarious thing is: As MUCH AS I WHINE ABOUT CLAUDE? -- It's better at the building everything, it's just that when I'm all said and done and trying to save money I have to debug everything. And it's always the API calls, not the front end.

Except when I literally broke the UI because the build errors we had two days ago were my fault.

"HOW" Literally - Using KNX Trainer

Well, first of all if you're a LOCAL user and you're able to toy around with it and create more work for me here's the link: https://github.com/Ktiseos-Nyx/Ktiseos-Nyx-Trainer/

Right now there is a WORKING (MOSTLY) VastAI template that is a LITTLE JANKY but it's useable. The cloudflare buttons via VastAI's portal don't pick up the trainer right away (probably because I'm not doing something correct) - however ONCE the service is started and up and running? You just go to the part where it says "Advanced Connect" and if it gievs you an IP address don't panic, but if it gives you a URL go for gold. The ports it's on will still work, it just won't always pick it up on the button side.

Once it's all installed if you ever make your own changes, just npm run build if it's a front end issue. If you're on Vast, you'll need to restart via the supervisor commands, but local just CRTL-C and restart front and back ends.

I don't have a TUTORIAL on how this works YET but my plan is once i get everything fixed is to attempt to do a recording of me setting everything up and casually making it easier for people to use an alternative option out there.

How to HELP - The "PLEASE PLEASE"

"WON'T YOU PLEASE PLEASE HELP ME" - The Beatles.

What i need: Confident people who want to volunteer to look into how to service this long term. We've vendored code to stop submodule hell (Only because for me that was breaking things so much that I didn't understand how to fix it.) - so if you're a Lycoris nerd and want to peek into what i'm doing wrong - help please!

What i also need: Focused nerds who want to usher in more trainer code that can boost A LONG SIDE sd-scripts. I'm looking at adding FLOW properly outside of Kohya-SS based code, and there's more context that would just make this so much cooler.

PLANS

My Future PLANS: Screw API calls, i kinda wanna in the next year attempt to re-package Forge OR (Or "AND") ComfyUI through this system. Probably FORGE more likely than Comfy, because ComfyUI could LITERALLY just be an API route through an iframe of sorts. I don't like gradio in that it's difficult to navigate how it works in comparison to HTML and otherwise- it's a wonderful system but it's also difficult to deal with.

In that future plan for forge: I would like to be able to support not just forge and a111 plugins but make a plugin system FOR the trainer.

Things otherwise i want to add to the system: Dataset tools, this is incredibly a valuable tool i've made and it's addition would be interesting to make an absolute easy to use "MONOLITH" In a good way system for people who need a WebUI that's different.

AI may be a bubble that a lot of us still want to burst because cost wise it's -- harming consumers (at the corporate level)... But i feel so much for the open source ethos, that I still want to keep going. I want to make an ecosystem that encompasses that easy to use feeling that we all got when we started using A111 - but making it way easier in some ways.