Update: The V1 80k is cooking
For the final train of V1 the full 80k dataset is required. With all the major caching, speed, and implementation problems resolved, the train can commence.
2 epochs VLM with 1 epoch Animetimm will be enough until we refine the json behavior. Even with those counts the training regiment is larger than the entirety of v0.5, considerably more images and a larger dispersion of bucketed subjects.
V2 Preparation Steps
We have benchmarked 15 QWEN models for their utilizable capacity with JSON vision tasks. https://huggingface.co/AbstractPhil/qwen-benchmark
Utilizing the strongest JSON caption structure we have devised a two stage processing with full vocabularies and a miniature LLM in planning to differentiate the most important elemental systems from each other.
Target: Qwen 3.5 4b as our VLM plain english captioner. Handles roughly the same output as the larger variants with a little less fidelity and a few different complexities. The similarity between the outputs and the speed for utilization is the main reason 4b was chosen for the VLM captioner.
Target: Qwen 3.5 9b as our JSON articulator for plain English to JSON processing. We will utilize the JSON translations from the more complex and more powerful json system.
Target: Qwen 3 VL 8b Uncensored as our JSON articulator for Booru to JSON processing. JoyCaption could not handle the processing to JSON effectively enough and the hallucination was problematic.
Target: Qwen 3.5 27b as our JSON articulator for Multilingual to JSON processing. The larger model showed a substantially more accurate response to json processing multilingual in valid format without requiring much attention or fixing. A larger variant would be ideal, but there's other problems related to those.
Current V2 Datasets
https://huggingface.co/datasets/AbstractPhil/qwen-synth-characters
https://huggingface.co/datasets/AbstractPhil/qwen-deepfashion
Subject Bucketing Supercharge
The models will be instructed with a multitude of system prompts before we settle on the final format for the JSON schema. The important structure revolves around the subject symbolic tree related to "top left to bottom right" subject fixated image ordering.
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
16 17 18 19 20
21 22 23 24 25For QWEN-centric image models the padding is TO THE LEFT; so the models often response like this instead it seems.
5 4 3 2 1
10 9 8 7 6
15 14 13 12 11
20 19 18 17 16
25 24 23 22 21So we'll need to test a few prototypes to see which is the most responsive with the JSON positional articulation. Ideally the system will contain 15 different vision tasks associated with roughly 1 million image composite system for it's finality.
So the adjudication principality of this behavior is very specifically aligned to one principle, the models decide the valuations - not a human. This is intentionally meant to be nearly fully automated, providing the models with all the necessary information, and then setting them lose onto a pod on their own being driven by Claude Ultracode with opus 4.8.
Below is the list of vision targets the models will be responsible for.
V1 is Strong, but too specific. Too shallow.
https://civitai.red/models/2730503/anima-jsonenglish?modelVersionId=3069633
The Qwen 3 0.5b knows some semantics, but the model is quite shallow. Very direct.
Qwen 3.5 0.8b isn't much smarter either. Essentially a toolchain mockup lookup that I jiggered into a symbolic lookup tree for subject association.
Not strong enough. Too shallow.
It was a good prototype and a good preview. Version 1 will be released soon for Anima and preparing for version 2 will be ready rapidly.
Current system
The subject-bucketed json data was prepared using Qwen 3.5 0.8b with a json tool activation finetune. This model is not smart, but the json did do some work.
The next model
Next up we're we're targeting 15 vision tasks and every single VLM on the list, targeting the single most likely to inference accurately model for all 15 vision tasks simultaneously with the JSON task.
Target: Behavioral Power
Semantics aren't enough. We need guarantees. Guaranteed json offset, guaranteed text, guaranteed associations, guaranteed rotation, depth, assessments, wireframes, and more.
Which produces valid json schema?
image classification
bounding box location
image text identification and accuracy checking
structural and spatial awareness
3d geometric object identification and awareness
camera rotational offset
subject fixation and awareness
semantic association
depth analysis
segmentation potential
vit accuracy to image prompting
outline and association testing
style identification and structural awareness
type differentiation with data types; json, yaml, MD, and a multitude of other potentials.
utilization and response to those types and the expected prompts
Structural Foundation
This is essentially a series of potentials that can in fact be achieved, maybe not with one model, however if one model can't handle them - a pile of them can.
This is not a tinker toy. This pile will be capable of forming the necessary JSON association required for a full JSON associative model.
Faster is obviously better as it currently operates, however I will also find the very best. The very best will be required for the foundational pieces, and those pieces must be as accurate as possible.
I am but one person.