5d (edited) • General discussion
Decluttering Model Training Process
The most exciting way to train AI is to let it learn real skills from synthetically generated objectives.
Suppose we want to learn photo decluttering. A problem with base model is that it will invent things - we don't want that. We want a very clear objective of ONLY removing the clutter. We define what's important: the model must preserve the room but remove the clutter. Meaning: no invented walls, no deleted walls, no reshaped walls. We can generate all of them with AI for training and then apply the model on real images and it will transfer the skill of decluttering.
Step 1: generate using image model (or get) clean, well lit, room images.
Step 2: generate using the same model exactly the same room, with exact same camera angle but cluttered
Step 3: train lora adapter to go from cluttered to decluttered image.
Here are all the prompts I actually used:
Cluttering prompt: ""Preserve the room exactly but add clutter, bad lighting, photo noise and make it into simply a badly taken photo. Be VERY careful NOT to change the room architecture. Keep it exactly the same as original photo. Do NOT change camera angle. Keep exact same camera angle. Do NOT add any text or camera date to the photo."
Training - 3 prompt variants, round-robin'd across 54 training images:
- A: "Declutter this photo. Remove all garbage and clutter. Keep the room's shape, architecture and camera angle exactly as it is. Fix the lighting so it looks bright and clean. Realphoto"
- B: "Remove all clutter, trash, and mess from this room. Do not change the walls, floor, windows, or camera angle. Make the lighting bright and even, like a professional real estate photo. Realphoto"
- C: "Make this into a clean, professionally lit real estate photo. Remove all clutter and personal items. Preserve the room's exact architecture and camera angle - do not change them. Realphoto"
Realphoto is just used as a lora activation keyword and is present in ever prompt. You don't have to use 3 different prompts since embeddings will be close but I found it gives slightly better result to vary the input prompts because it makes it a little more robust to different wordings.
Inference - the prompt used for every review batch, validation scoring, and final image generation throughout:
"Preserve the room exactly but remove all clutter, fix the lighting and remove photo noise and make it into simply a professionally taken real estate photo, mäklarfoto. Be VERY careful NOT to change the room architecture. Keep it exactly the same as original photo. Do NOT change camera angle. Keep exact same camera angle. Do NOT add any text or camera date to the photo."
Note this inference prompt is different in wording from all three training variants — it wasn't one of the three rotated captions, it's the original fixed prompt used since step 1 and kept constant for every generation call (training and inference use different prompt sets by design: training used the 3 rotating variants for robustness, inference always used this one fixed prompt for consistent comparison across checkpoints).
(I trained this on brain)
0
0 comments
Decluttering Model Training Process
powered by
skool.com/ai-software-engineering-1814
Getting you to 600k lines of code per week, fully automated.
Suggested communities
Powered by