Neelabh Kumar

Training a small model to run a house

I trained Qwen3-1.7B, an open model small enough to run on a consumer GPU, to control a house through Home Assistant. On the Home Assistant community's voice-command test it went from 75.0% to 89.8%.

The method is reinforcement learning with GRPO. The model never sees a correct answer. It tries each command eight times, a real Home Assistant checks which tries left the house right, and the model is nudged toward those. Training took seven hours on one RTX 3060, and the result is published as hua-1.7b.

The rest of this post explains how it works, step by step, down to the math of a single training step.

The training loop The training loop, as five boxes in a row: a command, what a person might say; the model, Qwen3-1.7B, writes eight answers; each answer runs in a real Home Assistant; the whole house is checked, 1 if right and 0 if not; the update nudges each answer up or down. A dashed line returns from the update to the next command: the loop ran for 1,600 steps. A note on Command: 'Unlock the August Lock', one of 4,318 sentences across 85 pretend homes. Above Check, the eight grades of one real step: 1 0 0 1 1 1 1 1, 6 of 8 right. “Unlock the August Lock”, one of 4,318 sentences across 85 pretend homes. Command what a person might say Model Qwen3-1.7B writes eight answers House each runs in a real Home Assistant Check whole house right? 1 or 0 one real step: 6 of 8 right Update GRPO nudges up or down 1,600 steps
Figure 1. The training loop, at a glance. Each part is explained in the sections below.

01 The tasks

403

tests across 85 pretend homes, 4,318 sentences in all.

The Home Assistant community has a public test that scores models on this, with a leaderboard. Each task is a sentence and a synthetic home. The model answers with a tool call, Home Assistant runs it, and the task passes only if the whole home ends up the way it should.

The test has three parts. The assist-mini set is 196 spoken commands in small homes with few devices, the one the leaderboard is sorted by. The assist set is 460 commands in a medium-sized home, written as corner cases to trip models up. The questions set is 370 questions about the house, where the right response is an answer, not an action.

What one test is What one test is, the August Lock test of one of our homes. The home: a living room with two locks, August Lock and Yale Deadbolt, both locked, and a sensor named after each lock. 11 ways to say it, among them 'Unlock the August Lock', 'Open up the August Lock' and 'August Lock, unlock it'. The end state it must reach: the August Lock unlocked, everything else as it was. the home Living Room August Lock locked Yale Deadbolt locked and a sensor named after each lock + 11 ways to say it “Unlock the August Lock” “Open up the August Lock” “August Lock, unlock it” and 8 more the end state August Lock unlocked everything else as it was
Figure 2. What one test is. One of our 403 tests: a home, the ways to say the command, and the end state.

Every answer can be checked by running it: the house ends up right or wrong, 1 or 0. That makes this a good fit for reinforcement learning.

I trained on my own tests, built by a generator in the same format as the community's. The community's three sets were kept out of training and used only to score the model.

02 How it works

The whole system The whole system in six parts. Tasks: 403 tests in 85 pretend homes, each a home, a command and how the home should end up. Verifier: each test is run in Home Assistant; all 403 pass. Pre-pass: the untrained model tries each sentence 8 times; 759 sentences were never right, 616 sometimes right and kept for training, 2,943 always right. Training: 1,600 steps in 7.3 hours; each step the model answers one of the 616 sentences 8 times, Home Assistant grades each answer, and the model is nudged toward the answers that worked; it is never shown a correct answer. One real step, Unlock the August Lock: six answers unlocked it, one tried to lock it (already locked, so nothing changed), one was unclear because its sensor has the same name. Evaluation, on the community's own tests: assist-mini 75.0 before and 89.8 after; assist 51.5 before and 68.0 after; questions 68.4 before and 70.0 after. Release: hua-1.7b on Hugging Face with the code to run it. A gold line joins the one Home Assistant to the parts that use it: it checks the tests and the pre-pass, grades every answer, and runs the final tests. Tasks 403 tests in 85 homes A pretend home, a command, and how the home should end up. Verifier All 403 pass Each test is run in Home Assistant. Pre-pass Each sentence is tried 8 times by the untrained model. A filled dot is a right answer. 759 never right 616 sometimes right kept: these can teach it 2,943 always right Training 1,600 steps, 7.3 hours Each step takes one of the 616 sentences. The model answers it 8 times, Home Assistant grades each answer, and the model is nudged toward the ones that worked. It is never shown a correct answer. model 8 answers 1 0 0 1 1 1 1 1 run each right? nudge leans toward the answers that worked One real step: “Unlock the August Lock” Six answers unlocked it. One tried to lock it, so it stayed locked. One was unclear: its sensor is also called August Lock. The six get nudged up, the two get nudged down. Evaluation The community’s own tests, never used in training. Same setup before and after. assist-mini 75.0 89.8 assist 51.5 68.0 questions 68.4 70.0 before after training Release hua-1.7b The trained model, free to download on Hugging Face, with the code to run it. Home Assistant checks the tests and the pre-pass, grades every answer, and runs the final tests.
Figure 3. The whole system. One Home Assistant, the gold line, does all the checking.

The model reads two things: Home Assistant's own prompt for the home, then the command. The prompt lists the rooms, each device with its name and its kind (light, lock, sensor), and the tools the model can use. A tool is an action Home Assistant can carry out, such as TurnOn or TurnOff. The model never changes the house itself. It writes a tool call: which tool, and which device to use it on. For "Unlock the August Lock", the right tool call is:

TurnOff
  name: August Lock
  kind: lock

The name picks the device. The kind is there because the lock's sensor is also named August Lock. On a lock, TurnOff means unlock. Home Assistant runs the call, and the scorer compares the whole home with the expected end state.

GRPO (Group Relative Policy Optimization) was introduced by DeepSeek in the DeepSeekMath paper (2024). The model gets one prompt and writes a group of answers to it. Each answer is scored. Answers that score above the group's average are made more likely, and answers below it less likely. Each answer is judged only against its own group: that is the "group relative" in the name.

Here, the prompt is one command and the group is eight answers. Each answer scores 1 if the house ends up right, 0 if not. Compare each answer with the group's average. Answers above it get their probability nudged up; answers below it get nudged down. That is one step. Move to the next command and repeat, 1,600 steps in all.

The numbers behind one stepFive short parts, worked on one real sentence.

1 The model answers eight times

The model gets "Unlock the August Lock" (the test from section 01) and answers it eight times. Each answer scores 1 if the house ends up right, 0 if not.

The model answers eight times, and each answer is scored The eight answers the untrained model gave to 'Unlock the August Lock', as three kinds. Six were TurnOff on the August Lock in the Living Room with the kind lock: unlocked it, score 1, push +0.54. One was TurnOn: it tried to lock the lock, which was already locked, score 0, push −1.62. One left out the kind of device: Home Assistant refused because two things are called August Lock, score 0, push −1.62. 6 answers: unlocked it TurnOff August Lock lock score 1 · push +0.54 1 answer: tried to lock it; it was already locked TurnOn August Lock lock score 0 · push −1.62 1 answer: refused: two things are called August Lock TurnOff August Lock none score 0 · push −1.62

The trainer also gave a point for any valid tool call. In 1,587 of the 1,600 steps all eight answers earned it, so it changed nothing.

2 Better answers are pushed up, worse ones down

The average score is 0.75. An answer above the average is made more likely. An answer below it is made less likely. The size of the push is the gap from the average, divided by the spread of the scores (0.463):

push (score − average) / spread

right answer (1 − 0.75) / 0.463 = +0.54

wrong answer (0 − 0.75) / 0.463 = −1.62

3 Equal scores push nothing

The pushes always add up to zero: six of +0.54 and two of −1.62. If all eight answers score the same, every push is zero and the model learns nothing. That is why the first run failed.

4 Only the unsure parts move

The model writes an answer one token at a time (a token is a word or part of a word), and it picks each token with some probability. A push up raises the probability of every token in that answer. A push down lowers it.

Before training, the model was already over 99.9% sure of 29 of the 32 tokens in the right answer, so those barely change. The learning happens at the three places where it was unsure, shown before training and after all 1,600 steps:

Each step pushes where the model was unsure The three choices where the model was unsure, before training and after all 1,600 steps: Off 92.3% or On 7.7% before, Off >99.9% after; go on 94.6% or stop 5.4% before, go on >99.9% after; lock 94.9% or sensor 5.1% before, lock >99.9% after. Off or On? On is the answer that kept it locked. before Off 92.3% · On 7.7% after Off >99.9% After the room: go on, or stop? Going on names the kind of device. Stopping is the answer that was refused. before go on 94.6% · stop 5.4% after go on >99.9% Which kind: lock or sensor? No answer picked the sensor, but it had a small chance. before lock 94.9% · sensor 5.1% after lock >99.9%

5 How the weights change

The loss is minus the push times the log probability of each token, averaged over all the tokens of the eight answers. Backpropagation gives the gradient, and each weight takes a small step against it:

p.data += -lr * p.grad

with lr = 5e-6, on the LoRA weights only. The real optimizer is AdamW, which sizes each weight's step separately; the idea is the same.

Each step took 16.4 seconds. Almost all of it is the model: writing the eight answers, then updating its weights. The scorer checks 62 answers a second.

03 What broke

90%

of the first run's steps had nothing to learn from: all eight answers scored the same.

The first run chose its training data by test, and that was the mistake. Before training, I asked the untrained model every sentence eight times and kept every test it passed only some of the time: 353 tests, 3,793 sentences. But a test has about eleven sentences, and each step uses only one. Inside a mixed test, most sentences still came back right all eight times or wrong all eight times. When the eight scores are equal, every answer sits at the average, so every nudge is zero and the step teaches nothing. I stopped the run at step 197.

04 The fix

616

of the 4,318 sentences, kept for training.

The fix is to choose by sentence, not by test: keep only the sentences the untrained model got right on some tries and wrong on others. The DAPO paper (2025) does the same thing during training and calls it dynamic sampling.

Eight tries per sentence Histogram of 4,318 sentences by how many of eight answers were right before training: 0 of 8: 759; 1 of 8: 69; 2 of 8: 43; 3 of 8: 35; 4 of 8: 61; 5 of 8: 63; 6 of 8: 81; 7 of 8: 264; 8 of 8: 2,943. The 616 sentences in between are the ones the model could learn from. sentences 1,000 2,000 0 0 1 2 3 4 5 6 7 8 759 2,943 never always 616 in between: the ones it can learn from right answers out of 8, before training (a filled dot is a right try)
Figure 4. Eight tries per sentence: mostly all right or all wrong. All 4,318 of our sentences, each answered eight times by the untrained model.
Where the learning happened Share of the eight answers that left the house right, per training step, smoothed over 51 steps, across 1,600 steps; 104 steps had a learning signal, a gradient that was not zero (the others had all eight answers graded the same), 27 of them in the first 50 steps and 10 after step 800. answers right, % · 51-step average 0 50 100 0 400 800 1,200 1,600 training step gold tick: a step with a learning signal 27 in the first 50 steps 10 after step 800
Figure 5. The learning happened in 104 of 1,600 steps. The model learned fast at first, then less and less as it got more sentences right. When all eight answers to a sentence are right, there is nothing left to learn from it.
After training, the same sentences come out right How many of 8 answers were right, per sentence, before (stock) and after training (hua-1.7b, checkpoint-1600). The 616 sentences it trained on: before 0: 0, 1: 69, 2: 43, 3: 35, 4: 61, 5: 63, 6: 81, 7: 264, 8: 0; after 0: 1, 1: 1, 2: 0, 3: 0, 4: 0, 5: 0, 6: 0, 7: 3, 8: 611; right answers 64.0% to 99.6%, 613 up, 1 same, 2 down. The 88 sentences of the 8 never-right tests, never trained on: before 0: 88, 1: 0, 2: 0, 3: 0, 4: 0, 5: 0, 6: 0, 7: 0, 8: 0; after 0: 7, 1: 2, 2: 0, 3: 2, 4: 2, 5: 3, 6: 2, 7: 9, 8: 61; right answers 0.0% to 84.4%, 81 up, 7 same, 0 down. before: the stock model, Sep 21 · after: hua-1.7b, checkpoint-1600, Sep 22 8 answers per sentence in each run, temperature 1.0 · our own tasks, not a benchmark it trained on the 616; it never saw the 88 (the 8 tests it never got right). after training before training 1 0 69 1 1 43 2 35 3 61 4 63 5 81 6 264 3 7 611 8 right answers out of 8 tries (a filled dot is a right try)
Figure 6a. After training, the 616 sentences it trained on come out right. Each sentence was tried eight times before training and eight times after. In each column, the grey bar counts the sentences with that many right tries before training, the blue bar after. These are our own sentences, not the benchmark.
After training, the same sentences come out right How many of 8 answers were right, per sentence, before (stock) and after training (hua-1.7b, checkpoint-1600). The 616 sentences it trained on: before 0: 0, 1: 69, 2: 43, 3: 35, 4: 61, 5: 63, 6: 81, 7: 264, 8: 0; after 0: 1, 1: 1, 2: 0, 3: 0, 4: 0, 5: 0, 6: 0, 7: 3, 8: 611; right answers 64.0% to 99.6%, 613 up, 1 same, 2 down. The 88 sentences of the 8 never-right tests, never trained on: before 0: 88, 1: 0, 2: 0, 3: 0, 4: 0, 5: 0, 6: 0, 7: 0, 8: 0; after 0: 7, 1: 2, 2: 0, 3: 2, 4: 2, 5: 3, 6: 2, 7: 9, 8: 61; right answers 0.0% to 84.4%, 81 up, 7 same, 0 down. before: the stock model, Sep 21 · after: hua-1.7b, checkpoint-1600, Sep 22 8 answers per sentence in each run, temperature 1.0 · our own tasks, not a benchmark it trained on the 616; it never saw the 88 (the 8 tests it never got right). after training before training 88 7 0 2 1 2 2 3 2 4 3 5 2 6 9 7 61 8 right answers out of 8 tries (a filled dot is a right try)
Figure 6b. Most of the 88 sentences it never trained on come out right too. They come from the 8 tests the model never got right before training, so they were left out of training.
After training, the same sentences come out right How many of 8 answers were right, per sentence, before (stock) and after training (hua-1.7b, checkpoint-1600). The 616 sentences it trained on: before 0: 0, 1: 69, 2: 43, 3: 35, 4: 61, 5: 63, 6: 81, 7: 264, 8: 0; after 0: 1, 1: 1, 2: 0, 3: 0, 4: 0, 5: 0, 6: 0, 7: 3, 8: 611; right answers 64.0% to 99.6%, 613 up, 1 same, 2 down. The 88 sentences of the 8 never-right tests, never trained on: before 0: 88, 1: 0, 2: 0, 3: 0, 4: 0, 5: 0, 6: 0, 7: 0, 8: 0; after 0: 7, 1: 2, 2: 0, 3: 2, 4: 2, 5: 3, 6: 2, 7: 9, 8: 61; right answers 0.0% to 84.4%, 81 up, 7 same, 0 down. before: the stock model, Sep 21 · after: hua-1.7b, checkpoint-1600, Sep 22 8 answers per sentence in each run, temperature 1.0 · our own tasks, not a benchmark it trained on the 616; it never saw the 88 (the 8 tests it never got right). tries right, % before training after training 64.0% 99.6% the 616 it trained on 0.0% 84.4% the 88 it never trained on
Figure 7. The same result as a share of right tries. Eight tries per sentence, on the sentences of Figures 6a and 6b, before and after training.

05 The result

75.0 89.8

on the community's voice-command test, before and after training.

Before and after training, beside the leaderboard's row Tests passed on the community's three test sets: assist-mini (196 tests): the leaderboard's row for the untrained Qwen3-1.7B 60.2% ±6.9 (4-bit through ollama, HA 2025.7.1), stock at bf16 75.0% ±6.1, hua-1.7b at bf16 89.8% ±4.2 (HA 2026.9.3); assist (460 tests): the leaderboard's row for the untrained Qwen3-1.7B 35.9% ±4.4 (4-bit through ollama, HA 2025.7.1), stock at bf16 51.5% ±4.6, hua-1.7b at bf16 68.0% ±4.3 (HA 2026.9.3); questions (370 tests): the leaderboard's row for the untrained Qwen3-1.7B 59.5% ±5.0 (4-bit through ollama, HA 2025.7.1), stock at bf16 68.4% ±4.7, hua-1.7b at bf16 70.0% ±4.7 (HA 2026.9.3). assist-mini 196 tests 60.2% stock 4-bit 75.0% stock bf16 89.8% hua-1.7b bf16 assist 460 tests 35.9% stock 4-bit 51.5% stock bf16 68.0% hua-1.7b bf16 questions 370 tests 59.5% stock 4-bit 68.4% stock bf16 70.0% hua-1.7b bf16 0 25 50 75 100
Figure 8. Training raised the two command scores by 14.8 and 16.5 points. Questions, which it never trained on, moved 1.6, inside the error bar. The first bar in each group is the leaderboard's own score for the untrained model, run at 4-bit (about 4 bits per weight instead of 16) through ollama on Home Assistant 2025.7.1. I ran the other two at full precision (bf16) on Home Assistant 2026.9.3. The error bars are the 95% interval.

The first two bars are the same untrained model, run two ways. The gap between them comes from the setup, not from training. Training is the step from the second bar to the third.

06 How it compares

89.8

on the same 196 tests: level with the leaderboard's 4B models, and with gpt‑oss‑120b.

Training put a 1.7B model level with models many times its size. On assist-mini, hua-1.7b scores above ministral-3-3b, a 3B model, and level with the leaderboard's 4B models. It is also level with gpt-oss-120b and qwen3-235b, which are about 70 and 140 times larger. On the larger assist test, the bigger models keep their lead: gpt-oss-120b scores 85.0, against 68.0 for hua-1.7b.

Where a 1.7B model lands on the board assist-mini, % of 196 tests passed, sorted: gemma4-e4b 91.3 (4-bit, HA 2026.2.3); hua-1.7b 89.8 (bf16, HA 2026.9.3); qwen3.5-4b 88.3 (4-bit, HA 2026.2.3); gemini-2.0-flash-lite 88.3 (api, HA 2025.4.3); gpt-oss-120b 87.8 (api, HA 2026.2.0b1); qwen3-235b-a22b-2507 87.2 (api, HA 2026.2.1); qwen3-4b-instruct 86.7 (4-bit, HA 2026.1.1); ministral-3-3b 81.6 (4-bit, HA 2026.1.1); qwen3-1.7b, stock 75.0 (bf16, HA 2026.9.3); gemma4-e2b 61.2 (4-bit, HA 2026.2.3); qwen3-1.7b 60.2 (4-bit, HA 2025.7.1); functiongemma-270m 9.4 (4-bit, HA 2026.9.2); needle3 5.6 (2-bit, HA 2026.9.2). board rows as published Sep 22, 2026 (its README); HA version at the right 4-bit: run locally at 4-bit (Q4_K_M or Q4_0), through ollama or llama.cpp api: a hosted service · bf16: ours, full precision, HA 2026.9.3, our own server each row's 95% interval is ±3 to ±7 points; rows within that of each other are level. A bracket joins qwen3-1.7b, stock to hua-1.7b: hua-1.7b is Qwen3-1.7B after training; both at bf16, 75.0 before and 89.8 after. gemma4-e4b 4-bit · 2026.2.3 91.3 hua-1.7b bf16 · 2026.9.3 89.8 qwen3.5-4b 4-bit · 2026.2.3 88.3 gemini-2.0-flash-lite api · 2025.4.3 88.3 gpt-oss-120b api · 2026.2.0b1 87.8 qwen3-235b-a22b-2507 api · 2026.2.1 87.2 qwen3-4b-instruct 4-bit · 2026.1.1 86.7 ministral-3-3b 4-bit · 2026.1.1 81.6 qwen3-1.7b, stock bf16 · 2026.9.3 75.0 gemma4-e2b 4-bit · 2026.2.3 61.2 qwen3-1.7b 4-bit · 2025.7.1 60.2 functiongemma-270m 4-bit · 2026.9.2 9.4 needle3 2-bit · 2026.9.2 5.6 hua-1.7b is Qwen3-1.7B after training. From 60.2 to 75.0: the same weights, 4-bit to bf16.
Figure 9. The leaderboard on assist-mini, as published on September 26: every model up to 4B, three larger ones, and hua-1.7b. Larger models that score above hua-1.7b are left out. The two bf16 rows are mine, run on a newer Home Assistant. The leaderboard's local models run compressed, at 4 bits per weight (needle3 at 2). "api" means a hosted service. Each score can be off by 3 to 7 points either way (the 95% interval), so rows that close together are level.

07 Why not just show it the answers

68.4 59.2

on questions about the house, after supervised fine-tuning on every answer. Trained by pass or fail, hua‑1.7b scored 70.0.

Supervised fine-tuning on all 4,318 stored answers gave the best command score of the four models, 94.4 on assist-mini, and cost 9 points on questions. The cause is what it was shown: every stored answer is a tool call, so it learned to answer questions with a tool call too. Pass-or-fail training on top could not repair this, for two reasons. It trains on commands only, so nothing pushed the questions back. And the model already got almost every command right all eight times, so only 10 of 1,600 steps changed anything. RL's Razor (2025) finds the same: reinforcement learning makes a model forget less than supervised fine-tuning.

Four versions of one model Tests passed on three benchmarks by four versions of Qwen3-1.7B, one serving route: stock: assist-mini 75.0%, assist 51.5%, questions 68.4%; hua-1.7b, pass/fail only: assist-mini 89.8%, assist 68.0%, questions 70.0%; shown every answer (SFT only): assist-mini 94.4%, assist 68.3%, questions 59.2%; shown every answer, then pass/fail (SFT + GRPO): assist-mini 93.9%, assist 68.3%, questions 62.2%. Home Assistant 2026.9.3 · full precision (bf16) · temperature 0.7; one serving route for every bar. assist-mini 196 tests stock 75.0 hua-1.7b, pass or fail 89.8 supervised fine-tuning 94.4 supervised fine-tuning, then pass or fail 93.9 assist 460 tests stock 51.5 hua-1.7b, pass or fail 68.0 supervised fine-tuning 68.3 supervised fine-tuning, then pass or fail 68.3 questions 370 tests stock 68.4 hua-1.7b, pass or fail 70.0 supervised fine-tuning 59.2 (−9.2) supervised fine-tuning, then pass or fail 62.2
Figure 10. Supervised fine-tuning helped on commands and hurt on questions. Four versions of Qwen3-1.7B, all at full precision (bf16) on Home Assistant 2026.9.3.

08 What is next

Three runs come next. The first applies the same recipe to a smaller model, FunctionGemma-270M, which scores 9.4 on the leaderboard today. The second is supervised fine-tuning on a set that includes questions, then pass-or-fail training on top, so the model keeps its questions score. The third is more pass-or-fail training on new tasks written for the misses that remain, most of them on the larger assist test.

The model, its serving code and the leaderboard scores are linked below.