A practical training guide

Get better results from your LoRA

A useful adapter starts with a clear goal, accurate examples, and questions that show whether it learned. GPUless gives you a place to prepare data, train, compare answers, and try an informed change.

The formatter gives you a draft. Qwen can help turn messy source text into training examples. It can miss qualifications, invent details, or ask weak questions. Review its output before training. A completed adapter and a lower training loss do not guarantee better answers. A verified download confirms artifact integrity, not learning quality.

1. Choose a goal
2. Review examples
3. Train and compare
4. Improve one thing

• Good examples
• Useful tests
• The loose score
• Your next attempt
• How attempts work

Start with one thing you want to improve

Write down a goal you can test: “Answer questions about this version’s lock bookkeeping, including its limits.” A focused section is easier to review than a whole book. Keep the product, platform, edition, and version attached to facts that depend on them.

A LoRA stores a small set of changes used alongside its original base model. It can help adapt behavior or knowledge, but it is not a dependable document database. If every answer must quote an exact, frequently changing document, giving the model relevant source text at question time may suit that need better. See Hugging Face’s LoRA reference for how adapters fit around a base model.

Write examples you would want the model to imitate

• Name the subject in the question. “Where does Maple v1 store its recovery tokens?” stands alone. “Where is it stored in the passage above?” needs context the model will not have.
• Answer exactly what was asked. A question about a maximum needs the number. A question about a container needs the container’s exact name. Keep necessary exceptions and limits.
• Teach the same fact in useful ways. Include a direct question, a natural paraphrase, an application, and a relevant boundary or false-premise correction. Copying one sentence many times gives you little evidence of transfer to a new question.
• Prefer complete, concise answers. Add enough explanation to make the fact useful. Remove plausible extras that the source does not support.
• Resolve contradictions first. Check identifiers, numbers, units, negation, and version differences. More examples cannot fix two incompatible target answers.

{"prompt":"How long is a Maple v1 recovery token valid?","completion":"A Maple v1 recovery token expires ten minutes after creation and can be used only once."}
{"prompt":"Can I reuse a Maple v1 recovery token before its ten minutes are up?","completion":"No. A Maple v1 recovery token can be used once, even within its ten-minute lifetime."}

Where GPUless helps

You can upload reviewed JSONL yourself or use the optional formatter as a starting point. Inspect the proposed records, fix weak questions, and check each answer against your source. Formatting does not start training; you choose when to submit a job.

Where source-based dataset building is enabled, GPUless also proposes facts, reviews examples against the passage, and keeps separate questions for checking the finished adapter. Those reviews are automated. They do not certify the source or replace your judgment. This helper accepts a focused passage up to 16 KiB; split larger material into coherent sections.

Learning focus selects where the adapter can change the model. It does not validate your data or promise a particular learning gain. Advanced settings remain available. When a recipe is labeled verified, its evidence applies to the stated base-model revision and task scope.

Choose tests before you train

1. Write five to ten questions you expect the adapter to answer, plus a few simple questions the base already handles. Keep them outside the training upload.
2. Use fresh wording and situations. Include exact details, limitations, and a question with a false premise when your source supports correcting it.
3. Ask the base model and adapter the same questions with the same prompt format and output budget. Where the studio offers Compare answers, use it on a completed job with an available registered base model. Otherwise, or for an uploaded model, download the adapter and compare with the base locally.
4. Read the actual answers. Check for correct facts, unsupported additions, contradictions, and new mistakes on your simple control questions.

If you turn a failed test into a training example, retire it as a fresh test and write a new one. Keep a few questions untouched until you are ready to assess your final candidate.

Use the same base-model revision and tokenizer when testing a download. Chat models need their expected chat template; a format mismatch can hurt answers independently of training. GPUless’s checked dataset path and bundled chat script use matching chat formatting. See the Transformers chat-template guide.

Treat the loose score as a clue

When automatic answer checks are available, the result compares base and adapter answers on a small set of questions generated from your source. A score such as Base 40% → Adapter 60% across five scored questions describes that sample only.

Open Inspect checked answers to see the questions, both answers, and reference answers. Automated graders can be wrong or share the same blind spot. Questions with uncertain grades are shown separately and excluded from the percentage. If nothing can be scored, there is no percentage.

These checks share the source used to prepare training data. They do not independently verify the source, measure broad model quality, or replace your own fresh questions. A higher score is encouraging; a small change on a tiny sample is weak evidence.

Make the next attempt teach you something

Keep a short note for each run: dataset revision, settings, what improved, what got worse, and the next change. Fix a clear data problem first. Change one major thing at a time so you can tell what helped.

Rank controls adapter capacity; epochs repeat the training examples; learning rate controls update size. Larger values do not guarantee better results. Avoid increasing every setting together to chase a score. Source coverage, trustworthy completions, and suitable model targets still matter.

Room to try, with a clear stopping point

When paid packs are enabled, the standard pack includes ten completed training attempts; the studio shows the configured price and allowance before purchase. Formatting, data validation, and answer checks do not use training attempts. A queued job reserves one attempt. Cancellation before completion or a service-side failure returns it.

A completed run uses an attempt even if the adapter needs more work. The allowance gives you room to compare and refine; it is not a promise that ten runs will achieve any particular result. Free pre-alpha mode requires no payment.

Used your pack? Take a moment to review your examples and compare your results. Choose a specific change you expect to help before starting another pack. You can keep inspecting and testing your completed adapters without using another training attempt; preview requests have rate limits.

When you have an adapter worth keeping, put your download to work with Transformers, or return to your studio to review the next attempt.