Serving fine-tuned personas side by side on one GPU — a hands-on guide to multi-LoRA with vLLM

So far, gcube Lab has covered fine-tuning and serving separately.

• [Fine-tuning Llama 3.1 on an RTX 5090] — our first hands-on QLoRA fine-tuning on gcube, where we also fixed bugs live along the way
• [Distributed LLM fine-tuning with gcube replicas: a hands-on multi-node guide to Axolotl + DeepSpeed] — where we sped up training with two RTX 5090s
• [vLLM vs LMDeploy vs Ollama: what happens as concurrency rises] — where we compared serving engines’ throughput with Qwen3-8B

This post connects those two threads. Once you’ve fine-tuned a few times, adapters start piling up, and a new question comes up:

“I built a separate adapter for each persona. How many GPUs do I need to serve them?”

The answer: one is enough. With vLLM’s multi-LoRA serving, you can attach several adapters to a single base model and run a server that answers each request with a different persona, all on one GPU. In this post, we share how we verified this on a gcube RTX 5090 (32 GB). (We also reproduced the whole procedure from scratch on a fresh workload.)

Note on language: The adapters were trained on Korean data, so the test prompts and model outputs are in Korean. Outputs are translated from Korean and marked as such.

Why multi-LoRA

As fine-tuning projects multiply, there are two main ways to serve them.

• Merge, then serve: Merge each LoRA into the base model and deploy it as a standalone model. You need as many GPUs (or model instances) as you have personas.
• Multi-LoRA serving: Load the base model once, attach multiple adapters, and pick one per request. The base model takes up most of the GPU memory, while a rank-32 LoRA adapter is only about 1% of the base model’s size, so attaching several doesn’t add much load.

If you have persona adapters left over from side projects, the latter is far more economical.

Environment

Step 1: Prepare two different adapters

To see the effect clearly, we trained two adapters with completely different tones.

• gyaru: a casual persona that talks in gyaru style, a playful, slang-heavy register borrowed from Japanese “gal” culture (300 samples from the existing 3,000-example gyaru-dataset)
• persona-b: a counseling AI persona that uses only formal language, with no emoji or casual speech (40 newly generated synthetic examples)

The key is to save the LoRA as an adapter, without merging it into the base model.

model = FastLanguageModel.get_peft_model(
    model, r=32,
    target_modules=["q_proj","k_proj","v_proj","o_proj",
                     "gate_proj","up_proj","down_proj"],
    lora_alpha=32, lora_dropout=0, bias="none",
    use_gradient_checkpointing="unsloth",
)
# ... train with SFTTrainer ...
model.save_pretrained(output_dir)      # don't call merge_and_unload()
tokenizer.save_pretrained(output_dir)

On the RTX 5090, training took about 24 seconds for 300 samples and 18 seconds for 40. If you open the saved directory, you’ll find only adapter_model.safetensors and adapter_config.json, with no full model weights. It saves exactly what you need and nothing more.

Step 2: Start a vLLM multi-LoRA server

vLLM supports multi-LoRA serving with a single flag, --enable-lora. List your adapters in --lora-modules in name=path format.

vllm serve Qwen/Qwen2.5-7B-Instruct \
  --enable-lora \
  --max-loras 2 \
  --max-lora-rank 32 \
  --lora-modules gyaru=/workspace/adapters/gyaru \
                 persona-b=/workspace/adapters/persona_b \
  --port 8002 \
  --gpu-memory-utilization 0.85

Loading the base model (bf16) took 15.4 GB of GPU memory. Calling /v1/models lists the base model and both adapters, each exposed as if it were a separate model.

["Qwen/Qwen2.5-7B-Instruct", "gyaru", "persona-b"]

From then on, you just put the adapter name in the model field of each request. There’s no need to start a separate server.

curl http://localhost:8002/v1/chat/completions \
  -d '{"model": "gyaru", "messages": [...]}'

curl http://localhost:8002/v1/chat/completions \
  -d '{"model": "persona-b", "messages": [...]}'

Step 3: Verifying — same question, different personas

We sent the same question, “How’s the weather today?” (오늘 날씨 어때?), to each adapter.

It’s hard to believe these came from the same base model, the same GPU and the same server process — the tone is that different. We double-checked with another question (“Give me some health tips”), and when we reproduced it from scratch in a different container, the responses came back with exactly the same wording.

Step 4: Latency and concurrent requests

Cold start: The first request after the server starts is slow because CUDA graphs are compiled for each adapter. The first gyaru request took 35 seconds.

Warm: Once compilation is done, latency drops to 0.6–0.75 seconds per request.

Concurrent requests: Even when you send requests to gyaru and persona-b at the same time, vLLM’s batching scheduler groups them automatically. With the two requests sent in parallel, the total time was about 1.0 second — actually faster than sending them one after the other (about 1.4–1.5 seconds) — and the two responses didn’t mix (no cross-contamination). Reproducing it in a new container gave almost the same result: 0.997 seconds.

Wrapping up

• If you save fine-tuned adapters without merging them, vLLM’s --enable-lora lets you serve several personas at once on one GPU.
• On a gcube RTX 5090 (32 GB), the base model (7B, bf16) and two adapters loaded together without issue, and concurrent requests were handled reliably through batching.
• The only thing to watch for is the compilation cold start on the first request (tens of seconds). For a production service, we recommend sending warm-up requests right after deployment.

If you have a pile of adapters fine-tuned for side projects, you don’t need multiple GPUs. On gcube, one is enough.