An AI that summarizes meeting notes your company’s way — serving the Korean small LLM Tri-7B on gcube with n8n automation
on this page
We build a pipeline that takes meeting notes and automatically produces action items organized by owner, task and due date, using a self-hosted Korean LLM. There are no external API costs, and internal meeting notes never leave the company. This post is set up so you can reproduce everything just by running the cells of a Jupyter notebook from top to bottom, with no terminal needed. Note on language: This tutorial is about a Korean-language LLM, so the meeting notes, prompts and model outputs are in Korean. Code is shown exactly as we ran it, with English translations in comments, and outputs are translated from Korean and marked as such.
1. Why self-hosting + (optional) fine-tuning instead of a general-purpose model
Give meeting notes to a general-purpose model like ChatGPT or Claude and ask it to “pull out the action items,” and you’ll get something reasonable. The problems come after that.
- Owners are addressed differently from your company’s conventions
- Department names are written differently every time (“Marketing Team” vs. “Marketing Division”)
- Due dates and formats vary, so someone has to fix them up before pasting into Slack or Notion
- And above all, meeting notes are sensitive internal information, which makes sending them to an external API uncomfortable
So this post focuses on two things:
- Self-hosting: run a Korean LLM directly on a gcube GPU so meeting notes never leave the company.
- (Optional) fine-tuning: teach the model your company’s own format, so it always produces the same format even with a short prompt.
In this post, we reproduce self-hosting, inference and n8n automation from start to finish, and take fine-tuning as far as showing that it extends from the same pipeline.
2. Choosing a model: why Tri-7B
Since the goal is automating Korean-language work, we need an open-source small LLM that’s strong in Korean. In this post we use Trillion Labs’ trillionlabs/Tri-7B.
- A 7.76B model trained on Korean, English and Japanese, strong at handling Korean nuance
- 32K context length → good for feeding in long meeting notes whole
- Its architecture is standard llama family, so tools like Unsloth and vLLM work as is with no extra setup (we confirm this later)
License note: Tri-7B is covered by the Trillion Labs AI Model License Agreement. If you fine-tune this model and distribute it externally, you must keep the
Tri-prefix in the model name and include the Notice file, and services with more than 1 million MAU or more than $10 million in annual revenue need a separate commercial license. Some versions are released under Apache 2.0, so check the LICENSE file in the model repository yourself before any real deployment.
3. Environment
| Item | Value |
|---|---|
| GPU | gcube RTX 5090 (32 GB VRAM) |
| Model | trillionlabs/Tri-7B |
| Serving | vLLM (OpenAI-compatible API), port 8001 |
| Automation | n8n (separate workload) |
| Way of working | Jupyter notebook (no terminal needed) |
Why port 8001: Depending on the gcube workload, port 8000 may already be taken by the default service. To avoid conflicts, this post uses 8001 throughout. If 8000 is free in your environment, you can use it; just change the port in the code below. Why a notebook: Even if you’re not comfortable with the terminal, you only need to run the cells in order, and each cell’s output stays in place, making results easy to check and share. Long-running servers like vLLM can run in the background with
subprocess.Popen, so the notebook doesn’t get blocked.
Create a new notebook (Python 3 (ipykernel)) in JupyterLab, then paste and run the cells below in order.
4. Following along in a notebook
Cell 1 — Check the environment
# Check the GPU
!nvidia-smi --query-gpu=name,memory.total,memory.used --format=csv
# Check key package versions
import torch, transformers
print('torch :', torch.__version__)
print('transformers:', transformers.__version__)
print('CUDA available:', torch.cuda.is_available())
print('GPU:', torch.cuda.get_device_name(0) if torch.cuda.is_available() else 'N/A')
Once you see the RTX 5090 and CUDA available (True), move on.
Cell 2 — Check that the model loads (transformers)
Because Tri-7B uses a standard architecture, it loads without trust_remote_code. The first run downloads the model (about 15 GB), so it takes a few minutes.
This requires outbound access from the notebook (that is, the gcube workload) to Hugging Face. If the download is blocked, check whether external access is allowed in your gcube network settings. Once downloaded, it’s cached and won’t be downloaded again.
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
MODEL = 'trillionlabs/Tri-7B'
tok = AutoTokenizer.from_pretrained(MODEL)
model = AutoModelForCausalLM.from_pretrained(MODEL, dtype=torch.bfloat16, device_map='auto')
print('OK: loaded with transformers')
# Free GPU memory for the next step (vLLM serving)
del model, tok
import gc; gc.collect(); torch.cuda.empty_cache()
print('GPU memory cleared')
If you see OK: loaded with transformers, the model runs fine on this GPU.
Cell 3 — (Optional) Fine-tuning: teaching the model your company’s format
If you only want to try serving, skip this cell and go to Cell 4. The base model alone summarizes well. However, as you’ll see in the inference results later, parts that depend on company- or situation-specific rules, such as date calculation, vary from run to run. To lock those down, you need fine-tuning that teaches the model your company’s meeting-notes format.
Important — order: Fine-tuning (training) and vLLM serving compete for the same GPU. Running both at once leaves too little memory and training fails. So finish fine-tuning first (this cell), free the GPU, then serve in Cell 4. If you’ve already started serving, shut it down with
pkill -f 'vllm serve'before training.
Preparing the dataset (do this on your own)
The heart of fine-tuning is data. Prepare your organization’s real meeting notes (anonymized), or data that reflects their format, as JSONL in the format below (ChatML/messages). We recommend 500–1,000 examples. How you build the dataset (anonymizing real data, generating synthetic data and so on) depends on your environment. Below is a minimal example just to show that fine-tuning actually runs.
import json
# ⚠️ Minimal example data — in practice, prepare hundreds of examples from your own company's meeting notes
# (system: "You are an assistant that organizes meeting notes into action items in owner / task / due date format."
# user: "Organize the action items from the following meeting notes: ..."
# assistant: the expected action items in your company's format)
samples = [
{"messages":[
{"role":"system","content":"당신은 회의록을 담당자/할 일/기한 형식의 액션아이템으로 정리하는 assistant입니다."},
{"role":"user","content":"다음 회의록에서 액션아이템을 정리해줘:\n참석자: 김대리, 이과장\n김대리: 랜딩페이지 시안 다음주 수요일까지 필요해요.\n이과장: 개발 초안은 화요일까지 드릴게요."},
{"role":"assistant","content":"- 담당자: 김대리 / 할 일: 랜딩페이지 시안 제작 / 기한: 다음주 수요일\n- 담당자: 이과장 / 할 일: 개발 초안 작성 / 기한: 다음주 화요일"}
]},
# ... (fill in your real data in the same format)
]
data = samples * 20 # duplicated to 60 examples for this demo (use diverse real data in practice)
with open('/workspace/meeting_action_items.jsonl','w',encoding='utf-8') as f:
for d in data:
f.write(json.dumps(d, ensure_ascii=False)+'\n')
print('Dataset created:', len(data), 'examples')
Running training (Unsloth QLoRA)
from unsloth import FastLanguageModel
from trl import SFTTrainer, SFTConfig
from datasets import load_dataset
import torch
# 1) Load the model in 4-bit
model, tokenizer = FastLanguageModel.from_pretrained(
model_name='trillionlabs/Tri-7B',
max_seq_length=2048,
load_in_4bit=True,
)
# 2) Attach LoRA adapters
model = FastLanguageModel.get_peft_model(
model, r=16,
target_modules=['q_proj','k_proj','v_proj','o_proj','gate_proj','up_proj','down_proj'],
lora_alpha=16, use_gradient_checkpointing='unsloth',
)
# 3) Load the dataset + apply the chat template
ds = load_dataset('json', data_files='/workspace/meeting_action_items.jsonl', split='train')
ds = ds.map(lambda ex: {'text': tokenizer.apply_chat_template(ex['messages'], tokenize=False)})
# 4) Train (max_steps=10 for this demo; adjust epochs/steps to your data in practice)
trainer = SFTTrainer(
model=model, tokenizer=tokenizer, train_dataset=ds,
args=SFTConfig(
per_device_train_batch_size=2, gradient_accumulation_steps=2,
max_steps=10, learning_rate=2e-4, logging_steps=1,
output_dir='/workspace/tri7b-meeting-lora', dataset_text_field='text',
max_seq_length=2048, report_to='none',
),
)
result = trainer.train()
print('Training complete! final_loss =', round(result.training_loss, 4))
# 5) Save the trained LoRA adapter
model.save_pretrained('/workspace/tri7b-meeting-lora')
tokenizer.save_pretrained('/workspace/tri7b-meeting-lora')
print('LoRA adapter saved: /workspace/tri7b-meeting-lora')
Actual run results (with the example data above):
Dataset created: 60 examples
...
Training complete! final_loss = 2.3759
LoRA adapter saved: /workspace/tri7b-meeting-lora
If training completes without errors, the fine-tuning pipeline works. From here, you only need to scale up the data and steps with real data.
Tri-7B is registered with
model_typellama, so Unsloth automatically recognizes it with “Fast Llama patching.” In other words, you can reuse existing QLoRA code for Llama or Qwen just by changing the model name.
Fully freeing the GPU after training (required before serving — restart the kernel)
If you ran fine-tuning, be sure to restart the kernel before moving on to serving (Cell 4). In the top menu, run Kernel → Restart Kernel.
Why
del+empty_cache()isn’t enough (measured): Even afterdel modelandtorch.cuda.empty_cache(), the GPU isn’t fully freed. In our test, about 7.4 GB stayed in use, and starting vLLM in that state makes serving fail with this error:
ValueError: Free memory on device cuda:0 (23.57/31.36 GiB) on startup is less than desired GPU memory utilization (0.85, ...). Decrease GPU memory utilization or reduce GPU memory used by other processes.The reason is that
del/empty_cache()only clears Python references; the CUDA context held by the kernel process that ran training isn’t released as long as the kernel is alive. Only restarting the kernel fully frees the GPU (check: usage drops back to 4 MiB), after which serving works normally.
Restarting the kernel clears all earlier variables (model, trainer and so on), but the trained LoRA adapter is already saved in /workspace/tri7b-meeting-lora, so you can use it for serving as is. After restarting, rerun only Cell 1 (the imports) and go to Cell 4.
Cell 4 — Start the vLLM server in the background
Because it runs in the background with subprocess.Popen, this cell finishes immediately while the server keeps running.
If you ran Cell 3 (fine-tuning): Make sure you restarted the kernel (Kernel → Restart Kernel) before this cell. Otherwise, the GPU memory held by the training kernel makes serving fail with
ValueError: Free memory ... is less than desired GPU memory utilization. After restarting, rerun just the imports in Cell 1 and come back to this cell. (If you skipped Cell 3 and are only serving, no restart is needed.) What to serve: If you skipped fine-tuning in Cell 3, serve the base model (trillionlabs/Tri-7B) as is, as shown below. If you did fine-tune, add--enable-lora --lora-modules meeting=/workspace/tri7b-meeting-loraaftervllm serveto serve the trained LoRA adapter alongside it, and setmodeltomeetingin your requests. The example below uses the base model.
import subprocess
PORT = 8001
log = open('/workspace/vllm.log', 'w')
proc = subprocess.Popen([
'vllm', 'serve', 'trillionlabs/Tri-7B',
'--host', '0.0.0.0', '--port', str(PORT),
'--max-model-len', '8192',
'--gpu-memory-utilization', '0.85',
'--enable-auto-tool-choice', '--tool-call-parser', 'hermes',
], stdout=log, stderr=subprocess.STDOUT)
print('vLLM started in the background, PID =', proc.pid)
What each flag means:
--host 0.0.0.0: binds to all interfaces so it can be reached from outside the container (that is, from the n8n workload)--port 8001: the service port (avoids a conflict with 8000)--max-model-len 8192: the maximum token length per request (adjust to the length of your meeting notes, up to 32768)--gpu-memory-utilization 0.85: use up to 85% of GPU memory--enable-auto-tool-choice --tool-call-parser hermes: OpenAI-compatible tool calling (recommended for compatibility with n8n, Open WebUI and others)
Next, wait until the server is fully up (usually 1–2 minutes for loading the model and preparing CUDA graphs).
import requests, time
BASE = '<http://localhost:8001>'
for i in range(60): # wait up to 5 minutes
try:
r = requests.get(f'{BASE}/v1/models', timeout=3)
if r.status_code == 200:
print('Server ready:', r.json()['data'][0]['id'])
break
except Exception:
pass
print(f' waiting... ({i*5}s)')
time.sleep(5)
else:
print('Timed out - check /workspace/vllm.log')
When you see Server ready: trillionlabs/Tri-7B, serving is up.
Cell 5 — Inference test (meeting notes → action items)
Check that the model actually summarizes meeting notes. n8n calls it in exactly the same way (the OpenAI-compatible /v1/chat/completions).
import requests
# Korean meeting notes used in the test (English translation):
# Date: 2026-07-08
# Attendees: Assistant Manager Kim (Marketing), Manager Lee (Development), Team Lead Park (Planning)
# Park: Our goal is to launch the new product landing page next week.
# Lee: Development will have a first draft done by Thursday.
# Kim: I'll have the copy and banner images ready by Friday.
# Park: Great. Let's all do QA together Friday afternoon.
meeting = '''회의 일시: 2026-07-08
참석자: 김대리(마케팅), 이과장(개발), 박팀장(기획)
박팀장: 다음 주 신제품 랜딩페이지 오픈이 목표입니다.
이과장: 개발은 목요일까지 초안 완료하겠습니다.
김대리: 카피와 배너 이미지는 제가 금요일까지 준비할게요.
박팀장: 좋습니다. QA는 금요일 오후에 다같이 진행하죠.'''
payload = {
'model': 'trillionlabs/Tri-7B',
'messages': [
# "You are an assistant that organizes meeting notes into action items in owner / task / due date format."
{'role': 'system', 'content': '당신은 회의록을 담당자/할 일/기한 형식의 액션아이템으로 정리하는 assistant입니다.'},
# "Organize the action items from the following meeting notes:"
{'role': 'user', 'content': '다음 회의록에서 액션아이템을 정리해줘:\n' + meeting},
],
'temperature': 0.3,
'max_tokens': 512,
}
r = requests.post('<http://localhost:8001/v1/chat/completions>', json=payload, timeout=60)
print(r.json()['choices'][0]['message']['content'])
Example output (Tri-7B, prompt only, no fine-tuning; translated from Korean):
Here are the action items extracted from the meeting notes, organized by owner / task / due date:
1. New product landing page launch
- Owner: Team Lead Park (Planning)
- Task: Lead planning and preparation of the new product landing page
- Due: 2026-07-15 (target: next week)
2. Complete development draft
- Owner: Manager Lee (Development)
- Task: Write and complete the first development draft for the new product
- Due: 2026-07-14 (by Thursday)
3. Prepare copy and banner images
- Owner: Assistant Manager Kim (Marketing)
- Task: Write promotional copy and create banner images for the new product
- Due: 2026-07-15 (by Friday)
4. Run QA (quality assurance)
- Owners: Team Lead Park (Planning), Manager Lee (Development), Assistant Manager Kim (Marketing) (joint)
- Task: QA the new product landing page and related elements
- Due: 2026-07-15 (Friday afternoon)
In the example above, the model assigned owners and tasks correctly, and the format comes out reliably with the base model alone. The dates, however, are off. The meeting was on Wednesday, 2026-07-08, so “by Thursday” should be 07-09 (or 07-16 if it means next week), but the model wrote 07-14, which is a Tuesday. “By Friday” likewise became 07-15, a Wednesday.
Note — the base model’s limits: Owner and task assignment is stable, but if you run the same meeting notes several times, the date calculation varies from run to run. Even the run above converted the weekdays into the wrong dates, and other runs put the meeting date instead. Lowering the temperature doesn’t eliminate this completely. Calculations that involve company- or situation-specific rules, such as converting a weekday into an actual date, are hard to stabilize with prompts alone; locking them down with the fine-tuning in section 6 is the reliable approach.
5. Automating with n8n
Now we connect this vLLM endpoint to an n8n workflow. Key assumption: if n8n runs as a separate workload, it must call the GPU workload’s address, not localhost.
5-1. Find the GPU workload’s address
Run the following in the notebook to find the GPU workload’s internal IP.
import socket
print('Container internal IP:', socket.gethostbyname(socket.gethostname()))
The IP it prints (e.g. 10.0.10.208) is the address n8n uses to call vLLM. In n8n, you access it as http://<this IP>:8001. This IP changes when the container restarts (it actually changed after a restart while we were verifying this post), so rerun this cell to check the current IP before configuring n8n.
Caution: This internal IP can change when the container restarts. Using the IP directly is fine for a one-off test, but for ongoing operation we recommend replacing it with a fixed service address provided by gcube, if available.
5-2. Building the n8n workflow
Create a new workflow in n8n and place the nodes in the following order.
Step 1 — Trigger
Choose a trigger that matches how meeting notes come in.
- Testing: Manual Trigger (“When clicking ‘Execute workflow’”)
- Production: Notion Trigger when a new page is created in Notion, or Slack Trigger for a Slack channel
Step 2 — HTTP Request node (calling vLLM)
Click + to the right of the Manual Trigger, add an HTTP Request node and configure it as follows.
- Method: POST
- URL:
http://<IP from 5-1>:8001/v1/chat/completions(e.g.http://10.0.10.208:8001/v1/chat/completions) - Send Body: on
- Body Content Type: JSON
- JSON Body:
{
"model": "trillionlabs/Tri-7B",
"messages": [
{"role": "system", "content": "당신은 회의록을 담당자/할 일/기한 형식의 액션아이템으로 정리하는 assistant입니다."},
{"role": "user", "content": "다음 회의록에서 액션아이템을 정리해줘:\n{{ $json.meeting_text }}"}
],
"temperature": 0.3,
"max_tokens": 512
}
The prompts are the same Korean prompts as in Cell 5. Replace {{ $json.meeting_text }} with the field that carries the meeting notes text from the previous step (the trigger or an extraction node). For a simple connection test, you can put fixed text here instead.
Step 3 — Parse the response (Edit Fields node)
The summary in the vLLM response sits deep in a path like choices[0].message.content. To make it easy to use in later nodes (Slack/Notion), pull this value out into a single flat field with an Edit Fields (Set) node.
Click + to the right of the HTTP Request node, add an Edit Fields node, and:
- Mode: Manual Mapping
- Add one field:
- Name:
action_items - Type: String
- Value:
{{ $json.choices[0].message.content }}
- Name:
Caution: You must enter the Value in Expression mode. If you leave the field as plain text and enter
{{ $json... }}, that string is saved literally. Turn on the expression toggle (the=icon) in the value field before entering it. The path$json.choices[0].message.contentfollows the OpenAI-compatible response structure, so it’s the same for both the base and fine-tuned models.
Step 4 — Deliver the result (Slack or Notion)
Click + to the right of Edit Fields to add a destination node. Both nodes need credentials first, which you have to set up with your own account.
Option A — Slack (Slack → Send a Message)
- Credential: connect your Slack account (OAuth or bot token)
- Resource: Message / Operation: Send
- Send Message To: Channel → choose the channel to post to
- Message Text:
{{ $json.action_items }}(expression mode)
Option B — Notion (Notion → Create a Page)
- Credential: connect a Notion integration
- Resource: Database Page / Operation: Create
- Database: choose the database for meeting notes
- Title: e.g.
{{ $now.format('yyyy-MM-dd') }} Meeting action items - Content property:
{{ $json.action_items }}(expression mode) - Prerequisites: ① get an integration token at notion.so/my-integrations, and ② connect (share) the integration to the target database via ··· → Connections at its top right. n8n can access the database only after both steps.
With this, the full flow is complete:
Trigger → HTTP Request (vLLM) → Edit Fields (action_items) → Slack or Notion
5-3. Run and check the results
Click Execute workflow at the top right. The HTTP Request node turns green when done, and you can see the action item summary in choices[0].message.content in the output panel.
Two common pitfalls (we hit both while reproducing this):
- “Method not allowed” because the request goes out as GET — If you don’t set the Method on n8n’s HTTP Request node, it defaults to GET.
/v1/chat/completionsaccepts only POST, so be sure to set Method toPOST.- Errors because
{{ $json.meeting_text }}is empty — If you run with only a Manual Trigger, there’s no previous node passing in meeting notes, so this expression is empty. During connection testing, replace it with actual meeting notes text.Tip for checking the connection first: Inference takes 10–20 seconds, so to quickly check reachability, temporarily change the URL to
.../v1/models(GET) and run it. If you get a model list JSON containingtrillionlabs/Tri-7B, the connection works; if you get a timeout or connection refused, recheck the IP/port or network policy. When you’re done, switch back toPOST .../v1/chat/completions.
With this, the whole flow runs unattended with no external APIs: meeting notes in → n8n → self-hosted Tri-7B → action items → automatically organized in Slack/Notion.
6. Beyond the example — scaling up fine-tuning
In Cell 3, we confirmed that fine-tuning actually runs (training completed on example data). Taking it to production quality is ultimately a question of data.
| Item | Example (Cell 3) | Recommended for production |
|---|---|---|
| Data source | 3 duplicated samples | Your company's real meeting notes (anonymized) |
| Scale | 60 examples (repeated) | 500–1,000 or more |
| max_steps | 10 (just to confirm it runs) | Set in epochs to fit your data |
The data must be your own company’s: The purpose of fine-tuning isn’t general summarization but reproducing your organization’s own meeting format and rules (how departments are written, how owners are addressed, converting weekdays to dates and so on). Train on another company’s examples and you’ll just get “a model that summarizes in someone else’s company voice.” If you have real meeting notes, anonymize attendee names and sensitive information before using them, and build and source your dataset in whatever way fits your environment.
After training, serve the LoRA adapter alongside the model in Cell 4 (--enable-lora --lora-modules ...) and change the model value in your inference and n8n requests to the trained adapter’s name. The automation pipeline (section 5) stays the same; you just swap the model.
7. Cleaning up the server
When you’re done, shut down vLLM to release the GPU.
proc.terminate()
print('vLLM stopped')
If you restarted the notebook kernel, the proc variable is gone, so you can also stop it like this.
import subprocess
subprocess.run(['pkill', '-f', 'vllm serve'])
print('Requested vLLM process shutdown')
If GPU memory still shows as used in
nvidia-smiafter you stop vLLM, the current kernel is holding a CUDA context. Restart the kernel with Kernel → Restart Kernel to release it completely.
8. Wrapping up
In this post, we solved a practical task, summarizing meeting notes, by running a Korean model (Tri-7B) directly on a gcube GPU without sending data outside the company. We then connected it to n8n to automate everything from meeting notes input to organizing and delivering the results.
In summary:
- Tri-7B uses the standard llama architecture, so transformers, Unsloth and vLLM work with no extra setup
- Opening an OpenAI-compatible endpoint with vLLM connects it directly to n8n, Open WebUI and more
- The base model alone gives good Korean summarization quality, and if you need to lock in your company’s format, the same pipeline extends to fine-tuning
If you want to automate work while handling internal data safely and without external API costs, spin up a GPU on gcube and try it yourself.