Planning Vietnamese LLM Fine-Tuning on an 84GB GPU

Why Plan First

Fine-tuning a Vietnamese LLM is not just running a training command. An 84GB GPU is a limited resource, so without upfront estimates, you can run out of memory, face longer runtimes, and see costs rise. A clear plan helps you choose the right model size, training configuration, and serving approach.

Choosing a 7B-32B Model Size

With one or a few 84GB cards, the 7B-32B range is worth considering. 7B models are usually easier to run and fit small datasets or quick experiments. Larger models in the 7B-32B range can deliver better quality, but they require tighter control over context length, batch size, and the number of parameters being updated. If you have only one card, prioritize a mid-sized model with quantization or adapters. If you have a few cards, you can consider 32B, but you need to account for both the training and serving pipelines.

LoRA/QLoRA and 84GB GPU Memory

LoRA/QLoRA reduces memory by updating only a subset of parameters. QLoRA typically combines quantization to keep the base weights at low precision while the adapter learns additional updates. This approach is especially useful when fine-tuning 27B or 32B models on an 84GB card. However, memory does not depend only on total parameter count. Prompt length, batch size, gradient checkpointing, number of adapters, and parameter type all matter. So, run a small configuration first before scaling up.

Estimating GPU Time and Cost

Cost can be estimated as: number of cards times 2.5 USD/hour/card, then multiplied by runtime. Before running, estimate the number of steps, epochs, batch size, sample length, and throughput. Run a small test set to measure actual speed and memory usage. With one card, runtime may be longer; with a few cards, consider scaling efficiency and synchronization complexity. Do not focus only on hourly price; calculate total time from data preparation, training, evaluation, to deployment.

Preparing Vietnamese Data

Data determines fine-tuning quality. You need to normalize encoding, diacritics, abbreviations, instructions, and context. Build separate training, validation, and evaluation sets. The data should represent real use cases: customer support, summarization, classification, instruction generation, or document processing. Remove duplicates and low-quality data. If data is limited, prioritize light fine-tuning, combined with prompt engineering or RAG, instead of forcing the model to learn too much from a small set.

Evaluation and Serving with vLLM

After fine-tuning, evaluate using the validation set, business criteria, and manual checks. Monitor latency, throughput, error rate, and user experience. vLLM is a suitable choice for serving inference after training. When configuring it, consider the 84GB card, context length, concurrent requests, and stability. If using a few cards, test scalability before production.

Strapi Vietnam Perspective

Based on Strapi Vietnam's operational experience, we are running a 27B FP8 model for internal use on a single card. This shows that with an 84GB GPU, a 27B model in FP8 format can be feasible for internal serving with a reasonable configuration. When planning fine-tuning, Strapi Vietnam usually starts with a mid-sized model, controls memory with LoRA/QLoRA, runs small-scale tests, and then scales up data and training time. The key is not to chase the largest model size, but to find the balance between quality, cost, and operations.