HOWAI - Artificial Intelligence Blog

News

Fine-Tuning LLMs with a Bigger Teacher: A Practical Approach to Domain Intelligence

January 5, 2026 Lorenzo Mascia 15 min read

What if you could train a domain-expert LLM without massive compute, huge datasets, or full retraining? Instead of fine-tuning a model only on human-curated data, I explore a hybrid approach: Use a very large, general-purpose LLM as a teacher, let it generate high-quality, domain-specific data, and perform efficient fine-tuning (LoRA / QLoRA) on a smaller model. The result? Strong domain specialization, low computational cost, and production-ready models.

This approach sits at the intersection of: Knowledge distillation • Synthetic data generation • Parameter-efficient fine-tuning

It's especially effective in structured and technical domains (network automation, CLI parsing, configuration models, APIs), where precision and consistency matter more than sheer model size. I've detailed the full idea, architecture, and trade-offs. If you're working with LLMs in enterprise, automation, or vertical domains, this might save you a lot of time and compute.

LLM

How Tokenization Works in Large Language Models

January 15, 2025 Lorenzo Mascia 12 min read

When we interact with Large Language Models, it feels natural but under the hood, LLMs never see raw text. They operate entirely on tokens. Understanding tokenization is crucial for using LLM APIs efficiently, controlling costs, debugging model behavior, and designing better prompts...

LLM

Understanding Embeddings in Large Language Models

February 3, 2025 Lorenzo Mascia 8 min read

Embeddings translate language into numbers, but more specifically, they encode meaning as position in space. Understanding how embeddings work is crucial to understanding why they are at the core of almost everything modern LLMs can do, from semantic search to RAG systems...

LLM

Tokenization & Embedding: How Text Becomes Meaning

February 22, 2025 Lorenzo Mascia 8 min read

Understanding the relationship between tokenization and embeddings is essential to grasp how raw text transforms into semantic meaning. Learn how token IDs become embeddings, how transformers add context, and why this pipeline is fundamental to every modern NLP system...

LLM

Order Matters: Positional Encoding in Large Language Models

March 12, 2025 Lorenzo Mascia 10 min read

Transformers do not naturally understand order. Words are processed in parallel, making them fast and scalable, but language is sequential. Learn how Positional Encoding solves this fundamental problem and why it's the bridge between parallel computation and sequential meaning...

LLM

Why Sinusoidal Functions? The Math Behind Positional Encoding

April 5, 2025 Lorenzo Mascia 12 min read

Why were sinusoidal functions chosen for positional encoding? Learn how sine and cosine waves transform discrete positions into continuous signals, enable relative distance computation, and turn order into geometry. Discover the mathematical elegance that makes transformers understand time through signals...

LLM

Stacking Layers: How Transformers Build Intelligence

April 28, 2025 Lorenzo Mascia 14 min read

At the foundation of every modern LLM lies repetition. By stacking many identical transformer layers, models transform raw tokens into deep semantic understanding. Discover how multi-head self-attention, feed-forward networks, and residual connections work together to build intelligence through layering...

LLM

Self-Attention: How a Model Understands Context

May 18, 2025 Lorenzo Mascia 11 min read

When people say models "understand context," they're talking about self-attention. Learn how Query, Key, and Value work together to let each word consult all others, resolving ambiguity and capturing relationships. Discover the mechanism that changed how machines process language...

LLM

Next Token Prediction: What the Model Is Really Doing

June 9, 2025 Lorenzo Mascia 10 min read

Everything a language model does traces back to one simple objective: predicting the next token. Learn how probability distributions drive generation, why reasoning emerges as a side effect, and the crucial distinction between prediction and understanding...

LLM

Training: How Large Language Models Learn

July 2, 2025 Lorenzo Mascia 11 min read

Training adjusts billions of parameters to improve next-token prediction. Learn about pre-training on massive datasets, cross-entropy loss, backpropagation, gradient descent, and why computational scale is inseparable from model intelligence...

LLM

What Is Fine-Tuning in Large Language Models ?

July 25, 2025 Lorenzo Mascia 15 min read

Pretrained LLMs are generalists. Fine-tuned LLMs are specialists. Fine-tuning is the process that turns a powerful but generic model into one that is aligned, domain-aware, and task-effective. Learn what fine-tuning is, why it exists, how it works, and when you actually need it...

LLM

Inference: What Happens When You Ask a Question

August 14, 2025 Lorenzo Mascia 10 min read

Inference is where training meets reality. Learn how prompts become tokens and embeddings, how the forward pass generates probability distributions, and how sampling strategies shape output. Discover why what feels like conversation is actually controlled probability unfolding token by token...

LLM

Limits of Large Language Models

September 8, 2025 Lorenzo Mascia 11 min read

LLMs are powerful, but understanding their limits is essential. Learn about hallucinations, lack of real-world grounding, learned biases, finite context windows, and why fluency doesn't equal understanding. Discover the systematic constraints that shape what these models can and cannot do...

LLM

Modern Extensions: How LLMs Go Beyond Pure Text Prediction

October 1, 2025 Lorenzo Mascia 12 min read

Modern extensions don't change LLMs—they amplify them. Learn about tool calling, RAG, agentic systems, and multimodality. Discover how intelligence emerges not from making models smarter, but from designing better systems around them where prediction meets action, grounding, and perception...

LLM

Model Quantization: Making Large Models Smaller, Faster, and Cheaper

November 5, 2025 Lorenzo Mascia 11 min read

Quantization reduces numerical precision to make models smaller and faster. Learn how reducing from 32-bit to 8-bit or lower improves memory usage and inference speed, the trade-offs between post-training and quantization-aware approaches, and why this optimization is essential for practical deployment...

LLM

LoRA and QLoRA: Fine-Tuning Large Models Without Breaking the Bank

December 3, 2025 Lorenzo Mascia 12 min read

Learn how LoRA and QLoRA make fine-tuning affordable and accessible. Discover low-rank adaptation, 4-bit quantization, and how to customize powerful models without massive compute resources. These methods democratize AI by enabling efficient, modular adaptation that preserves base model knowledge...

Unlock the Power of Artificial Intelligence

Fine-Tuning LLMs with a Bigger Teacher: A Practical Approach to Domain Intelligence

How Tokenization Works in Large Language Models

Understanding Embeddings in Large Language Models

Tokenization & Embedding: How Text Becomes Meaning

Order Matters: Positional Encoding in Large Language Models

Why Sinusoidal Functions? The Math Behind Positional Encoding

Stacking Layers: How Transformers Build Intelligence

Self-Attention: How a Model Understands Context

Next Token Prediction: What the Model Is Really Doing

Training: How Large Language Models Learn

What Is Fine-Tuning in Large Language Models ?

Inference: What Happens When You Ask a Question

Limits of Large Language Models

Modern Extensions: How LLMs Go Beyond Pure Text Prediction

Model Quantization: Making Large Models Smaller, Faster, and Cheaper

LoRA and QLoRA: Fine-Tuning Large Models Without Breaking the Bank