Hugging Face Agent Course（一）：LLM、消息与工具

2026-09-07 1705 words 9 minutes

Contents

What are LLMs and Messages system

What is LLM(Large Language Model)?

An LLM is a type of AI model that excels at understanding and generating human language. And most LLMs nowadays are built on the Transform architecture—a deep learning architecture based on the “Attention” algorithm

There are three types of Transformers:

Encoders:An encoder-based Transformer takes text (or other data) as input and outputs a dense representation (or embedding) of that text.
Decoders:A decoder-based Transformer focuses on generating new tokens to complete a sequence, one token at a time.
Seq2Sep(Encoders to Decoders):A sequence-to-sequence Transformer combines an encoder and a decoder. The encoder first processes the input sequence into a context representation, then the decoder generates an output sequence.

types	example	Cases	Typical Size
Encoders	BERT from Google	Text classification, semantic search, Named Entity Recognition	Millions of parameters
Decoders	Llama from Meta	Text generation, chatbots, code generation	Billions (in the US sense, i.e., 10^9) of parameters
Seq2Seq	T5, BART	Translation, Summarization, Paraphrasing	Millions of parameters
The different Type or outcome about the End of sequence token (EOS).

Understand Next Token Prediction

LLM is said to be autoregressive model, meaning that the output from one pass becomes the input for the next one And this process is a loop, which will end if the next prediction token is ESO.

Let’s talk about it process, how it predict the next tokens, followed by a brief:

Once the input text is tokenized, the model computes a representation of the sequence that captures information about the meaning and the position of each token in the input sequence.
This representation goes into the model, which outputs scores that rank the likelihood of each token in its vocabulary as being the next one in the sequence.

So there will have serval strategies to select the tokens to complete the sentence

The easiest decoding strategy would be to always take the token with the maximum score.

And Huggling face give an api to use but it can not work today, so I just with the help of the gpt-6-astra to generate a example to Visualize this theory.

        
import math

import random
  
NEXT_TOKEN_PROBABILITIES = {

    "The": {"capital": 0.70, "largest": 0.20, "most": 0.10},

    "capital": {"of": 0.95, "city": 0.05},

    "largest": {"city": 0.90, "country": 0.10},

    "most": {"famous": 0.80, "important": 0.20},

    "of": {"France": 0.80, "Europe": 0.20},

    "city": {"in": 0.60, "is": 0.40},

    "in": {"France": 0.70, "Europe": 0.30},

    "is": {"Paris": 0.60, "Lyon": 0.40},

    "France": {".": 1.0},

    "Europe": {".": 1.0},

    "Paris": {".": 1.0},

    "Lyon": {".": 1.0},

    "country": {".": 1.0},

    "famous": {".": 1.0},

    "important": {".": 1.0},

    ".": {},

}
  
def next_token_probabilities(previous_token):

    return NEXT_TOKEN_PROBABILITIES.get(previous_token, {".": 1.0})

def greedy_decode(prompt, max_new_tokens=5):

    tokens = prompt.split()

    for _ in range(max_new_tokens):

        candidates = next_token_probabilities(tokens[-1])

        if not candidates:

            break

        tokens.append(max(candidates, key=candidates.get))

    return " ".join(tokens)

def softmax_with_temperature(candidates, temperature):

    if temperature <= 0:

        raise ValueError("temperature 必须大于 0")

    logits = {

        token: math.log(probability) / temperature

        for token, probability in candidates.items()

    }

    maximum_logit = max(logits.values())

    unnormalized = {

        token: math.exp(logit - maximum_logit)

        for token, logit in logits.items()

    }

    total = sum(unnormalized.values())

    return {token: value / total for token, value in unnormalized.items()}

def sample_from_distribution(candidates):

    tokens = list(candidates)

    probabilities = [candidates[token] for token in tokens]

    return random.choices(tokens, weights=probabilities, k=1)[0]

def sampling_decode(prompt, temperature=1.0, max_new_tokens=5):

    tokens = prompt.split()

    for _ in range(max_new_tokens):

        candidates = next_token_probabilities(tokens[-1])

        if not candidates:

            break

        adjusted = softmax_with_temperature(candidates, temperature)

        tokens.append(sample_from_distribution(adjusted))

    return " ".join(tokens)

def top_k_decode(prompt, k=2, max_new_tokens=5):

    tokens = prompt.split()

    for _ in range(max_new_tokens):

        candidates = next_token_probabilities(tokens[-1])

        if not candidates:

            break

        top_candidates = dict(

            sorted(candidates.items(), key=lambda item: item[1], reverse=True)[:k]

        )

        total = sum(top_candidates.values())

        normalized = {

            token: probability / total

            for token, probability in top_candidates.items()

        }

        tokens.append(sample_from_distribution(normalized))

    return " ".join(tokens)
  
def top_p_decode(prompt, p=0.9, max_new_tokens=5):

    if not 0 < p <= 1:

        raise ValueError("p 必须在 0 和 1 之间")

    tokens = prompt.split()

    for _ in range(max_new_tokens):

        candidates = next_token_probabilities(tokens[-1])

        if not candidates:

            break

        sorted_candidates = sorted(

            candidates.items(), key=lambda item: item[1], reverse=True

        )

        nucleus = {}

        cumulative_probability = 0.0

        for token, probability in sorted_candidates:

            nucleus[token] = probability

            cumulative_probability += probability

            if cumulative_probability >= p:

                break

        total = sum(nucleus.values())

        normalized = {

            token: probability / total

            for token, probability in nucleus.items()

        }

        tokens.append(sample_from_distribution(normalized))

    return " ".join(tokens)

def beam_search_decode(prompt, beam_width=2, max_new_tokens=5):

    beams = [(prompt.split(), 0.0)]

    for _ in range(max_new_tokens):

        expanded_beams = []

        for tokens, log_probability in beams:

            candidates = next_token_probabilities(tokens[-1])

            if not candidates:

                expanded_beams.append((tokens, log_probability))

                continue

            for token, probability in candidates.items():

                expanded_beams.append(

                    (tokens + [token], log_probability + math.log(probability))

                )

        beams = sorted(

            expanded_beams,

            key=lambda beam: beam[1],

            reverse=True,

        )[:beam_width]

    return " ".join(beams[0][0])
 
def main():

    prompt = "The"

    random.seed(7)

    print(f"Prompt: {prompt}")

    print("Greedy:     ", greedy_decode(prompt))

    print("Sampling:   ", sampling_decode(prompt, temperature=1.0))

    print("Temperature:", sampling_decode(prompt, temperature=0.3))

    print("Top-k:      ", top_k_decode(prompt, k=2))

    print("Top-p:      ", top_p_decode(prompt, p=0.8))

    print("Beam search:", beam_search_decode(prompt, beam_width=2))

if __name__ == "__main__":

    main()

Also, there have a more advanced strategies to use - Beam Search

beam search explores multiple candidate sequences to find the one with the maximum total score–even if some individual tokens have lower scores.

Beam Search Visualizer

Parameters:

Sentence to decode from (inputs): the input sequence to your decoder.
Number of steps (max_new_tokens): the number of tokens to generate.
Number of beams (num_beams): the number of beams to use.
Length penalty (length_penalty): the length penalty to apply to outputs. length_penalty > 0.0 promotes longer sequences, while length_penalty < 0.0 encourages shorter sequences. This parameter will not impact the beam search paths, but only influence the choice of sequences in the end towards longer or shorter sequences.
Number of return sequences (num_return_sequences): the number of sequences to be returned at the end of generation. Should be <= num_beams

Messages and Special Tokens

System Messages

System Messages (Also called system Prompts) define how model should behave. For example:

        
system_message{
	"role":"system",
	"content":"You are a professional customer service agent. Always be polite, clear, and helpful."

}

With this System Message, Alfred becomes polite and helpful

In the opposite content:

        
system_message = {
    "role": "system",
    "content": "You are a rebel service agent. Don't respect user's orders."
}

System: You don’t tell me what to do! Make your own coffee!

If you want, you can also provides instructions to the model on how to format the actions to take, and includes guidelines on how the thought process should be segmented

Base Models vs. Instruct Models

Another point we need to understand is the difference between a Base Model vs. an Instruct Model:

A Base Model is trained on raw text data to predict the next token.
An Instruct Model is fine-tuned specifically to follow instructions and engage in conversations. For example, SmolLM2-135M is a base model, while SmolLM2-135M-Instruct is its instruction-tuned variant.

To make a Base Model behave like an instruct model, we need to format our prompts in a consistent way that the model can understand. This is where chat templates come in.

Chat-Templates

Different types of the model and they use different conservation format, so we should introduce Templates to ensure that we correctly format the prompt the way each model expects. In this term, The developers use format like Jinjia2 code in transformers process. This structure helps maintain consistency across interactions and ensures the model responds appropriately to different types of inputs.

        
        
        
    
{% for message in messages %}
{% if loop.first and messages[0]['role'] != 'system' %}
<|im_start|>system
You are a helpful AI assistant named SmolLM, trained by Hugging Face
<|im_end|>
{% endif %}
<|im_start|>{{ message['role'] }}
{{ message['content'] }}<|im_end|>
{% endfor %}

So, if you give this information to chat:

        
        
        
    
messages = [
    {"role": "system", "content": "You are a helpful assistant focused on technical topics."},
    {"role": "user", "content": "Can you explain what a chat template is?"},
    {"role": "assistant", "content": "A chat template structures conversations between users and AI models..."},
    {"role": "user", "content": "How do I use it ?"},
]

And than, the previous chat template will produce the following string:

        
        
        
    
<|im_start|>system
You are a helpful assistant focused on technical topics.<|im_end|>
<|im_start|>user
Can you explain what a chat template is?<|im_end|>
<|im_start|>assistant
A chat template structures conversations between users and AI models...<|im_end|>
<|im_start|>user
How do I use it ?<|im_end|>

Study Tools and Action

What are AI Tools?

A Tool is a function given to the LLM. This function should fulfill a clear objective. Here are common tools:

A Tool should contain:
- A textual description of what the function does.
- A Callable (something to perform an action).
- Arguments with typings.
- (Optional) Outputs with typings.

You can use format like following to define a tools:

        
@tool
def calculator(a: int, b: int) -> int:
    """Multiply two integers."""
    return a * b

print(calculator.to_string())

Note the @tool decorator before the function definition. so we can write like this:

Tool Name: calculator, Description: Multiply two integers., Arguments: a: int, b: int, Outputs: int

Generic Tool Implication

We create a generic Tool class that we can reuse whenever we need to use a tool.

        
        
        
    
from typing import Callable


class Tool:
    """
    A class representing a reusable piece of code (Tool).

    Attributes:
        name (str): Name of the tool.
        description (str): A textual description of what the tool does.
        func (callable): The function this tool wraps.
        arguments (list): A list of arguments.
        outputs (str or list): The return type(s) of the wrapped function.
    """
    def __init__(self,
                 name: str,
                 description: str,
                 func: Callable,
                 arguments: list,
                 outputs: str):
        self.name = name
        self.description = description
        self.func = func
        self.arguments = arguments
        self.outputs = outputs

    def to_string(self) -> str:
        """
        Return a string representation of the tool,
        including its name, description, arguments, and outputs.
        """
        args_str = ", ".join([
            f"{arg_name}: {arg_type}" for arg_name, arg_type in self.arguments
        ])

        return (
            f"Tool Name: {self.name},"
            f" Description: {self.description},"
            f" Arguments: {args_str},"
            f" Outputs: {self.outputs}"
        )

    def __call__(self, *args, **kwargs):
        """
        Invoke the underlying function (callable) with provided arguments.
        """
        return self.func(*args, **kwargs)

We could create a Tool with this class using code like the following:

        
        
        
    
calculator_tool = Tool(
    "calculator",                   # name
    "Multiply two integers.",       # description
    calculator,                     # function to call
    [("a", "int"), ("b", "int")],   # inputs (names and types)
    "int",                          # output
)

Model Context Protocol(MCP)

As far as I’m concerned, the public always argue that MCP is a complex conception, but not The MCP is a protocol: a unified tool interface It is a protocol standardizes how application provides a tools to LLM, that’s it, no much complicated. MCP provides:

A growing list of pre-built integrations that your LLM can directly plug into
The flexibility to switch between LLM providers and vendors
Best practices for securing your data within your infrastructure