5. Functions in depth

A function is an object created at runtime by def. It carries its compiled code, its default values and a reference to the scope where it was defined. Callbacks, closures and decorators all follow from that.

Functions are values

import math

def relu(x):
    return max(0.0, x)

activations = {"relu": relu, "tanh": math.tanh}
act = activations[config["activation"]]          # choose behavior from config
ranked = sorted(items, key=lambda r: r["score"], reverse=True)

Functions can be inspected like any object: relu.__name__, relu.__defaults__, relu.__code__.

Scopes: the LEGB rule

Python looks a name up in four places, in order: Local, Enclosing function, Global (the module), Builtins.

• Assigning to a name anywhere in a function makes it local for the whole function. Reading it before the assignment raises UnboundLocalError.
• global x and nonlocal x let you rebind outer names. You rarely need them; reaching for them is usually a design smell.
• Shadowing builtins (list = [...], id = 5, input = ...) breaks later code in that scope in confusing ways.

count = 0
def inc():
    count += 1      # UnboundLocalError: assignment makes count local

Closures

An inner function keeps references to the enclosing function’s variables, even after the outer function has returned.

def make_warmup(base_lr, warmup_steps):
    def lr_at(step):
        return base_lr * min(1.0, step / warmup_steps)
    return lr_at

schedule = make_warmup(3e-4, 1000)
schedule(500)       # 0.00015

The late-binding trap: a closure captures the variable, not its value at creation time.

fns = [lambda: i for i in range(3)]
[f() for f in fns]          # [2, 2, 2]
fns = [lambda i=i: i for i in range(3)]
[f() for f in fns]          # [0, 1, 2]: the default freezes the current value

The full parameter grammar

def f(pos_only, /, normal, *args, kw_only, **kwargs): ...

• Before /: positional-only parameters.
• *args: extra positional arguments, collected into a tuple.
• After * or *args: keyword-only parameters. They force readable calls such as train(model, lr=3e-4, epochs=10).
• **kwargs: extra keyword arguments, collected into a dict. Frameworks forward these through several layers, which is why a parameter’s documentation can live three classes away from the call you made.
• At the call site, f(*xs, **opts) spreads a sequence and a dict into arguments.

Lambdas

A lambda is a one-expression anonymous function, with no statements allowed. Use it for short key= functions and give anything longer a name.

Decorators

A decorator is a function that takes a function and returns a replacement. Writing @d above def f means exactly f = d(f).

import functools, time

def timed(fn):
    @functools.wraps(fn)                  # keep the name, docstring, signature
    def wrapper(*args, **kwargs):
        t0 = time.perf_counter()
        try:
            return fn(*args, **kwargs)
        finally:
            print(f"{fn.__name__}: {time.perf_counter() - t0:.3f}s")
    return wrapper

@timed
def embed(texts): ...

A decorator that takes arguments is one more layer: a function that returns a decorator.

def retry(times):
    def decorator(fn):
        @functools.wraps(fn)
        def wrapper(*args, **kwargs):
            for attempt in range(times):
                try:
                    return fn(*args, **kwargs)
                except ConnectionError:
                    if attempt == times - 1:
                        raise
        return wrapper
    return decorator

@retry(times=3)          # retry(3) runs first and returns the real decorator
def call_api(): ...

Where you will meet decorators in AI code: @torch.no_grad(), @torch.compile, @app.post("/chat") in FastAPI (which registers a route), @pytest.fixture, @dataclass, @property, and tool-registration decorators in agent frameworks. Those last ones read the function’s signature and docstring to build the JSON schema the model sees.

functools essentials

• partial(fn, pad_id=0) pre-fills arguments, for example to configure a collate function.
• cache / lru_cache memoize pure functions with hashable arguments. Avoid them on methods of large objects, because the cache keeps self alive.
• wraps belongs in every decorator you write.

Signatures and docstrings are data

inspect.signature(fn) reads a function’s parameters at runtime. Tool-calling libraries combine it with type hints and the docstring to generate the tool schema. In other words, your docstring literally becomes part of the prompt.

Questions to ask:

• What does this decorator replace the function with?
• Is this closure capturing a variable that changes later?
• Where do these **kwargs finally end up?