Using AI SDKs

There are two ways to talk to a model from Python: call the HTTP API yourself, or use a library. This page covers both, and when each is the right choice.

Everything here is Python. Frameworks get named so you recognise them later, but this tutorial does not teach them — that is what the classes and the missions are for.

The layers

Each layer sits on the one above. None is more correct; they trade control for convenience.

Raw HTTP

Worth doing once, because it shows there is no magic underneath.

import os
import httpx

response = httpx.post(
    "https://api.deepseek.com/chat/completions",
    headers={"Authorization": f"Bearer {os.getenv('DEEPSEEK_API_KEY')}"},
    json={
        "model": "deepseek-chat",
        "messages": [
            {"role": "system", "content": "You are a triage assistant."},
            {"role": "user", "content": "The database is unreachable."},
        ],
        "temperature": 0.2,
    },
    timeout=30,
)
response.raise_for_status()

data = response.json()
print(data["choices"][0]["message"]["content"])

That is a complete model call. A list of message dictionaries goes in; a nested dictionary comes back.

Nearly every provider follows this same shape, which is why one mental model covers most of them.

The message roles

The whole list is sent on every request. That list is the conversation memory — the model retains nothing between calls.

The settings that matter

• temperature — randomness. Low, around 0 to 0.3, for classification and extraction. Higher for drafting prose. For agent work you usually want it low.
• max_tokens — a cap on the reply length, and on what you pay for it.
• model — which model. A cheap fast one is often enough for routing and classification.

A provider SDK

from openai import OpenAI

client = OpenAI(
    api_key=os.getenv("DEEPSEEK_API_KEY"),
    base_url="https://api.deepseek.com",
)

response = client.chat.completions.create(
    model="deepseek-chat",
    messages=[{"role": "user", "content": "Hello"}],
)

print(response.choices[0].message.content)

The same request, with retries, timeouts and typed objects handled for you. Note the dot access — response.choices[0].message.content — rather than dictionary keys.

Many providers copy OpenAI's interface, so the same client often works against a different service by changing base_url. That is why you see the OpenAI library pointed at other providers.

Streaming

For anything a person is watching, stream it.

stream = client.chat.completions.create(
    model="deepseek-chat",
    messages=[{"role": "user", "content": "Explain retries"}],
    stream=True,
)

for chunk in stream:
    piece = chunk.choices[0].delta.content
    if piece:
        print(piece, end="", flush=True)

stream=True returns a generator instead of a finished reply. end="" stops a newline per chunk and flush=True shows it immediately.

Use streaming when a human is waiting. Use a normal call when the result feeds the next step of your program, because it is far easier to work with.

Agent frameworks

Once you want tools, memory, branching or retries, writing it yourself stops being worthwhile.

from langchain_core.tools import tool

@tool
def get_ticket(ticket_id: int) -> str:
    """Fetch an incident ticket by its id."""
    return db.fetch(ticket_id)

What a framework gives you:

• schema generation from your type hints and docstrings
• the agent loop, written and tested
• one interface across providers
• streaming, retries and tracing as options
• branching, cycles and state, in the graph libraries

The names you will meet:

They differ in style more than in substance, and all of them rest on the Python in this tutorial. Decorators, Pydantic models, TypedDict, functions passed as values, generators for streaming — that is what their source is made of.

Structured output

The feature that most changes how you build. Instead of parsing text, you declare the shape you want.

from pydantic import BaseModel, Field

class Triage(BaseModel):
    severity: str = Field(description="One of P1, P2, P3")
    owner: str = Field(description="Team that should take this")
    summary: str

structured = llm.with_structured_output(Triage)
result = structured.invoke("Database is down, users cannot log in")

print(result.severity)

result is a Triage object. No regex, no json.loads, no handling the day it wraps the JSON in a code fence.

If you take one thing from this page, take this. Most flaky agent code is text parsing that should have been a Pydantic model.

Choosing a layer

• Raw HTTP — learning, or a single call with no dependencies
• Provider SDK — one provider, straightforward calls, streaming
• Framework — tools, multi-step work, branching, or you want to switch providers later

Start at the lowest layer that does the job. A great deal of production work is one well-aimed model call, not an agent.

What to check before choosing a library

• does it support async? Without it you cannot run tool calls concurrently
• does it support streaming?
• does it support tool calling in the shape the provider expects?
• does it give structured output?
• is it maintained? This ecosystem moves fast and abandoned wrappers break

Common mistakes

• Reaching for a framework for a single model call.
• Parsing text by hand where structured output would have guaranteed the shape.
• No timeout on a model call.
• A high temperature on a classification task, then wondering why answers vary.
• Forgetting the whole message list is re-sent, so cost grows with conversation length.
• Copying a tutorial's model name without checking it still exists.

Practise this

• Make one raw HTTP call to a model with httpx and print the reply.
• Make the same call again with temperature at 0 and at 1, twice each, and compare.
• Stream a response and print it token by token.
• Define a Pydantic model for something you would want extracted, and write the field descriptions as if the model will read them — because it will.