Unlocking the Power of AI: Ollama LLM with Structured Output
🦙 Ollama introduction
Ollama is a powerful platform designed to make it easy for developers to run large language models (LLMs) directly on their local computers, without needing a big server or expensive hardware like high-end GPUs.
With Ollama, you can get up and running with models such as Llama 3.3, Phi 3, Mistral, Gemma 2, and more, all without the need for specialized cloud infrastructure.
In short, Ollama democratizes access to cutting-edge AI models, making them accessible on personal computers and empowering developers to build AI solutions without the burden of expensive server infrastructure.
🔑 Why Use JSON Schema Constraints in Ollama?
Ollama now supports to constrain the output to a json schema. https://ollama.com/blog/structured-outputs
Use cases for structured outputs include:
- Parsing data from documents
- Extracting data from images
- Structuring all language model responses
- More reliability and consistency than JSON mode
🚀 Getting Started
1. Install Ollama
Download Ollama from https://ollama.com/download. Choose the appropriate version for your platform and complete the installation.
2. Verify Installation
After installation, confirm the version by running:
ollama --version
expected output format:
ollama version is 0.5.1
3. Start Ollama Server
Run the following command to start the server:
ollama serve
By default, the server runs on port 11434. You can verify this by visiting http://localhost:11434.
4. Download the Ollama Model
I recommend the llama3.2 model for its compact size (2.0 GB) and impressive performance.
It features 3 billion parameters and outperforms many larger models in industry benchmarks.
Download the model with:
ollama run llama3.2
Verify the download:
ollama list
Expected output:
NAME ID SIZE MODIFIED
llama3.2:latest a80c4f17acd5 2.0 GB n weeks ago
▶️ Run using CURL
Run using curl directly to Ollama server http://localhost:11434.
There is no need to code, there is no setup, no need to install anything (*make sure you have curl installed).
Use Case 1: Extract Country Data
Input Prompt: "Tell me about Canada."
Schema
{
"type": "object",
"properties": {
"name": {
"type": "string"
},
"capital": {
"type": "string"
},
"languages": {
"type": "array",
"items": {
"type": "string"
}
}
},
"required": [
"name",
"capital",
"languages"
]
}
Curl Request
json schema above included in the request in field format.
Run the following command in the terminal:
curl --location 'http://127.0.0.1:11434/api/chat' \
--header 'Content-Type: application/json' \
--data '{
"model": "llama3.2",
"messages": [{"role": "user", "content": "Tell me about Canada."}],
"stream": false,
"format": {
"type": "object",
"properties": {
"name": { "type": "string" },
"capital": { "type": "string" },
"languages": { "type": "array", "items": { "type": "string" } }
},
"required": ["name", "capital", "languages"]
}
}'
Response
{
"model": "llama3.2",
"created_at": "2024-12-12T07:47:22.184905Z",
"message": {
"role": "assistant",
"content": "{ \"capital\": \"Ottawa\", \"languages\": [\"English\", \"French\"], \"name\":\"Canada\" }"
},
"done_reason": "stop",
"done": true,
"total_duration": 1232600000,
"load_duration": 33616458,
"prompt_eval_count": 30,
"prompt_eval_duration": 495000000,
"eval_count": 29,
"eval_duration": 700000000
}
see field message.content using the same json schema as above
{
"capital": "Ottawa",
"languages": [
"English",
"French"
],
"name": "Canada"
}
Use Case 2: Extract Pets Data
Prompt: "I have two pets. A cat named Luna who is 5 years old and loves playing with yarn. She has grey fur. I also have a 2-year-old black cat named Loki who loves tennis balls."
Schema
{
"$defs": {
"Pet": {
"properties": {
"age": {
"type": "integer"
},
"animal": {
"type": "string"
},
"color": {
"anyOf": [
{
"const": "black"
},
{
"const": "grey"
}
]
},
"favorite_toy": {
"anyOf": [
{
"const": "yarn"
},
{
"const": "tennis balls"
}
]
},
"name": {
"type": "string"
}
},
"required": [
"name",
"animal",
"age",
"color",
"favorite_toy"
],
"type": "object"
}
},
"properties": {
"pets": {
"type": "array",
"items": {
"$ref": "#/$defs/Pet"
}
}
},
"required": [
"pets"
],
"type": "object"
}
Curl Request
curl --location 'http://127.0.0.1:11434/api/chat' \
--header 'Content-Type: application/json' \
--data '{
"model": "llama3.2",
"messages": [{
"role": "user",
"content": "I have two pets. A cat named Luna who is 5 years old and loves playing with yarn. She has grey fur. I also have a 2-year-old black cat named Loki who loves tennis balls."
}],
"stream": false,
"format": {
"$defs": {
"Pet": {
"properties": {
"age": { "type": "integer" },
"animal": { "type": "string" },
"color": { "anyOf": [
{ "const": "black" },
{ "const": "grey" }
] },
"favorite_toy": { "anyOf": [
{ "const": "yarn" },
{ "const": "tennis balls" }
] },
"name": { "type": "string" }
},
"required": ["name", "animal", "age", "color", "favorite_toy"],
"type": "object"
}
},
"properties": {
"pets": {
"type": "array",
"items": { "$ref": "#/$defs/Pet" }
}
},
"required": ["pets"],
"type": "object"
}
}'
Response
{
"model": "llama3.2",
"created_at": "2024-12-12T08:32:37.316804Z",
"message": {
"role": "assistant",
"content": "{ \"pets\": [ { \"age\": 5, \"animal\": \"cat\", \"color\": \"grey\", \"favorite_toy\": \"yarn\" , \"name\": \"Luna\"}, { \"age\": 2, \"animal\": \"cat\", \"color\": \"black\", \"favorite_toy\": \"tennis balls\" , \"name\": \"Loki\"}] }"
},
"done_reason": "stop",
"done": true,
"total_duration": 7159731958,
"load_duration": 829545916,
"prompt_eval_count": 68,
"prompt_eval_duration": 4620000000,
"eval_count": 82,
"eval_duration": 1697000000
}
see field message.content using the same json schema as above
{
"pets": [
{
"age": 5,
"animal": "cat",
"color": "grey",
"favorite_toy": "yarn",
"name": "Luna"
},
{
"age": 2,
"animal": "cat",
"color": "black",
"favorite_toy": "tennis balls",
"name": "Loki"
}
]
}
▶️ Run using demo python api
Recommend to use python 3.10.* with venv
1. Create venv
Clone this repository
git clone https://github.com/harryosmar/ollama-demo-python.git
Run the following command in the terminal:
cd ollama-demo-python
python3 -m venv venv
2. Activate venv
Run the following command in the terminal:
source ./venv/bin/activate
3. Install requirements
Run the following command in the terminal:
pip install -r requirements.txt
4. Run app
Run the following command in the terminal:
flask --app main run
Start api server in http://127.0.0.1:5000
5. Swagger endpoints
swagger access link http://127.0.0.1:5000/apidocs/
🔥 Conclusion
By leveraging JSON schema constraints, Ollama ensures structured, reliable, and predictable outputs from LLMs. Whether you’re parsing documents, analyzing images, or building robust APIs, this feature delivers the consistency and accuracy you need.
🏅 Credits
Give credits to Matt Williams with his youtube channel https://www.youtube.com/@technovangelist
He was part of the founding Ollama team. Don't forget to check out & subscribe to his channel, help him to reach 1 million subscribers.
I learned a lot from his videos, thank you Matt, hats off to you! 👏