​
I. Introduction

The gpt-oss series is an open-source model first released by OpenAI in August 2025.
gpt-oss:20b (fits systems with ~16 GB memory) gpt-oss:120b (designed for ≥ 60 GB memory)
You can run it locally with Ollama. No cloud calls. Data stays on your machine, which helps with privacy and latency. Dify is an open-source platform for building AI Agents and Workflows. This guide shows how to run gpt-oss with Ollama and plug it into Dify for a private, high-performance setup.

​
II. Environment Setup

​
Step 1: Run gpt-oss with Ollama

  1. Install Ollama
Download and install for macOS, Windows, or Linux. If anything is unclear, see the official docs.
  1. Pull a model
# Recommended for most dev machines
ollama pull gpt-oss:20b

# For large GPUs or multi-GPU hosts
ollama pull gpt-oss:120b
Both models are pre‑quantized in MXFP4 , suitable for local deployment.
  1. Start Ollama
The default endpoint is http://localhost:11434.

​
Step 2: Install Dify locally

See the Dify docs for full instructions. Or follow the quick tutorial below.

​
Prerequisites

Install Docker and make sure Docker Engine is running. 1

​
Install steps

git clone https://github.com/langgenius/Dify.git
cd Dify/docker
cp .env.example .env
docker compose up -d
2 Open your local Dify instance and complete the initial setup. 3

​
III. Add the model and test chat

  1. In Settings → Model Providers → Ollama , click Add Ollama model type .
4
  1. Set Base URL to http://localhost:11434, select Model name gpt-oss:20b, and fill in the required fields.
5
  1. Create a blank template
6
  1. Select the app or workflow type you want to build.
7

​
IV. Verification and Usage

  • On Dify’s model testing page, send a prompt and check that the response looks correct.
  • In a workflow, add an LLM node, select gpt-oss:20b, and connect the nodes end to end.
8 9

​
V. Appendix — FAQ and tips

Model Pull Is Slow
  • Configure a Docker proxy or use an image mirror to speed up downloads.
Insufficient GPU Memory
  • Use gpt-oss:20b. You can enable CPU offloading, but responses will be slower.*
Port Access Issues
  • Check firewall rules, port bindings, and Docker network settings to ensure connectivity.