RafineAI

Open source · Self-hosted · MIT licensed

AI governance and an LLM gateway you run yourself

RafineAI sits between your apps and LLM providers (OpenAI, Anthropic, Gemini…). It authenticates traffic, applies security policy, records a full audit trail of every interaction, and ships a built-in chat panel — all from a single docker compose up.

Get started View on GitHub
  • Go gateway hot path — HMAC keys, zero DB round-trip
  • Async, batched audit logging that never blocks requests
  • Own your data — nothing leaves your infrastructure
RafineAI chat panel screenshot

Everything a self-hosted LLM gateway needs

Core proxying, policy, and audit — plus an enterprise feature set for teams that need more.

🔐

Zero-network auth

Gateway API keys are HMAC-SHA256 signed and verified with CPU math only — no database round-trip on the hot path.

🛡️

Policy & response masking

Enforce security policy on every request, and mask sensitive data (TCKN, IBAN, cards, API keys) in the model's reply too — even mid-stream.

📜

Full audit trail

Every interaction is logged asynchronously to a buffered, batch-written channel, so audit logging never blocks a request.

🧑‍🤝‍🧑

Teams & limits

Group users into teams with RPM and daily token limits, and per-provider access — enforced with the most restrictive limit in effect.

📚

RAG / NotebookLM

Index documents with pgvector embeddings and ask grounded questions; answers cite the source documents they came from.

📊

Usage dashboard

Requests, tokens, estimated cost, p95 latency, and error rate — with daily charts and a Prometheus /metrics endpoint.

Quick start

One command gets you a running stack with generated secrets and a seeded owner account.

1Clone & install

git clone https://github.com/RafineAI/rafineai-self-hosted.git
cd rafineai-self-hosted
./install.sh

install.sh checks dependencies, generates strong secrets, writes .env, starts the stack, waits for health, seeds the default owner account, and prints your login credentials.

2Or run it manually

cp .env.example .env      # edit secrets + provider keys
docker compose up -d      # pulls pre-built images

The panel is then available at http://localhost.

3Build from source (dev)

cp .env.example .env
docker compose -f docker-compose.dev.yml up --build

Prerequisites: Docker & Docker Compose v2, and a machine with ≥ 2 GB RAM.

Panel & admin guide

After install you get an owner account (owner@rafine.local by default). Here's what to do next.

  1. Create accounts. Add admin and user accounts from User Management.
  2. Configure LLM providers. In LLM Management, add a shared API key or set up per-user OAuth2.
  3. Invite users. Hand out the panel URL — users log in and start chatting immediately.
RoleCapabilities
ownerEverything. Single account, cannot be deleted.
adminManage users, providers, and policies; view all audit logs.
userChat; view only their own conversations.

See docs/FLOW.md for a full screenshot walkthrough — login, chat, providers, users, policy rules, alerts, and audit logs — and docs/DEPLOYMENT.md for building and pushing your own versioned images (GHCR, GCP Artifact Registry, or a private Docker Hub repo).