Universal LLM Inference Provider

Greenfield single-tenant LLM inference router with:

• OpenAI-compatible inference API
• FastAPI backend
• SQLAlchemy/Alembic-ready persistence layer
• Redis-compatible quota and affinity abstractions
• React admin frontend scaffold

Layout

• backend/ FastAPI application, routing engine, admin API, tests
• frontend/ React/Vite admin UI scaffold

Backend quick start

cd backend
python -m venv .venv
.venv\Scripts\activate
pip install -e .[test]
uvicorn app.main:app --reload

Frontend quick start

cd frontend
npm install
npm run dev

Notes

• Default runtime uses in-memory repositories and counters for local development.
• Database and Redis settings are exposed so the app can be wired to Postgres and Redis in deployment.
• The routing engine is covered by tests and is isolated from transport/persistence details.