skills.rest
VelkaressiaBlutkrone
VelkaressiaBlutkroneCOMMUNITY
Explore all skills by this author →
GitHub

python-endpoint

Generates FastAPI LLM inference endpoints with RAG context and Ollama chat integration.

Updated Feb 26, 2026
Search Tags:#fastapi#llm#rag#sse#ollama#chromadb#endpoint generation
One-click install
npx skills add https://github.com/VelkaressiaBlutkrone/spring-python-llm-exam-mng --skill python-endpoint-velkaressiablutkrone
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: python-endpoint
Source: https://github.com/VelkaressiaBlutkrone/spring-python-llm-exam-mng/tree/main/.claude/skills/python-endpoint
Command: npx skills add https://github.com/VelkaressiaBlutkrone/spring-python-llm-exam-mng --skill python-endpoint-velkaressiablutkrone

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? Adding a new LLM inference endpoint to a Python FastAPI service requires repeating boilerplate for system prompts, RAG context retrieval, Ollama API calls, and Spring Boot integration. This Skill codifies the project's established patterns so new domain endpoints are generated consistently. ## Core Features & Use Cases - Endpoint Scaffolding: Generates synchronous and SSE streaming FastAPI endpoints following the project's /infer/{domain} pattern with Ollama Chat API calls. - System Prompt Design: Enforces bilingual (English + Korean) prompt rules, Korean-only output, and structured answer formats. - Hybrid RAG Context: Builds context services combining ChromaDB vector search with MySQL FULLTEXT fallback, plus indexing scripts. - Use Case: When adding a new consultation domain (e.g., pharmacy Q&A) to the medical LLM system, generate the endpoint, context service, Spring WebClient integration, and tests in one consistent pass. ## Quick Start Ask the assistant to add a new FastAPI LLM inference endpoint for a specific domain using the python-endpoint pattern.

Frequently Asked Questions about python-endpoint

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I add a new FastAPI LLM inference endpoint?▼

Create a POST route under /infer/{domain} that builds a system prompt, retrieves RAG context, and calls the Ollama /api/chat endpoint with httpx. Return the generated text through a Pydantic response schema and log request and response lengths.

How to stream LLM responses with FastAPI and SSE?▼

Set stream=True in the Ollama chat payload and return a StreamingResponse with media_type text/event-stream. Use an async generator that yields tokens as they arrive, filtering special tokens and unwanted CJK characters in real time.

How do I combine ChromaDB vector search with MySQL FULLTEXT?▼

Query the ChromaDB collection first with the user query and collect the top-k documents. If vector search returns nothing or fails, fall back to a MySQL MATCH AGAINST boolean-mode query, then join results into a single context string.

Why does my LLM respond in Chinese instead of Korean?▼

Models like qwen2.5 may drift into Chinese without explicit constraints. Add bilingual instructions stating the model must respond only in Korean, explicitly forbid CJK characters, and filter special tokens from the output.

How does Spring Boot call the Python LLM endpoint?▼

Add a service method that uses an injected WebClient to POST to /infer/{domain} on the Python server. Apply the same error mapping pattern as existing services, handling timeouts and unavailable-service exceptions.

Related Skills

Explore More Software Engineering Skills→
kernex-devkernex-dev
1

ai-engineer

Design and implement RAG pipelines and LLM integrations for production AI systems.

Official
Advanced
PedroSantiagoDevPedroSantiagoDev

planning-java

Plan Java/Spring Boot project architecture and technology decisions.

Community
Basic
codingshotcodingshot
3

api-integrations-sunnahsleep

Integrate external APIs for prayer times, geolocation, and content delivery.

Community
Advanced
pagopapagopa
1

TechAIProjectJava

Generate and modify Java project components with JavaDoc and BDD-style unit tests.

Official
Intermediate