sayt2

Index and search Python dictionaries with ngram substring and BM25 queries.

Updated Apr 25, 2022
One-click install
npx skills add https://github.com/MacHu-GWU/afwf_github-project --skill sayt2
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: sayt2
Source: https://github.com/MacHu-GWU/afwf_github-project/tree/main/.claude/skills/sayt2
Command: npx skills add https://github.com/MacHu-GWU/afwf_github-project --skill sayt2

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve?

sayt2 provides a Python library to build a full-text search index from a list of dictionaries and query it with substring matching (ngram), BM25 full-text search, fuzzy search, range queries, sorting, and more -- all through a single DataSet object.

Core Features & Use Cases

  • Full-text search with BM25, substring search via ngram, and fuzzy search across stored fields.
  • Sorting, range queries, and caching via diskcache, with validation via pydantic and a Rust-backed Tantivy engine.
  • Use cases include building search-as-you-type interfaces and fast data retrieval for Python dictionaries in various applications.

Quick Start

Create a DataSet with a fields definition and a downloader, then call search to see results.

Frequently Asked Questions about sayt2

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I implement search-as-you-type in Python using dictionaries?▼

To build search-as-you-type in Python, load a list of dictionaries into a DataSet object and query using ngram-based substring matching and BM25 ranking to retrieve fast, ranked matches.

What is the best way to do full-text search and substring matching on Python dictionaries?▼

Full-text search and substring matching on Python dictionaries is best handled by indexing the data with a Rust-backed Tantivy engine, enabling BM25 ranking and ngram fields for partial text matches.

How does ngram substring search work with BM25 ranking?▼

Ngram substring search works by breaking text into character sequences for partial matching, while BM25 calculates relevance scores to rank the overall full-text search results returned by the query.

Can I use Tantivy with Python for fuzzy search and range queries?▼

Yes, you can use Tantivy with Python through specific bindings to execute fuzzy search and range queries, while leveraging pydantic validation and diskcache-based caching for the search results.

Do I need external dependencies to run a local search index with caching?▼

Running a local search index with caching requires no external dependencies, as the environment integrates a Rust-backed Tantivy engine and diskcache to handle indexing, pydantic validation, and storage automatically.