skills.rest
standardbeagle
standardbeagleOFFICIAL
Explore all skills by this author →
WebsiteGitHub

lci-benchmarks-evaluation

Maps lci benchmark surfaces and runs unbiased comparisons against competing code search tools.

Updated Apr 27, 2026
Search Tags:#benchmarking#code search#comparison#performance testing#evaluation methodology#regression gate#lci
One-click install
npx skills add https://github.com/standardbeagle/lci-cpp --skill lci-benchmarks-evaluation-standardbeagle
Or copy as Structured Prompt for Agent▼
Please help me install this Agent Skill.
Skill: lci-benchmarks-evaluation
Source: https://github.com/standardbeagle/lci-cpp/tree/main/.agents/skills/lci-benchmarks-evaluation
Command: npx skills add https://github.com/standardbeagle/lci-cpp --skill lci-benchmarks-evaluation-standardbeagle

SYSTEM DOCUMENTATION & REQUIREMENTS

What problem does it solve? It locates every place lci performance and correctness are measured (perf gate, goldens, real-project tests, repo-qa banks) and provides a rigorous procedure for honestly comparing lci against tools like ripgrep, Zoekt, Sourcegraph, ctags, LSP servers, Aider, Cursor, Serena, or GitHub code search. ## Core Features & Use Cases - Measurement surface map: Indexes the perf gate (bench_gate.py), integration goldens, real-project latency tests, and the benchmarks/repo-qa/ agent-level harness with exact file and line references. - Comparison methodology: A 10-step procedure covering manifest design, pre-registered predictions, independent oracles, disjoint arms, and evidence auditing. - Claim verification: Tables linking README claims (sub-millisecond search, context reduction) to the benchmarks that back or fail to back them. - Use Case: When asked whether lci beats ripgrep on literal search latency, use this Skill to pick the axis, build an oracle with rg --fixed-strings, run interleaved arms, and report with the audit template. ## Quick Start Use the lci-benchmarks-evaluation skill to find where lci's search latency is measured and design a fair comparison against ripgrep.

Frequently Asked Questions about lci-benchmarks-evaluation

High-intent search queries and answers about installing and using this skill.

FAQPage Schema
How do I run the lci performance benchmark gate?▼

Build the lci_benchmarks target, run it with --benchmark_filter=-RealProject and JSON output, then pass the result to scripts/bench_gate.py with the baseline at tests/benchmarks/baseline/linux-x64.json. The default regression threshold is 1.5x and the gate refuses judgment above load average 2.0.

How do I compare lci against ripgrep or Zoekt?▼

Pick one axis such as literal search latency, use rg --fixed-strings as the independent oracle, run both tools warm on the same pinned corpus with interleaved best-of-K timing, and report min and median. Follow the 10-step comparison procedure including pre-registered predictions.

What is the correctness oracle for lci search results?▼

Integration goldens under tests/integration/goldens/ are the sole correctness oracle since Go parity was removed. Set LCI_UPDATE_GOLDENS=1 to re-pin them, and use tests/search_rg_differential_test.cpp for literal search validation against rg.

Why does the lci perf gate report false regressions?▼

Running the gate under system load fakes regressions, with false positives up to 2.19x observed at load average 10 or higher. The gate demotes itself above load 2.0 and CI snapshots pre-run load to guard against this.

Does lci actually reduce agent token usage versus grep?▼

No committed measurement supports a specific context-reduction percentage. Repo-qa tier results instead show lci costing 1.5-2.5x agent tokens, with wins limited to completion-rate reliability on weak models at tier 3 difficulty.

Why do lci golden tests fail in a git worktree?▼

Goldens embed absolute repo-root paths such as http_reindex, so a worktree checkout fails them spuriously. Run golden tests from the primary checkout or account for the path difference before diagnosing a real regression.

Related Skills

Explore More Software Engineering Skills→
alibabaalibaba
2.2k

compile

Builds LoongCollector C++ and Go components with Make, CMake, and Docker.

Official
Intermediate
ag3nt3154ag3nt3154
1

pms-ingest

Classify raw files into wiki source directories and update wiki documents.

Community
Intermediate
mrrtsmrrts
1

run-experiment

Design and execute commit-bound natural experiments on live message corpora.

Community
Advanced
lazyFrogLOLlazyFrogLOL
127

query-optimize

Optimize slow SQL queries with plan analysis and index recommendations.

Community
Intermediate