claw-semantic-sim

What problem does it solve?

Biomedical research literature is often siloed and unevenly connected, making it difficult to quantify which diseases are isolated from cross-disciplinary knowledge and where research transfer is most likely to succeed. This Skill provides robust, reproducible metrics that reveal semantic isolation and knowledge transfer potential across the global biomedical corpus so researchers and funders can prioritise interventions and collaborations.

Core Features & Use Cases

• PubMed retrieval with quality filtering: checkpointed fetching of abstracts (2000–2025) with rate-limit handling and minimum coverage thresholds.
• PubMedBERT embeddings: batched 768-dimensional embeddings with Apple MPS acceleration and CPU fallback, stored in compressed HDF5 with checksums.
• Semantic equity metrics: computes Semantic Isolation Index (SII), Knowledge Transfer Potential (KTP), Research Clustering Coefficient (RCC), and temporal semantic drift.
• Analysis & visualization: generates publication-quality 4-panel figures, ranking tables, statistical tests (Welch's t-test, Cohen's d), and a reproducibility bundle.
• Use Case: rapidly identify neglected tropical diseases (NTDs) that occupy knowledge silos to guide research funding, collaboration, and literature synthesis.

Quick Start

Run the demo to generate the full four-panel report using precomputed embeddings by executing python semantic_sim.py --demo --output demo_report on your machine.