What problem does it solve? Sending structured data to LLMs consumes excessive tokens, and processing large JSON/YAML files with standard libraries causes memory exhaustion and slow performance. This Skill provides TOON encoding for 40-60% token reduction, high-performance serialization, and schema-based data validation. ## Core Features & Use Cases - TOON Encoding: Compress structured data with type markers (#, !, @, ~) to reduce LLM token usage by 40-60% with lossless round-trip decoding. - High-Performance JSON/YAML: Use orjson for 2-5x faster serialization and ijson for memory-efficient streaming of large files. - Data Validation & Schema Evolution: Validate data against schemas with type checking, custom rules, cross-field validation, and versioned migrations. - Intelligent Caching: Apply LRU eviction, memory-aware cache management, and tag-based invalidation for expensive operations. - Use Case: When building an API that feeds user records to an LLM, validate the records against a schema, encode them with TOON to fit the token budget, and decode the LLM's response back to standard dictionaries. ## Quick Start Ask the AI to encode a sample user dictionary with TOONEncoder and compare the token count against standard JSON output.
