S06-超长文本读取

Split the file into batches using line-number ranges sized to about 60% of the context window, then read each batch sequentially. Ensure every batch starts and ends on a complete record boundary so no record is cut in half.

Use each format's record delimiter as the chunk boundary: ER lines for WoS plain text, blank lines for RIS, and @ entry markers for BibTeX. After each read, verify the last line closes a record and extend to the next delimiter if not.

Batch size depends on the operation: use 60-80 records for counting and field extraction, 20-40 for semantic judgment, and 10-20 for complex reasoning like noise pattern detection. Cap total read content at 60% of the context window.

Yes. Record the last completed batch number and line position, verify previously processed records were saved, then continue from the next batch without reprocessing. A final check confirms processed count equals the expected total.

Chunked reading is unnecessary when the entire file fits within the context window, typically under 2MB or fewer than 1000 records. It is designed specifically for full-scan operations on oversized files.