
Chat Dataset Format Cleaner
Convert local chat JSONL into clean prompt/completion examples with a reproducible split. Supports message arrays, direct pairs, marker cleanup, basic length filters, and JSON summaries. Requires Python 3.10+. Source files are read only.
#엔지니어링
평점
더 많은 평가가 필요합니다
판매
0
사용 방법
다운로드
Chat Dataset Format Cleaner
Prepare clean prompt/completion JSONL from local chat exports. The utility supports message arrays and direct pairs, removes common turn wrappers, uses the final assistant response as the target, and creates a reproducible train/validation split.
- Python standard library only
- Read-only source handling
- Configurable split ratio and seed
- Length filters and record limit
- Human-readable or JSON summaries
Run python3 scripts/chat_dataset_cleaner.py examples/conversations.jsonl --output-dir /tmp/chat-clean --seed 42.


