Summary
Text::CSV_XS versions before 1.65 may truncate the end of a CSV record when a UTF-8-flagged field is followed by invalid UTF-8 bytes during output with print.
The missing bytes can include the final field data, closing quote, and line ending, potentially causing records to merge or causing readers to interpret subsequent records as part of a quoted field.
Text::CSV_XS versions before 1.65 may truncate the end of a CSV record when a UTF-8-flagged field is followed by invalid UTF-8 bytes during output with print.
The missing bytes can include the final field data, closing quote, and line ending, potentially causing records to merge or causing readers to interpret subsequent records as part of a quoted field.
Affected context
The issue occurs in Text::CSV_XS Perl code that writes records through print when all of the following conditions may apply:
- A record contains a field flagged as UTF-8.
- Invalid UTF-8 bytes occur later in the record, particularly near the final field.
- Output is buffered and processed across multiple flushes.
The combine operation is not affected; the problematic behavior is associated with record serialization through print.
The issue occurs in Text::CSV_XS Perl code that writes records through print when all of the following conditions may apply:
- A record contains a field flagged as UTF-8.
- Invalid UTF-8 bytes occur later in the record, particularly near the final field.
- Output is buffered and processed across multiple flushes.
The combine operation is not affected; the problematic behavior is associated with record serialization through print.
Exploit preconditions
- Attack surface: Applications that serialize CSV records using Text::CSV_XS
print - Required access: Ability to influence CSV field content or the data being exported
- Trigger point: Final record flushing when UTF-8-flagged content precedes invalid UTF-8 bytes
- Why exploitation is possible: The serializer retains trailing bytes while attempting to handle a character split across flushes but does not emit those retained bytes after the record's final flush
- Attack surface: Applications that serialize CSV records using Text::CSV_XS
print - Required access: Ability to influence CSV field content or the data being exported
- Trigger point: Final record flushing when UTF-8-flagged content precedes invalid UTF-8 bytes
- Why exploitation is possible: The serializer retains trailing bytes while attempting to handle a character split across flushes but does not emit those retained bytes after the record's final flush
Why this happens
The underlying bug is an incomplete output-buffering and encoding-handling path. When a UTF-8-flagged field is present, the serializer preserves trailing bytes that might form part of a split character across flush boundaries. If those bytes are invalid UTF-8, the final flush can complete without writing them.
This improperly neutralizes or transforms output data and delimiters, mapping to CWE-116: Improper Output Neutralization for Output Context. The resulting serialized stream may omit field content, quote characters, and record terminators, even though the print operation reports success.
The underlying bug is an incomplete output-buffering and encoding-handling path. When a UTF-8-flagged field is present, the serializer preserves trailing bytes that might form part of a split character across flush boundaries. If those bytes are invalid UTF-8, the final flush can complete without writing them.
This improperly neutralizes or transforms output data and delimiters, mapping to CWE-116: Improper Output Neutralization for Output Context. The resulting serialized stream may omit field content, quote characters, and record terminators, even though the print operation reports success.
Potential impact
- CSV field data may be silently truncated
- Closing quotes and line endings may be omitted
- Multiple logical records may be joined onto one physical line
- A CSV reader may treat subsequent records as part of an earlier quoted field
- Exported data may become inconsistent, misleading, or difficult to process reliably
- Downstream ingestion, reporting, or validation workflows could be disrupted
- CSV field data may be silently truncated
- Closing quotes and line endings may be omitted
- Multiple logical records may be joined onto one physical line
- A CSV reader may treat subsequent records as part of an earlier quoted field
- Exported data may become inconsistent, misleading, or difficult to process reliably
- Downstream ingestion, reporting, or validation workflows could be disrupted
General mitigation guidance
Mitigation generally focuses on ensuring that CSV output preserves field data, quoting, delimiters, and record boundaries across encoding errors and buffer flushes.
Common defensive patterns include:
- Use a maintained Text::CSV_XS release containing a correction for the serialization behavior.
- Normalize or reject invalid byte sequences before CSV serialization where the application permits.
- Establish an explicit text-encoding policy at input boundaries instead of mixing flagged text with arbitrary byte strings.
- Validate generated CSV by checking row counts, record terminators, quoting balance, and round-trip parsing.
- Treat serialization failures and encoding anomalies as errors rather than relying solely on a successful return value.
- Apply least privilege to export processes and limit the downstream authority of generated files.
Mitigation generally focuses on ensuring that CSV output preserves field data, quoting, delimiters, and record boundaries across encoding errors and buffer flushes.
Common defensive patterns include:
- Use a maintained Text::CSV_XS release containing a correction for the serialization behavior.
- Normalize or reject invalid byte sequences before CSV serialization where the application permits.
- Establish an explicit text-encoding policy at input boundaries instead of mixing flagged text with arbitrary byte strings.
- Validate generated CSV by checking row counts, record terminators, quoting balance, and round-trip parsing.
- Treat serialization failures and encoding anomalies as errors rather than relying solely on a successful return value.
- Apply least privilege to export processes and limit the downstream authority of generated files.
How teams detect issues like this
Detection approaches may include:
- Static analysis and code review searches for Text::CSV_XS
printcalls handling mixed UTF-8 and byte-oriented data. - Tests containing UTF-8-flagged fields followed by invalid or incomplete byte sequences.
- Boundary tests that force small output buffers or multiple flushes.
- Round-trip tests that compare original records with records parsed from generated CSV.
- Monitoring for unexpected row-count changes, unbalanced quotes, missing line endings, and merged records.
- Policy checks requiring explicit encoding validation before structured-data export.
Detection approaches may include:
- Static analysis and code review searches for Text::CSV_XS
printcalls handling mixed UTF-8 and byte-oriented data. - Tests containing UTF-8-flagged fields followed by invalid or incomplete byte sequences.
- Boundary tests that force small output buffers or multiple flushes.
- Round-trip tests that compare original records with records parsed from generated CSV.
- Monitoring for unexpected row-count changes, unbalanced quotes, missing line endings, and merged records.
- Policy checks requiring explicit encoding validation before structured-data export.
