Skip to content

Clear escape delimiter buffer before peek in isEscapeDelimiter#608

Merged
garydgregory merged 1 commit into
apache:masterfrom
rootvector2:escape-delimiter-buffer-clear
Jun 11, 2026
Merged

Clear escape delimiter buffer before peek in isEscapeDelimiter#608
garydgregory merged 1 commit into
apache:masterfrom
rootvector2:escape-delimiter-buffer-clear

Conversation

@rootvector2

Copy link
Copy Markdown
Contributor

isEscapeDelimiter() peeks the next characters into the reused escapeDelimiterBuf look-ahead without clearing it first, so a truncated escaped multi-character delimiter at EOF is completed from the stale bytes of an earlier full escaped delimiter and the lexer appends a delimiter the input never contained. clearing the buffer before the peek mirrors the delimiterBuf reset added in nextToken for the plain multi-character delimiter path.

found while reading the partial-delimiter-at-EOF fix (CSV-324): the escaped-delimiter sibling was left unguarded.

for delimiter [|] and escape !, parsing x![!|!]y![!| returns x[|]y[|] before the change and the correct x[|]y![!| after. regression tests added in LexerTest and CSVParserTest (both fail without the one-line change).

  • Read the contribution guidelines for this project.
  • Read the ASF Generative Tooling Guidance if you use Artificial Intelligence (AI).
  • I used AI to create any part of, or all of, this pull request. Which AI tool was used to create this pull request, and to what extent did it contribute?
  • Run a successful build using the default Maven goal with mvn; that's mvn on the command line by itself.
  • Write unit tests that match behavioral changes, where the tests fail if the changes to the runtime are not applied. This may not always be possible, but it is a best practice.
  • Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
  • Each commit in the pull request should have a meaningful subject line and body. Note that a maintainer may squash commits during the merge process.

@garydgregory garydgregory changed the title clear escape delimiter buffer before peek in isEscapeDelimiter Clear escape delimiter buffer before peek in isEscapeDelimiter Jun 11, 2026
@garydgregory garydgregory merged commit 6551fc4 into apache:master Jun 11, 2026
16 checks passed
@garydgregory

Copy link
Copy Markdown
Member

Thank you @rootvector2

PR merged 🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants