docs(react/powersync): document sync-stream SQL footguns#467
Conversation
📝 WalkthroughWalkthroughThis PR adds a "Sync Stream SQL — Known Footguns" section to the PowerSync README template, documenting server/client SQL behavior mismatches including unsupported JOIN clauses and operator semantic differences across the JavaScript-to-SQLite execution boundary. ChangesPowerSync README Enhancement
Estimated Code Review Effort🎯 1 (Trivial) | ⏱️ ~5 minutes Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/create/src/frameworks/react/add-ons/powersync/README.md.ejs`:
- Around line 32-35: Update the README wording: narrow the "**`JOIN` clauses**"
line to state that Sync Streams support `JOIN`/`INNER JOIN` but do not support
`LEFT`/`RIGHT`/`OUTER` (and describe the specific unsupported patterns) and
explicitly note the actual failure mode per powersync-ja/powersync-service#662
(silent zero-row sync vs proposed loud error), and revise the
`upper()`/`lower()`, `length()`, and `substring()`/`substr()` bullets so their
behavioral claims match the corresponding PRs (`#663`, `#664`, `#665`) — i.e., state
ASCII-only casing for `upper()`/`lower()`, UTF-16 code-unit vs code-point
differences for `length()`, and code-unit indexing/surrogate-splitting behavior
for `substring()`/`substr()` exactly as described in those PRs, removing any
overbroad or mismatched examples.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 183cb8f9-bfd5-48c3-994b-a78df138a885
📒 Files selected for processing (1)
packages/create/src/frameworks/react/add-ons/powersync/README.md.ejs
| - **`JOIN` clauses** are not currently supported in sync streams and silently sync zero rows. Rewrite as a single-table query with a correlated subquery in the `WHERE` clause. See [powersync-ja/powersync-service#662](https://github.com/powersync-ja/powersync-service/pull/662) for a loud error proposal. | ||
| - **`upper()` / `lower()`** apply locale-aware Unicode case folding (length-changing on `ß`, `fi`, Turkish dotted-i) where SQLite is ASCII-only. Avoid as bucket keys over non-ASCII data until [powersync-service#663](https://github.com/powersync-ja/powersync-service/pull/663) lands. | ||
| - **`length()`** returns a UTF-16 code-unit count where SQLite returns code points — diverges 2x on emoji, CJK Extension B-G, ancient scripts. Avoid in bucket-parameter filters until [#664](https://github.com/powersync-ja/powersync-service/pull/664). | ||
| - **`substring()` / `substr()`** indexes by UTF-16 code units and can split surrogate pairs, returning unpaired-surrogate strings. Avoid on user content with non-BMP code points until [#665](https://github.com/powersync-ja/powersync-service/pull/665). |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Description: Verify PowerSync upstream PR links exist and are accessible
echo "Checking PR `#662`..."
gh pr view 662 --repo powersync-ja/powersync-service --json title,url,state
echo "Checking PR `#663`..."
gh pr view 663 --repo powersync-ja/powersync-service --json title,url,state
echo "Checking PR `#664`..."
gh pr view 664 --repo powersync-ja/powersync-service --json title,url,state
echo "Checking PR `#665`..."
gh pr view 665 --repo powersync-ja/powersync-service --json title,url,stateRepository: TanStack/cli
Length of output: 807
🌐 Web query:
PowerSync sync-rules SQL JOIN support and known limitations
💡 Result:
PowerSync handles SQL JOIN support differently depending on the synchronization system being used: Sync Streams or the legacy Sync Rules [1]. Sync Streams (Recommended) Sync Streams support INNER JOINs, though with specific limitations [2][3]. - Supported Syntax: Only JOIN or INNER JOIN are permitted [2][3]. - Unsupported Joins: LEFT, RIGHT, and OUTER joins are not supported [2]. - Selection Constraint: All selected columns must originate from a single table (e.g., SELECT table1.* FROM table1 JOIN table2 ON ...) [2][3]. Selecting columns from multiple tables in a single query is invalid [2][3]. - Condition Constraint: Join conditions must be simple equality comparisons (e.g., table1.column = table2.column) [2][3]. Complex expressions in join conditions (e.g., a.x > b.y) are not supported [2]. Sync Rules (Legacy) Sync Rules do not support SQL JOINs [2][4][5]. For projects using this legacy system, developers typically use workarounds such as: - Denormalization: Adding direct foreign key relationships or auxiliary columns to the schema to avoid the need for joins [4]. - Materialized Views: For Postgres sources, using extensions like pg_ivm to create incrementally maintained materialized views that can be queried directly [4][6]. - Parameter Queries: Using bucket parameters to pass IDs that allow for filtering data without requiring joins [7]. Because Sync Rules are considered legacy, PowerSync recommends migrating to Sync Streams for more complex query requirements, as they provide significantly more expressive capabilities, including CTEs and subqueries alongside the supported JOIN syntax [8][1]. Regardless of the sync method, keep in mind that PowerSync manages data synchronization between the server and the local SQLite database; complex JOINs can still be performed locally within the client-side SQLite database after the data has been synced [6].
Citations:
- 1: https://docs.powersync.com/sync/rules/overview
- 2: https://docs.powersync.com/sync/supported-sql
- 3: https://docs.powersync.com/sync/streams/queries
- 4: https://docs.powersync.com/sync/rules/many-to-many-join-tables
- 5: https://stackoverflow.com/questions/79218804/powersync-sync-rule-issue-with-supabase-auth-relationship-data
- 6: https://powersync.com/blog/using-pg-ivm-to-enable-joins-in-powersync
- 7: https://docs.powersync.com/sync/rules/parameter-queries
- 8: https://docs.powersync.com/sync/streams/migration
🌐 Web query:
PowerSync JavaScript SQL evaluator UTF-16 string function behavior
💡 Result:
PowerSync does not implement custom JavaScript-level logic for string function behavior in its SQL evaluator; instead, it relies on SQLite’s native text processing capabilities [1][2]. In the PowerSync architecture, data is synced as SQLite TEXT columns, which are stored as UTF-8 encoded strings [3][4]. Key points regarding string behavior in PowerSync: 1. Storage and Encoding: PowerSync maps most string-like data types (including JSON and various custom types) to the SQLite TEXT storage class, which natively uses UTF-8 encoding [3][4]. 2. SQL Evaluator: Because PowerSync utilizes standard SQLite as its local database engine, string functions (such as length, substring, instr, upper, and lower) behave according to SQLite's built-in definitions [1]. These functions operate on the stored UTF-8 string data [1]. 3. JavaScript Interaction: While PowerSync’s client SDKs are built with JavaScript/TypeScript, the SQL evaluator is essentially "just SQLite" [2]. When your JavaScript code interacts with these functions via PowerSync’s query methods (e.g., execute, getAll), the JavaScript runtime and the SQLite engine handle string passing [5][2]. Standard JavaScript strings are UTF-16, but they are typically converted to UTF-8 when interacting with the underlying SQLite database layer [6][7][2]. 4. No UTF-16 Specific Functions: PowerSync does not provide a specific API for "UTF-16 string functions." Developers should rely on standard SQLite string functions, which are designed to be efficient for the UTF-8 storage format [1]. In summary, you should assume standard SQLite behavior for all string operations rather than JavaScript-specific UTF-16 behavior, as the SQL evaluator functions are executed against the UTF-8 encoded data within the SQLite engine [1][3][2].
Citations:
- 1: https://docs.powersync.com/sync/supported-sql
- 2: https://powersync.com/blog/speeding-up-powersync-with-a-sqlite-extension-written-in-rust
- 3: https://docs.powersync.com/sync/types
- 4: https://docs.powersync.com/client-sdks/advanced/query-json-in-sqlite
- 5: https://docs.powersync.com/client-sdks/reference/javascript-web
- 6: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String
- 7: https://mathiasbynens.be/notes/javascript-encoding
Fix Sync Streams JOIN wording to match PowerSync docs (and align string-function claims with PR details)
- PR links are valid:
powersync-ja/powersync-service#662,#663,#664, and#665all exist and resolve to the expected pulls. - The README’s JOIN statement (“JOIN clauses are not currently supported in sync streams”) is too broad: PowerSync docs say Sync Streams support
JOIN/INNER JOINwith limitations (while other join types likeLEFT/RIGHT/OUTERaren’t supported). Reword to describe the specific unsupported JOIN patterns and the actual failure mode (silent zero rows vs loud error) consistent with#662. - For
upper()/lower(),length(), andsubstring()/substr(), the PR titles indicate the intended fixes (ASCII-only casing; code points vs code units; code-point indexing), but the README’s detailed behavioral claims should be made to exactly match the PR descriptions (especially the “silent mismatch” mechanics and the edge-case examples) consistent with#663/#664/#665.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/create/src/frameworks/react/add-ons/powersync/README.md.ejs` around
lines 32 - 35, Update the README wording: narrow the "**`JOIN` clauses**" line
to state that Sync Streams support `JOIN`/`INNER JOIN` but do not support
`LEFT`/`RIGHT`/`OUTER` (and describe the specific unsupported patterns) and
explicitly note the actual failure mode per powersync-ja/powersync-service#662
(silent zero-row sync vs proposed loud error), and revise the
`upper()`/`lower()`, `length()`, and `substring()`/`substr()` bullets so their
behavioral claims match the corresponding PRs (`#663`, `#664`, `#665`) — i.e., state
ASCII-only casing for `upper()`/`lower()`, UTF-16 code-unit vs code-point
differences for `length()`, and code-unit indexing/surrogate-splitting behavior
for `substring()`/`substr()` exactly as described in those PRs, removing any
overbroad or mismatched examples.
1 participant
What
Adds a "Known Footguns" section to the PowerSync add-on README covering four silent-row-loss bugs in the upstream PowerSync sync-rules engine that affect any starter generated by this add-on, until the upstream fixes ship.
Why
The PowerSync add-on package.json pins
@powersync/web ^1.26.1. The sync-stream SQL evaluator in PowerSync up through that version has four operator implementations that disagree with the SQLite that runs on the client — bucket keys silently diverge and rows fail to sync without any error or log line:upper()/lower()use locale-aware Unicode case folding (length-changing onß,fi, Turkishi) where SQLite is ASCII-only.length()returns UTF-16 code-unit count where SQLite returns Unicode code points (2x divergence on emoji, CJK Extension B-G).substring()/substr()slices by UTF-16 code units and can return unpaired-surrogate strings.The fixes are open upstream:
length()code points)substring()code points)Until they land in
@powersync/service, this add-on quietly generates starters that will silently lose rows if user data contains any non-ASCII characters or any JOIN gets written intosync-rules.yaml. The README addition documents that explicitly with workarounds.Maintainer note
Disclosure: I authored the four upstream PRs linked here, plus four already-merged sync-rules fixes in the same engine. silentdrop is an open-source MIT correctness checker I built that catches this class of bug across JS DB layers. Happy to remove the silentdrop link if you prefer the section be vendor-neutral.
This is documentation-only — no code change in the generated app.
Summary by CodeRabbit