Skip to content

Fix pickling regression with scipy backend after multiple opens#11413

Open
C1-BA-B1-F3 wants to merge 1 commit into
pydata:mainfrom
C1-BA-B1-F3:fix-scipy-pickling-issue-11323
Open

Fix pickling regression with scipy backend after multiple opens#11413
C1-BA-B1-F3 wants to merge 1 commit into
pydata:mainfrom
C1-BA-B1-F3:fix-scipy-pickling-issue-11323

Conversation

@C1-BA-B1-F3

Copy link
Copy Markdown

Description

This PR fixes a pickling regression introduced in 2026.4.0 when using the scipy backend with file-like objects.

Problem: When opening multiple scipy-backed datasets from file-like objects (e.g., BytesIO), the flush_only_netcdf_file class was recreated inside _open_scipy_netcdf on each call. Each new class definition overwrote the previous one on _PickleWorkaround, breaking pickle's class-identity check for instances created by earlier calls.

Fix: Only set the class on _PickleWorkaround once (on the first call). Subsequent calls reuse the same class definition, ensuring pickle stability.

This is a minimal, focused fix that:

  1. Adds an _initialized flag to _PickleWorkaround
  2. Only sets the class attribute on the first call
  3. Adds a test case for the specific scenario from issue 2026.4.0 breaks pickling with backends.scipy_ #11323

Related Issues

Testing

Added a new test test_pickle_open_dataset_after_multiple_opens that verifies pickling works correctly after opening multiple datasets from the same bytes source.

All existing scipy backend tests continue to pass.


Minimal Complete Verifiable Example (now fixed):

import io
import pickle
import xarray as xr
import numpy as np

ds = xr.Dataset(
    {"foo": (("x",), np.arange(4, dtype=np.float64))},
    coords={"x": np.arange(4)},
)
buf = io.BytesIO()
ds.to_netcdf(buf, engine="scipy")

buf.seek(0)
ds1 = xr.open_dataset(buf, engine="scipy")
buf.seek(0)
ds2 = xr.open_dataset(buf, engine="scipy")

pickle.dumps(ds1)  # Previously raised PicklingError, now works

When opening multiple scipy-backed datasets from file-like objects,
the  class was recreated inside
on each call, overwriting the previous class definition on .
This broke pickle's class-identity check for instances created by earlier calls.

Fix: Only set the class on  once (first call). Subsequent
calls reuse the same class definition, ensuring pickle stability.

Closes pydata#11323
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2026.4.0 breaks pickling with backends.scipy_

1 participant