docs: add python / matplotlib requirement example by akihikokuroda · Pull Request #1293 · generative-computing/mellea

akihikokuroda · 2026-06-18T15:10:36Z

Pull Request

Issue

Fix: #1025

Description

Testing

Tests added to the respective file if code was changed
New code has 100% coverage if code was added
Ensure existing tests and github automation passes (a maintainer will kick off the github automation when the rest of the PR is populated)

Attribution

AI coding assistants used

Adding a new component, requirement, sampling strategy, or tool?

If your PR adds or modifies one of the types below, check the matching box. A checklist of type-specific review items will be posted as a comment.

Component
Requirement
Sampling Strategy
Tool

NOTE: Please ensure you have an issue that has been acknowledged by a core contributor and routed you to open a pull request against this repository. Otherwise, please open an issue before continuing with this pull request.

AngeloDanducci · 2026-06-18T19:14:57Z

+
+def main():
+    """Demonstrate complete pipeline: accept user input and generate graphs."""
+    import argparse


I'd make this top level.

The sys import is already at the top as well.

Moved to top and removed sys.

AngeloDanducci · 2026-06-18T19:21:06Z

+    """Extract Python code from LLM-generated text.
+
+    Looks for code blocks marked with triple backticks and python language tag.
+    Falls back to treating entire output as code if no markers found.
+    """
+    lines = generated_text.split("\n")
+    in_code_block = False
+    code_lines = []
+
+    for line in lines:
+        if line.strip().startswith("```python"):
+            in_code_block = True
+            continue
+        elif line.strip().startswith("```") and in_code_block:
+            in_code_block = False
+            continue
+
+        if in_code_block:
+            code_lines.append(line)
+
+    if code_lines:
+        return "\n".join(code_lines)
+    return generated_text


Can we replace this with PythonCodeExtraction?
Imports at top of file but something like this:

Suggested change

"""Extract Python code from LLM-generated text.

Looks for code blocks marked with triple backticks and python language tag.

Falls back to treating entire output as code if no markers found.

"""

lines = generated_text.split("\n")

in_code_block = False

code_lines = []

for line in lines:

if line.strip().startswith("```python"):

in_code_block = True

continue

elif line.strip().startswith("```") and in_code_block:

in_code_block = False

continue

if in_code_block:

code_lines.append(line)

if code_lines:

return "\n".join(code_lines)

return generated_text

def _extract_code_from_thunk(generated: object) -> str | None:

"""Extract the highest-scoring Python code block from a ModelOutputThunk."""

from mellea.stdlib.context import ChatContext

from mellea.core import ModelOutputThunk

ctx = ChatContext().add(generated) # type: ignore[arg-type]

result = PythonCodeExtraction().validation_fn(ctx)

return result.reason if result.as_bool() else None

updated to use "PythonCodeExtraction".

AngeloDanducci · 2026-06-18T19:24:26Z

+    return generated_text
+
+
+def execute_python_code(code: str, timeout: int = 10) -> dict:


Is this entirely covered by m.instruct with all_reqs below? This line line 277 to my understanding will cause m.instruct to perform PythonExecutionReq.validation_fn which executes the code as part of the validation _python_executes_without_error(). If so we can remove this.

If we remove this I think we can also remove import subprocess and import tempfile

That's correct. The code execution is a side effect of the requirement validation. The execution code is removed but left as comments. It explains why it is not necessary here but when it is necessary.

AngeloDanducci · 2026-06-18T19:35:53Z

+    generated_str = str(generated)
+    code = extract_python_code(generated_str)
+
+    print("\nGenerated code:")
+    print(code)
+
+    print("\nExecuting code...")
+    result = execute_python_code(code, timeout=15)
+
+    if not result["success"]:
+        print(f"  ✗ Error during execution: {result['error']}")
+        return
+
+    print("  ✓ Code executed successfully")
+    if result["output"]:
+        print(f"\n  Output: {result['output']}")
+
+    # Check if graph file was created
+    graph_path = Path(output_path)
+    if graph_path.exists():
+        print(f"\n  ✓ Graph saved to: {graph_path}")
+        print(f"    File size: {graph_path.stat().st_size} bytes")
+    else:
+        print(f"\n  ⚠ Graph file not found at {graph_path}")


If using the extract code suggestion and the removal above I think this can be simplified to

Suggested change

generated_str = str(generated)

code = extract_python_code(generated_str)

print("\nGenerated code:")

print(code)

print("\nExecuting code...")

result = execute_python_code(code, timeout=15)

if not result["success"]:

print(f" ✗ Error during execution: {result['error']}")

return

print(" ✓ Code executed successfully")

if result["output"]:

print(f"\n Output: {result['output']}")

# Check if graph file was created

graph_path = Path(output_path)

if graph_path.exists():

print(f"\n ✓ Graph saved to: {graph_path}")

print(f" File size: {graph_path.stat().st_size} bytes")

else:

print(f"\n ⚠ Graph file not found at {graph_path}")

code = _extract_code_from_thunk(generated)

print("\nGenerated code:")

print(code if code is not None else str(generated))

graph_path = Path(output_path)

if graph_path.exists():

print(f"\n ✓ Graph saved to: {graph_path}")

print(f" File size: {graph_path.stat().st_size} bytes")

else:

print(f"\n ⚠ Graph file not found at {graph_path}")

Thanks. I didn't do simplification as an example. I intentionally left code as comments.

psschwei

a few nits but otherwise LGTM

psschwei · 2026-06-18T21:40:48Z

+import mellea
+from mellea.stdlib.components import Message
+from mellea.stdlib.context import ChatContext
+from mellea.stdlib.requirements.plotting.matplotlib import (


nit: In this readme, this is from mellea.stdlib.requirements.plotting import PlotFileSaved, probably should use the same both places (not sure which is right)

This one was wrong. Fixed it.

psschwei · 2026-06-18T21:42:00Z

+    print("=" * 70)
+
+    # Use provided options or defaults
+    csv_path = args.csv if args.csv else "/tmp/sample_data.csv"


would it make sense to use tempfile instead of hardcoding a path?

Yes, it make sense. Updated it.

psschwei · 2026-06-18T21:43:09Z

+        "Nancy",
+        "Oscar",
+        "Piper",
+        "Quinn",


also on L179, is it ok to duplicate or should we use another Q name here?

it is removed.

planetf1 · 2026-06-19T14:15:52Z

+        print(f"\n  ✓ Graph saved to: {graph_path}")
+        print(f"    File size: {graph_path.stat().st_size} bytes")
+    else:
+        print(f"\n  ⚠ Graph file not found at {graph_path}")


The conftest passes the test when the script exits 0, and process_user_request always exits 0 — there's no sys.exit or raise when the graph isn't produced, just a print + return. So even with matplotlib installed, if the pipeline fails for any reason the test is recorded as passing whilst producing nothing. Worth raising here so failures are visible:

Suggested change

print(f"\n ⚠ Graph file not found at {graph_path}")

if not graph_path.exists():

raise RuntimeError(f"Graph was not created at {graph_path}")

print(f"\n ✓ Graph saved to: {graph_path}")

print(f" File size: {graph_path.stat().st_size} bytes")

planetf1 · 2026-06-19T14:23:21Z

+        PythonCodeExtraction(),
+        PythonSyntaxValid(),
+        PythonExecutionReq(
+            execution_tier="local",


execution_tier="local" triggers a library-level warning about running untrusted code without container isolation, and the README lists "CI/CD environments" as a use case for this example. Might be worth a comment pointing readers to execution_tier="docker" for untrusted inputs — keeps the local demo useful whilst teaching the safer pattern. (Context: commit 85c447e recently tightened the project's default posture here.)

I'll address this as a follow up.

planetf1 · 2026-06-19T14:24:06Z


 - matplotlib and numpy installed (auto-checked by requirements)
- Any code analysis backend (requirements use AST analysis)
+- Any code analysis backend (requirements use AST analysis)


This section describes the AST-only examples (matplotlib_plotting.py) — it doesn't cover code_generation_and_execution.py, which also needs an Ollama backend and a matplotlib runtime for the generated code to execute. Worth noting the distinction so readers know what to set up before running each example.

Added statements for code_generation_and_execution.py

planetf1 · 2026-06-19T14:25:01Z

Tiny one: the PR title has a typo — "requrement" should be "requirement". Worth fixing before merge as it ends up in the commit message.

planetf1 · 2026-06-19T14:25:26Z

+import mellea
+from mellea.stdlib.components import Message
+from mellea.stdlib.context import ChatContext
+from mellea.stdlib.requirements.plotting import MatplotlibHeadlessBackend, PlotFileSaved


matplotlib isn't a project dependency, so without it the repair loop spins 5 times against a ModuleNotFoundError (no specific handler for this case), then exits with ⚠ Graph file not found and no hint of the cause. The project convention (AGENTS.md §5) is to wrap optional imports in a try/except ImportError with a friendly message — that would both tell the user what to install and let the CI conftest skip the test cleanly (it already converts ImportError in subprocess stderr to pytest.skip). The README also needs an install step for the new example — the existing Requirements section refers to the AST-only examples and doesn't cover this one.

Changes made:

Friendly dependency error: Added try/except ImportError with a clear message telling users exactly what to install

README prerequisites: Added a "Prerequisites" section that documents:

How to start Ollama with the required model

How to install matplotlib and numpy

Clear requirements distinction: Updated the final "Requirements" section to explicitly separate the needs of each example

planetf1

Thanks for the example — the pipeline approach is nicely structured and the README additions are clear.

Two things I'd want addressed before merge:

matplotlib guard — matplotlib isn't a project dependency so the example can't succeed on a clean checkout. The repair loop exhausts silently with no useful message to the user (see inline comment).
Silent test pass — the conftest passes the test on exit 0, and process_user_request always exits 0 even when no graph is produced. The e2e test gives a false green in that case (see inline comment).

The other inline comments (execution tier caveat, README requirements section) are informational — happy for those to be addressed or tracked as follow-ups.

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>

akihikokuroda requested a review from a team as a code owner June 18, 2026 15:10

akihikokuroda requested review from markstur, planetf1 and psschwei June 18, 2026 15:10

github-actions Bot added the documentation Improvements or additions to documentation label Jun 18, 2026

AngeloDanducci reviewed Jun 18, 2026

View reviewed changes

akihikokuroda requested a review from AngeloDanducci June 18, 2026 21:37

psschwei reviewed Jun 18, 2026

View reviewed changes

akihikokuroda requested a review from psschwei June 18, 2026 22:23

planetf1 reviewed Jun 19, 2026

View reviewed changes

planetf1 requested changes Jun 19, 2026

View reviewed changes

akihikokuroda changed the title ~~docs: add python / matplotlib requrement example~~ docs: add python / matplotlib requirement example Jun 19, 2026

akihikokuroda requested a review from planetf1 June 19, 2026 15:45

akihikokuroda added 8 commits June 19, 2026 14:08

example for python requirements and samplings

a4e8f60

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>

add requirements

815ab51

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>

change python execution check requrement usage

57f1b1d

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>

review comments

9c0da95

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>

review comment

e8503b3

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>

review comment

1fb51fc

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>

review comment

c54570b

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>

improved prompt

11fbd13

Signed-off-by: Akihiko Kuroda <akihikokuroda2020@gmail.com>

akihikokuroda force-pushed the issue1025 branch from 0949db1 to 11fbd13 Compare June 19, 2026 23:08

		return generated_text


		def execute_python_code(code: str, timeout: int = 10) -> dict:

Conversation

akihikokuroda commented Jun 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request

Issue

Description

Testing

Attribution

Adding a new component, requirement, sampling strategy, or tool?

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AngeloDanducci Jun 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AngeloDanducci Jun 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

psschwei left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

planetf1 commented Jun 19, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

planetf1 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

akihikokuroda commented Jun 18, 2026 •

edited

Loading

AngeloDanducci Jun 18, 2026 •

edited

Loading

AngeloDanducci Jun 18, 2026 •

edited

Loading