Skip to content

[codex] Register Moondream2 inference-models adapter#2422

Closed
hansent wants to merge 3 commits into
mainfrom
hansent/fix-moondream2-lmm-registration
Closed

[codex] Register Moondream2 inference-models adapter#2422
hansent wants to merge 3 commits into
mainfrom
hansent/fix-moondream2-lmm-registration

Conversation

@hansent

@hansent hansent commented Jun 8, 2026

Copy link
Copy Markdown
Collaborator

Summary

Fixes Moondream2 registration when the server is running with USE_INFERENCE_MODELS=True.

The direct /infer/lmm path resolves Moondream2 generic IDs to ("lmm", "moondream2"), while the new inference-models registry exposes Moondream2 as a VLM model. Previously, the adapter swap only ran if an existing ("lmm", "moondream2") entry survived legacy registration, so deployments using the inference-models stack could hit ModelNotRecognisedError for ('lmm', 'moondream2').

Changes

  • Explicitly register InferenceModelsMoondream2Adapter for both ("lmm", "moondream2") and ("vlm", "moondream2") when MOONDREAM2_ENABLED and USE_INFERENCE_MODELS are enabled.
  • Set the Moondream2 adapter task_type to lmm, matching the other VLM-backed direct LMM adapters in core inference.
  • Add regression coverage for the lmm/vlm registry entries and adapter task type.

Validation

  • python -m py_compile inference/models/utils.py inference/models/moondream2/moondream2_inference_models.py tests/inference/unit_tests/models/test_moondream2_registration.py
  • PYTHONPATH=.:inference_models uv run --no-project --with pytest --with requests --with python-dotenv --with pillow --with numpy --with pydantic --with fastapi --with opencv-python-headless --with pycocotools python -m pytest tests/inference/unit_tests/models/test_moondream2_registration.py -q
  • PYTHONPATH=.:inference_models uv run --no-project --with black python -m black --check inference/models/utils.py inference/models/moondream2/moondream2_inference_models.py tests/inference/unit_tests/models/test_moondream2_registration.py

self.api_key = api_key if api_key else API_KEY

self.task_type = "llm"
self.task_type = "lmm"

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure about this one

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this is not needed

Comment thread inference/models/utils.py
InferenceModelsMoondream2Adapter,
)

ROBOFLOW_MODEL_TYPES[("lmm", "moondream2")] = InferenceModelsMoondream2Adapter

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

old registry and auth endpoint has it under lmm, new registry says vlm

@hansent hansent marked this pull request as ready for review June 10, 2026 17:25

@PawelPeczek-Roboflow PawelPeczek-Roboflow left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how about inference/models/utils.py line 925

            elif task == "lmm" and variant == "moondream2":
                from inference.models.moondream2.moondream2_inference_models import (
                    InferenceModelsMoondream2Adapter,
                )

                ROBOFLOW_MODEL_TYPES[(task, variant)] = InferenceModelsMoondream2Adapter
                ROBOFLOW_MODEL_TYPES[("vlm", "moondream2")] = (
                    InferenceModelsMoondream2Adapter
                )

are you sure your change is different than that?

@hansent

hansent commented Jun 18, 2026

Copy link
Copy Markdown
Collaborator Author

fixed as part of new auth endpoint change

@hansent hansent closed this Jun 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants