Ask Qwen Task#406
Conversation
Deploying geodatalytics with
|
| Latest commit: |
c9ad678
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://894360ba.geodatalytics.pages.dev |
| Branch Preview URL: | https://first-hf-task.geodatalytics.pages.dev |
Co-authored-by: Brian Helba <brian.helba@kitware.com>
| "role": "user", | ||
| "content": [ | ||
| {"type": "image_url", "image_url": {"url": thumbnail_uri}}, | ||
| {"type": "text", "text": f"{PROMPT_PREFIX} {text_prompt}"}, |
There was a problem hiding this comment.
Rather than prefixing the prompt, should we instead use a system message?
If we make this a general endpoint, that is where you would also have more instructions to try to avoid prompt hacking, if we care.
| chat = endpoint.client.chat_completion( | ||
| model="unsloth/Qwen3.5-9B-GGUF", | ||
| messages=messages, | ||
| max_tokens=max_tokens, |
There was a problem hiding this comment.
Should we be specifying a temperature?
| result.write_status("Encoding imagery...") | ||
| imagery_path = utilities.field_file_to_local_path(imagery.cloud_optimized_geotiff) | ||
| src = large_image.open(imagery_path) | ||
| thumbnail_bytes, _ = src.getThumbnail(THUMBNAIL_WIDTH, encoding="PNG") |
There was a problem hiding this comment.
We probably want pass the limit to both width and height, or a long narrow image will be too large. And, I think the maximum size image will depend on the model and which projector transformers or ollama uses for it.
| "pyvips==3.1.1.8.18.1", | ||
| "uvdat-flood-sim[large-image-writer]==1.0.4", | ||
| "xdg-base-dirs==6.0.2", | ||
| "huggingface-hub>=1.14.0", |
There was a problem hiding this comment.
| "huggingface-hub>=1.14.0", | |
| "huggingface-hub==1.14.0", |
Renovate will try to pin this, so you might as well do it now. I'm not saying this is generally a best practice for Python, but it's how Renovate works currently.
| from .analysis_type import AnalysisInputError, AnalysisTask, AnalysisType | ||
|
|
||
| ENDPOINT_NAMESPACE = "Kitware" | ||
| ENDPOINT_NAME = "qwen3-5-9b-gguf-ulh" |
There was a problem hiding this comment.
Although I previously suggested that we didn't have to keep this a secret, now that I see it hardcoded here, I'm worried that future open source users or developers will try to use our deployment-specific endpoint (I've seen this happen before).
I think we should make this a deployment-specific env var.
To avoid proliferation of multiple env vars, we could potentially pack this into a single URL.
3 participants
This PR adds a new analytics task called "Imagery: Ask Qwen". The task accepts three inputs: 1) an imagery raster, 2) a text prompt, and 3) a
max_tokensvalue (1k - 10k). The task leverages our first Huggingface inference endpoint, sending a lower resolution version of the imagery to the endpoint along with the question and returning the response to the user.The screenshots below show some example runs on the boston orthoimagery layer:
To support the large markdown responses, this PR makes a slight change to the Analytics Panel in the client, using
vue-markdown-renderfor a new output type "markdown".