biondizzle/vllm
1
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases Wiki Activity

Merge similar examples in offline_inference into single basic example (#12737)

Browse Source
This commit is contained in:
Harry Mellor
2025-02-20 12:53:51 +00:00
committed by GitHub
parent b69692a2d8
commit 992e5c3d34
29 changed files with 394 additions and 437 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
docs/source/models/pooling_models.md
View File

@@ -88,7 +88,7 @@ embeds = output.outputs.embedding
print(f"Embeddings: {embeds!r} (size={len(embeds)})")
```
A code example can be found here: <gh-file:examples/offline_inference/embedding.py>
A code example can be found here: <gh-file:examples/offline_inference/basic/embed.py>
### `LLM.classify`
@@ -103,7 +103,7 @@ probs = output.outputs.probs
print(f"Class Probabilities: {probs!r} (size={len(probs)})")
```
A code example can be found here: <gh-file:examples/offline_inference/classification.py>
A code example can be found here: <gh-file:examples/offline_inference/basic/classify.py>
### `LLM.score`
@@ -125,7 +125,7 @@ score = output.outputs.score
print(f"Score: {score}")
```
A code example can be found here: <gh-file:examples/offline_inference/scoring.py>
A code example can be found here: <gh-file:examples/offline_inference/basic/score.py>
## Online Serving