Skip to content

Commit 6bcc3eb

Browse files
ddebowczykclaude
andcommitted
feat(polyglot): add Perplexity embeddings driver
Add a `perplexity` Embeddings driver for pplx-embed-v1-{0.6b,4b} and the contextualized pplx-embed-context-v1-{0.6b,4b} models. - decode base64 signed int8 embeddings into float vectors; reject encodings other than base64_int8 - route contextualized models to /contextualizedembeddings, sending the request inputs as chunks of one document and flattening in chunk order - bundled `perplexity` and `perplexity-context` presets and model profiles with limits, default dimensions, and pricing - adapter, mock-HTTP, catalog and opt-in live tests; overview docs Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent ff3f925 commit 6bcc3eb

20 files changed

Lines changed: 607 additions & 1 deletion

‎packages/polyglot/docs/embeddings/overview.md‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -112,9 +112,36 @@ Polyglot ships with presets for the following providers:
112112
| `jina` | Jina | (configured per preset) | (varies) |
113113
| `mistral` | OpenAI-compatible | (configured per preset) | (varies) |
114114
| `ollama` | OpenAI-compatible | (configured per preset) | (varies) |
115+
| `perplexity` | Perplexity | `pplx-embed-v1-0.6b` | 1024 |
116+
| `perplexity-context` | Perplexity | `pplx-embed-context-v1-0.6b` | 1024 |
115117

116118
> **Note:** Mistral and Ollama use the OpenAI-compatible driver, since their APIs follow the same format.
117119
120+
### Perplexity
121+
122+
The `perplexity` driver reads `PERPLEXITY_API_KEY` and supports
123+
`pplx-embed-v1-0.6b`, `pplx-embed-v1-4b`, and the contextualized
124+
`pplx-embed-context-v1-0.6b` and `pplx-embed-context-v1-4b` models.
125+
126+
- Perplexity returns base64 signed int8 embeddings. The driver decodes them
127+
into float vectors in the range -128 to 127. They are not normalized, so
128+
compare them with cosine similarity rather than dot product or L2 distance.
129+
- Only the default `base64_int8` encoding is supported; `base64_binary` is
130+
rejected. Pass `['dimensions' => 256]` (128 up to the model maximum) through
131+
`withOptions()` to request shorter Matryoshka vectors.
132+
- Contextualized models use `/contextualizedembeddings`. All inputs of one
133+
request are sent as the chunks of a single document, and vectors come back
134+
in chunk order. Embed each document in its own request, and embed queries
135+
with the same contextualized model.
136+
- Standard models accept up to 512 inputs and 120,000 tokens per request; each
137+
input must stay under 32K tokens.
138+
139+
```php
140+
$chunks = Embeddings::using('perplexity-context')
141+
->withInputs(['First paragraph...', 'Second paragraph...'])
142+
->vectors();
143+
```
144+
118145
## Custom Driver Registration
119146

120147
You can register your own driver for providers not bundled with Polyglot by creating a custom `EmbeddingsDriverRegistry`:
Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
# Sources reviewed 2026-10-01:
2+
# https://docs.perplexity.ai/docs/embeddings/quickstart.md
3+
# https://docs.perplexity.ai/docs/embeddings/contextualized-embeddings.md
4+
# Embeddings are returned as unnormalized base64 int8 values; compare them by
5+
# cosine similarity. Matryoshka dimensions range from 128 to 1024.
6+
# The driver sends all request inputs as the chunks of one document.
7+
schemaVersion: 1
8+
version: '2026-10-01'
9+
profile:
10+
driver: perplexity
11+
model: pplx-embed-context-v1-0.6b
12+
maxInputs: 16000
13+
maxInputTokens: 32000
14+
defaultDimensions: 1024
15+
pricing:
16+
inputPerMToken: 0.008
Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
# Sources reviewed 2026-10-01:
2+
# https://docs.perplexity.ai/docs/embeddings/quickstart.md
3+
# https://docs.perplexity.ai/docs/embeddings/contextualized-embeddings.md
4+
# Embeddings are returned as unnormalized base64 int8 values; compare them by
5+
# cosine similarity. Matryoshka dimensions range from 128 to 2560.
6+
# The driver sends all request inputs as the chunks of one document.
7+
schemaVersion: 1
8+
version: '2026-10-01'
9+
profile:
10+
driver: perplexity
11+
model: pplx-embed-context-v1-4b
12+
maxInputs: 16000
13+
maxInputTokens: 32000
14+
defaultDimensions: 2560
15+
pricing:
16+
inputPerMToken: 0.05
Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
# Sources reviewed 2026-10-01:
2+
# https://docs.perplexity.ai/docs/embeddings/quickstart.md
3+
# https://docs.perplexity.ai/docs/embeddings/standard-embeddings.md
4+
# Embeddings are returned as unnormalized base64 int8 values; compare them by
5+
# cosine similarity. Matryoshka dimensions range from 128 to 1024.
6+
schemaVersion: 1
7+
version: '2026-10-01'
8+
profile:
9+
driver: perplexity
10+
model: pplx-embed-v1-0.6b
11+
maxInputs: 512
12+
maxInputTokens: 32000
13+
maxRequestTokens: 120000
14+
defaultDimensions: 1024
15+
pricing:
16+
inputPerMToken: 0.004
Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
# Sources reviewed 2026-10-01:
2+
# https://docs.perplexity.ai/docs/embeddings/quickstart.md
3+
# https://docs.perplexity.ai/docs/embeddings/standard-embeddings.md
4+
# Embeddings are returned as unnormalized base64 int8 values; compare them by
5+
# cosine similarity. Matryoshka dimensions range from 128 to 2560.
6+
schemaVersion: 1
7+
version: '2026-10-01'
8+
profile:
9+
driver: perplexity
10+
model: pplx-embed-v1-4b
11+
maxInputs: 512
12+
maxInputTokens: 32000
13+
maxRequestTokens: 120000
14+
defaultDimensions: 2560
15+
pricing:
16+
inputPerMToken: 0.03
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
driver: perplexity
2+
apiUrl: 'https://api.perplexity.ai/v1'
3+
apiKey: '${PERPLEXITY_API_KEY}'
4+
endpoint: /embeddings
5+
model: pplx-embed-context-v1-0.6b
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
driver: perplexity
2+
apiUrl: 'https://api.perplexity.ai/v1'
3+
apiKey: '${PERPLEXITY_API_KEY}'
4+
endpoint: /embeddings
5+
model: pplx-embed-v1-0.6b

‎packages/polyglot/src/Embeddings/Creation/EmbeddingsDriverRegistry.php‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@
1111
use Cognesy\Polyglot\Embeddings\Drivers\Gemini\GeminiDriver;
1212
use Cognesy\Polyglot\Embeddings\Drivers\Jina\JinaDriver;
1313
use Cognesy\Polyglot\Embeddings\Drivers\OpenAI\OpenAIDriver;
14+
use Cognesy\Polyglot\Embeddings\Drivers\Perplexity\PerplexityDriver;
1415
use InvalidArgumentException;
1516
use Override;
1617
use Psr\EventDispatcher\EventDispatcherInterface;
@@ -37,6 +38,7 @@ public static function default(): self {
3738
'mistral' => OpenAIDriver::class,
3839
'openai' => OpenAIDriver::class,
3940
'ollama' => OpenAIDriver::class,
41+
'perplexity' => PerplexityDriver::class,
4042
]);
4143
}
4244

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
<?php
2+
3+
declare(strict_types=1);
4+
5+
namespace Cognesy\Polyglot\Embeddings\Drivers\Perplexity;
6+
7+
use Cognesy\Polyglot\Embeddings\Config\EmbeddingsConfig;
8+
use Cognesy\Polyglot\Embeddings\Contracts\CanMapRequestBody;
9+
use Cognesy\Polyglot\Embeddings\Data\EmbeddingsRequest;
10+
use InvalidArgumentException;
11+
12+
/**
13+
* Standard models take a flat list of texts. Contextualized models take
14+
* documents as lists of chunks; the request inputs are sent as the chunks of
15+
* one document. Only `base64_int8` output is decodable into vectors.
16+
*/
17+
class PerplexityBodyFormat implements CanMapRequestBody
18+
{
19+
public const string ENCODING_FORMAT = 'base64_int8';
20+
21+
public function __construct(
22+
private readonly EmbeddingsConfig $config
23+
) {}
24+
25+
#[\Override]
26+
public function toRequestBody(EmbeddingsRequest $request): array
27+
{
28+
$model = $request->model() ?: $this->config->model;
29+
$options = $request->options();
30+
$encoding = $options['encoding_format'] ?? self::ENCODING_FORMAT;
31+
if ($encoding !== self::ENCODING_FORMAT) {
32+
throw new InvalidArgumentException(
33+
"Perplexity embeddings support only the '".self::ENCODING_FORMAT."' encoding format.",
34+
);
35+
}
36+
$inputs = $request->inputs();
37+
38+
return array_filter(array_merge($options, [
39+
'input' => match (true) {
40+
PerplexityModels::isContextualized($model) => [$inputs],
41+
default => $inputs,
42+
},
43+
'model' => $model,
44+
'encoding_format' => self::ENCODING_FORMAT,
45+
]), static fn (mixed $value): bool => (bool) $value);
46+
}
47+
}
Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
<?php declare(strict_types=1);
2+
3+
namespace Cognesy\Polyglot\Embeddings\Drivers\Perplexity;
4+
5+
use Cognesy\Http\Contracts\CanSendHttpRequests;
6+
use Cognesy\Polyglot\Embeddings\Config\EmbeddingsConfig;
7+
use Cognesy\Polyglot\Embeddings\Contracts\EmbedRequestAdapter;
8+
use Cognesy\Polyglot\Embeddings\Contracts\EmbedResponseAdapter;
9+
use Cognesy\Polyglot\Embeddings\Drivers\BaseEmbedDriver;
10+
use Cognesy\Polyglot\Embeddings\Drivers\OpenAI\OpenAIUsageFormat;
11+
use Psr\EventDispatcher\EventDispatcherInterface;
12+
13+
class PerplexityDriver extends BaseEmbedDriver
14+
{
15+
protected EmbedRequestAdapter $requestAdapter;
16+
protected EmbedResponseAdapter $responseAdapter;
17+
18+
public function __construct(
19+
protected EmbeddingsConfig $config,
20+
protected CanSendHttpRequests $httpClient,
21+
protected EventDispatcherInterface $events,
22+
) {
23+
$requestAdapter = new PerplexityRequestAdapter(
24+
$config,
25+
new PerplexityBodyFormat($config)
26+
);
27+
$responseAdapter = new PerplexityResponseAdapter(
28+
new OpenAIUsageFormat()
29+
);
30+
parent::__construct($config, $httpClient, $events, $requestAdapter, $responseAdapter);
31+
}
32+
}

0 commit comments

Comments
 (0)