Skip to content

What is FastTensors? How to set tensor_parallel flag? #13

Description

@esherriff

Please can you explain what the "FastTensors" option does, or better still give it a name that actually corresponds to a parameter on Tabby API's model load endpoint.

Is this related to the tensor_parallel parameter or not? If not how do I set that flag from this extension?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions