Skip to content

FTS index created with duckdb 1.3 doesn't work with 1.4 #19

Description

@zeppafi

Hi!

I noticed that fts index created with DuckDB 1.3 doesn't work with 1.4 (python client).

with duckdb 1.3.0:

import duckdb
print(duckdb.__version__)

con = duckdb.connect("made_with_duckdb_1_3_0.db")
con.sql("""--sql
CREATE OR REPLACE TABLE documents (
    document_identifier VARCHAR,
    text_content VARCHAR,
    author VARCHAR,
    doc_version INTEGER
);
""")

con.sql("""--sql
--sql
INSERT INTO documents
    VALUES ('doc1',
            'The mallard is a dabbling duck that breeds throughout the temperate.',
            'Hannes Mühleisen',
            3),
           ('doc2',
            'The cat is a domestic species of small carnivorous mammal.',
            'Laurens Kuiper',
            2
           );

PRAGMA create_fts_index(
    'documents', 'document_identifier', 'text_content', 'author', overwrite = 1
);
""")

con.close()

This works with duckdb 1.3.0, but doesn't work with 1.4.0 (check images):

import duckdb
print(duckdb.__version__)
with duckdb.connect("made_with_duckdb_1_3_0.db") as con:
    display(con.sql("""--sql
        SELECT document_identifier, text_content, score
        FROM (
            SELECT *, fts_main_documents.match_bm25(
                document_identifier,
                'Muhleisen',
                fields := 'author'
            ) AS score
            FROM documents
        ) sq
        WHERE score IS NOT NULL
        AND doc_version > 2
        ORDER BY score DESC;
    """))
Image Image

Following python exception is thrown when using with 1.4.0:

CatalogException                          Traceback (most recent call last)
Cell In[1], [line 3](vscode-notebook-cell:?execution_count=1&line=3)
      1 import duckdb
      2 print(duckdb.__version__)
----> [3](vscode-notebook-cell:?execution_count=1&line=3) with duckdb.connect("made_with_duckdb_1_3_0.db") as con:
      4     display(con.sql("""--sql
      5         SELECT document_identifier, text_content, score
      6         FROM (
   (...)     16         ORDER BY score DESC;
     17     """))

Cell In[1], [line 4](vscode-notebook-cell:?execution_count=1&line=4)
      2 print(duckdb.__version__)
      3 with duckdb.connect("made_with_duckdb_1_3_0.db") as con:
----> [4](vscode-notebook-cell:?execution_count=1&line=4)     display(con.sql("""--sql
      5         SELECT document_identifier, text_content, score
      6         FROM (
      7             SELECT *, fts_main_documents.match_bm25(
      8                 document_identifier,
      9                 'Muhleisen',
     10                 fields := 'author'
     11             ) AS score
     12             FROM documents
     13         ) sq
     14         WHERE score IS NOT NULL
     15         AND doc_version > 2
     16         ORDER BY score DESC;
     17     """))

CatalogException: Catalog Error: Table with name scores does not exist!
Did you mean "documents"?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions