Allow materialized views to be created from Qleverfile via CLI

[azaroth42]

Per the documentation there are two ways to write a materialized view:

# During indexing if specified in Qleverfile:
qlever index
# After indexing while the server is running:
qlever materialized-view $VIEW_NAME "SELECT ... { ... }"

The combination of the two would also be very useful, when you define the SPARQL in Qleverfile, but then want to load (or delete and reload) the view from the CLI against a running server. Especially during development of the view. I suggest:

# After indexing while the server is running, and the query is specified in Qleverfile:
qlever materialized-view $VIEW_NAME

(Happy to have a go at the above, if it's useful)

Also, is there a correct way to delete a view? At the moment I just delete $DB_NAME.view.$VIEW_NAME.* after stopping the server.

[azaroth42]

Standalone code that demonstrates the proposed functionality:

import json
import shlex
import time
from qlever.qleverfile import Qleverfile
from qlever.util import run_command

qf = Qleverfile.read("indexes/Qleverfile")
views = json.loads(qf["index"]["materialized_views"])

def build_view(qf, view_name, view_query):
    sparql_endpoint = f"{qf['server']['host_name']}:{qf['server']['port']}"
    access_token = qf["server"]["access_token"]

    url = f"{sparql_endpoint}?cmd=write-materialized-view&view-name={view_name}"
    materialized_view_cmd = (
        f"curl -s {shlex.quote(url)} "
        f"-H 'Authorization: Bearer {access_token}' "
        f"-H 'Content-type: application/sparql-query' "
        f"-d {shlex.quote(view_query)}"
    )

    # Run the command (and time it).
    print(f"Creating materialized view '{view_name}'...")
    time_start = time.monotonic()
    try:
        result = run_command(materialized_view_cmd, return_output=True)
    except Exception as e:
        print(f"Creating the materialized view failed: {e}")
        return False
    time_end = time.monotonic()
    duration_seconds = round(time_end - time_start)

    # Try to parse the result (should be JSON).
    try:
        result_json = json.loads(result)
        view_name = result_json.get("materialized-view-written")
        print(
            f"Materialized view '{view_name}' created successfully "
            f"in {duration_seconds:,} seconds"
        )
    except Exception as e:
        print(f'Failed to parse JSON from "{result}": {e}')
    return True


for k, v in views.items():
    build_view(qf, k, v)

[ullingerc]

@azaroth42 Thanks for your feedback. Two quick notes on this:

• As you already write, views can be written at index build time (by qlever index) when given via the MATERIALIZED_VIEWS field in the Qleverfile, see here: https://docs.qlever.dev/qleverfile/#section-index. Writing them by giving the name later could be implemented, so I'll leave this open.
• Deleting views of a running server is implemented in Qlever, but not merged into the qlever-control CLI program yet. So far, deleting the files is sufficient. See also Add option --delete to materialized-view command #312

[ullingerc]

@azaroth42 I have now opened PR #326 for the feature you suggested.