When trying to list the full size of the contents of each source of cpg-00016 according to this issue, only sources 1, 2 and 7 appear to have their workspace_dl content indexed:
df = (
index
.filter(pl.col("dataset_id").eq("cpg0016-jump"))
.filter(pl.col("workspace_dl").str.contains("workspace_dl"))
.select("workspace_dl", "dataset_id", "source_id")
.unique(subset=["source_id"])
.collect(streaming=True)
)
with pl.Config(fmt_str_lengths=1000, tbl_width_chars=1000):
print(df)
shape: (3, 3)
┌──────────────┬──────────────┬───────────┐
│ workspace_dl ┆ dataset_id ┆ source_id │
│ --- ┆ --- ┆ --- │
│ str ┆ str ┆ str │
╞══════════════╪══════════════╪═══════════╡
│ workspace_dl ┆ cpg0016-jump ┆ source_1 │
│ workspace_dl ┆ cpg0016-jump ┆ source_2 │
│ workspace_dl ┆ cpg0016-jump ┆ source_7 │
└──────────────┴──────────────┴───────────┘
Likely the other sources are using a different structure and are not recognized by the parser when making the index.
When trying to list the full size of the contents of each source of cpg-00016 according to this issue, only sources 1, 2 and 7 appear to have their workspace_dl content indexed:
Likely the other sources are using a different structure and are not recognized by the parser when making the index.