TierBase should be split up

[0Hughman0]

Currently TierBase represents a broad range of tiers.

DataSets don't have notebooks, therefore templates and meta and highlights etc.

TierBase should be split into two Base classes - FolderTierBase and NotebookTierBase.

A FolderTier is one with no notebook.

A NotebookTier is one with notebook, meta highlights etc.

This stops dodgy null implementations of the Notebook methods for DataSets.

It should be straightforward to incorporate is instance checks into the code and also into the jl_cassini_server. Providing a type would probably be very useful and a solution to poor handling of the top and bottom of the hierarchy by jl_cassini.

[0Hughman0]

See #109/ #127 for work

[0Hughman0]

Here are the current attributes:

class TierBase(ABC):

    rank: ClassVar[int] = -1
    id_regex: ClassVar[str] = r"(\d+)"

    gui_cls: Type[BaseTierGui[Self]] = BaseTierGui

    @cached_class_prop
    def hierarchy(cls) -> List[Type[TierBase]]:

    @cached_class_prop
    def pretty_type(cls) -> str:

    @cached_class_prop
    def short_type(cls) -> str:

    @cached_class_prop
    def name_part_template(cls) -> str:

    @cached_class_prop
    def name_part_regex(cls) -> str:

    @cached_class_prop
    def parent_cls(cls) -> Union[Type[TierBase], None]:

    @cached_class_prop
    def child_cls(cls) -> Union[Type[TierBase], None]:

    @cached_class_prop
    def default_template(cls) -> Union[Path, None]:

    @cached_class_prop
    def _meta_folder_name(cls) -> str:

    def __new__(cls, *args: str, **kwargs: Dict[str, Any]) -> TierBase:

    @classmethod
    def parse_name(cls, name: str) -> Tuple[str, ...]:

    @classmethod
    def _iter_meta_dir(cls, path: Path) -> Iterator[Tuple[str, ...]]:

    _identifiers: Tuple[str, ...]
    gui: BaseTierGui[Self]
    meta: Meta

    def __init__(self: Self, *args: str):

    def setup_files(
        self, template: Union[Path, None] = None, meta: Optional[MetaDict] = None
    ) -> None:

    description: MetaAttr[str, str] = MetaAttr()
    conclusion: MetaAttr[str, str] = MetaAttr()
    started: MetaAttr[str, datetime.datetime] = MetaAttr(str_to_date, date_to_str)

    @cached_prop
    def identifiers(self) -> Tuple[str, ...]:

    @cached_prop
    def name(self) -> str:

    @cached_prop
    def id(self) -> str:

    @cached_prop
    def folder(self) -> Path:

    @cached_prop
    def meta_file(self) -> Union[Path, None]:

    @cached_prop
    def highlights_file(self) -> Union[Path, None]:

    @cached_prop
    def file(self) -> Union[Path, None]:

    @cached_prop
    def parent(self) -> Union[TierBase, None]:

    @cached_prop
    def href(self) -> Union[str, None]:

    def exists(self) -> bool:

    def get_child(self, id: str) -> TierBase:

    def serialize(self) -> MetaDict:

    def children_df(
        self, *, include: Optional[List[str]] = None, exclude: Optional[List[str]]
    ) -> Union[pd.DataFrame, None]:

    @classmethod
    def get_templates(cls) -> List[Path]:

    def render_template(self, template_path: Path) -> str:

    def open_folder(self) -> None:

    def get_highlights(self) -> Union[HighlightsType, None]:

    def add_highlight(
        self, name: str, data: HighlightType, overwrite: bool = True
    ) -> None:

    def remove_highlight(self, name: str) -> None:

    def __truediv__(self, other: Any) -> Path:

    def __getitem__(self, item: str) -> TierBase:

    def __iter__(self) -> Iterator[Any]:

    def __repr__(self) -> str:

    def _repr_html_(self) -> str:

    def __getattr__(self, item: str) -> TierBase:

    def remove_files(self) -> None:

[0Hughman0]

The full definitions of FolderTierBase and NotebookBase must have:

class FolderTierBase(ABC):

    rank: ClassVar[int] = -1
    id_regex: ClassVar[str] = r"(\d+)"

    gui_cls: Type[BaseTierGui[Self]] = BaseTierGui

    @cached_class_prop
    def hierarchy(cls) -> List[Type[TierBase]]:

    @cached_class_prop
    def pretty_type(cls) -> str:

    @cached_class_prop
    def short_type(cls) -> str:

    @cached_class_prop
    def name_part_template(cls) -> str:

    @cached_class_prop
    def name_part_regex(cls) -> str:

    @cached_class_prop
    def parent_cls(cls) -> Union[Type[TierBase], None]:

    @cached_class_prop
    def child_cls(cls) -> Union[Type[TierBase], None]:

    def __new__(cls, *args: str, **kwargs: Dict[str, Any]) -> TierBase:

    @classmethod
    def parse_name(cls, name: str) -> Tuple[str, ...]:

    _identifiers: Tuple[str, ...]
    gui: BaseTierGui[Self]
    meta: Meta

    def __init__(self: Self, *args: str):

    def setup_files(
        self, template: Union[Path, None] = None, meta: Optional[MetaDict] = None
    ) -> None:

    @cached_prop
    def identifiers(self) -> Tuple[str, ...]:

    @cached_prop
    def name(self) -> str:

    @cached_prop
    def id(self) -> str:

    @cached_prop
    def folder(self) -> Path:

    @cached_prop
    def parent(self) -> Union[TierBase, None]:

    @cached_prop
    def href(self) -> Union[str, None]:

    def exists(self) -> bool:

    def get_child(self, id: str) -> TierBase:

    def open_folder(self) -> None:

    def __truediv__(self, other: Any) -> Path:

    def __getitem__(self, item: str) -> TierBase:

    def __iter__(self) -> Iterator[Any]:

    def __repr__(self) -> str:

    def _repr_html_(self) -> str:

    def __getattr__(self, item: str) -> TierBase:

    def remove_files(self) -> None:

class NotebookTierBase(ABC):

    rank: ClassVar[int] = -1
    id_regex: ClassVar[str] = r"(\d+)"

    gui_cls: Type[BaseTierGui[Self]] = BaseTierGui

    @cached_class_prop
    def hierarchy(cls) -> List[Type[TierBase]]:

    @cached_class_prop
    def pretty_type(cls) -> str:

    @cached_class_prop
    def short_type(cls) -> str:

    @cached_class_prop
    def name_part_template(cls) -> str:

    @cached_class_prop
    def name_part_regex(cls) -> str:

    @cached_class_prop
    def parent_cls(cls) -> Union[Type[TierBase], None]:

    @cached_class_prop
    def child_cls(cls) -> Union[Type[TierBase], None]:

    @cached_class_prop
    def default_template(cls) -> Union[Path, None]:

    @cached_class_prop
    def _meta_folder_name(cls) -> str:

    def __new__(cls, *args: str, **kwargs: Dict[str, Any]) -> TierBase:

    @classmethod
    def parse_name(cls, name: str) -> Tuple[str, ...]:

    @classmethod
    def _iter_meta_dir(cls, path: Path) -> Iterator[Tuple[str, ...]]:

    _identifiers: Tuple[str, ...]
    gui: BaseTierGui[Self]
    meta: Meta

    def __init__(self: Self, *args: str):

    def setup_files(
        self, template: Union[Path, None] = None, meta: Optional[MetaDict] = None
    ) -> None:

    description: MetaAttr[str, str] = MetaAttr()
    conclusion: MetaAttr[str, str] = MetaAttr()
    started: MetaAttr[str, datetime.datetime] = MetaAttr(str_to_date, date_to_str)

    @cached_prop
    def identifiers(self) -> Tuple[str, ...]:

    @cached_prop
    def name(self) -> str:

    @cached_prop
    def id(self) -> str:

    @cached_prop
    def folder(self) -> Path:

    @cached_prop
    def meta_file(self) -> Union[Path, None]:

    @cached_prop
    def highlights_file(self) -> Union[Path, None]:

    @cached_prop
    def file(self) -> Union[Path, None]:

    @cached_prop
    def parent(self) -> Union[TierBase, None]:

    @cached_prop
    def href(self) -> Union[str, None]:

    def exists(self) -> bool:

    def get_child(self, id: str) -> TierBase:

    def serialize(self) -> MetaDict:

    def children_df(
        self, *, include: Optional[List[str]] = None, exclude: Optional[List[str]]
    ) -> Union[pd.DataFrame, None]:

    @classmethod
    def get_templates(cls) -> List[Path]:

    def render_template(self, template_path: Path) -> str:

    def open_folder(self) -> None:

    def get_highlights(self) -> Union[HighlightsType, None]:

    def add_highlight(
        self, name: str, data: HighlightType, overwrite: bool = True
    ) -> None:

    def remove_highlight(self, name: str) -> None:

    def __truediv__(self, other: Any) -> Path:

    def __getitem__(self, item: str) -> TierBase:

    def __iter__(self) -> Iterator[Any]:

    def __repr__(self) -> str:

    def _repr_html_(self) -> str:

    def __getattr__(self, item: str) -> TierBase:

    def remove_files(self) -> None:

[0Hughman0]

Need to worry about how to iterate over children.

Children need not necessarily reside directly in the parent.folder.

Children obviously know where they are, as they know who their parent is.

My idea was that this means the Tier should know where all of its' siblings are.

For the simple recursive structure:

@classmethod
def iter_siblings(cls, parent):
     pass

For the way DataSets work

@classmethod
def iter_siblings(cls, parent):
     pass

This is going to be very confusing, and maybe it's just better to make them abstract and force users to define themselves...

[0Hughman0]

Yes, this is my preferred approach.

Make these methods abstract, provide sensible implementations in the defaults, these can serve as examples as to how this should work.

[0Hughman0]

There's actually no need for all these kinda confusing 'sensible default' implementations, because the default tiers can provide them...?

Feel like there's maybe too much magic going on.

[0Hughman0]

Actually, I checked the performance, and the caching really does make quite a big difference.

[0Hughman0]

FYI I checked the performance, and the caching really does make quite a big difference.

[0Hughman0]

Define 2 ABC, one TierABC, which assumes only a folder and a position in the hierarchy and NotebookABC, which assumes a notebook and meta file.

Where it's more obvious for a user to define their own explicitly, get rid of some of the magic properties.

Specialisations, like techniques, etc. can be implemented in the defaults/tiers.py.

Home can be a subclass of TierABC as can DataSets. Experiment and WorkPackage can subclass NotebookABC.

class TierBase(ABC):

    rank: ClassVar[int] = -1  # give to Project
    id_regex: ClassVar[str] = r"(\d+)"  # keep!


    @cached_class_prop
    def hierarchy(cls) -> List[Type[TierBase]]:  # probably get rid of, just use project.heirarchy.

    @cached_class_prop
    def pretty_type(cls) -> str: # make an attribute

    @cached_class_prop
    def short_type(cls) -> str: # make an attribute

    @cached_class_prop
    def name_part_template(cls) -> str:  # make an attribute

    @cached_class_prop
    def name_part_regex(cls) -> str:  # make an attribute

    def __new__(cls, *args: str, **kwargs: Dict[str, Any]) -> TierBase:

    _identifiers: Tuple[str, ...]

    def __init__(self: Self, *args: str):

    def setup_files(
        self, template: Union[Path, None] = None, meta: Optional[MetaDict] = None
    ) -> None:

    @cached_prop
    def identifiers(self) -> Tuple[str, ...]:

    @cached_prop
    def name(self) -> str:

    @cached_prop
    def id(self) -> str:

    @cached_prop
    def folder(self) -> Path:

    @cached_prop
    def parent(self) -> Union[TierBase, None]:

    @cached_prop
    def href(self) -> Union[str, None]:

    def exists(self) -> bool:

    def open_folder(self) -> None:

    def __truediv__(self, other: Any) -> Path:

    def __getitem__(self, item: str) -> TierBase:

    def __iter__(self) -> Iterator[Any]:

    def __repr__(self) -> str:

    def _repr_html_(self) -> str:

    def __getattr__(self, item: str) -> TierBase:

    def remove_files(self) -> None:

NotebookTier

class NotebookTier(TierABC):

    rank: ClassVar[int] = -1  # give to Project
    id_regex: ClassVar[str] = r"(\d+)"  # keep!

    gui_cls: Type[BaseTierGui[Self]] = BaseTierGui  # Keep!

    @cached_class_prop
    def hierarchy(cls) -> List[Type[TierBase]]:  # probably get rid of, just use project.heirarchy.

    @cached_class_prop
    def pretty_type(cls) -> str: # make an attribute

    @cached_class_prop
    def short_type(cls) -> str: # make an attribute

    @cached_class_prop
    def name_part_template(cls) -> str:  # make an attribute

    @cached_class_prop
    def name_part_regex(cls) -> str:  # make an attribute

    @cached_class_prop
    def parent_cls(cls) -> Union[Type[TierBase], None]: # get from project

    @cached_class_prop
    def child_cls(cls) -> Union[Type[TierBase], None]: # get from project

    @cached_class_prop
    def default_template(cls) -> Union[Path, None]:  # probably fine.

    @cached_class_prop
    def _meta_folder_name(cls) -> str:  # probably fine, let's force every class to have this at least

    def __new__(cls, *args: str, **kwargs: Dict[str, Any]) -> TierBase:

    @classmethod
    def parse_name(cls, name: str) -> Tuple[str, ...]:  # give to Project

    @classmethod
    def _iter_meta_dir(cls, path: Path) -> Iterator[Tuple[str, ...]]:  # _iter_child_meta?

    _identifiers: Tuple[str, ...]
    gui: BaseTierGui[Self]
    meta: Meta

    def __init__(self: Self, *args: str):

    def setup_files(
        self, template: Union[Path, None] = None, meta: Optional[MetaDict] = None
    ) -> None:

    description: MetaAttr[str, str] = MetaAttr()  # ... do I really want this in the base class?
    conclusion: MetaAttr[str, str] = MetaAttr()
    started: MetaAttr[str, datetime.datetime] = MetaAttr(str_to_date, date_to_str)

    @cached_prop
    def identifiers(self) -> Tuple[str, ...]:

    @cached_prop
    def name(self) -> str:

    @cached_prop
    def id(self) -> str:

    @cached_prop
    def folder(self) -> Path:

    @cached_prop
    def meta_file(self) -> Union[Path, None]:

    @cached_prop
    def highlights_file(self) -> Union[Path, None]:

    @cached_prop
    def file(self) -> Union[Path, None]:

    @cached_prop
    def parent(self) -> Union[TierBase, None]:

    @cached_prop
    def href(self) -> Union[str, None]:

    def exists(self) -> bool:

    def get_child(self, id: str) -> TierBase:

    def serialize(self) -> MetaDict:

    def children_df(
        self, *, include: Optional[List[str]] = None, exclude: Optional[List[str]]
    ) -> Union[pd.DataFrame, None]:

    @classmethod
    def get_templates(cls) -> List[Path]:

    def render_template(self, template_path: Path) -> str:

    def open_folder(self) -> None:

    def get_highlights(self) -> Union[HighlightsType, None]:

    def add_highlight(
        self, name: str, data: HighlightType, overwrite: bool = True
    ) -> None:

    def remove_highlight(self, name: str) -> None:

    def __truediv__(self, other: Any) -> Path:

    def __getitem__(self, item: str) -> TierBase:

    def __iter__(self) -> Iterator[Any]:

    def __repr__(self) -> str:

    def _repr_html_(self) -> str:

    def __getattr__(self, item: str) -> TierBase:

    def remove_files(self) -> None:

[0Hughman0]

Ok I actually will add back in HomeBase. The reason for this is that I want defaults/tiers.py to act as an example for how to build your own hierarchy. We want users to inherit from NotebookBase and FolderBase, but then Home from the default hierarchy? - That's inconsistent. Plus inheriting from the default hierarchy could lead to some unexpected behaviour, for example the default implementations e.g. of name_part_template will not work because they were overwritten in the default hierarchy implementations.

Adding a HomeBase will simplify defaults/tiers.py and mean we can instruct users to only inherit from these base classes when building their own hierarchies and to look to the defaults as an example.

[0Hughman0]

Here I am again...

I wonder if we should use a mixin?

I.e. a Notebook mixin and a Meta mixin?

I think this would work?

The problem we have is that Home isn't a Notebook or a Folder and that's annoying...

[0Hughman0]

Generally I think people don't like mixins because they're a bix strange, but I feel like they fit this use case really quite well...

[0Hughman0]

It looks like this would require Protocols and probably @runtime_checkable

https://mypy.readthedocs.io/en/stable/protocols.html#defining-subprotocols-and-subclassing-protocols

[0Hughman0]

We would have to do something smart about setup_files and exists and possibly href.

[0Hughman0]

I'm closing for now. They have been split up.

Mixins maybe. People generally turn their noses up at multiple inheritance.