You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, the implementation for the MVP uses pure functional approaches with in/out Tensors as effectively just fancy data storage. To make things convenient, I think going forward we should prefer an stateful approach like PyTorch. For example, instead of calling the raw matmul and raw add from Sachit's BLAS integration, we make a dedicated Linear class that is constructed with the Tensor to the data. We extrapolate this for all of the existing functions.
Action items:
Create a slew of objects that follow the same interface, which is that they expose a callable operator() with parameters const TensorBase<T1, Ext1>& in, TensorBase<T2, Ext2>& out and void return type. For example, Linear should be callable as Linear<TensorType1, TensorType2>{t1, t2}(t_in, t_out); and TensorType1 should be a rank-2 mat along with TensorType1 as a rank-1 mat, both enforced at comptime.
Do the same for convolve, batchnorm, maxpool, maybe a special Expression-based operation (not necessary), and anything else that would be convenient and in scope.
I think it could also be really interesting to consider using non-type template T* parameters since most if not all of our weights are externally linked, although I don't know how this could be done ergonomically. Feel free to look into this or ignore.
Currently, the implementation for the MVP uses pure functional approaches with in/out Tensors as effectively just fancy data storage. To make things convenient, I think going forward we should prefer an stateful approach like PyTorch. For example, instead of calling the raw matmul and raw add from Sachit's BLAS integration, we make a dedicated Linear class that is constructed with the Tensor to the data. We extrapolate this for all of the existing functions.
Action items:
Linear<TensorType1, TensorType2>{t1, t2}(t_in, t_out);and TensorType1 should be a rank-2 mat along with TensorType1 as a rank-1 mat, both enforced at comptime.