Should we include advanced functionality (statistics, anomaly detection) as modules as a part of the app?

[breznak]

..I got asked this from some feedback, basically some users would be interested in out-of-the-box advanced functionality, such as some statistics, anomaly detection (HTM, outliner detection), ...

For us it would mean switching from a simple visualization task to a more broad use-case monitoring and anomaly detection system.

I think this would be interesting if we still stick with the JS-only requirement and provide a very simple user experience, running from a browser.

I could imagine this as (optional) modules, that can be added as new series to the graph. For example:

• HTM anomaly model (JS-based)
• statistics https://github.com/nupic-community/nupic.visualizations/labels/statistics
• ...

Our goal should not be to develop the functionality ourselves, but rather reuse existing projects, only setting them up and integrating them for a convenient user-experience in the app.

[breznak]

This is a big change in our focus, but imho quite interesting and useful; what do you think @jefffohl ?

[divyangashah]

I think you should include anomaly detection on batch data with streaming data. It would not be a big change. what do you say @breznak?
I think its available even currently but not mentioned in documents. is it so?

[breznak]

@divyangashah any modules (eg. anomaly detection model, HTM) are currently not part of the system. What is present is the support for NuPIC/OPF format, so when you run anomaly detection on a NuPIC model, you can easily load the results here and show the anomaly score as needed.

It is a question of this Issue whether we should support "running built-in models on the data": it would mean a lot of work and shift from a clear, single use-case (visualizations) to a "do it all" aim. On the other hand, it could simplify life for the users.

[divyangashah]

@breznak As per my knowledge NAB - Numenta Anomaly Detection supports on streaming data.
but, is it supporting on batch data.
for. ex. I have a csv file with timestamp,pageviews than can I detect anomaly in that or can predict future pageviews using NAB?

[breznak]

NAB is a benchmark that is evaluated on streaming datasets, but it does not provide functionality to online monitor/predict values on a live stream of data. See RiverVIew for that. For your examples: if you modify your CSV for NAB-specific "format", you can detect anomalies in it with HTM and other algorithms. You could as well run a simple NuPIC model (see hotgym examples, videos) to detect anomalies in data using HTM.

[divyangashah]

@breznak thanks for response.
Actually, I was confuse about I can use NAB for my work or not and I was not getting proper response on numenta mailing list.
👍

[breznak]

(not saying how, but adding the need to decide to Milestone 2.)

[breznak]

htm-community/htm.JavaScript#2 is a JS port of HTM, not sure how much feature parity has been reached with the official NuPIC, but would be interesting project for a module

[jefffohl]

I am not sure how active Raf's project is. That said, it would be neat to have a server-side JS implementation of HTM. I started a few projects intending to do that, but it is a really big task, and right now my projects are stalled out.