Insyra - Create for the Next

繁體中文 | English

Insyra is a complete data analysis ecosystem for Go, spanning data loading and cleaning, statistics, visualization, machine learning, and deep learning. Every layer is checked against a reference implementation: statistics against R, models against scikit-learn, neural networks against PyTorch and onnxruntime. GPU acceleration, parallel processing, Python interop, and a CLI with its own REPL come built in.

Official Website: https://insyra.hazelnut-paradise.com

Documentation: https://hazelnutparadise.github.io/insyra/

Go.dev Package: https://pkg.go.dev/github.com/HazelnutParadise/insyra

Note

This project is evolving rapidly—please star and watch the repository to stay up to date with the latest changes!

Fast, Lovely, Easy To Use

The Insyra library is a dynamic and versatile tool designed for managing and analyzing data in Go. It offers a rich set of features for data manipulation, statistical calculations, data visualization, and more, making it an essential toolkit for developers handling complex data structures.

Tip

isr package provides Syntax Sugar!
Use isr for convenient, readable code, and the insyra main package where performance matters. The two mix freely.
For more details, please refer to the Documentation.

Note

If some functions or methods in the documentation are not working, it may be because the feature is not yet included in the latest release. Please refer to the documentation in the source code of the corresponding version in Releases.

Important

For any functions or methods not explicitly listed in Insyra documents, it indicates that the feature is still under active development. These experimental features might provide unstable results.
Please refer to our latest updates in Docs folder for more details.

Machine Learning and Deep Learning in Pure Go

You no longer need to leave Go to train a model. There is no Python runtime behind these packages, no cgo, and no external inference engine:

• ml gives you scikit-learn-style modeling: regressions (linear, ridge, lasso, logistic), decision trees, random forests, gradient boosting, pipelines, cross-validation, and grid search. Every estimator is verified against scikit-learn, R, or statsmodels, and fitted models export to ONNX.
• nn is a complete neural network engine:
  • Run real models. Published ONNX checkpoints load and run unmodified: MobileNetV2, MiniLM (a BERT-class encoder), FCN-ResNet50, fast-neural-style, and tiny-YOLOv3, each verified against onnxruntime.
  • Train in Go. An autodiff tape and a Sequential layer API (Dense, Conv2D, BatchNorm, MultiHeadAttention, and more) with AdamW, learning-rate schedules, and dropout. Gradients and optimizer steps match PyTorch.
  • Keep your weights portable. SafeTensors files load and save with PyTorch-compatible naming, so weights move freely between Go and torch, and trained models export back to ONNX.
  • Use the GPU without thinking about it. Large matrix products run on Metal, Vulkan, or DirectX 12 through a pure-Go WebGPU backend, fall back to a bit-identical CPU path when no device is available, and switch off with one line.

tape := nn.NewTape(42)
model, _ := nn.NewSequential(tape,
    nn.Dense(784, 128), nn.ReLU(), nn.Dropout(0.2), nn.Dense(128, 10),
)
logits, _ := model.Forward(tape, batch)
loss, _ := tape.SoftmaxCrossEntropy(logits, labels)
tape.Backward(loss)
tape.AdamW(1e-3, 1e-2)

AI / Agent Skills

This repository includes agent skills:

• skills/insyra: teaches AI agents to use Insyra in Go code: when to reach for it, how to think about DataList/DataTable, the conventions that hold across the library, and how to look up the exact API for the version a project uses.
• skills/use-insyra-cli: teaches agents to work through the Insyra CLI, REPL, .isr scripts and Go DSL: which mode to choose, how sessions and environments behave, how to keep work reproducible, and how to find each command's exact usage with insyra help.

Install the skills with:

npx skills add HazelnutParadise/insyra/skills

Quick picker:

• Use skills/insyra when the task is to write or modify Go code using Insyra APIs.
• Use skills/use-insyra-cli when the task should be done via insyra commands, REPL, or .isr scripts.
• Use both when you need a hybrid flow (CLI prototyping first, then productionize in Go code).

It is platform-agnostic and can be used with OpenClaw, Claude Code, opencode, or any skill-capable agent runtime.

Example prompts:

• "Use insyra to read data.csv, add a derived column with CCL, and export to output.csv."
• "Use insyra DataList to compute mean/std and show a quick preview."

Idensyra

We provide a mini Go IDE, Idensyra, which aims to make data analysis even more easier (though Insyra has already made it very easy).

Idensyra comes with Insyra pre-installed, and allows you to run Go code without installing Go environment!

Know more about Idensyra

Syralit

Want to turn your analysis into an interactive web app? We also build Syralit, a Go-native, Streamlit-inspired framework for data apps, dashboards, and AI tools. Write pure Go and get a live web UI, without touching JavaScript or any frontend build tooling.

Syralit ships with first-class Insyra integration: render DataTable/DataList directly, chart your data, and even run Insyra DSL scripts inside your app.

Know more about Syralit

Getting Started

Start Here: Guided Tutorials

If you want a practical, end-to-end way to learn Insyra, start with the guided tutorials.

• Tutorial hub: Docs/tutorials/README.md
• Featured tutorial: Sales Analysis End-to-End
• New tutorial tracks: data quality, parquet streaming, A/B statistics, RFM+CAI segmentation, yfinance trend, interactive plot dashboards, static gplot reports, LP capacity planning, Python + parallel batch.

The featured tutorial walks through a full workflow: CSV setup -> DataTable loading -> CCL enrichment -> sorting -> KPI aggregation -> CSV export.

For those new to Golang

Tip

Jump to Installation or Quick Example if you are familiar with Go.

1. Download and install Golang from here.

2. Set up your editor, we recommend using VSCode. Or even lighter weight, Idensyra.

3. Open or create a folder for your project, and open it in the editor.

4. Create a new project by running the following command:

   go mod init your_project_name

5. Install Insyra:

   go get github.com/HazelnutParadise/insyra/allpkgs

6. Create a new file, e.g., main.go, and write the following code:

   package main
   
   import (
       "fmt"
       "github.com/HazelnutParadise/insyra"
   )
   
   func main() {
       // Your code here
   }

7. Run your project:

   go run main.go

Installation

• To start using Insyra, install it with the following command:

  go get github.com/HazelnutParadise/insyra/allpkgs

• To use the optional acceleration runtime surface only:

  go get github.com/HazelnutParadise/insyra/accel

  accel is also included in allpkgs, so the standard install already covers it.

• Update Insyra to the latest version:

  go get -u github.com/HazelnutParadise/insyra/allpkgs

  or

  go get -u github.com/HazelnutParadise/insyra/allpkgs@latest

Quick Example

package main

import (
    "fmt"
    "github.com/HazelnutParadise/insyra"
)

func main() {
    dl := insyra.NewDataList(1, 2, 3, 4, 5)
    dl.Append(6)
    fmt.Println("DataList:", dl.Data())
    fmt.Println("Mean:", dl.Mean())
}

Syntactic Sugar

It is strongly recommended to use syntactic sugar since it is much more power and easier to use. For example, the above code can be written as:

package main

import (
 "fmt"

 "github.com/HazelnutParadise/insyra/isr"
)

func main() {
 dl := isr.DL.Of(1, 2, 3, 4, 5)
 dl.Append(6)
 dl.Show()
 fmt.Println("Mean:", dl.Mean())
}

To use the syntactic sugar, import github.com/HazelnutParadise/insyra/isr.

Console Preview with insyra.Show

Need a quick labelled look at any showable structure (like DataTable or DataList)? Use the package-level Show helper, which delegates to ShowRange under the hood and supports the same range arguments:

func main() {
    dt := insyra.NewDataTable(
        insyra.NewDataList("Alice", "Bob", "Charlie").SetName("Name"),
        insyra.NewDataList(28, 34, 29).SetName("Age"),
    ).SetName("Team Members")

    insyra.Show("Preview", dt, 2) // First two rows
}

Configuration

See Docs/Configuration.md.

CLI Quick Examples

Install the CLI (recommended):

go install github.com/HazelnutParadise/insyra/cmd/insyra@latest

The binary is installed to $GOBIN (or $GOPATH/bin if $GOBIN is not set).

Tip

On Windows, if insyra is not found, add %USERPROFILE%\\go\\bin (or your %GOBIN%) to PATH, then reopen your terminal.

Start REPL:

insyra

Run commands directly (non-REPL):

insyra newdl 1 2 3 4 5 as x
insyra mean x

Advanced command examples:

# Regression
insyra regression linear y x1 x2 as reg

# Hypothesis test
insyra ttest two group_a group_b equal

# Plot
insyra plot line sales save sales.html

# Fetch (Yahoo Finance)
insyra fetch yahoo AAPL quote as q

# Partial Parquet load (selected columns + row groups)
insyra load parquet data.parquet cols id,amount,status rowgroups 0,1 as t

# Reproducible sampling and train/test split
insyra sample t frac 0.1 seed 42 as preview
insyra split t train 0.8 seed 42 as train test

Tip

Use --env <name> to isolate analysis contexts, e.g. insyra --env exp1.

For full CLI + DSL documentation, see Docs/cli-dsl.md.

Thread Safety and Defensive Copies

• Defensive copies: Insyra returns defensive copies for all public data accessors. Any method that exposes internal slices, maps, or other mutable structures returns a copy so callers cannot mutate internal state unintentionally.
• Atomic operations: For safe concurrent multi-step operations, use the helper AtomicDo. AtomicDo serializes all operations for an instance via a sync.Mutex plus a goroutine-id holder (using petermattis/goid) for fast same-goroutine re-entry detection (see atomic.go). Per-call overhead is ~30 ns; same-goroutine re-entry runs inline without re-acquiring the lock.

DataList

The DataList is the core structure in Insyra, enabling the storage, management, and analysis of dynamic data collections. It offers various methods for data manipulation and statistical analysis.

For a complete list of methods and features, please refer to the DataList Documentation.

DataTable

The DataTable structure provides a tabular data representation, allowing for the storage and manipulation of data in a structured format. It offers methods for data filtering, sorting, and aggregation, making it a powerful tool for data analysis.

You can also convert between DataTables and CSV files with simply one line of code, enabling seamless integration with external data sources.

Error Handling (instance-level)

Both DataList and DataTable support instance-level error tracking for fluent/chained operations. Use Err() to obtain the last error on the instance (returns *ErrorInfo or nil) and ClearErr() to clear it.

Example:

// DataList example
dl := insyra.NewDataList(1,2,3).Sort().Reverse()
if err := dl.Err(); err != nil {
    fmt.Println("Error:", err.Message)
    dl.ClearErr()
}

// DataTable example (pseudo-args shown)
dt := insyra.NewDataTable(insyra.NewDataList(1), insyra.NewDataList(2)).SortBy(/*config*/)
if err := dt.Err(); err != nil {
    fmt.Println("Error:", err.Message)
    dt.ClearErr()
}

For more details, see the DataList Documentation and DataTable Documentation.

Column Calculation Language (CCL)

Insyra features a powerful Column Calculation Language (CCL) that works just like Excel formulas!

With CCL, you can:

• Create calculated columns using familiar Excel-like syntax
• Reference columns using Excel-style notation (A, B, C...)
• Use conditional logic with IF, AND, OR, and CASE functions
• Perform mathematical operations and string manipulations
• Execute chained comparisons like 1 < A <= 10 for range checks
• Access specific rows using the . operator (e.g., A.0) and reference all columns with @
• Use aggregate functions like SUM, AVG, COUNT, MAX, and MIN

// Add a column that classifies data based on values in column A
dt.AddColUsingCCL("category", "IF(A > 90, 'Excellent', IF(A > 70, 'Good', 'Average'))")

// Perform calculations just like in Excel
dt.AddColUsingCCL("total", "A + B + C")
dt.AddColUsingCCL("average", "AVG(A + B + C)")

// Use aggregate functions on rows or columns
dt.AddColUsingCCL("row_sum", "SUM(@.0)")

// Use range checks with chained comparisons (try this in Excel!)
dt.AddColUsingCCL("in_range", "IF(10 <= A <= 20, 'Yes', 'No')")

Parquet Integration

CCL can be applied directly during Parquet file reading to filter data at the source:

// Filter rows while reading - only matching rows are loaded into memory
dt, err := parquet.FilterWithCCL(ctx, "sales_data.parquet", "(['amount'] > 1000) && (['status'] = 'Active')")

// Apply CCL transformations directly on parquet files (streaming mode)
err := parquet.ApplyCCL(ctx, "data.parquet", "NEW('total') = A + B + C")

This approach reduces memory usage when working with large datasets by processing data in batches.

For a complete guide to CCL syntax and features, see the CCL Documentation.

For a complete list of DataTable methods and features, please refer to the DataTable Documentation.

Exact Decimals

When a number has to stay exact — money, rates, anything where 0.1 + 0.2 must be 0.3 — use github.com/TimLai666/go-decimal. It is the decimal type Insyra is built around: every finance function takes and returns it, and a Parquet Decimal128 column reads as it.

import "github.com/TimLai666/go-decimal/decimal"

ctx := decimal.Context{Scale: 2, Mode: decimal.RoundingModeHalfEven}
prices := []decimal.Decimal{
    decimal.MustParse(ctx, "19.99"),
    decimal.MustParse(ctx, "5.01"),
}
dl := insyra.NewDataList(prices) // one cell per price

How a cell treats a decimal (it sorts by value, but Mean and Sum need it converted first), and why this package rather than another, compared against the current versions of the alternatives, is in Exact Decimals.

Packages

Insyra also provides several expansion packages, each focusing on a specific aspect of data analysis.

Package	Description
isr	Syntactic sugar over Insyra for convenient code; use the main package where performance matters.
stats	Statistical functions for data analysis: skewness, kurtosis, moment calculations, and more.
ml	scikit-learn-style machine learning: regressions, trees, forests, boosting, pipelines, and model selection, verified against scikit-learn and R, with ONNX export.
nn	Pure-Go neural networks: runs real ONNX models verified against onnxruntime, trains with a PyTorch-verified tape and layer API, SafeTensors in/out, GPU-accelerated MatMul.
parallel	Parallel processing for data manipulation; runs any function and auto-waits for all goroutines.
accel	Opt-in GPU acceleration: device discovery, typed columnar projection, and real column reductions on a GPU. Pure Go, no CGO, nothing extra to install.
plot	Data visualization wrapping go-echarts.
gplot	Static charts via gonum/plot — fast, no Chrome, supports function plots.
csvxl	Work with Excel and CSV files (e.g. convert CSV to Excel).
parquet	Apache Parquet read/write, deeply integrated with DataTable/DataList; streaming, column-level reads, CCL filtering.
mkt	Marketing analytics: RFM, Customer Activity Index, and market-basket analysis.
finance	High-precision fixed-point finance: TVM, NPV/IRR, depreciation, bond pricing, and amortization schedules.
quant	Quantitative finance for strategy/backtest evaluation: Sharpe ratio, VaR/CVaR, Sortino and Calmar ratios, max drawdown, annualized return, market beta/CAPM, European option pricing and greeks, implied volatility, multi-factor attribution, PBO (CSCV), Deflated Sharpe Ratio, walk-forward validation, block-bootstrap path simulation with percentile bands, and mean-variance portfolio optimization with an efficient-frontier sweep.
quant/ta	Technical-analysis tools to test, not to trust: moving averages, MACD, KD, RSI, Bollinger bands, ATR, volume and breadth indicators, turning points and regimes, chart patterns, gaps and a shuffled-price control, with what our own study found about them
py	Run Python from Go with no manual environment setup; pass variables both ways.
pd	Pandas-like DataFrame helpers built on gpandas, with DataTable conversion.
datafetch	Easy data fetching: Google Maps store reviews, Yahoo Finance, Taiwan reverse geocoding, and TWSE/TPEx stock data.
lpgen	Generate linear programming (LP) models and export them as .lp files.
lp	Fully automatic LP solver using GLPK.
engine	Re-exports selected Insyra internals for reuse in other projects.

Advanced Usage

Beyond basic usage, Insyra provides extensive capabilities for handling different data types and performing complex statistical operations. Explore more in the detailed documentation.

Changelog

What is coming in the next release: CHANGELOG.md. Everything already published: GitHub Releases.

Contributing

Contributions are welcome! You can contribute to Insyra by:

• Issues: Reporting issues or suggesting new features.
• Pull Requests: Submitting pull requests to enhance the library.
• Discussions: Sharing your feedback and ideas to improve the project.

Contributors

License

Insyra is licensed under the MIT License. See the LICENSE file for more information.