Skip to content

YeDawa/Scimon

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

1,004 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation


License: MIT Built with Rust Platforms Documentation

What is Scimon?

Scimon is a fast, Rust-powered command-line tool for building document collections from a single, declarative list. You describe what you want in the Scimon language (.mon) and the interpreter does the rest β€” downloading papers, rendering Markdown and LaTeX to PDF, capturing AI conversations, generating QR codes, and more.

What began as a batch PDF downloader has grown into a small, friendly language: the syntax is intuitive and quick to write, with a clear separation between variables (single-line, e.g. path "...") and blocks (multi-line, e.g. downloads { ... }), all processed top to bottom.

Features

  • πŸ“₯ Batch downloads β€” list URLs and fetch them all, with per-line renaming (as "name.pdf"), skipping (!ignore), archive extraction (!unzip) and subfolders via group "name" { ... }.
  • πŸ” Ranges & fallbacks β€” expand a numeric range into many downloads ({2203.08877..08880}), list mirror URLs with ||, and retry flaky ones with !retry(3).
  • 🧩 Variables, functions, loops & conditionals β€” keep lists DRY with @var name "..." (used as ${name}), reusable fn name(args) { ... } templates, for x in [...] { ... } expansion, and if / else conditional blocks.
  • 🌐 Smart providers β€” Arxiv, Sci-Hub, Wikipedia/Wikisource, GitHub/GitLab and more are handled automatically.
  • πŸ’¬ AI conversations to PDF β€” paste a ChatGPT or Gemini share link and Scimon scrapes, cleans, and prints it (images inlined).
  • πŸ€– AI-generated documents β€” describe what you want in an ai { ... } block and Scimon writes the files for you via OpenRouter, as Markdown or rendered straight to PDF.
  • πŸ“ Built-in LaTeX compiler β€” turn .tex files into PDF with no TeX distribution installed (theorems, bibliography, acronyms, TikZ, pgfplots, and more).
  • πŸ“ Markdown rendering β€” render Markdown to styled PDF with MathJax and Mermaid support.
  • πŸ”’ Math to image β€” render formulas straight to PNG.
  • πŸ”— Merge PDFs β€” combine many PDFs into one with a glob or an ordered list (merge "papers/*.pdf" > "out.pdf", or merge [ "a.pdf", "b.pdf" ] > "out.pdf").
  • βœ‚οΈ Split PDFs β€” split a PDF into one file per page (split "doc.pdf" > "pages/page-{n}.pdf").
  • πŸ”ƒ Rotate PDFs β€” rotate every page by a multiple of 90Β° (rotate "scan.pdf" 90 > "out.pdf").
  • πŸ’§ Watermark PDFs β€” stamp every page with text or an image (watermark "doc.pdf" "DRAFT" > "out.pdf", or ... image "logo.png" > ...).
  • πŸ”„ Convert files β€” turn local files into other formats by extension (convert "doc.md" > "doc.pdf"): Markdownβ†’PDF/HTML/EPUB, LaTeXβ†’PDF, HTMLβ†’PDF.
  • πŸ”³ QR codes & covers β€” generate QR codes and extract document covers.
  • πŸ—œοΈ Compression & scripts β€” zip output folders and run Python/JavaScript/TypeScript steps (with a secure-by-default runner).
  • 🌍 Built-in web server β€” browse and preview the generated files in your browser (lightbox for images/PDFs, dark mode), via the serve command or server "PORT" in a list.
  • πŸ“¦ Composable packages β€” import "..." splices another list into yours from a local file, a remote URL, or a Monlib package.
  • 🎁 Distributable bundles β€” pack a list, its imported .mon files and its license into a single .scpkg with scimon pack, then scimon run it anywhere.

Requirements

  • Rust & Cargo (to build from source).
  • A Chromium/Chrome install β€” used to render HTML/Markdown/LaTeX to PDF.
  • pdfium binaries β€” only if you use the covers feature.

Installation

Build from source:

git clone https://github.com/YeDawa/Scimon.git
cd Scimon
cargo build --release

The binary is produced at target/release/scimon. See the build guide for details.

Quick start

Create a file named scimon.mon:

// where the files are saved
@var path "downloads/"

path "${path}"

downloads {
    https://arxiv.org/pdf/2203.08877 as "arxiv_paper.pdf"
    https://chatgpt.com/share/67c3f647-0bac-8005-abbb-012c3c1dafcc as "chat.pdf"
}

Tip

Scimon supports // line comments and /* ... */ block comments β€” see the comments guide.

Run it:

scimon run scimon.mon

A fuller example

Package metadata lives in a package.yml next to the list (like a crate's Cargo.toml):

# package.yml
name: "Scimon"
description: "A simple and powerful tool for downloading files, generating QR codes, compressing folders, and more."
author: "YeDawa"
license: "MIT"
privacy: "Public"
homepage: "https://scimon.dev"
// scimon.mon
@var path "downloads/"

fn arxiv(id, name) {
    https://arxiv.org/pdf/${id} as "${name}"
}

path "${path}"

copy "${path}backup/"

open "https://github.com/YeDawa/Scimon"

compress "folder.zip"

covers "${path}covers/"
qrcode "${path}qrcodes/"

math "2 + 2" > "${path}math.png"
math "2 + 3" > "${path}math1.png"

print "Hello, World!"

readme "https://gist.githubusercontent.com/Kremilly/5fd360d994bb0fe108b648d0e4c9e92f/raw/1ede0877f2bd023e77674eb89f4a0eb7d8f7e7da/readme-example.md"

downloads {
    arxiv("2203.08877", "arxiv_paper.pdf")

    for id in ["2106.09685", "1706.03762"] {
        https://arxiv.org/pdf/${id} as "${id}.pdf"
    }

    https://arxiv.org/pdf/{2405.01510..01513}
    https://primary.example/spec.pdf || https://mirror.example/spec.pdf as "spec.pdf" !retry(2)
    https://example.com/dataset.zip !unzip
    https://chatgpt.com/share/67c3f647-0bac-8005-abbb-012c3c1dafcc as "chatgpt_conversation.pdf"
    https://arxiv.org/pdf/2405.01513 !ignore
    https://www.sci-hub.se/10.1626/JCS.66.427
    https://cs.uwaterloo.ca/~jimmylin/publications/Busch_etal_ICDE2012.pdf

    group "readmes" {
        https://raw.githubusercontent.com/facebook/react/main/README.md
        https://raw.githubusercontent.com/h4cknlearn/architecture101/main/README.md !ignore
    }
    https://pt.wikisource.org/wiki/Manifesto_da_Guerrilha_do_Livre_Acesso !ignore
}

merge "${path}*.pdf" > "${path}all.pdf"

split "${path}all.pdf" > "${path}pages/page-{n}.pdf"

rotate "${path}all.pdf" 90 > "${path}all-rotated.pdf"

watermark "${path}all.pdf" "CONFIDENTIAL" > "${path}all-wm.pdf"

convert "${path}report.md" > "${path}report.pdf"

ai {
    "Write a short article about the Rust programming language" as "rust.md"
    "Explain quantum computing for beginners" as "quantum.pdf" with "anthropic/claude-3.5-sonnet"
}

commands {
    https://gist.githubusercontent.com/Kremilly/e0e0db11e43269da179adab610f38bb1/raw/6820be26a936a54bac713d03deb49edf804d0b6b/index.py
}

server "8080"

Note

The ai block generates documents using OpenRouter. Set your key with scimon options write-env (or edit the .env file) so that OPENROUTER_API_KEY="..." is defined. Each entry takes a prompt and an output file (as "name.md"): use a .md name to save raw Markdown or a .pdf name to render a styled PDF. Add with "provider/model" to override the default model.

Note

Save the file as scimon.mon, then run scimon run scimon.mon. With server "8080", the generated files are served at http://127.0.0.1:8080 until you stop it with Ctrl+C.

Packages

Bundle a list and everything needed to share it into a single distributable .scpkg file (a gzip-compressed tar, like a .crate). The entry list is always main.mon, and a bundle ships the source only β€” the package.yml manifest, the LICENSE, the README, and main.mon plus every .mon it pulls in through import:

scimon init                  # interactive: scaffold <name>/ with package.yml,
                             #   main.mon, README.md and a LICENSE (from SPDX)
scimon pack                  # β†’ <name>-<version>.scpkg (slugified) from main.mon
scimon info <name>.scpkg     # show a bundle's metadata and contents
scimon run <name>.scpkg      # extract a bundle and run its main.mon

scimon pull <package>          # download from Monlib, run it, drop the archive
scimon pull <package>@<version># pull a specific version

scimon init asks for the package fields and creates a new folder named after the package. scimon pack packs the current directory's main.mon. scimon pull fetches the .scpkg from Monlib (latest, or a pinned @version), runs it, and removes the downloaded archive β€” leaving only the extracted package. See the Packages guide for details.

Documentation

Full documentation is available at docs.scimon.dev.

Contributing

Contributions are welcome! Please read the contributing guide before opening an issue or pull request.

License

Licensed under the MIT License.

About

Scimon is a fast, Rust-powered command-line tool for building document collections from a single, declarative list. You describe what you want in the Scimon language (.mon) and the interpreter does the rest.

Topics

Resources

License

Contributing

Security policy

Stars

11 stars

Watchers

1 watching

Forks

Releases

No releases published

Sponsor this project

  •  

Contributors

Languages