feat: implement all improvements from plan

README.md

@@ -1,91 +1,164 @@
 # catsequences
 
-### A tool for concatenating multiple fasta alignments for supermatrix phylogenetic analyses
+### A tool for concatenating multiple FASTA alignments for supermatrix phylogenetic analyses
 
 Please reference this DOI: [![DOI](https://zenodo.org/badge/79135872.svg)](https://zenodo.org/badge/latestdoi/79135872)
 
+---
 
-Usage: `catsequences FILENAME`
+## Installation
 
-Where:
-  - `FILENAME` contains a list of files with sequences to be concatenated.
-  - The concatenated alignments are outputted to a file called `allseqs.fas`.
-  - Partition information will be written to the file `allseqs.partitions.txt`
+Download `catsequences.c` or clone the repository:
 
-The sequence names are read up to the first `.` `|` or ` ` (space) or the end of the name.
+```bash
+git clone https://github.com/ChrisCreevey/catsequences.git
+```
 
-**NOTE**: You can change the delimiters that are used to split the name by editing the code - see below for details
+Build:
 
+```bash
+cc catsequences.c -o catsequences -lm
+```
 
-----------------------------------
+Copy the executable to somewhere on your `PATH` (e.g. `~/bin`).
 
-# To install:
+---
 
-Download the file `catsequences.c` and or if you have git installed use the command:
-```
-git clone https://github.com/ChrisCreevey/catsequences.git
-```
+## Basic usage
 
-To build catsequences type:
 ```
-cc catsequences.c -o catsequences -lm
+catsequences [OPTIONS] LISTFILE
+catsequences [OPTIONS] FILE1.fasta FILE2.fasta ...
 ```
 
-Copy the executable `catsequences` to somewhere on your path like `~/bin` to have access to it from anywhere.
+**List-file mode** (original behaviour):
 
---------------------------------
+```bash
+ls *.fasta > list
+catsequences list
+```
+
+**Direct FASTA mode** (pass files on the command line):
 
-# To use:
+```bash
+catsequences gene1.fasta gene2.fasta gene3.fasta
+```
 
-Usage: 
+Glob wildcards are also accepted inside a list file:
 
 ```
-catsequences FILENAME
+genes/*.fasta
 ```
 
-  where `FILENAME` is a file containing a list of files with sequences to be concatenated.
+---
+
+## Options
+
+| Flag | Description |
+|------|-------------|
+| `-d DELIMS` | Name-delimiter characters (default: `" \|."`) — truncate the species name at the first occurrence of any character in this string |
+| `-o PREFIX` | Output filename prefix (default: `allseqs`) |
+| `-m CHAR` | Missing-data character (default: `?`; common alternatives: `-`, `N`) |
+| `-q` | Quiet — suppress all progress messages |
+| `-v` | Verbose — print per-file scan/read progress |
+| `-f FORMAT` | Partition file format: `generic` (default), `raxml`, `iqtree`, `nexus` |
+| `-F FORMAT` | Alignment output format: `fasta` (default), `nexus`, `phylip` |
+| `-O` | Write an occupancy matrix CSV |
+| `--min-taxa N` | Skip any gene file where fewer than `N` × 100 % of species are present (e.g. `0.5` = require at least 50 % occupancy) |
+| `--on-duplicate MODE` | What to do when a species name appears more than once in a file: `warn` (default — keep first, print a message), `error` (abort), `keep-last` (overwrite with the last copy) |
+| `--include FILE` | Only output species whose names appear in `FILE` (one name per line, without `>`) |
+| `--exclude FILE` | Exclude species whose names appear in `FILE` (one name per line, without `>`) |
+
+---
+
+## Outputs
+
+| File | Description |
+|------|-------------|
+| `<PREFIX>.fas` / `.nex` / `.phy` | Concatenated alignment |
+| `<PREFIX>.partitions.txt` | Partition coordinates |
+| `<PREFIX>.occupancy.txt` | Per-species and overall occupancy summary |
+| `<PREFIX>.occupancy.csv` | Occupancy matrix — present (1) / absent (0) per gene per species (written only with `-O`) |
 
-  - The concatenated alignments are outputted to a file called `allseqs.fas`
-
-  - Partition information will be written to the file `allseqs.partitions.txt`
+Species that are absent from a gene file are padded with the missing-data character for that gene's full length.
 
-  - The sequence names are read up to the first `.` `|` or ` ` (space) (or the end of the name)
+---
 
-The easiest way to create the list input file is to use a command like:
+## Partition format examples
+
+### generic (default)
 ```
-ls *.fasta > list
+gene1.fasta=1-500;
+gene2.fasta=501-1200;
+```
+
+### raxml (`-f raxml`)
+```
+DNA, gene1 = 1-500
+DNA, gene2 = 501-1200
+```
+
+### iqtree (`-f iqtree`)
+```
+gene1 = 1-500
+gene2 = 501-1200
 ```
-assuming that all your input file have the extension ".fasta"
 
-The software will put in '?' for the entire length of any gene for which the species is missing.
+### nexus (`-f nexus`)
+```
+#NEXUS
+begin sets;
+charset gene1 = 1-500;
+charset gene2 = 501-1200;
+end;
+```
 
+---
 
-# Trouble Shooting
+## Examples
 
-## If the total concatenated seuquences length is larger than defined in the code you recieve an ERROR saying:
+Run with a list file, writing RAxML partitions and a NEXUS alignment:
 
+```bash
+catsequences -f raxml -F nexus -o mymatrix list
 ```
-ERROR: The total concatenated sequence length is greater than the maximum length defined (1000000)!
-    To fix this increase the value of MAXLEN on line 15 of 'catsequences.c` and recompile the tool.
+
+Require at least 50 % of species per gene, write occupancy CSV:
+
+```bash
+catsequences --min-taxa 0.5 -O -o filtered list
 ```
 
-This line you want to change is:
+Include only a subset of species:
 
+```bash
+echo -e "Homo_sapiens\nPan_troglodytes\nGorilla_gorilla" > primates.txt
+catsequences --include primates.txt -o primates list
 ```
-#define MAXLEN 1000000
+
+Change the delimiter so only `|` splits names (not `.` or space):
+
+```bash
+catsequences -d '|' list
 ```
 
-Although this means that the total length of all the seqeunces you are trying to concatentate is greater than 1 million bp. 
-Please make sure that this is as expected before changing the code.
+---
 
-## I need a different delimiter character in the sequence names than the default "|", "." or " " (space)
+## Testing
 
-Add (or remove) characters between the `" "` as needed on line 16 of `catsequences.c`:
+Build and run the regression test suite:
 
-```
-#define NAMEDELIMITER " |." 
+```bash
+cc catsequences.c -o catsequences -lm
+bash tests/run_tests.sh
 ```
 
-Then recompile (build) the tool.
+---
 
+## Notes
 
+- Species name lengths and the number of species and genes are all determined
+  dynamically at run time — there are no hard-coded limits requiring recompilation.
+- The tool reads each input file twice: once to discover species and measure
+  sequence lengths, and once to read sequence data.  This keeps peak memory
+  proportional to `(number of species) × (total concatenated length)`.