This project fine-tunes FinBERT for cryptocurrency-specific sentiment analysis. The main idea was to build something that actually understands crypto news, not just generic financial sentiment. Regular sentiment models don't really get the nuances of how crypto Twitter talks or how news impacts token prices differently than traditional stocks.
Financial sentiment models exist, but they're trained on traditional finance. Crypto has its own vocabulary - "rugs", "pumps", "diamond hands", DeFi protocols, L2s, etc. FinBERT doesn't know these terms out of the box. So I built a pipeline to:
- Collect crypto news from multiple sources
- Label it using multiple LLMs for consensus
- Fine-tune FinBERT on this crypto-specific dataset
- Analyze what tokens are missing from the vocabulary
I'm pulling news from three sources:
CryptoPanic API - The main source. Fetches news for 14 major assets (BTC, ETH, SOL, NEAR, ICP, IMX, DYDX, ALGO, XRP, SHIB, LINK, BNB, SAND, DOGE). It's rate-limited to avoid hitting API limits, and uses async requests to speed things up.
CryptoNews API - Secondary source focusing on BTC, XRP, and DOGE. Similar setup with rate limiting.
Pre-scraped data - Had some data already scraped that included engagement metrics (likes, comments, etc.). This gets merged in too.
All sources go through duplicate detection using xxhash. The merging process normalizes everything into a single format - 69 columns tracking publication dates, titles, descriptions, URLs, sources, and metadata.
This was the interesting part. Instead of manually labeling thousands of articles (no thanks), I used three different LLMs to create a consensus label:
- GPT-4o-mini - Fast and cheap, good baseline
- Gemini 2.5 Flash - Different training, different perspective
- Mistral Medium - Open-weights model, less US-centric bias
Each model scores sentiment from -5 to +5 on three dimensions:
- Overall sentiment (what's the tone?)
- Short-term impact (will this move prices today?)
- Long-term impact (does this matter in 6 months?)
The models also extract which assets are mentioned and classify intensity (extreme, very high, moderate, mild, weak, balanced). Final labels come from averaging the three models' scores and mapping to negative/neutral/positive classes.
Why three models? Single-model labeling is biased. The consensus approach gives more reliable ground truth. If all three models agree something is negative, it probably is. If they disagree, it's actually neutral.
The training pipeline is in training_model.ipynb. Here's what happens:
- Data prep - Combine title and description into single text field, clean it up (remove emojis, normalize unicode)
- Split - 80/20 train/validation split, stratified by class to keep label distribution balanced
- Load FinBERT - Start with ProsusAI's pre-trained finbert model
- Fine-tune - 3 epochs, batch size 8, learning rate 2e-5, max length 512 tokens
- Evaluate - Track accuracy, F1, precision, recall on validation set
- Save - Model gets timestamped and saved to
trained_models/
Training uses early stopping with patience=3, so if validation loss doesn't improve for 3 checks, it stops. This prevents overfitting.
One thing I wanted to check was whether FinBERT's tokenizer actually knows crypto vocabulary. Turns out, it doesn't know a lot of terms. So I built two scripts:
token_check_fixed.py - Identifies words that get sub-tokenized (broken into pieces). For example, if "DYDX" becomes ["DY", "##DX"], that's inefficient and the model probably doesn't understand it as a single concept.
token_relevance_analyzer.py - Uses GPT to score tokens on relevance to crypto markets (1-10 scale). This helps prioritize which tokens to add to the vocabulary. A term like "staking" scores 10/10, while random crypto slang might score lower.
The analysis saves suggested tokens to CSV.
.
├── fetcher/
│ └── newsfetcher.py # CryptoPanic API fetcher
├── snapi_lls/
│ └── snapi.py # CryptoNews API fetcher
├── scraped/
│ └── scraped_prepration.ipynb # Pre-scraped data prep
├── sentiment_a/
│ └── structured.py # Multi-LLM sentiment labeling
├── training/
│ ├── data_preparation.py # Train/val splitting
│ ├── dataset_merger.py # Merge all data sources
│ ├── model_utils.py # FinBERT training logic
│ ├── token_check_fixed.py # Tokenizer vocab analysis
│ └── token_relevance_analyzer.py # GPT-based token scoring
├── hashsaver/
│ └── hashsaver.py # Duplicate detection
├── load_dataset.py # Dataset loader
└── training_model.ipynb # Main training pipeline
You'll need API keys in a .env file:
panic_api_key=your_cryptopanic_key
sn_lls_api=your_cryptonews_key
OPENAI_API_KEY=your_openai_key
GEMINI_API_KEY=your_gemini_key (optional)
MISTRAL_API_KEY=your_mistral_key (optional)
Dependencies (install these):
pip install torch transformers pandas numpy scikit-learn aiohttp aiolimiter xxhash langchain langchain-openai langchain-google-genai langchain-mistralai pydantic python-dotenv emoji
The main workflow is in training_model.ipynb. Run the cells in order:
- Fetch news from APIs
- Merge datasets
- Generate sentiment labels with multi-LLM consensus
- Prepare training data
- Train FinBERT
- Evaluate and save model
Once trained, you can load the model from trained_models/finbert_finetuned_YYYYMMDD_HHMMSS/ and use it for inference on new crypto news.
- Data fetching is solid. Async + rate limiting prevents API issues.
- Multi-LLM labeling works well. Consensus is much more reliable than single-model annotations.
- FinBERT training converges. Validation metrics look reasonable.
- Token analysis identified hundreds of missing crypto terms.
FinBERT is pre-trained on financial text (earnings calls, SEC filings, news). It already understands concepts like "earnings beat", "guidance", "bull market". This makes it a better starting point than vanilla BERT. The fine-tuning teaches it crypto-specific stuff on top of that financial foundation.
Haven't done extensive benchmarking yet, but initial tests show the fine-tuned model outperforms the base FinBERT on crypto news. The biggest improvements are on neutral class (base model tends to be too confident about positive/negative) and on crypto-specific terminology.
The model saves timestamped checkpoints, so you can compare different training runs.
Do whatever you want with this. If it's useful, cool. If you improve it, even better.