Question
How to store scraped data?
Answer
Set an output target: --out csv|jsonl|sqlite|postgres.
For SQLite: set SQLITE_PATH=./data/insta.sqlite.
For Postgres: set POSTGRES_URL=postgres://USER:PASS@HOST:5432/DB.
Use --append for simple adds or --upsert to dedupe by post_id, comment_id, and user_id. Run migrations/ to create tables, then schedule jobs via cron or a queue (BullMQ/Celery). ETL samples are provided in /etl/ to push clean datasets downstream.
Question
How to store scraped data?
Answer
Set an output target:
--out csv|jsonl|sqlite|postgres.For SQLite: set
SQLITE_PATH=./data/insta.sqlite.For Postgres: set
POSTGRES_URL=postgres://USER:PASS@HOST:5432/DB.Use
--appendfor simple adds or--upsertto dedupe bypost_id,comment_id, anduser_id. Runmigrations/to create tables, then schedule jobs via cron or a queue (BullMQ/Celery). ETL samples are provided in/etl/to push clean datasets downstream.