Before all, thank you so much, you did so great job.
0/
I downloaded + 1000 files (+ 2 TO).
Would be great to have an option be able to become a mirror.
1/
Password for go to settings :
- the input is not recognized as password by bitwarden
- and when i close page and back it does not remeber (when remember password is already checked)
2/
Downloading : when downloading hundreds of zim files
- we need click download button many times, there could be easy muti-selection and "download selected" button
- the menu block "installed, catalog, collections ..." is sitting one kilometer in the middle of the screen so you have to scroll that kilometer (better should be on top of the page)
- it starts downloading all files, and each file at 0.2 mb/s speed : perhaps do hew files simultaneously and queue all others and sort by ascending file size so small download first and fast
- perhaps use multiple columns in download section so one line can display 3 downloads, so we have shorter download block
- also perhaps 3 columns in the catalog detail block
- long blocks : partial downloads, downloads, ... perhaps would be better if we could fold/unfold this sections
- pause button on downloads, to be able to prioritize some files by pausing others
- downloading / completed : perhaps split them in two different tabs : will save space on the page
3/
Files already included in other files :
- perhaps show hierarchy tree so people know when they downloaded the big wikipedia/TED they do not need all small parts
- catalogs : if i have full wikipedia, catalog could say me that a have all articles and do not need to go inside to downolad other parts
- also user face a choice : when downloading TED the ALL TED big file is older BUT smaller TED files are fresher, i do not know if there is a way to get ALL articles + also extract fresher articles and replace older ones (perhaps there is something to do to parse all zims into a database and do smart update to keep database updated)
- also show if one big file is equal or not equal to sum of small files (ted, wikipedia) : so user has easier decision to download the big one or the small parts instead
4/
Integration with SearXNG search engine :
- this self hosted search engine can replace google, works well with AI (openwebui, openclaw ...)
- perhaps there is something to do with SearXNG team to facilitate Zimi integration with SearXNG
- also now with my integration trial all results go to "general" section in searxng, would be best that images and videos (for example TED videos) go into images and videos sections (i do not know how to do it)
- there possibly smarter way to properly integrate zimi and zim files with searxng tabs (general, images, video, science ...)
- also searxng time outs and i need to hit search multiple times then results from zimi show up (but it is already incredible to be able to search through all zim files)
- smart ranking articles : when getting zim results via SearXNG they arrive in certain order and are not ranked by revelance or date, perhaps there is something to do to analyze all this data with AI and get better ranking per date and/or revelance
5/
with many files (+1000) :
- there could be performance problems, i have redis valkey postgres mariadb ... perhaps propose possibility to use them to speed up results (i am not developer, do not know if this could help)
- i have 128gb ram : when having many files let user to choose what to load in ram/cache to speed up searches
6/ languages / images
- i an in France, so i am interested in : english, french, multi
- perhaps propose filter options in settings to only show user selected languages in GUI so all other files will be invisible and there will be less bloat
- also if i download all files in my selected languages the collections will show that i already have all files and there is nothing more to download
- when the same file is awailable with and without images, add preference setting to show one of them and filter other : i always take with images and skip no images (so no images is redundant bloat for me)
7/
- could this zim library be integrated with openwebui and other AI, an how to do it ?
- add documentation and use examples including how to use MCP with some self hosted apps
8/
- when i try to download all Canadian prepper files they do not all appear as downloaded
- updates : 5 remaining : how to see detail ?
- also one big wikipedia file appears as not updated, perhaps i messed something when downloaded files via script before having zimi (do not know how to solve this)
9/
- stats page : number of : files, searches, top 10 searches, other stats
- grafana integration
Best regards :)
Downloading :
Bitwarden :
Searxng :
Copilot/Gemini proposed this to do integration (i am not developer) :
# my setup use traefik with dynamic config and adguard to dns rewrite http/s to traefik for local https with no warnings
# searxng settings.yaml :
# zimi : /Volumes/mb-docker/appdata/br-searxng/engines/zimi.py
- name: my-zimi
shortcut: zm
engine: zimi
categories: general
timeout: 20.0 # Zimi can be slow on first boot
# searxng/engines/zimi.py :
from searx.utils import urlencode
# Base URL for the Zimi search API
search_url = 'https://zm.l.d0ge.ovh/search?'
def request(query, params):
if not query or not query.strip():
return params
params['url'] = search_url + urlencode({'q': query})
return params
def response(resp):
results = []
try:
data = resp.json()
except:
return []
for result in data.get('results', []):
zim_id = result.get('zim', '')
path = result.get('path', '')
# Build the Zimi reader URL
res_url = f"https://zm.l.d0ge.ovh/w/{zim_id}/{path}"
source_name = zim_id.replace('_', ' ').title()
results.append({
'url': res_url,
'title': f"[{source_name}] {result.get('title', 'No Title')}",
'content': result.get('snippet', ''),
'template': 'default.html',
'engine': 'zimi',
# Use 'score' here, but ensure it's a valid float.
# SearXNG uses this to initialize the metrics key.
'score': float(result.get('score', 1.0))
})
return results
# zimi compose :
services:
srv_zimi:
extends:
file: ../_common/all.yml
service: all
container_name: ${C_ZM}
image: ${C_ZM_IMG}
environment:
#- ZIM_DIR=/zims # Path to ZIM files
- ZIMI_MANAGE=1 # 1 Library manager. Set to 0 to disable.
- ZIMI_MANAGE_PASSWORD=${C_ZM_P} # (none) Protect library management
- ZIMI_AUTO_UPDATE=1 # 0 Auto-update ZIMs (1 to enable)
- ZIMI_UPDATE_FREQ=daily # weekly, daily, weekly, or monthly
- ZIMI_RATE_LIMIT=0 # 60 : API rate limit (requests/min/IP). 0 to disable.
volumes:
- ${REP_APPDATA}/${C_ZM}:/data
- vol-zim:/zims
ports:
- ${C_ZM_8899}:8899 # 22026
hostname: ${C_ZM_HOST} # zm
networks:
proxy:
ipv4_address: ${NET_TFK}.${C_ZM_NUM} # 154
mac_address: ${MAC_TFK}:${C_ZM_MAC} # 9A
# proxmox:
# ipv4_address: ${NET_PX}.${C_ZM_NUM}
# mac_address: ${MAC_PX}:${C_ZM_MAC}
#dns:
# - ${NET_PX_DNS}
networks:
proxy:
external: true
proxmox:
external: true
volumes:
vol-zim:
external: true
# searxng compose :
services:
srv_searxng:
extends:
file: ../_common/healthcheck.yml
service: searxng
container_name: ${C_SXN}
image: ${C_SXN_IMG}
volumes:
- ${REP_APPDATA}/${C_SXN}/config:/etc/searxng
#- ${REP_APPDATA}/${C_SXN}/themes:/usr/local/searxng/searx/static/themes
- ${REP_APPDATA}/${C_SXN}/tmp:/var/tmp
- ${REP_APPDATA}/${C_SXN}/ssl:/etc/ssl/certs
#
- ${REP_APPDATA}/${C_SXN}/engines/zimi.py:/usr/local/searxng/searx/engines/zimi.py:ro # zimi.lua
#
- ${REP_CACHE}/${C_SXN}:/var/cache/searxng
environment:
- SEARXNG_VALKEY_URL=redis://${C_VK_HOST}:6379/0
- SEARXNG_BASE_URL=https://${C_SXN_HOST}.${DOML}/
- SEARXNG_PORT=8080
- SEARXNG_API_URL=${C_SXN_URL}
- INSTANCE_NAME='dogeXng'
- SEARXNG_SECRET=${C_SXN_SECRET}
#- SEARXNG_SETTINGS_PATH=/etc/searxng/settings.yml
#
## ERR : ERROR:searx.botdetection: X-Forwarded-For nor X-Real-IP header is set! and no results returned
#- SEARXNG_OUTGOING__ENABLE_HTTP=true
## look at the headers provided by Traefik
#- SEARXNG_SERVER__INTERNAL_ADDRESS=127.0.0.1
#- SEARXNG_SERVER__BIND_ADDRESS=0.0.0.0:8080
## respect the proxy headers
#- SEARXNG_SERVER__PROXY_ADDRESS=192.168.51.0/24
cap_drop:
- ALL
cap_add:
- CHOWN
- SETGID
- SETUID
ports:
- ${C_SXN_8080}:8080
hostname: ${C_SXN_HOST}
networks:
proxy:
ipv4_address: ${NET_TFK}.${C_SXN_NUM} # 63
mac_address: ${MAC_TFK}:${C_SXN_MAC} # 3F
# proxmox:
# ipv4_address: ${NET_PX}.${C_SXN_NUM}
# mac_address: ${MAC_PX}:${C_SXN_MAC}
#dns:
# - ${NET_PX_DNS}
networks:
proxy:
external: true
proxmox:
external: true
Before all, thank you so much, you did so great job.
0/
I downloaded + 1000 files (+ 2 TO).
Would be great to have an option be able to become a mirror.
1/
Password for go to settings :
2/
Downloading : when downloading hundreds of zim files
3/
Files already included in other files :
4/
Integration with SearXNG search engine :
5/
with many files (+1000) :
6/ languages / images
7/
8/
9/
Best regards :)
Downloading :
Bitwarden :
Searxng :
Copilot/Gemini proposed this to do integration (i am not developer) :