iamzulx/crypto-rag
crypto-rag is an Indonesian-language CLI crypto assistant that pairs a static knowledge-base RAG pipeline with live market data tool calls, built on the principle that volatile numbers (price, funding rate, TVL, sentiment) must never be embedded in the vector store since they go stale instantly. It implements hybrid retrieval (BM25 keyword search + FAISS dense embeddings fused via Reciprocal Rank Fusion) over a 146-topic knowledge corpus, an optional cross-encoder reranker, and an intent router that dispatches price/derivatives/on-chain/sentiment queries to live sources (Binance, DefiLlama, Fear & Greed Index) versus concept questions to the corpus. A tool-calling agent handles multi-step queries, and an OpenAI-compatible LLM synthesis layer (works with StepFun, OpenAI, Groq, LM Studio, Ollama) grounds answers with citations and falls back to extractive answers when no LLM is configured.
What it claims
All market data is fetched without API keys from public sources: spot prices from six exchanges (Binance, OKX, Bybit, KuCoin, Kraken, Coinbase), Binance Futures, DefiLlama, Mempool.space and the Fear & Greed Index. Only the optional LLM layer needs a key. The README says live numbers "never pass through the LLM", so prices cannot be hallucinated.
Beyond what the summary covers, the tool also offers:
- Cross-exchange spread and arbitrage comparison
- Order-book slippage estimates for a given notional amount
- OHLC trend analysis with an SMA
- BTC mempool, fee and halving data, Ethereum EIP-1559 gas, and Solana inflation
- A watch mode with conditional price alerts
- A portfolio tracker with profit and loss
- Chat follow-up memory, so "berapa harganya sekarang" resolves to the previously named coin
- A coin-category filter over 615 tagged coins
How to use it
Install and build the indexes once:
python3 -m venv venv
venv/bin/pip install requests numpy faiss-cpu fastembed websocket-client
venv/bin/python fetch_data.py
venv/bin/python index.py
venv/bin/python knowledge_index.pyOptionally, enrich coin categories from CoinGecko and then rebuild:
venv/bin/python enrich_categories.py --pages 1
venv/bin/python index.pyTo enable the LLM layer, copy config.example.json to config.json and set OPENAI_BASE_URL, OPENAI_API_KEY and OPENAI_MODEL. The example uses StepFun step-3.5-flash. Environment variables of the same names override config.json. With neither set, the tool returns extractive answers.
Examples:
venv/bin/python rag.py # interactive chat (default)
venv/bin/python rag.py "jelaskan impermanent loss"
venv/bin/python rag.py --watch "ETH<1800" SOL
venv/bin/python rag.py "slippage beli BTC 5 juta"
venv/bin/python rag.py "portfolio tambah 0.5 BTC harga 60000"Useful flags:
--ruleforces the extractive path with no LLM.--rerankturns on the cross-encoder.--agentforces the tool-calling agent.--k Nsets the number of retrieved documents (default 5).--streamprints the answer token by token.--liveadds sub-second WebSocket prices.
Retrieval evaluation runs against an 87-query golden set:
venv/bin/python eval_rag.pyThe README does not publish any scores from this evaluation.
Performance as stated
- Startup: a one-shot run takes about 15–17 s, mostly spent loading the embedding model. In chat mode the model loads once, and later queries take 0.1–1 s.
- LLM synthesis: adds 8–9 s of API latency.
--streamreduces the wait before output appears, and--rulereturns instantly. - Cache TTLs: price 3 s, funding and overview 30–60 s, TVL 5 min. Repeat queries for the same coin are near-instant.
- Non-semantic queries (price, funding, TVL, category, market cap) skip the LLM and are fast once the model is warm.
Limits
- Both the corpus and the example queries are in Indonesian. Intent routing appears to be keyed to Indonesian phrasing, so check before using it with English queries.
- Live data depends on exchange availability. Coins without a trade stream, such as delisted ones, fall back to REST.
- Coin metadata and categories are static snapshots taken at index-build time. Keeping them current means re-running
fetch_data.pyandindex.py. - The README gives no retrieval-quality numbers, no licence and no test coverage.
- The README describes all output as educational only and not financial advice.
Organisations named here
- Binance
- OKX
- Bybit
- KuCoin
- Kraken
- Coinbase
- DefiLlama
- Mempool.space
- StepFun
- OpenAI
- Groq
- LM Studio
- Ollama
- CoinGecko
- Ethereum
- Bitcoin
- Fear and Greed Index
- Solana
Technologies named here
- Step 3.5 Flash
- FAISS
- fastembed
- BM25
- websocket-client
- Binance API / WebSocket
- DefiLlama
- Mempool.space
- Fear & Greed Index
- CoinGecko
- OpenAI-compatible API endpoints
- eval_rag.py
- rag.py
- OpenAI API
- Ollama
- Python
Techniques named here
- Retrieval-augmented generation
- Hybrid retrieval with Reciprocal Rank Fusion
- Cross-encoder reranking
- Intent routing
- Tool-calling agent
- Grounded LLM synthesis with citations
- Token-by-token streaming output
- Short-TTL caching
Related in llm-apps