writing
notes on ai, engineering, and building things.
publications
delta attention residuals for cross-layer information flow in indic models
accepted at the newinml workshop @ neurips 2026. keeps the standard residual stream but adds token-wise weighted aggregation of earlier layers' outputs, with a grouped "delta block" to cut routing and memory cost — lowest val loss and perplexity when fine-tuning qwen3-0.6b on hindi fineweb-2.
blogs
emotion concepts across model scale (submitted to arxiv, waiting for print)
a replication of anthropic's emotion-concepts study on 19 open base models (70m–13b; pythia, cerebras-gpt, qwen3.5) — emotion representations saturate early, around 1b parameters, at the same level across all three families.
which backbone, readout and calibration for system-1 decision models?
a common-ground benchmark of 50 model configurations on 22 zero-shot datasets — decision heads on llm backbones lead, joint choice readout beats per-option scoring, scale stops paying once the head is trained, and "zero-shot" nli leaderboards look contaminated.
world models and how they work and their lineage
tracing the path from pixel-space simulators (iris, storm, diamond) through latent-space compression, into dreamer's planning loop, and now watching jepa — my learnings on world models so far.
e3: issue-level backtesting for automated research critique
the research behind noteweave's review engine — an automated critic that flags unsupported claims, incomplete ablations, weak baselines, and data leakage risks, backtested against 100 iclr 2026 papers.
agentic search — how we built it at discoverminds.ai
the architecture behind arya: finding the right people in your network using agentic search
resources for classical ml, dl & genai
a curated list of courses, books, and resources for learning classical machine learning, deep learning, and generative ai — from nptel, iit, and beyond.