My Posts – Ishaan’s Blog

Answer Key: Linear Algebra Course Quiz

Award 2 points for a correct and explicit answer, 1 for the correct method with a minor error, and 0 for a missing or materially incorrect answer.

Linear Algebra Course: An Eight-Unit Study Guide and Quiz

This guide follows the public lessons retrieved from the Linear Algebra playlist. Members-only and unavailable lessons are excluded.

Ishaan Shrivastava

Answer Key: Essential Linear Algebra Quiz

Questions 1–10 are computational and worth 2 points each. Questions 11–12 are conceptual and worth 2 points each. Award 1 point for the correct method with an arithmetic or…

Essential Linear Algebra: A Five-Unit Quiz

Study one unit, solve its representative problem, and only then attempt the final quiz. The sequence follows the conceptual dependencies in the 3Blue1Brown Essence of Linear…

Ishaan Shrivastava

ANSWER KEY TO PRIVILEGED BASIS AND RELU QUIZ

Here is the answer sheet - index.qmd

Prerequisite Study Guide: ReLU Privileged-Basis Reconstruction

The target object is

Ishaan Shrivastava

Multivariable Statistics

While trying to understand what a privileged-basis means, I realised that I am significantly underequipped with the prerequisites. These are my notes for all the problems I…

Ishaan Shrivastava

WTF is going on inside Gemma? Thoughts, and some experiments with hybrid post-pre-norm and Per-Layer-Embeddings (PLE)

Does anyone have any idea of what Gemma-style Per-Layer-Embeddings actually do? Lets pull up Gemma 4 E4B for a closer look.

Ishaan Shrivastava

Part-Whole training

Reference - https://www.sciencedirect.com/science/article/abs/pii/0001691889900073?via%3Dihub

Ishaan Shrivastava

Reproducing Attention is All you Need, from scratch - PyTorch

Transformers are a sequence modelling and transduction architecture heavily leveraging self-attention in order to eliminate the need for convolutions or gating. Advantages…

Ishaan Shrivastava

Rotating Emphasis Manipulation

Reference - https://www.sciencedirect.com/science/article/abs/pii/0001691889900073?via%3Dihub

Ishaan Shrivastava

speculative-decoding

def remove_last_t_from_kvcache(kvcache, t): for i, (k_layer, v_layer) in enumerate(kvcache): if k_layer is not None: kvcache[i] = ( k_layer[:, :, :-t, :], # remove last…

Ishaan Shrivastava

Measure Theory

A sigma algebra \(\mathcal{A}\) on a set \(X\) is a nonempty set of subsets of \(X\) that is closed under complement, countable union, and countable intersection. In other…

Ishaan Shrivastava