Posts
-
How GPT-4 Sees Your Text: Building a BPE Tokenizer From Scratch in Mojo
A step-by-step build of a byte pair encoding tokenizer in Mojo, tracing every merge, every byte, and every design decision from raw text to token IDs. -
Attention Is (Almost) All You Need: What Reversing and Sorting Digits Taught Me About Transformers
Building minimal transformers for sequence reversal and sorting — and discovering that what fixes one architecture's instability doesn't transfer to the other. A journey through attention, gradient noise, and the realization that 'Attention Is All You Need' was never a recipe. -
From Bytes to Gradients: Tracing a Neural Network Through Tenmo, One Layer at a Time [Tenmo]
A line-by-line trace of one MNIST training step through Tenmo — from raw memory allocation, through SIMD-vectorized matmul and compile-time autograd, to the SGD parameter update — all in pure Mojo, no Python, no CUDA. -
Migrate kubernetes embedded etcd to external etcd - easy back and forth switch
Create a multi-master kubernetes cluster from the comfort of a shell menu without tweaking a thing. Front the apiservers with load balancer of your choice - haproxy/nginx/envoy. Do hassle free back and forth switch between embedded etcd and external etcd. -
grpc connect — rust, java and grpc-web
Gist: Route calls from browser(using grpc-web) to rust grpc application(implemented using tonic), which in turn delegates to java grpc and vice versa.
-
Nesting all the way - kubernetes on openstack on google compute engine
The gist: Can gcp/azure/AWS can be squeezed into one machine?
-
Exploring envoyproxy
With the adoption of micro-services, we need to tackle hosts of issues associated with remote calls and networking - because what used to be an in-process function call now becomes a RPC call that need to be handled by a service which needs to be discovered. Service discovery has its own issues - among others, the most important being able to discover services that are active. Once services are discovered - we need to handle uniform spread of requests among discovered service instances. Traffic encryption becomes another issue that we need to handle once a call goes out over the wire from one micro-service to another.
-
The essence of a monad
Many people when they encounter the term ‘monad’ for the the first time, they decide to look it up, at least the curious bunch. But when they do consult the blogs, articles, Wikipedia, haskell documentations- explaining the term monad - they become deeply confused and angry at themselves for not being able to make any sense of it - at least people like me, whose brain is not bigger than the size of a peanut. They curse themselves and grok more blogs, articles, try to learn haskell in the hope that, may be this time they would be able to grasp it - but to no avail. Their brain hurts and they become even more confused. As they keep persisting in their efforts -they come across monads being described as elephants, space suits/burritos/fluffy clouds and what not! Some say - you need to know category theory to understand monad, some say no you do not - while people trying understand monad remain in a state of utter confusion for months or maybe years on end. This happened to me also - couple of years back(As I said before my brain is not bigger than a peanut!).
-
Recovering from scala delimited continuation encounter
As I have said before, delimited continuations are wild beasts. Its one of those things that bends and hurts the mind. You might have to spend months in a state of delirium trying to figure them out! In the scala-lang.org’s delimited continuation page, one reader commented and I quote, “From my perspective, these are convoluted ways of adding numbers and I have no idea what is being gained or accomplished”. Reading his comment - I had laughed my heart out -really! I could understand his frustration - even though my plight was only marginally better! So, if this is the first time you are hearing about them - maybe you should do some googling for them before going through rest of the stuff in this post. Once you are sure that you are confused enough - you should take some rest and then start with this post. Hopefully, once you are through with this post - some of the mists would disappear and you will be better equipped to explore them further. So, without further ado, let’s get to them.
-
Scala for the uninitiated java developer: 22 scala things you may not have known
Scala’s popularity is rising among developers. Yet, many java developers have hardly paid any attention to it - thinking it is some sort of esoteric programming langauge isolated from the java main land. They are the target audience of this post. This post is not a tutorial - it presents the scala features with minimum technical detail and tries to show how java and scala reside under the same roof(JVM).
-
Iteratee fundamentals for the beginer
Iteratees are an alternative to commonly prevalent, imperative blocking IO. While reading from a stream, if is data is not available - the reading thread blocks - which is not desirable. The alernative would be to register a callback which gets invoked as data become available. Iteratees are callback handlers and much more. If you have heard about them but not sure what they are - then this post is for you.
-
Asynch concurrency: In the promised land of scala futures
The concept of future is not new - java added them in 1.5 -scala actors had futures from the start, lift webframework had its own futures. As more and more libraries and toolkits sprang up based on the scala language - many had futures with more or less similar functionalities - akka, playframework, twitter’s finagle etc had their respective implementations.
-
Delimited continuations
Understanding Scala's delimited continuations by building a framework to capture and remotely execute frozen computations.