This Week's Learning Nuggets (W25-2025)
As always, these notes capture my current understanding. Fully expect future‑me to refactor or flat‑out contradict some of this later.1. LLM Quantization Moves the Pareto FrontierReading: ParetoQ...
View ArticleThis Week’s Learning Nuggets (W24-2025)
A quick‑fire recap of what I learned from this week’s reading binge.1. Uber (2017) – “Michelangelo: ML at Uber Scale”Key takeawaysUber had a full feature‑store → training → online‑serving loop as early...
View ArticleThis Week’s Learning Nuggets (W23-2025)
Trying something new - quick, imperfect notes from what I read this week. If I got something wrong let me know!Nugget #1: Hugging Face Spark ConnectorWhat I used to think:Loading datasets into Spark...
View ArticleA Baby, a Bug, and a Pull Request to the Human Genome
A few weeks ago, I read a story that I haven’t been able to shake off. A baby named KJ Muldoon was born with a rare genetic disorder — one that, if untreated, would’ve ended his life before it truly...
View ArticleBeyond `kubectl apply`Giving Your Kubernetes Apps Superpowers with the...
Kubernetes 101 — Desired State MagicAlright, so you're deploying apps on Kubernetes. You probably know the drill: write some YAML defining Deployments, Services, maybe StatefulSets, run kubectl apply,...
View ArticleDebugging Pip Extras: A Deep Dive into .whl Files
BackgroundPython packaging has a nifty feature to include packages as an optional dependency for extended functionality. As an example, Amazon SageMaker released a new capability to use partner AI apps...
View ArticleWriting Into Dynamic Partitions Using Spark
Hive has this wonderful feature of partitioning — a way of dividing a table into related parts based on the values of certain columns. Using partitions, it’s easy to query a portion of data. Hive...
View ArticleParse Json in Hive Using Hive JSON Serde
In an earlier post, I wrote a custom UDF to read JSON into my table. Since then, I have also learned about and used the Hive-JSON-Serde. I will use the same example as before.{ "customer": {...
View ArticleWriting UDF To Parse JSON In Hive
Every so often, we need to perform data transformation in ways too complicated for SQL (even with the Custom UDF’s provided by hive). Let’s take JSON manipulation as an example.JSON is widely used to...
View ArticleAlways Specify Region When Calling DynamoDb from Hive
DynamoDb is a key-value storage store. One can query DynamoDb tables from Hive using the DynamoDBStorageHandler. It’s super easy to setup. Let’s say we have built a platform that collects data for...
View ArticleCamelCase Partition Column is a Bad Idea in Hive
Outside Java code I prefer snake_case over camelCase. This is mostly a preference without any strong good reason: Without a proper IDE I find it easier to read snake_case words than camelCase...
View ArticleReusing Hive Scripts
Amazon’s Elastic Data Pipeline does a fine job of scheduling data processing activities. It spawns a cluster and executes Hive script when the data becomes available. And after all the jobs have...
View ArticleScrapy | Crawl WhoScored For Football Stats
Earlier, I have written code to crawl Google Play, iTunes AppStore and Goal.com websites. But every time I re-wrote the code to get content from website, parse it using BeautifulSoup while maintaining...
View Article1-2 Oblivious Transfer
I learnt about oblivious transfer when reading up about garbled circuits. As an engineer this feels like a fascinating, almost magical protocol. And it’s not just me. Everyone I talked to about this...
View ArticleClik here to view.
Bowen Lookout and Yew Lake Loop
View from Bowen Lookout We had a fun Saturday, exploring live music scenes in Vancouver. The top recommendation in the group was Guilt & Co, which turned out to be an excellent pick! We stayed for...
View ArticleMaking Sense | Is Life Actually Worth Living?
This was an interesting discussion. It was a learning opportunity for me, as I have no prior opinion on anti-natalism. David has a unique perspective where he assigns different values to creation of...
View ArticleRise of Skywalker | Balance Restored?
I watched the final part of the Skywalker saga, and overall liked it. It had plenty of elements to invoke nostalgia and make me feel emotionally comfortable. However, I think the ending could have...
View ArticleWriting Into Dynamic Partitions Using Spark
Hive has this wonderful feature of partitioning — a way of dividing a table into related parts based on the values of certain columns. Using partitions, it’s easy to query a portion of data. Hive...
View ArticleParse Json in Hive Using Hive JSON Serde
In an earlier post I wrote a custom UDF to read JSON into my table. Since then, I have also learnt about and used the Hive-JSON-Serde. I will use the same example as before. { "customer": {...
View ArticleSPOJ | NICEDAY — The Day of the Competitors
Problem Contestants are evaluated in 3 competitions. We say that: A contestant A is better than B if A is ranked above B in all of the three competitions, they were evaluated in.A is an excellent...
View Article
