Data Formats for Row-Oriented Processing
Photo by Christopher Gower on UnsplashThe combination of pandas and parquet is the backbone of many data pipelines. However, it can, at times, be a badly performing solution. I would like to show one such case here.IntroductionPandas and parquet are both primarily column-oriented/analytical tools. They shine the most when your pipeline is,
Read moreDecoding a Heatmap: Using Espionage in the Battle for Riddler Nation
Unraveling enemy battle plans, one pixel at a timeImage by Bing Image Creator, with out-painting by Dall-E, upscaling by waifu2x, and color quantization by pngquant.This weekend marks the eighth and final Battle for Riddler Nation hosted by FiveThirtyEight. The first was held back in 2017. I’m fond of it because I
Read moreChris’ Corner: If these computers are so smart, how come I have tell them what to do?
It’s clear that this current generation of “AI” / LLM tools likes offering a “chat box” as the primary interaction model. Both Bard and OpenAI’s interface center a text input at the bottom of the screen (like most messaging clients) and you converse with it a bit like you would
Read moreThe hardest part of building software is not coding, it’s requirements
With all the articles about just how amazing all the developments in AI have been, there’s plenty of hand wringing around the possibility that we, as software developers, could soon be out of a job, replaced by artificial intelligence. They imagine all the business execs and product researchers will bypass
Read moreDrag equation exponent variation
The motion of a falling body of mass m is given by where the term −kvr accounts for drag due to air resistance. One can derive r = 2 under simple physical assumptions, but if I remember correctly other values of r may be more realistic in certain circumstances. I
Read moreHandling Errors in Go
Errors in Go can be cumbersome and verboseContinue reading on Better Programming »
Read moreThe Missing Runbook for Engineering Managers
Photo by Jonathan Chng on UnsplashDespite the technical nature of a Software Engineering Manager’s (EM) work, it is still a leadership position focused on empowering others. During my years at Facebook Reality Labs and WhatsApp, I explored multiple strategies to manage and support teams and ensure we achieved desired results and
Read moreNumbers don’t typically have many prime factors
Suppose you select a 100-digit number at random. How many distinct prime factors do you think it would have? The answer is smaller than you might think: most likely between 5 and 6. The function ω(n) returns the number of distinct prime factors of n [1]. A theorem of Hardy
Read moreKeep your UI test working when migrating from native Auth0 to WebAuth0 login
How we achieved that without any external libraryPhoto by Sigmund on UnsplashIntroductionsAuth0 is a popular identity and access management platform that provides developers an easy-to-use solution for securing their applications and APIs. With Auth0, developers can quickly add authentication and authorization to their applications without having to worry about the underlying infrastructure.
Read moreMixed Reality App Design Case Study
Use your curiosity to explore new design paths — step by step.Continue reading on Better Programming »
Read more