7 Ways to Speed Up Inference of Your Hosted LLMs
7 Ways To Speed Up Inference of Your Hosted LLMsTLDR; techniques to speed up inference of LLMs to increase token generation speed and reduce memory consumption: mixed-precision, Bfloat16, quantization, fine-tuning with adapters, pruning, continuous batching, and multiple GPUsImage generated with MidjourneyCompanies, from small startups to large corporations, want to utilize the power
Read moreQuick Tip: Controlling Windows with Python
Windows is entirely controllable from code, using the Win32 API. Stuart looks at ways to control the Windows OS with Python. Continue reading Quick Tip: Controlling Windows with Python on SitePoint.
Read moreExperimenting LlamaIndex RouterQueryEngine with Document Management
How RouterQueryEngine works in a DevSecOps chatbotContinue reading on Better Programming »
Read moreTrust Is the Key to Engineering Management
Trust is not assumed. It takes time and skill to earn itPhoto by Joshua Aragon on UnsplashIntroIn the latin etymology, the word “confidere” is the addition of two words: “cum” and “fidere.” These can be translated as “with” and “rely on.” It means that you entrust something precious to somebody.How accurate is
Read moreUnderstanding URL Routing in Flask
Learn about URL routing in Flask: defining routes, handling URL patterns, supporting HTTP methods, managing redirects and errors, and more. Continue reading Understanding URL Routing in Flask on SitePoint.
Read moreRussian transliteration hack
I mentioned in the previous post that I had been poking around in HTML entities and noticed symbols for Fourier transforms and such. I also noticed HTML entities for Cyrillic letters. These entities have the form & + transliteration + cy;. For example, the Cyrillic letter П is based on
Read moreSymbols for transforms
I was looking through HTML entities and ran across ℱ. I searched for all entities ending in trf; and also found ℳ, ℒ, and ℨ. Apparently “trf” stands “transform” and these symbols are intended to be used to represent the Fourier transform, Mellin transform, Laplace transform, and z-transform. You would
Read moreBuilding a Perl Module for Posting to Bluesky Social
The new social media network is growing by leaps and bounds. Learning to post to it with Perl offers lessons useful to coders of all…Continue reading on Better Programming »
Read moreLlamaIndex: How to Use Index Correctly
and understanding what use case for what type of indexWow, I am absolutely thrilled by the overwhelming response to my first post on building a PDF chatbot! I have received so many messages requesting Part 2, and I cannot express how grateful I am for your interest and enthusiasm, but I
Read moreJamstack is evolving toward a composable web (Ep. 588)
Dana Lawson, Senior VP of Engineering at Netlify, joins Ben and Ryan to talk about her path from the military to tech, how three years at GitHub continues to shape her perspective, and how composable architecture is turning web development into something resembling LEGOsⓇ (in a good way). Episode notes:
Read more