Weak-referencing is powerful new feature in JavaScript and specifically has tremendous value in building caches that can actually leverage and work in harmony JavaScript’s garbage collection (GC) mechanism to maximize efficiency in caching objects in a way that GC can collect cached objects, but cached objects still can be accessed from the up until the point when they collected. Here we introduce the weak-lru-cache package as a high-performance cache that uses weak-referencing values in combination with a LRU/LRFU expiration strategy.

Usage

We have previously written about the power of using weak-referenced maps for the purpose of caching. That previous post gives…


Virtually any interaction of data between computers, or storage of structured data involves serialization and deserialization, and therefore the performance of these methods is crucial for optimum application performance. In our clinical research applications, we deal with storing, searching, and updating massive amounts of structured data, and therefore we have invested in building the fastest serialization and deserialization techniques that are available with msgpackr and cbor-x, providing significantly better performance than typical JSON serialization. In this post, I wanted to share a few techniques that we used to achieve this performance.

The msgpackr package uses the MessagePack format, and cbor-x…


Key-value stores are a powerful tool for fast storage of data. We use LMDB, which provides phenomenal performance, for caching data. However, storage systems like LMDB have a low-level interface, using binary data for keys (and values). In JavaScript it is much more convenient to use JS primitives like strings and numbers as identifiers or keys (null and booleans are also primitives). What is the best way to convert these primitive values to binary form, as a Buffer in Node.js, for use in getting and storing values in key-value store? We have created the ordered-binary package to provide a single…


We switched to Lightning DB (LMDB) from LevelDB over a year ago for our caching server, and we have been thoroughly impressed with the performance, scalability, efficiency, integrity, and concurrency of LMDB. Simply put, LMDB is probably the most efficient key-value store available, and it also supports multi-process concurrency and has a robust, crash-proof design. We initially were interested in LMDB because it provided multiple-process support, which is the most robust mechanism for parallel execution in Node.js …


DPack is structured data format that can serialize and encode large data structures extremely efficiently, often with 2 to 3 times less space, and can parse faster than JSON or other common alternatives. In addition, dpack supports a broad and extensible range of data types, object and structure referencing, and supports blocks that allows for partial/deferred parsing. DPack is autonomous, no out of band information is necessary. …


Our engineering team at Doctor Evidence places a high value on a fast, efficient, and responsive development environment. Development speed is highly dependent on immediate feedback as you code. Ensuring that the result of any code change can be seen in seconds dramatically improves the ability to stay focused and quickly work through coding challenges. Every millisecond counts towards productivity in the edit/compile/test feedback loop.

Fast automatic reload after code changes

On the front-end, we use Webpack’s hot module reloading for instant feedback to code changes. We also have a NodeJS server that caches most data used on the front-end, allowing for quick application load times.


We use a NodeJS-based server in our stack that is responsible for transforming data and caching content in LevelDB for fast access in the front-end. This, like any other HTTP server, handles HTTP requests from users. And when we need to update our code and restart this server, we want to do this as unobtrusively as possible. Therefore, we employed a “graceful shutdown” process that restarts Node, avoiding any lost requests that we can.

Fortunately, Node is actually very well designed for graceful shutdowns, with several important features that facilitate a smooth exit. First, Node
naturally exits once there are no…


JavaScript’s meteoric rise, alongside the exponential growth of the NPM ecosystem, has given JavaScript developers unprecedented opportunities to use a rich variety of packages. But this is a luxury that we can also easily over-indulge in. A couple years ago, the left-pad removal left thousands of project builds broken, despite it being just a few lines of code. Recently a great post about the how easily security can be exploited through NPM packages has been widely read. We have also experienced pain when a dependency suddenly decided to add a polyfill that conflicted with our polyfills. …


Weak value maps are a powerful data structure that can enable efficient object management, tracking, and caching. While WeakMaps are well known, with their addition into core EcmaScript, and have a wide range of uses, the lesser known, weak value map structure serves a smaller, but valuable role, and is available as a package on Node for tracking identifiable objects, creating cache structures, and safe introspection of “in-use” objects.

So what is the difference EcmaScript’s WeakMap and a weak value map? With a WeakMap each of the keys is a weak reference (and the values are strong references from the…


One of the exciting new features in TypeScript is the ability to write your own plugins, or “custom transformers”. This was added in version 2.3 with little fanfare, but this extensibility is a powerful capability that expands the possibilities of the TypeScript engine. JavaScript developers have typically had to choose between Babel’s extensive plugin-based compiler with full modern EcmaScript feature transpilation, or TypeScript’s excellent type checking functionality. …

Kris Zyp

On JavaScript development, applied to medical research

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store