Back

Announcing fromfrom

• 22.03.19 •

Tomi Turtiainen • Software Developer

I’m announcing a new library called fromfrom. Watch this video (NOTE: the video has been recorded before the library had a name) or read on to learn more!

What is it

fromfrom is a library to transform sequences of data from one format to another. Here’s a simple example of how it works.

Embedded content: https://gist.github.com/tomi/4cc5d42b55746eb33df58be1386e977a

As you can probably guess, the name of the library comes from its API, which is basically a single function from. from takes a single parameter, the data you want to transform and wraps it to a Sequence. Now you can apply and chain all types of different transformations to the data until you want to convert it back to for example an array. To see the full list of available transformation, see the API documentation.

Motivation

Have you ever tried transforming an array of data from one format to another in JavaScript? JavaScript array has all these nice functions like filter, map and reduce, which do a pretty nice job. However, what do you do when you need to something more complicated, like taking only specific items, sorting them by some attribute and taking the first 10? Or grouping the items by some attribute and taking only specific properties per item? Or want to use some data structure than array?

I come from a .NET background and when I started working with JS some years ago, I was surprised and frustrated how lacking the JS “standard” library was. Even though the situation has improved with each new ECMAScript version, you are still pretty much forced to do any more complicated operations using the reduce function, which is not as readable as using a proper function for the task.

Things get even worse when you want to use different data structures than array (and you should use a correct data structure for the situation). Map and Set don’t have any transformation methods and you’re left with the only option to convert them to arrays and back. With objects the situation is a little better. ES5.1 gave us Object.keys, ES2017 Object.entries and the Object.fromEntries TC39 proposal is as of writing this in stage 4. Still the support could be better.

Some might say you can always use a library like lodash. For me, the way you do chaining in lodash has always felt a bit clumsy and implicit. When you chain operations in lodash with the chain function you invoke the execution with .value(). It’s hard to see what the actual data type and result of the chain is going to be without reading the entire chain. Also it doesn’t support native Sets or Maps. What Having been accustomed to the powerful LINQ expressions in .NET, I wanted to have a similar experience in JS. That is, to have a comprehensive set of transform functions as well as great tooling support. Now there are already many existing LINQ implementations in JS, but they don’t feel like JS. They use the exact same method names as in .NET and they might not support Sets, Maps or objects. To tackle these shortcomings and make my life a little easier I decided to write a new library and I set these design goals for myself:

  • Minimalistic: Do one thing and do it well
  • Simple: Have a clean and simple API
  • Familiar: Use familiar function names from JS
  • Small: No external dependencies
  • Type safe: Written in TypeScript and has type definitions included
  • Comprehensive: Support all native JS collections
  • Fast: Use lazy evaluation and pipelining

How does it work

Transforming data with fromfrom can be thought as a pipeline. Each element of the source sequence flows through the pipeline and the given transformation are applied one by one. Under the hood fromfrom uses iteration protocols, deferred execution and lazy evaluation.

Iteration protocols

Iteration protocols were added in ECMAScript 2015 to provide a coherent way to produce a sequence of values and iterate them. For example when you loop through all the elements of an array with a for..of construct you are using the iteration protocols.

There are two iteration protocols: the iterable protocol and the iterator protocol. The TypeScript definitions of these two protocols are as follows:

Embedded content: https://gist.github.com/tomi/66d127f3a3797ead2dbbd7d4fe3ce696

As can be seen, for an object to implement the iterable protocol, it must have a single function named Symbol.iterator that returns an iterator. Iterator provides functions to loop through all the data. Here’s a bit modified example from MDN that shows how this works in practice with an array, that implements the iterable protocol:

Embedded content: https://gist.github.com/tomi/b2cb2d70aa26e60f1fe9ec75459c4490

fromfrom uses the iteration protocols to iterate through the input data. All transformations create a new iterable that when iterated reads from the previous iterable, applies the transformation and then provides the output data. fromfrom works with any input data that implements the iterable protocol. From the built-in types for example Array, Set and Map implement the iteration protocols, but for example Objects do not. fromfrom provides its own iterable implementation for objects, so they can also be used as input data.

Deferred execution and lazy evaluation

Pipelines created by fromfrom use deferred execution and lazy evaluation. Deferred execution means that the pipeline is not executed until there’s a call that forces the evaluation. Calls that produce a built-in JS type force the evaluation, such as .toArray(), .some() or .first(). Here’s an example that demonstrates that:

Embedded content: https://gist.github.com/tomi/202c35caf7cfd2078f99fc6b6b4fb721

Lazy evaluation means that each element from the source sequence is processed through the entire pipeline as needed. The following example demonstrates that:

Embedded content: https://gist.github.com/tomi/4dd991ff05055b082d3adbbfe6c183cc

Exception to this rule are certain transformations that require the entire sequence to be eagerly evaluated. For example .sortBy() needs sort the entire collection first before it can return the first element.

Conclusion

fromfrom is available now in NPM. Try it out and let me know for example in Twitter what you think!

Sign up for Futurice news

Futu Connect is our semi-regular roundup of all things Futurice. Be in the know about career-changing events, industry-leading content, non-profit passion projects, and an ever-changing job board.

Enter your email address below.