Pipeline tap operator `|:`

What?

The idea is to have a pipeline tap |: operator, allowing to perform side effect inside a pipeline.
It returns the same value given in its input, while calling the given function on it.

(Using F# style pipeline)

const result = someData
  |> parse
  |> filter
  |: log
  |> format

Which would do the same as:

const result = someData
  |> parse
  |> filter
  |> x => { log(x); return x }
  |> format

Why?

The pipeline tap |: operator can become very handy when logging or debugging inside a pipeline, or even anywhere else in the code.

For example, if we want, for debugging purposes, to quickly log the return value of the following function:

function randInt (min, max) {
  return Math.floor( min + Math.random() * (max - min) );
}

we would have to refactor the code in order to be able to do so:

function randInt (min, max) {
  const value = Math.floor( min + Math.random() * (max - min) );
  console.log(value);
  return value;
}

with the pipeline tap |: operator, no refactoring would be needed, only appending the operator:

// Using F# syntax
function randInt (min, max) {
  return Math.floor( min + Math.random() * (max - min) )
    |: console.log;
}

// Using Hack syntax
function randInt (min, max) {
  return Math.floor( min + Math.random() * (max - min) )
    |: console.log("randInt:", ?);
}

Of course, as shown in the intro, it would also be very handy when used inside of a pipeline, so that the given input keeps flowing without being changed:

const sendMessage = () => chatBox
    |> ?.value
    |> formatMessage(?)
    |: broadcastMessage(?)
    |: appendMessageToHTML(?)

How?

This proposal depends on the final syntax chosen for pipelines (F# or Hack).
But in any case, the pipeline tap |: operator can easily be designed to work in both syntax.

F# style:

expression |: handler

would desugar to

expression |> x => (handler(x), x)

Hack style:

expression |: handler(?)

would desugar to:

expression |> (handler(?), ?)

Operator precedence

This operator precedence is the same than the pipeline |> operator, and is also left-to-right associative.

Pros and Cons

I'd like to discuss several pros and cons about this proposal:

Pros

Debugging heaven

You can almost log any values for debugging without having to heavily modify parts of your code for that:

  • Declared variables: const sum = a + b |: console.log
  • Function return: (a, b) => a + b |: console.log
  • Single value in expression: (a, b) => a + (b |: console.log)
  • Pipelines
  • ...

Syntax coherence

This point might be a little opinionated, but I do find the pipeline tap |: operator quite elegant, and coherent with the rest of the language.

When looking at a pipeline flow, like:

const result = someData
  |> parse
  |> filter
  |: log
  |> format

the | character visually continues the pipe along the normal |> pipeline, but we can see at a glance that it is different. When the > of the normal pipeline seems to indicate a movement forward, to indicate a change, the : on the other hand, seems to be just presenting the data to the given handler, and nothing more. I feel that it looks quite descriptive about what it does, and there is no real confusion possible about which operator is doing what.

Cons

I'll need your help on this part, because I don't see a lot of cons coming with this proposal :)

One could say that the tap |: operator could potentially lead to writing less clean code, as it would allow to call side-effect function anywhere in the code for any expression. But I personally think that most of its use outside a pipeline flow would be to create temporary debugging functions, and most of the "production-ready" uses are going to be wrote inside a pipeline flow, so I'm not worrying too much about this.

But this is only my opinion, please do not hesitate to give your own thoughts about it! :smiley:

5 Likes

ux-wise, you can achieve the same with a helper-function debugInline (used in this real-world example);

inline-debugging becomes trivial as follows:

#!/bin/sh

node -e '
// this inline-debugger can be safely copy-pasted into any script
// init debugInline
if (!globalThis.debugInline) {
    let consoleError;
    consoleError = console.error;
    globalThis.debugInline = function (...argList) {
    /*
     * this function will both print <argList> to stderr and
     * return <argList>[0]
     */
        consoleError("\n\ndebugInline");
        consoleError(...argList);
        consoleError("\n");
        return argList[0];
    };
}

function randInt(min, max) {
    // return Math.floor(
    //     min +
    //     Math.random() * (max - min)
    // );
    return Math.floor(
        debugInline(  min, "min"  ) +
        debugInline(  Math.random() * (max - min), "random*scale ")
    );
}
randInt(0, 10);
'

# stderr:
#
#
# debugInline
# 0 min
#
#
#
#
# debugInline
# 0.9995644826073868 random*scale
#
#
1 Like

Sometimes I hear people discourage writing code like the following, because of how difficult it is to debug when doing log-based debugging:

const doTransfomration = data => data
  .filter(x => x != null)
  .map(x => x ** 2)
  .filter(x => x > 100)

It's a shame because I personally find code like that to be very elegant, but I do feel their pain every time I'm trying to debug a longer expression. I was even toying around with the idea of proposing a new "debugger.log <expression>" operator for this exact purpose, but I like this pipeline tap idea even better, as it's more general-purpose, and can fit in the middle of a pipeline better.

4 Likes

Indeed, but actually this would question the whole principle of pipelining.

We would have the same concern about the |> pipeline operator, because we could still do :

filter(parse(data))

instead of:

data |> parse |> filter

It's just that in the end, the pipeline version is more linear, with less nested function calls, and more readable for large flows.

But that's a good point that if the |> pipeline operator doesn't make it to the end, the |: should not either, for the same reasons.

Also, I think using an helper-function inside a pipeline flow would feel really, really wrong:

const result = someData
  |> parse(?)
  |> debugInline(filter(?))
  |> format(?)

// It would be even worse if we do really need a side effect for production
// And not just temporary debugging
const sendMessage = () => chatBox
    |> ?.value
    |> formatMessage(?)
    |: tap(broadcastMessage(?))
    |: tap(appendMessageToHTML(?))
1 Like

Unfortunately, the |: tap operator would not be usable in the example you gave.
We wouldn't be able to write something like:

const doTransfomration = data => data
  .filter(x => x != null)
  .map(x => x ** 2)
  |: console.log
  .filter(x => x > 100)

because this would be actually parsed as:

const doTransfomration = data => data
  .filter(x => x != null)
  .map(x => x ** 2)
  |: console.log.filter(x => x > 100)

This would indeed be a limitation of the pipeline tap |: operator: it would not be possible to use it between two member access . operators :(

It won't be extremely convenient to use |: between .functionCall()s, but it still would be much better than our current alternative. Compare these examples of someone trying to quickly modify my original example to log out an intermediet value:

For reference, the original version:

const doTransfomration = data => data
  .filter(x => x != null)
  .map(x => x ** 2)
  .filter(x => x > 100)

Debugging with |:

const doTransfomration = data => (data
  .filter(x => x != null)
  |: console.log)
  .map(x => x ** 2)
  .filter(x => x > 100)

Debugging without |:

const doTransfomration = data => {
  const temp = data
    .filter(x => x != null)
  console.log(temp)
  return temp
    .map(x => x ** 2)
    .filter(x => x > 100)
}

Also, if all you want to do is log out the final return value, the |: is really nice to use.

You're right, that would still be a better solution that what exists today.

But that made me want to find an even better design solution to get taps to work with member access . operators, unfortunately none of the ideas I came with were generic / coherent / clean enough to get a chance to get in the language some day.

I think the least worst solution to solve specifically your example would be to propose the implementation of an Array.prototype.tap method, so that we can at least use taps when working on arrays:

const doTransfomration = data => data
  .filter(x => x != null)
  .tap(x => console.log(x))
  .map(x => x ** 2)
  .filter(x => x > 100)

But this would feel weird to have an array method that has nothing specific about arrays.
And of course, this methods would not be available when working with other objects, like a custom API / framework making extensive use of chaining:

$("#p1")
  .css("color", "red")
  // I'd want to log here
  .slideUp(2000)
  .slideDown(2000); 

So I think we can forgot about this idea.

There is more generic solution I came with, but I don't really like it either.
I'm still presenting it here in case it inspires someone to come up with a better solution:

The idea would be to have a .:( ... ) operator. It would have the same precedence that the member access . operator or the optional chaining ?. operator:

const doTransfomration = data => data
  .filter(x => x != null)
  .map(x => x ** 2)
  .:(x => console.log(x)) // log the value of x here
  .filter(x => x > 100)

The problem when designing an "operator-like" solution is that the member access . operator has the second highest operator precedence, so we have to use parentheses (which is the highest) in order to delimit our R.H.S from the next member access of the chain.

Also, as we want to indicate that it is a member-access operator, it has to begin with a dot .

And finally, as .( ... ) would not really be self-explanatory about what it does, I reused the same idea that for the pipeline tap |: operator: using a colon : to express the idea of the tap.

But in the end, that gives me this 4 characters long operator monstrosity!

If someone can find a clean solution for making taps work with member access, that would really be great!

I guess another option would be to find some way to mess with order of operations. For example, an optional operator to close the tap.

const doTransfomration = data => data
  .filter(x => x != null)
  .map(x => x ** 2)
  |: console.log(?) :|
  .filter(x => x > 100)

Just thought of another way to do it, without even using the tap operator.

const doTransfomration = data => data
  .filter(x => x != null)
  .map(x => x ** 2)
  |> console.log(?) || ?
  .filter(x => x > 100)

console.log(...) always returned undefined. So, console.log(...) || somethingElse will always cause the somethingElse to be evaluated and returned. This will effectivly cause the .filter() to be combined with the ?, which is exactly what we're wanting. |> console.log(?) || ?.filter(...)

This concept can also be used in a normal pipeline too:

const result = someData
  |> parse(?)
  |> filter(?)
  |> console.log(?) || ?
  |> format(?)

Not too shabby, eh?

This certainly satisfies my debugging needs - it's not difficult at all to put a ||? after the console.log, and it works in many scenarios.

Edit: I used || in these examples, because I often use || with console.log() when debugging expressions already. But, I guess a more general-purpose solution could be constructed by using the comma operator instead. This would allow us to have the tap operator, without extra syntax (though, I recognize that the comma operator isn't the most intuitive thing to use in production code).

const result = someData
  |> parse(?)
  |> filter(?)
  |> doSomethingWithValue(?), ?
  |> format(?)

if ux-objective is making caveman-debugging as ergonomic as possible,
again, the quick-and-dirty inline-debugger
requires less cognition and keystrokes than a tap-operator.

 const doTransfomration = data => data
   .filter(x => x != null)
-  .map(x => x ** 2)
+  .map(x => debugInline(x ** 2))
   .filter(x => x > 100)

The issue I have with that is that I would then have to add this debugging function somewhere to the project. That's fine on personal projects where I'm the only one working on it, but for shared projects, I wouldn't be able to add a function like that without everyone's wanting that function in there, and everyone's not going to want it. I certainly wouldn't want other people adding their personal debugging functions to a shared project I'm working on either, because I know I'm unlikely to use it, and that code will just rot once they stop contributing to the project.

What's needed is a portable solution that can be used, no matter which project I'm debugging and what set of "debugging helper functions" people may have added to it.

This works in your example because you are specifically calling an Array method to debug each element of the array. But what if you wanted to log the whole array ? Or what if you want to log inside a chain that does not provide the Array.prototype methods, like a jquery object?

What we are looking for here is a generic way of "tapping" inside a chained expression. In this case, inline-debugger will have the same limitation than the proposed |: pipe tap operator.

And in the idea, it would also be great that this solution is production-ready, so that it can be used not only for quick and dirty temporary debugging, but also for cleaner chainning workflow patterns if needed.

But as this issue came as a second thought along the main proposed pipeline tap |: operator, I think it is totally fine if we don't find something in this discussion to solve this. This would be great for sure, but not mandatory.

Actually this is quite clever :)
However I'm not 100% sure about it, I think this expression is too confusing and error-prone, because it does not work for the same reasons it looks to work. While it can indeed be really handy for quick and dirty debugging on chained expressions, I hope this will not become a common pattern among js developers, because such a tricky expression would end up in production code one day!

Indeed, that's the main purpose of the my proposed pipeline tap operator: being syntactic sugar for such an expression!

@clemdz, oh i see you wanted the whole array. yea i frequently use debugInline as follows which looks ugly (but gets the job done and i remove it after i finish debugging)

const doTransfomration = data => debugInline(data
  .filter(x => x != null)
  .map(x => x ** 2)
  .map(x => x ** 2)
)
  .filter(x => x > 100)

@theScottyJam , you've inspired me to create a nodejs pull-request to make debugInline part of nodejs-core ; )

2 Likes

Because this is relevant to this thread, I'll also note a stage 1 debugger.log() proposal I just found out about (here), that is intended to act similar to console.log(), but will also act as an identity function (it returns whatever is passed into it). If that goes through, we would be able to do this:

const result = someData
  |> parse(?)
  |> filter(?)
  |> Debugger.log(?)
  |> format(?)

This doesn't help for the general-purpose "tap" use case, but it does help with debugging.

Is this significantly better than a generic tap function?

const tap = fn => x => (void fn(x), x)

Hi @lightmare, thanks for your interest in the subject!

I think the question is more: "would this feature be needed / used enough to be worth having dedicated operator over a function?"

Looking at another (successful) operator as an example: I think same question could have been raised for the Nullish Coallescing operator ??, where we could have simply implemented a generic function instead:

const coalesce = (...values) => values.find(x => x !== undefined && x !== null); 

Turns out the use cases and needs were important enough to implement a new syntax.

About this tap operator, I obviously cannot speak for everyone. In my point of view, it could come really handy, especially when used along side the pipeline operator |>, making the code more linear and more readable. When the pipeline operator finally reach stage 4 (hopefully one day), I can feel that the need for pipeline taps will be felt among developers, so I'm just getting a little ahead of the subject, and already thinking / proposing solutions for this! :)

So actually I'm interested in your opinion, do you think it would be worth adding a dedicated operator for pipeline taps, or that a generic tap function would be sufficient to work with? (taking into account the needs you could have later when pipelines are a real thing)

(a ?? b) could not be implemented by a function. coalesce(a, b) always evaluates both arguments. Moreover, emulating it with ternary was verbose, and with non-trivial lhs you needed a temporary to avoid evaluating it twice.

If we're trying to measure the utility of an operator (over not having it), then ?? sets the bar very high. It is both widely useful and difficult to emulate. I'm not saying this |: operator is bad or something, but it's not in the same ballpark. It is a bit more focused, and easy to emulate.

I like how |: visually distinguishes pass-through lines. Beyond that, it saves typing 5 characters (regardless of which pipeline syntax gets accepted, tap function would lean into it)

expr |> tap(log, #)
// vs
expr |: log(#)

Honestly I don't think that's enough.

Indeed, the ?? operator may not have been the best example here! :smiley:
But at least I think you get my point, and in the end we both agree that we should have good reasons to have a new operator to the language.

Also about your last point, it depends of the chosen pipeline implementation: for the F# style, not only it saves 13 characters in this case, but it also removes the need for wrapper function:

expr |> x => tap(log, x)
// vs
expr |: log

So actually the need for a pipeline tap operator could be more or less important according to the chosen pipeline implementation.

1 Like

No, the tap function I showed above works for F# pipeline:

const tapF = fn => x => (void fn(x), x);

expr |> tapF(log) |> send;

For Hack-style it would be different:

const tapH = (fn, x) => (void fn(x), x);

expr |> tapH(log, #) |> send(#);

And for Elixir-style yet another:

const tapE = (x, fn) => (void fn(x), x);

expr |> tapE(log) |> send();
1 Like