Explicit exceptions - a solution to fragile code dealing with exceptions

I am a bit late to this thread but I believe the author is correct error handling in javascript is challenging. I just don't want the cure to be worse than the disease.

Could all of this be handled by having a c/c++ assert style function within a catch block?
Perhaps a catchAssert(e, conditions) ... ?

(I hate it when people chime in after the fact so if I sound like an ignormous it is because I am and will not be offended if you have already thought of this ?)

Nah, don't feel bad - I'm hoping to get a wide variety of opinions on this matter - the more heads we get into this problem, the more likely we can find the solution.

Could you elaborate a bit on this catchAssert idea and how it would work in Javascript? (Maybe an example?)

Great work, indeed, that will simplify reasoning!

(Also, sorry for the delay of the response, I've been quite busy recently!)

Actually (maybe I didn't say it explicitly), but in my proposal, there was also an idea of automatic propagation / escalation of the exception to the outer scope.

If the exception is declared in a function definition, when this function is called inside another caller function, the exception could be either:

  1. implicitly propagated (default behavior if there is no try / catch)
  2. explicitly caught and re-thrown (manual propagation)
  3. explicitly caught and re-thrown as an error (manual escalation)
  4. explicitly caught and a default value is provided instead

In cases (1) and (2), the caller function definition would have to explicitly declare the exception with the throws keyword, so that the propagation continues in the outer scope.

If a function tries to throw an exception that was not explicitly defined in its definition, it would be automatically escalated to an error, and the caller function will receive this error (instead of the exception that caused it).
This would be very simple to understand and to explain: a function cannot throw an exception it has not declared.

So for example, instead of (from your github example #2):

function getUsername(userId, { userIdDoingRequest, fallback = null }) throws UnauthorizedEx {
  try {
    return userManager.getProperty(userId, 'username', { userIdDoingRequest })
  } catch NotFoundEx {
    return fallback
  } catch UnauthorizedEx (ex) {
    throw ex
  } catch MissingPropEx {
    throw new Error('Unexpected failure to find the username property.')
  } catch Exception (ex) {
    throw new Error(`Unexpected exception: ${ex.message}`)
  }
}

we would be able to write directly:

function getUsername(userId, { userIdDoingRequest, fallback = null }) throws UnauthorizedEx {
  try {
    return userManager.getProperty(userId, 'username', { userIdDoingRequest })
  }

  // Provide a default value if not found
  // case (4)
  catch NotFoundEx {
    return fallback
  }

  // No need to catch UnauthorizedEx: it will be automatically propagated
  // (because declared in both `userManager.getProperty` and `getUsername` definitions)
  // case (1)

  // By definition, there is no need to catch MissingPropEx:
  // it will be automatically escalated because MissingPropEx is not declared in `getUsername` function definition
  // But as we want to throw a custom error, we still specify it
  // case (3)
  catch MissingPropEx {
    throw new Error('Unexpected failure to find the username property.')
  }

  // No need to catch any other exceptions: they would have been already escalated
  // And as it it more a "programmer" error than a "program" error
  // This would make little sense to try to catch those errors

Or without any comments:

function getUsername(userId, { userIdDoingRequest, fallback = null }) throws UnauthorizedEx {
  try {
    return userManager.getProperty(userId, 'username', { userIdDoingRequest })
  } catch NotFoundEx {
    return fallback
  } catch MissingPropEx {
    throw new Error('Unexpected failure to find the username property.')
  }

In the end, the code you write is not that different from your except keyword proposal, but from a "thrower" perspective instead of a "catcher" one. Instead of having the information about where an exception can be expected to be thrown, you have the information about what exceptions a function is expected to throw.

Both information are useful on their own:

  • The expect keyword, as you explained, is very useful to know what to expect from the function you are calling, at the place you are calling it. This helps a lot not to jump from file to file, in order to know what exceptions you expect the function to throw. Also, this would prevent developers from refactoring code badly, because they were not aware that the order of the functions was important.

  • The throws keyword in function definition, on the other hand, is very useful to get a description of every exceptions the function can be expected to thrown. Instead of seeking inside the function body to see what it could possibly throw, this is documented right away, in one and only place. The code is now self-documented, also allowing better intellisense from IDEs and linters. Also, the automatic unexpected exception escalation can be done directly before leaving the function scope, keeping everything encapsulated (which would be in my opinion more intuitive and logical).

About the second point, even if we can't directly see which exceptions could possibly be throw on function calls, linters and IDEs would be able to parse function definitions, and provide these information to developers, ensuring that exceptions are handled correctly. However, this would indeed be weaker than using the expect keyword, as not enforced directly by the semantics of the language.

For now, I think I would be more in favor of using the throws keyword inside function definitions, rather than the expect keyword on function calls. The pros I discussed above looks to me more interesting to have, and also because its cons could still be lowered by using linters or other helpers from outside the language. But of course this could still be debatable :)

I've made a PR on your github repository if you want to see the full example with this idea!

1 Like

Look up criticism of Java's checked exceptions. It'd be ill advised to push for this without being aware of the massive amount of criticism of that.

It's also worth noting that you can force consumers to do their due diligence by simply returning a {status, ...data} object where status could be "success" or whatever errors you want to feature. (Plus, JS doesn't have any decent syntax for catching specific exception types like Java does.)

3 Likes

@claudiameadows Thank you for helping research the criticism for Java's exception system - I knew sooner or later we would have to look into some of those issues and address them before getting too far, and the research you did can be a nice starting point into that.

I'll add one more link I remember enjoying. This compares Java's exception system to Rust's, and tries to figure out why Java's exception system is so disliked while Rust's exceptions are so praised.

As for "returning exceptions", that certainly helps a lot, but won't fix all issues, as mentioned here in the original post.

An interesting conversation happened in @Clemdz's pull request (here) that I thought I would mention here, along with some thoughts about it.

@Clemdz attempted to do a little refactoring on the code that was supposed to model his ideas, so that it better reflected what he was proposing. In the process of this refactor, he ran into this function (I'll post the version using the "excepts" syntax I had originally proposed in this post, as it's more concise)

function givePermissionToUsers(permission, userIds, { userIdDoingRequest }) {
  userManager.assertUserHasPermission(userIdDoingRequest, PERMISSIONS.view) excepts 'Unauthorized'
  userManager.assertUserHasPermission(userIdDoingRequest, PERMISSIONS.update) excepts 'Unauthorized'

  const users = []
  for (const userId of userIds) {
    users.push(
      userManager.getUser(userId, { userIdDoingRequest }) excepts 'NotFound'
    )
  }

  for (const user of users) {
    if (!user.permissions.includes(permission)) {
      const newPermissions = [...user.permissions, permission]
      userManager.setProperty(user.id, 'permissions', newPermissions, { userIdDoingRequest })
    }
  }
}

@Clemdz wasn't sure why there were two for loops, even after looking over the existing variations of this chunk of code, and decided to refactor it into one for loop.

Which was a breaking change.

In my original post (see here), I had mentioned that the problem I hoped to solve in this thread was that of trying to make it obvious when reordering certain statements would cause a breaking change. This function that was wrongly refactored was supposed to be a demonstration point of this type of breaking change I was hoping to avoid.

In the first for loop, we're retrieving all of the users using userManager.getUser(). This getUser() function could potentially throw a 'NotFound' exception if a particular user does not exist. If none of the getUser() function calls fail, then we proceed to update the permissions property on all of the users. By consolidating this into one for loop, we are now causing updates to start happening, before all of the NotFound checks were done. In a single-loop approach, if a getUser() call fails half-way through, then half of the users will end up with updated permissions and the other half will not.

When an exception is thrown, it should leave the system in a good state, so that the caller can handle the exception and recover from the issue (if it's not left in a good state, then a fatal error should be thrown instead) - I wouldn't really call "half of the users updated" a good state that can be recovered from (or, at the very least, it's not an ideal state to leave it in, but I'm sure there are some scenarios where this behavior isn't all that bad).

I take this event as living proof that none of the proposed explicit-exception systems really do a great job at solving the refactoring-issue I was hoping to find a solution for in this thread. Certainly, they help - because the exceptions are explicit, a reader of the code can see that certain exceptions were being thrown in the first for loop, and that this refactoring is indeed a breaking change (Without the exception being explicit, it would require the reader to follow the different function calls to see what kinds of exceptions different functions may throw) - but just because it's possible to figure out that the refactoring would be a breaking change, does not mean it's obvious or easily deducible. You almost have to be specifically looking for this type of thing in order to notice it.

@Clemdz mentioned that simply adding some comments explaining the odd ordering of the code would have gone a long way. I would add that unit tests can help catch these types of accidental breaking changes too. This series of events has caused me to think long and hard about this class of bugs, and I realized that I've been looking at this issue from the wrong angle the whole time. These "accidental refactoring" issues I'm trying to prevent are all related to trying to update a resource in an atomic way. In the code snippet presented above, I'm hoping to fetch all user information before updating it, in order to ensure I can perform the updates as a single atomic action - either all of the updates happen, or none of them do. Moving exception-throwing logic first is just one way to handle atomic updates, and only works in certain scenarios. Other techniques are sometimes needed, like rolling back an update if an exception does occur, etc. I'm now doing some additional research into this topic to see what types of libraries/patterns exist to help make atomic updates easier to work with and less error-prone.

The point I'm getting at here is that the original class of bugs I was hoping to solve in this thread is better solved through the means of atomic update libraries/patterns than through an explicit-exception system (though explicit exceptions certainly help out). I'm sure a lot of contributors to this thread are here mainly with the hopes of getting a better exception system in Javascript anyways (and probably didn't care as much about this atomic-bug thing as much as I did).

The tl;dr

I think we can change the direction of this thread to have the focus of enhancing Javascript's exception system in general, and forget about atomic-related bugs I was originally trying to address in my original post.

Alright @claudiameadows, I've read through the list of articles you've provided and tried to extract out all of the arguments against checked exceptions. I'm going to point them out and see how they could have been addressed in Java (when it was first designed). This will let us know how we can avoid these same pitfalls when designing an exception system.

Arguments against checked exceptions and how to deal with them.

1. Checked exceptions integrate poorly with higher-order functions.

I've certainly run into this before. I've wanted to use Java's fancy stream API, which contains useful functions such as a map function for arrays. However, while mapping elements, I needed to be able to throw a certain exception. This unfortunately was not possible because the map function does not declare in its "throws" clause that I might be throwing my custom exception.

This should be solvable by providing to the map function with a list of exceptions you expect it to propagate when calling it, similar to how you might provide type information to a generic function. As far as I can tell, this issue is mostly an annoyance because Java has yet to provide the language-level features to make exception handling work with higher-order functions.

I've also found this article about "anchored exceptions" that I understand is supposed to fix this issue in Java. It's a dense read, and I'm not sure how much I understood from it, but it's still interesting.

2. Adding a new exception to a type signature is a breaking change, and all client code has to update.

There are potentially two reasons why you might want to cause your function to throw a new exception:
A. What used to be a fatal error is now being treated as an exception (because you found that people actually wanted to recover from that scenario). This is probably the most common scenario, and one reason people get annoyed with Java's checked exceptions. This should not be treated as a breaking change to client code, and client's should not be forced to update their own code just to indicate that they still want to treat that exception as a fatal error.
B. What used to be working code is now being treated as an exception. This is a breaking change, and each user of your API should be forced to decide how to handle it.

There are a few ways this could be fixed. I'll point out a couple (this first one isn't necessarily the best idea, but it shows that this doesn't have to be an issue with checked exceptions). Make it so there are two types of exceptions you declare in the function signature - exceptions that you require the end-user to explicitly handle, and exceptions that you don't care if they handle or not (if they don't handle it, it'll auto-escalate). Another option is to loosen things up a bit and not force changes in thrown exceptions to be a breaking change in client code (the "A" scenario is more common anyway) - this also means you can't force a client to explicitly acknowledge each type of exception that might be thrown when calling a function.

3. Checked exceptions may force users to handle an exception they know shouldn't ever happen.

For example, right before indexing into an array, you might first check if your index is within bounds. When you actually index into an array, it might throw IndexOutOfBounds, which you then have to catch and handle, even though you know that exception should never happen (thus, this exception is really a fatal error in this specific scenario). This is a lot of extra annoying boilerplate.

This can be solved by providing terser syntax to automatically escalate exceptions as fatal errors. Right now, Java users have to do an entire try-catch, catch the specific exception, and rethrow it as an instance of RuntimeException.

4. Checked exceptions are verbose

With each solution we decide, there's always going to be a balance with how verbose it is to use, and how secure it makes us. Different syntax, etc, can help out, but there's always going to be a verbosity cost, and it's something we'll have to keep in mind.

Others?

These are the main issues I saw while reading through the articles. If I missed any important ones, please feel free to speak up and we can address it. But, I think the takeaway here is best described by the Anders Hejlsberg from @claudiameadows's last article: "checked exceptions are a wonderful feature. It's just that particular implementations can be problematic". In other words, with some tweaking, all of these things that people hate about checked exceptions can be solved. We just need to keep in mind the places where Java failed in our own design.



An another note, to continue much further with this general conversation about improving Javascript's exception system, I think it would be good to at least try and figure out some specific things we want in an exception system, and some problems we're hoping to solve. Note that the final solution may not be able to solve all of the presented problems (sometimes the extra syntax or verbosity is not worth it).

I'll start of by naming a few:

Problems the new exception system should solve:

  1. Provide a way to self-document what exceptions a particular function can throw/give.
  2. Provide a way to easily escalate exception into fatal errors when you know that particular exception should never happen. (This solves issue #3 with checked exceptions)
  3. It should be difficult to accidentally ignore important exceptions

Features the new exception system should provide:

  1. It should be possible to throw/give a new exception without forcing all users of your API to update their code (because it was a none-breaking change - this solves issue #2 with checked exceptions)
  2. When an unexpected exception occurs, it should be difficult to forget to escalate it (thus, letting it slip through, and potentially getting inappropriately caught and handled)
  3. It should integrate well with higher-order functions (This solves issue #1 with checked exceptions)

Feel free to suggest other points you feel an exception system should provide.

1 Like

Just adding a new edge to the related-things-on-the-Internet graph:

One of the TypeScript issues also discussing syntax for exceptions: Suggestion: `throws` clause and typed catch clause · Issue #13219 · microsoft/TypeScript · GitHub

3 Likes

What exactly do you mean by "fatal error"? Sounds ominous.

Just to be clear, I wasn't explicitly objecting, just saying you would have to address them to get very far. (Several TC39 members are ex-Java developers, and the similarity won't go missed.)

1 Like

In this thread, this is what I mean when I use these terms:

  • error = programmer error - you can catch them for logging purposes, etc, but you can never recover from them. I've been using the term "fatal error" interchangeably with "error" as the error should eventually be fatal. (e.g. InvalidParameter, AssertionError, etc)
  • exception = Something went wrong while doing a task, but the API user may choose to recover from the exception. (e.g. FileNotFoundException, IndexOutOfRangeException, etc)

I've also defined other phrases in the original post that people have been using back and forth, such as "Propagating an exception" and "Escalating an exception".

I previously talked about how simply "returning exceptions" isn't a great solution, because it doesn't self-document what exceptions are being returned when functions are simply propagating the exceptions down the call stack. I have recently realized that it doesn't have to be this way if you're using Typescript (Typescript isn't something I've had the opportunity to use much, but I would love to use it more - so excuse me if my Typescript example below isn't well-written).

With Typescript, one can declare what types of exceptions a specific function can return in each function signature, and Typescript can help make sure you handle all possible exceptions explicitly (usually). This sort of system certainly isn't perfect (and is probably a little verbose to use), but it is an improvement over what we've got today and might be enough for some people. Here's an example of how this might work:

// Exception utility class

class Exception <T> {
  constructor(public code: T) {}
}

// Example API

interface User {
  username: string,
}

// Notice how in the signature we explicitly declare each type of exception that can be returned.
// It gives this a similar feel to Java's checked exceptions, but without some of Java's issues.
function getUser(userId: number): User | Exception<'NOT_INITIALIZED' | 'NOT_FOUND'> {
  if (!initialized) return new Exception('NOT_INITIALIZED')
  const user = users.get(userId)
  if (!user) return new Exception('NOT_FOUND')
  return user
}

You can recover from all those kinds of errors, depending on the situation. "invalid parameter" could mean you handle it by showing the user an error in the UI.

This is true. Maybe I ought to stop giving example error/exception-types when defining them, as it's always situation-dependent. I think there's a bit of a fuzzy line here as to what is counted as an "exception" and what is counted as an "error", and ultimately it's up to the API designer to decide what constitutes as "misusing the API". The general rule of thumb I follow is "when in doubt, use an error. You can always turn them into exception later if a need arises to handle it (unless you're in Java)".

It's completely reasonable to design arrays that throw an "IndexOutOfRange" exception, allowing users to recover from this issue. It's also reasonable to design an array where the designers expect you to always check your index before indexing into it, and any out-of-bound issues will be thrown as an error. Yes it's possible for the user to recover from it, but that's not how the designers want you to deal with this issue.

There are some cases where an error will always be an error, like, if your program got itself into a bad state and you caught that issue (a common use for asserts).

So I misspoke a bit when I tried to define these two terms, thanks @ljharb for pointing this out. Maybe this is a better definition:

  • error = programmer error - you should never recover from these because they indicates there was a bug in the program (e.g. mis-used an API, program in a bad state, etc).
  • exception = Something went wrong while doing a task. If wanted, you can choose to recover from this issue.

I don't think it's possible for a function (eg) to be able to know, with authority, whether the caller can recover from it or not.

In other words, every single error, without exception (pun unavoidable) is potentially recoverable, and I suspect it simply wouldn't make any sense to attempt to classify some as "unrecoverable" from the implementation site.

1 Like

No, it's not possible for the API to know if the caller can recover or not. It's up to the caller to escalate the exception as an error if it can't handle it.

Even if you can recover from an error, doesn't mean you should. An exception is a binding contract between the API and the user. This contract states that the function will always provide the same exception (no matter how it's refactored), and the exception will always leave the API in a good state. Any error handling without such a contract will surely be unstable, which is why it's deemed "unrecoverable" (or in my updated definitions, I phrased it as "should not recover") - not because it's necessarily impossible to recover from it, but because you shouldn't - instead, you should follow whatever other means the API provides to ensure that error does not happen, or maybe ask the API owner to make this binding contract and turn the error into a proper exception. (Some people attempt to recover from these anyways, by using a fragile regex to match against the content of the error message - a message that was intended to provide debugging details, and could change at a moment's notice).

This particular definition does mean that, for example, when you run eval('x=') and it throws a syntax error, this is really an exception, not an error. A contract is in place (by standards) that ensures this behavior stays consistent and things are left in a good state. There may be times when a user finds it useful to catch and handle this SyntaxError. The standard committee probably categorized it as an error because most of the time this gets thrown because there really was a bug in the code. If we were being more rigorous, we would just call it a SyntaxException and let the user escalate it as an error if they think it indicates a bug in their code - this is why a simple escalation syntax can be useful, and why in the original proposal I gave from the first post, escalation was the default behavior (it was implicitly done if it wasn't handled any other way).

An exception is a binding contract between the API and the user. This contract states that the function will always provide the same exception (no matter how it's refactored), and the exception will always leave the API in a good state

This is decidedly not how I've designed any API I've ever built, nor how any of the standard JS methods are built. It is widely understood that future versions might stop throwing an exception, for example. Where in the JS ecosystem is there a norm that a function will always provide the same exception?

In general, why would anyone ever provide an exception code if they don't expect you to programmatically catch and handle that specific exception? And if they're expecting you to do that, then why would they change that exception on you and break your code? How could I reliably write race-condition resistant code when reading a file, if I'm not supposed to catch and handle node's file-not-found exception (because at any point they may choose to just stop throwing that exception, causing my code to break).

Maybe it was wrong for me to call a SyntaxError an exception if I'm supposed to understand that at any point the standard committee may change the type of error being thrown at a moment's notice. However, there's got to be some set of stable exceptions that are treated as part of a function's API that I can expect to always be present, otherwise, there's really no such thing as being able to reliably catch an exception, handle it, and continue execution.

1 Like

As far as the language is concerned, and most of user land, that’s not the case. node is a bit different - but its error codes don’t mean that error will always occur, it just means that if it does, you can handle it. It also doesn’t preclude new kinds of errors from being added later.

Why is node a special snowflake here? What's is it doing differently that makes it important for node to have well-documented exceptions and no one else?

Doesn't it? If I open a file, and the file doesn't exist, isn't it Node's responsibility to throw the file-not-found exception? If it only sometimes throws it, and it's not well documented the time it will and won't, then how can I ever rely on this behavior? If Node's exception behavior is inconsistent, then the only way I can reliably handle opening a potentially non-existing file is checking for its existence in advance, which exposes my code to race conditions.

You are certainly right here. This is one thing I dislike about Java's checked exceptions is that any new exception you add to the API becomes a breaking API change (even if its addition doesn't actually break anyone's code). Any exception system we design should keep this principle in mind.