Errors in Rust: A Formula

Errors are a multi layer cake. Errors in Rust have a taste factor because it falls in the “there is more than one way to do it” language camp. It mixes-in ML (as in: Ocaml), functional, and algol-like flavors for its constructs, and
takes the best ideas from other languages, and it has strict opinions on everything
safety: concurrency, memory access, sharing resources, and more.

This is why error handling in Rust is a multi layer cake.

→ → →
🙋‍♀️ Have questions? Want to share your experience?, follow me on Twitter 😎

PS: if you’re impatient you can skip to the bottom to review the formula/checklist for errors in Rust.

Choosing the Right Error Handling Philosophy

As a contrast, look at Go, which takes an arguably correct approach, it says:
“An error is a simple thing. Handle it at the callsite or bust”. And look at Java, which says “Throw it and someone
else will handle it” which, as history proves, can be very wrong, many times.

The trouble is that when errors are precieved to be a simple thing like in Go, they’re often a string, or an object that by the time it reaches the handler, it lacks context or information to make a smart error recovery decision. And with Java’s approach, an error is thrown and the stack is busted, add in a good measure of lack of best practices, and for each layer of bubbling up, you lose more and more context, and again — by the time the handler gets an error, they are in a bad position to make an informed decision.

An Error Philosophy in Rust

The basics of errors in Rust are these:

// define a Result type as a convenience, already encoding-in our Error type
type Result<T> = Result<T, ParserError>;

// an Error is a type, nothing special
#[derive(Debug)]
enum ParserError {
    EmptyText,
    Parse(String),
}
// return a result, which may or may not be an error. to find it `match` on its content
fn parse(..) -> Result<AST>{
  ..
}

You have a Result type, which encodes the Ok and Err states, which are represented as types you can create. The error type can be anything, but an enum is a smart idea (and the most popular one).

A function returns a Result.

For managing errors, Rust picks up on the gaps of other languages, and tells you:

• Handle errors on callsite, or
• Bubble up errors in an orderly fashion (also means: no throw/catch mechanism like in Java)

For handling errors Rust also gives you everything you need, as part of the language. Handling errors is first-class:

• Errors are encouraged to be richly typed (e.g. unlike Go)
• You are encouraged to match on errors, and Rust acknowledges that errors and failures are a rich experience, that can easily become a frustrating one and gives you ergonomics for those not so trivial error handling situations. Again, unlike Go.
• Passing errors around is similar to returning results from functions, which is what a programming language already knows how to do. Like Go, but unlike Java. It means that the programmer should not be caught off guard, and that every single powerful construct a programmer has to create or handle function results (aggregation, expression building, conversion, concurrency, etc.) is available to handle errors, unlike Go and unlike Java.

Because an error is just a return value, you can model this in any other programming language, right?

I can model errors as result types in Java, whats the big deal? and the big deal is how everything connects — when the standard library is all oriented towards this, the syntax factors-in errors as results, and the community and mindset is already there as well your get error-story-nirvana.

<rant>Errors are Not Simple </rant>

While this is not to dis Go, the difference between Go and Rust is a great illustration of how “language thinking” (e.g. Wharf theorem) affects code and cognitive load.

Every person that migrated from Go to Rust that I’ve met — told me
they can’t go back. That they can’t believe how they miscalculated how complex errors were, and that today they can see the amount of edge cases that their Go code was ignorant of.

It’s all about the cognitive load weight transfer: in C, errors are simple because C was an assembly preprocessor. That’s why the complexity is shifted towards the developer. In Go, which aims to classically improve on this, but still be inspired from the simplicity, the weight is shifted towards the developer as well. The difference is that in C you knew you have no safety net.

This is why they couldn’t believe how they were “sold” on the idea that errors are “one dimensional”, simple, (e.g. handle it or just bork, using the typical if err != nil muscle reflex you get in Go). Failures are not simple. Error cases are not trivial. This is why we have spaceships crash, people.

There’s no Free Lunch

Errors in Rust are a learning experience and a powerful tool. Errors aren’t that pesky thing you need to deal with, and a master of errors is a master of the language.

Rust offers tools to build a great error modeling and handling story that can power mission-critical software — if that’s your thing. For me, eventhough I don’t build software for spaceships and neuclear reactors, I always enjoyed building reliable software that doesn’t wake you up at night.

This is why errors in Rust require more investment from your side.

• Creating errors has more cognitive load. You can’t just new Error("oops") or return "can't load" or throw new Foobar(). Errors need to be created as a type and conform to a few rules.
• Connect with a failure story: to provide backtrace, root cause
• Connect with a user experience: provide debug and display implementation which cater to operator and end user respectively.
• Convert results sensibly and responsibly: you’ll quickly realize that a function returning Result<(), ErrA> does not work with an inner function returning Result<(), ErrB>.
• Handle the complexity of a real system, where you can get a multi-layered error: Error(DatabaseError(ConnectionError(PoolError(reason)))), each layer of that error onion, has a fork and a decision you need to explicitly code. This is a Good Thing, unless the library author did a lousy job modeling errors, in which case you're just compensating for their gaps, which is also a bug-preventing action, in anycase.
• Be responsible towards others. The errors you return may be consumed by others. You need to think with intent about what kind of error story they will have.

Structure, semantics, and user experience

We’re going to review some practical patterns, and design challenges. Everything here will be built with just two error libraries.

1. thiserror - for libraries and core
2. eyre (which is similar / drop-in alternative to anyhow) - optional, if you're targeting apps (CLI mostly)

We’ll review:

• How to create and add context to our own errors
• How to layer, nest, and wrap errors
• Map errors from dependencies into our own errors in a 1:1, 1:N, N:1 relationship

Creating errors

By keeping an enum form, you can create an error hierarchy that keeps context:

enum ParserError {
  EmptyFile(String) // the file path
  Json(serde_json::Error) // the original JSON parser error
  SyntaxError { line: String, pos: usize } // a complex error structure
}

When we want to create errors, or return errors we want to keep the following principles:

• Type less: use From extensively and automatic conversions using ?
• Use a ‘for-in’ loop instead of mapping for short-circuiting
• Collect Result<Vec> for aggregation, instead of Vec<Result<..>>

let res: Result<Vec<_>> = foo.map(..).collect();

• Keep the source errors at all times, when possible
• In Error, both display and debug are important: audience; display is for end users (e.g. redact secrets, make a long text shorter), and debug is for operators (e.g. troubleshooting and logging and diagnostics).
• No expect, no unwrap, no panic in your code unless it's a must (and mostly, due to a gap in one of the external libraries)

Crate-level and Module-level Errors

While you can definitely have a single Result type, with a single Error type for your entire crate, often you can benefit from sub-dividing your Error type into sub-errors relevant to your individual crate modules. When using a monorepo with multiple-crate workspace, this is inevitable.

The meaning of this is that you’ll also be sub-dividing your single Result type to many different Result types relevant to your sub-modules.

For example:

compiler/      -> (1) Result<T,E> = Result<.., Error>
  parser/      -> (2) Result<T,E> = Result<.., ParserError>
  scanner/     -> (3) Result<T,E> = Result<.., ScannerError>

Crate-level error type is (1)

Module-level error type is (2) and (3)

The three different Result types are different types. So when returning a Result<.., ParserError> to a compiler function expecting a Result<.., Error> your code will break.

And so, you’ll need to .map_err if the Result T is the same, or re-wrap your Result while passing the value upwards from downstream (parser) to upstream (compiler) dependencies.

Mapping Errors

Sometimes handling errors is simply mapping them into a different error kind, which means picking the original error apart, taking some context from it, or wrapping it as-is into a different error kind.

This requires a thoughful base error to start with. I find that most of the time it’s great to have this as a starting point (and I tend to copy it for every new module I create).

#[derive(thiserror::Error, Debug)]
pub enum Error {
    #[error("{0}")]
    Message(String),
    // a central IO wrapper
    #[error(transparent)]
    IO(#[from] std::io::Error),
    // will be used with `.map_err(Box::from)?;`
    #[error(transparent)]
    Any(#[from] Box<dyn std::error::Error + Send + Sync>),
    
    // some other common conversions
    #[error(transparent)]
    Pattern(#[from] regex::Error),
    #[error(transparent)]
    Json(#[from] serde_json::Error),
}

Wrapping serde_json::Error through Error::Json feels like repeating existing information and not adding any value. Are we just repeating stuff?

Well, there are two things happening here:

1. Consolidating different error types, possibly from different libraries, as one unified enum error type, which streamlines Result types significantly - this is key for useful error handling of your library by third parties. When you use a library, it's best to expect just one error type which tells you about all of the errors that are possible in this library.
2. Enjoying automatic conversions with #[from] to clean up your code, save typing, and save maintenance, by side-stepping the error conversion decision point. You can always go "manual" and remove #[from], and make the conversion at each code point in your code base yourself.
3. Keeping an escape hatch for those anyhow moments. Don’t care about an error? don’t know what to do with it? the library authors made it impossible to work with? Do this karate-chop: .map_err(Box::from)?; and wrap it with your own accessible Error type.

Mapping: N:1

Mapping many 3rd party error types to one of your error types: when would you want to do this?

• When one or more libraries have a too fine level of details in their errors. For example, when Error::HttpThrottling, Error::RateLimit, and Error::AccountDepleated are all the same, indicating: "you're out of your API credits, or you're abusing your credits -- chill out!".
• When you already know it’s game over. Knowing the specifics of the error won’t help your service. For example Error::DiskFull, Error::CorruptPartition, etc. Just wrap them under an MyError::Fatal and keep the orignal error in it, boxed, for more detail.
• When multiple different libraries are doing the same thing, and implement a provider architecture where you can swap different providers implementing a trait. For example, an Error::PostgresConnection, Error::MySqlConnectionPool, with a swappable DB provider in your crate, might mean just a MyError::Connection for you. Remember if the trait needs to be universal over providers, errors returned in trait functions should be too

You have two ways to create this kind of mapping.

1. N:1 Mapping — layer things and bubble up

Essentially we want to create a first-level aggregate error type, and a top-level aggregate of errors.

Say you have database providers, and then you have your crate which does data access.

First, create a first-level error type:

enum DbProviderError {
  // all of these are invariably common to all database
  // providers you're dealing with
  Connection(..)
  PoolLimit(..)
  SqlSyntax(..)
}

Having a trait for these providers, will return the above error, to align all the specifics of each and every different error from the different providers:

trait DbProvider {
  fn connect(..)  ->  Result<(), DbProviderError>
}

Finally, your crate which uses DbProvider, sets up the right provider, etc., needs to be able to accept DbProviderError:

enum MyError {
    //..
    Message(String)
    #[error(transparent)]
    DB(#[from] DbProviderError),
}

Now, using ? to convert a DB error to a crate-level error should create a nice and tidy error story.

NOTE: there will be cases where the two result types of your trait and your crate-level result type will not be compatible and converting through ? will not work. That's where you'll have to manually call .into() on a DbProviderError to turn it into a MyError when doing a .map_err or creating a new DbProviderError yourself.

An example:

fn embed_inputs(&self, inputs: &[&str]) -> Result<Vec<Vec<f32>>> {
      Err(EmbeddingError::Other(
          "an embedding cannot be made".to_string(),
      )
      .into())
}

Here, embed_inputs is in an embedding module, and it has its own error hierarchy and story, and its own Error type.

But, it returns a higher crate-level Result, and while it contains a crate-level Error which can convert an EmbeddingError with a from trait, it cannot be inferred automatically, so we're using into() on our Err directive.

Another way to manually convert is to call up the into trait directly, and then use a ? conversion:

fn embed_inputs(&self, inputs: &[&str]) -> Result<Vec<Vec<f32>>> {
  let res = provider
    .do_something()
    .map_err(Into::<..error type..>::into);
  Ok(res?)
}

A note about folder and module structure

When dealing with providers and providers implementing traits, we many times have to map N error types and variants into 1 variant of our own.

In that case we can create a similar looking error architecture, where each layer knows its own concrete errors.

root/
  error.rs
  providers/
    error.rs 
    providerA/
      error.rs
    providerB/
      error.rs
    ...

And then next step is to offer encapsulated error types and module-local conversions:

root/
  error.rs
    providers/
    error.rs 
      .. { 
        ProviderA(provider_a::error::ProviderAError)
        ProviderB
      }
    providerA/
    error.rs
      .. { 
        SqlConnectionError(extlib::conn:Error)
        DataTransferError(extlib::conn:Error)
      }
    providerB/
      error.rs
    ...

But often, we want to “group” provider errors without caring for the details inside each of those errors, because it’s too low level and because a user cannot handle those, we just have a provider error. In essence we’re grouping errors N to 1.

2. N:1 Mapping — put it in a box

If you don’t care about the specific DB provider error type, you can wrap and box it, and send it up with a MyError crate-level error:

enum MyError {
    //..
    Message(String)
    DB(Box<dyn std::error::Error + Send + Sync>),
}

And then, .map_err(Box::from).map_err(|e| MyError::DB(e)). Note that we're being very explicit here with .map_err, to provide space for more variants that are Box<dyn Error> under the MyError type.

Mapping 1:1

Mapping a single 3rd party error to a one of our own error variants (for the sake of this discussion we treat things like stdlib as 3rd party as well).

This lets us map into our error variants which allow us to jump through error hierarchies as they exist in our modules and crates.

For example for bubbling up through layers, viewed as a set of abstract actions:

3rd-party error -> (wrap!) -> ModuleError -> (wrap!) -> CrateError

Taking a concrete parser example and viewed as a tree:

// crate
ParserError::Invalid(
  // module
  ScannerError::BadInput(
    // 3rd party
    Regex::Error(..)
  )
)

Jumping through layers in code:

fn parse(..) -> Result<String, ParserError> {
 scan()?; // module error -> crate error
}

fn scan(..) -> Result<String, ScannerError> {
 scan_with_regex()?; // lib error -> module error
}

fn scan_with_regex(..) -> Result<String, Regex::Error> {
  ...
}

Do you need all these layer? many times you don’t. But, understanding this basic structuring of errors will let you understand other libraries, and you’ll be able to “cut out” the stuff you need from this bigger picture when you need it.

Mapping 1:N

This happens when you get one type of error and you need more granularity in your own code. A common example is an HTTPError that treats everything like an error but you know that a 404 is different than a 500, so you want different error handling strategies.

Rust does a great job by giving you a .map_err precisely for this. And with a match clause it's also ergonomically enjoyable:

.map_err(|e| match e {
  // use e.code to create your error variants
})

Error tricks

Ad-hoc into

fn do_something(..) -> Result<String> {
  foobar(..).map_err(Into<ModuleError>::into)?;
  ...
}

2-level From

Some times jumping up two layers of errors can be done at every callsite, but when it’s done enough times, it’s better to refactor it out into a From trait. This cleans up your code and centralizes your error decision making efficiently.

impl From<lib error> for Error {
    fn from(e: <lib error>) -> Self {
        Self::SomeCrateError(super::ModuleError::SomeModuleError(...))
    }
}

// some other place
fn do_something(..) -> Result<String> {
  foobar(..)?;
  ...
}

Quick box

If you have this kind of error:

#[derive(thiserror::Error, Debug)]
enum MyError {
    #[error("{0}")]
    Message(String)
    // note this variant
    #[error(transparent)]
    Any(#[from] Box<dyn std::error::Error + Send + Sync>),
}

Then you can do a quick karate-chop to convert what ever error to a MyError::Any.

foo(..).map_err(Box::from)?;

If you use anyhow, using MyError::Any is a good alternative to avoid adding up another library.

Handling errors

Match, map, wrap

Handling errors from the AWS SDK crate, we want to say that a missing parameter in ssm is OK and not an error case when deleting a parameter:

fn handle_delete(e: SdkError<DeleteParameterError>, pm: &PathMap) -> Result<()> {
    match e.into_service_error() {
        DeleteParameterError::ParameterNotFound(_) => {
            // we're ok
            Ok(())
        }
        e => Err(crate::Error::DeleteError {
            path: pm.path.to_string(),
            msg: e.to_string(),
        }),
    }
}

• Match — pick out the various error cases from the main Error type
• Map — create a different semantic for the original Result type (mapping an error case to an Ok case)
• Wrap — return a streamlined, familiar Error type that our crate will expose to end users
  composability

As a general rule of thumb, handling errors will always be a selection of (1) match, (2) map, (3) wrap, or all of those combined.

Exit handling

In many cases, you have no way to actually recover from an error. So the best you can do is print it out, and signal it appropriately (e.g. using proper Unix exit codes).

Here’s one example of such handling, where a program can report an expected error as part of a CmdResponse value, which is not a Rust Error, but also, given an unexpected error with Error, will report it in an orderly fashion.

const DEFAULT_ERR_EXIT_CODE: i32 = 1;
pub fn result_exit(res: Result<CmdResponse>) {
    let exit_with = match res {
        Ok(cmd) => {
            if let Some(message) = cmd.message {
                if exitcode::is_success(cmd.code) {
                    eprintln!("{message}");
                } else {
                    eprintln!("{} {}", style("error:").red().bold(), style(message).red());
                };
            }
            cmd.code
        }
        Err(e) => {
            eprintln!("error: {e:?}");
            DEFAULT_ERR_EXIT_CODE
        }
    };
    exit(exit_with)
}

Think about your surface area

Make sure your crate exposes a single error type.

Why?

• It will make your users be able to create a single from conversion, and be done with it
• Documentation of what to handle and how to handle is in one single place
• For those who want to cover all cases, matching all variants of a single Error enum covers it faithfull
• Reporting, debugging, and operability focus — while working with your crate, users are getting to know a single error type and its context intimately and so be able to handle it effectively whether within a debugging session or through code
• One error type means one kind of Result type, which is by itself a better API design
• If needed, nest other error types within that single error type in one of its variant

Errors: a Formula

Follow these steps for error nirvana in Rust.

1. Add and learn your dependencies

• thiserror for all errors in your crate
• eyre for CLIs

2. Create a base error type per crate

You can place it in your top level mod.rs or lib.rs.

// lib/mod.rs
#[derive(thiserror::Error, Debug)]
pub enum Error {
    #[error("{0}")]
    Message(String),
    // a central IO wrapper
    #[error(transparent)]
    IO(#[from] std::io::Error),
    // will be used with `.map_err(Box::from)?;`
    #[error(transparent)]
    Any(#[from] Box<dyn std::error::Error + Send + Sync>),
    
    // some other common conversions
    #[error(transparent)]
    Pattern(#[from] regex::Error),
    #[error(transparent)]
    Json(#[from] serde_json::Error),
}

3. Match, map, wrap

Take errors from dependencies, stdlib or your other modules, and where needed match and extract information, or map them with map_err, or wrap them into your own error type.

4. In your code, keep using ? conversions

Let the compiler help you and make automatic from conversions.

If an error from a 3rd party library cannot be automatically converted, add an enum variant to your top level crate error with the #[from] attribute. Verify that you're not creating competing variants (the compiler will let you know).

When trying to convert multiple layers of errors, code your own From trait to help centralize error making decision points.

impl From<RustBertError> for Error {
    fn from(e: RustBertError) -> Self {
        Self::EmbeddingErr(super::EmbeddingError::SentenceEmbedding(Box::from(e)))
    }
}

Remember, you can also .map_err into an error that can be converted via ? and from traits.

5. Create contextful variants but don’t overdo it

Create variants that contain information that is available when an error is created.

InvalidSyntax{ file: String, line: usize, reason: String }

But don’t overdo it by containing every single piece of information there is in the universe.

6. Think about operability

Ask yourself:

• Can a user do something to recover with the information you’re encoding with an error you’re creating?
• Is it an automatic recovery? or manual?
• Is there an importance for time? for space? for resources? hardware?
• Will errors appear in logs? what would they need to contain?
• After crashing, will an error give a user enough information to fix an issue?

7. When all else fails, Box it

Use a variant in your error that can just take a dyn Error if the originating error is not important but you have to create an ergonomic codebase.

🙋‍♀️ Have questions? want to say hi? follow me on Twitter