MEGA: Enchanting decoders (part 1)

MEGA: Enchanting decoders (part 1)

In this MEGA’s article, we’ll see some useful patterns to discover magic powers hidden inside Elm decoders.

Introduction: The gate to darkness

Once upon a time, in the Pure Language realm, everything was simple, bright and predictable.
That was until a despicable programmer fairy opened the gates to the realm of Impure Languages…

This could quite easily be your story as an Elm beginner. You were used to writing clean and composable code. But then you realised that everything’s sandboxed. So you still need to deal with stuff like browser, JavaScript and API calls.

And that’s when you probably met your first decoder.

A decoder is a piece of code that builds a parser to turn your serialized data into your Elm types. Otherwise, it fails.

It’s needed every time you need to obtain data at runtime. For instance, events, API calls, ports and subscriptions.

Out of the box Elm ships with basic JSON parsing capabilities, that allow you to deal with JavaScript and REST API.
Some libraries empower JSON decoding. And others allows us to deal with data different to JSON (like GraphQL).

In the following examples, I’ll show some interesting (I hope) cases of JSON parsing of a rest API.
And I’ll use elm-json-decode-pipeline, a great package that we use a lot here at Prima.

Master decoders and keep your world fair

Granted, decoders can be a bit of a headache. But once you master them, you can rid your Elm code of a lot of data consistency checks.

Decoders allow you to move all that “if-then” logic in one single point. You can then build your application model not around the backend, but around the user application needs and keep your code cleaner.

Do you remember the examples from Maybe maybe is not the right choice article?

Let’s imagine the situation where Gift is provided by a REST API like the following (I’m using jsdoc to describe explicit nullable fields, so imagine a case where we have some kind of type guarantee from BE). And you’re writing a Payment application:

Let’s consider the price key at first.

Do we need to have Maybe Float in our model?

The answer’s no. Because the API contract says that, in every legal case that value exists, there’s a number type and it’s always a price. So a more suitable data type for the frontend could be a Decimal of decimal package.

The package exposes a nice function:

So now we have two options:

  1. The not-so smart option. Decode a Maybe Float and keep that in our model. Every time we’ll need it we’ll turn that into a Decimal resolving two Maybes. Since we can’t handle Nothing variant (it’s a payment page) we’ll probably resolve the Nothing with a default.
  2. The smart option. Attempt to decode our Maybe Float into a Decimal and if we can’t… well probably something is going very wrong outside there!

Since we want to be smart let’s write our decimal decoder joining together existing ones with a pinch of custom logic to make it decode a Decimal or fail:

Now it’s time for Json.Decode.Pipeline ! We can use its required function to ensure that the price field exists and our new decoder to turn it into a Decimal:

That’s all. Now your Gift model has its nice and well typed price field!

So now imagine that we want to decode the Gift type with its optional greeting card pieces of information.

The first naive approach is to write a very simple decoder (because you know they can be a headache) that decodes a record like this:

Instead of something better shaped like:

As seen in the previous article, we prefer the second choice. So we’d need to convert that record into our union type.

But how?

Well, with a well-structured decoder, we can make all the “bogus” cases collapse into the decoding failure and clean everything up.

When you work with decoders, it helps to think locally. So we’ll try to break down the big problem into simpler tasks.

First of all, we’ll try to decode the raw pieces of information that we need to build a SimpleGift:

Let’s do the same with the other variant, reusing the previous decoders. We could rewrite a whole decoder. But I’m lazy and I can use Json.Decode.Pipeline.custom function:

Now we can simply combine them with Json.Decode.oneOf . This function tries to resolve the serialized data with a list of decoders until one succeeds (so pay attention to priority, in case you don’t have a discriminating value like hasGreetingCard ) :

That could become (with some love and modularization that I’ll leave to you) something like:

And that’s it. We’ve moved all our data coherence checks into a single point. Plus, we can expose only a simple fetch function, the whole decoder, or both as preferred.

We’ve reached more than one goal with that:

  • We wiped out a loosely typed maybish record in favour of a more fashionable union in the whole app.
  • We can move the whole business logic and consistency checks about Gift in a single point.
  • We can make our consistency checks more or less strict based on our BE and requirements. We can morph the raw data directly here, providing default cases instead of making everything fail. For example what happens if gift card values are truly nullable but you always provide them even if not given (with fallback values)? You can simply provide the decoder with your fallback in case of null:

Surely you’ve noticed how messy your update function becomes when you have more than one API call and a little logic (like some security checks on it):

You have to admit that it’s not nice or scalable. A lot of branches are probably the same. You can group the logic with functions but the update case will stay huge.

We can do better. Let me explain.

You probably noticed that your http calls always produce a Result Http.Error decodedData as come back result.
But Elm’s faces 2 possible failures.
The first is during the request process (that always comes out as a Response type):

The other possible failure happens during the decoding process and results in a parsing error:

This because you’re used to passing Http.expectJson as request expectation:

Such function takes the internal Response body type, if the response is ok, then attempts to decode the body with your decoder.

If Response indicates a failure or decoder fails then the result will be a Http.Error type that wraps the previous two errors:

It’s fast and simple. But there are a couple of drawbacks:

  1. It hides some Response information. (We can’t access the body in the case of a BadStatus. We can’t see response headers. And we can’t intercept decoding failures.)
  2. In the case of a reusable library, you’re demanding a lot of business logic of the consumer regarding the API.
  3. The component’s API can’t be self-explanatory.

We can fix almost all of these weak points though.

First of all, let’s define the Gift error that we’re interested to catch:

And after that, we can write our custom expect , that intercepts 401 status code and applies another decoder on that body:

We can now make our fetch function more expressive:

And change our custom expect function, adding the message retagging:

And finally, in our update function, we can collapse every common logic in one dedicated state, like this:

If you’ve reached this point (without a headache), you’ve gained an entry-level Decoder Master.

Wait a minute… all of this and I’m still an entry-level ??

Ehm yes… You’ve learned that decoders can be more than simple string parsers, and how to make them more effective in cleaning up a malformed server API.

You should be able to decode complex elm types and rewrite the basic “decode or fail” logic hidden inside HTTP requests.

This is great and helps make you a better Elm’s sorcerer. But there is still much to say about decoders. I’ll try to cover up the most interesting parts in future articles.

See you soon in the next castle!



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Ivan Gori

HCI obsessed, JS experimentalist, 3D enchanted and actually Elm apprentice —