Blog

Blog Post

Working with Erlang records in Elixir
Posted By nick, December 21, 2015

Elixir is a great way to hop on the power of the Erlang runtime with a more friendly syntax. Working with Erlang records can be a bit un-intuitive though, here’s how we are dealing with it.

The Elixir Record module handles interoperating with Erlang records, and we can use it for a quick test:

require Record
defmodule Person do
  Record.defrecord :person, [:name, age: 25]
end

require Person
default = Person.person
# {:person, nil, 25}
named = Person.person(name: "Bob Jones")
# {:person, "Bob Jones", 25}
name_and_age = Person.person(name: "Bob Jones", age: 55)
# {:person, "Bob Jones", 55}
name_and_age_modified = Person.person(name_and_age, age: 56)
# {:person, "Bob Jones", 56}

But how do we use existing Erlang records, defined externally? For example, we are using the erlcloud library to access Amazon S3 resources, some of which can be quite large. The erlcloud library sets a default timeout of 10 seconds for get_object requests, which doesn’t fly with objects that can be 100+ MB in size.

If we request a large enough file and it takes more than 10 seconds to download, we get this timeout:

:erlcloud_s3.get_object('my-s3-bucket-name', 'path/to/huge/file')
# ** (ErlangError) erlang error: {:aws_error, {:socket_error, :timeout}}
#     (erlcloud) src/erlcloud_s3.erl:1022: :erlcloud_s3.s3_request/8
#     (erlcloud) src/erlcloud_s3.erl:497: :erlcloud_s3.get_object/4

Each function in the erlcloud library takes an optional Config parameter, which is specified to be an aws_config record. The aws_config record is defined here, and includes a timeout field we can set, here’s a snippet:

-record(aws_config, {
          as_host="autoscaling.amazonaws.com"::string(),
          s3_scheme="https://"::string(),
          ...
          timeout=10000::timeout(),
          retry=fun erlcloud_retry:no_retry/1::erlcloud_retry:retry_fun()
         }).
-type(aws_config() :: #aws_config{}).

In an existing project, we can define a module to house records, or use them in an existing module, for example purposes we’ll make a new one. We use Record.extract/2 to pull in the existing definition.

defmodule MyProject.AWS.Records do
  require Record
  import Record, only: [defrecord: 2, extract: 2]

  defrecord :aws_config, extract(:aws_config, from_lib: "erlcloud/include/erlcloud_aws.hrl")
end

Where the from_lib path is the location of our erlcloud dependency’s include file mentioned above, relative to the library’s path.

Now we can use this to define a default record:

require MyProject.AWS.Records
MyProject.AWS.Records.aws_config
# {:aws_config, 'autoscaling.amazonaws.com', 'ec2.amazonaws.com',
#  'iam.amazonaws.com', 'sts.amazonaws.com', 'https://', 's3.amazonaws.com', 80,
#  ...
#  'https://', 'cloudtrail.amazonaws.com', 80, :undefined, :undefined, :undefined,
#  10000, false, :lhttpc, :default, &:erlcloud_retry.no_retry/1}

erlcloud will also offer up its default configuration, one with more useful values if we want to override just the timeout value. We get this via :erlcloud_aws.default_config:

default = :erlcloud_aws.default_config
# {:aws_config, 'autoscaling.amazonaws.com', 'ec2.amazonaws.com',
#  'email.us-east-1.amazonaws.com', ..., :undefined, 10000, false, :lhttpc,
#  :default, &:erlcloud_retry.no_retry/1}

From here we can make a new copy w/the timeout value changed to 30 seconds:

updated = MyProject.AWS.Records.aws_config(default, timeout: 30000)
# {:aws_config, 'autoscaling.amazonaws.com', 'ec2.amazonaws.com',
#  'email.us-east-1.amazonaws.com', ..., :undefined, 30000, false, :lhttpc,
#  :default, &:erlcloud_retry.no_retry/1}

Notice the 30000 where there once was 10000. From here, we can pass it on to the get_object function and have a little more patient request:

:erlcloud_s3.get_object('my-s3-bucket-name', 'path/to/huge/file', updated)
# [etag: '"078194bdc907e09b0456ca0112bc92c7"', content_length: '143432958',
#  content_type: 'binary/octet-stream', content_encoding: :undefined,
#  delete_marker: false, version_id: 'null', content: "..."]

Contact

Say Hello!

Contact
hello@codelever.com
(402) 885-9521
14301 FNB Parkway, Suite 100, Omaha, NE 68154
Interact