Nim - Playing with Records

   
   
 cli  
 code  
Article Series

This article series discuss more than 30 different programming languages. Please read overview before you read any of the details.

Playing with Records Related Articles.

Table of Content
Where to Discuss?

Local Group

Preface

Goal: A practical case to collect unique record fields using Nim.

Nim is surprisingly very easy, and also fun to work with. It is interesting, that Nim is using macro as syntatic sugar. But I still can’t predict yet how far the impact will go.

Reference Reading

Source Examples

You can obtain source examples here:

Common Use Case

Task: Get the unique tag string

Please read overview for more detail.

Prepopulated Data

Songs and Poetry

type
  StringSeq* = seq[string]
  Song* = tuple[title: string, tags: StringSeq]

let songs*: seq[Song] = @[
  ( title: "Cantaloupe Island",
    tags : @["60s", "jazz"]),
  ( title: "Let It Be",
    tags : @["60s", "rock"]),
  ( title: "Knockin' on Heaven's Door",
    tags : @["70s", "rock"]),
  ( title: "Emotion",
    tags : @["70s", "pop"]),
  ( title: "The River",
    tags : @[]),
]

Nim Solution

The Answer

There might be many ways to do things in Nim. One of them is this oneliner as below:

import sequtils, sets, MySongs
echo songs.mapIt(it.tags).foldl(a & b).toHashSet.toSeq

Enough with introduction, at this point we should go straight to coding.

Environment

No need any special setup. Just run and voila..!

1: Data Structure

We are going to use sequence throught out this article.

Simple Array

Before building a struct, I begin with simple array.

var tags: array[5, string] =
  ["rock", "jazz", "rock", "pop", "pop"]

echo tags

It is easy to dump variable in nim using echo. With the result similar as below array:

$ nim compile --run t01_tags
Hint: used config file '/etc/nim/nim.cfg' [Conf]
Hint: used config file '/etc/nim/config.nims' [Conf]
Hint: 7849 lines; 0.038s; 6.996MiB peakmem; Debug build; proj: /home/epsi/Documents/songs/nim/t01_tags; out: /home/epsi/Documents/songs/nim/t01_tags [SuccessX]
Hint: /home/epsi/Documents/songs/nim/t01_tags  [Exec]
["rock", "jazz", "rock", "pop", "pop"]

You can also run nim without making executable:

$ nim r t01_tags

Hints

There is a purpose of showing hint. But if you wish, it would be clearer if we surpress the hints.

$ nim compile --run --hints:off t01_tags.nim
["rock", "jazz", "rock", "pop", "pop"]

Or set it in nim.cfg

❯ cat /etc/nim/nim.cfg | grep hint
hint[LineTooLong]=off
#hint[XDeclaredButNotUsed]=off
hints=off

Now we can run the compilation quietly.

$ nim r t01_tags.nim
["rock", "jazz", "rock", "pop", "pop"]

The Song Tuple Type

We can continue our journey to records using type and tuple. Since we don’t know the length of the array, we use sequence instead.

type
  StringSeq = seq[string]
  Song = tuple[title: string, tags: StringSeq]

let song: Song = (
    title: "Cantaloupe Island",
    tags : @["60s", "jazz"]
  )

echo song

With the result similar as below record:

$ nim r t02_song.nim
(title: "Cantaloupe Island", tags: @["60s", "jazz"])

Nim: The Song Tuple Type

Array of Song Struct

Meet The Songs Array

From just karaoke, we can go pro with recording studio. From simple data, we can build a structure to solve our task.

type
  StringSeq = seq[string]
  Song = tuple[title: string, tags: StringSeq]

let songs: seq[Song] = @[
  ( title: "Cantaloupe Island",
    tags : @["60s", "jazz"]),
  ( title: "Let It Be",
    tags : @["60s", "rock"]),
  ( title: "Knockin' on Heaven's Door",
    tags : @["70s", "rock"]),
  ( title: "Emotion",
    tags : @["70s", "pop"]),
  ( title: "The River",
    tags : @[]),
]

for i, song in songs: writeLine(stdout, song)

It is more convenience to show the result using for, with the result similar as below sequential lines of Song type:

$ nim r t03_songs.nim
(title: "Cantaloupe Island", tags: @["60s", "jazz"])
(title: "Let It Be", tags: @["60s", "rock"])
(title: "Knockin\' on Heaven\'s Door", tags: @["70s", "rock"])
(title: "Emotion", tags: @["70s", "pop"])
(title: "The River", tags: @[])

Instead of just echo song, we can utilize writeLine(stdout, song).

Maybe Null

A not too simple, case example.

Consider make this data stucture more complex, with adding nullability option as shown in below type:

type
  StringSeq = seq[string]
  Song = tuple[title: string, tags: Option[StringSeq]]

The complete code can be shown as below:

import options

type
  StringSeq = seq[string]
  Song = tuple[title: string,
               tags: Option[StringSeq]]

let songs: seq[Song] = @[
  ( title: "Cantaloupe Island",
    tags : some(@["60s", "jazz"])),
  ( title: "Let It Be",
    tags : some(@["60s", "rock"])),
  ( title: "Knockin' on Heaven's Door",
    tags : some(@["70s", "rock"])),
  ( title: "Emotion",
    tags : some(@["70s", "pop"])),
  ( title: "The River",
    tags : none(StringSeq)),
]

for i, song in songs: song.echo

With the result similar as below record:

$ nim r t04_options.nim
(title: "Cantaloupe Island", tags: Some(@["60s", "jazz"]))
(title: "Let It Be", tags: Some(@["60s", "rock"]))
(title: "Knockin\' on Heaven\'s Door", tags: Some(@["70s", "rock"]))
(title: "Emotion", tags: Some(@["70s", "pop"]))
(title: "The River", tags: None[StringSeq])

This is just an example how maybe null can be done in Nim, but we will discard this Option example, for the rest of this article.

2: Separating Module

Since we need to reuse the songs record multiple times, it is a good idea to separate the record from logic.

Songs Module

The code can be shown as below:

type
  StringSeq* = seq[string]
  Song* = tuple[title: string, tags: StringSeq]

let songs*: seq[Song] = @[
  ( title: "Cantaloupe Island",
    tags : @["60s", "jazz"]),
  ( title: "Let It Be",
    tags : @["60s", "rock"]),
  ( title: "Knockin' on Heaven's Door",
    tags : @["70s", "rock"]),
  ( title: "Emotion",
    tags : @["70s", "pop"]),
  ( title: "The River",
    tags : @[]),
]

Nim: The Songs Module Containing Sequence of Tuples

Using Songs Module

Now we can have a very short code.

import MySongs

for i, song in songs: echo song

With the result exactly the same as above array.

Nim: Using Songs Module

3: Finishing The Task

MapIt, FilterIt, Flatten, Unique, Oneliner Chain

Extracting Fields

We can use map, or even better mapIt. And also chain the function with filterIt, in oneliner fashioned.

import sequtils
import MySongs

let tagsSeq = songs
  .mapIt(it.tags)
  .filterIt(it != @[])

for i, tags in tagsSeq: echo tags

With the result similar as below record:

$ nim compile --run --hints:off t06_filter
@["60s", "jazz"]
@["60s", "rock"]
@["70s", "rock"]
@["70s", "pop"]

The last tuple with empty sequence is not shown anymore.

Flatten

We can utilize foldl from sequtils library, to flatten a nested sequence. The & operator will concat sequence for each iteration. The a is accumulator, and the b is the value for each iteration.

import sequtils
import MySongs

let 
  tagsSeq: seq[StringSeq] = songs.mapIt(it.tags)
  tags: StringSeq = tagsSeq.foldl(a & b)

echo tags

With the result similar as below sequence:

$ nim compile --run --hints:off t07_flatten
@["60s", "jazz", "60s", "rock", "70s", "rock", "70s", "pop"]

For convenience, I add the data type, so you can analyze further.

Unique

In order to get to get distinct values, we can convert a sequence to either hash set or ordered set. So I guess our problem is solved completely.

import sequtils, sets, MySongs

songs
  .mapIt(it.tags)
  .foldl(a & b)
  .toOrderedSet
  .toSeq
  .echo

We can also chain every function, and convert back from hash set/ordered set to sequence, before finally echo the result, with the result similar as below sequence:

$ nim r t08_unique.nim
@["60s", "jazz", "rock", "70s", "pop"]

Nim: Finishing The Task

Now the code is very clear. We can understand exactly what it does, by just reading the code.

4: Alternate Approach

Since our introduction to nim above is too short, consider to makes it more complete with options, proc, alternate flatten written in imperative fashioned.

Songs Module with Option

import options

type
  StringSeq* = seq[string]
  Song* = tuple[title: string,
               tags: Option[StringSeq]]

let songs*: seq[Song] = @[
  ( title: "Cantaloupe Island",
    tags : some(@["60s", "jazz"])),
  ( title: "Let It Be",
    tags : some(@["60s", "rock"])),
  ( title: "Knockin' on Heaven's Door",
    tags : some(@["70s", "rock"])),
  ( title: "Emotion",
    tags : some(@["70s", "pop"])),
  ( title: "The River",
    tags : none(StringSeq)),
]

Custom Flatten Function

Now we can have other approach for flattening the tags This is simply flatten using all tags from all songs records, using nested for in loop.

import options
import MaSongs

proc flatten(songs: seq[Song]): seq[string] =
  var tags: seq[string] = @[]

  for i, song in songs:
    if (song.tags != none(StringSeq)):
      for ii, tag in song.tags.get():
        tags.add(tag)

  result = tags

echo songs.flatten()

With the result similar as below sequence:

$ nim r t09_flatten.nim
@["60s", "jazz", "60s", "rock", "70s", "rock", "70s", "pop"]

Nim: Custom Flatten Function in Imperative Fashioned

We are going to utilize this imperative fashioned as a show case, to pass data between thread using channel.

5: Concurrency with Channel

Concurrency in nim, can be handled through channel. Based on flatten function above, we are going to build show case to pass data between thread.

Reference

The Skeleton

Now we should be ready for the real demo. This is only consist of one short file.

First we need a global channel variable. Then we have these three functions below:

import options
import MaSongs

var chan: Channel[Option[string]]

proc sender(songs: seq[Song]) = …

proc receiver() = …

proc runBoth() = …

runBoth()
  1. Sender
  2. Receiver
  3. runBoth (optionally)

Sender and Receiver

We should prepare two functions, one for Sender, and the other one for Receiver.

The first one sending each results to Sender channel, as an option with either some value or none.

proc sender(songs: seq[Song]) =
  for i, song in songs:
    if (song.tags != none(StringSeq)):
      for ii, tag in song.tags.get():
        chan.send(some(tag))
  chan.send(none(string))

And from the Receiver channel, all the result would be gathered as sequence.

proc receiver() =
  var tags: seq[string] = @[]

  while true:
    let tried = chan.tryRecv()
    if tried.dataAvailable:
      if tried.msg != none(string):
        tags.add(tried.msg.get())
      else:
        echo tags
        break

This way we can send the result through a channel, instead of directly pushing the result to sequence.

Running Both Thread

They do not aware of each other presence.

Consider gather both function in a function.

proc runBoth() =
  # Initialize the channel.
  chan.open()

  # Launch the worker.
  var worker1: Thread[seq[Song]]
  createThread[seq[Song]](worker1, sender, songs)

  var worker2: Thread[void]
  createThread(worker2, receiver)

  # Wait for the thread to exit
  worker1.joinThread()
  worker2.joinThread()

  # Clean up the channel.
  chan.close()

runBoth()

With the result similar as below sequence:

$ nim r --threads:on t10_channel.nim
@["60s", "jazz", "60s", "rock", "70s", "rock", "70s", "pop"]

Nim: Concurrency with Channel: Sender and Receive

We are done with simple concurreny example. Concurency is not so complex after all.

What is Next 🤔?

Consider continue reading [ Crystal - Playing with Records ].