Python - Playing with Records - Part Three

Preface

Goal: A simple case example of concurrency using Python.

Preface

Goal: Continue Part Two

7: Approach in Solving Unique

A custom example, that you can rewrite.

An Algorithm Case

Why reinvent the wheel?

My intention is writing a custom pattern matching example, so later I can rewrite other algorithm based on this example. You can read the detail on the first article. The record overview.

Consider going back to simple data. We need to discuss about solving unique list.

x:xs Pattern Matching

The original x:xs pattern matching is made of two functions:

1. Exclude
2. Unique

Pattern matching in Python, is not as fancy as haskell, or even OCaml. But what we have in Python is enough, to solve our little problem.

Exclude Function

The exclude function is just a filter with below details:

• gitlab.com/…/python/21-exclude.py

tags = ['60s', 'jazz', '60s', 'rock',
        '70s', 'rock', '70s', 'pop']

def exclude(val, tags):
  return [tag for tag in tags if tag != val]

print(exclude('rock', tags))

With the result as below array:

❯ python 21-exclude.py
['60s', 'jazz', '60s', '70s', '70s', 'pop']

Recursive Unique Function

With exclude function above, we can build unique function recursively, as below code:

• gitlab.com/…/python/22-unique-a.py

from MySongs import songs

tags = ['60s', 'jazz', '60s', 'rock',
        '70s', 'rock', '70s', 'pop']

def exclude(val, tags):
  return [tag for tag in tags if tag != val]

def unique(tags):
  if len(tags) <= 1: return tags
  else:
    head, *tail = tags
    return [head] + unique(exclude(head, tail))

print(unique(tags))

• Although the syntax looks is, not exactly the same as functional programming in haskell, the algorithm is the same.

With the result as below array:

❯ python 22-unique-a.py
['60s', 'jazz', 'rock', '70s', 'pop']

Now we can apply the method to our unique song challenge.

• gitlab.com/…/python/22-unique-b.py

from MySongs import songs

tags = [
  tag for song in songs
  if 'tags' in song
    for tag in song['tags']
]

def exclude(val, tags):
  return [tag for tag in tags if tag != val]

def unique(tags):
  if len(tags) <= 1: return tags
  else:
    head, *tail = tags
    return [head] + unique(exclude(head, tail))

print(unique(tags))

• With the same result as below

❯ python 22-unique-b.py
['60s', 'jazz', 'rock', '70s', 'pop']

8: Generator

Yield

With imperative approach, we can also flatten tags.

We are going to consider this approach for concurrency topic.

Yield

Consider this function.

• gitlab.com/…/python/24-generator.py

def sender(songs):
  for song in songs:
    if 'tags' in song:
      for tag in song['tags']:
        yield tag

Entry Point

Then we can call later:

# main: entry point
from pprint import pprint
from MySongs import songs

pprint(list(sender(songs)))

With the result of

❯ python 24-generator.py
['60s', 'jazz', '60s', 'rock', '70s', 'rock', '70s', 'pop']

The Skeleton

With above code, we can build the basic of sender receiver function.

• gitlab.com/…/python/24-next.py

def receiver():
  # ...

def sender():
  # ...

# main: entry point
...

Sender

Yield

It is very similar to our previous function, but we add None value as a stopper.

def sender(songs):
  for song in songs:
    if 'tags' in song:
      for tag in song['tags']:
        yield tag
  yield None

This None value is yielded outside the loop.

Receiver

Next

Now we can build a while loop that will stop, whenever the value is None.

The next function will resume any yielded value, one value for each next call.

def receiver(mygenr):
  tags = []

  while True:
    item = next(mygenr)
    if item is None:
      return tags
    else: 
      if not (item in tags):
        tags.append(item)

We also test for unique value before appending to tags array.

Main

And finally run both function.

# main: entry point
from pprint import pprint
from MySongs import songs

mygenr = sender(songs)
pprint(receiver(mygenr))

With the result as below:

❯ python 24-next.py
['60s', 'jazz', 'rock', '70s', 'pop']

Now we can have unique tags, with very different approach, compare with our previous list comprehension.

9: Concurrency with Queue

Python support Actor Model pattern. We can rewrite previous process through queue.

Reference

• A synchronized queue class

The Skeleton

We should be ready for the real demo. This is only consist of one short file.

• gitlab.com/…/python/23-queue.py

import threading, queue
from MySongs import songs

q = queue.Queue()
tags = []

def receiver():
  # ...

def sender():
  # ...

threading.Thread(
  target=receiver, daemon=True
).start()

sender()
q.join()

1. Producer: sender()
2. Consumer: receiver()
3. Program entry point: main()

Producer and Consumer

We should prepare two functions:

1. One for Producer that put to queue,
2. And the other one for Consumer that get from queue.

Producer that put to queue.

def sender():
  for song in songs:
    if 'tags' in song:
      for tag in song['tags']:
        q.put(tag)
  q.put(None)

Consumer that get from queue.

def receiver():
  while True:
    item = q.get()
    if item is None:
      q.task_done()
      break
    else: 
      if not (item in tags):
        tags.append(item)
      q.task_done()

Running Both Routine

Pretty short right!

Consider gather both function in main entry point.

• gitlab.com/…/python/23-queue.py

def main():
  # turn-on the worker thread
  threading.Thread(
    target=receiver, daemon=True
  ).start()

  # block until all tasks are done
  sender()
  q.join()
  print(tags)

main()

With the result as below array:

❯ python 23-queue.py
['60s', 'jazz', 'rock', '70s', 'pop']

This is the end of our python journey in this article. We shall meet again in other article.

What is Next 🤔?

Consider continue reading [ Ruby - Playing with Records - Part One ].