Starting Worker from Python in a Docker container?


#1

I try to use Dramatiq within a docker container. I came up with the following program:

import requests
import dramatiq

from dramatiq.brokers.rabbitmq import RabbitmqBroker

rabbitmq_broker = RabbitmqBroker(url="amqp://thomas:xxx@quantsrv:5672")
dramatiq.set_broker(rabbitmq_broker)

@dramatiq.actor
def count_words(url):
  response = requests.get(url)
  count = len(response.text.split(" "))
  print(f"There are {count} words at {url!r}.")


if __name__ == '__main__':
    # note that we run in a docker container. Hence best strategy to fire
    # off a worker
    w = dramatiq.Worker(broker=rabbitmq_broker)
    w.start()

    count_words.send("http://www.spiegel.de//")
    count_words.send("http://example.com")

    w.stop()

I get a bunch of error messages caused by Prometheus:

Attaching to lobquant_base_1
base_1             | Unexpected failure in before_process_message.
base_1             | Traceback (most recent call last):
base_1             |   File "/opt/conda/lib/python3.6/site-packages/dramatiq/broker.py", line 89, in emit_before
base_1             |     getattr(middleware, "before_" + signal)(self, *args, **kwargs)
base_1             |   File "/opt/conda/lib/python3.6/site-packages/dramatiq/middleware/prometheus.py", line 155, in before_process_message
base_1             |     self.inprogress_messages.labels(*labels).inc()
base_1             | AttributeError: 'Prometheus' object has no attribute 'inprogress_messages'
base_1             | Unexpected failure in before_process_message.
base_1             | Traceback (most recent call last):
base_1             |   File "/opt/conda/lib/python3.6/site-packages/dramatiq/broker.py", line 89, in emit_before
base_1             |     getattr(middleware, "before_" + signal)(self, *args, **kwargs)
base_1             |   File "/opt/conda/lib/python3.6/site-packages/dramatiq/middleware/prometheus.py", line 155, in before_process_message
base_1             |     self.inprogress_messages.labels(*labels).inc()
base_1             | AttributeError: 'Prometheus' object has no attribute 'inprogress_messages'
base_1             | There are 338 words at 'http://example.com'.
base_1             | Unexpected failure in after_process_message.
base_1             | Traceback (most recent call last):

I also wonder whether I could do:

with dramatiq.worker(...) as www:
       www.send(fct, args)

#2

You should really use the CLI tool to launch workers, even in containers.

That said, the reason you’re seeing those errors is because you need to call

        broker.emit_after("process_boot")

before starting the worker.