Lessons Learned

contact@vinta.com.br

heroku's addons for memcached are not explicitly compatible with Celery as result backends, but you can use them with django-celery-results https://github.com/celery/django-celery-results

We created a custom cache for Celery and used Memcached Cloud. Our settings for this are looking like:

INSTALLED_APPS = [
    # ...
    'django_celery_results',
    # ...
]

CACHES['celery'] = {
    'BACKEND': 'django_bmemcached.memcached.BMemcached',
    'LOCATION': MEMCACHED_SERVERS,
    'OPTIONS': {
        'username': MEMCACHED_USERNAME,
        'password': MEMCACHED_PASSWORD,
    }
}

CELERY_RESULT_BACKEND = 'django-cache'
CELERY_CACHE_BACKEND = 'celery'

Obs.: To use Memcached Cloud on Django you have to follow these instructions

Did you like?
1

To list the current Celery tasks names in Redis queue, use:

import base64
from redis import StrictRedis
redis_client = StrictRedis.from_url(BROKER_URL)
default_queue = redis_client.lrange('celery', 0, -1)
print({json.loads(base64.b64decode(json.loads(message.decode('utf-8'))['body']).decode('utf-8'))['task'] for message in default_queue})

For reserved and active, use app.control.inspect(): http://docs.celeryproject.org/en/latest/userguide/workers.html#inspecting-workers

Did you like?
1

In Celery configuration use:

app.autodiscover_tasks(lambda: [n.name for n in apps.get_app_configs()])

instead of:

app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)

autodiscover_tasks() will not understand if you declare in INSTALLED_APPS some app through django AppConfig. (Ex: INSTALLED_APPS += ['users.apps.UsersConfig', ])

Did you like?
1