It is DjangoCon time! As avid participants, we are super excited for this year's edition of DjangoCon US. We've been participating in the event since 2017 and sponsoring it since 2018. This community built around the framework we love has a special place in our hearts, and we are excited to be able to participate again this year.
DjangoCon comes back in the same hybrid model they did last year. A necessity after the COVID-19 pandemic discouraged many people from attending in-person events. In their own words, the event: "has something for everyone, from the person who develops Django applications for a living to the person who just tinkers in their spare time."
If you still need to attend, we strongly recommend you check their profiles here and here. This way, you can check talks from past editions and stay in on the news about where the conference will happen next year!
This year, the conference will run for five days. It is distributed, as usual, between talks, tutorials, and sprints. One of our partners, Flávio Juvenal, will be giving a talk about Celery! It will air on October 17th, the event's second day, at 10:40 AM EDT. If that schedule doesn't work for you, no worries; a recording will be available on their YouTube channel.
Our talk at the event
Mixing reliability with Celery for delicious async tasks
Speaker: Flavio Juvenal
About the talk:
Celery is essential for asynchronous processing in Django backends. In multiple Django projects, we used far beyond the use case of sending emails without blocking HTTP responses. Celery helped us aggregate data, fill caches, run ETL workflows, parallelize heavy workloads, sync with external services, set up periodic background jobs, and more.
But as with any distributed system, running Celery reliably in production is challenging. Due to the many issues we've seen on Celery, we often considered replacing it with other task queues. But we have yet to find another library with the features Celery offers. So, we had to learn to work around its shortcomings and pitfalls. After years of running it in multiple Django projects, we faced and solved several reliability problems. We remediated concurrency hazards. We dealt with lost tasks on multiple edges of the architecture. We read tons of docs, articles, and issues to tweak settings properly. We fixed weird serialization bugs after version upgrades. We found what kind of monitoring really is needed.
In this talk, you will learn how to configure, use, and monitor Celery successfully in production. Celery performs well in simple contexts, and because of that, it might induce a false sense of safety that can be misleading as usage picks up and flows become more complex. Understanding how it can fail as projects grow will help developers prepare in advance.
