Make sure you are closing the DB connections after accessing Django ORM in your threads
I was writing a service which import various feeds and then post to another RESTful API, simple enough as below:
1 2 3 4 5 6 7 8 |
from django.db import connections class Receiver(PolymorphicModel): ... class Database(Receiver): name = models.CharField(max_length=255) def get_data_source(self): return connections[name] |
Then the feed model:
1 2 3 4 5 |
class Feed(models.Model): receiver = models.ForeignKey('Receiver') def get_data_source(self): return self.receiver.get_real_instance().get_data_source() |
Then run this feed in trivial code:
1 2 3 4 5 6 7 8 9 10 11 |
import threading function transform_listing(feed): cursor = feed.get_data_source().cursor() .... feed = Feed.objects.create(receiver=Database.objects.create(name='legacy')) for i in range(100): th = threading.Thread(transform_listing, [feed]) th.start() |
And you will quickly notice the threads complaining
1 |
Too many connections |
What it’s happening is Django will create a new connection per thread… Read More »