[Twisted-Python] Re: Memory size of Deferreds
MichaĆ Pasternak
michal.dtz at gmail.com
Tue May 20 12:54:34 MDT 2008
On 2008-05-19, 22:45:26
Martin Geisler <mg at daimi.au.dk> wrote:
> "Michal Pasternak" <michal.dtz at gmail.com> writes:
>
[...]
> I would love to get some advice on the design of my library -- it is
> the first thing I have made with Twisted. The goal of VIFF is to
> enable shared computations -- think addition and multiplication.
[...]
> def add(x, y):
> sum = gatherResults([x, y])
> sum.addCallback(lambda results: results[0] + results[1])
> return sum
>
> Multiplication is similar, but requires network communication -- that
> is why it returns a Deferred.
I don't think that there's anything wrong with this example in terms of
being written using Twisted Python.
> The important point is
> that as much as possible is done in parallel, that all operations run
> as soon as their operands are available. Twisted made this extremely
> easy thanks to the DeferredList and Deferred abstraction.
This is the point where I think, that you answered your question. If
you do stuff in parallel, you use more Deferreds and more memory.
I will repeat myself now: I would rather try to find other places to
optimize VIFF, than reducing memory footprint of Deferred. I don't have
a CS degree, I'm not a mathematician, but from what you wrote it seems
that maybe it would be worth to try implementing something stack-based
(like RPN), than the tree approach you seem to be doing now - of course
if having a stack is something you can have using secure multiparty
calculations.
--
m
More information about the Twisted-Python
mailing list