7:46:08 PM oubiwann: *waves at fzzzy*
7:46:13 PM fzzzy: hi
7:46:35 PM fzzzy: I just posted about isolation to the bug… I think it'll be the hardest part to get right
7:47:12 PM oubiwann: *reloads that page...*
7:50:09 PM oubiwann: fzzzy: I'm 100% on the first 2 paragraphs
7:50:18 PM oubiwann: but that last one has me scratching my chin...
7:50:37 PM fzzzy: process pool?
7:51:11 PM oubiwann: nah, the enforcing of data isolation via type checking (essentially)
7:51:23 PM oubiwann: I'd prefer to document the *right* way to do it
7:51:54 PM oubiwann: but allow people the freedom to experiment wildly (and of course, screw things up wildly too...)
7:51:58 PM fzzzy: so you want to be able to share mutable data structures between actors?
7:52:01 PM fzzzy: no, then it's not actors
7:52:11 PM fzzzy: so don't call it actors, call it something else
7:52:49 PM oubiwann: well, I'd like to document the right way, and say that anything beyond the documented way could be abuse, and thus not confromant to the actor model
7:53:06 PM oubiwann: fzzzy: but that's just my initial reaction
7:53:12 PM fzzzy: oubiwann: I mean just make it so every message has to be adaptable to some interface, and provide safe defaults, and don't teach noobs how to do it because they will do it wrong
7:53:55 PM fzzzy: oh, I see, I just read my post again, you were talking about deferToActor taking a filename
7:54:00 PM oubiwann: yup
7:54:00 PM fzzzy: I still think it's a good idea
7:54:17 PM oubiwann: yeah, I'm not saying it's a *bad* idea… just that I'm hesitating on it
7:54:18 PM fzzzy: I'm not sure what you mean about type checking
7:54:53 PM fzzzy: it has huge benefits
7:54:56 PM oubiwann: I was using the term very loosely: forcing the use of a filename, vs. allowing functions to be passed
7:55:06 PM fzzzy: if you can spawn a function then that severely limits what can be done
7:55:15 PM fzzzy: how do you pass the function to another process? to another machine?
7:56:20 PM fzzzy: if a big new api addition to twisted is going to be made, why not design it to support the multi core use case right out of the box
7:56:37 PM fzzzy: as well as distributed computing
7:56:57 PM oubiwann: man, I was really hiding  my head in the sand over the whole smells-like-rpc thing...
7:57:07 PM oubiwann: but yeah, we need to think about this now
7:57:42 PM fzzzy: especially since that's what actors are for, anyway
7:57:52 PM oubiwann: hehe, yeah...
7:57:54 PM fzzzy: maybe AMP can be used somehow
7:58:04 PM oubiwann: you beat me to it
7:58:12 PM oubiwann: that was the next thing I was going to say
7:58:31 PM oubiwann: (I'm actually looking at some example erlang code, and that's what made me think of it)
7:58:40 PM fzzzy: yeah it looks like amp is probably the thing to use
7:58:55 PM oubiwann: *sighs with relief*
7:59:11 PM oubiwann: yeah, that takes care of my rpc fears
7:59:38 PM fzzzy: if two actors are in-process, then a special fastpath can be written to avoid the serialization overhead
7:59:47 PM oubiwann: I love it
7:59:51 PM oubiwann: that's a great idea
7:59:59 PM itamar: if you're going to use a protocol... why not Erlang's protocol?
8:00:01 PM fzzzy: the problem with rpc is synchronous rpc
8:00:16 PM fzzzy: itamar: yeah, that's a good idea, but there's a problem, erlang's protocol is crap
8:00:22 PM oubiwann: hehe
8:00:23 PM itamar: or akka
8:00:34 PM fzzzy: I'd defer to dreid or bob's opinion on that though, they know more about it
8:00:35 PM oubiwann: therve's thing
8:00:58 PM fzzzy: supporting erlang protocol as an option is an excellent idea
8:01:09 PM oubiwann: *nods*
8:01:43 PM oubiwann: I need to look at twotp again...
8:02:16 PM fzzzy: so interop with other languages could be a good thing too
8:02:30 PM oubiwann: yeah, that's a huge selling point
8:02:54 PM fzzzy: like a small library for node that parses the wire proto and just does callbacks, no blocking selective receive
8:07:29 PM oubiwann: fzzzy: therve's twotp.client.NodeClientProtocol looks promising
8:07:50 PM fzzzy: cool
8:25:53 PM fzzzy: glyph: what do you think about tackling the multicore angle with this api? I thought of it because I saw on the hacker news comments to oubiwann's post that people were describing a pitfall to gevent being the single threaded nature of it
8:50:39 PM fzzzy: glyph: +1 multihost
8:51:07 PM fzzzy: it's really about thinking of the political angle that's going to gain the most mindshare
8:51:09 PM fzzzy: and that's a good one
8:51:46 PM fzzzy: and yeah, the stm stuff will be amazing
8:52:32 PM glyph: fzzzy: multihost does mean a fair amount of tooling in addition to the basic actor model stuff though
8:52:46 PM glyph: not a huge amount but enough that we shouldn't let that part of the toolchain fall through the cracks
8:52:49 PM fzzzy: glyph: not really, unless you mean implementing OTP
8:53:04 PM fzzzy: just making sure messages are serializable in the design is enough for now
8:53:16 PM fzzzy: actual supervisor trees can be implemented later
8:53:28 PM glyph: "instant out of the box super-scalable applications" is much better than "instant kinda out of the box super-scalable applications, if you write a bunch of code using spawnProcess that only a twisted wizard could figure out how to write"
8:53:44 PM glyph: fzzzy: oh, absolutely
8:53:49 PM glyph: fzzzy: gotta start somewhere
8:53:51 PM fzzzy: yeah, spawnProcess has been there for years, why not make it easy to use
8:54:03 PM glyph: fzzzy: I'm just saying that the sooner we get to a fully functional high-level system with this, the better
8:54:07 PM fzzzy: indeed
8:54:48 PM glyph: fzzzy: it's also a pretty strong sales pitch on top of what other microthreading libraries offer; if we have the tools built-in to scale to multiple workers immediately, then it's not just "microthreads", it's "web scale", which is a bigger-mindshare pitch
8:55:19 PM fzzzy: right, that's what started me thinking about it
8:55:32 PM fzzzy: why just catch up to the greenlet libraries, go past
8:55:41 PM glyph: and once we do have actors with serializable message-passing and a nice way to enforce and debug any non-serializable components which happen to slip in there, the extra code necessary to make it do this will be subtle, but not all that big
8:56:09 PM fzzzy: by the extra code you mean all the cross-machine coordination?
8:56:42 PM fzzzy: that's what I refer to as OTP, supervisor trees, generic protocols, etc
8:56:57 PM fzzzy: all the stuff that's above just the mailbox and cast in erlang
8:57:27 PM fzzzy: afk for a bit again
9:00:46 PM glyph: fzzzy: oh.  "OTP" to me invokes the specific wire protocol, not the general messaging theme
9:01:26 PM fzzzy: I'm back. sure, but there needs to be another name for the generic idea of coordination protocols on top of lower level protocols, then
9:01:29 PM glyph: fzzzy: the extra code is really just the machinery for bootstrapping the coordination - once you have point-to-point serialization of actor communication, you just need a way to relocate your logic to the appropriate place to make use of parallelism.
9:01:40 PM fzzzy: yes
9:01:59 PM glyph: so you need a way to say "spawn some subprocesses" and then also "here are some credentials and network locations for other processing nodes you can talk to"
9:02:05 PM glyph: the kind of crap that heroku will want to give you in an environment variable 
9:02:11 PM fzzzy: I usually just punt on it and make everything explicit, like in python-actors you can start up an http server and accept messages from it, but the sender has to know the address
9:02:28 PM glyph: None of this stuff is terribly rocket sciency, but the more you automate it the clearer the documentation can be
9:02:46 PM fzzzy: but I was planning on doing the higher level stuff once some clear lower-level winner appeared to me
9:03:33 PM glyph: fzzzy: OK.  My comments are just to make sure that we solve the whole problem and don't stall halfway through (as we sort of have on AMP, c.f. the lack of authentication and the lack of multiplexing which _habnabit is now fixing).
9:03:50 PM fzzzy: sure
9:04:00 PM glyph: fzzzy: they should not be interpreted to mean that you should put the cart before the horse here, and build an all-singing all-dancing ssh credentials management application before getting some simple serialization working 
9:04:09 PM fzzzy: heh
9:04:15 PM fzzzy: that sounds good too though
9:04:59 PM glyph: fzzzy: the _main_ thing I want to automate (later) is integrating some kind of deployment infrastructure along with node communication
9:05:09 PM glyph: so you can say "here's my function, figure out what it depends on, and move all that crap over there"
9:05:15 PM fzzzy: yes
9:05:28 PM fzzzy: but that is why I was arguing for spawn taking a file name
9:05:32 PM glyph: which you can totally do pretty straightforwardly by asking setuptools and twisted.python.modules which distribution the referenced function is a part of, what that distribution's data_files are, etc
9:05:53 PM fzzzy: as long as you can find all the dependencies and move them… that would be totally sweet
9:06:17 PM glyph: fzzzy: Well, as I said on the ticket, there are good ways to work from a function to a file (or file-like thing, which is why I don't want to actually pass a filename)
9:06:36 PM glyph: and it's entirely reasonable to raise RuntimeError("don't pass a lambda closure to this, you jackass")
9:07:03 PM fzzzy: personally if I build systems on this I would want each actor to be in it's own file, with no external dependencies other than what is available uniformly on the grid
9:07:16 PM fzzzy: but that's just a style thing
9:07:23 PM glyph: THE GRID
9:07:33 PM glyph: *puts on the Tron Legacy sound track and tries to pretend the plot of that movie made sense*
9:07:50 PM fzzzy: since actors all end up having a main loop, a file per actor makes a lot of sense
9:07:59 PM fzzzy: although a function per actor could make as much sense
9:08:13 PM fzzzy: but it doesn't as clearly get across that the module namespace cannot be used to share state
9:08:32 PM fzzzy: or worse, the module namespace could be used to share state, but only if they happen to be on the same machine in the same process
9:08:59 PM fzzzy: but of course every little object everywhere in python can be used to share state and destroy the abstraction
9:09:00 PM glyph: fzzzy: I think that's a documentation issue.  You could as easily pass a filename and still believe that the shared state is 'in the py file' and be surprised that it doesn't propagate
9:09:18 PM glyph: fzzzy: the function is the relevant thing though, the file name is just an implementation detail.
9:09:35 PM fzzzy: well I see the top level of the file as being the main
9:09:42 PM fzzzy: it's just a style thing
9:09:49 PM glyph: fzzzy: oh, I see what you're saying...
9:09:55 PM glyph: I'd rather the actor be function-scope than script-scope
9:09:58 PM fzzzy: it's just what I've come to prefer, that's all, not important
9:10:11 PM fzzzy: glyph: ok, I'll ask you what you think about that again in 5 years 
9:10:28 PM fzzzy: I think it'll be fine
9:10:56 PM itamar: I'd much prefer functions
9:11:04 PM itamar: otherwise even minimal examples end up with three files
9:11:11 PM fzzzy: yes.
9:11:15 PM fzzzy: that's a good point.
9:12:08 PM fzzzy: and in fact, that's exactly how it looks when I do it that way:
9:12:08 PM fzzzy: https://github.com/fzzzy/lambda.js/tree/master/src/examples
9:12:35 PM glyph: fzzzy: also, functions make testing easier.
9:12:52 PM fzzzy: yep
9:13:03 PM glyph: fzzzy: after all, you want to make Python look like _function_al programming here, not... scriptural programming
9:14:00 PM fzzzy: erlang being functional is a red herring
9:14:17 PM fzzzy: it has nothing to do with anything important
9:14:22 PM fzzzy: except immutable messages I guess
9:14:35 PM fzzzy: but still, +1 on functions
9:18:15 PM glyph: cool
9:23:24 PM oubiwann: wow, I step out of the room for a few minutes, and look what happens
9:23:32 PM oubiwann: that was awesome, btw 
9:24:23 PM glyph: fzzzy: we should maybe talk about amp.AnyOf at some point soon too
9:24:50 PM glyph: fzzzy: dynamic typing might be useful when serializing these messages, and AMP will be more compact and less error-prone in an interop context than JSON
9:25:03 PM glyph: (teratorn can write some workers in C, for example)
9:25:20 PM fzzzy: glyph: yes, use amp instead of json, please