wiki:Meetings/2009-Feb-02/irclogs
Last modified 3 years ago Last modified on 04/23/11 20:53:08
16:00 < lmb> fabbione: I watched DaVinci code yesterday, the italian bishop reminded me of you
16:00 < fabbione> lmb: ahahha
16:00 < fabbione> ok guys
16:00 < fabbione> welcome everybody
16:00 < fabbione> this is our second IRC meeting
16:00 < fabbione> agenda is here:
16:00 < fabbione> http://sources.redhat.com/cluster/wiki/Meetings/2009-Feb-02
16:00 < fabbione> who is online?
16:00 < jlbec> bueller
16:01 < lon> I am
16:01 < __McA__> Hi all
16:01  * kleind says hi
16:01 < chrissie> hiya
16:01 < lmb> Sir, yes, sir.
16:01 < dct> hi
16:01 < sdake> hi guys
16:01 < bob_home> I'm here. Although I'm not on the agenda.  :7)
16:01 < fabbione> awesome...
16:01 < AndyP> afternoon
16:01 < fabbione> sorry my adsl is a bit bumpy today
16:02 < fabbione> so i _might_ lag from time to time
16:02 < pmyers> greetings all
16:02 < fabbione> but i am here
16:02 < fabbione> so let's get started with agenda and try to get quickly to the Voice to the community
16:02 < fabbione> # corosync/openais status
16:02 < fabbione> sdake: go ahead
16:03 < fabbione> sdake: ?
16:03 < sdake> ya
16:03 < fabbione> # corosync/openais status
16:03 < sdake> well as most people know we are working on an ipc rewrite
16:03 < sdake> that is the main activity
16:03 < sdake> its coming along
16:04 < fabbione> ETA for a final version in corosync?
16:04 < sdake> theoretically this week
16:05 < fabbione> ok.. lmb spotted some memory leak
16:05 < sdake> yes it is still in development
16:05 < lmb> Yes, we see pretty bad memory leak here currently, but no more other crashes at this time
16:06 < fabbione> 640k ought to be enough for everybody</Bill Gates>
16:06 -!- kennepede [n=pkennedy@65.122.67.200] has joined #linux-cluster
16:06 < lmb> again it would be slightly easier to review if we had a branch/repo with individual commits rather than reissued patchkits, but that is more of a development methodology comment than a technical one
16:06 -!- binsi1 [n=christia@p4FCF91A5.dip0.t-ipconnect.de] has left #linux-cluster []
16:06 < lmb> I guess we need to look at valgrind, that should help considerably to spot memory leaks and corruption
16:07 -!- sbradley_ [n=sbradley@nc-71-50-142-54.dyn.embarqhsd.net] has joined #linux-cluster
16:07 < sdake> i ran through valgrind once
16:07 < lon> fabbione: 640k is a lot of memory
16:07 < sdake> but apparently I missed something
16:07 < sdake> the client or server is leaking?
16:07 < sdake> is that xen-03 backtrace fixed up for you lmb? 
16:08 < fabbione> (5 minutes warning now.. max another minute and we need to move forward)
16:08 < lmb> sdake: yes, now it simply runs out of memory within 2 minutes, but no more crash ;)
16:08 < lmb> it seems that both client and server are leaking
16:09 < sdake> lmb k 
16:09 < fabbione> ok guys 
16:09 < sdake> lmb we can sort it out after meeting
16:09 < fabbione> sdake: thanks
16:09 < lmb> right
16:09 < fabbione> next in line.. chrissie
16:09 < fabbione> cman status 
16:09 < chrissie> CMAN itself is now retired in 'master' cluster.git tree.
16:09 < chrissie> The quorum module is integrated into corosync as of a couple of weeks ago though sdake says he reserves the right to change bits ... I'm open to suggestions from anyone though.
16:09 < chrissie> There is still a compatibility libcman in cluster.git but I hope people will migrate away from it to more generic community-supported interfaces.
16:09 < chrissie> Most of the non-quorum features of cman are provided for in cpg, cfg or confdb.
16:09 < chrissie> We currently still use cman_tool (linked agaist libcman) to start 'cman', It might be replaced with a shell script using more generic binaries.
16:09 < chrissie> clvmd has grown a corosync/DLM interface which will be used in RHEL6 and beyond (even though cman is still in RHEL6). I've added the ability to choose cluster manager at startup.
16:10 < fabbione> wow... awesome... 
16:10 < fabbione> chrissie: we will need an example on how to move away from libcman
16:10 < fabbione> chrissie: once you are ready to do that, maybe migrate notifyd that's fairly stupid
16:11 < chrissie> yes, the code in libcman itself serves as an illustration in fact
16:11 < chrissie> apart from messageing, all its other features are implemented in terms of corosync interfaces
16:11 < chrissie> but for messaging I recommend cpg
16:12 < fabbione> question: if we remove libcman, and start using standard corosync interfaces, aren't we going to lose onwire compatibility?
16:12 < chrissie> I've written up a documenta bout all this which I'll publish shortly
16:12 < chrissie> fabbione: yes. but this is for beyond RHEL6
16:12 < fabbione> chrissie: right, but i think we will need to retain some kind of rolling upgrade compat from RHEL6 -> RHEL7...
16:12 < chrissie> clvmd in RHEL6 can do both
16:12 < fabbione> anyway there is a lot of time to look into it
16:13 < chrissie> libcman will do a lot of that, but it's almost impossible to do totally cleanly
16:13 < chrissie> we can't guarantee rolling upgrades forever ... can we ?
16:13 < fabbione> well only across major releases
16:13 < fabbione> RHEL5 -> RHEL6
16:13 < fabbione> RHEL6 -> RHEL7
16:13 < fabbione> we don't have to guarantee 5 -> 7
16:13 < dct> please no
16:13 < chrissie> no, that's absurd
16:14 < lmb> You should consider the impact of switching from rgmanager to pacemaker too, for example
16:14 -!- sbradley_ [n=sbradley@nc-71-50-142-54.dyn.embarqhsd.net] has quit []
16:14 < lon> oh, I'm well aware of that
16:14 -!- sbradley [n=sbradley@nc-71-50-142-54.dyn.embarqhsd.net] has quit ["leaving"]
16:14 < lmb> we looked at that and decided that rolling ugprades are a PITA and not cost-effective to support
16:14 < fabbione> dct: i thought you already had that done...
16:14  * lon still has to write the XSL conversion
16:14 < dct> RHEL5-RHEL6 was a bad idea, and I think that will become more and more obvious as it gets closer
16:14 < lmb> lon: oh I mean on-the-wire compatibility, not offline conversion while the cluster is down ;)
16:14 < fabbione> dct: it was a requirement.. :/
16:14 < dct> a bad one
16:14 < dct> misguided
16:15 < chrissie> customers like it ...
16:15 < dct> we should have objected to it more strongly :-)
16:15 < sdake> ya i have to agree with dct for once :)
16:15 < chrissie> taking a whole cluster down is a najor nuisance for them
16:15 < chrissie> its our job to take that nuisance on. not theirs
16:15 < chrissie> but not for every upgrade
16:15 < jlbec> yeah
16:15 < lmb> they like it but don't want to pay for it ;-)
16:15 < fabbione> ok, let's get 5->6 work....
16:15 < jlbec> we have the same experience in ocfs2
16:15 < fabbione> we will see what to do later in future
16:15 < jlbec> customers *hate* reboot-all-nodes changes
16:16 < lon> lmb: rgmanager/pacemaker is the least of our concerns.  ;)  "Freeze" in rgmanager, take down rgmanager, convert config, start pacemaker with everything unmanaged, and flip on the 'managed' bit, I think
16:16 < lon> lmb: the problem is we have to provide both rgmanager and pacemaker to do it
16:16 < fabbione> ok.. point taken.. let's discuss this when the time comes
16:16 < fabbione> in a year or two :)
16:16 < lon> cool
16:16 < chrissie> in actuality I don't think it'll be that bad. once we have everything in RHEL6 using cpg
16:16 < chrissie> which it should be capable of
16:17 < fabbione> chrissie: i don't think everything will be using cpg in rhel6... unless you have a hidden tree with everything converted :)
16:17 < chrissie> higher-level protocols are a differnt issue of course ... as dct will tell you ;)
16:17 < fabbione> indeed :)
16:17 < fabbione> ok let's move on
16:17 < fabbione> time is short
16:17 < fabbione> chrissie: thanks for the update.
16:17 < chrissie> well, in that case I could make libcman use it
16:17 < fabbione> dct: you are up next..
16:18 < dct> dlm: testing ocfs2+fsdlm locking and debugging problems that come up.
16:18 < dct> Most problems related to ocfs2's extensive use of cancelation.
16:18 < dct> control daemons: nothing happening
16:18 < dct> questions?
16:18 < fabbione> dct: are those tests relevant for STABLE3?
16:18 < jlbec> dct: do your control daemons have the lack-of-libcman that chrissie has made possible?
16:19 < dct> jlbec, they will in the newly created git trees eventually
16:19 < lmb> jlbec: we obviously have lack-of-cman controlds on SLE (running cpg+libdlm+pacemaker)
16:19 < dct> I'll get working on converting stuff in the new git trees soon I expect
16:19 < dct> e.g. dlm.git, which beekhof is using
16:20 < jlbec> dct: can you poke me when you do, I want to move ocfs2_controld to the new node interfaces, whatever they are
16:20 < dct> ok
16:20 < fabbione> maybe use dlm.git to merge?
16:20 < fabbione> is it even possible to merge those daemons?
16:20 < jlbec> this is all STABLE3, I assume, which is the base level we want to start production ocfs2 with.
16:20 < jlbec> we're not merging them
16:20 < jlbec> I'm just working with him to understand how to replace libcman access
16:21 < dct> STABLE3 in cluster.git isn't going to be changing much
16:21 < fabbione> jlbec: with the split tree, we can do releases at different speed 
16:21 < dct> the new split-up git trees are for new devel
16:21 < jlbec> so the libcman-excise is post sTABLE3?
16:21 < dct> yes
16:21 < chrissie> but you can still get started using the new corosync features
16:21 < chrissie> they are there anyway 
16:22 < chrissie> cman in STABLE3 plugs into quorum in corosync
16:22 < chrissie> so you can use -lquorum to get cman status
16:22 < dct> good point, you could possibly move ocfs2_controld away from cman and still have it work with our other STABLE3 components
16:22 < jlbec> this is preciesely what I want to learn, so I can plot ocfs2_controld, which has to handle all the generations :-)
16:23 < chrissie> all the corosync features that post-stable3 uses are already there in corosync now
16:23 < chrissie> its just that we don't use votequorum, we stil use cman in STABLE3
16:23 < dct> so you could possibly have one version of ocfs2_controld that worked on STABLE3 and post-STABLE3
16:23 < fabbione> unless the release manager goes crazy and decides to pull in bits from master....
16:23 < chrissie> easily
16:24 < jlbec> dct: that's ugly, but I'm thinking it might have to
16:24 < chrissie> dct: I already do this with clvmd ... that's th eupgrade path from RHEL6->7
16:24 < dct> wouldn't be ugly, all the api's you'd use would be the same in both, they'd be api's from corosync
16:24 < jlbec> dct: oh, I read that as "different versions for each"
16:25 < jlbec> dct: I want one daemon, one codebase, that works with both.
16:25 < dct> one daemon working everywhere :-)
16:25 < jlbec> ok, yay
16:25 < fabbione> ok awesome
16:25 < fabbione> dct: thanks for the report 
16:25 < dct> <fabbione> dct: are those tests relevant for STABLE3? 
16:25 < jlbec> I don't use a lot, just nodeid -> name, getting the cluster name, and kill_stack_node()
16:25  * lon readies the spew-queue
16:25 < dct> didn't follow fabbione's question
16:25 < fabbione> dct: ehe ok.. the other stuff was more important
16:27 < fabbione> dct: basically i was only asking if you were using STABLE3 for those ocfs2 <-> dlm tests and if there was any good outcome for us :)
16:27 < fabbione> anyway...
16:27 < fabbione> let's keep going
16:27 < fabbione> lon: you are up next
16:27 < dct> fabbione, not related to userspace
16:27 < dct> only kernel dlm
16:27 < fabbione> dct: ok thanks.
16:27 < lon> ok
16:27 < lon> * rgmanager - on target
16:27 < lon>   * Known Bugs - to be fixed before GA
16:27 < lon>     * rgmanager: Fencing node C when request from A-B occurs results in
16:27 < lon>       clusvcadm hang
16:27 < lon>     * rgmanager: Infinite loop caused in corosync after config update
16:27 < lon>   * ra2man (documentation generation) for resource agents complete/pushed
16:27 < lon>   * ra2oid (ldap OID generation) alpha/beta quality.  Pushed.
16:27 < lon> * qdiskd - on target
16:27 < lon> * Config schema generation - LATE; due 1/31
16:27 < lon>   * Have old relaxng schema; will use for template 
16:27 < lon>   * Some OID generation is complete (rgmanager, cman); still a ways to go
16:27 < lon>     on other parts.
16:27 < lon>   * Should be able to generate OIDs / RelaxNG stuff from fencing agent
16:27 < lon>     metadata.
16:27 < lon>   * Currently in a private branch.
16:27 < lon> * Cluster Configuration Documentation - LATE; due 1/31
16:27 < lon>   * Some generation from RAs complete (ra2man).  Need to do for fencing
16:28 < lon>     agents.
16:28 < lon>   * Have older full-schema docs; going to update and re-push
16:28 < lon>     to sources.redhat.com when completed.
16:28 < lon> * Resource agent merge
16:28 < lon>   * No work done.
16:28 < lon> * XSL conversion for rgmanager->pacemaker config file format
16:28 < lon>   * No work done.
16:28 < fabbione> ok.. hold on tight.. it did scroll off my terminal
16:28 < lon> sorry
16:29 < chrissie> lol
16:29 < fabbione> 16:27 < lon>     * rgmanager: Infinite loop caused in corosync after config update
16:29 < fabbione> wasn't this one fixed?
16:29 < lon> fabbione: you reported that one
16:29 < lon> just recently
16:29 < lon> there were two
16:29 < lon> the one in rgmanager where rgmanager spinned -- that's fixed
16:29 < fabbione> ok... i admit i can't remember
16:29 < lon> and one where rgmanager doesn't do something and corosync spins
16:29 < fabbione> ah right yes
16:30 < lon> the one where rgmanager doesn't do something isn't fixed yet
16:30 < chrissie> lon: for the OID schema generation .. where is the master list held ?
16:30 < fabbione> i think chrissie was able to reproduce it without rgmanager
16:30 < lon> chrissie: for ldap it's held currently in ... config/plugins/ldap/ldap-base.csv or something like this
16:30 < chrissie> yes, I did. I think it's fixed now
16:30 < sdake> probably ipc bug
16:30 < lon> chrissie: it's attr,name,[generated-oid]
16:30 < lon> er
16:30 < sdake> i blame it all on ipc
16:30 < lon> [attr|obj],name,[generated-oid]
16:31 < fabbione> lon: is all that stuff in your private branch?
16:31 < lon> chrissie: in retrospect we probably want a metaschema to genrate schemas
16:31 < lon> fabbione: yes, currently
16:31 < chrissie> I wonder if we need to keep it out of the ldap tree .. .those OIDs also could be used to SNMP
16:31 < chrissie> at least the same range is used for SNMP
16:31 < lon> chrissie: sounds fine with me; I didn't push anything because it's a complete mess.
16:31 < fabbione> chrissie: would you have time to start looking into that branch?
16:31 < fabbione> i also need to look at it to plug it in
16:31 < lon> and I'm wicked-late getting this stuff done, so we can rebuild it if we need to
16:32 < chrissie> yes, i should
16:32 < lon> it's a lot more work than I expected.
16:32 < chrissie> lon: what's the branch called ?
16:32 < lon> lhh_schema
16:32 < chrissie> ta
16:32 < lon> chrissie: run 'make all'
16:32 < lon> then
16:32 -!- abhi_home [n=Abhi@c-66-41-33-110.hsd1.mn.comcast.net] has joined #linux-cluster
16:32 < fabbione> lon: i'll need to know if we need to run this stuff at build time or at any random time before releases
16:32 < lon> pushd rgmanager/src/resources; make schema; popd
16:32 < lon> then
16:32 < lon> scripts/gen-schema
16:33 < lon> fabbione: I expect that we build the schema and commit it back to git before pulling a release
16:33 < fabbione> lon: ok so it's nothing we need to create at build time.. even better
16:33 < lon> and when we pull a release tarball
16:33 < fabbione> lon: can you send me and chrissie and email on how to do it, just in case...
16:33 < lon> pull the current schema for relaxng (not done at all) and ldap (partially done)
16:33 < lon> sure
16:33 < fabbione> perfect
16:33 < fabbione> lon: thanks for the report..
16:34 < fabbione> let's move on
16:34 < lon> cool 
16:34 < fabbione> fabbione: you are up next..
16:34 < fabbione> :)
16:34 < fabbione> - released cluster 3.0.0.alpha3 from STABLE3 branch.
16:34 < fabbione> - updated corosync/openais/cluster packages in Fedora rawhide.
16:34 < fabbione> - submitted review requests for new fence-agents and resource-agents packages for Fedora rawhide.
16:34 < fabbione> - done first cut support for pkgconfig support for corosync/openais/cluster libraries.
16:34 < fabbione> - merge libgfs2 fixes from master to STABLE3.
16:34 < fabbione> - completed automatic release scripts.
16:34 < fabbione> - fixed a corner case bug in liblogthread.
16:34 < fabbione> - ongoing discussion on libfence improvements with David.
16:34 < fabbione> - fixed libccs to cope with random config_version numbers.
16:34 < fabbione> - started working on corosync/openais/cluster packaging changes to support multiarch.
16:34 < fabbione> any questions?
16:35 < sdake> dumb question i suppose but what value does pkgconfig offer
16:35 < chrissie> lots
16:35 < fabbione> sdake: stuff like CFLAGS and LDFLAGS that an external app should use to build against one of your libs (for example)
16:35 < fabbione> sdake: so i can do: gcc -Wall `pkg-config --cflags liblogsys` myapp.c
16:36 < jlbec> sdake: finding libraries is hard, pkgconfig makes it easy
16:36 < jlbec> esp libraries in non-standard locations
16:36 < sdake> ok thanks
16:36 < fabbione> sdake: another example is pkg-config --libs liblogsys:
16:36 < fabbione> [root@daitarn-fedora devel]# pkg-config --libs liblogsys
16:36 < fabbione> -L/usr/lib64/corosync -llogsys  
16:37 < fabbione> it will tell me where to find the library and what options to link with it
16:37 < fabbione> without having to know anything
16:37 < sdake> ok thanks i understand 
16:37 < fabbione> if you change the location, the pkg-confnig is updated etc.
16:37 < fabbione> and you don't need to propagate all the info around..
16:37 < fabbione> sdake: i have done the work in CVS.. all is cleaned and nice.. next time just be one minute more patience :)
16:38 < fabbione> sdake: i did _NOT_ tag nor release
16:38 < fabbione> sdake: it's waiting your blessing
16:38 < fabbione> sdake: i test both clean install and upgrade paths and they are ok
16:38 < sdake> so which package names did we end up with?
16:38 < fabbione> sdake: it doesn't even break compatibilities for applications that still BuildRequire: corosync-devel as we provide them
16:39 < fabbione> corosync, corosynclib and corosynclib-devel + equivalents for openais
16:39 < sdake> sounds good
16:39 < fabbione> note that corosynclib-devel also Provides: corosync-devel
16:39 -!- sbradley [n=sbradley@nc-71-50-142-54.dyn.embarqhsd.net] has joined #linux-cluster
16:39 < fabbione> so if you do yum install corosync-devel you will simply get the new packages
16:39 < sdake> yes that sounds great
16:39 < fabbione> and i did test install with both 32 and 64bit libs on the same system and it was good
16:39 < sdake> thanks for the work
16:40 < fabbione> so unless you have any other comment, I'd like to push the crack into rawhide
16:40 < fabbione> (tomorrow morning)
16:40 < fabbione> together with the same changes to cluster
16:40 < fabbione> no worries... it was either doing the job or shipping dead horse heads around :P
16:40 < fabbione> ok.. i am done
16:40 < fabbione> let's give voice to "the community"
16:41 < fabbione> i see no entry in the agenda but i heard people had problems to edit the wiki
16:41 < fabbione> so i'll just do a quick round
16:41 < fabbione> lmb, bob_home, jlbec, anybody else?
16:41 < fabbione> anybody would like to report something they did on their stack?
16:41 < fabbione> AndyP: ?
16:41 < bob_home> I'm just plugging away at gfs and gfs2 bugzillas
16:42 < bob_home> There's a gfs2 problem that perhaps people should know about
16:42 < jlbec> dct found us a nice bug, so we fixed it
16:42 -!- edwardam [n=edwardam@99-7-171-19.lightspeed.sntcca.sbcglobal.net] has joined #linux-cluster
16:42 < AndyP> i was wondering about the roadmap for that libgfs2 cleanup but swhiteho doesn't seem to be here
16:42 < fabbione> bob_home: how is master coming along? more fixes we can pull into stable3?
16:42 < bob_home> If you have a gfs2 file system and its block size is something other than the default 4K, then a gfs2_grow operation miscalculates the new size
16:42 < fabbione> AndyP: let me try to ping him..
16:43 < AndyP> getting rid of those "die" calls sounds like a big job, i thought i might contribute some patches if i can
16:43 < fabbione> AndyP: i think is joining right now...
16:43 < bob_home> What this means is that if you extend your space by, let's say 1TB, then do gfs2_grow, it will only see 256GB of new free space if the block size is 1K
16:43 < bob_home> This is now fixed in all the branches of the git tree
16:43 < fabbione> bob_home: ah nice.. EAT MY DISK!
16:43 -!- swhiteho_ohnl [n=swhiteho@adsl-02-67.abel.net.uk] has joined #linux-cluster
16:43 < fabbione> speaking of the evil
16:43 < fabbione> 16:42 < AndyP> i was wondering about the roadmap for that libgfs2 cleanup but swhiteho doesn't seem to be here
16:44 < swhiteho_ohnl> ok, here I am :-)
16:44 < bob_home> We're doing a z-stream fix for 5.3.z that should be available shortly
16:44 < fabbione> 16:42 < fabbione> bob_home: how is master coming along? more fixes we can pull into stable3?
16:44 < fabbione> swhiteho_ohnl: ^^
16:44 < fabbione> 16:43 < AndyP> getting rid of those "die" calls sounds like a big job, i thought i might contribute some patches if i can
16:44 < fabbione> bob_home: ok cool.. thanks
16:44 < bob_home> The cleanup work on the master tree of gfs2 is coming along nicely
16:44 < swhiteho_ohnl> with a pause as I'm looking into a bug that dct reported recently
16:44 < bob_home> There still may be some cleanup, but I think we're winding down on that
16:44 < swhiteho_ohnl> there is still a lot to do
16:44  * lon loves closed->notabug
16:44 < bob_home> swhiteho_ohnl may have more planned
16:45 < swhiteho_ohnl> just not sure how much more will be done in the next short period since other things are more important now
16:45 < fabbione> swhiteho_ohnl: i need to know when I can finish to sync into STABLE3 as we are half way through
16:45 < swhiteho_ohnl> but anybody who wants to send patches to help clean it up, is very welcome to do so
16:45 -!- StuartMI [n=stukirk@71.238.151.24] has quit ["Lake Titicaca"]
16:45 < AndyP> ok that answers my question
16:46 < swhiteho_ohnl> killing die is a good plan
16:46 < fabbione> bob_home: i need to finish a little work for dct and now i have the bandwidth to do it...
16:46 -!- StuartMI [n=stukirk@nat/cisco/x-a70bdac9adc4f170] has joined #linux-cluster
16:46 < bob_home> I was kinda hoping abhi_home had some time to test the new master branch of gfs2 with F10
16:46 < fabbione> bob_home: next in line is going to be glock smith 
16:46 < swhiteho_ohnl> ideally there will be no calls to exit(), etc from the library
16:46 < swhiteho_ohnl> error codes should be returned to the calling application
16:47 < bob_home> I'd like to see the recent changes to master tested before we push them to the STABLE3 branch...
16:47 < fabbione> ok
16:47 < abhi_home> bob_home, I've got the smoke cluster setup with upstream code. Anything specific you'd like me to test on the master branch?
16:47 < bob_home> fabbione: good news on glock_smith; whenever you get time
16:47 < fabbione> ok <random> guys... 
16:48 < fabbione> thanks for reports
16:48 < bob_home> abhi_home: Just basic gfs2 functionality with the user space tools: mkfs.gfs2, gfs2_quota, gfs2_tool, gfs2_grow, etc.
16:48 -!- kanderso [n=kanderso@76.164.8.130] has joined #linux-cluster
16:48 < bob_home> heh 
16:48 < fabbione> is there anybody lese that would like to mention any activity?
16:48 < abhi_home> I see, ok. 
16:48 < lon> I'd like to apologize for my epic fail at getting the schema generation / documentation done in a timely manner for stable3.
16:48 < lmb> fabbione: Well, we've obviously been extremly busy with pacemaker/ocfs2 on openais in the last weeks, and it seems to be going pretty acceptably now
16:49 < fabbione> lon: ok :)
16:49 < fabbione> lmb: i think it's great to see the amount of work you guys are working so hard to push stuff to sdake
16:50 < fabbione> s/stuff to//g ;)
16:50 < sdake> ya i love fixing bugs
16:51 < fabbione> we have 10 minutes left
16:51 < lmb> fabbione: yeah, right now most of our issues are openais-centric; the rest is largely unchanged from previous releases and as such somewhat more stable
16:51 < lon> sdake: I almost just fell off my chair.
16:51 < sdake> i guess the sarcasm didn't come through irc lon.. :)
16:51 < fabbione> lmb: i am pleased to know it's going better.. memory leaks aside :)
16:52 < fabbione> ok.. last call for topics
16:52 < fabbione> anybody would like to add anything?
16:52 -!- refried [n=nstraz@dsl-216-227-89-100.fairpoint.net] has quit [Connection timed out]
16:52 < fabbione> 5
16:52 < fabbione> 4
16:52 -!- refried [n=nstraz@dsl-216-227-89-100.fairpoint.net] has joined #linux-cluster
16:52 < fabbione> 3
16:52 < fabbione> 2
16:52 < fabbione> 1
16:52 < fabbione> OK.. 
16:52 < fabbione> all done
16:52 < fabbione> thank you very much guys
16:52 < fabbione> it was nice to see you around
16:52 < sdake> lmb
16:53 < sdake> memory leak
16:53 < fabbione> thanks also for the early birds from SF :)
16:53 < sdake> have time to work on it now?
16:53 < fabbione> oh btw...
16:53 < sdake> ya don't forget arizona.
16:53 < fabbione> Lustre ROCKS!!! OCFS2 and GFS2 SUCK!!!!
16:53  * fabbione hids
16:53 < sdake> 8AM
16:53  * fabbione hides
16:53 < lmb> fabbione: bad pasta I see
16:53 < fabbione> ahha
16:53 -!- fbijlsma [n=fbijlsma@h-217.111.50.177.host.de.colt.net] has quit [Read error: 110 (Connection timed out)]
16:53 < AndyP> don't feed the troll (bad pasta)
16:54  * jlbec waits for fabbionefs to take over the world
16:54 < fabbione> jlbec: i started writing bendoverfs
16:54 < lmb> sdake: alas no, I'm sucked into some other issues right now, but I _think_ it should be trivial to reproduce, my feeling is that every single message leaks
16:54 < lon> jlbec: well, he calls it fabfs, because it means Fabio and also Fabulous
16:54 < lon> jlbec: oops, I wasn't supposed to talk about that.
16:54 < fabbione> jlbec: but people realized that it was just s/ext/bendover/ on ext3 :)
16:55 < sdake> are you calling openais_service_disconnect in your apps when disconnecting from openais?
16:55 -!- fbijlsma [n=fbijlsma@h-217.111.50.177.host.de.colt.net] has joined #linux-cluster
16:55 < jlbec> heh
16:56 < sdake> i think naming a filesystem after yourself could lead to jail time
16:56 -!- fbijlsma [n=fbijlsma@h-217.111.50.177.host.de.colt.net] has quit [Client Quit]
16:57 < fabbione> mount -t fabfs -o murderwife=1 /...
16:57 < jlbec> sdake: but naming it after the secret to french cooking means quick inclusion
16:58 -!- sullyvon [n=dsulliva@nat/redhat/x-6222767674c9bec1] has joined #linux-cluster