Open in new window / Try shogun cloud
--- Log opened Wed Aug 21 00:00:24 2013
shogun-buildbotbuild #954 of rpm1 - libshogun is complete: Success [build successful]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/rpm1%20-%20libshogun/builds/95400:01
-!- foulwall [~user@2001:da8:215:c252:7846:786c:36da:abfc] has quit [Ping timeout: 245 seconds]00:10
shogun-buildbotbuild #1278 of cyg1 - libshogun is complete: Failure [failed test]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/cyg1%20-%20libshogun/builds/1278  blamelist: Viktor Gal <viktor.gal@maeth.com>00:32
shogun-buildbotbuild #1572 of deb3 - modular_interfaces is complete: Failure [failed compile csharp_modular]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb3%20-%20modular_interfaces/builds/1572  blamelist: Viktor Gal <viktor.gal@maeth.com>01:16
-!- hushell [~hushell@8-92.ptpg.oregonstate.edu] has quit [Ping timeout: 260 seconds]01:58
-!- hushell [~hushell@8-92.ptpg.oregonstate.edu] has joined #shogun02:03
-!- hushell [~hushell@8-92.ptpg.oregonstate.edu] has quit [Ping timeout: 276 seconds]02:08
-!- travis-ci [~travis-ci@ec2-54-227-44-173.compute-1.amazonaws.com] has joined #shogun02:14
travis-ci[travis-ci] it's Viktor Gal's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: http://travis-ci.org/shogun-toolbox/shogun/builds/1042456602:14
-!- travis-ci [~travis-ci@ec2-54-227-44-173.compute-1.amazonaws.com] has left #shogun []02:14
-!- hushell [~hushell@8-92.ptpg.oregonstate.edu] has joined #shogun02:16
-!- shogun-notifier- [~irker@7nn.de] has quit [Quit: transmission timeout]02:18
-!- travis-ci [~travis-ci@ec2-184-73-69-105.compute-1.amazonaws.com] has joined #shogun02:32
travis-ci[travis-ci] it's Viktor Gal's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: http://travis-ci.org/shogun-toolbox/shogun/builds/1042461102:32
-!- travis-ci [~travis-ci@ec2-184-73-69-105.compute-1.amazonaws.com] has left #shogun []02:32
-!- hushell [~hushell@8-92.ptpg.oregonstate.edu] has quit [Ping timeout: 248 seconds]02:32
-!- pickle27 [~kevin@d67-193-243-174.home3.cgocable.net] has joined #shogun02:33
-!- zxtx [~zv@rrcs-74-62-200-195.west.biz.rr.com] has quit [Ping timeout: 264 seconds]04:12
-!- pickle27 [~kevin@d67-193-243-174.home3.cgocable.net] has left #shogun []04:33
-!- hushell [~hushell@c-98-232-178-161.hsd1.or.comcast.net] has joined #shogun06:58
-!- gsomix [~gsomix@95.67.149.216] has quit [Ping timeout: 264 seconds]07:53
-!- gsomix [~gsomix@80.234.99.172] has joined #shogun08:05
-!- lisitsyn1 [~lisitsyn@fb2-lo1.global63.net] has left #shogun []08:21
-!- zxtx [~zv@cpe-75-83-151-252.socal.res.rr.com] has joined #shogun08:26
@wikingsonney2k: ping09:02
-!- gsomix [~gsomix@80.234.99.172] has quit [Ping timeout: 264 seconds]09:17
-!- gsomix [~gsomix@80.234.99.172] has joined #shogun09:19
-!- gsomix_ [~gsomix@109.169.154.231] has joined #shogun09:56
-!- gsomix [~gsomix@80.234.99.172] has quit [Ping timeout: 260 seconds]09:59
-!- gsomix_ is now known as gsomix11:08
-!- yo [4e352d55@gateway/web/freenode/ip.78.53.45.85] has joined #shogun11:21
-!- yo is now known as Guest6074711:21
gsomixgood morning?11:22
-!- Guest60747 [4e352d55@gateway/web/freenode/ip.78.53.45.85] has quit [Ping timeout: 250 seconds]11:26
-!- van51 [~van51@ppp-94-66-22-124.home.otenet.gr] has joined #shogun11:36
@wikinghey11:38
gsomixwiking, hey ho11:43
gsomixsonney2k, around?11:44
gsomixwiking, are there nice and safe methods for use SGVector with offset? I want use it for Parsing.11:59
@wikinggsomix: whatyaneed exactly11:59
@wikingi mean SGVector.vector is just a simple T* array where T is a builtin type... so u can offset that for sure quite easily the only thing you'll always have to check yourself is that you are not trying to offset more than the length of the vector...12:00
gsomixsomething like SGVector<char> substring=string.get_vector_with_offset(start); where string is SGVector.12:01
gsomixwiking, yeah, but I want refcount for this.12:01
@wikinggsomix: aaaaah lool12:01
@wikingyou want to raise the refcount of the sgvector if you get a subset of it12:02
@wikingmmm12:02
gsomixyep, of course I can to make unsafe offset with pointers...12:02
@wikingmmm12:03
@wikingthere's this command12:03
@wikingSGVector<T> get()12:03
@wiking{12:03
@wikingreturn *this;12:03
@wiking}12:03
@wikingand according to the doxygen:  * @return the refcount increased vector12:04
@wikingwhich i dont see yet how it is done if you just return *this :P12:04
@wikingbut then again you should check it with .ref_count()12:05
@wikingbut if that works12:05
@wikingthen basically you can write the same type of function12:05
@wikingSGVector<T> get_vector_with_offset(start) {  return  }12:06
@wikingjust do the offsetting yourself12:06
@wikingawith the pointer of *this12:06
@wiking*with12:06
@wikinggsomix: but then again this is really just an assumption... you should check it simply with the ref_count() :P12:07
gsomixwiking, hm, ok. but there is needed to store offset in SGVector. additional data... I need think about it.12:09
-!- van51 [~van51@ppp-94-66-22-124.home.otenet.gr] has quit [Quit: Leaving.]12:11
-!- HeikoS [~heiko@nat-187-82.internal.eduroam.ucl.ac.uk] has joined #shogun12:36
-!- mode/#shogun [+o HeikoS] by ChanServ12:36
@wikingHeikoS: sorry man but that's not shogun's fault13:02
lisitsynwiking: yes it raises the RC13:17
lisitsynwiking: it *has* to call the constructor13:17
lisitsynRVO whatever but you need to create a copy13:18
@wikingah ok so it's calling the copy ctr13:21
@wikinglisitsyn: still here?13:29
-!- xBeast [~xBeast@den-69-171-160-70.evdo.leapwireless.net] has joined #shogun13:47
-!- gsomix [~gsomix@109.169.154.231] has quit [Ping timeout: 256 seconds]13:56
-!- xBeast [~xBeast@den-69-171-160-70.evdo.leapwireless.net] has left #shogun ["Leaving"]13:57
lisitsynwiking: yes13:58
@wikinglisitsyn: so i was thinking about adding a new serilization support, but then i realised that actually what would b good to have is to be able to (de)serialize not only to hdfs but as well deserialize a mahout Dense/SparseVector into SGVector/sparseVector14:03
@wikingthe first part is quite easy... i mean we just have to add like a SerializeAvroFile/Reader.h with the right implementation14:04
@wikinginsights?14:06
-!- van51 [~van51@ppp-94-66-22-124.home.otenet.gr] has joined #shogun14:06
lisitsynwiking: sorry don't get: what is the first part and the second part?14:13
-!- gsomix [~gsomix@109.169.154.231] has joined #shogun14:14
@wiking1) support serialization to hdfs14:14
@wiking2) support serialization of mahout's vector into shogun's vector14:14
@HeikoSwiking: well, I still cannot compile it, mpi *is* installed14:39
lisitsynwiking: I am not sure I have any insight on that actually14:57
lisitsynis there any specific problem?14:57
-!- van51 [~van51@ppp-94-66-22-124.home.otenet.gr] has quit [Quit: Leaving.]15:25
lisitsynHeikoS: are you going to nips this year?15:31
@HeikoSlisitsyn: hi! not clear yet, waiting for rewie answers15:32
lisitsynHeikoS: http://mloss.org/workshop/nips13/15:32
@HeikoSbut optimistic15:32
lisitsynI think it is for you then :)15:32
@HeikoSah cool! :)15:32
@HeikoSthanks for the info15:32
lisitsynHeikoS: lets prepare something15:32
@HeikoSlisitsyn: yeah why not :)15:32
@HeikoSshogun parallel stuff :D15:33
lisitsynI would like to go as well but that's too much travelling already :D15:33
@HeikoSwhy?15:35
@HeikoStravelling is nice :)15:36
lisitsynHeikoS: $$ :)15:36
@HeikoShehe15:37
-!- foulwall [~user@2001:da8:215:c252:893c:1950:94a2:dc48] has joined #shogun16:19
-!- pickle27 [~kevin@d67-193-243-174.home3.cgocable.net] has joined #shogun16:32
-!- shogun-notifier- [~irker@7nn.de] has joined #shogun16:47
shogun-notifier-shogun: Thoralf Klein :develop * d0bbb8f / src/shogun/features/streaming/ (2 files): https://github.com/shogun-toolbox/shogun/commit/d0bbb8ff4f40c453e79f6a8f71b2a32de608bf9116:47
shogun-notifier-shogun: Refactoring of StreamingSparseFeatures: Removed current_vector and current_length16:47
shogun-notifier-shogun: * Removed code duplication; simpler initialization.16:47
shogun-notifier-shogun: * Using non-refcounted sparse vector to avoid double-frees when using get_vector().16:47
shogun-notifier-shogun: Heiko Strathmann :develop * eb72b69 / src/shogun/features/streaming/ (2 files): https://github.com/shogun-toolbox/shogun/commit/eb72b6932aec410817eb76af44b781a9fc67de7e16:47
shogun-notifier-shogun: Merge pull request #1449 from tklein23/refactoring_streaming_sparse_features16:47
shogun-notifier-shogun:16:47
shogun-notifier-shogun: Refactoring of StreamingSparseFeatures: Removed members current_vector/_length16:47
shogun-buildbotbuild #1457 of deb2 - static_interfaces is complete: Failure [failed configure]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb2%20-%20static_interfaces/builds/1457  blamelist: Thoralf Klein <thoralf.klein@zib.de>16:54
shogun-buildbotbuild #1488 of bsd1 - libshogun is complete: Failure [failed test]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/bsd1%20-%20libshogun/builds/1488  blamelist: Thoralf Klein <thoralf.klein@zib.de>16:57
-!- van51 [~van51@ppp-94-66-22-124.home.otenet.gr] has joined #shogun16:57
-!- pickle27 [~kevin@d67-193-243-174.home3.cgocable.net] has left #shogun []16:59
-!- pickle27 [~kevin@d67-193-243-174.home3.cgocable.net] has joined #shogun17:10
@wikingre17:25
-!- HeikoS [~heiko@nat-187-82.internal.eduroam.ucl.ac.uk] has quit [Ping timeout: 246 seconds]17:31
shogun-buildbotbuild #1279 of cyg1 - libshogun is complete: Failure [failed test]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/cyg1%20-%20libshogun/builds/1279  blamelist: Thoralf Klein <thoralf.klein@zib.de>17:34
-!- HeikoS [~heiko@nat-187-82.internal.eduroam.ucl.ac.uk] has joined #shogun17:36
-!- mode/#shogun [+o HeikoS] by ChanServ17:37
shogun-buildbotbuild #1458 of deb2 - static_interfaces is complete: Failure [failed configure]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb2%20-%20static_interfaces/builds/1458  blamelist: Heiko Strathmann <heiko.strathmann@gmail.com>17:38
shogun-buildbotbuild #1489 of bsd1 - libshogun is complete: Failure [failed test]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/bsd1%20-%20libshogun/builds/1489  blamelist: Heiko Strathmann <heiko.strathmann@gmail.com>17:42
@wikinggsomix: here?17:45
gsomixwiking, yep17:45
@wikinggsomix: so just a quick question as you are really into this17:45
@wikingsay i want a new class that can generate SGVectors or SGMatrixes from files17:46
@wikingcurrently this is implemented with classes that are subclasses of File.h17:46
@wikingright?17:46
gsomixhm, yep. but it works with plain arrays, not SGVector's or SGMatrices.17:47
@wikingyeah i saw that17:48
@wikingbut then how one would get like a CFeatures* from a CSVFile at the moment?17:48
gsomixwiking, CFeatures' object should have method load(CFile*)17:49
@wikingand it does \o/17:49
@wikingok so basically what i would need then is open the file with CSVFile and then create a CFeatures object and call .load(CSVFile) on it17:50
@wikingand that'll basically load the csvfile content into a CFeatures.17:51
@wikingok and that is the same for SGMatrix17:51
@wikingi can see17:51
gsomixwiking, yeo17:52
gsomixhttps://github.com/shogun-toolbox/shogun/blob/develop/examples/undocumented/python_modular/kernel_anova_modular.py17:52
gsomixfor example17:52
@wikingok this is great17:53
@wikingthnx17:53
-!- lisitsyn [~lisitsin@mxs.kg.ru] has quit [Quit: Leaving.]17:54
-!- van511 [~van51@ppp-94-66-88-196.home.otenet.gr] has joined #shogun18:14
-!- van51 [~van51@ppp-94-66-22-124.home.otenet.gr] has quit [Ping timeout: 248 seconds]18:17
-!- travis-ci [~travis-ci@ec2-184-73-69-105.compute-1.amazonaws.com] has joined #shogun18:19
travis-ci[travis-ci] it's Heiko Strathmann's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: http://travis-ci.org/shogun-toolbox/shogun/builds/1045529718:19
-!- travis-ci [~travis-ci@ec2-184-73-69-105.compute-1.amazonaws.com] has left #shogun []18:19
shogun-buildbotbuild #1280 of cyg1 - libshogun is complete: Failure [failed test]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/cyg1%20-%20libshogun/builds/1280  blamelist: Heiko Strathmann <heiko.strathmann@gmail.com>18:19
gsomixok, all another works are done today.18:33
gsomixtime for shogun18:34
-!- lambday [67157c37@gateway/web/freenode/ip.103.21.124.55] has joined #shogun18:57
lambdayHeikoS: hi18:57
lambdaywiking: thanks man! :)19:01
-!- travis-ci [~travis-ci@ec2-184-73-69-105.compute-1.amazonaws.com] has joined #shogun19:03
travis-ci[travis-ci] it's Heiko Strathmann's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun: http://travis-ci.org/shogun-toolbox/shogun/builds/1045529719:03
-!- travis-ci [~travis-ci@ec2-184-73-69-105.compute-1.amazonaws.com] has left #shogun []19:03
@HeikoSlambday: hi!19:04
@HeikoSlambday: I have been starting to prototype the independent computation stuff for PBS in python, might be useful for us later19:06
lambdayHeikoS: PBS?19:06
@HeikoSthats a batch based cluster19:06
@HeikoSwhere one can submit independent jobs that are queued etc19:06
lambday:D19:06
lambdaywow!19:06
@HeikoSseems to work nice with our ideas19:07
@HeikoSbut still building things19:07
@HeikoSlambday: anyway, how are things?19:07
lambdayHeikoS: I was checking out ColPack19:07
@HeikoSlambday: ah cool, what do you think about it?19:07
lambdayHeikoS: and to use it, we need CRS format19:07
lambdaywhich eigen3 provides19:08
lambdayand we already have a method which convers SGSparseMatrix to eigen3 one19:08
@HeikoSlambday: nice!19:08
lambdaybut extra memory usage is a drawback19:08
lambdayanyway19:08
@HeikoSwell19:08
lambdaybut computing matrix power19:08
@HeikoSthats life19:08
lambdayis a bad thing :(19:08
@HeikoSI know19:08
@HeikoSkrylstat do it by hand?19:09
-!- van511 [~van51@ppp-94-66-88-196.home.otenet.gr] has quit [Quit: Leaving.]19:09
lambdayHeikoS: yeah they simply multiply matrices19:09
lambdayHeikoS: I wanted to test something19:09
@HeikoSlambday: what better ideas are there?19:09
lambdaysince its psd, we can use eigenvalues and eigenvectors19:10
@HeikoSfor matrix powers?19:10
lambdayand we already have Lanczos for that19:10
lambdayyes19:10
@HeikoSah19:10
@HeikoSthat would be sweet19:10
@HeikoSwould be good to compare the speed19:10
lambdayHeikoS: yes... otherwise I can add matrix-matrix product for SGSparseMatrix...19:11
lambdayusing mat-vec product that I already added19:11
@HeikoSlambday: yep19:11
lambdayso, I am thinking of adding just the coloring first19:12
@HeikoSlambday: ok19:12
lambdayand then the matrix power - straight forward19:12
@HeikoSthe only thing open is then what is fastest19:12
lambdaythen will add the eigenvalue/eigenvectors thing19:12
@HeikoSlambday: ok this sounds all very promising19:12
lambdayHeikoS: yes19:12
@HeikoShas colpack many dependencies?19:13
lambdayHeikoS: oh one thing... while passing SGVector as a param, is all the data copied or just the refcount increases?19:13
@HeikoScan we cmake it into shogun?, wiking?19:13
lambdayHeikoS: that's already done.... :D I talked to wiking yesterday and asked him to add it19:13
lambdayand he did :D19:13
@HeikoSlambday: SGVector copy by value creates another SGVector instance with the pointer pointing to the same memory19:13
@HeikoSand yes recount is increased19:13
@HeikoSlambday: whoooo19:13
@HeikoSwiking rocks doesnt he? :)19:13
lambdayno I can run ColPack without any dependency19:13
lambdayHeikoS: you bet :D19:14
lambdaythe cmake thing is frickin19:14
lambdayawesome!19:14
lambdayHeikoS: I just checked that wiking added ColPack check... now gotta do some test..19:14
@HeikoSlambday: indeed!19:15
@HeikoSlambday: ok then19:15
lambdayHeikoS: once I add this.. I'll be relieved! :D theoretically all modules will be implemented19:15
@HeikoSI am really curious how long this takes with colpack and your lancos idea19:15
lambdayrest, test and more tests19:15
@HeikoSsince with matlab it took ages19:15
lambdayHeikoS: yeah! but the power is small, then Lanczos can be a overkill19:16
@HeikoSso that I just dumped a fixed matrix colouring to disc and loaded it all over again19:16
@HeikoSlambday: lets see19:16
@HeikoSlambday: the ozone matrix is a good benchmark, use it19:16
lambdayalright.. yes19:16
lambdayHeikoS: can I add to shogun??19:16
@HeikoSits quite sparse so the conversion also should not be a problem19:16
lambdayadd the ozone matrix as data?19:16
@HeikoSlambday: good question19:16
@HeikoSwould be good let me check19:17
lambdayits 80 MB almost19:17
@HeikoSsonney2k, ^ too much?19:18
@HeikoSlambday: I just asked daniel lets see whether its even allowed19:18
@HeikoSI mean the matrix at least is ok19:18
lambdayHeikoS: what is?19:18
@HeikoSbut we rather want the dataset to generate the matrix from for different parameters19:18
@HeikoSthe matrix itself is not protected o whatever19:18
@HeikoSallowed to publish the data19:19
lambdayokay...hmm19:19
@HeikoSI will also ask Daniel to for advice on hard synthetic datasets19:19
@HeikoSlambday: btw the notebooks should also illustrate how to use all the solvers you wrote, people might be interested in that as a seperate module in shogun19:19
@HeikoSlambday: but that later, first lets make things work :)19:20
lambdaycurrently for log-det, we can only use 3 solvers...19:20
lambdaydirect one, cocg, and cg-m19:20
@HeikoSah yes not the individual one19:22
lambdayI am still confused about why the paper calls it cocg-m... :-/ if the data is real, then vec1.transpose()*vec2 is same as vec1.dot(vec2) :(19:22
@HeikoSlambday: dont know, dont worry about this19:22
lambdaythe only thing that changes from cocg and cg is this step!19:22
@HeikoSlambday: the individual with preconditioning should be optional, once all the rest works19:22
@HeikoSsince time is getting more tight these days19:22
lambdayyeah! :(19:23
lambdayand things are getting tougher and tougher on my side :(19:23
@HeikoSlambday: indeed, so lets focus on getting the core to work19:23
@HeikoSthen we can extend19:23
@HeikoSoh and documentation is also very important19:23
@HeikoSwe should go over all class docs etc to make things very clear19:24
@HeikoSso19:24
lambdayalright19:24
lambdayokay... how long would it take for me to add swig interfaces? (stupid question but I have no idea about this)19:24
@HeikoS1.) some tests on solvers19:24
@HeikoS2.) matrix product19:24
@HeikoS3.) colpack19:24
@HeikoS4.) implement probing19:24
@HeikoS5.) final example19:24
@HeikoS6.) tests19:24
@HeikoS7.) docs19:24
@HeikoSlambday: they are done extremley fast19:24
@HeikoSjust a few lines to add19:25
@HeikoSall built automagically19:25
lambdayclassic!19:25
@HeikoSstatic interfaces are more work though19:25
@HeikoSand I would like to have at least some basic ones19:25
@HeikoSfor matlab and R people19:25
lambdaywow this list seems great!... I am copying it to my mail :D19:25
lambdayerr..19:25
lambdayfor the ipython demo, we need python modular interface to work, right?19:26
@HeikoSlambday: yes19:26
@HeikoSlambday: but as said, modular interfaces are only 1h work19:26
lambdayHeikoS: phew! :D19:26
@HeikoSif you want, have  a look into src/interfaces/modular19:26
@HeikoSthere are .i files19:26
@HeikoSGaussianProcess.i GaussianProcess_includes.i19:26
@HeikoSyou have to create similar ones19:27
@HeikoSor even put it into math19:27
@HeikoSfor static ones, no idea, I have in fact never created some, but ask pickle27 I think he created some static ones19:27
lambdaywah19:28
lambdayalright... so, in the final example, would we just compute log-det or run the whole thing?19:28
@HeikoSlambday: but thats also not top priority19:28
@HeikoSlambday: yes19:28
lambdayand plot that graph that I saw in the ozone examplepage?19:28
@HeikoSlambday: so I would like a few things in fact19:28
@HeikoSlambday: first some basic examples how the log-det framework is used in c++ and in python modular19:29
@HeikoSwith a few cases covered ( no plots though, just basic usage)19:29
lambdayalright19:29
@HeikoSand then, have one big ipython notebook where you introduce your whole project, illustrate many of the methods, how they work, how they behave, have some examples, convergence plots, maybe the ozone data19:30
@HeikoSmaybe even a little MCMC on a GMRF19:30
@HeikoSeven the ozone maybe19:30
@HeikoSdepending on how much time we have19:30
@HeikoSshould leave 1-2 weeks at the end just for the notebook and examples19:30
lambdayHeikoS: alright...19:31
@HeikoSlambday: will be very cool! :)19:31
lambdayyeah! :)19:31
lambdayfeels bad that I could have done much more if I could give all my time but these f**king people! :'(19:32
lambdayanyway I will try to finish coloring by this week only19:33
lambdaylast night I checked what I had to do, now just gotta implement19:33
@HeikoSlambday: dont spend too much energy being angry, not worth it. You will finish the project anyhow and did a great job already, so dont feel bad19:33
@HeikoSlambday: cool, sounds good!19:33
lambday:(19:34
@HeikoSlambday: I hope to finish the cluster stuff this week so then next week I will test your stuff :D19:34
lambdayyeah you're right... better use my energy in something constructive19:34
lambdayhehe.... that's superb!19:34
lambdayalright..19:35
@HeikoScool man, keep me update how it is going19:35
lambdayHeikoS: are we gonna have another meeting before endsem?19:35
@HeikoSreally excited about the graph colouring stuff :)19:35
@HeikoSlambday: yes I will organize one soon19:35
lambdayHeikoS: official meeting19:35
lambdayalright :)19:36
lambdayme too19:36
lambdaylol@endsem,  I meant endterm :D19:36
@HeikoShehe :)19:38
@HeikoSstudents ;)19:38
@wiking HeikoS ok so i was digging19:40
@wikingHeikoS: got some time now?19:40
@HeikoSwiking: sorry just about to leave home19:42
@HeikoSwiking: what did you dig out?19:42
@wikingokok19:42
@wikingno worries we talk another time19:42
@HeikoSwiking: ok then, see you tomorrow!19:43
@HeikoSwiking: btw we could have another dev meeting some time soon! :)19:43
-!- HeikoS [~heiko@nat-187-82.internal.eduroam.ucl.ac.uk] has quit [Quit: Leaving.]19:43
-!- hushell [~hushell@c-98-232-178-161.hsd1.or.comcast.net] has quit [Ping timeout: 245 seconds]19:43
-!- shogun-notifier- [~irker@7nn.de] has quit [Quit: transmission timeout]19:47
shogun-buildbotbuild #1575 of deb3 - modular_interfaces is complete: Failure [failed compile csharp_modular]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb3%20-%20modular_interfaces/builds/1575  blamelist: Heiko Strathmann <heiko.strathmann@gmail.com>19:54
@wikinglambday: ping19:57
lambdaywiking: pong19:57
@wikinglambday: does the color graph library detection work for u?19:57
lambdaywiking: checking... :)19:57
lambdaywiking: it did detect19:58
@wikingcool19:58
-!- foulwall [~user@2001:da8:215:c252:893c:1950:94a2:dc48] has quit [Ping timeout: 245 seconds]19:58
lambdayyou rock!19:58
@wikinglambday: do u have time a bit now to talk about the computation engine you've developed?19:59
lambdaywiking: sure20:01
lambdayits still its infancy :(20:02
@wikinglambday: yeah but still cool20:02
@wikingi just want to talk about the backend20:02
lambdaybut this can be extended...20:02
@wikingwhat u had in mind for the real distributed engine20:03
lambdaywiking: please ask/suggest20:03
@wikingso let's say we have now the serialize engine for all the objects20:03
@wikingthat currently only supports serialization via file but that can be extended20:03
@wikingmy only question here is what should we implement as a parallel engine20:04
@wikingi.e. the new subclass of IndependentComputationEngine20:04
lambdaywiking: so far me and Heiko had discussed, it uses a job queue... and dispatches the jobs to different nodes... all these jobs has a compute method and they all know how to compute the individual results...20:05
lambdayand all these jobs submits their results to the job aggregator.. the computation engine waits until all the jobs are computed20:06
@wikingyep true20:06
@wikingbut what's the job engine20:06
@wiking?20:06
@wikingbecause so far it's abstract20:07
@wikingwe need a library or something that does all the underlying job20:07
@wikinglike finding a free node and sending the job there20:07
lambdaywiking: yeah and I guess these all will be within the parallel implementation of the job engine.. yes as of now its abstract only, the serial computation engine which is currently there just computes these jobs sequentially20:08
@wikinglambday: yeah i wouldn't recommend implementing the scheduling and all the networking stuff ourselves20:09
lambdayso, we need implementation of the submit_job method and wait_for_all...20:09
@wikingi was wondering whether you've been talking about what libraries to interface with20:09
lambdaywiking: we planned using mpi for that20:09
@wikingbecause for sure there are like millions of engines like this out there20:09
@wikinglambday: ookey that's an option20:10
lambdaywiking: Heiko is currently working on that for a project that needs log-det computation on cluster..20:10
@wikingyeah i got that T:)20:11
lambdaywiking: if you have any other suggestion please let us know... we've kept this outside the scope of gsoc, so not much we discussed about the future of it20:11
@wikinglambday: yeah that's totally out of scope of it20:12
@wikingi'm just thinking what we should support/implement in the future20:12
lambdaybut the job/engine design is flexible enough as per we think20:12
lambdaywhat idea do you have in mind for dispatching jobs other that mpi (I don't have experience in this yet)20:13
@wikinglambday: well that's the thing20:13
@wikingi'm looking into other cluster infrastructure implementations20:14
@wikingwhat we could use/support20:14
lambdaythat's super cool!20:14
@wikingi'm wondering now about hadoop20:14
lambdayah I was just about to say this20:15
@wikingbecause it's quite hot nowadays20:15
@wikingfor big data computation20:15
lambdayyeah it is...20:15
@wikingbut in the past when i was doing hadoop pipes20:15
@wiking(the c++ interface of hadoop) i had quite some troubles20:15
@wikingbut now i'm checking into it20:16
lambdaywhat's the main difference in openmpi and hadoop anyway?20:16
@wikinganyhow i'd like to support to read/write mahout vector,matrix so we'll have hdfs support in shogun for sure20:16
@wikinglambday: well there's a lot :)20:16
@wikingfor one hadoop is java native :P20:16
lambdayyeah that but functionally?20:16
@wikinglambday: and hadoop is bascially giving you a filessystem20:17
@wikinga distributed fs20:17
@wikingwhile openmpi is really just purely doing the communication among the nodes20:17
lambdaywiking: but when we have independent computation jobs (like the ones I created for log-det) we don't need to have any msg massing at all... do you think openmpi won't be a bottleneck for this case?20:18
@wikinglambday: what do you mean you dont need msg massing?20:18
lambdayerr20:18
lambdaypassing20:18
@wikinglambday: who's gonna take care of communicating amongst the nodes?20:19
@wikingand who's gonna know at all about the nodes20:19
lambdayya but these nodes can compute each of the tasks independently, so in that sense..20:20
lambdayI mean, in this scenario, what advantages does hadoop have over mpi20:21
@wikingyeah but then after the computation is finished20:21
@wikingsomehow it has to be signaled and sent back the result20:21
lambdayyeah that's true20:21
@wikingwell hadoop takes care of lot of things20:22
@wikinglike having a fault tolerant computation engine (i.e. when a node dies while doing a job)20:22
@wikingand then having the hdfs which all the nodes share20:22
@wikingand some funky optimization20:22
@wikinglike the scheduler of the job will assign the node to do the computation20:23
@wikingthat has the fastest way of accessing the data in the hdfs20:23
lambdayokay... that sounds great20:24
lambdaybut you had some troubles with the c++ interface... so would it fit in?20:24
@wikinglambday: i'm just cehcking the latest stable20:25
@wikingthe last time i've used this was like 1.5 years ago20:25
lambdayif it works - awesome.. I am sure Heiko can give some inputs... I will be really happy if I can be of any help in implementing any of these stuffs20:26
@wikingyeah i'll do today a test with this20:27
@wikingsee if i can get something out of the new version of hadoop and it's c++ api20:27
@wikingseems to be much more mature now20:27
lambdaythat's great20:27
-!- hushell [~hushell@8-92.ptpg.oregonstate.edu] has joined #shogun20:30
-!- naywhayare [~ryan@spoon.lugatgt.org] has quit [Ping timeout: 276 seconds]20:37
-!- zxtx [~zv@cpe-75-83-151-252.socal.res.rr.com] has quit [Ping timeout: 256 seconds]20:38
-!- naywhayare [~ryan@spoon.lugatgt.org] has joined #shogun20:43
shogun-buildbotbuild #23 of osx1 - libshogun is complete: Failure [failed test]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/osx1%20-%20libshogun/builds/23  blamelist: Heiko Strathmann <heiko.strathmann@gmail.com>, Thoralf Klein <thoralf.klein@zib.de>20:58
-!- lisitsyn [~lisitsyn@fb2-lo1.global63.net] has joined #shogun21:39
-!- zxtx [~zv@rrcs-76-79-81-162.west.biz.rr.com] has joined #shogun22:00
-!- lambday [67157c37@gateway/web/freenode/ip.103.21.124.55] has quit [Ping timeout: 250 seconds]22:03
-!- pickle27 [~kevin@d67-193-243-174.home3.cgocable.net] has left #shogun []22:33
-!- iglesiasg [~iglesias@89-77-4-38.dynamic.chello.pl] has joined #shogun23:12
-!- mode/#shogun [+o iglesiasg] by ChanServ23:12
@iglesiasggood evening people23:12
-!- shogun-notifier- [~irker@7nn.de] has joined #shogun23:13
shogun-notifier-shogun: Kevin :develop * 53014e3 / / (10 files): https://github.com/shogun-toolbox/shogun/commit/53014e3b94e82d366c5a9cdb730ec628eb813f3d23:13
shogun-notifier-shogun: added UWedgeSep and python modular examples for all current ICA methods23:13
shogun-notifier-shogun: Sergey Lisitsyn :develop * 7d6ee1d / / (10 files): https://github.com/shogun-toolbox/shogun/commit/7d6ee1dc3ce1d9cd1905c6b71e2dd81a314a20c523:13
shogun-notifier-shogun: Merge pull request #1447 from pickle27/develop23:13
shogun-notifier-shogun:23:13
shogun-notifier-shogun: added UWedgeSep Alg and python modular examples for all current ICA methods23:13
lisitsynhey iglesiasg23:14
@iglesiasglisitsyn, how is it going?23:15
shogun-notifier-shogun: hushell :develop * 6badc95 / / (5 files): https://github.com/shogun-toolbox/shogun/commit/6badc95efdf9816841dd1cc82e8e04cbde7e49d323:15
shogun-notifier-shogun: max product inference for factor graph model23:15
shogun-notifier-shogun: Fernando Iglesias :develop * 636a71b / / (5 files): https://github.com/shogun-toolbox/shogun/commit/636a71bb190549cdb43ec28f263cc5811a6bd7b023:15
shogun-notifier-shogun: Merge pull request #1379 from hushell/fg_dev23:15
shogun-notifier-shogun:23:15
shogun-notifier-shogun: max product inference for factor graph model23:15
lisitsynha23:15
lisitsyniglesiasg: ok, what about you?23:16
@iglesiasglisitsyn, I am good. This is my last night of holidays :(23:16
lisitsyniglesiasg: I am really curious why you have chose poland ;)23:17
@iglesiasglisitsyn, well it is cheap, close to home and with beach holidays ;)23:18
lisitsyniglesiasg: c'mon you are from spain, it is common destination for 99% of mid class russians ;)23:19
@iglesiasglisitsyn, Spain or Poland?23:19
lisitsynspain23:19
@iglesiasgas you may understand I have been in Spain quite a bit already hehe23:19
@iglesiasgI like to visit new places23:19
lisitsynhaha23:19
lisitsynyeah but beaches are better in spain right?23:20
@iglesiasgbut yeah, there are places left to visit in Spain for me23:20
gsomixiglesiasg, >> I like to visit new places | how about Russia next year? >:323:20
@iglesiasglisitsyn, there are better beaches in Spain than the one I am here23:20
@iglesiasggsomix, hehe why not!23:21
lisitsynyeah we discussed st. petersburg once :D23:21
@iglesiasggsomix, St. Petersburgh looks awesome23:21
lisitsynit is pretty swedish though23:21
@iglesiasgit will take me years before I get right the spelling hehe23:21
lisitsyniglesiasg: you should get to gsomix's origin place23:22
@iglesiasgwhere is it?23:22
gsomixhuh!23:22
@iglesiasgKamchatka is also impressive!23:22
lisitsynwell all around here23:22
lisitsynbut it is pure russia23:22
lisitsynnot the city so more pure for sure23:22
@iglesiasgI checked a bit more about both St. Petersburg and Kamchatka23:22
lisitsynkamchatka should be quite expensive23:22
gsomixiglesiasg, lisitsyn pure russia... no, no cities! I'm here now: 52.337542,49.92392223:24
lisitsyniglesiasg: yo yo saint P should be not that expensive comparing to kamchatka23:25
@iglesiasggsomix, google maps doesn't show me much about that location hehe23:25
lisitsyniglesiasg: there is freaking nothing :D23:25
@iglesiasglisitsyn, saint P is da city23:25
gsomixiglesiasg, indeed.23:25
@iglesiasghaha23:25
lisitsynit is like 50 kms and you are in kazakhstan23:26
@iglesiasgBorat?!23:26
lisitsynthe borat yes23:26
lisitsynhmm I think more23:27
@iglesiasgI have to start saving money, I want to go up to Kilimanjaro23:27
shogun-buildbotbuild #1490 of bsd1 - libshogun is complete: Failure [failed test]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/bsd1%20-%20libshogun/builds/1490  blamelist: hushell <hushell@hushell-U510.(none)>23:27
lisitsynmaybe ~100 or whatever23:27
shogun-buildbotbuild #1459 of deb2 - static_interfaces is complete: Failure [failed configure]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/deb2%20-%20static_interfaces/builds/1459  blamelist: hushell <hushell@hushell-U510.(none)>23:27
@iglesiasgcrazy expensive23:27
shogun-buildbotbuild #24 of osx1 - libshogun is complete: Failure [failed test]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/osx1%20-%20libshogun/builds/24  blamelist: hushell <hushell@hushell-U510.(none)>23:28
lisitsyniglesiasg: like gsoc I guess :D23:28
@iglesiasglisitsyn, so so23:29
shogun-buildbotbuild #1491 of bsd1 - libshogun is complete: Failure [failed test]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/bsd1%20-%20libshogun/builds/1491  blamelist: Fernando Iglesias <fernando.iglesiasg@gmail.com>23:32
shogun-buildbotbuild #25 of osx1 - libshogun is complete: Failure [failed test]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/osx1%20-%20libshogun/builds/25  blamelist: Fernando Iglesias <fernando.iglesiasg@gmail.com>23:33
hushelliglesiasg: Thanks! :)23:43
@iglesiasghushell, thanks to you ;)23:44
hushelliglesiasg: Next PR coming soon, I see how hard your job is, haha23:45
@iglesiasghehe23:45
-!- lisitsyn [~lisitsyn@fb2-lo1.global63.net] has quit [Quit: Leaving.]23:49
shogun-buildbotbuild #1281 of cyg1 - libshogun is complete: Failure [failed compile]  Build details are at http://www.shogun-toolbox.org/buildbot/builders/cyg1%20-%20libshogun/builds/1281  blamelist: hushell <hushell@hushell-U510.(none)>23:50
@iglesiasgall right guys, going to bed now. See you tomorrow afternoon!23:59
--- Log closed Thu Aug 22 00:00:25 2013