Open in new window / Try shogun cloud
--- Log opened Mon Jul 08 00:00:22 2013
@sonney2kvan51, so all the code is kind of in there and just needs some tweaks to work with shogun00:00
van51sonney2k: ok, nice00:01
van51sonney2k: I'll start it this week00:01
van51sonney2k: sent first PR00:06
shogun-notifier-shogun: van51 :develop * cb3d559 / / (3 files):
shogun-notifier-shogun: Added support for normalization in HashedDocDotfetures00:15
shogun-notifier-shogun: Soeren Sonnenburg :develop * 34c2f9c / / (3 files):
shogun-notifier-shogun: Merge pull request #1215 from van51/feature/hashing00:15
shogun-notifier-shogun: Added support for normalization in HashedDocDotfetures00:15
shogun-buildbotbuild #1209 of bsd1 - libshogun is complete: Failure [failed test]  Build details are at  blamelist: Soeren Sonnenburg <>00:31
@sonney2kvan51, ok that one is merged00:37
@sonney2kvan51, btw I realized that your DelimiterTokenizer skipping over multiple e.g. spaces might be a good thing when reading e.g space delimited files00:39
@sonney2kvan51, if it is not too hard it would be nice to bring this back as an option00:39
van51sonney2k: that was what I had in mind00:40
van51sonney2k: yeah it shouldn't be hard00:40
van51sonney2k: I'm solving some compiler complaints now and I'll push later on00:40
@sonney2kvan51, so for text based stuff I think n-grams are the best there is anyway, some people like to use n-words or so too. So that might be the only other tokenizer I can currently think of (except for some very sophisticated ones using stemming etc we shouldn't do without some use case)00:41
@sonney2kanyway later I better sleep now00:43
shogun-buildbotbuild #1210 of bsd1 - libshogun is complete: Failure [failed test]  Build details are at  blamelist: van51 <>00:43
-!- van51 [] has quit [Ping timeout: 246 seconds]00:45
-!- van51 [] has joined #shogun00:54
-!- nube [~rho@] has joined #shogun00:54
van51sonney2k: my connection decided to take a break00:56
van51sonney2k: yea n-words sounds reasonable to have as well and it shouldn't be too hard00:56
van51sonney2k: just an extension of DelimiterTokenizer00:56
van51I'll see it from tomorrow00:56
shogun-buildbotbuild #1331 of deb3 - modular_interfaces is complete: Failure [failed test python_modular]  Build details are at  blamelist: Soeren Sonnenburg <>01:12
shogun-buildbotbuild #1332 of deb3 - modular_interfaces is complete: Failure [failed test python_modular]  Build details are at  blamelist: van51 <>01:36
-!- foulwall [~user@2001:da8:215:6100:851a:56a4:fbd1:1de6] has joined #shogun02:57
@iglesiasghushell, hi there!03:03
@iglesiasghushell, everything going good with the factor graph?03:04
-!- shogun-notifier- [] has quit [Quit: transmission timeout]03:15
-!- van51 [] has quit [Quit: Leaving.]03:27
-!- nube [~rho@] has quit [Quit: Leaving.]03:55
shogun-buildbotbuild #451 of nightly_default is complete: Failure [failed test]  Build details are at
hushelliglesiasg: hi iglesiasg04:32
@iglesiasghello hello :)04:33
hushellThere are more work than my imagination :(04:33
hushellI am still working on it04:33
hushellfinishing it in few hours04:33
@iglesiasgthat sounds pretty good then!04:33
hushellokay, talk you later04:34
@iglesiasgsee you04:34
-!- iglesiasg [] has quit [Quit: Ex-Chat]04:52
-!- foulwall [~user@2001:da8:215:6100:851a:56a4:fbd1:1de6] has quit [Ping timeout: 245 seconds]05:38
-!- FSCV [~FSCV@] has quit [Quit: Leaving]06:57
-!- hushell [] has quit [Ping timeout: 268 seconds]07:20
-!- votjakovr [] has joined #shogun07:22
-!- nube [~rho@] has joined #shogun07:45
gsomixsonney2k, hello. I have started work on csv reader and write some stuff for van51. cu at evening, I'll send PR.07:50
gsomixneed to go now07:50
-!- foulwall [~user@2001:da8:215:c252:9dcb:c7a6:35c0:77ac] has joined #shogun08:25
foulwallping sonney2k08:26
@sonney2kfoulwall, hey there!08:27
@sonney2kgsomix, go go!08:27
foulwallHi sonney2k , could I use CNormOne to normalize the features? and I don't exactly know the mechanism in that.08:32
foulwallAh, I got from /examples, I'll have a try.08:39
@sonney2kfoulwall, sorry for what?08:40
foulwallsonney2k: the domain of features are not fixed to [0, 1] since the data importer, so I need to normalize it08:42
@sonney2kI still don't understand08:42
@sonney2kif the domain is not 0..1 why not keep it?08:42
foulwallsince some features in australian.libsvm.h5 are out of domain.08:43
@sonney2kfoulwall, coudn't you then just display the real feature range?08:44
@sonney2kI would prefer to keep the data as is08:44
@sonney2kfoulwall, oki gtg brb08:45
foulwallsonney2k: cu08:46
-!- sonne|work [] has joined #shogun08:57
sonne|workfoulwall: back08:57
-!- foulwall [~user@2001:da8:215:c252:9dcb:c7a6:35c0:77ac] has quit [Ping timeout: 245 seconds]08:58
-!- foulwall [~user@2001:da8:215:6100:2d48:68e4:4af2:4837] has joined #shogun09:10
sonne|workfoulwall: ok?09:11
foulwallHi sonne|work , I think we can do regress/classification/etc in [0,1] and display in its original domain? or just as you said do them in original domain and display09:13
sonne|workfoulwall: yeah display in its original domain and for now don't care about normalization. We can add a tab later allowing people to select how to   normalize the data09:15
-!- hushell [] has joined #shogun09:15
foulwallok, gotcha!09:16
hushellsonney2k: CDynamicArray cannot store CInheritedClass* type, which is subclass of CSGObject?10:01
hushellsonney2k: I got an error like this: /DynamicArray.h:625:4: error: invalid conversion from 'shogun::CFactorDataSource***' to 'shogun::CSGObject***'10:03
hushellsonney2k: Now I add (CSGObject***) to DynamicArray.h:625, i.e. m_parameters->add_vector((CSGObject***)&m_array.array . Is this the right way?10:13
sonne|workhushell: did you use DynamicObjectArray?10:33
sonne|workhushell: I need more context...10:34
-!- iglesiasg [] has joined #shogun10:38
-!- mode/#shogun [+o iglesiasg] by ChanServ10:39
-!- iglesiasg [] has quit [Client Quit]10:39
-!- iglesiasg [] has joined #shogun10:40
-!- mode/#shogun [+o iglesiasg] by ChanServ10:40
hushellsonne|work: no, I am using DynamicArray<CFactor*> to store a set of factors, which has to be dynamically increasing, and CFactor is an inherited class from CSGObject10:48
sonne|workhushell: then please use CDynamicObjectArray for that10:49
hushellsonne|work: DynamicObjectArray is not a template class10:49
hushellI need to explicitly specify10:49
sonne|workhushell: why?10:49
sonne|workCDynamicArray won't work with serialization10:50
hushellI have 2 such kind array, I mean in this way the definitions are clearer10:50
hushellnow I am doing like this: CDynamicArray<CFactor*> m_factors;10:51
hushell    // Data source objects if data is shared among multiple factors10:51
hushell    CDynamicArray<CFactorDataSource*> m_datasources;10:51
sonne|workhushell: CDynamicArray is not intended for SGObjects's - I know you have to use casts with DynamicObjectArray but you should rather use that10:52
hushellsonne|work: CDynamicObjectArray is the only way? how about DynArray?10:55
sonne|workthe only way if you want serialization10:55
hushellsonne|work: okay, then in deconstructor, do I need to call SG_UNREF for each element in DynamicObjectArray?10:57
hushellif each element is a pointer10:57
sonne|workit is made for SGObjects10:57
sonne|workso it will do REF/UNREF automatically10:57
hushellokay, got it10:57
hushellI am curious where the REF is set10:58
hushellSG_UNREF only let pointer to NULL, where the free operation being taken?11:00
hushellthe question may be quite silly :)11:00
sonne|workhushell: in SGObject's unref() call11:03
-!- HeikoS [] has joined #shogun11:07
-!- mode/#shogun [+o HeikoS] by ChanServ11:07
hushellsonne|work: Thanks! not be used to without delete11:08
sonne|workhushell: sorry what? it is calling delete in SGObject once the refcount reaches 011:08
hushellsonne|work: I mean when coding, I don't need to delete for myself :)11:11
sonne|workhushell: only call SG_UNREF for SGObject derived object's yes11:13
@HeikoSvotjakovr: hi!11:14
@iglesiasghushell: I used this DynamicObjectArray in the PrimalMosekSOSVM, take a look if you want to see how to use its API11:14
@iglesiasghushell: IIRC I spent a bit of time debugging until I got it working fine without leaks and so11:15
votjakovrHeikoS: hi!11:17
hushelliglesiasg: yours is the best reference :)11:19
hushellbut shall I use CDynamicObjectArray or CDynamicObjectArray& for a return type?11:19
@HeikoSvotjakovr: still no PR, how are things with that?11:21
@HeikoSvotjakovr: we have to be careful not to fall behind in schedule11:21
@iglesiasghushell: let me check a moment11:21
hushellIs not allow to use std::vector, right?11:22
@HeikoShushell: try to avoid this for class members11:22
@iglesiasghushell: internally it is fine11:23
hushellbut my case is members, okay let me make CDynamicObjectArray works11:24
@HeikoShushell: what objects do you want to put inside?11:24
@iglesiasghushell: I don't returning a CDynamicObjectArray would entail much overhead (it would copy a few numbers and a pointer, but not the whole array)11:24
@iglesiasgI don't think*11:24
@HeikoSiglesiasg: yeah thats pretty fine11:24
@HeikoShushell: CDynamicObjectArray is only for shogun class instances11:25
votjakovrHeikoS: ah, sorry, now i'm fixing docs, implementing p(y*|y)... it's the base of future work, then things will be much quicker11:25
@HeikoSotherwise DynamicArray or DynArray11:25
hushellI remember last time I got similar question for SGVector, but both case work well11:25
@HeikoSvotjakovr: so what about the numerical integration for p(y*|y) ?11:25
hushellHeikoS: i.e. CDynamicObjectArray also works for derived classes ?11:26
@HeikoShushell: yes11:28
@HeikoShushell: also it can do reference counting11:28
@HeikoShushell: you have to be a bit careful about that, there is a boolean flag that you can set11:28
@HeikoSits useful if you put objects into the array and then they ger all unrefed when the array is deleted11:29
@HeikoSso no need to delete the elements by hand11:29
votjakovrHeikoS: in process... i created class-wrapper for functions of one variable, is it ok?11:29
@HeikoSbut also if you dont want that to happen, you can turn it off11:29
@HeikoSvotjakovr: a new class? or in CStatistics (where I think it would be fine)11:29
votjakovrHeikoS: with overloaded operator ()11:29
@HeikoSvotjakovr: ehm, explain that :)11:29
-!- vgorbati [c3ee5cb1@gateway/web/freenode/ip.] has joined #shogun11:31
votjakovrHeikoS: we need to integrate function of one variable, i created class IntegrableFunction with virtual float64_t operator() (float64_t x)=0;, which return f(x)11:32
@HeikoSvotjakovr: I like that! does it work already?11:33
hushellHeikoS: Thanks! It make sense :)11:33
votjakovrHeikoS: not yet, but in progress...11:33
@HeikoSvotjakovr: ok, so what is missing for the PR?11:34
votjakovrHeikoS: should it be in Statistics?11:34
@HeikoSvotjakovr: I would have put the implementation into CStatistics as a function rather than having a new class, so only the Gauss-blabla method, but yours should be in mathematics folder then11:35
-!- zxtx_ is now known as zxtx11:36
hushellHow do I register a CDynamicObjectArray member? use SG_ADD((CSGObject**)...) or add_vector()/11:37
@HeikoShushell: the first, with a cast11:38
-!- vgorbati [c3ee5cb1@gateway/web/freenode/ip.] has quit [Quit: Page closed]11:38
hushellHeikoS: but it indeed a vector11:38
votjakovrHeikoS: ok, for the probit: fix docs + implement evaluate_log_probability for all likelihoods + implement get_probabilities() in GPC class + test - this will be very quickly11:38
@HeikoShushell: well no, it is a dynamically growing complex structure11:38
hushellHeikoS: okay, Thanks!11:39
@HeikoSvotjakovr: ok, sounds good, keep on then!11:39
@iglesiasgsonne|work, HeikoS, lisitsyn: did you have a look to the performance comparison in
@HeikoSiglesiasg: yes, we had a little discussion about that11:41
@iglesiasgI wonder, if I believe that our kNN is much faster (using the cover tree) than what they state there, is it ok to run experiments on our own and contact them?11:41
@iglesiasgI find it sort of lying, but no idea if this is a common case in research :)11:41
@HeikoSiglesiasg: While I agree, I am not sure whether this is worth it11:42
sonne|workiglesiasg: that was IIRC an old implementation of KNN before you did the covertree11:43
-!- lambday [67157d37@gateway/web/freenode/ip.] has joined #shogun11:43
sonne|workiglesiasg: naywhayare is the author of mlpack (here in the channel btw)11:43
lambdayHeikoS: hi11:43
sonne|worklambday: hey there !11:43
lambdaysonne|work: hi11:43
@HeikoSmaybe tell them11:43
sonne|worklambday: moin moin11:43
@iglesiasgsonney2k: but the paper is from 201311:43
@HeikoSlambday: hey!11:43
lambdaymoin moin :)11:43
sonne|workiglesiasg, HeikoS naywhayare knows11:43
@iglesiasgnaywhayare: morning!11:44
sonne|worklambday: exactly :)11:44
lambdayHeikoS: number of allocations for SG was way too higher, but for the sparse matrix I initially had, total number of bytes allocated was smaller for SG11:44
sonne|workHeikoS: can you provide a more reasonable sparse amtrix?11:44
sonne|workHeikoS, lambday: why do you care at all?11:44
@HeikoSsonne|work, lambday yes I can, just thinking of how to get it into shogun :)11:45
@HeikoSsonne|work: what do you mean?11:45
lambdaythe matrix is really huge11:45
sonne|workHeikoS: you could write a libsvm file11:45
sonne|workHeikoS: about the number of allocs11:46
@HeikoSsonne|work: just curious11:46
sonne|worklambday: are you talking about #allocs for the sparse matrix?11:46
sonne|workthen shogun will do one alloc per sparse vector of the matrix11:46
lambdaysonne|work: yes..11:46
sonne|workgood thing is that you can easily change one vector11:46
lambdaysonne|work: but how about the entries?11:46
sonne|workand grow the matrix11:47
sonne|worklambday: that was already for the entries11:47
@HeikoSlambday: so about these conversions from shogun to eigen311:47
@HeikoShow badly do they hurt us?11:47
sonne|workHeikoS: depends on the sparsity of the matrix11:47
lambdayHeikoS: pretty bad... if I don't get to keep eigen3's sparse as a member then I gotta change it every time I call apply11:47
lambdayand that would be A LOT11:48
lambdayfor cg11:48
-!- vgorbati [c3ee5cb1@gateway/web/freenode/ip.] has joined #shogun11:48
lambdayif I do get to keep eigen3's sparse, its just once11:48
@HeikoSlambday: ok sure, we cannot do this, but say if we have an eigen3 member what is not registered11:48
@HeikoSbut just remporary11:48
sonne|workHeikoS: computation wise already the test is 2 times slower11:48
sonne|workwith eigen311:48
lambdayHeikoS: temporary?11:48
sonne|workso at this stage it doesn't look like using eigen3 is a good idea for sparse11:48
@HeikoSsonne|work: yes, but we need eigen3's solvers at some point11:48
lambdaysonne|work: well, that depends on the sparsity pattern, doesn't it :(11:49
sonne|workHeikoS: for sparse matrices?11:49
@HeikoSsonne|work: yes, there is a case where we need to use plain vanille cg on sparse linear systems11:49
@HeikoSlambday: but for cocg_m you will implement that anyway by hand11:49
lambdayHeikoS: could you please elaborate on the temporary thing? I was thinking it of keeping it as a member, so it will stay as long asn that object last11:49
@HeikoSlambday: ah no, the triangular sparse solver is also there11:50
@HeikoSlambday: sorry, temporary in the sense that it is not a class member (and is not serialised then)11:50
@HeikoSlambday: you you have an object that can be accessed by the instance, but is lost after the instance is deleted11:50
@HeikoSlambday: so no sending around in networks etc11:50
@HeikoSlambday: what about the triangular solver in cocg_m ? for the shifts, how would you do that?11:51
@HeikoSsonne|work, lambday man I really did not expect that many framework challenges in this project ;)11:51
lambdaylol neither did I :D first complex, now this :D11:52
lambdayHeikoS: cocg_m just applies the operator to the vector :-/ the one that krylstat has11:52
lambdayfor shifted stuff they do it in two stages11:52
@HeikoScocg_m but we need a loop over the shifts and a backsolve in each iteration11:53
lambdaythey do it here11:54
@HeikoSI see11:55
lambdayHeikoS: and, also, I think I can get cocg_m under the same base11:56
@HeikoSlambday: ok then11:56
lambdaycause we have to send shifts anyway, why not weight?11:56
@HeikoSlambday: ok11:56
lambdaybut that should be optional11:56
lambdayI mean, having two solve methods for cocg11:56
lambdayone that returns the matrix and another the vector11:57
@HeikoSlambday: thats a good idea I think11:57
@HeikoSlambday: how big were the test matrices you used?11:57
lambdaybut about sparse -11:57
lambdaya mil x a mil11:57
@HeikoSI have a 200000 matrix which I used in experiments, will that allow to see differences?11:58
@HeikoSat least it has a realistic sparsity structure :)11:58
lambdayso things to check are (a) time (cpu/wall clock) (b) number of allocs (c) total bytes allocated11:59
@HeikoStime is most important, memory is just interesting12:00
sonne|workHeikoS: memory is clear just #of sparse vectors12:01
lambdayand log-det framework will work even if eigen3 is not there.. if we provide it an operator, an eigen solver and a linear solver that works without eigen3, it should work12:01
@HeikoSlambday: thats pretty cool then12:01
lambdayit all depends on the implementation12:01
@HeikoSlambday: question is what is best for the default case12:02
@HeikoSbecause letting users choose can create confusion if not documented properly12:02
sonne|workHeikoS: what is a 200000 matrix ?12:02
@HeikoSsonne|work: square and psd12:02
sonne|work200k x 200k in size?12:02
@HeikoSsonne|work: and badly conditioned12:02
@HeikoSlet me check12:03
lambdayHeikoS: is that for the ozone example?12:03
@HeikoSlambday:  yes thats the one12:03
lambdaycool!!! :D12:03
@HeikoSlambday: but it will be way easier to generate it from python12:03
@HeikoSsonne|work:  nnz(Q) =  372394212:05
sonne|workquite a lot12:05
lambdayHeikoS: could you please send me the script? I gotta read it from a file manually in cpp I guess..12:06
sonne|workso shogun's code would be twice as fast I would expect12:06
lambdayHeikoS: sonne|work: if that is the case (default) then we can just use shogun's for all our stuffs... just gotta add serialization and we're good to go12:07
@HeikoSsonne|work: ok good!12:07
@HeikoSlambday: that would be good12:07
@HeikoSlambday: I was a bit worried about the solvers (e.g. individual ones)12:07
lambdayHeikoS: yes I was just thinking that12:08
@HeikoSlambday: also we will need preconditioned CG at some point12:08
sonne|worklambday: well we should test this12:08
@HeikoSbut for that we can just do a conversation to eigen3 once and then work from there12:08
sonne|worklambday, HeikoS I expect eigen3 to be faster for extremely sparse matrices12:08
lambdaypreconditioning is just once per solve12:08
lambdaysonne|work: yes.. seems so12:08
* sonne|work food12:09
@HeikoSsonne|work: haha, enjoy! :)12:09
@HeikoSlambday: should I just send you this matrix as a matlab file?12:10
lambdayHeikoS: yes.. or the python script that generates it12:10
@HeikoSlambday: I read things from a file for that, but I can try to write something similar12:10
@HeikoSlambday: yes, I will first send matlab file and then a bit later a script to generate some large matrices randomly12:11
lambdayHeikoS: alright12:12
lambdayhow big is that matrix?12:12
lambdayoh wait!12:12
@HeikoSthe file?12:12
lambdayis that in the ozone code zip?12:12
lambdaythen I might actually have that12:12
@HeikoSoh yes in fact it is12:12
lambdaywait I am checking12:12
@HeikoSthere is also the code how to generate the matrix Q as a function of kappa12:12
@HeikoSand for that, some files are loaded12:12
@HeikoSlambday: this contains the building blocks12:13
@HeikoScheck mcmc_ozone.m12:13
@HeikoSload 'ozone_data.mat'12:13
@HeikoScreate_Q_matrix=@(kappa) GiCG+2*kappa^2*G+(kappa^4)*C0 + speye(size(A,2))*1E-10;12:13
@HeikoSand then you can to Q=create_Q_matrix(2^-10) or so12:13
lambdaywell, I can do that from octave? :-/ I don't have matlab12:14
@HeikoSshould work12:14
@HeikoSif not, let me know12:14
@iglesiasgHeikoS: let me ask you; say  I want to have a private method that returns some std::vector, I guess that is not good since vector should be included in the header. However, as I just want to use it internally -- it is private -- it feels like it should be ok being able to do it12:15
@iglesiasghave you faced that before?12:15
-!- nube [~rho@] has quit [Quit: Leaving.]12:16
-!- van51 [] has joined #shogun12:16
@HeikoSiglesiasg: I think that should be fine, however, it is not nice :)12:20
@HeikoSiglesiasg: why do you want the std::vector?12:20
@iglesiasgHeikoS: I would like to use std::vector because I want to have a vector of SGVectors12:21
@HeikoSiglesiasg: does it have to be dynamically growing?12:22
@iglesiasghowever I think I could manage using another thing in Shogun for that, it was a bad example12:22
@HeikoSif not, why not use a matrix?12:22
@iglesiasgno, it does not need to be12:22
@iglesiasgI want to index something like this12:22
@HeikoSthen you should definitely go for a matrix12:22
@iglesiasgC(i,j) where each of these elements is a vector12:22
@HeikoSso it is a 3d array?12:23
@iglesiasgsort of, but I would rather index it like that, with two indices12:23
@HeikoSiglesiasg: I see, and how do you represent that currently?12:24
lambdayHeikoS: alright, matrix loaded12:24
@HeikoSlambday: cool that it worked!12:24
lambday(although I didn't understand a bit of what's going on with the code!)12:24
lambdaybut then I write this to an ascii file and read it from shogun?12:24
@iglesiasgHeikoS: it is not represented yet, I want to first see if I can use something like std::vector< std::vector<SGVector>>12:24
@HeikoSlambday: yes, although I never loaded sparse matrices, sonne|work pointed out the svmfile might be a possibility12:25
van51iglesiasg: vector inception :O12:25
@iglesiasgHeikoS: apart from that, I am in a similar situation where I would like to use a std::set, and that a private method returns it12:25
@HeikoSiglesiasg: I see, let me think12:25
@iglesiasgvan51: maybe it looks weird yeah :S12:25
@HeikoSiglesiasg: I mean you could have a DynamicArray of DynamicObjectArrays12:26
@iglesiasgbut SGVector are not SGObject12:26
@HeikoSiglesiasg: the thing is: if you do the private method with std::vector you can never use it from another language than c++12:26
@HeikoSiglesiasg: you can use DynArray<float>12:26
@HeikoSthere is one array for CSGObject and one for generic stuff12:27
@HeikoSiglesiasg: I had these issues before and did not really come up with a nice solution12:27
@HeikoSiglesiasg: what about a large matrix?12:27
@HeikoSthen you just have to do index conversions?12:28
@iglesiasgyeah, but the indexing won't be nice12:28
@HeikoSiglesiasg: a block matrix12:28
-!- foulwall [~user@2001:da8:215:6100:2d48:68e4:4af2:4837] has quit [Quit: super]12:28
@HeikoSiglesiasg: you could have a private method for that12:28
@HeikoSthen it would be more generic12:28
@iglesiasgaham, that sounds more feasible12:28
@iglesiasgI guess I won't be able to use operators like () and [] though12:28
@HeikoSyou could add a method that takes a matrix and your three indices and then returns the coresponding thing for you12:29
@HeikoSiglesiasg: ah we really should implement the NDArray soon12:30
@HeikoSsince this would solve everything12:30
@iglesiasgnow that I think of the the SGMatrixList may help me for this too12:30
@iglesiasgI am actually more concerned now about the set thing12:30
@HeikoSiglesiasg: yes, it might, I have not used that yet though12:31
@HeikoSisnt there CSet? is that for CSGObject only?12:31
@iglesiasgHeikoS: I have seen the lib/Set but it does not seem to implement an intersection12:31
@HeikoSiglesiasg: ah yes, thats not a proper set12:32
@HeikoSiglesiasg: so these methods are private?12:32
@HeikoSmaybe you should just use std:: then12:32
@iglesiasgHeikoS: yes, they are private12:32
@HeikoStoo much hassle otherwise12:32
@HeikoSiglesiasg: I mean you can still unit-test them12:32
@HeikoSsince thats also in c++12:32
@iglesiasgbut about what you said that it won't be possible to call from other languages12:33
@iglesiasgsay I have a public train method12:33
@HeikoSbut people will not be able to call them, say if they want to unterstand things better12:33
@iglesiasgwell, but if I make them private12:33
@HeikoSiglesiasg: they cannot be called anyway12:33
@iglesiasgthe idea is that people can't call them12:33
@HeikoSyou are right12:33
@HeikoSiglesiasg: haha :) sorry12:33
@iglesiasg:) but one is going to be still able to call the public methods from target languages right?12:34
@HeikoSwell, it will definitely work, so why not do it12:34
@iglesiasgI am going to try to use the SGMatrixList then but the set will be included in the header I think12:34
@iglesiasgHeikoS: thanks!12:34
@HeikoSiglesiasg: to be honest I would not use the matrix list, our lists are quite messy in terms of memory management12:35
@iglesiasghaha ok12:37
votjakovrHeikoS: if you have a time please look at evaluate_means and evaluate_variances method of LikelihoodModel class, should i add labels as a parameter for that methods or we are only interested in y==1?12:39
@HeikoSvotjakovr: we should have both12:40
@HeikoSvotjakovr: so if labels==NULL (default) y==1 will be used12:40
@HeikoSvotjakovr: and for multiclass, things are a bit different12:40
votjakovrHeikoS: ok, i'll add12:40
van51sonne|work: I've added that option to DelimiterTokenizer12:42
van51sonne|work: I've sent a PR12:42
@HeikoSvotjakovr: please document that properly to avoid confusion :)12:46
-!- nube [~rho@] has joined #shogun12:47
votjakovrHeikoS: yeah :)12:49
-!- nube [~rho@] has quit [Ping timeout: 268 seconds]12:52
-!- foulwall [~foulwall@] has joined #shogun12:52
-!- votjakovr [] has quit [Quit: ERC Version 5.3 (IRC client for Emacs)]13:05
-!- iglesiasg [] has quit [Ping timeout: 260 seconds]13:06
sonne|workvan51: which one now?13:27
van51sonne|work: the delimiter PR is ready13:29
-!- shogun-notifier- [] has joined #shogun13:29
shogun-notifier-shogun: van51 :develop * c7a27ca / / (3 files):
shogun-notifier-shogun: Option to skip delimiters in CDelimiterTokenizer13:29
shogun-notifier-shogun: Soeren Sonnenburg :develop * 583fc6c / / (3 files):
shogun-notifier-shogun: Merge pull request #1217 from van51/feature/tokenizer13:29
shogun-notifier-shogun: Option to skip delimiters in CDelimiterTokenizer13:29
sonne|workvan51: yeah I thought you already did some skip 'n' delimiters13:29
sonne|workvan51: but thansk anway13:29
sonne|workshould be pretty useful for gsomix13:29
van51sonne|work: np13:30
van51sonne|work: I also liked that functionality13:30
sonne|workvan51: I am tempted to think that instead of skipping delimiters you could add an option to skip 'n' delimiters13:31
sonne|workso one would get multiple words13:31
sonne|workvan51: I mean I think the code might be actually pretty similar13:31
van51sonne|work: like the n-words you mentioned yesterday?13:31
sonne|workif you had some counter that counts down to 0 I mean13:32
sonne|workwhich would usually be 113:32
sonne|workjust thinking13:32
van51sonne|work: hm I get what you're saying13:33
shogun-buildbotbuild #1007 of cyg1 - libshogun is complete: Failure [failed configure]  Build details are at  blamelist: van51 <>13:34
van51sonne|work: but the idea behind this PR was to avoid situations like consecutive white space13:34
lambdayHeikoS: libsvm format is row#: col#:val, right? val only nz elements?13:35
van51sonne|work: it may be a bit trickier to maintain both functionalities simultaneously13:35
lambdayHeikoS: sonne|work how do I read from libsvm files in shogun?13:35
sonne|worklambday: we have function in CSparseFeatures13:36
van51sonne|work: I don't mind implementing that, but maybe it would be better in a seperate tokenizer like you proposed yesterday13:36
lambdaysonne|work: okay..13:36
sonne|workvan51: yeah maybe it is more explicit which is good13:37
sonne|worklambday:         CRegressionLabels* load_svmlight_file(char* fname, bool do_sort_features=true);13:37
lambdaysonne|work: yes..13:37
sonne|worklambday: that is what you need13:37
lambdaysonne|work: okay..13:38
lambdayits being huge13:38
-!- nube [~rho@] has joined #shogun13:39
sonne|worklambday: size?13:40
sonne|workI thought it was just 4 million nnz's?13:40
lambdaystill writing13:40
sonne|workahh yes the file itself is big yes13:40
lambdayI am writing from octave13:40
sonne|workbut loaded it would just be 50MB or so13:40
shogun-buildbotbuild #1211 of bsd1 - libshogun is complete: Failure [failed test]  Build details are at  blamelist: van51 <>13:41
shogun-buildbotbuild #1008 of cyg1 - libshogun is complete: Failure [failed configure]  Build details are at  blamelist: Soeren Sonnenburg <>13:45
sonne|workvan51: btw I had a look at the Streaming* code and I don't see any better way than storing the converted output13:53
sonne|workvan51: I mean a sparse vector13:53
shogun-buildbotbuild #1212 of bsd1 - libshogun is complete: Failure [failed test]  Build details are at  blamelist: Soeren Sonnenburg <>13:54
van51sonne|work: so it's going ok13:54
-!- foulwall` [~user@2001:da8:215:6100:259b:cbe5:416f:ba90] has joined #shogun13:59
-!- foulwall [~foulwall@] has quit [Quit: AndroIRC - Android IRC Client ( )]13:59
-!- iglesiasg [] has joined #shogun14:01
-!- mode/#shogun [+o iglesiasg] by ChanServ14:01
sonne|workvan51: yes just add a test. IIRC there was some code to turn streaming* features into normal SparseFeatures (or densefeatures?) - maybe it  would be good to compare data14:09
sonne|workiglesiasg: what do you want to do? you need nd-arrays?14:10
shogun-buildbotbuild #1333 of deb3 - modular_interfaces is complete: Failure [failed test python_modular]  Build details are at  blamelist: van51 <>14:11
@iglesiasgsonne|work: I want to index matrices like this C(i,j)14:17
@iglesiasgthis C(i,j) would be a matrix14:17
naywhayareiglesiasg: hello there14:20
naywhayareI believe the benchmarks were done before the cover tree support was added to shogun, if I remember right14:21
sonne|workiglesiasg: and SGMatrix C; is not enough?14:21
-!- nube [~rho@] has quit [Read error: Connection reset by peer]14:21
naywhayarehere is the script:
naywhayarelike every other library, I used the documentation to figure out what the simplest way to run kNN was14:22
naywhayare(that is, the way a new user would be most likely to approach it)14:23
@iglesiasgnaywhayare: aham ok, so the default option in mlpack is using cover tree / kd tree?14:25
@iglesiasgsonne|work: Wouldn't I need to do a SGMatrix<SGMatrix> then? AFAIK, that's not possible14:25
sonne|workiglesiasg: ahh matrix!14:25
naywhayareiglesiasg: the default option in mlpack is the kd-tree14:26
@iglesiasgnaywhayare: got it, all right14:26
-!- nube [~rho@] has joined #shogun14:26
sonne|workiglesiasg: is this matrix of constant size?14:27
sonne|workiglesiasg: I mean could we just use ndarrays?14:27
@iglesiasgsonne|work: yeah, however then I do not see how to use just two indices14:27
@iglesiasgusing a intermediate function as Heiko said14:27
@iglesiasgbut I wouldn't be able to use operators [] or () I think14:28
sonne|workiglesiasg: ahh ok and your matrixlist class is not an option?14:28
@iglesiasgsonne|work: yeah, I think it could14:28
@iglesiasgI can live doing14:28
@iglesiasgC(n*j + i)14:28
@iglesiasgor something like that14:29
sonne|workiglesiasg: you could add an operator in you matrixlist class too14:29
sonne|workso it would be exactly like this then14:29
@iglesiasgsonney2k: mmm it may be an option14:29
@iglesiasgsonne|work: wrong name :)14:30
@iglesiasghowever, I would like to still include set in the header14:30
@iglesiasgI don't say no way around that14:30
@iglesiasgI am just using it internally though, just need it in the header for the return value of a private method14:30
sonne|worksorry what? set in the header?14:36
sonne|workiglesiasg: ^14:37
@iglesiasginclude set in the header14:37
@iglesiasglike #include <set>, std::set14:38
@iglesiasgsonne|work: ^14:41
sonne|workahh ok then just add this function to modshogun_ignores.i14:42
sonne|workto exclude it from swig14:42
@iglesiasgit makes sense14:42
@iglesiasgdo I need to use some SWIG_IGNORE too?14:42
shogun-buildbotbuild #1334 of deb3 - modular_interfaces is complete: Failure [failed test python_modular]  Build details are at  blamelist: Soeren Sonnenburg <>14:46
sonne|workiglesiasg: no14:57
-!- van51 [] has quit [Quit: Leaving.]15:04
lisitsynheavy irc usage today!15:04
lisitsynit is monday btw15:04
lisitsyngsomix: hushell: foulwall: iglesiasg: lambday: you know what to do15:04
@iglesiasglisitsyn: sure :)15:08
@iglesiasgthanks for the reminder15:08
lambdaylisitsyn: yes :)15:11
foulwall`thanks for the reminder15:12
@HeikoSlambday: i am back15:26
lambdayHeikoS: wb :)15:26
@HeikoSlambday: whats up? :)15:27
lambdayHeikoS: the matrix is so f**king big :( :(15:27
lambdayHeikoS: its still writing :(15:27
lambdayHeikoS: in the meantime I tested to read from a svmlight file to a sparse matrix15:27
lambdayworks fine15:27
@HeikoSlambday: ascii is a bad format15:27
lambdaybut the good thing is, once I write it, I'll use it for all experiments later on :D15:28
@HeikoSlambday, sonne|work it would be good to have some std formats that we can read15:28
lambdayeven for the demo :D15:28
@HeikoSlike hdf515:28
@HeikoSor *.mat15:28
@HeikoSlambday: yes, thats good, its gonna be huge though15:28
lambdayHeikoS: *.mat will be really helpful I think15:29
lambdayHeikoS: I am guessing 100+ MB15:29
lambdaysonne|work: said loaded size will not be more than 50 MB15:29
sonne|workHeikoS: we can read hdf5 and the ascii format matlab does15:29
@HeikoSlambday: loaded is fine, its just a pain to read/write asci15:29
lambdayyes :(15:30
@HeikoSsonne|work: how easy is it to read hdf5?15:30
@HeikoSbecause matlab can export that15:30
sonne|work2 lines15:30
@HeikoSlambday: maybe this is better then?15:30
sonne|workHeikoS: actually new matlab is hdf515:30
lambdayso.. use SGSparseMatrix::load instead of that15:30
lambdaybut I gotta check how to write hdf5 from octave15:31
@HeikoSlambday: thats easy15:31
sonne|workf=HDF5File("fm_train_real.h5","r", "/data/doubles")15:31
sonne|work    feats2.load(f)15:31
@HeikoSI did that before15:31
sonne|work    lab2.load(f)15:31
lambdaysonne|work: ah thanks15:31
@HeikoScool! thx :)15:31
sonne|workis the example15:31
sonne|workbut it can be directly used on SG* datatypes15:32
lambdayHeikoS: so in octave, I just do something like hdf5write('myfile2.h5', 'blah-blah', Q); ?? Q=sparse matrix?15:36
-!- foulwall` [~user@2001:da8:215:6100:259b:cbe5:416f:ba90] has quit [Ping timeout: 264 seconds]15:37
@HeikoSlambday: yes its that simple,15:37
lambdayand then just use shogun's load?15:37
@HeikoSmaybe some details,15:37
@HeikoSjust try with a small matrix15:37
lambdayah crap! octave says "warning: the `hdf5write' function is not yet implemented in Octave"15:39
lambdayHeikoS: :'(15:39
@HeikoSlambday: nice ;)15:39
@HeikoSI can write you one15:39
@HeikoSfrom matlab15:39
@HeikoSIll test a small sparse matrix15:39
@HeikoSjust a sec15:39
lambdayHeikoS: alright15:39
@HeikoSlambday: ah computer overloaded ;)15:45
@HeikoStoo many things at once15:45
lambdayHeikoS: hehe :D15:45
@HeikoSlambday: I need more memory15:46
lambdayHeikoS: how much do you have?15:46
@HeikoS8 are not enough15:46
@HeikoScompiling a big thing currently, thats the problem, anyways, will now store the matrix15:46
sonne|workHeikoS: why are you not using libsvm format?15:46
@HeikoSsonne|work: lambday said it takes ages to write the file15:47
lambdaysonne|work: yes I am doing that15:47
@HeikoSlambday: is it yet finished btw?15:47
lambdaymeanwhile, we're trying with hdf515:47
lambdayHeikoS: nope!15:47
sonne|worklambday: it was just 4 million entries? why does it take so long then?15:48
sonne|worklambday: do you have the sparse matrix as scipy.sparse perhaps?15:48
sonne|workor in which format?15:48
lambdaysonne|work: no.. octave's sparse15:48
sonne|worklambday: but then you could pass this matrix to shogun15:49
lambdayoctave interface?15:50
sonne|worklambday: modular octave interface15:50
@HeikoSsonne|work: how does that work?15:50
sonne|workI don't understand the question15:52
sonne|workthe same way you do python with sparse15:52
sonne|workexamples/undocumented/octave_modular/ features_sparse_modular.m15:52
@HeikoSMatlab: Error using hdf5writec15:52
@HeikoSSparse data is not supported.15:52
@HeikoSsonne|work: we need the matrix from c++15:52
sonne|workHeikoS: well then save it from shogun15:53
@HeikoSsonne|work: ha! clever15:53
sonne|workthough I don't see why you need it from c++15:53
sonne|workyou could use the callbacks15:54
@HeikoSnot following15:54
sonne|workHeikoS: I mean the octave interface for the tests in the same way15:55
@HeikoSsonne|work: sorry I totally dont get what you are talking about. all we want to have is this matrix in available in c++ for testing(not example) how we do this: doesnt matter.15:56
@HeikoSbut I think a good way is to load it from a modular interface and save it in some shogun format15:56
sonne|workjsut save it as ascii file15:56
sonne|workI was under the impression you just need it now for the benchmark so you could have put the benchmark function into e.g. CSparseFeatures for the test and done15:57
@HeikoSsonne|work: ascii file takes long to save, lambday?15:57
@HeikoSgotta go, talk15:58
sonne|workcannot imagine that this is slow for just 4 million elements16:01
-!- foulwall` [~user@2001:da8:215:6100:9594:7f4:ea8a:ecb5] has joined #shogun16:04
-!- foulwall` [~user@2001:da8:215:6100:9594:7f4:ea8a:ecb5] has quit [Remote host closed the connection]16:06
-!- foulwall` [~user@2001:da8:215:6100:9594:7f4:ea8a:ecb5] has joined #shogun16:06
@iglesiasgwiking: hey, I think there might be a problem with the configure script recognizing gmock16:13
@iglesiasgwiking: it said that it was in my machine under /usr/src even though it was not16:14
-!- shogun-notifier- [] has quit [Quit: transmission timeout]16:29
-!- gsomix_ [~gsomix@] has joined #shogun16:32
wikingiglesiasg: doh16:32
wikingiglesiasg: can u send me the configure.log?16:32
-!- gsomix [~gsomix@] has quit [Ping timeout: 256 seconds]16:33
-!- foulwall` [~user@2001:da8:215:6100:9594:7f4:ea8a:ecb5] has quit [Ping timeout: 264 seconds]16:36
wikingiglesiasg: doh, this wasn't helpful :(16:49
@HeikoSlambday: was the matrix saved in ascii by now?17:05
lambdayHeikoS: yes :D17:05
@HeikoSlambday: ok, then I dont need to send another one :)17:05
lambdayHeikoS: yes.. its HUGE17:06
@HeikoSlambday: how big?17:06
@HeikoSlambday: might be a good idea to save it to hdf via some modular interface17:06
lambdayHeikoS: yup.. but first I wanna see which performs better17:07
@HeikoSlambday: yes go ahead :)17:07
lambdayargh... segfault!17:09
@HeikoSlambday: valgrind :)17:09
lambdayHeikoS: sonne|work unthreaded, unopenmp'd version - eigen3 wins by a small margin17:14
lambdaytrying others17:14
-!- zxtx [] has quit [Ping timeout: 245 seconds]17:15
lambdayHeikoS: sonne|work the difference was ~0.04 s17:16
@HeikoSlambday: well, so they are equal :)17:16
lambdayHeikoS: pretty much!17:17
lambdayI am hoping to get a better performance with the pthreaded one17:17
-!- foulwall [~user@2001:da8:215:c252:c82:7dd9:e65a:75e6] has joined #shogun17:17
@HeikoSlambday: yeah ...17:17
@HeikoSlambday: but thats not too interesting for us anyways ... but try it17:17
lambdayHeikoS: yeah just trying17:17
-!- iglesiasg [] has quit [Ping timeout: 240 seconds]17:19
lambdayHeikoS: sonne|work nah! same!17:19
lambdayoh wait!17:19
@HeikoSlambday: threaded will be more useful for larger matrices17:19
@HeikoSlambday: how long does on product take btw?17:20
lambdayHeikoS: sonne|work with threaded, shogun wins17:20
@HeikoSlambday: nice17:20
@HeikoSso matlab needs 0.012 on my machine for one product17:21
lambdayHeikoS: I am testing just the product, 100 times for each17:21
lambday0.005090 for SG17:21
lambday0.005908 for eigen317:22
@HeikoSthats faster17:22
lambdaybut I use corei7+16gb ram17:22
@HeikoSlambday: oh17:23
@HeikoSI dont have such things :)17:23
lambdayHeikoS: I use insti's :D17:23
lambdaymy laptop is 1GB :D17:23
lambdayso, I am testing with octave on mine17:24
@HeikoSoctave is very slow in most things17:24
@HeikoSasserts for example are incredibly slow17:25
-!- iglesiasg [] has joined #shogun17:25
-!- mode/#shogun [+o iglesiasg] by ChanServ17:25
lambdayHeikoS: 0.0141339317:26
lambdayso, I can start using SGSparse then17:27
lambdayso my todo list - (a) add SGSparse under param framework (b) add serialization (c) add mat-vec product method in SGSparseMatrix (HeikoS : do you think we should have the threaded one there as well?) (d) add sparse matrix operator17:29
@HeikoSlambday: looks good, one thing, lets do the serialisation a bit later ok? after the cocg_m work17:30
lambdayHeikoS: alright17:30
@HeikoSc) we should have this in17:30
@HeikoSif its not too much wokr17:30
lambdayHeikoS: alright :)17:30
lambdayHeikoS: lol I'll copy paste sonne|work's code :D17:30
@HeikoSlambday: thats totally fine17:31
@HeikoSlambday:  you could even go openmp here17:31
@HeikoSsince its easier17:31
lambdayHeikoS: yes but that doesn't help much! no idea why!17:32
-!- vgorbati [c3ee5cb1@gateway/web/freenode/ip.] has quit [Quit: Page closed]17:32
foulwallping lisitsyn17:32
-!- pickle27 [] has joined #shogun17:36
@sonney2klambday, hmmhh that is just too fast17:37
@sonney2klambday, I guess one cannot really gain anything by parallelizing stuff17:37
@sonney2kHeikoS, I tried openmp first but basically neither pthreads nor openmp really help17:37
lambdaysonney2k: but pthread helps :-/17:37
@sonney2klambday, what is the speedup?17:37
@sonney2klambday, how many CPU's did you use?17:37
@HeikoSlambday: we can also just leave the parallelization for now17:38
@HeikoScan always add that later17:38
@sonney2klambday, on my i7 at home it didn't help much so what is the speedup single core vs multiple?17:38
lambdayHeikoS: yeah17:38
@HeikoSits easy if there are unit tests that ensure that things compute exact products17:38
@sonney2kHeikoS, heh lambday already had that shogun's output is compared to eigen3's17:39
lambdaysonney2k: 0.006664 (single thread)17:39
lambdaysonney2k: 0.004662 (multiple)17:40
lambday(just for one multiplication)17:40
@sonney2kwith 2 threads. and if you use say 4 threads?17:40
lambdaysonney2k: that was 4 threads17:40
@sonney2kit is really fast anyways17:40
lambdayearlier I was using 217:40
@sonney2kHeikoS, will that operation be the bottleneck?17:40
lambdayHeikoS: I should add that unit-test anyway17:41
@HeikoSsonney2k: it is important, but when this stuff is parallelized on a larger scale anyway, we will usually only have 1cpu per job usually17:42
@HeikoSso its not very important to gain 20%17:42
@sonney2k30% on lambday's i717:43
@HeikoSsonne|work: we can speed things up in more clever ways (preconditioning etc)17:43
@HeikoSsonney2k: its easy to add later on right?17:43
@HeikoSand it would be good to have, but not crucial for now I would say17:43
@sonney2kHeikoS, well it is already there just use my code17:43
@sonney2kI don't care though17:44
@sonney2kI was just interested in the benchmark17:44
@sonney2klambday, waht was shogun single threaded?17:46
lambdaysonney2k: eigen3 was slightly faster17:46
@sonney2klambday, I meant the time17:46
lambdaysonney2k: wait17:46
lambdaysonney2k: [shogun] 0.006630, [eigen3] 0.00518017:47
lambday(just for one multiplication)17:47
@sonney2kalright gtg17:50
lambdaysonney2k: see you17:50
-!- van51 [] has joined #shogun17:51
lisitsynfoulwall: pong17:52
-!- iglesiasg [] has quit [Quit: Leaving]18:08
-!- sanyam [uid10602@gateway/web/] has quit [Ping timeout: 264 seconds]18:09
lambdayHeikoS: I'll be back later18:12
-!- lambday [67157d37@gateway/web/freenode/ip.] has quit []18:13
-!- foulwall [~user@2001:da8:215:c252:c82:7dd9:e65a:75e6] has quit [Remote host closed the connection]18:32
-!- van51 [] has quit [Quit: Leaving.]18:34
-!- van51 [] has joined #shogun18:41
-!- nube [~rho@] has quit [Quit: Leaving.]18:48
-!- nube [~rho@] has joined #shogun18:50
-!- zxtx [] has joined #shogun18:58
hushelllisitsyn: report coming soon :)19:18
pickle27lisitsyn: just sent my report19:24
pickle27lisitsyn: I'll be around for a while if you / anyone wants to discuss final examples19:24
-!- marcus_zoq [] has joined #shogun19:27
hushellHeikoS: Could you check for me this short code: ?19:27
hushellHeikoS: I am using DynamicObjectArray but get mem leak :(19:28
@HeikoShushell: what should I check it for?19:28
@HeikoSah I see19:28
marcus_zoqHello, I'm trying to run K-Means clustering with given starting points. But I get the following error: [ERROR] assertion idx_b<rhs->get_num_vectors() failed in file ../shogun/distance/Distance.h line 121. Perhaps anybody can help me?19:28
hushellHeikoS: mem leak detected in line 1919:28
@HeikoSmarcus_zoq: could you paste some code on pastebin or gist?19:29
@HeikoShushell: get_element increases REF19:30
@HeikoShushell: removing the print lines removes the leak19:30
hushellHeikoS: so this means I have to keep the pointer returned by get_element() ...19:32
@HeikoShushell: yes if you get_element you have to SG_UNREF afterwards19:33
@HeikoSmarcus_zoq: why this derives class?19:33
hushellHeikoS: Thanks very much! I know next time similar problem, I go to check SG_REF somewhere19:33
@HeikoShushell: no worries, these things are hard to catch if you dont know them, keep in mind that any method that returns a CSGObject should increase the ref-count, we need that for swig19:34
marcus_zoqHeikoS: I think shogun has no direct way to set initial centroids?19:35
@HeikoSmarcus_zoq: ah that might be ... maybe write to the mailing list on this one then, I have never used nor written the KMeans code19:37
marcus_zoqok, thanks19:38
@HeikoSmarcus_zoq: sonney2k might know, or iglesias (not here currently)19:38
-!- marcus_zoq [] has left #shogun []19:42
-!- votjakovr [] has joined #shogun19:43
lisitsynpickle27: hey19:44
lisitsynI am back19:44
@sonney2kHeikoS, hushell well any method that returns a *new* CSGObject does not need a SG_REF, only when you return an object that you use internally too19:45
@HeikoSsonney2k: yes sure, I forgot to say19:45
@HeikoSsonney2k: our kmeans seems a bit deprecated , s.o.19:46
@sonney2kHeikoS, only because we have no function to set the cluster centers?19:46
@HeikoSsonney2k: no I mean also the code with get/free feature vector19:47
@HeikoSsonney2k: is it as easy as add a method?19:47
pickle27lisitsyn: cool!19:47
pickle27lisitsyn: I'm working away on adding ICA right19:47
pickle27lisitsyn: right now I have it as a public CEmbeddingConverter, not sure if this is the right fit yet19:47
pickle27lisitsyn: I am just going to continue going forward and if I need to change the base class in the future I'll do so19:48
@sonney2kHeikoS, the code is pretty messy but it looks like the function clustknb can start with some predefined mean's19:51
@HeikoSsonney2k: yeah the things are also sometimes as rhs of the distance19:51
@HeikoSsonney2k: if you look at the code marcus_zoq:
@HeikoSthis is what he tried19:52
@sonney2klooks reasonable19:52
@HeikoSbut would be better if we offered that directly I would say19:52
@sonney2kHeikoS, file a bug report. I think the code should be heavily refactored too btw19:55
@sonney2kit is way to hard to read19:55
@HeikoSsonney2k: indeed ;)19:56
naywhayareboth marcus_zoq and I are looking into this, by the way (he is off getting dinner, so he says)19:57
naywhayare"way too hard to read" +100019:57
naywhayareI'll also add that the code did work with an older version of shogun (1.0.0 I think) but I haven't diffed them to see what changed19:58
votjakovrHeikoS: i've just sent a PR, please have a look at it19:58
pickle27lisitsyn: nvm the ICA methods are just vanilla converters now20:02
-!- shogun-notifier- [] has joined #shogun20:03
shogun-notifier-shogun: Roman Votyakov :develop * a70e031 / src/shogun/ (6 files):
shogun-notifier-shogun: add evaluate_log_probabilities() method for CLikelihoodModel class20:03
shogun-notifier-shogun: Heiko Strathmann :develop * d602f99 / src/shogun/ (6 files):
shogun-notifier-shogun: Merge pull request #1218 from votjakovr/feature/gp_refactoring20:03
shogun-notifier-shogun: add evaluate_log_probabilities() method for CLikelihoodModel class20:03
@HeikoSvotjakovr: cool!20:03
@HeikoSvotjakovr: how is the integral?20:03
@sonney2kHeikoS, no need for a smiley here. We should add a unit test and then sb. should rewrite this thing.20:04
-!- sanyam [uid10602@gateway/web/] has joined #shogun20:05
@HeikoSsonney2k: yeah rewrite should be best, unit test might be hard to do since implementations might differ, I would rather do this after the rewrite20:05
votjakovrHeikoS: not ready yet, but i'd like to send another PR with probit20:05
@HeikoSvotjakovr: go ahead, I will leave soon though20:05
lisitsynpickle27: sorry got distracted - preparing for my flight tomorrow20:06
pickle27lisitsyn: We can discuss more later, also I should have the first PR for the ICA algs soon ish (maybe not until tomorrow or so)20:06
@HeikoSsonney2k: thats a good task for someone who is interested in hacking shogun20:06
@HeikoSmaybe we can recruit someebody at the workshop20:07
votjakovrHeikoS: may i add also logit with 3 unimplemented methods?20:07
@HeikoSvotjakovr: yep thats fine, as long as you implement and test them in the near future20:07
votjakovrHeikoS: ok20:08
@sonney2kHeikoS, I would not change the implementation though. This is like 5 functions in one so a cleanup would help20:09
naywhayaresonney2k: HeikoS: are you interested in a patch when we dig to the bottom of the issue, or are you planning on a complete rewrite (so a patch is not relevant)?20:09
@HeikoSsonney2k: yeah maybe20:10
@sonney2knaywhayare, we are interested in a patch20:10
-!- zxtx [] has quit [Ping timeout: 264 seconds]20:10
@sonney2kI am not a big fan of rewrites (that will never happen)20:10
@HeikoSyes like cleanup and expose cluster centres20:10
naywhayaresonney2k: okay.  I don't know when either marcus or I will figure it out; this is difficult code to read and we're not familiar with the shogun design ideas :)20:11
naywhayaresonney2k: but I am beginning to understand what's going on so sooner or later I will get somewhere... :)20:11
@sonney2knaywhayare, this is pre-shogun code20:11
@sonney2ksome research code from the last century20:11
naywhayaresonney2k: haha, really from the 90s?20:11
naywhayareit has lasted quite a while :)20:12
@sonney2kahh what am I saying millenioum20:12
naywhayareboth are true :)20:12
@sonney2kyeah I 'rescued' it into shogun. at some point I understood it all but now I do regret that I didn't polish it when I did20:13
naywhayarewhen I took over mlpack development some years ago it had previously been done by a bunch of people who were interested in publishing papers and not code, so absolutely nothing was polished20:14
@sonney2knaywhayare, I think there are like 3-5 massacre classes in shogun. one is CHMM, then the legacy structured output code, kmeans and yay our string kernels :)20:14
@HeikoSsonney2k: there is also this EM thing somewhere ;)20:14
@sonney2knaywhayare, so then you know how it is20:14
naywhayareheh... I know how reading those goes...20:14
@sonney2kHeikoS, but that is 'new' GSoC code20:14
naywhayareyeah; when I discovered the executable that ran nearest neighbors, I noticed that the genius who wrote it did not give an option for the results to be saved20:15
naywhayareso the method would run on the dataset of your choice, complete the calculation, then exit, trashing the results20:15
naywhayarewho needs results anyway? </sarcasm>20:15
@sonney2knaywhayare, well actually shogun started out as my student research project20:15
@sonney2kIIRC 1999 with an HMM20:15
@sonney2kI wrote20:15
naywhayareit must be neat having seen it come such a long way20:15
@sonney2kback then it was very readable code20:15
@sonney2kuntil it got some treats (parallelized and future massacred)20:16
naywhayareI've avoided parallelization to some extent because of the nightmare it makes code into20:16
@sonney2kthe svm framework & kernels were done in 200020:16
@sonney2kand I basicaly wrote all code in trains20:16
@sonney2k(I had 7hr train rides back then)20:17
@sonney2ktwice a week20:17
naywhayarewow, that is a serious commute20:17
@sonney2knaywhayare, well we already ran it on some alpha cluster in 200020:18
@sonney2kdistributed on > 100 CPUs20:18
naywhayarewow, alpha, I don't think you can even find those anymore20:18
@sonney2kno but they were fast20:18
shogun-buildbotbuild #1213 of bsd1 - libshogun is complete: Failure [failed test]  Build details are at  blamelist: Heiko Strathmann <>20:18
@sonney2kand the cluster was fresh and empty and to our disposal20:18
naywhayareconvenient :)20:19
@sonney2kso I trained thousands of HMMs20:19
@sonney2kand later SVMs utilizing a kernel that was computed on HMMs on-the-fly20:19
@sonney2krequiring lots of memory20:19
@sonney2kI am glad to say that we have come a long way and I am not sure this code is even useful nowadays20:20
naywhayarenow I know a lot about the history of shogun. :)  also, I would not be surprised if someone out there is using your HMM code20:23
naywhayarethere is kind of a lack of HMM C++ code out there20:24
votjakovrHeikoS: i've sent20:24
naywhayarethere's HTK but last I checked it won't even compile on amd6420:24
@HeikoSvotjakovr: reading :)20:24
shogun-buildbotbuild #1009 of cyg1 - libshogun is complete: Success [build successful]  Build details are at
@sonney2knaywhayare, yeah but we lost multithreading during some refactoring (yes it was even more messy before!)20:25
shogun-buildbotbuild #1010 of cyg1 - libshogun is complete: Failure [failed configure]  Build details are at  blamelist: Roman Votyakov <>20:25
@sonney2knaywhayare, ohh and the first name of shogun was 'gf' for genefinder20:26
pickle27naywhayare: totally agree, there is lacking HMM c++ code and examples20:26
@sonney2kthis thing became pretty big before gee eff got a real name20:26
-!- zxtx [] has joined #shogun20:26
@sonney2kand we started with the cmdline itnerface then matlab20:27
@sonney2kcmdline for running things on the cluster and matlab to try things out20:27
@sonney2kbut then I just tried out swig for fun and I got addicted to python20:27
@sonney2kso that's it20:28
shogun-notifier-shogun: Roman Votyakov :develop * 3641eb7 / / (10 files):
shogun-notifier-shogun: add probit and logit likelihood models20:28
shogun-notifier-shogun: Heiko Strathmann :develop * 330626c / / (10 files):
shogun-notifier-shogun: Merge pull request #1219 from votjakovr/feature/gp_binary_classification20:28
shogun-notifier-shogun: add probit and logit likelihood models20:28
@HeikoSladies and gentlemen, we have the first binary GP classifier :)20:28
votjakovrYeah :)20:28
@sonney2knaywhayare, hardcore hackers like lisitsyn, HeikoS and all our gsoc'ers and bug reporters did the rest20:28
@HeikoSyeah :)20:28
shogun-buildbotbuild #1214 of bsd1 - libshogun is complete: Failure [failed test]  Build details are at  blamelist: Roman Votyakov <>20:28
@sonney2kvotjakovr, HeikoS, lisitsyn vodka for everyone!20:28
gsomix_good evening20:29
@sonney2kgsomix_, still in train?20:29
-!- gsomix_ is now known as gsomix20:29
@sonney2kgsomix, your keyword ;)20:29
gsomixsonney2k, nope. I will be in the train tomorrow.20:29
lisitsyngsomix: are you in train already?20:29
@HeikoSvotjakovr: nice, I will use this for the workshop if I can get a graphical example running20:29
@HeikoSgotta go, bye all!20:29
@sonney2kgsomix, could you please not use fgets instead of getline to fix the bsd1 buildbot failure20:29
pickle27also unfortunately I won't be joining for the workshop20:30
gsomixsonney2k, of course. preparing PR.20:30
pickle27too expensive, maybe next year! (hoping I can get my future employer to pick up the tab!)20:30
@sonney2knaywhayare, ohh btw I tried hard to merge collaborate with others but shogun was already too big (and the other projects too) so I think the destiny will be 1000 ml toolboxes or whatever20:30
@sonney2kpickle27, wiking has set up some live stream20:31
@sonney2kpickle27, I mean will be setting up20:31
@sonney2kand we will have recordings of the talks20:31
naywhayaresonney2k: I had the same thoughts early on -- why not merge with something else? -- but then I thought that everyone taking different approaches is not a bad thing20:31
@sonney2kto be put on youtube20:31
@sonney2kseems like 1080p even20:31
naywhayarethe general community that you have set up through MLOSS is a great thing though20:31
pickle27awesome I will deffs watch!20:31
naywhayaresome amount of collaboration between libraries is definitely useful, especially in interchangeable formats, etc.20:31
@sonney2knaywhayare, well I wish it would be building blocks20:32
@sonney2kI mean some kind of general framework everyone could hook to20:32
votjakovrHeikoS: good:) So next step: finish numerical evaluation of integrals and implement missing logit and student's methods and then EP i think20:32
pickle27yeah actually I had a random question about the ml community did anyone see that lib that Iglesiasg posted about20:32
naywhayarepickle27: I heard about it once20:32
@sonney2knaywhayare, yeah data formats... mldata was my attempt on that but I see some patterns evolving but much much slower than I expected20:32
pickle27seems like a waste of duplicated effort is all20:33
naywhayarepickle27: thanks ;)20:33
@HeikoSvotjakovr: go for it, this is really good stuff, looking forward to the other things :)20:33
@sonney2kpickle27, naywhayare is the author of mlpack (or one of the...)20:33
pickle27haha oops!20:33
@sonney2kpickle27, naywhayare but I think so too20:33
@HeikoSvotjakovr: ok running now, bye!20:33
naywhayarepickle27: it's okay :).  mlpack focuses on different things than shogun does20:34
pickle27well I was basically wondering if we talk to them at all so there is my answer20:34
@sonney2kI would join some other projects and not start a new one nowadays20:34
votjakovrHeikoS: see you20:34
pickle27yeah I hadn't looked into it20:34
naywhayarethe focus is tree-based algorithms20:34
@sonney2knaywhayare, well I hope so :)20:34
pickle27its good that we talk!20:34
@sonney2kyeah we have 2 trees in here20:34
-!- HeikoS [] has quit [Quit: Leaving.]20:34
@sonney2kone for string kernels20:34
@sonney2kand covertree20:35
naywhayarethe cover tree code is just jl's horrendous ugly code wrapped though20:35
naywhayarewhat a nightmare that code is20:35
@sonney2kexactly like k-means :)20:35
@sonney2kwritten by geniouses though20:35
naywhayaredefinitely :)20:35
naywhayareI met jl at ICML last week.  interesting guy, maybe a bit eccentric20:35
@sonney2knaywhayare, yeah I know him well.20:36
@sonney2khe was mentoring for shogun even20:36
naywhayareI didn't know that he was involved with shogun20:36
@sonney2knaywhayare, we tried to interface somehow20:36
@sonney2kbasically the streaming features in shogun are the result of this20:37
@sonney2kunfortunately we didn't have unit tests back then20:37
@sonney2kand our integration tests were b0rken / disabled20:37
naywhayareoh, so this was a collaboration between shogun and VW? or was this before then20:37
@sonney2kso some refactoring killed halve the code20:37
@sonney2knowadays we would do it differently20:38
@sonney2krather use VW as a library20:38
@sonney2kI mean have a student design soem clean interfaces to VW20:38
@sonney2kand then use it as lib20:38
@sonney2kwhich we didn't back then20:38
@sonney2kand the student disappeared an no one picked up20:39
@sonney2kproblem with online learnign is that you need HUGE data sets20:39
naywhayarehuge as in GB or as in TB?20:39
@sonney2kas in TB20:39
naywhayareoye, that gets to be a bit frustrating to deal with20:40
@sonney2kno I mean it will work with GB's too20:40
@sonney2kbut with GB's you can still use batch and get more reliable results20:40
naywhayareah, okay; even so, once you exceed the limit of the RAM things you have to rewrite to account for that20:40
naywhayarealthough I did have an idea recently; I wonder if it's in use anywhere: use mmap() to get a double* to your raw packed data matrix on disk, then wrap a matrix around that memory pointer, and let the OS handle smartly accessing it20:41
@sonney2knaywhayare, yeah but with shogun we tried to stretch that limit as far as possible by having all the native data representations20:41
naywhayareit wouldn't be optimal but it sure would be easy20:41
@sonney2knaywhayare, yeah I mmap'd in shogun too (and IIRC python numpy has that too)20:41
@sonney2kbut it is rather not doing the trick20:42
@sonney2kit is easy access though20:42
@sonney2kyou need some clever memory access pattern20:42
@sonney2ksome kind of mini batch20:42
naywhayareyeah, it's definitely not the smartest way to do that.  but you could do it and pester the OS guys to make it faster :)20:43
lisitsynsonney2k: have you prepared your talk?20:43
@sonney2kso load batches of data that fit in memory do some heavy stuff etc20:43
@sonney2klisitsyn, you are right20:43
@sonney2knot at all :/20:43
lisitsynsonney2k: the same here *crying*20:43
@sonney2knaywhayare, no it is not possible20:43
lisitsynI depart tomorrow AAAAA20:43
shogun-buildbotbuild #1335 of deb3 - modular_interfaces is complete: Failure [failed compile csharp_modular]  Build details are at  blamelist: Heiko Strathmann <>20:43
@sonney2klisitsyn, ohh man then hurry up! or do it in the plane/airport...20:44
lisitsynsonney2k: it is mostly on paper already20:44
lisitsyntime to convert it to impress.js20:44
@sonney2knaywhayare, mmap is fast but you need a lot of virtual memory (swap!)20:44
@sonney2knaywhayare, back in 2000 I had some RealFileFeatures (as they were called in shogun) which did some caching / reading from disk of features20:45
@sonney2kdamn they even exist today20:45
@sonney2ksomething to be removed :)20:46
@sonney2kso we did the caching of vectors manually20:46
naywhayarethere was someone in our lab who had refactored mlpack to read from databases; but he never once talked to me about it and graduated and I never heard about it again20:46
@sonney2kI guess mmap does it much better for random access patterns (though needing swap)20:46
naywhayareso I'm not actually sure if he ever did it or if his committee was asleep20:47
@sonney2kyeah I've seen papers about people doing SVMs directly from databases20:47
@sonney2knaywhayare, the biggest problem with any mloss project is to keep it alive and to get contributors20:51
pickle27how would one run a single unit test?20:51
@sonney2knaywhayare, that is if you are not a professor and have a team of students working on this (e.g. Weka, libsvm, liblinear, ...)20:51
@sonney2kpickle27, you need to set some environment varialbe IIRC GTEST_FILTER20:52
@sonney2kpickle27, google for it20:52
@sonney2kvotjakovr, do you already have an example for using GPC?20:54
shogun-buildbotbuild #1215 of bsd1 - libshogun is complete: Failure [failed test]  Build details are at  blamelist: Heiko Strathmann <>, Roman Votyakov <>20:54
naywhayaresonney2k: this is true.  I have a lot of trouble finding the time to balance both my research and mlpack20:54
naywhayarebut, the library is starting to get some momentum, and with GSoC it may find its feet20:55
votjakovrsonney2k: yep, but i need to polish it20:55
@sonney2knaywhayare, I know exactly what you are talking about...20:56
pickle27I have a lot of respect for researchers who also maintain open libs of their code20:57
-!- travis-ci [] has joined #shogun20:58
travis-ci[travis-ci] it's Heiko Strathmann's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun:
-!- travis-ci [] has left #shogun []20:58
lisitsynpickle27: run it with20:59
pickle27run what exactly? i tried make unit-tests --gtest-filter=Jade and it said unrecognized option21:00
votjakovrsonney2k: hmm, c# modular is failed again...21:00
lisitsynpickle27: not make21:00
lisitsyn./shogun-unit-test --gtest-filter=Jade21:00
votjakovrsonney2k: i'm not sure, but i think, because there are 2 SGVectors in parameters of a function21:02
lisitsynvotjakovr: yes your guess is right21:02
shogun-buildbotbuild #1011 of cyg1 - libshogun is complete: Success [build successful]  Build details are at
@sonney2kvotjakovr, then please add them to modshogun_ignores.i21:05
@sonney2kvotjakovr, in the very same way I did it last time - you will find your mean_* in there21:05
@sonney2kand send a PR21:05
shogun-buildbotbuild #1336 of deb3 - modular_interfaces is complete: Failure [failed compile csharp_modular]  Build details are at  blamelist: Roman Votyakov <>21:06
-!- travis-ci [] has joined #shogun21:08
travis-ci[travis-ci] it's Heiko Strathmann's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun:
-!- travis-ci [] has left #shogun []21:08
votjakovrsonney2k: ok, i'll do it21:10
shogun-buildbotbuild #1337 of deb3 - modular_interfaces is complete: Failure [failed compile csharp_modular]  Build details are at  blamelist: Heiko Strathmann <>, Roman Votyakov <>21:12
van51sonney2k: ping21:23
van51sonney2k: I had to make some changes to StreamingFileFromStringFeatures to get it working21:24
van51sonney2k: should I make a different PR for those?21:24
@sonney2kvan51, yes please21:26
pickle27lisitsyn: do I need to do anything for my code to become part of the wrappers?21:27
lisitsynpickle27: yes21:27
pickle27what do I need to do?21:27
lisitsynpickle27: I guess you want to make your class visible in swig interfaces, right?21:28
lisitsynpickle27: check interfaces/Converter.i21:28
lisitsynand interfaces/Converter_includes.i21:28
lisitsynpickle27: just add similar includes21:28
lisitsynone there and one here21:28
@sonney2kvan51, btw we should expose your new classes to python/csharp etc too21:31
@sonney2kvery easy though21:31
van51sonney2k: I made the PR21:32
van51sonney2k: sure.. is there something I can look at to get an idea of how to do it?21:32
shogun-notifier-shogun: van51 :develop * 2279539 / src/shogun/io/streaming/StreamingFileFromStringFeatures.h:
shogun-notifier-shogun: Changes in StreamingFileFromStringFeatures to make it work21:33
shogun-notifier-shogun: Soeren Sonnenburg :develop * 7862aad / src/shogun/io/streaming/StreamingFileFromStringFeatures.h:
shogun-notifier-shogun: Merge pull request #1220 from van51/develop21:33
shogun-notifier-shogun: Changes in StreamingFileFromStringFeatures to make it work21:33
van51sonney2k: good.. took me some time to find what was wrong there :/21:34
@sonney2kvan51, per class taht you want to expose it is adding 3 lines to .i files in shogun/src/interfaces/modular/21:34
@sonney2kvan51, these features need to use SGVector21:34
@sonney2kvan51, they are from the dawn of ages21:34
van51sonney2k: the streaming features?21:34
@sonney2kI mean before we had refcounted sgvector21:34
pickle27lisitsyn: I just made a PR for ICA with Jade21:35
@sonney2kso the problem now is that they don't de-alloc memory or do twice bah!21:35
van51sonney2k: ah ok, I thought you were saying I should switch to returning a SGVector instead of a SGSparseVector21:35
@sonney2kvan51, no21:35
van51sonney2k: yea it was really frustrating21:35
@sonney2kvan51, look at e.g. Clustering.i21:36
lisitsynpickle27: nice I'll check21:36
@sonney2kand then KMeans21:36
@sonney2kpickle27, that is also answering your question :)21:36
@sonney2kso CKMeans gets renamed to KMeans in e.g. python21:37
@sonney2kand you need the %include to get the class wrapped21:37
@sonney2kthen there is a Clustering_includes.i file21:38
@sonney2kwhere Kmeans is regularly #included21:38
@sonney2kthat's all21:38
shogun-buildbotbuild #1012 of cyg1 - libshogun is complete: Failure [failed configure]  Build details are at  blamelist: van51 <>21:38
van51sonney2k: ok. I'll add it to my todo list :)21:39
van51sonney2k: I want to get done with the streaming features today21:39
@sonney2kvan51, pickle27 now any type that returns an SGVector or other SG* datatype or gets it as argument is wrapped to the native' lang's representation21:39
@sonney2kfor example in octave or python a SGMatrix would be mapped to a matrix or numpy array21:39
@sonney2kvan51, yeah it is trivial anyways21:39
shogun-buildbotbuild #1216 of bsd1 - libshogun is complete: Failure [failed test]  Build details are at  blamelist: van51 <>21:46
pickle27Im trying a build of the python modular now lets see how it works!21:49
votjakovrsonney2k: i've just sent a PR21:52
-!- iglesiasg [] has joined #shogun21:53
-!- mode/#shogun [+o iglesiasg] by ChanServ21:53
shogun-notifier-shogun: Roman Votyakov :develop * a8efb26 / src/interfaces/modular/ (3 files):
shogun-notifier-shogun: add swig interfaces for GPC and update ignore list for c# modular21:54
shogun-notifier-shogun: Soeren Sonnenburg :develop * c80366d / src/interfaces/modular/ (3 files):
shogun-notifier-shogun: Merge pull request #1222 from votjakovr/develop21:54
shogun-notifier-shogun: add swig interfaces for GPC and update ignore list for c# modular21:54
@iglesiasghi all21:54
@sonney2kvotjakovr, excellent!21:54
@sonney2kvotjakovr, now if only the build would be back to green with that gp test...21:54
@sonney2kgsomix, and the bsd1 getline fgets...21:54
@sonney2kevening iglesiasg!21:55
votjakovrsonney2k: yep, but i still can't find what is actually wrong with it21:56
@sonney2kvotjakovr, the only way to figure this out is figure out which change caused this21:58
shogun-buildbotbuild #1217 of bsd1 - libshogun is complete: Failure [failed test]  Build details are at  blamelist: Soeren Sonnenburg <>21:58
-!- travis-ci [] has joined #shogun21:59
travis-ci[travis-ci] it's Soeren Sonnenburg's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun:
-!- travis-ci [] has left #shogun []21:59
lisitsyniglesiasg: hey21:59
@iglesiasglisitsyn: how is it going with the slides?22:00
lisitsyniglesiasg: terrible :D22:00
@iglesiasgI don't believe that!22:00
lisitsyniglesiasg: I am htmling them now22:01
@iglesiasglisitsyn: I bet they will look superb, looking forward to seeing them22:02
lisitsyniglesiasg: I have procrastinated too much22:02
@sonney2klisitsyn, can you do mine too :)22:02
pickle27hey guys 2 things22:03
@sonney2kI don't care if it is via procrastination or anything22:03
@iglesiasgsonney2k: you must be a professional already doing slides :)22:03
lisitsynsonney2k: I will not be in time22:03
pickle27first there is a prob in develop with Logit but Im sure you knew that22:03
lisitsynlike before flight :)22:03
@sonney2kiglesiasg, that doesn't make it more interesting22:03
@sonney2klisitsyn, me neither22:03
pickle27second what is the difference between the doc and undoc examples exactly?22:03
shogun-buildbotbuild #1338 of deb3 - modular_interfaces is complete: Failure [failed compile csharp_modular]  Build details are at  blamelist: van51 <>22:03
lisitsynsonney2k: but you won't take a flight ;)22:03
lisitsynpickle27: doc folder is autogenerated22:03
@sonney2kpickle27, documented examples are generated from undocumented ones22:04
@sonney2kand the descriptions22:04
pickle27kk I thought thats what happened22:04
@sonney2kmake examples does that22:04
@sonney2kpickle27, rationale is that you write an example in the 5 languages doing the same thing but document it just once22:04
pickle27logit crashes travis before I can see if my unit test runs on travis :(22:05
pickle27sonney2k: ooooh cool22:05
@sonney2kpickle27, blame votjakovr22:05
pickle27votjakovr: ! lol22:05
* sonney2k round 1 - fight!22:05
lisitsynAAAHH my plane is 13:50 UTC22:06
lisitsynwith crazy 1h transfer in SVO it must be funny hah22:07
pickle27ha regretting that upstream merge now, I could have waited lol22:07
@iglesiasglisitsyn: you are flying already tomorrow, aren't you?22:08
lisitsyniglesiasg: yes22:08
@sonney2kiglesiasg, when are you?22:08
lisitsyniglesiasg: in 17 hrs22:08
shogun-buildbotbuild #1013 of cyg1 - libshogun is complete: Success [build successful]  Build details are at
@sonney2kshogun-buildbot, dance22:09
@iglesiasglisitsyn: cool cool, will you be using there your current cell number?22:09
@iglesiasgsonney2k: on Wednesday22:09
@sonney2kI am damn impressed that cyg1 is happy22:09
lisitsyniglesiasg: not sure I will be using it heavily with its price22:09
@iglesiasgI see22:09
lisitsyniglesiasg: wifi for the win (I hope)22:09
@iglesiasgthen I will contact you through gtalk so we can meet on Thursday :)22:10
@iglesiasglet's hope I share room with nice people :P22:10
lisitsyniglesiasg: if wifi in the hotel is ok I will definitely be online some time morning and evening22:10
lisitsyniglesiasg: hah I hope so too22:11
lisitsyniglesiasg: when you're back to sweden?22:11
@iglesiasglisitsyn: on Sunday22:11
lisitsynI see22:11
@sonney2klisitsyn, you too right?22:12
lisitsynsonney2k: no I am departing monday morning22:13
lisitsynlike 8-50 or so22:13
@sonney2kiglesiasg, will you be at the hands on workshop on saturday/sunday?22:13
lisitsynI guess I'd have to get up eeearrly22:13
@iglesiasgsonney2k: sure!22:13
@sonney2klisitsyn, sxf or txl?22:13
lisitsynsonney2k: sxf22:13
@sonney2kohh man22:14
@sonney2kpoor guy22:14
@iglesiasgsonney2k: will you be able to be a little bit there too any of those days?22:14
@iglesiasgIIRC last time I heard of you mentioned you wouldn't22:14
lisitsynsonney2k: well I'll take taxi but I guess it is a hour still, right?22:14
@iglesiasgI heard of it*22:14
@sonney2kwell I am away wednesday,thursday,friday for the workshop22:14
@sonney2kyeah on friday22:14
@sonney2klisitsyn, $$$22:14
@iglesiasgsonney2k: but no Saturday nor Sunday?22:15
lisitsynsonney2k: 38 euro according to the website?22:15
@sonney2kmaybe a little bit of saturday22:15
@sonney2kbut I will be dead22:15
@iglesiasgsonney2k: is it SXF so bad? I read on the website it didn't take that long to the city centre22:15
@sonney2ka bit too much22:15
@iglesiasgnot that long ~ less than one hour22:15
@sonney2kno it is ok22:15
@iglesiasgin public transportation I am talking about22:16
@sonney2kbut you have to walk 10 mins to the s-bahn station22:16
lisitsynit takes a few hops to my hotel from SXF22:16
@sonney2kone cannot miss it though there is just one way to walk to22:16
@iglesiasgI really have to check how to download the map of the city in my phone before going!22:16
@sonney2kiglesiasg, android?22:16
@sonney2klong press22:16
@sonney2kthen download22:16
@iglesiasgsonney2k: long press in google maps pointing at Berlin?22:17
@iglesiasgyes, android22:17
@iglesiasgoh cool, it sounds easy enough22:18
@sonney2kiglesiasg, actually no22:18
@sonney2kiglesiasg, just menu22:18
@sonney2kthen download for offline use22:18
@iglesiasgI am reading
@sonney2kiglesiasg, lisitsyn weather is supposed to be very nice22:20
lisitsynthat's indeed cool22:20
@sonney2klike 25 C or so22:20
@sonney2knot too hot22:21
@iglesiasgyes! That sounds perfect22:21
shogun-buildbotbuild #1339 of deb3 - modular_interfaces is complete: Failure [failed compile csharp_modular]  Build details are at  blamelist: Soeren Sonnenburg <>22:21
@iglesiasgmore than 25 gets close to 30 and #@%& hot22:21
@iglesiasgI wonder how I survived that many years in Spain :-O22:21
@iglesiasgnice that Germany is in the offline list22:22
@sonney2kiglesiasg, at least you have sunlight in winter!22:22
@iglesiasgthis year was pretty good in that matter here in Stockholm actually22:23
@iglesiasglisitsyn: if it were in Russia or Spain we would be screwed -- no offline maps mode :S22:23
@sonney2kberlin had no sun the whole winter22:23
-!- travis-ci [] has joined #shogun22:23
travis-ci[travis-ci] it's Soeren Sonnenburg's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun:
-!- travis-ci [] has left #shogun []22:23
lisitsyniglesiasg: if it were in russia we all would be screwed22:23
@iglesiasglisitsyn: more reasons other than the offline maps?22:24
lisitsyniglesiasg: yes dangerous!22:25
@sonney2khaving a phone or what?22:25
@sonney2kto breathe?22:25
lisitsynI bet an foreigner in real russia (not like the center of moscow)22:26
lisitsynwill be despoiled very fast22:26
lisitsynlike once they realize you have no idea where to go22:27
shogun-notifier-shogun: Roman Votyakov :develop * f8704e4 / src/shogun/machine/gp/LogitLikelihood.cpp:
shogun-notifier-shogun: fix bug in CLogitLikelihood class22:27
shogun-notifier-shogun: Soeren Sonnenburg :develop * 4e6e445 / src/shogun/machine/gp/LogitLikelihood.cpp:
shogun-notifier-shogun: Merge pull request #1223 from votjakovr/develop22:27
shogun-notifier-shogun: fix bug in CLogitLikelihood class22:27
* sonney2k ZZzz22:28
@iglesiasgok I am saving 82MB of Berlin, that should be enough!22:28
lisitsyniglesiasg: I saved a bit too!22:28
* iglesiasg is surprised all the stuff you get to do while compiling after rebase!22:29
shogun-buildbotbuild #1218 of bsd1 - libshogun is complete: Failure [failed test]  Build details are at  blamelist: Soeren Sonnenburg <>, Roman Votyakov <>22:29
pickle27ahhhhhh I have merge files in my git commit now22:50
pickle27ah there we are22:51
pickle27lisitsyn: pending travis success I think I might be good to merge the first ICA technique22:52
pickle27lisitsyn: I am pretty happy with the current basic examples too!22:52
lisitsynpickle27: I'll check it in a bit22:53
pickle27yeah let Travis do his thing22:53
shogun-buildbotbuild #1219 of bsd1 - libshogun is complete: Failure [failed test]  Build details are at  blamelist: Soeren Sonnenburg <>, Roman Votyakov <>23:00
shogun-buildbotbuild #1340 of deb3 - modular_interfaces is complete: Failure [failed test python_modular]  Build details are at  blamelist: Soeren Sonnenburg <>, Roman Votyakov <>23:01
lisitsynpickle27: alright looks nice!23:03
lisitsynpickle27: thanks!23:03
shogun-notifier-shogun: Kevin :develop * e7f36b5 / / (7 files):
shogun-notifier-shogun: added Jade algorithm for ICA and BSS along with a unit test and example23:03
shogun-notifier-shogun: Sergey Lisitsyn :develop * 20f94d3 / / (7 files):
shogun-notifier-shogun: Merge pull request #1221 from pickle27/develop23:03
shogun-notifier-shogun: added Jade algorithm for ICA and BSS along with a unit test and example23:03
pickle27lisitsyn: thanks!23:06
van51question, suppose you have a PR standing by and things get merged in the meantime.. do that PR need to be rebased? :)23:10
@iglesiasghey hushell! I will have a look to the PR ASAP23:11
@iglesiasgsome time tomorrow afternoon probably23:11
@iglesiasghushell: you can continue working before we revise, right?23:11
-!- travis-ci [] has joined #shogun23:12
travis-ci[travis-ci] it's Soeren Sonnenburg's turn to pay the next round of drinks for the massacre he caused in shogun-toolbox/shogun:
-!- travis-ci [] has left #shogun []23:12
pickle27van51: if you don't merge it into your own branch then you can just make a PR23:16
pickle27but if you merge it in yourself then yes rebase23:17
pickle27thats what I just had to do because I wanted to merge votjakovr latest PR to see if Travis would run23:17
hushelliglesiasg: thanks! yes, I can keep working on unit tests23:20
hushelliglesiasg: But I may not be able to finish all of them today, cause I have to prepare a presentation for tomorrow23:21
@iglesiasghushell: sure, that's fine23:22
van51pickle27: ok thanks :)23:22
pickle27lisitsyn: woooo a clean build from my machine to the build bot!23:30
shogun-buildbotbuild #1341 of deb3 - modular_interfaces is complete: Failure [failed test python_modular]  Build details are at  blamelist: Soeren Sonnenburg <>, Roman Votyakov <>23:30
lisitsynpickle27: good!23:30
pickle27lisitsyn: I trust you saw my mail today about the final example?23:31
pickle27sonney2k seemed to think my modified idea would work23:31
lisitsynpickle27: sorry I am a bit hectic last days23:32
pickle27no worries!23:32
shogun-buildbotbuild #1220 of bsd1 - libshogun is complete: Failure [failed test]  Build details are at  blamelist: Sergey Lisitsyn <>, Kevin <>23:32
pickle27hmm wut23:32
pickle27I think this error is not from our merhe23:33
pickle27lisitsyn: I'll probably spend the rest of this week porting the rest of the algorithms and then start on the example after everyone is done with the workshop23:34
lisitsynpickle27: I am ok with the new approach yes23:35
lisitsynjust select a few tracks, mix then demix23:35
pickle27I'll try and pick some interesting tracks23:36
lisitsynpickle27: something from venetian snares23:36
lisitsynthere is a nice album of venetian snares23:36
pickle27haha maybe23:36
lisitsynwinnipeg is a frozen shitty hole or so23:36
pickle27I was thinking more along the lines of mashing together movie quotes or something but music too23:37
pickle27haha yeah winnipeg sucks!23:37
lisitsynpickle27: is it really that bad?23:37
pickle27I've actually never been there lol23:37
pickle27but I have been to Saskatchewan and it was fairly boring23:38
naywhayarelisitsyn: have you listened to Rossz Csillag Alatt Sz?letett?  I enjoyed that album a lot (okay, I know this is off-topic...)23:41
lisitsynnaywhayare: yes sure, interesting thing23:41
lisitsynnaywhayare: are you into such IDM music?23:43
lisitsynI am actually not but he got into my sight one day so I listened to it23:44
lisitsynpickle27: so winnipeg should be boring but anything else?23:44
-!- votjakovr [] has left #shogun ["Fallen asleep!"]23:44
lisitsynnaywhayare: szamar madar is kind of classic!23:45
naywhayareI am into IDM (although I think that's a stupid name for the genre)23:47
naywhayareI found the genre via Boards of Canada23:47
pickle27naywhayare: are you Canadian?23:47
naywhayarenope, I'm in Atlanta23:48
naywhayarebut I have been close to Canada before :)23:48
pickle27alright guys Im off to play some Frisbee, see you tomorrow!23:50
-!- pickle27 [] has quit [Quit: Leaving]23:52
--- Log closed Tue Jul 09 00:00:23 2013